MARKING SCHEMEBy TALK ON ITEM CONSTRUCTION ± SOUTHERN COLLEGE, JOHOR BAHRU, JOHOR 07 JULY 2009 ASSOC. PROF. DR. OTHMAN MD.JOHAN FAKULTI PENDIDIKAN UNIVERSITI TEKNOLOGI MALAYSIA Assoc. Prof. Dr. Othman Md. Johan 1 Holistic Scoring Method : Holistic Method: Read an entire response and assigning it to a category identified by a score or grade. See example attached. Develop a description of the type of response that would illustrate each category before you start. Try out this draft version using several actual papers. Re-examine the papers within a category to see if they are similar enough in quality to receive the same points or grade. Assoc. Prof. Dr. Othman Md. Johan 2 Holistic Scoring Method (Cont.) It may be faster to read essays holistically and provide only an overall score or grade However students do not receive much feedback about their strengths and weaknesses. Thus, brief comments on each paper to point out one or two strengths and/or weaknesses should be so students will have a better idea of why their responses received the scores they did. Assoc. Prof. Dr. Othman Md. Johan 3 Holistic Scoring Category AEXCELLENT (85 - 100) - A Markedly Exceptional Performance SUPERIOR (70 - 84)- Clearly Above Average Performance SATISFACTORY (55 - 69) - A Fully Competent Paper POOR (40 - 54) - A Marginally Acceptable Paper FAILING (0 - 39) - An Unacceptable Performance B- CDF- Assoc. Prof. Dr. Othman Md. Johan 4 Holistic Scoring Rubric : AEXCELLENT (85 - 100) - A Markedly Exceptional Performance a comprehensive grasp of the subject matter is demonstrated, including an in-depth understanding of the relevant concepts, theories, and issues related to the topic addressed an awareness of differing view-points is demonstrated and a rigorous assessment of these undertaken where relevant an ability to think critically is demonstrated in the analysis, synthesis and evaluation of relevant information a thoughtful statement of position is presented and defended through logical arguments and carefully selected supportive detail; the arguments presented build to a consistent conclusion Assoc. Prof. Dr. Othman Md. Johan 5 Holistic Scoring Rubric : AEXCELLENT (85 - 100) - A Markedly Exceptional Performance (Cont.) originality, insight, and creativity are demonstrated; the paper goes beyond repeating what others have said and contributes something new to our understanding of the topic a clear, fluent, and concise style highlights a well-written, tightly argued, and logically structured essay a virtually flawless mastery of all aspects of grammar, structure, and style is demonstrated Assoc. Prof. Dr. Othman Md. Johan 6 B- SUPERIOR (70 - 84)- Clearly Above Average Performance a thorough grasp of the subject matter is demonstrated an awareness of differing view-points is demonstrated and an assessment of these attempted where relevant the paper goes beyond description to interpretation, analysis, synthesis and evaluation a position is adopted and logically argued; appropriate supporting detail is supplied a clear style which communicates well (but may contain occasional or minor flaws in the mechanics of spelling, grammar, sentence structure, etc.) is evident in the logical presentation of a reasonable argument Assoc. Prof. Dr. Othman Md. Johan 7 C- SATISFACTORY (55 - 69) - A Fully Competent Paper a basic grasp of the subject matter is demonstrated accurate information incorporating relevant sources and references is conveyed a position is adopted and logically argued an adequate attempt at analysis, synthesis, interpretation or evaluation is evident an acceptable style demonstrates an awareness of, and attention to, the principles of paragraph development, sentence structure, grammar and spelling, etc. Assoc. Prof. Dr. Othman Md. Johan 8 D - POOR (40 - 54) - A Marginally Acceptable Paper a lack of familiarity with the subject matter is demonstrated through the omission of key material, or through the misinterpretation of important concepts, theories or issues a lack of critical thinking is evident in a paper which is more descriptive than interpretive; or in which the analysis and synthesis are logically flawed; or in which there is a reliance on assertion; or in which the relevance of supporting detail is questionable a position is not taken, is hard to determine, or is inconsistent with arguments or information presented in the paper Assoc. Prof. Dr. Othman Md. Johan 9 D - POOR (40 - 54) - A Marginally Acceptable Paper (Cont.) there is a lack of originality and an over-reliance on material presented in class or in the assigned readings written expression requires improvement in basic communication skills; or written communication is marred by inflated diction, overly complex sentence structures, or an obtuse style. Assoc. Prof. Dr. Othman Md. Johan 10 F - FAILING (0 - 39) - An Unacceptable Performance a basic lack of understanding of the subject matter is demonstrated through gross misinterpretation or omissions there is little attempt to go beyond description; or interpretation and analysis demonstrates gross error in logic or supporting detail little or no factual material is presented; or material presented contains gross factual error; or is completely irrelevant written expression is disorganized, incoherent, poorly expressed, and contains unacceptably frequent or serious errors in grammar, sentence structure, and spelling Assoc. Prof. Dr. Othman Md. Johan 11 F - FAILING (0 - 39) - An Unacceptable Performance OR an attempt is made to use other authors' work without providing proper acknowledgment an attempt is made to hand in a paper from another course an attempt is made to write a paper on a topic other than that approved in writing by the instructor - marking criteria compiled by R. Runté and K. Mazurek Assoc. Prof. Dr. Othman Md. Johan 12 Analytic Scoring Method Develop a list of the major elements you believe students should include in the ideal answer. This should be done before the test is printed so that you can determine the point value of the item and the suggested length, so that adjustments can be made if necessary. Analytic approach is easier to employ and is more suitable when the essay item requires a restricted or short answer. You can decide to give partial credit for each element in your list or simply to give full or no points for each item. Read a sample of the papers to check on your scoring scheme and make necessary adjustments. Assoc. Prof. Dr. Othman Md. Johan 13 Example of Analytic Marking Scheme 1 a) List 4 essential features of a facsimile (fax) in the context of the modern office [4 marks] (b) In a large office, describe two factors which should be considered in making a decision to purchase a facsimile machine [8 marks] (c) A company with a large travelling sales staff is considering the introduction of portable computers with modems for communication. Compile a summary report for your IT Manager on the possible benefits to the company of introducing such products [8 marks] Assoc. Prof. Dr. Othman Md. Johan 14 Marking Scheme: Ques- Part Performance tion Criteria 1 (a) Recall 4 features identified Mark Any 4 relevant points 4x1 marks Total Mark Outline Solution Transmits exact copy Transmits data from hard copy Sends text & graphics Uses phone line Can send & receive data outside office hours Can transmit via computer 4 Assoc. Prof. Dr. Othman Md. Johan 15 Marking Scheme: Ques- Part Performance tion Criteria 1 (b) Understanding of concepts & clarity of expression Mark Any 2 relevant points 2x4 marks Total Mark Outline Solution Convenience/speed 8 Volume usage Cost effectiveness Use outside office hours Different time zone Assoc. Prof. Dr. Othman Md. Johan 16 Marking Scheme: Ques- Part Performance tion Criteria 1 (c) Application Range of appropriate examples Good supporting evidence/ application to concepts Mark Total Mark Outline Solution Speed 4 marks 4 marks 8 Efficiency Immediacy Placing orders Ease of communication Assoc. Prof. Dr. Othman Md. Johan 17 Guidelines to Scoring Exams: Regardless of whether you decide to use an analytic or holistic method of scoring, there are several guidelines to consider with respect to the scoring of essay tests: Develop scoring criteria or model answers. If there are several essay questions, score all of the students responses to one question at a time. This improves consistency and reduces any "carryover" effects. Decide whether you are going to score factors other than content, such as spelling, language usage, and organization. Make sure that your students are aware if you are, and give a separate score for these factors. Assoc. Prof. Dr. Othman Md. Johan 18 Guidelines to Scoring Exams: Score essay responses anonymously to avoid any bias resulting from your familiarity with students and their previous work. One idea is to have students write their names on the backs of the answer sheets or booklets. Periodically check to see whether you have applied the criteria in the same way to later-scored answers as to earlier-scored ones. If you have a large number of papers to score, stop when you get tired so that your frame of mind will not cause your scoring to be inconsistent. When you start again, read over the last few papers you scored to be sure that your scoring was objective. Assoc. Prof. Dr. Othman Md. Johan 19 Assoc. Prof. Dr. Othman Md. Johan 20