| Chapter 10. Assessment | |
1 | Practicality (within the means of financial limitations, time constraints, ease of administration, and scoring and interpretation) - not excessively expensive, stays within appropriate time constraints, relatively easy to administer |
| 2 | Reliability : the degree to which a test gives consistent and dependable results. (1)Inter-rater reliability - Two or more scorers yield consistent score of the same test - Low reliability results from lack of attending to scoring criteria, inexperience, inattention, or even preconceived biases (2)Intra-rater reliability - One rater results in the same score in the case of scoring more than one time. - Low reliability results from unclear scoring criteria, fatigue, biases toward particular students, or simple carelessness. |
| 3 | Validity: the extent to a test actually measures what it is intended to measure (1) Content validity - the adequacy with which the test samples the content of the course. - the extent to which a test adequately measures a particular skill it sets out to measure (2) Face validity - the degree to which a test appears to measure the abilities it claims to measure based on the subjective judgement of an observer (3) Construct validity - the extent to which the items in a test reflect the essential aspects of the theory on which the test is based - the degree to which the test permit inferences about any underlying ability. (4) Criterion-related validity- empirical * Concurrent validity - how identical is the result of the test to the qualified test. *Predictive validity - the degree to which the test predicts the real-ability of students. |
| 4 | Authenticity - the degree of correspondence of the characteristics of a given language test to the features of a target language task. For authenticity in a test, a task is likely to be enacted in the 'real world'. *Conditions for authenticity The language in the test is as natural as possible. Items are contextualized rather isolated. Topics are meaningful, relevant, and interesting for the learner. Some thematic organization to items is provided, such as through a story line or episode. Tasks represent, or closely approximate, real-world tasks. |
| 5 | Wash-back effect: the effects of an assessment on teaching and learning prior to the assessment itself - provides the information that washes back to students in the form of useful diagnoses of strengths and weaknesses *Conditions for positive wash-back effect -The test has high content validity. -It reflects on the result of the assessment on the following lesson. |
| 6 | Formative vs Summative test (1) Formative test - on-going test / for further study / forming students' competence - evaluating students in the process of forming their competencies and skills - internalization of appropriate feedback on performance, with an eye toward the future continuation of learning (2) Summative test - aiming to measure or summarize what a students has grasped - occurring at the end of a course or unit of instruction - final exams in a course and general proficiency exam |
| 7 | Direct vs Indirect test (1) Direct : directly assess what someone intends to measure the effect of wash-back is high (2) Indirect : indirectly assess what they want to assess |
| 8 | Objective vs Subjective (1) objective : multiple choice / true or false / matching / high reliability / low validity (2) subjective : low reliability / high validity |
| 9 | Norm-referenced vs Criterion-referenced (1) Norm-referenced test (between individuals): measuring how the student does on the test compared to how others did on it (2) Criterion-referenced test (within individuals): measuring how well the student has met specific objectives in a certain area |
| 10 | Discrete-point vs intergrative (1) Discrete-point testing (behavioristic/structural approach with contrastive analysis) - knowledge of specific elements in phonology, grammar, and vocabulary in order to determine proficiency in the isolated skill areas (2) Integrative testing (developmentalistic/communicative approach) - examining the students' ability to use many skills simultaneously when accomplishing a task |
| 11 | Classification of assessment based on its purpose (1) Proficiency test: measuring global competence in a language (2) Diagnostic test: checking students' abilities for specific weaknesses and problems they may have encountered (3) Placement test: sorting students into groups according to their language ability at the beginning of a course (4) Achievement test: determining acquisition of course objectives at the end of a period |
| 12 | Integrative test (1) Cloze test - a sentence with a word left out should be filled with a calculated guess, using linguistic expectancies and background knowledge - integrative measure *Fixed-ratio deletion: typically every n'th word is deleted *Rational deletion : choosing deletions according to the specific standard allows the designer to avoid deleting words that would be difficult to predict from the context - Scoring method *Exact word method: get credits for a correct answer if and only if the word they write in any given blank is the exact word that was deleted from the original text in that place. *Acceptable word method : any response that is grammatically correct, and makes good sense in the context is given full credits as an acceptable answer. This method may promote positive wash-back, since it could encourage learners tu use their pragmatic expectancy creatively. (2) C-test - the second half of every other words is deleted, leaving the first and the last sentence of the passage intact. - a clue (half the word) serves as a stimulus for respondents to find the other half. |
| 13 | Alternative test - requires students to perform, create, produce or do something - real-world contexts or simulations - non-intrusive in that they extend the day-to-day classroom activities - allows students to be assessed on what they normally do in class - uses tasks that represent meaningful instructional activities - focuses on processes as well as products - taps into higher-level thinking and problem-solving skills - provides information about the strengths and weaknesses of students - open disclosure of standards and rating criteria |
| 14 | Examples of alternative assessment (1) Portfolio - a collection of student work that tells the viewer about the student - students should be a participant in the selection of his/her work *advantages -encouraged to use their teachers and classmates as resources encourage work in which the motivating factor is to become more precise writers -> encouraged to review and revise their drafts - stimulate reflective thinking - students can set their own criteria - fosters intrinsic motivation, responsibility, and ownership - promote student-teacher interaction with the teacher as facilitator - provides tangible evidence of a students' work - guidelines should be given to students - systematic periodic review and feedback should be present - the combination of self-assessment and teacher assessment probides students with the maximum benefit - students are allowed to participate in setting criteria (2) Journals - a log of one's thoughts, feelings, reactions, assessments, ideas, or progresses toward goals, usually written with little attention to structure, form, or correctness - to carry on a conversation with the teacher - to become better equipped to meet students' individual needs - to practice in the mechanics of writing, using writing as a thinking process - to offer various kinds of feedback - sensitively introduce students to the concept and the objectives of the journal - provide optimal feedback in your responses - provide formative washback comments (3) Self-assessment - students are asked to assess themselves each week according to the most appropriate grades listed on a simple form - show their forms at the end of the week and briefly discuss their results individually with a teacher - allows students to reevaluate the goals they have set to recognize their progress in relation to those goals, and identify new goals revealed as they progress - heightens their awareness of the goals and outcomes of the program and allows them to identify their strengths and need in relation to those outcomes - helps them identify how they learn best - gives students a voice in their education and in shaping the curriculum - guidelines * tell students the purpose of the assessment * define the tasks clearly * encourage impartial evaluation of performance or ability * ensure beneficial wash-back through follow-up tasks. |
| 15 | Scoring method (1) Holistic scoring method - lower reliability than analytic method - all aspects of writing ability is assessed at the same rate - captured in a single score based on a impression on the test - single score may mask an uneven writing profile (2) Analytic scoring method - more scales provide useful diagnostic information for placement - students' performance is assessed in terms of many discrete factors - time-consuming /expensive - structuralist approach - halo effect : results in rating one scale may influence the rating of others (3) Primary trait scoring - focuses on how well students can write within a narrowly defined rage of discourse" - emphasizes the task at hand and assigns a score based on the effectiveness of the text's achieving that one goal - for example, if the purpose or function of an essay is to persuade the reader to do something, the score for the writing would rise or fall on the accomplishment of that function |
| 16 | (1) Item facility - a statistics used to examine the percentage of students who correctly answered a given item - the more the number close to 1.00, the more it means the item was easy - the appropriate range is from 0.3 to 0.7 (2) Distractor analysis - how well individual distractors function as sorting out students who do not know the item (3) Item discrimination - a statistics that indicates the degree to which an item separates students who performed well from ones who did poorly on the test - the efficiency of distractors is the extent to which the distractors lure a sufficient number of test-taker (4) Item response distribution - combines information from both the distractor analysis and the item discrimination analysis |
다음검색