Applications of Rule-Space Methodology
Diagnostic Test Analys
相册封面To-date, rule-space methodology has been applied to diagnostic test analysis across a variety of disciplines including mathematics tests, English language tests, document literacy tasks, and a mantically-rich architecture knowledge test.
Mathematics tests. The earlier applications of RSM focus on simple and well-defined mathematics domains, such as algebra, signed-number, fractions and exponents. The goal of the rule-space method in this type of domain is mainly to diagno the malfunction of examinees’ abilities to perform problem-solving steps or to detect unusual respon patterns as when completing arithmetic items (e.g., Tatsuoka, 1981; Tatsuoka & Tatsuoka, 1982; Tatsuoka, 1983; Tatusoka & Tatsuoka, 1983; Tatsuoka, 1985). For example, Tatsuoka (1983, 1985) ud the rule-space approach to classify and diagno tho procedures with which students struggled within solving signed-number subtraction problems. RSM has also been applied to more complex mathematical domains such as algebra (e.g., Birenbaum, Kelly, & Tatsuoka, 1993; Birenbaum, et al., 1994), fraction addition problems (e.g., Tatsuoka & Tatsuoka, 1987) and multiplication and division with exponents (e.g., Birenbaum & Tatsuoka, 1993). In
恶毒
each ca for the remediation purpo of student learning, RSM analys provided uful and explicit results.
English language tests. English language testing is another important field to which RSM has been applied. Tests of English language examined with RSM include cond-language reading comprehension (Buck, Tatsuoka, & Kostin, 1997; Kasai, 1997; Scott, 1998), listening testing (Buck & Tatsuoka, 1998), and SAT I verbal-ntence completion (Buck et al., 1998). Findings in the studies indicate that RSM can be ud
to modify and validate a t of the hypothesized attributes underlying complex performance on English language tests and also provide diagnostic information to individual test-takers. For example, Buck and his colleagues (1998) ud rule space procedures to develop a t of attributes for the Sentence Completion Section of SAT-I Verbal. They identified 20 cognitive and linguistic attributes along with 3 interactive attributes. Bad on the attributes, 97% of students were successfully classified into knowledge states, suggesting that the cognitive and linguistic attributes they identified accurately reflected the students’ performance on the Sentence Completion Section of SAT-I Verbal.
Document literacy tasks. Kirsch and Monthal (1990) developed a t of 22 cognitive attributes that underlie the performance of young adults on document literacy tasks. They classified the 22 cognitive attributes into three categories: document variables bad on the structure and complexity of the document, task variables bad on the structural relation between the document and the accompanying question or directive, and process variables bad on strategies ud to relate information in the question or directive to information in the document. Eighty-nine percent of the variance in item difficulty was accounted for by the attributes. Bad on this t of identified cognitive attributes, Sheehan, Tatsuoka, and Lewis (1993) applied rule-space methodology to diagnosing and classifying the examinees’ knowledge states underlying the performance on the nationwide survey of literacy skills. The results revealed that 157 valid knowledge states were identified and 90% of the examinees were classified into one or more of the 157 knowledge states.
rude名词Semantically-rich architecture knowledge test. Another application of rule space
methodology was mantically-rich architecture knowledge asssment (Katz et al., 1998). Katz and her colleagues collected respons from 122 samples from students majoring in architecture, architecture interns, and professional architects on a 22-item test of architectural knowledge. They fir哄宝宝睡觉的故事
st defined a cognitive model consisting of three process groups: understanding the item, solving the items, and checking performance. Each group of attributes was applied to parate rule-space analys. The results showed that the highest classification rates were found using the attributes of the solving category. The results also supported that the rule-space diagnostic technique extended its applicability to a more abstract discipline.
Other Applications创业故事
Construct equivalence. In addition to diagnosis, RSM can provide uful information for other psychometric issues. Rule-space methodology can be ud to test construct equivalence between two different language versions of mathematics aptitude tests. Guerrero (2001) in her disrtation compared the RSM to traditional psychometric methods for evaluating construct equivalence of the mathematics ction of the Scholastic Aptitude Test I (SAT-I) and the Prueba de Aptitude Academica (PAA-M), which is a Spanish-language version of SAT-I. Results indicated that a traditional psychometric approach using structural equation models found a single common factor accounting for performance on both tests. However, the RSM approach suggested that construct equivalence of the two language versions of mathematics aptitude tests is questionable bad on the cognitive attributes.
Dimensionality examination. Scott (1998) ud learning paths results derived from rule-space analys to examine the dimensionality of the reading ction of the
TOEFL test bad on the different skills applied by examinees. In the rule space analys, she found three major learning paths. Three paths reprented three different learning patterns of cognitive skills ud by students. Path 1 was the main learning path and had the highest percent of classified students. Path 2 and Path 3 were minor learning paths, which had fewer classified students compared to Path 1. Scott ud the artificial data ts bad on learning paths for dimensionality examination. She examined the actual data t and four artificial data ts. The first data t consisted of students who were classified into Path 1. The cond data t consisted of students who were classified into Path 1 and Path 2. The third data t consisted of students who were classified into Path 1 and Path 3. The fourth data t consisted of students who were classified into Path 3 and Path 2. The findings in her study revealed that unidimensionality of the reading ction of the TOEFL test was confirmed by the actual data t and the data ts 1, 2, and 3, which consisted of students who were classified into only one learning path or similar learning paths. However, the data ts exhibited multidimensionality when the data ts included students who were classified into two distinct learning paths, such as path 2 and path 3.
正常胎动次数快乐随心
Suggestions for Further Rearch
In addition to the substantive results, the current study bring forth veral suggestions for further rearch in terms of cognitively diagnostic asssment. One suggestion related directly to the rule-space method is a cutoff point of probability for attribute mastery. Different cutoff values will affect the outputs of rule-space analysis, such as learning paths. In order to make learning paths most reliable, it is very important to decide an optimal cutoff score. How to determine an optimal cutoff score in the rule-space analysis is a crucial and interesting topic in the future rearch.
Item Bank and Test Development
Under traditional psychometric models, item development and lection for banking bad on IRT-related item properties, such as item difficulty and item discrimination, is considered as traditional banking. As a result, theoretical constructs of tests resulting from this type of the item bank is still vague. By applying cognitive-related psychometric models, such as rule-space methodology, to validating the required cognitive attributes of the test, banking items by cognitive attributes is possible as well as more effective and informative than traditional banking. By decomposing items into the cognitive attributes that are required to answer correctly, items can be designated by the patt
社区网格化erns of cognitive attributes and by their difficulties. Using item banks that include cognitive attributes, computerized diagnostic tests can be developed to help teachers diagno students’ misconceptions and computerized remedial instruction can also be created to assist students (Henson & Douglas, 2005).
Some specific suggestions regarding item bank structures for diagnostic asssment can also be made. One suggestion of the current study is the need for item banks with relatively equal distributions of attributes across items. The current study was a condary data analysis and cognitive attributes were defined bad on the existing tests, which led to unequal distribution of items involved in attributes. This problem was found in this study and might result in the unstable attribute mastery probability estimates for diagnostic asssment. In the future, in order to obtain stable and reliable estimates of attribute mastery probabilities for cognitively diagnostic asssment, rearchers should first define a t of item attributes, and then develop enough test items for each attribute.
Another suggestion that can be made in this study is related to linking and
