Knowledge structures and skill assignments: Structural tools for diagnostic assessment

The overall aim of any assessment is to determine the skills a subject masters, and the more skills one has to solve the problems, the more problems one will solve; this simple idea is at the basis of all common theories of ability tests. Apart from this idea, the theories diverge in their assumptions about ”latent traits” or ”ability parameters” or other constructs that ”represent” the data as good as possible. A direct connection between theory and data is the construction of a proper operationalization of the field of interest: The researcher formulates the skills a subject needs in order to be able to solve a problem. Alternatively, given a set S of skills, an expert indicates for each subset X of S which problems can be solved with the skills in X. In a second step, a scaling mechanism is chosen which interprets the test results in terms of the skills and provides useful feedback to subject and researcher about the subject’s skill state. Just as knowledge assessment by modern scaling theory such as knowledge structures can be regarded as a qualitative way of measurement, the aim is to provide a qualitative description of a subject’s skill state. This is based on the conviction that a sensible diagnostic cannot be a point estimate of the knowledge of an individual such as a test score, but that, in most cases, only a range of skills can be given, which are more or less mastered by an individual.

In this talk I will present a test theory that can be built using skill and problem assignments, and how empirical data can be used to explore an underlying skill or problem function.