INTRODUCTION

            Classifying student errors is beneficial in a pedagogical sense. It is perhaps a unique approach to economic education to focus on student errors, rather than measures of achievement. However, they are, in fact, complements to each other. One of the advantages of studying learning dysfunctions is that we can more readily identify problem areas and hence solutions to problems. Every instructor who sifts through student essays on economic applications immediately identifies mistakes and problems for that moment, and he or she is usually able to make judgments and evaluations on the seriousness of the error.

Rarely are the essay exams themselves looked at as a valid observation of student understanding to be scrutinized and recorded. On the other hand, instructors using multiple choice exams often perform item analysis on the various choices students make in order to determine which of the incorrect answers distracted the student enough to make the wrong choice. Item analysis of this type can an important investigation of student errors in order to improve exam questions. However, it may lead to simply making more clever and tricky distractors; which is not exactly conducive to learning. The point here is that the analysis of student errors on tests can provide the means to improve tests of that nature.

In this paper we seek to develop a system of classifying student errors from essay exam answers in the context of an introductory economics course.  Since the response is not finite as in the multiple choice test, a system of classification of errors must be developed. However, in this case the use of the information is not so much to make a better test, but to try to understand the nature of student thinking so that instruction can be improved.

This paper describes the subject of economic ideas upon which the error classification system is aimed. It is identified and its importance established. Subsequently, a classification system is developed and an application is described. The results of the application are then analyzed and some conclusions are drawn.

LITERATURE REVIEW

             A variety of researches encompassing eclectic academic disciplines have attempted to devise classification grading schemes. Lee (1990) proposes a system for classifying errors in linguistics and language teaching theory. The author suggested four complementary ways of categorizing errors: error vs. mistake; linguistic phenomenon; gravity of the error; and difficulty of correction. Hendrickson (1979) depicts a method for classifying errors of second language learners. Druger (2000) introduced  grading classification in the natural sciences in his study dealing with introductory college biology.

The theory of classifying errors is cogently analyzed in Valentine (1982). The author emphasizes the importance of sufficient evidence of discriminant validity in regards to evaluating errors classification schemes. Durfee (1993) emphasizes the need for positive feedback in regards to developing errors classification schemes. Livingston and Lewis (1995) introduced the concept of classification consistency and classification accuracy, whereas Hoffman and Wise (2000) introduced the concept of classification reliability.

THE COMPETITIVE MARKET MODEL  

The competitive market model is a fundamental part of most introductory economics courses and the texts which are used. This material generally includes a coverage of demand and supply, laying out the ceteris paribus demand curve including the principle of the law of demand. Market price and quantity determination is covered in schedules and graphs and the equilibrium process completes a verbal description of price determination. The uses of the model include a demonstration of reasons for price movements utilizing the relaxation of the ceteris paribus assumption for the various non- price factors of demand and supply. The list of these factors is fairly standard; income, price of related goods, number of buyers, tastes and preferences and expectations on the demand side; cost of resources, number of sellers, technology and expectations on the supply side. This model of the product market is also utilized to illustrate effects of  price fixing arrangement and tax burden. Elasticity measures allow certain generalizations about market operations, especially when connected to revenue generated by the market.  

Most introductory courses continue the development of the market system by illustrating the circular flow mechanism and the operation of factor markets including especially labor, and less frequently markets for capital and natural resources. These markets form the basis for a consideration of the functional distribution of income. Details of the role of profit round out the distribution aspects of the circular flow. The treatment of these aspects of the market are generally static with partial equilibrium structures utilized to demonstrate basic principles.

The understanding of the competitive market model described above is normally high on the priority list of learning objectives for the introductory economics course. While the introductory economics course typically shares the semester calendar with various macro topics, it is clear that with enough choices of topics to fill several semester calendars the competitive market model is the topic of choice for most economics instructors.  It is clearly important in understanding the economic environment in which we live.

Instructors may differ in the method and approach to teaching the competitive market model. Some may treat it as strictly an abstract cognitive exercise; others may insist on the development of applications under different circumstances. The premise of this paper is that the competitive market model is important enough with wide and diverse applicability I that students should be required to demonstrate competence in using it as a tool of analysis. This implies competence in correctly utilizing supply and demand diagrams and adequately explaining its conclusions and implications.

METHODOLOGY

The focus of this paper is on the classification and analysis of learning dysfunctions of students of the competitive market model. In particular the learning dysfunctions involve a large number of observable errors and characteristics of student answers to short word problems involving the supply and demand model. The list of errors includes 20 observable characteristics of the diagram submitted by the student in response to the word problem and seven characteristics of the written portion of the answer.

These 27 elements comprise the scope of the error set analyzed in this paper. From the classification of these 27 error types, it is necessary to proceed to meaningful analysis. Some aggregation will aid the analysis. The error types are collected into groups of similar errors in order to concentrate the data and make statistical analysis more meaningful. The error types are compiled into three major groups and a total or overall error category. These are:  

1.  Failure to attempt an answer

2.  Diagram errors

3.  Serious problems of written expression

            The failure to attempt an answer involves leaving out all or a portion of what the question asks for. The obvious cases are where there is a diagram present with no written essay, or a written answer without a diagram.  Turning in a blank page is also an indication of this type of error. The failure to attempt an answer includes observations regarding the students’ answer indicating frustration and impatience and uncertainty in formulating an answer. This category includes extreme sloppiness of  a diagram, one or more false starts or erasures, extreme grammatical faults that impede evaluation, and extremely sloppy penmanship that impedes evaluation.

Diagram errors include a wide variety of faults regarding the construction of the diagram. Diagrams in these particular word problems involve a one-factor relaxation of the ceteris paribus assumption on a normal supply and demand presentation either in the product market or the factor market. Errors include missing arts, logical errors, labeling detail faults, and others. The missing parts error pertains to student presented diagrams which were odd or abnormal, where no shift is indicated but the supply and demand remains intact, or where there is a missing supply or a missing demand curve. 

Logical errors include instances where: the wrong curve is shifted; the correct curve is shifted in the wrong direction; both supply and demand are shifted in a one shift problem; equilibrium price and quantity are out of correspondence with supply-demand intersections; supply and demand are placed on different graphs; and the students focus is on the wrong market in a problem involving substitutes or complements. Labeling problems also plague student response, though they may not be as serious as some other types of errors.  Labeling mistakes include supply and demand curves labeling reversed; odd-ball labels on supply or demand or on the axes which are left unexplained by the student; and made up numbers placed on the axes which are unrelated to the problem and other similar errors.

            Written expression errors involve characteristics of the written portion of the response with the following possible characteristics: extreme difficulty in stating cause and effect in the context of a one-factor shift in the market analysis; minimized verbal expression, that is, being too brief or crimped in the response so that it does not sufficiently underscore the diagrammatic analysis; and excessive verbiage which is incorrect, otherwise referred to as "shotgunning" or guessing.

Obviously any given student may not have many or even any of these error characteristics. It is only by analyzing the totality of these error patterns that we might learn something and be able to make teaching recommendations on how to assist students in minimizing errors.

This classification scheme was applied to an introductory economics class at the University of Texas-Pan American. The analysis which follows is based on these observations. The course was roughly divided into two parts, the first of which dealt with a variety of topics but focused on the microeconomics of market analysis, including both product and factor markets. The second half dealt with macro topics, including output, prices, unemployment and the like and focused on the aggregate supply and demand analyses.

The analysis presented here deals only with the micro portions of the analysis. The observations made in the application of the error classification system described above involved part of the examinations that were given. There were several questions involved, one in each of two exams.  These included a question on Exam 1 that involved the simple product market model of supply and demand and a two-part question on Exam 2 that involved the application of the market model for the product and one related to a related factor market--labor in this case. This two-part problem was meant to involve the interrelatedness of product and factor markets.  Students were schooled only on immediate impacts and were not expected to trace out general equilibrium repercussions on related product or factor markets. Additionally, at the end of the semester students were given an opportunity to retake either exam with a second chance test which covered similar material as was on the first test. Students were given the option to take this make up test or not; except those who had missed the earlier test were required to take the make up.

RESULTS

            Observations were taken on all word problem answers provided by the students in each of these exams. The specific wording of the word problems included is included in Appendix A. Classification summaries were made according to the system described above and dummy variables were employed. Table 1 presents the total errors by classification for the introductory economics class under observation.

The analysis which follows attempts to answer the question; when do students make errors? The question of  "when" errors are made focuses on the issue of whether students tend to repeat the same errors in subsequent tests. Here an association is made between Test 1 and Test 2 in terms of the type of error.

The fact that just about all students make errors at some time is not surprising; it becomes a task for student and instructor to learn from those mistakes and make the necessary adjustments, since these errors include less and more serious types of errors. It would be expected that even the better students make some errors.

It is also of interest as to whether student errors are persistent: do students repeat error?. Table 2 is a tabulation of all errors by type for the two exams relating to the micro concepts of the market model of supply and demand and the two makeup tests. These data indicate that in terms of total number of errors made, there is a drop of about 20 percent from the first to the second exam. However, this drop is entirely due to errors of expression not being repeated. Other types of errors: failure to attempt and diagram errors, stayed about the same from the first to the second test.

However, these totals do not tell the whole story. What is really of interest is whether or not any given student is repeating the same type of mistake. Taking Exams 1 and 2 together, four possibilities might be present. (Where 0 = no error, 1 = error)  

            Table 3 presents the cell counts for these four possibilities; with respect to the different error types. Under column A are those who made no error in either Exam l or Exam 2. The B column includes those who made no errors on the first exam, but made an error on the second. Since both tests involve applications of the basic market model, this category represents to some extent students who possessed the skills on the first exam but did not retain them. This is referred to as the forget factor. Under columns C and D are the number of students who had an error on the first exam. Column C indicates out of those how many did not repeat the error, while column D indicates those that repeated the error.

            Two ratios are included in Table 3; the forget factor or the percent of those who made no errors on Exam 1 in any given error category who made an error on Exam 2. The other ratio is the repeat factor or the percent of those making an error on Exam 1 who made the same type error on Exam 2. Overall, students had a high repeat factor 58.3 percent, that is, if they had an error of any type on Exam 1 then almost six out of ten also has an error of some type on Exam 2. There is no way of interpreting this figure as far as repeating the same type of error. However, more can be learned by narrowing the type of error when looking for repeated errors.  Overall at the narrower levels there were very low repetition rates with the exception of missing parts.  Here the repeat factor was 50 percent.

            The tendency to make errors on the second exam when the first was error free, or the forget factor, was most prevalent in diagram errors. In particular, the logical errors related to diagramming. Mistakes such as shifting the wrong curve or the right curve in the wrong direction may appear in the second test with new situational description. It may also be due to forgetting the details of the model. Also, the sloppiness errors had a 24.1 percent forget error factor which was also quite high. This is surprising since neatness or sloppiness is usually thought to be habitual in nature. Perhaps the high forget factor in this category is more related to the frustration coming through, of facing a more trying test the second time around. Otherwise, the forget factors quite low for other types of errors.

CONCLUSION

This paper has attempted to develop a classification system for categorizing student errors on word problems associated with the supply and demand market model. This classification system was applied in a class of introductory economics students at UT-Pan American. The indications from this application are that most students make errors at one time or another in the class.

            There was some tendency for students to repeat errors of diagramming dealing with missing parts and to make logical errors in graphing after a successful error free graph on an earlier test.

Future research should increase the sample size in order to improve the external validity of the classification system. It might also be important to consider some screening method at the beginning of the course to identify those students who might be predicted to have difficulty with graphic analysis, and instead of moving on, have a remedial or study session with that group to review some of the basics of graphs.

For the general student group it might be helpful if the course objectives point to a serious intent to have students empowered with the ability to apply the supply and demand market model to develop a series of practice problems which may be used to correct errors or notify the student of errors and which will not bare negatively on the students grade. If student learning involves making mistakes, there should be costless means of correcting these mistakes before tests which determine grades are given.