20 February 1998

Dr. Leon McGinnis
5544 Trowbridge Dr.
Dunwoody, GA 30338

Dear Dr. McGinnis:

We appreciated the opportunity to meet with you and your colleagues to discuss your concerns about the use of the SAT II Subject Tests in lieu of the CPC requirements for admission into the University System of Georgia. While the meeting did not seem to alleviate your concerns about the possible impact of the SAT II on the admission prospects of home school students, we would like to address certain issues that we feel have been misrepresented. In particular, it is our considered position that your conclusion is incorrect that only 16% of the test takers would be eligible for admission to USG institutions. Because that was publicly reported on the HEIR web page and probably contributes in part for the concern of large numbers of home school parents, this letter represents our interpretation of the data and we hope that you will share this with your home school colleagues.

In your analysis, you used the SAT I to predict the percent of students who would pass the various SAT II tests. Your approach assumes a positive correlation among scores on the SAT I and each of the SAT II tests, i.e., those who do well on SAT I are more likely to do well on SAT II and vice versa. While this is not an unreasonable assumption, the correlations between the SAT I and SAT II tests vary from subject to subject. In other words, any prediction would probably be better from some subject tests than for others. In those SAT II Subject Tests where the correlation with the SA I is low, any passing estimates are likely to have a great deal of error associated with them. Conversely, if the correlations were perfect, the predictions of the number of students passing each subject test would be without error and there would be no need for SAT II tests. The fact is, because the correlation is less than perfect, the SAT II test provide information beyond what is provided by the SAT I. The SAT Subject Tests assess subject matter content closely approximating the College Preparatory Curriculum rather than verbal and mathematical reasoning skills as do the SAT I tests. It should also be noted that the College Preparatory Curriculum is the foundation of the new University System of Georgia admissions requirements; therefore, a reasonable assessment of a body of knowledge equivalent to the COPC is essential. The SAT I score is used in admissions decisions in addition to the College Preparatory Curriculum information and the high school grade point average.

In the second part of your analysis, you estimate the percent of students who would pass the four subject tests (American History, Math 1C, Biology and Chemistry) to be 39%. Here, unlike the first analysis, you assume that there is no correlation among the four subject tests. In order to apply the multiplicative rule, i.e., multiplying the inferred probability of passing one test by the inferred probability of passing a second test and so on, it is necessary to assume that the four tests are independent. IN other words, independence assumes that if a student did well on one subject test, he or she is no more likely to do well on another subject test than any other student. This is an implausible assumption. All of these cognitive measures are positively correlated; therefore, the amount of overlap in these four tests would likely insure a much higher pass rate over the four tests than the 39% you arrived at. Your conclusion that if the test takers had been required to pass all six SAT II Subject Tests for which cut scores have been established, the pass rate would be 16% is especially faulty since, again, it assumes no correlation, i.e., that the test are independent, among the six tests. It should be noted, though, that at the cut scores we have set, the pass rates for the individual SAT II Subject Tests range from 72% to 88%.

We assume that the probability of anyone's passing a given SAT II Subject Test without adequate preparation in the subject area is near zero, but we do not know how many students will or will not pass an individual SAT II Subject Test. In the absence of any empirical evidence it is impossible to know if the SAT II cut scores are more difficult than, comparable to or less difficult than the CPC requirements. It should be noted that empirical evidence is not without its flaws since data frequently are specific to the group taking the test and not generalizable to different populations that were not included in the original data gathering. In the absence of such data, ETS has used a conventional and widely practiced method for setting the cut scores, i.e., expert judgement. The Angoff Method is commonly used to set standards in areas as diverse as K-12 achievement tests and professional licensure and certification tests. As evidence becomes available from various populations taking the SAT II tests, we are prepared to adjust the cut scores if such a decision is indicated. In the absence of such data, it would be premature to further adjust the cut score based on flawed statistical analyses of data from students who may or may not be like those for whom the test will be used (i.e., homeschoolers). I should also point out that we chose not to sue the Study Passing Scores that the standard setting exercise produced. As a matter of fairness and allowing for a certain amount of error that is inherent in any measurement of this type, we decided to set the passing scores for most of the tests at one standard error of measurement below the study passing scores. You should also know that over the next several months, we will be working with ETS to develop a compensatory matrix that will allow students to present a range of scores yet still meet our requirements.

We hope that you will consider presenting this information to others in the home school community. Feel free to use this letter.

Sincerely,

James Muyskins
Senior Vice Chancellor for Academic Affairs.