Analysis of Inequities in the Requirements for Admission to the University System of Georgia

Appendix: Statistical Analyses of the Fairness of SAT II Test Requirements

Background Information

When Dr. Barry Fullerton, Vice Chancellor for the GA Board of Regents, addressed the Georgia House Education Committee on July 29, 1997, he stated that the Board’s intent in using the SAT II tests was to insure that applicants from home study programs meet the same minimum requirements as graduates from accredited programs.

The USG contracted with the Educational Testing Service (ETS) to create a "passing score" for each SAT II test. The passing score for each test is one that the ETS predicts could be achieved by the group of "minimally knowledgeable" accredited applicants. The USG defined the "minimally knowledgeable" student as one who earned an SAT I composite score of 830 (in the 19^th percentile), completed the CPC course for the subject being reviewed with a "C", and had a HSGPA of 2.0. This minimally knowledgeable student would have an FI of 1830 and, therefore, would qualify only for admission to an associate-level college. The professional staff at ETS have made no claim that they cannot determine valid, verifiable "grade equivalent" scores.

Few, if any, accredited applicants who match the minimally knowledgeable profile take any SAT II tests, so no empirical evidence exists of the scores that they could achieve. Of the small percentage of students who did take one or more SAT II tests, most were seeking advanced college placement in the tested subject area or were applying to highly selective colleges. According to the College Board, the U.S. students who took at least one SAT II test in 1996-1997, produced a median composite score of 1195 (in the 79^th percentile) on the SAT I. In other words, there are 365 points and 60 percentile points separating the imagined minimally knowledgeable accredited student and the average of those students taking at least one SAT II test. As you can see, the students who traditionally take the SAT II tests are exceptional students ("very, very good students" in the words of Dr Fullerton.)

In the absence of empirical evidence, the ETS used the "Angoff" method for setting the passing scores. Rather than discuss the methodological flaws in this process, which are fundamental, simply consider the results.

Compare the ETS-predicted performance of the minimally knowledgeable accredited applicant with that of the students who actually have taken one or more the SAT II tests ("test takers"). Given the exceptional ability of the test takers and the fact that the vast majority of them graduated from an accredited program, then virtually all of them would have at least a C in the corresponding CPC course. If the SAT II standard was fair, then the pass rate for the test takers on the individual tests would be very close to 100%. However, as shown in Table 1 below, this is not the case. The passing score predicted by the ETS as being achievable by the minimally knowledgeable applicants is indicated for each of six SAT II tests in column 2. (For some tests the report lists two scores but not how to reconcile them, so both scores are included.) "Percentile" indicates the percentage of U.S. test takers in 1996-1997 who would have scored at that level or below. This value is reported in the "Admission Staff Handbook for the SAT Program: 1996-97" (an ETS Publication).

Incredible as it may seem, the results of this process predict that a minimally knowledgeable applicant would score better than more than:

• 42% of the "very, very good" students who took the Writing test,
• 49% of the "very, very good" students who took the American History test,
• up to 39% of the "very, very good" students who took the Math IC test,
• up to 20% of the "very, very good" students who took the Biology test,
• 36% of the "very, very good" students who took the Chemistry test, and
• 57% of the "very, very good" students who took the Physics test.

The ETS scores are so incredibly unbelievable that the USG has since made the completely arbitrary decision to "adjust" the passing scores down by one standard error.

Table 2 presents the passing scores presently required by the USG for the SAT II tests and it presents the performance of GA students who took one or more SAT II tests in 1998. The GA students who take at least one SAT II test are an even more exceptional group than the US SAT II test takers. In 1998, 78% of the GA SAT II test takers took three or fewer SAT II tests. The average of the composite SAT I scores for GA students who took an SAT II test was 1259 (87^th percentile), 429 points and 69 percentile points above that of an applicant matching the "minimally knowledgeable" profile.

The data on the SAT II test results for GA students in 1998 comes from the web site: http://collegeboard.org/sat/cbsenior/yr1998/ga/cbs1998.html. The BoR’s passing scores are from the USG Academic Affairs Handbook as posted at section 3.01.01.c.2.a (which is about half-way down the page) http://www.usg.edu/admin/accaff/handbook/section3/301-312/3.01-3.12.html, and, for the scores not posted there, from subsequent USG announcements. Column 1 lists the subject and the number of GA students who actually took the test in 1997-1998, column 2 shows their average SAT I composite score and corresponding percentile, and column 3 lists the average score they achieved on the subject test. These values are taken from the College Board web site. Column 4 lists the USG specified passing score. Column 5 indicates the percent of GA test takers who would have passed. This was determined by interpolating in the percentile distribution chart provided in the College Board web site and subtracting the percentile from 100%.

How many of these "very, very good" students, the actual SAT II test takers, have satisfied the new admissions requirements for non-accredited applicants?

Assuming a 100% correlation between success on any one test and success on any other, then 38% of these exceptional students would have passed all seven of the tests. (0.87**7 = 38%). Assuming no correlation between success on any one test and success on any other test, then, as few as 15% of these exceptional students passed seven of the tests. (80% x 77% x 77% x 56% x 83% x 87% x 80% = 15%) The actual result would be somewhere between the two extremes. Conservatively, more than 62% of these exceptional students would have had 2 or more deficiencies in the 16-unit CPC.

Obviously, the results for this group after taking seven SAT II tests and attempting to meet the requirements for a fourth mathematics CPC unit and attempting to demonstrate foreign language proficiency at the level of two years of high school "study" would be worse.

Please note that the above results were calculated with data from the Chemistry test (the best case) and not the Physics test. Each applicant may take either the Chemistry or the Physics test and a higher portion of Chemistry test takers would have met the prescribed cut score.

If the SAT II standard was fair, then accredited applicants, if held to that standard, would be enrolled into each sector at the same rates as when they had met the freshman admission requirements involving CPC and FI standards.

We have predicted the success rate of the actual GA SAT II test takers. If required to meet the SAT II standard, how would the performance of freshmen actually enrolled in the research university sector in 1996/1997 compare to that of the actual test takers?

In 1996/1997 (the last year for which the USG has published the data on their web site), the average of the composite SAT I scores for all research university freshmen was 1160 (in the 74^th percentile), 99 points and 13 percentile points below the average performance of the GA SAT II test takers. Most of the GA SAT II test takers were prepared for and seeking advanced placement in the tested subject. 78% of the GA SAT II test takers took three or fewer SAT II tests. Their average performance on each test is higher than what could be expected for the average of all research sector enrollees having to take seven SAT II tests. At best, the performance of research sector freshmen in 1996/1997 would have approached that of the exceptional test takers in meeting the SAT II standard. Conservatively, if required to take seven SAT II tests, more than 62% of the freshmen enrolled in the research sector in 1996 would have had 2 or more CPC deficiencies. Applicants deficient in the 16-unit CPC cannot be admitted to the research sector except as Presidential Exceptions.

If required to meet the SAT II standard, how would the performance of freshmen actually enrolled in the associate-level colleges in 1996/1997 compare to that of the actual test takers?

In 1996/1997, the average of the composite SAT I scores for associate-level college freshmen was 910 (in the 30^th percentile), 349 points and 57 percentile points below the average performance of the GA SAT II test takers. Any claim that the associate-level college freshmen could even approach the performance of the actual test takers in meeting the SAT II standard would be patently ridiculous. If all applicants to associate-level colleges in 1996 had been held to the standard for non-accredited applicants, then the associate-level colleges would have had only Presidential Exceptions in the freshman class.

About HEIR ||| Home ||| Contents ||| News ||| Opinion