Georgia’s merit-based "Helping Outstanding Pupils Educationally" (HOPE) scholarship program awards scholarships to Georgia high school students who graduate with a "B" grade point average and attend college in-state. This study examines whether there are systematic differences in how "B" averages are awarded to Georgia’s students across school systems; whether it is easier to earn a "B" average, and become eligible for a HOPE scholarship, in some of Georgia’s school systems than in others. This becomes an interesting question considering the dual political purposes behind the HOPE scholarship: firstly, to expand educational access to good students regardless of social or economic status, and, secondly, to encourage those students to remain in state for their education. A merit-based scholarship designed to expand educational access should not discriminate according to the county of a student’s origin. Thus, this study seeks to establish whether such differences in HOPE eligibility beyond that explained by educational performance exist as an indicator of a need for further research in this area.

This study employs a three-year panel set of HOPE eligibility and SAT data with Georgia’s public school systems (i.e. Georgia county and city school systems) as the cross-sectional unit. By comparing predicted and actual values for HOPE eligibility, primarily using an F-Test and Tukey’s HSD method of comparisons, the hypothesis that all high school "B" averages are created equal in Georgia is rejected. Given the same average educational aptitude or achievement, it is easier to earn a "B" GPA in high school, hence HOPE eligibility, in some school systems than in others.

For qualified graduates of Georgia’s high schools, the HOPE scholarship pays tuition and fees plus a book stipend to attend Georgia’s public colleges and universities. For qualified graduates attending in-state private colleges and universities on a full time basis, HOPE pays a cash award similar in size to the benefit paid to students attending public colleges. For a rising college freshman to qualify for HOPE, a Georgia high school student must graduate with a "B" or better cumulative grade point average. Georgia phased out all income caps on eligibility by 1996, but beginning in 2000 the state only considers GPA in core academic classes when determining eligibility. To retain the HOPE scholarship, students must maintain a "B" grade point average while in college.

Based on its perceived success, Georgia’s HOPE program has served as the model for numerous states enacting less comprehensive merit-based scholarship programs. Given this national interest and a growing wealth of data, scholars are examining the effects of the HOPE program. Rubenstein and Scafidi (2000) consider the incidence of the implicit lottery tax combined with the distribution of benefits from all lottery-funded educational programs, including HOPE. They conclude that non-white, lower income households tend to purchase more lottery products, hence bearing more of the tax burden, and receive fewer benefits than white and higher income households. They further find that HOPE benefits particularly accrue to higher income, better educated households.

Dee (1998) examines whether HOPE has resulted in immigration to Georgia. He finds that on the Georgia side of Metropolitan Statistical Areas (MSAs) that straddle Georgia’s borders, residential construction in Georgia increased by 30 percent relative to the non-Georgia areas, and the real value of this construction increased by 40 percent. Furthermore, using similarly specified models, Dee also finds that Georgia public elementary school enrollments were rising in these MSAs.

Several studies examine the HOPE scholarship in higher education. Dee and Jackson (1999) examine a cohort of Georgia Tech students to determine the individual probabilities of college students losing HOPE eligibility. They find a strong relationship between measures of student ability in high school and retention of HOPE scholarships, with better students more likely to retain HOPE. After controlling for choice of academic discipline and student ability, they find the probability of retaining HOPE does not differ dramatically by race or ethnicity. Of particular interest to this study they state, "the empirical relevance of unobserved student attributes is underscored by the joint significance of fixed effects for each student’s county of origin…." (p. 381). Thus, when discussing HOPE retention among college students, the county of a student’s origin is of particular importance. Their main finding is the dramatic differentials in the probability of retaining HOPE across academic majors. Their models report students majoring in engineering, computing, or the natural sciences are 21 percent to 51 percent more likely to lose HOPE than students majoring in other disciplines.

Dynarski (2000) and Cornwell, Mustard, and Sridhar (2000) investigate HOPE’s impact on college enrollment. Dynarski estimates the HOPE scholarship increased college attendance rates in Georgia seven to eight percent for 18 to 19 year-olds. Furthermore, white, higher-income families account for the largest portion of this increase. Cornwell, Mustard, and Sridhar discover HOPE has differing effects, depending on whether the school in question is public or private, and whether it is a two-year or four-year institution. They estimate HOPE has increased Georgia first-time freshman enrollment by eight percentage points, with most of the increase accounted for in four-year schools. They conclude that HOPE has served primarily to influence college choice (that Georgia retains her high school graduates) rather than expand access to higher education (that more of Georgia’s graduates go on to colleges and universities).

HOPE has been less examined at the high school level. Bugler, Henry, and Rubenstein (1999) examine whether HOPE has caused high school grade inflation. Bugler, et al. find that HOPE eligibility grew rapidly, from below 50 percent of Georgia’s high school graduates in 1993 to nearly 60 percent in 1999. Over the same period, Georgia’s average SAT scores and cumulative grade point averages also rose. Using graduates’ cumulative grade point averages and average SAT scores as their measures, Bugler, et al. find no evidence that HOPE has caused or accelerated grade inflation. While grade inflation may be occurring, they find it to be part of a long-term, national trend that predates HOPE.

This study assumes well-educated, well-prepared students will tend to do well on standardized tests of student preparation/ability/achievement. Additionally, they will earn "B" averages or better, thereby qualifying for HOPE. In this fashion, school systems whose students are better prepared on average, using nationally standardized measures, should also have higher than average percentages of HOPE qualifiers. The converse would also be true. If "B" GPAs are the same across school systems, school system average scores on standardized measures should be predictably correlated with HOPE eligibility. Therefore, comparison of predicted HOPE eligibility to actual HOPE eligibility will allow researchers to examine whether "B" GPAs are created equal across the state.

Each of Georgia’s counties has a single independent school district, often comprised of several high schools. Therefore, the HOPE eligibility of a particular county is the total number or percentage of all students who qualify from the various high schools within the county school district. In addition there are twenty-one city school districts located within various counties, but operated independently of the county system. When necessary, city system data was incorporated into county data. Each system’s average standardized test scores are regressed on each school system’s percentage of students qualifying for HOPE to generate residuals. Analysis of these residuals indicates whether certain school systems award more than average or less than average HOPE eligibility for similar average scholastic preparation/ability as measured by the standardized tests. For interested readers, our data is described in the Appendix.

This study’s empirical approach is discussed in greater detail in the Appendix. School system average SAT scores are used to predict an expected level of HOPE eligibility—i.e., the expected percentage of HOPE eligible students in the school system. Subtracting the predicted value of HOPE eligibility from each school system’s actual value generates a set of "differences." So, a school system can "under award" HOPE, where the actual HOPE eligibility is less than that predicted by the system’s average SAT score, or a school system can "over award" HOPE, where actual eligibility is greater than that predicted by the system’s average SAT score. As examples, the most extreme values uncovered by this procedure are as follows: Towns County awarded HOPE eligibility to 26 per cent more students than expected in 1998, while Glascock County awarded HOPE eligibility to nearly 32 per cent fewer students than expected in 1999. Using various statistical tools, the authors analyze these "differences" within and across school systems to determine whether there are systematic differences in the way HOPE eligibility is earned: do certain school systems consistently "over award" or "under award" HOPE eligibility relative to the state at large? Are "B" averages unequal as one goes from system to system?

For a first pass, a visual analysis was conducted. The "differences" were ordered by magnitude for each year, and then split into groups of 17 counties to approximate deciles (each group of 17 represents 9.82 per cent of the total). The top two groupings, that is, the groups with the greatest "over-award" of HOPE per year are reported in Table 4. The bottom two groupings, that is, the groups with the greatest "under-award" of HOPE per year are reported in Table 5.

In the top "over-award" grouping (Table 4), fifteen systems appear twice during the three-year sample. The likelihood of at least this many systems appearing two or more times simply by chance in the top "over-award" grouping is only 0.000084. Twenty-six systems appear at least twice in the top two groupings combined. The likelihood of such an extreme concentration occurring in the top two groupings at least twice by chance is 0.011. Eight systems appear in all three years. The likelihood of such an extreme concentration occurring in the top two groupings at least by chance is 0.000053. These findings imply that systems that "over award" HOPE eligibility in one year tend to do so in following years.

In the top "under-award" grouping (Table 5), fourteen systems appear at least twice during the three year period, and three appear all three years. The likelihood of observing at least this many systems in the worst "under award" grouping by chance is 0.00029. Looking at the top two "under-award" groupings together, twenty-six systems occupy 60.8% of all possible entries. The likelihood of such an extreme concentration is 0.025. Ten systems appear all three years. The likelihood of such an extreme concentration is 0.00000093. These findings imply that systems that "under award" HOPE eligibility in one year tend to do so in following years.

The "differences" are then examined for systems that make wide swings during the three years—systems appearing in the top group and in the bottom group at least once during the period. Only two such systems were detected. The likelihood of seeing this few systems with such wide swings from bottom "under award" to top "over award" and vice versa by coincidence is only 0.0052. Again, systems that "over award" HOPE eligibility in one year tend to do so in following years, and vice versa. Thus there exists evidence of more systems ranking similarly over the three year period than one would expect, and fewer systems making wide swings than one would expect.

Statistical tests were conducted to determine the expected percentage of HOPE qualifiers for each school system each year, to determine differences between the actual allocation percentage of HOPE qualifiers and the expected percentage of HOPE qualifiers (residuals), to test for system-to-system differences, and to explain the differences found. As indicated earlier, the expected percentage of HOPE qualifiers is determined from the regression equation obtained from each year’s data. These predicted values are then compared to the actual percentage of HOPE qualifiers, producing a "difference" (residual) for each system for each year. If GPAs are awarded equally, the average "difference" (residual) for each county should be zero. Analysis of variance (ANOVA) was used to test whether the average residuals were the same from school system to school system. The F test associated with this analysis supports the conclusion of school system to school system differences.

Once differences were confirmed, attention was shifted to categorizing the differences. Tukey’s HSD method of multiple comparisons was used. This approach allows researchers to group school systems together and talk about differences between groups. As shown in Table 6, Parts A through C, Tukey’s HSD method indicates that school systems should be separated into twenty-nine groups. Some groups contain a single school system while others contain multiple school systems. For example, Group 10 includes Valdosta City, Burke County, and Vidalia City. Rather than talking about each of the systems in this group separately, any result that applies to one of these systems will also apply to the other two systems in Group 10. Based on this analysis, all but 49 school systems (those in Group 15) are providing significantly different HOPE eligibility to students from at least one other school system in the state, after accounting for student academic aptitude.

These findings allow the authors to reject the hypothesis that all "B" averages are created equal: that for similar average SAT achievements there are systematic differences in HOPE eligibility across Georgia’s public school systems. Therefore, based on similar SAT scores, students who fail to qualify for HOPE in one school system may have qualified in another, and vice versa.

This study finds evidence that not all Georgia high school "B" averages are created equal. It finds systematic differences in the way Georgia’s public school systems award "B" averages and hence eligibility for HOPE scholarships. Based on average student achievement measured by average SAT scores, some school systems systematically award more HOPE eligibility relative to the state wide average. Conversely, some school systems systematically award less HOPE eligibility relative to the state wide average. Dee and Jackson find the county of a student’s origin is important in predicting HOPE retention among college students. Similarly, this study finds the county of a student’s origin is important in predicting initial HOPE eligibility.

These results seem contrary to the spirit and political appeal of merit-based scholarships. There may be many reasons for these results. However, given the correlation between economic and demographic factors and SAT scores, the authors are inclined to believe the explanation lies outside such factors. Rather, because the HOPE eligibility standard is based on subjectively assigned high school grades, citizen influence at the local level may be responsible. This study does not conclusively demonstrate this, but its current results indicate a need for further research.

Bradbury, John Charles and Noel D. Campbell, "Who Gets HOPE? A Political Economy Analysis of the Determinants of HOPE Eligibility," manuscript under review, North Georgia College and State university, 2001.

Bugler, Daniel T., Gary T. Henry and Ross Rubenstein, "An Evaluation of Georgia’s HOPE Scholarship Program: Effects of HOPE on Grade Inflation, Academic Performance and College Enrollment," Council for School Performance, 1999.

Cornwell-Mustard HOPE Scholarship page, www.terry.uga.edu/hope/home.html

Cornwell, Christopher M., David B. Mustard, Deepa J. Sridhar (2000) "The Enrollment Effects of Merit-Based Financial Aid: Evidence from Georgia’s HOPE Scholarship." University of Georgia Working Paper, Athens, GA.

Dee, Thomas S., Linda Jackson (1999) "Who Loses HOPE? Attrition from Georgia’s College Scholarship Program." Southern Economic Journal, V. 66, no. 2: 379-390.

Dee, Thomas S. (1998) "Tiebout Goes to College: Evidence from the HOPE Scholarship Program." Georgia Institute of Technology Working Paper, Atlanta, GA.

Dynarski, Susan (2000) "Hope for Whom? Financial Aid for the Middle Class and Its Impact on College Attendance." NBER Working Paper 7756, Cambridge, MA

Education Commission of the States, www.ecs.org

Georgia Public Education Report Card, 1995-2000, Georgia Department of Education, 205 Butler Street, Atlanta, GA 30334.

Rubenstein, Ross, and Benjamin P. Scafidi (2000) "Who Pays and Who Benefits? Examining the Distributional Consequences of the Georgia Lottery for Education." Andrew Young School for Policy Studies, Georgia State University, Atlanta, GA.

---

 

With important caveats (See Appendix.) some systems in Part A may be distinguished from some systems in Part C.

With important caveats (see Appendix) Systems in Part B may NOT be distinguished from systems in Parts A or C.

With important caveats (See Appendix.) some systems in Part C may be distinguished from some systems in Parts A.

Data Source: Data was drawn from the Georgia Department of Education’s Public Education Report Card. This study considers only public school systems, as comparable educational data are unavailable from private schools. The study examines 173 of Georgia’s county and city school systems, discarding seven systems for insufficient data from 1996-1997 to 1998-1999. Baker, Chattahoochee, Clay, Quitman, Schley, Taliaferro, and Webster counties were discarded. These years were chosen because there was no "structural" change in the HOPE program: all income caps were phased out by 1996, and the phase-in of "core" GPA requirements did not begin until 2000. Average SAT scores by school system (county) by year were chosen as the independent measure of student achievement. Other moments and measures of variation of this variable are not available. While average SAT scores may not be ideal, ease of access and general acceptance may warrant their use. Furthermore, such standardized scores provide a consistent measure of academic achievement that will not differ across school systems, as internal grading standards (leading to an assigned GPA) may differ. The number of students eligible for HOPE and the total graduating class are reported directly. From this data, the authors calculate the percentage of HOPE eligible students. Selected summary statistics are presented in Tables 1 and 2. Potential complications could arise because many students take the SAT in their junior year of high school, and many students take the test more than once. The data does not allow the authors to discriminate along such margins.

Regression Analysis and Generation of Residuals: System average SAT scores and YEAR is regressed on the percentage of graduating students eligible for HOPE using a first order linear regression model:

1. [HOPE = b₀ + b₁SAT + b₂YEAR + e]

to generate predicted values. Additionally, models including the percentage of students taking the SAT as a regressor were estimated. Different counties have different rates of students taking the SAT. Students with little or no interest in secondary education will forego taking the SAT. One can imagine a county in which relatively few students take the SAT, but those who do have the strongest interest in and best prospects for attending college. In this instance this study’s measure of system-wide average SAT scores would be biased, "skimming the cream" of the county’s students. Therefore, these models were re-estimated including the percentage of students taking the SAT. Quantitatively, the results did not differ markedly. Qualitatively, the analysis and conclusions were unchanged.

Subtracting predicted values of HOPE eligibility from actual values of HOPE eligibility generated the residuals; thus, residuals are measured in percentage of students. A positive residual indicates that school system awarded more HOPE eligibility (more "B" averages) than the state at large for similar average SAT scores. A negative residual indicates that school system awarded less HOPE eligibility (fewer "B" averages) than the state at large for similar average SAT scores. Regression results are presented in Table 3. Since differences were noted from year to year, individual models were fit for each year. Using the data for each school system for three years, homogeneity of variance across school systems was considered as well as homogeneity of variance across SAT scores. Less than 3.5% of the school systems show more variation from year to year than would be expected based on the data available. Homogeneity of variance across SAT scores was analyzed through traditional regression residual analysis, and homogeneity of variance across school systems was analyzed through quality control methods that address stability of variation from small samples.

Estimating Likelihood of Empirical Results: For each statement about likelihood of observing a certain grouping, the same approach was followed. The authors order the residuals by magnitude for each year, divide the residuals into approximate deciles (17 observations per group—9.82 % of the observations), and count the number of times the stated outcome is observed. The top two groupings, that is, the groups with the greatest "over-award" of HOPE per year are reported in Table 4. The bottom two groupings, that is, the groups with the greatest "under-award" of HOPE per year are reported in Table 5. Probability of such extreme groupings occurring when the residuals are rank ordered are calculated based on the binomial distribution:

2. P(X = x) = _nC_xp^x(1-p)^n-x x = 0, 1, 2, …, n

Where n = number of trials, X = number of successes observed, x = the number of successes of interest, and p = probability of success on a single trial, _nC_x = n!/x!(n-x)!

and

3. P(X ³ x) = P(X = x) + P(X = x+1) + … + P (X = n)

For example: Suppose one wants to determine the probability that at least fifteen school systems will appear in the top "over-award" group at least twice in the three year period.

P(X ³ 15) = P(X=15) + P(X=16 ) + …+ P(X=173) = 1 – [P(X=0) + P(X=1) + … + P(X=14)]

X = the number of systems appearing in the top tier two or more years in three years

n = the number of trials; i.e., the number of systems included = 173

p = the probability of success for any system; i.e., the probability a system is in the top tier two or more years where this is calculated by another binomial:

4. p = P(Y ³ 2) = S _y=2³ ₃C_y (17/173)^y (1-17/173)^3-y

Y = the number of times in the top tier

n_y = 3

p_y = 17 / 173; the likelihood of being in the top 17 of 173 by random assignment

In the top "over-award" grouping fifteen systems appear twice during the three-year sample. These fifteen systems account for 58.8% of the entries in this table. Based on calculations from the above formulas, the likelihood of at least this many systems appearing two or more times by chance is only 0.000084.

Similar calculations produce the additional results. In the second "over-award" grouping ten systems, 27.4% of the 51 possible entries are accounted for by six systems. Two systems appeared in this grouping all three years. Looking at the top two "over-award" groupings together, twenty-six systems appear at least twice in three years (with eight of these systems appearing all three years). The likelihood of at least extreme groupings occurring by chance are 0.011 and 0.000053 respectively.

In the top "under-award" (bottom) grouping fourteen systems appear at least twice during the three year period (with three of these appearing all three years). The chances of observing at least this many systems by coincidence are 0.00029 and 0.00064. In second-to-bottom grouping, eight systems appear twice in three years. Looking at the top two "under-award" (bottom) groupings together, twenty-six systems occupy 60.8% of the possible cells in the table (with ten of these systems appearing all three years). The likelihood of such extreme groupings is 0.025 and 0.00000093 respectively.

Seeking out systems that make wide swings during the three years (appearing in the top group and in the bottom group at least once during the period), only two systems are detected. The chance of seeing this few systems with such swings, by coincidence, is only 0.0052. If one considers school systems that appear in the top thirty-four and the bottom thirty-four during the three-year period, one finds eleven systems. The likelihood of this few systems making such a swing, by chance alone, is only 0.0000031.

Analyzing School System-to-School System Differences: The regression models developed for each year were used to generate residuals. In turn, these residuals were used as input to a one-way Analysis of Variance (with three observations per school system). ANOVA tests to see if effects are the same from school system to school system. The analysis supports the conclusion of system-to-system differences (F = 3.89; p-value = 3.95x10^-27). Since differences were noted, Tukey’s HSD method of multiple comparisons was used for post hoc comparison of school systems. The average residual was computed for each school system, these were ranked, and differences were analyzed. Differences of at least 20.99 were statistically significant (a = .05). Using Tukey’s procedure the authors are able to cluster the school systems into 29 groups. Conclusions for any school system within a group will apply to all school systems in that group. The average residuals and the groups are shown in Table 6. Results show statistically significant differences between Group 1 (Jefferson County) and any system in Groups 16 or above—i.e., the deviation in percent of students eligible for HOPE in Jefferson County is significantly lower than the deviation from 91 systems (almost 53% of the systems). Similarly, the following statements can be made in terms of statistically significant differences: Systems in Group 2 can be distinguished from systems in Groups 17 or above; Group 3 from Groups 18 or above; Group 4 from Groups 19 or above; Group 5 from Groups 20 or above; and so on through Group 14 from Group 29. Therefore, on the high end, the deviation for Bremen City is significantly higher than the deviations for 33 school systems (approximately 19%). The systems in Group 15 cannot be distinguished from any of the other groups. Therefore, all but 49 school systems (those in Group 15) are providing significantly different HOPE eligibility to students from at least one other school system in the state.

Note: The authors wish to thank Edward Lopez, John Charles Bradbury, and Carole Scott for helpful comments. All remaining errors are the authors’ responsibility.