The Gadfly, May 6, 2009
The Washington, D.C. public school system is among the nation’s worst. In fact, it’s relatively uncontroversial to say that public schools in D.C. are the worst in the nation—despite the District spending over $15,000 per pupil in its public school system, by far the highest in the nation and well above the national average of $10,071. Despite spending more per pupil than any state, students in the D.C. school system consistently rank among the lowest in critical learning areas. The National Assessment of Educational Progress (NAEPP)—an annual report released by the National Center for Educational Statistics (NCES)—reported in its 2007 study that students in Washington, D.C. were being outperformed by every state in the nation. The Heritage Foundation writes, “On the 2007 NAEP, 61 percent of fourth-grade students scored ‘below basic’ in reading, and 51 percent scored ‘below basic’ in math. Among eighth-grade students, 52 percent scored ‘below basic’ in reading, and 66 percent scored ‘below basic’ in math.” The NCES estimates that fewer than 60% of D.C. students graduate high school, and the Washington Post found that “only 9 percent of D.C. public school freshmen will complete college within five years of graduating from high school.” The high dropout rate and low rate of higher education has had a demonstrable effect on the rates of adult illiteracy, poverty, homelessness, incarceration and various other social pathologies.
Schools in Washington, D.C. are also mismanaged, violent, and falling apart. In June 2007, the Washington Post reported, “The schools spent $25 million on a computer system to manage personnel that had to be discarded because there was no accurate list of employees to use as a starting point. . . . It also lacks an accurate list of its 55,000-plus students, although it pays $900,000 to a consultant each year to keep count.” Another report shows that before the 2007-08 school year, “half of the city’s public schools [did not] have all of their textbooks arrive on time and [did] not have functioning air conditioning units.” Despite these failures, D.C. ranks first in the nation in the percentage of its budget spent on administration (44%), and last in percentage spent on teachers and instruction (56%).
In a biannual study measuring risky behavior among youth, the Department of Health and Human Services found that 14.4% of students in D.C. public schools skipped one day of school in the thirty days before the survey because “they felt unsafe at school or on their way to or from school.” This is almost three times higher than the national average. According to the Washington Post, “Over half of teenage students attend schools that meet the District’s definition of ‘persistently dangerous’ because of the number of violent crimes, according to an analysis of school reports.”
This is not to say that D.C. public schools are hopeless. The recent enactment of mayoral control has allowed Mayor Adrian Fenty and Schools Chancellor Michelle Rhee to enact radical changes in the public school system, and the District’s 72 public charter schools that service over 20,000 students have been largely successful. Another successful program in overhauling the state of public education in Washington, D.C. has been the much-talked-about (and in some circles much-maligned) D.C. Vouchers Program.
In January 2004, President Bush signed the District of Columbia School Choice Incentive Act, a five-year trial program that created the D.C. Opportunity Scholarship Program (OSP). Under the OSP, families in the District with an annual income of less than 185% of the poverty line are eligible for a $7,500 scholarship to attend a private school in Washington, D.C. The trial program was highly popular among parents, with over 7,200 eligible students applying for the 1,700 scholarships. Despite the OSP being a Republican program, Mayor Fenty, a Democrat, has been an ardent supporter and has made OSP, along with increased funding and public charter schools, part of his three-pronged plan for reforming D.C. schools.
The Three-Year Evaluation
In the legislation creating the Opportunity Scholarship Program, Congress required the Department of Education to commission annual studies of the effectiveness of the program. Specifically, the bill required that “the evaluation is conducted using the strongest possible research design for determining the effectiveness of the programs funded under this title.” The Department commissioned a team of researchers led by Patrick Wolf, Professor and 21st Century Chair in School Choice at the University of Arkansas to study effects after three years. The study compared the outcomes of applicants who were randomly chosen to receive an opportunity scholarship (via a lottery) with those eligible applicants who were not chosen to receive a scholarship. The former were assigned to the treatment group, while the latter served as a control group. To date, there have been five groups of applicants to the scholarship program, beginning in spring 2004 (cohort 1) and most recently in the spring of 2008 (cohort 5). In all, 7,852 students applied for voucher-based scholarships in D.C., of whom 5,331 were eligible for scholarships and 3,738 were awarded scholarships via a random lottery. For the 2008-09 school year, 1,714 students were enrolled in the OSP.
In order to gauge both short- and long-term effects of the scholarship program, the study was focused on those students whose outcomes could be tracked over three or more years (cohorts 1 and 2). Of the 5,818 applications in the first two years, 4,047 (approximately 70%) were eligible to enter the program. These eligible students were entered into a lottery, with certain groups of applicants being given higher priority in the lottery than others. Applicants attending “schools in need of improvement” (SINI), as categorized under No Child Left Behind, were given highest priority; then applicants attending non-SINI public schools; and finally those already attending private schools. Applicants already enrolled in private schools and some applicants in oversubscribed grades were excluded from the impact sample.
The impact sample for the study was composed of 2,308 applicants—492 from cohort 1 and 1,816 from cohort 2. Students in each of these cohorts were randomly assigned by the lottery to either the treatment group (meaning they received a scholarship) or control group (did not receive a scholarship). The treatment group comprised 1,387 students and the control group comprised 921. According to the authors, this impact sample of 2,308 represents one of the largest ever observed in voucher program analysis.
The evaluation collected four types of data: academic assessments, parent surveys, student surveys, and principal surveys. Academic assessment was based of the Stanford Achievement Test Series 9 (SAT-9), a series of standardized test administered to all students in the D.C. public schooling system and participating OSP schools. Several subgroups of public policy interest were designated prior to data collection. The subgroups are: SINI and non-SINI participants; high and low performance; males and females; K-8 and 9-12 students; and cohorts 1 and 2. Reports in 2007 and 2008 described the program impacts after one and two years. In neither of the previous two years were statistically significant academic gains observed overall or for students attending SINI schools. In years one and two, parent satisfaction surveys reported a positive impact on parental perception of the school, in terms of both academics and safety. Student, however, did not report being more satisfied or feeling safer in their new schools.
The 2009 study places its primary emphasis on academic achievement. The study found statistically significant improvements in reading overall and within five of the ten subgroups. Students attending private schools through the voucher program were reading four months ahead of those who did not receive scholarships—or, nearly a half-grade level ahead. The earliest participants in the program (cohort 1) were reading a full nineteen months ahead of their peers after three years participating in the program. This analysis, however, did not find statistically significant impacts on student math scores.
Of parents applying to the OSP in cohort 1, 17% listed school safety as the most important reason for seeking to exercise school choice. This was second only to academic quality (48%). Because there is no quantitative measure for school safety as there is for academic achievement (SAT-9s), researchers studied how parents and students perceived the levels of vandalism, cheating, bullying, drug distribution, drug and alcohol use, tardiness, truancy, fighting, racial conflict, weapons, and teacher absenteeism. Parents were asked whether each of these issues were “very serious,” “somewhat serious,” or “not serious” at their child’s school, and from those results developed a safety index of 0 to 10. Parents in the control group reported a mean safety index of 7.07, while parents in the treatment group reported a mean safety index of 8.08, a significant differential. One of the largest subgroup differentials was those in the treatment groups who left SINI schools. Those parents reported a safety differential of 1.16. Every subgroup of parents in the treatment reported significantly higher safety levels at their child’s new school.
Student safety reports asked students whether they personally had been victims of theft, drug dealing, assault, threats, bullying, or taunting, or had observed weapons on their school grounds. Student reports of improved safety, unlike parent reports, showed no statistically significant gains in safety, with only a .12 safety index differential. No subgroups reported statistically significant gains, either. Similarly, parents in the treatment group reported significantly higher opinions of their child’s new school in satisfaction surveys, while students reported little difference.
The first implication of the study is clear: give the OSP more time. The legislation creating the scholarship program included a sunset clause, which stipulates that, unless the program is extended by Congress, the program would expire at the end of the 2009-10 school year. The Washington Post, the more liberal of the two major Beltway dailies, wrote in an editorial, “The study’s findings are no slam-dunk for the program’s success, but they are, by no means, proof of failure. Indeed, for the first time, researchers found statistically significant improvement in reading test scores for students offered vouchers and that, at the very least, demands further study.” The Wall Street Journal further chimed in, “Voucher recipients are doing no better in math but they’re doing no worse. Which means that no voucher participant is in worse academic shape than before, and many students are much better off.”
Patience is a deadly virtue for elected officials, and it’s hard for members of Congress to see beyond their next elections. But policymakers should give the OSP an opportunity to succeed. In just three years, students are already performing significantly better in reading achievement. Generally, the benefits of voucher programs come with time. Jay Greene, a Senior Fellow at the Manhattan Institute, says that, “There are transition difficulties, a culture shock upon entering a school where you’re expected to pay attention, learn, do homework. But these results fit a pattern that we’ve seen in other evaluations of vouchers. Benefits compound over time.” There have been eight previous studies looking at the longer-term effects of voucher programs in various states and cities. Seven of these studies showed that students using scholarships outperform those who remained in public schools, and that these benefits increase over time.
Just two days before the report was released, Rachel Chaney, Education Fellow at the Pacific Research Institute, wrote, “[Opponents] point to a Department of Education study of the first two years of the program that shows no significant difference in test scores between students in voucher schools and otherwise. […] Multiple studies of state voucher programs [show that] improvement often becomes noticeable after three to four years of participation, a fact overlooked by those who use the two-year Department of Education study.” The subsequent release of the three-year report proved Chaney right, and voucher naysayers wrong.
Policymakers should also encourage early application to the OSP by giving younger students priority in the lottery. Elementary education provides a framework upon which later classes build. This is part of the reason why younger students and students in cohort 1 tend to outperform older students and those in later cohorts. On average, OSP students in K-8 were more than six months ahead of their peers who had not used an opportunity scholarship. This indicates that vouchers programs have a pronounced effect on younger students that appears to tapper off as the student gets older. Until the voucher program is expanded, special emphasis should be placed on K-8 students in order to get the most from limited funds.
This segues to the third policy implication: the voucher program should be extended. There have been four applicants for every scholarship available (meaning there is a large demand) and applicants represent only a fraction of all students eligible for the program in SINI schools (which means there are still many students who would benefit from school choice). The astounding demand is testament enough to the efficacy of the OSP. If the school voucher program didn’t work, parents wouldn’t apply to enter it. Moreover, the school system should be able to make money off the program. It costs over $15,000 for the District to educate a student for one year, and under the OSP only half of that ($7,500) is diverted to private schools. Granted, there are sunk costs such as computers and facilities and administrators, but schools should be able to save money when a student moves from a private to a public school. This is not the case for two reasons, the first of which is the aforementioned wasteful spending, discussed above and the second of which is discussed below. Expanding the voucher program would meet a critical demand as well as allow D.C. schools to more effectively target its funding to a smaller group of students.
In a recent editorial on National Review Online, Greene talks about a “rising tide” phenomenon that occurred in other voucher programs. For instance, Greene notes a study done in Milwaukee where researchers discovered that, not only had the program significantly helped voucher recipients, but “the academic performance of students remaining in Milwaukee’s public schools [improved] by about 12 percentage points over the history of the program.” Greene argues that these results prove that the voucher program motivated the entire school system to do better. This same phenomenon could not be found in D.C., Greene argues, for one reason: the school system’s failings continued to be subsidized. Greene writes, “D.C. public schools have essentially been protected financially against the loss of students to vouchers.” As long as failure is subsidized in public schools, the school system will not have an incentive to move students from failing public to thriving private schools, and thus no reason to expand the OSP despite its successes. Funding remains constant regardless of how many students defect to private schools. Stopping this policy would be a win-win for schools and students. Though schools would lose money overall, their spending per pupil would increase, and the bloated teacher-to-student ratio would be reduced. Students remaining in public schools would benefit as well because their schools would be competing with private schools for students and teachers, as opposed to remaining static regardless of how many students leave the public school system. Such competition would eventually mean higher standards, better teachers, higher compensation, lower overhead, and safer environments in public schools. Carolyn Hoxby, the Scott & Donya Bommer Professor of Economics at Stanford University, writes, “If every school in the nation were to face a high level of competition […] the productivity of America’s schools, in terms of students’ level of learning at a given level of spending, would be 28 percent higher than it is now.” Such competition, Hoxby notes, had a statistically significant effect on math, science, and language scores for Milwaukee students who attend schools most affected by vouchers.
The Death of Vouchers?
The D.C. Opportunity Scholarship Program may have died a quiet death earlier this year. Tucked into the $410 billion 2009 Omnibus Appropriations Act, and between the $787 billion stimulus package and $3.55 trillion budget, was a provision that would slash the voucher program at the end of the 2009-10 school year. The Wall Street Journal writes that this is more than simple policy disagreement between the conservative paper and the liberal Congress. “It’s bad enough,” the editorial board writes, “that Democrats are killing a program that parents love and is closing the achievement gap between poor minorities and whites. But as scandalous is that the Education Department almost certainly knew the results of this evaluation for months. […] Since Education officials are intimately involved in this process, they had to know what was in this evaluation even as Democrats passed (and Mr. Obama signed) language that ends the program after next year.” This bill passed over the objections of parents in D.C., Mayor Fenty, and remarkable data proving the positive impacts of voucher-based education reform.
The Constitution gives Congress the general jurisdiction over the District of Columbia. This makes the D.C. the only school district in the nation where the federal government has explicit power, and provides a unique macrocosm for education reform on the national stage. What works in Washington, D.C. will likely be mimicked across the nation, and what is rejected in D.C. will have slim chance of being applied elsewhere. That is why it is so important to not end the D.C. voucher program before it can be given the necessary time to succeed. Every other study indicates that student performances begin to increase significantly after the third or fourth year. The three-year report indicates that the D.C. voucher program is beginning to follow a similar pattern, and has already and will continue to produce significant increase in student achievement and overall satisfaction and safety.
Last month, while addressing the Hispanic Chamber of Commerce, President Obama said the litmus test for allocating education reform dollars is “not whether an idea is liberal or conservative, but whether it works.” Similarly, when addressing the merits of mayoral control in New York City, Education Secretary Arne Duncan said, “I’m looking at the data here in front of me. […] Graduation rates are up. Test scores are up. Teacher salaries are up. Social promotion was eliminated. Dramatically increasing parental choice. That’s real progress.” Many of these benefits are not unlike those produced by the D.C. voucher program, a program Messrs. Obama and Duncan are effectively trying to end.
This study indicates that the Opportunity Scholarship Program, in conjunction with charter schools and mayoral controls, could significantly reform the broken D.C. school system. The benefits of giving the OSP more time are significant: higher achievement scores, more high school graduates, and in turn more students who go on to college or trade school, less crime, less adult illiteracy, and so on. The disadvantages are small, and are mostly partisan and political. Students in Washington, D.C., don’t have the luxury of waiting for a politically palpable policy. They’re desperate for what works.