Education has a new god: data. It is believed to have the power to save American education and thus everything in education must be about data—collect more data about our children, evaluate teachers and administrators based on data, and reward and punish schools using data.
The US federal government has been dangling $4.35 billion to recruit worshipers through the Race to the Top (RTT) program, and possibly a lot more through the upcoming reauthorization of the Elementary and Secondary Education Act (ESEA). Already 40 states have somehow made the conversion, although only two (Delaware and Tennessee) were deemed to have developed a system powerful enough to generate more converts in schools and thus rewarded with a total of $600 million. Other states will have a second chance and many are working to rewrite their pledges.
Both Delaware and Tennessee did an excellent job pledging to build an education system around data. The Delaware application for RTT lists three strengths that “few states can match:”
- Delaware’s state-of-the-art system (emphasis original) captures longitudinal information about both students and teachers, and links them together. Today, the State can quickly analyze the performance of any teacher’s students over time, can track how graduates perform in college, and can link teachers to teacher preparation programs, providing rich opportunities to use data to drive performance at the system, school, and classroom levels.
- Delaware’s rigorous statewide educator evaluation system is based on the most respected standards for teaching and leading. The system provides a multi-measure assessment of performance that incorporates student growth as one of five components…an educator can only be rated effective if they demonstrate satisfactory levels of student growth.
- Delaware’s newly-defined regulatory framework for school turnaround gives the State the authority to intervene directly in failing schools and requires schools to demonstrate results b achieving AYP within two years. (p. A4-A5)
All three are about data. With the RTT money, Delaware promises to become even more faithful—moving from data-rich to data-driven (watch the panel interview on Youtube). The State will “administer up to three formative and summative assessments per year per student in core subjects” (p. B-10), provide “immediate information that a teacher will use” (p. B-10), “provide data coaches to aid in the use of assessment data,” make tenure and licensure decisions for new teachers based on student performance (D-8), and include in teacher and administrator annual evaluations (p. D-16). With all these plans, Delaware will be “poised to promote data-driven instruction across all schools” (p. B-23). (http://www2.ed.gov/programs/racetothetop/phase1-applications/delaware.pdf)
Tennessee embraces data as enthusiastically as Delaware. Its RTT application begins with a poetic piece that asks the reviewers to:
Imagine, for a moment, a new day that is coming for Tennessee’s children and families, teachers and principals, and the State’s economic future….It is a day when middle school students will have the benefit of formative assessments, with immediate results so their teachers can measure their progress and make adjustments in time for the annual test. And then principals will use that student achievement data as a significant part of more rigorous teacher evaluations, and work with their teachers to help them succeed (p. 10).
I could go on, but you could read the applications and/or watch their presentations/panel Q&A on youTube (quite fun, actually).
I am not at all opposed to data but unfortunately data are not god, but rather man-made artifacts. Therefore they cannot be perfect. There are good data and there are bad data. A good example of bad data and bad decision driven by bad data is in fact the Race to the Top evaluation process.
William Peterson and Richard Rothstein published an analysis of the results of RTT Phase 1 review as a briefing paper of Economic Policy Institute this week. They conclude that “In short, the Race to the Top 500-point rating system presents a patina of scientific objectivity, but in truth masks a subjective and somewhat random process.” (p. 8)
They first raise some basic questions about this complex rating system invented by the Department of Education:
What was involved in the decision to use a scale of 500 total points rather than, say, 10, or 100, or 1,000? Is there scientific support for the “State Success Factors” being 90.6% as important as the “Great Teachers and Leaders” factor? Should the “Great Teachers” maximum points be 140, or maybe 163, instead of 138? And are there missing factors, such as “Developing Techniques to Promote Creativity,” or others?
They then conducted some experiments with the weights assigned to each category. By shifting the weights slightly (by increasing only 3% to four sub categories), they found:
If the weights were increased in this manner (and the weights of the other 25 indicators reduced by roughly half a point,5 so the total would remain 100%), Tennessee would no longer have won the competition—Georgia would have won instead. The official result can’t be justified by a claim that Tennessee is more reform-minded than Georgia. (p. 4)
They also found that Pennsylvania’s excellent proposal to focus on early childhood education, a priority of President Obama, was not recognized:
Pennsylvania, in short, has now been told by the Department of Education that if it wants to compete successfully in the next round of RTT competition, then the state should downplay its focus on early childhood and science education, and put its efforts instead into categories that get more points but which, in fact, have a much weaker research base. (p. 5)
Moreover, the rating system punished Massachusetts for its intention to take a bit more time to think whether it wants to lower its standards in order to be aligned with the Common Core standards.
In essence, the analysis shows that although “the process (or RTT reviewing) was presented as objective and scientific. However, further examination suggests that the selection of Delaware and Tennessee was subjective and arbitrary, more a matter of bias or chance than a result of these states’ superior compliance with reform policies.” (p. 1-2)
Read the complete paper: Let’s do the numbers: Department of Education’s “Race to the Top” Program Offers Only a Muddled Path to the Finish Line
If the RTT reviewing process—an elaborate apparatus developed by a data-driven Secretary of Education and Department of Education to evaluate efforts by 40 states that involved governors, state legislators, state school boards, superintendents(commissioners), school administrators, business partners, and teacher unions for their chances to receive many billions of dollars—can be so flawed, how much faith can we really have in the data that will be collected about students, used to determine their learning process, to evaluate their teachers and principals, and judge their schools?
Even if we could collect “good data” about students, there are still some questions we must ask:
- What is the cost, both financial and human, of all this data collection? Can the money for buying/developing/administering tests be used for truly educational activities? Can teachers and students better spend their time on actual teaching and learning rather than reporting and studying for tests?
- What are the implications of collecting so much data about individual student, from kindergarten to college or even post-graduate level? When we are so concerned about the misuse of health data, should we also consider about academic and school behavior data of our children?
- Will data-driven instruction truly work? This seems a very mechanical and arrogant—as if we knew exactly how children learn and develop and could prescribe their course of development as well as their purpose for life. But we know children are living organisms, with intentions, agency, emotions, interest, dreams, and hopes. They cannot and should not be deemed needing intervention just because they did do well to meet the prescribed “grade-level” expectations.
Learning in the Age of AI 
Leave a reply to Art Cancel reply