Skip to main content
One in two district leaders report that the curriculum their district pilots is also the one they adopt.1 This pattern highlights a common area of vulnerability in the curriculum purchasing journey: once pilots are complete, decision-makers often lean toward the option they’ve already invested in, even if pilot outcomes don’t fully support that choice.
The winnow stage of the adoption journey surfaces new complexities that can shape a district’s path toward adopting high-quality instructional materials (HQIM). Winnow is the phase when districts analyze the results of pilots, weigh the evidence collected against district needs, and make a final purchase decision. It goes without saying that some of the most difficult and cognitively demanding decision-making in the adoption journey occurs at this stage. The process is even more taxing for districts with limited time or expertise to analyze data from pilots and conduct objective assessments. Smaller districts tend to struggle with this the most: only 55% have a process to assess the efficacy of a new curriculum, compared to 65% of larger districts.1 As such, some districts report not using formalized data at all, instead relying “more so on perception data” to inform their ultimate adoption decision.1
When winnowing down options is more of a gut-driven, intuitive process than an objective, deliberative process, cognitive biases can influence the ultimate decision of what curriculum to purchase. Many times, these biases naturally favor curricula that feel intuitively “right,” even if data says otherwise. In this article, we break down some of the most common challenges facing districts during the winnow stage, coupled with deliberation tools and de-biasing strategies that can support strong data evaluation. With these interventions, districts can empower adoption committees to make confident curriculum decisions that have a real and measurable impact on student success.
Transforming Complex Pilot Data into Usable Insights
Before districts can assess what’s working and what isn’t, they must first translate raw pilot data into something that’s easy to interpret and compare. From teacher feedback and classroom observation notes to rubric scores and student learning data, districts often struggle to synthesize this mix of qualitative and quantitative evidence. In fact, research from the EdSignals Studio shows that balancing and weighing multiple data sources is a key barrier to evidence use during curriculum adoption.2
Information overload, a state of overwhelm in which information volume exceeds our cognitive capacity to process it, can push decision-makers toward easily available evidence, even if this evidence isn’t necessarily the highest quality. For example, memorable comments from teachers may be weighed more heavily than evidence that’s more complicated to decipher, such as data on a curriculum’s cultural competence.1
To combat these issues, districts need structured tools to weigh evidence and compare piloted materials. These processes are best planned before piloting starts. For instance, establishing standardized feedback forms and dashboards ahead of time can prevent data from getting messy and unmanageable during pilots.1 Just as importantly, districts should involve a diverse set of stakeholders to make sense of the resulting data. Teachers and instructional leaders who participated in the pilot can surface valuable insights that may get overlooked without their involvement. At the same time, external organizations can help districts structure qualitative data for easy interpretation while building capacity to develop a data culture from within.3 The ultimate goal with external experts is not to outsource data assessment entirely, but to create systems where decision-makers in the district feel empowered rather than intimidated by data.
Altogether, including diverse stakeholders in the interpretation process helps districts decide what to adopt, and also how to implement it. By looking back on challenges that arose during piloting, districts can use the data to determine whether issues reflect problems with the curriculum itself or if gaps could be addressed through additional training and support for teachers. Overall, these insights help districts move from pilot data to implementation-ready adoption.
Building Processes for More Objective Winnowing
Because final purchase decisions are high-stakes and emotionally-charged, these decisions are especially prone to bias. As a result, districts may find themselves making decisions based on intuitive judgments rather than real evidence of impact. For example, the sunk cost fallacy can make districts feel they must adopt the curriculum that they invested resources into piloting, just to justify the intensive process and avoid wasting these efforts.1
The sunk cost effect interacts with other cognitive biases that favor familiarity, making the piloted curriculum even more attractive to districts that lean toward intuitive judgment. The mere exposure effect, for instance, describes how we tend to develop a preference for things that we encounter repeatedly. In curriculum adoption, committee members can start to favor materials they’ve spent a lot of time evaluating and testing, just because those materials have become familiar.4
Finally, confirmation bias, which pops up throughout the adoption process, can further influence how committee members perceive evidence from pilots.1 This bias encourages decision-makers to favor evidence that supports an existing belief—say, the idea that the piloted curriculum is the best option for the district. As committee members review pilot results, they may unintentionally search for or interpret information in a way that confirms these preexisting ideas while overlooking evidence that the product actually doesn’t perform well overall.
To prevent bias from creeping into the winnow phase of adoption, districts can leverage several de-biasing strategies and structured reflection tools:
- Pilot Valuation Exercise: Prompts district leaders to consider both the past and future costs of different scenarios, including those associated with adopting, adjusting, or abandoning the piloted curriculum.5 This can help decision-makers consider what will deliver the most value moving forward instead of focusing on invested resources.
- S.W.O.T. Analysis: Encourages district leaders to consider the Strengths, Weaknesses, Opportunities, and Threats associated with a curriculum option to avoid unintentionally favoring evidence that confirms existing beliefs.5
- Pre-Mortem Analysis: Asks decision-makers to imagine that their chosen curriculum has failed, prompting them to work backward and examine potential causes—including cognitive biases or assumptions—that may have led to this failure.5
Reducing Consensus Pressures in Group-Level Decision-Making
Even when individual district leaders are given tools and strategies to reduce the influence of cognitive biases, group dynamics can still shape how evidence is evaluated. For example, teams under high stress and time pressure can be especially vulnerable to groupthink, a psychological phenomenon where group members feel the need to suppress their dissenting views to reduce conflict and reach a point of agreement. In the final stages of curriculum adoption, when decisions often require consensus, individual team members may self-censor their opinions, creating an illusion of consensus. While this may help groups make decisions quickly, it limits the influence of diverse voices and unique perspectives—such as those of teachers and other stakeholders—on the strengths and weaknesses of piloted material.1
During this stage in curriculum adoption, districts need ways to maintain diverse stakeholder participation and ensure final decisions are not made by an illusion of unanimity, but by genuine agreement. One approach is to use formal decision processes that allow for independent deliberation. Having individual committee members initially conduct anonymous analyses of pilot results can help them form early judgments based on evidence before coming together as a group.4 Similarly, staggering discussion windows so that small groups get together before full-group discussions occur can create more space for diverse perspectives and evidence-driven reasoning.
When groups eventually come together, appointing a devil’s advocate can help increase deliberation by making disagreement a formal part of the process rather than something uncomfortable to be avoided.4 Districts can even seek external support in this area; while districts strongly prefer to make their own curriculum decisions, they highly value the organizational and interpersonal capabilities of external providers, suggesting that many are willing to work with partners for facilitation and moderation of committee meetings.6
Moving Forward with Confident Curriculum Decisions
Well-executed curriculum pilots produce a rich collection of meaningful data that reflects how well a given curriculum fits the unique needs of a district. To make the most of this valuable evidence, districts need a strong winnow phase that transforms that data into objective, actionable insights. This means having processes in place to make sense of varied evidence and combat the natural pull of biases that favor things like ease, familiarity, and group cohesion. With the right structures and supports, districts can turn the complexity of winnowing into an opportunity that leads to curriculum choices grounded in real evidence of what will work best for their students.
Sources
- EdSignals Studio, Smarter Demand: Dimensions of Quality in Purchasing Decisions, 2022
- EdSignals Studio, Smarter Demand: Dimensions of Quality in Purchasing Decisions, 2023
- Chow, K., Nakamura, J., & Nunn, S. (2021, June) Learning Before Going to Scale: An Introduction to Conducting Pilot Studies. The Institute of Education Sciences. https://ies.ed.gov/use-work/resource-library/resource/other-resource/learning-going-scale-introduction-conducting-pilot-studies
- EdSignals Studio, Market Analysis: K-12 Teacher Prep Decision Maps, 2020
- EdSignals Studio, Pilot Cohort, 2024
- EdReports, The Decision Lab. (September 16, 2025). Beyond Selection: Rethinking How Districts Adopt Curriculum. https://www.edreports.org/resources/article/beyond-selection
