Building Better Learning Tools: Frameworks and Standards for High-Quality EdTech

Educational technology (EdTech) for children encompasses a range of digital education tools, including apps, online games, videos, platforms, and programs. These tools have the potential to enhance students' learning and improve teachers' instructional methods, but they must be well-designed and of high quality. Researchers are actively exploring how EdTech quality varies across different subjects (like math or reading), contexts (such as home learning versus classroom settings), and types of learners (including both typically developing children and those with special needs).

Yet, as this research evolves, teachers, parents, and policymakers need immediate guidance to navigate the vast landscape of EdTech. They need clear criteria to identify high-quality tools for their children and school districts now. How can that be achieved, and why is it so challenging to determine what works in children’s EdTech? These questions are crucial, particularly given the volume of new EdTech products offering artificial intelligence (AI) solutions.

The need for a joint framework

A reliable evaluation of the quality of the 500,000 apps and thousands of learning platforms available on the market starts with an evaluation framework. To be effective, this framework must address multiple aspects of quality, including safety, effectiveness, responsiveness to marginalized groups, or environmental concerns. Currently, there are numerous frameworks for evaluating EdTech, with 65 frameworks focusing solely on efficacy. Thus, instead of creating yet another framework, there is a need for a consolidated "framework of frameworks" that integrates various approaches to quality into a single, comprehensive system.

A good umbrella framework should streamline quality assessment for users while ensuring fairness for the industry. This means that the framework needs to be usable for evaluating what is in the market now, while allowing space for innovation (what could be in the market tomorrow). Multiple parallel efforts with various expert groups selected by national governments and international agencies are currently underway to derive such a framework. The groups are grappling with the question of whether some EdTech solutions (such as those with AI) are appropriate for children at all, as well as how to maximize the benefits of EdTech that are essential for maintaining learning access for some children.

Veering on the quality assurance side, the Council of Europe’s strategic actions in digital transformation in education is devising a review system for EdTech solutions. Focusing more on quality aspirations, the United Nations has six working groups composed of 43 members with diverse expertise who are developing a safeguards framework for effective digital public infrastructure (the DPI Safeguards initiative) . In addition, the QUESA framework by the Asian Development Bank focuses on aspects of quality design and implementation practices for EdTech.

While these frameworks can help coordinate thinking and organize action, they alone are insufficient to protect students' rights and provide developers with the necessary guidance to design high-quality EdTech. The next steps for a comprehensive framework are first, to establish regulatory standards, and second, to develop industry-wide quality benchmarks that model and promote desirable qualities.

From frameworks to regulations

Given that EdTech deals with what is often referred to as "the new oil"—data—there are many data protection issues that need to be addressed. Moreover, because EdTech is used for children’s learning, which is a vulnerable and sensitive context, it is essential not only to distinguish between high and low-quality products but also to identify those of unacceptable quality. Quality assurance must include recognizing certain quality aspects that are simply non-negotiable and certain practices that are not permissible in classrooms.

This is where regulations, and certifications derived from regulations, come into play. Following the enactment of Europe’s AI Act, the first comprehensive legal framework for AI, a certification of EdTech solutions that use AI should be available to make it easy to spot which technologies comply and which do not. Similarly to GDPR, which was accompanied by certifications issued by data controllers who can ensure individual companies’ compliance, such certifications need to be issued by authorized bodies, or data protection regulators to ensure the providers are applying the criteria consistently. A good example is the evidence standard issued by the What Worked Clearinghouse, a US government-funded “source of scientific evidence on education programs, products, practices, and policies,” which is provided along with a list of certified providers.

From frameworks to benchmarks

The proliferation of certifications on the EdTech market shows that EdTech companies desire certification and are willing to submit evidence and work towards meeting the criteria. That indicates a unique opportunity to drive the quality in the EdTech field: certifications and their standards can act as aspirational goals for the EdTech industry to improve the quality of their solutions.

This is where certifications provided by training bodies and various independent organizations come into play. Some certifications are based on the completion of research training programs by the provider, such as EDUCATE Edwards, while others focus on co-design approaches with teachers, like LeanLab Co-design certifications, and some on specific commitments made by an EdTech organization, such as the Prioritizing racial equity certification by Digital Promise, which reflects a commitment to equity.

To ensure that the pursuit of quality is genuinely aspirational, it should facilitate various types of competition under fair and equitable rules for all participants. Imagine an "EdTech Olympics," where instead of countries competing against each other, global EdTech providers vie for recognition. By applying this concept to the EdTech sector, we could create a shared repository of tools that allows for meaningful comparisons across different options, enabling parents and teachers to evaluate and choose the best solutions for their needs.

Creating such an EdTech index or depository should ideally be automated to avoid delays and errors with manual updates. Thus far, various platforms aggregate EdTech certifications by provider, in the USA it is EdSurge Index, and LearnPlatform by Instructure, and internationally, the EdTech Marketplace Asia, and the UNICEF Tech4Good platform (Learning Cabinet of recommended solutions, mainly for Global South policymakers). Thus far, these depositories are aggregators, rather than comparison charts, because there is not a consolidated benchmark for the different certifications that could be compared against each other according to a set standard. For a depository of aggregations to serve a comparison purpose, there must be an agreed-upon benchmarking system.

Indexing and labelling

So that EdTech tools can be reliably and objectively compared for their quality, the system needs to enable the comparison and weighing of different certifications against each other, thereby providing an overall impact assessment.

A total score can only be calculated if there is a fair differentiation across different levels of quality. In research, this is referred to as the weight of evidence. Weight of evidence depends on different factors based on the evaluation question, requiring consensus among several experts, akin to a peer-review or college of reviewers approach for research studies. Furthermore, each type of impact necessitates different levels of evaluation, with distinct impact indicators to ascertain significance. For example, the quality of an EdTech tool demonstrated through a small-scale Masters thesis is of lower weight than the quality of a tool demonstrated by a longitudinal three year study by several research teams working together.

In addition to weighing the evidence of impact in each category, the system needs to allow for the impact prioritization of different evaluators. For example, the UN, from a human rights perspective, emphasizes that unless safety and inclusion are considered in the design of digital tools, their full potential cannot be realized. Teachers, on the other hand, might argue that unless EdTech tools contribute to learning, there is no point in having them in classrooms. Therefore, a good system for evaluating EdTech needs to account for the varying importance of different types of impact, both within each impact category and between categories. The International Certification of Evidence of Impact in Education has been working to pilot such a system to establish equivalence among all internationally available certifications for EdTech by creating benchmarks of impact.

The different types of frameworks, mechanisms or tools for certifications, and appropriate bodies for providing certifications are included in the table below.

What would an ideal certification system look like?

In summary, to eliminate ineffective or harmful tools from the market and drive the EdTech industry toward higher standards, a robust certification system grounded in a well-established, government-endorsed standard is essential. This standard must offer a comprehensive view of quality, avoiding the pitfalls of narrow definitions of EdTech impact, such as those seen in the US framework that overly emphasized randomized controlled trials at the expense of inclusive research and design. In contrast, the UN Secretary General’s Roadmap for Digital Cooperation highlights the importance of a broad evaluation framework that addresses both digital transformation and environmental sustainability.

For the certification system to be effective, it must be integrated into the financial framework of EdTech businesses, not solely driven by moral concerns. It should be demanded by the market—schools, users, and parents—making quality standards a key criterion in procurement protocols. Additionally, a unified repository of certified tools, rated on clear and fair criteria, will enable parents and educators to make informed choices. This approach could be pivotal in solving the problem of subpar EdTech reaching students and ensuring that high-quality tools make a meaningful impact.