By Keith Devlin @KeithDevlin@fediscience.org, @profkeithdevlin.bsky.social

As a developer of an education technology product, my eye was naturally caught by two recent news stories on the topic.

On February 27, the government finally agreed to re-start the two programs that fund most of the newest educational technology products that spin-out from universities. Those programs are the Small Business Innovation Research (SBIR) program and the Small Business Technology Transfer (STTR) program.  Each federal department has such  programs. For education technologies, that used to be the Department of Education, now effectively shut down.

SBIR, popularly described as “America’s Seed Fund”, is a competitive funding program that encourages domestic small businesses to engage in federal research and development (R&D) with high potential for commercialization. It provides non-dilutive, high-risk funding to develop innovative technologies.

STTR funds cooperative research and development (R&D) between small businesses and non-profit research institutions (like universities or labs), and is designed to bridge the gap between academic research and commercialization.

It was a tragedy that those programs were allowed to lapse in the first place, but America was founded as a democracy and its voters sometimes make surprising choices. In the case of education technology, not only were those programs allowed to lapse, but closure of the Department of Education was a key platform element of the winning party in the last presidential election. (The late Bob Moses would not have been surprised.)

The lapse had a considerable cost. SBIR/STTR funds more than 4,000 companies per year, over 1,000 of which are new to government contracting. Economic impact studies of the programs show a $22-33 return for every dollar invested, depending on the agency. SBIR/STTR supported companies generated 65,578 jobs per year over a 23-year period. 

The compromise legislation introduced a number of changes to the program that small businesses will need to be aware of. For more on this, see the press release.

Meanwhile, the effective closure of the Department of Education has hampered technology support. I checked the program website, and it was last updated in 2025. (The Institute of R&D Education Sciences, whose name is on the program website banner, was the subunit of the Dept of Ed that operated the SBIR program.)

The second news story that caught my eye was from Education Week. Written by Alyson Klein, dated February 11, it was titled What’s Worse for Students: A Boring Worksheet or Ineffective Ed Tech? Her article was based on a panel discussion organized by the Public Interest Privacy Center at George Washington University law school in Washington, D.C. on February 10.

The article includes arguments raised that support each of the two possible answers to the question. It’s a short piece, so I won’t quote from it here. (I will mention that the article mentions attempts to limit schools’ use of educational technology by passing new laws, and that part too would not have surprised Bob Moses.)

Simmering just beneath the surface throughout Klein’s article, is the issue included in its title: most current ed tech is ineffective. Which, I suggest, should be read with the assumed modification “ineffective at producing good learning.” Right now, the reading that I think reflects the reality of the SBIR program is “effective at trying to create a profitable company.”

Those two should, in principle, not be in conflict. But for a small, university spin-off ed tech startup, they are critically (and frequently fatally) so. I talked with a fair number of SBIR funded startups when my own startup (BrainQuake) was in the SBIR program (1015-2018) for the first time, with Phase I and II awards (together totaling $1.1M), and they pretty well all said the same thing: get a product out as fast as you can, and promote it as hard as you can to drive sales and then hang on until a large ed tech company buys you out. Hardly a good way to develop first rate, effective, and reliable educational technology. Yet, if the goal is enhancing children’s education, those are the adjectives we should aim for—and achieve.

In practical terms, the necessity of getting good sales figures quickly morphs into “don’t spend your time doing efficacy research beyond what you did (by obligation) when in the SBIR program.” (You need some concrete evidence on your Phase I product, both efficacy and a reasonable sales figure, to get a Phase II award, and your Phase II report should address both requirements.)

Most SBIR funded startups worked with one of several nonprofit education research organizations available, to conduct the efficacy study on student learning. We chose WestEd, a particularly large player in the field. (A significant portion of the funds awarded to the company goes to the organization that runs the studies.)

Living in and around Silicon Valley, I and my founding colleagues had little difficulty raising a total of $300,000 in angel investments to build and market an initial first product: an app called Wuzzit Trouble. (That original iOS app is still available from Amazon.) That app got us our SBIR Phase I award. When we completed our Phase II (over two years), we were able to release a greatly expanded learning app called (simply) BrainQuake, that we had built with funding from our Phase II award. The initial injection of sales (sparked to a large extent by our product being featured on the App Store) eventually slowed down, since we did no marketing.

We really did not want to make learning claims about our product until we had completed a large scale, randomized study on students’ learning. Our in-house studies of our released product, and those by WestEd, all showed significant learning. But to truly determine learning outcomes for a product like ours, you’d need many hundreds of students, plus their teachers.

West Ed does carry out such studies, but our small startup could not afford that; we’d need a federal award of at least a few millions, over 2-3 years. Both the NSF and the IES used to make awards of several millions of dollars to conduct such projects. (I’m a reviewer on one study of that nature on an ed tech product that’s midway through, though for all I know it too may have stalled, due to the current chaos in Washington.)

So, we teamed up with WestEd once more and were working on a large-scale award application for several million dollars when the mysterious American Electorate made its choice, and the Department of Education was no more.

But the fact is, regardless of who is in charge in Washington, the SBIR model has a major flaw. The IES demanded small study efficacy results from Phase I to get Phase II funding, and we had submitted them. But to get further SBIR awards, you need to have strong sales figures. And that requires marketing. I couldn’t in all conscience do the rounds of district mathematics administrations to pitch a sale of the BrainQuake product until we had completed a large-scale efficacy study. Based on the evidence we had from small scale studies, I believed in its efficacy. But belief, even an expert’s belief, is not proof. At heart, I’m a scientist, not a businessman. The same is surely true for any university scholar who starts a company.

The standard way startups get off the ground is to pitch to a Venture Capitalist. However, I and several of my small BrainQuake team know some Silicon Valley VC's (one of the most spectacularly successful entrepreneurs, now a leading VC, was my former student at Stanford back in the late 1980s), and they all said we should avoid going that route. They appreciated that, for a quality education technology product in the math ed space, you needed solid scientific evidence of efficacy, that would take far too long for a small startup to complete on VC funding. Those guys want a fast turnaround, with significant returns in a small number of years at the most. With large scale education studies, you’re looking at several years minimum. That’s not what they fund. (It’s different for many other areas; pharmaceuticals jumps to mind.)

So, from the outset we had to proceed slowly but steadily, functioning essentially as a non-profit organization. We were helped by some independently-funded studies of our product carried out by universities in the US and overseas, but they too were on a small scale, typically one school class or (as in the case of the WestEd SBIR collaborations) a small group of schools. Absent a marketing budget, we could not put resources into marketing; and SBIR funds could not be used that way.

With education technology, a funding program, if it is to result in effective new products, there has to be funding to run the studies that establish the efficacy, as well as funding to help it get a foothold in the market (i.e., marketing). In our case, none of us working on BrainQuake takes a salary (except when on a federal award, where such is mandatory); we operate as if we were volunteers at a nonprofit. We’re working on the product out of a passion to use our skillsets for the common good. (We all have day jobs, or are retired.)

One of the most successful technology development programs was, of course, DARPA, founded in 1958 in response to the Soviet Union’s launch of Sputnik. Through the 1960s and 70s, DARPA supported projects that gave us a very large sector of today's personal computer landscape, including the PC itself, and the Internet. They would fund promising ideas (and sometimes less promising ones) for as long as it took for the team to get them to work.

Jumping to today, if an idea is worth an award of $1M to a small startup to build and test a prototype, it’s surely worth continuing to fund the product development until it is ready to launch, to work properly, and be of benefit to society.

With educational technology, we’re not talking about social media apps or cute gadgets, rather products that can change people’s lives for the better, and help develop the next generation of skilled citizens. That doesn’t come cheaply. Well-designed learning studies are expensive and take a lot of time, both to plan and to run.

The now-defunct Department of Education used to give out major awards to fund the large scale validation studies required to bring a successful new product to the attention of school districts, and in fact at the start of last year we were working with WestEd to submit a proposal to do just that with BrainQuake.

A second Phase I SBIR award in 2021 had provided evidence that the assessments our product delivered in real-time, at essentially no cost, correlated with success on large scale standardized tests. (It was, to use the metaphor I came up with to explain what we had stumbled on, like a blood panel test that is quick and simple to perform, yet gives you a surprisingly broad picture of someone's overall health and fitness. Getting a quick read of a student’s overall mathematical ability was an exciting possibility we had not had in mind when we built the platform; we were focused on supplementary learning. As a result of our discovery, we brought onboard one of the US’s leading assessment experts in order to pursue the assessment possibility. He was sufficiently convinced by what we had that he threw in his lot with us. But we have no way to fund the necessary, large scale validation study. And without it, we can’t advance. Indeed, with an income stream from app sales resulting from a few social media posts and word-of-mouth recommendations, it’s not even clear we can survive a long period of no funding. Pretty well all income went into further R&D.

So, BrainQuake is now just one of many products that have been created and validated in small, short-term studies, but may never find their way into enough classrooms or students’ homes to make a difference. Which means many of those $1M awards have been effectively wasted, other than to fund the limited satisfaction the developers get from seeing their idea in working form. But that satisfaction quickly turns to frustration, when you have good reason to believe it could make a real difference.