Special thank you to Chris Gibson, Nathan Frey, Peyton Greenside, Ron Alfa, and several others for insightful conversations throughout the writing of this article.
One of the largest trends in AI this year has been the incredible spend by frontier labs on proprietary training data generated by external partners. While the labs’ spending metrics are rarely publicly disclosed, they can be estimated using numbers from the data labeling firms themselves. Back in June 2025, the largest players, Surge AI and Scale AI, were averaging ~$1B in revenue each, driven mainly by spend from the frontier labs.
Revenues for top data labeling companies. (Data from company disclosures; Source)
Extrapolating to the larger data market, it’s been estimated that top labs are shelling out $1-10B annually on proprietary dataset generation from several data providers. These numbers seem to only be increasing as bespoke datasets and reinforcement learning (RL) environments become cemented as core accelerants for improving model performance across disciplines.
If you simplify the axes of how large language models (LLMs) improve into (1) access to relevant training data, (2) advancements in model architecture, and (3) increased compute, this trend toward purchasing external datasets makes sense for a few reasons.
This data demand has manifested most widely across hot application domains like coding, finance, and computer use. The logic is rather simple, and the playbook seems to be working. Including more curated examples of Python code in your training data will likely make the LLM a better software engineer.
I am increasingly seeing these same trends discussed for the biological datasets used to train life science foundation models, with model developers making more licensing deals for access to task-relevant data. Including more high-quality antibody affinity measurements in your training data will likely make the generative protein model a better protein engineer.
There has also been a growing focus on improving the speed and rigor with which we collect these biological datasets to feed this demand. Autonomous life science robotics companies are now pitching themselves as data foundries to take advantage of this growing market in an effort to close the flywheel between data collection, model training, and evaluation.
These growing opportunities to collect, sell, buy, and train on biological data raise some important questions: what’s the right strategy for curating and training on datasets for biological foundation models, and what principles do and do not transfer from how we approached training LLMs with text data?
Despite the macro-level market parallels between frontier LLMs and biological foundation models, there are meaningful and nuanced differences that are worth discussing in depth.
From a data perspective, the historical playbook for LLM training — a predominant focus on data scale — will not transfer one-to-one to biological datasets. Instead, I argue that curating biological training datasets demands far more care in prioritizing data quality over quantity, and we should recognize this before we fall into the trap of scalemaxxing.
In the early days of modern LLM pretraining, the predominant consensus was to improve model performance through scale. More data, more parameters, more compute. This manifested with the field spending much of its time enumerating pretraining scaling laws — the slope of how model performance improved as a function of data, model size, compute, etc. We wrote about this topic in depth in a previous Dimension Research article.
In 2020, Kaplan et al. at OpenAI published some of the initial pretraining scaling laws, showing that increasing model size (parameters) faster than data (tokens) resulted in optimal performance for a fixed compute budget. In 2022, Hoffmann et al. at DeepMind published the “Chinchilla” scaling laws, which showed that Kaplan-based models were undertrained and that better model performance was achieved when training smaller models with more data (~20 tokens per parameter). Almost two years later, Sardana et al. at Databricks published a modification to Chinchilla scaling laws to take inference demand into account. With a fixed compute budget, it’s more economical to train smaller models with increasingly more data (~200-10,000 tokens per parameter) to achieve a desired performance because serving smaller models is ultimately cheaper for customers. (Source)
