At the University of Toronto Engineering, Professor Jason Hattrick-Simpers and his team have challenged a key assumption in AI: the need for large data sets for effective learning. Their research, published in Nature Communications, suggests that AI, including technologies like ChatGPT and DALL-E, might not require as much training data as previously thought.
Hattrick-Simpers is focused on developing new materials, such as catalysts for converting carbon into fuel and non-stick surfaces for airplanes. The challenge is the vast number of potential materials; for example, the Open Catalyst Project alone contains over 200 million data points on potential catalysts.
Traditionally, AI models in this field have relied on large datasets for training. However, this approach favors those with access to significant computing power, leaving others at a disadvantage. Furthermore, smaller datasets that are available tend to be narrow in scope, often focusing on existing materials, thus limiting the discovery of new, potentially more effective materials.
Dr. Kangming Li, a postdoctoral fellow in the team, compares this situation to understanding global student performance based on data from just one country. It's an incomplete approach that can miss out on broader insights.
Li's solution involves using subsets of large datasets that are easier to manage but still retain the diverse range of information from the original dataset. He tested this by training a computer model on both a full dataset and a subset 95% smaller. The results showed that the smaller dataset model performed comparably to the full dataset model when predicting properties of materials within the dataset's domain.
This finding suggests that large datasets might contain a significant amount of redundancy. It also aligns with a growing consensus in the AI community that quality of data can be more important than quantity. Smaller datasets, if well-curated, can be as effective as larger ones.
The study underscores the need for thoughtful dataset construction in AI, focusing on the richness of information rather than sheer volume. This approach could democratize AI research, making it more accessible to those with limited computing resources.
Read next: Decoding AI 'Hallucinations' - A Journey Beyond Misconceptions
Hattrick-Simpers is focused on developing new materials, such as catalysts for converting carbon into fuel and non-stick surfaces for airplanes. The challenge is the vast number of potential materials; for example, the Open Catalyst Project alone contains over 200 million data points on potential catalysts.
Traditionally, AI models in this field have relied on large datasets for training. However, this approach favors those with access to significant computing power, leaving others at a disadvantage. Furthermore, smaller datasets that are available tend to be narrow in scope, often focusing on existing materials, thus limiting the discovery of new, potentially more effective materials.
Dr. Kangming Li, a postdoctoral fellow in the team, compares this situation to understanding global student performance based on data from just one country. It's an incomplete approach that can miss out on broader insights.
Li's solution involves using subsets of large datasets that are easier to manage but still retain the diverse range of information from the original dataset. He tested this by training a computer model on both a full dataset and a subset 95% smaller. The results showed that the smaller dataset model performed comparably to the full dataset model when predicting properties of materials within the dataset's domain.
This finding suggests that large datasets might contain a significant amount of redundancy. It also aligns with a growing consensus in the AI community that quality of data can be more important than quantity. Smaller datasets, if well-curated, can be as effective as larger ones.
The study underscores the need for thoughtful dataset construction in AI, focusing on the richness of information rather than sheer volume. This approach could democratize AI research, making it more accessible to those with limited computing resources.
Read next: Decoding AI 'Hallucinations' - A Journey Beyond Misconceptions
