MIT researchers have hit a note that could make machine learning more efficient by using a century-old mathematical concept, according to a recent news release. In a cross-disciplinary twist, PhD student Behrooz Tahmasebi and associate professor Stefanie Jegelka, from the Department of Electrical Engineering and Computer Science, have adapted Weyl's law to streamline data complexity in neural networks. A seemingly abstract relation, Weyl's formula, originally linked to acoustics, now finds its beat in the digital age, targeting a reduction in the volume of training data required for artificial intelligence.
The duo's innovative approach earned a "Spotlight" designation at the prestigious December 2023 Neural Information Processing Systems conference. "To the best of my knowledge," Tahmasebi stated in an interview, "this is the first time Weyl’s law has been used to determine how machine learning can be enhanced by symmetry." They shrugged off traditional applications and carved a path through the algorithmic landscape, exposing how symmetries—unchanging properties under certain transformations—can drastically lower the hurdles of machine learning.
It's a concept that begs simplicity in a complex world. An algorithm, for instance, doesn't need to analyze every pixel of an image with mirror symmetry but only half of it, doubling efficiency. Similarly, recognizing a color sequence regardless of its order slashes search requirements from thousands to a sole possibility. This leverages the inherent symmetry, reducing the grunt work for AIs while preserving their ability to make accurate predictions, particularly vital when data is scarce, explained Soledad Villar, an applied mathematician at Johns Hopkins University.
In their robust paper, Tahmasebi and Jegelka not only proposed but mathematically proved two theorems demonstrating the achievable gains and confirming them as the best possible outcome. The formula they presented promises versatility. It works across known symmetries and is primed to handle those yet undiscovered. Such foresight reflects the long-standing quest for new symmetries in physics and suggests their method will grow even more formidable with time.
The groundwork laid by the MIT researchers branches beyond the theoretical, supporting an emerging sector called 'Geometric Deep Learning'. Computer scientist Haggai Maron from Technion and NVIDIA remarked on the distinct divergence their work shows from previous studies, noting its application potential in graph learning and 3D data, "The paper helps establish a theoretical basis to guide further developments in this rapidly expanding research area," Maron told MIT News. With every novel approach that aligns theoretical math with practical computing, AI's trek towards refinement continues, seemingly unhindered by the daunting complexities of its own nature.
