A Facebook AI Research (FAIR) team has developed a new, small design space called “RegNet” that surpasses traditional models available like Google’s and works five times faster on GPUs.
RegNet produces simple, fast and versatile networks and, in experiments, it has outperformed Google’s SOTA EfficientNet models, the researchers said in an article titled “Designing Network Design Spaces; published on the ArXiv pre-printing repository.
The researchers sought to “interpretability and discover general principles of design that describe networks that are simple, work well and are generalized in all contexts”.
The Facebook AI team performed controlled comparisons with EfficientNet without any improvement in training time and with the same training configuration.
Launched in 2019, Google’s EfficientNet uses a combination of NAS and model scaling rules and represents the current SOTA.
With comparable training parameters and flops, RegNet models outperformed EfficientNet models while being up to 5 times faster on GPUs.
Rather than designing and developing individual networks, the team focused on designing real network design spaces with huge and perhaps endless populations of model architectures.
The quality of the design space is analyzed using the empirical error distribution function (EDF).
Analysis of the RegNet design space also provided researchers with other unexpected information about network design.
They noticed, for example, that the depth of the best models is stable across the calculation regimes with an optimal depth of 20 blocks (60 layers).
“Although it is common to see modern mobile networks using reverse bottlenecks, researchers have noticed that the use of reverse bottlenecks degrades performance. The best models use no bottleneck or reverse bottleneck, the newspaper said.
Facebook’s AI research team recently developed a tool that deceives the facial recognition system to mistakenly identify a person in a video.
The “de-identification” system, which also works in live video, uses machine learning to change the main facial features of a subject in a video.
FAIR advances the state of the art in artificial intelligence through basic and applied research in open collaboration with the community.
The social media giant created the Facebook group AI Research (FAIR) in 2014 to advance the state of the art of AI through open research for the benefit of all.
Since then, FAIR has become an international research organization with laboratories in Menlo Park, New York, Paris, Montreal, Tel Aviv, Seattle, Pittsburgh and London.
