American researchers used video games to evaluate the effectiveness of artificial intelligence (AI) models, reports TechCrunch.
Specialists from the Halo Artificial Intelligence Laboratory at the University of California, San Diego, tested AI models using the legendary 1985 game Super Mario Bros. The developers created the GamingAgent framework, which helped the models reproduce and simulate the gaming experience.
Experts noted that the “Mario” game helped to assess how quickly each model can learn, make decisions, and develop a game strategy. As a result of the tests, Anthropic’s Claude 3.7 was the best, slightly outperforming Google’s Gemini 1.5 Pro and OpenAI’s GPT-4o.
However, the researchers concluded that even the most successful AI model played worse than any novice player. This is most likely because AI models need at least a second to make a decision, while “Mario” did not give them such an opportunity.
