The AI revolution continues to advance by leaps and bounds. Microsoft is developing a large language model (LLM) with about 500 billion parameters, as reported by The Information in a recent article. The LLM, internally known as MAI-1, is expected to make its debut this month.
When OpenAI introduced GPT-3 in mid-2020, it detailed that the initial version of the model had 175 billion parameters. The company revealed that GPT-4 is larger, but has not yet shared specific figures. Some reports suggest that OpenAI’s flagship includes 1.76 trillion parameters, while Google’s Gemini Ultra, which has comparable performance to GPT-4, allegedly has 1.6 trillion.
In this way, by having MAI-1 with 500 billion parameters, Microsoft’s new model could position itself as an intermediate option between GPT-3 and GPT-4. This configuration would allow the model to offer great response accuracy, but using much less energy than OpenAI’s flagship. This would result in lower inference costs for Microsoft.
According to The Information, the development of MAI-1 is being supervised by Mustafa Suleyman, founder of the company dedicated to the development of LLM Inflection AI, and who joined Microsoft in March, along with most of the employees of the new company. The executive previously co-founded DeepMind, Google’s AI research team.
With this in mind, it wouldn’t be too far-fetched to think that Microsoft is using training data and other assets from Inflection AI to power MAI-1. The model’s training dataset would also supposedly include various types of information, such as text generated by GPT-4 and web content. Furthermore, the Redmond-based company would be conducting the development process using a “large cluster of servers” equipped with Nvidia graphics cards.
The sources from The Information indicated that the company has not yet determined how it will use the MAI-1. If the model really has 500 billion parameters, it would be too complex to run on consumer devices, such as mobile phones. Therefore, it is very likely that Microsoft will implement the MAI-1 in its data centers, where the LLM could be integrated into services like Bing and Azure.
