Microsoft Corp (NASDAQ:MSFT) has unveiled the latest in its range of smaller, less computationally intensive artificial intelligence models.
‘Phi-3 Mini’ comprises a 3.8-billion-parameter language model trained on 3.3 trillion tokens.
For reference, ChatGPT, based on OpenAI's GPT-3.5 architecture, comprises approximately 175 billion parameters, making Phi-3 substantially less powerful than the industry benchmark.
However, Microsoft engineers believe Phi-3 bats above its average when it comes to AI computing.
“The innovation lies entirely in our dataset for training, a scaled-up version of one used for Phi-2, composed of heavily filtered web data and synthetic data,” reads a technical report on the model.
Microsoft released Phi-3’s predecessor Phi-2 last December.
According to the report, Phi-3 is small enough to be deployed on a smartphone, “yet it achieves a quality that seems on-par with… GPT-3.5”, referring to OpenAI’s groundbreaking ChatGPT large-language model (LLM).
However, Phi-3 is “still fundamentally limited by its size for certain tasks… The model simply does not have the capacity to store too much ‘factual knowledge’, which can be seen for example with low performance on TriviaQA.”
Eric Boyd, corporate vice president of Microsoft Azure AI Platform, told The Verge that Phi-3 is as capable as LLMs like GPT-3.5 “just in a smaller form factor.”
It excels at handling specific tasks by using smaller data sets and less computing power.
Essentially, Microsoft’s Phi-3 model can be more efficiently used for more surgical AI tasks than the blunt force offered by ChatGPT and other LLMs; a master of one as opposed to a jack of all trades, so to speak.
It forms part of a growing industry of small-language models (SLMs) such as Google’s Gemma 2B and 7B.
These models are part of a broader trend in AI research towards creating more accessible and sustainable AIs that can be widely deployed across various sectors without the infrastructural demands of larger models.
Specific use cases could include customer support chatbots, real-time language translation and content moderation.
Microsoft is set to release its third-quarter results this Thursday.
Bank of America analysts are projecting 1% upside to the Street’s revenue estimate of $60.5 billion.