A group of AI researchers at Microsoft published a research paper in arXiv about their testing on Orca-Math and how it is better than most LLMs in solving math equations. Many Large Language Models like ChatGPT are best in conversational skills but they also have the ability to solve math questions accurately. These models have been tested for their abilities by solving GSM8K, a data consisting of 8500 math questions of grade school levels.
Microsoft also developed an AI model named as Orca-Math whose speciality is solving math questions and equations. In their study, the researchers tested this AI model and found out that there is a great difference in Orca-Math and LLMs. As Orca Math is a Small Language Model, it couldn’t get to the level of LLMs in terms of conversational skills but as it was specifically designed to solve math questions, it was better than LLMs in terms of that.
Even though Orca-Math is termed as a Small Language Model, it is still larger than other SMLs. It has 7 billion total parameters, so it is still smaller than LLMs. As a result of testing with GSM8K, Orca Math scored 86.81% as compared to ChatGPT-4 that scored 97.0%. Llama-2 didn’t even fall close behind and only scored 14.6%. Microsoft said that despite being a SLM, Orca-Math managed to score this high because of its data training that is generally used for LLMs. SMLs can also perform well as LLMs if they have been trained well with specialized conditions.
Read next: The Case for and Against AI’s Role in Cybersecurity: Is AI a Hero or a Villain?
