Shen Yang, a professor and doctoral supervisor at Tsinghua University’s School of Journalism and Communication, recently published a report called the “Comprehensive Performance Evaluation Report of Large Language Models.” The report ranks the performance of various language models based on 20 indicators across three dimensions.

The evaluation included seven large language models: GPT-4, ChatGPT 3.5, Wenxin Yiyan, Tongyi Qianwen, Xunfei Xinghuo, Claude, and Tiangong. These models were assessed for their generation quality, usage and performance, security, and compliance.

Wenxin Yiyan emerged as the top performer among the Chinese language models. It scored the highest in the overall comprehensive score, surpassing ChatGPT. Notably, Wenxin Yiyan excelled in Chinese Semantic Understanding and outperformed GPT-4 in certain Chinese abilities.

The report highlighted Wenxin Yiyan’s exceptional semantic comprehension, particularly its deep understanding of Chinese culture, strong timeliness, and content security. These strengths can be attributed to its enhanced knowledge, retrieval, and dialogue technological innovations.

In terms of generation quality, Wenxin Yiyan scored 76.98%, placing second only to GPT-4. It also ranked first in partial Chinese semantic understanding with a score rate of 92%.

Wenxin Yiyan also took the lead in security compliance, achieving a score of 78.18% in content security, bias and fairness, and privacy protection, surpassing GPT-4.

Please note that the external jump links provided in the article are for reference purposes only.

The post Comprehensive Performance Evaluation of Large Language Models appeared first on TS2 SPACE.