Positive solution of AI big model： general VS vertical？

Our reporter Qin Xiao reports from Beijing.

As an important project of global science and technology development, artificial intelligence is also an important strategic pilot opportunity for China. With the commercial application of technologies such as large models, all countries in the world are incubating and gestating all kinds of general industrial large models.

However, compared with the previous rush into the general big model, how to develop a vertical model for industry segmentation based on the big model has attracted more attention. At present, many vertical models can be seen to be applied in financial, medical and trading scenarios.

Many people in the industry said in an interview with the reporter of China Business News that there will not be many pedestal models in the future due to the challenges such as high computing power demand, high training and reasoning costs, and poor data quality. Compared with the general large model, the vertical large model, as a brand-new productivity, will inevitably achieve cost reduction and efficiency improvement of enterprises with the continuous breakthrough of its bottom-level capabilities, which will bring iteration and reform of upper-level applications.

Multi-modal large model competition

The Research Report on the Map of China Artificial Intelligence Model released at Zhongguancun Forum 2023 shows that at present, the artificial intelligence model in China is developing vigorously. According to incomplete statistics, up to now, 79 large-scale models with parameters above 1 billion have been released nationwide. "Producers" include Internet giants, AI concept listed companies, leading server enterprises, research institutes and primary market startups.

However, looking at the big models in China, most of them focus on the language model.

Wang Jinqiao, executive deputy director of "Zidong Taichu" Large Model Research Center of Institute of Automation, Chinese Academy of Sciences and president of Wuhan Institute of Artificial Intelligence, told reporters, "ChatGPT and domestic large models basically focus on language models. We know that 70% of the world’s data depends on vision, 10% on touch, and others rely on hearing and other information. To achieve low-power and more human-like intelligence, we must integrate multimodal information."

Not only that, high cost and high threshold also restrict the landing of large models in commercial scenes. The parameters of Baidu’s ERNIE Bot model are above 200 billion, JD.COM’s Yanxi model is above 100 billion, Tencent’s mixed model and Huawei’s Pangu model are above trillion, and ali tong’s Yiqianwen model is above 10 trillion.

Wang Jinqiao said: "In the future, there won’t be many large base models, because the cost of large base models is high and the technical threshold is high, and the training of large base models basically costs about 10 million yuan of electricity, so not all the work can be undertaken. Many large domestic models still focus on vertical industry solutions. The models are not large, but they have some core data of industries and production data of business systems, so they have certain advantages in some industries. I believe that in the future, the models of’ base model+vertical class’ will be integrated. "

Zeng Dajun, deputy director of the Institute of Automation, China Academy of Sciences, said: "The challenge of computing power and energy consumption of the existing large models will prompt a lot of work to develop in the direction of domain specialization and lightweight, especially in the fields of finance, education, medical care and transportation. A lot of work is trying to reduce the cost of large models."

Liu Chen, founder and CEO of Lingdi Technology Style3D, also believes that it is not necessary for every industry to make a big model. For some enterprises, the cost of training basic models is very high, which is unrealistic for many enterprises. In the future, big models are often dominated by giants, and enterprises can use them directly, but industry models must be trained. In the future, big models will become the infrastructure of the whole society, and industry models will become the infrastructure of the industry.

In fact, some companies have realized this problem. After the initial iterative upgrade, Baidu, Ali, Huawei and other companies explored specific industry applications from the initial large model development.

For example, Zidong Taichu 2.0, a full-modal large model put forward by the Institute of Automation of China Academy of Sciences based on Shengsi, is positioned as a pedestal model with world knowledge, and its application demonstration is carried out in the four fields of smart cars, smart medical care, smart manufacturing and digital government affairs through industrial consortia and expert data.

Build an AI ecosystem together

Whether it is based on natural language processing, machine vision, deep learning, reinforcement learning, or vertical industry model, it is not a "toy" for Internet manufacturers, nor is it a project that can be completed behind closed doors.

OpenAI, which entered the spotlight because of the research and development of ChatGPT, is also controversial because of closed source. As one of the founders of OpenAI, Musk recently bombarded on social media: "OpenAI was originally created as an open source non-profit company to compete with Google, but now it has become a closed-source for-profit company, controlled by Microsoft … This is not my intention at all."

Sam Altman, CEO of OpenAI, said that there will be more open source in the future, but there is no specific model and timetable.

"Although the current big model competition is fierce, neither OpenAI nor Google has a moat, because’ open source’ is rising in the field of AI big models." In a document leaked by Google, Google internal researchers believe that the open source model may lead the future development of the big model. This document mentions that "the open source model is faster in iteration, more customizable and more private, and when the free and unrestricted substitutes are of the same quality, people will not pay for the restricted model".

Huang Tiejun, president of Zhiyuan Research Institute, said: "It is difficult for large model industries to form a monopoly, and it is necessary to build a closed-loop industry. In the long run, the big model is just a label, not a product or a company’s tool, so it is inevitable that the whole big model will be open source and open ecology. "

In China, the development of most artificial intelligence models adheres to open source innovation and ecological guidance. At present, more than half of the large models published in China have achieved open source. The Research Report of China Artificial Intelligence Large Model Map shows that Beijing, Guangdong and Shanghai rank in the top three in terms of open source quantity and influence. Universities and scientific research institutions are the main sources of open source. chatglm-6b of Tsinghua University, moss of Fudan University and Wenxin series of large models of Baidu rank among the top three in terms of open source influence.

Ding Cheng, chairman of the open source community of MindSpore, Shengsi, told reporters: "From the perspective of the open source community, we very much hope that the big model ecology can be integrated. In the process of large-scale model training, data is a very core asset. Each large-scale model training can contribute relevant data for open source sharing, and it can gather the power of the whole industry to achieve a better big model for China or all mankind. For the core assets of the algorithm, at present, it is true that each family has its own core weapon. If we are more open and share this thing in an open source way, we can exchange technical needs and realize a better big model for the whole industry. "

In Wang Jinqiao’s view, the industry thinks that ChatGPT represents the iPhone moment of artificial intelligence, and it can be seen that the iPhone’s ecology is closed and Android is open source, but now the code, model and parameters of OpenAI are not open, and no patents are applied. This year, a lot of open source and open models have emerged rapidly. "There are so many models in China, which depends on the contribution of the collective wisdom of the global open source system. Open source and openness can achieve our mutual integration. Moreover, more and more models will be open source and open, and more models will join the community to adapt to the domestic hardware, so as to establish an open source and open system equivalent to the whole country, so that we can realize the development of mutual assistance and integration based on single mode or multi-mode of Chinese. "

(Editor: Zhang Jingchao Proofreading: Zhai Jun)