GPT+通信白皮書-英文

ExecutiveSummary

Drivenbythejointeffortofkeytechnologiessuchasbigdataandcloud

computing,asizablenumberofthegenerativepre-trainedtransformer(GPT)large

models,representedbyChatGPT,haveemerged,showinghighlycreativecontent

generationcapabilitiesandprovidinghighlyintelligenthuman-computerinteraction

experience.Foralongtime,therehavebeenmanytechnicalproblemsin

communicationthataredifficulttomodelaccuratelyorsolveefficientlyusing

traditionalmethods.Meanwhile,GPTdemonstratesthepotentialtoimprovethe

performanceofinformationcommunicationservicesandintelligentautonomous

networks.Inaddition,therapiddevelopmentandbroadapplicationsofGPTalsoneed

tobesupportedbyacommunicationnetworkwithlargebandwidth,lowlatency,and

highreliability.

Therefore,fromtheperspectiveofcommunicationpractitioners,thiswhitepaper

explorestheinterrelationshipbetweenGPTandcommunication.Firstly,Chapter1

sketchestheconcept,developmentprocess,andresearchstatusofGPTlargemodels.

Secondly,Chapter2discussesthenewapplicationsofGPTinthecommunication

industry,andthepositionofGPTinnetworkintelligentautonomy.Thirdly,Chapter3

exploreshowthecommunicationnetworksenablethebroadapplicationsofGPT,and

givesatypicalideaoffuturenetworkdesign.Moreover,Chapter4analyzesthe

processofGPTandcommunicationfromindependentevolutiontocollaborative

development,aswellasapplicationsof“6G+GPT”empoweringthedigital

transformationofindustries.Inaddition,Chapter5pointsoutthefivemostobvious

problemsandchallengesintheintegrationprocessof“GPT+Communication”and

providessomesolutions.Subsequently,Chapter6putsforwardseveralsuggestionson

howGPTandthecommunicationindustrycandeveloptogether,aswellasthe

prospectsforthefuture.Finally,Chapter7concludesthiswhitepaper.

1/92

0Preface

Inrecentyears,asArtificialIntelligence(AI)technologyhascontinuedto

advance,particularlyintheareasofreinforcementlearning,largemodels,and

generativecontent,variousindustrieshavebeenactivelyexploringitsapplications.At

theendofNovember2022,OpenAIreleasedtherapidlypopularizedchatbot

ChatGPT,whichpossessesastonishingnaturallanguageunderstandingandgeneration

capabilities,attractingwidespreadattentionfromsociety.Subsequently,inMarch

2023,thelaunchoftheupgradedversionGPT-4multimodallargemodelreignited

enthusiasmforgenerativeAI,leadingtotheemergenceofnumerouslargemodelsin

quicksuccession.

Sincetheinceptionoftext-basedconversationalinteractions,GPThas

profoundlyimpactedpeople’sproductionandliveswithinafewshortyears,bringing

aboutsignificantchanges.Manypeoplebelievethatitwillcontinuetobring

disruptivechanges.BillGatespointedoutthatlargemodelsrepresentthemost

revolutionarytechnologicaladvancementinover40years;NVIDIACEOJensen

Huanglikenedtheemergenceoflargemodelstothe“iPhonemoment”ofAI;Baidu

CEORobinLiproposedthatlargemodelsarepreparedtochangetheworldatthe

2023ZhongguancunForum.FromtheripplescausedbyChatGPTtotheglobalwave

itunleashed,GPTlargemodelshavebecomeoneofthemostdiscussedtopicstoday,

signalingacrucialturningpointinthedevelopmentofgenerativeAI;theyear2023

willalsoundoubtedlyleaveasignificantmarkinthehistoryofAIdevelopment.

Asanindustryfacilitatinginformationexchangeandtransmissionamong

humans,nature,andmachines,thecommunicationindustryiscloselyintertwined

withthedevelopmentoflargemodeltechnology.Thecommunicationindustryitself

hasahighdegreeofdigitalizationandneedstohandlecomplexdata.Theintroduction

ofGPTcanstreamlineasignificantamountofwork,bringingaboutsignificant

capacityenhancementsforcommunicationoperators,particularlyintherealmsof

networkoperationsandmaintenance(O&M)andservicedelivery,makingthemmore

intelligent.Intheeraoflargemodels,withtheadvancementofGPTtechnology,the

demandforcomputingpower,data,andalgorithmswillexperienceexplosivegrowth,

requiringcommunicationinfrastructuretoprovidesupport.Inthefuture,howGPT

empowersthecommunicationindustryandhowthecommunicationindustrysupports

4/92

GPTarequestionsthateverycommunicationprofessionalshouldearnestly

contemplate.

Therefore,thiswhitepaperisbasedonthedevelopmenthistoryandlatest

researchadvancementsofGPTlargemodels.Ontheonehand,itelaboratesonthe

innovativeapplicationsofGPTwithinthecommunicationindustryinspecific

scenarios.Ontheotherhand,itinvestigateshowfuturecommunicationnetworks

providenativesupportforGPTintermsofarchitectureandkeytechnologies.

Subsequently,combiningGPTwithcommunication,itproposesaroadmapforthe

digitalandintelligenttransformationofkeyindustriesthroughtheircollaborative

development,whilealsopointingouttheproblemsandchallengesintheintegration

anddevelopmentprocess.Inresponsetotheseissues,correspondingdevelopment

recommendationsandprospectsareprovided.Finally,thewholecontentofthiswhite

paperissummarized.Thecompletechapterstructureofthiswhitepaperisillustrated

inFigure0-1below.

5/92

Figure0-1WhitePaperChapterStructureDiagram

ThiswhitepaperwasjointlyorganizedandauthoredbytheBeijingInstituteof

Technology,withparticipationfrom18entities,includingthethreemajortelecom

operators(ChinaMobile,ChinaUnicom,andChinaTelecom),seventop-tier

universities,threerenownedenterprises,andfiveleadingresearchinstitutesinthe

industry.Spanningovereightmonths,theprocessinvolvedthein-depthparticipation

ofover50expertsandscholars,fromconductingresearchandtrackingthecutting-

edgestatusofGPTlargemodelstoexploringtherelationshipbetweenGPTand

communication,conceptualizingtheoutlineofthewhitepaper,arrangingspecific

chaptercontent,andassigningwritingtasks.Itunderwentmorethantwentyroundsof

discussionsandrevisionsbeforereachingitscompletion.Duringthisperiod,some

participatingentitiesalsosuccessfullycollaboratedtoapplyforaninternational

6/92

cooperationprojectfromtheMinistryofScienceandTechnologyofthePeople’s

RepublicofChina,titled“ResearchonKeyTechnologiesofIntegrated

MultidimensionalIntelligentOrchestrationinCloudComputingNetworksBasedon

LargeModels,”therebybettersupportingthecompletionofthiswhitepaper.

WebelievethatAItechnologyisstillinarapidlydevelopingstage,andthe

integrationandmutualsupportbetweenGPTlargemodelsandcommunication

networkscancontinuallyexpandinnovativeapplicationscenariosandimprove

ecosystemdevelopment,thusjointlypromotingtechnologicalprogressandthe

developmentofvariousindustries.

7/92

1.GPTLeadstheTideofArtificialIntelligenceDevelopment

WiththeadvancementofAIanddeeplearningtechnologies,theconceptof

“l(fā)argemodels”hascomeintofocus,withChatGPTbeingthemostnotable.On

November30,2022,OpenAIofficiallyreleasedtheAIchatbotChatGPT,which

representsArtificialIntelligenceGeneratedContent(AIGC)inthefieldofnatural

language.Itspowerfulcapabilitieshavechangedthewaymanypeopleworkandlive,

sparkinganewwaveofAIgloballyandattractingwideattentionfrombothindustry

andacademia.OnMarch14,2023,theofficiallyreleasedGPT-4underwentfurther

upgrades,significantlyrelaxingtextinputrestrictions,improvingansweraccuracy,

andevenenablingdirectinputofimagestogeneratelyrics,creativetexts,etc.,with

stylevariations,onceagainshowcasingtheimpactofgenerativeAI.OnNovember7,

2023,atthefirst-everOpenAIDevDay,OpenAICEOAltmanshowcasedGPT-4

Turbototheworld.AsthelatestversionofGPT,ithasbeenupdatedinareassuchas

dataquality,imageprocessing,andspeechconversion,bringingdevelopersandusers

morepossibilitiesandopportunities.

So,whatareChatGPTandGPT?Whatdevelopmentjourneyhavethey

undergone?Andhowshouldtheybeunderstoodandapplied?Thischapterwillstart

withanexplorationofGPTlargemodels,introducingtheirbasicconcepts,

developmenthistory,andcurrentresearchstatustoprovidereaderswitha

comprehensiveandin-depthunderstandingofGPT.

1.1.BasicConceptsofGPT

1.1.1GenerativePre-trainedTransformer

GPTstandsforGenerativePre-trainedTransformer,originatingfromthefields

ofdeeplearningandnaturallanguageprocessing(NLP).Overthepastfewyears,

withtheadvancementofcomputingpowerandtheemergenceofbigdata,significant

breakthroughshavebeenmadeinthefieldofNLP.GPT,asanintegrationofaseries

ofNLPtechnologies,emergedinsuchacontext,asshowninFigure1-1.

G:Generative.ThisindicatesthatGPThastheabilitytospontaneouslygenerate

content.

P:Pre-trained.ThisindicatesthatGPThasundergonepre-trainingandisready

forimmediateuse.

8/92

T:Transformer.ThisindicatesthatGPTisbasedontheTransformerarchitecture

forlanguagemodeling.

Figure1-1MeaningofGPT

In2017,theGoogleteamfirstproposedtheTransformermodelbasedonthe

Self-AttentionMechanism(SAM)andappliedittoNLP[1].OpenAIappliedthis

technologyandreleasedtheearliestgenerationoflargemodels,GPT-1,in2018.Since

then,theparametersizeofeachgenerationofGPTmodelshasgrownexplosively.

TheparametersizeofGPT-2,releasedinFebruary2019,was1.5billion,whileGPT-3,

releasedinMay2020,directlyreached175billion.

ThemeteoricriseofChatGPTwasnotbychance.Itistheresultoftheeffortsof

manypeopleandalongperiodofevolution.TounderstandthedevelopmentofGPT,

oneshouldfirstgrasptheconceptoflargemodelsandTransformerarchitecture.

1.1.2LargeModel

Generally,beforeChatGPT,theAImodelsthatreceivedpublicattentionwere

mainlyusedforsingletasks.Forexample,“AlphaGo”,whichignitedtheentireAI

marketandprompteditsexplosivedevelopment,defeatedGoworldchampionLee

Sedolinthe“Manvs.Machine”matchin2016,basedonglobalGogamerecords.

However,fundamentally,theseAIdatamodels,whichfocusonspecifictasks,can

onlybecalled“smallmodels”comparedtoChatGPT.

Largemodelsrefertomachinelearningmodelswithhugeparameterscalesand

complexity.ThetermusuallyreferstoLargeLanguageModels(LLMs).Alanguage

modelisanAImodelthat,aftertraining,canunderstandandgeneratehuman

language,and“l(fā)arge”meansthatthemodel’sparametersareverylargerelativeto

“smallmodels.”

AsshowninFigure1-2,thisevolutionarytreetracesthedevelopmenthistoryof

9/92

largemodelsinrecentyears,highlightingsomeofthemostwell-knownmodels,with

modelsonthesamebranchbeingmorecloselyrelated[2].Solidsquaresrepresent

open-sourcemodels,whilehollowsquaresrepresentclosed-sourcemodels.Non-

Transformermodelsareshowningray,andamongTransformer-basedmodels,

Encodermodelsareinthepinkbranch,Decodermodelsareinthebluebranch,and

Encoder-Decodermodelsareinthegreenbranch.

Figure1-2EvolutionaryTreeofLargeModels

Basedonthisevolutionarytreediagram,wecanconcludethatDecoder-only

modelsaregraduallybecomingthedominantmodelsinLLMdevelopment,and

OpenAIcontinuestomaintainitsleadingpositioninLLM.Metahasmade

outstandingcontributionstoopen-sourceandLLMresearch,butthereisatrend

towardsclosed-sourcedevelopmentafterthelaunchofGPT-3.Inaddition,many

companiesandinstitutionsarestillactivelyexploringEncoder-Decodermodels,such

asGoogle.

Currently,majorinstitutionsabroadthatreleaselargemodelsincludeOpenAI,

Anthropic,Google,andMeta,withmodelparameterscalesmainlyinthetensand

hundredsofbillions.Uptonow,thetopGPTlargemodelsabroadincludeChatGPT,

Claude,Bard,andLlama.Amongthem,afterGooglereleasedthelatestnative

10/92

multimodallargemodel–Gemini,BardwasofficiallyrenamedGemini.

Inthisgloballycompetitivearena,Chinaisalsokeepingpace,developingmany

largemodels,includingTencent’s“Hybrid,”Alibaba’s“TongyiQianwen,”Huawei’s

“Pangu,”andChinaMobile’s“Jiutian”series.DatashowsthatasofOctober2023,

thereareatotalof254domesticcompanies,universities,andresearchinstituteswith

largemodelsofover1billionparameters,indicatingthatthe“battleofthehundred

models”istransitioningfromthepreviousstageof“beingborn”toanewstageof

“beingused.”Figure1-3showssomeofthelargemodelsdevelopedbydomesticand

foreigncompaniescurrently.

Figure1-3VariousTypesofLargeModels

1.1.3TransformerArchitecture

TheTransformerarchitectureisacrucialfoundationofGPT,whichisaneural

networkarchitecturebasedontheSAMandwidelyusedinlargemodelsinthefield

ofNLP.ItscorecomponentsaretheEncoderandDecoder.TheEncoderencodes

inputtextintoaseriesofvectors,whiletheDecoderdecodesthesevectorsonebyone

intooutputtext.BeforetheintroductionofTransformer,themainstreammodelsinthe

NLPfieldwereRecurrentNeuralNetworks(RNNs),whichusedrecursionand

convolutionalneuralnetworksforlanguagesequencetransformation.

InJune2017,theGoogleBrainteampublishedapapertitledAttentionisAllYou

NeedatthetopAIconferenceNeurIPS,proposinganewnetworkarchitecturecalled

Transformer.ItisentirelybasedontheSAM,abandoningrecursionandconvolution.

Afteronly12hoursoftrainingoneightP100GraphicsProcessingUnits(GPUs),

Transformerachievedhighertranslationquality[1],showcasingexcellentparallelism

11/92

andbecomingthemostadvancedLLMatthetime.

Figure1-4illustratesthenetworkstructureoftheTransformer.Itconsistsofa

seriesofEncodersandDecoders,eachcomprisingmulti-headattentionlayersandall-

inclusiveconnectedfeedforwardnetworks.GPT,similartotheDecoderpartof

Transformer,isanautoregressivemodel.

Figure1-4TransformerNetworkStructureDiagram

ThecorecomponentintheTransformeristhemulti-headattentionmechanism

module,asshowninFigure1-5.Itrequiresthreespecifiedinputs:Q(Query),K(Key),

andV(Value).Then,itcalculatesthesimilaritybetweeneachpairofQandKand

weightseachVbasedonthesimilaritytoobtaintheattentioncalculationresult.

12/92

Figure1-5Multi-HeadAttentionMechanismModule

Themulti-headattentionmechanismdoesnotcalculateattentiononlyoncebut

dividestheinputintosmallerblocksandthencalculatesthescaleddot-product

attentioninparalleloneachsubspace.Thisdesignallowseachattentionmechanism

tooptimizedifferentfeaturepartsofeachword,balancingthebiasesthatmayarise

fromthesameattentionmechanismandenablingthemodeltocapturesemantic

informationatdifferentlevels,therebyenhancingthemodel’sexpressivepowerand

improvingitseffectiveness.

1.2.DevelopmentHistoryofGPT

13/92

Figure1-6DevelopmentHistoryofGPT

ThedevelopmenthistoryofGPTcanbedividedintotwostages.Before

ChatGPT,theemphasiswasoncontinuouslyincreasingthebasicscaleoflarge

modelsandenhancingnewcapabilities.ChatGPTandGPT-4,ontheotherhand,

focusmoreonreinforcementlearningfromhumanfeedbacktounderstandhuman

intentandprovidebetterservices,asshowninFigure1-6.

①June2018:OpenAIpublishedthepaperImprovingLanguageUnderstanding

byGenerativePre-trainingandofficiallyreleasedGPT-1[3].

Basicapproach:Generativepre-training(unsupervised)+downstreamtask

fine-tuning(supervised).

BasedonaunidirectionalTransformerlanguagemodelwithadecoder

structure,consistingof12layers.

117millionparameters,5GBtrainingdata,relativelylimitedmodelsizeand

capabilities.

Contextwindow:512tokens.

②February2019:OpenAIpublishedthepaperLanguageModelsare

UnsupervisedMultitaskLearners,proposingthatlanguagemodelsareunsupervised

multitasklearners,andGPT-2wasborn[4].

Basicapproach:Removingsupervision,retainingonlyunsupervisedlearning.

48-layerTransformerstructure.

14/92

1.5billionparameters,andthetrainingdatavolumeincreasedto40GB.

Contextwindow:1024tokens.

③May2020:OpenAIpublishedthepaperLanguageModelsareFew-Shot

LearnersandintroducedtheGPT-3model[5].

Basicapproach:Unsupervisedlearning+in-contextlearning.

96-layermulti-headTransformer.

Thenumberofparametersincreasedto175billion,trainedon45TBoftext

data.

Contextwindow:2048tokens.

④March2022:OpenAIonceagainpublishedthepaperTrainingLanguage

ModelstoFollowInstructionswithHumanFeedback,introducingReinforcement

LearningfromHumanFeedback(RLHF),andlaunchedtheInstructGPTmodel[6].

Basicapproach:RLHF+fine-tuningtraining.

Enhancedhumanadjustmentofmodeloutput.

Resultsrankedinamoreunderstandablemanner.

ChatGPTisaderivativeofInstructGPT,andthetwohavethesamemodel

structureandtrainingmethod.Theonlydifferenceisthewaytheycollectdata.

ChatGPTfocusesmoreoninteractionintheformofdialogue.

⑤March2023:OpenAIreleasedthemultimodalpre-trainedlargemodelGPT-4,

onceagainundergoingsignificantupgrades.

Basicapproach:Multimodal.

Contextwindow:8195tokens.

1.8trillionparameters,13trilliontokentrainingdata.

Powerfulimagerecognitioncapabilities.

AlthoughthecurrentcapabilitiesofGPT-4inreal-worldscenariosmaynot

matchthoseofhumans,ithasdemonstratedsignificantlysuperiorabilitiesinvarious

professionalandacademicexams.EvenSATscores(whichcanbeunderstoodas

scoresfortheU.S.collegeadmissionstest)ofGPT-4havesurpassedthoseof90%of

testtakers,reachingthelevelrequiredforadmissiontotopuniversitiessuchas

HarvardandStanford.

1.3.CurrentResearchStatusofGPT

OnOctober12,2023,theanalysiscompanystateof.aireleasedtheStateofAI

Report2023.ThereportpointedoutthatOpenAI’sGPT-4remainsthemostpowerful

15/92

LLMglobally.GenerativeAIhaspropelledadvancementsinlifesciencesandhas

beenasaviorfortheventurecapitalindustry[7].Largemodelscontinuetoachieve

technologicalbreakthroughs,especiallyinthefieldoflifesciences,making

significantprogressinmolecularbiologyanddrugdiscovery.

OnDecember14,2023,Natureannouncedtenpeoplein2023.Notably,the

chatbotChatGPT,duetoitsdominanceofvariousnewsheadlinesin2023and

profoundimpactonthescientificcommunityandsocietyatlarge,wasincludedasthe

11th“non-humanmember”onthelist,recognizingthesignificantchangesbrought

aboutbygenerativeAItoscientificdevelopmentandprogress.Currently,both

domesticallyandabroad,researchonGPTlargemodelscontinuestodeepen,with

manyinstitutionsstartingtodeveloptheirownlargemodels,andtheapplication

scenariosarebecomingincreasinglydiverse.LargemodelsrepresentedbyChatGPT

haveofficiallyusheredintheeraofAI2.0.

1.3.1ForeinResearchStatus

1UnitedStates

IntheUnitedStates,startupslikeOpenAIandAnthropic,alongwithtechgiants

suchasMicrosoftandGoogle,areleadingtherapiddevelopmentoflargemodels.

Majorcompaniesarecontinuallyenhancingtheircompetitiveness.Googleinvested

$300millioninAnthropictocounterthethreatposedbyChatGPT,joining

reinforcementlearningfromartificialintelligencefeedback(RLAIF)toreducehuman

feedback.InDecember2022,GooglepublishedapapertitledConstitutionalAI:

HarmlessnessfromAIFeedback,introducingtheAImodelClaude.Buzzfeed,aUS

newmediagiant,sawitsstockpricetripleintwodaysafterannouncingplanstouse

ChatGPTtoassistcontentcreation.Microsoft,asthemaininvestorinOpenAI,isalso

usingChatGPTtoenhanceitsproductcompetitivenessandsupplementits

professionalknowledgeandmathematicalshortcomings.

2UnitedKingdom

InApril2023,theUKgovernmentannouncedthatitwouldprovide￡100million

ininitialfundingtotheteamresponsibleforbuildingtheUKversionofthe

foundationalAImodeltoacceleratethedevelopmentofAItechnologyintheUK.The

UKgovernmentstatedthatthisinvestmentwouldbeusedtofundnewteamsjointly

builtbythegovernmentandtheindustrytoensuretheUK’sAI“sovereign

capabilities.”Thegoalofthisinitiativeistopromotetheapplicationofsafeand

16/92

reliablefoundationalmodelsandstrivetobuildtheUKintoatechnological

“superpower”by2030.Inaddition,inresponsetothecontroversyovertheapplication

oflargemodelssuchasGPTinAIethics,theUKhasalsoissuedawhitepaperon

regulatorymeasuresandstatedthatregulatoryagencieswillnextissueguidelinesand

riskassessmenttemplatestovariousorganizations.Othertoolsandresourceswillbe

usedtoformulatespecificimplementationprincipleswithintheindustry.

③Europe

InFinland,FlowriteisanAI-basedwritingtoolthatcangenerateemails,

messages,andothercontentbyinputtingkeywords.IntheNetherlands,the

omnichannelcommunicationplatformMessageBirdlauncheditsownAIplatform

MessageBirdAI,whichcanunderstandthemeaningofcustomerinformationand

respondaccordingly.BotharebasedonGPT-3.Germanyisalsoconstantlycatching

upinthedevelopmentoflargemodels.Forexample,onMarch7,2023,Google

launchedthemultimodallargemodelPaLM-E,jointlydevelopedbytheTechnical

UniversityofBerlinandGoogle.

InFebruary2024,theEuropeangenerativeAIunicornMistralAIunveiledits

latestLLM,MistralLarge.Withacontextwindowof32Ktokens,thismodelsupports

English,French,Spanish,German,andItalian.Astheflagshipmodelnewlylaunched,

MistralLargedemonstratedoutstandingperformanceincommon-sensereasoningand

knowledgequizzes,scoringhigheroverallthanGeminiProandClaude2,secondonly

toGPT-4.

④SouthKorea

SouthKoreaisalsoamongtheearliestcountriestoengageinlargemodel

development.Currently,notablerepresentativesinthisfieldfromSouthKoreainclude

NAVER,Kakao,KT,SKT,andLG.SouthKorea’saccumulationofexpertisein

semiconductorchipspositionsitadvantageouslyintherealmoflargemodels.

Presently,SouthKoreansemiconductorcompaniesareactivelyformingalliancesto

tacklethecomputationalchallengesposedbylargemodeldevelopment.Bytheendof

2022,NAVERinitiatedcollaborationwithSamsungElectronicstodevelopnext-

generationAIchipsolutions,optimizingthembasedonNAVER’slargemodel,

HyperCLOVA.Moreover,SouthKoreahasmadeconsiderableexplorationsinthe

verticalapplicationsoflargemodels,suchasKoGPTinhealthcareandExaonein

biopharmaceuticalsandintelligentmanufacturing.

17/92

⑤Japan

Japan,asacountrywithalesscommonlanguage,facesthechallengeof

insufficientlinguisticdata.TheearliestpubliclylaunchedNLPlargemodelinJapan

wasNTELLILINKBackOffice,introducedin2020,capableofdocument

classification,knowledgereadingcomprehension,andautomaticsummarization,

amongotherfunctions.ItisanapplicationdevelopedbasedonGoogleBERT.

ThemoreJapanese-bloodedgenerativeAIsareactuallyHyperCLOVA,Rinna

andELYZAPencil,butHyperCLOVAandRinnaalsohaveforeigngenes.

HyperCLOVA,initiallylaunchedbytheSouthKoreansearchgiantNAVERin2021,

standsoutasthefirstLLMspecificallytailoredfortheJapanese.Itachievedfirst

placeinalltracksatthedialoguesystemlivecompetitionheldin2021.ELYZAPencil,

ontheotherhand,isanLLMintroducedbyanAIstartupaffiliatedwiththeMatsuo

LaboratoryattheUniversityofTokyo,markingJapan’sfirstgenuinepublicreleaseof

agenerativeAIproduct.

1.3.2DomesticResearchStatus

ManymightbelievethatChina’sjourneywithlargemodelsbeganwiththe

“ERNIEBot,”butinreality,it’smerelyaconversationaltoolpoweredbylarge

models.Largemodelswerealreadyintroduceddomesticallyasearlyas2019.Inthat

year,largemodelswereextensivelyappliedindrugdevelopment,promptingmajor

technologycompaniestoinitiatetheirownlargemodelprojects.InMarch2021,the

BeijingAcademyofArtificialIntelligenceunveiledChina’sfirstultra-large-scale

intelligentmodelsystem,“Wudao1.0.”Subsequently,inAprilofthesameyear,

AlibabaGrouplaunchedPLUG,thelargestpre-trainedlanguagemodelintheChinese

community,whichwaswidelyreferredtoasthe“ChineseversionofGPT-3”atthe

time.

Inrecentyears,significantprogresshasbeenmadedomesticallyinthefieldof

largemodels.Fromresearchinstitutionstoenterprises,therehasbeenasubstantial

increaseininvestmentinlargemodels,leadingtosignificantbreakthroughsin

algorithms,computingpower,data,andotherareas.Chinahasproducedabatchof

internationallycompetitivelargemodels,widelyappliedacrossvariousfields.

OnMarch16,2023,basedontheERNIElargemodel,Baidureleased“ERNIE

Bot,”China’sfirstChatGPT-likeproduct.OnMay6,2023,iFLYTEKlaunchedthe

ChineseversionofChatGPT,“SparkCognitiveLargeModel,”capableoftext

18/92

generation,languageunderstanding,knowledgequestionanswering,logicalreasoning,

mathematicalabilities,codingskills,andmultimodalcapabilities.

1.3.3InternationalOrganizations

Today,internationalorganizationssuchastheInternationalOrganizationfor

Standardization(ISO)andtheInternationalElectrotechnicalCommission(IEC)have

allcarriedoutstandardresearchonkeyterminologies.InMarch2023,theEuropean

TelecommunicationStandardsInstitute(ETSI)alsointroduced