2024 全球6G技術(shù)大會 -10.0A GPT and Communication

1/92

ExecutiveSummary

Drivenbythejointeffortofkeytechnologiessuchasbigdataandcloudcomputing,asizablenumberofthegenerativepre-trainedtransformer(GPT)largemodels,representedbyChatGPT,haveemerged,showinghighlycreativecontentgenerationcapabilitiesandprovidinghighlyintelligenthuman-computerinteractionexperience.Foralongtime,therehavebeenmanytechnicalproblemsincommunicationthataredifficulttomodelaccuratelyorsolveefficientlyusingtraditionalmethods.Meanwhile,GPTdemonstratesthepotentialtoimprovetheperformanceofinformationcommunicationservicesandintelligentautonomousnetworks.Inaddition,therapiddevelopmentandbroadapplicationsofGPTalsoneedtobesupportedbyacommunicationnetworkwithlargebandwidth,lowlatency,and

highreliability.

Therefore,fromtheperspectiveofcommunicationpractitioners,thiswhitepaperexplorestheinterrelationshipbetweenGPTandcommunication.Firstly,Chapter1sketchestheconcept,developmentprocess,andresearchstatusofGPTlargemodels.Secondly,Chapter2discussesthenewapplicationsofGPTinthecommunicationindustry,andthepositionofGPTinnetworkintelligentautonomy.Thirdly,Chapter3exploreshowthecommunicationnetworksenablethebroadapplicationsofGPT,andgivesatypicalideaoffuturenetworkdesign.Moreover,Chapter4analyzestheprocessofGPTandcommunicationfromindependentevolutiontocollaborativedevelopment,aswellasapplicationsof“6G+GPT”empoweringthedigitaltransformationofindustries.Inaddition,Chapter5pointsoutthefivemostobviousproblemsandchallengesintheintegrationprocessof“GPT+Communication”andprovidessomesolutions.Subsequently,Chapter6putsforwardseveralsuggestionsonhowGPTandthecommunicationindustrycandeveloptogether,aswellasthe

prospectsforthefuture.Finally,Chapter7concludesthiswhitepaper.

2/92

Contents

ExecutiveSummary 1

0Preface 4

1.GPTLeadstheTideofArtificialIntelligenceDevelopment 8

1.1.BasicConceptsofGPT 8

1.1.1GenerativePre-trainedTransformer 8

1.1.2LargeModel 9

1.1.3TransformerArchitecture 11

1.2.DevelopmentHistoryofGPT 13

1.3.CurrentResearchStatusofGPT 15

1.3.1ForeinResearchStatus 16

1.3.2DomesticResearchStatus 18

1.3.3InternationalOrganizations 19

2.GPTEmpowerstheCommunicationIndustry 20

2.1.GPTStimulatesNewApplicationsandReformsinCommunication 20

2.1.1IntelligentCustomerService 22

2.1.2AutomationSimulation 23

2.1.3EnhancedSemanticCommunication 24

2.1.4ReshapingtheFieldofChipDesign 25

2.2.GPTPromotesIntelligentAutonomyinCommunicationNetworks 26

2.2.1GPTReshapesNetworkPlanning 28

2.2.2GPTEnhancesSlicingDeployment 29

2.2.3GPTSimplifiesNetworkOperationsandMaintenance 30

2.2.4GPTAcceleratesNetworkOptimization 32

3.CommunicationNetworksEnableGPTUbiquitousApplications 35

3.1CommunicationNetworksGuaranteetheLandingofGPTApplications 35

3.2FutureNetworkTechnologySupportsGPTApplications 38

3.2.1TypicalApproachestoFutureNetworkDesign 38

3.2.26GNetworkwithNativeSupportforGPTApplications 39

3.3NewNetworkArchitectureSupportsGPTCapabilitySinking 41

3.3.1AdaptiveSlicing 41

3.3.2DistributedLearning 43

3.3.3EdgeIntelligence 43

4.CollaborativeDevelopmentofGPTandCommunication 46

4.1.GPTandCommunicationfromIndependentEvolutiontoCloseIntegration 46

4.1.1TrendsintheIntegrationofGPTandCommunication 46

4.1.2IntegrationofGPTand5GNetworks 47

4.2.IntegrationandDevelopmentofGPTwith6GCommunicationNetworks 48

4.2.1GPTSupportsMassiveDataProcessing 49

4.2.2GPTPromotesNetworkSelf-Service 50

3/92

4.2.3GPTAssistsinNetworkResourceOrchestration 50

4.2.4GPTConstructsNetworkEndogenousSecurity 50

4.3.“6G+GPT”EmpowersIndustryDigitalTransformation 51

4.3.1“6G+GPT”EmpowersSmartIndustry 52

4.3.2“6G+GPT”EmpowersSmartHealthcare 53

4.3.3“6G+GPT”EmpowersSmartTransportation 53

4.3.4“6G+GPT”EmpowersSmartAgriculture 54

4.3.5“6G+GPT”EmpowersSmartHome 55

4.3.6“6G+GPT”EmpowersDigitalEntertainment 55

5.ProblemsFacedbytheDevelopmentof“GPT+Communication”Integration56

5.1.ScarcityofHigh-QualityTrainingDatainCommunicationLeadstoPoorAccuracyand

GeneralizationofSpecializedModels

5

7

5.2.InsufficientOn-DeviceComputingPowerandHardwareResourcesPoseChallengesto

LightweightDeploymentofLargeModels

6

0

5.3.DifficultiesinCloud-Edge-TerminalHeterogeneousNetworkCollaborationLeadtoPoor

StabilityPerformanceofLargeModels

6

2

5.4.ServerInterconnectionBandwidthBottlenecksResultinLongTrainingTimeandLow

InferenceEfficiency

6

5

5.5.LaggingLegalRegulationsRelatedtoLargeModelsResultinHighRisksofSecurity,

Privacy,andEthicalIssues

6

7

6.DevelopmentRecommendationsandFutureProspects 71

6.1.DevelopmentRecommendations 71

6.1.1AcceleratingtheConstructionofAIComputingPowerandProvidingInfrastructure

Support

7

1

6.1.2StrengtheningJointTrainingofSchoolsandEnterprisestoFilltheGapin

InnovativeTalents

7

4

6.1.3AcceleratingtheFormulationofRelevantPoliciesandEstablishingPlatformsto

GuideDevelopment

7

6

6.2.FutureProspects 78

6.2.1BreakthroughsinCoreTechnologiesandSignificantEnhancementofKey

Capabilities

7

8

6.2.2ContinuousImprovementinSystemConstructionandRapidDevelopmentofthe

DigitalEconomy

8

0

6.2.3ExpansionofApplicationScenarios,GradualIntegrationandSymbiosis 82

7.Conclusion 84

References 85

Abbreviations 90

Acknowledgments 92

4/92

0Preface

Inrecentyears,asArtificialIntelligence(AI)technologyhascontinuedtoadvance,particularlyintheareasofreinforcementlearning,largemodels,andgenerativecontent,variousindustrieshavebeenactivelyexploringitsapplications.AttheendofNovember2022,OpenAIreleasedtherapidlypopularizedchatbotChatGPT,whichpossessesastonishingnaturallanguageunderstandingandgenerationcapabilities,attractingwidespreadattentionfromsociety.Subsequently,inMarch2023,thelaunchoftheupgradedversionGPT-4multimodallargemodelreignitedenthusiasmforgenerativeAI,leadingtotheemergenceofnumerouslargemodelsin

quicksuccession.

Sincetheinceptionoftext-basedconversationalinteractions,GPThasprofoundlyimpactedpeople’sproductionandliveswithinafewshortyears,bringingaboutsignificantchanges.Manypeoplebelievethatitwillcontinuetobringdisruptivechanges.BillGatespointedoutthatlargemodelsrepresentthemostrevolutionarytechnologicaladvancementinover40years;NVIDIACEOJensenHuanglikenedtheemergenceoflargemodelstothe“iPhonemoment”ofAI;BaiduCEORobinLiproposedthatlargemodelsarepreparedtochangetheworldatthe2023ZhongguancunForum.FromtheripplescausedbyChatGPTtotheglobalwaveitunleashed,GPTlargemodelshavebecomeoneofthemostdiscussedtopicstoday,signalingacrucialturningpointinthedevelopmentofgenerativeAI;theyear2023

willalsoundoubtedlyleaveasignificantmarkinthehistoryofAIdevelopment.

Asanindustryfacilitatinginformationexchangeandtransmissionamonghumans,nature,andmachines,thecommunicationindustryiscloselyintertwinedwiththedevelopmentoflargemodeltechnology.Thecommunicationindustryitselfhasahighdegreeofdigitalizationandneedstohandlecomplexdata.TheintroductionofGPTcanstreamlineasignificantamountofwork,bringingaboutsignificantcapacityenhancementsforcommunicationoperators,particularlyintherealmsofnetworkoperationsandmaintenance(O&M)andservicedelivery,makingthemmoreintelligent.Intheeraoflargemodels,withtheadvancementofGPTtechnology,thedemandforcomputingpower,data,andalgorithmswillexperienceexplosivegrowth,requiringcommunicationinfrastructuretoprovidesupport.Inthefuture,howGPT

empowersthecommunicationindustryandhowthecommunicationindustrysupports

5/92

GPTarequestionsthateverycommunicationprofessionalshouldearnestly

contemplate.

Therefore,thiswhitepaperisbasedonthedevelopmenthistoryandlatestresearchadvancementsofGPTlargemodels.Ontheonehand,itelaboratesontheinnovativeapplicationsofGPTwithinthecommunicationindustryinspecificscenarios.Ontheotherhand,itinvestigateshowfuturecommunicationnetworksprovidenativesupportforGPTintermsofarchitectureandkeytechnologies.Subsequently,combiningGPTwithcommunication,itproposesaroadmapforthedigitalandintelligenttransformationofkeyindustriesthroughtheircollaborativedevelopment,whilealsopointingouttheproblemsandchallengesintheintegrationanddevelopmentprocess.Inresponsetotheseissues,correspondingdevelopmentrecommendationsandprospectsareprovided.Finally,thewholecontentofthiswhitepaperissummarized.Thecompletechapterstructureofthiswhitepaperisillustrated

inFigure0-1below.

6/92

Figure0-1WhitePaperChapterStructureDiagram

ThiswhitepaperwasjointlyorganizedandauthoredbytheBeijingInstituteofTechnology,withparticipationfrom18entities,includingthethreemajortelecomoperators(ChinaMobile,ChinaUnicom,andChinaTelecom),seventop-tieruniversities,threerenownedenterprises,andfiveleadingresearchinstitutesintheindustry.Spanningovereightmonths,theprocessinvolvedthein-depthparticipationofover50expertsandscholars,fromconductingresearchandtrackingthecutting-edgestatusofGPTlargemodelstoexploringtherelationshipbetweenGPTandcommunication,conceptualizingtheoutlineofthewhitepaper,arrangingspecificchaptercontent,andassigningwritingtasks.Itunderwentmorethantwentyroundsofdiscussionsandrevisionsbeforereachingitscompletion.Duringthisperiod,some

participatingentitiesalsosuccessfullycollaboratedtoapplyforaninternational

7/92

cooperationprojectfromtheMinistryofScienceandTechnologyofthePeople’sRepublicofChina,titled“ResearchonKeyTechnologiesofIntegratedMultidimensionalIntelligentOrchestrationinCloudComputingNetworksBasedon

LargeModels,”therebybettersupportingthecompletionofthiswhitepaper.

WebelievethatAItechnologyisstillinarapidlydevelopingstage,andtheintegrationandmutualsupportbetweenGPTlargemodelsandcommunicationnetworkscancontinuallyexpandinnovativeapplicationscenariosandimproveecosystemdevelopment,thusjointlypromotingtechnologicalprogressandthe

developmentofvariousindustries.

8/92

1.GPTLeadstheTideofArtificialIntelligenceDevelopment

WiththeadvancementofAIanddeeplearningtechnologies,theconceptof“l(fā)argemodels”hascomeintofocus,withChatGPTbeingthemostnotable.OnNovember30,2022,OpenAIofficiallyreleasedtheAIchatbotChatGPT,whichrepresentsArtificialIntelligenceGeneratedContent(AIGC)inthefieldofnaturallanguage.Itspowerfulcapabilitieshavechangedthewaymanypeopleworkandlive,sparkinganewwaveofAIgloballyandattractingwideattentionfrombothindustryandacademia.OnMarch14,2023,theofficiallyreleasedGPT-4underwentfurtherupgrades,significantlyrelaxingtextinputrestrictions,improvingansweraccuracy,andevenenablingdirectinputofimagestogeneratelyrics,creativetexts,etc.,withstylevariations,onceagainshowcasingtheimpactofgenerativeAI.OnNovember7,2023,atthefirst-everOpenAIDevDay,OpenAICEOAltmanshowcasedGPT-4Turbototheworld.AsthelatestversionofGPT,ithasbeenupdatedinareassuchasdataquality,imageprocessing,andspeechconversion,bringingdevelopersandusers

morepossibilitiesandopportunities.

So,whatareChatGPTandGPT?Whatdevelopmentjourneyhavetheyundergone?Andhowshouldtheybeunderstoodandapplied?ThischapterwillstartwithanexplorationofGPTlargemodels,introducingtheirbasicconcepts,developmenthistory,andcurrentresearchstatustoprovidereaderswitha

comprehensiveandin-depthunderstandingofGPT.

1.1.BasicConceptsofGPT

1.1.1GenerativePre-trainedTransformer

GPTstandsforGenerativePre-trainedTransformer,originatingfromthefieldsofdeeplearningandnaturallanguageprocessing(NLP).Overthepastfewyears,withtheadvancementofcomputingpowerandtheemergenceofbigdata,significantbreakthroughshavebeenmadeinthefieldofNLP.GPT,asanintegrationofaseries

ofNLPtechnologies,emergedinsuchacontext,asshowninFigure1-1.

G:Generative.ThisindicatesthatGPThastheabilitytospontaneouslygenerate

content.

P:Pre-trained.ThisindicatesthatGPThasundergonepre-trainingandisready

forimmediateuse.

9/92

T:Transformer.ThisindicatesthatGPTisbasedontheTransformerarchitecture

forlanguagemodeling.

Figure1-1MeaningofGPT

In2017,theGoogleteamfirstproposedtheTransformermodelbasedontheSelf-AttentionMechanism(SAM)andappliedittoNLP[1].OpenAIappliedthistechnologyandreleasedtheearliestgenerationoflargemodels,GPT-1,in2018.Sincethen,theparametersizeofeachgenerationofGPTmodelshasgrownexplosively.TheparametersizeofGPT-2,releasedinFebruary2019,was1.5billion,whileGPT-3,

releasedinMay2020,directlyreached175billion.

ThemeteoricriseofChatGPTwasnotbychance.Itistheresultoftheeffortsofmanypeopleandalongperiodofevolution.TounderstandthedevelopmentofGPT,

oneshouldfirstgrasptheconceptoflargemodelsandTransformerarchitecture.

1.1.2LargeModel

Generally,beforeChatGPT,theAImodelsthatreceivedpublicattentionweremainlyusedforsingletasks.Forexample,“AlphaGo”,whichignitedtheentireAImarketandprompteditsexplosivedevelopment,defeatedGoworldchampionLeeSedolinthe“Manvs.Machine”matchin2016,basedonglobalGogamerecords.However,fundamentally,theseAIdatamodels,whichfocusonspecifictasks,can

onlybecalled“smallmodels”comparedtoChatGPT.

Largemodelsrefertomachinelearningmodelswithhugeparameterscalesandcomplexity.ThetermusuallyreferstoLargeLanguageModels(LLMs).AlanguagemodelisanAImodelthat,aftertraining,canunderstandandgeneratehumanlanguage,and“l(fā)arge”meansthatthemodel’sparametersareverylargerelativeto

“smallmodels.”

AsshowninFigure1-2,thisevolutionarytreetracesthedevelopmenthistoryof

10/92

largemodelsinrecentyears,highlightingsomeofthemostwell-knownmodels,withmodelsonthesamebranchbeingmorecloselyrelated[2].Solidsquaresrepresentopen-sourcemodels,whilehollowsquaresrepresentclosed-sourcemodels.Non-Transformermodelsareshowningray,andamongTransformer-basedmodels,Encodermodelsareinthepinkbranch,Decodermodelsareinthebluebranch,and

Encoder-Decodermodelsareinthegreenbranch.

Figure1-2EvolutionaryTreeofLargeModels

Basedonthisevolutionarytreediagram,wecanconcludethatDecoder-onlymodelsaregraduallybecomingthedominantmodelsinLLMdevelopment,andOpenAIcontinuestomaintainitsleadingpositioninLLM.Metahasmadeoutstandingcontributionstoopen-sourceandLLMresearch,butthereisatrendtowardsclosed-sourcedevelopmentafterthelaunchofGPT-3.Inaddition,manycompaniesandinstitutionsarestillactivelyexploringEncoder-Decodermodels,such

asGoogle.

Currently,majorinstitutionsabroadthatreleaselargemodelsincludeOpenAI,Anthropic,Google,andMeta,withmodelparameterscalesmainlyinthetensandhundredsofbillions.Uptonow,thetopGPTlargemodelsabroadincludeChatGPT,

Claude,Bard,andLlama.Amongthem,afterGooglereleasedthelatestnative

11/92

multimodallargemodel–Gemini,BardwasofficiallyrenamedGemini.

Inthisgloballycompetitivearena,Chinaisalsokeepingpace,developingmanylargemodels,includingTencent’s“Hybrid,”Alibaba’s“TongyiQianwen,”Huawei’s“Pangu,”andChinaMobile’s“Jiutian”series.DatashowsthatasofOctober2023,thereareatotalof254domesticcompanies,universities,andresearchinstituteswithlargemodelsofover1billionparameters,indicatingthatthe“battleofthehundredmodels”istransitioningfromthepreviousstageof“beingborn”toanewstageof“beingused.”Figure1-3showssomeofthelargemodelsdevelopedbydomesticand

foreigncompaniescurrently.

Figure1-3VariousTypesofLargeModels

1.1.3TransformerArchitecture

TheTransformerarchitectureisacrucialfoundationofGPT,whichisaneuralnetworkarchitecturebasedontheSAMandwidelyusedinlargemodelsinthefieldofNLP.ItscorecomponentsaretheEncoderandDecoder.TheEncoderencodesinputtextintoaseriesofvectors,whiletheDecoderdecodesthesevectorsonebyoneintooutputtext.BeforetheintroductionofTransformer,themainstreammodelsintheNLPfieldwereRecurrentNeuralNetworks(RNNs),whichusedrecursionand

convolutionalneuralnetworksforlanguagesequencetransformation.

InJune2017,theGoogleBrainteampublishedapapertitledAttentionisAllYouNeedatthetopAIconferenceNeurIPS,proposinganewnetworkarchitecturecalledTransformer.ItisentirelybasedontheSAM,abandoningrecursionandconvolution.Afteronly12hoursoftrainingoneightP100GraphicsProcessingUnits(GPUs),

Transformerachievedhighertranslationquality[1],showcasingexcellentparallelism

12/92

andbecomingthemostadvancedLLMatthetime.

Figure1-4illustratesthenetworkstructureoftheTransformer.ItconsistsofaseriesofEncodersandDecoders,eachcomprisingmulti-headattentionlayersandall-inclusiveconnectedfeedforwardnetworks.GPT,similartotheDecoderpartof

Transformer,isanautoregressivemodel.

Figure1-4TransformerNetworkStructureDiagram

ThecorecomponentintheTransformeristhemulti-headattentionmechanismmodule,asshowninFigure1-5.Itrequiresthreespecifiedinputs:Q(Query),K(Key),andV(Value).Then,itcalculatesthesimilaritybetweeneachpairofQandKand

weightseachVbasedonthesimilaritytoobtaintheattentioncalculationresult.

13/92

Figure1-5Multi-HeadAttentionMechanismModule

Themulti-headattentionmechanismdoesnotcalculateattentiononlyoncebutdividestheinputintosmallerblocksandthencalculatesthescaleddot-productattentioninparalleloneachsubspace.Thisdesignallowseachattentionmechanismtooptimizedifferentfeaturepartsofeachword,balancingthebiasesthatmayarisefromthesameattentionmechanismandenablingthemodeltocapturesemanticinformationatdifferentlevels,therebyenhancingthemodel’sexpressivepowerand

improvingitseffectiveness.

1.2.DevelopmentHistoryofGPT

14/92

Figure1-6DevelopmentHistoryofGPT

ThedevelopmenthistoryofGPTcanbedividedintotwostages.BeforeChatGPT,theemphasiswasoncontinuouslyincreasingthebasicscaleoflargemodelsandenhancingnewcapabilities.ChatGPTandGPT-4,ontheotherhand,focusmoreonreinforcementlearningfromhumanfeedbacktounderstandhuman

intentandprovidebetterservices,asshowninFigure1-6.

①June2018:OpenAIpublishedthepaperImprovingLanguageUnderstandingbyGenerativePre-trainingandofficiallyreleasedGPT-1[3].

.Basicapproach:Generativepre-training(unsupervised)+downstreamtask

fine-tuning(supervised).

.BasedonaunidirectionalTransformerlanguagemodelwithadecoder

structure,consistingof12layers.

.117millionparameters,5GBtrainingdata,relativelylimitedmodelsizeandcapabilities.

.Contextwindow:512tokens.

②February2019:OpenAIpublishedthepaperLanguageModelsareUnsupervisedMultitaskLearners,proposingthatlanguagemodelsareunsupervisedmultitasklearners,andGPT-2wasborn[4].

.Basicapproach:Removingsupervision,retainingonlyunsupervisedlearning.

.48-layerTransformerstructure.

15/92

.1.5billionparameters,andthetrainingdatavolumeincreasedto40GB.

.Contextwindow:1024tokens.

③May2020:OpenAIpublishedthepaperLanguageModelsareFew-Shot

LearnersandintroducedtheGPT-3model[5].

.Basicapproach:Unsupervisedlearning+in-contextlearning.

.96-layermulti-headTransformer.

.Thenumberofparametersincreasedto175billion,trainedon45TBoftextdata.

.Contextwindow:2048tokens.

④March2022:OpenAIonceagainpublishedthepaperTrainingLanguageModelstoFollowInstructionswithHumanFeedback,introducingReinforcementLearningfromHumanFeedback(RLHF),andlaunchedtheInstructGPTmodel[6].

.Basicapproach:RLHF+fine-tuningtraining.

.Enhancedhumanadjustmentofmodeloutput.

.Resultsrankedinamoreunderstandablemanner.

ChatGPTisaderivativeofInstructGPT,andthetwohavethesamemodelstructureandtrainingmethod.Theonlydifferenceisthewaytheycollectdata.

ChatGPTfocusesmoreoninteractionintheformofdialogue.

⑤March2023:OpenAIreleasedthemultimodalpre-trainedlargemodelGPT-4,

onceagainundergoingsignificantupgrades.

.Basicapproach:Multimodal.

.Contextwindow:8195tokens.

.1.8trillionparameters,13trilliontokentrainingdata.

.Powerfulimagerecognitioncapabilities.

AlthoughthecurrentcapabilitiesofGPT-4inreal-worldscenariosmaynotmatchthoseofhumans,ithasdemonstratedsignificantlysuperiorabilitiesinvariousprofessionalandacademicexams.EvenSATscores(whichcanbeunderstoodasscoresfortheU.S.collegeadmissionstest)ofGPT-4havesurpassedthoseof90%oftesttakers,reachingthelevelrequiredforadmissiontotopuniversitiessuchas

HarvardandStanford.

1.3.CurrentResearchStatusofGPT

OnOctober12,2023,theanalysiscompanystateof.aireleasedtheStateofAI

Report2023.ThereportpointedoutthatOpenAI’sGPT-4remainsthemostpowerful

16/92

LLMglobally.GenerativeAIhaspropelledadvancementsinlifesciencesandhasbeenasaviorfortheventurecapitalindustry[7].Largemodelscontinuetoachievetechnologicalbreakthroughs,especiallyinthefieldoflifesciences,making

significantprogressinmolecularbiologyanddrugdiscovery.

OnDecember14,2023,Natureannouncedtenpeoplein2023.Notably,thechatbotChatGPT,duetoitsdominanceofvariousnewsheadlinesin2023andprofoundimpactonthescientificcommunityandsocietyatlarge,wasincludedasthe11th“non-humanmember”onthelist,recognizingthesignificantchangesbroughtaboutbygenerativeAItoscientificdevelopmentandprogress.Currently,bothdomesticallyandabroad,researchonGPTlargemodelscontinuestodeepen,withmanyinstitutionsstartingtodeveloptheirownlargemodels,andtheapplicationscenariosarebecomingincreasinglydiverse.LargemodelsrepresentedbyChatGPT

haveofficiallyusheredintheeraofAI2.0.

1.3.1ForeinResearchStatus

1UnitedStates

IntheUnitedStates,startupslikeOpenAIandAnthropic,alongwithtechgiantssuchasMicrosoftandGoogle,areleadingtherapiddevelopmentoflargemodels.Majorcompaniesarecontinuallyenhancingtheircompetitiveness.Googleinvested$300millioninAnthropictocounterthethreatposedbyChatGPT,joiningreinforcementlearningfromartificialintelligencefeedback(RLAIF)toreducehumanfeedback.InDecember2022,GooglepublishedapapertitledConstitutionalAI:HarmlessnessfromAIFeedback,introducingtheAImodelClaude.Buzzfeed,aUSnewmediagiant,sawitsstockpricetripleintwodaysafterannouncingplanstouseChatGPTtoassistcontentcreation.Microsoft,asthemaininvestorinOpenAI,isalsousingChatGPTtoenhanceitsproductcompetitivenessandsupplementits

professionalknowledgeandmathematicalshortcomings.

2UnitedKingdom

InApril2023,theUKgovernmentannouncedthatitwouldprovide￡100millionininitialfundingtotheteamresponsibleforbuildingtheUKversionofthefoundationalAImodeltoacceleratethedevelopmentofAItechnologyintheUK.TheUKgovernmentstatedthatthisinvestmentwouldbeusedtofundnewteamsjointlybuiltbythegovernmentandtheindustrytoensuretheUK’sAI“sovereign

capabilities.”Thegoalofthisinitiativeistopromotetheapplicationofsafeand

17/92

reliablefoundationalmodelsandstrivetobuildtheUKintoatechnological“superpower”by2030.Inaddition,inresponsetothecontroversyovertheapplicationoflargemodelssuchasGPTinAIethics,theUKhasalsoissuedawhitepaperonregulatorymeasuresandstatedthatregulatoryagencieswillnextissueguidelinesandriskassessmenttemplatestovariousorganizations.Othertoolsandresourceswillbe

usedtoformulatespecificimplementationprincipleswithintheindustry.

③Europe

InFinland,FlowriteisanAI-basedwritingtoolthatcangenerateemails,messages,andothercontentbyinputtingkeywords.IntheNetherlands,theomnichannelcommunicationplatformMessageBirdlauncheditsownAIplatformMessageBirdAI,whichcanunderstandthemeaningofcustomerinformationandrespondaccordingly.BotharebasedonGPT-3.Germanyisalsoconstantlycatchingupinthedevelopmentoflargemodels.Forexample,onMarch7,2023,GooglelaunchedthemultimodallargemodelPaLM-E,jointlydevelopedbytheTechnical

UniversityofBerlinandGoogle.

InFebruary2024,theEuropeangenerativeAIun