【會(huì)議演講PPT】Gartner-開(kāi)始您的生成式AI之旅（APAC）

Gartner

Webinars

Gartnerdeliversactionable,objective

insight,guidanceandtoolstoenable

strongerperformanceonyour

organization’smissioncriticalpriorities

Enhanceyourwebinarexperience

Aska

Question

Download

Attachments

Watch

Again

2

RESTRICTEDDISTRIBUTION

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

ConnectwithGartner

GetStartedonYourGenerativeAI

Journey(APAC)

AlbertGauthier

SrDirectorAnalyst

RESTRICTEDDISTRIBUTION

3

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

WhatisArtificialIntelligence

?AIin2023describesstatisticalanalysisthathasbeenaroundfor50years.

?Setoftoolswrappedaroundto“tune”thestatisticalanalysis.

?Asrawcomputingcapabilitiesimprove,theabilitytoquicklyanalyzelargedatasetsandmodifythosedatasetshasimproved.

?ArtificialIntelligenceisthe“l(fā)atest”descriptionofoldtechnologies.

?CurrentMLisbasedalmostexclusivelyonStatisticalAnalysistofindpatterns

?Notintelligent.

4

RESTRICTEDDISTRIBUTION

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

Distinctions

?Theterm“AI”hasbeenusedtodescribemanythings.

–NLP(Alexa,Google)

–Linear/PolynomialRegressionAnalysis

–Probability

–NeuralNetworks(RemembertheTerminator)

–MachineLearning

–Stochasticbasedmodels(LLMincludingChatGPT)

5

RESTRICTEDDISTRIBUTION

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

UtopianandDystopianViewsofAI-Orthis.

6

RESTRICTEDDISTRIBUTION

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

TheHypeofGenerativeAI

?Headline:

HowgenerativeAIisrevolutionizingthefutureofsmartcities

?“Inconclusion,generativeAIhasthepotentialtorevolutionizethewayweplan,develop,andmanagesmartcities.Byprovidinginsightsintocitizenbehavior,trafficpatterns,andenvironmentalfactors,generativeAIcanhelpcitiesbecomemoreefficient,sustainable,andaccessible.”

RESTRICTEDDISTRIBUTION

7?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

WhataboutChatGPTandotherLLM?

?ChatGPTisa“LargeLanguageModel”(LLM)orFoundationalMachineLearningModel.OriginalmodelsbuiltonStochastics(somecallthestochasticparrots).

?ConversationalchatbotwithGenerativePretrainedTransformer(GPT).

?Is“generative”AImeaning,itcraftsaresponsewith“new”contentandtriestoformatitintonaturallanguage.

?(Stochasticsisthestudyofdatasetswithrandomprobabilitydistributionsthatcanbeanalyzedstatisticallybutnotpredicted.)Duetotheuncertaintypresentinastochastic

model,theresultsprovideanestimateoftheprobabilityofvariousoutcomes.

?Interesting,entertainingandwrappedinalotofhype(anyonerememberthemetaverse).

?GARTNER:IsChatGPTartificialgeneralintelligence?No.

RESTRICTEDDISTRIBUTION

8?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

StochasticModelling-5to95%probability

9

RESTRICTEDDISTRIBUTION

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

Howdoesitwork?

?Classifies“intent”likeAlexaorGooglewithconfidencescores(statistics)BUTusingstochastic-likeanalysis.

?Produces“constraints”toboundtheresponse.

?Trainedwithupto300BillionWordsfromvarioussources.

?Cansummarizeresponseswithmarginaldegreesofaccuracy(usecautiously)conditionaluponinput.

?Modelsarefine-tunedbyyourfeedback(unlessyouusetheAPI)

?Generatesoutputsbasedontrainedfoundationalmodels(i.e.Ifthemodelisnottrainedinaparticulararea,itdoesn’twork).

?Usesprobabilityanalysis.

?Determinethebest(mostprobable)pathbasedonyourinput.

RESTRICTEDDISTRIBUTION

10?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

Morespecifically,modelstrengthsinclude

?Generateandaugmentproseornarratives

?Codedevelopment,translation,explanationandaugmentation

?Summarizeandsimplifylong-formtexts.

?Classifycontentforsentimentorbytopicarea.

?Answerquestions,

?Translateandconvertlanguage(includingprogramminglanguages).

?Writtencontentaugmentationandcreation.

11

RESTRICTEDDISTRIBUTION

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

Whatitisn’t

?Accuratemuchofthetime.

?Equallystrongacrossalldomains…onlywhereit’strained.

?Sentient(Perceptive)…itisnotAI

?Insightfuli.e.)Givesyouthesameanswerifyouaskhowtobuildahigh-performanceteamofplumbersorbrain-surgeons.

?Reliable&Trustworthy(ieRequiresexpertreview).

?Abletobecustomizedortrainedwithyourdata.

?Notparticularlyinsightfulmuchofthetime.Regurgitatesprescribedpathsthroughthemodel.

12

RESTRICTEDDISTRIBUTION

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

Salesandmarketing:Engagewithpotentialcustomers

onawebsiteorinachatbot,andprovide

recommendationsandproductdescriptions.

HR:Createinterviewquestions,writeofferlettersandjobdescriptions,summarizeemployeesurveyresultsand

suggestemployeeengagementactivities.

Customerservice:Improvecustomer-facingchatbotbreadthandquality,effectivelyrespondtocustomerinquiriesand

complaints,andgeneratepersonalizedresponses.

Softwareprogramming:Generatecomputercodefromprose,convertcodefromoneprogramminglanguagetoanother,correcterroneouscodeandalsoexplaincode.

PopularUseCasesofChatGPT

ChatGPTCapabilities

?Createwrittencontent.

?Answerquestions(noncomputational)anddiscoverinformation.

?Transformthetone,formalityorwriting

genreoflanguageonrequest.

?Summarizeandclassifytext.

?Compareparagraphsandcorrect

grammar.

?Generateideas,suggestionsandkeypointsondifferenttopics.

?Classifyandcategorizecontentbasedontheexampleprovided.

?Generate,translate,explainandverifycomputercode.

?Translatetexttoinstructions,queryordifferentlanguage.

RESTRICTEDDISTRIBUTION

13?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

SelectEnterpriseUseCasesofChatGPT

Legalandcompliance:Draftandsummarizelegaldocuments,andcreatedraftcompliancepoliciesandtrainingmaterial.

Gartner’sGenerativeAIDefinition:

?Createsnewlyderivedcontent,strategies,designsandmethods.

?Learnsfromlargerepositoriesoforiginalsourcecontent.

RisksExecutivesShouldbeWatching

ΔHallucinations

ΔNoAttribution

ΔDataLeakage

WhatExactlyisGenAIinaProfessionalContext?

GenAICreates&LearnsWhatUseCasesAreEmergingforCXOs?

GartnerUseCasePrismforGenerativeAI

Gartner’sAIDefinition:

?Analyzesdatawithlogic-basedtechniqueslikeMachinelearning(ML)

?Interpretsevents,supportsandautomatedecisions(carefulhere).

14

RESTRICTEDDISTRIBUTION

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

KeyIssueTake-Away:

Foundationmodelsrepresentahugestep

changeinthefieldofAI,duetotheirmassivepretraining,whichmakesthemeffectiveat

few-shotandzero-shotlearning,enablingthemtobeversatile.

15

RESTRICTEDDISTRIBUTION

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

But…

?Toomanyorganizationsarejumpingintothetechnologywithoutunderstandingtheproblemandtheusecase

?ThisisgoingtocreatefailedPOCs

?Manyorganizationshaveasolutionlookingforaproblem.

?Drivenbyleadershipandthehypecycle.

?Havecompletemisunderstandingofhowthemodelsarebuilt,usecasesandlimitation.

16

RESTRICTEDDISTRIBUTION

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

GettingStartedinGenAIPilot

?Whatisyourusecase?

?WillOOTBmodelsuffice?

–PromptEngineering&TokenFiltering?

–ModelSelection?

–TextBasedUI(ChatGPT)

–API’sandApplicationEmbedding.

?IsModelAugmentationrequired?

–ModelSelection?

–ModelTraining,TestingandFeedback?

-APIsusedforTraining?

17

RESTRICTEDDISTRIBUTION

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

ShareableSummary

KeyFindings

?Themostsuccessfulpilotsfocusondemonstratingbusinesspotential,notontechnical

feasibility.OrganizationstendtoruntechnicaIpilotsthatsimplydemonstratethatitis

possibletobuildsomethingwithgenerativeAI,leadingtoonlyincrementalimprovementsandignoringthetransformativepotentialofthistechnology.

?ITleadersstruggletoidentifyandprioritizeimpactfulgenerativeAIusecasesduetothebroadandemergingnatureofthetechnology.

?MatureAIorganizationsinvolvebusinesspartnersandsoftwareengineersaskeymembersoftheirAIprojectsandpilotteams.

ViewDocument

RESTRICTEDDISTRIBUTION

18?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

?2023GartnerInc.and/oritsaffiliates.Allrightsreserved.

TextGeneration,Q&A,Summarization,Search,Classification,EntityExtraction,IntentRecognition,Translation,Rewrite,

TexttoSpeech

TexttoImage,ImageClassification,ObjectDetection,VideoClassification,ImagetoText

TexttoCode,CodeCompletion

?DrugDiscovery,GenomicSequencing,ChemicalFormulation

?Human-RobotInteraction

UseCasesforFoundationModels

NLP

ComputerVision

SoftwareEngineering

GeneralSciences

&Others

RESTRICTEDDISTRIBUTION

19?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

EnterpriseChatGPT/GPTUsageAreas:ProsandCons

Out-of-the-BoxModelUsage

?UsesChatGPTservice“asis,”nodirectaccesstoGPT-3.5model.

?Pro:Fasttomarket;limitedinvestments;gainexperience.

?Con:Limiteddifferentiation;controlrangeislimited.

Prompt

Engineering/

?Usestoolstocreate,tune,andevaluatepromptinputsandoutputs.

?Pro:BettertargetedChatGPTandGPT3results;lowstartupcosts.

InContextLearnigCon:Mustintegratewithbusinesssystemstointroducedata.

Deployment/FineTuningofCustom

Models

?Uses(builds/finetunes/licenses)GPTorotherlanguagemodelsdirectly.

?Pro:Customizedoroptimizedmodels,data,parametersandtuning.

?Con:Requiresaddedfundingandskills.ThisisnotChatGPT.

20

RESTRICTEDDISTRIBUTION

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

Out-of-the-BoxModelUsage

?Thisformofusageisbyfarthemostaccessibleandcommontoday.

?Text-basedwebchatinterface().APIrecentlyavailable.

?Formostusecases,outputmustbereviewedbyahuman,asitmaycontaininaccuraciesorunacceptablecontent.

?Enterprisesmayachieveusefulresultswithlimitedinvestmentsandskills.Butbecausemanyusersareinexperienced,theyriskoverlookingdata,securityandanalyticsrisks.

?Alimitationisthatthemodelcannotincludereal-time,currentorcustomdata.Nordoesitcoverrecenthistoricalevents(thoseafterDecember2021).However,newdatacanbeaddedviaa

promptatthetimeofinteraction.

21

RESTRICTEDDISTRIBUTION

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

PromptEngineering/InContextLearning

?PromptengineeringcanbeappliedtobothChatGPTandGPTusecases.Itinvolvesdevelopingasystematicapproachtocreating,tuning,andevaluatingresultsintermsofinputsandoutputstoandfromChatGPT.

?InChatGPT,thepromptisthecriticalelementdrivingresults.Smallchangestoaprompt’schoiceofwordsandwordordercanresultinsignificantchangesinoutput.Apromptcanalsocontain

datathatshouldbeincorporatedorconsideredwhengeneratingaresponse.

?Leadersshouldanticipatethatpromptengineeringisanewtechnicalskillthatwillneedtobedeveloped,alongwithrelatedtools.

?Insomecases,thisrequirementwillextendtobuildingaseparatelearningmodeltooptimizeprompts.

?InContextLearning,leveragingRetrievalAugmentedGeneration,isthedominantmodelinusebyorganizationsthatmustkeepdatasecureandregularlyupdatedatainanLLMcontext

22

RESTRICTEDDISTRIBUTION

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

Deployment/FineTuningofCustomModels

?Thisisthelikelylong-termapproachforsophisticatedsolutions.

?ThisapproachisnotpossiblewithChatGPT,asitdoesnotprovideuserswithaccesstocustomizeitsunderlyingmodel.

?BesidesGPT,otherfoundationmodelsexist.Somearespecialized.

?Customizingfoundationmodelsisacomplextaskthatrequiressignificantskills,datacurationandfunding.

?Enterprisesshouldanticipatearobustmarketforthird-partymodelscustomizedfordifferentusecases.

?Plannersshouldanticipatetheemergenceofthird-party,fit-for-purpose,specializedmodels.Buyingoneofthesemayproveabetterapproachformanyenterprisesthancustomizingamodelthemselves.

?Applicationsmayalsohaveprebuiltmodelsfortheirusers.

23

RESTRICTEDDISTRIBUTION

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

ViewDocument

RESTRICTEDDISTRIBUTION

24

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

5

?2023GartnerInc.and/oritsaffiliates.Allrightsreserved.

VALUE

●Benefits:

○Thesimplicityandversatilityofthisdesignpattern—ageneralpurposenaturallanguagegenerationtool—makesithighvaluein

complementingworkflowsoflanguageandsoftwareproduction.

●Drawbacks:

○Riskofincorrectorbiasedoutputs,requiringhumanqualitycontrolofgeneratedresponse.

○PotentialprivacyriskswhensharingIPorconfidentialinformation.

USECASES

●Codegeneration

●Ideageneration/brainstorming

●Copywriting/contentcreation

●Generalknowledgediscovery/search

●Basictranslation/NLPtasks

UseLLMs“As-Is”

Prompt

Response

SimplePrompt

User

LLM

Prompt+Content

Response

PromptWithContent

User

LLM

VALUE

●Benefits:

○ThepotentialtolinkLLMswithinternaldocumentdatabases,unlockinginsightsfrominternaldatawithLLMcapabilities.

○Thepotentialtohavemuchmoreaccurateandrecentinformation.

○Theresultingsystemcouldincludereferences/citationstotheoriginalsourcedocumentsfromwhichtheresponsewasgenerated.

●Drawbacks:

○LLMretrievalmodelscanstillbeinaccurateandhallucinate,albeittypicallylessthanwhenusingLLMswithoutretrieval.

○Requiresastronginformationclassificationtomitigateprivacyrisks.

○DataleakageriskifLLMandsearcharenotinthesameinfrastructure.

USECASES

●UsingLLMstoanswerquestionsaboutaninternal,privatedocumentdatabase

●AugmentingLLMswithwebsearchresults

LLMWithDocument

RetrievalorSearch

LLMandRetrieval

Prompt

Response

Retrieval/

Search

Model

LLMAPI

Prompt+Context

User

Interface

Query

Top

Docs

Document

Database

PrepTheServices

UserPrompt

4

IndexyourinternaldocswithAzureCognitive

services

1

AzureCognitiveServices

ConvertedChatGPT

Prompt

2

AzureOpenAI

MasterPrompt

Service

SetupyourownPrivate

InstanceofChatGPTw/API

8

3

9

Howitworks

Interactwithservices

PrompttranslatedtoaQueryof

IndexedData

5

6

MasterPrompt

7

ListofDocument

snippetsfromprivate

dataindex

HiddenfromUser

GeneratedSummary

Source1Source2

Groundingw/sources

DESIGNPATTERNAPPLICATION

EmbedLLM“As-Is”Into

anApplicationFrame

●Name:EmbedLLM“as-is”intoanapplicationframe

●Description:ExposingLLMcapabilitiesviaanapplicationframethatmakesAPIcallstotheLLMonthebackend

●Motivation:TobettercontrolandsecureadoptionofLLMcapabilities

●Solution:ServicecalledviaAPIandresultspresentedinUIframeinsidehostapplication

●Implementation:Implementedasanon-demanddiscoveryorcontentgenerationtool(inessence,anewtoolinaframeawaitingapromptfromuser)

VALUE

●Benefits:

○TakesadvantageofthebetterprivacyandsecurityprotectionsincludedinAPIofferings(ascomparedtotheend-userapplications).

○Easiertomonitorcompliancebyrecordingusageviatheproprietaryuserinterface.

○APIsgivemoreflexibilityforcreatingcomplexworkflows(forexample,addingautomatedcontrolsbeforesendingdatatotheAPI).

●Drawbacks:

○Volumeofuseandpricing:APIcostsneedtobemonitored.

○PrivateinstancesofLLMscouldbeeventuallybeoffereddirectlyby

vendors,changingthecost-benefitofbuildingaprivateuserinterface.

USECASES

●EnablingemployeeaccesstoLLMsinacontrolledenvironment

●AlltheusecasesintheUsingLLMsAs-Isapplyhere:codegeneration,Ideageneration/brainstorming,copywriting/contentcreation,generalknowledgediscovery,basicNLPtasks

Request

App

Frame

AppUserInterface

APICall

PromptResponse

LLMExposedWithinanApplicationFrame

User

Response

LLM

API

Non-LLM

prompt

UI

DESIGNPATTERNAPPLICATION

EmbedLLMIntoanApplicationWorkflow

●Name:LLMembeddedinanapplicationworkflow

●Description:Embeddingas-isLLMaspartofabroaderapplicationworkflow.ThisdiffersfromtheApplicationFramepatterninthatthisisnotjustawayto

exposeLLMAPIs,butawaytointegratethemaspartofacomplexapplication

●DataConsiderations:PotentialinconsistencybetweentheLLMandthehostapplicationcontextanddata

●Motivation:ToexpandthefunctionalityofanapplicationwithLLMcapabilities

●Solution:LLMcalledviaAPIbyapplicationandresultsprocessedbytheapplication

●Implementation:Canbeimplementedintwoways:

○Asasecondarysourceofcontentproactivelyqueriedbyapplicationandpresentedtotheuser

○WheretheLLMoutputdrivesanotherprocessintheapplicationandmayormaynotpresentresultsintheUI

VALUE

●Benefits:

○Enhancesthefunctionalityofanapplication

●Drawbacks:

○UsagecontrolsneedtobeimplementedtokeepAPIcostsundercontrol

○UsingtheLLMasacomponentofanapplicationmightbetooriskyforsomeusecases,requiringcarefulguardraildesign

USECASES

●EmbeddingLLMsintoproductivitysoftwareorcollaborationtools

●PresentingLLMoutputsalongsideexistingsearchresults

●Embeddedintoacontentmanagementsystem

●ChatbotapplicationexpandingvirtualassistantnetworkwithLLMs

Aproprietaryapp

augmentedwith

LLMs

Prompt

LLM

API

Response

LLMinanApplicationWorkflow

User

AppWorkflowTrigger

ProcessLLMResponse

Application

AppUserInterface

DESIGNPATTERNAPPLICATION

LLMasaSecondaryChatbot

●Name:LLMasasecondaryconversationalagent

●Description:AconversationalsystemroutesrequeststoanexistingchatbotortheLLMAPI.Thishandovercouldalsobedonefromtheexistingchatbot.

●Motivation:

○ToaddabroadgeneralknowledgeexperiencetoaconversationalUI

○Toenableopen-endedconversations

●Solution:Therearetwobroadapproaches:

○WhereachatbotorchestrationfunctionroutesauserquerytoeitheranexistingchatbotortheLLMAPI

○WhereexistingchatbothaslowconfidenceandhandsqueryovertotheLLM

●Implementation:Theincumbentconversationalsystemisresponsiblefor

invokingtheLLMbasedoncontextorenablingthechatbotsinitsnetworktofallback/handovertotheLLMbasedonconfidencelevels(orsomeotherfactor).

VALUE

●Benefits:

○Extendtheconversationalcapabilitiesofanexistingchatbotecosystem

●Drawbacks:

○LowconsistencyinresponsebetweentheLLMandtheexistingchatbot

○Riskinlowaccuracy/hallucinationscomingfromtheLLMresponses

○ExternalchatbotsmayrequiresendingcustomerdataintotheLLMAPI,potentiallycreatingaprivacyrisk

USECASES

●Improvingcustomerservicechatbots

●Augmentingnonplayablecharactersinvideogames

User

Interface

Orchestration&

RoutingLogic

1

LLM

API

OtherChatbot

2

Response

Handling

Option1.Routetoappropriatebot

Option2.Handoverwhenchatbot

confidenceislow

Response

LLMasSecondary

Agent

Non-LLM

promptUI

ShareableSummary

Recommendations

AsanITleaderfocusedonleveraginggenerativeAItocreatebusinessvalue,youshould:

?Runaworkshoptogenerateuse-caseideaswiththebusiness,focusingonthedisruptivepotentialofgenerativeAIandthewayinwhichitcanenablestrategicobjectives.

?Prioritizetheusecasesforyourpilotagainsttheirpotentialbusinessvalueandfeasibility.FocusonnomorethanafewusecasesforyourgenerativeAIpilot.

?Assembleasmallbutdiverseteam,includingbusinesspartners,softwaredevelopersandAIexperts.Dedicatethisfusionteamforthedurationofthepilot.

?Createaminimumviableproducttovalidateeachusecase.Identifythetargetbusinesskeyperformanceindicator(KPI)improvementhypothesis,anddefinethedeploymentapproachesandriskmitigationsrequiredtoquicklytestthishypothesis.

ViewDocument

4?2023GartnerInc.and/oritsaffiliates.Allrightsreserved.

Mitigate

Hallucinations

?DocumentLimitations

?ModelMonitoring

CollaborateAcrossStakeholders

?SeekDiversity

?PublishLessonsLearned

InstillResponsibleAIPractices

PreventMisuse

?UsageGuidelines

?Enforcement

32

RESTRICTEDDISTRIBUTION

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

AI

Experts

AllowUserstoReportIssues

AI

Services

AIUsers

CreateaFeedbackLoop

Communicate

Limitations

RESTRICTEDDISTRIBUTION

33?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

ConductAdversarial

Testingvia“RedTeaming”

Insteadofdoingextensive

annotation,theredteamconductsadversarialtesting,activelyseekingoutexampleswhereitfails.

Themodelisretrainedonthese

examples,withtheteamadding

newadversarialexamples—

continuingthisprocessuntilthey

closethelooponfindingfailures.

34

RESTRICTEDDISTRIBUTION

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

UpskillingRecommendations

?AlignyourskillsdevelopmentwiththedeploymentpatternsforLLMsthatbestfitsyourorganizationsusecaseandmaturity.Formostorganizations,thiswillconsistoftheadoptionofCOTSapplicationsincorporatingLLMs,orLLMapplications

thatleverageretrievalaugmentedgeneration(RAG).

?Crossfunctionaltechnicalteamsshouldupskillpromptengineering,knowledgegraphandLLMOpsskills.CitizenDataScientistsshoulddevelopprompt

engineeringskills.

?Learnfromproductmanagementbestpracticesandspendmoretimeon

discoverybeforeyoujumpintodelivery.Figuringoutwhattobuild,howtobuildit,andhowtobringittomarket,evenwithhelpfromasmartAIcopilot,isstilla

highlychallengingactivity.

?Architects:focusonimprovingTeam,ProcessesandOrganizationdesignwithmethodssuchasAgile,TeamTopologiesandWardleymappingtoenablethevelocity,serviceorientationandadaptabilitytochangerequiredbyAIadoption.

RESTRICTEDDISTRIBUTION

35?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

PromptEngineeringMethods&Skills

TechnicalSkills

SoftSkills

Tools

Core

Prompt

Formulation/Chaining

Writing/CommunicationBusinessDomain

Knowledge

PromptManagement

Valuable

AdvancedPromptingMethods

Prompt

Monitoring/RelevanceScoring

Creativity

Reasoning

ProductSense

ThinkingEnd-To-EndCollaboration

Prompt

Engineering/PromptInfrastructure

Search/Indexing/VectorDatabases

Specialized

SemanticSearch

Knowledge

Engineering

AdversarialPromptingPromptOptimization

Architecture

UserEmpathy

DesignThinking

Persuasion

Automation/Workflow

Platforms

SymbolicAIPluginsDataLabeling

36

RESTRICTEDDISTRIBUTION

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

Domainspecific

orGeneral

Purpose

ModelSize&Benchmarks

EcosystemorBuildyouown

OrganizationalSupport

ModelUpdateoptions

Execution

performance&

latency

LargeLanguageModel(LLM)

RolesandSkills

Considerations

QualityofDataSources

ArchitectureTransparency

Deploymentoptions

RESTRICTEDDISTRIBUTION

37?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.

FutureofGenerativeAI

?Rawpotentialisenormous—morepowerfulandversatilemodels,butsafetyandveracityremainquestionable.

?Willberapidlyembeddedintoconsumerandbusinessapps.

?Modelsizeswillcontinuetoscalebutclientswillprioritizecost,simplicity,security,transparencyanddomainspecificity.

?Emergenceofnewbusinessmodels&ecosystems.

?Growthinmultimodalmodels.

?Theconcentrationofpowerthatthisphenomenonentailsanditseffectsaren’tfullyunderstoodtoday.

?EverythingclaimstobepoweredbyAI.

RESTRICTEDDISTRIBUTION

38?2023G