OWASP：人工智能安全解決方案概覽指南2025（英文）

updatedQuarterly

|GenAI,LLMSecOpsandSecuritySolutionLandscape

RevisionHistory

Revision

Date

Authors

Description

.01

6/4/2024

ScottClinton

InitialDraftICharter

.05

8/10/2024

ScottClintonI

ContributorsInputs

Updatedwithinitialfeedback

.06

10/15/2024

ScottClintonI

ContributorsIReviewerInputs

Re-factorSolutions

LandscapecategoriesI

1.0

10/15/2024

ContributorsIReviewers

FinalReleaseCandidate

TheinformationprovidedinthisdocumentdoesnotIandisnotintendedtoIconstitutelegaladvice.Allinformationisforgeneralinformationalpurposesonly.Thisdocumentcontainslinkstootherthird-partywebsites.SuchlinksareonlyforconvenienceIandOWASPdoesnotrecommend

orendorsethecontentsofthethird-partysites.

LicenseandUsage

ThisdocumentisIicensedunderCreativeCommons,CCBY-SA4.0Youarefreeto:

●Share—copyandredistributethematerialinanymediumorformat

●Adapt—remixItransformIandbuilduponthematerialforanypurposeIevencommercially.

●Underthefollowingterms:

oAttribution—YoumustgiveappropriatecreditIprovidealinktothelicenseIandindicateifchangesweremade.Youmaydosoinanyreasonablemannerbutnotinanywaythatsuggeststhelicensorendorsesyouoryouruse.

oAttributionGuidelines-mustincludetheprojectnameaswellasthenameoftheassetReferenced

■OWASPTop10forLLMs-LLMSecOpsSolutionsLandscape

■OWASPTop10forLLMs-CyberSecuritySolutionandLLMSecOpsLandscapeGuide

●ShareAlike—IfyouremixItransformIorbuilduponthematerialIyoumustdistributeyourcontributionsunderthesamelicenseastheoriginal.

Linktofulllicensetext:

/licenses/by–sa/4.0/legalcode

TheinformationprovidedinthisdocumentdoesnotIandisnotintendedtoIconstitutelegaladvice.Allinformationisfor

generalinformationalpurposesonly.Thisdocumentcontainslinkstootherthird-partywebsites.Suchlinksareonlyfor

convenienceandOWASPdoesnotrecommendorendorsethecontentsofthethird-partysites.

Version1.01of34

|GenAI,LLMSecOpsandSecuritySolutionLandscape

Contents

WhoIsThisDocumentFor? 3

Objectives 3

Scope 3

Introduction 4

DefiningtheSecuritySolutionsLandscape 4

LandscapeConsiderations 4

LLMApplicationCategories,SecurityChallenges 5

StaticPromptAugmentationAppIications 6

AgenticAppIications 7

LLMPIug-ins,Extensions 8

CompIexAppIications 9

LLMDevelopmentandConsumptionModels 10

LLMOpsandLLMSecOpsDefined 11

AQuickOpsPrimer-FoundationforLLMOps 11

LLMOpsLifeCYcIeStages-FoundationforLLMDevSecOps 12

Scoping/PIanning 13

DataAugmentationandFine-Tuning 14

AppIicationDeveIopmentandExperimentation 14

TestandEvaIuation 15

ReIease 15

DepIoY 16

Operate 16

Monitor 17

Govern 18

MappingtotheOWASPTop10forLLMThreatModeI 18

AppIicationServices 19

ProductionServices 19

OWASPTop10forLLMsSolutionsLandscape 20

EmergingGenAI/LLM-SpecificSecuritYSoIutions 21

LLM&GenerativeAISecuritYSoIutions 22

SoIutionLandscapeMatrixDefinitions 22

LandscapeSoIutionMatrix 23

Acknowledgements 29

OWASPTop10forLLMProjectSponsors 30

References 31

ProjectSupporters 32

Version1.02of34

|GenAI,LLMSecOpsandSecuritySolutionLandscape

WhoIsThisDocumentFor?

ThisdocumentistailoredforadiverseaudiencecomprisingdevelopersIAppSecprofessionalsIDevSecOpsandMLSecOpsteamsIdataengineersIdatascientistsICISOsIandsecurityleaderswhoarefocusedondevelopingstrategiestosecureLargeLanguageModels(LLMs)andGenerativeAIapplications.ItprovidesareferenceguideofthesolutionsavailabletoaidinsecuringLLMapplicationsIequippingthemwiththeknowledgeandtoolsnecessarytobuildrobustIsecureAIapplications.

Objectives

ThisdocumentisintendedtobeacompaniontotheOWASPTop10forLargeLanguageModel(LLM)ApplicationsListandtheCISOCybersecurity&GovernanceChecklist.Itsprimaryobjectiveistoprovideareferenceresourcefororganizationsseekingtoaddresstheidenti?edrisksandenhancetheirsecurityprograms.Whilenotdesignedtobeanall-inclusiveresourceIthisdocumentoffersaresearchedpointofviewbasedonthetopsecuritycategoriesandemergingthreatareas.Itcapturesthemostimpactfulexistingandemergingcategories.BycategorizingIde?ningIandaligningapplicabletechnologysolutionareaswiththeemergingLLMandgenerativeAIthreatlandscapeIthisdocumentaimstosimplifyresearcheffortsandserveasasolutionsreferenceguide.

Scope

Thescopeofthisdocumentistocreateasharedde?nitionofsolutioncategoryareasthataddressthesecurityoftheLLMandgenerativeAIlifecycleIfromdevelopmenttodeploymentandusage.ThisalignmentsupportstheOWASPTop10ListForLLMsoutcomesandtheCISOCybersecurityandGovernanceChecklist.ToachievethisIthedocumentwillcreateaninitialframeworkandcategorydescriptorsIutilizingbothopen-sourcesolutionsandprovidingmechanismsforsolutionproviderstoaligntheirofferingswithspeci?ccoverageareasasexamplestosupporteachcategory.

Thedocumentadherestoseveralkeyrulestomaintainitsintegrityandusefulness:

●Vendor-AgnosticandOpenApproach:ItmaintainsaneutralstanceIavoidingrecommendationsofonetechnologyoveranotherIinsteadprovidingcategoryguidancewithchoicesandoptions.

●Straightforward,ActionableGuidance:ThedocumentoffersclearIactionableadvicethatorganizationscanreadilyimplement.

●CoordinatedKnowledgeGraph:ItincludescoordinatedtermsIde?nitionsIanddescriptionsforkeyconcepts.

●PointtoExistingStandards:WhereexistingstandardsorsourcesoftruthareavailableIthedocumentreferencestheseinsteadofcreatingnewsourcesIensuringconsistencyandreliability.

Version1.03of34

|GenAI,LLMSecOpsandSecuritySolutionLandscape

Introduction

WiththegrowthofGenerativeAIadoption,usage,andappIicationdeveIopmentcomesnewrisksthataffecthoworganizationsstrategizeandinvest.AstheserisksevoIve,sodoriskmitigationsoIutions,technoIogies,frameworks,andtaxonomies.ToaidsecurityIeadersinprioritization,conversationsaboutemergingtechnoIogyandsoIutionareasmustbeaIignedappropriateIytocIearIyunderstoodbusinessoutcomesforAIsecuritysoIutions.ThebusinessoutcomesofAIsecuritysolutionsmustbeproperlyde?nedtoaidsecurityleadersinbudgeting

ManyorganizationshavealreadyinvestedheavilyinvarioussecuritytoolsIsuchasvulnerabilitymanagementsystemsIidentityandaccessmanagement(IAM)solutionsIendpointsecurityIDynamicApplicationSecurityTesting(DAST)IobservabilityplatformsIandsecureCI/CD(ContinuousIntegration/ContinuousDeployment)toolsItonameafew.HoweverIthesetraditionalsecuritytoolsmaynotbesu代cienttofullyaddressthecomplexitiesofAIapplicationsIleadingtogapsinprotectionthatmaliciousactorscanexploit.ForexampleItraditionalsecuritytoolsmaynotsu代cientlyaddresstheuniquedatasecurityandsensitiveinformationdisclosureprotectioninthecontextofLLMandGenAIapplications.ThisincludesbutisnotlimitedtothechallengesofsecuringsensitivedatawithinpromptsIoutputsIandmodeltrainingdataIandthespeci?cmitigationstrategiessuchasencryptionIredactionIandaccesscontrolmechanisms.

EmergentsolutionslikeLLMFirewallsIAI-speci?cthreatdetectionsystemsIsecuremodeldeploymentplatformsIandAIgovernanceframeworksattempttoaddresstheuniquesecurityneedsofAI/MLapplications.HoweverItherapidevolutionofAI/MLtechnologyanditsapplicationshasdrivenanexplosionofsolutionapproachesIwhichhasonlyaddedtotheconfusionfacedbyorganizationsindeterminingwheretoallocatetheirsecuritybudgets.

DefiningtheSecuritySolutionsLandscape

TherehavebeenmanyapproachestocharacterizingthesolutionslandscapeforLargeLanguageModeltoolsandinfrastructure.InordertodevelopasolutionslandscapethatfocusesonthesecurityofLLMapplicationsacrossthelifecyclefromplanningIdevelopmentIdeploymentIandoperationItherearefourkeyareasofinputwehavefocusedontodevelopbothade?nitionforLargeLanguageModelDevSecOPsandrelatedsolutionslandscapecategories.

LandscapeConsiderations

ApplicationTypesandScope-whichimpactsthepeopleIprocessesIandtoolsneededbasedonthecomplexityoftheapplicationandtheLLMenvironmentIas-a-serviceIself-hostedIorcustom-built.

EmergingLLMSecOpsProcess-whilethisisaworkinprogressImanyarelookingtoadaptandadoptexistingDevOpsandMLOpsandassociatedsecuritypractices.Weexpectourde?nitiontoevolveasthedevelopmentprocessesforLLMapplicationsbegintomature.

Version1.04of34

|GenAI,LLMSecOpsandSecuritySolutionLandscape

ThreatandRiskModeling-understandingtherisksposedbyLLMsystemsIapplicationusageIormisuselikethoseoutlinedintheOWASPTop10forLLMsandGenerativeAIApplicationsIarekeytounderstandingwhichsolutionsarebestsuitedtoimprovethesecuritypostureandcombatarangeofattacks.

TrackingEmergingSolutions-manyexistingsecuritysolutionsareadaptingtosupportLLMdevelopmentwork?owsandusecaseshowevergiventhenatureofnewthreatsandevolvingtechnologyandarchitecturesnewtypesofLLM-speci?csecuritysolutionswillbenecessary.

LLMApplicationCategories,SecurityChallenges

OrganizationshavebeenleveragingMachineLearninginapplicationsfordecades.ThisoftenrequireddetailedexpertiseinDataScienceandextensivemodeltraining.GenerativeAIhaschangedthis.Speci?callyILargeLanguageModels(LLMs)havemademachinelearningtechnologywidelyaccessible.Theabilitytodynamicallyinteractinplainlanguagehasopenedthedoorforthecreationofanewclassofdata-drivenapplicationsandapplicationintegrations.FurthermoreIusageisnolongerlimitedtothehighlyskilledeffortsoftraditionaldevelopersanddatascientists.Pre-trainedmodelsenablenearlyanyonetoperformcomplexcomputationaltasksIregardlessofpriorexposuretoprogrammingorsecurity.OrganizationshavebeenleveragingMachineLearninginapplicationsfordecadesincludingNaturalLanguageProcessing(NLP)modelsthatoftenrequiredetailedexpertiseinDataScienceandextensivemodeltraining.

Withtheadventoftransformerstechnologyenablinggenerativecapabilitiescombinedwiththeeaseofaccessforpre-trainedas-a-servicemodelslikeChatGPTandotheras-a-serviceIFourmajorcategoriesofLLMApplicationArchitectureemerged;Prompt-centricIAIAgentsIPlug-ins/extensionsIandcomplexgenerativeAIapplicationwheretheLLMplaysakeyroleinalargerapplicationusecase.

(?gure:ApplicationCategories&SummaryAttributes)

HavingacommonviewoftypicalLLMapplicationarchitecturesIincludingagentsImodelsILLMsIandtheMLapplicationstackIiscrucialforde?ningandaligningtheapplicationstackIsecuritymodelIandapplicationofferings.BelowIwehaveprovidedashortdescriptionofkeycharacteristicsIusecasesIandsecuritychallengesforeachapplicationcategory.

Version1.05of34

|GenAI,LLMSecOpsandSecuritySolutionLandscape

StaticPromptAugmentationApplications

Theseapplicationsinvolvespeci?cstaticnaturaIIanguageinputstoguidethebehaviorofa

largelanguagemodel(LLM)towardgeneratingthedesiredoutput.Thistechniqueoptimizestheinteractionbetweentheuserandthemodelby?ne-tuningthephrasingIcontextIandinstructionsgiventotheLLM.Theseapplicationsallowuserstoaccomplishawiderangeoftasksbysimply

re?ninghowtheyaskquestionsorprovideinstructions.

KeyCharacteristics

oHumantomodel/modeltohumaninteractionandresponse

oStaticpromptaugmentation

oFlexibilityandCreativity

oSimplicityandAccessibility

oRapidPrototypingandExperimentation

UseCaseExamples

oExperimentation/RapidPrototyping

oContentGenerationTools

oTextSummarizationApplications

oQuestion-AnsweringSystems

oLanguageTranslationTools

oChatbotsandVirtualAssistants

SecurityChallenges

oPrompt-basedapplicationsfacesecurityriskslikepromptinjectionattacksand

dataleakagefrompoorlycraftedprompts.Lackofcontextorstatemanagement

canleadtounintendedoutputsIincreasingmisusevulnerability.User-generated

promptsmaycauseinconsistentorbiasedresponsesIriskingcomplianceorethicalviolations.EnsuringpromptintegrityIrobustinputvalidationIandsecuringtheLLMenvironmentarecrucialtomitigatetheserisks.

Version1.06of34

|GenAI,LLMSecOpsandSecuritySolutionLandscape

AgenticApplications

TheseapplicationsleverageLargeLanguageModels(LLMs)toautonomouslyorsemi-autonomouslyperformtasksImakedecisionsIandinteractwithusersorothersystems.TheseagentsaredesignedtoactonbehalfofusersIhandlingcomplexprocessesthatofteninvolvemultiplestepsIintegrationsIandreal-timedecision-making.TheyoperatewithalevelofautonomyIallowingthemtocompletetaskswithoutconstanthumanintervention.

KeyCharacteristics

oAutonomyandDecision-Making

oInteractionwithExternalSystems

oStateManagementandMemory

oComplexWork?owAutomation

oHuman-AgentCollaboration

UseCaseExamples

oVirtualAssistants

oCustomerSupportBots

oProcessAutomationAgents

oDataAnalysisandReportingAgents

oIntelligentPersonalizationAgents

oSecurityandComplianceAgents

SecurityChallenges

oAgentapplicationsIwiththeirautonomyandaccesstovarioussystemsImustbecarefullysecuredtopreventmisuse.Theyfacesecuritychallengeslike

unauthorizedaccessIincreasedexploitationrisksduetointeractionwithmultiplesystemsIandvulnerabilitiesindecision-makingprocesses.Ifsomeonegains

controlofanautonomousagent,theconsequencescouldbesevere,especiallyincriticalsystems.Ensuringrobustaccesscontrolsandencryptionmethodsto

protectagainstthisisessential.Ensuringdataintegrityandcon?dentialityis

criticalIasagentsoftenhandlesensitiveinformationitisimportanttosecuredataatallstagesIincludingat-restIinmotionIandaccessthroughsecuredAPIs.Theirautonomyalsoposesrisksofunintendedorharmfuldecisionswithoutoversight.RobustauthenticationIencryptionImonitoringIandfail-safemechanismsare

essentialtomitigatethesesecurityrisks.ObservabilityandTraceabilitysolutionsthatmonitortheentirelifecycleoftheAgents(DesignIDevelopmentIDeploymentIandVisibilityondecision-making)mustbeconsideredtoensurereal-time

correctionsusingahumans-in-the-loopprocesscanbeenforced.

Version1.07of34

|GenAI,LLMSecOpsandSecuritySolutionLandscape

LLMPlug-ins,Extensions

Plug-insareextensionsoradd-onsthatintegrateLLMsintoexistingapplicationsorplatformsIenablingthemtoprovideenhancedornewfunctionalities.Plug-instypicallyserveasabridgebetweentheLLMandtheapplicationIfacilitatingseamlessintegrationIsuchasaddingalanguagemodeltoawordprocessorforgrammarcorrectionorintegratingwithcustomerrelationshipmanagement(CRM)systemsforautomatedemailresponses.

Whileitcanbesometimesdi代culttodrawthelinebetweenAgentsandplug-insorextensionswhichareoftencomponentsoflargerapplicationsIonemeasureisthewayitisdeployedandused.ForexampleIaplug-inwouldbeapre-builtagendesignedforreusethatyoucallexplicitlyIthroughanAPIIoraspartofanLLMspluginorextensionframeworkvs.customcoderunninginthebackgroundonaperiodicbasis.

KeyCharacteristics

oModularityandFlexibility

oSeamlessIntegration

oTaskSpeci?cFocus

oEaseofDeploymentandUse

oRapidUpdatesandMaintenance

UseCaseExamples

oContentGenerationTools

oTextSummarizationApplications

SecurityChallenges

oPluginsinteractingwithsensitivedataorcriticalsystemsmustbecarefullyvettedforsecurityvulnerabilities.Poorlydesignedormaliciouspluginscancausedatabreachesorunauthorizedaccess.LLMpluginsfacechallengeslikecompatibilityissuesIwhereupdatescanintroducevulnerabilitiesIandintegrationwithsensitivesystemsincreasestheriskofdataleaks.EnsuringsecureAPIinteractionsIregularupdatesIandrobustaccesscontrolsiscrucial.Resource-intensivepluginsmaydegradeperformanceIriskingexploitation.

o

Version1.08of34

|GenAI,LLMSecOpsandSecuritySolutionLandscape

ComplexApplications

ComplexapplicationsaresophisticatedsoftwaresystemsthatdeeplyintegrateLargeLanguageModels(LLMs)asacentralcomponenttoprovideadvancedfunctionalitiesandsolutions.TheseapplicationsarecharacterizedbytheircomprehensivescopeIscalabilityIandtheintegrationofmultipletechnologiesandcomponents.TheyaretypicallydesignedtosolveintricateproblemsIofteninenterpriseenvironmentsIandrequireextensivedevelopmentIengineeringIandongoingmaintenanceefforts.

KeyCharacteristics

oMulti-componentarchitecturesaredesignedtoprocesspromptsfromothernon-humansystems.

oOftenusemultipleintegrationsIincludingothermodels.

oMulti-ComponentArchitecture

oScalabilityandPerformance

oAdvancedFeaturesandCustomization

oEnd-to-EndWork?owAutomation

UseCaseExamples

oLegalDocumentAnalysisPlatforms

oAutomatedFinancialReportingSystems

oCustomerServicePlatforms

oHealthcareDiagnostics

SecurityChallenges

oComplexLLMapplicationsfacemajorsecuritychallengesduetotheirintegrationwithmultiplesystemsandextensivedatahandling.TheseincludeAPIvulnerabilitiesIdatabreachesIandadversarialattacks.Thecomplexityincreasestheriskofmiscon?gurationsIleadingtounauthorizedaccessordataleaks.Managingcomplianceacrosscomponentsisalsodi代cult.RobustencryptionIaccesscontrolsIregularsecurityauditsIandcomprehensivemonitoringareessentialtoprotecttheseapplicationsfromsophisticatedthreatsandensuredatasecurity.

Version1.09of34

|GenAI,LLMSecOpsandSecuritySolutionLandscape

LLMDevelopmentandConsumptionModels

Oneofthe?rstconsiderationsforanorganizationisdecidingupontheapproachtoleveragingLLMcapabilitiesbasedonthetypeofapplicationandgoalsfortheproject.TodayIdevelopershaveachoiceoftwoprimarydeploymentmodelswhenimplementingLLM-basedapplicationsandsystems.

CreateaNewModel:ThetrainingprocessforcustomLLMsisintensiveIofteninvolvingdomain-speci?cdatasetsandextensive?ne-tuningtoachievedesiredperformancelevels.ThisapproachismoreakintoMLOpsbuildingMLmodelsfromthegroundupIwithdetaileddataanalysisIcollectionformattingIcleaningIandlabeling.Oneofthebene?tsofthisapproachisthatyouknowthelineageandsourceofthedatathemodelisbuiltonandcanattestdirectlytoitsvalidityand?t.HoweverIamajordownsideistheresourcesIcostIandexpertisenecessarytobuildItrainIandverifyamodelthatmeetstheprojectobjectives.CustomLLMsprovidetailoredsolutionsoptimizedforspeci?ctasksanddomainsIofferinghigheraccuracyandalignmentwithanorganization'sspeci?cneeds.

ConsumeandCustomizeExistingModels:Pre-trained(foundation)modelsIwhetherself-hostedorofferedasaserviceIsuchaswithChatGPTIBertandothersontheotherhandprovideamoreaccessibleentrypointfororganizations.ThesemodelscanbequicklydeployedviaAPIsIallowingforrapidsolutionvalidationandintegrationintoexistingsystems.TheLLMOpsprocessinthisscenarioemphasizescustomizationthrough?ne-tuningwithspeci?cdatasetsIensuringthemodelmeetstheapplication'suniquerequirementsIfollowedbyrobustdeploymentandmonitoringtomaintainperformanceandsecurity.

Version1.010of34

|GenAI,LLMSecOpsandSecuritySolutionLandscape

LLMOpsandLLMSecOpsDefined

HavingacommonviewoftypicalLLMapplicationarchitecturesIincludingagentsImodelsILLMsIandtheMLapplicationstackIiscrucialforde?ningandaligningtheapplicationstackandsecuritymodel.

(?gure:LLMOpsrelatedOperationsProcessforDataIMachineLearningandDevOps)

AQuickOPsPrimer-FoundationforLLMOPs

DevOpsIwhichemphasizescollaborationIautomationIandcontinuousintegrationanddeployment(CI/CD)Ihaslaidthegroundworkfore代cientsoftwaredevelopmentandoperations.BystreamliningthesoftwaredevelopmentlifecycleIDevOpsenablesrapidandreliabledeliveryofapplicationsIfosteringacultureofcollaborationbetweendevelopmentandoperationsteams.

DataOpsbuildsonDevOpsIwheredatapipelinesaremanagedwithsimilarautomationIversioncontrolIandcontinuousmonitoringIensuringdataqualityandcomplianceacrossthedatalifecycle.MLOpsalsoextendstheDevOpsprinciplestomachinelearningIfocusingontheuniquechallengesofmodeldevelopmentItrainingIdeploymentIandmonitoring.UtilizingDevOpsasafoundationensuresthatbothDataOpsandMLOpsinheritarobustinfrastructurethatprioritizese代ciencyIscalabilityIsecurityIandfasterinnovationindata-drivenandmachinelearningapplications.

MLOpsandDataOpsarefoundationaltoLLMOpsbecausetheyestablishthecriticalprocessesandinfrastructureneededformanagingthelifecycleoflargelanguagemodels(LLMs).DataOpsensuresthatdatapipelinesaree代cientlymanagedIfromdatacollectionandpreparationtostorageandretrievalIprovidinghigh-qualityIconsistentIandsecuredatathatLLMsrelyonfortrainingandinference.MLOpsextendstheseprinciplesbyautomatingandorchestratingthemachinelearninglifecycleIincludingmodeldevelopmentItrainingIdeploymentIandmonitoring.

Version1.011of34

|GenAI,LLMSecOpsandSecuritySolutionLandscape

LLMOpsandMLOpsIwhilerootedinthesamefoundationalprinciplesoflifecyclemanagementIdivergesigni?cantlyintheirfocusandrequirementsduetothespeci?cdemandsoflargelanguagemodels(LLMs).LLMOpsencompassesthecomplexitiesoftrainingIdeployingIandmanagingLLMsIwhichrequiresubstantialcomputationalresourcesandsophisticatedhandling.LLMOpsensurethatLLMsaree代cientlyintegratedintoproductionenvironmentsImonitoredforperformanceandbiasesIandupdatedasneededtomaintaintheireffectiveness.ThisholisticapproachensuresthatthedeploymentandoperationofLLMsarestreamlinedIscalableIandsecureIincludingconsiderationsfordatavalidationandprovenancetoensurethatthedatausedfortrainingand?ne-tuningLLMsistrustworthyandfreefromtampering.Thiscanincludetechniquesfordataauditingandveri?cation.

LLMOPsLifeCycleStages-FoundationforLLMDevSecOPs

AsmentionedearlierinthisdocumentItoalignsecuritysolutionsforLLMapplicationsforoursolutionguideweareusingtheLLMOpsprocesstode?nethesolutioncategoriessothattheyalignwiththechallengesdevelopersarefacingindevelopinganddeployingLLM-basedapplications.

(?gure:CombinedLLMCustomandLLMPre-TrainedImage)

TheLLMOpsprocessesdiffersigni?cantlybetweenusingpre-trainedLLMmodelsforapplicationdevelopmentandcreatingcustomLLMmodelsfromscratchusingopen-sourceandcustomdatasetsIwhichinheritmorefromMLOpspracticeswithsomeadditions.We?rstneedtode?nethestagesIthetypicaldevelopertasksIandthesecuritystepsateachstageofthelifecycle.

Version1.012of34

|GenAI,LLMSecOpsandSecuritySolutionLandscape

(?gure:LLMopsPre-TrainedProcessandSteps)

Thesephaseswehavede?nedinclude:Scope/PlanIModelFine-Tuning/DataAugmentationITest/EvaluateIReleaseIDeployIOperateIMonitorIandGovern.OfcourseIthisisaniterativeapproachIwhetheryouarepracticingwaterfallIagileIorahybridapproacheachofthesestepscanbeleveraged.

Scoping/Planning

Thefocusisonde?ningtheapplication'sgoalsIunderstandingthespeci?cneedstheLLMwilladdressIanddetermininghowthepre-trainedmodelwillbeintegratedintothelargersystem.ThisstageinvolvesgatheringrequirementsIassessingpotentialethicalandcomplianceconsiderationsIandsettingclearobjectivesforperformanceIscalabilityIanduserinteraction.TheoutcomeisadetailedprojectplanthatoutlinesthescopeIresourcesIandtimelinesneededtoimplementtheLLM-poweredapplicationsuccessfully.

TypicalActivities:

LLMOps

LLMSecOps

●

DataSuitability

●

AccessControlandAuthentication

●

ModelSelection

Planning

●

Requirem