Gartner：如何選擇正確的設(shè)計和技術(shù)來削弱人工智能（英文版）

Workshop:

HowtoSelecttheRightDesignandTechnologiestoDeriskAI

AndrewWallsAvivahLitan

?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.GartnerisaregisteredtrademarkofGartner,Inc.anditsaffiliates.Thispublicationmaynotbereproducedordistributedinanyform

withoutGartner'spriorwrittenpermission.ItconsistsoftheopinionsofGartner'sresearchorganization,whichshouldnotbeconstruedasstatementsoffact.Whiletheinformationcontainedinthispublicationhasbeenobtainedfromsourcesbelievedtobereliable,Gartnerdisclaimsallwarrantiesastotheaccuracy,completenessoradequacyofsuchinformation.AlthoughGartnerresearchmayaddresslegalandfinancialissues,Gartnerdoesnotprovidelegalorinvestmentadviceanditsresearchshouldnotbeconstruedorusedassuch.Youraccessanduseofthispublicationaregovernedby

Gartner’sUsagePolicy

.Gartnerpridesitselfonitsreputationforindependenceandobjectivity.Itsresearchisproducedindependentlybyitsresearchorganizationwithoutinputor

influencefromanythirdparty.Forfurtherinformation,see"

GuidingPrinciplesonIndependenceandObjectivity

."

TheWorkshop’sGoal

?We’reheretogetyouthinkingaboutthegovernance,designandtechnologyconsiderationstoderiskAIsolutions.

?You’llidentify:

—ThedifferenttypesofrisksassociatedwithAI.

—Mitigationapproaches—people,processesandtechnologies.—ApplythederiskingapproachtothreedifferentAIusecases.

?Willsplitintogroupsforexercisesandregrouptosharefindings.

2?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.GartnerisaregisteredtrademarkofGartner,Inc.anditsaffiliates.

AgendaandTiming

FullRoom

TableGroups

Timing

Introduction—Goal

3Minutes

DeriskingAIFramework

5Minutes

HowtoUseLabMaterials

5Minutes

LabExerciseNo.1

15Minutes

FindingsDiscussion

10Minutes

LabExerciseNo.2

15Minutes

FindingsDiscussion

10Minutes

LabExerciseNo.3

15Minutes

FindingsDiscussion

10Minutes

Closing

2Minutes

3?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.GartnerisaregisteredtrademarkofGartner,Inc.anditsaffiliates.

Artificialintelligence(AI)appliesadvancedanalysisandlogic-basedtechniques,including

machinelearning,tointerpretevents,supportandautomatedecisions,andtotakeactions.

4?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.GartnerisaregisteredtrademarkofGartner,Inc.anditsaffiliates.

RiskCategories

Standard

Compliance

?Regulatory

?Contract

Performance

Reputation

?Brand

?Failure

Operational

?SolutionOversight

?OmbudsmanProcess

Safety

?HarmtoHumans

?DamagetoProperty

Financial

?Fines

?LostRevenue

AI-Specific

Bias&Fairness

?SelectionBias

?Information

Privacy

?DataProtection

?Identity

Model&DataDrift

?UnanticipatedData

?UnanticipatedEvents

Explainability

?Transparency

?“BlackBox”

Security

?Sabotage

?DataPoisoning

5?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.GartnerisaregisteredtrademarkofGartner,Inc.anditsaffiliates.

Gartner’sAITRiSMFramework

AITrust,RiskandSecurityManagementPillars

Source:Gartner750738_C

6?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.GartnerisaregisteredtrademarkofGartner,Inc.anditsaffiliates.

AIDeriskingApproach

UseCase:

?Whatproblemarewesolving?

?Isthebenefitworththepotentialcost/risks?Isitfeasible?Isitethical?

Environment:

?WilltheAImakeautomateddecisions?Orwillahumanbeintheloop?

?WilltheAIneedtoworkinday/nightconditionsorpoorweather?Orremoteplaces(space,ocean)?

Risks:

?WhatarethedifferentpotentialrisksoftheAIsolution?Reputation,legal,compliance,financial,technology,securityorhumanimpact(physical/psychological)?

?Didweconsultwithrepresentativesofinvolvedandaffectedgroups(users,citizens,legal,etc.)?

AITechnologies:

?WhatAItechnologiesarerequired?Whatpotentialrisksdotheypose?E.g.,facialrecognition?

Mitigation/Controls:

?Whatarethedifferentmechanismstomitigaterisks?People,process,technology?

7?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.GartnerisaregisteredtrademarkofGartner,Inc.anditsaffiliates.

LabExercises

?Breakintogroups;eachgroupwillworkonallthreeusecases:—ObjectiveistofillintheAIderiskinganddesigntemplates.

—Willhave15minutestoworkoneachusecase;willregroupintolargerteamtosharefindingsfor10minutesbetweeneachusecase.

?Labhandouts—onepacketforeachusecase:

—Use-caseoverview

—AIderiskingapproach

—AIderiskingdesigntemplate—AIcategories

—AITRiSMframework

8?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.GartnerisaregisteredtrademarkofGartner,Inc.anditsaffiliates.

WhatWeWillDoinThisWorkshop:

IdentifytheRisksandMitigationApproachesforThreeAIUseCases

AI-Powered

CodeDev

Tool

UseCaseNo.1

AI-Powered

Recruiting

Tool

UseCaseNo.2

AI-PoweredBoss

UseCaseNo.3

9?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.GartnerisaregisteredtrademarkofGartner,Inc.anditsaffiliates.

UseCaseNo.1—AI-PoweredCodeGeneration

?Generatingapplicationsistime-

consumingandrequiresspecializedskills.

?WouldliketouseAIto:

—Generatecodefornewapplications.

—Generatecodetocustomizeexistingapplications.

—Reviewhuman-generatedcodeforareasrequiringimprovementandgeneratepatches.

?AndusegenerativeAItowritejobdescriptionsfordevelopers.

10?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.GartnerisaregisteredtrademarkofGartner,Inc.anditsaffiliates.

UseCaseNo.2—AI-PoweredRecruitingTool

?Findingnewtalentistime-consuming—scanningresumesandinterviewing

candidates.

?WouldliketouseAIto:

—Automaticallystripsprotected

characteristics(race,nationality,religion,ethnicity,age)

tonegatebiasrecruitmentprocess.—Performinitialcandidateinterviews

usingvirtualHRrepresentatives.

—Determinerolesuitabilitybyanalyzingcandidatevoiceandvideoduring

interviews.

11?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.GartnerisaregisteredtrademarkofGartner,Inc.anditsaffiliates.

UseCaseNo.3—AI-PoweredBoss

?Middlemanagementoftenaddslittlevaluetothebottomlineofacompanybutdifficultforseniorexecutivesto

directlysupervisehundredsofemployees.

?WouldliketouseAIto:

—Replacehumanmanagerswithavirtualboss.

—Willinteractwithemployeestosetandtrackgoals,willmonitorwork

performanceandprovidefeedback.

12?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.GartnerisaregisteredtrademarkofGartner,Inc.anditsaffiliates.

ExampleUseCase—AIJaywalkingCatcher

?Studieshaveshownthatmany

pedestrianaccidentsmaybecausedbyfailureofthepedestriantofollowtrafficlaws.

?WouldliketouseAIto:

—Detectjaywalking,capturetheirfacesandautomaticallynotifyandfinethem.

—Detectnon-jaywalking,capturetheirfacesandautomaticallysend

notificationandreward.

13?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.GartnerisaregisteredtrademarkofGartner,Inc.anditsaffiliates.

AIDe-riskingApproach:Example

UseCase:AIJaywalkingCatcher

Environment:

Outdoorcamerasoperatingunderdifferentlightingconditionsthatlikelywouldincludeinclementweathersuchasrain,snow,fog,smog.Couldbelotsofpeopleandtraffic

onroadsandsidewalks.

TypesofRisks:

Inabilitytoaccuratelymatchjaywalkertophotoofpersonindatabase,biasesbased

onskincolor,citizenstrickingsystembywearingmasksoffamouspeople,costs

ofhandlingcomplaintsandpotentiallawsuits,unauthorizedaccesstophotodatabase.

AITechnologies:

Facialrecognition,computervisiontodetectjaywalkingvswalkingincrosswalk.

Mitigation/Controls:

Trainingdatasetrepresentativeofdifferentraces,skincolors,genders,ages;

humanreinforcementlearningandhumanconfirmationbeforesendingletterwithfine.

14?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.GartnerisaregisteredtrademarkofGartner,Inc.anditsaffiliates.

AILifeCycleRiskandMitigation:Example

DataSelection

ModelBuild

ModelMonitor

Environment

TrainModel,TestandDeploy

Risks

●Inaccuracyinconfirmingpersoninimage/video.

●Inabilitytodeterminewhetherpersonwasinside/outside

crosswalk.

Risks

●Systemperformswellin

labenvironmentbutnotinrealworld.

●Systemnoteconomicaltoscaletobeused

broadly.

Risks

●Modelaccuracydegrades.

●Sourcefacedatabasenotmaintained.

●Toomanyfalsepositives.

●Requirestoomuch

resourcestomaintain.

Risks

●Camerasareexposedtoweather.

●Cameravandalism.

●Signal

●distortion/interception.

●Camerarecalibration.

●Citizencomplaints.

Insufficientnighttimelightingforcameras.

Risks

●Biased/incompletetrainingdatasets.

●Improperdatastorageleadingtoprivacyandsecurityrisks.

Mitigation

●Useofprebuiltfacialrecognitionmodelwith

●99+%accuracy.

crosswalkandwalklightnotgreen.

●Addbusinessrules

requiringhumanconfirmation.

Detecthumanoutside

Mitigation

●Useofsimulationtotestsystemacrossdifferentspectrumofpeople

(races,genders,etc.)

●andtrafficscenarios.

withcamerapositionedonstreetwithhigh

reportsofjaywalking.

TestHW/SWsystem

Mitigation

●Exploreautomatedmodel

●retraining/replacement.

●withfacialinfo.

andprivacyprotected.

●Cost-benefitanalysis

beforeexpandedrollout.

Publicdocumentationofhowsystemoperates

Identifyotherdatabases

Mitigation

●Real-worldtestingof

systembeforesending●finestocitizens.

verification/confirmationbeforesendingletter

withfines.

●Communicationto

citizensnotifying

purposeofcameras.

Human

Mitigation

●Varietyofdatafromdifferentraces,skintones,genders,ages.

●Usesyntheticdatatofillinincompletedataandprovidedifferent

angles/lightingofpeopleandroads,crosswalks.

15?2023Gartner,Inc.and/oritsaffiliates.Allrightsreserved.GartnerisaregisteredtrademarkofGartner,Inc.anditsaffiliates.

Recommendations

oFocusonAIusecasesthatprovidethebestROI:

thehighestcombinationofbusinessvalueandfeasibility.

oDevelopyourAI-enabledtool/processusingariskassessmentframework(e.g.,NISTAIRMF).

oIncludeadiversesetofreviewersattheprojectgateways:

e.g.,technology,bu