算法穩(wěn)定性：人工智能如何塑造未來(lái)威懾

ONFUTUREWAR

JUNE2024

AlgorithmicStability

HowAICouldShapetheFutureofDeterrence

ByBenjaminJensen,YasirAtalan,andJoseM.MaciasIII

Inthefuture…

?Stateswillintegrateartificialintelligenceandmachinelearning(AI/ML)intotheir

nationalsecurityenterprisestogaindecisionadvantagesovertheirrivals.Thequestionwillnotbeifawebofalgorithmsextendsacrossthemilitary,intelligencecommunity,and

foreignpolicydecisionmakinginstitutions,buthowlinesofcodeinteractwiththehumanclashofwillsattheheartofstrategy.

?Newtechnologywillchangethecharacterbutnotthenatureofstatecraftandstrategy.

Stateswillstillcombinediplomacy,economiccoercion,andinfluencecampaignswith

threatsofmilitaryforcetosignalrivalsandreassureallies.Humandecisionmaking,whileaugmentedbyalgorithms,remainscentraltostrategyformationandcrisismanagement.

?InformationaboutAI/MLcapabilitieswillinfluencehowstatesmanageescalation.

Escalationriskswillcontinuetoemergefromhowwarfightingchangesthebalanceof

informationavailabletoupdatemodelsandsupporthumandecisionmaking.IntelligencegapsonadversaryalgorithmsincreasethelikelihoodofescalationbutonlyoncestateshavecrossedtheRubiconandfightbeneaththenuclearthreshold.

Introduction

HowwilltheadoptionofAI/MLacrossastate’snationalsecurityenterpriseaffectcrisis

decisionmaking?Forexample,whatwouldtheCubanMissileCrisislooklikeatmachinespeed?Beyondcurrentpolicydebates,congressionaltestimony,newstrategies,andadrivetoidentify,test,andevaluatestandards,thereisafundamentalquestionofhowcomputeralgorithmswill

shapecrisisinteractionsbetweennuclearpowers.

1

Further,willrefinedAI/MLmodelspullpeoplebackfromthebrinkorpushthemovertheedgeduringcrisesthatareasmuchaboutfearand

emotionastheyarerationaldecisionmaking?Howwillhumansandmachinesinteractduringacrisisbetweennuclearpowers?

ThiseditionofOnFutureWarusesaseriesofwargamesasanexperimenttoanalyzehowplayerswith10ormoreyearsofnationalsecurityexperienceapproachcrisis

decisionmakinggivenvariablelevelsofknowledgeaboutarivalgreatpower’slevelofAI/MLintegrationacrossitsnationalsecurityenterprise.

Toanswerthisquestion,theCSISFuturesLabheldaseriesofcrisissimulationsinearly2023

analyzinghowAI/MLwillshapethefutureofdeterrence.Thegames—designedasarandomizedcontroltrial—exploredhumanuncertaintyregardingarivalgreatpower’slevelofAI/ML

integrationandhowthisfactoraffectedstrategicstabilityduringacrisis.

Twomajorfindingsemerged.First,acrossthesimulations,varyinglevelsofAI/MLcapabilities

hadnoobservableeffectonstrategyandageneraltrendoftryingtocombinemultiple

instrumentsofpowerwhenrespondingtoacrisis.WhiledatascienceandtheuseofAI/MLto

augmentstatecraftwillalmostcertainlybeadefiningfeatureofthenearfuture,thereappeartobeconstantsofstrategythatwillsurvivetheemergenceofsuchnewtechnologies.Diplomacy,economiccoercion,andinfluencecampaignswillsurviveevenasmachinescollectandprocessmoreinformationandhelpshapenationalsecuritydecisionmaking.AI/MLwillaugmentbutnotfundamentallychangestrategy.Thatsaid,thereisanurgentneedtostarttrainingnationalsecurityprofessionalstounderstandwhatAI/MLisandisnot,aswellashowitcansupporthumandecisionmakingduringforeignpolicycrises.

Second,hownationsfightintheshadowofnuclearweaponswillchangeasstatesselectively

targetthebattlenetworksoftheirrivals.EventhoughtheperceivedriskofescalationisnotlikelytobeaffectedbythebalanceofAI/MLcapabilities,thecriteriausedtoselectflexibleresponse

optionswillchange.Stateswillneedtobalancecounteringadversaryalgorithmswithensuringthattheydonotblindanadversaryandrisktriggeringa“dead-hand”—afastandautomated

systemdevelopedbytheSovietUniontolaunchnuclearweapons—escalationspiral.This

needtostriketherightbalanceinmilitarytargetingwillputanewpremiumonintelligence

collectionthatmapshowrivalstatesemployAI/MLcapabilitiesatthetactical,operational,andstrategiclevels.Itcouldalsochangehowstatesapproacharmscontrol,withanewemphasisonunderstandingwhereandhowAI/MLcapabilitiesaugmentcrisisdecisionmaking.

Deterrence,BattleNetworks,andAI/ML

Moderndeterrenceliteraturefocusesonhowstatesbargain,shortofwar,throughthreatsandcommitments.

2

Thesesignalschangehoweachsidecalculatesthecostsandbenefitsofgoingtowar,implyingthatthelessinformationeachsidehasaboutthebalanceofcapabilitiesandresolve,theharderitbecomestoencouragerestraint.

3

Signalingandcommunicationplaya

centralroleinhowstatesseektomanipulaterisktodenyanadversaryanadvantage,includingincentivesforseekingafaitaccomplibymilitaryforce.

4

Evenliteraturethatstressesthe

psychologicalandculturalantecedentsthatshapehowforeignpolicyleadersapproachcrisisdiplomacysharethisemphasisonthecentralroleofinformation.

5

Rationalcalculationsbreak

ALGORITHMICSTABILITY|2

downbasedonpastinformation(i.e.,howbeliefsshapeexpectations)andflawedweighting(e.g.,biasandprospecttheory).

Inmodernmilitaryplanningandoperations,informationismanagedthroughbattlenetworks.

6

Theabilitytoconductlong-rangeprecisionstrikesandtrackadversarytroopmovementsall

restsonaggregatingandanalyzingdata.ThislogicisafoundationfortheCoalitionJointAll-

DomainCommandandControl(CJADC2)network,whichisdesignedtopushandpulldata

acrossdistributednetworksofsensorsandshootersconnectedbyfastercommunication,

processing,anddecisionlayersinformedbyAI/ML.

7

ThenetworkisthenewtheoryofvictoryatthecenterofthenewJointWarfightingConcept,whichprioritizessynchronizingmultidomain

effectsintimeandspace.

8

Forthisreason,informationisnowakeycomponentofmilitarypowerand,byproxy,astate’sabilitytobargainwitharival.

9

Themoreinformationastatecanprocess,assistedbyalgorithms,themorelikelyitistoidentifywindowsofopportunityandriskaswellasalignends,ways,andmeanstogainarelativeadvantage.

Yet,mostoftheemergingliteratureonAI/MLandthefutureofwarfocusesmoreonriskand

ethicalconsiderationsthanbargainingadvantages.

10

First,multipleaccountsclaimthatAI/

MLwillcreatenewrisks,including“flashwars,”andarelikelytoproducedestabilizingeffects

alongmultiplevectors.

11

Thethinkinggoesthattheneweraofgreatpowercompetitionwill

bemarkedbyan“indelicatebalanceofterror”asRussia,China,andtheUnitedStatesracetoacquiretechnologicalgamechangers.

12

TotheextentthatAI/MLaltersmilitarypower,itcouldaffecthowstatesperceivethebalanceofpower.

13

Asperceptionsaboutpowerandinfluence

change,itcouldtriggerinadvertentescalationrisks.

14

Insidethebureaucracy,defenseplannerscouldrelyonbrittleandblack-boxedAI/MLrecommendationsthatcreatenewformsofstrategicinstability.

15

Atthetacticallevel,thespeedofautonomousweaponssystemscouldleadto

inadvertentescalationwhilealsounderminingsignalingcommitmentduringacrisis.

16

Therearetwoissueswiththeseclaims.First,argumentsaboutthedestabilizingroleofAI/MLhaveyettobeexploredbeyondliteraturereviews,alternativescenarios,andillustrativewargamesdesignedmoreforgainingperspectivethanforevaluatingstrategy.

17

Inother

words,hypothesesaboutriskandescalationhavenotbeentested.Second,accounts

aboutemergingtechnologyandinadvertentescalationoftendiscounttheroleofincreased

TheGame

Twotabletopexercisesanalyzinghowplayersdevelopflexible

deterrentandflexibleresponseoptionsduringacrisisbetweenrivalnuclearstates

Fakescenariotoreducebias

Playersallhadatleast10yearsofnationalsecurityexperience

Playersrandomlyassignedintodifferenttreatmentgroups

Threeroundsofcrisisinteractions,movingfromcompetitiontothe

earlystagesofamilitaryconflict

informationinreducingtensions.

18

Totheextentthatalgorithmsappliedacrossabattlenetworkhelpreduceuncertainty,theyarelikelytosupportdeterrence.Youcanneverliftthefogofwar,butyoucanmakeweatherforecastsanddescribe

whatisknown,unknown,andunknowable.Thisalternativelogicsitsatthefoundationofthe

“wargameasanexperiment”theCSISFuturesLabconstructedtoanalyzehowAI/MLcouldaffect

strategicstability.

19

WouldYouLiketoPlayaGame?

In2023,theCSISFuturesLabconducted

twotabletopexercisesexploringacrisis

scenarioinvolvingAI/ML’seffectonstrategic

decisionmaking.

20

Thetabletopexercisefocused

BENJAMINJENSEN,YASIRATALAN,ANDJOSEM.MACIASIII|3

onacrisisinvolvingathird-partystatebetweentworivals,eachofwhichhadnuclear

weaponsandasecond-strikecapability.Therivalstatesintheexercisewereabstractedto

removebiasaboutcurrentrivalriesthatdefinetheinternationalsystem,thusreducing,butnoteliminating,theriskofconfoundingfactorsskewinggameplay.

21

Asaresult,playersmadechoicesabouthowtorespondtoacrisisandwhichelementsofarivalstate’sbattlenetwork(i.e.,CJADC2)totarget.Thegamesconsistedof29individualplayers,eachwithover10yearsofnationalsecurityexperience.

22

Thegamedesignadaptedanearliertabletopexerciseusedtostudymoderncompetitionand

cyberescalationdynamicsknownasCorcyra.

23

Thisgameputplayersinafictionalscenario

involvingacrisisstandoffbetweentwonuclearrivals:GreenStateandPurpleStateoverasmallstate,OrangeState.GreenStateandOrangeStatearetreatyallies.PurpleStateandOrangeStatehaveamaritimeterritorialdispute.Players,assumingtheroleofGreenState,makedecisions

aboutthebestmixofflexibledeterrent(competition)andflexibleresponseoptionsasthe

crisisunfolds.Thegamescenario—involvingaterritorialdispute,enduringrivalry,andalliancenetworks—builtindynamicsassociatedwithescalationtofocusonlow-probability,high-

consequenceforeignpolicyevents.

24

TheuseoffakecountriesseekstomakeplayerslesslikelytointroducebiasintheresultsbasedonpriorbeliefsaboutcurrentpowerssuchastheUnitedStatesandChina.

Unknowntotheplayers,PurpleState’smovesanddecisionswerepredeterminedtowalkplayersuptheescalationladder,atechniqueknowninthewargamecommunityasa

1.5-sideddesign.Thisgamedesigncapturestheuncertaintyandinteractivecomplexity

(i.e.,reaction,counteraction)oftwo-sidedgamesbutbettersupportscapturingandcodingobservationsaboutplayerpreferencesandassessmentsofrisksandopportunities.

25

AppliedtoCorcyra,a1.5-sidedgamedesignenabledtheCSISFuturesLabtocollectdataonhowplayersmadedecisionsaboutcompetitionandconventionalconflictintheshadowofnuclearweapons.

Asthegameproceeded,playerswererandomlyassignedintotwodifferenttreatmentgroups

andledintodifferentroomswithafacilitator.Eachgroupwasgivenanidenticalgame

packetthatincludedanoverviewofthecrisisandmilitarybalancebetweenGreenStateand

PurpleStateaswellasthestandingpolicyobjectivesforGreenStatetodeterPurpleState’s

actionsagainstitstreatyallieswhilelimitingtheriskofabroaderwar(i.e.,extendedgeneral

deterrent).

26

Theonlydifferencebetweenthetwogroupswasintheintelligenceestimateof

theextentofPurpleState’sAI/MLcapabilitiesinrelationtoGreenState.AsseeninTable1,thisdifferencemeantthateverythingwasconstant(e.g.,militarybalance,policyobjectives,and

PurpleState’sactionsacrossthegame)exceptfortheknowledgetheplayersreceivedabout

theirrival’slevelofAI/MLcapabilities.Thisdesignreflectsafactorialvignettesurvey,controllingforwhetherAI/MLcapabilityisknownorunknown.

27

Overthecourseofthegame,eachgroup(i.e.,TreatmentAandB)respondedtoasetof

adversaryescalationvignettes.Unknowntotheparticipantswasthateachsubsequentmovewouldseetheirrivalescalateandmoveuptheescalationladdertoensureamoredynamicinterplaybetweencurrentdecisionmakingandmaintainingsufficientforcesandoptionsforfutureinteractions.Thedesignalsoensuredthatplayerswereforcedtoconfrontthethreatofconventionalstrikesonthenuclearenterpriseandthelimiteduseofnuclearweaponsinacounterforcerole.

ALGORITHMICSTABILITY|4

Table1:ExcerptsfromtheCorcyraGamePackets

TreatmentA:AI/MLCapabilitiesKnown

IntelligenceEstimateonAI/MLCapabilities

TreatmentB:AI/MLCapabilitiesUnknown

IntelligenceEstimateonAI/MLCapabilities

PurplestateisalsoknowntohaveimplementedawiderangeofAI/MLapplicationsinitsbattlenetworksto

catchuptoGreenState,whichhassimilarcapabilities.TheseapplicationshelpPurpleaircraft,missilebatteries,andwarshipssense,communication,andprocess(i.e.,identifypatterns,optimizetargeting)atmachinespeed.Theapplicationsalsoincludedecisionassistantsand

platformswithAI/MLsupportforlogisticsandtargeting.Over50percentofPurple’sconventionalforcearenownetworked,alevelthatmatchesGreenState.ThisAI/MLbattlenetworkconnectstoPurple’sNC3[nucleararmedcommand,control,andcommunications].

PurpleStateAI/MLcapabilitiesremainunknown.ThereareunconfirmedreportsPurpleStateismodernizing

tomatchGreenStateAI/MLapplicationsinitsbattle

networks,whichhelpGreenStateaircraft,missile

batteries,andwarshipssense,communication,and

process(i.e.,identifypatterns,optimizetargeting)at

machinespeed.TheapplicationsalsoincludedecisionassistantsandplatformswithAI/MLsupportforlogisticsandtargeting.ThisAI/MLbattlenetworkconnectsto

GreenState’sNC3.TheextentofanAI/MLbattlenetworkforPurpleStateremainsanintelgap

Source:CSISFuturesLab.

Thefirstroundofthegamedealtwithflexibledeterrentoptionsandcrisisresponse.

28

Afterwards,eachgroupreceivedabriefaboutacrisis—betweentheirtreatyally(OrangeState)andnuclearrival(PurpleState)—andeachplayer(GreenState)wasaskedtocraftaresponse

usingmultipleinstrumentsofpower(e.g.,diplomatic,informational,military,andeconomic)

bypickingthreeoptionsfromamenuof24preapprovedflexibledeterrentoptions.Each

instrumentofpowerhadsixresponseoptionsbasedontheimpliedlevelofescalation.This

designallowedtheCSISFuturesLabtoseeifvaryinglevelsofAI/MLcapabilitieshadaneffectonhowplayersapproachedcompetitionandcampaigningaspartofalargerdeterrentposture.

29

Specifically,itallowedtheresearchteamtotesthowthecapabilityaffectedoverallcompetitionstrategyandescalationdynamics.

ThesecondroundbeganwithalimitedmilitarystrikebyPurpleStateonOrangeState.PlayerswerebriefedthatPurpleStateconductedaseriesoflimitedstrikesonanairfieldandnaval

baseinOrangeState.OrangeStateintercepted50percentofthecruisemissilesintheattack,buttheremaining20—firedfromamixofPurpleStateaircraftandnavalwarships—damagedanOrangeStatefrigate,downedtwomaritimepatrolaircraft,anddestroyedanammunitiondepot.Initialreportssuggestedthatasmanyas20OrangeStatemilitarypersonnelwere

killedinaction,withanother30wounded.TherewerealsofourGreenStatemilitaryadvisersworkingonthebaseatthetimewhowerekilledintheattack.Theattackscoincidedwith

widespreadreportsofglobalpositioningsystem(GPS)denial,jamming,andcyberintrusionsinbothOrangeStateandGreenState.PurpleStatesaidtheattackwaslimitedtothemilitaryfacilityOrangeStatehadusedinpastprovocations,butitvowsbroaderattacksifOrange

StateorGreenStateresponds.Inthismanner,PurpleState’sconventionalmilitaryresponsereflectedcoreconceptsinmodernmilitarytheoryaboutmultidomainoperations,joint

firepowerstrikes,andsixth-generationwarfare.

30

Afterreceivingthisintelligenceupdate,playerswereaskedtonominateflexibleresponse

options.First,theplayershadtorecommendwhichofthemilitaryresponsesfromthemenuof24optionstheyrecommendedinresponsetoalimitedmilitarystrikebyPurpleStateonOrangeState.PlayersalsohadtospecifywhichlayerofPurpleState’sbattlenetworktheywantedto

BENJAMINJENSEN,YASIRATALAN,ANDJOSEM.MACIASIII|5

affectthroughtheirrecommendedresponse:sensing,processing,communicating,decision,oreffectors.Thisgamedesignlimitedtherangeofoptionsopentoplayerstosupportstatisticalanalysisandcomparisonbetweenthetwotreatments.

Thethirdroundexaminedifplayersadjustedtheirmilitaryresponseoptionstoconventionalstrikesontheirnuclearenterprise.PlayerswerebriefedthatPurpleStateconductedaseriesofconventionalstrikesagainstOrangeStateandGreenState.InOrangeState,thestrikes

targetedmajormilitaryfacilitiesandeventheOrangeStateleadership(bothmilitaryand

civilian)withamixofcruisemissiles,loiteringmunitions,andspecialoperationsforces(SOF)raids.TheattacksincludedstrikingGreenStateintelligencesatellitesandmajorearly-warningradarsaswellaskeyairfieldswhereGreenStatekeepsthemajorityofitsstrikeandbomberaircraftsquadrons.Themassprecisionconventionalstrikes—launchedlargelybyamixof

long-rangestrikeandbomberaircraftandsubmarines—alsotargetedkeyportfacilitiesin

GreenStateusedtoreloadverticallaunchcellsandsupportGreenStatesubmarineforces.

OrangeStatelostover30percentofitsmilitarycombatpowerand50percentofitscivilian

criticalinfrastructurerelatedtowatertreatment,energy,andtelecommunications.Itsleaderssurvivedthedecapitationstrikesbutareenactingcontinuityofgovernmentprotocols.GreenStatehassuffered10percentattritioninitsairandnavalforces.PurpleStatehasthreatenedthatitmaybeforcedtoexpandstrikes,toincludeusingnuclearweapons,ifGreenState

conductsadditionalmilitarystrikes.

WhattheGameRevealedabouttheFutureofDeterrence

FlexibleDeterrentOptions

Throughoutthegame,eachplayerhadthreeoptionsfromanycategory(i.e.,diplomatic,

informational,military,oreconomic)tochoose.Table2summarizeshowfrequentlyeachplayerselectedflexibleresponseoptionslinkedtodifferentinstrumentsofpowerduringtheinitial

crisisresponse(i.e.,Round1).Analyzingthesechoicesprovidesawindowintohowplayerswithatleast10yearsofnationalsecurityexperienceapproachedthe“ways”ofcrisismanagementtowardthe“end”ofreestablishingdeterrence.IfAI/MLisinherentlymoreescalationprone,onewouldexpecttoseeastatisticallysignificantdifferencebetweenthetreatments.

Therewasnostatisticallysignificantdifferencebetweenthetwotreatments.ThebalanceofAI/MLcapabilitiesdidnotalterhowplayersapproachedtheircompetitivestrategyandcrisismanagement.

31

Acrossthetreatments,playersdevelopedanintegratedapproach,

oftenchoosingmultipleinstrumentsofpowertopressuretheirrival,asopposedtofocusingonastrictlymilitaryresponse.Noplayerineithertreatmentusedonlyasingularinstrumentofpower,suchasonlyrespondingwithmilitaryoptions.Onthecontrary,over60percentof

playersineachtreatmentselectedmultipleinstrumentsofpower.

32

Inotherwords,thelevelofAI/MLcapabilitiesbetweenrivalsdidnotdirectlyalterearly-stagecrisisresponseandthe

developmentofflexibledeterrentoptionsdesignedtostabilizethesituationandreestablishdeterrence.ThisfindingrunscontrarytoperspectivesthatseeAI/MLasinherentlyescalatoryandriskprone.WhileAI/MLwillaugmentmultipleanalyticalprocessesfromthetacticalto

thestrategiclevel,nationalsecurityleaderswillstillmakecriticaldecisionsandseekwaystoslowdownandmanageacrisis.Theriskislikelymoreinhowpeopleinteractwithalgorithmsduringacrisisthanintheuseofdatascienceandmachinelearningtoanalyzeinformationandintelligence.AI/MLdoesnotposerisksonitsown.

ALGORITHMICSTABILITY|6

Table2:SelectedDeterrentOptions

InstrumentofPower

Diplomatic

Informational

Military

Economic

TreatmentA:KnownAI/ML

10

12

10

7

TreatmentB:

UnknownAI/ML

15

9

10

5

ContingencyTableforLayersChi-SquareTest1.76

P=0.62

Nostatisticallysignificantdifferencebetweenthetreatments.

Source:CSISFuturesLab.

DuringdiscussionsinRound1forTreatmentA(AI/MLCapabilitiesKnown),playersfocusedmoreonstrategyasitrelatestoshapinghumandecisionmaking.Inthefirstround,participantswere

dividedoverwhetherPurpleState’sactionswerehostileornotbutconcededthatPurpleState

wasattemptingtosignalthatGreenState’sactionsintheregionwereunwarranted.Ofnote,thisfocusonsignalingwasmoreabouthumanintentionthanalgorithmicassessments.TheplayersgenerallysawthecrisisasabouthumanleadersinrivalstatesseekingadvantagethroughnationalsecuritybureaucraciesaugmentedbyAI/MLcapabilities.Intheirdiscussions,humanswerein

theloopandfocusedonfindingwaystode-escalatethecrisiswithoutsignalingweakness.Thediscussionwaslessabouthowalgorithmsmightskewinformationandmoreabouthowtohavedirectcommunicationandclearmessagespassedthroughdiplomatstootherhumanleaders.Thislogicdrovetheplayerstofocusondevelopingthesituationthroughdiplomaticoutreachwhileincreasingintelligence,surveillance,andreconnaissance(ISR)assetsintheregion.

Althoughparticipantsagreedupontheutilityofdiplomacyasaresponsetothecrisis,theydivergedintheiropiniononwhethertheeffortsshouldpubliclycallforde-escalationor

privatelyusethirdpartiesorinfluencecampaignstowarnPurpleState’selitesaboutfurtherescalation.GroupmemberswidelysupportedincreasingISRassetsintheregiontoprovideearlywarningoffuturePurpleStatemilitarydeploymentsandincreaseprotectionoffriendlycommunicationsandintelligence-collectionassets.Additionally,groupmemberssoughttoprivatelyshareintelligencewithalliesintheregion.Again,AI/MLwaslessafocalpointthanamethodtogainbettersituationalawarenessandidentifythebestmechanismsforcrisis

communicationwiththerivalstate(i.e.,private,public,orthird-party).People,morethanmachines,werecentraltocrisismanagement.

WhereAI/MLdidenterthediscussionwaswithrespecttohowtherivalbattlenetworkswithhighdegreesofautomationcouldmisperceivethedeploymentofadditionalISRcapabilitiesduringacrisis.Althoughparticipantsstronglybelievedthatthisactionwasessential,theyrecognizedthepossibilitythatPurpleState’sAI/MLaugmentedstrategicwarningsystemsmaynotbeabletointerpretthedifferencebetweensignalingresolveordeliberateescalation.Atthesame

time,theythoughtthedirectcommunicationchannelsshouldaugmentthisriskandthatmostalgorithmicapplicationsanalyzingthedeploymentofISRassetswouldbemoreprobabilisticthandeterministic,thinkingintermsofconditionalprobabilitiesaboutescalationrisksratherthanjumpingtotheconclusionthatwarwasimminent.

BENJAMINJENSEN,YASIRATALAN,ANDJOSEM.MACIASIII|7

Putdifferently,sinceallAI/MLisbasedonlearningpatternsfromdataorrules-basedlogic,

theunderlyingpatternsofpastISRdeploymentswouldlikelymakethesystem—evenabsent

ahuman—unlikelytopredictalow-probabilityevent(i.e.,war)whenthereareothermore

commonoptionsassociatedwithaubiquitouseventsuchasincreasingintelligence.Ifanything,thereversewouldbemorelikely.Sincewarsandevenmilitarizedcrisesarerareevents,an

AI/MLalgorithm—dependingonthetrainingdataandparameters—wouldtreatthemassuch

absenthumanintervention.Thisdoesnotmeanthereisazeroprobabilityof“flashwars”andmisreadingthesituation.Rather,thepointisthatgiventhelownumberofwarsandmilitarizedcrisesbetweenstates,therearefewpatterns,andtherewillalmostcertainlybeahumanintheloop.Noseniornationalsecurityleaderislikelytocedealldecisionmakingtoanalgorithminacrisis.

TreatmentBsawasimilardiscussionbetweenplayersthatfocusedonusingmultipleinstrumentsofpowerandidentifyingde-escalationopportunities.Ofnote,italsosawmorediplomatic

outreach,withplayerswantingtoensuretheycouldcutthroughtheuncertaintyaboutPurpleState’sAI/MLcapabilitiesthroughdirectandindirectdiplomaticengagementsandcrisis

communication.InTreatmentB,concernaboutthestrengthsandlimitationsofanunknownAI/MLcapabilityinPurpleStatehadasignificantimpactongroupdecisionmaking.Groupmembers

desiredamutualunderstandingofGreenStateandPurpleState’sAI/MLcapabilities,believingthatgreaterclarityoverthebalanceanduseofAI/MLcapabilitiesbetweenrivalswouldleadtostability,ratherthanthecurrentinstabilitytheyfaced.

ThisfocusledparticipantstorecommenddeployingISRcapabilitiestoidentifyhowandif

PurpleStatewasusingAI-assistedISRthatmightmisreadtheirintentions.Someplayers

expressedconcernaboutwhethertheuseofISRwouldunderminetheireffortsatde-escalation.PlayerssuggestedthatifPurpleStatedetectedincreaseddeploymentofcollectionandearly-

warningassets,theywouldfeelpressuredtodeploymilitaryforcestopreventtheappearanceofacapabilitygap.

Ultimately,playersbelievedthatdeployingadditionalISRwasworththepotentialescalation

riskbecauseitwasessentialtoanalyzeadversaryactionsastheyunfolded.Thesegroups

soughttocombinediplomatic,flexibledeterrentoptionssuchasthird-partymediationand

redeploymentofnavalandaviationassetsawayfromPurpleStatetosignaladesiretode-

escalate.Inotherwords,nationalsecurityprofessionalswantedtoensuretherewasacheckonalgorithmicreasoningduringacrisisandwerewillingtodeployassetstothatendand