數(shù)據(jù)與模型安全 課件 第11周：聯(lián)邦學(xué)習(xí)

Federated

Learning姜育剛，馬興軍，吳祖煊/2017/04/federated-learning-collaborative.htmlRecap:week10口 CommonTamperingandDeepfakes口 ImageManipulationDetection口 VideoManipulationDetectionThisWeek口 FederatedLearning口 PrivacyinFederatedLearning口 RobustnessinFederatedLearning口 ChallengesandFutureResearchTraditionalMachineLearningDataModelDataandmodelinonesingleplaceTraditionalMachineLearningDataModelWhat

if

we

need

more

data?DataGatheringUsingmultipleGPUsFederatedLearning:Whatisit?Google:FederatedLearning:CollaborativeMachineLearningwithoutCentralizedTrainingDataFederatedLearning:Challenges,Methods,andFutureDirections,/pdf/1908.07873.pdfNextwordpredictiononmobile.FederatedMachineLearning:ConceptandApplications,/pdf/1902.04885.pdfHorizontalFL（橫著切）:samefeatures,differentsamplesFederatedLearning:TypesVerticalFL（縱著切）:samesamples,differentfeaturesFederatedLearning:TypesFederatedMachineLearning:ConceptandApplications,/pdf/1902.04885.pdfFederatedLearning:TypesFederatedTransferLearning:differentsamples,differentfeaturesFederatedMachineLearning:ConceptandApplications,/pdf/1902.04885.pdfCompareDifferentParadigmsFederatedMachineLearning:ConceptandApplications,/pdf/1902.04885.pdfCompareDifferentParadigmshttps:///projects/distributed-learning-and-collaborative-learning-1/overview/SplitLearningvsFederatedLearningFederatedLearningFrameworksHE:homomorphicencryption SS:secretSharingObjectivesandUpdatesinFLGlobalobjectiveLocalobjective:LocalUpdates:GlobalAggregation(e.g.FedAvg):FederatedLearning–MajorChallengesExpensiveCommunicationSystemsHeterogeneityStatisticalHeterogeneityPrivacyandSecurityConcernsFederatedLearning:Challenges,Methods,andFutureDirections,/pdf/1908.07873.pdfFederatedLearning-HorizontalFederatedLearning:Challenges,Methods,andFutureDirections,/pdf/1908.07873.pdfHFLcanfurtherbedividedinto…?PrivacyandSecurityThreatsLyuetal.“Privacyandrobustnessinfederatedlearning:Attacksanddefenses.”TNNLS,2022.SummaryofThreatModelsFLserver(insider)FLparticipants(insider)Eavesdroppers(outsider)Serviceusers(outsider)□InsidervsOutsider □InsiderAttacksByzantine:theworstattacker,knowseverythingaboutthesystem,doesnotobeytheprotocol,sendarbitraryupdates,evencolludewitheachother.Sybil:takingoverthenetworkbysimulatingmanydummyparticipants,out-votethehonestusersSemi-honestvsMaliciousSemi-honestsettingMalicioussettingTraining-timevsTest-timeStealprivatedata,stealmodel,corruptthemodel(trainingtime)Adversarialattack(testtime)SummaryofAttacksExistingattacksagainstserver-basedFLPoisoningAttacksDatapoisoningvsmodel(weight)poisoningDataPoisoningAttacksinTraditionalML□Dirty-labelPoisoningLabelflipping(onlychangelabels)Dirty-labelbackdoor(changeinputsandlabels)Clean-labelPoisoningClean-labelbackdoor(onlychangeinputs)DataPoisoningAttacksinTraditionalMLAsimplepatterncanmakethemodeltomemorizeFLPoisoningAttacks–ModelPoisoningMaincharacteristics:ChangelocalmodelweightsMostlyByzantineattack(attackercandoanythingtotheweights)CanattackByzantine-robustaggregationmechanismssuchasKrumandcoordinate-wisemedianinsteadofweightedaveragingKrum:PrivacyAttacksForeverycommunicationround,localclientshavethechancetoreverseengineerothers’gradients.Fromthereversedgradients,reverseengineer:RepresentationsMembershipPropertiesSensitiveattributesInVFL:featuresPrivacyAttacks–InferenceAttacksDeepmodelsundertheGAN:informationleakagefromcollaborativedeeplearning,CCS2017InferenceclassrepresentationsusingGANsCIFAR-10horseclassReconstructAlice’sfaceimagePrivacyAttacks–InferenceAttacksComprehensiveprivacyanalysisofdeeplearning:Passiveandactivewhite-boxinferenceattacksagainstcentralizedandfederatedlearning,S&P,2019Inferencemembership:Passiveattacks:observeandinference.Activeattacks:influencethetargetmodelinordertoextractmoreinformation.WeaknessofFL:FLcreatesanenvironmentfor(almost)white-boxattacksPrivacyAttacks–InferenceAttacksOtherinferenceattacks:inferringproperties,trainingdata,labels...DeepLeakagefromGradient(DLG)ImprovedDeepLeakagefromGradient(iDLG)…Defenses–PrivacyDefenseHomomorphic

Encryption:RSAEl

GamalPaillier…Homomorphic

properties:Allows

computation

directly

onencrypted

data(“可算不可見(jiàn)”)Needs

to

be

designed

for

eachalgorithmA

side

note:

attacking

encrypted

FL

is

challengingbut

still

possible!Defenses–PrivacyDefense2.

SecureMultipartyComputation(SMC,Yaosharing):SecureML(data-independentofflinephase+fastonlinephase)Offlinemultiplicationtriplets,truncate,sharingCharacteristics:HighlevelprivacyHighcomputationandcommunicationcostYao'sMillionaires'problemProtocolsforSecureComputations,AndrewChi-ChihYao,1982,UCBerkeleyDefenses–PrivacyDefense2.DifferentialPrivacy(DP):TypesofDP:LocalDPCentralizedDPDistributedDPDefenses–PrivacyDefenseDataflowofstatisticsunderLDP2.DifferentialPrivacy(DP):Defenses–PrivacyDefense2.DifferentialPrivacy(DP):TypesoffrequencyestimationDefenses–PrivacyDefense2.DifferentialPrivacy(DP):Real-worldapplications.Vanilla

FLM:ADPmechanismCentralized

DPM:ADPmechanismLocal

DPM:ADPmechanismE:encryptionD:decryptionDistributed

DPDefenses–ByzantineDefenseAlgorithm:Krum(forByzantinerobustness)Setting:nparticipants,fareByzantine,with??≥????+??Atcommunicationroundt,?? ?? ??serverreceives{????,????,…,????}foreach????:??selecttheclosest(L2distance)n-f-2intoset????compute??????????????=∑?? ??∈???? ????????????? ????????????=???=argmin{?????????????? ,…,??????????????}updateglobalparameter:????.??=????+??????????Blanchardetal.“Machinelearningwithadversaries:Byzantinetolerantgradientdescent.”NeurIPS,2017.Defenses–ByzantineDefenseAlgorithm:Krum(forByzantinerobustness)Blanchard

et

al.

“Machine

learning

with

adversaries:

Byzantine

tolerant

gradient

descent.”

NeurIPS,

2017.紅色：攻擊梯度藍(lán)色：真實(shí)梯度黑色：本地梯度黑色曲線：損失函數(shù)Defenses–ByzantineDefenseMorerobustaggregationmethods:Multi-Krum=Krum+Averaging=Krumrobustness+increasedconvergencespeedcoordinate-wisemedian,coordinate-wisetrimmedmeanmedianisnotgoodforconvergenceBulyan=Krum+trimmedmedianMedianandgeometric-median(RobustFederatedAggregation)RFA:approximategeometricmedian(notrobusttoByzantineattacks)Defenses–ByzantineDefenseModelpoisoningattackcanbreakKrumandcoordinate-wisemedianAnalyzingfederatedlearningthroughanadversariallens,ICML2019.??/:adversarialtargetclassr:numberofpoisonedsamples??0:cleandata1???2:estimationoftheglobalparametersReversedgradientsfromthelastround.Defenses–SybilDefenseFromtraditionalML:RejectonNegativeInfluence(RONI)WithacleanvalidationdatasetItrequiresuniformdistributioninnon-IIDsetting,notgood.FoolsGold:Sybilsharethesameobjective,driftsawayfromtheoriginalobjectiveCoreidea:cosinesimilarityFoolsGold:MitigatingSybilsinFederatedLearningPoisoning,/abs/1808.04866Defenses–SybilDefenseDistributedbackdoorattack(DBA)canbypassbothRFAandFoolsGold.DBA:Distributed

Backdoor

Attacks

against

Federated

Learning,

ICLR

2020.

Defenses

-

SummaryDefenseagainstFederatedLearningPoisoning.n:numberofparticipants.RemainingChallengesandFutureResearch□ CurseofdimensionalityLargermodelsaremorevulnerableSharingweights/gradientsmaynotbeagoodidea□ WeaknessesofcurrentattacksGANattackassumestheclassofdataisfromonesingleparticipantDLG/iDLGworkwithsecond-ordergradientmethod(expensive)andsmallminibatch-gradients(B=8)□ Vulnerabilitytofreeriders:pretendtohavedatabutnot.□ WeaknessofCurrentPrivacy-preservingTechniquesSecureaggregationismorevulnerabletopoisoningattacks