培訓效果評估-課件

MeasureTrainingResults SomeConceptsSomeexperiencesSomelessons FrankCaoLucentTechnologiesfcao@1MeasureTrainingResults SomeTrainingProcess NeedsAnalysisDesignDeliveryEvaluation2TrainingProcess NeedsAnalysiFourEvaluationLevelsReactionHowdoparticipantsfeelabouttheprogram?LearningTowhatextentdidtheparticipantsincreaseknowledge,improveskills,and/orchangeattitude?BehaviorTowhatextentdidtheirjobbehaviorchange?ResultsWhatfinalresultsoccurred?(Quantity,quality,safety,sales,costs,profits,ROI)3FourEvaluationLevelsReactionLevelOne:ReactionWhatisevaluated?ContentTrainerMethodologyMaterialFacilitiesLogisticsRegistration4LevelOne:ReactionWhatisevaExampleQuestionsProbabilityyouwilluseideasfromthissessioninyourworkContentrelevancetomyjobPracticalexamplesandexperienceSpeaker’sknowledgeofsubjectareaSpeaker’spresentation/facilitationskillsSpeaker’sabilitytorespondtoquestionsGroupparticipation5ExampleQuestionsProbabilityyLevelOne:ReactionWhatisevaluated?Overall6LevelOne:ReactionWhatisevaExampleQuestionsWhatdidyoulikemostabouttheprogram?Whatdidyoulikeleastabouttheprogram?Inwhatwayscouldthisprogrambeimproved?Wouldyourecommendthiscoursetootherswhoareinterestedinthesubject?Whichofthefollowingfeatureswereimportantinyourdecisiontoattend?Rankinorderofimportance)7ExampleQuestionsWhatdidyouLevelOne:ReactionHowtoevaluate?QuestionnairesInterviewFocusGroupPhonesurvey8LevelOne:ReactionHowtoevalLevelOne:ReactionQuestionnaire:scalesExcellent Well Fair PoorExcellent VeryGood Good Fair PoorStronglyAgree Disagree76543 21High Low 10 98 76 543 21

9LevelOne:ReactionQuestionnaiLevelOne:ReactionAdvantagesEasytoexecuteParticipantsarefreeSomerelationshiptoonjobperformance IssuesPurelysubjectiveHaloeffect(smilesheet)HorneffectCentraltendencyTimepressure10LevelOne:ReactionAdvantagesILevelOne:ReactionTip1:highlightpurpose,solicitcooperationTodeterminethedegreethattheseminarmetyourneeds,wewouldlikeyoutogiveusyourhonestopinion…Tomakeourfuturesessionsasmeaningfulaspossible,wewouldappreciateyourcandidevaluationofthisprogram.Yourcommentswillbeusedtoimprovefutureofferingsofthisseminar.11LevelOne:ReactionTip1:highLevelOne:ReactionTip2:EncouragecommentsCommentsaremoreaccuratethanscoresTip3:Setupbaselinescores

Historicaldata Benchmarkingdata Pilotgroupdata(Formativeevaluation)Tip4:Combinequestionnaires,interviews,focusgroupsTip5:instantfeedbackformultisubjectcourses12LevelOne:ReactionTip2:EncoLevelTwo:LearningHowtoevaluate?TestDemonstrationPresentationDiscussionRolePlay13LevelTwo:LearningHowtoevalLevelTwo:LearningAdvantagesPressuretoparticipantsPressuretotrainerFirststeptochangebehavior IssuesTensionReliabilityCreditabilityDifficultyDifferentiationNotthebestindicatorforonjobchange14LevelTwo:LearningAdvantagesILevelTwo:LearningTip1:FullyutilizetestTestisgoodtoevaluateknowledgebasedtrainingFunctionalskillsNewEmployeeOrientationSupervisorABCBeseriousabouttestresultofformaltrainingInstantselfscoringandsymbolicincentiveforinformaltrainingReliability,Creditability,DifficultyandDifferentiationSetupbaselineFinalscorevs.gain15LevelTwo:LearningTip1:Full1616LevelTwo:LearningTip2:SetuprulesinadvancefordemonstrationandpresentationTopicTimeCriteriatoevaluateEvaluatorsarekeysuccessfactorTip3:Peopledon’tspeakmightbebestlearnerTip4:Peoplegoodatroleplaymightnotbegoodlearner17LevelTwo:LearningTip2:SetExample:ManagementSkillsParticipantsselected3topicsfrom11TopicrequirementTimeandassessorsPracticeRightbeforethestage…...ParticipantsknewthetopicAssessorsinteractedwithandprovidedfeedbacktoparticipants18Example:ManagementSkillsPartiExample:LeadershipStyle Describeasituationwhenyouusedwhatyoulearnedaboutleadershipstylestoimprovetheeffectivenessofarelationship.Besuretocover:WhatmotivatedyoutoexaminethestyledifferenceWhatdidyoufindandhowWhatdidyoudotoflexyourstyleWhatweretheresults19Example:LeadershipStyle19Example:PerformanceAppraisal20Example:PerformanceAppraisalLevelThree:BehaviorHowtoevaluate?ObservationPerformanceAppraisalInformationSupervisorfeedback(oralorwritten)Customerfeedback(oralorwritten)Employeefeedback(oralorwritten)21LevelThree:BehaviorHowtoevLevelThree:BehaviorAdvantagesDirectpurposeoftrainingRecognizedbysupervisorandmanagementteamIssuesTimeconsumingExpertisescarcityCooperationfromsupervisor/customerMultiple“cause-effect”22LevelThree:BehaviorAdvantageLevelThree:BehaviorTip1:SelectprogramcarefullyBehaviorbasedCompanybehaviorProfessionalskillsManagerialskillsCustomerservice(smile,response)Trainer’scooperation

Trainer’sexpertiseinevaluationDocumentsreadybeforetraining23LevelThree:BehaviorTip1:SeLevelThree:BehaviorTip2:EvaluationtimeinaccordancewithskillspracticeTip3:Supervisor’scooperationiskeysuccessfactorCommitmentwhenregisterMutualbenefitBriefthekeypoints24LevelThree:BehaviorTip2:EvLevelThree:BehaviorTip4:“Takeadvantage”ofvendorKeybehavioridentificationQuestionnairesdesignTimeResultanalysis25LevelThree:BehaviorTip4:“TExample:TimeManagement26Example:TimeManagement26TOT:TransferofTrainingOnly%ofclassroomlearningwastransferredtowork!ThetransfermustbePLANNED!27TOT:TransferofTrainingOnlyTOT:TransferofTrainingA.Beforetaking…,couldyou…?B.Canyoucurrently…?C.Howoftenareyouapplying…onyourjob

TOToccurredifAisNBisYCis>=1Y NY N012345

28TOT:TransferofTrainingA.BeTOT:TransferofTrainingA:“Y”/All-PriorKnowledgeRateB:“N”/All-IncomprehensionRateC:“0”/All-DisconnectRateTOTRateAcceptableRate 0-66% 0-33% 0-33%Low Mid High0-33% 34-66% 67-100%29TOT:TransferofTrainingAccepLevelFour:ResultsHowtoevaluate?QuantityQualitySafetySalesCostsProfitsROI30LevelFour:ResultsHowtoevaluLevelFour:ResultsAdvantagesUltimatepurposeIssuesTimeconsumingExpertisescarcityCooperationfromManagementMultiple“cause-effect”31LevelFour:ResultsAdvantagesIsLevelFour:ResultsTip1:SelectprogramsthatothercausesarestableQualitytraining-errorrateSafetytraining-accidentrateSalestraining-revenuegrowthrate/orderloserate32LevelFour:ResultsTip1:SelecLevelFour:ResultsTip2:UseacontrolgroupifpracticalTip3:Allowtimeforpossibleresultstotakeplace33LevelFour:ResultsTip2:UseaExample:RetentionSkills34Example:RetentionSkills34Example:EMSProjectShopReengineeringCostReductionImproveCustomerServiceProcessOrderEntryviaConfigurationSystemChinaCBUARImprovement35Example:EMSProjectShopReengASTD1997Study36ASTD1997Study36演講完畢，謝謝觀看！演講完畢，謝謝觀看！MeasureTrainingResults SomeConceptsSomeexperiencesSomelessons FrankCaoLucentTechnologiesfcao@38MeasureTrainingResults SomeTrainingProcess NeedsAnalysisDesignDeliveryEvaluation39TrainingProcess NeedsAnalysiFourEvaluationLevelsReactionHowdoparticipantsfeelabouttheprogram?LearningTowhatextentdidtheparticipantsincreaseknowledge,improveskills,and/orchangeattitude?BehaviorTowhatextentdidtheirjobbehaviorchange?ResultsWhatfinalresultsoccurred?(Quantity,quality,safety,sales,costs,profits,ROI)40FourEvaluationLevelsReactionLevelOne:ReactionWhatisevaluated?ContentTrainerMethodologyMaterialFacilitiesLogisticsRegistration41LevelOne:ReactionWhatisevaExampleQuestionsProbabilityyouwilluseideasfromthissessioninyourworkContentrelevancetomyjobPracticalexamplesandexperienceSpeaker’sknowledgeofsubjectareaSpeaker’spresentation/facilitationskillsSpeaker’sabilitytorespondtoquestionsGroupparticipation42ExampleQuestionsProbabilityyLevelOne:ReactionWhatisevaluated?Overall43LevelOne:ReactionWhatisevaExampleQuestionsWhatdidyoulikemostabouttheprogram?Whatdidyoulikeleastabouttheprogram?Inwhatwayscouldthisprogrambeimproved?Wouldyourecommendthiscoursetootherswhoareinterestedinthesubject?Whichofthefollowingfeatureswereimportantinyourdecisiontoattend?Rankinorderofimportance)44ExampleQuestionsWhatdidyouLevelOne:ReactionHowtoevaluate?QuestionnairesInterviewFocusGroupPhonesurvey45LevelOne:ReactionHowtoevalLevelOne:ReactionQuestionnaire:scalesExcellent Well Fair PoorExcellent VeryGood Good Fair PoorStronglyAgree Disagree76543 21High Low 10 98 76 543 21

46LevelOne:ReactionQuestionnaiLevelOne:ReactionAdvantagesEasytoexecuteParticipantsarefreeSomerelationshiptoonjobperformance IssuesPurelysubjectiveHaloeffect(smilesheet)HorneffectCentraltendencyTimepressure47LevelOne:ReactionAdvantagesILevelOne:ReactionTip1:highlightpurpose,solicitcooperationTodeterminethedegreethattheseminarmetyourneeds,wewouldlikeyoutogiveusyourhonestopinion…Tomakeourfuturesessionsasmeaningfulaspossible,wewouldappreciateyourcandidevaluationofthisprogram.Yourcommentswillbeusedtoimprovefutureofferingsofthisseminar.48LevelOne:ReactionTip1:highLevelOne:ReactionTip2:EncouragecommentsCommentsaremoreaccuratethanscoresTip3:Setupbaselinescores

Historicaldata Benchmarkingdata Pilotgroupdata(Formativeevaluation)Tip4:Combinequestionnaires,interviews,focusgroupsTip5:instantfeedbackformultisubjectcourses49LevelOne:ReactionTip2:EncoLevelTwo:LearningHowtoevaluate?TestDemonstrationPresentationDiscussionRolePlay50LevelTwo:LearningHowtoevalLevelTwo:LearningAdvantagesPressuretoparticipantsPressuretotrainerFirststeptochangebehavior IssuesTensionReliabilityCreditabilityDifficultyDifferentiationNotthebestindicatorforonjobchange51LevelTwo:LearningAdvantagesILevelTwo:LearningTip1:FullyutilizetestTestisgoodtoevaluateknowledgebasedtrainingFunctionalskillsNewEmployeeOrientationSupervisorABCBeseriousabouttestresultofformaltrainingInstantselfscoringandsymbolicincentiveforinformaltrainingReliability,Creditability,DifficultyandDifferentiationSetupbaselineFinalscorevs.gain52LevelTwo:LearningTip1:Full5316LevelTwo:LearningTip2:SetuprulesinadvancefordemonstrationandpresentationTopicTimeCriteriatoevaluateEvaluatorsarekeysuccessfactorTip3:Peopledon’tspeakmightbebestlearnerTip4:Peoplegoodatroleplaymightnotbegoodlearner54LevelTwo:LearningTip2:SetExample:ManagementSkillsParticipantsselected3topicsfrom11TopicrequirementTimeandassessorsPracticeRightbeforethestage…...ParticipantsknewthetopicAssessorsinteractedwithandprovidedfeedbacktoparticipants55Example:ManagementSkillsPartiExample:LeadershipStyle Describeasituationwhenyouusedwhatyoulearnedaboutleadershipstylestoimprovetheeffectivenessofarelationship.Besuretocover:WhatmotivatedyoutoexaminethestyledifferenceWhatdidyoufindandhowWhatdidyoudotoflexyourstyleWhatweretheresults56Example:LeadershipStyle19Example:PerformanceAppraisal57Example:PerformanceAppraisalLevelThree:BehaviorHowtoevaluate?ObservationPerformanceAppraisalInformationSupervisorfeedback(oralorwritten)Customerfeedback(oralorwritten)Employeefeedback(oralorwritten)58LevelThree:BehaviorHowtoevLevelThree:BehaviorAdvantagesDirectpurposeoftrainingRecognizedbysupervisorandmanagementteamIssuesTimeconsumingExpertisescarcityCooperationfromsupervisor/customerMultiple“cause-effect”59LevelThree:BehaviorAdvantageLevelThree:BehaviorTip1:SelectprogramcarefullyBehaviorbasedCompanybehaviorProfessionalskillsManagerialskillsCustomerservice(smile,response)Trainer’scooperation

Trainer’sexpertiseinevaluationDocumentsreadybeforetraining60LevelThree:BehaviorTip1:SeLevelThree:BehaviorTip2:EvaluationtimeinaccordancewithskillspracticeTip3:Supervisor’scooperationiskeysuccessfactorCommitmentwhenregisterMutualbenefitBriefthekeypoints61LevelThree:BehaviorTip2:EvLevelThree:BehaviorTip4:“Takeadvantage”ofvendorKeybehavioridentificationQuestionnairesdesignTimeResultanalysis62LevelThree:BehaviorTip4:“TExample:TimeManagement63Example:TimeManagement26TOT:TransferofTrainingOnly%ofclassroomlearningwastransferredtowork!Thet