廣告調(diào)查（第二版）英文 指導(dǎo)手冊 Davis-Instructors-Manual-ch14 Experimentation

PAGE

AdvertisingResearch:Instructor’sManual

Copyright?2012PearsonEducation,Inc.publishingasPrenticeHall

AdvertisingResearch:Instructor’sManual

Copyright?2012PearsonEducation,Inc.publishingasPrenticeHall

PAGE

14.Experimentation

ChapterGoals

Afterreadingthischapter,studentswillhaveabetterunderstandingofthedifferencesbetweensurveyandexperimentalresearchaswellashowto:

? identifycharacteristicsofexperimentsanddistinguishexperimentalversusnonexperimentalapproachestodatacollection.

? explainthefactorsthatinfluenceanexperiment'sabilitytoprovideasoundbasisfordecisionmaking.

? describetheoptionsaresearcherhasindesigningexperiments.

? describehowadvertisersaddresstheissuesofinternalandexternalexperimentalvalidity.

NotestotheInstructor

TheChapterLectureprovidesaguidetokeytopicsandcontent.ThePowerPointslidesarenamed:davis_adresearch_ch14.ppt.

Therearetwoonlinesupplementalreadingsthatreinforceconceptsrelatedtoadvertisingexperimentation.

IABAdvertisingEffectivenessStudy

TheInternetAdvertisingBureau’s(IAB)AdvertisingEffectivenessStudyfundamentallyalteredhowadvertisersviewedbanneradvertisingand,asaresult,repositionedonlineadvertisingasalegitimateandimportantadvertisingmedium.Keepinmindthatwhilenoonedoubtstheimportanceofonlineadvertisingtoday,attitudeswereverydifferentin1997.Fewadvertisershadagoodunderstandingofhowadvertisinginthisnewmediumworkedand

evenfewerwereconfidentthatitevenworkedatall.Withinthiscontext,banneradvertisingwasnotviewedasacriticalcomponentofanadvertisingplan.Somefeltthatbanneradvertisingoccupiedthesamenicheasdirectmail,asbothformsofadvertisingcouldbemeasuredbyresponserate(click-throughinthecaseofbanneradvertising).Othersfeltthatbanneradvertisingwasequivalenttooutdooradvertisingasking“whatcanthoselittlebillboardsaccomplish?”TheIABAdvertisingEffectivenessStudyfundamentally

repositionedonlineadvertisingbydemonstratingthatadvertisinginthismediumworksmuchmoreliketelevisionadvertisingthandirectmailoroutdoor.Theresearchfoundthatafteronly

asingleexposure,onlineadvertisingcanincreasebrandawareness,enhanceproductperceptionsandimproveproductpurchaseintent.Perhapsmostimportantly,thesepositiveoutcomeswereachievedintheabsenceofclick-through.

Keysummarysectionsfromthe1997AdvertisingEffectivenessStudyareprovided.Reportpage6providesbackgroundonthestudyandinsightsintomethodologicalconsiderations.Reportpages12to15provideanoverviewoftheexperimentaldesign.Notehowkeyissuesofexperimentalresearch(sampling,randomassignment,experimentaldesignandtheuseofcontrolgroups)arealladdressed.Finally,reportpages7to11summarizethefindings.Notehowthepresentationoftheresultsofacomplexexperimentaldesigndonothavetobe

complexthemselves.Theresultsofthisresearcharepresentedinawaythathighlightsthekeyfindingsandbuildsapersuasive,easytounderstandexplanationofhowbanneradvertisingworked.

MassiveVideoGameAdvertisingTest

Jumper(themovie),releasedin2008,starredHaydenChristensen,JamieBellandRachelBilson.Theoverlapbetweenthemovie’sassumedtargetaudienceandvideogameplayersmotivatedresearchtodetermineifadvertisingJumper’sDVDreleaseinvideogameswasappropriateandeffective.

MassiveIncorporatedesignedanexecutedanexemplary“real-world”experimenttodeterminewhethertheplacementofin-gameadvertisingfortheJumperDVDreleaseimprovedattitudestowardthemovieandfosteredincreasedinterest(includingpurchaseintent)intheDVD.Massiveexecutedamulti-title,multiplecreative,timesensitivein-gameadvertisingcampaignandthensurveyedgamersplayingSkateandRainbow6:Vegas2todeterminetheadvertisingcampaign’simpactonpre-identifiedkeymeasures.(ExamplesofadvertisingplacementinthesegamesareshowninFigure14.5and14.6.)MassiveworkedwithInterpretResearchtorecruit603consolegamersaged13to34intoacontrolgroup(whichdidnotseethein-gameads)andatestgroup(whichdidseetheads).BothgroupsansweredanonlinequestionnairethatmeasuredtheirawarenessandopinionsofthefilmandpurchaseintentfortheDVD.Theresultswereimpressive.Amonggamerswhosawtheadstherewas:

? a39%increaseinawarenessfortheJumperDVDrelease.

? over100%increaseinfilmratingsandpurchaseconsideration.

? upto80%increasesinpositiveratingsofthefilm’sattributes.

Importantly,gamerswhosawtheadswereverypositive.Thepresenceoftheadswerefelttoimprovethegameexperiencebyaddingrealismand“l(fā)ookingcool.”

AswiththeIABStudy,notehowgreatcareistakentomakecertainthattheexperimentiswell-designedandexecuted.Also,notehowdataispresentedinwaythatmakesiteasyforusersoftheresearchtoseeandunderstandkeyfindingsandimplications.

ChapterLecture

I.DescriptiveVersusExperimentalResearch

Slide14-

2

Slide14-

3

Slide14-

4

Slide14-

5

Descriptive(survey)researchexaminesconsumers'attitudes,beliefs,lifestylesandbehaviorsataparticularpointintime.Experimentalresearchmovesbeyonddescription.Goalofexperimentistodeterminecausality-theeffectofchangesinoneareaononeormoreotherareas.

A.ExampleofSurveyVs.ExperimentalResults

Imaginethatanadvertiserusesbanneradsandwantstoconductresearchtodeterminewhichaspectsofbanneradsaremostlikelytomotivateindividualsto“click.”Theysurveyarandomsampleoftargetaudience,askingthemtoindicatewhichbanneradscharacteristicsaremostlikelytoinfluenceclick-through.Individualssaytheyaremostinfluencedbyadsconsideredrelevantandwhichcontaindollar-offcodesanddiscounts.

Whilefindingsnotunexpected,theyaresurprisedatthelow

levelofinfluenceattributedtoadsize.ResultsshowninSlide14-4.

Advertiserdecidestofurtherexploretherelationshipbetweenbanneradcharacteristicsandclick-throughinanexperiment.Randomlyselectmembersoftargetaudienceandmanipulatecharacteristicsofbanneradstowhichtheyareexposed.Manipulationallowsyoutoisolatetheeffectsofrelevance,discountcodesandadsize.Thentrackandmeasurehowmanipulationsaffectclick-through.Resultsindicatethatrelevance,discountcodeandadsizeareallequallyimportantoverall:click-throughishigherforrelevantversusnonrelevantads,higherforadswithdiscountcodesversusthosewithout,andhigherforlargerversussmallerads.

ResultsshowninSlide14-5.

Outcomeisimportant.Surveyresultswouldhaveledadvertisertoconcludethatadsizeisnotimportant,whichitclearlyis.Butexperimentcanprovideevengreaterinsightintotherelationshipbetweenbannercharacteristicsandclick-though.The

experimentfoundthat:

Slide14-

6

Slide14-

7

Slide14-

8

Slide14-

9

? Thereisarelationshipbetweendiscountcodesandrelevance:discountcodesareanimportantinfluencebutonlywhenthelevelofrelevanceislow.Whenanadisseenasrelevant,itisequallylikelytobeclickeduponwhetheradiscountcodeispresentornot.Whenanadisseenasnotrelevant,click-throughincreaseswhentheadcontainsadiscountcode.

? Thereisarelationshipbetweenadsizeandrelevance:adsizeisimportantbutonlywhenadsdonotcontainadiscountcode.Largeradsareequallylikelytobeclickeduponwhetheradiscountcodeispresentornot.Whenanadissmaller,click-throughincreaseswhentheadcontainsadiscountcode.

Resultsindicatethatinfluencesonclick-thougharemuchmorecomplexthanindicatedinthesurvey,

B.LimitationsofSurveyResults

Differenceinpriorexamplebetweensurveyandexperimentalresultsillustrateslimitationsofsurveyresearchforexploringindividuals’behaviorsandinfluencesonthesebehaviors.Surveyresultsbecomelessreliableforexplainingbehaviorswhen:

? surveyquestionsrequirerecall,especiallyoverlongertimeperiods,

? individualsareaskedforintrospectioninareasofbehaviortheydonotnormallythinkabout,

? individualsareaskedtousememorytoattributecauseandeffect,

? theremaybecomplex,multiple,orinterrelatedinfluencesonbehavior.

II.TheCharacteristicsofExperiments

Stepsinexperimentation:

? identifywhatyouneedtolearn,

? takerelevantactions(conducttheexperimentbymanipulatingoneor

morevariables),

? observeeffectsandconsequencesofactionsonothervariables,andthen

? determinecausality-extenttowhichobservedeffectscanbeattributedtoactionstaken.

Considerfollowingexample:

Peterwantstodevelopapizzacrustwithbettertexturethanthecrustshehasmadeinthepast.So,heconductsanexperimenttoseehowtheamountofwateraddedtothebatteraffectscrusttexture.Petermixesthreedifferentbatchesofcrustmakingcertainthatheusesthesameingredientsinthesamequantityineachcrustexceptfortheamountofwater.CrustAismadewithonecupofwater,CrustBcontains11/2cupsofwaterwhileCrustCcontainstwocupsofwater.Allingredientsareaddedandmixedinthesameorder.Peter'sovenislargeenoughto

holdonecrustatatime.Hebakeseachcrustforthesameamountoftimeatthesametemperature.Hemixeseachcrustbatterimmediatelybeforebakingandletstheovencooldownbetweeneachbaking.Peterthenobservesandevaluateseachcrustwhenitcomesoutoftheoven.HedecidesthatCrustChasabettertexture:itisfirmer,crisper(withoutburning),andhasabettercolor.PeterdecidesthattherecipeforCrustCisthebetterofthethreerecipes.

Slide14-

10

Slide14-

11

Slide14-

12

Slide14-

13

Referringbacktothefoursteps:Peterdecidedthathewantedtounderstandtheeffectofwatercontentoncrusttexture,hetooktherelevantactionbyvaryingtheamountofwaterintherecipes,heobservedtheeffectofwatervariationoncrusttextureandhedecided,becauseofthecarehetookinthedesignandconductoftheexperiment,thattherewascausalityandthatcouldhaveconfidenceintheconclusionthattheamountofwaterinCrustCresultsinthebestcrust.

A.ComponentsofanExperiment

Everyexperimenthasatleastoneindependentvariable,onedependentvariableandamanipulation.

Independentvariableiswhattheexperimentmanipulates.TheamountofwaterinPeter'scrustexperimentwastheindependentvariable.

Dependentvariableiswhattheresearcherisinterestedinexplaining.Itisthemeasureusedtoevaluateinfluenceofindependentvariable.Dependent

variablesinPeter'scrustexperimentwereratingsofcrustfirmness,crispness,andcolor.

Aneasywaytorememberdifferenceistoinsertnamesoftwovariablesinthissentence:

Slide14-

14

Slide14-

15

Slide14-

16

Slide14-

17

Slide14-

18

(Independentvariable)causesachangein(DependentVariable)anditisn'tpossiblethat(DependentVariable)couldcauseachangein(IndependentVariable).”

Forexample:

(Theamountofwater)causesachangein(pizzacrustcharacteristics)anditisn'tpossiblethat(pizzacrustcharacteristics)couldcauseachangein(theamountofwater).

B.RequirementsforCausality

Beforeacausalrelationshipcanbeaccepted,severalcriticalcriteriamustbesatisfied.

1.Eventsmusttakeplaceinproperorder.Foroneeventtocauseanotheritmustprecedeit.Peter'sexperimentsatisfiesthiscriterionbecausehismanipulationofwatercontentprecededcrustevaluation.

2.Eventsmustshowanexplicitrelationship.Requiresthatcausesandeffectsmustoccurorvarytogether.Peter'sexperimentsatisfiesthiscriterionbecausecrustqualityvariedinapredictablewayastheresultofwatermanipulation.

3.Alternativeexplanationsmustbereducedandeliminatedwheneverpossible.Whenallreasonablealternativeexplanationsforresultsareeliminatedthencanhavemoreconfidencethatmanipulationsofindependentvariablecausedobservedchangesindependentvariable.ConsiderwhathappenswhenSallyconductsherownpizzaexperiment:

SallybeginsherexperimentinexactlythesamewayasPeter.Shecarefullyprepareseachrecipe,manipulatingonlythewatercontent.However,becausesheispressedfortimeandhasashortageofpizzapans,shebakesCrustAinherovenandpan,shebakesCrustBinaneighbor'soveninadifferentbrandofpizzapan,andshebakesCrustCinasecondneighbor'sovenusingathirdbrandofpan.

Failstoeliminatetworeasonablealternativeexplanationsofresults.

Differencesinqualitycouldbeduetodifferencesinovensordifferencesinpans.

Slide14-

19

Slide14-

20

Slide14-

21

Slide14-

22

4.Relationshipmustshowstrengthofassociation.Thecauseandeffectrelationshipmustbesupportedandverifiedbysignificantoutcomesfromappropriatestatisticaltests.

III.InternalValidity

Allfourcriteriafortheacceptanceofcausationareessential.Thirdcriterionisperhapsthemostcrucialbecauseitaffectsanexperiment'sinternalvalidity.Internalvalidityreferstotheextentthatonecaneliminatealternativeexplanationsfortheobservedexperimentalresults.

Peter'spizzaexperimenthadhighinternalvaliditybecauseofthecarehetookinthepreparationandbakingofeachcrust.Consistencyintheseareashelpedtoeliminatealternativeexplanations.Sally'spizzaexperimentlackedinternalvaliditybecausetherewereseveralreasonablealternativeexplanationsoftheresults.CrustdifferencesinSally'sexperimentcouldbeattributedtodifferencesinovensorpans.

Researchershaveidentifiedseveraldifferenttypesofproblems(typicallyreferredtoas“threats”)thatcanraisealternativeexplanationsoftherelationshipbetweenindependentanddependentvariables.Thesethreatstointernalvalidityfallintofiveareas:

? problemsassociatedwithaninitial,pretestsurvey

(premeasurement,interactionandtesting)

? problemsduetodatacollection(instrumentation)

? problemsassociatedwiththesample(maturation,selectionandmortality)

? problemscausedbythestudycontext(history)

? problemsattributabletoresearcherbehavior(bias)

A.ProblemsAssociatedWithInitial,PretestSurvey

1.PremeasurementandInteraction

Premeasurementandinteractionproblemsarecloselyrelated.Theyoccurwhenindividualsareinterviewedatstartofanexperiment,beforetheyareexposedtotheindependentvariable.

Slide14-

23

Slide14-

24

Premeasurementproblemsoccurwheneveraninterviewadministeredbeforethestartofanexperimenthasadirecteffectontherespondent'sattitudes,actionsorbehaviorsduringtheexperiment,forexample:

TomresidesinatestmarketselectedbyWisk,amanufactureroflaundrydetergent.Wiskisabouttoconductatestofanewadvertisingcampaigninthetestmarket.Tomisselectedtoparticipateintheresearchstudythatwilltrackchangesinproductperceptionsandpurchasebehaviors.BeforethestartofthestudyTomcompletesaquestionnairethataskshimtoratefivebrandsoflaundrydetergentandtodescribehispurchasebehaviorsofthepastfourmonths.Afterhecompletesthequestionnaire,hesaystohimself:"I'verecentlytriedallthebrandsthequestionnaireaskedmeaboutexceptWisk.Ihaven'tboughtWiskinalongtime.Iwonderwhy?NexttimeIthinkI'llgetsome."Tom'snextpurchaseoflaundrydetergentisWisk.FourweekslaterTomfillsoutanotherquestionnairethatshowshehaspurchasedWisk.

WiskattributesthispositiveshiftinTom'sbehaviortotheeffectsoftheadvertising.However,Tom'spurchasehadnothingtodowiththeadvertisingtest.Infact,hedidnotevenseeanyWiskadvertising.Tom'spurchasewasdirectlyaffectedbythequestionnairehefilledoutbeforethestartofthetest.

Interactionproblemsoccurwheneveraninterviewadministeredbeforestartofexperimentaffectsrespondent'ssensitivityorresponsivenesstoindependentvariable,forexample:

Mary,isselectedtoparticipateinaresearchstudydesignedtoprobetheinfluenceofheathcareadvocacyadvertisingonconsumerattitudes.Beforethestartofthestudyandtheadvertisingcampaigns,Marycompletesaquestionnairethatexploresherattitudestowardhealthcarereformandhealthcare-relatedadvocacyadvertising.Attheendofthreemonths,duringwhichtimetheadvertisingappears,Marycompletesanothersurveythatagainexploresherattitudestowardhealthcarereformandhealthcare-relatedadvocacyadvertising.Maryshowsalargeshiftinattitudebetweenthefirstandsecondquestionnaires.

ThestudysponsorattributesthechangesinMary'sattitudestotheimpactoftheadvertisingandgeneralizesthesechangestothebroaderpopulation.But,Maryisnotrepresentativeofthe

broaderpopulationbecauseherbehaviorschangedafterfillingouttheinitialquestionnaire.AfterMarycompletedthefirstquestionnaire,shesaidtoherself:"That'saninterestingtopic.Ihaven'tseenanyadvertisingabouthealthcarereform,yet.I'dbetterwatchoutforit."ThefirstquestionnairesensitizedMary.Shewasmuchmorelikelythantheaverageconsumertowatchforthisadvertising.BecauseherchangeinattitudemaybelargerthanthetypicalconsumerwhowasnotsensitizedandwhopaidlessattentiontotheadvertisingthandidMary.

2.Testing

Slide14-

25

Slide14-

26

Slide14-

27

Resultsfromrepeatedadministrationsofthesamequestionnaireorsurvey,whenitisadministeredoncebeforetreatment(pretest)andonceagainaftertreatment(post-test).Itispossiblethatchangesinscoreshappensnotbecauseofindependentvariable’seffectbutratherbecauseofrepeatedtesting,forexample:

? Onbasisofsurveyresponses,consumermightappeartoshowbetterunderstandingofproductbenefitsafterexposuretoatestadwhenthisunderstandingiscomparedtoresponsesgivenonpretest.Thisimprovement,however,mightnotnecessarilyreflectanychangeinknowledgeorunderstanding,insteaditmightsimplyreflectrespondents’betterabilityto“correctly”answertestquestionsonthesecondtry.

B.ProblemsDuetoDataCollection:Instrumentation

Instrumentationreferstochangesmadetomeasurementinstrumentordatarecordingtechniquesduringanexperiment.Thisaffectsinternalvaliditybecauseitisnotknownwhethertoattributedifferencesobservedbeforeandaftertheexperimentalmanipulationtothemanipulationitselfortochangesinthemeasurementinstrument,recordingtechniques,datacollectionmethods,orinconsistentinterviewing.

C.ProblemsAssociatedWiththeSample

1.Maturation

Respondents'attitudes,behaviors,andphysiologychangeduringanexperiment.Theycanbecometired,hungry,thirsty,bored,and/ordisinterestedastheexperimentprogresses.Maturationthreatstointernalvalidityrefertothesetypesofchanges.Forexample,respondentsattheendofexperimentmaybelessinterestedthanwhentheybegan.Thismaycausethemtoprovidesuperficialanswersattheendofexperiment.

2.SelectionandMortality

Thesearerelatedtocompositionandcharacteristicsofgroupscomprisinganexperiment.

Slide14-

28

Slide14-

29

Slide14-

30

Slide14-

31

Selectionthreatsoccurwheneveratestandcontrolgroupsdifferinrelevantdemographics,attitudes,behaviors,orinaninclinedwaytorespondtoanindependentvariable.Itposessignificantproblemswithdatainterpretationbecauseitisnotknownwhethertoattributedifferencesinresponsebetweengroupstoinitialgroupdifferencesortoexperimentalmanipulation.

Mortalitythreatsarisewhenrespondentsdropoutofastudybetweenpretestandpost-testand,asaresult,itcannotbedeterminedifchangesbetweentwotestsaretheresultofmanipulationoraretheresultofadifferentgroupcharacteristicsattwostagesofmeasurement.Thiscanoccurwhetherornotthereisacontrolgroup.

D.ProblemsAssociatedWiththeStudyContext:History

Thisreferstoanyeventsorinfluencesbeyondthoseintentionallymanipulatedbyaresearcherthatoccurduringanexperimentandthathavethepotentialtoaffecttheexperimentaloutcome.Theycancomefromtwosources.

? Somehistoricalthreatstheareresultofcircumstancesbeyondanyperson'scontrol.Forexample:anexperimentdesignedtomeasuretheeffectofvariouspromotionsonsoftdrinkconsumptionmightbedistortedbyseveralweeksofunexpectedunseasonablyhotorcoldweather.

? Somehistoricalthreatsaretheresultofdeliberateactionstakenbyothers.WondrahandlotionwastestmarketedinMilwaukee.Duringthetest,largeandatypicaldiscountswereofferedbyWondra'scompetitorsignificantlydistortingsalesfigures.

E.ProblemsAssociatedWiththeResearcher:ResearcherBias

Occurswheneveractionsoftheexperimenter(whetherornotintentional)biastheoutcomeoftheexperiment.Biascanoccurduringanypartofexperiment,forexample:

? Canbiasresultsattheverystartoftheresearchbecauseofthewayindividualsareassignedtodifferentgroups.Whenrandomsamplingisnotused,aresearchercanassign“better”peopletothetestconditions,where“better”isdefined“asmostlikely

respondinthedesiredwaytotheindependentvariable.”

? Canbiasresultsduringanexperimentbysubtlytreatingindividualsindifferentgroupsdifferently,wheredifferenceintreatmentreflectsresearcher’sexpectationsaboutexperiment’soutcome,whichissubtlycommunicatedtoparticipants.

? Canbiasresultsduringdataanalysisthroughinconsistentdataprocessing.Wheninterpretingopen-endedresponses,forexample,aresearcher’sanalysisandclassificationofambiguousresponsescanbeinfluencedbywhattheresearcherwantsoutcomeofexperimenttobe.

Threatstointernalvaliditycan,toagreaterorlesserextent,becontrolledthroughexperimentaldesign.Somedesignsattempttocontrolallthreatstointernalvaliditywhileotherdesignsattempt(orareableto)controlonlya

few.Goodresearchdesignrequiresabalancebetweenaccuracyofinformationcollectedandcostrequiredtocollectthatinformation.Theresearchermustidentifyexperimentaldesignthat,givenmagnitudeofdecisionandinformationalneeds,providesthebestbalancebetweenaccuracyandcost.

UseemailcampaignbyAmericanSavingsAssociation(ASA)asbasisfordiscussionofdifferentapproaches.

IV.Quasi-ExperimentalDesigns

Arenottrueexperiments.Attempttouncoveracausalrelationship,butdosorelativelypoorlybecausetheyfailtoeliminatealargenumberofthreatstointernalvalidity.Asaresult,outcomesofquasi-experimentaldesignsaredifficulttointerpretandareonlyslightimprovementsoverdescriptivestudiesasabasisfordecision-making.

Thethreecommontypesofquasi-experimentaldesignswhichcouldbeusedtheare:

? onegrouppost-testonly

? onegrouppretesttopost-test

? twogrouppost-testwithcontrol

Slide14-

32

A.OneGroupPost-testOnly

Takessinglegroupofindividuals,exposesthemtoexperimentalmanipulationandthenmeasuresdependentvariable(s)aspartofthepost-test.

UseofthisdesignintheAmericanSavingsAssociationemailcampaignwouldresultinfollowingquasi-experiment:

TheAmericanSavingsAssociationemailstoagroupofindividuals,inthiscasethosewhohaveprovidedtheiremailaddressontheASAwebsite.Oneweekafterthelastmailing,eachindividualiscontactedbyemailandaskedtocompleteanonlinequestionnaire,whichmeasuresperceptionsofthestability,friendliness,andcommunityinvolvementoftheircommunity'sSavings&Loans.TheAmericanSavings

Associationlooksattheopinionsexpressedonthequestionnaire

andthendecideswhethertheemailcampaignwassuccessful.Thisisaveryweakdesign.

? Becausesampleisconvenienceratherthanrandom,cannotbegeneralizedtobroaderadultpopulation.

? Becausethereisnocontrolorreferencegroup,Associationisforcedtorelyentirelyonjudgmenttointerprettheresults.Forexample,assumethat"friendliness"ismeasuredusingafive-pointscaleinwhicha'5'isextremelyfriendly.Isanaveragefriendlinessratingof3.0goodorbad?TheAssociationhasnowayofknowingbecauseitdidnotknowhow"friendly"Savings&Loanswereperceivedtobepriortothestartofthecampaign.

? Cannotwithanydegreeofconfidenceattributepost-testratingstotheemailcampaign.Theseratings(reflectinghistoricalthreatstointernalvalidity)mayhavebeeninfluencedbynewsstories,personalexperienceswithSavings&Loansorwordofmouthoccurringduringthemonthlongemailcampaign.

? Designfailstocontrolforseveraladditionalthreatstointernalvalidity,specificallymaturation,selection,andmortality.

Slide14-

33

B.OneGroupPre-testtoPost-test

Alsoknownas"beforeandafterwithoutcontrol.”Issimilartopriordesignexceptthatapre-measureistakenbeforeexperimentalmanipulationoftheindependentvariable.Impactofexperimentaltreatmentcompareslevelsofdependentmeasureinthepost-treatmentmeasuretolevelsobservedinthepretreatmentmeasure.

UseofthisdesigninAmericanSavingsAssociationemailcampaignwould

resultinfollowingquasi-experiment:

AmericanSavingsAssociationselectsarandomsampleofadultsforstudyparticipation,EachindividualiscontactedbyemailandinterviewedbeforefirstemailingregardingtheirperceptionsofSavings&Loanstability,friendliness,andcommunityinvolvement.TheAssociationbeginsitstestemailingoneweekafterconclusionofthesepretestinterviews.Oneweekafterlastemailing,eachindividualiscontactedbyemailandaskedtocompleteanonlinequestionnaire,whichmeasuresperceptionsofstability,friendliness,andcommunityinvolvementoftheircommunity'sSavings&Loans.TheAmericanSavingsAssociationcomparestheopinionsexpressedonpost-emailingquestionnairetopre-emailinglevelsandthendecideswhetheranychangesbetweenthepretestandpost-testlevelsreflectasuccessfulcampaign.Forexample,if“safety”ratingsonthepretestaveraged“2”andonthepost-testaveraged“4”theAssociationwouldconcludethatthecampaignsuccessfullyimproved“safety”perceptions.

Weaknessesbecomeevidentwhenonetriestoassigncauseandeffectbyclaimingthattheemailcampaignwasresponsiblefortheriseinpositiveattitudes.

? Thereisnocontrolforanyhistoricalthreatstointernalvalidity

? Designfailstoeliminateanyproblemsassociatedwithinitialpret