hoc

1

PostHocTestsinANOVA

ThishandoutprovidesinformationontheuofposthoctestsintheAnalysisofVariance

(ANOVA).Posthoctestsaredesignedforsituationsinwhichtherearcherhasalreadyobtained

asignificantomnibusF-testwithafactorthatconsistsofthreeormoremeansandadditional

explorationofthedifferencesamongmeansisneededtoprovidespecificinformationonwhich

meansaresignificantlydifferentfromeachother.

Forexample,thedatafile“”(availableonthewebsite),containstwofactors,gender

andexperienceandonedependentmeasure,ngtheGLM-

UnianovaprocedureinSPSSproducesthefollowingANOVAsourcetable:

TestsofBetween-SubjectsEffects

DependentVariable:SpatialAbilityScoreErrors

SourceTypeIII

Sumof

Squares

dfMean

Square

FSig.

Corrected

Model

102.575714.65415.029.000

Intercept403.2251403.225413.564.000

GENDER60.025160.02561.564.000

EXPER28.27539.4259.667.000

GENDER*

EXPER

14.27534.7584.880.007

Error31.20032.975

Total537.00040

Corrected

Total

133.77539

aRSquared=.767(AdjustedRSquared=.716)

Inspectionofthesourcetableshowsthatboththemaineffectsandtheinteractioneffectare

dereffectcanbeinterpreteddirectlysincethereareonlytwolevelsofthe

retationofeithertheExperiencemaineffectortheGenderbyExperienceinteraction

isambiguous,however,delaytestingand

cernnowishowtodetermine

whichofthemeansforthefourExperiencegroups(etablebelow)aresignificantlydifferent

fromtheothers.

Thefirstposthoc,ginalsolutiontothisproblem,developedbyFisher,

wastoexploreallpossiblepair-wicomparisonsofmeanscomprisingafactorusingthe

ocedurewasnamedtheLeastSignificantDifference(LSD)

stsignificantdifferencebetweentwomeansiscalculatedby:

_________

LSD=to2MSE/n*

wheretisthecritical,tabledvalueofthet-distributionwiththedfassociatedwithMSE,the

2

denominatoroftheFstatisticandn*isthenumberofscoresudtocalculatethemeansof

critical

xample,tat"=.05,two-tailed,withdf=32is2.0369,MSEfromthe

sourcetableaboveis0.975,andn*is10scorespermean.

___________________

LSD=to2MSE/n*=2.0369o2(.975/10=0.6360

SotheLSDorminimumdifferencebetweenapairofmeansnecessaryforstatisticalsignificanceis

0.636.

Inordertocomparethiscriticalvalueordifferenceforallourmeansitisufultoorganizethe

,thenumberofpair-wicomparisonsamongmeanscanbecalculatedusing

theformula:k(k-1)/2,wherek=

prentexample,theexperiencefactorhasfourlevelssok=4andtherearek(k-1)/2=4(3)/2=6

uniquepairsofmeansthatcanbecompared.

ExperiencewithMechanicalProblems

DependentVariable:SpatialAbilityScoreErrors

M

95%

Confidence

Interval

ExperienceLowerBoundUpperBound

Alot2.100.3121.4642.736

FairAmount2.700.3122.0643.336

Some3.600.3122.9644.236

Littletonone4.300.3123.6644.936

Thetablewewillconstructisatableshowingtheobtainedmeansontherowsandcolumnsand

subtracteddifferencesbetweeneachpairofmeansintheinteriorcellsproducingatableof

tructthetablefollow

thesteps:1)rankthemeansfromlargesttosmallest,2)createtablerowsbeginningwiththe

largestmeanandgoingthroughthenext-to-smallestmean,3)createtablecolumnsstartingwith

thesmallestmeanandgoingthroughthenext-to-largestmean,4)computetheabsolutedifference

betweeneachrowandcolumninterction/rentexamplethisresultsinthetableof

absolutemeandifferencesbelow:

2.12.73.6

4.32.21.60.7

3.61.50.9

2.70.6

NowapplyingourLSDvalueof.636tothemeandifferencesinthetable,itcanbeenthatall

differencesamongthemeansaresignificantat"=.05exceptthelastdifferencebetweenthe

3

unately,thep-valuesassociatedwiththemultipleLSDtestsare

hesamplingdistributionfortassumesonlyonet-testfromanygivensample,

substantialalphaslippagehasoccurredbecau6testshavebeenperformedonthesamesample.

Thetruealphalevelgivenmultipletestsorcomparisonscanbeestimatedas1-(1-"),wherec

c

=thetotalnumberofcomparisons,contrasts,rentexample

1-(1-")=1-(1-.05)=.ultipletestinginthissituation,thetruevalueof

c6

alphaisapproximately.26ratherthan.05.

Anumberofdifferentsolutionsandcorrectionshavebeendevelopedtodealwiththisproblem

andproduceposthocteststhatcorrectformultipletestssothatacorrectalphalevelis

loftheapproaches

arediscusdbelow.

Tukey’’stestwasdevelopedinreactiontotheLSDtestandstudieshave

showntheprocedureaccuratelymaintainsalphalevelsattheirintendedvaluesaslongas

statisticalmodelassumptionsaremet(i.e.,normality,homogeneity,independence).Tukey’sHSD

wasdesignedforasituationwithequalsamplesizespergroup,butcanbeadaptedtounequal

samplesizesaswell(thesimplestadaptationustheharmonicmeanofn-sizesasn*).The

formulaforTukey’sis:

_________

HSD=qoMSE/n*

whereq=therelevantcriticalvalueofthestudentizedrangestatisticandn*isthenumberof

ationofTukey’sfortheprent

exampleproducesthefollowing:

_________________

HSD=qoMSE/n*=3.83o.975/10=1.1957

Theqvalueof3.83isobtainedbyreferencetothestudentizedrangestatistictablelookingupthe

qvalueforanalphaof.05,df=<=32,k=p=r=edifferencesinthetableofmean

differencesbelowthataremarkedbytheasteriskxceedtheHSDcriticaldifferenceandare

significantatp<.attwodifferencessignificantwithLSDarenownotsignificant.

2.12.73.6

4.32.2*1.6*0.7

3.61.5*0.9

2.70.6

Scheffe’e’sprocedureisperhapsthemostpopularoftheposthocprocedures,the

mostflexible,e’sprocedurecorrectsalphaforallpair-wior

simplecomparisonsofmeans,x

ult,Scheffe’sisalsothe

4

e’sisprentedandcalculatedbelowforourpair-

wisituationforpurposofcomparisonandbecauScheffe’siscommonlyappliedinthis

situation,butitshouldberecognizedthatScheffe’sisapoorchoiceofproceduresunlesscomplex

comparisonsarebeingmade.

Forpair-wicomparisons,Scheffe’scanbecomputedasfollows:

critical12

o(k-1)FoMSE(1/n+1/n)

Inourexample:

o(3)(2.9011)o.975(.1+.1)=2.9501=1.3027

Andreferringtothetableofmeandifferencesabove,itcanbeenthat,despitethemore

stringentcriticaldifferenceforScheffe’stest,inthisparticularexample,thesamemeandifferences

aresignificantasfoundusingTukey’sprocedure.

sa

Tukey--

Forsythe’sposthocprocedureisamodificationoftheScheffetestforsituationswith

’sMultipleRangetestandtheNewman-Keulstestprovide

differentcriticaldifferencevaluesforparticularcomparisonsofmeansdependingonhowadjacent

stshavebeencriticizedfornotprovidingsufficientprotectionagainstalpha

rinformationonthetestsandrelatedissuesin

contrastormultiplecomparisontestsisavailablefromKirk(1982)orWiner,Brown,andMichels

(1991).

ldbeapparentfromtheforegoingdiscussion,there

ceduresdifferintheamountand

actofthedifferencescanbeeninthetableof

criticalvaluesfortheprentexampleshownbelow:

CriticalDifference

LSD0.6360

Tukey1.1957

Scheffe1.3027

Themostimportantissueistochooaprocedurewhichproperlyandreliablyadjustsforthe

ghScheffe’s

procedureisthemostpopularduetoitsconrvatism,itisactuallywastefulofstatisticalpower

anlpairsof

meansarebeingcompared,Tukey’ialdesignsituations,other

posthocproceduresmayalsobepreferableandshouldbeexploredasalternatives.

©Stevens,1999