conditional

Conditionalindependence

1

Conditionalindependence

llreprentsa

ntsR,BandYarereprentedbytheareasshadedred,blue

probabilitiesoftheeventsareshadedareaswith

examplesRandBareconditionallyindependentgivenY

becau:Pr(RcapBmidY)=Pr(RmidY)Pr(BmidY),Toethatthisistheca,

oneneedstorealithatPr(R∩B|Y)istheprobabilityofanoverlapofRandBintheY

,inthepictureontheleft,therearetwosquareswhereRandBoverlapwithin

theYarea,andtheYareahastwelvesquares,Pr(R∩B|Y)=tfrac{2}{12}=

tfrac{1}{6}.Similarly,Pr(R|Y)=tfrac{4}{12}=tfrac{1}{3}andPr(B|Y)=

tfrac{6}{12}=tfrac{1}{2}.butnotconditionallyindependentgivennotY

becau:Pr(RcapBmidtext{not}Y)not=Pr(Rmidmbox{not}Y)Pr(Bmid

text{not}Y).,

Inprobabilitytheory,twoeventsRand

Bareconditionallyindependent

givenathirdeventYprecilyifthe

occurrenceornon-occurrenceofRand

theoccurrenceornon-occurrenceofB

areindependenteventsintheir

conditionalprobabilitydistribution

rwords,RandBare

conditionallyindependentifandonly

if,givenknowledgeofwhetherY

occurs,knowledgeofwhetherRoccurs

providesnoinformationonthe

likelihoodofBoccurring,and

knowledgeofwhetherBoccurs

providesnoinformationonthe

likehoodofRoccurring.

Inthestandardnotationofprobability

theory,RandBareconditionally

independentgivenYifandonlyif

orequivalently,

TworandomvariablesXandYareconditionallyindependentgivenathirdrandomvariableZifandonlyiftheyare

,XandYareconditionallyindependent

givenZifandonlyif,givenanyvalueofZ,theprobabilitydistributionofXisthesameforallvaluesofYandthe

probabilitydistributionofYisthesameforallvaluesofX.

TwoeventsRandBareconditionallyindependentgivenaσ-algebraΣif

wheredenotestheconditionalexpectationoftheindicatorfunctionoftheevent,,giventhe

,

TworandomvariablesXandYareconditionallyindependentgivenaσ-algebraΣiftheaboveequationholdsforall

Rinσ(X)andBinσ(Y).

TworandomvariablesXandYareconditionallyindependentgivenarandomvariableWiftheyareindependent

givenσ(W):theσ-commonlywritten:

Thisisread"XisindependentofY,givenW";theconditioningappliestothewholestatement:"(Xisindependent

ofY)givenW".

Conditionalindependence

2

IfWassumesacountabletofvalues,thisiquivalenttotheconditionalindependenceofXandYfortheeventsof

theform[W = w].Conditionalindependenceofmorethantwoevents,orofmorethantworandomvariables,is

definedanalogously.

ThefollowingtwoexamplesshowthatX⊥YneitherimpliesnorisimpliedbyX⊥Y|,suppoWis0with

= 0takeXandYtobeindependent,eachhavingthevalue0with

probability0.99, = 1,XandYareagainindependent,butthistimetheytakethevalue

⊥Y | dYaredependent,becauPr(X = 0)

becauPr(X = 0)= 0.5,butifY = 0thenit'sverylikelythatW = 0andthusthatX = 0aswell,so

Pr(X = 0|Y = 0) > condexample,suppoX⊥Y,eachtakingthevalues0and1withprobability 0.5.

LetWbetheproductX×enW = 0,Pr(X = 0) = 2/3,butPr(X = 0|Y = 0) = 1/2,soX ⊥ Y |

inMurphy'stutorial

[2]

whereXandYtakethevalues"brainy"and

"sporty".

UsinBayesianstatistics

Letpbetheproportionofvoterswhowillvote"yes"nganopinionpoll,one

= 1, ..., n,letX

i

= 1or0accordingastheithchonvoterwill

orwillnotvote"yes".

Inafrequentistapproachtostatisticalinferenceonewouldnotattributeanyprobabilitydistributiontop(unlessthe

probabilitiescouldbesomehowinterpretedasrelativefrequenciesofoccurrenceofsomeeventorasproportionsof

somepopulation)andonewouldsaythatX

1

,...,X

n

areindependentrandomvariables.

Bycontrast,inaBayesianapproachtostatisticalinference,onewouldassignaprobabilitydistributiontop

regardlessofthenon-existenceofanysuch"frequency"interpretation,andonewouldconstruetheprobabilitiesas

dmodel,therandomvariables

X

1

, ..., X

n

arenotindependent,icular,ifalarge

numberoftheXsareobrvedtobeequalto1,thatwouldimplyahighconditionalprobability,giventhat

obrvation,thatpisnear1,andthusahighconditionalprobability,giventhatobrvation,thatthenextXtobe

obrvedwillbeequalto1.

Rulesofconditionalindependence

Atofrulesgoverningstatementsofconditionalindependencehavebeenderivedfromthebasicdefinition.

[3][4]

Note:sincetheimplicationsholdforanyprobabilityspace,theywillstillholdifconsidersasub-univerby

conditioningeverythingonanothervariable,mple,wouldalsomeanthat

.

Note:below,thecommacanbereadas,andthuscanbevisualizedasaVennDiagram.

Conditionalindependence

3

Symmetry

Decomposition

Proof:

• (meaningof)

• (ignorevariablebyintegratingitout)

•

repeatprooftoshowindependenceofXandB.

Weakunion

Contraction

Contraction-weak-union-decomposition

Puttingtheabovethreetogether,wehave:

Interction

IftheprobabilitiesofX,A,Bareallpositive,thenthefollowingalsoholds:

References

[1]Toethatthisistheca,oneneedstorealithatPr(R∩B|Y),inthepicture

ontheleft,therearetwosquareswhereRandBoverlapwithintheYarea,andtheYareahastwelvesquares,Pr(R∩B|Y)==.

Similarly,Pr(R|Y)==andPr(B|Y)==.

[2]/~murphyk/Bayes/

[3]Dawid,A.P.(1979)."ConditionalIndependenceinStatisticalTheory".JournaloftheRoyalStatisticalSocietySeriesB41(1):1–31.

2984718.

[4]JPearl,Causality:Models,Reasoning,andInference,2000,CambridgeUniversityPress

ArticleSourcesandContributors

4

ArticleSourcesandContributors

Conditionalindependence Source:/w/?oldid=418113252 Contributors:3mta3,Alansohn,AzaToth,Brighterorange,Btyner,Cesarth,CharlesMatthews,

Circeus,CitrusLover,DGJM,Ddxc,Dominus,Duoduoduo,Epachamo,Fresheneesz,GarethOwen,Giftlite,JeffG.,Mackem,Melcombe,MichaelHardy,Ms2ger,Mtcv,Nasz,Neilc,

Ninjagecko,Ogai,OlegAlexandrov,,Qwfp,Redrocket,Rkashuba,Splat2010,Tsirel,31anonymoudits

ImageSources,LicensandContributors

Image: Source:/w/?title=File:Conditional_ Licen:CreativeCommonsAttribution-Sharealike3.0,2.5,2.0,1.0

Contributors:diaLaterversion(s)dia.

Licen

CreativeCommonsAttribution-ShareAlike3.0Unported

/licens/by-sa/3.0/