Lecture 6:
Classification and marginalization
Bayesian modeling
1.Specify the generative model
2.Specify the inference process: how the obrver computes and reads out the posterior (on a single trial)
3.Calculate behavior across many trials: distribution of the MAP estimate
Generative model
•p (measurements | state of the world)
•Describes the statistical structure of the world and/or task : how the measurements ari stochastically from the state-of-the-world variable.
•Also called noi model or forward model
Generative models en so far
s
x s
x V
x A Combining a measurement with a prior
Cue combination
C x
or
discrete variable
()()()
()()()()1||11log
log 1||11p C x p x C p C d x p C x p x C p C ======−=−=−Log posterior ratio (or log odds ratio):
()()
()()
|11log
log
|11p x C p C p x C p C ===+=−=−log likelihood ratio (LLR)
log prior ratio
Tuesday: binary decisions
()0d x >MAP decision rule:d
Confidence:
Goals for today
•See examples of other generative models in natural and psychophysical tasks
•Understand how to derive the posterior for any generative model: marginalization •Work out a detailed example: visual arch
–Binary decision
–Weighting by reliability
Examples of other generative models
Classification
Two difficulties for an obrver:
•Noi: internal reprentation varies •Ambiguity: different caus, same sound
Ambiguity
a trapezoid
a rectangle on a road
a weird wire frame
C
s
x
class
stimulus
internal reprentation
Generative model
Ambiguity p (s |C )
Sensory noi p (x |s )
Irrelevant variable
Kersten and Yuille, 2003
The same object looks differently when viewed from a different angle.
Generative model
I
s θ
object identity viewing angle
image Each node comes with a probability
distribution
A
B C
B
C
A
A
B
C
p(A)
p(B|A)p (C|A)
How does an obrver do Bayesian inference for a given generative model?
(Step 2)
Inference
Computing the posterior distribution over the state-of-the-world variable bad on the obrvations
A
B C
B
C
A
A
B
C
General approach
1.Compute the joint probability distribution by
“following the arrows”.
2.Compute any conditional probability
distribution by marginalizing the joint
distribution. Conditional independence A
B C
()()()()
,,||
p A B C p A p B A p C A
=
()
()
()
()()()
()()()
,,|| |,
,||
A
p A B C p A p B A p C A p A B C
p B C p A p B A p C A
==∑
Discounting
B
C
A
()()()()
,,|,p A B C p A p B p C A B =()()
()
()()()
()()()
,|,,||,B
A B
p A p B p C A B p A C p A C p C p A p B p C A B ==
∑∑Markov chain
A
B C
()()()()
,,||p A B C p A p B A p C B =()()()
()()()()()
()()()
,,,,||,|,,||B
B
A B
A B
p A B C p A p B A p C B p A C p A C p C p A B C p A p B A p C B =
=
=
∑∑∑∑In computing the posterior, a Bayesian obrver marginalizes (sums or integrates) over every variable in the generative model other than the obrvations and the state-of-the-world variable of interest.
More complex example
B
C D
E
F G
A
Homework: Compute p (A |E,F ) bad on the conditional probabilities indicated in this generative model.
Step 3: Respon/estimate distribution
()ˆargmax |s
s
p s x =MAP estimation
Estimate distribution
()ˆ|p s
s ()
()()()
==>|ˆ1|Pr p x C p C C d x k Continuous:Binary:
P r o b a b i l i t y
ˆs
()ˆ|p s
s Fal alarm rate
D e t e c t i o n r a t e
()0
d x >Binary:Worked example: visual arch
(a bit more general than needed)
Visual arch Visual arch in laboratory
Was the target prent?
Step 1: Generative model
1. Choo trial type (target abnt or prent with equal probability)x x x x x x 3. At each location, t orientation bad on
whether a target or a distractor is there: s T or s D
2. If the target is prent, choo its location (equal probabilities)
4. Internal obrvations (Gaussian noi)
X
T s D
s target distractor
Generative model
T
Global target prence:Yes/no T 1,…,T N
Local target prence:Yes/no
s 1
s 2s N
…
Stimuli
…
x N
x 2x 1Internal
reprentations
Write down the probability distributions
associated with each node
()1212,0,0,0,,...,|0N N T T T p T T T T δδδ== ()112,0
,1,0
1
1
,,...,|1i N N
N T T T i p T T T T N
δ
δδ===
∑ ()()|0i i i D p s T s s δ==−()()|1i i i T p s T s s δ==−()()2
2
221|2i i i x s i i i
p x s e
σπσ
−−
=
()()010.5
p T p T ====Step 2: Inference
Calculate the probability that the target is prent (or abnt) given the internal obrvations of the orientations:
()
121|,,...,N p T x x x =Do this using the generative model and the rules of probability calculus
