The utility of mental images:How to construct stable mental models in an
unstable image medium
Bettina Berendt
Graduiertenkolleg Kognitionswisnschaft
(Doctoral Programme in Cognitive Science),University of Hamburg
Vogt-K¨o lln-Str.30,D-22527Hamburg,Germany
berendt@informatik.uni-hamburg.de
Abstract
We propo a strategy of how to construct uful mental im-ages given that the are inexact.Inexactness is modelled as constructed image element(point)positions reprented prob-abilistically,which leads to slight changes over time in the spa-tial relations extracted from this image by inspection.This may make the extracted information unstable or even incorrect.We show that the propo
d strategy‘regulari’achieves maximal ufulness of mental images constructed in this way if the im-ages are to rve as depictions of mental models,which should be stable over time.Strategy choice is shown to be the optimal solution to a decision problem.We also discuss empirical data which support the claim of the existence of this strategy.
Introduction
It is one of the few relatively uncontroversial statements about mental images that the do not behave like external pic-torial reprentations such as pictures or diagrams in every way.Some differences have been demonstrated by experi-ments showing a failure to make discoveries in rotated mental images(=find emerging shapes)that are immediately obvi-ous if the same image is rotated in perception(Reisberg& Chambers,1991,Slezak,1992).This contrasts withfindings that this kind of discovery is possible when different images are (Finke&Slayton,1988).One explanation for thefindings is that disoveries are possible if the images are simple enough and impossible otherwi,becau imagery is‘inexact’,i.e.cannot reprent and/or processfine details which are nsitive to small metrical changes(,(Lo-gie,1995)).Recent experiments by Denis,Gonc¸alves and Memmi(1995)have demonstrated an‘inexactness’of men-tal images when the do not have to be transformed(ro-tated),but inspected(scanned).The reaction time patterns tha
t Denis and his colleagues obtained led them to propo and model that positions in a mental image are reprented metrically,but within an uncertainty region around the‘exact’value.That a mental image should be‘inexact’in this n is to be expected given current theories of how the imagery system works:for example,Kosslyn(1980,1994)and Logie (1995)suggest a continued re-generation of mental images.
While this explanation is not generally accepted(some claim that interpreted patterns are imaged differently from abstract (Logie,1995,Slezak,1992)),it is difficult to decide this issue given the evidence available so far.
Thanks to Gerhard Strube for suggesting this link to me.
This pos an interesting problem:This inexactness may lead to the inability of imagers to remember certain details in perceived images and/or to operate on them,which is un-avoidable.However,what about images in who construc-tion people have a certain freedom?We assume that often, imagers u that freedom in a way which takes advantage of the capabilities of the imagery system and is minimally af-fected by its limitations–in short,that imagers u visuali-sation strategies to make the most of the resource‘imagery system’.
The aim of this paper is to describe such a strategy,which we call the‘regularisation strategy’.The strategy consists of the avoidance of unnecessary equalities of image elements (points).We model it as(an aspect of)the optimal solution to the decision problem of how to construct mental im-ages which are maximally uful for the construction and reprentation of stable mental models.(It is likely that various strategies cooperate towards that goal.)We havefirst propod the existence of this strategy in(Berendt,1996)–it is the purpo of this paper to derive and describe it formally and to show its general applicability.
The paper is organid as follows:In ction1,we dis-cuss the empirical data whichfirst motivated us to propo the regularisation strategy.In ction2,we define‘uful-ness’or‘utility’of a mental image via the degree of stability of the(spatial)mental model depicted in it.We also define the‘inexactness’of a mental image and the‘stability’of a mental model.In ction3,we show the decision problem which the imager faces,and show that the solution involves regularisation.In ction4,we show an example of the other step,the choice of image construction and inspection param-eters.We conclude by discussing some open questions and further rearch problems.
Empirical motivation and support for the
regularisation strategy
As we have pointed out above,we are interested in strategies that people employ when they have a certain control over how The equalities may also be called‘singularities’as oppod to‘regularities’,which gives the strategy its name.
We shall talk about“optimal solutions whether they are the re-sult of a totally unconstrained maximisation,a maximisation un-der numerical constraints,or a maximisation under constraints of bounded rationality.火腿的做法
1
they construct a mental what they choo to depict in it.This is only relevant where there are veral possibili-ties of what the image may various shapes and numbers of image elements.If the image is ud tofind the solution to a task,all the possibilities should be valid solu-tions to the task.This kind of non-uniqueness of solutions is one of the key subjects of mental models rearch.Here,we concentrate on mental models generated in order to answer questions about relations between entities bad on a logic describing the relations,a generalisation of the three-term ries introduced by Johnson-Laird(1972),and we concen-trate on spatial mental models,since we assume their repre-ntation is an image.In this context,the veral possibilities of what the imag
e may contain can be detailed as follows: If people u imagery to construct mental models and have some freedom in the answer they give,can knowledge about the imagery system help us predict which of all the logically possible mental models are more likely to be constructed? We assume that asking subjects to just give one possible an-swer will give us a mental model which is more likely to be constructed,becau its reprentation has properties which make the best u of the system it is reprented in,which in our ca is the imagery system.
The data that motivated our investigations were obtained by Knauff,Rauh and Schlieder(1995),who ud Allen infer-ences as three-term ries.The logic underlying the infer-ences is that of the‘Allen relations’.The are the relations which can hold between two intervals,corresponding to re-lations between the start-and endpoints of the two intervals, between which only the ordinal relations’is before/smaller than’,’is equal to’,and’is after/larger than’are distinguished. Apart from equality,there are12such relations.Examples of Allen relations and a typical Allen inference are shown infig.1,for full details,(Allen,1983,Knauff et al.,1995).As in other mental models tasks,the interesting inference questions are tho that have more than one logi-cally possible solution.Knauff et al.(1995)investigated this question by training subjects in the understanding of Allen relations and then asking them to provide one answer to each of the Allen inference questions.They aggregated the an
swers over subjects and found evidence of preferred mental models:In multiple-solution compositions(72out of the144 compositions),a great majority of subjects cho the same solution.
In an attempt to characteri the preferences,we ob-rved(Berendt,1996)that in nearly all multiple-model cas,the preferred mental models were characterid by an avoidance of possible,but unnecessary equalities.For exam-ple,the preferred mental model of the composition infig.1 was“is during”and not“starts”.This obr-vation led us to postulate the existence of the visualisation strategy‘regulari’,on the basis of which we formulated a “overlaps”would also be a regular solution.However, the decision which regular solution to choo is made at a different level.See(Berendt,1996)for a discussion of this question.
Figure1:A typical Allen inference showing examples of Allen relations between intervals,the logically possible mod-els,and the empirically preferred mental model as found by Knauff et al.(1995)
very simple metrical computational theory of mental images containing mental models formed in doing Allen inferences (Berendt,1996).Here,we shall develop this idea further by showing formally why regularising is a strategy which is gen-erally uful for images of spatial mental models.
Of cour,our computational theory does not prove that
a metrical image construction process is indeed employed
in Allen inferences,as the existence of an alternative the-ory bad on ordinal information only(Schlieder,in prep.) shows(for a comparison of the two computational theories, e(Berendt,1996)).However,even though the ordinal the-ory can also describe mental model construction in an image (Schlieder,1996),its underlying assumption that a repren-tation of the ordinal relations between the interval start-and endpoints is fundamental for construction and inspection pro-cess cannot explain thefinding that possible,but unneces-sary equalities em to be avoided.As we shall show below, our assumption that a reprentation of metrical relations be-tween the interval start-and endpoints is fundamental for con-struction and inspection process can
explain thefinding.
It may be argued that the prevalence of inequalities in the results is just the result of simple statistics:If there is any noi in the image,it is highly unlikely that two points,even if constructed at the same spot,are found to be equal at in-spection.However,this in itlf would not explain an ap-parent systematicity in the preferred mental models,which ems to be the outcome of a systematic feature of the con-struction process.We argue that the construction process is purpofully‘designed’in order to implement the visu-alisation strategy.Also,only systematic construction and inspection process are able to ensure stable mental mod-els,which is of general importance if images are to be ud as reprentation medium for mental models.
2
Utility of mental images:inexactness of mental images vs.stability of mental models
The utility of a mental image is its ability to assist the rea-soner in whatever task she is trying to accomplish.This abil-ity depends on the ability of the mental image to reprent tho properties of the reprented domain and the ability of the imagery system to perform tho construction,trans-formation,and inspection operations on this image wich are needed to obtain the required answer.‘R
eprenting’involves maintenance:What information is extracted from inspection of the image should remain stable over time.What the properties and operations are depends on the task at hand.In many tasks,knowledge about the order or arrangement of im-age elements is needed.This is the ca in tasks like compar-ing the relative heights of people or the left/right/front/back relations between objects as ud (Mani&Johnson-Laird,1982).In Allen inferences,one needs to be able to reprent and extract the ordinal relations between the inter-vals’start-and endpoints to determine the answer(the Allen relation between intervals and,which depends on the or-dinal relations between the start-and endpoints and only on them).
If mental images are inexact,the information extracted from inspection of an image varies metrically with time.We model this inexactness as a conquence of an unstable im-age medium as follows:Let be the description of the intended position of an image element in terms of the ref-erence frame ud.For simplicity,we consider only points as image elements(the inexactness of image elements like lines may behave differently).The intended position is computed by a construction process.In general,it might be described as a two-dimensional position in a2D image;in the ca of Allen inferences,it might also be a2D position,as ourfigures may suggest,but it might also be a1D position,since the task itlf concerns only one dimension(e(Schlieder,1996)for a discussion of var
ious possibilities).Let measure the dis-persion characteristic of and constant throughout the image medium.Then image element is reprented as a random variable with mean and measure of stan-dard deviation).Every inspection operationfirst retrieves a value from this distribution,which is then interpreted at a certain granularity modelled by a grid of size,producing an inspection value.
Instability can occur in two forms:Two points con-structed as equal can appear unequal on inspection(Constr-Eq:,but).We call the conditional probability of this error.Alternatively,two points constructed as unequal can appear equal or unequal in the wrong direction on inspection(Constr-Non-Eq: ,but,or,but
).We call the conditional probability of this error. Fig.2shows why this kind of error is a problem.The im-age construction strategy determines how many point pairs This is what we called‘sketch interpretation’in(Berendt, 1996).
Regular image:
Non-regular image:
UNSTABLE
mental model
mental model
STABLE If the constructed value of C’s startpoint is ...
"A is during C"
- smaller than that of A (<=> mental model ):
A
C
B
"A is during C".
every inspection will very likely yield a different relation:
"C’s startpoint is smaller than/larger than/equal to A’s startpoint"
and therefore DIFFERENT mental models:
every inspection will very likely yield the same relation:
"A is during/starts/overlaps C".
C
B
A
Figure2:Avoiding equalities leads to more stable men-tal models(example Allen inferences,e abvove).Circles around a constructed value indicate the regions in which val-ues are found at inspection.
are constructed as equal or unequal,respectively.Therefore the overall probability of making an inspection error in the comparison of two points is
error Constr-Eq Constr-Non-Eq
(1)
The lower the overall probability of making such errors, the more stable the mental model.We therefore define the (expected)utility of an image as the overall probability of not making such errors for a given pair of points(since one does not know a priori how many pairs of points one will have to examine,this ems a nsible measure):
Constr-Eq Constr-Non-Eq
(2)
Since any two points are either constructed as equal or as un-equal,we have
Constr-Eq Constr-Non-Eq(3) so
Constr-Non-Eq(4) It is very likely that this is only one aspect of a general de-scription of‘stability’or more general‘quality’of mental image reprentations of mental models.Therefore,various strategies will probably cooperate towards that goal.We be-lieve,however,that the aspect we describe can be treated in-dependently,and that the optimisation of this utility is a nec-essary condition of any optimal solution to the general‘sta-bility’or‘quality’problem.
3
Regularisation as utility maximisation
A mental model is(maximally)stable iff the probability of errors is iff utility is maximal.This allows us to model the situation underlying this strategy as a deci-sion problem:The imager has control over a certain t of parameters,and if she is‘rational’,she will t the in a way that maximis the expected utility.As with other strategies, of cour,this need not be a conscious decision.We model images as determined by the following parameters:
their‘degree of inexactness’,modelled by measure of dis-persion,which the imager has no control over,
their‘degree of regularity’,which is modelled by Constr-Non-Eq.This has a lower bound:the pro-portion of point pairs that must be constructed as unequal, since they must be different points.It also has an upper bound given by1minus the proportion of point pairs that must be constructed as equal,since they must be the same point.All remaining point pairs are‘regularisation can-didates’,i.e.they can,but do not have to be constructed as unequal.For simplicity,we assume that to regulari means to construct all of the as unequal,so that there are only two values of.
the minimum distance between two points constructed as unequal,which is influenced by the construction strat-egy(and therefore assumed to be under the control of the imager),and
the‘level of granularity’at which the image is inspected, modelled by the size of a grid laid over the image(again assumed to be under the control of the imager).( (Buckley&Gillmann,1974),for an early account offlexi-ble u of such a grid on mental reprentations.)
In principle,the utility of the image and therefore its op-timal solution depend on all three parameters under the im-ager’s control.So ideally,the parameters should be t con-jointly when all the information about an image is there. However,it ems nsible to make the following assump-tions in addition to image inexactness:
bounded rationality:The imager does not know exactly all the relevant parameters,and she is unwilling or unable to perform a new optimisation before the generation of each new image.
limited maximal size of the image:This corresponds to the size-resolution tradeoff in mental images as found,for ex-ample,by Finke and Kosslyn(1980).It implies that min-imum distance(and therefore also grid size)cannot be t at arbitrarily high values,since otherwi,one might ‘run out of space’in imaging.Also,smaller images may generally be more desirable,becau they economi on scanning
operations.
a generally higher proportion of necessarily unequal points than of necessarily equal points among the t of all images
generated:this is a conquence of what the objects look like that images are usually ud to depict.
写给女朋友的情书The three assumptions lead to the likelihood of a generally employed image spacing strategy,which is the numerical
relationship between minimum distance and grid size in
dependence on dispersion and maximimum image size.We
再造句
assume that the values of and are kept constant over dif-
ferent images,and that they reprent‘nsible’(i.e.opti-mising)values under the assumption that there are generally
more points to be distinguished than there are points intended
to be equal.Moreover,they are the result of constrained op-
timisation:Becau of image size limitations,they tend to
be rather smaller than their unconstrained optima.We will
show an example of an optimal‘image spacing strategy’in the next ction.It should be pointed out that becau of all
the default assumptions and the usage of their result(the
image spacing strategy),generated images will not always be
徒善不足以为政the best hey should be en as the result of sat-
isficing.
Under the assumptions,the decision problem is greatly
simplified:The problem is reduced to deciding whether it is
worthwhile to regulari.Fig.3shows the structure of the
decision problem.is the proportion of points constructed
as unequal(indexed by strategy),and and are the prob-
abilities of errors as in equation(1).(Assigning utilities of1 and0,respectively,to making no error or an error,shows how
the overall utility is derived.)
It is unlikely that the proportion of either pairs of points
which have to be constructed as unequal anyway or the pro-
portion of regularisation candidates is known a priori.There-
fore,‘regulari’is only a nsible choice if it increas utility over a wide range of the proportions.We have tofind out
whether the utility increas when one when
the proportion of point pairs constructed as unequal increas: reg no-reg):
reg no-reg(5) It is worthwhile to regulari iff.
This is indeed the ca:There are always many ways of constructing and inspecting images which satisfy this prop-
erty.To e why,consider the dependency of the inspection
errors on the controllable parameters of construction(mini-
mum distance)and inspection(grid size):For grid size
0,the probability of two random variables being in the same
grid cell(=taking on the exact same value)is0.So,the probability of two points constructed as equal being read as
unequal will be maximal().,the probability of two points constructed as unequal with distance being read
as equal or unequal in the wrong direction will be minimal
().(Since the probability of a error for point pairs constructed as unequal at distances larger than is even It must be acknowledged that at the moment,this is a claim rather than a proven statement.The assumption ems intuitive given common images,but needs more detailed investigation.
4
α
1-1-β
1-β
α
1-reg
reg
+ n U = (1-n )reg
α
(1- )β(1- )β(1- ) + n no-reg
α
(1- )regulari
don’t
regulari
Constr-Eq
Constr-Non-Eq
n no-reg
no-reg
(1-n )no error no error
error
error
αβ
0000
1
1
1
1
Constr-Eq
Constr-Non-Eq
reg
(1-n )n reg
error
error
no error no error
从零开始练瑜伽αβ
no-reg no-reg
(1-n )U =
Figure 3:The decision tree showing the possible outcomes depending on whether an image is regularid or not (e text).The box denotes a decision node,circles denote ran-dom events during inspection,with probabilities shown at the outgoing branches.
lower,we feel safe in using this upper bound on in our def-inition of utility.)For any given ,enlarging the grid will reduce the error,but increa the error.Increasing will not affect the error (since only changes the relation of points constructed as unequal),but decrea the error for any given grid size:The further the means of two random variables are apart,the less likely it is that they take on val-
ues clo to each other or in the wrong order.This produces curves like tho shown in fig.4(the exact shape depends on the distribution).Variations in also only decrea the value at which the and curves interct,,but do not change the relative positions of the two functions.So there are always combinations of and such that .And the are small values of both and ;with small relative to ,since this is the better choice for images with higher proportions of inequalities.In other words,it is very likely that and derived from the general image spacing strategy will have values which make ,which means that regularisation is worthwhile.
Since this result is true over a wide range of images (and over a wide range of tho fixed parameters that might change the optimal solution),the decision problem remains the same throughout a wide range of situations,which makes it nsi-ble to assume that it is not reconsidered each time,but ‘com-piled’into a generally applied strategy.
For particular class of tasks,the strategy has to be ud as a basis for a construction process.For the class ‘Allen inferences’,we show this in (Berendt,1996).In the design
g
1
Pr
-> increas ->
m
α
β
β1
2
学习实践活动Figure 4:and errors depending on minimum distance
and grid size .is only affected by ;the curve shifts downward for increasing .
of this construction process,it should be kept in mind that the process should produce distances between points that are not smaller than the minimum distance identified by the solu-tion to the construction and inspection parameter optimisation problem.At the same time,the image must not be
come too large,becau otherwi the imager might ‘run out of space’or have to employ complex image transformation operations to continue using the image.论文形式
An example of an image spacing strategy
An image spacing strategy consists in the choice of the best values for the image construction parameter minimum dis-tance and the image inspection parameter grid size .Here,we will show an example of the design of such a strategy under the following simplifying,but generalisable,assump-tions:
The reprentation in the image is esntially 1D.This can be the ca,for example,in Allen inferences (e ction 3).By ‘esntially’,we mean that the comparison opera-tions are only carried out from left to right,so that we could in principle have mental images that look like fig.s 1and 2,where we abstract from the possibility that the bars rep-renting the Allen intervals change their vertical position in a way which adverly affects inspection for 1D order relations.
Randomness can be reprented by a distribution over a finite interval of length ,which is centered at the con-structed value of a point.
Maximal image size is large enough.We show this un-constrained optimum becau we believe it makes things clearer.A size limitation may necessitate smaller values
立志小故事For this reason,the exact mathematics of movement adjustment in the construction process in (Berendt,1996)may have to be refor-mulated,without however changing the qualitative process descrip-tion.Thanks to Christoph Schlieder for pointing out this problem to me.
5
(b)
α = 1/2(a)α = 0g
m
(c)
β = 0
(d)β = 0
Figure 5:Optimal values for minimum distance and grid size if uncertainty is limited to a finite interval
of length ,and image size is large enough not to be constraining.(a)best possible grid alignment for error;(b)worst possible grid alignment for error;(a)best possible grid alignment for error;(b)worst possible grid alignment for error
of and ,who conquences we have discusd in the preceding ction.
The problem of choosing an optimal -combination is similar to problems of choosing the optimal sampling rate in measurement theory (given ,choo ).Here,we shall only discuss the importance of the results for our argument here.It can be shown easily that for all (i.e.some points must be constructed equal,becau they are intended to be the same),the optimal minimum distance is and the optimum grid size is .For the unlikely ca that (all points are intended to be different),optimal is and optimal 0.The most important point to note is that the optimal values do not depend on .The values of and are shown in fig.5.So if space is not a problem (and could be t to their optimal values):either grid alignment is no problem,,and utility is independent of ,so regularisation caus no decrea in utility,or grid align-ment is a problem,
,and regularisation is an advantage.However,as soon as space is a and are smaller their optimal values such that and regularisation is an advantage.
When the uncertainty interval is not fiin cas where the distribution can be modelled by a normal distribu-tion,the shapes of the functions change,with the main result that optimal grid size and minimum distance decrea contin-uously with .Since this does not affect the principal argu-ment,we will not discuss this ca further here.
Some open questions and directions for further
work
We have shown in this paper that on the background theory that mental images are inexact,the visualisation strategy ‘reg-ulari’is (an aspect of)the best solution to the decision prob-lem of how to construct images of maximally stable mental models.We have also shown (Berendt,1996)for a partic-ular class of mental model construction tasks that the strat-egy immediately leads to the formulation of a very simple construction process,which provides a computational theory which explains empirical data obtained for this task well.Ob-viously,further experiments designed to test directly for the existence of this strategy are needed.Also,the assumptions about image construction and inspection parameters that we have described in this paper should be subjected to empirical scrutiny.
In this context,the model of image inexactness and its con-quences will have to be refined.For example,it is likely that inexactness is not uniform across the image.Also,we are working on relating our model of inexactness to other com-monly ud models like resolution decrea over time.A very crucial aspect of our model is the process of re-generating the image.We have assumed that this is an unbiad pro-cess in the n of the values being disperd around the same mean value (=the ‘constructed value’)in concutive inspection steps.This requires the assumption of some un-derlying model,from which the ‘constructed value’can be retrieved or regenerated.If there is no such model,or it is not ud,as Kosslyn’s (1980)operation REGENERA TE as-sumes,a re-drawing at time would more likely depend on the value at time than on the ‘constructed value’,which would better be described by a random variable with a new mean.This would not change the general argument put forward here,but the exact ramifications need to be in-vestigated.More wide-ranging questions about this putative model include:What kind of information does it is it an ordinal or a metrical model (e also (Schlieder,in prep.,Berendt,1996,Schlieder,1996),or do various models interact (like Kosslyn’s (1994)‘categorical’and ‘coordinate’models)?Are there different class of models accounting for the different behaviour in imagery tasks of different tasks and stimuli?Are different models operated on differently,and how?Further rearch is clearly needed to model and test the various possibilities.
Acknowledgements
I wish to thank Christoph Schlieder,Gerhard Strube,Markus Knauff,Reinhold Rauh,Marina Yoveva,Alexander Petrov,Peter Slezak,Bruce Burns and Longin Latecki for discus-sion and comments,and the Graduiertenkolleg Kognition-swisnschaft (Doctoral Programme in Cognitive Science)of Hamburg University for financing my work.6
