Feedback-labelling synergies in judgmental stock price forecasting
Paul Goodwin a,*,Dilek O自我介绍英语作文
¨nkal-Atay b,1,Mary E.Thomson c,2,Andrew C.Pollock d,3,Alex Macaulay d,4
a
The Management School,University of Bath,Claverton Down,Bath BA27AY,UK b
Faculty of Business Administration,Bilkent University,Ankara 06533,Turkey c
Glasgow Caledonian Business School,Cowcaddens Road,Glasgow G40BA,UK
ncha touchd
Division of Mathematics,School of Computing and Mathematical Sciences,Glasgow Caledonian University,
Cowcaddens Road,Glasgow G40BA,UK
Received 1October 2002;accepted 5December 2002
Abstract
Rearch has suggested that outcome feedback is less effective than other forms of feedback in promoting learning by urs of decision support systems.However,if circumstances can be identified where the effectiveness of outcome feedback can be improved,this offers considerable advantages,given its lower computational demands,ea of understanding and immediacy.An experiment in stock price forecasting was ud to compare the effectiveness of outcome and performance feedback:(i)when different forms of probability forecast were required,and (ii)with and without the prence of contextual information provided as labels.For interval forecasts,the effectiveness of outcome feedback came clo to that of performance feedback,as long as labels were provided.For directional probability forecasts,outcome feedback was not effective,even if labels were supplied.Implications are discusd and future rearch directions are suggested.D 2003Elvier B.V .All rights rerved.
Keywords:Forecasting;Judgment;Feedback;Calibration;Stock price;Contextual information
1.Introduction
Forecasting and decision support systems are partly systems for learning.One of their objectives is t
o
improve management judgment by fostering under-standing and insights and by allowing appropriate access to relevant information [16].Feedback is the key information element of systems that are intended to help urs to learn.By providing managers with timely feedback,it is hoped that they will learn about the deficiencies in their current judgmental strategies and hence enhance the strategies over time.When a system is being ud to support forecasting,feedback can be provided in a number of forms [6,10].The simplest form is outcome feedback ,where the manager is simply informed of the actual outcome of an event that was being forecasted.Performance feedback pro-vides the forecaster with a measure of his or her
0167-9236/$-e front matter D 2003Elvier B.V .All rights rerved.doi:10.1016/S0167-9236(03)00002-2
leafage*Corresponding author.Tel.:+44-122-538-3594;fax:+44-122-582-6473.
E-mail address:mnspg@management.bath.ac.uk
(P.Goodwin),onkal@ (D.O
¨nkal-Atay),al.ac.uk (M.E.Thomson),a.c.pollock@gcal.ac.uk (A.C.Pollock),A.B.Macaulay@gcal.ac.uk (A.Macaulay).
1
Tel.:+90-312-290-1596;fax:+90-312-266-4958.2
Tel.:+44-141-331-8954;fax:+44-141-331-3229.3
Tel.:+44-141-331-3613;fax:+44-141-331-3608.4
maintain名词Tel.:+44-141-331-3052;fax:+44-141-331-3608./locate/dsw
Decision Support Systems 37(2004)175–
186
forecasting accuracy or bias.Process feedback in-volves the estimation of a model of the forecaster’s judgmental strategy.By feeding this model back to the forecaster,it is hoped that insights will be gained into possible ways of improving this strategy.Finally,task properties feedback delivers statistical information on the forecasting task (e.g.it may provide statistical measures of trends or correlations between the forecast variable and independent variables).
Most of the rearch literature on management judgment under uncertainty suggests that outcome feedback is less effective than other forms in promot-ing learning (e.g.Refs.[6,33]).For example,much rearch into the accuracy of judgmental forecasts has found that forecasters tend to focus too much on the latest obrvation (e.g.the latest stock value)which will inevitably contain noi.The result is that they e evidence of new,but fal,systematic patterns in the latest obrvation [31]and overreact to it.Becau outcome feedback draws attention to the latest obr-vation it exacerbates this tendency.This means that a long ries of trials may be needed to distinguish between the systematic and random elements of the information received by the forecaster [31].5In con-trast,by averaging results over more than one period (or over more than one ries if cross-ctional data is being ud),other forms of feedback are likely to reduce the attention
that is paid to the most recent obrvation and to filter out the noi from the feed-back.For example,performance feedback may be prented in the form of the mean forecast error,or in the ca of categorical forecasts,the percentage of forecasts that were correct.
However,if conditions could be found where out-come feedback does encourage learning as efficiently (or nearly as efficiently)as other forms of feedback,then this would yield considerable benefits to urs and designers of support systems.This is becau outcome feedback overcomes,or at least reduces,various shortcomings of the other forms.
Firstly,outcome feedback is easier to provide and is likely to be more easily understood by the forecaster.
Converly,the provision of performance feedback,for instance,can involve difficult choices on which performance measure to provide—each measure will only relate to one aspect of performance,but providing veral measures may confu the forecaster.More-over,some measures may be difficult to comprehend and will therefore require that the forecaster is trained in their u.Process feedback will require the identi-fication of cues that the forecaster is assumed to be using,with no guarantee that the cues have really been ud.Also,multicollinearity in the cues means that there will be la
rge standard errors associated with the estimates of the weights that the forecaster is attaching to the cues.Task properties feedback requires regular statistical patterns in past data.By defini-tion,the characteristics are often abnt in tasks where management judgment is preferred to statis-tical methods.
Secondly,when judgments are being made in re-lation to a single variable over time ,outcome feedback will not be contaminated by old obrvations when circumstances are changing.Becau performance and process feedback are measured over a number of periods,they may lag behind changing performance or changes in the strategies being ud by the fore-caster.Also,veral periods must elap before a meaningful measure of performance,or a reliable model of the judgmental process,can be obtained.For cross-ctional data ,outcome feedback can be provided for each variable and,as such,is not merely an average of potentially different performances (or strategies)on different types of ries.Furthermore,a reasonably large number of judgments over different ries are required in order to obtain reliable estimates of performance or a reliable estimate of the process model.
As we discuss below,there are some indications in the literature of situations that may be favourable to outcome feedback.The relate to (i)the nature of the forecast that is required,and (ii)the type of in
forma-tion that is supplied with the feedback—in particular,whether the past history of the forecast variable is accompanied by an informative label.
This paper describes an experiment that was ud to investigate the effects of the factors in an important application area:stock price forecasting.Financial forecasting is an area where human judgment is particularly prevalent [8,35,45]and the specific role
5It is possible that,in some circumstances,outcome feedback is actually damaging to the quality of judgments.However,since in most practical forecasting tasks it will be difficult to avoid the forecaster having access to outcome feedback,the identification of factors that will mitigate its effects would then be of interest.
P .Goodwin et al./Decision Support Systems 37(2004)175–186
英标读音176
第83届奥斯卡颁奖典礼of judgment in forecasting stock prices has itlf received particular attention from the rearch com-munity(e Refs.[7,21,26,32–34,41,48]).The paper compares the effectiveness of outcome feedback under different conditions with that of performance feed-back.Performance feedback was ud as th
e bench-mark becau,of the other feedback types,it is likely to be the most relevant to financial forecasting and most acceptable to forecasters.The paper is structured as follows.First,a literature review is ud to explain why outcome feedback may be more effective when particular types of forecasts are required and why feedback type and label provision might be expected to have interactive effects.Then details of the experi-ment are discusd,followed by analysis and discus-sion.The paper concludes with suggestions for further rearch.
2.Literature review
2.1.Feedback and type of forecast
There is some evidence in the literature that the effectiveness of outcome feedback is related to the nature of the forecast that is required.It ems that outcome feedback is unlikely to be effective when point forecasts are Ref.[24]).Point forecasts merely provide an estimate of the exact value that the forecast variable will assume at a specified time in the he stock price of company X at the end of trading tomorrow will be$3).As indicated earlier,this is probably becau outcome feedback exacerbates the tendency to read system into the noi that is associated with the most recent obrvation. Point forecasts fail to communicate the level of uncer-tainty that is asso
ciated with the forecast.In contrast, judgmental interval ‘‘I am90%con-fident that the closing stock price will be between$2.4 and$3.6’’)do indicate the level of uncertainty,and there is some evidence that outcome feedback is effective in improving the.Usually,the estimated intervals are too narrow for the specified level of confidence,but a study by O’Connor and Lawrence [29]found that outcome feedback was effective in widening the intervals.This may be becau the differ-ence between the reported outcome and the original forecast draws attention to the inherent uncertainty associated with the forecasting task.There is also some evidence that categorical probability the probability that it will rain during the next24h)can be improved by outcome feedback.Indeed,the almost perfect calibration of US weather forecasters has been partly attributed to the fact that the forecasters receive regular and speedy outcome feedback relating to their forecasts[3].Directional probability ‘‘I am80%confident that the stock price at the end of trading in ven days time will be lower than the current price’’)6can be regarded as a special ca of categorical probability forecasts and may therefore also benefit from outcome feedback.However,in the financial forecasting context,O¨nkal and Muradoglu [33]have shown performance feedback to be more effective than simple outcome feedback in improving the accuracy of stock price forecasts expresd as probabilities over multiple price-change intervals. 2.2.The effect of providing labels
In helping the forecaster,a support system can provide various levels and types of information.Time ries information indicates the past history of the forecast variable,enabling trends or other patterns to be identified and the volatility of the variable to be assd.Contextual information refers to information about the forecast variable over and above the ries history.For example,it might refer to information of a company takeover.It also includes labels,which simply indicate the nature of the he name of the company who past stock prices are being displayed).As we indicate below,rearch suggests that labels can have a profound effect on judgmental forecasts.It is also notable that many financial fore-casters ba their estimates only on time ries he u of specific labels is abnt).For instance,chartists do not u any contextual informa-tion due to their belief that all indicators of economic,political,psychological or otherwi)are reflected in the pattern of the price ries itlf and, therefore,a study of past price movements is all that is needed to forecast future movements[27,28].
Labels are a particularly interesting form of con-textual information that can have powerful effects on
美国大城市排名6This type of forecast is preferred over the multiple-interval format by both financial professionals[44]and theorists[17].
P.Goodwin et al./Decision Support Systems37(2004)175–186177
the accuracy of judgmental forecasts.The effects can occur becau labels create expectations about the form and nature of the time ries[39].Labels which create expectations that are congruent with the statis-tical structure of the task can improve the accuracy of prediction becau they increa knowledge of this structure.They also improve the consistency of pre-diction becau they reduce the need to arch for and test a large variety of hypothes about the nature of the data.However,Sniezek[39]found that even neutral labels which give a context to the task,but give no information about statistical structure, such as‘‘weather’’and‘‘marketing variable’’)can aid performance,possibly becau they offer a context to a task that allows the judge to create,with the obrved data,a congruent,and hence consistent,interpretation. Even non-expert forecasters may benefit from labels in this way.
A concept that is cloly related to congruence is label specificity.For example a graph can simply display the general label‘‘sales’’or it could display the more specific label‘‘sales of mobile phones by the Acme phone company’’.It is possible that specificity can have a profound effect on the way a ries is interpreted.Beach et al[9]have distinguished between the u of aleatory and epistemic strategies in judgmental forecasting.Aleatory strategies catego-ri elements by their class membership,rather than their unique iven that you are a member of a particular profe
ssion,you have a70% probability of living to an age of80or more).If only a general label like‘‘sales’’is prented,a time ries can only be en as a member of the class of sales time ries which may be perceived to behave in a stereotypical way.For example,consider the u of the label‘‘sales’’in a judgmental forecasting study by Lawrence and Makridakis[19].Despite the fact that graphs of the time ries manifested an upward linear trend,the label may have caud subjects to forecast damped growth,becau sales ries typically have this pattern.Hence the u of general labels may cau forecasters to pay less attention to the specific char-acteristics of the ries and more to the perceived characteristics of the stereotypical pattern.In forecast-ing stock prices,this may involve perceptions such as ‘‘recent gains are usually subquently reduced by profit taking’’.Nonspecialist forecasters may not have such perceptions and they may have difficulty in making any n of movements in the stock price time ries.
bathingsuit>美国学院In contrast,epistemic strategies u information on the unique characteristics of the element in question (u are30years old,eat healthily,exerci regularly,do not smoke etc.,so you have85%chance of living to an age of80or more).Providing a specific label might therefore be expected to promote epistemic reasoning with a greater focus on the individual features of the time ries.This may be beneficial if it enables the forecaster to incorporate company-spe-cific knowledge
into the interpretation of the graph and the forecast—with‘important’movements in the time ries being more salient and more meaningful.For example,in stock price forecasting,this may involve considerations like‘‘this company is in the aerospace industry and given recent bad news about this industry I expect the slight downward movement in the share price to continue’’.It will be detrimental if it encour-ages the forecaster to attempt to explain specific movements in the ries that are best regarded as noi [13].
Finally,Beach et al.[9]have also suggested that one of the determinants of the motivation to produce accurate forecasts is the quality and amount of infor-mation available to the forecaster—other things being equal,the more adequate the information,the greater the expectation of forecasting accuracy.Hence,the motivation for accurate forecasts may be expected to increa if‘general labels’are replaced by more specific labels.Again,this means that even non-expert subjects may be expected to improve their perform-ance as the specificity of the labels increas.
2.3.The interaction between labels and feedback
One important area that has been underexplored in the literature is the possible interaction between feed-back types and the extent of availability of contextual information.Yet such interactions may be
of consid-erable interest.For example,in a cue probability learning task,Adelman[1]found that the provision of congruent labels led to no difference in performance in a cue probability learning task between task proper-ties and outcome feedback.Adelman suggested that this may have resulted becau the labels implicitly provided accurate information about the statistical structure of the task thereby matching the information
P.Goodwin et al./Decision Support Systems37(2004)175–186 178
that was explicitly provided by the task properties feedback.It is thus possible that,by providing advance information about the data,meaningful labels can add to task knowledge and enhance the rate of learning that would be achieved by feedback alone.
In a judgmental time ries forecasting task,it is possible to hypothesi about the effect of interactions between the specificity of the label provided and the type of feedback.Both the labels and the feedback can be viewed in terms of their likely effects on the attention that the judgmental forecaster will pay to the time ries pattern.At one extreme,the abnce of specific labels and the provision of only outcome feedback are both likely to reduce the salience of the overall time ries pattern provided and encourage a focus on the most recent value.When specific labels are provided
with the outcome feedback,they are likely to increa the salience and meaningfulness of the particular time ries pattern and hence improve forecast accuracy.In particular,considering the entire time ries should improve the forecaster’s asssment of the amount of uncertainty associated with the forecast variable and hence improve interval forecasts so that their width is more appropriate for the level of confidence that is being expresd.
Where performance feedback is a summary meas-ure taken across a number of time ries,it should rve to alert the forecaster to general deficiencies in his or her forecasting strategy and engender reflection on how improvements might be achieved.This would also encourage the forecaster to attend to the entire time ries pattern,even when no specific labels are supplied(and even where outcome feedback is also provided)and may account for the benefits of perform-ance feedback reported in the O¨nkal and Muradoglu study[33].The interesting question is whether provid-ing specific labels yields any added value in the prence of performance feedback.If both specific labels and performance feedback improve perform-ance by directing attention to the overall time ries pattern,then the effect of one of the information types may be subsumed within that of the other.
advancement3.Method
Participants were undergraduate business students from Bilkent University who were taking a forecasting cour.Participation was voluntary and no compen-sation was provided.The subjects were randomly assigned to four groups bad on the type of feedback (outcome vs.performance feedback)and the provision of labels(names of stocks provided). Performance feedback was provided in the form of calibration feedback.Calibration refers to the corre-spondence between the forecaster’s probabilities and the attained proportion of correct forecasts.For exam-ple,if a perfectly calibrated forecaster is expressing his or her forecasts as90%prediction intervals,we would expect90%of the intervals to include the true value of the variable being forecast.Similarly,when this forecaster states that her/she is80%confident that a stock price will move in a particular direction,we would expect the predicted direction to be correct on 80%of occasions.Calibration is an integral aspect of performance and is therefore a natural choice for performance feedback(for extensive reviews of this literature,e Refs.[22,25]).Note that all subjects were provided with outcome feedback in order to provide a realistic simulation of stock market fore-casting.In a practical situation,it is very unlikely that a forecast would be made without the forecaster having knowledge of the most recent obrvation. Moreover,the denial of access to this information would mean that the task became progressively more difficult as forecasters were forced to make forecasts with increasing lead times.It would therefore mean that lead time was confounded with abs
ence of out-come feedback in the experimental design.Note also that the designation‘‘no-labels’’(below)means that ‘‘no specific labels’’were provided since a general label‘‘stock prices’’is implied by the nature of the task.
Fifty-nine students completed the3-week long experiment.In particular,the groups were organized as follows:
G1:outcome feedback,no-labels group(n=14), G2:outcome feedback,labels group(n=12),
G3:calibration feedback,no-labels group(n=17), G4:calibration feedback,labels group(n=16).
For each of the three ssions,participants were requested to provide weekly interval and probability forecasts for the closing stock prices of30randomly lected companies from the Istanbul Stock Exchange.
P.Goodwin et al./Decision Support Systems37(2004)175–186179
Selection of the weekly forecast horizon was dictated by the conditions prevailing in emerging markets.Ordering of 30stocks was randomized individually for each subject for each ssion.All subjects were given the weekly closing stock prices (i.e.the closing stock prices for each Friday)for t
he previous 52weeks in graphical form;and,so that subjects had appropriate information to provide numerical values for credible intervals,the data were also prented for the previous 12weeks in tabular form.The name of each stock was provided to the subjects in the labels groups (i.e.G2and G4),whereas the stock names were not revealed to the subjects in the no-labels groups (i.e.G1and G3).At the beginning of cond and third ssions,participants in G1and G2received outcome feedback (i.e.previous Friday’s closing pri-ces marked on the graphical and tabular information forms).Subjects in G3and G4received calibration feedback in addition to the outcome feedback.Specif-ically,subjects in groups 3and 4were given (1)closing prices of the previous week shown on the tabular and graphical forms;(2)individual calibration scores computed from the previous week’s probability forecasts,along with detailed information on the proportion of correct forecasts and relative frequency of u for each probability category employed by the participant;and (3)percentage of their prediction intervals that actually contained the realized stock price (i.e.an index of interval calibration).
At the beginning of the first ssion,concepts of ‘‘subjective probability’’,‘‘prediction intervals’’and ‘‘probability forecasting’’were discusd,and their role in financial forecasting was emphasized.Exam-ples were given and the participants were informed that certain scores of forecast
ing performance would be computed from their individual forecasts,and that they could earn their best potential score by stating their true opinions without hedging or bluffing.Also,the students in no-contextual-information groups were specifically instructed to ba their forecasts only on the price information prented,without trying to uncover the names of individual stocks.The subjects were warned of the particular significance of basing their forecasts solely on the prented time ries information.
In each ssion,the participants were instructed to provide a prediction interval for the closing price of each of the stocks being considered.In stating the prediction interval,each subject gave the highest and the lowest predicted closing price for each stock such that he/she was 90%confident that this range would include the actual closing price.Participants were also asked to make directional probability forecasts for the closing prices of stocks.In particular,each subject was requested to indicate whether (s)he believed that the stock price for the current Friday would (a)increa,or (b)decrea or stay the same in comparison with the previous Friday’s closing stock price.Following this direction indication,each subject was asked to convey his/her degree of belief with a subjective probability for the predicted direction of price change (i.e.,probability that the weekly price change would actually fall in the direction indicated by the subject).Since a direction of
price change was given first,the following probability would have to lie between 0.5and 1.0.A sample form for reporting predictions is prented in Fig.1.
Fig.1.Sample form for reporting judgmental forecasts.
P .Goodwin et al./Decision Support Systems 37(2004)175–186
180
