ANNEXURES
Annexure 1 - Transportation Modelling - An
Esoteric Exercise by W.R. Blunden....... 338
Annexure 2
- Cost-Benefit Analysis in the
Kyeemagh-Chullora Road Inquiry
by M.E. Beesley......................... 349
Kyeemagh-Chullora Road Inquiry
by M.E. Beesley......................... 349
Annexure 3
- Discussion of the Economic
Analysis of Highway Improvements
by W.R. Blunden......................... 372
Analysis of Highway Improvements
by W.R. Blunden......................... 372
Annexure
2 - COST BENEFIT ANALYSIS AND THE
KYEEMAGH-CHULLORA ROAD INQUIRY
M.E. BEESLEY.
KYEEMAGH-CHULLORA ROAD INQUIRY
M.E. BEESLEY.
A. THE USES OF COST BENEFIT ANALYSIS
1. By "cost benefit" analysis is meant the measuring
of impacts on interests prospectively affected by a
(road) project in such a way that these can be directly
compared and aggregated to form a statement about the
project's worth. The relation of the sum of benefits to
the sum of costs can be expressed in many ways, by
deduction of one from the other, computing ratios etc.
Costs and benefits occur over time, so problems of
forecasting and dealing with their incidence over time
arise. Expressing the results and recognising the passage
of time, while raising intricate problems of discounting
and dealing with risk, do not greatly distinguish the
cost-benefit from ordinary financial appraisal, which has
similarly to deal with these problems. Cost-benefit
analysis raises most controversy among those who are
invited to use it, or who are affected by it, because of
its special characteristics.
of impacts on interests prospectively affected by a
(road) project in such a way that these can be directly
compared and aggregated to form a statement about the
project's worth. The relation of the sum of benefits to
the sum of costs can be expressed in many ways, by
deduction of one from the other, computing ratios etc.
Costs and benefits occur over time, so problems of
forecasting and dealing with their incidence over time
arise. Expressing the results and recognising the passage
of time, while raising intricate problems of discounting
and dealing with risk, do not greatly distinguish the
cost-benefit from ordinary financial appraisal, which has
similarly to deal with these problems. Cost-benefit
analysis raises most controversy among those who are
invited to use it, or who are affected by it, because of
its special characteristics.
2. These are:
(i) that it aims to be comprehensive - it
looks at all important impacts of a given project. This
means that, in principle, it goes beyond the immediate
users of the project to consider other interests such
as the community at large providing the resources for
the investment, or those subject to environmental impacts
such as sufferers from noise. Hence its more apt name
"Social Cost Benefit Analysis" (1). But the
possibilities of doing this are much more constrained by
its second characteristic:-
looks at all important impacts of a given project. This
means that, in principle, it goes beyond the immediate
users of the project to consider other interests such
as the community at large providing the resources for
the investment, or those subject to environmental impacts
such as sufferers from noise. Hence its more apt name
"Social Cost Benefit Analysis" (1). But the
possibilities of doing this are much more constrained by
its second characteristic:-
1. For
convenience, the initials ‘CBA' are used here.
-349-
(ii) it must necessarily be concerned both with
impacts which are readily measurable by easily
referred-to market prices and with those which are not,
such as the value of leisure time saved, yet every
item must be measured in terms of a single yardstick,
which, faute de mieux, must be dollars. This need to
measure in a common unit leads to the third
important characteristic, which is that:-
impacts which are readily measurable by easily
referred-to market prices and with those which are not,
such as the value of leisure time saved, yet every
item must be measured in terms of a single yardstick,
which, faute de mieux, must be dollars. This need to
measure in a common unit leads to the third
important characteristic, which is that:-
(iii) the evidence sought to yield the relevant values
is essentially inferences from consumers' behaviour
The question posed is what would he or she pay for the
benefits arising from a project, or not to have to
bear the costs involved?
is essentially inferences from consumers' behaviour
The question posed is what would he or she pay for the
benefits arising from a project, or not to have to
bear the costs involved?
3. This is answered by referring to how consumers
behave, as when paying more to save time on a journey.
That the evidence in practice must be derived in this
way is the source of prolonged debate among
economists, because of the (more or less) questionable
assumption that what consumers are prepared to pay is
itself a good measure of consumer welfare. However,
most practical users of cost benefit analysis have
plenty of more important sources of worry about the
project calculation that this for example, how the
project(s) are defined, what is done about forecasting
the quantities of traffic and persons involved and the
question of whether the consumers' willingness to pay
has been correctly measured etc. The significant point
about the evidence, first, that it does refer strictly
to consumers in their several roles with respect to
the project, as road users, as householders, as
sufferers of pollution. Consumer based values turn up
throughout the analysis, valuing resources and
measuring all kinds of impacts. Second, it is by no
means the only possible basis for valuation.
behave, as when paying more to save time on a journey.
That the evidence in practice must be derived in this
way is the source of prolonged debate among
economists, because of the (more or less) questionable
assumption that what consumers are prepared to pay is
itself a good measure of consumer welfare. However,
most practical users of cost benefit analysis have
plenty of more important sources of worry about the
project calculation that this for example, how the
project(s) are defined, what is done about forecasting
the quantities of traffic and persons involved and the
question of whether the consumers' willingness to pay
has been correctly measured etc. The significant point
about the evidence, first, that it does refer strictly
to consumers in their several roles with respect to
the project, as road users, as householders, as
sufferers of pollution. Consumer based values turn up
throughout the analysis, valuing resources and
measuring all kinds of impacts. Second, it is by no
means the only possible basis for valuation.
4. Consumers are not necessarily the ultimate arbiters
of values in practice. Governments regularly over-ride
of values in practice. Governments regularly over-ride
-350-
them, as in deciding that education is a service
which should be consumed even if the receiver is not
able - or indeed willing - to pay for it. In fact,
in choosing among projects, governments themselves are
implying that values are inherent in them. For example
in refusing to extend the budget devoted to, say,
defence, government is impliedly setting a limit on the
value to be derived from extra resources devoted to
defence, in analogous manner to the consumer refusing
to spend more than $X on time saving. Whatever is
the implied bases of valuation in what the Government
does in these cases, it certainly cannot be described.
as based simply on consumer preferences.
which should be consumed even if the receiver is not
able - or indeed willing - to pay for it. In fact,
in choosing among projects, governments themselves are
implying that values are inherent in them. For example
in refusing to extend the budget devoted to, say,
defence, government is impliedly setting a limit on the
value to be derived from extra resources devoted to
defence, in analogous manner to the consumer refusing
to spend more than $X on time saving. Whatever is
the implied bases of valuation in what the Government
does in these cases, it certainly cannot be described.
as based simply on consumer preferences.
5. CBA therefore represents an alternative valuation
system, one that has the virtue of attempting to
treat a variety of projects in a consistent manner,
and one that explores consequences widely, but with a
means to do so (consumer based valuations) which do not
necessarily or wholly accord with typical government
objectives (1) . From the administrative point of view,
cost benefit analysis is an alternative means to make
decisions, and thus is likely to be regarded as partly.
friendly (in upsetting methods which are deemed to need
reform) and partly as potentially unfriendly (in
substituting consumers as sovereign).
system, one that has the virtue of attempting to
treat a variety of projects in a consistent manner,
and one that explores consequences widely, but with a
means to do so (consumer based valuations) which do not
necessarily or wholly accord with typical government
objectives (1) . From the administrative point of view,
cost benefit analysis is an alternative means to make
decisions, and thus is likely to be regarded as partly.
friendly (in upsetting methods which are deemed to need
reform) and partly as potentially unfriendly (in
substituting consumers as sovereign).
5. The growth of CBA in governmental decision making
is largely to be explained by dissatisfaction with
existing practices. These have often been implicitly,
and on occasion embarrassingly explicitly,
inconsistent in choice of projects. They have lacked
objective means for imputing values, have displayed
limited ability to cope with comparisons involving
other uses of government funds, etc. CBA, in setting
out to overcome shortcomings like these, is a mixed
blessing, however. It certainly is typically more
is largely to be explained by dissatisfaction with
existing practices. These have often been implicitly,
and on occasion embarrassingly explicitly,
inconsistent in choice of projects. They have lacked
objective means for imputing values, have displayed
limited ability to cope with comparisons involving
other uses of government funds, etc. CBA, in setting
out to overcome shortcomings like these, is a mixed
blessing, however. It certainly is typically more
1. The
values entering cost benefit analysis are
frequently modified to "correct" for distortions
of market prices, and to reflect particular
scarcities not shown up in them, as seen by
governments, for various reasons e.g. foreign
exchange in undeveloped countries. The intention is
to simulate what would have happened were the markets
not distorted; there is no challenge to the basic
reference point in what consumers (and producers) would
do, in the absence of the factors causing distortion.
frequently modified to "correct" for distortions
of market prices, and to reflect particular
scarcities not shown up in them, as seen by
governments, for various reasons e.g. foreign
exchange in undeveloped countries. The intention is
to simulate what would have happened were the markets
not distorted; there is no challenge to the basic
reference point in what consumers (and producers) would
do, in the absence of the factors causing distortion.
-351-
comprehensive, but this throws into sharp relief
the areas of impact which are measured comprehensively,
and those which, though not less important
intrinsically, cannot be measured. Numbers may acquire
undue weight, and are usually hard to set alongside
others on equal terms. It certainly typically imports
much more concern for objective evidence about value;
but its own basis, consumer behaviour, is not
unchallengeable. (In the heat of controversy about the
nature of evidence used in CBA, it is however,
frequently forgotten that justification of values
implicit in alternative ways of making choices is far
more flimsy, if present at all). CBA forces more
precision of thinking about the definition of projects,
the comparisons to be made, the consistent treatment of
long-1ived assets, what is meant by costs, and who bear
them, etc. By the same token, it cannot give all the
answers, and its adoption in practice means adopting
its purveyors-in-chief, professional economists, not
always the most welcome of bed-fellows. It also tends
to discover interests quite latent until investigations
are made, and raises awkward questions for example,
about the adequacy of compensation to be paid for
bearing the costs, without, of course, providing the
means of dealing with such issues. In short, CBA is a
technique to be regarded with circumspection, but its
development to help fill governmental needs in decision
making was inevitable.
7. It is not surprising, therefore, that CBA has had a
chequered career in public inquiries. In roads, it is
a well-established part of regular investment appraisal,
as for example in all major road projects in the UK.
However, its exact scope and its role in the road
decision making process as a whole is by no means settled
there. This, as in most countries, is a series of stops
stretching over many years, from initial identification
of the "need" for a new road, through many stages of
project definition, design, approval and adoption as part
chequered career in public inquiries. In roads, it is
a well-established part of regular investment appraisal,
as for example in all major road projects in the UK.
However, its exact scope and its role in the road
decision making process as a whole is by no means settled
there. This, as in most countries, is a series of stops
stretching over many years, from initial identification
of the "need" for a new road, through many stages of
project definition, design, approval and adoption as part
-352-
of the road budget, to construction. A public inquiry
is frequently involved. Currently, the official
standing Advisory committee on Trunk Road Assessment
in the UK, successor to the Leitch Committee, advocates
using a type of assessment in which CBA is part of a
comprehensive statement of impacts on all major
interests as affected by the project in question,
whether amenable to measurement or not. Although this
"framework" for assessment is at present focused on the
problem of choosing between routes to be followed by a
major road investment, and thus on the need of the
public inquiry phase of the process, it is hoped that
the framework will have a growing influence on the
rest of it. The framework itself can be regarded as
a form of CBA; indeed its progenitor was N. Lichfield’s
"Planning Balance Sheet", recognised as a form of CBA.
But it is the elements valued in money items which are
the distinctive contribution of the CBA technique, and
they form part of the framework.
is frequently involved. Currently, the official
standing Advisory committee on Trunk Road Assessment
in the UK, successor to the Leitch Committee, advocates
using a type of assessment in which CBA is part of a
comprehensive statement of impacts on all major
interests as affected by the project in question,
whether amenable to measurement or not. Although this
"framework" for assessment is at present focused on the
problem of choosing between routes to be followed by a
major road investment, and thus on the need of the
public inquiry phase of the process, it is hoped that
the framework will have a growing influence on the
rest of it. The framework itself can be regarded as
a form of CBA; indeed its progenitor was N. Lichfield’s
"Planning Balance Sheet", recognised as a form of CBA.
But it is the elements valued in money items which are
the distinctive contribution of the CBA technique, and
they form part of the framework.
8. The elements measured in money terms which are
incorporated into the framework are those which have
for many years been the mainstay of economic analysis
of roads. On the cost side, they are the discounted
present values of resources for building and maintaining
the option in question (at the Treasury-set discount
rate of 10%). On the benefit side, they are the
present value of changes in road users’ times, in
business and leisure, changes in operating costs of
vehicles, and in accident savings. Benefits are thus
changes in prospective users' costs. No attempt is
made to incorporate in the CBA calculations other
"social" costs; these are the subject of other measures
and comments in the "framework". So, in the case of
road evaluation, (measured) CBA is a valued, but
limited, part of wider assessment, which in turn is
only a part of a sequential decision process. CBA has
very little influence, for example, on the initial
selection of projects; and very little on the total
amount of money devoted to roads, a decision involving
incorporated into the framework are those which have
for many years been the mainstay of economic analysis
of roads. On the cost side, they are the discounted
present values of resources for building and maintaining
the option in question (at the Treasury-set discount
rate of 10%). On the benefit side, they are the
present value of changes in road users’ times, in
business and leisure, changes in operating costs of
vehicles, and in accident savings. Benefits are thus
changes in prospective users' costs. No attempt is
made to incorporate in the CBA calculations other
"social" costs; these are the subject of other measures
and comments in the "framework". So, in the case of
road evaluation, (measured) CBA is a valued, but
limited, part of wider assessment, which in turn is
only a part of a sequential decision process. CBA has
very little influence, for example, on the initial
selection of projects; and very little on the total
amount of money devoted to roads, a decision involving
-353-
the Treasury, and thus alternative, non-road uses of
funds.
9. CBA is thus accepted and institutionalised in road
decisions in the UK. Where made the centre-piece of
the decision making process itself, it has, by contrast
enjoyed much less acceptance. The outstanding example
of this was in the Roskill Inquiry into the Third
London Airport, of 1971, which was by far the largest
(and many would say the best) CBA ever attempted in the
UK. This was not only a comprehensive attempt at
measurement of impacts, in which specific values were
summed up and put on the four option sites, but it also
served as the means by which debate about the choice was
conducted. The CBA elements were subjected to detailed.
scrutiny by QC's leading for each "defendant" option
(no one site wanted an airport, of course), their juniors
and by a long sequence of professional economists, many
of great renown. During the course of the Inquiry, and
partly in response to the debate, the CBA elements were
rejigged and refined by the Inquiry's economic research
team, whose cost represented a large part of the
Inquiry's outlay.
decisions in the UK. Where made the centre-piece of
the decision making process itself, it has, by contrast
enjoyed much less acceptance. The outstanding example
of this was in the Roskill Inquiry into the Third
London Airport, of 1971, which was by far the largest
(and many would say the best) CBA ever attempted in the
UK. This was not only a comprehensive attempt at
measurement of impacts, in which specific values were
summed up and put on the four option sites, but it also
served as the means by which debate about the choice was
conducted. The CBA elements were subjected to detailed.
scrutiny by QC's leading for each "defendant" option
(no one site wanted an airport, of course), their juniors
and by a long sequence of professional economists, many
of great renown. During the course of the Inquiry, and
partly in response to the debate, the CBA elements were
rejigged and refined by the Inquiry's economic research
team, whose cost represented a large part of the
Inquiry's outlay.
10. The outcome was a decision by 5 Commissioners in
favour of one site, and one dissenting report in favour
of another. The majority decision was not accepted by
the responsible Minister, who favoured the dissenter's.
Subsequently this solution was itself abandoned; recently
a new decision has been made reviving an option not
among Roskill's four. This time, the government of the
day has been careful not to have a public discussion based.
on a CBA, though there has been much developing of the
Roskill analysis as input to the decision. The moral of
this long-running saga is that CBA is an excellent way
to generate and encourage discussion; and uncertainty
among decision-makers. To many this is precisely its
great virtue, because it forces consideration of options,
and their relative merits, in a consistent even if, in
the end, limited, account of effects on public welfare.
favour of one site, and one dissenting report in favour
of another. The majority decision was not accepted by
the responsible Minister, who favoured the dissenter's.
Subsequently this solution was itself abandoned; recently
a new decision has been made reviving an option not
among Roskill's four. This time, the government of the
day has been careful not to have a public discussion based.
on a CBA, though there has been much developing of the
Roskill analysis as input to the decision. The moral of
this long-running saga is that CBA is an excellent way
to generate and encourage discussion; and uncertainty
among decision-makers. To many this is precisely its
great virtue, because it forces consideration of options,
and their relative merits, in a consistent even if, in
the end, limited, account of effects on public welfare.
-354-
11. In the events leading up to the setting up of the
Leitch Committee in 1978, and thus the eventual
establishing of the "framework", measured CBA items were
indeed one of the means to express discontent about the
prospect of a road in a locality. Public road inquiries
became for a time unworkable because of protests, among
which were prominent and surprising establishment figures.
But the assumptions on which traffic forecasts were based
provided an even more productive source for ammunition.
Perhaps the most telling discontent was about the manifest
one-sidedness of the method of inquiry, which was focused
on one proposal. This could not, in principle, lead to
discussion of alternative options on equal terms. Also
the inspectorate for Inquiries was not drawn independently
from the Ministry responsible for the final decision.
Thus the present position is that it is believed that
sufficient attention has been given to airing and responding
to the more important complaints, including attending to
the point about independence. Measured. CBA items, them-
selves limited in the way described earlier, have been
supplemented by the "framework". Whether these changes
will diffuse the opposition when the difficult (usually
urban) cases arise in future is a moot point.
12. The Kyeemagh-Chullora Inquiry, as the first of its
kind, and having a constitution and terms of reference
permitting a wide consideration of options, has fortunately
not inherited problems of the kind which have beset UK
inquiries. But the UK history does suggest that CBA has
significance for the Kyeemagh-Chullora Inquiry in these
ways: -
kind, and having a constitution and terms of reference
permitting a wide consideration of options, has fortunately
not inherited problems of the kind which have beset UK
inquiries. But the UK history does suggest that CBA has
significance for the Kyeemagh-Chullora Inquiry in these
ways: -
(a) CBA is certainly not a means to
make
the relevant decisions, but it, has a
role to raise questions about options
systematically.
the relevant decisions, but it, has a
role to raise questions about options
systematically.
(b) Such logical comparisons will
involve
measurement, because this is the only
means to get them aligned.
measurement, because this is the only
means to get them aligned.
(c) Measured items will comprise
only a
subset of relevant factors.
subset of relevant factors.
-355-
(d) There is a well-established subset
used regularly in CBA work on
roads, on which it would be useful
to focus, because to do so enables
CBA experience to be called upon.
(e) Those proposing options, and
advocating a specific decision,
should be aware of the strengths
and limitations of CBA in a given
decision-making process, and where it
is likely to raise controversy as
well as help settle it.
advocating a specific decision,
should be aware of the strengths
and limitations of CBA in a given
decision-making process, and where it
is likely to raise controversy as
well as help settle it.
B. THE SUBMISSIONS TO THE INQUIRY
13. The last two conclusions bear on the submission to
the Inquiry by the D.M.R., P.E.C. and U.T.S. Because
DMR has ultimate responsibility for road building - in
terms of the above argument it contains most of the
decision making process it is useful to concentrate
mostly on DMR’s submissions. Questions of the technical
merit of the work on which submissions were based are first
raised; these have in mind the state of the art as it is
generally practiced, not some ideal model of CBA work.
Then the treatment of the inquiry process by the
department is considered. The question in mind here is
the constructive one - what could be done to improve the
way in which presentations were managed were a similar need
to arise?
the Inquiry by the D.M.R., P.E.C. and U.T.S. Because
DMR has ultimate responsibility for road building - in
terms of the above argument it contains most of the
decision making process it is useful to concentrate
mostly on DMR’s submissions. Questions of the technical
merit of the work on which submissions were based are first
raised; these have in mind the state of the art as it is
generally practiced, not some ideal model of CBA work.
Then the treatment of the inquiry process by the
department is considered. The question in mind here is
the constructive one - what could be done to improve the
way in which presentations were managed were a similar need
to arise?
14. The quality of a CBA analysis depends chiefly on how:
(i) it deals with the question of defining the options
and interprets results;
(ii) the quality of the underlying
information about
traffic flows, and
traffic flows, and
(iii) how it treats the items of
cost and benefit
entering into the account of the values of the
options. (The first two are common to any logical
process of comparison).
entering into the account of the values of the
options. (The first two are common to any logical
process of comparison).
These points may be taken in turn.
15. (i) Defining the Options
Defining the options analysed is crucial in at least three
ways. First the set analysed in detail is necessarily
limited. (In the case at hand there were five, in the
original set on which public comment was invited. Two
of them were Bexley Road sub-options). There is always
an implied assumption in presenting a limited set that,
out of a possible larger set of options, in some sense
the "best" have been selected for detailed analysis.
A review of the larger set could not, by definition,
have the same degree of attention; nevertheless normally
some kind of coarse but formal sieving process would be
gone through to reduce the number. No mention of such a
process occurs in the submissions. Legitimate doubts are
raised about implicit rejections of options which may have
turned out to be "better". This came home to roost in the
present inquiry when the question of the eastern
("terminal") end of the study area arose - what of an
option incorporating a route north of the airport? DMR
was unable to speculate about its costs or benefits.
ways. First the set analysed in detail is necessarily
limited. (In the case at hand there were five, in the
original set on which public comment was invited. Two
of them were Bexley Road sub-options). There is always
an implied assumption in presenting a limited set that,
out of a possible larger set of options, in some sense
the "best" have been selected for detailed analysis.
A review of the larger set could not, by definition,
have the same degree of attention; nevertheless normally
some kind of coarse but formal sieving process would be
gone through to reduce the number. No mention of such a
process occurs in the submissions. Legitimate doubts are
raised about implicit rejections of options which may have
turned out to be "better". This came home to roost in the
present inquiry when the question of the eastern
("terminal") end of the study area arose - what of an
option incorporating a route north of the airport? DMR
was unable to speculate about its costs or benefits.
16. The definition of options greatly affects the
results. First, one has to assume a base-line against
which changes in benefits and costs associated with an
option are to be measured. The obvious base-line option is
"do nothing". But no exercise can in practice assume that
the base is, literally, "do nothing’. In the present
instance, one can not assume that road investment or
betterment in Sydney will remain still, making the options
studied the only relevant shift in investment which will
occur. The parties submitting evidence were less than clear
about how this problem was handled even if it was handled
explicitly, i.e., with a conscious attempt to be consistent.
The assumptions about what was or was not assumed to be
"committed", or in the programme, or actually budgeted for,
etc., seemed to be different at different times. Second,
options can be subdivided, or "staged". This produces, in
effect, another base-line - one invests in the best “bit”
or "stage" first, and then the next. The relevant
computation becomes the change from the new base. Failure
results. First, one has to assume a base-line against
which changes in benefits and costs associated with an
option are to be measured. The obvious base-line option is
"do nothing". But no exercise can in practice assume that
the base is, literally, "do nothing’. In the present
instance, one can not assume that road investment or
betterment in Sydney will remain still, making the options
studied the only relevant shift in investment which will
occur. The parties submitting evidence were less than clear
about how this problem was handled even if it was handled
explicitly, i.e., with a conscious attempt to be consistent.
The assumptions about what was or was not assumed to be
"committed", or in the programme, or actually budgeted for,
etc., seemed to be different at different times. Second,
options can be subdivided, or "staged". This produces, in
effect, another base-line - one invests in the best “bit”
or "stage" first, and then the next. The relevant
computation becomes the change from the new base. Failure
-357-
to distinguish separable options in this sense will
mislead; the results for the less favourable option
may be unsatisfactory where it is distinguished.
This became important in discussion of the "extended
base" case. Once the feasibility of separating the
eastern parts of this had been established, the relevant
calculations become what are termed the "marginal"
calculations, having considerably less impressive
(positive) results that the results for the eastern
options plus the works further west, viewed as joint
projects. This was true of both the major options
considered at that point, namely the Cooks River and
S.W. Freeway. The eastern options results were
presented as in some sense of lesser importance (see
conclusions pg. 74, D.L. report).
17. More fundamentally for the present Inquiry,
analysts have to be clear whether options are expected
to be directly complementary or competitive, or only
indirectly related through the common need to finance
and construct road investment. Complementary means that
the worth of one option depends on another; both gain if
undertaken. Competitive means that if one is undertaken,
the other’s results are much worse, and vice versa. The
submissions seemed to reflect the underlying view that
any particular option contained complementary sections;
as we have just seen, this was not supported on more
detailed analysis. But the whole Inquiry was posited
on the assumption that each original option was
competitive - these were supposed to be routes which
were alternative ways of performing basically a similar
service to the public - with greater or lesser
anticipated efficiency and effects, to be sure. The
test of this hypothesis was never undertaken. It would
have required running the original options in
combinations, and not separately against a common base,
as was done. Were this test to have been made, it is
quite possible, indeed I think likely, that the results
would have shown that the options had strong independence.
analysts have to be clear whether options are expected
to be directly complementary or competitive, or only
indirectly related through the common need to finance
and construct road investment. Complementary means that
the worth of one option depends on another; both gain if
undertaken. Competitive means that if one is undertaken,
the other’s results are much worse, and vice versa. The
submissions seemed to reflect the underlying view that
any particular option contained complementary sections;
as we have just seen, this was not supported on more
detailed analysis. But the whole Inquiry was posited
on the assumption that each original option was
competitive - these were supposed to be routes which
were alternative ways of performing basically a similar
service to the public - with greater or lesser
anticipated efficiency and effects, to be sure. The
test of this hypothesis was never undertaken. It would
have required running the original options in
combinations, and not separately against a common base,
as was done. Were this test to have been made, it is
quite possible, indeed I think likely, that the results
would have shown that the options had strong independence.
-358-
Such an interpretation can certainly be placed on
the apparent mystery of the S.W. Freeway and
Cooks River option results. (See Cameron et. al's
letter of 2nd April, 1980). When the 1976 trip tables
were used instead of the 1991 trip tables, the S.W.
option was more sharply differentiated in benefit terms
from the Cooks River option. This was interpreted
(see pg. 2 of letter) as due to accident costs; yet
inspection of the comparison discloses a greater
discrepancy in time costs (1). The distribution of trips
as between 1976 and 1991 is expected to alter
geographically. The plain fact may be that this change
affects one option much more than the other; i.e., the
options serve substantially different movements.
18. The relevant comparison for road projects which are
not inter-related in their effects is with
opportunities open to the competent authority to use
road funds elsewhere. Are they substantially better or
worse? This question is also pertinent when the inter-
relations have been demonstrated, and the ranking of the
options determined as a result. One is then interested
in where the cut-off in terms of acceptance will lie.
Unfortunately, no evidence bearing on this issue was
offered, although there were from time to time references
to the expected size of cost benefit ratios in road
analysis in general. In fact, there is some evidence
which could have been drawn upon. This, unfortunately,
does not concern the situation in N.S.W. or Sydney, as
would have been far more relevant; whether the
comparative calculations for other 1ocaI roads exist
not inter-related in their effects is with
opportunities open to the competent authority to use
road funds elsewhere. Are they substantially better or
worse? This question is also pertinent when the inter-
relations have been demonstrated, and the ranking of the
options determined as a result. One is then interested
in where the cut-off in terms of acceptance will lie.
Unfortunately, no evidence bearing on this issue was
offered, although there were from time to time references
to the expected size of cost benefit ratios in road
analysis in general. In fact, there is some evidence
which could have been drawn upon. This, unfortunately,
does not concern the situation in N.S.W. or Sydney, as
would have been far more relevant; whether the
comparative calculations for other 1ocaI roads exist
___________________________________________________________
1. The table is:-
$m Benefits
|
||||
Operating
|
Time
|
Accidents
|
Total
|
|
Cooks River
|
.667
|
.716
|
-1.185
|
0.198
|
S.W. Freeway
|
.908
|
2.927
|
.156
|
3.991
|
From this:
|
-----
|
-----
|
-----
|
-----
|
Net Swing to
S.W.
|
+.241
|
+2.211
|
+1.341
|
+3.793
|
____________________________________________________________
-359-
was not disclosed. However, Hastings and Taplin's
work on Transport Investment in South Australia, a
Ten-Year View (1) indicates that the marginal cost-
benefit ratio for South Australia's roads at 1978
were as follows:-
work on Transport Investment in South Australia, a
Ten-Year View (1) indicates that the marginal cost-
benefit ratio for South Australia's roads at 1978
were as follows:-
National Highways 1.6
Rural Arterial 2.5
Rural Local 2.3
Urban Arterial 2.0
Rural Arterial 2.5
Rural Local 2.3
Urban Arterial 2.0
"Marginal" here means the lowest accepted ratio in the
accepted set; so all the rest of the projects had ratios
superior to these. These broadly agree with Bureau of
Transport Economics own estimate of 1.6 nationwide.
(10% discount rates were used, as in the main evidence
to the K/C Inquiry). The nearest equivalent category
to the Inquiry's options is Urban Arterial; there is no
reason to suppose that N.S.W. has a more generous
attitude towards roads, or less opportunities to use
the resources productively, than South Australia. If
this is a fair interpretation, the eastern part of the
S.W. Freeway and Cooks River options would each be
highly doubtful investments, in CBA terms, when measured
correctly, that is as additions to investments further
west. On the other hand, it is clear that these more
western road investments, not the immediate concern on
the Inquiry, are highly acceptable investments on the
CBA test. The most striking thing is the contrast
between results in CBA terms. Even though the evidence
is so scanty, it seems highly likely that the true margin of
acceptance or rejection lies in between the two sets of
investment, and much closer to the eastern than the
western. Failure to throw direct light on this
handicapped discussion greatly.
accepted set; so all the rest of the projects had ratios
superior to these. These broadly agree with Bureau of
Transport Economics own estimate of 1.6 nationwide.
(10% discount rates were used, as in the main evidence
to the K/C Inquiry). The nearest equivalent category
to the Inquiry's options is Urban Arterial; there is no
reason to suppose that N.S.W. has a more generous
attitude towards roads, or less opportunities to use
the resources productively, than South Australia. If
this is a fair interpretation, the eastern part of the
S.W. Freeway and Cooks River options would each be
highly doubtful investments, in CBA terms, when measured
correctly, that is as additions to investments further
west. On the other hand, it is clear that these more
western road investments, not the immediate concern on
the Inquiry, are highly acceptable investments on the
CBA test. The most striking thing is the contrast
between results in CBA terms. Even though the evidence
is so scanty, it seems highly likely that the true margin of
acceptance or rejection lies in between the two sets of
investment, and much closer to the eastern than the
western. Failure to throw direct light on this
handicapped discussion greatly.
19. A CBA may have better or worse foundations in the
underlying data, but it is important that what is done
is interpreted correctly. The PEC/DMR joint report
underlying data, but it is important that what is done
is interpreted correctly. The PEC/DMR joint report
1.
Director-General of Transport, South Australia,
December, 1978.
December, 1978.
-360-
presented the most comprehensive account of the five
options. The measured CBA work was set in a wider
context rather reminiscent of the Leitch framework.
Besides the traditional CBA road items, such impacts
as number of houses resumed, or parks and houses
affected etc., were noted. This was not a full impact
statement; notable omissions where any attempt to
measure the number of persons in households in proximity
to the different routes affected, or the intensity of
shopping uses along them. This is essential to
forming a balanced view of the populations principally
at risk - or in other words, those chiefly standing to
gain or lose in the choice between options. However,
what was done was useful information; what of its
interpretation?
20. In contrast to the impression given by the
commentary in the report, it would have been a natural
conclusion from considering the analysis only and one
stresses this last qualification - that one of the
Harrow Road options was revealed superior. Though
measurement in CBA terms was not attempted beyond the
traditional road user etc., items, the quantities on
the other dimensions can be compared. It turns out that
with a relatively minor exception - the number of
houses partly affected, and that only compared with the
S.W. option, the Bestic Road dominates the others; (1)
i.e., on every dimension indicates lower negative
impacts. This means that, however highly these negative
impacts were to be valued, it is highly probable that
it would still turn out that a completely calculated
CBA would have shown this option to be "superior",
(it and Harrow Road also had superior calculated CBA
ratios, unsurprisingly in view of the low cost of the
options). This was no doubt an unwelcome result, for
different reasons, to each of the departments, and that
it ran counter to their intuition was also doubtless
commentary in the report, it would have been a natural
conclusion from considering the analysis only and one
stresses this last qualification - that one of the
Harrow Road options was revealed superior. Though
measurement in CBA terms was not attempted beyond the
traditional road user etc., items, the quantities on
the other dimensions can be compared. It turns out that
with a relatively minor exception - the number of
houses partly affected, and that only compared with the
S.W. option, the Bestic Road dominates the others; (1)
i.e., on every dimension indicates lower negative
impacts. This means that, however highly these negative
impacts were to be valued, it is highly probable that
it would still turn out that a completely calculated
CBA would have shown this option to be "superior",
(it and Harrow Road also had superior calculated CBA
ratios, unsurprisingly in view of the low cost of the
options). This was no doubt an unwelcome result, for
different reasons, to each of the departments, and that
it ran counter to their intuition was also doubtless
1. 80
houses, compared. with 51 on the S.W. Freeway.
-361-
partly due at least to the acknowledged omissions in
the analysis. Nevertheless, the conclusions should.
have been squarely drawn and their implications faced.
(At the time the report was submitted, no CBA ratios
were available for the Bexley Road options; these
appear in a later submission. But it is not likely
that the way in which they would turn out was not very
foreseeable at that stage - the Bexley Road options'
costs were known to be only about one-third of those of
the other options).
(ii) Quality of Information
21. Passing to the question of the data used in the
several submissions to underpin and value items in the
measured CBA, there emerges a-strong contrast between
the output of the models used to produce the relevant
traffic volumes, and the valuation of benefit and cost
changes. The former can only be described as revealed
as quite inadequate; the latter have their
difficulties, but one cannot regard these as so serious
in comparison. The latter are, after all, conventional
and limited to road users and suppliers benefits and
costs. The former are essential whether CBA is used or
not; they are at the heart of attempts to plan roads
according to likely demands.
several submissions to underpin and value items in the
measured CBA, there emerges a-strong contrast between
the output of the models used to produce the relevant
traffic volumes, and the valuation of benefit and cost
changes. The former can only be described as revealed
as quite inadequate; the latter have their
difficulties, but one cannot regard these as so serious
in comparison. The latter are, after all, conventional
and limited to road users and suppliers benefits and
costs. The former are essential whether CBA is used or
not; they are at the heart of attempts to plan roads
according to likely demands.
22. It is not feasible to enter into a detailed account
here of the difficulties encountered in trying to
derive a fairly straight forward account of predicted
traffic flows for the options. Among other things, the
options themselves were changing; omissions and mistakes
in the trip tables had to be corrected and, all through,
considerable doubt existed about the plausibility of
the basic population and employment distributions, and
their likely change over time. That responsibility for
three important phases of estimation - population and
employment prediction, option definition, and traffic
modelling is divided (PEC, DMR and UTS respectively) no
here of the difficulties encountered in trying to
derive a fairly straight forward account of predicted
traffic flows for the options. Among other things, the
options themselves were changing; omissions and mistakes
in the trip tables had to be corrected and, all through,
considerable doubt existed about the plausibility of
the basic population and employment distributions, and
their likely change over time. That responsibility for
three important phases of estimation - population and
employment prediction, option definition, and traffic
modelling is divided (PEC, DMR and UTS respectively) no
-362-
doubt played its part in generating uncertainty.
(It did not help, either, that valuation of CBA items
was also in part delegated to consultants, thus
inserting another party in the process). At no time
was a reasoned account given of why Sydney is expected
to develop in population and employment in the way
expected. But the problems of predicting population
and employment are shared in greater or lesser degree
in urban planning the world over. One can discern
that the basic problem in the present case lay
essentially elsewhere. This was that questions were
being asked of a total modelling process which was not
geared to answer them, and would not have been even if
there had been no complications of the sort just
mentioned.
23. The K/C Inquiry is concerned with impacts on
specific sets of streets. It has to be interested in
details of traffic composition on those streets, for
heavy lorries are among the most important reasons for
public disquiet. The modelling process, on the other
hand, embodies a traditional urban transport study
approach. The question being asked of it is what is
the optimum strategy for road investment on the Sydney
scale? The critical methodological point, perhaps not
appreciated fully in the hearings, is that in order to
answer the question about investment in any given
possible road network, the lever of detail of description
of flows has to be greater than that for which the
answers are sought. The network used as descriptors in
the Inquiry were developed in order to illuminate
investment at a higher level e.g., priorities in
major corridors. But somehow the need to have any
information about the case at hand overcame
methodological good sense. One then had the absurd
spectacle of debates about expected flows on particular
links of the network, how they were individually expected
to grow, etc. At this level, considerable uncertainty is
quite acceptable in principle so long as aggregation over
specific sets of streets. It has to be interested in
details of traffic composition on those streets, for
heavy lorries are among the most important reasons for
public disquiet. The modelling process, on the other
hand, embodies a traditional urban transport study
approach. The question being asked of it is what is
the optimum strategy for road investment on the Sydney
scale? The critical methodological point, perhaps not
appreciated fully in the hearings, is that in order to
answer the question about investment in any given
possible road network, the lever of detail of description
of flows has to be greater than that for which the
answers are sought. The network used as descriptors in
the Inquiry were developed in order to illuminate
investment at a higher level e.g., priorities in
major corridors. But somehow the need to have any
information about the case at hand overcame
methodological good sense. One then had the absurd
spectacle of debates about expected flows on particular
links of the network, how they were individually expected
to grow, etc. At this level, considerable uncertainty is
quite acceptable in principle so long as aggregation over
-363-
observations is acceptable. For the original model
purposes, it was; for inquiry purposes, it was not.
purposes, it was; for inquiry purposes, it was not.
24. There are ways to tackle this problem of level or,
alternatively, plausibly to substitute another approach.
If one is to retain the basic model, one has specifically
to "window" an area of the size covered by the Inquiry,
and develop accounts of flows likely to come across the
"window frame", concentrating attention on an additional
and detailed model of movement within. Alternatively,
one could quite reasonably have taken the view that
traffic as a whole in the most relevant area, was not
really likely to change much anyway, except in its
composition, over the foreseeable future. One could then
have concentrated on considering the effects of a range
of possible moves to combat environmental deterioration
and, just as important, to improve the experience of the
population at present exposed. (It has to be remembered
that, in any area like the Inquiry's, rather fully
developed in urban terms, most of the benefits and costs
are going to be borne by interests already in it, however
these are reckoned.) Relevant measures would include
positive and negative routeing rules for lorries, of
different sizes; and measures to combat noise impacts,
etc., by screening etc., as well as detailed small
investments such as widening, limited by-pass work etc.
The main data for these investigations would have been
specially drawn samples.
alternatively, plausibly to substitute another approach.
If one is to retain the basic model, one has specifically
to "window" an area of the size covered by the Inquiry,
and develop accounts of flows likely to come across the
"window frame", concentrating attention on an additional
and detailed model of movement within. Alternatively,
one could quite reasonably have taken the view that
traffic as a whole in the most relevant area, was not
really likely to change much anyway, except in its
composition, over the foreseeable future. One could then
have concentrated on considering the effects of a range
of possible moves to combat environmental deterioration
and, just as important, to improve the experience of the
population at present exposed. (It has to be remembered
that, in any area like the Inquiry's, rather fully
developed in urban terms, most of the benefits and costs
are going to be borne by interests already in it, however
these are reckoned.) Relevant measures would include
positive and negative routeing rules for lorries, of
different sizes; and measures to combat noise impacts,
etc., by screening etc., as well as detailed small
investments such as widening, limited by-pass work etc.
The main data for these investigations would have been
specially drawn samples.
25. Such alternative approaches could also have avoided
another main difficulty with using the output of the
basic model - namely its dependence on peak hour flows.
The problems facing the Inquiry inevitably lead to a
concentration on time-of-day effects, because of concern
with particular classes of vehicles, particularly heavy
lorries. There was much discussion, also, on the bases
for pumping up peak-hour observations to become annual
another main difficulty with using the output of the
basic model - namely its dependence on peak hour flows.
The problems facing the Inquiry inevitably lead to a
concentration on time-of-day effects, because of concern
with particular classes of vehicles, particularly heavy
lorries. There was much discussion, also, on the bases
for pumping up peak-hour observations to become annual
-364-
equivalents. What is reasonable to be done in these
respects much depends, again, on time of day, week,
etc., observations in the particular area under
consideration. Such observations were not available to
the Inquiry, or it seems, to the departments, and timing
of the Inquiry of course prevented specific modifications
of existing models from being mounted. It would
nevertheless have been helpful to have recognised the
methodological issues confronting evidence to the Inquiry.
Such a review, for example, would strongly have warned
against admitting as evidence any comparisons not
involving very large differences in CBA results. Such
differences did in fact appear, but only when the proper
calculations of options were done, as noted above. More
important, it would have saved the resources and time to
produce more relevant data and to discuss them.
respects much depends, again, on time of day, week,
etc., observations in the particular area under
consideration. Such observations were not available to
the Inquiry, or it seems, to the departments, and timing
of the Inquiry of course prevented specific modifications
of existing models from being mounted. It would
nevertheless have been helpful to have recognised the
methodological issues confronting evidence to the Inquiry.
Such a review, for example, would strongly have warned
against admitting as evidence any comparisons not
involving very large differences in CBA results. Such
differences did in fact appear, but only when the proper
calculations of options were done, as noted above. More
important, it would have saved the resources and time to
produce more relevant data and to discuss them.
26. One further point of interpretation should perhaps
be added at this point. The submissions presented in
terms of Net Present Values (NPV), Benefit Cost Ratios
(BC) and the Internal Rate of Return (IRR), without any
explanation of their relevance to decisions. In fact,
NPV is used where there is no constraint on the total
amount available to be devoted to investment at the
going rate, in this case assumed to be 10% in real terms
for the central calculations. If there is a budget
constraint - that is, if the total funds available are
not enough to exhaust all the "good" projects at 10%, i.e.,
all those with a positive NPV at 10%, then some form of
recognition of the need to ration has to occur. In this,
BC ratios and/or IRR calculations become relevant. The
argument about using the different measures is therefore
essentially about the likely scale and state of the road
budget (in N.S.W. or Sydney). If it is limitedr as seems
likely, what does that imply in terms of the required B/C
ratio or IRR? A recognition that one cannot interpret the
results without such a discussion would have been helpful.
In other words, there was required some view on how PEC
and DMR foresee the justification of the level of road
investment developing in the region as a whole.
be added at this point. The submissions presented in
terms of Net Present Values (NPV), Benefit Cost Ratios
(BC) and the Internal Rate of Return (IRR), without any
explanation of their relevance to decisions. In fact,
NPV is used where there is no constraint on the total
amount available to be devoted to investment at the
going rate, in this case assumed to be 10% in real terms
for the central calculations. If there is a budget
constraint - that is, if the total funds available are
not enough to exhaust all the "good" projects at 10%, i.e.,
all those with a positive NPV at 10%, then some form of
recognition of the need to ration has to occur. In this,
BC ratios and/or IRR calculations become relevant. The
argument about using the different measures is therefore
essentially about the likely scale and state of the road
budget (in N.S.W. or Sydney). If it is limitedr as seems
likely, what does that imply in terms of the required B/C
ratio or IRR? A recognition that one cannot interpret the
results without such a discussion would have been helpful.
In other words, there was required some view on how PEC
and DMR foresee the justification of the level of road
investment developing in the region as a whole.
-365-
(iii) Valuation of the CBA Items
27. Turning to the valuation of items in the measured
CBA works: as already noted these followed well trodden
lines. On the benefit side, concern was expressed about
one feature of the main item entering the calculations,
time values, and in particular the value of small time
savings. That these were included conforms with
standard CBA practice, defensible in my view both
pragmatically and in terms of principle. Uncertainty
about this was probably enhanced by the fact that DMR had
not, in work previous to the Inquiry submissions, included
small time savings (for example in the Johnston's Creek
assessment), and by the inherent difficulty in exposing
the economic issues though these have been well canvassed
in the recent literature. Distinctions among different
classes of vehicles operating costs were not as detailed
as in UK work, but this is rather unlikely to have affected
the outcomes the models were capable of producing. (If
windowing or other techniques had been used, more detail
here would have been essential.) Accident costs were also
dealt with conventionally.
CBA works: as already noted these followed well trodden
lines. On the benefit side, concern was expressed about
one feature of the main item entering the calculations,
time values, and in particular the value of small time
savings. That these were included conforms with
standard CBA practice, defensible in my view both
pragmatically and in terms of principle. Uncertainty
about this was probably enhanced by the fact that DMR had
not, in work previous to the Inquiry submissions, included
small time savings (for example in the Johnston's Creek
assessment), and by the inherent difficulty in exposing
the economic issues though these have been well canvassed
in the recent literature. Distinctions among different
classes of vehicles operating costs were not as detailed
as in UK work, but this is rather unlikely to have affected
the outcomes the models were capable of producing. (If
windowing or other techniques had been used, more detail
here would have been essential.) Accident costs were also
dealt with conventionally.
28. On the cost side, however, greater doubts must enter.
The derivation of capital costs were not explained in
sufficient detail to judge such issues as how they might
be affected by changes in option definition, alternative
staging of work, and liability to differential price
rises with time. Also, a problem of principle emerges
which the evidence does not tackle. This is the value
(losses) to be associated with reservation of land for
possible future incorporation in the road network. The DMR
appear to argue along the following lines: if for any
reason, including the fact that the CB ratio is not
"acceptable", the department decides that the road in
question will not be built in the foreseeable future, it
will nevertheless reserve the land, to be acquired later.
Reservation is assumed to be justified because of the
very high, indeed prohibitive cost, of resumption later.
The derivation of capital costs were not explained in
sufficient detail to judge such issues as how they might
be affected by changes in option definition, alternative
staging of work, and liability to differential price
rises with time. Also, a problem of principle emerges
which the evidence does not tackle. This is the value
(losses) to be associated with reservation of land for
possible future incorporation in the road network. The DMR
appear to argue along the following lines: if for any
reason, including the fact that the CB ratio is not
"acceptable", the department decides that the road in
question will not be built in the foreseeable future, it
will nevertheless reserve the land, to be acquired later.
Reservation is assumed to be justified because of the
very high, indeed prohibitive cost, of resumption later.
-366-
This line of argument is difficult to accept at face
value. First, the cost benefit ratios should include the
present value of future alternative uses for the
reserved land. One may seriously doubt whether they in
fact do so. Second, the implicit argument is that DMR
expect the cost-benefit ratio to become more favourable
("acceptable") at some future time, a calculation in which
resumption costs will include the cost of acquiring the
land plus demolition etc. If this is so, it should be
demonstrated, or at least shown to be a reasonable
probability. It is unreasonable to argue as if resumption
costs are infinite.
C. HANDLING THE INQUIRY
29. A number of criticisms of the way in which DMR in
particular handled the Inquiry are implicit in this
discussion. Before launching on these, it should be
stressed that what the Inquiry's demands would be, and how
they might develop, could scarcely be anticipated by any
participant. The Inquiry was the first of its kind so
far as the principal actors were concerned. Hindsight
is only useful as an indicator of what can in practice be
done for the future. Thus, the aim of the following
paragraphs is to make constructive suggestions. They do
not consider the reciprocal question of how the terms of
reference or the procedures of the Inquiry might themselves
have been modified.
particular handled the Inquiry are implicit in this
discussion. Before launching on these, it should be
stressed that what the Inquiry's demands would be, and how
they might develop, could scarcely be anticipated by any
participant. The Inquiry was the first of its kind so
far as the principal actors were concerned. Hindsight
is only useful as an indicator of what can in practice be
done for the future. Thus, the aim of the following
paragraphs is to make constructive suggestions. They do
not consider the reciprocal question of how the terms of
reference or the procedures of the Inquiry might themselves
have been modified.
30. The experience as a whole leaves a strong impression
that there were two basic reasons why so much misunder-
standing, and therefore the need for subsequent explication
occurred, misunderstanding which is perhaps best
epitomised by the Commissioner's comment on the last day
of the DMR's oral submission:... "it takes a fairly careful
reading of the department's (July, 1979) submission to
work out that the department is not advocating the
building of the study option (South West)” (pg. 96, 18th
March, 1980). The first is that, because all the options
that there were two basic reasons why so much misunder-
standing, and therefore the need for subsequent explication
occurred, misunderstanding which is perhaps best
epitomised by the Commissioner's comment on the last day
of the DMR's oral submission:... "it takes a fairly careful
reading of the department's (July, 1979) submission to
work out that the department is not advocating the
building of the study option (South West)” (pg. 96, 18th
March, 1980). The first is that, because all the options
-367-
were of low priority in the DMR’s estimation, all
that was required for the Inquiry in its view was some
kind of holding operation, not a detailed justification
of highly specific routes. The second is that, in any
case, economic arguments about the worth of road
investment, of which CBA is but part, are not seen as
particularly relevant in the current state of the
department's work. This is clearly shown by the
department's willingness to subcontract economic work,
its deeming matters of economic consistency between
documents unimportant (1), and failing adequately to
monitor statements made in its name (2).
that was required for the Inquiry in its view was some
kind of holding operation, not a detailed justification
of highly specific routes. The second is that, in any
case, economic arguments about the worth of road
investment, of which CBA is but part, are not seen as
particularly relevant in the current state of the
department's work. This is clearly shown by the
department's willingness to subcontract economic work,
its deeming matters of economic consistency between
documents unimportant (1), and failing adequately to
monitor statements made in its name (2).
31. These perceptions of DMR were in effect strongly
challenged. Once the decision to inform the public of
what was being considered was made, precision in defence
of preferences became inevitable. The orange document’s
options, ill prepared as some of them have been, became
highly specific and widely known these were what the
interested parties thenceforth argued about. Cost benefit
analysis, in the sense of both values and underlying
models, did become an active issue, but the groundwork for
economic understanding had not, been laid. No sense of the
uses and limits of CBA could be conveyed, because their
uses and limits had not, it seems, been decided internally
in DMR. The combination of these factors undermined the
cohesion and consistency of the presentations.
challenged. Once the decision to inform the public of
what was being considered was made, precision in defence
of preferences became inevitable. The orange document’s
options, ill prepared as some of them have been, became
highly specific and widely known these were what the
interested parties thenceforth argued about. Cost benefit
analysis, in the sense of both values and underlying
models, did become an active issue, but the groundwork for
economic understanding had not, been laid. No sense of the
uses and limits of CBA could be conveyed, because their
uses and limits had not, it seems, been decided internally
in DMR. The combination of these factors undermined the
cohesion and consistency of the presentations.
32. The first inference for future inquiries is that, for
these public occasions at least, a commitment to some
form of CBA is inevitable. This follows from earlier
arguments about the nature of CBA and inquiries. Of all
techniques of evaluation it comes, warts and all, closest
to fulfilling the need to compare and weigh by reference to
external evidence. If not done explicitly, the choice
process will perform a CBA of some sort implicitly. Better
these public occasions at least, a commitment to some
form of CBA is inevitable. This follows from earlier
arguments about the nature of CBA and inquiries. Of all
techniques of evaluation it comes, warts and all, closest
to fulfilling the need to compare and weigh by reference to
external evidence. If not done explicitly, the choice
process will perform a CBA of some sort implicitly. Better
1. The
Johnston’s Creek Study, the PEC/DMR orange document;
the De Leuw Cather Study for example have substantial
variations all unexplained and indeed unacknowledged.
the De Leuw Cather Study for example have substantial
variations all unexplained and indeed unacknowledged.
2. Even as
late as April 2nd, DMR simply passed on to the
Inquiry a letter of explanation from sub-consultants.
As seen above, this was not satisfactory.
Inquiry a letter of explanation from sub-consultants.
As seen above, this was not satisfactory.
-368-
to be explicit, and thus recognise its limitations. And,
as a standard part of the analytical equipment to
evaluate roads, it can hardly be ignored anyway. The
second strong inference from the recent experience applies
to the whole production of evidence from models it is
extremely unlikely that a model can be taken off the
shelf which can be satisfactorily bent to answer the
exact problems put. There must be horses for courses.
The work must derive from the options to be considered;
and if comparison means estimation of new data, this
must be done, however crude it is by reference to the
complex, up-and-running, models. In the K/C Inquiry,
the problem of appraisal was to cope with detailed routes
which, however, had strong independence. The model
strategy should have been deployed accordingly, as
outlined earlier. Of course, existing work must be drawn
and built upon. But the methodological issue of the line
of attack must be settled independently. In the next
Inquiry, the up-and-running models may well legitimately
have a larger contribution. If so, this is fortunate;
but this conclusion must be arrived at, not assumed.
as a standard part of the analytical equipment to
evaluate roads, it can hardly be ignored anyway. The
second strong inference from the recent experience applies
to the whole production of evidence from models it is
extremely unlikely that a model can be taken off the
shelf which can be satisfactorily bent to answer the
exact problems put. There must be horses for courses.
The work must derive from the options to be considered;
and if comparison means estimation of new data, this
must be done, however crude it is by reference to the
complex, up-and-running, models. In the K/C Inquiry,
the problem of appraisal was to cope with detailed routes
which, however, had strong independence. The model
strategy should have been deployed accordingly, as
outlined earlier. Of course, existing work must be drawn
and built upon. But the methodological issue of the line
of attack must be settled independently. In the next
Inquiry, the up-and-running models may well legitimately
have a larger contribution. If so, this is fortunate;
but this conclusion must be arrived at, not assumed.
33. Earlier arguments have indicated several desirable
lines of development of CBA items; but the greatest need
is to decide the basic set of interests to be noted in
comparisons. The nomination of the potentially relevant
set is essentially a policy judgement. In the case at
hand, one assumes this is the prime responsibility of the
Inquiry; it has to decide who is to be considered and who
is not. In this it is useful to have a standard
comprehensive list of interests already drawn up for
characteristic road situations; the obvious candidate is
the UK Standing Committee’s “framework". Clearly PEC
and DMR would do well to anticipate demands for measuring
impacts in these dimensions. But selection among the
possible interests to be considered is necessary for two
reasons: relevance to the options being discussed, and
likely cost of estimation. Since selection is primarily
lines of development of CBA items; but the greatest need
is to decide the basic set of interests to be noted in
comparisons. The nomination of the potentially relevant
set is essentially a policy judgement. In the case at
hand, one assumes this is the prime responsibility of the
Inquiry; it has to decide who is to be considered and who
is not. In this it is useful to have a standard
comprehensive list of interests already drawn up for
characteristic road situations; the obvious candidate is
the UK Standing Committee’s “framework". Clearly PEC
and DMR would do well to anticipate demands for measuring
impacts in these dimensions. But selection among the
possible interests to be considered is necessary for two
reasons: relevance to the options being discussed, and
likely cost of estimation. Since selection is primarily
-369-
the Inquiry’s duty, and it will wish to keep open the
possibility of discerning: extra interests, some
consultation between Inquiry and PEC/DMR experts is
highly desirable before analysis proceeds. The Inquiry must
anticipate problems of filling its requirements. From
this it also follows that the most urgent need for
technical CBA development is of the concept of populations
(of the several interests) “at risk" in the sense defined
earlier. A sense of how many outside the road user set,
were being affected differentially by the options are
conspicuously lacking in the current exercise.
possibility of discerning: extra interests, some
consultation between Inquiry and PEC/DMR experts is
highly desirable before analysis proceeds. The Inquiry must
anticipate problems of filling its requirements. From
this it also follows that the most urgent need for
technical CBA development is of the concept of populations
(of the several interests) “at risk" in the sense defined
earlier. A sense of how many outside the road user set,
were being affected differentially by the options are
conspicuously lacking in the current exercise.
34. The "experts" on models are now, however,
concentrated
elsewhere specifically in UTS. Most roads in modelling
turned out to lead back to UTS, and its evidence was clear
and disinterested; the limits of what the existing models
were capable (or incapable) of was never concealed. The
problem was that this expertise was not harnessed
effectively. It would certainly be wrong, and in any case
presumptuous, to comment here on the thinking which has led.
to the present division of labour between departments.
However, there was difficulty in following lines of Inquiry
further, both because of the division of labour itself and
because of the special position of confidence held by UTS.
Future progress in modelling for inquiry purposes probably
require attention to this. In connection with this, it
should be remarked that there is no particular merit, in
these matters, in avoiding duplication of skills. It may
well be useful for DMR to acquire more in-house specialists,
but it is certainly useful for an inquiry to have its own
in-house expertise, simply because to appreciate and develop
outside contribution, to profit from them, implies having
knowledgeable people to receive and transmit technical
contributions.
elsewhere specifically in UTS. Most roads in modelling
turned out to lead back to UTS, and its evidence was clear
and disinterested; the limits of what the existing models
were capable (or incapable) of was never concealed. The
problem was that this expertise was not harnessed
effectively. It would certainly be wrong, and in any case
presumptuous, to comment here on the thinking which has led.
to the present division of labour between departments.
However, there was difficulty in following lines of Inquiry
further, both because of the division of labour itself and
because of the special position of confidence held by UTS.
Future progress in modelling for inquiry purposes probably
require attention to this. In connection with this, it
should be remarked that there is no particular merit, in
these matters, in avoiding duplication of skills. It may
well be useful for DMR to acquire more in-house specialists,
but it is certainly useful for an inquiry to have its own
in-house expertise, simply because to appreciate and develop
outside contribution, to profit from them, implies having
knowledgeable people to receive and transmit technical
contributions.
35. Finally, some indications for the future handling of
the inquiry process at a more detailed level should be
noted. First, the DMR’s letter of 7th January noted a
the inquiry process at a more detailed level should be
noted. First, the DMR’s letter of 7th January noted a
-370-
a number of issues relating to its consultant's
submission, under the DMR imprimatur, of the previous
September. Substantively, these were quite important;
more particularly the fact that the "eastern terminal”
argument could not be resolved; and the concession
that more "marginal" calculations should have been done.
But content aside, these were matters which should have
been picked up when the consultant reported. They
undoubtedly would have been, had the Department been
aware of its own need, at that time, for expert receptors
and transmitters of information. In other words, its
own economic expertise should have been deployed on these
functions. Also the warning, sounded above, about the
interpretation of individual link data is indicative of
a more general need. This is to settle a Departmental
view on matters of uncertainty surrounding all data
estimates, including the output of models. It was quite
ridiculous to have been forced under questioning into a
repudiation of individual numbers, where the data are and
are meant to be, subject to averaging. Similar points
of detailed control over numbers used emerge in the
Department's letter of 8th April. It is difficult to
conceive how, for example, reasonable statements about
future investment plans can be made if the point about
being unable to estimate cost changes until the financial
year is completed is to be taken literally, as an indicator
of the state of the Department's information on how cost
levels are changing. So also for property acquisition
costs; the doubts expressed in the letter bode ill for the
economic propriety of statements about reservation of land
for future construction, as argued earlier.
D. CONCLUSIONS
36. CBA did not play as useful a part in the Inquiry
submissions as it might have done , for a variety of reasons
CBA itself is by no means a sufficient, but it is a
necessary, tool for decision-making, and its strengths
submissions as it might have done , for a variety of reasons
CBA itself is by no means a sufficient, but it is a
necessary, tool for decision-making, and its strengths
-371-
and weaknesses were not recognised explicitly. The
Inquiry process itself inhibited this, because its wide
terms of reference, plus a focus upon very specific
road proposals, encouraged the maximum of skeptical probing
of what CBA work was done, without effective means to
modify the CBA analysis. The CBA analysis reflected the
low priority accorded to the options in DMR's view of
road priorities in Sydney. But by far the greatest
technical shortcoming would have applied to any means of
appraisal the underlying models of expected traffic
flows, and their composition, were not such as to yield
an appropriate account of the impacts of the specific
options. CBA analysis also raised questions, as yet
unsettled, about the economic strategy underlying present
road policy, including the relation between the total
budgets likely to be available for road projects in the
future and the evaluation of particular schemes, and the
preservation of routes for future, but undated, construction.
37. In terms of the needs of a future Inquiry, certain
changes in the disposition of departmental resources and
in the presentation of evidence seem desirable. Because
the subject of inquiries is specific urban routes,
reliance cannot wholly be placed on the output from the
existing Sydney-wide models; windowing, or other approaches,
must be added. Where this development should take place
presents problems which it would be absurd to attempt to
solve because of inquiry needs alone. UTS has the
expertise, DMR the ultimate responsibility for producing
and acting on proposals. It has not proved satisfactory
for DMR to delegate responsibility for submissions to
others. A larger, more direct DMR involvement seems,
on the face of it, desirable. At all events, the
development of CBA in a wider framework, to include the
systematic treatment of measurable impacts of interests
not in the traditional road user and provider set, and
recording non-measurable impacts, would be desirable from
the Inquiry's point of view. The dimensions of this, which
involves initial policy judgements, should be agreed
between the major Departments concerned, and the
Commissioner.
changes in the disposition of departmental resources and
in the presentation of evidence seem desirable. Because
the subject of inquiries is specific urban routes,
reliance cannot wholly be placed on the output from the
existing Sydney-wide models; windowing, or other approaches,
must be added. Where this development should take place
presents problems which it would be absurd to attempt to
solve because of inquiry needs alone. UTS has the
expertise, DMR the ultimate responsibility for producing
and acting on proposals. It has not proved satisfactory
for DMR to delegate responsibility for submissions to
others. A larger, more direct DMR involvement seems,
on the face of it, desirable. At all events, the
development of CBA in a wider framework, to include the
systematic treatment of measurable impacts of interests
not in the traditional road user and provider set, and
recording non-measurable impacts, would be desirable from
the Inquiry's point of view. The dimensions of this, which
involves initial policy judgements, should be agreed
between the major Departments concerned, and the
Commissioner.
- end of Annexure 2 -
