Tuesday 29 March 2016

Some sobering research on infrastructure

A wee while back I posted about the outstanding service a modest piece of infrastructure had provided to the Auckland region, and went on to make the case that there could often be unexpected spinoff benefits from infrastructure investment, over and above those envisaged at the time: I mentioned the example of the Auckland Harbour Bridge in particular. Hence and otherwise I got to the point that with government borrowing costs currently so low, there was a good case for the government to accelerate or increase the infrastructure spend.

But I was given pause for thought by a comment from 'Wellington Haiku' - I don't know for sure who that is, but one Wellington organisation is given to announcing its research in haiku - who referred me to the work done by Bent Flyvbjerg at Oxford University's Saïd Business School, and in particular to his paper, "Survival of the unfittest: why the worst infrastructure gets built—and what we can do about it".

The gist of his argument is that infrastructure projects typically cost far more, and are far less productive, than imagined in their planning:
Cost overruns in the order of 50 per cent in real terms are common for major infrastructure, and overruns above 100 per cent are not uncommon. Demand and benefit forecasts that are wrong by 20–70 per cent compared with actual development are common.
The reason that they turn out that way, is that infrastructure planners have been gaming the people who pay for the projects:
promoters and forecasters intentionally use the following formula in order to secure approval and funding for their projects:
               underestimated costs+overestimated benefits=funding
Using this formula...results in an inverted Darwinism, i.e the survival of the unfittest. It is not the best projects that get implemented, but the projects that look best on paper. And the projects that look best on paper are the projects with the largest cost underestimates and benefit overestimates, other things being equal...Therefore the projects that have been made to look best on paper in this manner become the worst, or unfittest, projects in reality, in the sense that they are the very projects that will encounter most problems during construction and operations in terms of the largest cost overruns,benefit shortfalls, and risks of non-viability.
And Flyvbjerg goes on to recommend a whole series of sensible improvements to the governance of infrastructure investment, including the compulsory introduction of private capital as 'skin in the game' to incentivise better financial performance.

I have to say, post Flyvbjerg, my optimism about infrastructure has taken something of a knock. But as I brooded a bit, I'm not sure that everything Flyvbjerg said made total sense to me.

One thing that seemed a bit strange was that, if a high proportion of infrastructure budgets really do run well adrift of plan, why isn't there more of a political fuss? We've had a few projects go awry in New Zealand - the INCIS police computer system, the Novopay payroll system - and they've invariably been heavily picked over afterwards (both were subject to formal ministerial inquiries). Why, if they really do systematically go off the rails, aren't we seeing infrastructure projects in the dock every other day?

And I'm not sure why the project paymasters would keep getting gulled - you'd think they'd wise up, and that there'd be a kind of arms race to build ever stronger protection against ever more blatant puffery. At one institution I know, for example, projects needed to meet an internal rate of return target of 19% and a payback period of two years - ruled designed to filter out all but the strongest cases (though people still tried brazen reverse-engineering of project numbers to fit the criteria). The NZ Treasury's required 8% real rate of return is arguably a similar device.

And even if project promoters are gaming the system over and over, you'd think the best projects would still be the ones most easy to manipulate to show supersized net benefits, and it could well be that even though they didn't live up to the hype, they were still well worth doing. The metric of success Flyvbjerg uses (actual versus expected usage) is only a partial view of overall value: what if a new road was underutilised relative to forecast, but was used enough to relieve a major bottleneck on a big highway? What if (like the Auckland Harbour Bridge) it unleashed a whole new avenue of development wholly outside the original cost-benefit calculus?

All that said, Flyvbjerg's general point must surely be right: people need to be realists about the often exaggerated anticipated benefits from infrastructure. But my point about unanticipated benefits in at least some cases must surely be right, too: who'd have thought, for example, that the Plain Old Telephone Service ('POTS') carrying voice over copper wires would end up supporting ADSL and VDSL speed broadband?

The reality is that every infrastructure investment, like any other investment, extinguishes some previous options, and creates new ones, and we may not even know what some of those options are, let alone their value, until after the event (maybe long after, as with the Harbour Bridge). As I said in the original post, "That doesn't mean that every social planner should have free rein to add on arbitrary benefits on a "build it and they will come" basis". Investment is an experiment, with uncertain results: there will be losers (and probably more losers than I previously thought), but if you make an over-cautious number of experiments, you'll also truncate your chances of the unanticipated payoffs.

5 comments:

  1. NZTA occasionally does post-implementation reviews of its projects. I did a bit of quick-and-dirty analysis of the results here: http://transportblog.co.nz/2014/11/17/road-funding-survival-of-the-un-fittest/

    Costs seem to blow out, but there doesn't seem to be a trend towards systematically over-inflating benefits. And while NZTA's PIR dataset doesn't include a lot of really large projects, it may suggest that NZ's performance is better than what Flyvbjerg observes internationally.

    Another data point is that Auckland's motorway and arterial road network was originally (1956) projected to cost £15 million - or around $800 million in today's dollars. By 1962 they'd raised the cost estimates to £40 million ($2 billion today). That's still a laughable under-estimate...

    One reason to expect optimism bias to be a perennial issue for public infrastructure projects is that political actors can benefit from announcing or opening projects, even if they're not actually socially beneficial. Voters reward politicians who are "doing stuff", and the costs are often sufficiently dispersed to be invisible.

    Private participation in project financing hasn't necessarily enhanced financial discipline. The Australian experience with private toll roads has led private investors to favour an "availability payment" model in which demand risks remain with the government. Consequently, an availability payment PPP model doesn't offer _that_ many advantages over standard design-and-build contracts.

    ReplyDelete
    Replies
    1. One reason why the cost overruns are less than found in the international studies that is that databases from megaprojects: projects can be more than $1 billion.

      A number of the post-implementation reviews are a very small road upgrades and so forth of a couple of million dollars. If you cannot spend few million dollars and stay in budget, things would be pretty hopeless.

      Delete
  2. Thank you. That's very helpful indeed. I did wonder myself if Flyvbjerg's data set of large to very large projects (which you'd think would carry more intrinsic risk than run of the mill smaller ones) might not be representative of New Zealand.

    ReplyDelete
  3. The first estimate of the cost of building Stadium New Zealand on the Auckland waterfront was $500 million. 2 weeks later also the estimate was $900 million. Cabinet pulled the plug on that despite the heady excitement of hosting the Rugby World Cup

    ReplyDelete
  4. I'm not Motu! I can only dream of such poetical and economic heights.

    Seems to me Mr Nunns has the right of it with his comments about the political economy aspects. There's much more political gain in announcing new stuff than in going back later and assessing how far the expected benefits have actually been achieved in practice. This applies much more broadly than to infrastructure construction projects, of course, and it's not by any means limited to the NZ context.

    ReplyDelete

Hi - sorry about the Captcha step for real people like yourself commenting, it's to baffle the bots