Alex Budzier’s Presentation to the London meetup
On 10 April Alex Budzier kindly delivered a talk entitled Changing How Projects Are Delivered: Insights from studying 11,000 projects to the project data analytics meetup. I have been following his work for years and it has really helped to shape my thinking on how data can shape how we deliver projects. It is often used at public enquiries and expert witness testimonies; he is delivering some exceptional work. I was overjoyed when he agreed to share his work with our meetup community.
Alex started his presentation referencing the mayor of Rome citing his research as a reason for Rome pulling out of the Olympics. A man of international repute. He presented a slide capturing the frequency of cost overrun on a range of projects, for example rail projects have a mean cost overrun 38%, with 7 out of 10 projects experiencing an overrun; IT projects have a 74% cost overrun, with 4 out of 10 projects experiencing an overrun – which highlights the existence of a ‘fat tail’. This helps to reinforce the hypothesis that different types of projects have a different predisposition to variance. Something we should be able to prove through data analytics.
The killer slide for me was where he demonstrated that only 0.5% of projects are delivered on time & on cost & on benefits. He concluded that we must try something different and I agree. The PMI Pulse of the Profession charts also illustrate that project delivery performance has largely been static for the last 8 years. We must try something different.
The key take homes for me were:
- We are on a current journey where we are seeking ever more experienced people and better models to give us better estimates. We get them involved in risk assessments and underpin it with detailed monte carlo analysis; which implies a degree of precision. I liked the quote “Its optimism dressed up as science”. But the evidence indicates that people’s judgement is biased and this is hard coded into each of us. We are on the wrong path. The exhaust plume of data emitted from projects enables us to moderate the bias. Alex is currently doing this at project level using reference class forecasting. But as we get more forensic data we can take the principles of reference class forecasting to a WBS level and use graph database technologies to interconnect the schedule, risk, contract and a raft of other ‘features’ to understand the predisposition of projects to variance.
- 1 in 3 hours worked in the UK are spent working on projects. Productivity in the 2 project focused sectors (construction and professional services) has not improved in the last 20 years. By delivering projects differently we have a huge opportunity to make a difference.
- His work on government 3 point estimates were compelling. The outside view, i.e. ral data from comparable projects illustrates that we are seriously underestimating the range of outcomes, particularly at the top end.
- He cautioned that if we open up the data into the public domain it can drive the wrong incentives, with people manipulating the data. He provided clear evidence of this from a US dataset. Public officials were clearly adjusting the data to tell the story that they think people want to be told. I acknowledge this, but is it a reason to not open up the data or do we need to manage it? The creation of data trusts should also help. But I would also counsel that data analytics can help to highlight where data is being manipulated and discourage it, with consequences for people who have gamed it. We are entering a new era.
- The data illustrates the variance in out-turn for certain types of projects. By engaging clients in this discussion we provide an opportunity for them to understand how to manage the factors that influence out-turn rather than seek the most economically advantageous tender. We change the way bids are assessed.
- We also need to change human behaviour. Alex highlighted that currently most project data is of poor quality. My own perception on this is people are ‘feeding the machine’ and don’t see the value of investing effort in it. If we can give them insights from the data they are getting value from it, which in turn drives up quality. Furthermore, if we can use data science to assess and report data quality, then we can develop change programmes to improve it. I agree with Alex that its not solely a technology driven solution.
- Alex highlighted that we need to get better at data auditing, data quality etc. I would argue that we can use data science to do a lot of this.
The slides to Alex’s presentation can be found here:
A video of Alex’s presentation can be found here:
Martin Paver is CEO/Founder of Projecting Success. In Dec 2017 he saw the opportunity to transform how projects could be delivered through the application of advanced data analytics and founded the London Project Data Analytics meetup, which has expanded throughout the UK and has grown to a community of ~3000 people. He delivers strategic and tactical solutions that integrate project management and leading edge data science and analytics.
Please share with us any comments or feedback. Many thanks.