Lessons Learned from Government ICT Projects 2012

Report | 28-03-2013

Conclusions

We drew the following conclusions from our assessment of how the government had formulated and applied a number of important management instruments.

Chief Information Officer
CIOs have been appointed at all ministries since 2009. The CIOs hold different positions within the ministries, ranging from deputy secretary general to separate CIO officers. The CIOs are not yet working in the fashion we had envisaged in the report we published in 2008. They concentrate chiefly on operational management and management of the ICT infrastructure. This is useful but there is a risk that the information underpinning policy will not receive the attention it deserves.

Project portfolio management
A start has been made with project portfolio management but it is not yet a mature instrument at all ministries. Frameworks have been agreed for portfolio management but they are concerned chiefly with the largest and riskiest ICT projects.

ICT feasibility tests
ICT feasibility tests have been carried out 18 times since 2007. Three tests considered projects on a list of large and/or high risk projects. Most of the large and high risk projects were therefore not tested. Only limited feasibility tests were carried out because:

• project organisations were not familiar with the tests;
• the tests were sometimes considered to be too demanding and less exacting forms of market research were preferred.

Sourcing assessment framework
Sourcing is a process to determine whether certain activities can be carried out internally, in cooperation with third parties or subcontracted. In some areas, the sourcing assessment framework, dating from September 2012, is based on abstract propositions and principles and their implications for the sourcing assessment are not always clear. Some principles are therefore difficult to test in specific sourcing assessments.

Business cases
Business cases are designed to underpin ICT projects (i.e. provide commercial justification for them). The business cases for central government do not have a standard format. The Government Portfolio Management Manual, however, names elements that should be included in a business case. Differences in the business cases we examined make it difficult to compare government projects. Furthermore, the business cases did not contain all the elements named in the manual. The main weaknesses related to:

• information on the financing method;
• market research (or information on existing solutions);
• planned and current contracts;
• the contracting strategy.

ICT dashboard
The ICT dashboard is in effect a website version of the list of large and high risk ICT projects presented in the Central Government Annual Operational Management Report. Its information value can be strengthened in several areas.

Gateway Reviews
The Gateway Review is a quality assessment to test programmes and projects at key decision points during their life cycles. Gateway Reviews are regularly carried out of large and/or high risk ICT projects. More than 100 have been carried out in the past four years. Experience with Gateway Reviews has been largely favourable among both project leaders and CIOs.

Recommendations

CIO

• Have CIOs give higher priority to information supply in their policy fields, including the management of interministerial ICT projects.
• Strengthen the CIO's position as a link between the organisation and the information system by appointing the CIO to the management board at ministries where this is not yet the case.

Project portfolio management

• Make more use of project portfolio management on ICT projects.
• Apply the project portfolio management frameworks to smaller projects, too.
• Give the CIO a key role in setting priorities by having the CIO give an opinion before the start and at every milestone of an ICT project that only the most senior ministerial heads can depart from with good reason.

ICT feasibility tests

• Increase familiarity with the tests among programme and project managers and make them as accessible as possible.
• Ensure that the tests are based on strictly defined specifications so that market parties are able to give appropriate advice.
• Specifically state what will be done with the outcomes of the feasibility tests in the remainder of the ICT project.

Sourcing assessment framework

• Work out the principles described in the assessment framework at a lower level of abstraction and include them in a concrete checklist to facilitate the use and application of the framework.
• Formulate clear criteria on when external sourcing is not possible for security reasons.

Business cases

• Tighten up the Portfolio Management Manual requirements for the specification of business cases (particularly regarding cost and benefit elements) and apply the requirements in business cases in the manual.
• Prepare business cases as a management instrument not only before the start but also during the course of a project and pay more attention to managing the benefits (i.e. monitoring the achievement of the benefits).

ICT dashboard

• The reporting format should also consider the financial and non-financial benefits and the management, operating and maintenance costs over the entire expected life of the systems delivered by a project.
• Tighten up the specification requirements for project costs so that the cost of internal personnel and the cost of each project stage are known at all times.
• Ensure that the project accounts are consistent with the financial accounts. Furthermore, strengthen the quality assessment of the completeness and accuracy of the information disclosed in the annual reports and the ICT dashboard.
• Update the information from the ICT dashboard whenever estimates of an ICT project are revised.
• Adapt the presentation in the ICT dashboard so that it instantly displays changes in estimates from original positions, even if the changes have been approved when a project is revised.

Gateway Reviews

• Continue to use this instrument and the professional assistance provided for it.
• Subject projects to external or internal audit where the Gateway Reviews display limitations in terms of depth and/or independence.
• Encourage the learning process by disseminating lessons learned from reviews carried out.

Response

The Minister for Housing and the Central Government Sector recognises that some instruments require further development and professionalisation. He will adopt many of our recommendations. He has undertaken, for example, to: 

• extend project portfolio management to projects with a multiyear ICT component of more than €5 million;
• apply the comply or explain principle in ICT feasibility tests of projects that are expected to cost more then €20 million;
• have the central government CIO and the ministerial CIOs oversee compliance with existing business case requirements;
• include the financial and non-financial benefits and the management, operating and maintenance costs of projects in the central government ICT dashboard as from 2013.

The minister will not adopt our recommendation to appoint all CIOs to their ministries' management boards. In his opinion, it is sufficient that the CIOs have access to the management board when necessary. We think it is a shame that the minister will not adopt this recommendation because it is vital to strengthen a CIO's involvement in the policy field. We would refer the minister to the opportunities that ICT offers for a continuous and open supply of information between the various part of central government, between central government and local authorities and between central government and the House of Representatives. The supply of information is vital to steer policy implementation accountability using new techniques.