4.1 This report has focused on the ATO's income tax release and other areas of the Change Program in so far as they relate to the income tax release. For the reasons outlined in the IGT submission guidelines, a staged approach has been adopted for this review, and as a consequence, the IGT has only sought to address matters within that scope. However, during the review some matters outside this scope did arise and where appropriate these have been identified for further review consideration.
4.2 Overall, the Change Program was an ambitious and far-reaching project that aimed to deliver a range of significant capabilities designed to improve tax administration into the future. Although it may be too early to determine the proportion of benefits as against the costs of the project, assessments commissioned by the ATO quantified some benefits and expect other benefits to be realised in the short-term and accumulate further into the future. Whilst investigating the events leading up to and including the deployment of the income tax release and identifying areas for improvement set out below, the IGT specifically acknowledges the commitment from relevant ATO staff and contractors on a sustained basis over a number of years that was required for the project. Without this work, the project would not have achieved the deliverables it has to date.
The Change Program — a large, complex information and communications technology (ICT) project
4.3 The Change Program was a large, complex ICT project that took around seven years to deliver. The Change Program itself was subject to constant change with components removed and added to the initial contract scope.
4.4 The ATO had sought to replace over 180 ICT systems with a number of integrated systems. These integrated systems would help the ATO to administer Australia's taxation and superannuation laws, which for 2008-09 included:
- collecting $264.5 billion in revenue, involving the processing of 50 million forms and making of 16.9 million electronic payments
- managing 22.7 million taxpayer accounts involving the handling of 12.5 million phone calls, receiving 4 million pieces of correspondence and receiving 41 million electronic lodgements
- managing the work of around 24,800 ATO officers on an operating budget of around $3.05 billion.
LARGE ICT PROJECT — RISK RETURN OR COST BENEFIT DECISION
4.5 The key issue for Government, ATO management and the community at large is whether the ATO carries out its work of tax administration efficiently and effectively. The ATO uses a combination of manual and ICT systems to complete this work. The important consideration is how this work is to be carried out effectively and at lowest possible overall cost to the community.
4.6 ATO management uses ICT systems extensively in discharging this responsibility because, like other large organisations, it could not meet community expectations any other way. Indeed, the ATO is a significant software house in its own right, particularly after the implementation of the key Change Program releases. In designing ICT systems, a cost benefit or risk return trade off decision is required by management. This is a difficult but important matter. The question is, 'what cost is acceptable for a given level of risk in developing and maintaining an ICT system?' Or to express it another way, how close to perfect does the ICT system need to be? Is it acceptable to have certain defects or problems? If so, what kind and for how long? The ATO, like any agency, must identify and measure these costs and benefits, both tangible and intangible, and report transparently on these matters to ensure there is proper community understanding.
4.7 Certain software systems have an extremely low level of defect or problem tolerance. At one extreme, a nuclear power station or human life support system may have nil problem or defect tolerance level due to their potential cost consequences, but the associated cost of system design and its testing is much greater. Where should the tax administration ICT system be placed on this risk spectrum? Should the risks be analogous to a banking and payment transfer system?
4.8 Importantly, in using the system there is a direct cost of the system borne by the community which is outside of the direct cost to Government. This direct community cost represents a significant cost to the economy and needs to be similarly taken into account in a meaningful fashion in assessing the tax administration system's effectiveness. Indeed, many of the taxpayer and tax practitioner frustrations arise from the problems exported by the system onto them as a consequence of the new ICT systems' implementation.
4.9 The IGT has not had the opportunity to conduct any research on this matter, but it may be that community attitudes and expectations have shifted in relation to the level of direct costs that they are willing to bear in the running of the tax system, as being fair and reasonable in this context.
4.10 There may be some benefit in considering this further in the context of determining the value of goodwill and possible compensation for large ICT system short-term dislocation for disadvantaged parties (be they taxpayer or tax practitioners), where the greater good of the community is served through the operation of new ICT system.
4.11 Given the potential systemic risk for serious community dislocation from large ICT projects, it may be that the level of risk for return, or cost and benefit trade off, is one that Government may wish to consider both in more detail and also more broadly in a whole of Government sense and not just left solely to a given agency.
4.12 In relation to the Change Program, a number of key risks existed for ATO management, including the following:
- Contract execution risk — a risk that the contractor did not have the resource capacity, skills and expertise to deliver the required ICT systems. The ATO sought to minimise this risk by engaging one of the larger contractors in the industry who it believed had a proven ability to deliver tax administration systems.
- Technology risk — a risk that the technology either did not exist, or was not sufficiently proven to be reliable in providing services on the same scale and context for which the ATO required in administering the tax system. The ATO sought to minimise this risk by basing its design on an existing system, namely Singapore's Tax Administration System (TAS).
- Financial risks — a risk that the contract was not performed within the scope initially identified and therefore may lead to cost blow-outs and extended periods for delivery. The ATO sought to minimise this risk by concluding a fixed price contract not to exceed $230.7 million for outcomes.
- Contract capture risk — a risk that the ATO would became 'captive' to a contractor's judgments, to the ATO's own detriment. The ATO sought to minimise this risk by engaging two independent assurers, Capgemini and Aquitaine (although Aquitaine was engaged some years after the contract was started).
- Government policy change risk — a risk that delivery of a new Government policy would require the ATO to reallocate those resources needed for the contract's timely and cost effective completion, resulting in cost blow-outs and extended timeframes for delivery. This risk was difficult to mitigate directly because the ATO cannot reasonably ask the Government to refrain from delivering any new taxation or retirement income policy pending the completion of an ICT change. However, it is clear that the longer the contract, the greater the size of the project and the potential for that risk to be realised.
4.13 A number of other issues arose from the nature of the Change Program. The size and complexity of the Change Program demanded a high level of focus and energy over a many years. The Change Program was a highly stressful exercise that took a toll on ATO personnel. Realistically, this stretched their capacity to continue to deliver and operate as an effective team over such a long period. Certain dysfunctional behaviours were identified by the ATO assurance providers at times in this context.78
4.14 The ATO Change program system architecture design approach also has an important history. The ATO initially employed a more integrated system design approach, rather than a more modularised one. The ATO has advised that the adopted design reflected the latest proven technology available at the time.
4.15 The ATO also wished to minimise the technology risk by seeking a proven system. The TAS was built by Accenture and had attractive elements. It had proven reliable and had functional design features that appealed to the ATO. At the time, there was no reliable off-the-shelf technology solutions for interfaces between modules that facilitated such a development approach.
4.16 However, we are advised that the more recent industry thinking has shifted towards a more 'modular' software design approach, particularly now that effective interface technology solutions are available off-the-shelf (so called 'middle-ware').
4.17 Changes to the design of the system were inevitable in such a large, complex project, especially where Government policy initiatives must be implemented due to legislative enactment. Strong discipline is required to incorporate these changes into an already complex system design because of the high risk of costs overruns and delays in delivery.
4.18 At this stage, it is difficult for the IGT to assess the extent to which changes due to Government policy initiatives impacted on the ATO's discipline of maintaining the program's original scope and, therefore, costs (see for example, the extracts of Capgemini's reports in relation to the testing of the income tax release, as set out in Chapter 3). This issue may be examined in any future review of the management of the contract delivery (see below).
4.19 An integrated system that takes around seven years to deliver by a single contractor increases the overall level of risk. Contract management over such a long duration becomes increasingly difficult for a range of practical, commercial and organisational reasons. The likelihood of major Government policy initiatives arising during lengthy projects is significant.
4.20 Fixed price contracts for projects over a long duration also increase the risk of non-delivery in part or whole in the event that the contract becomes uneconomical for the contractor. This can pressure the Government agency to become captive to the contract and accept sub-optimal performance as a means to realise some benefits from the contract.
4.21 In a state government environment, the Victorian Auditor-General has also observed from his audits of a number of major, complex ICT projects over five years that fixed price contracts for large, complex ICT projects are often problematic and often deliver reduced functionality, as outlined below:
Be aware of the risks of a fixed-price contract
Issues we have observed
Fixed-price contracts are one response to the desire for certainty, transparency and probity in acquiring and using ICT resources. Such contracts can be an effective way of managing small, tightly specified projects. For larger, more complex projects; however, fixed-price contracts are often problematic.
Complex projects are not typically, (and often cannot be) completely specified in advance—the details of later stages of the project are determined by the outcomes of earlier stages. This is not necessarily an indication of weak or incomplete planning, but rather simply a recognition that knowledge will increase as the project progresses.
In this environment, using a fixed-price contract for the entire project is at best optimistic and at worst deceptive.
The result often is a project delivered with reduced functionality—for the purposes of staying within the agreed budget—or a project delivered with increased cost to provide the agreed functionality.
Practical steps to take
'Chunk' large investments
Agencies that use fixed-price contracts for ICT projects should attempt to break the projects into small pieces and contract for each piece separately.
This will require additional time for contract negotiation and funding approval, but will increase the likelihood that the expected functionality is actually delivered at the expected cost.
Agencies that use a single fixed-price contract for a large, complex ICT project should make contingency plans for the likely outcomes of overspending and under delivery.79
Need for modularised approach to large, complex ICT contracts
4.22 In light of the above risks and observations, in future, ICT projects of this size and complexity should seek to address these risks and consider a modularised design approach, minimising single contractor concentration risks, and carefully consider the commercial realities of pricing and rewarding contractor performance.
Need for independent Government governance
4.23 The ATO's independent assurers, Capgemini and Aquitaine, and the ATO's post-implementation Release 3 reviewer, CPT Global, are strongly of the view that a close working relationship between the ATO and Accenture was essential to ensure both parties were aligned and working towards the same project outcomes.
4.24 The ATO endorsed this approach as it also wanted to ensure it obtained a full understanding of the new system software by, in effect, co-developing it, so it could largely maintain the new technology in-house in the future.
4.25 Such an approach also seeks to avoid antagonistic, legalistic relationship management and better facilitates actual system software delivery. There is a residual risk that parties can become too close and lose strategic direction or get captured in a situation. The ATO sought to mitigate this risk via two main checks:
- engaging two independent assurers who, as a condition of engagement, knew they would be prevented from bidding for other the Change Program contract work
- ATO business line staff (and not the Change Program staff, be they from ATO or Accenture) were responsible for checking that the software in production met the required outcomes.
4.26 Generally, in large ICT projects such as the Change Program, there is a need for a cohesive team working together in a partnership sense as well as having the right checks and balances so that they deliver effective outcomes and robust project scrutiny.
4.27 The process for selecting the correct contractors, independent assurers and scrutineers for the risk assessment and advisory task in this context is absolutely critical for large ICT projects. An agency's peak decision-making body appropriately augmented with an independent, authoritative and specifically skilled government representative (external to the agency embarking on the project), as well as business-line personnel, is an important governance matter which also mitigates against any perception that there may not be appropriate checks in place.
4.28 One of the causes of adverse impacts on taxpayers was the delays and errors attributable to the initial lack of a full and operationally effective interface between the ATO, Centrelink and the Child Support Agency as discussed in Chapter 3 and Appendix 11. Many refunds were delayed and some were incorrectly issued when they should have been garnished.
4.29 All parties expressed frustration at the lack of effective test interfaces between government agencies in a genuine end-to-end test environment. In relation to the income tax release, a major impediment was the absence of an ATO, Centrelink or Child Support Agency test environment in which the interface could be tested end-to-end without affecting their clients' 'live' data. With the benefit of hindsight, many government officers interviewed (both ATO and others) cited a need for joint testing using real data in larger volumes, testing of the flow-on effects of using that data, and testing over a few cycles with a large scope of cases and scenarios. Inherently, this would also require a shared understanding of each agency's business requirements at a more granular level.
4.30 More broadly, Government agencies have an ever increasing inter-dependence on each others' data. Agencies now exchange and re-exchange citizen data many times over in an iterative fashion for a range of reasons. The data is used iteratively in the determination of income tax assessments and social transfer payments and also for direct offset of citizen refunds from Government against debts to Government.
4.31 This indicates an increasing need for Government ICT contracts to specifically consider significant inter-agency software interfaces and the needed for end-to-end test environments in other agencies as part of the development process. It also raises important privacy and related integrity matters.
4.32 The form or structure of the underlying data itself is also an important consideration. Where there are consistent standards across programs, the interfaces and interactions between different Government systems should be more reliably developed and maintained.
4.33 Agreed standards for data and testing (that are appropriately modified for best practice emergence) should enable more robust and reliable data exchange via interfaces, maintaining greater data integrity across Government. A key question arises as to how this should be achieved and who is accountable for ICT program governance scrutiny and the integrity of standards setting and compliance.
4.34 The IGT considers that the ATO as one of the largest government agencies is, in a prima facie sense, much better placed to manage large ICT projects of this nature than many other agencies due to its resourcing and scale of activities. Had the ATO not had this considerable organisational scale, crisis management culture and perhaps a touch of good fortune, it would not have been able to achieve this outcome effectively. Even the ATO was heavily tested and it is considered by certain parties to be in the vanguard of government ICT development.
4.35 The IGT remit only addresses the Australian tax administration system, and while there are clear learnings from this experience for the ATO, it also raises a much broader consideration for government in relation to large ICT projects and significant inter-agency data exchanges.
4.36 Important management features of the Change Program were considered and addressed by ATO management. The IGT recognises this and the recommendation below incorporates certain features adopted by ATO management for the Change Program. The important issue for the IGT is that certain standards are set for large projects development as a key management requirement and that these are maintained into the future to reduce associated risks.
For the purpose of minimising risks arising in future large scale ICT projects, the IGT recommends that the Government consider requiring the ATO, and agencies with which it has ICT interfaces, to:
- employ best practice modularised design approaches, avoid single contractor over-reliance, avoid lengthy projects and consider the commercial realities of pricing and rewarding contractor performance;
- establish improved governance and scrutiny functions by ensuring:
- the agency's peak decision-making body (appropriately augmented with skilled and experienced ICT and key business line personnel) is directly responsible for managing the oversight of the project;
- an independent, authoritative and ICT-skilled government representative (from outside the contracting agency) be a mandatory addition to the augmented agency's peak decision-making body for the project's duration (including post implementation follow up where appropriate); and
- the independent government representative report directly to Government and be required to furnish appropriate, periodic written progress reports that are publicly released at appropriate times; and,
- fully explore formal intra-Governmental protocols or standards to provide reliable system function testing, both internal to the agency and for system interfaces between relevant Government agencies, during any ICT upgrades or changes, including:
- standards for system data form, structure and definitions;
- specifying how intra-Government testing should be conducted and who is accountable for that testing and ensuring compliance with the standards; and,
- requiring periodic review processes to ensure such protocols or standards conform to current best practice.
This recommendation is a matter for Government.
ATO income tax release and ATO communications
4.37 In making the following observations and recommendations, the IGT has relied on the reports produced by the independent assurers, Capgemini and Aquitaine, and the Release 3 post-implementation reviewer, CPT Global. It is acknowledged that these parties were contracted by the ATO, therefore the IGT accepts that there may be a perception that such relationships are not completely independent as direct IGT appointments would be. The IGT has also relied on interviews with Accenture, ATO officers (past and present) and the ATO's own documentation.
Income Tax Release Testing — behind schedule
4.38 The ATO planned (in August 2008) for income tax release product testing to be completed by the end of May 2009, with user acceptance testing occurring from May to June 2009, so that the ATO could then assess the impact of the new system on its business practices and workforce for a six-month period.
4.39 However, it was clear that up until January 2010, the ATO was behind in its planned testing schedule.80 The ATO advised that this was due to a combination of factors, including:
- the unavailability of environments for extended periods at critical times
- the fact that the TaxTime 2009 build took more work than estimated due to new legislative requirements and late amendments
- the difficulty and level of complexity involved in maintaining the ATO's pre-existing legacy systems and the new Integrated Core Processing (ICP) system in parallel.
4.40 It also appears that the ATO was aware on a number of occasions that changes in design also materially contributed to the reduced timeframes for, and compression of, testing. In relation to the impact of these types of changes, Capgemini reported to the ATO in June 2008 that:
30. In the previous releases, the IA [Capgemini] observed that there has been multiple occurrences of late design change requests leading to the slippage of design schedule. This then has a cascading effect on the downstream activities affecting the overall delivery schedule. It is common and inevitable that requirements constantly change and the business knowledge is ever-evolving. The IA expects that the Change Program [CP] adheres to the design stage schedules that are agreed and endorsed in the plan, and most of all, that the CP implements the new Design Approach.81
4.41 Capgemini also reported to the ATO in November 2008 that:
The effort required to design and deliver CRs [Change Requests] is frequently underestimated.
Unanticipated CRs are being approved with limited consideration of the overall impact of the schedule.
71% of the critical resources in the CP [Change Program] are within design creating consequential delays / impacts
Planning has not adapted for a parallel release paradigm, causing compounding impacts.82
4.42 The ATO was aware that not adhering to delivery timeframes was a material risk. Since the FBT release's deployment in April 2008, Capgemini had made the ATO aware that, based on prior experience, there was a real risk that a failure to adhere to estimated timeframes for delivery would reduce the quality of the software product released into production:
Principles for Delivery
In the agreed and endorsed Delivery Methods and Plans, the delivery approach contains many valuable elements. Historically, the IA [Capgemini] has observed deviations from the delivery principles in the actual execution, and would like to see specific measures in place that ensure best practices and the delivery approach are rigorously followed. This will ensure that the solution delivered will be aligned to the documented Quality Plans.
29. In previous reports, the IA has reported that build activity eroded into testing, and that the production environment was used as testing bed. There have been many defects emerging after technical deployment into production because of insufficient testing and the lack of a real Level 4 environment that mirrors the exact environment in production, as seen since Release 3.1a FBT has gone live. The IA expects the Change Program to adhere to rigorous quality standards with regard to IPT, Partnership Testing, UAT [User Acceptance Testing] and Business Pilot, including the use of a true Level 4 environment that enables proper test of the system before it goes into production. This will help prevent the majority of defects from emerging in production.83
4.43 Capgemini repeatedly advised the ATO from December 2008 that product testing for the income tax release was behind schedule. In June 2008, Aquitaine also advised the ATO that compressing timeframes by conducting parallel product testing with business testing was also 'highly unrealistic'. It observed, among other things, that longer delivery times were needed to increase the quality of testing to ensure a high confidence in the delivery of future releases.84
4.44 It is clear that contrary to initial plans for the income tax release, the upstream delays cascaded into downstream work, contributing to significant defects being released into production (albeit that the ATO sought to reduce impacts on taxpayers and tax practitioners through mitigation mechanisms). Although the ATO had planned from August 2008 to have a stable code base, 'synched' with the TaxTime 2009 code, delivered in July 2009, the income tax release product testing was not complete until January 2010.
4.45 The ATO originally planned to conduct end-to-end business testing on a stable codebase for six months, but this business testing was repeatedly delayed and ultimately done in parallel with the product testing.
4.46 In taking a staged approach to the review, the IGT is not in a position to precisely determine why the income tax release was not delivered by June 2009 without significant further review and directly engaging independent ICT expertise.
4.47 Such a further review would examine the extent to which the independent assurers' recommendations were implemented and, if not implemented in full, whether that failure was a material contributing factor to the unexpected schedule slippage and reduced standard of quality of implementation into production.
ATO 'go live' decision
4.48 The ATO's independent assurers say that in large ICT projects it is not unusual for problems to arise after the deployment of the software. Accenture expressed the same view. Given the systems and business readiness assessment support provided by all the relevant parties, the more important issue was to identify problems and understand their impact and the nature of the required action. There are a range of reasons that have been advanced in support of this view and a number of them are discussed below.
4.49 At the outset, it is important to acknowledge that there is a point in time at which further testing delivers diminishing returns and the costs of testing outweigh the benefits (the diagram below provides a visual representation of this principle).
Number of defects resolved over time
4.50 In relation to the income tax release, it was clear that in January 2010 the income tax release was not in an ideal state in terms of the number of existing defects or defects likely to arise in production. At the time, there were 229 known 'Severity 2' defects.85 The ATO was also aware that an unknown number of unidentified defects would likely arise in production, as the rate of new defects emerging to the date of deployment of the income tax release was substantial.
4.51 More testing would have been beneficial. However, further testing would delay release deployment for another year, as January was the optimal deployment time because it minimised potential adverse community impacts. An April deployment date did not give the ATO enough of a settling in period and restricted its capacity to deal with any crises before the peak processing period in July to November.
4.52 Delaying the deployment for another year would not have addressed the interface issues with external systems unless the ATO and other agencies changed the way testing was done.
4.53 There were also perceived risks that the future viability of the project could not be sustained — for example, key people would likely move on, taking critical knowledge with them.
4.54 At the time of the decision, the ATO had considered the costs and benefits of going live. It also received substantial independent advice on the extent of the likely errors that could eventuate and the resulting impacts. Aquitaine had advised the ATO that as a result of known assessment defects it had estimated there would be an estimated 50,000 individual assessments affected annually. Aquitaine also advised that due to the ramp up and activities around the maintenance release update and preparation for TaxTime 10, it would be reasonable to expect widespread delays in processing affecting most taxpayers across the board until the end of the year.
4.55 Capgemini advised that their role was to assure the technical implementation of the release on the basis of technical elements only, not the business risks for the ATO. They noted that there were significant risks in other areas (that is, non-technical elements), but that there were mitigation strategies in place for these risks and that ultimately it was a decision for the ATO as to whether these non-technical risks were acceptable.
4.56 The ATO was confident that the incorrect assessments would be caught by the safety net, therefore no incorrect assessments would issue but would rather be delayed.
4.57 The ATO was also confident in its post-deployment problem mitigation mechanisms (see Chapter 3). The ATO could bring substantial resources to bear to ensure that all significant defects could be managed by preventing incorrect assessments from issuing to taxpayers and that those defects would be progressively resolved during the year.
4.58 One of these problem mitigation mechanisms was to fix defects through emergency fixes (e-fixes). As at 5 May 2010, 395 e-fixes were deployed, an average of approximately 28 per week from the date of deployment, although a number were the ATO manually manipulating systems controls, such as 'turning on and off' the safety net. (See the diagram below).
Number of defects resolved over time
4.59 It is clear that from the rate and number of e-fixes that needed to be deployed within the first three months of the income tax release's deployment, the ATO was in an invidious position where testing was effectively being extended into production (see also Capgemini's observations on previous releases in the extract reproduced in Chapter 3).
4.60 Having found itself in such a position, based on:
- the independent assurers' opinions, as well as that of Accenture and the ATO itself and inferences that may be reasonably drawn from them, and
- the cost and risk of further delay in deploying the income tax release,
the IGT has concluded that the ATO had little choice but to go live when it did.
4.61 Having made the go live decision, the approach to problems, defects and wider system difficulties, including how these were to be communicated to the community, required careful consideration and timely remediation by the ATO.
ATO did not communicate the significant risk of potential external impacts adequately
4.62 As stated earlier, the ATO did carry out its planned communication and intelligence collection strategy. This strategy included providing information to taxpayers, businesses and tax practitioners on the Change Program, particularly in relation to the income tax release. However, this plan and subsequent ATO communications ultimately proved to be inadequate in alerting the taxpaying community to adopt strategies that would minimise any potential adverse impacts.
4.63 At the time of deciding to deploy the income tax release, the ATO estimated that there would be significant risks.
4.64 In terms of anticipated delays in return and amendment processing, if full system functionality was achieved by May 2010, the ATO estimated that it would meet its service standards (for example, process 94 per cent of individual electronic income tax returns within 14 days) by the 2010-11 income year. If functionality was not achieved by May 2010, the ATO estimated that it would likely achieve these service standards by the 2011-12 income year.
4.65 The ATO also estimated that around 170,000 taxpayers would ask for priority processing of refunds over the February to April (inclusive) period, requiring up to approximately 520 full-time equivalent staff to action them.
4.66 The ATO also estimated that there may be significant impacts on taxpayers and tax practitioners: ...
5. External Readiness
The external community have been provided with the information that can be provided to them at this stage. Our focus has been on tax agents, BAS Service Providers, legal practitioners and large corporates. ...
However, it should be noted that other than messages to lodge early and potential impacts on processing in the new year, there has been no direct impact on the external community at this time. With the potential for significant systems errors impacting on certain classes of clients in their assessments or accounts' records, coupled with the general difficulties of a deployment of this size, the level of tolerance from the business community and tax practitioners in particular will be greatly tested.
Through the external forums it is clear that the large corporates, professional associations and tax agents do appreciate that the system will not be without significant impact on the ATO service delivery. As experienced in the tax bonus [initiative, it was shown that] as soon as there is an impact on their individual practices there is a point the ATO risks the lose [sic] of patience from the community. This has been sought to be managed through planned communication and intelligence collection processes to keep the external community informed on how to work with the new system. The success of this will not only be influenced by our actions but also the level of leeway given to the ATO over a long period.
Tax professions have advised that while they appreciate the size of the deployment and will be understanding; their members will need to be able to explain to their client's reasons for delay and any inaccuracy. How long this can be accepted will be determined by their general perceptions and feedback based on perhaps isolated instances, rather than the rate of our systems corrections or ongoing contingencies.
Based on the identified systems issues at this time, it is reasonable to assume that there is a greatly increased risk that the tax profession generally or the representative groups could much earlier than previously anticipated, lose confidence in the ATO's data integrity and processing ability or their belief that the system was ready for deployment from their perspective.86
4.67 In light of the above, the ATO publicly communicated general warnings that delays were likely be experienced until March 2010. However, the ATO did not communicate:
- the potential risks, such as there may be unforeseen circumstances and then update the public if and when these unforseen circumstances were encountered; or
- the invidious position that the ATO found itself in January 2010.
4.68 Further, when problems were encountered after deployment, the ATO did not adequately communicate (until 25 August 2010) those problems in a manner, or on a timely basis, which would have materially assisted relevant taxpayers to minimise the adverse impact on them.
4.69 The ATO did not communicate the existence of many errors until after the problems had been purportedly fixed. For example, on 29 March 2010, the ATO told the public87 of two errors (or 'glitches') three weeks after the ATO was aware of the problem and after many people had taken time to understand the error and contact the ATO to try to resolve their confusion (see also Appendix 11).
4.70 The ATO also did not adequately convey the reasons for problems encountered. For example, the ATO's 15 April 2010 website update indicated that around 30,000 returns were subject to a pre-issue compliance review (high risk refunds):
The reality is that some cases take longer to process and we would always hold some up for legitimate reasons.
For example, we would not release refunds that appeared to be fraudulent or where people may owe money to the Commonwealth, for example, other agencies such as Centrelink and the Child Support Agency. Sometimes, we also check information reported in tax returns where we find discrepancies or need more information on particular claims.
Of the estimated 100,000, approximately 30,000 returns are in this category.88
4.71 However, the ATO information did not cover all reasons. There were some returns that were not awaiting information from taxpayers or being reviewed by officers, as indicated in the website update. They were prevented from being resolved because of other systems problems, such as problems preventing the processing of amendment cases or shipping and entertainers' returns. Although the system had attached high risk refund review items to these returns, staff were unable to process these returns for significant periods because of other problems with the systems that prevented the removal of the review item (see Appendix 11). Another example is the ATO's 15 March 2010 website update which gave the impression that the problems holding up the approximate 200,000 returns held in the safety net were 'minor' in nature:
At our last update on 2 March 2010, we were on track to issue the remaining stockpiled refunds and assessments for income tax returns lodged in February by the end of last week.
Last week we experienced some minor problems which have delayed us issuing some of those remaining stockpiled refunds and assessments while we ensure the integrity of our data.
There are approximately 200,000 stockpiled assessments yet to issue (of which we estimate 100,000 are refunds). These include assessments which involve a baby bonus, entrepreneur tax offset, primary production averaging, exempt foreign employment income, special professional averaging, eligible termination payments or superannuation lump sum payments and non-resident withholding tax.
We have fixed these minor problems and can start releasing most of these refunds and assessments from today (with the exception of assessments involving non-resident withholding tax).89
4.72 However, the errors preventing the issuing of assessments for these returns would have likely been classified as Severity 1 defects because the errors may have led to incorrect assessments being issued, if it were not for the existence of the safety net.
4.73 It is clear that with the benefit of hindsight, the ATO's publicly expressed confidence in resolving delays by March 2010 was optimistic. However, if the ATO had given early, clear warnings of the potential problems in a manner that enabled people to take action to minimise the potential risks, then it is likely that the community would have given the ATO more leeway, suffered the impacts with a reduced level of emotive reaction and been better placed to minimise the adverse impacts on themselves.
4.74 The ATO could have better managed expectations by communicating the worst case scenario as well as the best case scenario and the risks associated with each of these scenarios for taxpayers. For example, in the dynamic environment of fixing problems in the first couple of months of deployment, the ATO should have recognised that taxpayers and tax practitioners (although not happy that matters were not running smoothly) only wanted to be assured that the ATO was aware of the problem, working to address it and could give a future date for the taxpayer to take action if they had not received by that date either their NOA or a further update. Taxpayers did not necessarily want a firm undertaking on when returns would issue if that could not be met with a reasonable degree of certainty.
4.75 Tax practitioners wanted advance and specific warning of potential adverse impacts and made in a manner which would allow them to take action to mitigate them. This was made clear to the ATO at a meeting between the ATO and tax professional bodies on 22 April 2010. The ATO responded to this by issuing on 28 May 2010 a list of issues it was working on or had recently resolved.90 However, until 25 August 2010, the ATO did not do this with sufficient specificity. Without detailed knowledge of the likely risks, one can form the impression that the ATO perceived the delays to be insubstantial, small in number or that the majority of delayed returns were due to compliance risk checks.
4.76 Generally where the ATO did seek to warn parties, the word 'delay' proved too simple and subtle in communication. If there is a problem, it needs to be communicated transparently and specifically rather than just the consequence of the problem, being a delay. Such an approach did not allow people to assess their own position and risk effectively.
4.77 During August 2010, some tax practitioners with business models that offer fast refunds advised the ATO (for example, at the 6 August 2010 ATO-Tax Practitioner Forum meeting) that they were experiencing processing difficulties. Compared to previous years, these practitioners were experiencing an approximate 30 per cent drop in numbers of NOAs issuing within 14 days of lodgement. The ATO did not acknowledge that such delays existed until 20 August 2010.91
4.78 However, based on the ATO's 10 September 2010 website communication,92 these tax practitioners' observations appear to be justified.
To ensure that the community is fully informed and is in a position to take appropriate action with respect to potential adverse effects of significant software releases or large ICT implementations (including TaxTime requirements), the IGT recommends that the ATO, in future, communicate via the appropriate mediums and in real time:
- the risks, in specific, meaningful and transparent terms;
- the impact of the risks; and
- any changes to the identification or impact of those risks, (including additions, solutions or mitigation strategies).
For major ICT deployments (such as TaxTime), the ATO will incorporate into our organisational project management methodology a requirement that specific consideration should be given to communicating:
- the risks, in specific, meaningful and transparent terms;
- the impact of the risks; and
- any changes to the identification or impact of those risks, (including additions, solutions or mitigation strategies).
ATO did its best in the circumstances to fix problems as they arose
4.79 Notwithstanding the concerns regarding the ATO's communications, the ATO did its best in the given circumstances to fix problems as they arose, diverting significant resources to mitigate and resolve issues. It was suggested to the IGT that the ATO has a good crisis management culture. It was also fortunate that certain events and strategies did come together at the right time to facilitate this outcome. In saying this, it is equally important to recognise, however, that in certain cases individual taxpayers and tax practitioners suffered (and in some cases are still suffering) adverse impacts from the new systems' introduction until the problems were fixed.
4.80 Set out below is a discussion of some of the ways in which the ATO sought to mitigate risk and resolve issues.
4.81 In production, the income tax release was subject to an average of just under 28 e-fixes per week during the first 3 months (as at 3 May 2010, 382 e-fixes were deployed from the date of deployment).
4.82 CPT Global has since advised the ATO that e-fixes have the potential to create significant problems in production (as two of the main problems were caused by e-fixes with limited testing) and that ad hoc e-fixes should be consolidated into releases to enable greater testing.
4.83 The safety net became an important feature of the system. In fact, without it, the ATO may not have decided to go live when it did.
4.84 The safety net was a late feature that was designed after the October 2009 pre-deployment testing identified numbers of errors that could not be fixed before deployment and for which there was no developed workaround. It was a preventative measure designed to minimise such adverse impacts.
4.85 The ATO successfully used the safety net to minimise the risk of incorrect assessments being issued. Around 180,000 returns were held in the safety net over differing periods of time pending the resolution of identified problems.
4.86 There was also potential to improve the functionality of the safety net by allowing it to distinguish on the basis of combinations of data fields. It is triggered if an identified field is present. In some circumstances, it is not the existence of a particular data field that is of concern, but its combination with other data fields that may cause problems — therefore, some returns may be caught by the safety net unnecessarily. An improved functionality would allow the ATO to release returns which, although they have a particular data field present, do not otherwise give rise to concern.
4.87 It should be noted, however, that there are downsides to the overuse of the safety net. It should not become a permanent feature because, without a strong business impact focus, it may create a temptation to delay resolution of issues and provide a false sense of security. By its very nature, it also ensures that certain taxpayers will experience delays in the issuing of their NOAs.
Integrated support model
4.88 The ATO had a disciplined problem identification, escalation and prioritisation system — the integrated support model (see Appendix 8). This allowed any ATO officer to raise an issue, have it considered and prioritised for resolution. This system helped the ATO to quickly respond to identified problems on the basis of risk.
4.89 Overall, over 3900 issues, including around 2300 problems with business processes, were identified and managed by the ATO through its integrated support model.
4.90 This model could be further improved if tax practitioners were incorporated into it. Tax practitioners provide another reasonable perspective to the functioning of the tax administration system. As is evident by some of the problems encountered, tax practitioners were sometimes the first to identify those problems.
Manual processing of assessments
4.91 The ATO also undertook to manually process (when requested by taxpayers) around 7000 returns that were significantly delayed. Manually processing a tax return required the ATO to invest significant resources, not only in being able to deliver a refund quickly to taxpayers, but also in terms of the remedial work that needs to be done to correctly reflect the calculations on the ICP system.
Staff levels and internal management reporting
4.92 The ATO had ample numbers of staff to work on mitigation strategies. However, the initial management reporting did not enable the ATO to best understand how the system worked and where forms were in the system. Specifically, it did not enable the ATO to identify 'hot spots' so that problems could be proactively mitigated. Initial reporting somewhat hamstrung the ATO's ability to identify where staff should be deployed. If the ATO had deployed the income tax release with better reporting on review items, then problems would have been identified and resolved earlier. However, as internal reporting developed, targeted deployment of staff improved in effectiveness. Better familiarity with the system should also improve the ATO's ongoing ability to accurately predict the volume and type of work and also the time at which it will arrive.
4.93 The IGT found that the ATO's internal management reporting is not as comprehensive or detailed in relation to particular aspects of the ICT output performance or processing details. As noted earlier, the ATO prioritised resources towards the key income tax release system processing for go live readiness. Aspects of management reporting were affected by this decision. The ATO had a range of broad aggregate information on how the income tax release system was progressing. However, specific details of exactly what kinds of returns were moving through the system and their associated level of complexity was not readily available. Many concerns were raised in this context by tax practitioners in particular as to what kinds of returns were being successfully processed, rather than understanding just the broad system aggregate number the ATO were announcing.
4.94 A collateral impact of the ICT system being so large and complex is that particular categories of taxpayer or tax practitioner may have concerns or problems that are not readily seen in the management reporting from the ICT systems. It may be that other reporting mechanisms do seek to overcome any perceived shortfalls in service delivery, but there are risks that they may not work.
4.95 Clearly from the materials and submissions the IGT has received, some real problems for specific taxpayers and tax practitioners continue to arise. Whether these are so called 'glitches' or bigger administrative problems, it is difficult to assess at a high level without more granular reporting processes. As noted previously, there needs to be some recognition of these difficulties and the impact that they have on the affected parties. Where the ICT systems are said by the ATO to be working effectively, the assumption is that they are meeting the expectations of both Government and the broader community as to what is acceptable cost effective service delivery in this context.
4.96 A similar concern was raised regarding the impact that these difficulties may have in certain hardship situations. The ATO does seek to address hardship situations as raised by affected parties, but perhaps this area requires deeper consideration in a proactive management sense as other agencies like Centrelink recognise.
The IGT recommends that the ATO design and implement improved ICT reporting of output performance and processing details, in consultation with external stakeholders, to ensure there is better transparency and understanding of the system's operation, for example, the type of returns that are not successfully processed as opposed to broad system aggregate statistics.
The ATO will continue to publish regular "issues logs" on the Tax Professionals section of the ATO website to alert tax agents of any issues as they arise where the processing of returns may be delayed.
Further, the ATO will continue its consultation with tax agents and professional associations to assist them to better understand and utilise information on the Tax Agent Portal.
4.97 The ATO also relied upon a strong relationship with the Child Support Agency (CSA) and Centrelink. This relationship enabled the ATO to quickly identify and resolve problems (subject to the two areas noted below) based on frequent and routine inter-agency contact with people who knew how the problems would affect clients. This interaction was based on a shared understanding that where potential problems might arise there was a need for both agencies to take action to resolve them. For example, the ATO and Centrelink shared call centre scripting to enable taxpayers to receive consistent messages from both agencies.
4.98 However, quicker remedial action by the ATO to address certain problems would have reduced the extent of delays and therefore numbers of complaints and resulting reverse work flows. For example, the ATO told the CSA that a certain problem was not a priority to resolve. The ATO escalated the priority for resolution (3-4 months after the problem was identified).
4.99 The issues of inter-agency compensation is worthy of consideration in this context. The CSA needed to dedicate resources to work around the problems that the ATO's systems created. It may be a difficult area, but with ever increasing complexity in ICT data interchange between Government agencies, this may be a very significant issue in future where formal protocols are appropriate. To do otherwise may send the wrong signal to larger agencies about the true cost impact in a whole-of-Government sense. Naturally, as with any important working relationship, the matters would need to be of sufficient significance to warrant engaging in the process or effecting actual payments.
Economic accountability for adverse economic impacts on taxpayers and tax practitioners
4.100 Currently there is a range of compensation options open to claimants who believe they have been adversely affected by the income tax release problems and insufficient ATO communication, including:
- payments made under the Taxation (Interest on Overpayments and Early Payments) Act 1983 — to compensate taxpayers for the time-value of taxpayers' refunds where the ATO takes more than 30 days to issue credit NOAs after tax returns are lodged
- payments made under the Commonwealth Scheme for Compensation for Detriment caused by Defective Administration (the CDDA Scheme)
- act of grace payments made under section 33 of the Financial Management and Accountability Act 1997
- ex gratia payments (the power to make these payments emanates from the Government's executive powers under section 61 of the Constitution).
4.101 Many tax practitioners have asked the IGT what steps they could take to seek compensation for the losses they claim to have incurred. This indicates a need for the ATO to better advise the community on their avenues for compensation.
For the purpose of addressing tax practitioners' lack of awareness of the appropriate avenues for compensation claims, the ATO should ensure that it specifically notifies tax practitioners of the different avenues for compensation claims, including how to make claims for:
- payments made under the Scheme for Compensation for Detriment caused by Defective Administration;
- act of grace payments made under section 33 of the Financial Management and Accountability Act 1997;
- ex gratia payments; and
- any other relevant legislative or administrative compensation payments that may exist.
The ATO website provides information on how taxpayers and tax agents can make a claim for compensation. This includes information about when losses can be compensated, service standards and where further information can be obtained. The ATO website also provides an application form which details what information is required to be submitted as part of a claim.
The ATO will highlight this information and the relevant application forms for tax agents in future communications.
4.102 The IGT also considers that the ATO should assess, as part of the initial design of the project, the potential detriments that a large scale ICT project may impose on taxpayers. This should include principles or guidelines against which claims for compensation could be assessed and processed to evaluate claims for compensation. Such an assessment should help focus the ATO on the ground rules for which compensation is payable and focus initial risk assessments as projects develop. The ATO would benefit from the reduced resources needed to consider numbers of claims. The IGT has not had the opportunity to conduct any significant research into international norms that may exist in this regard, but it is certainly an issue of some concern to the community.
4.103 The IGT remit is only directed at the ATO, but as noted earlier the IGT considers that developing a proactive approach to the issue of compensation is worthy of consideration by other Government agencies.
For the purpose of minimising risks arising in future large, complex ICT projects, the IGT recommends that the ATO consider for future projects, whether it should have guidelines in place early in the development of the project to assess and process claims for compensation by members of the community for substantial detrimental impacts imposed.
Agree in part
For future large and complex ICT projects the ATO will consider whether additional guidelines are needed to assess and process possible compensation cases.
4.104 As a result of problems arising from the income tax release deployment, the main loss suffered by taxpayers was the time value for money held by the ATO for extended periods of time. Compensation for this type of loss is currently addressed by the Taxation (Interest on Overpayments and Early Payments) Act 1983. The ATO had encountered difficulties in meeting its obligations to make such payments to these types of taxpayers due to problems with the ICP system. However, in September 2010 the ATO completed the payment of interest to those who are known to be owed amounts greater than $50. Amounts less than $50 have not yet been posted to accounts; the proposed ATO course of action is to post the credits to the accounts and then write them off. The credits will then be re-raised when future action occurs on the account. This process is to stop small dollar amounts being released/issued to agents when they are posted to the account.
4.105 Tax practitioners claimed, amongst other things, that they incurred the following types of losses:
- damage to their reputation by reason of ATO communications that did not acknowledge ATO problems
- lost time in dealing with the ATO and their clients in attempting to resolve problems, which they would not have done so if the ATO had acknowledged the specific problems and given expected dates for resolution or update
- obtaining third party finance to maintain cash flows for periods after 1 March 2010 where the level of expected refunds did not occur and the actual levels did not allow outgoings to be met.
4.106 In late April 2010, the ATO moved to reduce the risk of reputational damage to tax practitioners by issuing an open letter containing an apology from the Commissioner for the delays experienced.
4.107 In late August 2010, the ATO also published a list of known issues in a manner that would help taxpayers and tax practitioners to reduce unnecessary costs in following up certain types of returns. A list of some of the known issues had been published earlier, however, this list was not communicated in a manner that helped taxpayers and tax practitioners to minimise their costs. This is because it was communicated after the issues had been resolved or had not specifically identified the problems.
4.108 In relation to the impacts of delayed returns on certain tax practitioners' cash flows, the IGT considers that the ATO has been sufficiently aware for a number of years of the business model of certain tax practitioners — that is, those that rely on high volume, prompt taxpayer refunds. These businesses typically minimised risk by accepting certain clients who were of lower risk of delayed returns (for example, salary and wage earners with lower risk profiles).
4.109 Tax practitioners were also concerned with the process for claiming compensation. They claimed that, given the large numbers of practitioners affected and the likely low amounts involved, the costs for claiming compensation was disproportionate to the likely amounts payable. They suggested that the ATO could implement an expedited compensation system which imposed minimal cost to tax practitioners and provided clear criteria for compensation.
4.110 The ATO advises, on the basis of its internal advice, that compensation is not payable under the CDDA scheme because there is no defective administration:
Has there been defective administration by the Tax Office?
26. The answer to this is no.
27. The implementation of the Tax Office's new system is a major upgrade, involving the transfer of a significant amount of data and complex systems. There have been no critical systems problems, and overall it is working well. A considerable number of returns have been processed and refunds have been paid.
28. We have accepted that there have been processing delays brought about by the implementation of our new system, but we do not consider that our actions in managing this implementation give rise to compensation. The Tax Office remains committed to ensure the reliability of our processes and the integrity of our information, even if this slows down the implementation and processing times overall. Given the magnitude of the systems overhaul, we do not consider that the consequential delays in the processing of returns, activity statements and related documents were either unreasonable or avoidable. Specifically, we do not consider that the time taken to implement our Change program and process tax returns amounts to defective administration within the meaning of the CDDA scheme.
29. In determining whether there has been defective administration, the test is not what would or should have occurred in a perfect world, but what a reasonable person would expect given the same circumstances, same powers and access to resources. The reality is that no implementation of a major computer upgrade of the kind undertaken by the Tax Office could be achieved without some delay or minor processing issues. This had been acknowledged by the Tax Office in its public broadcasts, and the timing of the implementation over the Australia Day long weekend in 2010 was chosen due to the reduced impact on taxpayers and tax agents. Accordingly, the fact that there have been delays and some processing issues does not mean that there is defective administration. The assessment of defective administration must be based on what another reasonable agency could achieve with the same circumstances, powers and resources, and such a comparison would not lead to a conclusion that the Tax Office has been defective or unreasonable in its implementation. ...
48. Adopting this uniform approach is not to say that under no circumstances would a compensation claim relating to delayed refunds or delayed processing not be considered and paid, but in order to accept a claim, a decision maker would need to be satisfied that there had been defective administration in the processing of the income tax return other than the mere fact of the implementation of the Change program.93
4.111 It could be argued that all tax practitioners were aware that a project of this size was likely to adversely impact certain taxpayers and tax practitioners. However, the IGT considers that this conduct is not the only relevant ATO conduct that has caused taxpayers and tax practitioners' loss. The ATO also:
- was aware of the problems that were likely to be encountered and that these would result in delays to issuing refunds causing loss to taxpayers and tax practitioners
- communicated that general delays would be experienced from December 2009 to March 2010, however, did not communicate the delays occurring after 1 March 2010 in manner that allowed taxpayers and tax practitioners to take action to mitigate their losses (see Appendix 11).
4.112 The IGT also considers that it would have been reasonable to alert these taxpayers and tax practitioners to specific problems or potential problems that impacted on them so that:
- it allowed taxpayers and tax practitioners to take alternative action to mitigate their losses
- the agency could minimise the number and negative emotive tone of taxpayers and tax practitioners' contacts with the agency.
4.113 On the above basis, the IGT considers that the ATO should, in consultation with the tax practitioner community, robustly and openly reconsider its position on compensation claims under the CDDA scheme in light of the facts above and, in the event that the ATO decides to change its position on these claims, reconsider the process by which such claims be made.
For the purpose of addressing tax practitioners' concerns with the basis for, and process to obtain, compensation, the IGT recommends that the ATO work with the tax practitioner community to robustly and openly reconsider its position on compensation claims under the CDDA scheme and the process by which such claims should be made.
The CDDA scheme does not operate on the basis which involves consultation about findings of defective administration. The ATO considers each case on its merits.
As at 30 November 2010, the ATO has received 59 claims from taxpayers and 35 claims from tax agents for a total of 94 claims. Each of these claims for compensation under the CDDA scheme are considered on their merits. For an individual taxpayer, interest is paid with the delayed refund if a notice of assessment is issued more than 30 days after the income tax return is lodged. The ATO will continue to consider current claims and any future claims received on their merits.
Change Program impacts on ATO staff
4.114 The IGT received a number of submissions from ATO staff concerning a range of issues, including the following:
- Front line ATO staff cited increased stress levels dealing with taxpayers and tax practitioners in an environment where they considered that they could not offer reasonable solutions.
- Processing staff cited increased stress levels dealing with increased workloads, with workdays sometimes extending until the early hours of the morning.
- Accounting staff cited concerns with back-end accounting integrity and reconciliation issues arising in the new system.
- Interpretative assistance staff and compliance assurance staff considered that the CMWS reduced functionality and productiveness.
- A range of staff considered that the CMWS did not comply with occupational health and safety requirements and that the ATO may have also breached disability discrimination law.
- A range of staff were dissatisfied with ATO management's consideration of their input into the system's design and usability and the ATO approach to change management.
4.115 ATO management advises that substantial work was done within the organisation to minimise the risk of adverse impacts on ATO staff. ATO management advises that it considered that it had ensured that its business leaders were involved in the design and configuration of the system.
4.116 The ATO understands that there were concerns around the customisation of the CWMS. ATO management considers that the new CWMS imposed new disciplines and visibility on case work. Managers were now able to understand workloads of staff and how they were progressing against case plans.
4.117 The IGT understands that the ATO decided to implement a 'one-size fits all' CWMS, balancing customisation against the adaptability of the system into the future — the ATO's priority at the time was to implement a CWMS and consider customisation after the completion of the Change Program. Submissions to the IGT assert that there are now other products on the market, such as Infopath, that have the potential to provide ATO staff in differing areas with effective front-end customisation of the Siebel CWMS. This would mean that the Siebel CWMS itself would remain untouched, only the user interface would be tailored to suit particular ATO functionalities. Whilst the IGT has not fully investigated the possibility of such software solutions, it seems that customisation of a front-end interface with Siebel would be likely to increase user acceptance.
4.118 Although the ATO had taken substantial steps to minimise the adverse impacts on staff, the level of ATO staff angst indicates that further work is to be done.
For the purpose of addressing ATO staff concerns in relation to the systems arising as a consequence of the Change Program, the ATO should conduct open and frank post-implementation consultation with its staff and:
- understand the causes for ATO staff concerns; and
- communicate the ATO's consideration of those concerns including what action, if any, the ATO intends to take in relation to each particular concern.
Agree in part
The ATO undertook extensive consultation with our people during the various phases of the Change Program. The engagement of our people to ATO values, objectives and priorities is very important to us and we will continue to consult with our people to understand and where possible, resolve any issues which may be a concern for them.
The ATO's ICT capability development and ongoing care and maintenance of the new systems
4.119 Having spent just under $800 million in implementing the Change Program (see further discussion below) steps need to be taken to minimise the risk of the ATO finding itself in similar circumstances to that which required the initiation of the Change Program. The new system needs to be maintained and steps taken to make full use of its value.
4.120 One of the main reasons for the need to change the ATO's existing legacy systems was that these systems were encumbered with a range of limitations with the software's design. For example, the ATO's main system to process income tax returns, the NTS, was based on a batch processing system which impeded tax practitioners and taxpayers' real time access to data held on that system. As the batch processing and edit checks were performed overnight, it prevented real-time correction of forms with multiple errors.
4.121 There is a risk that a similar deterioration of adaptability could happen with the new systems. This is discussed below.
CPT Global's recommendations
4.122 The ATO engaged an independent ICT specialist, CPT Global, to conduct a technical review of the income tax release in order to learn from what had taken place and determine a forward strategy with TaxTime 10 looming.
4.123 Upon the completion of CPT Global's review, the IGT was able to study the resulting report and held discussions with the author. CPT Global made a number of observations and recommendations (which are set out in Appendix 13). Many of the observations and recommendations are consistent with the IGT findings.
The IGT recommends that:
- the ATO continue to consider CPT Global's recommendations (as set out in appendix 13) and publicly report on their progressive implementation; and
- in the event that the ATO does not implement any of CPT Global's recommendations, the ATO provide public reasons for not so doing.
The ATO's Change Program Steering Committee agreed on 19 August 2010 to implement all of the recommendations made by CPT Global. Work on all recommendations has commenced. Some recommendations have already been implemented. A report on progress will be provided on the ATO's website.
Implementing new Government tax and retirement savings policy in the new ICT environment
4.124 The cost savings or benefits of the new ICT system may not be fully realised if the design of future Government tax and retirement savings initiatives fall outside the functionality of the new system. A new tax or retirement savings policy may incur more set up cost if the ATO cannot accommodate it within the current system functionality.
4.125 The policy and legislative design process for new tax measures currently requires that the ATO provide advice to Treasury on administration issues. The ATO/Treasury protocol — Tax Policy and Legislation sets out that the ATO will provide advice on administrative issues, including the design and build of systems to administer tax measures. The IGT reinforces the need for this advice to include that avoidable costs have not been incurred for the ATO to redesign systems due to a policy or legislative mismatch with existing ATO systems.
Changes in scope of the Change Program
Change Program finished on 30 June 2010, with work continuing until 30 June 2011
4.126 The ATO has advised that the contract with Accenture finished on 30 June 2010, with work under the contract continuing until 30 June 2011. During the life of the contract there were substantial changes.
4.127 The ATO attributes one of the main changes to the contract to the requirement to implement the Government's new superannuation policy — Change Order 38 to Work Order 9, which incorporated the superannuation changes into the contract and cost around $204 million in direct contract costs.
4.128 Although implementing the Government's new superannuation policy was incorporated into the contract, some significant deliverables were also later excluded from the scope of the contract. The ATO has decided not to complete the Superannuation Guarantee and Business Activity Statement (BAS) releases under the contract. The ATO has negotiated a refund from Accenture. However, the ATO has recently issued a 'work order' to Accenture to do work on the design of the Income Tax Instalments release. At this stage it is unclear how this work will be funded or performed.
4.129 Notwithstanding these major changes, there remains other major uncompleted work.
4.130 The ATO systems still have no single client register or a single back-end accounting system. Very general estimates are that it may need another 30,000 to 40,000 work days to complete this work. It is unclear whether this work will be completed in the future.
4.131 In particular, during the course of this review, ATO personnel had raised concerns with internal reconciliation processes relating to the back-end accounting systems and receipt allocation to taxpayer accounts. It would be appropriate for the ATO to arrange an internal audit review of these specific back-end accounting system issues, whether the above work is completed or not.
4.132 The above indicates the need for a broader review to examine the exact scope and cost of the Change Program, including variations and an assessment of indirect costs (such as staff involved in the implementation but not included in the Change Program cost centre).
4.133 As previously noted, the IGT has taken a staged approach with this review and has not sought to review contract performance due to current resource constraints and the need to engage specific ICT expertise. A review of the performance under this contract would be useful for broader understanding of large ICT project management in the future and should be considered at a later stage.
4.134 Such a review could also examine whether anything could be learnt to minimise the risk of under-estimation in the planning phase of work needed to deliver such products and how that work should best be tracked to provide an evidentiary basis for more accurate predictions of completion dates.
The costs and benefits of the Change Program
4.135 The ATO recently engaged Aquitaine to assess the costs and benefits of the Change Program. In its final report, issued on 22 September 2010, Aquitaine set out the costs of the Change Program as follows:94
Costs of the Change Program
Source: EST Change Program Program Management Office, ATO Finance
4.136 It is also important to take note of Aquitaine's conclusions, some of which are set out below:
As at its completion in June 2010 the Change Program had cost $799 million, of which $582.1 million was self funded by the ATO from the expected efficiency gains. The balance was funded by government to implement a range of major legislative reforms between 2005 and 2010.
The Change Program delivered a series of major releases of new capabilities from 2003 to 2010. The final planned release, to convert [Business Activity Statement] BAS-related products such as GST [Goods and Services Tax] and PAYGI [Pay As You Go Instalments] to the new Integrated Core Processing System (ICP), was not completed. We have not adjusted the planned benefits for the BAS Release and so have tracked against the originally planned budget and benefits.
In 2004, the Change Program business case had identified $155 million in direct and specific efficiency gains expected to be enabled by the new capabilities to be introduced. When converted to 2010 equivalent value, this represents $183 million in expected benefits. However the many changes in the ATO since 2004 made these specific benefits difficult to track. Therefore in this review, we have analysed the benefits from a top-down perspective — examining the changes in the ATO from 2003 to 2010 and then seeking to understand any linkage to the Change Program capabilities.
Overall, we have identified $147-153 million in annual efficiency benefits in the Operations and Compliance Sub-Plans. This represents a 4 year payback period on the self-funded component of the Change Program. These benefits have come from three principal areas:
- Headcount reductions in Client Account Services;
- Headcount reductions in Customer Service & Solutions;
- Productivity gains in MEI [the ATO's Micro-Enterprise and Individuals business line] for desk-based audit activities supported by analytical models. …
The non-financial benefits to the ATO are also quite extensive, in particular the use of enterprise systems to replace many fragmented product-based systems. We have contrasted the "before" and "after" situations to illustrate the extent of this change. Overall, we believe that the ATO has gained significant benefits in a more flexible workforce using common enterprise systems. This allows the ATO, in large part, to assign work nationally based on need and the national availability of the required skills. Teams can be reassigned to different work as required and work can be reallocated electronically to meet targeted service standards. Most of these benefits stem from the introduction of enterprise work management and case management, which are now mature products and well integrated into the ATO's business processes.
We note that the enterprise systems in concert with an improved use of analytics for case selection have contributed to creating a more transparent and accountable ATO. …
We have observed a relatively long lead time for the realisation of benefits of 2-3 years following a particular release, as new systems and processes are assimilated into the organisation. Consistent with this, for some major deliverables where benefits were expected, we have found the new systems and processes still too recent to determine whether the expected outcomes will be achieved. These include:
- The introduction of new systems for income tax processing. This was delivered in January 2010 and resulted in delays to processing returns while residual defects were eliminated, it is now processing peak volumes satisfactorily;
- New systems and processes for provision of interpretive assistance to the community. This has experienced difficulties in some aspects of its functionality, such as correspondence generation, and is currently being reviewed and refined to address these;
- The expected reduced effort in modifying systems to deliver new tax products. While we see promising signs of gains in productivity and flexibility, the recent introduction of the full ICP system has provided insufficient evidence as yet of substantially faster effort;
- Similarly, the annual systems rollover of the tax year appears to have reduced in effort and time, but on the basis of one year's data we regard this as too early to be conclusive.
Overall, the Change Program has been an ambitious and far reaching undertaking for the ATO. Not all of the planned scope was completed, and a number of major legislative reforms were required to be introduced to addition to the already major program of work. We believe that the ATO has derived significant benefits in internal efficiencies from the program, as indicated by a 4 year simple payback calculation on the $582 million self-funded investment. This is based on the benefits observed and does not take account of benefits that have yet to be realised from the more recent releases. Moreover, the ATO has created a strong capability that extensively rationalises their internal processes and systems and which provides a strong platform for future development. The short term issues following the Income Tax release should be seen in this context.95
4.137 Aquitaine's report provides a useful first step in assessing the costs and benefits of the Change Program. It may be useful to conduct a further cost/benefit analysis sometime after the Change Program work is completed and the new ICT systems have achieved full functionality and are well-settled operationally. In the meantime, there are likely to be many ongoing opinions on what should be considered in such an assessment and how they should be accounted for — for example, should the costs include those borne by the wider community in relation to problems encountered post-deployment as well? To this end, as a starting point, the ATO should release relevant information in its possession to promote open and transparent discussion in this regard.
For the purpose of improving the transparency surrounding the assessment of the costs and benefits of the Change Program, the IGT recommends that the ATO publish in full Aquitaine's final report on its review of the costs and benefits of the Change Program.
The ATO will publish on the ATO website the final Aquitaine report on the Costs and Benefits of the Change Program.
78 Aquitaine Consulting, Review of the Change Program at the June 2008 Replan, report to the Australian Taxation Office, Canberra, 15 July 2008, p. 1.
79 Victorian Auditor-General's Office, Investing Smarter in Public Sector ICT, July 2008, Melbourne, p. 29.
80 See for example, Capgemini's report, Australian Taxation Office Easier, Cheaper and More Personalised Change Program, Independent Assurer Report Version 1.0, Period covering 7th February 2009 - 27th February 2009, report to the Australian Taxation Office, Canberra, February 2009, pp. 3-4.
81 Capgemini, Australian Taxation Office Easier, Cheaper and More Personalised Change Program, Independent Assurer Report Version 1.1, Period covering 1st June 2008 - 30th June 2008, report to the Australian Taxation Office, Canberra, June 2008, p. 17.
82 Capgemini, Australian Taxation Office Easier, Cheaper and More Personalised Change Program, Independent Assurer Report Version 1.0, Period covering 10th November 2008 - 5th December 2008, report to the Australian Taxation Office, Canberra, November 2008, p. 8.
83 Capgemini, Australian Taxation Office Easier, Cheaper and More Personalised Change Program, Independent Assurer Report Version 1.1, Period covering 1st June 2008 - 30th June 2008, report to the Australian Taxation Office, Canberra, June 2008, pp. 16-17.
84 Aquitaine Consulting, Review of the Change Program at the June 2008 Replan, report to the Australian Taxation Office, Canberra, 15 July 2008, p. 1.
85 A Severity 2 defect is defined as 'the incident restricts the usability of the application/system, but the application/ system itself is running. There is no sustainable workaround available': CPT Global, Release 3 — Income Tax Implementation Review, report to the Australian Taxation Office, August 2010, p. 29.
86 Australian Taxation Office, Business Readiness: Executive Summary, Business Readiness Assessment for Change Program Release 3 Income Tax, document attached to the agenda for the 22 December 2009 meeting of the Change Program Steering Committee, pp. 1-6.
87 Australian Taxation Office, Latest update from Second Commissioner David Butler - 29 March 2010.
88 Australian Taxation Office, Processing status of stockpiled tax returns - latest update 15 April.
89 Australian Taxation Office, Latest update from Second Commissioner David Butler - 15 March 2010.
90 Australian, Taxation Office, Income tax processing - issues and status - 28 May 2010.
91 Australian, Taxation Office, Progress report 4 - income tax returns since 1 July 2010.
92 Australian, Taxation Office, Progress report 7 - income tax returns since 1 July 2010, 10 September 2010
93 Australian Taxation Office, Internal ATO correspondence, 13 April 2010.
94 Aquitaine Consulting, Review of the Benefits from the Change Program: Final report, report to the Australian Taxation Office, Canberra, 22 September 2010, pp. 6-7.
95 ibid., pp. i -iii.