The ITRIP models
2.1 The ITRIP comprises a suite of income tax risk models which use 'a series of known behavioural attributes and fraud indicators to build a risk profile for individual income tax returns'.44 The current suite of models are the identity crime and network detection model, the 'expert business rules' model and the pattern detection model. Each of these is discussed below.
Identity crime and network detection model
2.2 The ATO has advised that the identity crime and network detection model 'uses known attributes of identity fraud to detect high risk returns using pre-set thresholds. Networks linked to this high risk population are subsequently identified and their size, value and growth assessed'.45
2.3 The identity crime and network detection model aims to detect cases of potential identity fraud, such as where a taxpayer's details have been stolen and a tax return lodged in their name with any refund due being paid to another person or entity other than the taxpayer themselves. This model also utilises analytics to:
- identify commonalities between different returns;
- identify possible networks of perpetrators of fraud lodging such returns; and
- to seek to address this risk by attacking the networks rather than the individual preparers.
2.4 The ATO advises that this model is currently applied to lodgements received from all channels. It is owned and managed by the Serious Non-Compliance (SNC) business line. Due to the high risk and severity of identity crime, the inputs to this model are considered highly sensitive and the ATO quarantines access and authorisation to modify the model to a limited group of officers within that business line.
Expert business rules model
2.5 The ATO defines the expert business rules model as:
… a suite of rules constituting both label-based rules with risk-based criteria, and watch lists of suspect individual identifiers (tax file numbers, bank accounts, IP addresses and ESIDs (computer identifiers), ABNs). The structure is flexible to facilitate response to emerging fraud issues.46
2.6 The expert business rules model seeks to identify abnormalities recorded at specific labels within the tax return, claims in risk areas identified by the ATO or income tax returns which bear suspicious indicators on the ATO's watch lists, such as TFNs or bank accounts the ATO considers to be compromised.
2.7 To illustrate certain risks the expert business rules seek to address, the following sample is provided:47
- claims for spouse offset to which the taxpayer is not entitled;
- claims for education tax refund when the taxpayer appears to be too young to support a dependent school age child;
- inflated amounts of tax withheld or fabricated employer details in relation to payment summaries; or
- returns which contain references to TFNs or bank accounts which are currently on the ATO's watch lists as representing a risk to the revenue or the taxpayer.
2.8 The expert business rules model is generally responsible for stopping the majority of returns which are held for review under the ITRIP. In particular, the ATO notes that the rules relating to spouse offset, education tax refund and payment summary fraud (as outlined above) stopped the majority of returns which were reviewed under the ITRIP in 2011-12.48 This may change from year to year as the mix of models applied to the ITRIP changes depending on the risks needing to be addressed.
Pattern detection model
2.9 The pattern detection model uses data mining to identify organised tax evasion patterns or groups. Such patterns may include multiple returns which share abnormal commonality suggesting that the returns may have been prepared by one person or a group of people. The ATO refers to this person or group as the 'guiding mind'.49 A particular focus of the pattern detection model is identifying income tax returns which are prepared by a single or groups of unregistered tax agents.
2.10 The ATO has advised that as at 1 July 2012, the pattern detection model was only applied to returns lodged through its e-tax system. However, the ATO is currently undertaking a project to expand the application of this model to other lodgement channels such as mail lodgements and lodgements made by telephone.50 Some expansion of the model had occurred by the end of August 2012.
Updating the ITRIP models
2.11 The ATO has advised that the ITRIP models are dynamic and changes to them may be effected within a period of between 48 hours to two weeks. It notes that this is a significant improvement from the hard-coded approach adopted in the HRR program.51
2.12 The ATO formally reviews the ITRIP models and expert business rules on an annual basis, incorporating observations regarding adjustment rates from different models and rules, intelligence on areas of potential fraud and changes to legislation (such as an increase in the tax free threshold). The formal review may lead to a refinement of existing models and business rules, or new business rules being developed and implemented with the approval of the Assistant Deputy Commissioner, MEI Active Compliance.52
2.13 The ATO notes that prior to implementing any new business rules, 'a sample of post-issue audit cases is undertaken to ensure the intended risk is being targeted appropriately'.53
Effectiveness of the ITRIP models
ITRIP strike rates
2.14 A strike rate is one means of assessing the effectiveness of a particular approach or strategy. A strike rate may generally be defined as the proportion of selected taxpayer cases, in which the relevant risk was confirmed resulting in a positive outcome or action, measured against the total population. Those taxpayers incorrectly selected or whose tax returns were released without action are not included and are considered to be false positives in the risk identification process.54
2.15 As the ITRIP incorporates a number of different models which are aimed at detecting and addressing different risks (for example, overstated claims, identity crime and fraud), a positive outcome may include a reduction of refunds claimed, a referral for prosecutorial action or an increase in taxpayers' returns to reflect claims to which the taxpayer may be entitled but not otherwise claimed.
Overall ITRIP strike rates
2.16 The ATO's strike rates for the ITRIP in aggregate for all risk models used are outlined in Table 6.
2.17 In the ITRIP's first full year of operation, 2009-10, it was only applied to e-tax lodgements for the purposes of identifying unregistered preparers and fraudulent returns through the pattern detection and identity crime and network detection models. Limiting the ITRIP to e-tax and only applying selected models resulted in a high strike rate of 93 per cent.55
2.18 However, with the expansion of the ITRIP to lodgements through all other channels and the inclusion of expert business rules to identify errors or incorrect claims within specific labels, a general decline in its overall strike rate has resulted. Between 2009-10 and 2012-13, the overall strike rate reduced from 93 per cent to 68 per cent, as shown in Table 6. However, the average adjustment from ITRIP returns increased between 2011-12 and 2012-13 from $2500 to $4098.56
|Strike Rate (%)||93||71||76||68|
|Average adjustments ($)||N/A||3,323||2,500||4,098|
Source: ATO, Evolution of Income Tax Refund Program 2009-13; ATO, 2011 ITRI Final Treatment Evaluation; Senate Estimates Briefing October 2012; ATO information dated 5 June 2013.
2.19 The ATO acknowledges the decline in the ITRIP's overall strike rate and has undertaken work to specifically address the models and expert business rules which it considers to have contributed to this overall decline. As part of this program of work, the ATO has more actively monitored and refined the outputs of the affected models and adjusted low risk parameters in a more timely manner to ensure that cases are not unnecessarily held up for review.57
Identity crime and network detection model strike rate
2.20 In addition to the overall strike rate, the ATO's data also captures the strike rates in relation to returns which are stopped by its identity crime and network detection model. This particular strike rate is a subset of the overall ITRIP strike rates outlined in Table 6.
2.21 The identity crime model strike rates for the years 2010-11 and 2011-12 as reported to the Senate are outlined in Table 7.
|Year||Total income tax returns reviewed by ATO||Revenue protected by review
|Returns stopped by the ICM||Returns adjusted from ICM cases||Percentage of returns adjusted in ICM cases||Revenue protected by ICM review
|Average revenue protected by ICM cases
Source: ATO, Senate Estimates Briefing October 2012
2.22 Table 7, above, shows that of all the returns stopped by the ITRIP, only a fraction were as a result of risks identified by the identity crime and network detection model. In 2010-11, this accounted for 7,269 cases (being 21 per cent of total returns reviewed by the ATO). In 2011-12, 7,924 cases were stopped by the identity crime and network detection model, accounting for about 10 per cent of total returns reviewed by the ATO.
2.23 Of those returns which were stopped by the identity crime model, the ATO reports that it made adjustments in 6,427 cases (or 88 per cent) in 2010-11 and 4,894 cases (or 62 per cent) in 2011-12. The average adjustment rate, however, has increased between the two years from an average of $2,349 to $4,413 per case.
Expert business rules and pattern detection strike rates
2.24 The ATO's data in relation to the strike rates for its expert business rules and pattern detection model are not reported publicly. These are collated and analysed by the ATO for internal evaluation purposes. Like the identity crime model statistics above, strike rates in relation to the expert business rules and pattern detection models are also subsets of the overall ITRIP strike rates outlined in Table 6, above.
|Channel||Model||Returns completed||Total returns adjusted||Strike rate (%)||Total adjusted ($)||Average adjusted amount ($)|
|Agent||Expert business rules||7,573||5,174||68||14,928,127||2,885|
|E-tax||Expert business rules||4,449||3,283||74||8,946,432||2,725|
|Other self||Expert business rules||2,995||2,690||90||19,783,862||7,355|
|Unknown||Expert business rules||239||192||80||1,426,935||7,432|
Source: ATO, 2011 ITRI Final Treatment Evaluation
2.25 As outlined in Table 8, above, in 2010-11, the ATO reported that the pattern detection and expert business rules models had a strike rate of 86 per cent and 74 per cent respectively when applied to tax returns lodged through e-tax.58 The strike rates for expert business rules applied to tax agent lodgements and lodgements by taxpayers through channels other than e-tax were 66 per cent and 88 per cent, respectively.59
2.26 At the IGT's request, the ATO has constructed a three year comparison of the strike rates achieved by the expert business rules and pattern detection models for 2010-11 (based on the table above), 2011-12 and 2012-13 (sourced from other internal evaluation documents). This comparison is outlined in Table 9 below.
|E-tax||Identity crime (IDC)||78%||68%||66%|
|Expert business rules||74%||83%||74%|
|Tax agent||Expert business rules||66%||69%||57%|
|Non e-tax or other self preparer||Expert business rules||88%||89%||72%|
|All channels and models||71%||76%||68%|
2.27 The IGT notes that, like the overall strike rates and those for the identity crime and network detection model above, there is a general declining trend in the strike rates for the expert business rules and pattern detection models across all channels. The IGT notes that the declining trend is most pronounced in relation to the expert business rules applied to tax agent lodgements and lodgements by unrepresented taxpayers through channels other than e-tax.
2.28 This declining trend could suggest a need for the ATO to more closely monitor the performance of the expert business rules model and to undertake more stringent sample testing before applying these rules broadly. There may be valid reasons for this decline. However, the complete 2012-13 year return data is not yet available and the ATO has not yet undertaken a full evaluation. Accordingly, the IGT is of the view that it would be premature to draw any final conclusions.
2.29 While the statistics in Tables 6, 7 and 9 show some decline in strike rates, both for the ITRIP overall and the models individually, the ATO notes these strike rates continue to exceed its own expectations of achieving a 55 per cent strike rate.60 It is unclear to the IGT how the ATO arrived at this expectation. The IGT has requested details of the ATO's considerations in developing this 55 per cent benchmark but as at the date of this report, has not been provided with those details.
2.30 The IGT also notes that when compared with other compliance verification strategies, such as the ATO's use of benchmarks in the cash economy,61 the strike rates in the ITRIP are considerably higher. The comparison of the strike rates between different compliance initiatives is indicative only. This is necessarily so as these programs adopt different methodologies and target different risks and taxpayer populations.
2.31 As a general observation, the IGT believes that it is difficult to assess whether a particular strike rate accurately indicates the effectiveness of a risk assessment tool when the rate of non-compliance within the relevant population is unknown.
2.32 It is also important to point out that while examining strike rates can provide a broad overview of the general effectiveness of certain projects or strategies, caution must be exercised as these are not necessarily conclusive. This is because a number of other factors can also directly affect strike rates. In the case of the ITRIP, strike rates and average adjustment figures may be attributable to factors such as changes to underlying legislation,62 improved taxpayer compliance with fewer errors in lodgements, more sophisticated methodologies for identifying fraudulent returns or changes made to relevant risk models between the years.63
2.33 Notwithstanding the comparatively high strike rates above, the impact of holding a return that is later confirmed to be of lower or no risk can be significant. As discussed later in this report, delayed processing of income tax returns generates a considerable level of taxpayer and tax agent complaints. The IGT notes that as part of the ATO's intended improvement work, the ATO should focus on reducing the numbers of these types of returns from being stopped.
Level of fraud detection from the ITRIP
2.34 One of the key aims of the ITRIP is to identify and stop claims which are potentially fraudulent.64 Where a return is stopped by the ITRIP on the basis of suspected fraud and the ATO reviewing officer cannot substantiate the claim or satisfy themselves of the absence of the fraud risk, there are three options available to ATO officers to escalate the potentially fraudulent case. These options are:65
- comprehensive referrals for investigation and prosecution by the SNC business line under a joint funding agreement. The ATO notes that these referrals involve the most egregious cases of non-complying taxpayers, repeat offenders and taxpayers who fail to comply with the ATO's notices to give information under section 264 of the Income Tax Assessment Act 1936 (ITAA 1936). The ATO acknowledges that these referrals, when compared with other approaches to dealing with fraud, are comparatively low. The ATO estimates that, between 2011-12 and 2014-15, 25 matters will be referred by the SNC business line to the Commonwealth Director of Public Prosecutions for action. In addition, the ATO expects to complete 150 in-house prosecution matters;66
- identity fraud cases which are reported to the SNC business line by the CAS business line but in which prosecution activity is generally not suitable because of difficulties in identifying those suspected to have committed the fraud. Information from the ATO indicates that, in 2011-12, there were 1,915 such cases reported while, in 2012-13 (as at 3 June 2013), 3,240 cases have been reported;67 and
- fraudulent and dishonest behaviour cases which are reported to the SNC business line in accordance with the ATO's internal corporate management practice statement on fraud control.68 The ATO notes that all cases in which a penalty of more than 50 per cent (for recklessness) is imposed are reported to the SNC business line under this category. Where the ATO considers a case may be appropriate for prosecution, this is referred to the SNC business line as a comprehensive referral. In 2011-12, the ATO reported 1,645 such cases, while in the current year; it has reported 660 cases (as at February 2013).69
2.35 The ATO notes that, in 2010-11, there were difficulties in specifically identifying the number of comprehensive referrals to SNC from the ITRIP.70 The recorded information for 2010-11 shows that the MEI business line comprehensively referred 356 cases to the SNC business line, of which the ATO can only identify two being as a result of the ITRIP.71 This accounts for approximately two per cent of all tax returns in which there was an adjustment in favour of the revenue.
2.36 In 2011-12 and 2012-13, the MEI business line undertook more comprehensive recording of its referrals and their progress and outcomes through the use of a manual spreadsheet. These records indicate that 170 and 112 cases were referred to the SNC business line in 2011-12 and 2012-13, respectively.72 In 2011-12 and 2012-13, the level of referrals to the SNC business line was less than one per cent of cases adjusted in favour of the revenue. As a fraction of total returns stopped by the ITRIP, the levels of referrals across all years represent an even smaller fraction.
2.37 The ATO acknowledges the low levels of comprehensive referrals to the SNC business line for investigation. The ATO notes that this level of referral is directly related to a joint funding agreement between the SNC business line and the MEI business line for the investigation and prosecution of fraud cases arising out of the ITRIP between 2011-12 and 2014-15.
2.38 However, the ATO also notes that in addition to comprehensive referrals to the SNC business line, it also takes action to treat cases in which fraud is identified but which are not suitable for prosecution. Such cases include instances of identity crime. In these cases, TFNs are stolen and fraudulent income tax returns are lodged claiming refunds.
2.39 The ATO has advised that prosecution action may be frustrated in cases of identity crime due to difficulties associated with identifying the perpetrators. However, the ATO has also advised that it takes a number of other remedial actions such as requiring proof of identity for issuing refunds and cancelling compromised TFNs and reissuing new ones. Accordingly, the ATO considers that a measure of comprehensive referrals to the SNC business line alone is not an accurate measure of the effectiveness of the ITRIP as a fraud detection and treatment mechanism.
2.40 The IGT notes the ATO's advice that the low levels of fraud referrals and prosecution action undertaken by the ATO is limited by internal funding decisions and also by difficulties associated with identifying the perpetrators of fraud. The IGT also notes the treatments applied by the ATO to cases in which prosecution is not suitable. This is reflected in the relatively high quantity of cases identified and adjusted on the basis of identity crime and pattern detection. As illustrated in Tables 7, 8 and 9, a total of 9,379 and 10,02873 cases were adjusted as a result of identity crime and pattern detection in 2010-11 and 2011-12, respectively.
2.41 Notwithstanding the above, the IGT is of the view that prosecutorial action and specific court rulings in fraud cases serve as the most visible and public outcomes of the ATO's action against fraud. Absent these, there is a risk of a general public perception that the ATO is ineffective or that its program of work in relation to refund integrity is disproportionate to the risks identified.
2.42 It is also worth noting that in addition to the above reasons, the low levels of referrals may be due to the vast majority of individual taxpayers voluntarily complying with their taxation obligations and that discrepancies may be the result of errors ranging from simple mistakes through to negligent statements, rather than dishonest or fraudulent behaviour.
2.43 Having already examined the overall ITRIP strike rates as a general measure of effectiveness, the levels of referrals to the SNC business line raise two further questions:
- should the ATO continue to group incorrect or potentially fraudulent claims together when publicly referring to the ITRIP; and
- is the pre-issue compliance action taken by the ATO proportional to the materiality of the risk posed by these income tax returns?
2.44 These questions are examined in Chapters 5 and 6, respectively.
2.45 At this point, it is important to note the ATO's current limitations on capturing and reporting data in relation to SNC referrals from the ITRIP. As noted earlier, since 2011-12, the MEI business line has utilised a manual spreadsheet to track those ITRIP cases which have been referred to the SNC business line. The spreadsheet also records the nature of the risk, the reasons for referral and the outcomes of any investigation or prosecution action taken against the taxpayer.74
2.46 The ATO advises that this spreadsheet is maintained by the MEI business line for its own purposes and does not interface with reporting by the SNC business line. Moreover, as the spreadsheet is manually maintained, any reporting must also be manually generated through a review of the cases recorded and notes made in relation to the progress of those matters.
2.47 While the IGT acknowledges that the spreadsheet represents an improvement on the more generic records of the earlier year, there is a risk that a manually maintained document may be inadvertently corrupted, data lost or information being inaccurate or not up to date. As the referrals are tied closely to the SNC business line's own work and investigations, the absence of cohesion between the reporting mechanisms also creates some concern.
2.48 Given a key outcome from the ITRIP is the prevention of fraudulent claims, the IGT considers it imperative that there is appropriate record-keeping and reporting to enable the ATO to identify and assess the level of accuracy with which the ITRIP detects cases of potential fraud and the effectiveness of the related strategies. Such data would not only assist the ATO to continually update and refine its fraud detection parameters and strategies but, when publicly reported, would also enhances community confidence in the ATO's efforts to address fraud more accurately with minimal compliance costs.
The IGT recommends that the ATO:
- ensures that the MEI, CAS and SNC business lines collaborate to develop a comprehensive record-keeping system to identify and report on:
- the number of cases referred from the ITRIP to the SNC business line for fraud investigation and the number of these cases where actual fraud was established;
- the number of cases in which identity crime was observed and those in which the ATO suspects that false tax returns were lodged by perpetrators of crime; and
- the actions taken by the ATO to address the fraud and the outcomes of such action.
- distils common factors in cases where fraud activity has been identified and use such findings in its annual review process of the ITRIP models to improve their ability to detect potential fraud.
44 ATO, Senate Estimates Briefing, October 2012.
45 ATO, 'Risk Treatment Plan - Income Tax Refund Integrity' (September 2012), internal ATO document, p. 8.
46 Above n 45.
47 ATO, '2011-12 Expert Business Rules Quick Reference Guide' (2012), internal ATO document.
48 Above n 13, page 5.
49 ATO, 'Risk Assessment - Income Tax Refund Integrity' (January 2012), internal ATO document, p. 11.
50 Above n 45.
51 ATO, Communication with the IGT, 18 February 2013.
52 ATO, Communication with the IGT, 12 April 2013.
54 Above n 26.
55 Above n 16.
56 Above n 16.
57 ATO, communication with the IGT, 17 May 2013, p. 2.
58 ATO, 'Micro Enterprises and Individuals Final Treatment Evaluation - Income Tax Refund Integrity Project (2010-11),' (17 February 2012), internal ATO document, p. 14.
60 ATO, 'Individuals Pre-Issue Overarching Strategy' (13 March 2012), internal ATO document, p. 11.
61 IGT, Review into the ATO's use of benchmarking to target the cash economy, 4 October 2012, p. 62.
62 For example, Tax Laws Amendment (2012 Measures No. 1) Act 2012 which phased out the Dependent Spouse Offset.
63 Above n 58, p. 5.
64 Above n 13, page 4.
65 Above n 57.
66 Above n 57, p. 7.
67 ATO, Communication with the IGT, 5 June 2013.
68 Corporate Management Practice Statement PS CM 2007/02 Fraud Control and the Prosecution Process.
69 Above n 57, p. 8.
70 ATO, Communication with the IGT, 21 March 2013, p. 5.
72 ATO, Communication with the IGT, 18 July 2013.
73 This figure comprises 4,894 identity crime cases as outlined in Table 7 and 5,134 pattern detection cases (extrapolated from 5,769 total returns stopped as a result of pattern detection and an 89 per cent strike rate as indicated in Table 9); ATO, 'Income Tax Return Integrity - 2011-12' (22 April 2013), draft internal ATO document.
74 Above n 72.