Checks and Audits on CHAS Subsidy Calculations

Assoc Prof Walter Theseira asked the Minister for Health in light of the IT error in the provision of CHAS subsidies (a) how frequently are checks for accuracy and reliability performed on such IT systems and how are these checks performed; (b) to what extent will the costs of incorrectly providing higher subsidy tiers be recoverable from the system vendor; (c) what are the principles determining which healthcare IT systems are outsourced; and (d) whether the outsourcing of means-testing IT systems will be reviewed so as to build key competencies within Government.

Assoc Prof Daniel Goh Pei Siong asked the Minister for Health concerning the computer system error resulting in inaccurate CHAS subsidies being given to 7,700 recipients (a) whether there was feedback by the recipients about the errors; (b) if so, how was the feedback responded to; (c) how was the computer system error discovered; and (d) what was the cost of the error to the Ministry.

Thank you. Sir, the error occurred in the computerised means-test system that calculates the healthcare subsidy tiers which individuals are eligible for, based on their income information. This central system is owned by MOH and managed by our vendor NCS Pte Ltd.

On 16 September 2018, the system was migrated to a new Government data centre, due to the Government’s planned decommissioning of the old data centre where the system was originally located. One of the files uploaded by NCS to one of the servers during the migration was of a wrong version. As a result, healthcare subsidy tiers of some individuals were computed without the corresponding full income information of these individuals.

The first run of CHAS means-tests after the system migration took place on 18 September. The first discrepancy in means-test results was identified by the CHAS processing team on 24 September, and NCS was immediately alerted. The team then subsequently identified five more cases between 9 October and 2 November, and alerted NCS on each occasion. Three of the six cases were applications that required some manual verification from the CHAS processing team, who detected the discrepancies. The other three cases were appeals from CHAS applicants. Two were assessed to be legitimate appeals for higher subsidies and were approved.

The majority of the affected individuals received higher subsidies and they do not need to return the subsidies disbursed. We estimate the amount to be about $2 million, and we are in discussions with NCS to recover the amount from them. Those who received lower subsidies will have the shortfall reimbursed, and their subsidies set to the correct levels going forward.

Dr Chia Shi-Lu and Assoc Prof Walter Theseira asked about the checks and audits on the system and means-test results.

To ensure the means-test status and healthcare subsidy levels are updated, a refresh of an individual’s data is automatically triggered every two years, or sooner should an individual requests for an updated means-test or makes an appeal. We agree with Dr Chia Shi-Lu that it is useful to inform CHAS beneficiaries of their subsidies at each deduction so that they can also verify their subsidy levels and balances are accurate and updated. We plan to look into this during future rounds of system enhancements.

As for the means-test system, the regime of checks and audits are in line with overall Government IT Policies. These cover areas including software development, test management, systems resiliency and cybersecurity. Audits are conducted regularly. Recent audits on the means-test system include one by GovTech on IT management process compliance from Jul to Oct 2018, and another initiated by MOH on cybersecurity in Aug 2018.

Where changes or enhancements are made to the system, MOH adheres to the quality assurance process stipulated for Government agencies. This would include requirements on software component testing, end-to-end system integration testing, user acceptance testing, performance test, security test and production environment shakedown tests. For some systems, a set of transactions may also be performed at launch or migration to verify accuracy before going “live”. Additional monitoring may be undertaken during the initial period of time after going “live”.

The extent of testing in each case would depend on the assessed risk. The means-test system concerned is a mature system that has been running since 2012. Based on the scope of this migration, the tests and validations were scoped and performed accordingly. Thereafter, the means-test system was migrated to the new data center. Unfortunately, NCS deployed a wrong version of a software file to one of the servers in the new data centre. In other words, the scope of test and post production validation was appropriate and in accordance to the project’s assessed risk but the wrong version of the file was thereafter used, instead of the one that had been tested. And that applied to one of the servers.

Dr Chia Shi-Lu and Assoc Prof Walter Theseira also raised questions on outsourcing against in-house development. To deliver digital solutions and deploy IT systems effectively, the Government adopts a mix of IT development approaches, including outsourcing, co-sourcing and also in-sourcing. If the solution is commercially mature, Government agencies are more likely to outsource, so as to better manage cost and leverage on the capacity, competency and also the scale available in the market. Increasingly, the Government is also co-sourcing the development of critical systems with key vendors to ensure high systems quality and reliability, as well as inter-operability across Government. Finally, full in-house development is done to develop capability which would enable us to develop software to better respond to our evolving policies and requirements.

In public healthcare, we take a similar approach. This allows us to move at the pace needed to support the volume of IT system requirements across the healthcare system, while ensuring that core capabilities continue to be developed internally.

As we tap on technological solutions to raise productivity and improve the provision of healthcare services and support to Singaporeans, we are keenly aware that system vulnerabilities and risks exist and they do have to be carefully managed. Where IT systems are outsourced to vendors, we also have to closely supervise their work, monitor their performance and manage vendor risks.

We constantly seek to improve our quality assurance frameworks and review IT processes and outsourcing. Following this incident, MOH has started a review of our testing and deployment processes to identify areas that can be further strengthened. For example, we will be implementing an independent Quality Assurance review for every major system change henceforth. We will also look into automating some parts of our testing and deployment processes, including the detection of anomalous transaction outcomes. We will continue to learn from our experiences to build a stronger and more robust system.

I thank the Senior Minister of State. Two questions. The first has to do with the scale of the error and the cost because the error did not just happen for CHAS but also affected outpatient subsidies, MediShield Life premium subsidies, subsidies for intermediate and long-term care and the disability assistance schemes. So, what is the scale of the error beyond CHAS and what is the cost.

The second question is if it affected these other subsidy schemes too, does it mean that the means-testing data is so integrated that one error in one part would affect all the other parts with regards to a particular individual, for example, who is receiving subsidies across all these different schemes.

In terms of the scale of the error and the remediation efforts that are taken, and the cost that it will take, the $2 million that I estimated would approximate all the costs that would be involved in correcting this error. In terms of the information that is affected, income information for a particular individual would remain similar. So, yes, if one aspect of the information is compromised, then it may affect the way in which that individual's subsidy tiers are calculated. The way in which we design the system is for convenience so there is a similar back end to all of these subsidy tiers, and each time a request is being made, one goes back to the base information for the particular individual.

So, to that extent, yes, Assoc Prof Goh is right. But ultimately, the integrity of the system depends on the various cross checks and the verifications that are done before the information is then put out. In this case, there was one particular file that affected the base information, which is why when computing subsidy levels, the information was incomplete and therefore, some subsidy levels were wrongly assigned.

I thank the Senior Minister of State. I have three clarifying questions. The first is that it appears from the timeline there was about a one-and-half months' lapse between discovery of the first discrepancy and the conclusion that it was a systemic error. So, is it the case that the vendor did not conclude there was a need to check for whether there was a systemic error upon being notified of the very first discrepancy?

The second question is, is it possible for the Ministry to clarify today the extent of recoverability of costs from the vendor?

And the last question is, will the decision to outsource this particular type of system which is means-testing, will that be reviewed in light of this case?

The timeline I gave earlier takes into account the amount of time for the CHAS team to pick up the error, flag it to NCS and within about three weeks or so, the error was fixed. What the vendor did not realise at that point in time was the fixing of that error meant that all cases moving forward would be correct, but it did not account for the fact that those that had been processed up to that point in time still had mismatched subsidy information. The error itself was fixed a few weeks later. So, from that time onwards, no other errors would occur. Between that time and when this was then announced, steps were taken to identify the scope of the persons who were involved, how each of them were affected – whether up or down – and the extent to which remediation would take place. And once that was determined, this was announced back in February, earlier this month.

As far as the extent of recovery is concerned, as I had mentioned, the $2 million would cover an estimate of what it would cost to remediate this problem. As Assoc Prof Theseira appreciates, if we keep the subsidy level and honour the subsidy level despite it being erroneous, there will continue to be a degree to which further subsidies that were higher than what they were supposed to be, might be granted. And so, we are looking at all of this with the vendor and working out the sum that would approximate all of this and come to an arrangement with the vendor.

On the last question by Assoc Prof Theseira, this system still remains the most convenient because it allows there to be one avenue where the information is contained and one source at which all of these would be determined when an applicant comes and makes an application for subsidy, whether it is CHAS, long-term care or for some other subsidy schemes. So, it remains the most efficient. And the best way in which this can be fixed and to ensure that this problem does not arise again, is to really strengthen and make more resilient the way in which testing is done, either at the time migration takes place or at some stage thereafter, to continue to monitor even after it goes "live", as I had mentioned earlier.

Fixing the problem at its root requires these levels of resiliency to be improved, not so much to change the system architecture altogether.