MAS orders DBS and Citi to review outage

The Monetary Authority of Singapore (MAS) has issued orders to DBS Bank and Citibank, requiring them to conduct a detailed investigation into their recent service outage and the subsequent failure to fully recover their systems within the stipulated timeframe. The regulatory authority in Singapore has specified that banks’ critical systems should not be non-operational for more than four hours in any 12-month period. Furthermore, these systems are required to have robust backup data centres in place, which must be periodically tested to ensure their functionality in the event of primary data center failures.

Both DBS Bank and Citibank in Singapore had their services supported by an Equinix data center, in compliance with the regulator’s guidelines. On 14th October, a situation arose where the primary data centres of both banks experienced issues, leading them to activate their backup data centres. However, they were unable to fully restore their systems within the mandated timeframe.

In response to the disruption, the banks implemented contingency measures, including alternative arrangements for credit card transactions and extended branch hours to mitigate customer inconvenience.

The root cause of the data center outage was traced back to a third-party contractor who had mistakenly sent a signal to shut off the valves providing chilled water into the buffer tanks during a planned system upgrade. This resulted in an increase in temperature in some of the data center halls, which, in turn, affected the networking equipment that supported the backed-up data. The incident not only impacted the banks but also affected other services that were reliant on the same data center, including social media platforms such as Facebook, Instagram, and WhatsApp.

MAS is now directing both DBS and Citibank to carry out a thorough investigation into the incident. The regulator has indicated that it will take appropriate supervisory actions once it has gathered all the necessary facts. It has emphasised that no IT system is infallible and that both banks and customers should have contingency plans in place to address service disruptions caused by IT outages. This incident serves as a reminder of the importance of robust backup and recovery strategies for critical financial systems.