It took a few days but RIM finally provided some details on the outage that knocked out its BlackBerry email service from Tuesday evening to Wednesday mid-morning. The company ruled out causes like security and capacity issues, hardware failure and core software bugs. Instead, the outage was caused by “the introduction of a new, non-critical system routine” designed to improve cache performance. When RIM realized that they could not fix the problem, they initiated a rollback that also ran into problems, causing further delays.
RIM has already indicated that it will improve its testing and recovery processes to avoid other outages.