More Information on Blackberry outage…


 


Following my earlier blog post this morning – Business and trade press including Bloomberg, WNBC and PC Advisor have reported all day on the disruption caused to millions of BlackBerry users when a system failure disabled push email for eleven hours last night. According to RIM officials, all halted e mails will now be sent on in stages so as not to overload devices.


BlackBerry Service Disrupted; No Timeframe for Restoration

Gwen Ackerman, Bloomberg, April 18, 2007

http://www.bloomberg.com/apps/news?pid=20601082&sid=a_r1Lp6zBi_0&refer=canada

April 18 (Bloomberg) — Research In Motion Ltd.’s BlackBerry service experienced disruptions in North America and other parts of the world, affecting an unspecified number of the company’s 8 million subscribers. The company hasn’t provided a timeframe for when the service might be restored.

“Please be advised that we are currently experiencing a service interruption that is causing delays in sending and receiving messages,” said a recording on a customer service number in Vancouver, Canada. The company “will provide updates as soon as they become available.” Officials at the Waterloo, Ontario-based company could not be reached for comment.

The disruption may be limited to North America, with some problems in other parts of the world for customers who receive their e-mails through North American servers. Mobile-phone companies in Asia and Europe said they haven’t experienced problems. Research In Motion had about 8 million subscribers at the end of the fiscal fourth quarter ended in February.

Before today, shares of Research In Motion had gained 2.7 percent in Nasdaq Stock Market trading.

`Paralyzing’

General Motors Corp. spokesman Chris Preus, who is based in Detroit, was one of several U.S.-based participants on a press trip in China who had intermittent or blacked out BlackBerry coverage today.

The BlackBerry service is “the only way I have to communicate,” said Preuss, who was in Liuzhou, China, for a press trip of a General Motors joint venture factory. “When I lose it, it’s paralyzing.”

Blackberry users helpless after outage

John Blau, PC Advisor, April 18, 2007

http://www.pcadvisor.co.uk/news/index.cfm?newsid=9040

Research In Motion’s Blackberry service down

The BlackBerry wireless email service from Research In Motion has suffered a widespread outage starting in the US last night.

Customers on the BlackBerry Forums discussion board complained of having no service, while callers to the BlackBerry US technical support line were still greeted with the following message early Wednesday morning in the US: “We are currently experiencing a service interruption that is causing delays in sending or receiving messages. We apologise for the inconvenience and will provide updates as soon as they become available.”

New York television news channel NewsChannel4 reported on Tuesday night that the problem affected “all users in the Western hemisphere.” Postings to BlackBerry Forums indicated that the problem may be limited to North America, however.

“Officials with RIM said they are trying to reset the system and told NewsChannel4 that they are concerned that the backlog of data, which will rush through when it comes back on line, could cause a bigger problem,” the news station reported on its website.

RIM officials advised people who use Blackberry as a major way of communications to make back-up plans, the TV station reported.

A trouble ticket from T-Mobile said customers in all markets were experiencing “intermittent issues when sending and receiving e-mail from BlackBerry devices,” according to one of the forum postings. It said the problem affected all markets and all geographies.

RIM didn’t immediately return calls for comment.

NewsChannel4 Exclusive: Massive System Failure Affecting Blackberry Users

WNBC.com, April 18, 2007

http://www.wnbc.com/news/12339359/detail.html?taf=ny

NEW YORK — NewsChannel 4 learned of a massive system failure that affected all blackberry users in the western hemisphere late Tuesday.

The RIM Company, which stands for Research In Motion, developed blackberry technology and said its infrastructure failed around 8 p.m. Tuesday and was until about 7 a.m. Wenesday.

E-mails were not being pushed to portable blackberry devices.

Officials with RIM said they tried to reset the system and they were concerned that the backlog of data, which could cause a bigger problem as it rushes through now that the system appears to be online.

RIM officials said messages would be sent out in stages so the system does not crash.

RIM officials recommend all who depend on their blackberry as a major way of communication should make some back-up plans in case more problems occur Wednesday morning.

For the latest on this issue, stay with NewsChannel 4 and WNBC.com.

Now this isn’t the first time the RIM NOC has been down but prior incidents have just affected Blackberry users on particular operator networks.   From what I’ve managed to pull together. 


  • The failure covered the entire North American region, as indicated by RIM officials in media stories.  The main NOC in Waterloo appears to have been the problem.
  • The failure started about 8PM EST last night, and the outage was fixed “around 4am”.  There is still a backlog of traffic running through the NOC. 
  • RIM **does** have failover built into its NOC, so if a Waterloo outage occurs, traffic should be able to run through the UK NOC.  Obviously this didn’t happenRIM admits that obviously this feature did not work.  

RIM play very heavily on the supposed benefits of a NOC based model introducing reliability however these are obviously completely eroded when you lose a service for such a long time and impact so many people.


Comments (10)
  1. Greg Lowe says:

    So, doesn’t the average IT Professional feel comfortable knowing that a server/infrastructure that they have no control over can negatively impact their users in such a major way?

    At least with Direct Push, we can fire the bonehead that tripped on the power cord (or installed the latest patch without testing)

  2. Cuando se tiene que ofrecer servicio de correo móvil a nuestros usuarios, en estos momentos tenemos dos

  3. Fixer says:

    Wow. With all this hoo haa about feeling vulnerable with no data etc. what ever happened to the idea of using the device as a phone too! Inconvenient for the crackberry addicts but an instant backup in many cases.

    PS like the NOC NOC joke…just a bit of fun

  4. AndyB says:

    Yet again, RIM shoots itself (rather painfully IMO) in the foot.

    I just hope some of the FUD distributed about Exchange ActiveSync to promote RIM is now consigned to the trash where it belongs…

    Keep up the good work Jason.

  5. anon says:

    To NOC or not to NOC, that was the question.

    I’m sure we all know the answer now…!

  6. Fixer says:

    Wow. With all this hoo haa about feeling vulnerable with no data etc. what ever happened to the idea of using the device as a phone too! Inconvenient for the crackberry addicts but an instant backup in many cases.

    PS like the NOC NOC joke…just a bit of fun

  7. jrdepriest says:

    What’s the FUD with ActiveSync?  My phone get’s emails before my workstation does, so I’d say it works pretty darn good.

  8. Calendar, Email, Tasks, and Projects: The Exchange Dilemma Exchange Server Dirty Shutdown Error Exchange

  9. After the earlier outage this year … the Blackberry Internet Service went down on Friday according

  10. I had to smile seeing this website however I doubt RIM's PR or Legal teams shared my humour. RimOutages

Comments are closed.