Power Failure




As you’ve likely noticed, we’ve been experiencing problems with the Photosynth site for a few hours.  Around 7pm PST one of the data centers hosting our servers experienced a power failure.  By unfortunate chance we were upgrading some of our database servers at the same time which caused a cascading failure.  Our engineers are working around the clock to restore the service but in the meantime you may see missing thumbnails, our maintenance page, or other errors on the site while we troubleshoot.  Synthing will also be impacted and you may see slow uploads or synthing may not work at all.

 

We’ll post an update here when everything is in the clear.  Thanks for your patience.

 

UPDATE: 4/14 10am PST – We’ve got most systems back online now (most content on the site should be viewable, and we’re accepting uploads).  We’re still working through some details and will update this post once we’ve got a more complete status.

 

UPDATE: 4/14 12:25  PST – As many of you have reported, there are still a large number of “missing” synths that produce the message “This synth could not be loaded. The owner many have deleted it.” We are restoring these synths at the moment, and sometime tomorrow all synths created before 6pm PST on April 13th should be back online. Synths created during our power failure (7pm — 11pm PST) are currently in limbo. The may get published tomorrow, but if you want access to them now, you’ll need to re-synth them. I’ll update this thread when the restored data is back online. Thank you for bearing with us.

 

UPDATE: 4/16 19:00 PST — All the content has been restored. Thanks for your patience, and please let us know (at photosynth@live.com) if you see any further issues.

 

UPDATE: 4/22 — We had an issue unrelated to the power outage Monday night that caused a problem on our view page.  This too has been fixed, and all synths should be viewable.  Please let us know if you see any further problems.

 

 


Comments (8)

  1. Kit says:

    I thought Photosynth is hosted on Windows Azure or managed by GFS.

  2. Nate Lawrence says:

    Hi, Kit,

    If you have your web browser’s status bar enabled or are running any sort of HTML traffic monitoring program such as Wireshark or Fiddler, you can see that most Photosynth data comes from a Microsoft Labs account with Limelight Networks (the URL should look something like http://mslabs-nnn.vo.llnwd.net/name.of.file)

    Windows Azure didn’t even get into Community Tech Preview until after Photosynth had launched (that Photosynth launch date being, as I’m sure you’re aware, well after the Photosynth team would have established their hosting solutions).

  3. Hi chaps

    If you had a better forum or if you emailed members then you could have distributed this news a little more efficiently.

    G

  4. Frank says:

    I still can’t load any synths. Has the problem recurred?

  5. I’m wondering the same, not seeing any and I started loading one and its taking forever

  6. Julien says:

    Too bad, Photosynth is unavailible…

  7. Julien says:

    Still unavailible here.. too bad!

  8. kudrna78 says:

    I cant see any of my already uploaded synths and also new ones…whats the problem with it ? Its stil related to the power failture ?