Background processing in ASP.NET


As part of my bicycle climbs website, I need to spend some time calling a web service to fetch the elevation of 250 different points. Each call takes a few seconds.


Ideally, what I would have is a way to start the processing but not have it block my normal page processing operation.


Ideas? I looked at the MSDN docs here, but that requires me to continually refresh the page. I could do that if necessary if there’s no other way.


Comments (11)

  1. Josh Twist says:

    AJAX?

    This sounds like the perfect use for it.

    You could even use MS’s Atlas framework.

  2. Dan McKinley says:

    I’m not sure I completely understand your problem, but there’s no reason you couldn’t start a worker thread for this purpose. If it’s a one-time event you could stick it in Application_Start.

  3. Jim K says:

    If you want users to be able to view and use a page while the elevation data is being loaded, the best approach is AJAX and if possible MS Atlas Framework.

  4. Chris Lundie says:

    Another technique is to use a hidden iframe that loads the "slow" stuff, and it can use a line of javascript to update the parent page when it’s ready.

  5. Jeff Parker says:

    http://msdn.microsoft.com/msdnmag/issues/05/01/ASPNETPerformance/

    I hope this helps you out Eric, this is an older article but Tip Number 6 how to do Background Processing in asp.net I think should fit the bill.

  6. dhchait says:

    The way I’ve done this for big, mission-critical production systems (which may be different than what you need) is to write, via some persistent mechanism (ie write a records in a database table, post an MSMQ message, etc), the data about the longrunning work to be done. In your example, you’d write 250 records, each with say an X,Y coordinate, a status of pending, and an elevation of NULL.

    Then you have a separate .exe which runs (as a service) and periodically polls for new records

    (something like "SELECT * FROM workItem WHERE Status="Pending"), and processes that work, ten writes the value in the record and sets status to "Complete". Obviously you put in logic for other status such as error, timed out, etc. as needed.

    Your asp is then very simple – user sumbits request, they are redirected to a status page, which can either be some simple interstitial page with a meta-refresh (or fancy javascript0, or to a boring "pending jobs" page which just shows the table of pending jobs. To get super fancy, you can even let them do things to those jobs, like say cancel or pause them, move them up/down in priority, resumbit them, etc.

    Other benefits:

    Since the website code is so loosely coupled to the "doing work" code, you can for instance, shut down the service for a bit and not affect the users – their jobs will just stay pending for a little longer. Manageable!

    You can easily unit test the "doing work" bit without having to putz around in your web code. Testable!

    You can add workers trivially – just set up more boxes and point them to the "work item" database. Scalable!

  7. Lee says:

    How often do the elevations change? 🙂

    Why don’t you parse the data into a format most useful for your application, and hard code it? You probably want to write a tool to converts the real underlying data to your special format, but there is no need for the site to be truly dynamic.

    That’s the approach I took for my Commute Times survey: http://www.leeholmes.com/projects/commute

  8. ericgu says:

    Lee,

    The user draws a path on the map, and I fetch the elevations from the USGS web service based on that path.

    I’d looked at getting my own copy of the elevation data. I might be able to do it if I was only concerned about Washington, but at the level of resolution I want, the Puget Sound area by itself is well over a Gig of data, and I’d like to cover the whole US, so it just isn’t practical.

  9. Ricky Dhatt says:

    Eric – I’d be interested to know what approach you finally ended up implementing.