SEAMonster: A .NET-Based Seam Carving Implementation


To avoid any potential confusion, this is a personal, spare-time project that has nothing to do with the cool Seadragon technology.


Before you read my post, you should watch the short 4 1/2 minute video that demonstrates an image resizing technique called seam carving. The technique was presented at this year’s SIGGRAPH 2007 conference by Shai Avidan and Ariel Shamir. If you want to know all of the details, check out the paper they presented called Seam Carving for Content-Aware Image Resizing (20MB PDF). Shai and Ariel outline a relatively simple algorithm for finding a “seam” of pixels that is least likely to be missed when it’s removed from an image. The algorithm looks for connected pixels of low energy, where energy refers to a measure of visual detail. By iterating the algorithm, an image can be resized while maintaining its general structure. Unlike a typical stretch operation in your favorite graphics application, the seam carving technique resists squashing or distorting the image.


When I first saw their video, I thought it was magic. Not only do they demonstrate resizing, but they also show how their algorithm can easily remove specific content (like a person or object) from a scene. It’s easy to imagine how images can be resized using this technique for display on smaller devices like a cell phone. In their paper, Shai and Ariel briefly describe a method that would allow an image format with extended seam carving data to easily resize on-the-fly. Very cool.


Inspired by their work, I thought I’d play around with the algorithm and methods in my spare time. So, over the course of the past week or so, I’ve put together a .NET-based implementation of seam carving that I affectionately call SEAMonster. The code has evolved from a very simple prototype to something a bit more robust. As these things go, the architecture is a bit shaky at the moment, but it does work. The early version of the application can only open JPG files, doesn’t support image expansion, and the user interface is pretty minimal. However, it’s far enough along that someone might find it interesting.


If you’d like to take it for a spin, download SEAMonster_0.1.zip (63KB). The application requires the Microsoft .NET Framework Version 2.0 if you don’t already have it installed on your machine. In lieu of documentation, I encourage you to watch the 8 minute video introduction I recorded to quickly get up-to-speed (note that you may need to download the TechSmith Screen Capture Codec to properly view this video).


When I have the code under control, I plan to upload the project to CodePlex so that the developer community can contribute. All of the image manipulation functionality is encapsulated in a DLL, so it should be usable in most scenarios. There is a lot of opportunity for improvement and optimization in these algorithms, and I expect a lot of innovation to happen over the next few months. It should be fun to watch.


Here are some other seam carving resources that you may find interesting:



If you have any comments about this early version of SEAMonster, or if you have suggestions for the version I plan to release on CodePlex, please leave feedback (or contact me directly).

Comments (22)

  1. Vous avez peut être vu cette présentation d’une technique impressionnante de redimensionnement d’images

  2. Brian Brewder says:

    I just posted this on the Paint.Net forums. Do you have any plans of creating a plugin for PDN?

  3. mswanson says:

    Brian…I don’t have any plans to create a plug-in. However, once I’ve uploaded the SEAMonster code to CodePlex, it shouldn’t be too much work for someone to do this. The logic is implemented as a library which should make it relatively easy to adapt to other uses.

  4. Kevin says:

    Wow, you weren’t kidding about it being CPU intensive. I have a 2.5 meg image and it’s taking a long time to do operations on a quad top of line Intel cpu.  Is the algorithm something that can be spread out across CPUs?  And also, are there plans to put your app on codeplex as well (with source)? Using larger images than the screen do not work very well since there are no scrollbars :(  

    I am completely amazed however, Great work!! I’ve never seen this before and is REALLY interesting! I can’t wait till the library lets you stretch too :>

  5. mswanson says:

    Kevin…yes, it is very CPU intensive. This is the same story for every implementation I’ve seen. I’m sure the algorithm will see improvements over time. Also, as you mention, the algorithm is very well suited to multi-core/CPUs, and performance should increase dramatically when that functionality is added.

    Related to CodePlex, yes, as mentioned in the post, I do plan to upload it so that people much smarter than me can make contributions. :)

  6. Nice job.  Great to see this in .NET.  I think the Seam Carving just might be the killer app to drive home those dual and multi-core CPUs when photoshop or design can do this.  C’mon don’t tell me you aren’t going to upgrade all your PCs then when your friends are Seam Carving with impunity.

    Doing this for web would be nice as well, imagine all those sites with skewed imagery and distorted aspect ratios self correcting themselves stylistically with Seam Carving.  A man can dream….

  7. Pressacco says:

    Very interesting concept.  I look forward to trying out your source code.

  8. Brian Shapiro says:

    This is neat, and is good for photo manipulation. I kind of doubt the use as suggested in the opening of the video though, that we supposedly need the ability of images to resize like this inside of browser windows.

  9. Brain_ReCall says:

    Hi Mike! Nice GUI. I was considering trying a .NET implementation, but ran into threading issues that I couldn’t easily resolve, so I kind of gave up on that :)

    A bunch of work later and the algorithm is getting fast. Down to about 15 seconds for 1024×678 down to 800×600. Unfortunetly, I don’t see many more drastic performance increases.

  10. mswanson says:

    Thanks for the compliment, Brain. It’s really more of a "fancy test harness" than anything else. :) 15 seconds for that resize is great. It’s definitely an expensive algorithm if you want the best fidelity.

  11. Mark Van Dyke says:

    Awesome! I saw that original demo and was amazed. Cool to see it in .NET. Can’t wait to try it out.

  12. Sometimes you read people's blogs and you get to know their hobbies and roughly what filed of work

  13. Ian Hickman says:

    I am amazed by this technique, and how quickly there are a million different implementations of it!

  14. Ross Snowden says:

    Would love to see this feature somehow integrated into Live Gallery.

    I’ve used the flash based rsizr (or whatever you call it) and it seemed to work in much the same way.

  15. Maybe the algorithm is the perfect opportunity to work with the PFX framework.

  16. Dan Luong says:

    Hi thanks for providing helpful resources about seam carving. I was able to implement my own version of the algorithm in MATLAB with a GUI and complete documentation so that it may be helpful for people trying to understand the algorithm. My code and tutorial can be downloaded at:

    http://danluong.com/2007/12/21/seam-carving-matlab-implementation-tutorial/

  17. pbz says:

    Wouldn’t this be nice if we could implement it in Silverlight since it’s so CPU intensive?

  18. mswanson says:

    @pbz: I had originally wanted to implement this using Silverlight, but there is no way to directly manipulate the individual pixels of a bitmap image. Since the algorithm depends on that feature, I had to use another method.

  19. Maybe this should go to my photo blog, but since it has an interesting software development aspect, let’s…

  20. Hi there Mike, I’m very interested on this Seam Carving’s Algorithm.

    I’m currently doing a research about this, and this research is for my study on IT.

    But I still don’t get the big picture on how this “Seam Carving” technique could be implemented into something big ?

    Please email me @ aditpnyu[at]gmail[dot]com if you have some advised on this.

    I would appreciate your help

    Thanks,

    Aditya Perdana