Human Movements – The Easiest Motion Capture File Format


This is the second post of the “Reallistic Human Movements with Silverlight” series. I’ll spend a little time talking about the .bvh file format structure. I’m still working on the code and that’s why I’ll make this post short in order to get back to the mocap code. There is a link to more relevant info at the bottom.


I updated to the motion capture source code (download attachment to this blog) that contains a major fix related to joint rotation.


 


The latest sample is here: http://nokola.com/mocap  


Some people thought the sample is a saved video – it is not. The sample shows rendering of 3D motion capture data at real time on Silverlight. Try out the new “body fat” slider! If you click “Add another person” then “Show Info” you’ll see the x, y, z angles for joint rotations in realtime.


Before going on, a quick overview of one way animation studios do motion capture today:


1.      A human (e.g. actor or martial artist) is placed in a special costume for motion capture


2.      There are numerous cameras surrounding that person (e.g. 24 cameras) that take videos of the person


3.      Software processes the videos to “understand” where the person’s hands, feet, etc are and saves this information in a motion capture file


 


One such file format is .BVH (Biovision motion capture).


The bvh contains 2 sections:


1.      Skeleton data describing the hierarchy and initial pose of the human skeleton whose motion is being captured


2.      Frame data describing how the human’s joints move over time


The skeleton data looks something like this:


HIERARCHY


ROOT Hips


{


               OFFSET   0.00        0.00        0.00


               CHANNELS 6 Xposition Yposition Zposition Zrotation Xrotation Yrotation


               JOINT Chest


               {


                               OFFSET   0.00       5.21       0.00


                               CHANNELS 3 Zrotation Xrotation Yrotation


                               JOINT Neck


                               {


                                              OFFSET   0.00       18.65     0.00


                                              CHANNELS 3 Zrotation Xrotation Yrotation


                                              JOINT Head


                                              {


                                                             OFFSET   0.00       5.45       0.00


                                                             CHANNELS 3 Zrotation Xrotation Yrotation


                                                             End Site


                                                             {


                                                                            OFFSET   0.00       3.87        0.00


                                                             }


                                              }


                               }



For each joint, we have its 3D offset relative to its parent joint (Hips is the parent of Chest) and its channel information.


You can read more about this here: http://www.cs.wisc.edu/graphics/Courses/cs-838-1999/Jeff/BVH.html


One thing missing from the above page is that it is very important to note the order of the channels –this is the order in which you have to rotate the joints. For example order of “Zrotation Xrotation Yrotation” means that you have to rotate by z,x,y in that order.


The bug that I fixed in my code was related to that. If you don’t follow the order in the file, you’ll end up with weirdly moving figures or jitter in the playback.


The second part of the .bvh file is the frame information – I encourage you to read about it here: http://www.cs.wisc.edu/graphics/Courses/cs-838-1999/Jeff/BVH.html


This resource explains it pretty well!


 


More Info 


One person also asked me “how do I capture my own motion data?” Usually this is pretty costly (e.g. $10,000-$20,000), but could be made with lower cost too!

I’ll talk a little bit about this in the next post. In the meantime you can search for “motion capture single camera” or “markerless motion capture” on the internet. This seems to be the cheaper way to do it (cheaper as in $50-$500). Recent scientific developments (last few months) in markerless motion capture really open the possibility for the public to use this technology!

MocapView.zip

Comments (13)

  1. Nikola, previously on showing his Shock game , is back with a new interesting project, a Motion Capture

  2. Scott says:

    I love the silverlight implementation of a .bvh player.  Congratulations.

    I’m a programmer who golfs and have been using silverlight since it came out.  It would be interesting to try and figure out the body angles for golf swing analysis in Silverlight!

    I downloaded your sample to learn  more about the subject in my "spare" time.  Any insight on using it for golf would be appreciated!

    Thank you,

    Scott

    scott.dugan@hotmail.com

  3. nikola says:

    Hi Scott,

    Thanks for the comment and I’m glad you like it.

    golf is easy: check out the motion library here: http://sites.google.com/a/cgspeed.com/cgspeed/motion-capture/3dsmax-friendly-release-of-cmu-motion-database

    subject #63 and #64 are golfers. Click the "full motion list" on the above site to see the motion description.

    I got the motions for my demo from the same library. You can download the .bvh files from there and use them directly in the mocap viewer on http://nokola.com/mocap.html

  4. Mike says:

    This is an AWESOME demo!!!

    Do you have plans to make the BVH reader support more 3D programs?  I noticed that the current code expects the root object to be the HIPS.  Some applications like Poser make WORLD_ROOT the root object.

  5. nikola says:

    Hi Mike, Thank you for the support!

    I think the source should load Poser bvh files fine, although haven’t tried it. It’s not looking for HIPS in particular and any root should do (the hips was just for the example).

    I have some plans for the reader (making a game with reallistic movements), but want to finish my space game first 🙂

    Are you interested in making something with this Silverlght mocap viewer?

    Thanks,

  6. Mike says:

    Yes.  I was very excited about making an action game when I saw your post.  It looks like your application has tremendous potential.  I have been looking for a way to do something like this for a while.

    I tried importing a bvh file of a simple walk cycle that I created using Poser 6.  But, it gave me the following runtime error.

    “Unsupported number of channels for ROOT element: 3”.  

    I was guessing that it was caused by Poser giving me a different root element.  But, I wasn’t sure.  I can e-mail you the BVH file if you would like to see it.

    Thanks,

    Mike

  7. nikola says:

    ah, yes, I know about this…Loader.cs expects 6 channel format for the root:

    CHANNELS 6 Xposition Yposition Zposition Zrotation Xrotation Yrotation

    I just did it 6-channel, since it was faster to get the program up and running.

    The Poser file you have probably has only rotation, not movement info (in which case it can be ignored).

    After I get my game close to done, I’ll probably go and fix it up – until then, if you’re interested in loading Poser files, you can add a quick hack to Loader.cs: bool has6Channels = true, then change the source                 if (tokens[tokenIndex] != "6") {

                       AddErrorMessage("Unsupported number of channels for ROOT element: " + tokens[tokenIndex]);

                       return false;

                   }

    to also check for 3 channels. and update the:

               if (nodeType == NodeType.Root) {

    to work for 3 as well as 6 channels (just assume the translation channels are 0.0 if they are not present)

  8. Mike says:

    Thanks.  I’ll work on making these changes.

    Keep up the good work!!!

  9. nikola says:

    Thanks!!! 🙂

  10. Prav says:

    Hi,

    Can you point me to some nice tutorials or articles on

    1)How I can obtain joint angles from a mocap file(asf/bvh whichever is simpler)?

    2)Capturing important keyframes from the mocap file.

    Are there libraries in matlab which i can utilise to extract the keyframes?

    I am not from a CS background and so, i dont know anything about computer graphics.I am a masters student in robotics and am attempting to use some techniques to fit the mocap data to a humanoid robot.

    Many thanks,

    Prav

  11. nikola says:

    Hi Prav,

    I found this for Matlab: http://www.cs.man.ac.uk/~neill/mocap/

    Not sure how good is it, since I don’t have Matlab to try it with.

    The .bvh file format is the simplest in my opinion. It’s just a text file that can be opened and looked at with notepad. Each "frame" is on a single line and consists of all joint angles in the skeleton.

    As far as I know the .bvh file does not store "important keyframe information". If you want to transition between motions though (e.g. from "kick" to "punch"), you can interpolate the angles between any frame of the "kick" motion and the "punch" motion. The good thing is that the motion capture database here:      http://sites.google.com/a/cgspeed.com/cgspeed/motion-capture/3dsmax-friendly-release-of-cmu-motion-database

    uses the same skeletons for all files, so interpolating between frames would be easy.

    btw, controlling robot with motion capture is pretty cool! if you have something publicly share-able, please let me know!

  12. nikola says:

    forgot something – in the motion capture source code http://blogs.msdn.com/nikola/attachment/9655943.ashx the "loading" is separated from the view (graphics). If you translate Loader.cs, Motion.cs and Node.cs into Matlab, you’ll have the loader.