Using OMSpy to “fix” the bug with the image rotator

  • Article

  • I wanted to show the output of OMSpy in case someone wants to get very bold and use it to update the XML on a OneNote page to work around the limitations I mentioned on Monday with the image rotator.

  • Let's start with a simple image to use for testing:

  • And if I right click on the image and select "Copy Text from Image," this is the text I get:

  •  

  • ---------------

  • pride and Prejudice

  • by ane Austen

  • chapter 1

  • It is a truth universally acknowledged, that a single man in

  • possession of a good fortune, must be In want of a wife. However little known the feelings or views of such a man may

  • be on his first entering a neighbourhood, this truth is so well fixed in the minds of the surrounding families, that he is

  • considered the rightful property of some one or other of their daughters.

  •  

  • --------------

  • This is pretty good. The first "p" needs to be capitalized, and poor Jane lost her "J." The rest of the text looks accurate to me (but the fonts are a little off).

  • If I use OMSpy to look at the data on the page, I first start OMSpy and use the tree control to open the page I want to modify. Here's what that looked like to me:

  • <one:OCRData lang="en-US">

  • <one:OCRText>

  • <![CDATA[pride and Prejudice

  • by ane Austen

  • chapter 1

  • It is a truth universally acknowledged, that a single man in

  • possession of a good fortune, must be In want of a wife. However

  • little known the feelings or views of such a man may

  • be on his first entering a neighbourhood, this truth is so well

  • fixed in the minds of the surrounding families, that he is

  • considered the rightful property of some one or other of their

  • daughters.]]>

  • </one:OCRText>

  • I broke up the XML to make it a bit easier to read. The first node just tells me this is OneNote's OCR data using English (and more specifically, my system is set to use US English, as opposed to the Queen's English, for instance). The next tag specifies that the actual text OneNote found is coming. It's contained in the CDATA[ ] element found next.

  • Then a closing tag and that's it for the text. For images that are upside down

  • Next up are a whole slew of lines that look like this:

  • <one:OCRToken startPos="0" region="0" line="0" x="1.5" y="2.25" width="30.0" height="8.25" />

  • <one:OCRToken startPos="6" region="0" line="0" x="39.75" y="2.25" width="17.25000190734863" height="8.25" />

  • <one:OCRToken startPos="10" region="0" line="0" x="65.25" y="2.25" width="55.50000381469727" height="10.5" />

  • These track the (x,y) coordinates of each word OneNote. A quick example might help explain what these are for. If I search for Prejudice, I see this in the first image above:

  • The word "prejudice" is highlighted based on the (x,y) coordinates specified here. In this case, the word is the third word found in line 0, so OneNote moved over 62.25 pixels to the right, then down 2.25 pixels to draw a rectangle 55.5 pixels wide and 10.5 pixels tall. The "region" attribute corresponds to the "paragraph" where the word occurs, with "paragraph" being loosely defined since the text you are extracting may not use paragraphs as we typically expect them. The details of the XML schema can be found here. Since even this small bit of text has many words in it, the list of words and (x,y) coordinates here is quite long.

  • Anyway, since the image is going to be flipped, rotated and otherwise skewed, this data will soon be NA.

  • So to delete all the OCR data with OMSpy, I selected everything from the oneOCRData tag to its close - this includes all the OCRToken data. Then I deleted it from OMSpy. Then I clicked UpdateContent and the XML, which no longer has the OCR data for the image, was put onto the page replacing everything that had been there. Here's what the page data looked like at that point:

  • To me as a user, nothing on the page looked different after I updated the content. When I right clicked the rotated image and selected to extract the text from the image, OneNote triggered its OCR code to recreate the index for the words in the image. Had I performed a search immediately for "pride," though, nothing would have been found since I had just deleted the index.

  • Note: If you don't delete exactly the OCRData node and its closing node, you will get error number

  • HRESULT: 0x80042000

  • Which means you have invalid XML and the API won't let you update the page. Double check the XML structure and try again! In my first image above you can see I did not delete the opening XML tag for OCRData so this is the error I got when I tried to apply my changes after deleting the highlighted information.

  • Have fun and remember to back up your notebooks before modifying them!

  • Questions, comments, concerns and criticisms always welcome,

  • John