How embedded files now work in OneNote B2TR (and will in RTM)

  • Article

I saw this on the OneNote public newsgroup and I wanted to share with all of you, it was from David Rasmussen:

How embedded files now work in OneNote B2TR (and will in RTM)

I see there have been some interesting threads on embedded files, and apologize for my absence in responding to them. Rather than replying on what became a very deep thread, I thought I'd start a new one with a clear explanation of how things work.

1.    As of B2TR "inserted files" (aka embedded files) are actually stored within the .one section files. That is, when you insert a file on a page in OneNote, it gets stored in the relevant section file in your notebook folders. You no longer will see "thicket" folders (xx_onfiles) alongside the seciton file, they are not used anymore.

2.    We made this change for a number of reasons, but mainly because lots of other replication systems underneath us (e.g. Windows Offline Files caching, Folder Share, Groove, other sync tools you might use) could do some pretty nasty things to us when they are moving/copying/renaming the section files separately from the embedded files in the folder. That might even result in loss of those files for the user. That would be bad, we take data integrity very seriously so we decided to be more robust when co-existing with these tools. Second, users could get themselves in a mess (e.g. move the .one section files but not the thicket folder with it etc.). That would also be bad, could result in data loss etc.

3.    There is an exception to the above change. When we are syncing to a notebook on SharePoint we still leave the embedded files in separate "thicket" folders alongside the section file. This is for performance reasons because we have to send whole files on SharePoint when making updates, we can't just selectively update some bits within the file. So big files are expensive. Also, the sync/replication systems mentioned above (Windows Offline Files, Folder Share, Groove etc.) don't generally operate on SharePoint document libraries, so that problem doesn't exist there.

4.    So the definitive repository of your embedded files is in the .one section files along with your notebooks. Just like all notebook data, we do replicate them down to our cache for a whole bunch of reasons discussed in earlier threads (performance, offline availability, etc.). In our cache, for performance reasons we store them as separate files still (the way we store things in our cache vs user files is completely separable and we have different performance / architecture considerations for each). This in no way means that this is the only or definitive place where these files are stored (i.e. it's just a cached copy for our operational needs).

5.    Our cache is a "garbage collected" store. In simple terms, this means that when you delete something or close a notebook, it won't disappear from there immediately. When we've accumulated a certain % of no longer in use stuff, and the machine is not busy we come along later and clean things up, remove things that are no longer referenced etc. You can always force this operation at anytime by choosing the "Optimize All Files Now" button in the Tools->Options->Save tab. You will note when you do this that it can take some time, that's why we don't do these operations immediately everytime you delete/close something. The garbage collected store results in much better performance for the user.

6.    I STRONGLY recommend you ignore the cache folder in the system "Local Settings" directory and certainly don't go messing with it and messing around with files in there. That's probably not going to lead to a happy result for you. If your notebooks are fully in sync and you're a super power user, and you want to rebuild the cache, yes that can be done (basically delete ALL the files there including the .onecache file itself, I wouldn't mess with them individually). This may have been helpful in earlier beta builds when we still had sync issues. At B2TR and RTM though I really can't think of a good reason at all to recommend you do this.

7.    One very unfortunate limitation of what we could with embedded files in this version is that our search will not produce hits within the embedded files. I won't go into all the technical details and issues dealing with indexing engines etc., but I will say this pains us greatly, it's somethign we really wanted to do, we thought long and hard about it, but we just couldn't do that work in this release. It's a lot of work to do right and we focused most of our search energy on just getting fast indexed search for OneNote content right.

8.    Audio and video recordings made with OneNote are treated just like other embedded files. All the same issues, challenges etc. apply so we handle them in the same way.

Thanks David!