Ahhh "foreach". Usage of foreach can be either a blessing or a curse. Unfortunately, a lot of developers don't understand that it can cause issues. The need to avoid it when code in a foreach will do an operation (move, delete, etc.) is something which blows the minds of even seasoned developers.
What you will see happen when an operation like a delete or move is done on the iterating item is that the position of where the underlying code is in the collection will be thrown off and the next item to be processed will be skipped. This can/will cause items to not get processed and can otherwise introduce leaks into your application. This isn't limited to working with .NET, OOM or when COM objects are used – it happens for all types of code.
To work around this, you should look at using a reverse for-each loop on the index of the item in the collection instead. Yes, sometimes the best way to do things is backwards.