Quantification vs Gut feeling

I was working on a pretty tricky issue a while ago. It was related to the Visual Studio IDE high memory consumption and slowness and I found that there are a lot of files in the Source Folder. So much so, that my gut feeling was saying that it is the source of all evil. But, troubleshooting doesn't really go well with gut feeling.You need quantification... you need proof!

Have a look at the image below (I created these junk folders just to give you an idea of how it roughly looks like). In the following folder structure, almost every folder contains a folder called _wow. The project was a big one and contained hundreds of folder and thousands of file... yeah, honestly! I just wanted to know what is the size of all the _wow folders combined. Oh, and I forgot to say that I am very lazy and instead of relying on counting them manually I thought of using a couple of tools to make my job easier.

image

It is pretty simple, and believe me this trick is really handy sometimes!

So, the first thing I did was to switch over to the cmd window and navigated to MyFolder. Then I listed out the directory with dir /s /b > MyOutput.txt which gives an output something like this...

C:\temp\MyFolder>dir /s /b
C:\temp\MyFolder\1
C:\temp\MyFolder\2
C:\temp\MyFolder\3
C:\temp\MyFolder\_wow
C:\temp\MyFolder\1\55
C:\temp\MyFolder\1\66
C:\temp\MyFolder\1\77
C:\temp\MyFolder\1\88
C:\temp\MyFolder\1\99
C:\temp\MyFolder\1\_wow
C:\temp\MyFolder\1\55\2
C:\temp\MyFolder\1\55\3
C:\temp\MyFolder\1\55\_wow
C:\temp\MyFolder\1\66\2
C:\temp\MyFolder\1\66\3
C:\temp\MyFolder\1\66\_wow
C:\temp\MyFolder\1\77\_wow
C:\temp\MyFolder\1\88\_wow
C:\temp\MyFolder\1\99\_wow
C:\temp\MyFolder\_wow\MyFile.txt

Now, open the MyOutput.txt file in MS Excel.

Select the column 1, and choose the ribbon and select Conditional Formatting -> High light cells Rules -> Text That Contains.

image

Choose a fill like the following...

image

From the ribbon, click Sort & Filter -> Custom Sort

image

Drop down the Sort on menu and select -> Cell Color
Click on No Cell Color button and select the color.
The output is something like the following...

image

Now, you can select all these colored cells, create a new text file and paste it in there. Let's call it... FindSize.txt.

I have written a small exe, which can count the size of all these files for you.

image

Last but not the least, have a look at the output of the problem which I was talking about earlier...

image

As you can see the gut feeling was not completely wrong... BUT quantity makes it a bit more palpable! Besides, as you can see the size of all the 1647 files combined is just about 25 MB. The performance problem may be due to the number and type of these files, but the high memory consumption is more likely not caused by this. We'll see later! After all, it is taking just 25 MB odd. The problem is not solved yet, but I am sure we will nail it down soon.

Download the utility here if you like!

Until next time Wave
Rahul

Share this post :

CalculateFileSizeByGivingFileNames.zip