How do I compress files (via NTFS compression) from the command line?


A customer wanted to know whether there was a way to compress files and directories (in the sense of NTFS compression) from a script or from the command line. They knew about the Properties dialog, but they wanted something scriptable.

The command-line tool for this is COMPACT.EXE. Type compact /? for usage information.

The customer liaison was grateful for this information.

Thanks for the prompt response, and yes, this will meet our customer’s need to compress specific files such as *.docx under a particular directory and all its subdirectories.

Um, *.docx files are already compressed. Compressing them again gains you nothing.

Bonus reading: Functions for manipulating documents which follow the Open Package Conventions are available in both managed and unmanaged versions. Check out the Packaging Team Blog for more information, including a comparison of the managed and unmanaged versions.

Comments (33)
  1. Adam Rosenfield says:

    Yep, .docx files are just zip files in disguise.  It always saddens me when I see compressed archives full of other compressed files like JPEGs, MP3s, other zip files, etc.

  2. Crescens2k says:

    Unless they are in there with the store compression setting. Then you know they are using the format to distribute the files in one archive.

  3. SimonRev says:

    @Adam,

    Yes and no.  Certainly you don't gain anything (size wise) by trying to compress a bunch of jpegs, people may decide to put them in an archive like a zip file for the benefit of having them all in a single file.

    Now you can, of course, tell your compression program not to actually compress the files, but unless you are making a huge archive, the time savings of not compressing doesn't offset the extra time needed to actually figure out how to tell your compression software not to bother compressing the files.

  4. ERock says:

    Well, it may not do anything for disk space savings, but it makes them blue in the Explorer window, which means it'll just make the users happy thinking magic was made. I recall delighting a user group with a pdf writer that was installed so they could use Word to print to PDF, and then use Adobe to print the very same PDF.

    Some people like the crusts cut off their sandwiches, after all.

  5. DrkMatter says:

    @Adam Rosenfield

    On the other hand, can't you possibly make additional size gains if very similar documents are pooled in the same archive? I know very little about zip compression, so I do not know if compression occurs on a per file basis or on the whole archive.

  6. RichB says:

    Um, *.docx files are already compressed. Compressing them again gains you nothing.

    In this particular case yes. But as a nitpicker, the statement is not generally true. If you use a compression method which uses solid compression – such as CAB files or zipped tarballs – then compressing again may in fact help.

    en.wikipedia.org/…/Solid_compression

    It's slightly ironic that Microsoft implemented docx files with .zip compression – a non solid compression technology versus CAB compression which has been in Windows for….oh….16 years.

  7. Cesar says:

    @Crescens2k, @SimonRev: Most compression programs can detect compressing will gain nothing and automatically store the file uncompressed without you having to do anything.

  8. Cesar says:

    @RichB: Solid compression means you have to decompress everything when you are only interested in a single file. You might gain compression but lose performance. The old zip format has a good tradeoff between compression and performance, with the bonus that it is a de facto standard everyone understands.

  9. RichB says:

    @Cesar Yea, I knew that.

  10. voo says:

    Hard to make any general statements about compression – after all there are not only dozens of different algorithms, but also lots of different settings. Especially compressing small files into one large archive with a solid block size can reduce size quite nicely even for compressed files.

    IE: ~200jpegs compressed with 7z on some higher settings reduces the filesize from 90mb to 71mb. Is a more than 20% compression for files that are backed up or used rarely that bad? Hardly – and for backups always handy, you get a checksum to see if something goes wrong. So being extremely sad about someone compressing jpegs has no real basis..

  11. voo says:

    @tsrblke I assume (purely speculation!) why gmail can't use folders is, that it would basically allow them (and therefore all websites) to traverse the whole filesystem of a user (ie they get the folder and to add everything in there they have to traverse it). While having to compress/archive everything is a bit annoying, I still think that's preferable to allowing websites access to the filesystem..

  12. ALS says:

    $5 says the user compressed the files, saw little or no space savings, and first complained about compress/ntfs compression not working well.

  13. Nathan Samson says:

    @tsrblke

    As voo pointed out, this is an issue with browser API's / security concerns. Flash could maybe help here, but that is not a solution everywhere for everyone.

    But actually attaching (multiple) files in gmail is quite easy nowadays

    Step 1: Install a decent browser (Firefox 4/5 or Chrome will do, maybe even opera and IE8, but I'm not sure and in IE's case I highly doubt it).

    Step 2: Open windows explorer (or in my case nautilus, the (Linux) Gnome File Browser)

    Step 3: Select your files you want to attach

    Step 4: Drag them to the attachment "box" (A big dropzone will activate once you hover over it — including subject line)

    Step 5: See your files attached all at once, and being uploaded in the background

    One single addition would be worthwile though: Having the multiple="multiple" attribute on the file input widget, so you can select multiple files at once from within the browser instead of only with drag and drop. But this is even less supported in browsers for now AFAIK.

    As fas a I know other (web)mail programs (hotmail, outlook web app, yahoo, …) are even less advanced

  14. Troll says:

    Why is the Compression GUI so un-evolved and basic/dumbed down? It's there since NT 3.51 and so many years later, it's still a checkbox? We had "Compress old files" option in Disk Cleanup but that was also removed in Vista because it slowed the Disk Cleanup utility even before it showed its options. There is an excellent open source tool called NTFS Compressor (sourceforge.net/…/download) for advanced options. Something like this should be integrated into the compression GUI instead of just a checkbox. It has configurable options for file types to exclude, compress by other attributes, folder paths to exclude, compress by size or % save disk space, skip recompressing already NTFS compressed files. Even compact.exe has plenty of switches. Windows 8 or 9 maybe?

    [No matter where you put an advanced setting, somebody will tell you that you are an idiot. -Raymond]
  15. Troll says:

    "No matter where you put an advanced setting, somebody will tell you that you are an idiot."

    This would mean there should be no improvement after several versions to the GUI? Probably, that's why the issue of not being able to set security permissions on multiple files in Vista/7 is "by design" and not being fixed.

    [That's a great idea. It's on the list. (Actually, this probably wouldn't make the list, since the target audience for this feature would be people comfortable with command line tools in the first place. Like the IT Administrators who update multiple ACLs with icacls.) And the multiple-file permission issue is not by design out of laziness. It's by design because of an actual design issue. -Raymond]
  16. Troll says:

    The multiple-file permission can be fixed by using this dialog for UAC: img511.imageshack.us/…/alternateelevationdialo.png (it's already used for all other attribute related operations) instead of the regular UAC dialog.

  17. Joshua says:

    I would tell Troll to shut up but this time Raymond introduced the topic.

    Either way, just turn UAC off instead of complain about it.

  18. Joe says:

    The real question is where is the command line utility to force the preview image database to be rebuilt in entirety?

  19. Brian says:

    The Packaging Team Blog. 5 posts from 2 years ago. Now there's a stable technology.

  20. Troll says:

    I would tell Joshua to shut up for topics where he doesn't know what we are discussing. Turning off UAC doesn't bring back a removed feature.

  21. Rangoric says:

    @Troll

    That feature was removed for a reason. You didn't even read the linked post which goes over this very issue.

  22. voo says:

    I'm not exactly sure how we even ended up discussing this topic. But to recapitulate: People are complaining because they have to use a command line tool to change ACLs of several files without changing everything in a whole folder?

    So people who really should be fine with the CLI (yeah I'm pretty sure my mom has no idea what ACLs are – and why would she?) doing a rare corner case of something that isn't done that often to begin with are complaining that it may take half a minute longer to do so?

    Well I'm sure after MS has implemented all other less obscure features that profit a larger user base in Windows 30, they'll look at that again xX

  23. James Schend says:

    Guys, his NAME is TROLL. Do you think he might be trolling you? (It doesn't help that Raymond got trolled too.) Sheesh, this is Internet Forum 101 stuff here.

    [I try not to discriminate based on a person's name. Maybe I should. -Raymond]
  24. Cesar says:

    > Um, *.docx files are already compressed. Compressing them again gains you nothing.

    Do not be so sure. AFAIK, *.docx are compressed using deflate, the same as gzip. Try the following on a Unix system:

    $ dd if=/dev/zero of=testfile bs=1024 count=1024

    $ ls -l testfile

    -rw-r–r– 1 cesar cesar 1048576 2011-06-22 12:11 testfile

    $ gzip -9v testfile

    testfile: 99.9% — replaced with testfile.gz

    $ ls -l testfile.gz

    -rw-r–r– 1 cesar cesar 1060 2011-06-22 12:11 testfile.gz

    $ mv testfile.gz testfile2

    $ gzip -9v testfile2

    testfile2: 95.6% — replaced with testfile2.gz

    $ ls -l testfile2.gz

    -rw-r–r– 1 cesar cesar 83 2011-06-22 12:11 testfile2.gz

    This creates a 1 megabyte file of zeros, compresses it with gzip to 1060 bytes, and compresses it *again* with gzip to 83 bytes.

    While with real files the effects are not as dramatic as with a zero-filled file, they do exist (and I have seen it happen). This is because deflate is limited in the amount of redundancy it can remove – in particular, it has a very small window size (around 32 KiB). If you have enough redundancy in your file, the deflate output itself will be repetitive and can be compressed again, even by the same algorithm.

    (In case some are still doubting this works in real files, I tried with a real file. I randomly chose /lib/libc-2.11.1.so on this machine which has 1572232 bytes. The first gzip pass took it down to 702418 bytes, and the second gzip pass took it down to 700796 bytes.)

    [News flash: NTFS compression is not the same as gzip compression. (Think: Random access.) -Raymond]
  25. tsrblke says:

    Yeah,I understand that it gains you no size space, but honestly neither would compressing .doc files unless you had thousands of them, relative to average drive space (even 1 year ago when this was written!).  But I do parrot the others here, what it gains is reduction to 1 file, which may be what they wanted.  If I have 20-30 of these and I need to send them to someone via email, it's a heck of a lot easier to send them ZIPed then it is to add each of them to the document.  I consider that a flaw of most email program though (I'm staring at you Gmail in particular) why can't I "attach folder?"  But I digress.

    [The customer was not looking for file consolidation either. They had some *.docx files and wanted to apply NTFS compression to them. Even though applying NTFS compression to *.docx files is basically useless. -Raymond]
  26. AK says:

    Roll is a real surname. His first initial may be "T".

  27. Joshua says:

    [I try not to discriminate based on a person's name. Maybe I should. -Raymond]

    The guy admitted he chose the name deliberately a few tens of blog entries back.

  28. Neil says:

    Novell NetWare used a different compression system (whole-file compression, decompress on use, throw away compressed or uncompressed version depending on heuristics). There were a number of relevant attributes, "please compress now", "file is currently compressed", "compression did not reduce file allocation size" (or "can't compress" for short).

  29. Troll says:

    @Rangoric, I have read that post a million times. You didn't read the comments in that post or mine here at all which apply to that post as well or else you wouldn't have said that. Raymond's (or Microsoft's) argument the UAC dialog NEEDS to display the file names on which you are operating is just a silly excuse. No file operations on multiple files done thru properties need to display file names all at once. Either they can show the file name one by one as operating on each file like the Attributes/copying dialogs do or they don't show any name at all (neither did Windows 2000 or XP showed the file names of files whose permissions you are setting). The alternative security dialog used when applying attributes like Compressed or Encrypted can very well be used when applying security permissions. Problem fixed. Or they can elevate the entire property sheet like certain device manager pages can. Either ways, there are a dozen ways to fix the problem but MS doesn't care enough because it's already been 2 releases post XP.

    And don't assume just because my username is troll, I am trolling.

  30. tsrblke says:

    @Raymond,

    Well then my assumptions were entirely unwarrented then.  That will likely teach me…who am I kidding it will teach me nothing. Compression is complex and I don't really fault the end user for not getting it.  Plus if you compress enough things you'll likely gain 1KB space difference in a rounding error at some point I imagine, which the user thinks is compression!

    Space is so cheap I often wonder why we bother anymore with it. I remember having a conversation with my wife who was using a commandline to compress stuff for work.  It was a bunch of text files (which do compress well) but I remember asking her "Why? They're like 100kbps each, even if you produce 5/day it'll take you tens of years to fill up a standard (500 gig) harddrive assuming you never purge anything."  Her response "They told me to!"

  31. Bart says:

    Maybe they're just used to the years of MS claims that Office files were "highly compressed" when in reality they were not. So when .docx came along, with yet another claim that the documents would be "as small as they possible could be", the customer did what they were getting used to – ignore that piece of marketing blurb.

    As for being sad about packing, say, 200 jpg files in one zip file–I'm sure that whoever wrote that never had to save those 200 jpegs as single attachments, one by one. I prefer saving a single zip file and then unpacking it. But that's just me.

  32. David Walker says:

    I learned the command line to uncompress all files from a drive.  On older, slower computers, a few years back, when the disk space had grown faster than the CPU and memory had, it seemed like a logical thing to do.

    My Dad's computer has a 500 GB data drive (and a 500 GB backup drive) but even though the disks contain many of the scientific papers that he wrote or is working on, many stock market data files, and lots of family pictures, he is only using 15 GB.  I told him that he would have to do better than that…

  33. David Walker says:

    I know that writing blogs can be unrewarding, but the packaging team blog is a very poor example.  5 entries from 2009, and that's the extent of the blog.  That is more than I have blogged, but… even with the best intentions…

    Kudos to Raymond for doing such a good job!

Comments are closed.