WofCompressed streams in Windows 10

On windows 10, there is a new 'System Compression' option that compresses files using reparse points. This is not the NTFS-based compression that earlier versions of windows utilized, its different. This post is about the new compression scheme and how it affects forensic analysts.

With windows 10, a lot of details are automatically managed without user input and this is one of them. Windows can determine if the compression will be beneficial to the host system and automatically trigger it! This usually happens when you upgrade as opposed to clean installing the OS. Some users have reported seeing it as an option in 'Disk Cleanup' too.

Windows provides a utility called Compact.exe to do this processing manually. Using it, you can compress/decompress files and folders or simply query a system to determine if it will be beneficial at all on a specific volume. The compression algorithms are XPRESS (4K, 8K, 16K) or LZX. While the files are compressed on disk, if an application opens/reads such a file, it is still getting the original decompressed data and all decompression is handled on the fly automatically by windows 10.

Figure 1 - Compact.exe and its command usage info
The command 'compact /exe <file>' will compress any file (not just exe)

Lets get to the point, how does this impact forensics? 

Well, as of now, no tools will recognize and decompress these files. Hence, you can't read, keyword search or extract these files in their original uncompressed form.

Tools tested

Here is a list of tools tested so far:

Tool	Version	Support (as of 10/26/2016)
SIFT Workstation	3	No
Autopsy	4.2.0	No
FTK	6.0.1.30	No
Xways Forensic	19.0	No
Encase	8.01	No

How it works?

System compression utilizes reparse points and creates a new Alternate Data Stream (ADS) having the name 'WofCompressedData'. The compressed data is stored here. Reparse points are an NTFS feature that allow custom implementation like this. However this means that other applications that are not aware of this custom implementation will not be able to read/write to that file. In encase (or other forensic tools), you can see the file and the WofCompressedData stream. Clicking on the file just shows the contents to be all zeroes. Clicking on the stream, you can get the compressed data, but as of now, no automatic transparent decompression (as it does with NTFS compressed files). This is seen in screenshot below.

Note - This isn't to be confused with WOFF compression, which is a compression scheme used in Web Open Font Format!

Figure 2 - Encase shows the WofCompressedData stream. The file's original data was all text.

If you mount a volume containing such compressed files in SIFT Workstation or any linux system (they all use the same NTFS-3g FUSE driver), you will see the message 'Unsupported reparse point' when trying to list these files. Trying to access file contents will result in errors as seen in screenshot below.

Figure 3 - Files DW20.exe and upgrader_default.log are compressed here

If you attach a windows 10 formatted volume/disk to a Windows 7 system, you won't be able to access files as it does not know how to read them. See screenshot below:

Figure 3 - Notepad trying to view upgrader_default.log file (which is compressed)

Workarounds (till supported is added in by tool developers)

For Linux

If you use SIFT or another Linux system to do your forensics, the fix is simple. A few months back, Eric Biggers wrote a plugin to handle this. Its a plugin to the ntfs-3g FUSE driver. Its available here:
https://github.com/ebiggers/ntfs-3g-system-compression

For this, you will first need to download, compile and install the latest version of the ntfs-3g driver (but not from Tuxera, that one is missing a file!); then proceed to download, compile and install the above mentioned plugin. You can get this working on SIFT with roughly the following steps:

1. Go to https://launchpad.net/ubuntu/+source/ntfs-3g and download the source code for the latest stable release, right now its ntfs-3g_2016.2.22AR.1.orig.tar.gz.
2. Unzip and extract the file downloaded.
3. Open Terminal and browse to the extracted folder.
4. Compile and install using commands:

./configure
make
sudo make install

4. Go to https://github.com/ebiggers/ntfs-3g-system-compression and download the entire code as a zip file.
5. Unzip and extract the archive.
6. Open Terminal and browse to the extracted folder.
7. A few more tools need to be installed to compile this, so run the following commands:

sudo apt-get update
sudo apt-get install autoconf automake libtool

8. Run following commands to generate a configure script:

mkdir m4
autoreconf -i

9. Compile and install

./configure
make
sudo make install

10. If all went well (without errors), you are done!

Now you should be able to view and read those files normally, all decompression is handled on the fly automatically!

Figure 4 - No errors seen listing or reading files after installing the system compression plugin

For Windows

If you use Windows as your host machine for forensics processing, then you should only use a Windows 10 machine for processing evidence files that contain windows 10 images. This applies to tasks such as antivirus scanning, where you would typically share the entire disk out using Disk emulation (if you use Encase) which allow windows to parse and interpret the disk. This would only work (to read system compressed files) if the host system is Windows 10.

If you are looking to identify the system compressed files, you could filter on all files with ADS streams that have the name 'WofCompressedData'.

Fortunately, by default windows only compresses system files (EXE/DLL in windows and system32) and not user files, so you should mostly be fine. However, users can compress any file manually using the compact command.