Inside Encrypting File System, Part 2

Learn how Win2K encrypts and decrypts files

Last month, I began this two-part series by introducing the basics of Encrypting File System (EFS). I concluded by discussing how EFS generates keys and stores the keys in an EFS attribute with the file the keys will encrypt. This month, I conclude my walk through the encryption process. I also discuss the decryption process and other functionality that the EFS driver provides, including encrypted file backup and restore and the ability to view information about encrypted files.

Encrypting File Data, Continued
I concluded Part 1 at the point at which EFSEncryptFileSrv finishes creating Data Decryption Field (DDF) and Data Recovery Field (DRF) key fields for a file that is undergoing encryption. Figure 1, page 54, depicts the encryption process' flow. After EfsEncryptFileSrv constructs the necessary information for a file a user wants to encrypt, EFS can begin encrypting the file. The Local Security Authority Server's (LSASRV's) EncryptFileSrv function guides this phase of the encryption process. First, EncryptFileSrv creates a backup file, efs0.tmp, for the file undergoing encryption. (EncryptFileSrv uses higher numbers in the backup filename if other backup files exist.) EncryptFileSrv creates the backup file in the directory that contains the file undergoing encryption. Then, EFS applies a restrictive security descriptor to the backup file so that only the OS (i.e., the System account) can access the file's contents. EFS initializes the log file that EfsRpcEncryptFileSrv created in the first phase of the encryption process. Finally, EFS records in the log file that EncryptFileSrv created the backup file. EFS encrypts the original file only after the file is completely backed up.

EncryptFileSrv next prepares to send the EFS device driver a command to add to the original file the EFS information that EncryptFileSrv just created. EncryptFileSrv encrypts the command with DESX (a stronger variant of Data Encryption Standard—DES) and a session key before sending the command. When the system initializes, LSASRV uses the CryptoAPI function CryptGenRandom to produce a 128-bit number to serve as the session key. The EFS driver asks LSASRV for the session key via local procedure call (LPC); when the driver has the key, the driver can decrypt the control commands that EncryptFileSrv encrypted with the session key. EFS encrypts control commands so that malicious users can't develop programs to send commands specifying that files adopt incorrect EFS information, or that destroy files' valid EFS information. (A file with invalid EFS information is essentially destroyed because EFS can't then interpret the file's data correctly.)

NTFS receives the command to add the EFS information to the original file, but because NTFS doesn't understand EFS commands, NTFS calls the EFS driver function EfsFileControl. EfsFileControl uses its copy of the session key to decrypt the control command, then calls the EfsSetEncrypt function. EfsSetEncrypt takes the EFS information that LSASRV sent and uses exported NTFS functions to apply the information to the file. The exported NTFS functions let EFS add meta data information to NTFS files. The EFS driver uses NtOfsCreateAttributeEx, NtOfsSetLength, NtOfsPutData, and NtOfsCloseAttribute to create the $EFS NTFS meta data attribute (which is new to Windows 2000—Win2K) and copy the EFS information to the attribute. Simultaneously, the EFS driver returns a block of context data to NTFS and associates the data with the file's NTFS in-memory management data. NTFS will pass this data, or context information, to EFS whenever NTFS invokes an EFS driver callback function for the file. EFS makes use of the context information to store an unencrypted version of the file's file encryption key (FEK).

Execution returns to EncryptFileSrv, which copies the contents of the file undergoing encryption to the backup file. When the backup copy is complete, including backups of all alternate data streams, EncryptFileSrv records in the log file that the backup file is up-to-date. EncryptFileSrv then sends another DESX-encrypted command to NTFS to tell NTFS to encrypt the contents of the original file.

When NTFS receives the EFS command to encrypt the file, NTFS deletes the contents of the original file and copies the backup data to the file. After NTFS copies each section of the file, NTFS flushes the section's data from the file system cache, which prompts the Cache Manager to tell NTFS to write the file's data to disk. Because the file is marked as encrypted, at this point in the file-writing process, NTFS calls EFS to encrypt the data before NTFS writes the data to disk. The EFS function EfsWrite uses the unencrypted FEK in the file's context information to perform DESX encryption of the file, one sector (512 bytes) at a time.

On Win2K versions approved for export outside the United States, the EFS driver implements a 56-bit key DESX encryption. I stated in Part 1 that a FEK is 128 bits long; however, only the first 56 bits constitute the DESX key for the EFS export version. For the US-only version of Win2K, the key is 128 bits long, and the entire FEK constitutes the key.

After EFS encrypts the file, EncryptFileSrv records in the log file that the encryption was successful and deletes the file's backup copy. Finally, EncryptFileSrv deletes the log file and returns control to the application that requested the file's encryption.

Table 1 summarizes the steps EFS performs to encrypt a file. If the system crashes during the encryption process, either the original file remains intact or the backup file contains a consistent copy. When LSASRV initializes after a system crash, it looks for log files under the System Volume Information subdirectory on each NTFS drive on the system. If LSASRV finds one or more log files, it examines their contents and determines how recovery should take place. LSASRV deletes the log file and the corresponding backup file if the original file wasn't modified at the time of the crash; otherwise, LSASRV copies the backup file over the original, partially encrypted file, then deletes the log and backup. After EFS processes log files, the file system will be in a consistent state with respect to encryption, with no loss of user data.

As I stated in Part 1, the OS can designate directories as encrypted. EFS automatically encrypts any files a user moves into or creates in an encrypted directory.

The Decryption Process
The decryption process begins when a user opens an encrypted file. NTFS examines the file's attributes when opening the file, then executes the callback function EfsOpenFile in the EFS driver. EfsOpenFile reads the EFS attribute associated with the encrypted file. To read the attribute, EfsOpenFile calls several of the EFS support functions that NTFS exports for EFS's use. First, EfsOpenFile calls NtOfsCreateAttributeEx to open the attribute. NtOfsCreateAttributeEx supports opening any of a file's NTFS attributes, including the file's name attribute (which stores the file's name and attribute data), security attribute, and data attributes (which store the file data). For a refresher course on attributes, see NT Internals: "Inside NTFS," January 1998. To use EfsOpenFile to open the EFS attribute, EFS specifies $EFS as the attribute name and the numeric identifier of the $EFS attribute.

EfsOpenFile next opens the EFS attribute and calls the NTFS function NtOfsQueryLength to determine the attribute's length. Then, EfsOpenFile allocates a buffer to store the attribute's contents, calls NtOfsMapAttribute to obtain a virtual memory pointer to the contents, and copies the contents into the buffer. EfsOpenFile returns the EFS attribute data and bookkeeping information to NTFS. If EfsOpenFile can't open the EFS attribute, NTFS fails the file's open operation and returns an appropriate error code to the application that tried to open the file.

If the EFS attribute opens successfully, NTFS completes the necessary steps to open the file. When the file-open operation completes, NTFS invokes the EFS callback function EFSFilePostCreate. NTFS passes the context information from EfsOpenFile as a parameter to EFSFilePostCreate. EFSFilePostCreate ensures that the user opening the file has access to the file's encrypted data (i.e., that an encrypted FEK in either the DDF or DRF key rings corresponds to a public key/private key pair associated with the user). As EFS performs this validation, EFS obtains the file's decrypted FEK to use in subsequent data operations the user might perform on the file.

EFSFilePostCreate can't decrypt a FEK and relies on LSASRV (which can use the CryptoAPI) to perform FEK decryption. Before EFSFilePostCreate sends an LPC message to LSASRV, EFSFilePostCreate extracts the security ID (SID) of the user opening the file from the current process token. Then, EFSFilePostCreate attaches to the lsass.exe process (LSASRV's location). When a kernel thread attaches to a process, the thread associates with the process' (in this case lsass.exe) user-mode virtual memory; the thread doesn't attach to the virtual memory of the process to which the thread belongs (in this case the process opening the file). EFSFilePostCreate allocates a buffer in the LSASS process' virtual memory and copies the EFS context information—which includes the EFS attribute data that EfsOpenFile passed to EFSFilePostCreate indirectly—into the buffer. Because EFSFilePostCreate is now executing as a thread in the lsass.exe process (which executes in the System account) instead of as a thread in the process executing in the user's account that is opening the file, EFSFilePostCreate must impersonate the user opening the file. EFSFilePostCreate calls the PsImpersonateClient Security Manager function to accomplish the impersonation. Now, all the information LSASRV needs is in place, and EFSFilePostCreate sends an LPC message by way of the ksecdd.sys driver to LSASRV. The LPC message asks LSASRV to obtain the decrypted form of the encrypted FEK in the EFS attribute data that corresponds to the user the thread is now impersonating.

When LSASRV receives the LPC message, LSASRV executes the EFS function LpcEfsDecryptFek. To execute in the security context of the user opening the file, rather than in the System security context, LpcEfsDecryptFek uses an LPC impersonation facility that changes the function's security context to the user's security context. As I described in Part 1, both EFS and the CryptoAPI require access to a user's Registry profile settings; therefore, LpcEfsDecryptFek executes the userenv.dll (User Environment DLL) LoadUserProfile API to bring the user's profile into the Registry, if the profile isn't already loaded.

LpcEfsDecryptFek calls the DecryptFek function to perform decryption. DecryptFek doesn't know which of the encrypted FEKs in the DDF and DRF key rings correspond to the user opening the file. To find the user's FEK, DecryptFek proceeds through each key field in the EFS attribute data, using the user's private key to try to decrypt each FEK. Figure 2, reprinted from Part 1, shows an example EFS attribute's format. For each key, DecryptFek calls the function GetFekFromEncryptedKeys, which attempts to decrypt a DDF or DRF key entry's FEK. GetFekFromEncryptedKeys first obtains a CryptoAPI reference to the user's private key. Last month, I explained that a certificate hash that the encryption process stores in a user's private key entry uniquely identifies a user's key pair. Because a user can have more than one EFS public key/private key pair, GetFekFromEncryptedKeys relies on the certificate hash in the key entry to identify the private key it will use to try to decrypt the encrypted FEK stored in that field. Because EFS stores EFS key pairs in the My certificate storage area, GetFekFromEncryptedKeys specifies the My storage area when requesting the CryptoAPI function CertOpenSystemStore to open the appropriate certificate store. GetFekFromEncryptedKeys gives CertFindCertificateInStore the current key entry's certificate hash, and CertFindCertificateInStore searches the My store for the key pair the certificate hash identifies.

If CertFindCertificateInStore doesn't find the key pair's certificate in the certificate store, it returns an error. In that case, GetFekFromEncryptedKeys moves on to the next key field. If DecryptFek can't decrypt any DDF or DRF key field's FEK, the user can't obtain the file's FEK. Consequently, EFS denies access to the application opening the file. However, if CertFindCertificateInStore is successful, DecryptFek uses CryptAcquireContext to open a cryptographic session with the security provider that issued the key pair the certificate hash designates. To open the session, CryptAcquireContext requires the name of the container storing the key pair, the name of the cryptographic provider, and the cryptographic provider's type. DecryptFek obtains this information by invoking the CertGetCertificateContextProperty CryptoAPI before calling CryptAcquireContext.

Although an EFS attribute's key fields store the key pair container name, provider name, and provider type, in Win2K beta 3, DecryptFek doesn't use the key fields' data. Win2K's release version EFS might use this information, however, to help it directly access a key pair by using CryptAcquireContext. Using the data stored in the key fields means EFS won't have to query the cryptographic provider for the container and provider name and provider type via CertGetCertificateContextProperty. This optimization can significantly speed EFS's access to encrypted files.

After DecryptFek opens a cryptographic session with the provider, DecryptFek accesses the user's private key via a call to CryptGetUserKey. DecryptFek never directly accesses the user's private key; rather, DecryptFek uses the call to CryptGetUserKey to tell the provider that DecryptFek wants to perform subsequent cryptographic functions with the user's key pair. Finally, DecryptFek invokes the CryptDecrypt CryptoAPI function to decrypt the FEK with the user's private key. As an extra security measure, DecryptFek calls the function EfspValidateEfsStream after CryptDecrypt decrypts the FEK. EfspValidateEfsStream obtains a Message Digest 5 (MD5) hash of the EFS attribute and decrypted FEK and compares the MD5 hash with the hash value that the EFS attribute's header stores. A mismatch between the hashes indicates that the EFS attribute is corrupt. In this case, EFS reads the current list of recovery agents, recomputes the checksum, applies the new EFS attribute to the file, then rebuilds the EFS attribute's DRF entries. By updating the DRF entries, EFS ensures that recovery agents can always access a file's data, even when the file's DDF entries are corrupted.

Because DecryptFek processes DDF and DRF key rings when decrypting a FEK, DecryptFek automatically performs file-recovery operations. If a recovery agent that isn't registered to access an encrypted file (i.e., it doesn't have a corresponding field in the DDF key ring) tries to access a file, EFS will let the recovery agent gain access because the agent has access to a key pair for a key field in the DRF key ring. EFS adds recovery-agent DRF entries to a DRF key ring for each recovery key pair in the system. A systems administrator can make any number of users recovery agents by assigning them access to an EFS recovery key pair.

Keeping EFS Attribute Information Up-to-Date
EFS ensures that the information the EFS key fields store is always current. For example, users might receive new EFS certificates, or they might receive a new EFS key pair if their original pair is compromised. If the key field for which DecryptFek successfully decrypted the FEK is in the DDF key ring of the EFS attribute, DecryptFek calls UserKeyCurrent. If the field is in the DRF key ring, DecryptFek calls RecoveryInformationCurrent. These functions complete the same basic operation: They both compare the SID, cryptographic provider name, cryptographic container name, and cryptographic hash value stored in the EFS attribute with the user's SID and the properties of the user's current EFS cryptographic key pair. The HKEY_CURRENT_USERSoftware MicrosoftWindowsNTCurrentVersionEFSCurrentKeys CertificateHash Registry key stores the user's active EFS key pair certificate hash. If any of the components in the key field don't match the current key pair's properties, including the user's SID, EFS must update the key field.

To update a key field in the DDF, DecryptFek calls ReplaceUserKey. To update a key field in the DRF, DecryptFek calls UpdateRecoveryInformation. These functions perform the same steps that ConstructKeyRing performs when a user encrypts a file for the first time. However, ReplaceUserKey and UpdateRecoveryInformation update the key fields instead of creating new fields. Both functions add the updated field to the end of the DDF or DRF key ring of the EFS attribute, then delete the old field.

After DecryptFek successfully obtains the file's FEK, control returns to LpcEfsDecryptFek, which allocates a virtual memory buffer and copies the FEK into the buffer. If the user's profile loaded specifically for the FEK decryption, LpcEfsDecryptFek unloads the profile and returns the decrypted FEK to the EFS driver via an LPC reply message. The EFS driver's EFSFilePostCreate function copies the decrypted FEK from the buffer into a kernel buffer. Then, in a somewhat odd move, EFSFilePostCreate sends a control command to itself by way of the NTFS driver. This control command executes the SetEfsData function in the EFS driver, which takes the decrypted FEK and associates it with the opened file. The control command also updates the file's EFS attribute if EFS updated a DDF or DRF field. As with all control commands that the EFS driver interprets, EFS encrypts this command using DESX and the EFS session key before sending it.

Decrypted FEK Caching
Traveling the path from EFS to LSASRV and back can take a relatively long time—in the process of decrypting a FEK, CryptoAPI uses results in more than 2000 Registry API calls and 400 file-system accesses on a typical system. The EFS driver, with the aid of NTFS, uses a cache to try to avoid this expense.

As I've described, NTFS keeps the EFS context information that the EFS driver associates with a file, and NTFS passes the context information to EFS when invoking an EFS callback function. NTFS stores EFS context information with NTFS's file context information, or file control block (FCB). After a user opens a file, NTFS optimizes subsequent opens of that file by keeping the FCB in an FCB table. By doing so, NTFS doesn't need to reconstruct the context information. When NTFS calls EFSFilePostCreate, NTFS passes the FCB to EFS so that EFS can see whether the FCB stores EFS's context information. If the FCB does store EFS's context information, EFS can choose to bypass the FEK decryption because EFS's context data stores the decrypted FEK.

EFS maintains a cache that stores user SID/EFS attribute-hash pairs. After a FEK is successfully decrypted, EFSFilePostCreate calls EfsRefreshCache to create a cache entry for the decrypted FEK. Thus, before EFSFilePostCreate goes to the trouble of requesting LSASRV to decrypt a file's FEK, EFSFilePostCreate looks in the FCB that NTFS handed to EFS to see whether EFS's context information is present. If the FCB contains the EFS context data, EFS has already decrypted the file's FEK. Therefore, EFSFilePostCreate calls EfsFindInCache to look for an entry in the EFS cache that contains the current user's SID (extracted from the token of the process opening the file) and the EFS attribute hash (taken from the FCB's EFS context information). If EfsFindInCache finds this entry, EfsFindInCache runs a check to determine whether the entry was created more than 5 seconds earlier. If the entry is older than 5 seconds, EFS discards the entry and follows the usual path through LSASRV to decrypt the FEK. If the entry is younger than 5 seconds, EFS uses the copy of the decrypted FEK in the EFS context data that the FCB has already associated with the file.

Decrypting File Data
After an application opens an encrypted file, the application can read from and write to the file. NTFS calls the EFS callback EfsRead to decrypt file data as NTFS reads the data from the disk, and before NTFS places the data in the file system cache. Similarly, when an application writes data to a file, the data remains in unencrypted form in the file system cache until the application or the Cache Manager uses NTFS to flush the data back to disk. When an encrypted file's data writes back from the cache to the disk, NTFS calls the EFS callback EfsWrite to encrypt the data.

NTFS passes the file's FCB as a parameter to both EfsRead and EfsWrite. The FCB contains the file's EFS context data, and the EFS driver uses the file's decrypted FEK, which the EFS context data includes, to perform DESX decryption or encryption of file data buffers. The EFS driver performs encryption and decryption in 512-byte units. The 512-byte size is the most convenient for the EFS driver, because disk reads and writes occur in multiples of the 512-byte sector.

Backing Up Encrypted Files
An important aspect of any file encryption facility's design is that file data is never available in unencrypted form except to applications that access the file via the encryption facility. This restriction particularly affects backup utilities, in which archival media store files. EFS addresses this problem by providing a facility for backup utilities so that the utilities can back up and restore files in their encrypted states. Thus, backup utilities don't have to be able to decrypt file data, nor do the utilities decrypt file data in their backup procedures.

Backup utilities use the new EFS APIs OpenEncryptedFileRaw, ReadEncryptedFileRaw, and WriteEncryptedFileRaw in Win2K to access a file's encrypted contents. The advapi32.dll library provides these APIs, which all use LPC to invoke corresponding functions in LSASRV. For example, after a backup utility opens a file for raw access during a backup operation, the utility calls ReadEncryptedFileRaw to obtain the file data. The LSASRV function EfsReadFileRaw issues control commands (which the EFS session key encrypts with DESX) to the NTFS driver to first read the file's EFS attribute, then the encrypted contents.

EfsReadFileRaw might have to perform multiple read operations to read a large file. As EfsReadFileRaw reads each portion of such a file, LSASRV sends a remote procedure call (RPC) message to advapi32.dll that executes a callback function that the backup program specified when it issued the ReadEncryptedFileRaw API. EfsReadFileRaw hands the encrypted data it just read to the callback function, which can write the data to the backup media. Backup utilities restore encrypted files in a similar manner. The utilities call the WriteEncryptedFileRaw API, which invokes a call-back function in the backup program to obtain the unencrypted data from the backup media while LSASRV's EfsWriteFileRaw function is restoring the file's contents.

Viewing EFS Information
EFS has other APIs that applications can use to manipulate encrypted files. For example, applications use AddUsersToEncryptedFile to give additional users access to an encrypted file and RemoveUsersFromEncryptedFile to revoke users' access to an encrypted file. Applications use QueryUsersOnEncryptedFile to obtain information about a file's associated DDF and DRF key fields. QueryUsersOnEncryptedFile returns the SID, certificate hash value, and display information that each DDF and DRF key field contains. Screen 1 shows the output of EFSDump, a utility I developed that displays information that QueryUsersOnEncryptedFile returns. You can see that the file mshtmled.dll has one DDF entry for user Joe and one DRF entry for Mark Russinovich, which is the only recovery agent currently registered on the system. You can download EFSDump, with full source code, from http://www.sysinternals.com/misc.htm.

EFS Caveats
As you use EFS to protect your data, you need to be aware of some important concerns. First, compression and encryption aren't compatible. If you specify both attributes for a file, encryption will override compression.

A more important concern is that even after you encrypt an existing file, the file's original unencrypted data remains where NTFS stored the file on disk before you encrypted it. NTFS eventually reuses this disk space for storing other files and directories, but until NTFS reallocates the space, you can use a disk editor to bypass NTFS (e.g., from within NT or from a DOS boot disk) and directly access the unencrypted file data. The only way you can prevent such access is to run a utility that overwrites all free file system areas to destroy vulnerable information in the disk's unallocated portions. You can download one such utility—Sdelete—including its source code, from http://www.sysinternals.com/sdelete.htm.