App-V and Application Compatibility Shims
Introduction
Since the release of Windows Vista application compatibility shims have been getting plenty of attention as organizations try to find solutions for applications that have migration issues when a new operating system is introduced into their organizations infrastructure. Interest from organizations looking to deploy Windows 7 has brought even more attention to the technology and in a way spurred further development of the Application Compatibility Toolkit; a free toolset provided by Microsoft to identify and manage application compatibility issues. This toolkit provides the ability to create what are called shims to that allow older applications to work with newer operating systems. Most of the confusion I see out in the App-V community is when to use App-V to solve a compatibility issue and when to use the Application Compatibility Toolkit so let’s explain what App-V can do then take it from there.
App-V and Application Compatibility
App-V is sometimes referred to as an application compatibility technology but it really isn’t a good technology in that context. App-V can do many wonderful things but the architecture doesn’t really address the majority of programming level incompatibilities with a new operating system. App-V has a virtual registry and virtual file system which is part of the SystemGuard™ technology that isolates the application from the operating system and other applications. The SystemGuard™ allows applications that are not well written for users who do not have local administrator rights to run the application in a sandbox where they are effectively a local administrator when it comes to the virtual file system and virtual registry.
High Level View of an App-V Application
On more rare occasions App-V may be used to overcome issues with software installers on newer operating systems by sequencing (packaging) the application on an older operating system then running that sequence on a newer operating system. Also in even more rare occasions this process allows the application to be packaged with older shared DLLs that the application may be compatible with but this sometimes does not work out because newer operating systems may not be compatible with these old DLLs that exist inside the sequence. In short we have learned the following about App-V and application compatibility.
· Can be used to mitigate issues where applications need admin access to registry and file system.
· Using an older operating system application installer compatibility issues can be overcome.
· On a rare occasion older DLLs can be captured inside the sequence to provide compatibility.
Application Shimming
Application shimming however is a completely different technology and approach to application compatibility. In a way this is the way application compatibility should be done because the application compatibility shims behave much like downgrading DLLs to older versions on a system but the way this effect is achieved isn’t quite so simple but the end result is more complete.
Some of the best hacks I’ve seen on the Windows platform involve what is called API hooking where the API call from an application is intercepted and redirected using linking. How this works in a nutshell is that the Windows Portable Executable (PE) using the Common Object File Format (COFF) specification contains headers (for things such as shared libraries) that provide a layer that can be used for indirection between the application and the linked file. At the core of this is the Import Address Table (IAT) that facilitates this indirection functionality. Essentially the table is built when the application is loaded but the shim engine can change some of the data being populated into this table to redirect API calls that normally go to the operating system.
Application Calling Windows Function Using the Import Address Table
In this example a Windows API call is put through the Import Address Table which then ensures that the correct shared library is called to fulfill the request.
Using the Import Address Table to Access a Shim
This example is much like the last except the Import Address Table redirects the call to a specific shim that imitates the API but has a different behaviour.
With the call being redirected to alternate code we can provide functionality to the application that didn’t previously exist in the operating system but what actually supplies this functionality? Most shims use helper DLLs that aren’t technically part of the operating system but are loaded by the shim engine when the application executable loads. Many off these DLLs are prefixed with AC* and exist in the C:\Windows\AppPatch folder of the system where they affect the behaviour of many system APIs. For example these are the DLLs on my Win7 x64 test machine.
These shims are applied based on properties of the executable so that the correct shim(s) are loaded for the application. Typically acgeneral.dll or aclayers.dll is used at this step to provide very generic compatibility fixes between the application and operating system such as operating system specific behaviours in older operating systems. The next type of shim is acspecific.dll where specific behaviours are applied such as a fix to an API to make it work correctly may break an application therefore a shim is needed to provide a behaviour the application required to operate properly even though the application is relying on a bug to function correctly. The third type provides flags to least privileged user account (LUA) or the application installer. The final type involves in memory patching of the executable rather than a traditional API hook.
Now you might be wondering with all this hacking going on is my system going to be safe and secure? All the code that runs within the shims must adhere to Windows programming standards and that includes the Windows security model. You may be able to lie to an application that it is an administrator but unless the user is actually elevated or the registry / file ACLs are loosened the application will still fail due to lack of privileges.
App-V and the Shim Engine
Now that we have an idea as to how both technologies address application compatibility how do we get these two technologies to work together so that we can have the best of both worlds? Well the answer is somewhat complicated so we will address how the shim engine and App-V can interact. The shim engine is loaded by kernel32.dll which is in user mode but it is essentially a part of the operating system. If we look at how App-V isolates applications we know that applications are isolated from each other to a great deal leaving only copy / paste, DDE and File Type Associations as methods of communication between virtual applications. Virtual applications can use COM objects, OLE interfaces, printers etc… from the operating system but the operating system is limited to copy / paste, DDE and File Association as methods of communication with the other virtual application(s).
App-V Communication Overview
What this all means is that the shim engine cannot see shims inside sequences because of the isolation that makes the product such a useful tool. When you apply a shim using the sdbinst.exe utility the shim settings are applied inside the sequence so that when the application is run the shim engine cannot go inside the sequence and read the required configuration information to modify the import address table. For example you can find your application compatibility flags in the local registry under the following locations.
[HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AppCompatFlags
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\AppCompatFlags
Conclusion
So how do we take advantage of shims with App-V? Well the first and most important lesson is that the shim must be installed to the operating system in order for it to function correctly with the shim engine. The virtual application can take advantage of the shim because its import address table can be built properly by the operating system and will redirect / modify API calls handled by the shim engine to effectively fix the application. Now that I’ve explained the architecture you can look forward to my follow up post that will focus on how to apply a shim that can be used with a virtual application and more importantly how to manage shims for virtual applications.