Further Explorations in Developing Accessible Windows Applications

You've probably heard the roar of the accessibility train: It's that noise from your company's executives complaining constantly about application accessibility. Since these folks pay your salary, it's in your best interest to turn that noise into action items on your work plan.

In this article, I want to talk specifically about the tools and frameworks that you can use to make your application more accessible. In my previous article about accessibility, "How to Code WPF Applications for Accessibility," I walked you through the process of making your Windows Presentation Foundation (WPF) application accessible. Although WPF makes it significantly easier to build accessibility into .NET applications, you will still need to do additional development work so that your applications meet all accessibility mandates.

Much of my first accessibility article was built on working with the Microsoft User Interface Automation (UIA) framework, but I didn't spend much time discussing its internals. The idea in this article is to expose you to some of the internals of UIA so that you can:
1.    Provide thought leadership in the area of accessibility.
2.    Provide workable solutions to meet accessibility requirements.
3.    Identify and repair any technical issues that arise.

Thought Leadership in Action
Thought leadership is a big deal these days. If you don't understand accessibility and its impact, you aren't in a position to lead from the front. I think of thought leadership as the exact opposite of the mindless babble that encourages executives to purchase products first, then find uses for them later. Rather, thought leadership engages the client in a probing dialogue about the pros and cons of a specific technology.

It goes a step further by laying out a path toward technology adoption. As an example, consider thought leadership around UIA. A typical adoption strategy for UIA focuses on the value-add versus the status quo. It puts a dollar figure on the cost involved, measures the effort, and numbers the technical resources required to go from drawing board to a production roll-out. It outlines the best practices for developers and prepares mitigation strategies for times when things go "bump in the night."

The thought leadership plan calculates the risk involved in the adoption process so that decisions are made from a platform of knowledge, not based on ignorance and magic pixie dust. Accessibility thought leaders are walking, talking value additions to the companies that they work for because they help move a company toward accessibility. And that is no easy feat!

If you intend to be a thought leader in accessibility, you need to focus on some key reasons why accessibility is important. You'll also need to recommend the best way to implement accessibility today. I'll focus on the latter, as I assume that you have had "accessibility 101" training. The choices available today for implementing accessibility in Windows applications lie between Microsoft Active Accessibility (MSAA) and UIA. There are no other choices on the Microsoft Windows platform.

MSAA is the legacy accessibility framework that was introduced with Windows 95. It is based on COM and has run its course. That sums it up pretty nicely! UIA is the new accessibility framework for Microsoft Windows, introduced with Windows Vista and intended to replace MSAA. If you're considering developing applications to meet accessibility mandates, you should use UIA.

As a thought leader, you'll point out that applications based on UIA can expose more information to accessible clients than MSAA. Development cycles will be required to augment the basic information. Basic information is typically restricted to some control properties such as visibility, UI tree structure information, and events.

The fact that MSAA is based on COM means that accessibility interfaces are "set in stone." Unless Microsoft releases a new interface—another "stone"—you won't be able to take advantage of any new features available today. For instance, multi-touch screens that are available on mobile platforms have no native support for accessibility via MSAA. That will require development effort.

On the plus side of things, MSAA is everywhere! There's a risk that what you implement today in UIA may not work as intended when consumed by legacy clients, or may work with a reduced feature set than what the application was designed to deliver. I'll show you an example of this later. The point is that you'll need to know these exact limitations before making any recommendations. You'll also need to know how to work around these limitations.

When you recommend UIA as an accessibility framework, you'll need to be able to discuss its internals with confidence. Let's turn the focus of the discussion to Windows Automation 3. Figure 1 shows a simplified representation of the Windows Automation API containing the runtime assemblies and accessibility frameworks.

Figure 1: Windows Automation API 3.0 package

As shown in Figure 1, Windows Automation 3 contains both MSAA and UIA. There are a few other interfaces that are part of the collection, but I've chosen to ignore them to keep things simple.

Clients that interface with applications built on Windows Automation 3 can use either framework by consuming that particular interface. That way, both legacy and modern assistive technology can participate in the accessibility dialog. UI Automation is not just available for managed applications. Accessible clients and providers can be created without the use of managed code.

Windows Automation 3 ships with every current version of Windows. If the OS contains WPF, it also contains Windows Automation 3. From a development perspective, your shop has all the tools in place for building and supporting accessible applications. There is no other piece of software or hardware that you need to purchase or install.

Most developers incorrectly associate Windows Automation with testing methodologies. That's only partly true; it does allow the automation of other applications. However, as Figure 1 shows, it also rolls MSAA and UIA under one umbrella.

Let's say you create a WPF application that reads data from a database and displays that data in a grid. That application, referred to as an application provider, will contain references to the Windows Automation 3 assembly, as Figure 2 shows.

Figure 2: Windows Automation API assemblies

On the other hand, if you're creating an assistive tool such as a narrator, you'll need to select the appropriate .NET client profile, as shown in Figure 3.

Figure 3: Client profiles in Visual Studio 2010

Then, add the specific UIA assemblies depending on the type of feature set you are including. For instance, a narrator may include the UIA Provider assemblies because it consumes the provider interfaces of the application that is running. It may also contain the UIA Client assemblies.

If you choose not to include the automation assemblies, you'll need to hook into accessibility by implementing the IAccessible interface for unmanaged code. I'll discuss this in more detail later. For now, you need to understand that the dialogue between the narrator and provider will not be the best it can possibly be because that conversation has to be translated and mapped to common pieces that both applications understand.

Here's an example of what I mean. Consider that one of the controls on the WPF application provider is a range control. When that control is queried by the narrator, it returns the current value and the lower and upper bounds representing the valid range of values for that control: 14, 0, and 25 respectively. Narrators based on UIA understand 14, 0, and 25. Narrators based on MSAA can understand only the current value of 14. The valid range, defined by its minimum and maximum value, is lost in translation.

There's no way to fix this except through development effort. If there is any good news in that, it is that the application can still present some useful information—the current value of 14—to the assistive tool. And perhaps that is sufficient?!

The lesson here is that accessibility is a two-way street. In addition to implementing UIA, you also need to recommend the use of adaptive technology that supports UIA. As a thought leader, you'll present three options to your client.

In most cases, small companies can simply mandate the use of a particular version of software. The upgrade path is not particularly problematic and doesn't require that much effort. Larger companies sit on the other end of the spectrum in that regard.

For instance, consider that the US federal government can't simply mandate its accessibility staff to start using the JAWS 11 screen reader, a version that supports UIA. It's impractical because licensing, upgrades, software testing, and roll-out cycles can easily span a number of years. Uncle Sam has to incrementally work toward achieving that goal, perhaps by using a combination of all three recommendations.

Quirks and Design Limitations
There's another scenario that I need to discuss. Consider the case where there is no source code for an executable running in production. If that application must be made accessible, you'll need to proceed with a good dose of caution.

If you are puzzled about applications with no source code, welcome to my world! There are several reasons including, but not limited to:

While I make light of the first reason, it does happen with disturbing frequency. I have actually been assigned to projects where nobody has seen the source code in years. However, the application is up and running in production without issue.

How are you supposed to implement accessibility in this application? Actually, the courts don't care! Applications must follow federal accessibility mandates. Target.com knows this first-hand, as it cost the company at least $6 million (not counting lawyer fees and other expenses) to settle a class-action lawsuit brought by National Federation of the Blind.

Although the choices are extremely limited for these ugly scenarios, you do have some wiggle room depending on your appetite for risk. You'll need to replace the application with one that is accessible. There is no getting around this obstacle. To buy you some time while the replacement application is being built or purchased, you can modify the existing application at runtime to make it accessible.

Providing Workable Solutions
Let me explain this idea a bit more. In general, assistive tools can usually infer accessibility information from accessibility providers based on how that application lays out its UI components. For instance, a textbox control situated next to a label control on screen leads the assistive tool to infer that the label is associated with the text box. It can then present an accurate UI picture to the end user. However, this approach does not work if the label does not sit directly in front of the textbox. If your "no source code" application isn't designed like this, then you need to understand that there is no supported way to correct this problem.

Notice how carefully I worded that last sentence. There is a way, but it is unsupported! So here are the goods. You can programmatically inject accessibility into the application if you use a diagnostic tool from Microsoft called ManagedSpy.

ManagedSpy uses unorthodox techniques to inject code into the target process. The injected code uses reflection to alter the state of the object and its properties in the target process. It accomplishes this feat by breaking into the running process and dynamically rewriting the code. I first learned about this tool when a co-worker asked me to help troubleshoot an accessibility issue.

You'll see from Figure 4 that the before and after snapshots taken by AccExplorer32 show that the AccessibleName property has changed as a result of the injection.

Figure 4: Accessibility injection at runtime

Figure 5 shows the code.

Figure 5: ManagedSpy application used to infiltrate a running process

ControlProxy cp = ControlProxy.FromHandle(Injector.Handle);
cp.SetValue("AccessibleName", "A new test value.");

The code snippet is very simple and doesn't require a detailed examination. ControlProxy is a type that is defined in the Microsoft.ManagedSpy namespace and requires a reference to the ManagedSpyLib assembly. A ControlProxy represents a window running in another process. I also cheat by passing in the handle of the same running application (Injector.Handle), so it isn't really breaking into an external running process, but you get the idea.

Please note that the support statement for put_accName clearly states that this is not supported. I like to think of this approach as a measure of last resort.

I mention this technique only as a possible approach to a specific problem. Since the approach is unsupported, I have not included it as part of the code download for this article. That said, there is nothing else required than what is presented in Figure 5 to make this work if you decide to do this on your own. As a word of caution, that approach can be used to change any property for the running process, not just the accessible property.

Note that this is an entirely different concept than automating an application from UIA. UIA can only modify properties and behaviors that are modifiable as per the control pattern specification. This means that UIA can modify a TextBox control's text property; it cannot modify the accessible name property because that property does not have a value pattern associated with it.

There is another technique called MSAA Dynamic Annotation that is somewhat related to this approach. The difference between Dynamic Annotation and the ManagedSpy approach is that Dynamic Annotation gives the caller the opportunity to override the accessible value being returned from the target application provider. This approach is both recommended and supported by Microsoft.

ManagedSpy, on the other hand, changes the accessible property value in the target application provider. There is never a need to override values from that point on because these values are now correct in the target. The advantage of the ManagedSpy approach is that, once the data is adjusted inside the target application, any process—whether running in-process or out-of-process and interacting with the target application server—now receives that updated value.

However, with Dynamic Annotation, only the client application running in-process sees the new value. Other accessible clients will not see this new value. As of this writing, UIA supports Dynamic Annotation.

Identifying and Resolving Accessibility-Related Issues
The design goal of UIA is to improve the user experience by overcoming some of the shortfalls of MSAA that I listed earlier. UIA is based on patterns. A pattern is a rough equivalent of an interface except that a pattern contains some other important ingredients. Patterns are based on a set of published guidelines. Published guidelines allow developers to make correct assumptions about implementation details and to consume interfaces with confidence.

The chief culprit for these many issues in MSAA is IAccessible, a COM interface that supports accessibility and forms the backbone of MSAA. IAccessible suffers from a number of limitations: \[begin bullet list\]

It cannot be extended because it is an interface. And by definition, interfaces are not modifiable. You'll need to wrap and subclass the type to add your own feature set. And that process is typically very error prone.

Communication using IAccessible is slow. In fact, precisely because of this handicap, MSAA implementations run in the same process with the application server by default (MSAA applications are not called providers as in UIA). Note that implementations can run out-of-process as well using Distributed COM.

MSAA's implementation of IAccessible cannot run in the same process as a 64-bit application because it is a 32-bit process. You'll need to recompile your application server to target a 32-bit platform.

IAccessible is difficult to use correctly. \[end bullet list\]

UIA's backbone is built on IRawElementProviderSimple interface, not on IAccessible. This interface avoids all the shortcomings of IAccessible outlined previously while preserving all the benefits. UIA introduces another COM interface called IAccessibleEx that is intended as a short-term replacement for IAccessible. The specific responsibility of IAccessibleEx is to serve as a bridge between MSAA and UIA. It is easier to use than IAccessible and provides high-level services that do some of the heavy lifting for you.

IAccessibleEx also solves another thorny problem that needs some attention. IAccessible can refer to a parent or a child in the hierarchy of controls available to assistive tools. IAccessible differentiates these elements by exposing a ChildID parameter. A value of zero indicates that IAccessible is referring to an accessible object that is a parent. A nonzero value represents a child. Therefore, a single IAccessible can potentially represent any number of accessible objects in the UI control tree.

As is often the case, callers abuse the specification by passing in incorrect arguments to the ChildID parameter. The problem arises when code attempts to extend the interface. At this point, calling code cannot be absolutely sure which accessible object is being referenced in the control tree hierarchy, leading to inconsistencies in navigation and accessible information retrieval.

For this to work consistently, callers would be required to follow the specification by always passing in a valid ChildID parameter. Obviously, that approach is not enforceable because of language limitations in COM. IAccessibleEx overcomes this limitation by transitioning an IAccessible to an IRawElementProviderSimple from the UIA framework.

IRawElementProviderSimple can only refer to a single UI element. Therefore, it does not suffer the extensibility issues that plague IAccessible. When you examine an IRawElementProviderSimple, you know with confidence that it is a single UI element. As an added bonus, there is deep support for navigation of the UI control tree hierarchy as well as the ability to search and filter the control tree using different views.

What this means is that UIA performs better than MSAA, does not require in-process execution, can be consumed by 64-bit applications by default, and returns much more information to accessibility clients.

In the long term, UIA provider interfaces will replace IAccessible. MSAA clients can use IAccessibleEx to expose UIA properties and patterns without having to modify existing legacy code. MSDN provides detailed guidelines on implementing IAccessibleEx.

Geared Up for Accessibility
In this article, I've endeavored to present a detailed view of the modern accessibility frameworks and how they work toward improving the end-user accessibility experience. Looking back from here, appreciate that WPF makes great strides in meeting accessibility mandates. Frameworks such as UIA and Windows Automation 3 make the implementation task much easier as well. There are also potent tools like UI Spy and UIA Verify that may be used to enforce accessibility correctness.

If you are serious about implementing accessibility why not consider hiring a person with a disability to work with your team. There's no better opportunity to learn.


Alvin Bruney is a longtime ASP.NET MVP and author of five books. His current book, ASP.NET 4 by Example, is currently available for $20 on www.lulu.com/owc.