How disability helped change Microsoft’s design principles for Cortana and Bing

Imagine for a minute, if you will, waking up without the ability to walk, see, or hear. August de los Reyes, the design lead for Xbox, experienced just that, and his discovery has helped him shape and redefine Microsoft’s Cortana experience.

In an interview with Cliff Kuang of FastCo.Design, de los Reyes examines how much of the recent technology enjoyed by all came by way of empathy and a need for accessibility by the impaired, and how that realization helped focus Cortana’s development.

Recently, suffering from a fractured spine that permanently robbed him of his ability to walk, de los Reyes was thrust into a parallel universe that remarkably resembled the world he once took for granted. Previously overlooked obstacles such as curbs and overturned trash cans now presented the wheelchair-bound de los Reyes opportunities for innovative thought, along with a few frustrating realizations.

“After the accident, he had methodically set about trying to do as many things as he’d ever done. Yet the limitations soon become obvious. He’d try to meet friends at a favorite restaurant, only to discover that he couldn’t quite get inside because of one tiny curb that some contractor had overlooked. He’d be steering his wheelchair down the sidewalk, only to be met with a tipped-over garbage can, which would force him to circumnavigate an entire block.”

Rather than become sullen or dejected, de los Reyes saw the obstacles as a limitation of many thoughtless designers around him. The idea of disability being a personal problem rather than being a problem of unfortunate careless architecture is what ‘radicalized’ de los Reyes and prompted him to lend a hand in redefining how Cortana’s design could help everyone.

When newly appointed CEO of Microsoft, Satya Nadella promoted Albert Shum to Microsoft’s design director for the entire company, de los Reyes saw an opportunity to lend a hand. Shum, primarily responsible for Microsoft’s infamous flat design (for better or worse), had to chew on what exactly design meant beyond a two-dimensional sense, and de los Reyes was there to help. Using his previous curb experience as a metaphor, de los Reyes prompted the idea of universal design. Something as simple as a curb cut out benefits almost everyone while hindering relatively no one. Microsoft’s designs needed to be as abstractly and finite as a curb cut-out design.

Paired with Shum and Microsoft’s principle design director Kat Holmes, de los Reyes sought to do more than turn Microsoft into a curb cut-out company. Holmes, tasked with figuring out Microsoft’s digital assistant Cortana would behave, came across de los Reyes ideas on universal design. Interestingly enough, de los Reyes ideas helped Cortana become more than a Siri clone that attempted to ape human-like qualities.

“Making Cortana behave like a human would be like the new skeuomorphism.”

The three design heads came to the understanding that making Cortana pretend to be something it is not, would not only be poorly received by the masses but go against modern design principles established in the 20^th century. After studying actual personal assistants and watching the movie Her, Holmes sought the help of Harvard’s Allen Sayegh and Ontario College’s Jutta Treviranus alongside de los Reyes to infuse Cortana with what is dubbed as inclusive design. The concept of inclusive design stems from the study of building adaptive products for often overlooked communities such as the blind, dyslexic, deaf, or mute.

As Kuang puts it:

Let’s say you’d like to build a phone that’s easier to interact with while you’re driving. You could just try to study people driving with their phones. Or you could actually study how the blind use their phones. How do they know when their phones are paired with another device? What aural feedback do apps need to provide, when opened? You could build those features into a phone, so that by serving someone disabled, you serve everyone else better.

The Microsoft design group sees it as “reframing disability as an opportunity.” The design principles of inclusive design are being mandated throughout the company and have given birth to several dozen projects seeking to improve the lives of users universally. A newer project that deals with font and system text wrapping should be surfacing soon from the company to help those with dyslexia read more easily.

Even more impressive is landmark functionality being built into Bing maps. While a user is navigating by landmarks, Bing maps need to work that much harder to stay updated and relevant; studies have shown that women often navigate by landmarks and visual cues over more rigid cardinal directions. Rather than, “head southeast for 6.7 miles and turn right on Park Ave,” Bing Maps will begin to use slightly more nuanced language such as “head south for 6.7 miles, turn right at Dunkin Doughnuts. Now you’re on Park Ave.”

Similar to de los Reyes, subtle observations and implantations such as the Bing Maps landmark navigation or Cortana’s ability to adapt to its surrounding input, Microsoft is building products and processes that benefit everyone. Microsoft’s adoption of inclusive design may seem similar to kitchen sink approach it takes with Windows development, but the understanding behind it is miles apart.

Rather than building for specific use-cases in a sort of whack-a-mole approach, those in charge of design and functionality within Microsoft are beginning to research overall usability and flexible elements from the beginning. For more details on de los Reyes and the direction Microsoft is headed, please take time to read Cliff Kuang’s brilliant piece.