windows vista speech recognition

This browser is no longer supported.

Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.

Talking Windows

Exploring New Speech Recognition And Synthesis APIs In Windows Vista

• Robert Brown

This article is based on a prerelease version of WinFX. All information contained herein is subject to change.

Elements of Speech Talking to Windows Vista Windows Vista Speech APIs System.Speech.Synthesis System.Speech.Recognition Telephony Applications Conclusion

Microsoft has been researching and developing speech technologies for over a decade. In 1993, the company hired Xuedong (XD) Huang, Fil Alleva, and Mei-Yuh Hwang—three of the four people responsible for the Carnegie Mellon University Sphinx-II speech recognition system, which achieved fame in the speech world in 1992 due to its unprecedented accuracy. Right from the start, with the formation of the Speech API (SAPI) 1.0 team in 1994, Microsoft was driven to create a speech technology that was both accurate and accessible to developers through a powerful API. The team has continued to grow and over the years has released a series of increasingly powerful speech platforms.

In recent years, Microsoft has placed an increasing emphasis on bringing speech technologies into mainstream usage. This focus has led to products such as Speech Server, which is used to implement speech-enabled telephony systems, and Voice Command, which allows users to control Windows Mobile® devices using speech commands. So it should come as no surprise that the speech team at Microsoft has been far from idle in the development of Windows Vista™. The strategy of coupling powerful speech technology with a powerful API has continued right through to Windows Vista.

Windows Vista includes a built-in speech recognition user interface designed specifically for users who need to control Windows® and enter text without using a keyboard or mouse. There is also a state-of-the-art general purpose speech recognition engine. Not only is this an extremely accurate engine, but it's also available in a variety of languages. Windows Vista also includes the first of the new generation of speech synthesizers to come out of Microsoft, completely rewritten to take advantage of the latest techniques.

On the developer front, Windows Vista includes a new WinFX® namespace, System.Speech. This allows developers to easily speech-enable Windows Forms applications and apps based on the Windows Presentation Framework. In addition, there's an updated COM Speech API (SAPI 5.3) to give native code access to the enhanced speech capabilities of the platform. For more information on this, see the "New to SAPI 5.3" sidebar.

Elements of Speech

The concept of speech technology really encompasses two technologies: synthesizers and recognizers (see Figure 1 ). A speech synthesizer takes text as input and produces an audio stream as output. Speech synthesis is also referred to as text-to-speech (TTS). A speech recognizer, on the other hand, does the opposite. It takes an audio stream as input, and turns it into a text transcription.

Figure 1 Speech Recognition and Synthesis

A lot has to happen for a synthesizer to accurately convert a string of characters into an audio stream that sounds just as the words would be spoken. The easiest way to imagine how this works is to picture the front end and back end of a two-part system.

The front end specializes in the analysis of text using natural language rules. It analyzes a string of characters to figure out where the words are (which is easy to do in English, but not as easy in languages such as Chinese and Japanese). This front end also figures out details like functions and parts of speech—for instance, which words are proper nouns, numbers, and so forth; where sentences begin and end; whether a phrase is a question or a statement; and whether a statement is past, present, or future tense.

All of these elements are critical to the selection of appropriate pronunciations and intonations for words, phrases, and sentences. Consider that in English, a question usually ends with a rising pitch, or that the word "read" is pronounced very differently depending on its tense. Clearly, understanding how a word or phrase is being used is a critical aspect of interpreting text into sound. To further complicate matters, the rules are slightly different for each language. So, as you can imagine, the front end must do some very sophisticated analysis.

The back end has quite a different task. It takes the analysis done by the front end and, through some non-trivial analysis of its own, generates the appropriate sounds for the input text. Older synthesizers (and today's synthesizers with the smallest footprints) generate the individual sounds algorithmically, resulting in a very robotic sound. Modern synthesizers, such as the one in Windows Vista, utilize a database of sound segments built from hours and hours of recorded speech. The effectiveness of the back end depends on how good it is at selecting the appropriate sound segments for any given input and smoothly splicing them together.

If this all sounds vastly complicated, well, it is. Having these-text- to speech capabilities built into the operating system is very advantageous, as it allows applications to just use this technology. There's no need to go create your own speech engines. As you'll see later in the article, you can invoke all of this processing with a single function call. Lucky you!

Speech recognition is even more complicated than speech synthesis. However, it too can be thought of as having a front end and a back end. The front end processes the audio stream, isolating segments of sound that are probably speech and converting them into a series of numeric values that characterize the vocal sounds in the signal. The back end is a specialized search engine that takes the output produced by the front end and searches across three databases: an acoustic model, a lexicon, and a language model. The acoustic model represents the acoustic sounds of a language, and can be trained to recognize the characteristics of a particular user's speech patterns and acoustic environments. The lexicon lists a large number of the words in the language, along with information on how to pronounce each word. The language model represents the ways in which the words of a language are combined.

Neither of these models is trivial. It's impossible to specify exactly what speech sounds like. And human speech rarely follows strict and formal grammar rules that can be easily defined. An indispensable factor in producing good models is the acquisition of very large volumes of representative data. An equally important factor is the sophistication of the techniques used to analyze that data to produce the actual models.

Of course, no word has ever been said exactly the same way twice, so the recognizer is never going to find an exact match. And for any given segment of sound, there are very many things the speaker could potentially be saying. The quality of a recognizer is determined by how good it is at refining its search, eliminating the poor matches, and selecting the more likely matches. A recognizer's accuracy relies on it having good language and acoustic models, and good algorithms both for processing sound and for searching across the models. The better the models and algorithms, the fewer the errors that are made, and the quicker the results are found. Needless to say, this is a difficult technology to get right.

While the built-in language model of a recognizer is intended to represent a comprehensive language domain (such as everyday spoken English), any given application will often have very specific language model requirements. A particular application will generally only require certain utterances that have particular semantic meaning to that application. Hence, rather than using the general purpose language model, an application should use a grammar that constrains the recognizer to listen only for speech that the application cares about. This has a number of benefits: it increases the accuracy of recognition, it guarantees that all recognition results are meaningful to the application, and it enables the recognition engine to specify the semantic values inherent in the recognized text. Figure 2 shows one example of how these benefits can be put to use in a real-world scenario.

Figure 2** Using Speech Recognition for Application Input **

Talking to Windows Vista

Accuracy is only part of the equation. With the Windows Vista speech recognition technology, Microsoft has a goal of providing an end-to-end speech experience that addresses key features that users need in a built-in desktop speech recognition experience. This includes an interactive tutorial that explains how to use speech recognition technology and helps the user train the system to understand the user's speech.

The system includes built-in commands for controlling Windows—allowing you to start, switch between, and close applications using commands such as "Start Notepad" and "Switch to Calculator." Users can control on-screen interface elements like menus and buttons by speaking commands like "File" and "Open." There's also support for emulating the mouse and keyboard by giving commands such as "Press shift control left arrow 3 times."

Windows Vista speech technology includes built-in dictation capabilities (for converting the user's voice into text) and edit controls (for inserting, correcting, and manipulating text in documents). You can correct misrecognized words by redictating, choosing alternatives, or spelling. For example, "Correct Robot, Robert." Or "Spell it R, O, B, E, R as in rabbit, T as in telephone." You can also speak commands to select text, navigate inside a document, and make edits—for instance, "Select 'My name is,'" "Go after Robert," or "Capitalize Brown."

The user interface is designed to be unobtrusive, yet to keep the user in control of the speech system at all times (see Figure 3 ). You have easy access to the microphone state, which includes a sleeping mode. Text feedback tells the user what the system is doing, and provides instructions to the user. There's also a user interface used for clarifying what the user has said–when the user utters a command that can be interpreted in multiple ways, the system uses this interface to clarify what was intended. Meanwhile, ongoing use allows the underlying models to adapt continually improve accuracy over time.

Figure 3** Speech UI in Windows Vista **

To enable built-in speech functionality, from the Start Menu choose All Programs | Accessories | Accessibility and click Speech Recognition. The first time you do this, the system will step you through the tutorial, where you'll be introduced to some basic commands. You also get the option of enabling background language model adaptation, by which the system will read through your documents and e-mail in the background to adapt the language model to better match the way you express yourself. There are a variety of things the default settings enable. I recommend that you ask the system "what can I say" and then browse the topics.

But you're a developer, so why do you care about all this user experience stuff? The reason this is relevant to developers is that this is default functionality provided by the operating system. This is functionality that your applications will automatically get. The speech technology uses the Windows accessibility interfaces to discover the capabilities of each application; it then provides a spoken UI for each. If a user says the name of an accessible element, then the system will invoke the default function of that element. Hence, if you have built an accessible application, you have by default built a speech-enabled application.

Windows Vista Speech APIs

Windows Vista can automatically speech-enable any accessible application. This is fantastic news if you want to let users control your application with simple voice commands. But you may want to provide a speech-enabled user interface that is more sophisticated or tailored than the generic speech-enabled UI that Windows Vista will automatically supply.

There are numerous examples of why you might need to do this. If, for example, your user has a job that requires her to use her hands at all times. Any time she needs to hold a mouse or tap a key on the keyboard is time that her hands are removed from the job—this may compromise safety or reduce productivity. The same could be true for users who need their eyes to be looking at something other than a computer screen. Or, say your application has a very large number of functions that get lost in toolbar menus. Speech commands can flatten out deep menu structures, offering fast access to hundreds of commands. If your users ever say "that's easier said than done," they may be right.

In Windows Vista, there are two speech APIs:

• SAPI 5.3 for native applications
• The System.Speech.Recognition and System.Speech.Synthesis namespaces in WinFX

Figure 4 illustrates how each of these APIs relates to applications and the underlying recognition and synthesis engines.

Figure 4** Speech APIs in Windows Vista **

The speech recognition engine is accessed via SAPI. Even the classes in the System.Speech.Recognition namespaces wrap the functionality exposed by SAPI. (This is an implementation detail of Windows Vista that may change in future releases, but it's worth bearing in mind.) The speech synthesis engine, on the other hand, is accessed directly by the classes in System.Speech.Synthesis or, alternatively, by SAPI when used in an unmanaged application.

Both types implement the SAPI device driver interface (DDI), which is an API that makes engines interchangeable to the layers above them, much like the way device driver APIs make hardware devices interchangeable to the software that uses them. This means that developers who use SAPI or System.Speech are still free to use other engines that implement the SAPI DDI (and many do).

Notice in Figure 4 that the synthesis engine is always instantiated in the same process as the application, but the recognition engine can be instantiated in another process called SAPISVR.EXE. This provides a shared recognition engine that can be used simultaneously by multiple applications. This design has a number of benefits. First, recognizers generally require considerably more run-time resources than synthesizers, and sharing a recognizer is an effective way to reduce the overhead. Second, the shared recognizer is also used by the built-in speech functionality of Windows Vista. Therefore, apps that use the shared recognizer can benefit from the system's microphone and feedback UI. There's no additional code to write, and no new UI for the user to learn.New to SAPI 5.3

SAPI 5.3 is an incremental update to SAPI 5.1. The core mission and architecture for SAPI are unchanged. SAPI 5.3 adds performance improvements, overall enhancements to security and stability, and a variety of new functionality, including:

W3C Speech Synthesis Markup Language SAPI 5.3 supports the W3C Speech Synthesis Markup Language (SSML) version 1.0. SSML provides the ability to mark up voice characteristics, speed, volume, pitch, emphasis, and pronunciation so that a developer can make TTS sound more natural in their application.

W3C Speech Recognition Grammar Specification SAPI 5.3 adds support for the definition of context-free grammars using the W3C Speech Recognition Grammar Specification (SRGS), with these two important constraints: it does not support the use of SRGS to specify dual-tone modulated-frequency (touch-tone) grammars, and it only supports the expression of SRGS as XML—not as Augmented Backus-Naur Form (ABNF).

Semantic Interpretation SAPI 5.3 enables an SRGS grammar to be annotated with JScript® for semantic interpretation, so that a recognition result may contain not only the recognized text, but the semantic interpretation of that text. This makes it easier for apps to consume recognition results, and empower grammar authors to provide a full spectrum of semantic processing beyond what could be achieved with name-value pairs.

User-Specified "Shortcuts" in Lexicons This is the ability to add a string to the lexicon and associate it with a shortcut word. When dictating, the user can say the shortcut word and the recognizer will return the expanded string.

As an example, a developer could create a shortcut for a location so that a user could say "my address" and the actual data would be passed to the application as "123 Smith Street, Apt. 7C, Bloggsville 98765, USA". The following code sets up the lexicon shortcut:

When this code is used, the shortcut is added to the speech lexicon. Every time a user says "my address," the actual address is returned as the transcribed text.

Discovery of Engine Pronunciations SAPI 5.3 enables applications to query the Windows Vista recognition and synthesis engines for the pronunciations they use for particular words. This API will tell the application not only the pronunciation, but how that pronunciation was derived.

System.Speech.Synthesis

Let's take a look at some examples of how to use speech synthesis from a managed application. In the grand tradition of UI output examples, I'll start with an application that simply says "Hello, world," shown in Figure 5 . This example is a bare-bones console application as freshly created in Visual C#®, with three lines added. The first added line simply introduces the System.Speech.Synthesis namespace. The second declares and instantiates an instance of SpeechSynthesizer, which represents exactly what its name suggests: a speech synthesizer. The third added line is a call to SpeakText. This is all that's needed to invoke the synthesizer!

Figure 5 Saying Hello

By default, the SpeechSynthesizer class uses the synthesizer that is nominated as default in the Speech control panel. But it can use any SAPI DDI-compliant synthesizer.

The next example (see Figure 6 ) shows how this can be done, using the old Sam voice from Windows 2000 and Windows XP, and the new Anna and Microsoft® Lili voices from Windows Vista. (Note that this and all remaining System.Speech.Synthesis examples use the same code framework as the first example, and just replaces the body of Main.) This example shows three instances of the SelectVoice method using the name of the desired synthesizer. It also demonstrates the use of the Windows Vista Chinese synthesizer, Lili. Incidentally, Lili also speaks English very nicely.

Figure 6 Hearing Voices

In both of these examples, I use the synthesis API much as I would a console API: an application simply sends characters, which are rendered immediately in series. But for more sophisticated output, it's easier to think of synthesis as the equivalent of document rendering, where the input to the synthesizer is a document that contains not only the content to be rendered, but also the various effects and settings that are to be applied at specific points in the content.

Much like an XHTML document can describe the rendering style and structure to be applied to specific pieces of content on a Web page, the SpeechSynthesizer class can consume an XML document format called Speech Synthesis Markup Language (SSML). The W3C SSML recommendation ( www.w3.org/TR/speech-synthesis ) is very readable, so I'm not going to dive into describing SSML in this article. Suffice it to say, an application can simply load an SSML document directly into the synthesizer and have it rendered. Here's an example that loads and renders an SSML file:

A convenient alternative to authoring an SSML file is to use the PromptBuilder class in System.Speech.Synthesis. PromptBuilder can express almost everything an SSML document can express, and is much easier to use (see Figure 7 ). The general model for creating sophisticated synthesis is to first use a PromptBuilder to build the prompt exactly the way you want it, and then use the Synthesizer's Speak or SpeakAsync method to render it.

Figure 7 Using PromptBuilder

Figure 7 illustrates a number of powerful capabilities of the PromptBuilder. The first thing to point out is that it generates a document with a hierarchical structure. The example uses one speaking style nested within another. At the beginning of the document, I start the speaking style I want used for the entire document. Then about halfway through, I switch to a different style to provide emphasis. When I end this style, the document automatically reverts to the previous style.

The example also shows a number of other handy capabilities. The AppendAudio function causes a WAV file to be spliced into the output, with a textual equivalent to be used if the WAV file can't be found. The AppendTextWithPronunciation function allows you to specify the precise pronunciation of a word. A speech synthesis engine already knows how to pronounce most of the words in general use in a language, through a combination of an extensive lexicon and algorithms for deriving the pronunciation of unknown words. But this won't work for all words, such as some specialized terms or brand names. For example, "WinFX" would probably be pronounced as "winfeks". Instead, I use the International Phonetic Alphabet to describe "WinFX" as "wɪnɛfɛks", where the letter "ɪ" is Unicode character 0x026A (the "i" sound in the word "fish", as opposed to the "i" sound in the word "five") and the letter "ɛ" is Unicode character 0x025B (the General American "e" sound in the word "bed").

In general, a synthesis engine can distinguish between acronyms and capitalized words. But occasionally you'll find an acronym that the engine's heuristics incorrectly deduce to be a word. So you can use the AppendTextWithHint function to identify a token as an acronym. There are a variety of nuances you can introduce with the PromptBuilder. My example is more illustrative than exhaustive.

Another benefit of separating content specification from run-time rendering is that you are then free to decouple the application from the specific content it renders. You can use PromptBuilder to persist its prompt as SSML to be loaded by another part of the application, or a different application entirely. The following code writes to an SSML file with PromptBuilder:

Another way to decouple content production is to render the entire prompt to an audio file for later playback:

Whether to use SSML markup or the PromptBuilder class is probably a matter of stylistic preference. You should use whichever you feel more comfortable with.

One final note about SSML and PromptBuilder is that the capabilities of every synthesizer will be slightly different. Therefore, the specific behaviors you request with either of these mechanisms should be thought of as advisory requests that the engine will apply if it is capable of doing so.

System.Speech.Recognition

While you could use the general dictation language model in an application, you would very rapidly encounter a number of application development hurdles regarding what to do with the recognition results. For example, imagine a pizza ordering system. A user could say "I'd like a pepperoni pizza" and the result would contain this string. But it could also contain "I'd like pepper on a plaza" or a variety of similar sounding statements, depending on the nuances of the user's pronunciation or the background noise conditions. Similarly, the user could say "Mary had a little lamb" and the result would contain this, even though it's meaningless to a pizza ordering system. All of these erroneous results are useless to the application. Hence an application should always provide a grammar that describes specifically what the application is listening for.

In Figure 8 , I've started with a bare-bones Windows Forms application and added a handful of lines to achieve basic speech recognition. First, I introduce the System.Speech.Recognition namespace, and then instantiate a SpeechRecognizer object. Then I do three things in Form1_Load: build a grammar, attach an event handler to that grammar so that I can receive the SpeechRecognized events for that grammar, and then load the grammar into the recognizer. At this point, the recognizer will start listening for speech that fits the patterns defined by the grammar. When it recognizes something that fits the grammar, the grammar's SpeechRecognized event handler is invoked. The event handler itself accesses the Result object and works with the recognized text.

Figure 8 Ordering a Pizza

The System.Speech.Recognition API supports the W3C Speech Recognition Grammar Specification (SRGS), documented at www.w3.org/TR/speech-grammar . The API even provides a set of classes for creating and working with SRGS XML documents. But for most cases, SRGS is overkill, so the API also provides the GrammarBuilder class that suffices nicely for our pizza ordering system.

The GrammarBuilder lets you assemble a grammar from a set of phrases and choices. In Figure 8 I've eliminated the problem of listening for utterances I don't care about ("Mary had a little lamb"), and constrained the engine so that it can make much better choices between ambiguous sounds. It won't even consider the word "plaza" when the user mispronounces "pizza". So in a handful of lines, I've vastly increased the accuracy of the system. But there are still a couple of problems with the grammar.

The approach of exhaustively listing every possible thing a user can say is tedious, error prone, difficult to maintain, and only practically achievable for very small grammars. It is preferable to define a grammar that defines the ways in which words can be combined. Also, if the application cares about the size, toppings, and type of crust, then the developer has quite a task to parse these values out of the result string. It's much more convenient if the recognition system can identify these semantic properties in the results. This is very easy to do with System.Speech.Recognition and the Windows Vista recognition engine.

Figure 9 shows how to use the Choices class to assemble grammars where the user says something from a list of alternatives. In this code, the contents of each Choices instance are specified in the constructor as a sequence of string parameters. But you have a lot of other options for populating Choices: you can iteratively add new phrases, construct Choices from an array, add Choices to Choices to build the complex combinatorial rules that humans understand, or add GrammarBuilder instances to Choices to build increasingly flexible grammars (as demonstrated by the Permutations part of the example).

Figure 9 Using Choices to Assemble Grammars

Figure 9 also shows how to tag results with semantic values. When using GrammarBuilder, you can append Choices to the grammar, and attach a semantic value to that choice, as can be seen in the example with statements like this:

Sometimes a particular utterance will have an implied semantic value that was never uttered. For example, if the user doesn't specify a pizza size, the grammar can specify the size as "regular", as seen with statements like this:

Fetching the semantic values from the results is done by accessing RecognitionEventArgs.Result.Semantics[<name>].

Telephony Applications

This article only touches on the capabilities of the new speech technology included in Windows Vista. Here are several resources to help you learn more:

• Windows SDK
• Microsoft Speech Server
• Windows Accessibility

The following members of the Microsoft speech team have active blogs that you may find useful:

• Jay Waltmunson
• Jen Anderson
• Philipp Schmid
• Richard Sprague
• Rob Chambers

One of the biggest growth areas for speech applications is in speech-enabled telephony systems. Many of the principles are the same as with desktop speech: recognition and synthesis are still the keystone technologies, and good grammar and prompt design are critical.

There are a number of other factors necessary for the development of these applications. A telephony application needs a completely different acoustic model. It needs to be able to interface with telephony systems, and because there's no GUI, it needs to manage a spoken dialog with the user. A telephony application also needs to be scalable so that it can service as many simultaneous calls as possible without compromising performance.

Designing, tuning, deploying, and hosting a speech-enabled telephony application is a non-trivial project that the Microsoft Speech Server platform and SDK have been developed to address.

Windows Vista contains a new, more powerful desktop speech platform that is built into the OS. The intuitive UI and powerful APIs make it easy for end users and developers to tap into this technology. If you have the latest Beta build of Windows Vista, you can start playing with these new features immediately.

The Windows Vista Speech Recognition Web site, should be live by the time you're reading this. For links to other sources of information about Microsoft speech technologies, see the "Resources" sidebar.

Robert Brown is a Lead Program Manager on the Microsoft Speech & Natural Language team. Robert joined Microsoft in 1995 and has worked on VOIP and messaging technologies, Speech Server, and the speech platform in Windows Vista. Special thanks to Robert Stumberger, Rob Chambers and other members of the Microsoft speech team for their contribution to this article.

Additional resources

October 09, 2023

Share this page

Operate your PC hands-free with Speech Recognition

If you’re looking for ways to engage with your computer without using your hands, you can operate your Windows 11  PC using your voice with Speech Recognition. Learn about Windows 11’s Speech Recognition features and how to activate them on your device for hands-free access.

What is Windows 11 Speech Recognition

Speech Recognition is a powerful tool that greatly simplifies the way you use your PC. It allows you to control your computer using only your voice. Speech Recognition software makes it possible to start programs, navigate menus, write text, search the internet, and access different parts of your computer just by talking into your PC’s mic. By getting to know your specific voice, Speech Recognition can improve its capabilities and do the best job possible of following your commands.

How to set up and activate Speech Recognition

To get Speech Recognition up and running on your Windows 11 computer, there are a few steps to follow:

Set up your microphone

Make sure your microphone is set up correctly:

• Select the Windows logo key followed by the Settings icon.
• Navigate to Time & language > Speech .
• Under Microphone , select Get Started .
• The Speech wizard window will open, where you can ensure that your microphone is working properly.

Set up Speech Recognition

In this step, you will teach your device to recognize your voice:

• Select Windows logo key + Ctrl + S .
• The Set Up Speech Recognition wizard window will open.
• Select Next and follow the instructions.

Turn Speech Recognition on and off

Once Speech Recognition is set up on your computer, make sure it’s activated and learn how to quickly turn it on and off with these steps:

• Navigate to Accessibility > Speech .
• Toggle on Windows Speech Recognition .
• You can now turn Speech Recognition on or off by selecting Windows logo key + Ctrl + S .

That’s it! You’re ready to use these Speech Recognition commands  to operate your device with your voice.

Enjoy the Windows 11 Speech Recognition tool to open windows, search the web, and so much more, all hands-free. For other tips on making the most of Windows 11 head to the Windows Learning Center .

Products featured in this article

More articles

How to find and enjoy your computer's accessibility settings

Find the features to help with specific vision, hearing, or mobility needs.

Boost your productivity with a voice recorder for your PC

Turn your speech into text instantly with the voice typing feature in Windows 11.

Boost productivity with these Windows keyboard shortcuts

Take advantage of these Windows 11 keyboard shortcuts to accomplish tasks quickly.

Scott Hanselman

Speech recognition in windows vista - i'm listening.

Of course I was excited to hear that Windows Vista would include lots of new speech recognition features, and today I finally got to try them out.  I plugged in my Logitech USB headset and ran through the tutorial.

You really have to try it to fully understand the improvements that have been made to accessibility in Windows Vista.  While this entire blog post was dictated using the Built-in speech features in Vista, the dictation features, frankly aren't that impressive.  To be clear, they work, and they work well.  But it's the interface, the user experience, that's so amazing.

But these are speech-specific things, what was really interesting to me is how easy it is to interact with the entire system, the shell, without touching your mouse.  This is going to be Huge for people who CAN'T touch the mouse.

One of the most clever user interface experiences is the "show numbers" interface. When you're using Windows Vista voice recognition and you tell it to "show numbers," the current window has numbered regions overlaid on a user interface elements, so that they can be easily selected just by saying a number.

For example, notice the interface of Windows Live Writer as seen below.  Even though the default interface will click when I say - meaning if I simply say "insert picture" the system will click the Insert Picture user interface element just because it's on the screen - if there's a user interface on it like a toolbar button or something that is difficult to express verbally, I can click it easily using show numbers.

The same feature is used when selecting words that appear multiple times within a chunk of text.  For example if a paragraph contained the name 'Hanselman' four times and I said "Select Hanselman," each instance of the word would have been numbered overlaid allowing me to quickly indicate the one I meant. 

I'm not familiar with the Windows Speech API, but it'll be interesting to see how vendors like the folks at Dragon Naturally Speaking are meant to integrate their speech recognition algorithms to the existing interface experience provided by Vista out of the box.

As the one who fortunately does have the use of both my hands, I find speech to be the most valuable when I can have one hand on the keyboard, one hand on the mouse, and be speaking simultaneously.  It's certainly true that I can talk faster than I can type, and it's very very difficult to beat really good speech recognition software by just typing. 

It's worth noting that they've removed all of the speech recognition features from Office 2007 and there are a number of people who were considerably torqued about that decision.  That said, if you're into speech recognition or you use speech recognition software in your everyday life, the improvements in a speech in Vista are reason enough to upgrade your OS.

And sure, it's not perfect, but I'm using a crappy microphone in a noisy room on a slowish machine while speaking quietly so as not to wake the baby.  Not too shabby.

About Scott

Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.

Comments are closed.

Disclaimer: The opinions expressed herein are my own personal opinions and do not represent my employer's view in any way.

• Windows vNext
• Write for us
• Send news tip

Microsoft kills off Vista era Speech Recognition on the newest Windows versions

Sayan Sen Neowin · Dec 7, 2023 02:50 EST with 6 comments

Microsoft has added another feature to its long list of deprecated Windows features. Today, the company has announced that Windows speech recognition is no longer being developed. Instead, it is being replaced by Voice Access on Windows 11 23H2 and Windows 11 22H2.

On its website, the company writes:

Windows speech recognition is deprecated and is no longer being developed. This feature is being replaced with voice access. Voice access is available for Windows 11, version 22H2, or later devices.

A few days ago, Microsoft highlighted the Accessibility features it added in 2023, including Voice Access. Other such features include Live Captions, Narrator Voices, and Narrator Extensions. You can read about these in our dedicated article .

About the feature, Microsoft writes :

Voice access in Windows 11 is a new experience that enables everyone, including people with mobility disabilities, to control their PC and author text using their voice. For example, you can open and switch between apps, browse the web, and read and author emails using your voice. Voice access uses modern, on-device speech recognition to accurately recognize speech and works without an internet connection.

For those not old enough to know, Microsoft introduced Speech Recognition as a separate application back in 2006 with Windows Vista in order to make the OS more accessible. Unfortunately for Microsoft, it did not foresee threat actors abusing the feature .

Back in 2016, Microsoft hit a couple of impressive milestones in regard to speech recognition. In September of the year, Microsoft announced that it achieved the lowest word error rate , though IBM again beat that a year later in 2017 and by a fair margin too . Following that, in October of 2016, Microsoft announced that it had achieved human-like speech recognition too .

You can read about all the deprecated features Microsoft announced recently at this link .

• Windows vista
• Windows 11 23h2
• Windows 11 22h2
• Accessibility
• Deprecated features
• Speech recognition
• Voice access

Meta begins rolling out E2E encryption on Messenger chats and calls

Google shares details on how to restore Google Drive backups after a bug wiped off user data

Subscribe to our newsletter, community activity.

• new pc or gpu? in Hardware Hangout
• Acronyms.... in Jokes & Funny Stuff
• SpaceX Super Heavy and Starship updates in Science News & Discussion
• Every time you take a bath a BIOS pun happens.... in Jokes & Funny Stuff
• Will this system run Windows? in Hardware Hangout
• Concerned on External Drive that Fell Over While it was off in Hardware Hangout
• is the Asus Geforce GTX 1660 reliable ? and the drivers? in Hardware Hangout
• is there a way to export registry key but in an incremental way , many copies ? in Microsoft (Windows)
• What are you listening to? in Wall of Sound
• ROG CROSSHAIR X670E HERO BIOS 2007 Not Updating in Hardware Hangout

Software Stories

TeamViewer 15.53.6

System Informer 3.0.7578

WinToUSB 8.8

Pale Moon 33.1.0

Trending stories.

Microsoft raising Windows 11 24H2 system requirement to block CPUs without SSE4.2 and PopCnt

The new Microsoft Planner adds more task features designed for frontline workers

Dell explains how to check if your Windows 11 computer is 'AI-Enabled'

New version of Tiny11 Builder lets you debloat any Windows 11 build or version

Join the conversation.

Login or Sign Up to read and post a comment.

6 Comments - Add comment

Report comment.

Please enter your reason for reporting this comment.

Dangbei Atom, a HDR10/HLG laser projector with Dolby sound, Chromecast, Google TV and more

Forget Ubuntu 24.04 LTS, what you really want to download this month is Fedora Silverblue 40

fedora silverblue

Windows 11 22635.3500 add new account manager and more

windows 11 insider preview

Edifier NeoBuds Pro 2: Superb audio in a premium package snubbed by battery life

Windows 11 26200 adds Widgets improvements and more

Gulikit KK3 Max - premium controller with refined performance almost all the time

China set to send a new crew to its space station this week

The oukitel rt8 is a 20,000mah rugged tablet with 48 mp camera.

oukitel rt8

GameSir Nova, with hall effect sticks, 250Hz Bluetooth & A+ calibration

Edifier stax spirit s3 planar magnetic headphones — magnets how worky, how to turn off the new ads in discord, even without nitro.

Use voice typing to talk instead of type on your PC

With voice typing, you can enter text on your PC by speaking. Voice typing uses online speech recognition, which is powered by Azure Speech services.

How to start voice typing

To use voice typing, you'll need to be connected to the internet, have a working microphone, and have your cursor in a text box.

Once you turn on voice typing, it will start listening automatically. Wait for the "Listening..." alert before you start speaking.

Note:  Press Windows logo key + Alt + H to navigate through the voice typing menu with your keyboard. 

Install a voice typing language

You can use a voice typing language that's different than the one you've chosen for Windows. Here's how:

Select Start > Settings > Time & language > Language & region .

Find Preferred languages in the list and select Add a language .

Search for the language you'd like to install, then select Next .

Select Next or install any optional language features you'd like to use. These features, including speech recognition, aren't required for voice typing to work.

To see this feature's supported languages, see the list in this article.

Switch voice typing languages

To switch voice typing languages, you'll need to change the input language you use. Here's how:

Select the language switcher in the corner of your taskbar

Press Windows logo key + Spacebar on a hardware keyboard

Press the language switcher in the bottom right of the touch keyboard

Supported languages

These languages support voice typing in Windows 11:

• Chinese (Simplified, China)
• Chinese (Traditional, Hong Kong SAR)

Chinese (Traditional, Taiwan)

• Dutch (Netherlands)
• English (Australia)
• English (Canada)
• English (India)
• English (New Zealand)
• English (United Kingdom)
• English (United States)
• French (Canada)
• French (France)

Italian (Italy)

• Norwegian (Bokmål)

Portuguese (Brazil)

• Portuguese (Portugal)
• Romanian (Romania)
• Spanish (Mexico)
• Spanish (Spain)
• Swedish (Sweden)
• Tamil (India)

Voice typing commands

Use voice typing commands to quickly edit text by saying things like "delete that" or "select that".

The following list tells you what you can say. To view supported commands for other languages, change the dropdown to your desired language.

• Select your desired language
• Chinese (Traditional, Taiwan)
• Croatian (Croatia)

German (Germany)

Note:  If a word or phrase is selected, speaking any of the “delete that” commands will remove it.

Punctuation commands

Use voice typing commands to insert punctuation marks.

Use dictation to convert spoken words into text anywhere on your PC with Windows 10. Dictation uses speech recognition, which is built into Windows 10, so there's nothing you need to download and install to use it.

To start dictating, select a text field and press the Windows logo key + H to open the dictation toolbar. Then say whatever’s on your mind.  To stop dictating at any time while you're dictating, say “Stop dictation.”

If you’re using a tablet or a touchscreen, tap the microphone button on the touch keyboard to start dictating. Tap it again to stop dictation, or say "Stop dictation."

To find out more about speech recognition, read Use voice recognition in Windows  . To learn how to set up your microphone, read How to set up and test microphones in Windows .

To use dictation, your PC needs to be connected to the internet.

Dictation commands

Use dictation commands to tell you PC what to do, like “delete that” or “select the previous word.”

The following table tells you what you can say. If a word or phrase is in bold , it's an example. Replace it with similar words to get the result you want.

Dictating letters, numbers, punctuation, and symbols

You can dictate most numbers and punctuation by saying the number or punctuation character. To dictate letters and symbols, say "start spelling." Then say the symbol or letter, or use the ICAO phonetic alphabet.

To dictate an uppercase letter, say “uppercase” before the letter. For example, “uppercase A” or “uppercase alpha.” When you’re done, say “stop spelling.”

Here are the punctuation characters and symbols you can dictate.

Dictation commands are available in US English only.

You can dictate basic text, symbols, letters, and numbers in these languages:

Simplified Chinese

English (Australia, Canada, India, United Kingdom)

French (France, Canada)

Spanish (Mexico, Spain)

To dictate in other languages, Use voice recognition in Windows .

Need more help?

Want more options.

Explore subscription benefits, browse training courses, learn how to secure your device, and more.

Microsoft 365 subscription benefits

Microsoft 365 training

Microsoft security

Accessibility center

Communities help you ask and answer questions, give feedback, and hear from experts with rich knowledge.

Ask the Microsoft Community

Microsoft Tech Community

Windows Insiders

Microsoft 365 Insiders

Find solutions to common problems or get help from a support agent.

Online support

Was this information helpful?

Thank you for your feedback.

• Windows 10 PC Apps & Games
• Windows Leaks & Rumor
• Privacy Policy
• Select Theme: Light Dim Dark

Windows 11 KB5036980 turns on Start menu ads (direct download .msu)

Dell details Windows 11 AI requirements, but it’s not enough for…

Copilot on Windows 11 & 10 doubles characters limit, gets upload…

Windows 11 KB5036985 beta adds a new Microsoft account manager to…

Windows 11 Build 26200 leaks AI features: aihost.exe, screen understanding, discovery

Windows 10 KB5036979 pushes Microsoft account (direct download .msu)

Windows 10’s new feature wants you to create a Microsoft account;…

Microsoft says it accidentally added Copilot app to Windows Server 2022

Windows 10 KB5036892 out with desktop features (direct download links)

• Windows 11 24H2
• Windows 11 LTSC
• Snapdragon X Elite
• Windows 11 issues
• Windows 10 issues
• New Outlook
• Get Moment 5
• Windows 10 ISO
• Windows 11 ISO

Microsoft is killing off Windows Vista-era Speech Recognition on Windows 11

Microsoft is ending support for the Speech Recognition feature in Windows with an upcoming release of Windows 11. With Speech Recognition, you could teach the operating system to understand the sound of your voice and open apps or dictate commands, but it’s time to say goodbye to the legacy feature.

Microsoft believes the future is in the new Voice access feature of Windows 11, which supports multiple languages and is powered by AI.

With the release of Windows 11, Microsoft added many new features to the OS—a new Start menu, widgets, redesigned File Explorer and many more. While many feature additions were welcomed, users were also notified about feature deprecations. The latest addition to the deprecation list is Windows Speech Recognition.

We have now spotted that the feature is getting deprecated in Windows 11. The news comes straight from Microsoft from their Deprecated features list . Notably, WordPad and Cortana were two of the most prominent features to get axed. Other features going away include Mail & Calendar , Tips app, and Steps Recorder .

Introduced with Windows Vista, Windows Speech Recognition is an important functionality built into the OS. It allows you to interact with Windows using voice commands. It also served as an important tool for those with accessibility issues.

The feature can be set by launching the Set up Speech Recognitio n wizard and pressing the Windows + Ctrl + S keys on your keyboard together. Users should enable their Microphone by heading to Settings -> Time & language -> Microphone and clicking Get Started .

According to Microsoft, the feature will be replaced by Voice Access.

In Windows 11, you can launch Voice Access by searching for it from the Search Bar or Settings.

After turning on the Voice Access toggle, you will need download the Language packs if they are not installed. Voice Access is only available in English in the United States, United Kingdom, New Zealand, Australia and Canada.

Voice Access supports most of the features available in Windows Speech Recognition, which is currently being developed. More supported regions are to be added in future Windows updates.

Microsoft mentions that Speech Recognition will be discontinued starting December 2023 and will no longer receive updates.

While still present, we wouldn’t be surprised if Microsoft completely removes Speech Recognition in a future Windows update. If you use the feature, we recommend switching to Voice Access.

It is also worth noting that Voice access is available only for Windows 11, version 22H2, or later versions.

About The Author

Pallav Chakraborty

Pallav is a dedicated journalist and writer at Windows Latest, where he crafts thought-provoking articles that provide readers with deep insights into Microsoft and Windows. Pallav's investigative journalism has been referred by reputed publications like TechRadar over the years.

RELATED ARTICLES MORE FROM AUTHOR

Dell details windows 11 ai requirements, but it’s not enough for all features, copilot on windows 11 & 10 doubles characters limit, gets upload files feature, notebook, windows 11 kb5036985 beta adds a new microsoft account manager to start menu.

Windows 11: Control Panel is losing another feature (Font page) to Settings

Windows 10’s new feature wants you to create a Microsoft account; ditch local accounts

Windows 10 KB5034441 0x80070643 not fixed even after three months

Featured in

• Terms Of Use

• All about AI
• Google Bard
• Inflection AI Pi Chatbot
• Anthropic Claude
• Generative AI
• AI Image Generation
• AI Regulation
• AI Research
• Large Language Models (LLM)
• Surface Pro
• Surface Laptop
• Surface Book
• Surface Duo
• Surface Neo
• Surface Studio
• Surface Hub
• Surface Pen
• Surface Headphones
• Surface Earbuds
• About WinBuzzer
• Follow Us: PUSH, Feeds, Social
• Write for Us
• Cookie Policy and Privacy Policy
• Terms of Service

Windows 11 Voice Access to Replace Outdated Vista-Era Speech Recognition Technology.

Windows Speech Recognition is being retired, replaced by the more advanced and offline-capable Voice Access in Windows 11.

Microsoft has announced the deprecation of Windows Speech Recognition , indicating a strategic shift towards the more modern and sophisticated Voice Access technology in Windows 11 versions 22H2 and 23H2. The company has affirmed that the traditional speech recognition service will no longer receive updates, cementing Voice Access as the primary mode of voice-operated control and text authorship on the platform.

Voice Access Takes the Reins

Voice Access is designed to empower all users, especially those with mobility disabilities, to navigate and control their computers using voice commands. The new feature enables tasks such as opening applications, web browsing, and composing emails solely through vocal instructions. Voice Access operates on modern on-device speech recognition technology, enhancing accuracy with the compelling advantage of functioning offline, thus ensuring reliable performance without necessitating an internet connection.

A Look Back at Speech Recognition

Speech Recognition was initially introduced as a stand-alone feature in 2006 with the launch of Windows Vista , with the intention of enhancing the operating system’s accessibility. However, it faced unexpected challenges, including exploitation by malicious actors. Notably, Microsoft reported significant achievements in speech recognition technology in 2016, reaching a milestone in September by attaining a record-low word error rate, and in the following month, by matching human parity in recognition accuracy. Despite these advancements, the evolution of technology and user needs has spurred the transition to the more versatile Voice Access.

Reinforcing the Future of Accessibility

As part of its commitment to inclusive technology, Microsoft has enriched its accessibility offerings, integrating features such as Live Captions, enhanced Narrator Voices, and Narrator Extensions, alongside Voice Access. Together, these innovations reflect the company’s ongoing dedication to facilitating a user-friendly and barrier-free computing experience for a diverse user base.

The full list of features Microsoft has deprecated in favor of more advanced technologies, as well as further details on the accessibility enhancements added to Windows 11, is available for review on the company’s official website .

• Desktop Operating Systems
• Microsoft Windows
• Operating Systems
• Speech Recognition
• Voice Access
• voice commands
• Windows 11 Voice Access
• Windows Vista

Recent News

Windows Server 2025 to Feature Enhanced Hyper-V Capabilities

Adobe Premiere Pro Enhances Video Editing with Four New AI Tools

Subscribe to WinBuzzer on Google News

Microsoft moves on from 'speech recognition' on Windows 11

The Windows Vista-era speech recognition feature has been replaced by voice access.

What you need to know

• Windows speech recognition has been deprecated and will no longer be developed.
• Microsoft replaced Windows speech recognition with voice access, which first rolled out in 2022.
• Voice access allows you to control your PC with voice commands and is much more capable than Windows speech recognition.

You can say goodbye to Windows speech recognition. Microsoft announced the deprecation of the feature recently. Going forward, Windows speech recognition will not be actively developed and will be replaced by voice access.

"Windows speech recognition is deprecated and is no longer being developed. This feature is being replaced with voice access. Voice access is available for Windows 11, version 22H2, or later devices," explains Microsoft in a support document.

Speech recognition first shipped with Windows Vista to improve accessibility. Voice access is a relatively new feature that replaces Windows speech recognition while also having significantly more capabilities.

Voice access entered testing among Windows Insiders late in 2021 and rolled out to general users in 2022. The feature is available on PCs running Windows 11 version 22H2 or later.

While systems on older versions of Windows do not support voice access, speech recognition should still work. Microsoft has stopped development of the feature, but it has not removed it from older versions of Windows.

Since adding voice access to Windows 11, Microsoft has improved the feature . The company added support for more dialects, including English -UK, English – India, English – New Zealand, English – Canada, and English – Australia. Several commands have been added over time as well.

Windows 11 includes several features that improve accessibility , including voice access, live captions, eye tracking , and Windows narrator. The tech giant announced a five-year commitment to help people with disabilities back in 2021. In addition to efforts to make Windows 11 more accessible, Microsoft has an Xbox Adaptive Controller and the Seeing AI app, which recently launched for Android .

Get the Windows Central Newsletter

All the latest news, reviews, and guides for Windows and Xbox diehards.

Sean Endicott brings nearly a decade of experience covering Microsoft and Windows news to Windows Central. He joined our team in 2017 as an app reviewer and now heads up our day-to-day news coverage. If you have a news tip or an app to review, hit him up at  [email protected] .

• 2 Why THIS Kickstarter-backed game is the must-play Xbox Game Pass title this month
• 3 From new Xbox games to AR glasses, here are my favorite things I saw at my very first GDC
• 4 How to turn down brightness on Windows 11
• 5 Microsoft News Roundup: Xbox takes over PlayStation Store, Fallout tops charts, and Microsoft's project Stargate

This blog post is older than a year. The information provided below may be outdated.

Issue regarding Windows Vista Speech Recognition

Hey everyone this is Adrian and I am writing to try and clear up some concerns regarding a recently reported vulnerability in the Speech Recognition feature of Windows Vista. An issue has been identified publicly where an attacker could use the speech recognition capability of Windows Vista to cause the system to take undesired actions. While it is technically possible, there are some things that should be considered when trying to determine what the threat of exposure is to your Windows Vista system.

In order for the attack to be successful, the targeted system would need to have the speech recognition feature previously activated and configured. Additionally the system would need to have speakers and a microphone installed and turned on. The exploit scenario would involve the speech recognition feature picking up commands through the microphone such as “copy”, “delete”, ”shutdown”, etc. and acting on them. These commands would be coming from an audio file that is being played through the speakers. Of course this would be heard and the actions taken would be visible to the user if they were in front of the PC during the attempted exploitation. It is not possible through the use of voice commands to get the system to perform privileged functions such as creating a user without being prompted by UAC for Administrator credentials. The UAC prompt cannot be manipulated by voice commands by default. There are also additional barriers that would make an attack difficult including speaker and microphone placement, microphone feedback, and the clarity of the dictation.

You may ask why this is new to Windows Vista as previous versions of the operating system do not appear affected. Windows Vista’s sophisticated speech recognition allows for easier operation and extended support for commands. This has been largely used to help facilitate computing use especially for users that are affected by dexterity difficulties or impairments. You can learn more about Windows Vista’s accessibility tools including speech recognition by going to http://www.microsoft.com/industry/healthcare/providers/businessvalue/housecalls/accessibletech.mspx .

While we are taking the reports seriously and investigating them accordingly I am confident in saying that there is little if any need to worry about the effects of this issue on your new Windows Vista installation.

*This posting is provided “AS IS” with no warranties, and confers no rights.*

How satisfied are you with the MSRC Blog?

Your detailed feedback helps us improve your experience. Please enter between 10 and 2,000 characters.

Thank you for your feedback!

We'll review your input and work on improving the site.