Brave has accessibility issues when I try to use Dragon NaturallySpeaking to control my computer by voice

[Mike.Antonacci]

Description of the issue: Most browsers allow me to directly dictate text using Dragon NaturallySpeaking, but Brave does not. Additionally, Brave is much less likely to allow me to click items on a page such as saying “click Listen Podcast” to click the button whose text reads Listen Podcast. Since I must control my computer by voice due to issues with my hands, this makes Brave almost unworkable for me, I’m at least 50% slower due to this. I’m hoping it’s due to a setting that I can change easily, I really would like to continue using brave but right now it’s not very viable.

I’ve included a screencast I made showing several of these issues in real time, comparing Firefox (expected behavior) with Brave (actual behavior). I didn’t see an easy way to directly upload it to this forum, so here’s a link to it in Google Drive: https://drive.google.com/file/d/1c9ggMKrc_1UukRMP4cg7JhAtdh02JYvW/view?usp=sharing

How can this issue be reproduced?

1. Open Dragon NaturallySpeaking and Brave
2. Navigate to any website that lets you input text, such as a form to fill out with your name and contact info
3. Speak the text you want to input into the website

Expected result:
I am able to input the text directly into the website rather than through Dragon’s dictation box and then using the command “click Transfer” to copy/paste the text into the website.

Second issue:

1. Open Dragon NaturallySpeaking and Brave
2. Navigate to a website that has checkboxes that can be clicked by voice, such as Habitica
3. Speak the text of the checkbox so that it clicks. Expected result: saying the full text of the checkbox or the partial text of the checkbox both click the checkbox. Actual result: speaking partial text of the checkbox does not work.

Third issue:

1. Open Dragon NaturallySpeaking and Brave
2. Navigate to a website that has buttons with text on them, including folder names in Gmail.
3. Speak “click [button name]” to click the button whose text reads [button name]. Expected behavior is that you click the button. Actual behavior is that Brave does not click the button, instead interpreting it as dictation (i.e. typing text) instead of a command to click

Fourth issue:

1. Open Dragon NaturallySpeaking and Brave
2. Use command “go to address bar” to select the address bar
3. Say the URL you want to navigate to. Expected result is it types directly in the address bar. Actual result is the dictation box opens, and when I transfer the text to the address bar, it does not paste over the text that had been in the address bar.

Fifth issue:

1. Open Dragon NaturallySpeaking and Brave
2. Use command “Click LastPass” to open it
3. Say the name of the website saved into Last Pass so that Last Pass searches for it. Expected behavior is the name of the website goes into the search field and Last Pass automatically filters/searches. Actual behavior is the dictation box opens, and when you say “click transfer”, it instead transfers to the address bar in Brave

Brave Version( check About Brave):
Version 1.27.109 Chromium: 92.0.4515.115 (Official Build) (64-bit)

Additional Information:
Shields up/down has no change on the behavior.
Firefox, Chrome, and Edge all work with expected behavior, but Brave does not.

[Mattches]

@Mike.Antonacci,
Thank you for reporting and for the well written/detailed information. Really appreciate it. I’m reaching out to the team about this now.

[sampson]

Update:

Please go to brave://accessibility/ via your address bar, and enable the following:

• Native Accessibility API support
• Web accessibility
• HTML

Upon enabling these, please restart Brave and let us know if this improves your experience.

–

First off, what a top-notch report, @Mike.Antonacci I assume you’ve assisted projects in the past with understanding some of these a11y issues; I’m blown away at how well you’ve put together this feedback. Thank you!

Second, Dragon Naturally Speaking is really impressive. I first used it back around 2003, but not much since then. What a11y I have done over the years was largely around screen-readers, and making sure they were able to track context and more on pages. Out of curiosity, how much time did you have to spend training the software on your voice? If I recall correctly, back when I first used it, it was only a few minutes of reading a short story—curious if that has changed any.

Regarding the issues you’ve captured here, is this something we would be able to reproduce without a paid account? If not, would you be willing and able to join one of us over Zoom for a remote-troubleshooting call? I’m curious if Dragon leverages an in-browser extension, or if it operates exclusively from a parallel program running on the same machine.

Because the software works better in Chrome than in Brave, could you provide screenshots of the about://version page from both browsers? I’m primarily interested in the “Command Line” portion, which looks like this:

Lastly, I was curious about Dragon’s own settings. I noticed when you were dictating input into the pop-up box, that Dragon was configured to Use automatic Dictation Box for this application. There was also a icon at the top-right; can you explain these settings, and share what other settings exist for per-application interoperability, please?

Again, if these are questions we can answer on our own by using a demo of the app, please let me know. I’m more than happy to download the software and work towards making routine testing a part of our regular process.

Thank you again for the excellent report; I hope to see this matter resolved for you soon!

[sampson]

@Mike.Antonacci, I’m also curious if you’ve attempted to use Brave Beta or Brave Nightly builds of the browser as well, and if they behaved any differently for you.

[Mike.Antonacci]

Thanks for the fast and thorough response! And I appreciate your kind words about me documenting it, I figure I should give you as much info as I can about what I tried and what worked and what didn’t, because if I don’t know how to solve it, the more info you have, the easier it will be for you

brave://accessibility/ already had the first two options checked, so I enabled HTML. Unfortunately, no change in behavior. Attempted with Beta and Nightly, same behavior as before. I also enabled the screen reader checkbox, no change in behavior.

Dragon does offer an in-browser extension for Chrome (which I in turn installed for Brave) but I’m not exactly sure what it does. It supposedly offers an extension for Firefox but all of the times I’ve tried to download it, it says the file is corrupted, so I’ve never used the Firefox extension. I also tried disabling the extension in Brave, but the behavior still the same.

For the dictation box, I tried unchecking the “use automatic dictation box”, it solves some issues but not others. Without the dictation box, it assumes that anything I’m saying is dictated text, which is fine if I have my cursor in a place where I want to dictate text, but doesn’t work if I’m trying to say a command instead of dictate text

• Solved: dictating text into the address bar or into LastPass
• Not solved: clicking buttons by saying “click [button name]”. Another thing I tried: There’s a setting within Dragon that can force it to recognize something as dictation vs command, holding down the shift key forces it to interpret as dictation and holding down control key forces it to interpret as command; I held down the control key and it gave the error “please say that again” which means it couldn’t find a way to execute the command, which I interpret it to mean that it couldn’t find the button labeled [button name].

As best I can tell, Dragon doesn’t offer any sort of free trial, so that’s unfortunate. I’d be open to some sort of video call/screen share, I’m not really available next week due to my work schedule, but week of August 9 is fairly open at the moment.

Dictation box settings look like this, it seems that they are primarily for what to do with the copy/paste commands.

Within the options/settings for Dragon itself, this was the only page that looks relevant to me. Clicking on “more commands” in the bottom right had options for commands native to Microsoft Word, Excel, Facebook, and Twitter, so it didn’t look relevant and I didn’t take a screenshot.

About: version for both chrome and brave
Chrome

Brave

I didn’t spend a ton of time training Dragon. I think I had to read a couple paragraphs initially, had it scanned through my saved documents and emails to get a sense of my vocabulary, and then it continually learns as I go. For example, since Jon is more common than John in my usage, it defaults to the former.

[sampson]

Thank you for the prompt and detailed follow-up. If you don’t mind, let’s try one additional thing. Please close Brave entirely, and then open the Run Dialog on Windows (Windows Key + R, or Windows Key and search Run). In the input, type and submit brave --force-renderer-accessibility.

That will launch a new Brave window; please let me know if there’s any improvement with that instance.

If these measures don’t work, it may be that Dragon NaturallySpeaking has a hard-coded limitation. The Chromium project talks about how Chrome is detected by these assistive technologies, and it could involve code that looks for Chrome and Firefox specifically. If this is the case, we may need to make changes in Brave (to mimic Chrome).

[Mike.Antonacci]

No improvement with that instance (and for whatever reason, using that command forced Brave Nightly to open, not the release version).

In case it’s helpful for you, I looked more on Dragon’s website regarding the browser extension and what it does, “Without the extension installed, you’ll need to use your mouse and keyboard to click Web page objects. You might also need to use the Dictation Box to dictate text into fields and Web applications.”

https://www.nuance.com/products/help/dragon/dragon-for-pc/enx/professionalgroup/main/Content/Web/about_the_dragon_web_extension.htm

https://www.nuance.com/products/help/dragon/dragon-for-pc/enx/professionalgroup/main/Content/Web/working_with_chrome.htm

It looks like that as of right now, Dragon only has a browser extension for Chrome and Internet Explorer, but not Firefox (so maybe Firefox already has the functionality that the browser extension enables and therefore doesn’t need an extension)

[sampson]

If there is an extension for Chrome, it should also work in Brave. Brave supports the same extension architecture as Chrome, which makes nearly every Chrome extension is also a Brave extension. If you find that the extension still doesn’t deliver a similar experience to Chrome, the next best step would be for us to schedule a remote troubleshooting call and take a closer look. I’m happy to accommodate your schedule

[Mike.Antonacci]

Sent you a message to set up a time!

[Mike.Antonacci]

@sampson Is there anything I should be doing for my end to set up a time with you? I’m unsure whether I should do that in this thread, or message you directly, or something else entirely