Brave Leo answer corrections

Hello! I just saw Brave Leo’s announcement and rushed to try it out (like I’m sure many people did, given that it seems to be down for me :sweat_smile:)

First off, I think Brave Leo will be a great feature if it works properly! I hope it scales well; it’s good to have more competition for ChatGPT and Microsoft Bing.

I enjoy breaking AI language models, so if I find any way to break the model, I’ll post it here. I would also be happy to give feedback directly in Brave (like how ChatGPT has the :+1::-1:), and I would be more than happy to give the fine folks working at Brave access to (most of) my conversation history as sample data - whatever I can do to help make Leo more helpful to others!

Here are some examples I found so far of where the AI gave unexpected answers:


Leo seems to be down for me for now, but when it’s back up, I’ll enjoy breaking testing Brave Leo out!

I see that Leo is based on Lambda Llama with no extra fine-tuning, so hopefully, I can help provide examples of where the AI model may need to be improved.

I look forward to seeing where Leo goes!

No it’s not. It’s from Meta AI. You may want to read the official notice at

The model behind Leo is Llama 2, a source-available large language model released by Meta with a special focus on safety.

Umm, not down for me. What are you seeing that you mention it was down?

Yeah, it’ll be getting there. Keep in mind it’s only in Nightly that it’s available. Nightly is always the testing and development version. So it’s generally not going to be anywhere near perfect and things will change.

And yeah, I tested the time one. My answer is slightly different. I tried plugging in a state just in case that mattered, but no go.

1 Like


First off, I’m sorry that my last post was so negative! I updated it to be less so.

Sorry about getting the model wrong - you’re right. With so many AI models to keep track of, I guess I got them confused! :sweat_smile: I had seen, but misread it. I’ll update my original post to match that.

When it was down, I was getting a bunch of messages like this for any query I gave it:

It seems to be back up now. It was funny, however; I was sending some messages correcting it where it left off a question mark, then I started typing in internet slang to see how it responded. Ironically, it stopped working right after I told it “Lol get rekt”

I also get that it’s a nightly release with lots of work coming up; it doesn’t yet have access to the internet, for example, but come will eventually thru Brave Search.

I’m sorry if my last post came off as argumentative, negative, or anything like that. I read it back, and I probably updated it to make it more friendly. I enjoy playing with and breaking ChatGPT and working with things like Lakera’s Gandalf, so I’ll enjoy having a new toy to play with. I’m a dev, I wrote my senior thesis on ChatGPT, and I enjoy trying to push systems to their limits to try to break them!

I really hope this works well. I trust Brave as my browser and my search engine and I recommend it to everyone. I also know that AI takes a lot of processing power, so it can get costly fast, especially as this scales up.

I’m really positive and hopeful for Leo’s future, and I look forward to seeing where it goes! :grinning:

Have a great day!


Ah, I see now @2br-2b. It’s doing to me what you saw yesterday.

@rebron2000 and @Mattches do you guys know if this is going to be added to anything for us to know if it’s ever down?

We’ll get it added to when we get to release. We also have a little bit clearer messaging on the error messages. We’ll need to see how we improve those errors messages as well.

1 Like

Here’s some of the conversations I’ve had with Leo. It seems to be a bit confused, but I’m sure it’ll improve over time! :sweat_smile:



(edited to add space between conversations)

Well, as I was thinking thru and testing out some responses for my next post, you went ahead and released a roadmap! I was coming up with a short list of improvements, and you already covered almost all of them!

Between the time Leo was first released and now, I’ve noticed it becoming significantly more accurate, especially with technical questions. Great job with the rapid improvements and amazing roadmap!

Here are the features I was going to ask about but which you have already announced:

  • Having Brave Leo on its own page rather than only in a sidebar

  • Using Brave’s sync capabilities to sync a chat history

  • The ability to customize a system prompt

  • More clear messages if you hit a rate limit

And honestly? Those were the major additions I wanted! You are knocking it out of the park with this!

The only other changes I had wanted were small ones:

  • Since Leo speaks using Markdown formatting (for example, code boxes ````), it would be nice if the page supported that

  • The ability to edit one of your previous messages or refresh the AI’s output if it’s incorrect. You mentioned the ability to resume a past conversation, so I would think there’d be a way to change the past conversation, so this is sort of already covered

  • When I enter a prompt, it often unselects the input box at the bottom after my message. It could be nice if the box remained selected to allow for more rapid responses

Now that you also said you’re looking into adding support for local models, that could be amazing! I’m sorry that my original post sounded so negative, and I look forward to seeing where Leo goes!

Here are a few random conversations I had: