I got it to work after clearing data and resetting permissions for WhatsApp
After doing that you have to link Brave with Whatsapp again.
What I noticed is that if I update Whatsapp it stops working (only on Brave, on Chrome it works perfectly fine after updating)