it would be cool if brave remember the choice you make and not have to click on “remember decision” all the time
Is it on desktop?
First, let check something. Did you have some or all settings enabled under Preferences/Settings > Security > Private Data? Enable Saved site settings and permissions under Private Data will clear those type of data (permissions like location access, etc) every time you close Brave.