So I switched to the new Brave Wallet after Brave decided to enforce it and prevent reverting to the old Metamask fork.
Their official page about it claims
Extensions slow down your device. Since Brave Wallet is built directly into the browser, it takes no extra processes to run, and burns less CPU & memory on your device.
But after a little while using it, I have to admit I can’t say where the performance improvement is when the Wallet button on the Toolbar is often unresponsive or when it takes up to a minute for transaction windows to show up.