After a bit of digging, and for added context, turns out I forgot that I enabled “MFAA” setting in Nvidia control panel because I was testing some things:
As soon as I disabled it, hardware acceleration works normally now with the default settings!
I probed into this a little more because my Visual Studio Code instance was having issues as well with HWA.