@BiigYoshii Most likely you have it turned on in your YouTube settings. I’m not sure if this link will take you to it, but check out https://www.youtube.com/account_playback or otherwise YouTube → Settings → Playback & Performance
You’ll see checkboxes that talk about captions. You’ll want to disable those.
I could be wrong, but I’m assuming that’s your issue. If so, then your settings there prioritize captions and would try to show them on each video. So your toggling temporarily does it, but until you toggle the global default…
Then again, hard to say. I know with the auto-generated captions unchecked, mine rarely shows anything. If yours is already off, then might need to try to clear your cookies/site settings from the browser and give it another shot.