When in Tablet Mode for Windows 11 and using vertical tabs, the X for closing the tab only displays on the tab that you were on prior to switching regardless of what tab is currently selected.
Another small thing would be that the icons for vertical tabs are hidden in tablet mode compared to the vertical tabs on desktop.
(Desktop Mode screenshot shown below)