I recommend moving to VSCodium and using this extension:
https://github.com/jeanp413/open-remote-ssh
and run vscode-server inside a VM or container
Would that really help, if you install the extension it's running in the VSCodium instance on your machine... or is it running in the vscode-server?
loading story #50303876