Whoa, this was driving me nuts, so glad I got it sorted.

When I used Remote Desktop on my Mac’s Parallel’s VM running Windows 10, every time I typed the pipe key (|) I got double quotes (“) and every time I typed a backslash (\) I got a single tick (‘) and my quote (“) key gave me a tilde (~).

Made it very hard to work in putty while shelled into a remote machine on the remote machine because I couldn’t type | pipe character which is kinda important for Linux. Anyway, I found the answer on the Parallels forum from a guy in Italy who was having a similar problem. 

I had to edit the keyboard settings on the version of Windows running in Parallels (not on the Remote Desktop machine) to use just US QWERTY instead of US (Apple). Logged out, and everything is working!