I've noticed a strange thing: the PowerShell remoting works much faster if you use the remote computer's IP address for connection rather than a DNS name. Well, not everything is faster but the re-connections. PowerShell uses the HTTP (or HTTPS) protocol to send the data. This connection is cached between the successive commands but if you don't enter anything for a few minutes, this connection gets dropped and then re-established when you enter the next command. This re-establishment, and also the initial connection, works much faster with the IP address than with the DNS name. The effect is most pronounced if connecting over a WAN, such as to a VM in Azure. Don't know why. There shouldn't be that much delay with the DNS. It's a mystery.
Of course, this won't work with HTTPS connections, HTTPS connections require that the machine name matches the name in the certificate.
P.S. Here is the real answer: this happens if the user is not domain-joined, and the user name was specified without an explicit domain, like you would typically do wne connecting to an Azure machine. The slowness happens in an attempt to use the domain of the machine you're connecting from. The workaround is to use the explicit domain or more exactly non-domain that says that the user belongs to the remote machine, i.e. instead of "myuser" use ".\myuser".