32

I'm configuring an OpenVPN (version 2.3.10) server on a Windows 2012 server but I cannot make it to work.

The server is behind a router and I opened the 1194 port and created a rule to forward traffic on this port to the server.

Here is the log I see on the server when I try to connect from a client:

Mon Mar 21 11:11:47 2016 XX.XX.XX.XX:57804 TLS: Initial packet from [AF_INET]XX.XX.XX.XX:57804, sid=fdf7a7ac 0264c7f3
Mon Mar 21 11:12:38 2016 XX.XX.XX.XX:55938 TLS: Initial packet from [AF_INET]XX.XX.XX.XX:55938, sid=1f242a3f e454a525
Mon Mar 21 11:12:48 2016 XX.XX.XX.XX:57804 TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
Mon Mar 21 11:12:48 2016 XX.XX.XX.XX:57804 TLS Error: TLS handshake failed
Mon Mar 21 11:12:48 2016 XX.XX.XX.XX:57804 SIGUSR1[soft,tls-error] received, client-instance restarting

Where XX.XX.XX.XX is the ip of the client. So I understand from this that the client at least is able to arrive at the server, so there's no routing or firewall issues.

I followed the description provided here Easy Windows Guide Any ideas?

MadHatter
  • 80,590
vmasanas
  • 443
  • 1
  • 4
  • 5
  • 2
    Assuming the two lots of XX.XX.XX.XX represent the same address (please consider not obfuscating such things), I'm interested by the change in source port numbers (57804, 55938). That suggests to me that there's an unreliable NAT in the way, which is most often the case for UDP. Are you using UDP or TCP transport, and if the former, can you try the latter and see if the problem goes away? – MadHatter Mar 23 '16 at 07:35
  • give I see this message on the server console I undertand that at least the client can get there, so I was assuming NAT was not the issue. Am I wrong here? – vmasanas Mar 23 '16 at 09:02
  • 2
    Not all NAT is created equal. Some have very short-lived state table entries, particularly for UDP, and OpenVPN won't take well to changes in the the source port. Please answer the question I asked, and if appropriate, try the change I suggested. – MadHatter Mar 23 '16 at 09:05
  • I'm not that experienced here, so can you tell me how to check whether I'm using UDP or TCP? – vmasanas Mar 23 '16 at 09:14
  • Well, you could try man openvpn and look for something that controls protocol. Don't forget to change it on both client and server, if you do decide to do the test. – MadHatter Mar 23 '16 at 09:16
  • Hey, that was the problem! I was using UDP and changing to TPC just solved the problem. Thanks a lot. – vmasanas Mar 23 '16 at 10:25

12 Answers12

34

What's interesting is how the port number changes mid-stream:

Mon Mar 21 11:11:47 2016 XX.XX.XX.XX:57804 TLS: Initial packet from [AF_INET]XX.XX.XX.XX:57804, sid=fdf7a7ac 0264c7f3

Mon Mar 21 11:12:38 2016 XX.XX.XX.XX:55938 TLS: Initial packet from [AF_INET]XX.XX.XX.XX:55938, sid=1f242a3f e454a525

This makes me think that, somewhere between client and server, there is a misbehaving NAT device, a device with very short-lived state table entries, which is changing the source port number that it applies to the client's established stream, causing the server to think that two short-lived communications are in progress, instead of one continuous one.

Such devices generally only do this with UDP, so I have advised you to confirm that you are using UDP, and try TCP instead. This you have done, and found that it fixes the problem. The next step is to identify the misbehaving NAT device, hit it with a club hammer, and replace it with one that doesn't make the cardinal mistake of assuming that all UDP communications are ephemeral; but you have indicated that you're happy with changing to TCP as a workaround, and so the matter is concluded.

MadHatter
  • 80,590
  • Is there any benefit on using UDP instead of TCP? TCP is working for me now to unless there's any downside I'm ok with it. – vmasanas Mar 23 '16 at 10:57
  • @vmasanas encapsulating TCP in TCP is always a bit grim on lossy links, because the retransmissions step on each other's feet. UDP is best if you can work out what's going wrong - but since this is likely to be client-side, it may prove very tricky. – MadHatter Mar 23 '16 at 11:38
  • didn't work for me. i set both sides to tcp and the port is still changing in the server tcpdump. – Inanc Gumus Jun 30 '17 at 06:54
  • @inanc I am very sorry to hear that you have a particularly useless NAT device in your traffic flow. You will need to sort that out before you can do anything that involves long-lived network state. – MadHatter Jun 30 '17 at 15:33
  • @MadHatter np. the same config was working with udp in my home network. now, I'm in vacation, and here, it doesn't work, no matter with udp or tcp. – Inanc Gumus Jul 01 '17 at 15:45
  • switching to tcp fixed it for me thx – Joseph Khella Jun 18 '22 at 13:51
7

This is one of the most common error in setting up Openvpn and there is a FAQ entry for this. I'm going to quote this here:

TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)

One of the most common problems in setting up OpenVPN is that the two OpenVPN daemons on either side of the connection are unable to establish a TCP or UDP connection with each other.

This is almost a result of:

  • A perimeter firewall on the server's network is filtering out incoming OpenVPN packets (by default OpenVPN uses UDP or TCP port number 1194).
  • A software firewall running on the OpenVPN server machine itself is filtering incoming connections on port 1194. Be aware that many OSes will block incoming connections by default, unless configured otherwise.
  • A NAT gateway on the server's network does not have a port forward rule for TCP/UDP 1194 to the internal address of the OpenVPN server machine.
  • The OpenVPN client config does not have the correct server address in its config file. The remote directive in the client config file must point to either the server itself or the public IP address of the server network's gateway.
  • Another possible cause is that the windows firewall is blocking access for the openvpn.exe binary. You may need to whitelist (add it to the "Exceptions" list) it for OpenVPN to work.

It's highly likely that any of these is causing the same problem in your case too. So just go through the list one by one to resolve it.

Ref: TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)

MadHatter
  • 80,590
Diamond
  • 9,131
  • I've gone through these points but not sure if I'm missing something: 1.for the moment firewalls are off in both client and server, 2. same, 3.the router have a forwarding rule to the server and I see the traffic appearing on the server console, 4.client has correct ip address, 5. no firewall that I can tell. – vmasanas Mar 23 '16 at 09:11
  • Well, honestly I can't think of anything else at the moment. To be sure, how is the network configuration in windows server? Does it have multiple gateway by chance? Is it's default gateway pointed to the router? If yes, then the rest you can test is, as MadHatter suggested, to test with tcp. – Diamond Mar 23 '16 at 09:33
  • If anyone feels like lending a hand, I've posted (yet another) TLS handshake failure question here: https://serverfault.com/questions/867599/openvpn-tls-handshake-failing-what-else-could-it-be – Ola Tuvesson Aug 08 '17 at 19:56
  • 1
    One other thing to look out for is high latency between the two hosts. I was just scratching my head on this extensively, and for some godforsaken reason I was getting 6000ms+ ping round trips in one direction (client to server), but not the other way around. This was causing key negotiation to take more than 60s. Rebooting the router provided by my ISP solved the issue. Probably a rare edge case, but hopefully that helps someone out. – s.co.tt Apr 25 '19 at 14:09
  • @s.co.tt This did help me out. I wish I tried out your trick earlier. – sofname Aug 16 '23 at 00:27
4

I was getting TLS key negotiation timeouts like this. But in my case I realised that the remote link was a local IP address.

The VPN on our pfSense firewall had mistakenly been put on the LAN interface instead of the WAN interface, and so the exported config was set to try and connect to the firewall's LAN IP address - which was never going to work with the client naturally being on a different LAN.

I think the main takeaways from this are:

  • Getting a key negotiation timeout does not necessarily mean you've even managed to connect to anything.

    So at this stage it may still be worth checking you're actually connecting to the right place, and there are no firewall rules blocking the connection, etc. Particularly if your configuration has been automatically generated.

    Note that getting a login prompt does not mean that you're connected, since OpenVPN asks for your credentials before trying to connect.

  • Make sure your VPN server is listening on the right interface.

    (Of course, this is one of a number of server-side misconfigurations that could occur, such as firewall rules, putting the wrong port number, intermixing TCP and UDP, etc.)

mwfearnley
  • 860
  • As you commented in the last topic. I once got this problem because the client was set up for UDP, but the server required TCP. So this is yet another root cause for the staggeringly long list of possible problems. – brandizzi Sep 27 '23 at 20:18
3

I had the same error and no advice helped, everything seemed to be fine: IPs, ports, firewall, everything. Gone insane for 2 hours.

Solution was to change the protocol from UDP to TCP in the client config (apparently I disabled UDP on purpose a long while ago).

Hope this helps someone :)

LE: this solved my problem but it's not the best approach as per below comments. You should use UDP instead of TCP. It helped me because I had different settings between the client and the server configs.

bosch
  • 185
  • You should know that encapsulating TCP inside of TCP is very likely to cause very bad performance issues, due to both TCP stacks trying to compete against each other. – EEAA Jun 01 '17 at 20:19
  • Indeed, it works like crap. Although I don't understand what you said, the performance is very poor. Should I switch to UDP then? – bosch Jun 01 '17 at 20:20
  • 2
    Yes. UDP is the standard for VPN implementations, for good reason. TCP has error-correction functionality - the ability to re-transmit lost packets, etc. If you encapsulate TCP inside of TCP, and you incur packet loss, both TCP stacks (the "inside" traffic as well as the encrypted OpenVPN traffic) will try and correct for the lost packets. When you encapsulate TCP in UDP, this isn't a problem - only the inside traffic will re-try. – EEAA Jun 01 '17 at 20:23
  • Thanks for the hint and for the explanations. I switched to UDP and keep an eye on the connections. Also, I need to read some more about the stacks... – bosch Jun 01 '17 at 20:30
  • A helpful page explaining why TCP over TCP is a bad idea: http://sites.inka.de/bigred/devel/tcp-tcp.html – mwfearnley Aug 08 '18 at 09:07
2

None of the solutions mentioned earlier worked. In my case, even though the client log showed same error TLS Error: TLS key negotiation failed to occur within 60 seconds, the server logs showed VERIFY ERROR: depth=0, error=CRL has expired.

On the server, following steps resolved the connection issue:

# cd <easyrsa folder>
# ./easyrsa gen-crl
above command generates new crl.pem file (in my case in pki folder)
using chown/chmod make sure 'pki/crl.pem' is readable by openvpn server (for example: chmod 640 pki/crl.pem)
# systemctl restart openvpn
mpprdev
  • 151
2

I also came across the TLS key negotiation failed to occur within 60 seconds problem.

From the official suggestion, as Diamant post, there must be something wrong in the network connection. However, neither the firewall nor the NAT cause the problem.

In my case, I first checked the connection by nc -uvz xxx.xxx.xxx.xxx 1194. The link is OK.

Besides, several other vpn clients within the same LAN work fine.

From somewhere I noticed that udp connection has some problems in response or port forward.

So I stop the running vpn clients from the largest ip to the hanging client, e.g, from "10.8.0.100" to "10.8.0.50".

Then start the stopped vpn clients in reverse.

Bang! All the vpn clients work propoerly.

In conclusion, there is a chance leads to TLS key negotiation failed to occur within 60 seconds problem that multiple vpn clients within a LAN starting in a wrong sequence.

samson.wang
  • 21
1

There seems to be a lot of different causes for the error - I was seeing this on the server for one client, but successfully connecting with another (the latter client being an android device using the OpenVPN Connect App).

What it turned out to be in my case is that I'd neglected to include a CommonName value when creating the server certificate - the app was ignoring this mistake but the other clients (OpenVPN plugin for Network Manager and pfSense) were validating this and refusing to continue the connection. This could be found within the client logs, but all that was visible on the server-side logs was:

TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
TLS Error: TLS handshake failed
Adam Luchjenbroers
  • 238
1

Note that you can get a TLS key negotiation error, without successfully connecting to the OpenVPN server - or even successfully connecting to anything at all!

I modified a VPN config to connect to localhost, on a port that wasn't listening on anything:

OpenVPN 2.4.6 x86_64-w64-mingw32 [SSL (OpenSSL)] [LZO] [LZ4] [PKCS11] [AEAD] built on Apr 26 2018
Windows version 6.2 (Windows 8 or greater) 64bit
library versions: OpenSSL 1.1.0h  27 Mar 2018, LZO 2.10
TCP/UDP: Preserving recently used remote address: [AF_INET]127.0.0.1:12345
UDP link local (bound): [AF_INET][undef]:0
UDP link remote: [AF_INET]127.0.0.1:12345
TLS Error: TLS key negotiation failed to occur within 60 seconds (check your network connectivity)
TLS Error: TLS handshake failed
SIGUSR1[soft,tls-error] received, process restarting
...

The error can lull you into a false sense that you're talking to a VPN server.

You may even get prompted for credentials first, but nothing outside your computer has actually asked for them.

mwfearnley
  • 860
  • Note: this error doesn't mean you're not talking to an OpenVPN server. The server may not be replying to you because it doesn't like your attempts to authenticate - e.g. there's a problem with your certificate. Be sure to check the server logs. – mwfearnley Feb 22 '23 at 16:14
1

I ran into this error in AWS, where OpenVPN was installed on a server with a public IP, but on an instance which was in a private subnet, i.e. a subnet which didn't have a route to an internet gateway.

Once I deployed OpenVPN on a server within a public subnet, it all worked nicely :)

On public/private subnets in AWS: https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html

Zoltán
  • 217
0

One possible reason is if the server requires TLS version newer then the TLS supported by the client. i.e 1.2 vs 1.0.

The obvious thing to try is to update the OpenVPN client, or modify the server side to accept TLS 1.0.

kenlukas
  • 3,206
ozk
  • 1
0

You should create a SSL/TLS certificate on OMV and then enable secure connection SSL/TLS and add the created certificate. So simple!

user576785
  • 1
0

Are there more than two NetCards on your OVPN-Server?

In UDP, the source IP is not checked when responding. OVPN-Server will use the default configuration to send UDP data, so we need to specify the NetCard in server.conf.

local the-IP-in-client.ovpn

Gerald Schneider
  • 25,025
  • 8
  • 61
  • 90
eulermate
  • 1
  • 1
  • 1