VPN Troubleshooting and Configuration

What is a Network?

What is Encryption?

What is a VPN?

Introduction

A VPN connection is created in two phases:

In Phase 1 (ISAKMP SA; SA = Security Association) the VPN gateways (peers) authenticate each other using a PSK (PSK = Pre-Shared Key) and then create and exchange a new encryption key to encrypt all network traffic passed between them. This SA is now established and is only used to exchange new keys and DPD (Dead Peer Detection) messages.
- ISAKMP SA (Phase 1) exchanges are denoted by STATE_MAIN_X# entries.
VPN gateways only proceed with creating Phase 2 (IPSec SA) if Phase 1 is established successfully. In Phase 2, IPSec connection configurations are exchanged and negotiated. This SA connects the two gateways and is used for data exchange between the clients on each of these networks.
- IPSec SA (Phase 2) exchanges are denoted by Quick Mode entries.

Most frequently establishing a VPN tunnel fails during Phase 1 (ISAKMP SA) establishment, caused by either wrong configurations on one of the gateways or by routers/firewalls/ACLs (ACL = Access Control List) between the two VPN gateways.

If the establishment of a VPN tunnel fails, inspect the status of the VPN tunnel to get more information.

The following four situations are most likely to occur:

VPN connection does not begin ISAKMP SA (Phase 1)
ISAKMP SA (Phase 1) not established
IPSec SA (Phase 2) not established
Problems transferring data through an established VPN tunnel (ISAKMP SA and IPSec SA established and routed)

If the establishment of the ISAKMP SA or the IPSec SA fails (Situation 2 or 3), in most cases the VPN logs of both VPN gateways will need to be reviewed to locate the reason for failure. Requesting the right-side side to inspect their VPN logs will also help to determine any issues reported on their side.

Troubleshooting

You can check a VPN tunnel status quickly by using: $ ipsec auto --status

The VPN is not successfully established and routed (unrouted):

000 "cloudsite/0x1": 172.30.0.1/32===172.24.32.11[54.252.137.61,+S=C]...203.177.192.2<203.177.192.2>[+S=C]===10.225.10.4/32; unrouted; eroute owner: #0

The VPN is successfully established and routed (erouted):

000 "cloudsite/0x1": 172.30.0.1/32===172.24.32.11[54.252.137.61,+S=C]...203.177.192.2<203.177.192.2>[+S=C]===10.225.10.4/32; erouted; eroute owner: #0

The VPN Service is not running:

whack: Pluto is not running (no "/var/run/pluto/pluto.ctl")`

You will need to restart the VPN service:

$ ipsec auto --delete <connection_name>

$ ipsec auto --start <connection_name>

$ ipsec auto --ready

If you get:

Redirecting to /bin/systemctl restart ipsec.service
Job for ipsec.service failed because the control process exited with error code. See "systemctl status ipsec.service" and "journalctl -xe" for deta

Try to check if the connection name has any spaces in it. (Additional statuses exist, such as: Prospective erouted and erouted HOLD)

Tunnels not established and routed successfully require a review of the logging output to troubleshoot what may be at fault.

Phase 1 tunnel exchanges are noted by the presence of STATE_MAIN_X# entries in the logging output.

Phase 2 tunnel exchanges are noted by the presence of Quick Mode entries in the logging output.

(It is important to note that variations in right-side gateway appliance vendors can cause differing output in the VPN logs to other right-sides using the same VPN configurations. EX: Cisco appliances versus Palo Alto appliances. As such, many of the core issues shown by the various logging output below have similar resolution steps.)

1. VPN connection does not begin ISAKMP SA (Phase 1)

To determine if the VPN connection has begun to negotiate the ISAKMP SA (Phase 1) tunnel, review the VPN logs and note if you see output like:

Aug 26 02:22:58 cloudsite pluto[17601]: "cloudsite/0x10" #1: initiating Main Mode
Aug 26 02:22:58 cloudsite pluto[17601]: "cloudsite/0x10" #1: received Vendor ID payload [RFC 3947] method set to=109
Aug 26 02:22:58 cloudsite pluto[17601]: "cloudsite/0x10" #1: ignoring Vendor ID payload [FRAGMENTATION c0000000]
Aug 26 02:22:58 cloudsite pluto[17601]: "cloudsite/0x10" #1: enabling possible NAT-traversal with method 4
Aug 26 02:22:58 cloudsite pluto[17601]: "cloudsite/0x10" #1: transition from state STATE_MAIN_I1 to state STATE_MAIN_I2

As this output shows that the VPN tunnel is initiating STATE_MAIN negotiations which are the exchanges to establish the ISAKMP SA (Phase 1) tunnel. If you see STATE_MAIN exchanges, then the tunnel is negotiating the ISAKMP SA (Phase 1) tunnel and you can move on to the next troubleshooting area: ISAKMP SA (Phase 1) could not be established

If output like above is not present, failed initiation of negotiating the ISAKMP SA (Phase 1) tunnel will resemble the following in the logs:

Aug 29 00:39:46 cloudsite pluto[23091]: initiating all conns with alias='cloudsite'
Aug 29 00:39:46 cloudsite pluto[23091]: "cloudsite/0x1" #1: initiating Main Mode
Aug 29 00:39:46 cloudsite pluto[23091]: "cloudsite/0x1" #1: ignoring informational payload, type INVALID_ID_INFORMATION msgid=00000000
Aug 29 00:39:46 cloudsite pluto[23091]: "cloudsite/0x1" #1: received and ignored informational message
Aug 29 00:45:46 cloudsite pluto[23091]: pending Quick Mode with 203.177.192.2 "cloudsite/0x1" took too long -- replacing phase 1
Aug 29 00:45:46 cloudsite pluto[23091]: "cloudsite/0x1" #2: initiating Main Mode to replace #1

And output of $ ipsec auto --status | grep "/0x" | grep routed will show:

000 "cloudsite/0x1": 172.30.0.1/32===172.24.32.11[54.252.137.61,+S=C]...203.177.192.2<203.177.192.2>[+S=C]===10.225.10.4/32; unrouted; eroute owner: #0

When the tunnel initiation is unable to get any kind of reply to move forward in the establishment of the ISAKMP SA (Phase 1) tunnel, this indicates that there is a connectivity or other networking issue between the left-side gateway and the right-side gateway.

Common troubleshooting steps are to work with the right-side to review their VPN logs and see if traffic coming from the left-side is reaching their gateway (incoming traffic). If there is no incoming traffic, the right-side can review their network for any blockers such as Firewalls/ACLs/Intrusion Detection Systems that may be blocking incoming traffic. (UDP ports 500 and 4500.)

If traffic from the left-side is reaching their gateway and the right-side gateway is responding with traffic, then you can review if we are receiving traffic from their side (outgoing traffic) using tcpdump (See: Commands). If we see no traffic outgoing from their gateway to our side, the right-side can review their network for any blockers such as Firewalls/ACLs/Intrusion Detection Systems that may be blocking outgoing traffic.

2. ISAKMP SA (Phase 1) could not established

The ISAKMP SA (Phase 1) tunnel is established using MAIN MODE provided by the Internet Key Exchange (IKE) protocol. (MAIN MODE is the only mode supported in the left-side.)

In MAIN MODE, three pairs of messages are exchanged between both VPN gateways. Keep the following diagram in mind when troubleshooting ISAKMP SA (Phase 1) problems.

The Initiator (the left-side) begins by sending a message out called STATE_MAIN_I1 to the Responder side. The Responder will respond with STATE_MAIN_R1. Once received, the Initiator will send out STATE_MAIN_I2 to the Responder. The Responder will then respond with STATE_MAIN_R2. Once received, the Initiator will send out STATE_MAIN_I3 and the Responder will respond with STATE_MAIN_R3. The state changes are all reflected in the VPN logs and show the progression through all of the messages needed to be exchanged to establish the ISAKMP SA (Phase 1) tunnel.

NOTE: The VPN connection is established through UDP port 500 traffic for ISAKMP SA (Phase 1) exchanges. If the configurations have NAT-T (NAT-T = Network Address Translation - Traversal) enabled or PFS (PFS = Perfect Forward Secrecy), starting with the third MAIN MODE message (STATE_MAIN_I3/STATE_MAIN_R3) the exchange will go through UDP port 4500.

Problems usually occur at the marked points in the above diagram when:

The initiator does not receive a response from the responder.
The initiator receives an unexpected packet or an error message from the responder.

2.1 Problem 1 - Log Examples of Issues at Each Message Exchange

2.1.1 Initiator Log - Issue with STATE_MAIN_I1 exchange:

Aug 26 02:22:58 cloudsite pluto[17601]: "cloudsite/0x10" #1: initiating Main Mode
Aug 26 02:22:58 cloudsite pluto[17601]: "cloudsite/0x1
0" #1: received Vendor ID payload [RFC 3947] method set to=109
Aug 26 02:22:58 cloudsite pluto[17601]: "cloudsite/0x10" #1: ignoring Venador ID payload [FRAGMENTATION c0000000]
Aug 26 02:22:58 cloudsite pluto[17601]: "cloudsite/0x10" #1: enabling possible NAT-traversal with method 4
Aug 26 02:22:58 cloudsite pluto[17601]: "cloudsite/0x10" #1: STATE_MAIN_I1: sent MI1, expecting MR1

Likely an issue with UDP port 500 (Incoming or outgoing traffic) being blocked on the right-side side. Or likely an issue on the right-side gateway (in right-side gateway VPN logs) concerning no matching encryption algorithm/hash algorithm for Phase 1 settings.

2.1.2 Initiator Log - Issue with STATE_MAIN_I2 exchange:

08:53:48.15255 "cloudsite" #2: received Vendor ID payload [Openswan (this version) 2.6.24 ]
08:53:48.15259 "cloudsite" #2: received Vendor ID payload [Dead Peer Detection]
08:53:48.15263 "cloudsite" #2: received Vendor ID payload [RFC 3947] method set to=109
08:53:48.15267 "cloudsite" #2: ignoring Vendor ID payload [FRAGMENTATION c0000000]
08:53:48.15275 "cloudsite" #2: enabling possible NAT-traversal with method 4
08:53:48.37178 "cloudsite" #2: transition from state STATE_MAIN_I1 to state STATE_MAIN_I2
08:53:48.37186 "cloudsite" #2: STATE_MAIN_I2: sent MI2, expecting MR2

Likely an issue on the right-side gateway (output may be present in the right-side gateway VPN logs) concerning no matching Diffie-Hellman Group with our provided configurations.

2.1.3 Initiator Log - Issue with STATE_MAIN_I3 exchange:

08:53:50.72881 "cloudsite" #2: NAT-Traversal: Result using RFC 3947 (NAT-Traversal): both are NATed
08:53:50.72892 "cloudsite" #2: I am sending my cert
08:53:50.72896 "cloudsite" #2: I am sending a certificate request
08:53:50.72942 "cloudsite" #2: transition from state STATE_MAIN_I2 to state STATE_MAIN_I3
08:53:50.72961 "cloudsite" #2: STATE_MAIN_I3: sent MI3, expecting MR3

Likely an issue with UDP port 4500 being blocked on the right-side side (incoming or outgoing). (Only for NAT-T or PFS enabled tunnels.)

2.1.4 Initiator Log - Successful ISAKMP SA (Phase 1) exchange

08:53:50.97229 "cloudsite" #2: issuer cacert not found
08:53:50.97233 "cloudsite" #2: X.509 certificate rejected
08:53:50.97236 "cloudsite" #2: transition from state STATE_MAIN_I3 to state STATE_MAIN_I4
08:53:50.97244 "cloudsite" #2: STATE_MAIN_I4: ISAKMP SA established {auth=OAKLEY_PRESHARED_KEY cipher=aes_256 prf=oakley_sha group=modp1024}

2.2 Problem 2 - Log Examples of Issues with unexpected packets or error messages

2.2.1 Initiator - Pending Quick Mode with 1.2.3.4 took too long - replacing phase 1

08:56:40.12570 "cloudsite" #6: initiating Main Mode
09:02:50.03792 pending Quick Mode with 77.245.33.66 "cloudsite" took too long -- replacing phase 1
09:02:50.03804 "cloudsite" #7: initiating Main Mode to replace #6
09:04:50.04538 pending Quick Mode with 77.245.33.66 "cloudsite" took too long -- replacing phase 1
09:04:50.04550 "cloudsite" #8: initiating Main Mode to replace #7

The initiator initiates the VPN connection by sending the first MAIN MODE message but there is no response from the responder. The initiator keeps trying to initiate the VPN connection. Important to inspect the VPN logs on the right-side side to see whether these messages have reached their gateway. Possibly a blocker on UDP port 500 for Phase 1 tunnel completions or UDP port 4500 traffic for Phase 2 initiation.

2.2.2 Initiator - Possible authentication failure: no acceptable response to our first encrypted message

09:54:06.14104 "cloudsite" #55: initiating Main Mode
09:54:08.02489 "cloudsite" #55: received Vendor ID payload [Openswan (this version) 2.6.24 ]
09:54:08.02493 "cloudsite" #55: received Vendor ID payload [Dead Peer Detection]
09:54:08.02497 "cloudsite" #55: received Vendor ID payload [RFC 3947] method set to=109
09:54:08.02509 "cloudsite" #55: enabling possible NAT-traversal with method 4
09:54:08.35894 "cloudsite" #55: transition from state STATE_MAIN_I1 to state STATE_MAIN_I2
09:54:08.35902 "cloudsite" #55: STATE_MAIN_I2: sent MI2, expecting MR2
09:54:10.71933 "cloudsite" #55: NAT-Traversal: Result using RFC 3947 (NAT-Traversal): both are NATed
09:54:10.71945 "cloudsite" #55: I am sending my cert
09:54:10.71948 "cloudsite" #55: I am sending a certificate request
09:54:10.72057 "cloudsite" #55: transition from state STATE_MAIN_I2 to state STATE_MAIN_I3
09:54:10.72076 "cloudsite" #55: STATE_MAIN_I3: sent MI3, expecting MR3
09:54:20.23466 "cloudsite" #55: discarding duplicate packet; already STATE_MAIN_I3
09:54:40.23282 "cloudsite" #55: discarding duplicate packet; already STATE_MAIN_I3
09:55:21.23123 "cloudsite" #55: max number of retransmissions (2) reached STATE_MAIN_I3. Possible authentication failure:
no acceptable response to our first encrypted message

The initiator has sent the third MAIN MODE message and expects a STATE_MAIN_R3 response. But the responder returns another STATE_MAIN_R2 instead. This shows as “discarding duplicate packet”. If the VPN is configured with NAT-T enabled on port 500. Investigate if the right-side side is blocking UDP port 4500 via Firewall/Router/ACL.

2.2.3 Initiator - ignoring informational payload, type INVALID_ID_INFORMATION

10:00:07.10837 "cloudsite" #61: initiating Main Mode
10:00:09.02070 "cloudsite" #61: received Vendor ID payload [Openswan (this version) 2.6.24 ]
10:00:09.02074 "cloudsite" #61: received Vendor ID payload [Dead Peer Detection]
10:00:09.02077 "cloudsite" #61: received Vendor ID payload [RFC 3947] method set to=109
10:00:09.02089 "cloudsite" #61: enabling possible NAT-traversal with method 4
10:00:09.34262 "cloudsite" #61: transition from state STATE_MAIN_I1 to state STATE_MAIN_I2
10:00:09.34270 "cloudsite" #61: STATE_MAIN_I2: sent MI2, expecting MR2
10:00:11.70805 "cloudsite" #61: NAT-Traversal: Result using RFC 3947 (NAT-Traversal): both are NATed
10:00:11.70817 "cloudsite" #61: I am sending my cert
10:00:11.70821 "cloudsite" #61: I am sending a certificate request
10:00:11.70929 "cloudsite" #61: transition from state STATE_MAIN_I2 to state STATE_MAIN_I3
10:00:11.70948 "cloudsite" #61: STATE_MAIN_I3: sent MI3, expecting MR3
10:00:11.71746 "cloudsite" #61: ignoring informational payload, type INVALID_ID_INFORMATION msgid=00000000
10:00:11.71750 "cloudsite" #61: received and ignored informational message

The initiator has sent the third MAIN MODE message and expects STATE_MAIN_R3 in response from the responder. This message contains the initiator’s certificate and/or hashed value of the PSK and expects the necessary information from the responder. But the responder did not send its certificate and/or hashed value of the PSK, instead it returns an informational payload with INVALID_ID_INFORMATION. This usually indicates a PSK mismatch. Ensure the same PSK is in place in the configurations for both the left-side gateway and the right-side gateway.

3. IPSec SA (Phase 2) could not be established

The IPSec SA (Phase 2) tunnel is established using QUICK MODE provided by the Internet Key Exchange (IKE) protocol. Again, three messages are exchanged in this mode. Phase 2 uses UDP port 4500.

If the establishment of the IPSec SA fails, it is most commonly caused by a configuration mismatch between each gateway. Either the configured VPN networks do not match or there is a mismatch in the configured encryption algorithm and/or hash algorithm. PFS may also be enabled on one gateway but not the other.

3.1 Log Examples of Issues with IPSec SA (Phase 2) exchanges

3.1.1 Initiator - ignoring informational payload, type NO_PROPOSAL_CHOSEN

15:50:00.48413 "cloudsite" #80: initiating Main Mode
----------  Establishment of the ISAKMP SA  ----------15:50:05.34633 "cloudsite" #80: STATE_MAIN_I4: ISAKMP SA established {auth=OAKLEY_PRESHARED_KEY cipher=aes_256 prf=oakley_sha group=modp1024}
15:50:05.34638 "cloudsite" #80: Dead Peer Detection (RFC 3706): enabled
15:50:05.34642 "cloudsite" #81: initiating Quick Mode PSK+ENCRYPT+TUNNEL+PFS+UP+IKEv2ALLOW+SAREFTRACK {using isakmp#1 msgid:507fcfd2 proposal=AES(12)_256-SHA1(2)_160 pfsgroup=OAKLEY_GROUP_MODP1024}
15:50:05.64835 "cloudsite" #80: ignoring informational payload, type NO_PROPOSAL_CHOSEN msgid=00000000
15:50:05.64839 "cloudsite" #80: received and ignored informational message

This usually indicates that the configured encryption or hash algorithms do not match for the IPSec SA (Phase 2) settings. Check on the Phase 2 encryption algorithm, the hash algorithm, and the DH Group (Diffie-Hellman Group) setting.

3.1.2 Initiator - ignoring informational payload, type INVALID_ID_INFORMATION

16:08:21.07207 "cloudsite" #104: initiating Main Mode
----------  Establishment of the ISAKMP SA  ----------16:08:25.85346 "cloudsite" #104: STATE_MAIN_I4: ISAKMP SA established {auth=OAKLEY_RSA_SIG
cipher=oakley_3des_cbc_192 prf=oakley_md5 group=modp8192}
16:08:25.85351 "cloudsite" #104: Dead Peer Detection (RFC 3706): enabled
16:08:25.85354 "cloudsite" #105: initiating Quick Mode PSK+ENCRYPT+TUNNEL+PFS+UP+IKEv2ALLOW+SAREFTRACK {using isakmp#1 msgid:507fcfd2 proposal=AES(12)_256-SHA1(2)_160 pfsgroup=OAKLEY_GROUP_MODP1024}
16:08:26.20417 "cloudsite" #104: ignoring informational payload, type INVALID_ID_INFORMATION msgid=00000000
16:08:26.20422 "cloudsite" #104: received and ignored informational message

3.1.3 Initiator - No acceptable response to our first Quick Mode message: perhaps peer likes no proposal

09:20:12.96824 "cloudsite" #15: initiating Main Mode
----------  Establishment of the ISAKMP SA  ----------09:20:17.64568 "cloudsite" #15: STATE_MAIN_I4: ISAKMP SA established i{auth=OAKLEY_PRESHARED_KEY cipher=aes_256 prf=oakley_sha group=modp1024}
09:20:17.64573 "cloudsite" #15: Dead Peer Detection (RFC 3706): enabled
09:20:17.64577 "cloudsite" #16: initiating Quick Mode PSK+ENCRYPT+TUNNEL+PFS+UP+IKEv2ALLOW+SAREFTRACK {using isakmp#1 msgid:507fcfd2 proposal=AES(12)_256-SHA1(2)_160 pfsgroup=OAKLEY_GROUP_MODP1024}
09:21:27.63790 "cloudsite" #16: max number of retransmissions (2) reached STATE_QUICK_I1. No acceptable response to our
first Quick Mode message: perhaps peer likes no proposal

This is usually encountered if the right-side gateway has PFS disabled but the initiator (the left-side) side is enabled. Ensure PFS is enabled or disabled on both sides. Disabling PFS on both sides can aid in troubleshooting any additional issues that also may occur.

If all phase 2 settings are correct then have the right-side check left-side’s encryption domain on their end. Ensure that instead of /24 they are using /32. For example, ensure the left-side private IP right-side has on their end is 172.30.0.1/32 and not 172.30.0.0/24

3.1.4 Initiator - received Delete SA payload: deleting ISAKMP State #1

16:08:21.07207 "cloudsite" #104: initiating Main Mode
----------  Establishment of the ISAKMP SA  ----------16:08:25.85346 "cloudsite" #104: STATE_MAIN_I4: ISAKMP SA established {auth=OAKLEY_PRESHARED_KEY cipher=aes_256 prf=oakley_sha group=modp1024}
16:08:25.85351 "cloudsite" #104: Dead Peer Detection (RFC 3706): enabled
16:08:25.85354 "cloudsite" #105: initiating Quick Mode PSK+ENCRYPT+TUNNEL+PFS+UP+IKEv2ALLOW+SAREFTRACK {using isakmp#1 msgid:507fcfd2 proposal=AES(12)_256-SHA1(2)_160 pfsgroup=OAKLEY_GROUP_MODP1024}
16:08:26.20422 "cloudsite" #106: received Delete SA payload: deleting ISAKMP State #1
16:08:26.20422 "cloudsite" #107: packet from 130.220.255.62:4500: received and ignored informational message
16:08:26.20422 "cloudsite" #108: phase2 initiation failed because parent ISAKMP #1 is gone

Usually not encountered on tunnel creation but during rekeying. The right-side has their gateway configuration incorrect and is causing the ISAKMP SA (Phase 1) tunnel to be deleted by sending a Delete SA payload. Any errors encountered on the right-side VPN gateway with the IPSec SA (Phase 2) packets being sent by our gateway could cause a Delete SA payload to be sent in an attempt to start a rekeying phase. However, in the first initiation of the VPN tunnel, rekeying is not allowed as there is no initial keying present. Ensure lifetime settings match for both gateways.

4. Problems transferring data through an established VPN tunnel

If the VPN tunnel is completely established and routed (erouted) problems transferring data through the VPN tunnel are usually not caused by any VPN issues.

5. Other Problems

5.1 VPN tunnel fails after 24 hours

This problem usually occurs if the VPN gateways cannot rekey successfully for the ISAKMP SA (Phase 1) tunnel. Ensure again that the configurations match on each VPN gateway and double check the lifetime values for the ISAKMP SA (Phase 1) and IPSec SA (Phase 2) configurations.

Also ensure that the right-side gateway does not have any settings that disconnect/terminate the VPN tunnel if idle as this can cause rekeying issues.

5.2 Problems transferring large sets of data over tunnel

This problem is usually caused by network configurations limiting the Maximum Transfer Unit (MTU) size of network packets exchanged over the VPN tunnel.

This can be solved by changing the MTU size setting for the VPN tunnel via cloud custom property to an applicable value as reviewed and approved by the left-side architects.

5.3 Configuration constraints when connecting to AWS VPN connections

Sometimes a requirement will be to configure a VPN to connect to an AWS VPC. These configurations have a special limitation that, when not observed, can cause some strange issues.

Catalyst: Adding multiple security associations to a single VPN tunnel that is hooked up to AWS

Symptoms: Connectivity to either of the tunnels will be interrupted for what appears to be no reason. It will appear like an issue on the right-side side, but will start and stop working with no changes

Remediation: Change the tunnel configuration to only have a single security association with a suitable range to cover all required endpoints.

Reference: “You are limited to 1 unique Security Association (SA) pair per tunnel (1 inbound and 1 outbound), and therefore 2 unique SA pairs in total for 2 tunnels (4 SAs).”

When initially establishing the connection, the right-side will require us to provide the gateway IP first. This is because the peer gateway IP (ours) is required when creating a Virtual Private Gateway in the AWS console

5.4 Problem with SSL handshake due to left-side not receiving certificate

Web service calls over VPN fail and a 503 is received on the frontend. Openssl s_client -showcerts -connect <h>:<p> returns no certificate.

This can be solved by changing the MTU size setting for the VPN tunnel to an applicable value.

6. Configuration

6.1 ipsec.conf file

The ipsec.conf file contains the configurations to be used for each VPN tunnel connection configured. There can be more than one VPN configuration on a given server. Each connection section begins with the ‘conn’ label.

ipsec.conf template example:

conn ###right-sideAlias###
left=%defaultroute
leftid=###PUBLIC_IP_ADDRESS###
leftsubnet=###PRIVATE_IP_ADDRESS###/32
right=###right-side_VPN_GATEWAY###
rightsubnets={###right-side_VPN_PEER_BLOCK/24,###right-side_VPN_PEERS###/32}
authby=secret
auto=add/start (add waits for right-side gateway to initiate. Start initiates from the cloud side. Use start as default setting.)
pfs=no/yes (Option2/Option1 - depends on option selected on worksheet)
ike=aes256-sha2_256-modp1024
phase2alg=aes256-sha2_256
ikelifetime=28800s
salifetime=3600s
dpdaction=restart
dpddelay=30
dpdtimeout=120

6.2 networking.sh

This file is actually a bash script that can run during the cloud site build that puts into place any OS level network routes necessary for the VPN tunnel network traffic.

networking.sh example file contents:

#!/bin/bash
route add -net 10.0.0.0/24 gw 172.30.0.1
route add -host 192.168.1.3 gw 172.30.0.1

Adding entries to networking.sh should be required when:

right-side is not using DNS based resolution.
right-side is using private IP addresses that are outside of Class B/C spaces (Class A public internet equivalent IP address spaces).
right-side is specifically using an IP address in Admin Console/left-side configurations and unable to connect to the endpoint over the VPN tunnel.
right-side has all end user access to the left-side site going over the VPN tunnel.
right-side has been allowed to use addresses for their network in 10.0.0.0/8 or 172.0.0.0/8 address spaces (outside default 172.30.0.0/24 for cloud).

7. Commands

7.1 VPN Specific

Check status of VPN tunnel:

$ ipsec auto --status

Check status of VPN tunnel to ensure it is just routed or not:

$ ipsec auto --status | grep "/0x" | grep routed

Check status if all VPN tunnels (internal and external):

$ ipsec status | grep routed

See full state of LibreSwan IPSec tunnel:

$ ipsec barf

Check VPN tunnel initialization:

$ tail -f /usr/local/left-side/ipsec.d/logs/ipsec.log.<YYYY-MM-DD>

$ ipsec auto --delete <connection_name>

$ ipsec auto --ready

$ ipsec auto --start <connection_name>

7.2 Linux OS Network Routing

View routes on instance:

$ route

How to delete routes created in networking.sh:

$ route del -net 10.1.0.0 netmask 255.255.0.0 gw 10.2.0.1

Adding and Removing a Network Space:

$ route add -net 10.10.10.0/24 gw 172.30.0.1

$ route del -net 10.10.10.0/24 gw 172.30.0.1

Adding and Removing a Specific Host/ IP:

$ route add -host 10.10.10.45 gw 172.30.0.1

$ route del -host 10.10.10.45 gw 172.30.0.1

Adding Network Interface for Primary VPN Tunnel

$ ifconfig eth0:0 ###PRIVATE_IP_ADDRESS### netmask 255.255.255.255

Adding Additional Network Interface for Tertiary VPN Tunnels

$ ifconfig eth0:1 ###PRIVATE_IP_ADDRESS### netmask 255.255.255.255

7.3 Generate Network Traffic

Run a Ping:

$ ping -I ###left-side_PRIVATE_IP### -i 5 ###right-side_IP###

Example:

ping -I 172.30.0.1 -i 5 10.2.32.64
- -I: interface, IP address to originate from
- -i: interval, the interval the ping sends out traffic on

Telnet: (Ensure that a route to the IP endpoint you are trying to reach is present.)

$ telnet ###endpoint_Ip### ###port_num###

Example: telnet 123.456.789.12 80

Check for resolution of a web service WSDL:

$ curl --interface "172.30.0.1" http://10.2.32.46:7001/EmployeeService/EmployeeService?WSDL

7.4 Verify Traffic Exchanges for the VPN tunnel:

See Phase 1 and Phase 2 traffic going over the VPN tunnel:

$ tcpdump udp -vv -n \`port 500 or port 4500\`

See Phase 2 traffic going over the VPN tunnel:

$ tcpdump | grep ESP

7.5 DNS Lookup via Command Line

$ nslookup

> server ##dns_server_ip##
(if using right-side DNS server. Otherwise, defaults to the left-side DNS server.)
...output showing the DNS server is set to access over port 53…
> ##URL_to_get_IP_from##
...should show output resolving the URL to an IP address…

7.6 LDAP Server Testing

$ telnet ##ldap_server_ip## ##port 389 for ldap or port 636 for ldaps##

ldapsearch -uvxW -H ldap://##ldap_Server_ip##:##port 389 for ldap or 636 for ldaps## -D "##Search bind DN##" -b "##Base DN##" "##Search Filter##" -z 5

Glossary

IPSec - Internet Protocol Security (IPsec) is a network protocol suite that authenticates and encrypts the packets of data sent over a network.

ISAKMP - ISAKMP (Internet Security Association and Key Management Protocol) is a protocol defined by RFC 2408 for establishing Security Associations (SA) and cryptographic keys in an Internet environment.

unrouted - The VPN tunnel is not routed. There is an issue in either the Phase 1 or Phase 2 tunnel creation.

prospective erouted - The VPN tunnel shows as routed correctly based on the tunnel states for Phase 1 and Phase 2 tunnels but there has been no traffic passed over the VPN tunnel to indicate complete success. This can also show up if the keying has expired on the tunnel and a rekey phase is expected to be incoming from the right-side gateway as the tunnel may still be valid for traffic to pass through but the state is in flux.

erouted HOLD - The VPN tunnel was successfully routed and a rekeying initiation was made through a Delete SA payload. The gateway expects a new keying phase to begin initiated by the right-side tunnel or a rekeying has been initiated but an error or other issue has prevented this from completing and no specific errors have been received.

erouted - The VPN tunnel is successfully established (e) and routed (routed) (established + routed == erouted). Both the Phase 1 and Phase 2 tunnels are also in a current keying phase with valid keys in place for both tunnels.

Security Association (SA) - A Security Association is a logical connection between two devices transferring data (here VPN gateways). An SA provides data protection for unidirectional traffic by using the defined IPSec protocols. An IPSec tunnel typically consists of two unidirectional SAs, which together provide a protected, full-duplex data channel. The SAs allow an enterprise to control exactly what resources may communicate securely, according to security policy. To do this an enterprise can set up multiple SAs to enable multiple secure VPNs, as well as define SAs within the VPN to support different departments and business partners.

Pre-Shared Key (PSK) - In cryptography, a pre-shared key (PSK) is a shared secret value which was previously shared between two parties using some secure channel before it needs to be used/implemented.

NOTE: PSK is not used for encryption. PSK is used for authentication. In Main Mode, the first message is sent by the Initiator, and the second message is sent by the Responder. Both of these messages include what is known in the cryptography world as a Nonce. A Nonce is simply a randomly generated number to use in key generation. (Nonce = Number used ONCE). So, after message 1 and message 2, both sides know each other’s Nonces. The Nonces are combined with the Pre-Shared-Key to create a Seed value for generating

Maximum Transfer Unit (MTU) - In computer networking, the maximum transmission unit (MTU) is the size of the largest network layer protocol data unit that can be communicated in a single network transaction.

Diffie Hellman Group (DH Group) - Diffie-Hellman is an asymmetric key algorithm used for public key cryptography. As well as IPSec it is also used for SSL, SSH, PGP and other PKI systems. less The Diffie-Hellman algorithm was created to address the issue of secure encrypted keys being attacked over the internet when in transmission by using the Diffie-Hellman algorithm in distributing symmetric keys securely over the internet. So Diffie-Hellman (DH) allows two devices to establish a shared secret over an unsecure network. In terms of VPN it is used in the in the IKE or Phase1 part of setting up the VPN tunnel.

Diffie-Hellman group 1 - 768 bit modulus - UNSUPPORTED Diffie-Hellman group 2 - 1024 bit modulus - AVOID Diffie-Hellman group 5 - 1536 bit modulus - ACCEPTABLE Diffie-Hellman group 14 - 2048 bit modulus – ACCEPTABLE

Diffie-Hellman group 19 - 256 bit elliptic curve – SUPPORTED IN CLOUD Diffie-Hellman group 20 - 384 bit elliptic curve – SUPPORTED Diffie-Hellman group 21 - 521 bit elliptic curve – SUPPORTED Diffie-Hellman group 24 - modular exponentiation group with a 2048-bit modulus and 256-bit prime order subgroup

Perfect Forward Secrecy (PFS) - In cryptography, forward secrecy (FS), also known as perfect forward secrecy (PFS), is a property of secure communication protocols in which a security secretcompromise of long-term keys does not compromise past session keys. Forward secrecy protects past sessions against future compromises of secret keys or passwords.

Internet Key Exchange (IKE) - In computing, Internet Key Exchange (IKE, sometimes IKEv1 or IKEv2, depending on version) is the protocol used to set up a security association (SA) in the IPsec protocol suite. IKE builds upon the Oakley protocol and ISAKMP.

Encryption Algorithm - Encryption is the process of encoding data so that only a computer with the right decoder will be able to read and use it. VPN uses protocols and some encryption algorithms for the ultimate privacy protection and security of network traffic. AES (Advanced Encryption Standard) is a secure algorithm used in symmetric key encryption. It supports various key lengths of 128, 192, and 256 bit, the longer the key length would be the stronger the encryption which also means it takes more time in processing which results in slower connection speed.

Hash Algorithm - A hash function is any function that can be used to map data of arbitrary size to data of a fixed size. The values returned by a hash function are called hash values, hash codes, digests, or simply hashes. Hashing algorithms have evolved into HMACs, which combine the proven security of hashing algorithms with additional cryptographic functions. The hash produced is encrypted with the sender’s private key, resulting in a keyed checksum as output. Secure Hash Algorithm (SHA) was created by Cisco and this algorithm is both very secure and strong in that it requires both the sender and receiver to imply use with this algorithm while encrypting and decrypting the message or the data traveling through the VPN tunnel.

Network Address Translation (NAT) - Network address translation (NAT) is a method of remapping one IP address space into another by modifying network address information in Internet Protocol (IP) datagram packet headers while they are in transit across a traffic routing device. The technique was originally used for ease of rerouting traffic in IP networks without readdressing every host.

NAT-T - Network address translation traversal (NAT-T) is a computer networking technique of establishing and maintaining Internet protocol connections across gateways that implement network address translation (NAT). NAT breaks the principle of end-to-end connectivity originally envisioned in the design of the Internet.

Access Control List (ACL) - An access control list (ACL) is a list of permissions attached to an object. An ACL specifies which objects are granted access to other objects, as well as what operations are allowed on given objects. On some types of proprietary computer-hardware (in particular routers and switches), an access control list provides rules that are applied to port numbers and/or IP addresses that are available on a host or other network appliance, each with a list of hosts and/or networks permitted to use the service.

Dead Peer Detection (DPD) - Dead Peer Detection (DPD) is a method of detecting a dead Internet Key Exchange (IKE) peer. The method uses IPsec traffic patterns to minimize the number of messages required to confirm the availability of a peer. When a dead peer is detected, the functional peer can attempt to re-initiate a VPN tunnel from scratch with the dead peer in an attempt to re-establish and route the VPN tunnel(s).