High Availability Wireguard VPN Infrastructure

A project to build a high availability WireGuard infrastructure in AWS

Started : 17-Aug-26

Notes

Requirements and beginnings

Created : 18 Aug 26 | Modified : 25 Aug 26 14:38

The initial requirements for the project, and some preliminary findings

An existing infrastructure to manage connected clients is approaching some limits in terms of scalability.  In addition, the existing infrastructure provides no resilience and is dependant on a single server instance hosted in AWS.

Typically the single server instance isn't an issue, however when the server requires maintenance that requires a reboot, the connected clients all attempt to re-establish connections, resulting in a 'galloping herd' problem once the server is available that results in high server load and loss of connectivity to the clients for up to a half an hour before connections to the existing 7,000+ active clients are re-established and services return to normal.  As there is only a single server hosted in AWS, the existing infrastructure is also at risk in case of an AWS Availability Zone outage.

The current method of connection is limited in its maximum count of clients connected - and we're approaching the limit in a few years to that maximum limit (note that active clients are only a subset of all possible clients, but the infrastructure needs to account for the possibility of all possible clients connecting)

Our initial requirements for this project are;
* must provide resilience to individual server loss
* must provide resilience to AWS Availability Zone (AZ) outage
* not allow individual clients to be able to connect to each other - only permit connection to/from maintenance servers also hosted in AWS
* must allow scaling beyond 32,767 active clients
* allow for server-initiated traffic to the connected clients
* only permit known clients to connect to the maintenance servers

[Wireguard VPN](https://www.wireguard.com/) has been suggested as a start point in order to set up a shared VPN between the clients and maintenance servers that fits a number of the requirements.

The linked 'High Availability Wireguard on AWS' article also provides a good start point for achieving a number of other requirements.

Bookmarks

High Availability WireGuard on AWS

Set up WireGuard in a high availability configuration on AWS

Secure VPN client registration

Created : 24 Aug 26 | Modified : 24 Aug 26 11:41

Ensure that clients connecting to the VPN are able to do so in a secure manner

The intent is that once the use of the VPN has been proven with test clients locally, as part of regular updates to clients, these will be progressively migrated from the existing mechanism of connection to the support servers to use the VPN instead.

Many of the current clients are remote - which requires a secure means for the remote clients to register with the VPN.  Manual configuration of migrated clients for the Wireguard server/s config is not practical.

Clients currently send telemetry data via mqtt (AWS IoT Core).  An existing secure mechanism exists to have clients registered (and reregistered if necessary) and issued with the certificates needed to connect and publish to mqtt.  As part of this, a client specific private key is provided to the client during registration.  We will make use of the existing private key known to already exist on the clients to sign a JWT that will be sent along with the clients Wireguard Public Key to a new vpn registration API endpoint.

The AWS IoT Core Certificate can be queried for the public key associated with the certificate.  This public key can then be used to validate that the JWT sent by the client was signed with the private key issued to the client.

The API will use the JWT to verify that the registration request has come from the client and has not been spoofed by another client.  All APIs are secured with TLS, so the end to end validity of the request/response is sound.

The vpn-registration will provide to the caller the Wireguard public key for the Wireguard instances, the allowed IPs the client is able to access within the VPN and the IP address for the Wireguard endpoint.

The vpn-registration API will be deployed into AWS API Gateway along side the existing registration API used to register clients for access to AWS IoT Core.

Child notes

Fetch of AWS IoT Core Thing Principals

Created 08/24/2026 12:18 · Modified 08/25/2026 14:39

Routing From Maintenance Servers to Clients

Created : 28 Aug 26 | Modified : 28 Aug 26 08:29

How do we connect from the existing maintenance servers to VPN connected clients?

Bookmarks

Linux GRE tunnels: configuration, MTU, and when to use them over WireGuard

How to build IP-over-IP GRE tunnels on Linux with iproute2, the MTU math, keepalive options, and where GRE still fits in a world of WireGuard and VXLAN.
MSS, MSS Clamping, PMTUD, and MTU

Infrastructure

Created : 28 Aug 26

The infrastructure necessary to provide scalability and fault tolerance

Bookmarks

High Availability WireGuard on AWS

Set up WireGuard in a high availability configuration on AWS

Attachments

High Availability WireGuard on AWS

This article will demonstrate how to set up WireGuard in a high availability configuration on AWS (Amazon Web Services).

Child notes

Minimal Policy for Global Accelerator creation

Created 08/28/2026 08:53 · Modified 08/31/2026 17:53

Provides the bare minimum permissions for an IAM user to be able to Create/Update an AWS Global Accelerator

```
  AcceleratorPolicy:
    Type: AWS::IAM::ManagedPolicy
    Properties:
      PolicyDocument:
        Version: "2012-10-17"
        Statement:
          -
            Sid: AcceleratorAccess
            Action:
              - globalaccelerator:CreateAccelerator
              - globalaccelerator:UpdateAccelerator
              - globalaccelerator:CreateEndpointGroup
              - globalaccelerator:UpdateEndpointGroup
              - globalaccelerator:CreateListener
              - globalaccelerator:UpdateListener
              - globalaccelerator:TagResource
              - globalaccelerator:UntagResource
              - globalaccelerator:DescribeAccelerator
              - globalaccelerator:DescribeAcceleratorAttributes
              - globalaccelerator:DescribeEndpointGroup
              - globalaccelerator:DescribeListener
            Effect: Allow
            Resource:
            - !Sub 'arn:aws:globalaccelerator::${AWS::AccountId}:accelerator/*'
      ManagedPolicyName: AcceleratorPolicy
			```

Things To Do

Prototype AWS Infrastructure

Decide on AWS services needed for prototype & build out

Test Framework

Testing to assist with initial scale and prove the approach in DEV is required.

Document the architecture and decisions in ADR

Write up the Architecture Design Record

Publish registered clients to Wireguard instances

The peer IPs allocated to the clients during registration as well as the Wireguard public keys need to be published to each of the Wireguard hosts

Routing from maintenance servers to Wireguard hosts

Wireguard clients connecting will end up on one of x Wireguard hosts behind a load balancer. Maintenance server need to know how to route to the client connected to Wireguard Host X

Maintenance scripts need to be 'connection' agnostic

The various maintenance scripts all need to be able to either connect to clients via VPN or via the existing connection mechanism

Notify WireGuard hosts from vpn-register

To avoid having to have the WireGuard servers poll for changes to registered clients, have registration push notifications via SQS/SNS to trigger an agent on the WireGuard servers to refresh config

Secure VPN client registration

The client Wireguard client needs a way to securely register with the Wireguard server/s

Done

Back to Projects