Requirements and beginnings

Created 08/18/2026 11:40 · Last modified 08/25/2026 14:38

AWS

The initial requirements for the project, and some preliminary findings

An existing infrastructure to manage connected clients is approaching some limits in terms of scalability.  In addition, the existing infrastructure provides no resilience and is dependant on a single server instance hosted in AWS.

Typically the single server instance isn't an issue, however when the server requires maintenance that requires a reboot, the connected clients all attempt to re-establish connections, resulting in a 'galloping herd' problem once the server is available that results in high server load and loss of connectivity to the clients for up to a half an hour before connections to the existing 7,000+ active clients are re-established and services return to normal.  As there is only a single server hosted in AWS, the existing infrastructure is also at risk in case of an AWS Availability Zone outage.

The current method of connection is limited in its maximum count of clients connected - and we're approaching the limit in a few years to that maximum limit (note that active clients are only a subset of all possible clients, but the infrastructure needs to account for the possibility of all possible clients connecting)

Our initial requirements for this project are;
* must provide resilience to individual server loss
* must provide resilience to AWS Availability Zone (AZ) outage
* not allow individual clients to be able to connect to each other - only permit connection to/from maintenance servers also hosted in AWS
* must allow scaling beyond 32,767 active clients
* allow for server-initiated traffic to the connected clients
* only permit known clients to connect to the maintenance servers

[Wireguard VPN](https://www.wireguard.com/) has been suggested as a start point in order to set up a shared VPN between the clients and maintenance servers that fits a number of the requirements.

The linked 'High Availability Wireguard on AWS' article also provides a good start point for achieving a number of other requirements.

References

High Availability WireGuard on AWS

Set up WireGuard in a high availability configuration on AWS

Back to Notes