At System Initiative, we use Tailscale for secure networking in our production infrastructure. We chose Tailscale specifically as it’s easy to automate its management and setup. This makes it perfect to integrate with System Initiative.

System Initiative deploys its infrastructure across multiple AWS accounts and multiple regions. When we bring a new segment of infrastructure under our management, one of the first things we do is to ensure that we deploy Tailscale into it. We have a standard way to deploy Tailscale, so for us, it’s a matter of “copying and pasting” a set of components in our graph. The set of components looks as follows:

How does it work?

You can see from the image above how the components relate to each other. We store our Tailscale API Key in AWS Secrets Manager and use an AWS IAM Role to allow Secrets Manager access to read the Tailscale API Key. The configuration of our IAM Role to allow that access is as follows:

{
  "RoleName": "tailscale",
  "AssumeRolePolicyDocument": {
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Action": [
          "sts:AssumeRole"
        ],
        "Principal": {
          "Service": [
            "ec2.amazonaws.com"
          ]
        }
      }
    ]
  }
}

The IAM Role also has an AWS Managed Role Policy (arn:aws:iam::aws:policy/SecretsManagerReadWrite) attached. This managed role policy is what allows the interaction with Secrets Manager. We use an AWS EC2 Launch Template and an AWS AutoScaling Group to spin up a Tailscale exit node. The UserData we pass to the EC2 Launch Template allows the instance to start and register the Tailscale service with the Tailscale account.

#!/bin/bash

exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1

# Enable ip_forward to allow advertising routes
echo 'net.ipv4.ip_forward = 1' | sudo tee -a /etc/sysctl.conf
echo 'net.ipv6.conf.all.forwarding = 1' | sudo tee -a /etc/sysctl.conf
sudo sysctl -p /etc/sysctl.conf

# Install tailscale
sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://pkgs.tailscale.com/stable/amazon-linux/2/tailscale.repo
sudo yum install -y tailscale

# Setup tailscale
sudo systemctl enable --now tailscaled

TAILSCALE_KEY=$(aws secretsmanager get-secret-value --region us-east-1 --secret-id tailscale-key | jq -r '.SecretString')

# Wait a few for tailscaled to come up
sleep 5

# Start tailscale
sudo tailscale up \
    --advertise-routes=10.0.0.0/16 \
    --authkey=${TAILSCALE_KEY} \
    --hostname=demo-tailscale-exitnode

The AutoScaling Group has a desired count of 1, which means that if a node fails, AWS will bring a new node into its place and use the UserData specified in the Launch Template. This allows us to regularly refresh our Tailscale instances to ensure they are security-patched and updated. The last piece of the infrastructure required is an AWS EC2 Security Group and the associated Ingress rule to allow SSH to port 22.

Future Plans

As part of System Initiative’s Roadmap, we plan to introduce “management nodes.” You can think of management nodes as a mix of templates and workflows. The introduction of management nodes means that we won’t have to copy and paste a set of components; we would instead drag a management node for Tailscale infrastructure onto our canvas, and the workflow would already understand all of the pieces of infrastructure it would need to deploy and manage. We can’t wait to release that feature so we can share this simple, yet powerful, configuration with you all!

Come join us on Discord and talk to us there. You can message me there directly at SI_Stack72. We are excited to see all the cool things you build with System Initiative and Tailscale!