Husarnet Docker Usage

info

Supported architectures: x64 (amd64), armhf, arm64

Docker technically is using Linux under the hood, but we've made a couple of extra tweaks specific for Docker so it's easier to use, hence it got a special "platform" entry in our ecosystem.

The first question one must ask is which method of integration will be best for a particular usecase. In general we recommend the sidecar method described below, but other options (like building the Husarnet into your Docker images) are possible too. Some of them will be described below.

Here's a table that should help you choose:

Pros and cons of both solutions

	Sidecar	Built in binary
Updates	Updates are provided by Husarnet	You need to keep track of the Husarnet Client updates in your images yourself
Ease of use	Add our container to compose file, change the network namespace of you workloads and you're done!	You need to keep track of our releases more closely as we occasionally change the dependencies/modify scripts, etc.
Shared Husarnet IP and configuration	Yes. In as many containers as you want	In a basic form no. You'd need to replicate our shared network setup in order to make it so
Can be run in a single container	No. In a single compose setup - yes, but in a single container - no.	Yes. You can easily start it using a single docker command invocation if you want
Can my app run without NET_ADMIN permission	Yes. Only Husarnet container needs to have NET_ADMIN permission to make an initial setup	No. Hence you're bundling your app and Husarnet your container needs to be running with a superset of all permissions needed for these
Can it run on Kubernetes	Yes. Sidecar pattern is a recommended way of augmenting your pods with new features	Yes, but it's not a recommended setup in any of the cases (even if you know what you're doing)

Shared environment variables

Variable name	Aliases	Example value
HUSARNET_JOIN_CODE	JOINCODE HUSARNET_JOINCODE	fc94:b01d:1803:8dd8:b293:5c7d:7639:932a/UcofxnS3MzRunForest94Q UcofxnS3MzRunForest94Q
HUSARNET_HOSTNAME	HOSTNAME	my-app
HUSARNET_DEBUG		1, true
HUSARNET_WAIT_HOSTNAMES	WAIT_HOSTNAMES	hostname1 hostname2 hostname1,hostname2

info

Aliases, HUSARNET_WAIT_HOSTNAMES and HUSARNET_DEBUG are available only if you're using the official Docker image for Husarnet. In case you're making your own you'll need to provide equivalent mechanisms yourself (in case you need them). You'll find tips how to do that in appriopriate section below.

HUSARNET_JOIN_CODE

This is the usual join code that you'll find in the dashboard. You can use both the full version, or you can strip the websetup address and use the short one. There are a couple of aliases provided to keep compatibility with the old versions, but you really should be using HUSARNET_JOIN_CODE as it's the one parsed using the actual Husarnet Daemon.

Also - yes, you now can provide a join code directly to the daemon with an environment variable and there's no need for extra husarnet join calls in that case. This works on a non-Docker Husarnet too!

HUSARNET_HOSTNAME

Docker containers will have some random names assigned to them which is not really useful when you want to manage them from the Dashboard. Regular calls for husarnet join can have an additional hostname argument provided so you can override the default hostname and this is the variable doing exactly that - all join calls will have it's value as hostname. This does not change the hostname of a whole container, but it does change the one reported to the dashboard.

HUSARNET_DEBUG

This variable is specific to the Husarnet Docker sidecar and not available in the regular binary. With this flag all output from Husarnet container will be suppresed. As our container is a bit verbose we chose to add this feature. For most cases though, providing this flag is a good idea as any actual error messages will be printed there.

HUSARNET_WAIT_HOSTNAMES

This variable is specific to the Husarnet Docker sidecar and not available in the regular binary. By default healthcheck available in the image will wait for a Husarnet Daemon to become responsive via the API. If you use this variable though it won't mark itself as "healthy" unless those specific hostnames are available. In case of a first run this means that Husarnet daemon must be healthy, connected to base server and websetup, registered registered in websetup and given the updates for host table. For all of the following runs the requirements may be loosened up a bit, but the guarantee is that those hostnames will resolve (via the /etc/hosts).

This may be useful i.e. when your apps are connected to your nginx server acting as a reverse proxy via Husarnet. Nginx needs the backend names to resolve before it's able to start so with this variable you'd be able to provide it with them.

Using sidecar image with a Docker compose

There are three basic methods of using a Husarnet sidecar.

First is an easy one - you already have an app running in compose and want to do as little changes to it's deployment as possible, so you want to augment it with Husarnet, not the other way around.

Second is a little bit more convoluted - you want multiple apps running in compose to have a single Husarnet IP - this will require a little bit more changes in the existing declarations but it's actually the most recommended way for workloads like ROS.

Third option is a "big gun" (necessary in some setups though) - deploying Husarnet in a network namespace of a host running Docker. This will make both all the apps and the host be able to connect to other Husarnet hosts and make them available via the Husarnet network. This is the most straightforward when it comes to port mappings, but this means that you can't be running a separate standalone Husarnet instance on a Docker host.

Examples

info

We will be using pidpawel's CDI as an example app. It will allow us to configure the port it's listening on internally. It will be an important part of the setup as in some cases Husarnet will run from the same network namespace as the app - thus the app will be exposed using the internal port numbers, not the ones stated in ports section.

info

If you'll be testing using IPv6 addresses and not hostnames, remember that those need to be enclosed in square brackets when used in URLs. Example: http://[fc94:…]:8080

Adding Husarnet to existing/single container

services:
  my-app:
    image: pidpawel/container-debugging-image:latest
    restart: unless-stopped

    environment:
      # This means our app will listen on port 80 inside it's network namespace
      HTTP_PORT: 80
      DEBUG_HTTP: true
      DEBUG_SERVER: true
      DEBUG_CLIENT: true

    ports:
      - "8080:80"

    # Husarnet is using IPv6 for it's overlay network so we need to make sure
    # it's not disabled in the app container either.
    # **You need to add it to your app in most cases**
    sysctls:
      - net.ipv6.conf.all.disable_ipv6=0

  husarnet:
    image: husarnet/husarnet:latest

    volumes:
      # This will persist your Husarnet Client keys, thus IP of the container
      # will be stable/the same between (re)boots.
      # Feel free to choose any volume storage method of your liking
      - ./husarnet-config:/var/lib/husarnet

    # This is required to create new interface for communication over Husarnet
    cap_add:
      - NET_ADMIN

    # See `Shared environment variables` section
    environment:
      - HUSARNET_HOSTNAME=my-app-deployment
      - HUSARNET_JOIN_CODE=UcofxnS3MzGo69poeuA94Q
      - HUSARNET_DEBUG=1

    # This line actually "injects" Husarnet Daemon into your app's container network
    network_mode: service:my-app

Some explanation is required for this example. Here we're "injecting" the Husarnet Daemon to the network created especially for your app's container. This means that:

1. Husarnet network will be available only for my-app container
2. And, more importantly, you'll be exposing the network image seen from inside that container. This means that via the Husarnet network other hosts will need to connect to port 80 and not 8080 (as would be the case from other hosts in the local network or the Docker host). This is a super important difference and also one that's easy to forget.
3. It's not possible to utilize the depends_on and healthcheck - it would result in a cyclic dependency

Adding Husarnet to multiple containers

This requires adding your apps to the Husarnet sidecar's networks (so it's the other way around) and comes with it's own set of quirks.

services:
  my-app1:
    image: pidpawel/container-debugging-image:latest
    restart: unless-stopped

    environment:
      # This means our app1 will listen on port 81 inside **Husarnet's** network namespace
      HTTP_PORT: 81
      DEBUG_HTTP: true
      DEBUG_SERVER: true
      DEBUG_CLIENT: true

    # This section makes your app1 move into the namespace of Husarnet.
    # This means you no longer can configure ports and other networking settings here
    # and all of those configs need to be moved to declaration of service `husarnet`.
    # This will also wait on Husarnet to actually start before starting your app
    network_mode: service:husarnet
    depends_on:
      husarnet: { condition: service_healthy }

  my-app2:
    image: pidpawel/container-debugging-image:latest
    restart: unless-stopped

    environment:
      # This means our app2 will listen on port 82 inside **Husarnet's** network namespace
      HTTP_PORT: 82
      DEBUG_HTTP: true
      DEBUG_SERVER: true
      DEBUG_CLIENT: true

    # This section makes your app2 move into the namespace of Husarnet.
    # This means you no longer can configure ports and other networking settings here
    # and all of those configs need to be moved to declaration of service `husarnet`
    # This will also wait on Husarnet to actually start before starting your app
    network_mode: service:husarnet
    depends_on:
      husarnet: { condition: service_healthy }

  husarnet:
    image: husarnet/husarnet:latest

    volumes:
      # This will persist your Husarnet Client keys, thus IP of the container
      # will be stable/the same between (re)boots.
      # Feel free to choose any volume storage method of your liking
      - ./husarnet-config:/var/lib/husarnet

    # This is required to create new interface for communication over Husarnet
    cap_add:
      - NET_ADMIN

    # Husarnet is using IPv6 for it's overlay network so we need to make sure
    # it's not disabled
    sysctls:
      - net.ipv6.conf.all.disable_ipv6=0

    # See `Shared environment variables` section
    environment:
      - HUSARNET_HOSTNAME=my-app-deployment
      - HUSARNET_JOIN_CODE=UcofxnS3MzGo69poeuA94Q
      - HUSARNET_DEBUG=1

    # As your apps are now running in Husarnet's container network namespace,
    # you need to move all the usual networking-related declarations here
    ports:
      - "8081:81"  # for app1
      - "8082:82"  # for app2

As you can see, config here is much different from a previous case. The main implications of this setup are:

1. All of your apps are running in the same network namespace, which means that they can't be running on the same ports internally.
2. Husarnet will still expose the internal ports as it's running in that namespace too.
3. You need to move all your networking related declarations from your apps to the husarnet service definition. Most importantly this goes for ports definitions. If you want to expose your services additionally to hosts from local network (possibly on different ports) you need to make do in husarnet service definition.
4. This is actually an optional step - your apps can utilize the healthchecks and dependencies do Husarnet starts before your app.

Adding Husarnet to the whole machine

Before you start using this config, make sure that you change sysctl -w net.ipv6.conf.all.disable_ipv6=0. As Husarnet is running in your host's network you need to make sure it's capable of handling IPv6 traffic for Husarnet's overlay network.

services:
  my-app:
    image: pidpawel/container-debugging-image:latest
    restart: unless-stopped

    environment:
      # This means our app will listen on port 80 inside it's network namespace
      HTTP_PORT: 80
      DEBUG_HTTP: true
      DEBUG_SERVER: true
      DEBUG_CLIENT: true

    # This will expose your app on port 8080 for **both** the external world
    # and Husarnet network in this case
    ports:
      - "8080:80"

  husarnet:
    image: husarnet/husarnet:latest

    volumes:
      # This will persist your Husarnet Client keys, thus IP of the container
      # will be stable/the same between (re)boots.
      # Feel free to choose any volume storage method of your liking
      - ./husarnet-config:/var/lib/husarnet

    # This is required to create new interface for communication over Husarnet
    cap_add:
      - NET_ADMIN

    # See `Shared environment variables` section
    environment:
      - HUSARNET_HOSTNAME=my-app-deployment
      - HUSARNET_JOIN_CODE=UcofxnS3MzGo69poeuA94Q
      - HUSARNET_DEBUG=1

    # This will make Husarnet use your host's networking namespace
    network_mode: host

In this example the deployment of your app can be done in a totally separate compose file or even not in a Docker. It's left here as a pure example but can be easily decoupled. As per the quirks for this deployment:

1. You can't be running Husarnet on the host and in a container with network_mode: host at the same time.
2. There can only be one container running Husarnet with network_mode: host at any given time. Because of that it may be a good idea to have a totally separate compose file, just for Husarnet and treat it as a global one.
3. Husarnet network won't have access to any of "internal" ports of your app. You'll "see" exactly the same view of the network your Docker host will.
4. Because of that you need to use ports from ports declaration of your app - this means via Husarnet you'll need to use port 8080 and not 80 (in this particular example).

Adding Husarnet to your own container

FROM ubuntu:22.04
ARG DEBIAN_FRONTEND=noninteractive

RUN apt-get update -y && apt-get install -y curl
RUN curl -s https://install.husarnet.com/install.sh | bash

# python3 -m http.server is an example app here
# Feel free to use any method of running husarnet-daemon, but make sure it
# actually runs in the background (you can even use supervisord for that)
SHELL ["bash", "-c"]
CMD (husarnet-daemon &) && python3 -m http.server --bind ::

# If you want to suppress the logs from Husarnet
# (equivalent to running Husarnet without HUSARNET_DEBUG)
# SHELL ["bash", "-c"]
# CMD (husarnet-daemon 2>/dev/null &) && python3 -m http.server --bind ::

There are many installation methods available - you can use the recommended install script, you can get the package from install.husarnet.com manually and dpkg -i it, you can get a tar file from that same page, or even you can build Husarnet yourself and embed it (see platforms/docker for some guidance on that). We do recommend sticking to the install script though - we're keeping that method most updated (including some platform fixes like the recent changes in apt-key management).

You don't need to start husarnet join manually. You can use environment variables to set the join code and reported hostname. See the Shared environment variables table above. Have in mind that in this setup aliases won't work and you need to use the full variable names from the left most column.

If you want to use healthchecks you need to integrate Husarnet healthcheck with your own. In the most generic case running husarnet daemon wait daemon in addition to your app's healthcheck will sufice. If you need a more complicated setup have a look at husarnet daemon wait --help - you'll find there many other checks and some may fit your case without requiring you to write any additional code.

Running custom image via Docker CLI

# Technically you shouldn't need to run `docker` with `sudo` but we're storing
# Husarnet config in the same directory so there's a permission mismatch
sudo docker build -t husarnet-combo .

# You can skip the `--rm -it` options depending on your case
docker run --rm -it \
  --sysctl net.ipv6.conf.all.disable_ipv6=0 \
  --cap-add NET_ADMIN \
  --volume "${PWD}"/husarnet-config:/var/lib/husarnet \
  --env HUSARNET_HOSTNAME=my-app-deployment \
  --env HUSARNET_JOIN_CODE=UcofxnS3MzGo69poeuA94Q \
  --env HUSARNET_DEBUG=1 \
  --publish 8000:8000 \
  husarnet-combo

The remarks are exactly the same as they're for the compose version below:

1. Sysctl is needed to enable IPv6 handling in the kernel. Husarnet uses it in it's overlay network.
2. NET_ADMIN capability is required to create a hnet0 interface.
3. Storing volumes is totally up to you but if you want a stable identity (IP address on Husarnet network) you need to persist /var/lib/husarnet somehow.
4. Environment variables are the recommended way of configuring the daemon.
5. --publish declaration is for python3 -m http.server that we're using as an example app. Fell free to change it as you wish.

Running custom image via Docker compose

services:
  husarnet-combo:
    build: .
    image: husarnet-combo

    volumes:
      # This will persist your Husarnet Client keys, thus IP of the container
      # will be stable/the same between (re)boots.
      # Feel free to choose any volume storage method of your liking
      - ./husarnet-config:/var/lib/husarnet

    # This is required to create new interface for communication over Husarnet
    cap_add:
      - NET_ADMIN

    # Husarnet is using IPv6 for it's overlay network so we need to make sure
    # it's not disabled
    sysctls:
      - net.ipv6.conf.all.disable_ipv6=0

    # See `Shared environment variables` section
    environment:
      - HUSARNET_HOSTNAME=my-app-deployment
      - HUSARNET_JOIN_CODE=UcofxnS3MzGo69poeuA94Q
      - HUSARNET_DEBUG=1

    # Those ports come from `python3 -m http.server` that we're using as an example app.
    # Feel free to change them as you wish
    ports:
      - "8000:8000"