Skip to main content

How Husarnet works?

Husarnet is a global P2P network layer dedicated especially for robotic and IoT use cases with first class ROS support. It can however be used in much more areas such as, for example, automotive industry (V2V/V2X) or connecting distributed, critical resources like Smart Grids. It can be used in use cases typical for VPN networks as well.

Thanks to Husarnet you can connect your robots, servers, laptops and even microcontrollers to a single network which is independent from any external infrastructure. All traffic goes directly between your devices (when possible).

High-level architecture​

Other solutions​

Let's talk about the solutions used by classic VPN technologies/providers first.

Standard solution for connecting devices over the Internet uses client-server architecture. All devices connected over the Internet can communicate with each other only via the central server. This architecture seems to be simple, but you have to be aware of its drawbacks.

In case of a self-hosted service:

  • you have to maintain a central server
  • providing a reasonable level of encryption incurs a heavy load on the server, sometimes needing a specialized hardware

In case of a externally hosted service:

  • users can be spied by the service provider
  • devices will cease to work when the service provider stops supporting the servers

In any case:

  • devices need Internet access to function, LAN connectivity doesn't suffice
  • larger latency in communication between elements connected to the system
  • central server is a single point of failure

Solution provided by Husarnet​

Husarnet implements a handful of solutions that are meant to make your life better.

First of all Husarnet is using peer to peer architecture whenever possible. This means that the data exchanged between your devices is sent directly between them, with no central server in between if they can connect to each other directly. If they can't, there's a fallback method of exchanging the data over Husarnet servers, but don't worry - it's still encrypted so we won't be able to see the content of those messages. You'll also be able to check whether the connection is direct or tunnelled.

Secondly the traffic is sent using UDP instead of TCP which helps with reducing the latency over the network. This is one of the techniques that online games are using to make the experience for players better. There's a TCP fallback for very specific cases, but it's a last-resort.

We're running a microservices based architecture, where each of the components is responsible for a single task, which enables us to scale the infrastructure quickly and efficiently. For example - we're able to run multiple, geographically distributed "Base Servers" that are responsible for helping your devices connect to each other, and in case they're unable to do so - forward their traffic. This makes sure that you're getting as low latency as possible and makes the whole infrastructure more resilient to failures.

Husarnet Client is as standalone as possible - all the encryption, authorization and authentication is done locally - this makes you, the user in control of what's happening with your data/network. This also means that, for already established direct connections, the Client is able to forward traffic even if our Base Servers are down!

Husarnet Client, whenever possible, will try to connect to other devices via the local connections - it'll use either multicast messages, IPv6 link-local communication or a Base Server (by providing it with a list of both local and public addresses a Client can be reached at, which it can forward to the interested parties) to advertise its presence and contact details.

Husarnet, in it's core, is one big, automatically routed, IPv6 network (essentially what is called a software defined network). Running Husarnet daemon creates a virtual network interface with an unique Husarnet IPv6 address and associated fc94::/16 route. By default a Client will allow communication only with hosts on it's whitelist. Whitelist can be managed locally, using the CLI or remotely with the Husarnet Dashboard. This whitelist system can also be disabled entirely allowing connections from any other host in the network.

The nodes are identified by their 112-bit IPv6 addresses. The IPv6 address is based on hash of the node's public key. All connections are also authenticated by the IPv6 address. This property makes it possible to establish connection authenticity without any trusted third party, basing only on the IPv6 address! The connections are also always encrypted.

Husarnet Dashboard (and Husarnet Dashboard API) provide an easy way of managing your devices - you can create virtual networks there, add your devices to many virtual networks at once or even share your virtual networks with others! The Dashboard manages whitelists on your devices so only desired connections are allowed by the devices themselves. (Communication of the Clients with the Dashboard is being done via the special service on the Husarnet network called "Websetup".)

Husarnet server stack can be hosted in your infrastructure (self-hosted) in some of the plans. This means the whole stack - Base Server, Dashboard and Websetup. No connections to our infrastructure will be done, so you can even run it on the networks with no Internet connectivity.

Husarnet security​

Security was of an uttermost importance when designing Husarnet.

Cryptography: Husarnet uses X25519 from libsodium for key exchange, with ephemeral Curve25519 keys for forward secrecy. The hash of initial public key is validated to match the IPv6 address. The packets are encrypted using libsodium ChaCha20-Poly1305 secretbox construction with a random 192-bit nonce.

Runtime safety: Husarnet is written in C++ using modern memory-safe constructs. Linux version drops all capabilities after initializing. It retains access to /etc/hosts and /etc/hostname via a helper process.

If Husarnet instance is not connected to the Husarnet Dashboard, the whitelist and /etc/hosts can only be changed by a local root user. Otherwise, the owner of the Husarnet Dashboard account can also influence the configuration by adding the device to networks.

Details​

Minimal workflow​

So… what does the client actually do?

  1. Grabs the License from a Dashboard
  2. Connects to one of the Base Servers specified in the License
  3. Connects to the Websetup Server and updates it's settings

…and it's ready to rock!

How do licenses work?​

In order to support self-hosted deployments we introduced a licensing system into our infrastructure. The first thing Husarnet Client will do after start is grab a License file from a server (configured with husarnet setup-server [address] CLI command). This License file is hosted by the Husarnet Dashboard under http://dashboard-url/license.json. Two important remarks here. 1) It's hosted using plain HTTP. 2) The public, global infrastructure maintained by us, uses the license mechanism too (under http://app.husarnet.com/license.json, and if you ever want to move your Client back to public infrastructure, you can do so with husarnet setup-server app.husarnet.com).

Apart from authorizing the deployments, License files contain all the information crucial the the operation of a given network - mainly addresses of the Base Servers (IPv4, public ones) and an address of Websetup Server (IPv6, Husarnet-internal).

What's the role of a Base Server and how the connection is made?​

Once the Client gets all the metadata required for operation it needs, it'll try to connect to one of the Base Servers. Base Servers are essentially responsible for efficient handling of the data plane of the network. They'll help Clients connect to each other directly (either by passing their public and private contact info between them or by performing NAT traversal) and, in case Clients are unable to connect directly, they'll forward the (encrypted) data between them.

In case of the connection there are a couple of mechanisms here, most of which are meant to improve the chances of establishing a successful connection. For example a Client is usually presented with multiple Base Servers so it can fall back to others in case one is unavailable. Additionally, connection to the Base Server can be established both using the UDP protocol (that's the preferred one due to the lower latency) (port 5582) and TCP (on port 443 so it's able to pass some "clever" firewall setups).

What does Websetup do?​

Websetup is a special service available inside of the Husarnet Network (in fact it'll be seen as an almost regular peer) that is responsible for keeping Husarnet Client configuration in sync with one in the Husarnet Dashboard. It's most important task is to update the whitelist every time you make a change in the Dashboard, like add a Client to a new network. In such case whitelists on all other devices have to be updated to allow communication with this new host, and the whitelist on the new host has to be updated with entries for all the existing hosts.

What does Dashboard do?​

Dashboard is a command center for managing all your Clients. It's the place where you'll claim the ownership of Husarnet Client instances and add them to one or many networks, or even share them with others if you want to!

The public Dashboard is hosted at https://app.husarnet.com

Most of the settings stored in a Dashboard can also be changed using the Husarnet Dashboard API. If you need a programmatic way of controlling your networks - that's a solution for you!

Software Defined Network​

Husarnet at it's core is a Software Defined Network. All the data forwarding (data-plane) is handled by Husarnet Clients and Husarnet Base Servers and all the logic (control-plane) is configured using Husarnet Dashboard (API) and Husarnet Websetup, and enforced by Husarnet Clients.

Each of the Clients has a global, stable (as in non-changing in time) and unique (IPv6) address and all the connections between the Clients are guarded by the whitelists. (Have in mind that despite whitelist consisting of IPv6 addresses it actually is holding a cryptographic identities of other nodes. All the packets that can be accepted are required to be signed with a key matching that IPv6 address).

Whitelists can be managed using the Husarnet Dashboard (API) or can be managed locally using the CLI.

Hostname management​

Similar to the whitelist algorithm, there's a mechanism of keeping your /etc/hosts updated with the names of other hosts in your networks. It too can be managed either manually with a CLI or remotely using the Dashboard (and Websetup). It allows you to connect to your other devices using a convenient names, instead of long IPv6 addresses.

If you're having issues with the connections try our Linux Troubleshooting Guide or Windows Troubleshooting Guide.