Connect securely with your business

This manual describes the design and operation of Aclass Tier and its associated services, applications, and libraries. It is intended for IT professionals, network administrators, information security experts, and developers.

The first section (2) of this guide explains the high-level design and operation of Aclass Tier and is written for those with at least intermediate knowledge of topics like TCP/IP and Ethernet networks. It is not required reading for most users, but understanding how things work in detail helps to clarify everything else and greatly aids in troubleshooting if something goes wrong.

The remaining sections deal more specifically with deployment and management.

The Aclass Tier network hypervisor is a standalone network virtualization engine that implements an Ethernet virtualization layer similar to VXLAN over a globally encrypted peer-to-peer network.

The protocol used by Aclass Tier is original, although it has similarities to VXLAN and IPSec. It has two conceptually separate but tightly coupled layers in the sense of the OSI model: VL1 and VL2. VL1 is the underlying peer-to-peer transport layer, the “virtual cable,” while VL2 is an emulated Ethernet layer that provides operating systems and applications with a familiar communication medium.

In conventional networks, L1 (OSI layer 1) refers to the actual CAT5/CAT6 cables or wireless radio channels that transport data, and the physical transceiver chips that modulate and demodulate them. VL1 is a peer-to-peer network that does the same thing using encryption, authentication, and a bunch of network tricks to dynamically create virtual cables as needed.

Currently, Aclass uses external servers and provides its own servers for secure VPN network intercommunication. There are twelve root servers organized into two clusters of six members distributed across the major continents and multiple network providers. Almost everyone has one within 100 ms of network latency from their location.

A node can “orbit” any number of moons. A moon is just a convenient way to add user-defined root servers to the pool. Users can create moons to reduce dependency on Aclass Tier infrastructure or to locate root servers closer for better performance. For local SDN use, a cluster of root servers can be located within a building or data center so that Aclass Tier can continue to function normally if Internet connectivity is lost.

Nodes start without direct links to each other, only upwards to the roots (planet and moons). Each peer in VL1 has a unique 40-bit global address (10 hexadecimal digits), but unlike IP addresses, these are opaque cryptographic identifiers that do not encode any routing information. To communicate, peers first send packets “upward” the tree and, as they traverse the network, opportunistically create direct links along the way. The tree constantly tries to “collapse” to optimize itself to the traffic pattern it carries.

1. A wants to send a packet to B, but as it has no direct route, it sends it upstream to R (a root).
2. If R has a direct link to B, it forwards the packet there. If not, it sends the packet upstream until it reaches planetary roots. Planetary roots know all the nodes, so the packet will eventually reach B if B is connected.
3. R also sends a message called rendezvous to A that contains clues about how it might reach B. Meanwhile, the root forwarding the packet to B sends rendezvous informing B of how it might reach A.
4. A and B receive their rendezvous messages and try to send test messages to each other, possibly succeeding in punching holes in any NAT or stateful firewall that is in the way. If this works, a direct link is established, and packets no longer need to take the scenic route.

Since the roots forward the packets, A and B can communicate instantly. A and B then attempt to establish a peer-to-peer direct connection. If successful, a faster and lower latency link is achieved. We call this link provision-triggered transport, as it is the packet forwarding itself that triggers the peer-to-peer network to attempt the direct connection.

VL1 never gives up. If a direct route cannot be established, communication can continue through relay (slower). Attempts to establish a direct connection continue forever periodically. VL1 also has other features to establish direct connectivity, including LAN peer discovery, prediction of ports to traverse symmetric IPv4 NATs, and explicit port mapping using uPnP and/or NAT-PMP if available on the local physical LAN.

Each node is uniquely identified in VL1 by a 40-bit (10 hexadecimal digits) Aclass Tier address, which is calculated from the public part of a public/private key pair. The address, public key, and private key of a node form its identity.

In devices running Aclass Tier, the node’s identity is stored in identity.public and identity.secret in the main service directory.

When Aclass Tier is first started, it generates a new identity and attempts to announce it to the network. In the highly unlikely event that the unique 40-bit address of the identity is already taken, it discards it and generates another.

Identities are claimed on a first-come, first-served basis and currently expire after 60 days of inactivity. If a long inactive device returns, it can reclaim its identity unless its address has been taken in the meantime (again, highly unlikely).

The address derivation algorithm used to calculate addresses from public keys imposes a computational cost barrier against the intentional generation of a collision. Currently, it would take approximately 10,000 years of CPU time to do so (assuming, for example, an Intel 3ghz core). This is expensive but not impossible, but it is only the first line of defense. After generating a collision, an attacker would have to compromise all upstream nodes, network controllers, and anything else that has recently communicated with the target node and replace their cached identities.

Aclass Tier addresses are, once announced and claimed, a very secure method of unique identification.

When a node attempts to send a message to another node whose identity is not cached, it sends a whois query to a root. The roots provide an authorized identity cache.

If you don’t know much about cryptography, you can skip this section. TL;DR: the packets are end-to-end encrypted and cannot be read by the roots or anyone else, and we use modern 256-bit cryptography in the forms recommended by professional cryptographers who created it.

Asymmetric public key encryption is Curve25519/Ed25519, a variant of 256-bit elliptic curve.

Each VL1 packet is end-to-end encrypted using (starting from the current version) Salsa20 256-bit and authenticated using the message authentication algorithm (MAC) Poly1305. MAC is calculated after encryption (encrypt-then-MAC) and the encryption/MAC composition used is identical to the NaCl reference implementation.

Currently, we do not implement forward secrecy or other cryptographic features with state in VL1. We do not do so for the sake of simplicity, reliability, and code footprint, and because frequent state changes make features such as batching and error switching much harder to implement.

We may implement forward secrecy in the future. For those who want this level of security today, we recommend using other cryptographic protocols such as SSL or SSH over Aclass Tier. These protocols typically implement forward secrecy, but their use over Aclass Tier also provides the secondary benefit of defense in depth. Most cryptography is not compromised by a failure in encryption, but by errors in implementation. If you use two secure transports, the chances of a critical failure being discovered in both at the same time are very low. The CPU overhead of double encryption is not significant for most workloads.

To support the use of Aclass Tier as a high-performance SDN/NFV protocol over physically secure networks, the protocol supports a feature called trust paths. It is possible to configure all devices linked to a particular Aclass Tier network to bypass encryption and authentication of traffic through a designated physical path. This can significantly reduce CPU usage in high-traffic situations, but at the cost of virtually all transport security.

Trust paths do not prevent communication with devices from other locations, as traffic through other paths will still be encrypted and authenticated normally.

We do not recommend using this feature unless you really need the performance and know what you are doing. We also recommend thinking carefully before disabling transport security on a private cloud network. Larger cloud providers such as Amazon and Azure tend to provide good network segregation, but many cheaper providers offer private networks that are “party lines” and are not much more secure than the open Internet.

VL2 is built on top of VL1 and transported through it, inheriting VL1’s endpoint encryption and authentication, and can use VL1’s asymmetric keys to sign and verify credentials. VL1 also allows us to implement VL2 without worrying about the underlying physical network topology. Connectivity and routing efficiency issues are VL1’s concerns. It’s important to understand that there is no relationship between VL2 virtual networks and VL1 routes. Like VLAN multiplexing on a wired LAN, two nodes sharing multiple network memberships in common will only have one VL1 route (virtual cable) between them.

Each VL2 network (VLAN) is identified by a 64-bit (16 hexadecimal digits) Aclass Tier network ID, which contains the 40-bit Aclass Tier address of the network controller and a 24-bit number identifying the network on the controller.

Network ID: 8056c2e21c123456
| |
| Network number on the controller
|
Aclass Tier address of the controller

When a node joins a network or requests a network configuration update, it sends a network configuration query message (via VL1) to the network controller. The controller can then use the node’s VL1 address to look it up on the network and send it the appropriate certificates, credentials, and configuration information. From the perspective of VL2 virtual networks, Aclass Tier VL1 addresses can be thought of as port numbers on a huge global-scale virtual switch.

A common misunderstanding is to confuse network controllers with root servers (planet and moons). Root servers are connection facilitators that operate at the VL1 level. Network controllers are configuration managers and certification authorities that belong to VL2. Generally, root servers do not join or control virtual networks, and network controllers are not root servers, although it is possible for a node to do both.

Network controllers serve as certification authorities for Aclass Tier virtual networks. As such, their identity.secret files must be closely monitored and securely backed up. Compromising a controller’s secret key would allow an attacker to issue fraudulent network configurations or admit unauthorized members, while losing the secret key results in the loss of the ability to control the network in any way or issue configuration updates, effectively rendering the network unusable.

It is important that the system clocks of controllers are relatively accurate (within 30 to 60 seconds) and protected against remote manipulation. Many cloud providers provide secure time sources, either directly through the hypervisor or via NTP servers within their networks.

Credentials are only issued to their owners, and nodes wishing to communicate with other nodes in the network send them peer-to-peer. This allows networks to grow to enormous sizes without requiring nodes to cache a large number of credentials or constantly query the controller.

Aclass Tier networks support multicast through a simple publish/subscribe system.

When a node wants to receive multicasts for a specific multicast group, it announces its membership to that group to other members of the network it communicates with and to the network controller. When a node wants to send a multicast, it checks its cache of recent announcements and periodically requests additional announcements.

Multicast (Ethernet ff:ff:ff:ff:ff:ff) is treated as a multicast group to which all members subscribe. It can be disabled at the network level to reduce traffic if not needed. IPv4 ARP receives special treatment (see below) and will continue to work if normal broadcast is disabled.

Multicasts are propagated using simple sender-side replication. This places all outgoing multicast bandwidth load on the sender and minimizes multicast latency. Network configurations contain a network-wide multicast limit configurable on the network controller. This specifies the maximum number of other nodes any node will send a multicast to. If the number of known recipients in a particular multicast group exceeds the multicast limit, the sender chooses a random subset.

There is no global limit on multicast recipients, but setting a very high multicast limit in very large networks could cause significant bandwidth overload.

IPv4 ARP is based on simple Ethernet broadcast and scales poorly in large or distributed networks. To improve ARP scalability, Aclass Tier generates a unique multicast group for each detected IPv4 address in its system and then transparently intercepts ARP queries and sends them only to the correct group. This converts ARP into a narrowcast or multicast protocol (like IPv6 NDP) and allows IPv4 ARP to function reliably in wide area networks without excessive bandwidth consumption. A similar strategy is applied under the hood of a number of enterprise switches and WiFi routers designed for deployment in extremely large LANs. This ARP emulation mode is transparent to the operating system and application layers, but means that packet trackers will not see all ARP queries in a virtual network in the way they normally do in smaller wired LANs.

IPv6 addresses are large enough to easily encode Aclass Tier addresses. For faster operation and better scaling, we have implemented several special IPv6 addressing modes that allow the local node to emulate NDP. These are the Aclass Tier IPv6 address assignment schemes rfc4193 and 6plane. If these addressing schemes are enabled on a network, nodes locally intercept outgoing NDP queries for matching addresses and generate false NDP responses locally.

Both modes drastically reduce the latency of the initial connection between members of the network. 6plane further exploits NDP emulation to transparently assign a complete /80 IPv6 prefix to each node without requiring any additional entries in the routing table. This is intended for virtual machine and container hosts that wish to self-assign IPv6 addresses to guests and is very useful in microservices architecture backplane networks.

Finally, NDP emulation offers a security advantage. Aclass Tier addresses are cryptographically authenticated, and since Ethernet MAC addresses on networks are computed from Aclass Tier addresses, they are also secure. Therefore, the IPv6 addressing modes emulated by NDP are not vulnerable to NDP response spoofing.

Normal IPv6 addresses not emulated by NDP (including link-local addresses) can coexist with NDP-emulated addressing schemes. Any NDP queries that do not match NDP-emulated addresses are sent via normal multicast.

Aclass Tier emulates a true Ethernet switch. This includes the ability to bridge L2 other Ethernet networks (wired LAN, WiFi, virtual backplanes, etc.) to virtual networks using conventional Ethernet bridges.

To act as a bridge, a member of the network must be designated as such by the controller. This is for security reasons, as normal network members are not allowed to send traffic from any source other than their MAC address. Designated bridges also receive special treatment by the multicast algorithm, which aggressively and directly requests their subscriptions to groups and replicates all broadcast traffic and ARP requests to them. As a result, bridge nodes experience a slightly higher multicast bandwidth overload.

Bridging has been extensively tested on Linux using the native Linux kernel bridge, which cleanly handles network MTU mismatches. There are third-party reports on bridging operation on other platforms. The details of bridge configuration, including how to selectively block traffic such as DHCP that may not be desired over the bridge, are beyond the scope of this manual.

It is possible to deactivate access control in an Aclass Tier network. Members of a public network do not check membership certificates, and new members of a public network are automatically marked as authorized by their host controller. It is not possible to deauthorize a member of a public network.

On the other hand, rules are enforced, so it is possible to implement a special-purpose public network that only allows access to a few things or only allows a restricted subset of traffic.

Public networks are useful for testing and “party lines” among peers for games, chat, and other applications. Participants in public networks are advised to pay special attention to security. If you join a public network, be careful not to expose vulnerable services or accidentally share private files through open network shares or HTTP servers. Make sure your operating system, applications, and services are fully updated.

ffSSSSEEEE000000
| | | |
| | | End of port range (hex)
| | Start of port range (hex)
Reserved Aclass Tier address prefix indicating a controllerless network

Ad-hoc networks are public networks (without access control) that do not have a network controller. Instead, their configuration and other credentials are generated locally. Ad-hoc networks only allow IPv6 UDP and TCP unicast traffic (no multicast or broadcast) using emulated IPv6 NDP addresses in 6plane format. Additionally, an ad-hoc network ID encodes a range of IP ports. UDP packets and TCP SYN (open connection) packets are only allowed to destination ports within the encoded range.

For example, ff00160016000000 is an ad-hoc network that only allows SSH, while ff0000ffff000000 is an ad-hoc network that allows any UDP or TCP port.

Note that these networks are public and anyone in the world can join them. Care should be taken to avoid exposing vulnerable services or sharing unwanted files or other resources.

In a future version (not yet implemented in version 1.4.2), Aclass Tier will support new traffic prioritization rules based on link measurement functions. There will be nine categories available for traffic classification: [0, 1, 2, 3, (4), 5, 6, 7, 8], each with a geometrically increasing priority, with block 4 being the default in the absence of a classification rule for a specific traffic type.

For example, let’s say you want to prioritize your VoIP traffic over standard web traffic:

priority 6
ip protocol udp
and dport 5060-5065
and sport 5060-5065
;

priority 3
ip protocol tcp
and dport 80
and sport 80
;

This would place VoIP traffic on ports 5060 to 5065 at a higher priority 6 than standard web traffic on port 80 in block 3.