Chapter 2. Network
This chapter briefly describes the network configuration defining our virtual organization. The way this network is built, and its characteristics, have crucially determined the choices we made afterwards, so it is necessary to start from here to understand the rest of the infrastracture.
The network configuration is mantained on each server using CFengine (see chapter 6).
2.1. Network Structure
The various server are phisically located in different countries and each one has its own public IP address. One of the characteristics of this configuration is the low bandwidth available for server intercommunication: the geographical distribution of the servers means that the server-to-server communication channel shares the same connection of the normal communication, the one on which people use the various services (see Appendix B for a more in-depth study). This is the single most important difference between the model we are describing and traditional clustering and high-availability solutions, that generally imply the existence of some high-speed local interconnection between the servers.
The internal communication channel uses a VPN (implemented through Tinc) connecting the various nodes of the network. This configuration allows for encrypted communications without the overhead of the cryptographic handshake on each connection (as SSL does).
We consider the VPN a trusted connection: the simple presence on the VPN means the server can be trusted as part of the network; the single connections on this ring do not need to be encrypted (since tinc already provides for it on a lower layer). Many internal administrative services (as well as the synchronizing mechanisms) are available only on the VPN interfaces.
The full structure of the network includes several other servers, organized in concentric and connected rings, defining the level of "reliability" of the servers. For instance we can consider that nodes on ring 1 (the ring numbering starts from 0) have only a restricted access to the users LDAP database. These boxes are in fact used only for minor services, for backups and other side-tasks as relay connection for the ssh and vpn access of the network administrators (a single non-public point of access to the network with the ensuing traffic properly routed anonymously allows the administrator to mantain a sufficient degree of privacy :).
In the node connecting the different rings, firewalls are set up to manage the flows of data between one ring and the other. Another possibility we still haven't checked out thoroughly is to set up servers with this specific inter-connecting role (gateway servers in the picture) instead of burdenind already overbusy servers with this function. The nodes connecting different rings should be multiple, but we are still studying the best way to route the traffic between the rings even if one of the nodes is unreachable.
2.1.1. VPN
Tinc is a VPN daemon with automated routing functions able to mantain N hosts connected with point-to-point connections. It also manages authentications and encrypting on an internal autonomous set of RSA keys that have then to be distributed accordingly to the software needs on the various servers.
It has been chosen among other solutions since it's very easy to install and allows to hide point-to-point connections between the single servers: it presents the OS with a single network interface, transparently managing the routing with its own internal procedures (e.g.: it is as though the boxes were all connecting to one another on a local network, or even better, since the software looks for alternate traffic routes if the direct connection between two nodes is failing).
Accordingly, we have chosen for our internal connections the network 172.16.0.0/16, distributing the ip addresses as follows (the addresses will be then added to the /etc/hosts file distributed through CFengine):
test1 172.16.1.1
test2 172.16.1.2
test3 172.16.1.3
To avoid confusion, it can be useful to choose for these addresses a domain different from the one that will be used for public IP addresses: for example a subdomain (test1.vpn.domain.org) or a common name-suffix (test1-vpn.domain.org) could be used.
As regards the configuration, we have created base configuration files for a network called ring0 (corresponding to the innermost ring in the trust-based VPN scheme) in /configfiles/ring0/common/tinc/ring0. Apart from the basic tinc-up and tinc-down files, the tinc.conf file is the main configuration location and it contains the following directives:
# Sample tinc configuration file
# The name of the machine
# Name = test1
# Connections to be done
ConnectTo = test1
ConnectTo = test2
ConnectTo = test3
# The tap device to use
Device = /dev/net/tun
Note that the names of the servers used in this file need not be valid dns fully qualified hostnames, but need to correspond to the files in the ring0/hosts directory. It's only for simplicity with CFengine use that the names we have chosen are identical to the names assigned to the hosts.
The /configfiles/ring0/common/tinc/ring0/hosts directory should contain two files for each server, identified with their names with no domain suffix: NAME and NAME.priv. The RSA keys can be generated with the command:
$ tincd -n ring0 -K
We then need to edit the file with the public key to add some options, as in the following example:
# The real IP address of this tinc host. Can be used by other tinc hosts. Address = 192.168.1.21 # Portnumber for incoming connections. Default is 655. Port = 655 # Subnet on the virtual private network that is local for this host. Subnet = 172.16.1.2/32 # Connection with the other ring # Subnet = 172.17.0.0/16 #IndirectData = Yes #TCPOnly = Yes # The public key generated by `tincd -n example -K' is stored here -----BEGIN RSA PUBLIC KEY----- MIGJAoGBAJGedJTd4GnIe1VssM+ROBwsMzRXbdI/reZvkLmji3YK0HJcyDIKnRZ2 /ikPJNyH1bKSWlqds28j4eG6ENM5ZjaWDETztW6OyNOt4vDxAXEQRF50WLBL5BOk e6bXlPoOtXWrVK/ZpBiDl86XpEdm0DHhETB2Cit9KNAXcW2ajnabAgMBAAE= -----END RSA PUBLIC KEY-----