Inferno Networking

Any Inferno system configuration may be made up of many distributed clients and servers. These distributed machines can be called Inferno agents, since the Inferno system supports peer-to-peer networking: any machine can function as both server and client. The Inferno system does not dictate any rules or restrictions on where these machines are located - they may be in the next room or across the country. Additionally, machines running the Inferno system may be connected to each other by a variety of physical media and network protocols as required to maximize connectivity and throughput.

The Inferno system hides network complexity from applications. When an application wishes to access services that reside on remote servers, a few simple operations are used to construct a single, locally-represented namespace that may encompass multiple machines and multiple networks.

Like other services in the Inferno system, network interfaces are represented as file systems in a local namespace. The Styx communications protocol unifies different parts of the local namespace that may be constructed from the vast collection of files on the network. File operations between any two machines are accomplished by sending requests and receiving replies via Styx messages. The protocol supports remote access to files and allows a remote machine to use these interfaces as gateways.

Styx messages are not manipulated directly by an application developer. Any file operation generates the appropriate Styx messages to perform the necessary operations on the namespace. The exact sequence of messages differs depending on the call, but the complexity is concealed from the developer, who simply invokes familiar open, read, and similar file system operations.

Styx runs on top of any reliable transport level protocol, making the underlying network connection transparent to applications. Styx relies on several properties of the underlying transport protocol. For example, it assumes that messages arrive reliably and in sequence.

Establishing a network namespace

A server file system is incorporated into a client namespace using the kernel mount and bind calls. For example, if a client has access only to a TCP/IP network and requires access to other networks, the client may mount these networks from a server that has access to them by constructing a union of the client and server directories conventionally named /net. When attached to the local namespace, the remote server networks appear to be local to the client.

Figure 1-3 shows the /net file tree for a client that only has access to a TCP/IP connection. The /net file tree on the server called <server> has access to multiple connections.

Figure 1-3 An Example of Two Network File Trees

The client creates a namespace that incorporates the /net file system from the server by issuing a mount and bind sequence of commands similar to the following from the Inferno shell:

$ mount net!server /n/remote
$ bind -a /n/remote/net /net

After the mount and bind commands have executed successfully, all of the listed networks that are connected to the <server> are available to the client. The client can now send requests intended for these networks via the server. The server thus acts as a gateway without the need for a client protocol stack.

Following the mount / bind sequence described above, a list of directories at the client machine would look like the following:

	$ ls /net
	cs
	atm
	udp
	tcp
	tcp

Note that there are two /net/tcp directories resulting from the union of the local and remote directory structures. Both the local and remote versions of this directory are part of the local namespace. Only the first, on the local machine, is accessible; using the -b option in the bind command would make only the remote /net/tcp directory accessible.

Network Connection Server Applications

A client attached to a server may be able to use all of the networks to which the server has access.

The Inferno server application named Cs hides the details of call setup from applications. Cs is provided as a demonstration server application.

A network connection is established in two phases. The Cs application is first contacted to translate a logical network name into a device name and a connect address string. The Cs service reads a logical name such as tcp!<server>!styx and writes the device name and connect address string:

/net/tcp/clone 

connect 35.104.9.52!6666

The path name identifies the network device; the connect address string is used to set up a connection. To establish the connection, the client then opens /net/tcp/clone and writes the control message into it.

When /net/tcp/clone is opened, it actually returns a file descriptor to:

/net/tcp/<number>/ctl

where <number> is a numeric directory name (Figure 1-3). Text commands are then written to the ctl file to control the connection.

Each network directory contains a clone file and a directory for each connection of that type, numbered 0 through n, where n+1 is the number of connections of a single type. Opening the clone file finds and reserves an unused connection directory and opens its ctl file. The file descriptor returned to the client by the open call points to the ctl file. Each connection directory contains files to control a single connection and to send and receive information to it.

TCP Connections

TCP is the most common protocol used in the Inferno system. A TCP connection is represented by the files indicated in Figure 1-3 under the /net/tcp/0 directory. The organization of the TCP file system is similar to that of other network connections types. The function of each of these files is described in the following sections.

Control file

Reading the ctl file descriptor returns a text string containing the connection number. The client program uses this value to construct the full name of the connection directory. The client program can read and write the file descriptor returned by the open call to send and receive control messages to the driver or connection. A connection remains established while any of the files in the connection directory are referenced or until a close message is received from the network.

Data file

The data file provides access to the media. Reading the data file returns data received across the network, blocking the file until some data is available. Writing to the data file prefixes a header and queues data for transmission.

Status file

The status file may be read to yield protocol-dependent information about the state of the interface.

Local and remote files

The files local and remote contain an IP address and port number for each end of the connection.

Listen file

The listen file is used to accept the incoming calls from the network. If a process opens the listen file, the process will block until an incoming call is received.

[Top] [Prev] [Next] [Bottom]

infernosupport@lucent.com