The Network Manager

This chapter covers the following topics:

Introduction

The Network Manager (Net) gives QNX users a seamless extension of the operating system's powerful messaging capabilities. Communicating directly with the Microkernel, the Network Manager enhances QNX's message-passing IPC by efficiently propagating messages to remote machines. In addition, the Network Manager offers three advanced features:

Network Manager responsibilities

The Network Manager is responsible for propagating the QNX messaging primitives across a local area network. The standard messaging primitives used in local messaging are used unmodified in remote messaging. In other words, there's no special “network” Send(), Receive(), or Reply().

An independent module

The Network Manager does not have to be built into the operating system image. It may be started and stopped at any time to provide or remove network messaging capabilities.

When the Network Manager starts, it registers with the Process Manager and Microkernel. This activates existing code within the two that interfaces to the Network Manager. This means that network messaging and remote process creation are not just a layer added onto the operating system. Network messaging is integrated into the very heart of the messaging and process-management primitives themselves.

This deep integration at the lowest level gives QNX its network transparency and qualifies it as a fully distributed operating system. Since applications access all services via messaging, and since the Network Manager allows messages to flow transparently on the network, QNX nodes perform together as a single, logical computer.

Since the Network Manager and Microkernel are distinct, the Microkernel can achieve independence from the network hardware, and non-networked machines can benefit from reduced code requirements.

Microkernel/Network Manager interface

The Microkernel and Process Manager interface to the Network Manager via a special nonblocking memory queue. This queue is essentially a list of transmissions to be performed by the Network Manager. The entries contain all the information for a particular operation (e.g. Send(), Reply(), VC creation, remote signal propagation, etc.).

Another resource the operating system uses to provide transparent messaging is the virtual circuit buffer. Allocated when a process creates a VC, virtual circuit buffers hold the data during the completion of a messaging transaction to another node.

The following diagrams outline the flow of data and control for sending and receiving remote messages.

Sending a message to a remote node

A process issues a Send() or Reply() to a remote node.

fig: ./images/sendrmot.gif

In the case of a Send() or Reply() to a remote node, the following events occur:

  1. The process calls Send() or Reply() and the Microkernel puts the data from the process's data space into the virtual circuit buffer associated with the VC specified by the Send() or Reply().
  2. The Microkernel enqueues to the Network Manager a time-ordered queue entry identifying the sender, the remote receiver, and pointers to the data in the virtual circuit buffer. If the queue was previously empty, the Network Manager's proxy is triggered to let it know new work has arrived.
  3. The Network Manager dequeues the queue entry.
  4. The Network Manager passes the entry to an appropriate network driver.
  5. The network driver begins transmission on the network media and is responsible for delivery.

In the case of a signal propagation or VC creation, the Process Manager rather than the Microkernel would enqueue a control packet. Nevertheless, the Network Manager would transmit the packet to its destination.

Receiving the message on the remote node

A process receives a remote Send() or Reply().

fig: ./images/recvrmot.gif

Assuming the remote node will have sent a message as previously described, the following events occur on the node receiving the message:

  1. The Network driver puts the message data from the network media into the appropriate virtual circuit buffer.
  2. The Network driver informs the Network Manager that the reception is complete.
  3. The Network Manager uses a private kernel call to inform the Microkernel that the reception is complete.
  4. The Microkernel puts the data from the virtual circuit buffer into the process's buffer (assuming it was RECEIVE- or REPLY-blocked on this virtual circuit).

Any control packets the Network Manager receives are forwarded immediately to the Process Manager via the standard Send() primitive. These control packets are used for signal propagation and VC creation.

Network drivers

Like the Filesystem Manager and the Device Manager, the Network Manager contains no hardware-specific code. This functionality is provided by network card drivers. The Network Manager can support multiple network drivers at one time. Each driver typically supports a single network card. You may have drivers/cards of the same type or different types — for example, two Ethernet drivers/cards or perhaps an Ethernet driver/card and an Arcnet driver/card.

Shared memory queues provide the interface between the Network Manager and drivers. This interface has been designed to obtain the maximum performance possible. The driver determines the appropriate protocol for the network media.

The driver is responsible for packetization, sequencing, and retransmission if reliable guaranteed data transmission to a remote physical node is requested. This is the default for all QNX messaging primitives. This design allows QNX to easily support new network hardware and protocols by writing or modifying only a network driver.

Node and network identifiers

Each node in a local area network is identified by two numbers:

  • its physical node ID (or IDs if the node has more than one network card)
  • its logical node ID.

Physical node ID

The physical node ID is determined by the hardware. Network cards communicate with each other by specifying the physical node ID of the remote node they wish to talk to. In the case of Ethernet and Token Ring, this represents a large number that is difficult for people — and utilities — to deal with. For example, each Ethernet and Token Ring card is shipped with a unique 48-bit physical node ID, conforming to the IEEE 802 standard. Arcnet cards, on the other hand, have only an 8-bit ID.

Physical node IDs entail a significant drawback: when interconnecting some networks, addresses may conflict (especially in the case of Arcnet) or be of a radically different format.

Logical node ID

To overcome the above problems with physical node IDs, each QNX node is given a logical node ID. All QNX processes deal with logical node IDs. The physical node IDs are hidden from processes running on QNX.

Logical node IDs simplify network and application licensing. They also make it easy for some utilities that may wish to poll the network using a simple loop, where the logical node ID goes from 1 to the number of nodes.

The mapping between logical and physical node IDs is done by the Network Manager. The driver is given a physical ID by the Network Manager when asked to transmit data to another node.

The logical node ID is typically assigned sequential numbers starting at 1. For example, a node with an Ethernet card may be given a logical node ID of 2, which is mapped to the physical node ID of 00:00:c0:46:93:30.

Note that the logical node ID must be unique for all nodes across all interconnected QNX networks in order for network bridging to work.

Logical network ID

The network ID identifies a particular logical network. A logical network is any hardware that allows a network driver to directly communicate with a network driver on another node. This may be as simple as a serial port or as complex as an Ethernet network with hardware bridges.

In the following diagram, node 7 has two network cards that allow the node to access nodes on logical networks 1 and 2. Nodes 8 and 9 both have three cards, connecting them to networks 1, 2, and 3.

Note that every logical node ID is unique across all three logical networks.

Logical network and logical node IDs are assigned by the system administrator. For more information, see Network Installation in the Installation & Configuration guide.

Multiple physical networks happily coexist via logical networks.

fig: ./images/multinet.gif

Choosing a network

In the case where nodes are connected by more than one logical network, the Network Manager has a choice of which network to use when transmitting to a remote node. For example, in the above diagram, node 7 could transmit to node 8 using the driver attached to logical network 1 or logical network 2.

Load balancing

Network throughput is determined by a combination of the speed of the computer and the speed of the network. If your computer can provide data faster than the network can accept it, then the network will limit your throughput.

For example, two Pentium computers connected by a 10BASE-T Ethernet network will be limited to a maximum of 1.1 million bytes per second, which is the data rate provided by the Ethernet hardware. If, however, you were to place two Ethernet cards in each computer and connect them with separate cables, the Network Manager could transmit data over both networks simultaneously. Under a heavy load, this would provide up to twice as much throughput as a single network.

The Network Manager will automatically load-balance by choosing an appropriate network driver. In our example above, if a transmission is in progress from node 7 to node 8 on logical network 1, and another transmission to node 8 is initiated on node 7, then logical network 2 will automatically be chosen to transmit the data.

Fault tolerance

When nodes are connected by two or more networks, there's more than a single path to use for communication. If a card in one network fails in a way that prevents any communication on that network, the Network Manager will automatically re-route all data through another network. This happens automatically without any involvement on the part of the application software, and results in transparent network fault tolerance. If cables for the different networks are run using separate routes, you'll also be protected against an accidental cable cut.

You can also construct tandem systems in which two machines are connected by a fast network for normal operation and by a cheaper, slower network (e.g. a serial link) that remains on standby. If the first network fails, communication will continue, although throughput would naturally be reduced.

Bridging between QNX LANs

The Network Manager allows any node to act as a bridge between two separate IEEE 802-based QNX networks.

Since QNX uses the same packet format and protocol on all IEEE 802-based networks, you can create bridges between Ethernet, Token Ring, and FDDI networks.

Arcnet networks cannot be bridged.

Consider the following diagram, where node 17 and node 18 are on one network, and node 18 and node 19 are on another network:

Network bridging between two IEEE 802-based QNX LANs.

fig: ./images/relay.gif

Nodes 17 and 18 are on the same network, so they can talk directly to each other. The same is true for nodes 18 and 19. But how can node 17 talk to node 19?

Since both the LANS involved are IEEE 802-based, node 18 automatically bridges packets, allowing node 17 and node 19 to create a virtual circuit. Although they're not connected to the same LAN, nodes 17 and 19 can nevertheless talk to each other as if they were.