This site has been archived. For information on the GN Project’s eduPERT initiative please visit https://archive.geant.org/projects/gn3/geant/services/edupert/Pages/Home.html

InfiniBand

InfiniBand is a high-performance network architecture developed by the IBTA (InfiniBand Trade Association). IBTA is a group of 220 or more companies founded in 1999 with the purpose of developing a technology which would substantially outstrip the capabilities of industry-standard I/O systems.

The InfiniBand architecture (IBA) is mainly based on switching fabric and is designed for use in I/O networks, such as storage area networks (SAN) or cluster networks, where low latency plays an extremely important role.

A large IBA network is composed of different subnets joined together by routers.

IBA Subnet

The elements of a subnet are end nodes, switches, links, and a subnet manager.

InfiniBandSubnet.png

End nodes exchange messages routed by switches over links. No routing exists in IBA. A subnet discovery is performed at the beginning by the Subnet Manager.

IBA offers point-to-point bidirectional links with five different signalling data rates:

  • SDR - Single Data Rate with signalling rate equal to 2.5 Gbit/s in each direction per connection;
  • DDR - Double Data Rate with 5 Gbit/s per lane;
  • QDR - Quad Data Rate with 10 Gbit/s per lane;
  • FDR - Fourteen Data Rate with 14.0625 Gbit/s per lane;
  • EDR - Enhanced Data Rated with 25.78125 Gbit/s per lane.
    The physical links can be aggregated in parallel (4X, 8X or 12X) to achieve greater bandwidth.
  • The IBTA‘s InfiniBand™ roadmap:

IBTAsBandwidth.png

In IBA, QoS is supported through Virtual Lanes (VL) which also provide a mechanism to avoid Head of Line (HoL) blocking. VLs are separate logical communication links which share a single physical link. As a packet traverses the subnet, a Service Level (SL) is defined to ensure its QoS level. Each link along a path can have a different VL, and the SL provides each link with a desired priority of communication.

Each switch/router has an SL to VL mapping table that is set by the subnet manager to keep the proper priority with the number of VLs supported on each link. Therefore, the InfiniBand Architecture can ensure end-to-end QoS through switches, routers and over the long haul.

References

– Main.AlessandraScicchitano - 03 May 2012

  • No labels