Agenda
- Overview – What is OFUV
- Architecture
- OFUV Components (userland)
- What were the challenges
- Installed files
- Kernel code
- Gates, source code layout and build
- Testing
- References
OFED components provided by OFUV
- Libraries:
- libibverbs, libmthca, libmlx4, librdmacm, libsdp
- Internal foundation libraries for utility implementation
- libibcommon, libibmad, libibumad, libosmvendor, libopensm, libosmcomp, ibdiagnet, libibis and libibdm.
- Commands and Utilities
- perftest, rds-tools, qperf, infiniband-diags & ibutils (ibdiagnet)
- See PSARC 2010/346 for details
OFUV components – libibverbs
- Allows userspace processes to use IB verbs.
- Direct hardware access for fast path operations.
- Need a device-specific plug-in module (libmthca or libmlx4).
- Uses special character device files (see PSARC/2010/239):
- ‘/devices/ib/sol_uverbs@0:uverbs#’ (# – minor node number)
- Detailed description of each library interface and example binaries can be found in the man pages (see PSARC/2010/346)
- Also delivers the ibv_* commands/tools/utilites
OFUV components – libmthca & libmlx4
- Provide device-specific userspace driver for use with libibverbs
- Allows programs to use HCA hardware directly from userspace.
- libmthca (for HCAs driven by the tavor(7D) driver) .
- libmlx4 (for HCAs driven by the hermon(7D) driver).
- Works as a plug-in module for libibverbs.
- Loaded & used automatically by programs linked with libibverbs
OFUV components – librdmacm
- Provides a userspace communication management API.
- Used to setup/teardown reliable connected and unreliable datagram communication.
- Based on sockets, but adapted for queue pair (QP) semantics.
- Works in conjunction with the verbs API (libibverbs – provides the
- interfaces needed to send and receive data.
- Uses a single device file regardless of the number of adapters or
- ports present (see PSARC/2010/239):
- ‘/devices/ib/sol_uverbs@0:ucma’ device.
- Also provides the rping, mckey, udaddy & ucmatose tools
- Detailed description of each library interface and example
- binaries can be found in the man pages (See PSARC 2010/346).
OFUV components – libsdp
- Sockets interposition library
- Transparently redirects sockets applications to use the Sockets Direct Protocol (sdp(7d)) driver instead of TCP/IP over IB.
- Not setup automatically. It can be used in one of two ways:
- LD_PRELOAD environment variable
- crle(1)
- Supports two modes of configuration:
- simple – converts all AF_INET socket(2) calls into AF_INET_SDP
- selective – a configuration file (libsdp.conf) selects which sockets will use SDP and which will not.
- See libsdp(3lib) and libsdp.conf(4) for more details.
OFUV components – Internal libraries #1
- Consumed by other OFUV libraries and utilities, no consumer
- interfaces defined.
- libibcommon
- provides common utility functions for OFUV diagnostic and management tools.
- libibmad
- provides low layer IB functions for use by other OFED MAD based IB diagnostic/management utilities and libraries, specifically libibumad.
- libibumad provides the user MAD library functions that interface into the sol_umad kernel component (see PSARC/2010/239).
OFUV components – Internal libraries #2
- libosmvendor, libopensm and libosmcomp
- A suite of shared libraries that are part of the OpenSM package but are
- also used by other OFED applications.
- Can be installed and used without OpenSM to satisfy dependencies of
- other applications.
- ibdiagnet, libibis and libibdm
- part of “ibutils” component, which provides diagnostic tools and utilities
- and a TCL toolkit for writing utilities.
- ibdiagnet provides a TCL IB network diagnostics library, is dependent
- on the ibis and ibdm packages.
- libibis is a TCL shell extended with an interface for sending and
- receiving IB management datagrams (MADs).
- libibdm provides a set of utilities that is based on a common data model
- to hold a complete fabric at both the system and node levels.
OFUV components – perftest
- Binaries that provide IB user verbs micro-benchmarks.
- ib_rdma_bw – streaming bandwidth test with RDMA write transactions
- ib_rdma_lat – latency test with RDMA write transactions
- ib_read_bw – bandwidth test with RDMA read transactions
- ib_read_lat – latency test with RDMA read transactions
- ib_send_bw – bandwidth test with send transactions
- ib_send_lat – latency test with send transactions
- ib_write_bw – bandwidth test with RDMA write transactions
- ib_write_bw_postlist – bandwidth test with list of RDMA write transactions
- ib_write_lat – latency test with RDMA write transactions
© 2010, www.oracledatabase12g.com. 版权所有.文章允许转载,但必须以链接方式注明源地址,否则追究法律责任.
相关文章 | Related posts:
最新评论