Ubuntu 10.04 LTE do not support Infiniband. NO opensm package.

Upgrading from 10.04 LTE to 10.10

install the update-manager-core package if it is not already installed:

sudo apt-get install update-manager-core

Edit /etc/update-manager/release-upgrades and set Prompt=normal;

Launch the upgrade tool with the command

sudo do-release-upgrade -d

and follow the on-screen instructions.

Checking hardware

root@DKAM1:~# <input>apt-get -y install infiniband-diags</input>
root@DKAM1:~# <input>apt-get -y install opensm</input>
root@DKAM1:~# <input>modprobe ib_sa</input>
root@DKAM1:~# <input>modprobe ib_cm</input>
root@DKAM1:~# <input>modprobe ib_umad</input>
root@DKAM1:~# <input>modprobe ib_addr</input>
root@DKAM1:~# <input>modprobe ib_uverbs</input>
root@DKAM1:~# <input>modprobe ib_ipoib</input>
root@DKAM1:~# <input>modprobe ib_ipath</input>
<notice>May report device busy. Then it's already loaded</notice>
root@DKAM1:~# <input>modprobe ib_qib</input>
root@DKAM1:~# <input>ifconfig ib0</input>
ib0       Link encap:UNSPEC  HWaddr 80-00-00-02-FE-80-00-00-00-00-00-00-00-00-00-00
          BROADCAST MULTICAST  MTU:4092  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:256
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
root@DKAM1:~# <input> ibstat</input>
CA 'qib0'
        CA type: InfiniPath_QLE7340
        Number of ports: 1
        Firmware version:
        Hardware version: 2
        Node GUID: 0x0011750000788b2c
        System image GUID: 0x0011750000788b2c
        Port 1:
                State: Down
                Physical state: Polling
                Rate: 10
                Base lid: 65535
                LMC: 0
                SM lid: 65535
                Capability mask: 0x07610868
                Port GUID: 0x0011750000788b2c
root@DKAM1:~# <input>ibstatus</input>
Infiniband device 'qib0' port 1 status:
        default gid:     fe80:0000:0000:0000:0011:7500:0078:8b2c
        base lid:        0xffff
        sm lid:          0xffff
        state:           1: <notice>DOWN</notice>
        phys state:      2: <notice>Polling</notice>
        rate:            <notice>10 Gb/sec (4X)</notice>

Getting the Infinitiband hardware up

To initialize the infinitiband hardware you need a Infiniband Subnet Manager and Administration entity, then the adapters on that subnet should initialize.

root@node2:~# <input>ibstatus</input>
Infiniband device 'qib0' port 1 status:
        default gid:     fe80:0000:0000:0000:0011:7500:0078:8c6c
        base lid:        0x1
        sm lid:          0x5
        state:           4: <notice>ACTIVE</notice>
        phys state:      5: <notice>LinkUp</notice>
        rate:            <notice>40 Gb/sec (4X QDR)</notice>
root@node2:~# <input>ifconfig ib0</input>
ib0       Link encap:UNSPEC  HWaddr 80-00-00-03-FE-80-00-00-00-00-00-00-00-00-00-00
          inet addr:10.0.1.11  Bcast:10.0.1.255  Mask:255.255.255.0
          inet6 addr: fe80::211:7500:78:8c6c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:2044  Metric:1
          RX packets:3734630 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3725918 errors:0 dropped:7 overruns:0 carrier:0
          collisions:0 txqueuelen:256
          RX bytes:4634441688 (4.6 GB)  TX bytes:4649148404 (4.6 GB)

Configuration

add the following modules to /etc/modules

ib_sa
ib_cm
ib_umad
ib_addr
ib_uverbs
ib_ipoib
ib_ipath
ib_qib

Install opensm

root@DKAM1:~# <input>apt-get -y install opensm</input>
root@DKAM1:~# <input>apt-get -y install infiniband-diags</input>

Add ib0 interface to /etc/network/interfaces

auto ib0
iface ib0 inet static
  address 10.0.1.12
  netmask 255.255.255.0
  gateway 10.0.1.1

Links

Navigation menu

Ubuntu Infiniband

Contents

Upgrading from 10.04 LTE to 10.10

Checking hardware

Getting the Infinitiband hardware up

Configuration

Links