Latest revision as of 05:08, 28 April 2012

Ubuntu 10.04 LTE do not support Infiniband. NO opensm package.

Upgrading from 10.04 LTE to 10.10

install the update-manager-core package if it is not already installed:

sudo apt-get install update-manager-core

Edit /etc/update-manager/release-upgrades and set Prompt=normal;

Launch the upgrade tool with the command

sudo do-release-upgrade -d

and follow the on-screen instructions.

Checking hardware

root@DKAM1:~# <input>apt-get -y install infiniband-diags</input>
root@DKAM1:~# <input>apt-get -y install opensm</input>
root@DKAM1:~# <input>modprobe ib_sa</input>
root@DKAM1:~# <input>modprobe ib_cm</input>
root@DKAM1:~# <input>modprobe ib_umad</input>
root@DKAM1:~# <input>modprobe ib_addr</input>
root@DKAM1:~# <input>modprobe ib_uverbs</input>
root@DKAM1:~# <input>modprobe ib_ipoib</input>
root@DKAM1:~# <input>modprobe ib_ipath</input>
<notice>May report device busy. Then it's already loaded</notice>
root@DKAM1:~# <input>modprobe ib_qib</input>
root@DKAM1:~# <input>ifconfig ib0</input>
ib0       Link encap:UNSPEC  HWaddr 80-00-00-02-FE-80-00-00-00-00-00-00-00-00-00-00
          BROADCAST MULTICAST  MTU:4092  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:256
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
root@DKAM1:~# <input> ibstat</input>
CA 'qib0'
        CA type: InfiniPath_QLE7340
        Number of ports: 1
        Firmware version:
        Hardware version: 2
        Node GUID: 0x0011750000788b2c
        System image GUID: 0x0011750000788b2c
        Port 1:
                State: Down
                Physical state: Polling
                Rate: 10
                Base lid: 65535
                LMC: 0
                SM lid: 65535
                Capability mask: 0x07610868
                Port GUID: 0x0011750000788b2c
root@DKAM1:~# <input>ibstatus</input>
Infiniband device 'qib0' port 1 status:
        default gid:     fe80:0000:0000:0000:0011:7500:0078:8b2c
        base lid:        0xffff
        sm lid:          0xffff
        state:           1: <notice>DOWN</notice>
        phys state:      2: <notice>Polling</notice>
        rate:            <notice>10 Gb/sec (4X)</notice>

Getting the Infinitiband hardware up

To initialize the infinitiband hardware you need a Infiniband Subnet Manager and Administration entity, then the adapters on that subnet should initialize.

root@node2:~# <input>ibstatus</input>
Infiniband device 'qib0' port 1 status:
        default gid:     fe80:0000:0000:0000:0011:7500:0078:8c6c
        base lid:        0x1
        sm lid:          0x5
        state:           4: <notice>ACTIVE</notice>
        phys state:      5: <notice>LinkUp</notice>
        rate:            <notice>40 Gb/sec (4X QDR)</notice>
root@node2:~# <input>ifconfig ib0</input>
ib0       Link encap:UNSPEC  HWaddr 80-00-00-03-FE-80-00-00-00-00-00-00-00-00-00-00
          inet addr:10.0.1.11  Bcast:10.0.1.255  Mask:255.255.255.0
          inet6 addr: fe80::211:7500:78:8c6c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:2044  Metric:1
          RX packets:3734630 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3725918 errors:0 dropped:7 overruns:0 carrier:0
          collisions:0 txqueuelen:256
          RX bytes:4634441688 (4.6 GB)  TX bytes:4649148404 (4.6 GB)

Configuration

add the following modules to /etc/modules

ib_sa
ib_cm
ib_umad
ib_addr
ib_uverbs
ib_ipoib
ib_ipath
ib_qib

Install opensm

root@DKAM1:~# <input>apt-get -y install opensm</input>
root@DKAM1:~# <input>apt-get -y install infiniband-diags</input>

Add ib0 interface to /etc/network/interfaces

auto ib0
iface ib0 inet static
  address 10.0.1.12
  netmask 255.255.255.0
  gateway 10.0.1.1

Links

Navigation menu

Difference between revisions of "Ubuntu Infiniband"