Difference between revisions of "Ubuntu Infiniband"
From Teknologisk videncenter
m (→Configuration) |
m (→Links) |
||
(4 intermediate revisions by the same user not shown) | |||
Line 58: | Line 58: | ||
phys state: 2: <notice>Polling</notice> | phys state: 2: <notice>Polling</notice> | ||
rate: <notice>10 Gb/sec (4X)</notice> | rate: <notice>10 Gb/sec (4X)</notice> | ||
+ | |||
+ | </source> | ||
+ | ==Getting the Infinitiband hardware up== | ||
+ | To initialize the infinitiband hardware you need a [[InfiniBand|Infiniband Subnet Manager and Administration]] entity, then the adapters on that subnet should initialize. | ||
+ | <source lang=cli> | ||
+ | |||
+ | root@node2:~# <input>ibstatus</input> | ||
+ | Infiniband device 'qib0' port 1 status: | ||
+ | default gid: fe80:0000:0000:0000:0011:7500:0078:8c6c | ||
+ | base lid: 0x1 | ||
+ | sm lid: 0x5 | ||
+ | state: 4: <notice>ACTIVE</notice> | ||
+ | phys state: 5: <notice>LinkUp</notice> | ||
+ | rate: <notice>40 Gb/sec (4X QDR)</notice> | ||
+ | root@node2:~# <input>ifconfig ib0</input> | ||
+ | ib0 Link encap:UNSPEC HWaddr 80-00-00-03-FE-80-00-00-00-00-00-00-00-00-00-00 | ||
+ | inet addr:10.0.1.11 Bcast:10.0.1.255 Mask:255.255.255.0 | ||
+ | inet6 addr: fe80::211:7500:78:8c6c/64 Scope:Link | ||
+ | UP BROADCAST RUNNING MULTICAST MTU:2044 Metric:1 | ||
+ | RX packets:3734630 errors:0 dropped:0 overruns:0 frame:0 | ||
+ | TX packets:3725918 errors:0 dropped:7 overruns:0 carrier:0 | ||
+ | collisions:0 txqueuelen:256 | ||
+ | RX bytes:4634441688 (4.6 GB) TX bytes:4649148404 (4.6 GB) | ||
</source> | </source> | ||
Line 76: | Line 99: | ||
<source lang=cli> | <source lang=cli> | ||
root@DKAM1:~# <input>apt-get -y install opensm</input> | root@DKAM1:~# <input>apt-get -y install opensm</input> | ||
− | root@DKAM1:~# <input>apt-get install | + | root@DKAM1:~# <input>apt-get -y install infiniband-diags</input> |
</source> | </source> | ||
Add '''ib0''' interface to '''/etc/network/interfaces''' | Add '''ib0''' interface to '''/etc/network/interfaces''' | ||
Line 93: | Line 116: | ||
*[http://en.wikipedia.org/wiki/InfiniBand Information about Infiniband] | *[http://en.wikipedia.org/wiki/InfiniBand Information about Infiniband] | ||
{{Source cli}} | {{Source cli}} | ||
− | [[Category:Ubuntu]] | + | [[Category:Ubuntu]][[Category:Infiniband]] |
Latest revision as of 06:08, 28 April 2012
Ubuntu 10.04 LTE do not support Infiniband. NO opensm package.
Contents
Upgrading from 10.04 LTE to 10.10
install the update-manager-core package if it is not already installed:
sudo apt-get install update-manager-core
Edit /etc/update-manager/release-upgrades and set Prompt=normal;
Launch the upgrade tool with the command
sudo do-release-upgrade -d
and follow the on-screen instructions.
Checking hardware
root@DKAM1:~# <input>apt-get -y install infiniband-diags</input>
root@DKAM1:~# <input>apt-get -y install opensm</input>
root@DKAM1:~# <input>modprobe ib_sa</input>
root@DKAM1:~# <input>modprobe ib_cm</input>
root@DKAM1:~# <input>modprobe ib_umad</input>
root@DKAM1:~# <input>modprobe ib_addr</input>
root@DKAM1:~# <input>modprobe ib_uverbs</input>
root@DKAM1:~# <input>modprobe ib_ipoib</input>
root@DKAM1:~# <input>modprobe ib_ipath</input>
<notice>May report device busy. Then it's already loaded</notice>
root@DKAM1:~# <input>modprobe ib_qib</input>
root@DKAM1:~# <input>ifconfig ib0</input>
ib0 Link encap:UNSPEC HWaddr 80-00-00-02-FE-80-00-00-00-00-00-00-00-00-00-00
BROADCAST MULTICAST MTU:4092 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:256
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
root@DKAM1:~# <input> ibstat</input>
CA 'qib0'
CA type: InfiniPath_QLE7340
Number of ports: 1
Firmware version:
Hardware version: 2
Node GUID: 0x0011750000788b2c
System image GUID: 0x0011750000788b2c
Port 1:
State: Down
Physical state: Polling
Rate: 10
Base lid: 65535
LMC: 0
SM lid: 65535
Capability mask: 0x07610868
Port GUID: 0x0011750000788b2c
root@DKAM1:~# <input>ibstatus</input>
Infiniband device 'qib0' port 1 status:
default gid: fe80:0000:0000:0000:0011:7500:0078:8b2c
base lid: 0xffff
sm lid: 0xffff
state: 1: <notice>DOWN</notice>
phys state: 2: <notice>Polling</notice>
rate: <notice>10 Gb/sec (4X)</notice>
Getting the Infinitiband hardware up
To initialize the infinitiband hardware you need a Infiniband Subnet Manager and Administration entity, then the adapters on that subnet should initialize.
root@node2:~# <input>ibstatus</input>
Infiniband device 'qib0' port 1 status:
default gid: fe80:0000:0000:0000:0011:7500:0078:8c6c
base lid: 0x1
sm lid: 0x5
state: 4: <notice>ACTIVE</notice>
phys state: 5: <notice>LinkUp</notice>
rate: <notice>40 Gb/sec (4X QDR)</notice>
root@node2:~# <input>ifconfig ib0</input>
ib0 Link encap:UNSPEC HWaddr 80-00-00-03-FE-80-00-00-00-00-00-00-00-00-00-00
inet addr:10.0.1.11 Bcast:10.0.1.255 Mask:255.255.255.0
inet6 addr: fe80::211:7500:78:8c6c/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:2044 Metric:1
RX packets:3734630 errors:0 dropped:0 overruns:0 frame:0
TX packets:3725918 errors:0 dropped:7 overruns:0 carrier:0
collisions:0 txqueuelen:256
RX bytes:4634441688 (4.6 GB) TX bytes:4649148404 (4.6 GB)
Configuration
add the following modules to /etc/modules
ib_sa
ib_cm
ib_umad
ib_addr
ib_uverbs
ib_ipoib
ib_ipath
ib_qib
Install opensm
root@DKAM1:~# <input>apt-get -y install opensm</input>
root@DKAM1:~# <input>apt-get -y install infiniband-diags</input>
Add ib0 interface to /etc/network/interfaces
auto ib0
iface ib0 inet static
address 10.0.1.12
netmask 255.255.255.0
gateway 10.0.1.1