* sata_mv success on 2.6.16-rc6-mm2 (was: Re: 2.6.16-rc6-mm2)
2006-03-18 12:40 2.6.16-rc6-mm2 Andrew Morton
@ 2006-03-18 15:10 ` Sander
2006-03-18 17:25 ` 2.6.16-rc6-mm2: new RDMA CM EXPORT_SYMBOL's Adrian Bunk
` (13 subsequent siblings)
14 siblings, 0 replies; 97+ messages in thread
From: Sander @ 2006-03-18 15:10 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-ide, jgarzik, lkml
Andrew Morton wrote (ao):
> ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc6/2.6.16-rc6-mm2/
Good news! This kernel lets me use eight Maxtor sata disks in raid5 over
nForce4 and Marvell MV88SX6081.
I do:
mdadm -C -l5 -n8 /dev/md0 /dev/sda1 /dev/sdb1 /dev/sdc1 \
/dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1
mke2fs -j -m1 /dev/md0
mount -o data=writeback,nobh /dev/md0 /mnt
for i in `seq 10`
do dd if=/dev/zero of=bigfile.$i bs=1024k count=10000
done
md5sum bigfile.*
rm bigfile.*
and
for i in `seq 10`
do dd if=/dev/zero of=bigfile.$i bs=1024k count=10000 &
done
sleep 1200
md5sum bigfile.*
rm bigfile.*
and
for i in `seq 4`
do ( dd if=/dev/zero of=bigfile.$i bs=1024k count=10000 ; \
time md5sum bigfile.$i ) &
done
I experience no crash and no data corruption with these simple tests.
2.6.16-rc6 crashes with no output of the crash on netconsole (have to
find a screen to connect to the system).
2.6.16-rc3 crashes too:
http://www.ussg.iu.edu/hypermail/linux/kernel/0602.2/0360.html
I haven't tried 2.6.16-rc6-mm1.
With 2.6.16-rc6-mm2 I do get "BUG: warning ..." and "Call Trace: ..."
messages during boot (see dmesg below), which are not there with 2.6.16-rc6.
But that seems not to cause any problems.
I also see these appear during the tests:
[ 3584.499415] ata5: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ 0xb/47/00
[ 3584.499469] ata5: status=0xd0 { Busy }
[ 4430.676302] ata9: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ 0xb/47/00
[ 4430.676355] ata9: status=0xd0 { Busy }
But with no visible effect.
Thanks!
Kind regards, Sander
[ 82.078411] libata version 1.20 loaded.
[ 82.078449] sata_nv 0000:00:07.0: version 0.8
[ 82.078457] ACPI (acpi_bus-0216): Device 'SAT0' is not power manageable [20060210]
[ 82.078917] ACPI: PCI Interrupt Link [LTID] enabled at IRQ 23
[ 82.078967] GSI 20 sharing vector 0xD1 and IRQ 20
[ 82.079015] ACPI: PCI Interrupt 0000:00:07.0[A] -> Link [LTID] -> GSI 23 (level, high) -> IRQ 20
[ 82.079376] PCI: Setting latency timer of device 0000:00:07.0 to 64
[ 82.079446] ata1: SATA max UDMA/133 cmd 0x1440 ctl 0x1436 bmdma 0x1410 irq 20
[ 82.079520] ata2: SATA max UDMA/133 cmd 0x1438 ctl 0x1432 bmdma 0x1418 irq 20
[ 82.295014] ata1: SATA link up 3.0 Gbps (SStatus 123)
[ 82.494850] ata1: dev 0 cfg 49:2f00 82:7c6b 83:7f69 84:4773 85:7c69 86:3e01 87:4763 88:407f
[ 82.494854] ata1: dev 0 ATA-7, max UDMA/133, 976773168 sectors: LBA48
[ 82.496131] nv_sata: Primary device added
[ 82.496184] nv_sata: Primary device removed
[ 82.496228] nv_sata: Secondary device added
[ 82.496273] nv_sata: Secondary device removed
[ 82.524820] ata1: dev 0 configured for UDMA/133
[ 82.524865] scsi0 : sata_nv
[ 82.744454] ata2: SATA link down (SStatus 0)
[ 82.744500] scsi1 : sata_nv
[ 82.744637] Vendor: ATA Model: Maxtor 7H500F0 Rev: HA43
[ 82.745399] Type: Direct-Access ANSI SCSI revision: 05
[ 82.745582] ACPI (acpi_bus-0216): Device 'SAT1' is not power manageable [20060210]
[ 82.746058] ACPI: PCI Interrupt Link [LSI1] enabled at IRQ 22
[ 82.746108] GSI 21 sharing vector 0xD9 and IRQ 21
[ 82.746157] ACPI: PCI Interrupt 0000:00:08.0[A] -> Link [LSI1] -> GSI 22 (level, high) -> IRQ 21
[ 82.746485] PCI: Setting latency timer of device 0000:00:08.0 to 64
[ 82.746551] ata3: SATA max UDMA/133 cmd 0x1458 ctl 0x144E bmdma 0x1420 irq 21
[ 82.746628] ata4: SATA max UDMA/133 cmd 0x1450 ctl 0x144A bmdma 0x1428 irq 21
[ 82.964178] ata3: SATA link down (SStatus 0)
[ 82.964222] scsi2 : sata_nv
[ 83.183902] ata4: SATA link down (SStatus 0)
[ 83.183946] scsi3 : sata_nv
[ 83.184069] sata_mv 0000:0a:03.0: version 0.6
[ 83.184129] GSI 22 sharing vector 0xE1 and IRQ 22
[ 83.184174] ACPI: PCI Interrupt 0000:0a:03.0[A] -> GSI 30 (level, low) -> IRQ 22
[ 83.187740] sata_mv 0000:0a:03.0: 32 slots 8 ports SCSI mode IRQ via INTx
[ 83.187857] ata5: SATA max UDMA/133 cmd 0x0 ctl 0xFFFFC20000222120 bmdma 0x0 irq 22
[ 83.187951] ata6: SATA max UDMA/133 cmd 0x0 ctl 0xFFFFC20000224120 bmdma 0x0 irq 22
[ 83.188042] ata7: SATA max UDMA/133 cmd 0x0 ctl 0xFFFFC20000226120 bmdma 0x0 irq 22
[ 83.188132] ata8: SATA max UDMA/133 cmd 0x0 ctl 0xFFFFC20000228120 bmdma 0x0 irq 22
[ 83.188222] ata9: SATA max UDMA/133 cmd 0x0 ctl 0xFFFFC20000232120 bmdma 0x0 irq 22
[ 83.188314] ata10: SATA max UDMA/133 cmd 0x0 ctl 0xFFFFC20000234120 bmdma 0x0 irq 22
[ 83.188404] ata11: SATA max UDMA/133 cmd 0x0 ctl 0xFFFFC20000236120 bmdma 0x0 irq 22
[ 83.188495] ata12: SATA max UDMA/133 cmd 0x0 ctl 0xFFFFC20000238120 bmdma 0x0 irq 22
[ 83.244314] BUG: warning at drivers/scsi/sata_mv.c:1896/__msleep()
[ 83.244362]
[ 83.244362] Call Trace: <IRQ> <ffffffff80408d00>{__mv_phy_reset+242}
[ 83.244509] <ffffffff804081ec>{mv_channel_reset+133} <ffffffff80409361>{mv_interrupt+565}
[ 83.244669] <ffffffff8023bf2c>{handle_IRQ_event+41} <ffffffff8023bffa>{__do_IRQ+155}
[ 83.244828] <ffffffff8020c13b>{do_IRQ+59} <ffffffff8020834c>{default_idle+0}
[ 83.244989] <ffffffff80209d64>{ret_from_intr+0} <EOI> <ffffffff804eb96d>{thread_return+86}
[ 83.245172] <ffffffff80208379>{default_idle+45} <ffffffff80208406>{cpu_idle+98}
[ 83.245329] <ffffffff807c4018>{start_secondary+1224}
[ 86.569852] ata5: dev 0 cfg 49:2f00 82:7c6b 83:7f09 84:4063 85:7c69 86:3e01 87:4063 88:007f
[ 86.569856] ata5: dev 0 ATA-7, max UDMA/133, 586114704 sectors: LBA48
[ 86.570840] BUG: warning at drivers/scsi/libata-core.c:3884/__ata_qc_complete()
[ 86.570902]
[ 86.570903] Call Trace: <IRQ> <ffffffff80401694>{__ata_qc_complete+98}
[ 86.571046] <ffffffff8040939e>{mv_interrupt+626} <ffffffff8023bf2c>{handle_IRQ_event+41}
[ 86.571208] <ffffffff8023bffa>{__do_IRQ+155} <ffffffff8020c13b>{do_IRQ+59}
[ 86.571365] <ffffffff80209d64>{ret_from_intr+0} <EOI> <ffffffff804eccd2>{__reacquire_kernel_lock+39}
[ 86.571552] <ffffffff804eb9bd>{thread_return+166} <ffffffff804ebfe5>{schedule_timeout+138}
[ 86.571713] <ffffffff8022a66c>{process_timeout+0} <ffffffff804eaeaf>{wait_for_completion_timeout+137}
[ 86.571878] <ffffffff8021cc6f>{default_wake_function+0} <ffffffff80407402>{ata_exec_command+36}
[ 86.572042] <ffffffff804038b8>{ata_exec_internal+266} <ffffffff80404473>{ata_set_mode+667}
[ 86.575479] <ffffffff8023c4d1>{request_irq+130} <ffffffff804049f4>{ata_device_add+1163}
[ 86.575639] <ffffffff80409a41>{mv_init_one+1547} <ffffffff8036c8b2>{pci_device_probe+221}
[ 86.575799] <ffffffff803b999d>{driver_probe_device+82} <ffffffff803b9a52>{__driver_attach+0}
[ 86.575959] <ffffffff803b9aa8>{__driver_attach+86} <ffffffff803b8f5a>{bus_for_each_dev+67}
[ 86.576121] <ffffffff803b925a>{bus_add_driver+116} <ffffffff8036c918>{pci_bus_match+0}
[ 86.576279] <ffffffff8036c418>{__pci_register_driver+85} <ffffffff8020720c>{init+455}
[ 86.576438] <ffffffff8020a6b6>{child_rip+8} <ffffffff80207045>{init+0}
[ 86.576582] <ffffffff8020a6ae>{child_rip+0}
[ 86.599812] ata5: dev 0 configured for UDMA/133
[ 86.599856] scsi4 : sata_mv
[ 89.975572] ata6: dev 0 cfg 49:2f00 82:7c6b 83:7f09 84:4673 85:7c69 86:3e21 87:4663 88:007f
[ 89.975576] ata6: dev 0 ATA-7, max UDMA/133, 586114704 sectors: LBA48
[ 89.976575] BUG: warning at drivers/scsi/libata-core.c:3884/__ata_qc_complete()
[ 89.976637]
[ 89.976638] Call Trace: <IRQ> <ffffffff80401694>{__ata_qc_complete+98}
[ 89.976780] <ffffffff8040939e>{mv_interrupt+626} <ffffffff8023bf2c>{handle_IRQ_event+41}
[ 89.976943] <ffffffff8023bffa>{__do_IRQ+155} <ffffffff8020c13b>{do_IRQ+59}
[ 89.977100] <ffffffff80209d64>{ret_from_intr+0} <EOI> <ffffffff804eb96d>{thread_return+86}
[ 89.977287] <ffffffff804ebfe5>{schedule_timeout+138} <ffffffff8022a66c>{process_timeout+0}
[ 89.977450] <ffffffff804eaeaf>{wait_for_completion_timeout+137}
[ 89.977546] <ffffffff8021cc6f>{default_wake_function+0} <ffffffff80407402>{ata_exec_command+36}
[ 89.977710] <ffffffff804038b8>{ata_exec_internal+266} <ffffffff80404473>{ata_set_mode+667}
[ 89.977871] <ffffffff804049f4>{ata_device_add+1163} <ffffffff80409a41>{mv_init_one+1547}
[ 89.978032] <ffffffff8036c8b2>{pci_device_probe+221} <ffffffff803b999d>{driver_probe_device+82}
[ 89.978195] <ffffffff803b9a52>{__driver_attach+0} <ffffffff803b9aa8>{__driver_attach+86}
[ 89.978355] <ffffffff803b8f5a>{bus_for_each_dev+67} <ffffffff803b925a>{bus_add_driver+116}
[ 89.978514] <ffffffff8036c918>{pci_bus_match+0} <ffffffff8036c418>{__pci_register_driver+85}
[ 89.978676] <ffffffff8020720c>{init+455} <ffffffff8020a6b6>{child_rip+8}
[ 89.978832] <ffffffff80207045>{init+0} <ffffffff8020a6ae>{child_rip+0}
[ 90.005534] ata6: dev 0 configured for UDMA/133
[ 90.005579] scsi5 : sata_mv
[ 93.381295] ata7: dev 0 cfg 49:2f00 82:7c6b 83:7f09 84:4063 85:7c69 86:3e01 87:4063 88:007f
[ 93.381299] ata7: dev 0 ATA-7, max UDMA/133, 586114704 sectors: LBA48
[ 93.382285] BUG: warning at drivers/scsi/libata-core.c:3884/__ata_qc_complete()
[ 93.382347]
[ 93.382348] Call Trace: <IRQ> <ffffffff80401694>{__ata_qc_complete+98}
[ 93.382491] <ffffffff8040939e>{mv_interrupt+626} <ffffffff8023bf2c>{handle_IRQ_event+41}
[ 93.382652] <ffffffff8023bffa>{__do_IRQ+155} <ffffffff8020c13b>{do_IRQ+59}
[ 93.382810] <ffffffff80209d64>{ret_from_intr+0} <EOI> <ffffffff804ecb0e>{_spin_lock_irqsave+2}
[ 93.382996] <ffffffff8022a1d9>{lock_timer_base+27} <ffffffff8022addd>{try_to_del_timer_sync+22}
[ 93.383157] <ffffffff8022ae2c>{del_timer_sync+12} <ffffffff804ebfed>{schedule_timeout+146}
[ 93.383317] <ffffffff8022a66c>{process_timeout+0} <ffffffff804eaeaf>{wait_for_completion_timeout+137}
[ 93.383480] <ffffffff8021cc6f>{default_wake_function+0} <ffffffff80407402>{ata_exec_command+36}
[ 93.383642] <ffffffff804038b8>{ata_exec_internal+266} <ffffffff80404473>{ata_set_mode+667}
[ 93.383803] <ffffffff804049f4>{ata_device_add+1163} <ffffffff80409a41>{mv_init_one+1547}
[ 93.383965] <ffffffff8036c8b2>{pci_device_probe+221} <ffffffff803b999d>{driver_probe_device+82}
[ 93.384125] <ffffffff803b9a52>{__driver_attach+0} <ffffffff803b9aa8>{__driver_attach+86}
[ 93.384285] <ffffffff803b8f5a>{bus_for_each_dev+67} <ffffffff803b925a>{bus_add_driver+116}
[ 93.384446] <ffffffff8036c918>{pci_bus_match+0} <ffffffff8036c418>{__pci_register_driver+85}
[ 93.384606] <ffffffff8020720c>{init+455} <ffffffff8020a6b6>{child_rip+8}
[ 93.384762] <ffffffff80207045>{init+0} <ffffffff8020a6ae>{child_rip+0}
[ 93.411257] ata7: dev 0 configured for UDMA/133
[ 93.411304] scsi6 : sata_mv
[ 96.787019] ata8: dev 0 cfg 49:2f00 82:7c6b 83:7f09 84:4063 85:7c69 86:3e01 87:4063 88:007f
[ 96.787023] ata8: dev 0 ATA-7, max UDMA/133, 586114704 sectors: LBA48
[ 96.788019] BUG: warning at drivers/scsi/libata-core.c:3884/__ata_qc_complete()
[ 96.788081]
[ 96.788081] Call Trace: <IRQ> <ffffffff80401694>{__ata_qc_complete+98}
[ 96.788224] <ffffffff8040939e>{mv_interrupt+626} <ffffffff8023bf2c>{handle_IRQ_event+41}
[ 96.788385] <ffffffff8023bffa>{__do_IRQ+155} <ffffffff8020c13b>{do_IRQ+59}
[ 96.788541] <ffffffff80209d64>{ret_from_intr+0} <EOI> <ffffffff8022a1c4>{lock_timer_base+6}
[ 96.788726] <ffffffff8022addd>{try_to_del_timer_sync+22} <ffffffff8022ae2c>{del_timer_sync+12}
[ 96.788886] <ffffffff804ebfed>{schedule_timeout+146} <ffffffff8022a66c>{process_timeout+0}
[ 96.789046] <ffffffff804eaeaf>{wait_for_completion_timeout+137}
[ 96.789142] <ffffffff8021cc6f>{default_wake_function+0} <ffffffff804ecb5d>{_spin_unlock_irqrestore+8}
[ 96.789302] <ffffffff804038b8>{ata_exec_internal+266} <ffffffff80404473>{ata_set_mode+667}
[ 96.789462] <ffffffff804049f4>{ata_device_add+1163} <ffffffff80409a41>{mv_init_one+1547}
[ 96.789625] <ffffffff8036c8b2>{pci_device_probe+221} <ffffffff803b999d>{driver_probe_device+82}
[ 96.789787] <ffffffff803b9a52>{__driver_attach+0} <ffffffff803b9aa8>{__driver_attach+86}
[ 96.789947] <ffffffff803b8f5a>{bus_for_each_dev+67} <ffffffff803b925a>{bus_add_driver+116}
[ 96.790110] <ffffffff8036c918>{pci_bus_match+0} <ffffffff8036c418>{__pci_register_driver+85}
[ 96.790270] <ffffffff8020720c>{init+455} <ffffffff8020a6b6>{child_rip+8}
[ 96.790426] <ffffffff80207045>{init+0} <ffffffff8020a6ae>{child_rip+0}
[ 96.816981] ata8: dev 0 configured for UDMA/133
[ 96.817025] scsi7 : sata_mv
[ 100.192743] ata9: dev 0 cfg 49:2f00 82:7c6b 83:7f09 84:4673 85:7c69 86:3e21 87:4663 88:007f
[ 100.192747] ata9: dev 0 ATA-7, max UDMA/133, 586114704 sectors: LBA48
[ 100.193757] BUG: warning at drivers/scsi/libata-core.c:3884/__ata_qc_complete()
[ 100.193819]
[ 100.193820] Call Trace: <IRQ> <ffffffff80401694>{__ata_qc_complete+98}
[ 100.193963] <ffffffff8040939e>{mv_interrupt+626} <ffffffff8023bf2c>{handle_IRQ_event+41}
[ 100.194124] <ffffffff8023bffa>{__do_IRQ+155} <ffffffff8020c13b>{do_IRQ+59}
[ 100.194283] <ffffffff80209d64>{ret_from_intr+0} <EOI> <ffffffff804eb973>{thread_return+92}
[ 100.194469] <ffffffff804ebfe5>{schedule_timeout+138} <ffffffff8022a66c>{process_timeout+0}
[ 100.194629] <ffffffff804eaeaf>{wait_for_completion_timeout+137}
[ 100.194724] <ffffffff8021cc6f>{default_wake_function+0} <ffffffff80407402>{ata_exec_command+36}
[ 100.194887] <ffffffff804038b8>{ata_exec_internal+266} <ffffffff80404473>{ata_set_mode+667}
[ 100.195048] <ffffffff804049f4>{ata_device_add+1163} <ffffffff80409a41>{mv_init_one+1547}
[ 100.195208] <ffffffff8036c8b2>{pci_device_probe+221} <ffffffff803b999d>{driver_probe_device+82}
[ 100.195372] <ffffffff803b9a52>{__driver_attach+0} <ffffffff803b9aa8>{__driver_attach+86}
[ 100.195531] <ffffffff803b8f5a>{bus_for_each_dev+67} <ffffffff803b925a>{bus_add_driver+116}
[ 100.195690] <ffffffff8036c918>{pci_bus_match+0} <ffffffff8036c418>{__pci_register_driver+85}
[ 100.195853] <ffffffff8020720c>{init+455} <ffffffff8020a6b6>{child_rip+8}
[ 100.196010] <ffffffff80207045>{init+0} <ffffffff8020a6ae>{child_rip+0}
[ 100.222704] ata9: dev 0 configured for UDMA/133
[ 100.222749] scsi8 : sata_mv
[ 103.598467] ata10: dev 0 cfg 49:2f00 82:7c6b 83:7f09 84:4063 85:7c69 86:3e01 87:4063 88:007f
[ 103.598471] ata10: dev 0 ATA-7, max UDMA/133, 586114704 sectors: LBA48
[ 103.599467] BUG: warning at drivers/scsi/libata-core.c:3884/__ata_qc_complete()
[ 103.599530]
[ 103.599531] Call Trace: <IRQ> <ffffffff80401694>{__ata_qc_complete+98}
[ 103.599674] <ffffffff8040939e>{mv_interrupt+626} <ffffffff8023bf2c>{handle_IRQ_event+41}
[ 103.599834] <ffffffff8023bffa>{__do_IRQ+155} <ffffffff8020c13b>{do_IRQ+59}
[ 103.599992] <ffffffff80209d64>{ret_from_intr+0} <EOI> <ffffffff804eccd2>{__reacquire_kernel_lock+39}
[ 103.600179] <ffffffff804eb9bd>{thread_return+166} <ffffffff804ebfe5>{schedule_timeout+138}
[ 103.600340] <ffffffff8022a66c>{process_timeout+0} <ffffffff804eaeaf>{wait_for_completion_timeout+137}
[ 103.600502] <ffffffff8021cc6f>{default_wake_function+0} <ffffffff80407402>{ata_exec_command+36}
[ 103.600666] <ffffffff804038b8>{ata_exec_internal+266} <ffffffff80404473>{ata_set_mode+667}
[ 103.600826] <ffffffff804049f4>{ata_device_add+1163} <ffffffff80409a41>{mv_init_one+1547}
[ 103.600986] <ffffffff8036c8b2>{pci_device_probe+221} <ffffffff803b999d>{driver_probe_device+82}
[ 103.601152] <ffffffff803b9a52>{__driver_attach+0} <ffffffff803b9aa8>{__driver_attach+86}
[ 103.601314] <ffffffff803b8f5a>{bus_for_each_dev+67} <ffffffff803b925a>{bus_add_driver+116}
[ 103.604759] <ffffffff8036c918>{pci_bus_match+0} <ffffffff8036c418>{__pci_register_driver+85}
[ 103.604919] <ffffffff8020720c>{init+455} <ffffffff8020a6b6>{child_rip+8}
[ 103.605075] <ffffffff80207045>{init+0} <ffffffff8020a6ae>{child_rip+0}
[ 103.628428] ata10: dev 0 configured for UDMA/133
[ 103.628473] scsi9 : sata_mv
[ 106.994203] ata11: dev 0 cfg 49:2f00 82:7c6b 83:7f09 84:4673 85:7c69 86:3e21 87:4663 88:007f
[ 106.994207] ata11: dev 0 ATA-7, max UDMA/133, 586114704 sectors: LBA48
[ 106.995198] BUG: warning at drivers/scsi/libata-core.c:3884/__ata_qc_complete()
[ 106.995261]
[ 106.995262] Call Trace: <IRQ> <ffffffff80401694>{__ata_qc_complete+98}
[ 106.995405] <ffffffff8040939e>{mv_interrupt+626} <ffffffff8023bf2c>{handle_IRQ_event+41}
[ 106.995565] <ffffffff8023bffa>{__do_IRQ+155} <ffffffff8020c13b>{do_IRQ+59}
[ 106.995722] <ffffffff80209d64>{ret_from_intr+0} <EOI> <ffffffff804eb917>{thread_return+0}
[ 106.995907] <ffffffff804eb9bd>{thread_return+166} <ffffffff804eb9ec>{thread_return+213}
[ 106.996069] <ffffffff804ebfe5>{schedule_timeout+138} <ffffffff8022a66c>{process_timeout+0}
[ 106.996228] <ffffffff804eaeaf>{wait_for_completion_timeout+137}
[ 106.996325] <ffffffff8021cc6f>{default_wake_function+0} <ffffffff80407402>{ata_exec_command+36}
[ 106.996486] <ffffffff804038b8>{ata_exec_internal+266} <ffffffff80404473>{ata_set_mode+667}
[ 106.996646] <ffffffff804049f4>{ata_device_add+1163} <ffffffff80409a41>{mv_init_one+1547}
[ 106.996808] <ffffffff8036c8b2>{pci_device_probe+221} <ffffffff803b999d>{driver_probe_device+82}
[ 106.996969] <ffffffff803b9a52>{__driver_attach+0} <ffffffff803b9aa8>{__driver_attach+86}
[ 106.997129] <ffffffff803b8f5a>{bus_for_each_dev+67} <ffffffff803b925a>{bus_add_driver+116}
[ 106.997291] <ffffffff8036c918>{pci_bus_match+0} <ffffffff8036c418>{__pci_register_driver+85}
[ 106.997453] <ffffffff8020720c>{init+455} <ffffffff8020a6b6>{child_rip+8}
[ 106.997611] <ffffffff80207045>{init+0} <ffffffff8020a6ae>{child_rip+0}
[ 107.024165] ata11: dev 0 configured for UDMA/133
[ 107.024210] scsi10 : sata_mv
[ 107.073907] ata12: no device found (phy stat 00000000)
[ 107.073953] scsi11 : sata_mv
[ 107.074082] Vendor: ATA Model: Maxtor 6B300S0 Rev: BANC
[ 107.074841] Type: Direct-Access ANSI SCSI revision: 05
[ 107.075044] Vendor: ATA Model: Maxtor 6L300S0 Rev: BACE
[ 107.075798] Type: Direct-Access ANSI SCSI revision: 05
[ 107.075994] Vendor: ATA Model: Maxtor 6B300S0 Rev: BANC
[ 107.076749] Type: Direct-Access ANSI SCSI revision: 05
[ 107.076944] Vendor: ATA Model: Maxtor 6B300S0 Rev: BANC
[ 107.077701] Type: Direct-Access ANSI SCSI revision: 05
[ 107.077900] Vendor: ATA Model: Maxtor 6L300S0 Rev: BACE
[ 107.078653] Type: Direct-Access ANSI SCSI revision: 05
[ 107.078857] Vendor: ATA Model: Maxtor 6B300S0 Rev: BANC
[ 107.079610] Type: Direct-Access ANSI SCSI revision: 05
[ 107.079806] Vendor: ATA Model: Maxtor 6L300S0 Rev: BACE
[ 107.080559] Type: Direct-Access ANSI SCSI revision: 05
[ 107.080785] SCSI device sda: 976773168 512-byte hdwr sectors (500108 MB)
[ 107.080848] sda: Write Protect is off
[ 107.080893] sda: Mode Sense: 00 3a 00 00
[ 107.080905] SCSI device sda: drive cache: write back
[ 107.081014] SCSI device sda: 976773168 512-byte hdwr sectors (500108 MB)
[ 107.081069] sda: Write Protect is off
[ 107.081112] sda: Mode Sense: 00 3a 00 00
[ 107.081124] SCSI device sda: drive cache: write back
[ 107.081170] sda: sda1
[ 107.099237] sd 0:0:0:0: Attached scsi disk sda
etc, etc.
--
Humilis IT Services and Solutions
http://www.humilis.net
^ permalink raw reply [flat|nested] 97+ messages in thread* 2.6.16-rc6-mm2: new RDMA CM EXPORT_SYMBOL's
2006-03-18 12:40 2.6.16-rc6-mm2 Andrew Morton
2006-03-18 15:10 ` sata_mv success on 2.6.16-rc6-mm2 (was: Re: 2.6.16-rc6-mm2) Sander
@ 2006-03-18 17:25 ` Adrian Bunk
2006-03-19 1:44 ` Tom Tucker
2006-03-18 18:45 ` [-mm patch] drivers/edac/e752x_edac.c: make sysbus_message static Adrian Bunk
` (12 subsequent siblings)
14 siblings, 1 reply; 97+ messages in thread
From: Adrian Bunk @ 2006-03-18 17:25 UTC (permalink / raw)
To: Andrew Morton, Sean Hefty; +Cc: linux-kernel, Roland Dreier, openib-general
On Sat, Mar 18, 2006 at 04:40:56AM -0800, Andrew Morton wrote:
>...
> Changes since 2.6.16-rc6-mm1:
>...
> git-infiniband.patch
>...
> git trees.
>...
I'm not exactly happy that this tree adds tons of RDMA CM
EXPORT_SYMBOL's that are neither currently used nor _GPL.
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: 2.6.16-rc6-mm2: new RDMA CM EXPORT_SYMBOL's
2006-03-18 17:25 ` 2.6.16-rc6-mm2: new RDMA CM EXPORT_SYMBOL's Adrian Bunk
@ 2006-03-19 1:44 ` Tom Tucker
2006-03-19 1:57 ` Lee Revell
0 siblings, 1 reply; 97+ messages in thread
From: Tom Tucker @ 2006-03-19 1:44 UTC (permalink / raw)
To: Adrian Bunk
Cc: Andrew Morton, Sean Hefty, linux-kernel, Roland Dreier,
openib-general
On Sat, 2006-03-18 at 18:25 +0100, Adrian Bunk wrote:
> On Sat, Mar 18, 2006 at 04:40:56AM -0800, Andrew Morton wrote:
> >...
> > Changes since 2.6.16-rc6-mm1:
> >...
> > git-infiniband.patch
> >...
> > git trees.
> >...
>
> I'm not exactly happy that this tree adds tons of RDMA CM
> EXPORT_SYMBOL's that are neither currently used nor _GPL.
I apologize in advance for my ignorance, but I don't understand how 'new
symbols' could pass the test of 'currently used'.
> cu
> Adrian
>
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: 2.6.16-rc6-mm2: new RDMA CM EXPORT_SYMBOL's
2006-03-19 1:44 ` Tom Tucker
@ 2006-03-19 1:57 ` Lee Revell
0 siblings, 0 replies; 97+ messages in thread
From: Lee Revell @ 2006-03-19 1:57 UTC (permalink / raw)
To: tom
Cc: Adrian Bunk, Andrew Morton, Sean Hefty, linux-kernel,
Roland Dreier, openib-general
On Sat, 2006-03-18 at 19:44 -0600, Tom Tucker wrote:
> I apologize in advance for my ignorance, but I don't understand how
> 'new symbols' could pass the test of 'currently used'.
>
Normally when adding new exports you would submit the code that uses
them at the same time.
Lee
^ permalink raw reply [flat|nested] 97+ messages in thread
* [-mm patch] drivers/edac/e752x_edac.c: make sysbus_message static
2006-03-18 12:40 2.6.16-rc6-mm2 Andrew Morton
2006-03-18 15:10 ` sata_mv success on 2.6.16-rc6-mm2 (was: Re: 2.6.16-rc6-mm2) Sander
2006-03-18 17:25 ` 2.6.16-rc6-mm2: new RDMA CM EXPORT_SYMBOL's Adrian Bunk
@ 2006-03-18 18:45 ` Adrian Bunk
2006-03-18 18:45 ` [-mm patch] drivers/scsi/aic7xxx/aic79xx_core.c: make ahd_match_scb() static Adrian Bunk
` (11 subsequent siblings)
14 siblings, 0 replies; 97+ messages in thread
From: Adrian Bunk @ 2006-03-18 18:45 UTC (permalink / raw)
To: Andrew Morton, Dave Peterson; +Cc: linux-kernel
On Sat, Mar 18, 2006 at 04:40:56AM -0800, Andrew Morton wrote:
>...
> Changes since 2.6.16-rc6-mm1:
>...
> +edac-use-sysbus_message-in-e752x-code.patch
>...
> EDAC updates
>...
This patch makes the needlessly global sysbus_message static.
Signed-off-by: Adrian Bunk <bunk@stusta.de>
--- linux-2.6.16-rc6-mm2-full/drivers/edac/e752x_edac.c.old 2006-03-18 17:57:13.000000000 +0100
+++ linux-2.6.16-rc6-mm2-full/drivers/edac/e752x_edac.c 2006-03-18 17:57:36.000000000 +0100
@@ -472,7 +472,7 @@
do_membuf_error(errors);
}
-char *sysbus_message[10] = {
+static char *sysbus_message[10] = {
"Addr or Request Parity",
"Data Strobe Glitch",
"Addr Strobe Glitch",
^ permalink raw reply [flat|nested] 97+ messages in thread* [-mm patch] drivers/scsi/aic7xxx/aic79xx_core.c: make ahd_match_scb() static
2006-03-18 12:40 2.6.16-rc6-mm2 Andrew Morton
` (2 preceding siblings ...)
2006-03-18 18:45 ` [-mm patch] drivers/edac/e752x_edac.c: make sysbus_message static Adrian Bunk
@ 2006-03-18 18:45 ` Adrian Bunk
2006-03-18 18:45 ` [-mm patch] nfs4proc.c: make _nfs4_proc_setclientid_confirm() static Adrian Bunk
` (10 subsequent siblings)
14 siblings, 0 replies; 97+ messages in thread
From: Adrian Bunk @ 2006-03-18 18:45 UTC (permalink / raw)
To: Andrew Morton, James.Bottomley; +Cc: linux-kernel, linux-scsi
On Sat, Mar 18, 2006 at 04:40:56AM -0800, Andrew Morton wrote:
>...
> Changes since 2.6.16-rc6-mm1:
>...
> git-scsi-misc.patch
>...
> git trees.
ahd_match_scb() can now become static.
Signed-off-by: Adrian Bunk <bunk@stusta.de>
---
drivers/scsi/aic7xxx/aic79xx.h | 3 ---
drivers/scsi/aic7xxx/aic79xx_core.c | 5 ++++-
2 files changed, 4 insertions(+), 4 deletions(-)
--- linux-2.6.16-rc6-mm2-full/drivers/scsi/aic7xxx/aic79xx.h.old 2006-03-18 18:33:30.000000000 +0100
+++ linux-2.6.16-rc6-mm2-full/drivers/scsi/aic7xxx/aic79xx.h 2006-03-18 18:33:38.000000000 +0100
@@ -1347,9 +1347,6 @@
/************************** SCB and SCB queue management **********************/
void ahd_qinfifo_requeue_tail(struct ahd_softc *ahd,
struct scb *scb);
-int ahd_match_scb(struct ahd_softc *ahd, struct scb *scb,
- int target, char channel, int lun,
- u_int tag, role_t role);
/****************************** Initialization ********************************/
struct ahd_softc *ahd_alloc(void *platform_arg, char *name);
--- linux-2.6.16-rc6-mm2-full/drivers/scsi/aic7xxx/aic79xx_core.c.old 2006-03-18 18:33:45.000000000 +0100
+++ linux-2.6.16-rc6-mm2-full/drivers/scsi/aic7xxx/aic79xx_core.c 2006-03-18 18:36:31.000000000 +0100
@@ -263,6 +263,9 @@
u_int mincmds);
static int ahd_verify_vpd_cksum(struct vpd_config *vpd);
static int ahd_wait_seeprom(struct ahd_softc *ahd);
+static int ahd_match_scb(struct ahd_softc *ahd, struct scb *scb,
+ int target, char channel, int lun,
+ u_int tag, role_t role);
/******************************** Private Inlines *****************************/
@@ -7236,7 +7239,7 @@
}
/************************** SCB and SCB queue management **********************/
-int
+static int
ahd_match_scb(struct ahd_softc *ahd, struct scb *scb, int target,
char channel, int lun, u_int tag, role_t role)
{
^ permalink raw reply [flat|nested] 97+ messages in thread* [-mm patch] nfs4proc.c: make _nfs4_proc_setclientid_confirm() static
2006-03-18 12:40 2.6.16-rc6-mm2 Andrew Morton
` (3 preceding siblings ...)
2006-03-18 18:45 ` [-mm patch] drivers/scsi/aic7xxx/aic79xx_core.c: make ahd_match_scb() static Adrian Bunk
@ 2006-03-18 18:45 ` Adrian Bunk
2006-03-18 18:45 ` [-mm patch] net/core/scm.c: make scm_detach_fds() static Adrian Bunk
` (9 subsequent siblings)
14 siblings, 0 replies; 97+ messages in thread
From: Adrian Bunk @ 2006-03-18 18:45 UTC (permalink / raw)
To: Andrew Morton, trond.myklebust; +Cc: linux-kernel
On Sat, Mar 18, 2006 at 04:40:56AM -0800, Andrew Morton wrote:
>...
> Boilerplate:
>...
> Changes since 2.6.16-rc6-mm1:
>...
> git-nfs.patch
>...
> git trees.
>...
This patch makes a needlessly global function static.
Signed-off-by: Adrian Bunk <bunk@stusta.de>
--- linux-2.6.16-rc6-mm2-full/fs/nfs/nfs4proc.c.old 2006-03-18 18:40:47.000000000 +0100
+++ linux-2.6.16-rc6-mm2-full/fs/nfs/nfs4proc.c 2006-03-18 18:41:01.000000000 +0100
@@ -2849,7 +2849,8 @@
return status;
}
-int _nfs4_proc_setclientid_confirm(struct nfs4_client *clp, struct rpc_cred *cred)
+static int _nfs4_proc_setclientid_confirm(struct nfs4_client *clp,
+ struct rpc_cred *cred)
{
struct nfs_fsinfo fsinfo;
struct rpc_message msg = {
^ permalink raw reply [flat|nested] 97+ messages in thread* [-mm patch] net/core/scm.c: make scm_detach_fds() static
2006-03-18 12:40 2.6.16-rc6-mm2 Andrew Morton
` (4 preceding siblings ...)
2006-03-18 18:45 ` [-mm patch] nfs4proc.c: make _nfs4_proc_setclientid_confirm() static Adrian Bunk
@ 2006-03-18 18:45 ` Adrian Bunk
2006-03-18 20:21 ` 2.6.16-rc6-mm2 Rafael J. Wysocki
` (8 subsequent siblings)
14 siblings, 0 replies; 97+ messages in thread
From: Adrian Bunk @ 2006-03-18 18:45 UTC (permalink / raw)
To: Andrew Morton, davem; +Cc: linux-kernel, netdev
On Sat, Mar 18, 2006 at 04:40:56AM -0800, Andrew Morton wrote:
>...
> Changes since 2.6.16-rc6-mm1:
>...
> git-net.patch
>...
> git trees.
>...
We can now make scm_detach_fds() static.
Signed-off-by: Adrian Bunk <bunk@stusta.de>
---
include/net/scm.h | 1 -
net/core/scm.c | 3 +--
2 files changed, 1 insertion(+), 3 deletions(-)
--- linux-2.6.16-rc6-mm2-full/include/net/scm.h.old 2006-03-18 18:48:20.000000000 +0100
+++ linux-2.6.16-rc6-mm2-full/include/net/scm.h 2006-03-18 18:48:25.000000000 +0100
@@ -24,7 +24,6 @@
unsigned long seq; /* Connection seqno */
};
-extern void scm_detach_fds(struct msghdr *msg, struct scm_cookie *scm);
extern void scm_detach_fds_compat(struct msghdr *msg, struct scm_cookie *scm);
extern void __scm_destroy(struct scm_cookie *scm);
extern struct scm_fp_list * scm_fp_dup(struct scm_fp_list *fpl);
--- linux-2.6.16-rc6-mm2-full/net/core/scm.c.old 2006-03-18 18:48:35.000000000 +0100
+++ linux-2.6.16-rc6-mm2-full/net/core/scm.c 2006-03-18 18:48:46.000000000 +0100
@@ -210,7 +210,7 @@
return err;
}
-void scm_detach_fds(struct msghdr *msg, struct scm_cookie *scm)
+static void scm_detach_fds(struct msghdr *msg, struct scm_cookie *scm)
{
struct cmsghdr __user *cm = (struct cmsghdr __user*)msg->msg_control;
@@ -329,5 +329,4 @@
EXPORT_SYMBOL(scm_send);
EXPORT_SYMBOL(scm_recv);
EXPORT_SYMBOL(put_cmsg);
-EXPORT_SYMBOL(scm_detach_fds);
EXPORT_SYMBOL(scm_fp_dup);
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: 2.6.16-rc6-mm2
2006-03-18 12:40 2.6.16-rc6-mm2 Andrew Morton
` (5 preceding siblings ...)
2006-03-18 18:45 ` [-mm patch] net/core/scm.c: make scm_detach_fds() static Adrian Bunk
@ 2006-03-18 20:21 ` Rafael J. Wysocki
2006-03-18 20:54 ` 2.6.16-rc6-mm2 Andrew Morton
2006-03-19 1:09 ` 2.6.16-rc6-mm2 uninitialized online_policy_cpus.bits[0] Con Kolivas
` (7 subsequent siblings)
14 siblings, 1 reply; 97+ messages in thread
From: Rafael J. Wysocki @ 2006-03-18 20:21 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel
On Saturday 18 March 2006 13:40, Andrew Morton wrote:
>
> ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc6/2.6.16-rc6-mm2/
I get the following oops from it 100% of the time (on boot):
Trying to free ramdisk memory ... ----------- [cut here ] --------- [please bite here ] ---------
Kernel BUG at fs/buffer.c:1629
invalid opcode: 0000 [1] PREEMPT
last sysfs file: /block/hdc/range
CPU 0
Modules linked in:
Pid: 1, comm: swapper Not tainted 2.6.16-rc6-mm2 #16
RIP: 0010:[<ffffffff80280d9a>] <ffffffff80280d9a>{block_invalidatepage+202}
RSP: 0000:ffff81005ff07ac8 EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff810037c09138 RCX: ffff810037c09228
RDX: 0000000000000000 RSI: ffff81005ff07a88 RDI: 0000000000000000
RBP: ffff81005ff07af8 R08: 0000000000000002 R09: ffff81005ff07a99
R10: ffff810037c09138 R11: 0000000000000010 R12: ffff810037c09138
R13: 0000000000001000 R14: ffff810037c09138 R15: ffff810001c34e28
FS: 0000000000000000(0000) GS:ffffffff80690000(0000) knlGS:0000000000000000
CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 00002afbe89a3000 CR3: 000000005fe36000 CR4: 00000000000006e0
Process swapper (pid: 1, threadinfo ffff81005ff06000, task ffff81005ff05530)
Stack: 0000000000000000 ffff810001c34e28 0000000000000002 0000000000000001
ffff810037df47e0 ffffffffffffffff ffff81005ff07b08 ffffffff8027f7b3
ffff81005ff07b28 ffffffff802610d5
Call Trace: <ffffffff8027f7b3>{do_invalidatepage+35}
<ffffffff802610d5>{truncate_complete_page+37} <ffffffff8026154f>{truncate_inode_pages_range+207}
<ffffffff802617c0>{truncate_inode_pages+16} <ffffffff803bac96>{rd_ioctl+86}
<ffffffff8034541d>{blkdev_driver_ioctl+109} <ffffffff80345574>{blkdev_ioctl+164}
<ffffffff8046bb1d>{_spin_unlock_irqrestore+29} <ffffffff8022cdb7>{release_console_sem+423}
<ffffffff8022da48>{vprintk+824} <ffffffff8046bbd3>{_spin_unlock+19}
<ffffffff80284dec>{put_super+44} <ffffffff80284f91>{deactivate_super+145}
<ffffffff8028617b>{block_ioctl+27} <ffffffff80292801>{do_ioctl+49}
<ffffffff80292b1b>{vfs_ioctl+683} <ffffffff80292baa>{sys_ioctl+106}
<ffffffff8069c6d1>{initrd_load+737} <ffffffff80699e1b>{prepare_namespace+139}
<ffffffff8020722c>{init+460} <ffffffff8046bad4>{_spin_unlock_irq+20}
<ffffffff80228b09>{schedule_tail+73} <ffffffff8020a8de>{child_rip+8}
<ffffffff80207060>{init+0} <ffffffff8020a8d6>{child_rip+0}
Code: 0f 0b 68 b9 80 49 80 c2 5d 06 48 8b 5d d8 4c 8b 65 e0 4c 8b
RIP <ffffffff80280d9a>{block_invalidatepage+202} RSP <ffff81005ff07ac8>
<0>Kernel panic - not syncing: Attempted to kill init!
Of course booting with "noinitrd" helps.
Greetings,
Rafael
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: 2.6.16-rc6-mm2
2006-03-18 20:21 ` 2.6.16-rc6-mm2 Rafael J. Wysocki
@ 2006-03-18 20:54 ` Andrew Morton
2006-03-18 21:24 ` 2.6.16-rc6-mm2 Rafael J. Wysocki
` (2 more replies)
0 siblings, 3 replies; 97+ messages in thread
From: Andrew Morton @ 2006-03-18 20:54 UTC (permalink / raw)
To: Rafael J. Wysocki; +Cc: linux-kernel, Neil Brown
"Rafael J. Wysocki" <rjw@sisk.pl> wrote:
>
> On Saturday 18 March 2006 13:40, Andrew Morton wrote:
> >
> > ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc6/2.6.16-rc6-mm2/
>
> I get the following oops from it 100% of the time (on boot):
>
> Trying to free ramdisk memory ... ----------- [cut here ] --------- [please bite here ] ---------
> Kernel BUG at fs/buffer.c:1629
> invalid opcode: 0000 [1] PREEMPT
> last sysfs file: /block/hdc/range
> CPU 0
> Modules linked in:
> Pid: 1, comm: swapper Not tainted 2.6.16-rc6-mm2 #16
> RIP: 0010:[<ffffffff80280d9a>] <ffffffff80280d9a>{block_invalidatepage+202}
> RSP: 0000:ffff81005ff07ac8 EFLAGS: 00010246
> RAX: 0000000000000000 RBX: ffff810037c09138 RCX: ffff810037c09228
> RDX: 0000000000000000 RSI: ffff81005ff07a88 RDI: 0000000000000000
> RBP: ffff81005ff07af8 R08: 0000000000000002 R09: ffff81005ff07a99
> R10: ffff810037c09138 R11: 0000000000000010 R12: ffff810037c09138
> R13: 0000000000001000 R14: ffff810037c09138 R15: ffff810001c34e28
> FS: 0000000000000000(0000) GS:ffffffff80690000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
> CR2: 00002afbe89a3000 CR3: 000000005fe36000 CR4: 00000000000006e0
> Process swapper (pid: 1, threadinfo ffff81005ff06000, task ffff81005ff05530)
> Stack: 0000000000000000 ffff810001c34e28 0000000000000002 0000000000000001
> ffff810037df47e0 ffffffffffffffff ffff81005ff07b08 ffffffff8027f7b3
> ffff81005ff07b28 ffffffff802610d5
> Call Trace: <ffffffff8027f7b3>{do_invalidatepage+35}
> <ffffffff802610d5>{truncate_complete_page+37} <ffffffff8026154f>{truncate_inode_pages_range+207}
> <ffffffff802617c0>{truncate_inode_pages+16} <ffffffff803bac96>{rd_ioctl+86}
Yeah, ramdisk does strange things.
That's two. I guess I need to see if Neil left any other little timebombs
in there for us ;)
--- devel/fs/buffer.c~make-address_space_operations-invalidatepage-return-void-fix 2006-03-18 12:52:37.000000000 -0800
+++ devel-akpm/fs/buffer.c 2006-03-18 12:53:04.000000000 -0800
@@ -1624,10 +1624,8 @@ void block_invalidatepage(struct page *p
* The get_block cached value has been unconditionally invalidated,
* so real IO is not possible anymore.
*/
- if (offset == 0) {
- int ret = try_to_release_page(page, 0);
- BUG_ON(!ret);
- }
+ if (offset == 0)
+ try_to_release_page(page, 0);
out:
return;
}
_
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: 2.6.16-rc6-mm2
2006-03-18 20:54 ` 2.6.16-rc6-mm2 Andrew Morton
@ 2006-03-18 21:24 ` Rafael J. Wysocki
2006-03-18 21:25 ` Emulex IP over FC support shogunx
2006-03-19 23:03 ` 2.6.16-rc6-mm2 - BUG when flushing a ramdisk Neil Brown
2 siblings, 0 replies; 97+ messages in thread
From: Rafael J. Wysocki @ 2006-03-18 21:24 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel, Neil Brown
On Saturday 18 March 2006 21:54, Andrew Morton wrote:
> "Rafael J. Wysocki" <rjw@sisk.pl> wrote:
> >
> > On Saturday 18 March 2006 13:40, Andrew Morton wrote:
> > >
> > > ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc6/2.6.16-rc6-mm2/
> >
> > I get the following oops from it 100% of the time (on boot):
> >
> > Trying to free ramdisk memory ... ----------- [cut here ] --------- [please bite here ] ---------
> > Kernel BUG at fs/buffer.c:1629
> > invalid opcode: 0000 [1] PREEMPT
> > last sysfs file: /block/hdc/range
> > CPU 0
> > Modules linked in:
> > Pid: 1, comm: swapper Not tainted 2.6.16-rc6-mm2 #16
> > RIP: 0010:[<ffffffff80280d9a>] <ffffffff80280d9a>{block_invalidatepage+202}
> > RSP: 0000:ffff81005ff07ac8 EFLAGS: 00010246
> > RAX: 0000000000000000 RBX: ffff810037c09138 RCX: ffff810037c09228
> > RDX: 0000000000000000 RSI: ffff81005ff07a88 RDI: 0000000000000000
> > RBP: ffff81005ff07af8 R08: 0000000000000002 R09: ffff81005ff07a99
> > R10: ffff810037c09138 R11: 0000000000000010 R12: ffff810037c09138
> > R13: 0000000000001000 R14: ffff810037c09138 R15: ffff810001c34e28
> > FS: 0000000000000000(0000) GS:ffffffff80690000(0000) knlGS:0000000000000000
> > CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
> > CR2: 00002afbe89a3000 CR3: 000000005fe36000 CR4: 00000000000006e0
> > Process swapper (pid: 1, threadinfo ffff81005ff06000, task ffff81005ff05530)
> > Stack: 0000000000000000 ffff810001c34e28 0000000000000002 0000000000000001
> > ffff810037df47e0 ffffffffffffffff ffff81005ff07b08 ffffffff8027f7b3
> > ffff81005ff07b28 ffffffff802610d5
> > Call Trace: <ffffffff8027f7b3>{do_invalidatepage+35}
> > <ffffffff802610d5>{truncate_complete_page+37} <ffffffff8026154f>{truncate_inode_pages_range+207}
> > <ffffffff802617c0>{truncate_inode_pages+16} <ffffffff803bac96>{rd_ioctl+86}
>
> Yeah, ramdisk does strange things.
>
> That's two. I guess I need to see if Neil left any other little timebombs
> in there for us ;)
>
> --- devel/fs/buffer.c~make-address_space_operations-invalidatepage-return-void-fix 2006-03-18 12:52:37.000000000 -0800
> +++ devel-akpm/fs/buffer.c 2006-03-18 12:53:04.000000000 -0800
> @@ -1624,10 +1624,8 @@ void block_invalidatepage(struct page *p
> * The get_block cached value has been unconditionally invalidated,
> * so real IO is not possible anymore.
> */
> - if (offset == 0) {
> - int ret = try_to_release_page(page, 0);
> - BUG_ON(!ret);
> - }
> + if (offset == 0)
> + try_to_release_page(page, 0);
> out:
> return;
> }
> _
That helps, thanks.
^ permalink raw reply [flat|nested] 97+ messages in thread* Emulex IP over FC support.
2006-03-18 20:54 ` 2.6.16-rc6-mm2 Andrew Morton
2006-03-18 21:24 ` 2.6.16-rc6-mm2 Rafael J. Wysocki
@ 2006-03-18 21:25 ` shogunx
2006-03-19 23:03 ` 2.6.16-rc6-mm2 - BUG when flushing a ramdisk Neil Brown
2 siblings, 0 replies; 97+ messages in thread
From: shogunx @ 2006-03-18 21:25 UTC (permalink / raw)
To: linux-kernel
Hi all,
Does the current crop of kernel drivers for Emulex HBA's support IP over
FC?
Thanx,
Scott
sleekfreak pirate broadcast
http://sleekfreak.ath.cx:81/
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: 2.6.16-rc6-mm2 - BUG when flushing a ramdisk
2006-03-18 20:54 ` 2.6.16-rc6-mm2 Andrew Morton
2006-03-18 21:24 ` 2.6.16-rc6-mm2 Rafael J. Wysocki
2006-03-18 21:25 ` Emulex IP over FC support shogunx
@ 2006-03-19 23:03 ` Neil Brown
2 siblings, 0 replies; 97+ messages in thread
From: Neil Brown @ 2006-03-19 23:03 UTC (permalink / raw)
To: Andrew Morton; +Cc: Rafael J. Wysocki, linux-kernel
On Saturday March 18, akpm@osdl.org wrote:
> "Rafael J. Wysocki" <rjw@sisk.pl> wrote:
> >
> > On Saturday 18 March 2006 13:40, Andrew Morton wrote:
> > >
> > > ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc6/2.6.16-rc6-mm2/
> >
> > I get the following oops from it 100% of the time (on boot):
> >
> > Trying to free ramdisk memory ... ----------- [cut here ] --------- [please bite here ] ---------
> > Kernel BUG at fs/buffer.c:1629
> > invalid opcode: 0000 [1] PREEMPT
> > last sysfs file: /block/hdc/range
> > CPU 0
> > Modules linked in:
> > Pid: 1, comm: swapper Not tainted 2.6.16-rc6-mm2 #16
> > RIP: 0010:[<ffffffff80280d9a>] <ffffffff80280d9a>{block_invalidatepage+202}
> > RSP: 0000:ffff81005ff07ac8 EFLAGS: 00010246
> > RAX: 0000000000000000 RBX: ffff810037c09138 RCX: ffff810037c09228
> > RDX: 0000000000000000 RSI: ffff81005ff07a88 RDI: 0000000000000000
> > RBP: ffff81005ff07af8 R08: 0000000000000002 R09: ffff81005ff07a99
> > R10: ffff810037c09138 R11: 0000000000000010 R12: ffff810037c09138
> > R13: 0000000000001000 R14: ffff810037c09138 R15: ffff810001c34e28
> > FS: 0000000000000000(0000) GS:ffffffff80690000(0000) knlGS:0000000000000000
> > CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b
> > CR2: 00002afbe89a3000 CR3: 000000005fe36000 CR4: 00000000000006e0
> > Process swapper (pid: 1, threadinfo ffff81005ff06000, task ffff81005ff05530)
> > Stack: 0000000000000000 ffff810001c34e28 0000000000000002 0000000000000001
> > ffff810037df47e0 ffffffffffffffff ffff81005ff07b08 ffffffff8027f7b3
> > ffff81005ff07b28 ffffffff802610d5
> > Call Trace: <ffffffff8027f7b3>{do_invalidatepage+35}
> > <ffffffff802610d5>{truncate_complete_page+37} <ffffffff8026154f>{truncate_inode_pages_range+207}
> > <ffffffff802617c0>{truncate_inode_pages+16} <ffffffff803bac96>{rd_ioctl+86}
>
> Yeah, ramdisk does strange things.
Well... someone is doing strange things....
This BUG would only hit if a page used by ramdisk had buffers
attached, but as far as I can see, rd.c never attaches buffers to its
pages, or sets the PagePrivate flag. So this cannot happen :-)
If there are no users of the page (as one expects given that rd_ioctl
only called truncate_inode_pages is bd_openers is nice and low), then
try_to_release_page should succeed.
On the other hand, if the page could still be in use, then rd_ioctl
should really be calling invalidate_inode_pages() (probably via
invalidate_bdev) rather than truncate_inode_pages.
Either way, I'm not convinced that removing the BUG is the "right"
solution, though I concede that it might be the "practical" solution.
It seems very likely that the failure of try_to_release_page at this
point means that some buffer_heads will be leaking, but as I cannot
reproduce the problem by just playing with a ramdisk (the reported
problem happens during early boot from an initrd) I cannot point to a
clear leak for you....
Maybe the "right" thing to do is for rd.c to define an invalidate_page
function that removes dirty buffers (which try_to_release_page won't),
but that would only make sense if rd.c put buffers on the page in the
first place....
Confused!
>
> That's two. I guess I need to see if Neil left any other little timebombs
> in there for us ;)
No, I think you've got them all now. One in block_invalidatepage, one
in journal_invalidatepage, and one in jfs that has been confirmed as
'OK'.
NeilBrown
>
> --- devel/fs/buffer.c~make-address_space_operations-invalidatepage-return-void-fix 2006-03-18 12:52:37.000000000 -0800
> +++ devel-akpm/fs/buffer.c 2006-03-18 12:53:04.000000000 -0800
> @@ -1624,10 +1624,8 @@ void block_invalidatepage(struct page *p
> * The get_block cached value has been unconditionally invalidated,
> * so real IO is not possible anymore.
> */
> - if (offset == 0) {
> - int ret = try_to_release_page(page, 0);
> - BUG_ON(!ret);
> - }
> + if (offset == 0)
> + try_to_release_page(page, 0);
> out:
> return;
> }
> _
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: 2.6.16-rc6-mm2 uninitialized online_policy_cpus.bits[0]
2006-03-18 12:40 2.6.16-rc6-mm2 Andrew Morton
` (6 preceding siblings ...)
2006-03-18 20:21 ` 2.6.16-rc6-mm2 Rafael J. Wysocki
@ 2006-03-19 1:09 ` Con Kolivas
2006-03-19 1:29 ` Con Kolivas
2006-03-19 1:35 ` Andrew Morton
2006-03-20 3:26 ` New Areca driver in 2.6.16-rc6-mm2 Dax Kelson
` (6 subsequent siblings)
14 siblings, 2 replies; 97+ messages in thread
From: Con Kolivas @ 2006-03-19 1:09 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel
Wonder if this is related to rc6's oops?
gcc 4.0.3
CC [M] arch/i386/kernel/cpu/cpufreq/speedstep-centrino.o
arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c: In function
'centrino_target':
include/linux/bitmap.h:170: warning: 'online_policy_cpus.bits[0]' is used
uninitialized in this function
CC [M] arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.o
arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.c: In function
'acpi_cpufreq_target':
include/linux/bitmap.h:170: warning: 'online_policy_cpus.bits[0]' is used
uninitialized in this function
config here:
http://ck.kolivas.org/crap/lapconfig
Con
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: 2.6.16-rc6-mm2 uninitialized online_policy_cpus.bits[0]
2006-03-19 1:09 ` 2.6.16-rc6-mm2 uninitialized online_policy_cpus.bits[0] Con Kolivas
@ 2006-03-19 1:29 ` Con Kolivas
2006-03-19 1:35 ` Andrew Morton
1 sibling, 0 replies; 97+ messages in thread
From: Con Kolivas @ 2006-03-19 1:29 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel
On Sunday 19 March 2006 12:09, Con Kolivas wrote:
> Wonder if this is related to rc6's oops?
> gcc 4.0.3
>
> CC [M] arch/i386/kernel/cpu/cpufreq/speedstep-centrino.o
> arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c: In function
> 'centrino_target':
> include/linux/bitmap.h:170: warning: 'online_policy_cpus.bits[0]' is used
> uninitialized in this function
> CC [M] arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.o
> arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.c: In function
> 'acpi_cpufreq_target':
> include/linux/bitmap.h:170: warning: 'online_policy_cpus.bits[0]' is used
> uninitialized in this function
>
> config here:
> http://ck.kolivas.org/crap/lapconfig
Sorry wrong one:
http://ck.kolivas.org/crap/lap_config
Con
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: 2.6.16-rc6-mm2 uninitialized online_policy_cpus.bits[0]
2006-03-19 1:09 ` 2.6.16-rc6-mm2 uninitialized online_policy_cpus.bits[0] Con Kolivas
2006-03-19 1:29 ` Con Kolivas
@ 2006-03-19 1:35 ` Andrew Morton
2006-03-19 2:37 ` Con Kolivas
2006-03-19 6:13 ` Venkatesh Pallipadi
1 sibling, 2 replies; 97+ messages in thread
From: Andrew Morton @ 2006-03-19 1:35 UTC (permalink / raw)
To: Con Kolivas; +Cc: linux-kernel, Venkatesh Pallipadi, Len Brown, linux-acpi
Con Kolivas <kernel@kolivas.org> wrote:
>
> Wonder if this is related to rc6's oops?
> gcc 4.0.3
>
> CC [M] arch/i386/kernel/cpu/cpufreq/speedstep-centrino.o
> arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c: In function
> 'centrino_target':
> include/linux/bitmap.h:170: warning: 'online_policy_cpus.bits[0]' is used
> uninitialized in this function
> CC [M] arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.o
> arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.c: In function
> 'acpi_cpufreq_target':
> include/linux/bitmap.h:170: warning: 'online_policy_cpus.bits[0]' is used
> uninitialized in this function
Well conceivably. That warning is a consequence of my quick hack to make
the ACPI tree compile on uniprocessor.
ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc6/2.6.16-rc6-mm2/broken-out/git-acpi-up-fix.patch
My patch is, as the compiler points out, wrong.
I've sent that patch two or three times to the APCI maintainers, to the
ACPI mailing list and to the author of the original buggy patch. The
response thus far has been dead silence.
IOW, despite my efforts, the ACPI tree has been in a non-compiling state on
uniprocessor since February 11.
This is pathetic. People are trying to get things done here and ACPI is
getting in the way. But *need* to get the ACPI development tree out for
people to test else we'll never be able to take another ACPI update into
mainline.
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: 2.6.16-rc6-mm2 uninitialized online_policy_cpus.bits[0]
2006-03-19 1:35 ` Andrew Morton
@ 2006-03-19 2:37 ` Con Kolivas
2006-03-19 6:13 ` Venkatesh Pallipadi
1 sibling, 0 replies; 97+ messages in thread
From: Con Kolivas @ 2006-03-19 2:37 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel, Venkatesh Pallipadi, Len Brown, linux-acpi
On Sunday 19 March 2006 12:35, Andrew Morton wrote:
> Con Kolivas <kernel@kolivas.org> wrote:
> > Wonder if this is related to rc6's oops?
> > gcc 4.0.3
> >
> > CC [M] arch/i386/kernel/cpu/cpufreq/speedstep-centrino.o
> > arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c: In function
> > 'centrino_target':
> > include/linux/bitmap.h:170: warning: 'online_policy_cpus.bits[0]' is
> > used uninitialized in this function
> > CC [M] arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.o
> > arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.c: In function
> > 'acpi_cpufreq_target':
> > include/linux/bitmap.h:170: warning: 'online_policy_cpus.bits[0]' is
> > used uninitialized in this function
>
> Well conceivably. That warning is a consequence of my quick hack to make
> the ACPI tree compile on uniprocessor.
>
> ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc6/2.
>6.16-rc6-mm2/broken-out/git-acpi-up-fix.patch
>
> My patch is, as the compiler points out, wrong.
>
> I've sent that patch two or three times to the APCI maintainers, to the
> ACPI mailing list and to the author of the original buggy patch. The
> response thus far has been dead silence.
Well this will end up being the wrong place to do it but I needed it to work
now so this patch fixes it for me. Dunno what else it will break. Works on
a couple of configs fine.
Cheers,
Con
---
Hacky workaround for cpu_online_map not being defined
arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.c | 2 --
arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c | 2 --
kernel/sched.c | 4 ++++
3 files changed, 4 insertions(+), 4 deletions(-)
Index: linux-2.6.16-rc6-mm2/arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.c
===================================================================
--- linux-2.6.16-rc6-mm2.orig/arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.c 2006-03-19 11:15:05.000000000 +1100
+++ linux-2.6.16-rc6-mm2/arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.c 2006-03-19 12:45:56.000000000 +1100
@@ -225,10 +225,8 @@ acpi_cpufreq_target (
freqs.old = data->freq_table[cur_state].frequency;
freqs.new = data->freq_table[next_state].frequency;
-#ifdef CONFIG_SMP
/* cpufreq holds the hotplug lock, so we are safe from here on */
cpus_and(online_policy_cpus, cpu_online_map, policy->cpus);
-#endif
for_each_cpu_mask(j, online_policy_cpus) {
freqs.cpu = j;
Index: linux-2.6.16-rc6-mm2/arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c
===================================================================
--- linux-2.6.16-rc6-mm2.orig/arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c 2006-03-19 11:15:05.000000000 +1100
+++ linux-2.6.16-rc6-mm2/arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c 2006-03-19 12:45:42.000000000 +1100
@@ -652,10 +652,8 @@ static int centrino_target (struct cpufr
return -EINVAL;
}
-#ifdef CONFIG_SMP
/* cpufreq holds the hotplug lock, so we are safe from here on */
cpus_and(online_policy_cpus, cpu_online_map, policy->cpus);
-#endif
saved_mask = current->cpus_allowed;
first_cpu = 1;
Index: linux-2.6.16-rc6-mm2/kernel/sched.c
===================================================================
--- linux-2.6.16-rc6-mm2.orig/kernel/sched.c 2006-03-19 13:25:25.000000000 +1100
+++ linux-2.6.16-rc6-mm2/kernel/sched.c 2006-03-19 13:25:36.000000000 +1100
@@ -6366,6 +6366,10 @@ void __init sched_init_smp(void)
init_sched_domain_sysctl();
}
#else
+/* bitmap of online cpus */
+cpumask_t cpu_online_map __read_mostly;
+EXPORT_SYMBOL(cpu_online_map);
+
void __init sched_init_smp(void)
{
}
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: 2.6.16-rc6-mm2 uninitialized online_policy_cpus.bits[0]
2006-03-19 1:35 ` Andrew Morton
2006-03-19 2:37 ` Con Kolivas
@ 2006-03-19 6:13 ` Venkatesh Pallipadi
1 sibling, 0 replies; 97+ messages in thread
From: Venkatesh Pallipadi @ 2006-03-19 6:13 UTC (permalink / raw)
To: Andrew Morton
Cc: Con Kolivas, linux-kernel, Venkatesh Pallipadi, Len Brown,
linux-acpi
On Sat, Mar 18, 2006 at 05:35:12PM -0800, Andrew Morton wrote:
> Con Kolivas <kernel@kolivas.org> wrote:
> >
> > Wonder if this is related to rc6's oops?
> > gcc 4.0.3
> >
> > CC [M] arch/i386/kernel/cpu/cpufreq/speedstep-centrino.o
> > arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c: In function
> > 'centrino_target':
> > include/linux/bitmap.h:170: warning: 'online_policy_cpus.bits[0]' is used
> > uninitialized in this function
> > CC [M] arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.o
> > arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.c: In function
> > 'acpi_cpufreq_target':
> > include/linux/bitmap.h:170: warning: 'online_policy_cpus.bits[0]' is used
> > uninitialized in this function
>
> Well conceivably. That warning is a consequence of my quick hack to make
> the ACPI tree compile on uniprocessor.
>
> ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc6/2.6.16-rc6-mm2/broken-out/git-acpi-up-fix.patch
>
> My patch is, as the compiler points out, wrong.
>
> I've sent that patch two or three times to the APCI maintainers, to the
> ACPI mailing list and to the author of the original buggy patch. The
> response thus far has been dead silence.
>
> IOW, despite my efforts, the ACPI tree has been in a non-compiling state on
> uniprocessor since February 11.
>
> This is pathetic. People are trying to get things done here and ACPI is
> getting in the way.
Oops. Sorry. It is me who dropped the ball here. I somehow assumed that
your original patch fixed the issue and didn't look at the actual code.
Here is the patch against mm.
Thanks,
Venki
Fix the UP build breakage in acpi-cpufreq and speedstep-centrino due to
previous p-state software coordination patch.
Signed-off-by: Venkatesh Pallipadi <venkatesh.pallipadi@intel.com>
diff -purN linux-2.6.15/arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.c linux-2.6.15-new/arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.c
--- linux-2.6.15/arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.c 2006-03-18 19:41:25.000000000 -0800
+++ linux-2.6.15-new/arch/i386/kernel/cpu/cpufreq/acpi-cpufreq.c 2006-03-18 19:48:10.000000000 -0800
@@ -225,9 +225,11 @@ acpi_cpufreq_target (
freqs.old = data->freq_table[cur_state].frequency;
freqs.new = data->freq_table[next_state].frequency;
-#ifdef CONFIG_SMP
+#ifdef CONFIG_HOTPLUG_CPU
/* cpufreq holds the hotplug lock, so we are safe from here on */
cpus_and(online_policy_cpus, cpu_online_map, policy->cpus);
+#else
+ online_policy_cpus = policy->cpus;
#endif
for_each_cpu_mask(j, online_policy_cpus) {
diff -purN linux-2.6.15/arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c linux-2.6.15-new/arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c
--- linux-2.6.15/arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c 2006-03-18 19:41:25.000000000 -0800
+++ linux-2.6.15-new/arch/i386/kernel/cpu/cpufreq/speedstep-centrino.c 2006-03-18 19:47:46.000000000 -0800
@@ -652,9 +652,11 @@ static int centrino_target (struct cpufr
return -EINVAL;
}
-#ifdef CONFIG_SMP
+#ifdef CONFIG_HOTPLUG_CPU
/* cpufreq holds the hotplug lock, so we are safe from here on */
cpus_and(online_policy_cpus, cpu_online_map, policy->cpus);
+#else
+ online_policy_cpus = policy->cpus;
#endif
saved_mask = current->cpus_allowed;
^ permalink raw reply [flat|nested] 97+ messages in thread
* New Areca driver in 2.6.16-rc6-mm2
2006-03-18 12:40 2.6.16-rc6-mm2 Andrew Morton
` (7 preceding siblings ...)
2006-03-19 1:09 ` 2.6.16-rc6-mm2 uninitialized online_policy_cpus.bits[0] Con Kolivas
@ 2006-03-20 3:26 ` Dax Kelson
2006-03-21 23:49 ` Chris Caputo
2006-03-20 13:02 ` RAID5 grow success (was: Re: 2.6.16-rc6-mm2) Sander
` (5 subsequent siblings)
14 siblings, 1 reply; 97+ messages in thread
From: Dax Kelson @ 2006-03-20 3:26 UTC (permalink / raw)
To: linux-kernel; +Cc: hch, James.Bottomley, erich
On Sat, 18 Mar 2006, Andrew Morton wrote:
> SCSI fixes
>
> +areca-raid-linux-scsi-driver-update4.patch
>
> Update areca-raid-linux-scsi-driver.patch
Has anyone had a chance to review this new update to see if it now passes
muster for mainline inclusion?
As a owner of Areca hardware I sure appreciate the responsiveness and work
they have done in respects to Linux support. Thanks to Randy, Christoph,
Randy and others for their work as well!
Dax Kelson
===========
ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc6/2.6.16-rc6-mm2/broken-out/areca-raid-linux-scsi-driver.patch
ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc6/2.6.16-rc6-mm2/broken-out/areca-raid-linux-scsi-driver-update4.patch
From: Erich Chen <erich@areca.com.tw>
1- remove internal queueing
2- remove odd ioctls
3- remove useless forward prototypes
4- give types like ACB useful names
5- give variable useful names, especially follow kernel conventions,
e.g. a struct pci_dev is usually named pdev
6- kill ->proc_info method
7- use normal comment style even for comments not fitting into the
kernel-doc item above. kill useless separator comments without
text
8- convert arcmsr_show_firmware_info to useful one value per
file attributes.
9- convert arcmsr_show_driver_state to useful one value per
file attributes.
10- remove never called release method in the host template
11- remove shutdown notifier, add pci_driver ->shutdown method instead
12- remove CameCase PCI Ids. The vendor Id should go into pci_ids.h,
the device ids either removed or spelled the normal linux way
13- arcmsr_do_interrupt should stop walking the global host list
and use the private data passed to request_irq
14- the global host list go away completely
15- locking to be redone.
16- arcmsr_device_probe rewritten to do goto-based
error unwinding.
17- remove msi options
18- remove arcmsr_scsi_host_template_init
19- the hardware documentation move from arcmsr.h
into a separate file (Documentation/scsi/arcmsr_spec.txt)
20- remove the SCSISTAT_* defines
21- split arcmsr.c into arcmsr_attr.c from arcmsr_hba.c
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: New Areca driver in 2.6.16-rc6-mm2
2006-03-20 3:26 ` New Areca driver in 2.6.16-rc6-mm2 Dax Kelson
@ 2006-03-21 23:49 ` Chris Caputo
2006-03-22 3:37 ` Matti Aarnio
0 siblings, 1 reply; 97+ messages in thread
From: Chris Caputo @ 2006-03-21 23:49 UTC (permalink / raw)
To: Dax Kelson; +Cc: linux-kernel, erich
On Sun, 19 Mar 2006, Dax Kelson wrote:
> On Sat, 18 Mar 2006, Andrew Morton wrote:
> > SCSI fixes
> >
> > +areca-raid-linux-scsi-driver-update4.patch
> >
> > Update areca-raid-linux-scsi-driver.patch
>
> Has anyone had a chance to review this new update to see if it now passes
> muster for mainline inclusion?
Unfortunately when the new driver is applied to 2.6.15.6 a bonnie++ test
results in the following endless spew:
...
attempt to access beyond end of device
sdb1: rw=0, want=134744080, limit=128002016
attempt to access beyond end of device
sdb1: rw=0, want=134744080, limit=128002016
attempt to access beyond end of device
sdb1: rw=0, want=134744080, limit=128002016
attempt to access beyond end of device
sdb1: rw=0, want=134744080, limit=128002016
attempt to access beyond end of device
sdb1: rw=0, want=134744080, limit=128002016
attempt to access beyond end of device
sdb1: rw=0, want=134744080, limit=128002016
attempt to access beyond end of device
sdb1: rw=0, want=134744080, limit=128002016
...
I have emailed the details to Erich.
Chris
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: New Areca driver in 2.6.16-rc6-mm2
2006-03-21 23:49 ` Chris Caputo
@ 2006-03-22 3:37 ` Matti Aarnio
2006-03-22 3:56 ` Randy.Dunlap
2006-03-22 4:41 ` erich
0 siblings, 2 replies; 97+ messages in thread
From: Matti Aarnio @ 2006-03-22 3:37 UTC (permalink / raw)
To: Chris Caputo; +Cc: Dax Kelson, linux-kernel, erich
On Tue, Mar 21, 2006 at 11:49:32PM +0000, Chris Caputo wrote:
> On Sun, 19 Mar 2006, Dax Kelson wrote:
> > On Sat, 18 Mar 2006, Andrew Morton wrote:
> > > SCSI fixes
> > >
> > > +areca-raid-linux-scsi-driver-update4.patch
> > >
> > > Update areca-raid-linux-scsi-driver.patch
> >
> > Has anyone had a chance to review this new update to see if it now passes
> > muster for mainline inclusion?
>
> Unfortunately when the new driver is applied to 2.6.15.6 a bonnie++ test
> results in the following endless spew:
Curious... I didn't encounter this phenomena, but then, my 0.75 TB
raid5 volume is practically empty...
For the development phase it would be most useful, if the driver
would be available in similar "this will compile for your currently
running kernel, or some other you care to name and have its config.h
files at hand" as e.g. Nvidia drivers are (except that arcmsr is
in "all source form", whereas NV has this magic object blob..)
Such would allow (at least for me) to have a wee bit faster cycle
with "pick vendor kernel, add this and that custom module"
I was apalled to learn that full cycle kernel compilation takes
_hours_ these days (Pentium-4 HT, 2.4 GHz, 2 GB memory -- and it
is about as slow as my first kernel compilation experience with
a 386/33MHz way back in ...)
> ...
> attempt to access beyond end of device
> sdb1: rw=0, want=134744080, limit=128002016
> ...
>
> I have emailed the details to Erich.
>
> Chris
/Matti Aarnio
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: New Areca driver in 2.6.16-rc6-mm2
2006-03-22 3:37 ` Matti Aarnio
@ 2006-03-22 3:56 ` Randy.Dunlap
2006-03-22 3:56 ` Al Viro
2006-03-22 4:41 ` erich
1 sibling, 1 reply; 97+ messages in thread
From: Randy.Dunlap @ 2006-03-22 3:56 UTC (permalink / raw)
To: Matti Aarnio; +Cc: ccaputo, dax, linux-kernel, erich
On Wed, 22 Mar 2006 05:37:18 +0200 Matti Aarnio wrote:
>
> I was apalled to learn that full cycle kernel compilation takes
> _hours_ these days (Pentium-4 HT, 2.4 GHz, 2 GB memory -- and it
> is about as slow as my first kernel compilation experience with
> a 386/33MHz way back in ...)
You mean allmodconfig or allyesconfig, right?
Yes, it does take a l o n g time.
---
~Randy
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: New Areca driver in 2.6.16-rc6-mm2
2006-03-22 3:56 ` Randy.Dunlap
@ 2006-03-22 3:56 ` Al Viro
0 siblings, 0 replies; 97+ messages in thread
From: Al Viro @ 2006-03-22 3:56 UTC (permalink / raw)
To: Randy.Dunlap; +Cc: Matti Aarnio, ccaputo, dax, linux-kernel, erich
On Tue, Mar 21, 2006 at 07:56:26PM -0800, Randy.Dunlap wrote:
> On Wed, 22 Mar 2006 05:37:18 +0200 Matti Aarnio wrote:
>
>
> >
> > I was apalled to learn that full cycle kernel compilation takes
> > _hours_ these days (Pentium-4 HT, 2.4 GHz, 2 GB memory -- and it
> > is about as slow as my first kernel compilation experience with
> > a 386/33MHz way back in ...)
>
> You mean allmodconfig or allyesconfig, right?
> Yes, it does take a l o n g time.
Kill CONFIG_DEBUG_INFO and it'll go much faster...
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: New Areca driver in 2.6.16-rc6-mm2
2006-03-22 3:37 ` Matti Aarnio
2006-03-22 3:56 ` Randy.Dunlap
@ 2006-03-22 4:41 ` erich
1 sibling, 0 replies; 97+ messages in thread
From: erich @ 2006-03-22 4:41 UTC (permalink / raw)
To: Matti Aarnio; +Cc: Dax Kelson, linux-kernel, Randy.Dunlap, Al Viro
Dear All,
I had met this problem at my Fab before.
But this problem seem not came from the issue of driver version change.
I had test it with my older version of areca driver and it cause same
problem.
I did this testing with EXT2 file system and I got dump messages as
following message.
But When I used EXT3 file system and run same testing, it worked fine.
attempt to access beyond end of device
sdb1: rw=0, want=134744080, limit=128002016
attempt to access beyond end of device
sdb1: rw=0, want=134744080, limit=128002016
attempt to access beyond end of device
sdb1: rw=0, want=134744080, limit=128002016
attempt to access beyond end of device
sdb1: rw=0, want=134744080, limit=128002016
attempt to access beyond end of device
sdb1: rw=0, want=134744080, limit=128002016
attempt to access beyond end of device
sdb1: rw=0, want=134744080, limit=128002016
attempt to access beyond end of device
sdb1: rw=0, want=134744080, limit=128002016
Now I have time to research this problem.
Hope that I can give you more information about it.
Best Regards
Erich Chen
----- Original Message -----
From: "Matti Aarnio" <matti.aarnio@zmailer.org>
To: "Chris Caputo" <ccaputo@alt.net>
Cc: "Dax Kelson" <dax@gurulabs.com>; <linux-kernel@vger.kernel.org>;
<erich@areca.com.tw>
Sent: Wednesday, March 22, 2006 11:37 AM
Subject: Re: New Areca driver in 2.6.16-rc6-mm2
> On Tue, Mar 21, 2006 at 11:49:32PM +0000, Chris Caputo wrote:
>> On Sun, 19 Mar 2006, Dax Kelson wrote:
>> > On Sat, 18 Mar 2006, Andrew Morton wrote:
>> > > SCSI fixes
>> > >
>> > > +areca-raid-linux-scsi-driver-update4.patch
>> > >
>> > > Update areca-raid-linux-scsi-driver.patch
>> >
>> > Has anyone had a chance to review this new update to see if it now
>> > passes
>> > muster for mainline inclusion?
>>
>> Unfortunately when the new driver is applied to 2.6.15.6 a bonnie++ test
>> results in the following endless spew:
>
> Curious... I didn't encounter this phenomena, but then, my 0.75 TB
> raid5 volume is practically empty...
>
> For the development phase it would be most useful, if the driver
> would be available in similar "this will compile for your currently
> running kernel, or some other you care to name and have its config.h
> files at hand" as e.g. Nvidia drivers are (except that arcmsr is
> in "all source form", whereas NV has this magic object blob..)
>
> Such would allow (at least for me) to have a wee bit faster cycle
> with "pick vendor kernel, add this and that custom module"
>
> I was apalled to learn that full cycle kernel compilation takes
> _hours_ these days (Pentium-4 HT, 2.4 GHz, 2 GB memory -- and it
> is about as slow as my first kernel compilation experience with
> a 386/33MHz way back in ...)
>
>> ...
>> attempt to access beyond end of device
>> sdb1: rw=0, want=134744080, limit=128002016
>> ...
>>
>> I have emailed the details to Erich.
>>
>> Chris
>
> /Matti Aarnio
^ permalink raw reply [flat|nested] 97+ messages in thread
* RAID5 grow success (was: Re: 2.6.16-rc6-mm2)
2006-03-18 12:40 2.6.16-rc6-mm2 Andrew Morton
` (8 preceding siblings ...)
2006-03-20 3:26 ` New Areca driver in 2.6.16-rc6-mm2 Dax Kelson
@ 2006-03-20 13:02 ` Sander
2006-03-20 13:33 ` Some sata_mv error messages " Sander
` (4 subsequent siblings)
14 siblings, 0 replies; 97+ messages in thread
From: Sander @ 2006-03-20 13:02 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel, neilb, linux-raid
Andrew Morton wrote (ao):
> - Lots of MD and DM updates
I would like to report that growing an online raid5 device works like a
charm:
mdadm --version
mdadm - v2.4-pre1 - Not For Production Use - 20 March 2006
mdadm -C -l5 -n3 /dev/md0 /dev/sda1 /dev/sdb1 /dev/sdc1
While performing:
for i in `seq 4`
do dd if=/dev/zero of=bigfile.$i bs=1024k count=10000
done
md5sum bigfile.*
I do:
mdadm /dev/md0 -a /dev/sdd1
mdadm --grow /dev/md0 --raid-disks=4
When the dd and md5sum finishes, I umount and:
e2fsck -f /dev/md0
resize2fs -p /dev/md0
After mounting again the disk indeed is bigger, and the md5sum still
matches.
FWIW, I now try to add four more spares at once, and grow the raid5
again. It seems to work:
# mdadm /dev/md0 -a /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1
mdadm: added /dev/sde1
mdadm: added /dev/sdf1
mdadm: added /dev/sdg1
mdadm: added /dev/sdh1
# mdadm --grow /dev/md0 --raid-disks=8
mdadm: Need to backup 448K of critical section..
mdadm: ... critical section passed.
#
It is still reshapeing ATM.
Thanks!
Sander
--
Humilis IT Services and Solutions
http://www.humilis.net
^ permalink raw reply [flat|nested] 97+ messages in thread* Some sata_mv error messages (was: Re: 2.6.16-rc6-mm2)
2006-03-18 12:40 2.6.16-rc6-mm2 Andrew Morton
` (9 preceding siblings ...)
2006-03-20 13:02 ` RAID5 grow success (was: Re: 2.6.16-rc6-mm2) Sander
@ 2006-03-20 13:33 ` Sander
2006-03-21 1:02 ` Some sata_mv error messages Jeff Garzik
2006-03-21 20:27 ` 2.6.16-rc6-mm2: reiser4 BUG when unmounting fs Laurent Riffard
` (3 subsequent siblings)
14 siblings, 1 reply; 97+ messages in thread
From: Sander @ 2006-03-20 13:33 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel, linux-ide, jgarzik, lkml
Hi all,
While sata_mv in 2.6.16-rc6-mm2 seems stable (yah!) compared to
2.6.16-rc6 (no crashes, no data corruption), I still get these messages:
[ 3962.139906] ata5: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ 0xb/47/00
[ 3962.139959] ata5: status=0xd0 { Busy }
[ 6105.948045] ata6: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ 0xb/47/00
[ 6105.948097] ata6: status=0xd0 { Busy }
[ 7981.164936] ata5: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ 0xb/47/00
[ 7981.164991] ata5: status=0xd0 { Busy }
[ 8273.951019] ata7: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ 0xb/47/00
[ 8273.951072] ata7: status=0xd0 { Busy }
[ 9903.032350] ata8: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ 0xb/47/00
[ 9903.032402] ata8: status=0xd0 { Busy }
I'm not entirely sure this is only happens on sata_mv (Marvell
MV88SX6081) as out of eight disks only one is connected to the onboard
sata_nv (nVidia) and the error doesn't happen very often. But I'll keep
an eye on it.
Are these messages somehow dangerous or otherwise indicating a
potentional serious problem? A google search came up with a few links,
but none of them helped me understand the messages.
Thanks!
Sander
--
Humilis IT Services and Solutions
http://www.humilis.net
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: Some sata_mv error messages
2006-03-20 13:33 ` Some sata_mv error messages " Sander
@ 2006-03-21 1:02 ` Jeff Garzik
2006-03-21 2:25 ` Dave Jones
` (2 more replies)
0 siblings, 3 replies; 97+ messages in thread
From: Jeff Garzik @ 2006-03-21 1:02 UTC (permalink / raw)
To: sander; +Cc: Andrew Morton, linux-kernel, linux-ide, lkml
Sander wrote:
> Hi all,
>
> While sata_mv in 2.6.16-rc6-mm2 seems stable (yah!) compared to
> 2.6.16-rc6 (no crashes, no data corruption), I still get these messages:
>
> [ 3962.139906] ata5: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ 0xb/47/00
> [ 3962.139959] ata5: status=0xd0 { Busy }
>
> [ 6105.948045] ata6: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ 0xb/47/00
> [ 6105.948097] ata6: status=0xd0 { Busy }
>
> [ 7981.164936] ata5: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ 0xb/47/00
> [ 7981.164991] ata5: status=0xd0 { Busy }
>
> [ 8273.951019] ata7: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ 0xb/47/00
> [ 8273.951072] ata7: status=0xd0 { Busy }
>
> [ 9903.032350] ata8: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ 0xb/47/00
> [ 9903.032402] ata8: status=0xd0 { Busy }
>
>
> I'm not entirely sure this is only happens on sata_mv (Marvell
> MV88SX6081) as out of eight disks only one is connected to the onboard
> sata_nv (nVidia) and the error doesn't happen very often. But I'll keep
> an eye on it.
>
> Are these messages somehow dangerous or otherwise indicating a
> potentional serious problem? A google search came up with a few links,
> but none of them helped me understand the messages.
Without answering your specific question, just remember that sata_mv is
considerly "highly experimental" right now, and still needs some
workarounds for hardware errata.
For now, the goal is a system that doesn't crash and doesn't corrupt
data. If its occasionally slow or spits out a few errors, but otherwise
still works, that's pretty darned good :)
Jeff
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: Some sata_mv error messages
2006-03-21 1:02 ` Some sata_mv error messages Jeff Garzik
@ 2006-03-21 2:25 ` Dave Jones
2006-03-21 2:35 ` Jeff Garzik
2006-03-21 4:48 ` Mark Lord
2006-03-21 7:26 ` Sander
2 siblings, 1 reply; 97+ messages in thread
From: Dave Jones @ 2006-03-21 2:25 UTC (permalink / raw)
To: Jeff Garzik; +Cc: sander, Andrew Morton, linux-kernel, linux-ide, lkml, pjones
On Mon, Mar 20, 2006 at 08:02:06PM -0500, Jeff Garzik wrote:
> Sander wrote:
> >Hi all,
> >
> >While sata_mv in 2.6.16-rc6-mm2 seems stable (yah!) compared to
> >2.6.16-rc6 (no crashes, no data corruption), I still get these messages:
> >
> >[ 3962.139906] ata5: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ
> >0xb/47/00
> >[ 3962.139959] ata5: status=0xd0 { Busy }
> >
> >[ 6105.948045] ata6: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ
> >0xb/47/00
> >[ 6105.948097] ata6: status=0xd0 { Busy }
> >
> >[ 7981.164936] ata5: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ
> >0xb/47/00
> >[ 7981.164991] ata5: status=0xd0 { Busy }
> >
> >[ 8273.951019] ata7: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ
> >0xb/47/00
> >[ 8273.951072] ata7: status=0xd0 { Busy }
> >
> >[ 9903.032350] ata8: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ
> >0xb/47/00
> >[ 9903.032402] ata8: status=0xd0 { Busy }
> >
> >
> >I'm not entirely sure this is only happens on sata_mv (Marvell
> >MV88SX6081) as out of eight disks only one is connected to the onboard
> >sata_nv (nVidia) and the error doesn't happen very often. But I'll keep
> >an eye on it.
> >
> >Are these messages somehow dangerous or otherwise indicating a
> >potentional serious problem? A google search came up with a few links,
> >but none of them helped me understand the messages.
>
> Without answering your specific question, just remember that sata_mv is
> considerly "highly experimental" right now, and still needs some
> workarounds for hardware errata.
>
> For now, the goal is a system that doesn't crash and doesn't corrupt
> data. If its occasionally slow or spits out a few errors, but otherwise
> still works, that's pretty darned good :)
Reminds me, this message (though different error codes) gets spewed to the
console a lot when haldaemon polls SATA CD drives.
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=183348
This wasn't occasional, this was every few seconds, making the box
pretty much unusable.
Dave
--
http://www.codemonkey.org.uk
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: Some sata_mv error messages
2006-03-21 2:25 ` Dave Jones
@ 2006-03-21 2:35 ` Jeff Garzik
2006-03-21 16:06 ` Peter Jones
0 siblings, 1 reply; 97+ messages in thread
From: Jeff Garzik @ 2006-03-21 2:35 UTC (permalink / raw)
To: Dave Jones; +Cc: sander, Andrew Morton, linux-kernel, linux-ide, lkml, pjones
Dave Jones wrote:
> On Mon, Mar 20, 2006 at 08:02:06PM -0500, Jeff Garzik wrote:
> > Sander wrote:
> > >Hi all,
> > >
> > >While sata_mv in 2.6.16-rc6-mm2 seems stable (yah!) compared to
> > >2.6.16-rc6 (no crashes, no data corruption), I still get these messages:
> > >
> > >[ 3962.139906] ata5: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ
> > >0xb/47/00
> > >[ 3962.139959] ata5: status=0xd0 { Busy }
> > >
> > >[ 6105.948045] ata6: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ
> > >0xb/47/00
> > >[ 6105.948097] ata6: status=0xd0 { Busy }
> > >
> > >[ 7981.164936] ata5: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ
> > >0xb/47/00
> > >[ 7981.164991] ata5: status=0xd0 { Busy }
> > >
> > >[ 8273.951019] ata7: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ
> > >0xb/47/00
> > >[ 8273.951072] ata7: status=0xd0 { Busy }
> > >
> > >[ 9903.032350] ata8: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ
> > >0xb/47/00
> > >[ 9903.032402] ata8: status=0xd0 { Busy }
> > >
> > >
> > >I'm not entirely sure this is only happens on sata_mv (Marvell
> > >MV88SX6081) as out of eight disks only one is connected to the onboard
> > >sata_nv (nVidia) and the error doesn't happen very often. But I'll keep
> > >an eye on it.
> > >
> > >Are these messages somehow dangerous or otherwise indicating a
> > >potentional serious problem? A google search came up with a few links,
> > >but none of them helped me understand the messages.
> >
> > Without answering your specific question, just remember that sata_mv is
> > considerly "highly experimental" right now, and still needs some
> > workarounds for hardware errata.
> >
> > For now, the goal is a system that doesn't crash and doesn't corrupt
> > data. If its occasionally slow or spits out a few errors, but otherwise
> > still works, that's pretty darned good :)
>
> Reminds me, this message (though different error codes) gets spewed to the
> console a lot when haldaemon polls SATA CD drives.
> https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=183348
Yeah, libata is way too noisy on ATAPI. In ATA, errors were rare so you
needed all the info. In ATAPI, some "errors" are expected, and simply
reported back to the application. Unfortunately, the solution is
observation and iteration:
Console spews messages. Users yell, and paste said messages. Observe
ATAPI behavior and return codes, and adjust libata. Repeat.
As an aside, most recent SATA cd/dvd drives support "asynchronous
notification", which means you don't have the poll the drive. HAL needs
to support that.
> This wasn't occasional, this was every few seconds, making the box
> pretty much unusable.
The entire box or just the console?
Jeff
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: Some sata_mv error messages
2006-03-21 2:35 ` Jeff Garzik
@ 2006-03-21 16:06 ` Peter Jones
0 siblings, 0 replies; 97+ messages in thread
From: Peter Jones @ 2006-03-21 16:06 UTC (permalink / raw)
To: Jeff Garzik
Cc: Dave Jones, sander, Andrew Morton, linux-kernel, linux-ide, lkml
On Mon, 2006-03-20 at 21:35 -0500, Jeff Garzik wrote:
> Dave Jones wrote:
> > This wasn't occasional, this was every few seconds, making the box
> > pretty much unusable.
>
> The entire box or just the console?
Just the console.
--
Peter
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: Some sata_mv error messages
2006-03-21 1:02 ` Some sata_mv error messages Jeff Garzik
2006-03-21 2:25 ` Dave Jones
@ 2006-03-21 4:48 ` Mark Lord
2006-03-21 7:28 ` Sander
2006-03-21 19:33 ` Denis Leroy
2006-03-21 7:26 ` Sander
2 siblings, 2 replies; 97+ messages in thread
From: Mark Lord @ 2006-03-21 4:48 UTC (permalink / raw)
To: Jeff Garzik; +Cc: sander, Andrew Morton, linux-kernel, linux-ide, lkml
Jeff Garzik wrote:
>
> Without answering your specific question, just remember that sata_mv is
> considerly "highly experimental" right now, and still needs some
> workarounds for hardware errata.
>
> For now, the goal is a system that doesn't crash and doesn't corrupt
> data. If its occasionally slow or spits out a few errors, but otherwise
> still works, that's pretty darned good :)
I'm currently working with the original authors of sata_mv, and have taken
over maintenance of it for now. It should progress from "highly experimental"
to "production quality" over the next month or so.
The (mucho) updated driver I'm using here now is already much improved
in many ways. At some point, I'll break it out into patches for Jeff.
But there's one MAJOR bugfix patch that I'll release here shortly,
to go with the interrupt handler fix already posted.
Cheers
Mark
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: Some sata_mv error messages
2006-03-21 4:48 ` Mark Lord
@ 2006-03-21 7:28 ` Sander
2006-03-21 19:33 ` Denis Leroy
1 sibling, 0 replies; 97+ messages in thread
From: Sander @ 2006-03-21 7:28 UTC (permalink / raw)
To: Mark Lord
Cc: Jeff Garzik, sander, Andrew Morton, linux-kernel, linux-ide, lkml
Mark Lord wrote (ao):
> Jeff Garzik wrote:
> >For now, the goal is a system that doesn't crash and doesn't corrupt
> >data. If its occasionally slow or spits out a few errors, but
> >otherwise still works, that's pretty darned good :)
>
> I'm currently working with the original authors of sata_mv, and have
> taken over maintenance of it for now. It should progress from "highly
> experimental" to "production quality" over the next month or so.
Thanks a lot :-)
> The (mucho) updated driver I'm using here now is already much improved
> in many ways. At some point, I'll break it out into patches for Jeff.
>
> But there's one MAJOR bugfix patch that I'll release here shortly, to
> go with the interrupt handler fix already posted.
I've actually a patched kernel ready to test your patch. Only need to
boot the server, so will do that this afternoon and report back.
Sander
--
Humilis IT Services and Solutions
http://www.humilis.net
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: Some sata_mv error messages
2006-03-21 4:48 ` Mark Lord
2006-03-21 7:28 ` Sander
@ 2006-03-21 19:33 ` Denis Leroy
2006-03-21 19:37 ` Jeff Garzik
2006-03-21 19:42 ` Mark Lord
1 sibling, 2 replies; 97+ messages in thread
From: Denis Leroy @ 2006-03-21 19:33 UTC (permalink / raw)
To: Mark Lord
Cc: Jeff Garzik, sander, Andrew Morton, linux-kernel, linux-ide, lkml
Mark Lord wrote:
> Jeff Garzik wrote:
>>
>> Without answering your specific question, just remember that sata_mv
>> is considerly "highly experimental" right now, and still needs some
>> workarounds for hardware errata.
>>
>> For now, the goal is a system that doesn't crash and doesn't corrupt
>> data. If its occasionally slow or spits out a few errors, but
>> otherwise still works, that's pretty darned good :)
>
> I'm currently working with the original authors of sata_mv, and have taken
> over maintenance of it for now. It should progress from "highly
> experimental"
> to "production quality" over the next month or so.
>
> The (mucho) updated driver I'm using here now is already much improved
> in many ways. At some point, I'll break it out into patches for Jeff.
>
> But there's one MAJOR bugfix patch that I'll release here shortly,
> to go with the interrupt handler fix already posted.
This is great news. Is there any relationship between the development of
this driver and the one maintained by Marvell that's available from
their web site ? Their latest version (3.6.1) is released under the GPL,
and is a very solid driver based our experience with it over the past
few years, though it's targeted at older versions of the linux kernel
and needs some porting to 2.6.16 (and contains redundant stuff like its
own scsi-ata layer).
Denis Leroy
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: Some sata_mv error messages
2006-03-21 19:33 ` Denis Leroy
@ 2006-03-21 19:37 ` Jeff Garzik
2006-03-21 19:42 ` Mark Lord
1 sibling, 0 replies; 97+ messages in thread
From: Jeff Garzik @ 2006-03-21 19:37 UTC (permalink / raw)
To: Denis Leroy
Cc: Mark Lord, sander, Andrew Morton, linux-kernel, linux-ide, lkml
Denis Leroy wrote:
> This is great news. Is there any relationship between the development of
> this driver and the one maintained by Marvell that's available from
> their web site ? Their latest version (3.6.1) is released under the GPL,
> and is a very solid driver based our experience with it over the past
> few years, though it's targeted at older versions of the linux kernel
> and needs some porting to 2.6.16 (and contains redundant stuff like its
> own scsi-ata layer).
There is a we-use-that-driver-as-documentation-sometimes relationship.
It's quite un-Linux for a Linux driver, and others have reported that
sata_mv in its current state is more stable for them than the Marvell
behemoth GPL driver.
Jeff
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: Some sata_mv error messages
2006-03-21 19:33 ` Denis Leroy
2006-03-21 19:37 ` Jeff Garzik
@ 2006-03-21 19:42 ` Mark Lord
1 sibling, 0 replies; 97+ messages in thread
From: Mark Lord @ 2006-03-21 19:42 UTC (permalink / raw)
To: Denis Leroy
Cc: Jeff Garzik, sander, Andrew Morton, linux-kernel, linux-ide, lkml
Denis Leroy wrote:
>
> This is great news. Is there any relationship between the development of
> this driver and the one maintained by Marvell that's available from
> their web site ? Their latest version (3.6.1) is released under the GPL,
>
No relationship, other than that I plan to look through their driver
for more clues to how they handle certain errata and stuff.
My employer for this project has fairly detailed information
from Marvell as well.
Cheers
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: Some sata_mv error messages
2006-03-21 1:02 ` Some sata_mv error messages Jeff Garzik
2006-03-21 2:25 ` Dave Jones
2006-03-21 4:48 ` Mark Lord
@ 2006-03-21 7:26 ` Sander
2 siblings, 0 replies; 97+ messages in thread
From: Sander @ 2006-03-21 7:26 UTC (permalink / raw)
To: Jeff Garzik; +Cc: sander, Andrew Morton, linux-kernel, linux-ide, lkml
Jeff Garzik wrote (ao):
> Sander wrote:
> >While sata_mv in 2.6.16-rc6-mm2 seems stable (yah!) compared to
> >2.6.16-rc6 (no crashes, no data corruption), I still get these messages:
> >
> >[ 3962.139906] ata5: translated ATA stat/err 0xd0/00 to SCSI SK/ASC/ASCQ
> >0xb/47/00
> >[ 3962.139959] ata5: status=0xd0 { Busy }
> >I'm not entirely sure this is only happens on sata_mv (Marvell
> >MV88SX6081) as out of eight disks only one is connected to the onboard
> >sata_nv (nVidia) and the error doesn't happen very often. But I'll keep
> >an eye on it.
> >
> >Are these messages somehow dangerous or otherwise indicating a
> >potentional serious problem? A google search came up with a few links,
> >but none of them helped me understand the messages.
>
> Without answering your specific question, just remember that sata_mv is
> considerly "highly experimental" right now, and still needs some
> workarounds for hardware errata.
>
> For now, the goal is a system that doesn't crash and doesn't corrupt
> data. If its occasionally slow or spits out a few errors, but
> otherwise still works, that's pretty darned good :)
I fully agree!
I am aware that sata_mv is early beta, and kernel 2.6.16-rc6-mm2 is
actually the first kernel which lets me really use my Marvell
controller (users should give feedback on that right ;-). Kudos to you
all! (I don't know what happened between -rc6 and -rc6-mm2).
I know quite some people have their Marvell controller on a shelf
because they could not get it to work reliably. Now they can give it
another try..
Btw, without the hardware errata workarounds implemented yet, will it
eventually corrupt data for a fact? Are there any tests which will
trigger bugs, other than my simple dd-over-raid5?
Sander
--
Humilis IT Services and Solutions
http://www.humilis.net
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: 2.6.16-rc6-mm2: reiser4 BUG when unmounting fs
2006-03-18 12:40 2.6.16-rc6-mm2 Andrew Morton
` (10 preceding siblings ...)
2006-03-20 13:33 ` Some sata_mv error messages " Sander
@ 2006-03-21 20:27 ` Laurent Riffard
2006-03-21 20:38 ` Laurent Riffard
2006-03-21 22:54 ` 2.6.16-rc6-mm2: Why is CONFIG_MIGRATION available for everyone? Adrian Bunk
` (2 subsequent siblings)
14 siblings, 1 reply; 97+ messages in thread
From: Laurent Riffard @ 2006-03-21 20:27 UTC (permalink / raw)
To: Andrew Morton, Kernel development list, reiserfs-list
[-- Attachment #1: Type: text/plain, Size: 3340 bytes --]
Le 18.03.2006 13:40, Andrew Morton a écrit :
> ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc6/2.6.16-rc6-mm2/
Hello,
This BUG is 100% reproducible. Simply boot to runlevel 1 and then
unmount a reiser4 fs:
------------[ cut here ]------------
kernel BUG at fs/inode.c:1146!
invalid opcode: 0000 [#1]
last sysfs file: /firmware/edd/int13_dev81/legacy_max_head
Modules linked in: floppy ide_cd cdrom loop aes dm_crypt nls_iso8859_1 nls_cp850 vfat fat reiser4 reiserfs raid1 md_mod via_agp agpgart video joydev usbhid ohci1394 ieee1394 uhci_hcd usbcore dm_mirror dm_mod
CPU: 0
EIP: 0060:[<c015af27>] Not tainted VLI
EFLAGS: 00010246 (2.6.16-rc6-mm2 #130)
EIP is at iput+0x1c/0x6a
eax: c031efa0 ebx: c15fe2f4 ecx: de85e958 edx: de85e958
esi: c15fe260 edi: c1720c00 ebp: da2f2ea0 esp: da2f2e9c
ds: 007b es: 007b ss: 0068
Process umount (pid: 3002, threadinfo=da2f2000 task=dff43a70)
Stack: <0>de85e85c da2f2eb0 c014daa6 de85e85c de85e9a8 da2f2ec0 c015b5aa de85e85c
df79c200 da2f2ed0 c015bacc de85e85c df79c400 da2f2edc c015af72 c1720c00
da2f2ee8 e0cbc2a3 df79c200 da2f2efc e0cc2079 df79c200 df79c248 c1720df8
Call Trace:
[<c0103a36>] show_stack_log_lvl+0x8b/0x95
[<c0103b6e>] show_registers+0x12e/0x194
[<c0103e67>] die+0x14e/0x1db
[<c01040bf>] do_trap+0x7c/0x96
[<c010431e>] do_invalid_op+0x89/0x93
[<c01034db>] error_code+0x4f/0x54
[<c014daa6>] bd_forget+0x63/0x67
[<c015b5aa>] clear_inode+0xb1/0xd2
[<c015bacc>] generic_drop_inode+0x107/0x119
[<c015af72>] iput+0x67/0x6a
[<e0cbc2a3>] done_formatted_fake+0x99/0xa8 [reiser4]
[<e0cc2079>] reiser4_put_super+0xec/0x11b [reiser4]
[<c014c862>] generic_shutdown_super+0x80/0x11a
[<c014c91c>] kill_block_super+0x20/0x32
[<c014ca89>] deactivate_super+0x51/0x64
[<c015cc17>] mntput_no_expire+0x38/0x4b
[<c0151e39>] path_release_on_umount+0x15/0x18
[<c015d8d9>] sys_umount+0x1ca/0x1d4
[<c01029bb>] sysenter_past_esp+0x54/0x75
Code: 89 03 74 03 89 58 04 89 1a 89 53 04 5b c9 c3 55 85 c0 89 e5 53 89 c3 74 5d 83 bb 4c 01 00 00 20 8b 80 a8 00 00 00 8b 40 20 75 08 <0f> 0b 7a 04 ac c0 28 c0 85 c0 74 0b 8b 50 14 85 d2 74 04 89 d8
BUG: warning at kernel/exit.c:847/do_exit()
[<c0103c0a>] show_trace+0xd/0xf
[<c0103c21>] dump_stack+0x15/0x17
[<c011823d>] do_exit+0x3d/0x65b
[<c0103eec>] die+0x1d3/0x1db
[<c01040bf>] do_trap+0x7c/0x96
[<c010431e>] do_invalid_op+0x89/0x93
[<c01034db>] error_code+0x4f/0x54
[<c014daa6>] bd_forget+0x63/0x67
[<c015b5aa>] clear_inode+0xb1/0xd2
[<c015bacc>] generic_drop_inode+0x107/0x119
[<c015af72>] iput+0x67/0x6a
[<e0cbc2a3>] done_formatted_fake+0x99/0xa8 [reiser4]
[<e0cc2079>] reiser4_put_super+0xec/0x11b [reiser4]
[<c014c862>] generic_shutdown_super+0x80/0x11a
[<c014c91c>] kill_block_super+0x20/0x32
[<c014ca89>] deactivate_super+0x51/0x64
[<c015cc17>] mntput_no_expire+0x38/0x4b
[<c0151e39>] path_release_on_umount+0x15/0x18
[<c015d8d9>] sys_umount+0x1ca/0x1d4
[<c01029bb>] sysenter_past_esp+0x54/0x75
BUG: umount/3002, lock held at task exit time!
[df79c248] {alloc_super}
.. held by: umount: 3002 [dff43a70, 124]
... acquired at: generic_shutdown_super+0x56/0x11a
2.6.16-rc6-mm1 was OK with the same config file (I did "make oldconfig"
before compiling 2.6.16-rc6-mm2).
Attached files:
- full dmesg
- .config
~~
laurent
[-- Attachment #2: kernel.log --]
[-- Type: text/x-log, Size: 13117 bytes --]
Linux version 2.6.16-rc6-mm2 (laurent@antares.localdomain) (gcc version 4.0.3 (4.0.3-0.20060215.2mdk for Mandriva Linux release 2006.1)) #130 Mon Mar 20 01:28:15 CET 2006
BIOS-provided physical RAM map:
sanitize start
sanitize end
copy_e820_map() start: 0000000000000000 size: 000000000009fc00 end: 000000000009fc00 type: 1
copy_e820_map() type is E820_RAM
add_memory_region(0000000000000000, 000000000009fc00, 1)
copy_e820_map() start: 000000000009fc00 size: 0000000000000400 end: 00000000000a0000 type: 2
add_memory_region(000000000009fc00, 0000000000000400, 2)
copy_e820_map() start: 00000000000f0000 size: 0000000000010000 end: 0000000000100000 type: 2
add_memory_region(00000000000f0000, 0000000000010000, 2)
copy_e820_map() start: 0000000000100000 size: 000000001feec000 end: 000000001ffec000 type: 1
copy_e820_map() type is E820_RAM
add_memory_region(0000000000100000, 000000001feec000, 1)
copy_e820_map() start: 000000001ffec000 size: 0000000000003000 end: 000000001ffef000 type: 3
add_memory_region(000000001ffec000, 0000000000003000, 3)
copy_e820_map() start: 000000001ffef000 size: 0000000000010000 end: 000000001ffff000 type: 2
add_memory_region(000000001ffef000, 0000000000010000, 2)
copy_e820_map() start: 000000001ffff000 size: 0000000000001000 end: 0000000020000000 type: 4
add_memory_region(000000001ffff000, 0000000000001000, 4)
copy_e820_map() start: 00000000ffff0000 size: 0000000000010000 end: 0000000100000000 type: 2
add_memory_region(00000000ffff0000, 0000000000010000, 2)
BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 000000001ffec000 (usable)
BIOS-e820: 000000001ffec000 - 000000001ffef000 (ACPI data)
BIOS-e820: 000000001ffef000 - 000000001ffff000 (reserved)
BIOS-e820: 000000001ffff000 - 0000000020000000 (ACPI NVS)
BIOS-e820: 00000000ffff0000 - 0000000100000000 (reserved)
511MB LOWMEM available.
DMI 2.3 present.
ACPI: PM-Timer IO Port: 0xe408
Allocating PCI resources starting at 30000000 (gap: 20000000:dfff0000)
Built 1 zonelists
Kernel command line: root=/dev/vglinux1/lvroot video=vesafb:ywrap,mtrr splash=silent resume=/dev/hdb6 netconsole=@192.163.0.3/,@192.168.0.1/00:0E:9B:91:ED:72 init 1
netconsole: local port 6665
netconsole: local IP 192.163.0.3
netconsole: interface eth0
netconsole: remote port 6666
netconsole: remote IP 192.168.0.1
netconsole: remote ethernet address 00:0e:9b:91:ed:72
Local APIC disabled by BIOS -- you can enable it with "lapic"
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
CPU 0 irqstacks, hard=c0388000 soft=c0389000
PID hash table entries: 2048 (order: 11, 8192 bytes)
Detected 1410.463 MHz processor.
Using pmtmr for high-res timesource
Console: colour VGA+ 80x25
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Memory: 516112k/524208k available (1478k kernel code, 7532k reserved, 932k data, 156k init, 0k highmem)
Checking if this processor honours the WP bit even in supervisor mode... Ok.
Calibrating delay using timer specific routine.. 2823.56 BogoMIPS (lpj=5647128)
Mount-cache hash table entries: 512
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 256K (64 bytes/line)
Intel machine check architecture supported.
Intel machine check reporting enabled on CPU#0.
CPU: AMD Athlon(TM) XP 1600+ stepping 02
Checking 'hlt' instruction... OK.
SMP alternatives: switching to UP code
Freeing SMP alternatives: 0k freed
tbxface-0109 [02] load_tables : ACPI Tables successfully acquired
Parsing all Control Methods:
Table [DSDT](id 0005) - 356 Objects with 38 Devices 115 Methods 24 Regions
ACPI Namespace successfully loaded at root c03a6358
ACPI: setting ELCR to 0200 (from 0c20)
evxfevnt-0091 [03] enable : Transition to ACPI mode successful
checking if image is initramfs...it isn't (no cpio magic); looks like an initrd
Freeing initrd memory: 222k freed
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: PCI BIOS revision 2.10 entry at 0xf1180, last bus=1
ACPI: Subsystem revision 20060210
evgpeblk-0950 [06] ev_create_gpe_block : GPE 00 to 0F [_GPE] 2 regs on int 0x9
evgpeblk-1047 [05] ev_initialize_gpe_bloc: Found 4 Wake, Enabled 0 Runtime GPEs in this block
Completing Region/Field/Buffer/Package initialization:.......................................................
Initialized 23/24 Regions 2/2 Fields 19/19 Buffers 11/14 Packages (365 nodes)
Executing all Device _STA and_INI methods:.........................................
41 Devices found - executed 0 _STA, 0 _INI methods
ACPI: Interpreter enabled
ACPI: Using PIC for interrupt routing
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 9 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 9 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 *5 6 7 9 10 11 12 14 15)
ACPI: PCI Root Bridge [PCI0] (0000:00)
ACPI: Assume root bridge [\_SB_.PCI0] bus is 0
PCI quirk: region e200-e27f claimed by vt82c686 HW-mon
PCI quirk: region e800-e80f claimed by vt82c686 SMB
PCI: Using ACPI for IRQ routing
PCI: If a device doesn't work, try "pci=routeirq". If it helps, post a report
Setting up standard PCI resources
PCI: Bridge: 0000:00:01.0
IO window: disabled.
MEM window: c6000000-c7efffff
PREFETCH window: c7f00000-cfffffff
Simple Boot Flag at 0x3a set to 0x1
Initializing Cryptographic API
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
Applying VIA southbridge workaround.
PCI: Disabling Via external APIC routing
ACPI: Power Button (FF) [PWRF]
ACPI: Power Button (CM) [PWRB]
ACPI: CPU0 (power states: C1[C1] C2[C2])
ACPI: Processor [CPU0] (supports 16 throttling states)
RAMDISK driver initialized: 16 RAM disks of 4096K size 1024 blocksize
ne2k-pci.c:v1.03 9/22/2003 D. Becker/P. Gortmaker
http://www.scyld.com/network/ne2k-pci.html
ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 10
ACPI: PCI Interrupt 0000:00:0b.0[A] -> Link [LNKB] -> GSI 10 (level, low) -> IRQ 10
eth0: RealTek RTL-8029 found at 0xa400, IRQ 10, 00:40:95:46:6E:2D.
netconsole: device eth0 not up yet, forcing it
netconsole: carrier detect appears untrustworthy, waiting 4 seconds
netconsole: network logging started
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
VP_IDE: IDE controller at PCI slot 0000:00:04.1
ACPI (acpi_bus-0216): Device 'IDE0' is not power manageable [20060210]
VP_IDE: chipset revision 6
VP_IDE: not 100% native mode: will probe irqs later
VP_IDE: VIA vt82c686b (rev 40) IDE UDMA100 controller on pci0000:00:04.1
ide0: BM-DMA at 0xd800-0xd807, BIOS settings: hda:DMA, hdb:DMA
ide1: BM-DMA at 0xd808-0xd80f, BIOS settings: hdc:DMA, hdd:DMA
hda: ST340016A, ATA DISK drive
hdb: Maxtor 6Y080L0, ATA DISK drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
hdc: HL-DT-ST DVDRAM GSA-4165B, ATAPI CD/DVD-ROM drive
hdd: CD-950E/AKU, ATAPI CD/DVD-ROM drive
ide1 at 0x170-0x177,0x376 on irq 15
hda: max request size: 128KiB
hda: 78165360 sectors (40020 MB) w/2048KiB Cache, CHS=65535/16/63, UDMA(100)
hda: cache flushes not supported
hda: hda1 hda2 hda3 < hda5 hda6 hda7 hda8 hda9 hda10 > hda4
hdb: max request size: 128KiB
hdb: 160086528 sectors (81964 MB) w/2048KiB Cache, CHS=65535/16/63, UDMA(100)
hdb: cache flushes supported
hdb: hdb1 hdb2 < hdb5 hdb6 hdb7 hdb8 hdb9 >
serio: i8042 AUX port at 0x60,0x64 irq 12
serio: i8042 KBD port at 0x60,0x64 irq 1
mice: PS/2 mouse device common for all mice
NET: Registered protocol family 2
input: AT Translated Set 2 keyboard as /class/input/input0
IP route cache hash table entries: 4096 (order: 2, 16384 bytes)
TCP established hash table entries: 16384 (order: 4, 65536 bytes)
TCP bind hash table entries: 8192 (order: 3, 32768 bytes)
TCP: Hash tables configured (established 16384 bind 8192)
TCP reno registered
TCP bic registered
NET: Registered protocol family 1
Using IPI Shortcut mode
ACPI: wakeup devices: PWRB PCI0 UAR1 UAR2 USB0 USB1
ACPI: (supports S0 S1 S3 S4 S5)
BIOS EDD facility v0.16 2004-Jun-25, 2 devices found
RAMDISK: Compressed image found at block 0
VFS: Mounted root (ext2 filesystem).
device-mapper: 4.6.0-ioctl (2006-02-17) initialised: dm-devel@redhat.com
input: ImExPS/2 Generic Explorer Mouse as /class/input/input1
kjournald starting. Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
Freeing unused kernel memory: 156k freed
Write protecting the kernel read-only data: 653k
input: THRUSTMASTER FireStorm Dual Analog 2 as /class/input/input2
input: USB HID v1.10 Gamepad [THRUSTMASTER FireStorm Dual Analog 2] on usb-0000:00:04.2-2
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.6:USB HID core driver
------------[ cut here ]------------
kernel BUG at fs/inode.c:1146!
invalid opcode: 0000 [#1]
last sysfs file: /block/hdd/removable
Modules linked in: ide_cd cdrom loop aes dm_crypt nls_iso8859_1 nls_cp850 vfat fat reiser4 reiserfs raid1 md_mod via_agp agpgart video joydev usbhid ohci1394 ieee1394 uhci_hcd usbcore dm_mirror dm_mod
CPU: 0
EIP: 0060:[<c015af27>] Not tainted VLI
EFLAGS: 00010246 (2.6.16-rc6-mm2 #130)
EIP is at iput+0x1c/0x6a
eax: c031efa0 ebx: c15fe2f4 ecx: de851958 edx: de851958
esi: c15fe260 edi: df525c00 ebp: dede3e90 esp: dede3e8c
ds: 007b es: 007b ss: 0068
Process umount (pid: 1177, threadinfo=dede3000 task=dfe6aa50)
Stack: <0>de85185c dede3ea0 c014daa6 de85185c de8519a8 dede3eb0 c015b5aa de85185c
dfc31a00 dede3ec0 c015bacc de85185c df9d7800 dede3ecc c015af72 df525c00
dede3ed8 e0cbc2a3 dfc31a00 dede3eec e0cc2079 dfc31a00 dfc31a48 df525df8
Call Trace:
[<c0103a36>] show_stack_log_lvl+0x8b/0x95
[<c0103b6e>] show_registers+0x12e/0x194
[<c0103e67>] die+0x14e/0x1db
[<c01040bf>] do_trap+0x7c/0x96
[<c010431e>] do_invalid_op+0x89/0x93
[<c01034db>] error_code+0x4f/0x54
[<c014daa6>] bd_forget+0x63/0x67
[<c015b5aa>] clear_inode+0xb1/0xd2
[<c015bacc>] generic_drop_inode+0x107/0x119
[<c015af72>] iput+0x67/0x6a
[<e0cbc2a3>] done_formatted_fake+0x99/0xa8 [reiser4]
[<e0cc2079>] reiser4_put_super+0xec/0x11b [reiser4]
[<c014c862>] generic_shutdown_super+0x80/0x11a
[<c014c91c>] kill_block_super+0x20/0x32
[<c014ca89>] deactivate_super+0x51/0x64
[<c015cc17>] mntput_no_expire+0x38/0x4b
[<c0151e39>] path_release_on_umount+0x15/0x18
[<c015d8d9>] sys_umount+0x1ca/0x1d4
[<c015d8f0>] sys_oldumount+0xd/0xf
[<c01029bb>] sysenter_past_esp+0x54/0x75
Code: 89 03 74 03 89 58 04 89 1a 89 53 04 5b c9 c3 55 85 c0 89 e5 53 89 c3 74 5d 83 bb 4c 01 00 00 20 8b 80 a8 00 00 00 8b 40 20 75 08 <0f> 0b 7a 04 ac c0 28 c0 85 c0 74 0b 8b 50 14 85 d2 74 04 89 d8
BUG: warning at kernel/exit.c:847/do_exit()
[<c0103c0a>] show_trace+0xd/0xf
[<c0103c21>] dump_stack+0x15/0x17
[<c011823d>] do_exit+0x3d/0x65b
[<c0103eec>] die+0x1d3/0x1db
[<c01040bf>] do_trap+0x7c/0x96
[<c010431e>] do_invalid_op+0x89/0x93
[<c01034db>] error_code+0x4f/0x54
[<c014daa6>] bd_forget+0x63/0x67
[<c015b5aa>] clear_inode+0xb1/0xd2
[<c015bacc>] generic_drop_inode+0x107/0x119
[<c015af72>] iput+0x67/0x6a
[<e0cbc2a3>] done_formatted_fake+0x99/0xa8 [reiser4]
[<e0cc2079>] reiser4_put_super+0xec/0x11b [reiser4]
[<c014c862>] generic_shutdown_super+0x80/0x11a
[<c014c91c>] kill_block_super+0x20/0x32
[<c014ca89>] deactivate_super+0x51/0x64
[<c015cc17>] mntput_no_expire+0x38/0x4b
[<c0151e39>] path_release_on_umount+0x15/0x18
[<c015d8d9>] sys_umount+0x1ca/0x1d4
[<c015d8f0>] sys_oldumount+0xd/0xf
[<c01029bb>] sysenter_past_esp+0x54/0x75
BUG: umount/1177, lock held at task exit time!
[dfc31a48] {alloc_super}
.. held by: umount: 1177 [dfe6aa50, 118]
... acquired at: generic_shutdown_super+0x56/0x11a
SysRq : Emergency Sync
SysRq : Emergency Remount R/O
SysRq : Resetting
BUG: warning at drivers/pci/search.c:234/pci_get_subsys()
[<c0103c0a>] show_trace+0xd/0xf
[<c0103c21>] dump_stack+0x15/0x17
[<c01aa72d>] pci_get_subsys+0x3f/0xb4
[<c01aa7b0>] pci_get_device+0xe/0x10
[<c0110053>] mach_reboot_fixups+0x15/0x2e
[<c010e2fe>] machine_emergency_restart+0x26/0xcd
[<c011ff2d>] emergency_restart+0x8/0xa
[<c02025de>] sysrq_handle_reboot+0x9/0xb
[<c020277f>] __handle_sysrq+0x72/0xe0
[<c0202800>] handle_sysrq+0x13/0x16
[<c01fd9c8>] kbd_event+0x266/0x413
[<c0218f27>] input_event+0x3a7/0x427
[<c021aeb1>] atkbd_report_key+0x5e/0x7e
[<c021b325>] atkbd_interrupt+0x454/0x4f0
[<c0216169>] serio_interrupt+0x18/0x3f
[<c0216636>] i8042_interrupt+0x1c0/0x1d2
[<c012f3be>] handle_IRQ_event+0x26/0x51
[<c012f441>] __do_IRQ+0x58/0x9b
[<c0104c54>] do_IRQ+0x61/0x80
=======================
[<c01033ee>] common_interrupt+0x1a/0x20
[<c0101a20>] cpu_idle+0x42/0x5a
[<c010023e>] rest_init+0x1e/0x20
[<c035c655>] start_kernel+0x29e/0x2a0
[<c0100199>] 0xc0100199
No reboot fixup found for your hardware
[-- Attachment #3: config-2.6.16-rc6-mm2 --]
[-- Type: text/plain, Size: 41944 bytes --]
#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.16-rc6-mm2
# Mon Mar 20 00:33:05 2006
#
CONFIG_X86_32=y
CONFIG_SEMAPHORE_SLEEPERS=y
CONFIG_X86=y
CONFIG_MMU=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_GENERIC_HWEIGHT=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
CONFIG_DMI=y
#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
CONFIG_BROKEN_ON_SMP=y
CONFIG_INIT_ENV_ARG_LIMIT=32
#
# General setup
#
CONFIG_LOCALVERSION=""
CONFIG_LOCALVERSION_AUTO=y
CONFIG_SWAP=y
CONFIG_SWAP_PREFETCH=y
CONFIG_SYSVIPC=y
CONFIG_POSIX_MQUEUE=y
# CONFIG_BSD_PROCESS_ACCT is not set
CONFIG_SYSCTL=y
# CONFIG_AUDIT is not set
# CONFIG_IKCONFIG is not set
# CONFIG_RELAY is not set
CONFIG_INITRAMFS_SOURCE=""
CONFIG_UID16=y
CONFIG_VM86=y
CONFIG_CC_OPTIMIZE_FOR_SIZE=y
CONFIG_EMBEDDED=y
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_ALL is not set
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_SHMEM=y
CONFIG_SLAB=y
# CONFIG_TINY_SHMEM is not set
CONFIG_BASE_SMALL=0
# CONFIG_SLOB is not set
#
# Loadable module support
#
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODULE_FORCE_UNLOAD=y
# CONFIG_MODVERSIONS is not set
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_KMOD=y
#
# Block layer
#
# CONFIG_LBD is not set
# CONFIG_BLK_DEV_IO_TRACE is not set
# CONFIG_LSF is not set
#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
# CONFIG_DEFAULT_AS is not set
# CONFIG_DEFAULT_DEADLINE is not set
CONFIG_DEFAULT_CFQ=y
# CONFIG_DEFAULT_NOOP is not set
CONFIG_DEFAULT_IOSCHED="cfq"
#
# Processor type and features
#
CONFIG_X86_PC=y
# CONFIG_X86_ELAN is not set
# CONFIG_X86_VOYAGER is not set
# CONFIG_X86_NUMAQ is not set
# CONFIG_X86_SUMMIT is not set
# CONFIG_X86_BIGSMP is not set
# CONFIG_X86_VISWS is not set
# CONFIG_X86_GENERICARCH is not set
# CONFIG_X86_ES7000 is not set
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
# CONFIG_M686 is not set
# CONFIG_MPENTIUMII is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUMM is not set
# CONFIG_MPENTIUM4 is not set
# CONFIG_MK6 is not set
CONFIG_MK7=y
# CONFIG_MK8 is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MEFFICEON is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP2 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MGEODEGX1 is not set
# CONFIG_MGEODE_LX is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
# CONFIG_X86_GENERIC is not set
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
CONFIG_X86_CMPXCHG64=y
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_X86_USE_3DNOW=y
CONFIG_X86_TSC=y
CONFIG_HPET_TIMER=y
# CONFIG_SMP is not set
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set
CONFIG_X86_UP_APIC=y
CONFIG_X86_UP_IOAPIC=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_MCE=y
CONFIG_X86_MCE_NONFATAL=m
CONFIG_X86_MCE_P4THERMAL=y
# CONFIG_TOSHIBA is not set
# CONFIG_I8K is not set
CONFIG_X86_REBOOTFIXUPS=y
# CONFIG_MICROCODE is not set
CONFIG_X86_MSR=m
# CONFIG_X86_CPUID is not set
#
# Firmware Drivers
#
CONFIG_EDD=y
# CONFIG_DELL_RBU is not set
# CONFIG_DCDBAS is not set
CONFIG_NOHIGHMEM=y
# CONFIG_HIGHMEM4G is not set
# CONFIG_HIGHMEM64G is not set
CONFIG_VMSPLIT_3G=y
# CONFIG_VMSPLIT_3G_OPT is not set
# CONFIG_VMSPLIT_2G is not set
# CONFIG_VMSPLIT_1G is not set
CONFIG_PAGE_OFFSET=0xC0000000
CONFIG_ARCH_FLATMEM_ENABLE=y
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_FLATMEM_MANUAL=y
# CONFIG_DISCONTIGMEM_MANUAL is not set
# CONFIG_SPARSEMEM_MANUAL is not set
CONFIG_FLATMEM=y
CONFIG_FLAT_NODE_MEM_MAP=y
CONFIG_SPARSEMEM_STATIC=y
CONFIG_SPLIT_PTLOCK_CPUS=4
# CONFIG_MIGRATION is not set
# CONFIG_MATH_EMULATION is not set
CONFIG_MTRR=y
# CONFIG_EFI is not set
CONFIG_REGPARM=y
CONFIG_SECCOMP=y
# CONFIG_HZ_100 is not set
CONFIG_HZ_250=y
# CONFIG_HZ_1000 is not set
CONFIG_HZ=250
CONFIG_KEXEC=y
CONFIG_PHYSICAL_START=0x100000
CONFIG_DOUBLEFAULT=y
#
# Power management options (ACPI, APM)
#
CONFIG_PM=y
CONFIG_PM_LEGACY=y
# CONFIG_PM_DEBUG is not set
CONFIG_SOFTWARE_SUSPEND=y
CONFIG_PM_STD_PARTITION="/dev/hdb6"
#
# ACPI (Advanced Configuration and Power Interface) Support
#
CONFIG_ACPI=y
CONFIG_ACPI_SLEEP=y
CONFIG_ACPI_SLEEP_PROC_FS=y
# CONFIG_ACPI_SLEEP_PROC_SLEEP is not set
CONFIG_ACPI_AC=y
CONFIG_ACPI_BATTERY=y
CONFIG_ACPI_BUTTON=y
CONFIG_ACPI_VIDEO=m
CONFIG_ACPI_HOTKEY=m
CONFIG_ACPI_FAN=y
CONFIG_ACPI_PROCESSOR=y
CONFIG_ACPI_THERMAL=y
# CONFIG_ACPI_ASUS is not set
# CONFIG_ACPI_IBM is not set
# CONFIG_ACPI_TOSHIBA is not set
# CONFIG_ACPI_SONY is not set
CONFIG_ACPI_BLACKLIST_YEAR=0
CONFIG_ACPI_DEBUG=y
CONFIG_ACPI_EC=y
CONFIG_ACPI_POWER=y
CONFIG_ACPI_SYSTEM=y
CONFIG_X86_PM_TIMER=y
# CONFIG_ACPI_CONTAINER is not set
# CONFIG_ACPI_SCI_EMULATE is not set
#
# APM (Advanced Power Management) BIOS Support
#
# CONFIG_APM is not set
#
# CPU Frequency scaling
#
# CONFIG_CPU_FREQ is not set
#
# Bus options (PCI, PCMCIA, EISA, MCA, ISA)
#
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GOMMCONFIG is not set
# CONFIG_PCI_GODIRECT is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
# CONFIG_PCIEPORTBUS is not set
# CONFIG_PCI_MSI is not set
# CONFIG_PCI_DEBUG is not set
CONFIG_ISA_DMA_API=y
CONFIG_ISA=y
# CONFIG_EISA is not set
# CONFIG_MCA is not set
# CONFIG_SCx200 is not set
#
# PCCARD (PCMCIA/CardBus) support
#
# CONFIG_PCCARD is not set
#
# PCI Hotplug Support
#
# CONFIG_HOTPLUG_PCI is not set
#
# Executable file formats
#
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_AOUT=m
CONFIG_BINFMT_MISC=m
#
# Networking
#
CONFIG_NET=y
#
# Networking options
#
# CONFIG_NETDEBUG is not set
CONFIG_PACKET=m
# CONFIG_PACKET_MMAP is not set
CONFIG_UNIX=y
CONFIG_XFRM=y
CONFIG_XFRM_USER=m
# CONFIG_NET_KEY is not set
CONFIG_INET=y
# CONFIG_IP_MULTICAST is not set
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_ASK_IP_FIB_HASH=y
# CONFIG_IP_FIB_TRIE is not set
CONFIG_IP_FIB_HASH=y
# CONFIG_IP_MULTIPLE_TABLES is not set
# CONFIG_IP_ROUTE_MULTIPATH is not set
CONFIG_IP_ROUTE_VERBOSE=y
# CONFIG_IP_PNP is not set
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE is not set
# CONFIG_ARPD is not set
CONFIG_SYN_COOKIES=y
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
CONFIG_INET_IPCOMP=m
CONFIG_INET_TUNNEL=m
CONFIG_INET_DIAG=m
CONFIG_INET_TCP_DIAG=m
# CONFIG_TCP_CONG_ADVANCED is not set
CONFIG_TCP_CONG_BIC=y
#
# IP: Virtual Server Configuration
#
# CONFIG_IP_VS is not set
# CONFIG_IPV6 is not set
CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set
#
# Core Netfilter Configuration
#
# CONFIG_NETFILTER_NETLINK is not set
# CONFIG_NETFILTER_XTABLES is not set
#
# IP: Netfilter Configuration
#
CONFIG_IP_NF_CONNTRACK=m
# CONFIG_IP_NF_CT_ACCT is not set
# CONFIG_IP_NF_CONNTRACK_MARK is not set
# CONFIG_IP_NF_CONNTRACK_EVENTS is not set
# CONFIG_IP_NF_CT_PROTO_SCTP is not set
CONFIG_IP_NF_FTP=m
CONFIG_IP_NF_IRC=m
# CONFIG_IP_NF_NETBIOS_NS is not set
CONFIG_IP_NF_TFTP=m
CONFIG_IP_NF_AMANDA=m
# CONFIG_IP_NF_PPTP is not set
CONFIG_IP_NF_QUEUE=m
#
# DCCP Configuration (EXPERIMENTAL)
#
# CONFIG_IP_DCCP is not set
#
# SCTP Configuration (EXPERIMENTAL)
#
# CONFIG_IP_SCTP is not set
#
# TIPC Configuration (EXPERIMENTAL)
#
# CONFIG_TIPC is not set
# CONFIG_ATM is not set
# CONFIG_BRIDGE is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_NET_DIVERT is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
#
# QoS and/or fair queueing
#
CONFIG_NET_SCHED=y
CONFIG_NET_SCH_CLK_JIFFIES=y
# CONFIG_NET_SCH_CLK_GETTIMEOFDAY is not set
# CONFIG_NET_SCH_CLK_CPU is not set
#
# Queueing/Scheduling
#
CONFIG_NET_SCH_CBQ=m
CONFIG_NET_SCH_HTB=m
CONFIG_NET_SCH_HFSC=m
CONFIG_NET_SCH_PRIO=m
CONFIG_NET_SCH_RED=m
CONFIG_NET_SCH_SFQ=m
CONFIG_NET_SCH_TEQL=m
CONFIG_NET_SCH_TBF=m
CONFIG_NET_SCH_GRED=m
CONFIG_NET_SCH_DSMARK=m
# CONFIG_NET_SCH_NETEM is not set
CONFIG_NET_SCH_INGRESS=m
#
# Classification
#
CONFIG_NET_CLS=y
CONFIG_NET_CLS_BASIC=m
CONFIG_NET_CLS_TCINDEX=m
CONFIG_NET_CLS_ROUTE4=m
CONFIG_NET_CLS_ROUTE=y
CONFIG_NET_CLS_FW=m
CONFIG_NET_CLS_U32=m
CONFIG_CLS_U32_PERF=y
CONFIG_CLS_U32_MARK=y
CONFIG_NET_CLS_RSVP=m
# CONFIG_NET_CLS_RSVP6 is not set
# CONFIG_NET_EMATCH is not set
# CONFIG_NET_CLS_ACT is not set
CONFIG_NET_CLS_POLICE=y
# CONFIG_NET_CLS_IND is not set
CONFIG_NET_ESTIMATOR=y
#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_HAMRADIO is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
# CONFIG_IEEE80211 is not set
CONFIG_WIRELESS_EXT=y
#
# Device Drivers
#
#
# Generic Driver Options
#
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
CONFIG_FW_LOADER=m
# CONFIG_DEBUG_DRIVER is not set
#
# Connector - unified userspace <-> kernelspace linker
#
# CONFIG_CONNECTOR is not set
#
# Memory Technology Devices (MTD)
#
# CONFIG_MTD is not set
#
# Parallel port support
#
CONFIG_PARPORT=m
CONFIG_PARPORT_PC=m
# CONFIG_PARPORT_SERIAL is not set
# CONFIG_PARPORT_PC_FIFO is not set
# CONFIG_PARPORT_PC_SUPERIO is not set
# CONFIG_PARPORT_GSC is not set
CONFIG_PARPORT_1284=y
#
# Plug and Play support
#
# CONFIG_PNP is not set
#
# Block devices
#
CONFIG_BLK_DEV_FD=m
# CONFIG_BLK_DEV_XD is not set
# CONFIG_PARIDE is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
# CONFIG_BLK_DEV_COW_COMMON is not set
CONFIG_BLK_DEV_LOOP=m
# CONFIG_BLK_DEV_CRYPTOLOOP is not set
# CONFIG_BLK_DEV_NBD is not set
# CONFIG_BLK_DEV_SX8 is not set
CONFIG_BLK_DEV_UB=m
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_COUNT=16
CONFIG_BLK_DEV_RAM_SIZE=4096
CONFIG_BLK_DEV_INITRD=y
CONFIG_CDROM_PKTCDVD=m
CONFIG_CDROM_PKTCDVD_BUFFERS=8
# CONFIG_CDROM_PKTCDVD_WCACHE is not set
# CONFIG_ATA_OVER_ETH is not set
#
# ATA/ATAPI/MFM/RLL support
#
CONFIG_IDE=y
CONFIG_BLK_DEV_IDE=y
#
# Please see Documentation/ide.txt for help/info on IDE drives
#
# CONFIG_BLK_DEV_IDE_SATA is not set
# CONFIG_BLK_DEV_HD_IDE is not set
CONFIG_BLK_DEV_IDEDISK=y
# CONFIG_IDEDISK_MULTI_MODE is not set
CONFIG_BLK_DEV_IDECD=m
# CONFIG_BLK_DEV_IDETAPE is not set
# CONFIG_BLK_DEV_IDEFLOPPY is not set
CONFIG_BLK_DEV_IDESCSI=m
# CONFIG_IDE_TASK_IOCTL is not set
#
# IDE chipset support/bugfixes
#
CONFIG_IDE_GENERIC=m
# CONFIG_BLK_DEV_CMD640 is not set
CONFIG_BLK_DEV_IDEPCI=y
CONFIG_IDEPCI_SHARE_IRQ=y
# CONFIG_BLK_DEV_OFFBOARD is not set
# CONFIG_BLK_DEV_GENERIC is not set
# CONFIG_BLK_DEV_OPTI621 is not set
# CONFIG_BLK_DEV_RZ1000 is not set
CONFIG_BLK_DEV_IDEDMA_PCI=y
# CONFIG_BLK_DEV_IDEDMA_FORCED is not set
CONFIG_IDEDMA_PCI_AUTO=y
# CONFIG_IDEDMA_ONLYDISK is not set
# CONFIG_BLK_DEV_AEC62XX is not set
# CONFIG_BLK_DEV_ALI15X3 is not set
# CONFIG_BLK_DEV_AMD74XX is not set
# CONFIG_BLK_DEV_ATIIXP is not set
# CONFIG_BLK_DEV_CMD64X is not set
# CONFIG_BLK_DEV_TRIFLEX is not set
# CONFIG_BLK_DEV_CY82C693 is not set
# CONFIG_BLK_DEV_CS5520 is not set
# CONFIG_BLK_DEV_CS5530 is not set
# CONFIG_BLK_DEV_CS5535 is not set
# CONFIG_BLK_DEV_HPT34X is not set
# CONFIG_BLK_DEV_HPT366 is not set
# CONFIG_BLK_DEV_SC1200 is not set
# CONFIG_BLK_DEV_PIIX is not set
# CONFIG_BLK_DEV_IT821X is not set
# CONFIG_BLK_DEV_NS87415 is not set
# CONFIG_BLK_DEV_PDC202XX_OLD is not set
# CONFIG_BLK_DEV_PDC202XX_NEW is not set
# CONFIG_BLK_DEV_SVWKS is not set
# CONFIG_BLK_DEV_SIIMAGE is not set
# CONFIG_BLK_DEV_SIS5513 is not set
# CONFIG_BLK_DEV_SLC90E66 is not set
# CONFIG_BLK_DEV_TRM290 is not set
CONFIG_BLK_DEV_VIA82CXXX=y
# CONFIG_IDE_ARM is not set
# CONFIG_IDE_CHIPSETS is not set
CONFIG_BLK_DEV_IDEDMA=y
# CONFIG_IDEDMA_IVB is not set
CONFIG_IDEDMA_AUTO=y
# CONFIG_BLK_DEV_HD is not set
#
# SCSI device support
#
# CONFIG_RAID_ATTRS is not set
CONFIG_SCSI=m
# CONFIG_SCSI_TGT is not set
CONFIG_SCSI_PROC_FS=y
#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=m
CONFIG_CHR_DEV_ST=m
# CONFIG_CHR_DEV_OSST is not set
CONFIG_BLK_DEV_SR=m
CONFIG_BLK_DEV_SR_VENDOR=y
CONFIG_CHR_DEV_SG=m
# CONFIG_CHR_DEV_SCH is not set
#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs
#
CONFIG_SCSI_MULTI_LUN=y
CONFIG_SCSI_CONSTANTS=y
# CONFIG_SCSI_LOGGING is not set
#
# SCSI Transports
#
CONFIG_SCSI_SPI_ATTRS=m
CONFIG_SCSI_FC_ATTRS=m
# CONFIG_SCSI_ISCSI_ATTRS is not set
# CONFIG_SCSI_SAS_ATTRS is not set
# CONFIG_SAS_CLASS is not set
#
# SCSI low-level drivers
#
# CONFIG_ISCSI_TCP is not set
# CONFIG_BLK_DEV_3W_XXXX_RAID is not set
# CONFIG_SCSI_3W_9XXX is not set
# CONFIG_SCSI_7000FASST is not set
# CONFIG_SCSI_ACARD is not set
# CONFIG_SCSI_AHA152X is not set
# CONFIG_SCSI_AHA1542 is not set
# CONFIG_SCSI_AACRAID is not set
# CONFIG_SCSI_AIC7XXX is not set
# CONFIG_SCSI_AIC7XXX_OLD is not set
# CONFIG_SCSI_AIC79XX is not set
# CONFIG_SCSI_DPT_I2O is not set
# CONFIG_SCSI_IN2000 is not set
# CONFIG_SCSI_ARCMSR is not set
# CONFIG_MEGARAID_NEWGEN is not set
# CONFIG_MEGARAID_LEGACY is not set
# CONFIG_MEGARAID_SAS is not set
# CONFIG_SCSI_SATA is not set
# CONFIG_SCSI_BUSLOGIC is not set
# CONFIG_SCSI_DMX3191D is not set
# CONFIG_SCSI_DTC3280 is not set
# CONFIG_SCSI_EATA is not set
# CONFIG_SCSI_FUTURE_DOMAIN is not set
# CONFIG_SCSI_GDTH is not set
# CONFIG_SCSI_GENERIC_NCR5380 is not set
# CONFIG_SCSI_GENERIC_NCR5380_MMIO is not set
# CONFIG_SCSI_IPS is not set
# CONFIG_SCSI_INITIO is not set
# CONFIG_SCSI_INIA100 is not set
# CONFIG_SCSI_PPA is not set
# CONFIG_SCSI_IMM is not set
# CONFIG_SCSI_NCR53C406A is not set
# CONFIG_SCSI_SYM53C8XX_2 is not set
# CONFIG_SCSI_IPR is not set
# CONFIG_SCSI_PAS16 is not set
# CONFIG_SCSI_PSI240I is not set
# CONFIG_SCSI_QLOGIC_FAS is not set
# CONFIG_SCSI_QLOGIC_FC is not set
# CONFIG_SCSI_QLOGIC_1280 is not set
# CONFIG_SCSI_QLA_FC is not set
# CONFIG_SCSI_LPFC is not set
# CONFIG_SCSI_SYM53C416 is not set
# CONFIG_SCSI_DC395x is not set
# CONFIG_SCSI_DC390T is not set
# CONFIG_SCSI_T128 is not set
# CONFIG_SCSI_U14_34F is not set
# CONFIG_SCSI_ULTRASTOR is not set
# CONFIG_SCSI_NSP32 is not set
# CONFIG_SCSI_DEBUG is not set
#
# Old CD-ROM drivers (not SCSI, not IDE)
#
# CONFIG_CD_NO_IDESCSI is not set
#
# Multi-device support (RAID and LVM)
#
CONFIG_MD=y
CONFIG_BLK_DEV_MD=m
CONFIG_MD_LINEAR=m
CONFIG_MD_RAID0=m
CONFIG_MD_RAID1=m
# CONFIG_MD_RAID10 is not set
# CONFIG_MD_RAID5 is not set
# CONFIG_MD_RAID6 is not set
# CONFIG_MD_MULTIPATH is not set
# CONFIG_MD_FAULTY is not set
CONFIG_BLK_DEV_DM=m
CONFIG_DM_CRYPT=m
# CONFIG_DM_SNAPSHOT is not set
CONFIG_DM_MIRROR=m
# CONFIG_DM_ZERO is not set
# CONFIG_DM_MULTIPATH is not set
#
# Fusion MPT device support
#
# CONFIG_FUSION is not set
# CONFIG_FUSION_SPI is not set
# CONFIG_FUSION_FC is not set
# CONFIG_FUSION_SAS is not set
#
# IEEE 1394 (FireWire) support
#
CONFIG_IEEE1394=m
#
# Subsystem Options
#
# CONFIG_IEEE1394_VERBOSEDEBUG is not set
CONFIG_IEEE1394_OUI_DB=y
# CONFIG_IEEE1394_EXTRA_CONFIG_ROMS is not set
# CONFIG_IEEE1394_EXPORT_FULL_API is not set
#
# Device Drivers
#
# CONFIG_IEEE1394_PCILYNX is not set
CONFIG_IEEE1394_OHCI1394=m
#
# Protocol Drivers
#
CONFIG_IEEE1394_VIDEO1394=m
# CONFIG_IEEE1394_SBP2 is not set
# CONFIG_IEEE1394_ETH1394 is not set
CONFIG_IEEE1394_DV1394=m
CONFIG_IEEE1394_RAWIO=m
#
# I2O device support
#
# CONFIG_I2O is not set
#
# Network device support
#
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m
# CONFIG_BONDING is not set
# CONFIG_EQUALIZER is not set
CONFIG_TUN=m
#
# ARCnet devices
#
# CONFIG_ARCNET is not set
#
# PHY device support
#
# CONFIG_PHYLIB is not set
#
# Ethernet (10 or 100Mbit)
#
CONFIG_NET_ETHERNET=y
CONFIG_MII=m
# CONFIG_HAPPYMEAL is not set
# CONFIG_SUNGEM is not set
# CONFIG_CASSINI is not set
# CONFIG_NET_VENDOR_3COM is not set
# CONFIG_LANCE is not set
# CONFIG_NET_VENDOR_SMC is not set
# CONFIG_NET_VENDOR_RACAL is not set
#
# Tulip family network device support
#
# CONFIG_NET_TULIP is not set
# CONFIG_AT1700 is not set
# CONFIG_DEPCA is not set
# CONFIG_HP100 is not set
# CONFIG_NET_ISA is not set
CONFIG_NET_PCI=y
# CONFIG_PCNET32 is not set
# CONFIG_AMD8111_ETH is not set
# CONFIG_ADAPTEC_STARFIRE is not set
# CONFIG_AC3200 is not set
# CONFIG_APRICOT is not set
# CONFIG_B44 is not set
# CONFIG_FORCEDETH is not set
# CONFIG_CS89x0 is not set
# CONFIG_DGRS is not set
# CONFIG_EEPRO100 is not set
# CONFIG_E100 is not set
# CONFIG_FEALNX is not set
# CONFIG_NATSEMI is not set
CONFIG_NE2K_PCI=y
# CONFIG_8139CP is not set
# CONFIG_8139TOO is not set
# CONFIG_SIS900 is not set
# CONFIG_EPIC100 is not set
# CONFIG_SUNDANCE is not set
# CONFIG_TLAN is not set
# CONFIG_VIA_RHINE is not set
# CONFIG_NET_POCKET is not set
#
# Ethernet (1000 Mbit)
#
# CONFIG_ACENIC is not set
# CONFIG_DL2K is not set
# CONFIG_E1000 is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
# CONFIG_R8169 is not set
# CONFIG_SIS190 is not set
# CONFIG_SKGE is not set
# CONFIG_SKY2 is not set
# CONFIG_SK98LIN is not set
# CONFIG_VIA_VELOCITY is not set
# CONFIG_TIGON3 is not set
# CONFIG_BNX2 is not set
#
# Ethernet (10000 Mbit)
#
# CONFIG_CHELSIO_T1 is not set
# CONFIG_IXGB is not set
# CONFIG_S2IO is not set
#
# Token Ring devices
#
# CONFIG_TR is not set
#
# Wireless LAN (non-hamradio)
#
CONFIG_NET_RADIO=y
#
# Obsolete Wireless cards support (pre-802.11)
#
# CONFIG_STRIP is not set
# CONFIG_ARLAN is not set
# CONFIG_WAVELAN is not set
#
# Wireless 802.11b ISA/PCI cards support
#
# CONFIG_IPW2100 is not set
# CONFIG_IPW2200 is not set
# CONFIG_AIRO is not set
# CONFIG_HERMES is not set
# CONFIG_ATMEL is not set
#
# Prism GT/Duette 802.11(a/b/g) PCI/Cardbus support
#
CONFIG_PRISM54=m
# CONFIG_HOSTAP is not set
# CONFIG_ACX is not set
CONFIG_NET_WIRELESS=y
#
# Wan interfaces
#
# CONFIG_WAN is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_PLIP is not set
CONFIG_PPP=m
# CONFIG_PPP_MULTILINK is not set
CONFIG_PPP_FILTER=y
CONFIG_PPP_ASYNC=m
CONFIG_PPP_SYNC_TTY=m
CONFIG_PPP_DEFLATE=m
CONFIG_PPP_BSDCOMP=m
# CONFIG_PPP_MPPE is not set
CONFIG_PPPOE=m
# CONFIG_SLIP is not set
# CONFIG_NET_FC is not set
# CONFIG_SHAPER is not set
CONFIG_NETCONSOLE=y
CONFIG_NETPOLL=y
# CONFIG_NETPOLL_RX is not set
# CONFIG_NETPOLL_TRAP is not set
CONFIG_NET_POLL_CONTROLLER=y
#
# ISDN subsystem
#
# CONFIG_ISDN is not set
#
# Telephony Support
#
# CONFIG_PHONE is not set
#
# Input device support
#
CONFIG_INPUT=y
#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
CONFIG_INPUT_JOYDEV=m
# CONFIG_INPUT_TSDEV is not set
# CONFIG_INPUT_EVDEV is not set
# CONFIG_INPUT_EVBUG is not set
#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not set
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
# CONFIG_MOUSE_SERIAL is not set
# CONFIG_MOUSE_INPORT is not set
# CONFIG_MOUSE_LOGIBM is not set
# CONFIG_MOUSE_PC110PAD is not set
# CONFIG_MOUSE_VSXXXAA is not set
CONFIG_INPUT_JOYSTICK=y
CONFIG_JOYSTICK_ANALOG=m
# CONFIG_JOYSTICK_A3D is not set
# CONFIG_JOYSTICK_ADI is not set
# CONFIG_JOYSTICK_COBRA is not set
# CONFIG_JOYSTICK_GF2K is not set
# CONFIG_JOYSTICK_GRIP is not set
# CONFIG_JOYSTICK_GRIP_MP is not set
# CONFIG_JOYSTICK_GUILLEMOT is not set
# CONFIG_JOYSTICK_INTERACT is not set
# CONFIG_JOYSTICK_SIDEWINDER is not set
# CONFIG_JOYSTICK_TMDC is not set
# CONFIG_JOYSTICK_IFORCE is not set
# CONFIG_JOYSTICK_WARRIOR is not set
# CONFIG_JOYSTICK_MAGELLAN is not set
# CONFIG_JOYSTICK_SPACEORB is not set
# CONFIG_JOYSTICK_SPACEBALL is not set
# CONFIG_JOYSTICK_STINGER is not set
# CONFIG_JOYSTICK_TWIDJOY is not set
# CONFIG_JOYSTICK_DB9 is not set
# CONFIG_JOYSTICK_GAMECON is not set
# CONFIG_JOYSTICK_TURBOGRAFX is not set
# CONFIG_JOYSTICK_JOYDUMP is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
CONFIG_INPUT_MISC=y
CONFIG_INPUT_PCSPKR=m
# CONFIG_INPUT_WISTRON_BTNS is not set
# CONFIG_INPUT_UINPUT is not set
#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
# CONFIG_SERIO_SERPORT is not set
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PARKBD is not set
# CONFIG_SERIO_PCIPS2 is not set
CONFIG_SERIO_LIBPS2=y
# CONFIG_SERIO_RAW is not set
CONFIG_GAMEPORT=m
# CONFIG_GAMEPORT_NS558 is not set
# CONFIG_GAMEPORT_L4 is not set
# CONFIG_GAMEPORT_EMU10K1 is not set
# CONFIG_GAMEPORT_FM801 is not set
#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_SERIAL_NONSTANDARD is not set
#
# Serial drivers
#
CONFIG_SERIAL_8250=m
CONFIG_SERIAL_8250_NR_UARTS=4
CONFIG_SERIAL_8250_RUNTIME_UARTS=4
# CONFIG_SERIAL_8250_EXTENDED is not set
#
# Non-8250 serial port support
#
CONFIG_SERIAL_CORE=m
# CONFIG_SERIAL_JSM is not set
CONFIG_UNIX98_PTYS=y
# CONFIG_LEGACY_PTYS is not set
CONFIG_PRINTER=m
# CONFIG_LP_CONSOLE is not set
# CONFIG_PPDEV is not set
# CONFIG_TIPAR is not set
#
# IPMI
#
# CONFIG_IPMI_HANDLER is not set
#
# Watchdog Cards
#
# CONFIG_WATCHDOG is not set
# CONFIG_HW_RANDOM is not set
# CONFIG_NVRAM is not set
CONFIG_RTC=m
# CONFIG_GEN_RTC is not set
# CONFIG_DTLK is not set
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
# CONFIG_SONYPI is not set
#
# Ftape, the floppy tape device driver
#
# CONFIG_FTAPE is not set
CONFIG_AGP=m
# CONFIG_AGP_ALI is not set
# CONFIG_AGP_ATI is not set
# CONFIG_AGP_AMD is not set
# CONFIG_AGP_AMD64 is not set
# CONFIG_AGP_INTEL is not set
# CONFIG_AGP_NVIDIA is not set
# CONFIG_AGP_SIS is not set
# CONFIG_AGP_SWORKS is not set
CONFIG_AGP_VIA=m
# CONFIG_AGP_EFFICEON is not set
CONFIG_DRM=m
# CONFIG_DRM_TDFX is not set
# CONFIG_DRM_R128 is not set
# CONFIG_DRM_RADEON is not set
# CONFIG_DRM_MGA is not set
# CONFIG_DRM_SIS is not set
# CONFIG_DRM_VIA is not set
# CONFIG_DRM_SAVAGE is not set
# CONFIG_MWAVE is not set
# CONFIG_CS5535_GPIO is not set
# CONFIG_RAW_DRIVER is not set
# CONFIG_HPET is not set
# CONFIG_HANGCHECK_TIMER is not set
#
# TPM devices
#
# CONFIG_TCG_TPM is not set
# CONFIG_TELCLOCK is not set
#
# I2C support
#
CONFIG_I2C=y
CONFIG_I2C_CHARDEV=m
#
# I2C Algorithms
#
CONFIG_I2C_ALGOBIT=m
CONFIG_I2C_ALGOPCF=m
CONFIG_I2C_ALGOPCA=m
#
# I2C Hardware Bus support
#
# CONFIG_I2C_ALI1535 is not set
# CONFIG_I2C_ALI1563 is not set
# CONFIG_I2C_ALI15X3 is not set
# CONFIG_I2C_AMD756 is not set
# CONFIG_I2C_AMD8111 is not set
# CONFIG_I2C_ELEKTOR is not set
# CONFIG_I2C_I801 is not set
# CONFIG_I2C_I810 is not set
# CONFIG_I2C_PIIX4 is not set
CONFIG_I2C_ISA=m
# CONFIG_I2C_NFORCE2 is not set
# CONFIG_I2C_PARPORT is not set
# CONFIG_I2C_PARPORT_LIGHT is not set
# CONFIG_I2C_PROSAVAGE is not set
# CONFIG_I2C_SAVAGE4 is not set
# CONFIG_SCx200_ACB is not set
# CONFIG_I2C_SIS5595 is not set
# CONFIG_I2C_SIS630 is not set
# CONFIG_I2C_SIS96X is not set
# CONFIG_I2C_STUB is not set
CONFIG_I2C_VIA=m
CONFIG_I2C_VIAPRO=m
# CONFIG_I2C_VOODOO3 is not set
# CONFIG_I2C_PCA_ISA is not set
#
# Miscellaneous I2C Chip support
#
# CONFIG_SENSORS_DS1337 is not set
# CONFIG_SENSORS_DS1374 is not set
# CONFIG_SENSORS_EEPROM is not set
# CONFIG_SENSORS_PCF8574 is not set
# CONFIG_SENSORS_PCA9539 is not set
# CONFIG_SENSORS_PCF8591 is not set
# CONFIG_SENSORS_MAX6875 is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_I2C_DEBUG_CHIP is not set
#
# SPI support
#
# CONFIG_SPI is not set
# CONFIG_SPI_MASTER is not set
#
# Dallas's 1-wire bus
#
# CONFIG_W1 is not set
#
# Hardware Monitoring support
#
CONFIG_HWMON=y
CONFIG_HWMON_VID=m
# CONFIG_SENSORS_ADM1021 is not set
# CONFIG_SENSORS_ADM1025 is not set
# CONFIG_SENSORS_ADM1026 is not set
# CONFIG_SENSORS_ADM1031 is not set
# CONFIG_SENSORS_ADM9240 is not set
# CONFIG_SENSORS_ASB100 is not set
# CONFIG_SENSORS_ATXP1 is not set
# CONFIG_SENSORS_DS1621 is not set
# CONFIG_SENSORS_F71805F is not set
# CONFIG_SENSORS_FSCHER is not set
# CONFIG_SENSORS_FSCPOS is not set
# CONFIG_SENSORS_GL518SM is not set
# CONFIG_SENSORS_GL520SM is not set
# CONFIG_SENSORS_IT87 is not set
# CONFIG_SENSORS_LM63 is not set
# CONFIG_SENSORS_LM75 is not set
# CONFIG_SENSORS_LM77 is not set
# CONFIG_SENSORS_LM78 is not set
CONFIG_SENSORS_LM80=m
# CONFIG_SENSORS_LM83 is not set
# CONFIG_SENSORS_LM85 is not set
# CONFIG_SENSORS_LM87 is not set
# CONFIG_SENSORS_LM90 is not set
# CONFIG_SENSORS_LM92 is not set
# CONFIG_SENSORS_MAX1619 is not set
# CONFIG_SENSORS_PC87360 is not set
# CONFIG_SENSORS_SIS5595 is not set
# CONFIG_SENSORS_SMSC47M1 is not set
# CONFIG_SENSORS_SMSC47B397 is not set
CONFIG_SENSORS_VIA686A=m
# CONFIG_SENSORS_VT8231 is not set
CONFIG_SENSORS_W83781D=m
# CONFIG_SENSORS_W83792D is not set
# CONFIG_SENSORS_W83L785TS is not set
# CONFIG_SENSORS_W83627HF is not set
# CONFIG_SENSORS_W83627EHF is not set
# CONFIG_SENSORS_HDAPS is not set
# CONFIG_HWMON_DEBUG_CHIP is not set
#
# Misc devices
#
# CONFIG_IBM_ASM is not set
#
# Multimedia devices
#
CONFIG_VIDEO_DEV=m
#
# Video For Linux
#
#
# Video Adapters
#
# CONFIG_VIDEO_ADV_DEBUG is not set
# CONFIG_VIDEO_BT848 is not set
# CONFIG_VIDEO_PMS is not set
# CONFIG_VIDEO_BWQCAM is not set
# CONFIG_VIDEO_CQCAM is not set
# CONFIG_VIDEO_W9966 is not set
# CONFIG_VIDEO_CPIA is not set
# CONFIG_VIDEO_CPIA2 is not set
# CONFIG_VIDEO_SAA5246A is not set
# CONFIG_VIDEO_SAA5249 is not set
# CONFIG_TUNER_3036 is not set
# CONFIG_VIDEO_STRADIS is not set
# CONFIG_VIDEO_ZORAN is not set
# CONFIG_VIDEO_SAA7134 is not set
# CONFIG_VIDEO_MXB is not set
# CONFIG_VIDEO_DPC is not set
# CONFIG_VIDEO_HEXIUM_ORION is not set
# CONFIG_VIDEO_HEXIUM_GEMINI is not set
# CONFIG_VIDEO_CX88 is not set
# CONFIG_VIDEO_EM28XX is not set
# CONFIG_VIDEO_OVCAMCHIP is not set
# CONFIG_VIDEO_MSP3400 is not set
# CONFIG_VIDEO_CS53L32A is not set
# CONFIG_VIDEO_WM8775 is not set
# CONFIG_VIDEO_CX25840 is not set
# CONFIG_VIDEO_SAA711X is not set
# CONFIG_VIDEO_SAA7127 is not set
#
# Radio Adapters
#
# CONFIG_RADIO_CADET is not set
# CONFIG_RADIO_RTRACK is not set
# CONFIG_RADIO_RTRACK2 is not set
# CONFIG_RADIO_AZTECH is not set
# CONFIG_RADIO_GEMTEK is not set
# CONFIG_RADIO_GEMTEK_PCI is not set
# CONFIG_RADIO_MAXIRADIO is not set
# CONFIG_RADIO_MAESTRO is not set
# CONFIG_RADIO_SF16FMI is not set
# CONFIG_RADIO_SF16FMR2 is not set
# CONFIG_RADIO_TERRATEC is not set
# CONFIG_RADIO_TRUST is not set
# CONFIG_RADIO_TYPHOON is not set
# CONFIG_RADIO_ZOLTRIX is not set
#
# Digital Video Broadcasting Devices
#
# CONFIG_DVB is not set
#
# Graphics support
#
CONFIG_FB=y
CONFIG_FB_CFB_FILLRECT=y
CONFIG_FB_CFB_COPYAREA=y
CONFIG_FB_CFB_IMAGEBLIT=y
# CONFIG_FB_MACMODES is not set
CONFIG_FB_FIRMWARE_EDID=y
CONFIG_FB_MODE_HELPERS=y
# CONFIG_FB_TILEBLITTING is not set
# CONFIG_FB_CIRRUS is not set
# CONFIG_FB_PM2 is not set
# CONFIG_FB_CYBER2000 is not set
# CONFIG_FB_ARC is not set
# CONFIG_FB_ASILIANT is not set
# CONFIG_FB_IMSTT is not set
# CONFIG_FB_VGA16 is not set
CONFIG_FB_VESA=y
CONFIG_VIDEO_SELECT=y
# CONFIG_FB_HGA is not set
# CONFIG_FB_S1D13XXX is not set
CONFIG_FB_NVIDIA=m
CONFIG_FB_NVIDIA_I2C=y
CONFIG_FB_RIVA=m
CONFIG_FB_RIVA_I2C=y
# CONFIG_FB_RIVA_DEBUG is not set
# CONFIG_FB_I810 is not set
# CONFIG_FB_INTEL is not set
# CONFIG_FB_MATROX is not set
# CONFIG_FB_RADEON_OLD is not set
# CONFIG_FB_RADEON is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_ATY is not set
# CONFIG_FB_SAVAGE is not set
# CONFIG_FB_SIS is not set
# CONFIG_FB_NEOMAGIC is not set
# CONFIG_FB_KYRO is not set
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
# CONFIG_FB_CYBLA is not set
# CONFIG_FB_TRIDENT is not set
# CONFIG_FB_GEODE is not set
# CONFIG_FB_VIRTUAL is not set
#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
# CONFIG_VGACON_SOFT_SCROLLBACK is not set
# CONFIG_MDA_CONSOLE is not set
CONFIG_DUMMY_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE=y
# CONFIG_FRAMEBUFFER_CONSOLE_ROTATION is not set
# CONFIG_FONTS is not set
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y
#
# Logo configuration
#
CONFIG_LOGO=y
CONFIG_LOGO_LINUX_MONO=y
CONFIG_LOGO_LINUX_VGA16=y
CONFIG_LOGO_LINUX_CLUT224=y
# CONFIG_BACKLIGHT_LCD_SUPPORT is not set
#
# Sound
#
CONFIG_SOUND=m
#
# Advanced Linux Sound Architecture
#
CONFIG_SND=m
CONFIG_SND_TIMER=m
CONFIG_SND_PCM=m
CONFIG_SND_RAWMIDI=m
CONFIG_SND_SEQUENCER=m
# CONFIG_SND_SEQ_DUMMY is not set
CONFIG_SND_OSSEMUL=y
CONFIG_SND_MIXER_OSS=m
CONFIG_SND_PCM_OSS=m
CONFIG_SND_PCM_OSS_PLUGINS=y
CONFIG_SND_SEQUENCER_OSS=y
CONFIG_SND_RTCTIMER=m
CONFIG_SND_SEQ_RTCTIMER_DEFAULT=y
CONFIG_SND_DYNAMIC_MINORS=y
# CONFIG_SND_SUPPORT_OLD_API is not set
# CONFIG_SND_VERBOSE_PROCFS is not set
# CONFIG_SND_VERBOSE_PRINTK is not set
# CONFIG_SND_DEBUG is not set
#
# Generic devices
#
CONFIG_SND_AC97_CODEC=m
CONFIG_SND_AC97_BUS=m
# CONFIG_SND_DUMMY is not set
# CONFIG_SND_VIRMIDI is not set
# CONFIG_SND_MTPAV is not set
# CONFIG_SND_SERIAL_U16550 is not set
# CONFIG_SND_MPU401 is not set
#
# ISA devices
#
# CONFIG_SND_AD1848 is not set
# CONFIG_SND_CMI8330 is not set
# CONFIG_SND_CS4231 is not set
# CONFIG_SND_CS4232 is not set
# CONFIG_SND_CS4236 is not set
# CONFIG_SND_ES1688 is not set
# CONFIG_SND_ES18XX is not set
# CONFIG_SND_GUSCLASSIC is not set
# CONFIG_SND_GUSEXTREME is not set
# CONFIG_SND_GUSMAX is not set
# CONFIG_SND_OPL3SA2 is not set
# CONFIG_SND_OPTI92X_AD1848 is not set
# CONFIG_SND_OPTI92X_CS4231 is not set
# CONFIG_SND_OPTI93X is not set
# CONFIG_SND_SB8 is not set
# CONFIG_SND_SB16 is not set
# CONFIG_SND_SBAWE is not set
# CONFIG_SND_SGALAXY is not set
# CONFIG_SND_SSCAPE is not set
# CONFIG_SND_WAVEFRONT is not set
#
# PCI devices
#
# CONFIG_SND_AD1889 is not set
# CONFIG_SND_ALS4000 is not set
# CONFIG_SND_ALI5451 is not set
# CONFIG_SND_ATIIXP is not set
# CONFIG_SND_ATIIXP_MODEM is not set
# CONFIG_SND_AU8810 is not set
# CONFIG_SND_AU8820 is not set
# CONFIG_SND_AU8830 is not set
# CONFIG_SND_AZT3328 is not set
# CONFIG_SND_BT87X is not set
# CONFIG_SND_CA0106 is not set
# CONFIG_SND_CMIPCI is not set
# CONFIG_SND_CS4281 is not set
# CONFIG_SND_CS46XX is not set
# CONFIG_SND_CS5535AUDIO is not set
# CONFIG_SND_EMU10K1 is not set
# CONFIG_SND_EMU10K1X is not set
# CONFIG_SND_ENS1370 is not set
CONFIG_SND_ENS1371=m
# CONFIG_SND_ES1938 is not set
# CONFIG_SND_ES1968 is not set
# CONFIG_SND_FM801 is not set
# CONFIG_SND_HDA_INTEL is not set
# CONFIG_SND_HDSP is not set
# CONFIG_SND_HDSPM is not set
# CONFIG_SND_ICE1712 is not set
# CONFIG_SND_ICE1724 is not set
# CONFIG_SND_INTEL8X0 is not set
# CONFIG_SND_INTEL8X0M is not set
# CONFIG_SND_KORG1212 is not set
# CONFIG_SND_MAESTRO3 is not set
# CONFIG_SND_MIXART is not set
# CONFIG_SND_NM256 is not set
# CONFIG_SND_PCXHR is not set
# CONFIG_SND_RME32 is not set
# CONFIG_SND_RME96 is not set
# CONFIG_SND_RME9652 is not set
# CONFIG_SND_SONICVIBES is not set
# CONFIG_SND_TRIDENT is not set
# CONFIG_SND_VIA82XX is not set
# CONFIG_SND_VIA82XX_MODEM is not set
# CONFIG_SND_VX222 is not set
# CONFIG_SND_YMFPCI is not set
#
# USB devices
#
# CONFIG_SND_USB_AUDIO is not set
# CONFIG_SND_USB_USX2Y is not set
#
# Open Sound System
#
# CONFIG_SOUND_PRIME is not set
#
# USB support
#
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
CONFIG_USB_ARCH_HAS_EHCI=y
CONFIG_USB=m
# CONFIG_USB_DEBUG is not set
#
# Miscellaneous USB options
#
CONFIG_USB_DEVICEFS=y
CONFIG_USB_BANDWIDTH=y
CONFIG_USB_DYNAMIC_MINORS=y
CONFIG_USB_SUSPEND=y
# CONFIG_USB_OTG is not set
#
# USB Host Controller Drivers
#
CONFIG_USB_EHCI_HCD=m
# CONFIG_USB_EHCI_SPLIT_ISO is not set
CONFIG_USB_EHCI_ROOT_HUB_TT=y
# CONFIG_USB_ISP116X_HCD is not set
# CONFIG_USB_OHCI_HCD is not set
CONFIG_USB_UHCI_HCD=m
# CONFIG_USB_SL811_HCD is not set
#
# USB Device Class drivers
#
# CONFIG_USB_ACM is not set
CONFIG_USB_PRINTER=m
#
# NOTE: USB_STORAGE enables SCSI, and 'SCSI disk support'
#
#
# may also be needed; see USB_STORAGE Help for more information
#
CONFIG_USB_STORAGE=m
# CONFIG_USB_STORAGE_DEBUG is not set
# CONFIG_USB_STORAGE_DATAFAB is not set
# CONFIG_USB_STORAGE_FREECOM is not set
# CONFIG_USB_STORAGE_ISD200 is not set
# CONFIG_USB_STORAGE_DPCM is not set
# CONFIG_USB_STORAGE_USBAT is not set
# CONFIG_USB_STORAGE_SDDR09 is not set
# CONFIG_USB_STORAGE_SDDR55 is not set
# CONFIG_USB_STORAGE_JUMPSHOT is not set
# CONFIG_USB_STORAGE_ALAUDA is not set
# CONFIG_USB_LIBUSUAL is not set
#
# USB Input Devices
#
CONFIG_USB_HID=m
CONFIG_USB_HIDINPUT=y
# CONFIG_USB_HIDINPUT_POWERBOOK is not set
# CONFIG_HID_FF is not set
# CONFIG_USB_HIDDEV is not set
#
# USB HID Boot Protocol drivers
#
# CONFIG_USB_KBD is not set
# CONFIG_USB_MOUSE is not set
# CONFIG_USB_AIPTEK is not set
# CONFIG_USB_WACOM is not set
# CONFIG_USB_ACECAD is not set
# CONFIG_USB_KBTAB is not set
# CONFIG_USB_POWERMATE is not set
# CONFIG_USB_MTOUCH is not set
# CONFIG_USB_ITMTOUCH is not set
# CONFIG_USB_EGALAX is not set
# CONFIG_USB_YEALINK is not set
# CONFIG_USB_XPAD is not set
# CONFIG_USB_ATI_REMOTE is not set
# CONFIG_USB_ATI_REMOTE2 is not set
# CONFIG_USB_KEYSPAN_REMOTE is not set
# CONFIG_USB_APPLETOUCH is not set
#
# USB Imaging devices
#
# CONFIG_USB_MDC800 is not set
# CONFIG_USB_MICROTEK is not set
#
# USB Multimedia devices
#
# CONFIG_USB_DABUSB is not set
# CONFIG_USB_VICAM is not set
# CONFIG_USB_DSBR is not set
# CONFIG_USB_ET61X251 is not set
# CONFIG_USB_IBMCAM is not set
# CONFIG_USB_KONICAWC is not set
# CONFIG_USB_OV511 is not set
# CONFIG_USB_SE401 is not set
# CONFIG_USB_SN9C102 is not set
# CONFIG_USB_STV680 is not set
# CONFIG_USB_ZC0301 is not set
# CONFIG_USB_PWC is not set
#
# USB Network Adapters
#
# CONFIG_USB_CATC is not set
# CONFIG_USB_KAWETH is not set
# CONFIG_USB_PEGASUS is not set
# CONFIG_USB_RTL8150 is not set
# CONFIG_USB_USBNET is not set
# CONFIG_USB_ZD1201 is not set
# CONFIG_USB_MON is not set
#
# USB port drivers
#
# CONFIG_USB_USS720 is not set
#
# USB Serial Converter support
#
# CONFIG_USB_SERIAL is not set
#
# USB Miscellaneous drivers
#
# CONFIG_USB_EMI62 is not set
# CONFIG_USB_EMI26 is not set
# CONFIG_USB_AUERSWALD is not set
# CONFIG_USB_RIO500 is not set
# CONFIG_USB_LEGOTOWER is not set
# CONFIG_USB_LCD is not set
# CONFIG_USB_LED is not set
# CONFIG_USB_CYTHERM is not set
# CONFIG_USB_GOTEMP is not set
# CONFIG_USB_PHIDGETKIT is not set
# CONFIG_USB_PHIDGETSERVO is not set
# CONFIG_USB_IDMOUSE is not set
# CONFIG_USB_SISUSBVGA is not set
# CONFIG_USB_LD is not set
# CONFIG_USB_TEST is not set
#
# USB DSL modem support
#
#
# USB Gadget Support
#
# CONFIG_USB_GADGET is not set
#
# MMC/SD Card support
#
# CONFIG_MMC is not set
#
# LED devices
#
# CONFIG_NEW_LEDS is not set
#
# InfiniBand support
#
# CONFIG_INFINIBAND is not set
#
# EDAC - error detection and reporting (RAS) (EXPERIMENTAL)
#
CONFIG_EDAC=m
#
# Reporting subsystems
#
# CONFIG_EDAC_DEBUG is not set
CONFIG_EDAC_MM_EDAC=m
CONFIG_EDAC_AMD76X=m
CONFIG_EDAC_E7XXX=m
# CONFIG_EDAC_E752X is not set
CONFIG_EDAC_I82875P=m
# CONFIG_EDAC_I82860 is not set
CONFIG_EDAC_R82600=m
CONFIG_EDAC_POLL=y
#
# Distributed Lock Manager
#
# CONFIG_DLM is not set
#
# Real Time Clock
#
# CONFIG_RTC_CLASS is not set
#
# File systems
#
CONFIG_EXT2_FS=y
CONFIG_EXT2_FS_XATTR=y
# CONFIG_EXT2_FS_POSIX_ACL is not set
# CONFIG_EXT2_FS_SECURITY is not set
CONFIG_EXT2_FS_XIP=y
CONFIG_FS_XIP=y
CONFIG_EXT3_FS=y
CONFIG_EXT3_FS_XATTR=y
CONFIG_EXT3_FS_POSIX_ACL=y
# CONFIG_EXT3_FS_SECURITY is not set
CONFIG_JBD=y
# CONFIG_JBD_DEBUG is not set
CONFIG_FS_MBCACHE=y
CONFIG_REISER4_FS=m
CONFIG_REISER4_DEBUG=y
CONFIG_REISERFS_FS=m
# CONFIG_REISERFS_CHECK is not set
# CONFIG_REISERFS_PROC_INFO is not set
# CONFIG_REISERFS_FS_XATTR is not set
# CONFIG_JFS_FS is not set
CONFIG_FS_POSIX_ACL=y
# CONFIG_XFS_FS is not set
# CONFIG_OCFS2_FS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_ROMFS_FS is not set
CONFIG_INOTIFY=y
# CONFIG_QUOTA is not set
CONFIG_DNOTIFY=y
# CONFIG_AUTOFS_FS is not set
CONFIG_AUTOFS4_FS=m
# CONFIG_FUSE_FS is not set
#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=m
CONFIG_JOLIET=y
# CONFIG_ZISOFS is not set
CONFIG_UDF_FS=m
CONFIG_UDF_NLS=y
#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=m
CONFIG_MSDOS_FS=m
CONFIG_VFAT_FS=m
CONFIG_FAT_DEFAULT_CODEPAGE=850
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
# CONFIG_NTFS_FS is not set
#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
CONFIG_RAMFS=y
CONFIG_CONFIGFS_FS=m
#
# Miscellaneous filesystems
#
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_CRAMFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set
#
# Network File Systems
#
CONFIG_NFS_FS=m
CONFIG_NFS_V3=y
# CONFIG_NFS_V3_ACL is not set
CONFIG_NFS_V4=y
# CONFIG_NFS_DIRECTIO is not set
CONFIG_NFSD=m
CONFIG_NFSD_V3=y
# CONFIG_NFSD_V3_ACL is not set
CONFIG_NFSD_V4=y
CONFIG_NFSD_TCP=y
CONFIG_LOCKD=m
CONFIG_LOCKD_V4=y
CONFIG_EXPORTFS=m
CONFIG_NFS_COMMON=y
CONFIG_SUNRPC=m
CONFIG_SUNRPC_GSS=m
CONFIG_RPCSEC_GSS_KRB5=m
# CONFIG_RPCSEC_GSS_SPKM3 is not set
CONFIG_SMB_FS=m
# CONFIG_SMB_NLS_DEFAULT is not set
CONFIG_CIFS=m
CONFIG_CIFS_STATS=y
# CONFIG_CIFS_STATS2 is not set
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
# CONFIG_CIFS_EXPERIMENTAL is not set
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
# CONFIG_AFS_FS is not set
# CONFIG_9P_FS is not set
#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
# CONFIG_ACORN_PARTITION is not set
# CONFIG_OSF_PARTITION is not set
# CONFIG_AMIGA_PARTITION is not set
# CONFIG_ATARI_PARTITION is not set
# CONFIG_MAC_PARTITION is not set
CONFIG_MSDOS_PARTITION=y
# CONFIG_BSD_DISKLABEL is not set
# CONFIG_MINIX_SUBPARTITION is not set
# CONFIG_SOLARIS_X86_PARTITION is not set
# CONFIG_UNIXWARE_DISKLABEL is not set
# CONFIG_LDM_PARTITION is not set
# CONFIG_SGI_PARTITION is not set
# CONFIG_ULTRIX_PARTITION is not set
# CONFIG_SUN_PARTITION is not set
# CONFIG_KARMA_PARTITION is not set
# CONFIG_EFI_PARTITION is not set
#
# Native Language Support
#
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=m
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
CONFIG_NLS_CODEPAGE_850=m
# CONFIG_NLS_CODEPAGE_852 is not set
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not set
# CONFIG_NLS_CODEPAGE_863 is not set
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not set
# CONFIG_NLS_CODEPAGE_936 is not set
# CONFIG_NLS_CODEPAGE_950 is not set
# CONFIG_NLS_CODEPAGE_932 is not set
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not set
# CONFIG_NLS_ISO8859_8 is not set
# CONFIG_NLS_CODEPAGE_1250 is not set
# CONFIG_NLS_CODEPAGE_1251 is not set
CONFIG_NLS_ASCII=m
CONFIG_NLS_ISO8859_1=m
# CONFIG_NLS_ISO8859_2 is not set
# CONFIG_NLS_ISO8859_3 is not set
# CONFIG_NLS_ISO8859_4 is not set
# CONFIG_NLS_ISO8859_5 is not set
# CONFIG_NLS_ISO8859_6 is not set
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
# CONFIG_NLS_ISO8859_14 is not set
CONFIG_NLS_ISO8859_15=m
# CONFIG_NLS_KOI8_R is not set
# CONFIG_NLS_KOI8_U is not set
# CONFIG_NLS_UTF8 is not set
#
# Instrumentation Support
#
# CONFIG_PROFILING is not set
# CONFIG_KPROBES is not set
#
# Kernel hacking
#
# CONFIG_PRINTK_TIME is not set
CONFIG_MAGIC_SYSRQ=y
CONFIG_DEBUG_SHIRQ=y
CONFIG_DEBUG_KERNEL=y
CONFIG_LOG_BUF_SHIFT=14
CONFIG_DETECT_SOFTLOCKUP=y
# CONFIG_SCHEDSTATS is not set
# CONFIG_DEBUG_SLAB is not set
CONFIG_DEBUG_MUTEXES=y
# CONFIG_DEBUG_SPINLOCK is not set
CONFIG_DEBUG_SPINLOCK_SLEEP=y
# CONFIG_DEBUG_KOBJECT is not set
CONFIG_DEBUG_BUGVERBOSE=y
# CONFIG_DEBUG_INFO is not set
# CONFIG_PAGE_OWNER is not set
# CONFIG_DEBUG_FS is not set
# CONFIG_DEBUG_VM is not set
CONFIG_FRAME_POINTER=y
CONFIG_UNWIND_INFO=y
CONFIG_FORCED_INLINING=y
# CONFIG_RCU_TORTURE_TEST is not set
# CONFIG_DEBUG_SYNCHRO_TEST is not set
CONFIG_EARLY_PRINTK=y
CONFIG_DEBUG_STACKOVERFLOW=y
# CONFIG_DEBUG_STACK_USAGE is not set
CONFIG_STACK_BACKTRACE_COLS=1
#
# Page alloc debug is incompatible with Software Suspend on i386
#
CONFIG_DEBUG_RODATA=y
CONFIG_DEBUG_INITDATA=y
CONFIG_4KSTACKS=y
CONFIG_X86_FIND_SMP_CONFIG=y
CONFIG_X86_MPPARSE=y
# CONFIG_KGDB is not set
#
# Security options
#
# CONFIG_KEYS is not set
# CONFIG_SECURITY is not set
#
# Cryptographic options
#
CONFIG_CRYPTO=y
CONFIG_CRYPTO_HMAC=y
# CONFIG_CRYPTO_NULL is not set
# CONFIG_CRYPTO_MD4 is not set
CONFIG_CRYPTO_MD5=y
CONFIG_CRYPTO_SHA1=m
# CONFIG_CRYPTO_SHA256 is not set
# CONFIG_CRYPTO_SHA512 is not set
# CONFIG_CRYPTO_WP512 is not set
# CONFIG_CRYPTO_TGR192 is not set
CONFIG_CRYPTO_DES=m
CONFIG_CRYPTO_BLOWFISH=m
# CONFIG_CRYPTO_TWOFISH is not set
# CONFIG_CRYPTO_SERPENT is not set
CONFIG_CRYPTO_AES=m
CONFIG_CRYPTO_AES_586=m
# CONFIG_CRYPTO_CAST5 is not set
# CONFIG_CRYPTO_CAST6 is not set
# CONFIG_CRYPTO_TEA is not set
CONFIG_CRYPTO_ARC4=m
# CONFIG_CRYPTO_KHAZAD is not set
# CONFIG_CRYPTO_ANUBIS is not set
CONFIG_CRYPTO_DEFLATE=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_CRC32C=m
# CONFIG_CRYPTO_TEST is not set
#
# Hardware crypto devices
#
# CONFIG_CRYPTO_DEV_PADLOCK is not set
#
# Library routines
#
CONFIG_CRC_CCITT=m
# CONFIG_CRC16 is not set
CONFIG_CRC32=y
CONFIG_LIBCRC32C=m
CONFIG_ZLIB_INFLATE=m
CONFIG_ZLIB_DEFLATE=m
CONFIG_GENERIC_HARDIRQS=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_X86_BIOS_REBOOT=y
CONFIG_KTIME_SCALAR=y
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: 2.6.16-rc6-mm2: reiser4 BUG when unmounting fs
2006-03-21 20:27 ` 2.6.16-rc6-mm2: reiser4 BUG when unmounting fs Laurent Riffard
@ 2006-03-21 20:38 ` Laurent Riffard
2006-03-22 7:43 ` Vladimir V. Saveliev
0 siblings, 1 reply; 97+ messages in thread
From: Laurent Riffard @ 2006-03-21 20:38 UTC (permalink / raw)
To: Andrew Morton; +Cc: Kernel development list, reiserfs-list
Le 21.03.2006 21:27, Laurent Riffard a écrit :
> Le 18.03.2006 13:40, Andrew Morton a écrit :
>
>>ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc6/2.6.16-rc6-mm2/
>
>
> Hello,
>
> This BUG is 100% reproducible. Simply boot to runlevel 1 and then
> unmount a reiser4 fs:
Oops! Somebody already reported it: http://lkml.org/lkml/2006/3/21/88.
Sorry for the noise...
--
laurent
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: 2.6.16-rc6-mm2: reiser4 BUG when unmounting fs
2006-03-21 20:38 ` Laurent Riffard
@ 2006-03-22 7:43 ` Vladimir V. Saveliev
2006-03-22 18:38 ` Laurent Riffard
2006-03-23 17:32 ` Alexander Gran
0 siblings, 2 replies; 97+ messages in thread
From: Vladimir V. Saveliev @ 2006-03-22 7:43 UTC (permalink / raw)
To: Laurent Riffard; +Cc: Andrew Morton, Kernel development list, reiserfs-list
[-- Attachment #1: Type: text/plain, Size: 430 bytes --]
Hello
On Tue, 2006-03-21 at 21:38 +0100, Laurent Riffard wrote:
> Le 21.03.2006 21:27, Laurent Riffard a écrit :
> > Le 18.03.2006 13:40, Andrew Morton a écrit :
> >
> >>ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc6/2.6.16-rc6-mm2/
> >
> >
> > Hello,
> >
> > This BUG is 100% reproducible. Simply boot to runlevel 1 and then
> > unmount a reiser4 fs:
>
The attached patch fixes the problem.
[-- Attachment #2: reiser4-cleanup_init_fake_inode.patch --]
[-- Type: text/x-patch, Size: 731 bytes --]
fs/reiser4/page_cache.c | 4 ----
1 files changed, 4 deletions(-)
diff -puN fs/reiser4/page_cache.c~reiser4-fix-bd_inode fs/reiser4/page_cache.c
--- linux-2.6.16-rc6-mm2/fs/reiser4/page_cache.c~reiser4-fix-bd_inode 2006-03-21 06:42:42.000000000 +0300
+++ linux-2.6.16-rc6-mm2-vs/fs/reiser4/page_cache.c 2006-03-21 07:21:54.000000000 +0300
@@ -198,10 +198,6 @@ init_fake_inode(struct super_block *supe
{
assert("nikita-2168", fake->i_state & I_NEW);
fake->i_mapping->a_ops = &formatted_fake_as_ops;
- fake->i_blkbits = super->s_blocksize_bits;
- fake->i_size = ~0ull;
- fake->i_rdev = super->s_bdev->bd_dev;
- fake->i_bdev = super->s_bdev;
*pfake = fake;
/* NOTE-NIKITA something else? */
unlock_new_inode(fake);
_
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: 2.6.16-rc6-mm2: reiser4 BUG when unmounting fs
2006-03-22 7:43 ` Vladimir V. Saveliev
@ 2006-03-22 18:38 ` Laurent Riffard
2006-03-23 17:32 ` Alexander Gran
1 sibling, 0 replies; 97+ messages in thread
From: Laurent Riffard @ 2006-03-22 18:38 UTC (permalink / raw)
To: Vladimir V. Saveliev
Cc: Andrew Morton, Kernel development list, reiserfs-list
Le 22.03.2006 08:43, Vladimir V. Saveliev a écrit :
> Hello
> On Tue, 2006-03-21 at 21:38 +0100, Laurent Riffard wrote:
>>Le 21.03.2006 21:27, Laurent Riffard a écrit :
>>
>>>Le 18.03.2006 13:40, Andrew Morton a écrit :
>>>>ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc6/2.6.16-rc6-mm2/
>>>
>>>Hello,
>>>
>>>This BUG is 100% reproducible. Simply boot to runlevel 1 and then
>>>unmount a reiser4 fs:
>>
>
> The attached patch fixes the problem.
Ok, it works fine now.
Thanks.
> ------------------------------------------------------------------------
>
> fs/reiser4/page_cache.c | 4 ----
> 1 files changed, 4 deletions(-)
>
> diff -puN fs/reiser4/page_cache.c~reiser4-fix-bd_inode fs/reiser4/page_cache.c
> --- linux-2.6.16-rc6-mm2/fs/reiser4/page_cache.c~reiser4-fix-bd_inode 2006-03-21 06:42:42.000000000 +0300
> +++ linux-2.6.16-rc6-mm2-vs/fs/reiser4/page_cache.c 2006-03-21 07:21:54.000000000 +0300
> @@ -198,10 +198,6 @@ init_fake_inode(struct super_block *supe
> {
> assert("nikita-2168", fake->i_state & I_NEW);
> fake->i_mapping->a_ops = &formatted_fake_as_ops;
> - fake->i_blkbits = super->s_blocksize_bits;
> - fake->i_size = ~0ull;
> - fake->i_rdev = super->s_bdev->bd_dev;
> - fake->i_bdev = super->s_bdev;
> *pfake = fake;
> /* NOTE-NIKITA something else? */
> unlock_new_inode(fake);
>
> _
--
laurent
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: 2.6.16-rc6-mm2: reiser4 BUG when unmounting fs
2006-03-22 7:43 ` Vladimir V. Saveliev
2006-03-22 18:38 ` Laurent Riffard
@ 2006-03-23 17:32 ` Alexander Gran
1 sibling, 0 replies; 97+ messages in thread
From: Alexander Gran @ 2006-03-23 17:32 UTC (permalink / raw)
To: Vladimir V. Saveliev
Cc: Laurent Riffard, Andrew Morton, Kernel development list,
reiserfs-list
[-- Attachment #1: Type: text/plain, Size: 266 bytes --]
Am Mittwoch, 22. März 2006 08:43 schrieb Vladimir V. Saveliev:
> The attached patch fixes the problem.
confirmed, works in 2.6.16-mm1.
regards
Alex
--
Encrypted Mails welcome.
PGP-Key at http://zodiac.dnsalias.org/misc/pgpkey.asc | Key-ID: 0x6D7DD291
[-- Attachment #2: Type: application/pgp-signature, Size: 191 bytes --]
^ permalink raw reply [flat|nested] 97+ messages in thread
* 2.6.16-rc6-mm2: Why is CONFIG_MIGRATION available for everyone?
2006-03-18 12:40 2.6.16-rc6-mm2 Andrew Morton
` (11 preceding siblings ...)
2006-03-21 20:27 ` 2.6.16-rc6-mm2: reiser4 BUG when unmounting fs Laurent Riffard
@ 2006-03-21 22:54 ` Adrian Bunk
2006-03-21 22:57 ` Christoph Lameter
2006-03-21 23:40 ` Christoph Lameter
2006-03-22 8:41 ` 2.6.16-rc6-mm2 J.A. Magallon
2006-03-22 10:40 ` [2.6 patch] kernel/time.c: remove unused pps_* variables Adrian Bunk
14 siblings, 2 replies; 97+ messages in thread
From: Adrian Bunk @ 2006-03-21 22:54 UTC (permalink / raw)
To: Andrew Morton, Christoph Lameter; +Cc: linux-kernel
On Sat, Mar 18, 2006 at 04:40:56AM -0800, Andrew Morton wrote:
>...
> Changes since 2.6.16-rc6-mm1:
>...
> +page-migration-reorg.patch
>...
> Reorganise the page migration code
>...
The patch description includes:
5. Make it possible to configure NUMA systems without page migration
and non-NUMA systems with page migration.
I don't see the point in making this option visible for the majority of
users on non-NUMA systems who will never need it.
When is it required on non-NUMA systems?
Memory hotplug?
Can we express this explicitely?
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: 2.6.16-rc6-mm2: Why is CONFIG_MIGRATION available for everyone?
2006-03-21 22:54 ` 2.6.16-rc6-mm2: Why is CONFIG_MIGRATION available for everyone? Adrian Bunk
@ 2006-03-21 22:57 ` Christoph Lameter
2006-03-21 23:40 ` Christoph Lameter
1 sibling, 0 replies; 97+ messages in thread
From: Christoph Lameter @ 2006-03-21 22:57 UTC (permalink / raw)
To: Adrian Bunk; +Cc: Andrew Morton, linux-kernel
On Tue, 21 Mar 2006, Adrian Bunk wrote:
> 5. Make it possible to configure NUMA systems without page migration
> and non-NUMA systems with page migration.
>
>
> I don't see the point in making this option visible for the majority of
> users on non-NUMA systems who will never need it.
>
> When is it required on non-NUMA systems?
> Memory hotplug?
That is one. It also may be useful for transferring pages between various
zones. Could be used to free up space in ZONE_DMA etc. Right now none of
these uses exist so we could disable it by default for now?
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: 2.6.16-rc6-mm2: Why is CONFIG_MIGRATION available for everyone?
2006-03-21 22:54 ` 2.6.16-rc6-mm2: Why is CONFIG_MIGRATION available for everyone? Adrian Bunk
2006-03-21 22:57 ` Christoph Lameter
@ 2006-03-21 23:40 ` Christoph Lameter
1 sibling, 0 replies; 97+ messages in thread
From: Christoph Lameter @ 2006-03-21 23:40 UTC (permalink / raw)
To: Adrian Bunk; +Cc: Andrew Morton, linux-kernel
On Tue, 21 Mar 2006, Adrian Bunk wrote:
> Can we express this explicitely?
How about this fix?
Make page migration dependent on swap and NUMA. The page migration code
could function without NUMA but we currently have no users for the
non-NUMA case.
Signed-off-by: Christoph Lameter <clameter@sgi.com>
Index: linux-2.6.16-rc6-mm2/mm/Kconfig
===================================================================
--- linux-2.6.16-rc6-mm2.orig/mm/Kconfig 2006-03-21 14:51:37.000000000 -0800
+++ linux-2.6.16-rc6-mm2/mm/Kconfig 2006-03-21 15:36:25.000000000 -0800
@@ -138,8 +138,8 @@ config SPLIT_PTLOCK_CPUS
#
config MIGRATION
bool "Page migration"
- def_bool y if NUMA || SPARSEMEM || DISCONTIGMEM
- depends on SWAP
+ def_bool y if NUMA
+ depends on SWAP && NUMA
help
Allows the migration of the physical location of pages of processes
while the virtual addresses are not changed. This is useful for
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: 2.6.16-rc6-mm2
2006-03-18 12:40 2.6.16-rc6-mm2 Andrew Morton
` (12 preceding siblings ...)
2006-03-21 22:54 ` 2.6.16-rc6-mm2: Why is CONFIG_MIGRATION available for everyone? Adrian Bunk
@ 2006-03-22 8:41 ` J.A. Magallon
2006-03-22 8:48 ` 2.6.16-rc6-mm2 Andrew Morton
2006-03-22 10:40 ` [2.6 patch] kernel/time.c: remove unused pps_* variables Adrian Bunk
14 siblings, 1 reply; 97+ messages in thread
From: J.A. Magallon @ 2006-03-22 8:41 UTC (permalink / raw)
To: Andrew Morton, Linux-Kernel,
[-- Attachment #1: Type: text/plain, Size: 916 bytes --]
On Sat, 18 Mar 2006 04:40:56 -0800, Andrew Morton <akpm@osdl.org> wrote:
>
> ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc6/2.6.16-rc6-mm2/
>
>
> - John's time rework patches were dropped - they're being reworked.
>
> - Lots of MD and DM updates
>
Mmmm, somthing strange is in this kernel. Is hangs the box in the middle
of the night, it looks like it got stuck on the scsi disk on an AHC
controller...I get no info on syslog.
Are there any changes in aic drivers ? It also has a raid array, perhaps
some change in md code is borking when cron jobs are run in the night...
--
J.A. Magallon <jamagallon()able!es> \ Software is like sex:
werewolf!able!es \ It's better when it's free
Mandriva Linux release 2006.1 (Cooker) for i586
Linux 2.6.15-jam20 (gcc 4.0.3 (4.0.3-1mdk for Mandriva Linux release 2006.1))
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 191 bytes --]
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: 2.6.16-rc6-mm2
2006-03-22 8:41 ` 2.6.16-rc6-mm2 J.A. Magallon
@ 2006-03-22 8:48 ` Andrew Morton
0 siblings, 0 replies; 97+ messages in thread
From: Andrew Morton @ 2006-03-22 8:48 UTC (permalink / raw)
To: J.A. Magallon; +Cc: linux-kernel
"J.A. Magallon" <jamagallon@able.es> wrote:
>
> On Sat, 18 Mar 2006 04:40:56 -0800, Andrew Morton <akpm@osdl.org> wrote:
>
> >
> > ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.16-rc6/2.6.16-rc6-mm2/
> >
> >
> > - John's time rework patches were dropped - they're being reworked.
> >
> > - Lots of MD and DM updates
> >
>
> Mmmm, somthing strange is in this kernel. Is hangs the box in the middle
> of the night, it looks like it got stuck on the scsi disk on an AHC
> controller...I get no info on syslog.
>
> Are there any changes in aic drivers ? It also has a raid array, perhaps
> some change in md code is borking when cron jobs are run in the night...
diffstat will tell.
b/drivers/scsi/aic7xxx/aic79xx_core.c | 33
b/drivers/scsi/aic7xxx/aic79xx_osm.c | 559
b/drivers/scsi/aic7xxx/aic79xx_osm.h | 7
b/drivers/scsi/aic7xxx/aic7xxx_core.c | 24
b/drivers/scsi/aic7xxx/aic7xxx_osm.c | 45
b/drivers/scsi/aic7xxx/aic7xxx_osm.h | 5
There are large numbers of changes to scsi core as well. And MD. And
everything else.
Can no info be obtained from sysrq-P or sysrq-T?
If it's running X then I'd suggest you quit from X overnight, see if
anything pops up on the screen, and to simplify using sysrq.
^ permalink raw reply [flat|nested] 97+ messages in thread
* [2.6 patch] kernel/time.c: remove unused pps_* variables
2006-03-18 12:40 2.6.16-rc6-mm2 Andrew Morton
` (13 preceding siblings ...)
2006-03-22 8:41 ` 2.6.16-rc6-mm2 J.A. Magallon
@ 2006-03-22 10:40 ` Adrian Bunk
14 siblings, 0 replies; 97+ messages in thread
From: Adrian Bunk @ 2006-03-22 10:40 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel
On Sat, Mar 18, 2006 at 04:40:56AM -0800, Andrew Morton wrote:
>...
> Changes since 2.6.16-rc6-mm1:
>...
> -kernel-timec-remove-unused-pps_-variables.patch
>
> Dropped - being redone.
>...
This patch was unrelated, but seems to have been dropped due to context
changes.
A version that applies against 2.6.16-rc6-mm2 is below.
cu
Adrian
<-- snip -->
From: Adrian Bunk <bunk@stusta.de>
AFAIR there is a patch floating around that might use these variables,
but this patch is still unmerged and my patch can easily be undone.
Signed-off-by: Adrian Bunk <bunk@stusta.de>
---
include/linux/timex.h | 8 --------
kernel/time.c | 23 ++++++-----------------
2 files changed, 6 insertions(+), 25 deletions(-)
--- linux-2.6.16-rc6-mm2-full/include/linux/timex.h.old 2006-03-21 23:30:45.000000000 +0100
+++ linux-2.6.16-rc6-mm2-full/include/linux/timex.h 2006-03-21 23:31:20.000000000 +0100
@@ -247,19 +247,11 @@
extern long time_next_adjust; /* Value for time_adjust at next tick */
/* interface variables pps->timer interrupt */
-extern long pps_offset; /* pps time offset (us) */
extern long pps_jitter; /* time dispersion (jitter) (us) */
extern long pps_freq; /* frequency offset (scaled ppm) */
extern long pps_stabil; /* frequency dispersion (scaled ppm) */
extern long pps_valid; /* pps signal watchdog counter */
-/* interface variables pps->adjtimex */
-extern int pps_shift; /* interval duration (s) (shift) */
-extern long pps_jitcnt; /* jitter limit exceeded */
-extern long pps_calcnt; /* calibration intervals */
-extern long pps_errcnt; /* calibration errors */
-extern long pps_stbcnt; /* stability limit exceeded */
-
/**
* ntp_clear - Clears the NTP state variables
*
--- linux-2.6.16-rc6-mm2-full/kernel/time.c.old 2006-03-21 23:30:57.000000000 +0100
+++ linux-2.6.16-rc6-mm2-full/kernel/time.c 2006-03-21 23:31:58.000000000 +0100
@@ -202,7 +202,6 @@
return do_sys_settimeofday(tv ? &new_ts : NULL, tz ? &new_tz : NULL);
}
-long pps_offset; /* pps time offset (us) */
long pps_jitter = MAXTIME; /* time dispersion (jitter) (us) */
long pps_freq; /* frequency offset (scaled ppm) */
@@ -210,16 +209,6 @@
long pps_valid = PPS_VALID; /* pps signal watchdog counter */
-int pps_shift = PPS_SHIFT; /* interval duration (s) (shift) */
-
-long pps_jitcnt; /* jitter limit exceeded */
-long pps_calcnt; /* calibration intervals */
-long pps_errcnt; /* calibration errors */
-long pps_stbcnt; /* stability limit exceeded */
-
-/* hook for a loadable hardpps kernel module */
-void (*hardpps_ptr)(struct timeval *);
-
/* we call this to notify the arch when the clock is being
* controlled. If no such arch routine, do nothing.
*/
@@ -315,7 +304,7 @@
else if ( time_status & (STA_PLL | STA_PPSTIME) ) {
ltemp = (time_status & (STA_PPSTIME | STA_PPSSIGNAL)) ==
(STA_PPSTIME | STA_PPSSIGNAL) ?
- pps_offset : txc->offset;
+ 0 : txc->offset;
/*
* Scale the phase adjustment and
@@ -390,12 +379,12 @@
txc->tick = tick_usec;
txc->ppsfreq = pps_freq;
txc->jitter = pps_jitter >> PPS_AVG;
- txc->shift = pps_shift;
+ txc->shift = 0;
txc->stabil = pps_stabil;
- txc->jitcnt = pps_jitcnt;
- txc->calcnt = pps_calcnt;
- txc->errcnt = pps_errcnt;
- txc->stbcnt = pps_stbcnt;
+ txc->jitcnt = 0;
+ txc->calcnt = 0;
+ txc->errcnt = 0;
+ txc->stbcnt = 0;
write_sequnlock_irq(&xtime_lock);
do_gettimeofday(&txc->time);
notify_arch_cmos_timer();
^ permalink raw reply [flat|nested] 97+ messages in thread
* [PATCH 1/5] Infiniband: connection abstraction
2006-02-01 20:03 [PATCH 0/5] Infiniband: connection abstraction Sean Hefty
@ 2006-02-01 20:07 ` Sean Hefty
2006-02-01 20:10 ` [PATCH 2/5] " Sean Hefty
` (4 subsequent siblings)
5 siblings, 0 replies; 97+ messages in thread
From: Sean Hefty @ 2006-02-01 20:07 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: openib-general
The following patch provides common handling for marshalling data between
Userspace clients and kernel mode Infiniband drivers.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
---
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/Makefile
linux-2.6.ib/drivers/infiniband/core/Makefile
--- linux-2.6.git/drivers/infiniband/core/Makefile 2006-01-16 10:25:27.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/Makefile 2006-01-16 15:34:15.000000000 -0800
@@ -16,4 +16,5 @@ ib_umad-y := user_mad.o
ib_ucm-y := ucm.o
-ib_uverbs-y := uverbs_main.o uverbs_cmd.o uverbs_mem.o
+ib_uverbs-y := uverbs_main.o uverbs_cmd.o uverbs_mem.o \
+ uverbs_marshall.o
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/ucm.c
linux-2.6.ib/drivers/infiniband/core/ucm.c
--- linux-2.6.git/drivers/infiniband/core/ucm.c 2006-01-16 10:25:26.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/ucm.c 2006-01-16 15:34:15.000000000 -0800
@@ -30,7 +30,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- * $Id: ucm.c 2594 2005-06-13 19:46:02Z libor $
+ * $Id: ucm.c 4311 2005-12-05 18:42:01Z sean.hefty $
*/
#include <linux/init.h>
#include <linux/fs.h>
@@ -48,6 +48,7 @@
#include <rdma/ib_cm.h>
#include <rdma/ib_user_cm.h>
+#include <rdma/ib_marshall.h>
MODULE_AUTHOR("Libor Michalek");
MODULE_DESCRIPTION("InfiniBand userspace Connection Manager access");
@@ -203,36 +204,6 @@ error:
return NULL;
}
-static void ib_ucm_event_path_get(struct ib_ucm_path_rec *upath,
- struct ib_sa_path_rec *kpath)
-{
- if (!kpath || !upath)
- return;
-
- memcpy(upath->dgid, kpath->dgid.raw, sizeof *upath->dgid);
- memcpy(upath->sgid, kpath->sgid.raw, sizeof *upath->sgid);
-
- upath->dlid = kpath->dlid;
- upath->slid = kpath->slid;
- upath->raw_traffic = kpath->raw_traffic;
- upath->flow_label = kpath->flow_label;
- upath->hop_limit = kpath->hop_limit;
- upath->traffic_class = kpath->traffic_class;
- upath->reversible = kpath->reversible;
- upath->numb_path = kpath->numb_path;
- upath->pkey = kpath->pkey;
- upath->sl = kpath->sl;
- upath->mtu_selector = kpath->mtu_selector;
- upath->mtu = kpath->mtu;
- upath->rate_selector = kpath->rate_selector;
- upath->rate = kpath->rate;
- upath->packet_life_time = kpath->packet_life_time;
- upath->preference = kpath->preference;
-
- upath->packet_life_time_selector =
- kpath->packet_life_time_selector;
-}
-
static void ib_ucm_event_req_get(struct ib_ucm_req_event_resp *ureq,
struct ib_cm_req_event_param *kreq)
{
@@ -251,8 +222,10 @@ static void ib_ucm_event_req_get(struct
ureq->srq = kreq->srq;
ureq->port = kreq->port;
- ib_ucm_event_path_get(&ureq->primary_path, kreq->primary_path);
- ib_ucm_event_path_get(&ureq->alternate_path, kreq->alternate_path);
+ ib_copy_path_rec_to_user(&ureq->primary_path, kreq->primary_path);
+ if (kreq->alternate_path)
+ ib_copy_path_rec_to_user(&ureq->alternate_path,
+ kreq->alternate_path);
}
static void ib_ucm_event_rep_get(struct ib_ucm_rep_event_resp *urep,
@@ -322,8 +295,8 @@ static int ib_ucm_event_process(struct i
info = evt->param.rej_rcvd.ari;
break;
case IB_CM_LAP_RECEIVED:
- ib_ucm_event_path_get(&uvt->resp.u.lap_resp.path,
- evt->param.lap_rcvd.alternate_path);
+ ib_copy_path_rec_to_user(&uvt->resp.u.lap_resp.path,
+ evt->param.lap_rcvd.alternate_path);
uvt->data_len = IB_CM_LAP_PRIVATE_DATA_SIZE;
uvt->resp.present = IB_UCM_PRES_ALTERNATE;
break;
@@ -635,65 +608,11 @@ static ssize_t ib_ucm_attr_id(struct ib_
return result;
}
-static void ib_ucm_copy_ah_attr(struct ib_ucm_ah_attr *dest_attr,
- struct ib_ah_attr *src_attr)
-{
- memcpy(dest_attr->grh_dgid, src_attr->grh.dgid.raw,
- sizeof src_attr->grh.dgid);
- dest_attr->grh_flow_label = src_attr->grh.flow_label;
- dest_attr->grh_sgid_index = src_attr->grh.sgid_index;
- dest_attr->grh_hop_limit = src_attr->grh.hop_limit;
- dest_attr->grh_traffic_class = src_attr->grh.traffic_class;
-
- dest_attr->dlid = src_attr->dlid;
- dest_attr->sl = src_attr->sl;
- dest_attr->src_path_bits = src_attr->src_path_bits;
- dest_attr->static_rate = src_attr->static_rate;
- dest_attr->is_global = (src_attr->ah_flags & IB_AH_GRH);
- dest_attr->port_num = src_attr->port_num;
-}
-
-static void ib_ucm_copy_qp_attr(struct ib_ucm_init_qp_attr_resp *dest_attr,
- struct ib_qp_attr *src_attr)
-{
- dest_attr->cur_qp_state = src_attr->cur_qp_state;
- dest_attr->path_mtu = src_attr->path_mtu;
- dest_attr->path_mig_state = src_attr->path_mig_state;
- dest_attr->qkey = src_attr->qkey;
- dest_attr->rq_psn = src_attr->rq_psn;
- dest_attr->sq_psn = src_attr->sq_psn;
- dest_attr->dest_qp_num = src_attr->dest_qp_num;
- dest_attr->qp_access_flags = src_attr->qp_access_flags;
-
- dest_attr->max_send_wr = src_attr->cap.max_send_wr;
- dest_attr->max_recv_wr = src_attr->cap.max_recv_wr;
- dest_attr->max_send_sge = src_attr->cap.max_send_sge;
- dest_attr->max_recv_sge = src_attr->cap.max_recv_sge;
- dest_attr->max_inline_data = src_attr->cap.max_inline_data;
-
- ib_ucm_copy_ah_attr(&dest_attr->ah_attr, &src_attr->ah_attr);
- ib_ucm_copy_ah_attr(&dest_attr->alt_ah_attr, &src_attr->alt_ah_attr);
-
- dest_attr->pkey_index = src_attr->pkey_index;
- dest_attr->alt_pkey_index = src_attr->alt_pkey_index;
- dest_attr->en_sqd_async_notify = src_attr->en_sqd_async_notify;
- dest_attr->sq_draining = src_attr->sq_draining;
- dest_attr->max_rd_atomic = src_attr->max_rd_atomic;
- dest_attr->max_dest_rd_atomic = src_attr->max_dest_rd_atomic;
- dest_attr->min_rnr_timer = src_attr->min_rnr_timer;
- dest_attr->port_num = src_attr->port_num;
- dest_attr->timeout = src_attr->timeout;
- dest_attr->retry_cnt = src_attr->retry_cnt;
- dest_attr->rnr_retry = src_attr->rnr_retry;
- dest_attr->alt_port_num = src_attr->alt_port_num;
- dest_attr->alt_timeout = src_attr->alt_timeout;
-}
-
static ssize_t ib_ucm_init_qp_attr(struct ib_ucm_file *file,
const char __user *inbuf,
int in_len, int out_len)
{
- struct ib_ucm_init_qp_attr_resp resp;
+ struct ib_uverbs_qp_attr resp;
struct ib_ucm_init_qp_attr cmd;
struct ib_ucm_context *ctx;
struct ib_qp_attr qp_attr;
@@ -716,7 +635,7 @@ static ssize_t ib_ucm_init_qp_attr(struc
if (result)
goto out;
- ib_ucm_copy_qp_attr(&resp, &qp_attr);
+ ib_copy_qp_attr_to_user(&resp, &qp_attr);
if (copy_to_user((void __user *)(unsigned long)cmd.response,
&resp, sizeof(resp)))
@@ -791,7 +710,7 @@ static int ib_ucm_alloc_data(const void
static int ib_ucm_path_get(struct ib_sa_path_rec **path, u64 src)
{
- struct ib_ucm_path_rec ucm_path;
+ struct ib_user_path_rec upath;
struct ib_sa_path_rec *sa_path;
*path = NULL;
@@ -803,36 +722,14 @@ static int ib_ucm_path_get(struct ib_sa_
if (!sa_path)
return -ENOMEM;
- if (copy_from_user(&ucm_path, (void __user *)(unsigned long)src,
- sizeof(ucm_path))) {
+ if (copy_from_user(&upath, (void __user *)(unsigned long)src,
+ sizeof(upath))) {
kfree(sa_path);
return -EFAULT;
}
- memcpy(sa_path->dgid.raw, ucm_path.dgid, sizeof sa_path->dgid);
- memcpy(sa_path->sgid.raw, ucm_path.sgid, sizeof sa_path->sgid);
-
- sa_path->dlid = ucm_path.dlid;
- sa_path->slid = ucm_path.slid;
- sa_path->raw_traffic = ucm_path.raw_traffic;
- sa_path->flow_label = ucm_path.flow_label;
- sa_path->hop_limit = ucm_path.hop_limit;
- sa_path->traffic_class = ucm_path.traffic_class;
- sa_path->reversible = ucm_path.reversible;
- sa_path->numb_path = ucm_path.numb_path;
- sa_path->pkey = ucm_path.pkey;
- sa_path->sl = ucm_path.sl;
- sa_path->mtu_selector = ucm_path.mtu_selector;
- sa_path->mtu = ucm_path.mtu;
- sa_path->rate_selector = ucm_path.rate_selector;
- sa_path->rate = ucm_path.rate;
- sa_path->packet_life_time = ucm_path.packet_life_time;
- sa_path->preference = ucm_path.preference;
-
- sa_path->packet_life_time_selector =
- ucm_path.packet_life_time_selector;
-
+ ib_copy_path_rec_from_user(sa_path, &upath);
*path = sa_path;
return 0;
}
@@ -1243,8 +1140,10 @@ static unsigned int ib_ucm_poll(struct f
poll_wait(filp, &file->poll_wait, wait);
+ down(&file->mutex);
if (!list_empty(&file->events))
mask = POLLIN | POLLRDNORM;
+ up(&file->mutex);
return mask;
}
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/uverbs_marshall.c
linux-2.6.ib/drivers/infiniband/core/uverbs_marshall.c
--- linux-2.6.git/drivers/infiniband/core/uverbs_marshall.c 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/uverbs_marshall.c 2006-01-16 15:34:15.000000000 -0800
@@ -0,0 +1,138 @@
+/*
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <rdma/ib_marshall.h>
+
+static void ib_copy_ah_attr_to_user(struct ib_uverbs_ah_attr *dst,
+ struct ib_ah_attr *src)
+{
+ memcpy(dst->grh.dgid, src->grh.dgid.raw, sizeof src->grh.dgid);
+ dst->grh.flow_label = src->grh.flow_label;
+ dst->grh.sgid_index = src->grh.sgid_index;
+ dst->grh.hop_limit = src->grh.hop_limit;
+ dst->grh.traffic_class = src->grh.traffic_class;
+ dst->dlid = src->dlid;
+ dst->sl = src->sl;
+ dst->src_path_bits = src->src_path_bits;
+ dst->static_rate = src->static_rate;
+ dst->is_global = src->ah_flags & IB_AH_GRH ? 1 : 0;
+ dst->port_num = src->port_num;
+}
+
+void ib_copy_qp_attr_to_user(struct ib_uverbs_qp_attr *dst,
+ struct ib_qp_attr *src)
+{
+ dst->cur_qp_state = src->cur_qp_state;
+ dst->path_mtu = src->path_mtu;
+ dst->path_mig_state = src->path_mig_state;
+ dst->qkey = src->qkey;
+ dst->rq_psn = src->rq_psn;
+ dst->sq_psn = src->sq_psn;
+ dst->dest_qp_num = src->dest_qp_num;
+ dst->qp_access_flags = src->qp_access_flags;
+
+ dst->max_send_wr = src->cap.max_send_wr;
+ dst->max_recv_wr = src->cap.max_recv_wr;
+ dst->max_send_sge = src->cap.max_send_sge;
+ dst->max_recv_sge = src->cap.max_recv_sge;
+ dst->max_inline_data = src->cap.max_inline_data;
+
+ ib_copy_ah_attr_to_user(&dst->ah_attr, &src->ah_attr);
+ ib_copy_ah_attr_to_user(&dst->alt_ah_attr, &src->alt_ah_attr);
+
+ dst->pkey_index = src->pkey_index;
+ dst->alt_pkey_index = src->alt_pkey_index;
+ dst->en_sqd_async_notify = src->en_sqd_async_notify;
+ dst->sq_draining = src->sq_draining;
+ dst->max_rd_atomic = src->max_rd_atomic;
+ dst->max_dest_rd_atomic = src->max_dest_rd_atomic;
+ dst->min_rnr_timer = src->min_rnr_timer;
+ dst->port_num = src->port_num;
+ dst->timeout = src->timeout;
+ dst->retry_cnt = src->retry_cnt;
+ dst->rnr_retry = src->rnr_retry;
+ dst->alt_port_num = src->alt_port_num;
+ dst->alt_timeout = src->alt_timeout;
+}
+EXPORT_SYMBOL(ib_copy_qp_attr_to_user);
+
+void ib_copy_path_rec_to_user(struct ib_user_path_rec *dst,
+ struct ib_sa_path_rec *src)
+{
+ memcpy(dst->dgid, src->dgid.raw, sizeof src->dgid);
+ memcpy(dst->sgid, src->sgid.raw, sizeof src->sgid);
+
+ dst->dlid = src->dlid;
+ dst->slid = src->slid;
+ dst->raw_traffic = src->raw_traffic;
+ dst->flow_label = src->flow_label;
+ dst->hop_limit = src->hop_limit;
+ dst->traffic_class = src->traffic_class;
+ dst->reversible = src->reversible;
+ dst->numb_path = src->numb_path;
+ dst->pkey = src->pkey;
+ dst->sl = src->sl;
+ dst->mtu_selector = src->mtu_selector;
+ dst->mtu = src->mtu;
+ dst->rate_selector = src->rate_selector;
+ dst->rate = src->rate;
+ dst->packet_life_time = src->packet_life_time;
+ dst->preference = src->preference;
+ dst->packet_life_time_selector = src->packet_life_time_selector;
+}
+EXPORT_SYMBOL(ib_copy_path_rec_to_user);
+
+void ib_copy_path_rec_from_user(struct ib_sa_path_rec *dst,
+ struct ib_user_path_rec *src)
+{
+ memcpy(dst->dgid.raw, src->dgid, sizeof dst->dgid);
+ memcpy(dst->sgid.raw, src->sgid, sizeof dst->sgid);
+
+ dst->dlid = src->dlid;
+ dst->slid = src->slid;
+ dst->raw_traffic = src->raw_traffic;
+ dst->flow_label = src->flow_label;
+ dst->hop_limit = src->hop_limit;
+ dst->traffic_class = src->traffic_class;
+ dst->reversible = src->reversible;
+ dst->numb_path = src->numb_path;
+ dst->pkey = src->pkey;
+ dst->sl = src->sl;
+ dst->mtu_selector = src->mtu_selector;
+ dst->mtu = src->mtu;
+ dst->rate_selector = src->rate_selector;
+ dst->rate = src->rate;
+ dst->packet_life_time = src->packet_life_time;
+ dst->preference = src->preference;
+ dst->packet_life_time_selector = src->packet_life_time_selector;
+}
+EXPORT_SYMBOL(ib_copy_path_rec_from_user);
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/ib_marshall.h
linux-2.6.ib/include/rdma/ib_marshall.h
--- linux-2.6.git/include/rdma/ib_marshall.h 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/include/rdma/ib_marshall.h 2006-01-16 15:34:15.000000000 -0800
@@ -0,0 +1,50 @@
+/*
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#if !defined(IB_USER_MARSHALL_H)
+#define IB_USER_MARSHALL_H
+
+#include <rdma/ib_verbs.h>
+#include <rdma/ib_sa.h>
+#include <rdma/ib_user_verbs.h>
+#include <rdma/ib_user_sa.h>
+
+void ib_copy_qp_attr_to_user(struct ib_uverbs_qp_attr *dst,
+ struct ib_qp_attr *src);
+
+void ib_copy_path_rec_to_user(struct ib_user_path_rec *dst,
+ struct ib_sa_path_rec *src);
+
+void ib_copy_path_rec_from_user(struct ib_sa_path_rec *dst,
+ struct ib_user_path_rec *src);
+
+#endif /* IB_USER_MARSHALL_H */
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/ib_user_cm.h
linux-2.6.ib/include/rdma/ib_user_cm.h
--- linux-2.6.git/include/rdma/ib_user_cm.h 2006-01-16 10:26:47.000000000 -0800
+++ linux-2.6.ib/include/rdma/ib_user_cm.h 2006-01-16 15:34:15.000000000 -0800
@@ -30,13 +30,13 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- * $Id: ib_user_cm.h 2576 2005-06-09 17:00:30Z libor $
+ * $Id: ib_user_cm.h 4019 2005-11-11 00:33:09Z sean.hefty $
*/
#ifndef IB_USER_CM_H
#define IB_USER_CM_H
-#include <linux/types.h>
+#include <rdma/ib_user_sa.h>
#define IB_USER_CM_ABI_VERSION 4
@@ -110,58 +110,6 @@ struct ib_ucm_init_qp_attr {
__u32 qp_state;
};
-struct ib_ucm_ah_attr {
- __u8 grh_dgid[16];
- __u32 grh_flow_label;
- __u16 dlid;
- __u16 reserved;
- __u8 grh_sgid_index;
- __u8 grh_hop_limit;
- __u8 grh_traffic_class;
- __u8 sl;
- __u8 src_path_bits;
- __u8 static_rate;
- __u8 is_global;
- __u8 port_num;
-};
-
-struct ib_ucm_init_qp_attr_resp {
- __u32 qp_attr_mask;
- __u32 qp_state;
- __u32 cur_qp_state;
- __u32 path_mtu;
- __u32 path_mig_state;
- __u32 qkey;
- __u32 rq_psn;
- __u32 sq_psn;
- __u32 dest_qp_num;
- __u32 qp_access_flags;
-
- struct ib_ucm_ah_attr ah_attr;
- struct ib_ucm_ah_attr alt_ah_attr;
-
- /* ib_qp_cap */
- __u32 max_send_wr;
- __u32 max_recv_wr;
- __u32 max_send_sge;
- __u32 max_recv_sge;
- __u32 max_inline_data;
-
- __u16 pkey_index;
- __u16 alt_pkey_index;
- __u8 en_sqd_async_notify;
- __u8 sq_draining;
- __u8 max_rd_atomic;
- __u8 max_dest_rd_atomic;
- __u8 min_rnr_timer;
- __u8 port_num;
- __u8 timeout;
- __u8 retry_cnt;
- __u8 rnr_retry;
- __u8 alt_port_num;
- __u8 alt_timeout;
-};
-
struct ib_ucm_listen {
__be64 service_id;
__be64 service_mask;
@@ -180,28 +128,6 @@ struct ib_ucm_private_data {
__u8 reserved[3];
};
-struct ib_ucm_path_rec {
- __u8 dgid[16];
- __u8 sgid[16];
- __be16 dlid;
- __be16 slid;
- __u32 raw_traffic;
- __be32 flow_label;
- __u32 reversible;
- __u32 mtu;
- __be16 pkey;
- __u8 hop_limit;
- __u8 traffic_class;
- __u8 numb_path;
- __u8 sl;
- __u8 mtu_selector;
- __u8 rate_selector;
- __u8 rate;
- __u8 packet_life_time_selector;
- __u8 packet_life_time;
- __u8 preference;
-};
-
struct ib_ucm_req {
__u32 id;
__u32 qpn;
@@ -304,8 +230,8 @@ struct ib_ucm_event_get {
};
struct ib_ucm_req_event_resp {
- struct ib_ucm_path_rec primary_path;
- struct ib_ucm_path_rec alternate_path;
+ struct ib_user_path_rec primary_path;
+ struct ib_user_path_rec alternate_path;
__be64 remote_ca_guid;
__u32 remote_qkey;
__u32 remote_qpn;
@@ -349,7 +275,7 @@ struct ib_ucm_mra_event_resp {
};
struct ib_ucm_lap_event_resp {
- struct ib_ucm_path_rec path;
+ struct ib_user_path_rec path;
};
struct ib_ucm_apr_event_resp {
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/ib_user_sa.h
linux-2.6.ib/include/rdma/ib_user_sa.h
--- linux-2.6.git/include/rdma/ib_user_sa.h 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/include/rdma/ib_user_sa.h 2006-01-16 15:34:15.000000000 -0800
@@ -0,0 +1,60 @@
+/*
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef IB_USER_SA_H
+#define IB_USER_SA_H
+
+#include <linux/types.h>
+
+struct ib_user_path_rec {
+ __u8 dgid[16];
+ __u8 sgid[16];
+ __be16 dlid;
+ __be16 slid;
+ __u32 raw_traffic;
+ __be32 flow_label;
+ __u32 reversible;
+ __u32 mtu;
+ __be16 pkey;
+ __u8 hop_limit;
+ __u8 traffic_class;
+ __u8 numb_path;
+ __u8 sl;
+ __u8 mtu_selector;
+ __u8 rate_selector;
+ __u8 rate;
+ __u8 packet_life_time_selector;
+ __u8 packet_life_time;
+ __u8 preference;
+};
+
+#endif /* IB_USER_SA_H */
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/ib_user_verbs.h
linux-2.6.ib/include/rdma/ib_user_verbs.h
--- linux-2.6.git/include/rdma/ib_user_verbs.h 2006-01-16 10:26:47.000000000 -0800
+++ linux-2.6.ib/include/rdma/ib_user_verbs.h 2006-01-16 15:34:15.000000000 -0800
@@ -31,7 +31,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- * $Id: ib_user_verbs.h 2708 2005-06-24 17:27:21Z roland $
+ * $Id: ib_user_verbs.h 4019 2005-11-11 00:33:09Z sean.hefty $
*/
#ifndef IB_USER_VERBS_H
@@ -311,6 +311,64 @@ struct ib_uverbs_destroy_cq_resp {
__u32 async_events_reported;
};
+struct ib_uverbs_global_route {
+ __u8 dgid[16];
+ __u32 flow_label;
+ __u8 sgid_index;
+ __u8 hop_limit;
+ __u8 traffic_class;
+ __u8 reserved;
+};
+
+struct ib_uverbs_ah_attr {
+ struct ib_uverbs_global_route grh;
+ __u16 dlid;
+ __u8 sl;
+ __u8 src_path_bits;
+ __u8 static_rate;
+ __u8 is_global;
+ __u8 port_num;
+ __u8 reserved;
+};
+
+struct ib_uverbs_qp_attr {
+ __u32 qp_attr_mask;
+ __u32 qp_state;
+ __u32 cur_qp_state;
+ __u32 path_mtu;
+ __u32 path_mig_state;
+ __u32 qkey;
+ __u32 rq_psn;
+ __u32 sq_psn;
+ __u32 dest_qp_num;
+ __u32 qp_access_flags;
+
+ struct ib_uverbs_ah_attr ah_attr;
+ struct ib_uverbs_ah_attr alt_ah_attr;
+
+ /* ib_qp_cap */
+ __u32 max_send_wr;
+ __u32 max_recv_wr;
+ __u32 max_send_sge;
+ __u32 max_recv_sge;
+ __u32 max_inline_data;
+
+ __u16 pkey_index;
+ __u16 alt_pkey_index;
+ __u8 en_sqd_async_notify;
+ __u8 sq_draining;
+ __u8 max_rd_atomic;
+ __u8 max_dest_rd_atomic;
+ __u8 min_rnr_timer;
+ __u8 port_num;
+ __u8 timeout;
+ __u8 retry_cnt;
+ __u8 rnr_retry;
+ __u8 alt_port_num;
+ __u8 alt_timeout;
+ __u8 reserved[5];
+};
+
struct ib_uverbs_create_qp {
__u64 response;
__u64 user_handle;
@@ -487,26 +545,6 @@ struct ib_uverbs_post_srq_recv_resp {
__u32 bad_wr;
};
-struct ib_uverbs_global_route {
- __u8 dgid[16];
- __u32 flow_label;
- __u8 sgid_index;
- __u8 hop_limit;
- __u8 traffic_class;
- __u8 reserved;
-};
-
-struct ib_uverbs_ah_attr {
- struct ib_uverbs_global_route grh;
- __u16 dlid;
- __u8 sl;
- __u8 src_path_bits;
- __u8 static_rate;
- __u8 is_global;
- __u8 port_num;
- __u8 reserved;
-};
-
struct ib_uverbs_create_ah {
__u64 response;
__u64 user_handle;
^ permalink raw reply [flat|nested] 97+ messages in thread* [PATCH 2/5] Infiniband: connection abstraction
2006-02-01 20:03 [PATCH 0/5] Infiniband: connection abstraction Sean Hefty
2006-02-01 20:07 ` [PATCH 1/5] " Sean Hefty
@ 2006-02-01 20:10 ` Sean Hefty
2006-02-01 20:15 ` [PATCH 3/5] " Sean Hefty
` (3 subsequent siblings)
5 siblings, 0 replies; 97+ messages in thread
From: Sean Hefty @ 2006-02-01 20:10 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: openib-general
The following patch extends matching connection requests to listens in the
Infiniband CM to include private data.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
---
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/cm.c
linux-2.6.ib/drivers/infiniband/core/cm.c
--- linux-2.6.git/drivers/infiniband/core/cm.c 2006-01-16 10:25:26.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/cm.c 2006-01-16 16:03:35.000000000 -0800
@@ -32,7 +32,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- * $Id: cm.c 2821 2005-07-08 17:07:28Z sean.hefty $
+ * $Id: cm.c 4311 2005-12-05 18:42:01Z sean.hefty $
*/
#include <linux/dma-mapping.h>
#include <linux/err.h>
@@ -130,6 +130,7 @@ struct cm_id_private {
/* todo: use alternate port on send failure */
struct cm_av av;
struct cm_av alt_av;
+ struct ib_cm_compare_data *compare_data;
void *private_data;
__be64 tid;
@@ -355,6 +356,41 @@ static struct cm_id_private * cm_acquire
return cm_id_priv;
}
+static void cm_mask_copy(u8 *dst, u8 *src, u8 *mask)
+{
+ int i;
+
+ for (i = 0; i < IB_CM_COMPARE_SIZE / sizeof(unsigned long); i++)
+ ((unsigned long *) dst)[i] = ((unsigned long *) src)[i] &
+ ((unsigned long *) mask)[i];
+}
+
+static int cm_compare_data(struct ib_cm_compare_data *src_data,
+ struct ib_cm_compare_data *dst_data)
+{
+ u8 src[IB_CM_COMPARE_SIZE];
+ u8 dst[IB_CM_COMPARE_SIZE];
+
+ if (!src_data || !dst_data)
+ return 0;
+
+ cm_mask_copy(src, src_data->data, dst_data->mask);
+ cm_mask_copy(dst, dst_data->data, src_data->mask);
+ return memcmp(src, dst, IB_CM_COMPARE_SIZE);
+}
+
+static int cm_compare_private_data(u8 *private_data,
+ struct ib_cm_compare_data *dst_data)
+{
+ u8 src[IB_CM_COMPARE_SIZE];
+
+ if (!dst_data)
+ return 0;
+
+ cm_mask_copy(src, private_data, dst_data->mask);
+ return memcmp(src, dst_data->data, IB_CM_COMPARE_SIZE);
+}
+
static struct cm_id_private * cm_insert_listen(struct cm_id_private *cm_id_priv)
{
struct rb_node **link = &cm.listen_service_table.rb_node;
@@ -362,14 +397,18 @@ static struct cm_id_private * cm_insert_
struct cm_id_private *cur_cm_id_priv;
__be64 service_id = cm_id_priv->id.service_id;
__be64 service_mask = cm_id_priv->id.service_mask;
+ int data_cmp;
while (*link) {
parent = *link;
cur_cm_id_priv = rb_entry(parent, struct cm_id_private,
service_node);
+ data_cmp = cm_compare_data(cm_id_priv->compare_data,
+ cur_cm_id_priv->compare_data);
if ((cur_cm_id_priv->id.service_mask & service_id) ==
(service_mask & cur_cm_id_priv->id.service_id) &&
- (cm_id_priv->id.device == cur_cm_id_priv->id.device))
+ (cm_id_priv->id.device == cur_cm_id_priv->id.device) &&
+ !data_cmp)
return cur_cm_id_priv;
if (cm_id_priv->id.device < cur_cm_id_priv->id.device)
@@ -378,6 +417,10 @@ static struct cm_id_private * cm_insert_
link = &(*link)->rb_right;
else if (service_id < cur_cm_id_priv->id.service_id)
link = &(*link)->rb_left;
+ else if (service_id > cur_cm_id_priv->id.service_id)
+ link = &(*link)->rb_right;
+ else if (data_cmp < 0)
+ link = &(*link)->rb_left;
else
link = &(*link)->rb_right;
}
@@ -387,16 +430,20 @@ static struct cm_id_private * cm_insert_
}
static struct cm_id_private * cm_find_listen(struct ib_device *device,
- __be64 service_id)
+ __be64 service_id,
+ u8 *private_data)
{
struct rb_node *node = cm.listen_service_table.rb_node;
struct cm_id_private *cm_id_priv;
+ int data_cmp;
while (node) {
cm_id_priv = rb_entry(node, struct cm_id_private, service_node);
+ data_cmp = cm_compare_private_data(private_data,
+ cm_id_priv->compare_data);
if ((cm_id_priv->id.service_mask & service_id) ==
cm_id_priv->id.service_id &&
- (cm_id_priv->id.device == device))
+ (cm_id_priv->id.device == device) && !data_cmp)
return cm_id_priv;
if (device < cm_id_priv->id.device)
@@ -405,6 +452,10 @@ static struct cm_id_private * cm_find_li
node = node->rb_right;
else if (service_id < cm_id_priv->id.service_id)
node = node->rb_left;
+ else if (service_id > cm_id_priv->id.service_id)
+ node = node->rb_right;
+ else if (data_cmp < 0)
+ node = node->rb_left;
else
node = node->rb_right;
}
@@ -728,15 +779,14 @@ retest:
wait_event(cm_id_priv->wait, !atomic_read(&cm_id_priv->refcount));
while ((work = cm_dequeue_work(cm_id_priv)) != NULL)
cm_free_work(work);
- if (cm_id_priv->private_data && cm_id_priv->private_data_len)
- kfree(cm_id_priv->private_data);
+ kfree(cm_id_priv->compare_data);
+ kfree(cm_id_priv->private_data);
kfree(cm_id_priv);
}
EXPORT_SYMBOL(ib_destroy_cm_id);
-int ib_cm_listen(struct ib_cm_id *cm_id,
- __be64 service_id,
- __be64 service_mask)
+int ib_cm_listen(struct ib_cm_id *cm_id, __be64 service_id, __be64 service_mask,
+ struct ib_cm_compare_data *compare_data)
{
struct cm_id_private *cm_id_priv, *cur_cm_id_priv;
unsigned long flags;
@@ -750,7 +800,19 @@ int ib_cm_listen(struct ib_cm_id *cm_id,
return -EINVAL;
cm_id_priv = container_of(cm_id, struct cm_id_private, id);
- BUG_ON(cm_id->state != IB_CM_IDLE);
+ if (cm_id->state != IB_CM_IDLE)
+ return -EINVAL;
+
+ if (compare_data) {
+ cm_id_priv->compare_data = kzalloc(sizeof *compare_data,
+ GFP_KERNEL);
+ if (!cm_id_priv->compare_data)
+ return -ENOMEM;
+ cm_mask_copy(cm_id_priv->compare_data->data,
+ compare_data->data, compare_data->mask);
+ memcpy(cm_id_priv->compare_data->mask, compare_data->mask,
+ IB_CM_COMPARE_SIZE);
+ }
cm_id->state = IB_CM_LISTEN;
@@ -767,6 +829,8 @@ int ib_cm_listen(struct ib_cm_id *cm_id,
if (cur_cm_id_priv) {
cm_id->state = IB_CM_IDLE;
+ kfree(cm_id_priv->compare_data);
+ cm_id_priv->compare_data = NULL;
ret = -EBUSY;
}
return ret;
@@ -1239,7 +1303,8 @@ static struct cm_id_private * cm_match_r
/* Find matching listen request. */
listen_cm_id_priv = cm_find_listen(cm_id_priv->id.device,
- req_msg->service_id);
+ req_msg->service_id,
+ req_msg->private_data);
if (!listen_cm_id_priv) {
spin_unlock_irqrestore(&cm.lock, flags);
cm_issue_rej(work->port, work->mad_recv_wc,
@@ -2646,7 +2711,8 @@ static int cm_sidr_req_handler(struct cm
goto out; /* Duplicate message. */
}
cur_cm_id_priv = cm_find_listen(cm_id->device,
- sidr_req_msg->service_id);
+ sidr_req_msg->service_id,
+ sidr_req_msg->private_data);
if (!cur_cm_id_priv) {
rb_erase(&cm_id_priv->sidr_id_node, &cm.remote_sidr_table);
spin_unlock_irqrestore(&cm.lock, flags);
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/ucm.c
linux-2.6.ib/drivers/infiniband/core/ucm.c
--- linux-2.6.git/drivers/infiniband/core/ucm.c 2006-01-16 16:03:08.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/ucm.c 2006-01-16 16:03:35.000000000 -0800
@@ -646,6 +646,17 @@ out:
return result;
}
+static int ucm_validate_listen(__be64 service_id, __be64 service_mask)
+{
+ service_id &= service_mask;
+
+ if (((service_id & IB_CMA_SERVICE_ID_MASK) == IB_CMA_SERVICE_ID) ||
+ ((service_id & IB_SDP_SERVICE_ID_MASK) == IB_SDP_SERVICE_ID))
+ return -EINVAL;
+
+ return 0;
+}
+
static ssize_t ib_ucm_listen(struct ib_ucm_file *file,
const char __user *inbuf,
int in_len, int out_len)
@@ -661,7 +672,13 @@ static ssize_t ib_ucm_listen(struct ib_u
if (IS_ERR(ctx))
return PTR_ERR(ctx);
- result = ib_cm_listen(ctx->cm_id, cmd.service_id, cmd.service_mask);
+ result = ucm_validate_listen(cmd.service_id, cmd.service_mask);
+ if (result)
+ goto out;
+
+ result = ib_cm_listen(ctx->cm_id, cmd.service_id, cmd.service_mask,
+ NULL);
+out:
ib_ucm_ctx_put(ctx);
return result;
}
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/ib_cm.h
linux-2.6.ib/include/rdma/ib_cm.h
--- linux-2.6.git/include/rdma/ib_cm.h 2006-01-16 10:26:47.000000000 -0800
+++ linux-2.6.ib/include/rdma/ib_cm.h 2006-01-16 16:03:35.000000000 -0800
@@ -32,7 +32,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- * $Id: ib_cm.h 2730 2005-06-28 16:43:03Z sean.hefty $
+ * $Id: ib_cm.h 4311 2005-12-05 18:42:01Z sean.hefty $
*/
#if !defined(IB_CM_H)
#define IB_CM_H
@@ -102,7 +102,8 @@ enum ib_cm_data_size {
IB_CM_APR_INFO_LENGTH = 72,
IB_CM_SIDR_REQ_PRIVATE_DATA_SIZE = 216,
IB_CM_SIDR_REP_PRIVATE_DATA_SIZE = 136,
- IB_CM_SIDR_REP_INFO_LENGTH = 72
+ IB_CM_SIDR_REP_INFO_LENGTH = 72,
+ IB_CM_COMPARE_SIZE = 64
};
struct ib_cm_id;
@@ -238,7 +239,6 @@ struct ib_cm_sidr_rep_event_param {
u32 qpn;
void *info;
u8 info_len;
-
};
struct ib_cm_event {
@@ -317,6 +317,15 @@ void ib_destroy_cm_id(struct ib_cm_id *c
#define IB_SERVICE_ID_AGN_MASK __constant_cpu_to_be64(0xFF00000000000000ULL)
#define IB_CM_ASSIGN_SERVICE_ID __constant_cpu_to_be64(0x0200000000000000ULL)
+#define IB_CMA_SERVICE_ID __constant_cpu_to_be64(0x0000000001000000ULL)
+#define IB_CMA_SERVICE_ID_MASK __constant_cpu_to_be64(0xFFFFFFFFFF000000ULL)
+#define IB_SDP_SERVICE_ID __constant_cpu_to_be64(0x0000000000010000ULL)
+#define IB_SDP_SERVICE_ID_MASK __constant_cpu_to_be64(0xFFFFFFFFFFFF0000ULL)
+
+struct ib_cm_compare_data {
+ u8 data[IB_CM_COMPARE_SIZE];
+ u8 mask[IB_CM_COMPARE_SIZE];
+};
/**
* ib_cm_listen - Initiates listening on the specified service ID for
@@ -330,10 +339,12 @@ void ib_destroy_cm_id(struct ib_cm_id *c
* range of service IDs. If set to 0, the service ID is matched
* exactly. This parameter is ignored if %service_id is set to
* IB_CM_ASSIGN_SERVICE_ID.
+ * @compare_data: This parameter is optional. It specifies data that must
+ * appear in the private data of a connection request for the specified
+ * listen request.
*/
-int ib_cm_listen(struct ib_cm_id *cm_id,
- __be64 service_id,
- __be64 service_mask);
+int ib_cm_listen(struct ib_cm_id *cm_id, __be64 service_id, __be64 service_mask,
+ struct ib_cm_compare_data *compare_data);
struct ib_cm_req_param {
struct ib_sa_path_rec *primary_path;
^ permalink raw reply [flat|nested] 97+ messages in thread* [PATCH 3/5] Infiniband: connection abstraction
2006-02-01 20:03 [PATCH 0/5] Infiniband: connection abstraction Sean Hefty
2006-02-01 20:07 ` [PATCH 1/5] " Sean Hefty
2006-02-01 20:10 ` [PATCH 2/5] " Sean Hefty
@ 2006-02-01 20:15 ` Sean Hefty
2006-03-03 21:14 ` [PATCH 3/5] export of ip_dev_find as part of Infiniband " Sean Hefty
2006-02-01 20:18 ` [PATCH 4/5] Infiniband: " Sean Hefty
` (2 subsequent siblings)
5 siblings, 1 reply; 97+ messages in thread
From: Sean Hefty @ 2006-02-01 20:15 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: openib-general
The following provides an address translation service that maps IP addresses
to Infiniband addresses (GIDs) using IPoIB.
This patch exports ip_dev_find() to locate a net_device given an IP address.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
---
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/addr.c
linux-2.6.ib/drivers/infiniband/core/addr.c
--- linux-2.6.git/drivers/infiniband/core/addr.c 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/addr.c 2006-01-16 16:14:24.000000000 -0800
@@ -0,0 +1,356 @@
+/*
+ * Copyright (c) 2005 Voltaire Inc. All rights reserved.
+ * Copyright (c) 2002-2005, Network Appliance, Inc. All rights reserved.
+ * Copyright (c) 1999-2005, Mellanox Technologies, Inc. All rights reserved.
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ */
+#include <linux/inetdevice.h>
+#include <linux/workqueue.h>
+#include <net/arp.h>
+#include <net/neighbour.h>
+#include <net/route.h>
+#include <rdma/ib_addr.h>
+
+MODULE_AUTHOR("Sean Hefty");
+MODULE_DESCRIPTION("IB Address Translation");
+MODULE_LICENSE("Dual BSD/GPL");
+
+struct addr_req {
+ struct list_head list;
+ struct sockaddr src_addr;
+ struct sockaddr dst_addr;
+ struct rdma_dev_addr *addr;
+ void *context;
+ void (*callback)(int status, struct sockaddr *src_addr,
+ struct rdma_dev_addr *addr, void *context);
+ unsigned long timeout;
+ int status;
+};
+
+static void process_req(void *data);
+
+static DEFINE_MUTEX(lock);
+static LIST_HEAD(req_list);
+static DECLARE_WORK(work, process_req, NULL);
+struct workqueue_struct *rdma_wq;
+EXPORT_SYMBOL(rdma_wq);
+
+static int copy_addr(struct rdma_dev_addr *dev_addr, struct net_device *dev,
+ unsigned char *dst_dev_addr)
+{
+ switch (dev->type) {
+ case ARPHRD_INFINIBAND:
+ dev_addr->dev_type = IB_NODE_CA;
+ break;
+ default:
+ return -EADDRNOTAVAIL;
+ }
+
+ memcpy(dev_addr->src_dev_addr, dev->dev_addr, MAX_ADDR_LEN);
+ memcpy(dev_addr->broadcast, dev->broadcast, MAX_ADDR_LEN);
+ if (dst_dev_addr)
+ memcpy(dev_addr->dst_dev_addr, dst_dev_addr, MAX_ADDR_LEN);
+ return 0;
+}
+
+int rdma_translate_ip(struct sockaddr *addr, struct rdma_dev_addr *dev_addr)
+{
+ struct net_device *dev;
+ u32 ip = ((struct sockaddr_in *) addr)->sin_addr.s_addr;
+ int ret;
+
+ dev = ip_dev_find(ip);
+ if (!dev)
+ return -EADDRNOTAVAIL;
+
+ ret = copy_addr(dev_addr, dev, NULL);
+ dev_put(dev);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_translate_ip);
+
+static void set_timeout(unsigned long time)
+{
+ unsigned long delay;
+
+ cancel_delayed_work(&work);
+
+ delay = time - jiffies;
+ if ((long)delay <= 0)
+ delay = 1;
+
+ queue_delayed_work(rdma_wq, &work, delay);
+}
+
+static void queue_req(struct addr_req *req)
+{
+ struct addr_req *temp_req;
+
+ mutex_lock(&lock);
+ list_for_each_entry_reverse(temp_req, &req_list, list) {
+ if (time_after(req->timeout, temp_req->timeout))
+ break;
+ }
+
+ list_add(&req->list, &temp_req->list);
+
+ if (req_list.next == &req->list)
+ set_timeout(req->timeout);
+ mutex_unlock(&lock);
+}
+
+static void addr_send_arp(struct sockaddr_in *dst_in)
+{
+ struct rtable *rt;
+ struct flowi fl;
+ u32 dst_ip = dst_in->sin_addr.s_addr;
+
+ memset(&fl, 0, sizeof fl);
+ fl.nl_u.ip4_u.daddr = dst_ip;
+ if (ip_route_output_key(&rt, &fl))
+ return;
+
+ arp_send(ARPOP_REQUEST, ETH_P_ARP, rt->rt_gateway, rt->idev->dev,
+ rt->rt_src, NULL, rt->idev->dev->dev_addr, NULL);
+ ip_rt_put(rt);
+}
+
+static int addr_resolve_remote(struct sockaddr_in *src_in,
+ struct sockaddr_in *dst_in,
+ struct rdma_dev_addr *addr)
+{
+ u32 src_ip = src_in->sin_addr.s_addr;
+ u32 dst_ip = dst_in->sin_addr.s_addr;
+ struct flowi fl;
+ struct rtable *rt;
+ struct neighbour *neigh;
+ int ret;
+
+ memset(&fl, 0, sizeof fl);
+ fl.nl_u.ip4_u.daddr = dst_ip;
+ fl.nl_u.ip4_u.saddr = src_ip;
+ ret = ip_route_output_key(&rt, &fl);
+ if (ret)
+ goto out;
+
+ neigh = neigh_lookup(&arp_tbl, &rt->rt_gateway, rt->idev->dev);
+ if (!neigh) {
+ ret = -ENODATA;
+ goto err1;
+ }
+
+ if (!(neigh->nud_state & NUD_VALID)) {
+ ret = -ENODATA;
+ goto err2;
+ }
+
+ if (!src_ip) {
+ src_in->sin_family = dst_in->sin_family;
+ src_in->sin_addr.s_addr = rt->rt_src;
+ }
+
+ ret = copy_addr(addr, neigh->dev, neigh->ha);
+err2:
+ neigh_release(neigh);
+err1:
+ ip_rt_put(rt);
+out:
+ return ret;
+}
+
+static void process_req(void *data)
+{
+ struct addr_req *req, *temp_req;
+ struct sockaddr_in *src_in, *dst_in;
+ struct list_head done_list;
+
+ INIT_LIST_HEAD(&done_list);
+
+ mutex_lock(&lock);
+ list_for_each_entry_safe(req, temp_req, &req_list, list) {
+ if (req->status) {
+ src_in = (struct sockaddr_in *) &req->src_addr;
+ dst_in = (struct sockaddr_in *) &req->dst_addr;
+ req->status = addr_resolve_remote(src_in, dst_in,
+ req->addr);
+ }
+ if (req->status && time_after(jiffies, req->timeout))
+ req->status = -ETIMEDOUT;
+ else if (req->status == -ENODATA)
+ continue;
+
+ list_del(&req->list);
+ list_add_tail(&req->list, &done_list);
+ }
+
+ if (!list_empty(&req_list)) {
+ req = list_entry(req_list.next, struct addr_req, list);
+ set_timeout(req->timeout);
+ }
+ mutex_unlock(&lock);
+
+ list_for_each_entry_safe(req, temp_req, &done_list, list) {
+ list_del(&req->list);
+ req->callback(req->status, &req->src_addr, req->addr,
+ req->context);
+ kfree(req);
+ }
+}
+
+static int addr_resolve_local(struct sockaddr_in *src_in,
+ struct sockaddr_in *dst_in,
+ struct rdma_dev_addr *addr)
+{
+ struct net_device *dev;
+ u32 src_ip = src_in->sin_addr.s_addr;
+ u32 dst_ip = dst_in->sin_addr.s_addr;
+ int ret;
+
+ dev = ip_dev_find(dst_ip);
+ if (!dev)
+ return -EADDRNOTAVAIL;
+
+ if (!src_ip) {
+ src_in->sin_family = dst_in->sin_family;
+ src_in->sin_addr.s_addr = dst_ip;
+ ret = copy_addr(addr, dev, dev->dev_addr);
+ } else {
+ ret = rdma_translate_ip((struct sockaddr *)src_in, addr);
+ if (!ret)
+ memcpy(addr->dst_dev_addr, dev->dev_addr, MAX_ADDR_LEN);
+ }
+
+ dev_put(dev);
+ return ret;
+}
+
+int rdma_resolve_ip(struct sockaddr *src_addr, struct sockaddr *dst_addr,
+ struct rdma_dev_addr *addr, int timeout_ms,
+ void (*callback)(int status, struct sockaddr *src_addr,
+ struct rdma_dev_addr *addr, void *context),
+ void *context)
+{
+ struct sockaddr_in *src_in, *dst_in;
+ struct addr_req *req;
+ int ret = 0;
+
+ req = kmalloc(sizeof *req, GFP_KERNEL);
+ if (!req)
+ return -ENOMEM;
+ memset(req, 0, sizeof *req);
+
+ if (src_addr)
+ memcpy(&req->src_addr, src_addr, ip_addr_size(src_addr));
+ memcpy(&req->dst_addr, dst_addr, ip_addr_size(dst_addr));
+ req->addr = addr;
+ req->callback = callback;
+ req->context = context;
+
+ src_in = (struct sockaddr_in *) &req->src_addr;
+ dst_in = (struct sockaddr_in *) &req->dst_addr;
+
+ req->status = addr_resolve_local(src_in, dst_in, addr);
+ if (req->status == -EADDRNOTAVAIL)
+ req->status = addr_resolve_remote(src_in, dst_in, addr);
+
+ switch (req->status) {
+ case 0:
+ req->timeout = jiffies;
+ queue_req(req);
+ break;
+ case -ENODATA:
+ req->timeout = msecs_to_jiffies(timeout_ms) + jiffies;
+ queue_req(req);
+ addr_send_arp(dst_in);
+ break;
+ default:
+ ret = req->status;
+ kfree(req);
+ break;
+ }
+ return ret;
+}
+EXPORT_SYMBOL(rdma_resolve_ip);
+
+void rdma_addr_cancel(struct rdma_dev_addr *addr)
+{
+ struct addr_req *req, *temp_req;
+
+ mutex_lock(&lock);
+ list_for_each_entry_safe(req, temp_req, &req_list, list) {
+ if (req->addr == addr) {
+ req->status = -ECANCELED;
+ req->timeout = jiffies;
+ list_del(&req->list);
+ list_add(&req->list, &req_list);
+ set_timeout(req->timeout);
+ break;
+ }
+ }
+ mutex_unlock(&lock);
+}
+EXPORT_SYMBOL(rdma_addr_cancel);
+
+static int addr_arp_recv(struct sk_buff *skb, struct net_device *dev,
+ struct packet_type *pkt, struct net_device *orig_dev)
+{
+ struct arphdr *arp_hdr;
+
+ arp_hdr = (struct arphdr *) skb->nh.raw;
+
+ if (dev->type == ARPHRD_INFINIBAND &&
+ (arp_hdr->ar_op == __constant_htons(ARPOP_REQUEST) ||
+ arp_hdr->ar_op == __constant_htons(ARPOP_REPLY)))
+ set_timeout(jiffies);
+
+ kfree_skb(skb);
+ return 0;
+}
+
+static struct packet_type addr_arp = {
+ .type = __constant_htons(ETH_P_ARP),
+ .func = addr_arp_recv,
+ .af_packet_priv = (void*) 1,
+};
+
+static int addr_init(void)
+{
+ rdma_wq = create_singlethread_workqueue("rdma_wq");
+ if (!rdma_wq)
+ return -ENOMEM;
+
+ dev_add_pack(&addr_arp);
+ return 0;
+}
+
+static void addr_cleanup(void)
+{
+ dev_remove_pack(&addr_arp);
+ destroy_workqueue(rdma_wq);
+}
+
+module_init(addr_init);
+module_exit(addr_cleanup);
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/Makefile
linux-2.6.ib/drivers/infiniband/core/Makefile
--- linux-2.6.git/drivers/infiniband/core/Makefile 2006-01-16 16:03:08.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/Makefile 2006-01-16 16:14:24.000000000 -0800
@@ -1,5 +1,5 @@
obj-$(CONFIG_INFINIBAND) += ib_core.o ib_mad.o ib_sa.o \
- ib_cm.o
+ ib_cm.o ib_addr.o
obj-$(CONFIG_INFINIBAND_USER_MAD) += ib_umad.o
obj-$(CONFIG_INFINIBAND_USER_ACCESS) += ib_uverbs.o ib_ucm.o
@@ -12,6 +12,8 @@ ib_sa-y := sa_query.o
ib_cm-y := cm.o
+ib_addr-y := addr.o
+
ib_umad-y := user_mad.o
ib_ucm-y := ucm.o
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/ib_addr.h
linux-2.6.ib/include/rdma/ib_addr.h
--- linux-2.6.git/include/rdma/ib_addr.h 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/include/rdma/ib_addr.h 2006-01-16 16:14:24.000000000 -0800
@@ -0,0 +1,117 @@
+/*
+ * Copyright (c) 2005 Voltaire Inc. All rights reserved.
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ *
+ */
+
+#if !defined(IB_ADDR_H)
+#define IB_ADDR_H
+
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/netdevice.h>
+#include <linux/socket.h>
+#include <rdma/ib_verbs.h>
+
+extern struct workqueue_struct *rdma_wq;
+
+struct rdma_dev_addr {
+ unsigned char src_dev_addr[MAX_ADDR_LEN];
+ unsigned char dst_dev_addr[MAX_ADDR_LEN];
+ unsigned char broadcast[MAX_ADDR_LEN];
+ enum ib_node_type dev_type;
+};
+
+/**
+ * rdma_translate_ip - Translate a local IP address to an RDMA hardware
+ * address.
+ */
+int rdma_translate_ip(struct sockaddr *addr, struct rdma_dev_addr *dev_addr);
+
+/**
+ * rdma_resolve_ip - Resolve source and destination IP addresses to
+ * RDMA hardware addresses.
+ * @src_addr: An optional source address to use in the resolution. If a
+ * source address is not provided, a usable address will be returned via
+ * the callback.
+ * @dst_addr: The destination address to resolve.
+ * @addr: A reference to a data location that will receive the resolved
+ * addresses. The data location must remain valid until the callback has
+ * been invoked.
+ * @timeout_ms: Amount of time to wait for the address resolution to complete.
+ * @callback: Call invoked once address resolution has completed, timed out,
+ * or been canceled. A status of 0 indicates success.
+ * @context: User-specified context associated with the call.
+ */
+int rdma_resolve_ip(struct sockaddr *src_addr, struct sockaddr *dst_addr,
+ struct rdma_dev_addr *addr, int timeout_ms,
+ void (*callback)(int status, struct sockaddr *src_addr,
+ struct rdma_dev_addr *addr, void *context),
+ void *context);
+
+void rdma_addr_cancel(struct rdma_dev_addr *addr);
+
+static inline int ip_addr_size(struct sockaddr *addr)
+{
+ return addr->sa_family == AF_INET6 ?
+ sizeof(struct sockaddr_in6) : sizeof(struct sockaddr_in);
+}
+
+static inline u16 ib_addr_get_pkey(struct rdma_dev_addr *dev_addr)
+{
+ return ((u16)dev_addr->broadcast[8] << 8) | (u16)dev_addr->broadcast[9];
+}
+
+static inline void ib_addr_set_pkey(struct rdma_dev_addr *dev_addr, u16 pkey)
+{
+ dev_addr->broadcast[8] = pkey >> 8;
+ dev_addr->broadcast[9] = (unsigned char) pkey;
+}
+
+static inline union ib_gid* ib_addr_get_sgid(struct rdma_dev_addr *dev_addr)
+{
+ return (union ib_gid *) (dev_addr->src_dev_addr + 4);
+}
+
+static inline void ib_addr_set_sgid(struct rdma_dev_addr *dev_addr,
+ union ib_gid *gid)
+{
+ memcpy(dev_addr->src_dev_addr + 4, gid, sizeof *gid);
+}
+
+static inline union ib_gid* ib_addr_get_dgid(struct rdma_dev_addr *dev_addr)
+{
+ return (union ib_gid *) (dev_addr->dst_dev_addr + 4);
+}
+
+static inline void ib_addr_set_dgid(struct rdma_dev_addr *dev_addr,
+ union ib_gid *gid)
+{
+ memcpy(dev_addr->dst_dev_addr + 4, gid, sizeof *gid);
+}
+
+#endif /* IB_ADDR_H */
+
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/net/ipv4/fib_frontend.c
linux-2.6.ib/net/ipv4/fib_frontend.c
--- linux-2.6.git/net/ipv4/fib_frontend.c 2006-01-16 10:28:29.000000000 -0800
+++ linux-2.6.ib/net/ipv4/fib_frontend.c 2006-01-16 16:14:24.000000000 -0800
@@ -666,4 +666,5 @@ void __init ip_fib_init(void)
}
EXPORT_SYMBOL(inet_addr_type);
+EXPORT_SYMBOL(ip_dev_find);
EXPORT_SYMBOL(ip_rt_ioctl);
^ permalink raw reply [flat|nested] 97+ messages in thread* [PATCH 3/5] export of ip_dev_find as part of Infiniband connection abstraction
2006-02-01 20:15 ` [PATCH 3/5] " Sean Hefty
@ 2006-03-03 21:14 ` Sean Hefty
0 siblings, 0 replies; 97+ messages in thread
From: Sean Hefty @ 2006-03-03 21:14 UTC (permalink / raw)
To: Hefty, Sean, netdev, linux-kernel; +Cc: openib-general
I wanted to make doubly sure that this didn't get lost in the patch series, but
ip_dev_find() is re-exported. The use is shown below.
- Sean
>+int rdma_translate_ip(struct sockaddr *addr, struct rdma_dev_addr *dev_addr)
>+{
>+ struct net_device *dev;
>+ u32 ip = ((struct sockaddr_in *) addr)->sin_addr.s_addr;
>+ int ret;
>+
>+ dev = ip_dev_find(ip);
>+ if (!dev)
>+ return -EADDRNOTAVAIL;
>+
>+ ret = copy_addr(dev_addr, dev, NULL);
>+ dev_put(dev);
>+ return ret;
>+}
{snip}
>+static int addr_resolve_local(struct sockaddr_in *src_in,
>+ struct sockaddr_in *dst_in,
>+ struct rdma_dev_addr *addr)
>+{
>+ struct net_device *dev;
>+ u32 src_ip = src_in->sin_addr.s_addr;
>+ u32 dst_ip = dst_in->sin_addr.s_addr;
>+ int ret;
>+
>+ dev = ip_dev_find(dst_ip);
>+ if (!dev)
>+ return -EADDRNOTAVAIL;
>+
>+ if (!src_ip) {
>+ src_in->sin_family = dst_in->sin_family;
>+ src_in->sin_addr.s_addr = dst_ip;
>+ ret = copy_addr(addr, dev, dev->dev_addr);
>+ } else {
>+ ret = rdma_translate_ip((struct sockaddr *)src_in, addr);
>+ if (!ret)
>+ memcpy(addr->dst_dev_addr, dev->dev_addr, MAX_ADDR_LEN);
>+ }
>+
>+ dev_put(dev);
>+ return ret;
>+}
{snip}
>diff -uprN -X linux-2.6.git/Documentation/dontdiff
>linux-2.6.git/net/ipv4/fib_frontend.c
>linux-2.6.ib/net/ipv4/fib_frontend.c
>--- linux-2.6.git/net/ipv4/fib_frontend.c 2006-01-16 10:28:29.000000000
-0800
>+++ linux-2.6.ib/net/ipv4/fib_frontend.c 2006-01-16 16:14:24.000000000
-0800
>@@ -666,4 +666,5 @@ void __init ip_fib_init(void)
> }
>
> EXPORT_SYMBOL(inet_addr_type);
>+EXPORT_SYMBOL(ip_dev_find);
> EXPORT_SYMBOL(ip_rt_ioctl);
^ permalink raw reply [flat|nested] 97+ messages in thread
* [PATCH 4/5] Infiniband: connection abstraction
2006-02-01 20:03 [PATCH 0/5] Infiniband: connection abstraction Sean Hefty
` (2 preceding siblings ...)
2006-02-01 20:15 ` [PATCH 3/5] " Sean Hefty
@ 2006-02-01 20:18 ` Sean Hefty
2006-02-01 20:19 ` [PATCH 5/5] " Sean Hefty
2006-03-03 21:13 ` [PATCH 0/5] " Sean Hefty
5 siblings, 0 replies; 97+ messages in thread
From: Sean Hefty @ 2006-02-01 20:18 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: openib-general
The following patch implements a kernel mode connection management agent
over Infiniband that connects based on IP addresses.
The agent defines a generic RDMA connection abstraction to support clients
wanting to connect over different RDMA devices.
It also handles RDMA device hotplug events on behalf of clients.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
---
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/cma.c
linux-2.6.ib/drivers/infiniband/core/cma.c
--- linux-2.6.git/drivers/infiniband/core/cma.c 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/cma.c 2006-01-16 16:17:34.000000000 -0800
@@ -0,0 +1,1639 @@
+/*
+ * Copyright (c) 2005 Voltaire Inc. All rights reserved.
+ * Copyright (c) 2002-2005, Network Appliance, Inc. All rights reserved.
+ * Copyright (c) 1999-2005, Mellanox Technologies, Inc. All rights reserved.
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ *
+ */
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/random.h>
+#include <rdma/rdma_cm.h>
+#include <rdma/ib_cache.h>
+#include <rdma/ib_cm.h>
+#include <rdma/ib_sa.h>
+
+MODULE_AUTHOR("Sean Hefty");
+MODULE_DESCRIPTION("Generic RDMA CM Agent");
+MODULE_LICENSE("Dual BSD/GPL");
+
+#define CMA_CM_RESPONSE_TIMEOUT 20
+#define CMA_MAX_CM_RETRIES 3
+
+static void cma_add_one(struct ib_device *device);
+static void cma_remove_one(struct ib_device *device);
+
+static struct ib_client cma_client = {
+ .name = "cma",
+ .add = cma_add_one,
+ .remove = cma_remove_one
+};
+
+static LIST_HEAD(dev_list);
+static LIST_HEAD(listen_any_list);
+static DEFINE_MUTEX(lock);
+
+struct cma_device {
+ struct list_head list;
+ struct ib_device *device;
+ __be64 node_guid;
+ wait_queue_head_t wait;
+ atomic_t refcount;
+ struct list_head id_list;
+};
+
+enum cma_state {
+ CMA_IDLE,
+ CMA_ADDR_QUERY,
+ CMA_ADDR_RESOLVED,
+ CMA_ROUTE_QUERY,
+ CMA_ROUTE_RESOLVED,
+ CMA_CONNECT,
+ CMA_ADDR_BOUND,
+ CMA_LISTEN,
+ CMA_DEVICE_REMOVAL,
+ CMA_DESTROYING
+};
+
+/*
+ * Device removal can occur at anytime, so we need extra handling to
+ * serialize notifying the user of device removal with other callbacks.
+ * We do this by disabling removal notification while a callback is in process,
+ * and reporting it after the callback completes.
+ */
+struct rdma_id_private {
+ struct rdma_cm_id id;
+
+ struct list_head list;
+ struct list_head listen_list;
+ struct cma_device *cma_dev;
+
+ enum cma_state state;
+ spinlock_t lock;
+ wait_queue_head_t wait;
+ atomic_t refcount;
+ wait_queue_head_t wait_remove;
+ atomic_t dev_remove;
+
+ int backlog;
+ int timeout_ms;
+ struct ib_sa_query *query;
+ int query_id;
+ struct ib_cm_id *cm_id;
+
+ u32 seq_num;
+ u32 qp_num;
+ enum ib_qp_type qp_type;
+ u8 srq;
+};
+
+struct cma_work {
+ struct work_struct work;
+ struct rdma_id_private *id;
+};
+
+union cma_ip_addr {
+ struct in6_addr ip6;
+ struct {
+ __u32 pad[3];
+ __u32 addr;
+ } ip4;
+};
+
+struct cma_hdr {
+ u8 cma_version;
+ u8 ip_version; /* IP version: 7:4 */
+ __u16 port;
+ union cma_ip_addr src_addr;
+ union cma_ip_addr dst_addr;
+};
+
+struct sdp_hh {
+ u8 sdp_version;
+ u8 ip_version; /* IP version: 7:4 */
+ u8 sdp_specific1[10];
+ __u16 port;
+ __u16 sdp_specific2;
+ union cma_ip_addr src_addr;
+ union cma_ip_addr dst_addr;
+};
+
+#define CMA_VERSION 0x00
+#define SDP_VERSION 0x22
+
+static int cma_comp(struct rdma_id_private *id_priv, enum cma_state comp)
+{
+ unsigned long flags;
+ int ret;
+
+ spin_lock_irqsave(&id_priv->lock, flags);
+ ret = (id_priv->state == comp);
+ spin_unlock_irqrestore(&id_priv->lock, flags);
+ return ret;
+}
+
+static int cma_comp_exch(struct rdma_id_private *id_priv,
+ enum cma_state comp, enum cma_state exch)
+{
+ unsigned long flags;
+ int ret;
+
+ spin_lock_irqsave(&id_priv->lock, flags);
+ if ((ret = (id_priv->state == comp)))
+ id_priv->state = exch;
+ spin_unlock_irqrestore(&id_priv->lock, flags);
+ return ret;
+}
+
+static enum cma_state cma_exch(struct rdma_id_private *id_priv,
+ enum cma_state exch)
+{
+ unsigned long flags;
+ enum cma_state old;
+
+ spin_lock_irqsave(&id_priv->lock, flags);
+ old = id_priv->state;
+ id_priv->state = exch;
+ spin_unlock_irqrestore(&id_priv->lock, flags);
+ return old;
+}
+
+static inline u8 cma_get_ip_ver(struct cma_hdr *hdr)
+{
+ return hdr->ip_version >> 4;
+}
+
+static inline void cma_set_ip_ver(struct cma_hdr *hdr, u8 ip_ver)
+{
+ hdr->ip_version = (ip_ver << 4) | (hdr->ip_version & 0xF);
+}
+
+static inline u8 sdp_get_ip_ver(struct sdp_hh *hh)
+{
+ return hh->ip_version >> 4;
+}
+
+static inline void sdp_set_ip_ver(struct sdp_hh *hh, u8 ip_ver)
+{
+ hh->ip_version = (ip_ver << 4) | (hh->ip_version & 0xF);
+}
+
+static void cma_attach_to_dev(struct rdma_id_private *id_priv,
+ struct cma_device *cma_dev)
+{
+ atomic_inc(&cma_dev->refcount);
+ id_priv->cma_dev = cma_dev;
+ id_priv->id.device = cma_dev->device;
+ list_add_tail(&id_priv->list, &cma_dev->id_list);
+}
+
+static void cma_detach_from_dev(struct rdma_id_private *id_priv)
+{
+ list_del(&id_priv->list);
+ if (atomic_dec_and_test(&id_priv->cma_dev->refcount))
+ wake_up(&id_priv->cma_dev->wait);
+ id_priv->cma_dev = NULL;
+}
+
+static int cma_acquire_ib_dev(struct rdma_id_private *id_priv)
+{
+ struct cma_device *cma_dev;
+ union ib_gid *gid;
+ int ret = -ENODEV;
+
+ gid = ib_addr_get_sgid(&id_priv->id.route.addr.dev_addr);
+
+ mutex_lock(&lock);
+ list_for_each_entry(cma_dev, &dev_list, list) {
+ ret = ib_find_cached_gid(cma_dev->device, gid,
+ &id_priv->id.port_num, NULL);
+ if (!ret) {
+ cma_attach_to_dev(id_priv, cma_dev);
+ break;
+ }
+ }
+ mutex_unlock(&lock);
+ return ret;
+}
+
+static int cma_acquire_dev(struct rdma_id_private *id_priv)
+{
+ switch (id_priv->id.route.addr.dev_addr.dev_type) {
+ case IB_NODE_CA:
+ return cma_acquire_ib_dev(id_priv);
+ default:
+ return -ENODEV;
+ }
+}
+
+static void cma_deref_id(struct rdma_id_private *id_priv)
+{
+ if (atomic_dec_and_test(&id_priv->refcount))
+ wake_up(&id_priv->wait);
+}
+
+static void cma_release_remove(struct rdma_id_private *id_priv)
+{
+ if (atomic_dec_and_test(&id_priv->dev_remove))
+ wake_up(&id_priv->wait_remove);
+}
+
+struct rdma_cm_id* rdma_create_id(rdma_cm_event_handler event_handler,
+ void *context, enum rdma_port_space ps)
+{
+ struct rdma_id_private *id_priv;
+
+ id_priv = kzalloc(sizeof *id_priv, GFP_KERNEL);
+ if (!id_priv)
+ return ERR_PTR(-ENOMEM);
+
+ id_priv->state = CMA_IDLE;
+ id_priv->id.context = context;
+ id_priv->id.event_handler = event_handler;
+ id_priv->id.ps = ps;
+ spin_lock_init(&id_priv->lock);
+ init_waitqueue_head(&id_priv->wait);
+ atomic_set(&id_priv->refcount, 1);
+ init_waitqueue_head(&id_priv->wait_remove);
+ atomic_set(&id_priv->dev_remove, 0);
+ INIT_LIST_HEAD(&id_priv->listen_list);
+ get_random_bytes(&id_priv->seq_num, sizeof id_priv->seq_num);
+
+ return &id_priv->id;
+}
+EXPORT_SYMBOL(rdma_create_id);
+
+static int cma_init_ib_qp(struct rdma_id_private *id_priv, struct ib_qp *qp)
+{
+ struct ib_qp_attr qp_attr;
+ struct rdma_dev_addr *dev_addr;
+ int ret;
+
+ dev_addr = &id_priv->id.route.addr.dev_addr;
+ ret = ib_find_cached_pkey(id_priv->id.device, id_priv->id.port_num,
+ ib_addr_get_pkey(dev_addr),
+ &qp_attr.pkey_index);
+ if (ret)
+ return ret;
+
+ qp_attr.qp_state = IB_QPS_INIT;
+ qp_attr.qp_access_flags = IB_ACCESS_LOCAL_WRITE;
+ qp_attr.port_num = id_priv->id.port_num;
+ return ib_modify_qp(qp, &qp_attr, IB_QP_STATE | IB_QP_ACCESS_FLAGS |
+ IB_QP_PKEY_INDEX | IB_QP_PORT);
+}
+
+int rdma_create_qp(struct rdma_cm_id *id, struct ib_pd *pd,
+ struct ib_qp_init_attr *qp_init_attr)
+{
+ struct rdma_id_private *id_priv;
+ struct ib_qp *qp;
+ int ret;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ if (id->device != pd->device)
+ return -EINVAL;
+
+ qp = ib_create_qp(pd, qp_init_attr);
+ if (IS_ERR(qp))
+ return PTR_ERR(qp);
+
+ switch (id->device->node_type) {
+ case IB_NODE_CA:
+ ret = cma_init_ib_qp(id_priv, qp);
+ break;
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+
+ if (ret)
+ goto err;
+
+ id->qp = qp;
+ id_priv->qp_num = qp->qp_num;
+ id_priv->qp_type = qp->qp_type;
+ id_priv->srq = (qp->srq != NULL);
+ return 0;
+err:
+ ib_destroy_qp(qp);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_create_qp);
+
+void rdma_destroy_qp(struct rdma_cm_id *id)
+{
+ ib_destroy_qp(id->qp);
+}
+EXPORT_SYMBOL(rdma_destroy_qp);
+
+static int cma_modify_qp_rtr(struct rdma_cm_id *id)
+{
+ struct ib_qp_attr qp_attr;
+ int qp_attr_mask, ret;
+
+ if (!id->qp)
+ return 0;
+
+ /* Need to update QP attributes from default values. */
+ qp_attr.qp_state = IB_QPS_INIT;
+ ret = rdma_init_qp_attr(id, &qp_attr, &qp_attr_mask);
+ if (ret)
+ return ret;
+
+ ret = ib_modify_qp(id->qp, &qp_attr, qp_attr_mask);
+ if (ret)
+ return ret;
+
+ qp_attr.qp_state = IB_QPS_RTR;
+ ret = rdma_init_qp_attr(id, &qp_attr, &qp_attr_mask);
+ if (ret)
+ return ret;
+
+ return ib_modify_qp(id->qp, &qp_attr, qp_attr_mask);
+}
+
+static int cma_modify_qp_rts(struct rdma_cm_id *id)
+{
+ struct ib_qp_attr qp_attr;
+ int qp_attr_mask, ret;
+
+ if (!id->qp)
+ return 0;
+
+ qp_attr.qp_state = IB_QPS_RTS;
+ ret = rdma_init_qp_attr(id, &qp_attr, &qp_attr_mask);
+ if (ret)
+ return ret;
+
+ return ib_modify_qp(id->qp, &qp_attr, qp_attr_mask);
+}
+
+static int cma_modify_qp_err(struct rdma_cm_id *id)
+{
+ struct ib_qp_attr qp_attr;
+
+ if (!id->qp)
+ return 0;
+
+ qp_attr.qp_state = IB_QPS_ERR;
+ return ib_modify_qp(id->qp, &qp_attr, IB_QP_STATE);
+}
+
+int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
+ int *qp_attr_mask)
+{
+ struct rdma_id_private *id_priv;
+ int ret;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ switch (id_priv->id.device->node_type) {
+ case IB_NODE_CA:
+ ret = ib_cm_init_qp_attr(id_priv->cm_id, qp_attr,
+ qp_attr_mask);
+ if (qp_attr->qp_state == IB_QPS_RTR)
+ qp_attr->rq_psn = id_priv->seq_num;
+ break;
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL(rdma_init_qp_attr);
+
+static inline int cma_any_addr(struct sockaddr *addr)
+{
+ struct in6_addr *ip6;
+
+ if (addr->sa_family == AF_INET)
+ return ((struct sockaddr_in *) addr)->sin_addr.s_addr ==
+ INADDR_ANY;
+ else {
+ ip6 = &((struct sockaddr_in6 *) addr)->sin6_addr;
+ return (ip6->s6_addr32[0] | ip6->s6_addr32[1] |
+ ip6->s6_addr32[3] | ip6->s6_addr32[4]) == 0;
+ }
+}
+
+static inline int cma_loopback_addr(struct sockaddr *addr)
+{
+ return ((struct sockaddr_in *) addr)->sin_addr.s_addr ==
+ ntohl(INADDR_LOOPBACK);
+}
+
+static int cma_get_net_info(void *hdr, enum rdma_port_space ps,
+ u8 *ip_ver, __u16 *port,
+ union cma_ip_addr **src, union cma_ip_addr **dst)
+{
+ switch (ps) {
+ case RDMA_PS_SDP:
+ if (((struct sdp_hh *) hdr)->sdp_version != SDP_VERSION)
+ return -EINVAL;
+
+ *ip_ver = sdp_get_ip_ver(hdr);
+ *port = ((struct sdp_hh *) hdr)->port;
+ *src = &((struct sdp_hh *) hdr)->src_addr;
+ *dst = &((struct sdp_hh *) hdr)->dst_addr;
+ break;
+ default:
+ if (((struct cma_hdr *) hdr)->cma_version != CMA_VERSION)
+ return -EINVAL;
+
+ *ip_ver = cma_get_ip_ver(hdr);
+ *port = ((struct cma_hdr *) hdr)->port;
+ *src = &((struct cma_hdr *) hdr)->src_addr;
+ *dst = &((struct cma_hdr *) hdr)->dst_addr;
+ break;
+ }
+ return 0;
+}
+
+static void cma_save_net_info(struct rdma_addr *addr,
+ struct rdma_addr *listen_addr,
+ u8 ip_ver, __u16 port,
+ union cma_ip_addr *src, union cma_ip_addr *dst)
+{
+ struct sockaddr_in *listen4, *ip4;
+ struct sockaddr_in6 *listen6, *ip6;
+
+ switch (ip_ver) {
+ case 4:
+ listen4 = (struct sockaddr_in *) &listen_addr->src_addr;
+ ip4 = (struct sockaddr_in *) &addr->src_addr;
+ ip4->sin_family = listen4->sin_family;
+ ip4->sin_addr.s_addr = dst->ip4.addr;
+ ip4->sin_port = listen4->sin_port;
+
+ ip4 = (struct sockaddr_in *) &addr->dst_addr;
+ ip4->sin_family = listen4->sin_family;
+ ip4->sin_addr.s_addr = src->ip4.addr;
+ ip4->sin_port = port;
+ break;
+ case 6:
+ listen6 = (struct sockaddr_in6 *) &listen_addr->src_addr;
+ ip6 = (struct sockaddr_in6 *) &addr->src_addr;
+ ip6->sin6_family = listen6->sin6_family;
+ ip6->sin6_addr = dst->ip6;
+ ip6->sin6_port = listen6->sin6_port;
+
+ ip6 = (struct sockaddr_in6 *) &addr->dst_addr;
+ ip6->sin6_family = listen6->sin6_family;
+ ip6->sin6_addr = src->ip6;
+ ip6->sin6_port = port;
+ break;
+ default:
+ break;
+ }
+}
+
+static inline int cma_user_data_offset(enum rdma_port_space ps)
+{
+ switch (ps) {
+ case RDMA_PS_SDP:
+ return 0;
+ default:
+ return sizeof(struct cma_hdr);
+ }
+}
+
+static int cma_notify_user(struct rdma_id_private *id_priv,
+ enum rdma_cm_event_type type, int status,
+ void *data, u8 data_len)
+{
+ struct rdma_cm_event event;
+
+ event.event = type;
+ event.status = status;
+ event.private_data = data;
+ event.private_data_len = data_len;
+
+ return id_priv->id.event_handler(&id_priv->id, &event);
+}
+
+static void cma_cancel_addr(struct rdma_id_private *id_priv)
+{
+ switch (id_priv->id.device->node_type) {
+ case IB_NODE_CA:
+ rdma_addr_cancel(&id_priv->id.route.addr.dev_addr);
+ break;
+ default:
+ break;
+ }
+}
+
+static void cma_cancel_route(struct rdma_id_private *id_priv)
+{
+ switch (id_priv->id.device->node_type) {
+ case IB_NODE_CA:
+ ib_sa_cancel_query(id_priv->query_id, id_priv->query);
+ break;
+ default:
+ break;
+ }
+}
+
+static inline int cma_internal_listen(struct rdma_id_private *id_priv)
+{
+ return (id_priv->state == CMA_LISTEN) && id_priv->cma_dev &&
+ cma_any_addr(&id_priv->id.route.addr.src_addr);
+}
+
+static void cma_destroy_listen(struct rdma_id_private *id_priv)
+{
+ cma_exch(id_priv, CMA_DESTROYING);
+
+ if (id_priv->cm_id && !IS_ERR(id_priv->cm_id))
+ ib_destroy_cm_id(id_priv->cm_id);
+
+ list_del(&id_priv->listen_list);
+ if (id_priv->cma_dev)
+ cma_detach_from_dev(id_priv);
+
+ atomic_dec(&id_priv->refcount);
+ wait_event(id_priv->wait, !atomic_read(&id_priv->refcount));
+
+ kfree(id_priv);
+}
+
+static void cma_cancel_listens(struct rdma_id_private *id_priv)
+{
+ struct rdma_id_private *dev_id_priv;
+
+ mutex_lock(&lock);
+ list_del(&id_priv->list);
+
+ while (!list_empty(&id_priv->listen_list)) {
+ dev_id_priv = list_entry(id_priv->listen_list.next,
+ struct rdma_id_private, listen_list);
+ cma_destroy_listen(dev_id_priv);
+ }
+ mutex_unlock(&lock);
+}
+
+static void cma_cancel_operation(struct rdma_id_private *id_priv,
+ enum cma_state state)
+{
+ switch (state) {
+ case CMA_ADDR_QUERY:
+ cma_cancel_addr(id_priv);
+ break;
+ case CMA_ROUTE_QUERY:
+ cma_cancel_route(id_priv);
+ break;
+ case CMA_LISTEN:
+ if (cma_any_addr(&id_priv->id.route.addr.src_addr) &&
+ !id_priv->cma_dev)
+ cma_cancel_listens(id_priv);
+ break;
+ default:
+ break;
+ }
+}
+
+void rdma_destroy_id(struct rdma_cm_id *id)
+{
+ struct rdma_id_private *id_priv;
+ enum cma_state state;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ state = cma_exch(id_priv, CMA_DESTROYING);
+ cma_cancel_operation(id_priv, state);
+
+ if (id_priv->cm_id && !IS_ERR(id_priv->cm_id))
+ ib_destroy_cm_id(id_priv->cm_id);
+
+ if (id_priv->cma_dev) {
+ mutex_lock(&lock);
+ cma_detach_from_dev(id_priv);
+ mutex_unlock(&lock);
+ }
+
+ atomic_dec(&id_priv->refcount);
+ wait_event(id_priv->wait, !atomic_read(&id_priv->refcount));
+
+ kfree(id_priv->id.route.path_rec);
+ kfree(id_priv);
+}
+EXPORT_SYMBOL(rdma_destroy_id);
+
+static int cma_rep_recv(struct rdma_id_private *id_priv)
+{
+ int ret;
+
+ ret = cma_modify_qp_rtr(&id_priv->id);
+ if (ret)
+ goto reject;
+
+ ret = cma_modify_qp_rts(&id_priv->id);
+ if (ret)
+ goto reject;
+
+ ret = ib_send_cm_rtu(id_priv->cm_id, NULL, 0);
+ if (ret)
+ goto reject;
+
+ return 0;
+reject:
+ cma_modify_qp_err(&id_priv->id);
+ ib_send_cm_rej(id_priv->cm_id, IB_CM_REJ_CONSUMER_DEFINED,
+ NULL, 0, NULL, 0);
+ return ret;
+}
+
+static int cma_rtu_recv(struct rdma_id_private *id_priv)
+{
+ int ret;
+
+ ret = cma_modify_qp_rts(&id_priv->id);
+ if (ret)
+ goto reject;
+
+ return 0;
+reject:
+ cma_modify_qp_err(&id_priv->id);
+ ib_send_cm_rej(id_priv->cm_id, IB_CM_REJ_CONSUMER_DEFINED,
+ NULL, 0, NULL, 0);
+ return ret;
+}
+
+static int cma_ib_handler(struct ib_cm_id *cm_id, struct ib_cm_event *ib_event)
+{
+ struct rdma_id_private *id_priv = cm_id->context;
+ enum rdma_cm_event_type event;
+ u8 private_data_len = 0;
+ int ret = 0, status = 0;
+
+ if (!cma_comp(id_priv, CMA_CONNECT))
+ return 0;
+
+ atomic_inc(&id_priv->dev_remove);
+ switch (ib_event->event) {
+ case IB_CM_REQ_ERROR:
+ case IB_CM_REP_ERROR:
+ event = RDMA_CM_EVENT_UNREACHABLE;
+ status = -ETIMEDOUT;
+ break;
+ case IB_CM_REP_RECEIVED:
+ if (id_priv->id.qp) {
+ status = cma_rep_recv(id_priv);
+ event = status ? RDMA_CM_EVENT_CONNECT_ERROR :
+ RDMA_CM_EVENT_ESTABLISHED;
+ } else
+ event = RDMA_CM_EVENT_CONNECT_RESPONSE;
+ private_data_len = IB_CM_REP_PRIVATE_DATA_SIZE;
+ break;
+ case IB_CM_RTU_RECEIVED:
+ status = cma_rtu_recv(id_priv);
+ event = status ? RDMA_CM_EVENT_CONNECT_ERROR :
+ RDMA_CM_EVENT_ESTABLISHED;
+ break;
+ case IB_CM_DREQ_ERROR:
+ status = -ETIMEDOUT; /* fall through */
+ case IB_CM_DREQ_RECEIVED:
+ case IB_CM_DREP_RECEIVED:
+ event = RDMA_CM_EVENT_DISCONNECTED;
+ break;
+ case IB_CM_TIMEWAIT_EXIT:
+ case IB_CM_MRA_RECEIVED:
+ /* ignore event */
+ goto out;
+ case IB_CM_REJ_RECEIVED:
+ cma_modify_qp_err(&id_priv->id);
+ status = ib_event->param.rej_rcvd.reason;
+ event = RDMA_CM_EVENT_REJECTED;
+ break;
+ default:
+ printk(KERN_ERR "RDMA CMA: unexpected IB CM event: %d",
+ ib_event->event);
+ goto out;
+ }
+
+ ret = cma_notify_user(id_priv, event, status, ib_event->private_data,
+ private_data_len);
+ if (ret) {
+ /* Destroy the CM ID by returning a non-zero value. */
+ id_priv->cm_id = NULL;
+ cma_exch(id_priv, CMA_DESTROYING);
+ cma_release_remove(id_priv);
+ rdma_destroy_id(&id_priv->id);
+ return ret;
+ }
+out:
+ cma_release_remove(id_priv);
+ return ret;
+}
+
+static struct rdma_id_private* cma_new_id(struct rdma_cm_id *listen_id,
+ struct ib_cm_event *ib_event)
+{
+ struct rdma_id_private *id_priv;
+ struct rdma_cm_id *id;
+ struct rdma_route *rt;
+ union cma_ip_addr *src, *dst;
+ __u16 port;
+ u8 ip_ver;
+
+ id = rdma_create_id(listen_id->event_handler, listen_id->context,
+ listen_id->ps);
+ if (IS_ERR(id))
+ return NULL;
+
+ rt = &id->route;
+ rt->num_paths = ib_event->param.req_rcvd.alternate_path ? 2 : 1;
+ rt->path_rec = kmalloc(sizeof *rt->path_rec * rt->num_paths, GFP_KERNEL);
+ if (!rt->path_rec)
+ goto err;
+
+ if (cma_get_net_info(ib_event->private_data, listen_id->ps,
+ &ip_ver, &port, &src, &dst))
+ goto err;
+
+ cma_save_net_info(&id->route.addr, &listen_id->route.addr,
+ ip_ver, port, src, dst);
+ rt->path_rec[0] = *ib_event->param.req_rcvd.primary_path;
+ if (rt->num_paths == 2)
+ rt->path_rec[1] = *ib_event->param.req_rcvd.alternate_path;
+
+ ib_addr_set_sgid(&rt->addr.dev_addr, &rt->path_rec[0].sgid);
+ ib_addr_set_dgid(&rt->addr.dev_addr, &rt->path_rec[0].dgid);
+ ib_addr_set_pkey(&rt->addr.dev_addr, be16_to_cpu(rt->path_rec[0].pkey));
+ rt->addr.dev_addr.dev_type = IB_NODE_CA;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ id_priv->state = CMA_CONNECT;
+ return id_priv;
+err:
+ rdma_destroy_id(id);
+ return NULL;
+}
+
+static int cma_req_handler(struct ib_cm_id *cm_id, struct ib_cm_event *ib_event)
+{
+ struct rdma_id_private *listen_id, *conn_id;
+ int offset, ret;
+
+ listen_id = cm_id->context;
+ atomic_inc(&listen_id->dev_remove);
+ if (!cma_comp(listen_id, CMA_LISTEN)) {
+ ret = -ECONNABORTED;
+ goto out;
+ }
+
+ conn_id = cma_new_id(&listen_id->id, ib_event);
+ if (!conn_id) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ atomic_inc(&conn_id->dev_remove);
+ ret = cma_acquire_ib_dev(conn_id);
+ if (ret) {
+ ret = -ENODEV;
+ cma_release_remove(conn_id);
+ rdma_destroy_id(&conn_id->id);
+ goto out;
+ }
+
+ conn_id->cm_id = cm_id;
+ cm_id->context = conn_id;
+ cm_id->cm_handler = cma_ib_handler;
+
+ offset = cma_user_data_offset(listen_id->id.ps);
+ ret = cma_notify_user(conn_id, RDMA_CM_EVENT_CONNECT_REQUEST, 0,
+ ib_event->private_data + offset,
+ IB_CM_REQ_PRIVATE_DATA_SIZE - offset);
+ if (ret) {
+ /* Destroy the CM ID by returning a non-zero value. */
+ conn_id->cm_id = NULL;
+ cma_exch(conn_id, CMA_DESTROYING);
+ cma_release_remove(conn_id);
+ rdma_destroy_id(&conn_id->id);
+ }
+out:
+ cma_release_remove(listen_id);
+ return ret;
+}
+
+static __be64 cma_get_service_id(enum rdma_port_space ps, struct sockaddr *addr)
+{
+ return cpu_to_be64(((u64)ps << 16) +
+ be16_to_cpu(((struct sockaddr_in *) addr)->sin_port));
+}
+
+static void cma_set_compare_data(struct sockaddr *addr,
+ struct ib_cm_compare_data *compare)
+{
+ struct cma_hdr *data, *mask;
+
+ memset(compare, 0, sizeof *compare);
+ data = (void *) compare->data;
+ mask = (void *) compare->mask;
+
+ switch (addr->sa_family) {
+ case AF_INET:
+ cma_set_ip_ver(data, 4);
+ cma_set_ip_ver(mask, 0xF);
+ data->dst_addr.ip4.addr = ((struct sockaddr_in *) addr)->
+ sin_addr.s_addr;
+ mask->dst_addr.ip4.addr = ~0;
+ break;
+ case AF_INET6:
+ cma_set_ip_ver(data, 6);
+ cma_set_ip_ver(mask, 0xF);
+ data->dst_addr.ip6 = ((struct sockaddr_in6 *) addr)->
+ sin6_addr;
+ memset(&mask->dst_addr.ip6, 1, sizeof mask->dst_addr.ip6);
+ break;
+ default:
+ break;
+ }
+}
+
+static int cma_ib_listen(struct rdma_id_private *id_priv)
+{
+ struct ib_cm_compare_data compare_data;
+ struct sockaddr *addr;
+ __be64 svc_id;
+ int ret;
+
+ id_priv->cm_id = ib_create_cm_id(id_priv->id.device, cma_req_handler,
+ id_priv);
+ if (IS_ERR(id_priv->cm_id))
+ return PTR_ERR(id_priv->cm_id);
+
+ addr = &id_priv->id.route.addr.src_addr;
+ svc_id = cma_get_service_id(id_priv->id.ps, addr);
+ if (cma_any_addr(addr))
+ ret = ib_cm_listen(id_priv->cm_id, svc_id, 0, NULL);
+ else {
+ cma_set_compare_data(addr, &compare_data);
+ ret = ib_cm_listen(id_priv->cm_id, svc_id, 0, &compare_data);
+ }
+
+ if (ret) {
+ ib_destroy_cm_id(id_priv->cm_id);
+ id_priv->cm_id = NULL;
+ }
+
+ return ret;
+}
+
+static int cma_duplicate_listen(struct rdma_id_private *id_priv)
+{
+ struct rdma_id_private *cur_id_priv;
+ struct sockaddr_in *cur_addr, *new_addr;
+
+ new_addr = (struct sockaddr_in *) &id_priv->id.route.addr.src_addr;
+ list_for_each_entry(cur_id_priv, &listen_any_list, listen_list) {
+ cur_addr = (struct sockaddr_in *)
+ &cur_id_priv->id.route.addr.src_addr;
+ if (cur_addr->sin_port == new_addr->sin_port)
+ return -EADDRINUSE;
+ }
+ return 0;
+}
+
+static int cma_listen_handler(struct rdma_cm_id *id,
+ struct rdma_cm_event *event)
+{
+ struct rdma_id_private *id_priv = id->context;
+
+ id->context = id_priv->id.context;
+ id->event_handler = id_priv->id.event_handler;
+ return id_priv->id.event_handler(id, event);
+}
+
+static void cma_listen_on_dev(struct rdma_id_private *id_priv,
+ struct cma_device *cma_dev)
+{
+ struct rdma_id_private *dev_id_priv;
+ struct rdma_cm_id *id;
+ int ret;
+
+ id = rdma_create_id(cma_listen_handler, id_priv, id_priv->id.ps);
+ if (IS_ERR(id))
+ return;
+
+ dev_id_priv = container_of(id, struct rdma_id_private, id);
+ ret = rdma_bind_addr(id, &id_priv->id.route.addr.src_addr);
+ if (ret)
+ goto err;
+
+ cma_attach_to_dev(dev_id_priv, cma_dev);
+ list_add_tail(&dev_id_priv->listen_list, &id_priv->listen_list);
+
+ ret = rdma_listen(id, id_priv->backlog);
+ if (ret)
+ goto err;
+
+ return;
+err:
+ cma_destroy_listen(dev_id_priv);
+}
+
+static int cma_listen_on_all(struct rdma_id_private *id_priv)
+{
+ struct cma_device *cma_dev;
+ int ret;
+
+ mutex_lock(&lock);
+ ret = cma_duplicate_listen(id_priv);
+ if (ret)
+ goto out;
+
+ list_add_tail(&id_priv->list, &listen_any_list);
+ list_for_each_entry(cma_dev, &dev_list, list)
+ cma_listen_on_dev(id_priv, cma_dev);
+out:
+ mutex_unlock(&lock);
+ return ret;
+}
+
+int rdma_listen(struct rdma_cm_id *id, int backlog)
+{
+ struct rdma_id_private *id_priv;
+ int ret;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ if (!cma_comp_exch(id_priv, CMA_ADDR_BOUND, CMA_LISTEN))
+ return -EINVAL;
+
+ if (id->device) {
+ switch (id->device->node_type) {
+ case IB_NODE_CA:
+ ret = cma_ib_listen(id_priv);
+ break;
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+ } else
+ ret = cma_listen_on_all(id_priv);
+
+ if (ret)
+ goto err;
+
+ id_priv->backlog = backlog;
+ return 0;
+err:
+ cma_comp_exch(id_priv, CMA_LISTEN, CMA_ADDR_BOUND);
+ return ret;
+};
+EXPORT_SYMBOL(rdma_listen);
+
+static void cma_query_handler(int status, struct ib_sa_path_rec *path_rec,
+ void *context)
+{
+ struct rdma_id_private *id_priv = context;
+ struct rdma_route *route = &id_priv->id.route;
+ enum rdma_cm_event_type event = RDMA_CM_EVENT_ROUTE_RESOLVED;
+
+ atomic_inc(&id_priv->dev_remove);
+ if (!status) {
+ route->path_rec = kmalloc(sizeof *route->path_rec, GFP_KERNEL);
+ if (route->path_rec) {
+ route->num_paths = 1;
+ *route->path_rec = *path_rec;
+ if (!cma_comp_exch(id_priv, CMA_ROUTE_QUERY,
+ CMA_ROUTE_RESOLVED)) {
+ kfree(route->path_rec);
+ goto out;
+ }
+ } else
+ status = -ENOMEM;
+ }
+
+ if (status) {
+ if (!cma_comp_exch(id_priv, CMA_ROUTE_QUERY, CMA_ADDR_RESOLVED))
+ goto out;
+ event = RDMA_CM_EVENT_ROUTE_ERROR;
+ }
+
+ if (cma_notify_user(id_priv, event, status, NULL, 0)) {
+ cma_exch(id_priv, CMA_DESTROYING);
+ cma_release_remove(id_priv);
+ cma_deref_id(id_priv);
+ rdma_destroy_id(&id_priv->id);
+ return;
+ }
+out:
+ cma_release_remove(id_priv);
+ cma_deref_id(id_priv);
+}
+
+static int cma_resolve_ib_route(struct rdma_id_private *id_priv, int timeout_ms)
+{
+ struct rdma_dev_addr *addr = &id_priv->id.route.addr.dev_addr;
+ struct ib_sa_path_rec path_rec;
+
+ memset(&path_rec, 0, sizeof path_rec);
+ path_rec.sgid = *ib_addr_get_sgid(addr);
+ path_rec.dgid = *ib_addr_get_dgid(addr);
+ path_rec.pkey = cpu_to_be16(ib_addr_get_pkey(addr));
+ path_rec.numb_path = 1;
+
+ id_priv->query_id = ib_sa_path_rec_get(id_priv->id.device,
+ id_priv->id.port_num, &path_rec,
+ IB_SA_PATH_REC_DGID | IB_SA_PATH_REC_SGID |
+ IB_SA_PATH_REC_PKEY | IB_SA_PATH_REC_NUMB_PATH,
+ timeout_ms, GFP_KERNEL,
+ cma_query_handler, id_priv, &id_priv->query);
+
+ return (id_priv->query_id < 0) ? id_priv->query_id : 0;
+}
+
+int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms)
+{
+ struct rdma_id_private *id_priv;
+ int ret;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ if (!cma_comp_exch(id_priv, CMA_ADDR_RESOLVED, CMA_ROUTE_QUERY))
+ return -EINVAL;
+
+ atomic_inc(&id_priv->refcount);
+ switch (id->device->node_type) {
+ case IB_NODE_CA:
+ ret = cma_resolve_ib_route(id_priv, timeout_ms);
+ break;
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+ if (ret)
+ goto err;
+
+ return 0;
+err:
+ cma_comp_exch(id_priv, CMA_ROUTE_QUERY, CMA_ADDR_RESOLVED);
+ cma_deref_id(id_priv);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_resolve_route);
+
+static int cma_bind_loopback(struct rdma_id_private *id_priv)
+{
+ struct cma_device *cma_dev;
+ union ib_gid *gid;
+ u16 pkey;
+ int ret;
+
+ mutex_lock(&lock);
+ if (list_empty(&dev_list)) {
+ ret = -ENODEV;
+ goto out;
+ }
+
+ cma_dev = list_entry(dev_list.next, struct cma_device, list);
+ gid = ib_addr_get_sgid(&id_priv->id.route.addr.dev_addr);
+ ret = ib_get_cached_gid(cma_dev->device, 1, 0, gid);
+ if (ret)
+ goto out;
+
+ ret = ib_get_cached_pkey(cma_dev->device, 1, 0, &pkey);
+ if (ret)
+ goto out;
+
+ ib_addr_set_pkey(&id_priv->id.route.addr.dev_addr, pkey);
+ id_priv->id.port_num = 1;
+ cma_attach_to_dev(id_priv, cma_dev);
+out:
+ mutex_unlock(&lock);
+ return ret;
+}
+
+static void addr_handler(int status, struct sockaddr *src_addr,
+ struct rdma_dev_addr *dev_addr, void *context)
+{
+ struct rdma_id_private *id_priv = context;
+ enum rdma_cm_event_type event;
+ enum cma_state old_state;
+
+ atomic_inc(&id_priv->dev_remove);
+ if (!id_priv->cma_dev) {
+ old_state = CMA_IDLE;
+ if (!status)
+ status = cma_acquire_dev(id_priv);
+ } else
+ old_state = CMA_ADDR_BOUND;
+
+ if (status) {
+ if (!cma_comp_exch(id_priv, CMA_ADDR_QUERY, old_state))
+ goto out;
+ event = RDMA_CM_EVENT_ADDR_ERROR;
+ } else {
+ if (!cma_comp_exch(id_priv, CMA_ADDR_QUERY, CMA_ADDR_RESOLVED))
+ goto out;
+ memcpy(&id_priv->id.route.addr.src_addr, src_addr,
+ ip_addr_size(src_addr));
+ event = RDMA_CM_EVENT_ADDR_RESOLVED;
+ }
+
+ if (cma_notify_user(id_priv, event, status, NULL, 0)) {
+ cma_exch(id_priv, CMA_DESTROYING);
+ cma_release_remove(id_priv);
+ cma_deref_id(id_priv);
+ rdma_destroy_id(&id_priv->id);
+ return;
+ }
+out:
+ cma_release_remove(id_priv);
+ cma_deref_id(id_priv);
+}
+
+static void loopback_addr_handler(void *data)
+{
+ struct cma_work *work = data;
+ struct rdma_id_private *id_priv = work->id;
+
+ kfree(work);
+ atomic_inc(&id_priv->dev_remove);
+
+ if (!cma_comp_exch(id_priv, CMA_ADDR_QUERY, CMA_ADDR_RESOLVED))
+ goto out;
+
+ if (cma_notify_user(id_priv, RDMA_CM_EVENT_ADDR_RESOLVED, 0, NULL, 0)) {
+ cma_exch(id_priv, CMA_DESTROYING);
+ cma_release_remove(id_priv);
+ cma_deref_id(id_priv);
+ rdma_destroy_id(&id_priv->id);
+ return;
+ }
+out:
+ cma_release_remove(id_priv);
+ cma_deref_id(id_priv);
+}
+
+static int cma_resolve_loopback(struct rdma_id_private *id_priv,
+ struct sockaddr *src_addr, enum cma_state state)
+{
+ struct cma_work *work;
+ struct rdma_dev_addr *dev_addr;
+ int ret;
+
+ work = kmalloc(sizeof *work, GFP_KERNEL);
+ if (!work)
+ return -ENOMEM;
+
+ if (state == CMA_IDLE) {
+ ret = cma_bind_loopback(id_priv);
+ if (ret)
+ goto err;
+ dev_addr = &id_priv->id.route.addr.dev_addr;
+ ib_addr_set_dgid(dev_addr, ib_addr_get_sgid(dev_addr));
+ if (!src_addr || cma_any_addr(src_addr))
+ src_addr = &id_priv->id.route.addr.dst_addr;
+ memcpy(&id_priv->id.route.addr.src_addr, src_addr,
+ ip_addr_size(src_addr));
+ }
+
+ work->id = id_priv;
+ INIT_WORK(&work->work, loopback_addr_handler, work);
+ queue_work(rdma_wq, &work->work);
+ return 0;
+err:
+ kfree(work);
+ return ret;
+}
+
+int rdma_resolve_addr(struct rdma_cm_id *id, struct sockaddr *src_addr,
+ struct sockaddr *dst_addr, int timeout_ms)
+{
+ struct rdma_id_private *id_priv;
+ enum cma_state expected_state;
+ int ret;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ if (id_priv->cma_dev) {
+ expected_state = CMA_ADDR_BOUND;
+ src_addr = &id->route.addr.src_addr;
+ } else
+ expected_state = CMA_IDLE;
+
+ if (!cma_comp_exch(id_priv, expected_state, CMA_ADDR_QUERY))
+ return -EINVAL;
+
+ atomic_inc(&id_priv->refcount);
+ memcpy(&id->route.addr.dst_addr, dst_addr, ip_addr_size(dst_addr));
+ if (cma_loopback_addr(dst_addr))
+ ret = cma_resolve_loopback(id_priv, src_addr, expected_state);
+ else
+ ret = rdma_resolve_ip(src_addr, dst_addr,
+ &id->route.addr.dev_addr,
+ timeout_ms, addr_handler, id_priv);
+ if (ret)
+ goto err;
+
+ return 0;
+err:
+ cma_comp_exch(id_priv, CMA_ADDR_QUERY, expected_state);
+ cma_deref_id(id_priv);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_resolve_addr);
+
+int rdma_bind_addr(struct rdma_cm_id *id, struct sockaddr *addr)
+{
+ struct rdma_id_private *id_priv;
+ struct rdma_dev_addr *dev_addr;
+ int ret;
+
+ if (addr->sa_family != AF_INET)
+ return -EINVAL;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ if (!cma_comp_exch(id_priv, CMA_IDLE, CMA_ADDR_BOUND))
+ return -EINVAL;
+
+ if (cma_any_addr(addr)) {
+ ret = 0;
+ } else if (cma_loopback_addr(addr)) {
+ ret = cma_bind_loopback(id_priv);
+ } else {
+ dev_addr = &id->route.addr.dev_addr;
+ ret = rdma_translate_ip(addr, dev_addr);
+ if (!ret)
+ ret = cma_acquire_dev(id_priv);
+ }
+
+ if (ret)
+ goto err;
+
+ memcpy(&id->route.addr.src_addr, addr, ip_addr_size(addr));
+ return 0;
+err:
+ cma_comp_exch(id_priv, CMA_ADDR_BOUND, CMA_IDLE);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_bind_addr);
+
+static void cma_format_hdr(void *hdr, enum rdma_port_space ps,
+ struct rdma_route *route)
+{
+ struct sockaddr_in *src4, *dst4;
+ struct cma_hdr *cma_hdr;
+ struct sdp_hh *sdp_hdr;
+
+ src4 = (struct sockaddr_in *) &route->addr.src_addr;
+ dst4 = (struct sockaddr_in *) &route->addr.dst_addr;
+
+ switch (ps) {
+ case RDMA_PS_SDP:
+ sdp_hdr = hdr;
+ sdp_hdr->sdp_version = SDP_VERSION;
+ sdp_set_ip_ver(sdp_hdr, 4);
+ sdp_hdr->src_addr.ip4.addr = src4->sin_addr.s_addr;
+ sdp_hdr->dst_addr.ip4.addr = dst4->sin_addr.s_addr;
+ sdp_hdr->port = src4->sin_port;
+ break;
+ default:
+ cma_hdr = hdr;
+ cma_hdr->cma_version = CMA_VERSION;
+ cma_set_ip_ver(cma_hdr, 4);
+ cma_hdr->src_addr.ip4.addr = src4->sin_addr.s_addr;
+ cma_hdr->dst_addr.ip4.addr = dst4->sin_addr.s_addr;
+ cma_hdr->port = src4->sin_port;
+ break;
+ }
+}
+
+static int cma_connect_ib(struct rdma_id_private *id_priv,
+ struct rdma_conn_param *conn_param)
+{
+ struct ib_cm_req_param req;
+ struct rdma_route *route;
+ void *private_data;
+ int offset, ret;
+
+ memset(&req, 0, sizeof req);
+ offset = cma_user_data_offset(id_priv->id.ps);
+ req.private_data_len = offset + conn_param->private_data_len;
+ private_data = kzalloc(req.private_data_len, GFP_ATOMIC);
+ if (!private_data)
+ return -ENOMEM;
+
+ if (conn_param->private_data && conn_param->private_data_len)
+ memcpy(private_data + offset, conn_param->private_data,
+ conn_param->private_data_len);
+
+ id_priv->cm_id = ib_create_cm_id(id_priv->id.device, cma_ib_handler,
+ id_priv);
+ if (IS_ERR(id_priv->cm_id)) {
+ ret = PTR_ERR(id_priv->cm_id);
+ goto out;
+ }
+
+ route = &id_priv->id.route;
+ cma_format_hdr(private_data, id_priv->id.ps, route);
+ req.private_data = private_data;
+
+ req.primary_path = &route->path_rec[0];
+ if (route->num_paths == 2)
+ req.alternate_path = &route->path_rec[1];
+
+ req.service_id = cma_get_service_id(id_priv->id.ps,
+ &route->addr.dst_addr);
+ req.qp_num = id_priv->qp_num;
+ req.qp_type = id_priv->qp_type;
+ req.starting_psn = id_priv->seq_num;
+ req.responder_resources = conn_param->responder_resources;
+ req.initiator_depth = conn_param->initiator_depth;
+ req.flow_control = conn_param->flow_control;
+ req.retry_count = conn_param->retry_count;
+ req.rnr_retry_count = conn_param->rnr_retry_count;
+ req.remote_cm_response_timeout = CMA_CM_RESPONSE_TIMEOUT;
+ req.local_cm_response_timeout = CMA_CM_RESPONSE_TIMEOUT;
+ req.max_cm_retries = CMA_MAX_CM_RETRIES;
+ req.srq = id_priv->srq ? 1 : 0;
+
+ ret = ib_send_cm_req(id_priv->cm_id, &req);
+out:
+ kfree(private_data);
+ return ret;
+}
+
+int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
+{
+ struct rdma_id_private *id_priv;
+ int ret;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ if (!cma_comp_exch(id_priv, CMA_ROUTE_RESOLVED, CMA_CONNECT))
+ return -EINVAL;
+
+ if (!id->qp) {
+ id_priv->qp_num = conn_param->qp_num;
+ id_priv->qp_type = conn_param->qp_type;
+ id_priv->srq = conn_param->srq;
+ }
+
+ switch (id->device->node_type) {
+ case IB_NODE_CA:
+ ret = cma_connect_ib(id_priv, conn_param);
+ break;
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+ if (ret)
+ goto err;
+
+ return 0;
+err:
+ cma_comp_exch(id_priv, CMA_CONNECT, CMA_ROUTE_RESOLVED);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_connect);
+
+static int cma_accept_ib(struct rdma_id_private *id_priv,
+ struct rdma_conn_param *conn_param)
+{
+ struct ib_cm_rep_param rep;
+ int ret;
+
+ ret = cma_modify_qp_rtr(&id_priv->id);
+ if (ret)
+ return ret;
+
+ memset(&rep, 0, sizeof rep);
+ rep.qp_num = id_priv->qp_num;
+ rep.starting_psn = id_priv->seq_num;
+ rep.private_data = conn_param->private_data;
+ rep.private_data_len = conn_param->private_data_len;
+ rep.responder_resources = conn_param->responder_resources;
+ rep.initiator_depth = conn_param->initiator_depth;
+ rep.target_ack_delay = CMA_CM_RESPONSE_TIMEOUT;
+ rep.failover_accepted = 0;
+ rep.flow_control = conn_param->flow_control;
+ rep.rnr_retry_count = conn_param->rnr_retry_count;
+ rep.srq = id_priv->srq ? 1 : 0;
+
+ return ib_send_cm_rep(id_priv->cm_id, &rep);
+}
+
+int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
+{
+ struct rdma_id_private *id_priv;
+ int ret;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ if (!cma_comp(id_priv, CMA_CONNECT))
+ return -EINVAL;
+
+ if (!id->qp && conn_param) {
+ id_priv->qp_num = conn_param->qp_num;
+ id_priv->qp_type = conn_param->qp_type;
+ id_priv->srq = conn_param->srq;
+ }
+
+ switch (id->device->node_type) {
+ case IB_NODE_CA:
+ if (conn_param)
+ ret = cma_accept_ib(id_priv, conn_param);
+ else
+ ret = cma_rep_recv(id_priv);
+ break;
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+
+ if (ret)
+ goto reject;
+
+ return 0;
+reject:
+ cma_modify_qp_err(id);
+ rdma_reject(id, NULL, 0);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_accept);
+
+int rdma_reject(struct rdma_cm_id *id, const void *private_data,
+ u8 private_data_len)
+{
+ struct rdma_id_private *id_priv;
+ int ret;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ if (!cma_comp(id_priv, CMA_CONNECT))
+ return -EINVAL;
+
+ switch (id->device->node_type) {
+ case IB_NODE_CA:
+ ret = ib_send_cm_rej(id_priv->cm_id, IB_CM_REJ_CONSUMER_DEFINED,
+ NULL, 0, private_data, private_data_len);
+ break;
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+ return ret;
+};
+EXPORT_SYMBOL(rdma_reject);
+
+int rdma_disconnect(struct rdma_cm_id *id)
+{
+ struct rdma_id_private *id_priv;
+ int ret;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ if (!cma_comp(id_priv, CMA_CONNECT))
+ return -EINVAL;
+
+ ret = cma_modify_qp_err(id);
+ if (ret)
+ goto out;
+
+ switch (id->device->node_type) {
+ case IB_NODE_CA:
+ /* Initiate or respond to a disconnect. */
+ if (ib_send_cm_dreq(id_priv->cm_id, NULL, 0))
+ ib_send_cm_drep(id_priv->cm_id, NULL, 0);
+ break;
+ default:
+ break;
+ }
+out:
+ return ret;
+}
+EXPORT_SYMBOL(rdma_disconnect);
+
+static void cma_add_one(struct ib_device *device)
+{
+ struct cma_device *cma_dev;
+ struct rdma_id_private *id_priv;
+
+ cma_dev = kmalloc(sizeof *cma_dev, GFP_KERNEL);
+ if (!cma_dev)
+ return;
+
+ cma_dev->device = device;
+ cma_dev->node_guid = device->node_guid;
+ if (!cma_dev->node_guid)
+ goto err;
+
+ init_waitqueue_head(&cma_dev->wait);
+ atomic_set(&cma_dev->refcount, 1);
+ INIT_LIST_HEAD(&cma_dev->id_list);
+ ib_set_client_data(device, &cma_client, cma_dev);
+
+ mutex_lock(&lock);
+ list_add_tail(&cma_dev->list, &dev_list);
+ list_for_each_entry(id_priv, &listen_any_list, list)
+ cma_listen_on_dev(id_priv, cma_dev);
+ mutex_unlock(&lock);
+ return;
+err:
+ kfree(cma_dev);
+}
+
+static int cma_remove_id_dev(struct rdma_id_private *id_priv)
+{
+ enum cma_state state;
+
+ /* Record that we want to remove the device */
+ state = cma_exch(id_priv, CMA_DEVICE_REMOVAL);
+ if (state == CMA_DESTROYING)
+ return 0;
+
+ cma_cancel_operation(id_priv, state);
+ wait_event(id_priv->wait_remove, !atomic_read(&id_priv->dev_remove));
+
+ /* Check for destruction from another callback. */
+ if (!cma_comp(id_priv, CMA_DEVICE_REMOVAL))
+ return 0;
+
+ return cma_notify_user(id_priv, RDMA_CM_EVENT_DEVICE_REMOVAL,
+ 0, NULL, 0);
+}
+
+static void cma_process_remove(struct cma_device *cma_dev)
+{
+ struct list_head remove_list;
+ struct rdma_id_private *id_priv;
+ int ret;
+
+ INIT_LIST_HEAD(&remove_list);
+
+ mutex_lock(&lock);
+ while (!list_empty(&cma_dev->id_list)) {
+ id_priv = list_entry(cma_dev->id_list.next,
+ struct rdma_id_private, list);
+
+ if (cma_internal_listen(id_priv)) {
+ cma_destroy_listen(id_priv);
+ continue;
+ }
+
+ list_del(&id_priv->list);
+ list_add_tail(&id_priv->list, &remove_list);
+ atomic_inc(&id_priv->refcount);
+ mutex_unlock(&lock);
+
+ ret = cma_remove_id_dev(id_priv);
+ cma_deref_id(id_priv);
+ if (ret)
+ rdma_destroy_id(&id_priv->id);
+
+ mutex_lock(&lock);
+ }
+ mutex_unlock(&lock);
+
+ atomic_dec(&cma_dev->refcount);
+ wait_event(cma_dev->wait, !atomic_read(&cma_dev->refcount));
+}
+
+static void cma_remove_one(struct ib_device *device)
+{
+ struct cma_device *cma_dev;
+
+ cma_dev = ib_get_client_data(device, &cma_client);
+ if (!cma_dev)
+ return;
+
+ mutex_lock(&lock);
+ list_del(&cma_dev->list);
+ mutex_unlock(&lock);
+
+ cma_process_remove(cma_dev);
+ kfree(cma_dev);
+}
+
+static int cma_init(void)
+{
+ return ib_register_client(&cma_client);
+}
+
+static void cma_cleanup(void)
+{
+ ib_unregister_client(&cma_client);
+}
+
+module_init(cma_init);
+module_exit(cma_cleanup);
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/Makefile
linux-2.6.ib/drivers/infiniband/core/Makefile
--- linux-2.6.git/drivers/infiniband/core/Makefile 2006-01-16 16:16:18.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/Makefile 2006-01-16 16:35:48.000000000 -0800
@@ -1,5 +1,5 @@
obj-$(CONFIG_INFINIBAND) += ib_core.o ib_mad.o ib_sa.o \
- ib_cm.o ib_addr.o
+ ib_cm.o ib_addr.o rdma_cm.o
obj-$(CONFIG_INFINIBAND_USER_MAD) += ib_umad.o
obj-$(CONFIG_INFINIBAND_USER_ACCESS) += ib_uverbs.o ib_ucm.o
@@ -12,6 +12,8 @@ ib_sa-y := sa_query.o
ib_cm-y := cm.o
+rdma_cm-y := cma.o
+
ib_addr-y := addr.o
ib_umad-y := user_mad.o
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/rdma_cm.h
linux-2.6.ib/include/rdma/rdma_cm.h
--- linux-2.6.git/include/rdma/rdma_cm.h 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/include/rdma/rdma_cm.h 2006-01-16 16:19:12.000000000 -0800
@@ -0,0 +1,255 @@
+/*
+ * Copyright (c) 2005 Voltaire Inc. All rights reserved.
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ *
+ */
+
+#if !defined(RDMA_CM_H)
+#define RDMA_CM_H
+
+#include <linux/socket.h>
+#include <linux/in6.h>
+#include <rdma/ib_addr.h>
+#include <rdma/ib_sa.h>
+
+/*
+ * Upon receiving a device removal event, users must destroy the associated
+ * RDMA identifier and release all resources allocated with the device.
+ */
+enum rdma_cm_event_type {
+ RDMA_CM_EVENT_ADDR_RESOLVED,
+ RDMA_CM_EVENT_ADDR_ERROR,
+ RDMA_CM_EVENT_ROUTE_RESOLVED,
+ RDMA_CM_EVENT_ROUTE_ERROR,
+ RDMA_CM_EVENT_CONNECT_REQUEST,
+ RDMA_CM_EVENT_CONNECT_RESPONSE,
+ RDMA_CM_EVENT_CONNECT_ERROR,
+ RDMA_CM_EVENT_UNREACHABLE,
+ RDMA_CM_EVENT_REJECTED,
+ RDMA_CM_EVENT_ESTABLISHED,
+ RDMA_CM_EVENT_DISCONNECTED,
+ RDMA_CM_EVENT_DEVICE_REMOVAL,
+};
+
+enum rdma_port_space {
+ RDMA_PS_SDP = 0x0001,
+ RDMA_PS_TCP = 0x0106,
+ RDMA_PS_UDP = 0x0111,
+ RDMA_PS_SCTP = 0x0183
+};
+
+struct rdma_addr {
+ struct sockaddr src_addr;
+ u8 src_pad[sizeof(struct sockaddr_in6) -
+ sizeof(struct sockaddr)];
+ struct sockaddr dst_addr;
+ u8 dst_pad[sizeof(struct sockaddr_in6) -
+ sizeof(struct sockaddr)];
+ struct rdma_dev_addr dev_addr;
+};
+
+struct rdma_route {
+ struct rdma_addr addr;
+ struct ib_sa_path_rec *path_rec;
+ int num_paths;
+};
+
+struct rdma_cm_event {
+ enum rdma_cm_event_type event;
+ int status;
+ void *private_data;
+ u8 private_data_len;
+};
+
+struct rdma_cm_id;
+
+/**
+ * rdma_cm_event_handler - Callback used to report user events.
+ *
+ * Notes: Users may not call rdma_destroy_id from this callback to destroy
+ * the passed in id, or a corresponding listen id. Returning a
+ * non-zero value from the callback will destroy the corresponding id.
+ */
+typedef int (*rdma_cm_event_handler)(struct rdma_cm_id *id,
+ struct rdma_cm_event *event);
+
+struct rdma_cm_id {
+ struct ib_device *device;
+ void *context;
+ struct ib_qp *qp;
+ rdma_cm_event_handler event_handler;
+ struct rdma_route route;
+ enum rdma_port_space ps;
+ u8 port_num;
+};
+
+/**
+ * rdma_create_id - Create an RDMA identifier.
+ *
+ * @event_handler: User callback invoked to report events associated with the
+ * returned rdma_id.
+ * @context: User specified context associated with the id.
+ * @ps: RDMA port space.
+ */
+struct rdma_cm_id* rdma_create_id(rdma_cm_event_handler event_handler,
+ void *context, enum rdma_port_space ps);
+
+void rdma_destroy_id(struct rdma_cm_id *id);
+
+/**
+ * rdma_bind_addr - Bind an RDMA identifier to a source address and
+ * associated RDMA device, if needed.
+ *
+ * @id: RDMA identifier.
+ * @addr: Local address information. Wildcard values are permitted.
+ *
+ * This associates a source address with the RDMA identifier before calling
+ * rdma_listen. If a specific local address is given, the RDMA identifier will
+ * be bound to a local RDMA device.
+ */
+int rdma_bind_addr(struct rdma_cm_id *id, struct sockaddr *addr);
+
+/**
+ * rdma_resolve_addr - Resolve destination and optional source addresses
+ * from IP addresses to an RDMA address. If successful, the specified
+ * rdma_cm_id will be bound to a local device.
+ *
+ * @id: RDMA identifier.
+ * @src_addr: Source address information. This parameter may be NULL.
+ * @dst_addr: Destination address information.
+ * @timeout_ms: Time to wait for resolution to complete.
+ */
+int rdma_resolve_addr(struct rdma_cm_id *id, struct sockaddr *src_addr,
+ struct sockaddr *dst_addr, int timeout_ms);
+
+/**
+ * rdma_resolve_route - Resolve the RDMA address bound to the RDMA identifier
+ * into route information needed to establish a connection.
+ *
+ * This is called on the client side of a connection.
+ * Users must have first called rdma_resolve_addr to resolve a dst_addr
+ * into an RDMA address before calling this routine.
+ */
+int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms);
+
+/**
+ * rdma_create_qp - Allocate a QP and associate it with the specified RDMA
+ * identifier.
+ *
+ * QPs allocated to an rdma_cm_id will automatically be transitioned by the CMA
+ * through their states.
+ */
+int rdma_create_qp(struct rdma_cm_id *id, struct ib_pd *pd,
+ struct ib_qp_init_attr *qp_init_attr);
+
+/**
+ * rdma_destroy_qp - Deallocate the QP associated with the specified RDMA
+ * identifier.
+ *
+ * Users must destroy any QP associated with an RDMA identifier before
+ * destroying the RDMA ID.
+ */
+void rdma_destroy_qp(struct rdma_cm_id *id);
+
+/**
+ * rdma_init_qp_attr - Initializes the QP attributes for use in transitioning
+ * to a specified QP state.
+ * @id: Communication identifier associated with the QP attributes to
+ * initialize.
+ * @qp_attr: On input, specifies the desired QP state. On output, the
+ * mandatory and desired optional attributes will be set in order to
+ * modify the QP to the specified state.
+ * @qp_attr_mask: The QP attribute mask that may be used to transition the
+ * QP to the specified state.
+ *
+ * Users must set the @qp_attr->qp_state to the desired QP state. This call
+ * will set all required attributes for the given transition, along with
+ * known optional attributes. Users may override the attributes returned from
+ * this call before calling ib_modify_qp.
+ *
+ * Users that wish to have their QP automatically transitioned through its
+ * states can associate a QP with the rdma_cm_id by calling rdma_create_qp().
+ */
+int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
+ int *qp_attr_mask);
+
+struct rdma_conn_param {
+ const void *private_data;
+ u8 private_data_len;
+ u8 responder_resources;
+ u8 initiator_depth;
+ u8 flow_control;
+ u8 retry_count; /* ignored when accepting */
+ u8 rnr_retry_count;
+ /* Fields below ignored if a QP is created on the rdma_cm_id. */
+ u8 srq;
+ u32 qp_num;
+ enum ib_qp_type qp_type;
+};
+
+/**
+ * rdma_connect - Initiate an active connection request.
+ *
+ * Users must have resolved a route for the rdma_cm_id to connect with
+ * by having called rdma_resolve_route before calling this routine.
+ */
+int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param);
+
+/**
+ * rdma_listen - This function is called by the passive side to
+ * listen for incoming connection requests.
+ *
+ * Users must have bound the rdma_cm_id to a local address by calling
+ * rdma_bind_addr before calling this routine.
+ */
+int rdma_listen(struct rdma_cm_id *id, int backlog);
+
+/**
+ * rdma_accept - Called to accept a connection request or response.
+ * @id: Connection identifier associated with the request.
+ * @conn_param: Information needed to establish the connection. This must be
+ * provided if accepting a connection request. If accepting a connection
+ * response, this parameter must be NULL.
+ *
+ * Typically, this routine is only called by the listener to accept a connection
+ * request. It must also be called on the active side of a connection if the
+ * user is performing their own QP transitions.
+ */
+int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param);
+
+/**
+ * rdma_reject - Called on the passive side to reject a connection request.
+ */
+int rdma_reject(struct rdma_cm_id *id, const void *private_data,
+ u8 private_data_len);
+
+/**
+ * rdma_disconnect - This function disconnects the associated QP.
+ */
+int rdma_disconnect(struct rdma_cm_id *id);
+
+#endif /* RDMA_CM_H */
+
^ permalink raw reply [flat|nested] 97+ messages in thread* [PATCH 5/5] Infiniband: connection abstraction
2006-02-01 20:03 [PATCH 0/5] Infiniband: connection abstraction Sean Hefty
` (3 preceding siblings ...)
2006-02-01 20:18 ` [PATCH 4/5] Infiniband: " Sean Hefty
@ 2006-02-01 20:19 ` Sean Hefty
2006-03-03 21:13 ` [PATCH 0/5] " Sean Hefty
5 siblings, 0 replies; 97+ messages in thread
From: Sean Hefty @ 2006-02-01 20:19 UTC (permalink / raw)
To: netdev, linux-kernel; +Cc: openib-general
This patch adds the kernel component to support the userspace Infiniband/RDMA
connection agent library.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
---
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/Makefile
linux-2.6.ib/drivers/infiniband/core/Makefile
--- linux-2.6.git/drivers/infiniband/core/Makefile 2006-01-16 16:58:58.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/Makefile 2006-01-16 16:55:25.000000000 -0800
@@ -1,5 +1,5 @@
obj-$(CONFIG_INFINIBAND) += ib_core.o ib_mad.o ib_sa.o \
- ib_cm.o ib_addr.o rdma_cm.o
+ ib_cm.o ib_addr.o rdma_cm.o rdma_ucm.o
obj-$(CONFIG_INFINIBAND_USER_MAD) += ib_umad.o
obj-$(CONFIG_INFINIBAND_USER_ACCESS) += ib_uverbs.o ib_ucm.o
@@ -14,6 +14,8 @@ ib_cm-y := cm.o
rdma_cm-y := cma.o
+rdma_ucm-y := ucma.o
+
ib_addr-y := addr.o
ib_umad-y := user_mad.o
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/ucma.c
linux-2.6.ib/drivers/infiniband/core/ucma.c
--- linux-2.6.git/drivers/infiniband/core/ucma.c 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/ucma.c 2006-01-16 16:54:31.000000000 -0800
@@ -0,0 +1,788 @@
+/*
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <linux/poll.h>
+#include <linux/idr.h>
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/miscdevice.h>
+
+#include <rdma/rdma_user_cm.h>
+#include <rdma/ib_marshall.h>
+#include <rdma/rdma_cm.h>
+
+MODULE_AUTHOR("Sean Hefty");
+MODULE_DESCRIPTION("RDMA Userspace Connection Manager Access");
+MODULE_LICENSE("Dual BSD/GPL");
+
+enum {
+ UCMA_MAX_BACKLOG = 128
+};
+
+struct ucma_file {
+ struct mutex file_mutex;
+ struct file *filp;
+ struct list_head ctxs;
+ struct list_head events;
+ wait_queue_head_t poll_wait;
+};
+
+struct ucma_context {
+ int id;
+ wait_queue_head_t wait;
+ atomic_t ref;
+ int events_reported;
+ int backlog;
+
+ struct ucma_file *file;
+ struct rdma_cm_id *cm_id;
+ __u64 uid;
+
+ struct list_head events; /* list of pending events. */
+ struct list_head file_list; /* member in file ctx list */
+};
+
+struct ucma_event {
+ struct ucma_context *ctx;
+ struct list_head file_list; /* member in file event list */
+ struct list_head ctx_list; /* member in ctx event list */
+ struct rdma_cm_id *cm_id;
+ struct rdma_ucm_event_resp resp;
+};
+
+static DEFINE_MUTEX(ctx_mutex);
+static DEFINE_IDR(ctx_idr);
+
+static struct ucma_context* ucma_get_ctx(struct ucma_file *file, int id)
+{
+ struct ucma_context *ctx;
+
+ mutex_lock(&ctx_mutex);
+ ctx = idr_find(&ctx_idr, id);
+ if (!ctx)
+ ctx = ERR_PTR(-ENOENT);
+ else if (ctx->file != file)
+ ctx = ERR_PTR(-EINVAL);
+ else
+ atomic_inc(&ctx->ref);
+ mutex_unlock(&ctx_mutex);
+
+ return ctx;
+}
+
+static void ucma_put_ctx(struct ucma_context *ctx)
+{
+ if (atomic_dec_and_test(&ctx->ref))
+ wake_up(&ctx->wait);
+}
+
+static void ucma_cleanup_events(struct ucma_context *ctx)
+{
+ struct ucma_event *uevent;
+
+ mutex_lock(&ctx->file->file_mutex);
+ list_del(&ctx->file_list);
+ while (!list_empty(&ctx->events)) {
+
+ uevent = list_entry(ctx->events.next, struct ucma_event,
+ ctx_list);
+ list_del(&uevent->file_list);
+ list_del(&uevent->ctx_list);
+
+ /* clear incoming connections. */
+ if (uevent->resp.event == RDMA_CM_EVENT_CONNECT_REQUEST)
+ rdma_destroy_id(uevent->cm_id);
+
+ kfree(uevent);
+ }
+ mutex_unlock(&ctx->file->file_mutex);
+}
+
+static struct ucma_context* ucma_alloc_ctx(struct ucma_file *file)
+{
+ struct ucma_context *ctx;
+ int ret;
+
+ ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+ if (!ctx)
+ return NULL;
+
+ atomic_set(&ctx->ref, 1);
+ init_waitqueue_head(&ctx->wait);
+ ctx->file = file;
+ INIT_LIST_HEAD(&ctx->events);
+
+ do {
+ ret = idr_pre_get(&ctx_idr, GFP_KERNEL);
+ if (!ret)
+ goto error;
+
+ mutex_lock(&ctx_mutex);
+ ret = idr_get_new(&ctx_idr, ctx, &ctx->id);
+ mutex_unlock(&ctx_mutex);
+ } while (ret == -EAGAIN);
+
+ if (ret)
+ goto error;
+
+ list_add_tail(&ctx->file_list, &file->ctxs);
+ return ctx;
+
+error:
+ kfree(ctx);
+ return NULL;
+}
+
+static int ucma_event_handler(struct rdma_cm_id *cm_id,
+ struct rdma_cm_event *event)
+{
+ struct ucma_event *uevent;
+ struct ucma_context *ctx = cm_id->context;
+ int ret = 0;
+
+ uevent = kzalloc(sizeof(*uevent), GFP_KERNEL);
+ if (!uevent)
+ return event->event == RDMA_CM_EVENT_CONNECT_REQUEST;
+
+ uevent->ctx = ctx;
+ uevent->cm_id = cm_id;
+ uevent->resp.uid = ctx->uid;
+ uevent->resp.id = ctx->id;
+ uevent->resp.event = event->event;
+ uevent->resp.status = event->status;
+ if ((uevent->resp.private_data_len = event->private_data_len))
+ memcpy(uevent->resp.private_data, event->private_data,
+ event->private_data_len);
+
+ mutex_lock(&ctx->file->file_mutex);
+ if (event->event == RDMA_CM_EVENT_CONNECT_REQUEST) {
+ if (!ctx->backlog) {
+ ret = -EDQUOT;
+ goto out;
+ }
+ ctx->backlog--;
+ }
+ list_add_tail(&uevent->file_list, &ctx->file->events);
+ list_add_tail(&uevent->ctx_list, &ctx->events);
+ wake_up_interruptible(&ctx->file->poll_wait);
+out:
+ mutex_unlock(&ctx->file->file_mutex);
+ return ret;
+}
+
+static ssize_t ucma_get_event(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct ucma_context *ctx;
+ struct rdma_ucm_get_event cmd;
+ struct ucma_event *uevent;
+ int ret = 0;
+ DEFINE_WAIT(wait);
+
+ if (out_len < sizeof(struct rdma_ucm_event_resp))
+ return -ENOSPC;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ mutex_lock(&file->file_mutex);
+ while (list_empty(&file->events)) {
+ if (file->filp->f_flags & O_NONBLOCK) {
+ ret = -EAGAIN;
+ break;
+ }
+
+ if (signal_pending(current)) {
+ ret = -ERESTARTSYS;
+ break;
+ }
+
+ prepare_to_wait(&file->poll_wait, &wait, TASK_INTERRUPTIBLE);
+ mutex_unlock(&file->file_mutex);
+ schedule();
+ mutex_lock(&file->file_mutex);
+ finish_wait(&file->poll_wait, &wait);
+ }
+
+ if (ret)
+ goto done;
+
+ uevent = list_entry(file->events.next, struct ucma_event, file_list);
+
+ if (uevent->resp.event == RDMA_CM_EVENT_CONNECT_REQUEST) {
+ ctx = ucma_alloc_ctx(file);
+ if (!ctx) {
+ ret = -ENOMEM;
+ goto done;
+ }
+ uevent->ctx->backlog++;
+ ctx->cm_id = uevent->cm_id;
+ ctx->cm_id->context = ctx;
+ uevent->resp.id = ctx->id;
+ }
+
+ if (copy_to_user((void __user *)(unsigned long)cmd.response,
+ &uevent->resp, sizeof(uevent->resp))) {
+ ret = -EFAULT;
+ goto done;
+ }
+
+ list_del(&uevent->file_list);
+ list_del(&uevent->ctx_list);
+ uevent->ctx->events_reported++;
+ kfree(uevent);
+done:
+ mutex_unlock(&file->file_mutex);
+ return ret;
+}
+
+static ssize_t ucma_create_id(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_create_id cmd;
+ struct rdma_ucm_create_id_resp resp;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (out_len < sizeof(resp))
+ return -ENOSPC;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ mutex_lock(&file->file_mutex);
+ ctx = ucma_alloc_ctx(file);
+ mutex_unlock(&file->file_mutex);
+ if (!ctx)
+ return -ENOMEM;
+
+ ctx->uid = cmd.uid;
+ ctx->cm_id = rdma_create_id(ucma_event_handler, ctx, RDMA_PS_TCP);
+ if (IS_ERR(ctx->cm_id)) {
+ ret = PTR_ERR(ctx->cm_id);
+ goto err1;
+ }
+
+ resp.id = ctx->id;
+ if (copy_to_user((void __user *)(unsigned long)cmd.response,
+ &resp, sizeof(resp))) {
+ ret = -EFAULT;
+ goto err2;
+ }
+ return 0;
+
+err2:
+ rdma_destroy_id(ctx->cm_id);
+err1:
+ mutex_lock(&ctx_mutex);
+ idr_remove(&ctx_idr, ctx->id);
+ mutex_unlock(&ctx_mutex);
+ kfree(ctx);
+ return ret;
+}
+
+static ssize_t ucma_destroy_id(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_destroy_id cmd;
+ struct rdma_ucm_destroy_id_resp resp;
+ struct ucma_context *ctx;
+ int ret = 0;
+
+ if (out_len < sizeof(resp))
+ return -ENOSPC;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ mutex_lock(&ctx_mutex);
+ ctx = idr_find(&ctx_idr, cmd.id);
+ if (!ctx)
+ ctx = ERR_PTR(-ENOENT);
+ else if (ctx->file != file)
+ ctx = ERR_PTR(-EINVAL);
+ else
+ idr_remove(&ctx_idr, ctx->id);
+ mutex_unlock(&ctx_mutex);
+
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ atomic_dec(&ctx->ref);
+ wait_event(ctx->wait, !atomic_read(&ctx->ref));
+
+ /* No new events will be generated after destroying the id. */
+ rdma_destroy_id(ctx->cm_id);
+ /* Cleanup events not yet reported to the user. */
+ ucma_cleanup_events(ctx);
+
+ resp.events_reported = ctx->events_reported;
+ if (copy_to_user((void __user *)(unsigned long)cmd.response,
+ &resp, sizeof(resp)))
+ ret = -EFAULT;
+
+ kfree(ctx);
+ return ret;
+}
+
+static ssize_t ucma_bind_addr(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_bind_addr cmd;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ret = rdma_bind_addr(ctx->cm_id, (struct sockaddr *) &cmd.addr);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_resolve_addr(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_resolve_addr cmd;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ret = rdma_resolve_addr(ctx->cm_id, (struct sockaddr *) &cmd.src_addr,
+ (struct sockaddr *) &cmd.dst_addr,
+ cmd.timeout_ms);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_resolve_route(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_resolve_route cmd;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ret = rdma_resolve_route(ctx->cm_id, cmd.timeout_ms);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static void ucma_copy_ib_route(struct rdma_ucm_query_route_resp *resp,
+ struct rdma_route *route)
+{
+ struct rdma_dev_addr *dev_addr;
+
+ resp->num_paths = route->num_paths;
+ switch (route->num_paths) {
+ case 0:
+ dev_addr = &route->addr.dev_addr;
+ memcpy(&resp->ib_route[0].dgid, ib_addr_get_dgid(dev_addr),
+ sizeof(union ib_gid));
+ memcpy(&resp->ib_route[0].sgid, ib_addr_get_sgid(dev_addr),
+ sizeof(union ib_gid));
+ resp->ib_route[0].pkey = cpu_to_be16(ib_addr_get_pkey(dev_addr));
+ break;
+ case 2:
+ ib_copy_path_rec_to_user(&resp->ib_route[1],
+ &route->path_rec[1]);
+ /* fall through */
+ case 1:
+ ib_copy_path_rec_to_user(&resp->ib_route[0],
+ &route->path_rec[0]);
+ break;
+ default:
+ break;
+ }
+}
+
+static ssize_t ucma_query_route(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_query_route cmd;
+ struct rdma_ucm_query_route_resp resp;
+ struct ucma_context *ctx;
+ struct sockaddr *addr;
+ int ret = 0;
+
+ if (out_len < sizeof(resp))
+ return -ENOSPC;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ if (!ctx->cm_id->device) {
+ ret = -ENODEV;
+ goto out;
+ }
+
+ addr = &ctx->cm_id->route.addr.src_addr;
+ memcpy(&resp.src_addr, addr, addr->sa_family == AF_INET ?
+ sizeof(struct sockaddr_in) :
+ sizeof(struct sockaddr_in6));
+ addr = &ctx->cm_id->route.addr.dst_addr;
+ memcpy(&resp.dst_addr, addr, addr->sa_family == AF_INET ?
+ sizeof(struct sockaddr_in) :
+ sizeof(struct sockaddr_in6));
+ resp.node_guid = ctx->cm_id->device->node_guid;
+ resp.port_num = ctx->cm_id->port_num;
+ switch (ctx->cm_id->device->node_type) {
+ case IB_NODE_CA:
+ ucma_copy_ib_route(&resp, &ctx->cm_id->route);
+ default:
+ break;
+ }
+
+ if (copy_to_user((void __user *)(unsigned long)cmd.response,
+ &resp, sizeof(resp)))
+ ret = -EFAULT;
+
+out:
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static void ucma_copy_conn_param(struct rdma_conn_param *dst_conn,
+ struct rdma_ucm_conn_param *src_conn)
+{
+ dst_conn->private_data = src_conn->private_data;
+ dst_conn->private_data_len = src_conn->private_data_len;
+ dst_conn->responder_resources =src_conn->responder_resources;
+ dst_conn->initiator_depth = src_conn->initiator_depth;
+ dst_conn->flow_control = src_conn->flow_control;
+ dst_conn->retry_count = src_conn->retry_count;
+ dst_conn->rnr_retry_count = src_conn->rnr_retry_count;
+ dst_conn->srq = src_conn->srq;
+ dst_conn->qp_num = src_conn->qp_num;
+ dst_conn->qp_type = src_conn->qp_type;
+}
+
+static ssize_t ucma_connect(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_connect cmd;
+ struct rdma_conn_param conn_param;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ if (!cmd.conn_param.valid)
+ return -EINVAL;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ucma_copy_conn_param(&conn_param, &cmd.conn_param);
+ ret = rdma_connect(ctx->cm_id, &conn_param);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_listen(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_listen cmd;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ctx->backlog = cmd.backlog > 0 && cmd.backlog < UCMA_MAX_BACKLOG ?
+ cmd.backlog : UCMA_MAX_BACKLOG;
+ ret = rdma_listen(ctx->cm_id, ctx->backlog);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_accept(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_accept cmd;
+ struct rdma_conn_param conn_param;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ if (cmd.conn_param.valid) {
+ ctx->uid = cmd.uid;
+ ucma_copy_conn_param(&conn_param, &cmd.conn_param);
+ ret = rdma_accept(ctx->cm_id, &conn_param);
+ } else
+ ret = rdma_accept(ctx->cm_id, NULL);
+
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_reject(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_reject cmd;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ret = rdma_reject(ctx->cm_id, cmd.private_data, cmd.private_data_len);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_disconnect(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_disconnect cmd;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ret = rdma_disconnect(ctx->cm_id);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_init_qp_attr(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_init_qp_attr cmd;
+ struct ib_uverbs_qp_attr resp;
+ struct ucma_context *ctx;
+ struct ib_qp_attr qp_attr;
+ int ret;
+
+ if (out_len < sizeof(resp))
+ return -ENOSPC;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ resp.qp_attr_mask = 0;
+ memset(&qp_attr, 0, sizeof qp_attr);
+ qp_attr.qp_state = cmd.qp_state;
+ ret = rdma_init_qp_attr(ctx->cm_id, &qp_attr, &resp.qp_attr_mask);
+ if (ret)
+ goto out;
+
+ ib_copy_qp_attr_to_user(&resp, &qp_attr);
+ if (copy_to_user((void __user *)(unsigned long)cmd.response,
+ &resp, sizeof(resp)))
+ ret = -EFAULT;
+
+out:
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t (*ucma_cmd_table[])(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len) = {
+ [RDMA_USER_CM_CMD_CREATE_ID] = ucma_create_id,
+ [RDMA_USER_CM_CMD_DESTROY_ID] = ucma_destroy_id,
+ [RDMA_USER_CM_CMD_BIND_ADDR] = ucma_bind_addr,
+ [RDMA_USER_CM_CMD_RESOLVE_ADDR] = ucma_resolve_addr,
+ [RDMA_USER_CM_CMD_RESOLVE_ROUTE]= ucma_resolve_route,
+ [RDMA_USER_CM_CMD_QUERY_ROUTE] = ucma_query_route,
+ [RDMA_USER_CM_CMD_CONNECT] = ucma_connect,
+ [RDMA_USER_CM_CMD_LISTEN] = ucma_listen,
+ [RDMA_USER_CM_CMD_ACCEPT] = ucma_accept,
+ [RDMA_USER_CM_CMD_REJECT] = ucma_reject,
+ [RDMA_USER_CM_CMD_DISCONNECT] = ucma_disconnect,
+ [RDMA_USER_CM_CMD_INIT_QP_ATTR] = ucma_init_qp_attr,
+ [RDMA_USER_CM_CMD_GET_EVENT] = ucma_get_event
+};
+
+static ssize_t ucma_write(struct file *filp, const char __user *buf,
+ size_t len, loff_t *pos)
+{
+ struct ucma_file *file = filp->private_data;
+ struct rdma_ucm_cmd_hdr hdr;
+ ssize_t ret;
+
+ if (len < sizeof(hdr))
+ return -EINVAL;
+
+ if (copy_from_user(&hdr, buf, sizeof(hdr)))
+ return -EFAULT;
+
+ if (hdr.cmd < 0 || hdr.cmd >= ARRAY_SIZE(ucma_cmd_table))
+ return -EINVAL;
+
+ if (hdr.in + sizeof(hdr) > len)
+ return -EINVAL;
+
+ ret = ucma_cmd_table[hdr.cmd](file, buf + sizeof(hdr), hdr.in, hdr.out);
+ if (!ret)
+ ret = len;
+
+ return ret;
+}
+
+static unsigned int ucma_poll(struct file *filp, struct poll_table_struct *wait)
+{
+ struct ucma_file *file = filp->private_data;
+ unsigned int mask = 0;
+
+ poll_wait(filp, &file->poll_wait, wait);
+
+ mutex_lock(&file->file_mutex);
+ if (!list_empty(&file->events))
+ mask = POLLIN | POLLRDNORM;
+ mutex_unlock(&file->file_mutex);
+
+ return mask;
+}
+
+static int ucma_open(struct inode *inode, struct file *filp)
+{
+ struct ucma_file *file;
+
+ file = kmalloc(sizeof *file, GFP_KERNEL);
+ if (!file)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&file->events);
+ INIT_LIST_HEAD(&file->ctxs);
+ init_waitqueue_head(&file->poll_wait);
+ mutex_init(&file->file_mutex);
+
+ filp->private_data = file;
+ file->filp = filp;
+ return 0;
+}
+
+static int ucma_close(struct inode *inode, struct file *filp)
+{
+ struct ucma_file *file = filp->private_data;
+ struct ucma_context *ctx;
+
+ mutex_lock(&file->file_mutex);
+ while (!list_empty(&file->ctxs)) {
+ ctx = list_entry(file->ctxs.next, struct ucma_context,
+ file_list);
+ mutex_unlock(&file->file_mutex);
+
+ mutex_lock(&ctx_mutex);
+ idr_remove(&ctx_idr, ctx->id);
+ mutex_unlock(&ctx_mutex);
+
+ rdma_destroy_id(ctx->cm_id);
+ ucma_cleanup_events(ctx);
+ kfree(ctx);
+
+ mutex_lock(&file->file_mutex);
+ }
+ mutex_unlock(&file->file_mutex);
+ kfree(file);
+ return 0;
+}
+
+static struct file_operations ucma_fops = {
+ .owner = THIS_MODULE,
+ .open = ucma_open,
+ .release = ucma_close,
+ .write = ucma_write,
+ .poll = ucma_poll,
+};
+
+static struct miscdevice ucma_misc = {
+ .minor = MISC_DYNAMIC_MINOR,
+ .name = "rdma_cm",
+ .fops = &ucma_fops,
+};
+
+static int __init ucma_init(void)
+{
+ return misc_register(&ucma_misc);
+}
+
+static void __exit ucma_cleanup(void)
+{
+ misc_deregister(&ucma_misc);
+ idr_destroy(&ctx_idr);
+}
+
+module_init(ucma_init);
+module_exit(ucma_cleanup);
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/rdma_user_cm.h
linux-2.6.ib/include/rdma/rdma_user_cm.h
--- linux-2.6.git/include/rdma/rdma_user_cm.h 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/include/rdma/rdma_user_cm.h 2006-01-16 16:54:55.000000000 -0800
@@ -0,0 +1,186 @@
+/*
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef RDMA_USER_CM_H
+#define RDMA_USER_CM_H
+
+#include <linux/types.h>
+#include <linux/in6.h>
+#include <rdma/ib_user_verbs.h>
+#include <rdma/ib_user_sa.h>
+
+#define RDMA_USER_CM_ABI_VERSION 1
+
+#define RDMA_MAX_PRIVATE_DATA 256
+
+enum {
+ RDMA_USER_CM_CMD_CREATE_ID,
+ RDMA_USER_CM_CMD_DESTROY_ID,
+ RDMA_USER_CM_CMD_BIND_ADDR,
+ RDMA_USER_CM_CMD_RESOLVE_ADDR,
+ RDMA_USER_CM_CMD_RESOLVE_ROUTE,
+ RDMA_USER_CM_CMD_QUERY_ROUTE,
+ RDMA_USER_CM_CMD_CONNECT,
+ RDMA_USER_CM_CMD_LISTEN,
+ RDMA_USER_CM_CMD_ACCEPT,
+ RDMA_USER_CM_CMD_REJECT,
+ RDMA_USER_CM_CMD_DISCONNECT,
+ RDMA_USER_CM_CMD_INIT_QP_ATTR,
+ RDMA_USER_CM_CMD_GET_EVENT
+};
+
+/*
+ * command ABI structures.
+ */
+struct rdma_ucm_cmd_hdr {
+ __u32 cmd;
+ __u16 in;
+ __u16 out;
+};
+
+struct rdma_ucm_create_id {
+ __u64 uid;
+ __u64 response;
+};
+
+struct rdma_ucm_create_id_resp {
+ __u32 id;
+};
+
+struct rdma_ucm_destroy_id {
+ __u64 response;
+ __u32 id;
+ __u32 reserved;
+};
+
+struct rdma_ucm_destroy_id_resp {
+ __u32 events_reported;
+};
+
+struct rdma_ucm_bind_addr {
+ __u64 response;
+ struct sockaddr_in6 addr;
+ __u32 id;
+};
+
+struct rdma_ucm_resolve_addr {
+ struct sockaddr_in6 src_addr;
+ struct sockaddr_in6 dst_addr;
+ __u32 id;
+ __u32 timeout_ms;
+};
+
+struct rdma_ucm_resolve_route {
+ __u32 id;
+ __u32 timeout_ms;
+};
+
+struct rdma_ucm_query_route {
+ __u64 response;
+ __u32 id;
+ __u32 reserved;
+};
+
+struct rdma_ucm_query_route_resp {
+ __u64 node_guid;
+ struct ib_user_path_rec ib_route[2];
+ struct sockaddr_in6 src_addr;
+ struct sockaddr_in6 dst_addr;
+ __u32 num_paths;
+ __u8 port_num;
+ __u8 reserved[3];
+};
+
+struct rdma_ucm_conn_param {
+ __u32 qp_num;
+ __u32 qp_type;
+ __u8 private_data[RDMA_MAX_PRIVATE_DATA];
+ __u8 private_data_len;
+ __u8 srq;
+ __u8 responder_resources;
+ __u8 initiator_depth;
+ __u8 flow_control;
+ __u8 retry_count;
+ __u8 rnr_retry_count;
+ __u8 valid;
+};
+
+struct rdma_ucm_connect {
+ struct rdma_ucm_conn_param conn_param;
+ __u32 id;
+ __u32 reserved;
+};
+
+struct rdma_ucm_listen {
+ __u32 id;
+ __u32 backlog;
+};
+
+struct rdma_ucm_accept {
+ __u64 uid;
+ struct rdma_ucm_conn_param conn_param;
+ __u32 id;
+ __u32 reserved;
+};
+
+struct rdma_ucm_reject {
+ __u32 id;
+ __u8 private_data_len;
+ __u8 reserved[3];
+ __u8 private_data[RDMA_MAX_PRIVATE_DATA];
+};
+
+struct rdma_ucm_disconnect {
+ __u32 id;
+};
+
+struct rdma_ucm_init_qp_attr {
+ __u64 response;
+ __u32 id;
+ __u32 qp_state;
+};
+
+struct rdma_ucm_get_event {
+ __u64 response;
+};
+
+struct rdma_ucm_event_resp {
+ __u64 uid;
+ __u32 id;
+ __u32 event;
+ __u32 status;
+ __u8 private_data_len;
+ __u8 reserved[3];
+ __u8 private_data[RDMA_MAX_PRIVATE_DATA];
+};
+
+#endif /* RDMA_USER_CM_H */
^ permalink raw reply [flat|nested] 97+ messages in thread* [PATCH 0/5] Infiniband: connection abstraction
2006-02-01 20:03 [PATCH 0/5] Infiniband: connection abstraction Sean Hefty
` (4 preceding siblings ...)
2006-02-01 20:19 ` [PATCH 5/5] " Sean Hefty
@ 2006-03-03 21:13 ` Sean Hefty
2006-03-03 22:53 ` [openib-general] " Roland Dreier
5 siblings, 1 reply; 97+ messages in thread
From: Sean Hefty @ 2006-03-03 21:13 UTC (permalink / raw)
To: Hefty, Sean, netdev, linux-kernel; +Cc: openib-general
>Here's an updated version of these patches based on feedback. (The license
>did not change and continues to match that of the other Infiniband code.)
>Please consider for inclusion in 2.6.17.
This is just a ping for anymore feedback to this patch series, so that I can
respond to any requests before 2.6.17 opens up.
I can resubmit the patches if necessary.
Thanks,
Sean
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: [openib-general] [PATCH 0/5] Infiniband: connection abstraction
2006-03-03 21:13 ` [PATCH 0/5] " Sean Hefty
@ 2006-03-03 22:53 ` Roland Dreier
2006-03-06 18:59 ` [PATCH 1/6] IB: common handling for marshalling parameters to/from userspace Sean Hefty
` (5 more replies)
0 siblings, 6 replies; 97+ messages in thread
From: Roland Dreier @ 2006-03-03 22:53 UTC (permalink / raw)
To: Sean Hefty; +Cc: netdev, linux-kernel, openib-general
Sean> I can resubmit the patches if necessary.
Yes, can you please send out the patches again, with descriptive
subjects for each patch and with the ip_dev_find() re-export split
into its own patch? That would make it easier for me to pull into my
git tree.
Thanks,
Roland
^ permalink raw reply [flat|nested] 97+ messages in thread* [PATCH 1/6] IB: common handling for marshalling parameters to/from userspace
2006-03-03 22:53 ` [openib-general] " Roland Dreier
@ 2006-03-06 18:59 ` Sean Hefty
2006-03-19 1:19 ` 2.6.16-rc6-mm2: new RDMA CM EXPORT_SYMBOL's Andrew Morton
2006-03-06 19:04 ` [PATCH 2/6] IB: match connection requests based on private data Sean Hefty
` (4 subsequent siblings)
5 siblings, 1 reply; 97+ messages in thread
From: Sean Hefty @ 2006-03-06 18:59 UTC (permalink / raw)
To: 'Roland Dreier'; +Cc: netdev, linux-kernel, openib-general
Provide common handling for marshalling data between userspace clients
and kernel mode Infiniband drivers.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
---
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/Makefile
linux-2.6.ib/drivers/infiniband/core/Makefile
--- linux-2.6.git/drivers/infiniband/core/Makefile 2006-01-16 10:25:27.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/Makefile 2006-01-16 15:34:15.000000000 -0800
@@ -16,4 +16,5 @@ ib_umad-y := user_mad.o
ib_ucm-y := ucm.o
-ib_uverbs-y := uverbs_main.o uverbs_cmd.o uverbs_mem.o
+ib_uverbs-y := uverbs_main.o uverbs_cmd.o uverbs_mem.o \
+ uverbs_marshall.o
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/ucm.c
linux-2.6.ib/drivers/infiniband/core/ucm.c
--- linux-2.6.git/drivers/infiniband/core/ucm.c 2006-01-16 10:25:26.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/ucm.c 2006-01-16 15:34:15.000000000 -0800
@@ -30,7 +30,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- * $Id: ucm.c 2594 2005-06-13 19:46:02Z libor $
+ * $Id: ucm.c 4311 2005-12-05 18:42:01Z sean.hefty $
*/
#include <linux/init.h>
#include <linux/fs.h>
@@ -48,6 +48,7 @@
#include <rdma/ib_cm.h>
#include <rdma/ib_user_cm.h>
+#include <rdma/ib_marshall.h>
MODULE_AUTHOR("Libor Michalek");
MODULE_DESCRIPTION("InfiniBand userspace Connection Manager access");
@@ -203,36 +204,6 @@ error:
return NULL;
}
-static void ib_ucm_event_path_get(struct ib_ucm_path_rec *upath,
- struct ib_sa_path_rec *kpath)
-{
- if (!kpath || !upath)
- return;
-
- memcpy(upath->dgid, kpath->dgid.raw, sizeof *upath->dgid);
- memcpy(upath->sgid, kpath->sgid.raw, sizeof *upath->sgid);
-
- upath->dlid = kpath->dlid;
- upath->slid = kpath->slid;
- upath->raw_traffic = kpath->raw_traffic;
- upath->flow_label = kpath->flow_label;
- upath->hop_limit = kpath->hop_limit;
- upath->traffic_class = kpath->traffic_class;
- upath->reversible = kpath->reversible;
- upath->numb_path = kpath->numb_path;
- upath->pkey = kpath->pkey;
- upath->sl = kpath->sl;
- upath->mtu_selector = kpath->mtu_selector;
- upath->mtu = kpath->mtu;
- upath->rate_selector = kpath->rate_selector;
- upath->rate = kpath->rate;
- upath->packet_life_time = kpath->packet_life_time;
- upath->preference = kpath->preference;
-
- upath->packet_life_time_selector =
- kpath->packet_life_time_selector;
-}
-
static void ib_ucm_event_req_get(struct ib_ucm_req_event_resp *ureq,
struct ib_cm_req_event_param *kreq)
{
@@ -251,8 +222,10 @@ static void ib_ucm_event_req_get(struct
ureq->srq = kreq->srq;
ureq->port = kreq->port;
- ib_ucm_event_path_get(&ureq->primary_path, kreq->primary_path);
- ib_ucm_event_path_get(&ureq->alternate_path, kreq->alternate_path);
+ ib_copy_path_rec_to_user(&ureq->primary_path, kreq->primary_path);
+ if (kreq->alternate_path)
+ ib_copy_path_rec_to_user(&ureq->alternate_path,
+ kreq->alternate_path);
}
static void ib_ucm_event_rep_get(struct ib_ucm_rep_event_resp *urep,
@@ -322,8 +295,8 @@ static int ib_ucm_event_process(struct i
info = evt->param.rej_rcvd.ari;
break;
case IB_CM_LAP_RECEIVED:
- ib_ucm_event_path_get(&uvt->resp.u.lap_resp.path,
- evt->param.lap_rcvd.alternate_path);
+ ib_copy_path_rec_to_user(&uvt->resp.u.lap_resp.path,
+ evt->param.lap_rcvd.alternate_path);
uvt->data_len = IB_CM_LAP_PRIVATE_DATA_SIZE;
uvt->resp.present = IB_UCM_PRES_ALTERNATE;
break;
@@ -635,65 +608,11 @@ static ssize_t ib_ucm_attr_id(struct ib_
return result;
}
-static void ib_ucm_copy_ah_attr(struct ib_ucm_ah_attr *dest_attr,
- struct ib_ah_attr *src_attr)
-{
- memcpy(dest_attr->grh_dgid, src_attr->grh.dgid.raw,
- sizeof src_attr->grh.dgid);
- dest_attr->grh_flow_label = src_attr->grh.flow_label;
- dest_attr->grh_sgid_index = src_attr->grh.sgid_index;
- dest_attr->grh_hop_limit = src_attr->grh.hop_limit;
- dest_attr->grh_traffic_class = src_attr->grh.traffic_class;
-
- dest_attr->dlid = src_attr->dlid;
- dest_attr->sl = src_attr->sl;
- dest_attr->src_path_bits = src_attr->src_path_bits;
- dest_attr->static_rate = src_attr->static_rate;
- dest_attr->is_global = (src_attr->ah_flags & IB_AH_GRH);
- dest_attr->port_num = src_attr->port_num;
-}
-
-static void ib_ucm_copy_qp_attr(struct ib_ucm_init_qp_attr_resp *dest_attr,
- struct ib_qp_attr *src_attr)
-{
- dest_attr->cur_qp_state = src_attr->cur_qp_state;
- dest_attr->path_mtu = src_attr->path_mtu;
- dest_attr->path_mig_state = src_attr->path_mig_state;
- dest_attr->qkey = src_attr->qkey;
- dest_attr->rq_psn = src_attr->rq_psn;
- dest_attr->sq_psn = src_attr->sq_psn;
- dest_attr->dest_qp_num = src_attr->dest_qp_num;
- dest_attr->qp_access_flags = src_attr->qp_access_flags;
-
- dest_attr->max_send_wr = src_attr->cap.max_send_wr;
- dest_attr->max_recv_wr = src_attr->cap.max_recv_wr;
- dest_attr->max_send_sge = src_attr->cap.max_send_sge;
- dest_attr->max_recv_sge = src_attr->cap.max_recv_sge;
- dest_attr->max_inline_data = src_attr->cap.max_inline_data;
-
- ib_ucm_copy_ah_attr(&dest_attr->ah_attr, &src_attr->ah_attr);
- ib_ucm_copy_ah_attr(&dest_attr->alt_ah_attr, &src_attr->alt_ah_attr);
-
- dest_attr->pkey_index = src_attr->pkey_index;
- dest_attr->alt_pkey_index = src_attr->alt_pkey_index;
- dest_attr->en_sqd_async_notify = src_attr->en_sqd_async_notify;
- dest_attr->sq_draining = src_attr->sq_draining;
- dest_attr->max_rd_atomic = src_attr->max_rd_atomic;
- dest_attr->max_dest_rd_atomic = src_attr->max_dest_rd_atomic;
- dest_attr->min_rnr_timer = src_attr->min_rnr_timer;
- dest_attr->port_num = src_attr->port_num;
- dest_attr->timeout = src_attr->timeout;
- dest_attr->retry_cnt = src_attr->retry_cnt;
- dest_attr->rnr_retry = src_attr->rnr_retry;
- dest_attr->alt_port_num = src_attr->alt_port_num;
- dest_attr->alt_timeout = src_attr->alt_timeout;
-}
-
static ssize_t ib_ucm_init_qp_attr(struct ib_ucm_file *file,
const char __user *inbuf,
int in_len, int out_len)
{
- struct ib_ucm_init_qp_attr_resp resp;
+ struct ib_uverbs_qp_attr resp;
struct ib_ucm_init_qp_attr cmd;
struct ib_ucm_context *ctx;
struct ib_qp_attr qp_attr;
@@ -716,7 +635,7 @@ static ssize_t ib_ucm_init_qp_attr(struc
if (result)
goto out;
- ib_ucm_copy_qp_attr(&resp, &qp_attr);
+ ib_copy_qp_attr_to_user(&resp, &qp_attr);
if (copy_to_user((void __user *)(unsigned long)cmd.response,
&resp, sizeof(resp)))
@@ -791,7 +710,7 @@ static int ib_ucm_alloc_data(const void
static int ib_ucm_path_get(struct ib_sa_path_rec **path, u64 src)
{
- struct ib_ucm_path_rec ucm_path;
+ struct ib_user_path_rec upath;
struct ib_sa_path_rec *sa_path;
*path = NULL;
@@ -803,36 +722,14 @@ static int ib_ucm_path_get(struct ib_sa_
if (!sa_path)
return -ENOMEM;
- if (copy_from_user(&ucm_path, (void __user *)(unsigned long)src,
- sizeof(ucm_path))) {
+ if (copy_from_user(&upath, (void __user *)(unsigned long)src,
+ sizeof(upath))) {
kfree(sa_path);
return -EFAULT;
}
- memcpy(sa_path->dgid.raw, ucm_path.dgid, sizeof sa_path->dgid);
- memcpy(sa_path->sgid.raw, ucm_path.sgid, sizeof sa_path->sgid);
-
- sa_path->dlid = ucm_path.dlid;
- sa_path->slid = ucm_path.slid;
- sa_path->raw_traffic = ucm_path.raw_traffic;
- sa_path->flow_label = ucm_path.flow_label;
- sa_path->hop_limit = ucm_path.hop_limit;
- sa_path->traffic_class = ucm_path.traffic_class;
- sa_path->reversible = ucm_path.reversible;
- sa_path->numb_path = ucm_path.numb_path;
- sa_path->pkey = ucm_path.pkey;
- sa_path->sl = ucm_path.sl;
- sa_path->mtu_selector = ucm_path.mtu_selector;
- sa_path->mtu = ucm_path.mtu;
- sa_path->rate_selector = ucm_path.rate_selector;
- sa_path->rate = ucm_path.rate;
- sa_path->packet_life_time = ucm_path.packet_life_time;
- sa_path->preference = ucm_path.preference;
-
- sa_path->packet_life_time_selector =
- ucm_path.packet_life_time_selector;
-
+ ib_copy_path_rec_from_user(sa_path, &upath);
*path = sa_path;
return 0;
}
@@ -1243,8 +1140,10 @@ static unsigned int ib_ucm_poll(struct f
poll_wait(filp, &file->poll_wait, wait);
+ down(&file->mutex);
if (!list_empty(&file->events))
mask = POLLIN | POLLRDNORM;
+ up(&file->mutex);
return mask;
}
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/uverbs_marshall.c
linux-2.6.ib/drivers/infiniband/core/uverbs_marshall.c
--- linux-2.6.git/drivers/infiniband/core/uverbs_marshall.c 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/uverbs_marshall.c 2006-01-16 15:34:15.000000000 -0800
@@ -0,0 +1,138 @@
+/*
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <rdma/ib_marshall.h>
+
+static void ib_copy_ah_attr_to_user(struct ib_uverbs_ah_attr *dst,
+ struct ib_ah_attr *src)
+{
+ memcpy(dst->grh.dgid, src->grh.dgid.raw, sizeof src->grh.dgid);
+ dst->grh.flow_label = src->grh.flow_label;
+ dst->grh.sgid_index = src->grh.sgid_index;
+ dst->grh.hop_limit = src->grh.hop_limit;
+ dst->grh.traffic_class = src->grh.traffic_class;
+ dst->dlid = src->dlid;
+ dst->sl = src->sl;
+ dst->src_path_bits = src->src_path_bits;
+ dst->static_rate = src->static_rate;
+ dst->is_global = src->ah_flags & IB_AH_GRH ? 1 : 0;
+ dst->port_num = src->port_num;
+}
+
+void ib_copy_qp_attr_to_user(struct ib_uverbs_qp_attr *dst,
+ struct ib_qp_attr *src)
+{
+ dst->cur_qp_state = src->cur_qp_state;
+ dst->path_mtu = src->path_mtu;
+ dst->path_mig_state = src->path_mig_state;
+ dst->qkey = src->qkey;
+ dst->rq_psn = src->rq_psn;
+ dst->sq_psn = src->sq_psn;
+ dst->dest_qp_num = src->dest_qp_num;
+ dst->qp_access_flags = src->qp_access_flags;
+
+ dst->max_send_wr = src->cap.max_send_wr;
+ dst->max_recv_wr = src->cap.max_recv_wr;
+ dst->max_send_sge = src->cap.max_send_sge;
+ dst->max_recv_sge = src->cap.max_recv_sge;
+ dst->max_inline_data = src->cap.max_inline_data;
+
+ ib_copy_ah_attr_to_user(&dst->ah_attr, &src->ah_attr);
+ ib_copy_ah_attr_to_user(&dst->alt_ah_attr, &src->alt_ah_attr);
+
+ dst->pkey_index = src->pkey_index;
+ dst->alt_pkey_index = src->alt_pkey_index;
+ dst->en_sqd_async_notify = src->en_sqd_async_notify;
+ dst->sq_draining = src->sq_draining;
+ dst->max_rd_atomic = src->max_rd_atomic;
+ dst->max_dest_rd_atomic = src->max_dest_rd_atomic;
+ dst->min_rnr_timer = src->min_rnr_timer;
+ dst->port_num = src->port_num;
+ dst->timeout = src->timeout;
+ dst->retry_cnt = src->retry_cnt;
+ dst->rnr_retry = src->rnr_retry;
+ dst->alt_port_num = src->alt_port_num;
+ dst->alt_timeout = src->alt_timeout;
+}
+EXPORT_SYMBOL(ib_copy_qp_attr_to_user);
+
+void ib_copy_path_rec_to_user(struct ib_user_path_rec *dst,
+ struct ib_sa_path_rec *src)
+{
+ memcpy(dst->dgid, src->dgid.raw, sizeof src->dgid);
+ memcpy(dst->sgid, src->sgid.raw, sizeof src->sgid);
+
+ dst->dlid = src->dlid;
+ dst->slid = src->slid;
+ dst->raw_traffic = src->raw_traffic;
+ dst->flow_label = src->flow_label;
+ dst->hop_limit = src->hop_limit;
+ dst->traffic_class = src->traffic_class;
+ dst->reversible = src->reversible;
+ dst->numb_path = src->numb_path;
+ dst->pkey = src->pkey;
+ dst->sl = src->sl;
+ dst->mtu_selector = src->mtu_selector;
+ dst->mtu = src->mtu;
+ dst->rate_selector = src->rate_selector;
+ dst->rate = src->rate;
+ dst->packet_life_time = src->packet_life_time;
+ dst->preference = src->preference;
+ dst->packet_life_time_selector = src->packet_life_time_selector;
+}
+EXPORT_SYMBOL(ib_copy_path_rec_to_user);
+
+void ib_copy_path_rec_from_user(struct ib_sa_path_rec *dst,
+ struct ib_user_path_rec *src)
+{
+ memcpy(dst->dgid.raw, src->dgid, sizeof dst->dgid);
+ memcpy(dst->sgid.raw, src->sgid, sizeof dst->sgid);
+
+ dst->dlid = src->dlid;
+ dst->slid = src->slid;
+ dst->raw_traffic = src->raw_traffic;
+ dst->flow_label = src->flow_label;
+ dst->hop_limit = src->hop_limit;
+ dst->traffic_class = src->traffic_class;
+ dst->reversible = src->reversible;
+ dst->numb_path = src->numb_path;
+ dst->pkey = src->pkey;
+ dst->sl = src->sl;
+ dst->mtu_selector = src->mtu_selector;
+ dst->mtu = src->mtu;
+ dst->rate_selector = src->rate_selector;
+ dst->rate = src->rate;
+ dst->packet_life_time = src->packet_life_time;
+ dst->preference = src->preference;
+ dst->packet_life_time_selector = src->packet_life_time_selector;
+}
+EXPORT_SYMBOL(ib_copy_path_rec_from_user);
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/ib_marshall.h
linux-2.6.ib/include/rdma/ib_marshall.h
--- linux-2.6.git/include/rdma/ib_marshall.h 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/include/rdma/ib_marshall.h 2006-01-16 15:34:15.000000000 -0800
@@ -0,0 +1,50 @@
+/*
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#if !defined(IB_USER_MARSHALL_H)
+#define IB_USER_MARSHALL_H
+
+#include <rdma/ib_verbs.h>
+#include <rdma/ib_sa.h>
+#include <rdma/ib_user_verbs.h>
+#include <rdma/ib_user_sa.h>
+
+void ib_copy_qp_attr_to_user(struct ib_uverbs_qp_attr *dst,
+ struct ib_qp_attr *src);
+
+void ib_copy_path_rec_to_user(struct ib_user_path_rec *dst,
+ struct ib_sa_path_rec *src);
+
+void ib_copy_path_rec_from_user(struct ib_sa_path_rec *dst,
+ struct ib_user_path_rec *src);
+
+#endif /* IB_USER_MARSHALL_H */
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/ib_user_cm.h
linux-2.6.ib/include/rdma/ib_user_cm.h
--- linux-2.6.git/include/rdma/ib_user_cm.h 2006-01-16 10:26:47.000000000 -0800
+++ linux-2.6.ib/include/rdma/ib_user_cm.h 2006-01-16 15:34:15.000000000 -0800
@@ -30,13 +30,13 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- * $Id: ib_user_cm.h 2576 2005-06-09 17:00:30Z libor $
+ * $Id: ib_user_cm.h 4019 2005-11-11 00:33:09Z sean.hefty $
*/
#ifndef IB_USER_CM_H
#define IB_USER_CM_H
-#include <linux/types.h>
+#include <rdma/ib_user_sa.h>
#define IB_USER_CM_ABI_VERSION 4
@@ -110,58 +110,6 @@ struct ib_ucm_init_qp_attr {
__u32 qp_state;
};
-struct ib_ucm_ah_attr {
- __u8 grh_dgid[16];
- __u32 grh_flow_label;
- __u16 dlid;
- __u16 reserved;
- __u8 grh_sgid_index;
- __u8 grh_hop_limit;
- __u8 grh_traffic_class;
- __u8 sl;
- __u8 src_path_bits;
- __u8 static_rate;
- __u8 is_global;
- __u8 port_num;
-};
-
-struct ib_ucm_init_qp_attr_resp {
- __u32 qp_attr_mask;
- __u32 qp_state;
- __u32 cur_qp_state;
- __u32 path_mtu;
- __u32 path_mig_state;
- __u32 qkey;
- __u32 rq_psn;
- __u32 sq_psn;
- __u32 dest_qp_num;
- __u32 qp_access_flags;
-
- struct ib_ucm_ah_attr ah_attr;
- struct ib_ucm_ah_attr alt_ah_attr;
-
- /* ib_qp_cap */
- __u32 max_send_wr;
- __u32 max_recv_wr;
- __u32 max_send_sge;
- __u32 max_recv_sge;
- __u32 max_inline_data;
-
- __u16 pkey_index;
- __u16 alt_pkey_index;
- __u8 en_sqd_async_notify;
- __u8 sq_draining;
- __u8 max_rd_atomic;
- __u8 max_dest_rd_atomic;
- __u8 min_rnr_timer;
- __u8 port_num;
- __u8 timeout;
- __u8 retry_cnt;
- __u8 rnr_retry;
- __u8 alt_port_num;
- __u8 alt_timeout;
-};
-
struct ib_ucm_listen {
__be64 service_id;
__be64 service_mask;
@@ -180,28 +128,6 @@ struct ib_ucm_private_data {
__u8 reserved[3];
};
-struct ib_ucm_path_rec {
- __u8 dgid[16];
- __u8 sgid[16];
- __be16 dlid;
- __be16 slid;
- __u32 raw_traffic;
- __be32 flow_label;
- __u32 reversible;
- __u32 mtu;
- __be16 pkey;
- __u8 hop_limit;
- __u8 traffic_class;
- __u8 numb_path;
- __u8 sl;
- __u8 mtu_selector;
- __u8 rate_selector;
- __u8 rate;
- __u8 packet_life_time_selector;
- __u8 packet_life_time;
- __u8 preference;
-};
-
struct ib_ucm_req {
__u32 id;
__u32 qpn;
@@ -304,8 +230,8 @@ struct ib_ucm_event_get {
};
struct ib_ucm_req_event_resp {
- struct ib_ucm_path_rec primary_path;
- struct ib_ucm_path_rec alternate_path;
+ struct ib_user_path_rec primary_path;
+ struct ib_user_path_rec alternate_path;
__be64 remote_ca_guid;
__u32 remote_qkey;
__u32 remote_qpn;
@@ -349,7 +275,7 @@ struct ib_ucm_mra_event_resp {
};
struct ib_ucm_lap_event_resp {
- struct ib_ucm_path_rec path;
+ struct ib_user_path_rec path;
};
struct ib_ucm_apr_event_resp {
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/ib_user_sa.h
linux-2.6.ib/include/rdma/ib_user_sa.h
--- linux-2.6.git/include/rdma/ib_user_sa.h 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/include/rdma/ib_user_sa.h 2006-01-16 15:34:15.000000000 -0800
@@ -0,0 +1,60 @@
+/*
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef IB_USER_SA_H
+#define IB_USER_SA_H
+
+#include <linux/types.h>
+
+struct ib_user_path_rec {
+ __u8 dgid[16];
+ __u8 sgid[16];
+ __be16 dlid;
+ __be16 slid;
+ __u32 raw_traffic;
+ __be32 flow_label;
+ __u32 reversible;
+ __u32 mtu;
+ __be16 pkey;
+ __u8 hop_limit;
+ __u8 traffic_class;
+ __u8 numb_path;
+ __u8 sl;
+ __u8 mtu_selector;
+ __u8 rate_selector;
+ __u8 rate;
+ __u8 packet_life_time_selector;
+ __u8 packet_life_time;
+ __u8 preference;
+};
+
+#endif /* IB_USER_SA_H */
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/ib_user_verbs.h
linux-2.6.ib/include/rdma/ib_user_verbs.h
--- linux-2.6.git/include/rdma/ib_user_verbs.h 2006-01-16 10:26:47.000000000 -0800
+++ linux-2.6.ib/include/rdma/ib_user_verbs.h 2006-01-16 15:34:15.000000000 -0800
@@ -31,7 +31,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- * $Id: ib_user_verbs.h 2708 2005-06-24 17:27:21Z roland $
+ * $Id: ib_user_verbs.h 4019 2005-11-11 00:33:09Z sean.hefty $
*/
#ifndef IB_USER_VERBS_H
@@ -311,6 +311,64 @@ struct ib_uverbs_destroy_cq_resp {
__u32 async_events_reported;
};
+struct ib_uverbs_global_route {
+ __u8 dgid[16];
+ __u32 flow_label;
+ __u8 sgid_index;
+ __u8 hop_limit;
+ __u8 traffic_class;
+ __u8 reserved;
+};
+
+struct ib_uverbs_ah_attr {
+ struct ib_uverbs_global_route grh;
+ __u16 dlid;
+ __u8 sl;
+ __u8 src_path_bits;
+ __u8 static_rate;
+ __u8 is_global;
+ __u8 port_num;
+ __u8 reserved;
+};
+
+struct ib_uverbs_qp_attr {
+ __u32 qp_attr_mask;
+ __u32 qp_state;
+ __u32 cur_qp_state;
+ __u32 path_mtu;
+ __u32 path_mig_state;
+ __u32 qkey;
+ __u32 rq_psn;
+ __u32 sq_psn;
+ __u32 dest_qp_num;
+ __u32 qp_access_flags;
+
+ struct ib_uverbs_ah_attr ah_attr;
+ struct ib_uverbs_ah_attr alt_ah_attr;
+
+ /* ib_qp_cap */
+ __u32 max_send_wr;
+ __u32 max_recv_wr;
+ __u32 max_send_sge;
+ __u32 max_recv_sge;
+ __u32 max_inline_data;
+
+ __u16 pkey_index;
+ __u16 alt_pkey_index;
+ __u8 en_sqd_async_notify;
+ __u8 sq_draining;
+ __u8 max_rd_atomic;
+ __u8 max_dest_rd_atomic;
+ __u8 min_rnr_timer;
+ __u8 port_num;
+ __u8 timeout;
+ __u8 retry_cnt;
+ __u8 rnr_retry;
+ __u8 alt_port_num;
+ __u8 alt_timeout;
+ __u8 reserved[5];
+};
+
struct ib_uverbs_create_qp {
__u64 response;
__u64 user_handle;
@@ -487,26 +545,6 @@ struct ib_uverbs_post_srq_recv_resp {
__u32 bad_wr;
};
-struct ib_uverbs_global_route {
- __u8 dgid[16];
- __u32 flow_label;
- __u8 sgid_index;
- __u8 hop_limit;
- __u8 traffic_class;
- __u8 reserved;
-};
-
-struct ib_uverbs_ah_attr {
- struct ib_uverbs_global_route grh;
- __u16 dlid;
- __u8 sl;
- __u8 src_path_bits;
- __u8 static_rate;
- __u8 is_global;
- __u8 port_num;
- __u8 reserved;
-};
-
struct ib_uverbs_create_ah {
__u64 response;
__u64 user_handle;
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: 2.6.16-rc6-mm2: new RDMA CM EXPORT_SYMBOL's
2006-03-06 18:59 ` [PATCH 1/6] IB: common handling for marshalling parameters to/from userspace Sean Hefty
@ 2006-03-19 1:19 ` Andrew Morton
2006-03-19 4:11 ` Matthew Frost
2006-03-20 18:47 ` Roland Dreier
0 siblings, 2 replies; 97+ messages in thread
From: Andrew Morton @ 2006-03-19 1:19 UTC (permalink / raw)
To: Sean Hefty; +Cc: bunk, linux-kernel, rolandd, openib-general
"Sean Hefty" <sean.hefty@intel.com> wrote:
>
> >I'm not exactly happy that this tree adds tons of RDMA CM
> >EXPORT_SYMBOL's that are neither currently used nor _GPL.
>
> The symbols are used by the kernel component that provides userspace support.
>
There's brevity and then there is obscurity.
+EXPORT_SYMBOL(rdma_wq);
+EXPORT_SYMBOL(rdma_translate_ip);
+EXPORT_SYMBOL(rdma_resolve_ip);
+EXPORT_SYMBOL(rdma_addr_cancel);
+EXPORT_SYMBOL(rdma_create_id);
+EXPORT_SYMBOL(rdma_create_qp);
+EXPORT_SYMBOL(rdma_destroy_qp);
+EXPORT_SYMBOL(rdma_init_qp_attr);
+EXPORT_SYMBOL(rdma_destroy_id);
+EXPORT_SYMBOL(rdma_listen);
+EXPORT_SYMBOL(rdma_resolve_route);
+EXPORT_SYMBOL(rdma_resolve_addr);
+EXPORT_SYMBOL(rdma_bind_addr);
+EXPORT_SYMBOL(rdma_connect);
+EXPORT_SYMBOL(rdma_accept);
+EXPORT_SYMBOL(rdma_reject);
+EXPORT_SYMBOL(rdma_disconnect);
+EXPORT_SYMBOL(ib_get_rmpp_segment);
+EXPORT_SYMBOL(ib_copy_qp_attr_to_user);
+EXPORT_SYMBOL(ib_copy_path_rec_to_user);
+EXPORT_SYMBOL(ib_copy_path_rec_from_user);
+EXPORT_SYMBOL(ib_modify_qp_is_ok);
+EXPORT_SYMBOL(ip_dev_find);
Some or all of those exports have no in-tree users.
What code will use them?
Is it planned that this code be merged?
Please explain the thinking behind the choice of a non-GPL export. (Yes,
we discussed this when inifiniband was first merged, but it doesn't hurt to
reiterate).
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: 2.6.16-rc6-mm2: new RDMA CM EXPORT_SYMBOL's
2006-03-19 1:19 ` 2.6.16-rc6-mm2: new RDMA CM EXPORT_SYMBOL's Andrew Morton
@ 2006-03-19 4:11 ` Matthew Frost
2006-03-20 18:32 ` [openib-general] " Sean Hefty
2006-03-20 18:47 ` Roland Dreier
1 sibling, 1 reply; 97+ messages in thread
From: Matthew Frost @ 2006-03-19 4:11 UTC (permalink / raw)
To: Andrew Morton, Sean Hefty; +Cc: bunk, linux-kernel, rolandd, openib-general
--- Andrew Morton <akpm@osdl.org> wrote:
> "Sean Hefty" <sean.hefty@intel.com> wrote:
> >
> > >I'm not exactly happy that this tree adds tons of RDMA CM
> > >EXPORT_SYMBOL's that are neither currently used nor _GPL.
> >
> > The symbols are used by the kernel component that provides userspace
> support.
> >
>
> There's brevity and then there is obscurity.
>
To the point. I, insightful betimes, but a non-user of the technology,
can grep TFM's and find out what the names could mean, but we're left
guessing at what some of these *do*. Translating names falls into the
"any idiot can" category of data mining, but if you code for them, we can
see context. If you named them more transparently, we might even use
them right. Maybe just comment well?
RDMA
+EXPORT_SYMBOL(rdma_wq); Work Queue (do what to it?)
+EXPORT_SYMBOL(rdma_translate_ip); Translate IP Address
+EXPORT_SYMBOL(rdma_resolve_ip); Resolve IP Address
+EXPORT_SYMBOL(rdma_addr_cancel); Address Cancel (memory?)
+EXPORT_SYMBOL(rdma_create_id); Create (?) ID
+EXPORT_SYMBOL(rdma_create_qp); Create Queue Pair (WQ,CQ)
+EXPORT_SYMBOL(rdma_destroy_qp); Destroy Queue Pair (WQ,CQ)
+EXPORT_SYMBOL(rdma_init_qp_attr); Set Initial Queue Pair Attributes (?)
+EXPORT_SYMBOL(rdma_destroy_id); Destroy (?) ID
+EXPORT_SYMBOL(rdma_listen); Listen (to ... socket, port, pipe, what?)
+EXPORT_SYMBOL(rdma_resolve_route); Resolve Route (datagram path?)
+EXPORT_SYMBOL(rdma_resolve_addr); Resolve Address (memory?)
+EXPORT_SYMBOL(rdma_bind_addr); Bind Address (memory?)
+EXPORT_SYMBOL(rdma_connect); Connect
+EXPORT_SYMBOL(rdma_accept); Accept
+EXPORT_SYMBOL(rdma_reject); Reject
+EXPORT_SYMBOL(rdma_disconnect); Disconnect
Address vs. IP - I know we're talking about a net/dma kluge here, but the
twin usage is bugging me. I'm intuiting the _addr as memory addresses,
rather than IP addresses, which seem to be _ip, but my poor gray goo
suffers pointer overload.
INFINIBAND
+EXPORT_SYMBOL(ib_get_rmpp_segment); Reliable MultiPacket Protocol
+EXPORT_SYMBOL(ib_copy_qp_attr_to_user); Push Queue Pair Attribute
+EXPORT_SYMBOL(ib_copy_path_rec_to_user); Push Path Record
+EXPORT_SYMBOL(ib_copy_path_rec_from_user); Retrieve Path Record
+EXPORT_SYMBOL(ib_modify_qp_is_ok); Yes, Modify Queue Pair, or "QP is
OK", or "QP was Modified OK"?
+EXPORT_SYMBOL(ip_dev_find); Find IP device (sub(/ip/, "ib")? find the
network interface device?)
>
> Some or all of those exports have no in-tree users.
>
> What code will use them?
>
> Is it planned that this code be merged?
*Can* work that uses them be merged? Code that requires a non-GPL
export?
>
> Please explain the thinking behind the choice of a non-GPL export.
> (Yes, we discussed this when inifiniband was first merged, but it
> doesn't hurt to reiterate).
>
Color me curious ...
Frosty
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel"
> in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: [openib-general] Re: 2.6.16-rc6-mm2: new RDMA CM EXPORT_SYMBOL's
2006-03-19 4:11 ` Matthew Frost
@ 2006-03-20 18:32 ` Sean Hefty
2006-03-20 18:47 ` Arjan van de Ven
0 siblings, 1 reply; 97+ messages in thread
From: Sean Hefty @ 2006-03-20 18:32 UTC (permalink / raw)
To: Matthew Frost
Cc: Andrew Morton, Sean Hefty, linux-kernel, openib-general, bunk
Matthew Frost wrote:
> To the point. I, insightful betimes, but a non-user of the technology,
> can grep TFM's and find out what the names could mean, but we're left
> guessing at what some of these *do*. Translating names falls into the
> "any idiot can" category of data mining, but if you code for them, we can
> see context. If you named them more transparently, we might even use
> them right. Maybe just comment well?
Documentation in the form of comments are provided in the header files. I can
clarify the calls if needed. To help understand the structure of this patch,
three modules were submitted:
ib_addr - maps IP addresses to an RDMA device.
rdma_cm - Adds connection management over Infiniband using IP addressing. This
exports most of the symbols below.
rdma_ucm - Exports the rdma_cm functionality to userspace. This uses most of
the exported symbols (starting at rdma_create_id and below).
> +EXPORT_SYMBOL(rdma_wq); Work Queue (do what to it?)
This is used by ib_addr and rdma_cm modules to invoke user callbacks.
Additional code not yet ready for merging will also make use of this work queue.
The intent is to re-use this work queue, rather than each module creating
their own.
> +EXPORT_SYMBOL(rdma_translate_ip); Translate IP Address
> +EXPORT_SYMBOL(rdma_resolve_ip); Resolve IP Address
> +EXPORT_SYMBOL(rdma_addr_cancel); Address Cancel (memory?)
These exports are from the ib_addr module. The routines are called by the
rdma_cm module. The first two map an IP address to a local Infiniband device
address. Rdma_resolve_ip is an asynchronous call, so rdma_addr_cancel is used
to cancel its operation. 'rdma_cancel_resolve_ip' might have been a clearer name.
> +EXPORT_SYMBOL(rdma_create_id); Create (?) ID
The rdma_cm_id created by this call is required for the calls below.
Conceptually, it may help to think of an rdma_cm_id as somewhat like a socket.
> +EXPORT_SYMBOL(rdma_create_qp); Create Queue Pair (WQ,CQ)
> +EXPORT_SYMBOL(rdma_destroy_qp); Destroy Queue Pair (WQ,CQ)
'rdma_create_qp' associates a QP with an rdma_cm_id, so that the rdma_cm can
perform the QP transitions for the user during connection establishment.
> +EXPORT_SYMBOL(rdma_init_qp_attr); Set Initial Queue Pair Attributes (?)
This initializes the QP attributes for a user that wants to manually perform QP
transitions. It is provided mainly for userspace support.
> +EXPORT_SYMBOL(rdma_destroy_id); Destroy (?) ID
> +EXPORT_SYMBOL(rdma_listen); Listen (to ... socket, port, pipe, what?)
Listens across RDMA devices for connection requests. The listen is on an
rdma_cm_id.
> +EXPORT_SYMBOL(rdma_resolve_route); Resolve Route (datagram path?)
In Infiniband terms, this obtains a path record from the subnet manager. The
path record specifies the route through the subnet that packets between two
connected queue pairs will take.
> +EXPORT_SYMBOL(rdma_resolve_addr); Resolve Address (memory?)
This converts struct sockaddr to RDMA addresses. It ends up calling
rdma_translate_ip and rdma_resolve_ip, but performs some additional work to
handle device hotplug events.
> +EXPORT_SYMBOL(rdma_bind_addr); Bind Address (memory?)
Associates an rdma_cm_id to a specific struct sockaddr.
> +EXPORT_SYMBOL(rdma_connect); Connect
> +EXPORT_SYMBOL(rdma_accept); Accept
> +EXPORT_SYMBOL(rdma_reject); Reject
> +EXPORT_SYMBOL(rdma_disconnect); Disconnect
>
> Address vs. IP - I know we're talking about a net/dma kluge here, but the
> twin usage is bugging me. I'm intuiting the _addr as memory addresses,
> rather than IP addresses, which seem to be _ip, but my poor gray goo
> suffers pointer overload.
Maybe the naming is off here. I used _ip when referring specifically to an IP
address, and _addr when using a struct sockaddr. In some cases, such as
rdma_bind_addr, the 'address' may be nothing more than a port number.
> +EXPORT_SYMBOL(ib_get_rmpp_segment); Reliable MultiPacket Protocol
This is from a separate patch. It is exported by the ib_mad module, and used by
the ib_umad module. There is at least one out of tree module (not yet ready for
merging) that will make use of it.
> +EXPORT_SYMBOL(ib_copy_qp_attr_to_user); Push Queue Pair Attribute
> +EXPORT_SYMBOL(ib_copy_path_rec_to_user); Push Path Record
> +EXPORT_SYMBOL(ib_copy_path_rec_from_user); Retrieve Path Record
These are used by ib_uverbs, ib_ucm, and rdma_ucm modules.
> +EXPORT_SYMBOL(ib_modify_qp_is_ok); Yes, Modify Queue Pair, or "QP is
> OK", or "QP was Modified OK"?
This is from a separate patch. It should be exported by ib_verbs, and used by
ib_mthca. The call verifies that the settings to modify a QP from one state to
the next are valid. I believe that the check used to be part of the ib_mthca
driver itself, but additional kernel drivers, such as the ipath driver recently
submitted for inclusion, now make use of this routine.
> +EXPORT_SYMBOL(ip_dev_find); Find IP device (sub(/ip/, "ib")? find the
> network interface device?)
This is the network call in fib_frontend.c. It is being re-exported (the export
was removed a couple of versions ago) for use by ib_addr.
>>Please explain the thinking behind the choice of a non-GPL export.
>>(Yes, we discussed this when inifiniband was first merged, but it
>>doesn't hurt to reiterate).
The agreement made within the OpenIB community, from where this code originates,
is that all source code be licensed under a dual license of BSD/GPL. I am not a
lawyer, so I don't know the implications of changing the exports to be GPL only,
given the OpenIB license. But my understanding is that makes using those
functions less attractive.
- Sean
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: [openib-general] Re: 2.6.16-rc6-mm2: new RDMA CM EXPORT_SYMBOL's
2006-03-20 18:32 ` [openib-general] " Sean Hefty
@ 2006-03-20 18:47 ` Arjan van de Ven
0 siblings, 0 replies; 97+ messages in thread
From: Arjan van de Ven @ 2006-03-20 18:47 UTC (permalink / raw)
To: Sean Hefty
Cc: Matthew Frost, Andrew Morton, Sean Hefty, linux-kernel,
openib-general, bunk
> >>Please explain the thinking behind the choice of a non-GPL export.
> >>(Yes, we discussed this when inifiniband was first merged, but it
> >>doesn't hurt to reiterate).
>
> The agreement made within the OpenIB community, from where this code originates,
> is that all source code be licensed under a dual license of BSD/GPL. I am not a
> lawyer, so I don't know the implications of changing the exports to be GPL only,
> given the OpenIB license. But my understanding is that makes using those
> functions less attractive.
no actually ;)
but I understood OpenIB to be "GPL when used in the linux kernel,
optionally BSD when outside linux". Since EXPORT_SYMBOL is highly linux
kernel specific... the _GPL should be just fine
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: 2.6.16-rc6-mm2: new RDMA CM EXPORT_SYMBOL's
2006-03-19 1:19 ` 2.6.16-rc6-mm2: new RDMA CM EXPORT_SYMBOL's Andrew Morton
2006-03-19 4:11 ` Matthew Frost
@ 2006-03-20 18:47 ` Roland Dreier
2006-03-20 19:18 ` [openib-general] " Sean Hefty
1 sibling, 1 reply; 97+ messages in thread
From: Roland Dreier @ 2006-03-20 18:47 UTC (permalink / raw)
To: Andrew Morton; +Cc: Sean Hefty, bunk, linux-kernel, rolandd, openib-general
Putting this in -mm has generated more discussion than three postings
of the complete patch series to lkml. Andrew, please don't ever turn
your powers to evil ;)
Andrew> What code will use them?
Andrew> Is it planned that this code be merged?
Anyway, there are three main users of these APIs:
- A kernel module that allows userspace to use this IP-based RDMA
connection abstraction. It's my fault that this didn't make it
into -mm: I didn't think it was quite as baked as the kernel side,
and it's also more important to have the userspace interface stable
before merging it upstream, so I didn't put it in my git tree.
However, it should go into -mm, so I'll include it for next time.
- iSER ("iSCSI over IB"). The feeling I get from Christoph et al is
that this is pretty much ready to merge as well, but I'm not sure
if it's being staged anywhere.
- NFS/RDMA. This is not ready to merge yet but it is being worked on
with the idea of going upstream as soon as it's ready.
Looking at this list of exports, I do see a couple of things that
could maybe be improved:
> +EXPORT_SYMBOL(rdma_wq);
> +EXPORT_SYMBOL(rdma_translate_ip);
> +EXPORT_SYMBOL(rdma_resolve_ip);
> +EXPORT_SYMBOL(rdma_addr_cancel);
First, rdma_wq seems like a strange internal thing to be exporting.
Sean, why does more than one module need to use the same workqueue?
Second, the naming of rdma_addr_cancel() is not that symmetric with
the rdma_{translate,resolve}_ip() functions. Unfortunately I'm just
clever enough to criticize, not clever enough to come up with a better
suggestion.
Andrew> Please explain the thinking behind the choice of a non-GPL
Andrew> export. (Yes, we discussed this when inifiniband was
Andrew> first merged, but it doesn't hurt to reiterate).
I don't think there's any deep thought behind it, except that these
exports (for the most part) form a high-level coherent API and there's
no desire by the original author to restrict things.
ip_dev_find() was originally a non-GPL export, then removed for a
while. So it makes sense to just revert the first change.
- R.
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: [openib-general] Re: 2.6.16-rc6-mm2: new RDMA CM EXPORT_SYMBOL's
2006-03-20 18:47 ` Roland Dreier
@ 2006-03-20 19:18 ` Sean Hefty
0 siblings, 0 replies; 97+ messages in thread
From: Sean Hefty @ 2006-03-20 19:18 UTC (permalink / raw)
To: Roland Dreier; +Cc: Andrew Morton, linux-kernel, openib-general, bunk
Roland Dreier wrote:
> Looking at this list of exports, I do see a couple of things that
> could maybe be improved:
>
> > +EXPORT_SYMBOL(rdma_wq);
> > +EXPORT_SYMBOL(rdma_translate_ip);
> > +EXPORT_SYMBOL(rdma_resolve_ip);
> > +EXPORT_SYMBOL(rdma_addr_cancel);
>
> First, rdma_wq seems like a strange internal thing to be exporting.
> Sean, why does more than one module need to use the same workqueue?
This is simply an attempt to reduce/combine work queues used by the Infiniband
code. This keeps the threading a little simpler in the rdma_cm, since all
callbacks are invoked using the same work queue. (I'm also using this with the
local SA/multicast code, but that's not ready for merging.)
> Second, the naming of rdma_addr_cancel() is not that symmetric with
> the rdma_{translate,resolve}_ip() functions. Unfortunately I'm just
> clever enough to criticize, not clever enough to come up with a better
> suggestion.
rdma_resolve_ip() is the operation actually being canceled if that makes it
easier to come up with a better name.
- Sean
^ permalink raw reply [flat|nested] 97+ messages in thread
* [PATCH 2/6] IB: match connection requests based on private data
2006-03-03 22:53 ` [openib-general] " Roland Dreier
2006-03-06 18:59 ` [PATCH 1/6] IB: common handling for marshalling parameters to/from userspace Sean Hefty
@ 2006-03-06 19:04 ` Sean Hefty
2006-03-06 22:05 ` [openib-general] " Sean Hefty
2006-03-06 23:29 ` [PATCH 2/6 v2] " Sean Hefty
2006-03-06 19:07 ` [PATCH 3/6] net/IB: export ip_dev_find Sean Hefty
` (3 subsequent siblings)
5 siblings, 2 replies; 97+ messages in thread
From: Sean Hefty @ 2006-03-06 19:04 UTC (permalink / raw)
To: 'Roland Dreier'; +Cc: netdev, linux-kernel, openib-general
Extend matching connection requests to listens in the Infiniband CM to include
private data checks.
This allows applications to listen on the same service identifier, with private
data directing the request to the appropriate application.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
---
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/cm.c
linux-2.6.ib/drivers/infiniband/core/cm.c
--- linux-2.6.git/drivers/infiniband/core/cm.c 2006-01-16 10:25:26.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/cm.c 2006-01-16 16:03:35.000000000 -0800
@@ -32,7 +32,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- * $Id: cm.c 2821 2005-07-08 17:07:28Z sean.hefty $
+ * $Id: cm.c 4311 2005-12-05 18:42:01Z sean.hefty $
*/
#include <linux/dma-mapping.h>
#include <linux/err.h>
@@ -130,6 +130,7 @@ struct cm_id_private {
/* todo: use alternate port on send failure */
struct cm_av av;
struct cm_av alt_av;
+ struct ib_cm_private_data_compare *compare_data;
void *private_data;
__be64 tid;
@@ -355,6 +356,40 @@ static struct cm_id_private * cm_acquire
return cm_id_priv;
}
+static void cm_mask_compare_data(u8 *dst, u8 *src, u8 *mask)
+{
+ int i;
+
+ for (i = 0; i < IB_CM_PRIVATE_DATA_COMPARE_SIZE; i++)
+ dst[i] = src[i] & mask[i];
+}
+
+static int cm_compare_data(struct ib_cm_private_data_compare *src_data,
+ struct ib_cm_private_data_compare *dst_data)
+{
+ u8 src[IB_CM_PRIVATE_DATA_COMPARE_SIZE];
+ u8 dst[IB_CM_PRIVATE_DATA_COMPARE_SIZE];
+
+ if (!src_data || !dst_data)
+ return 0;
+
+ cm_mask_compare_data(src, src_data->data, dst_data->mask);
+ cm_mask_compare_data(dst, dst_data->data, src_data->mask);
+ return memcmp(src, dst, IB_CM_PRIVATE_DATA_COMPARE_SIZE);
+}
+
+static int cm_compare_private_data(u8 *private_data,
+ struct ib_cm_private_data_compare *dst_data)
+{
+ u8 src[IB_CM_PRIVATE_DATA_COMPARE_SIZE];
+
+ if (!dst_data)
+ return 0;
+
+ cm_mask_compare_data(src, private_data, dst_data->mask);
+ return memcmp(src, dst_data->data, IB_CM_PRIVATE_DATA_COMPARE_SIZE);
+}
+
static struct cm_id_private * cm_insert_listen(struct cm_id_private *cm_id_priv)
{
struct rb_node **link = &cm.listen_service_table.rb_node;
@@ -362,14 +397,18 @@ static struct cm_id_private * cm_insert_
struct cm_id_private *cur_cm_id_priv;
__be64 service_id = cm_id_priv->id.service_id;
__be64 service_mask = cm_id_priv->id.service_mask;
+ int data_cmp;
while (*link) {
parent = *link;
cur_cm_id_priv = rb_entry(parent, struct cm_id_private,
service_node);
+ data_cmp = cm_compare_data(cm_id_priv->compare_data,
+ cur_cm_id_priv->compare_data);
if ((cur_cm_id_priv->id.service_mask & service_id) ==
(service_mask & cur_cm_id_priv->id.service_id) &&
- (cm_id_priv->id.device == cur_cm_id_priv->id.device))
+ (cm_id_priv->id.device == cur_cm_id_priv->id.device) &&
+ !data_cmp)
return cur_cm_id_priv;
if (cm_id_priv->id.device < cur_cm_id_priv->id.device)
@@ -378,6 +417,10 @@ static struct cm_id_private * cm_insert_
link = &(*link)->rb_right;
else if (service_id < cur_cm_id_priv->id.service_id)
link = &(*link)->rb_left;
+ else if (service_id > cur_cm_id_priv->id.service_id)
+ link = &(*link)->rb_right;
+ else if (data_cmp < 0)
+ link = &(*link)->rb_left;
else
link = &(*link)->rb_right;
}
@@ -387,16 +430,20 @@ static struct cm_id_private * cm_insert_
}
static struct cm_id_private * cm_find_listen(struct ib_device *device,
- __be64 service_id)
+ __be64 service_id,
+ u8 *private_data)
{
struct rb_node *node = cm.listen_service_table.rb_node;
struct cm_id_private *cm_id_priv;
+ int data_cmp;
while (node) {
cm_id_priv = rb_entry(node, struct cm_id_private, service_node);
+ data_cmp = cm_compare_private_data(private_data,
+ cm_id_priv->compare_data);
if ((cm_id_priv->id.service_mask & service_id) ==
cm_id_priv->id.service_id &&
- (cm_id_priv->id.device == device))
+ (cm_id_priv->id.device == device) && !data_cmp)
return cm_id_priv;
if (device < cm_id_priv->id.device)
@@ -405,6 +452,10 @@ static struct cm_id_private * cm_find_li
node = node->rb_right;
else if (service_id < cm_id_priv->id.service_id)
node = node->rb_left;
+ else if (service_id > cm_id_priv->id.service_id)
+ node = node->rb_right;
+ else if (data_cmp < 0)
+ node = node->rb_left;
else
node = node->rb_right;
}
@@ -728,15 +779,14 @@ retest:
wait_event(cm_id_priv->wait, !atomic_read(&cm_id_priv->refcount));
while ((work = cm_dequeue_work(cm_id_priv)) != NULL)
cm_free_work(work);
- if (cm_id_priv->private_data && cm_id_priv->private_data_len)
- kfree(cm_id_priv->private_data);
+ kfree(cm_id_priv->compare_data);
+ kfree(cm_id_priv->private_data);
kfree(cm_id_priv);
}
EXPORT_SYMBOL(ib_destroy_cm_id);
-int ib_cm_listen(struct ib_cm_id *cm_id,
- __be64 service_id,
- __be64 service_mask)
+int ib_cm_listen(struct ib_cm_id *cm_id, __be64 service_id, __be64 service_mask,
+ struct ib_cm_private_data_compare *compare_data)
{
struct cm_id_private *cm_id_priv, *cur_cm_id_priv;
unsigned long flags;
@@ -750,7 +800,19 @@ int ib_cm_listen(struct ib_cm_id *cm_id,
return -EINVAL;
cm_id_priv = container_of(cm_id, struct cm_id_private, id);
- BUG_ON(cm_id->state != IB_CM_IDLE);
+ if (cm_id->state != IB_CM_IDLE)
+ return -EINVAL;
+
+ if (compare_data) {
+ cm_id_priv->compare_data = kzalloc(sizeof *compare_data,
+ GFP_KERNEL);
+ if (!cm_id_priv->compare_data)
+ return -ENOMEM;
+ cm_mask_compare_data(cm_id_priv->compare_data->data,
+ compare_data->data, compare_data->mask);
+ memcpy(cm_id_priv->compare_data->mask, compare_data->mask,
+ IB_CM_PRIVATE_DATA_COMPARE_SIZE);
+ }
cm_id->state = IB_CM_LISTEN;
@@ -767,6 +829,8 @@ int ib_cm_listen(struct ib_cm_id *cm_id,
if (cur_cm_id_priv) {
cm_id->state = IB_CM_IDLE;
+ kfree(cm_id_priv->compare_data);
+ cm_id_priv->compare_data = NULL;
ret = -EBUSY;
}
return ret;
@@ -1239,7 +1303,8 @@ static struct cm_id_private * cm_match_r
/* Find matching listen request. */
listen_cm_id_priv = cm_find_listen(cm_id_priv->id.device,
- req_msg->service_id);
+ req_msg->service_id,
+ req_msg->private_data);
if (!listen_cm_id_priv) {
spin_unlock_irqrestore(&cm.lock, flags);
cm_issue_rej(work->port, work->mad_recv_wc,
@@ -2646,7 +2711,8 @@ static int cm_sidr_req_handler(struct cm
goto out; /* Duplicate message. */
}
cur_cm_id_priv = cm_find_listen(cm_id->device,
- sidr_req_msg->service_id);
+ sidr_req_msg->service_id,
+ sidr_req_msg->private_data);
if (!cur_cm_id_priv) {
rb_erase(&cm_id_priv->sidr_id_node, &cm.remote_sidr_table);
spin_unlock_irqrestore(&cm.lock, flags);
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/ucm.c
linux-2.6.ib/drivers/infiniband/core/ucm.c
--- linux-2.6.git/drivers/infiniband/core/ucm.c 2006-01-16 16:03:08.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/ucm.c 2006-01-16 16:03:35.000000000 -0800
@@ -646,6 +646,17 @@ out:
return result;
}
+static int ucm_validate_listen(__be64 service_id, __be64 service_mask)
+{
+ service_id &= service_mask;
+
+ if (((service_id & IB_CMA_SERVICE_ID_MASK) == IB_CMA_SERVICE_ID) ||
+ ((service_id & IB_SDP_SERVICE_ID_MASK) == IB_SDP_SERVICE_ID))
+ return -EINVAL;
+
+ return 0;
+}
+
static ssize_t ib_ucm_listen(struct ib_ucm_file *file,
const char __user *inbuf,
int in_len, int out_len)
@@ -661,7 +672,13 @@ static ssize_t ib_ucm_listen(struct ib_u
if (IS_ERR(ctx))
return PTR_ERR(ctx);
- result = ib_cm_listen(ctx->cm_id, cmd.service_id, cmd.service_mask);
+ result = ucm_validate_listen(cmd.service_id, cmd.service_mask);
+ if (result)
+ goto out;
+
+ result = ib_cm_listen(ctx->cm_id, cmd.service_id, cmd.service_mask,
+ NULL);
+out:
ib_ucm_ctx_put(ctx);
return result;
}
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/ib_cm.h
linux-2.6.ib/include/rdma/ib_cm.h
--- linux-2.6.git/include/rdma/ib_cm.h 2006-01-16 10:26:47.000000000 -0800
+++ linux-2.6.ib/include/rdma/ib_cm.h 2006-01-16 16:03:35.000000000 -0800
@@ -32,7 +32,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- * $Id: ib_cm.h 2730 2005-06-28 16:43:03Z sean.hefty $
+ * $Id: ib_cm.h 4311 2005-12-05 18:42:01Z sean.hefty $
*/
#if !defined(IB_CM_H)
#define IB_CM_H
@@ -102,7 +102,8 @@ enum ib_cm_data_size {
IB_CM_APR_INFO_LENGTH = 72,
IB_CM_SIDR_REQ_PRIVATE_DATA_SIZE = 216,
IB_CM_SIDR_REP_PRIVATE_DATA_SIZE = 136,
- IB_CM_SIDR_REP_INFO_LENGTH = 72
+ IB_CM_SIDR_REP_INFO_LENGTH = 72,
+ IB_CM_PRIVATE_DATA_COMPARE_SIZE = 64
};
struct ib_cm_id;
@@ -238,7 +239,6 @@ struct ib_cm_sidr_rep_event_param {
u32 qpn;
void *info;
u8 info_len;
-
};
struct ib_cm_event {
@@ -317,6 +317,15 @@ void ib_destroy_cm_id(struct ib_cm_id *c
#define IB_SERVICE_ID_AGN_MASK __constant_cpu_to_be64(0xFF00000000000000ULL)
#define IB_CM_ASSIGN_SERVICE_ID __constant_cpu_to_be64(0x0200000000000000ULL)
+#define IB_CMA_SERVICE_ID __constant_cpu_to_be64(0x0000000001000000ULL)
+#define IB_CMA_SERVICE_ID_MASK __constant_cpu_to_be64(0xFFFFFFFFFF000000ULL)
+#define IB_SDP_SERVICE_ID __constant_cpu_to_be64(0x0000000000010000ULL)
+#define IB_SDP_SERVICE_ID_MASK __constant_cpu_to_be64(0xFFFFFFFFFFFF0000ULL)
+
+struct ib_cm_private_data_compare {
+ u8 data[IB_CM_PRIVATE_DATA_COMPARE_SIZE];
+ u8 mask[IB_CM_PRIVATE_DATA_COMPARE_SIZE];
+};
/**
* ib_cm_listen - Initiates listening on the specified service ID for
@@ -330,10 +339,12 @@ void ib_destroy_cm_id(struct ib_cm_id *c
* range of service IDs. If set to 0, the service ID is matched
* exactly. This parameter is ignored if %service_id is set to
* IB_CM_ASSIGN_SERVICE_ID.
+ * @compare_data: This parameter is optional. It specifies data that must
+ * appear in the private data of a connection request for the specified
+ * listen request.
*/
-int ib_cm_listen(struct ib_cm_id *cm_id,
- __be64 service_id,
- __be64 service_mask);
+int ib_cm_listen(struct ib_cm_id *cm_id, __be64 service_id, __be64 service_mask,
+ struct ib_cm_private_data_compare *compare_data);
struct ib_cm_req_param {
struct ib_sa_path_rec *primary_path;
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: [openib-general] [PATCH 2/6] IB: match connection requests based on private data
2006-03-06 19:04 ` [PATCH 2/6] IB: match connection requests based on private data Sean Hefty
@ 2006-03-06 22:05 ` Sean Hefty
2006-03-06 23:29 ` [PATCH 2/6 v2] " Sean Hefty
1 sibling, 0 replies; 97+ messages in thread
From: Sean Hefty @ 2006-03-06 22:05 UTC (permalink / raw)
To: Sean Hefty; +Cc: 'Roland Dreier', netdev, linux-kernel, openib-general
Sean Hefty wrote:
> +static void cm_mask_compare_data(u8 *dst, u8 *src, u8 *mask)
> +{
> + int i;
> +
> + for (i = 0; i < IB_CM_PRIVATE_DATA_COMPARE_SIZE; i++)
> + dst[i] = src[i] & mask[i];
> +}
> +
> +static int cm_compare_data(struct ib_cm_private_data_compare *src_data,
> + struct ib_cm_private_data_compare *dst_data)
> +{
> + u8 src[IB_CM_PRIVATE_DATA_COMPARE_SIZE];
> + u8 dst[IB_CM_PRIVATE_DATA_COMPARE_SIZE];
Ugh. I sent the wrong patch series. This was the original set of patches,
before any feedback was incorporated. I will need to resend patches 2, 4, 5,
and 6. Sorry about this.
- Sean
^ permalink raw reply [flat|nested] 97+ messages in thread* [PATCH 2/6 v2] IB: match connection requests based on private data
2006-03-06 19:04 ` [PATCH 2/6] IB: match connection requests based on private data Sean Hefty
2006-03-06 22:05 ` [openib-general] " Sean Hefty
@ 2006-03-06 23:29 ` Sean Hefty
1 sibling, 0 replies; 97+ messages in thread
From: Sean Hefty @ 2006-03-06 23:29 UTC (permalink / raw)
To: Hefty, Sean, 'Roland Dreier', netdev, linux-kernel; +Cc: openib-general
Extend matching connection requests to listens in the Infiniband CM to include
private data checks.
This allows applications to listen on the same service identifier, with private
data directing the request to the appropriate application.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
---
This should be the correct patch that incorporates feedback from the initial
submission. Sorry about the mis-post.
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/cm.c
linux-2.6.ib/drivers/infiniband/core/cm.c
--- linux-2.6.git/drivers/infiniband/core/cm.c 2006-01-16 10:25:26.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/cm.c 2006-01-16 16:03:35.000000000 -0800
@@ -32,7 +32,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- * $Id: cm.c 2821 2005-07-08 17:07:28Z sean.hefty $
+ * $Id: cm.c 4311 2005-12-05 18:42:01Z sean.hefty $
*/
#include <linux/dma-mapping.h>
#include <linux/err.h>
@@ -130,6 +130,7 @@ struct cm_id_private {
/* todo: use alternate port on send failure */
struct cm_av av;
struct cm_av alt_av;
+ struct ib_cm_compare_data *compare_data;
void *private_data;
__be64 tid;
@@ -355,6 +356,41 @@ static struct cm_id_private * cm_acquire
return cm_id_priv;
}
+static void cm_mask_copy(u8 *dst, u8 *src, u8 *mask)
+{
+ int i;
+
+ for (i = 0; i < IB_CM_COMPARE_SIZE / sizeof(unsigned long); i++)
+ ((unsigned long *) dst)[i] = ((unsigned long *) src)[i] &
+ ((unsigned long *) mask)[i];
+}
+
+static int cm_compare_data(struct ib_cm_compare_data *src_data,
+ struct ib_cm_compare_data *dst_data)
+{
+ u8 src[IB_CM_COMPARE_SIZE];
+ u8 dst[IB_CM_COMPARE_SIZE];
+
+ if (!src_data || !dst_data)
+ return 0;
+
+ cm_mask_copy(src, src_data->data, dst_data->mask);
+ cm_mask_copy(dst, dst_data->data, src_data->mask);
+ return memcmp(src, dst, IB_CM_COMPARE_SIZE);
+}
+
+static int cm_compare_private_data(u8 *private_data,
+ struct ib_cm_compare_data *dst_data)
+{
+ u8 src[IB_CM_COMPARE_SIZE];
+
+ if (!dst_data)
+ return 0;
+
+ cm_mask_copy(src, private_data, dst_data->mask);
+ return memcmp(src, dst_data->data, IB_CM_COMPARE_SIZE);
+}
+
static struct cm_id_private * cm_insert_listen(struct cm_id_private *cm_id_priv)
{
struct rb_node **link = &cm.listen_service_table.rb_node;
@@ -362,14 +397,18 @@ static struct cm_id_private * cm_insert_
struct cm_id_private *cur_cm_id_priv;
__be64 service_id = cm_id_priv->id.service_id;
__be64 service_mask = cm_id_priv->id.service_mask;
+ int data_cmp;
while (*link) {
parent = *link;
cur_cm_id_priv = rb_entry(parent, struct cm_id_private,
service_node);
+ data_cmp = cm_compare_data(cm_id_priv->compare_data,
+ cur_cm_id_priv->compare_data);
if ((cur_cm_id_priv->id.service_mask & service_id) ==
(service_mask & cur_cm_id_priv->id.service_id) &&
- (cm_id_priv->id.device == cur_cm_id_priv->id.device))
+ (cm_id_priv->id.device == cur_cm_id_priv->id.device) &&
+ !data_cmp)
return cur_cm_id_priv;
if (cm_id_priv->id.device < cur_cm_id_priv->id.device)
@@ -378,6 +417,10 @@ static struct cm_id_private * cm_insert_
link = &(*link)->rb_right;
else if (service_id < cur_cm_id_priv->id.service_id)
link = &(*link)->rb_left;
+ else if (service_id > cur_cm_id_priv->id.service_id)
+ link = &(*link)->rb_right;
+ else if (data_cmp < 0)
+ link = &(*link)->rb_left;
else
link = &(*link)->rb_right;
}
@@ -387,16 +430,20 @@ static struct cm_id_private * cm_insert_
}
static struct cm_id_private * cm_find_listen(struct ib_device *device,
- __be64 service_id)
+ __be64 service_id,
+ u8 *private_data)
{
struct rb_node *node = cm.listen_service_table.rb_node;
struct cm_id_private *cm_id_priv;
+ int data_cmp;
while (node) {
cm_id_priv = rb_entry(node, struct cm_id_private, service_node);
+ data_cmp = cm_compare_private_data(private_data,
+ cm_id_priv->compare_data);
if ((cm_id_priv->id.service_mask & service_id) ==
cm_id_priv->id.service_id &&
- (cm_id_priv->id.device == device))
+ (cm_id_priv->id.device == device) && !data_cmp)
return cm_id_priv;
if (device < cm_id_priv->id.device)
@@ -405,6 +452,10 @@ static struct cm_id_private * cm_find_li
node = node->rb_right;
else if (service_id < cm_id_priv->id.service_id)
node = node->rb_left;
+ else if (service_id > cm_id_priv->id.service_id)
+ node = node->rb_right;
+ else if (data_cmp < 0)
+ node = node->rb_left;
else
node = node->rb_right;
}
@@ -728,15 +779,14 @@ retest:
wait_event(cm_id_priv->wait, !atomic_read(&cm_id_priv->refcount));
while ((work = cm_dequeue_work(cm_id_priv)) != NULL)
cm_free_work(work);
- if (cm_id_priv->private_data && cm_id_priv->private_data_len)
- kfree(cm_id_priv->private_data);
+ kfree(cm_id_priv->compare_data);
+ kfree(cm_id_priv->private_data);
kfree(cm_id_priv);
}
EXPORT_SYMBOL(ib_destroy_cm_id);
-int ib_cm_listen(struct ib_cm_id *cm_id,
- __be64 service_id,
- __be64 service_mask)
+int ib_cm_listen(struct ib_cm_id *cm_id, __be64 service_id, __be64 service_mask,
+ struct ib_cm_compare_data *compare_data)
{
struct cm_id_private *cm_id_priv, *cur_cm_id_priv;
unsigned long flags;
@@ -750,7 +800,19 @@ int ib_cm_listen(struct ib_cm_id *cm_id,
return -EINVAL;
cm_id_priv = container_of(cm_id, struct cm_id_private, id);
- BUG_ON(cm_id->state != IB_CM_IDLE);
+ if (cm_id->state != IB_CM_IDLE)
+ return -EINVAL;
+
+ if (compare_data) {
+ cm_id_priv->compare_data = kzalloc(sizeof *compare_data,
+ GFP_KERNEL);
+ if (!cm_id_priv->compare_data)
+ return -ENOMEM;
+ cm_mask_copy(cm_id_priv->compare_data->data,
+ compare_data->data, compare_data->mask);
+ memcpy(cm_id_priv->compare_data->mask, compare_data->mask,
+ IB_CM_COMPARE_SIZE);
+ }
cm_id->state = IB_CM_LISTEN;
@@ -767,6 +829,8 @@ int ib_cm_listen(struct ib_cm_id *cm_id,
if (cur_cm_id_priv) {
cm_id->state = IB_CM_IDLE;
+ kfree(cm_id_priv->compare_data);
+ cm_id_priv->compare_data = NULL;
ret = -EBUSY;
}
return ret;
@@ -1239,7 +1303,8 @@ static struct cm_id_private * cm_match_r
/* Find matching listen request. */
listen_cm_id_priv = cm_find_listen(cm_id_priv->id.device,
- req_msg->service_id);
+ req_msg->service_id,
+ req_msg->private_data);
if (!listen_cm_id_priv) {
spin_unlock_irqrestore(&cm.lock, flags);
cm_issue_rej(work->port, work->mad_recv_wc,
@@ -2646,7 +2711,8 @@ static int cm_sidr_req_handler(struct cm
goto out; /* Duplicate message. */
}
cur_cm_id_priv = cm_find_listen(cm_id->device,
- sidr_req_msg->service_id);
+ sidr_req_msg->service_id,
+ sidr_req_msg->private_data);
if (!cur_cm_id_priv) {
rb_erase(&cm_id_priv->sidr_id_node, &cm.remote_sidr_table);
spin_unlock_irqrestore(&cm.lock, flags);
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/ucm.c
linux-2.6.ib/drivers/infiniband/core/ucm.c
--- linux-2.6.git/drivers/infiniband/core/ucm.c 2006-01-16 16:03:08.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/ucm.c 2006-01-16 16:03:35.000000000 -0800
@@ -646,6 +646,17 @@ out:
return result;
}
+static int ucm_validate_listen(__be64 service_id, __be64 service_mask)
+{
+ service_id &= service_mask;
+
+ if (((service_id & IB_CMA_SERVICE_ID_MASK) == IB_CMA_SERVICE_ID) ||
+ ((service_id & IB_SDP_SERVICE_ID_MASK) == IB_SDP_SERVICE_ID))
+ return -EINVAL;
+
+ return 0;
+}
+
static ssize_t ib_ucm_listen(struct ib_ucm_file *file,
const char __user *inbuf,
int in_len, int out_len)
@@ -661,7 +672,13 @@ static ssize_t ib_ucm_listen(struct ib_u
if (IS_ERR(ctx))
return PTR_ERR(ctx);
- result = ib_cm_listen(ctx->cm_id, cmd.service_id, cmd.service_mask);
+ result = ucm_validate_listen(cmd.service_id, cmd.service_mask);
+ if (result)
+ goto out;
+
+ result = ib_cm_listen(ctx->cm_id, cmd.service_id, cmd.service_mask,
+ NULL);
+out:
ib_ucm_ctx_put(ctx);
return result;
}
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/ib_cm.h
linux-2.6.ib/include/rdma/ib_cm.h
--- linux-2.6.git/include/rdma/ib_cm.h 2006-01-16 10:26:47.000000000 -0800
+++ linux-2.6.ib/include/rdma/ib_cm.h 2006-01-16 16:03:35.000000000 -0800
@@ -32,7 +32,7 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
- * $Id: ib_cm.h 2730 2005-06-28 16:43:03Z sean.hefty $
+ * $Id: ib_cm.h 4311 2005-12-05 18:42:01Z sean.hefty $
*/
#if !defined(IB_CM_H)
#define IB_CM_H
@@ -102,7 +102,8 @@ enum ib_cm_data_size {
IB_CM_APR_INFO_LENGTH = 72,
IB_CM_SIDR_REQ_PRIVATE_DATA_SIZE = 216,
IB_CM_SIDR_REP_PRIVATE_DATA_SIZE = 136,
- IB_CM_SIDR_REP_INFO_LENGTH = 72
+ IB_CM_SIDR_REP_INFO_LENGTH = 72,
+ IB_CM_COMPARE_SIZE = 64
};
struct ib_cm_id;
@@ -238,7 +239,6 @@ struct ib_cm_sidr_rep_event_param {
u32 qpn;
void *info;
u8 info_len;
-
};
struct ib_cm_event {
@@ -317,6 +317,15 @@ void ib_destroy_cm_id(struct ib_cm_id *c
#define IB_SERVICE_ID_AGN_MASK __constant_cpu_to_be64(0xFF00000000000000ULL)
#define IB_CM_ASSIGN_SERVICE_ID __constant_cpu_to_be64(0x0200000000000000ULL)
+#define IB_CMA_SERVICE_ID __constant_cpu_to_be64(0x0000000001000000ULL)
+#define IB_CMA_SERVICE_ID_MASK __constant_cpu_to_be64(0xFFFFFFFFFF000000ULL)
+#define IB_SDP_SERVICE_ID __constant_cpu_to_be64(0x0000000000010000ULL)
+#define IB_SDP_SERVICE_ID_MASK __constant_cpu_to_be64(0xFFFFFFFFFFFF0000ULL)
+
+struct ib_cm_compare_data {
+ u8 data[IB_CM_COMPARE_SIZE];
+ u8 mask[IB_CM_COMPARE_SIZE];
+};
/**
* ib_cm_listen - Initiates listening on the specified service ID for
@@ -330,10 +339,12 @@ void ib_destroy_cm_id(struct ib_cm_id *c
* range of service IDs. If set to 0, the service ID is matched
* exactly. This parameter is ignored if %service_id is set to
* IB_CM_ASSIGN_SERVICE_ID.
+ * @compare_data: This parameter is optional. It specifies data that must
+ * appear in the private data of a connection request for the specified
+ * listen request.
*/
-int ib_cm_listen(struct ib_cm_id *cm_id,
- __be64 service_id,
- __be64 service_mask);
+int ib_cm_listen(struct ib_cm_id *cm_id, __be64 service_id, __be64 service_mask,
+ struct ib_cm_compare_data *compare_data);
struct ib_cm_req_param {
struct ib_sa_path_rec *primary_path;
^ permalink raw reply [flat|nested] 97+ messages in thread
* [PATCH 3/6] net/IB: export ip_dev_find
2006-03-03 22:53 ` [openib-general] " Roland Dreier
2006-03-06 18:59 ` [PATCH 1/6] IB: common handling for marshalling parameters to/from userspace Sean Hefty
2006-03-06 19:04 ` [PATCH 2/6] IB: match connection requests based on private data Sean Hefty
@ 2006-03-06 19:07 ` Sean Hefty
2006-03-06 21:31 ` Roland Dreier
2006-03-06 19:10 ` [PATCH 4/6] IB: address translation to map IP to IB addresses (GIDs) Sean Hefty
` (2 subsequent siblings)
5 siblings, 1 reply; 97+ messages in thread
From: Sean Hefty @ 2006-03-06 19:07 UTC (permalink / raw)
To: 'Roland Dreier', linux-kernel, netdev; +Cc: openib-general
Export ip_dev_find to allow locating a net_device given an IP address.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
---
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/net/ipv4/fib_frontend.c
linux-2.6.ib/net/ipv4/fib_frontend.c
--- linux-2.6.git/net/ipv4/fib_frontend.c 2006-01-16 10:28:29.000000000 -0800
+++ linux-2.6.ib/net/ipv4/fib_frontend.c 2006-01-16 16:14:24.000000000 -0800
@@ -666,4 +666,5 @@ void __init ip_fib_init(void)
}
EXPORT_SYMBOL(inet_addr_type);
+EXPORT_SYMBOL(ip_dev_find);
EXPORT_SYMBOL(ip_rt_ioctl);
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: [PATCH 3/6] net/IB: export ip_dev_find
2006-03-06 19:07 ` [PATCH 3/6] net/IB: export ip_dev_find Sean Hefty
@ 2006-03-06 21:31 ` Roland Dreier
2006-03-06 21:42 ` David S. Miller
0 siblings, 1 reply; 97+ messages in thread
From: Roland Dreier @ 2006-03-06 21:31 UTC (permalink / raw)
To: Sean Hefty; +Cc: linux-kernel, netdev, openib-general
Sean> Export ip_dev_find to allow locating a net_device given an
Sean> IP address.
My plan is to queue all of this stuff for merging in 2.6.17.
Is there any objection from netdev or openib-general people?
I just looked back, and the original "unexport ip_dev_find()" patch
was a de-Bunk-ing change. Now that there is a modular user, is there
any problem with re-exporting it?
Thanks,
Roland
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: [PATCH 3/6] net/IB: export ip_dev_find
2006-03-06 21:31 ` Roland Dreier
@ 2006-03-06 21:42 ` David S. Miller
0 siblings, 0 replies; 97+ messages in thread
From: David S. Miller @ 2006-03-06 21:42 UTC (permalink / raw)
To: rdreier; +Cc: sean.hefty, linux-kernel, netdev, openib-general
From: Roland Dreier <rdreier@cisco.com>
Date: Mon, 06 Mar 2006 13:31:05 -0800
> Sean> Export ip_dev_find to allow locating a net_device given an
> Sean> IP address.
>
> My plan is to queue all of this stuff for merging in 2.6.17.
>
> Is there any objection from netdev or openib-general people?
>
> I just looked back, and the original "unexport ip_dev_find()" patch
> was a de-Bunk-ing change. Now that there is a modular user, is there
> any problem with re-exporting it?
I'm fine with re-exporting it.
^ permalink raw reply [flat|nested] 97+ messages in thread
* [PATCH 4/6] IB: address translation to map IP to IB addresses (GIDs)
2006-03-03 22:53 ` [openib-general] " Roland Dreier
` (2 preceding siblings ...)
2006-03-06 19:07 ` [PATCH 3/6] net/IB: export ip_dev_find Sean Hefty
@ 2006-03-06 19:10 ` Sean Hefty
2006-03-06 23:31 ` [PATCH 4/6 v2] IB: address translation to map IP toIB " Sean Hefty
2006-03-06 19:18 ` [PATCH 5/6] IB: IP address based RDMA connection manager Sean Hefty
2006-03-06 19:21 ` [PATCH 6/6] IB: userspace support for " Sean Hefty
5 siblings, 1 reply; 97+ messages in thread
From: Sean Hefty @ 2006-03-06 19:10 UTC (permalink / raw)
To: 'Roland Dreier', linux-kernel, netdev; +Cc: openib-general
Add an address translation service that maps IP addresses to Infiniband
GID addresses using IPoIB.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
---
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/addr.c
linux-2.6.ib/drivers/infiniband/core/addr.c
--- linux-2.6.git/drivers/infiniband/core/addr.c 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/addr.c 2006-01-16 16:14:24.000000000 -0800
@@ -0,0 +1,356 @@
+/*
+ * Copyright (c) 2005 Voltaire Inc. All rights reserved.
+ * Copyright (c) 2002-2005, Network Appliance, Inc. All rights reserved.
+ * Copyright (c) 1999-2005, Mellanox Technologies, Inc. All rights reserved.
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ */
+#include <linux/inetdevice.h>
+#include <linux/workqueue.h>
+#include <net/arp.h>
+#include <net/neighbour.h>
+#include <net/route.h>
+#include <rdma/ib_addr.h>
+
+MODULE_AUTHOR("Sean Hefty");
+MODULE_DESCRIPTION("IB Address Translation");
+MODULE_LICENSE("Dual BSD/GPL");
+
+struct addr_req {
+ struct list_head list;
+ struct sockaddr src_addr;
+ struct sockaddr dst_addr;
+ struct rdma_dev_addr *addr;
+ void *context;
+ void (*callback)(int status, struct sockaddr *src_addr,
+ struct rdma_dev_addr *addr, void *context);
+ unsigned long timeout;
+ int status;
+};
+
+static void process_req(void *data);
+
+static DECLARE_MUTEX(mutex);
+static LIST_HEAD(req_list);
+static DECLARE_WORK(work, process_req, NULL);
+struct workqueue_struct *rdma_wq;
+EXPORT_SYMBOL(rdma_wq);
+
+static int copy_addr(struct rdma_dev_addr *dev_addr, struct net_device *dev,
+ unsigned char *dst_dev_addr)
+{
+ switch (dev->type) {
+ case ARPHRD_INFINIBAND:
+ dev_addr->dev_type = IB_NODE_CA;
+ break;
+ default:
+ return -EADDRNOTAVAIL;
+ }
+
+ memcpy(dev_addr->src_dev_addr, dev->dev_addr, MAX_ADDR_LEN);
+ memcpy(dev_addr->broadcast, dev->broadcast, MAX_ADDR_LEN);
+ if (dst_dev_addr)
+ memcpy(dev_addr->dst_dev_addr, dst_dev_addr, MAX_ADDR_LEN);
+ return 0;
+}
+
+int rdma_translate_ip(struct sockaddr *addr, struct rdma_dev_addr *dev_addr)
+{
+ struct net_device *dev;
+ u32 ip = ((struct sockaddr_in *) addr)->sin_addr.s_addr;
+ int ret;
+
+ dev = ip_dev_find(ip);
+ if (!dev)
+ return -EADDRNOTAVAIL;
+
+ ret = copy_addr(dev_addr, dev, NULL);
+ dev_put(dev);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_translate_ip);
+
+static void set_timeout(unsigned long time)
+{
+ unsigned long delay;
+
+ cancel_delayed_work(&work);
+
+ delay = time - jiffies;
+ if ((long)delay <= 0)
+ delay = 1;
+
+ queue_delayed_work(rdma_wq, &work, delay);
+}
+
+static void queue_req(struct addr_req *req)
+{
+ struct addr_req *temp_req;
+
+ down(&mutex);
+ list_for_each_entry_reverse(temp_req, &req_list, list) {
+ if (time_after(req->timeout, temp_req->timeout))
+ break;
+ }
+
+ list_add(&req->list, &temp_req->list);
+
+ if (req_list.next == &req->list)
+ set_timeout(req->timeout);
+ up(&mutex);
+}
+
+static void addr_send_arp(struct sockaddr_in *dst_in)
+{
+ struct rtable *rt;
+ struct flowi fl;
+ u32 dst_ip = dst_in->sin_addr.s_addr;
+
+ memset(&fl, 0, sizeof fl);
+ fl.nl_u.ip4_u.daddr = dst_ip;
+ if (ip_route_output_key(&rt, &fl))
+ return;
+
+ arp_send(ARPOP_REQUEST, ETH_P_ARP, rt->rt_gateway, rt->idev->dev,
+ rt->rt_src, NULL, rt->idev->dev->dev_addr, NULL);
+ ip_rt_put(rt);
+}
+
+static int addr_resolve_remote(struct sockaddr_in *src_in,
+ struct sockaddr_in *dst_in,
+ struct rdma_dev_addr *addr)
+{
+ u32 src_ip = src_in->sin_addr.s_addr;
+ u32 dst_ip = dst_in->sin_addr.s_addr;
+ struct flowi fl;
+ struct rtable *rt;
+ struct neighbour *neigh;
+ int ret;
+
+ memset(&fl, 0, sizeof fl);
+ fl.nl_u.ip4_u.daddr = dst_ip;
+ fl.nl_u.ip4_u.saddr = src_ip;
+ ret = ip_route_output_key(&rt, &fl);
+ if (ret)
+ goto out;
+
+ neigh = neigh_lookup(&arp_tbl, &rt->rt_gateway, rt->idev->dev);
+ if (!neigh) {
+ ret = -ENODATA;
+ goto err1;
+ }
+
+ if (!(neigh->nud_state & NUD_VALID)) {
+ ret = -ENODATA;
+ goto err2;
+ }
+
+ if (!src_ip) {
+ src_in->sin_family = dst_in->sin_family;
+ src_in->sin_addr.s_addr = rt->rt_src;
+ }
+
+ ret = copy_addr(addr, neigh->dev, neigh->ha);
+err2:
+ neigh_release(neigh);
+err1:
+ ip_rt_put(rt);
+out:
+ return ret;
+}
+
+static void process_req(void *data)
+{
+ struct addr_req *req, *temp_req;
+ struct sockaddr_in *src_in, *dst_in;
+ struct list_head done_list;
+
+ INIT_LIST_HEAD(&done_list);
+
+ down(&mutex);
+ list_for_each_entry_safe(req, temp_req, &req_list, list) {
+ if (req->status) {
+ src_in = (struct sockaddr_in *) &req->src_addr;
+ dst_in = (struct sockaddr_in *) &req->dst_addr;
+ req->status = addr_resolve_remote(src_in, dst_in,
+ req->addr);
+ }
+ if (req->status && time_after(jiffies, req->timeout))
+ req->status = -ETIMEDOUT;
+ else if (req->status == -ENODATA)
+ continue;
+
+ list_del(&req->list);
+ list_add_tail(&req->list, &done_list);
+ }
+
+ if (!list_empty(&req_list)) {
+ req = list_entry(req_list.next, struct addr_req, list);
+ set_timeout(req->timeout);
+ }
+ up(&mutex);
+
+ list_for_each_entry_safe(req, temp_req, &done_list, list) {
+ list_del(&req->list);
+ req->callback(req->status, &req->src_addr, req->addr,
+ req->context);
+ kfree(req);
+ }
+}
+
+static int addr_resolve_local(struct sockaddr_in *src_in,
+ struct sockaddr_in *dst_in,
+ struct rdma_dev_addr *addr)
+{
+ struct net_device *dev;
+ u32 src_ip = src_in->sin_addr.s_addr;
+ u32 dst_ip = dst_in->sin_addr.s_addr;
+ int ret;
+
+ dev = ip_dev_find(dst_ip);
+ if (!dev)
+ return -EADDRNOTAVAIL;
+
+ if (!src_ip) {
+ src_in->sin_family = dst_in->sin_family;
+ src_in->sin_addr.s_addr = dst_ip;
+ ret = copy_addr(addr, dev, dev->dev_addr);
+ } else {
+ ret = rdma_translate_ip((struct sockaddr *)src_in, addr);
+ if (!ret)
+ memcpy(addr->dst_dev_addr, dev->dev_addr, MAX_ADDR_LEN);
+ }
+
+ dev_put(dev);
+ return ret;
+}
+
+int rdma_resolve_ip(struct sockaddr *src_addr, struct sockaddr *dst_addr,
+ struct rdma_dev_addr *addr, int timeout_ms,
+ void (*callback)(int status, struct sockaddr *src_addr,
+ struct rdma_dev_addr *addr, void *context),
+ void *context)
+{
+ struct sockaddr_in *src_in, *dst_in;
+ struct addr_req *req;
+ int ret = 0;
+
+ req = kmalloc(sizeof *req, GFP_KERNEL);
+ if (!req)
+ return -ENOMEM;
+ memset(req, 0, sizeof *req);
+
+ if (src_addr)
+ memcpy(&req->src_addr, src_addr, ip_addr_size(src_addr));
+ memcpy(&req->dst_addr, dst_addr, ip_addr_size(dst_addr));
+ req->addr = addr;
+ req->callback = callback;
+ req->context = context;
+
+ src_in = (struct sockaddr_in *) &req->src_addr;
+ dst_in = (struct sockaddr_in *) &req->dst_addr;
+
+ req->status = addr_resolve_local(src_in, dst_in, addr);
+ if (req->status == -EADDRNOTAVAIL)
+ req->status = addr_resolve_remote(src_in, dst_in, addr);
+
+ switch (req->status) {
+ case 0:
+ req->timeout = jiffies;
+ queue_req(req);
+ break;
+ case -ENODATA:
+ req->timeout = msecs_to_jiffies(timeout_ms) + jiffies;
+ queue_req(req);
+ addr_send_arp(dst_in);
+ break;
+ default:
+ ret = req->status;
+ kfree(req);
+ break;
+ }
+ return ret;
+}
+EXPORT_SYMBOL(rdma_resolve_ip);
+
+void rdma_addr_cancel(struct rdma_dev_addr *addr)
+{
+ struct addr_req *req, *temp_req;
+
+ up(&mutex);
+ list_for_each_entry_safe(req, temp_req, &req_list, list) {
+ if (req->addr == addr) {
+ req->status = -ECANCELED;
+ req->timeout = jiffies;
+ list_del(&req->list);
+ list_add(&req->list, &req_list);
+ set_timeout(req->timeout);
+ break;
+ }
+ }
+ up(&mutex);
+}
+EXPORT_SYMBOL(rdma_addr_cancel);
+
+static int addr_arp_recv(struct sk_buff *skb, struct net_device *dev,
+ struct packet_type *pkt, struct net_device *orig_dev)
+{
+ struct arphdr *arp_hdr;
+
+ arp_hdr = (struct arphdr *) skb->nh.raw;
+
+ if (dev->type == ARPHRD_INFINIBAND &&
+ (arp_hdr->ar_op == __constant_htons(ARPOP_REQUEST) ||
+ arp_hdr->ar_op == __constant_htons(ARPOP_REPLY)))
+ set_timeout(jiffies);
+
+ kfree_skb(skb);
+ return 0;
+}
+
+static struct packet_type addr_arp = {
+ .type = __constant_htons(ETH_P_ARP),
+ .func = addr_arp_recv,
+ .af_packet_priv = (void*) 1,
+};
+
+static int addr_init(void)
+{
+ rdma_wq = create_singlethread_workqueue("rdma_wq");
+ if (!rdma_wq)
+ return -ENOMEM;
+
+ dev_add_pack(&addr_arp);
+ return 0;
+}
+
+static void addr_cleanup(void)
+{
+ dev_remove_pack(&addr_arp);
+ destroy_workqueue(rdma_wq);
+}
+
+module_init(addr_init);
+module_exit(addr_cleanup);
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/Makefile
linux-2.6.ib/drivers/infiniband/core/Makefile
--- linux-2.6.git/drivers/infiniband/core/Makefile 2006-01-16 16:03:08.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/Makefile 2006-01-16 16:14:24.000000000 -0800
@@ -1,5 +1,5 @@
obj-$(CONFIG_INFINIBAND) += ib_core.o ib_mad.o ib_sa.o \
- ib_cm.o
+ ib_cm.o ib_addr.o
obj-$(CONFIG_INFINIBAND_USER_MAD) += ib_umad.o
obj-$(CONFIG_INFINIBAND_USER_ACCESS) += ib_uverbs.o ib_ucm.o
@@ -12,6 +12,8 @@ ib_sa-y := sa_query.o
ib_cm-y := cm.o
+ib_addr-y := addr.o
+
ib_umad-y := user_mad.o
ib_ucm-y := ucm.o
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/ib_addr.h
linux-2.6.ib/include/rdma/ib_addr.h
--- linux-2.6.git/include/rdma/ib_addr.h 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/include/rdma/ib_addr.h 2006-01-16 16:14:24.000000000 -0800
@@ -0,0 +1,117 @@
+/*
+ * Copyright (c) 2005 Voltaire Inc. All rights reserved.
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ *
+ */
+
+#if !defined(IB_ADDR_H)
+#define IB_ADDR_H
+
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/netdevice.h>
+#include <linux/socket.h>
+#include <rdma/ib_verbs.h>
+
+extern struct workqueue_struct *rdma_wq;
+
+struct rdma_dev_addr {
+ unsigned char src_dev_addr[MAX_ADDR_LEN];
+ unsigned char dst_dev_addr[MAX_ADDR_LEN];
+ unsigned char broadcast[MAX_ADDR_LEN];
+ enum ib_node_type dev_type;
+};
+
+/**
+ * rdma_translate_ip - Translate a local IP address to an RDMA hardware
+ * address.
+ */
+int rdma_translate_ip(struct sockaddr *addr, struct rdma_dev_addr *dev_addr);
+
+/**
+ * rdma_resolve_ip - Resolve source and destination IP addresses to
+ * RDMA hardware addresses.
+ * @src_addr: An optional source address to use in the resolution. If a
+ * source address is not provided, a usable address will be returned via
+ * the callback.
+ * @dst_addr: The destination address to resolve.
+ * @addr: A reference to a data location that will receive the resolved
+ * addresses. The data location must remain valid until the callback has
+ * been invoked.
+ * @timeout_ms: Amount of time to wait for the address resolution to complete.
+ * @callback: Call invoked once address resolution has completed, timed out,
+ * or been canceled. A status of 0 indicates success.
+ * @context: User-specified context associated with the call.
+ */
+int rdma_resolve_ip(struct sockaddr *src_addr, struct sockaddr *dst_addr,
+ struct rdma_dev_addr *addr, int timeout_ms,
+ void (*callback)(int status, struct sockaddr *src_addr,
+ struct rdma_dev_addr *addr, void *context),
+ void *context);
+
+void rdma_addr_cancel(struct rdma_dev_addr *addr);
+
+static inline int ip_addr_size(struct sockaddr *addr)
+{
+ return addr->sa_family == AF_INET6 ?
+ sizeof(struct sockaddr_in6) : sizeof(struct sockaddr_in);
+}
+
+static inline u16 ib_addr_get_pkey(struct rdma_dev_addr *dev_addr)
+{
+ return ((u16)dev_addr->broadcast[8] << 8) | (u16)dev_addr->broadcast[9];
+}
+
+static inline void ib_addr_set_pkey(struct rdma_dev_addr *dev_addr, u16 pkey)
+{
+ dev_addr->broadcast[8] = pkey >> 8;
+ dev_addr->broadcast[9] = (unsigned char) pkey;
+}
+
+static inline union ib_gid* ib_addr_get_sgid(struct rdma_dev_addr *dev_addr)
+{
+ return (union ib_gid *) (dev_addr->src_dev_addr + 4);
+}
+
+static inline void ib_addr_set_sgid(struct rdma_dev_addr *dev_addr,
+ union ib_gid *gid)
+{
+ memcpy(dev_addr->src_dev_addr + 4, gid, sizeof *gid);
+}
+
+static inline union ib_gid* ib_addr_get_dgid(struct rdma_dev_addr *dev_addr)
+{
+ return (union ib_gid *) (dev_addr->dst_dev_addr + 4);
+}
+
+static inline void ib_addr_set_dgid(struct rdma_dev_addr *dev_addr,
+ union ib_gid *gid)
+{
+ memcpy(dev_addr->dst_dev_addr + 4, gid, sizeof *gid);
+}
+
+#endif /* IB_ADDR_H */
+
^ permalink raw reply [flat|nested] 97+ messages in thread* [PATCH 4/6 v2] IB: address translation to map IP toIB addresses (GIDs)
2006-03-06 19:10 ` [PATCH 4/6] IB: address translation to map IP to IB addresses (GIDs) Sean Hefty
@ 2006-03-06 23:31 ` Sean Hefty
2006-03-11 1:14 ` Roland Dreier
2006-03-21 20:57 ` Roland Dreier
0 siblings, 2 replies; 97+ messages in thread
From: Sean Hefty @ 2006-03-06 23:31 UTC (permalink / raw)
To: Hefty, Sean, 'Roland Dreier', linux-kernel, netdev; +Cc: openib-general
Add an address translation service that maps IP addresses to Infiniband
GID addresses using IPoIB.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
---
This should be the correct patch. The only difference between this and the mis-post
is the use of mutex_lock/unlock in place of up/down.
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/addr.c
linux-2.6.ib/drivers/infiniband/core/addr.c
--- linux-2.6.git/drivers/infiniband/core/addr.c 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/addr.c 2006-01-16 16:14:24.000000000 -0800
@@ -0,0 +1,356 @@
+/*
+ * Copyright (c) 2005 Voltaire Inc. All rights reserved.
+ * Copyright (c) 2002-2005, Network Appliance, Inc. All rights reserved.
+ * Copyright (c) 1999-2005, Mellanox Technologies, Inc. All rights reserved.
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ */
+#include <linux/inetdevice.h>
+#include <linux/workqueue.h>
+#include <net/arp.h>
+#include <net/neighbour.h>
+#include <net/route.h>
+#include <rdma/ib_addr.h>
+
+MODULE_AUTHOR("Sean Hefty");
+MODULE_DESCRIPTION("IB Address Translation");
+MODULE_LICENSE("Dual BSD/GPL");
+
+struct addr_req {
+ struct list_head list;
+ struct sockaddr src_addr;
+ struct sockaddr dst_addr;
+ struct rdma_dev_addr *addr;
+ void *context;
+ void (*callback)(int status, struct sockaddr *src_addr,
+ struct rdma_dev_addr *addr, void *context);
+ unsigned long timeout;
+ int status;
+};
+
+static void process_req(void *data);
+
+static DEFINE_MUTEX(lock);
+static LIST_HEAD(req_list);
+static DECLARE_WORK(work, process_req, NULL);
+struct workqueue_struct *rdma_wq;
+EXPORT_SYMBOL(rdma_wq);
+
+static int copy_addr(struct rdma_dev_addr *dev_addr, struct net_device *dev,
+ unsigned char *dst_dev_addr)
+{
+ switch (dev->type) {
+ case ARPHRD_INFINIBAND:
+ dev_addr->dev_type = IB_NODE_CA;
+ break;
+ default:
+ return -EADDRNOTAVAIL;
+ }
+
+ memcpy(dev_addr->src_dev_addr, dev->dev_addr, MAX_ADDR_LEN);
+ memcpy(dev_addr->broadcast, dev->broadcast, MAX_ADDR_LEN);
+ if (dst_dev_addr)
+ memcpy(dev_addr->dst_dev_addr, dst_dev_addr, MAX_ADDR_LEN);
+ return 0;
+}
+
+int rdma_translate_ip(struct sockaddr *addr, struct rdma_dev_addr *dev_addr)
+{
+ struct net_device *dev;
+ u32 ip = ((struct sockaddr_in *) addr)->sin_addr.s_addr;
+ int ret;
+
+ dev = ip_dev_find(ip);
+ if (!dev)
+ return -EADDRNOTAVAIL;
+
+ ret = copy_addr(dev_addr, dev, NULL);
+ dev_put(dev);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_translate_ip);
+
+static void set_timeout(unsigned long time)
+{
+ unsigned long delay;
+
+ cancel_delayed_work(&work);
+
+ delay = time - jiffies;
+ if ((long)delay <= 0)
+ delay = 1;
+
+ queue_delayed_work(rdma_wq, &work, delay);
+}
+
+static void queue_req(struct addr_req *req)
+{
+ struct addr_req *temp_req;
+
+ mutex_lock(&lock);
+ list_for_each_entry_reverse(temp_req, &req_list, list) {
+ if (time_after(req->timeout, temp_req->timeout))
+ break;
+ }
+
+ list_add(&req->list, &temp_req->list);
+
+ if (req_list.next == &req->list)
+ set_timeout(req->timeout);
+ mutex_unlock(&lock);
+}
+
+static void addr_send_arp(struct sockaddr_in *dst_in)
+{
+ struct rtable *rt;
+ struct flowi fl;
+ u32 dst_ip = dst_in->sin_addr.s_addr;
+
+ memset(&fl, 0, sizeof fl);
+ fl.nl_u.ip4_u.daddr = dst_ip;
+ if (ip_route_output_key(&rt, &fl))
+ return;
+
+ arp_send(ARPOP_REQUEST, ETH_P_ARP, rt->rt_gateway, rt->idev->dev,
+ rt->rt_src, NULL, rt->idev->dev->dev_addr, NULL);
+ ip_rt_put(rt);
+}
+
+static int addr_resolve_remote(struct sockaddr_in *src_in,
+ struct sockaddr_in *dst_in,
+ struct rdma_dev_addr *addr)
+{
+ u32 src_ip = src_in->sin_addr.s_addr;
+ u32 dst_ip = dst_in->sin_addr.s_addr;
+ struct flowi fl;
+ struct rtable *rt;
+ struct neighbour *neigh;
+ int ret;
+
+ memset(&fl, 0, sizeof fl);
+ fl.nl_u.ip4_u.daddr = dst_ip;
+ fl.nl_u.ip4_u.saddr = src_ip;
+ ret = ip_route_output_key(&rt, &fl);
+ if (ret)
+ goto out;
+
+ neigh = neigh_lookup(&arp_tbl, &rt->rt_gateway, rt->idev->dev);
+ if (!neigh) {
+ ret = -ENODATA;
+ goto err1;
+ }
+
+ if (!(neigh->nud_state & NUD_VALID)) {
+ ret = -ENODATA;
+ goto err2;
+ }
+
+ if (!src_ip) {
+ src_in->sin_family = dst_in->sin_family;
+ src_in->sin_addr.s_addr = rt->rt_src;
+ }
+
+ ret = copy_addr(addr, neigh->dev, neigh->ha);
+err2:
+ neigh_release(neigh);
+err1:
+ ip_rt_put(rt);
+out:
+ return ret;
+}
+
+static void process_req(void *data)
+{
+ struct addr_req *req, *temp_req;
+ struct sockaddr_in *src_in, *dst_in;
+ struct list_head done_list;
+
+ INIT_LIST_HEAD(&done_list);
+
+ mutex_lock(&lock);
+ list_for_each_entry_safe(req, temp_req, &req_list, list) {
+ if (req->status) {
+ src_in = (struct sockaddr_in *) &req->src_addr;
+ dst_in = (struct sockaddr_in *) &req->dst_addr;
+ req->status = addr_resolve_remote(src_in, dst_in,
+ req->addr);
+ }
+ if (req->status && time_after(jiffies, req->timeout))
+ req->status = -ETIMEDOUT;
+ else if (req->status == -ENODATA)
+ continue;
+
+ list_del(&req->list);
+ list_add_tail(&req->list, &done_list);
+ }
+
+ if (!list_empty(&req_list)) {
+ req = list_entry(req_list.next, struct addr_req, list);
+ set_timeout(req->timeout);
+ }
+ mutex_unlock(&lock);
+
+ list_for_each_entry_safe(req, temp_req, &done_list, list) {
+ list_del(&req->list);
+ req->callback(req->status, &req->src_addr, req->addr,
+ req->context);
+ kfree(req);
+ }
+}
+
+static int addr_resolve_local(struct sockaddr_in *src_in,
+ struct sockaddr_in *dst_in,
+ struct rdma_dev_addr *addr)
+{
+ struct net_device *dev;
+ u32 src_ip = src_in->sin_addr.s_addr;
+ u32 dst_ip = dst_in->sin_addr.s_addr;
+ int ret;
+
+ dev = ip_dev_find(dst_ip);
+ if (!dev)
+ return -EADDRNOTAVAIL;
+
+ if (!src_ip) {
+ src_in->sin_family = dst_in->sin_family;
+ src_in->sin_addr.s_addr = dst_ip;
+ ret = copy_addr(addr, dev, dev->dev_addr);
+ } else {
+ ret = rdma_translate_ip((struct sockaddr *)src_in, addr);
+ if (!ret)
+ memcpy(addr->dst_dev_addr, dev->dev_addr, MAX_ADDR_LEN);
+ }
+
+ dev_put(dev);
+ return ret;
+}
+
+int rdma_resolve_ip(struct sockaddr *src_addr, struct sockaddr *dst_addr,
+ struct rdma_dev_addr *addr, int timeout_ms,
+ void (*callback)(int status, struct sockaddr *src_addr,
+ struct rdma_dev_addr *addr, void *context),
+ void *context)
+{
+ struct sockaddr_in *src_in, *dst_in;
+ struct addr_req *req;
+ int ret = 0;
+
+ req = kmalloc(sizeof *req, GFP_KERNEL);
+ if (!req)
+ return -ENOMEM;
+ memset(req, 0, sizeof *req);
+
+ if (src_addr)
+ memcpy(&req->src_addr, src_addr, ip_addr_size(src_addr));
+ memcpy(&req->dst_addr, dst_addr, ip_addr_size(dst_addr));
+ req->addr = addr;
+ req->callback = callback;
+ req->context = context;
+
+ src_in = (struct sockaddr_in *) &req->src_addr;
+ dst_in = (struct sockaddr_in *) &req->dst_addr;
+
+ req->status = addr_resolve_local(src_in, dst_in, addr);
+ if (req->status == -EADDRNOTAVAIL)
+ req->status = addr_resolve_remote(src_in, dst_in, addr);
+
+ switch (req->status) {
+ case 0:
+ req->timeout = jiffies;
+ queue_req(req);
+ break;
+ case -ENODATA:
+ req->timeout = msecs_to_jiffies(timeout_ms) + jiffies;
+ queue_req(req);
+ addr_send_arp(dst_in);
+ break;
+ default:
+ ret = req->status;
+ kfree(req);
+ break;
+ }
+ return ret;
+}
+EXPORT_SYMBOL(rdma_resolve_ip);
+
+void rdma_addr_cancel(struct rdma_dev_addr *addr)
+{
+ struct addr_req *req, *temp_req;
+
+ mutex_lock(&lock);
+ list_for_each_entry_safe(req, temp_req, &req_list, list) {
+ if (req->addr == addr) {
+ req->status = -ECANCELED;
+ req->timeout = jiffies;
+ list_del(&req->list);
+ list_add(&req->list, &req_list);
+ set_timeout(req->timeout);
+ break;
+ }
+ }
+ mutex_unlock(&lock);
+}
+EXPORT_SYMBOL(rdma_addr_cancel);
+
+static int addr_arp_recv(struct sk_buff *skb, struct net_device *dev,
+ struct packet_type *pkt, struct net_device *orig_dev)
+{
+ struct arphdr *arp_hdr;
+
+ arp_hdr = (struct arphdr *) skb->nh.raw;
+
+ if (dev->type == ARPHRD_INFINIBAND &&
+ (arp_hdr->ar_op == __constant_htons(ARPOP_REQUEST) ||
+ arp_hdr->ar_op == __constant_htons(ARPOP_REPLY)))
+ set_timeout(jiffies);
+
+ kfree_skb(skb);
+ return 0;
+}
+
+static struct packet_type addr_arp = {
+ .type = __constant_htons(ETH_P_ARP),
+ .func = addr_arp_recv,
+ .af_packet_priv = (void*) 1,
+};
+
+static int addr_init(void)
+{
+ rdma_wq = create_singlethread_workqueue("rdma_wq");
+ if (!rdma_wq)
+ return -ENOMEM;
+
+ dev_add_pack(&addr_arp);
+ return 0;
+}
+
+static void addr_cleanup(void)
+{
+ dev_remove_pack(&addr_arp);
+ destroy_workqueue(rdma_wq);
+}
+
+module_init(addr_init);
+module_exit(addr_cleanup);
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/Makefile
linux-2.6.ib/drivers/infiniband/core/Makefile
--- linux-2.6.git/drivers/infiniband/core/Makefile 2006-01-16 16:03:08.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/Makefile 2006-01-16 16:14:24.000000000 -0800
@@ -1,5 +1,5 @@
obj-$(CONFIG_INFINIBAND) += ib_core.o ib_mad.o ib_sa.o \
- ib_cm.o
+ ib_cm.o ib_addr.o
obj-$(CONFIG_INFINIBAND_USER_MAD) += ib_umad.o
obj-$(CONFIG_INFINIBAND_USER_ACCESS) += ib_uverbs.o ib_ucm.o
@@ -12,6 +12,8 @@ ib_sa-y := sa_query.o
ib_cm-y := cm.o
+ib_addr-y := addr.o
+
ib_umad-y := user_mad.o
ib_ucm-y := ucm.o
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/ib_addr.h
linux-2.6.ib/include/rdma/ib_addr.h
--- linux-2.6.git/include/rdma/ib_addr.h 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/include/rdma/ib_addr.h 2006-01-16 16:14:24.000000000 -0800
@@ -0,0 +1,117 @@
+/*
+ * Copyright (c) 2005 Voltaire Inc. All rights reserved.
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ *
+ */
+
+#if !defined(IB_ADDR_H)
+#define IB_ADDR_H
+
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/netdevice.h>
+#include <linux/socket.h>
+#include <rdma/ib_verbs.h>
+
+extern struct workqueue_struct *rdma_wq;
+
+struct rdma_dev_addr {
+ unsigned char src_dev_addr[MAX_ADDR_LEN];
+ unsigned char dst_dev_addr[MAX_ADDR_LEN];
+ unsigned char broadcast[MAX_ADDR_LEN];
+ enum ib_node_type dev_type;
+};
+
+/**
+ * rdma_translate_ip - Translate a local IP address to an RDMA hardware
+ * address.
+ */
+int rdma_translate_ip(struct sockaddr *addr, struct rdma_dev_addr *dev_addr);
+
+/**
+ * rdma_resolve_ip - Resolve source and destination IP addresses to
+ * RDMA hardware addresses.
+ * @src_addr: An optional source address to use in the resolution. If a
+ * source address is not provided, a usable address will be returned via
+ * the callback.
+ * @dst_addr: The destination address to resolve.
+ * @addr: A reference to a data location that will receive the resolved
+ * addresses. The data location must remain valid until the callback has
+ * been invoked.
+ * @timeout_ms: Amount of time to wait for the address resolution to complete.
+ * @callback: Call invoked once address resolution has completed, timed out,
+ * or been canceled. A status of 0 indicates success.
+ * @context: User-specified context associated with the call.
+ */
+int rdma_resolve_ip(struct sockaddr *src_addr, struct sockaddr *dst_addr,
+ struct rdma_dev_addr *addr, int timeout_ms,
+ void (*callback)(int status, struct sockaddr *src_addr,
+ struct rdma_dev_addr *addr, void *context),
+ void *context);
+
+void rdma_addr_cancel(struct rdma_dev_addr *addr);
+
+static inline int ip_addr_size(struct sockaddr *addr)
+{
+ return addr->sa_family == AF_INET6 ?
+ sizeof(struct sockaddr_in6) : sizeof(struct sockaddr_in);
+}
+
+static inline u16 ib_addr_get_pkey(struct rdma_dev_addr *dev_addr)
+{
+ return ((u16)dev_addr->broadcast[8] << 8) | (u16)dev_addr->broadcast[9];
+}
+
+static inline void ib_addr_set_pkey(struct rdma_dev_addr *dev_addr, u16 pkey)
+{
+ dev_addr->broadcast[8] = pkey >> 8;
+ dev_addr->broadcast[9] = (unsigned char) pkey;
+}
+
+static inline union ib_gid* ib_addr_get_sgid(struct rdma_dev_addr *dev_addr)
+{
+ return (union ib_gid *) (dev_addr->src_dev_addr + 4);
+}
+
+static inline void ib_addr_set_sgid(struct rdma_dev_addr *dev_addr,
+ union ib_gid *gid)
+{
+ memcpy(dev_addr->src_dev_addr + 4, gid, sizeof *gid);
+}
+
+static inline union ib_gid* ib_addr_get_dgid(struct rdma_dev_addr *dev_addr)
+{
+ return (union ib_gid *) (dev_addr->dst_dev_addr + 4);
+}
+
+static inline void ib_addr_set_dgid(struct rdma_dev_addr *dev_addr,
+ union ib_gid *gid)
+{
+ memcpy(dev_addr->dst_dev_addr + 4, gid, sizeof *gid);
+}
+
+#endif /* IB_ADDR_H */
+
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: [PATCH 4/6 v2] IB: address translation to map IP toIB addresses (GIDs)
2006-03-06 23:31 ` [PATCH 4/6 v2] IB: address translation to map IP toIB " Sean Hefty
@ 2006-03-11 1:14 ` Roland Dreier
2006-03-11 6:10 ` Sean Hefty
2006-03-21 20:57 ` Roland Dreier
1 sibling, 1 reply; 97+ messages in thread
From: Roland Dreier @ 2006-03-11 1:14 UTC (permalink / raw)
To: Sean Hefty; +Cc: linux-kernel, netdev, openib-general
The ib_addr module depends on CONFIG_INET, because it uses symbols
like arp_tbl, which are only exported if INET is enabled.
I fixed this up by creating a new (non-user-visible) config symbol to
control when ib_addr is built -- I put the following diff on top of
your patch in my tree:
diff --git a/drivers/infiniband/Kconfig b/drivers/infiniband/Kconfig
index bdf0891..48c8bb5 100644
--- a/drivers/infiniband/Kconfig
+++ b/drivers/infiniband/Kconfig
@@ -29,6 +29,11 @@ config INFINIBAND_USER_ACCESS
libibverbs, libibcm and a hardware driver library from
<http://www.openib.org>.
+config INFINIBAND_ADDR_TRANS
+ tristate
+ depends on INFINIBAND && INET
+ default y
+
source "drivers/infiniband/hw/mthca/Kconfig"
source "drivers/infiniband/ulp/ipoib/Kconfig"
diff --git a/drivers/infiniband/core/Makefile b/drivers/infiniband/core/Makefile
index 2393e9d..935851d 100644
--- a/drivers/infiniband/core/Makefile
+++ b/drivers/infiniband/core/Makefile
@@ -1,7 +1,8 @@
obj-$(CONFIG_INFINIBAND) += ib_core.o ib_mad.o ib_sa.o \
- ib_cm.o ib_addr.o
+ ib_cm.o
obj-$(CONFIG_INFINIBAND_USER_MAD) += ib_umad.o
obj-$(CONFIG_INFINIBAND_USER_ACCESS) += ib_uverbs.o ib_ucm.o
+obj-$(CONFIG_INFINIBAND_ADDR_TRANS) += ib_addr.o
ib_core-y := packer.o ud_header.o verbs.o sysfs.o \
device.o fmr_pool.o cache.o
^ permalink raw reply related [flat|nested] 97+ messages in thread
* RE: [PATCH 4/6 v2] IB: address translation to map IP toIB addresses (GIDs)
2006-03-11 1:14 ` Roland Dreier
@ 2006-03-11 6:10 ` Sean Hefty
0 siblings, 0 replies; 97+ messages in thread
From: Sean Hefty @ 2006-03-11 6:10 UTC (permalink / raw)
To: 'Roland Dreier'; +Cc: linux-kernel, netdev, openib-general
>The ib_addr module depends on CONFIG_INET, because it uses symbols
>like arp_tbl, which are only exported if INET is enabled.
>
>I fixed this up by creating a new (non-user-visible) config symbol to
>control when ib_addr is built -- I put the following diff on top of
>your patch in my tree:
Thanks!
-Sean
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: [PATCH 4/6 v2] IB: address translation to map IP toIB addresses (GIDs)
2006-03-06 23:31 ` [PATCH 4/6 v2] IB: address translation to map IP toIB " Sean Hefty
2006-03-11 1:14 ` Roland Dreier
@ 2006-03-21 20:57 ` Roland Dreier
2006-03-21 21:08 ` [openib-general] " Sean Hefty
1 sibling, 1 reply; 97+ messages in thread
From: Roland Dreier @ 2006-03-21 20:57 UTC (permalink / raw)
To: Sean Hefty; +Cc: linux-kernel, netdev, openib-general
> +struct workqueue_struct *rdma_wq;
> +EXPORT_SYMBOL(rdma_wq);
Sean, I don't think I saw an answer when I asked you this before. Why
is ib_addr exporting a workqueue? Is there some sort of ordering
constraint that is forcing other modules to go through the same
workqueue for things?
This seems like a very fragile internal thing to be exposing, and I'm
wondering if there's a better way to handle it.
- R.
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: [openib-general] Re: [PATCH 4/6 v2] IB: address translation to map IP toIB addresses (GIDs)
2006-03-21 20:57 ` Roland Dreier
@ 2006-03-21 21:08 ` Sean Hefty
2006-03-21 22:39 ` Roland Dreier
0 siblings, 1 reply; 97+ messages in thread
From: Sean Hefty @ 2006-03-21 21:08 UTC (permalink / raw)
To: Roland Dreier; +Cc: Sean Hefty, netdev, linux-kernel, openib-general
Roland Dreier wrote:
> > +struct workqueue_struct *rdma_wq;
> > +EXPORT_SYMBOL(rdma_wq);
>
> Sean, I don't think I saw an answer when I asked you this before. Why
> is ib_addr exporting a workqueue? Is there some sort of ordering
> constraint that is forcing other modules to go through the same
> workqueue for things?
>
> This seems like a very fragile internal thing to be exposing, and I'm
> wondering if there's a better way to handle it.
I responded in a different thread, but here's what I wrote:
"This is simply an attempt to reduce/combine work queues used by the Infiniband
code. This keeps the threading a little simpler in the rdma_cm, since all
callbacks are invoked using the same work queue. (I'm also using this with the
local SA/multicast code, but that's not ready for merging.)"
There's no specific ordering constraint that's required. We're just ending up
with several Infiniband modules creating their own work queues (ib_mad, ib_cm,
ib_addr, rdma_cm, plus a couple more in modules under development), and this is
an attempt to reduce that. If having separate work queues would work better,
there shouldn't be anything that prevents this.
- Sean
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: [openib-general] Re: [PATCH 4/6 v2] IB: address translation to map IP toIB addresses (GIDs)
2006-03-21 21:08 ` [openib-general] " Sean Hefty
@ 2006-03-21 22:39 ` Roland Dreier
0 siblings, 0 replies; 97+ messages in thread
From: Roland Dreier @ 2006-03-21 22:39 UTC (permalink / raw)
To: Sean Hefty; +Cc: Sean Hefty, netdev, linux-kernel, openib-general
Sean> "This is simply an attempt to reduce/combine work queues
Sean> used by the Infiniband code. This keeps the threading a
Sean> little simpler in the rdma_cm, since all callbacks are
Sean> invoked using the same work queue. (I'm also using this
Sean> with the local SA/multicast code, but that's not ready for
Sean> merging.)"
How does it keep the threading model simpler? Is this an inter-module
dependency.
Sean> There's no specific ordering constraint that's required.
Sean> We're just ending up with several Infiniband modules
Sean> creating their own work queues (ib_mad, ib_cm, ib_addr,
Sean> rdma_cm, plus a couple more in modules under development),
Sean> and this is an attempt to reduce that. If having separate
Sean> work queues would work better, there shouldn't be anything
Sean> that prevents this.
It seems like it would be cleaner for each module to have its own
workqueue if it needs one. There's also schedule_work(), although
that goes to a multi-threaded workqueue. Michael Tsirkin has
suggested creating a system-wide single-threaded workqueue (ie
something like schedule_ordered_work()) for everyone that occasionally
needs a single-threaded workqueue.
- R.
^ permalink raw reply [flat|nested] 97+ messages in thread
* [PATCH 5/6] IB: IP address based RDMA connection manager
2006-03-03 22:53 ` [openib-general] " Roland Dreier
` (3 preceding siblings ...)
2006-03-06 19:10 ` [PATCH 4/6] IB: address translation to map IP to IB addresses (GIDs) Sean Hefty
@ 2006-03-06 19:18 ` Sean Hefty
2006-03-06 19:21 ` [PATCH 6/6] IB: userspace support for " Sean Hefty
5 siblings, 0 replies; 97+ messages in thread
From: Sean Hefty @ 2006-03-06 19:18 UTC (permalink / raw)
To: 'Roland Dreier', netdev, linux-kernel; +Cc: openib-general
Kernel mode connection management agent over Infiniband that connects based
on IP addresses. The agent defines a generic RDMA connection abstraction
to support clients wanting to connect over different RDMA devices.
Agent also handles RDMA device hotplug events on behalf of clients.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
---
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/cma.c
linux-2.6.ib/drivers/infiniband/core/cma.c
--- linux-2.6.git/drivers/infiniband/core/cma.c 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/cma.c 2006-01-16 16:17:34.000000000 -0800
@@ -0,0 +1,1639 @@
+/*
+ * Copyright (c) 2005 Voltaire Inc. All rights reserved.
+ * Copyright (c) 2002-2005, Network Appliance, Inc. All rights reserved.
+ * Copyright (c) 1999-2005, Mellanox Technologies, Inc. All rights reserved.
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ *
+ */
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/random.h>
+#include <rdma/rdma_cm.h>
+#include <rdma/ib_cache.h>
+#include <rdma/ib_cm.h>
+#include <rdma/ib_sa.h>
+
+MODULE_AUTHOR("Guy German");
+MODULE_DESCRIPTION("Generic RDMA CM Agent");
+MODULE_LICENSE("Dual BSD/GPL");
+
+#define CMA_CM_RESPONSE_TIMEOUT 20
+#define CMA_MAX_CM_RETRIES 3
+
+static void cma_add_one(struct ib_device *device);
+static void cma_remove_one(struct ib_device *device);
+
+static struct ib_client cma_client = {
+ .name = "cma",
+ .add = cma_add_one,
+ .remove = cma_remove_one
+};
+
+static LIST_HEAD(dev_list);
+static LIST_HEAD(listen_any_list);
+static DECLARE_MUTEX(mutex);
+
+struct cma_device {
+ struct list_head list;
+ struct ib_device *device;
+ __be64 node_guid;
+ wait_queue_head_t wait;
+ atomic_t refcount;
+ struct list_head id_list;
+};
+
+enum cma_state {
+ CMA_IDLE,
+ CMA_ADDR_QUERY,
+ CMA_ADDR_RESOLVED,
+ CMA_ROUTE_QUERY,
+ CMA_ROUTE_RESOLVED,
+ CMA_CONNECT,
+ CMA_ADDR_BOUND,
+ CMA_LISTEN,
+ CMA_DEVICE_REMOVAL,
+ CMA_DESTROYING
+};
+
+/*
+ * Device removal can occur at anytime, so we need extra handling to
+ * serialize notifying the user of device removal with other callbacks.
+ * We do this by disabling removal notification while a callback is in process,
+ * and reporting it after the callback completes.
+ */
+struct rdma_id_private {
+ struct rdma_cm_id id;
+
+ struct list_head list;
+ struct list_head listen_list;
+ struct cma_device *cma_dev;
+
+ enum cma_state state;
+ spinlock_t lock;
+ wait_queue_head_t wait;
+ atomic_t refcount;
+ wait_queue_head_t wait_remove;
+ atomic_t dev_remove;
+
+ int backlog;
+ int timeout_ms;
+ struct ib_sa_query *query;
+ int query_id;
+ struct ib_cm_id *cm_id;
+
+ u32 seq_num;
+ u32 qp_num;
+ enum ib_qp_type qp_type;
+ u8 srq;
+};
+
+struct cma_work {
+ struct work_struct work;
+ struct rdma_id_private *id;
+};
+
+union cma_ip_addr {
+ struct in6_addr ip6;
+ struct {
+ __u32 pad[3];
+ __u32 addr;
+ } ip4;
+};
+
+struct cma_hdr {
+ u8 cma_version;
+ u8 ip_version; /* IP version: 7:4 */
+ __u16 port;
+ union cma_ip_addr src_addr;
+ union cma_ip_addr dst_addr;
+};
+
+struct sdp_hh {
+ u8 sdp_version;
+ u8 ip_version; /* IP version: 7:4 */
+ u8 sdp_specific1[10];
+ __u16 port;
+ __u16 sdp_specific2;
+ union cma_ip_addr src_addr;
+ union cma_ip_addr dst_addr;
+};
+
+#define CMA_VERSION 0x10
+#define SDP_VERSION 0x22
+
+static int cma_comp(struct rdma_id_private *id_priv, enum cma_state comp)
+{
+ unsigned long flags;
+ int ret;
+
+ spin_lock_irqsave(&id_priv->lock, flags);
+ ret = (id_priv->state == comp);
+ spin_unlock_irqrestore(&id_priv->lock, flags);
+ return ret;
+}
+
+static int cma_comp_exch(struct rdma_id_private *id_priv,
+ enum cma_state comp, enum cma_state exch)
+{
+ unsigned long flags;
+ int ret;
+
+ spin_lock_irqsave(&id_priv->lock, flags);
+ if ((ret = (id_priv->state == comp)))
+ id_priv->state = exch;
+ spin_unlock_irqrestore(&id_priv->lock, flags);
+ return ret;
+}
+
+static enum cma_state cma_exch(struct rdma_id_private *id_priv,
+ enum cma_state exch)
+{
+ unsigned long flags;
+ enum cma_state old;
+
+ spin_lock_irqsave(&id_priv->lock, flags);
+ old = id_priv->state;
+ id_priv->state = exch;
+ spin_unlock_irqrestore(&id_priv->lock, flags);
+ return old;
+}
+
+static inline u8 cma_get_ip_ver(struct cma_hdr *hdr)
+{
+ return hdr->ip_version >> 4;
+}
+
+static inline void cma_set_ip_ver(struct cma_hdr *hdr, u8 ip_ver)
+{
+ hdr->ip_version = (ip_ver << 4) | (hdr->ip_version & 0xF);
+}
+
+static inline u8 sdp_get_ip_ver(struct sdp_hh *hh)
+{
+ return hh->ip_version >> 4;
+}
+
+static inline void sdp_set_ip_ver(struct sdp_hh *hh, u8 ip_ver)
+{
+ hh->ip_version = (ip_ver << 4) | (hh->ip_version & 0xF);
+}
+
+static void cma_attach_to_dev(struct rdma_id_private *id_priv,
+ struct cma_device *cma_dev)
+{
+ atomic_inc(&cma_dev->refcount);
+ id_priv->cma_dev = cma_dev;
+ id_priv->id.device = cma_dev->device;
+ list_add_tail(&id_priv->list, &cma_dev->id_list);
+}
+
+static void cma_detach_from_dev(struct rdma_id_private *id_priv)
+{
+ list_del(&id_priv->list);
+ if (atomic_dec_and_test(&id_priv->cma_dev->refcount))
+ wake_up(&id_priv->cma_dev->wait);
+ id_priv->cma_dev = NULL;
+}
+
+static int cma_acquire_ib_dev(struct rdma_id_private *id_priv)
+{
+ struct cma_device *cma_dev;
+ union ib_gid *gid;
+ int ret = -ENODEV;
+
+ gid = ib_addr_get_sgid(&id_priv->id.route.addr.dev_addr);
+
+ down(&mutex);
+ list_for_each_entry(cma_dev, &dev_list, list) {
+ ret = ib_find_cached_gid(cma_dev->device, gid,
+ &id_priv->id.port_num, NULL);
+ if (!ret) {
+ cma_attach_to_dev(id_priv, cma_dev);
+ break;
+ }
+ }
+ up(&mutex);
+ return ret;
+}
+
+static int cma_acquire_dev(struct rdma_id_private *id_priv)
+{
+ switch (id_priv->id.route.addr.dev_addr.dev_type) {
+ case IB_NODE_CA:
+ return cma_acquire_ib_dev(id_priv);
+ default:
+ return -ENODEV;
+ }
+}
+
+static void cma_deref_id(struct rdma_id_private *id_priv)
+{
+ if (atomic_dec_and_test(&id_priv->refcount))
+ wake_up(&id_priv->wait);
+}
+
+static void cma_release_remove(struct rdma_id_private *id_priv)
+{
+ if (atomic_dec_and_test(&id_priv->dev_remove))
+ wake_up(&id_priv->wait_remove);
+}
+
+struct rdma_cm_id* rdma_create_id(rdma_cm_event_handler event_handler,
+ void *context, enum rdma_port_space ps)
+{
+ struct rdma_id_private *id_priv;
+
+ id_priv = kzalloc(sizeof *id_priv, GFP_KERNEL);
+ if (!id_priv)
+ return ERR_PTR(-ENOMEM);
+
+ id_priv->state = CMA_IDLE;
+ id_priv->id.context = context;
+ id_priv->id.event_handler = event_handler;
+ id_priv->id.ps = ps;
+ spin_lock_init(&id_priv->lock);
+ init_waitqueue_head(&id_priv->wait);
+ atomic_set(&id_priv->refcount, 1);
+ init_waitqueue_head(&id_priv->wait_remove);
+ atomic_set(&id_priv->dev_remove, 0);
+ INIT_LIST_HEAD(&id_priv->listen_list);
+ get_random_bytes(&id_priv->seq_num, sizeof id_priv->seq_num);
+
+ return &id_priv->id;
+}
+EXPORT_SYMBOL(rdma_create_id);
+
+static int cma_init_ib_qp(struct rdma_id_private *id_priv, struct ib_qp *qp)
+{
+ struct ib_qp_attr qp_attr;
+ struct rdma_dev_addr *dev_addr;
+ int ret;
+
+ dev_addr = &id_priv->id.route.addr.dev_addr;
+ ret = ib_find_cached_pkey(id_priv->id.device, id_priv->id.port_num,
+ ib_addr_get_pkey(dev_addr),
+ &qp_attr.pkey_index);
+ if (ret)
+ return ret;
+
+ qp_attr.qp_state = IB_QPS_INIT;
+ qp_attr.qp_access_flags = IB_ACCESS_LOCAL_WRITE;
+ qp_attr.port_num = id_priv->id.port_num;
+ return ib_modify_qp(qp, &qp_attr, IB_QP_STATE | IB_QP_ACCESS_FLAGS |
+ IB_QP_PKEY_INDEX | IB_QP_PORT);
+}
+
+int rdma_create_qp(struct rdma_cm_id *id, struct ib_pd *pd,
+ struct ib_qp_init_attr *qp_init_attr)
+{
+ struct rdma_id_private *id_priv;
+ struct ib_qp *qp;
+ int ret;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ if (id->device != pd->device)
+ return -EINVAL;
+
+ qp = ib_create_qp(pd, qp_init_attr);
+ if (IS_ERR(qp))
+ return PTR_ERR(qp);
+
+ switch (id->device->node_type) {
+ case IB_NODE_CA:
+ ret = cma_init_ib_qp(id_priv, qp);
+ break;
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+
+ if (ret)
+ goto err;
+
+ id->qp = qp;
+ id_priv->qp_num = qp->qp_num;
+ id_priv->qp_type = qp->qp_type;
+ id_priv->srq = (qp->srq != NULL);
+ return 0;
+err:
+ ib_destroy_qp(qp);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_create_qp);
+
+void rdma_destroy_qp(struct rdma_cm_id *id)
+{
+ ib_destroy_qp(id->qp);
+}
+EXPORT_SYMBOL(rdma_destroy_qp);
+
+static int cma_modify_qp_rtr(struct rdma_cm_id *id)
+{
+ struct ib_qp_attr qp_attr;
+ int qp_attr_mask, ret;
+
+ if (!id->qp)
+ return 0;
+
+ /* Need to update QP attributes from default values. */
+ qp_attr.qp_state = IB_QPS_INIT;
+ ret = rdma_init_qp_attr(id, &qp_attr, &qp_attr_mask);
+ if (ret)
+ return ret;
+
+ ret = ib_modify_qp(id->qp, &qp_attr, qp_attr_mask);
+ if (ret)
+ return ret;
+
+ qp_attr.qp_state = IB_QPS_RTR;
+ ret = rdma_init_qp_attr(id, &qp_attr, &qp_attr_mask);
+ if (ret)
+ return ret;
+
+ return ib_modify_qp(id->qp, &qp_attr, qp_attr_mask);
+}
+
+static int cma_modify_qp_rts(struct rdma_cm_id *id)
+{
+ struct ib_qp_attr qp_attr;
+ int qp_attr_mask, ret;
+
+ if (!id->qp)
+ return 0;
+
+ qp_attr.qp_state = IB_QPS_RTS;
+ ret = rdma_init_qp_attr(id, &qp_attr, &qp_attr_mask);
+ if (ret)
+ return ret;
+
+ return ib_modify_qp(id->qp, &qp_attr, qp_attr_mask);
+}
+
+static int cma_modify_qp_err(struct rdma_cm_id *id)
+{
+ struct ib_qp_attr qp_attr;
+
+ if (!id->qp)
+ return 0;
+
+ qp_attr.qp_state = IB_QPS_ERR;
+ return ib_modify_qp(id->qp, &qp_attr, IB_QP_STATE);
+}
+
+int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
+ int *qp_attr_mask)
+{
+ struct rdma_id_private *id_priv;
+ int ret;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ switch (id_priv->id.device->node_type) {
+ case IB_NODE_CA:
+ ret = ib_cm_init_qp_attr(id_priv->cm_id, qp_attr,
+ qp_attr_mask);
+ if (qp_attr->qp_state == IB_QPS_RTR)
+ qp_attr->rq_psn = id_priv->seq_num;
+ break;
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL(rdma_init_qp_attr);
+
+static inline int cma_any_addr(struct sockaddr *addr)
+{
+ struct in6_addr *ip6;
+
+ if (addr->sa_family == AF_INET)
+ return ((struct sockaddr_in *) addr)->sin_addr.s_addr ==
+ INADDR_ANY;
+ else {
+ ip6 = &((struct sockaddr_in6 *) addr)->sin6_addr;
+ return (ip6->s6_addr32[0] | ip6->s6_addr32[1] |
+ ip6->s6_addr32[3] | ip6->s6_addr32[4]) == 0;
+ }
+}
+
+static inline int cma_loopback_addr(struct sockaddr *addr)
+{
+ return ((struct sockaddr_in *) addr)->sin_addr.s_addr ==
+ ntohl(INADDR_LOOPBACK);
+}
+
+static int cma_get_net_info(void *hdr, enum rdma_port_space ps,
+ u8 *ip_ver, __u16 *port,
+ union cma_ip_addr **src, union cma_ip_addr **dst)
+{
+ switch (ps) {
+ case RDMA_PS_SDP:
+ if (((struct sdp_hh *) hdr)->sdp_version != SDP_VERSION)
+ return -EINVAL;
+
+ *ip_ver = sdp_get_ip_ver(hdr);
+ *port = ((struct sdp_hh *) hdr)->port;
+ *src = &((struct sdp_hh *) hdr)->src_addr;
+ *dst = &((struct sdp_hh *) hdr)->dst_addr;
+ break;
+ default:
+ if (((struct cma_hdr *) hdr)->cma_version != CMA_VERSION)
+ return -EINVAL;
+
+ *ip_ver = cma_get_ip_ver(hdr);
+ *port = ((struct cma_hdr *) hdr)->port;
+ *src = &((struct cma_hdr *) hdr)->src_addr;
+ *dst = &((struct cma_hdr *) hdr)->dst_addr;
+ break;
+ }
+ return 0;
+}
+
+static void cma_save_net_info(struct rdma_addr *addr,
+ struct rdma_addr *listen_addr,
+ u8 ip_ver, __u16 port,
+ union cma_ip_addr *src, union cma_ip_addr *dst)
+{
+ struct sockaddr_in *listen4, *ip4;
+ struct sockaddr_in6 *listen6, *ip6;
+
+ switch (ip_ver) {
+ case 4:
+ listen4 = (struct sockaddr_in *) &listen_addr->src_addr;
+ ip4 = (struct sockaddr_in *) &addr->src_addr;
+ ip4->sin_family = listen4->sin_family;
+ ip4->sin_addr.s_addr = dst->ip4.addr;
+ ip4->sin_port = listen4->sin_port;
+
+ ip4 = (struct sockaddr_in *) &addr->dst_addr;
+ ip4->sin_family = listen4->sin_family;
+ ip4->sin_addr.s_addr = src->ip4.addr;
+ ip4->sin_port = port;
+ break;
+ case 6:
+ listen6 = (struct sockaddr_in6 *) &listen_addr->src_addr;
+ ip6 = (struct sockaddr_in6 *) &addr->src_addr;
+ ip6->sin6_family = listen6->sin6_family;
+ ip6->sin6_addr = dst->ip6;
+ ip6->sin6_port = listen6->sin6_port;
+
+ ip6 = (struct sockaddr_in6 *) &addr->dst_addr;
+ ip6->sin6_family = listen6->sin6_family;
+ ip6->sin6_addr = src->ip6;
+ ip6->sin6_port = port;
+ break;
+ default:
+ break;
+ }
+}
+
+static inline int cma_user_data_offset(enum rdma_port_space ps)
+{
+ switch (ps) {
+ case RDMA_PS_SDP:
+ return 0;
+ default:
+ return sizeof(struct cma_hdr);
+ }
+}
+
+static int cma_notify_user(struct rdma_id_private *id_priv,
+ enum rdma_cm_event_type type, int status,
+ void *data, u8 data_len)
+{
+ struct rdma_cm_event event;
+
+ event.event = type;
+ event.status = status;
+ event.private_data = data;
+ event.private_data_len = data_len;
+
+ return id_priv->id.event_handler(&id_priv->id, &event);
+}
+
+static void cma_cancel_addr(struct rdma_id_private *id_priv)
+{
+ switch (id_priv->id.device->node_type) {
+ case IB_NODE_CA:
+ rdma_addr_cancel(&id_priv->id.route.addr.dev_addr);
+ break;
+ default:
+ break;
+ }
+}
+
+static void cma_cancel_route(struct rdma_id_private *id_priv)
+{
+ switch (id_priv->id.device->node_type) {
+ case IB_NODE_CA:
+ ib_sa_cancel_query(id_priv->query_id, id_priv->query);
+ break;
+ default:
+ break;
+ }
+}
+
+static inline int cma_internal_listen(struct rdma_id_private *id_priv)
+{
+ return (id_priv->state == CMA_LISTEN) && id_priv->cma_dev &&
+ cma_any_addr(&id_priv->id.route.addr.src_addr);
+}
+
+static void cma_destroy_listen(struct rdma_id_private *id_priv)
+{
+ cma_exch(id_priv, CMA_DESTROYING);
+
+ if (id_priv->cm_id && !IS_ERR(id_priv->cm_id))
+ ib_destroy_cm_id(id_priv->cm_id);
+
+ list_del(&id_priv->listen_list);
+ if (id_priv->cma_dev)
+ cma_detach_from_dev(id_priv);
+
+ atomic_dec(&id_priv->refcount);
+ wait_event(id_priv->wait, !atomic_read(&id_priv->refcount));
+
+ kfree(id_priv);
+}
+
+static void cma_cancel_listens(struct rdma_id_private *id_priv)
+{
+ struct rdma_id_private *dev_id_priv;
+
+ down(&mutex);
+ list_del(&id_priv->list);
+
+ while (!list_empty(&id_priv->listen_list)) {
+ dev_id_priv = list_entry(id_priv->listen_list.next,
+ struct rdma_id_private, listen_list);
+ cma_destroy_listen(dev_id_priv);
+ }
+ up(&mutex);
+}
+
+static void cma_cancel_operation(struct rdma_id_private *id_priv,
+ enum cma_state state)
+{
+ switch (state) {
+ case CMA_ADDR_QUERY:
+ cma_cancel_addr(id_priv);
+ break;
+ case CMA_ROUTE_QUERY:
+ cma_cancel_route(id_priv);
+ break;
+ case CMA_LISTEN:
+ if (cma_any_addr(&id_priv->id.route.addr.src_addr) &&
+ !id_priv->cma_dev)
+ cma_cancel_listens(id_priv);
+ break;
+ default:
+ break;
+ }
+}
+
+void rdma_destroy_id(struct rdma_cm_id *id)
+{
+ struct rdma_id_private *id_priv;
+ enum cma_state state;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ state = cma_exch(id_priv, CMA_DESTROYING);
+ cma_cancel_operation(id_priv, state);
+
+ if (id_priv->cm_id && !IS_ERR(id_priv->cm_id))
+ ib_destroy_cm_id(id_priv->cm_id);
+
+ if (id_priv->cma_dev) {
+ down(&mutex);
+ cma_detach_from_dev(id_priv);
+ up(&mutex);
+ }
+
+ atomic_dec(&id_priv->refcount);
+ wait_event(id_priv->wait, !atomic_read(&id_priv->refcount));
+
+ kfree(id_priv->id.route.path_rec);
+ kfree(id_priv);
+}
+EXPORT_SYMBOL(rdma_destroy_id);
+
+static int cma_rep_recv(struct rdma_id_private *id_priv)
+{
+ int ret;
+
+ ret = cma_modify_qp_rtr(&id_priv->id);
+ if (ret)
+ goto reject;
+
+ ret = cma_modify_qp_rts(&id_priv->id);
+ if (ret)
+ goto reject;
+
+ ret = ib_send_cm_rtu(id_priv->cm_id, NULL, 0);
+ if (ret)
+ goto reject;
+
+ return 0;
+reject:
+ cma_modify_qp_err(&id_priv->id);
+ ib_send_cm_rej(id_priv->cm_id, IB_CM_REJ_CONSUMER_DEFINED,
+ NULL, 0, NULL, 0);
+ return ret;
+}
+
+static int cma_rtu_recv(struct rdma_id_private *id_priv)
+{
+ int ret;
+
+ ret = cma_modify_qp_rts(&id_priv->id);
+ if (ret)
+ goto reject;
+
+ return 0;
+reject:
+ cma_modify_qp_err(&id_priv->id);
+ ib_send_cm_rej(id_priv->cm_id, IB_CM_REJ_CONSUMER_DEFINED,
+ NULL, 0, NULL, 0);
+ return ret;
+}
+
+static int cma_ib_handler(struct ib_cm_id *cm_id, struct ib_cm_event *ib_event)
+{
+ struct rdma_id_private *id_priv = cm_id->context;
+ enum rdma_cm_event_type event;
+ u8 private_data_len = 0;
+ int ret = 0, status = 0;
+
+ if (!cma_comp(id_priv, CMA_CONNECT))
+ return 0;
+
+ atomic_inc(&id_priv->dev_remove);
+ switch (ib_event->event) {
+ case IB_CM_REQ_ERROR:
+ case IB_CM_REP_ERROR:
+ event = RDMA_CM_EVENT_UNREACHABLE;
+ status = -ETIMEDOUT;
+ break;
+ case IB_CM_REP_RECEIVED:
+ if (id_priv->id.qp) {
+ status = cma_rep_recv(id_priv);
+ event = status ? RDMA_CM_EVENT_CONNECT_ERROR :
+ RDMA_CM_EVENT_ESTABLISHED;
+ } else
+ event = RDMA_CM_EVENT_CONNECT_RESPONSE;
+ private_data_len = IB_CM_REP_PRIVATE_DATA_SIZE;
+ break;
+ case IB_CM_RTU_RECEIVED:
+ status = cma_rtu_recv(id_priv);
+ event = status ? RDMA_CM_EVENT_CONNECT_ERROR :
+ RDMA_CM_EVENT_ESTABLISHED;
+ break;
+ case IB_CM_DREQ_ERROR:
+ status = -ETIMEDOUT; /* fall through */
+ case IB_CM_DREQ_RECEIVED:
+ case IB_CM_DREP_RECEIVED:
+ event = RDMA_CM_EVENT_DISCONNECTED;
+ break;
+ case IB_CM_TIMEWAIT_EXIT:
+ case IB_CM_MRA_RECEIVED:
+ /* ignore event */
+ goto out;
+ case IB_CM_REJ_RECEIVED:
+ cma_modify_qp_err(&id_priv->id);
+ status = ib_event->param.rej_rcvd.reason;
+ event = RDMA_CM_EVENT_REJECTED;
+ break;
+ default:
+ printk(KERN_ERR "RDMA CMA: unexpected IB CM event: %d",
+ ib_event->event);
+ goto out;
+ }
+
+ ret = cma_notify_user(id_priv, event, status, ib_event->private_data,
+ private_data_len);
+ if (ret) {
+ /* Destroy the CM ID by returning a non-zero value. */
+ id_priv->cm_id = NULL;
+ cma_exch(id_priv, CMA_DESTROYING);
+ cma_release_remove(id_priv);
+ rdma_destroy_id(&id_priv->id);
+ return ret;
+ }
+out:
+ cma_release_remove(id_priv);
+ return ret;
+}
+
+static struct rdma_id_private* cma_new_id(struct rdma_cm_id *listen_id,
+ struct ib_cm_event *ib_event)
+{
+ struct rdma_id_private *id_priv;
+ struct rdma_cm_id *id;
+ struct rdma_route *rt;
+ union cma_ip_addr *src, *dst;
+ __u16 port;
+ u8 ip_ver;
+
+ id = rdma_create_id(listen_id->event_handler, listen_id->context,
+ listen_id->ps);
+ if (IS_ERR(id))
+ return NULL;
+
+ rt = &id->route;
+ rt->num_paths = ib_event->param.req_rcvd.alternate_path ? 2 : 1;
+ rt->path_rec = kmalloc(sizeof *rt->path_rec * rt->num_paths, GFP_KERNEL);
+ if (!rt->path_rec)
+ goto err;
+
+ if (cma_get_net_info(ib_event->private_data, listen_id->ps,
+ &ip_ver, &port, &src, &dst))
+ goto err;
+
+ cma_save_net_info(&id->route.addr, &listen_id->route.addr,
+ ip_ver, port, src, dst);
+ rt->path_rec[0] = *ib_event->param.req_rcvd.primary_path;
+ if (rt->num_paths == 2)
+ rt->path_rec[1] = *ib_event->param.req_rcvd.alternate_path;
+
+ ib_addr_set_sgid(&rt->addr.dev_addr, &rt->path_rec[0].sgid);
+ ib_addr_set_dgid(&rt->addr.dev_addr, &rt->path_rec[0].dgid);
+ ib_addr_set_pkey(&rt->addr.dev_addr, be16_to_cpu(rt->path_rec[0].pkey));
+ rt->addr.dev_addr.dev_type = IB_NODE_CA;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ id_priv->state = CMA_CONNECT;
+ return id_priv;
+err:
+ rdma_destroy_id(id);
+ return NULL;
+}
+
+static int cma_req_handler(struct ib_cm_id *cm_id, struct ib_cm_event *ib_event)
+{
+ struct rdma_id_private *listen_id, *conn_id;
+ int offset, ret;
+
+ listen_id = cm_id->context;
+ atomic_inc(&listen_id->dev_remove);
+ if (!cma_comp(listen_id, CMA_LISTEN)) {
+ ret = -ECONNABORTED;
+ goto out;
+ }
+
+ conn_id = cma_new_id(&listen_id->id, ib_event);
+ if (!conn_id) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
+ atomic_inc(&conn_id->dev_remove);
+ ret = cma_acquire_ib_dev(conn_id);
+ if (ret) {
+ ret = -ENODEV;
+ cma_release_remove(conn_id);
+ rdma_destroy_id(&conn_id->id);
+ goto out;
+ }
+
+ conn_id->cm_id = cm_id;
+ cm_id->context = conn_id;
+ cm_id->cm_handler = cma_ib_handler;
+
+ offset = cma_user_data_offset(listen_id->id.ps);
+ ret = cma_notify_user(conn_id, RDMA_CM_EVENT_CONNECT_REQUEST, 0,
+ ib_event->private_data + offset,
+ IB_CM_REQ_PRIVATE_DATA_SIZE - offset);
+ if (ret) {
+ /* Destroy the CM ID by returning a non-zero value. */
+ conn_id->cm_id = NULL;
+ cma_exch(conn_id, CMA_DESTROYING);
+ cma_release_remove(conn_id);
+ rdma_destroy_id(&conn_id->id);
+ }
+out:
+ cma_release_remove(listen_id);
+ return ret;
+}
+
+static __be64 cma_get_service_id(enum rdma_port_space ps, struct sockaddr *addr)
+{
+ return cpu_to_be64(((u64)ps << 16) +
+ ((struct sockaddr_in *) addr)->sin_port);
+}
+
+static void cma_set_compare_data(struct sockaddr *addr,
+ struct ib_cm_private_data_compare *compare)
+{
+ struct cma_hdr *data, *mask;
+
+ memset(compare, 0, sizeof *compare);
+ data = (void *) compare->data;
+ mask = (void *) compare->mask;
+
+ switch (addr->sa_family) {
+ case AF_INET:
+ cma_set_ip_ver(data, 4);
+ cma_set_ip_ver(mask, 0xF);
+ data->dst_addr.ip4.addr = ((struct sockaddr_in *) addr)->
+ sin_addr.s_addr;
+ mask->dst_addr.ip4.addr = ~0;
+ break;
+ case AF_INET6:
+ cma_set_ip_ver(data, 6);
+ cma_set_ip_ver(mask, 0xF);
+ data->dst_addr.ip6 = ((struct sockaddr_in6 *) addr)->
+ sin6_addr;
+ memset(&mask->dst_addr.ip6, 1, sizeof mask->dst_addr.ip6);
+ break;
+ default:
+ break;
+ }
+}
+
+static int cma_ib_listen(struct rdma_id_private *id_priv)
+{
+ struct ib_cm_private_data_compare compare_data;
+ struct sockaddr *addr;
+ __be64 svc_id;
+ int ret;
+
+ id_priv->cm_id = ib_create_cm_id(id_priv->id.device, cma_req_handler,
+ id_priv);
+ if (IS_ERR(id_priv->cm_id))
+ return PTR_ERR(id_priv->cm_id);
+
+ addr = &id_priv->id.route.addr.src_addr;
+ svc_id = cma_get_service_id(id_priv->id.ps, addr);
+ if (cma_any_addr(addr))
+ ret = ib_cm_listen(id_priv->cm_id, svc_id, 0, NULL);
+ else {
+ cma_set_compare_data(addr, &compare_data);
+ ret = ib_cm_listen(id_priv->cm_id, svc_id, 0, &compare_data);
+ }
+
+ if (ret) {
+ ib_destroy_cm_id(id_priv->cm_id);
+ id_priv->cm_id = NULL;
+ }
+
+ return ret;
+}
+
+static int cma_duplicate_listen(struct rdma_id_private *id_priv)
+{
+ struct rdma_id_private *cur_id_priv;
+ struct sockaddr_in *cur_addr, *new_addr;
+
+ new_addr = (struct sockaddr_in *) &id_priv->id.route.addr.src_addr;
+ list_for_each_entry(cur_id_priv, &listen_any_list, listen_list) {
+ cur_addr = (struct sockaddr_in *)
+ &cur_id_priv->id.route.addr.src_addr;
+ if (cur_addr->sin_port == new_addr->sin_port)
+ return -EADDRINUSE;
+ }
+ return 0;
+}
+
+static int cma_listen_handler(struct rdma_cm_id *id,
+ struct rdma_cm_event *event)
+{
+ struct rdma_id_private *id_priv = id->context;
+
+ id->context = id_priv->id.context;
+ id->event_handler = id_priv->id.event_handler;
+ return id_priv->id.event_handler(id, event);
+}
+
+static void cma_listen_on_dev(struct rdma_id_private *id_priv,
+ struct cma_device *cma_dev)
+{
+ struct rdma_id_private *dev_id_priv;
+ struct rdma_cm_id *id;
+ int ret;
+
+ id = rdma_create_id(cma_listen_handler, id_priv, id_priv->id.ps);
+ if (IS_ERR(id))
+ return;
+
+ dev_id_priv = container_of(id, struct rdma_id_private, id);
+ ret = rdma_bind_addr(id, &id_priv->id.route.addr.src_addr);
+ if (ret)
+ goto err;
+
+ cma_attach_to_dev(dev_id_priv, cma_dev);
+ list_add_tail(&dev_id_priv->listen_list, &id_priv->listen_list);
+
+ ret = rdma_listen(id, id_priv->backlog);
+ if (ret)
+ goto err;
+
+ return;
+err:
+ cma_destroy_listen(dev_id_priv);
+}
+
+static int cma_listen_on_all(struct rdma_id_private *id_priv)
+{
+ struct cma_device *cma_dev;
+ int ret;
+
+ down(&mutex);
+ ret = cma_duplicate_listen(id_priv);
+ if (ret)
+ goto out;
+
+ list_add_tail(&id_priv->list, &listen_any_list);
+ list_for_each_entry(cma_dev, &dev_list, list)
+ cma_listen_on_dev(id_priv, cma_dev);
+out:
+ up(&mutex);
+ return ret;
+}
+
+int rdma_listen(struct rdma_cm_id *id, int backlog)
+{
+ struct rdma_id_private *id_priv;
+ int ret;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ if (!cma_comp_exch(id_priv, CMA_ADDR_BOUND, CMA_LISTEN))
+ return -EINVAL;
+
+ if (id->device) {
+ switch (id->device->node_type) {
+ case IB_NODE_CA:
+ ret = cma_ib_listen(id_priv);
+ break;
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+ } else
+ ret = cma_listen_on_all(id_priv);
+
+ if (ret)
+ goto err;
+
+ id_priv->backlog = backlog;
+ return 0;
+err:
+ cma_comp_exch(id_priv, CMA_LISTEN, CMA_ADDR_BOUND);
+ return ret;
+};
+EXPORT_SYMBOL(rdma_listen);
+
+static void cma_query_handler(int status, struct ib_sa_path_rec *path_rec,
+ void *context)
+{
+ struct rdma_id_private *id_priv = context;
+ struct rdma_route *route = &id_priv->id.route;
+ enum rdma_cm_event_type event = RDMA_CM_EVENT_ROUTE_RESOLVED;
+
+ atomic_inc(&id_priv->dev_remove);
+ if (!status) {
+ route->path_rec = kmalloc(sizeof *route->path_rec, GFP_KERNEL);
+ if (route->path_rec) {
+ route->num_paths = 1;
+ *route->path_rec = *path_rec;
+ if (!cma_comp_exch(id_priv, CMA_ROUTE_QUERY,
+ CMA_ROUTE_RESOLVED)) {
+ kfree(route->path_rec);
+ goto out;
+ }
+ } else
+ status = -ENOMEM;
+ }
+
+ if (status) {
+ if (!cma_comp_exch(id_priv, CMA_ROUTE_QUERY, CMA_ADDR_RESOLVED))
+ goto out;
+ event = RDMA_CM_EVENT_ROUTE_ERROR;
+ }
+
+ if (cma_notify_user(id_priv, event, status, NULL, 0)) {
+ cma_exch(id_priv, CMA_DESTROYING);
+ cma_release_remove(id_priv);
+ cma_deref_id(id_priv);
+ rdma_destroy_id(&id_priv->id);
+ return;
+ }
+out:
+ cma_release_remove(id_priv);
+ cma_deref_id(id_priv);
+}
+
+static int cma_resolve_ib_route(struct rdma_id_private *id_priv, int timeout_ms)
+{
+ struct rdma_dev_addr *addr = &id_priv->id.route.addr.dev_addr;
+ struct ib_sa_path_rec path_rec;
+
+ memset(&path_rec, 0, sizeof path_rec);
+ path_rec.sgid = *ib_addr_get_sgid(addr);
+ path_rec.dgid = *ib_addr_get_dgid(addr);
+ path_rec.pkey = cpu_to_be16(ib_addr_get_pkey(addr));
+ path_rec.numb_path = 1;
+
+ id_priv->query_id = ib_sa_path_rec_get(id_priv->id.device,
+ id_priv->id.port_num, &path_rec,
+ IB_SA_PATH_REC_DGID | IB_SA_PATH_REC_SGID |
+ IB_SA_PATH_REC_PKEY | IB_SA_PATH_REC_NUMB_PATH,
+ timeout_ms, GFP_KERNEL,
+ cma_query_handler, id_priv, &id_priv->query);
+
+ return (id_priv->query_id < 0) ? id_priv->query_id : 0;
+}
+
+int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms)
+{
+ struct rdma_id_private *id_priv;
+ int ret;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ if (!cma_comp_exch(id_priv, CMA_ADDR_RESOLVED, CMA_ROUTE_QUERY))
+ return -EINVAL;
+
+ atomic_inc(&id_priv->refcount);
+ switch (id->device->node_type) {
+ case IB_NODE_CA:
+ ret = cma_resolve_ib_route(id_priv, timeout_ms);
+ break;
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+ if (ret)
+ goto err;
+
+ return 0;
+err:
+ cma_comp_exch(id_priv, CMA_ROUTE_QUERY, CMA_ADDR_RESOLVED);
+ cma_deref_id(id_priv);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_resolve_route);
+
+static int cma_bind_loopback(struct rdma_id_private *id_priv)
+{
+ struct cma_device *cma_dev;
+ union ib_gid *gid;
+ u16 pkey;
+ int ret;
+
+ down(&mutex);
+ if (list_empty(&dev_list)) {
+ ret = -ENODEV;
+ goto out;
+ }
+
+ cma_dev = list_entry(dev_list.next, struct cma_device, list);
+ gid = ib_addr_get_sgid(&id_priv->id.route.addr.dev_addr);
+ ret = ib_get_cached_gid(cma_dev->device, 1, 0, gid);
+ if (ret)
+ goto out;
+
+ ret = ib_get_cached_pkey(cma_dev->device, 1, 0, &pkey);
+ if (ret)
+ goto out;
+
+ ib_addr_set_pkey(&id_priv->id.route.addr.dev_addr, pkey);
+ id_priv->id.port_num = 1;
+ cma_attach_to_dev(id_priv, cma_dev);
+out:
+ up(&mutex);
+ return ret;
+}
+
+static void addr_handler(int status, struct sockaddr *src_addr,
+ struct rdma_dev_addr *dev_addr, void *context)
+{
+ struct rdma_id_private *id_priv = context;
+ enum rdma_cm_event_type event;
+ enum cma_state old_state;
+
+ atomic_inc(&id_priv->dev_remove);
+ if (!id_priv->cma_dev) {
+ old_state = CMA_IDLE;
+ if (!status)
+ status = cma_acquire_dev(id_priv);
+ } else
+ old_state = CMA_ADDR_BOUND;
+
+ if (status) {
+ if (!cma_comp_exch(id_priv, CMA_ADDR_QUERY, old_state))
+ goto out;
+ event = RDMA_CM_EVENT_ADDR_ERROR;
+ } else {
+ if (!cma_comp_exch(id_priv, CMA_ADDR_QUERY, CMA_ADDR_RESOLVED))
+ goto out;
+ memcpy(&id_priv->id.route.addr.src_addr, src_addr,
+ ip_addr_size(src_addr));
+ event = RDMA_CM_EVENT_ADDR_RESOLVED;
+ }
+
+ if (cma_notify_user(id_priv, event, status, NULL, 0)) {
+ cma_exch(id_priv, CMA_DESTROYING);
+ cma_release_remove(id_priv);
+ cma_deref_id(id_priv);
+ rdma_destroy_id(&id_priv->id);
+ return;
+ }
+out:
+ cma_release_remove(id_priv);
+ cma_deref_id(id_priv);
+}
+
+static void loopback_addr_handler(void *data)
+{
+ struct cma_work *work = data;
+ struct rdma_id_private *id_priv = work->id;
+
+ kfree(work);
+ atomic_inc(&id_priv->dev_remove);
+
+ if (!cma_comp_exch(id_priv, CMA_ADDR_QUERY, CMA_ADDR_RESOLVED))
+ goto out;
+
+ if (cma_notify_user(id_priv, RDMA_CM_EVENT_ADDR_RESOLVED, 0, NULL, 0)) {
+ cma_exch(id_priv, CMA_DESTROYING);
+ cma_release_remove(id_priv);
+ cma_deref_id(id_priv);
+ rdma_destroy_id(&id_priv->id);
+ return;
+ }
+out:
+ cma_release_remove(id_priv);
+ cma_deref_id(id_priv);
+}
+
+static int cma_resolve_loopback(struct rdma_id_private *id_priv,
+ struct sockaddr *src_addr, enum cma_state state)
+{
+ struct cma_work *work;
+ struct rdma_dev_addr *dev_addr;
+ int ret;
+
+ work = kmalloc(sizeof *work, GFP_KERNEL);
+ if (!work)
+ return -ENOMEM;
+
+ if (state == CMA_IDLE) {
+ ret = cma_bind_loopback(id_priv);
+ if (ret)
+ goto err;
+ dev_addr = &id_priv->id.route.addr.dev_addr;
+ ib_addr_set_dgid(dev_addr, ib_addr_get_sgid(dev_addr));
+ if (!src_addr || cma_any_addr(src_addr))
+ src_addr = &id_priv->id.route.addr.dst_addr;
+ memcpy(&id_priv->id.route.addr.src_addr, src_addr,
+ ip_addr_size(src_addr));
+ }
+
+ work->id = id_priv;
+ INIT_WORK(&work->work, loopback_addr_handler, work);
+ queue_work(rdma_wq, &work->work);
+ return 0;
+err:
+ kfree(work);
+ return ret;
+}
+
+int rdma_resolve_addr(struct rdma_cm_id *id, struct sockaddr *src_addr,
+ struct sockaddr *dst_addr, int timeout_ms)
+{
+ struct rdma_id_private *id_priv;
+ enum cma_state expected_state;
+ int ret;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ if (id_priv->cma_dev) {
+ expected_state = CMA_ADDR_BOUND;
+ src_addr = &id->route.addr.src_addr;
+ } else
+ expected_state = CMA_IDLE;
+
+ if (!cma_comp_exch(id_priv, expected_state, CMA_ADDR_QUERY))
+ return -EINVAL;
+
+ atomic_inc(&id_priv->refcount);
+ memcpy(&id->route.addr.dst_addr, dst_addr, ip_addr_size(dst_addr));
+ if (cma_loopback_addr(dst_addr))
+ ret = cma_resolve_loopback(id_priv, src_addr, expected_state);
+ else
+ ret = rdma_resolve_ip(src_addr, dst_addr,
+ &id->route.addr.dev_addr,
+ timeout_ms, addr_handler, id_priv);
+ if (ret)
+ goto err;
+
+ return 0;
+err:
+ cma_comp_exch(id_priv, CMA_ADDR_QUERY, expected_state);
+ cma_deref_id(id_priv);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_resolve_addr);
+
+int rdma_bind_addr(struct rdma_cm_id *id, struct sockaddr *addr)
+{
+ struct rdma_id_private *id_priv;
+ struct rdma_dev_addr *dev_addr;
+ int ret;
+
+ if (addr->sa_family != AF_INET)
+ return -EINVAL;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ if (!cma_comp_exch(id_priv, CMA_IDLE, CMA_ADDR_BOUND))
+ return -EINVAL;
+
+ if (cma_any_addr(addr)) {
+ ret = 0;
+ } else if (cma_loopback_addr(addr)) {
+ ret = cma_bind_loopback(id_priv);
+ } else {
+ dev_addr = &id->route.addr.dev_addr;
+ ret = rdma_translate_ip(addr, dev_addr);
+ if (!ret)
+ ret = cma_acquire_dev(id_priv);
+ }
+
+ if (ret)
+ goto err;
+
+ memcpy(&id->route.addr.src_addr, addr, ip_addr_size(addr));
+ return 0;
+err:
+ cma_comp_exch(id_priv, CMA_ADDR_BOUND, CMA_IDLE);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_bind_addr);
+
+static void cma_format_hdr(void *hdr, enum rdma_port_space ps,
+ struct rdma_route *route)
+{
+ struct sockaddr_in *src4, *dst4;
+ struct cma_hdr *cma_hdr;
+ struct sdp_hh *sdp_hdr;
+
+ src4 = (struct sockaddr_in *) &route->addr.src_addr;
+ dst4 = (struct sockaddr_in *) &route->addr.dst_addr;
+
+ switch (ps) {
+ case RDMA_PS_SDP:
+ sdp_hdr = hdr;
+ sdp_hdr->sdp_version = SDP_VERSION;
+ sdp_set_ip_ver(sdp_hdr, 4);
+ sdp_hdr->src_addr.ip4.addr = src4->sin_addr.s_addr;
+ sdp_hdr->dst_addr.ip4.addr = dst4->sin_addr.s_addr;
+ sdp_hdr->port = src4->sin_port;
+ break;
+ default:
+ cma_hdr = hdr;
+ cma_hdr->cma_version = CMA_VERSION;
+ cma_set_ip_ver(cma_hdr, 4);
+ cma_hdr->src_addr.ip4.addr = src4->sin_addr.s_addr;
+ cma_hdr->dst_addr.ip4.addr = dst4->sin_addr.s_addr;
+ cma_hdr->port = src4->sin_port;
+ break;
+ }
+}
+
+static int cma_connect_ib(struct rdma_id_private *id_priv,
+ struct rdma_conn_param *conn_param)
+{
+ struct ib_cm_req_param req;
+ struct rdma_route *route;
+ void *private_data;
+ int offset, ret;
+
+ memset(&req, 0, sizeof req);
+ offset = cma_user_data_offset(id_priv->id.ps);
+ req.private_data_len = offset + conn_param->private_data_len;
+ private_data = kzalloc(req.private_data_len, GFP_ATOMIC);
+ if (!private_data)
+ return -ENOMEM;
+
+ if (conn_param->private_data && conn_param->private_data_len)
+ memcpy(private_data + offset, conn_param->private_data,
+ conn_param->private_data_len);
+
+ id_priv->cm_id = ib_create_cm_id(id_priv->id.device, cma_ib_handler,
+ id_priv);
+ if (IS_ERR(id_priv->cm_id)) {
+ ret = PTR_ERR(id_priv->cm_id);
+ goto out;
+ }
+
+ route = &id_priv->id.route;
+ cma_format_hdr(private_data, id_priv->id.ps, route);
+ req.private_data = private_data;
+
+ req.primary_path = &route->path_rec[0];
+ if (route->num_paths == 2)
+ req.alternate_path = &route->path_rec[1];
+
+ req.service_id = cma_get_service_id(id_priv->id.ps,
+ &route->addr.dst_addr);
+ req.qp_num = id_priv->qp_num;
+ req.qp_type = id_priv->qp_type;
+ req.starting_psn = id_priv->seq_num;
+ req.responder_resources = conn_param->responder_resources;
+ req.initiator_depth = conn_param->initiator_depth;
+ req.flow_control = conn_param->flow_control;
+ req.retry_count = conn_param->retry_count;
+ req.rnr_retry_count = conn_param->rnr_retry_count;
+ req.remote_cm_response_timeout = CMA_CM_RESPONSE_TIMEOUT;
+ req.local_cm_response_timeout = CMA_CM_RESPONSE_TIMEOUT;
+ req.max_cm_retries = CMA_MAX_CM_RETRIES;
+ req.srq = id_priv->srq ? 1 : 0;
+
+ ret = ib_send_cm_req(id_priv->cm_id, &req);
+out:
+ kfree(private_data);
+ return ret;
+}
+
+int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
+{
+ struct rdma_id_private *id_priv;
+ int ret;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ if (!cma_comp_exch(id_priv, CMA_ROUTE_RESOLVED, CMA_CONNECT))
+ return -EINVAL;
+
+ if (!id->qp) {
+ id_priv->qp_num = conn_param->qp_num;
+ id_priv->qp_type = conn_param->qp_type;
+ id_priv->srq = conn_param->srq;
+ }
+
+ switch (id->device->node_type) {
+ case IB_NODE_CA:
+ ret = cma_connect_ib(id_priv, conn_param);
+ break;
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+ if (ret)
+ goto err;
+
+ return 0;
+err:
+ cma_comp_exch(id_priv, CMA_CONNECT, CMA_ROUTE_RESOLVED);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_connect);
+
+static int cma_accept_ib(struct rdma_id_private *id_priv,
+ struct rdma_conn_param *conn_param)
+{
+ struct ib_cm_rep_param rep;
+ int ret;
+
+ ret = cma_modify_qp_rtr(&id_priv->id);
+ if (ret)
+ return ret;
+
+ memset(&rep, 0, sizeof rep);
+ rep.qp_num = id_priv->qp_num;
+ rep.starting_psn = id_priv->seq_num;
+ rep.private_data = conn_param->private_data;
+ rep.private_data_len = conn_param->private_data_len;
+ rep.responder_resources = conn_param->responder_resources;
+ rep.initiator_depth = conn_param->initiator_depth;
+ rep.target_ack_delay = CMA_CM_RESPONSE_TIMEOUT;
+ rep.failover_accepted = 0;
+ rep.flow_control = conn_param->flow_control;
+ rep.rnr_retry_count = conn_param->rnr_retry_count;
+ rep.srq = id_priv->srq ? 1 : 0;
+
+ return ib_send_cm_rep(id_priv->cm_id, &rep);
+}
+
+int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
+{
+ struct rdma_id_private *id_priv;
+ int ret;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ if (!cma_comp(id_priv, CMA_CONNECT))
+ return -EINVAL;
+
+ if (!id->qp && conn_param) {
+ id_priv->qp_num = conn_param->qp_num;
+ id_priv->qp_type = conn_param->qp_type;
+ id_priv->srq = conn_param->srq;
+ }
+
+ switch (id->device->node_type) {
+ case IB_NODE_CA:
+ if (conn_param)
+ ret = cma_accept_ib(id_priv, conn_param);
+ else
+ ret = cma_rep_recv(id_priv);
+ break;
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+
+ if (ret)
+ goto reject;
+
+ return 0;
+reject:
+ cma_modify_qp_err(id);
+ rdma_reject(id, NULL, 0);
+ return ret;
+}
+EXPORT_SYMBOL(rdma_accept);
+
+int rdma_reject(struct rdma_cm_id *id, const void *private_data,
+ u8 private_data_len)
+{
+ struct rdma_id_private *id_priv;
+ int ret;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ if (!cma_comp(id_priv, CMA_CONNECT))
+ return -EINVAL;
+
+ switch (id->device->node_type) {
+ case IB_NODE_CA:
+ ret = ib_send_cm_rej(id_priv->cm_id, IB_CM_REJ_CONSUMER_DEFINED,
+ NULL, 0, private_data, private_data_len);
+ break;
+ default:
+ ret = -ENOSYS;
+ break;
+ }
+ return ret;
+};
+EXPORT_SYMBOL(rdma_reject);
+
+int rdma_disconnect(struct rdma_cm_id *id)
+{
+ struct rdma_id_private *id_priv;
+ int ret;
+
+ id_priv = container_of(id, struct rdma_id_private, id);
+ if (!cma_comp(id_priv, CMA_CONNECT))
+ return -EINVAL;
+
+ ret = cma_modify_qp_err(id);
+ if (ret)
+ goto out;
+
+ switch (id->device->node_type) {
+ case IB_NODE_CA:
+ /* Initiate or respond to a disconnect. */
+ if (ib_send_cm_dreq(id_priv->cm_id, NULL, 0))
+ ib_send_cm_drep(id_priv->cm_id, NULL, 0);
+ break;
+ default:
+ break;
+ }
+out:
+ return ret;
+}
+EXPORT_SYMBOL(rdma_disconnect);
+
+static void cma_add_one(struct ib_device *device)
+{
+ struct cma_device *cma_dev;
+ struct rdma_id_private *id_priv;
+
+ cma_dev = kmalloc(sizeof *cma_dev, GFP_KERNEL);
+ if (!cma_dev)
+ return;
+
+ cma_dev->device = device;
+ cma_dev->node_guid = device->node_guid;
+ if (!cma_dev->node_guid)
+ goto err;
+
+ init_waitqueue_head(&cma_dev->wait);
+ atomic_set(&cma_dev->refcount, 1);
+ INIT_LIST_HEAD(&cma_dev->id_list);
+ ib_set_client_data(device, &cma_client, cma_dev);
+
+ down(&mutex);
+ list_add_tail(&cma_dev->list, &dev_list);
+ list_for_each_entry(id_priv, &listen_any_list, list)
+ cma_listen_on_dev(id_priv, cma_dev);
+ up(&mutex);
+ return;
+err:
+ kfree(cma_dev);
+}
+
+static int cma_remove_id_dev(struct rdma_id_private *id_priv)
+{
+ enum cma_state state;
+
+ /* Record that we want to remove the device */
+ state = cma_exch(id_priv, CMA_DEVICE_REMOVAL);
+ if (state == CMA_DESTROYING)
+ return 0;
+
+ cma_cancel_operation(id_priv, state);
+ wait_event(id_priv->wait_remove, !atomic_read(&id_priv->dev_remove));
+
+ /* Check for destruction from another callback. */
+ if (!cma_comp(id_priv, CMA_DEVICE_REMOVAL))
+ return 0;
+
+ return cma_notify_user(id_priv, RDMA_CM_EVENT_DEVICE_REMOVAL,
+ 0, NULL, 0);
+}
+
+static void cma_process_remove(struct cma_device *cma_dev)
+{
+ struct list_head remove_list;
+ struct rdma_id_private *id_priv;
+ int ret;
+
+ INIT_LIST_HEAD(&remove_list);
+
+ down(&mutex);
+ while (!list_empty(&cma_dev->id_list)) {
+ id_priv = list_entry(cma_dev->id_list.next,
+ struct rdma_id_private, list);
+
+ if (cma_internal_listen(id_priv)) {
+ cma_destroy_listen(id_priv);
+ continue;
+ }
+
+ list_del(&id_priv->list);
+ list_add_tail(&id_priv->list, &remove_list);
+ atomic_inc(&id_priv->refcount);
+ up(&mutex);
+
+ ret = cma_remove_id_dev(id_priv);
+ cma_deref_id(id_priv);
+ if (ret)
+ rdma_destroy_id(&id_priv->id);
+
+ down(&mutex);
+ }
+ up(&mutex);
+
+ atomic_dec(&cma_dev->refcount);
+ wait_event(cma_dev->wait, !atomic_read(&cma_dev->refcount));
+}
+
+static void cma_remove_one(struct ib_device *device)
+{
+ struct cma_device *cma_dev;
+
+ cma_dev = ib_get_client_data(device, &cma_client);
+ if (!cma_dev)
+ return;
+
+ down(&mutex);
+ list_del(&cma_dev->list);
+ up(&mutex);
+
+ cma_process_remove(cma_dev);
+ kfree(cma_dev);
+}
+
+static int cma_init(void)
+{
+ return ib_register_client(&cma_client);
+}
+
+static void cma_cleanup(void)
+{
+ ib_unregister_client(&cma_client);
+}
+
+module_init(cma_init);
+module_exit(cma_cleanup);
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/Makefile
linux-2.6.ib/drivers/infiniband/core/Makefile
--- linux-2.6.git/drivers/infiniband/core/Makefile 2006-01-16 16:16:18.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/Makefile 2006-01-16 16:35:48.000000000 -0800
@@ -1,5 +1,5 @@
obj-$(CONFIG_INFINIBAND) += ib_core.o ib_mad.o ib_sa.o \
- ib_cm.o ib_addr.o
+ ib_cm.o ib_addr.o rdma_cm.o
obj-$(CONFIG_INFINIBAND_USER_MAD) += ib_umad.o
obj-$(CONFIG_INFINIBAND_USER_ACCESS) += ib_uverbs.o ib_ucm.o
@@ -12,6 +12,8 @@ ib_sa-y := sa_query.o
ib_cm-y := cm.o
+rdma_cm-y := cma.o
+
ib_addr-y := addr.o
ib_umad-y := user_mad.o
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/rdma_cm.h
linux-2.6.ib/include/rdma/rdma_cm.h
--- linux-2.6.git/include/rdma/rdma_cm.h 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/include/rdma/rdma_cm.h 2006-01-16 16:19:12.000000000 -0800
@@ -0,0 +1,255 @@
+/*
+ * Copyright (c) 2005 Voltaire Inc. All rights reserved.
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This Software is licensed under one of the following licenses:
+ *
+ * 1) under the terms of the "Common Public License 1.0" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/cpl.php.
+ *
+ * 2) under the terms of the "The BSD License" a copy of which is
+ * available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/bsd-license.php.
+ *
+ * 3) under the terms of the "GNU General Public License (GPL) Version 2" a
+ * copy of which is available from the Open Source Initiative, see
+ * http://www.opensource.org/licenses/gpl-license.php.
+ *
+ * Licensee has the right to choose one of the above licenses.
+ *
+ * Redistributions of source code must retain the above copyright
+ * notice and one of the license notices.
+ *
+ * Redistributions in binary form must reproduce both the above copyright
+ * notice, one of the license notices in the documentation
+ * and/or other materials provided with the distribution.
+ *
+ */
+
+#if !defined(RDMA_CM_H)
+#define RDMA_CM_H
+
+#include <linux/socket.h>
+#include <linux/in6.h>
+#include <rdma/ib_addr.h>
+#include <rdma/ib_sa.h>
+
+/*
+ * Upon receiving a device removal event, users must destroy the associated
+ * RDMA identifier and release all resources allocated with the device.
+ */
+enum rdma_cm_event_type {
+ RDMA_CM_EVENT_ADDR_RESOLVED,
+ RDMA_CM_EVENT_ADDR_ERROR,
+ RDMA_CM_EVENT_ROUTE_RESOLVED,
+ RDMA_CM_EVENT_ROUTE_ERROR,
+ RDMA_CM_EVENT_CONNECT_REQUEST,
+ RDMA_CM_EVENT_CONNECT_RESPONSE,
+ RDMA_CM_EVENT_CONNECT_ERROR,
+ RDMA_CM_EVENT_UNREACHABLE,
+ RDMA_CM_EVENT_REJECTED,
+ RDMA_CM_EVENT_ESTABLISHED,
+ RDMA_CM_EVENT_DISCONNECTED,
+ RDMA_CM_EVENT_DEVICE_REMOVAL,
+};
+
+enum rdma_port_space {
+ RDMA_PS_SDP = 0x0001,
+ RDMA_PS_TCP = 0x0106,
+ RDMA_PS_UDP = 0x0111,
+ RDMA_PS_SCTP = 0x0183
+};
+
+struct rdma_addr {
+ struct sockaddr src_addr;
+ u8 src_pad[sizeof(struct sockaddr_in6) -
+ sizeof(struct sockaddr)];
+ struct sockaddr dst_addr;
+ u8 dst_pad[sizeof(struct sockaddr_in6) -
+ sizeof(struct sockaddr)];
+ struct rdma_dev_addr dev_addr;
+};
+
+struct rdma_route {
+ struct rdma_addr addr;
+ struct ib_sa_path_rec *path_rec;
+ int num_paths;
+};
+
+struct rdma_cm_event {
+ enum rdma_cm_event_type event;
+ int status;
+ void *private_data;
+ u8 private_data_len;
+};
+
+struct rdma_cm_id;
+
+/**
+ * rdma_cm_event_handler - Callback used to report user events.
+ *
+ * Notes: Users may not call rdma_destroy_id from this callback to destroy
+ * the passed in id, or a corresponding listen id. Returning a
+ * non-zero value from the callback will destroy the corresponding id.
+ */
+typedef int (*rdma_cm_event_handler)(struct rdma_cm_id *id,
+ struct rdma_cm_event *event);
+
+struct rdma_cm_id {
+ struct ib_device *device;
+ void *context;
+ struct ib_qp *qp;
+ rdma_cm_event_handler event_handler;
+ struct rdma_route route;
+ enum rdma_port_space ps;
+ u8 port_num;
+};
+
+/**
+ * rdma_create_id - Create an RDMA identifier.
+ *
+ * @event_handler: User callback invoked to report events associated with the
+ * returned rdma_id.
+ * @context: User specified context associated with the id.
+ * @ps: RDMA port space.
+ */
+struct rdma_cm_id* rdma_create_id(rdma_cm_event_handler event_handler,
+ void *context, enum rdma_port_space ps);
+
+void rdma_destroy_id(struct rdma_cm_id *id);
+
+/**
+ * rdma_bind_addr - Bind an RDMA identifier to a source address and
+ * associated RDMA device, if needed.
+ *
+ * @id: RDMA identifier.
+ * @addr: Local address information. Wildcard values are permitted.
+ *
+ * This associates a source address with the RDMA identifier before calling
+ * rdma_listen. If a specific local address is given, the RDMA identifier will
+ * be bound to a local RDMA device.
+ */
+int rdma_bind_addr(struct rdma_cm_id *id, struct sockaddr *addr);
+
+/**
+ * rdma_resolve_addr - Resolve destination and optional source addresses
+ * from IP addresses to an RDMA address. If successful, the specified
+ * rdma_cm_id will be bound to a local device.
+ *
+ * @id: RDMA identifier.
+ * @src_addr: Source address information. This parameter may be NULL.
+ * @dst_addr: Destination address information.
+ * @timeout_ms: Time to wait for resolution to complete.
+ */
+int rdma_resolve_addr(struct rdma_cm_id *id, struct sockaddr *src_addr,
+ struct sockaddr *dst_addr, int timeout_ms);
+
+/**
+ * rdma_resolve_route - Resolve the RDMA address bound to the RDMA identifier
+ * into route information needed to establish a connection.
+ *
+ * This is called on the client side of a connection.
+ * Users must have first called rdma_resolve_addr to resolve a dst_addr
+ * into an RDMA address before calling this routine.
+ */
+int rdma_resolve_route(struct rdma_cm_id *id, int timeout_ms);
+
+/**
+ * rdma_create_qp - Allocate a QP and associate it with the specified RDMA
+ * identifier.
+ *
+ * QPs allocated to an rdma_cm_id will automatically be transitioned by the CMA
+ * through their states.
+ */
+int rdma_create_qp(struct rdma_cm_id *id, struct ib_pd *pd,
+ struct ib_qp_init_attr *qp_init_attr);
+
+/**
+ * rdma_destroy_qp - Deallocate the QP associated with the specified RDMA
+ * identifier.
+ *
+ * Users must destroy any QP associated with an RDMA identifier before
+ * destroying the RDMA ID.
+ */
+void rdma_destroy_qp(struct rdma_cm_id *id);
+
+/**
+ * rdma_init_qp_attr - Initializes the QP attributes for use in transitioning
+ * to a specified QP state.
+ * @id: Communication identifier associated with the QP attributes to
+ * initialize.
+ * @qp_attr: On input, specifies the desired QP state. On output, the
+ * mandatory and desired optional attributes will be set in order to
+ * modify the QP to the specified state.
+ * @qp_attr_mask: The QP attribute mask that may be used to transition the
+ * QP to the specified state.
+ *
+ * Users must set the @qp_attr->qp_state to the desired QP state. This call
+ * will set all required attributes for the given transition, along with
+ * known optional attributes. Users may override the attributes returned from
+ * this call before calling ib_modify_qp.
+ *
+ * Users that wish to have their QP automatically transitioned through its
+ * states can associate a QP with the rdma_cm_id by calling rdma_create_qp().
+ */
+int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
+ int *qp_attr_mask);
+
+struct rdma_conn_param {
+ const void *private_data;
+ u8 private_data_len;
+ u8 responder_resources;
+ u8 initiator_depth;
+ u8 flow_control;
+ u8 retry_count; /* ignored when accepting */
+ u8 rnr_retry_count;
+ /* Fields below ignored if a QP is created on the rdma_cm_id. */
+ u8 srq;
+ u32 qp_num;
+ enum ib_qp_type qp_type;
+};
+
+/**
+ * rdma_connect - Initiate an active connection request.
+ *
+ * Users must have resolved a route for the rdma_cm_id to connect with
+ * by having called rdma_resolve_route before calling this routine.
+ */
+int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param);
+
+/**
+ * rdma_listen - This function is called by the passive side to
+ * listen for incoming connection requests.
+ *
+ * Users must have bound the rdma_cm_id to a local address by calling
+ * rdma_bind_addr before calling this routine.
+ */
+int rdma_listen(struct rdma_cm_id *id, int backlog);
+
+/**
+ * rdma_accept - Called to accept a connection request or response.
+ * @id: Connection identifier associated with the request.
+ * @conn_param: Information needed to establish the connection. This must be
+ * provided if accepting a connection request. If accepting a connection
+ * response, this parameter must be NULL.
+ *
+ * Typically, this routine is only called by the listener to accept a connection
+ * request. It must also be called on the active side of a connection if the
+ * user is performing their own QP transitions.
+ */
+int rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param);
+
+/**
+ * rdma_reject - Called on the passive side to reject a connection request.
+ */
+int rdma_reject(struct rdma_cm_id *id, const void *private_data,
+ u8 private_data_len);
+
+/**
+ * rdma_disconnect - This function disconnects the associated QP.
+ */
+int rdma_disconnect(struct rdma_cm_id *id);
+
+#endif /* RDMA_CM_H */
+
^ permalink raw reply [flat|nested] 97+ messages in thread* [PATCH 6/6] IB: userspace support for RDMA connection manager
2006-03-03 22:53 ` [openib-general] " Roland Dreier
` (4 preceding siblings ...)
2006-03-06 19:18 ` [PATCH 5/6] IB: IP address based RDMA connection manager Sean Hefty
@ 2006-03-06 19:21 ` Sean Hefty
2006-03-06 21:33 ` Roland Dreier
` (2 more replies)
5 siblings, 3 replies; 97+ messages in thread
From: Sean Hefty @ 2006-03-06 19:21 UTC (permalink / raw)
To: 'Roland Dreier', netdev, linux-kernel; +Cc: openib-general
Kernel component necessary to support the userspace RDMA connection management
library.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
---
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/Makefile
linux-2.6.ib/drivers/infiniband/core/Makefile
--- linux-2.6.git/drivers/infiniband/core/Makefile 2006-01-16 16:58:58.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/Makefile 2006-01-16 16:55:25.000000000 -0800
@@ -1,5 +1,5 @@
obj-$(CONFIG_INFINIBAND) += ib_core.o ib_mad.o ib_sa.o \
- ib_cm.o ib_addr.o rdma_cm.o
+ ib_cm.o ib_addr.o rdma_cm.o rdma_ucm.o
obj-$(CONFIG_INFINIBAND_USER_MAD) += ib_umad.o
obj-$(CONFIG_INFINIBAND_USER_ACCESS) += ib_uverbs.o ib_ucm.o
@@ -14,6 +14,8 @@ ib_cm-y := cm.o
rdma_cm-y := cma.o
+rdma_ucm-y := ucma.o
+
ib_addr-y := addr.o
ib_umad-y := user_mad.o
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/ucma.c
linux-2.6.ib/drivers/infiniband/core/ucma.c
--- linux-2.6.git/drivers/infiniband/core/ucma.c 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/ucma.c 2006-01-16 16:54:31.000000000 -0800
@@ -0,0 +1,788 @@
+/*
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <linux/poll.h>
+#include <linux/idr.h>
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/miscdevice.h>
+
+#include <rdma/rdma_user_cm.h>
+#include <rdma/ib_marshall.h>
+#include <rdma/rdma_cm.h>
+
+MODULE_AUTHOR("Sean Hefty");
+MODULE_DESCRIPTION("RDMA Userspace Connection Manager Access");
+MODULE_LICENSE("Dual BSD/GPL");
+
+enum {
+ UCMA_MAX_BACKLOG = 128
+};
+
+struct ucma_file {
+ struct semaphore mutex;
+ struct file *filp;
+ struct list_head ctxs;
+ struct list_head events;
+ wait_queue_head_t poll_wait;
+};
+
+struct ucma_context {
+ int id;
+ wait_queue_head_t wait;
+ atomic_t ref;
+ int events_reported;
+ int backlog;
+
+ struct ucma_file *file;
+ struct rdma_cm_id *cm_id;
+ __u64 uid;
+
+ struct list_head events; /* list of pending events. */
+ struct list_head file_list; /* member in file ctx list */
+};
+
+struct ucma_event {
+ struct ucma_context *ctx;
+ struct list_head file_list; /* member in file event list */
+ struct list_head ctx_list; /* member in ctx event list */
+ struct rdma_cm_id *cm_id;
+ struct rdma_ucm_event_resp resp;
+};
+
+static DECLARE_MUTEX(ctx_mutex);
+static DEFINE_IDR(ctx_idr);
+
+static struct ucma_context* ucma_get_ctx(struct ucma_file *file, int id)
+{
+ struct ucma_context *ctx;
+
+ down(&ctx_mutex);
+ ctx = idr_find(&ctx_idr, id);
+ if (!ctx)
+ ctx = ERR_PTR(-ENOENT);
+ else if (ctx->file != file)
+ ctx = ERR_PTR(-EINVAL);
+ else
+ atomic_inc(&ctx->ref);
+ up(&ctx_mutex);
+
+ return ctx;
+}
+
+static void ucma_put_ctx(struct ucma_context *ctx)
+{
+ if (atomic_dec_and_test(&ctx->ref))
+ wake_up(&ctx->wait);
+}
+
+static void ucma_cleanup_events(struct ucma_context *ctx)
+{
+ struct ucma_event *uevent;
+
+ down(&ctx->file->mutex);
+ list_del(&ctx->file_list);
+ while (!list_empty(&ctx->events)) {
+
+ uevent = list_entry(ctx->events.next, struct ucma_event,
+ ctx_list);
+ list_del(&uevent->file_list);
+ list_del(&uevent->ctx_list);
+
+ /* clear incoming connections. */
+ if (uevent->resp.event == RDMA_CM_EVENT_CONNECT_REQUEST)
+ rdma_destroy_id(uevent->cm_id);
+
+ kfree(uevent);
+ }
+ up(&ctx->file->mutex);
+}
+
+static struct ucma_context* ucma_alloc_ctx(struct ucma_file *file)
+{
+ struct ucma_context *ctx;
+ int ret;
+
+ ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+ if (!ctx)
+ return NULL;
+
+ atomic_set(&ctx->ref, 1);
+ init_waitqueue_head(&ctx->wait);
+ ctx->file = file;
+ INIT_LIST_HEAD(&ctx->events);
+
+ do {
+ ret = idr_pre_get(&ctx_idr, GFP_KERNEL);
+ if (!ret)
+ goto error;
+
+ down(&ctx_mutex);
+ ret = idr_get_new(&ctx_idr, ctx, &ctx->id);
+ up(&ctx_mutex);
+ } while (ret == -EAGAIN);
+
+ if (ret)
+ goto error;
+
+ list_add_tail(&ctx->file_list, &file->ctxs);
+ return ctx;
+
+error:
+ kfree(ctx);
+ return NULL;
+}
+
+static int ucma_event_handler(struct rdma_cm_id *cm_id,
+ struct rdma_cm_event *event)
+{
+ struct ucma_event *uevent;
+ struct ucma_context *ctx = cm_id->context;
+ int ret = 0;
+
+ uevent = kzalloc(sizeof(*uevent), GFP_KERNEL);
+ if (!uevent)
+ return event->event == RDMA_CM_EVENT_CONNECT_REQUEST;
+
+ uevent->ctx = ctx;
+ uevent->cm_id = cm_id;
+ uevent->resp.uid = ctx->uid;
+ uevent->resp.id = ctx->id;
+ uevent->resp.event = event->event;
+ uevent->resp.status = event->status;
+ if ((uevent->resp.private_data_len = event->private_data_len))
+ memcpy(uevent->resp.private_data, event->private_data,
+ event->private_data_len);
+
+ down(&ctx->file->mutex);
+ if (event->event == RDMA_CM_EVENT_CONNECT_REQUEST) {
+ if (!ctx->backlog) {
+ ret = -EDQUOT;
+ goto out;
+ }
+ ctx->backlog--;
+ }
+ list_add_tail(&uevent->file_list, &ctx->file->events);
+ list_add_tail(&uevent->ctx_list, &ctx->events);
+ wake_up_interruptible(&ctx->file->poll_wait);
+out:
+ up(&ctx->file->mutex);
+ return ret;
+}
+
+static ssize_t ucma_get_event(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct ucma_context *ctx;
+ struct rdma_ucm_get_event cmd;
+ struct ucma_event *uevent;
+ int ret = 0;
+ DEFINE_WAIT(wait);
+
+ if (out_len < sizeof(struct rdma_ucm_event_resp))
+ return -ENOSPC;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ down(&file->mutex);
+ while (list_empty(&file->events)) {
+ if (file->filp->f_flags & O_NONBLOCK) {
+ ret = -EAGAIN;
+ break;
+ }
+
+ if (signal_pending(current)) {
+ ret = -ERESTARTSYS;
+ break;
+ }
+
+ prepare_to_wait(&file->poll_wait, &wait, TASK_INTERRUPTIBLE);
+ up(&file->mutex);
+ schedule();
+ down(&file->mutex);
+ finish_wait(&file->poll_wait, &wait);
+ }
+
+ if (ret)
+ goto done;
+
+ uevent = list_entry(file->events.next, struct ucma_event, file_list);
+
+ if (uevent->resp.event == RDMA_CM_EVENT_CONNECT_REQUEST) {
+ ctx = ucma_alloc_ctx(file);
+ if (!ctx) {
+ ret = -ENOMEM;
+ goto done;
+ }
+ uevent->ctx->backlog++;
+ ctx->cm_id = uevent->cm_id;
+ ctx->cm_id->context = ctx;
+ uevent->resp.id = ctx->id;
+ }
+
+ if (copy_to_user((void __user *)(unsigned long)cmd.response,
+ &uevent->resp, sizeof(uevent->resp))) {
+ ret = -EFAULT;
+ goto done;
+ }
+
+ list_del(&uevent->file_list);
+ list_del(&uevent->ctx_list);
+ uevent->ctx->events_reported++;
+ kfree(uevent);
+done:
+ up(&file->mutex);
+ return ret;
+}
+
+static ssize_t ucma_create_id(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_create_id cmd;
+ struct rdma_ucm_create_id_resp resp;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (out_len < sizeof(resp))
+ return -ENOSPC;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ down(&file->mutex);
+ ctx = ucma_alloc_ctx(file);
+ up(&file->mutex);
+ if (!ctx)
+ return -ENOMEM;
+
+ ctx->uid = cmd.uid;
+ ctx->cm_id = rdma_create_id(ucma_event_handler, ctx, RDMA_PS_TCP);
+ if (IS_ERR(ctx->cm_id)) {
+ ret = PTR_ERR(ctx->cm_id);
+ goto err1;
+ }
+
+ resp.id = ctx->id;
+ if (copy_to_user((void __user *)(unsigned long)cmd.response,
+ &resp, sizeof(resp))) {
+ ret = -EFAULT;
+ goto err2;
+ }
+ return 0;
+
+err2:
+ rdma_destroy_id(ctx->cm_id);
+err1:
+ down(&ctx_mutex);
+ idr_remove(&ctx_idr, ctx->id);
+ up(&ctx_mutex);
+ kfree(ctx);
+ return ret;
+}
+
+static ssize_t ucma_destroy_id(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_destroy_id cmd;
+ struct rdma_ucm_destroy_id_resp resp;
+ struct ucma_context *ctx;
+ int ret = 0;
+
+ if (out_len < sizeof(resp))
+ return -ENOSPC;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ down(&ctx_mutex);
+ ctx = idr_find(&ctx_idr, cmd.id);
+ if (!ctx)
+ ctx = ERR_PTR(-ENOENT);
+ else if (ctx->file != file)
+ ctx = ERR_PTR(-EINVAL);
+ else
+ idr_remove(&ctx_idr, ctx->id);
+ up(&ctx_mutex);
+
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ atomic_dec(&ctx->ref);
+ wait_event(ctx->wait, !atomic_read(&ctx->ref));
+
+ /* No new events will be generated after destroying the id. */
+ rdma_destroy_id(ctx->cm_id);
+ /* Cleanup events not yet reported to the user. */
+ ucma_cleanup_events(ctx);
+
+ resp.events_reported = ctx->events_reported;
+ if (copy_to_user((void __user *)(unsigned long)cmd.response,
+ &resp, sizeof(resp)))
+ ret = -EFAULT;
+
+ kfree(ctx);
+ return ret;
+}
+
+static ssize_t ucma_bind_addr(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_bind_addr cmd;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ret = rdma_bind_addr(ctx->cm_id, (struct sockaddr *) &cmd.addr);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_resolve_addr(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_resolve_addr cmd;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ret = rdma_resolve_addr(ctx->cm_id, (struct sockaddr *) &cmd.src_addr,
+ (struct sockaddr *) &cmd.dst_addr,
+ cmd.timeout_ms);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_resolve_route(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_resolve_route cmd;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ret = rdma_resolve_route(ctx->cm_id, cmd.timeout_ms);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static void ucma_copy_ib_route(struct rdma_ucm_query_route_resp *resp,
+ struct rdma_route *route)
+{
+ struct rdma_dev_addr *dev_addr;
+
+ resp->num_paths = route->num_paths;
+ switch (route->num_paths) {
+ case 0:
+ dev_addr = &route->addr.dev_addr;
+ memcpy(&resp->ib_route[0].dgid, ib_addr_get_dgid(dev_addr),
+ sizeof(union ib_gid));
+ memcpy(&resp->ib_route[0].sgid, ib_addr_get_sgid(dev_addr),
+ sizeof(union ib_gid));
+ resp->ib_route[0].pkey = cpu_to_be16(ib_addr_get_pkey(dev_addr));
+ break;
+ case 2:
+ ib_copy_path_rec_to_user(&resp->ib_route[1],
+ &route->path_rec[1]);
+ /* fall through */
+ case 1:
+ ib_copy_path_rec_to_user(&resp->ib_route[0],
+ &route->path_rec[0]);
+ break;
+ default:
+ break;
+ }
+}
+
+static ssize_t ucma_query_route(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_query_route cmd;
+ struct rdma_ucm_query_route_resp resp;
+ struct ucma_context *ctx;
+ struct sockaddr *addr;
+ int ret = 0;
+
+ if (out_len < sizeof(resp))
+ return -ENOSPC;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ if (!ctx->cm_id->device) {
+ ret = -ENODEV;
+ goto out;
+ }
+
+ addr = &ctx->cm_id->route.addr.src_addr;
+ memcpy(&resp.src_addr, addr, addr->sa_family == AF_INET ?
+ sizeof(struct sockaddr_in) :
+ sizeof(struct sockaddr_in6));
+ addr = &ctx->cm_id->route.addr.dst_addr;
+ memcpy(&resp.dst_addr, addr, addr->sa_family == AF_INET ?
+ sizeof(struct sockaddr_in) :
+ sizeof(struct sockaddr_in6));
+ resp.node_guid = ctx->cm_id->device->node_guid;
+ resp.port_num = ctx->cm_id->port_num;
+ switch (ctx->cm_id->device->node_type) {
+ case IB_NODE_CA:
+ ucma_copy_ib_route(&resp, &ctx->cm_id->route);
+ default:
+ break;
+ }
+
+ if (copy_to_user((void __user *)(unsigned long)cmd.response,
+ &resp, sizeof(resp)))
+ ret = -EFAULT;
+
+out:
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static void ucma_copy_conn_param(struct rdma_conn_param *dst_conn,
+ struct rdma_ucm_conn_param *src_conn)
+{
+ dst_conn->private_data = src_conn->private_data;
+ dst_conn->private_data_len = src_conn->private_data_len;
+ dst_conn->responder_resources =src_conn->responder_resources;
+ dst_conn->initiator_depth = src_conn->initiator_depth;
+ dst_conn->flow_control = src_conn->flow_control;
+ dst_conn->retry_count = src_conn->retry_count;
+ dst_conn->rnr_retry_count = src_conn->rnr_retry_count;
+ dst_conn->srq = src_conn->srq;
+ dst_conn->qp_num = src_conn->qp_num;
+ dst_conn->qp_type = src_conn->qp_type;
+}
+
+static ssize_t ucma_connect(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_connect cmd;
+ struct rdma_conn_param conn_param;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ if (!cmd.conn_param.valid)
+ return -EINVAL;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ucma_copy_conn_param(&conn_param, &cmd.conn_param);
+ ret = rdma_connect(ctx->cm_id, &conn_param);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_listen(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_listen cmd;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ctx->backlog = cmd.backlog > 0 && cmd.backlog < UCMA_MAX_BACKLOG ?
+ cmd.backlog : UCMA_MAX_BACKLOG;
+ ret = rdma_listen(ctx->cm_id, ctx->backlog);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_accept(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_accept cmd;
+ struct rdma_conn_param conn_param;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ if (cmd.conn_param.valid) {
+ ctx->uid = cmd.uid;
+ ucma_copy_conn_param(&conn_param, &cmd.conn_param);
+ ret = rdma_accept(ctx->cm_id, &conn_param);
+ } else
+ ret = rdma_accept(ctx->cm_id, NULL);
+
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_reject(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_reject cmd;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ret = rdma_reject(ctx->cm_id, cmd.private_data, cmd.private_data_len);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_disconnect(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_disconnect cmd;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ret = rdma_disconnect(ctx->cm_id);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_init_qp_attr(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_init_qp_attr cmd;
+ struct ib_uverbs_qp_attr resp;
+ struct ucma_context *ctx;
+ struct ib_qp_attr qp_attr;
+ int ret;
+
+ if (out_len < sizeof(resp))
+ return -ENOSPC;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ resp.qp_attr_mask = 0;
+ memset(&qp_attr, 0, sizeof qp_attr);
+ qp_attr.qp_state = cmd.qp_state;
+ ret = rdma_init_qp_attr(ctx->cm_id, &qp_attr, &resp.qp_attr_mask);
+ if (ret)
+ goto out;
+
+ ib_copy_qp_attr_to_user(&resp, &qp_attr);
+ if (copy_to_user((void __user *)(unsigned long)cmd.response,
+ &resp, sizeof(resp)))
+ ret = -EFAULT;
+
+out:
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t (*ucma_cmd_table[])(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len) = {
+ [RDMA_USER_CM_CMD_CREATE_ID] = ucma_create_id,
+ [RDMA_USER_CM_CMD_DESTROY_ID] = ucma_destroy_id,
+ [RDMA_USER_CM_CMD_BIND_ADDR] = ucma_bind_addr,
+ [RDMA_USER_CM_CMD_RESOLVE_ADDR] = ucma_resolve_addr,
+ [RDMA_USER_CM_CMD_RESOLVE_ROUTE]= ucma_resolve_route,
+ [RDMA_USER_CM_CMD_QUERY_ROUTE] = ucma_query_route,
+ [RDMA_USER_CM_CMD_CONNECT] = ucma_connect,
+ [RDMA_USER_CM_CMD_LISTEN] = ucma_listen,
+ [RDMA_USER_CM_CMD_ACCEPT] = ucma_accept,
+ [RDMA_USER_CM_CMD_REJECT] = ucma_reject,
+ [RDMA_USER_CM_CMD_DISCONNECT] = ucma_disconnect,
+ [RDMA_USER_CM_CMD_INIT_QP_ATTR] = ucma_init_qp_attr,
+ [RDMA_USER_CM_CMD_GET_EVENT] = ucma_get_event
+};
+
+static ssize_t ucma_write(struct file *filp, const char __user *buf,
+ size_t len, loff_t *pos)
+{
+ struct ucma_file *file = filp->private_data;
+ struct rdma_ucm_cmd_hdr hdr;
+ ssize_t ret;
+
+ if (len < sizeof(hdr))
+ return -EINVAL;
+
+ if (copy_from_user(&hdr, buf, sizeof(hdr)))
+ return -EFAULT;
+
+ if (hdr.cmd < 0 || hdr.cmd >= ARRAY_SIZE(ucma_cmd_table))
+ return -EINVAL;
+
+ if (hdr.in + sizeof(hdr) > len)
+ return -EINVAL;
+
+ ret = ucma_cmd_table[hdr.cmd](file, buf + sizeof(hdr), hdr.in, hdr.out);
+ if (!ret)
+ ret = len;
+
+ return ret;
+}
+
+static unsigned int ucma_poll(struct file *filp, struct poll_table_struct *wait)
+{
+ struct ucma_file *file = filp->private_data;
+ unsigned int mask = 0;
+
+ poll_wait(filp, &file->poll_wait, wait);
+
+ down(&file->mutex);
+ if (!list_empty(&file->events))
+ mask = POLLIN | POLLRDNORM;
+ up(&file->mutex);
+
+ return mask;
+}
+
+static int ucma_open(struct inode *inode, struct file *filp)
+{
+ struct ucma_file *file;
+
+ file = kmalloc(sizeof *file, GFP_KERNEL);
+ if (!file)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&file->events);
+ INIT_LIST_HEAD(&file->ctxs);
+ init_waitqueue_head(&file->poll_wait);
+ init_MUTEX(&file->mutex);
+
+ filp->private_data = file;
+ file->filp = filp;
+ return 0;
+}
+
+static int ucma_close(struct inode *inode, struct file *filp)
+{
+ struct ucma_file *file = filp->private_data;
+ struct ucma_context *ctx;
+
+ down(&file->mutex);
+ while (!list_empty(&file->ctxs)) {
+ ctx = list_entry(file->ctxs.next, struct ucma_context,
+ file_list);
+ up(&file->mutex);
+
+ down(&ctx_mutex);
+ idr_remove(&ctx_idr, ctx->id);
+ up(&ctx_mutex);
+
+ rdma_destroy_id(ctx->cm_id);
+ ucma_cleanup_events(ctx);
+ kfree(ctx);
+
+ down(&file->mutex);
+ }
+ up(&file->mutex);
+ kfree(file);
+ return 0;
+}
+
+static struct file_operations ucma_fops = {
+ .owner = THIS_MODULE,
+ .open = ucma_open,
+ .release = ucma_close,
+ .write = ucma_write,
+ .poll = ucma_poll,
+};
+
+static struct miscdevice ucma_misc = {
+ .minor = MISC_DYNAMIC_MINOR,
+ .name = "rdma_cm",
+ .fops = &ucma_fops,
+};
+
+static int __init ucma_init(void)
+{
+ return misc_register(&ucma_misc);
+}
+
+static void __exit ucma_cleanup(void)
+{
+ misc_deregister(&ucma_misc);
+ idr_destroy(&ctx_idr);
+}
+
+module_init(ucma_init);
+module_exit(ucma_cleanup);
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/rdma_user_cm.h
linux-2.6.ib/include/rdma/rdma_user_cm.h
--- linux-2.6.git/include/rdma/rdma_user_cm.h 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/include/rdma/rdma_user_cm.h 2006-01-16 16:54:55.000000000 -0800
@@ -0,0 +1,186 @@
+/*
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef RDMA_USER_CM_H
+#define RDMA_USER_CM_H
+
+#include <linux/types.h>
+#include <linux/in6.h>
+#include <rdma/ib_user_verbs.h>
+#include <rdma/ib_user_sa.h>
+
+#define RDMA_USER_CM_ABI_VERSION 1
+
+#define RDMA_MAX_PRIVATE_DATA 256
+
+enum {
+ RDMA_USER_CM_CMD_CREATE_ID,
+ RDMA_USER_CM_CMD_DESTROY_ID,
+ RDMA_USER_CM_CMD_BIND_ADDR,
+ RDMA_USER_CM_CMD_RESOLVE_ADDR,
+ RDMA_USER_CM_CMD_RESOLVE_ROUTE,
+ RDMA_USER_CM_CMD_QUERY_ROUTE,
+ RDMA_USER_CM_CMD_CONNECT,
+ RDMA_USER_CM_CMD_LISTEN,
+ RDMA_USER_CM_CMD_ACCEPT,
+ RDMA_USER_CM_CMD_REJECT,
+ RDMA_USER_CM_CMD_DISCONNECT,
+ RDMA_USER_CM_CMD_INIT_QP_ATTR,
+ RDMA_USER_CM_CMD_GET_EVENT
+};
+
+/*
+ * command ABI structures.
+ */
+struct rdma_ucm_cmd_hdr {
+ __u32 cmd;
+ __u16 in;
+ __u16 out;
+};
+
+struct rdma_ucm_create_id {
+ __u64 uid;
+ __u64 response;
+};
+
+struct rdma_ucm_create_id_resp {
+ __u32 id;
+};
+
+struct rdma_ucm_destroy_id {
+ __u64 response;
+ __u32 id;
+ __u32 reserved;
+};
+
+struct rdma_ucm_destroy_id_resp {
+ __u32 events_reported;
+};
+
+struct rdma_ucm_bind_addr {
+ __u64 response;
+ struct sockaddr_in6 addr;
+ __u32 id;
+};
+
+struct rdma_ucm_resolve_addr {
+ struct sockaddr_in6 src_addr;
+ struct sockaddr_in6 dst_addr;
+ __u32 id;
+ __u32 timeout_ms;
+};
+
+struct rdma_ucm_resolve_route {
+ __u32 id;
+ __u32 timeout_ms;
+};
+
+struct rdma_ucm_query_route {
+ __u64 response;
+ __u32 id;
+ __u32 reserved;
+};
+
+struct rdma_ucm_query_route_resp {
+ __u64 node_guid;
+ struct ib_user_path_rec ib_route[2];
+ struct sockaddr_in6 src_addr;
+ struct sockaddr_in6 dst_addr;
+ __u32 num_paths;
+ __u8 port_num;
+ __u8 reserved[3];
+};
+
+struct rdma_ucm_conn_param {
+ __u32 qp_num;
+ __u32 qp_type;
+ __u8 private_data[RDMA_MAX_PRIVATE_DATA];
+ __u8 private_data_len;
+ __u8 srq;
+ __u8 responder_resources;
+ __u8 initiator_depth;
+ __u8 flow_control;
+ __u8 retry_count;
+ __u8 rnr_retry_count;
+ __u8 valid;
+};
+
+struct rdma_ucm_connect {
+ struct rdma_ucm_conn_param conn_param;
+ __u32 id;
+ __u32 reserved;
+};
+
+struct rdma_ucm_listen {
+ __u32 id;
+ __u32 backlog;
+};
+
+struct rdma_ucm_accept {
+ __u64 uid;
+ struct rdma_ucm_conn_param conn_param;
+ __u32 id;
+ __u32 reserved;
+};
+
+struct rdma_ucm_reject {
+ __u32 id;
+ __u8 private_data_len;
+ __u8 reserved[3];
+ __u8 private_data[RDMA_MAX_PRIVATE_DATA];
+};
+
+struct rdma_ucm_disconnect {
+ __u32 id;
+};
+
+struct rdma_ucm_init_qp_attr {
+ __u64 response;
+ __u32 id;
+ __u32 qp_state;
+};
+
+struct rdma_ucm_get_event {
+ __u64 response;
+};
+
+struct rdma_ucm_event_resp {
+ __u64 uid;
+ __u32 id;
+ __u32 event;
+ __u32 status;
+ __u8 private_data_len;
+ __u8 reserved[3];
+ __u8 private_data[RDMA_MAX_PRIVATE_DATA];
+};
+
+#endif /* RDMA_USER_CM_H */
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: [PATCH 6/6] IB: userspace support for RDMA connection manager
2006-03-06 19:21 ` [PATCH 6/6] IB: userspace support for " Sean Hefty
@ 2006-03-06 21:33 ` Roland Dreier
2006-03-06 21:42 ` [openib-general] " Sean Hefty
2006-03-06 21:35 ` Roland Dreier
2006-03-06 23:41 ` [PATCH 6/6 v2] " Sean Hefty
2 siblings, 1 reply; 97+ messages in thread
From: Roland Dreier @ 2006-03-06 21:33 UTC (permalink / raw)
To: Sean Hefty; +Cc: netdev, linux-kernel, openib-general
> +struct rdma_ucm_query_route_resp {
> + __u64 node_guid;
> + struct ib_user_path_rec ib_route[2];
> + struct sockaddr_in6 src_addr;
> + struct sockaddr_in6 dst_addr;
> + __u32 num_paths;
> + __u8 port_num;
> + __u8 reserved[3];
> +};
Is there a 32-bit/64-bit compatibility problem here? From a quick
look, struct sockaddr_in6 is not 8-byte aligned.
- R.
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: [openib-general] Re: [PATCH 6/6] IB: userspace support for RDMA connection manager
2006-03-06 21:33 ` Roland Dreier
@ 2006-03-06 21:42 ` Sean Hefty
2006-03-06 21:58 ` Roland Dreier
0 siblings, 1 reply; 97+ messages in thread
From: Sean Hefty @ 2006-03-06 21:42 UTC (permalink / raw)
To: Roland Dreier; +Cc: Sean Hefty, netdev, linux-kernel, openib-general
Roland Dreier wrote:
> > +struct rdma_ucm_query_route_resp {
> > + __u64 node_guid;
> > + struct ib_user_path_rec ib_route[2];
> > + struct sockaddr_in6 src_addr;
> > + struct sockaddr_in6 dst_addr;
> > + __u32 num_paths;
> > + __u8 port_num;
> > + __u8 reserved[3];
> > +};
>
> Is there a 32-bit/64-bit compatibility problem here? From a quick
> look, struct sockaddr_in6 is not 8-byte aligned.
Unless I miss counted, they should be aligned. ib_user_path_rec is defined near
the end of patch 1/6.
+struct ib_user_path_rec {
+ __u8 dgid[16];
+ __u8 sgid[16];
+ __be16 dlid;
+ __be16 slid;
+ __u32 raw_traffic;
+ __be32 flow_label;
+ __u32 reversible;
+ __u32 mtu;
+ __be16 pkey;
+ __u8 hop_limit;
+ __u8 traffic_class;
+ __u8 numb_path;
+ __u8 sl;
+ __u8 mtu_selector;
+ __u8 rate_selector;
+ __u8 rate;
+ __u8 packet_life_time_selector;
+ __u8 packet_life_time;
+ __u8 preference;
+};
- Sean
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: [openib-general] Re: [PATCH 6/6] IB: userspace support for RDMA connection manager
2006-03-06 21:42 ` [openib-general] " Sean Hefty
@ 2006-03-06 21:58 ` Roland Dreier
2006-03-06 22:28 ` David S. Miller
0 siblings, 1 reply; 97+ messages in thread
From: Roland Dreier @ 2006-03-06 21:58 UTC (permalink / raw)
To: Sean Hefty; +Cc: Sean Hefty, netdev, linux-kernel, openib-general
Sean> Unless I miss counted, they should be aligned.
Sean> ib_user_path_rec is defined near the end of patch 1/6.
You're right. struct sockaddr_in6 is 28 bytes long (not a multiple of
8) but gcc seems to lay everything out the same on 32-bit and 64-bit
architectures just the same.
- R.
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: [openib-general] Re: [PATCH 6/6] IB: userspace support for RDMA connection manager
2006-03-06 21:58 ` Roland Dreier
@ 2006-03-06 22:28 ` David S. Miller
2006-03-06 22:32 ` Roland Dreier
0 siblings, 1 reply; 97+ messages in thread
From: David S. Miller @ 2006-03-06 22:28 UTC (permalink / raw)
To: rdreier; +Cc: mshefty, sean.hefty, netdev, linux-kernel, openib-general
From: Roland Dreier <rdreier@cisco.com>
Date: Mon, 06 Mar 2006 13:58:32 -0800
> Sean> Unless I miss counted, they should be aligned.
> Sean> ib_user_path_rec is defined near the end of patch 1/6.
>
> You're right. struct sockaddr_in6 is 28 bytes long (not a multiple of
> 8) but gcc seems to lay everything out the same on 32-bit and 64-bit
> architectures just the same.
Please make sure you check "x86_64 vs. x86", and then something like
"powerpc64 vs. powerpc32" or "sparc64 vs. sparc32", as those are the
two different classes of ABI layouts.
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: [openib-general] Re: [PATCH 6/6] IB: userspace support for RDMA connection manager
2006-03-06 22:28 ` David S. Miller
@ 2006-03-06 22:32 ` Roland Dreier
2006-03-06 22:39 ` David S. Miller
0 siblings, 1 reply; 97+ messages in thread
From: Roland Dreier @ 2006-03-06 22:32 UTC (permalink / raw)
To: David S. Miller; +Cc: mshefty, sean.hefty, netdev, linux-kernel, openib-general
David> Please make sure you check "x86_64 vs. x86", and then
David> something like "powerpc64 vs. powerpc32" or "sparc64
David> vs. sparc32", as those are the two different classes of ABI
David> layouts.
Yes, I tried ppc64 vs ppc and it still comes out the same.
Unfortunately I don't have any sparc handy to try.
The fundamental question seems to be whether things like
struct foo {
struct sockaddr_in6 src;
struct sockaddr_in6 dst;
};
and
struct bar {
struct sockaddr_in6 a;
__u32 b;
};
end up being packed, even though struct sockaddr_in6 is 28 bytes in
size. And as far as I can tell, they always do, I guess because the
individual fields of struct sockaddr_in6 are all <= 32 bits.
- R.
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: [openib-general] Re: [PATCH 6/6] IB: userspace support for RDMA connection manager
2006-03-06 22:32 ` Roland Dreier
@ 2006-03-06 22:39 ` David S. Miller
2006-03-06 22:41 ` Roland Dreier
0 siblings, 1 reply; 97+ messages in thread
From: David S. Miller @ 2006-03-06 22:39 UTC (permalink / raw)
To: rdreier; +Cc: mshefty, sean.hefty, netdev, linux-kernel, openib-general
From: Roland Dreier <rdreier@cisco.com>
Date: Mon, 06 Mar 2006 14:32:28 -0800
> The fundamental question seems to be whether things like
>
> struct foo {
> struct sockaddr_in6 src;
> struct sockaddr_in6 dst;
> };
>
> and
>
> struct bar {
> struct sockaddr_in6 a;
> __u32 b;
> };
I wrote a test program and it looks ok:
davem@sunset:~/src/GIT/sparc-2.6.17$ gcc -m32 -O -o foo foo.c
davem@sunset:~/src/GIT/sparc-2.6.17$ ./foo
SPARC32
foo src: 0
foo dst: 28
bar a: 0
bar b: 28
davem@sunset:~/src/GIT/sparc-2.6.17$ gcc -m64 -O -o foo foo.c
davem@sunset:~/src/GIT/sparc-2.6.17$ ./foo
SPARC64
foo src: 0
foo dst: 28
bar a: 0
bar b: 28
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: [openib-general] Re: [PATCH 6/6] IB: userspace support for RDMA connection manager
2006-03-06 22:39 ` David S. Miller
@ 2006-03-06 22:41 ` Roland Dreier
2006-03-06 22:50 ` David S. Miller
0 siblings, 1 reply; 97+ messages in thread
From: Roland Dreier @ 2006-03-06 22:41 UTC (permalink / raw)
To: David S. Miller; +Cc: mshefty, sean.hefty, netdev, linux-kernel, openib-general
David> I wrote a test program and it looks ok:
Cool, thanks.
I should look into getting some niagara machines to test with -- with
PCIe slots they should actually be good for IB testing.
- R.
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: [openib-general] Re: [PATCH 6/6] IB: userspace support for RDMA connection manager
2006-03-06 22:41 ` Roland Dreier
@ 2006-03-06 22:50 ` David S. Miller
2006-03-06 23:40 ` Roland Dreier
0 siblings, 1 reply; 97+ messages in thread
From: David S. Miller @ 2006-03-06 22:50 UTC (permalink / raw)
To: rdreier; +Cc: mshefty, sean.hefty, netdev, linux-kernel, openib-general
From: Roland Dreier <rdreier@cisco.com>
Date: Mon, 06 Mar 2006 14:41:21 -0800
> I should look into getting some niagara machines to test with -- with
> PCIe slots they should actually be good for IB testing.
You'll be cpu limited until we have Van Jacobson net channels.
Also, since our existing Linux "generic" MSI code is so riddled with
x86'isms (it was written by an Intel person, so this is just the
status quo), it will be a while before MSI interrupts are supported on
sparc64.
I haven't gotten around to working on that problem yet, and PPC needs
this work too as they now have MSI capable PCI controllers.
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: [openib-general] Re: [PATCH 6/6] IB: userspace support for RDMA connection manager
2006-03-06 22:50 ` David S. Miller
@ 2006-03-06 23:40 ` Roland Dreier
2006-03-07 0:05 ` Bryan O'Sullivan
2006-03-07 0:26 ` David S. Miller
0 siblings, 2 replies; 97+ messages in thread
From: Roland Dreier @ 2006-03-06 23:40 UTC (permalink / raw)
To: David S. Miller; +Cc: mshefty, sean.hefty, netdev, linux-kernel, openib-general
Roland> I should look into getting some niagara machines to test
Roland> with -- with PCIe slots they should actually be good for
Roland> IB testing.
David> You'll be cpu limited until we have Van Jacobson net
David> channels.
For IPoIB maybe but not for native IB which offloads all transport to
the HCA... I'd be surprised if the bottleneck were anywhere other than
the bus, even on niagara.
David> Also, since our existing Linux "generic" MSI code is so
David> riddled with x86'isms (it was written by an Intel person,
David> so this is just the status quo), it will be a while before
David> MSI interrupts are supported on sparc64.
Yeah, I've always wanted to make the MSI stuff generic and handle the
embedded ppc chips that have MSI, but I've never had a good enough
reason to really work on it -- it's just been at the level of "that
would be fun." Now that IBM cares I hope it will get done soon.
Anyway IB works fine with standard INTx interrupts -- MSI is just icing.
The Niagara boxes seem like a fun toy if I can get budget for it --
and 32 threads are probably good for flushing out SMP races.
- R.
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: [openib-general] Re: [PATCH 6/6] IB: userspace support for RDMA connection manager
2006-03-06 23:40 ` Roland Dreier
@ 2006-03-07 0:05 ` Bryan O'Sullivan
2006-03-07 0:10 ` Roland Dreier
2006-03-07 0:26 ` David S. Miller
1 sibling, 1 reply; 97+ messages in thread
From: Bryan O'Sullivan @ 2006-03-07 0:05 UTC (permalink / raw)
To: Roland Dreier; +Cc: David S. Miller, linux-kernel, openib-general, netdev
On Mon, 2006-03-06 at 15:40 -0800, Roland Dreier wrote:
> Anyway IB works fine with standard INTx interrupts -- MSI is just icing.
Depends on the driver. Ours needs the interrupt vector rather than the
number, which means we don't work without CONFIG_PCI_MSI. That is,
unless there's some other way to get the vector that I don't know about
(entirely likely).
<b
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: [openib-general] Re: [PATCH 6/6] IB: userspace support for RDMA connection manager
2006-03-07 0:05 ` Bryan O'Sullivan
@ 2006-03-07 0:10 ` Roland Dreier
0 siblings, 0 replies; 97+ messages in thread
From: Roland Dreier @ 2006-03-07 0:10 UTC (permalink / raw)
To: Bryan O'Sullivan
Cc: netdev, David S. Miller, openib-general, linux-kernel
Bryan> Depends on the driver. Ours needs the interrupt vector
Bryan> rather than the number, which means we don't work without
Bryan> CONFIG_PCI_MSI. That is, unless there's some other way to
Bryan> get the vector that I don't know about (entirely likely).
OK, fair enough. But that's a quirk of the x86 architecture that
you're pretty tied to -- you don't actually need MSI, you just need
the interrupt numbering that is enabled on x86 with that config option.
Since Niagara boxes don't have HT anyway it's kind of a moot point.
Although I wonder what you would have to do to make your device work
with something like the MIPS SoCs that have HT...
- R.
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: [openib-general] Re: [PATCH 6/6] IB: userspace support for RDMA connection manager
2006-03-06 23:40 ` Roland Dreier
2006-03-07 0:05 ` Bryan O'Sullivan
@ 2006-03-07 0:26 ` David S. Miller
1 sibling, 0 replies; 97+ messages in thread
From: David S. Miller @ 2006-03-07 0:26 UTC (permalink / raw)
To: rdreier; +Cc: mshefty, sean.hefty, netdev, linux-kernel, openib-general
From: Roland Dreier <rdreier@cisco.com>
Date: Mon, 06 Mar 2006 15:40:56 -0800
> and 32 threads are probably good for flushing out SMP races.
Indeed, guess what I've been spending most of my time working
on lately? :)
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: [PATCH 6/6] IB: userspace support for RDMA connection manager
2006-03-06 19:21 ` [PATCH 6/6] IB: userspace support for " Sean Hefty
2006-03-06 21:33 ` Roland Dreier
@ 2006-03-06 21:35 ` Roland Dreier
2006-03-06 21:43 ` [openib-general] " Sean Hefty
2006-03-06 23:41 ` [PATCH 6/6 v2] " Sean Hefty
2 siblings, 1 reply; 97+ messages in thread
From: Roland Dreier @ 2006-03-06 21:35 UTC (permalink / raw)
To: Sean Hefty; +Cc: netdev, linux-kernel, openib-general
I think it makes sense to merge patches 1-5 independently of this
patch. The kernel interface is needed by iSER and NFS/RDMA, and
maintaining compatibility isn't a huge deal, so we can merge it now
(assuming it looks mergable).
On the other hand I think it would be good to let this userspace
interface cook a little more, say in -mm.
Anyone have any problems with that plan?
- R.
^ permalink raw reply [flat|nested] 97+ messages in thread
* [PATCH 6/6 v2] IB: userspace support for RDMA connection manager
2006-03-06 19:21 ` [PATCH 6/6] IB: userspace support for " Sean Hefty
2006-03-06 21:33 ` Roland Dreier
2006-03-06 21:35 ` Roland Dreier
@ 2006-03-06 23:41 ` Sean Hefty
2006-03-22 1:40 ` Roland Dreier
2 siblings, 1 reply; 97+ messages in thread
From: Sean Hefty @ 2006-03-06 23:41 UTC (permalink / raw)
To: Hefty, Sean, 'Roland Dreier', netdev, linux-kernel; +Cc: openib-general
Kernel component necessary to support the userspace RDMA connection management
library.
Signed-off-by: Sean Hefty <sean.hefty@intel.com>
---
Discussion on the list suggested giving the userspace interface more time to
develop, which seems reasonable.
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/Makefile
linux-2.6.ib/drivers/infiniband/core/Makefile
--- linux-2.6.git/drivers/infiniband/core/Makefile 2006-01-16 16:58:58.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/Makefile 2006-01-16 16:55:25.000000000 -0800
@@ -1,5 +1,5 @@
obj-$(CONFIG_INFINIBAND) += ib_core.o ib_mad.o ib_sa.o \
- ib_cm.o ib_addr.o rdma_cm.o
+ ib_cm.o ib_addr.o rdma_cm.o rdma_ucm.o
obj-$(CONFIG_INFINIBAND_USER_MAD) += ib_umad.o
obj-$(CONFIG_INFINIBAND_USER_ACCESS) += ib_uverbs.o ib_ucm.o
@@ -14,6 +14,8 @@ ib_cm-y := cm.o
rdma_cm-y := cma.o
+rdma_ucm-y := ucma.o
+
ib_addr-y := addr.o
ib_umad-y := user_mad.o
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/drivers/infiniband/core/ucma.c
linux-2.6.ib/drivers/infiniband/core/ucma.c
--- linux-2.6.git/drivers/infiniband/core/ucma.c 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/drivers/infiniband/core/ucma.c 2006-01-16 16:54:31.000000000 -0800
@@ -0,0 +1,788 @@
+/*
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#include <linux/poll.h>
+#include <linux/idr.h>
+#include <linux/in.h>
+#include <linux/in6.h>
+#include <linux/miscdevice.h>
+
+#include <rdma/rdma_user_cm.h>
+#include <rdma/ib_marshall.h>
+#include <rdma/rdma_cm.h>
+
+MODULE_AUTHOR("Sean Hefty");
+MODULE_DESCRIPTION("RDMA Userspace Connection Manager Access");
+MODULE_LICENSE("Dual BSD/GPL");
+
+enum {
+ UCMA_MAX_BACKLOG = 128
+};
+
+struct ucma_file {
+ struct mutex file_mutex;
+ struct file *filp;
+ struct list_head ctxs;
+ struct list_head events;
+ wait_queue_head_t poll_wait;
+};
+
+struct ucma_context {
+ int id;
+ wait_queue_head_t wait;
+ atomic_t ref;
+ int events_reported;
+ int backlog;
+
+ struct ucma_file *file;
+ struct rdma_cm_id *cm_id;
+ __u64 uid;
+
+ struct list_head events; /* list of pending events. */
+ struct list_head file_list; /* member in file ctx list */
+};
+
+struct ucma_event {
+ struct ucma_context *ctx;
+ struct list_head file_list; /* member in file event list */
+ struct list_head ctx_list; /* member in ctx event list */
+ struct rdma_cm_id *cm_id;
+ struct rdma_ucm_event_resp resp;
+};
+
+static DEFINE_MUTEX(ctx_mutex);
+static DEFINE_IDR(ctx_idr);
+
+static struct ucma_context* ucma_get_ctx(struct ucma_file *file, int id)
+{
+ struct ucma_context *ctx;
+
+ mutex_lock(&ctx_mutex);
+ ctx = idr_find(&ctx_idr, id);
+ if (!ctx)
+ ctx = ERR_PTR(-ENOENT);
+ else if (ctx->file != file)
+ ctx = ERR_PTR(-EINVAL);
+ else
+ atomic_inc(&ctx->ref);
+ mutex_unlock(&ctx_mutex);
+
+ return ctx;
+}
+
+static void ucma_put_ctx(struct ucma_context *ctx)
+{
+ if (atomic_dec_and_test(&ctx->ref))
+ wake_up(&ctx->wait);
+}
+
+static void ucma_cleanup_events(struct ucma_context *ctx)
+{
+ struct ucma_event *uevent;
+
+ mutex_lock(&ctx->file->file_mutex);
+ list_del(&ctx->file_list);
+ while (!list_empty(&ctx->events)) {
+
+ uevent = list_entry(ctx->events.next, struct ucma_event,
+ ctx_list);
+ list_del(&uevent->file_list);
+ list_del(&uevent->ctx_list);
+
+ /* clear incoming connections. */
+ if (uevent->resp.event == RDMA_CM_EVENT_CONNECT_REQUEST)
+ rdma_destroy_id(uevent->cm_id);
+
+ kfree(uevent);
+ }
+ mutex_unlock(&ctx->file->file_mutex);
+}
+
+static struct ucma_context* ucma_alloc_ctx(struct ucma_file *file)
+{
+ struct ucma_context *ctx;
+ int ret;
+
+ ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+ if (!ctx)
+ return NULL;
+
+ atomic_set(&ctx->ref, 1);
+ init_waitqueue_head(&ctx->wait);
+ ctx->file = file;
+ INIT_LIST_HEAD(&ctx->events);
+
+ do {
+ ret = idr_pre_get(&ctx_idr, GFP_KERNEL);
+ if (!ret)
+ goto error;
+
+ mutex_lock(&ctx_mutex);
+ ret = idr_get_new(&ctx_idr, ctx, &ctx->id);
+ mutex_unlock(&ctx_mutex);
+ } while (ret == -EAGAIN);
+
+ if (ret)
+ goto error;
+
+ list_add_tail(&ctx->file_list, &file->ctxs);
+ return ctx;
+
+error:
+ kfree(ctx);
+ return NULL;
+}
+
+static int ucma_event_handler(struct rdma_cm_id *cm_id,
+ struct rdma_cm_event *event)
+{
+ struct ucma_event *uevent;
+ struct ucma_context *ctx = cm_id->context;
+ int ret = 0;
+
+ uevent = kzalloc(sizeof(*uevent), GFP_KERNEL);
+ if (!uevent)
+ return event->event == RDMA_CM_EVENT_CONNECT_REQUEST;
+
+ uevent->ctx = ctx;
+ uevent->cm_id = cm_id;
+ uevent->resp.uid = ctx->uid;
+ uevent->resp.id = ctx->id;
+ uevent->resp.event = event->event;
+ uevent->resp.status = event->status;
+ if ((uevent->resp.private_data_len = event->private_data_len))
+ memcpy(uevent->resp.private_data, event->private_data,
+ event->private_data_len);
+
+ mutex_lock(&ctx->file->file_mutex);
+ if (event->event == RDMA_CM_EVENT_CONNECT_REQUEST) {
+ if (!ctx->backlog) {
+ ret = -EDQUOT;
+ goto out;
+ }
+ ctx->backlog--;
+ }
+ list_add_tail(&uevent->file_list, &ctx->file->events);
+ list_add_tail(&uevent->ctx_list, &ctx->events);
+ wake_up_interruptible(&ctx->file->poll_wait);
+out:
+ mutex_unlock(&ctx->file->file_mutex);
+ return ret;
+}
+
+static ssize_t ucma_get_event(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct ucma_context *ctx;
+ struct rdma_ucm_get_event cmd;
+ struct ucma_event *uevent;
+ int ret = 0;
+ DEFINE_WAIT(wait);
+
+ if (out_len < sizeof(struct rdma_ucm_event_resp))
+ return -ENOSPC;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ mutex_lock(&file->file_mutex);
+ while (list_empty(&file->events)) {
+ if (file->filp->f_flags & O_NONBLOCK) {
+ ret = -EAGAIN;
+ break;
+ }
+
+ if (signal_pending(current)) {
+ ret = -ERESTARTSYS;
+ break;
+ }
+
+ prepare_to_wait(&file->poll_wait, &wait, TASK_INTERRUPTIBLE);
+ mutex_unlock(&file->file_mutex);
+ schedule();
+ mutex_lock(&file->file_mutex);
+ finish_wait(&file->poll_wait, &wait);
+ }
+
+ if (ret)
+ goto done;
+
+ uevent = list_entry(file->events.next, struct ucma_event, file_list);
+
+ if (uevent->resp.event == RDMA_CM_EVENT_CONNECT_REQUEST) {
+ ctx = ucma_alloc_ctx(file);
+ if (!ctx) {
+ ret = -ENOMEM;
+ goto done;
+ }
+ uevent->ctx->backlog++;
+ ctx->cm_id = uevent->cm_id;
+ ctx->cm_id->context = ctx;
+ uevent->resp.id = ctx->id;
+ }
+
+ if (copy_to_user((void __user *)(unsigned long)cmd.response,
+ &uevent->resp, sizeof(uevent->resp))) {
+ ret = -EFAULT;
+ goto done;
+ }
+
+ list_del(&uevent->file_list);
+ list_del(&uevent->ctx_list);
+ uevent->ctx->events_reported++;
+ kfree(uevent);
+done:
+ mutex_unlock(&file->file_mutex);
+ return ret;
+}
+
+static ssize_t ucma_create_id(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_create_id cmd;
+ struct rdma_ucm_create_id_resp resp;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (out_len < sizeof(resp))
+ return -ENOSPC;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ mutex_lock(&file->file_mutex);
+ ctx = ucma_alloc_ctx(file);
+ mutex_unlock(&file->file_mutex);
+ if (!ctx)
+ return -ENOMEM;
+
+ ctx->uid = cmd.uid;
+ ctx->cm_id = rdma_create_id(ucma_event_handler, ctx, RDMA_PS_TCP);
+ if (IS_ERR(ctx->cm_id)) {
+ ret = PTR_ERR(ctx->cm_id);
+ goto err1;
+ }
+
+ resp.id = ctx->id;
+ if (copy_to_user((void __user *)(unsigned long)cmd.response,
+ &resp, sizeof(resp))) {
+ ret = -EFAULT;
+ goto err2;
+ }
+ return 0;
+
+err2:
+ rdma_destroy_id(ctx->cm_id);
+err1:
+ mutex_lock(&ctx_mutex);
+ idr_remove(&ctx_idr, ctx->id);
+ mutex_unlock(&ctx_mutex);
+ kfree(ctx);
+ return ret;
+}
+
+static ssize_t ucma_destroy_id(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_destroy_id cmd;
+ struct rdma_ucm_destroy_id_resp resp;
+ struct ucma_context *ctx;
+ int ret = 0;
+
+ if (out_len < sizeof(resp))
+ return -ENOSPC;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ mutex_lock(&ctx_mutex);
+ ctx = idr_find(&ctx_idr, cmd.id);
+ if (!ctx)
+ ctx = ERR_PTR(-ENOENT);
+ else if (ctx->file != file)
+ ctx = ERR_PTR(-EINVAL);
+ else
+ idr_remove(&ctx_idr, ctx->id);
+ mutex_unlock(&ctx_mutex);
+
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ atomic_dec(&ctx->ref);
+ wait_event(ctx->wait, !atomic_read(&ctx->ref));
+
+ /* No new events will be generated after destroying the id. */
+ rdma_destroy_id(ctx->cm_id);
+ /* Cleanup events not yet reported to the user. */
+ ucma_cleanup_events(ctx);
+
+ resp.events_reported = ctx->events_reported;
+ if (copy_to_user((void __user *)(unsigned long)cmd.response,
+ &resp, sizeof(resp)))
+ ret = -EFAULT;
+
+ kfree(ctx);
+ return ret;
+}
+
+static ssize_t ucma_bind_addr(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_bind_addr cmd;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ret = rdma_bind_addr(ctx->cm_id, (struct sockaddr *) &cmd.addr);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_resolve_addr(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_resolve_addr cmd;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ret = rdma_resolve_addr(ctx->cm_id, (struct sockaddr *) &cmd.src_addr,
+ (struct sockaddr *) &cmd.dst_addr,
+ cmd.timeout_ms);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_resolve_route(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_resolve_route cmd;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ret = rdma_resolve_route(ctx->cm_id, cmd.timeout_ms);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static void ucma_copy_ib_route(struct rdma_ucm_query_route_resp *resp,
+ struct rdma_route *route)
+{
+ struct rdma_dev_addr *dev_addr;
+
+ resp->num_paths = route->num_paths;
+ switch (route->num_paths) {
+ case 0:
+ dev_addr = &route->addr.dev_addr;
+ memcpy(&resp->ib_route[0].dgid, ib_addr_get_dgid(dev_addr),
+ sizeof(union ib_gid));
+ memcpy(&resp->ib_route[0].sgid, ib_addr_get_sgid(dev_addr),
+ sizeof(union ib_gid));
+ resp->ib_route[0].pkey = cpu_to_be16(ib_addr_get_pkey(dev_addr));
+ break;
+ case 2:
+ ib_copy_path_rec_to_user(&resp->ib_route[1],
+ &route->path_rec[1]);
+ /* fall through */
+ case 1:
+ ib_copy_path_rec_to_user(&resp->ib_route[0],
+ &route->path_rec[0]);
+ break;
+ default:
+ break;
+ }
+}
+
+static ssize_t ucma_query_route(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_query_route cmd;
+ struct rdma_ucm_query_route_resp resp;
+ struct ucma_context *ctx;
+ struct sockaddr *addr;
+ int ret = 0;
+
+ if (out_len < sizeof(resp))
+ return -ENOSPC;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ if (!ctx->cm_id->device) {
+ ret = -ENODEV;
+ goto out;
+ }
+
+ addr = &ctx->cm_id->route.addr.src_addr;
+ memcpy(&resp.src_addr, addr, addr->sa_family == AF_INET ?
+ sizeof(struct sockaddr_in) :
+ sizeof(struct sockaddr_in6));
+ addr = &ctx->cm_id->route.addr.dst_addr;
+ memcpy(&resp.dst_addr, addr, addr->sa_family == AF_INET ?
+ sizeof(struct sockaddr_in) :
+ sizeof(struct sockaddr_in6));
+ resp.node_guid = ctx->cm_id->device->node_guid;
+ resp.port_num = ctx->cm_id->port_num;
+ switch (ctx->cm_id->device->node_type) {
+ case IB_NODE_CA:
+ ucma_copy_ib_route(&resp, &ctx->cm_id->route);
+ default:
+ break;
+ }
+
+ if (copy_to_user((void __user *)(unsigned long)cmd.response,
+ &resp, sizeof(resp)))
+ ret = -EFAULT;
+
+out:
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static void ucma_copy_conn_param(struct rdma_conn_param *dst_conn,
+ struct rdma_ucm_conn_param *src_conn)
+{
+ dst_conn->private_data = src_conn->private_data;
+ dst_conn->private_data_len = src_conn->private_data_len;
+ dst_conn->responder_resources =src_conn->responder_resources;
+ dst_conn->initiator_depth = src_conn->initiator_depth;
+ dst_conn->flow_control = src_conn->flow_control;
+ dst_conn->retry_count = src_conn->retry_count;
+ dst_conn->rnr_retry_count = src_conn->rnr_retry_count;
+ dst_conn->srq = src_conn->srq;
+ dst_conn->qp_num = src_conn->qp_num;
+ dst_conn->qp_type = src_conn->qp_type;
+}
+
+static ssize_t ucma_connect(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_connect cmd;
+ struct rdma_conn_param conn_param;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ if (!cmd.conn_param.valid)
+ return -EINVAL;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ucma_copy_conn_param(&conn_param, &cmd.conn_param);
+ ret = rdma_connect(ctx->cm_id, &conn_param);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_listen(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_listen cmd;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ctx->backlog = cmd.backlog > 0 && cmd.backlog < UCMA_MAX_BACKLOG ?
+ cmd.backlog : UCMA_MAX_BACKLOG;
+ ret = rdma_listen(ctx->cm_id, ctx->backlog);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_accept(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_accept cmd;
+ struct rdma_conn_param conn_param;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ if (cmd.conn_param.valid) {
+ ctx->uid = cmd.uid;
+ ucma_copy_conn_param(&conn_param, &cmd.conn_param);
+ ret = rdma_accept(ctx->cm_id, &conn_param);
+ } else
+ ret = rdma_accept(ctx->cm_id, NULL);
+
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_reject(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_reject cmd;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ret = rdma_reject(ctx->cm_id, cmd.private_data, cmd.private_data_len);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_disconnect(struct ucma_file *file, const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_disconnect cmd;
+ struct ucma_context *ctx;
+ int ret;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ ret = rdma_disconnect(ctx->cm_id);
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t ucma_init_qp_attr(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len)
+{
+ struct rdma_ucm_init_qp_attr cmd;
+ struct ib_uverbs_qp_attr resp;
+ struct ucma_context *ctx;
+ struct ib_qp_attr qp_attr;
+ int ret;
+
+ if (out_len < sizeof(resp))
+ return -ENOSPC;
+
+ if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
+ return -EFAULT;
+
+ ctx = ucma_get_ctx(file, cmd.id);
+ if (IS_ERR(ctx))
+ return PTR_ERR(ctx);
+
+ resp.qp_attr_mask = 0;
+ memset(&qp_attr, 0, sizeof qp_attr);
+ qp_attr.qp_state = cmd.qp_state;
+ ret = rdma_init_qp_attr(ctx->cm_id, &qp_attr, &resp.qp_attr_mask);
+ if (ret)
+ goto out;
+
+ ib_copy_qp_attr_to_user(&resp, &qp_attr);
+ if (copy_to_user((void __user *)(unsigned long)cmd.response,
+ &resp, sizeof(resp)))
+ ret = -EFAULT;
+
+out:
+ ucma_put_ctx(ctx);
+ return ret;
+}
+
+static ssize_t (*ucma_cmd_table[])(struct ucma_file *file,
+ const char __user *inbuf,
+ int in_len, int out_len) = {
+ [RDMA_USER_CM_CMD_CREATE_ID] = ucma_create_id,
+ [RDMA_USER_CM_CMD_DESTROY_ID] = ucma_destroy_id,
+ [RDMA_USER_CM_CMD_BIND_ADDR] = ucma_bind_addr,
+ [RDMA_USER_CM_CMD_RESOLVE_ADDR] = ucma_resolve_addr,
+ [RDMA_USER_CM_CMD_RESOLVE_ROUTE]= ucma_resolve_route,
+ [RDMA_USER_CM_CMD_QUERY_ROUTE] = ucma_query_route,
+ [RDMA_USER_CM_CMD_CONNECT] = ucma_connect,
+ [RDMA_USER_CM_CMD_LISTEN] = ucma_listen,
+ [RDMA_USER_CM_CMD_ACCEPT] = ucma_accept,
+ [RDMA_USER_CM_CMD_REJECT] = ucma_reject,
+ [RDMA_USER_CM_CMD_DISCONNECT] = ucma_disconnect,
+ [RDMA_USER_CM_CMD_INIT_QP_ATTR] = ucma_init_qp_attr,
+ [RDMA_USER_CM_CMD_GET_EVENT] = ucma_get_event
+};
+
+static ssize_t ucma_write(struct file *filp, const char __user *buf,
+ size_t len, loff_t *pos)
+{
+ struct ucma_file *file = filp->private_data;
+ struct rdma_ucm_cmd_hdr hdr;
+ ssize_t ret;
+
+ if (len < sizeof(hdr))
+ return -EINVAL;
+
+ if (copy_from_user(&hdr, buf, sizeof(hdr)))
+ return -EFAULT;
+
+ if (hdr.cmd < 0 || hdr.cmd >= ARRAY_SIZE(ucma_cmd_table))
+ return -EINVAL;
+
+ if (hdr.in + sizeof(hdr) > len)
+ return -EINVAL;
+
+ ret = ucma_cmd_table[hdr.cmd](file, buf + sizeof(hdr), hdr.in, hdr.out);
+ if (!ret)
+ ret = len;
+
+ return ret;
+}
+
+static unsigned int ucma_poll(struct file *filp, struct poll_table_struct *wait)
+{
+ struct ucma_file *file = filp->private_data;
+ unsigned int mask = 0;
+
+ poll_wait(filp, &file->poll_wait, wait);
+
+ mutex_lock(&file->file_mutex);
+ if (!list_empty(&file->events))
+ mask = POLLIN | POLLRDNORM;
+ mutex_unlock(&file->file_mutex);
+
+ return mask;
+}
+
+static int ucma_open(struct inode *inode, struct file *filp)
+{
+ struct ucma_file *file;
+
+ file = kmalloc(sizeof *file, GFP_KERNEL);
+ if (!file)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&file->events);
+ INIT_LIST_HEAD(&file->ctxs);
+ init_waitqueue_head(&file->poll_wait);
+ mutex_init(&file->file_mutex);
+
+ filp->private_data = file;
+ file->filp = filp;
+ return 0;
+}
+
+static int ucma_close(struct inode *inode, struct file *filp)
+{
+ struct ucma_file *file = filp->private_data;
+ struct ucma_context *ctx;
+
+ mutex_lock(&file->file_mutex);
+ while (!list_empty(&file->ctxs)) {
+ ctx = list_entry(file->ctxs.next, struct ucma_context,
+ file_list);
+ mutex_unlock(&file->file_mutex);
+
+ mutex_lock(&ctx_mutex);
+ idr_remove(&ctx_idr, ctx->id);
+ mutex_unlock(&ctx_mutex);
+
+ rdma_destroy_id(ctx->cm_id);
+ ucma_cleanup_events(ctx);
+ kfree(ctx);
+
+ mutex_lock(&file->file_mutex);
+ }
+ mutex_unlock(&file->file_mutex);
+ kfree(file);
+ return 0;
+}
+
+static struct file_operations ucma_fops = {
+ .owner = THIS_MODULE,
+ .open = ucma_open,
+ .release = ucma_close,
+ .write = ucma_write,
+ .poll = ucma_poll,
+};
+
+static struct miscdevice ucma_misc = {
+ .minor = MISC_DYNAMIC_MINOR,
+ .name = "rdma_cm",
+ .fops = &ucma_fops,
+};
+
+static int __init ucma_init(void)
+{
+ return misc_register(&ucma_misc);
+}
+
+static void __exit ucma_cleanup(void)
+{
+ misc_deregister(&ucma_misc);
+ idr_destroy(&ctx_idr);
+}
+
+module_init(ucma_init);
+module_exit(ucma_cleanup);
diff -uprN -X linux-2.6.git/Documentation/dontdiff
linux-2.6.git/include/rdma/rdma_user_cm.h
linux-2.6.ib/include/rdma/rdma_user_cm.h
--- linux-2.6.git/include/rdma/rdma_user_cm.h 1969-12-31 16:00:00.000000000 -0800
+++ linux-2.6.ib/include/rdma/rdma_user_cm.h 2006-01-16 16:54:55.000000000 -0800
@@ -0,0 +1,186 @@
+/*
+ * Copyright (c) 2005 Intel Corporation. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#ifndef RDMA_USER_CM_H
+#define RDMA_USER_CM_H
+
+#include <linux/types.h>
+#include <linux/in6.h>
+#include <rdma/ib_user_verbs.h>
+#include <rdma/ib_user_sa.h>
+
+#define RDMA_USER_CM_ABI_VERSION 1
+
+#define RDMA_MAX_PRIVATE_DATA 256
+
+enum {
+ RDMA_USER_CM_CMD_CREATE_ID,
+ RDMA_USER_CM_CMD_DESTROY_ID,
+ RDMA_USER_CM_CMD_BIND_ADDR,
+ RDMA_USER_CM_CMD_RESOLVE_ADDR,
+ RDMA_USER_CM_CMD_RESOLVE_ROUTE,
+ RDMA_USER_CM_CMD_QUERY_ROUTE,
+ RDMA_USER_CM_CMD_CONNECT,
+ RDMA_USER_CM_CMD_LISTEN,
+ RDMA_USER_CM_CMD_ACCEPT,
+ RDMA_USER_CM_CMD_REJECT,
+ RDMA_USER_CM_CMD_DISCONNECT,
+ RDMA_USER_CM_CMD_INIT_QP_ATTR,
+ RDMA_USER_CM_CMD_GET_EVENT
+};
+
+/*
+ * command ABI structures.
+ */
+struct rdma_ucm_cmd_hdr {
+ __u32 cmd;
+ __u16 in;
+ __u16 out;
+};
+
+struct rdma_ucm_create_id {
+ __u64 uid;
+ __u64 response;
+};
+
+struct rdma_ucm_create_id_resp {
+ __u32 id;
+};
+
+struct rdma_ucm_destroy_id {
+ __u64 response;
+ __u32 id;
+ __u32 reserved;
+};
+
+struct rdma_ucm_destroy_id_resp {
+ __u32 events_reported;
+};
+
+struct rdma_ucm_bind_addr {
+ __u64 response;
+ struct sockaddr_in6 addr;
+ __u32 id;
+};
+
+struct rdma_ucm_resolve_addr {
+ struct sockaddr_in6 src_addr;
+ struct sockaddr_in6 dst_addr;
+ __u32 id;
+ __u32 timeout_ms;
+};
+
+struct rdma_ucm_resolve_route {
+ __u32 id;
+ __u32 timeout_ms;
+};
+
+struct rdma_ucm_query_route {
+ __u64 response;
+ __u32 id;
+ __u32 reserved;
+};
+
+struct rdma_ucm_query_route_resp {
+ __u64 node_guid;
+ struct ib_user_path_rec ib_route[2];
+ struct sockaddr_in6 src_addr;
+ struct sockaddr_in6 dst_addr;
+ __u32 num_paths;
+ __u8 port_num;
+ __u8 reserved[3];
+};
+
+struct rdma_ucm_conn_param {
+ __u32 qp_num;
+ __u32 qp_type;
+ __u8 private_data[RDMA_MAX_PRIVATE_DATA];
+ __u8 private_data_len;
+ __u8 srq;
+ __u8 responder_resources;
+ __u8 initiator_depth;
+ __u8 flow_control;
+ __u8 retry_count;
+ __u8 rnr_retry_count;
+ __u8 valid;
+};
+
+struct rdma_ucm_connect {
+ struct rdma_ucm_conn_param conn_param;
+ __u32 id;
+ __u32 reserved;
+};
+
+struct rdma_ucm_listen {
+ __u32 id;
+ __u32 backlog;
+};
+
+struct rdma_ucm_accept {
+ __u64 uid;
+ struct rdma_ucm_conn_param conn_param;
+ __u32 id;
+ __u32 reserved;
+};
+
+struct rdma_ucm_reject {
+ __u32 id;
+ __u8 private_data_len;
+ __u8 reserved[3];
+ __u8 private_data[RDMA_MAX_PRIVATE_DATA];
+};
+
+struct rdma_ucm_disconnect {
+ __u32 id;
+};
+
+struct rdma_ucm_init_qp_attr {
+ __u64 response;
+ __u32 id;
+ __u32 qp_state;
+};
+
+struct rdma_ucm_get_event {
+ __u64 response;
+};
+
+struct rdma_ucm_event_resp {
+ __u64 uid;
+ __u32 id;
+ __u32 event;
+ __u32 status;
+ __u8 private_data_len;
+ __u8 reserved[3];
+ __u8 private_data[RDMA_MAX_PRIVATE_DATA];
+};
+
+#endif /* RDMA_USER_CM_H */
^ permalink raw reply [flat|nested] 97+ messages in thread* Re: [PATCH 6/6 v2] IB: userspace support for RDMA connection manager
2006-03-06 23:41 ` [PATCH 6/6 v2] " Sean Hefty
@ 2006-03-22 1:40 ` Roland Dreier
2006-03-22 8:30 ` Michael S. Tsirkin
0 siblings, 1 reply; 97+ messages in thread
From: Roland Dreier @ 2006-03-22 1:40 UTC (permalink / raw)
To: Sean Hefty; +Cc: netdev, linux-kernel, openib-general
I added this patch to the rdma_cm branch in my git tree. When I was
doing that, I noticed that it builds rdma_ucm.ko unconditionally. It
seems that we want this to depend on CONFIG_INFINIBAND_USER_ACCESS,
since that controls ib_uverbs.ko and ib_ucm.ko.
To do this I rejiggered the Kconfig and Makefile changes I made
before. I made CONFIG_INFINIBAND_ADDR_TRANS into a bool (instead of a
tristate), so that it's 'y' if INFINIBAND and INET are on, and made
the top of the Makefile look like:
infiniband-$(CONFIG_INFINIBAND_ADDR_TRANS) := ib_addr.o rdma_cm.o
user_access-$(CONFIG_INFINIBAND_ADDR_TRANS) := rdma_ucm.o
obj-$(CONFIG_INFINIBAND) += ib_core.o ib_mad.o ib_sa.o \
ib_cm.o $(infiniband-y)
obj-$(CONFIG_INFINIBAND_USER_MAD) += ib_umad.o
obj-$(CONFIG_INFINIBAND_USER_ACCESS) += ib_uverbs.o ib_ucm.o $(user_access-y)
I'm pretty sure this does exactly what we want.
- R.
^ permalink raw reply [flat|nested] 97+ messages in thread
* Re: [PATCH 6/6 v2] IB: userspace support for RDMA connection manager
2006-03-22 1:40 ` Roland Dreier
@ 2006-03-22 8:30 ` Michael S. Tsirkin
2006-03-22 15:18 ` Roland Dreier
0 siblings, 1 reply; 97+ messages in thread
From: Michael S. Tsirkin @ 2006-03-22 8:30 UTC (permalink / raw)
To: Roland Dreier; +Cc: Sean Hefty, netdev, linux-kernel, openib-general
Quoting r. Roland Dreier <rdreier@cisco.com>:
> Subject: Re: [PATCH 6/6 v2] IB: userspace support for RDMA connection manager
>
> I added this patch to the rdma_cm branch in my git tree.
BTW, is there some way to see your git tree e.g. with gitweb?
--
Michael S. Tsirkin
Staff Engineer, Mellanox Technologies
^ permalink raw reply [flat|nested] 97+ messages in thread