* Re: [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset [not found] <200610212100.k9LL0GtC018787@hera.kernel.org> @ 2006-10-22 3:51 ` Muli Ben-Yehuda 2006-10-22 5:13 ` Yinghai Lu ` (4 more replies) 0 siblings, 5 replies; 24+ messages in thread From: Muli Ben-Yehuda @ 2006-10-22 3:51 UTC (permalink / raw) To: Yinghai Lu, Eric W. Biederman; +Cc: Linux Kernel Mailing List, Andi Kleen On Sat, Oct 21, 2006 at 09:00:17PM +0000, Linux Kernel Mailing List wrote: > commit 45edfd1db02f818b3dc7e4743ee8585af6b78f78 > tree cc7a524069ee23c49237c417299e5aa2f93205e0 > parent 926fafebc48a3218fac675f12974f9a46473bd40 > author Yinghai Lu <yinghai.lu@amd.com> 1161448621 +0200 > committer Andi Kleen <andi@basil.nowhere.org> 1161448621 +0200 > > [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset > > typo with cpu instead of new_cpu This patch breaks my x366 machine: aic94xx: device 0000:01:02.0: SAS addr 5005076a0112df00, PCBA SN , 8 phys, 8 enabled phys, flash present, BIOS build 1323 aic94xx: couldn't get irq 25 for 0000:01:02.0 ACPI: PCI interrupt for device 0000:01:02.0 disabled aic94xx: probe of 0000:01:02.0 failed with error -38 Reverting it allows it to boot again. Since the patch is "obviously correct", it must be uncovering some other problem with the genirq code. Cheers, Muli kernel (hd0,1)/boot/calgary/bzImage root=/dev/sda2 console=tty0 console=ttyS1,1 9200 [Linux-bzImage, setup=0x1c00, size=0x2bd61b] initrd (hd0,1)/boot/calgary/aic94xxfw.initramfs.gz [Linux-initrd @ 0x37071000, 0xf7ea24 bytes] savedefault Linux version 2.6.19-rc2mx (muli@cluwyn) (gcc version 4.0.3) #11 SMP Sun Oct 22 04:25:51 IST 2006 Command line: root=/dev/sda2 console=tty0 console=ttyS1,19200 BIOS-provided physical RAM map: BIOS-e820: 0000000000000000 - 0000000000099000 (usable) BIOS-e820: 0000000000099000 - 00000000000a0000 (reserved) BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved) BIOS-e820: 0000000000100000 - 00000000e7f9c640 (usable) BIOS-e820: 00000000e7f9c640 - 00000000e7fa6a40 (ACPI data) BIOS-e820: 00000000e7fa6a40 - 00000000e8000000 (reserved) BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved) BIOS-e820: 0000000100000000 - 0000000198000000 (usable) end_pfn_map = 1671168 DMI 2.3 present. Zone PFN ranges: DMA 0 -> 4096 DMA32 4096 -> 1048576 Normal 1048576 -> 1671168 early_node_map[3] active PFN ranges 0: 0 -> 153 0: 20x06] enabled) Processor #6 ACPI: LAPIC (acpi_id[0x03] lapic_id[0x07] enabled) Processor #7 ACPI: LAPIC_NMI (acpi_id[0x00] dfl dfl lint[0x1]) ACPI: LAPIC_NMI (acpi_id[0x01] dfl dfl lint[0x1]) ACPI: LAPIC_NMI (acpi_id[0x02] dfl dfl lint[0x1]) ACPI: LAPIC_NMI (acpi_id[0x03] dfl dfl lint[0x1]) ACPI: IOAPIC (id[0x0f] address[0xfec0_OVR (bus 0 bus_irq 8 global_irq 8 low edge) ACPI: INT_SRC_OVR (bus 0 bus_irq 14 global_irq 14 low edge) Setting APIC routing to flat Using ACPI (MADT) for SMP configuration information Nosave address range: 0000000000099000 - 00000000000a0000 Nosave address range: 00000000000a0000 - 00000000000e0000 Nosave address range: 00000000000e0000 - 0000000000100000 Nosave address range: 00000000e7f9c000 - 00000000e7f9d000 Nosave address range: 00000000e7f9d000 - 00000000e7fa6000 Nosave address range: 00000000e7fa6000 - 00000000e7fa7000 Nosave address range: 00000000e7fa7000 - 00000000e8000000 Nosave address range: 00000000e8000000 - 00000000fec00000 Nosave address range: 00000000fec00000 - 0000000100000000 Allocating PCI resources starting at ea000000 (gap: e8000000:16c00000) PERCPU: Allocating 34048 bytes of per cpu data Built 1 zonelists. Total pages: 1534074 Kernel command line: root=/dev/sda2 console=tty0 console=ttyS1,19200 Initializing CPU#0 PID hash table entries: 4096 (order: 12, 32768 bytes) Console: colour VGA+ 80x25 Lock dependency validator: Copyright (c) 2006 Re 4096 memory used by lock dependency info: 1328 kB per task-struct memory footprint: 1680 bytes Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes) Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes) Checking aperture... PCI-DMA: Calgary IOMMU detected. PCI-DMA: Calgary TCE table spec is 7, CONFIG_IOMMU_DEBUG is enabled. Memory: 6082452k/6684672k available (3735k kernel code, 207748k reserved, 2686k data, 276k init) Calibrating delay using timer specific routine.. 6346.24 BogoMIPS (lpj=12692491)Mount-cache hash table entries: 256 CPU: Trace cache: 12K uops, L1 D cache: 16K CPU: L2 cache: 1024K using mwait in idle threads. CPU: Physical Processor ID: 0 CPU: Processor Core ID: 0 CPU0: Thermal monitoring enabled (TM1) Freeing SMP alternatives: 32k freed ACPI: Core revision 20060707 ..MP-BIOS bug: 8254 timer not connected to IO-APIC Using local APIC timer interrupts. result 10425644 Detected 10.425 MHz APIC timer. lockdep: not fixing up alternatives. Booting processor 1/4 APIC 0x1 Initializing CPU#1 Calibrating delay using timer specific routine.. 6339.07 BogoMIPS (lpj=12678150)CPU: Trace cache: 12K uops, L1 D cache: Initializing CPU#2 Calibrating delay using timer specific routine.. 6339.11 BogoMIPS (lpj=12678228)CPU: Trace cache: 12K uops, L1 D cache: 16K CPU: L2 cache: 1024K CPU: Physical Processor ID: 3 CPU: Processor Core ID: 0 CPU2: Thermal monitoring enabled (TM1) Intel(R) Pentium(R) 4 CPU 3.16GHz stepping 09 lockdep: not fixing up alternatives. Booting processor 3/4 APIC 0x7 Initializing CPU#3 Calibrating delay using timer specific routine.. 6339.20 BogoMIPS (lpj=12678401)CPU: Trace cache: 12K uops, L1 D cache: 16K CPU: L2 cache: 1024K CPU: Physical Processor ID: 3 CPU: Processor Core ID: 0 CPU3: Thermal monitoring enabled (TM1) Intel(R) Pentium(R) 4 CPU 3.16GHz stepping 09 Brought up 4 CPUs testing NMI watchdog ... OK. time.c: Using 3.579545 MHz WALL PM GTOD PIT/TSC timer. time.c: Detected 3169.440 MHz processor. migration_cost=12,795 checking if image is initramfs... it is Freeing initrd memory: 15866k freed NET: Registered protocol family 16 ACPI: bus type pci registered PCI: Using configuration type 1 ACPI: Interpreter enabled ACPI: Using IOAPIC for interrupt routing ACPI: PCI Root Bridge [VP00] (0000:00) PCI: Ignoring BAR0-3 of IDE controller 0000:00:0f.1 ACPI: PCI Root Bridge [VP01] (0000:01) ACPI: PCI Root Bridge [VP02] (0000:02) ACPI: PCI Root Bridge [VP03] (0000:04) ACPI: PCI Root Bridge [VP04] (0000:06) ACPI: PCI Root Bridge [VP05] (0000:08) ACPI: PCI Root Bridge [VP06] (0000:0a) ACPI: PCI Root Bridge [VP07] (0000:0c) SCSI subsystem initialized usbcore: registered new interface driver usbfs usbcore: registered new interface driver hub usbcore: registered new device driver usb PCI: Using ACPI for IRQ routing PCI: If a device doesn't work, try "pci=routeirq". If it helps, post a report PCI-DMA: Using Calgary IOMMU Calgary: enabling translation on PHB 0x0 Calgary: errant DMAs will now be prevented on this bus. Calgary: enabling translation on PHB 0x1 Calgary: errant DMAs will now be prevented on this bus. Calgary: enabling translation on PHB 0x2 Calgary: errant DMAs will now be prevented on this bus. PCI-GART: No AMD northbridge found. NET: Registered protocol family 2 IP route cache hash table entries: 262144 (order: 9, 2097152 bytes) TCP established hash table entries: 65536 (order: 9, 3670016 bytes) TCP bind hash table entries: 32768 (order: 8, 1835008 bytes) TCP: Hash tables configured (established 65536 bind 32768) TCP reno registered Total HugeTLB memory allocated, 0 Installing knfsd (copyright (C) 1996 okir@monad.swb.de). io scheduler noop registered io scheduler anticipatory registered (default) io scheduler deadline registered io scheduler cfq registered ACPI: PCI Interrupt 0000:00:01.0[A] -> GSI 16 (level, low) -> IRQ 16 radeonfb: Found Intel x86 BIOS ROM Image radeonfb: Retrieved PLL infos from BIOS radeonfb: Reference=27.00 MHz (RefDiv=60) Memory=143.00 Mhz, System=143.00 MHz radeonfb: PLL min 12000 max 35000 i2c_adapter i2c-1: unable to read EDID block. i2c_adapter i2c-1: unable to read EDID block. i2c_adapter i2c-1: unable to read EDID block. i2c_adapter i2c-2: unable to read EDID block. i2c_adapter i2c-2: unable to read EDID block. i2c_adapter i2c-2: unable to read EDID block. radeonfb: Monitor 1 type DFP found radeonfb: EDID probed radeonfb: Monitor 2 type CRT found Console: switching to colour frame buffer device 128x48 radeonfb (0000:00:01.0): ATI Radeon QY tridentfb: Trident framebuffer 0.7.8-NEWAPI initializing hgafb: HGA card not detected. hgafb: probe of hgafb.0 failed with error -22 vga16fb: mapped to 0xffff8100000a0000 fb1: VGA16 VGA frame buffer device fb2: Virtual frame buffer device, using 1024K of video memory ACPI: Power Button (FF) [PWRF] ibm_acpi: ec object not found Linux agpgart interface v0.101 (c) Dave Jones ipmi message handler version 39.0 ipmi device interface Hangcheck: starting hangcheck timer 0.9.0 (tick is 180 seconds, margin is 60 seconds). Hangcheck: Using monotonic_clock(). Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing disabled serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize loop: loaded (max 8 devices) ibmasm: IBM ASM Service Processor Driver version 1.0 loaded ACPI: PCI Interrupt 0000:02:01.0[A] -> GSI 18 (level, low) -> IRQ 18 3c59x: Donald Becker and others. www.scyld.com/network/vortex.html 0000:02:01.0: 3Com PCI 3c905C Tornado at ffffc20000042000. tg3.c:v3.67 (October 18, 2006) ACPI: PCI Interrupt 0000:01:01.0[A] -> GSI 24 (level, low) -> IRQ 24 eth1: Tigon3 [partno(BCM95704A6) rev 2100 PHY(5704)] (PCIX:66MHz:64-bit) 10/100/1000BaseT Ethernet 00:0d:60:98:74:22 eth1: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[1] Split[0] WireSpeed[1] TSOcap[0] eth1: dma_rwctrl[769f0000] dma_mask[64-bit] ACPI: PCI Interrupt 0000:01:01.1[B] -> GSI 28 (level, low) -> IRQ 28 eth2: Tigon3 [partno(BCM95704A6) rev 2100 PHY(5704)] (PCIX:66MHz:64-bit) 10/100/1000BaseT Ethernet 00:0d:60:98:74:23 eth2: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] Split[0] WireSpeed[1] TSOcap[1] eth2: dma_rwctrl[769f0000] dma_mask[64-bit] Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2 ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx SvrWks CSB6: IDE controller at PCI slot 0000:00:0f.1 SvrWks CSB6: chipset revision 160 SvrWks CSB6: not 100% native mode: will probe irqs later ide0: BM-DMA at 0x0700-0x0707, BIOS settings: hda:DMA, hdb:DMA SvrWks CSB6: simplex device: DMA disabled ide1: SvrWks CSB6 Bus-Master DMA disabled (BIOS) hda: HL-DT-STDVD-ROM GDR8082N, ATAPI CD/DVD-ROM<7>Losing some ticks... checking if CPU frequency changed. drive ide0 at 0x1f0-0x1f7,0x3f6 on irq 14 hda: ATAPI 24X DVD-ROM drive, 256kB Cache Uniform CD-ROM driver Revision: 3.20 usbmon: debugfs is not available ACPI: PCI Interrupt 0000:00:03.0[A] -> GSI 20 (level, low) -> IRQ 20 ohci_hcd 0000:00:03.0: OHCI Host Controller ohci_hcd 0000:00:03.0: new USB bus registered, assigned bus number 1 ohci_hcd 0000:00:03.0: irq 20, io mem 0xf2c10000 usb usb1: Product: OHCI Host Controller usb usb1: Manufacturer: Linux 2.6.19-rc2mx ohci_hcd usb usb1: SerialNumber: 0000:00:03.0 usb usb1: configuration #1 chosen from 1 choice hub 1-0:1.0: USB hub found hub 1-0:1.0: 2 ports detected ACPI: PCI Interrupt 0000:00:03.1[B] -> GSI 20 (level, low) -> IRQ 20 ohci_hcd 0000:00:03.1: OHCI Host Controller ohci_hcd 0000:00:03.1: new USB bus registered, assigned bus number 2 ohci_hcd 0000:00:03.1: irq 20, io mem 0xf2c11000 usb usb2: Product: OHCI Host Controller usb usb2: Manufacturer: Linux 2.6.19-rc2mx ohci_hcd usb usb2: SerialNumber: 0000:00:03.1 usb usb2: configuration #1 chosen from 1 choice hub 2-0:1.0: USB hub found hub 2-0:1.0: 2 ports detected USB Universal Host Controller Interface driver v3.0 serio: i8042 KBD port at 0x60,0x64 irq 1 serio: i8042 AUX port at 0x60,0x64 irq 12 mice: PS/2 mouse device common for all mice input: PC Speaker as /class/input/input0 input: AT Translated Set 2 keyboard as /class/input/input1 i2c /dev entries driver i2c-parport: adapter type unspecified md: linear personality registered for level -1 IBM TrackPoint firmware: 0x0b, buttons: 3/3 md: raid0 personality registered for level 0 input: TPPS/2 IBM TrackPoint as /class/input/input2 md: raid1 personality registered for level 1 md: multipath personality registered for level -4 device-mapper: ioctl: 4.10.0-ioctl (2006-09-14) initialised: dm-devel@redhat.comdevice-mapper: multipath: version 1.0.5 loaded device-mapper: multipath round-robin: version 1.0.0 loaded device-mapper: multipath emc: version 0.0.3 loaded EDAC MC: Ver: 2.0.1 Oct 22 2006 pktgen v2.68: Packet Generator for packet performance testing. u32 classifier OLD policer on IPv4 over IPv4 tunneling driver GRE over IPv4 tunneling driver TCP cubic registered Initializing XFRM netlink socket NET: Registered protocol family 1 NET: Registered protocol family 17 NET: Registered protocol family 15 802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com> All bugs added by David S. Miller <davem@redhat.com> SCTP: Hash tables configured (established 37449 bind 37449) Freeing unused kernel memory: 276k freed running (1:0) /init hello worlaic94xx: Adaptec aic94xx SAS/SATA driver version 1.0.2 loaded d from the initrACPI: PCI Interrupt 0000:01:02.0[A] -> d1! hello wGSI 25 (level, low) -> IRQ 25 orld from the inaic94xx: found Adaptec AIC-9410W SAS/SATA Host Adapter, device 0000:01:02.0 itrd 2! helloscsi0 : aic94xx world from the aic94xx: BIOS present (1,1), 1323 initrd 3! Loading kernel/drivers/scsi/aic94xxaic94xx: ue num:2, ue size:88 /aic94xx.ko aic94xx: manuf sect SAS_ADDR 5005076a0112df00 aic94xx: manuf sect PCBA SN aic94xx: ms: num_phy_desc: 8 aic94xx: ms: phy0: ENEBLEABLE aic94xx: ms: phy1: ENEBLEABLE aic94xx: ms: phy2: ENEBLEABLE aic94xx: ms: phy3: ENEBLEABLE aic94xx: ms: phy4: ENEBLEABLE aic94xx: ms: phy5: ENEBLEABLE aic94xx: ms: phy6: ENEBLEABLE aic94xx: ms: phy7: ENEBLEABLE aic94xx: ms: max_phys:0x8, num_phys:0x8 aic94xx: ms: enabled_phys:0xff aic94xx: ctrla: phy0: sas_ flags:0x0 aic94xx: ctrla: phy3: sas_addr: 5005076a0112df00, sas rate:0x9-0x8, sata rate:0x0-0x0, flags:0x0 aic94xx: ctrla: phy4: sas_addr: 5005076a0112df00, sas rate:0x9-0x8, sata rate:0x0-0x0, flags:0x0 aic94xx: ctrla: phy5: sas_addr: 5005076a0112df00, sas rate:0x9-0x8, sata rate:0x0-0x0, flags:0x0 aic94xx: ctrla: phy6: sas_addr: 5005076a0112df00, sas rate:0x9-0x8, sata rate:0x0-0x0, flags:0x0 aic94xx: ctrla: phy7: sas_addr: 5005076a0112df00, sas rate:0x9-0x8, sata rate:0x0-0x0, flags:0x0 aic94xx: max_scbs:512, max_ddbs:128 aic94xx: setting phy0 addr to 5005076a0112df00 aic94xx: setting phy1 addr to 5005076a0112df00 aic94xx: setting phy2 addr to 5005076a0112df00 aic94xx: setting phy3 addr to 5005076a0112df00 aic94xx: setting phy4 addr to 5005076a0112df00 aic94xx: setting phy5 addr to 5005076a0112df00 aic94xx: setting phy6 addr to 5005076a0112df00 aic94xx: setting phy7 addr to 5005076a0112df00 aic94xx: num_edbs:21 aic94xx: num_escbs:3 aic94xx: using sequencer Razor_10a1 aic94xx: downloading CSEQ... aic94xx: dma-ing 8192 bytes aic94xx: verified 8192 bytes, passed aic94xx: downloading LSEQs... aic94xx: dma-ing 14336 bytes aic94xx: LSEQ0 verified 14336 bytes, passed aic94xx: LSEQ1 verified 14336 bytes, passed aic94xx: LSEQ2 verified 14336 bytes, passed aic94xx: LSEQ3 verified 14336 bytes, passed aic94xx: LSEQ4 verified 14336 bytes, passed aic94xx: LSEQ5 verified 14336 bytes, passed aic94xx: LSEQ6 verified 14336 bytes, passed aic94xx: LSEQ7 verified 14336 bytes, passed aic94xx: max_scbs:446 aic94xx: first_scb_site_no:0x20 aic94xx: last_scb_site_no:0x1fe aic94xx: First SCB dma_handle: 0xd000 aic94xx: device 0000:01:02.0: SAS addr 5005076a0112df00, PCBA SN , 8 phys, 8 enabled phys, flash present, BIOS build 1323 aic94xx: couldn't get irq 25 for 0000:01:02.0 ACPI: PCI interrupt for device 0000:01:02.0 disabled aic94xx: probe of 0000:01:02.0 failed with error -38 hello world from the initrd 4! creating device nodes ... waiting for block device node: /dev/sda2 ........................ mount -o ro /dev/sda2 mount: special device /dev/sda2 does not exist unable to mount root filesystem on /dev/sda2 Kernel panic - not syncing: Attempted to kill init! <0>Rebooting in 42 seconds.. ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset 2006-10-22 3:51 ` [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset Muli Ben-Yehuda @ 2006-10-22 5:13 ` Yinghai Lu 2006-10-22 6:55 ` yhlu ` (3 subsequent siblings) 4 siblings, 0 replies; 24+ messages in thread From: Yinghai Lu @ 2006-10-22 5:13 UTC (permalink / raw) To: Muli Ben-Yehuda; +Cc: Eric W. Biederman, Linux Kernel Mailing List, Andi Kleen On 10/21/06, Muli Ben-Yehuda <muli@il.ibm.com> wrote: > On Sat, Oct 21, 2006 at 09:00:17PM +0000, Linux Kernel Mailing List wrote: > Booting processor 1/4 APIC 0x1 > Initializing CPU#1 > Calibrating delay using timer specific routine.. 6339.07 BogoMIPS (lpj=12678150)CPU: Trace cache: 12K uops, L1 D cache: Initializing CPU#2 > Calibrating delay using timer specific routine.. 6339.11 BogoMIPS (lpj=12678228)CPU: Trace cache: 12K uops, L1 D cache: 16K > CPU: L2 cache: 1024K > CPU: Physical Processor ID: 3 > CPU: Processor Core ID: 0 > CPU2: Thermal monitoring enabled (TM1) > Intel(R) Pentium(R) 4 CPU 3.16GHz stepping 09 > lockdep: not fixing up alternatives. > Booting processor 3/4 APIC 0x7 > Initializing CPU#3 > Calibrating delay using timer specific routine.. 6339.20 BogoMIPS (lpj=12678401)CPU: Trace cache: 12K uops, L1 D cache: 16K > CPU: L2 cache: 1024K > CPU: Physical Processor ID: 3 > CPU: Processor Core ID: 0 > CPU3: Thermal monitoring enabled (TM1) > Intel(R) Pentium(R) 4 CPU 3.16GHz stepping 09 > Brought up 4 CPUs It seems it tried to initialize CPU2, before CPU1 is all set. current code still initialize CPU one by one, because there is some share data structure. YH ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset 2006-10-22 3:51 ` [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset Muli Ben-Yehuda 2006-10-22 5:13 ` Yinghai Lu @ 2006-10-22 6:55 ` yhlu 2006-10-22 8:50 ` Muli Ben-Yehuda 2006-10-22 8:50 ` Eric W. Biederman 2006-10-22 8:28 ` Yinghai Lu ` (2 subsequent siblings) 4 siblings, 2 replies; 24+ messages in thread From: yhlu @ 2006-10-22 6:55 UTC (permalink / raw) To: Muli Ben-Yehuda; +Cc: Eric W. Biederman, Linux Kernel Mailing List, Andi Kleen can you try to add command line pci=routeirq? also if put the driver in the kernel? YH ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset 2006-10-22 6:55 ` yhlu @ 2006-10-22 8:50 ` Muli Ben-Yehuda 2006-10-22 8:50 ` Eric W. Biederman 1 sibling, 0 replies; 24+ messages in thread From: Muli Ben-Yehuda @ 2006-10-22 8:50 UTC (permalink / raw) To: yhlu; +Cc: Eric W. Biederman, Linux Kernel Mailing List, Andi Kleen On Sat, Oct 21, 2006 at 11:55:28PM -0700, yhlu wrote: > can you try to add command line pci=routeirq? Works. > also if put the driver in the kernel? Can't do that - it needs to be loaded late enough for the userspace firmware request mechanism to work. Cheers, Muli ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset 2006-10-22 6:55 ` yhlu 2006-10-22 8:50 ` Muli Ben-Yehuda @ 2006-10-22 8:50 ` Eric W. Biederman 1 sibling, 0 replies; 24+ messages in thread From: Eric W. Biederman @ 2006-10-22 8:50 UTC (permalink / raw) To: yhlu; +Cc: Muli Ben-Yehuda, Linux Kernel Mailing List, Andi Kleen yhlu <yhlu.kernel@gmail.com> writes: > can you try to add command line pci=routeirq? > > also if put the driver in the kernel? YH we have an irq number so it doesn't appear to be a routing problem. Much more likely that the vector allocator decided to fail. Probably just some stupid thing with vector_irq. Eric ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset 2006-10-22 3:51 ` [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset Muli Ben-Yehuda 2006-10-22 5:13 ` Yinghai Lu 2006-10-22 6:55 ` yhlu @ 2006-10-22 8:28 ` Yinghai Lu 2006-10-22 8:50 ` Muli Ben-Yehuda 2006-10-22 8:35 ` Eric W. Biederman 2006-10-22 13:29 ` [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset Andi Kleen 4 siblings, 1 reply; 24+ messages in thread From: Yinghai Lu @ 2006-10-22 8:28 UTC (permalink / raw) To: Muli Ben-Yehuda; +Cc: Eric W. Biederman, Linux Kernel Mailing List, Andi Kleen [-- Attachment #1: Type: text/plain, Size: 642 bytes --] On 10/21/06, Muli Ben-Yehuda <muli@il.ibm.com> wrote: > > > > typo with cpu instead of new_cpu > > This patch breaks my x366 machine: > > aic94xx: device 0000:01:02.0: SAS addr 5005076a0112df00, PCBA SN , 8 phys, 8 enabled phys, flash present, BIOS build 1323 > aic94xx: couldn't get irq 25 for 0000:01:02.0 > ACPI: PCI interrupt for device 0000:01:02.0 disabled > aic94xx: probe of 0000:01:02.0 failed with error -38 > > Reverting it allows it to boot again. Since the patch is "obviously > correct", it must be uncovering some other problem with the genirq > code. > can you try attached patch? it use cpu_online_map instead of 0xff. YH [-- Attachment #2: vector_allocation_domain.diff --] [-- Type: text/x-patch, Size: 485 bytes --] diff --git a/arch/x86_64/kernel/genapic_flat.c b/arch/x86_64/kernel/genapic_flat.c index 7c01db8..490df69 100644 --- a/arch/x86_64/kernel/genapic_flat.c +++ b/arch/x86_64/kernel/genapic_flat.c @@ -32,8 +32,7 @@ static cpumask_t flat_vector_allocation_ * deliver interrupts to the wrong hyperthread when only one * hyperthread was specified in the interrupt desitination. */ - cpumask_t domain = { { [0] = APIC_ALL_CPUS, } }; - return domain; + return cpu_online_map; } /* ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset 2006-10-22 8:28 ` Yinghai Lu @ 2006-10-22 8:50 ` Muli Ben-Yehuda 2006-10-22 8:58 ` Eric W. Biederman 2006-10-22 16:02 ` yhlu 0 siblings, 2 replies; 24+ messages in thread From: Muli Ben-Yehuda @ 2006-10-22 8:50 UTC (permalink / raw) To: Yinghai Lu; +Cc: Eric W. Biederman, Linux Kernel Mailing List, Andi Kleen On Sun, Oct 22, 2006 at 01:28:03AM -0700, Yinghai Lu wrote: > On 10/21/06, Muli Ben-Yehuda <muli@il.ibm.com> wrote: > >> > >> typo with cpu instead of new_cpu > > > >This patch breaks my x366 machine: > > > >aic94xx: device 0000:01:02.0: SAS addr 5005076a0112df00, PCBA SN , 8 phys, > >8 enabled phys, flash present, BIOS build 1323 > >aic94xx: couldn't get irq 25 for 0000:01:02.0 > >ACPI: PCI interrupt for device 0000:01:02.0 disabled > >aic94xx: probe of 0000:01:02.0 failed with error -38 > > > >Reverting it allows it to boot again. Since the patch is "obviously > >correct", it must be uncovering some other problem with the genirq > >code. > > > > can you try attached patch? it use cpu_online_map instead of 0xff. Works! Thanks, Muli ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset 2006-10-22 8:50 ` Muli Ben-Yehuda @ 2006-10-22 8:58 ` Eric W. Biederman 2006-10-22 16:02 ` yhlu 1 sibling, 0 replies; 24+ messages in thread From: Eric W. Biederman @ 2006-10-22 8:58 UTC (permalink / raw) To: Muli Ben-Yehuda; +Cc: Yinghai Lu, Linux Kernel Mailing List, Andi Kleen Muli Ben-Yehuda <muli@il.ibm.com> writes: > Works! Cool so we have a work around. So it looks like we need to be a little more careful with vector_irq, and it's initialization when cpus start up, or are not running. Eric ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset 2006-10-22 8:50 ` Muli Ben-Yehuda 2006-10-22 8:58 ` Eric W. Biederman @ 2006-10-22 16:02 ` yhlu 2006-10-22 16:19 ` yhlu 2006-10-22 16:20 ` Muli Ben-Yehuda 1 sibling, 2 replies; 24+ messages in thread From: yhlu @ 2006-10-22 16:02 UTC (permalink / raw) To: Muli Ben-Yehuda; +Cc: Eric W. Biederman, Linux Kernel Mailing List, Andi Kleen You must have NR_CPUS=4. Also you genapic is running on flat. (logical) Can you try to set NR_CPUS=8 and without this patch? YH On 10/22/06, Muli Ben-Yehuda <muli@il.ibm.com> wrote: > On Sun, Oct 22, 2006 at 01:28:03AM -0700, Yinghai Lu wrote: > > On 10/21/06, Muli Ben-Yehuda <muli@il.ibm.com> wrote: > > >> > > >> typo with cpu instead of new_cpu > > > > > >This patch breaks my x366 machine: > > > > > >aic94xx: device 0000:01:02.0: SAS addr 5005076a0112df00, PCBA SN , 8 phys, > > >8 enabled phys, flash present, BIOS build 1323 > > >aic94xx: couldn't get irq 25 for 0000:01:02.0 > > >ACPI: PCI interrupt for device 0000:01:02.0 disabled > > >aic94xx: probe of 0000:01:02.0 failed with error -38 > > > > > >Reverting it allows it to boot again. Since the patch is "obviously > > >correct", it must be uncovering some other problem with the genirq > > >code. > > > > > > > can you try attached patch? it use cpu_online_map instead of 0xff. > > Works! > > Thanks, > Muli > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset 2006-10-22 16:02 ` yhlu @ 2006-10-22 16:19 ` yhlu 2006-10-22 21:33 ` Andi Kleen 2006-10-22 16:20 ` Muli Ben-Yehuda 1 sibling, 1 reply; 24+ messages in thread From: yhlu @ 2006-10-22 16:19 UTC (permalink / raw) To: Muli Ben-Yehuda; +Cc: Eric W. Biederman, Linux Kernel Mailing List, Andi Kleen andi, the per_cpu only can be used with online cpus? YH On 10/22/06, yhlu <yhlu.kernel@gmail.com> wrote: > You must have NR_CPUS=4. > > Also you genapic is running on flat. (logical) > > Can you try to set NR_CPUS=8 and without this patch? > > YH > > On 10/22/06, Muli Ben-Yehuda <muli@il.ibm.com> wrote: > > On Sun, Oct 22, 2006 at 01:28:03AM -0700, Yinghai Lu wrote: > > > On 10/21/06, Muli Ben-Yehuda <muli@il.ibm.com> wrote: > > > >> > > > >> typo with cpu instead of new_cpu > > > > > > > >This patch breaks my x366 machine: > > > > > > > >aic94xx: device 0000:01:02.0: SAS addr 5005076a0112df00, PCBA SN , 8 phys, > > > >8 enabled phys, flash present, BIOS build 1323 > > > >aic94xx: couldn't get irq 25 for 0000:01:02.0 > > > >ACPI: PCI interrupt for device 0000:01:02.0 disabled > > > >aic94xx: probe of 0000:01:02.0 failed with error -38 > > > > > > > >Reverting it allows it to boot again. Since the patch is "obviously > > > >correct", it must be uncovering some other problem with the genirq > > > >code. > > > > > > > > > > can you try attached patch? it use cpu_online_map instead of 0xff. > > > > Works! > > > > Thanks, > > Muli > > - > > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > > Please read the FAQ at http://www.tux.org/lkml/ > > > ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset 2006-10-22 16:19 ` yhlu @ 2006-10-22 21:33 ` Andi Kleen 0 siblings, 0 replies; 24+ messages in thread From: Andi Kleen @ 2006-10-22 21:33 UTC (permalink / raw) To: yhlu; +Cc: Muli Ben-Yehuda, Eric W. Biederman, Linux Kernel Mailing List On Sunday 22 October 2006 18:19, yhlu wrote: > andi, > > the per_cpu only can be used with online cpus? It can be used for all possible cpus. This means some subsystems initialize their state only for online CPUs, but the compile time initialization is available for all possible ones. -Andi ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset 2006-10-22 16:02 ` yhlu 2006-10-22 16:19 ` yhlu @ 2006-10-22 16:20 ` Muli Ben-Yehuda 2006-10-22 16:28 ` yhlu 1 sibling, 1 reply; 24+ messages in thread From: Muli Ben-Yehuda @ 2006-10-22 16:20 UTC (permalink / raw) To: yhlu; +Cc: Eric W. Biederman, Linux Kernel Mailing List, Andi Kleen On Sun, Oct 22, 2006 at 09:02:19AM -0700, yhlu wrote: > You must have NR_CPUS=4. Nope, I have NR_CPUS=8 > Also you genapic is running on flat. (logical) Right > Can you try to set NR_CPUS=8 and without this patch? All of the tests I ran were with NR_CPUS=8. Shall I try NR_CPUS=4? Cheers, Muli ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset 2006-10-22 16:20 ` Muli Ben-Yehuda @ 2006-10-22 16:28 ` yhlu 0 siblings, 0 replies; 24+ messages in thread From: yhlu @ 2006-10-22 16:28 UTC (permalink / raw) To: Muli Ben-Yehuda; +Cc: Eric W. Biederman, Linux Kernel Mailing List, Andi Kleen On 10/22/06, Muli Ben-Yehuda <muli@il.ibm.com> wrote: > All of the tests I ran were with NR_CPUS=8. Shall I try NR_CPUS=4? So per_cpu only can be used onlined cpus? I wonder if you try NR_CPUS without the patch, you will get more strange result. YH ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset 2006-10-22 3:51 ` [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset Muli Ben-Yehuda ` (2 preceding siblings ...) 2006-10-22 8:28 ` Yinghai Lu @ 2006-10-22 8:35 ` Eric W. Biederman 2006-10-22 8:52 ` Muli Ben-Yehuda 2006-10-22 13:29 ` [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset Andi Kleen 4 siblings, 1 reply; 24+ messages in thread From: Eric W. Biederman @ 2006-10-22 8:35 UTC (permalink / raw) To: Muli Ben-Yehuda; +Cc: Yinghai Lu, Linux Kernel Mailing List, Andi Kleen Muli Ben-Yehuda <muli@il.ibm.com> writes: > On Sat, Oct 21, 2006 at 09:00:17PM +0000, Linux Kernel Mailing List wrote: > >> commit 45edfd1db02f818b3dc7e4743ee8585af6b78f78 >> tree cc7a524069ee23c49237c417299e5aa2f93205e0 >> parent 926fafebc48a3218fac675f12974f9a46473bd40 >> author Yinghai Lu <yinghai.lu@amd.com> 1161448621 +0200 >> committer Andi Kleen <andi@basil.nowhere.org> 1161448621 +0200 >> >> [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and > offset >> >> typo with cpu instead of new_cpu > > This patch breaks my x366 machine: > > aic94xx: device 0000:01:02.0: SAS addr 5005076a0112df00, PCBA SN , 8 phys, 8 > enabled phys, flash present, BIOS build 1323 > aic94xx: couldn't get irq 25 for 0000:01:02.0 > ACPI: PCI interrupt for device 0000:01:02.0 disabled > aic94xx: probe of 0000:01:02.0 failed with error -38 > > Reverting it allows it to boot again. Since the patch is "obviously > correct", it must be uncovering some other problem with the genirq > code. > Ugh. This is no fun at all. I am assuming this is with cpu hotplug disabled in flat mode. I need to step back and read the code but it appears that request_irq(25) is failing. Which implies that the vector assignment is failing for some strange reason. So what we need to do is figure out what those data structures look like on your machine and see if we can figure out a plausible explanation for why there would be no free vectors. Taking a wild stab in the dark my hunch it is this bit of code that is failing. for_each_cpu_mask(new_cpu, domain) if (per_cpu(vector_irq, new_cpu)[vector] != -1) goto next; Which would suggest that vector_irq is improperly setup on one of the cpus I am looking at. It might be something stupid like I need to filter domain with cpu_online_map. If my wild set of hypothesis are true the patch below might make those symptoms go away. It isn't a real fix by any means but it is an easy test patch I can generate to generate these giant leaps of deduction, I'm taking in the middle of the night :) Eric diff --git a/arch/x86_64/kernel/genapic_flat.c b/arch/x86_64/kernel/genapic_flat.c index 7c01db8..986fa4b 100644 --- a/arch/x86_64/kernel/genapic_flat.c +++ b/arch/x86_64/kernel/genapic_flat.c @@ -33,7 +33,7 @@ static cpumask_t flat_vector_allocation_ * hyperthread was specified in the interrupt desitination. */ cpumask_t domain = { { [0] = APIC_ALL_CPUS, } }; - return domain; + return cpu_online_map; } /* ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset 2006-10-22 8:35 ` Eric W. Biederman @ 2006-10-22 8:52 ` Muli Ben-Yehuda 2006-10-22 9:10 ` Eric W. Biederman 2006-10-23 4:32 ` [PATCH 1/2] x86_64 irq: Simplify the vector allocator Eric W. Biederman 0 siblings, 2 replies; 24+ messages in thread From: Muli Ben-Yehuda @ 2006-10-22 8:52 UTC (permalink / raw) To: Eric W. Biederman; +Cc: Yinghai Lu, Linux Kernel Mailing List, Andi Kleen On Sun, Oct 22, 2006 at 02:35:19AM -0600, Eric W. Biederman wrote: > Ugh. This is no fun at all. I am assuming this is with cpu hotplug > disabled in flat mode. Correct. > If my wild set of hypothesis are true the patch below might make those > symptoms go away. It isn't a real fix by any means but it is an > easy test patch I can generate to generate these giant leaps > of deduction, I'm taking in the middle of the night :) Yeap, this fixes it. Thanks to you and YL for the quick debugging! May I suggest you CC me in the future on patches in this area and I'll give them a quick spin before they hit mainline? less pain for everyone involved :-) Cheers, Muli ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset 2006-10-22 8:52 ` Muli Ben-Yehuda @ 2006-10-22 9:10 ` Eric W. Biederman 2006-10-23 4:32 ` [PATCH 1/2] x86_64 irq: Simplify the vector allocator Eric W. Biederman 1 sibling, 0 replies; 24+ messages in thread From: Eric W. Biederman @ 2006-10-22 9:10 UTC (permalink / raw) To: Muli Ben-Yehuda; +Cc: Yinghai Lu, Linux Kernel Mailing List, Andi Kleen Muli Ben-Yehuda <muli@il.ibm.com> writes: > On Sun, Oct 22, 2006 at 02:35:19AM -0600, Eric W. Biederman wrote: > >> Ugh. This is no fun at all. I am assuming this is with cpu hotplug >> disabled in flat mode. > > Correct. > >> If my wild set of hypothesis are true the patch below might make those >> symptoms go away. It isn't a real fix by any means but it is an >> easy test patch I can generate to generate these giant leaps >> of deduction, I'm taking in the middle of the night :) > > Yeap, this fixes it. Thanks to you and YL for the quick debugging! > > May I suggest you CC me in the future on patches in this area and I'll > give them a quick spin before they hit mainline? less pain for > everyone involved :-) :) Well we still need to fix it right so there should be a couple of more iterations. I need to sleep on the problem first though :) Eric ^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH 1/2] x86_64 irq: Simplify the vector allocator. 2006-10-22 8:52 ` Muli Ben-Yehuda 2006-10-22 9:10 ` Eric W. Biederman @ 2006-10-23 4:32 ` Eric W. Biederman 2006-10-23 4:35 ` [PATCH 2/2] x86_64 irq: Only look at per_cpu data for online cpus Eric W. Biederman ` (2 more replies) 1 sibling, 3 replies; 24+ messages in thread From: Eric W. Biederman @ 2006-10-23 4:32 UTC (permalink / raw) To: Andi Kleen Cc: Muli Ben-Yehuda, Yinghai Lu, Linux Kernel Mailing List, Andrew Morton There is no reason to remember a per cpu position of which vector to try. Keeping a global position is simpler and more likely to result in a global vector allocation even if I don't need or require it. For level triggered interrupts this means we are less likely to acknowledge another cpus irq, and cause the level triggered irq to harmlessly refire. This simplification makes it easier to only access data structures of online cpus, by having fewer special cases to deal with. Signed-off-by: Eric W. Biederman <ebiederm@xmission.com> --- arch/x86_64/kernel/io_apic.c | 20 +++++++------------- 1 files changed, 7 insertions(+), 13 deletions(-) diff --git a/arch/x86_64/kernel/io_apic.c b/arch/x86_64/kernel/io_apic.c index b000017..0e89ae7 100644 --- a/arch/x86_64/kernel/io_apic.c +++ b/arch/x86_64/kernel/io_apic.c @@ -612,10 +612,7 @@ static int __assign_irq_vector(int irq, * Also, we've got to be careful not to trash gate * 0x80, because int 0x80 is hm, kind of importantish. ;) */ - static struct { - int vector; - int offset; - } pos[NR_CPUS] = { [ 0 ... NR_CPUS - 1] = {FIRST_DEVICE_VECTOR, 0} }; + static int current_vector = FIRST_DEVICE_VECTOR, current_offset = 0; int old_vector = -1; int cpu; @@ -631,14 +628,13 @@ static int __assign_irq_vector(int irq, for_each_cpu_mask(cpu, mask) { cpumask_t domain; - int first, new_cpu; + int new_cpu; int vector, offset; domain = vector_allocation_domain(cpu); - first = first_cpu(domain); - vector = pos[first].vector; - offset = pos[first].offset; + vector = current_vector; + offset = current_offset; next: vector += 8; if (vector >= FIRST_SYSTEM_VECTOR) { @@ -646,7 +642,7 @@ next: offset = (offset + 1) % 8; vector = FIRST_DEVICE_VECTOR + offset; } - if (unlikely(pos[first].vector == vector)) + if (unlikely(current_vector == vector)) continue; if (vector == IA32_SYSCALL_VECTOR) goto next; @@ -654,10 +650,8 @@ next: if (per_cpu(vector_irq, new_cpu)[vector] != -1) goto next; /* Found one! */ - for_each_cpu_mask(new_cpu, domain) { - pos[new_cpu].vector = vector; - pos[new_cpu].offset = offset; - } + current_vector = vector; + current_offset = offset; if (old_vector >= 0) { int old_cpu; for_each_cpu_mask(old_cpu, irq_domain[irq]) -- 1.4.2.rc3.g7e18e-dirty ^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH 2/2] x86_64 irq: Only look at per_cpu data for online cpus. 2006-10-23 4:32 ` [PATCH 1/2] x86_64 irq: Simplify the vector allocator Eric W. Biederman @ 2006-10-23 4:35 ` Eric W. Biederman 2006-10-23 6:17 ` yhlu 2006-10-23 8:14 ` Muli Ben-Yehuda 2006-10-23 6:15 ` [PATCH 1/2] x86_64 irq: Simplify the vector allocator yhlu 2006-10-24 5:29 ` Andi Kleen 2 siblings, 2 replies; 24+ messages in thread From: Eric W. Biederman @ 2006-10-23 4:35 UTC (permalink / raw) To: Andi Kleen Cc: Muli Ben-Yehuda, Yinghai Lu, Linux Kernel Mailing List, Andrew Morton When I generalized __assign_irq_vector I failed to pay attention to what happens when you access a per cpu data structure for a cpu that is not online. It is an undefined case making any code that does it have undefined behavior as well. The code still needs to be able to allocate a vector across cpus that are not online to properly handle combinations like lowest priority interrupt delivery and cpu_hotplug. Not that we can do that today but the infrastructure shouldn't prevent it. So this patch updates the places where we touch per cpu data to only touch online cpus, it makes cpu vector allocation an atomic operation with respect to cpu hotplug, and it updates the cpu start code to properly initialize vector_irq so we don't have inconsistencies. Signed-off-by: Eric W. Biederman <ebiederm@xmission.com> --- arch/x86_64/kernel/io_apic.c | 42 +++++++++++++++++++++++++++++++++++++----- arch/x86_64/kernel/smpboot.c | 7 ++++++- include/asm-x86_64/hw_irq.h | 2 ++ 3 files changed, 45 insertions(+), 6 deletions(-) diff --git a/arch/x86_64/kernel/io_apic.c b/arch/x86_64/kernel/io_apic.c index 0e89ae7..df33383 100644 --- a/arch/x86_64/kernel/io_apic.c +++ b/arch/x86_64/kernel/io_apic.c @@ -63,7 +63,7 @@ int timer_over_8254 __initdata = 1; static struct { int pin, apic; } ioapic_i8259 = { -1, -1 }; static DEFINE_SPINLOCK(ioapic_lock); -static DEFINE_SPINLOCK(vector_lock); +DEFINE_SPINLOCK(vector_lock); /* * # of IRQ routing registers @@ -618,6 +618,9 @@ static int __assign_irq_vector(int irq, BUG_ON((unsigned)irq >= NR_IRQ_VECTORS); + /* Only try and allocate irqs on cpus that are present */ + cpus_and(mask, mask, cpu_online_map); + if (irq_vector[irq] > 0) old_vector = irq_vector[irq]; if (old_vector > 0) { @@ -627,11 +630,12 @@ static int __assign_irq_vector(int irq, } for_each_cpu_mask(cpu, mask) { - cpumask_t domain; + cpumask_t domain, new_mask; int new_cpu; int vector, offset; domain = vector_allocation_domain(cpu); + cpus_and(new_mask, domain, cpu_online_map); vector = current_vector; offset = current_offset; @@ -646,18 +650,20 @@ next: continue; if (vector == IA32_SYSCALL_VECTOR) goto next; - for_each_cpu_mask(new_cpu, domain) + for_each_cpu_mask(new_cpu, new_mask) if (per_cpu(vector_irq, new_cpu)[vector] != -1) goto next; /* Found one! */ current_vector = vector; current_offset = offset; if (old_vector >= 0) { + cpumask_t old_mask; int old_cpu; - for_each_cpu_mask(old_cpu, irq_domain[irq]) + cpus_and(old_mask, irq_domain[irq], cpu_online_map); + for_each_cpu_mask(old_cpu, old_mask) per_cpu(vector_irq, old_cpu)[old_vector] = -1; } - for_each_cpu_mask(new_cpu, domain) + for_each_cpu_mask(new_cpu, new_mask) per_cpu(vector_irq, new_cpu)[vector] = irq; irq_vector[irq] = vector; irq_domain[irq] = domain; @@ -678,6 +684,32 @@ static int assign_irq_vector(int irq, cp return vector; } +void __setup_vector_irq(int cpu) +{ + /* Initialize vector_irq on a new cpu */ + /* This function must be called with vector_lock held */ + unsigned long flags; + int irq, vector; + + + /* Mark the inuse vectors */ + for (irq = 0; irq < NR_IRQ_VECTORS; ++irq) { + if (!cpu_isset(cpu, irq_domain[irq])) + continue; + vector = irq_vector[irq]; + per_cpu(vector_irq, cpu)[vector] = irq; + } + /* Mark the free vectors */ + for (vector = 0; vector < NR_VECTORS; ++vector) { + irq = per_cpu(vector_irq, cpu)[vector]; + if (irq < 0) + continue; + if (!cpu_isset(cpu, irq_domain[irq])) + per_cpu(vector_irq, cpu)[vector] = -1; + } +} + + extern void (*interrupt[NR_IRQS])(void); static struct irq_chip ioapic_chip; diff --git a/arch/x86_64/kernel/smpboot.c b/arch/x86_64/kernel/smpboot.c index 7b7a687..62c2e74 100644 --- a/arch/x86_64/kernel/smpboot.c +++ b/arch/x86_64/kernel/smpboot.c @@ -581,12 +581,16 @@ void __cpuinit start_secondary(void) * smp_call_function(). */ lock_ipi_call_lock(); + spin_lock(&vector_lock); + /* Setup the per cpu irq handling data structures */ + __setup_vector_irq(smp_processor_id()); /* * Allow the master to continue. */ cpu_set(smp_processor_id(), cpu_online_map); per_cpu(cpu_state, smp_processor_id()) = CPU_ONLINE; + spin_unlock(&vector_lock); unlock_ipi_call_lock(); cpu_idle(); @@ -799,7 +803,6 @@ static int __cpuinit do_boot_cpu(int cpu cpu, node); } - alternatives_smp_switch(1); c_idle.idle = get_idle_for_cpu(cpu); @@ -1246,8 +1249,10 @@ int __cpu_disable(void) local_irq_disable(); remove_siblinginfo(cpu); + spin_lock(&vector_lock); /* It's now safe to remove this processor from the online map */ cpu_clear(cpu, cpu_online_map); + spin_unlock(&vector_lock); remove_cpu_from_maps(); fixup_irqs(cpu_online_map); return 0; diff --git a/include/asm-x86_64/hw_irq.h b/include/asm-x86_64/hw_irq.h index 792dd52..179cce7 100644 --- a/include/asm-x86_64/hw_irq.h +++ b/include/asm-x86_64/hw_irq.h @@ -76,6 +76,8 @@ #define FIRST_SYSTEM_VECTOR 0xef /* du #ifndef __ASSEMBLY__ typedef int vector_irq_t[NR_VECTORS]; DECLARE_PER_CPU(vector_irq_t, vector_irq); +extern void __setup_vector_irq(int cpu); +extern spinlock_t vector_lock; /* * Various low-level irq details needed by irq.c, process.c, -- 1.4.2.rc3.g7e18e-dirty ^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH 2/2] x86_64 irq: Only look at per_cpu data for online cpus. 2006-10-23 4:35 ` [PATCH 2/2] x86_64 irq: Only look at per_cpu data for online cpus Eric W. Biederman @ 2006-10-23 6:17 ` yhlu 2006-10-23 8:14 ` Muli Ben-Yehuda 1 sibling, 0 replies; 24+ messages in thread From: yhlu @ 2006-10-23 6:17 UTC (permalink / raw) To: Eric W. Biederman Cc: Andi Kleen, Muli Ben-Yehuda, Linux Kernel Mailing List, Andrew Morton On 10/22/06, Eric W. Biederman <ebiederm@xmission.com> wrote: > > When I generalized __assign_irq_vector I failed to pay attention > to what happens when you access a per cpu data structure for > a cpu that is not online. It is an undefined case making any > code that does it have undefined behavior as well. > > The code still needs to be able to allocate a vector across cpus > that are not online to properly handle combinations like lowest > priority interrupt delivery and cpu_hotplug. Not that we can do > that today but the infrastructure shouldn't prevent it. > > So this patch updates the places where we touch per cpu data > to only touch online cpus, it makes cpu vector allocation > an atomic operation with respect to cpu hotplug, and it updates > the cpu start code to properly initialize vector_irq so we > don't have inconsistencies. > > Signed-off-by: Eric W. Biederman <ebiederm@xmission.com> Acked-by: Yinghai Lu <yinghai.lu@amd.com> ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 2/2] x86_64 irq: Only look at per_cpu data for online cpus. 2006-10-23 4:35 ` [PATCH 2/2] x86_64 irq: Only look at per_cpu data for online cpus Eric W. Biederman 2006-10-23 6:17 ` yhlu @ 2006-10-23 8:14 ` Muli Ben-Yehuda 1 sibling, 0 replies; 24+ messages in thread From: Muli Ben-Yehuda @ 2006-10-23 8:14 UTC (permalink / raw) To: Eric W. Biederman Cc: Andi Kleen, Yinghai Lu, Linux Kernel Mailing List, Andrew Morton On Sun, Oct 22, 2006 at 10:35:34PM -0600, Eric W. Biederman wrote: > > When I generalized __assign_irq_vector I failed to pay attention > to what happens when you access a per cpu data structure for > a cpu that is not online. It is an undefined case making any > code that does it have undefined behavior as well. > > The code still needs to be able to allocate a vector across cpus > that are not online to properly handle combinations like lowest > priority interrupt delivery and cpu_hotplug. Not that we can do > that today but the infrastructure shouldn't prevent it. > > So this patch updates the places where we touch per cpu data > to only touch online cpus, it makes cpu vector allocation > an atomic operation with respect to cpu hotplug, and it updates > the cpu start code to properly initialize vector_irq so we > don't have inconsistencies. > > Signed-off-by: Eric W. Biederman <ebiederm@xmission.com> I tried 1/2 and 2/2 at the same time and it booted, so good work :-) I'm stressing the machine a little now, will let you know if anything out of the ordinary comes up. Acked-by: Muli Ben-Yehuda <muli@il.ibm.com> Cheers, Muli ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 1/2] x86_64 irq: Simplify the vector allocator. 2006-10-23 4:32 ` [PATCH 1/2] x86_64 irq: Simplify the vector allocator Eric W. Biederman 2006-10-23 4:35 ` [PATCH 2/2] x86_64 irq: Only look at per_cpu data for online cpus Eric W. Biederman @ 2006-10-23 6:15 ` yhlu 2006-10-24 5:29 ` Andi Kleen 2 siblings, 0 replies; 24+ messages in thread From: yhlu @ 2006-10-23 6:15 UTC (permalink / raw) To: Eric W. Biederman Cc: Andi Kleen, Muli Ben-Yehuda, Linux Kernel Mailing List, Andrew Morton On 10/22/06, Eric W. Biederman <ebiederm@xmission.com> wrote: > > There is no reason to remember a per cpu position of which vector > to try. Keeping a global position is simpler and more likely to > result in a global vector allocation even if I don't need or require > it. For level triggered interrupts this means we are less likely to > acknowledge another cpus irq, and cause the level triggered irq to > harmlessly refire. > > This simplification makes it easier to only access data structures > of online cpus, by having fewer special cases to deal with. > > > Signed-off-by: Eric W. Biederman <ebiederm@xmission.com> Good, It will keep increase vector, and only try to use same vector for different cpu when vector is used up. Acked-by: Yinghai Lu <yinghai.lu@amd.com> ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH 1/2] x86_64 irq: Simplify the vector allocator. 2006-10-23 4:32 ` [PATCH 1/2] x86_64 irq: Simplify the vector allocator Eric W. Biederman 2006-10-23 4:35 ` [PATCH 2/2] x86_64 irq: Only look at per_cpu data for online cpus Eric W. Biederman 2006-10-23 6:15 ` [PATCH 1/2] x86_64 irq: Simplify the vector allocator yhlu @ 2006-10-24 5:29 ` Andi Kleen 2 siblings, 0 replies; 24+ messages in thread From: Andi Kleen @ 2006-10-24 5:29 UTC (permalink / raw) To: Eric W. Biederman Cc: Muli Ben-Yehuda, Yinghai Lu, Linux Kernel Mailing List, Andrew Morton On Sunday 22 October 2006 21:32, Eric W. Biederman wrote: > There is no reason to remember a per cpu position of which vector > to try. Keeping a global position is simpler and more likely to > result in a global vector allocation even if I don't need or require > it. For level triggered interrupts this means we are less likely to > acknowledge another cpus irq, and cause the level triggered irq to > harmlessly refire. > > This simplification makes it easier to only access data structures > of online cpus, by having fewer special cases to deal with. Shouldn't this and the following patch be done on i386 too? -Andi ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset 2006-10-22 3:51 ` [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset Muli Ben-Yehuda ` (3 preceding siblings ...) 2006-10-22 8:35 ` Eric W. Biederman @ 2006-10-22 13:29 ` Andi Kleen 2006-10-22 16:31 ` Muli Ben-Yehuda 4 siblings, 1 reply; 24+ messages in thread From: Andi Kleen @ 2006-10-22 13:29 UTC (permalink / raw) To: Muli Ben-Yehuda; +Cc: Yinghai Lu, Eric W. Biederman, Linux Kernel Mailing List > This patch breaks my x366 machine: > > aic94xx: device 0000:01:02.0: SAS addr 5005076a0112df00, PCBA SN , 8 phys, 8 enabled phys, flash present, BIOS build 1323 > aic94xx: couldn't get irq 25 for 0000:01:02.0 > ACPI: PCI interrupt for device 0000:01:02.0 disabled > aic94xx: probe of 0000:01:02.0 failed with error -38 > > Reverting it allows it to boot again. Since the patch is "obviously > correct", it must be uncovering some other problem with the genirq > code. I wonder if the machine works when booted with a 32bit kernel? Presumably Eric made less typos when doing the i386 genirq code @) -Andi ^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset 2006-10-22 13:29 ` [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset Andi Kleen @ 2006-10-22 16:31 ` Muli Ben-Yehuda 0 siblings, 0 replies; 24+ messages in thread From: Muli Ben-Yehuda @ 2006-10-22 16:31 UTC (permalink / raw) To: Andi Kleen; +Cc: Yinghai Lu, Eric W. Biederman, Linux Kernel Mailing List On Sun, Oct 22, 2006 at 03:29:38PM +0200, Andi Kleen wrote: > > > This patch breaks my x366 machine: > > > > aic94xx: device 0000:01:02.0: SAS addr 5005076a0112df00, PCBA SN , 8 phys, 8 enabled phys, flash present, BIOS build 1323 > > aic94xx: couldn't get irq 25 for 0000:01:02.0 > > ACPI: PCI interrupt for device 0000:01:02.0 disabled > > aic94xx: probe of 0000:01:02.0 failed with error -38 > > > > Reverting it allows it to boot again. Since the patch is "obviously > > correct", it must be uncovering some other problem with the genirq > > code. > > I wonder if the machine works when booted with a 32bit kernel? Mostly... I don't have a 32-bit initrd environment for aic94xx, so I gave it a spin with aic94xx built in and without firmware. It made it as far as trying to mount the root device, and then sat there spitting this out continously: atkbd.c: Spurious ACK on isa0060/serio0. Some program might be trying access hardware directly I can put together a 32-bit initrd and try booting all the way if it'll be a useful experiment, let me know. Cheers, Muli ^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2006-10-24 12:29 UTC | newest]
Thread overview: 24+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <200610212100.k9LL0GtC018787@hera.kernel.org>
2006-10-22 3:51 ` [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset Muli Ben-Yehuda
2006-10-22 5:13 ` Yinghai Lu
2006-10-22 6:55 ` yhlu
2006-10-22 8:50 ` Muli Ben-Yehuda
2006-10-22 8:50 ` Eric W. Biederman
2006-10-22 8:28 ` Yinghai Lu
2006-10-22 8:50 ` Muli Ben-Yehuda
2006-10-22 8:58 ` Eric W. Biederman
2006-10-22 16:02 ` yhlu
2006-10-22 16:19 ` yhlu
2006-10-22 21:33 ` Andi Kleen
2006-10-22 16:20 ` Muli Ben-Yehuda
2006-10-22 16:28 ` yhlu
2006-10-22 8:35 ` Eric W. Biederman
2006-10-22 8:52 ` Muli Ben-Yehuda
2006-10-22 9:10 ` Eric W. Biederman
2006-10-23 4:32 ` [PATCH 1/2] x86_64 irq: Simplify the vector allocator Eric W. Biederman
2006-10-23 4:35 ` [PATCH 2/2] x86_64 irq: Only look at per_cpu data for online cpus Eric W. Biederman
2006-10-23 6:17 ` yhlu
2006-10-23 8:14 ` Muli Ben-Yehuda
2006-10-23 6:15 ` [PATCH 1/2] x86_64 irq: Simplify the vector allocator yhlu
2006-10-24 5:29 ` Andi Kleen
2006-10-22 13:29 ` [PATCH] x86-64: typo in __assign_irq_vector when updating pos for vector and offset Andi Kleen
2006-10-22 16:31 ` Muli Ben-Yehuda
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox