* lapic & npt @ 2008-06-04 16:11 Gerd Hoffmann 2008-06-04 16:16 ` Avi Kivity 2008-06-04 18:56 ` Vringfd Dong, Eddie 0 siblings, 2 replies; 7+ messages in thread From: Gerd Hoffmann @ 2008-06-04 16:11 UTC (permalink / raw) To: kvm Hi, I've just noticed that xenner doesn't work for 64bit xen guest kernels on my new shiny barcelona box. The VM crashes when trying to access the lapic. lapic setup is done before idt setup, register dump looks like init state, thus it most likely is a triple fault resetting the vcpu. Turning off npt fixes it. xenner maps the lapic to ffff820000000000 (64bit guests) or fe900000 (32bit guests). 32bit works fine even with npt enabled. host kernel is fedora 9 with kvm-69 modules. ideas anyone? cheers, Gerd -- http://kraxel.fedorapeople.org/xenner/ ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: lapic & npt 2008-06-04 16:11 lapic & npt Gerd Hoffmann @ 2008-06-04 16:16 ` Avi Kivity 2008-06-06 14:45 ` Gerd Hoffmann 2008-06-30 10:21 ` Gerd Hoffmann 2008-06-04 18:56 ` Vringfd Dong, Eddie 1 sibling, 2 replies; 7+ messages in thread From: Avi Kivity @ 2008-06-04 16:16 UTC (permalink / raw) To: Gerd Hoffmann; +Cc: kvm Gerd Hoffmann wrote: > Hi, > > I've just noticed that xenner doesn't work for 64bit xen guest kernels > on my new shiny barcelona box. The VM crashes when trying to access the > lapic. lapic setup is done before idt setup, register dump looks like > init state, thus it most likely is a triple fault resetting the vcpu. > Turning off npt fixes it. > > xenner maps the lapic to ffff820000000000 (64bit guests) or fe900000 > (32bit guests). 32bit works fine even with npt enabled. > > host kernel is fedora 9 with kvm-69 modules. > > ideas anyone? > > Turn on logging in lapic.c. See if something turns up. If not, the page tables are probably set up incorrectly, but in a way that kvm doesn't notice. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: lapic & npt 2008-06-04 16:16 ` Avi Kivity @ 2008-06-06 14:45 ` Gerd Hoffmann 2008-06-06 16:30 ` Avi Kivity 2008-06-30 10:21 ` Gerd Hoffmann 1 sibling, 1 reply; 7+ messages in thread From: Gerd Hoffmann @ 2008-06-06 14:45 UTC (permalink / raw) To: Avi Kivity; +Cc: kvm Avi Kivity wrote: > Gerd Hoffmann wrote: >> Hi, >> >> I've just noticed that xenner doesn't work for 64bit xen guest kernels >> on my new shiny barcelona box. The VM crashes when trying to access the >> lapic. lapic setup is done before idt setup, register dump looks like >> init state, thus it most likely is a triple fault resetting the vcpu. >> Turning off npt fixes it. >> >> xenner maps the lapic to ffff820000000000 (64bit guests) or fe900000 >> (32bit guests). 32bit works fine even with npt enabled. >> >> host kernel is fedora 9 with kvm-69 modules. >> >> ideas anyone? > > Turn on logging in lapic.c. See if something turns up. > > If not, the page tables are probably set up incorrectly, but in a way > that kvm doesn't notice. Figured meanwhile the instruction accessing the lapic register somehow beames me into my gpf handler. With a quite strange rsp value. Hmm. To be analyzed further after kvm forum I guess, unless someone has a bright idea ... cheers, Gerd -- http://kraxel.fedorapeople.org/xenner/ ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: lapic & npt 2008-06-06 14:45 ` Gerd Hoffmann @ 2008-06-06 16:30 ` Avi Kivity 2008-06-27 14:42 ` Gerd Hoffmann 0 siblings, 1 reply; 7+ messages in thread From: Avi Kivity @ 2008-06-06 16:30 UTC (permalink / raw) To: Gerd Hoffmann; +Cc: kvm Gerd Hoffmann wrote: > Avi Kivity wrote: > >> Gerd Hoffmann wrote: >> >>> Hi, >>> >>> I've just noticed that xenner doesn't work for 64bit xen guest kernels >>> on my new shiny barcelona box. The VM crashes when trying to access the >>> lapic. lapic setup is done before idt setup, register dump looks like >>> init state, thus it most likely is a triple fault resetting the vcpu. >>> Turning off npt fixes it. >>> >>> xenner maps the lapic to ffff820000000000 (64bit guests) or fe900000 >>> (32bit guests). 32bit works fine even with npt enabled. >>> >>> host kernel is fedora 9 with kvm-69 modules. >>> >>> ideas anyone? >>> >> Turn on logging in lapic.c. See if something turns up. >> >> If not, the page tables are probably set up incorrectly, but in a way >> that kvm doesn't notice. >> > > Figured meanwhile the instruction accessing the lapic register somehow > beames me into my gpf handler. With a quite strange rsp value. Hmm. > To be analyzed further after kvm forum I guess, unless someone has a > bright idea ... > x86-64 will gpf if an address is not canonical. ffff820000000000 is canonical, but perhaps a type killed a zero somewhere? -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: lapic & npt 2008-06-06 16:30 ` Avi Kivity @ 2008-06-27 14:42 ` Gerd Hoffmann 0 siblings, 0 replies; 7+ messages in thread From: Gerd Hoffmann @ 2008-06-27 14:42 UTC (permalink / raw) To: Avi Kivity; +Cc: kvm Hi, >> Figured meanwhile the instruction accessing the lapic register somehow >> beames me into my gpf handler. With a quite strange rsp value. Hmm. >> To be analyzed further after kvm forum I guess, unless someone has a >> bright idea ... > x86-64 will gpf if an address is not canonical. ffff820000000000 is > canonical, but perhaps a type killed a zero somewhere? Doesn't look like. Tried to analyze that. Compiled lastest kvm.git kernel (to get kvmtrace support), booted, and noticed "kvm-amd npt=0" doesn't work at all ... cheers, Gerd -- http://kraxel.fedorapeople.org/xenner/ ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: lapic & npt 2008-06-04 16:16 ` Avi Kivity 2008-06-06 14:45 ` Gerd Hoffmann @ 2008-06-30 10:21 ` Gerd Hoffmann 1 sibling, 0 replies; 7+ messages in thread From: Gerd Hoffmann @ 2008-06-30 10:21 UTC (permalink / raw) To: Avi Kivity; +Cc: kvm [-- Attachment #1: Type: text/plain, Size: 1394 bytes --] Avi Kivity wrote: > Gerd Hoffmann wrote: >> Hi, >> >> I've just noticed that xenner doesn't work for 64bit xen guest kernels >> on my new shiny barcelona box. The VM crashes when trying to access the >> lapic. lapic setup is done before idt setup, register dump looks like >> init state, thus it most likely is a triple fault resetting the vcpu. >> Turning off npt fixes it. >> >> xenner maps the lapic to ffff820000000000 (64bit guests) or fe900000 >> (32bit guests). 32bit works fine even with npt enabled. >> >> host kernel is fedora 9 with kvm-69 modules. >> >> ideas anyone? > > Turn on logging in lapic.c. See if something turns up. It doesn't come that far according to kvmtrace. > If not, the page tables are probably set up incorrectly, but in a way > that kvm doesn't notice. Dunno. Tried kvmtrace and got the results attached without and with NTP. Cutted down to the important sequence, starting with the apic MSR read. After the msr access there are some page faults (some page table pages are allocated and used to map the apic, thus likely triggering shadow pt updates). Then the apic access (id register). With npt kvm doesn't see the apic access at all. Also the TDP_FAULT line looks fishy to me. The "virt = ... " isn't a (guest) virtual address. btw: the iowrite to 0x00ea is console output. ideas anyone? Gerd -- http://kraxel.fedorapeople.org/xenner/ [-- Attachment #2: no-npt --] [-- Type: text/plain, Size: 1705 bytes --] 1673786946412 (+ 6068) VMEXIT vcpu = 0x00000000 pid = 0x0000102d [ exitcode = 0x0000007c, rip = 0xffff8300 0000a1d2 ] 0 (+ 0) MSR_READ vcpu = 0x00000000 pid = 0x0000102d [ MSR# = 0x0000001b, data = 0x00000000 fee00900 ] 1673786950748 (+ 4336) VMENTRY vcpu = 0x00000000 pid = 0x0000102d 1673786957068 (+ 6320) VMEXIT vcpu = 0x00000000 pid = 0x0000102d [ exitcode = 0x00000003, rip = 0xffff8300 00006ceb ] 0 (+ 0) CR_READ vcpu = 0x00000000 pid = 0x0000102d [ CR# = 3, value = 0x00000000 017a3000 ] 1673786966522 (+ 9454) VMENTRY vcpu = 0x00000000 pid = 0x0000102d 1673786973678 (+ 7156) VMEXIT vcpu = 0x00000000 pid = 0x0000102d [ exitcode = 0x0000004e, rip = 0xffff8300 00004023 ] 0 (+ 0) PAGE_FAULT vcpu = 0x00000000 pid = 0x0000102d [ errorcode = 0x00000002, virt = 0xffff8300 00048000 ] 1673787002404 (+ 28726) VMENTRY vcpu = 0x00000000 pid = 0x0000102d 1673787034034 (+ 31630) VMEXIT vcpu = 0x00000000 pid = 0x0000102d [ exitcode = 0x0000004e, rip = 0xffff8300 00004023 ] 0 (+ 0) PAGE_FAULT vcpu = 0x00000000 pid = 0x0000102d [ errorcode = 0x00000002, virt = 0xffff8300 00049000 ] 1673787049388 (+ 15354) VMENTRY vcpu = 0x00000000 pid = 0x0000102d 1673787080890 (+ 31502) VMEXIT vcpu = 0x00000000 pid = 0x0000102d [ exitcode = 0x0000004e, rip = 0xffff8300 0000a1ef ] 0 (+ 0) PAGE_FAULT vcpu = 0x00000000 pid = 0x0000102d [ errorcode = 0x00000000, virt = 0xffff8200 00000020 ] 0 (+ 0) APIC_ACCESS vcpu = 0x00000000 pid = 0x0000102d [ offset = 0x00000020 ] 1673787096270 (+ 15380) VMENTRY vcpu = 0x00000000 pid = 0x0000102d [-- Attachment #3: npt --] [-- Type: text/plain, Size: 1603 bytes --] 1834321779170 (+ 6312) VMEXIT vcpu = 0x00000000 pid = 0x00001080 [ exitcode = 0x0000007c, rip = 0xffff8300 0000a1d2 ] 0 (+ 0) MSR_READ vcpu = 0x00000000 pid = 0x00001080 [ MSR# = 0x0000001b, data = 0x00000000 fee00900 ] 1834321783872 (+ 4702) VMENTRY vcpu = 0x00000000 pid = 0x00001080 1834321793790 (+ 9918) VMEXIT vcpu = 0x00000000 pid = 0x00001080 [ exitcode = 0x00000400, rip = 0xffff8300 00004023 ] 0 (+ 0) TDP_FAULT vcpu = 0x00000000 pid = 0x00001080 [ errorcode = 0x00000006, virt = 0x00000000 00048000 ] 1834321818582 (+ 24792) VMENTRY vcpu = 0x00000000 pid = 0x00001080 1834321851480 (+ 32898) VMEXIT vcpu = 0x00000000 pid = 0x00001080 [ exitcode = 0x00000400, rip = 0xffff8300 00004023 ] 0 (+ 0) TDP_FAULT vcpu = 0x00000000 pid = 0x00001080 [ errorcode = 0x00000006, virt = 0x00000000 00049000 ] 1834321870858 (+ 19378) VMENTRY vcpu = 0x00000000 pid = 0x00001080 1834321906650 (+ 35792) VMEXIT vcpu = 0x00000000 pid = 0x00001080 [ exitcode = 0x00000400, rip = 0xffff8300 000053b1 ] 0 (+ 0) TDP_FAULT vcpu = 0x00000000 pid = 0x00001080 [ errorcode = 0x00000004, virt = 0x00000000 0000d000 ] 1834321912818 (+ 6168) VMENTRY vcpu = 0x00000000 pid = 0x00001080 1834321931382 (+ 18564) VMEXIT vcpu = 0x00000000 pid = 0x00001080 [ exitcode = 0x0000007b, rip = 0xffff8300 0000b256 ] 0 (+ 0) IO_WRITE vcpu = 0x00000000 pid = 0x00001080 [ port = 0x00ea, size = 1 ] 1834322090822 (+ 159440) VMENTRY vcpu = 0x00000000 pid = 0x00001080 ^ permalink raw reply [flat|nested] 7+ messages in thread
* Vringfd 2008-06-04 16:11 lapic & npt Gerd Hoffmann 2008-06-04 16:16 ` Avi Kivity @ 2008-06-04 18:56 ` Dong, Eddie 1 sibling, 0 replies; 7+ messages in thread From: Dong, Eddie @ 2008-06-04 18:56 UTC (permalink / raw) To: kvm; +Cc: Dong, Eddie Rusty/Anthony/Avi and all: I did a read at Rusty's April 18 post of vringfd patch, though I didn't find explicit code for how to call get_user_skb_frags, it seems to be clear the user only need to trigger a "flush_tx" command. All the rest will be automatically turned on, fetching the ring and send to bridge etc, which is perfect from performance point of view. Now I am working on enabling VMDq which provide HW pre-sorting mechanism base on MAC address, it can definetely take the advantage of vringfd with additional advantage of 0-copy in receive side, which makes VMDq enabled virtio-net fly with 0-copy for both transmit & receive. The only thing an VMDq agency (vringfd similar components in kernel) may bother user level is initialization and notification given that vringfd already solved the ring operations, since KVM virtual interrupt module is in kernel side, it is pretty easy to make user free of receiving packet at run time. Any comments? Thanks, eddie ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2008-06-30 10:21 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2008-06-04 16:11 lapic & npt Gerd Hoffmann 2008-06-04 16:16 ` Avi Kivity 2008-06-06 14:45 ` Gerd Hoffmann 2008-06-06 16:30 ` Avi Kivity 2008-06-27 14:42 ` Gerd Hoffmann 2008-06-30 10:21 ` Gerd Hoffmann 2008-06-04 18:56 ` Vringfd Dong, Eddie
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox