public inbox for kvm@vger.kernel.org
 help / color / mirror / Atom feed
* Guest memory backed by PCI BAR (x86)
@ 2015-03-25 15:56 Nate Case
  2015-03-26 14:02 ` Paolo Bonzini
  0 siblings, 1 reply; 10+ messages in thread
From: Nate Case @ 2015-03-25 15:56 UTC (permalink / raw)
  To: kvm

Hello,

I have an unusual goal of presenting SDRAM located on a real PCIe device
(exposed via BAR) to a guest as normal memory.  Eventually I'd like to
split up guest memory between PCI memory and host memory as different
NUMA nodes to optimize performance;  but for now I'm focusing on just
getting the guest to use memory over PCIe entirely, ignoring the
awful performance.

My first attempt was to modify qemu's "-mem-path" parameter to also
support mmap()able files in addition to hugetlbfs paths.  This was
fairly straightforward, and using a tmpfs file (backed by host SDRAM)
as guest RAM appears to work fine.

I was hoping I could then use a PCI sysfs resource file instead of a
tmpfs file (i.e., /sys/bus/pci/devices/dddd:bb:ss.f/resourceN) to
achieve the desired effect.  But I haven't been able to get Linux or
memtest86+ to boot with this arrangement.  It only boots when KVM
acceleration is disabled.

When KVM acceleration is enabled, SeaBIOS seems to function fine
running out of PCI memory space, but booting the OS resets.
Specifically, the following happens (I'll stick with the memtest86+
5.01 test case for simplicity):

setup.S of memtest:

---[snip]---
  /*
   * Note that the short jump isn't strictly needed, althought there are
   * reasons why it might be a good idea. It won't hurt in any case.
   */
          movw    $0x0001, %ax    # protected mode (PE) bit
          lmsw    %ax             # This is it#
          jmp     flush_instr
  flush_instr:
          movw    $KERNEL_DS, %ax
          movw    %ax, %ds
          movw    %ax, %es
          movw    %ax, %ss    <---- broken here
          movw    %ax, %fs
          movw    %ax, %gs
---[snip]---

In this portion that's attempting to enter protected mode, the
"movw %ax, %ss" instruction execution results in the guest resetting.
gdb shows that after stepping through this instruction, CS:IP points
to F000:E05B (entry_post within SeaBIOS).

I'm new to KVM, so I'm hoping someone could provide some guidance on
what might be wrong, what you'd recommend to get this working, or how
I might debug this better.

I'm using qemu 2.2.0 and an EL6 kernel (2.6.32-358.23.2.el6.x86_64).

Thanks,

Nate

^ permalink raw reply	[flat|nested] 10+ messages in thread
end of thread, other threads:[~2015-03-27 15:27 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2015-03-25 15:56 Guest memory backed by PCI BAR (x86) Nate Case
2015-03-26 14:02 ` Paolo Bonzini
2015-03-26 16:01   ` Nate Case
2015-03-26 16:07     ` Paolo Bonzini
2015-03-26 16:34       ` Nate Case
2015-03-26 16:40         ` Paolo Bonzini
2015-03-26 16:52           ` Nate Case
2015-03-26 17:04             ` Paolo Bonzini
2015-03-26 17:14               ` Nate Case
2015-03-27 15:27               ` Nate Case

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox