Re: [PATCH][RESEND v5 3/3] Add a Hyper-V Dynamic Memory Protocol driver (hv-balloon)

[David Hildenbrand <david@redhat.com>]

On 22.06.23 13:12, Maciej S. Szmigiero wrote:
> On 22.06.2023 13:01, David Hildenbrand wrote:
>> [...]
>>
>>>>>> We'd use a memory region container as device memory region (like [1]) and would have to handle the !memdev case (I can help with that). > Into that, you can map the RAM memory region on demand (and eventually even using multiple slots like [1]).
>>>>>>
>>>>>> (2) Use a single virtual DIMM and (un)plug that on demand. Let the machine code handle (un)plugging of the device.
>>>>>>
>>>>>>
>>>>>> (1) feels cleanest to me, although it will require a bit more work.
>>>>>>
>>>>>
>>>>> I also think approach (1) makes more sense as it avoids memslot metadata
>>>>> overhead for not-yet-hot-added parts of the memory backing device.
>>>>>
>>>>> Not sure what you mean that the !memdev case would be problematic in this
>>>>> case - it is working in the current driver shape so why would adding
>>>>> potential memory subregions (used in the memdev case) change that?
>>>>
>>>> I'm thinking about the case where you have a hv-balloon device without a memdev.
>>>>
>>>> Without -m X,maxmem=y we don't currently expect to have memory devices around
>>>> (and especially them getting (un)plugged. But why should we "force" to set the
>>>> "maxmem" option
>>>
>>> I guess it's only a small change to QEMU to allow having hv-balloon
>>> device (without a memdev) even in the case where there's no "maxmem"
>>> option given on the QEMU command line.
>>>
>>>>
>>>> I hope I'll find some time soonish to prototype what I have in mind, to see
>>>> if it could be made working.
>>>>
>>>
>>> Okay, so I'll wait for your prototype before commencing further work on
>>> the next version of this driver.
>>
>> About to have something simplistic running -- I think. Want to test with a Linux VM, but I don't seem to get it working (also without my changes).
>>
>>
>> #!/bin/bash
>>
>> build/qemu-system-x86_64 \
>>       --enable-kvm \
>>       -m 4G,maxmem=36G \
>>       -cpu host,hv-syndbg=on,hv-synic,hv-relaxed,hv-vpindex \
>>       -smp 16 \
>>       -nographic \
>>       -nodefaults \
>>       -net nic -net user \
>>       -chardev stdio,nosignal,id=serial \
>>       -hda Fedora-Cloud-Base-37-1.7.x86_64.qcow2 \
>>       -cdrom /home/dhildenb/git/cloud-init/cloud-init.iso \
>>       -device isa-serial,chardev=serial \
>>       -chardev socket,id=monitor,path=/var/tmp/mon_src,server,nowait \
>>       -mon chardev=monitor,mode=readline \
>>       -device vmbus-bridge \
>>       -object memory-backend-ram,size=2G,id=mem0 \
>>       -device hv-balloon,id=hv1,memdev=mem0
>>
>>
>>
>> [root@vm-0 ~]# uname -r
>> 6.3.5-100.fc37.x86_64
>> [root@vm-0 ~]# modprobe hv_balloon
>> modprobe: ERROR: could not insert 'hv_balloon': No such device
>>
>>
>> Any magic flag I am missing? Or is there something preventing this to work with Linux VMs?
>>
> 
> Haven't tested the driver with Linux guests in a long time (as it is
> targeting Windows), but I think you need to disable KVM PV interface for
> the Hyper-V one to be detected by Linux.
> 
> Something like adding "kvm=off" to "-cpu" and seeing in the dmesg whether
> the detected hypervisor is now Hyper-V.
> 
> Also, you need to disable S4 in the guest for hot-add capability to work
> (I'm adding "-global ICH9-LPC.disable_s4=1" with q35 machine for this).
> 
> Would also suggest adding "--trace 'hv_balloon_*' --trace 'memory_device_*'"
> to QEMU command line to see what's happening.

VM is not happy:

[    1.908595] BUG: kernel NULL pointer dereference, address: 0000000000000007
[    1.908837] #PF: supervisor read access in kernel mode
[    1.908837] #PF: error_code(0x0000) - not-present page
[    1.908837] PGD 0 P4D 0
[    1.908837] Oops: 0000 [#1] PREEMPT SMP NOPTI
[    1.908837] CPU: 13 PID: 492 Comm: (udev-worker) Not tainted 6.3.5-100.fc37.x86_64 #1
[    1.908837] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.16.2-0-gea1b7a073390-p4
[    1.908837] RIP: 0010:acpi_ns_lookup+0x8f/0x4c0
[    1.908837] Code: 8b 3d f5 eb 1c 03 83 05 52 ec 1c 03 01 48 85 ff 0f 84 51 03 00 00 44 89 c3 4c 89 cb
[    1.908837] RSP: 0018:ffff95b680ad7950 EFLAGS: 00010286
[    1.908837] RAX: ffff95b680ad79e0 RBX: 0000000000000002 RCX: 0000000000000003
[    1.908837] RDX: 0000000000000000 RSI: ffff8a0283a3c558 RDI: ffffffffa4b376e0
[    1.908837] RBP: 0000000000000000 R08: 0000000000000002 R09: 0000000000000000
[    1.908837] R10: ffff8a02811034ec R11: 0000000000000000 R12: ffffffffffffffff
[    1.908837] R13: ffff8a02811034e8 R14: ffff8a02811034e8 R15: 0000000000000000
[    1.908837] FS:  00007f3bb2e7d0c0(0000) GS:ffff8a02bbd40000(0000) knlGS:0000000000000000
[    1.908837] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    1.908837] CR2: 0000000000000007 CR3: 0000000100a58002 CR4: 0000000000770ee0
[    1.908837] PKRU: 55555554
[    1.908837] Call Trace:
[    1.908837]  <TASK>
[    1.908837]  ? __die+0x23/0x70
[    1.908837]  ? page_fault_oops+0x171/0x4e0
[    1.908837]  ? prepare_alloc_pages.constprop.0+0xf6/0x1a0
[    1.908837]  ? exc_page_fault+0x74/0x170
[    1.908837]  ? asm_exc_page_fault+0x26/0x30
[    1.908837]  ? acpi_ns_lookup+0x8f/0x4c0
[    1.908837]  acpi_ns_get_node_unlocked+0xdd/0x110
[    1.908837]  ? down_timeout+0x3e/0x60
[    1.908837]  ? acpi_ns_get_node+0x3e/0x60
[    1.908837]  acpi_ns_get_node+0x3e/0x60
[    1.908837]  acpi_ns_evaluate+0x1cb/0x2d0
[    1.908837]  acpi_ut_evaluate_object+0x68/0x1c0
[    1.908837]  acpi_rs_get_method_data+0x37/0x80
[    1.908837]  ? __pfx_vmbus_walk_resources+0x10/0x10 [hv_vmbus]
[    1.908837]  acpi_walk_resources+0x91/0xe0
[    1.908837]  vmbus_acpi_add+0x87/0x170 [hv_vmbus]
[    1.908837]  acpi_device_probe+0x47/0x160
[    1.908837]  really_probe+0x19f/0x400
[    1.908837]  ? __pfx___driver_attach+0x10/0x10
[    1.908837]  __driver_probe_device+0x78/0x160
[    1.908837]  driver_probe_device+0x1f/0x90
[    1.908837]  __driver_attach+0xd2/0x1c0
[    1.908837]  bus_for_each_dev+0x85/0xd0
[    1.908837]  bus_add_driver+0x116/0x220
[    1.908837]  driver_register+0x59/0x100
[    1.908837]  ? __pfx_init_module+0x10/0x10 [hv_vmbus]
[    1.908837]  hv_acpi_init+0x39/0xff0 [hv_vmbus]
[    1.908837]  ? __pfx_init_module+0x10/0x10 [hv_vmbus]
[    1.908837]  do_one_initcall+0x5a/0x240
[    1.908837]  do_init_module+0x4a/0x210
[    1.908837]  __do_sys_init_module+0x17f/0x1b0
[    1.908837]  do_syscall_64+0x5c/0x90
[    1.908837]  ? handle_mm_fault+0x11e/0x310
[    1.908837]  ? do_user_addr_fault+0x1e0/0x720
[    1.908837]  ? exc_page_fault+0x74/0x170
[    1.908837]  entry_SYSCALL_64_after_hwframe+0x72/0xdc


Maybe I'll have to install a Windows guest :/

-- 
Cheers,

David / dhildenb