[PATCH] target/i386/kvm: Fix Live Migration failure on Intel Ice Lake after OS upgrade (Ubuntu 20.04 to 22.04)

[PATCH] target/i386/kvm: Fix Live Migration failure on Intel Ice Lake after OS upgrade (Ubuntu 20.04 to 22.04)

[fatih şener]

[-- Attachment #1: Type: text/plain, Size: 1734 bytes --]

Dear QEMU Developers,

We have identified a critical live migration failure on Intel Ice Lake
platforms when using *CPU host-passthrough* in OpenStack environments. The
issue specifically surfaced after upgrading compute nodes from *Ubuntu
20.04 to 22.04*.

*Technical Analysis:* The migration fails with KVM_SET_MSRS error while
restoring MSR_IA32_PERF_GLOBAL_CTRL (0x38f). On Ice Lake CPUs, this
register reports 0xf000000ff due to the 4th fixed-function counter (Perf
Mon v5).

We observed that while the source host (20.04) allows this value, the
target host (22.04 with a newer Kernel/KVM) enforces a stricter mask of
0x7000000ff. Even with identical hardware, the migration is rejected
because the target KVM treats the 4th bit as reserved/invalid.

*Proposed Fix:* We propose intercepting the MSR write in
target/i386/kvm/kvm.c to mask the value back to 0x7000000ff if 0xf000000ff
is detected. This ensures backward compatibility across different kernel
versions and stabilizes migration on Ice Lake hardware.

/* target/i386/kvm/kvm.c */
for (i = 0; i < cpu->kvm_msr_buf->nmsrs; i++) {
    if (cpu->kvm_msr_buf->entries[i].index == 0x38f &&
        cpu->kvm_msr_buf->entries[i].data == 0xf000000ffULL) {
        cpu->kvm_msr_buf->entries[i].data = 0x7000000ffULL;
    }
}

We have verified that this fix stabilizes our production environment and
allows successful live migration between hosts with different kernel
versions on Ice Lake hardware. We are looking forward to your technical
feedback on this implementation. If there is a more architectural way to
handle these MSR bitmasks upstream, we would be happy to discuss it. Best
regards,
-- 
● Fatih Şener - fatihsenerr@gmail.com

[-- Attachment #2: Type: text/html, Size: 2140 bytes --]

Re: [PATCH] target/i386/kvm: Fix Live Migration failure on Intel Ice Lake after OS upgrade (Ubuntu 20.04 to 22.04)

[Daniel P. Berrangé]

On Tue, May 12, 2026 at 11:02:37AM +0300, fatih şener wrote:
> Dear QEMU Developers,
> 
> We have identified a critical live migration failure on Intel Ice Lake
> platforms when using *CPU host-passthrough* in OpenStack environments. The
> issue specifically surfaced after upgrading compute nodes from *Ubuntu
> 20.04 to 22.04*.

IMHO that is not a valid usage scenario.

host-passthrough is only supportable with live migration if both sides
of the migration are *identical*. That means same CPU models, same
microcode version, same BIOS version, same BIOS settings, same kernel
version, same QEMU version. Vary any one of those factors as it looses
any guarantee of live migration safety for host-passthrough.

If those requirements can't be guaranteed, then we have a wide choice
of named CPU models that support migration across hosts with different
setups.

> *Technical Analysis:* The migration fails with KVM_SET_MSRS error while
> restoring MSR_IA32_PERF_GLOBAL_CTRL (0x38f). On Ice Lake CPUs, this
> register reports 0xf000000ff due to the 4th fixed-function counter (Perf
> Mon v5).
> 
> We observed that while the source host (20.04) allows this value, the
> target host (22.04 with a newer Kernel/KVM) enforces a stricter mask of
> 0x7000000ff. Even with identical hardware, the migration is rejected
> because the target KVM treats the 4th bit as reserved/invalid.
> 
> *Proposed Fix:* We propose intercepting the MSR write in
> target/i386/kvm/kvm.c to mask the value back to 0x7000000ff if 0xf000000ff
> is detected. This ensures backward compatibility across different kernel
> versions and stabilizes migration on Ice Lake hardware.
> 
> /* target/i386/kvm/kvm.c */
> for (i = 0; i < cpu->kvm_msr_buf->nmsrs; i++) {
>     if (cpu->kvm_msr_buf->entries[i].index == 0x38f &&
>         cpu->kvm_msr_buf->entries[i].data == 0xf000000ffULL) {
>         cpu->kvm_msr_buf->entries[i].data = 0x7000000ffULL;
>     }
> }
> 
> We have verified that this fix stabilizes our production environment and
> allows successful live migration between hosts with different kernel
> versions on Ice Lake hardware. We are looking forward to your technical
> feedback on this implementation. If there is a more architectural way to
> handle these MSR bitmasks upstream, we would be happy to discuss it. Best
> regards,
> -- 
> ● Fatih Şener - fatihsenerr@gmail.com

With regards,
Daniel
-- 
|: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
|: https://libvirt.org          ~~          https://entangle-photo.org :|
|: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|

Re: [PATCH] target/i386/kvm: Fix Live Migration failure on Intel Ice Lake after OS upgrade (Ubuntu 20.04 to 22.04)

[fatih şener]

[-- Attachment #1: Type: text/plain, Size: 3400 bytes --]

Thank you for the clarification and technical insight.
We understand that this migration scenario is not considered fully
supported with host-passthrough across differing kernel/QEMU environments.
The environment was originally designed around maximum CPU feature exposure
and performance requirements.
We are currently evaluating moving away from host-passthrough for future
upgrade compatibility.

Thank you again for your response.

Best regards,
Fatih

Daniel P. Berrangé <berrange@redhat.com>, 12 May 2026 Sal, 20:33 tarihinde
şunu yazdı:

> On Tue, May 12, 2026 at 11:02:37AM +0300, fatih şener wrote:
> > Dear QEMU Developers,
> >
> > We have identified a critical live migration failure on Intel Ice Lake
> > platforms when using *CPU host-passthrough* in OpenStack environments.
> The
> > issue specifically surfaced after upgrading compute nodes from *Ubuntu
> > 20.04 to 22.04*.
>
> IMHO that is not a valid usage scenario.
>
> host-passthrough is only supportable with live migration if both sides
> of the migration are *identical*. That means same CPU models, same
> microcode version, same BIOS version, same BIOS settings, same kernel
> version, same QEMU version. Vary any one of those factors as it looses
> any guarantee of live migration safety for host-passthrough.
>
> If those requirements can't be guaranteed, then we have a wide choice
> of named CPU models that support migration across hosts with different
> setups.
>
> > *Technical Analysis:* The migration fails with KVM_SET_MSRS error while
> > restoring MSR_IA32_PERF_GLOBAL_CTRL (0x38f). On Ice Lake CPUs, this
> > register reports 0xf000000ff due to the 4th fixed-function counter (Perf
> > Mon v5).
> >
> > We observed that while the source host (20.04) allows this value, the
> > target host (22.04 with a newer Kernel/KVM) enforces a stricter mask of
> > 0x7000000ff. Even with identical hardware, the migration is rejected
> > because the target KVM treats the 4th bit as reserved/invalid.
> >
> > *Proposed Fix:* We propose intercepting the MSR write in
> > target/i386/kvm/kvm.c to mask the value back to 0x7000000ff if
> 0xf000000ff
> > is detected. This ensures backward compatibility across different kernel
> > versions and stabilizes migration on Ice Lake hardware.
> >
> > /* target/i386/kvm/kvm.c */
> > for (i = 0; i < cpu->kvm_msr_buf->nmsrs; i++) {
> >     if (cpu->kvm_msr_buf->entries[i].index == 0x38f &&
> >         cpu->kvm_msr_buf->entries[i].data == 0xf000000ffULL) {
> >         cpu->kvm_msr_buf->entries[i].data = 0x7000000ffULL;
> >     }
> > }
> >
> > We have verified that this fix stabilizes our production environment and
> > allows successful live migration between hosts with different kernel
> > versions on Ice Lake hardware. We are looking forward to your technical
> > feedback on this implementation. If there is a more architectural way to
> > handle these MSR bitmasks upstream, we would be happy to discuss it. Best
> > regards,
> > --
> > ● Fatih Şener - fatihsenerr@gmail.com
>
> With regards,
> Daniel
> --
> |: https://berrange.com       ~~        https://hachyderm.io/@berrange :|
> |: https://libvirt.org          ~~          https://entangle-photo.org :|
> |: https://pixelfed.art/berrange   ~~    https://fstop138.berrange.com :|
>
>

-- 
● Fatih Şener - fatihsenerr@gmail.com

[-- Attachment #2: Type: text/html, Size: 4770 bytes --]