All of lore.kernel.org
 help / color / mirror / Atom feed
From: Marc Zyngier <maz@kernel.org>
To: Peter Xu <peterx@redhat.com>
Cc: kvm@vger.kernel.org, catalin.marinas@arm.com,
	andrew.jones@linux.dev, dmatlack@google.com, will@kernel.org,
	shan.gavin@gmail.com, bgardon@google.com, kvmarm@lists.linux.dev,
	pbonzini@redhat.com, zhenyzha@redhat.com, shuah@kernel.org,
	kvmarm@lists.cs.columbia.edu, ajones@ventanamicro.com
Subject: Re: [PATCH v8 3/7] KVM: Support dirty ring in conjunction with bitmap
Date: Mon, 07 Nov 2022 15:30:59 +0000	[thread overview]
Message-ID: <86zgd2pyrw.wl-maz@kernel.org> (raw)
In-Reply-To: <Y2kdXTn6X3O08sFv@x1n>

On Mon, 07 Nov 2022 14:59:41 +0000,
Peter Xu <peterx@redhat.com> wrote:
> 
> On Mon, Nov 07, 2022 at 09:21:35AM +0000, Marc Zyngier wrote:
> > On Sun, 06 Nov 2022 21:06:43 +0000,
> > Peter Xu <peterx@redhat.com> wrote:
> > > 
> > > It's definitely not the case for x86, but if it's true for
> > > arm64, then could the DMA be spread across all the guest pages?
> > > If it's also true, I really don't know how this will work..
> > 
> > Of course, all pages can be the target of DMA. It works the same way
> > it works for the ITS: you sync the state, you obtain the dirty bits,
> > you move on.
> > 
> > And mimicking what x86 does is really not my concern (if you still
> > think that arm64 is just another flavour of x86, stay tuned!  ;-).
> 
> I didn't mean so, I should probably stop mentioning x86. :)

Please! I turned off my last x86 development machine over the weekend,
and my x86 laptop is now a glorified window manager... ;-)

> I had some sense already from the topics in past few years of kvm forum.
> Yeah I'll be looking forward to anything more coming.

Yup. Hopefully we won't have to wait for too long to see this stuff (I
had good discussions on the subject at both KF and Plumbers in Dublin
earlier this year).

> > > We're only syncing the dirty bitmap once right now with the
> > > protocol.  If that can cover most of the guest mem, it's same as
> > > non-live.  If we sync it periodically, then it's the same as
> > > enabling dirty-log alone and the rings are useless.
> > 
> > I'm glad that you finally accept it: the ring *ARE* useless in the
> > general sense. Only limited, CPU-only workloads can make any use of
> > the current design. This probably covers a large proportion of what
> > the cloud vendors do, but this doesn't work for general situations
> > where you have a stream of dirty pages originating outside of the
> > CPUs.
> 
> The ring itself is really not the thing to blame, IMHO it's a good attempt
> to try out de-coupling guest size in regard of dirty tracking from kvm.  It
> may not be perfect, but it may still service some of the goals, e.g., at
> least it allows the user app to detect per-vcpu information and also since
> there's the ring full events we can do something more than before like the
> vcpu throttling that China Telecom does with the ring structures.

I don't disagree with that: for vcpu-based workloads, the rings are
great and doing their job. It's just that there is another side to
this problem, and you'll have to deal with both eventually. We're just
ahead of the curve here...

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

WARNING: multiple messages have this Message-ID (diff)
From: Marc Zyngier <maz@kernel.org>
To: Peter Xu <peterx@redhat.com>
Cc: Gavin Shan <gshan@redhat.com>,
	kvmarm@lists.linux.dev, kvmarm@lists.cs.columbia.edu,
	kvm@vger.kernel.org, shuah@kernel.org, catalin.marinas@arm.com,
	andrew.jones@linux.dev, ajones@ventanamicro.com,
	bgardon@google.com, dmatlack@google.com, will@kernel.org,
	suzuki.poulose@arm.com, alexandru.elisei@arm.com,
	pbonzini@redhat.com, seanjc@google.com, oliver.upton@linux.dev,
	zhenyzha@redhat.com, shan.gavin@gmail.com
Subject: Re: [PATCH v8 3/7] KVM: Support dirty ring in conjunction with bitmap
Date: Mon, 07 Nov 2022 15:30:59 +0000	[thread overview]
Message-ID: <86zgd2pyrw.wl-maz@kernel.org> (raw)
Message-ID: <20221107153059.cm7q0dLCLP0jSj1uEm1UPs-3vlCCqfF3Bzz-_FP58Ts@z> (raw)
In-Reply-To: <Y2kdXTn6X3O08sFv@x1n>

On Mon, 07 Nov 2022 14:59:41 +0000,
Peter Xu <peterx@redhat.com> wrote:
> 
> On Mon, Nov 07, 2022 at 09:21:35AM +0000, Marc Zyngier wrote:
> > On Sun, 06 Nov 2022 21:06:43 +0000,
> > Peter Xu <peterx@redhat.com> wrote:
> > > 
> > > It's definitely not the case for x86, but if it's true for
> > > arm64, then could the DMA be spread across all the guest pages?
> > > If it's also true, I really don't know how this will work..
> > 
> > Of course, all pages can be the target of DMA. It works the same way
> > it works for the ITS: you sync the state, you obtain the dirty bits,
> > you move on.
> > 
> > And mimicking what x86 does is really not my concern (if you still
> > think that arm64 is just another flavour of x86, stay tuned!  ;-).
> 
> I didn't mean so, I should probably stop mentioning x86. :)

Please! I turned off my last x86 development machine over the weekend,
and my x86 laptop is now a glorified window manager... ;-)

> I had some sense already from the topics in past few years of kvm forum.
> Yeah I'll be looking forward to anything more coming.

Yup. Hopefully we won't have to wait for too long to see this stuff (I
had good discussions on the subject at both KF and Plumbers in Dublin
earlier this year).

> > > We're only syncing the dirty bitmap once right now with the
> > > protocol.  If that can cover most of the guest mem, it's same as
> > > non-live.  If we sync it periodically, then it's the same as
> > > enabling dirty-log alone and the rings are useless.
> > 
> > I'm glad that you finally accept it: the ring *ARE* useless in the
> > general sense. Only limited, CPU-only workloads can make any use of
> > the current design. This probably covers a large proportion of what
> > the cloud vendors do, but this doesn't work for general situations
> > where you have a stream of dirty pages originating outside of the
> > CPUs.
> 
> The ring itself is really not the thing to blame, IMHO it's a good attempt
> to try out de-coupling guest size in regard of dirty tracking from kvm.  It
> may not be perfect, but it may still service some of the goals, e.g., at
> least it allows the user app to detect per-vcpu information and also since
> there's the ring full events we can do something more than before like the
> vcpu throttling that China Telecom does with the ring structures.

I don't disagree with that: for vcpu-based workloads, the rings are
great and doing their job. It's just that there is another side to
this problem, and you'll have to deal with both eventually. We're just
ahead of the curve here...

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

  reply	other threads:[~2022-11-07 15:31 UTC|newest]

Thread overview: 66+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-11-04 23:40 [PATCH v8 0/7] KVM: arm64: Enable ring-based dirty memory tracking Gavin Shan
2022-11-04 23:40 ` Gavin Shan
2022-11-04 23:40 ` [PATCH v8 1/7] KVM: x86: Introduce KVM_REQ_DIRTY_RING_SOFT_FULL Gavin Shan
2022-11-04 23:40   ` Gavin Shan
2022-11-04 23:40 ` [PATCH v8 2/7] KVM: Move declaration of kvm_cpu_dirty_log_size() to kvm_dirty_ring.h Gavin Shan
2022-11-04 23:40   ` Gavin Shan
2022-11-04 23:40 ` [PATCH v8 3/7] KVM: Support dirty ring in conjunction with bitmap Gavin Shan
2022-11-04 23:40   ` Gavin Shan
2022-11-06 15:43   ` Marc Zyngier
2022-11-06 15:43     ` Marc Zyngier
2022-11-06 16:22     ` Peter Xu
2022-11-06 16:22       ` Peter Xu
2022-11-06 20:12       ` Marc Zyngier
2022-11-06 20:12         ` Marc Zyngier
2022-11-06 21:06         ` Peter Xu
2022-11-06 21:06           ` Peter Xu
2022-11-06 21:23           ` Gavin Shan
2022-11-06 21:23             ` Gavin Shan
2022-11-07  9:38             ` Marc Zyngier
2022-11-07  9:38               ` Marc Zyngier
2022-11-07 14:29               ` Peter Xu
2022-11-07 14:29                 ` Peter Xu
2022-11-07  9:21           ` Marc Zyngier
2022-11-07  9:21             ` Marc Zyngier
2022-11-07 14:59             ` Peter Xu
2022-11-07 14:59               ` Peter Xu
2022-11-07 15:30               ` Marc Zyngier [this message]
2022-11-07 15:30                 ` Marc Zyngier
2022-11-06 21:40     ` Gavin Shan
2022-11-06 21:40       ` Gavin Shan
2022-11-07  9:45       ` Marc Zyngier
2022-11-07  9:45         ` Marc Zyngier
2022-11-07 10:45   ` Gavin Shan
2022-11-07 10:45     ` Gavin Shan
2022-11-07 11:33     ` Marc Zyngier
2022-11-07 11:33       ` Marc Zyngier
2022-11-07 23:53       ` Gavin Shan
2022-11-07 23:53         ` Gavin Shan
2022-11-07 16:05   ` Sean Christopherson
2022-11-07 16:05     ` Sean Christopherson
2022-11-08  0:44     ` Gavin Shan
2022-11-08  0:44       ` Gavin Shan
2022-11-08  1:13       ` Oliver Upton
2022-11-08  1:13         ` Oliver Upton
2022-11-08  3:30         ` Gavin Shan
2022-11-08  3:30           ` Gavin Shan
2022-11-04 23:40 ` [PATCH v8 4/7] KVM: arm64: Enable ring-based dirty memory tracking Gavin Shan
2022-11-04 23:40   ` Gavin Shan
2022-11-06 15:50   ` Marc Zyngier
2022-11-06 15:50     ` Marc Zyngier
2022-11-06 21:46     ` Gavin Shan
2022-11-06 21:46       ` Gavin Shan
2022-11-07  9:47       ` Marc Zyngier
2022-11-07  9:47         ` Marc Zyngier
2022-11-07 10:47         ` Gavin Shan
2022-11-07 10:47           ` Gavin Shan
2022-11-04 23:40 ` [PATCH v8 5/7] KVM: selftests: Use host page size to map ring buffer in dirty_log_test Gavin Shan
2022-11-04 23:40   ` Gavin Shan
2022-11-04 23:40 ` [PATCH v8 6/7] KVM: selftests: Clear dirty ring states between two modes " Gavin Shan
2022-11-04 23:40   ` Gavin Shan
2022-11-04 23:40 ` [PATCH v8 7/7] KVM: selftests: Automate choosing dirty ring size " Gavin Shan
2022-11-04 23:40   ` Gavin Shan
2022-11-06 16:08 ` [PATCH v8 0/7] KVM: arm64: Enable ring-based dirty memory tracking Marc Zyngier
2022-11-06 16:08   ` Marc Zyngier
2022-11-06 21:50   ` Gavin Shan
2022-11-06 21:50     ` Gavin Shan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=86zgd2pyrw.wl-maz@kernel.org \
    --to=maz@kernel.org \
    --cc=ajones@ventanamicro.com \
    --cc=andrew.jones@linux.dev \
    --cc=bgardon@google.com \
    --cc=catalin.marinas@arm.com \
    --cc=dmatlack@google.com \
    --cc=kvm@vger.kernel.org \
    --cc=kvmarm@lists.cs.columbia.edu \
    --cc=kvmarm@lists.linux.dev \
    --cc=pbonzini@redhat.com \
    --cc=peterx@redhat.com \
    --cc=shan.gavin@gmail.com \
    --cc=shuah@kernel.org \
    --cc=will@kernel.org \
    --cc=zhenyzha@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.