* [virtio-dev] Re: [virtio-comment] Re: About custom device counter
2023-06-15 6:23 ` Xuan Zhuo
@ 2023-06-16 0:57 ` Jason Wang
2023-06-16 1:36 ` Xuan Zhuo
2023-06-16 14:13 ` Michael S. Tsirkin
2023-06-19 14:25 ` Michael S. Tsirkin
2 siblings, 1 reply; 20+ messages in thread
From: Jason Wang @ 2023-06-16 0:57 UTC (permalink / raw)
To: Xuan Zhuo; +Cc: Michael S. Tsirkin, virtio-dev, virtio-comment, Parav Pandit
On Thu, Jun 15, 2023 at 2:34 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
>
> On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > channel for driver and device, and both key and value are provided by device.
> > > > >
> > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > still necessary to discuss this again.
> > > > >
> > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > time.
> > > > >
> > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > why does it affect live migration? These counters we plan to give to users
> > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > may change during the migration process. For example, the driver is migrated
> > > > > to some new devices. These devices support some new counters. I think users
> > > > > should be able to see these new counters. These new counters may be the
> > > > > purpose of the migration.
> > > > >
> > > > > We don't need to support live migration at this point.
> > > > >
> > > > > Thanks.
> > > >
> > > >
> > > > Why not just document the thing?
> > >
> > > Device Counter or Device Stat, we discussed this issue earlier,
> > > So I want to slove the problem migration firstly.
> > >
> > >
> > > Thanks.
> >
> > Sorry have trouble parsing this.
> >
> > Can we get examples of counters from various vendors, so that
> > it's clear what kind of thing this, and it becomes convincing
> > that abstracting this in virtio spec is pointless?
>
> Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> these VMs share the capabilities of the same host. In order to ensure that each
> VM will not affect others, the network card(virtio-net) capability of each VM is
> limited. When these users purchase VMs, this limit has already been determined.
> So if the network card traffic of a vm exceeds the upper limit, packet loss will
> occur. It is necessary for us to count these packet losses. And the device
> should expose to the user.
>
>
> Such as "session", our dpu supports tcp connection tracking, but there is an
> upper limit to the number of connections, and if it exceeds, packet loss will
> also occur.
>
> These are two commonly used scenarios, I hope they can help you.
But you didn't explain why these can't be abstracted in the virtio spec?
Thanks
>
> Thanks.
>
>
> >
> > Also, am I right assuming that in virtualized settings this is mostly
> > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > host wants to figure out source of issues?
> >
> >
> >
> >
> > >
> > > >
> > > > --
> > > > MST
> > > >
> > > >
> > > > This publicly archived list offers a means to provide input to the
> > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > >
> > > > In order to verify user consent to the Feedback License terms and
> > > > to minimize spam in the list archive, subscription is required
> > > > before posting.
> > > >
> > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > Join OASIS: https://www.oasis-open.org/join/
> > > >
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 20+ messages in thread
* [virtio-dev] Re: [virtio-comment] Re: About custom device counter
2023-06-16 0:57 ` Jason Wang
@ 2023-06-16 1:36 ` Xuan Zhuo
2023-06-16 14:42 ` Michael S. Tsirkin
0 siblings, 1 reply; 20+ messages in thread
From: Xuan Zhuo @ 2023-06-16 1:36 UTC (permalink / raw)
To: Jason Wang; +Cc: Michael S. Tsirkin, virtio-dev, virtio-comment, Parav Pandit
On Fri, 16 Jun 2023 08:57:01 +0800, Jason Wang <jasowang@redhat.com> wrote:
> On Thu, Jun 15, 2023 at 2:34 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> >
> > On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > > channel for driver and device, and both key and value are provided by device.
> > > > > >
> > > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > > still necessary to discuss this again.
> > > > > >
> > > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > > time.
> > > > > >
> > > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > > why does it affect live migration? These counters we plan to give to users
> > > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > > may change during the migration process. For example, the driver is migrated
> > > > > > to some new devices. These devices support some new counters. I think users
> > > > > > should be able to see these new counters. These new counters may be the
> > > > > > purpose of the migration.
> > > > > >
> > > > > > We don't need to support live migration at this point.
> > > > > >
> > > > > > Thanks.
> > > > >
> > > > >
> > > > > Why not just document the thing?
> > > >
> > > > Device Counter or Device Stat, we discussed this issue earlier,
> > > > So I want to slove the problem migration firstly.
> > > >
> > > >
> > > > Thanks.
> > >
> > > Sorry have trouble parsing this.
> > >
> > > Can we get examples of counters from various vendors, so that
> > > it's clear what kind of thing this, and it becomes convincing
> > > that abstracting this in virtio spec is pointless?
> >
> > Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> > these VMs share the capabilities of the same host. In order to ensure that each
> > VM will not affect others, the network card(virtio-net) capability of each VM is
> > limited. When these users purchase VMs, this limit has already been determined.
> > So if the network card traffic of a vm exceeds the upper limit, packet loss will
> > occur. It is necessary for us to count these packet losses. And the device
> > should expose to the user.
> >
> >
> > Such as "session", our dpu supports tcp connection tracking, but there is an
> > upper limit to the number of connections, and if it exceeds, packet loss will
> > also occur.
> >
> > These are two commonly used scenarios, I hope they can help you.
>
> But you didn't explain why these can't be abstracted in the virtio spec?
As far as I know, there are no such concepts, and I don't like to force these
concepts to be abstracted.
Instead of forcing abstraction, I think it is more convenient to directly
provide a vendor channel.
Thanks.
>
> Thanks
>
> >
> > Thanks.
> >
> >
> > >
> > > Also, am I right assuming that in virtualized settings this is mostly
> > > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > > host wants to figure out source of issues?
> > >
> > >
> > >
> > >
> > > >
> > > > >
> > > > > --
> > > > > MST
> > > > >
> > > > >
> > > > > This publicly archived list offers a means to provide input to the
> > > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > > >
> > > > > In order to verify user consent to the Feedback License terms and
> > > > > to minimize spam in the list archive, subscription is required
> > > > > before posting.
> > > > >
> > > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > > Join OASIS: https://www.oasis-open.org/join/
> > > > >
> > >
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 20+ messages in thread
* [virtio-dev] Re: [virtio-comment] Re: About custom device counter
2023-06-16 1:36 ` Xuan Zhuo
@ 2023-06-16 14:42 ` Michael S. Tsirkin
0 siblings, 0 replies; 20+ messages in thread
From: Michael S. Tsirkin @ 2023-06-16 14:42 UTC (permalink / raw)
To: Xuan Zhuo; +Cc: Jason Wang, virtio-dev, virtio-comment, Parav Pandit
On Fri, Jun 16, 2023 at 09:36:45AM +0800, Xuan Zhuo wrote:
> On Fri, 16 Jun 2023 08:57:01 +0800, Jason Wang <jasowang@redhat.com> wrote:
> > On Thu, Jun 15, 2023 at 2:34 PM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:
> > >
> > > On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > > > channel for driver and device, and both key and value are provided by device.
> > > > > > >
> > > > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > > > still necessary to discuss this again.
> > > > > > >
> > > > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > > > time.
> > > > > > >
> > > > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > > > why does it affect live migration? These counters we plan to give to users
> > > > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > > > may change during the migration process. For example, the driver is migrated
> > > > > > > to some new devices. These devices support some new counters. I think users
> > > > > > > should be able to see these new counters. These new counters may be the
> > > > > > > purpose of the migration.
> > > > > > >
> > > > > > > We don't need to support live migration at this point.
> > > > > > >
> > > > > > > Thanks.
> > > > > >
> > > > > >
> > > > > > Why not just document the thing?
> > > > >
> > > > > Device Counter or Device Stat, we discussed this issue earlier,
> > > > > So I want to slove the problem migration firstly.
> > > > >
> > > > >
> > > > > Thanks.
> > > >
> > > > Sorry have trouble parsing this.
> > > >
> > > > Can we get examples of counters from various vendors, so that
> > > > it's clear what kind of thing this, and it becomes convincing
> > > > that abstracting this in virtio spec is pointless?
> > >
> > > Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> > > these VMs share the capabilities of the same host. In order to ensure that each
> > > VM will not affect others, the network card(virtio-net) capability of each VM is
> > > limited. When these users purchase VMs, this limit has already been determined.
> > > So if the network card traffic of a vm exceeds the upper limit, packet loss will
> > > occur. It is necessary for us to count these packet losses. And the device
> > > should expose to the user.
> > >
> > >
> > > Such as "session", our dpu supports tcp connection tracking, but there is an
> > > upper limit to the number of connections, and if it exceeds, packet loss will
> > > also occur.
> > >
> > > These are two commonly used scenarios, I hope they can help you.
> >
> > But you didn't explain why these can't be abstracted in the virtio spec?
>
>
> As far as I know, there are no such concepts, and I don't like to force these
> concepts to be abstracted.
>
> Instead of forcing abstraction, I think it is more convenient to directly
> provide a vendor channel.
>
> Thanks.
Bypassing standartiziation always seems convenient short term.
>
> >
> > Thanks
> >
> > >
> > > Thanks.
> > >
> > >
> > > >
> > > > Also, am I right assuming that in virtualized settings this is mostly
> > > > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > > > host wants to figure out source of issues?
> > > >
> > > >
> > > >
> > > >
> > > > >
> > > > > >
> > > > > > --
> > > > > > MST
> > > > > >
> > > > > >
> > > > > > This publicly archived list offers a means to provide input to the
> > > > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > > > >
> > > > > > In order to verify user consent to the Feedback License terms and
> > > > > > to minimize spam in the list archive, subscription is required
> > > > > > before posting.
> > > > > >
> > > > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > > > Join OASIS: https://www.oasis-open.org/join/
> > > > > >
> > > >
> > >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 20+ messages in thread
* [virtio-dev] Re: [virtio-comment] Re: About custom device counter
2023-06-15 6:23 ` Xuan Zhuo
2023-06-16 0:57 ` Jason Wang
@ 2023-06-16 14:13 ` Michael S. Tsirkin
2023-06-19 1:44 ` Xuan Zhuo
2023-06-19 12:33 ` [virtio-dev] " Parav Pandit
2023-06-19 14:25 ` Michael S. Tsirkin
2 siblings, 2 replies; 20+ messages in thread
From: Michael S. Tsirkin @ 2023-06-16 14:13 UTC (permalink / raw)
To: Xuan Zhuo; +Cc: virtio-dev, virtio-comment, Parav Pandit, Jason Wang
On Thu, Jun 15, 2023 at 02:23:31PM +0800, Xuan Zhuo wrote:
> On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > channel for driver and device, and both key and value are provided by device.
> > > > >
> > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > still necessary to discuss this again.
> > > > >
> > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > time.
> > > > >
> > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > why does it affect live migration? These counters we plan to give to users
> > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > may change during the migration process. For example, the driver is migrated
> > > > > to some new devices. These devices support some new counters. I think users
> > > > > should be able to see these new counters. These new counters may be the
> > > > > purpose of the migration.
> > > > >
> > > > > We don't need to support live migration at this point.
> > > > >
> > > > > Thanks.
> > > >
> > > >
> > > > Why not just document the thing?
> > >
> > > Device Counter or Device Stat, we discussed this issue earlier,
> > > So I want to slove the problem migration firstly.
> > >
> > >
> > > Thanks.
> >
> > Sorry have trouble parsing this.
> >
> > Can we get examples of counters from various vendors, so that
> > it's clear what kind of thing this, and it becomes convincing
> > that abstracting this in virtio spec is pointless?
>
> Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> these VMs share the capabilities of the same host. In order to ensure that each
> VM will not affect others, the network card(virtio-net) capability of each VM is
> limited. When these users purchase VMs, this limit has already been determined.
> So if the network card traffic of a vm exceeds the upper limit, packet loss will
> occur. It is necessary for us to count these packet losses. And the device
> should expose to the user.
OK so here, driver is expected to rate limit yes?
> Such as "session", our dpu supports tcp connection tracking, but there is an
> upper limit to the number of connections, and if it exceeds, packet loss will
> also occur.
What can be done here? Just don't run so many apps?
> These are two commonly used scenarios, I hope they can help you.
>
> Thanks.
>
>
> >
> > Also, am I right assuming that in virtualized settings this is mostly
> > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > host wants to figure out source of issues?
> >
> >
> >
> >
> > >
> > > >
> > > > --
> > > > MST
> > > >
> > > >
> > > > This publicly archived list offers a means to provide input to the
> > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > >
> > > > In order to verify user consent to the Feedback License terms and
> > > > to minimize spam in the list archive, subscription is required
> > > > before posting.
> > > >
> > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > Join OASIS: https://www.oasis-open.org/join/
> > > >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 20+ messages in thread
* [virtio-dev] Re: [virtio-comment] Re: About custom device counter
2023-06-16 14:13 ` Michael S. Tsirkin
@ 2023-06-19 1:44 ` Xuan Zhuo
2023-06-19 12:33 ` [virtio-dev] " Parav Pandit
1 sibling, 0 replies; 20+ messages in thread
From: Xuan Zhuo @ 2023-06-19 1:44 UTC (permalink / raw)
To: Michael S. Tsirkin; +Cc: virtio-dev, virtio-comment, Parav Pandit, Jason Wang
On Fri, 16 Jun 2023 10:13:23 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Thu, Jun 15, 2023 at 02:23:31PM +0800, Xuan Zhuo wrote:
> > On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > > channel for driver and device, and both key and value are provided by device.
> > > > > >
> > > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > > still necessary to discuss this again.
> > > > > >
> > > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > > time.
> > > > > >
> > > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > > why does it affect live migration? These counters we plan to give to users
> > > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > > may change during the migration process. For example, the driver is migrated
> > > > > > to some new devices. These devices support some new counters. I think users
> > > > > > should be able to see these new counters. These new counters may be the
> > > > > > purpose of the migration.
> > > > > >
> > > > > > We don't need to support live migration at this point.
> > > > > >
> > > > > > Thanks.
> > > > >
> > > > >
> > > > > Why not just document the thing?
> > > >
> > > > Device Counter or Device Stat, we discussed this issue earlier,
> > > > So I want to slove the problem migration firstly.
> > > >
> > > >
> > > > Thanks.
> > >
> > > Sorry have trouble parsing this.
> > >
> > > Can we get examples of counters from various vendors, so that
> > > it's clear what kind of thing this, and it becomes convincing
> > > that abstracting this in virtio spec is pointless?
> >
> > Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> > these VMs share the capabilities of the same host. In order to ensure that each
> > VM will not affect others, the network card(virtio-net) capability of each VM is
> > limited. When these users purchase VMs, this limit has already been determined.
> > So if the network card traffic of a vm exceeds the upper limit, packet loss will
> > occur. It is necessary for us to count these packet losses. And the device
> > should expose to the user.
>
> OK so here, driver is expected to rate limit yes?
The counter is for the user, not for the driver. Let the user know that some
packets are dropped by rate limit. If they need more traffic, then users may
need to buy a larger network card.
The driver does not know about this.
>
> > Such as "session", our dpu supports tcp connection tracking, but there is an
> > upper limit to the number of connections, and if it exceeds, packet loss will
> > also occur.
>
> What can be done here? Just don't run so many apps?
This function will be used in some special scenarios, I am not particularly
sure. But we do have such a need.
I just gave two examples. In fact, there are many other scenarios. In the cloud
scenario, we still have many strange features for customers. We want to embrace
the virtio, so we're not trying to do something that isn't compliant, but we do
currently have the problem that in our backend, we have a lot of functionality
that can't be exposed to the driver or the guest os by the standard virtio
channels. This actually affects the use of the virtio standard in cloud vendors.
For example, some cloud manufacturers are turning to use custom equipment.
We believe that virtio/virtio-net should also accelerate its evolution and
become more modern, so as to meet the needs of the market.
Thanks.
>
> > These are two commonly used scenarios, I hope they can help you.
> >
> > Thanks.
> >
> >
> > >
> > > Also, am I right assuming that in virtualized settings this is mostly
> > > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > > host wants to figure out source of issues?
> > >
> > >
> > >
> > >
> > > >
> > > > >
> > > > > --
> > > > > MST
> > > > >
> > > > >
> > > > > This publicly archived list offers a means to provide input to the
> > > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > > >
> > > > > In order to verify user consent to the Feedback License terms and
> > > > > to minimize spam in the list archive, subscription is required
> > > > > before posting.
> > > > >
> > > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > > Join OASIS: https://www.oasis-open.org/join/
> > > > >
> > >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 20+ messages in thread
* [virtio-dev] RE: [virtio-comment] Re: About custom device counter
2023-06-16 14:13 ` Michael S. Tsirkin
2023-06-19 1:44 ` Xuan Zhuo
@ 2023-06-19 12:33 ` Parav Pandit
2023-06-19 14:20 ` [virtio-dev] " Michael S. Tsirkin
1 sibling, 1 reply; 20+ messages in thread
From: Parav Pandit @ 2023-06-19 12:33 UTC (permalink / raw)
To: Michael S. Tsirkin, Xuan Zhuo
Cc: virtio-dev@lists.oasis-open.org,
virtio-comment@lists.oasis-open.org, Jason Wang
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Friday, June 16, 2023 10:13 AM
> > Such as "limit", in a cloud scenario, multiple users purchase
> > different VMs, and these VMs share the capabilities of the same host.
> > In order to ensure that each VM will not affect others, the network
> > card(virtio-net) capability of each VM is limited. When these users purchase
> VMs, this limit has already been determined.
> > So if the network card traffic of a vm exceeds the upper limit, packet
> > loss will occur. It is necessary for us to count these packet losses.
> > And the device should expose to the user.
>
> OK so here, driver is expected to rate limit yes?
>
Some implementations of txq are lossy and some are not creating backpressure/flow control to driver so driver can rate limit it naturally.
So a tx packet drop counter is needed to cover the lossy implementations which is abstract enough regardless of reason on why device dropped it.
A more granular counter becomes vendor specific that we can possibly avoid or place under different command.
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 20+ messages in thread
* [virtio-dev] Re: [virtio-comment] Re: About custom device counter
2023-06-19 12:33 ` [virtio-dev] " Parav Pandit
@ 2023-06-19 14:20 ` Michael S. Tsirkin
2023-06-19 20:43 ` [virtio-dev] " Parav Pandit
2023-06-20 2:45 ` [virtio-dev] " Xuan Zhuo
0 siblings, 2 replies; 20+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 14:20 UTC (permalink / raw)
To: Parav Pandit
Cc: Xuan Zhuo, virtio-dev@lists.oasis-open.org,
virtio-comment@lists.oasis-open.org, Jason Wang
On Mon, Jun 19, 2023 at 12:33:33PM +0000, Parav Pandit wrote:
>
>
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Friday, June 16, 2023 10:13 AM
> > > Such as "limit", in a cloud scenario, multiple users purchase
> > > different VMs, and these VMs share the capabilities of the same host.
> > > In order to ensure that each VM will not affect others, the network
> > > card(virtio-net) capability of each VM is limited. When these users purchase
> > VMs, this limit has already been determined.
> > > So if the network card traffic of a vm exceeds the upper limit, packet
> > > loss will occur. It is necessary for us to count these packet losses.
> > > And the device should expose to the user.
> >
> > OK so here, driver is expected to rate limit yes?
> >
> Some implementations of txq are lossy and some are not creating backpressure/flow control to driver so driver can rate limit it naturally.
> So a tx packet drop counter is needed to cover the lossy implementations which is abstract enough regardless of reason on why device dropped it.
Yes but Xuan Zhuo here is saying that he wants a register that is
like link speed, not a counter.
> A more granular counter becomes vendor specific that we can possibly avoid or place under different command.
--
MST
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 20+ messages in thread
* [virtio-dev] RE: [virtio-comment] Re: About custom device counter
2023-06-19 14:20 ` [virtio-dev] " Michael S. Tsirkin
@ 2023-06-19 20:43 ` Parav Pandit
2023-06-20 2:45 ` [virtio-dev] " Xuan Zhuo
1 sibling, 0 replies; 20+ messages in thread
From: Parav Pandit @ 2023-06-19 20:43 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Xuan Zhuo, virtio-dev@lists.oasis-open.org,
virtio-comment@lists.oasis-open.org, Jason Wang
> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Monday, June 19, 2023 10:21 AM
>
> Yes but Xuan Zhuo here is saying that he wants a register that is like link speed,
> not a counter.
>
I understand it as counter because Xuan is saying counter.
He wrote " The counter is for the user, not for the driver".
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 20+ messages in thread
* [virtio-dev] Re: [virtio-comment] Re: About custom device counter
2023-06-19 14:20 ` [virtio-dev] " Michael S. Tsirkin
2023-06-19 20:43 ` [virtio-dev] " Parav Pandit
@ 2023-06-20 2:45 ` Xuan Zhuo
1 sibling, 0 replies; 20+ messages in thread
From: Xuan Zhuo @ 2023-06-20 2:45 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: virtio-dev@lists.oasis-open.org,
virtio-comment@lists.oasis-open.org, Jason Wang, Parav Pandit
On Mon, 19 Jun 2023 10:20:56 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Mon, Jun 19, 2023 at 12:33:33PM +0000, Parav Pandit wrote:
> >
> >
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Friday, June 16, 2023 10:13 AM
> > > > Such as "limit", in a cloud scenario, multiple users purchase
> > > > different VMs, and these VMs share the capabilities of the same host.
> > > > In order to ensure that each VM will not affect others, the network
> > > > card(virtio-net) capability of each VM is limited. When these users purchase
> > > VMs, this limit has already been determined.
> > > > So if the network card traffic of a vm exceeds the upper limit, packet
> > > > loss will occur. It is necessary for us to count these packet losses.
> > > > And the device should expose to the user.
> > >
> > > OK so here, driver is expected to rate limit yes?
> > >
> > Some implementations of txq are lossy and some are not creating backpressure/flow control to driver so driver can rate limit it naturally.
> > So a tx packet drop counter is needed to cover the lossy implementations which is abstract enough regardless of reason on why device dropped it.
>
>
> Yes but Xuan Zhuo here is saying that he wants a register that is
> like link speed, not a counter.
>
A counter.
This is a capability, but it only exists between the device and the user. The
capability is set when the user buys a network card, so it has nothing to do
with the driver. As a result, some packets were lost, and we hope that these
counter can be passed to users. Because this capability has nothing to do
with the virtio spec, we always want it to be a vendor custom counter.
Of course, some parameters can be abstracted, and we are very happy to do so.
But I am worried that this will lead to inaccurate expressions of meaning.
Anyway we can try something out, I'll sort out some of the counters we'd like to
have. There are also some that I think we want to customize. We discuss further
based on these counters.
Thanks.
>
> > A more granular counter becomes vendor specific that we can possibly avoid or place under different command.
>
>
> --
> MST
>
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 20+ messages in thread
* [virtio-dev] Re: [virtio-comment] Re: About custom device counter
2023-06-15 6:23 ` Xuan Zhuo
2023-06-16 0:57 ` Jason Wang
2023-06-16 14:13 ` Michael S. Tsirkin
@ 2023-06-19 14:25 ` Michael S. Tsirkin
2023-06-20 2:55 ` Xuan Zhuo
2 siblings, 1 reply; 20+ messages in thread
From: Michael S. Tsirkin @ 2023-06-19 14:25 UTC (permalink / raw)
To: Xuan Zhuo; +Cc: virtio-dev, virtio-comment, Parav Pandit, Jason Wang
On Thu, Jun 15, 2023 at 02:23:31PM +0800, Xuan Zhuo wrote:
> On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > channel for driver and device, and both key and value are provided by device.
> > > > >
> > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > still necessary to discuss this again.
> > > > >
> > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > time.
> > > > >
> > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > why does it affect live migration? These counters we plan to give to users
> > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > may change during the migration process. For example, the driver is migrated
> > > > > to some new devices. These devices support some new counters. I think users
> > > > > should be able to see these new counters. These new counters may be the
> > > > > purpose of the migration.
> > > > >
> > > > > We don't need to support live migration at this point.
> > > > >
> > > > > Thanks.
> > > >
> > > >
> > > > Why not just document the thing?
> > >
> > > Device Counter or Device Stat, we discussed this issue earlier,
> > > So I want to slove the problem migration firstly.
> > >
> > >
> > > Thanks.
> >
> > Sorry have trouble parsing this.
> >
> > Can we get examples of counters from various vendors, so that
> > it's clear what kind of thing this, and it becomes convincing
> > that abstracting this in virtio spec is pointless?
>
> Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> these VMs share the capabilities of the same host. In order to ensure that each
> VM will not affect others, the network card(virtio-net) capability of each VM is
> limited. When these users purchase VMs, this limit has already been determined.
> So if the network card traffic of a vm exceeds the upper limit, packet loss will
> occur. It is necessary for us to count these packet losses. And the device
> should expose to the user.
>
>
> Such as "session", our dpu supports tcp connection tracking, but there is an
> upper limit to the number of connections, and if it exceeds, packet loss will
> also occur.
>
> These are two commonly used scenarios, I hope they can help you.
>
> Thanks.
So not a counter at all really. More a capability.
My rule of thumb is that if something can be relevant to multiple
vendors then it should not be a vendor specific capaiblity.
What you have described can easily apply to a software
virtio imlementation, so it's relevant for multiple
vendors.
So please try to add this to the spec as a generic capability.
Yes adding vendor specific extensions is of course less work
for vendors. But the ecosystem is only strong if everyone is
working on the same spec and codebase.
>
> >
> > Also, am I right assuming that in virtualized settings this is mostly
> > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > host wants to figure out source of issues?
> >
> >
> >
> >
> > >
> > > >
> > > > --
> > > > MST
> > > >
> > > >
> > > > This publicly archived list offers a means to provide input to the
> > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > >
> > > > In order to verify user consent to the Feedback License terms and
> > > > to minimize spam in the list archive, subscription is required
> > > > before posting.
> > > >
> > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > Join OASIS: https://www.oasis-open.org/join/
> > > >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 20+ messages in thread
* [virtio-dev] Re: [virtio-comment] Re: About custom device counter
2023-06-19 14:25 ` Michael S. Tsirkin
@ 2023-06-20 2:55 ` Xuan Zhuo
0 siblings, 0 replies; 20+ messages in thread
From: Xuan Zhuo @ 2023-06-20 2:55 UTC (permalink / raw)
To: Michael S. Tsirkin; +Cc: virtio-dev, virtio-comment, Parav Pandit, Jason Wang
On Mon, 19 Jun 2023 10:25:38 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> On Thu, Jun 15, 2023 at 02:23:31PM +0800, Xuan Zhuo wrote:
> > On Tue, 13 Jun 2023 05:02:01 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > On Mon, Jun 12, 2023 at 07:27:15PM +0800, Xuan Zhuo wrote:
> > > > On Mon, 12 Jun 2023 02:25:42 -0400, "Michael S. Tsirkin" <mst@redhat.com> wrote:
> > > > > On Mon, Jun 12, 2023 at 02:12:35PM +0800, Xuan Zhuo wrote:
> > > > > > We hope to support device custom counter. That is, virtio spec provides a
> > > > > > channel for driver and device, and both key and value are provided by device.
> > > > > >
> > > > > > We discussed this issue earlier, and after some internal practice, I think it is
> > > > > > still necessary to discuss this again.
> > > > > >
> > > > > > It is very important, each cloud vendor will always have some special counters,
> > > > > > these counters may not exist in another vendor. At the same time, if we have
> > > > > > to discuss it in the spec every time we add a counter, or add a feature, I
> > > > > > think it is very inconvenient. Manufacturers may add some new counters at any
> > > > > > time based on some new requirements. Some counters may also be removed at any
> > > > > > time.
> > > > > >
> > > > > > Of course I know that doing this might hurt migration. But what I want to say is,
> > > > > > why does it affect live migration? These counters we plan to give to users
> > > > > > through ethtool -S, and some changes have taken place in the ethtool counters
> > > > > > output by users. Does this have any practical impact? Or do we directly use
> > > > > > some other output in a way, we can clearly tell the user that these counters
> > > > > > may change during the migration process. For example, the driver is migrated
> > > > > > to some new devices. These devices support some new counters. I think users
> > > > > > should be able to see these new counters. These new counters may be the
> > > > > > purpose of the migration.
> > > > > >
> > > > > > We don't need to support live migration at this point.
> > > > > >
> > > > > > Thanks.
> > > > >
> > > > >
> > > > > Why not just document the thing?
> > > >
> > > > Device Counter or Device Stat, we discussed this issue earlier,
> > > > So I want to slove the problem migration firstly.
> > > >
> > > >
> > > > Thanks.
> > >
> > > Sorry have trouble parsing this.
> > >
> > > Can we get examples of counters from various vendors, so that
> > > it's clear what kind of thing this, and it becomes convincing
> > > that abstracting this in virtio spec is pointless?
> >
> > Such as "limit", in a cloud scenario, multiple users purchase different VMs, and
> > these VMs share the capabilities of the same host. In order to ensure that each
> > VM will not affect others, the network card(virtio-net) capability of each VM is
> > limited. When these users purchase VMs, this limit has already been determined.
> > So if the network card traffic of a vm exceeds the upper limit, packet loss will
> > occur. It is necessary for us to count these packet losses. And the device
> > should expose to the user.
> >
> >
> > Such as "session", our dpu supports tcp connection tracking, but there is an
> > upper limit to the number of connections, and if it exceeds, packet loss will
> > also occur.
> >
> > These are two commonly used scenarios, I hope they can help you.
> >
> > Thanks.
>
>
> So not a counter at all really. More a capability.
Explain in other mail.
>
> My rule of thumb is that if something can be relevant to multiple
> vendors then it should not be a vendor specific capaiblity.
> What you have described can easily apply to a software
> virtio imlementation, so it's relevant for multiple
> vendors.
Great!!
>
> So please try to add this to the spec as a generic capability.
> Yes adding vendor specific extensions is of course less work
> for vendors. But the ecosystem is only strong if everyone is
> working on the same spec and codebase.
I agree.
Thank.
>
> >
> > >
> > > Also, am I right assuming that in virtualized settings this is mostly
> > > relevant for hosts? E.g. guests see a vendor neutral virtio but
> > > host wants to figure out source of issues?
> > >
> > >
> > >
> > >
> > > >
> > > > >
> > > > > --
> > > > > MST
> > > > >
> > > > >
> > > > > This publicly archived list offers a means to provide input to the
> > > > > OASIS Virtual I/O Device (VIRTIO) TC.
> > > > >
> > > > > In order to verify user consent to the Feedback License terms and
> > > > > to minimize spam in the list archive, subscription is required
> > > > > before posting.
> > > > >
> > > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > > > List help: virtio-comment-help@lists.oasis-open.org
> > > > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > > > Committee: https://www.oasis-open.org/committees/virtio/
> > > > > Join OASIS: https://www.oasis-open.org/join/
> > > > >
> > >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
^ permalink raw reply [flat|nested] 20+ messages in thread