All of lore.kernel.org
 help / color / mirror / Atom feed
From: John Fastabend <john.fastabend@gmail.com>
To: intel-wired-lan@osuosl.org
Subject: [Intel-wired-lan] [PATCH] ixgbe: take online CPU number as MQ max limit when alloc_etherdev_mq()
Date: Mon, 16 May 2016 10:14:04 -0700	[thread overview]
Message-ID: <5739FFDC.80908@gmail.com> (raw)
In-Reply-To: <CAKgT0UfJqu5sa7U4HKLFk0h3NjqavaAY0xfsahZ32MvnePFHdg@mail.gmail.com>

[...]

>>> ixgbe_main.c.  All you are doing with this patch is denying the user
>>> choice with this change as they then are not allowed to set more
>>
>>   Yes, it is purposed to deny configuration that doesn't benefit.
> 
> Doesn't benefit who?  It is obvious you don't understand how DCB is
> meant to work since you are assuming the queues are throw-away.
> Anyone who makes use of the ability to prioritize their traffic would
> likely have a different opinion.


+1 this is actually needed so that when DCB is turned on we can
see both prioritize between tcs (DCB feature) but also do not
see a performance degradation with just a single TC transmitting.

If we break this (and its happened occasionally) we end up with
bug reports so its clear to me folks care about it.

> 
>>> queues.  Even if they find your decision was wrong for their
>>> configuration.
>>>
>>> - Alex
>>>
>>  Thanks,
>>  Ethan
> 
> Your response clearly points out you don't understand DCB.  I suggest
> you take another look at how things are actually being configured.  I
> believe what you will find is that the current implementation is
> basing things on the number of online CPUs already based on the
> ring_feature[RING_F_RSS].limit value.  All that is happening is that
> you are getting that value multiplied by the number of TCs and the RSS
> value is reduced if the result is greater than 64 based on the maximum
> number of queues.
> 
> With your code on an 8 core system you go from being able to perform
> RSS over 8 queues to only being able to perform RSS over 1 queue when
> you enable DCB.  There was a bug a long time ago where this actually
> didn't provide any gain because the interrupt allocation was binding
> all 8 RSS queues to a single q_vector, but that has long since been
> fixed and what you should be seeing is that RSS will spread traffic
> across either 8 or 16 queues when DCB is enabled in either 8 or 4 TC
> mode.
> 
> My advice would be to use a netperf TCP_CRR test and watch what queues
> and what interrupts traffic is being delivered to.  Then if you have
> DCB enabled on both ends you might try changing the priority of your
> netperf session and watch what happens when you switch between TCs.
> What you should find is that you will shift between groups of queues
> and as you do so you should not have any active queues overlapping
> unless you have less interrupts than CPUs.
> 

Yep.

Thanks,
John

> - Alex
> _______________________________________________
> Intel-wired-lan mailing list
> Intel-wired-lan at lists.osuosl.org
> http://lists.osuosl.org/mailman/listinfo/intel-wired-lan
> 


WARNING: multiple messages have this Message-ID (diff)
From: John Fastabend <john.fastabend@gmail.com>
To: Alexander Duyck <alexander.duyck@gmail.com>,
	ethan zhao <ethan.zhao@oracle.com>
Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	intel-wired-lan <intel-wired-lan@lists.osuosl.org>,
	Netdev <netdev@vger.kernel.org>,
	Ethan Zhao <ethan.kernel@gmail.com>
Subject: Re: [Intel-wired-lan] [PATCH] ixgbe: take online CPU number as MQ max limit when alloc_etherdev_mq()
Date: Mon, 16 May 2016 10:14:04 -0700	[thread overview]
Message-ID: <5739FFDC.80908@gmail.com> (raw)
In-Reply-To: <CAKgT0UfJqu5sa7U4HKLFk0h3NjqavaAY0xfsahZ32MvnePFHdg@mail.gmail.com>

[...]

>>> ixgbe_main.c.  All you are doing with this patch is denying the user
>>> choice with this change as they then are not allowed to set more
>>
>>   Yes, it is purposed to deny configuration that doesn't benefit.
> 
> Doesn't benefit who?  It is obvious you don't understand how DCB is
> meant to work since you are assuming the queues are throw-away.
> Anyone who makes use of the ability to prioritize their traffic would
> likely have a different opinion.


+1 this is actually needed so that when DCB is turned on we can
see both prioritize between tcs (DCB feature) but also do not
see a performance degradation with just a single TC transmitting.

If we break this (and its happened occasionally) we end up with
bug reports so its clear to me folks care about it.

> 
>>> queues.  Even if they find your decision was wrong for their
>>> configuration.
>>>
>>> - Alex
>>>
>>  Thanks,
>>  Ethan
> 
> Your response clearly points out you don't understand DCB.  I suggest
> you take another look at how things are actually being configured.  I
> believe what you will find is that the current implementation is
> basing things on the number of online CPUs already based on the
> ring_feature[RING_F_RSS].limit value.  All that is happening is that
> you are getting that value multiplied by the number of TCs and the RSS
> value is reduced if the result is greater than 64 based on the maximum
> number of queues.
> 
> With your code on an 8 core system you go from being able to perform
> RSS over 8 queues to only being able to perform RSS over 1 queue when
> you enable DCB.  There was a bug a long time ago where this actually
> didn't provide any gain because the interrupt allocation was binding
> all 8 RSS queues to a single q_vector, but that has long since been
> fixed and what you should be seeing is that RSS will spread traffic
> across either 8 or 16 queues when DCB is enabled in either 8 or 4 TC
> mode.
> 
> My advice would be to use a netperf TCP_CRR test and watch what queues
> and what interrupts traffic is being delivered to.  Then if you have
> DCB enabled on both ends you might try changing the priority of your
> netperf session and watch what happens when you switch between TCs.
> What you should find is that you will shift between groups of queues
> and as you do so you should not have any active queues overlapping
> unless you have less interrupts than CPUs.
> 

Yep.

Thanks,
John

> - Alex
> _______________________________________________
> Intel-wired-lan mailing list
> Intel-wired-lan@lists.osuosl.org
> http://lists.osuosl.org/mailman/listinfo/intel-wired-lan
> 

  reply	other threads:[~2016-05-16 17:14 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-05-13  5:56 [Intel-wired-lan] [PATCH] ixgbe: take online CPU number as MQ max limit when alloc_etherdev_mq() Ethan Zhao
2016-05-13  5:56 ` Ethan Zhao
2016-05-13 12:52 ` [Intel-wired-lan] " Sergei Shtylyov
2016-05-13 12:52   ` Sergei Shtylyov
2016-05-16  5:38   ` [Intel-wired-lan] " ethan zhao
2016-05-16  5:38     ` ethan zhao
2016-05-13 16:46 ` [Intel-wired-lan] " Alexander Duyck
2016-05-13 16:46   ` Alexander Duyck
2016-05-16  2:59   ` [Intel-wired-lan] " ethan zhao
2016-05-16  2:59     ` ethan zhao
2016-05-16 16:09     ` [Intel-wired-lan] " Alexander Duyck
2016-05-16 16:09       ` Alexander Duyck
2016-05-16 17:14       ` John Fastabend [this message]
2016-05-16 17:14         ` [Intel-wired-lan] " John Fastabend
2016-05-17  9:00       ` ethan zhao
2016-05-17  9:00         ` ethan zhao
2016-05-17 15:58         ` [Intel-wired-lan] " Alexander Duyck
2016-05-17 15:58           ` Alexander Duyck
2016-05-16 20:58 ` [Intel-wired-lan] " Jeff Kirsher
2016-05-16 20:58   ` Jeff Kirsher
2016-05-16 20:58   ` Jeff Kirsher

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=5739FFDC.80908@gmail.com \
    --to=john.fastabend@gmail.com \
    --cc=intel-wired-lan@osuosl.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.