iwlwifi aggregation info

iwlwifi aggregation info

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 468 bytes --]

Hey,

Can you tell me how aggregation works on hw? It seems to me that there's
a problem:

Say a station enables aggregation on a low-priority TID, manages to fill
the hw queue because high-prio frames are eating up all the airtime and
then disables aggregation again. Now another station enables aggregation
on a high-priority TID, and gets assigned to the same hw queue. Now
frames for that station will be stalled, no? Seems like a bug to me.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: iwlwifi aggregation info

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 629 bytes --]


> Say a station enables aggregation on a low-priority TID, manages to fill
> the hw queue because high-prio frames are eating up all the airtime and
> then disables aggregation again. Now another station enables aggregation
> on a high-priority TID, and gets assigned to the same hw queue. Now
> frames for that station will be stalled, no? Seems like a bug to me.

Never mind, the queue won't get reassigned. I was just confused by a
bunch of superfluous code in iwl-tx.c. You cannot ever run into having
IWL_EMPTYING_HW_QUEUE_ADDBA set because you wait for the queue to drain
before fully disabling it.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: iwlwifi aggregation info

[Tomas Winkler]

On Tue, Jul 29, 2008 at 2:36 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
>
>> Say a station enables aggregation on a low-priority TID, manages to fill
>> the hw queue because high-prio frames are eating up all the airtime and
>> then disables aggregation again. Now another station enables aggregation
>> on a high-priority TID, and gets assigned to the same hw queue. Now
>> frames for that station will be stalled, no? Seems like a bug to me.

aggregation queue can still can tx packets in singles so such stall
doesn't happen even in theory.

> Never mind, the queue won't get reassigned. I was just confused by a
> bunch of superfluous code in iwl-tx.c. You cannot ever run into having
> IWL_EMPTYING_HW_QUEUE_ADDBA set because you wait for the queue to drain
> before fully disabling it.

Correct.

Tomas

Re: iwlwifi aggregation info

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 460 bytes --]

On Tue, 2008-07-29 at 15:25 +0300, Tomas Winkler wrote:

> > Never mind, the queue won't get reassigned. I was just confused by a
> > bunch of superfluous code in iwl-tx.c. You cannot ever run into having
> > IWL_EMPTYING_HW_QUEUE_ADDBA set because you wait for the queue to drain
> > before fully disabling it.
> 
> Correct.

I suppose you won't mind me removing that particular code then? I'm
working on the agg refactoring right now.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: iwlwifi aggregation info

[Tomas Winkler]

On Tue, Jul 29, 2008 at 3:27 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
> On Tue, 2008-07-29 at 15:25 +0300, Tomas Winkler wrote:
>
>> > Never mind, the queue won't get reassigned. I was just confused by a
>> > bunch of superfluous code in iwl-tx.c. You cannot ever run into having
>> > IWL_EMPTYING_HW_QUEUE_ADDBA set because you wait for the queue to drain
>> > before fully disabling it.
>>
>> Correct.
>
> I suppose you won't mind me removing that particular code then? I'm
> working on the agg refactoring right now.

Sorry for misleading I'm not in focus today :). You cannot remove this
code. This is form
This is required for emptying packets from legacy queue before
starting aggregation queue.
You cannot transmit from both queues for the same tid  Aggregation is
sensitive on ordering.
Tomas

Re: iwlwifi aggregation info

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 581 bytes --]

On Tue, 2008-07-29 at 15:35 +0300, Tomas Winkler wrote:

> Sorry for misleading I'm not in focus today :). You cannot remove this
> code. This is form
> This is required for emptying packets from legacy queue before
> starting aggregation queue.
> You cannot transmit from both queues for the same tid  Aggregation is
> sensitive on ordering.

I see, it's required because mac80211 would otherwise start putting
packets into the new agg queue and then that could be used by the hw
before all frames from that sta/tid in the regular queue were sent,
right?

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: iwlwifi aggregation info

[Tomas Winkler]

On Tue, Jul 29, 2008 at 3:53 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
> On Tue, 2008-07-29 at 15:35 +0300, Tomas Winkler wrote:
>
>> Sorry for misleading I'm not in focus today :). You cannot remove this
>> code. This is form
>> This is required for emptying packets from legacy queue before
>> starting aggregation queue.
>> You cannot transmit from both queues for the same tid  Aggregation is
>> sensitive on ordering.
>
> I see, it's required because mac80211 would otherwise start putting
> packets into the new agg queue and then that could be used by the hw
> before all frames from that sta/tid in the regular queue were sent,
> right?

Correct.
Other thing is you probably not seeing the latest code as well. Yi is
not publish new code before John is merging what we've already sent
out. I also have something still in the drawer.
This quite bad position still have to fix bugs on old MQ because the
new one cannot be ported at all to old kernels and already face the
breakage in the new one.
I'm starting to be frustrated about this development model it looks
like there is no single stable point of this code base.

Tomas

Re: iwlwifi aggregation info

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 811 bytes --]

On Tue, 2008-07-29 at 16:04 +0300, Tomas Winkler wrote:

> Correct.

Ok. That means it can go away with the stuff I'm doing right now.

> Other thing is you probably not seeing the latest code as well. Yi is
> not publish new code before John is merging what we've already sent
> out. I also have something still in the drawer.

Heh ok, then instead of fixing iwlwifi I'll just fix up mac80211 and
give you the patches to look at aggregation, seeing that Ron is on
vacation for another week.

What I discussed with Ron (and Jouni) is that we should not use extra
software queues for aggregation, because
 - doing it requires requeue which is not possible with tx info intact
 - doing it breaks access semantics in the case where not just hw but
   also sw queues start filling up

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: iwlwifi aggregation info

[Tomas Winkler]

On Tue, Jul 29, 2008 at 4:07 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
> On Tue, 2008-07-29 at 16:04 +0300, Tomas Winkler wrote:
>
>> Correct.
>
> Ok. That means it can go away with the stuff I'm doing right now.
>
>> Other thing is you probably not seeing the latest code as well. Yi is
>> not publish new code before John is merging what we've already sent
>> out. I also have something still in the drawer.
>
> Heh ok, then instead of fixing iwlwifi I'll just fix up mac80211 and
> give you the patches to look at aggregation, seeing that Ron is on
> vacation for another week.

iwlwifi-2.6.git master branch (kernel.org)  has most updated visible
code if you want to make same effort.

>
> What I discussed with Ron (and Jouni) is that we should not use extra
> software queues for aggregation, because
>  - doing it requires requeue which is not possible with tx info intact
>  - doing it breaks access semantics in the case where not just hw but
>   also sw queues start filling up

This will require another queueing under what we already have. You
still need ability to stop and start the packet stream.
At that time it seems to be stupid not to reuse queueing.
I'm not familiar enough we the scheduler but hopefully it doesn't mean
to implement it in the driver again.  Do we really putting correct
effort where needed maybe altering scheduler would be better
direction.  I'm really shutting from the waist here....
Tomas

Re: iwlwifi aggregation info

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 606 bytes --]

On Tue, 2008-07-29 at 16:18 +0300, Tomas Winkler wrote:

> This will require another queueing under what we already have. You
> still need ability to stop and start the packet stream.
> At that time it seems to be stupid not to reuse queueing.

That's where you're actually wrong. If the aggregation queue is full,
since they both go to the same FIFO, we should actually stop the queue
that corresponds to the AC the aggregation goes into, otherwise we
favour stations that enable aggregation over those who don't and they
unfairly get more downstream throughput (if we are the AP).

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: iwlwifi aggregation info

[Tomas Winkler]

On Tue, Jul 29, 2008 at 4:23 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
> On Tue, 2008-07-29 at 16:18 +0300, Tomas Winkler wrote:
>
>> This will require another queueing under what we already have. You
>> still need ability to stop and start the packet stream.
>> At that time it seems to be stupid not to reuse queueing.
>
> That's where you're actually wrong. If the aggregation queue is full,
> since they both go to the same FIFO, we should actually stop the queue
> that corresponds to the AC the aggregation goes into, otherwise we
> favour stations that enable aggregation over those who don't and they
> unfairly get more downstream throughput (if we are the AP).

No the mistake is on your side. FIFO is not relevant here. What is
important is whether HW queue is full or not.
Except the AC Back off scheduling done by FIFO there is BA protocol
which my stall HW queue. It doesn't mean that other packets with the
same FIFO should be stalled. You need additional SW queue that can be
stopped.
Tomas

Re: iwlwifi aggregation info

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 693 bytes --]

On Tue, 2008-07-29 at 16:43 +0300, Tomas Winkler wrote:

> No the mistake is on your side. FIFO is not relevant here. What is
> important is whether HW queue is full or not.
> Except the AC Back off scheduling done by FIFO there is BA protocol
> which my stall HW queue. It doesn't mean that other packets with the
> same FIFO should be stalled. You need additional SW queue that can be
> stopped.

HW queues are irrelevant, you're the only vendor that implements it with
HW queues. If you think they're required for correct operation, clearly
you're wrong.

I'm just trying to fix the bugs and find a way to do this with hardware
that doesn't have extra HW queues.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: iwlwifi aggregation info

[Tomas Winkler]

On Tue, Jul 29, 2008 at 4:46 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
> On Tue, 2008-07-29 at 16:43 +0300, Tomas Winkler wrote:
>
>> No the mistake is on your side. FIFO is not relevant here. What is
>> important is whether HW queue is full or not.
>> Except the AC Back off scheduling done by FIFO there is BA protocol
>> which my stall HW queue. It doesn't mean that other packets with the
>> same FIFO should be stalled. You need additional SW queue that can be
>> stopped.
>
> HW queues are irrelevant, you're the only vendor that implements it with
> HW queues. If you think they're required for correct operation, clearly
> you're wrong.

Please put some reasoning behind this 'wrong'?  Different doesn't mean
wrong and 'only one' certainly doesn't mean wrong.
And it also doesn't mean that this hw should not operates correctly
under Linux. I also cannot publish any performance comparison but it
doesn't look wrong at all.

If Intel is the only vendor implements this that way that we may push
the extra queuing into driver
but so far I've seen only athk9 with 11n.

> I'm just trying to fix the bugs and find a way to do this with hardware
> that doesn't have extra HW queues.

I don't want to even answer to this.

Tomas

Re: iwlwifi aggregation info

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 1371 bytes --]

On Tue, 2008-07-29 at 17:06 +0300, Tomas Winkler wrote:

> Please put some reasoning behind this 'wrong'?  Different doesn't mean
> wrong and 'only one' certainly doesn't mean wrong.
> And it also doesn't mean that this hw should not operates correctly
> under Linux. I also cannot publish any performance comparison but it
> doesn't look wrong at all.

Well I read that that you said one needs hardware queues for
correctness, which clearly cannot be the case since neither Atheros nor
Broadcom have hardware queues used for this.

> If Intel is the only vendor implements this that way that we may push
> the extra queuing into driver
> but so far I've seen only athk9 with 11n.

I think we need a terminology update and a bit of a big picture thing
here.

First of all, let me ask a question: Why should stations that enable
aggregation be treated preferentially by giving them an extra qdisc?

As far as I'm concerned, they should _not_ be, and thus their packets
should flow through the qdisc for the same AC that packets from non-agg
stations go through. If this AC queue gets full because it's background
and video is hogging the air, then for fairness reasons we really should
stop the whole AC and not let aggregation frames continue to flow.

I think we're talking about two different queue stopping issues here
maybe?

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: iwlwifi aggregation info

[Tomas Winkler]

On Tue, Jul 29, 2008 at 5:21 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
> On Tue, 2008-07-29 at 17:06 +0300, Tomas Winkler wrote:
>
>> Please put some reasoning behind this 'wrong'?  Different doesn't mean
>> wrong and 'only one' certainly doesn't mean wrong.
>> And it also doesn't mean that this hw should not operates correctly
>> under Linux. I also cannot publish any performance comparison but it
>> doesn't look wrong at all.
>
> Well I read that that you said one needs hardware queues for
> correctness, which clearly cannot be the case since neither Atheros nor
> Broadcom have hardware queues used for this.
>
>> If Intel is the only vendor implements this that way that we may push
>> the extra queuing into driver
>> but so far I've seen only athk9 with 11n.
>
> I think we need a terminology update and a bit of a big picture thing
> here.
>
> First of all, let me ask a question: Why should stations that enable
> aggregation be treated preferentially by giving them an extra qdisc?
>

> As far as I'm concerned, they should _not_ be, and thus their packets
> should flow through the qdisc for the same AC that packets from non-agg
> stations go through. If this AC queue gets full because it's background
> and video is hogging the air, then for fairness reasons we really should
> stop the whole AC and not let aggregation frames continue to flow.
>

So why do you need 4 HW queues for QoS, every vendor now implements it
that way today. There is only one medium, you don't put 4 packets on
the air at the same time.  Now imaging that withing single queue you
have another priority level why it is wrong to add another queue for
it?

In aggregation a bunch of packets need to be transmitted as a single
entity.  Aggregation on the air is not interleaved by other packets
There is a strict timing between packets. While there no such
distinction in application level and packets arriving interleaved to
the driver. So aggregation is definitely stream of its own rate and
from obvious reasons we need buffering for it.
Even if you don't have HW queue you still you need something that
selects bunch of packets for <sta,tid> pair put them on the air. It's
much easier if they are already queued in a single queue, maybe other
vendors have or other way to do it. This is very legitimate way.

I would say that not add HW queue might be the question of silicon
real estate for low cost products I also would handle it in SW.
You must count number of packets you are able to put in one
aggregation according TXOP etc. I guess just that HW do it faster and
more accurate and therefore can utilize medium better then sw which
has unknown delay in computation.

ipw2100 has QoS implemented in sw and it works in general but it's
headache and it's not accurate.

> I think we're talking about two different queue stopping issues here
> maybe?

Maybe
Tomas

Re: iwlwifi aggregation info

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 746 bytes --]

On Tue, 2008-07-29 at 18:55 +0300, Tomas Winkler wrote:

> So why do you need 4 HW queues for QoS, every vendor now implements it
> that way today. There is only one medium, you don't put 4 packets on
> the air at the same time.

The single medium part is true, but the scheduling decision is best made
at the air interface, otherwise you'd need to be able to kill the hw
fifo when a high-prio frame comes in to preempt other frames.

> Now imaging that withing single queue you
> have another priority level why it is wrong to add another queue for
> it?

What makes you think aggregation is another priority level though? I
don't see any evidence that it is, and everybody I've asked so far seems
to agree with me.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

RE: iwlwifi aggregation info

[Friedrich.Beckmann]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

from my understanding the reason for hardware aggregation
queues is not a different priority level as for the AC queues.
But there are two things that make hardware aggregation queues
reasonable:

a) Packets which are aggregated must be in sequence and they
must belong to the same RA/TID. So if you mix packets which
belong to that aggregation process with packets which do no
not belong to that RA/TID, then you must either break the
aggregation process or you must somehow "jump" over the
packets to find suitable canditates for aggregation.

I think a random access to a queue makes it really
difficult to handle the hardware queues as it is done
today, because today it is assumed that packets are
dequeued in the order in that they are enqueued.

Therefore it is more likely that you have to break the
aggregation process when you find a packet that does not
fit to the current aggregation process.

Assume that you have the following packets in a queue, where
the packet is described by the tuple [RA/TID/sequence number].
The first packet would go to RA 1 with TID 1 and has
sequence number 5.

[1/1/5] - [1/1/6] - [1/1/7] - [2/1/3] - [1/1/8] - [1/1/9]
                                ^
                          This packets breaks aggregation

Now the fourth packet [2/1/3] would break the aggregation process
because it does not belong to RA=3D1/TID=3D1.

b) The length of the aggregate should be adapted to
the transmit opportunity (TXOP). The longer the aggregates,
the more efficient is the transmission. The most efficient
way is when you adapt the length of the aggregate to your
remaining transmit opportunity time (TXOP). Imagine you have
a TXOP of 1ms, then you should ideally adapt the aggregate
such that it takes approx. 1ms. But when you have consumed for
other packets, e.g. already 200=B5s, then you have only 800=B5s left
and you should tailor the aggregate that it fits in the 800=B5s.

I guess this process does depend on the current transmit situation
in a very similar way as the AC queues. The problem of the scheduling
between the hardware AC queue and the hardware aggregation queue where
all packets belong also to one AC is maybe not of that importance, comp=
ared
to the increase of transmit efficiency?

Regards

=46riedrich


> -----Original Message-----
> From: linux-wireless-owner@vger.kernel.org
> [mailto:linux-wireless-owner@vger.kernel.org] On Behalf Of
> Johannes Berg
> Sent: Wednesday, July 30, 2008 11:54 AM
> To: Tomas Winkler
> Cc: linux-wireless; Jouni Malinen
> Subject: Re: iwlwifi aggregation info
>
> On Tue, 2008-07-29 at 18:55 +0300, Tomas Winkler wrote:
>
> > So why do you need 4 HW queues for QoS, every vendor now
> implements it
> > that way today. There is only one medium, you don't put 4
> packets on
> > the air at the same time.
>
> The single medium part is true, but the scheduling decision
> is best made at the air interface, otherwise you'd need to be
> able to kill the hw fifo when a high-prio frame comes in to
> preempt other frames.
>
> > Now imaging that withing single queue you have another
> priority level
> > why it is wrong to add another queue for it?
>
> What makes you think aggregation is another priority level
> though? I don't see any evidence that it is, and everybody
> I've asked so far seems to agree with me.
>
> johannes
>

-----BEGIN PGP SIGNATURE-----
Version: SECUDE secure mail 4.2.8.1
Comment: SECUDE Office Security Suite - http://www.secude.com

iQA/AwUBSJBKTYmGfrssAHC0EQJ/qwCffSqp6zAdRw1YmvEPXxfYG0TuBpgAoMfRP+4syjw=
M
qTzjq6eDkEAQVlBk
=3DYDp3
-----END PGP SIGNATURE-----
--
To unsubscribe from this list: send the line "unsubscribe linux-wireles=
s" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: iwlwifi aggregation info

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 2109 bytes --]


> from my understanding the reason for hardware aggregation
> queues is not a different priority level as for the AC queues.
> But there are two things that make hardware aggregation queues
> reasonable:
> 
> a) Packets which are aggregated must be in sequence and they
> must belong to the same RA/TID. So if you mix packets which
> belong to that aggregation process with packets which do no
> not belong to that RA/TID, then you must either break the
> aggregation process or you must somehow "jump" over the
> packets to find suitable canditates for aggregation.

Yes, I know this.

> I think a random access to a queue makes it really
> difficult to handle the hardware queues as it is done
> today, because today it is assumed that packets are
> dequeued in the order in that they are enqueued.
> 
> Therefore it is more likely that you have to break the
> aggregation process when you find a packet that does not
> fit to the current aggregation process.

Well other drivers would have to handle the aggregation in software,
obviously, and only put the packets onto the queue once enough have been
collected.

I'm fairly do understand what's going on.

How does the hardware make the scheduling decision between the regular
and a-mpdu queue? If it sends out a bunch of aggregated frames whenever
there are enough, how is that _not_ unfair to non-agg stations?

The way I would implement it (and I guess atheros/broadcom do) is to
queue up frames for a station like NAPI does: either for a certain time
(low traffic, and ampdu will be disabled again) or until the txop window
is full, and then queue all of them to the right fifo/hw queue. Which,
in my software design, simply means deferring them a bit inside the
driver. Or for your driver, sticking them into a separate queue. But if
they don't come in via the same qdisc, I don't see how it can be fair.
Please let's get the upper layer considerations before we talk about the
implementation though. Do you not agree that giving an aggregation flow
a separate qdisc is unfair within that AC?

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: iwlwifi aggregation info

[Tomas Winkler]

On Wed, Jul 30, 2008 at 4:19 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
>
>> from my understanding the reason for hardware aggregation
>> queues is not a different priority level as for the AC queues.
>> But there are two things that make hardware aggregation queues
>> reasonable:
>>
>> a) Packets which are aggregated must be in sequence and they
>> must belong to the same RA/TID. So if you mix packets which
>> belong to that aggregation process with packets which do no
>> not belong to that RA/TID, then you must either break the
>> aggregation process or you must somehow "jump" over the
>> packets to find suitable canditates for aggregation.
>
> Yes, I know this.
>
>> I think a random access to a queue makes it really
>> difficult to handle the hardware queues as it is done
>> today, because today it is assumed that packets are
>> dequeued in the order in that they are enqueued.
>>
>> Therefore it is more likely that you have to break the
>> aggregation process when you find a packet that does not
>> fit to the current aggregation process.
>
> Well other drivers would have to handle the aggregation in software,
> obviously, and only put the packets onto the queue once enough have been
> collected.
>
> I'm fairly do understand what's going on.
>
> How does the hardware make the scheduling decision between the regular
> and a-mpdu queue? If it sends out a bunch of aggregated frames whenever
> there are enough, how is that _not_ unfair to non-agg stations?
>
> The way I would implement it (and I guess atheros/broadcom do) is to
> queue up frames for a station like NAPI does: either for a certain time
> (low traffic, and ampdu will be disabled again) or until the txop window
> is full, and then queue all of them to the right fifo/hw queue. Which,
> in my software design, simply means deferring them a bit inside the
> driver. Or for your driver, sticking them into a separate queue. But if
> they don't come in via the same qdisc, I don't see how it can be fair.
> Please let's get the upper layer considerations before we talk about the
> implementation though. Do you not agree that giving an aggregation flow
> a separate qdisc is unfair within that AC?

I'm think your miss understanding is that HW FIFO != HW QUEUE.
HW FIFO takes gives fairness in level of AC.. HW QUEUE just piles up
packets for HW scheduler.
qdisc should be just provide simple buffering
Tomas

Re: iwlwifi aggregation info

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 690 bytes --]

On Wed, 2008-07-30 at 16:45 +0300, Tomas Winkler wrote:
> > Do you not agree that giving an aggregation flow
> > a separate qdisc is unfair within that AC?
> 
> I'm think your miss understanding is that HW FIFO != HW QUEUE.
> HW FIFO takes gives fairness in level of AC.. HW QUEUE just piles up
> packets for HW scheduler.

I know, but I cannot make out how the hw queue is pulled into the hw
fifo fairly. I'm not talking about fair media access, I'm talking about
fair number of frames in congestion.

> qdisc should be just provide simple buffering

Well qdiscs don't just do that, depending on the configuration, so we
shouldn't rely on them doing just that.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: iwlwifi aggregation info

[Tomas Winkler]

On Wed, Jul 30, 2008 at 4:50 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
> On Wed, 2008-07-30 at 16:45 +0300, Tomas Winkler wrote:
>> > Do you not agree that giving an aggregation flow
>> > a separate qdisc is unfair within that AC?
>>
>> I'm think your miss understanding is that HW FIFO != HW QUEUE.
>> HW FIFO takes gives fairness in level of AC.. HW QUEUE just piles up
>> packets for HW scheduler.
>
> I know, but I cannot make out how the hw queue is pulled into the hw
> fifo fairly. I'm not talking about fair media access, I'm talking about
> fair number of frames in congestion.
>
>> qdisc should be just provide simple buffering
>
> Well qdiscs don't just do that, depending on the configuration, so we
> shouldn't rely on them doing just that.

I know but this was the primary purpose of MQ to push the scheduling
decision to HW if I'm not mistaken.
We really don't need anything else just queueing. For example what the
prioritization which is done in current wme.c
is wrong and it won't pass certification and creates starvation.
Tested and proved.
Tomas

Re: iwlwifi aggregation info

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 997 bytes --]

On Wed, 2008-07-30 at 16:59 +0300, Tomas Winkler wrote:

> > Well qdiscs don't just do that, depending on the configuration, so we
> > shouldn't rely on them doing just that.
> 
> I know but this was the primary purpose of MQ to push the scheduling
> decision to HW if I'm not mistaken.

I thought it was more about the upper level locking, ethernet hw really
just round-robins most of the time. Although it will, of course, be used
for scheduling decisions.

Is aggregation really a scheduling decision though?

> We really don't need anything else just queueing.

Exactly, but qdiscs do most definitely not provide just queueing, and we
should not deprive the user of the ability to use qdiscs on wireless.

>  For example what the
> prioritization which is done in current wme.c
> is wrong and it won't pass certification and creates starvation.
> Tested and proved.

Can you explain how starvation happens? In what scenarios? With or
without aggregation?

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: iwlwifi aggregation info

[Tomas Winkler]

On Wed, Jul 30, 2008 at 6:19 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
> On Wed, 2008-07-30 at 16:59 +0300, Tomas Winkler wrote:
>
>> > Well qdiscs don't just do that, depending on the configuration, so we
>> > shouldn't rely on them doing just that.
>>
>> I know but this was the primary purpose of MQ to push the scheduling
>> decision to HW if I'm not mistaken.
>
> I thought it was more about the upper level locking, ethernet hw really
> just round-robins most of the time. Although it will, of course, be used
> for scheduling decisions.
>
> Is aggregation really a scheduling decision though?
>
>> We really don't need anything else just queueing.
>
> Exactly, but qdiscs do most definitely not provide just queueing, and we
> should not deprive the user of the ability to use qdiscs on wireless.
>
>>  For example what the
>> prioritization which is done in current wme.c
>> is wrong and it won't pass certification and creates starvation.
>> Tested and proved.
>
> Can you explain how starvation happens? In what scenarios? With or
> without aggregation?

for (queue = 0; queue < QD_NUM(hw); queue++)

This always starts 0  prioritize dequeue the first frame. But what we
need is RR and let HW to prioritize the transmission according AC
Tomas

Re: iwlwifi aggregation info

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 468 bytes --]

On Wed, 2008-07-30 at 19:08 +0300, Tomas Winkler wrote:

> > Can you explain how starvation happens? In what scenarios? With or
> > without aggregation?
> 
> for (queue = 0; queue < QD_NUM(hw); queue++)
> 
> This always starts 0  prioritize dequeue the first frame. But what we
> need is RR and let HW to prioritize the transmission according AC

That's from the old code not having real MQ though, right? I'm totally
not concerned about that.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: iwlwifi aggregation info

[Tomas Winkler]

On Thu, Jul 31, 2008 at 4:05 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
> On Wed, 2008-07-30 at 19:08 +0300, Tomas Winkler wrote:
>
>> > Can you explain how starvation happens? In what scenarios? With or
>> > without aggregation?
>>
>> for (queue = 0; queue < QD_NUM(hw); queue++)
>>
>> This always starts 0  prioritize dequeue the first frame. But what we
>> need is RR and let HW to prioritize the transmission according AC
>
> That's from the old code not having real MQ though, right? I'm totally
> not concerned about that.

Correct. The bottom line is that correct behavior MQ would be shift
scheduling decision to the HW.
Tomas

Re: iwlwifi aggregation info

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 892 bytes --]

On Thu, 2008-07-31 at 21:14 +0300, Tomas Winkler wrote:
> On Thu, Jul 31, 2008 at 4:05 PM, Johannes Berg
> <johannes@sipsolutions.net> wrote:
> > On Wed, 2008-07-30 at 19:08 +0300, Tomas Winkler wrote:
> >
> >> > Can you explain how starvation happens? In what scenarios? With or
> >> > without aggregation?
> >>
> >> for (queue = 0; queue < QD_NUM(hw); queue++)
> >>
> >> This always starts 0  prioritize dequeue the first frame. But what we
> >> need is RR and let HW to prioritize the transmission according AC
> >
> > That's from the old code not having real MQ though, right? I'm totally
> > not concerned about that.
> 
> Correct. The bottom line is that correct behavior MQ would be shift
> scheduling decision to the HW.

Right. Which brings us back to the original point, why does the hw need
to make the scheduling decision between agg and non-agg?

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: iwlwifi aggregation info

[Tomas Winkler]

On Thu, Jul 31, 2008 at 9:23 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
> On Thu, 2008-07-31 at 21:14 +0300, Tomas Winkler wrote:
>> On Thu, Jul 31, 2008 at 4:05 PM, Johannes Berg
>> <johannes@sipsolutions.net> wrote:
>> > On Wed, 2008-07-30 at 19:08 +0300, Tomas Winkler wrote:
>> >
>> >> > Can you explain how starvation happens? In what scenarios? With or
>> >> > without aggregation?
>> >>
>> >> for (queue = 0; queue < QD_NUM(hw); queue++)
>> >>
>> >> This always starts 0  prioritize dequeue the first frame. But what we
>> >> need is RR and let HW to prioritize the transmission according AC
>> >
>> > That's from the old code not having real MQ though, right? I'm totally
>> > not concerned about that.
>>
>> Correct. The bottom line is that correct behavior MQ would be shift
>> scheduling decision to the HW.
>
> Right. Which brings us back to the original point, why does the hw need
> to make the scheduling decision between agg and non-agg?

There is no scheduling between aag and legacy queue in the sense of
qdisc .The aggregation need to be taken from single stream as
explained before,
Iwlwifi has HW support for it that that's the whole story we just need
queueing support from the software buffering stopping and starting
queue and last but not least there is a classification just an
extension of the regular AC scheduling.  The fairness between legacy
and agg queue must be provided by actually 'not scheduling'
If you don't want to experiment but provide QoS by spec the scheduling
is best to leave to HW.

Thanks
Tomas

Re: iwlwifi aggregation info

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 1559 bytes --]

Had to think about this for a bit...

> > Right. Which brings us back to the original point, why does the hw need
> > to make the scheduling decision between agg and non-agg?
> 
> There is no scheduling between aag and legacy queue in the sense of
> qdisc .

Right. So why are you saying we should have a separate qdisc for it?

> The aggregation need to be taken from single stream as
> explained before,

I think we simply agree on that. Which brings me back to my original
point: to provide fairness within that stream we shouldn't have separate
qdiscs for agg/non-agg parts of the stream.

> Iwlwifi has HW support for it that that's the whole story we just need
> queueing support from the software buffering stopping and starting
> queue and last but not least there is a classification just an
> extension of the regular AC scheduling.  The fairness between legacy
> and agg queue must be provided by actually 'not scheduling'

I don't understand what you mean by "not scheduling". AIUI from the
specs, there is no scheduling between aggregation/non-aggregation
queues, or "within an AC" as I would say it.

Therefore, I think we should remove the extra software queues and split
up the single-AC stream into the different hardware queues in the
driver, to be reunited in the FIFOs.

> If you don't want to experiment but provide QoS by spec the scheduling
> is best to leave to HW.

I don't contend that only the hw is able to make QoS scheduling
decisions, as they depend on air access etc. after all.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: iwlwifi aggregation info

[Tomas Winkler]

On Fri, Aug 1, 2008 at 3:09 PM, Johannes Berg <johannes@sipsolutions.net> wrote:
> Had to think about this for a bit...
>
>> > Right. Which brings us back to the original point, why does the hw need
>> > to make the scheduling decision between agg and non-agg?
>>
>> There is no scheduling between aag and legacy queue in the sense of
>> qdisc .
>
> Right. So why are you saying we should have a separate qdisc for it?

I need a sw queue for it.

>> The aggregation need to be taken from single stream as
>> explained before,
>
> I think we simply agree on that. Which brings me back to my original
> point: to provide fairness within that stream we shouldn't have separate
> qdiscs for agg/non-agg parts of the stream.

You agree on the fact that it's a seperate stream but you still
doesn't want separate queue for it....

>> Iwlwifi has HW support for it that that's the whole story we just need
>> queueing support from the software buffering stopping and starting
>> queue and last but not least there is a classification just an
>> extension of the regular AC scheduling.  The fairness between legacy
>> and agg queue must be provided by actually 'not scheduling'
>
> I don't understand what you mean by "not scheduling".

Not scheduling mean not string to prioritize streams in SW. I guess it means RR.

 AIUI from the
> specs, there is no scheduling between aggregation/non-aggregation
> queues, or "within an AC" as I would say it.
>
> Therefore, I think we should remove the extra software queues and split
> up the single-AC stream into the different hardware queues in the
> driver, to be reunited in the FIFOs.

Aggregation is a separate stream even on the air it has it's own
rhythm.  For example
from AP perspective you an have 3 streams for the same TID for 3
stations. Each station
has it's own rate of processing aggregation stream.  It may vary on
number of packets and size of the aggregation
this is determine in association time.
So shell I stop the whole AC queue just because on station is slower?
Tomas

Re: iwlwifi aggregation info

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 4016 bytes --]

On Fri, 2008-08-01 at 15:40 +0300, Tomas Winkler wrote:

> >> > Right. Which brings us back to the original point, why does the hw need
> >> > to make the scheduling decision between agg and non-agg?
> >>
> >> There is no scheduling between aag and legacy queue in the sense of
> >> qdisc .
> >
> > Right. So why are you saying we should have a separate qdisc for it?
> 
> I need a sw queue for it.

Sure, you need to buffer packets, but that's a whole different beast.

> >> The aggregation need to be taken from single stream as
> >> explained before,
> >
> > I think we simply agree on that. Which brings me back to my original
> > point: to provide fairness within that stream we shouldn't have separate
> > qdiscs for agg/non-agg parts of the stream.
> 
> You agree on the fact that it's a seperate stream but you still
> doesn't want separate queue for it....

Actually, no, I don't think it's a separate stream, I think aggregation
frames are taken from the AC stream. I thought you meant that.

> >> Iwlwifi has HW support for it that that's the whole story we just need
> >> queueing support from the software buffering stopping and starting
> >> queue and last but not least there is a classification just an
> >> extension of the regular AC scheduling.  The fairness between legacy
> >> and agg queue must be provided by actually 'not scheduling'
> >
> > I don't understand what you mean by "not scheduling".
> 
> Not scheduling mean not string to prioritize streams in SW. I guess it means RR.

But that's entirely user dependent if we allow actual qdiscs.

>  AIUI from the
> > specs, there is no scheduling between aggregation/non-aggregation
> > queues, or "within an AC" as I would say it.
> >
> > Therefore, I think we should remove the extra software queues and split
> > up the single-AC stream into the different hardware queues in the
> > driver, to be reunited in the FIFOs.
> 
> Aggregation is a separate stream even on the air it has it's own
> rhythm.  For example
> from AP perspective you an have 3 streams for the same TID for 3
> stations. Each station
> has it's own rate of processing aggregation stream.  It may vary on
> number of packets and size of the aggregation
> this is determine in association time.
> So shell I stop the whole AC queue just because on station is slower?

If you want to draw up scenarios here, how about this one:

From an AP perspective, assume three streams for the same TID for three
stations. No aggregation involved. Say they all receive a fixed bitrate
video stream. Assume you need all the airtime to transmit the three
video streams when all stations are fairly close. Now one of them is
further away and is a bit slower than the others. This will lead to
contention, right? So now, frames will queue up in the AC queue for this
TID, penalising all three stations. Do you agree so far?

Now, enter aggregation. The way I think of it (and Jouni seems to agree)
is that the spec intended for aggregation to be a mechanism to reduce
the required airtime by making the transmission more efficient with less
space.

On the other hand, you seem to think of aggregation as a QoS mechanism,
and I think that's wrong.

Ideally, for the first two fast stations would allow enough airtime to
be freed up so that the third station can still receive the video
stream.

When that is not the case, however, we disagree. I think that because
aggregation isn't a QoS mechanism, it should behave the same way as in
the case where no stations have aggregation enabled, and stall the whole
queue. On the other hand, you think it is a QoS mechanism, and let
streams for the fast stations be interleaved with the slow station,
leaving only frames for the slow station piling up.

Is my analysis of what you are saying correct? If so, I think we cannot
solve the fundamental disagreement here without actually going and
asking for other people's opinion on the 11n draft.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: iwlwifi aggregation info

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 807 bytes --]


> When that is not the case, however, we disagree. I think that because
> aggregation isn't a QoS mechanism, it should behave the same way as in
> the case where no stations have aggregation enabled, and stall the whole
> queue. On the other hand, you think it is a QoS mechanism, and let
> streams for the fast stations be interleaved with the slow station,
> leaving only frames for the slow station piling up.

I just found IEEE 802.11-2007 subclause 9.10 which actually explains all
the block-ack business without aggregation, but I assume that
aggregation now just means that instead of sending
	mpdu + sifs + (mpdu + sifs)* + blockackreq
you send simply
	a-mpdu

I see nothing in 9.10 that supports the view that aggregation/block-ack
should create a new traffic stream.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: iwlwifi aggregation info

[Tomas Winkler]

On Thu, Aug 7, 2008 at 12:45 AM, Johannes Berg
<johannes@sipsolutions.net> wrote:
>
>> When that is not the case, however, we disagree. I think that because
>> aggregation isn't a QoS mechanism, it should behave the same way as in
>> the case where no stations have aggregation enabled, and stall the whole
>> queue. On the other hand, you think it is a QoS mechanism, and let
>> streams for the fast stations be interleaved with the slow station,
>> leaving only frames for the slow station piling up.
>
> I just found IEEE 802.11-2007 subclause 9.10 which actually explains all
> the block-ack business without aggregation, but I assume that
> aggregation now just means that instead of sending
>        mpdu + sifs + (mpdu + sifs)* + blockackreq
> you send simply
>        a-mpdu
>
> I see nothing in 9.10 that supports the view that aggregation/block-ack
> should create a new traffic stream.

It doesn't have to be spelled, it's clear from the fact that traffic
is defined by <TID,RA>  pair.
A-MPDU stuff is simplified version of what you've read there.
Tomas

Re: iwlwifi aggregation info

[Tomas Winkler]

On Fri, Aug 1, 2008 at 3:54 PM, Johannes Berg <johannes@sipsolutions.net> wrote:
> On Fri, 2008-08-01 at 15:40 +0300, Tomas Winkler wrote:
>> > Right. So why are you saying we should have a separate qdisc for it?
>>
>> I need a sw queue for it.
>
> Sure, you need to buffer packets, but that's a whole different beast.

So this is the most important point of this conversation... What do
you call this beast!
What I'm trying to say all the way along I don't need qdisc per se I
need like 80% of it.
So it was already implemented in qdisc so it was utilized.
I'm not aware of anything else that is already implemented but you can
always open my eyes and I'm not trying to be cynical.

>
>> >> The aggregation need to be taken from single stream as
>> >> explained before,
>> >
>> > I think we simply agree on that. Which brings me back to my original
>> > point: to provide fairness within that stream we shouldn't have separate
>> > qdiscs for agg/non-agg parts of the stream.
>>
>> You agree on the fact that it's a seperate stream but you still
>> doesn't want separate queue for it....
>
> Actually, no, I don't think it's a separate stream, I think aggregation
> frames are taken from the AC stream. I thought you meant that.

This is really theoretical question. Wireless has defined 4 AC stream
for simplicity
so you map 2 TIDs out of 8 into 4 AC stream. One possible view
of aggregation you don't map TID into AC stream but run it along to
get more fine grained.
Otherwise why don't we do <AC, RA> but <TID,RA> . Just a possible
view. after all it's mapped back to AC.


>> Not scheduling mean not string to prioritize streams in SW. I guess it means RR.
>
> But that's entirely user dependent if we allow actual qdiscs.

True but there is s always some sane default.

>
>
> If you want to draw up scenarios here, how about this one:
>
> From an AP perspective, assume three streams for the same TID for three
> stations. No aggregation involved. Say they all receive a fixed bitrate
> video stream. Assume you need all the airtime to transmit the three
> video streams when all stations are fairly close. Now one of them is
> further away and is a bit slower than the others. This will lead to
> contention, right? So now, frames will queue up in the AC queue for this
> TID, penalising all three stations. Do you agree so far?

You are looking at the wrong point. All stations got fair amount of
air space according AC and TXOP but then number of bytes that are
transmitted is different. While the slow station will use this time to
transfer packet on low rate with many retries faster station will fill
it with many bytes at high rates.I hope it's clear that fixed rate
video stream is not implies wireless rate.
I believe the upper level in this case will take flow control decision
and release packet pressure on the slow station.

> Now, enter aggregation. The way I think of it (and Jouni seems to agree)
> is that the spec intended for aggregation to be a mechanism to reduce
> the required airtime by making the transmission more efficient with less
> space.
Exactly, but if you don't have mechanism to fill up the TXOP
efficiently such as have packets ready in a separate queue it just
won't fly.

-------
I suggest to start at some more productive point.
So what is important is that we agree on one thing that we need sw
queue/buffer (forget qdisc for now) for aggregation queue and the
question is how to implement it efficiently  and how to resolve
cleanly the lack of requeueing in the new MQ. .
-------
Thanks
Tomas

mac80211 aggregation (was: iwlwifi aggregation info)

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 4149 bytes --]

On Thu, 2008-08-07 at 01:31 +0300, Tomas Winkler wrote:
> On Fri, Aug 1, 2008 at 3:54 PM, Johannes Berg <johannes@sipsolutions.net> wrote:
> > On Fri, 2008-08-01 at 15:40 +0300, Tomas Winkler wrote:
> >> > Right. So why are you saying we should have a separate qdisc for it?
> >>
> >> I need a sw queue for it.
> >
> > Sure, you need to buffer packets, but that's a whole different beast.
> 
> So this is the most important point of this conversation... What do
> you call this beast!

Ok I see where we're going. You're thinking of "struct
sk_buff_head" (which is part of most qdiscs) while I always thought you
actually wanted a qdisc.

> What I'm trying to say all the way along I don't need qdisc per se I
> need like 80% of it.

Right.

> So it was already implemented in qdisc so it was utilized.
> I'm not aware of anything else that is already implemented but you can
> always open my eyes and I'm not trying to be cynical.

Well there's struct sk_buff_head that allows you to queue up frames
anywhere you want them to be, we use it for example for PS-buffered
frames. Of course that means we need to implement it ourselves.

> This is really theoretical question. Wireless has defined 4 AC stream
> for simplicity
> so you map 2 TIDs out of 8 into 4 AC stream. One possible view
> of aggregation you don't map TID into AC stream but run it along to
> get more fine grained.
> Otherwise why don't we do <AC, RA> but <TID,RA> . Just a possible
> view. after all it's mapped back to AC.

Well 802.11 isn't really clear about this, it talks about QSTAs having
no more than one frame per <TID,RA,SA> outstanding at a time. I don't
have access to wi-fi documents but I suspect they add further
constraints because it's not feasible to have that many packets buffered
on the hardware.

> >> Not scheduling mean not string to prioritize streams in SW. I guess it means RR.
> >
> > But that's entirely user dependent if we allow actual qdiscs.
> 
> True but there is s always some sane default.

So we should disallow changing qdiscs? I'm not entirely against that,
but I think if we go that way we should force it in the code.

> I suggest to start at some more productive point.
> So what is important is that we agree on one thing that we need sw
> queue/buffer (forget qdisc for now) for aggregation queue and the
> question is how to implement it efficiently  and how to resolve
> cleanly the lack of requeueing in the new MQ. .

Well we can clearly do requeuing as much as we like, but we lose the
skb->cb over it. In fact, the master_start_xmit now makes that explicit
by memsetting it to 0.

I've looked into the Atheros driver, they way the implement it is that
when frames for an aggregation session are passed to the ->tx() call
they simply buffer them in software and when they have enough they
enqueue all of them to the hardware queue for the particular AC they map
into. This scales much better, with 16 TIDs and many stations you can
that way possibly support a lot of concurrent aggregation sessions.

Anyway, I'm confident you can work it out, I don't have time any more
for it right now, as I'll announce in a separate email. I had a patch
that mostly went the "just have queues per AC" route, but it was
incomplete, if anybody wants to take a look it's at
http://johannes.sipsolutions.net/patches/kernel/all/2008-08-04-12%3a26/006-fix-skb-cb-agg.patch

Here's a recap of the current issues that I'm aware of:

      * select_queue cannot really look into any state because due to
        the lack of a single TX lock there's nothing to prevent it
        racing against the code starting/stopping the aggregation
      * you cannot re-queue and assume that skb->cb will be left intact,
        but that's codified now by clearing skb->cb on master_start_xmit
      * the current implementation of ieee80211_requeue is racy against
        select_queue, select_queue is called under RCU lock though so
        you can use synchronize_rcu to avoid the race, this requires
        moving ieee80211_requeue into process context though

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

RE: mac80211 aggregation (was: iwlwifi aggregation info)

[Friedrich.Beckmann]

>
> > This is really theoretical question. Wireless has defined 4
> AC stream
> > for simplicity so you map 2 TIDs out of 8 into 4 AC stream. One
> > possible view of aggregation you don't map TID into AC
> stream but run
> > it along to get more fine grained.
> > Otherwise why don't we do <AC, RA> but <TID,RA> . Just a possible
> > view. after all it's mapped back to AC.
>
> Well 802.11 isn't really clear about this, it talks about
> QSTAs having no more than one frame per <TID,RA,SA>
> outstanding at a time. I don't have access to wi-fi documents
> but I suspect they add further constraints because it's not
> feasible to have that many packets buffered on the hardware.

The aggregation is introduced in 802.11n and A-MPDU aggregation
always comes together with block acknowledgement. The reason is that
the component frames of an A-MPDU aggregate are individually subject
to retransmission but not the aggregate itself.

Block Acknowlegment allows to have a configurable number of
outstanding packets (up to 128) which you can transmit before
you require an acknowledgement. Therefore you also have to
keep the packets for which you do not have already an
acknowledgement. This is similar to TCP windows.

Each block acknowledgement and aggregation process is
identfied by the <TID, RA, SA> tuple. Having this ID has
as a consequence that you have one sequence number counter
for these packets, allowing the duplicate detection and retransmission.

Maximum number of outstanding packets = 1 is only valid
for the standard acknowledgement procedure (= one ack per packet).

The AC has first hand only to do with the access priority to the
air, but as a consequence the order of packets is only guaranteed
within each AC. So each AC needs an individual TID and as a result an
individual sequence number counter in order to do
the duplicate detection.

>
>
> I've looked into the Atheros driver, they way the implement
> it is that when frames for an aggregation session are passed
> to the ->tx() call they simply buffer them in software and
> when they have enough they enqueue all of them to the
> hardware queue for the particular AC they map into. This
> scales much better, with 16 TIDs and many stations you can
> that way possibly support a lot of concurrent aggregation sessions.
>
> Anyway, I'm confident you can work it out, I don't have time
> any more for it right now, as I'll announce in a separate
> email. I had a patch that mostly went the "just have queues
> per AC" route, but it was incomplete, if anybody wants to
> take a look it's at
> http://johannes.sipsolutions.net/patches/kernel/all/2008-08-04
> -12%3a26/006-fix-skb-cb-agg.patch
>
> Here's a recap of the current issues that I'm aware of:
>
>       * select_queue cannot really look into any state because due to
>         the lack of a single TX lock there's nothing to prevent it
>         racing against the code starting/stopping the aggregation
>       * you cannot re-queue and assume that skb->cb will be
> left intact,
>         but that's codified now by clearing skb->cb on
> master_start_xmit
>       * the current implementation of ieee80211_requeue is
> racy against
>         select_queue, select_queue is called under RCU lock though so
>         you can use synchronize_rcu to avoid the race, this requires
>         moving ieee80211_requeue into process context though
>
> johannes
>

The atheros way is for sure possible but it comes with some constraints
in building the aggregates.

a) You can not dynamically adapt the A-MPDU to the remaining TXOP.
b) You need to start a retransmission process of the aggregate packets in
   the AC queue immediatly, because otherwise you cannot release the
   buffers.

Regards

Friedrich

RE: mac80211 aggregation (was: iwlwifi aggregation info)

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 2021 bytes --]

On Thu, 2008-08-07 at 14:21 +0200, Friedrich.Beckmann@infineon.com
wrote:

> The aggregation is introduced in 802.11n and A-MPDU aggregation
> always comes together with block acknowledgement. The reason is that
> the component frames of an A-MPDU aggregate are individually subject
> to retransmission but not the aggregate itself.

Sure, but for the purpose of the discussion the actual aggregation
doesn't matter much, and the block-ack was introduced by 11e

> Block Acknowlegment allows to have a configurable number of
> outstanding packets (up to 128) which you can transmit before
> you require an acknowledgement. Therefore you also have to
> keep the packets for which you do not have already an
> acknowledgement. This is similar to TCP windows.

Sure, I can read 11e too :)

> The AC has first hand only to do with the access priority to the
> air, but as a consequence the order of packets is only guaranteed
> within each AC. So each AC needs an individual TID and as a result an
> individual sequence number counter in order to do
> the duplicate detection.

This is a bit confusing to me, I don't think each AC has a TID, rather
more than one TIDs map into a single AC.

> The atheros way is for sure possible but it comes with some constraints
> in building the aggregates.
> 
> a) You can not dynamically adapt the A-MPDU to the remaining TXOP.
> b) You need to start a retransmission process of the aggregate packets in
>    the AC queue immediatly, because otherwise you cannot release the
>    buffers.

I wouldn't be so sure about either of these, Atheros hardware has an
elaborate scheme for suppressing frames and asking software to requeue
them to the hardware at a later time. I suspect they can use it here to
adapt the A-MPDU to the TXOP and suppress the remaining MPDUs if they
need to, and tell the driver which of the aggregate packets needs to be
retried later. I suppose that would need fairly shallow DMA queues, but
that's doable.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

RE: mac80211 aggregation (was: iwlwifi aggregation info)

[Friedrich.Beckmann]

>
> > The AC has first hand only to do with the access priority
> to the air,
> > but as a consequence the order of packets is only guaranteed within
> > each AC. So each AC needs an individual TID and as a result an
> > individual sequence number counter in order to do the duplicate
> > detection.
>
> This is a bit confusing to me, I don't think each AC has a
> TID, rather more than one TIDs map into a single AC.

That is true. You cannot map packets with the same TID to
several AC queues. The reason is retransmission. Each TID stream
has to be mapped to one AC. My understanding from the previous email
was different.

>
> > The atheros way is for sure possible but it comes with some
> > constraints in building the aggregates.
> >
> > a) You can not dynamically adapt the A-MPDU to the remaining TXOP.
> > b) You need to start a retransmission process of the
> aggregate packets in
> >    the AC queue immediatly, because otherwise you cannot release the
> >    buffers.
>
> I wouldn't be so sure about either of these, Atheros hardware
> has an elaborate scheme for suppressing frames and asking
> software to requeue them to the hardware at a later time. I
> suspect they can use it here to adapt the A-MPDU to the TXOP
> and suppress the remaining MPDUs if they need to, and tell
> the driver which of the aggregate packets needs to be retried
> later. I suppose that would need fairly shallow DMA queues,
> but that's doable.
>
> johannes
>

You are right, with software requeue of frames you get rid of the
retransmission problem. With the tailoring of the A-MPDU to the
TXOP I am more concerned of not having enough packets. In order to
have some efficient aggregation you need to collect a number of
packets but you have to make sure that if you have just one packet
that you release it after some waiting time.

Regards

Friedrich

Re: iwlwifi aggregation info

[Tomas Winkler]

On Wed, Jul 30, 2008 at 12:53 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
> On Tue, 2008-07-29 at 18:55 +0300, Tomas Winkler wrote:
>
>> So why do you need 4 HW queues for QoS, every vendor now implements it
>> that way today. There is only one medium, you don't put 4 packets on
>> the air at the same time.
>
> The single medium part is true, but the scheduling decision is best made
> at the air interface, otherwise you'd need to be able to kill the hw
> fifo when a high-prio frame comes in to preempt other frames.

The scheduling decision is made there. high priority packet is
scheduled to different hw fifo.
This issue is really no concern here.

>> Now imaging that withing single queue you
>> have another priority level why it is wrong to add another queue for
>> it?
>
> What makes you think aggregation is another priority level though? I
> don't see any evidence that it is, and everybody I've asked so far seems
> to agree with me.

Not it's not priority level in proper sense I just want to make the
picture tangible to you. There is world 'imagine'  in the sentence.
I've focused on less important part of the explanation.
What is more important that it's different stream I hope that
Frederich made it clearer

Tomas