[RFC] Redesigning aggregation on mac80211

[RFC] Redesigning aggregation on mac80211

[Luis R. Rodriguez]

We're considering redesigning how aggregation works on mac80211 to
ensure its vendor neutral and to remove as much as redundant code in
drivers. We'd like some feedback from people and driver developers,
even those who haven't *yet* posted their new shiny 11n drivers, like
Marvell or Ralink driver developers.

First let me review the current design and how the different hardware
supported uses aggregation *as I understand it* so far.

I'll start with Atheros hardware and mac80211. Atheros hardware uses 4
main HW queues, one for each AC, and we also have one for beacons,
another for queued stuff after beacon (aka "Crap After Beacon") where
things like queued frames for broadcast / multicast or for frames for
stations in power save mode, and I think another hw queue for uapsd.
Right now mac80211 queues skb's for drivers into the netdev_queues.
Under the current mac80211 design we have 4 netdev_queues, one for
each AC and then there is a number of "aggregation queues". We
essentially just declare one "aggregation" queue as we do our own
queue mapping internally to our own hardware queues -- and that
consist of just mapping frames to each HQ queue as per AC.

Intel's hardware seems to have a hardware scheduler for aggregation
and although it does seem you are sending frames onto a real hardware
queue you are actually passing it to the hardware so it can itself do
the hardware scheduling onto a proper AC queue somewhere or something
(perhaps someone from Intel can elaborate a little more on this). The
Intel drivers make use of the "aggregation queues" more as buffer
space in case of retransmit on failed frames. So these aren't actually
HW queues per se as we thought they were.

We were thinking of just ditching the "aggregation queues" because
they seemed to represent actual hardware queues (and it seems they are
but in a different way) but its now evident the "aggregation queues"
are more useful as a way to move the queuing outside of the driver and
make it part of the stack.

Ivo, are you familiar yet with how aggregation works on Ralink
hardware? Does the new 11n hardware require firmware? Specifically
we're curious about how the hardware queues are managed.

Anyone know about Marvell's stuff? How about Broadcom?

  Luis

Re: [RFC] Redesigning aggregation on mac80211

[Johannes Berg]

[-- Attachment #1: Type: text/plain, Size: 1152 bytes --]

On Mon, 2008-09-15 at 12:03 -0700, Luis R. Rodriguez wrote:

> I'll start with Atheros hardware and mac80211. Atheros hardware uses 4
> main HW queues, one for each AC, and we also have one for beacons,
> another for queued stuff after beacon (aka "Crap After Beacon") where
> things like queued frames for broadcast / multicast or for frames for
> stations in power save mode, and I think another hw queue for uapsd.
> Right now mac80211 queues skb's for drivers into the netdev_queues.
> Under the current mac80211 design we have 4 netdev_queues, one for
> each AC and then there is a number of "aggregation queues". We
> essentially just declare one "aggregation" queue as we do our own
> queue mapping internally to our own hardware queues -- and that
> consist of just mapping frames to each HQ queue as per AC.

Except that declaring one "aggregation queue" also limits you to a
single aggregation session, no big deal in STA mode but punishable by
death in AP mode.

> Anyone know about Marvell's stuff? How about Broadcom?

Consider Broadcom's stuff to work just as yours. Details may differ, but
not by much.

johannes

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [RFC] Redesigning aggregation on mac80211

[Luis R. Rodriguez]

On Mon, Sep 15, 2008 at 12:15 PM, Johannes Berg
<johannes@sipsolutions.net> wrote:
> On Mon, 2008-09-15 at 12:03 -0700, Luis R. Rodriguez wrote:
>
>> I'll start with Atheros hardware and mac80211. Atheros hardware uses 4
>> main HW queues, one for each AC, and we also have one for beacons,
>> another for queued stuff after beacon (aka "Crap After Beacon") where
>> things like queued frames for broadcast / multicast or for frames for
>> stations in power save mode, and I think another hw queue for uapsd.
>> Right now mac80211 queues skb's for drivers into the netdev_queues.
>> Under the current mac80211 design we have 4 netdev_queues, one for
>> each AC and then there is a number of "aggregation queues". We
>> essentially just declare one "aggregation" queue as we do our own
>> queue mapping internally to our own hardware queues -- and that
>> consist of just mapping frames to each HQ queue as per AC.
>
> Except that declaring one "aggregation queue" also limits you to a
> single aggregation session, no big deal in STA mode but punishable by
> death in AP mode.

Or IBSS/Mesh although I think only IBSS speaks aggregation as per the
specs so far.

  Luis

Re: [RFC] Redesigning aggregation on mac80211

[Ivo van Doorn]

Hi,

> We're considering redesigning how aggregation works on mac80211 to
> ensure its vendor neutral and to remove as much as redundant code in
> drivers. We'd like some feedback from people and driver developers,
> even those who haven't *yet* posted their new shiny 11n drivers, like
> Marvell or Ralink driver developers.

Well technically speaking there are 2 Ralink drivers in the Kernel already
which *should* support aggregation (rt61 and rt73).

I'm sorry I have not read the entire thread on the new aggregation support
yet, but I'll write what I know from rt61/rt73/rt2800 here anyway. :)

> Ivo, are you familiar yet with how aggregation works on Ralink
> hardware? Does the new 11n hardware require firmware? Specifically
> we're curious about how the hardware queues are managed.

Ralink defines 4 queues, 1 Management queue (can be abused as regular TX queue)
and a Beacon queue.

There is no such thing as seperate aggregation queues, as soon as the device
wants to perform aggregation it grabs a normal entry from the TX queue,
and assignes all frames to it.

rt61: 5 addresses can be provided for Aggrated frames, I am not sure
if the first is a header only frame or not.
rt73: Not sure what should be done
rt2800: 2 addresses can be provided, where the first is a header/descriptor
only address. In the second address frames can be provided.

Because the aggregated frame is created directly into a TX entry, no other
frames should go through that queue until the aggregated frame has been
completed.

I'm afraid that is currently all the information I got at this time,
It is only a little, but I never got around to fully look into the aggregation support,
although I definately want it implemented in the drivers to increase the
performance a bit. :)

Ivo

Re: [RFC] Redesigning aggregation on mac80211

[Tomas Winkler]

On Tue, Sep 16, 2008 at 9:05 PM, Ivo van Doorn <ivdoorn@gmail.com> wrote:
> Hi,
>
>> We're considering redesigning how aggregation works on mac80211 to
>> ensure its vendor neutral and to remove as much as redundant code in
>> drivers. We'd like some feedback from people and driver developers,
>> even those who haven't *yet* posted their new shiny 11n drivers, like
>> Marvell or Ralink driver developers.
>
> Well technically speaking there are 2 Ralink drivers in the Kernel already
> which *should* support aggregation (rt61 and rt73).
>
> I'm sorry I have not read the entire thread on the new aggregation support
> yet, but I'll write what I know from rt61/rt73/rt2800 here anyway. :)
>
>> Ivo, are you familiar yet with how aggregation works on Ralink
>> hardware? Does the new 11n hardware require firmware? Specifically
>> we're curious about how the hardware queues are managed.
>
> Ralink defines 4 queues, 1 Management queue (can be abused as regular TX queue)
> and a Beacon queue.
>
> There is no such thing as seperate aggregation queues, as soon as the device
> wants to perform aggregation it grabs a normal entry from the TX queue,
> and assignes all frames to it.
>
> rt61: 5 addresses can be provided for Aggrated frames, I am not sure
> if the first is a header only frame or not.
> rt73: Not sure what should be done
> rt2800: 2 addresses can be provided, where the first is a header/descriptor
> only address. In the second address frames can be provided.

> Because the aggregated frame is created directly into a TX entry, no other
> frames should go through that queue until the aggregated frame has been
> completed.
>

What type of aggregation is this? AMSDU or AMPDU? In AMPDU you need
somehow handle
out of order resend of the packets.  rt61 and rt2800 looks more like
AMSDU (Not sure) we don't support that in iwlwifi TX side of it yet,
it just doesn't outperform AMPDU aggregation.
Tomas


> I'm afraid that is currently all the information I got at this time,
> It is only a little, but I never got around to fully look into the aggregation support,
> although I definately want it implemented in the drivers to increase the
> performance a bit. :)

Need to find out what kind of aggregation the cards support. For TX
AMSDU we need add more code into mac80211, the AMSDU sub frames have
14 bytes (eth) headers.

Thanks
Tomas

Re: [RFC] Redesigning aggregation on mac80211

[Ivo van Doorn]

On Thursday 18 September 2008, Tomas Winkler wrote:
> On Tue, Sep 16, 2008 at 9:05 PM, Ivo van Doorn <ivdoorn@gmail.com> wrote:
> > Hi,
> >
> >> We're considering redesigning how aggregation works on mac80211 to
> >> ensure its vendor neutral and to remove as much as redundant code in
> >> drivers. We'd like some feedback from people and driver developers,
> >> even those who haven't *yet* posted their new shiny 11n drivers, like
> >> Marvell or Ralink driver developers.
> >
> > Well technically speaking there are 2 Ralink drivers in the Kernel already
> > which *should* support aggregation (rt61 and rt73).
> >
> > I'm sorry I have not read the entire thread on the new aggregation support
> > yet, but I'll write what I know from rt61/rt73/rt2800 here anyway. :)
> >
> >> Ivo, are you familiar yet with how aggregation works on Ralink
> >> hardware? Does the new 11n hardware require firmware? Specifically
> >> we're curious about how the hardware queues are managed.
> >
> > Ralink defines 4 queues, 1 Management queue (can be abused as regular TX queue)
> > and a Beacon queue.
> >
> > There is no such thing as seperate aggregation queues, as soon as the device
> > wants to perform aggregation it grabs a normal entry from the TX queue,
> > and assignes all frames to it.
> >
> > rt61: 5 addresses can be provided for Aggrated frames, I am not sure
> > if the first is a header only frame or not.
> > rt73: Not sure what should be done
> > rt2800: 2 addresses can be provided, where the first is a header/descriptor
> > only address. In the second address frames can be provided.
> 
> > Because the aggregated frame is created directly into a TX entry, no other
> > frames should go through that queue until the aggregated frame has been
> > completed.
> >
> 
> What type of aggregation is this? AMSDU or AMPDU? In AMPDU you need
> somehow handle
> out of order resend of the packets.  rt61 and rt2800 looks more like
> AMSDU (Not sure) we don't support that in iwlwifi TX side of it yet,
> it just doesn't outperform AMPDU aggregation.
> Tomas

It is a bit confusing since comments, variable names are a conflicting eachother
in the legacy driver code. Comments state MSDU where variables are named MPDU
and later the same variable is accompanies with comments regarding MPDU.

I really have to look deeper in the actual implementation to see what the driver
is doing exactly before I can answer the question what aggregation it is using.

> > I'm afraid that is currently all the information I got at this time,
> > It is only a little, but I never got around to fully look into the aggregation support,
> > although I definately want it implemented in the drivers to increase the
> > performance a bit. :)
> 
> Need to find out what kind of aggregation the cards support. For TX
> AMSDU we need add more code into mac80211, the AMSDU sub frames have
> 14 bytes (eth) headers.

Ivo