net-2.6.24 GIT state

net-2.6.24 GIT state

[David Miller]

Ok, as is clear for some the net-2.6.24 tree has been cut
and is at:

	kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6.24.git

Contents:

1) napi_struct V6 :-)
2) LRO with EHEA and Myri10g ports
3) mac80211 merge with John Linville
4) Some netlink bits from Thomas Graf
5) VETH driver

For shits and grins I build a powerpc cross environment and used
this to fix up the EHEA build fallout.  That means nobody with
these chips did any testing at all, please do so if you can.

SIS190 didn't build because it had a half-done NAPI implementation
(actually closer to not done) and it did netif_poll_{enable,disable}()
(and wrongly) in it's open and close routines.  Doing this
unconditionally, even when non-NAPI, is just wrong and therefore since
the NAPI related bits were miniscule I just ripped out all of it to
fix that.

Natsemi and pasemi_mac were missing a few netif_poll_{disable,enable}()
conversions to napi_{enable,disable}().

I found one driver whose conversion got missed, ibm_emac, but I can't
enable that in the powerpc64 build because it's guarded against when
PPC_MERGE is enabled which ppc64 sets unconditionally.

I might take a stab at converting that guy (it's similar to
EHEA and can use multi-queue NAPI quite directly although
the access back up to the parent netdev is a bit tricky)
but if someone can beat me to it... Hint hint!

I'm now doing a powerpc allmodconfig build to try and shake out
any other build regressions added by the net-2.6.24 tree changes.

I'll be going over the TCP bits Ilpo and I have been discussing
then expect me to disappear back to driver hacking :-)  As is,
there should be enough in there for people to play with,
regression test, and improve.

Thanks.

Re: net-2.6.24 GIT state

[Ilpo Järvinen]

On Thu, 9 Aug 2007, David Miller wrote:

> Ok, as is clear for some the net-2.6.24 tree has been cut
> and is at:
> 
> 	kernel.org:/pub/scm/linux/kernel/git/davem/net-2.6.24.git
> 
> I'll be going over the TCP bits Ilpo and I have been discussing

...In case I didn't make it clear previously, you'll notice that there are 
some non-submitted changes rebased as well on the top of it.

> then expect me to disappear back to driver hacking :-)  As is,
> there should be enough in there for people to play with,
> regression test, and improve.

At least these are trivial to take (after rebase, some come nicely even 
from the before tree):

40564051bdb237231e625ba674fdedf6e8373027 [TCP]: Remove num_acked>0 checks from cong.ctrl mods pkts_acked
4c035cd78a6b60710b0dda4f62339df070f761c8 [TCP]: Add tcp_dec_pcount_approx int variant
ed2753b0b463df41c10121ce494d51428047fcbc [TCP]: Move code from tcp_ecn.h to tcp*.c and tcp.h & remove it
c215314c6ea0261865d3df09f375016f3dcadeba [TCP]: Access to highest_sack obsoletes forward_cnt_hint
9e5f432fb247af2f0062c3c57d7f45d511692f26 [TCP] FRTO: remove unnecessary fackets/sacked_out recounting
96001b306c60cb969d456ac70e376220db97e54a [TCP]: Move Reno SACKed_out counter functions earlier
318cf753170367504cfac07ac89e97befb1ca501 [TCP]: Extract DSACK detection code from tcp_sacktag_write_queue().
a2539974b098065ebe02ad11a6411df4f56ba0ed [TCP]: Rexmit hint must be cleared instead of setting it
  - Could be combined with the extrator as it's really a bug fix, once 
    rxmit cnt hint gets dropped, we can add this back as optimization :-).
2fea67411f0c89642fa4abc0490b65c7852a1202 [TCP]: Extracted rexmit hint clearing from the LOST marking code
1fba6548b2ecc2d513981898247472d1183329c5 [TCP]: Add highest_sack seqno, points to globally highest SACK

...I was claiming incorrectly last time that highest_sack stuff needs 
SACK-block validation, it does not as it takes seq from skb->seq, not from 
the receiver like I thought :-). ...Redoing the validator (the improved 
version) on top of net-2.6.24 is on my "high priority list" though, so 
maybe I'll have it already on today.

Left_out removal should be safe too: after my analysis, it seems safe as 
most of the functions are if-modified-then-sync type, some sync always. 
Clean_rtx_queue leaves left_out to old state which is the hardest thing to 
track. However, TCP should always execute fastretrans_alert (that syncs 
near entry) if sacked_out+lost_out is (or was) non-zero, so it shouldn't 
leak stale state anywhere. Not going to fastretrans_alert would be IMHO a 
bug that should be fixed. I found one such case which is fixed in [TCP]: 
Keep state in Disorder also if only lost_out > 0). Also FRTO that is 
called before fastretrans_alert sync at entry, so it shouldn't be a 
problem to it either. 

...Taking IsReno/IsFack removal is probably going to be hardest one, as it 
will conflict a lot. ...Maybe some sed work in patches would reduce # of 
conflicts instead of trying cherry-pick+rebase.

-- 
 i.

Re: net-2.6.24 GIT state

[David Miller]

From: "Ilpo_Järvinen" <ilpo.jarvinen@helsinki.fi>
Date: Thu, 9 Aug 2007 12:50:51 +0300 (EEST)

> At least these are trivial to take (after rebase, some come nicely even 
> from the before tree):
> 
> 40564051bdb237231e625ba674fdedf6e8373027 [TCP]: Remove num_acked>0 checks from cong.ctrl mods pkts_acked
> 4c035cd78a6b60710b0dda4f62339df070f761c8 [TCP]: Add tcp_dec_pcount_approx int variant
> ed2753b0b463df41c10121ce494d51428047fcbc [TCP]: Move code from tcp_ecn.h to tcp*.c and tcp.h & remove it
> c215314c6ea0261865d3df09f375016f3dcadeba [TCP]: Access to highest_sack obsoletes forward_cnt_hint
> 9e5f432fb247af2f0062c3c57d7f45d511692f26 [TCP] FRTO: remove unnecessary fackets/sacked_out recounting
> 96001b306c60cb969d456ac70e376220db97e54a [TCP]: Move Reno SACKed_out counter functions earlier
> 318cf753170367504cfac07ac89e97befb1ca501 [TCP]: Extract DSACK detection code from tcp_sacktag_write_queue().
> a2539974b098065ebe02ad11a6411df4f56ba0ed [TCP]: Rexmit hint must be cleared instead of setting it
>   - Could be combined with the extrator as it's really a bug fix, once 
>     rxmit cnt hint gets dropped, we can add this back as optimization :-).
> 2fea67411f0c89642fa4abc0490b65c7852a1202 [TCP]: Extracted rexmit hint clearing from the LOST marking code
> 1fba6548b2ecc2d513981898247472d1183329c5 [TCP]: Add highest_sack seqno, points to globally highest SACK

Ok, in order to get this started I merged the above patches
into net-2.6.24

I'm exhausted and done for today. :-)

We can work on the other bits next.  Note that patches like the
"Create tcp_sacktag_foo" ones are just mechanical transformations
and thus if the conflicts are huge they can just be reimplemented
by hand.

> ...Taking IsReno/IsFack removal is probably going to be hardest one, as it 
> will conflict a lot. ...Maybe some sed work in patches would reduce # of 
> conflicts instead of trying cherry-pick+rebase.

Yes, such patch hand editing techniques can help in these cases.

Re: net-2.6.24 GIT state

[Ilpo Järvinen]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1326 bytes --]

On Thu, 9 Aug 2007, David Miller wrote:

> From: "Ilpo_Järvinen" <ilpo.jarvinen@helsinki.fi>
> Date: Thu, 9 Aug 2007 12:50:51 +0300 (EEST)
> 
> Ok, in order to get this started I merged the above patches
> into net-2.6.24
> 
> I'm exhausted and done for today. :-)

:-)

> We can work on the other bits next. 

Next step:

http://www.cs.helsinki.fi/u/ijjarvin/tcp-rebase/after-reorder/

...I put some effort to reorganization and there's the result, please 
_don't_ take the new SACK block-validator yet as I haven't yet put it 
under any testing (it's the only difference this reordering of patches 
seems to have introduced :-)). The first five commits touch left_out (4 
latest fix bug and "flaws" I made in the first commit), I could combine 
them into a single one if you want to.

...Notice that part of the next step includes work that hasn't yet been 
posted to netdev at all (was from my not yet submitted wip portion of 
it). Would you prefer me to post them using the usual procedure?

> > ...Taking IsReno/IsFack removal is probably going to be hardest one, as it 
> > will conflict a lot. ...Maybe some sed work in patches would reduce # of 
> > conflicts instead of trying cherry-pick+rebase.
> 
> Yes, such patch hand editing techniques can help in these cases.

...it helped a lot in the process :-).


-- 
 i.

Re: net-2.6.24 GIT state

[David Miller]

From: "Ilpo_Järvinen" <ilpo.jarvinen@helsinki.fi>
Date: Thu, 9 Aug 2007 17:40:07 +0300 (EEST)

> Next step:
> 
> http://www.cs.helsinki.fi/u/ijjarvin/tcp-rebase/after-reorder/
> 
> ...I put some effort to reorganization and there's the result, please 
> _don't_ take the new SACK block-validator yet as I haven't yet put it 
> under any testing (it's the only difference this reordering of patches 
> seems to have introduced :-)). The first five commits touch left_out (4 
> latest fix bug and "flaws" I made in the first commit), I could combine 
> them into a single one if you want to.

I applied everything up until the SACK validator to net-2.6.24

Thanks for doing all of this merging work, I very much appreciate
it.

> ...Notice that part of the next step includes work that hasn't yet been 
> posted to netdev at all (was from my not yet submitted wip portion of 
> it). Would you prefer me to post them using the usual procedure?

Everything I hit today which had not been posted before was trivial
fix or a reasonable small cleanup.

Why don't you just post the patches to the list one-by-one from
now on so we can review them in parallel with merging into
net-2.6.24?

Thanks again Ilpo!

Re: net-2.6.24 GIT state

[Ilpo Järvinen]

On Thu, 9 Aug 2007, David Miller wrote:

> I applied everything up until the SACK validator to net-2.6.24

Ok, thanks.

> Everything I hit today which had not been posted before was trivial
> fix or a reasonable small cleanup.

...Yeah, I know that but don't want to give impression that something 
gets "sneaked" in...

> Why don't you just post the patches to the list one-by-one from
> now on so we can review them in parallel with merging into
> net-2.6.24?

Ok, will do. ...I thought that at some point that would be wise way 
forward...


-- 
 i.