Patch-o-matic cleanup

Patch-o-matic cleanup

[Patrick McHardy]

Following (quite late) the decision taken at the netfilter workshop to
clean up the pomng repository by removing old patches and moving some
patches out of the netfilter repository, I've started by removing the
obsolete ipsec* patches and the policy match. Both have been merged
and were pretty outdated.

If noone can name good reasons against it, I will also remove the
following patches in a few days:

- NETLINK: obsolete
- NETMAP: in mainline for a long time
- TRACE: don't recall the reason, but its on my list :)
- comment: in mainline
- conntrack_locking: seriously outdated, if someone wants to resurrect
  this patch he can use the history
- dropped_table: also don't recall the reason
- goto: in mainline
- ipq_vmark: obsoleted by nfnetlink_queue
- mport: mainline multiport supports all of its features
- netfilter-docbook: outdated
- owner-socketlookup: unfixable broken
- pool: obsoleted by ipset
- pptp-conntrack-nat: in mainline now, patches are missing critical
  fixes
- unclean: bad idea

Re: Patch-o-matic cleanup

[Stephen Jones]

Patrick McHardy wrote:
> Following (quite late) the decision taken at the netfilter workshop to
> clean up the pomng repository by removing old patches and moving some
> patches out of the netfilter repository, I've started by removing the
> obsolete ipsec* patches and the policy match. Both have been merged
> and were pretty outdated.
> 
> If noone can name good reasons against it, I will also remove the
> following patches in a few days:
> 

< snip >

> - pptp-conntrack-nat: in mainline now, patches are missing critical
>   fixes
Hi Patrick,

Is pptp-conntrack-nat functionality going to be included in the 2.4.x 
mainline also?  Otherwise, the patches still appear to be necessary for 
2.4.x kernels.  Thanks for all of your hard work!

SJ

> - unclean: bad idea
> 
> 
> 

Re: Patch-o-matic cleanup

[Patrick McHardy]

Stephen Jones wrote:
> Patrick McHardy wrote:
> 
>> - pptp-conntrack-nat: in mainline now, patches are missing critical
>>   fixes
> 
> Hi Patrick,
> 
> Is pptp-conntrack-nat functionality going to be included in the 2.4.x
> mainline also?  Otherwise, the patches still appear to be necessary for
> 2.4.x kernels.

No, 2.4 is in absolute stable mode. The pptp patches for 2.4 are missing
a number of important bugfixes, I wouldn't recommend using them in their
present form. We don't have the time to maintain old versions forever,
which is why we decided to remove this old stuff.

Re: Patch-o-matic cleanup

[Stephen Jones]

Patrick McHardy wrote:
> Stephen Jones wrote:
> 
>>Patrick McHardy wrote:
>>
>>
>>>- pptp-conntrack-nat: in mainline now, patches are missing critical
>>>  fixes
>>
>>Hi Patrick,
>>
>>Is pptp-conntrack-nat functionality going to be included in the 2.4.x
>>mainline also?  Otherwise, the patches still appear to be necessary for
>>2.4.x kernels.
> 
> 
> No, 2.4 is in absolute stable mode. The pptp patches for 2.4 are missing
> a number of important bugfixes, I wouldn't recommend using them in their
> present form. We don't have the time to maintain old versions forever,
> which is why we decided to remove this old stuff.
> 
> 
Ok, good enough, and completely understandable.  Unfortunately, for me, 
there are still quite a few items (not netfilter related) that work on 
2.4.x that do not yet work on 2.6.x series kernels that force me into a 
holding pattern on the 2.4.3x kernels for now.  I'll keep an archive of 
the last known working iptables + pom-ng snapshot if I need to recompile 
again in the future.  The pptp conntrack functionality has been "mostly" 
working for us for several years now. The problems with it are rare and 
tend to occur when the linux firewall is managing many concurrent 
tunnels, otherwise it appears to work just fine. Thank you for your reply.

SJ

Re: Patch-o-matic cleanup

[Krzysztof Oledzki]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 599 bytes --]



On Mon, 3 Apr 2006, Patrick McHardy wrote:

> Following (quite late) the decision taken at the netfilter workshop to
> clean up the pomng repository by removing old patches and moving some
> patches out of the netfilter repository, I've started by removing the
> obsolete ipsec* patches and the policy match. Both have been merged
> and were pretty outdated.
>
> If noone can name good reasons against it, I will also remove the
> following patches in a few days:
<CUT>
> - goto: in mainline
Not in 2.4 and version from pom-ng is 2.4-only.

Best regards,

 					Krzysztof Olędzki

Re: Patch-o-matic cleanup

[Samuel Díaz García]

Patrick, after the last "cleanup" (18/9/2005?), many patches were removed
and 2.4/2.6 compatibility broken, for example the "condition" match over
2.6 or CONNMARK over 2.4.

I think, that if devteam are going to invert time to "cleanup" pomng,
would be fantastic that old ussefull and broken patches (or teorically
merged into kernel main base) must be reviewed.

For example, you put pptp_conntrack to be deleted, but I used it into 2.4
kernel versions, where it isn't into mainline.

That are the ticks I think you will need to take care of.

Good work, regards.

-- 
Samuel Díaz García
ArcosCom Wireless, S.L.L.

CIF: B11828068
c/ Romero Gago, 19
Arcos de la Frontera
11630 - Cadiz

http://www.arcoscom.com

mailto:samueldg@arcoscom.com
msn: samueldg@arcoscom.com

Tlfn.: 956 70 13 15
Fax:   956 70 34 83


El Lun, 3 de Abril de 2006, 20:12, Patrick McHardy escribió:
> Following (quite late) the decision taken at the netfilter workshop to
> clean up the pomng repository by removing old patches and moving some
> patches out of the netfilter repository, I've started by removing the
> obsolete ipsec* patches and the policy match. Both have been merged
> and were pretty outdated.
>
> If noone can name good reasons against it, I will also remove the
> following patches in a few days:
>
> - NETLINK: obsolete
> - NETMAP: in mainline for a long time
> - TRACE: don't recall the reason, but its on my list :)
> - comment: in mainline
> - conntrack_locking: seriously outdated, if someone wants to resurrect
>   this patch he can use the history
> - dropped_table: also don't recall the reason
> - goto: in mainline
> - ipq_vmark: obsoleted by nfnetlink_queue
> - mport: mainline multiport supports all of its features
> - netfilter-docbook: outdated
> - owner-socketlookup: unfixable broken
> - pool: obsoleted by ipset
> - pptp-conntrack-nat: in mainline now, patches are missing critical
>   fixes
> - unclean: bad idea
>
>

Re: Patch-o-matic cleanup

[Patrick McHardy]

Samuel Díaz García wrote:
> Patrick, after the last "cleanup" (18/9/2005?), many patches were removed
> and 2.4/2.6 compatibility broken, for example the "condition" match over
> 2.6 or CONNMARK over 2.4.
> 
> I think, that if devteam are going to invert time to "cleanup" pomng,
> would be fantastic that old ussefull and broken patches (or teorically
> merged into kernel main base) must be reviewed.

Yes, that was the idea, most useful patches will be merged, a few useful
but obscure will be kept and all others will be deleted. There will also
be a new feature to download patches from external sources when using
patch-o-matic, so if anyone wants to keep maintaining a specific patch,
we can just include a line in the sources.list.

> For example, you put pptp_conntrack to be deleted, but I used it into 2.4
> kernel versions, where it isn't into mainline.
> 
> That are the ticks I think you will need to take care of.

Most effort goes into 2.6 these days, and nobody seems to have the time
to maintain old versions. The PPtP-helper in pom is missing important
bugfixes, so as long as noone fixes those, it is better to remove it
than to distribute versions known to have critical bugs IMO.

This is the time for users to step up :)

Re: Patch-o-matic cleanup

[Carl-Daniel Hailfinger]

Hi,

Patrick McHardy schrieb:
> If noone can name good reasons against it, I will also remove the
> following patches in a few days:
> 
> - unclean: bad idea

Please leave that one in. I know it's a bad idea, but in non-production
setups LOGging all packets matched by unclean gives interesting insights
into network traffic. So it's a valuable educational tool.



Regards,
Carl-Daniel
-- 
http://www.hailfinger.org/

Re: Patch-o-matic cleanup

[Samuel Díaz García]

Patrick, after the last "cleanup" (18/9/2005?), many patches were removed
and 2.4/2.6 compatibility broken, for example the "condition" match over
2.6 or CONNMARK over 2.4.

I think, that if devteam are going to invert time to "cleanup" pomng,
would be fantastic that old ussefull and broken patches (or teorically
merged into kernel main base) must be reviewed.

For example, you put pptp_conntrack to be deleted, but I used it into 2.4
kernel versions, where it isn't into mainline.

That are the ticks I think you will need to take care of.

Good work, regards.

Patrick McHardy escribió:
> Following (quite late) the decision taken at the netfilter workshop to
> clean up the pomng repository by removing old patches and moving some
> patches out of the netfilter repository, I've started by removing the
> obsolete ipsec* patches and the policy match. Both have been merged
> and were pretty outdated.
> 
> If noone can name good reasons against it, I will also remove the
> following patches in a few days:
> 
> - NETLINK: obsolete
> - NETMAP: in mainline for a long time
> - TRACE: don't recall the reason, but its on my list :)
> - comment: in mainline
> - conntrack_locking: seriously outdated, if someone wants to resurrect
>   this patch he can use the history
> - dropped_table: also don't recall the reason
> - goto: in mainline
> - ipq_vmark: obsoleted by nfnetlink_queue
> - mport: mainline multiport supports all of its features
> - netfilter-docbook: outdated
> - owner-socketlookup: unfixable broken
> - pool: obsoleted by ipset
> - pptp-conntrack-nat: in mainline now, patches are missing critical
>   fixes
> - unclean: bad idea
> 
> 
> 

-- 
    Samuel Díaz García
     Director Gerente
ArcosCom Wireless, S.L.L.

CIF: B11828068
c/ Romero Gago, 19
Arcos de la Frontera
11630 - Cadiz

http://www.arcoscom.com

mailto:samueldg@arcoscom.com
msn: samueldg@arcoscom.com

Móvil: 651 93 72 48
Tlfn.: 956 70 13 15
Fax:   956 70 34 83

Update: Patch-o-matic cleanup

[Patrick McHardy]

Patrick McHardy wrote:
> Following (quite late) the decision taken at the netfilter workshop to
> clean up the pomng repository by removing old patches and moving some
> patches out of the netfilter repository, I've started by removing the
> obsolete ipsec* patches and the policy match. Both have been merged
> and were pretty outdated.
> 
> If noone can name good reasons against it, I will also remove the
> following patches in a few days:
> [...]

After receiving multiple "please keep this patch" mails, it seems I
started the wrong way.

Removing old patches is part of a greater plan to reduce the content of
pom to only those things the netfilter team has an interest in
maintaining, and most of these things will be merged in not too long
time. _All_ other patches will be removed and the "runme" tool will
be enhanced by an apt-get like mechanism to download patches from
external sources, so people interested in keeping patches around can
maintain them themselves and publish them somewhere for others to use.
This should benefit both us and the authors since they usually maintain
their own versions of the patches anyway and periodically send us
updates to include in pom, which unnecessarily cost time for everyone
involved.

This is the full list of patches to be removed. Instructions how to set
up a pomng repository are at the end of this mail, I'll commit the code
soon.

- ACCOUNT
- IPMARK
- NETLINK
- NETMAP
- TARPIT
- TCPLAG
- TRACE
- XOR
- account
- comment
- connlimit
- connrate
- conntrack_locking
- cuseeme-nat
- dropped-table
- expire
- fuzzy
- geoip
- goto
- ip_queue_vwmark
- ipp2p
- mport
- nat-reservations
- netfilter-docbook
- osf
- owner-socketlookup
- pool
- pptp-conntrack-nat
- psd
- time
- tproxy

The patches will be removed in 30 days, authors or other people that
want to maintain them in an external repository are welcome to send
the URL to be included in the distrubuted sources.list.

How to set up a repository:

The repository must be accessible in a way supported by curl (HTTP or
FTP for example). The name of each patchlet is listed in a file named
"index". Each patchlet must be contained in a .tar.gz file, which
contains a directory named like the patchlet itself. The directories
contents are similar to those in patchlets/ today, but the Repository
specified in the info file must be "external".

So to create a repository from one of the patches currently in pom:

- cd patchlets
- edit $patch/info, change repository to "external"
- tar cz $patch > /wwwroot/pom-repo/$patch.tar.gz
- echo $patch >> /wwwroot/pom-repo/index

As I said above, I'll commit the code for this soon.

Re: Update: Patch-o-matic cleanup

[Carl-Daniel Hailfinger]

Hi Patrick,

Patrick McHardy schrieb:
> 
> After receiving multiple "please keep this patch" mails, it seems I
> started the wrong way.
> 
> Removing old patches is part of a greater plan to reduce the content of
> pom to only those things the netfilter team has an interest in
> maintaining, and most of these things will be merged in not too long
> time. _All_ other patches will be removed and the "runme" tool will
> be enhanced by an apt-get like mechanism to download patches from
> external sources[...]

But why keep a few patches in pom and throw out the rest? Wouldn't it
make sense to reduce pom to a simple script and throw out all patches?
The few patches maintained by the netfilter team could be moved to a
separate pom-hotfix package and be automatically downloaded by the
pom script.

OTOH, having a pom-ng that can be applied without a network connection
was very helpful for me in the past. So if you are really going to
discontinue pom-ng as is, I offer to take over maintenance. Fetching
the different packages from locations in a list and assembling them
into one pom-full package automatically can't be that difficult.

Regards,
Carl-Daniel
-- 
http://www.hailfinger.org/

Re: Update: Patch-o-matic cleanup

[Patrick McHardy]

Carl-Daniel Hailfinger wrote:
> Hi Patrick,
> 
> Patrick McHardy schrieb:
> 
>>After receiving multiple "please keep this patch" mails, it seems I
>>started the wrong way.
>>
>>Removing old patches is part of a greater plan to reduce the content of
>>pom to only those things the netfilter team has an interest in
>>maintaining, and most of these things will be merged in not too long
>>time. _All_ other patches will be removed and the "runme" tool will
>>be enhanced by an apt-get like mechanism to download patches from
>>external sources[...]
> 
> 
> But why keep a few patches in pom and throw out the rest? Wouldn't it
> make sense to reduce pom to a simple script and throw out all patches?
> The few patches maintained by the netfilter team could be moved to a
> separate pom-hotfix package and be automatically downloaded by the
> pom script.

We could do that, but currently we want to keep a few patches while
minimizing our effort, the easiest way to do that is to just keep
them where they are.

> OTOH, having a pom-ng that can be applied without a network connection
> was very helpful for me in the past. So if you are really going to
> discontinue pom-ng as is, I offer to take over maintenance. Fetching
> the different packages from locations in a list and assembling them
> into one pom-full package automatically can't be that difficult.

I'm not sure if we want to discontinue official pom-releases entirely,
but you are of course free to maintain and release version which
include external patches.

Re: Update: Patch-o-matic cleanup

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 433 bytes --]

Thanks Patrick, for taking care of this long-over-due issue.
-- 
- Harald Welte <laforge@netfilter.org>                 http://netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Type: application/pgp-signature, Size: 191 bytes --]

Re: Update: Patch-o-matic cleanup

[Thomas Jarosch]

Hi Patrick,

> Patrick McHardy wrote:
> How to set up a repository:
>
> The repository must be accessible in a way supported by curl (HTTP or
> FTP for example). The name of each patchlet is listed in a file named
> "index". Each patchlet must be contained in a .tar.gz file, which
> contains a directory named like the patchlet itself. The directories
> contents are similar to those in patchlets/ today, but the Repository
> specified in the info file must be "external".

I've set up a repository for ACCOUNT here:

http://www.intra2net.com/de/produkte/opensource/ipt_account/

Hope everything is ok.

The tarball is currently named ACCOUNT.tar.gz.
Is there a way to add a version number to it?
Or is it safe to include some kind of "Changelog"
file in the distribution?

btw: "README.new_patches" doesn't know anything
about the "Repository: external" stuff yet.

Cheers,
Thomas

Re: Update: Patch-o-matic cleanup

[Patrick McHardy]

Thomas Jarosch wrote:
> Hi Patrick,
> 
>> Patrick McHardy wrote:
>> How to set up a repository:
>>
>> The repository must be accessible in a way supported by curl (HTTP or
>> FTP for example). The name of each patchlet is listed in a file named
>> "index". Each patchlet must be contained in a .tar.gz file, which
>> contains a directory named like the patchlet itself. The directories
>> contents are similar to those in patchlets/ today, but the Repository
>> specified in the info file must be "external".
> 
> 
> I've set up a repository for ACCOUNT here:
> 
> http://www.intra2net.com/de/produkte/opensource/ipt_account/
> 
> Hope everything is ok.

Thanks. I'll add all repositories people have mailed to me in the
next days and will start removing things. I'll give it a short
test run then.

> The tarball is currently named ACCOUNT.tar.gz.
> Is there a way to add a version number to it?
> Or is it safe to include some kind of "Changelog"
> file in the distribution?

Changelog file (or just the info file) should be fine. There is no
support for seperate versions yet .. pomng itself doesn't know
anything about versions, so I'm not sure how cleanly this could
be done.

> btw: "README.new_patches" doesn't know anything
> about the "Repository: external" stuff yet.

I'll update that as well.

Re: Update: Patch-o-matic cleanup

[Patrick McHardy]

Patrick McHardy wrote:
> This is the full list of patches to be removed. Instructions how to set
> up a pomng repository are at the end of this mail, I'll commit the code
> soon.
> 
> - ACCOUNT
> - IPMARK
> - NETLINK
> - NETMAP
> - TARPIT
> - TCPLAG
> - TRACE
> - XOR
> - account
> - comment
> - connlimit
> - connrate
> - conntrack_locking
> - cuseeme-nat
> - dropped-table
> - expire
> - fuzzy
> - geoip
> - goto
> - ip_queue_vwmark
> - ipp2p
> - mport
> - nat-reservations
> - netfilter-docbook
> - osf
> - owner-socketlookup
> - pool
> - pptp-conntrack-nat
> - psd
> - time
> - tproxy
> 
> The patches will be removed in 30 days, authors or other people that
> want to maintain them in an external repository are welcome to send
> the URL to be included in the distrubuted sources.list.

I've now removed all patches that didn't find a new maintainer or
which already have an external repository. I'll keep the ones where
people are still setting up the repository for a couple more days,
these are:

IPMARK                  Grzegorz Janoszka <Grzegorz@Janoszka.pl>
TARPIT                  Krzysztof Oldzki <olenf@ans.pl>
connlimit               Krzysztof Oldzki <olenf@ans.pl>
ipp2p                   Krzysztof Oldzki <olenf@ans.pl>
time                    Krzysztof Oldzki <olenf@ans.pl>

Please send me the URLs of the repositories when they are up. Anyone
wanting to take care of an already removed patchlet is of course also
invited to send me a URL.

Re: Update: Patch-o-matic cleanup

[Carl-Daniel Hailfinger]

Hi,

Patrick McHardy wrote:
> 
> I've now removed all patches that didn't find a new maintainer or
> which already have an external repository. I'll keep the ones where
> people are still setting up the repository for a couple more days,
> these are:

would it be possible for you to generate a list of the patches which
didn't find a new maintainer and have no external repository in sight?

Apologies if I overlooked such a list.

Regards,
Carl-Daniel
-- 
http://www.hailfinger.org/

Re: Update: Patch-o-matic cleanup

[Patrick McHardy]

Carl-Daniel Hailfinger wrote:
> Hi,
> 
> Patrick McHardy wrote:
> 
>>I've now removed all patches that didn't find a new maintainer or
>>which already have an external repository. I'll keep the ones where
>>people are still setting up the repository for a couple more days,
>>these are:
> 
> 
> would it be possible for you to generate a list of the patches which
> didn't find a new maintainer and have no external repository in sight?

Unfortunately I just deleted my list together with the patches.
But anything is better than cleaning up ipt_recent, so here is
a new one :)

- account
- ACCOUNT
- comment
- connrate
- conntrack_locking
- cuseeme-nat
- dropped-table
- expire
- fuzzy
- goto
- h323-conntrack-nat
- ip_queue_vwmark
- MARK-operations
- mport
- nat-reservations
- netfilter-docbook
- NETLINK
- NETMAP
- nth
- osf
- owner-socketlookup
- pool
- pptp-conntrack-nat
- psd
- quota
- random
- TCPLAG
- tproxy
- TRACE
- unclean
- XOR

Some of these are already in mainline (or something comparable) or will
be soon:

- comment
- goto
- h323
- ip_queue_vwmark
- MARK-operations
- mport
- nth
- random
- quota
- pptp

The tproxy guys will probably set up their own repository, at least
thats what they said at the last workshop.