Re: Fixing ipkg-make-index slowness

[Richard Purdie <rpurdie@rpsys.net>]

Paul,

On Fri, 2007-01-05 at 09:33 +0200, Paul Sokolovsky wrote:
> 1. md5sum thrashing
> 
> In the summer, RP introduced patch for i-m-i ironically called
> index_speedup.patch:

There is nothing ironic about it, it does speed up the process ;-).
md5suming the files was faster than actually parsing them.

> Richard, what were the reasons for such conservative file matching?
> Filename matching should be just enough, as per OE convention, any
> package source changes leading to changes in the package metadata must
> lead to bumping of package recipe's PR, and that in turn updates package
> filename. Whoever don't follow PR update convention either call for
> trouble, and no checks could really help them, or know what they do
> (like bother to rebuild Packages from scratch). In this regard,
> checking ipk size is the great convenience for adventurous, because
> it's really high probability that update of any package metadata will
> lead to change of file size due to compression. In other words, I
> propose to remove md5sum check.

See Koen's reply.

The way forward is to store more information in the index, namely the
last modified timestamp for the file. If you add that and compare
timestamp+size, you can safely skip the md5sum check.

I can't remember why I didn't do that, it could have been a time thing.
It does mean adding a new field to the Packages file and that could also
have had implications elsewhere, I can't remember...

Richard