Re: [PATCH 1/2] diffcore-rename: support rename cache

["Nguyen Thai Ngoc Duy" <pclouds@gmail.com>]

On 11/8/08, Jeff King <peff@peff.net> wrote:
> On Sat, Nov 08, 2008 at 10:24:10AM +0100, Yann Dirson wrote:
>
>  > Well, it could be that an out-of-pathspec pair would have a better
>  > score than an in-pathspec one.  Maybe cache recording should be turned
>  > off when doing pathspec limitation ?
>
>
> One thing I notice is that the cache works at the level of "here is the
>  best rename for this commit." Maybe it could go down a level and say
>  "here is the inexact rename score between these blobs". Then you would
>  still find the best score between two blobs each time, but save the
>  really computationally intensive part (which is comparing the actual
>  _content_ of the blobs).
>  That should work in the face of path limiting or any other option,
>  because it is caching something immutable: this is the similarity score
>  between two pieces of content. And then you get arbitrary tree-to-tree
>  speedups for free, since such a cache would be valid for every commit.

I did that and realized the cost was not from each diff, in
--find-copies-harder case, but from the number of diffs you had to do.
Even with exact matching on linux-2.6.git, it could take significant
time (it was about 5 minutes in no-cache case, 1 minute without exact
match cache, and less than 1 sec if everything is cached).

>
>  The downsides are:
>
>   - your cache is potentially bigger, since you are caching the score of
>    every pair you look at, instead of just "good" pairs (OTOH, you are
>    not doing a per-commit cache, which helps reduce the size)

It is huge if you accidentially add --find-copies-harder to your
command, considering that every new file will be compared against
every files in tree (about 25k).

>   - you can still "lie" about a score to pre-seed imported SVN renames,
>    but such lying will actually apply to all commits.
-- 
Duy