Re: Consist timestamps within a checkout/clone

["Ævar Arnfjörð Bjarmason" <avarab@gmail.com>]

On Mon, Oct 31 2022, Mark Hills wrote:

> On Mon, 31 Oct 2022, Taylor Blau wrote:
>
>> On Mon, Oct 31, 2022 at 09:21:20PM +0100, Ævar Arnfjörð Bjarmason wrote:
>> > I think you're almost certainly running into the parallel checkout,
>> > which is new in that revision range. Try tweaking checkout.workers and
>> > checkout.thresholdForParallelism (see "man git-config").
>> >
>> > I can't say without looking at the code/Makefile (and even then, I don't
>> > have time to dig here:), but if I had to bet I'd say that your
>> > dependencies have probably always been broken with these checked-in
>> > files, but they happend to work out if they were checked out in sorted
>> > order.
>> >
>> > And now with the parallel checkout they're not guaranteed to do that, as
>> > some workers will "race ahead" and finish in an unpredictable order.
>> 
>> Doesn't checkout.thresholdForParallelism only matter when
>> checkout.workers != 1?
>> 
>> So what you wrote seems like a reasonable explanation, but only if the
>> original reporter set checkout.workers to imply the non-sequential
>> behavior in the first place.
>> 
>> That said...
>> 
>>   - I also don't know off-hand of a place where we've defined the order
>>     where Git will checkout files in the working copy. So depending on
>>     that behavior isn't a safe thing to do.
>> 
>>   - Committing build artifacts into your repository is generally
>>     discouraged.
>
> If it's undefined and never implemented this is reasonable.
>
> But "generally" is a caveat, so while I agree with the statement it also 
> implies there's valid cases outside of that. Ones which used to work, too.
>
> Here are some useful cases I have seen for the combination of build rule + 
> checked in file:
>
> - part of a build requires licensed software that's not always available
>
> - part of the build requires large memory that other builders generally do 
>   not have available
>
> - part of the build process uses a different platform or some other system 
>   requirement
>
> - to fetch data eg. from a URL, with a record of the URL/automation but 
>   also a copy of the file as a record and for offline use
>
> So it's useful, to retain repeatable automation but not always build from 
> square one.
>
> Generally discouraged to check in build results yes, but I've found it 
> very practical.
>  
>> So while I'd guess that setting `checkout.workers` back to "1" (if it 
>> wasn't already) will probably restore the existing behavior, counting on 
>> that behavior in the first place is wrong.
>
> I think perhaps the tail is wagging the dog here, though.
>
> It's 'wrong' because it doesn't work; but I haven't seen anything to make 
> me think this is fundamentally or theoretically flawed.
>
> If we had a transactional file system we'd reasonably expect a checkout to 
> be an atomic operation -- same timestamp on the files created in that 
> step. A discrepancy in timestamps would be considered incorrect; it would 
> imply an 'order' to the checkout which, as you say, is order-less.
>
> Sowhat could be the bad outcomes if Git created files stamped with the 
> point in time of the "git checkout"?

I agree that it's practical in some scenarios, including checking in
built assets.

But those that are doing that need to be aware that combining that sort
of thing with source control tends to upend your build system's idea of
the world.

E.g. until recently in git.git we had a po/git.pot in-tree, which is a
"compiled file" (although a plain-text one) that was checked in, and
dealing with that in make's dependency graph was a (minor) pain
sometimes.