Re: [Buildroot] [PATCH 1/2] Revert "Create a .gitignore file in the CANONICAL_O directory"

[Brandon Maier via buildroot <buildroot@buildroot.org>]

Hi Thomas, James,

On Thu Jul 25, 2024 at 6:04 AM UTC, James Hilliard wrote:
> On Wed, Jul 24, 2024 at 11:51 PM Thomas Petazzoni via buildroot
> <buildroot@buildroot.org> wrote:
> >
> > Hello Brandon,
> >
> > On Wed, 24 Jul 2024 21:23:20 +0000
> > Brandon Maier <brandon.maier@collins.com> wrote:
> >
> > > The packages 'python-jsonschema-specifications', 'pydantic-core', and
> > > likely other packages based on Python maturin silently stopped
> > > installing files to the host and target directories after commit
> > > a14c862c08.
> > >
> > > A similar issue was reported by the Conda project[1]. It seems some
> > > build tools scan for gitignore files to decide what files to build
> > > and install.
> > >
> > > I attempted various combinations of gitignore patterns including
> > > mimicking the Conda project's gitignore[2], but the only thing that has
> > > worked reliably is the original `/output` ignore in the root Buildroot
> > > directory.
> >
> > Thanks for working on this. However, it's really messed up. Why would
> > these tools be bothered by a .gitignore file? Do you have some more
> > details?
>
> So it's the exact same reason ripgrep is affected by .gitignore files,
> maturin actually uses the ignore crate from the ripgrep project:
> https://crates.io/crates/ignore
>
> Note that the ignore crate doesn't entirely correctly replicate git's
> ignore logic as otherwise this shouldn't have triggered any issues.

It does seem to replicate git's ignore logic correctly. The issue is
that we download python-jsonschema as a tarball which does not include
the '.git' directory. So when `maturin` tries to use gitignore's, it
does do what git would do, it searches upwards until it finds
Buildroot's '.git' directory and treats it as the repo.

It seems to me then the fix is one of
- Buildroot needs to switch to using the git repo directly
- Buildroot needs to create an empty git repo in build/python-jsonschema/
- Maturin needs to stop searching for git repos at its pyproject
  directory, which James' Pull Request is doing

And I agree fixing it in Maturin is the correct place. As that will also
fix it for anyone using the tarballs, like Conda. Though it does make me
paranoid about what other build tools out there might be expecting a git
repo and then silently falling back to the Buildroot repo...

What was surprising to me is that the `/output` pattern in Buildroot's
root gitignore doesn't cause this problem. This appears to be a bug in
ripgrep's logic, and I found this long thread that seems related. So if
this ever gets fixed then these packages will be broken again.

https://github.com/BurntSushi/ripgrep/issues/829

>
> >
> > I think I'm going to apply the revert to fix the immediate build
> > issues, but I would really really like to fix the real problem, and
> > allow us to generate this .gitignore file in the output directory.
>
> Yeah, I've been working on a fix but it has some issues, maybe
> someone can help fix the tests:
> https://github.com/PyO3/maturin/pull/2158

Thank you James for looking at this so quickly,
Brandon

>
> >
> > Thomas
> > --
> > Thomas Petazzoni, co-owner and CEO, Bootlin
> > Embedded Linux and Kernel engineering and training
> > https://bootlin.com
> > _______________________________________________
> > buildroot mailing list
> > buildroot@buildroot.org
> > https://lists.buildroot.org/mailman/listinfo/buildroot
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot