[Buildroot] Flutter-engine package help wanted.

[Buildroot] Flutter-engine package help wanted.

[Adam Duskett]

[-- Attachment #1.1: Type: text/plain, Size: 3888 bytes --]

Hello! I have a task to port Flutter to Buildroot and to make the packages
suitable for the master branch.

I have talked with Yann in IRC about the problems facing the first package,
flutter-engine [https://github.com/flutter/engine]. However, after another
full
day of hacking away at the package, it's time to talk about a significant
problem I must overcome.

Here is a summary of the issue I am facing and the main reason for the
email:

Flutter is developed and maintained by Google.
Of course, it stands to reason that Flutter does not use a conventional
configure/build system. Oh no, it couldn't be that easy. Instead, Flutter
uses
gclient and gn from the chromium depot_tools repository, which contains all
sorts of exciting utilities. One of which is gclient.

What is gclient? From:
https://www.chromium.org/developers/how-tos/depottools

gclient is a Python script to manage a workspace of modular dependencies
that
are each checked out independently from different subversion or git
repositories. Features include:

  - Dependencies can be specified on a per-OS basis.
  - Dependencies can be specified relative to their parent dependency.
  - Variables can be used to abstract concepts.
  - Hooks can be specified to be run after a checkout.
  - .gclient and DEPS are Python scripts. You can hack in easily or add
     additional configuration data.

.gclient file: It's the primary file. It is, in fact, a Python script. It
specifies the following variables:

  - solutions: an array of dictionaries specifying the projects that will be
               fetched.
  - hooks: additional hooks to be run when this meta checkout is synced.
  - target_os: an optional array of (target) operating systems to fetch
               OS-specific dependencies for.
  - cache_dir: Primarily for bots, multiple working sets use a single git
               cache.


Why is this a significant issue?

The release tarballs from https://github.com/flutter/engine are in no state
to
compile! No, they are only for the use of gclient to download and build a
source directory structure suitable to build the Flutter engine! If you
download, extract and attempt to run
./tools/gn --no-goma --no-prebuilt-dart-sdk,
you receive the error message:
No such file or directory: 'flutter/flutter/third_party/gn/gn.'

But wait! Wasn't the gn binary just called? No, that's a wrapper in the
Flutter
source tree that formats arguments to call the real gn binary. The real gn
is
not provided in the tarball but is downloaded via gclient
(among many other repositories.)

The above leads to the question: What do we do about this? We can't use the
existing download infrastructure to download the tarball from Git Hub, as
that
isn't meant for us. The gclient download works if called out directly via a
pre_download hook; however, this leads to other issues in recreating a
consistent tarball because gclient writes the date to various files all
around
the directory, thus changing the shasum every time I attempt to generate a
tarball. I can sed the dates in a flutter-engine-gen-tarball script I've
made,
which solves the problem; however, there is one last hurdle to overcome:
When I
compress the directory created by gclient, any .git folder in a subdirectory
also changes when running mk_tar_gz from the script. The .git directories,
of
course, can be removed, and that DOES create a reproducible tarball, but the
build system relies on the .git directories to fill in a bunch of
information
before configuring, causing failures.

I won't ask the best course of action because anything we decide will be
pretty
ugly, thanks to Google and its decisions. Instead, I will ask, what is the
least-bad solution that can be upstreamed to the master branch?

If you want to play around with any of this, you can clone
git@gitlab.com:aduskett/buildroot-flutter.git and build the
flutter_defconfig.

Thanks!
Adam

[-- Attachment #1.2: Type: text/html, Size: 4389 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Flutter-engine package help wanted.

[Peter Korsgaard]

>>>>> "Adam" == Adam Duskett <aduskett@gmail.com> writes:

 > Hello! I have a task to port Flutter to Buildroot and to make the packages
 > suitable for the master branch.

[snip lots of details]

Gaah, sounds like a big mess :/ Oh well, I guess that is life. Good luck
with it ;)

I take it that you have already looked at
https://github.com/sony/flutter-embedded-linux and the meta-flutter
layer? From a quick look it seems to largely do the same (but skip the
reproducible tarball).

I take it that you have already played around with Flutter? How useful
(features, but also size and build time) is it in the context of
Buildroot?

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Flutter-engine package help wanted.

[Adam Duskett]

[-- Attachment #1.1: Type: text/plain, Size: 1138 bytes --]

Hey Peter!

One thought I had that I didn’t bring up in the first email: Another
possible solution is to create and upload a tarball to the buildroot
sources server, but each tarball is around 2.1GiB in size!

I have been using meta-flutter to port the package to Buildroot, but
haven’t quite got it to compile yet!

Adam
On Sat, Jul 15, 2023 at 9:24 AM Peter Korsgaard <peter@korsgaard.com> wrote:

> >>>>> "Adam" == Adam Duskett <aduskett@gmail.com> writes:
>
>  > Hello! I have a task to port Flutter to Buildroot and to make the
> packages
>  > suitable for the master branch.
>
> [snip lots of details]
>
> Gaah, sounds like a big mess :/ Oh well, I guess that is life. Good luck
> with it ;)
>
> I take it that you have already looked at
> https://github.com/sony/flutter-embedded-linux and the meta-flutter
> layer? From a quick look it seems to largely do the same (but skip the
> reproducible tarball).
>
> I take it that you have already played around with Flutter? How useful
> (features, but also size and build time) is it in the context of
> Buildroot?
>
> --
> Bye, Peter Korsgaard
>

[-- Attachment #1.2: Type: text/html, Size: 1837 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Flutter-engine package help wanted.

[Peter Korsgaard]

>>>>> "Adam" == Adam Duskett <aduskett@gmail.com> writes:

 > Hey Peter!
 > One thought I had that I didn’t bring up in the first email: Another
 > possible solution is to create and upload a tarball to the buildroot
 > sources server, but each tarball is around 2.1GiB in size!

So far the s.b.o is just a fallback and something required for an active
package. I would prefer to keep it like that if at all possible.


 > I have been using meta-flutter to port the package to Buildroot, but
 > haven’t quite got it to compile yet!

OK.

-- 
Bye, Peter Korsgaard
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Flutter-engine package help wanted.

[Arnout Vandecappelle via buildroot]

On 15/07/2023 05:02, Adam Duskett wrote:
> Hello! I have a task to port Flutter to Buildroot and to make the packages
> suitable for the master branch.
> 
> I have talked with Yann in IRC about the problems facing the first package,
> flutter-engine [https://github.com/flutter/engine 
> <https://github.com/flutter/engine>]. However, after another full
> day of hacking away at the package, it's time to talk about a significant
> problem I must overcome.
> 
> Here is a summary of the issue I am facing and the main reason for the email:
> 
> Flutter is developed and maintained by Google.
> Of course, it stands to reason that Flutter does not use a conventional
> configure/build system. Oh no, it couldn't be that easy. Instead, Flutter uses
> gclient and gn from the chromium depot_tools repository, which contains all
> sorts of exciting utilities. One of which is gclient.

  My first reaction was: oh, we'll need specific download infrastructure like 
for go and cargo...

> What is gclient? From: https://www.chromium.org/developers/how-tos/depottools 
> <https://www.chromium.org/developers/how-tos/depottools>
> 
> gclient is a Python script to manage a workspace of modular dependencies that
> are each checked out independently from different subversion or git
> repositories. Features include:
> 
>    - Dependencies can be specified on a per-OS basis.
>    - Dependencies can be specified relative to their parent dependency.
>    - Variables can be used to abstract concepts.
>    - Hooks can be specified to be run after a checkout.
>    - .gclient and DEPS are Python scripts. You can hack in easily or add
>       additional configuration data.

  I haven't looked at the details, but it sounds like even with separate 
download infrastructure it may be tricky. It sounds like it might be difficult 
to decouple the download from the configuration and/or build...

  (And also bad: this stuff is running arbitrary code before checking the hash. 
But we have the same problem with go and cargo already as well.)

> .gclient file: It's the primary file. It is, in fact, a Python script. It
> specifies the following variables:
> 
>    - solutions: an array of dictionaries specifying the projects that will be
>                 fetched.
>    - hooks: additional hooks to be run when this meta checkout is synced.
>    - target_os: an optional array of (target) operating systems to fetch
>                 OS-specific dependencies for.
>    - cache_dir: Primarily for bots, multiple working sets use a single git
>                 cache.
> 
> 
> Why is this a significant issue?
> 
> The release tarballs from https://github.com/flutter/engine 
> <https://github.com/flutter/engine> are in no state to
> compile! No, they are only for the use of gclient to download and build a
> source directory structure suitable to build the Flutter engine! If you
> download, extract and attempt to run
> ./tools/gn --no-goma --no-prebuilt-dart-sdk,
> you receive the error message:
> No such file or directory: 'flutter/flutter/third_party/gn/gn.'
> 
> But wait! Wasn't the gn binary just called? No, that's a wrapper in the Flutter
> source tree that formats arguments to call the real gn binary. The real gn is
> not provided in the tarball but is downloaded via gclient
> (among many other repositories.)
> 
> The above leads to the question: What do we do about this? We can't use the
> existing download infrastructure to download the tarball from Git Hub, as that
> isn't meant for us. The gclient download works if called out directly via a
> pre_download hook; however, this leads to other issues in recreating a
> consistent tarball because gclient writes the date to various files all around
> the directory, thus changing the shasum every time I attempt to generate a
> tarball. I can sed the dates in a flutter-engine-gen-tarball script I've made,

  You could also use fakedate to pre-set the date to a fixed value.

  Or could you patch gclient to not do that rewriting?

> which solves the problem; however, there is one last hurdle to overcome: When I
> compress the directory created by gclient, any .git folder in a subdirectory
> also changes when running mk_tar_gz from the script.

  Huh? That shouldn't happen... mk_tar_gz doesn't do anything that modifies the 
filesystem, other than changing access time filestamps. And it certainly doesn't 
do anything specific for .git directories.

> The .git directories, of
> course, can be removed, and that DOES create a reproducible tarball, but the
> build system relies on the .git directories to fill in a bunch of information
> before configuring, causing failures.
> 
> I won't ask the best course of action because anything we decide will be pretty
> ugly, thanks to Google and its decisions. Instead, I will ask, what is the
> least-bad solution that can be upstreamed to the master branch?

  If we're going for ugly solutions: one option would be to download everything 
that gclient downloads in a pre-download hook, create an extra tarball out of 
it, and in a pre-extract hook extract that tarball. That way, what we present to 
gn in the configure and build steps looks like an already-expanded gclient tree 
and it won't download again. It's of course horrible to find out what all needs 
to be downloaded and where to put this stuff, but you can e.g. write a script 
that monkey-patches gclient to get URLs instead of actually downloading. But in 
the end, I guess there's not much improvement compared to just doing the 
download by calling gclient/gn.

  Regards,
  Arnout


> 
> If you want to play around with any of this, you can clone
> git@gitlab.com:aduskett/buildroot-flutter.git and build the flutter_defconfig.
> 
> Thanks!
> Adam
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Flutter-engine package help wanted.

[Adam Duskett]

[-- Attachment #1.1: Type: text/plain, Size: 6658 bytes --]

Hey Arnout!

On Sun, Jul 16, 2023 at 8:32 AM Arnout Vandecappelle <arnout@mind.be> wrote:

>
>
> On 15/07/2023 05:02, Adam Duskett wrote:
> > Hello! I have a task to port Flutter to Buildroot and to make the
> packages
> > suitable for the master branch.
> >
> > I have talked with Yann in IRC about the problems facing the first
> package,
> > flutter-engine [https://github.com/flutter/engine
> > <https://github.com/flutter/engine>]. However, after another full
> > day of hacking away at the package, it's time to talk about a significant
> > problem I must overcome.
> >
> > Here is a summary of the issue I am facing and the main reason for the
> email:
> >
> > Flutter is developed and maintained by Google.
> > Of course, it stands to reason that Flutter does not use a conventional
> > configure/build system. Oh no, it couldn't be that easy. Instead,
> Flutter uses
> > gclient and gn from the chromium depot_tools repository, which contains
> all
> > sorts of exciting utilities. One of which is gclient.
>
>   My first reaction was: oh, we'll need specific download infrastructure
> like
> for go and cargo...
>
> > What is gclient? From:
> https://www.chromium.org/developers/how-tos/depottools
> > <https://www.chromium.org/developers/how-tos/depottools>
> >
> > gclient is a Python script to manage a workspace of modular dependencies
> that
> > are each checked out independently from different subversion or git
> > repositories. Features include:
> >
> >    - Dependencies can be specified on a per-OS basis.
> >    - Dependencies can be specified relative to their parent dependency.
> >    - Variables can be used to abstract concepts.
> >    - Hooks can be specified to be run after a checkout.
> >    - .gclient and DEPS are Python scripts. You can hack in easily or add
> >       additional configuration data.
>
>   I haven't looked at the details, but it sounds like even with separate
> download infrastructure it may be tricky. It sounds like it might be
> difficult
> to decouple the download from the configuration and/or build..
>


>   (And also bad: this stuff is running arbitrary code before checking the
> hash.
> But we have the same problem with go and cargo already as well.)
>
> Extremely difficult unfortunately. :(


> > .gclient file: It's the primary file. It is, in fact, a Python script. It
> > specifies the following variables:
> >
> >    - solutions: an array of dictionaries specifying the projects that
> will be
> >                 fetched.
> >    - hooks: additional hooks to be run when this meta checkout is synced.
> >    - target_os: an optional array of (target) operating systems to fetch
> >                 OS-specific dependencies for.
> >    - cache_dir: Primarily for bots, multiple working sets use a single
> git
> >                 cache.
> >
> >
> > Why is this a significant issue?
> >
> > The release tarballs from https://github.com/flutter/engine
> > <https://github.com/flutter/engine> are in no state to
> > compile! No, they are only for the use of gclient to download and build a
> > source directory structure suitable to build the Flutter engine! If you
> > download, extract and attempt to run
> > ./tools/gn --no-goma --no-prebuilt-dart-sdk,
> > you receive the error message:
> > No such file or directory: 'flutter/flutter/third_party/gn/gn.'
> >
> > But wait! Wasn't the gn binary just called? No, that's a wrapper in the
> Flutter
> > source tree that formats arguments to call the real gn binary. The real
> gn is
> > not provided in the tarball but is downloaded via gclient
> > (among many other repositories.)
> >
> > The above leads to the question: What do we do about this? We can't use
> the
> > existing download infrastructure to download the tarball from Git Hub,
> as that
> > isn't meant for us. The gclient download works if called out directly
> via a
> > pre_download hook; however, this leads to other issues in recreating a
> > consistent tarball because gclient writes the date to various files all
> around
> > the directory, thus changing the shasum every time I attempt to generate
> a
> > tarball. I can sed the dates in a flutter-engine-gen-tarball script I've
> made,
>
>   You could also use fakedate to pre-set the date to a fixed value.
>
>   Or could you patch gclient to not do that rewriting?
>
> I didn't think about fakedate, I'll play around with it!


> > which solves the problem; however, there is one last hurdle to overcome:
> When I
> > compress the directory created by gclient, any .git folder in a
> subdirectory
> > also changes when running mk_tar_gz from the script.
>
>   Huh? That shouldn't happen... mk_tar_gz doesn't do anything that
> modifies the
> filesystem, other than changing access time filestamps. And it certainly
> doesn't
> do anything specific for .git directories.
>
> Yeah, I misremembered. It's actually gclient (again) creating new
checkouts on a clone with the current date.


> > The .git directories, of
> > course, can be removed, and that DOES create a reproducible tarball, but
> the
> > build system relies on the .git directories to fill in a bunch of
> information
> > before configuring, causing failures.
> >
> > I won't ask the best course of action because anything we decide will be
> pretty
> > ugly, thanks to Google and its decisions. Instead, I will ask, what is
> the
> > least-bad solution that can be upstreamed to the master branch?
>
>   If we're going for ugly solutions: one option would be to download
> everything
> that gclient downloads in a pre-download hook, create an extra tarball out
> of
> it, and in a pre-extract hook extract that tarball. That way, what we
> present to
> gn in the configure and build steps looks like an already-expanded gclient
> tree
> and it won't download again. It's of course horrible to find out what all
> needs
> to be downloaded and where to put this stuff, but you can e.g. write a
> script
> that monkey-patches gclient to get URLs instead of actually downloading.
> But in
> the end, I guess there's not much improvement compared to just doing the
> download by calling gclient/gn.
>

A pre-download hook is exactly what I used at first, but creating a
reproducible
tarball was.. painful. I wasn't aware of fakedate, however, so I'll give
that a try!

  Regards,
>   Arnout
>
>
> >
> > If you want to play around with any of this, you can clone
> > git@gitlab.com:aduskett/buildroot-flutter.git and build the
> flutter_defconfig.
> >
> > Thanks!
> > Adam
>

[-- Attachment #1.2: Type: text/html, Size: 8821 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Flutter-engine package help wanted.

[Yann E. MORIN]

Adam, All,

On 2023-07-14 20:02 -0700, Adam Duskett spake thusly:
> Hello! I have a task to port Flutter to Buildroot and to make the packages
> suitable for the master branch.
> 
> I have talked with Yann in IRC about the problems facing the first package,
> flutter-engine [ [1]https://github.com/flutter/engine]. However, after another full
> day of hacking away at the package, it's time to talk about a significant
> problem I must overcome.
> 
> Here is a summary of the issue I am facing and the main reason for the email:
> 
> Flutter is developed and maintained by Google.
> Of course, it stands to reason that Flutter does not use a conventional
> configure/build system. Oh no, it couldn't be that easy. Instead, Flutter uses
> gclient and gn from the chromium depot_tools repository, which contains all
> sorts of exciting utilities. One of which is gclient.
> 
> What is gclient? From: [2]https://www.chromium.org/developers/how-tos/depottools
> 
> gclient is a Python script to manage a workspace of modular dependencies that
> are each checked out independently from different subversion or git
> repositories. Features include:
> 
>   - Dependencies can be specified on a per-OS basis.
>   - Dependencies can be specified relative to their parent dependency.
>   - Variables can be used to abstract concepts.
>   - Hooks can be specified to be run after a checkout.
>   - .gclient and DEPS are Python scripts. You can hack in easily or add
>      additional configuration data.
> 
> .gclient file: It's the primary file. It is, in fact, a Python script. It
> specifies the following variables:
> 
>   - solutions: an array of dictionaries specifying the projects that will be
>                fetched.
>   - hooks: additional hooks to be run when this meta checkout is synced.
>   - target_os: an optional array of (target) operating systems to fetch
>                OS-specific dependencies for.
>   - cache_dir: Primarily for bots, multiple working sets use a single git
>                cache.
> 
> Why is this a significant issue?
> 
> The release tarballs from [3]https://github.com/flutter/engine are in no state to
> compile! No, they are only for the use of gclient to download and build a
> source directory structure suitable to build the Flutter engine! If you
> download, extract and attempt to run
> ./tools/gn --no-goma --no-prebuilt-dart-sdk,
> you receive the error message:
> No such file or directory: 'flutter/flutter/third_party/gn/gn.'
> 
> But wait! Wasn't the gn binary just called? No, that's a wrapper in the Flutter
> source tree that formats arguments to call the real gn binary. The real gn is
> not provided in the tarball but is downloaded via gclient
> (among many other repositories.)
> 
> The above leads to the question: What do we do about this? We can't use the
> existing download infrastructure to download the tarball from Git Hub, as that
> isn't meant for us. The gclient download works if called out directly via a
> pre_download hook; however, this leads to other issues in recreating a
> consistent tarball because gclient writes the date to various files all around
> the directory, thus changing the shasum every time I attempt to generate a
> tarball. I can sed the dates in a flutter-engine-gen-tarball script I've made,
> which solves the problem; however, there is one last hurdle to overcome: When I
> compress the directory created by gclient, any .git folder in a subdirectory
> also changes when running mk_tar_gz from the script. The .git directories, of
> course, can be removed, and that DOES create a reproducible tarball, but the
> build system relies on the .git directories to fill in a bunch of information
> before configuring, causing failures.
> 
> I won't ask the best course of action because anything we decide will be pretty
> ugly, thanks to Google and its decisions. Instead, I will ask, what is the
> least-bad solution that can be upstreamed to the master branch?

So, my overall idea was:

    FLUTTER_VERSION = some-hash-or-tag
    FLUTTER_SITE = https://github.com/flutter/engine
    FLUTTER_SITE_METHOD = git   # or download a tarball

    define FLUTTER_POPULATE_WITH_GCLIENT
        rm -rf $(@D)/foo-temp-trash  # in case previous make failed somehow and we needed to restart
        mkdir $(@D)/foo-temp-trash
        tar xf $(FLUTTER_DL_DIR)/$(FLUTTER_SOURCE) -C $(@D)/foo-temp-trash
        cd $(@D)/foo-temp-trash && gclient --some-freaking-options
        . ./support/download/helpers && mk_tar_gz \
            $(@D)/foo-temp-trash \
            flutter-$(FLUTTER_VERSION) \
            $(some-date-which-one-pray-tell) \
            $(FLUTTER_DL_DIR)/flutter-$(FLUTTER_VERSION).br.tar.gz
        rm -rf $(@D)/foo-temp-trash
    endef
    FLUTTER_POST_DOWNLOAD_HOOKS += FLUTTER_POPULATE_WITH_GCLIENT

    FLUTTER_EXTRACT_GCLIENT_STUFF
        # Better use the existing macros that those proper decompressor
        # and tar macro that already strips component; read the code, Luke
        tar xf $(FLUTTER_DL_DIR)/flutter-$(FLUTTER_VERSION).br.tar.gz --strip-component=1 -C $(@D)
    endef
    FLUTTER_POST_EXTRACT_HOOKS += FLUTTER_EXTRACT_GCLIENT_STUFF

And that's basically it. No .hash file because gclient does soe
git-clone, those full repositories are needed at build time, and thus
there is no telling whatstate those local clones will be in (besides a
lot of dates fluctuating...)

Regards,
Yann E. MORIN.

> If you want to play around with any of this, you can clone
> git@gitlab.com:aduskett/buildroot-flutter.git and build the flutter_defconfig.
> 
> Thanks!
> Adam
> 
> Links:
> 1. https://github.com/flutter/engine
> 2. https://www.chromium.org/developers/how-tos/depottools
> 3. https://github.com/flutter/engine

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Flutter-engine package help wanted.

[Yann E. MORIN]

Adam, All,

Folloing our discussion on IRC, it turns out that I got it partly
(mostly) wrong.

On 2023-07-17 20:38 +0200, Yann E. MORIN spake thusly:
> On 2023-07-14 20:02 -0700, Adam Duskett spake thusly:

So, we do not need to download anything from the flutter repo; we just
need a file, .gclient, that contains directives for gclient to perform
the actual download.

As an aside, gclient is like repo (another google tool, I guess), right?

So, basically, what you did in your .mk looks like I would have done,
except no using _DL_DIR as a scratch-dir to do the download, but in a
sub-dir of the package dir (see my previous reply).

Finally: gclient runs git-clone, we can't have stability in that, it's
not reproducible.

If the .git dirs really need to be present at configure/build time,
we're borked, we can't hash the generated tarball.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Flutter-engine package help wanted.

[Adam Duskett]

[-- Attachment #1.1: Type: text/plain, Size: 1821 bytes --]

Thanks for the reply! I am going through your notes on the repository now!

As for needing the .git dirs, indeed. I found someone who wants to make the
yocto tarball reproducible.
In the end he gave up:
https://github.com/meta-flutter/meta-flutter/issues/271


On Mon, Jul 17, 2023 at 12:16 PM Yann E. MORIN <yann.morin.1998@free.fr>
wrote:

> Adam, All,
>
> Folloing our discussion on IRC, it turns out that I got it partly
> (mostly) wrong.
>
> On 2023-07-17 20:38 +0200, Yann E. MORIN spake thusly:
> > On 2023-07-14 20:02 -0700, Adam Duskett spake thusly:
>
> So, we do not need to download anything from the flutter repo; we just
> need a file, .gclient, that contains directives for gclient to perform
> the actual download.
>
> As an aside, gclient is like repo (another google tool, I guess), right?
>
> So, basically, what you did in your .mk looks like I would have done,
> except no using _DL_DIR as a scratch-dir to do the download, but in a
> sub-dir of the package dir (see my previous reply).
>
> Finally: gclient runs git-clone, we can't have stability in that, it's
> not reproducible.
>
> If the .git dirs really need to be present at configure/build time,
> we're borked, we can't hash the generated tarball.
>
> Regards,
> Yann E. MORIN.
>
> --
>
> .-----------------.--------------------.------------------.--------------------.
> |  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics'
> conspiracy: |
> | +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___
>      |
> | +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is
> no  |
> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v
>  conspiracy.  |
>
> '------------------------------^-------^------------------^--------------------'
>

[-- Attachment #1.2: Type: text/html, Size: 2432 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Flutter-engine package help wanted.

[Adam Duskett]

[-- Attachment #1.1: Type: text/plain, Size: 2728 bytes --]

All;

I updated the above buildroot repository on the flutter-dev branch, and now
compiling flutter works with one manual step that I need to resolve.

While compiling, it seems like the LDPATHS for one third-party module can't
find libgcc.a, crtbeginS.o, and crtendS.o. These libraries, of course, are
found in the toolchains directory. If I manually create symlinks to those
files in the staging/lib directory, flutter-engine compiles successfully
without warning.

Feel free to review the flutter-engine package and give compiling a try:
```
git clone https://gitlab.com/aduskett/buildroot-flutter.git -b flutter-dev
cd buildroot-flutter
make flutter_defconfig
make flutter-engine
```

Any thoughts on the LDPATH issue?

Thanks!

Adam


On Mon, Jul 17, 2023 at 12:33 PM Adam Duskett <aduskett@gmail.com> wrote:

> Thanks for the reply! I am going through your notes on the repository now!
>
> As for needing the .git dirs, indeed. I found someone who wants to make
> the yocto tarball reproducible.
> In the end he gave up:
> https://github.com/meta-flutter/meta-flutter/issues/271
>
>
> On Mon, Jul 17, 2023 at 12:16 PM Yann E. MORIN <yann.morin.1998@free.fr>
> wrote:
>
>> Adam, All,
>>
>> Folloing our discussion on IRC, it turns out that I got it partly
>> (mostly) wrong.
>>
>> On 2023-07-17 20:38 +0200, Yann E. MORIN spake thusly:
>> > On 2023-07-14 20:02 -0700, Adam Duskett spake thusly:
>>
>> So, we do not need to download anything from the flutter repo; we just
>> need a file, .gclient, that contains directives for gclient to perform
>> the actual download.
>>
>> As an aside, gclient is like repo (another google tool, I guess), right?
>>
>> So, basically, what you did in your .mk looks like I would have done,
>> except no using _DL_DIR as a scratch-dir to do the download, but in a
>> sub-dir of the package dir (see my previous reply).
>>
>> Finally: gclient runs git-clone, we can't have stability in that, it's
>> not reproducible.
>>
>> If the .git dirs really need to be present at configure/build time,
>> we're borked, we can't hash the generated tarball.
>>
>> Regards,
>> Yann E. MORIN.
>>
>> --
>>
>> .-----------------.--------------------.------------------.--------------------.
>> |  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics'
>> conspiracy: |
>> | +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___
>>      |
>> | +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There
>> is no  |
>> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v
>>  conspiracy.  |
>>
>> '------------------------------^-------^------------------^--------------------'
>>
>

[-- Attachment #1.2: Type: text/html, Size: 3819 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] Flutter-engine package help wanted.

[Adam Duskett]

[-- Attachment #1.1: Type: text/plain, Size: 3346 bytes --]

All;

The https://gitlab.com/aduskett/buildroot-flutter.git  project is now
updated a nearly completed
flutter-engine package. I do have one concern found in the .mk file
(
https://gitlab.com/aduskett/buildroot-flutter/-/tree/flutter-dev/package/flutter-engine
)

Any help on
https://gitlab.com/aduskett/buildroot-flutter/-/blob/flutter-dev/package/flutter-engine/flutter-engine.mk#L216
would be much appreciated!

Adam


On Tue, Jul 18, 2023 at 7:04 PM Adam Duskett <aduskett@gmail.com> wrote:

> All;
>
> I updated the above buildroot repository on the flutter-dev branch, and now
> compiling flutter works with one manual step that I need to resolve.
>
> While compiling, it seems like the LDPATHS for one third-party module can't
> find libgcc.a, crtbeginS.o, and crtendS.o. These libraries, of course, are
> found in the toolchains directory. If I manually create symlinks to those
> files in the staging/lib directory, flutter-engine compiles successfully
> without warning.
>
> Feel free to review the flutter-engine package and give compiling a try:
> ```
> git clone https://gitlab.com/aduskett/buildroot-flutter.git -b flutter-dev
> cd buildroot-flutter
> make flutter_defconfig
> make flutter-engine
> ```
>
> Any thoughts on the LDPATH issue?
>
> Thanks!
>
> Adam
>
>
> On Mon, Jul 17, 2023 at 12:33 PM Adam Duskett <aduskett@gmail.com> wrote:
>
>> Thanks for the reply! I am going through your notes on the repository now!
>>
>> As for needing the .git dirs, indeed. I found someone who wants to make
>> the yocto tarball reproducible.
>> In the end he gave up:
>> https://github.com/meta-flutter/meta-flutter/issues/271
>>
>>
>> On Mon, Jul 17, 2023 at 12:16 PM Yann E. MORIN <yann.morin.1998@free.fr>
>> wrote:
>>
>>> Adam, All,
>>>
>>> Folloing our discussion on IRC, it turns out that I got it partly
>>> (mostly) wrong.
>>>
>>> On 2023-07-17 20:38 +0200, Yann E. MORIN spake thusly:
>>> > On 2023-07-14 20:02 -0700, Adam Duskett spake thusly:
>>>
>>> So, we do not need to download anything from the flutter repo; we just
>>> need a file, .gclient, that contains directives for gclient to perform
>>> the actual download.
>>>
>>> As an aside, gclient is like repo (another google tool, I guess), right?
>>>
>>> So, basically, what you did in your .mk looks like I would have done,
>>> except no using _DL_DIR as a scratch-dir to do the download, but in a
>>> sub-dir of the package dir (see my previous reply).
>>>
>>> Finally: gclient runs git-clone, we can't have stability in that, it's
>>> not reproducible.
>>>
>>> If the .git dirs really need to be present at configure/build time,
>>> we're borked, we can't hash the generated tarball.
>>>
>>> Regards,
>>> Yann E. MORIN.
>>>
>>> --
>>>
>>> .-----------------.--------------------.------------------.--------------------.
>>> |  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics'
>>> conspiracy: |
>>> | +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___
>>>        |
>>> | +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There
>>> is no  |
>>> | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v
>>>  conspiracy.  |
>>>
>>> '------------------------------^-------^------------------^--------------------'
>>>
>>

[-- Attachment #1.2: Type: text/html, Size: 5053 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot