On Thu, 2022-03-24 at 23:11 +0100, Ferry Toth wrote:
> Op 24-03-2022 om 16:36 schreef Ferry Toth:
> > Op 24-03-2022 om 13:03 schreef Richard Purdie:
> > > On Thu, 2022-03-24 at 12:23 +0100, Ferry Toth wrote:
> > > > > On Wed, 2022-03-23 at 19:34 +0100, Ferry Toth wrote:
> > > > > > I forgot to add a cover letter, sorry for that. The 2 patches  together
> > > > > > implement DEB repository signing.
> > > > > > 
> > > > > > This is necessary since Gatesgarth |apt| (1.8.2) has become more strict
> > > > > > and doesn’t allow unsigned repositories by default.
> > > > > > 
> > > > > > It is possible to override this behavior |but||| is more work then to
> > > > > > enable signed DEB repositories. These patches makes DEB a first class
> > > > > > citizen as IPK and RPM.
> > > > > > 
> > > > > > Patches have been in use in meta-intel-edison since Gatesgarth, see
> > > > > > https://edison-fw.github.io/meta-intel-edison/5.0-Creating-a-deb-
> > > > > > repository.html\
> > > > > What puzzles me is that we can build root filesystems using apt, we test
> > > > > this on
> > > > > the autobuilder. Saying repositories are broken since gatesgarth therefore
> > > > > seems
> > > > > confusing in the commit message.
> 
> The Development Task manual in 3.22.4.3.3 names it a repository, as I 
> believe is common for a distributions server holding pre-built packages. 
> But I could change the name to package feed if that would be better.
> 
> What I meant to say is that apt requires signed package feeds since 
> 1.8.2 and disabling that is a workaround.

I mean to distinguish between the use of repositories at rootfs creation time
compared to repositories used later to update an image. Since I know
repositories are used to build the rootfs, the commit message did confuse me at
first since rootfs construction works.



> 
> > > > Good question. When I (meta-intel-edison) build the rootfs using DEB's it just
> > > > works.
> > > > Could it be that during rootfs build dpkg is used and not apt? I think I have
> > > > seen that in the logs.
> > > It definitely uses apt.
> Checking logs I see you are correct.
> > > > Of course apt uses dpkg to install a package as well, but it refuses to
> > > > download the package from a repo when it's not signed.
> > > Perhaps the difference is the packages are local and not remote?
> 
> No, for generating the rootfs it appears the sources list file has:
> 
> deb [trusted=yes] file:/path.../oe-rootfs-repo/edison/ ./
> 
> So apparently has been taken care by disabling the signing requirement. 
> [trusted=yes] is part of that workaround.

That does give another way to work around this then...


> 
> > > > > I guess we must configure apt to override that during the rootfs process and
> > > > > likely an end user with a remote feed could do the same, possibly with a
> > > > > warning
> > > > > from apt?
> Yes. But it is a pain to google to find how to do that.
> > > >   I believe there is no issue during rootfs generation.
> > > >   
> > > > > I'm also worried that there isn't any automated testing of this change. The
> > > > > reason I worry is that since we don't show any testing failures right now,
> > > > > there
> > > > > is clearly a hole in our automated testing coverage and there is no
> > > > > guarantee
> > > > > that this feature will keep working. It is these smaller corner case issues
> > > > > which tend to make or break the project's experience as if a feature is
> > > > > present,
> > > > > people expect it to work. Can we improve the testing situation?
> > > > It doesn't seem to be a particularly volatile area in the code. I refreshed
> > > > Xavier's patches for Gatesgarth, and am actively using unchanged patch on
> > > > Honisiter.
> > > > I don't know how the automated testing is working but I guess for RPM a repo
> > > > is generated using a small layer? And then tested on a qemu running the
> > > > rootfs?
> > > > Should be almost same for deb/apt, maybe could be modified from rpm test?
> > > I think the rpm test is test_testimage_dnf in
> > > meta/lib/oeqa/selftest/cases/runtime_test.py. You'd run it with:
> > > 
> > > oe-selftest -r runtime_test.TestImage.test_testimage_dnf
> > I'll have a look.
> 
> Pfff. These tests are O(1) more then the signing code they test. I'm 
> trying this original test (on rpm) now to see if I can get that to run 
> and understand how it works.

What you're asking to work is a complex scenario so the test isn't going to be
simple.

FWIW there is also oeqa/runtime/cases/apt.py. That test checks if a package can
be installed from a remote package feed (no signing). It is doing:

echo deb [ allow-insecure=yes ] %s ./ > sources.list

to work around the signing issue.

> Am I right that meta/lib/oeqa/selftest/cases/signing.py tests the actual 
> package feed signing?

No. That tests that signed packages in rpm works rather than a signed remote
repository which is different.

Cheers,

Richard