From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Message-ID: <3be0df41af485685253bd0ed79396fd136c210e6.camel@linuxfoundation.org> Subject: Re: [PATCH v1 1/2] gpg-sign: Add parameters to gpg signature function From: "Richard Purdie" Date: Thu, 24 Mar 2022 22:20:37 +0000 In-Reply-To: <0791e55a-cba3-3d7f-8a17-96b1591454a1@gmail.com> References: <20220322211949.7423-1-fntoth@gmail.com> <6dad87edfe80040678963ca18842a3ddf337e35c.camel@linuxfoundation.org> <9c616e34-d31b-dcab-657a-268698d34150@gmail.com> <0791e55a-cba3-3d7f-8a17-96b1591454a1@gmail.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=-EVJiuUdYIZHFm4Hs02LU" List-id: To: Ferry Toth , openembedded-core@lists.openembedded.org Cc: Xavier Berger --=-EVJiuUdYIZHFm4Hs02LU Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit On Thu, 2022-03-24 at 23:11 +0100, Ferry Toth wrote: > Op 24-03-2022 om 16:36 schreef Ferry Toth: > > Op 24-03-2022 om 13:03 schreef Richard Purdie: > > > On Thu, 2022-03-24 at 12:23 +0100, Ferry Toth wrote: > > > > > On Wed, 2022-03-23 at 19:34 +0100, Ferry Toth wrote: > > > > > > I forgot to add a cover letter, sorry for that. The 2 patches  together > > > > > > implement DEB repository signing. > > > > > > > > > > > > This is necessary since Gatesgarth |apt| (1.8.2) has become more strict > > > > > > and doesn’t allow unsigned repositories by default. > > > > > > > > > > > > It is possible to override this behavior |but||| is more work then to > > > > > > enable signed DEB repositories. These patches makes DEB a first class > > > > > > citizen as IPK and RPM. > > > > > > > > > > > > Patches have been in use in meta-intel-edison since Gatesgarth, see > > > > > > https://edison-fw.github.io/meta-intel-edison/5.0-Creating-a-deb- > > > > > > repository.html\ > > > > > What puzzles me is that we can build root filesystems using apt, we test > > > > > this on > > > > > the autobuilder. Saying repositories are broken since gatesgarth therefore > > > > > seems > > > > > confusing in the commit message. > > The Development Task manual in 3.22.4.3.3 names it a repository, as I > believe is common for a distributions server holding pre-built packages. > But I could change the name to package feed if that would be better. > > What I meant to say is that apt requires signed package feeds since > 1.8.2 and disabling that is a workaround. I mean to distinguish between the use of repositories at rootfs creation time compared to repositories used later to update an image. Since I know repositories are used to build the rootfs, the commit message did confuse me at first since rootfs construction works. > > > > > Good question. When I (meta-intel-edison) build the rootfs using DEB's it just > > > > works. > > > > Could it be that during rootfs build dpkg is used and not apt? I think I have > > > > seen that in the logs. > > > It definitely uses apt. > Checking logs I see you are correct. > > > > Of course apt uses dpkg to install a package as well, but it refuses to > > > > download the package from a repo when it's not signed. > > > Perhaps the difference is the packages are local and not remote? > > No, for generating the rootfs it appears the sources list file has: > > deb [trusted=yes] file:/path.../oe-rootfs-repo/edison/ ./ > > So apparently has been taken care by disabling the signing requirement. > [trusted=yes] is part of that workaround. That does give another way to work around this then... > > > > > > I guess we must configure apt to override that during the rootfs process and > > > > > likely an end user with a remote feed could do the same, possibly with a > > > > > warning > > > > > from apt? > Yes. But it is a pain to google to find how to do that. > > > >  I believe there is no issue during rootfs generation. > > > > > > > > > I'm also worried that there isn't any automated testing of this change. The > > > > > reason I worry is that since we don't show any testing failures right now, > > > > > there > > > > > is clearly a hole in our automated testing coverage and there is no > > > > > guarantee > > > > > that this feature will keep working. It is these smaller corner case issues > > > > > which tend to make or break the project's experience as if a feature is > > > > > present, > > > > > people expect it to work. Can we improve the testing situation? > > > > It doesn't seem to be a particularly volatile area in the code. I refreshed > > > > Xavier's patches for Gatesgarth, and am actively using unchanged patch on > > > > Honisiter. > > > > I don't know how the automated testing is working but I guess for RPM a repo > > > > is generated using a small layer? And then tested on a qemu running the > > > > rootfs? > > > > Should be almost same for deb/apt, maybe could be modified from rpm test? > > > I think the rpm test is test_testimage_dnf in > > > meta/lib/oeqa/selftest/cases/runtime_test.py. You'd run it with: > > > > > > oe-selftest -r runtime_test.TestImage.test_testimage_dnf > > I'll have a look. > > Pfff. These tests are O(1) more then the signing code they test. I'm > trying this original test (on rpm) now to see if I can get that to run > and understand how it works. What you're asking to work is a complex scenario so the test isn't going to be simple. FWIW there is also oeqa/runtime/cases/apt.py. That test checks if a package can be installed from a remote package feed (no signing). It is doing: echo deb [ allow-insecure=yes ] %s ./ > sources.list to work around the signing issue. > Am I right that meta/lib/oeqa/selftest/cases/signing.py tests the actual > package feed signing? No. That tests that signed packages in rpm works rather than a signed remote repository which is different. Cheers, Richard --=-EVJiuUdYIZHFm4Hs02LU Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable
On Thu, 2022-03-24 at 23:11 +0100, Ferry Toth= wrote:
Op 24-03-2022 om 16:36 schreef=
 Ferry Toth:
Op 24-03-2022 om 13:03 sc=
hreef Richard Purdie:
On Thu, 2022-03-=
24 at 12:23 +0100, Ferry Toth wrote:
On Wed, 2022-03-23 at 19:34 +0100, Ferry Toth w=
rote:
I forgot to add a cover letter, =
sorry for that. The 2 patches  together
implement DEB reposi=
tory signing.

This is necessary since Gatesgarth |=
apt| (1.8.2) has become more strict
and doesn=E2=80=99t allow uns=
igned repositories by default.

It is possible to o=
verride this behavior |but||| is more work then to
enable signed =
DEB repositories. These patches makes DEB a first class
citizen a=
s IPK and RPM.

Patches have been in use in meta-in=
tel-edison since Gatesgarth, see
https://edison-fw.github.io/me=
ta-intel-edison/5.0-Creating-a-deb-
repository.html\
What puzzles me is that we can build root filesystems using =
apt, we test
this on
the autobuilder. Saying repositori=
es are broken since gatesgarth therefore
seems
confusin=
g in the commit message.

The Development Task manual in 3.22.4.3.3 names =
it a repository, as I 
believe is common for a distributions serv=
er holding pre-built packages. 
But I could change the name to pa=
ckage feed if that would be better.

What I meant t=
o say is that apt requires signed package feeds since 
1.8.2 and =
disabling that is a workaround.

I mea= n to distinguish between the use of repositories at rootfs creation time co= mpared to repositories used later to update an image. Since I know reposito= ries are used to build the rootfs, the commit message did confuse me at fir= st since rootfs construction works.




Good questi=
on. When I (meta-intel-edison) build the rootfs using DEB's it just
works.
Could it be that during rootfs build dpkg is used and n=
ot apt? I think I have
seen that in the logs.
<= pre>It definitely uses apt.
Checking lo=
gs I see you are correct.
Of course apt uses dpkg to insta=
ll a package as well, but it refuses to
download the package from=
 a repo when it's not signed.
Perhaps the difference=
 is the packages are local and not remote?
<= pre>
No, for generating the rootfs it appears the sources lis=
t file has:

deb [trusted=3Dyes] file:/path.../oe-r=
ootfs-repo/edison/ ./

So apparently has been taken=
 care by disabling the signing requirement. 
[trusted=3Dyes] is p=
art of that workaround.

That does giv= e another way to work around this then...



  I believe there is no issue d=
uring rootfs generation.
I'm also worried that there isn't any automated testing of this change. Th= e
reason I worry is that since we don't show any testing failures=
 right now,
there
is clearly a hole in our automated te=
sting coverage and there is no
guarantee
that this feat=
ure will keep working. It is these smaller corner case issues
whi=
ch tend to make or break the project's experience as if a feature is
<= pre>present,
people expect it to work. Can we improve the testing=
 situation?
It doesn't seem to be a particularly vol=
atile area in the code. I refreshed
Xavier's patches for Gatesgar=
th, and am actively using unchanged patch on
Honisiter.
I don't know how the automated testing is working but I guess for RPM a re= po
is generated using a small layer? And then tested on a qemu ru=
nning the
rootfs?
Should be almost same for deb/apt, ma=
ybe could be modified from rpm test?
I think the rpm=
 test is test_testimage_dnf in
meta/lib/oeqa/selftest/cases/runti=
me_test.py. You'd run it with:

oe-selftest -r runt=
ime_test.TestImage.test_testimage_dnf
I'll have a lo=
ok.

Pfff. These tests are O(1) more t=
hen the signing code they test. I'm 
trying this original test (o=
n rpm) now to see if I can get that to run 
and understand how it=
 works.

What you're asking to work is= a complex scenario so the test isn't going to be simple.

FWIW there is also oeqa/runtime/cases/apt.py. That test checks if a= package can be installed from a remote package feed (no signing). It is do= ing:

echo deb [ allow-insecure=3Dyes ] %s ./ > = sources.list

to work around the signing issue.

Am I right that meta/lib/o=
eqa/selftest/cases/signing.py tests the actual 
package feed sign=
ing?

No. That tests that signed packa= ges in rpm works rather than a signed remote repository which is different.=

Cheers,

Richard
--=-EVJiuUdYIZHFm4Hs02LU--