[PATCH] gitlab-ci: add smoke test for fuzzers

[PATCH] gitlab-ci: add smoke test for fuzzers

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 838 bytes --]

Our GitLab CI setup has a test gap where the fuzzers aren't exercised at
all. Add a smoke test, similar to the one we have in GitHub Workflows.

Signed-off-by: Patrick Steinhardt <ps@pks.im>
---

As identified by Junio in <xmqqwmoi31aw.fsf@gitster.g>.

Patrick

 .gitlab-ci.yml | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index c0fa2fe90b..33d53b3ec7 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -93,6 +93,15 @@ test:osx:
       - t/failed-test-artifacts
     when: on_failure
 
+test:fuzz-smoke-tests:
+  image: ubuntu:latest
+  variables:
+    CC: clang
+  before_script:
+    - ./ci/install-docker-dependencies.sh
+  script:
+    - ./ci/run-build-and-minimal-fuzzers.sh
+
 static-analysis:
   image: ubuntu:22.04
   variables:
-- 
2.45.0-rc1


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] gitlab-ci: add smoke test for fuzzers

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 482 bytes --]

On Mon, Apr 29, 2024 at 08:13:23AM +0200, Patrick Steinhardt wrote:
> Our GitLab CI setup has a test gap where the fuzzers aren't exercised at
> all. Add a smoke test, similar to the one we have in GitHub Workflows.
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> ---
> 
> As identified by Junio in <xmqqwmoi31aw.fsf@gitster.g>.
> 
> Patrick

I forgot to add the link to a successful run of this job:
https://gitlab.com/gitlab-org/git/-/jobs/6735705569

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] gitlab-ci: add smoke test for fuzzers

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> On Mon, Apr 29, 2024 at 08:13:23AM +0200, Patrick Steinhardt wrote:
>> Our GitLab CI setup has a test gap where the fuzzers aren't exercised at
>> all. Add a smoke test, similar to the one we have in GitHub Workflows.
>> 
>> Signed-off-by: Patrick Steinhardt <ps@pks.im>
>> ---
>> 
>> As identified by Junio in <xmqqwmoi31aw.fsf@gitster.g>.
>> 
>> Patrick
>
> I forgot to add the link to a successful run of this job:
> https://gitlab.com/gitlab-org/git/-/jobs/6735705569

Thanks.  I wonder if we can somehow automate a change like this.

Seeing how simple this fix has become thanks to the use of
before_script/script pair that merely point at ci/*.sh scripts,
perhaps we have already extracted enough commonalities as a set of
shell scripts in ci/ hierarchy.  I wonder if we can have a common
"source" that is "compiled" into .gitlab-ci.yml and its counterpart
for GitHub Actions?

Or perhaps a linter that can say things like "ah, you are adding
this new test to one, but not touching the other, shouldn't you?",
and "you are tweaking this existing test in one, but shouldn't you
be doing the same to the other?"

Re: [PATCH] gitlab-ci: add smoke test for fuzzers

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 1838 bytes --]

On Mon, Apr 29, 2024 at 08:37:49AM -0700, Junio C Hamano wrote:
> Patrick Steinhardt <ps@pks.im> writes:
> 
> > On Mon, Apr 29, 2024 at 08:13:23AM +0200, Patrick Steinhardt wrote:
> >> Our GitLab CI setup has a test gap where the fuzzers aren't exercised at
> >> all. Add a smoke test, similar to the one we have in GitHub Workflows.
> >> 
> >> Signed-off-by: Patrick Steinhardt <ps@pks.im>
> >> ---
> >> 
> >> As identified by Junio in <xmqqwmoi31aw.fsf@gitster.g>.
> >> 
> >> Patrick
> >
> > I forgot to add the link to a successful run of this job:
> > https://gitlab.com/gitlab-org/git/-/jobs/6735705569
> 
> Thanks.  I wonder if we can somehow automate a change like this.
> 
> Seeing how simple this fix has become thanks to the use of
> before_script/script pair that merely point at ci/*.sh scripts,
> perhaps we have already extracted enough commonalities as a set of
> shell scripts in ci/ hierarchy.  I wonder if we can have a common
> "source" that is "compiled" into .gitlab-ci.yml and its counterpart
> for GitHub Actions?
> 
> Or perhaps a linter that can say things like "ah, you are adding
> this new test to one, but not touching the other, shouldn't you?",
> and "you are tweaking this existing test in one, but shouldn't you
> be doing the same to the other?"

We probably could, yeah. The question is whether it would really be
worth it in the end. GitLab CI is still a relatively new addition, and
thus it needs to catch up with what GitHub Workflows has. But once that
is done I don't expect there to be a ton of changes to the CI setup, and
the few new additions that we gain once in a while should be relatively
easy to spot during review.

So if anybody is up for it then I'm happy to review that. But I don't
think there would be enough value to do it myself.

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] gitlab-ci: add smoke test for fuzzers

[Patrick Steinhardt]

[-- Attachment #1: Type: text/plain, Size: 449 bytes --]

On Mon, Apr 29, 2024 at 08:13:23AM +0200, Patrick Steinhardt wrote:
> Our GitLab CI setup has a test gap where the fuzzers aren't exercised at
> all. Add a smoke test, similar to the one we have in GitHub Workflows.
> 
> Signed-off-by: Patrick Steinhardt <ps@pks.im>

Junio, I haven't seen this topic getting picked up in your "What's
cooking" report yet. Is this intentional or did it simply fall through
the cracks?

Thanks!

Patrick

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] gitlab-ci: add smoke test for fuzzers

[Junio C Hamano]

Patrick Steinhardt <ps@pks.im> writes:

> On Mon, Apr 29, 2024 at 08:13:23AM +0200, Patrick Steinhardt wrote:
>> Our GitLab CI setup has a test gap where the fuzzers aren't exercised at
>> all. Add a smoke test, similar to the one we have in GitHub Workflows.
>> 
>> Signed-off-by: Patrick Steinhardt <ps@pks.im>
>
> Junio, I haven't seen this topic getting picked up in your "What's
> cooking" report yet. Is this intentional or did it simply fall through
> the cracks?

The latter.  Thanks for a ping that is designed to make it easy to
find the patch (by going to the message it is a response to).