From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BF2E3C47082 for ; Tue, 8 Jun 2021 13:37:55 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 14FD0610FB for ; Tue, 8 Jun 2021 13:37:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 14FD0610FB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:48090 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lqbvV-0002Iy-RL for qemu-devel@archiver.kernel.org; Tue, 08 Jun 2021 09:37:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45302) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lqbuc-0001Sq-F7 for qemu-devel@nongnu.org; Tue, 08 Jun 2021 09:36:58 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:24161) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lqbuZ-0006Ks-DM for qemu-devel@nongnu.org; Tue, 08 Jun 2021 09:36:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623159414; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=HdAhJfaJyTMgITmmhXtvcltY8dnDS6eECWpJf+fBY3U=; b=GGeIwPOjY1EdFMe7uwU5e6ZrrQbX2xfmO+scN9RUTMoRPMlcTYh7sjnsuSiJVIZ5/PE+ha QZVqZPO3L+QUITqUVwlVVEIpsbTDLGKF0GLR04Rgux70HCAy1HX8lg0AUbYcyQjRmeTVhK Jvr7+S29geyEIenZNFjeMb7riHB+2qA= Received: from mail-lj1-f200.google.com (mail-lj1-f200.google.com [209.85.208.200]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-392-PCEiYfuzODqUDu7i-MvpZg-1; Tue, 08 Jun 2021 09:36:50 -0400 X-MC-Unique: PCEiYfuzODqUDu7i-MvpZg-1 Received: by mail-lj1-f200.google.com with SMTP id f22-20020a2e38160000b0290130cf22822aso7949329lja.16 for ; Tue, 08 Jun 2021 06:36:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=HdAhJfaJyTMgITmmhXtvcltY8dnDS6eECWpJf+fBY3U=; b=mlTv1N+PLhGzHVqGyuGFkoQWJlmCDlM0WyuS3wuMDV04Or0L08cKLIe28Mj/+mDNUx Nl7N4gflYIwcH78s0RYidarvMU1EyOmJHQrFuYYGQXoiod5va84sfE+NdBLlgtJIJpjG ilkFZKHOrI4vkcOU2S2hwk2RLvQP4XbwfkXjpJRWnwCjITUcz0MVkeLc0x6v8lbp1xA1 PsMopziJ6jUtC2l14PRMqVQnk+HndRDLIw03ZWoLYO5YcXg/OrMdEucWp7YU8TOfcrV+ hUkE8YLeY+W2batDGUyWTgHME4flHxpuJpIUDDFdTg35+8gr6Yhpbms/yzwCFLo/4aY2 gYjw== X-Gm-Message-State: AOAM531wFTAvyR+R45rrNpJoETwjhq5T8b6Oe4zW4Bld3vJnvwEqdxHW Mp1OmZufYKKfSYZc9KAvp/0TjXx+u+0i/mTFv2YZ1JWzMv8Cnpv0pE09Ewv06scYcU0vRUHoQ2s CXR77f/HuBnS3brAGRbOlhuzxp99FB30= X-Received: by 2002:a05:6512:3a84:: with SMTP id q4mr8916964lfu.626.1623159408693; Tue, 08 Jun 2021 06:36:48 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyMr+GUMXf64X4XXN0vD02Fh+P/hamiCAO53goL2nIeABikHQ4iu/2o/cybIBwCdNianeAVmNujhan+M6Y06LU= X-Received: by 2002:a05:6512:3a84:: with SMTP id q4mr8916940lfu.626.1623159408414; Tue, 08 Jun 2021 06:36:48 -0700 (PDT) MIME-Version: 1.0 References: <20210608031425.833536-1-crosa@redhat.com> <20210608031425.833536-5-crosa@redhat.com> <9770910a-f586-0b79-395c-7154c4693690@amsat.org> In-Reply-To: <9770910a-f586-0b79-395c-7154c4693690@amsat.org> From: Cleber Rosa Junior Date: Tue, 8 Jun 2021 09:36:37 -0400 Message-ID: Subject: Re: [PATCH v6 4/4] Jobs based on custom runners: add job definitions for QEMU's machines To: =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=crosa@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: multipart/alternative; boundary="000000000000f3ebd905c4413fca" Received-SPF: pass client-ip=170.10.133.124; envelope-from=crosa@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.197, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Peter Maydell , Thomas Huth , "Daniel P . Berrange" , Beraldo Leal , Erik Skultety , Stefan Hajnoczi , =?UTF-8?Q?Philippe_Mathieu=2DDaud=C3=A9?= , qemu-devel , Wainer dos Santos Moschetta , Andrea Bolognani , Willian Rampazzo , Willian Rampazzo , =?UTF-8?B?QWxleCBCZW5uw6ll?= , Eduardo Habkost Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" --000000000000f3ebd905c4413fca Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, Jun 8, 2021 at 2:30 AM Philippe Mathieu-Daud=C3=A9 wrote: > Hi Alex, Stefan, > > On 6/8/21 5:14 AM, Cleber Rosa wrote: > > The QEMU project has two machines (aarch64 and s390x) that can be used > > for jobs that do build and run tests. > > AFAIK there is more hardware available to the project, so I'm wondering > what happened to the rest, is it a deliberate choice to start small? > Hi Phil, Yes, this series was deliberately focused on the first two machines owned and available to QEMU. > What will happen with the rest, since we are wasting resources? > The plan is to allow all machines (currently available and to-be available) to be connected as custom runners. This hopefully gets that started. About "more hardware available to the project", there's one VM from fosshost which was made available not long ago, and which I set up even more recently, which could be used as a gitlab runner too. But, even though some new hardware resource is available (and wasted?), the human resources to maintain them have not been properly determined, so I believe it's a good decision to start with the machines that have been operational for long, and that already have to the best of my knowledge, people maintaining them. I also see a "Debian unstable mips64el (Debian) @ cipunited.cn" registered as a runner, but I don't have extra information about it. Besides that, I'll send another series shortly, that builds upon this series, and adds a Red Hat focused job, on a Red Hat managed machine. This should be what other entities should be capable of doing and allowed to do. > Who has access to what and should do what (setup)? How is this list of > hw managed btw? Should there be some public visibility (i.e. Wiki)? > These are good questions, and I believe Alex can answer them about those two machines. Even though I hooked them up to GitLab, AFAICT he is the ultimate admin (maybe Peter too?). About hardware management, it has been suggested to use either the Wiki or a MAINTAINERS entry. This is still unresolved and feedback would be appreciated. For me to propose a MAINTAINERS entry, say, on a v7, I'd need the information on who is managing them. > Is there a document explaining what are the steps to follow for an > entity to donate / sponsor hardware? Where would it be stored, should > this hw be shipped somewhere? What documentation should be provided for > its system administration? > > In case an entity manages hosting and maintenance, can the QEMU project > share the power bill? Up to which amount? > Similar question if a sysadmin has to be paid. > > If the QEMU project can't spend money on CI, what is expected from > resource donators? Simply hardware + power (electricity) + network > traffic? Also sysadmining and monitoring? Do we expect some kind of > reliability on the data stored here or can we assume disposable / > transient runners? > Should donators also provide storage? Do we have a minimum storage > requirement? > > Should we provide some guideline explaining any code might be run by > our contributors on these runners and some security measures have to > be taken / considered? > > Sponsors usually expect some advertising to thanks them, and often > regular reports on how their resources are used, else they might not > renew their offer. Who should care to keep the relationship with > sponsors? > > Where is defined what belong to the project, who is responsible, who can > request access to this hardware, what resource can be used? > > You obviously directed the question towards Alex and Stefan (rightfully so), so I won't attempt to answer these ones at this point. > More generically, what is the process for a private / corporate entity > to register a runner to the project? (how did it work for this aarch64 > and s390x one?) > The steps are listed on the documentation. Basically anyone with knowledge of the "registration token" can add new machines to GitLab as runners. For the two aarch64 and s390x, it was a matter of following the documentation steps which basically involve: 1) providing the hostname(s) in the inventory file 2) provide the "registration token" in the vars.yml file 3) running the playbooks > > What else am I missing? > > I think you're missing the answers to all your good questions :). And I understand that are a lot of them (from everyone, including myself). The dilemma here is: should we activate the machines already available, and learn in practice, what's missing? I honestly believe we should. Thanks, - Cleber. > Thanks, > > Phil. > > > This introduces those jobs, > > which are a mapping of custom scripts used for the same purpose. > > > > Signed-off-by: Cleber Rosa > > --- > > .gitlab-ci.d/custom-runners.yml | 208 ++++++++++++++++++++++++++++++++ > > 1 file changed, 208 insertions(+) > > --000000000000f3ebd905c4413fca Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Tue, Jun 8, 2021 at 2:30 AM Philip= pe Mathieu-Daud=C3=A9 <f4bug@amsat.or= g> wrote:
Hi Alex, Stefan,

On 6/8/21 5:14 AM, Cleber Rosa wrote:
> The QEMU project has two machines (aarch64 and s390x) that can be used=
> for jobs that do build and run tests.

AFAIK there is more hardware available to the project, so I'm wondering=
what happened to the rest, is it a deliberate choice to start small?

Hi Phil,

Yes, this s= eries was deliberately focused on the first two machines owned and availabl= e to QEMU.=C2=A0
=C2=A0
What will happen with the rest, since we are wasting resources?

The plan is to allow all machines (currently avail= able and to-be available) to be connected as custom runners.=C2=A0 This hop= efully gets that started.

About "more hardwar= e available to the project", there's one VM from fosshost which wa= s made available not long ago, and which I set up even more recently, which= could be used as a gitlab runner too.=C2=A0 But, even though some new hard= ware resource is available (and wasted?), the human resources to maintain t= hem have not been properly determined, so I believe it's a good decisio= n to start with the machines that have been operational for long, and that = already have to the best of my knowledge, people maintaining them.

I also see a "Debian unstable mips64el (Debian) @ cipunited.cn" registered as a runner,= but I don't have extra information about it.

<= div>Besides that, I'll send another series shortly, that builds upon th= is series, and adds a Red Hat focused job, on a Red Hat managed machine.=C2= =A0 This should be what other entities should be capable of doing and allow= ed to do.
=C2=A0
Who has access to what and should do what (setup)? How is this list of
hw managed btw? Should there be some public visibility (i.e. Wiki)?

These are good questions, and I believe Alex c= an answer them about those two machines.=C2=A0 Even though I hooked them up= to GitLab, AFAICT he is the ultimate admin (maybe Peter too?).
<= br>
About hardware management, it has been suggested to use eithe= r the Wiki or a MAINTAINERS entry.=C2=A0 This is still unresolved and feedb= ack would be appreciated.=C2=A0 For me to propose a MAINTAINERS entry, say,= on a v7, I'd need the information on who is managing them.
<= br>

Is there a document explaining what are the steps to follow for an
entity to donate / sponsor hardware? Where would it be stored, should
this hw be shipped somewhere? What documentation should be provided for
its system administration?

In case an entity manages hosting and maintenance, can the QEMU project
share the power bill? Up to which amount?
Similar question if a sysadmin has to be paid.

If the QEMU project can't spend money on CI, what is expected from
resource donators? Simply hardware + power (electricity) + network
traffic? Also sysadmining and monitoring? Do we expect some kind of
reliability on the data stored here or can we assume disposable /
transient runners?
Should donators also provide storage? Do we have a minimum storage
requirement?

Should we provide some guideline explaining any code might be run by
our contributors on these runners and some security measures have to
be taken / considered?

Sponsors usually expect some advertising to thanks them, and often
regular reports on how their resources are used, else they might not
renew their offer. Who should care to keep the relationship with
sponsors?

Where is defined what belong to the project, who is responsible, who can request access to this hardware, what resource can be used?


You obviously directed the question to= wards Alex and Stefan (rightfully so), so I won't attempt to answer the= se ones at this point.
=C2=A0
More generically, what is the process for a private / corporate entity
to register a runner to the project? (how did it work for this aarch64
and s390x one?)

The steps are listed on= the documentation.=C2=A0 Basically anyone with knowledge of the "regi= stration token" can add new machines to GitLab as runners.=C2=A0 For t= he two aarch64 and s390x, it was a matter of following the documentation st= eps which basically involve:

1) providing the host= name(s) in the inventory file
2) provide the "registration t= oken" in the vars.yml file
3) running the playbooks
=C2=A0

What else am I missing?


I think you're missing the answers= to all your good questions :).

And I understand t= hat are a lot of them (from everyone, including myself).=C2=A0 The dilemma = here is: should we activate the machines already available, and learn in pr= actice, what's missing?=C2=A0 I honestly believe we should.
<= br>
Thanks,
- Cleber.
=C2=A0
Thanks,

Phil.

> This introduces those jobs,
> which are a mapping of custom scripts used for the same purpose.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> ---
>=C2=A0 .gitlab-ci.d/custom-runners.yml | 208 ++++++++++++++++++++++++++= ++++++
>=C2=A0 1 file changed, 208 insertions(+)

--000000000000f3ebd905c4413fca--