Re: [RFC PATCH] terraform/OCI: Grab secrets from ~/.oci/config

[Chuck Lever <chuck.lever@oracle.com>]

On 4/4/25 3:19 PM, Luis Chamberlain wrote:
> On Fri, Apr 04, 2025 at 02:35:19PM -0400, Chuck Lever wrote:
>> On 4/4/25 2:28 PM, Luis Chamberlain wrote:
>>> On Fri, Apr 04, 2025 at 02:24:37PM -0400, Chuck Lever wrote:
>>>> On 4/4/25 2:06 PM, Luis Chamberlain wrote:
>>>>> On Fri, Apr 04, 2025 at 12:10:49PM -0400, Chuck Lever wrote:
>>>>>> On a related topic, if we ever want to fully support running the kdevops
>>>>>> /control host/ in the cloud, terraform supports an authentication
>>>>>> mechanism that just uses the local instance's service principal, so
>>>>>> no extra authentication material is needed for provisioning the test
>>>>>> runners as separate cloud instances. Interesting to consider.
>>>>>
>>>>> Sorry I failed to understand this, what is mean by separate cloud
>>>>> instances?
>>>>
>>>> The usual situation is the control host (where terraform runs) is
>>>> outside the cloud. Like, on your workstation. I'm talking about a
>>>> scenario where kdevops and terraform are running on an instance in the
>>>> same cloud as where your target nodes are going to run.
>>>>
>>>> In that case, terraform can scoop up the service principal for that
>>>> instance, and use it in place of dot file authentication parameters.
>>>> So you don't have to maintain the dot file on the instance that is
>>>> running terraform, if it is already in the cloud.
>>>
>>> I see, so you first bring that cloud instance, and then use that as
>>> your command and control for test nodes. Does that first cloud instance
>>> need to be instantiated through another kdevops setup?
>>
>> No, you can create it with terraform or from the cloud console. Start it
>> up when you want, or leave it running.
>>
>> I'm wondering if we can get a cloud devops pipeline to trigger it when
>> it sees a PR against a watched repo. Haven't really thought that
>> through.
> 
> OK well so when we add a new commit to kdevops we should ideally try to
> run a test if the commit is related to terraform or all commits to just
> verify we don't break cloud instance support. Given we already rely on
> dedicated github hosted servers, if such hosted servers have the
> credential files in place then the below would work to automatically
> trigger a cloud instance test.
> 
> For example below could be a .github/workflows/aws.yml which would
> run on every single commit we submit to kdevops. This would just
> do a bringup and destroy.
> 
> If the console creates a cloud instance, and github can get
> access to it as a dedicated self hosted runner, and it will
> persist then that guest can simply just be a bare bones guest
> to run ansible as the command and control center. And the example
> below could be used to push github actions onto the runner.
> 
> Could this step be removed and have github directly do the bringup?

I think you can hide the authentication material in one or more GitHub
secrets, if we were to explicitly enable support for passing terraform
authentication parameters via environment variables.

Then the GH Action can just run the terraform steps ("make bringup"
and "make destroy") without effort. But more below.


> I suspect we can rely on github docker containers which just run
> kdevops configs for the cloud for it, but they won't be hosted on
> the respective cloud of choice. Other than that, it would be nice
> if github integrated support for instantiating a simple guest just
> to be command control. Otherwise a dedicated self hosted runner outside
> of the cloud could just have the credentials file.

As far as I can tell, the major four all implement devops pipelines that
can hook into GitHub pull requests and bring up instances and what not.
Seems like that might obviate the need for managing authentication
material on GitHub itself...?

Check out AWS CloudBuild:

https://docs.aws.amazon.com/codebuild/latest/userguide/concepts.html

You give it a little yaml file that tells it to watch a GitHub repo
and then execute steps on AWS when it notices a PR. Very much like a
GitHub Action, but the runner is on AWS and not GitHub, and because
the activity is triggered by an AWS service (CloudBuild) the API
authentication step is already done.

OCI calls this "Instance Principal Authentication":

https://docs.oracle.com/en-us/iaas/Content/API/Concepts/sdk_authentication_methods.htm#sdk_authentication_methods_instance_principaldita

Basically the service principal in the instance where terraform is
running is used to provision new terraform resources.

So we set up one of these CloudBuild projects for each cloud provider,
and for any PR in linux-kdevops/kdevops, it will try a bringup/destroy
sequence on that cloud. The build logs can be placed in a public spot
on the AWS web site.

</handwave>

So I would need to add a Kconfig widget to each cloud provider's
submenu to select "Instance Principal Authentication" (or whatever
terminology that cloud provider users) and then kdevops will set up
the terraform provider block properly for that scenario.


> The kdeops-ci repo [0] has sufficient workflow branch examples, some of which
> we are already using for different subsystems to trigger runs with
> kevops on dedicated servers. If a subsystem wanted to leverage the cloud
> their defconfig would just have the setup for the respective cloud of
> choice. For example defconfigs/linux-xfs-kpd is what se use for testing
> pushes to branches to the linux-kdevops/linux-xfs-kpd and only Carlos
> has access to push to that repo. Which dedicated self hosted runner is
> opaque to kdevops, but it can easily have cloud credentials to easily
> do bringup.
> 
> I hadn't enabled the below on kdevops just because of lack of time of
> testing, and trying to then optimize the cost so that a super bare bones
> cloud istanace would be used for every kdevops commit pushed.

As long as the instance configurations are fairly minimal, they can
be kept in the free tier on most if not all cloud providers. You do
need an active credit card to open a cloud account, though.


> However,
> that's a bit extreme of course, and we can tone it down to only commits
> perhaps that touch kdevops. But if that works, then by all means so
> would a push to a git tree as we do with linux-kdevops/linux-xfs-kpd
> using the examples in the branches for /linux-kdevops/kdevops-ci.
> 
> [0] https://github.com/linux-kdevops/kdevops-ci
> 
> name: Run kdevops on self-hosted runner for aws cloud instances
> 
> on:
>   workflow_dispatch:  # Add this for manual triggering of the workflow
> 
> jobs:
>   run-kdevops:
>     name: Run kdevops CI
>     runs-on: [self-hosted, Linux, X64]
>     steps:
>       - name: Checkout repository
>         uses: actions/checkout@v4
> 
>       - name: Set CI metadata for kdevops-results-archive
>         run: |
>           echo "$(basename ${{ github.repository }})" > ci.trigger
>           LOG=$(git log -1 --pretty=format:"%s") > ci.subject
>           echo "cloud aws:" $LOG" > ci.subject
>           # Start out pessimistic
>           echo "not ok" > ci.result
>           echo "Nothing to write home about." > ci.commit_extra
> 
>       - name: Set kdevops path
>         run: echo "KDEVOPS_PATH=$GITHUB_WORKSPACE" >> $GITHUB_ENV
> 
>       - name: Configure git
>         run: |
>           git config --global --add safe.directory '*'
>           git config --global user.name "kdevops"
>           git config --global user.email "kdevops@lists.linux.dev"
> 
>       - name: Run kdevops make defconfig-repo
>         run: |
>           KDEVOPS_TREE_REF="${{ github.event_name == 'pull_request' && github.event.pull_request.head.sha || github.sha }}"
>           SHORT_PREFIX="$(echo ${KDEVOPS_TREE_REF:0:12})"
>           make KDEVOPS_HOSTS_PREFIX="$SHORT_PREFIX" defconfig-aws
> 
>       - name: Run kdevops make
>         run: |
>           make -j$(nproc)
> 
>       - name: Run kdevops make bringup
>         run: |
>           make bringup
> 
>       - name: Just check the kernel
>         run: |
>           make uname
>           echo "ok" > ci.result
> 
>       - name: Start SSH Agent
>         if: always()  # Ensure this step runs even if previous steps failed
>         uses: webfactory/ssh-agent@v0.9.0
>         with:
>           ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
> 
>       - name: Build our kdevops archive results
>         if: always() # This ensures the step runs even if previous steps failed
>         run: |
>           make ci-archive
> 
>       - name: Upload our kdevops results archive
>         if: always() # This ensures the step runs even if previous steps failed
>         uses: actions/upload-artifact@v4
>         with:
>           name: kdevops-ci-results
>           path: ${{ env.KDEVOPS_PATH }}/archive/*.zip
> 
>       # Ensure make destroy always runs, even on failure
>       - name: Run kdevops make destroy
>         if: always()  # This ensures the step runs even if previous steps failed
>         run: |
>           make destroy


-- 
Chuck Lever