Http access token fetching with gitsm fetcher

Http access token fetching with gitsm fetcher

[alexzoyv]

[-- Attachment #1.1: Type: text/plain, Size: 543 bytes --]

Greetings!
I try to use gitsm fetcher to fetch bitbucket repository with read-only https access token. SRC_URI looks like this
> 
> 
> SRC_URI =
> "gitsm://******.git;protocol=https;branch=${BRANCH};user=azoykin:${TOKEN}"
> 
> 

This recipe fetches parent repository, but is unable to fetch submodule, writing fatal: could not read Username for 'https://tps-git.topcon.com': No such device or address in log.do_fetch
It happens because url is passed to fetcher function already with user string. I suggest a patch to fix this issue

[-- Attachment #1.2: Type: text/html, Size: 605 bytes --]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: gits.patch --]
[-- Type: text/x-patch; name="gits.patch", Size: 704 bytes --]

diff --git a/lib/bb/fetch2/gitsm.py b/lib/bb/fetch2/gitsm.py
index c5f7c03c..3a025624 100644
--- a/lib/bb/fetch2/gitsm.py
+++ b/lib/bb/fetch2/gitsm.py
@@ -100,6 +100,10 @@ class GitSM(Git):
                 # Properly formated URL already
                 proto = uris[module].split(':', 1)[0]
                 url = uris[module].replace('%s:' % proto, 'gitsm:', 1)
+                # remove user+password from string to add them as parameter
+                if ud.user in url:
+                    url = url.replace(ud.user + '@', '')
+                    url += ";user=%s" % ud.user
             else:
                 if ":" in uris[module]:
                     # Most likely an SSH style reference

Re: [yocto] Http access token fetching with gitsm fetcher

[Adrian Freihofer]

[-- Attachment #1: Type: text/plain, Size: 1128 bytes --]

Did you already consider to write the credentials in a  .netrc file?

Regards
Adrian

<alexzoyv@gmail.com> schrieb am Fr., 20. Okt. 2023, 13:55:

> Greetings!
> I try to use gitsm fetcher to fetch bitbucket repository with read-only
> https access token. SRC_URI looks like this
>
> SRC_URI =
> "gitsm://******.git;protocol=https;branch=${BRANCH};user=azoykin:${TOKEN}"
>
> This recipe fetches parent repository, but is unable to fetch submodule,
> writing *fatal: could not read Username for 'https://tps-git.topcon.com
> <https://tps-git.topcon.com>': No such device or address *in log.do_fetch
> It happens because url is passed to fetcher function already with user
> string. I suggest a patch to fix this issue
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#61430):
> https://lists.yoctoproject.org/g/yocto/message/61430
> Mute This Topic: https://lists.yoctoproject.org/mt/102079282/4454582
> Group Owner: yocto+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [
> adrian.freihofer@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 2088 bytes --]

Re: [yocto] Http access token fetching with gitsm fetcher

[Yoann CONGAL]

Hi,

Le ven. 20 oct. 2023 à 15:32, Adrian Freihofer
<adrian.freihofer@gmail.com> a écrit :
> Did you already consider to write the credentials in a  .netrc file?
>
> Regards
> Adrian
>
> <alexzoyv@gmail.com> schrieb am Fr., 20. Okt. 2023, 13:55:
>>
>> Greetings!
>> I try to use gitsm fetcher to fetch bitbucket repository with read-only https access token. SRC_URI looks like this
>>
>> SRC_URI = "gitsm://******.git;protocol=https;branch=${BRANCH};user=azoykin:${TOKEN}"
>>
>> This recipe fetches parent repository, but is unable to fetch submodule, writing fatal: could not read Username for 'https://tps-git.topcon.com': No such device or address in log.do_fetch
>> It happens because url is passed to fetcher function already with user string. I suggest a patch to fix this issue

I'm sorry but this is a known WONTFIX bug :
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13550
See : https://git.openembedded.org/bitbake/tree/lib/bb/fetch2/gitsm.py#n127

The only supported way to do this (that I know of) is to use .netrc as
Adrian suggested.

Regards,
-- 
Yoann Congal
Smile ECS - Tech expert

Re: Http access token fetching with gitsm fetcher

[alexzoyv]

[-- Attachment #1: Type: text/plain, Size: 1403 bytes --]

Adrian, Yoann,
Thank you very much and sorry for bringing it up.
.netrc does fix my problem.
Alex
On Fri, Oct 20, 2023 at 04:43 PM, Yoann Congal wrote:

> 
> Hi,
> 
> Le ven. 20 oct. 2023 à 15:32, Adrian Freihofer
> <adrian.freihofer@gmail.com> a écrit :
> 
>> Did you already consider to write the credentials in a .netrc file?
>> 
>> Regards
>> Adrian
>> 
>> <alexzoyv@gmail.com> schrieb am Fr., 20. Okt. 2023, 13:55:
>> 
>>> 
>>> Greetings!
>>> I try to use gitsm fetcher to fetch bitbucket repository with read-only
>>> https access token. SRC_URI looks like this
>>> 
>>> SRC_URI =
>>> "gitsm://******.git;protocol=https;branch=${BRANCH};user=azoykin:${TOKEN}"
>>> 
>>> 
>>> This recipe fetches parent repository, but is unable to fetch submodule,
>>> writing fatal: could not read Username for ' https://tps-git.topcon.com' :
>>> No such device or address in log.do_fetch
>>> It happens because url is passed to fetcher function already with user
>>> string. I suggest a patch to fix this issue
>> 
>> 
> 
> I'm sorry but this is a known WONTFIX bug :
> https://bugzilla.yoctoproject.org/show_bug.cgi?id=13550
> See : https://git.openembedded.org/bitbake/tree/lib/bb/fetch2/gitsm.py#n127
> 
> 
> The only supported way to do this (that I know of) is to use .netrc as
> Adrian suggested.
> 
> Regards,
> --
> Yoann Congal
> Smile ECS - Tech expert

[-- Attachment #2: Type: text/html, Size: 1782 bytes --]

Re: [yocto] Http access token fetching with gitsm fetcher

[Altous, Salahaldeen]

[-- Attachment #1: Type: text/plain, Size: 733 bytes --]

Hi Adrian,

Hi Yoann,

I have two project when is using dunfell and another one with using Kirkstone. with the Dunfell project I was using one recipe with fetching from git repo like this

SRC_URI = "git://${ COMPANY_GIT_REPO }/path/to/project.git;protocol=https; *user=token_name:${TOKEN}"*

The https+Token was working fine with Dunfell, now I have ported this recipe to kirkstone but it is not working and I get this error

fatal: could not read Password for 'https://git@COMPANY_GIT_REPO': No such device or address

Is this known WONTFIX bug was also in dunfell? or introduced only in the new yocto version?

is this only applied for gitsm or also for git?

Thanks in advance.

Regards,

Salahaldeen Altous

[-- Attachment #2: Type: text/html, Size: 2551 bytes --]

RE: [yocto] Http access token fetching with gitsm fetcher

[Janne Kiiskila]

[-- Attachment #1: Type: text/plain, Size: 1404 bytes --]

  *   SRC_URI = "git://${COMPANY_GIT_REPO}/path/to/project.git;protocol=https;user=token_name:${TOKEN}"
  *   The https+Token was working fine with Dunfell, now I have ported this recipe to kirkstone but it is not working and I get this error
  *   fatal: could not read Password for 'https://git@COMPANY_GIT_REPO': No such device or address

 Finally, something I might be able to contribute with.

  1.  Do not put ever any credentials to any recipes. That’s not the right place for them and they will leak.
  2.  Credentials are a build machine configuration issue. You can use git redirects for this purpose.

This type of thing is only required for private repositories typically, here is an example for a GitHub action

https://github.com/PelionIoT/actions

that does it. But, essentially it boils down to configuring git:



          git config --global user.name "Your BOT"

          git config --global user.email your@email.com<mailto:your@email.com>

          git config --global url.https://${{ inputs.github_token }}@github.com/<https://$%7b%7b%20inputs.github_token%20%7d%7d@github.com/>.insteadOf git@github.com:<mailto:git@github.com:>

          git config --global url.https://${{ inputs.github_token }}@github<https://$%7b%7b%20inputs.github_token%20%7d%7d@github>.insteadOf https://github

I hope this helps.



Best Regards,



Janne Kiiskilä

[-- Attachment #2: Type: text/html, Size: 10507 bytes --]

Re: [yocto] Http access token fetching with gitsm fetcher

[Altous, Salahaldeen]

[-- Attachment #1: Type: text/plain, Size: 1744 bytes --]

Hi Janne,

On Fri, Apr 19, 2024 at 06:44 AM, Janne Kiiskila wrote:

> 
> 
> 
> * SRC_URI = "git://${ COMPANY_GIT_REPO }/path/to/project.git;protocol=https;
> *user=token_name:${TOKEN}"*
> * The https+Token was working fine with Dunfell, now I have ported this
> recipe to kirkstone but it is not working and I get this error
> * fatal: could not read Password for 'https://git@COMPANY_GIT_REPO': No
> such device or address
> 
> 
> 
> 
> Finally, something I might be able to contribute with.
> 
> * Do not put ever any credentials to any recipes. That’s not the right
> place for them and they will leak.
> * Credentials are a build machine configuration issue. You can use git
> redirects for this purpose.
> 
> 
> This type of thing is only required for private repositories typically,
> here is an example for a GitHub action
> 
> 
> 
> https://github.com/PelionIoT/actions
> 
> 
> 
> that does it. But, essentially it boils down to configuring git:
> 
> 
> 
> 
> 
> 
> 
> git config --global user.name "Your BOT"
> 
> 
> 
> git config --global user.email your@email.com
> 
> 
> 
> git config --global url. https://${{ inputs.github_token }}@github.com/ (
> https://$%7b%7b%20inputs.github_token%20%7d%7d@github.com/ ).insteadOf git@github.com:
> 
> 
> 
> 
> git config --global url. https://${{ inputs.github_token }}@github (
> https://$%7b%7b%20inputs.github_token%20%7d%7d@github ).insteadOf https://github
> 
> 
> 
> 
> I hope this helps.
> 
> 
> 
> 
> 
> 
> 

thanks for your help, I will give it a try.

> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> Best Regards,
> 
> 
> 
> 
> 
> 
> 
> Janne Kiiskilä
> 
> 
> 

Regards,

Salahaldeen Altous

[-- Attachment #2: Type: text/html, Size: 4210 bytes --]