Kernel customized do_fetch issue

Kernel customized do_fetch issue

[Joel (Xi Zhou) Zhou]

[-- Attachment #1: Type: text/plain, Size: 811 bytes --]

Hi all,

I created a do_fetch() for checkout in-house git repo, which is working fine with OE.
do_fetch() {
    cd ${WORKDIR}
    rm -rf ${PN}-${PV}
    git clone ssh://svcswgit@git-ccxsw.inhouse.com/linux-lsk ${PN}-${PV}
    cd ${PN}-${PV}
    git checkout ${KBRANCH}
}

With Yocto, the do_fetch() does its job, but do_configure create an issue.
do_configure_prepend() {
    cp ${WORKDIR}/${KERNEL_CONFIG_FILE} ${S}/.config
    oe_runmake oldconfig
}
The do_configure error message:
| make: *** No rule to make target `oldconfig'.  Stop.

Basically, the kernel source checkout by do_fetch() is delete/erase while running do_configure().
I suspect some tasks between do_fetch and do_configure are doing some magic work of moving the kernel  source around.
Any suggestion?

Thanks,
Joel

[-- Attachment #2: Type: text/html, Size: 3686 bytes --]

Re: Kernel customized do_fetch issue

[Bruce Ashfield]

On 2015-04-29 01:39 PM, Joel (Xi Zhou) Zhou wrote:
> Hi all,
>
> I created a do_fetch() for checkout in-house git repo, which is working
> fine with OE.
>
> do_fetch() {
>
>      cd ${WORKDIR}
>
>      rm -rf ${PN}-${PV}
>
>      git clone ssh://svcswgit@git-ccxsw.inhouse.com/linux-lsk ${PN}-${PV}
>
>      cd ${PN}-${PV}
>
>      git checkout ${KBRANCH}
>
> }
>
> With Yocto, the do_fetch() does its job, but do_configure create an issue.
>
> do_configure_prepend() {
>
>      cp ${WORKDIR}/${KERNEL_CONFIG_FILE} ${S}/.config
>
>      oe_runmake oldconfig
>
> }
>
> The do_configure error message:
>
> | make: *** No rule to make target `oldconfig'.  Stop.
>
> Basically, the kernel source checkout by do_fetch() is delete/erase
> while running do_configure().
>
> I suspect some tasks between do_fetch and do_configure are doing some
> magic work of moving the kernel  source around.

It's better if you can post your entire kernel recipe, and what
branch/release you are using.

There are steps that move the kernel source into work-shared, so that
may be impacting your flow.

But the question has to be asked. Why exactly are you manually fetching
the kernel ? The fetcher can take care of most everything.

If you need to modify the source directory later, have a look at the
steps that I'm taking in kernel-yocto.bbclass, since that works properly
within the fetcher and build infrastructure.

Bruce

>
> Any suggestion?
>
> Thanks,
>
> Joel
>
>
>

Re: Kernel customized do_fetch issue

[Joel (Xi Zhou) Zhou]

> But the question has to be asked. Why exactly are you manually fetching
> the kernel ? The fetcher can take care of most everything.

The whole story is starting at the url of our git repo. We have a git URL like:
	ssh://svcswgit@git-ccxsw.inhouse.com/linux-lsk

So the SRC_URI like this, but the fetcher try to apply "scp" over it.
	KBRANCH= 3.14_common_dev
	SRC_URI = "ssh://svcswgit@git-ccxsw.inhouse.com/linux-lsk;bareclone=1;branch=${KBRANCH}"
Error:
	ERROR: Fetcher failure: Fetch command failed with exit code 1, output:
	FATAL: unknown git/gitolite command: 'scp -r -f linux-lsk'

Then I change it  according the suggestion in this mailing list:
	SRC_URI = "git://svcswgit@git-ccxsw.inhouse.com/linux-lsk;bareclone=1;branch=${KBRANCH};protocol=ssh"

This time "bitbake linux-lsk -c fetch" return no error, but the source folder in working directory is empty. 
	In log.do_fetch
	DEBUG: Fetcher failure: Fetch command failed with exit code 8, output:
	http://downloads.yoctoproject.org/mirror/sources/git2_git-ccxsw.rtp. inhouse.com. linux-lsk.tar.gz:
	2015-04-29 12:49:36 ERROR 404: Not Found.

Thanks,
Joel

Re: Kernel customized do_fetch issue

[Bruce Ashfield]

On 2015-04-29 02:08 PM, Joel (Xi Zhou) Zhou wrote:
>> But the question has to be asked. Why exactly are you manually fetching
>> the kernel ? The fetcher can take care of most everything.
>
> The whole story is starting at the url of our git repo. We have a git URL like:
> 	ssh://svcswgit@git-ccxsw.inhouse.com/linux-lsk
>
> So the SRC_URI like this, but the fetcher try to apply "scp" over it.
> 	KBRANCH= 3.14_common_dev
> 	SRC_URI = "ssh://svcswgit@git-ccxsw.inhouse.com/linux-lsk;bareclone=1;branch=${KBRANCH}"
> Error:
> 	ERROR: Fetcher failure: Fetch command failed with exit code 1, output:
> 	FATAL: unknown git/gitolite command: 'scp -r -f linux-lsk'
>
> Then I change it  according the suggestion in this mailing list:
> 	SRC_URI = "git://svcswgit@git-ccxsw.inhouse.com/linux-lsk;bareclone=1;branch=${KBRANCH};protocol=ssh"
>
> This time "bitbake linux-lsk -c fetch" return no error, but the source folder in working directory is empty.
> 	In log.do_fetch
> 	DEBUG: Fetcher failure: Fetch command failed with exit code 8, output:
> 	http://downloads.yoctoproject.org/mirror/sources/git2_git-ccxsw.rtp. inhouse.com. linux-lsk.tar.gz:
> 	2015-04-29 12:49:36 ERROR 404: Not Found.

I'd expect that it part of the log, since obviously the mirrors aren't
going to have a copy of your kernel.

You really aren't seeing anything land in the build/downloads/git2/
directory structure ? ssh fetches seem to work here.

You'd be better of working through the issues and getting the right
SRC_URI specification for the fetcher, since taking the fetch into a
custom routine is going to short circuit parts of the build .. and you'll
have to take care of them yourself.

Cheers,

Bruce

>
> Thanks,
> Joel
>

Re: Kernel customized do_fetch issue

[Joel (Xi Zhou) Zhou]

Thanks Bruce!
> You really aren't seeing anything land in the build/downloads/git2/
> directory structure ? ssh fetches seem to work here.

I do see something in build/downloads/git2/git-ccxsw.xxx.com.linux-lsk, but it looks like an empty git.
	HEAD  branches	config	description  hooks  info  objects  packed-refs	refs

I bring up this issue to IT department, what they said is, the Yocto expects the url contains  host/repo, while gitolit requires host:repo.
This is interesting since the Yocto documentation explicitly mentions gitolite in the SCM section:
http://www.yoctoproject.org/docs/current/dev-manual/dev-manual.html#best-practices-source-control-management

Thanks,
Joel