newbie questions

newbie questions

[Pierre-Louis Bossart]

Hi,
I am trying to figure out how complicated it'd be to add the sound subsystem to the list of supported packages - there are many cases where I only want the latest and greatest sound drivers and nothing else beyond the minimal dependencies. Bear with me if I am asking silly questions, I am not familiar at all with the backports environment and only tried to progress from the wiki and code.

question1: is there a way the list of packages can be selected? I'd like to use the backports infrastructure only for what I care about (media, usb, audio). It'd be really nice if I could limit the updates to the parts I care about (media, usb, sound) and remove all the networking part I don't really have a use for. It's fine if the releases include everything, I only want to limit the changes in my own workspace.

question2: I am not sure I understand how the code is generated. Nothing seems to work/compile even without my changes... If I use linux-next with the next-20150731 tag and the latest backport code, I get:

~/backports$ ./gentree.py --clean --verbose --gitdebug --git-revision  v4.1.6  ../linux-next ../linux-next-backport
Get original source files from git ...
Applying patches from patches to ../linux-next-backport ...
Applying patch 0000-upstream-backport-changes/0005-uapi-compromise/mwifiex.patch
> patching file drivers/net/wireless/mwifiex/decl.h
> Hunk #1 succeeded at 26 with fuzz 1.
> patching file drivers/net/wireless/mwifiex/sta_rx.c
Applying patch backport-adjustments/crypto-ccm.patch
> patching file compat/crypto-ccm.c
> Hunk #2 succeeded at 872 (offset -60 lines).
> Hunk #3 succeeded at 898 (offset -60 lines).
Applying patch backport-adjustments/devcoredump.patch
> patching file compat/drivers-base-devcoredump.c
> patching file include/linux/backport-devcoredump.h
Not applying backport-adjustments/flow_dissector.patch, not needed
Applying patch collateral-evolutions/media/0001-dma_mmap_coherent-revert.patch
> patching file drivers/media/v4l2-core/videobuf2-dma-contig.c
Applying patch collateral-evolutions/media/0002-no_dmabuf/v4l2.patch
> patching file drivers/media/v4l2-core/Kconfig
> Hunk #1 succeeded at 68 (offset -11 lines).
> Hunk #2 succeeded at 79 (offset -11 lines).
> patching file drivers/media/v4l2-core/v4l2-mem2mem.c
> Hunk #1 succeeded at 440 (offset -19 lines).
> Hunk #2 succeeded at 454 (offset -19 lines).
> Hunk #3 succeeded at 805 with fuzz 2 (offset -36 lines).
> Hunk #4 succeeded at 814 (offset -36 lines).
> patching file drivers/media/v4l2-core/videobuf2-core.c
> Hunk #1 succeeded at 252 (offset -1 lines).
> Hunk #2 succeeded at 281 (offset -1 lines).
> Hunk #3 succeeded at 425 (offset -1 lines).
> Hunk #4 succeeded at 787 (offset -1 lines).
> Hunk #5 succeeded at 801 (offset -1 lines).
> Hunk #6 succeeded at 835 (offset -1 lines).
> Hunk #7 succeeded at 1502 (offset -5 lines).
> Hunk #8 succeeded at 1621 (offset -5 lines).
> Hunk #9 succeeded at 1684 (offset -5 lines).
> Hunk #10 succeeded at 2074 (offset -10 lines).
> Hunk #11 succeeded at 2085 (offset -10 lines).
> Hunk #12 succeeded at 2094 (offset -10 lines).
> Hunk #13 succeeded at 2401 (offset -14 lines).
> Hunk #14 succeeded at 2479 (offset -14 lines).
> Hunk #15 succeeded at 3417 (offset -22 lines).
> Hunk #16 succeeded at 3427 (offset -22 lines).
> patching file drivers/media/v4l2-core/videobuf2-dma-contig.c
> patching file drivers/media/v4l2-core/videobuf2-vmalloc.c
> patching file include/media/v4l2-mem2mem.h
> Hunk #1 succeeded at 119 (offset -2 lines).
> Hunk #2 succeeded at 244 (offset -2 lines).
> patching file include/media/videobuf2-core.h
> Hunk #3 succeeded at 463 (offset -4 lines).
> Hunk #4 succeeded at 630 (offset -13 lines).
> patching file drivers/media/usb/uvc/uvc_queue.c
> Hunk #1 FAILED at 270.
> Hunk #2 succeeded at 269 (offset -12 lines).
> 1 out of 2 hunks FAILED -- saving rejects to file drivers/media/usb/uvc/uvc_queue.c.rej
> patching file drivers/media/v4l2-core/videobuf2-dma-sg.c
> Hunk #1 succeeded at 373 (offset -4 lines).
Traceback (most recent call last):
  File "./gentree.py", line 1091, in <module>
    ret = _main()
  File "./gentree.py", line 724, in _main
    logwrite=logwrite)
  File "./gentree.py", line 906, in process
    apply_patches(args, "backport", source_dir, 'patches', bpid.target_dir, logwrite)
  File "./gentree.py", line 552, in apply_patches
    raise Exception('Patch failed')
Exception: Patch failed

Now if I use branch origin/linux-4.1.y, the tree is generated but usually doesn't compile (random options selected)

Can anyone help with a known config that works?


question 3: how exactly does ckmake work and what am I missing?
:~/backports$ ./devel/ckmake 
[snip] backports/Makefile does not exist

Thanks,
-Pierre
--
To unsubscribe from this list: send the line "unsubscribe backports" in

Re: newbie questions

[Johannes Berg]

Hi,

> question1: is there a way the list of packages can be selected? I'd 
> like to use the backports infrastructure only for what I care about 
> (media, usb, audio). It'd be really nice if I could limit the updates 
> to the parts I care about (media, usb, sound) and remove all the 
> networking part I don't really have a use for. It's fine if the 
> releases include everything, I only want to limit the changes in my 
> own workspace.

In general, this is possible. Check out the copy-list file. You can
edit this, and it'll copy only what's listed there.

In order to add sound though, you'll also have to at least change
backport/Makefile.kernel and backport/Kconfig.sources to include the
sound directory in the kconfig and build.

The existing semantic patches will be applied to it automatically, but
it seems fairly unlikely that it'll all compile cleanly without more
effort.

> question2: I am not sure I understand how the code is generated. 
> Nothing seems to work/compile even without my changes... If I use 
> linux-next with the next-20150731 tag and the latest backport code, I 
> get:
> 
> ~/backports$ ./gentree.py --clean --verbose --gitdebug --git-revision
>   v4.1.6  ../linux-next ../linux-next-backport

You're not using next-20150731 tag? You're using v4.1.6 tag. If you
checked out linux-next in ../linux-next, then you can simply remove the
"--git-revision ..." argument and make the script a bit faster even.

> Exception: Patch failed

This is because you're not using the right tag.

> Now if I use branch origin/linux-4.1.y, the tree is generated but 
> usually doesn't compile (random options selected)
> 
> Can anyone help with a known config that works?
> 
> 
> question 3: how exactly does ckmake work and what am I missing?
> :~/backports$ ./devel/ckmake 
> [snip] backports/Makefile does not exist
> 

Don't worry about it for now. It's just a test tool to compile against
a lot of trees - you need to run it on the result of backport.

johannes
--
To unsubscribe from this list: send the line "unsubscribe backports" in

Re: newbie questions

[Pierre-Louis Bossart]

Thanks for the quick answers Johannes, more below.

On 08/31/2015 02:43 AM, Johannes Berg wrote:
> Hi,
> 
>> question1: is there a way the list of packages can be selected? I'd 
>> like to use the backports infrastructure only for what I care about 
>> (media, usb, audio). It'd be really nice if I could limit the updates 
>> to the parts I care about (media, usb, sound) and remove all the 
>> networking part I don't really have a use for. It's fine if the 
>> releases include everything, I only want to limit the changes in my 
>> own workspace.
> 
> In general, this is possible. Check out the copy-list file. You can
> edit this, and it'll copy only what's listed there.

ok. I was thinking of a more automated way that doesn't require me to maintain a branch but that'll do for now.

> In order to add sound though, you'll also have to at least change
> backport/Makefile.kernel and backport/Kconfig.sources to include the
> sound directory in the kconfig and build.

yes that part works fine. I still have to figure out how Kconfig works in the backports, I didn't get the explanations.

> The existing semantic patches will be applied to it automatically, but
> it seems fairly unlikely that it'll all compile cleanly without more
> effort.

indeed.

>> question2: I am not sure I understand how the code is generated. 
>> Nothing seems to work/compile even without my changes... If I use 
>> linux-next with the next-20150731 tag and the latest backport code, I 
>> get:
>>
>> ~/backports$ ./gentree.py --clean --verbose --gitdebug --git-revision
>>   v4.1.6  ../linux-next ../linux-next-backport
> 
> You're not using next-20150731 tag? You're using v4.1.6 tag. If you
> checked out linux-next in ../linux-next, then you can simply remove the
> "--git-revision ..." argument and make the script a bit faster even.

humm... What I wanted is backport next-20150731 to v4.1.6. linux-next is checked out to next-20150731.
which of the two does git-revision refer two? the linux-next origin or the destination?

> 
>> Exception: Patch failed
> 
> This is because you're not using the right tag.
> 
>> Now if I use branch origin/linux-4.1.y, the tree is generated but 
>> usually doesn't compile (random options selected)
>>
>> Can anyone help with a known config that works?
>>
>>
>> question 3: how exactly does ckmake work and what am I missing?
>> :~/backports$ ./devel/ckmake 
>> [snip] backports/Makefile does not exist
>>
> 
> Don't worry about it for now. It's just a test tool to compile against
> a lot of trees - you need to run it on the result of backport.
> 
> johannes
> --
> To unsubscribe from this list: send the line "unsubscribe backports" in
> 

--
To unsubscribe from this list: send the line "unsubscribe backports" in

Re: newbie questions

[Johannes Berg]

On Mon, 2015-08-31 at 07:50 -0500, Pierre-Louis Bossart wrote:
> 
> > In general, this is possible. Check out the copy-list file. You can
> > edit this, and it'll copy only what's listed there.
> 
> ok. I was thinking of a more automated way that doesn't require me to 
> maintain a branch but that'll do for now.

maintaining a branch of the copy-list is simple though.

> > In order to add sound though, you'll also have to at least change
> > backport/Makefile.kernel and backport/Kconfig.sources to include 
> > the
> > sound directory in the kconfig and build.
> 
> yes that part works fine. I still have to figure out how Kconfig 
> works in the backports, I didn't get the explanations.

You can submit this part upstream btw - it'll just be disabled in the
default copy-list... unless you also submit that upstream.

> humm... What I wanted is backport next-20150731 to v4.1.6. linux-next 
> is checked out to next-20150731.
> which of the two does git-revision refer two? the linux-next origin 
> or the destination?

The *to* version is never relevant. A given backport will compile
against many different *to* versions.

johannes
--
To unsubscribe from this list: send the line "unsubscribe backports" in

Re: newbie questions

[Pierre-Louis Bossart]

>> humm... What I wanted is backport next-20150731 to v4.1.6. linux-next 
>> is checked out to next-20150731.
>> which of the two does git-revision refer two? the linux-next origin 
>> or the destination?
> 
> The *to* version is never relevant. A given backport will compile
> against many different *to* versions.

I guess I completely missed the concept. I was thinking that the gentree.py command would only port and adjust the delta between linux-next and the version I wanted. Looks like the backport is really an add-on that will apply to multiple versions. Not sure I understand how successive changes in the tree are handled if there is a single backport.

--
To unsubscribe from this list: send the line "unsubscribe backports" in

Re: newbie questions

[Johannes Berg]

On Mon, 2015-08-31 at 08:21 -0500, Pierre-Louis Bossart wrote:
> > 
> > The *to* version is never relevant. A given backport will compile
> > against many different *to* versions.
> 
> I guess I completely missed the concept. I was thinking that the 
> gentree.py command would only port and adjust the delta between linux
> -next and the version I wanted. Looks like the backport is really an 
> add-on that will apply to multiple versions. Not sure I understand 
> how successive changes in the tree are handled if there is a single 
> backport.

Well, there's a single *from* version, as you say that was currently
"next-20150731" (or that was the one you used). The backport git
repository is maintained in lockstep with the *from* version (although
there's usually quite a bit of wiggle room)

The result, the output of backports, will/should compile against any
kernel starting from the earliest supported, all the way up to the
*from* version, right now I think that's 3.0 until 4.1 or so.

johannes
--
To unsubscribe from this list: send the line "unsubscribe backports" in

Re: newbie questions

[Pierre-Louis Bossart]

[-- Attachment #1: Type: text/plain, Size: 1393 bytes --]

On 08/31/2015 08:33 AM, Johannes Berg wrote:
> On Mon, 2015-08-31 at 08:21 -0500, Pierre-Louis Bossart wrote:
>>>
>>> The *to* version is never relevant. A given backport will compile
>>> against many different *to* versions.
>>
>> I guess I completely missed the concept. I was thinking that the 
>> gentree.py command would only port and adjust the delta between linux
>> -next and the version I wanted. Looks like the backport is really an 
>> add-on that will apply to multiple versions. Not sure I understand 
>> how successive changes in the tree are handled if there is a single 
>> backport.
> 
> Well, there's a single *from* version, as you say that was currently
> "next-20150731" (or that was the one you used). The backport git
> repository is maintained in lockstep with the *from* version (although
> there's usually quite a bit of wiggle room)
> 
> The result, the output of backports, will/should compile against any
> kernel starting from the earliest supported, all the way up to the
> *from* version, right now I think that's 3.0 until 4.1 or so. 

What I was really looking for is the --integrate option that does merge the backport into an existing tree.
This option is broken btw on newer kernels due to a change in the kernel Makefile in October 2014, the attached patch provides a correction (not sure how to use the right patch for the right kernel though).
-Pierre


[-- Attachment #2: 0001-enable-backports-built-in.patch --]
[-- Type: text/x-patch, Size: 1628 bytes --]

Allow backports to be integrated into vmlinux with newer Makefiles

diff --git a/Makefile b/Makefile
index f5c8983..31c7cbf 100644
--- a/Makefile
+++ b/Makefile
@@ -555,6 +555,7 @@ scripts: scripts_basic include/config/auto.conf include/config/tristate.conf \
 	$(Q)$(MAKE) $(build)=$(@)
 
 # Objects we will link into vmlinux / subdirs we need to visit
+backports-y 	:= backports/
 init-y		:= init/
 drivers-y	:= drivers/ sound/ firmware/
 net-y		:= net/
@@ -887,11 +888,13 @@ core-y		+= kernel/ mm/ fs/ ipc/ security/ crypto/ block/
 
 vmlinux-dirs	:= $(patsubst %/,%,$(filter %/, $(init-y) $(init-m) \
 		     $(core-y) $(core-m) $(drivers-y) $(drivers-m) \
+		     $(backports-y) $(backports-m) \
 		     $(net-y) $(net-m) $(libs-y) $(libs-m)))
 
 vmlinux-alldirs	:= $(sort $(vmlinux-dirs) $(patsubst %/,%,$(filter %/, \
-		     $(init-) $(core-) $(drivers-) $(net-) $(libs-))))
+		     $(init-) $(core-) $(drivers-) $(backports-) $(net-) $(libs-))))
 
+backports-y	:= $(patsubst %/, %/built-in.o, $(backports-y))
 init-y		:= $(patsubst %/, %/built-in.o, $(init-y))
 core-y		:= $(patsubst %/, %/built-in.o, $(core-y))
 drivers-y	:= $(patsubst %/, %/built-in.o, $(drivers-y))
@@ -902,7 +905,7 @@ libs-y		:= $(libs-y1) $(libs-y2)
 
 # Externally visible symbols (used by link-vmlinux.sh)
 export KBUILD_VMLINUX_INIT := $(head-y) $(init-y)
-export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y) $(drivers-y) $(net-y)
+export KBUILD_VMLINUX_MAIN := $(core-y) $(libs-y) $(drivers-y) $(net-y) $(backports-y)
 export KBUILD_LDS          := arch/$(SRCARCH)/kernel/vmlinux.lds
 export LDFLAGS_vmlinux
 # used by scripts/pacmage/Makefile

Re: newbie questions

[Johannes Berg]

On Mon, 2015-08-31 at 09:26 -0500, Pierre-Louis Bossart wrote:
> 
> What I was really looking for is the --integrate option that does 
> merge the backport into an existing tree.
> 

Ah. I'd suggest you not actually start with that though - it's more
complex and I'd think it's much easier to start getting all of this off
the ground as modules. Assuming the drivers you care about can be built
as modules, of course.

johannes
--
To unsubscribe from this list: send the line "unsubscribe backports" in