How to verify linux-next

How to verify linux-next

[Pintu Kumar]

** Re sending **

Hi,

I have a general question.
How do we normally verify linux-next tree?

I wanted to work on linux-next but I am facing some issues.
I could able to build linux-next for both x86 and arm, but I could not
verify it on any machine.
Currently I don't have a real Linux PC to boot with linux-next kernel.
So I am trying to find alternative ways, like using the virtual box or qemu-arm.
1) For Oracle virtual box 5.1.26 with ubuntu-32 bit, the linux-next
kernel is not booting.
2) For qemu-arm (versatilepb), I am able to build the kernel, but I
could not figure out which rootfs to use with it.
I tried creating minimal rootfs using busybox, but it does not contain
enough interface. I am not able to open multiple terminal and also
could not setup ssh to access it using PUTTY.

So, if you know of any better rootfs to use with qemu-arm please let me know.
Or, if you know of any better option to use linux-next please tell me.
It will be really helpful.


Thank You!
Regards,
Pintu

On Fri, Sep 29, 2017 at 3:12 PM, Pintu Kumar <pintu.ping@gmail.com> wrote:
> Hi,
>
> I have a general question.
> How do we normally verify linux-next tree?
>
> I wanted to work on linux-next but I am facing some issues.
> I could able to build linux-next for both x86 and arm, but I could not
> verify it on any machine.
> Currently I don't have a real Linux PC to boot with linux-next kernel.
> So I am trying to find alternative ways, like using the virtual box or qemu-arm.
> 1) For Oracle virtual box 5.1.26 with ubuntu-32 bit, the linux-next
> kernel is not booting.
> 2) For qemu-arm (versatilepb), I am able to build the kernel, but I
> could not figure out which rootfs to use with it.
> I tried creating minimal rootfs using busybox, but it does not contain
> enough interface. I am not able to open multiple terminal and also
> could not setup ssh to access it using PUTTY.
>
> So, if you know of any better rootfs to use with qemu-arm please let me know.
> Or, if you know of any better option to use linux-next please tell me.
> It will be really helpful.
>
>
> Thank You!
> Regards,
> Pintu

How to verify linux-next

[Pintu Kumar]

Hi,

I have a general question.
How do we normally verify linux-next tree?

I wanted to work on linux-next but I am facing some issues.
I could able to build linux-next for both x86 and arm, but I could not
verify it on any machine.
Currently I don't have a real Linux PC to boot with linux-next kernel.
So I am trying to find alternative ways, like using the virtual box or qemu-arm.
1) For Oracle virtual box 5.1.26 with ubuntu-32 bit, the linux-next
kernel is not booting.
2) For qemu-arm (versatilepb), I am able to build the kernel, but I
could not figure out which rootfs to use with it.
I tried creating minimal rootfs using busybox, but it does not contain
enough interface. I am not able to open multiple terminal and also
could not setup ssh to access it using PUTTY.

So, if you know of any better rootfs to use with qemu-arm please let me know.

Or, if you know of any better option to use linux-next please tell me.
It will be really helpful.


Thank You!
Regards,
Pintu

How to verify linux-next

[valdis.kletnieks at vt.edu]

On Fri, 29 Sep 2017 16:08:07 +0530, Pintu Kumar said:

> I have a general question.
> How do we normally verify linux-next tree?

The same exact way you "verify" any other Linux kernel, for whatever
definition of "verify" you plan to use.

> 1) For Oracle virtual box 5.1.26 with ubuntu-32 bit, the linux-next
> kernel is not booting.

Does an Ubuntu kernel boot correctly under VirtualBox?  If not, fix
that issue first.  Also, "is not booting" isn't detailed enough for anybody
to make even a guess as to what's wrong.

Also, note that 5.1.28 is out.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 486 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20170929/73b70e79/attachment.bin 

How to verify linux-next

[Damian Tometzki]

Hello,

i can tell you ubuntu 16.04 works under virtualbox 5.1.28 with the
current linux-next kernel

My maschine:
Host: Windows 10
Guest: Ubuntu 16.04

Am Freitag, den 29.09.2017, 08:41 -0400 schrieb
valdis.kletnieks at vt.edu:
> On Fri, 29 Sep 2017 16:08:07 +0530, Pintu Kumar said:
> 
> > 
> > I have a general question.
> > How do we normally verify linux-next tree?
> The same exact way you "verify" any other Linux kernel, for whatever
> definition of "verify" you plan to use.
> 
> > 
> > 1) For Oracle virtual box 5.1.26 with ubuntu-32 bit, the linux-next
> > kernel is not booting.
> Does an Ubuntu kernel boot correctly under VirtualBox???If not, fix
> that issue first.??Also, "is not booting" isn't detailed enough for
> anybody
> to make even a guess as to what's wrong.
> 
> Also, note that 5.1.28 is out.

How to verify linux-next

[Damian Tometzki]

Hello,

Ubuntu 16.04 with current linux-next Kernel workson virtualbox 5.1.28

Host: Windows 10
Guest: Ubuntu 16.04

Best regards
Damian


Am Freitag, den 29.09.2017, 08:41 -0400 schrieb
valdis.kletnieks at vt.edu:
> On Fri, 29 Sep 2017 16:08:07 +0530, Pintu Kumar said:
> 
> > 
> > I have a general question.
> > How do we normally verify linux-next tree?
> The same exact way you "verify" any other Linux kernel, for whatever
> definition of "verify" you plan to use.
> 
> > 
> > 1) For Oracle virtual box 5.1.26 with ubuntu-32 bit, the linux-next
> > kernel is not booting.
> Does an Ubuntu kernel boot correctly under VirtualBox???If not, fix
> that issue first.??Also, "is not booting" isn't detailed enough for
> anybody
> to make even a guess as to what's wrong.
> 
> Also, note that 5.1.28 is out.

How to verify linux-next

[Pintu Kumar]

On Fri, Sep 29, 2017 at 6:44 PM, Damian Tometzki
<damian.tometzki@icloud.com> wrote:
> Hello,
>
> Ubuntu 16.04 with current linux-next Kernel workson virtualbox 5.1.28
>
> Host: Windows 10
> Guest: Ubuntu 16.04
>
> Best regards
> Damian
>
>
> Am Freitag, den 29.09.2017, 08:41 -0400 schrieb
> valdis.kletnieks at vt.edu:
>> On Fri, 29 Sep 2017 16:08:07 +0530, Pintu Kumar said:
>>
>> >
>> > I have a general question.
>> > How do we normally verify linux-next tree?
>> The same exact way you "verify" any other Linux kernel, for whatever
>> definition of "verify" you plan to use.
>>
>> >
>> > 1) For Oracle virtual box 5.1.26 with ubuntu-32 bit, the linux-next
>> > kernel is not booting.
>> Does an Ubuntu kernel boot correctly under VirtualBox?  If not, fix
>> that issue first.  Also, "is not booting" isn't detailed enough for
>> anybody
>> to make even a guess as to what's wrong.
>>
>> Also, note that 5.1.28 is out.

Ok, I just updated to 5.1.28. And my Ubuntu version is already 16.04.
Let me try again if it works.
Thanks all for your reply.

BTW, I am more interested in my another query about QEMU arm.
This will be much quicker and easy for me.
But the problem is I wanted to use multiple ssh shell on qemu.
Also I needed a pre-built rootfs image for qemu-arm, cortex-a9
versatilepb machine.
It should have networking and ssh built-in so that I can connect to it
using PUTTY client.
Note that I could able to build my own minimal busybox and boot qemu
using linux-next (non graphical mode).
But I could able to get only one shell.
I need to test some client/server problem, so I need multiple shell.
I am not able to configure net/ssh on this qemu system.

So, I have 2 things to ask:
1) If you have pointers on how to setup ssh/net connection on QEMU
with busybox, do let me know.
2) Let, please point me to a pre-built qemu-arm busy box image with
full features.


Thanks,
Pintu

How to verify linux-next

[valdis.kletnieks at vt.edu]

On Fri, 29 Sep 2017 19:56:41 +0530, Pintu Kumar said:

> 1) If you have pointers on how to setup ssh/net connection on QEMU
> with busybox, do let me know.

Busybox doesn't do that as far as I know, as it's intended as a single-user
/sbin/init replacement. You'll need a full-featured userspace with an actual
init daemon (sysvinit, systemd, etc) and an ssh daemon (openssh, or if you want
something smaller, dropbear - works well on embedded things like routers...)
And of course things like /bin/login and all the /etc files that needs in order to work...
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 486 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20170929/4d817175/attachment.bin 

How to verify linux-next

[Kamil Konieczny]

On 29.09.2017 17:45, valdis.kletnieks at vt.edu wrote:
> On Fri, 29 Sep 2017 19:56:41 +0530, Pintu Kumar said:
> 
>> 1) If you have pointers on how to setup ssh/net connection on QEMU
>> with busybox, do let me know.
> 
> Busybox doesn't do that as far as I know, as it's intended as a single-user
> /sbin/init replacement. You'll need a full-featured userspace with an actual
> init daemon (sysvinit, systemd, etc) and an ssh daemon (openssh, or if you want [...]

What about /usr/bin/ssh as init replacement ?

-- 
Best regards,
Kamil Konieczny
Samsung R&D Institute Poland

How to verify linux-next

[valdis.kletnieks at vt.edu]

On Mon, 02 Oct 2017 10:11:34 +0200, Kamil Konieczny said:

> What about /usr/bin/ssh as init replacement ?

Well, if you are OK with your system panicking right away. :)

(Hint - the init process needs to be something that can run as a daemon).

If you use /usr/sbin/sshd, that has a *slightly* better chance of working,
except then you will need a *lot* of custom coding in your initramfs to do all
the system setup usually done by init during boot - mounting file systems,
configuring network interfaces, etc etc etc.  Plus, sshd probably doesn't have
some of the code needed for a true init process, such as reaping child
processes it didn't spawn itself, cleanly shutting down/rebooting, and so on...

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 486 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20171002/5b25db4e/attachment.bin 

How to verify linux-next

[Pintu Kumar]

On Fri, Sep 29, 2017 at 7:56 PM, Pintu Kumar <pintu.ping@gmail.com> wrote:
> On Fri, Sep 29, 2017 at 6:44 PM, Damian Tometzki
> <damian.tometzki@icloud.com> wrote:
>> Hello,
>>
>> Ubuntu 16.04 with current linux-next Kernel workson virtualbox 5.1.28
>>
>> Host: Windows 10
>> Guest: Ubuntu 16.04
>>
>> Best regards
>> Damian
>>
>>
>> Am Freitag, den 29.09.2017, 08:41 -0400 schrieb
>> valdis.kletnieks at vt.edu:
>>> On Fri, 29 Sep 2017 16:08:07 +0530, Pintu Kumar said:
>>>
>>> >
>>> > I have a general question.
>>> > How do we normally verify linux-next tree?
>>> The same exact way you "verify" any other Linux kernel, for whatever
>>> definition of "verify" you plan to use.
>>>
>>> >
>>> > 1) For Oracle virtual box 5.1.26 with ubuntu-32 bit, the linux-next
>>> > kernel is not booting.
>>> Does an Ubuntu kernel boot correctly under VirtualBox?  If not, fix
>>> that issue first.  Also, "is not booting" isn't detailed enough for
>>> anybody
>>> to make even a guess as to what's wrong.
>>>
>>> Also, note that 5.1.28 is out.
>
> Ok, I just updated to 5.1.28. And my Ubuntu version is already 16.04.
> Let me try again if it works.
> Thanks all for your reply.
>

Now, with vbox 5.1.28, I am getting below build failure with linux-next.
Any quick pointers on this, if anybody faced similar issue.

In file included from ./arch/x86/include/asm/atomic.h:7:0,
                 from ./include/linux/atomic.h:4,
                 from ./include/linux/mm_types_task.h:12,
                 from ./include/linux/mm_types.h:4,
                 from arch/x86/kvm/irq.h:25,
                 from arch/x86/kvm/vmx.c:19:
arch/x86/kvm/vmx.c: In function ?__pi_post_block?:
./arch/x86/include/asm/cmpxchg.h:129:2: warning: ?__ret? is used
uninitialized in this function [-Wuninitialized]
  __ret;        \
  ^
./arch/x86/include/asm/cmpxchg.h:86:21: note: ?__ret? was declared here
  __typeof__(*(ptr)) __ret;     \
                     ^
./arch/x86/include/asm/cmpxchg.h:133:2: note: in expansion of macro
?__raw_cmpxchg?
  __raw_cmpxchg((ptr), (old), (new), (size), LOCK_PREFIX)
  ^
./arch/x86/include/asm/cmpxchg.h:148:2: note: in expansion of macro ?__cmpxchg?
  __cmpxchg(ptr, old, new, sizeof(*(ptr)))
  ^
arch/x86/kvm/vmx.c:11732:11: note: in expansion of macro ?cmpxchg?
  } while (cmpxchg(&pi_desc->control, old.control,
           ^
  CC      kernel/trace/trace_seq.o
  CC      kernel/trace/trace_stat.o
In function ?__pi_post_block?,
    inlined from ?pi_post_block? at arch/x86/kvm/vmx.c:11831:2,
    inlined from ?vmx_post_block? at arch/x86/kvm/vmx.c:11840:2:
./arch/x86/include/asm/cmpxchg.h:127:3: error: call to
?__cmpxchg_wrong_size? declared with attribute error: Bad argument
size for cmpxchg
   __cmpxchg_wrong_size();     \
   ^



> BTW, I am more interested in my another query about QEMU arm.
> This will be much quicker and easy for me.
> But the problem is I wanted to use multiple ssh shell on qemu.
> Also I needed a pre-built rootfs image for qemu-arm, cortex-a9
> versatilepb machine.
> It should have networking and ssh built-in so that I can connect to it
> using PUTTY client.
> Note that I could able to build my own minimal busybox and boot qemu
> using linux-next (non graphical mode).
> But I could able to get only one shell.
> I need to test some client/server problem, so I need multiple shell.
> I am not able to configure net/ssh on this qemu system.
>
> So, I have 2 things to ask:
> 1) If you have pointers on how to setup ssh/net connection on QEMU
> with busybox, do let me know.
> 2) Let, please point me to a pre-built qemu-arm busy box image with
> full features.
>
>
> Thanks,
> Pintu

How to verify linux-next

[Theodore Ts'o]

On Fri, Sep 29, 2017 at 07:56:41PM +0530, Pintu Kumar wrote:
> BTW, I am more interested in my another query about QEMU arm.
> This will be much quicker and easy for me.
> But the problem is I wanted to use multiple ssh shell on qemu.
> Also I needed a pre-built rootfs image for qemu-arm, cortex-a9
> versatilepb machine.

If you want to get more useful help, it might be interesting if you
were to specify exactly what kind of "verification" you are interested
in doing.  What sort of kernel testing are you interested in doing?
What part of the kernel are you interested in testing?  The fact that
you are trying to test both a Ubuntu x86 box as will as a virtual ARM
box makes it unclear what part of the kernel you are most intested in
testing.

In particular, why do you care about accessing the VM via ssh /
networking?  What sort of testing do you plan to do after manage to
get the kernel running?   And do you care what distribution you use?

I have a huge amount of test automation built for testing kernel file
systems.  This includes building root_fs images for x86 for use with
kvm[1], and arm chroots for use in testing Android systems[2].  There
is also a turn-key images for running tests using the Google Cloud
Platform[3], and even a Dockerfile[4] so people can run kernel tests
using a private Kubernetes cluster.

[1] https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-xfstests.md
[2] https://thunk.org/android-xfstests
[3] https://thunk.org/gce-xfstests
[4] https://github.com/tytso/xfstests-bld/blob/master/kvm-xfstests/Dockerfile

If you don't have a file-system centric view of the world, and want to
do more generalized kernel testing, the above might not be as
interesting to you, although some of the utilities in the xfstests-bld
git tree for setting up and building in build chroots, using
debootstrap to create root_fs.img files, scripts for manipulating
xUnit test result files (the XML format used by Jenkins), using 9p to
communicate between the host system running qemu/kvm and the test VM,
etc.

The point is that if you really want to get serious about kernel
testing, you should really think hard about test automation.  And in
that world, using networking often makes things harder, not easier.
For kvm-xfstests we just do our communications using the serial port,
which made it easy for us to adapt things for android-xfstests, where
we comunicate test runner script via "adb shell".   For gce-xfstests
things _do_ get a bit more complicated, where the test summary gets
e-mail'ed back to the developer, while the full set of test artifacts
are archived on Google Cloud Storage.  But one of the most powerful
things about my setup is vast majority of the test automation code
stays the same regardless of whether the kernel being tested is being
run in KVM, on a physical Android hardware, or in the Cloud using
GCE.

> 2) Let, please point me to a pre-built qemu-arm busy box image with
> full features.

Define "full features".  Busy box images are generally _not_ full
featured.  There is a reason why I use a minimal Debian system; a lot
of the tests I run require bash, and modern shell utilities, and
Python so I can have scripts which manipulate xUnit XML files.
Nevertheless, the complete x86 test VM is still only 87 megs, which is
still small enough that it doesn't cause me any problems.

On the other hand, since I find networking in the test VM to be
completely superfluous (and in fact, gets in the way, since a VM which
is on the corporate network can be a security problem, and may run
afoul of corporate I/T security policies --- and if you don't have
those kinds of security policies, you really should....).  So my
root_fs's general have no networking support whatsoever.  It keeps
$WORK's secops team *much* happier.  :-)

					- Ted

How to verify linux-next

[Pintu Kumar]

Thanks Mr. Tso for your reply.
Please find my reply inline.

On Sat, Sep 30, 2017 at 3:20 AM, Theodore Ts'o <tytso@mit.edu> wrote:
> On Fri, Sep 29, 2017 at 07:56:41PM +0530, Pintu Kumar wrote:
>> BTW, I am more interested in my another query about QEMU arm.
>> This will be much quicker and easy for me.
>> But the problem is I wanted to use multiple ssh shell on qemu.
>> Also I needed a pre-built rootfs image for qemu-arm, cortex-a9
>> versatilepb machine.
>
> If you want to get more useful help, it might be interesting if you
> were to specify exactly what kind of "verification" you are interested
> in doing.  What sort of kernel testing are you interested in doing?
> What part of the kernel are you interested in testing?  The fact that
> you are trying to test both a Ubuntu x86 box as will as a virtual ARM
> box makes it unclear what part of the kernel you are most intested in
> testing.
>
I need to submit a patch to mainline which should be verified against
linux-next tree with latest API.
My patch is already working with 4.10 LTS version but I need to upgrade.

> In particular, why do you care about accessing the VM via ssh /
> networking?  What sort of testing do you plan to do after manage to
> get the kernel running?   And do you care what distribution you use?
>
My patch is related to some test utility based on client/server model.
So, I need 2 terminal, one for server and one for client.
No, I really don't care about distribution, whichever works this way is
good for me. So I a trying both ways: Ububntu(x86) or qemu (arm).
The point is, I should be able to test my patch with linux-next.

> I have a huge amount of test automation built for testing kernel file
> systems.  This includes building root_fs images for x86 for use with
> kvm[1], and arm chroots for use in testing Android systems[2].  There
> is also a turn-key images for running tests using the Google Cloud
> Platform[3], and even a Dockerfile[4] so people can run kernel tests
> using a private Kubernetes cluster.
>
> [1] https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-xfstests.md
> [2] https://thunk.org/android-xfstests
> [3] https://thunk.org/gce-xfstests
> [4] https://github.com/tytso/xfstests-bld/blob/master/kvm-xfstests/Dockerfile
>
> If you don't have a file-system centric view of the world, and want to
> do more generalized kernel testing, the above might not be as
> interesting to you, although some of the utilities in the xfstests-bld
> git tree for setting up and building in build chroots, using
> debootstrap to create root_fs.img files, scripts for manipulating
> xUnit test result files (the XML format used by Jenkins), using 9p to
> communicate between the host system running qemu/kvm and the test VM,
> etc.
>
> The point is that if you really want to get serious about kernel
> testing, you should really think hard about test automation.  And in
> that world, using networking often makes things harder, not easier.
> For kvm-xfstests we just do our communications using the serial port,
> which made it easy for us to adapt things for android-xfstests, where
> we comunicate test runner script via "adb shell".   For gce-xfstests
> things _do_ get a bit more complicated, where the test summary gets
> e-mail'ed back to the developer, while the full set of test artifacts
> are archived on Google Cloud Storage.  But one of the most powerful
> things about my setup is vast majority of the test automation code
> stays the same regardless of whether the kernel being tested is being
> run in KVM, on a physical Android hardware, or in the Cloud using
> GCE.
>
>> 2) Let, please point me to a pre-built qemu-arm busy box image with
>> full features.
>
> Define "full features".  Busy box images are generally _not_ full
> featured.  There is a reason why I use a minimal Debian system; a lot
> of the tests I run require bash, and modern shell utilities, and
> Python so I can have scripts which manipulate xUnit XML files.
> Nevertheless, the complete x86 test VM is still only 87 megs, which is
> still small enough that it doesn't cause me any problems.
>
Qemu Busybox -> full feature for me means: I should be able to connect
Qemu with my PUTTY session, and open 2 terminal.
Moreover, I should be able to do scp to my qemu machine from my
ubuntu pc.

> On the other hand, since I find networking in the test VM to be
> completely superfluous (and in fact, gets in the way, since a VM which
> is on the corporate network can be a security problem, and may run
> afoul of corporate I/T security policies --- and if you don't have
> those kinds of security policies, you really should....).  So my
> root_fs's general have no networking support whatsoever.  It keeps
> $WORK's secops team *much* happier.  :-)
>

I am really sorry for the confusion.
Ok, lets talk one by one.
1) How to resolve linux-next build error with ubuntu virtual box 5.1.28

Any quick pointers on this will really help me to quickly verify my
patch and submit.

In file included from ./arch/x86/include/asm/atomic.h:7:0,
                 from ./include/linux/atomic.h:4,
                 from ./include/linux/mm_types_task.h:12,
                 from ./include/linux/mm_types.h:4,
                 from arch/x86/kvm/irq.h:25,
                 from arch/x86/kvm/vmx.c:19:
arch/x86/kvm/vmx.c: In function ?__pi_post_block?:
./arch/x86/include/asm/cmpxchg.h:129:2: warning: ?__ret? is used
uninitialized in this function [-Wuninitialized]
  __ret;        \
  ^
./arch/x86/include/asm/cmpxchg.h:86:21: note: ?__ret? was declared here
  __typeof__(*(ptr)) __ret;     \
                     ^
./arch/x86/include/asm/cmpxchg.h:133:2: note: in expansion of macro
?__raw_cmpxchg?
  __raw_cmpxchg((ptr), (old), (new), (size), LOCK_PREFIX)
  ^
./arch/x86/include/asm/cmpxchg.h:148:2: note: in expansion of macro ?__cmpxchg?
  __cmpxchg(ptr, old, new, sizeof(*(ptr)))
  ^
arch/x86/kvm/vmx.c:11732:11: note: in expansion of macro ?cmpxchg?
  } while (cmpxchg(&pi_desc->control, old.control,
           ^
  CC      kernel/trace/trace_seq.o
  CC      kernel/trace/trace_stat.o
In function ?__pi_post_block?,
    inlined from ?pi_post_block? at arch/x86/kvm/vmx.c:11831:2,
    inlined from ?vmx_post_block? at arch/x86/kvm/vmx.c:11840:2:
./arch/x86/include/asm/cmpxchg.h:127:3: error: call to
?__cmpxchg_wrong_size? declared with attribute error: Bad argument
size for cmpxchg
   __cmpxchg_wrong_size();     \




>                                         - Ted

How to verify linux-next

[Theodore Ts'o]

On Sat, Sep 30, 2017 at 09:28:09AM +0530, Pintu Kumar wrote:
> I need to submit a patch to mainline which should be verified against
> linux-next tree with latest API.

If you want to verify a patch that you intend to submit upstream, my
suggestion is to *not* use linux-next, but rather use the latest
tagged -rc from Linus's tree.  So for example, you might want to use
v4.14-rc2 as your base, and then apply your patch on top of v4.14-rc2.
And then test v4.14-rc2.  That way you don't need to worry about
debugging problems that might be caused by code in other people's
development trees.

If you know which subsystem tree your commit is going to be sent to,
you might use as your base the current development branch of that
subsystem tree.  But in general, it's fine to use something like
v4.14-rc2; if the subsystem maintainer you plan to be submitting your
patch has other preference, he or she will let you know, or take care
of rebasing your patch onto his subsystme tree.

> My patch is related to some test utility based on client/server model.
> So, I need 2 terminal, one for server and one for client.

That implies you're running the commands to run the test by hand.  In
the ideal world, tests should be automated, even those that are using
client/server so that tests can be run unattended, over and over
again.

For example, here's an example of test involving a client and a server
in xfstests:

https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/tree/tests/generic/131

See?  No terminal required, and certainly not two terminals!

Remember, it's important not just to run one test, because the risk is
that fixing one bug might cause a test regression somewhere else.  So
when I "validate" a kernel, I'm running thousands of tests, just to
test the ext4 file system.  For each bug that we fix, we try to add a
new automated test, so we can be sure that some future change doesn't
cause a bug to reappear.  And if you're running hundreds or thousands
of tests, you certainly aren't going to be wanting to manually set up
each test by using putty to login to the VM using ssh!

> 1) How to resolve linux-next build error with ubuntu virtual box 5.1.28

Virtual box is not relevant.  What is relevant is the kernel config
file you are using, and what compiler version / distro are you using
to build the kernel.  And as I said, you're better off using something
like v4.14-rc2 instead of linux-next.

					- Ted

How to verify linux-next

[Pintu Kumar]

On Sat, Sep 30, 2017 at 10:55 AM, Theodore Ts'o <tytso@mit.edu> wrote:
> On Sat, Sep 30, 2017 at 09:28:09AM +0530, Pintu Kumar wrote:
>> I need to submit a patch to mainline which should be verified against
>> linux-next tree with latest API.
>
> If you want to verify a patch that you intend to submit upstream, my
> suggestion is to *not* use linux-next, but rather use the latest
> tagged -rc from Linus's tree.  So for example, you might want to use
> v4.14-rc2 as your base, and then apply your patch on top of v4.14-rc2.
> And then test v4.14-rc2.  That way you don't need to worry about
> debugging problems that might be caused by code in other people's
> development trees.
>
> If you know which subsystem tree your commit is going to be sent to,
> you might use as your base the current development branch of that
> subsystem tree.  But in general, it's fine to use something like
> v4.14-rc2; if the subsystem maintainer you plan to be submitting your
> patch has other preference, he or she will let you know, or take care
> of rebasing your patch onto his subsystme tree.
>
>> My patch is related to some test utility based on client/server model.
>> So, I need 2 terminal, one for server and one for client.
>
> That implies you're running the commands to run the test by hand.  In
> the ideal world, tests should be automated, even those that are using
> client/server so that tests can be run unattended, over and over
> again.
>
> For example, here's an example of test involving a client and a server
> in xfstests:
>
> https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/tree/tests/generic/131
>
> See?  No terminal required, and certainly not two terminals!
>
> Remember, it's important not just to run one test, because the risk is
> that fixing one bug might cause a test regression somewhere else.  So
> when I "validate" a kernel, I'm running thousands of tests, just to
> test the ext4 file system.  For each bug that we fix, we try to add a
> new automated test, so we can be sure that some future change doesn't
> cause a bug to reappear.  And if you're running hundreds or thousands
> of tests, you certainly aren't going to be wanting to manually set up
> each test by using putty to login to the VM using ssh!
>
>> 1) How to resolve linux-next build error with ubuntu virtual box 5.1.28
>
> Virtual box is not relevant.  What is relevant is the kernel config
> file you are using, and what compiler version / distro are you using
> to build the kernel.  And as I said, you're better off using something
> like v4.14-rc2 instead of linux-next.
>

Ok thank you so much for your reply.
Now I am able to boot with v4.14-rc2. But now I am facing another problem.
Now, I am not able to connect to internet from virtual box.
When I switch back to the default 4.10 the internet works normally.
I think the dlclient stopped working.
I am getting continuous logs related to apparmor like this:
apparmor="DENIED" comm=dhclient
apparmor="DENIED" comm=cups-browsed

With 4.10, I tried installing apparmor-utils and then reboot with
4.14-rc2, but it did not help.
Any suggestions on this?



>                                         - Ted

How to verify linux-next

[Damian Tometzki]

Am Sonntag, den 01.10.2017, 21:58 +0530 schrieb Pintu Kumar:
> On Sat, Sep 30, 2017 at 10:55 AM, Theodore Ts'o <tytso@mit.edu>
> wrote:
> > 
> > On Sat, Sep 30, 2017 at 09:28:09AM +0530, Pintu Kumar wrote:
> > > 
> > > I need to submit a patch to mainline which should be verified
> > > against
> > > linux-next tree with latest API.
> > If you want to verify a patch that you intend to submit upstream,
> > my
> > suggestion is to *not* use linux-next, but rather use the latest
> > tagged -rc from Linus's tree.??So for example, you might want to
> > use
> > v4.14-rc2 as your base, and then apply your patch on top of v4.14-
> > rc2.
> > And then test v4.14-rc2.??That way you don't need to worry about
> > debugging problems that might be caused by code in other people's
> > development trees.
> > 
> > If you know which subsystem tree your commit is going to be sent
> > to,
> > you might use as your base the current development branch of that
> > subsystem tree.??But in general, it's fine to use something like
> > v4.14-rc2; if the subsystem maintainer you plan to be submitting
> > your
> > patch has other preference, he or she will let you know, or take
> > care
> > of rebasing your patch onto his subsystme tree.
> > 
> > > 
> > > My patch is related to some test utility based on client/server
> > > model.
> > > So, I need 2 terminal, one for server and one for client.
> > That implies you're running the commands to run the test by
> > hand.??In
> > the ideal world, tests should be automated, even those that are
> > using
> > client/server so that tests can be run unattended, over and over
> > again.
> > 
> > For example, here's an example of test involving a client and a
> > server
> > in xfstests:
> > 
> > https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/tree/tests/g
> > eneric/131
> > 
> > See???No terminal required, and certainly not two terminals!
> > 
> > Remember, it's important not just to run one test, because the risk
> > is
> > that fixing one bug might cause a test regression somewhere
> > else.??So
> > when I "validate" a kernel, I'm running thousands of tests, just to
> > test the ext4 file system.??For each bug that we fix, we try to add
> > a
> > new automated test, so we can be sure that some future change
> > doesn't
> > cause a bug to reappear.??And if you're running hundreds or
> > thousands
> > of tests, you certainly aren't going to be wanting to manually set
> > up
> > each test by using putty to login to the VM using ssh!
> > 
> > > 
> > > 1) How to resolve linux-next build error with ubuntu virtual box
> > > 5.1.28
> > Virtual box is not relevant.??What is relevant is the kernel config
> > file you are using, and what compiler version / distro are you
> > using
> > to build the kernel.??And as I said, you're better off using
> > something
> > like v4.14-rc2 instead of linux-next.
> > 
> Ok thank you so much for your reply.
> Now I am able to boot with v4.14-rc2. But now I am facing another
> problem.
> Now, I am not able to connect to internet from virtual box.
> When I switch back to the default 4.10 the internet works normally.
> I think the dlclient stopped working.
> I am getting continuous logs related to apparmor like this:
> apparmor="DENIED" comm=dhclient
> apparmor="DENIED" comm=cups-browsed
> 
> With 4.10, I tried installing apparmor-utils and then reboot with
> 4.14-rc2, but it did not help.
> Any suggestions on this?

Hello,

i resolved the issue with:
sudo /etc/init.d/apparmor stop

Damian

> 
> 
> > 
> > ????????????????????????????????????????- Ted

How to verify linux-next

[Randy Dunlap]

On 10/01/17 09:44, Damian Tometzki wrote:
> 
> Am Sonntag, den 01.10.2017, 21:58 +0530 schrieb Pintu Kumar:
>> On Sat, Sep 30, 2017 at 10:55 AM, Theodore Ts'o <tytso@mit.edu>
>> wrote:
>>>
>>> On Sat, Sep 30, 2017 at 09:28:09AM +0530, Pintu Kumar wrote:
>>>>
>>>> I need to submit a patch to mainline which should be verified
>>>> against
>>>> linux-next tree with latest API.
>>> If you want to verify a patch that you intend to submit upstream,
>>> my
>>> suggestion is to *not* use linux-next, but rather use the latest
>>> tagged -rc from Linus's tree.??So for example, you might want to
>>> use
>>> v4.14-rc2 as your base, and then apply your patch on top of v4.14-
>>> rc2.
>>> And then test v4.14-rc2.??That way you don't need to worry about
>>> debugging problems that might be caused by code in other people's
>>> development trees.
>>>
>>> If you know which subsystem tree your commit is going to be sent
>>> to,
>>> you might use as your base the current development branch of that
>>> subsystem tree.??But in general, it's fine to use something like
>>> v4.14-rc2; if the subsystem maintainer you plan to be submitting
>>> your
>>> patch has other preference, he or she will let you know, or take
>>> care
>>> of rebasing your patch onto his subsystme tree.
>>>
>>>>
>>>> My patch is related to some test utility based on client/server
>>>> model.
>>>> So, I need 2 terminal, one for server and one for client.
>>> That implies you're running the commands to run the test by
>>> hand.??In
>>> the ideal world, tests should be automated, even those that are
>>> using
>>> client/server so that tests can be run unattended, over and over
>>> again.
>>>
>>> For example, here's an example of test involving a client and a
>>> server
>>> in xfstests:
>>>
>>> https://git.kernel.org/pub/scm/fs/xfs/xfstests-dev.git/tree/tests/g
>>> eneric/131
>>>
>>> See???No terminal required, and certainly not two terminals!
>>>
>>> Remember, it's important not just to run one test, because the risk
>>> is
>>> that fixing one bug might cause a test regression somewhere
>>> else.??So
>>> when I "validate" a kernel, I'm running thousands of tests, just to
>>> test the ext4 file system.??For each bug that we fix, we try to add
>>> a
>>> new automated test, so we can be sure that some future change
>>> doesn't
>>> cause a bug to reappear.??And if you're running hundreds or
>>> thousands
>>> of tests, you certainly aren't going to be wanting to manually set
>>> up
>>> each test by using putty to login to the VM using ssh!
>>>
>>>>
>>>> 1) How to resolve linux-next build error with ubuntu virtual box
>>>> 5.1.28
>>> Virtual box is not relevant.??What is relevant is the kernel config
>>> file you are using, and what compiler version / distro are you
>>> using
>>> to build the kernel.??And as I said, you're better off using
>>> something
>>> like v4.14-rc2 instead of linux-next.
>>>
>> Ok thank you so much for your reply.
>> Now I am able to boot with v4.14-rc2. But now I am facing another
>> problem.
>> Now, I am not able to connect to internet from virtual box.
>> When I switch back to the default 4.10 the internet works normally.
>> I think the dlclient stopped working.
>> I am getting continuous logs related to apparmor like this:
>> apparmor="DENIED" comm=dhclient
>> apparmor="DENIED" comm=cups-browsed
>>
>> With 4.10, I tried installing apparmor-utils and then reboot with
>> 4.14-rc2, but it did not help.
>> Any suggestions on this?
> 
> Hello,
> 
> i resolved the issue with:
> sudo /etc/init.d/apparmor stop

or boot with: apparmor=0


-- 
~Randy

How to verify linux-next

[Mike Galbraith]

On Sun, 2017-10-01 at 09:48 -0700, Randy Dunlap wrote:
> On 10/01/17 09:44, Damian Tometzki wrote:
> 
> > i resolved the issue with:
> > sudo /etc/init.d/apparmor stop
> 
> or boot with: apparmor=0

or systemctl mask apparmor

How to verify linux-next

[Pintu Kumar]

On Sun, Oct 1, 2017 at 10:33 PM, Mike Galbraith <efault@gmx.de> wrote:
> On Sun, 2017-10-01 at 09:48 -0700, Randy Dunlap wrote:
>> On 10/01/17 09:44, Damian Tometzki wrote:
>>
>> > i resolved the issue with:
>> > sudo /etc/init.d/apparmor stop
>>
>> or boot with: apparmor=0
>
> or systemctl mask apparmor

Ok, thanks everyone for your quick help.
Stopping apparmor really helped to restore the connection.

Mr. Tso, thanks for your help too for QEMU.
After finishing my patch next I will looking into QEMU issues as per
your suggestions.

How to verify linux-next

[Theodore Ts'o]

On Sun, Oct 01, 2017 at 09:58:37PM +0530, Pintu Kumar wrote:
> Ok thank you so much for your reply.
> Now I am able to boot with v4.14-rc2. But now I am facing another problem.
> Now, I am not able to connect to internet from virtual box.
> When I switch back to the default 4.10 the internet works normally.
> I think the dlclient stopped working.
> I am getting continuous logs related to apparmor like this:
> apparmor="DENIED" comm=dhclient
> apparmor="DENIED" comm=cups-browsed

How about compiling the kernel without apparmor?

Here are some very minimal kernel configs that are designed to work
with qemu and Google Compute Engine (if you use the 64-bit configs):

https://github.com/tytso/xfstests-bld/tree/master/kernel-configs

Grab the latest 4.9 kernel configs (e.g., [1]); it will work with
4.14-rc2, copy it into your kernel build tree as .config, and then run
"make olddefconfig".  That's why I use with qemu (32-bit and 64-bit)
and Google Compute Engine using a minimal debian chroot, and it works
Just Fine.

[1] https://github.com/tytso/xfstests-bld/blob/master/kernel-configs/x86_64-config-4.9

If you want an example of how to build a automatically generated test
appliance image (which is repeatably built, and can be used as part of
a discplined release engineering process), please see:

https://github.com/tytso/xfstests-bld/blob/master/kvm-xfstests/test-appliance/gen-image

Cheers,

					- Ted

P.S.  And here is an example of how a completely automated kernel
testing report might look like.  I ran a single command, and a few
hours later, the following lands in my e-mail's inbox.  Note the lack
of using putty to manually configure a single test, and not a single
terminal was opened during the entire test run.  :-)

TESTRUNID: ltm-20171001045435
KERNEL:    kernel 4.14.0-rc2-ext4-00005-gcdfa281a30f5 #513 SMP Sun Oct 1 00:52:09 EDT 2017 x86_64
CMDLINE:   --kernel gs://gce-xfstests/bzImage full
CPUS:      2
MEM:       7680

ext4/4k 337 tests, 23 skipped, 6 errors, 10569 seconds
   generic/233 generic/361 generic/388 generic/451 generic/456 generic/459 
ext4/1k 337 tests, 35 skipped, 8 errors, 12440 seconds
   generic/018 generic/232 generic/273 generic/361 generic/388 generic/451 
   generic/456 generic/459 
ext4/ext3 336 tests, 73 skipped, 6 errors, 10639 seconds
   generic/232 generic/361 generic/382 generic/388 generic/451 generic/459 
ext4/encrypt 327 tests, 97 skipped, 8 errors, 6785 seconds
   ext4/022 ext4/028 generic/081 generic/361 generic/382 generic/388 
   generic/459 shared/298 
ext4/nojournal 334 tests, 60 skipped, 6 errors, 6107 seconds
   ext4/301 generic/113 generic/232 generic/441 generic/451 generic/459 
ext4/ext3conv 336 tests, 23 skipped, 4 errors, 10039 seconds
   generic/361 generic/388 generic/451 generic/459 
ext4/adv 335 tests, 28 skipped, 8 errors, 8732 seconds
   generic/232 generic/233 generic/361 generic/388 generic/399 generic/451 
   generic/456 generic/459 
ext4/dioread_nolock 336 tests, 23 skipped, 5 errors, 7434 seconds
   generic/081 generic/233 generic/388 generic/451 generic/459 
ext4/data_journal 336 tests, 33 skipped, 5 errors, 13027 seconds
   generic/361 generic/371 generic/388 generic/441 generic/459 
ext4/bigalloc_1k 317 tests, 44 skipped, 10 errors, 8506 seconds
   generic/204 generic/235 generic/269 generic/273 generic/361 generic/388 
   generic/422 generic/451 generic/456 generic/459 

FSTESTIMG: gce-xfstests/xfstests-201709302211
FSTESTPRJ: gce-xfstests
FSTESTVER: e2fsprogs v1.43.6-85-g7595699d0 (Wed, 6 Sep 2017 22:04:14 -0400)
FSTESTVER: fio  fio-2.21 (Thu, 15 Jun 2017 12:25:03 -0600)
FSTESTVER: quota  16f31b1 (Tue, 5 Sep 2017 16:53:11 +0200)
FSTESTVER: stress-ng 977ae35 (Wed, 6 Sep 2017 23:45:03 -0400)
FSTESTVER: xfsprogs v4.12.0 (Thu, 20 Jul 2017 10:57:14 -0500)
FSTESTVER: xfstests-bld 0d85f98 (Sat, 30 Sep 2017 21:42:59 -0400)
FSTESTVER: xfstests linux-v3.8-1693-ga71d59bc (Fri, 29 Sep 2017 23:56:42 -0400)
FSTESTSET: -g auto
FSTESTOPT: aex