Re: [PATCH 6/6] Add dockerfile

Re: [PATCH 6/6] Add dockerfile

[Dmitry Monakhov]

Theodore Ts'o <tytso@mit.edu> writes:

> On Fri, Aug 19, 2016 at 12:54:11AM +0400, Dmitry Monakhov wrote:
>> Dockerfile is good way to create build environments
>> Let's keep it as a reference build script.
>> 
>> XXX: Currently Ted does not have hub.docker account
>>      so this docker file points to my hub. Should be updated.
>
> I guess I'm a little confused about how this Dockerfile works.  First
> of all, says "From Debian", so I guess the idea is to be based on an
> unspecified version of Debian (why not debian:jessie?).  But then it
> has apt-get commands?  How does that work?
Yes, this is an official debian image https://hub.docker.com/_/debian .
plain 'debian' means 'debian:jessie', but you can change it to 'debian:sid'
This image already has apt-get  (otherwise it will be not very useful)
>
> And if the idea is to fetch a prebuilt docker image, and we're running
> it with docker run --privilege, I'm not sure what value using docker
> run is really providing.  We're not using the docker container
> features.   And typing
>
> docker run -i -t --privileged ... \
> 	"kvm-xfstests.sh --kernel /tmp/bzImage --update-files --update-xfstests-tar  smoke"
>
> Is just *awkward*.  The whole point is to let kvm-xfstests read the
> kernel from your ~/.config/gce-xfstests, so your work flow can be
> something like this:
>
Actually main idea is to have build environment for xfstests-bld development
plus automatic builds gives us CI. This is important because
xfstests-blk build is silently broken since May 30 due to 82c3a179
(fix proposed: https://github.com/dmonakhov/xfsprogs/commit/5dc7a0805d6b7a265863413a8e1b7acb191d5280)
But this was uncovered because most people do not build xfstests-bld
from scratch very often, we all too busy. Let's machine do that :)
In fact other good candidate for CI is https://travis-ci.com/

Once we have Dockerfile which encapsulates all dependencies
one can create xfstets-blk from scratch simply typing
$ docker build -t my-xfstests-bld-image --file Dockerfile
Or use precreated environment
$ docker pull XXX/xfstests-bld

Once image is created 
Log in to container and start hacking xfstest or e2fsprogs etc.
$ docker run -i -t XXX/xfstests-bld
$ emacs xfstest/tests/ext4/023 ....
$ kvm-xfstests --update-xfstest ext4/023
...

Or run push your kernel patches to container via bindmount
and then submit it to gce.
$ docker run -v my-stuff:/my-stuff XXX/xfstests-bld \
                git clone kernel.git && \
                cd kernel && \
                git am -s /my-stuff/patch.mbox && \
                make O=/build/ext4 -j16 && \
                cp /build/ext4/arch/x86/boot/bzImage /my-stuff && \
                cp my-stuff/gce-config ~/.config/gce-config && \
                gce-xfstests.sh --kernel /my-stuff/bzImage smoke

Other useful feature is ability to speed xfstests-bld by splitting
full test-set to groups and run kvm-xfstests instances in parallel.
For example:
docker run -v my-kernel:/my-kernel XXX/xfstests-bld  \
       kvm-xfstests.sh --kernel /my-kernel/bzImage -c 1k > 1k.log &
docker run -v my-kernel:/my-kernel XXX/xfstests-bld  \
       kvm-xfstests.sh --kernel /my-kernel/bzImage -c 4k > 4k.log &
docker run -v my-kernel:/my-kernel XXX/xfstests-bld  \
       kvm-xfstests.sh --kernel /my-kernel/bzImage -c dax > dax.log &
.....
w/o sand box they will conflict due to net ports (localhost:7500 etc)
In my case speed up is about x10
> git am -s ~/mbox/patch.mbox
> ../make-ext4      # shortcut for something like "make O=/build/ext4 -j16"
> kvm-xfstests smoke   # ~/.config/gce-xfststs sets GCE_KERNEL=/build/ext4
>
> needing to type a long docker run command doesn't seem to add any
> value.
>
> Confused,
>
> 					- Ted
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 6/6] Add dockerfile

[Theodore Ts'o]

On Fri, Aug 19, 2016 at 12:27:43PM +0300, Dmitry Monakhov wrote:
> Actually main idea is to have build environment for xfstests-bld development
> plus automatic builds gives us CI. This is important because
> xfstests-blk build is silently broken since May 30 due to 82c3a179
> (fix proposed: https://github.com/dmonakhov/xfsprogs/commit/5dc7a0805d6b7a265863413a8e1b7acb191d5280)
> But this was uncovered because most people do not build xfstests-bld
> from scratch very often, we all too busy. Let's machine do that :)
> In fact other good candidate for CI is https://travis-ci.com/

Actually, I'm rebuilding all the time, but I'm not upgrading to the
latest bleeding edge version of xfsprogs all the time.  In fact, I'm
currently still using xfsprogs v4.5.0, since that's plenty for the
testing I'm doing.  I have historically pinned the commit Id's for
certain projects where newer versions were either (a) known to be
flaky, or (b) known to be broken in some way -- that's why we
currently have explicit commit ID's for fio and quota.  Perhaps I
should add one for xfsprogs as well, so that we only upgrade to newer
versions when it known to be needed and where it's helpful.

> Once we have Dockerfile which encapsulates all dependencies
> one can create xfstets-blk from scratch simply typing
> $ docker build -t my-xfstests-bld-image --file Dockerfile
> Or use precreated environment
> $ docker pull XXX/xfstests-bld

If you use the precreated environment that will have a fixed commit
ID.  So the advantage of doing continuous integration of the bleeding
edge version of xfsprogs only works if you are using the "docker
build" line, right.

> Once image is created 
> Log in to container and start hacking xfstest or e2fsprogs etc.
> $ docker run -i -t XXX/xfstests-bld
> $ emacs xfstest/tests/ext4/023 ....
> $ kvm-xfstests --update-xfstest ext4/023
> ...

I'm not sure I see the advantage of doing this in a container, I
guess.  I just do in my standard laptop environment today.

> Other useful feature is ability to speed xfstests-bld by splitting
> full test-set to groups and run kvm-xfstests instances in parallel.
> For example:
> docker run -v my-kernel:/my-kernel XXX/xfstests-bld  \
>        kvm-xfstests.sh --kernel /my-kernel/bzImage -c 1k > 1k.log &
> docker run -v my-kernel:/my-kernel XXX/xfstests-bld  \
>        kvm-xfstests.sh --kernel /my-kernel/bzImage -c 4k > 4k.log &
> docker run -v my-kernel:/my-kernel XXX/xfstests-bld  \
>        kvm-xfstests.sh --kernel /my-kernel/bzImage -c dax > dax.log &
> .....
> w/o sand box they will conflict due to net ports (localhost:7500 etc)

... and because you will need separate scratch disks for each of the
docker runs.

OK, I can see how that's certainly an advantage..  This is what I use
gce-xfstests for, myself.  :-)

The main thing I don't really like about the Dockerfile is that with
the exception of the last use case, everyone would really need to have
their own customized version of the Docker file, right?  And for
someone who isn't Docker aware, I'm concerned that it might be very
confusing to use, especially compared to all of the documentation I
currently have for kvm-xfstests and gce-xfstests.

Hmm, I'll have to think about this and play with it a bit more...

     	       	  	      	       	    - Ted

Re: [PATCH 6/6] Add dockerfile

[Dmitry Monakhov]

[-- Attachment #1: Type: text/plain, Size: 4976 bytes --]

Theodore Ts'o <tytso@mit.edu> writes:

> On Fri, Aug 19, 2016 at 12:27:43PM +0300, Dmitry Monakhov wrote:
>> Actually main idea is to have build environment for xfstests-bld development
>> plus automatic builds gives us CI. This is important because
>> xfstests-blk build is silently broken since May 30 due to 82c3a179
>> (fix proposed: https://github.com/dmonakhov/xfsprogs/commit/5dc7a0805d6b7a265863413a8e1b7acb191d5280)
>> But this was uncovered because most people do not build xfstests-bld
>> from scratch very often, we all too busy. Let's machine do that :)
>> In fact other good candidate for CI is https://travis-ci.com/
>
> Actually, I'm rebuilding all the time, but I'm not upgrading to the
> latest bleeding edge version of xfsprogs all the time.  In fact, I'm
> currently still using xfsprogs v4.5.0, since that's plenty for the
> testing I'm doing.  I have historically pinned the commit Id's for
> certain projects where newer versions were either (a) known to be
> flaky, or (b) known to be broken in some way -- that's why we
> currently have explicit commit ID's for fio and quota.  Perhaps I
> should add one for xfsprogs as well, so that we only upgrade to newer
> versions when it known to be needed and where it's helpful.
Yes. That is reasonable point.
My original idea was to check that any commit of xfstests-bld will
healthy with default config. And if github/docker build farm may do it
for us for free is it not bad.
>
>> Once we have Dockerfile which encapsulates all dependencies
>> one can create xfstets-blk from scratch simply typing
>> $ docker build -t my-xfstests-bld-image --file Dockerfile
>> Or use precreated environment
>> $ docker pull XXX/xfstests-bld
>
> If you use the precreated environment that will have a fixed commit
> ID.  So the advantage of doing continuous integration of the bleeding
> edge version of xfsprogs only works if you are using the "docker
> build" line, right.
>
>> Once image is created 
>> Log in to container and start hacking xfstest or e2fsprogs etc.
>> $ docker run -i -t XXX/xfstests-bld
>> $ emacs xfstest/tests/ext4/023 ....
>> $ kvm-xfstests --update-xfstest ext4/023
>> ...
>
> I'm not sure I see the advantage of doing this in a container, I
> guess.  I just do in my standard laptop environment today.
I can not because I laptop from famous thinkpad t430 series with
flaky SSD which starts to return EIO after intensive load.
https://forums.lenovo.com/t5/ThinkPad-T400-T500-and-newer-T/T430s-Intel-SSD-520-180GB-issue/td-p/888083

Also. What about RH/SUSE/Fedora or even Gentoo people? They are exit :)
And if we give them just chance to try debian w/o complexities
maybe they will become debian adepts.
>
>> Other useful feature is ability to speed xfstests-bld by splitting
>> full test-set to groups and run kvm-xfstests instances in parallel.
>> For example:
>> docker run -v my-kernel:/my-kernel XXX/xfstests-bld  \
>>        kvm-xfstests.sh --kernel /my-kernel/bzImage -c 1k > 1k.log &
>> docker run -v my-kernel:/my-kernel XXX/xfstests-bld  \
>>        kvm-xfstests.sh --kernel /my-kernel/bzImage -c 4k > 4k.log &
>> docker run -v my-kernel:/my-kernel XXX/xfstests-bld  \
>>        kvm-xfstests.sh --kernel /my-kernel/bzImage -c dax > dax.log &
>> .....
>> w/o sand box they will conflict due to net ports (localhost:7500 etc)
>
> ... and because you will need separate scratch disks for each of the
> docker runs.
Space is not a problem for most people. Average disk size is more that 500Gb
>
> OK, I can see how that's certainly an advantage..  This is what I use
> gce-xfstests for, myself.  :-)
BTW: I would be good thing to have some sort of job preparation mode.
where test does not executed, but only listed. So one can get lists
of tests and then divide it to groups and run in parallel:
$ kvm-xfstests.sh -c 4k -g auto --list > all_tests.txt
# Next line divides all_tests.txt in 8 parts and run them parallel
$ parallel --joblog job.log -j 8 -m -a all_tests.txt "kvm-xfstests.sh -c 4k"
I'll send you a patch.
BTW: Sometimes many parallel jobs provoke BUGON on host kernel due to
some bug in KVM  :)
>
> The main thing I don't really like about the Dockerfile is that with
> the exception of the last use case, everyone would really need to have
> their own customized version of the Docker file, right?  And for
> someone who isn't Docker aware, I'm concerned that it might be very
> confusing to use, especially compared to all of the documentation I
> currently have for kvm-xfstests and gce-xfstests.
Absolutely no. Dockerfile is just script which used for container
creation(similar gen-image). As normal script it should be as generic
as possible. If I've embedded some ad-hoc references to my-config this
is just a common error than every scripts writers do :(
>
> Hmm, I'll have to think about this and play with it a bit more...
>
>      	       	  	      	       	    - Ted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 472 bytes --]

Re: [PATCH 6/6] Add dockerfile

[Theodore Ts'o]

On Sat, Aug 20, 2016 at 02:31:26PM +0300, Dmitry Monakhov wrote:
> > I'm not sure I see the advantage of doing this in a container, I
> > guess.  I just do in my standard laptop environment today.
> I can not because I laptop from famous thinkpad t430 series with
> flaky SSD which starts to return EIO after intensive load.
> https://forums.lenovo.com/t5/ThinkPad-T400-T500-and-newer-T/T430s-Intel-SSD-520-180GB-issue/td-p/888083

Hmm, I never buy a Lenovo SSD; they're overpriced.  So I always get my
Thinkpads with a 500mb HDD, and then replace it with a Samsumg 840/850
EVO/PRO SSD.

> Also. What about RH/SUSE/Fedora or even Gentoo people? They are exit :)
> And if we give them just chance to try debian w/o complexities
> maybe they will become debian adepts.

You only need Debain to build the image.  If you are just using the
pre-built root_fs.img, you can just download it and go.

Speaking of which, that's one thing which I don't understand about
your Dockerfile.  In the build step you just download the prebuilt
root_fs.img from kernel.org.  There are comments in your Dockerfile which say:

# Fetch and build xfstests-bld
# In order to build root_image from scratch privileged operation are required,
# so it is impossible during build stage.
# One can make it like this:
# docker run -i -t --privileged --rm dmonakhov/xfstests-bld "cd kvm-xfstests/test-appliance && ./gen-image"
# During build stage we simply fetch image precreated by tytso@

... but while this will create a new root_fs.img file, as soon as
you're you're done the container will exit, and then the root_fs.img
file will be blown away, right?

The other thing I'll note here is that the Dockerfile is serving two
use cases.  One is to user Docker to do automated build testing.  The
other is for people who want to run multiple instances of kvm-xfstests
using the container services.

For the second, we don't really want to include all of the build
artifacts and build and source trees in Docker image that the users
will have to pull down over the network.  And using kvm-xfstests
--update-xfstests means a lot of extra wasted I/O, since we're
packaging up the tar file, then copying it into file, and then
unpacking it inside the guest VM, etc.

So for the second case, you'd really want to create a Docker image
which had the minimum packages needed to actually *run* the tests,
with the pre-installed root_fs image.

For the first case, all you need to do is to have the Dockerfile
create the xfstests.tar.gz file.  I suppose this could be used as a
poor man's build engine if you don't want to run the build on your
local machine.  What I do is my top-level config.custom has:

BUILD_ENV="schroot -c jessie64 --"
SUDO_ENV="schroot -c jessie64 -u root --"

That's my pristine build chroot, and so when I run ./do-all, what gets
executed is:

schroot -c jessie64 -- make clean
schroot -c jessie64 -- make
schroot -c jessie64 -- make tarball
cd kvm-xfstests/test-appliance
schroot -c jessie64 -u root -- ./gen-image

I'm pretty sure this is going to be faster than waiting for Docker to
build the image, and then for me to download the image and extract the
xfststs.tar.gz file.  (But then again, I have a fully working laptop
with functional SSD's not sourced by Lenovo :-).

     		      	  	     	    - Ted

Re: [PATCH 6/6] Add dockerfile

[Theodore Ts'o]

Ok, I've checked in a Dockerfile into the xfstests-bld repository, and
played with it some, and I have a couple of observations:

First of all, despite some work cleaning up the Dockerfile, the
resulting image is somewhere between 150% and 200% larger than what it
would be if we build root_fs.img outside of the Docker.  A bunch of
the wasted space is simply because we have to include a 47 MB
xfstests.tar.gz file which then has to get re-inserted into the
root_fs image via the --update-xfstests command line.

Secondly, as an automated build procedure, it's rather lacking in two
ways.  First, it doesn't do the gen-image step, and secondly it only
builds the 64-bit x86_64 binaries --- and I normally like to use the
32-bit i386 image, since that can be used to test 32-bit and 64-bit
binaries, and it also forces us to test the 32/64-bit ioctl compat
code.  So it's really only useful as a CI mechanism --- and in order
to use it I have to give Docker read/write access to my github
repositories.  I might consider creating a throwaway github repository
on bitbucket just for the CI effort, but it's not high on my priority
list, since it doesn't test the gen-image part of the build process.

Third, it's a bit more inconvenient to use than the comments in your
Dockerfile would imply.  The command:

docker run -i -t --privileged --rm dmonakhov/xfstests-bld \
   kvm-xfstests.sh --kernel /tmp/bzImage --update-files --update-xfstests-tar smoke

... won't work because the docker because the image won't have
/tmp/bzImage.  So you would need to add "-v /tmp/bzImage:/tmp/bzImage"
to the command line, making it even more unweildy.

BTW, with changes I've just commited, we can drop the --kernel since
we now default to ~/linux, and so the command would look like this:

docker run -i -t -rm -v /build/ext4:/root/linux --privileged \
    tytso/xfstests-bld kvm-xfstests --update-files --update-xfstests smoke

So if I am going to publish something to the Docker Hub, it would an
addition to my current release process, where I would build the
root_fs using my existing Debian build chroots, and just create a
minimal tytso/kvm-xfstests image which would just have the needed
files, and would probably end up weighing in around 200-225 MB.  The
user wouldn't need to specify the --update-files and --update-xfstests
flags, and it would start faster.

On the other hand, I'm not at all convinced this is actually a great
way to run kvm-xfstests; for one thing, the log file is trapped inside
the Docker container, and so you would need to manually extract it in
order to keep a history of past test runs.  (This is also the
challenge of just sharding the test runs; collating the test results
becomes a big pain.)  And the whole concept of running a VM inside a
docker container reminds me a bit of the "Hitler uses Docker" rant at:
https://youtu.be/PivpCKEiQOQ?t=2m27s

I'm also not wild about encouraging people to run random Docker images
they download over the network with docker run --privileged.  That's
right up there with running with scissors, encouraging people to give
Docker read/write access to their github accounts, using the same
password across hundreds of web sites, etc.....

If the real goal is to allow people to shard the tests so it can be
run across multiple VM's, it might be better to give kvm-xfstests some
options so a different set of disks are used and either a different
set of ports, or just network ports altogether.

Cheers,

					- Ted

Re: [PATCH 6/6] Add dockerfile

[Dmitry Monakhov]

[-- Attachment #1: Type: text/plain, Size: 4760 bytes --]

Theodore Ts'o <tytso@mit.edu> writes:

> On Sat, Aug 20, 2016 at 02:31:26PM +0300, Dmitry Monakhov wrote:
>> > I'm not sure I see the advantage of doing this in a container, I
>> > guess.  I just do in my standard laptop environment today.
>> I can not because I laptop from famous thinkpad t430 series with
>> flaky SSD which starts to return EIO after intensive load.
>> https://forums.lenovo.com/t5/ThinkPad-T400-T500-and-newer-T/T430s-Intel-SSD-520-180GB-issue/td-p/888083
>
> Hmm, I never buy a Lenovo SSD; they're overpriced.  So I always get my
> Thinkpads with a 500mb HDD, and then replace it with a Samsumg 840/850
> EVO/PRO SSD.
>
>> Also. What about RH/SUSE/Fedora or even Gentoo people? They are exit :)
>> And if we give them just chance to try debian w/o complexities
>> maybe they will become debian adepts.
>
> You only need Debain to build the image.  If you are just using the
> pre-built root_fs.img, you can just download it and go.
>
> Speaking of which, that's one thing which I don't understand about
> your Dockerfile.  In the build step you just download the prebuilt
> root_fs.img from kernel.org.  There are comments in your Dockerfile which say:
>
> # Fetch and build xfstests-bld
> # In order to build root_image from scratch privileged operation are required,
> # so it is impossible during build stage.
> # One can make it like this:
> # docker run -i -t --privileged --rm dmonakhov/xfstests-bld "cd kvm-xfstests/test-appliance && ./gen-image"
> # During build stage we simply fetch image precreated by tytso@
>
> ... but while this will create a new root_fs.img file, as soon as
> you're you're done the container will exit, and then the root_fs.img
> file will be blown away, right?
Yes. This is my typo. Generated image should be saved somewhere
before container stopped. So command should be
docker run -i -t --privileged -v ~/my-data:/shared --rm dmonakhov/xfstests-bld \
    "cd kvm-xfstests/test-appliance \
    && ./gen-image \
    && cp root_fs.img /shared/"
Probably it should be moved to dedicated script, see later.
>
> The other thing I'll note here is that the Dockerfile is serving two
> use cases.  One is to user Docker to do automated build testing.  The
> other is for people who want to run multiple instances of kvm-xfstests
> using the container services.
>
> For the second, we don't really want to include all of the build
> artifacts and build and source trees in Docker image that the users
> will have to pull down over the network.  And using kvm-xfstests
> --update-xfstests means a lot of extra wasted I/O, since we're
> packaging up the tar file, then copying it into file, and then
> unpacking it inside the guest VM, etc.
>
> So for the second case, you'd really want to create a Docker image
> which had the minimum packages needed to actually *run* the tests,
> with the pre-installed root_fs image.
Yes. That is reasonable idea. Actually I already have it in my queue.
Probably it it reasonable to create dedicated directory ./docker
and place all docker related crap there:
# build scripts
docker/Dockerfile.build-env
docker/Dockerfile.run-env
# And some run scripts
# Build all environment from scratch
docker/docker-build.sh
# Move config, kernel, root_fs.img inside container and run {gce,kvm}-xfstests.sh
docker/docker-kvm-xfstests.sh 
docker/docker-gce-xfstests.sh
> For the first case, all you need to do is to have the Dockerfile
> create the xfstests.tar.gz file.  I suppose this could be used as a
> poor man's build engine if you don't want to run the build on your
> local machine.  What I do is my top-level config.custom has:
>
> BUILD_ENV="schroot -c jessie64 --"
> SUDO_ENV="schroot -c jessie64 -u root --"
>
> That's my pristine build chroot, and so when I run ./do-all, what gets
> executed is:
>
> schroot -c jessie64 -- make clean
> schroot -c jessie64 -- make
> schroot -c jessie64 -- make tarball
> cd kvm-xfstests/test-appliance
> schroot -c jessie64 -u root -- ./gen-image
>
> I'm pretty sure this is going to be faster than waiting for Docker to
> build the image, and then for me to download the image and extract the
> xfststs.tar.gz file.  (But then again, I have a fully working laptop
> with functional SSD's not sourced by Lenovo :-).
Definitely. But in your case we still need jessie64's chroot.
It should be prepared somehow. My point is that if we give an automated build
script this may help people to start with xfstests with less pain.
In my experience time spend on investigation of installation process
always longer than time to download precreated VM/container :)
Let's call this approach "Lazy developers are welcome"
>      		      	  	     	    - Ted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 472 bytes --]

Re: [PATCH 6/6] Add dockerfile

[Theodore Ts'o]

On Sun, Aug 21, 2016 at 03:02:38PM +0300, Dmitry Monakhov wrote:
> Definitely. But in your case we still need jessie64's chroot.
> It should be prepared somehow. My point is that if we give an automated build
> script this may help people to start with xfstests with less pain.
> In my experience time spend on investigation of installation process
> always longer than time to download precreated VM/container :)
> Let's call this approach "Lazy developers are welcome"

Well, for the really lazy developers, all they need to do now is:

git clone git://git.kernel.org/pub/scm/fs/ext2/xfstests-bld.git
cd xfstests-bld/kvm-xfstests
make install

:-)

This will download the root_fs.img from www.kernel.org, and install
kvm-xfstests into their home directory.

(Or they can substite the last command with: "sudo make
prefix=/usr/local install")

This should work for both Fedora and Debian systems.  There will be
some packages you will need to install (e.g., kvm) but I suspect many
people will have all of the prequisites installed already.

Cheers,

       	    	     	    			  - Ted

Re: [PATCH 6/6] Add dockerfile

[Theodore Ts'o]

On Sun, Aug 21, 2016 at 03:02:38PM +0300, Dmitry Monakhov wrote:
> Probably it it reasonable to create dedicated directory ./docker
> and place all docker related crap there:
> # build scripts
> docker/Dockerfile.build-env
> docker/Dockerfile.run-env
> # And some run scripts
> # Build all environment from scratch
> docker/docker-build.sh
> # Move config, kernel, root_fs.img inside container and run {gce,kvm}-xfstests.sh
> docker/docker-kvm-xfstests.sh 
> docker/docker-gce-xfstests.sh

I thought about that, but my current thinking is to have one
Dockerfile which is in the top-level directory, and one which is in
the kvm-xfstests directory.

I've already created a tytso/xfststs-bld Docker image which was
generated using:

docker build -t tytso/xfstests-bld https://github.com/tytso/xfstests-bld.git

By putting the Dockerfile in the top-level directory, I can use the
command "COPY . /devel/xfstests-bld" in the Dockerfile, and so we can
get a clean build build that way.

By putting a Dockerfile in the kvm-xfstests directory, then all I have
to do is to put a "COPY . /usr/local/lib/kvm-xfstests" which will copy
all of the kvm-xfstests directory directly where we want the files in
the kvm-xfstests Docker image.  I'm assuming that my release process
will first build root_fs.i386 and root_fs.x86_64 using debian chroots,
and then upload them to www.kernel.org.  I'll then run:

docker build -t tytso/kvm-xfstests https://github.com/tytso/xfstests-bld.git:kvm-xfstests

... and then the Dockerfile will pull the root_fs.i386 from
ftp.kernel.org.

What do you think?

						- Ted

Re: [PATCH 6/6] Add dockerfile

[Dmitry Monakhov]

[-- Attachment #1: Type: text/plain, Size: 2082 bytes --]

Theodore Ts'o <tytso@mit.edu> writes:

> On Sun, Aug 21, 2016 at 03:02:38PM +0300, Dmitry Monakhov wrote:
>> Probably it it reasonable to create dedicated directory ./docker
>> and place all docker related crap there:
>> # build scripts
>> docker/Dockerfile.build-env
>> docker/Dockerfile.run-env
>> # And some run scripts
>> # Build all environment from scratch
>> docker/docker-build.sh
>> # Move config, kernel, root_fs.img inside container and run {gce,kvm}-xfstests.sh
>> docker/docker-kvm-xfstests.sh 
>> docker/docker-gce-xfstests.sh
>
> I thought about that, but my current thinking is to have one
> Dockerfile which is in the top-level directory, and one which is in
> the kvm-xfstests directory.
>
> I've already created a tytso/xfststs-bld Docker image which was
> generated using:
>
> docker build -t tytso/xfstests-bld https://github.com/tytso/xfstests-bld.git
Nice. Now image is no longer generated by Russian hacker. Hitler
finally can sleep well :)
>
> By putting the Dockerfile in the top-level directory, I can use the
> command "COPY . /devel/xfstests-bld" in the Dockerfile, and so we can
> get a clean build build that way.
Yes. That is reasonable. I do not know that build context can emerge from URL.
>
> By putting a Dockerfile in the kvm-xfstests directory, then all I have
> to do is to put a "COPY . /usr/local/lib/kvm-xfstests" which will copy
> all of the kvm-xfstests directory directly where we want the files in
> the kvm-xfstests Docker image.  I'm assuming that my release process
> will first build root_fs.i386 and root_fs.x86_64 using debian chroots,
> and then upload them to www.kernel.org.  I'll then run:
>
> docker build -t tytso/kvm-xfstests https://github.com/tytso/xfstests-bld.git:kvm-xfstests
Cool. The only minor comment is that command should be:
docker build -t tytso/kvm-xfstests -f kvm-xfstests/Dockerfile https://github.com/tytso/xfstests-bld.git:kvm-xfstests
>
> ... and then the Dockerfile will pull the root_fs.i386 from
> ftp.kernel.org.
>
> What do you think?
>
> 						- Ted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 472 bytes --]