From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754147AbcHSJ1t (ORCPT ); Fri, 19 Aug 2016 05:27:49 -0400 Received: from mail-lf0-f68.google.com ([209.85.215.68]:34384 "EHLO mail-lf0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751701AbcHSJ1r (ORCPT ); Fri, 19 Aug 2016 05:27:47 -0400 From: Dmitry Monakhov To: "Theodore Ts'o" Cc: linux-kernel@vger.kernel.org Subject: Re: [PATCH 6/6] Add dockerfile In-Reply-To: <20160819052106.GC10888@thunk.org> References: <1471553651-9547-1-git-send-email-dmonakhov@openvz.org> <1471553651-9547-7-git-send-email-dmonakhov@openvz.org> <20160819052106.GC10888@thunk.org> User-Agent: Notmuch/0.18.1 (http://notmuchmail.org) Emacs/24.4.1 (x86_64-pc-linux-gnu) Date: Fri, 19 Aug 2016 12:27:43 +0300 Message-ID: <87eg5l6rcg.fsf@openvz.org> MIME-Version: 1.0 Content-Type: text/plain Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Theodore Ts'o writes: > On Fri, Aug 19, 2016 at 12:54:11AM +0400, Dmitry Monakhov wrote: >> Dockerfile is good way to create build environments >> Let's keep it as a reference build script. >> >> XXX: Currently Ted does not have hub.docker account >> so this docker file points to my hub. Should be updated. > > I guess I'm a little confused about how this Dockerfile works. First > of all, says "From Debian", so I guess the idea is to be based on an > unspecified version of Debian (why not debian:jessie?). But then it > has apt-get commands? How does that work? Yes, this is an official debian image https://hub.docker.com/_/debian . plain 'debian' means 'debian:jessie', but you can change it to 'debian:sid' This image already has apt-get (otherwise it will be not very useful) > > And if the idea is to fetch a prebuilt docker image, and we're running > it with docker run --privilege, I'm not sure what value using docker > run is really providing. We're not using the docker container > features. And typing > > docker run -i -t --privileged ... \ > "kvm-xfstests.sh --kernel /tmp/bzImage --update-files --update-xfstests-tar smoke" > > Is just *awkward*. The whole point is to let kvm-xfstests read the > kernel from your ~/.config/gce-xfstests, so your work flow can be > something like this: > Actually main idea is to have build environment for xfstests-bld development plus automatic builds gives us CI. This is important because xfstests-blk build is silently broken since May 30 due to 82c3a179 (fix proposed: https://github.com/dmonakhov/xfsprogs/commit/5dc7a0805d6b7a265863413a8e1b7acb191d5280) But this was uncovered because most people do not build xfstests-bld from scratch very often, we all too busy. Let's machine do that :) In fact other good candidate for CI is https://travis-ci.com/ Once we have Dockerfile which encapsulates all dependencies one can create xfstets-blk from scratch simply typing $ docker build -t my-xfstests-bld-image --file Dockerfile Or use precreated environment $ docker pull XXX/xfstests-bld Once image is created Log in to container and start hacking xfstest or e2fsprogs etc. $ docker run -i -t XXX/xfstests-bld $ emacs xfstest/tests/ext4/023 .... $ kvm-xfstests --update-xfstest ext4/023 ... Or run push your kernel patches to container via bindmount and then submit it to gce. $ docker run -v my-stuff:/my-stuff XXX/xfstests-bld \ git clone kernel.git && \ cd kernel && \ git am -s /my-stuff/patch.mbox && \ make O=/build/ext4 -j16 && \ cp /build/ext4/arch/x86/boot/bzImage /my-stuff && \ cp my-stuff/gce-config ~/.config/gce-config && \ gce-xfstests.sh --kernel /my-stuff/bzImage smoke Other useful feature is ability to speed xfstests-bld by splitting full test-set to groups and run kvm-xfstests instances in parallel. For example: docker run -v my-kernel:/my-kernel XXX/xfstests-bld \ kvm-xfstests.sh --kernel /my-kernel/bzImage -c 1k > 1k.log & docker run -v my-kernel:/my-kernel XXX/xfstests-bld \ kvm-xfstests.sh --kernel /my-kernel/bzImage -c 4k > 4k.log & docker run -v my-kernel:/my-kernel XXX/xfstests-bld \ kvm-xfstests.sh --kernel /my-kernel/bzImage -c dax > dax.log & ..... w/o sand box they will conflict due to net ports (localhost:7500 etc) In my case speed up is about x10 > git am -s ~/mbox/patch.mbox > ../make-ext4 # shortcut for something like "make O=/build/ext4 -j16" > kvm-xfstests smoke # ~/.config/gce-xfststs sets GCE_KERNEL=/build/ext4 > > needing to type a long docker run command doesn't seem to add any > value. > > Confused, > > - Ted > -- > To unsubscribe from this list: send the line "unsubscribe linux-ext4" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html