* Re: [adhoc xen-unstable bisection] complete test-amd64-i386-xl
[not found] <E1ZYQs7-0006Wu-Vx@osstest.xs.citrite.net>
@ 2015-09-07 8:53 ` Ian Campbell
0 siblings, 0 replies; only message in thread
From: Ian Campbell @ 2015-09-07 8:53 UTC (permalink / raw)
To: ian.jackson, Wei Liu, Jan Beulich; +Cc: Andrew Cooper, xen-devel
This is the result of the bisection of the migration issue with 32-bit on
Linux 4.1 onwards.
This is just for info as I believe the issue is already under discussion
etc elsewhere.
I've copied the final graph to
http://xenbits.xen.org/people/ianc/tmp/adhoc/test-amd64-i386-xl..html
Ian.
On Sun, 2015-09-06 at 04:44 +0100, Platform Team regression test user wrote:
> branch xen-unstable
> xen branch xen-unstable
> job test-amd64-i386-xl
> test guest-saverestore
>
> Tree: linux git://xenbits.xen.org/linux-pvops.git
> Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
> Tree: qemu git://xenbits.xen.org/staging/qemu-xen-unstable.git
> Tree: qemuu git://xenbits.xen.org/staging/qemu-upstream-unstable.git
> Tree: xen git://xenbits.xen.org/xen.git
[...snip first pass blaming a merge changeset which I aborted but was still reported here...]
> *** Found and reproduced problem changeset ***
>
> Bug is in tree: linux git://xenbits.xen.org/linux-pvops.git
> Bug introduced: 132978b94e66f8ad7d20790f8332f0e9c1426029
> Bug not present: fbe1bf140671619508dfa575d74a185ae53c5dbb
>
>
> commit 132978b94e66f8ad7d20790f8332f0e9c1426029
> Author: Jan Beulich <JBeulich@suse.com>
> Date: Fri Dec 19 16:10:54 2014 +0000
>
> x86: Fix step size adjustment during initial memory mapping
>
> The old scheme can lead to failure in certain cases - the
> problem is that after bumping step_size the next (non-final)
> iteration is only guaranteed to make available a memory block
> the size of what step_size was before. E.g. for a memory block
> [0,3004600000) we'd have:
>
> iter> > start> > > end> > > step> > > amount
> 1> > 3004400000> > 30045fffff> > 2M> > > 2M
> 2> > 3004000000> > 30043fffff> > 64M> > > 4M
> 3> > 3000000000> > 3003ffffff> > 2G> > > 64M
> 4> > 2000000000> > 2fffffffff> > 64G> > > 64G
>
> Yet to map 64G with 4k pages (as happens e.g. under PV Xen) we
> need slightly over 128M, but the first three iterations made
> only about 70M available.
>
> The condition (new_mapped_ram_size > mapped_ram_size) for
> bumping step_size is just not suitable. Instead we want to bump
> it when we know we have enough memory available to cover a block
> of the new step_size. And rather than making that condition more
> complicated than needed, simply adjust step_size by the largest
> possible factor we know we can cover at that point - which is
> shifting it left by one less than the difference between page
> table level shifts. (Interestingly the original STEP_SIZE_SHIFT
> definition had a comment hinting at that having been the
> intention, just that it should have been PUD_SHIFT-PMD_SHIFT-1
> instead of (PUD_SHIFT-PMD_SHIFT)/2, and of course for non-PAE
> 32-bit we can't really use these two constants as they're equal
> there.)
>
> Furthermore the comment in get_new_step_size() didn't get
> updated when the bottom-down mapping logic got added. Yet while
> an overflow (flushing step_size to zero) of the shift doesn't
> matter for the top-down method, it does for bottom-up because
> round_up(x, 0) = 0, and an upper range boundary of zero can't
> really work well.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> Acked-by: Yinghai Lu <yinghai@kernel.org>
> Link: http://lkml.kernel.org/r/54945C1E020000780005114E@mail.emea.novell.com
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
>
>
> For bisection revision-tuple graph see:
> http://osstest.xs.citrite.net/~osstest/testlogs/results-adhoc/bisect/xen-unstable/test-amd64-i386-xl..html
> Revision IDs in each graph node refer, respectively, to the Trees above.
>
> ----------------------------------------
> Searching for failure / basis pass:
> 37864 fail [host=rice-weevil] / 37863 ok.
> Failure / basis pass flights: 37864 / 37863
> (tree with no url: ovmf)
> (tree with no url: seabios)
> Tree: linux git://xenbits.xen.org/linux-pvops.git
> Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
> Tree: qemu git://xenbits.xen.org/staging/qemu-xen-unstable.git
> Tree: qemuu git://xenbits.xen.org/staging/qemu-upstream-unstable.git
> Tree: xen git://xenbits.xen.org/xen.git
> Latest eaa27f34e91a14cdceed26ed6c6793ec1d186115 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> Basis pass 97bf6af1f928216fd6c5a66e8a57bfa95a659672 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> Generating revisions with ./adhoc-revtuple-generator git://xenbits.xen.org/linux-pvops.git#97bf6af1f928216fd6c5a66e8a57bfa95a659672-eaa27f34e91a14cdceed26ed6c6793ec1d186115 git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860 git://xenbits.xen.org/staging/qemu-xen-unstable.git#7f057440b31da38196e3398fd1b618fc36ad97d6-7f057440b31da38196e3398fd1b618fc36ad97d6 git://xenbits.xen.org/staging/qemu-upstream-unstable.git#bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa-bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa git://xenbits.xen.org/xen.git#145a8004a7d659668d5a3b0ad9868d7678b24822-145a8004a7d659668d5a3b0ad9868d7678b24822
> + exec
> + sh -xe
> + cd /export/home/osstest/repos/linux-pvops
> + git remote set-url origin git://drall.uk.xensource.com:9419/git://xenbits.xen.org/linux-pvops.git
> + git fetch -p origin +refs/heads/*:refs/remotes/origin/*
> + exec
> + sh -xe
> + cd /export/home/osstest/repos/linux-pvops
> + git remote set-url origin git://drall.uk.xensource.com:9419/git://xenbits.xen.org/linux-pvops.git
> + git fetch -p origin +refs/heads/*:refs/remotes/origin/*
> Loaded 1103 nodes in revision graph
> Searching for test results:
> 34279 [host=worm-moth]
> 37011 [host=bush-cricket]
> 37043 [host=grain-weevil]
> 37077 [host=scape-moth]
> 37289 [host=lace-bug]
> 37292 [host=bush-cricket]
> 37295 [host=scape-moth]
> 37298 pass irrelevant
> 37326 [host=moss-bug]
> 37331 [host=lace-bug]
> 37332 [host=itch-mite]
> 37377 [host=leaf-beetle]
> 37415 [host=grain-weevil]
> 37473 [host=gall-mite]
> 37484 [host=potato-beetle]
> 37536 []
> 37546 [host=lace-bug]
> 37552 [host=scape-moth]
> 37589 pass irrelevant
> 37719 [host=grain-weevil]
> 37711 pass irrelevant
> 37748 [host=potato-beetle]
> 37736 [host=scape-moth]
> 37751 [host=gall-mite]
> 37760 [host=grain-weevil]
> 37863 pass 97bf6af1f928216fd6c5a66e8a57bfa95a659672 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37865 pass b7392d2247cfe6771f95d256374f1a8e6a6f48d6 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37874 fail eaa27f34e91a14cdceed26ed6c6793ec1d186115 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37875 pass b1940cd21c0f4abdce101253e860feff547291b0 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37877 blocked aa9291355e19f804570466756ed7d874cd2e99ff c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37867 pass b1940cd21c0f4abdce101253e860feff547291b0 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37864 fail eaa27f34e91a14cdceed26ed6c6793ec1d186115 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37876 fail eaa27f34e91a14cdceed26ed6c6793ec1d186115 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37881 pass 11c8f01b423b2d9742ce21e44cb7da7b55429ad5 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37887 blocked 75069f2b5bfb5164beafaf3da597279c25b5535a c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37888 blocked eb74926920cfa756087a82e0b081df837177cb95 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37892 blocked 03c751a5e10caafbb6d1afcaf1ea67f2153c3193 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37893 pass b800c91a0517071156e772d4fb329ad33590da62 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37894 pass ddb321a8dd158520d97ed1cbade1d4ac36b6af31 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37895 fail 505569d208e61ab14f4b87957be0970ab33eb319 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37896 pass 5ab551d662396f8437ec5aba12210b7a67eb492b c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37898 fail 505569d208e61ab14f4b87957be0970ab33eb319 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37861 pass irrelevant
> 37899 pass 5ab551d662396f8437ec5aba12210b7a67eb492b c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37901 pass 97bf6af1f928216fd6c5a66e8a57bfa95a659672 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37902 fail eaa27f34e91a14cdceed26ed6c6793ec1d186115 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37903 fail ea174f4c4f6135e30a4e1e8c4511980338238b16 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37904 pass fbe1bf140671619508dfa575d74a185ae53c5dbb c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37905 fail 132978b94e66f8ad7d20790f8332f0e9c1426029 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37907 fail 505569d208e61ab14f4b87957be0970ab33eb319 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37908 pass 5ab551d662396f8437ec5aba12210b7a67eb492b c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37909 fail 505569d208e61ab14f4b87957be0970ab33eb319 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37910 pass fbe1bf140671619508dfa575d74a185ae53c5dbb c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37911 fail 132978b94e66f8ad7d20790f8332f0e9c1426029 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37912 pass fbe1bf140671619508dfa575d74a185ae53c5dbb c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37914 fail 132978b94e66f8ad7d20790f8332f0e9c1426029 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> Searching for interesting versions
> Result found: flight 37863 (pass), for basis pass
> Result found: flight 37864 (fail), for basis failure
> Repro found: flight 37901 (pass), for basis pass
> Repro found: flight 37902 (fail), for basis failure
> 0 revisions at fbe1bf140671619508dfa575d74a185ae53c5dbb c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> No revisions left to test, checking graph state.
> Result found: flight 37896 (pass), for last pass
> Result found: flight 37898 (fail), for first failure
> Repro found: flight 37899 (pass), for last pass
> Repro found: flight 37907 (fail), for first failure
> Repro found: flight 37908 (pass), for last pass
> Repro found: flight 37909 (fail), for first failure
>
> *** Found and reproduced problem changeset ***
>
> Bug is in tree: linux git://xenbits.xen.org/linux-pvops.git
> Bug introduced: 505569d208e61ab14f4b87957be0970ab33eb319
> Bug not present: 5ab551d662396f8437ec5aba12210b7a67eb492b
>
> + exec
> + sh -xe
> + cd /export/home/osstest/repos/linux-pvops
> + git remote set-url origin git://drall.uk.xensource.com:9419/git://xenbits.xen.org/linux-pvops.git
> + git fetch -p origin +refs/heads/*:refs/remotes/origin/*
>
> commit 505569d208e61ab14f4b87957be0970ab33eb319
> Merge: 5ab551d 2aba73a
> Author: Linus Torvalds <torvalds@linux-foundation.org>
> Date: Sun Jan 11 11:53:46 2015 -0800
>
> Merge branch 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
>
> Pull x86 fixes from Ingo Molnar:
> "Misc fixes: two vdso fixes, two kbuild fixes and a boot failure fix
> with certain odd memory mappings"
>
> * 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
> x86, vdso: Use asm volatile in __getcpu
> x86/build: Clean auto-generated processor feature files
> x86: Fix mkcapflags.sh bash-ism
> x86: Fix step size adjustment during initial memory mapping
> x86_64, vdso: Fix the vdso address randomization algorithm
>
> commit 2aba73a6146bb85d4a42386ca41dec0f4aa4b3ad
> Merge: 280dbc5 1ddf0b1
> Author: Ingo Molnar <mingo@kernel.org>
> Date: Thu Jan 1 22:21:22 2015 +0100
>
> Merge tag 'pr-20141223-x86-vdso' of git://git.kernel.org/pub/scm/linux/kernel/git/luto/linux into x86/urgent
>
> Pull VDSO fix from Andy Lutomirski:
>
> "This is hopefully the last vdso fix for 3.19. It should be very
> safe (it just adds a volatile).
>
> I don't think it fixes an actual bug (the __getcpu calls in the
> pvclock code may not have been needed in the first place), but
> discussion on that point is ongoing.
>
> It also fixes a big performance issue in 3.18 and earlier in which
> the lsl instructions in vclock_gettime got hoisted so far up the
> function that they happened even when the function they were in was
> never called. n 3.19, the performance issue seems to be gone due to
> the whims of my compiler and some interaction with a branch that's
> now gone.
>
> I'll hopefully have a much bigger overhaul of the pvclock code
> for 3.20, but it needs careful review."
>
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
>
> commit 1ddf0b1b11aa8a90cef6706e935fc31c75c406ba
> Author: Andy Lutomirski <luto@amacapital.net>
> Date: Sun Dec 21 08:57:46 2014 -0800
>
> x86, vdso: Use asm volatile in __getcpu
>
> In Linux 3.18 and below, GCC hoists the lsl instructions in the
> pvclock code all the way to the beginning of __vdso_clock_gettime,
> slowing the non-paravirt case significantly. For unknown reasons,
> presumably related to the removal of a branch, the performance issue
> is gone as of
>
> e76b027e6408 x86,vdso: Use LSL unconditionally for vgetcpu
>
> but I don't trust GCC enough to expect the problem to stay fixed.
>
> There should be no correctness issue, because the __getcpu calls in
> __vdso_vlock_gettime were never necessary in the first place.
>
> Note to stable maintainers: In 3.18 and below, depending on
> configuration, gcc 4.9.2 generates code like this:
>
> 9c3: 44 0f 03 e8 lsl %ax,%r13d
> 9c7: 45 89 eb mov %r13d,%r11d
> 9ca: 0f 03 d8 lsl %ax,%ebx
>
> This patch won't apply as is to any released kernel, but I'll send a
> trivial backported version if needed.
>
> Fixes: 51c19b4f5927 x86: vdso: pvclock gettime support
> Cc: stable@vger.kernel.org # 3.8+
> Cc: Marcelo Tosatti <mtosatti@redhat.com>
> Acked-by: Paolo Bonzini <pbonzini@redhat.com>
> Signed-off-by: Andy Lutomirski <luto@amacapital.net>
>
> commit 280dbc572357eb50184663fc9e4aaf09c8141e9b
> Author: Bjørn Mork <bjorn@mork.no>
> Date: Tue Dec 23 12:57:43 2014 +0100
>
> x86/build: Clean auto-generated processor feature files
>
> Commit 9def39be4e96 ("x86: Support compiling out human-friendly
> processor feature names") made two source file targets
> conditional. Such conditional targets will not be cleaned
> automatically by make mrproper.
>
> Fix by adding explicit clean-files targets for the two files.
>
> Fixes: 9def39be4e96 ("x86: Support compiling out human-friendly processor feature names")
> Signed-off-by: Bjørn Mork <bjorn@mork.no>
> Cc: Josh Triplett <josh@joshtriplett.org>
> Link: http://lkml.kernel.org/r/1419335863-10608-1-git-send-email-bjorn@mork.no
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
>
> commit ea174f4c4f6135e30a4e1e8c4511980338238b16
> Author: Sylvain BERTRAND <sylvain.bertrand@gmail.com>
> Date: Tue Dec 23 13:39:12 2014 +0100
>
> x86: Fix mkcapflags.sh bash-ism
>
> Chocked while compiling linux with dash shell instead of bash
> shell. See:
>
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html
>
> Signed-off-by: Sylvain BERTRAND <sylvain.bertrand@gmail.com>
> Link: http://lkml.kernel.org/r/20141223123912.GA1386@localhost.localdomain
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
>
> commit 132978b94e66f8ad7d20790f8332f0e9c1426029
> Author: Jan Beulich <JBeulich@suse.com>
> Date: Fri Dec 19 16:10:54 2014 +0000
>
> x86: Fix step size adjustment during initial memory mapping
>
> The old scheme can lead to failure in certain cases - the
> problem is that after bumping step_size the next (non-final)
> iteration is only guaranteed to make available a memory block
> the size of what step_size was before. E.g. for a memory block
> [0,3004600000) we'd have:
>
> iter> > start> > > end> > > step> > > amount
> 1> > 3004400000> > 30045fffff> > 2M> > > 2M
> 2> > 3004000000> > 30043fffff> > 64M> > > 4M
> 3> > 3000000000> > 3003ffffff> > 2G> > > 64M
> 4> > 2000000000> > 2fffffffff> > 64G> > > 64G
>
> Yet to map 64G with 4k pages (as happens e.g. under PV Xen) we
> need slightly over 128M, but the first three iterations made
> only about 70M available.
>
> The condition (new_mapped_ram_size > mapped_ram_size) for
> bumping step_size is just not suitable. Instead we want to bump
> it when we know we have enough memory available to cover a block
> of the new step_size. And rather than making that condition more
> complicated than needed, simply adjust step_size by the largest
> possible factor we know we can cover at that point - which is
> shifting it left by one less than the difference between page
> table level shifts. (Interestingly the original STEP_SIZE_SHIFT
> definition had a comment hinting at that having been the
> intention, just that it should have been PUD_SHIFT-PMD_SHIFT-1
> instead of (PUD_SHIFT-PMD_SHIFT)/2, and of course for non-PAE
> 32-bit we can't really use these two constants as they're equal
> there.)
>
> Furthermore the comment in get_new_step_size() didn't get
> updated when the bottom-down mapping logic got added. Yet while
> an overflow (flushing step_size to zero) of the shift doesn't
> matter for the top-down method, it does for bottom-up because
> round_up(x, 0) = 0, and an upper range boundary of zero can't
> really work well.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> Acked-by: Yinghai Lu <yinghai@kernel.org>
> Link: http://lkml.kernel.org/r/54945C1E020000780005114E@mail.emea.novell.com
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
>
> commit fbe1bf140671619508dfa575d74a185ae53c5dbb
> Merge: 97bf6af 394f56f
> Author: Ingo Molnar <mingo@kernel.org>
> Date: Sun Dec 21 11:16:49 2014 +0100
>
> Merge tag 'pr-20141220-x86-vdso' of git://git.kernel.org/pub/scm/linux/kernel/git/luto/linux into x86/urgent
>
> Pull a VDSO fix from Andy Lutomirski:
>
> "One vdso fix for a longstanding ASLR bug that's been in the news lately.
>
> The vdso base address has always been randomized, and I don't think there's
> anything particularly wrong with the range over which it's randomized,
> but the implementation seems to have been buggy since the very beginning.
>
> This fixes the implementation to remove a large bias that caused a small
> fraction of possible vdso load addresess to be vastly more likely than
> the rest of the possible addresses."
>
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
>
> commit 394f56fe480140877304d342dec46d50dc823d46
> Author: Andy Lutomirski <luto@amacapital.net>
> Date: Fri Dec 19 16:04:11 2014 -0800
>
> x86_64, vdso: Fix the vdso address randomization algorithm
>
> The theory behind vdso randomization is that it's mapped at a random
> offset above the top of the stack. To avoid wasting a page of
> memory for an extra page table, the vdso isn't supposed to extend
> past the lowest PMD into which it can fit. Other than that, the
> address should be a uniformly distributed address that meets all of
> the alignment requirements.
>
> The current algorithm is buggy: the vdso has about a 50% probability
> of being at the very end of a PMD. The current algorithm also has a
> decent chance of failing outright due to incorrect handling of the
> case where the top of the stack is near the top of its PMD.
>
> This fixes the implementation. The paxtest estimate of vdso
> "randomisation" improves from 11 bits to 18 bits. (Disclaimer: I
> don't know what the paxtest code is actually calculating.)
>
> It's worth noting that this algorithm is inherently biased: the vdso
> is more likely to end up near the end of its PMD than near the
> beginning. Ideally we would either nix the PMD sharing requirement
> or jointly randomize the vdso and the stack to reduce the bias.
>
> In the mean time, this is a considerable improvement with basically
> no risk of compatibility issues, since the allowed outputs of the
> algorithm are unchanged.
>
> As an easy test, doing this:
>
> for i in `seq 10000`
> do grep -P vdso /proc/self/maps |cut -d- -f1
> done |sort |uniq -d
>
> used to produce lots of output (1445 lines on my most recent run).
> A tiny subset looks like this:
>
> 7fffdfffe000
> 7fffe01fe000
> 7fffe05fe000
> 7fffe07fe000
> 7fffe09fe000
> 7fffe0bfe000
> 7fffe0dfe000
>
> Note the suspicious fe000 endings. With the fix, I get a much more
> palatable 76 repeated addresses.
>
> Reviewed-by: Kees Cook <keescook@chromium.org>
> Cc: stable@vger.kernel.org
> Signed-off-by: Andy Lutomirski <luto@amacapital.net>
>
> Result found: flight 37904 (pass), for last pass
> Result found: flight 37905 (fail), for first failure
> Repro found: flight 37910 (pass), for last pass
> Repro found: flight 37911 (fail), for first failure
> Repro found: flight 37912 (pass), for last pass
> Repro found: flight 37914 (fail), for first failure
>
> *** Found and reproduced problem changeset ***
>
> Bug is in tree: linux git://xenbits.xen.org/linux-pvops.git
> Bug introduced: 132978b94e66f8ad7d20790f8332f0e9c1426029
> Bug not present: fbe1bf140671619508dfa575d74a185ae53c5dbb
>
> + exec
> + sh -xe
> + cd /export/home/osstest/repos/linux-pvops
> + git remote set-url origin git://drall.uk.xensource.com:9419/git://xenbits.xen.org/linux-pvops.git
> + git fetch -p origin +refs/heads/*:refs/remotes/origin/*
>
> commit 132978b94e66f8ad7d20790f8332f0e9c1426029
> Author: Jan Beulich <JBeulich@suse.com>
> Date: Fri Dec 19 16:10:54 2014 +0000
>
> x86: Fix step size adjustment during initial memory mapping
>
> The old scheme can lead to failure in certain cases - the
> problem is that after bumping step_size the next (non-final)
> iteration is only guaranteed to make available a memory block
> the size of what step_size was before. E.g. for a memory block
> [0,3004600000) we'd have:
>
> iter> > start> > > end> > > step> > > amount
> 1> > 3004400000> > 30045fffff> > 2M> > > 2M
> 2> > 3004000000> > 30043fffff> > 64M> > > 4M
> 3> > 3000000000> > 3003ffffff> > 2G> > > 64M
> 4> > 2000000000> > 2fffffffff> > 64G> > > 64G
>
> Yet to map 64G with 4k pages (as happens e.g. under PV Xen) we
> need slightly over 128M, but the first three iterations made
> only about 70M available.
>
> The condition (new_mapped_ram_size > mapped_ram_size) for
> bumping step_size is just not suitable. Instead we want to bump
> it when we know we have enough memory available to cover a block
> of the new step_size. And rather than making that condition more
> complicated than needed, simply adjust step_size by the largest
> possible factor we know we can cover at that point - which is
> shifting it left by one less than the difference between page
> table level shifts. (Interestingly the original STEP_SIZE_SHIFT
> definition had a comment hinting at that having been the
> intention, just that it should have been PUD_SHIFT-PMD_SHIFT-1
> instead of (PUD_SHIFT-PMD_SHIFT)/2, and of course for non-PAE
> 32-bit we can't really use these two constants as they're equal
> there.)
>
> Furthermore the comment in get_new_step_size() didn't get
> updated when the bottom-down mapping logic got added. Yet while
> an overflow (flushing step_size to zero) of the shift doesn't
> matter for the top-down method, it does for bottom-up because
> round_up(x, 0) = 0, and an upper range boundary of zero can't
> really work well.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> Acked-by: Yinghai Lu <yinghai@kernel.org>
> Link: http://lkml.kernel.org/r/54945C1E020000780005114E@mail.emea.novell.com
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
>
> Revision graph left in /home/osstest/results-adhoc/bisect/xen-unstable/test-amd64-i386-xl..{dot,ps,png,html}.
> ----------------------------------------
> 37914: trouble: broken/fail
>
> flight 37914 xen-unstable adhoc-bisect [real]
> http://osstest.xs.citrite.net/~osstest/testlogs/logs/37914/
>
> Failures and problems with tests :-(
>
> Tests which did not succeed,
> including tests which could not be run:
> 37837.build-i386 broken
> 37837.build-amd64 broken
> test-amd64-i386-xl 14 guest-saverestore fail baseline untested
>
>
> jobs:
> 37837.build-amd64 broken
> 37837.build-i386 broken
> test-amd64-i386-xl fail
>
>
> ------------------------------------------------------------
> sg-report-flight on osstest.xs.citrite.net
> logs: /home/osstest/logs
> images: /home/osstest/images
>
> Logs, config files, etc. are available at
> http://osstest.xs.citrite.net/~osstest/testlogs/logs
>
> Test harness code can be found at
> http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] only message in thread