From: Ian Campbell <ian.campbell@citrix.com>
To: ian.jackson@eu.citrix.com, Wei Liu <wei.liu2@citrix.com>,
Jan Beulich <JBeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>, xen-devel@lists.xen.org
Subject: Re: [adhoc xen-unstable bisection] complete test-amd64-i386-xl
Date: Mon, 7 Sep 2015 09:53:51 +0100 [thread overview]
Message-ID: <1441616031.25589.35.camel@citrix.com> (raw)
In-Reply-To: <E1ZYQs7-0006Wu-Vx@osstest.xs.citrite.net>
This is the result of the bisection of the migration issue with 32-bit on
Linux 4.1 onwards.
This is just for info as I believe the issue is already under discussion
etc elsewhere.
I've copied the final graph to
http://xenbits.xen.org/people/ianc/tmp/adhoc/test-amd64-i386-xl..html
Ian.
On Sun, 2015-09-06 at 04:44 +0100, Platform Team regression test user wrote:
> branch xen-unstable
> xen branch xen-unstable
> job test-amd64-i386-xl
> test guest-saverestore
>
> Tree: linux git://xenbits.xen.org/linux-pvops.git
> Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
> Tree: qemu git://xenbits.xen.org/staging/qemu-xen-unstable.git
> Tree: qemuu git://xenbits.xen.org/staging/qemu-upstream-unstable.git
> Tree: xen git://xenbits.xen.org/xen.git
[...snip first pass blaming a merge changeset which I aborted but was still reported here...]
> *** Found and reproduced problem changeset ***
>
> Bug is in tree: linux git://xenbits.xen.org/linux-pvops.git
> Bug introduced: 132978b94e66f8ad7d20790f8332f0e9c1426029
> Bug not present: fbe1bf140671619508dfa575d74a185ae53c5dbb
>
>
> commit 132978b94e66f8ad7d20790f8332f0e9c1426029
> Author: Jan Beulich <JBeulich@suse.com>
> Date: Fri Dec 19 16:10:54 2014 +0000
>
> x86: Fix step size adjustment during initial memory mapping
>
> The old scheme can lead to failure in certain cases - the
> problem is that after bumping step_size the next (non-final)
> iteration is only guaranteed to make available a memory block
> the size of what step_size was before. E.g. for a memory block
> [0,3004600000) we'd have:
>
> iter> > start> > > end> > > step> > > amount
> 1> > 3004400000> > 30045fffff> > 2M> > > 2M
> 2> > 3004000000> > 30043fffff> > 64M> > > 4M
> 3> > 3000000000> > 3003ffffff> > 2G> > > 64M
> 4> > 2000000000> > 2fffffffff> > 64G> > > 64G
>
> Yet to map 64G with 4k pages (as happens e.g. under PV Xen) we
> need slightly over 128M, but the first three iterations made
> only about 70M available.
>
> The condition (new_mapped_ram_size > mapped_ram_size) for
> bumping step_size is just not suitable. Instead we want to bump
> it when we know we have enough memory available to cover a block
> of the new step_size. And rather than making that condition more
> complicated than needed, simply adjust step_size by the largest
> possible factor we know we can cover at that point - which is
> shifting it left by one less than the difference between page
> table level shifts. (Interestingly the original STEP_SIZE_SHIFT
> definition had a comment hinting at that having been the
> intention, just that it should have been PUD_SHIFT-PMD_SHIFT-1
> instead of (PUD_SHIFT-PMD_SHIFT)/2, and of course for non-PAE
> 32-bit we can't really use these two constants as they're equal
> there.)
>
> Furthermore the comment in get_new_step_size() didn't get
> updated when the bottom-down mapping logic got added. Yet while
> an overflow (flushing step_size to zero) of the shift doesn't
> matter for the top-down method, it does for bottom-up because
> round_up(x, 0) = 0, and an upper range boundary of zero can't
> really work well.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> Acked-by: Yinghai Lu <yinghai@kernel.org>
> Link: http://lkml.kernel.org/r/54945C1E020000780005114E@mail.emea.novell.com
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
>
>
> For bisection revision-tuple graph see:
> http://osstest.xs.citrite.net/~osstest/testlogs/results-adhoc/bisect/xen-unstable/test-amd64-i386-xl..html
> Revision IDs in each graph node refer, respectively, to the Trees above.
>
> ----------------------------------------
> Searching for failure / basis pass:
> 37864 fail [host=rice-weevil] / 37863 ok.
> Failure / basis pass flights: 37864 / 37863
> (tree with no url: ovmf)
> (tree with no url: seabios)
> Tree: linux git://xenbits.xen.org/linux-pvops.git
> Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
> Tree: qemu git://xenbits.xen.org/staging/qemu-xen-unstable.git
> Tree: qemuu git://xenbits.xen.org/staging/qemu-upstream-unstable.git
> Tree: xen git://xenbits.xen.org/xen.git
> Latest eaa27f34e91a14cdceed26ed6c6793ec1d186115 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> Basis pass 97bf6af1f928216fd6c5a66e8a57bfa95a659672 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> Generating revisions with ./adhoc-revtuple-generator git://xenbits.xen.org/linux-pvops.git#97bf6af1f928216fd6c5a66e8a57bfa95a659672-eaa27f34e91a14cdceed26ed6c6793ec1d186115 git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860 git://xenbits.xen.org/staging/qemu-xen-unstable.git#7f057440b31da38196e3398fd1b618fc36ad97d6-7f057440b31da38196e3398fd1b618fc36ad97d6 git://xenbits.xen.org/staging/qemu-upstream-unstable.git#bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa-bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa git://xenbits.xen.org/xen.git#145a8004a7d659668d5a3b0ad9868d7678b24822-145a8004a7d659668d5a3b0ad9868d7678b24822
> + exec
> + sh -xe
> + cd /export/home/osstest/repos/linux-pvops
> + git remote set-url origin git://drall.uk.xensource.com:9419/git://xenbits.xen.org/linux-pvops.git
> + git fetch -p origin +refs/heads/*:refs/remotes/origin/*
> + exec
> + sh -xe
> + cd /export/home/osstest/repos/linux-pvops
> + git remote set-url origin git://drall.uk.xensource.com:9419/git://xenbits.xen.org/linux-pvops.git
> + git fetch -p origin +refs/heads/*:refs/remotes/origin/*
> Loaded 1103 nodes in revision graph
> Searching for test results:
> 34279 [host=worm-moth]
> 37011 [host=bush-cricket]
> 37043 [host=grain-weevil]
> 37077 [host=scape-moth]
> 37289 [host=lace-bug]
> 37292 [host=bush-cricket]
> 37295 [host=scape-moth]
> 37298 pass irrelevant
> 37326 [host=moss-bug]
> 37331 [host=lace-bug]
> 37332 [host=itch-mite]
> 37377 [host=leaf-beetle]
> 37415 [host=grain-weevil]
> 37473 [host=gall-mite]
> 37484 [host=potato-beetle]
> 37536 []
> 37546 [host=lace-bug]
> 37552 [host=scape-moth]
> 37589 pass irrelevant
> 37719 [host=grain-weevil]
> 37711 pass irrelevant
> 37748 [host=potato-beetle]
> 37736 [host=scape-moth]
> 37751 [host=gall-mite]
> 37760 [host=grain-weevil]
> 37863 pass 97bf6af1f928216fd6c5a66e8a57bfa95a659672 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37865 pass b7392d2247cfe6771f95d256374f1a8e6a6f48d6 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37874 fail eaa27f34e91a14cdceed26ed6c6793ec1d186115 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37875 pass b1940cd21c0f4abdce101253e860feff547291b0 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37877 blocked aa9291355e19f804570466756ed7d874cd2e99ff c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37867 pass b1940cd21c0f4abdce101253e860feff547291b0 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37864 fail eaa27f34e91a14cdceed26ed6c6793ec1d186115 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37876 fail eaa27f34e91a14cdceed26ed6c6793ec1d186115 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37881 pass 11c8f01b423b2d9742ce21e44cb7da7b55429ad5 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37887 blocked 75069f2b5bfb5164beafaf3da597279c25b5535a c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37888 blocked eb74926920cfa756087a82e0b081df837177cb95 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37892 blocked 03c751a5e10caafbb6d1afcaf1ea67f2153c3193 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37893 pass b800c91a0517071156e772d4fb329ad33590da62 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37894 pass ddb321a8dd158520d97ed1cbade1d4ac36b6af31 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37895 fail 505569d208e61ab14f4b87957be0970ab33eb319 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37896 pass 5ab551d662396f8437ec5aba12210b7a67eb492b c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37898 fail 505569d208e61ab14f4b87957be0970ab33eb319 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37861 pass irrelevant
> 37899 pass 5ab551d662396f8437ec5aba12210b7a67eb492b c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37901 pass 97bf6af1f928216fd6c5a66e8a57bfa95a659672 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37902 fail eaa27f34e91a14cdceed26ed6c6793ec1d186115 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37903 fail ea174f4c4f6135e30a4e1e8c4511980338238b16 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37904 pass fbe1bf140671619508dfa575d74a185ae53c5dbb c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37905 fail 132978b94e66f8ad7d20790f8332f0e9c1426029 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37907 fail 505569d208e61ab14f4b87957be0970ab33eb319 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37908 pass 5ab551d662396f8437ec5aba12210b7a67eb492b c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37909 fail 505569d208e61ab14f4b87957be0970ab33eb319 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37910 pass fbe1bf140671619508dfa575d74a185ae53c5dbb c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37911 fail 132978b94e66f8ad7d20790f8332f0e9c1426029 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37912 pass fbe1bf140671619508dfa575d74a185ae53c5dbb c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> 37914 fail 132978b94e66f8ad7d20790f8332f0e9c1426029 c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> Searching for interesting versions
> Result found: flight 37863 (pass), for basis pass
> Result found: flight 37864 (fail), for basis failure
> Repro found: flight 37901 (pass), for basis pass
> Repro found: flight 37902 (fail), for basis failure
> 0 revisions at fbe1bf140671619508dfa575d74a185ae53c5dbb c530a75c1e6a472b0eb9558310b518f0dfcd8860 7f057440b31da38196e3398fd1b618fc36ad97d6 bcf35eec0b621c46dbf0aeb40c6bc06b5d3981aa 145a8004a7d659668d5a3b0ad9868d7678b24822
> No revisions left to test, checking graph state.
> Result found: flight 37896 (pass), for last pass
> Result found: flight 37898 (fail), for first failure
> Repro found: flight 37899 (pass), for last pass
> Repro found: flight 37907 (fail), for first failure
> Repro found: flight 37908 (pass), for last pass
> Repro found: flight 37909 (fail), for first failure
>
> *** Found and reproduced problem changeset ***
>
> Bug is in tree: linux git://xenbits.xen.org/linux-pvops.git
> Bug introduced: 505569d208e61ab14f4b87957be0970ab33eb319
> Bug not present: 5ab551d662396f8437ec5aba12210b7a67eb492b
>
> + exec
> + sh -xe
> + cd /export/home/osstest/repos/linux-pvops
> + git remote set-url origin git://drall.uk.xensource.com:9419/git://xenbits.xen.org/linux-pvops.git
> + git fetch -p origin +refs/heads/*:refs/remotes/origin/*
>
> commit 505569d208e61ab14f4b87957be0970ab33eb319
> Merge: 5ab551d 2aba73a
> Author: Linus Torvalds <torvalds@linux-foundation.org>
> Date: Sun Jan 11 11:53:46 2015 -0800
>
> Merge branch 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
>
> Pull x86 fixes from Ingo Molnar:
> "Misc fixes: two vdso fixes, two kbuild fixes and a boot failure fix
> with certain odd memory mappings"
>
> * 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
> x86, vdso: Use asm volatile in __getcpu
> x86/build: Clean auto-generated processor feature files
> x86: Fix mkcapflags.sh bash-ism
> x86: Fix step size adjustment during initial memory mapping
> x86_64, vdso: Fix the vdso address randomization algorithm
>
> commit 2aba73a6146bb85d4a42386ca41dec0f4aa4b3ad
> Merge: 280dbc5 1ddf0b1
> Author: Ingo Molnar <mingo@kernel.org>
> Date: Thu Jan 1 22:21:22 2015 +0100
>
> Merge tag 'pr-20141223-x86-vdso' of git://git.kernel.org/pub/scm/linux/kernel/git/luto/linux into x86/urgent
>
> Pull VDSO fix from Andy Lutomirski:
>
> "This is hopefully the last vdso fix for 3.19. It should be very
> safe (it just adds a volatile).
>
> I don't think it fixes an actual bug (the __getcpu calls in the
> pvclock code may not have been needed in the first place), but
> discussion on that point is ongoing.
>
> It also fixes a big performance issue in 3.18 and earlier in which
> the lsl instructions in vclock_gettime got hoisted so far up the
> function that they happened even when the function they were in was
> never called. n 3.19, the performance issue seems to be gone due to
> the whims of my compiler and some interaction with a branch that's
> now gone.
>
> I'll hopefully have a much bigger overhaul of the pvclock code
> for 3.20, but it needs careful review."
>
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
>
> commit 1ddf0b1b11aa8a90cef6706e935fc31c75c406ba
> Author: Andy Lutomirski <luto@amacapital.net>
> Date: Sun Dec 21 08:57:46 2014 -0800
>
> x86, vdso: Use asm volatile in __getcpu
>
> In Linux 3.18 and below, GCC hoists the lsl instructions in the
> pvclock code all the way to the beginning of __vdso_clock_gettime,
> slowing the non-paravirt case significantly. For unknown reasons,
> presumably related to the removal of a branch, the performance issue
> is gone as of
>
> e76b027e6408 x86,vdso: Use LSL unconditionally for vgetcpu
>
> but I don't trust GCC enough to expect the problem to stay fixed.
>
> There should be no correctness issue, because the __getcpu calls in
> __vdso_vlock_gettime were never necessary in the first place.
>
> Note to stable maintainers: In 3.18 and below, depending on
> configuration, gcc 4.9.2 generates code like this:
>
> 9c3: 44 0f 03 e8 lsl %ax,%r13d
> 9c7: 45 89 eb mov %r13d,%r11d
> 9ca: 0f 03 d8 lsl %ax,%ebx
>
> This patch won't apply as is to any released kernel, but I'll send a
> trivial backported version if needed.
>
> Fixes: 51c19b4f5927 x86: vdso: pvclock gettime support
> Cc: stable@vger.kernel.org # 3.8+
> Cc: Marcelo Tosatti <mtosatti@redhat.com>
> Acked-by: Paolo Bonzini <pbonzini@redhat.com>
> Signed-off-by: Andy Lutomirski <luto@amacapital.net>
>
> commit 280dbc572357eb50184663fc9e4aaf09c8141e9b
> Author: Bjørn Mork <bjorn@mork.no>
> Date: Tue Dec 23 12:57:43 2014 +0100
>
> x86/build: Clean auto-generated processor feature files
>
> Commit 9def39be4e96 ("x86: Support compiling out human-friendly
> processor feature names") made two source file targets
> conditional. Such conditional targets will not be cleaned
> automatically by make mrproper.
>
> Fix by adding explicit clean-files targets for the two files.
>
> Fixes: 9def39be4e96 ("x86: Support compiling out human-friendly processor feature names")
> Signed-off-by: Bjørn Mork <bjorn@mork.no>
> Cc: Josh Triplett <josh@joshtriplett.org>
> Link: http://lkml.kernel.org/r/1419335863-10608-1-git-send-email-bjorn@mork.no
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
>
> commit ea174f4c4f6135e30a4e1e8c4511980338238b16
> Author: Sylvain BERTRAND <sylvain.bertrand@gmail.com>
> Date: Tue Dec 23 13:39:12 2014 +0100
>
> x86: Fix mkcapflags.sh bash-ism
>
> Chocked while compiling linux with dash shell instead of bash
> shell. See:
>
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/test.html
>
> Signed-off-by: Sylvain BERTRAND <sylvain.bertrand@gmail.com>
> Link: http://lkml.kernel.org/r/20141223123912.GA1386@localhost.localdomain
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
>
> commit 132978b94e66f8ad7d20790f8332f0e9c1426029
> Author: Jan Beulich <JBeulich@suse.com>
> Date: Fri Dec 19 16:10:54 2014 +0000
>
> x86: Fix step size adjustment during initial memory mapping
>
> The old scheme can lead to failure in certain cases - the
> problem is that after bumping step_size the next (non-final)
> iteration is only guaranteed to make available a memory block
> the size of what step_size was before. E.g. for a memory block
> [0,3004600000) we'd have:
>
> iter> > start> > > end> > > step> > > amount
> 1> > 3004400000> > 30045fffff> > 2M> > > 2M
> 2> > 3004000000> > 30043fffff> > 64M> > > 4M
> 3> > 3000000000> > 3003ffffff> > 2G> > > 64M
> 4> > 2000000000> > 2fffffffff> > 64G> > > 64G
>
> Yet to map 64G with 4k pages (as happens e.g. under PV Xen) we
> need slightly over 128M, but the first three iterations made
> only about 70M available.
>
> The condition (new_mapped_ram_size > mapped_ram_size) for
> bumping step_size is just not suitable. Instead we want to bump
> it when we know we have enough memory available to cover a block
> of the new step_size. And rather than making that condition more
> complicated than needed, simply adjust step_size by the largest
> possible factor we know we can cover at that point - which is
> shifting it left by one less than the difference between page
> table level shifts. (Interestingly the original STEP_SIZE_SHIFT
> definition had a comment hinting at that having been the
> intention, just that it should have been PUD_SHIFT-PMD_SHIFT-1
> instead of (PUD_SHIFT-PMD_SHIFT)/2, and of course for non-PAE
> 32-bit we can't really use these two constants as they're equal
> there.)
>
> Furthermore the comment in get_new_step_size() didn't get
> updated when the bottom-down mapping logic got added. Yet while
> an overflow (flushing step_size to zero) of the shift doesn't
> matter for the top-down method, it does for bottom-up because
> round_up(x, 0) = 0, and an upper range boundary of zero can't
> really work well.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> Acked-by: Yinghai Lu <yinghai@kernel.org>
> Link: http://lkml.kernel.org/r/54945C1E020000780005114E@mail.emea.novell.com
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
>
> commit fbe1bf140671619508dfa575d74a185ae53c5dbb
> Merge: 97bf6af 394f56f
> Author: Ingo Molnar <mingo@kernel.org>
> Date: Sun Dec 21 11:16:49 2014 +0100
>
> Merge tag 'pr-20141220-x86-vdso' of git://git.kernel.org/pub/scm/linux/kernel/git/luto/linux into x86/urgent
>
> Pull a VDSO fix from Andy Lutomirski:
>
> "One vdso fix for a longstanding ASLR bug that's been in the news lately.
>
> The vdso base address has always been randomized, and I don't think there's
> anything particularly wrong with the range over which it's randomized,
> but the implementation seems to have been buggy since the very beginning.
>
> This fixes the implementation to remove a large bias that caused a small
> fraction of possible vdso load addresess to be vastly more likely than
> the rest of the possible addresses."
>
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
>
> commit 394f56fe480140877304d342dec46d50dc823d46
> Author: Andy Lutomirski <luto@amacapital.net>
> Date: Fri Dec 19 16:04:11 2014 -0800
>
> x86_64, vdso: Fix the vdso address randomization algorithm
>
> The theory behind vdso randomization is that it's mapped at a random
> offset above the top of the stack. To avoid wasting a page of
> memory for an extra page table, the vdso isn't supposed to extend
> past the lowest PMD into which it can fit. Other than that, the
> address should be a uniformly distributed address that meets all of
> the alignment requirements.
>
> The current algorithm is buggy: the vdso has about a 50% probability
> of being at the very end of a PMD. The current algorithm also has a
> decent chance of failing outright due to incorrect handling of the
> case where the top of the stack is near the top of its PMD.
>
> This fixes the implementation. The paxtest estimate of vdso
> "randomisation" improves from 11 bits to 18 bits. (Disclaimer: I
> don't know what the paxtest code is actually calculating.)
>
> It's worth noting that this algorithm is inherently biased: the vdso
> is more likely to end up near the end of its PMD than near the
> beginning. Ideally we would either nix the PMD sharing requirement
> or jointly randomize the vdso and the stack to reduce the bias.
>
> In the mean time, this is a considerable improvement with basically
> no risk of compatibility issues, since the allowed outputs of the
> algorithm are unchanged.
>
> As an easy test, doing this:
>
> for i in `seq 10000`
> do grep -P vdso /proc/self/maps |cut -d- -f1
> done |sort |uniq -d
>
> used to produce lots of output (1445 lines on my most recent run).
> A tiny subset looks like this:
>
> 7fffdfffe000
> 7fffe01fe000
> 7fffe05fe000
> 7fffe07fe000
> 7fffe09fe000
> 7fffe0bfe000
> 7fffe0dfe000
>
> Note the suspicious fe000 endings. With the fix, I get a much more
> palatable 76 repeated addresses.
>
> Reviewed-by: Kees Cook <keescook@chromium.org>
> Cc: stable@vger.kernel.org
> Signed-off-by: Andy Lutomirski <luto@amacapital.net>
>
> Result found: flight 37904 (pass), for last pass
> Result found: flight 37905 (fail), for first failure
> Repro found: flight 37910 (pass), for last pass
> Repro found: flight 37911 (fail), for first failure
> Repro found: flight 37912 (pass), for last pass
> Repro found: flight 37914 (fail), for first failure
>
> *** Found and reproduced problem changeset ***
>
> Bug is in tree: linux git://xenbits.xen.org/linux-pvops.git
> Bug introduced: 132978b94e66f8ad7d20790f8332f0e9c1426029
> Bug not present: fbe1bf140671619508dfa575d74a185ae53c5dbb
>
> + exec
> + sh -xe
> + cd /export/home/osstest/repos/linux-pvops
> + git remote set-url origin git://drall.uk.xensource.com:9419/git://xenbits.xen.org/linux-pvops.git
> + git fetch -p origin +refs/heads/*:refs/remotes/origin/*
>
> commit 132978b94e66f8ad7d20790f8332f0e9c1426029
> Author: Jan Beulich <JBeulich@suse.com>
> Date: Fri Dec 19 16:10:54 2014 +0000
>
> x86: Fix step size adjustment during initial memory mapping
>
> The old scheme can lead to failure in certain cases - the
> problem is that after bumping step_size the next (non-final)
> iteration is only guaranteed to make available a memory block
> the size of what step_size was before. E.g. for a memory block
> [0,3004600000) we'd have:
>
> iter> > start> > > end> > > step> > > amount
> 1> > 3004400000> > 30045fffff> > 2M> > > 2M
> 2> > 3004000000> > 30043fffff> > 64M> > > 4M
> 3> > 3000000000> > 3003ffffff> > 2G> > > 64M
> 4> > 2000000000> > 2fffffffff> > 64G> > > 64G
>
> Yet to map 64G with 4k pages (as happens e.g. under PV Xen) we
> need slightly over 128M, but the first three iterations made
> only about 70M available.
>
> The condition (new_mapped_ram_size > mapped_ram_size) for
> bumping step_size is just not suitable. Instead we want to bump
> it when we know we have enough memory available to cover a block
> of the new step_size. And rather than making that condition more
> complicated than needed, simply adjust step_size by the largest
> possible factor we know we can cover at that point - which is
> shifting it left by one less than the difference between page
> table level shifts. (Interestingly the original STEP_SIZE_SHIFT
> definition had a comment hinting at that having been the
> intention, just that it should have been PUD_SHIFT-PMD_SHIFT-1
> instead of (PUD_SHIFT-PMD_SHIFT)/2, and of course for non-PAE
> 32-bit we can't really use these two constants as they're equal
> there.)
>
> Furthermore the comment in get_new_step_size() didn't get
> updated when the bottom-down mapping logic got added. Yet while
> an overflow (flushing step_size to zero) of the shift doesn't
> matter for the top-down method, it does for bottom-up because
> round_up(x, 0) = 0, and an upper range boundary of zero can't
> really work well.
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> Acked-by: Yinghai Lu <yinghai@kernel.org>
> Link: http://lkml.kernel.org/r/54945C1E020000780005114E@mail.emea.novell.com
> Signed-off-by: Ingo Molnar <mingo@kernel.org>
>
> Revision graph left in /home/osstest/results-adhoc/bisect/xen-unstable/test-amd64-i386-xl..{dot,ps,png,html}.
> ----------------------------------------
> 37914: trouble: broken/fail
>
> flight 37914 xen-unstable adhoc-bisect [real]
> http://osstest.xs.citrite.net/~osstest/testlogs/logs/37914/
>
> Failures and problems with tests :-(
>
> Tests which did not succeed,
> including tests which could not be run:
> 37837.build-i386 broken
> 37837.build-amd64 broken
> test-amd64-i386-xl 14 guest-saverestore fail baseline untested
>
>
> jobs:
> 37837.build-amd64 broken
> 37837.build-i386 broken
> test-amd64-i386-xl fail
>
>
> ------------------------------------------------------------
> sg-report-flight on osstest.xs.citrite.net
> logs: /home/osstest/logs
> images: /home/osstest/images
>
> Logs, config files, etc. are available at
> http://osstest.xs.citrite.net/~osstest/testlogs/logs
>
> Test harness code can be found at
> http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
parent reply other threads:[~2015-09-07 8:53 UTC|newest]
Thread overview: expand[flat|nested] mbox.gz Atom feed
[parent not found: <E1ZYQs7-0006Wu-Vx@osstest.xs.citrite.net>]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1441616031.25589.35.camel@citrix.com \
--to=ian.campbell@citrix.com \
--cc=JBeulich@suse.com \
--cc=andrew.cooper3@citrix.com \
--cc=ian.jackson@eu.citrix.com \
--cc=wei.liu2@citrix.com \
--cc=xen-devel@lists.xen.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.