randomized placement of x86_64 vdso

randomized placement of x86_64 vdso

[Nathan Lynch]

Hi x86/vdso people,

I've been working on adding a vDSO to 32-bit ARM, and Kees suggested I
look at x86_64's algorithm for placing the vDSO at a randomized offset
above the stack VMA.  I found that when the stack top occupies the
last slot in the PTE (is that the right term?), the vdso_addr routine
returns an address below mm->start_stack, equivalent to
(mm->start_stack & PAGE_MASK).  For instance if mm->start_stack is
0x7fff3ffffc96, vdso_addr returns 0x7fff3ffff000.

Since the address returned is always already occupied by the stack,
get_unmapped_area detects the collision and falls back to
vm_unmapped_area.  This results in the vdso being placed in the
address space next to libraries etc.  While this is generally
unnoticeable and doesn't break anything, it does mean that the vdso is
placed below the stack when there is actually room above the stack.
To me it also seems uncomfortably close to placing the vdso in the way
of downward expansion of the stack.

I don't have a patch because I'm not sure what the algorithm should
be, but thought I would bring it up as vdso_addr doesn't seem to be
behaving as intended in all cases.


Thanks,
Nathan

Re: randomized placement of x86_64 vdso

[H. Peter Anvin]

On 04/21/2014 09:52 AM, Nathan Lynch wrote:
> Hi x86/vdso people,
> 
> I've been working on adding a vDSO to 32-bit ARM, and Kees suggested I
> look at x86_64's algorithm for placing the vDSO at a randomized offset
> above the stack VMA.  I found that when the stack top occupies the
> last slot in the PTE (is that the right term?), the vdso_addr routine
> returns an address below mm->start_stack, equivalent to
> (mm->start_stack & PAGE_MASK).  For instance if mm->start_stack is
> 0x7fff3ffffc96, vdso_addr returns 0x7fff3ffff000.
> 
> Since the address returned is always already occupied by the stack,
> get_unmapped_area detects the collision and falls back to
> vm_unmapped_area.  This results in the vdso being placed in the
> address space next to libraries etc.  While this is generally
> unnoticeable and doesn't break anything, it does mean that the vdso is
> placed below the stack when there is actually room above the stack.
> To me it also seems uncomfortably close to placing the vdso in the way
> of downward expansion of the stack.
> 
> I don't have a patch because I'm not sure what the algorithm should
> be, but thought I would bring it up as vdso_addr doesn't seem to be
> behaving as intended in all cases.
> 

If the stack occupies the last possible page, how can you say there is
"space above the stack"?

	-hpa

Re: randomized placement of x86_64 vdso

[Nathan Lynch]

On 04/23/2014 11:30 AM, H. Peter Anvin wrote:
> On 04/21/2014 09:52 AM, Nathan Lynch wrote:
>> Hi x86/vdso people,
>>
>> I've been working on adding a vDSO to 32-bit ARM, and Kees suggested I
>> look at x86_64's algorithm for placing the vDSO at a randomized offset
>> above the stack VMA.  I found that when the stack top occupies the
>> last slot in the PTE (is that the right term?), the vdso_addr routine
>> returns an address below mm->start_stack, equivalent to
>> (mm->start_stack & PAGE_MASK).  For instance if mm->start_stack is
>> 0x7fff3ffffc96, vdso_addr returns 0x7fff3ffff000.
>>
>> Since the address returned is always already occupied by the stack,
>> get_unmapped_area detects the collision and falls back to
>> vm_unmapped_area.  This results in the vdso being placed in the
>> address space next to libraries etc.  While this is generally
>> unnoticeable and doesn't break anything, it does mean that the vdso is
>> placed below the stack when there is actually room above the stack.
>> To me it also seems uncomfortably close to placing the vdso in the way
>> of downward expansion of the stack.
>>
>> I don't have a patch because I'm not sure what the algorithm should
>> be, but thought I would bring it up as vdso_addr doesn't seem to be
>> behaving as intended in all cases.
>>
> 
> If the stack occupies the last possible page, how can you say there is
> "space above the stack"?

Sorry for being unclear.  I probably am getting terminology wrong.  What
I'm trying to express is that if the stack top is in the last page of
its last-level page table (which may be the last possible page, but
that's not really the interesting case), vdso_addr returns an address
below mm->start_stack.

If you do a lot of execs with the following debug patch applied,
you should see occasional prints like:

got addr 0x7f9a2ba16000, asked 0x7fffa7bff000, start_stack=0x7fffa7bffc96
got addr 0x7f3877ff1000, asked 0x7fffd9bff000, start_stack=0x7fffd9bffc96
got addr 0x7f96e3637000, asked 0x7ffff39ff000, start_stack=0x7ffff39ffc96
got addr 0x7fb70588d000, asked 0x7fff271ff000, start_stack=0x7fff271ffc96
got addr 0x7f7957171000, asked 0x7fff71dff000, start_stack=0x7fff71dffc96

Hopefully this better illustrates.

diff --git a/arch/x86/vdso/vma.c b/arch/x86/vdso/vma.c
index 1ad102613127..06c51329d1b3 100644
--- a/arch/x86/vdso/vma.c
+++ b/arch/x86/vdso/vma.c
@@ -157,15 +157,17 @@ static int setup_additional_pages(struct linux_binprm *bprm,
 				  unsigned size)
 {
 	struct mm_struct *mm = current->mm;
-	unsigned long addr;
+	unsigned long addr, hint;
 	int ret;
 
 	if (!vdso_enabled)
 		return 0;
 
 	down_write(&mm->mmap_sem);
-	addr = vdso_addr(mm->start_stack, size);
-	addr = get_unmapped_area(NULL, addr, size, 0, 0);
+	hint = vdso_addr(mm->start_stack, size);
+	addr = get_unmapped_area(NULL, hint, size, 0, 0);
+	if (addr != hint)
+		pr_info("got addr 0x%lx, asked 0x%lx\n", addr, hint);
 	if (IS_ERR_VALUE(addr)) {
 		ret = addr;
 		goto up_fail;

Re: randomized placement of x86_64 vdso

[Kees Cook]

On Wed, Apr 23, 2014 at 10:06 AM, Nathan Lynch <Nathan_Lynch@mentor.com> wrote:
> On 04/23/2014 11:30 AM, H. Peter Anvin wrote:
>> On 04/21/2014 09:52 AM, Nathan Lynch wrote:
>>> Hi x86/vdso people,
>>>
>>> I've been working on adding a vDSO to 32-bit ARM, and Kees suggested I
>>> look at x86_64's algorithm for placing the vDSO at a randomized offset
>>> above the stack VMA.  I found that when the stack top occupies the
>>> last slot in the PTE (is that the right term?), the vdso_addr routine
>>> returns an address below mm->start_stack, equivalent to
>>> (mm->start_stack & PAGE_MASK).  For instance if mm->start_stack is
>>> 0x7fff3ffffc96, vdso_addr returns 0x7fff3ffff000.
>>>
>>> Since the address returned is always already occupied by the stack,
>>> get_unmapped_area detects the collision and falls back to
>>> vm_unmapped_area.  This results in the vdso being placed in the
>>> address space next to libraries etc.  While this is generally
>>> unnoticeable and doesn't break anything, it does mean that the vdso is
>>> placed below the stack when there is actually room above the stack.
>>> To me it also seems uncomfortably close to placing the vdso in the way
>>> of downward expansion of the stack.
>>>
>>> I don't have a patch because I'm not sure what the algorithm should
>>> be, but thought I would bring it up as vdso_addr doesn't seem to be
>>> behaving as intended in all cases.
>>>
>>
>> If the stack occupies the last possible page, how can you say there is
>> "space above the stack"?
>
> Sorry for being unclear.  I probably am getting terminology wrong.  What
> I'm trying to express is that if the stack top is in the last page of
> its last-level page table (which may be the last possible page, but
> that's not really the interesting case), vdso_addr returns an address
> below mm->start_stack.

It seems like this is avoidable, then? From your example, it seems
like we lose the separated randomization in this case, but we don't
need to? Do you have a suggestion for what could be done to fix this?

-Kees

>
> If you do a lot of execs with the following debug patch applied,
> you should see occasional prints like:
>
> got addr 0x7f9a2ba16000, asked 0x7fffa7bff000, start_stack=0x7fffa7bffc96
> got addr 0x7f3877ff1000, asked 0x7fffd9bff000, start_stack=0x7fffd9bffc96
> got addr 0x7f96e3637000, asked 0x7ffff39ff000, start_stack=0x7ffff39ffc96
> got addr 0x7fb70588d000, asked 0x7fff271ff000, start_stack=0x7fff271ffc96
> got addr 0x7f7957171000, asked 0x7fff71dff000, start_stack=0x7fff71dffc96
>
> Hopefully this better illustrates.
>
> diff --git a/arch/x86/vdso/vma.c b/arch/x86/vdso/vma.c
> index 1ad102613127..06c51329d1b3 100644
> --- a/arch/x86/vdso/vma.c
> +++ b/arch/x86/vdso/vma.c
> @@ -157,15 +157,17 @@ static int setup_additional_pages(struct linux_binprm *bprm,
>                                   unsigned size)
>  {
>         struct mm_struct *mm = current->mm;
> -       unsigned long addr;
> +       unsigned long addr, hint;
>         int ret;
>
>         if (!vdso_enabled)
>                 return 0;
>
>         down_write(&mm->mmap_sem);
> -       addr = vdso_addr(mm->start_stack, size);
> -       addr = get_unmapped_area(NULL, addr, size, 0, 0);
> +       hint = vdso_addr(mm->start_stack, size);
> +       addr = get_unmapped_area(NULL, hint, size, 0, 0);
> +       if (addr != hint)
> +               pr_info("got addr 0x%lx, asked 0x%lx\n", addr, hint);
>         if (IS_ERR_VALUE(addr)) {
>                 ret = addr;
>                 goto up_fail;
>



-- 
Kees Cook
Chrome OS Security

Re: randomized placement of x86_64 vdso

[Andy Lutomirski]

On Wed, Apr 30, 2014 at 10:47 AM, Kees Cook <keescook@google.com> wrote:
> On Wed, Apr 23, 2014 at 10:06 AM, Nathan Lynch <Nathan_Lynch@mentor.com> wrote:
>> On 04/23/2014 11:30 AM, H. Peter Anvin wrote:
>>> On 04/21/2014 09:52 AM, Nathan Lynch wrote:
>>>> Hi x86/vdso people,
>>>>
>>>> I've been working on adding a vDSO to 32-bit ARM, and Kees suggested I
>>>> look at x86_64's algorithm for placing the vDSO at a randomized offset
>>>> above the stack VMA.  I found that when the stack top occupies the
>>>> last slot in the PTE (is that the right term?), the vdso_addr routine
>>>> returns an address below mm->start_stack, equivalent to
>>>> (mm->start_stack & PAGE_MASK).  For instance if mm->start_stack is
>>>> 0x7fff3ffffc96, vdso_addr returns 0x7fff3ffff000.
>>>>
>>>> Since the address returned is always already occupied by the stack,
>>>> get_unmapped_area detects the collision and falls back to
>>>> vm_unmapped_area.  This results in the vdso being placed in the
>>>> address space next to libraries etc.  While this is generally
>>>> unnoticeable and doesn't break anything, it does mean that the vdso is
>>>> placed below the stack when there is actually room above the stack.
>>>> To me it also seems uncomfortably close to placing the vdso in the way
>>>> of downward expansion of the stack.
>>>>
>>>> I don't have a patch because I'm not sure what the algorithm should
>>>> be, but thought I would bring it up as vdso_addr doesn't seem to be
>>>> behaving as intended in all cases.
>>>>
>>>
>>> If the stack occupies the last possible page, how can you say there is
>>> "space above the stack"?
>>
>> Sorry for being unclear.  I probably am getting terminology wrong.  What
>> I'm trying to express is that if the stack top is in the last page of
>> its last-level page table (which may be the last possible page, but
>> that's not really the interesting case), vdso_addr returns an address
>> below mm->start_stack.
>
> It seems like this is avoidable, then? From your example, it seems
> like we lose the separated randomization in this case, but we don't
> need to? Do you have a suggestion for what could be done to fix this?

I don't understand why the vDSO should be special here.  Either the
standard logic for randomizing the placement of DSOs is good, in which
case it should be good for the vDSO too, or I think we should fix it
for everything.

--Andy

Re: randomized placement of x86_64 vdso

[Kees Cook]

On Wed, Apr 30, 2014 at 10:52 AM, Andy Lutomirski <luto@amacapital.net> wrote:
> On Wed, Apr 30, 2014 at 10:47 AM, Kees Cook <keescook@google.com> wrote:
>> On Wed, Apr 23, 2014 at 10:06 AM, Nathan Lynch <Nathan_Lynch@mentor.com> wrote:
>>> On 04/23/2014 11:30 AM, H. Peter Anvin wrote:
>>>> On 04/21/2014 09:52 AM, Nathan Lynch wrote:
>>>>> Hi x86/vdso people,
>>>>>
>>>>> I've been working on adding a vDSO to 32-bit ARM, and Kees suggested I
>>>>> look at x86_64's algorithm for placing the vDSO at a randomized offset
>>>>> above the stack VMA.  I found that when the stack top occupies the
>>>>> last slot in the PTE (is that the right term?), the vdso_addr routine
>>>>> returns an address below mm->start_stack, equivalent to
>>>>> (mm->start_stack & PAGE_MASK).  For instance if mm->start_stack is
>>>>> 0x7fff3ffffc96, vdso_addr returns 0x7fff3ffff000.
>>>>>
>>>>> Since the address returned is always already occupied by the stack,
>>>>> get_unmapped_area detects the collision and falls back to
>>>>> vm_unmapped_area.  This results in the vdso being placed in the
>>>>> address space next to libraries etc.  While this is generally
>>>>> unnoticeable and doesn't break anything, it does mean that the vdso is
>>>>> placed below the stack when there is actually room above the stack.
>>>>> To me it also seems uncomfortably close to placing the vdso in the way
>>>>> of downward expansion of the stack.
>>>>>
>>>>> I don't have a patch because I'm not sure what the algorithm should
>>>>> be, but thought I would bring it up as vdso_addr doesn't seem to be
>>>>> behaving as intended in all cases.
>>>>>
>>>>
>>>> If the stack occupies the last possible page, how can you say there is
>>>> "space above the stack"?
>>>
>>> Sorry for being unclear.  I probably am getting terminology wrong.  What
>>> I'm trying to express is that if the stack top is in the last page of
>>> its last-level page table (which may be the last possible page, but
>>> that's not really the interesting case), vdso_addr returns an address
>>> below mm->start_stack.
>>
>> It seems like this is avoidable, then? From your example, it seems
>> like we lose the separated randomization in this case, but we don't
>> need to? Do you have a suggestion for what could be done to fix this?
>
> I don't understand why the vDSO should be special here.  Either the
> standard logic for randomizing the placement of DSOs is good, in which
> case it should be good for the vDSO too, or I think we should fix it
> for everything.

The issue is specific to the vdso randomizing-near-the-stack code;
regular mmap randomization is operating correctly. The reason for
randomizing stack, vdso, and mmap separately is to avoid correlation
of leaked offsets in one to the other regions.

-Kees

-- 
Kees Cook
Chrome OS Security

Re: randomized placement of x86_64 vdso

[Andy Lutomirski]

On Wed, Apr 30, 2014 at 11:13 AM, Kees Cook <keescook@google.com> wrote:
> On Wed, Apr 30, 2014 at 10:52 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>> On Wed, Apr 30, 2014 at 10:47 AM, Kees Cook <keescook@google.com> wrote:
>>> On Wed, Apr 23, 2014 at 10:06 AM, Nathan Lynch <Nathan_Lynch@mentor.com> wrote:
>>>> On 04/23/2014 11:30 AM, H. Peter Anvin wrote:
>>>>> On 04/21/2014 09:52 AM, Nathan Lynch wrote:
>>>>>> Hi x86/vdso people,
>>>>>>
>>>>>> I've been working on adding a vDSO to 32-bit ARM, and Kees suggested I
>>>>>> look at x86_64's algorithm for placing the vDSO at a randomized offset
>>>>>> above the stack VMA.  I found that when the stack top occupies the
>>>>>> last slot in the PTE (is that the right term?), the vdso_addr routine
>>>>>> returns an address below mm->start_stack, equivalent to
>>>>>> (mm->start_stack & PAGE_MASK).  For instance if mm->start_stack is
>>>>>> 0x7fff3ffffc96, vdso_addr returns 0x7fff3ffff000.
>>>>>>
>>>>>> Since the address returned is always already occupied by the stack,
>>>>>> get_unmapped_area detects the collision and falls back to
>>>>>> vm_unmapped_area.  This results in the vdso being placed in the
>>>>>> address space next to libraries etc.  While this is generally
>>>>>> unnoticeable and doesn't break anything, it does mean that the vdso is
>>>>>> placed below the stack when there is actually room above the stack.
>>>>>> To me it also seems uncomfortably close to placing the vdso in the way
>>>>>> of downward expansion of the stack.
>>>>>>
>>>>>> I don't have a patch because I'm not sure what the algorithm should
>>>>>> be, but thought I would bring it up as vdso_addr doesn't seem to be
>>>>>> behaving as intended in all cases.
>>>>>>
>>>>>
>>>>> If the stack occupies the last possible page, how can you say there is
>>>>> "space above the stack"?
>>>>
>>>> Sorry for being unclear.  I probably am getting terminology wrong.  What
>>>> I'm trying to express is that if the stack top is in the last page of
>>>> its last-level page table (which may be the last possible page, but
>>>> that's not really the interesting case), vdso_addr returns an address
>>>> below mm->start_stack.
>>>
>>> It seems like this is avoidable, then? From your example, it seems
>>> like we lose the separated randomization in this case, but we don't
>>> need to? Do you have a suggestion for what could be done to fix this?
>>
>> I don't understand why the vDSO should be special here.  Either the
>> standard logic for randomizing the placement of DSOs is good, in which
>> case it should be good for the vDSO too, or I think we should fix it
>> for everything.
>
> The issue is specific to the vdso randomizing-near-the-stack code;
> regular mmap randomization is operating correctly. The reason for
> randomizing stack, vdso, and mmap separately is to avoid correlation
> of leaked offsets in one to the other regions.

I understand why the offset from the stack to the vDSO should be
randomized, and why the offset from the stack to, say, glibc should be
randomized.  What I don't get is why the offset from glibc to the vDSO
should be randomized but the offset from glibc to openssl should be
deterministic.  Or am I misunderstanding?

--Andy

Re: randomized placement of x86_64 vdso

[Kees Cook]

On Wed, Apr 30, 2014 at 11:30 AM, Andy Lutomirski <luto@amacapital.net> wrote:
> On Wed, Apr 30, 2014 at 11:13 AM, Kees Cook <keescook@google.com> wrote:
>> On Wed, Apr 30, 2014 at 10:52 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>>> On Wed, Apr 30, 2014 at 10:47 AM, Kees Cook <keescook@google.com> wrote:
>>>> On Wed, Apr 23, 2014 at 10:06 AM, Nathan Lynch <Nathan_Lynch@mentor.com> wrote:
>>>>> On 04/23/2014 11:30 AM, H. Peter Anvin wrote:
>>>>>> On 04/21/2014 09:52 AM, Nathan Lynch wrote:
>>>>>>> Hi x86/vdso people,
>>>>>>>
>>>>>>> I've been working on adding a vDSO to 32-bit ARM, and Kees suggested I
>>>>>>> look at x86_64's algorithm for placing the vDSO at a randomized offset
>>>>>>> above the stack VMA.  I found that when the stack top occupies the
>>>>>>> last slot in the PTE (is that the right term?), the vdso_addr routine
>>>>>>> returns an address below mm->start_stack, equivalent to
>>>>>>> (mm->start_stack & PAGE_MASK).  For instance if mm->start_stack is
>>>>>>> 0x7fff3ffffc96, vdso_addr returns 0x7fff3ffff000.
>>>>>>>
>>>>>>> Since the address returned is always already occupied by the stack,
>>>>>>> get_unmapped_area detects the collision and falls back to
>>>>>>> vm_unmapped_area.  This results in the vdso being placed in the
>>>>>>> address space next to libraries etc.  While this is generally
>>>>>>> unnoticeable and doesn't break anything, it does mean that the vdso is
>>>>>>> placed below the stack when there is actually room above the stack.
>>>>>>> To me it also seems uncomfortably close to placing the vdso in the way
>>>>>>> of downward expansion of the stack.
>>>>>>>
>>>>>>> I don't have a patch because I'm not sure what the algorithm should
>>>>>>> be, but thought I would bring it up as vdso_addr doesn't seem to be
>>>>>>> behaving as intended in all cases.
>>>>>>>
>>>>>>
>>>>>> If the stack occupies the last possible page, how can you say there is
>>>>>> "space above the stack"?
>>>>>
>>>>> Sorry for being unclear.  I probably am getting terminology wrong.  What
>>>>> I'm trying to express is that if the stack top is in the last page of
>>>>> its last-level page table (which may be the last possible page, but
>>>>> that's not really the interesting case), vdso_addr returns an address
>>>>> below mm->start_stack.
>>>>
>>>> It seems like this is avoidable, then? From your example, it seems
>>>> like we lose the separated randomization in this case, but we don't
>>>> need to? Do you have a suggestion for what could be done to fix this?
>>>
>>> I don't understand why the vDSO should be special here.  Either the
>>> standard logic for randomizing the placement of DSOs is good, in which
>>> case it should be good for the vDSO too, or I think we should fix it
>>> for everything.
>>
>> The issue is specific to the vdso randomizing-near-the-stack code;
>> regular mmap randomization is operating correctly. The reason for
>> randomizing stack, vdso, and mmap separately is to avoid correlation
>> of leaked offsets in one to the other regions.
>
> I understand why the offset from the stack to the vDSO should be
> randomized, and why the offset from the stack to, say, glibc should be
> randomized.  What I don't get is why the offset from glibc to the vDSO
> should be randomized but the offset from glibc to openssl should be
> deterministic.  Or am I misunderstanding?

Sure, I agree. In a perfect world, every DSO would have unrelated
offsets. Ignoring that for the moment, my take on this is that stack,
vDSO, and mmap have their locations tracked/exposed separately (SP
register, AUXV, and syscall return respectively). As such, they should
have separate randomization to avoid having any one method lead to the
discovery of all the offsets (i.e. the vDSO location is communicated
to the process in a different way than other DSOs that come into the
process via the mmap syscall).

As for per-mmap randomization, I would love this, but no one has
proposed algorithms that aren't worse than single base offset
randomization. This is especially true under 32-bit where attempting
to balance the randomization against fragmentation just leads to
non-random locations under certain situations/processes/DSOs, etc. It
could be worth revisiting this for 64-bit, but I feel like it's not a
high priority.

-Kees

-- 
Kees Cook
Chrome OS Security

Re: randomized placement of x86_64 vdso

[Andy Lutomirski]

On Wed, Apr 30, 2014 at 1:05 PM, Kees Cook <keescook@google.com> wrote:
> On Wed, Apr 30, 2014 at 11:30 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>> On Wed, Apr 30, 2014 at 11:13 AM, Kees Cook <keescook@google.com> wrote:
>>> On Wed, Apr 30, 2014 at 10:52 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>>>> On Wed, Apr 30, 2014 at 10:47 AM, Kees Cook <keescook@google.com> wrote:
>>>>> On Wed, Apr 23, 2014 at 10:06 AM, Nathan Lynch <Nathan_Lynch@mentor.com> wrote:
>>>>>> On 04/23/2014 11:30 AM, H. Peter Anvin wrote:
>>>>>>> On 04/21/2014 09:52 AM, Nathan Lynch wrote:
>>>>>>>> Hi x86/vdso people,
>>>>>>>>
>>>>>>>> I've been working on adding a vDSO to 32-bit ARM, and Kees suggested I
>>>>>>>> look at x86_64's algorithm for placing the vDSO at a randomized offset
>>>>>>>> above the stack VMA.  I found that when the stack top occupies the
>>>>>>>> last slot in the PTE (is that the right term?), the vdso_addr routine
>>>>>>>> returns an address below mm->start_stack, equivalent to
>>>>>>>> (mm->start_stack & PAGE_MASK).  For instance if mm->start_stack is
>>>>>>>> 0x7fff3ffffc96, vdso_addr returns 0x7fff3ffff000.
>>>>>>>>
>>>>>>>> Since the address returned is always already occupied by the stack,
>>>>>>>> get_unmapped_area detects the collision and falls back to
>>>>>>>> vm_unmapped_area.  This results in the vdso being placed in the
>>>>>>>> address space next to libraries etc.  While this is generally
>>>>>>>> unnoticeable and doesn't break anything, it does mean that the vdso is
>>>>>>>> placed below the stack when there is actually room above the stack.
>>>>>>>> To me it also seems uncomfortably close to placing the vdso in the way
>>>>>>>> of downward expansion of the stack.
>>>>>>>>
>>>>>>>> I don't have a patch because I'm not sure what the algorithm should
>>>>>>>> be, but thought I would bring it up as vdso_addr doesn't seem to be
>>>>>>>> behaving as intended in all cases.
>>>>>>>>
>>>>>>>
>>>>>>> If the stack occupies the last possible page, how can you say there is
>>>>>>> "space above the stack"?
>>>>>>
>>>>>> Sorry for being unclear.  I probably am getting terminology wrong.  What
>>>>>> I'm trying to express is that if the stack top is in the last page of
>>>>>> its last-level page table (which may be the last possible page, but
>>>>>> that's not really the interesting case), vdso_addr returns an address
>>>>>> below mm->start_stack.
>>>>>
>>>>> It seems like this is avoidable, then? From your example, it seems
>>>>> like we lose the separated randomization in this case, but we don't
>>>>> need to? Do you have a suggestion for what could be done to fix this?
>>>>
>>>> I don't understand why the vDSO should be special here.  Either the
>>>> standard logic for randomizing the placement of DSOs is good, in which
>>>> case it should be good for the vDSO too, or I think we should fix it
>>>> for everything.
>>>
>>> The issue is specific to the vdso randomizing-near-the-stack code;
>>> regular mmap randomization is operating correctly. The reason for
>>> randomizing stack, vdso, and mmap separately is to avoid correlation
>>> of leaked offsets in one to the other regions.
>>
>> I understand why the offset from the stack to the vDSO should be
>> randomized, and why the offset from the stack to, say, glibc should be
>> randomized.  What I don't get is why the offset from glibc to the vDSO
>> should be randomized but the offset from glibc to openssl should be
>> deterministic.  Or am I misunderstanding?
>
> Sure, I agree. In a perfect world, every DSO would have unrelated
> offsets. Ignoring that for the moment, my take on this is that stack,
> vDSO, and mmap have their locations tracked/exposed separately (SP
> register, AUXV, and syscall return respectively). As such, they should
> have separate randomization to avoid having any one method lead to the
> discovery of all the offsets (i.e. the vDSO location is communicated
> to the process in a different way than other DSOs that come into the
> process via the mmap syscall).
>
> As for per-mmap randomization, I would love this, but no one has
> proposed algorithms that aren't worse than single base offset
> randomization. This is especially true under 32-bit where attempting
> to balance the randomization against fragmentation just leads to
> non-random locations under certain situations/processes/DSOs, etc. It
> could be worth revisiting this for 64-bit, but I feel like it's not a
> high priority.

I think that the offending vDSO randomization code doesn't even run on 32-bit.

If anyone wants to fix this up (do something intelligent on 32-bit and
do something more intelligent on 64-bit), it might be easier if based
on my vdso cleanups
(https://git.kernel.org/cgit/linux/kernel/git/luto/linux.git/log/?h=vdso/cleanups).
 The code in question is a lot less twisted with those patches
applied.

--Andy

-- 
Andy Lutomirski
AMA Capital Management, LLC