Inexplicable PROT_EXEC flag set on mmap callback

[kennethadammiller@gmail.com (Kenneth Adam Miller)]

In fact, it's being set to 0xff exactly, not just the VM_EXEC flag being
set. vma->vm_flags & VM_EXEC resolves true, because vma->vm_flags is 0xff

On Thu, Jan 14, 2016 at 11:04 AM, Kenneth Adam Miller <
kennethadammiller@gmail.com> wrote:

> I have a custom drive and userland program pair that I'm using for a very
> special use case at my workplace where we are mapping specific physical
> address ranges into userland memory with a mmap callback. Everything works
> together well with a C userland program that calls into our driver's ioctl
> and mmap definitions, but for our case we are using an alternative systems
> language just for the userland program. That mmap call is failing (properly
> as we want) out from the driver's mmap implementation due to the fact that
> the vm_flags have the VM_EXEC flag set. We do not want users to be able to
> map the memory range as executable, so the driver should check for this as
> it does. The issue is in the fact that somewhere between where mmap is
> called and when the parameters are given to the driver, the vma->vm_flags
> are being set to 255. I've manually checked the values being given to the
> mmap call in our non-C binary, and they are *equivalent* in value to that
> of the C program.
>
> My question is, is there anything that can cause the vma->vm_flags to be
> changed in the trip between when the user land program calls mmap and when
> control is delivered to the mmap callback?
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20160114/3d094999/attachment.html