Re: [PATCH 00/12] [RFC] x86: Memory Protection Keys

Re: [PATCH 00/12] [RFC] x86: Memory Protection Keys

[Christian Borntraeger]

Am 07.05.2015 um 20:09 schrieb Dave Hansen:
> On 05/07/2015 10:57 AM, Ingo Molnar wrote:
>>>> There are two new instructions (RDPKRU/WRPKRU) for reading and 
>>>> writing to the new register.  The feature is only available in 
>>>> 64-bit mode, even though there is theoretically space in the PAE 
>>>> PTEs.  These permissions are enforced on data access only and have 
>>>> no effect on instruction fetches.
>> So I'm wondering what the primary usecases are for this feature?
>> Could you outline applications/workloads/libraries that would
>> benefit from this?
> 
> There are lots of things that folks would _like_ to mprotect(), but end
> up not being feasible because of the overhead of going and mucking with
> thousands of PTEs and shooting down remote TLBs every time you want to
> change protections.

These protection bits would need to be cached in TLBs as well, no?
So the saving would come by switching the PKRU instead of the page bits.

This all looks like s390 storage keys (with the key in pagetables instead
of a dedicated place). There we also have 16 values for the key and 4 bits 
in the PSW that describe the thread local key both are matched.
There is an additional field F (fetch protection) that decides, if the
key value is used for stores or for stores+fetches.

Re: [PATCH 00/12] [RFC] x86: Memory Protection Keys

[Dave Hansen]

On 05/07/2015 12:22 PM, Christian Borntraeger wrote:
> Am 07.05.2015 um 20:09 schrieb Dave Hansen:
>> On 05/07/2015 10:57 AM, Ingo Molnar wrote:
>>>>> There are two new instructions (RDPKRU/WRPKRU) for reading and 
>>>>> writing to the new register.  The feature is only available in 
>>>>> 64-bit mode, even though there is theoretically space in the PAE 
>>>>> PTEs.  These permissions are enforced on data access only and have 
>>>>> no effect on instruction fetches.
>>> So I'm wondering what the primary usecases are for this feature?
>>> Could you outline applications/workloads/libraries that would
>>> benefit from this?
>>
>> There are lots of things that folks would _like_ to mprotect(), but end
>> up not being feasible because of the overhead of going and mucking with
>> thousands of PTEs and shooting down remote TLBs every time you want to
>> change protections.
> 
> These protection bits would need to be cached in TLBs as well, no?

Yes, they are cached in the TLBs.  It's actually explicitly called out
in the documentation.

> So the saving would come by switching the PKRU instead of the page bits.

Right.

> This all looks like s390 storage keys (with the key in pagetables instead
> of a dedicated place). There we also have 16 values for the key and 4 bits 
> in the PSW that describe the thread local key both are matched.
> There is an additional field F (fetch protection) that decides, if the
> key value is used for stores or for stores+fetches.

OK, so a thread can only be in one domain at a time?

That's a bit different than x86 where each page can be in one protection
domain, but each CPU thread can independently enable/disable access to
each of the 16 protection domains.

Re: [PATCH 00/12] [RFC] x86: Memory Protection Keys

[Christian Borntraeger]

Am 07.05.2015 um 21:29 schrieb Dave Hansen:
> On 05/07/2015 12:22 PM, Christian Borntraeger wrote:
>> Am 07.05.2015 um 20:09 schrieb Dave Hansen:
>>> On 05/07/2015 10:57 AM, Ingo Molnar wrote:
>>>>>> There are two new instructions (RDPKRU/WRPKRU) for reading and 
>>>>>> writing to the new register.  The feature is only available in 
>>>>>> 64-bit mode, even though there is theoretically space in the PAE 
>>>>>> PTEs.  These permissions are enforced on data access only and have 
>>>>>> no effect on instruction fetches.
>>>> So I'm wondering what the primary usecases are for this feature?
>>>> Could you outline applications/workloads/libraries that would
>>>> benefit from this?
>>>
>>> There are lots of things that folks would _like_ to mprotect(), but end
>>> up not being feasible because of the overhead of going and mucking with
>>> thousands of PTEs and shooting down remote TLBs every time you want to
>>> change protections.
>>
>> These protection bits would need to be cached in TLBs as well, no?
> 
> Yes, they are cached in the TLBs.  It's actually explicitly called out
> in the documentation.
> 
>> So the saving would come by switching the PKRU instead of the page bits.
> 
> Right.
> 
>> This all looks like s390 storage keys (with the key in pagetables instead
>> of a dedicated place). There we also have 16 values for the key and 4 bits 
>> in the PSW that describe the thread local key both are matched.
>> There is an additional field F (fetch protection) that decides, if the
>> key value is used for stores or for stores+fetches.
> 
> OK, so a thread can only be in one domain at a time?

Via the PSW yes.
Actually the docs talk about access key, which is usually the PSW. There are
some instructions like MOVE WITH KEY that allow to specify the key for this
specific instruction. For compiled code these insructions are not used in 
Linux and I can not really see a way to implement that properly. Furthermore
enabling these key ops has other implications which are unwanted.

 
> That's a bit different than x86 where each page can be in one protection
> domain, but each CPU thread can independently enable/disable access to
> each of the 16 protection domains.
> 

Re: [PATCH 00/12] [RFC] x86: Memory Protection Keys

[Dave Hansen]

On 05/07/2015 12:45 PM, Christian Borntraeger wrote:
>>> >> This all looks like s390 storage keys (with the key in pagetables instead
>>> >> of a dedicated place). There we also have 16 values for the key and 4 bits 
>>> >> in the PSW that describe the thread local key both are matched.
>>> >> There is an additional field F (fetch protection) that decides, if the
>>> >> key value is used for stores or for stores+fetches.
>> > 
>> > OK, so a thread can only be in one domain at a time?
> Via the PSW yes.
> Actually the docs talk about access key, which is usually the PSW. There are
> some instructions like MOVE WITH KEY that allow to specify the key for this
> specific instruction. For compiled code these insructions are not used in 
> Linux and I can not really see a way to implement that properly. Furthermore
> enabling these key ops has other implications which are unwanted.

OK, so we have to basic operations that need to be done for
protection/storage/$FOO keys:

1. Assign a key (or set of keys) to a memory area
2. Have a thread request the access (read and/or write) to a set of
   areas be acquired or revoked.

For (2) on x86, we basically allow any combination of keys and r/w
permissions.  On s390, we would need to ensure that acces to only one
key was allowed at a time.

BTW, do the s390 keys affect instructions and data, or data only?

The x86 ones affect data only.

Re: [PATCH 00/12] [RFC] x86: Memory Protection Keys

[Christian Borntraeger]

Am 07.05.2015 um 21:49 schrieb Dave Hansen:
> On 05/07/2015 12:45 PM, Christian Borntraeger wrote:
>>>>>> This all looks like s390 storage keys (with the key in pagetables instead
>>>>>> of a dedicated place). There we also have 16 values for the key and 4 bits 
>>>>>> in the PSW that describe the thread local key both are matched.
>>>>>> There is an additional field F (fetch protection) that decides, if the
>>>>>> key value is used for stores or for stores+fetches.
>>>>
>>>> OK, so a thread can only be in one domain at a time?
>> Via the PSW yes.
>> Actually the docs talk about access key, which is usually the PSW. There are
>> some instructions like MOVE WITH KEY that allow to specify the key for this
>> specific instruction. For compiled code these insructions are not used in 
>> Linux and I can not really see a way to implement that properly. Furthermore
>> enabling these key ops has other implications which are unwanted.
> 
> OK, so we have to basic operations that need to be done for
> protection/storage/$FOO keys:
> 
> 1. Assign a key (or set of keys) to a memory area
> 2. Have a thread request the access (read and/or write) to a set of
>    areas be acquired or revoked.
> 
> For (2) on x86, we basically allow any combination of keys and r/w
> permissions.  On s390, we would need to ensure that acces to only one
> key was allowed at a time.
> 
> BTW, do the s390 keys affect instructions and data, or data only?

Both. In fact its also used for I/O. Maybe that also points out the
biggest difference. the storage key is a property of the physical page
frame (and not of the virtual page defined by the page tables).
So we cannot really use that for shared memory and then set different
protection keys in different mappings.

 
> The x86 ones affect data only.
>