Regarding mmap synchronization.

Regarding mmap synchronization.

[mindentropy]

Hi,

  I have mmaped a circular queue buffer created in the kernel.  Now I want to 
mmap the read and write pointers in the queue but I am not sure how to 
synchronize the access of the pointers between the kernel and userspace(while 
checking sizes for overflow and underflow). How should I go about doing this?

Thanks.

Regarding mmap synchronization.

[Dave Hylands]

Hi,

On Sun, Sep 18, 2011 at 11:02 AM, mindentropy <mindentropy@gmail.com> wrote:
> Hi,
>
> ?I have mmaped a circular queue buffer created in the kernel. ?Now I want to
> mmap the read and write pointers in the queue but I am not sure how to
> synchronize the access of the pointers between the kernel and userspace(while
> checking sizes for overflow and underflow). How should I go about doing this?

The way I normally deal with this is to use 2 indicies, a get index
and a put index. One of the indicies if only ever written by kernel
space, and the other is only ever written by user space.

Normally, I would arrange the get and put pointers to be in uncached
memory. If they're in cached memory, I would ensure that they're on
different cache lines.

You make the circular buffer be a power of 2 in size, and you
determine the number of items in the queue by subtracting the get
index from the put index. To retrieve  the items from the queue, you
apply a mask.

Lets say you're using 32-bit indicies and you have a max of 512 items
in the queue. You would mask the index with ~0x1FF in order to
determine the real index value.

If the items in the circular buffer are in cached memory, then I
normally try to make each item be an exact multiple of the cache line
size. I find using uncached memory is generally better for this type
of thing (the accesses are slower, but may be faster after accounting
for the cache management).

If you want to communicate in both directions, then you create a
separate queue for each direction.

-- 
Dave Hylands
Shuswap, BC, Canada
http://www.davehylands.com

Regarding mmap synchronization.

[mindentropy]

On Monday 19 Sep 2011 12:19:29 AM Dave Hylands wrote:

> The way I normally deal with this is to use 2 indicies, a get index
> and a put index. One of the indicies if only ever written by kernel
> space, and the other is only ever written by user space.
>
That is the setup I have now.

> 
> You make the circular buffer be a power of 2 in size, and you
> determine the number of items in the queue by subtracting the get
> index from the put index. 
I am worried about the subtraction. Is it safe to subtract when the put index 
is in the process of incrementing by the kernel?
My queue size is always a power of two and avoid the modulus operation by '&' 
with (2^n)-1.

> If the items in the circular buffer are in cached memory, then I
> normally try to make each item be an exact multiple of the cache line
> size. I find using uncached memory is generally better for this type
> of thing (the accesses are slower, but may be faster after accounting
> for the cache management).
> 
I am having a chunked buffer. So the queue items are memory chunks ranging from 
PAGE_SIZE*n to PAGE_SIZE*1. So I have a index to the chunk, index to item 
inside the chunk for read and write. For the user its invisible and sees it as 
one big chunk of memory with a read and a write index.

Regarding mmap synchronization.

[Dave Hylands]

HI,

On Sun, Sep 18, 2011 at 12:12 PM, mindentropy <mindentropy@gmail.com> wrote:
> On Monday 19 Sep 2011 12:19:29 AM Dave Hylands wrote:
>
>> The way I normally deal with this is to use 2 indicies, a get index
>> and a put index. One of the indicies if only ever written by kernel
>> space, and the other is only ever written by user space.
>>
> That is the setup I have now.
>
>>
>> You make the circular buffer be a power of 2 in size, and you
>> determine the number of items in the queue by subtracting the get
>> index from the put index.
> I am worried about the subtraction. Is it safe to subtract when the put index
> is in the process of incrementing by the kernel?
> My queue size is always a power of two and avoid the modulus operation by '&'
> with (2^n)-1.

The worst case that happens is that you miss the increment by the
other side. This means that you either miss a new item added onto the
queue (if you're reading) or you miss new space added to the queue (if
you're writing). Either case is non-fatal. You don't corrupt anything.
You'll pick up the new item/space the next time you check.

>> If the items in the circular buffer are in cached memory, then I
>> normally try to make each item be an exact multiple of the cache line
>> size. I find using uncached memory is generally better for this type
>> of thing (the accesses are slower, but may be faster after accounting
>> for the cache management).
>>
> I am having a chunked buffer. So the queue items are memory chunks ranging from
> PAGE_SIZE*n to PAGE_SIZE*1. So I have a index to the chunk, index to item
> inside the chunk for read and write. For the user its invisible and sees it as
> one big chunk of memory with a read and a write index.

Seems reasonable. The big thingis to ensure that when you add an item
to the queue, that the item is written fully before incrementing the
index. If both the memory for the items and the memory for the index
is uncached (and declared volatile), then this should be fine.

-- 
Dave Hylands
Shuswap, BC, Canada
http://www.davehylands.com