[Qemu-devel] DIRTY_MEMORY_BLOCK_SIZE;

[Qemu-devel] DIRTY_MEMORY_BLOCK_SIZE;

[ali saeedi]

Hello
what does 'DIRTY_MEMORY_BLOCK_SIZE' mean?
is it the number of words in a block? or number of pages in a block? or
number of bytes in a block?
thanks a lot

Re: [Qemu-devel] DIRTY_MEMORY_BLOCK_SIZE;

[Dr. David Alan Gilbert]

* ali saeedi (ali.saeedi56@gmail.com) wrote:
> Hello
> what does 'DIRTY_MEMORY_BLOCK_SIZE' mean?
> is it the number of words in a block? or number of pages in a block? or
> number of bytes in a block?
> thanks a lot

(cc'ing Stefan)
I think that DIRTY_MEMORY_BLOCK_SIZE is the number of TARGET_PAGEs
within one DIRTY_MEMORY_BLOCK
So with the common 4k target page that's 4k*256k*8=8GB/dirty memory
block - note these are just the size of structure sin qemu, it's still
got the ranularity ot mark individual target pages as dirty.

Dave

--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] DIRTY_MEMORY_BLOCK_SIZE;

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1769 bytes --]

On Fri, Jun 30, 2017 at 12:26:10PM +0100, Dr. David Alan Gilbert wrote:
> * ali saeedi (ali.saeedi56@gmail.com) wrote:
> > Hello
> > what does 'DIRTY_MEMORY_BLOCK_SIZE' mean?
> > is it the number of words in a block? or number of pages in a block? or
> > number of bytes in a block?
> > thanks a lot
> 
> (cc'ing Stefan)
> I think that DIRTY_MEMORY_BLOCK_SIZE is the number of TARGET_PAGEs
> within one DIRTY_MEMORY_BLOCK
> So with the common 4k target page that's 4k*256k*8=8GB/dirty memory
> block - note these are just the size of structure sin qemu, it's still
> got the ranularity ot mark individual target pages as dirty.

Right, the calculation is shown in the comment above the code:

 *   rcu_read_lock();
 *
 *   DirtyMemoryBlocks *blocks =
 *       atomic_rcu_read(&ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION]);
 *
 *   ram_addr_t idx = (addr >> TARGET_PAGE_BITS) / DIRTY_MEMORY_BLOCK_SIZE;
 *   unsigned long *block = blocks.blocks[idx];
 *   ...access block bitmap...
 *
 *   rcu_read_unlock();

Rather than focussing on DIRTY_MEMORY_BLOCK_SIZE, make sure you
understand how DirtyMemoryBlocks works.  It is an array of bitmap
pointers.

Instead of directly indexing into a single huge dirty memory bitmap,
QEMU divides the dirty memory bitmap into fixed-sized chunks.  Each
chunk covers DIRTY_MEMORY_BLOCK_SIZE pages.

The reason for this layer of indirection is so that the dirty memory
bitmap can be accessed without taking a traditional lock (just RCU) and
also supports memory hotplug.

Without indirection it would be difficult to grow the bitmap while other
threads are writing to it.  Thanks to the indirection, it's possible to
allocate new chunks and continue using the old chunks when growth
occurs.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [Qemu-devel] DIRTY_MEMORY_BLOCK_SIZE;

[Dr. David Alan Gilbert]

* Stefan Hajnoczi (stefanha@redhat.com) wrote:
> On Fri, Jun 30, 2017 at 12:26:10PM +0100, Dr. David Alan Gilbert wrote:
> > * ali saeedi (ali.saeedi56@gmail.com) wrote:
> > > Hello
> > > what does 'DIRTY_MEMORY_BLOCK_SIZE' mean?
> > > is it the number of words in a block? or number of pages in a block? or
> > > number of bytes in a block?
> > > thanks a lot
> > 
> > (cc'ing Stefan)
> > I think that DIRTY_MEMORY_BLOCK_SIZE is the number of TARGET_PAGEs
> > within one DIRTY_MEMORY_BLOCK
> > So with the common 4k target page that's 4k*256k*8=8GB/dirty memory
> > block - note these are just the size of structure sin qemu, it's still
> > got the ranularity ot mark individual target pages as dirty.
> 
> Right, the calculation is shown in the comment above the code:
> 
>  *   rcu_read_lock();
>  *
>  *   DirtyMemoryBlocks *blocks =
>  *       atomic_rcu_read(&ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION]);
>  *
>  *   ram_addr_t idx = (addr >> TARGET_PAGE_BITS) / DIRTY_MEMORY_BLOCK_SIZE;
>  *   unsigned long *block = blocks.blocks[idx];
>  *   ...access block bitmap...
>  *
>  *   rcu_read_unlock();
> 
> Rather than focussing on DIRTY_MEMORY_BLOCK_SIZE, make sure you
> understand how DirtyMemoryBlocks works.  It is an array of bitmap
> pointers.
> 
> Instead of directly indexing into a single huge dirty memory bitmap,
> QEMU divides the dirty memory bitmap into fixed-sized chunks.  Each
> chunk covers DIRTY_MEMORY_BLOCK_SIZE pages.
> 
> The reason for this layer of indirection is so that the dirty memory
> bitmap can be accessed without taking a traditional lock (just RCU) and
> also supports memory hotplug.
> 
> Without indirection it would be difficult to grow the bitmap while other
> threads are writing to it.  Thanks to the indirection, it's possible to
> allocate new chunks and continue using the old chunks when growth
> occurs.

I guess this works like the old non-chunk version, in that there's no
direct correspondence between DirtyMemoryBlocks and RAMBlock's - i.e.
one RAMBlock might span two DirtyMemoryBlocks even if it's smaller
than DIRTY_MEMORY_BLOCK_SIZE.

Dave


--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] DIRTY_MEMORY_BLOCK_SIZE;

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 2339 bytes --]

On Wed, Jul 05, 2017 at 05:06:41PM +0100, Dr. David Alan Gilbert wrote:
> * Stefan Hajnoczi (stefanha@redhat.com) wrote:
> > On Fri, Jun 30, 2017 at 12:26:10PM +0100, Dr. David Alan Gilbert wrote:
> > > * ali saeedi (ali.saeedi56@gmail.com) wrote:
> > > > Hello
> > > > what does 'DIRTY_MEMORY_BLOCK_SIZE' mean?
> > > > is it the number of words in a block? or number of pages in a block? or
> > > > number of bytes in a block?
> > > > thanks a lot
> > > 
> > > (cc'ing Stefan)
> > > I think that DIRTY_MEMORY_BLOCK_SIZE is the number of TARGET_PAGEs
> > > within one DIRTY_MEMORY_BLOCK
> > > So with the common 4k target page that's 4k*256k*8=8GB/dirty memory
> > > block - note these are just the size of structure sin qemu, it's still
> > > got the ranularity ot mark individual target pages as dirty.
> > 
> > Right, the calculation is shown in the comment above the code:
> > 
> >  *   rcu_read_lock();
> >  *
> >  *   DirtyMemoryBlocks *blocks =
> >  *       atomic_rcu_read(&ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION]);
> >  *
> >  *   ram_addr_t idx = (addr >> TARGET_PAGE_BITS) / DIRTY_MEMORY_BLOCK_SIZE;
> >  *   unsigned long *block = blocks.blocks[idx];
> >  *   ...access block bitmap...
> >  *
> >  *   rcu_read_unlock();
> > 
> > Rather than focussing on DIRTY_MEMORY_BLOCK_SIZE, make sure you
> > understand how DirtyMemoryBlocks works.  It is an array of bitmap
> > pointers.
> > 
> > Instead of directly indexing into a single huge dirty memory bitmap,
> > QEMU divides the dirty memory bitmap into fixed-sized chunks.  Each
> > chunk covers DIRTY_MEMORY_BLOCK_SIZE pages.
> > 
> > The reason for this layer of indirection is so that the dirty memory
> > bitmap can be accessed without taking a traditional lock (just RCU) and
> > also supports memory hotplug.
> > 
> > Without indirection it would be difficult to grow the bitmap while other
> > threads are writing to it.  Thanks to the indirection, it's possible to
> > allocate new chunks and continue using the old chunks when growth
> > occurs.
> 
> I guess this works like the old non-chunk version, in that there's no
> direct correspondence between DirtyMemoryBlocks and RAMBlock's - i.e.
> one RAMBlock might span two DirtyMemoryBlocks even if it's smaller
> than DIRTY_MEMORY_BLOCK_SIZE.

Yes.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]