From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50954) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1e1dTW-0004j7-Ae for qemu-devel@nongnu.org; Mon, 09 Oct 2017 15:12:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1e1dTS-0002rI-TX for qemu-devel@nongnu.org; Mon, 09 Oct 2017 15:12:26 -0400 Received: from mx1.redhat.com ([209.132.183.28]:50710) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1e1dTS-0002q0-Iw for qemu-devel@nongnu.org; Mon, 09 Oct 2017 15:12:22 -0400 Date: Mon, 9 Oct 2017 20:12:15 +0100 From: "Dr. David Alan Gilbert" Message-ID: <20171009191214.GA3481@work-vm> References: <1504081950-2528-1-git-send-email-peterx@redhat.com> <1504081950-2528-21-git-send-email-peterx@redhat.com> <20170922110542.GD2620@work-vm> <20170927100416.GA17044@pxdev.xzpeter.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170927100416.GA17044@pxdev.xzpeter.org> Subject: Re: [Qemu-devel] [RFC v2 20/33] migration: new message MIG_RP_MSG_RECV_BITMAP List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Peter Xu Cc: qemu-devel@nongnu.org, Laurent Vivier , "Daniel P . Berrange" , Alexey Perevalov , Juan Quintela , Andrea Arcangeli * Peter Xu (peterx@redhat.com) wrote: > On Fri, Sep 22, 2017 at 12:05:42PM +0100, Dr. David Alan Gilbert wrote: > > [...] > > > > diff --git a/migration/ram.c b/migration/ram.c > > > index 7e20097..5d938e3 100644 > > > --- a/migration/ram.c > > > +++ b/migration/ram.c > > > @@ -182,6 +182,70 @@ void ramblock_recv_bitmap_clear(RAMBlock *rb, void *host_addr) > > > clear_bit(ramblock_recv_bitmap_offset(host_addr, rb), rb->receivedmap); > > > } > > > > > > +#define RAMBLOCK_RECV_BITMAP_ENDING (0x0123456789abcdefULL) > > > + > > > +/* > > > + * Format: bitmap_size (8 bytes) + whole_bitmap (N bytes). > > > + * > > > + * Returns >0 if success with sent bytes, or <0 if error. > > > + */ > > > +int64_t ramblock_recv_bitmap_send(QEMUFile *file, > > > + const char *block_name) > > > +{ > > > + RAMBlock *block = qemu_ram_block_by_name(block_name); > > > + unsigned long *le_bitmap, nbits; > > > + uint64_t size; > > > + > > > + if (!block) { > > > + error_report("%s: invalid block name: %s", __func__, block_name); > > > + return -1; > > > + } > > > + > > > + nbits = block->used_length >> TARGET_PAGE_BITS; > > > + > > > + /* > > > + * Make sure the tmp bitmap buffer is big enough, e.g., on 32bit > > > + * machines we may need 4 more bytes for padding (see below > > > + * comment). So extend it a bit before hand. > > > + */ > > > + le_bitmap = bitmap_new(nbits + BITS_PER_LONG); > > > > I do worry what will happen on really huge RAMBlocks; the worst case is > > that this temporary bitmap is a few GB. > > IIUC the bitmap ratio is 32K, so the ramblock will be 1GB only if the > guest RAM region size is 1GB * 32K = 32TB. > > Then, can I just assume allocating (only) 1GB temporary memory for a > guest using more than 32TB memory not a problem? :-) > > I hope I didn't calculate it wrongly though. No, I think that's right; I was off a few bits. If they've got 32TB of RAM, then 1GB is probably no issue. Dave > > > > > + /* > > > + * Always use little endian when sending the bitmap. This is > > > + * required that when source and destination VMs are not using the > > > + * same endianess. (Note: big endian won't work.) > > > + */ > > > + bitmap_to_le(le_bitmap, block->receivedmap, nbits); > > > + > > > + /* Size of the bitmap, in bytes */ > > > + size = nbits / 8; > > > + > > > + /* > > > + * size is always aligned to 8 bytes for 64bit machines, but it > > > + * may not be true for 32bit machines. We need this padding to > > > + * make sure the migration can survive even between 32bit and > > > + * 64bit machines. > > > + */ > > > + size = ROUND_UP(size, 8); > > > + > > > + qemu_put_be64(file, size); > > > + qemu_put_buffer(file, (const uint8_t *)le_bitmap, size); > > > + /* > > > + * Mark as an end, in case the middle part is screwed up due to > > > + * some "misterious" reason. > > > + */ > > > + qemu_put_be64(file, RAMBLOCK_RECV_BITMAP_ENDING); > > > + qemu_fflush(file); > > > + > > > + free(le_bitmap); > > > + > > > + if (qemu_file_get_error(file)) { > > > + return qemu_file_get_error(file); > > > + } > > > + > > > + return size + sizeof(size); > > > +} > > > + > > > /* > > > * An outstanding page request, on the source, having been received > > > * and queued > > > @@ -2706,6 +2770,83 @@ static int ram_load(QEMUFile *f, void *opaque, int version_id) > > > return ret; > > > } > > > > > > +/* > > > + * Read the received bitmap, revert it as the initial dirty bitmap. > > > + * This is only used when the postcopy migration is paused but wants > > > + * to resume from a middle point. > > > + */ > > > +int ram_dirty_bitmap_reload(MigrationState *s, RAMBlock *block) > > > +{ > > > + int ret = -EINVAL; > > > + QEMUFile *file = s->rp_state.from_dst_file; > > > + unsigned long *le_bitmap, nbits = block->used_length >> TARGET_PAGE_BITS; > > > + uint64_t local_size = nbits / 8; > > > + uint64_t size, end_mark; > > > + > > > + if (s->state != MIGRATION_STATUS_POSTCOPY_RECOVER) { > > > + error_report("%s: incorrect state %s", __func__, > > > + MigrationStatus_lookup[s->state]); > > > + return -EINVAL; > > > + } > > > + > > > + /* > > > + * Note: see comments in ramblock_recv_bitmap_send() on why we > > > + * need the endianess convertion, and the paddings. > > > + */ > > > + local_size = ROUND_UP(local_size, 8); > > > + > > > + /* Add addings */ > > > + le_bitmap = bitmap_new(nbits + BITS_PER_LONG); > > > + > > > + size = qemu_get_be64(file); > > > + > > > + /* The size of the bitmap should match with our ramblock */ > > > + if (size != local_size) { > > > + error_report("%s: ramblock '%s' bitmap size mismatch " > > > + "(0x%lx != 0x%lx)", __func__, block->idstr, > > > + size, local_size); > > > > You need to use PRIx64 formatters there - %lx isn't portable. > > Yes. Fixing. > > > > > > + ret = -EINVAL; > > > + goto out; > > > + } > > > + > > > + size = qemu_get_buffer(file, (uint8_t *)le_bitmap, local_size); > > > + end_mark = qemu_get_be64(file); > > > + > > > + ret = qemu_file_get_error(file); > > > + if (ret || size != local_size) { > > > + error_report("%s: read bitmap failed for ramblock '%s': %d", > > > + __func__, block->idstr, ret); > > > > You might like to include size/local_size in the error. > > Will do. Thanks, > > -- > Peter Xu -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK