From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A4551C4167B for ; Mon, 4 Dec 2023 21:10:30 +0000 (UTC) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rAGBy-0000F3-2I; Mon, 04 Dec 2023 16:09:26 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rAGBw-0000Ec-Li for qemu-devel@nongnu.org; Mon, 04 Dec 2023 16:09:24 -0500 Received: from smtp-out1.suse.de ([2a07:de40:b251:101:10:150:64:1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1rAGBu-0001DO-MY for qemu-devel@nongnu.org; Mon, 04 Dec 2023 16:09:24 -0500 Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 42CA2220EA; Mon, 4 Dec 2023 21:09:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1701724160; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=23xF8tz7SKQ7vw6c5OzqegT1oL9r3QamZrFg0X0JcBU=; b=kQT/qqtKTXE2uyW1J+SRAKvlSGo/+jGimv1A4l3uBjOrnGNrUPuJ5VXR3MzPwSS8CA0PrU tEgATRaavivfplFMC8wPUPIY1ZMVZ54G04Sx541Qa7mgu1XiGTABgnZoBOvWLi5sQwVvrC S/uMcBneConiRRfw4JuZr5lVVn/khwk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1701724160; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=23xF8tz7SKQ7vw6c5OzqegT1oL9r3QamZrFg0X0JcBU=; b=VOOtfGHDUlESkVApamXx3jM6ZbCb6PSUNPB2CKPVi6eeXlAL8Zwl0c0QSpITsiv+m4pCPO opmEd73WsnPQVqDg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id B6C781398A; Mon, 4 Dec 2023 21:09:19 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id t/9HHv8/bmU6VAAAD6G6ig (envelope-from ); Mon, 04 Dec 2023 21:09:19 +0000 From: Fabiano Rosas To: Peter Xu Cc: Steven Sistare , qemu-devel@nongnu.org, Juan Quintela , Paolo Bonzini , Thomas Huth , =?utf-8?Q?Daniel_P=2E_Berrang=C3=A9?= , Leonardo Bras Subject: Re: [PATCH V6 05/14] migration: propagate suspended runstate In-Reply-To: References: <1701380247-340457-1-git-send-email-steven.sistare@oracle.com> <1701380247-340457-6-git-send-email-steven.sistare@oracle.com> <87r0k1n4r7.fsf@suse.de> Date: Mon, 04 Dec 2023 18:09:16 -0300 Message-ID: <87o7f5n08z.fsf@suse.de> MIME-Version: 1.0 Content-Type: text/plain Authentication-Results: smtp-out1.suse.de; none X-Spamd-Result: default: False [-4.30 / 50.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; BAYES_HAM(-3.00)[100.00%]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_THREE(0.00)[3]; DKIM_SIGNED(0.00)[suse.de:s=susede2_rsa,suse.de:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-0.994]; RCPT_COUNT_SEVEN(0.00)[8]; FUZZY_BLOCKED(0.00)[rspamd.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] Received-SPF: pass client-ip=2a07:de40:b251:101:10:150:64:1; envelope-from=farosas@suse.de; helo=smtp-out1.suse.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Peter Xu writes: > On Mon, Dec 04, 2023 at 04:31:56PM -0300, Fabiano Rosas wrote: >> Peter Xu writes: >> >> > On Fri, Dec 01, 2023 at 11:23:33AM -0500, Steven Sistare wrote: >> >> >> @@ -109,6 +117,7 @@ static int global_state_post_load(void *opaque, int version_id) >> >> >> return -EINVAL; >> >> >> } >> >> >> s->state = r; >> >> >> + vm_set_suspended(s->vm_was_suspended || r == RUN_STATE_SUSPENDED); >> >> > >> >> > IIUC current vm_was_suspended (based on my read of your patch) was not the >> >> > same as a boolean representing "whether VM is suspended", but only a >> >> > temporary field to remember that for a VM stop request. To be explicit, I >> >> > didn't see this flag set in qemu_system_suspend() in your previous patch. >> >> > >> >> > If so, we can already do: >> >> > >> >> > vm_set_suspended(s->vm_was_suspended); >> >> > >> >> > Irrelevant of RUN_STATE_SUSPENDED? >> >> >> >> We need both terms of the expression. >> >> >> >> If the vm *is* suspended (RUN_STATE_SUSPENDED), then vm_was_suspended = false. >> >> We call global_state_store prior to vm_stop_force_state, so the incoming >> >> side sees s->state = RUN_STATE_SUSPENDED and s->vm_was_suspended = false. >> > >> > Right. >> > >> >> However, the runstate is RUN_STATE_INMIGRATE. When incoming finishes by >> >> calling vm_start, we need to restore the suspended state. Thus in >> >> global_state_post_load, we must set vm_was_suspended = true. >> > >> > With above, shouldn't global_state_get_runstate() (on dest) fetch SUSPENDED >> > already? Then I think it should call vm_start(SUSPENDED) if to start. >> > >> > Maybe you're talking about the special case where autostart==false? We >> > used to have this (existing process_incoming_migration_bh()): >> > >> > if (!global_state_received() || >> > global_state_get_runstate() == RUN_STATE_RUNNING) { >> > if (autostart) { >> > vm_start(); >> > } else { >> > runstate_set(RUN_STATE_PAUSED); >> > } >> > } >> > >> > If so maybe I get you, because in the "else" path we do seem to lose the >> > SUSPENDED state again, but in that case IMHO we should logically set >> > vm_was_suspended only when we "lose" it - we didn't lose it during >> > migration, but only until we decided to switch to PAUSED (due to >> > autostart==false). IOW, change above to something like: >> > >> > state = global_state_get_runstate(); >> > if (!global_state_received() || runstate_is_alive(state)) { >> > if (autostart) { >> > vm_start(state); >> > } else { >> > if (runstate_is_suspended(state)) { >> > /* Remember suspended state before setting system to STOPed */ >> > vm_was_suspended = true; >> > } >> > runstate_set(RUN_STATE_PAUSED); >> > } >> > } >> > >> > It may or may not have a functional difference even if current patch, >> > though. However maybe clearer to follow vm_was_suspended's strict >> > definition. >> > >> >> >> >> If the vm *was* suspended, but is currently stopped (eg RUN_STATE_PAUSED), >> >> then vm_was_suspended = true. Migration from that state sets >> >> vm_was_suspended = s->vm_was_suspended = true in global_state_post_load and >> >> ends with runstate_set(RUN_STATE_PAUSED). >> >> >> >> I will add a comment here in the code. >> >> >> >> >> return 0; >> >> >> } >> >> >> @@ -134,6 +143,7 @@ static const VMStateDescription vmstate_globalstate = { >> >> >> .fields = (VMStateField[]) { >> >> >> VMSTATE_UINT32(size, GlobalState), >> >> >> VMSTATE_BUFFER(runstate, GlobalState), >> >> >> + VMSTATE_BOOL(vm_was_suspended, GlobalState), >> >> >> VMSTATE_END_OF_LIST() >> >> >> }, >> >> >> }; >> >> > >> >> > I think this will break migration between old/new, unfortunately. And >> >> > since the global state exist mostly for every VM, all VM setup should be >> >> > affected, and over all archs. >> >> >> >> Thanks, I keep forgetting that my binary tricks are no good here. However, >> >> I have one other trick up my sleeve, which is to store vm_was_running in >> >> global_state.runstate[strlen(runstate) + 2]. It is forwards and backwards >> >> compatible, since that byte is always 0 in older qemu. It can be implemented >> >> with a few lines of code change confined to global_state.c, versus many lines >> >> spread across files to do it the conventional way using a compat property and >> >> a subsection. Sound OK? >> > >> > Tricky! But sounds okay to me. I think you're inventing some of your own >> > way of being compatible, not relying on machine type as a benefit. If go >> > this route please document clearly on the layout and also what it looked >> > like in old binaries. >> > >> > I think maybe it'll be good to keep using strings, so in the new binaries >> > we allow >1 strings, then we define properly on those strings (index 0: >> > runstate, existed since start; index 2: suspended, perhaps using "1"/"0" to >> > express, while 0x00 means old binary, etc.). >> > >> > I hope this trick will need less code than the subsection solution, >> > otherwise I'd still consider going with that, which is the "common >> > solution". >> > >> > Let's also see whether Juan/Fabiano/others has any opinions. >> >> Can't we pack the structure and just go ahead and slash 'runstate' in >> half? That would claim some unused bytes for future backward >> compatibility issues. > > What I meant is something like: > > runstate[100] = {"str1", 0x00, "str2", 0x00, ...} > > Where str1 is runstate, and str2 can be either "0"/"1" to reflect suspended > value. We define all the strings separated by 0x00, then IIUC we save the > most chars for potential future extension of this string. > > Thanks, Right, I got your point. I just think we could avoid designing this new string format by creating new fields with the extra space: typedef struct QEMU_PACKED { uint32_t size; uint8_t runstate[50]; uint8_t unused[50]; RunState state; bool received; } GlobalState; In my mind this works seamlessly, or am I mistaken? In any case, a oneshot hack might be better than both our suggestions because we can just clean it up a couple of releases from now as if nothing happened.