From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp153-170.sina.com.cn (smtp153-170.sina.com.cn [61.135.153.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F754296BB8 for ; Thu, 20 Nov 2025 22:57:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=61.135.153.170 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763679429; cv=none; b=FdGmGybSY+DLh5bWiVAH5VMWm1uMHewneC1mVFsVlVHORioopDW4n0oCRbZHl6fkFafkV+nxZOVMjxQQmrw3bDAdyv592RyhkDGqXhAOAbR4rHorSjXVe+CwETi1kXA5qzoUTU1A6nATbmIlrjJwtcr8TJqEm+nVG/KeSHzb2oA= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1763679429; c=relaxed/simple; bh=iYpdSFAPLWF1JJB05P3xm96IY5tfcFWvVUoZnD9CSng=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SY7JMGDLJT+KiUZXs3mrsl1z/hQrSvN10NUSsmIU7ThE3p6y8w5sFWDSVgiYPHRGn3wG/kOA+g7cJJ7QOo2glQtm6NlDGfoUtGHzUDQbAX8vGHrwR0r/SEr1SJw0OugqwK4qT7qydyAj4J4htKYemf3BFpauyhv+DqOqTZRfOIU= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=sina.com; spf=pass smtp.mailfrom=sina.com; dkim=pass (1024-bit key) header.d=sina.com header.i=@sina.com header.b=Z7IUQatf; arc=none smtp.client-ip=61.135.153.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=sina.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sina.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=sina.com header.i=@sina.com header.b="Z7IUQatf" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sina.com; s=201208; t=1763679420; bh=AK7LGWM+2Nio24/xJncC7D3KwRSlaSj/rWk+0RgzeA0=; h=From:Subject:Date:Message-ID; b=Z7IUQatfRNCproUmB5hAaJSPkwPArrubO3/unjfgWTEPwgY1IF2G5VLvKkid7lenk kPVv3fHXMOyViJ89UQ7Q+GVbXIv6kmReC6h0/D5XgBmGEe+eP3lL4JWEoXvtmzOy4c WQtUtHERbHXrcW+FLxMl53tqCDBlpPhsq/qyfrXk= X-SMAIL-HELO: localhost.localdomain Received: from unknown (HELO localhost.localdomain)([114.249.57.85]) by sina.com (10.54.253.34) with ESMTP id 691F9C8D0000127A; Thu, 21 Nov 2025 06:56:15 +0800 (CST) X-Sender: hdanton@sina.com X-Auth-ID: hdanton@sina.com Authentication-Results: sina.com; spf=none smtp.mailfrom=hdanton@sina.com; dkim=none header.i=none; dmarc=none action=none header.from=hdanton@sina.com X-SMAIL-MID: 4598566292012 X-SMAIL-UIID: 43C261139DA74804A015B9DDE12C2E88-20251121-065615-1 From: Hillf Danton To: Shrikanth Hegde Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, seanjc@google.com, kprateek.nayak@amd.com Subject: Re: [PATCH 01/17] sched/docs: Document cpu_paravirt_mask and Paravirt CPU concept Date: Fri, 21 Nov 2025 06:56:00 +0800 Message-ID: <20251120225603.9460-1-hdanton@sina.com> In-Reply-To: References: <20251119124449.1149616-1-sshegde@linux.ibm.com> <20251119214857.9436-1-hdanton@sina.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit On Thu, 20 Nov 2025 20:24:13 +0530 Shrikanth Hegde wrote: > On 11/20/25 3:18 AM, Hillf Danton wrote: > > On Wed, 19 Nov 2025 18:14:33 +0530 Shrikanth Hegde wrote: > >> Add documentation for new cpumask called cpu_paravirt_mask. This could > >> help users in understanding what this mask and the concept behind it. > >> > >> Signed-off-by: Shrikanth Hegde > >> --- > >> Documentation/scheduler/sched-arch.rst | 37 ++++++++++++++++++++++++++ > >> 1 file changed, 37 insertions(+) > >> > >> diff --git a/Documentation/scheduler/sched-arch.rst b/Documentation/scheduler/sched-arch.rst > >> index ed07efea7d02..6972c295013d 100644 > >> --- a/Documentation/scheduler/sched-arch.rst > >> +++ b/Documentation/scheduler/sched-arch.rst > >> @@ -62,6 +62,43 @@ Your cpu_idle routines need to obey the following rules: > >> arch/x86/kernel/process.c has examples of both polling and > >> sleeping idle functions. > >> > >> +Paravirt CPUs > >> +============= > >> + > >> +Under virtualised environments it is possible to overcommit CPU resources. > >> +i.e sum of virtual CPU(vCPU) of all VM's is greater than number of physical > >> +CPUs(pCPU). Under such conditions when all or many VM's have high utilization, > >> +hypervisor won't be able to satisfy the CPU requirement and has to context > >> +switch within or across VM. i.e hypervisor need to preempt one vCPU to run > >> +another. This is called vCPU preemption. This is more expensive compared to > >> +task context switch within a vCPU. > >> + > > What is missing is > > 1) vCPU preemption is X% more expensive compared to task context switch within a vCPU. > > > > This would change from arch to arch IMO. Will try to get numbers from PowerVM hypervisor. > > >> +In such cases it is better that VM's co-ordinate among themselves and ask for > >> +less CPU by not using some of the vCPUs. Such vCPUs where workload can be > >> +avoided at the moment for less vCPU preemption are called as "Paravirt CPUs". > >> +Note that when the pCPU contention goes away, these vCPUs can be used again > >> +by the workload. > >> + > > 2) given X, how to work out Y, the number of Paravirt CPUs for the simple > > scenario like 8 pCPUs and 16 vCPUs (8 vCPUs from VM1, 8 vCPUs from VM2)? > > > > Y need not be dependent on X. Note CPUs are marked as paravirt only when both VM's > end up consuming all the CPU resource. > To check that dependence, the frequence of vCPU preemption can be set to 100HZ and the frequence of task context switch within a vCPU to 250HZ, on top of __zero__ Y (actually what we can do before this work), to compare with the result of whatever Y this work can select. BTW workload on vCPU can be compiling linux kernel with -j 8. > Different cases: > 1. VM1 is idle and VM2 is idle - No vCPUs are marked as paravirt. > 2. VM1 is 100% busy and VM2 is idle - No steal time seen - No vCPUs is marked as paravirt. > 3. VM1 is idle and VM2 is 100% busy - No steal time seen - No vCPUs is marked as paravirt. > 4. VM1 is 100% busy and VM2 is 100% busy - 50% steal time would be seen in each - > Since there are only 8 pCPUs (assuming each VM1 is allocated equally), 4 vCPUs in > each VM will be marked as paravirt. Workload consolidates to remaining 4 vCPUs and > hence no steal time will seen. Benefit would seen since host doesn't need to change > expensive VM context switches.