From mboxrd@z Thu Jan 1 00:00:00 1970 From: Julien Grall Subject: Re: [RFC 0/4] Support more than 8 vcpus on arm64 with GICv3 Date: Thu, 14 May 2015 18:48:46 +0100 Message-ID: <5554DFFE.6090909@citrix.com> References: <1431612863-3762-1-git-send-email-cbz@baozis.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta3.messagelabs.com ([195.245.230.39]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1YsxGa-0007aD-8i for xen-devel@lists.xenproject.org; Thu, 14 May 2015 17:49:52 +0000 In-Reply-To: <1431612863-3762-1-git-send-email-cbz@baozis.org> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Chen Baozi , xen-devel@lists.xenproject.org Cc: julien.grall@citrix.com, Chen Baozi , ian.campbell@citrix.com List-Id: xen-devel@lists.xenproject.org Hi Chen, On 14/05/15 15:14, Chen Baozi wrote: > From: Chen Baozi > > Currently the number of vcpus on arm64 with GICv3 is limited up to 8 due > to the fixed size of redistributor mmio region. In this patch series, I > postpone setting the size of GICR0 to the point when max_vcpus of a domU is > determined to support more than 8 redistributors. I don't think postponing is necessary. We have plenty of space in the RAM guest layout to reserve a region for 128 redistributors. If the guest is trying to access to wrong re-distributor, it will receive a data abort. It's already the case today when the guest is using less than 8 vCPUs. Although you would need to reshuffle a bit the layout. With your solution, if the guest is using 128 vCPUs it will overlap the grant-table region, magic page (xenstore, xenconsole,...) and the beginning of the RAM. whoops ;). > However, I am not quite sure that decoupling the rdist base and size > setting of domU to different functions is appropriate, though I am now > able to create both a dom0 and a domU of 8+ vcpus with these patches. > So any comments/suggestions are welcomed. I'm afraid to say that your suggestion is only enough to support up to 16 vCPUs per guest. The vGIC is only using the affinity 0 of the MPIDR (AFF1, AFF2 and AFF3 are ignored). Affinity 0 represents the CPU in a cluster and AFF{1,2,3} the cluster ID. Each cluster can support up to 16 CPUs. You will also need to change the way to domain is creating the MIPDR, currently it considers that AFF0 == vcpu_id (see vcpu_initialise). Lastly, you need more care for the GICv2 case. We don't want a user to create a guest with more than 8 vCPUs. Even though supporting more than 16 vCPUs would nice, it would require more work. I would be fine if you decide to only bump to 16 vCPUs for now. Regards, -- Julien Grall