From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Vrabel Subject: Re: [PATCHv3 3/4] xen: use ticket locks for spin locks Date: Thu, 30 Apr 2015 12:55:07 +0100 Message-ID: <5542181B.2020806@citrix.com> References: <1429611088-23950-1-git-send-email-david.vrabel@citrix.com> <1429611088-23950-4-git-send-email-david.vrabel@citrix.com> <5538FA930200007800075199@mail.emea.novell.com> <553FADA4.9080605@citrix.com> <554022AE02000078000CDFD8@mail.emea.novell.com> <5540F6E3.7030908@citrix.com> <55417DA602000078000CE131@mail.emea.novell.com> <20150430100932.GB13575@deinos.phlegethon.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1Ynn45-0002Ww-GS for xen-devel@lists.xenproject.org; Thu, 30 Apr 2015 11:55:37 +0000 In-Reply-To: <20150430100932.GB13575@deinos.phlegethon.org> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Tim Deegan , Jan Beulich Cc: keir@xen.org, ian.campbell@citrix.com, stefano.stabellini@eu.citrix.com, andrew.cooper3@citrix.com, jennifer.herbert@citrix.com, david.vrabel@citrix.com, xen-devel@lists.xenproject.org List-Id: xen-devel@lists.xenproject.org On 30/04/15 11:09, Tim Deegan wrote: > At 00:56 +0100 on 30 Apr (1430355366), Jan Beulich wrote: >>>>> David Vrabel 04/29/15 5:28 PM >>> >>> On 29/04/15 00:15, Jan Beulich wrote: >>>>>>> David Vrabel 04/28/15 6:16 PM >>> >>>>> Are there any structures whose size you're particularly concerned about? >>>> >>>> No specific ones (but of course structures with an inherent size constraint >>>> - like struct domain and struct vcpu - are, with all of their sub-structures, >>>> primary candidates). I just recall that in some cases (and this may no longer >>>> apply due to later additions) structures got arranged specifically taking in >>>> mind the 2-byte size of the locks, and the growth here may thus mean a >>>> structure size growth of more than just two bytes. >>> >>> Spin locks are currently 4 bytes (2 bytes for the lock, plus 2 bytes for >>> the recurse_{cnt,cpu}), and ticket locks are now 8 bytes (an increase in >>> 4 bytes). >>> >>> struct vcpu does not increase in size (there is 56, now 48 bytes, of >>> padding before the arch field). >>> >>> struct domain increases from 3480 to 3544 bytes. >> >> Odd - this suggests there's still some 64-byte alignment somewhere, but I >> can't immediately spot it. Only struct vcpu has an obvious 64-byte aligned >> field (struct pi_desc) afaics. > > I got slightly different numbers to David (I suspect just starting > from a different baseline) but the biggest hole came from struct > arch_domain being __cacheline_aligned (i.e. 128-byte aligned). > > There are some holes above it in struct domain which should pack in > nicely to counteract the larger spinlocks, e.g. this was enough to > naturally align arch_domain again in my tree: > > @@ -334,9 +334,9 @@ struct domain > spinlock_t rangesets_lock; > > /* Event channel information. */ > + unsigned int max_evtchns; > struct evtchn *evtchn; /* first bucket only */ > struct evtchn **evtchn_group[NR_EVTCHN_GROUPS]; /* all other buckets */ > - unsigned int max_evtchns; > unsigned int max_evtchn_port; > spinlock_t event_lock; > const struct evtchn_port_ops *evtchn_port_ops; This change makes struct domain 8 bytes bigger with ticket locks. On x86 there are 12 spinlocks in struct domain which is 48 bytes extra. I've clawed back 16 bytes by repacking struct paging_domain and struct hvm_domain. David