Re: [RFC PATCH 2/3] Dynamically allocate domain->evtchn, also bump EVTCHNS_PER_BUCKET to 512.

[Wei Liu <Wei.Liu2@citrix.com>]

On Wed, 2013-01-02 at 13:38 +0000, David Vrabel wrote:
> On 31/12/12 18:22, Wei Liu wrote:
> > From: Wei Liu <liuw@liuw.name>
> 
> The changeset description needs to say why you're increasing
> EVTCHNS_PER_BUCKET.  I can't tell why.
> 

Here is the tedious maths. My thought was that it is not very
interesting to see this in a change log, so I dropped it. But I will add
this in my later re-post of this series...

#define EVTCHNS_PER_BUCKET ???
#define NR_EVTCHN_BUCKETS (NR_EVENT_CHANNEL_L3/EVTCHNS_PER_BUCKET)
d->evtchn = xzalloc_array(struct evtchn *, NR_EVTCHN_BUCKETS);

We need to allow for 3-level evtchn, so use NR_EVENT_CHANNELS_L3 to
calculate NR_EVTCHN_BUCKETS.

For 64 bit build, NR_EVENT_CHANNELS_L3 is 256k. The original value of
EVTCHNS_PER_BUCKET is 128, which means NR_EVTCHN_BUCKETS=2048, thus
d->evtchn has size of 2048*8 = 16KB = 4 pages. Given that only Dom0 or
driver domain will need 3-level event channel, this is really overkill
for most guests.

If we bump EVTCHNS_PER_BUCKET to 512, d->evtchn becomes 512 * 8 = 4KB =
1 page, which I think is more space efficient for most guests.

Of course, bumping EVTCHNS_PER_BUCKETS will cause overhead in other
place. Xen allocates a buckets-size of struct evtchn at a time. So the
overhead is:

sizeof(struct evtchn) * (512-128) ~= 18 bytes * 384

So the conclusion is:

1 page + sizeof(struct evtchn) * 384  < 4 pages

IMHO this is a good reason to bump EVTCHNS_PER_BUCKETS to 512.


Wei.

> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > ---
> >  xen/common/event_channel.c |    8 ++++++++
> >  xen/include/xen/sched.h    |    4 ++--
> >  2 files changed, 10 insertions(+), 2 deletions(-)
> > 
> > diff --git a/xen/common/event_channel.c b/xen/common/event_channel.c
> > index 87e422e..9898f8e 100644
> > --- a/xen/common/event_channel.c
> > +++ b/xen/common/event_channel.c
> > @@ -1172,6 +1172,12 @@ void notify_via_xen_event_channel(struct domain *ld, int lport)
> >  
> >  int evtchn_init(struct domain *d)
> >  {
> > +    d->evtchn = (struct evtchn **)
> > +        xzalloc_array(struct evtchn *, NR_EVTCHN_BUCKETS);
> 
> Don't need the cast here?
> 
> David