[RFC] Extending numbers of event channels

[Wei Liu <Wei.Liu2@citrix.com>]

Hi all

There has been discussion on extending number of event channels back in
September [0].

Regarding Jan's comment in [0], I don't think allowing user to specify
arbitrary number of levels a good idea. Because only the last level
should be shared among vcpus, other level should be in percpu struct to
allow for quicker lookup. The idea to let user specify levels will be
too complicated in implementation and blow up percpu section (since the
size grows exponentially). Three levels should be quite enough. See
maths below.

Number of event channels:
 * 32bit: 1024 * sizeof(unsigned long long) * BITS_PER_BYTE = 64k
 * 64bit: 4096 * sizeof(unsigned long long) * BITS_PER_BYTE = 512k
Basically the third level is a new ABI, so I choose to use unsigned long
long here to get more event channels.

Pages occupied by the third level (if PAGE_SIZE=4k):
 * 32bit: 64k  / 8 / 4k = 2
 * 64bit: 512k / 8 / 4k = 16

Making second level percpu will incur overhead. In fact we move the
array in shared info into percpu struct:
 * 32bit: sizeof(unsigned long) * 8 * sizeof(unsigned long) = 128 byte
 * 64bit: sizeof(unsigned long) * 8 * sizeof(unsigned long) = 512 byte

What concerns me is that the struct evtchn buckets are allocated all at
once during initialization phrase. To save memory inside Xen, the
internal allocation/free scheme for evtchn needs to be modified. Ian
suggested we do small number of buckets at start of day then dynamically
fault in more as required.

To sum up:
     1. Guest should allocate pages for third level evtchn.
     2. Guest should register third level pages via a new hypercall op.
     3. Hypervisor should setup third level evtchn in that hypercall op.
     4. Only last level (third in this case) should be shared among
        vcpus.
     5. Need a flexible allocation/free scheme of struct evtchn.
     6. Debug dumping should use snapshot to avoid holding event lock
        for too long. (Jan's concern in [0])

Any comments are welcomed.


Wei.

[0] http://thread.gmane.org/gmane.comp.emulators.xen.devel/139921