From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Jan Beulich" Subject: Re: eliminating 166G limit (was Re: Problem with nr_nodes on large memory NUMA machine) Date: Tue, 27 Nov 2007 09:00:44 +0000 Message-ID: <474BEACC.76E4.0078.0@novell.com> References: <474BE6C2.76E4.0078.0@novell.com> Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: Content-Disposition: inline List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Keir Fraser Cc: xen-devel@lists.xensource.com List-Id: xen-devel@lists.xenproject.org >>> Keir Fraser 27.11.07 09:56 >>> >On 27/11/07 08:43, "Jan Beulich" wrote: > >> I think page allocation in this path isn't nice, at least not without = success >> guarantee (not the least because because netback doesn't check return >> values). I would therefore rather see a solution in placing the burden = of >> ensuring accessibility on the producer (netback) of the page, and fail = the >> transfer if the destination domain can't access the page (whether to be >> nice and try an allocate-and-copy operation here is a secondary thing). >>=20 >> Netback would then need to determine the address size of netfront's = domain >> (just like blkback and blktap do, except that HVM domains should also = be >> treated as not requiring address restriction), and have two pools of = pages >> for use in transfers - one unrestricted and one limited to 37 address = bits >> (the >> two could be folded for resource efficiency if the machine has less = than >> 128G). Besides that, netback would also start checking return values of = the >> multicall pieces. > >I don't get how your netback approach works. The pages we transfer do not >originate from netback, so it has little control over them. And, even if = it >did, when we allocate pages for network receive we do not know which >domain's packet will end up in each buffer. Oh, right, I mixed up old_mfn and new_mfn in netbk_gop_frag(). Nevertheless= netback could take care of this by doing the copying there, as at that = point i already knows the destination domain. >Personally I think doing it in Xen is perfectly good enough for supporting= >this very out-of-date network receive mechanism. I'm not just concerned about netback here. The interface exists, and other users might show up and/or exist already. Whether it would be acceptable for them to do allocation and copying is unknown. You'd therefore either need a way to prevent future users of the transfer mechanism, or set = proper requirements on its use. I think that placing extra requirements on the = user of the interface is better than introducing extra (possibly hard to = reproduce/ recognize/debug) possibilities of failure. Jan