From mboxrd@z Thu Jan 1 00:00:00 1970 From: Gerd Hoffmann Subject: Re: userspace block backend / gntdev problems Date: Fri, 04 Jan 2008 16:24:59 +0100 Message-ID: <477E4FCB.6080900@redhat.com> References: <477E3925.7070404@redhat.com> <1D19FC42-377A-47C7-8B6F-5BD56284C117@cl.cam.ac.uk> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="------------090203090404070103060200" Return-path: In-Reply-To: <1D19FC42-377A-47C7-8B6F-5BD56284C117@cl.cam.ac.uk> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Derek Murray Cc: Xen Development Mailing List List-Id: xen-devel@lists.xenproject.org This is a multi-part message in MIME format. --------------090203090404070103060200 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Derek Murray wrote: > The 128-grant limit is fairly arbitrary, and I wanted to see how people > were using gntdev before changing this. The reason for using a > fixed-size array is that it gives us O(1)-time mapping and unmapping of > single grants, which I anticipated would be the most frequently-used > case. Ok, try a hash instead of a list then ;) >> Second problem is that batched grant mappings (using >> xc_gnttab_map_grant_refs) don't work reliable. Symtoms I see are random >> failures with ENOMEM for no obvious reason (128 grant limit is *far* >> away). > > If it's failing with ENOMEM, a possible reason is that the address space > for mapping grants within gntdev (the array I mentioned above) is > becoming fragmented. Are you combining the mapping of single grants and > batches within the same gntdev instance? Yes, I'm mixing up single and batched maps (the later can have different sizes too, depending on the requests coming in, in the 1 -> 11 range). But I've seen ENOMEM failures with *only* the shared ring being mapped, i.e. one of 128 slots being used. That can't be fragmentation ... >> Also host kernel crashes (kernel 2.6.21-2952.fc8xen). > > When does this happen? Could you post the kernel OOPS? Dunno what exactly triggers it. Oops attached. cheers, Gerd --------------090203090404070103060200 Content-Type: text/plain; name="gntdev-oops" Content-Transfer-Encoding: base64 Content-Disposition: inline; filename="gntdev-oops" QlVHOiB1bmFibGUgdG8gaGFuZGxlIGtlcm5lbCBOVUxMIHBvaW50ZXIgZGVyZWZlcmVuY2Ug YXQgdmlydHVhbCBhZGRyZXNzIDAwMDAwMDAwCiBwcmludGluZyBlaXA6CjAxNDNlMDAwIC0+ ICpwZGUgPSAwMDAwMDAwMDo1MDE2ZTAwMQoyYzc2ZTAwMCAtPiAqcG1lID0gMDAwMDAwMDA6 MDAwMDAwMDAKT29wczogMDAwMCBbIzFdClNNUCAKbGFzdCBzeXNmcyBmaWxlOiAvZGV2aWNl cy94ZW4tYmFja2VuZC92YmQtMS01MTcxMi9zdGF0aXN0aWNzL3dyX3NlY3QKTW9kdWxlcyBs aW5rZWQgaW46IGlwdF9NQVNRVUVSQURFKFUpIGlwdGFibGVfbmF0KFUpIG5mX25hdChVKSBu Zl9jb25udHJhY2tfaXB2NChVKSB4dF9zdGF0ZShVKSBuZl9jb25udHJhY2soVSkgbmZuZXRs aW5rKFUpIGlwdF9SRUpFQ1QoVSkgeHRfdGNwdWRwKFUpIGlwdGFibGVfZmlsdGVyKFUpIGlw X3RhYmxlcyhVKSB4X3RhYmxlcyhVKSBicmlkZ2UoVSkgbmZzZChVKSBleHBvcnRmcyhVKSBs b2NrZChVKSBuZnNfYWNsKFUpIGF1dG9mczQoVSkgc3VucnBjKFUpIGlwdjYoVSkgZXh0MihV KSBsb29wKFUpIGRtX211bHRpcGF0aChVKSBuZXRiayhVKSBibGtiayhVKSA4MjUwX3BucChV KSA4MjUwX3BjaShVKSBzbmRfaGRhX2ludGVsKFUpIHNuZF9oZGFfY29kZWMoVSkgc25kX3Nl cV9kdW1teShVKSBzbmRfc2VxX29zcyhVKSBzbmRfc2VxX21pZGlfZXZlbnQoVSkgc25kX3Nl cShVKSBzbmRfc2VxX2RldmljZShVKSBzbmRfcGNtX29zcyhVKSBzbmRfbWl4ZXJfb3NzKFUp IHNuZF9wY20oVSkgaTJjX2k4MDEoVSkgcGFycG9ydF9wYyhVKSBzbmRfdGltZXIoVSkgaTJj X2NvcmUoVSkgc25kKFUpIHBhcnBvcnQoVSkgODI1MChVKSBlMTAwMChVKSBwY3Nwa3IoVSkg c291bmRjb3JlKFUpIHNlcmlvX3JhdyhVKSBzZXJpYWxfY29yZShVKSBhdGFfZ2VuZXJpYyhV KSBzbmRfcGFnZV9hbGxvYyhVKSBzcl9tb2QoVSkgc2coVSkgY2Ryb20oVSkgYXRhX3BpaXgo VSkgZG1fc25hcHNob3QoVSkgZG1femVybyhVKSBkbV9taXJyb3IoVSkgZG1fbW9kKFUpIGFo Y2koVSkgbGliYXRhKFUpIHNkX21vZChVKSBzY3NpX21vZChVKSBleHQzKFUpIGpiZChVKSBt YmNhY2hlKFUpIHVoY2lfaGNkKFUpIG9oY2lfaGNkKFUpIGVoY2lfaGNkKFUpCkNQVTogICAg MApFSVA6ICAgIDAwNjE6WzxjMTBlODViYT5dICAgIE5vdCB0YWludGVkIFZMSQpFRkxBR1M6 IDAwMDEwMjgyICAgKDIuNi4yMS0yOTUyLmZjOHhlbiAjMSkKRUlQIGlzIGF0IF9fc3luY19z aW5nbGUrMHgxYy8weDE5NwplYXg6IDAwMDAwMDAwICAgZWJ4OiAwMDA1YTZjYSAgIGVjeDog MDAwMDAwMDIgICBlZHg6IDAwMDAwMDAwCmVzaTogMDAwMDAwMDAgICBlZGk6IDAwMDAwMDAw ICAgZWJwOiAwMDAwMDQwMCAgIGVzcDogYzEzNmNlODAKZHM6IDAwN2IgICBlczogMDA3YiAg IGZzOiAwMGQ4ICBnczogMDAwMCAgc3M6IDAwNjkKUHJvY2VzcyBzd2FwcGVyIChwaWQ6IDAs IHRpPWMxMzZjMDAwIHRhc2s9YzEyZDQyNjAgdGFzay50aT1jMTMxNDAwMCkKU3RhY2s6IDAw MDAwMDAyIGVkNmExMDAwIGMxYzVkMTAwIGMxYzVkMTAwIGMxMzZjZWU4IGMxYzVkNWMwIDAw MDAwMDAwIGMxMDJiN2ExIAogICAgICAgMDAwNWE2Y2EgMDAwMDAwMDAgMDAwMDAwMDAgZWQ2 YTEwMDAgYzEwZTg3ZGIgMDAwMDA0MDAgMDAwMDAwMDIgMDAwMDA0MDAgCiAgICAgICAwMDAw MDAwMCAwMDAwMDQwMCAwMDAwMDAwMCBlYzdmYjQ4MCBjMTBlOGEzZSAwMDAwMDAwMiAwMDAw MDAwMSBjMWQ4Nzg0OCAKQ2FsbCBUcmFjZToKIFs8YzEwMmI3YTE+XSBsb2NrX3RpbWVyX2Jh c2UrMHgxOS8weDM1CiBbPGMxMGU4N2RiPl0gdW5tYXBfc2luZ2xlKzB4NTUvMHhkMgogWzxj MTBlOGEzZT5dIHN3aW90bGJfdW5tYXBfc2crMHgxMDMvMHgxMjAKIFs8ZWUxMDdmZWM+XSBh dGFfc2dfY2xlYW4rMHgxMDMvMHgxYjkgW2xpYmF0YV0KIFs8ZWUxMDgwZjA+XSBfX2F0YV9x Y19jb21wbGV0ZSsweDRlLzB4OTIgW2xpYmF0YV0KIFs8YzEwMDk4NTk+XSB0aW1lcl9pbnRl cnJ1cHQrMHg1YTQvMHg1YjcKIFs8ZWUxMGJjNzA+XSBhdGFfcWNfY29tcGxldGVfbXVsdGlw bGUrMHg4Ny8weDlkIFtsaWJhdGFdCiBbPGVlMGU1ZjIyPl0gYWhjaV9pbnRlcnJ1cHQrMHgy ZmYvMHg0YmQgW2FoY2ldCiBbPGMxMDRhNTNhPl0gaGFuZGxlX0lSUV9ldmVudCsweDM2LzB4 NmUKIFs8YzEwNGI5ZjI+XSBoYW5kbGVfbGV2ZWxfaXJxKzB4ODEvMHhjNwogWzxjMTA0Yjk3 MT5dIGhhbmRsZV9sZXZlbF9pcnErMHgwLzB4YzcKIFs8YzEwMDcxOWE+XSBkb19JUlErMHhh Yy8weGQyCiBbPGMxMDM2Y2I2Pl0ga3RpbWVfZ2V0KzB4Zi8weDJiCiBbPGMxMTRmMDc2Pl0g ZXZ0Y2huX2RvX3VwY2FsbCsweDgyLzB4ZGIKIFs8YzEwMDU4NWU+XSBoeXBlcnZpc29yX2Nh bGxiYWNrKzB4NDYvMHg0ZQogWzxjMTAwODg0MD5dIHJhd19zYWZlX2hhbHQrMHhiMy8weGQ1 CiBbPGMxMDA0NTJlPl0geGVuX2lkbGUrMHgzMS8weDVjCiBbPGMxMDAzNDM1Pl0gY3B1X2lk bGUrMHhhMy8weGJjCiBbPGMxMzE5YmU0Pl0gc3RhcnRfa2VybmVsKzB4NDgxLzB4NDg5CiBb PGMxMzE5MjVhPl0gdW5rbm93bl9ib290b3B0aW9uKzB4MC8weDIwMgogPT09PT09PT09PT09 PT09PT09PT09PT0KQ29kZTogYzggMDkgZDAgNWEgMGYgOTQgYzAgNTkgMGYgYjYgYzAgNWIg NWUgNWYgYzMgNTUgNTcgNTYgODkgYzYgNTMgODMgZWMgMjAgODkgNGMgMjQgMDQgOGIgNGMg MjQgMzggODkgNTQgMjQgMTggOGIgNmMgMjQgMzQgODkgMGMgMjQgPDhiPiAwOCBjMSBlOSAx ZSA2OSBjOSA4MCAxMiAwMCAwMCA4MSBjMSAwMCA5ZSAyZCBjMSA4YiA5OSAwYyAxMiAKRUlQ OiBbPGMxMGU4NWJhPl0gX19zeW5jX3NpbmdsZSsweDFjLzB4MTk3IFNTOkVTUCAwMDY5OmMx MzZjZTgwCktlcm5lbCBwYW5pYyAtIG5vdCBzeW5jaW5nOiBGYXRhbCBleGNlcHRpb24gaW4g aW50ZXJydXB0CihYRU4pIERvbWFpbiAwIGNyYXNoZWQ6IHJlYm9vdGluZyBtYWNoaW5lIGlu IDUgc2Vjb25kcy4KCg== --------------090203090404070103060200 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --------------090203090404070103060200--