From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Kamble, Nitin A" Subject: Host Numa informtion in dom0 Date: Fri, 29 Jan 2010 15:05:08 -0800 Message-ID: <8EA2C2C4116BF44AB370468FBF85A7770123904A29@orsmsx504.amr.corp.intel.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="_004_8EA2C2C4116BF44AB370468FBF85A7770123904A29orsmsx504amrc_" Return-path: Content-Language: en-US List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org --_004_8EA2C2C4116BF44AB370468FBF85A7770123904A29orsmsx504amrc_ Content-Type: multipart/alternative; boundary="_000_8EA2C2C4116BF44AB370468FBF85A7770123904A29orsmsx504amrc_" --_000_8EA2C2C4116BF44AB370468FBF85A7770123904A29orsmsx504amrc_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Keir, Attached is the patch which exposes the host numa information to dom0. W= ith the patch "xm info" command now also gives the cpu topology & host numa= information. This will be later used to build guest numa support. The patch basically changes physinfo sysctl, and adds topology_info & numa_= info sysctls, and also changes the python & libxc code accordingly. Please apply. Thanks & Regards, Nitin --_000_8EA2C2C4116BF44AB370468FBF85A7770123904A29orsmsx504amrc_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable

Hi Keir,

   Attached is the patch which exposes the h= ost numa information to dom0. With the patch “xm info” command now = also gives the cpu topology & host numa information. This will be later used= to build guest numa support.

The patch basically changes physinfo sysctl, and adds topology_info & numa_info sysctls, and also changes the python & li= bxc code accordingly.

 

Please apply.

 

Thanks & Regards,

Nitin

 

 

--_000_8EA2C2C4116BF44AB370468FBF85A7770123904A29orsmsx504amrc_-- --_004_8EA2C2C4116BF44AB370468FBF85A7770123904A29orsmsx504amrc_ Content-Type: application/octet-stream; name="numactl_patch_20100128_1.diff" Content-Description: numactl_patch_20100128_1.diff Content-Disposition: attachment; filename="numactl_patch_20100128_1.diff"; size=30175; creation-date="Thu, 28 Jan 2010 13:38:36 GMT"; modification-date="Thu, 28 Jan 2010 13:37:21 GMT" Content-Transfer-Encoding: base64 ZGlmZiAtciAyNjM2ZTU2MTk3MDggdG9vbHMvbGlieGMveGNfbWlzYy5jCi0tLSBhL3Rvb2xzL2xp YnhjL3hjX21pc2MuYwlUdWUgSmFuIDI2IDE1OjU0OjQwIDIwMTAgKzAwMDAKKysrIGIvdG9vbHMv bGlieGMveGNfbWlzYy5jCVRodSBKYW4gMjggMTQ6MTc6MzIgMjAxMCAtMDgwMApAQCAtODAsNiAr ODAsNDMgQEAKICAgICByZXR1cm4gMDsKIH0KIAoraW50IHhjX3RvcG9sb2d5aW5mbyhpbnQgeGNf aGFuZGxlLAorICAgICAgICAgICAgICAgIHhjX3RvcG9sb2d5aW5mb190ICpwdXRfaW5mbykKK3sK KyAgICBpbnQgcmV0OworICAgIERFQ0xBUkVfU1lTQ1RMOworCisgICAgc3lzY3RsLmNtZCA9IFhF Tl9TWVNDVExfdG9wb2xvZ3lpbmZvOworCisgICAgbWVtY3B5KCZzeXNjdGwudS50b3BvbG9neWlu Zm8sIHB1dF9pbmZvLCBzaXplb2YoKnB1dF9pbmZvKSk7CisKKyAgICBpZiAoIChyZXQgPSBkb19z eXNjdGwoeGNfaGFuZGxlLCAmc3lzY3RsKSkgIT0gMCApCisgICAgICAgIHJldHVybiByZXQ7CisK KyAgICBtZW1jcHkocHV0X2luZm8sICZzeXNjdGwudS50b3BvbG9neWluZm8sIHNpemVvZigqcHV0 X2luZm8pKTsKKworICAgIHJldHVybiAwOworfQorCitpbnQgeGNfbnVtYWluZm8oaW50IHhjX2hh bmRsZSwKKyAgICAgICAgICAgICAgICB4Y19udW1haW5mb190ICpwdXRfaW5mbykKK3sKKyAgICBp bnQgcmV0OworICAgIERFQ0xBUkVfU1lTQ1RMOworCisgICAgc3lzY3RsLmNtZCA9IFhFTl9TWVND VExfbnVtYWluZm87CisKKyAgICBtZW1jcHkoJnN5c2N0bC51Lm51bWFpbmZvLCBwdXRfaW5mbywg c2l6ZW9mKCpwdXRfaW5mbykpOworCisgICAgaWYgKChyZXQgPSBkb19zeXNjdGwoeGNfaGFuZGxl LCAmc3lzY3RsKSkgIT0gMCkKKyAgICAgICAgcmV0dXJuIHJldDsKKworICAgIG1lbWNweShwdXRf aW5mbywgJnN5c2N0bC51Lm51bWFpbmZvLCBzaXplb2YoKnB1dF9pbmZvKSk7CisKKyAgICByZXR1 cm4gMDsKK30KKworCiBpbnQgeGNfc2NoZWRfaWQoaW50IHhjX2hhbmRsZSwKICAgICAgICAgICAg ICAgICBpbnQgKnNjaGVkX2lkKQogewpkaWZmIC1yIDI2MzZlNTYxOTcwOCB0b29scy9saWJ4Yy94 ZW5jdHJsLmgKLS0tIGEvdG9vbHMvbGlieGMveGVuY3RybC5oCVR1ZSBKYW4gMjYgMTU6NTQ6NDAg MjAxMCArMDAwMAorKysgYi90b29scy9saWJ4Yy94ZW5jdHJsLmgJVGh1IEphbiAyOCAxNDoxNzoz MiAyMDEwIC0wODAwCkBAIC02MDksOSArNjA5LDE5IEBACiBpbnQgeGNfc2VuZF9kZWJ1Z19rZXlz KGludCB4Y19oYW5kbGUsIGNoYXIgKmtleXMpOwogCiB0eXBlZGVmIHhlbl9zeXNjdGxfcGh5c2lu Zm9fdCB4Y19waHlzaW5mb190OwordHlwZWRlZiB4ZW5fc3lzY3RsX3RvcG9sb2d5aW5mb190IHhj X3RvcG9sb2d5aW5mb190OwordHlwZWRlZiB4ZW5fc3lzY3RsX251bWFpbmZvX3QgeGNfbnVtYWlu Zm9fdDsKKwogdHlwZWRlZiB1aW50MzJfdCB4Y19jcHVfdG9fbm9kZV90OwotaW50IHhjX3BoeXNp bmZvKGludCB4Y19oYW5kbGUsCi0gICAgICAgICAgICAgICAgeGNfcGh5c2luZm9fdCAqaW5mbyk7 Cit0eXBlZGVmIHVpbnQzMl90IHhjX2NwdV90b19zb2NrZXRfdDsKK3R5cGVkZWYgdWludDMyX3Qg eGNfY3B1X3RvX2NvcmVfdDsKK3R5cGVkZWYgdWludDY0X3QgeGNfbm9kZV90b19tZW1zaXplX3Q7 Cit0eXBlZGVmIHVpbnQ2NF90IHhjX25vZGVfdG9fbWVtZnJlZV90OwordHlwZWRlZiB1aW50MzJf dCB4Y19ub2RlX3RvX25vZGVfZGlzdF90OworCitpbnQgeGNfcGh5c2luZm8oaW50IHhjX2hhbmRs ZSwgeGNfcGh5c2luZm9fdCAqaW5mbyk7CitpbnQgeGNfdG9wb2xvZ3lpbmZvKGludCB4Y19oYW5k bGUsIHhjX3RvcG9sb2d5aW5mb190ICppbmZvKTsKK2ludCB4Y19udW1haW5mbyhpbnQgeGNfaGFu ZGxlLCB4Y19udW1haW5mb190ICppbmZvKTsKIAogaW50IHhjX3NjaGVkX2lkKGludCB4Y19oYW5k bGUsCiAgICAgICAgICAgICAgICAgaW50ICpzY2hlZF9pZCk7CmRpZmYgLXIgMjYzNmU1NjE5NzA4 IHRvb2xzL3B5dGhvbi94ZW4vbG93bGV2ZWwveGMveGMuYwotLS0gYS90b29scy9weXRob24veGVu L2xvd2xldmVsL3hjL3hjLmMJVHVlIEphbiAyNiAxNTo1NDo0MCAyMDEwICswMDAwCisrKyBiL3Rv b2xzL3B5dGhvbi94ZW4vbG93bGV2ZWwveGMveGMuYwlUaHUgSmFuIDI4IDE0OjE3OjMyIDIwMTAg LTA4MDAKQEAgLTEwOTcsMTA1ICsxMDk3LDE3OCBAQAogICAgIHJldHVybiBQeUxvbmdfRnJvbVVu c2lnbmVkTG9uZyhwYWdlc190b19raWIocGFnZXMpKTsKIH0KIAotCiBzdGF0aWMgUHlPYmplY3Qg KnB5eGNfcGh5c2luZm8oWGNPYmplY3QgKnNlbGYpCiB7Ci0jZGVmaW5lIE1BWF9DUFVfSUQgMjU1 Ci0gICAgeGNfcGh5c2luZm9fdCBpbmZvOworICAgIHhjX3BoeXNpbmZvX3QgcGluZm87CiAgICAg Y2hhciBjcHVfY2FwWzEyOF0sIHZpcnRfY2Fwc1sxMjhdLCAqcDsKLSAgICBpbnQgaSwgaiwgbWF4 X2NwdV9pZCwgbnJfbm9kZXMgPSAwOwotICAgIHVpbnQ2NF90IGZyZWVfaGVhcDsKLSAgICBQeU9i amVjdCAqcmV0X29iaiwgKm5vZGVfdG9fY3B1X29iaiwgKm5vZGVfdG9fbWVtb3J5X29iajsKLSAg ICBQeU9iamVjdCAqbm9kZV90b19kbWEzMl9tZW1fb2JqOwotICAgIHhjX2NwdV90b19ub2RlX3Qg bWFwW01BWF9DUFVfSUQgKyAxXTsKKyAgICBpbnQgaTsKICAgICBjb25zdCBjaGFyICp2aXJ0Y2Fw X25hbWVzW10gPSB7ICJodm0iLCAiaHZtX2RpcmVjdGlvIiB9OwogCi0gICAgc2V0X3hlbl9ndWVz dF9oYW5kbGUoaW5mby5jcHVfdG9fbm9kZSwgbWFwKTsKLSAgICBpbmZvLm1heF9jcHVfaWQgPSBN QVhfQ1BVX0lEOwotCi0gICAgaWYgKCB4Y19waHlzaW5mbyhzZWxmLT54Y19oYW5kbGUsICZpbmZv KSAhPSAwICkKKyAgICBpZiAoIHhjX3BoeXNpbmZvKHNlbGYtPnhjX2hhbmRsZSwgJnBpbmZvKSAh PSAwICkKICAgICAgICAgcmV0dXJuIHB5eGNfZXJyb3JfdG9fZXhjZXB0aW9uKCk7CiAKICAgICBw ID0gY3B1X2NhcDsKICAgICAqcCA9ICdcMCc7Ci0gICAgZm9yICggaSA9IDA7IGkgPCBzaXplb2Yo aW5mby5od19jYXApLzQ7IGkrKyApCi0gICAgICAgIHAgKz0gc3ByaW50ZihwLCAiJTA4eDoiLCBp bmZvLmh3X2NhcFtpXSk7CisgICAgZm9yICggaSA9IDA7IGkgPCBzaXplb2YocGluZm8uaHdfY2Fw KS80OyBpKysgKQorICAgICAgICBwICs9IHNwcmludGYocCwgIiUwOHg6IiwgcGluZm8uaHdfY2Fw W2ldKTsKICAgICAqKHAtMSkgPSAwOwogCiAgICAgcCA9IHZpcnRfY2FwczsKICAgICAqcCA9ICdc MCc7CiAgICAgZm9yICggaSA9IDA7IGkgPCAyOyBpKysgKQotICAgICAgICBpZiAoIChpbmZvLmNh cGFiaWxpdGllcyA+PiBpKSAmIDEgKQorICAgICAgICBpZiAoIChwaW5mby5jYXBhYmlsaXRpZXMg Pj4gaSkgJiAxICkKICAgICAgICAgICBwICs9IHNwcmludGYocCwgIiVzICIsIHZpcnRjYXBfbmFt ZXNbaV0pOwogICAgIGlmICggcCAhPSB2aXJ0X2NhcHMgKQogICAgICAgKihwLTEpID0gJ1wwJzsK IAotICAgIG1heF9jcHVfaWQgPSBpbmZvLm1heF9jcHVfaWQ7Ci0gICAgaWYgKCBtYXhfY3B1X2lk ID4gTUFYX0NQVV9JRCApCi0gICAgICAgIG1heF9jcHVfaWQgPSBNQVhfQ1BVX0lEOworICAgIHJl dHVybiBQeV9CdWlsZFZhbHVlKCJ7czppLHM6aSxzOmksczppLHM6aSxzOmwsczpsLHM6bCxzOmks czpzLHM6c30iLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICJucl9ub2RlcyIsICAgICAg ICAgcGluZm8ubnJfbm9kZXMsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgInRocmVhZHNf cGVyX2NvcmUiLCBwaW5mby50aHJlYWRzX3Blcl9jb3JlLAorICAgICAgICAgICAgICAgICAgICAg ICAgICAgICJjb3Jlc19wZXJfc29ja2V0IiwgcGluZm8uY29yZXNfcGVyX3NvY2tldCwKKyAgICAg ICAgICAgICAgICAgICAgICAgICAgICAic29ja2V0c19wZXJfbm9kZSIsIHBpbmZvLnNvY2tldHNf cGVyX25vZGUsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgIm5yX2NwdXMiLCAgICAgICAg ICBwaW5mby5ucl9jcHVzLCAKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAidG90YWxfbWVt b3J5IiwgICAgIHBhZ2VzX3RvX2tpYihwaW5mby50b3RhbF9wYWdlcyksCisgICAgICAgICAgICAg ICAgICAgICAgICAgICAgImZyZWVfbWVtb3J5IiwgICAgICBwYWdlc190b19raWIocGluZm8uZnJl ZV9wYWdlcyksCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgInNjcnViX21lbW9yeSIsICAg ICBwYWdlc190b19raWIocGluZm8uc2NydWJfcGFnZXMpLAorICAgICAgICAgICAgICAgICAgICAg ICAgICAgICJjcHVfa2h6IiwgICAgICAgICAgcGluZm8uY3B1X2toeiwKKyAgICAgICAgICAgICAg ICAgICAgICAgICAgICAiaHdfY2FwcyIsICAgICAgICAgIGNwdV9jYXAsCisgICAgICAgICAgICAg ICAgICAgICAgICAgICAgInZpcnRfY2FwcyIsICAgICAgICB2aXJ0X2NhcHMpOworfQorCitzdGF0 aWMgUHlPYmplY3QgKnB5eGNfdG9wb2xvZ3lpbmZvKFhjT2JqZWN0ICpzZWxmKQoreworI2RlZmlu ZSBNQVhfQ1BVX0lOREVYIDI1NQorICAgIHhjX3RvcG9sb2d5aW5mb190IHRpbmZvOworICAgIGlu dCBpLCBtYXhfY3B1X2luZGV4OworICAgIFB5T2JqZWN0ICpyZXRfb2JqOworICAgIFB5T2JqZWN0 ICpjcHVfdG9fY29yZV9vYmosICpjcHVfdG9fc29ja2V0X29iaiwgKmNwdV90b19ub2RlX29iajsK KyAgICB4Y19jcHVfdG9fY29yZV90IGNvcmVtYXBbTUFYX0NQVV9JTkRFWCArIDFdOworICAgIHhj X2NwdV90b19zb2NrZXRfdCBzb2NrZXRtYXBbTUFYX0NQVV9JTkRFWCArIDFdOworICAgIHhjX2Nw dV90b19ub2RlX3Qgbm9kZW1hcFtNQVhfQ1BVX0lOREVYICsgMV07CisKKworICAgIHNldF94ZW5f Z3Vlc3RfaGFuZGxlKHRpbmZvLmNwdV90b19jb3JlLCBjb3JlbWFwKTsKKyAgICBzZXRfeGVuX2d1 ZXN0X2hhbmRsZSh0aW5mby5jcHVfdG9fc29ja2V0LCBzb2NrZXRtYXApOworICAgIHNldF94ZW5f Z3Vlc3RfaGFuZGxlKHRpbmZvLmNwdV90b19ub2RlLCBub2RlbWFwKTsKKyAgICB0aW5mby5tYXhf Y3B1X2luZGV4ID0gTUFYX0NQVV9JTkRFWDsKKworICAgIGlmICggeGNfdG9wb2xvZ3lpbmZvKHNl bGYtPnhjX2hhbmRsZSwgJnRpbmZvKSAhPSAwICkKKyAgICAgICAgcmV0dXJuIHB5eGNfZXJyb3Jf dG9fZXhjZXB0aW9uKCk7CisKKyAgICBtYXhfY3B1X2luZGV4ID0gdGluZm8ubWF4X2NwdV9pbmRl eDsKKyAgICBpZiAoIG1heF9jcHVfaW5kZXggPiBNQVhfQ1BVX0lOREVYICkKKyAgICAgICAgbWF4 X2NwdV9pbmRleCA9IE1BWF9DUFVfSU5ERVg7CisKKyAgICAvKiBDb25zdHJ1Y3QgY3B1LXRvLSog bGlzdHMuICovCisgICAgY3B1X3RvX2NvcmVfb2JqID0gUHlMaXN0X05ldygwKTsKKyAgICBjcHVf dG9fc29ja2V0X29iaiA9IFB5TGlzdF9OZXcoMCk7CisgICAgY3B1X3RvX25vZGVfb2JqID0gUHlM aXN0X05ldygwKTsKKyAgICBmb3IgKCBpID0gMDsgaSA8IG1heF9jcHVfaW5kZXg7IGkrKyApCisg ICAgeworICAgICAgICBQeU9iamVjdCAqcHlpbnQ7CisKKyAgICAgICAgcHlpbnQgPSBQeUludF9G cm9tTG9uZyhjb3JlbWFwW2ldKTsKKyAgICAgICAgUHlMaXN0X0FwcGVuZChjcHVfdG9fY29yZV9v YmosIHB5aW50KTsKKyAgICAgICAgUHlfREVDUkVGKHB5aW50KTsKKworICAgICAgICBweWludCA9 IFB5SW50X0Zyb21Mb25nKHNvY2tldG1hcFtpXSk7CisgICAgICAgIFB5TGlzdF9BcHBlbmQoY3B1 X3RvX3NvY2tldF9vYmosIHB5aW50KTsKKyAgICAgICAgUHlfREVDUkVGKHB5aW50KTsKKworICAg ICAgICBweWludCA9IFB5SW50X0Zyb21Mb25nKG5vZGVtYXBbaV0pOworICAgICAgICBQeUxpc3Rf QXBwZW5kKGNwdV90b19ub2RlX29iaiwgcHlpbnQpOworICAgICAgICBQeV9ERUNSRUYocHlpbnQp OworICAgIH0KKworICAgIHJldF9vYmogPSBQeV9CdWlsZFZhbHVlKCJ7czppfSIsICJtYXhfY3B1 X2luZGV4IiwgbWF4X2NwdV9pbmRleCk7CisKKyAgICBQeURpY3RfU2V0SXRlbVN0cmluZyhyZXRf b2JqLCAiY3B1X3RvX2NvcmUiLCBjcHVfdG9fY29yZV9vYmopOworICAgIFB5X0RFQ1JFRihjcHVf dG9fY29yZV9vYmopOworCisgICAgUHlEaWN0X1NldEl0ZW1TdHJpbmcocmV0X29iaiwgImNwdV90 b19zb2NrZXQiLCBjcHVfdG9fc29ja2V0X29iaik7CisgICAgUHlfREVDUkVGKGNwdV90b19zb2Nr ZXRfb2JqKTsKKyAKKyAgICBQeURpY3RfU2V0SXRlbVN0cmluZyhyZXRfb2JqLCAiY3B1X3RvX25v ZGUiLCBjcHVfdG9fbm9kZV9vYmopOworICAgIFB5X0RFQ1JFRihjcHVfdG9fbm9kZV9vYmopOwor IAorICAgIHJldHVybiByZXRfb2JqOworI3VuZGVmIE1BWF9DUFVfSU5ERVgKK30KKworc3RhdGlj IFB5T2JqZWN0ICpweXhjX251bWFpbmZvKFhjT2JqZWN0ICpzZWxmKQoreworI2RlZmluZSBNQVhf Tk9ERV9JTkRFWCAzMQorICAgIHhjX251bWFpbmZvX3QgbmluZm87CisgICAgaW50IGksIGosIG1h eF9ub2RlX2luZGV4OworICAgIHVpbnQ2NF90IGZyZWVfaGVhcDsKKyAgICBQeU9iamVjdCAqcmV0 X29iajsKKyAgICBQeU9iamVjdCAqbm9kZV90b19tZW1zaXplX29iaiwgKm5vZGVfdG9fbWVtZnJl ZV9vYmo7CisgICAgUHlPYmplY3QgKm5vZGVfdG9fZG1hMzJfbWVtX29iaiwgKm5vZGVfdG9fbm9k ZV9kaXN0X29iajsKKyAgICB4Y19ub2RlX3RvX21lbXNpemVfdCBub2RlX21lbXNpemVbTUFYX05P REVfSU5ERVggKyAxXTsKKyAgICB4Y19ub2RlX3RvX21lbWZyZWVfdCBub2RlX21lbWZyZWVbTUFY X05PREVfSU5ERVggKyAxXTsKKyAgICB4Y19ub2RlX3RvX25vZGVfZGlzdF90IG5vZGVzX2Rpc3Rb KE1BWF9OT0RFX0lOREVYICogTUFYX05PREVfSU5ERVgpICsgMV07CisKKyAgICBzZXRfeGVuX2d1 ZXN0X2hhbmRsZShuaW5mby5ub2RlX3RvX21lbXNpemUsIG5vZGVfbWVtc2l6ZSk7CisgICAgc2V0 X3hlbl9ndWVzdF9oYW5kbGUobmluZm8ubm9kZV90b19tZW1mcmVlLCBub2RlX21lbWZyZWUpOwor ICAgIHNldF94ZW5fZ3Vlc3RfaGFuZGxlKG5pbmZvLm5vZGVfdG9fbm9kZV9kaXN0YW5jZSwgbm9k ZXNfZGlzdCk7CisgICAgbmluZm8ubWF4X25vZGVfaW5kZXggPSBNQVhfTk9ERV9JTkRFWDsKKyAg ICBpZiggeGNfbnVtYWluZm8oc2VsZi0+eGNfaGFuZGxlLCAmbmluZm8pICE9IDAgKQorICAgICAg ICByZXR1cm4gcHl4Y19lcnJvcl90b19leGNlcHRpb24oKTsKKworICAgIG1heF9ub2RlX2luZGV4 ID0gbmluZm8ubWF4X25vZGVfaW5kZXg7CisgICAgaWYgKCBtYXhfbm9kZV9pbmRleCA+IE1BWF9O T0RFX0lOREVYICkKKyAgICAgICAgbWF4X25vZGVfaW5kZXggPSBNQVhfTk9ERV9JTkRFWDsKIAog ICAgIC8qIENvbnN0cnVjdCBub2RlLXRvLSogbGlzdHMuICovCi0gICAgbm9kZV90b19jcHVfb2Jq ID0gUHlMaXN0X05ldygwKTsKLSAgICBub2RlX3RvX21lbW9yeV9vYmogPSBQeUxpc3RfTmV3KDAp OworICAgIG5vZGVfdG9fbWVtc2l6ZV9vYmogPSBQeUxpc3RfTmV3KDApOworICAgIG5vZGVfdG9f bWVtZnJlZV9vYmogPSBQeUxpc3RfTmV3KDApOwogICAgIG5vZGVfdG9fZG1hMzJfbWVtX29iaiA9 IFB5TGlzdF9OZXcoMCk7Ci0gICAgZm9yICggaSA9IDA7IGkgPD0gaW5mby5tYXhfbm9kZV9pZDsg aSsrICkKKyAgICBub2RlX3RvX25vZGVfZGlzdF9vYmogPSBQeUxpc3RfTmV3KDApOworICAgIGZv ciAoIGkgPSAwOyBpIDwgbWF4X25vZGVfaW5kZXg7IGkrKyApCiAgICAgewotICAgICAgICBpbnQg bm9kZV9leGlzdHMgPSAwOwogICAgICAgICBQeU9iamVjdCAqcHlpbnQ7CiAKLSAgICAgICAgLyog Q1BVcy4gKi8KLSAgICAgICAgUHlPYmplY3QgKmNwdXMgPSBQeUxpc3RfTmV3KDApOwotICAgICAg ICBmb3IgKCBqID0gMDsgaiA8PSBtYXhfY3B1X2lkOyBqKysgKQotICAgICAgICB7Ci0gICAgICAg ICAgICBpZiAoIGkgIT0gbWFwW2pdICkKLSAgICAgICAgICAgICAgICBjb250aW51ZTsKLSAgICAg ICAgICAgIHB5aW50ID0gUHlJbnRfRnJvbUxvbmcoaik7Ci0gICAgICAgICAgICBQeUxpc3RfQXBw ZW5kKGNwdXMsIHB5aW50KTsKLSAgICAgICAgICAgIFB5X0RFQ1JFRihweWludCk7Ci0gICAgICAg ICAgICBub2RlX2V4aXN0cyA9IDE7Ci0gICAgICAgIH0KLSAgICAgICAgUHlMaXN0X0FwcGVuZChu b2RlX3RvX2NwdV9vYmosIGNwdXMpOyAKLSAgICAgICAgUHlfREVDUkVGKGNwdXMpOworICAgICAg ICAvKiBUb3RhbCBNZW1vcnkgKi8KKyAgICAgICAgcHlpbnQgPSBQeUludF9Gcm9tTG9uZyhub2Rl X21lbXNpemVbaV0gPj4gMjApOyAvKiBNQiAqLworICAgICAgICBQeUxpc3RfQXBwZW5kKG5vZGVf dG9fbWVtc2l6ZV9vYmosIHB5aW50KTsKKyAgICAgICAgUHlfREVDUkVGKHB5aW50KTsKIAotICAg ICAgICAvKiBNZW1vcnkuICovCi0gICAgICAgIHhjX2F2YWlsaGVhcChzZWxmLT54Y19oYW5kbGUs IDAsIDAsIGksICZmcmVlX2hlYXApOwotICAgICAgICBub2RlX2V4aXN0cyA9IG5vZGVfZXhpc3Rz IHx8IChmcmVlX2hlYXAgIT0gMCk7Ci0gICAgICAgIHB5aW50ID0gUHlJbnRfRnJvbUxvbmcoZnJl ZV9oZWFwIC8gMTAyNCk7Ci0gICAgICAgIFB5TGlzdF9BcHBlbmQobm9kZV90b19tZW1vcnlfb2Jq LCBweWludCk7CisgICAgICAgIC8qIEZyZWUgTWVtb3J5ICovCisgICAgICAgIHB5aW50ID0gUHlJ bnRfRnJvbUxvbmcobm9kZV9tZW1mcmVlW2ldID4+IDIwKTsgLyogTUIgKi8KKyAgICAgICAgUHlM aXN0X0FwcGVuZChub2RlX3RvX21lbWZyZWVfb2JqLCBweWludCk7CiAgICAgICAgIFB5X0RFQ1JF RihweWludCk7CiAKICAgICAgICAgLyogRE1BIG1lbW9yeS4gKi8KICAgICAgICAgeGNfYXZhaWxo ZWFwKHNlbGYtPnhjX2hhbmRsZSwgMCwgMzIsIGksICZmcmVlX2hlYXApOwotICAgICAgICBweWlu dCA9IFB5SW50X0Zyb21Mb25nKGZyZWVfaGVhcCAvIDEwMjQpOworICAgICAgICBweWludCA9IFB5 SW50X0Zyb21Mb25nKGZyZWVfaGVhcCA+PiAyMCk7IC8qIE1CICovCiAgICAgICAgIFB5TGlzdF9B cHBlbmQobm9kZV90b19kbWEzMl9tZW1fb2JqLCBweWludCk7CiAgICAgICAgIFB5X0RFQ1JFRihw eWludCk7CiAKLSAgICAgICAgaWYgKCBub2RlX2V4aXN0cyApCi0gICAgICAgICAgICBucl9ub2Rl cysrOworICAgICAgICAvKiBOb2RlIHRvIE5vZGUgRGlzdGFuY2UgKi8KKyAgICAgICAgZm9yICgg aiA9IDA7IGogPCBuaW5mby5tYXhfbm9kZV9pbmRleDsgaisrICkKKyAgICAgICAgeworICAgICAg ICAgICAgcHlpbnQgPSBQeUludF9Gcm9tTG9uZyhub2Rlc19kaXN0WyhpICogbmluZm8ubWF4X25v ZGVfaW5kZXgpICsgal0pOworICAgICAgICAgICAgUHlMaXN0X0FwcGVuZChub2RlX3RvX25vZGVf ZGlzdF9vYmosIHB5aW50KTsKKyAgICAgICAgICAgIFB5X0RFQ1JFRihweWludCk7CisgICAgICAg IH0KICAgICB9CiAKLSAgICByZXRfb2JqID0gUHlfQnVpbGRWYWx1ZSgie3M6aSxzOmksczppLHM6 aSxzOmksczppLHM6bCxzOmwsczpsLHM6aSxzOnM6czpzfSIsCi0gICAgICAgICAgICAgICAgICAg ICAgICAgICAgIm5yX25vZGVzIiwgICAgICAgICBucl9ub2RlcywKLSAgICAgICAgICAgICAgICAg ICAgICAgICAgICAibWF4X25vZGVfaWQiLCAgICAgIGluZm8ubWF4X25vZGVfaWQsCi0gICAgICAg ICAgICAgICAgICAgICAgICAgICAgIm1heF9jcHVfaWQiLCAgICAgICBpbmZvLm1heF9jcHVfaWQs Ci0gICAgICAgICAgICAgICAgICAgICAgICAgICAgInRocmVhZHNfcGVyX2NvcmUiLCBpbmZvLnRo cmVhZHNfcGVyX2NvcmUsCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAgImNvcmVzX3Blcl9z b2NrZXQiLCBpbmZvLmNvcmVzX3Blcl9zb2NrZXQsCi0gICAgICAgICAgICAgICAgICAgICAgICAg ICAgIm5yX2NwdXMiLCAgICAgICAgICBpbmZvLm5yX2NwdXMsIAotICAgICAgICAgICAgICAgICAg ICAgICAgICAgICJ0b3RhbF9tZW1vcnkiLCAgICAgcGFnZXNfdG9fa2liKGluZm8udG90YWxfcGFn ZXMpLAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICJmcmVlX21lbW9yeSIsICAgICAgcGFn ZXNfdG9fa2liKGluZm8uZnJlZV9wYWdlcyksCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAg InNjcnViX21lbW9yeSIsICAgICBwYWdlc190b19raWIoaW5mby5zY3J1Yl9wYWdlcyksCi0gICAg ICAgICAgICAgICAgICAgICAgICAgICAgImNwdV9raHoiLCAgICAgICAgICBpbmZvLmNwdV9raHos Ci0gICAgICAgICAgICAgICAgICAgICAgICAgICAgImh3X2NhcHMiLCAgICAgICAgICBjcHVfY2Fw LAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICJ2aXJ0X2NhcHMiLCAgICAgICAgdmlydF9j YXBzKTsKLSAgICBQeURpY3RfU2V0SXRlbVN0cmluZyhyZXRfb2JqLCAibm9kZV90b19jcHUiLCBu b2RlX3RvX2NwdV9vYmopOwotICAgIFB5X0RFQ1JFRihub2RlX3RvX2NwdV9vYmopOwotICAgIFB5 RGljdF9TZXRJdGVtU3RyaW5nKHJldF9vYmosICJub2RlX3RvX21lbW9yeSIsIG5vZGVfdG9fbWVt b3J5X29iaik7Ci0gICAgUHlfREVDUkVGKG5vZGVfdG9fbWVtb3J5X29iaik7CisgICAgcmV0X29i aiA9IFB5X0J1aWxkVmFsdWUoIntzOml9IiwgIm1heF9ub2RlX2luZGV4IiwgbWF4X25vZGVfaW5k ZXgpOworCisgICAgUHlEaWN0X1NldEl0ZW1TdHJpbmcocmV0X29iaiwgIm5vZGVfbWVtc2l6ZSIs IG5vZGVfdG9fbWVtc2l6ZV9vYmopOworICAgIFB5X0RFQ1JFRihub2RlX3RvX21lbXNpemVfb2Jq KTsKKworICAgIFB5RGljdF9TZXRJdGVtU3RyaW5nKHJldF9vYmosICJub2RlX21lbWZyZWUiLCBu b2RlX3RvX21lbWZyZWVfb2JqKTsKKyAgICBQeV9ERUNSRUYobm9kZV90b19tZW1mcmVlX29iaik7 CisKICAgICBQeURpY3RfU2V0SXRlbVN0cmluZyhyZXRfb2JqLCAibm9kZV90b19kbWEzMl9tZW0i LCBub2RlX3RvX2RtYTMyX21lbV9vYmopOwogICAgIFB5X0RFQ1JFRihub2RlX3RvX2RtYTMyX21l bV9vYmopOworCisgICAgUHlEaWN0X1NldEl0ZW1TdHJpbmcocmV0X29iaiwgIm5vZGVfdG9fbm9k ZV9kaXN0Iiwgbm9kZV90b19ub2RlX2Rpc3Rfb2JqKTsKKyAgICBQeV9ERUNSRUYobm9kZV90b19u b2RlX2Rpc3Rfb2JqKTsKICAKICAgICByZXR1cm4gcmV0X29iajsKLSN1bmRlZiBNQVhfQ1BVX0lE CisjdW5kZWYgTUFYX05PREVfSU5ERVgKIH0KIAogc3RhdGljIFB5T2JqZWN0ICpweXhjX3hlbmlu Zm8oWGNPYmplY3QgKnNlbGYpCkBAIC0yMDA0LDYgKzIwNzcsMjAgQEAKICAgICAgICJSZXR1cm5z IFtkaWN0XTogaW5mb3JtYXRpb24gYWJvdXQgdGhlIGhhcmR3YXJlIgogICAgICAgIiAgICAgICAg W05vbmVdOiBvbiBmYWlsdXJlLlxuIiB9LAogCisgICAgeyAidG9wb2xvZ3lpbmZvIiwKKyAgICAg IChQeUNGdW5jdGlvbilweXhjX3RvcG9sb2d5aW5mbywKKyAgICAgIE1FVEhfTk9BUkdTLCAiXG4i CisgICAgICAiR2V0IGluZm9ybWF0aW9uIGFib3V0IHRoZSBjcHUgdG9wb2xvZ3kgb24gdGhlIGhv c3QgbWFjaGluZVxuIgorICAgICAgIlJldHVybnMgW2RpY3RdOiBpbmZvcm1hdGlvbiBhYm91dCB0 aGUgY3B1IHRvcG9sb2d5IG9uIGhvc3QiCisgICAgICAiICAgICAgICBbTm9uZV06IG9uIGZhaWx1 cmUuXG4iIH0sCisKKyAgICB7ICJudW1haW5mbyIsCisgICAgICAoUHlDRnVuY3Rpb24pcHl4Y19u dW1haW5mbywKKyAgICAgIE1FVEhfTk9BUkdTLCAiXG4iCisgICAgICAiR2V0IE5VTUEgaW5mb3Jt YXRpb24gb24gdGhlIGhvc3QgbWFjaGluZVxuIgorICAgICAgIlJldHVybnMgW2RpY3RdOiBOVU1B IGluZm9ybWF0aW9uIG9uIGhvc3QiCisgICAgICAiICAgICAgICBbTm9uZV06IG9uIGZhaWx1cmUu XG4iIH0sCisKICAgICB7ICJ4ZW5pbmZvIiwKICAgICAgIChQeUNGdW5jdGlvbilweXhjX3hlbmlu Zm8sCiAgICAgICBNRVRIX05PQVJHUywgIlxuIgpkaWZmIC1yIDI2MzZlNTYxOTcwOCB0b29scy9w eXRob24veGVuL3hlbmQvWGVuZE5vZGUucHkKLS0tIGEvdG9vbHMvcHl0aG9uL3hlbi94ZW5kL1hl bmROb2RlLnB5CVR1ZSBKYW4gMjYgMTU6NTQ6NDAgMjAxMCArMDAwMAorKysgYi90b29scy9weXRo b24veGVuL3hlbmQvWGVuZE5vZGUucHkJVGh1IEphbiAyOCAxNDoxNzozMiAyMDEwIC0wODAwCkBA IC04NzQsNjYgKzg3NCw3MSBAQAogICAgIGRlZiBsaXN0X3RvX3N0cnJhbmdlKHNlbGYsbGlzdCk6 CiAgICAgICAgIHJldHVybiBzZWxmLmZvcm1hdF9wYWlycyhzZWxmLmxpc3RfdG9fcmFuZ2VwYWly cyhsaXN0KSkKIAotICAgIGRlZiBmb3JtYXRfbm9kZV90b19jcHUoc2VsZiwgcGluZm8pOgotICAg ICAgICBzdHI9JycKLSAgICAgICAgd2hpdGVzcGFjZT0nJworICAgIGRlZiBmb3JtYXRfY3B1X3Rv X2NvcmVfc29ja2V0X25vZGUoc2VsZiwgdGluZm8pOgogICAgICAgICB0cnk6Ci0gICAgICAgICAg ICBub2RlX3RvX2NwdT1waW5mb1snbm9kZV90b19jcHUnXQotICAgICAgICAgICAgZm9yIGkgaW4g cmFuZ2UoMCwgcGluZm9bJ21heF9ub2RlX2lkJ10rMSk6Ci0gICAgICAgICAgICAgICAgc3RyKz0n JXNub2RlJWQ6JXNcbicgJSAod2hpdGVzcGFjZSwKLSAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICBpLCAKLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg c2VsZi5saXN0X3RvX3N0cnJhbmdlKG5vZGVfdG9fY3B1W2ldKSkKLSAgICAgICAgICAgICAgICB3 aGl0ZXNwYWNlPSclMjVzJyAlICcnICAgICAgICAKLSAgICAgICAgZXhjZXB0OgotICAgICAgICAg ICAgc3RyPSdub25lXG4nCi0gICAgICAgIHJldHVybiBzdHJbOi0xXTsKLSAgICBkZWYgZm9ybWF0 X25vZGVfdG9fbWVtb3J5KHNlbGYsIHBpbmZvLCBrZXkpOgotICAgICAgICBzdHI9JycKLSAgICAg ICAgd2hpdGVzcGFjZT0nJwotICAgICAgICB0cnk6Ci0gICAgICAgICAgICBub2RlX3RvX21lbW9y eT1waW5mb1trZXldCi0gICAgICAgICAgICBmb3IgaSBpbiByYW5nZSgwLCBwaW5mb1snbWF4X25v ZGVfaWQnXSsxKToKLSAgICAgICAgICAgICAgICBzdHIrPSclc25vZGUlZDolZFxuJyAlICh3aGl0 ZXNwYWNlLAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGksCi0gICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbm9kZV90b19tZW1vcnlbaV0gLyAx MDI0KQotICAgICAgICAgICAgICAgIHdoaXRlc3BhY2U9JyUyNXMnICUgJycKKyAgICAgICAgICAg IG5yX2NwdXM9dGluZm9bJ21heF9jcHVfaW5kZXgnXQorICAgICAgICAgICAgc3RyPSdcbmNwdTog ICAgY29yZSAgICBzb2NrZXQgICAgIG5vZGVcbicKKyAgICAgICAgICAgIGZvciBpIGluIHJhbmdl KDAsIG5yX2NwdXMpOgorICAgICAgICAgICAgICAgIHN0cis9JyUzZDolOGQgJThkICU4ZFxuJyAl IChpLCAKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHRpbmZvWydj cHVfdG9fY29yZSddW2ldLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgdGluZm9bJ2NwdV90b19zb2NrZXQnXVtpXSwKKyAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIHRpbmZvWydjcHVfdG9fbm9kZSddW2ldKQogICAgICAgICBleGNlcHQ6 CiAgICAgICAgICAgICBzdHI9J25vbmVcbicKICAgICAgICAgcmV0dXJuIHN0cls6LTFdOwogCisg ICAgZGVmIGZvcm1hdF9udW1hX2luZm8oc2VsZiwgbmluZm8pOgorICAgICAgICB0cnk6CisgICAg ICAgICAgICBucl9ub2Rlcz1uaW5mb1snbWF4X25vZGVfaW5kZXgnXQorICAgICAgICAgICAgc3Ry PSdcbm5vZGU6IFRvdGFsTWVtb3J5IEZyZWVNZW1vcnkgZG1hMzJNZW1vcnkgTm9kZURpc3Q6Jwor ICAgICAgICAgICAgZm9yIGkgaW4gcmFuZ2UoMCwgbnJfbm9kZXMpOgorICAgICAgICAgICAgICAg IHN0cis9JyU0ZCAnICUgaQorICAgICAgICAgICAgc3RyKz0nXG4nCisgICAgICAgICAgICBmb3Ig aSBpbiByYW5nZSgwLCBucl9ub2Rlcyk6CisgICAgICAgICAgICAgICAgc3RyKz0nJTRkOiAgJThk TUIgJThkTUIgICU4ZE1CICAgICAgICAgOicgJSAoaSwgCisgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIG5pbmZvWydub2RlX21lbXNpemUnXVtpXSwKKyAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgbmluZm9bJ25vZGVfbWVtZnJlZSddW2ldLAorICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBuaW5mb1snbm9kZV90b19kbWEzMl9tZW0n XVtpXSkKKyAgICAgICAgICAgICAgICBmb3IgaiBpbiByYW5nZSgwLCBucl9ub2Rlcyk6CisgICAg ICAgICAgICAgICAgICAgIHN0cis9JyU0ZCAnICUgbmluZm9bJ25vZGVfdG9fbm9kZV9kaXN0J11b KGkqbnJfbm9kZXMpK2pdCisgICAgICAgICAgICAgICAgc3RyKz0nXG4nCisgICAgICAgIGV4Y2Vw dDoKKyAgICAgICAgICAgIHN0cj0nbm9uZVxuJworICAgICAgICByZXR1cm4gc3RyWzotMV07CiAK ICAgICBkZWYgcGh5c2luZm8oc2VsZik6CiAgICAgICAgIGluZm8gPSBzZWxmLnhjLnBoeXNpbmZv KCkKKyAgICAgICAgdGluZm8gPSBzZWxmLnhjLnRvcG9sb2d5aW5mbygpCisgICAgICAgIG5pbmZv ID0gc2VsZi54Yy5udW1haW5mbygpCiAKICAgICAgICAgaW5mb1snY3B1X21oeiddID0gaW5mb1sn Y3B1X2toeiddIC8gMTAwMAogICAgICAgICAKICAgICAgICAgIyBwaHlzaW5mbyBpcyBpbiBLaUIs IG5lZWQgaXQgaW4gTWlCCiAgICAgICAgIGluZm9bJ3RvdGFsX21lbW9yeSddID0gaW5mb1sndG90 YWxfbWVtb3J5J10gLyAxMDI0CiAgICAgICAgIGluZm9bJ2ZyZWVfbWVtb3J5J10gID0gaW5mb1sn ZnJlZV9tZW1vcnknXSAvIDEwMjQKLSAgICAgICAgaW5mb1snbm9kZV90b19jcHUnXSAgPSBzZWxm LmZvcm1hdF9ub2RlX3RvX2NwdShpbmZvKQotICAgICAgICBpbmZvWydub2RlX3RvX21lbW9yeSdd ID0gXAotICAgICAgICAgICAgc2VsZi5mb3JtYXRfbm9kZV90b19tZW1vcnkoaW5mbywgJ25vZGVf dG9fbWVtb3J5JykKLSAgICAgICAgaW5mb1snbm9kZV90b19kbWEzMl9tZW0nXSA9IFwKLSAgICAg ICAgICAgIHNlbGYuZm9ybWF0X25vZGVfdG9fbWVtb3J5KGluZm8sICdub2RlX3RvX2RtYTMyX21l bScpCisKKyAgICAgICAgaW5mb1snY3B1X3RvcG9sb2d5J10gID0gXAorICAgICAgICAgICAgIHNl bGYuZm9ybWF0X2NwdV90b19jb3JlX3NvY2tldF9ub2RlKHRpbmZvKQorCisgICAgICAgIGluZm9b J251bWFfaW5mbyddICA9IFwKKyAgICAgICAgICAgICBzZWxmLmZvcm1hdF9udW1hX2luZm8obmlu Zm8pCiAKICAgICAgICAgSVRFTV9PUkRFUiA9IFsnbnJfY3B1cycsCiAgICAgICAgICAgICAgICAg ICAgICAgJ25yX25vZGVzJywKICAgICAgICAgICAgICAgICAgICAgICAnY29yZXNfcGVyX3NvY2tl dCcsCiAgICAgICAgICAgICAgICAgICAgICAgJ3RocmVhZHNfcGVyX2NvcmUnLAorICAgICAgICAg ICAgICAgICAgICAgICdzb2NrZXRzX3Blcl9ub2RlJywKICAgICAgICAgICAgICAgICAgICAgICAn Y3B1X21oeicsCiAgICAgICAgICAgICAgICAgICAgICAgJ2h3X2NhcHMnLAogICAgICAgICAgICAg ICAgICAgICAgICd2aXJ0X2NhcHMnLAogICAgICAgICAgICAgICAgICAgICAgICd0b3RhbF9tZW1v cnknLAogICAgICAgICAgICAgICAgICAgICAgICdmcmVlX21lbW9yeScsCi0gICAgICAgICAgICAg ICAgICAgICAgJ25vZGVfdG9fY3B1JywKLSAgICAgICAgICAgICAgICAgICAgICAnbm9kZV90b19t ZW1vcnknLAotICAgICAgICAgICAgICAgICAgICAgICdub2RlX3RvX2RtYTMyX21lbScsCi0gICAg ICAgICAgICAgICAgICAgICAgJ21heF9ub2RlX2lkJworICAgICAgICAgICAgICAgICAgICAgICdj cHVfdG9wb2xvZ3knLAorICAgICAgICAgICAgICAgICAgICAgICdudW1hX2luZm8nLAogICAgICAg ICAgICAgICAgICAgICAgIF0KIAogICAgICAgICByZXR1cm4gW1trLCBpbmZvW2tdXSBmb3IgayBp biBJVEVNX09SREVSXQogCi0KICAgICBkZWYgcGNpaW5mbyhzZWxmKToKICAgICAgICAgZnJvbSB4 ZW4ueGVuZC5zZXJ2ZXIucGNpaWYgaW1wb3J0IGdldF9hbGxfYXNzaWduZWRfcGNpX2RldmljZXMK ICAgICAgICAgYXNzaWduZWRfZGV2cyA9IGdldF9hbGxfYXNzaWduZWRfcGNpX2RldmljZXMoKQpk aWZmIC1yIDI2MzZlNTYxOTcwOCB0b29scy9weXRob24veGVuL3hlbmQvYmFsbG9vbi5weQotLS0g YS90b29scy9weXRob24veGVuL3hlbmQvYmFsbG9vbi5weQlUdWUgSmFuIDI2IDE1OjU0OjQwIDIw MTAgKzAwMDAKKysrIGIvdG9vbHMvcHl0aG9uL3hlbi94ZW5kL2JhbGxvb24ucHkJVGh1IEphbiAy OCAxNDoxNzozMiAyMDEwIC0wODAwCkBAIC0xODQsMTUgKzE4NCwxMSBAQAogICAgICAgICAgICAg d2FpdHNjcnViID0gMQogICAgICAgICAgICAgdmNwdXMgPSBkb21pbmZvLmluZm9bJ2NwdXMnXVsw XQogICAgICAgICAgICAgZm9yIHZjcHUgaW4gdmNwdXM6Ci0gICAgICAgICAgICAgICAgbm9kZW51 bSA9IDAKLSAgICAgICAgICAgICAgICBmb3Igbm9kZSBpbiBwaHlzaW5mb1snbm9kZV90b19jcHUn XToKLSAgICAgICAgICAgICAgICAgICAgZm9yIGNwdSBpbiBub2RlOgotICAgICAgICAgICAgICAg ICAgICAgICAgaWYgdmNwdSA9PSBjcHU6Ci0gICAgICAgICAgICAgICAgICAgICAgICAgICAgaWYg b2xkbm9kZSA9PSAtMToKLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgb2xkbm9kZSA9 IG5vZGVudW0KLSAgICAgICAgICAgICAgICAgICAgICAgICAgICBlbGlmIG9sZG5vZGUgIT0gbm9k ZW51bToKLSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgd2FpdHNjcnViID0gMAotICAg ICAgICAgICAgICAgICAgICBub2RlbnVtID0gbm9kZW51bSArIDEKKyAgICAgICAgICAgICAgICBu b2RlbnVtID0geGMubnVtYWluZm8oKVsnY3B1X3RvX25vZGUnXVtjcHVdCisgICAgICAgICAgICAg ICAgaWYgb2xkbm9kZSA9PSAtMToKKyAgICAgICAgICAgICAgICAgICAgb2xkbm9kZSA9IG5vZGVu dW0KKyAgICAgICAgICAgICAgICBlbGlmIG9sZG5vZGUgIT0gbm9kZW51bToKKyAgICAgICAgICAg ICAgICAgICAgd2FpdHNjcnViID0gMAogCiAgICAgICAgICAgICBpZiB3YWl0c2NydWIgPT0gMSBh bmQgc2NydWJfbWVtID4gMDoKICAgICAgICAgICAgICAgICBsb2cuZGVidWcoIndhaXQgZm9yIHNj cnViICVzIiwgc2NydWJfbWVtKQpkaWZmIC1yIDI2MzZlNTYxOTcwOCB4ZW4vYXJjaC94ODYvc3lz Y3RsLmMKLS0tIGEveGVuL2FyY2gveDg2L3N5c2N0bC5jCVR1ZSBKYW4gMjYgMTU6NTQ6NDAgMjAx MCArMDAwMAorKysgYi94ZW4vYXJjaC94ODYvc3lzY3RsLmMJVGh1IEphbiAyOCAxNDoxNzozMiAy MDEwIC0wODAwCkBAIC0zNSw2ICszNSw4IEBACiAgICAgcmV0dXJuIGNwdV9kb3duKGNwdSk7CiB9 CiAKK2V4dGVybiBpbnQgX19ub2RlX2Rpc3RhbmNlKGludCBhLCBpbnQgYik7CisKIGxvbmcgYXJj aF9kb19zeXNjdGwoCiAgICAgc3RydWN0IHhlbl9zeXNjdGwgKnN5c2N0bCwgWEVOX0dVRVNUX0hB TkRMRSh4ZW5fc3lzY3RsX3QpIHVfc3lzY3RsKQogewpAQCAtNDUsMjUgKzQ3LDIyIEBACiAKICAg ICBjYXNlIFhFTl9TWVNDVExfcGh5c2luZm86CiAgICAgewotICAgICAgICB1aW50MzJfdCBpLCBt YXhfYXJyYXlfZW50OwotICAgICAgICBYRU5fR1VFU1RfSEFORExFXzY0KHVpbnQzMikgY3B1X3Rv X25vZGVfYXJyOwotCiAgICAgICAgIHhlbl9zeXNjdGxfcGh5c2luZm9fdCAqcGkgPSAmc3lzY3Rs LT51LnBoeXNpbmZvOwogCiAgICAgICAgIHJldCA9IHhzbV9waHlzaW5mbygpOwogICAgICAgICBp ZiAoIHJldCApCiAgICAgICAgICAgICBicmVhazsKIAotICAgICAgICBtYXhfYXJyYXlfZW50ID0g cGktPm1heF9jcHVfaWQ7Ci0gICAgICAgIGNwdV90b19ub2RlX2FyciA9IHBpLT5jcHVfdG9fbm9k ZTsKIAogICAgICAgICBtZW1zZXQocGksIDAsIHNpemVvZigqcGkpKTsKLSAgICAgICAgcGktPmNw dV90b19ub2RlID0gY3B1X3RvX25vZGVfYXJyOwogICAgICAgICBwaS0+dGhyZWFkc19wZXJfY29y ZSA9CiAgICAgICAgICAgICBjcHVzX3dlaWdodChwZXJfY3B1KGNwdV9zaWJsaW5nX21hcCwgMCkp OwogICAgICAgICBwaS0+Y29yZXNfcGVyX3NvY2tldCA9CiAgICAgICAgICAgICBjcHVzX3dlaWdo dChwZXJfY3B1KGNwdV9jb3JlX21hcCwgMCkpIC8gcGktPnRocmVhZHNfcGVyX2NvcmU7CiAgICAg ICAgIHBpLT5ucl9jcHVzID0gKHUzMiludW1fb25saW5lX2NwdXMoKTsKKyAgICAgICAgcGktPm5y X25vZGVzID0gKHUzMiludW1fb25saW5lX25vZGVzKCk7CisgICAgICAgIHBpLT5zb2NrZXRzX3Bl cl9ub2RlID0gIHBpLT5ucl9jcHVzIC8gCisgICAgICAgICAgICAgICAgICAgICAocGktPm5yX25v ZGVzICogcGktPmNvcmVzX3Blcl9zb2NrZXQgKiBwaS0+dGhyZWFkc19wZXJfY29yZSk7CiAgICAg ICAgIHBpLT50b3RhbF9wYWdlcyA9IHRvdGFsX3BhZ2VzOwogICAgICAgICBwaS0+ZnJlZV9wYWdl cyA9IGF2YWlsX2RvbWhlYXBfcGFnZXMoKTsKICAgICAgICAgcGktPnNjcnViX3BhZ2VzID0gMDsK QEAgLTc0LDE1ICs3Myw1NiBAQAogICAgICAgICBpZiAoIGlvbW11X2VuYWJsZWQgKQogICAgICAg ICAgICAgcGktPmNhcGFiaWxpdGllcyB8PSBYRU5fU1lTQ1RMX1BIWVNDQVBfaHZtX2RpcmVjdGlv OwogCi0gICAgICAgIHBpLT5tYXhfbm9kZV9pZCA9IGxhc3Rfbm9kZShub2RlX29ubGluZV9tYXAp OwotICAgICAgICBwaS0+bWF4X2NwdV9pZCA9IGxhc3RfY3B1KGNwdV9vbmxpbmVfbWFwKTsKLSAg ICAgICAgbWF4X2FycmF5X2VudCA9IG1pbl90KHVpbnQzMl90LCBtYXhfYXJyYXlfZW50LCBwaS0+ bWF4X2NwdV9pZCk7CisgICAgICAgIGlmICggY29weV90b19ndWVzdCh1X3N5c2N0bCwgc3lzY3Rs LCAxKSApCisgICAgICAgICAgICByZXQgPSAtRUZBVUxUOworICAgIH0KKyAgICBicmVhazsKKyAg ICAgICAgCisgICAgY2FzZSBYRU5fU1lTQ1RMX3RvcG9sb2d5aW5mbzoKKyAgICB7CisgICAgICAg IHVpbnQzMl90IGksIG1heF9jcHVfaW5kZXg7CisgICAgICAgIFhFTl9HVUVTVF9IQU5ETEVfNjQo dWludDMyKSBjcHVfdG9fY29yZV9hcnI7CisgICAgICAgIFhFTl9HVUVTVF9IQU5ETEVfNjQodWlu dDMyKSBjcHVfdG9fc29ja2V0X2FycjsKKyAgICAgICAgWEVOX0dVRVNUX0hBTkRMRV82NCh1aW50 MzIpIGNwdV90b19ub2RlX2FycjsKKworICAgICAgICB4ZW5fc3lzY3RsX3RvcG9sb2d5aW5mb190 ICp0aSA9ICZzeXNjdGwtPnUudG9wb2xvZ3lpbmZvOworCisgICAgICAgIG1heF9jcHVfaW5kZXgg PSB0aS0+bWF4X2NwdV9pbmRleDsKKyAgICAgICAgY3B1X3RvX2NvcmVfYXJyID0gdGktPmNwdV90 b19jb3JlOworICAgICAgICBjcHVfdG9fc29ja2V0X2FyciA9IHRpLT5jcHVfdG9fc29ja2V0Owor ICAgICAgICBjcHVfdG9fbm9kZV9hcnIgPSB0aS0+Y3B1X3RvX25vZGU7CisKKyAgICAgICAgbWVt c2V0KHRpLCAwLCBzaXplb2YoKnRpKSk7CisgICAgICAgIHRpLT5jcHVfdG9fY29yZSA9IGNwdV90 b19jb3JlX2FycjsKKyAgICAgICAgdGktPmNwdV90b19zb2NrZXQgPSBjcHVfdG9fc29ja2V0X2Fy cjsKKyAgICAgICAgdGktPmNwdV90b19ub2RlID0gY3B1X3RvX25vZGVfYXJyOworCisgICAgICAg IG1heF9jcHVfaW5kZXggPSBtaW5fdCh1aW50MzJfdCwgbWF4X2NwdV9pbmRleCwgbnVtX29ubGlu ZV9jcHVzKCkpOworICAgICAgICB0aS0+bWF4X2NwdV9pbmRleCA9IG1heF9jcHVfaW5kZXg7CiAK ICAgICAgICAgcmV0ID0gMDsKIAotICAgICAgICBpZiAoICFndWVzdF9oYW5kbGVfaXNfbnVsbChj cHVfdG9fbm9kZV9hcnIpICkKKyAgICAgICAgZm9yICggaSA9IDA7IGkgPCBtYXhfY3B1X2luZGV4 OyBpKysgKQogICAgICAgICB7Ci0gICAgICAgICAgICBmb3IgKCBpID0gMDsgaSA8PSBtYXhfYXJy YXlfZW50OyBpKysgKQorICAgICAgICAgICAgaWYgKCAhZ3Vlc3RfaGFuZGxlX2lzX251bGwoY3B1 X3RvX2NvcmVfYXJyKSApCisgICAgICAgICAgICB7CisgICAgICAgICAgICAgICAgdWludDMyX3Qg Y29yZSA9IGNwdV9vbmxpbmUoaSkgPyBjcHVfdG9fY29yZShpKSA6IH4wdTsKKyAgICAgICAgICAg ICAgICBpZiAoIGNvcHlfdG9fZ3Vlc3Rfb2Zmc2V0KGNwdV90b19jb3JlX2FyciwgaSwgJmNvcmUs IDEpICkKKyAgICAgICAgICAgICAgICB7CisgICAgICAgICAgICAgICAgICAgIHJldCA9IC1FRkFV TFQ7CisgICAgICAgICAgICAgICAgICAgIGJyZWFrOworICAgICAgICAgICAgICAgIH0KKyAgICAg ICAgICAgIH0KKyAgICAgICAgICAgIGlmICggIWd1ZXN0X2hhbmRsZV9pc19udWxsKGNwdV90b19z b2NrZXRfYXJyKSApCisgICAgICAgICAgICB7CisgICAgICAgICAgICAgICAgdWludDMyX3Qgc29j a2V0ID0gY3B1X29ubGluZShpKSA/IGNwdV90b19zb2NrZXQoaSkgOiB+MHU7CisgICAgICAgICAg ICAgICAgaWYgKCBjb3B5X3RvX2d1ZXN0X29mZnNldChjcHVfdG9fc29ja2V0X2FyciwgaSwgJnNv Y2tldCwgMSkgKQorICAgICAgICAgICAgICAgIHsKKyAgICAgICAgICAgICAgICAgICAgcmV0ID0g LUVGQVVMVDsKKyAgICAgICAgICAgICAgICAgICAgYnJlYWs7CisgICAgICAgICAgICAgICAgfQor ICAgICAgICAgICAgfQorICAgICAgICAgICAgaWYgKCAhZ3Vlc3RfaGFuZGxlX2lzX251bGwoY3B1 X3RvX25vZGVfYXJyKSApCiAgICAgICAgICAgICB7CiAgICAgICAgICAgICAgICAgdWludDMyX3Qg bm9kZSA9IGNwdV9vbmxpbmUoaSkgPyBjcHVfdG9fbm9kZShpKSA6IH4wdTsKICAgICAgICAgICAg ICAgICBpZiAoIGNvcHlfdG9fZ3Vlc3Rfb2Zmc2V0KGNwdV90b19ub2RlX2FyciwgaSwgJm5vZGUs IDEpICkKQEAgLTkzLDYgKzEzMyw4MiBAQAogICAgICAgICAgICAgfQogICAgICAgICB9CiAKKyAg ICAgICAgaWYgKHJldCkKKyAgICAgICAgICAgIGJyZWFrOworIAorICAgICAgICBpZiAoIGNvcHlf dG9fZ3Vlc3QodV9zeXNjdGwsIHN5c2N0bCwgMSkgKQorICAgICAgICAgICAgcmV0ID0gLUVGQVVM VDsKKyAgICB9CisgICAgYnJlYWs7CisKKyAgICBjYXNlIFhFTl9TWVNDVExfbnVtYWluZm86Cisg ICAgeworICAgICAgICB1aW50MzJfdCBpLCBtYXhfbm9kZV9pbmRleDsKKyAgICAgICAgWEVOX0dV RVNUX0hBTkRMRV82NCh1aW50NjQpIG5vZGVfdG9fbWVtc2l6ZV9hcnI7CisgICAgICAgIFhFTl9H VUVTVF9IQU5ETEVfNjQodWludDY0KSBub2RlX3RvX21lbWZyZWVfYXJyOworICAgICAgICBYRU5f R1VFU1RfSEFORExFXzY0KHVpbnQzMikgbm9kZV90b19ub2RlX2Rpc3RhbmNlX2FycjsKKworICAg ICAgICB4ZW5fc3lzY3RsX251bWFpbmZvX3QgKm5pID0gJnN5c2N0bC0+dS5udW1haW5mbzsKKwor ICAgICAgICBtYXhfbm9kZV9pbmRleCA9IG5pLT5tYXhfbm9kZV9pbmRleDsKKyAgICAgICAgbm9k ZV90b19tZW1zaXplX2FyciA9IG5pLT5ub2RlX3RvX21lbXNpemU7CisgICAgICAgIG5vZGVfdG9f bWVtZnJlZV9hcnIgPSBuaS0+bm9kZV90b19tZW1mcmVlOworICAgICAgICBub2RlX3RvX25vZGVf ZGlzdGFuY2VfYXJyID0gbmktPm5vZGVfdG9fbm9kZV9kaXN0YW5jZTsKKworICAgICAgICBtZW1z ZXQobmksIDAsIHNpemVvZigqbmkpKTsKKyAgICAgICAgbmktPm5vZGVfdG9fbWVtc2l6ZSA9IG5v ZGVfdG9fbWVtc2l6ZV9hcnI7CisgICAgICAgIG5pLT5ub2RlX3RvX21lbWZyZWUgPSBub2RlX3Rv X21lbWZyZWVfYXJyOworICAgICAgICBuaS0+bm9kZV90b19ub2RlX2Rpc3RhbmNlID0gbm9kZV90 b19ub2RlX2Rpc3RhbmNlX2FycjsKKworICAgICAgICBtYXhfbm9kZV9pbmRleCA9IG1pbl90KHVp bnQzMl90LCBtYXhfbm9kZV9pbmRleCwgbnVtX29ubGluZV9ub2RlcygpKTsKKyAgICAgICAgbmkt Pm1heF9ub2RlX2luZGV4ID0gbWF4X25vZGVfaW5kZXg7CisKKyAgICAgICAgcmV0ID0gMDsKKwor ICAgICAgICBmb3IgKCBpID0gMDsgaSA8IG1heF9ub2RlX2luZGV4OyBpKysgKQorICAgICAgICB7 CisgICAgICAgICAgICBpZiAoICFndWVzdF9oYW5kbGVfaXNfbnVsbChub2RlX3RvX21lbXNpemVf YXJyKSApCisgICAgICAgICAgICB7CisgICAgICAgICAgICAgICAgdWludDY0X3QgbWVtc2l6ZSA9 IG5vZGVfb25saW5lKGkpID8gCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG5v ZGVfc3Bhbm5lZF9wYWdlcyhpKSA8PCBQQUdFX1NISUZUIDogMHVsOworICAgICAgICAgICAgICAg IGlmICggY29weV90b19ndWVzdF9vZmZzZXQobm9kZV90b19tZW1zaXplX2FyciwgaSwgJm1lbXNp emUsIDEpICkKKyAgICAgICAgICAgICAgICB7CisgICAgICAgICAgICAgICAgICAgIHJldCA9IC1F RkFVTFQ7CisgICAgICAgICAgICAgICAgICAgIGJyZWFrOworICAgICAgICAgICAgICAgIH0KKyAg ICAgICAgICAgIH0KKyAgICAgICAgICAgIGlmICggIWd1ZXN0X2hhbmRsZV9pc19udWxsKG5vZGVf dG9fbWVtZnJlZV9hcnIpICkKKyAgICAgICAgICAgIHsKKyAgICAgICAgICAgICAgICB1aW50NjRf dCBtZW1mcmVlID0gbm9kZV9vbmxpbmUoaSkgPyAKKyAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgYXZhaWxfbm9kZV9oZWFwX3BhZ2VzKGkpIDw8IFBBR0VfU0hJRlQgOiAwdWw7Cisg ICAgICAgICAgICAgICAgaWYgKCBjb3B5X3RvX2d1ZXN0X29mZnNldChub2RlX3RvX21lbWZyZWVf YXJyLCBpLCAmbWVtZnJlZSwgMSkgKQorICAgICAgICAgICAgICAgIHsKKyAgICAgICAgICAgICAg ICAgICAgcmV0ID0gLUVGQVVMVDsKKyAgICAgICAgICAgICAgICAgICAgYnJlYWs7CisgICAgICAg ICAgICAgICAgfQorICAgICAgICAgICAgfQorCisgICAgICAgICAgICBpZiAoICFndWVzdF9oYW5k bGVfaXNfbnVsbChub2RlX3RvX25vZGVfZGlzdGFuY2VfYXJyKSApCisJICAgIHsKKyAgICAgICAg ICAgICAgICBpbnQgajsKKyAgICAgICAgICAgICAgICBmb3IgKCBqID0gMDsgaiA8IG1heF9ub2Rl X2luZGV4OyBqKyspCisgICAgICAgICAgICAgICAgeworICAgICAgICAgICAgICAgICAgICB1aW50 MzJfdCBkaXN0YW5jZSA9IH4wdTsKKyAgICAgICAgICAgICAgICAgICAgaWYgKG5vZGVfb25saW5l KGkpICYmIG5vZGVfb25saW5lIChqKSkgCisgICAgICAgICAgICAgICAgICAgICAgICBkaXN0YW5j ZSA9IF9fbm9kZV9kaXN0YW5jZShpLCBqKTsKKyAgICAgICAgICAgICAgICAgICAgCisgICAgICAg ICAgICAgICAgICAgIGlmICggY29weV90b19ndWVzdF9vZmZzZXQobm9kZV90b19ub2RlX2Rpc3Rh bmNlX2FyciwgCisgICAgICAgICAgICAgICAgICAgICAgICAgKGkgKiBtYXhfbm9kZV9pbmRleCAr IGopLCAmZGlzdGFuY2UsIDEpICkKKyAgICAgICAgICAgICAgICAgICAgeworICAgICAgICAgICAg ICAgICAgICAgICAgcmV0ID0gLUVGQVVMVDsKKyAgICAgICAgICAgICAgICAgICAgICAgIGJyZWFr OworICAgICAgICAgICAgICAgICAgICB9CisgICAgICAgICAgICAgICAgfQorICAgICAgICAgICAg fQorICAgICAgICB9CisgICAgICAgIGlmIChyZXQpCisgICAgICAgICAgICBicmVhazsKKwogICAg ICAgICBpZiAoIGNvcHlfdG9fZ3Vlc3QodV9zeXNjdGwsIHN5c2N0bCwgMSkgKQogICAgICAgICAg ICAgcmV0ID0gLUVGQVVMVDsKICAgICB9CmRpZmYgLXIgMjYzNmU1NjE5NzA4IHhlbi9jb21tb24v cGFnZV9hbGxvYy5jCi0tLSBhL3hlbi9jb21tb24vcGFnZV9hbGxvYy5jCVR1ZSBKYW4gMjYgMTU6 NTQ6NDAgMjAxMCArMDAwMAorKysgYi94ZW4vY29tbW9uL3BhZ2VfYWxsb2MuYwlUaHUgSmFuIDI4 IDE0OjE3OjMyIDIwMTAgLTA4MDAKQEAgLTEyMTEsNiArMTIxMSwxMiBAQAogICAgICAgICAgICAg ICAgICAgICAgICAgICAgIC0xKTsKIH0KIAordW5zaWduZWQgbG9uZyBhdmFpbF9ub2RlX2hlYXBf cGFnZXModW5zaWduZWQgaW50IG5vZGVpZCkKK3sKKyAgICByZXR1cm4gYXZhaWxfaGVhcF9wYWdl cyhNRU1aT05FX1hFTiwgTlJfWk9ORVMgLTEsIG5vZGVpZCk7Cit9CisKKwogc3RhdGljIHZvaWQg cGFnZWFsbG9jX2luZm8odW5zaWduZWQgY2hhciBrZXkpCiB7CiAgICAgdW5zaWduZWQgaW50IHpv bmUgPSBNRU1aT05FX1hFTjsKZGlmZiAtciAyNjM2ZTU2MTk3MDggeGVuL2luY2x1ZGUvYXNtLXg4 Ni9udW1hLmgKLS0tIGEveGVuL2luY2x1ZGUvYXNtLXg4Ni9udW1hLmgJVHVlIEphbiAyNiAxNTo1 NDo0MCAyMDEwICswMDAwCisrKyBiL3hlbi9pbmNsdWRlL2FzbS14ODYvbnVtYS5oCVRodSBKYW4g MjggMTQ6MTc6MzIgMjAxMCAtMDgwMApAQCAtNzMsNiArNzMsNyBAQAogI2RlZmluZSBOT0RFX0RB VEEobmlkKQkJKCYobm9kZV9kYXRhW25pZF0pKQogCiAjZGVmaW5lIG5vZGVfc3RhcnRfcGZuKG5p ZCkJKE5PREVfREFUQShuaWQpLT5ub2RlX3N0YXJ0X3BmbikKKyNkZWZpbmUgbm9kZV9zcGFubmVk X3BhZ2VzKG5pZCkJKE5PREVfREFUQShuaWQpLT5ub2RlX3NwYW5uZWRfcGFnZXMpCiAjZGVmaW5l IG5vZGVfZW5kX3BmbihuaWQpICAgICAgIChOT0RFX0RBVEEobmlkKS0+bm9kZV9zdGFydF9wZm4g KyBcCiAJCQkJIE5PREVfREFUQShuaWQpLT5ub2RlX3NwYW5uZWRfcGFnZXMpCiAKZGlmZiAtciAy NjM2ZTU2MTk3MDggeGVuL2luY2x1ZGUvcHVibGljL3N5c2N0bC5oCi0tLSBhL3hlbi9pbmNsdWRl L3B1YmxpYy9zeXNjdGwuaAlUdWUgSmFuIDI2IDE1OjU0OjQwIDIwMTAgKzAwMDAKKysrIGIveGVu L2luY2x1ZGUvcHVibGljL3N5c2N0bC5oCVRodSBKYW4gMjggMTQ6MTc6MzIgMjAxMCAtMDgwMApA QCAtMzQsNyArMzQsNyBAQAogI2luY2x1ZGUgInhlbi5oIgogI2luY2x1ZGUgImRvbWN0bC5oIgog Ci0jZGVmaW5lIFhFTl9TWVNDVExfSU5URVJGQUNFX1ZFUlNJT04gMHgwMDAwMDAwNworI2RlZmlu ZSBYRU5fU1lTQ1RMX0lOVEVSRkFDRV9WRVJTSU9OIDB4MDAwMDAwMDgKIAogLyoKICAqIFJlYWQg Y29uc29sZSBjb250ZW50IGZyb20gWGVuIGJ1ZmZlciByaW5nLgpAQCAtOTMsMzAgKzkzLDE1IEBA CiBzdHJ1Y3QgeGVuX3N5c2N0bF9waHlzaW5mbyB7CiAgICAgdWludDMyX3QgdGhyZWFkc19wZXJf Y29yZTsKICAgICB1aW50MzJfdCBjb3Jlc19wZXJfc29ja2V0OworICAgIHVpbnQzMl90IHNvY2tl dHNfcGVyX25vZGU7CiAgICAgdWludDMyX3QgbnJfY3B1czsKLSAgICB1aW50MzJfdCBtYXhfbm9k ZV9pZDsKKyAgICB1aW50MzJfdCBucl9ub2RlczsKICAgICB1aW50MzJfdCBjcHVfa2h6OwogICAg IHVpbnQ2NF9hbGlnbmVkX3QgdG90YWxfcGFnZXM7CiAgICAgdWludDY0X2FsaWduZWRfdCBmcmVl X3BhZ2VzOwogICAgIHVpbnQ2NF9hbGlnbmVkX3Qgc2NydWJfcGFnZXM7CiAgICAgdWludDMyX3Qg aHdfY2FwWzhdOwogCi0gICAgLyoKLSAgICAgKiBJTjogbWF4aW11bSBhZGRyZXNzYWJsZSBlbnRy eSBpbiB0aGUgY2FsbGVyLXByb3ZpZGVkIGNwdV90b19ub2RlIGFycmF5LgotICAgICAqIE9VVDog bGFyZ2VzdCBjcHUgaWRlbnRpZmllciBpbiB0aGUgc3lzdGVtLgotICAgICAqIElmIE9VVCBpcyBn cmVhdGVyIHRoYW4gSU4gdGhlbiB0aGUgY3B1X3RvX25vZGUgYXJyYXkgaXMgdHJ1bmNhdGVkIQot ICAgICAqLwotICAgIHVpbnQzMl90IG1heF9jcHVfaWQ7Ci0gICAgLyoKLSAgICAgKiBJZiBub3Qg TlVMTCwgdGhpcyBhcnJheSBpcyBmaWxsZWQgd2l0aCBub2RlIGlkZW50aWZpZXIgZm9yIGVhY2gg Y3B1LgotICAgICAqIElmIGEgY3B1IGhhcyBubyBub2RlIGluZm9ybWF0aW9uIChlLmcuLCBjcHUg bm90IHByZXNlbnQpIHRoZW4gdGhlCi0gICAgICogc2VudGluZWwgdmFsdWUgfjB1IGlzIHdyaXR0 ZW4uCi0gICAgICogVGhlIHNpemUgb2YgdGhpcyBhcnJheSBpcyBzcGVjaWZpZWQgYnkgdGhlIGNh bGxlciBpbiBAbWF4X2NwdV9pZC4KLSAgICAgKiBJZiB0aGUgYWN0dWFsIEBtYXhfY3B1X2lkIGlz IHNtYWxsZXIgdGhhbiB0aGUgYXJyYXkgdGhlbiB0aGUgdHJhaWxpbmcKLSAgICAgKiBlbGVtZW50 cyBvZiB0aGUgYXJyYXkgd2lsbCBub3QgYmUgd3JpdHRlbiBieSB0aGUgc3lzY3RsLgotICAgICAq LwotICAgIFhFTl9HVUVTVF9IQU5ETEVfNjQodWludDMyKSBjcHVfdG9fbm9kZTsKLQogICAgIC8q IFhFTl9TWVNDVExfUEhZU0NBUF8/Pz8gKi8KICAgICB1aW50MzJfdCBjYXBhYmlsaXRpZXM7CiB9 OwpAQCAtNDg2LDYgKzQ3MSw3MyBAQAogdHlwZWRlZiBzdHJ1Y3QgeGVuX3N5c2N0bF9sb2NrcHJv Zl9vcCB4ZW5fc3lzY3RsX2xvY2twcm9mX29wX3Q7CiBERUZJTkVfWEVOX0dVRVNUX0hBTkRMRSh4 ZW5fc3lzY3RsX2xvY2twcm9mX29wX3QpOwogCisjZGVmaW5lIFhFTl9TWVNDVExfdG9wb2xvZ3lp bmZvICAgICAgICAgMTYgCitzdHJ1Y3QgeGVuX3N5c2N0bF90b3BvbG9neWluZm8geworCisgICAg LyoKKyAgICAgKiBJTjogbWF4aW11bSBhZGRyZXNzYWJsZSBlbnRyeSBpbiB0aGUgY2FsbGVyLXBy b3ZpZGVkIGNwdV90b19jb3JlLCAKKyAgICAgKiBjcHVfdG9fc29ja2V0ICYgY3B1X3RvX25vZGUg YXJyYXlzLgorICAgICAqIE9VVDogbGFyZ2VzdCBjcHUgaWRlbnRpZmllciBpbiB0aGUgc3lzdGVt LgorICAgICAqIElmIE9VVCBpcyBncmVhdGVyIHRoYW4gSU4gdGhlbiB0aGUgY3B1X3RvX25vZGUg YXJyYXkgaXMgdHJ1bmNhdGVkIQorICAgICAqLworICAgIHVpbnQzMl90IG1heF9jcHVfaW5kZXg7 CisKKyAgICAvKgorICAgICAqIElmIG5vdCBOVUxMLCB0aGlzIGFycmF5IGlzIGZpbGxlZCB3aXRo IGNvcmUvc29ja2V0L25vZGUgaWRlbnRpZmllciBmb3IgCisgICAgICogZWFjaCBjcHUuCisgICAg ICogSWYgYSBjcHUgaGFzIG5vIGNvcmUvc29ja2V0L25vZGUgaW5mb3JtYXRpb24gKGUuZy4sIGNw dSBub3QgcHJlc2VudCkgCisgICAgICogdGhlbiB0aGUgc2VudGluZWwgdmFsdWUgfjB1IGlzIHdy aXR0ZW4uCisgICAgICogVGhlIHNpemUgb2YgdGhpcyBhcnJheSBpcyBzcGVjaWZpZWQgYnkgdGhl IGNhbGxlciBpbiBAbWF4X2NwdV9pbmRleC4KKyAgICAgKiBJZiB0aGUgYWN0dWFsIEBtYXhfY3B1 X2luZGV4IGlzIHNtYWxsZXIgdGhhbiB0aGUgYXJyYXkgdGhlbiB0aGUgdHJhaWxpbmcKKyAgICAg KiBlbGVtZW50cyBvZiB0aGUgYXJyYXkgd2lsbCBub3QgYmUgd3JpdHRlbiBieSB0aGUgc3lzY3Rs LgorICAgICAqLworICAgIFhFTl9HVUVTVF9IQU5ETEVfNjQodWludDMyKSBjcHVfdG9fY29yZTsK KyAgICBYRU5fR1VFU1RfSEFORExFXzY0KHVpbnQzMikgY3B1X3RvX3NvY2tldDsKKyAgICBYRU5f R1VFU1RfSEFORExFXzY0KHVpbnQzMikgY3B1X3RvX25vZGU7ICAvKiBub2RlX251bWJlciAqLwor Cit9OwordHlwZWRlZiBzdHJ1Y3QgeGVuX3N5c2N0bF90b3BvbG9neWluZm8geGVuX3N5c2N0bF90 b3BvbG9neWluZm9fdDsKK0RFRklORV9YRU5fR1VFU1RfSEFORExFKHhlbl9zeXNjdGxfdG9wb2xv Z3lpbmZvX3QpOworCisjZGVmaW5lIFhFTl9TWVNDVExfbnVtYWluZm8gICAgICAgICAgMTcJCitz dHJ1Y3QgeGVuX3N5c2N0bF9udW1haW5mbyB7CisgICAgLyoKKyAgICAgKiBJTjogbWF4aW11bSBh ZGRyZXNzYWJsZSBlbnRyeSBpbiB0aGUgY2FsbGVyLXByb3ZpZGVkIG5vZGVfbnVtYmVycywgCisg ICAgICogbm9kZV90b19tZW1zaXplICYgbm9kZV90b19tZW1mcmVlIGFycmF5cy4KKyAgICAgKiBP VVQ6IGxhcmdlc3QgcG9zc2libGUgbm9kZSBpbmRleCBmb3IgdGhlIHN5c3RlbS4KKyAgICAgKiBJ ZiBPVVQgaXMgZ3JlYXRlciB0aGFuIElOIHRoZW4gdGhlc2UgYXJyYXlzIGFyZSB0cnVuY2F0ZWQh CisgICAgICovCisgICAgdWludDMyX3QgbWF4X25vZGVfaW5kZXg7CisKKyAgICAvKiBGb3Igbm9k ZV90b19tZW1zaXplICYgbm9kZV90b19tZW1mcmVlIGFycmF5cywgdGhlIAorICAgICAqIGVudHJ5 IHdpdGggc2FtZSBpbmRleCBjb3Jyb3Nwb25kcyB0byB0aGUgc2FtZSBub2RlLgorICAgICAqIElm IGEgZW50cnkgaGFzIG5vIG5vZGUgaW5mb3JtYXRpb24gKGUuZy4sIG5vZGUgbm90IHByZXNlbnQp IHRoZW4gdGhlIAorICAgICAqIHNlbnRpbmVsIHZhbHVlIH4wdSBpcyB3cml0dGVuIGZvcl9ub2Rl X251bWJlciwgYW5kIHZhbHVlIDB1IGlzIHdyaXR0ZW4gCisgICAgICogZm9yIG5vZGVfdG9fbWVt c2l6ZSAmIG5vZGVfdG9fbWVtZnJlZS4KKyAgICAgKiBUaGUgc2l6ZSBvZiB0aGlzIGFycmF5IGlz IHNwZWNpZmllZCBieSB0aGUgY2FsbGVyIGluIEBtYXhfbm9kZV9pbmRleC4gCisgICAgICogSWYg dGhlIGFjdHVhbCBAbWF4X25vZGVfaW5kZXggaXMgc21hbGxlciB0aGFuIHRoZSBhcnJheSB0aGVu IHRoZSAKKyAgICAgKiB0cmFpbGluZyBlbGVtZW50cyBvZiB0aGUgYXJyYXkgd2lsbCBub3QgYmUg d3JpdHRlbiBieSB0aGUgc3lzY3RsLgorICAgICAqLworICAgIFhFTl9HVUVTVF9IQU5ETEVfNjQo dWludDY0KSBub2RlX3RvX21lbXNpemU7CisgICAgWEVOX0dVRVNUX0hBTkRMRV82NCh1aW50NjQp IG5vZGVfdG9fbWVtZnJlZTsKKworCisgICAgLyogbm9kZV90b19ub2RlX2Rpc3RhbmNlIGlzIGFy cmF5IG9mIHNpemUgKG5yX25vZGVzICogbnJfbm9kZXMpIGxpc3RpbmcKKyAgICAgKiBtZW1vcnkg YWNjZXNzIGRpc3RhbmNlcyBiZXR3ZWVuIG5vZGVzLiBpJ3RoICBlbnRyeSBpbiB0aGUgYXJyYXkg CisgICAgICogc3BlY2lmaWVzIGRpc3RhbmNlIGJldHdlZW4gbm9kZSAoaSAvIG5yX25vZGVzKSAm IG5vZGUgKGkgJSBucl9ub2RlcykKKyAgICAgKiBJZiBhIGVudHJ5IGhhcyBubyBub2RlIGRpc3Rh bmNlIGluZm9ybWF0aW9uIChlLmcuLCBub2RlIG5vdCBwcmVzZW50KSAKKyAgICAgKiB0aGVuIHRo ZSBzZW50aW5lbCB2YWx1ZSB+MHUgaXMgd3JpdHRlbi4KKyAgICAgKiBUaGUgc2l6ZSBvZiB0aGlz IGFycmF5IGlzIHNwZWNpZmllZCBieSB0aGUgY2FsbGVyIGluIAorICAgICAqIEBtYXhfbm9kZV9k aXN0YW5jZV9pbmRleC4gSWYgdGhlIG1heF9ub2RlX2luZGV4Km1heF9ub2RlX2luZGV4IGlzIAor ICAgICAqIHNtYWxsZXIgdGhhbiB0aGUgYXJyYXkgdGhlbiB0aGUgdHJhaWxpbmcgZWxlbWVudHMg b2YgdGhlIGFycmF5IHdpbGwgCisgICAgICogbm90IGJlIHdyaXR0ZW4gYnkgdGhlIHN5c2N0bC4K KyAgICAgKi8KKyAgICBYRU5fR1VFU1RfSEFORExFXzY0KHVpbnQzMikgbm9kZV90b19ub2RlX2Rp c3RhbmNlOworfTsKK3R5cGVkZWYgc3RydWN0IHhlbl9zeXNjdGxfbnVtYWluZm8geGVuX3N5c2N0 bF9udW1haW5mb190OworREVGSU5FX1hFTl9HVUVTVF9IQU5ETEUoeGVuX3N5c2N0bF9udW1haW5m b190KTsKKworCiBzdHJ1Y3QgeGVuX3N5c2N0bCB7CiAgICAgdWludDMyX3QgY21kOwogICAgIHVp bnQzMl90IGludGVyZmFjZV92ZXJzaW9uOyAvKiBYRU5fU1lTQ1RMX0lOVEVSRkFDRV9WRVJTSU9O ICovCkBAIC00OTMsNiArNTQ1LDggQEAKICAgICAgICAgc3RydWN0IHhlbl9zeXNjdGxfcmVhZGNv bnNvbGUgICAgICAgcmVhZGNvbnNvbGU7CiAgICAgICAgIHN0cnVjdCB4ZW5fc3lzY3RsX3RidWZf b3AgICAgICAgICAgIHRidWZfb3A7CiAgICAgICAgIHN0cnVjdCB4ZW5fc3lzY3RsX3BoeXNpbmZv ICAgICAgICAgIHBoeXNpbmZvOworICAgICAgICBzdHJ1Y3QgeGVuX3N5c2N0bF90b3BvbG9neWlu Zm8gICAgICB0b3BvbG9neWluZm87CisgICAgICAgIHN0cnVjdCB4ZW5fc3lzY3RsX251bWFpbmZv ICAgICAgICAgIG51bWFpbmZvOwogICAgICAgICBzdHJ1Y3QgeGVuX3N5c2N0bF9zY2hlZF9pZCAg ICAgICAgICBzY2hlZF9pZDsKICAgICAgICAgc3RydWN0IHhlbl9zeXNjdGxfcGVyZmNfb3AgICAg ICAgICAgcGVyZmNfb3A7CiAgICAgICAgIHN0cnVjdCB4ZW5fc3lzY3RsX2dldGRvbWFpbmluZm9s aXN0IGdldGRvbWFpbmluZm9saXN0OwpkaWZmIC1yIDI2MzZlNTYxOTcwOCB4ZW4vaW5jbHVkZS94 ZW4vbW0uaAotLS0gYS94ZW4vaW5jbHVkZS94ZW4vbW0uaAlUdWUgSmFuIDI2IDE1OjU0OjQwIDIw MTAgKzAwMDAKKysrIGIveGVuL2luY2x1ZGUveGVuL21tLmgJVGh1IEphbiAyOCAxNDoxNzozMiAy MDEwIC0wODAwCkBAIC01Nyw2ICs1Nyw3IEBACiB1bnNpZ25lZCBsb25nIGF2YWlsX2RvbWhlYXBf cGFnZXNfcmVnaW9uKAogICAgIHVuc2lnbmVkIGludCBub2RlLCB1bnNpZ25lZCBpbnQgbWluX3dp ZHRoLCB1bnNpZ25lZCBpbnQgbWF4X3dpZHRoKTsKIHVuc2lnbmVkIGxvbmcgYXZhaWxfZG9taGVh cF9wYWdlcyh2b2lkKTsKK3Vuc2lnbmVkIGxvbmcgYXZhaWxfbm9kZV9oZWFwX3BhZ2VzKHVuc2ln bmVkIGludCk7CiAjZGVmaW5lIGFsbG9jX2RvbWhlYXBfcGFnZShkLGYpIChhbGxvY19kb21oZWFw X3BhZ2VzKGQsMCxmKSkKICNkZWZpbmUgZnJlZV9kb21oZWFwX3BhZ2UocCkgIChmcmVlX2RvbWhl YXBfcGFnZXMocCwwKSkKIHVuc2lnbmVkIGludCBvbmxpbmVfcGFnZSh1bnNpZ25lZCBsb25nIG1m biwgdWludDMyX3QgKnN0YXR1cyk7Cg== --_004_8EA2C2C4116BF44AB370468FBF85A7770123904A29orsmsx504amrc_ Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --_004_8EA2C2C4116BF44AB370468FBF85A7770123904A29orsmsx504amrc_-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Keir Fraser Subject: Re: Host Numa informtion in dom0 Date: Sat, 30 Jan 2010 08:09:10 +0000 Message-ID: References: <8EA2C2C4116BF44AB370468FBF85A7770123904A29@orsmsx504.amr.corp.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <8EA2C2C4116BF44AB370468FBF85A7770123904A29@orsmsx504.amr.corp.intel.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: "Kamble, Nitin A" , "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org I'll apply post 4.0.0. Please also supply a signed-off-by line. -- Keir On 29/01/2010 23:05, "Kamble, Nitin A" wrote: > Hi Keir, > Attached is the patch which exposes the host numa information to dom0.= With > the patch =B3xm info=B2 command now also gives the cpu topology & host numa > information. This will be later used to build guest numa support. > The patch basically changes physinfo sysctl, and adds topology_info & > numa_info sysctls, and also changes the python & libxc code accordingly. > =20 > Please apply. > =20 > Thanks & Regards, > Nitin > =20 > =20 >=20 From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Kamble, Nitin A" Subject: RE: Host Numa informtion in dom0 Date: Sun, 31 Jan 2010 18:21:27 -0800 Message-ID: <8EA2C2C4116BF44AB370468FBF85A7770123994784@orsmsx504.amr.corp.intel.com> References: <8EA2C2C4116BF44AB370468FBF85A7770123904A29@orsmsx504.amr.corp.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: Content-Language: en-US List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Keir Fraser , "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org Thanks Keir, Signed-Off-by: Nitin A Kamble Regards, Nitin -----Original Message----- From: Keir Fraser [mailto:keir.fraser@eu.citrix.com]=20 Sent: Saturday, January 30, 2010 12:09 AM To: Kamble, Nitin A; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Host Numa informtion in dom0 I'll apply post 4.0.0. Please also supply a signed-off-by line. -- Keir On 29/01/2010 23:05, "Kamble, Nitin A" wrote: > Hi Keir, > Attached is the patch which exposes the host numa information to dom0.= With > the patch =B3xm info=B2 command now also gives the cpu topology & host nu= ma > information. This will be later used to build guest numa support. > The patch basically changes physinfo sysctl, and adds topology_info & > numa_info sysctls, and also changes the python & libxc code accordingly. > =20 > Please apply. > =20 > Thanks & Regards, > Nitin > =20 > =20 >=20 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andre Przywara Subject: Re: Host Numa informtion in dom0 Date: Mon, 1 Feb 2010 11:23:04 +0100 Message-ID: <4B66AB88.6090208@amd.com> References: <8EA2C2C4116BF44AB370468FBF85A7770123904A29@orsmsx504.amr.corp.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <8EA2C2C4116BF44AB370468FBF85A7770123904A29@orsmsx504.amr.corp.intel.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: "Kamble, Nitin A" Cc: "xen-devel@lists.xensource.com" , Keir Fraser List-Id: xen-devel@lists.xenproject.org Kamble, Nitin A wrote: > Hi Keir, >=20 > Attached is the patch which exposes the host numa information to=20 > dom0. With the patch =93xm info=94 command now also gives the cpu topol= ogy &=20 > host numa information. This will be later used to build guest numa supp= ort. What information are you missing from the current physinfo? As far as I=20 can see, only the total amount of memory per node is not provided. But=20 one could get this info from parsing the SRAT table in Dom0, which is at=20 least mapped into Dom0's memory. Or do you want to provide NUMA information to all PV guests (but then it=20 cannot be a sysctl)? This would be helpful, as this would avoid to=20 enable ACPI parsing in PV Linux for NUMA guest support. Beside that I have to oppose the introduction of sockets_per_node again.=20 Future AMD processors will feature _two_ nodes on _one_ socket, so this=20 variable should hold 1/2, but this will be rounded to zero. I think this=20 information is pretty useless anyway, as the number of sockets is mostly=20 interesting for licensing purposes, where a single number is sufficient.=20 For scheduling purposes cache topology is more important. My NUMA guest patches (currently for HVM only) are doing fine, I will=20 try to send out a RFC patches this week. I think they don't interfere=20 with this patch, but if you have other patches in development, we should=20 sync on this. The scope of my patches is to let the user (or xend) describe a guest's=20 topology (either by specifying only the number of guest nodes in the=20 config file or by explicitly describing the whole NUMA topology). Some=20 code will assign host nodes to the guest nodes (I am not sure yet=20 whether this really belongs into xend as it currently does, or is better=20 done in libxc, where libxenlight would also benefit). Then libxc's hvm_build_* will pass that info into the hvm_info_table,=20 where code in the hvmloader will generate an appropriate SRAT table. An extension of this would be to let Xen automatically decide whether a=20 split of the resources is necessary (because there is not enough memory=20 available (anymore) on one node). Looking forward to comments... Regards, Andre. --=20 Andre Przywara AMD-Operating System Research Center (OSRC), Dresden, Germany Tel: +49 351 448 3567 12 ----to satisfy European Law for business letters: Advanced Micro Devices GmbH Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen Geschaeftsfuehrer: Andrew Bowd; Thomas M. McCoy; Giuliano Meroni Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen Registergericht Muenchen, HRB Nr. 43632 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dulloor Subject: Re: Host Numa informtion in dom0 Date: Mon, 1 Feb 2010 12:53:37 -0500 Message-ID: <940bcfd21002010953y74e43db7h838f5021207bfa8f@mail.gmail.com> References: <8EA2C2C4116BF44AB370468FBF85A7770123904A29@orsmsx504.amr.corp.intel.com> <4B66AB88.6090208@amd.com> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <4B66AB88.6090208@amd.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Andre Przywara Cc: "xen-devel@lists.xensource.com" , Keir Fraser List-Id: xen-devel@lists.xenproject.org > Beside that I have to oppose the introduction of sockets_per_node again. > Future AMD processors will feature _two_ nodes on _one_ socket, so this > variable should hold 1/2, but this will be rounded to zero. I think this > information is pretty useless anyway, as the number of sockets is mostly > interesting for licensing purposes, where a single number is sufficient. I sent a similar patch (was using to enlist pcpu-tuples and in vcpu-pin/unpin) and I didn't pursue it because of this same argument. When we talk of cpu topology, that's how it is currently : nodes-socket-cpu-core. Don't sockets also figure in the cache and interconnect hierarchy ? What would be the hierarchy in those future AMD processors ? Even Keir and Ian Pratt initially wanted the pcpu-tuples to be listed that way. So, it would be helpful to make a call and move ahea= d. -dulloor On Mon, Feb 1, 2010 at 5:23 AM, Andre Przywara wro= te: > Kamble, Nitin A wrote: >> >> Hi Keir, >> >> =A0 Attached is the patch which exposes the host numa information to dom= 0. >> With the patch =93xm info=94 command now also gives the cpu topology & h= ost numa >> information. This will be later used to build guest numa support. > > What information are you missing from the current physinfo? As far as I c= an > see, only the total amount of memory per node is not provided. But one co= uld > get this info from parsing the SRAT table in Dom0, which is at least mapp= ed > into Dom0's memory. > Or do you want to provide NUMA information to all PV guests (but then it > cannot be a sysctl)? This would be helpful, as this would avoid to enable > ACPI parsing in PV Linux for NUMA guest support. > > Beside that I have to oppose the introduction of sockets_per_node again. > Future AMD processors will feature _two_ nodes on _one_ socket, so this > variable should hold 1/2, but this will be rounded to zero. I think this > information is pretty useless anyway, as the number of sockets is mostly > interesting for licensing purposes, where a single number is sufficient. > =A0For scheduling purposes cache topology is more important. > > My NUMA guest patches (currently for HVM only) are doing fine, I will try= to > send out a RFC patches this week. I think they don't interfere with this > patch, but if you have other patches in development, we should sync on th= is. > The scope of my patches is to let the user (or xend) describe a guest's > =A0topology (either by specifying only the number of guest nodes in the c= onfig > file or by explicitly describing the whole NUMA topology). Some code will > assign host nodes to the guest nodes (I am not sure yet whether this real= ly > belongs into xend as it currently does, or is better done in libxc, where > libxenlight would also benefit). > Then libxc's hvm_build_* will pass that info into the hvm_info_table, whe= re > code in the hvmloader will generate an appropriate SRAT table. > An extension of this would be to let Xen automatically decide whether a > split of the resources is necessary (because there is not enough memory > available (anymore) on one node). > > Looking forward to comments... > > Regards, > Andre. > > -- > Andre Przywara > AMD-Operating System Research Center (OSRC), Dresden, Germany > Tel: +49 351 448 3567 12 > ----to satisfy European Law for business letters: > Advanced Micro Devices GmbH > Karl-Hammerschmidt-Str. 34, 85609 Dornach b. Muenchen > Geschaeftsfuehrer: Andrew Bowd; Thomas M. McCoy; Giuliano Meroni > Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen > Registergericht Muenchen, HRB Nr. 43632 > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andre Przywara Subject: Re: Host Numa informtion in dom0 Date: Mon, 1 Feb 2010 22:39:36 +0100 Message-ID: <4B674A18.8010106@amd.com> References: <8EA2C2C4116BF44AB370468FBF85A7770123904A29@orsmsx504.amr.corp.intel.com> <4B66AB88.6090208@amd.com> <940bcfd21002010953y74e43db7h838f5021207bfa8f@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <940bcfd21002010953y74e43db7h838f5021207bfa8f@mail.gmail.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Dulloor Cc: "xen-devel@lists.xensource.com" , Keir Fraser List-Id: xen-devel@lists.xenproject.org Dulloor wrote: >> Beside that I have to oppose the introduction of sockets_per_node agai= n. >> Future AMD processors will feature _two_ nodes on _one_ socket, so thi= s >> variable should hold 1/2, but this will be rounded to zero. I think th= is >> information is pretty useless anyway, as the number of sockets is most= ly >> interesting for licensing purposes, where a single number is sufficien= t. >=20 > I sent a similar patch (was using to enlist pcpu-tuples and in > vcpu-pin/unpin) and I didn't pursue it because of this same argument. > When we talk of cpu topology, that's how it is currently : > nodes-socket-cpu-core. Don't sockets also figure in the cache and > interconnect hierarchy ? Not necessarily. Think of Intel's Core2Quad, they have two separate L2=20 caches each associated to two of the four cores in one socket. If you=20 move from core0 to core2 then AFAIK the cost would be very similar to=20 moving to another processor socket. So in fact the term socket does not=20 help here. The situation is similar to the new AMD CPUs, just that it replaces "L2=20 cache" with "node" (aka shared memory controller, which also matches=20 shared L3 cache). In fact the cost of moving from one node to the=20 neighbor in the same socket is exactly the same as moving to another=20 socket. > What would be the hierarchy in those future AMD processors ? Even Keir > and Ian Pratt initially wanted the pcpu-tuples > to be listed that way. So, it would be helpful to make a call and move = ahead. You could create variables like cores_per_socket and cores_per_node,=20 this would solve this issue for now. Actually better would be an array=20 mapping cores (or threads) to {nodes,sockets,L[123]_caches}, as this=20 would allow asymmetrical configurations (useful for guests). In the past there once was a socket_per_node value in physinfo, but it=20 has been removed. It was not used anywhere, and multiplying the whole=20 chain of x_per_y sometimes ended up in wrong values anyway. Anyway, if you insist on this value it will hold bogus values for the=20 upcoming processors. If it will be zero, you end up in trouble when=20 multiplying or dividing with it, and letting it be one is also wrong. I am sorry to spoil this whole game, but that it's how it is. If you or Nitin show me how the socket_per_node variable should be used,=20 we can maybe find a pleasing solution. Regards, Andre. >=20 > On Mon, Feb 1, 2010 at 5:23 AM, Andre Przywara = wrote: >> Kamble, Nitin A wrote: >>> Hi Keir, >>> >>> Attached is the patch which exposes the host numa information to do= m0. >>> With the patch =93xm info=94 command now also gives the cpu topology = & host numa >>> information. This will be later used to build guest numa support. >> What information are you missing from the current physinfo? As far as = I can >> see, only the total amount of memory per node is not provided. But one= could >> get this info from parsing the SRAT table in Dom0, which is at least m= apped >> into Dom0's memory. >> Or do you want to provide NUMA information to all PV guests (but then = it >> cannot be a sysctl)? This would be helpful, as this would avoid to ena= ble >> ACPI parsing in PV Linux for NUMA guest support. >> >> Beside that I have to oppose the introduction of sockets_per_node agai= n. >> Future AMD processors will feature _two_ nodes on _one_ socket, so thi= s >> variable should hold 1/2, but this will be rounded to zero. I think th= is >> information is pretty useless anyway, as the number of sockets is most= ly >> interesting for licensing purposes, where a single number is sufficien= t. >> For scheduling purposes cache topology is more important. >> >> My NUMA guest patches (currently for HVM only) are doing fine, I will = try to >> send out a RFC patches this week. I think they don't interfere with th= is >> patch, but if you have other patches in development, we should sync on= this. >> The scope of my patches is to let the user (or xend) describe a guest'= s >> topology (either by specifying only the number of guest nodes in the = config >> file or by explicitly describing the whole NUMA topology). Some code w= ill >> assign host nodes to the guest nodes (I am not sure yet whether this r= eally >> belongs into xend as it currently does, or is better done in libxc, wh= ere >> libxenlight would also benefit). >> Then libxc's hvm_build_* will pass that info into the hvm_info_table, = where >> code in the hvmloader will generate an appropriate SRAT table. >> An extension of this would be to let Xen automatically decide whether = a >> split of the resources is necessary (because there is not enough memor= y >> available (anymore) on one node). >> >> Looking forward to comments... >> >> Regards, >> Andre. >> From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Kamble, Nitin A" Subject: RE: Host Numa informtion in dom0 Date: Mon, 1 Feb 2010 15:21:58 -0800 Message-ID: <8EA2C2C4116BF44AB370468FBF85A7770123995244@orsmsx504.amr.corp.intel.com> References: <8EA2C2C4116BF44AB370468FBF85A7770123904A29@orsmsx504.amr.corp.intel.com> <4B66AB88.6090208@amd.com> <940bcfd21002010953y74e43db7h838f5021207bfa8f@mail.gmail.com> <4B674A18.8010106@amd.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <4B674A18.8010106@amd.com> Content-Language: en-US List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Andre Przywara , Dulloor Cc: Keir, "xen-devel@lists.xensource.com" , Fraser List-Id: xen-devel@lists.xenproject.org Andre, Dulloor, Some of us are also busy cooking guest numa patches for xen. I think we s= hould sync up, so that it works well for both.=20 And sockets_per_node can be taken out if it is issue to you. That was add= ed to assist the user in specifying the numa topology for the guest. It is = not strictly required, and can be taken out without any harm. Thanks & Regards, Nitin -----Original Message----- From: Andre Przywara [mailto:andre.przywara@amd.com]=20 Sent: Monday, February 01, 2010 1:40 PM To: Dulloor Cc: Kamble, Nitin A; xen-devel@lists.xensource.com; Keir Fraser Subject: Re: [Xen-devel] Host Numa informtion in dom0 Dulloor wrote: >> Beside that I have to oppose the introduction of sockets_per_node again. >> Future AMD processors will feature _two_ nodes on _one_ socket, so this >> variable should hold 1/2, but this will be rounded to zero. I think this >> information is pretty useless anyway, as the number of sockets is mostly >> interesting for licensing purposes, where a single number is sufficient. >=20 > I sent a similar patch (was using to enlist pcpu-tuples and in > vcpu-pin/unpin) and I didn't pursue it because of this same argument. > When we talk of cpu topology, that's how it is currently : > nodes-socket-cpu-core. Don't sockets also figure in the cache and > interconnect hierarchy ? Not necessarily. Think of Intel's Core2Quad, they have two separate L2=20 caches each associated to two of the four cores in one socket. If you=20 move from core0 to core2 then AFAIK the cost would be very similar to=20 moving to another processor socket. So in fact the term socket does not=20 help here. The situation is similar to the new AMD CPUs, just that it replaces "L2=20 cache" with "node" (aka shared memory controller, which also matches=20 shared L3 cache). In fact the cost of moving from one node to the=20 neighbor in the same socket is exactly the same as moving to another=20 socket. > What would be the hierarchy in those future AMD processors ? Even Keir > and Ian Pratt initially wanted the pcpu-tuples > to be listed that way. So, it would be helpful to make a call and move ah= ead. You could create variables like cores_per_socket and cores_per_node,=20 this would solve this issue for now. Actually better would be an array=20 mapping cores (or threads) to {nodes,sockets,L[123]_caches}, as this=20 would allow asymmetrical configurations (useful for guests). In the past there once was a socket_per_node value in physinfo, but it=20 has been removed. It was not used anywhere, and multiplying the whole=20 chain of x_per_y sometimes ended up in wrong values anyway. Anyway, if you insist on this value it will hold bogus values for the=20 upcoming processors. If it will be zero, you end up in trouble when=20 multiplying or dividing with it, and letting it be one is also wrong. I am sorry to spoil this whole game, but that it's how it is. If you or Nitin show me how the socket_per_node variable should be used,=20 we can maybe find a pleasing solution. Regards, Andre. >=20 > On Mon, Feb 1, 2010 at 5:23 AM, Andre Przywara w= rote: >> Kamble, Nitin A wrote: >>> Hi Keir, >>> >>> Attached is the patch which exposes the host numa information to dom0= . >>> With the patch "xm info" command now also gives the cpu topology & host= numa >>> information. This will be later used to build guest numa support. >> What information are you missing from the current physinfo? As far as I = can >> see, only the total amount of memory per node is not provided. But one c= ould >> get this info from parsing the SRAT table in Dom0, which is at least map= ped >> into Dom0's memory. >> Or do you want to provide NUMA information to all PV guests (but then it >> cannot be a sysctl)? This would be helpful, as this would avoid to enabl= e >> ACPI parsing in PV Linux for NUMA guest support. >> >> Beside that I have to oppose the introduction of sockets_per_node again. >> Future AMD processors will feature _two_ nodes on _one_ socket, so this >> variable should hold 1/2, but this will be rounded to zero. I think this >> information is pretty useless anyway, as the number of sockets is mostly >> interesting for licensing purposes, where a single number is sufficient. >> For scheduling purposes cache topology is more important. >> >> My NUMA guest patches (currently for HVM only) are doing fine, I will tr= y to >> send out a RFC patches this week. I think they don't interfere with this >> patch, but if you have other patches in development, we should sync on t= his. >> The scope of my patches is to let the user (or xend) describe a guest's >> topology (either by specifying only the number of guest nodes in the co= nfig >> file or by explicitly describing the whole NUMA topology). Some code wil= l >> assign host nodes to the guest nodes (I am not sure yet whether this rea= lly >> belongs into xend as it currently does, or is better done in libxc, wher= e >> libxenlight would also benefit). >> Then libxc's hvm_build_* will pass that info into the hvm_info_table, wh= ere >> code in the hvmloader will generate an appropriate SRAT table. >> An extension of this would be to let Xen automatically decide whether a >> split of the resources is necessary (because there is not enough memory >> available (anymore) on one node). >> >> Looking forward to comments... >> >> Regards, >> Andre. >> From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ian Pratt Subject: RE: Host Numa informtion in dom0 Date: Fri, 5 Feb 2010 17:39:09 +0000 Message-ID: <4FA716B1526C7C4DB0375C6DADBC4EA342D83D9EBC@LONPMAILBOX01.citrite.net> References: <8EA2C2C4116BF44AB370468FBF85A7770123904A29@orsmsx504.amr.corp.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <8EA2C2C4116BF44AB370468FBF85A7770123904A29@orsmsx504.amr.corp.intel.com> Content-Language: en-US List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: "Kamble, Nitin A" , "xen-devel@lists.xensource.com" Cc: Ian Pratt List-Id: xen-devel@lists.xenproject.org > Attached is the patch which exposes the host numa information to dom0. > With the patch "xm info" command now also gives the cpu topology & host > numa information. This will be later used to build guest numa support. >=20 > The patch basically changes physinfo sysctl, and adds topology_info & > numa_info sysctls, and also changes the python & libxc code accordingly. It would be good to have a discussion about how we should expose NUMA infor= mation to guests.=20 I believe we can control the desired allocation of memory from nodes and cr= eation of guest NUMA tables using VCPU affinity masks combined with a new b= oolean option to enable exposure of NUMA information to guests. For each guest VCPU, we should inspect its affinity mask to see which nodes= the VCPU is able to run on, thus building a set of 'allowed node' masks. W= e should then compare all the 'allowed node' masks to see how many unique n= ode masks there are -- this corresponds to the number of NUMA nodes that we= wish to expose to the guest if this guest has NUMA enabled. We would aport= ion the guest's pseudo-physical memory equally between these virtual NUMA n= odes. If guest NUMA is disabled, we just use a single node mask which is the unio= n of the per-VCPU node masks. Where allowed node masks span more than one physical node, we should alloca= te memory to the guest's virtual node by pseudo randomly striping memory al= locations (in 2MB chunks) from across the specified physical nodes. [pseudo= random is probably better than round robin] Make sense? I can provide some worked exampled. As regards the socket vs node terminology, I agree the variables are probab= ly badly named and would perhaps best be called 'node' and 'supernode'. The= key thing is that the toolstack should allow hierarchy to be expressed whe= n specifying CPUs (using a dotted notation) rather than having to specify t= he enumerated CPU number. Best, Ian From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dan Magenheimer Subject: RE: RE: Host Numa informtion in dom0 Date: Fri, 5 Feb 2010 14:33:19 -0600 (CST) Message-ID: <06d2c17f-cf69-4ef6-ab29-163bb3038bdb@default> References: <8EA2C2C4116BF44AB370468FBF85A7770123904A29@orsmsx504.amr.corp.intel.com 4FA716B1526C7C4DB0375C6DADBC4EA342D83D9EBC@LONPMAILBOX01.citrite.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <4FA716B1526C7C4DB0375C6DADBC4EA342D83D9EBC@LONPMAILBOX01.citrite.net> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Ian Pratt , "Kamble, Nitin A" , xen-devel@lists.xensource.com, Andre Przywara List-Id: xen-devel@lists.xenproject.org It would be good if the discussion includes how guest NUMA works with (or is exclusive of) migration/save/restore. Also, the discussion should include the interaction (or exclusivity from) the various Xen RAM utilization technologies -- tmem, page sharing/swapping, and PoD. Obviously it would be great if Xen could provide both optimal affinity/performance and optimal flexibility and resource utilization, but I suspect that will be a VERY difficult combination. > -----Original Message----- > From: Ian Pratt [mailto:Ian.Pratt@eu.citrix.com] > Sent: Friday, February 05, 2010 10:39 AM > To: Kamble, Nitin A; xen-devel@lists.xensource.com > Cc: Ian Pratt > Subject: [Xen-devel] RE: Host Numa informtion in dom0 >=20 > > Attached is the patch which exposes the host numa information to > dom0. > > With the patch "xm info" command now also gives the cpu topology & > host > > numa information. This will be later used to build guest numa > support. > > > > The patch basically changes physinfo sysctl, and adds topology_info & > > numa_info sysctls, and also changes the python & libxc code > accordingly. >=20 >=20 > It would be good to have a discussion about how we should expose NUMA > information to guests. >=20 > I believe we can control the desired allocation of memory from nodes > and creation of guest NUMA tables using VCPU affinity masks combined > with a new boolean option to enable exposure of NUMA information to > guests. >=20 > For each guest VCPU, we should inspect its affinity mask to see which > nodes the VCPU is able to run on, thus building a set of 'allowed node' > masks. We should then compare all the 'allowed node' masks to see how > many unique node masks there are -- this corresponds to the number of > NUMA nodes that we wish to expose to the guest if this guest has NUMA > enabled. We would aportion the guest's pseudo-physical memory equally > between these virtual NUMA nodes. >=20 > If guest NUMA is disabled, we just use a single node mask which is the > union of the per-VCPU node masks. >=20 > Where allowed node masks span more than one physical node, we should > allocate memory to the guest's virtual node by pseudo randomly striping > memory allocations (in 2MB chunks) from across the specified physical > nodes. [pseudo random is probably better than round robin] >=20 > Make sense? I can provide some worked exampled. >=20 > As regards the socket vs node terminology, I agree the variables are > probably badly named and would perhaps best be called 'node' and > 'supernode'. The key thing is that the toolstack should allow hierarchy > to be expressed when specifying CPUs (using a dotted notation) rather > than having to specify the enumerated CPU number. >=20 >=20 > Best, > Ian >=20 >=20 > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Nakajima, Jun" Subject: RE: RE: Host Numa informtion in dom0 Date: Tue, 9 Feb 2010 14:03:19 -0800 Message-ID: <54B2EB610B7F1340BB6A0D4CA04A4F1012BABA32@orsmsx505.amr.corp.intel.com> References: <8EA2C2C4116BF44AB370468FBF85A7770123904A29@orsmsx504.amr.corp.intel.com 4FA716B1526C7C4DB0375C6DADBC4EA342D83D9EBC@LONPMAILBOX01.citrite.net> <06d2c17f-cf69-4ef6-ab29-163bb3038bdb@default> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <06d2c17f-cf69-4ef6-ab29-163bb3038bdb@default> Content-Language: en-US List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Dan Magenheimer , Ian Pratt , "Kamble, Nitin A" , "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org Dan Magenheimer wrote on Fri, 5 Feb 2010 at 12:33:19: > It would be good if the discussion includes how guest NUMA > works with (or is exclusive of) migration/save/restore. Also, > the discussion should include the interaction (or exclusivity > from) the various Xen RAM utilization technologies -- tmem, > page sharing/swapping, and PoD. Obviously it would be great > if Xen could provide both optimal affinity/performance and optimal > flexibility and resource utilization, but I suspect that will > be a VERY difficult combination. >=20 I think migration/save/restore should be excluded at this point, to keep th= e design/implementation simple; it's a performance/scalability feature. Jun ___ Intel Open Source Technology Center From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Nakajima, Jun" Subject: RE: Host Numa informtion in dom0 Date: Tue, 9 Feb 2010 14:56:45 -0800 Message-ID: <54B2EB610B7F1340BB6A0D4CA04A4F1012BABB2E@orsmsx505.amr.corp.intel.com> References: <8EA2C2C4116BF44AB370468FBF85A7770123904A29@orsmsx504.amr.corp.intel.com> <4FA716B1526C7C4DB0375C6DADBC4EA342D83D9EBC@LONPMAILBOX01.citrite.net> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <4FA716B1526C7C4DB0375C6DADBC4EA342D83D9EBC@LONPMAILBOX01.citrite.net> Content-Language: en-US List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Ian Pratt , "Kamble, Nitin A" , "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org Ian Pratt wrote on Fri, 5 Feb 2010 at 09:39:09: >> Attached is the patch which exposes the host numa information to > dom0. >> With the patch "xm info" command now also gives the cpu topology & host >> numa information. This will be later used to build guest numa support. >>=20 >> The patch basically changes physinfo sysctl, and adds topology_info & >> numa_info sysctls, and also changes the python & libxc code > accordingly. >=20 >=20 > It would be good to have a discussion about how we should expose NUMA > information to guests. >=20 > I believe we can control the desired allocation of memory from nodes and > creation of guest NUMA tables using VCPU affinity masks combined with a > new boolean option to enable exposure of NUMA information to guests. >=20 I agree.=20 > For each guest VCPU, we should inspect its affinity mask to see which > nodes the VCPU is able to run on, thus building a set of 'allowed node' > masks. We should then compare all the 'allowed node' masks to see how > many unique node masks there are -- this corresponds to the number of > NUMA nodes that we wish to expose to the guest if this guest has NUMA > enabled. We would aportion the guest's pseudo-physical memory equally > between these virtual NUMA nodes. >=20 Right. > If guest NUMA is disabled, we just use a single node mask which is the > union of the per-VCPU node masks. >=20 > Where allowed node masks span more than one physical node, we should > allocate memory to the guest's virtual node by pseudo randomly striping > memory allocations (in 2MB chunks) from across the specified physical > nodes. [pseudo random is probably better than round robin] Do we really want to support this? I don't think the allowed node masks sho= uld span more than one physical NUMA node. We also need to look at I/O devi= ces as well. >=20 > Make sense? I can provide some worked exampled. >=20 Examples are appreciated. Thanks, Jun ___ Intel Open Source Technology Center From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dan Magenheimer Subject: RE: RE: Host Numa informtion in dom0 Date: Tue, 9 Feb 2010 19:25:49 -0800 (PST) Message-ID: References: <8EA2C2C4116BF44AB370468FBF85A7770123904A29@orsmsx504.amr.corp.intel.com> <4FA716B1526C7C4DB0375C6DADBC4EA342D83D9EBC@LONPMAILBOX01.citrite.net> <06d2c17f-cf69-4ef6-ab29-163bb3038bdb@default 54B2EB610B7F1340BB6A0D4CA04A4F1012BABA32@orsmsx505.amr.corp.intel.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="__126577235365929072abhmt007" Return-path: In-Reply-To: <54B2EB610B7F1340BB6A0D4CA04A4F1012BABA32@orsmsx505.amr.corp.intel.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: "Nakajima, Jun" , Ian Pratt , "Kamble, Nitin A" , xen-devel@lists.xensource.com, Andre Przywara List-Id: xen-devel@lists.xenproject.org --__126577235365929072abhmt007 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable While I am in agreement in general, my point is that we need to avoid misleading virtualization users by somehow making it clear that "pinning NUMA memory" to get performance advantages results in significant losses in flexibility. For example, it won't be intuitive to users/admins that starting guest A and then starting guest B may result in very different performance profile for A's applications than starting guest B and then starting guest A. This may be obvious for other flexibility limiters such as PCI passthrough, but I suspect the vast majority of users (at least outside of the HPC community) for the next few years are not going to accept that one chunk of memory=20 is *that* different from another chunk of memory. > -----Original Message----- > From: Nakajima, Jun [mailto:jun.nakajima@intel.com] > Sent: Tuesday, February 09, 2010 3:03 PM > To: Dan Magenheimer; Ian Pratt; Kamble, Nitin A; xen- > devel@lists.xensource.com; Andre Przywara > Subject: RE: [Xen-devel] RE: Host Numa informtion in dom0 >=20 > Dan Magenheimer wrote on Fri, 5 Feb 2010 at 12:33:19: >=20 > > It would be good if the discussion includes how guest NUMA > > works with (or is exclusive of) migration/save/restore. Also, > > the discussion should include the interaction (or exclusivity > > from) the various Xen RAM utilization technologies -- tmem, > > page sharing/swapping, and PoD. Obviously it would be great > > if Xen could provide both optimal affinity/performance and optimal > > flexibility and resource utilization, but I suspect that will > > be a VERY difficult combination. > > >=20 > I think migration/save/restore should be excluded at this point, to > keep the design/implementation simple; it's a performance/scalability > feature. >=20 > Jun > ___ > Intel Open Source Technology Center >=20 >=20 >=20 --__126577235365929072abhmt007 Content-Type: application/ms-tnef; name="winmail.dat" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="winmail.dat" eJ8+IgEAAQaQCAAEAAAAAAABAAEAAQeQBgAIAAAA5AQAAAAAAADoAAEDkAYA4AUAAAEAAAACAQkQ AQAAAM8FAADLBQAA+ggAAExaRnU7hGByAwAKAHJjcGcxMjUiMgNDdGV4BUJiaf5kBAADMAEDAfcK gAKkA+T/BxMCgBBzAFAEVghVB7IRpScOUQMBAgBjaArAc2XcdDIGAAbDEaUzBEYUN94wEqwRswjv Cfc7GJ8OMHY1EaIMYGMAUAsJAWQzRjYW0AumIFdoAxBlwCBJIGFtIAuAHeDvCcIHgAIwHhJnCfAE kAdAgCwgbXkgcG8LgJse0QQgdBRgBUB3ZQqirwqAH0AJgCBgbx3gdh/wLSFwbQQAHaBhD0BuZygg dmkAIHUHQGl6ZSCQaQIgIHUUkBSAIApiH8BzA3BlaG93qyDkAMBrIpJpBUBjIlFzBcAgcyJwC4AD ACKhTjhVTUEfoB6QBbB5Imshgh8gdCDkcASQAhBy7QOBYx2wInB2AHABkB8gwwQgGKBzdWx0BCAe IbUAkGcDAGYN4CmBIBgwLwQQB5Ag5B4hZh2geGkDDyAjMHR5LiAgRm8FsQ7AHfALUGUfkCWRd7UC ICcFQGIdsCABdSWQ3Gl2INUhkSPDLyJwIiC7BjEgc3MBkAAgIpJnClDXMRARQB3gbiFxaAnwMQfr IOQxpEIfoGEfwCoEHhL7L0AnkCAPQAEgBJAesiiZvyg1A2ArAB2hKMERQCcEIPxhcAtQKxEjciBT MpgxlW5CIOQyLzGWLiDkIORUzx2ABCA0Ei6hb2Ii0Ahg+wQgN3JvMmEFwCxpK2AHcD8lkCPiKiAU UDoVBCBQQx8d0AqwBBAgcANgdWdouR+QYnUFQB3QKiBzKJB+YwVAMmEiwECgBUAAwGrfBbA/MjcQ IOQjxCggkSJRXzHRCGAqUA8wPXFmQrNIZ0DQJbADcG11AwAs8Cl/N2NCwh9ADtAg5DVwB+B5/yXh N9EYoEewPlAfEB/xIrD9IZJjKSAFMSBzAiAdsBRQfUbQa0XSJ1QK4ywRBCAq/SByKjU5A1IyET5U Sx08C6g+IC1QYk8FEGcLgHcHQAXQB5BzKbFQY0/mRr0DYToHsCUwQ2AHcGEfkHJKRtAgWwDAAxAh kDpaakbQLlEAUwRAIAFl9GwuRpFdT+YGYAIwUsAaVDGxZDQgH5BGZWJmciMQNREwOR+QAdAxoRbQ MzowM0DATU/mUlRUICBEA5FNKbFuszJwB3FyOx3AA5FQH2C7AkBaYEsd8AJgLeFOLxH1A6BBWmB4 CfBR9wEAL0DMbEAjMDEQcy5cQSRQ7whwKSBVYlpgQTIwSRFawCh6eXcKwGFVt3ViJmpCgVLAUkVS wFtYa1xRXQNdYJNIK4AFQE6+dQDAHhEowiNzHiFkA3D/AUBP9U/mWX0gsANgDrBKwfVScWkfkDVX ElfkIJEOIMQ6M1hAMTk6Y65QQP5JLiIqMCFwLqFJgARwHhB7RfQPQWMjwACQYwMlwHV/AQAEICSR MZUnAmh4LkBy/msEIAPwIHBEsAWxD1EOwP9rYQCQL0BD0UcgIiAJwCNj9i9RcC9ALyoBIZAYoC0R tEFsJFAsaHhqXXMkkP9pYmtFQrNVEh9gQpAjgm4C/252LOFoeANSRyBCxAUQPdLzYPEH8EFNI7Aj cCM3DrD/FFBJQBgwUOAHkVBgIGAnUf9xKQqwHyBysXbxIqBv4F9QjzgAIpEfkDpyUG9ELRH6Tz2k bB/ALhNpZhigIJD/aHhqIXdiBaBpYjbxItBFsbsG4G3RbwUwUzEDIGEBIPsLgCzhLyibMjGAlXWp Psn/OnIqAV4Dd9pBzyCDAxCDCcMuoWJgVkVSWTUzDeD/NIJGkQ8gUQAjcjwFaNFjrv8d0CBwC4BL UG9fcGNyti6h/25zAQAhcCCRi9EEIB/jH5DrIZBP5msJ4HBqRAeQKsH+LwdwLcEeoiNkAJAtslpg /yWQN8I12m/gKyALYCy0T+Z/NXAgkAhwXkBjrlOBT+Zfv5dgizdVInyQKJADoFOE5PZUeLYfwEMe sQSQY65jrAUg5H2ckACq9wEDkAYAjAEAAAEAAAACAbmACCAGAAAAAADAAAAAAAAARgAAAAAghQAA AQAAAGQBAAACAQQAAAAAAAAABVJlcGx5CElQTS5Ob3RlB01lc3NhZ2UCUkUFAAAAAAAAAAABAAAA AAAAAAIAAABmAAAAAgAAAAEAAAAMUmVwbHkgdG8gQWxsCElQTS5Ob3RlB01lc3NhZ2UCUkUFAAAA AAAAAAABAAAAAAAAAAIAAABnAAAAAwAAAAIAAAAHRm9yd2FyZAhJUE0uTm90ZQdNZXNzYWdlAkZX BQAAAAAAAAAAAQAAAAAAAAACAAAAaAAAAAQAAAADAAAAD1JlcGx5IHRvIEZvbGRlcghJUE0uUG9z dARQb3N0AAUAAAAAAAAAAAEAAAAAAAAAAgAAAGwAAAAIAAAABAEFUgBlAHAAbAB5AAJSAEUADFIA ZQBwAGwAeQAgAHQAbwAgAEEAbABsAAJSAEUAB0YAbwByAHcAYQByAGQAAkYAVwAPUgBlAHAAbAB5 ACAAdABvACAARgBvAGwAZABlAHIAAJ47 --__126577235365929072abhmt007 Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel --__126577235365929072abhmt007-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ian Pratt Subject: RE: Host Numa informtion in dom0 Date: Thu, 11 Feb 2010 15:21:41 +0000 Message-ID: <4FA716B1526C7C4DB0375C6DADBC4EA342D83DA06B@LONPMAILBOX01.citrite.net> References: <8EA2C2C4116BF44AB370468FBF85A7770123904A29@orsmsx504.amr.corp.intel.com> <4FA716B1526C7C4DB0375C6DADBC4EA342D83D9EBC@LONPMAILBOX01.citrite.net> <54B2EB610B7F1340BB6A0D4CA04A4F1012BABB2E@orsmsx505.amr.corp.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <54B2EB610B7F1340BB6A0D4CA04A4F1012BABB2E@orsmsx505.amr.corp.intel.com> Content-Language: en-US List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: "Nakajima, Jun" , "Kamble, Nitin A" , "xen-devel@lists.xensource.com" Cc: Ian Pratt List-Id: xen-devel@lists.xenproject.org > > If guest NUMA is disabled, we just use a single node mask which is the > > union of the per-VCPU node masks. > > > > Where allowed node masks span more than one physical node, we should > > allocate memory to the guest's virtual node by pseudo randomly striping > > memory allocations (in 2MB chunks) from across the specified physical > > nodes. [pseudo random is probably better than round robin] >=20 > Do we really want to support this? I don't think the allowed node masks > should span more than one physical NUMA node. We also need to look at I/O > devices as well. Given that we definitely need this striping code in the case where the gues= t is non NUMA, I'd be inclined to still allow it to be used even if the gue= st has multiple NUMA nodes. It could come in handy where there is a hierarc= hy between physical NUMA nodes, enabling for example striping to be used be= tween a pair of 'close' nodes, while exposing the higher-level topology of = sets of the paired nodes to be exposed to the guest.=20 Ian From mboxrd@z Thu Jan 1 00:00:00 1970 From: Bruce Edge Subject: Re: Host Numa informtion in dom0 Date: Wed, 26 May 2010 10:31:09 -0700 Message-ID: References: <8EA2C2C4116BF44AB370468FBF85A7770123904A29@orsmsx504.amr.corp.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <8EA2C2C4116BF44AB370468FBF85A7770123904A29@orsmsx504.amr.corp.intel.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com Cc: "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org I'm getting a python traceback when I try to start a VM that I tracked down (I think) to this patch: Traceback (most recent call last): =A0=A0File "/usr/local/lib/python2.6/dist-packages/xen/util/xmlrpclib2.py", line 131, in _marshaled_dispatch =A0=A0 =A0response =3D self._dispatch(method, params) =A0=A0File "/usr/lib/python2.6/SimpleXMLRPCServer.py", line 418, in _dispat= ch =A0=A0 =A0return func(*params) =A0=A0File "/usr/local/lib/python2.6/dist-packages/xen/xend/server/XMLRPCSe= rver.py", line 80, in domain_create =A0=A0 =A0info =3D XendDomain.instance().domain_create(config) =A0=A0File "/usr/local/lib/python2.6/dist-packages/xen/xend/XendDomain.py", line 982, in domain_create =A0=A0 =A0dominfo =3D XendDomainInfo.create(config) =A0=A0File "/usr/local/lib/python2.6/dist-packages/xen/xend/XendDomainInfo.= py", line 106, in create =A0=A0 =A0vm.start() =A0=A0File "/usr/local/lib/python2.6/dist-packages/xen/xend/XendDomainInfo.= py", line 470, in start =A0=A0 =A0XendTask.log_progress(0, 30, self._constructDomain) =A0=A0File "/usr/local/lib/python2.6/dist-packages/xen/xend/XendTask.py", line 209, in log_progress =A0=A0 =A0retval =3D func(*args, **kwds) =A0=A0File "/usr/local/lib/python2.6/dist-packages/xen/xend/XendDomainInfo.= py", line 2530, in _constructDomain =A0=A0 =A0balloon.free(16*1024, self) # 16MB should be plenty =A0=A0File "/usr/local/lib/python2.6/dist-packages/xen/xend/balloon.py", line 187, in free =A0=A0 =A0nodenum =3D xc.numainfo()['cpu_to_node'][cpu] KeyError: 'cpu_to_node' release =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0: 2.6.32.12 version =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0: #1 SMP Wed May 5 21:52:23 PDT 2010 machine =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0: x86_64 nr_cpus =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0: 16 nr_nodes =A0 =A0 =A0 =A0 =A0 =A0 =A0 : 2 cores_per_socket =A0 =A0 =A0 : 4 threads_per_core =A0 =A0 =A0 : 2 cpu_mhz =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0: 2533 hw_caps =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0: bfebfbff:28100800:00000000:00001b40:009ce3bd:00000000:00000001:00000000 virt_caps =A0 =A0 =A0 =A0 =A0 =A0 =A0: hvm hvm_directio total_memory =A0 =A0 =A0 =A0 =A0 : 12277 free_memory =A0 =A0 =A0 =A0 =A0 =A0: 11629 free_cpus =A0 =A0 =A0 =A0 =A0 =A0 =A0: 0 xen_major =A0 =A0 =A0 =A0 =A0 =A0 =A0: 4 xen_minor =A0 =A0 =A0 =A0 =A0 =A0 =A0: 1 xen_extra =A0 =A0 =A0 =A0 =A0 =A0 =A0: -unstable xen_caps =A0 =A0 =A0 =A0 =A0 =A0 =A0 : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3= .0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 xen_scheduler =A0 =A0 =A0 =A0 =A0: credit xen_pagesize =A0 =A0 =A0 =A0 =A0 : 4096 platform_params =A0 =A0 =A0 =A0: virt_start=3D0xffff800000000000 xen_changeset =A0 =A0 =A0 =A0 =A0: Sat May 22 06:36:41 2010 +0100 21446:934= 10e5e4ad8 xen_commandline =A0 =A0 =A0 =A0: dummy=3Ddummy console=3Dcom1 115200,8n1 dom0_mem=3D512M dom0_max_vcpus=3D1 dom0_vcpus_pin=3Dtrue iommu=3D1,passthrough,no-intremap loglvl=3Dall loglvl_guest=3Dall loglevl= =3D10 debug acpi=3Dforce apic=3Don apic_verbosity=3Dverbose numa=3Don cc_compiler =A0 =A0 =A0 =A0 =A0 =A0: gcc version 4.3.3 (Ubuntu 4.3.3-5ubunt= u4) cc_compile_by =A0 =A0 =A0 =A0 =A0: bedge cc_compile_domain =A0 =A0 =A0: cc_compile_date =A0 =A0 =A0 =A0: Tue May 25 14:51:02 PDT 2010 xend_config_format =A0 =A0 : 4 -Bruce On Fri, Jan 29, 2010 at 4:05 PM, Kamble, Nitin A wrote: > > Hi Keir, > > =A0 =A0Attached is the patch which exposes the host numa information to d= om0. With the patch =93xm info=94 command now also gives the cpu topology &= host numa information. This will be later used to build guest numa support= . > > The patch basically changes physinfo sysctl, and adds topology_info & num= a_info sysctls, and also changes the python & libxc code accordingly. > > > > Please apply. > > > > Thanks & Regards, > > Nitin > > > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >