From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.3 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 28A4EC433E1 for ; Sat, 22 Aug 2020 07:09:43 +0000 (UTC) Received: from whitealder.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F1827208E4 for ; Sat, 22 Aug 2020 07:09:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="rxsqS8Pr" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F1827208E4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id E376B87D81; Sat, 22 Aug 2020 07:09:42 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 4nSCPQRH51VF; Sat, 22 Aug 2020 07:09:34 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by whitealder.osuosl.org (Postfix) with ESMTP id B57D487A60; Sat, 22 Aug 2020 07:09:33 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 8DA1EC089E; Sat, 22 Aug 2020 07:09:33 +0000 (UTC) Received: from silver.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 43FE0C0051 for ; Sat, 22 Aug 2020 04:28:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id 2248D203F1 for ; Sat, 22 Aug 2020 04:28:29 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id z3ZdxVyaRTwA for ; Sat, 22 Aug 2020 04:28:25 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pl1-f196.google.com (mail-pl1-f196.google.com [209.85.214.196]) by silver.osuosl.org (Postfix) with ESMTPS id 0E4CA203AE for ; Sat, 22 Aug 2020 04:28:25 +0000 (UTC) Received: by mail-pl1-f196.google.com with SMTP id k13so1773382plk.13 for ; Fri, 21 Aug 2020 21:28:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=aOT5t5tsGk7zRkfUzo++dr5YPDmf2c1MUVXVEa0SPao=; b=rxsqS8Pri0Qn5Ds/DSubiRLLrZ4K+SYlbOi4VVrpZ8DYheYQemgKiTMN0HKrAkNaNG UKIlmhhC30qGforLqflJRG84EP/bpSnmc3UilciUq08oxv1K7pCMFMTdysN1hA/VgHXk sr2gcZF+jTB1A4NwBXm1mtbkhalu01RDSjxoeBFjalhZTWMEoFiKIBIPlAS/f4ksgmvk Q01C++UaDK/zxG2BauSnyV4II4mmvj2TjEpaP8iWoL18U7D1SM/iaIXrTfBE/OLnlq30 X9ezI8seJUQzlq4jWyNMpqUMo/QoKJ14raVkgZRw7iIGeMUAVoMZAS6fqBLibjaidGKI gwaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aOT5t5tsGk7zRkfUzo++dr5YPDmf2c1MUVXVEa0SPao=; b=O+3tR/7Jh40U0qpGvdAGgGkdD1wJbxxi6hukXHRbG1yRAmA12ioEMUplW+Irj1UGJ1 hfZR2/aw7+d1rXjb5yE+/DuIeP3eSJIWkkxiTDmF1rVh8/tBdNIYhlq6JQ0sN/ZKQVm2 umDxLGnlf9Jes81Q+S17ygUhyhgbqhp/R+m1xdT8XF3BAOPOgvMNtcLi+16Ckf96r6LM rfM/K3Ss0QzSpJOaTzTT72uxeGcrK1ci6+hDHo+4VUquj44W7QpkuvglvFH9ioErDyNi se26LPiLJ9eDoCt8iOPfab5hOPYRNGrLQa8FbeXHtQvig21XOi9CXho3n2Up6XR5EKvQ 5X0A== X-Gm-Message-State: AOAM5334PcBUlsCRkgzhSNyYyjmyn4ZvETGe+t523a7C7dMZYCqGdflR 5NlgA54jEM70hkJGvlHJlcEomLX/SU1knaNU X-Google-Smtp-Source: ABdhPJwQZqduhA5Sqkeshos1ceV39Qc74CqyJ4sHhm2xbB00DhuIm77th+n707XKkjffOZ+s4R0hzA== X-Received: by 2002:a17:90a:9c3:: with SMTP id 61mr638955pjo.191.1598070503753; Fri, 21 Aug 2020 21:28:23 -0700 (PDT) Received: from jacob-builder.jf.intel.com ([192.55.55.43]) by smtp.gmail.com with ESMTPSA id q5sm3341582pgi.31.2020.08.21.21.28.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Aug 2020 21:28:23 -0700 (PDT) From: Jacob Pan X-Google-Original-From: Jacob Pan To: iommu@lists.linux-foundation.org, LKML , Jean-Philippe Brucker , "Lu Baolu" , Joerg Roedel , David Woodhouse Subject: [PATCH v2 1/9] docs: Document IO Address Space ID (IOASID) APIs Date: Fri, 21 Aug 2020 21:35:10 -0700 Message-Id: <1598070918-21321-2-git-send-email-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1598070918-21321-1-git-send-email-jacob.jun.pan@linux.intel.com> References: <1598070918-21321-1-git-send-email-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 X-Mailman-Approved-At: Sat, 22 Aug 2020 07:09:31 +0000 Cc: "Tian, Kevin" , Raj Ashok , Wu Hao X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" SU9BU0lEIGlzIHVzZWQgdG8gaWRlbnRpZnkgYWRkcmVzcyBzcGFjZXMgdGhhdCBjYW4gYmUgdGFy Z2V0ZWQgYnkgZGV2aWNlCkRNQS4gSXQgaXMgYSBzeXN0ZW0td2lkZSByZXNvdXJjZSB0aGF0IGlz IGVzc2VudGlhbCB0byBpdHMgbWFueSB1c2Vycy4KVGhpcyBkb2N1bWVudCBpcyBhbiBhdHRlbXB0 IHRvIGhlbHAgZGV2ZWxvcGVycyBmcm9tIGFsbCB2ZW5kb3JzIG5hdmlnYXRlCnRoZSBBUElzLiBB dCB0aGlzIHRpbWUsIEFSTSBTTU1VIGFuZCBJbnRlbOKAmXMgU2NhbGFibGUgSU8gVmlydHVhbGl6 YXRpb24KKFNJT1YpIGVuYWJsZWQgcGxhdGZvcm1zIGFyZSB0aGUgcHJpbWFyeSB1c2VycyBvZiBJ T0FTSUQuIEV4YW1wbGVzIG9mCmhvdyBTSU9WIGNvbXBvbmVudHMgaW50ZXJhY3Qgd2l0aCBJT0FT SUQgQVBJcyBhcmUgcHJvdmlkZWQgaW4gdGhhdCBtYW55CkFQSXMgYXJlIGRyaXZlbiBieSB0aGUg cmVxdWlyZW1lbnRzIGZyb20gU0lPVi4KClNpZ25lZC1vZmYtYnk6IExpdSBZaSBMIDx5aS5sLmxp dUBpbnRlbC5jb20+ClNpZ25lZC1vZmYtYnk6IFd1IEhhbyA8aGFvLnd1QGludGVsLmNvbT4KU2ln bmVkLW9mZi1ieTogSmFjb2IgUGFuIDxqYWNvYi5qdW4ucGFuQGxpbnV4LmludGVsLmNvbT4KLS0t CiBEb2N1bWVudGF0aW9uL2lvYXNpZC5yc3QgfCA2MTggKysrKysrKysrKysrKysrKysrKysrKysr KysrKysrKysrKysrKysrKysrKysrKysKIDEgZmlsZSBjaGFuZ2VkLCA2MTggaW5zZXJ0aW9ucygr KQogY3JlYXRlIG1vZGUgMTAwNjQ0IERvY3VtZW50YXRpb24vaW9hc2lkLnJzdAoKZGlmZiAtLWdp dCBhL0RvY3VtZW50YXRpb24vaW9hc2lkLnJzdCBiL0RvY3VtZW50YXRpb24vaW9hc2lkLnJzdApu ZXcgZmlsZSBtb2RlIDEwMDY0NAppbmRleCAwMDAwMDAwMDAwMDAuLmI2YThjZGM4ODVmZgotLS0g L2Rldi9udWxsCisrKyBiL0RvY3VtZW50YXRpb24vaW9hc2lkLnJzdApAQCAtMCwwICsxLDYxOCBA QAorLi4gaW9hc2lkOgorCis9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09CitJ TyBBZGRyZXNzIFNwYWNlIElECis9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 CisKK0lPQVNJRCBpcyBhIGdlbmVyaWMgbmFtZSBmb3IgUENJZSBQcm9jZXNzIEFkZHJlc3MgSUQg KFBBU0lEKSBvciBBUk0KK1NNTVUgc3ViLXN0cmVhbSBJRC4gQW4gSU9BU0lEIGlkZW50aWZpZXMg YW4gYWRkcmVzcyBzcGFjZSB0aGF0IERNQQorcmVxdWVzdHMgY2FuIHRhcmdldC4KKworVGhlIHBy aW1hcnkgdXNlIGNhc2VzIGZvciBJT0FTSUQgYXJlIFNoYXJlZCBWaXJ0dWFsIEFkZHJlc3MgKFNW QSkgYW5kCitJTyBWaXJ0dWFsIEFkZHJlc3MgKElPVkEpLiBIb3dldmVyLCB0aGUgcmVxdWlyZW1l bnRzIGZvciBJT0FTSUQKK21hbmFnZW1lbnQgY2FuIHZhcnkgYW1vbmcgaGFyZHdhcmUgYXJjaGl0 ZWN0dXJlcy4KKworVGhpcyBkb2N1bWVudCBjb3ZlcnMgdGhlIGdlbmVyaWMgZmVhdHVyZXMgc3Vw cG9ydGVkIGJ5IElPQVNJRAorQVBJcy4gVmVuZG9yLXNwZWNpZmljIHVzZSBjYXNlcyBhcmUgYWxz byBpbGx1c3RyYXRlZCB3aXRoIEludGVsJ3MgVlQtZAorYmFzZWQgcGxhdGZvcm1zIGFzIHRoZSBm aXJzdCBleGFtcGxlLgorCisuLiBjb250ZW50czo6IDpsb2NhbDoKKworR2xvc3NhcnkKKz09PT09 PT09CitQQVNJRCAtIFByb2Nlc3MgQWRkcmVzcyBTcGFjZSBJRAorCitJT0FTSUQgLSBJTyBBZGRy ZXNzIFNwYWNlIElEIChnZW5lcmljIHRlcm0gZm9yIFBDSWUgUEFTSUQgYW5kCitzdWItc3RyZWFt IElEIGluIFNNTVUpCisKK1NWQS9TVk0gLSBTaGFyZWQgVmlydHVhbCBBZGRyZXNzaW5nL01lbW9y eQorCitFTlFDTUQgLSBOZXcgSW50ZWwgWDg2IElTQSBmb3IgZWZmaWNpZW50IHdvcmtxdWV1ZSBz dWJtaXNzaW9uIFsxXQorCitEU0EgLSBJbnRlbCBEYXRhIFN0cmVhbWluZyBBY2NlbGVyYXRvciBb Ml0KKworVkRDTSAtIFZpcnR1YWwgZGV2aWNlIGNvbXBvc2l0aW9uIG1vZHVsZSBbM10KKworU0lP ViAtIEludGVsIFNjYWxhYmxlIElPIFZpcnR1YWxpemF0aW9uCisKKworS2V5IENvbmNlcHRzCis9 PT09PT09PT09PT0KKworSU9BU0lEIFNldAorLS0tLS0tLS0tLS0KK0FuIElPQVNJRCBzZXQgaXMg YSBncm91cCBvZiBJT0FTSURzIGFsbG9jYXRlZCBmcm9tIHRoZSBzeXN0ZW0td2lkZQorSU9BU0lE IHBvb2wuIEFuIElPQVNJRCBzZXQgaXMgY3JlYXRlZCBhbmQgY2FuIGJlIGlkZW50aWZpZWQgYnkg YQordG9rZW4gb2YgdTY0LiBSZWZlciB0byBJT0FTSUQgc2V0IEFQSXMgZm9yIG1vcmUgZGV0YWls cy4KKworSU9BU0lEIHNldCBpcyBwYXJ0aWN1bGFybHkgdXNlZnVsIGZvciBndWVzdCBTVkEgd2hl cmUgZWFjaCBndWVzdCBjb3VsZAoraGF2ZSBpdHMgb3duIElPQVNJRCBzZXQgZm9yIHNlY3VyaXR5 IGFuZCBlZmZpY2llbmN5IHJlYXNvbnMuCisKK0lPQVNJRCBTZXQgUHJpdmF0ZSBJRCAoU1BJRCkK Ky0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KK1NQSURzIGFyZSBpbnRyb2R1Y2VkIGFzIElP QVNJRHMgd2l0aGluIGl0cyBzZXQuIEVhY2ggU1BJRCBtYXBzIHRvIGEKK3N5c3RlbS13aWRlIElP QVNJRCBidXQgdGhlIG5hbWVzcGFjZSBvZiBTUElEIGlzIHdpdGhpbiBpdHMgSU9BU0lECitzZXQu IFNQSURzIGNhbiBiZSB1c2VkIGFzIGd1ZXN0IElPQVNJRHMgd2hlcmUgZWFjaCBndWVzdCBjb3Vs ZCBkbworSU9BU0lEIGFsbG9jYXRpb24gZnJvbSBpdHMgb3duIHBvb2wgYW5kIG1hcCB0aGVtIHRv IGhvc3QgcGh5c2ljYWwKK0lPQVNJRHMuIFNQSURzIGFyZSBwYXJ0aWN1bGFybHkgdXNlZnVsIGZv ciBzdXBwb3J0aW5nIGxpdmUgbWlncmF0aW9uCit3aGVyZSBkZWNvdXBsaW5nIGd1ZXN0IGFuZCBo b3N0IHBoeXNpY2FsIHJlc291cmNlcyBhcmUgbmVjZXNzYXJ5LgorCitGb3IgZXhhbXBsZSwgdHdv IFZNcyBjYW4gYm90aCBhbGxvY2F0ZSBndWVzdCBQQVNJRC9TUElEICMxMDEgYnV0IG1hcCB0bwor ZGlmZmVyZW50IGhvc3QgUEFTSURzICMyMDEgYW5kICMyMDIgcmVzcGVjdGl2ZWx5IGFzIHNob3du IGluIHRoZQorZGlhZ3JhbSBiZWxvdy4KKzo6CisKKyAuLS0tLS0tLS0tLS0tLS0tLS0tLiAgICAu LS0tLS0tLS0tLS0tLS0tLS0tLgorIHwgICBWTSAxICAgICAgICAgICB8ICAgIHwgICBWTSAyICAg ICAgICAgICB8CisgfCAgICAgICAgICAgICAgICAgIHwgICAgfCAgICAgICAgICAgICAgICAgIHwK KyB8LS0tLS0tLS0tLS0tLS0tLS0tfCAgICB8LS0tLS0tLS0tLS0tLS0tLS0tfAorIHwgR1BBU0lE L1NQSUQgMTAxICB8ICAgIHwgR1BBU0lEL1NQSUQgMTAxICB8CisgJy0tLS0tLS0tLS0tLS0tLS0t LScgICAgLS0tLS0tLS0tLS0tLS0tLS0tLScgICAgIEd1ZXN0CisgX19fX19fX19fX3xfX19fX19f X19fX19fX19fX19fX19ffF9fX19fX19fX19fX19fX19fX19fX18KKyAgICAgICAgICAgfCAgICAg ICAgICAgICAgICAgICAgICB8ICAgICAgICAgICAgICAgSG9zdAorICAgICAgICAgICB2ICAgICAg ICAgICAgICAgICAgICAgIHYKKyAuLS0tLS0tLS0tLS0tLS0tLS0tLiAgICAuLS0tLS0tLS0tLS0t LS0tLS0tLgorIHwgSG9zdCBJT0FTSUQgMjAxICB8ICAgIHwgSG9zdCBJT0FTSUQgMjAyICB8Cisg Jy0tLS0tLS0tLS0tLS0tLS0tLScgICAgJy0tLS0tLS0tLS0tLS0tLS0tLScKKyB8ICAgSU9BU0lE IHNldCAxICAgfCAgICB8ICAgSU9BU0lEIHNldCAyICAgfAorICctLS0tLS0tLS0tLS0tLS0tLS0n ICAgICctLS0tLS0tLS0tLS0tLS0tLS0nCisKK0d1ZXN0IFBBU0lEIGlzIHRyZWF0ZWQgYXMgSU9B U0lEIHNldCBwcml2YXRlIElEIChTUElEKSB3aXRoaW4gYW4KK0lPQVNJRCBzZXQsIG1hcHBpbmdz IGJldHdlZW4gZ3Vlc3QgYW5kIGhvc3QgSU9BU0lEcyBhcmUgc3RvcmVkIGluIHRoZQorc2V0IGZv ciBpbnF1aXJ5LgorCitJT0FTSUQgQVBJcworPT09PT09PT09PT0KK1RvIGdldCB0aGUgSU9BU0lE IEFQSXMsIHVzZXJzIG11c3QgI2luY2x1ZGUgPGxpbnV4L2lvYXNpZC5oPi4gVGhlc2UgQVBJcwor c2VydmUgdGhlIGZvbGxvd2luZyBmdW5jdGlvbmFsaXRpZXM6CisKKyAgLSBJT0FTSUQgYWxsb2Nh dGlvbi9GcmVlCisgIC0gR3JvdXAgbWFuYWdlbWVudCBpbiB0aGUgZm9ybSBvZiBpb2FzaWRfc2V0 CisgIC0gUHJpdmF0ZSBkYXRhIHN0b3JhZ2UgYW5kIGxvb2t1cAorICAtIFJlZmVyZW5jZSBjb3Vu dGluZworICAtIEV2ZW50IG5vdGlmaWNhdGlvbiBpbiBjYXNlIG9mIHN0YXRlIGNoYW5nZQorCitJ T0FTSUQgU2V0IExldmVsIEFQSXMKKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCitGb3IgdXNl IGNhc2VzIHN1Y2ggYXMgZ3Vlc3QgU1ZBIGl0IGlzIG5lY2Vzc2FyeSB0byBtYW5hZ2UgSU9BU0lE cyBhdAorYSBncm91cCBsZXZlbC4gRm9yIGV4YW1wbGUsIFZNcyBtYXkgYWxsb2NhdGUgbXVsdGlw bGUgSU9BU0lEcyBmb3IKK2d1ZXN0IHByb2Nlc3MgYWRkcmVzcyBzaGFyaW5nICh2U1ZBKS4gSXQg aXMgaW1wZXJhdGl2ZSB0byBlbmZvcmNlCitWTS1JT0FTSUQgb3duZXJzaGlwIHN1Y2ggdGhhdCBt YWxpY2lvdXMgZ3Vlc3QgY2Fubm90IHRhcmdldCBETUEKK3RyYWZmaWMgb3V0c2lkZSBpdHMgb3du IElPQVNJRHMsIG9yIGZyZWUgYW4gYWN0aXZlIElPQVNJRCBiZWxvbmcgdG8KK2Fub3RoZXIgVk0u Cis6OgorCisgc3RydWN0IGlvYXNpZF9zZXQgKmlvYXNpZF9hbGxvY19zZXQodm9pZCAqdG9rZW4s IGlvYXNpZF90IHF1b3RhLCB1MzIgdHlwZSkKKworIGludCBpb2FzaWRfYWRqdXN0X3NldChzdHJ1 Y3QgaW9hc2lkX3NldCAqc2V0LCBpbnQgcXVvdGEpOworCisgdm9pZCBpb2FzaWRfc2V0X2dldChz dHJ1Y3QgaW9hc2lkX3NldCAqc2V0KQorCisgdm9pZCBpb2FzaWRfc2V0X3B1dChzdHJ1Y3QgaW9h c2lkX3NldCAqc2V0KQorCisgdm9pZCBpb2FzaWRfc2V0X2dldF9sb2NrZWQoc3RydWN0IGlvYXNp ZF9zZXQgKnNldCkKKworIHZvaWQgaW9hc2lkX3NldF9wdXRfbG9ja2VkKHN0cnVjdCBpb2FzaWRf c2V0ICpzZXQpCisKKyBpbnQgaW9hc2lkX3NldF9mb3JfZWFjaF9pb2FzaWQoc3RydWN0IGlvYXNp ZF9zZXQgKnNkYXRhLAorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB2b2lkICgqZm4p KGlvYXNpZF90IGlkLCB2b2lkICpkYXRhKSwKKwkJCQl2b2lkICpkYXRhKQorCisKK0lPQVNJRCBz ZXQgY29uY2VwdCBpcyBpbnRyb2R1Y2VkIHRvIHJlcHJlc2VudCBzdWNoIElPQVNJRCBncm91cHMu IEVhY2gKK0lPQVNJRCBzZXQgaXMgY3JlYXRlZCB3aXRoIGEgdG9rZW4gd2hpY2ggY2FuIGJlIG9u ZSBvZiB0aGUgZm9sbG93aW5nCit0eXBlczoKKworIC0gSU9BU0lEX1NFVF9UWVBFX05VTEwgKEFy Yml0cmFyeSB1NjQgdmFsdWUpCisgLSBJT0FTSURfU0VUX1RZUEVfTU0gKFNldCB0b2tlbiBpcyBh IG1tX3N0cnVjdCkKKworVGhlIGV4cGxpY2l0IE1NIHRva2VuIHR5cGUgaXMgdXNlZnVsIHdoZW4g bXVsdGlwbGUgdXNlcnMgb2YgYW4gSU9BU0lECitzZXQgdW5kZXIgdGhlIHNhbWUgcHJvY2VzcyBu ZWVkIHRvIGNvbW11bmljYXRlIGFib3V0IHRoZWlyIHNoYXJlZCBJT0FTSURzLgorRS5nLiBBbiBJ T0FTSUQgc2V0IGNyZWF0ZWQgYnkgVkZJTyBmb3Igb25lIGd1ZXN0IGNhbiBiZSBhc3NvY2lhdGVk Cit3aXRoIHRoZSBLVk0gaW5zdGFuY2UgZm9yIHRoZSBzYW1lIGd1ZXN0IHNpbmNlIHRoZXkgc2hh cmUgYSBjb21tb24gbW1fc3RydWN0LgorCitUaGUgSU9BU0lEIHNldCBBUElzIHNlcnZlIHRoZSBm b2xsb3dpbmcgcHVycG9zZXM6CisKKyAtIE93bmVyc2hpcC9wZXJtaXNzaW9uIGVuZm9yY2VtZW50 CisgLSBUYWtlIGNvbGxlY3RpdmUgYWN0aW9ucywgZS5nLiBmcmVlIGFuIGVudGlyZSBzZXQKKyAt IEV2ZW50IG5vdGlmaWNhdGlvbnMgd2l0aGluIGEgc2V0CisgLSBMb29rIHVwIGEgc2V0IGJhc2Vk IG9uIHRva2VuCisgLSBRdW90YSBlbmZvcmNlbWVudAorCitJbmRpdmlkdWFsIElPQVNJRCBBUElz CistLS0tLS0tLS0tLS0tLS0tLS0tLS0tCitPbmNlIGFuIGlvYXNpZF9zZXQgaXMgY3JlYXRlZCwg SU9BU0lEcyBjYW4gYmUgYWxsb2NhdGVkIGZyb20gdGhlIHNldC4KK1dpdGhpbiB0aGUgSU9BU0lE IHNldCBuYW1lc3BhY2UsIHNldCBwcml2YXRlIElEIChTUElEKSBpcyBzdXBwb3J0ZWQuIEluCit0 aGUgVk0gdXNlIGNhc2UsIFNQSUQgY2FuIGJlIHVzZWQgZm9yIHN0b3JpbmcgZ3Vlc3QgUEFTSUQu CisKKzo6CisKKyBpb2FzaWRfdCBpb2FzaWRfYWxsb2Moc3RydWN0IGlvYXNpZF9zZXQgKnNldCwg aW9hc2lkX3QgbWluLCBpb2FzaWRfdCBtYXgsCisgICAgICAgICAgICAgICAgICAgICAgIHZvaWQg KnByaXZhdGUpOworCisgaW50IGlvYXNpZF9nZXQoc3RydWN0IGlvYXNpZF9zZXQgKnNldCwgaW9h c2lkX3QgaW9hc2lkKTsKKworIHZvaWQgaW9hc2lkX3B1dChzdHJ1Y3QgaW9hc2lkX3NldCAqc2V0 LCBpb2FzaWRfdCBpb2FzaWQpOworCisgaW50IGlvYXNpZF9nZXRfbG9ja2VkKHN0cnVjdCBpb2Fz aWRfc2V0ICpzZXQsIGlvYXNpZF90IGlvYXNpZCk7CisKKyB2b2lkIGlvYXNpZF9wdXRfbG9ja2Vk KHN0cnVjdCBpb2FzaWRfc2V0ICpzZXQsIGlvYXNpZF90IGlvYXNpZCk7CisKKyB2b2lkICppb2Fz aWRfZmluZChzdHJ1Y3QgaW9hc2lkX3NldCAqc2V0LCBpb2FzaWRfdCBpb2FzaWQsCisgICAgICAg ICAgICAgICAgICAgYm9vbCAoKmdldHRlcikodm9pZCAqKSk7CisKKyBpb2FzaWRfdCBpb2FzaWRf ZmluZF9ieV9zcGlkKHN0cnVjdCBpb2FzaWRfc2V0ICpzZXQsIGlvYXNpZF90IHNwaWQpCisKKyBp bnQgaW9hc2lkX2F0dGFjaF9kYXRhKHN0cnVjdCBpb2FzaWRfc2V0ICpzZXQsIGlvYXNpZF90IGlv YXNpZCwKKyAgICAgICAgICAgICAgICAgICAgICAgIHZvaWQgKmRhdGEpOworIGludCBpb2FzaWRf YXR0YWNoX3NwaWQoc3RydWN0IGlvYXNpZF9zZXQgKnNldCwgaW9hc2lkX3QgaW9hc2lkLAorICAg ICAgICAgICAgICAgICAgICAgICAgaW9hc2lkX3Qgc3NpZCk7CisKKworTm90aWZpY2F0aW9ucwor LS0tLS0tLS0tLS0tLQorQW4gSU9BU0lEIG1heSBoYXZlIG11bHRpcGxlIHVzZXJzLCBlYWNoIHVz ZXIgbWF5IGhhdmUgaGFyZHdhcmUgY29udGV4dAorYXNzb2NpYXRlZCB3aXRoIGFuIElPQVNJRC4g V2hlbiB0aGUgc3RhdHVzIG9mIGFuIElPQVNJRCBjaGFuZ2VzLAorZS5nLiBhbiBJT0FTSUQgaXMg YmVpbmcgZnJlZWQsIHVzZXJzIG5lZWQgdG8gYmUgbm90aWZpZWQgc3VjaCB0aGF0IHRoZQorYXNz b2NpYXRlZCBoYXJkd2FyZSBjb250ZXh0IGNhbiBiZSBjbGVhcmVkLCBmbHVzaGVkLCBhbmQgZHJh aW5lZC4KKworOjoKKworIGludCBpb2FzaWRfcmVnaXN0ZXJfbm90aWZpZXIoc3RydWN0IGlvYXNp ZF9zZXQgKnNldCwgc3RydWN0CisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBub3RpZmll cl9ibG9jayAqbmIpCisKKyB2b2lkIGlvYXNpZF91bnJlZ2lzdGVyX25vdGlmaWVyKHN0cnVjdCBp b2FzaWRfc2V0ICpzZXQsCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHJ1Y3Qg bm90aWZpZXJfYmxvY2sgKm5iKQorCisgaW50IGlvYXNpZF9yZWdpc3Rlcl9ub3RpZmllcl9tbShz dHJ1Y3QgbW1fc3RydWN0ICptbSwgc3RydWN0CisgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICBub3RpZmllcl9ibG9jayAqbmIpCisKKyB2b2lkIGlvYXNpZF91bnJlZ2lzdGVyX25vdGlm aWVyX21tKHN0cnVjdCBtbV9zdHJ1Y3QgKm1tLCBzdHJ1Y3QKKyAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIG5vdGlmaWVyX2Jsb2NrICpuYikKKworIGludCBpb2FzaWRfbm90aWZ5 KGlvYXNpZF90IGlvYXNpZCwgZW51bSBpb2FzaWRfbm90aWZ5X3ZhbCBjbWQsCisgICAgICAgICAg ICAgICAgICAgdW5zaWduZWQgaW50IGZsYWdzKQorCisKK0V2ZW50cworfn5+fn5+CitOb3RpZmlj YXRpb24gZXZlbnRzIGFyZSBwZXJ0aW5lbnQgdG8gaW5kaXZpZHVhbCBJT0FTSURzLCB0aGV5IGNh biBiZQorb25lIG9mIHRoZSBmb2xsb3dpbmc6CisKKyAtIEFMTE9DCisgLSBGUkVFCisgLSBCSU5E CisgLSBVTkJJTkQKKworT3JkZXJpbmcKK35+fn5+fn5+CitPcmRlcmluZyBpcyBzdXBwb3J0ZWQg YnkgSU9BU0lEIG5vdGlmaWNhdGlvbiBwcmlvcml0aWVzIGFzIHRoZQorZm9sbG93aW5nIChpbiBh c2NlbmRpbmcgb3JkZXIpOgorCis6OgorCisgZW51bSBpb2FzaWRfbm90aWZpZXJfcHJpb3Mgewor CUlPQVNJRF9QUklPX0xBU1QsCisJSU9BU0lEX1BSSU9fSU9NTVUsCisJSU9BU0lEX1BSSU9fREVW SUNFLAorCUlPQVNJRF9QUklPX0NQVSwKKyB9OworCitUaGUgdHlwaWNhbCB1c2UgY2FzZSBpcyB3 aGVuIGFuIElPQVNJRCBpcyBmcmVlZCBkdWUgdG8gYW4gZXhjZXB0aW9uLCBETUEKK3NvdXJjZSBz aG91bGQgYmUgcXVpZXNjZWQgYmVmb3JlIHRlYXJpbmcgZG93biBvdGhlciBoYXJkd2FyZSBjb250 ZXh0cworaW4gdGhlIHN5c3RlbS4gVGhpcyB3aWxsIHJlZHVjZSB0aGUgY2h1cm4gaW4gaGFuZGxp bmcgZmF1bHRzLiBETUEgd29yaworc3VibWlzc2lvbiBpcyBwZXJmb3JtZWQgYnkgdGhlIENQVSB3 aGljaCBpcyBncmFudGVkIGhpZ2hlciBwcmlvcml0eSB0aGFuCitkZXZpY2VzLgorCisKK1Njb3Bl cworfn5+fn5+CitUaGVyZSBhcmUgdHdvIHR5cGVzIG9mIG5vdGlmaWVycyBpbiBJT0FTSUQgY29y ZTogc3lzdGVtLXdpZGUgYW5kCitpb2FzaWRfc2V0LXdpZGUuCisKK1N5c3RlbS13aWRlIG5vdGlm aWVyIGlzIGNhdGVyaW5nIGZvciB1c2VycyB0aGF0IG5lZWQgdG8gaGFuZGxlIGFsbAorSU9BU0lE cyBpbiB0aGUgc3lzdGVtLiBFLmcuIFRoZSBJT01NVSBkcml2ZXIgaGFuZGxlcyBhbGwgSU9BU0lE cy4KKworUGVyIGlvYXNpZF9zZXQgbm90aWZpZXIgY2FuIGJlIHVzZWQgYnkgVk0gc3BlY2lmaWMg Y29tcG9uZW50cyBzdWNoIGFzCitLVk0uIEFmdGVyIGFsbCwgZWFjaCBLVk0gaW5zdGFuY2Ugb25s eSBjYXJlcyBhYm91dCBJT0FTSURzIHdpdGhpbiBpdHMKK293biBzZXQuCisKKworQXRvbWljaXR5 Cit+fn5+fn5+fn4KK0lPQVNJRCBub3RpZmllcnMgYXJlIGF0b21pYyBkdWUgdG8gc3BpbmxvY2tz IHVzZWQgaW5zaWRlIHRoZSBJT0FTSUQKK2NvcmUuIEZvciB0YXNrcyBjYW5ub3QgYmUgY29tcGxl dGVkIGluIHRoZSBub3RpZmllciBoYW5kbGVyLCBhc3luYyB3b3JrCitjYW4gYmUgc3VibWl0dGVk IHRvIGNvbXBsZXRlIHRoZSB3b3JrIGxhdGVyIGFzIGxvbmcgYXMgdGhlcmUgaXMgbm8KK29yZGVy aW5nIHJlcXVpcmVtZW50LgorCitSZWZlcmVuY2UgY291bnRpbmcKKy0tLS0tLS0tLS0tLS0tLS0t LQorSU9BU0lEIGxpZmVjeWNsZSBtYW5hZ2VtZW50IGlzIGJhc2VkIG9uIHJlZmVyZW5jZSBjb3Vu dGluZy4gVXNlcnMgb2YKK0lPQVNJRCBpbnRlbmQgdG8gYWxpZ24gbGlmZWN5Y2xlIHdpdGggdGhl IElPQVNJRCBuZWVkIHRvIGhvbGQKK3JlZmVyZW5jZSBvZiB0aGUgSU9BU0lELiBJT0FTSUQgd2ls bCBub3QgYmUgcmV0dXJuZWQgdG8gdGhlIHBvb2wgZm9yCithbGxvY2F0aW9uIHVudGlsIGFsbCBy ZWZlcmVuY2VzIGFyZSBkcm9wcGVkLiBDYWxsaW5nIGlvYXNpZF9mcmVlKCkKK3dpbGwgbWFyayB0 aGUgSU9BU0lEIGFzIEZSRUVfUEVORElORyBpZiB0aGUgSU9BU0lEIGhhcyBvdXRzdGFuZGluZwor cmVmZXJlbmNlLiBpb2FzaWRfZ2V0KCkgaXMgbm90IGFsbG93ZWQgb25jZSBhbiBJT0FTSUQgaXMg aW4gdGhlCitGUkVFX1BFTkRJTkcgc3RhdGUuCisKK0V2ZW50IG5vdGlmaWNhdGlvbnMgYXJlIHVz ZWQgdG8gaW5mb3JtIHVzZXJzIG9mIElPQVNJRCBzdGF0dXMgY2hhbmdlLgorSU9BU0lEX0ZSRUUg ZXZlbnQgcHJvbXB0cyB1c2VycyB0byBkcm9wIHRoZWlyIHJlZmVyZW5jZXMgYWZ0ZXIKK2NsZWFy aW5nIGl0cyBjb250ZXh0LgorCitGb3IgZXhhbXBsZSwgb24gVlQtZCBwbGF0Zm9ybSB3aGVuIGFu IElPQVNJRCBpcyBmcmVlZCwgdGVhcmRvd24KK2FjdGlvbnMgYXJlIHBlcmZvcm1lZCBvbiBLVk0s IGRldmljZSBkcml2ZXIsIGFuZCBJT01NVSBkcml2ZXIuCitLVk0gc2hhbGwgcmVnaXN0ZXIgbm90 aWZpZXIgYmxvY2sgd2l0aDo6CisKKyBzdGF0aWMgc3RydWN0IG5vdGlmaWVyX2Jsb2NrIHBhc2lk X25iX2t2bSA9IHsKKwkubm90aWZpZXJfY2FsbCA9IHBhc2lkX3N0YXR1c19jaGFuZ2Vfa3ZtLAor CS5wcmlvcml0eSAgICAgID0gSU9BU0lEX1BSSU9fQ1BVLAorIH07CisKK1ZEQ00gZHJpdmVyIHNo YWxsIHJlZ2lzdGVyIG5vdGlmaWVyIGJsb2NrIHdpdGg6OgorCisgc3RhdGljIHN0cnVjdCBub3Rp Zmllcl9ibG9jayBwYXNpZF9uYl92ZGNtID0geworCS5ub3RpZmllcl9jYWxsID0gcGFzaWRfc3Rh dHVzX2NoYW5nZV92ZGNtLAorCS5wcmlvcml0eSAgICAgID0gSU9BU0lEX1BSSU9fREVWSUNFLAor IH07CisKK0luIGJvdGggY2FzZXMsIG5vdGlmaWVyIGJsb2NrcyBzaGFsbCBiZSByZWdpc3RlcmVk IG9uIHRoZSBJT0FTSUQgc2V0CitzdWNoIHRoYXQgKm9ubHkqIGV2ZW50cyBmcm9tIHRoZSBtYXRj aGluZyBWTSBpcyByZWNlaXZlZC4KKworSWYgS1ZNIGF0dGVtcHRzIHRvIHJlZ2lzdGVyIG5vdGlm aWVyIGJsb2NrIGJlZm9yZSB0aGUgSU9BU0lEIHNldCBpcworY3JlYXRlZCBmb3IgdGhlIE1NIHRv a2VuLCB0aGUgbm90aWZpZXIgYmxvY2sgd2lsbCBiZSBwbGFjZWQgb24gYQorcGVuZGluZyBsaXN0 IGluc2lkZSBJT0FTSUQgY29yZS4gT25jZSB0aGUgdG9rZW4gbWF0Y2hpbmcgSU9BU0lEIHNldAor aXMgY3JlYXRlZCwgSU9BU0lEIHdpbGwgcmVnaXN0ZXIgdGhlIG5vdGlmaWVyIGJsb2NrIGF1dG9t YXRpY2FsbHkuCitJT0FTSUQgY29yZSBkb2VzIG5vdCByZXBsYXkgZXZlbnRzIGZvciB0aGUgZXhp c3RpbmcgSU9BU0lEcyBpbiB0aGUKK3NldC4gRm9yIElPQVNJRCBzZXQgb2YgTU0gdHlwZSwgbm90 aWZpY2F0aW9uIGJsb2NrcyBjYW4gYmUgcmVnaXN0ZXJlZAorb24gZW1wdHkgc2V0cyBvbmx5LiBU aGlzIGlzIHRvIGF2b2lkIGxvc3QgZXZlbnRzLgorCitJT01NVSBkcml2ZXIgc2hhbGwgcmVnaXN0 ZXIgbm90aWZpZXIgYmxvY2sgb24gZ2xvYmFsIGNoYWluOjoKKworIHN0YXRpYyBzdHJ1Y3Qgbm90 aWZpZXJfYmxvY2sgcGFzaWRfbmJfdnRkID0geworCS5ub3RpZmllcl9jYWxsID0gcGFzaWRfc3Rh dHVzX2NoYW5nZV92dGQsCisJLnByaW9yaXR5ICAgICAgPSBJT0FTSURfUFJJT19JT01NVSwKKyB9 OworCitDdXN0b20gYWxsb2NhdG9yIEFQSXMKKy0tLS0tLS0tLS0tLS0tLS0tLS0tLQorCis6Ogor CisgaW50IGlvYXNpZF9yZWdpc3Rlcl9hbGxvY2F0b3Ioc3RydWN0IGlvYXNpZF9hbGxvY2F0b3Jf b3BzICphbGxvY2F0b3IpOworCisgdm9pZCBpb2FzaWRfdW5yZWdpc3Rlcl9hbGxvY2F0b3Ioc3Ry dWN0IGlvYXNpZF9hbGxvY2F0b3Jfb3BzICphbGxvY2F0b3IpOworCitBbGxvY2F0b3IgQ2hvaWNl cworfn5+fn5+fn5+fn5+fn5+fn4KK0lPQVNJRHMgYXJlIGFsbG9jYXRlZCBmb3IgYm90aCBob3N0 IGFuZCBndWVzdCBTVkEvSU9WQSB1c2FnZS4gSG93ZXZlciwKK2FsbG9jYXRvcnMgY2FuIGJlIGRp ZmZlcmVudC4gRm9yIGV4YW1wbGUsIG9uIFZULWQgZ3Vlc3QgUEFTSUQKK2FsbG9jYXRpb24gbXVz dCBiZSBwZXJmb3JtZWQgdmlhIGEgdmlydHVhbCBjb21tYW5kIGludGVyZmFjZSB3aGljaCBpcwor ZW11bGF0ZWQgYnkgVk1NLgorCitJT0FTSUQgY29yZSBoYXMgdGhlIG5vdGlvbiBvZiAiY3VzdG9t IGFsbG9jYXRvciIgc3VjaCB0aGF0IGd1ZXN0IGNhbgorcmVnaXN0ZXIgdmlydHVhbCBjb21tYW5k IGFsbG9jYXRvciB0aGF0IHByZWNlZGVzIHRoZSBkZWZhdWx0IG9uZS4KKworTmFtZXNwYWNlcwor fn5+fn5+fn5+fgorSU9BU0lEcyBhcmUgbGltaXRlZCBzeXN0ZW0gcmVzb3VyY2VzIHRoYXQgZGVm YXVsdCB0byAyMCBiaXRzIGluCitzaXplLiBTaW5jZSBlYWNoIGRldmljZSBoYXMgaXRzIG93biB0 YWJsZSwgdGhlb3JldGljYWxseSB0aGUgbmFtZXNwYWNlCitjYW4gYmUgcGVyIGRldmljZSBhbHNv LiBIb3dldmVyLCBmb3Igc2VjdXJpdHkgcmVhc29ucyBzaGFyaW5nIFBBU0lECit0YWJsZXMgYW1v bmcgZGV2aWNlcyBhcmUgbm90IGdvb2QgZm9yIGlzb2xhdGlvbi4gVGhlcmVmb3JlLCBJT0FTSUQK K25hbWVzcGFjZSBpcyBzeXN0ZW0td2lkZS4KKworVGhlcmUgYXJlIGFsc28gb3RoZXIgcmVhc29u cyB0byBoYXZlIHRoaXMgc2ltcGxlciBzeXN0ZW0td2lkZQorbmFtZXNwYWNlLiBUYWtlIFZULWQg YXMgYW4gZXhhbXBsZSwgVlQtZCBzdXBwb3J0cyBzaGFyZWQgd29ya3F1ZXVlCithbmQgRU5RQ01E WzFdIHdoZXJlIG9uZSBJT0FTSUQgY291bGQgYmUgdXNlZCB0byBzdWJtaXQgd29yayBvbgorbXVs dGlwbGUgZGV2aWNlcyB0aGF0IGFyZSBzaGFyZWQgd2l0aCBvdGhlciBWTXMuIFRoaXMgcmVxdWly ZXMgSU9BU0lECit0byBiZSBzeXN0ZW0td2lkZS4gVGhpcyBpcyBhbHNvIHRoZSByZWFzb24gd2h5 IGd1ZXN0cyBtdXN0IHVzZSBhbgorZW11bGF0ZWQgdmlydHVhbCBjb21tYW5kIGludGVyZmFjZSB0 byBhbGxvY2F0ZSBJT0FTSUQgZnJvbSB0aGUgaG9zdC4KKworCitMaWZlIGN5Y2xlCis9PT09PT09 PT09CitUaGlzIHNlY3Rpb24gY292ZXJzIElPQVNJRCBsaWZlY3ljbGUgbWFuYWdlbWVudCBmb3Ig Ym90aCBiYXJlLW1ldGFsCithbmQgZ3Vlc3QgdXNhZ2VzLiBJbiBiYXJlLW1ldGFsIFNWQSwgTU1V IG5vdGlmaWVyIGlzIGRpcmVjdGx5IGhvb2tlZAordXAgd2l0aCBJT01NVSBkcml2ZXIsIHRoZXJl Zm9yZSB0aGUgcHJvY2VzcyBhZGRyZXNzIHNwYWNlIChNTSkKK2xpZmVjeWNsZSBpcyBhbGlnbmVk IHdpdGggSU9BU0lELgorCitIb3dldmVyLCBndWVzdCBNTVUgbm90aWZpZXIgaXMgbm90IGF2YWls YWJsZSB0byBob3N0IElPTU1VIGRyaXZlciwKK3doZW4gZ3Vlc3QgTU0gdGVybWluYXRlcyB1bmV4 cGVjdGVkbHksIHRoZSBldmVudHMgaGF2ZSB0byBnbyB0aHJvdWdoCitWRklPIGFuZCBJT01NVSBV QVBJIHRvIHJlYWNoIGhvc3QgSU9NTVUgZHJpdmVyLiBUaGVyZSBhcmUgYWxzbyBtb3JlCitwYXJ0 aWVzIGludm9sdmVkIGluIGd1ZXN0IFNWQSwgZS5nLiBvbiBJbnRlbCBWVC1kIHBsYXRmb3JtLCBJ T0FTSURzCithcmUgdXNlZCBieSBJT01NVSBkcml2ZXIsIEtWTSwgVkRDTSwgYW5kIFZGSU8uCisK K05hdGl2ZSBJT0FTSUQgTGlmZSBDeWNsZSAoVlQtZCBFeGFtcGxlKQorLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCisKK1RoZSBub3JtYWwgZmxvdyBvZiBuYXRpdmUgU1ZB IGNvZGUgd2l0aCBJbnRlbCBEYXRhIFN0cmVhbWluZworQWNjZWxlcmF0b3IoRFNBKSBbMl0gYXMg ZXhhbXBsZToKKworMS4gSG9zdCB1c2VyIG9wZW5zIGFjY2VsZXJhdG9yIEZELCBlLmcuIERTQSBk cml2ZXIsIG9yIHVhY2NlOworMi4gRFNBIGRyaXZlciBhbGxvY2F0ZSBXUSwgZG8gc3ZhX2JpbmRf ZGV2aWNlKCk7CiszLiBJT01NVSBkcml2ZXIgY2FsbHMgaW9hc2lkX2FsbG9jKCksIHRoZW4gYmlu ZCBQQVNJRCB3aXRoIGRldmljZSwKKyAgIG1tdV9ub3RpZmllcl9nZXQoKQorNC4gRE1BIHN0YXJ0 cyBieSBEU0EgZHJpdmVyIHVzZXJzcGFjZQorNS4gRFNBIHVzZXJzcGFjZSBjbG9zZSBGRAorNi4g RFNBL3VhY2NlIGtlcm5lbCBkcml2ZXIgaGFuZGxlcyBGRC5jbG9zZSgpCis3LiBEU0EgZHJpdmVy IHN0b3BzIERNQQorOC4gRFNBIGRyaXZlciBjYWxscyBzdmFfdW5iaW5kX2RldmljZSgpOworOS4g SU9NTVUgZHJpdmVyIGRvZXMgdW5iaW5kLCBjbGVhcnMgUEFTSUQgY29udGV4dCBpbiBJT01NVSwg Zmx1c2gKKyAgIFRMQnMuIG1tdV9ub3RpZmllcl9wdXQoKSBjYWxsZWQuCisxMC4gbW11X25vdGlm aWVyLnJlbGVhc2UoKSBjYWxsZWQsIElPTU1VIFNWQSBjb2RlIGNhbGxzIGlvYXNpZF9mcmVlKCkq CisxMS4gVGhlIElPQVNJRCBpcyByZXR1cm5lZCB0byB0aGUgcG9vbCwgcmVjbGFpbWVkLgorCis6 OgorCisgICAqIFdpdGggRU5RQ01ELCBQQVNJRCB1c2VkIG9uIFZULWQgaXMgbm90IHJlbGVhc2Vk IGluIG1tdV9ub3RpZmllcigpIGJ1dAorICAgICBtbWRyb3AoKS4gbW1kcm9wIGNvbWVzIGFmdGVy IEZEIGNsb3NlLiBTaG91bGQgbm90IG1hdHRlci4KKyAgICAgSWYgdGhlIHVzZXIgcHJvY2VzcyBk aWVzIHVuZXhwZWN0ZWRseSwgU3RlcCAjMTAgbWF5IGNvbWUgYmVmb3JlCisgICAgIFN0ZXAgIzUs IGluIGJldHdlZW4sIGFsbCBETUEgZmF1bHRzIGRpc2NhcmRlZC4gUFJRIHJlc3BvbmRlZCB3aXRo CisgICAgIGNvZGUgSU5WQUxJRCBSRVFVRVNULgorCitEdXJpbmcgdGhlIG5vcm1hbCB0ZWFyZG93 biwgdGhlIGZvbGxvd2luZyB0aHJlZSBzdGVwcyB3b3VsZCBoYXBwZW4gaW4KK29yZGVyOgorCisx LiBEZXZpY2UgZHJpdmVyIHN0b3BzIERNQSByZXF1ZXN0CisyLiBJT01NVSBkcml2ZXIgdW5iaW5k cyBQQVNJRCBhbmQgbW0sIGZsdXNoIGFsbCBUTEJzLCBkcmFpbiBpbi1mbGlnaHQKKyAgIHJlcXVl c3RzLgorMy4gSU9BU0lEIGZyZWVkCisKK0V4Y2VwdGlvbiBoYXBwZW5zIHdoZW4gcHJvY2VzcyB0 ZXJtaW5hdGVzICpiZWZvcmUqIGRldmljZSBkcml2ZXIgc3RvcHMKK0RNQSBhbmQgY2FsbCBJT01N VSBkcml2ZXIgdG8gdW5iaW5kLiBUaGUgZmxvdyBvZiBwcm9jZXNzIGV4aXN0cyBhcmUgYXMKK2Zv bGxvd3M6CisKKzo6CisKKyAgIGRvX2V4aXQoKSB7CisJZXhpdF9tbSgpIHsKKwkJbW1fcHV0KCk7 CisJCWV4aXRfbW1hcCgpIHsKKwkJCWludGVsX2ludmFsaWRhdGVfcmFuZ2UoKSAvL21tdSBub3Rp ZmllcgorCQkJdGxiX2ZpbmlzaF9tbXUoKQorCQkJbW11X25vdGlmaWVyX3JlbGVhc2UobW0pIHsK KwkJCQlpbnRlbF9pb21tdV9yZWxlYXNlKCkgeworICAgWzJdCQkJCQlpbnRlbF9pb21tdV90ZWFy ZG93bl9wYXNpZCgpOworICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGlu dGVsX2lvbW11X2ZsdXNoX3RsYnMoKTsKKwkJCQl9CisJCQkJLy8gdGxiX2ludmFsaWRhdGVfcmFu Z2UgY2IgcmVtb3ZlZAorCQkJfQorCQkJdW5tYXBfdm1hcygpOworICAgICAgICAgICAgICAgICAg ICAgICAgZnJlZV9wZ3RhYmxlcygpOyAvLyBJT01NVSBjYW5ub3Qgd2FsayBQR1QgYWZ0ZXIgdGhp cworCQl9OworCX0KKwlleGl0X2ZpbGVzKHRzaykgeworCQljbG9zZV9maWxlcygpIHsKKwkJCWRz YV9jbG9zZSgpOworICAgWzFdCQkJZHNhX3N0b3BfZG1hKCk7CisgICAgICAgICAgICAgICAgICAg ICAgICBpbnRlbF9zdm1fdW5iaW5kX3Bhc2lkKCk7IC8vbm90aGluZyB0byBkbworCQl9CisJfQor ICAgfQorCisgICBtbWRyb3AoKSAvKiBzb21lIHJhbmRvbSB0aW1lIGxhdGVyLCBsYXp5IG1tIHVz ZXIgKi8geworICAgCW1tX2ZyZWVfcGdkKCk7CisgICAgICAgIGRlc3Ryb3lfY29udGV4dChtbSk7 IHsKKyAgIFszXQkgICAgICAgIGlvYXNpZF9mcmVlKCk7CisJfQorICAgfQorCitBcyBzaG93biBp biB0aGUgbGlzdCBhYm92ZSwgc3RlcCAjMiBjb3VsZCBoYXBwZW4gYmVmb3JlCisjMS4gVW5yZWNv dmVyYWJsZShVUikgZmF1bHRzIGNvdWxkIGhhcHBlbiBiZXR3ZWVuICMyIGFuZCAjMS4KKworQWxz byBub3RpY2UgdGhhdCBUTEIgaW52YWxpZGF0aW9uIG9jY3VycyBhdCBtbXVfbm90aWZpZXIKK2lu dmFsaWRhdGVfcmFuZ2UgY2FsbGJhY2sgYXMgd2VsbCBhcyB0aGUgcmVsZWFzZSBjYWxsYmFjay4g VGhlIHJlYXNvbgoraXMgdGhhdCByZWxlYXNlIGNhbGxiYWNrIHdpbGwgZGVsZXRlIElPTU1VIGRy aXZlciBmcm9tIHRoZSBub3RpZmllcgorY2hhaW4gd2hpY2ggbWF5IHNraXAgaW52YWxpZGF0ZV9y YW5nZSgpIGNhbGxzIGR1cmluZyB0aGUgZXhpdCBwYXRoLgorCitUbyBhdm9pZCB1bm5lY2Vzc2Fy eSByZXBvcnRpbmcgb2YgVVIgZmF1bHQsIElPTU1VIGRyaXZlciBzaGFsbCBkaXNhYmxlCitmYXVs dCByZXBvcnRpbmcgYWZ0ZXIgZnJlZSBhbmQgYmVmb3JlIHVuYmluZC4KKworR3Vlc3QgSU9BU0lE IExpZmUgQ3ljbGUgKFZULWQgRXhhbXBsZSkKKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tCitHdWVzdCBJT0FTSUQgbGlmZSBjeWNsZSBzdGFydHMgd2l0aCBndWVzdCBkcml2 ZXIgb3BlbigpLCB0aGlzIGNvdWxkIGJlCit1YWNjZSBvciBpbmRpdmlkdWFsIGFjY2VsZXJhdG9y IGRyaXZlciBzdWNoIGFzIERTQS4gQXQgRkQgb3BlbiwKK3N2YV9iaW5kX2RldmljZSgpIGlzIGNh bGxlZCB3aGljaCB0cmlnZ2VycyBhIHNlcmllcyBvZiBhY3Rpb25zLgorCitUaGUgZXhhbXBsZSBi ZWxvdyBpcyBhbiBpbGx1c3RyYXRpb24gb2YgKm5vcm1hbCogb3BlcmF0aW9ucyB0aGF0CitpbnZv bHZlcyAqYWxsKiB0aGUgU1cgY29tcG9uZW50cyBpbiBWVC1kLiBUaGUgZmxvdyBjYW4gYmUgc2lt cGxlciBpZgorbm8gRU5RQ01EIGlzIHN1cHBvcnRlZC4KKworOjoKKworICAgICBWRklPICAgICAg ICBJT01NVSAgICAgICAgS1ZNICAgICAgICBWRENNICAgICAgICBJT0FTSUQgICAgICAgUmVmCisg ICAuLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4u Li4uLi4uLi4uLi4KKyAgIDEgICAgICAgICAgICAgaW9hc2lkX3JlZ2lzdGVyX25vdGlmaWVyL19t bSgpCisgICAyIGlvYXNpZF9hbGxvYygpICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAxCisgICAzIGJpbmRfZ3Bhc2lkKCkKKyAgIDQgICAgICAgICAgICAgaW9t bXVfYmluZCgpLT5pb2FzaWRfZ2V0KCkgICAgICAgICAgICAgICAgICAgICAgIDIKKyAgIDUgICAg ICAgICAgICAgaW9hc2lkX25vdGlmeShCSU5EKQorICAgNiAgICAgICAgICAgICAgICAgICAgICAg ICAgLT4gaW9hc2lkX2dldCgpICAgICAgICAgICAgICAgICAgICAgMworICAgNyAgICAgICAgICAg ICAgICAgICAgICAgICAgLT4gdm1jc191cGRhdGVfYXRvbWljKCkKKyAgIDggbWRldl93cml0ZShn cGFzaWQpCisgICA5ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaHBhc2lkPQor ICAgMTAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGZpbmRfYnlfc3BpZChncGFz aWQpICAgICAgNAorICAgMTEgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHZkZXZf d3JpdGUoaHBhc2lkKQorICAgMTIgLS0tLS0tLS0gR1VFU1QgU1RBUlRTIERNQSAtLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLQorICAgMTMgLS0tLS0tLS0gR1VFU1QgU1RPUFMgRE1BIC0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tCisgICAxNCBtZGV2X2NsZWFyKGdwYXNpZCkKKyAgIDE1ICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB2ZGV2X2NsZWFyKGhwYXNpZCkKKyAgIDE2ICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBpb2FzaWRfcHV0KCkgICAgICAgICAgICAg ICAzCisgICAxNyB1bmJpbmRfZ3Bhc2lkKCkKKyAgIDE4ICAgICAgICAgICAgaW9tbXVfdWJpbmQo KQorICAgMTkgICAgICAgICAgICBpb2FzaWRfbm90aWZ5KFVOQklORCkKKyAgIDIwICAgICAgICAg ICAgICAgICAgICAgICAgICAtPiB2bWNzX3VwZGF0ZV9hdG9taWMoKQorICAgMjEgICAgICAgICAg ICAgICAgICAgICAgICAgIC0+IGlvYXNpZF9wdXQoKSAgICAgICAgICAgICAgICAgICAgIDIKKyAg IDIyIGlvYXNpZF9mcmVlKCkgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAxCisgICAyMyAgICAgICAgICAgIGlvYXNpZF9wdXQoKSAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgMAorICAgMjQgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgUmVjbGFpbWVkCisgICAtLS0tLS0tLS0tLS0tLSBOZXcgTGlm ZSBDeWNsZSBCZWdpbiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCisgICAxICBpb2FzaWRf YWxsb2MoKSAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAtPiAgICAgICAgICAgMQor CisgICBOb3RlOiBJT0FTSUQgTm90aWZpY2F0aW9uIEV2ZW50czogRlJFRSwgQklORCwgVU5CSU5E CisKK0V4Y2VwdGlvbiBjYXNlcyBhcmlzZSB3aGVuIGEgZ3Vlc3QgY3Jhc2hlcyBvciBhIG1hbGlj aW91cyBndWVzdAorYXR0ZW1wdHMgdG8gY2F1c2UgZGlzcnVwdGlvbiBvbiB0aGUgaG9zdCBzeXN0 ZW0uIFRoZSBmYXVsdCBoYW5kbGluZworcnVsZXMgYXJlOgorCisxLiBJT0FTSUQgZnJlZSBtdXN0 ICphbHdheXMqIHN1Y2NlZWQuCisyLiBBbiBpbmFjdGl2ZSBwZXJpb2QgbWF5IGJlIHJlcXVpcmVk IGJlZm9yZSB0aGUgZnJlZWQgSU9BU0lEIGlzCisgICByZWNsYWltZWQuIER1cmluZyB0aGlzIHBl cmlvZCwgY29uc3VtZXJzIG9mIElPQVNJRCBwZXJmb3JtIGNsZWFudXAuCiszLiBNYWxmdW5jdGlv biBpcyBsaW1pdGVkIHRvIHRoZSBndWVzdCBvd25lZCByZXNvdXJjZXMgZm9yIGFsbAorICAgcHJv Z3JhbW1pbmcgZXJyb3JzLgorCitUaGUgcHJpbWFyeSBzb3VyY2Ugb2YgZXhjZXB0aW9uIGlzIHdo ZW4gdGhlIGZvbGxvd2luZyBhcmUgb3V0IG9mCitvcmRlcjoKKworMS4gU3RhcnQvU3RvcCBvZiBE TUEgYWN0aXZpdHkKKyAgIChHdWVzdCBkZXZpY2UgZHJpdmVyLCBtZGV2IHZpYSBWRklPKQorMi4g U2V0dXAvVGVhcmRvd24gb2YgSU9NTVUgUEFTSUQgY29udGV4dCwgSU9UTEIsIERldlRMQiBmbHVz aGVzCisgICAoSG9zdCBJT01NVSBkcml2ZXIgYmluZC91bmJpbmQpCiszLiBTZXR1cC9UZWFyZG93 biBvZiBWTUNTIFBBU0lEIHRyYW5zbGF0aW9uIHRhYmxlIGVudHJpZXMgKEtWTSkgaW4KKyAgIGNh c2Ugb2YgRU5RQ01ECis0LiBQcm9ncmFtbWluZy9DbGVhcmluZyBob3N0IFBBU0lEIGluIFZEQ00g KEhvc3QgVkRDTSBkcml2ZXIpCis1LiBJT0FTSUQgYWxsb2MvZnJlZSAoSG9zdCBJT0FTSUQpCisK K1ZGSU8gaXMgdGhlICpvbmx5KiB1c2VyLWtlcm5lbCBpbnRlcmZhY2UsIHdoaWNoIGlzIHVsdGlt YXRlbHkKK3Jlc3BvbnNpYmxlIGZvciBleGNlcHRpb24gaGFuZGxpbmdzLgorCisjMSBpcyBwcm9j ZXNzZWQgdGhlIHNhbWUgd2F5IGFzIHRoZSBhc3NpZ25lZCBkZXZpY2UgdG9kYXkgYmFzZWQgb24K K2RldmljZSBmaWxlIGRlc2NyaXB0b3JzIGFuZCBldmVudHMuIFRoZXJlIGlzIG5vIHNwZWNpYWwg aGFuZGxpbmcuCisKKyMzIGlzIGJhc2VkIG9uIGJpbmQvdW5iaW5kIGV2ZW50cyBlbWl0dGVkIGJ5 ICMyLgorCisjNCBpcyBuYXR1cmFsbHkgYWxpZ25lZCB3aXRoIElPQVNJRCBsaWZlIGN5Y2xlIGlu IHRoYXQgYW4gaWxsZWdhbAorZ3Vlc3QgUEFTSUQgcHJvZ3JhbW1pbmcgd291bGQgZmFpbCBpbiBv YnRhaW5pbmcgcmVmZXJlbmNlIG9mIHRoZQorbWF0Y2hpbmcgaG9zdCBJT0FTSUQuCisKKyM1IGlz IHNpbWlsYXIgdG8gIzQuIFRoZSBmYXVsdCB3aWxsIGJlIHJlcG9ydGVkIHRvIHRoZSB1c2VyIGlm IFBBU0lECit1c2VkIGluIHRoZSBFTlFDTUQgaXMgbm90IHNldCB1cCBpbiBWTUNTIFBBU0lEIHRy YW5zbGF0aW9uIHRhYmxlLgorCitUaGVyZWZvcmUsIHRoZSByZW1haW5pbmcgb3V0IG9mIG9yZGVy IHByb2JsZW0gaXMgYmV0d2VlbiAjMiBhbmQKKyM1LiBJLmUuIHVuYmluZCB2cy4gZnJlZS4gTW9y ZSBzcGVjaWZpY2FsbHksIGZyZWUgYmVmb3JlIHVuYmluZC4KKworSU9BU0lEIG5vdGlmaWVyIGFu ZCByZWZjb3VudGluZyBhcmUgdXNlZCB0byBlbnN1cmUgb3JkZXIuIEZvbGxvd2luZworYSBwdWJs aXNoZXItc3Vic2NyaWJlciBwYXR0ZXJuIHdoZXJlOgorCistIFB1Ymxpc2hlcnM6IFZGSU8gJiBJ T01NVQorLSBTdWJzY3JpYmVyczogS1ZNLCBWRENNLCBJT01NVQorCitJT0FTSUQgbm90aWZpZXIg aXMgYXRvbWljIHdoaWNoIHJlcXVpcmVzIHN1YnNjcmliZXJzIHRvIGRvIHF1aWNrCitoYW5kbGlu ZyBvZiB0aGUgZXZlbnQgaW4gdGhlIGF0b21pYyBjb250ZXh0LiBXb3JrcXVldWUgY2FuIGJlIHVz ZWQgZm9yCithbnkgcHJvY2Vzc2luZyB0aGF0IHJlcXVpcmVzIHRocmVhZCBjb250ZXh0LiBJT0FT SUQgcmVmZXJlbmNlIG11c3QgYmUKK2FjcXVpcmVkIGJlZm9yZSByZWNlaXZpbmcgdGhlIEZSRUUg ZXZlbnQuIFRoZSByZWZlcmVuY2UgbXVzdCBiZQorZHJvcHBlZCBhdCB0aGUgZW5kIG9mIHRoZSBw cm9jZXNzaW5nIGluIG9yZGVyIHRvIHJldHVybiB0aGUgSU9BU0lEIHRvCit0aGUgcG9vbC4KKwor TGV0J3MgZXhhbWluZSB0aGUgSU9BU0lEIGxpZmUgY3ljbGUgYWdhaW4gd2hlbiBmcmVlIGhhcHBl bnMgKmJlZm9yZSoKK3VuYmluZC4gVGhpcyBjb3VsZCBiZSBhIHJlc3VsdCBvZiBtaXNiZWhhdmlu ZyBndWVzdHMgb3IgY3Jhc2guIEFzc3VtaW5nCitWRklPIGNhbm5vdCBlbmZvcmNlIHVuYmluZC0+ ZnJlZSBvcmRlci4gTm90aWNlIHRoYXQgdGhlIHNldHVwIHBhcnQgdXAKK3VudGlsIHN0ZXAgIzEy IGlzIGlkZW50aWNhbCB0byB0aGUgbm9ybWFsIGNhc2UsIHRoZSBmbG93IGJlbG93IHN0YXJ0cwor d2l0aCBzdGVwIDEzLgorCis6OgorCisgICAgIFZGSU8gICAgICAgIElPTU1VICAgICAgICBLVk0g ICAgICAgIFZEQ00gICAgICAgIElPQVNJRCAgICAgICBSZWYKKyAgIC4uLi4uLi4uLi4uLi4uLi4u Li4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLi4uLgorICAgMTMg LS0tLS0tLS0gR1VFU1QgU1RBUlRTIERNQSAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQorICAg MTQgLS0tLS0tLS0gKkdVRVNUIE1JU0JFSEFWRVMhISEqIC0tLS0tLS0tLS0tLS0tLS0KKyAgIDE1 IGlvYXNpZF9mcmVlKCkKKyAgIDE2ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgaW9hc2lkX25vdGlmeShGUkVFKQorICAgMTcgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICBtYXJrX2lvYXNpZF9pbmFjdGl2ZVsxXQorICAgMTggICAg ICAgICAgICAgICAgICAgICAgICAgIGt2bV9uYl9oYW5kbGVyKEZSRUUpCisgICAxOSAgICAgICAg ICAgICAgICAgICAgICAgICAgdm1jc191cGRhdGVfYXRvbWljKCkKKyAgIDIwICAgICAgICAgICAg ICAgICAgICAgICAgICBpb2FzaWRfcHV0X2xvY2tlZCgpICAgLT4gICAgICAgICAgIDMKKyAgIDIx ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB2ZGNtX25iX2hhbmRsZXIoRlJFRSkK KyAgIDIyICAgICAgICAgICAgaW9tbV9uYl9oYW5kbGVyKEZSRUUpCisgICAyMyBpb2FzaWRfZnJl ZSgpIHJldHVybnNbMl0gICAgICAgICAgc2NoZWR1bGVfd29yaygpICAgICAgICAgICAyCisgICAy NCAgICAgICAgICAgIHNjaGVkdWxlX3dvcmsoKSAgICAgICAgdmRldl9jbGVhcl93ayhocGFzaWQp CisgICAyNSAgICAgICAgICAgIHRlYXJkb3duX3Bhc2lkX3drKCkKKyAgIDI2ICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICBpb2FzaWRfcHV0KCkgLT4gICAgICAgICAgIDEKKyAgIDI3 ICAgICAgICAgICAgaW9hc2lkX3B1dCgpICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIDAKKyAgIDI4ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIFJlY2xhaW1lZAorICAgMjkgdW5iaW5kX2dwYXNpZCgpCisgICAzMCAgICAgICAgICAgIGlv bW11X3VuYmluZCgpLT5pb2FzaWRfZmluZCgpIEZhaWxzWzNdCisgICAtLS0tLS0tLS0tLS0tLSBO ZXcgTGlmZSBDeWNsZSBCZWdpbiAtLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCisKK05vdGU6 CisKKzEuIEJ5IG1hcmtpbmcgSU9BU0lEIGluYWN0aXZlIGF0IHN0ZXAgIzE3LCBubyBuZXcgcmVm ZXJlbmNlcyBjYW4gYmUKKyAgIGhlbGQuIGlvYXNpZF9nZXQvZmluZCgpIHdpbGwgcmV0dXJuIC1F Tk9FTlQ7CisyLiBBZnRlciBzdGVwICMyMywgYWxsIGV2ZW50cyBjYW4gZ28gb3V0IG9mIG9yZGVy LiBTaGFsbCBub3QgYWZmZWN0CisgICB0aGUgb3V0Y29tZS4KKzMuIElPTU1VIGRyaXZlciBmYWls cyB0byBmaW5kIHByaXZhdGUgZGF0YSBmb3IgdW5iaW5kaW5nLiBJZiB1bmJpbmQgaXMKKyAgIGNh bGxlZCBhZnRlciB0aGUgc2FtZSBJT0FTSUQgaXMgYWxsb2NhdGVkIGZvciB0aGUgc2FtZSBndWVz dCBhZ2FpbiwKKyAgIHRoaXMgaXMgYSBwcm9ncmFtbWluZyBlcnJvci4gVGhlIGRhbWFnZSBpcyBs aW1pdGVkIHRvIHRoZSBndWVzdAorICAgaXRzZWxmIHNpbmNlIHVuYmluZCBwZXJmb3JtcyBwZXJt aXNzaW9uIGNoZWNraW5nIGJhc2VkIG9uIHRoZQorICAgSU9BU0lEIHNldCBhc3NvY2lhdGVkIHdp dGggdGhlIGd1ZXN0IHByb2Nlc3MuCisKK0tWTSBQQVNJRCBUcmFuc2xhdGlvbiBUYWJsZSBVcGRh dGVzCit+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgorUGVyIFZNIFBBU0lEIHRy YW5zbGF0aW9uIHRhYmxlIGlzIG1haW50YWluZWQgYnkgS1ZNIGluIG9yZGVyIHRvCitzdXBwb3J0 IEVOUUNNRCBpbiB0aGUgZ3Vlc3QuIFRoZSB0YWJsZSBjb250YWlucyBob3N0LWd1ZXN0IFBBU0lE Cit0cmFuc2xhdGlvbnMgdG8gYmUgY29uc3VtZWQgYnkgQ1BVIHVjb2RlLiBUaGUgc3luY2hyb25p emF0aW9uIG9mIHRoZQorUEFTSUQgc3RhdGVzIGRlcGVuZHMgb24gVkZJTy9JT01NVSBkcml2ZXIs IHdoZXJlIElPQ1RMIGFuZCBhdG9taWMKK25vdGlmaWVycyBhcmUgdXNlZC4gS1ZNIG11c3QgcmVn aXN0ZXIgSU9BU0lEIG5vdGlmaWVyIHBlciBWTSBpbnN0YW5jZQorZHVyaW5nIGxhdW5jaCB0aW1l LiBUaGUgZm9sbG93aW5nIGV2ZW50cyBhcmUgaGFuZGxlZDoKKworMS4gQklORC9VTkJJTkQKKzIu IEZSRUUKKworUnVsZXM6CisKKzEuIE11bHRpcGxlIGRldmljZXMgY2FuIGJpbmQgd2l0aCB0aGUg c2FtZSBQQVNJRCwgdGhpcyBjYW4gYmUgZGlmZmVyZW50IFBDSQorICAgZGV2aWNlcyBvciBtZGV2 cyB3aXRoaW4gdGhlIHNhbWUgUENJIGRldmljZS4gSG93ZXZlciwgb25seSB0aGUKKyAgICpmaXJz dCogQklORCBhbmQgKmxhc3QqIFVOQklORCBlbWl0IG5vdGlmaWNhdGlvbnMuCisyLiBJT0FTSUQg Y29kZSBpcyByZXNwb25zaWJsZSBmb3IgZW5zdXJpbmcgdGhlIGNvcnJlY3RuZXNzIG9mIEgtRwor ICAgUEFTSUQgbWFwcGluZy4gVGhlcmUgaXMgbm8gbmVlZCBmb3IgS1ZNIHRvIHZhbGlkYXRlIHRo ZQorICAgbm90aWZpY2F0aW9uIGRhdGEuCiszLiBXaGVuIFVOQklORCBoYXBwZW5zICphZnRlciog RlJFRSwgS1ZNIHdpbGwgc2VlIGVycm9yIGluCisgICBpb2FzaWRfZ2V0KCkgZXZlbiB3aGVuIHRo ZSByZWNsYWltIGlzIG5vdCBkb25lLiBJT01NVSBkcml2ZXIgd2lsbAorICAgYWxzbyBhdm9pZCBz ZW5kaW5nIFVOQklORCBpZiB0aGUgUEFTSUQgaXMgYWxyZWFkeSBGUkVFLgorNC4gV2hlbiBLVk0g dGVybWluYXRlcyAqYmVmb3JlKiBGUkVFICYgVU5CSU5ELCByZWZlcmVuY2VzIHdpbGwgYmUKKyAg IGRyb3BwZWQgZm9yIGFsbCBob3N0IFBBU0lEcy4KKworVkRDTSBQQVNJRCBQcm9ncmFtbWluZwor fn5+fn5+fn5+fn5+fn5+fn5+fn5+fgorVkRDTSBjb21wb3NlcyB2aXJ0dWFsIGRldmljZXMgYW5k IGV4cG9zZXMgdGhlbSB0byB0aGUgZ3Vlc3RzLiBXaGVuCit0aGUgZ3Vlc3QgYWxsb2NhdGVzIGEg UEFTSUQgdGhlbiBwcm9ncmFtIGl0IHRvIHRoZSB2aXJ0dWFsIGRldmljZSwgVkRDTQoraW50ZXJj ZXB0cyB0aGUgcHJvZ3JhbW1pbmcgYXR0ZW1wdCB0aGVuIHByb2dyYW0gdGhlIG1hdGNoaW5nIGhv c3QKK1BBU0lEIG9uIHRvIHRoZSBoYXJkd2FyZS4KK0NvbnZlcnNlbHksIHdoZW4gYSBkZXZpY2Ug aXMgZ29pbmcgYXdheSwgVkRDTSBtdXN0IGJlIGluZm9ybWVkIHN1Y2gKK3RoYXQgUEFTSUQgY29u dGV4dCBvbiB0aGUgaGFyZHdhcmUgY2FuIGJlIGNsZWFyZWQuIFRoZXJlIGNvdWxkIGJlCittdWx0 aXBsZSBtZGV2cyBhc3NpZ25lZCB0byBkaWZmZXJlbnQgZ3Vlc3RzIGluIHRoZSBzYW1lIFZEQ00u IFNpbmNlCit0aGUgUEFTSUQgdGFibGUgaXMgc2hhcmVkIGF0IFBDSSBkZXZpY2UgbGV2ZWwsIGxh enkgY2xlYXJpbmcgaXMgbm90CitzZWN1cmUuIEEgbWFsaWNpb3VzIGd1ZXN0IGNhbiBhdHRhY2sg YnkgdXNpbmcgbmV3bHkgZnJlZWQgUEFTSURzIHRoYXQKK2FyZSBhbGxvY2F0ZWQgYnkgYW5vdGhl ciBndWVzdC4KKworQnkgaG9sZGluZyBhIHJlZmVyZW5jZSBvZiB0aGUgUEFTSUQgdW50aWwgVkRD TSBjbGVhbnMgdXAgdGhlIEhXIGNvbnRleHQsCitpdCBpcyBndWFyYW50ZWVkIHRoYXQgUEFTSUQg bGlmZSBjeWNsZXMgZG8gbm90IGNyb3NzIHdpdGhpbiB0aGUgc2FtZQorZGV2aWNlLgorCisKK1Jl ZmVyZW5jZQorPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PQorMS4gaHR0cHM6Ly9zb2Z0d2FyZS5pbnRlbC5jb20vc2l0ZXMvZGVmYXVsdC9maWxlcy9t YW5hZ2VkL2M1LzE1L2FyY2hpdGVjdHVyZS1pbnN0cnVjdGlvbi1zZXQtZXh0ZW5zaW9ucy1wcm9n cmFtbWluZy1yZWZlcmVuY2UucGRmCisKKzIuIGh0dHBzOi8vMDEub3JnL2Jsb2dzLzIwMTkvaW50 cm9kdWNpbmctaW50ZWwtZGF0YS1zdHJlYW1pbmctYWNjZWxlcmF0b3IKKworMy4gaHR0cHM6Ly9z b2Z0d2FyZS5pbnRlbC5jb20vZW4tdXMvZG93bmxvYWQvaW50ZWwtZGF0YS1zdHJlYW1pbmctYWNj ZWxlcmF0b3ItcHJlbGltaW5hcnktYXJjaGl0ZWN0dXJlLXNwZWNpZmljYXRpb24KLS0gCjIuNy40 CgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwppb21tdSBt YWlsaW5nIGxpc3QKaW9tbXVAbGlzdHMubGludXgtZm91bmRhdGlvbi5vcmcKaHR0cHM6Ly9saXN0 cy5saW51eGZvdW5kYXRpb24ub3JnL21haWxtYW4vbGlzdGluZm8vaW9tbXU= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7D89C433E1 for ; Sat, 22 Aug 2020 04:28:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6FDCF2071A for ; Sat, 22 Aug 2020 04:28:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="rxsqS8Pr" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726218AbgHVE2h (ORCPT ); Sat, 22 Aug 2020 00:28:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60842 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725807AbgHVE2Y (ORCPT ); Sat, 22 Aug 2020 00:28:24 -0400 Received: from mail-pj1-x1041.google.com (mail-pj1-x1041.google.com [IPv6:2607:f8b0:4864:20::1041]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 890C7C061573 for ; Fri, 21 Aug 2020 21:28:24 -0700 (PDT) Received: by mail-pj1-x1041.google.com with SMTP id q93so1656738pjq.0 for ; Fri, 21 Aug 2020 21:28:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=aOT5t5tsGk7zRkfUzo++dr5YPDmf2c1MUVXVEa0SPao=; b=rxsqS8Pri0Qn5Ds/DSubiRLLrZ4K+SYlbOi4VVrpZ8DYheYQemgKiTMN0HKrAkNaNG UKIlmhhC30qGforLqflJRG84EP/bpSnmc3UilciUq08oxv1K7pCMFMTdysN1hA/VgHXk sr2gcZF+jTB1A4NwBXm1mtbkhalu01RDSjxoeBFjalhZTWMEoFiKIBIPlAS/f4ksgmvk Q01C++UaDK/zxG2BauSnyV4II4mmvj2TjEpaP8iWoL18U7D1SM/iaIXrTfBE/OLnlq30 X9ezI8seJUQzlq4jWyNMpqUMo/QoKJ14raVkgZRw7iIGeMUAVoMZAS6fqBLibjaidGKI gwaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=aOT5t5tsGk7zRkfUzo++dr5YPDmf2c1MUVXVEa0SPao=; b=hxftA5s5DL33A/ryMAG7soJgMZkVlQWQQE6Z80Bd5ce2qDIlfIZolpWbwvxTVPo8b2 lTxDVUMdVlz8rjfIcJsSCFyPk1YbIvr82ln0rO6WT1N7Hife1VJyBtZva/UXwpV5wKTG rlAP0prpkAFpRvyO9N1Pp52q5vKfCKV6NF3SbmnCBCH4WZl1RExtsuSXhHaI0FtN37k+ fkdjN+9/FGu+Q1aEE5ouSXDKnaYXAjPML5M1AbsXfCrzLYiaAmkzT2mB02ZvYW9XVgh/ JeijPfgXpu3kQun9mtgtJYQrreajIo6CKXplA92gH7kupRgXHbAIrriOgFYGTwZi1ES2 Qj8g== X-Gm-Message-State: AOAM533lSFUpe353UbgD+keR30/mK3XG2CHQgcWgoNPl0rVQRUm438b2 aCSBLHHoKxZV9IxCYWtPcag= X-Google-Smtp-Source: ABdhPJwQZqduhA5Sqkeshos1ceV39Qc74CqyJ4sHhm2xbB00DhuIm77th+n707XKkjffOZ+s4R0hzA== X-Received: by 2002:a17:90a:9c3:: with SMTP id 61mr638955pjo.191.1598070503753; Fri, 21 Aug 2020 21:28:23 -0700 (PDT) Received: from jacob-builder.jf.intel.com ([192.55.55.43]) by smtp.gmail.com with ESMTPSA id q5sm3341582pgi.31.2020.08.21.21.28.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Aug 2020 21:28:23 -0700 (PDT) From: Jacob Pan X-Google-Original-From: Jacob Pan To: iommu@lists.linux-foundation.org, LKML , Jean-Philippe Brucker , "Lu Baolu" , Joerg Roedel , David Woodhouse Cc: Yi Liu , "Tian, Kevin" , Raj Ashok , Eric Auger , Wu Hao Subject: [PATCH v2 1/9] docs: Document IO Address Space ID (IOASID) APIs Date: Fri, 21 Aug 2020 21:35:10 -0700 Message-Id: <1598070918-21321-2-git-send-email-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1598070918-21321-1-git-send-email-jacob.jun.pan@linux.intel.com> References: <1598070918-21321-1-git-send-email-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org IOASID is used to identify address spaces that can be targeted by device DMA. It is a system-wide resource that is essential to its many users. This document is an attempt to help developers from all vendors navigate the APIs. At this time, ARM SMMU and Intel’s Scalable IO Virtualization (SIOV) enabled platforms are the primary users of IOASID. Examples of how SIOV components interact with IOASID APIs are provided in that many APIs are driven by the requirements from SIOV. Signed-off-by: Liu Yi L Signed-off-by: Wu Hao Signed-off-by: Jacob Pan --- Documentation/ioasid.rst | 618 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 618 insertions(+) create mode 100644 Documentation/ioasid.rst diff --git a/Documentation/ioasid.rst b/Documentation/ioasid.rst new file mode 100644 index 000000000000..b6a8cdc885ff --- /dev/null +++ b/Documentation/ioasid.rst @@ -0,0 +1,618 @@ +.. ioasid: + +===================================== +IO Address Space ID +===================================== + +IOASID is a generic name for PCIe Process Address ID (PASID) or ARM +SMMU sub-stream ID. An IOASID identifies an address space that DMA +requests can target. + +The primary use cases for IOASID are Shared Virtual Address (SVA) and +IO Virtual Address (IOVA). However, the requirements for IOASID +management can vary among hardware architectures. + +This document covers the generic features supported by IOASID +APIs. Vendor-specific use cases are also illustrated with Intel's VT-d +based platforms as the first example. + +.. contents:: :local: + +Glossary +======== +PASID - Process Address Space ID + +IOASID - IO Address Space ID (generic term for PCIe PASID and +sub-stream ID in SMMU) + +SVA/SVM - Shared Virtual Addressing/Memory + +ENQCMD - New Intel X86 ISA for efficient workqueue submission [1] + +DSA - Intel Data Streaming Accelerator [2] + +VDCM - Virtual device composition module [3] + +SIOV - Intel Scalable IO Virtualization + + +Key Concepts +============ + +IOASID Set +----------- +An IOASID set is a group of IOASIDs allocated from the system-wide +IOASID pool. An IOASID set is created and can be identified by a +token of u64. Refer to IOASID set APIs for more details. + +IOASID set is particularly useful for guest SVA where each guest could +have its own IOASID set for security and efficiency reasons. + +IOASID Set Private ID (SPID) +---------------------------- +SPIDs are introduced as IOASIDs within its set. Each SPID maps to a +system-wide IOASID but the namespace of SPID is within its IOASID +set. SPIDs can be used as guest IOASIDs where each guest could do +IOASID allocation from its own pool and map them to host physical +IOASIDs. SPIDs are particularly useful for supporting live migration +where decoupling guest and host physical resources are necessary. + +For example, two VMs can both allocate guest PASID/SPID #101 but map to +different host PASIDs #201 and #202 respectively as shown in the +diagram below. +:: + + .------------------. .------------------. + | VM 1 | | VM 2 | + | | | | + |------------------| |------------------| + | GPASID/SPID 101 | | GPASID/SPID 101 | + '------------------' -------------------' Guest + __________|______________________|______________________ + | | Host + v v + .------------------. .------------------. + | Host IOASID 201 | | Host IOASID 202 | + '------------------' '------------------' + | IOASID set 1 | | IOASID set 2 | + '------------------' '------------------' + +Guest PASID is treated as IOASID set private ID (SPID) within an +IOASID set, mappings between guest and host IOASIDs are stored in the +set for inquiry. + +IOASID APIs +=========== +To get the IOASID APIs, users must #include . These APIs +serve the following functionalities: + + - IOASID allocation/Free + - Group management in the form of ioasid_set + - Private data storage and lookup + - Reference counting + - Event notification in case of state change + +IOASID Set Level APIs +-------------------------- +For use cases such as guest SVA it is necessary to manage IOASIDs at +a group level. For example, VMs may allocate multiple IOASIDs for +guest process address sharing (vSVA). It is imperative to enforce +VM-IOASID ownership such that malicious guest cannot target DMA +traffic outside its own IOASIDs, or free an active IOASID belong to +another VM. +:: + + struct ioasid_set *ioasid_alloc_set(void *token, ioasid_t quota, u32 type) + + int ioasid_adjust_set(struct ioasid_set *set, int quota); + + void ioasid_set_get(struct ioasid_set *set) + + void ioasid_set_put(struct ioasid_set *set) + + void ioasid_set_get_locked(struct ioasid_set *set) + + void ioasid_set_put_locked(struct ioasid_set *set) + + int ioasid_set_for_each_ioasid(struct ioasid_set *sdata, + void (*fn)(ioasid_t id, void *data), + void *data) + + +IOASID set concept is introduced to represent such IOASID groups. Each +IOASID set is created with a token which can be one of the following +types: + + - IOASID_SET_TYPE_NULL (Arbitrary u64 value) + - IOASID_SET_TYPE_MM (Set token is a mm_struct) + +The explicit MM token type is useful when multiple users of an IOASID +set under the same process need to communicate about their shared IOASIDs. +E.g. An IOASID set created by VFIO for one guest can be associated +with the KVM instance for the same guest since they share a common mm_struct. + +The IOASID set APIs serve the following purposes: + + - Ownership/permission enforcement + - Take collective actions, e.g. free an entire set + - Event notifications within a set + - Look up a set based on token + - Quota enforcement + +Individual IOASID APIs +---------------------- +Once an ioasid_set is created, IOASIDs can be allocated from the set. +Within the IOASID set namespace, set private ID (SPID) is supported. In +the VM use case, SPID can be used for storing guest PASID. + +:: + + ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_t min, ioasid_t max, + void *private); + + int ioasid_get(struct ioasid_set *set, ioasid_t ioasid); + + void ioasid_put(struct ioasid_set *set, ioasid_t ioasid); + + int ioasid_get_locked(struct ioasid_set *set, ioasid_t ioasid); + + void ioasid_put_locked(struct ioasid_set *set, ioasid_t ioasid); + + void *ioasid_find(struct ioasid_set *set, ioasid_t ioasid, + bool (*getter)(void *)); + + ioasid_t ioasid_find_by_spid(struct ioasid_set *set, ioasid_t spid) + + int ioasid_attach_data(struct ioasid_set *set, ioasid_t ioasid, + void *data); + int ioasid_attach_spid(struct ioasid_set *set, ioasid_t ioasid, + ioasid_t ssid); + + +Notifications +------------- +An IOASID may have multiple users, each user may have hardware context +associated with an IOASID. When the status of an IOASID changes, +e.g. an IOASID is being freed, users need to be notified such that the +associated hardware context can be cleared, flushed, and drained. + +:: + + int ioasid_register_notifier(struct ioasid_set *set, struct + notifier_block *nb) + + void ioasid_unregister_notifier(struct ioasid_set *set, + struct notifier_block *nb) + + int ioasid_register_notifier_mm(struct mm_struct *mm, struct + notifier_block *nb) + + void ioasid_unregister_notifier_mm(struct mm_struct *mm, struct + notifier_block *nb) + + int ioasid_notify(ioasid_t ioasid, enum ioasid_notify_val cmd, + unsigned int flags) + + +Events +~~~~~~ +Notification events are pertinent to individual IOASIDs, they can be +one of the following: + + - ALLOC + - FREE + - BIND + - UNBIND + +Ordering +~~~~~~~~ +Ordering is supported by IOASID notification priorities as the +following (in ascending order): + +:: + + enum ioasid_notifier_prios { + IOASID_PRIO_LAST, + IOASID_PRIO_IOMMU, + IOASID_PRIO_DEVICE, + IOASID_PRIO_CPU, + }; + +The typical use case is when an IOASID is freed due to an exception, DMA +source should be quiesced before tearing down other hardware contexts +in the system. This will reduce the churn in handling faults. DMA work +submission is performed by the CPU which is granted higher priority than +devices. + + +Scopes +~~~~~~ +There are two types of notifiers in IOASID core: system-wide and +ioasid_set-wide. + +System-wide notifier is catering for users that need to handle all +IOASIDs in the system. E.g. The IOMMU driver handles all IOASIDs. + +Per ioasid_set notifier can be used by VM specific components such as +KVM. After all, each KVM instance only cares about IOASIDs within its +own set. + + +Atomicity +~~~~~~~~~ +IOASID notifiers are atomic due to spinlocks used inside the IOASID +core. For tasks cannot be completed in the notifier handler, async work +can be submitted to complete the work later as long as there is no +ordering requirement. + +Reference counting +------------------ +IOASID lifecycle management is based on reference counting. Users of +IOASID intend to align lifecycle with the IOASID need to hold +reference of the IOASID. IOASID will not be returned to the pool for +allocation until all references are dropped. Calling ioasid_free() +will mark the IOASID as FREE_PENDING if the IOASID has outstanding +reference. ioasid_get() is not allowed once an IOASID is in the +FREE_PENDING state. + +Event notifications are used to inform users of IOASID status change. +IOASID_FREE event prompts users to drop their references after +clearing its context. + +For example, on VT-d platform when an IOASID is freed, teardown +actions are performed on KVM, device driver, and IOMMU driver. +KVM shall register notifier block with:: + + static struct notifier_block pasid_nb_kvm = { + .notifier_call = pasid_status_change_kvm, + .priority = IOASID_PRIO_CPU, + }; + +VDCM driver shall register notifier block with:: + + static struct notifier_block pasid_nb_vdcm = { + .notifier_call = pasid_status_change_vdcm, + .priority = IOASID_PRIO_DEVICE, + }; + +In both cases, notifier blocks shall be registered on the IOASID set +such that *only* events from the matching VM is received. + +If KVM attempts to register notifier block before the IOASID set is +created for the MM token, the notifier block will be placed on a +pending list inside IOASID core. Once the token matching IOASID set +is created, IOASID will register the notifier block automatically. +IOASID core does not replay events for the existing IOASIDs in the +set. For IOASID set of MM type, notification blocks can be registered +on empty sets only. This is to avoid lost events. + +IOMMU driver shall register notifier block on global chain:: + + static struct notifier_block pasid_nb_vtd = { + .notifier_call = pasid_status_change_vtd, + .priority = IOASID_PRIO_IOMMU, + }; + +Custom allocator APIs +--------------------- + +:: + + int ioasid_register_allocator(struct ioasid_allocator_ops *allocator); + + void ioasid_unregister_allocator(struct ioasid_allocator_ops *allocator); + +Allocator Choices +~~~~~~~~~~~~~~~~~ +IOASIDs are allocated for both host and guest SVA/IOVA usage. However, +allocators can be different. For example, on VT-d guest PASID +allocation must be performed via a virtual command interface which is +emulated by VMM. + +IOASID core has the notion of "custom allocator" such that guest can +register virtual command allocator that precedes the default one. + +Namespaces +~~~~~~~~~~ +IOASIDs are limited system resources that default to 20 bits in +size. Since each device has its own table, theoretically the namespace +can be per device also. However, for security reasons sharing PASID +tables among devices are not good for isolation. Therefore, IOASID +namespace is system-wide. + +There are also other reasons to have this simpler system-wide +namespace. Take VT-d as an example, VT-d supports shared workqueue +and ENQCMD[1] where one IOASID could be used to submit work on +multiple devices that are shared with other VMs. This requires IOASID +to be system-wide. This is also the reason why guests must use an +emulated virtual command interface to allocate IOASID from the host. + + +Life cycle +========== +This section covers IOASID lifecycle management for both bare-metal +and guest usages. In bare-metal SVA, MMU notifier is directly hooked +up with IOMMU driver, therefore the process address space (MM) +lifecycle is aligned with IOASID. + +However, guest MMU notifier is not available to host IOMMU driver, +when guest MM terminates unexpectedly, the events have to go through +VFIO and IOMMU UAPI to reach host IOMMU driver. There are also more +parties involved in guest SVA, e.g. on Intel VT-d platform, IOASIDs +are used by IOMMU driver, KVM, VDCM, and VFIO. + +Native IOASID Life Cycle (VT-d Example) +--------------------------------------- + +The normal flow of native SVA code with Intel Data Streaming +Accelerator(DSA) [2] as example: + +1. Host user opens accelerator FD, e.g. DSA driver, or uacce; +2. DSA driver allocate WQ, do sva_bind_device(); +3. IOMMU driver calls ioasid_alloc(), then bind PASID with device, + mmu_notifier_get() +4. DMA starts by DSA driver userspace +5. DSA userspace close FD +6. DSA/uacce kernel driver handles FD.close() +7. DSA driver stops DMA +8. DSA driver calls sva_unbind_device(); +9. IOMMU driver does unbind, clears PASID context in IOMMU, flush + TLBs. mmu_notifier_put() called. +10. mmu_notifier.release() called, IOMMU SVA code calls ioasid_free()* +11. The IOASID is returned to the pool, reclaimed. + +:: + + * With ENQCMD, PASID used on VT-d is not released in mmu_notifier() but + mmdrop(). mmdrop comes after FD close. Should not matter. + If the user process dies unexpectedly, Step #10 may come before + Step #5, in between, all DMA faults discarded. PRQ responded with + code INVALID REQUEST. + +During the normal teardown, the following three steps would happen in +order: + +1. Device driver stops DMA request +2. IOMMU driver unbinds PASID and mm, flush all TLBs, drain in-flight + requests. +3. IOASID freed + +Exception happens when process terminates *before* device driver stops +DMA and call IOMMU driver to unbind. The flow of process exists are as +follows: + +:: + + do_exit() { + exit_mm() { + mm_put(); + exit_mmap() { + intel_invalidate_range() //mmu notifier + tlb_finish_mmu() + mmu_notifier_release(mm) { + intel_iommu_release() { + [2] intel_iommu_teardown_pasid(); + intel_iommu_flush_tlbs(); + } + // tlb_invalidate_range cb removed + } + unmap_vmas(); + free_pgtables(); // IOMMU cannot walk PGT after this + }; + } + exit_files(tsk) { + close_files() { + dsa_close(); + [1] dsa_stop_dma(); + intel_svm_unbind_pasid(); //nothing to do + } + } + } + + mmdrop() /* some random time later, lazy mm user */ { + mm_free_pgd(); + destroy_context(mm); { + [3] ioasid_free(); + } + } + +As shown in the list above, step #2 could happen before +#1. Unrecoverable(UR) faults could happen between #2 and #1. + +Also notice that TLB invalidation occurs at mmu_notifier +invalidate_range callback as well as the release callback. The reason +is that release callback will delete IOMMU driver from the notifier +chain which may skip invalidate_range() calls during the exit path. + +To avoid unnecessary reporting of UR fault, IOMMU driver shall disable +fault reporting after free and before unbind. + +Guest IOASID Life Cycle (VT-d Example) +-------------------------------------- +Guest IOASID life cycle starts with guest driver open(), this could be +uacce or individual accelerator driver such as DSA. At FD open, +sva_bind_device() is called which triggers a series of actions. + +The example below is an illustration of *normal* operations that +involves *all* the SW components in VT-d. The flow can be simpler if +no ENQCMD is supported. + +:: + + VFIO IOMMU KVM VDCM IOASID Ref + .................................................................. + 1 ioasid_register_notifier/_mm() + 2 ioasid_alloc() 1 + 3 bind_gpasid() + 4 iommu_bind()->ioasid_get() 2 + 5 ioasid_notify(BIND) + 6 -> ioasid_get() 3 + 7 -> vmcs_update_atomic() + 8 mdev_write(gpasid) + 9 hpasid= + 10 find_by_spid(gpasid) 4 + 11 vdev_write(hpasid) + 12 -------- GUEST STARTS DMA -------------------------- + 13 -------- GUEST STOPS DMA -------------------------- + 14 mdev_clear(gpasid) + 15 vdev_clear(hpasid) + 16 ioasid_put() 3 + 17 unbind_gpasid() + 18 iommu_ubind() + 19 ioasid_notify(UNBIND) + 20 -> vmcs_update_atomic() + 21 -> ioasid_put() 2 + 22 ioasid_free() 1 + 23 ioasid_put() 0 + 24 Reclaimed + -------------- New Life Cycle Begin ---------------------------- + 1 ioasid_alloc() -> 1 + + Note: IOASID Notification Events: FREE, BIND, UNBIND + +Exception cases arise when a guest crashes or a malicious guest +attempts to cause disruption on the host system. The fault handling +rules are: + +1. IOASID free must *always* succeed. +2. An inactive period may be required before the freed IOASID is + reclaimed. During this period, consumers of IOASID perform cleanup. +3. Malfunction is limited to the guest owned resources for all + programming errors. + +The primary source of exception is when the following are out of +order: + +1. Start/Stop of DMA activity + (Guest device driver, mdev via VFIO) +2. Setup/Teardown of IOMMU PASID context, IOTLB, DevTLB flushes + (Host IOMMU driver bind/unbind) +3. Setup/Teardown of VMCS PASID translation table entries (KVM) in + case of ENQCMD +4. Programming/Clearing host PASID in VDCM (Host VDCM driver) +5. IOASID alloc/free (Host IOASID) + +VFIO is the *only* user-kernel interface, which is ultimately +responsible for exception handlings. + +#1 is processed the same way as the assigned device today based on +device file descriptors and events. There is no special handling. + +#3 is based on bind/unbind events emitted by #2. + +#4 is naturally aligned with IOASID life cycle in that an illegal +guest PASID programming would fail in obtaining reference of the +matching host IOASID. + +#5 is similar to #4. The fault will be reported to the user if PASID +used in the ENQCMD is not set up in VMCS PASID translation table. + +Therefore, the remaining out of order problem is between #2 and +#5. I.e. unbind vs. free. More specifically, free before unbind. + +IOASID notifier and refcounting are used to ensure order. Following +a publisher-subscriber pattern where: + +- Publishers: VFIO & IOMMU +- Subscribers: KVM, VDCM, IOMMU + +IOASID notifier is atomic which requires subscribers to do quick +handling of the event in the atomic context. Workqueue can be used for +any processing that requires thread context. IOASID reference must be +acquired before receiving the FREE event. The reference must be +dropped at the end of the processing in order to return the IOASID to +the pool. + +Let's examine the IOASID life cycle again when free happens *before* +unbind. This could be a result of misbehaving guests or crash. Assuming +VFIO cannot enforce unbind->free order. Notice that the setup part up +until step #12 is identical to the normal case, the flow below starts +with step 13. + +:: + + VFIO IOMMU KVM VDCM IOASID Ref + .................................................................. + 13 -------- GUEST STARTS DMA -------------------------- + 14 -------- *GUEST MISBEHAVES!!!* ---------------- + 15 ioasid_free() + 16 ioasid_notify(FREE) + 17 mark_ioasid_inactive[1] + 18 kvm_nb_handler(FREE) + 19 vmcs_update_atomic() + 20 ioasid_put_locked() -> 3 + 21 vdcm_nb_handler(FREE) + 22 iomm_nb_handler(FREE) + 23 ioasid_free() returns[2] schedule_work() 2 + 24 schedule_work() vdev_clear_wk(hpasid) + 25 teardown_pasid_wk() + 26 ioasid_put() -> 1 + 27 ioasid_put() 0 + 28 Reclaimed + 29 unbind_gpasid() + 30 iommu_unbind()->ioasid_find() Fails[3] + -------------- New Life Cycle Begin ---------------------------- + +Note: + +1. By marking IOASID inactive at step #17, no new references can be + held. ioasid_get/find() will return -ENOENT; +2. After step #23, all events can go out of order. Shall not affect + the outcome. +3. IOMMU driver fails to find private data for unbinding. If unbind is + called after the same IOASID is allocated for the same guest again, + this is a programming error. The damage is limited to the guest + itself since unbind performs permission checking based on the + IOASID set associated with the guest process. + +KVM PASID Translation Table Updates +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Per VM PASID translation table is maintained by KVM in order to +support ENQCMD in the guest. The table contains host-guest PASID +translations to be consumed by CPU ucode. The synchronization of the +PASID states depends on VFIO/IOMMU driver, where IOCTL and atomic +notifiers are used. KVM must register IOASID notifier per VM instance +during launch time. The following events are handled: + +1. BIND/UNBIND +2. FREE + +Rules: + +1. Multiple devices can bind with the same PASID, this can be different PCI + devices or mdevs within the same PCI device. However, only the + *first* BIND and *last* UNBIND emit notifications. +2. IOASID code is responsible for ensuring the correctness of H-G + PASID mapping. There is no need for KVM to validate the + notification data. +3. When UNBIND happens *after* FREE, KVM will see error in + ioasid_get() even when the reclaim is not done. IOMMU driver will + also avoid sending UNBIND if the PASID is already FREE. +4. When KVM terminates *before* FREE & UNBIND, references will be + dropped for all host PASIDs. + +VDCM PASID Programming +~~~~~~~~~~~~~~~~~~~~~~ +VDCM composes virtual devices and exposes them to the guests. When +the guest allocates a PASID then program it to the virtual device, VDCM +intercepts the programming attempt then program the matching host +PASID on to the hardware. +Conversely, when a device is going away, VDCM must be informed such +that PASID context on the hardware can be cleared. There could be +multiple mdevs assigned to different guests in the same VDCM. Since +the PASID table is shared at PCI device level, lazy clearing is not +secure. A malicious guest can attack by using newly freed PASIDs that +are allocated by another guest. + +By holding a reference of the PASID until VDCM cleans up the HW context, +it is guaranteed that PASID life cycles do not cross within the same +device. + + +Reference +==================================================== +1. https://software.intel.com/sites/default/files/managed/c5/15/architecture-instruction-set-extensions-programming-reference.pdf + +2. https://01.org/blogs/2019/introducing-intel-data-streaming-accelerator + +3. https://software.intel.com/en-us/download/intel-data-streaming-accelerator-preliminary-architecture-specification -- 2.7.4