From mboxrd@z Thu Jan 1 00:00:00 1970 From: Robin Murphy Subject: Re: iommu/arm-smmu-v2 ASID/VMID calculation Date: Tue, 26 Jan 2016 11:48:26 +0000 Message-ID: <56A75D0A.7000806@arm.com> References: <198F501C-8D30-4EB5-BC40-4F40BB75D40B@caviumnetworks.com> <20160125170312.GJ22927@arm.com> <6F24A28A-6302-4C48-A933-B47A9735808C@caviumnetworks.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <6F24A28A-6302-4C48-A933-B47A9735808C-M3mlKVOIwJVv6pq1l3V1OdBPR1lH4CV8@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: "Chalamarla, Tirumalesh" , Will Deacon Cc: "iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org" , linux-arm-kernel List-Id: iommu@lists.linux-foundation.org T24gMjYvMDEvMTYgMDM6MTEsIENoYWxhbWFybGEsIFRpcnVtYWxlc2ggd3JvdGU6Cj4gb25lIG15 IGNvbGxlYWd1ZSBwb2ludHMgb3V0LCBBU0lEUE5FIGFsc28gaW1wbGVtZW50YXRpb24gZGVmaW5l ZC4KCkFTSURQTkUgb25seSBjb3ZlcnMgd2hldGhlciB0aGUgU01NVSdzIFRMQnMgbWF5IGlnbm9y ZSBicm9hZGNhc3QgCl9pbnZhbGlkYXRpb25fIG9yIG5vdCwgbm90IGxvb2t1cHMuIEJyb2FkY2Fz dCBUTEIgbG9va3VwcyBhcmUgbm90IGEgdGhpbmcuCgo+IFRoYW5rcywKPiBUaXJ1bWFsZXNoLgo+ Cj4KPgo+IE9uIDEvMjUvMTYsIDQ6NDggUE0sICJsaW51eC1hcm0ta2VybmVsIG9uIGJlaGFsZiBv ZiBDaGFsYW1hcmxhLCBUaXJ1bWFsZXNoIiA8bGludXgtYXJtLWtlcm5lbC1ib3VuY2VzQGxpc3Rz LmluZnJhZGVhZC5vcmcgb24gYmVoYWxmIG9mIFRpcnVtYWxlc2guQ2hhbGFtYXJsYUBjYXZpdW1u ZXR3b3Jrcy5jb20+IHdyb3RlOgo+Cj4+IEkgZG9u4oCZdCB0aGluayBpdCBmb2xsb3dlZCBzcGVj Lgo+Pgo+PiBUaGUgU01NVSBzcGVjIChJSEkwMDYyQykgc2F5cyAoc2VjdGlvbiAyLjIpOgo+Pgo+ PiAgICAtIFRoZSBleGFjdCBiZWhhdmlvciBvZiBhbnkgVExCIGZ1bmN0aW9uYWxpdHkgaXMgSU1Q TEVNRU5UQVRJT04gREVGSU5FRAo+Pgo+PiBTZWN0aW9uIDIuMiBkZXNjcmliZXMgdGhlIGNvbnN0 cmFpbnRzIG9uIHNoYXJpbmcgKGUuZy4gbXVsdGlwbGUgY29udGV4dHMKPj4gdXNpbmcgdGhlIHNh bWUgQVNJRCBzYXlzKToKPj4KPj4gICAgICAgIklmIG11bHRpcGxlIGNvbnRleHQgYmFua3MgaGF2 ZSBodGUgc2FtZSBhdHRyaWJ1dGVzIGJ1dCBkZXNjcmliZQo+PiAgICAgICAgZGlmZmVyZW50IHRy YW5zbGF0aW9ucywgdGhlIHJlc3VsdHMgb2YgYSBUTEIgbG9va3VwIGFyZSBVTlBSRURJQ1RBQkxF Ii4KCk11bHRpcGxlIGNvbnRleHQgYmFua3Mgd2l0aGluIHRoZSBzYW1lIFNNTVUsIHRoYXQgaXMu Cgo+PiBUaGUgcHJlc2VudCBzbW11IGRyaXZlciBhc3N1bWVzIEFTSUQgc2hvdWxkIG9ubHkgYmUg dW5pcXVlIHBlciBTTU1VLCB0aGlzIG1pZ2h0IG5vdCBiZSB0cnVlIGZvciBhbGwKPj4gSW1wbGVt ZW50YXRpb25zLgoKV2hlcmUgIlNNTVUiIGltcGxpZXMgInNldCBvZiBjb250ZXh0IGJhbmtzIGJl aGluZCBhIHNpbmdsZSBwcm9ncmFtbWluZyAKaW50ZXJmYWNlIiwgaS5lLiBpdCBzaG91bGQgYmUg dHJ1ZSBmb3IgYWxsIGFyY2hpdGVjdHVyYWxseS1jb21wbGlhbnQgCmltcGxlbWVudGF0aW9ucy4K CldoYXQgaWYgeW91IHRha2UgeW91ciBzeXN0ZW0sIGZpcmUgdXAgdHdvIGNvcmVzIHdpdGggU01Q IGRpc2FibGVkLCBhbmQgCnJ1biBhbiBpbmRlcGVuZGVudCBpbnN0YW5jZSBvZiBMaW51eCBvbiBl YWNoLCB3aXRoIGFsbCB0aGUgc3lzdGVtIApyZXNvdXJjZXMgZGl2aWRlZCB1cCBlcXVhbGx5IGJl dHdlZW4gdGhlbSBhbmQgbm8gY29tbXVuaWNhdGlvbj8gSWYgb25lIApPUyB1c2luZyBvbmUgU01N VSBjYW4gbWFrZSB0aGUgb3RoZXIgT1MgdXNpbmcgYSBkaWZmZXJlbnQgU01NVSBnbyB3cm9uZywg CnRoZW4gdGhlIGhhcmR3YXJlIGlzIHNpbXBseSBicm9rZW4uIFN1cmUsIHdlIGNhbiBwcm9iYWJs eSBxdWlyayBBU0lEIAphbGxvY2F0aW9uIGluIHRoZSBkcml2ZXIsIGJ1dCBpdCByYWlzZXMgdGhl IHF1ZXN0aW9uIG9mIHdoYXQgZWxzZSB3ZSAKd291bGQgbmVlZCB0byBkbyB0byBrZWVwIFNNTVVz IHdoaWNoIGJlbGlldmUgdGhleSdyZSBzdXBwb3NlZCB0byBiZSAKaW5kZXBlbmRlbnQgZnJvbSBE b1MnaW5nIGVhY2ggb3RoZXIuLgoKUm9iaW4uCgo+PiBPbiAxLzI1LzE2LCA5OjAzIEFNLCAiV2ls bCBEZWFjb24iIDx3aWxsLmRlYWNvbkBhcm0uY29tPiB3cm90ZToKPj4KPj4+IE9uIFRodSwgSmFu IDIxLCAyMDE2IGF0IDA2OjUyOjM0UE0gKzAwMDAsIENoYWxhbWFybGEsIFRpcnVtYWxlc2ggd3Jv dGU6Cj4+Pj4gSGkgV2lsbCwKPj4+Cj4+PiBIZWxsbywKPj4+Cj4+Pj4gQ3VycmVudCBBU0lEL1ZN SUQgY2FsY3VsYXRpb24gbG9naWMgbWFrZXMgbG90IG9mIGFzc3VtcHRpb24gYWJvdXQgaW50ZXJu YWwgVExCCj4+Pj4gaW1wbGVtZW50YXRpb24gb2YgU01NVSwKPj4+Cj4+PiBOb3QgcmVhbGx5LiBJ dCBtYWtlcyBhc3N1bXB0aW9ucyB0aGF0IHRoZSBoYXJkd2FyZSBmb2xsb3dzIHRoZSBhcmNoaXRl Y3R1cmUsCj4+PiB3aGljaCBpcyBoYXJkbHkgdW5yZWFzb25hYmxlIGFzIGEgc3RhcnRpbmcgcG9p bnQuCj4+Pgo+Pj4+IFN5c3RlbXMgbGlrZSBUaHVuZGVyWCBoYXZlIG1vcmUgdGhhbiBvbmUgc21t dSBpbiB0aGUgc3lzdGVtIGFuZCBpdCBjYW4gdXNlIHNhbWUKPj4+PiBUTEJzIHdpdGggbW9yZSB0 aGFuIG9uZSBvZiB0aGVtIGFuZCBleHBlY3RzIEFTSUQgdG8gYmUgdW5pcXVlCj4+Pgo+Pj4gLi4u IGJ1dCB0aGF0J3MgYnJva2VuLiBJZiB5b3UgYnVpbHQgYSBzeXN0ZW0gd2hlcmUgdGhlIENQVXMg c2hhcmVkIGEgVExCLAo+Pj4geW91IHdvdWxkIHJ1biBpbnRvIGlzc3VlcyBhcyB3ZWxsLgo+Pj4K Pj4+IEhvdyBkb2VzIHRoaXMgd29yayB3aXRoIHRoaW5ncyBsaWtlIGFybV9zbW11X3RsYl9zeW5j IGFuZCB0aGUgVExCR1NUQVRVUwo+Pj4gcmVnaXN0ZXI/Cj4+Pgo+Pj4+IEN1cnJlbnQgbG9naWMK Pj4+Pgo+Pj4+ICNkZWZpbmUgQVJNX1NNTVVfQ0JfQVNJRChjZmcpICAgICAgICAgICAoKGNmZykt PmNibmR4KQo+Pj4+ICNkZWZpbmUgQVJNX1NNTVVfQ0JfVk1JRChjZmcpICAgICAgICAgICAoKGNm ZyktPmNibmR4ICsgMSkKPj4+Pgo+Pj4+Cj4+Pj4gQ2FuIHRoaXMgYmUgcmVwbGFjZWQgYnkgc29t ZXRoaW5nIGxpa2UKPj4+Pgo+Pj4+Cj4+Pj4gI2RlZmluZSBBUk1fU01NVV9DQl9BU0lEKGNmZywg c21tdSkgICAgICAgICAgICAgKCgoc21tdSktPmlkeCA8PCAoc21tdSktPmFzaWRfc2hpZnQpIHwg KGNmZyktPmNibmR4KQo+Pj4+ICNkZWZpbmUgQVJNX1NNTVVfQ0JfVk1JRChjZmcsIHNtbXUpICAg ICAgICAgICAgICgoKHNtbXUpLT5pZHggPDwgKHNtbXUpLT52bWlkX3NoaWZ0KSB8IChjZmcpLT5j Ym5keCArIDEpCj4+Pj4KPj4+Pgo+Pj4+IElkeCBhbmQgc2hpZnQgY2FuIGJlIHBhc3NlZCBmcm9t IGRldmljZS10cmVlLgo+Pj4+Cj4+Pj4KPj4+PiBQbGVhc2UgbGV0IG1lIGtub3cgaWYgdGhpcyBp cyBhY2NlcHRhYmxlLCBJIHdpbGwgcHJlcGFyZSBhIHByb3BlciBwYXRjaAo+Pj4+IGFuZCBzZW5k IHRvIGxpc3QuCj4+Pj4KPj4+Pgo+Pj4+IElmIHRoaXMgaXMgbm90IGFjY2VwdGFibGUgdGhyb3Vn aCBhbiBhbHRlcm5hdGl2ZSBzdWdnZXN0aW9uLgo+Pj4KPj4+IElmIHdlJ3JlIGdvaW5nIHRvIHB1 dCBzb21ldGhpbmcgaW50byB0aGUgZGV2aWNlLXRyZWUsIHRoZW4gaXQgc2hvdWxkIGJlCj4+PiBh biBlcnJhdHVtIHByb3BlcnR5IGRlc2NyaWJpbmcgdGhlIG9mZnNldCB0byBiZSBhcHBsaWVkIHRv IEFTSUQvVk1JRHMuCj4+PiBZb3UgYWxzbyBuZWVkIHRvIHRha2UgY2FyZSBub3QgdG8gZGVzY3Jp YmUgb3ZlcmxhcHBpbmcgbnVtYmVyc3BhY2VzLgo+PiBJIGRvbuKAmXQgdGhpbmsgdGhhdCBpcyBh biBlcnJhdGEuCj4+Pgo+Pj4gSG93IG1hbnkgY29udGV4dCBiYW5rcyBkbyB5b3UgaW1wbGVtZW50 IGluIGVhY2ggU01NVT8KPj4gMTI4Lgo+Pj4KPj4+IFdpbGwKPj4gX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX18KPj4gbGludXgtYXJtLWtlcm5lbCBtYWlsaW5n IGxpc3QKPj4gbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnCj4+IGh0dHA6Ly9s aXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtYXJtLWtlcm5lbAo+IF9f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCj4gaW9tbXUgbWFp bGluZyBsaXN0Cj4gaW9tbXVAbGlzdHMubGludXgtZm91bmRhdGlvbi5vcmcKPiBodHRwczovL2xp c3RzLmxpbnV4Zm91bmRhdGlvbi5vcmcvbWFpbG1hbi9saXN0aW5mby9pb21tdQo+CgpfX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwppb21tdSBtYWlsaW5nIGxp c3QKaW9tbXVAbGlzdHMubGludXgtZm91bmRhdGlvbi5vcmcKaHR0cHM6Ly9saXN0cy5saW51eGZv dW5kYXRpb24ub3JnL21haWxtYW4vbGlzdGluZm8vaW9tbXU= From mboxrd@z Thu Jan 1 00:00:00 1970 From: robin.murphy@arm.com (Robin Murphy) Date: Tue, 26 Jan 2016 11:48:26 +0000 Subject: iommu/arm-smmu-v2 ASID/VMID calculation In-Reply-To: <6F24A28A-6302-4C48-A933-B47A9735808C@caviumnetworks.com> References: <198F501C-8D30-4EB5-BC40-4F40BB75D40B@caviumnetworks.com> <20160125170312.GJ22927@arm.com> <6F24A28A-6302-4C48-A933-B47A9735808C@caviumnetworks.com> Message-ID: <56A75D0A.7000806@arm.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On 26/01/16 03:11, Chalamarla, Tirumalesh wrote: > one my colleague points out, ASIDPNE also implementation defined. ASIDPNE only covers whether the SMMU's TLBs may ignore broadcast _invalidation_ or not, not lookups. Broadcast TLB lookups are not a thing. > Thanks, > Tirumalesh. > > > > On 1/25/16, 4:48 PM, "linux-arm-kernel on behalf of Chalamarla, Tirumalesh" wrote: > >> I don?t think it followed spec. >> >> The SMMU spec (IHI0062C) says (section 2.2): >> >> - The exact behavior of any TLB functionality is IMPLEMENTATION DEFINED >> >> Section 2.2 describes the constraints on sharing (e.g. multiple contexts >> using the same ASID says): >> >> "If multiple context banks have hte same attributes but describe >> different translations, the results of a TLB lookup are UNPREDICTABLE". Multiple context banks within the same SMMU, that is. >> The present smmu driver assumes ASID should only be unique per SMMU, this might not be true for all >> Implementations. Where "SMMU" implies "set of context banks behind a single programming interface", i.e. it should be true for all architecturally-compliant implementations. What if you take your system, fire up two cores with SMP disabled, and run an independent instance of Linux on each, with all the system resources divided up equally between them and no communication? If one OS using one SMMU can make the other OS using a different SMMU go wrong, then the hardware is simply broken. Sure, we can probably quirk ASID allocation in the driver, but it raises the question of what else we would need to do to keep SMMUs which believe they're supposed to be independent from DoS'ing each other.. Robin. >> On 1/25/16, 9:03 AM, "Will Deacon" wrote: >> >>> On Thu, Jan 21, 2016 at 06:52:34PM +0000, Chalamarla, Tirumalesh wrote: >>>> Hi Will, >>> >>> Hello, >>> >>>> Current ASID/VMID calculation logic makes lot of assumption about internal TLB >>>> implementation of SMMU, >>> >>> Not really. It makes assumptions that the hardware follows the architecture, >>> which is hardly unreasonable as a starting point. >>> >>>> Systems like ThunderX have more than one smmu in the system and it can use same >>>> TLBs with more than one of them and expects ASID to be unique >>> >>> ... but that's broken. If you built a system where the CPUs shared a TLB, >>> you would run into issues as well. >>> >>> How does this work with things like arm_smmu_tlb_sync and the TLBGSTATUS >>> register? >>> >>>> Current logic >>>> >>>> #define ARM_SMMU_CB_ASID(cfg) ((cfg)->cbndx) >>>> #define ARM_SMMU_CB_VMID(cfg) ((cfg)->cbndx + 1) >>>> >>>> >>>> Can this be replaced by something like >>>> >>>> >>>> #define ARM_SMMU_CB_ASID(cfg, smmu) (((smmu)->idx << (smmu)->asid_shift) | (cfg)->cbndx) >>>> #define ARM_SMMU_CB_VMID(cfg, smmu) (((smmu)->idx << (smmu)->vmid_shift) | (cfg)->cbndx + 1) >>>> >>>> >>>> Idx and shift can be passed from device-tree. >>>> >>>> >>>> Please let me know if this is acceptable, I will prepare a proper patch >>>> and send to list. >>>> >>>> >>>> If this is not acceptable through an alternative suggestion. >>> >>> If we're going to put something into the device-tree, then it should be >>> an erratum property describing the offset to be applied to ASID/VMIDs. >>> You also need to take care not to describe overlapping numberspaces. >> I don?t think that is an errata. >>> >>> How many context banks do you implement in each SMMU? >> 128. >>> >>> Will >> _______________________________________________ >> linux-arm-kernel mailing list >> linux-arm-kernel at lists.infradead.org >> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel > _______________________________________________ > iommu mailing list > iommu at lists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/iommu >