From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jason Gunthorpe Subject: Re: [PATCH 2/2] IB/mad: Use IDR for agent IDs Date: Tue, 12 Jun 2018 08:33:59 -0600 Message-ID: <20180612143359.GE6328@mellanox.com> References: <20180608174218.32455-3-willy@infradead.org> <20180610063028.GH12407@mtr-leonro.mtl.com> <20180610104305.GA9284@bombadil.infradead.org> <20180610122505.GM12407@mtr-leonro.mtl.com> <20180610203027.GF5560@mellanox.com> <20180611043425.GA21382@mtr-leonro.mtl.com> <20180611044203.GA32562@mellanox.com> <20180611091914.00007858@dev.mellanox.co.il> <20180611161918.GF5815@mellanox.com> <20180612075942.00005061@dev.mellanox.co.il> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: <20180612075942.00005061@dev.mellanox.co.il> Sender: linux-kernel-owner@vger.kernel.org To: jackm Cc: Leon Romanovsky , Matthew Wilcox , hans.westgaard.ry@oracle.com, Doug Ledford , Matthew Wilcox , linux-rdma@vger.kernel.org, =?utf-8?B?SMOla29u?= Bugge , Parav Pandit , Pravin Shedge , linux-kernel@vger.kernel.org List-Id: linux-rdma@vger.kernel.org On Tue, Jun 12, 2018 at 07:59:42AM +0300, jackm wrote: > On Mon, 11 Jun 2018 10:19:18 -0600 > Jason Gunthorpe wrote: > > > On Mon, Jun 11, 2018 at 09:19:14AM +0300, jackm wrote: > > > On Sun, 10 Jun 2018 22:42:03 -0600 > > > Jason Gunthorpe wrote: > > > > > > > Er, the spec has nothing to do with this. In Linux the TID is made > > > > unique because the core code provides 32 bits that are unique and > > > > the user provides another 32 bits that are unique. The driver > > > > cannot change any of those bits without risking non-uniquenes, > > > > which is exactly the bug mlx4 created when it stepped outside its > > > > bounds and improperly overrode bits in the TID for its own > > > > internal use. > > > > > > Actually, the opposite is true here. When SRIOV is active, each VM > > > generates its *own* TIDs -- with 32 bits of agent number and 32 bits > > > of counter. > > > > And it does it while re-using the LRH of the host, so all VMs and the > > host are now forced to share a TID space, yes I know. > > > > > There is a chance that two different VMs can generate the same TID! > > > Encoding the slave (VM) number in the packet actually guarantees > > > uniqueness here. > > > > Virtualizing the TID in the driver would be fine, but it must > > virtualize all the TIDs (even those generated by the HOST). > > It DOES do so. The host slave id is 0. Slave numbers start with 1. > If the MS byte contains a zero after paravirtualization, the MAD > was sent by the host. > In fact, ALL mads are paravirtualized -- including those to/from the host. Just assuming the byte is 0 and replacing it with something else is *NOT* virtualization. Jason