* [Lustre-devel] client side reply handling @ 2009-12-09 15:05 Peter Braam 2009-12-15 16:28 ` Yuriy Umanets 0 siblings, 1 reply; 6+ messages in thread From: Peter Braam @ 2009-12-09 15:05 UTC (permalink / raw) To: lustre-devel Hi, I wonder if LNET doesn't have an atomic operation that unlinks the packet from the delivery process upon receiving a packet. Iirc it does. Why wouldn't one use that? Peter -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20091209/27c0c4d8/attachment.htm> ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Lustre-devel] client side reply handling 2009-12-09 15:05 [Lustre-devel] client side reply handling Peter Braam @ 2009-12-15 16:28 ` Yuriy Umanets 2009-12-16 2:01 ` Peter Braam 0 siblings, 1 reply; 6+ messages in thread From: Yuriy Umanets @ 2009-12-15 16:28 UTC (permalink / raw) To: lustre-devel 2009/12/9 Peter Braam <peter.braam@clusterstor.com> > Hi, > > hi Peter, > I wonder if LNET doesn't have an atomic operation that unlinks the packet > from the delivery process upon receiving a packet. Iirc it does. Why > wouldn't one use that? > That is probably because it still may be in use by upper layers (ptlrpc, etc), i.e. it is up to upper layer to decide should its buffers be still linked for access or not. Thanks. > > Peter > > _______________________________________________ > Lustre-devel mailing list > Lustre-devel at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-devel > > -- umka -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20091215/fd7d522c/attachment.htm> ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Lustre-devel] client side reply handling 2009-12-15 16:28 ` Yuriy Umanets @ 2009-12-16 2:01 ` Peter Braam 2009-12-16 2:48 ` Isaac Huang 2009-12-16 2:55 ` Liang Zhen 0 siblings, 2 replies; 6+ messages in thread From: Peter Braam @ 2009-12-16 2:01 UTC (permalink / raw) To: lustre-devel No - buffers can and probably should (because, for example, re-delivery may leaver buffer state undefined) be unlinked from the match list before passing the buffer up to any other layer. Portals/LNET certainly can do this. The question is why isn't this done, Eric, Isaac, Andreas? Peter On Tue, Dec 15, 2009 at 9:28 AM, Yuriy Umanets < yuriy.umanets@clusterstor.com> wrote: > > > 2009/12/9 Peter Braam <peter.braam@clusterstor.com> > >> Hi, >> >> hi Peter, > > >> I wonder if LNET doesn't have an atomic operation that unlinks the packet >> from the delivery process upon receiving a packet. Iirc it does. Why >> wouldn't one use that? >> > That is probably because it still may be in use by upper layers (ptlrpc, > etc), i.e. it is up to upper layer to decide should its buffers be still > linked for access or not. > > Thanks. > >> >> Peter >> >> _______________________________________________ >> Lustre-devel mailing list >> Lustre-devel at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-devel >> >> > > > -- > umka > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.lustre.org/pipermail/lustre-devel-lustre.org/attachments/20091215/21a6c9bb/attachment.htm> ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Lustre-devel] client side reply handling 2009-12-16 2:01 ` Peter Braam @ 2009-12-16 2:48 ` Isaac Huang 2009-12-16 2:51 ` Isaac Huang 2009-12-16 2:55 ` Liang Zhen 1 sibling, 1 reply; 6+ messages in thread From: Isaac Huang @ 2009-12-16 2:48 UTC (permalink / raw) To: lustre-devel On Tue, Dec 15, 2009 at 07:01:33PM -0700, Peter Braam wrote: > No - buffers can and probably should (because, for example, re-delivery > may leaver buffer state undefined) be unlinked from the match list > before passing the buffer up to any other layer. Portals/LNET > certainly can do this. Hi Peter, LNet does have a MD option to automatically unlink the buffer once it's been exhausted. Duplicated delivery shouldn't cause any problem because MD offset is increased locally (i.e. a dup would go into a different offset), unless the MD has enabled peers to manage offset remotely, which is now true for PTLRPC reply buffers (because now early replies and real reply share a same reply buffer). Isaac ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Lustre-devel] client side reply handling 2009-12-16 2:48 ` Isaac Huang @ 2009-12-16 2:51 ` Isaac Huang 0 siblings, 0 replies; 6+ messages in thread From: Isaac Huang @ 2009-12-16 2:51 UTC (permalink / raw) To: lustre-devel On Tue, Dec 15, 2009 at 09:48:18PM -0500, Isaac Huang wrote: > On Tue, Dec 15, 2009 at 07:01:33PM -0700, Peter Braam wrote: > > No - buffers can and probably should (because, for example, re-delivery > > may leaver buffer state undefined) be unlinked from the match list > > before passing the buffer up to any other layer. Portals/LNET > > certainly can do this. > > Hi Peter, LNet does have a MD option to automatically unlink the > buffer once it's been exhausted. Duplicated delivery shouldn't cause any > problem because MD offset is increased locally (i.e. a dup would go > into a different offset), unless the MD has enabled peers to manage > offset remotely, which is now true for PTLRPC reply buffers (because > now early replies and real reply share a same reply buffer). There was a recent discussion on this topic, under a similar subject: http://lists.lustre.org/pipermail/lustre-devel/2009-December/003231.html Isaac ^ permalink raw reply [flat|nested] 6+ messages in thread
* [Lustre-devel] client side reply handling 2009-12-16 2:01 ` Peter Braam 2009-12-16 2:48 ` Isaac Huang @ 2009-12-16 2:55 ` Liang Zhen 1 sibling, 0 replies; 6+ messages in thread From: Liang Zhen @ 2009-12-16 2:55 UTC (permalink / raw) To: lustre-devel Peter, It depends on how buffers are attached to portal, ptlrpc has three cases of this: 1. If buffer is for incoming request (not for RDMA, so it's not unique match) the MD is constructed with lnet_md_t::threshold = LNET_MD_THRESH_INF, so it will be unlinked only for two cases: a) user call LNetMDUnlink explicitly b) the buffer is exhausted (lnet_md_exhausted() return 1 in LNet) Once LNet get a request for that buffer, it will change offset of the buffer so it wouldn't be overwritten by the next match, it's possible that upper layer is handing the request, and the buffer is still on match list 2. If buffer is for bulk (it's unique) the MD is constructed with lnet_md_t::threshold = 1, and LNetMDAttach(...LNET_UNLINK...) so LNet will auto-unlink the buffer from match list once it got a match, I think this is the case you want 3. If the buffer is for reply (it's unique as well) the MD is constructed with lnet_md_t::threshold = LNET_MD_THRESH_INF, so it's same as 1), the buffer wouldn't be unlinked unless user call LNetMDUnlink or LNetMEUnlink, so safety of reply buffer can be guaranteed only if user is sure the buffer is unlinked. Regards Liang Peter Braam wrote: > No - buffers can and probably should (because, for example, > re-delivery may leaver buffer state undefined) be unlinked from the > match list before passing the buffer up to any other layer. > Portals/LNET certainly can do this. > > The question is why isn't this done, Eric, Isaac, Andreas? > > Peter > > On Tue, Dec 15, 2009 at 9:28 AM, Yuriy Umanets > <yuriy.umanets at clusterstor.com <mailto:yuriy.umanets@clusterstor.com>> > wrote: > > > > 2009/12/9 Peter Braam <peter.braam@clusterstor.com > <mailto:peter.braam@clusterstor.com>> > > Hi, > > hi Peter, > > > I wonder if LNET doesn't have an atomic operation that unlinks > the packet from the delivery process upon receiving a packet. > Iirc it does. Why wouldn't one use that? > > That is probably because it still may be in use by upper layers > (ptlrpc, etc), i.e. it is up to upper layer to decide should its > buffers be still linked for access or not. > > Thanks. > > > Peter > > _______________________________________________ > Lustre-devel mailing list > Lustre-devel at lists.lustre.org > <mailto:Lustre-devel@lists.lustre.org> > http://lists.lustre.org/mailman/listinfo/lustre-devel > > > > > -- > umka > > > ------------------------------------------------------------------------ > > _______________________________________________ > Lustre-devel mailing list > Lustre-devel at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-devel > ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2009-12-16 2:55 UTC | newest] Thread overview: 6+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2009-12-09 15:05 [Lustre-devel] client side reply handling Peter Braam 2009-12-15 16:28 ` Yuriy Umanets 2009-12-16 2:01 ` Peter Braam 2009-12-16 2:48 ` Isaac Huang 2009-12-16 2:51 ` Isaac Huang 2009-12-16 2:55 ` Liang Zhen
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.