From: Tony Lu <tonylu@linux.alibaba.com>
To: Jan Karcher <jaka@linux.ibm.com>
Cc: "D. Wythe" <alibuda@linux.alibaba.com>,
kuba@kernel.org, davem@davemloft.net, netdev@vger.kernel.org,
linux-s390@vger.kernel.org, linux-rdma@vger.kernel.org
Subject: Re: [PATCH net-next v3 00/10] optimize the parallelism of SMC-R connections
Date: Fri, 28 Oct 2022 13:29:25 +0800 [thread overview]
Message-ID: <Y1totRGH3t9u3QlQ@TonyMac-Alibaba> (raw)
In-Reply-To: <35d14144-28f7-6129-d6d3-ba16dae7a646@linux.ibm.com>
On Wed, Oct 26, 2022 at 03:12:48PM +0200, Jan Karcher wrote:
>
>
> On 25/10/2022 08:13, Tony Lu wrote:
> > On Mon, Oct 24, 2022 at 03:10:54PM +0200, Jan Karcher wrote:
> > > Hi D. Wythe,
> > >
> > > I reply with the feedback on your fix to your v4 fix.
> > >
> > > Regarding your questions:
> > > We are aware of this situation and we are currently evaluating how we want
> > > to deal with SMC-D in the future because as of right now i can understand
> > > your frustration regarding the SMC-D testing.
> > > Please give me some time to hit up the right people and collect some
> > > information to answer your question. I'll let you know as soon as i have an
> > > answer.
>
> Hi Tony (and D.),
> >
> > Hi Jan,
> >
> > We sent a RFC [1] to mock SMC-D device for inter-VM communication. The
> > original purpose is not to test, but for now it could be useful for the
> > people who are going to test without physical devices in the community.
>
> I'm aware of the RFC and various people in IBM looked over it.
>
> As stated in the last mail we are aware that the entanglement between SMC-D
> and ISM is causing problems for the community.
> To give you a little insight:
>
> In order to improve the code quality and usability for the broader community
> we are working on placing an API between SMC-D and the ISM device. If this
> API is complete it will be easier to use different "devices" for SMC-D. One
> could be your device driver for inter-VM communication (ivshmem).
> Another one could be a "Dummy-Device" which just implements the required
> interface which acts as a loopback device. This would work only in a single
> Linux instance, thus would be the perfect device to test SMC-D logic for the
> broad community.
That sounds great :-) It will provide many possibilities.
> We would hope that these changes remove the hardware restrictions and that
> the community picks up the idea and implements devices and improves SMC
> (including SMC-D and SMC-R) even more in the future!
>
> As i said - and also teased by Alexandra in a respond to your RFC - this API
> feature is currently being developed and in our internal reviews. This would
> make your idea with the inter-VM communication a lot easier and would
> provide a clean base to build upon in the future.
Great +1.
>
> >
> > This driver basically works but I would improve it for testing. Before
> > that, what do you think about it?
>
> I think it is a great idea and we should definetly give it a shot! I'm also
> putting a lot in code quality and future maintainability. The API is a key
> feature there improving the usability for the community and our work as
> maintainers. So - for the sake of the future of the SMC code base - I'd like
> to wait with putting your changes upstream for the API and use your idea to
> see if fits our (and your) requirements.
Sure. We are very much looking forward to the new API :-) Maybe we can
discuss this API in the mail list, and I'd like to adapt it first.
>
> >
> > And where to put this driver? In kernel with SMC code or merge into
> > separate SMC test cases. I haven't made up my mind yet.
>
> We are not sure either currently, and have to think about that for a bit. I
> think your driver could be a classic driver, since it is usable for a real
> world problem (communication between two VMs on the same host). If we look
> at the "Dummy-Device" above we see that it does not provide any value beside
> testing. Feel free to share your ideas on that topic.
I agree with this. SMC would provides a common ability that drivers can
be introduced as classic driver for every individual device, maybe likes
ethernet cards with their drivers.
And for dummy devices, I have an idea that provides the same
shared-memory ability for same host, just like loopback devices for
TCP/IP. SMC-D with loopback devices also shows better performance compared
with other protocols.
Maybe SMC can cover all the scenes including inter-host (SMC-R),
inter-VM (SMC-D) and inter-local-process (SMC-D) communication with the
fastest path, and make things easier for user-space. I'd like to share
this RFC later when it's ready.
>
> >
> > [1] https://lore.kernel.org/netdev/20220720170048.20806-1-tonylu@linux.alibaba.com/
> >
> > Cheers,
> > Tony Lu
>
> A friendly disclaimer: Even tho this API feature is pretty far in the
> development process it can always be that we decide to drop it, if it does
> not meet our quality expectations. But of course we'll keep you updated.
>
No problem. If there has something that we can involve, we'd be pleasure
to do that.
Cheers,
Tony Lu
> - Jan
prev parent reply other threads:[~2022-10-28 5:29 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-10-20 6:43 [PATCH net-next v3 00/10] optimize the parallelism of SMC-R connections D.Wythe
2022-10-20 6:43 ` [PATCH net-next v3 01/10] net/smc: remove locks smc_client_lgr_pending and smc_server_lgr_pending D.Wythe
2022-10-20 6:43 ` [PATCH net-next v3 02/10] net/smc: fix SMC_CLC_DECL_ERR_REGRMB without smc_server_lgr_pending D.Wythe
2022-10-20 6:43 ` [PATCH net-next v3 03/10] net/smc: allow confirm/delete rkey response deliver multiplex D.Wythe
2022-10-20 6:43 ` [PATCH net-next v3 04/10] net/smc: make SMC_LLC_FLOW_RKEY run concurrently D.Wythe
2022-10-20 6:43 ` [PATCH net-next v3 05/10] net/smc: llc_conf_mutex refactor, replace it with rw_semaphore D.Wythe
2022-10-20 6:43 ` [PATCH net-next v3 06/10] net/smc: use read semaphores to reduce unnecessary blocking in smc_buf_create() & smcr_buf_unuse() D.Wythe
2022-10-20 6:43 ` [PATCH net-next v3 07/10] net/smc: reduce unnecessary blocking in smcr_lgr_reg_rmbs() D.Wythe
2022-10-20 6:43 ` [PATCH net-next v3 08/10] net/smc: replace mutex rmbs_lock and sndbufs_lock with rw_semaphore D.Wythe
2022-10-20 6:43 ` [PATCH net-next v3 09/10] net/smc: Fix potential panic dues to unprotected smc_llc_srv_add_link() D.Wythe
2022-10-20 6:43 ` [PATCH net-next v3 10/10] net/smc: fix application data exception D.Wythe
2022-10-20 7:00 ` [PATCH net-next v3 00/10] optimize the parallelism of SMC-R connections D. Wythe
2022-10-20 7:24 ` Jan Karcher
2022-10-21 11:57 ` Jan Karcher
2022-10-21 15:57 ` D. Wythe
2022-10-24 13:10 ` Jan Karcher
2022-10-25 6:13 ` Tony Lu
2022-10-26 13:12 ` Jan Karcher
2022-10-28 5:29 ` Tony Lu [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Y1totRGH3t9u3QlQ@TonyMac-Alibaba \
--to=tonylu@linux.alibaba.com \
--cc=alibuda@linux.alibaba.com \
--cc=davem@davemloft.net \
--cc=jaka@linux.ibm.com \
--cc=kuba@kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox