Re: [RFC PATCH net-next v4 0/9] net/smc: Introduce SMC-D-based OS internal communication acceleration

[Wen Gu <guwen@linux.alibaba.com>]

On 2023/4/6 22:27, Niklas Schnelle wrote:

> On Thu, 2023-04-06 at 13:14 +0200, Alexandra Winter wrote:
>>
>> On 05.04.23 19:04, Niklas Schnelle wrote:
>>> One more question though, what about the SEID why does that have to be
>>> fixed and at least partially match what ISM devices use? I think I'm
>>> missing some SMC protocol/design detail here. I'm guessing this would
>>> require a protocol change?
>>>
>>> Thanks,
>>> Niklas
>>
>> Niklas,
>> in the initial SMC CLC handshake the client and server exchange the SEID (one per peer system)
>> and up to 8 proposals for SMC-D interfaces.
>> Wen's current proposal assumes that smc-d loopback can be one of these 8 proposed interfaces,
>> iiuc. So on s390 the proposal can contain ISM devices and a smc-d loopback device at the same time.
>> If one of the peers is e.g. an older Linux version, it will just ignore the loopback-device
>> in the list (Don't find a match for CHID 0xFFFF) and use an ISM interface for SMC-D if possible.
>> Therefor it is important that the SEID is used in the same way as it is today in the handshake.
>>
>> If we decide for some reason (virtio-ism open issues?) that a protocol change/extension is
>> required/wanted, then it is a new game and we can come up with new identifiers, but we may
>> lose compatibility to backlevel systems.
>>
>> Alexandra
> 
> Ok that makes sense to me. I was looking at the code in patch 4 of this
> series and there it looks to me like SMC-D loopback as implemented
> would always use the newly added SMCD_DEFAULT_V2_SEID might have
> misread it though. From your description I think that would be wrong,
> if a SEID is defined as on s390 it should use that SEID in the CLC for
> all SMC variants. Similarly on other architectures it should use the
> same SEID for SMC-D as for SMC-R, right? Also with partially match I
> was actually wrong the SMCD_DEFAULT_V2_SEID.seid_string starts with
> "IBM-DEF-ISMSEID…" while on s390's existing ISM we use "IBM-SYSZ-
> ISMSEID…" so if SMC-D loopback correctly uses the shared SEID on s390
> we can already only get GID.DMB collisions only on the same mainframe.
> 
> Thanks,
> Niklas

SMC stack uses a global variable smc_ism_v2_system_eid to indicate
the only one SEID of system. Because all ISMv2 devices return the same
SEID, SEID of the first registered ISMv2 device will be assigned to
smc_ism_v2_system_eid.

Now we have extension SMC-D devices, loopback or virtio-ism device,
and this may need a little change.

My original idea was that

- Extension SMC-D devices always return SMCD_DEFAULT_V2_SEID as SEID.
- If there is only extension device in the system, smc_ism_v2_system_eid
   will record SMCD_DEFAULT_V2_SEID returned by SMC-D extension device.
- If extension devices coexist with ISM devices on s390, smc_ism_v2_system_eid
   will record SEID of ISM devices.

But inspired by your comments, I find the original idea has some problems
in situation that one side has only extension devices but the other side
has both extension and ISM devices. Although they can communicate through
the extension devices(virtio-ism), SMC-D connection is unavailable due to
the different SEIDs.

So as you suggested, the extension devices (loopback or virtio-ism) should
use the same way as ISM device to get the shared SEID on s390 arch.

And on arch other than s390, extension SMC-D device can use a fixed SEID like
SMCD_DEFAULT_V2_SEID here if we do not require SMC-D communication between
different architectures.

Thanks,
Wen Gu