[LSF/MM/BPF BoF] Session for CXL memory

[LSF/MM/BPF BoF] Session for CXL memory

[Viacheslav A.Dubeyko]

Hello,

I believe CXL memory is hot topic now. I believe we have multiple topics
for discussion. I personally would like to discuss CXL Fabric Manager
and vision of FM architecture implementation. I am going to share the topic
in separate email. I would like to suggest a special session for CXL memory
related topics.

How everybody feels about it?

Thanks,
Slava.
 

Re: [LSF/MM/BPF BoF] Session for CXL memory

[Viacheslav A.Dubeyko]

CC: LSF/MM/BPF mailing list. Sorry, missed the list.

> On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <viacheslav.dubeyko@bytedance.com> wrote:
> 
> Hello,
> 
> I believe CXL memory is hot topic now. I believe we have multiple topics
> for discussion. I personally would like to discuss CXL Fabric Manager
> and vision of FM architecture implementation. I am going to share the topic
> in separate email. I would like to suggest a special session for CXL memory
> related topics.
> 
> How everybody feels about it?
> 
> Thanks,
> Slava.
> 

Re: [LSF/MM/BPF BoF] Session for CXL memory

[David Rientjes]

On Fri, 6 Jan 2023, Viacheslav A.Dubeyko wrote:

> CC: LSF/MM/BPF mailing list. Sorry, missed the list.
> 
> > On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <viacheslav.dubeyko@bytedance.com> wrote:
> > 
> > Hello,
> > 
> > I believe CXL memory is hot topic now. I believe we have multiple topics
> > for discussion. I personally would like to discuss CXL Fabric Manager
> > and vision of FM architecture implementation. I am going to share the topic
> > in separate email. I would like to suggest a special session for CXL memory
> > related topics.
> > 
> > How everybody feels about it?
> > 

I think this makes a lot of sense, thanks for suggesting it.

Should this be a BoF or just a normal topic proposal?  I assume that there 
could be several different topics of interest all related to CXL.mem.

Specifically interesting would be the division of work between the kernel 
and userspace to manage memory placement on systems with locally attached 
CXL.  And, further, what APIs userspace would have at its disposal for 
explicit optimization of this placement that would exist beyond what is 
available for NUMA.

I assume we might also want to chat about CXL 2.0 extensions that would be 
useful, especially for cloud providers.

I'd be happy to join in any of the proposals for these topics, just let me 
know!

Re: [LSF/MM/BPF BoF] Session for CXL memory

[Davidlohr Bueso]

On Sun, 22 Jan 2023, David Rientjes wrote:

>On Fri, 6 Jan 2023, Viacheslav A.Dubeyko wrote:
>
>> CC: LSF/MM/BPF mailing list. Sorry, missed the list.
>>
>> > On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <viacheslav.dubeyko@bytedance.com> wrote:
>> >
>> > Hello,
>> >
>> > I believe CXL memory is hot topic now. I believe we have multiple topics
>> > for discussion. I personally would like to discuss CXL Fabric Manager
>> > and vision of FM architecture implementation. I am going to share the topic
>> > in separate email. I would like to suggest a special session for CXL memory
>> > related topics.
>> >
>> > How everybody feels about it?
>> >
>
>I think this makes a lot of sense, thanks for suggesting it.

Yeah, considering all the CXL interest in last year's lsfmm, I think it would
be good to have a session.

>Should this be a BoF or just a normal topic proposal?  I assume that there
>could be several different topics of interest all related to CXL.mem.

fwiw last year there were 4-5 talks that directly touched on CXL (like half
day).

>Specifically interesting would be the division of work between the kernel
>and userspace to manage memory placement on systems with locally attached
>CXL.  And, further, what APIs userspace would have at its disposal for
>explicit optimization of this placement that would exist beyond what is
>available for NUMA.

+1

>
>I assume we might also want to chat about CXL 2.0 extensions that would be
>useful, especially for cloud providers.
>
>I'd be happy to join in any of the proposals for these topics, just let me
>know!

Likewise.

Thanks,
Davidlohr

Re: [LSF/MM/BPF BoF] Session for CXL memory

[Duen-wen Hsiao]

[-- Attachment #1: Type: text/plain, Size: 1605 bytes --]

Good morning,

I would also be interested in the discussion as well. Please include me for
the CXL related discussion.

Thanks.

Duen-wen Hsiao

On Sun, Jan 22, 2023 at 9:51 PM David Rientjes <rientjes@google.com> wrote:

> On Fri, 6 Jan 2023, Viacheslav A.Dubeyko wrote:
>
> > CC: LSF/MM/BPF mailing list. Sorry, missed the list.
> >
> > > On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <
> viacheslav.dubeyko@bytedance.com> wrote:
> > >
> > > Hello,
> > >
> > > I believe CXL memory is hot topic now. I believe we have multiple
> topics
> > > for discussion. I personally would like to discuss CXL Fabric Manager
> > > and vision of FM architecture implementation. I am going to share the
> topic
> > > in separate email. I would like to suggest a special session for CXL
> memory
> > > related topics.
> > >
> > > How everybody feels about it?
> > >
>
> I think this makes a lot of sense, thanks for suggesting it.
>
> Should this be a BoF or just a normal topic proposal?  I assume that there
> could be several different topics of interest all related to CXL.mem.
>
> Specifically interesting would be the division of work between the kernel
> and userspace to manage memory placement on systems with locally attached
> CXL.  And, further, what APIs userspace would have at its disposal for
> explicit optimization of this placement that would exist beyond what is
> available for NUMA.
>
> I assume we might also want to chat about CXL 2.0 extensions that would be
> useful, especially for cloud providers.
>
> I'd be happy to join in any of the proposals for these topics, just let me
> know!
>

[-- Attachment #2: Type: text/html, Size: 2222 bytes --]

Re: [LSF/MM/BPF BoF] Session for CXL memory

[Adam Manzanares]

On Sun, Jan 22, 2023 at 09:51:19PM -0800, David Rientjes wrote:
> On Fri, 6 Jan 2023, Viacheslav A.Dubeyko wrote:
> 
> > CC: LSF/MM/BPF mailing list. Sorry, missed the list.
> > 
> > > On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <viacheslav.dubeyko@bytedance.com> wrote:
> > > 
> > > Hello,
> > > 
> > > I believe CXL memory is hot topic now. I believe we have multiple topics
> > > for discussion. I personally would like to discuss CXL Fabric Manager
> > > and vision of FM architecture implementation. I am going to share the topic
> > > in separate email. I would like to suggest a special session for CXL memory
> > > related topics.
> > > 
> > > How everybody feels about it?
> > > 
> 
> I think this makes a lot of sense, thanks for suggesting it.
> 
> Should this be a BoF or just a normal topic proposal?  I assume that there 
> could be several different topics of interest all related to CXL.mem.
>

+1 for a normal topic proposal.

> Specifically interesting would be the division of work between the kernel 
> and userspace to manage memory placement on systems with locally attached 
> CXL.  And, further, what APIs userspace would have at its disposal for 
> explicit optimization of this placement that would exist beyond what is 
> available for NUMA.

Given virtual hierarchies can exist I think this becomes a very important
topic of discussion. Locally attached CXL is dynamic, and the CXL hierarchy as 
well as device characteristics are factors in the performance expectations of
the host.

In addition, I would like to discuss the best way to emulate such topologies.
QEMU is a great tool for functional testing, I am not sure if there are ways
to leverage it in order to emulate performance characteristics of CXL devices
and hierarchies. 

> 
> I assume we might also want to chat about CXL 2.0 extensions that would be 
> useful, especially for cloud providers.
>

The session about DCDs at Plumbers comes to mind here. In addition, I think this
leads into a general discussion of how the dynamic nature of CXL (devices and 
topologies) will be heavily dependent on hot add and remove of system memory. 
We have some exploratory work in this area and I am cc'ing the relevant people.

> I'd be happy to join in any of the proposals for these topics, just let me 
> know!

+1 for me.

Re: [LSF/MM/BPF BoF] Session for CXL memory

[Matthew Wilcox]

On Mon, Jan 23, 2023 at 05:46:00PM +0000, Adam Manzanares wrote:
> The session about DCDs at Plumbers comes to mind here. In addition, I think this

What's a DCD?

Re: [External] [LSF/MM/BPF BoF] Session for CXL memory

[Viacheslav A.Dubeyko]

> On Jan 23, 2023, at 10:29 AM, Matthew Wilcox <willy@infradead.org> wrote:
> 
> On Mon, Jan 23, 2023 at 05:46:00PM +0000, Adam Manzanares wrote:
>> The session about DCDs at Plumbers comes to mind here. In addition, I think this
> 
> What's a DCD?

DCD - Dynamic Capacity Device (CXL Specification 3.0)

Re: [LSF/MM/BPF BoF] Session for CXL memory

[Adam Manzanares]

On Mon, Jan 23, 2023 at 06:29:52PM +0000, Matthew Wilcox wrote:
> On Mon, Jan 23, 2023 at 05:46:00PM +0000, Adam Manzanares wrote:
> > The session about DCDs at Plumbers comes to mind here. In addition, I think this
> 
> What's a DCD?

Dynamic capacity device. Here is the link to the Plumbers talk:


https://lpc.events/event/16/contributions/1252/

RE: [LSF/MM/BPF BoF] Session for CXL memory

[Gregory Price]

Dynamic Capacity Device

-----Original Message-----
From: Matthew Wilcox <willy@infradead.org> 
Sent: Monday, January 23, 2023 1:30 PM
To: Adam Manzanares <a.manzanares@samsung.com>
Cc: David Rientjes <rientjes@google.com>; Viacheslav A.Dubeyko <viacheslav.dubeyko@bytedance.com>; lsf-pc@lists.linux-foundation.org; linux-cxl@vger.kernel.org; linux-mm@kvack.org; Dan Williams <dan.j.williams@intel.com>; Jonathan Cameron <jonathan.cameron@huawei.com>; Duen-wen Hsiao <duenwen@google.com>; Johannes Weiner <hannes@cmpxchg.org>; Fan Ni <fan.ni@samsung.com>
Subject: Re: [LSF/MM/BPF BoF] Session for CXL memory

On Mon, Jan 23, 2023 at 05:46:00PM +0000, Adam Manzanares wrote:
> The session about DCDs at Plumbers comes to mind here. In addition, I think this

What's a DCD?

Re: [External] [LSF/MM/BPF BoF] Session for CXL memory

[Viacheslav A.Dubeyko]

> On Jan 23, 2023, at 9:46 AM, Adam Manzanares <a.manzanares@samsung.com> wrote:
> 
> On Sun, Jan 22, 2023 at 09:51:19PM -0800, David Rientjes wrote:
>> On Fri, 6 Jan 2023, Viacheslav A.Dubeyko wrote:
>> 
>>> CC: LSF/MM/BPF mailing list. Sorry, missed the list.
>>> 
>>>> On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <viacheslav.dubeyko@bytedance.com> wrote:
>>>> 
>>>> Hello,
>>>> 
>>>> I believe CXL memory is hot topic now. I believe we have multiple topics
>>>> for discussion. I personally would like to discuss CXL Fabric Manager
>>>> and vision of FM architecture implementation. I am going to share the topic
>>>> in separate email. I would like to suggest a special session for CXL memory
>>>> related topics.
>>>> 
>>>> How everybody feels about it?
>>>> 
>> 
>> I think this makes a lot of sense, thanks for suggesting it.
>> 
>> Should this be a BoF or just a normal topic proposal?  I assume that there 
>> could be several different topics of interest all related to CXL.mem.
>> 
> 
> +1 for a normal topic proposal.

By the way, Samsung SMDK (memory development kit) suggested memory model
in user-space and kernel memory subsystem modification. I assume that we need
to discuss the memory model and kernel-space modification. So, it could be one of
the CXL related topic. Could we expect that Samsung guys deliver a talk?

Thanks,
Slava.

Re: [External] [LSF/MM/BPF BoF] Session for CXL memory

[Adam Manzanares]

On Mon, Jan 23, 2023 at 10:30:34AM -0800, Viacheslav A.Dubeyko wrote:
> 
> 
> > On Jan 23, 2023, at 9:46 AM, Adam Manzanares <a.manzanares@samsung.com> wrote:
> > 
> > On Sun, Jan 22, 2023 at 09:51:19PM -0800, David Rientjes wrote:
> >> On Fri, 6 Jan 2023, Viacheslav A.Dubeyko wrote:
> >> 
> >>> CC: LSF/MM/BPF mailing list. Sorry, missed the list.
> >>> 
> >>>> On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <viacheslav.dubeyko@bytedance.com> wrote:
> >>>> 
> >>>> Hello,
> >>>> 
> >>>> I believe CXL memory is hot topic now. I believe we have multiple topics
> >>>> for discussion. I personally would like to discuss CXL Fabric Manager
> >>>> and vision of FM architecture implementation. I am going to share the topic
> >>>> in separate email. I would like to suggest a special session for CXL memory
> >>>> related topics.
> >>>> 
> >>>> How everybody feels about it?
> >>>> 
> >> 
> >> I think this makes a lot of sense, thanks for suggesting it.
> >> 
> >> Should this be a BoF or just a normal topic proposal?  I assume that there 
> >> could be several different topics of interest all related to CXL.mem.
> >> 
> > 
> > +1 for a normal topic proposal.
> 
> By the way, Samsung SMDK (memory development kit) suggested memory model
> in user-space and kernel memory subsystem modification. I assume that we need
> to discuss the memory model and kernel-space modification. So, it could be one of
> the CXL related topic. Could we expect that Samsung guys deliver a talk?

+Kyungsan Kim

It is my understanding that Kyungsan, who is the lead for the SMDK project is 
interested in sharing with the community some lessons learned from working with 
CXL attached memory and a CFP is planned. 

> 
> Thanks,
> Slava.
> 

Re: [External] [LSF/MM/BPF BoF] Session for CXL memory

[Viacheslav A.Dubeyko]

> On Jan 26, 2023, at 8:58 AM, Adam Manzanares <a.manzanares@samsung.com> wrote:
> 
> On Mon, Jan 23, 2023 at 10:30:34AM -0800, Viacheslav A.Dubeyko wrote:
>> 
>> 
>>> On Jan 23, 2023, at 9:46 AM, Adam Manzanares <a.manzanares@samsung.com> wrote:
>>> 
>>> On Sun, Jan 22, 2023 at 09:51:19PM -0800, David Rientjes wrote:
>>>> On Fri, 6 Jan 2023, Viacheslav A.Dubeyko wrote:
>>>> 
>>>>> CC: LSF/MM/BPF mailing list. Sorry, missed the list.
>>>>> 
>>>>>> On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <viacheslav.dubeyko@bytedance.com> wrote:
>>>>>> 
>>>>>> Hello,
>>>>>> 
>>>>>> I believe CXL memory is hot topic now. I believe we have multiple topics
>>>>>> for discussion. I personally would like to discuss CXL Fabric Manager
>>>>>> and vision of FM architecture implementation. I am going to share the topic
>>>>>> in separate email. I would like to suggest a special session for CXL memory
>>>>>> related topics.
>>>>>> 
>>>>>> How everybody feels about it?
>>>>>> 
>>>> 
>>>> I think this makes a lot of sense, thanks for suggesting it.
>>>> 
>>>> Should this be a BoF or just a normal topic proposal?  I assume that there 
>>>> could be several different topics of interest all related to CXL.mem.
>>>> 
>>> 
>>> +1 for a normal topic proposal.
>> 
>> By the way, Samsung SMDK (memory development kit) suggested memory model
>> in user-space and kernel memory subsystem modification. I assume that we need
>> to discuss the memory model and kernel-space modification. So, it could be one of
>> the CXL related topic. Could we expect that Samsung guys deliver a talk?
> 
> +Kyungsan Kim
> 
> It is my understanding that Kyungsan, who is the lead for the SMDK project is 
> interested in sharing with the community some lessons learned from working with 
> CXL attached memory and a CFP is planned. 
> 

Sounds pretty great for me. :)

Thanks,
Slava.

Re: [External] [LSF/MM/BPF BoF] Session for CXL memory

[MTK]

[-- Attachment #1: Type: text/plain, Size: 2649 bytes --]

Thank you Mr. Slava and Adam for introducing our research.

If I was given an opportunity, I would like to share and discuss
- A case of memory tiering solution for CXL memory, Samsung SMDK, which
works across userspace and kernelspace : why and how
- Some lessens learned from Industry while CXL integration
- Proposal for possible Linux MM expansions for CXL memory

Regarding FM topic, I'm also interested in what we should supports for CXL
3.0 and later topology.

All the best
Kyungsan

2023년 1월 27일 (금) 오전 4:05, Viacheslav A.Dubeyko <
viacheslav.dubeyko@bytedance.com>님이 작성:

>
>
> > On Jan 26, 2023, at 8:58 AM, Adam Manzanares <a.manzanares@samsung.com>
> wrote:
> >
> > On Mon, Jan 23, 2023 at 10:30:34AM -0800, Viacheslav A.Dubeyko wrote:
> >>
> >>
> >>> On Jan 23, 2023, at 9:46 AM, Adam Manzanares <a.manzanares@samsung.com>
> wrote:
> >>>
> >>> On Sun, Jan 22, 2023 at 09:51:19PM -0800, David Rientjes wrote:
> >>>> On Fri, 6 Jan 2023, Viacheslav A.Dubeyko wrote:
> >>>>
> >>>>> CC: LSF/MM/BPF mailing list. Sorry, missed the list.
> >>>>>
> >>>>>> On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <
> viacheslav.dubeyko@bytedance.com> wrote:
> >>>>>>
> >>>>>> Hello,
> >>>>>>
> >>>>>> I believe CXL memory is hot topic now. I believe we have multiple
> topics
> >>>>>> for discussion. I personally would like to discuss CXL Fabric
> Manager
> >>>>>> and vision of FM architecture implementation. I am going to share
> the topic
> >>>>>> in separate email. I would like to suggest a special session for
> CXL memory
> >>>>>> related topics.
> >>>>>>
> >>>>>> How everybody feels about it?
> >>>>>>
> >>>>
> >>>> I think this makes a lot of sense, thanks for suggesting it.
> >>>>
> >>>> Should this be a BoF or just a normal topic proposal?  I assume that
> there
> >>>> could be several different topics of interest all related to CXL.mem.
> >>>>
> >>>
> >>> +1 for a normal topic proposal.
> >>
> >> By the way, Samsung SMDK (memory development kit) suggested memory model
> >> in user-space and kernel memory subsystem modification. I assume that
> we need
> >> to discuss the memory model and kernel-space modification. So, it could
> be one of
> >> the CXL related topic. Could we expect that Samsung guys deliver a talk?
> >
> > +Kyungsan Kim
> >
> > It is my understanding that Kyungsan, who is the lead for the SMDK
> project is
> > interested in sharing with the community some lessons learned from
> working with
> > CXL attached memory and a CFP is planned.
> >
>
> Sounds pretty great for me. :)
>
> Thanks,
> Slava.
>
>

[-- Attachment #2: Type: text/html, Size: 4061 bytes --]

Re: [External] [LSF/MM/BPF BoF] Session for CXL memory

[MTK]

[-- Attachment #1: Type: text/plain, Size: 2933 bytes --]

I'm sorry, forwarding as plaintext.

On Sun, Jan 29, 2023 at 10:45 AM MTK <kim1158@gmail.com> wrote:
>
> Thank you Mr. Slava and Adam for introducing our research.
>
> If I was given an opportunity, I would like to share and discuss
> - A case of memory tiering solution for CXL memory, Samsung SMDK, which
works across userspace and kernelspace : why and how
> - Some lessens learned from Industry while CXL integration
> - Proposal for possible Linux MM expansions for CXL memory
>
> Regarding FM topic, I'm also interested in what we should supports for
CXL 3.0 and later topology.
>
> All the best
> Kyungsan
>
> 2023년 1월 27일 (금) 오전 4:05, Viacheslav A.Dubeyko <
viacheslav.dubeyko@bytedance.com>님이 작성:
>>
>>
>>
>> > On Jan 26, 2023, at 8:58 AM, Adam Manzanares <a.manzanares@samsung.com>
wrote:
>> >
>> > On Mon, Jan 23, 2023 at 10:30:34AM -0800, Viacheslav A.Dubeyko wrote:
>> >>
>> >>
>> >>> On Jan 23, 2023, at 9:46 AM, Adam Manzanares <
a.manzanares@samsung.com> wrote:
>> >>>
>> >>> On Sun, Jan 22, 2023 at 09:51:19PM -0800, David Rientjes wrote:
>> >>>> On Fri, 6 Jan 2023, Viacheslav A.Dubeyko wrote:
>> >>>>
>> >>>>> CC: LSF/MM/BPF mailing list. Sorry, missed the list.
>> >>>>>
>> >>>>>> On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <
viacheslav.dubeyko@bytedance.com> wrote:
>> >>>>>>
>> >>>>>> Hello,
>> >>>>>>
>> >>>>>> I believe CXL memory is hot topic now. I believe we have multiple
topics
>> >>>>>> for discussion. I personally would like to discuss CXL Fabric
Manager
>> >>>>>> and vision of FM architecture implementation. I am going to share
the topic
>> >>>>>> in separate email. I would like to suggest a special session for
CXL memory
>> >>>>>> related topics.
>> >>>>>>
>> >>>>>> How everybody feels about it?
>> >>>>>>
>> >>>>
>> >>>> I think this makes a lot of sense, thanks for suggesting it.
>> >>>>
>> >>>> Should this be a BoF or just a normal topic proposal?  I assume
that there
>> >>>> could be several different topics of interest all related to
CXL.mem.
>> >>>>
>> >>>
>> >>> +1 for a normal topic proposal.
>> >>
>> >> By the way, Samsung SMDK (memory development kit) suggested memory
model
>> >> in user-space and kernel memory subsystem modification. I assume that
we need
>> >> to discuss the memory model and kernel-space modification. So, it
could be one of
>> >> the CXL related topic. Could we expect that Samsung guys deliver a
talk?
>> >
>> > +Kyungsan Kim
>> >
>> > It is my understanding that Kyungsan, who is the lead for the SMDK
project is
>> > interested in sharing with the community some lessons learned from
working with
>> > CXL attached memory and a CFP is planned.
>> >
>>
>> Sounds pretty great for me. :)
>>
>> Thanks,
>> Slava.
>>


-- 
------------------------------------------------------------
the person who practices a truth goes toward light.

[-- Attachment #2: Type: text/html, Size: 4178 bytes --]

Re: [External] [LSF/MM/BPF BoF] Session for CXL memory

[Viacheslav A.Dubeyko]

Hi Kyungsan,

> On Jan 28, 2023, at 5:45 PM, MTK <kim1158@gmail.com> wrote:
> 
> Thank you Mr. Slava and Adam for introducing our research.
> 
> If I was given an opportunity, I would like to share and discuss 
> - A case of memory tiering solution for CXL memory, Samsung SMDK, which works across userspace and kernelspace : why and how
> - Some lessens learned from Industry while CXL integration
> - Proposal for possible Linux MM expansions for CXL memory
> 
> Regarding FM topic, I'm also interested in what we should supports for CXL 3.0 and later topology.
> 

Please, send the topic suggestion then. :) You can check LSF/MM/BPF CFP page for guidance.

Thanks,
Slava.

Re: [External] [LSF/MM/BPF BoF] Session for CXL memory

[Viacheslav A.Dubeyko]

> On Jan 22, 2023, at 9:51 PM, David Rientjes <rientjes@google.com> wrote:
> 
> On Fri, 6 Jan 2023, Viacheslav A.Dubeyko wrote:
> 
>> CC: LSF/MM/BPF mailing list. Sorry, missed the list.
>> 
>>> On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <viacheslav.dubeyko@bytedance.com> wrote:
>>> 
>>> Hello,
>>> 
>>> I believe CXL memory is hot topic now. I believe we have multiple topics
>>> for discussion. I personally would like to discuss CXL Fabric Manager
>>> and vision of FM architecture implementation. I am going to share the topic
>>> in separate email. I would like to suggest a special session for CXL memory
>>> related topics.
>>> 
>>> How everybody feels about it?
>>> 
> 
> I think this makes a lot of sense, thanks for suggesting it.
> 
> Should this be a BoF or just a normal topic proposal?  I assume that there 
> could be several different topics of interest all related to CXL.mem.
> 

My point is to have a dedicated CXL session where we can discuss
CXL related topics. And we can have likewise session if several CXL
related topics will be suggested. :)

> Specifically interesting would be the division of work between the kernel 
> and userspace to manage memory placement on systems with locally attached 
> CXL.  And, further, what APIs userspace would have at its disposal for 
> explicit optimization of this placement that would exist beyond what is 
> available for NUMA.
> 

Yes, I think this could be important discussion. Because, for example, I am working
on figuring out how functionality can be distributed among user-space, kernel-space,
and firmware for the case of Fabric Manager. And I believe it could be a good topic
that I have in mind.

> I assume we might also want to chat about CXL 2.0 extensions that would be 
> useful, especially for cloud providers.
> 

What CXL 2.0 extensions would be useful for cloud providers from your point of view?

Thanks,
Slava.

Re: [Lsf-pc] [External] [LSF/MM/BPF BoF] Session for CXL memory

[Dan Williams]

Viacheslav A.Dubeyko wrote:
> 
> 
> > On Jan 22, 2023, at 9:51 PM, David Rientjes <rientjes@google.com> wrote:
> > 
> > On Fri, 6 Jan 2023, Viacheslav A.Dubeyko wrote:
> > 
> >> CC: LSF/MM/BPF mailing list. Sorry, missed the list.
> >> 
> >>> On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <viacheslav.dubeyko@bytedance.com> wrote:
> >>> 
> >>> Hello,
> >>> 
> >>> I believe CXL memory is hot topic now. I believe we have multiple topics
> >>> for discussion. I personally would like to discuss CXL Fabric Manager
> >>> and vision of FM architecture implementation. I am going to share the topic
> >>> in separate email. I would like to suggest a special session for CXL memory
> >>> related topics.
> >>> 
> >>> How everybody feels about it?
> >>> 
> > 
> > I think this makes a lot of sense, thanks for suggesting it.
> > 
> > Should this be a BoF or just a normal topic proposal?  I assume that there 
> > could be several different topics of interest all related to CXL.mem.
> > 
> 
> My point is to have a dedicated CXL session where we can discuss
> CXL related topics. And we can have likewise session if several CXL
> related topics will be suggested. :)

It worked well last time to have individual CXL topic proposals that
could later be grouped into multiple sessions by the planning committee.

So yes, as you mention above do send separate mails with the finer
grained CXL topic in the "Subject:" so it does not get lost in the
shuffle.

> > Specifically interesting would be the division of work between the kernel 
> > and userspace to manage memory placement on systems with locally attached 
> > CXL.  And, further, what APIs userspace would have at its disposal for 
> > explicit optimization of this placement that would exist beyond what is 
> > available for NUMA.
> > 
> 
> Yes, I think this could be important discussion. Because, for example, I am working
> on figuring out how functionality can be distributed among user-space, kernel-space,
> and firmware for the case of Fabric Manager. And I believe it could be a good topic
> that I have in mind.
> 
> > I assume we might also want to chat about CXL 2.0 extensions that would be 
> > useful, especially for cloud providers.
> > 
> 
> What CXL 2.0 extensions would be useful for cloud providers from your point of view?

...or even CXL 3.0 given that specification is released now.

Re: [LSF/MM/BPF BoF] Session for CXL memory

[Yang Shi]

On Sun, Jan 22, 2023 at 9:51 PM David Rientjes <rientjes@google.com> wrote:
>
> On Fri, 6 Jan 2023, Viacheslav A.Dubeyko wrote:
>
> > CC: LSF/MM/BPF mailing list. Sorry, missed the list.
> >
> > > On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <viacheslav.dubeyko@bytedance.com> wrote:
> > >
> > > Hello,
> > >
> > > I believe CXL memory is hot topic now. I believe we have multiple topics
> > > for discussion. I personally would like to discuss CXL Fabric Manager
> > > and vision of FM architecture implementation. I am going to share the topic
> > > in separate email. I would like to suggest a special session for CXL memory
> > > related topics.
> > >
> > > How everybody feels about it?
> > >
>
> I think this makes a lot of sense, thanks for suggesting it.
>
> Should this be a BoF or just a normal topic proposal?  I assume that there
> could be several different topics of interest all related to CXL.mem.
>
> Specifically interesting would be the division of work between the kernel
> and userspace to manage memory placement on systems with locally attached
> CXL.  And, further, what APIs userspace would have at its disposal for
> explicit optimization of this placement that would exist beyond what is
> available for NUMA.

Yeah, in addition, how CXL hierarchy interacts with memory tiering may
be an interesting topic too.

>
> I assume we might also want to chat about CXL 2.0 extensions that would be
> useful, especially for cloud providers.
>
> I'd be happy to join in any of the proposals for these topics, just let me
> know!

+1 for me.

>

Re: [LSF/MM/BPF BoF] Session for CXL memory

[Wei Xu]

On Mon, Jan 23, 2023 at 4:22 PM Yang Shi <shy828301@gmail.com> wrote:
>
> On Sun, Jan 22, 2023 at 9:51 PM David Rientjes <rientjes@google.com> wrote:
> >
> > On Fri, 6 Jan 2023, Viacheslav A.Dubeyko wrote:
> >
> > > CC: LSF/MM/BPF mailing list. Sorry, missed the list.
> > >
> > > > On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <viacheslav.dubeyko@bytedance.com> wrote:
> > > >
> > > > Hello,
> > > >
> > > > I believe CXL memory is hot topic now. I believe we have multiple topics
> > > > for discussion. I personally would like to discuss CXL Fabric Manager
> > > > and vision of FM architecture implementation. I am going to share the topic
> > > > in separate email. I would like to suggest a special session for CXL memory
> > > > related topics.
> > > >
> > > > How everybody feels about it?
> > > >
> >
> > I think this makes a lot of sense, thanks for suggesting it.
> >
> > Should this be a BoF or just a normal topic proposal?  I assume that there
> > could be several different topics of interest all related to CXL.mem.
> >
> > Specifically interesting would be the division of work between the kernel
> > and userspace to manage memory placement on systems with locally attached
> > CXL.  And, further, what APIs userspace would have at its disposal for
> > explicit optimization of this placement that would exist beyond what is
> > available for NUMA.
>
> Yeah, in addition, how CXL hierarchy interacts with memory tiering may
> be an interesting topic too.
>
> >
> > I assume we might also want to chat about CXL 2.0 extensions that would be
> > useful, especially for cloud providers.
> >
> > I'd be happy to join in any of the proposals for these topics, just let me
> > know!
>
> +1 for me.
>

I am also interested in these discussions.

Re: [LSF/MM/BPF BoF] Session for CXL memory

[Zhu Yanjun]

在 2023/1/24 8:57, Wei Xu 写道:
> On Mon, Jan 23, 2023 at 4:22 PM Yang Shi <shy828301@gmail.com> wrote:
>>
>> On Sun, Jan 22, 2023 at 9:51 PM David Rientjes <rientjes@google.com> wrote:
>>>
>>> On Fri, 6 Jan 2023, Viacheslav A.Dubeyko wrote:
>>>
>>>> CC: LSF/MM/BPF mailing list. Sorry, missed the list.
>>>>
>>>>> On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <viacheslav.dubeyko@bytedance.com> wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>> I believe CXL memory is hot topic now. I believe we have multiple topics
>>>>> for discussion. I personally would like to discuss CXL Fabric Manager
>>>>> and vision of FM architecture implementation. I am going to share the topic
>>>>> in separate email. I would like to suggest a special session for CXL memory
>>>>> related topics.
>>>>>
>>>>> How everybody feels about it?
>>>>>
>>>
>>> I think this makes a lot of sense, thanks for suggesting it.
>>>
>>> Should this be a BoF or just a normal topic proposal?  I assume that there
>>> could be several different topics of interest all related to CXL.mem.
>>>
>>> Specifically interesting would be the division of work between the kernel
>>> and userspace to manage memory placement on systems with locally attached
>>> CXL.  And, further, what APIs userspace would have at its disposal for
>>> explicit optimization of this placement that would exist beyond what is
>>> available for NUMA.
>>
>> Yeah, in addition, how CXL hierarchy interacts with memory tiering may
>> be an interesting topic too.
>>
>>>
>>> I assume we might also want to chat about CXL 2.0 extensions that would be
>>> useful, especially for cloud providers.
>>>
>>> I'd be happy to join in any of the proposals for these topics, just let me
>>> know!
>>
>> +1 for me.
>>
> 
> I am also interested in these discussions.

+1， I am interested in this.

> 

Re: [LSF/MM/BPF BoF] Session for CXL memory

[Dragan Stancevic]

On 1/25/23 09:04, Zhu Yanjun wrote:
> 在 2023/1/24 8:57, Wei Xu 写道:
>> On Mon, Jan 23, 2023 at 4:22 PM Yang Shi <shy828301@gmail.com> wrote:
>>>
>>> On Sun, Jan 22, 2023 at 9:51 PM David Rientjes <rientjes@google.com> 
>>> wrote:
>>>>
>>>> On Fri, 6 Jan 2023, Viacheslav A.Dubeyko wrote:
>>>>
>>>>> CC: LSF/MM/BPF mailing list. Sorry, missed the list.
>>>>>
>>>>>> On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko 
>>>>>> <viacheslav.dubeyko@bytedance.com> wrote:
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I believe CXL memory is hot topic now. I believe we have multiple 
>>>>>> topics
>>>>>> for discussion. I personally would like to discuss CXL Fabric Manager
>>>>>> and vision of FM architecture implementation. I am going to share 
>>>>>> the topic
>>>>>> in separate email. I would like to suggest a special session for 
>>>>>> CXL memory
>>>>>> related topics.
>>>>>>
>>>>>> How everybody feels about it?
>>>>>>
>>>>
>>>> I think this makes a lot of sense, thanks for suggesting it.
>>>>
>>>> Should this be a BoF or just a normal topic proposal?  I assume that 
>>>> there
>>>> could be several different topics of interest all related to CXL.mem.
>>>>
>>>> Specifically interesting would be the division of work between the 
>>>> kernel
>>>> and userspace to manage memory placement on systems with locally 
>>>> attached
>>>> CXL.  And, further, what APIs userspace would have at its disposal for
>>>> explicit optimization of this placement that would exist beyond what is
>>>> available for NUMA.
>>>
>>> Yeah, in addition, how CXL hierarchy interacts with memory tiering may
>>> be an interesting topic too.
>>>
>>>>
>>>> I assume we might also want to chat about CXL 2.0 extensions that 
>>>> would be
>>>> useful, especially for cloud providers.
>>>>
>>>> I'd be happy to join in any of the proposals for these topics, just 
>>>> let me
>>>> know!
>>>
>>> +1 for me.
>>>
>>
>> I am also interested in these discussions.
> 
> +1， I am interested in this.

+1, also interested in cloud aspects


--
Peace can only come as a natural consequence
of universal enlightenment -Dr. Nikola Tesla

Re: [LSF/MM/BPF BoF] Session for CXL memory

[Aneesh Kumar K.V]

Yang Shi <shy828301@gmail.com> writes:

> On Sun, Jan 22, 2023 at 9:51 PM David Rientjes <rientjes@google.com> wrote:
>>
>> On Fri, 6 Jan 2023, Viacheslav A.Dubeyko wrote:
>>
>> > CC: LSF/MM/BPF mailing list. Sorry, missed the list.
>> >
>> > > On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <viacheslav.dubeyko@bytedance.com> wrote:
>> > >
>> > > Hello,
>> > >
>> > > I believe CXL memory is hot topic now. I believe we have multiple topics
>> > > for discussion. I personally would like to discuss CXL Fabric Manager
>> > > and vision of FM architecture implementation. I am going to share the topic
>> > > in separate email. I would like to suggest a special session for CXL memory
>> > > related topics.
>> > >
>> > > How everybody feels about it?
>> > >
>>
>> I think this makes a lot of sense, thanks for suggesting it.
>>
>> Should this be a BoF or just a normal topic proposal?  I assume that there
>> could be several different topics of interest all related to CXL.mem.
>>
>> Specifically interesting would be the division of work between the kernel
>> and userspace to manage memory placement on systems with locally attached
>> CXL.  And, further, what APIs userspace would have at its disposal for
>> explicit optimization of this placement that would exist beyond what is
>> available for NUMA.
>
> Yeah, in addition, how CXL hierarchy interacts with memory tiering may
> be an interesting topic too.
>
>>
>> I assume we might also want to chat about CXL 2.0 extensions that would be
>> useful, especially for cloud providers.
>>
>> I'd be happy to join in any of the proposals for these topics, just let me
>> know!
>
> +1 for me.
>

I would also be interested in finding out what we learned about the
device attributes that people want to use for building hierarchy. We
still haven't hooked up HMAT/CDAT to memory tiers. It would be good to
understand and discuss on how we should make progress here.

-aneesh

Re: [LSF/MM/BPF BoF] Session for CXL memory

[Jonathan Cameron]

On Fri, 6 Jan 2023 14:20:42 -0800
"Viacheslav A.Dubeyko" <viacheslav.dubeyko@bytedance.com> wrote:

> CC: LSF/MM/BPF mailing list. Sorry, missed the list.
> 
> > On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <viacheslav.dubeyko@bytedance.com> wrote:
> > 
> > Hello,
> > 
> > I believe CXL memory is hot topic now. I believe we have multiple topics
> > for discussion. I personally would like to discuss CXL Fabric Manager
> > and vision of FM architecture implementation. 

The fabric manager is rather disconnected from the host side of things (all
out of band communications), so it would be a stretch to build a stand alone
topic around that for LSF-MM. If we have the right people in the room /
online, it would be good to discuss it (so part of a wider sessions on CXL).
I'd love to get some traction before LSFMM though! Host aspects such as
Dynamic Capacity Devices and sharing feel more LSFMM suitable.
 
> >I am going to share the topic in separate email.

Did I miss the email, or not sent yet?  That topic is obscure enough we definitely
need some background if anyone outside of CXL folk is going to have any idea what
we are talking about.

> > would like to suggest a special session for CXL memory
> > related topics.
> > 
> > How everybody feels about it?

A lot of the interesting bits currently strike me as rather speculative
(no code), so sessions might not be as productive as shooting at an
implementation. That's less true of FM stuff as we really do need
an outline of an architecture plus some planning on that.

Could we have something to shoot at for other topics in the
time frame?  maybe...

I'm definitely interested in general in CXL enablement topics
and FM side of things.

Jonathan


> > 
> > Thanks,
> > Slava.
> >   
> 

Re: [External] [LSF/MM/BPF BoF] Session for CXL memory

[Viacheslav A.Dubeyko]

> On Jan 24, 2023, at 2:12 AM, Jonathan Cameron <Jonathan.Cameron@Huawei.com> wrote:
> 
> On Fri, 6 Jan 2023 14:20:42 -0800
> "Viacheslav A.Dubeyko" <viacheslav.dubeyko@bytedance.com> wrote:
> 
>> CC: LSF/MM/BPF mailing list. Sorry, missed the list.
>> 
>>> On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <viacheslav.dubeyko@bytedance.com> wrote:
>>> 
>>> Hello,
>>> 
>>> I believe CXL memory is hot topic now. I believe we have multiple topics
>>> for discussion. I personally would like to discuss CXL Fabric Manager
>>> and vision of FM architecture implementation. 
> 
> The fabric manager is rather disconnected from the host side of things (all
> out of band communications), so it would be a stretch to build a stand alone
> topic around that for LSF-MM. If we have the right people in the room /
> online, it would be good to discuss it (so part of a wider sessions on CXL).
> I'd love to get some traction before LSFMM though! Host aspects such as
> Dynamic Capacity Devices and sharing feel more LSFMM suitable.
> 
>>> I am going to share the topic in separate email.
> 
> Did I miss the email, or not sent yet?  That topic is obscure enough we definitely
> need some background if anyone outside of CXL folk is going to have any idea what
> we are talking about.
> 

You missed nothing. :) I am still polishing the FM related topic. Sorry, I was busy
with other tasks. 

>>> would like to suggest a special session for CXL memory
>>> related topics.
>>> 
>>> How everybody feels about it?
> 
> A lot of the interesting bits currently strike me as rather speculative
> (no code), so sessions might not be as productive as shooting at an
> implementation. That's less true of FM stuff as we really do need
> an outline of an architecture plus some planning on that.
> 
> Could we have something to shoot at for other topics in the
> time frame?  maybe...
> 

I believe even discussion before implementation could make sense.
Because, it provides the way to make the architecture/API vision more
clear and understandable by everyone sometimes. :)

Thanks,
Slava.

Re: [External] [LSF/MM/BPF BoF] Session for CXL memory

[Dan Williams]

Viacheslav A.Dubeyko wrote:
> 
> 
> > On Jan 24, 2023, at 2:12 AM, Jonathan Cameron <Jonathan.Cameron@Huawei.com> wrote:
> > 
> > On Fri, 6 Jan 2023 14:20:42 -0800
> > "Viacheslav A.Dubeyko" <viacheslav.dubeyko@bytedance.com> wrote:
> > 
> >> CC: LSF/MM/BPF mailing list. Sorry, missed the list.
> >> 
> >>> On Jan 6, 2023, at 11:51 AM, Viacheslav A.Dubeyko <viacheslav.dubeyko@bytedance.com> wrote:
> >>> 
> >>> Hello,
> >>> 
> >>> I believe CXL memory is hot topic now. I believe we have multiple topics
> >>> for discussion. I personally would like to discuss CXL Fabric Manager
> >>> and vision of FM architecture implementation. 
> > 
> > The fabric manager is rather disconnected from the host side of things (all
> > out of band communications), so it would be a stretch to build a stand alone
> > topic around that for LSF-MM. If we have the right people in the room /
> > online, it would be good to discuss it (so part of a wider sessions on CXL).
> > I'd love to get some traction before LSFMM though! Host aspects such as
> > Dynamic Capacity Devices and sharing feel more LSFMM suitable.
> > 
> >>> I am going to share the topic in separate email.
> > 
> > Did I miss the email, or not sent yet?  That topic is obscure enough we definitely
> > need some background if anyone outside of CXL folk is going to have any idea what
> > we are talking about.
> > 
> 
> You missed nothing. :) I am still polishing the FM related topic. Sorry, I was busy
> with other tasks. 
> 
> >>> would like to suggest a special session for CXL memory
> >>> related topics.
> >>> 
> >>> How everybody feels about it?
> > 
> > A lot of the interesting bits currently strike me as rather speculative
> > (no code), so sessions might not be as productive as shooting at an
> > implementation. That's less true of FM stuff as we really do need
> > an outline of an architecture plus some planning on that.
> > 
> > Could we have something to shoot at for other topics in the
> > time frame?  maybe...
> > 
> 
> I believe even discussion before implementation could make sense.
> Because, it provides the way to make the architecture/API vision more
> clear and understandable by everyone sometimes. :)

I have the sense that a CXL Fabric Manager discussion is more
appropriate as a BoF that can pull in some storage folks that have
experience with out-of-band device control. Otherwise, for the core-MM
crowd it is not as relevant since that policy and control is outside of
the kernel. So, I would hope to have CXL topics that have core-MM
implications as the first order of business and then have a BoF for
those that also want to talk about FM collaboration.