Re: [ndctl PATCH 0/7] Add partitioning support for CXL memdevs

[Alison Schofield <alison.schofield@intel.com>]

Thanks for the review Ira!
There's one question back at you wrt adding MAN page to cover letter.
Otherwise, I'm absorbing all your feedback.


On Thu, Jan 06, 2022 at 12:32:46PM -0800, Ira Weiny wrote:
> On Mon, Jan 03, 2022 at 12:16:11PM -0800, Schofield, Alison wrote:
> > From: Alison Schofield <alison.schofield@intel.com>
>  
> First, thanks for taking this on!  :-D
> 
> > To support changing partitions on CXL memdevs, first provide access to device
> > partitioning info.
> >
> 
> How about:
> 
> Users will want to configure CXL memdevs on CXL devices which support
> partitioning.  CXL provides get and set partition info mailbox command to do
> this.
> 
> Add support to retrieve partition info and set partition info.
> 
> ...
> 

Will reword, thanks.

> >
> >
> > The first 4 patches add accessors to all
> > the partition info a CXL command parser needs in order to validate
> > the command. This info is added to cxl list to assist the user in
> > creating valid partition requests.
> 
> Great but what is a valid partition request?
>

I didn't regurgitate the restrictions here in the cover letter, nor
in the patch commit logs. They are only visible in the MAN page, and
I reference the MAN page in the cxl set-partition-info command patch.
I could add a pretty version of the MAN page here to give the cover
letter more details.

Would you like to see the MAN page here?

> > 
> > # cxl list -MP
> > [
> >   {
> >     "memdev":"mem0",
> >     "pmem_size":0,
> >     "ram_size":273535729664,
> >     "partition":{
> >       "active_volatile_capacity":273535729664,
> >       "active_persistent_capacity":0,
> >       "next_volatile_capacity":268435456,
> >       "next_persistent_capacity":273267294208,
> >       "total_capacity":273535729664,
> >       "volatile_only_capacity":0,
> >       "persistent_only_capacity":0,
> >       "partition_alignment":268435456
> >     }
> >   }
> > ]
> > 
> > Next introduce libcxl ioctl() interfaces for the SET_PARTITION_INFO
> > mailbox command and the new CXL command. cxl-cli does the constraints
> > checking. It does not offer the IMMEDIATE mode option since we do not
> > have driver support for that yet.
> 
> How about something like 'cxl-cli restricts the use of the IMMEDIATE flag until
> such time as the driver supports it'?

Those words are really not true. cxl-cli doesn't restrict the flags use,
it simply does not offer the option.

> 
> But we probably should just let the command through and rely on the driver to
> do what it does...
> 
Nah. I don't think I should add -i --immediate to the command yet.

(I think we covered this in chatroom)

> > 
> > # cxl set-partition-info
> >  usage: cxl set-partition-info <mem0> [<mem1>..<memN>] [<options>]
> > 
> >     -v, --verbose         turn on debug
> >     -s, --volatile_size <n>
> >                           next volatile partition size in bytes
> > 
> > Guessing that
>   ^^^^^^^^^^^^^
>   Delete
> 
> 'A libcxl user can...'
> 
> >

Will delete.


> > a libcxl user could send the SET_PARTITION_INFO mailbox
> > command outside of cxl-cli tool, so a kernel patch that disables the
> > immediate bit, on the receiving end of the ioctl, follows.
> 
> cool!
> 
> > 
> > It may be simpler to block the immediate bit in the libcxl API today,
> > (and as I write this cover letter I'm wondering just how far this goes
> > astray ;)) However, the kernel patch to peek in the payload sets us on
> > the path of inspecting set-partition-info mailbox commands in the future,
> > when immediate mode support is required.
> 
> I'd just delete this.  I think it is best to leave the kernel to the
> enforcement and not complicate the user space.

got it.

> 
> Ira
> 
> > 
> > Testing - so far I've only tested w one memdev in a Simics env. So,
> > next will be growing that Simics config, using cxl_test env, and 
> > adding a unit test.
> > 
> > Alison Schofield (7):
> >   libcxl: add GET_PARTITION_INFO mailbox command and accessors
> >   libcxl: add accessors for capacity fields of the IDENTIFY command
> >   libcxl: apply CXL_CAPACITY_MULTIPLIER to partition alignment field
> >   cxl: add memdev partition information to cxl-list
> >   libcxl: add interfaces for SET_PARTITION_INFO mailbox command
> >   ndctl, util: use 'unsigned long long' type in OPT_U64 define
> >   cxl: add command set-partition-info
> > 
> >  Documentation/cxl/cxl-list.txt               |  23 ++++
> >  Documentation/cxl/cxl-set-partition-info.txt |  27 +++++
> >  Documentation/cxl/partition-description.txt  |  15 +++
> >  Documentation/cxl/partition-options.txt      |  19 +++
> >  Documentation/cxl/Makefile.am                |   3 +-
> >  cxl/builtin.h                                |   1 +
> >  cxl/lib/private.h                            |  19 +++
> >  cxl/libcxl.h                                 |  12 ++
> >  util/json.h                                  |   1 +
> >  util/parse-options.h                         |   2 +-
> >  util/size.h                                  |   1 +
> >  cxl/cxl.c                                    |   1 +
> >  cxl/lib/libcxl.c                             | 117 ++++++++++++++++++-
> >  cxl/lib/libcxl.sym                           |  11 ++
> >  cxl/list.c                                   |   5 +
> >  cxl/memdev.c                                 |  89 ++++++++++++++
> >  util/json.c                                  | 112 ++++++++++++++++++
> >  17 files changed, 455 insertions(+), 3 deletions(-)
> >  create mode 100644 Documentation/cxl/cxl-set-partition-info.txt
> >  create mode 100644 Documentation/cxl/partition-description.txt
> >  create mode 100644 Documentation/cxl/partition-options.txt
> > 
> > -- 
> > 2.31.1
> >