Re: [PATCH v3 3/3] libnvdimm: Add sysfs numa_node to NVDIMM devices

Toshi Kani
2015-06-24T16:38:46Z

On Tue, 2015-06-23 at 17:26 -0700, Dan Williams wrote:
> On Fri, Jun 19, 2015 at 11:18 AM, Toshi Kani <toshi.kani@hp.com> wrote:
> > Add support of sysfs 'numa_node' to I/O-related NVDIMM devices
> > under /sys/bus/nd/devices, regionN, namespaceN.0, and bttN.
> > When bttN is not set up, its numa_node returns -1 (NUMA_NO_NODE).
> >
> > An example of numa_node values on a 2-socket system with a single
> > NVDIMM range on each socket is shown below.
> > /sys/bus/nd/devices
> > |-- btt0/numa_node:-1
> > |-- btt1/numa_node:0
> > |-- namespace0.0/numa_node:0
> > |-- namespace1.0/numa_node:1
> > |-- region0/numa_node:0
> > |-- region1/numa_node:1
> >
> > These numa_node files are then linked under the block class of
> > their device names.
> > /sys/class/block/pmem0/device/numa_node:0
> > /sys/class/block/pmem0s/device/numa_node:0
> > /sys/class/block/pmem1/device/numa_node:1
> >
> > This enables numactl(8) to accept 'block:' and 'file:' paths of
> > pmem and btt devices as shown in the examples below.
> > numactl --preferred block:pmem0 --show
> > numactl --preferred file:/dev/pmem0s --show
> >
> > Signed-off-by: Toshi Kani <toshi.kani@hp.com>
> > ---
> > drivers/acpi/nfit.c | 1 +
> > drivers/nvdimm/btt_devs.c | 1 +
> > drivers/nvdimm/bus.c | 30 ++++++++++++++++++++++++++++++
> > drivers/nvdimm/namespace_devs.c | 1 +
> > include/linux/libnvdimm.h | 1 +
> > 5 files changed, 34 insertions(+)
> >
> > diff --git a/drivers/acpi/nfit.c b/drivers/acpi/nfit.c
> > index 5997753..9cb63ac 100644
> > --- a/drivers/acpi/nfit.c
> > +++ b/drivers/acpi/nfit.c
> > @@ -873,6 +873,7 @@ static const struct attribute_group *acpi_nfit_region_attribute_groups[] = {
> > &nd_region_attribute_group,
> > &nd_mapping_attribute_group,
> > &nd_device_attribute_group,
> > + &nd_numa_attribute_group,
> > &acpi_nfit_region_attribute_group,
> > NULL,
> > };
> > diff --git a/drivers/nvdimm/btt_devs.c b/drivers/nvdimm/btt_devs.c
> > index bcf77dc..a7b192f 100644
> > --- a/drivers/nvdimm/btt_devs.c
> > +++ b/drivers/nvdimm/btt_devs.c
> > @@ -308,6 +308,7 @@ static struct attribute_group nd_btt_attribute_group = {
> > static const struct attribute_group *nd_btt_attribute_groups[] = {
> > &nd_btt_attribute_group,
> > &nd_device_attribute_group,
> > + &nd_numa_attribute_group,
> > NULL,
> > };
> >
> > diff --git a/drivers/nvdimm/bus.c b/drivers/nvdimm/bus.c
> > index 67525f9..03c0ee1 100644
> > --- a/drivers/nvdimm/bus.c
> > +++ b/drivers/nvdimm/bus.c
> > @@ -420,6 +420,36 @@ struct attribute_group nd_device_attribute_group = {
> > };
> > EXPORT_SYMBOL_GPL(nd_device_attribute_group);
> >
> > +static ssize_t numa_node_show(struct device *dev,
> > + struct device_attribute *attr, char *buf)
> > +{
> > + return sprintf(buf, "%d\n", dev_to_node(dev));
> > +}
>
> So patch 2 collided with the requested BTT stacking rework and
> prompted me to take a closer look. Shouldn't numa_node_show() be
> changed like this?

numa_node_show() is listed in its own nd_numa_attribute_group for using
is_visible. This nd_numa_attribute_group is then listed by region
(acpi_nfit_region_attribute_groups), namespace
(nd_namespace_attribute_groups), and btt (nd_btt_attribute_groups).
Therefore, numa_node_show() is only called with these device objects.
So, I do not think we need such change. Or are you suggesting to change
the way attribute group is set?

Thanks,
-Toshi

> @@ -273,7 +273,12 @@ EXPORT_SYMBOL_GPL(nd_device_attribute_group);
> static ssize_t numa_node_show(struct device *dev,
> struct device_attribute *attr, char *buf)
> {
> - return sprintf(buf, "%d\n", dev_to_node(dev));
> + if (is_nd_region(dev))
> + return sprintf(buf, "%d\n", dev_to_node(dev));
> + else if (is_nd_region(dev->parent))
> + return sprintf(buf, "%d\n", dev_to_node(dev->parent));
> + else
> + return sprintf(buf, "-1\n");
> }
> static DEVICE_ATTR_RO(numa_node);