sysfs for my chips

sysfs for my chips

[Benjamin Herrenschmidt]

Hi Greg !

(random CC list of clueful people)

On some new powerpc platforms (non-hypervisor or rather linux is the
hypervisor), I want to expose a bunch of stuff per "chip", the chips
being currently the processor chips and the "centaurs" (think of them as
the bottom half of the memory controllers).

Among other, I want a sysfs file in there to access "xscom" on the chip
which is a sideband bus used for low level stuff (think jtag on steroid)
which we can use, among others, for chip health monitoring, general
debugging and diagnostics, etc...

I might add more such as VPD, model information, etc... later or at
least a link to corresponding device-tree node.

How do you suggest I expose that ? So far I've been thinking about
something like

	/sys/chips/{processor,centaur}/chip#/files

or to avoid namespace clashes

	/sys/firmware/chips/{processor,centaur}/chip#/files

Or maybe just

	/sys/firmware/chips/chip#/files

(the chip type can be inferred from the chip#, they use the same space
at least as far my firmware exposes them to Linux)

(the actual access to xscom goes via firmware tho it makes *some* sense)

But I could instead create platform devices corresponding to the
device-tree representation of each of those chips ... and have the
platform devices contain the magic attributes. That's a bit more
convoluted though.

What's the current trend of the day for that sort of thing ? I'd rather
avoid yet-another-chardev-with-ioctl's here ...

Cheers,
Ben.

[PATCH] sysfs/bin: Fix size handling overflow for bin_attribute

[Benjamin Herrenschmidt]

While looking at the code, I noticed that bin_attribute read() and write()
ops copy the inode size into an int for futher comparisons.

Some bin_attributes can be fairly large. For example, pci creates some for
BARs set to the BAR size and giant BARs are around the corner, so this is
going to break something somewhere eventually.

Let's use the right type.

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
---

I noticed that while messing around with my "xscom" file which I had
originally set to be LONG_MAX :-)

I eventually decided to use an i_size of 0 instead which seems to be
the way that sort of "special" file tend to be done (it's a bridge to
a special sideband bus in the system which has a sparse addressing which
is splattered all over 64 bits though I've somewhat "compressed" it to 63
for the sake of sysfs).

Note that I noticed that late and don't have a good test case for it,
but code inspection didn't seem to show anything else cropping i_size. 

diff --git a/fs/sysfs/bin.c b/fs/sysfs/bin.c
index c590cab..373ddcf 100644
--- a/fs/sysfs/bin.c
+++ b/fs/sysfs/bin.c
@@ -69,7 +69,7 @@ static ssize_t
 read(struct file *file, char __user *userbuf, size_t bytes, loff_t *off)
 {
 	struct bin_buffer *bb = file->private_data;
-	int size = file_inode(file)->i_size;
+	loff_t size = file_inode(file)->i_size;
 	loff_t offs = *off;
 	int count = min_t(size_t, bytes, PAGE_SIZE);
 	char *temp;
@@ -139,7 +139,7 @@ static ssize_t write(struct file *file, const char __user *userbuf,
 		     size_t bytes, loff_t *off)
 {
 	struct bin_buffer *bb = file->private_data;
-	int size = file_inode(file)->i_size;
+	loff_t size = file_inode(file)->i_size;
 	loff_t offs = *off;
 	int count = min_t(size_t, bytes, PAGE_SIZE);
 	char *temp;

Re: [PATCH] sysfs/bin: Fix size handling overflow for bin_attribute

[Greg KH]

On Thu, Oct 10, 2013 at 06:03:55PM +1100, Benjamin Herrenschmidt wrote:
> While looking at the code, I noticed that bin_attribute read() and write()
> ops copy the inode size into an int for futher comparisons.
> 
> Some bin_attributes can be fairly large. For example, pci creates some for
> BARs set to the BAR size and giant BARs are around the corner, so this is
> going to break something somewhere eventually.
> 
> Let's use the right type.
> 
> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>

This file (fs/sysfs/bin.c) is no longer in linux-next as Tejun merged it
with the other file handling code in sysfs, to remove a bunch of
duplicated logic.

So, could you look at linux-next and see if this is still an issue there
or not?

thanks,

greg k-h

Re: [PATCH] sysfs/bin: Fix size handling overflow for bin_attribute

[Greg KH]

On Thu, Oct 10, 2013 at 10:38:47AM -0700, Greg KH wrote:
> On Thu, Oct 10, 2013 at 06:03:55PM +1100, Benjamin Herrenschmidt wrote:
> > While looking at the code, I noticed that bin_attribute read() and write()
> > ops copy the inode size into an int for futher comparisons.
> > 
> > Some bin_attributes can be fairly large. For example, pci creates some for
> > BARs set to the BAR size and giant BARs are around the corner, so this is
> > going to break something somewhere eventually.
> > 
> > Let's use the right type.
> > 
> > Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> 
> This file (fs/sysfs/bin.c) is no longer in linux-next as Tejun merged it
> with the other file handling code in sysfs, to remove a bunch of
> duplicated logic.
> 
> So, could you look at linux-next and see if this is still an issue there
> or not?

Ah, nevermind, I just looked, and it still is an issue there, I'll
hand-edit the patch to apply properly, thanks.

greg k-h

Re: [PATCH] sysfs/bin: Fix size handling overflow for bin_attribute

[Benjamin Herrenschmidt]

On Thu, 2013-10-10 at 10:40 -0700, Greg KH wrote:
> > This file (fs/sysfs/bin.c) is no longer in linux-next as Tejun merged it
> > with the other file handling code in sysfs, to remove a bunch of
> > duplicated logic.
> > 
> > So, could you look at linux-next and see if this is still an issue there
> > or not?
> 
> Ah, nevermind, I just looked, and it still is an issue there, I'll
> hand-edit the patch to apply properly, thanks.

Ah sorry, nice to see that Tejun cleaned that stuff up, it took me a while
to find my way around :-)

Cheers,
Ben.

Re: sysfs for my chips

[Greg KH]

On Thu, Oct 10, 2013 at 03:19:48PM +1100, Benjamin Herrenschmidt wrote:
> Hi Greg !
> 
> (random CC list of clueful people)
> 
> On some new powerpc platforms (non-hypervisor or rather linux is the
> hypervisor), I want to expose a bunch of stuff per "chip", the chips
> being currently the processor chips and the "centaurs" (think of them as
> the bottom half of the memory controllers).
> 
> Among other, I want a sysfs file in there to access "xscom" on the chip
> which is a sideband bus used for low level stuff (think jtag on steroid)
> which we can use, among others, for chip health monitoring, general
> debugging and diagnostics, etc...
> 
> I might add more such as VPD, model information, etc... later or at
> least a link to corresponding device-tree node.

So a mixture of things that traditionally have been in /proc/cpuinfo?

I've always wanted to see the cpuinfo files turn into sysfs files, so
that tools can parse them "properly" and not have to do different things
on different arches, like the proc/cpuinfo file is today (a mess).

> How do you suggest I expose that ? So far I've been thinking about
> something like
> 
> 	/sys/chips/{processor,centaur}/chip#/files
> 
> or to avoid namespace clashes
> 
> 	/sys/firmware/chips/{processor,centaur}/chip#/files
> 
> Or maybe just
> 
> 	/sys/firmware/chips/chip#/files
> 
> (the chip type can be inferred from the chip#, they use the same space
> at least as far my firmware exposes them to Linux)
> 
> (the actual access to xscom goes via firmware tho it makes *some* sense)
> 
> But I could instead create platform devices corresponding to the
> device-tree representation of each of those chips ... and have the
> platform devices contain the magic attributes. That's a bit more
> convoluted though.

We already create platform devices for the cpus in the system in
/sys/devices/system/, so can you just hang the attribute files off of
those platform devices?

And system devices seem like the correct thing for your "chips" that are
not cpus, right?

thanks,

greg k-h

Re: sysfs for my chips

[Benjamin Herrenschmidt]

On Thu, 2013-10-10 at 10:44 -0700, Greg KH wrote:
> On Thu, Oct 10, 2013 at 03:19:48PM +1100, Benjamin Herrenschmidt wrote:
> > Hi Greg !
> > 
> > (random CC list of clueful people)
> > 
> > On some new powerpc platforms (non-hypervisor or rather linux is the
> > hypervisor), I want to expose a bunch of stuff per "chip", the chips
> > being currently the processor chips and the "centaurs" (think of them as
> > the bottom half of the memory controllers).
> > 
> > Among other, I want a sysfs file in there to access "xscom" on the chip
> > which is a sideband bus used for low level stuff (think jtag on steroid)
> > which we can use, among others, for chip health monitoring, general
> > debugging and diagnostics, etc...
> > 
> > I might add more such as VPD, model information, etc... later or at
> > least a link to corresponding device-tree node.
> 
> So a mixture of things that traditionally have been in /proc/cpuinfo?

Not really no. /proc/cpuinfo is per-thread, and potentially might have a
core number and a few global things. Here I'm talking mostly about

  - A pseudo-file that gives access to a sideband HW bus to perform
read/write of system registers essentially, which is per physical
"chip" (chip in this context is processor chip, which can have lots of
cores/threads, but also can be our memory buffer chips).

  - The other stuff is much more than the kind of one-liner than one
finds in cpuinfo. For now I'm just slapping the usual "devspec" file
that contains the corresponding device-tree path and whatever additional
info can be retrieved from there. Things like VPDs for example can be
pretty big.

> I've always wanted to see the cpuinfo files turn into sysfs files, so
> that tools can parse them "properly" and not have to do different things
> on different arches, like the proc/cpuinfo file is today (a mess).
> 
> > How do you suggest I expose that ? So far I've been thinking about
> > something like
> > 
> > 	/sys/chips/{processor,centaur}/chip#/files
> > 
> > or to avoid namespace clashes
> > 
> > 	/sys/firmware/chips/{processor,centaur}/chip#/files
> > 
> > Or maybe just
> > 
> > 	/sys/firmware/chips/chip#/files
> > 
> > (the chip type can be inferred from the chip#, they use the same space
> > at least as far my firmware exposes them to Linux)
> > 
> > (the actual access to xscom goes via firmware tho it makes *some* sense)
> > 
> > But I could instead create platform devices corresponding to the
> > device-tree representation of each of those chips ... and have the
> > platform devices contain the magic attributes. That's a bit more
> > convoluted though.
> 
> We already create platform devices for the cpus in the system in
> /sys/devices/system/, so can you just hang the attribute files off of
> those platform devices?

This is not CPUs. CPUs are threads really. Even if they were cores, that
still wouldn't cut it. I'm looking at chips here and not all of them
actually processor chips. The SCOM bus address space is global to a
physical chip and allows to access various resources that are only
remotely related to the cores on it.

> And system devices seem like the correct thing for your "chips" that are
> not cpus, right?

Well, I could use system devices for everything I don't see much point,
since those things won't have a "driver" per-se, I don't actually need
any of the other infrastructure provided by a system device here. I've
posted a patch (which I should have labelled RFC) there with my current
experimental code:

http://patchwork.ozlabs.org/patch/282167/

Right now I do /sys/firmware/scom/<chip#>/{devspec,access} but of course
I can easily change that.

Cheers,
Ben.

> thanks,
> 
> greg k-h
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: sysfs for my chips

[Geert Uytterhoeven]

On Thu, Oct 10, 2013 at 10:01 PM, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
>> > Among other, I want a sysfs file in there to access "xscom" on the chip
>> > which is a sideband bus used for low level stuff (think jtag on steroid)
>> > which we can use, among others, for chip health monitoring, general
>> > debugging and diagnostics, etc...
>> >
>> > I might add more such as VPD, model information, etc... later or at
>> > least a link to corresponding device-tree node.
>>
>> So a mixture of things that traditionally have been in /proc/cpuinfo?
>
> Not really no. /proc/cpuinfo is per-thread, and potentially might have a
> core number and a few global things. Here I'm talking mostly about
>
>   - A pseudo-file that gives access to a sideband HW bus to perform
> read/write of system registers essentially, which is per physical
> "chip" (chip in this context is processor chip, which can have lots of
> cores/threads, but also can be our memory buffer chips).
>
>   - The other stuff is much more than the kind of one-liner than one
> finds in cpuinfo. For now I'm just slapping the usual "devspec" file
> that contains the corresponding device-tree path and whatever additional
> info can be retrieved from there. Things like VPDs for example can be
> pretty big.
>
>> I've always wanted to see the cpuinfo files turn into sysfs files, so
>> that tools can parse them "properly" and not have to do different things
>> on different arches, like the proc/cpuinfo file is today (a mess).
>>
>> > How do you suggest I expose that ? So far I've been thinking about
>> > something like
>> >
>> >     /sys/chips/{processor,centaur}/chip#/files
>> >
>> > or to avoid namespace clashes
>> >
>> >     /sys/firmware/chips/{processor,centaur}/chip#/files
>> >
>> > Or maybe just
>> >
>> >     /sys/firmware/chips/chip#/files
>> >
>> > (the chip type can be inferred from the chip#, they use the same space
>> > at least as far my firmware exposes them to Linux)
>> >
>> > (the actual access to xscom goes via firmware tho it makes *some* sense)
>> >
>> > But I could instead create platform devices corresponding to the
>> > device-tree representation of each of those chips ... and have the
>> > platform devices contain the magic attributes. That's a bit more
>> > convoluted though.
>>
>> We already create platform devices for the cpus in the system in
>> /sys/devices/system/, so can you just hang the attribute files off of
>> those platform devices?
>
> This is not CPUs. CPUs are threads really. Even if they were cores, that
> still wouldn't cut it. I'm looking at chips here and not all of them
> actually processor chips. The SCOM bus address space is global to a
> physical chip and allows to access various resources that are only
> remotely related to the cores on it.

What about a "bus" for the sideband bus? That allows to decouple it
from cores, and allows to support non-processor chips, too?

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: sysfs for my chips

[Benjamin Herrenschmidt]

On Thu, 2013-10-10 at 22:26 +0200, Geert Uytterhoeven wrote:
> > This is not CPUs. CPUs are threads really. Even if they were cores,
> that
> > still wouldn't cut it. I'm looking at chips here and not all of them
> > actually processor chips. The SCOM bus address space is global to a
> > physical chip and allows to access various resources that are only
> > remotely related to the cores on it.
> 
> What about a "bus" for the sideband bus? That allows to decouple it
> from cores, and allows to support non-processor chips, too?

Not sure what you mean .... create a linux bus type with devices on
it ? 

That's really overkill I think... doable though.

I think sysdev, despite my previous qualms, might be the best way... we
could have a "driver" in the form of the actual code that provide that
sideband bus access. Right now the platform registers a single global
"scom controller", we could make a sysdev instead with an instance per
"chip" I suppose...

Though there's still some kind of namespace issue, if I start creating
something like /sys/devices/system/chip/<id>/... but ...

That means doing something very platform specific in a location that is
very "generic" with a name that's likely to clash with something else
later on unless we start introducing a standard concept of "chip" with
associated properties but what do that really mean ? With things like
DCMs etc... the concept of "chip" is blurry at best.

Also what about other chips that do not participate in that xscom/scom
sideband mechanism ? like PCIe device chips, or other random stuff on
the mobo ?

I'm trying to solve a very specific problem here, which is to provide a
userspace scom access interface. I was trying to avoid a /dev/scom
because I would need 32-bit minors or do ioctls and because it seemed
simpler to just put it in sysfs somewhere but maybe that won't fly.

A last option is to put it in /sys/firmware/opal (OPAL is our firmware
name). Under there, I can put a lot of stuff that's essentially
firmware/platform specific. However our "scom" infrastructure is
somewhat generic and is used by at least one platform that isn't using
OPAL (though arguably that platform is dead: wsp).

Cheers,
Ben.

Re: sysfs for my chips

[Geert Uytterhoeven]

On Thu, Oct 10, 2013 at 11:30 PM, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> On Thu, 2013-10-10 at 22:26 +0200, Geert Uytterhoeven wrote:
>> > This is not CPUs. CPUs are threads really. Even if they were cores,
>> that
>> > still wouldn't cut it. I'm looking at chips here and not all of them
>> > actually processor chips. The SCOM bus address space is global to a
>> > physical chip and allows to access various resources that are only
>> > remotely related to the cores on it.
>>
>> What about a "bus" for the sideband bus? That allows to decouple it
>> from cores, and allows to support non-processor chips, too?
>
> Not sure what you mean .... create a linux bus type with devices on
> it ?

Yes, that's what I meant.

>From the nodes on that bus you can have symlinks in sysfs to e.g. the CPUs
in the rest of the sysfs tree.

It's a bit like smbus/i2c connecting various peripherals that may also be
connected in some other way (e.g. media devices).

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: sysfs for my chips

[Benjamin Herrenschmidt]

On Fri, 2013-10-11 at 08:52 +0200, Geert Uytterhoeven wrote:
> > Not sure what you mean .... create a linux bus type with devices on
> > it ?
> 
> Yes, that's what I meant.
> 
> >From the nodes on that bus you can have symlinks in sysfs to e.g. the CPUs
> in the rest of the sysfs tree.
> 
> It's a bit like smbus/i2c connecting various peripherals that may also be
> connected in some other way (e.g. media devices).

It's fairly overkill though ... we don't really plan to expose much of
these things to Linux anyway, it's mostly buried in firmware, I just
want scom access to userspace for debug/diagnostics.

I don't want it in debugfs however because we might want some "health
monitoring" daemon running in userspace that uses it to check some of
the built-in CE statistics etc... in the various chips and handle some
of the repair work. I prefer having that stuff in userspace (it's fairly
complex) than in firmware but it will get all the data it needs from the
device-tree, there's no point trying to reproduce the whole chip
hierarchy and sub hierarchy in Linux.

Cheers,
Ben.