Re: [PATCH] Enhancement of /proc/partitions output (2.5.1-pre5)

Re: [PATCH] Enhancement of /proc/partitions output (2.5.1-pre5)

[Andries.Brouwer]

    From: Anton Altaparmakov <aia21@cus.cam.ac.uk>

    Linus,

    Please consider below patch which adds the starting sector and number of
    sectors to /proc/partitions.

    It works fine here and I find having this information output can be very
    useful (especially when the values in the kernel don't match the values
    output by fdisk for example).

Of course this breaks all programs that use /proc/partitions,
like fdisk and mount. Every system that uses mount-by-label will be broken.

Andries

Re: [PATCH] Enhancement of /proc/partitions output (2.5.1-pre5)

[Andries.Brouwer]

    From: Anton Altaparmakov <aia21@cam.ac.uk>

    btw. You sent a patch to fsdevel some time ago converting parts of the 
    kernel to use bytes instead of 1024 byte blocks. Have you got an 
    updated/more complete version of that?

Yes. This summer I constructed the first dozen patches or so of
a path that would step by step change the present situation into one
without the `arrays', and with proper partition handling (namely not
buried down in the drivers), and with sizes as 64-bit byte amounts etc.
You can still find them on ftp.kernel.org.
Part of this was applied to 2.4.

Now that 2.5 has opened, I started (an hour ago) moving this stuff to 2.5.
(But will be abroad next week.)


    >    And I agree with Andreas the partition type would be useful
    >     in the display, too.
    >
    >I don't.
    >
    >This type is irrelevant. It would be very bad to make it available.
    >People might start using it, and that can only cause problems.
    >Moreover, usually there is no type, and in the future that I plan
    >there will never be a type. If there is a type, *fdisk will tell you.

    I am afraid I disagree. - Type is important when a partitioned device is 
    being worked on. LDM for example needs to know the types in order to make 
    sure not to take over a non-dynamic disk by mistake / to know that it is a 
    dynamic disk.

You illustrate what I say. It is vary bad to make types available,
since they do not exist. And ignorant people might start using them.

There are lots of partitioning systems in common use. And there is
no reason why LDM or MD RAID can sit on top of a DOS partition only.
Consequently, any dependence on types here is a bug.
Moreover, DOS type partitioning is dying. It cannot describe partitions
larger than 1 or 2 TB.

    .. I.e. where several different partitions have to be combined
    in various ways to give new devices? What are your thoughts
    on this? And do you have or are you aware of any code for
    these more advanced uses of partitioned devices?

No - but things will improve fully automatically. The MD code is
terrible (seen from a kdev_t point of view) and requires a lot of
restructuring.

Andries

Re: [PATCH] Enhancement of /proc/partitions output (2.5.1-pre5)

[Andries.Brouwer]

    From: Anton Altaparmakov <aia21@cam.ac.uk>

    > It will still break things.

    Sure. I anticipated it would break things but then again it is
    2.5.1-pre5 and not 2.4.x.

True. But this is the reporting side of the block device stuff.
And all is going to be changed. It is better to wait until the
new situation has been established.

    >Adding such stuff to the kernel would be especially unfortunate if,
    >as I did in my version of the block layer, the unit becomes byte
    >instead of sector. We would have to change again.

    So you are going to change it anyway at some point?

Who am I? I have plans, but other people may have other plans,
and it is not me who decides.

    We could just make the changes at the same time in that case?
    If it is in bytes even better. Just give start byte and length, 

But why not do it in user space?

# ./blockdev --report /dev/hda?
RO    RA   SSZ   BSZ   StartSec     Size    Device
rw     8   512  1024   16450560          2  /dev/hda1
rw     8   512  1024          0          0  /dev/hda2
rw     8   512  1024          0          0  /dev/hda3
rw     8   512  1024         63   16450497  /dev/hda4
rw     8   512  4096   16450623    4096512  /dev/hda5
rw     8   512  4096   20547198   13108977  /dev/hda6
rw     8   512  1024   33656238      80262  /dev/hda7
rw     8   512  1024     481950    4209030  /dev/hda8
rw     8   512  1024      64260     417690  /dev/hda9

(util-linux-2.11n)

    And I agree with Andreas the partition type would be useful
    in the display, too.

I don't.

This type is irrelevant. It would be very bad to make it available.
People might start using it, and that can only cause problems.
Moreover, usually there is no type, and in the future that I plan
there will never be a type. If there is a type, *fdisk will tell you.

[We all have plans, and some of these will actually be implemented.
In util-linux I already distribute for quite some time code that will
do partition setup. That means that no partition code is required
in the kernel. If 2.5 adds an initrd or so to the kernel, so that
Linux boots in its own ramdisk, then all partition reading code
can be thrown out of the kernel. That is one of my goals.
User space comes and tells the kernel: on device /dev/hda,
let hda7 be the interval of length 80262 sectors, starting at
sector 33656238. It is possible today, but not many people
use it. The plan is to make that the only way to get partitions.]

    If it really is only a list of available devices why show the size in 
    blocks at all?

History. It happened to be implemented like that.
There were some reasons, but not very strong ones.

Andries

Re: [PATCH] Enhancement of /proc/partitions output (2.5.1-pre5)

[Anton Altaparmakov]

At 22:43 01/12/01, Andries.Brouwer@cwi.nl wrote:
>     From: Anton Altaparmakov <aia21@cam.ac.uk>
>
>     > It will still break things.
>
>     Sure. I anticipated it would break things but then again it is
>     2.5.1-pre5 and not 2.4.x.
>
>True. But this is the reporting side of the block device stuff.
>And all is going to be changed. It is better to wait until the
>new situation has been established.

That is true.

>    >Adding such stuff to the kernel would be especially unfortunate if,
>     >as I did in my version of the block layer, the unit becomes byte
>     >instead of sector. We would have to change again.
>
>     So you are going to change it anyway at some point?
>
>Who am I? I have plans, but other people may have other plans,

Sorry, I meant: s/going/planning/

>and it is not me who decides.

True.

btw. You sent a patch to fsdevel some time ago converting parts of the 
kernel to use bytes instead of 1024 byte blocks. Have you got an 
updated/more complete version of that?

>    We could just make the changes at the same time in that case?
>     If it is in bytes even better. Just give start byte and length,
>
>But why not do it in user space?
>
># ./blockdev --report /dev/hda?
>RO    RA   SSZ   BSZ   StartSec     Size    Device
>rw     8   512  1024   16450560          2  /dev/hda1
>rw     8   512  1024          0          0  /dev/hda2
>rw     8   512  1024          0          0  /dev/hda3
>rw     8   512  1024         63   16450497  /dev/hda4
>rw     8   512  4096   16450623    4096512  /dev/hda5
>rw     8   512  4096   20547198   13108977  /dev/hda6
>rw     8   512  1024   33656238      80262  /dev/hda7
>rw     8   512  1024     481950    4209030  /dev/hda8
>rw     8   512  1024      64260     417690  /dev/hda9
>
>(util-linux-2.11n)

That's great! Thanks for the info.

>    And I agree with Andreas the partition type would be useful
>     in the display, too.
>
>I don't.
>
>This type is irrelevant. It would be very bad to make it available.
>People might start using it, and that can only cause problems.
>Moreover, usually there is no type, and in the future that I plan
>there will never be a type. If there is a type, *fdisk will tell you.

I am afraid I disagree. - Type is important when a partitioned device is 
being worked on. LDM for example needs to know the types in order to make 
sure not to take over a non-dynamic disk by mistake / to know that it is a 
dynamic disk.

MD RAID uses partition types for autodiscovery of raid partitions during boot.

And there might be other uses... For example IMHO it would make perfect 
sense to determine which fs to attempt to mount on a partition depending on 
its type.

>[We all have plans, and some of these will actually be implemented.
>In util-linux I already distribute for quite some time code that will
>do partition setup. That means that no partition code is required
>in the kernel. If 2.5 adds an initrd or so to the kernel, so that
>Linux boots in its own ramdisk, then all partition reading code
>can be thrown out of the kernel. That is one of my goals.
>User space comes and tells the kernel: on device /dev/hda,
>let hda7 be the interval of length 80262 sectors, starting at
>sector 33656238. It is possible today, but not many people
>use it. The plan is to make that the only way to get partitions.]

That sounds cool for normal partitions but how is handling of things like 
MD RAID and LDM going to be dealt with in that case?

I.e. where several different partitions have to be combined in various ways 
to give new devices? What are your thoughts on this? And do you have or are 
you aware of any code for these more advanced uses of partitioned devices?

(I will have a look at util-linux when I have some time.)

>    If it really is only a list of available devices why show the size in
>     blocks at all?
>
>History. It happened to be implemented like that.
>There were some reasons, but not very strong ones.

The usual answer in other words...

Best regards,

         Anton


-- 
   "I've not lost my mind. It's backed up on tape somewhere." - Unknown
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: [PATCH] Enhancement of /proc/partitions output (2.5.1-pre5)

[Andries.Brouwer]

    From: Andreas Dilger <adilger@turbolabs.com>

    > Please consider below patch which adds the starting sector and number of
    > sectors to /proc/partitions.

    Please do not accept as-is.  This breaks the format of /proc/partitions
    terribly, and all of the code that looks at it (fsck, mount, etc).

Indeed.

    Rather add the start_sect and nr_sects parameters _after_ the name
    parameter, and it will be "mostly" ok.

No. It will still break things.

    While we are at it, how about adding the partition type to the output?

The present /proc/partitions has minimal content.
Its only function is to report to user space which block devices
are known to the kernel. User space is unable to figure that out
on its own without an unreasonable amount of probing.

But once you know what devices exist, it is very easy for user space
to report all desired properties of these devices.
Do you want starting sector and size of /dev/hde4?

# hdparm -g /dev/hde4

/dev/hde4:
 geometry     = 70780/16/63, sectors = 1670760, start = 69673905

Adding such stuff to the kernel would be especially unfortunate if,
as I did in my version of the block layer, the unit becomes byte
instead of sector. We would have to change again.


Andries

Re: [PATCH] Enhancement of /proc/partitions output (2.5.1-pre5)

[Anton Altaparmakov]

At 11:47 01/12/01, Andries.Brouwer@cwi.nl wrote:
>     From: Andreas Dilger <adilger@turbolabs.com>
>
>     > Please consider below patch which adds the starting sector and 
> number of
>     > sectors to /proc/partitions.
>
>     Please do not accept as-is.  This breaks the format of /proc/partitions
>     terribly, and all of the code that looks at it (fsck, mount, etc).
>
>Indeed.
>
>     Rather add the start_sect and nr_sects parameters _after_ the name
>     parameter, and it will be "mostly" ok.
>
>No. It will still break things.

Sure. I anticipated it would break things but then again it is 2.5.1-pre5 
and not 2.4.x.

>    While we are at it, how about adding the partition type to the output?
>
>The present /proc/partitions has minimal content.
>Its only function is to report to user space which block devices
>are known to the kernel. User space is unable to figure that out
>on its own without an unreasonable amount of probing.
>
>But once you know what devices exist, it is very easy for user space
>to report all desired properties of these devices.
>Do you want starting sector and size of /dev/hde4?
>
># hdparm -g /dev/hde4

Ah, thanks! I didn't know that one.

>/dev/hde4:
>  geometry     = 70780/16/63, sectors = 1670760, start = 69673905
>
>Adding such stuff to the kernel would be especially unfortunate if,
>as I did in my version of the block layer, the unit becomes byte
>instead of sector. We would have to change again.

So you are going to change it anyway at some point? We could just make the 
changes at the same time in that case? If it is in bytes even better. Just 
give start byte and length, who cares about number of blocks anyway... (-; 
(Yes I can see there would be a problem with distinguishing the two but 
that can be solved.)

This is not a performance critical piece of code and having just the size 
of the partition is no way near as useful as having the start of it, too. 
And I agree with Andreas the partition type would be useful in the display, 
too.

If it really is only a list of available devices why show the size in 
blocks at all?

Utilities will adapt. That's what Documentation/Changes is for and this is 
a development kernel after all.

It's just the same as fairly recently my network refused to work because my 
network utilities disagreed with my kernel on how to communicate with each 
other...

Anton


-- 
   "I've not lost my mind. It's backed up on tape somewhere." - Unknown
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

Re: [PATCH] Enhancement of /proc/partitions output (2.5.1-pre5)

[Marcel J.E. Mol]

On Sat, Dec 01, 2001 at 11:47:34AM +0000, Andries.Brouwer@cwi.nl wrote:
>     From: Andreas Dilger <adilger@turbolabs.com>
> 
>     > Please consider below patch which adds the starting sector and number of
>     > sectors to /proc/partitions.
> 
>     Please do not accept as-is.  This breaks the format of /proc/partitions
>     terribly, and all of the code that looks at it (fsck, mount, etc).
> 
> Indeed.
> 
>     Rather add the start_sect and nr_sects parameters _after_ the name
>     parameter, and it will be "mostly" ok.
> 
> No. It will still break things.

Why, all fields are defined quite well. So usertools can easily get the info they
need based on 'column' number. If adding extra fields at the end breaks programs I
guess these tools should be fixed. 

RedHat kernels already have extended /proc/partitions: the sar patches...
It would ne nice to add these to the 2.5 kernel.

-Marcel
-- 
     ======--------         Marcel J.E. Mol                MESA Consulting B.V.
    =======---------        ph. +31-(0)6-54724868          P.O. Box 112
    =======---------        marcel@mesa.nl                 2630 AC  Nootdorp
__==== www.mesa.nl ---____U_n_i_x______I_n_t_e_r_n_e_t____ The Netherlands ____
 They couldn't think of a number,           Linux user 1148  --  counter.li.org
    so they gave me a name!  -- Rupert Hine  --  www.ruperthine.com

Re: [PATCH] Enhancement of /proc/partitions output (2.5.1-pre5)

[Bill Davidsen]

On Sat, 1 Dec 2001 Andries.Brouwer@cwi.nl wrote:

> The present /proc/partitions has minimal content.
> Its only function is to report to user space which block devices
> are known to the kernel. User space is unable to figure that out
> on its own without an unreasonable amount of probing.
> 
> But once you know what devices exist, it is very easy for user space
> to report all desired properties of these devices.
> Do you want starting sector and size of /dev/hde4?
> 
> # hdparm -g /dev/hde4
> 
> /dev/hde4:
>  geometry     = 70780/16/63, sectors = 1670760, start = 69673905

Really? Do you run hdparm setuid root or allow read access of hde? In any
case, allowing easy access from user space via hdparm trickes me as a
major security issue. The geometry is in /proc/ide/hd?/geometry, but the
other info is not readily available to other than root user in most
systems. I think his intention was to make it more generally available. I
don't see any reason to require root to see this information, and I think
reducing the need for root is a good thing for security.

It would be highly desirable to define new proc data as subject to
addition of more information per line at any time, and as long as the
fields are added at the end of the line a parser can easily be written not
to break. As you note, changing existing information presents other
problems and will break things, since /proc/partitions was not defines as
extensible.

I personally would have no problem adding this to 2.5, lots of things will
change, but I agree that a change to 2.4 is not practical as a retrofit.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: [PATCH] Enhancement of /proc/partitions output (2.5.1-pre5)

[RaúlNúñez de Arenas Coronado]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 394 bytes --]

    Hello Anton :)

>Please consider below patch which adds the starting sector and number of
>sectors to /proc/partitions.

    This is a *great* idea. I was wondering why this information is
not included by default :)))

    I have a somewhat 'special' partition scheme and that information
is vital for me. Thanks a lot for the patch. I hope that Marcello
accepts it for 2.4.17...

    Raúl

Re: [PATCH] Enhancement of /proc/partitions output (2.5.1-pre5)

[Marcus Meissner]

In article <E16A5XI-0005Lr-00@DervishD.viadomus.com> you wrote:
>     Hello Anton :)

>>Please consider below patch which adds the starting sector and number of
>>sectors to /proc/partitions.

>     This is a *great* idea. I was wondering why this information is
> not included by default :)))

>     I have a somewhat 'special' partition scheme and that information
> is vital for me. Thanks a lot for the patch. I hope that Marcello
> accepts it for 2.4.17...

Marcelo, don't apply to this to 2.4, a lot of userland applications rely
on it.

Ciao, Marcus

Re: [PATCH] Enhancement of /proc/partitions output (2.5.1-pre5)

[Anton Altaparmakov]

At 08:47 01/12/01, Marcus Meissner wrote:
>In article <E16A5XI-0005Lr-00@DervishD.viadomus.com> you wrote:
> >     Hello Anton :)
>
> >>Please consider below patch which adds the starting sector and number of
> >>sectors to /proc/partitions.
>
> >     This is a *great* idea. I was wondering why this information is
> > not included by default :)))
>
> >     I have a somewhat 'special' partition scheme and that information
> > is vital for me. Thanks a lot for the patch. I hope that Marcello
> > accepts it for 2.4.17...
>
>Marcelo, don't apply to this to 2.4, a lot of userland applications rely
>on it.

Absolutely, the subject does say 2.5.1-pre5. I never intended it for 2.4.x...

Anton


-- 
   "I've not lost my mind. It's backed up on tape somewhere." - Unknown
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

[PATCH] Enhancement of /proc/partitions output (2.5.1-pre5)

[Anton Altaparmakov]

Linus,

Please consider below patch which adds the starting sector and number of
sectors to /proc/partitions.

It works fine here and I find having this information output can be very
useful (especially when the values in the kernel don't match the values
output by fdisk for example).

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS maintainer / WWW: http://linux-ntfs.sf.net/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/


diff -u -urN linux-2.5.1-pre5-vanilla/drivers/block/genhd.c l251p5/drivers/block/genhd.c
--- linux-2.5.1-pre5-vanilla/drivers/block/genhd.c	Sat Dec  1 06:09:57 2001
+++ l251p5/drivers/block/genhd.c	Sat Dec  1 03:57:57 2001
@@ -149,7 +149,8 @@
 	char buf[64];
 	int len, n;
 
-	len = sprintf(page, "major minor  #blocks  name\n\n");
+	len = sprintf(page, "major minor   #blocks  start_sect   nr_sects  "
+			"name\n\n");
 	read_lock(&gendisk_lock);
 	for (gp = gendisk_head; gp; gp = gp->next) {
 		for (n = 0; n < (gp->nr_real << gp->minor_shift); n++) {
@@ -157,8 +158,10 @@
 				continue;
 
 			len += snprintf(page + len, 63,
-					"%4d  %4d %10d %s\n",
+					"%4d  %4d %10d  %10lu %10lu  %s\n",
 					gp->major, n, gp->sizes[n],
+					gp->part[n].start_sect,
+					gp->part[n].nr_sects,
 					disk_name(gp, n, buf));
 			if (len < offset)
 				offset -= len, len = 0;

Re: [PATCH] Enhancement of /proc/partitions output (2.5.1-pre5)

[Andreas Dilger]

On Dec 01, 2001  07:08 +0000, Anton Altaparmakov wrote:
> Linus,
> Please consider below patch which adds the starting sector and number of
> sectors to /proc/partitions.
> 
> It works fine here and I find having this information output can be very
> useful (especially when the values in the kernel don't match the values
> output by fdisk for example).

Please do not accept as-is.  This breaks the format of /proc/partitions
terribly, and all of the code that looks at it (fsck, mount, etc).  Rather
add the start_sect and nr_sects parameters _after_ the name parameter,
and it will be "mostly" ok.

While we are at it, how about adding the partition type to the output?
This is the only reason that I ever need to re-parse the on-disk crud
from DOS partition tables, and the kernel already handles it when it
is parsing the partition table.  For the "partitions" that don't have
DOS-style partition types, we can either just put a string (like LVM
or MD) which may be useful for a variety of other reasons.

(Purely hand-edited version of Anton's patch changing it to have the
 new fields at the end, may not apply, may not line up strings properly.
 Doesn't include suggested parition type field either).

On a similar note - Linus, will you accept a patch which fixes the block
device naming, to make it a per driver issue, rather than embedding it
into the core code?  I have one sitting around from way back, that has
been itching to get in.

Cheers, Andreas
============================================================================
diff -u -urN linux-2.5.1-pre5-vanilla/drivers/block/genhd.c l251p5/drivers/block/genhd.c
--- linux-2.5.1-pre5-vanilla/drivers/block/genhd.c	Sat Dec  1 06:09:57 2001
+++ l251p5/drivers/block/genhd.c	Sat Dec  1 03:57:57 2001
@@ -149,7 +149,8 @@
 	char buf[64];
 	int len, n;
 
-	len = sprintf(page, "major minor  #blocks  name\n\n");
+	len = sprintf(page, "major minor %9s %-16s %10s %10s\n",
+                     "#blocks", "name", "start_sect", "nr_sects");
 	read_lock(&gendisk_lock);
 	for (gp = gendisk_head; gp; gp = gp->next) {
 		for (n = 0; n < (gp->nr_real << gp->minor_shift); n++) {
@@ -157,8 +158,10 @@
 				continue;
 
 			len += snprintf(page + len, 63,
-					"%4d  %4d %10d %s\n",
+					"%4d  %4d %10d %-16s %10lu %10lu\n",
 					gp->major, n, gp->sizes[n],
-					disk_name(gp, n, buf));
+					disk_name(gp, n, buf),
+					gp->part[n].start_sect,
+					gp->part[n].nr_sects);
 			if (len < offset)
 				offset -= len, len = 0;
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/