[linux-lvm] Writing forward compatible applications using /proc

[linux-lvm] Writing forward compatible applications using /proc

[Jason Tackaberry]

I have some ideas for a comprehensive GUI disk administration tool, and
I am first thinking about some of the issues I'll face regarding
interfacing with controllers, disks, partitions, md, and LVM.

I am curious if it's a silly idea to write an application that relies on
the data in /proc?  Do the kernel developers attempt to keep the data
and format provided in /proc as stable as possible?  Or is it fair game
for trivial changes that could break applications that parse it?  Should
I use ioctl instead?

I don't mind having to release a new update to be compatible with a new
format in /proc every once and a while, but if it's constantly a moving
target I may want to look at alternatives.

I also have the same question with LVM related tools.  Is it sane to try
to wrap a GUI around lvm tools or is there some API approach that would
be better?

Cheers,
Jason.

Re: [linux-lvm] Writing forward compatible applications using /proc

[José Luis Domingo López]

On Saturday, 11 August 2001, at 23:57:18 -0400,
Jason Tackaberry wrote:

> [...]
> I am curious if it's a silly idea to write an application that relies on
> the data in /proc?  Do the kernel developers attempt to keep the data
> and format provided in /proc as stable as possible?  Or is it fair game
> for trivial changes that could break applications that parse it?  Should
> I use ioctl instead?
> 
As far as I know, there are plans to remove much of the thingss that
currently appear under /proc. It seems that only information about
processes currently running will remain under /proc.

Kernel parameters, system information, kernel variables and kernel
tunables will probably disappear and will be replaced by some sort of
user-space application that will allow query and modfication of kernel
values and tunables. Something similar to Solaris' ndd, AFAIK.

Greetings.

-- 
Jos� Luis Domingo L�pez
Linux Registered User #189436     Debian GNU/Linux Potato (P166 64 MB RAM)
 
jdomingo EN internautas PUNTO org  => � Spam ? Atente a las consecuencias
jdomingo AT internautas DOT   org  => Spam at your own risk

Re: [linux-lvm] Writing forward compatible applications using /proc

[Jason Tackaberry]

* Jos� Luis Domingo L�pez (jdomingo@internautas.org) [010812 07:57]:
> Kernel parameters, system information, kernel variables and kernel
> tunables will probably disappear and will be replaced by some sort of
> user-space application that will allow query and modfication of kernel
> values and tunables. Something similar to Solaris' ndd, AFAIK.

Any ideas when this will be implemented?  I suppose I could build it
around /proc, but just make it modular enough that I can switch to some
ndd-alike in the future.

Jason.

Re: [linux-lvm] Writing forward compatible applications using /proc

[Steven Lembark]

-- José Luis Domingo López <jdomingo@internautas.org>

> As far as I know, there are plans to remove much of the thingss that
> currently appear under /proc. It seems that only information about
> processes currently running will remain under /proc.
>
> Kernel parameters, system information, kernel variables and kernel
> tunables will probably disappear and will be replaced by some sort of
> user-space application that will allow query and modfication of kernel
> values and tunables. Something similar to Solaris' ndd, AFAIK.

Too bad.  Having a consistent interface for examining the values (vs
relying on a -- possibly out of date -- command line tool) is a nice
feature for system management.  Being able to 'cat blah' is uniform
across versions.  This is especially nice for checking scsi params,
etc, that effect LVM and friends.

Any ides why they the /proc information is being dropped?

--
Steven Lembark                                               2930 W. Palmer
Workhorse Computing                                       Chicago, IL 60647
                                                            +1 800 762 1582

Re: [linux-lvm] Writing forward compatible applications using /proc

[Eric M. Hopper]

[-- Attachment #1: Type: text/plain, Size: 1116 bytes --]

On 12 Aug 2001 13:54:02 +0000, José Luis Domingo López wrote:
> As far as I know, there are plans to remove much of the thingss that
> currently appear under /proc. It seems that only information about
> processes currently running will remain under /proc.
> 
> Kernel parameters, system information, kernel variables and kernel
> tunables will probably disappear and will be replaced by some sort of
> user-space application that will allow query and modfication of kernel
> values and tunables. Something similar to Solaris' ndd, AFAIK.

This is sad.  The whole point of the filesystem under Unix is to provide
a unified namespace for all the system objects you might want to talk
about.

I can see adding a new filesystem for kernel parameters that's
completely seperate from /proc, but getting rid of the ability to have
access to them directly through the filesystem is a bad idea IMHO.

:-(,
-- 
The best we can hope for concerning the people at large is that they
be properly armed.  -- Alexander Hamilton
-- Eric Hopper (hopper@omnifarious.org 
http://www.omnifarious.org/~hopper) --

[-- Attachment #2: Type: application/pgp-signature, Size: 228 bytes --]

Re: [linux-lvm] Writing forward compatible applications using /proc

[José Luis Domingo López]

On Sunday, 12 August 2001, at 11:38:46 -0400,
Jason Tackaberry wrote:

> * Jos� Luis Domingo L�pez (jdomingo@internautas.org) [010812 07:57]:
> > Kernel parameters, system information, kernel variables and kernel
> > tunables will probably disappear and will be replaced by some sort of
> > user-space application that will allow query and modfication of kernel
> > values and tunables. Something similar to Solaris' ndd, AFAIK.
> 
> Any ideas when this will be implemented?  I suppose I could build it
> around /proc, but just make it modular enough that I can switch to some
> ndd-alike in the future.
> 
Don't know about exact timeframes, but it seems one of the things on the
linux kernel 2.5.x TODO list. Maybe its time to ask about it on the linux
kernel mailing list :)

Greetings.

-- 
Jos� Luis Domingo L�pez
Linux Registered User #189436     Debian GNU/Linux Potato (P166 64 MB RAM)
 
jdomingo EN internautas PUNTO org  => � Spam ? Atente a las consecuencias
jdomingo AT internautas DOT   org  => Spam at your own risk

Re: [linux-lvm] Writing forward compatible applications using /proc

[Joe Thornber]

On Sat, Aug 11, 2001 at 11:57:18PM -0400, Jason Tackaberry wrote:
> I have some ideas for a comprehensive GUI disk administration tool, and
> I am first thinking about some of the issues I'll face regarding
> interfacing with controllers, disks, partitions, md, and LVM.
> 
> I am curious if it's a silly idea to write an application that relies on
> the data in /proc?  Do the kernel developers attempt to keep the data
> and format provided in /proc as stable as possible?  Or is it fair game
> for trivial changes that could break applications that parse it?  Should
> I use ioctl instead?

We won't change the format of the /proc data for the 1.0 series of LVM
(including 0.9.1 betas).  I do *not* suggest you use the ioctls directly, at
some point the ioctl interface will change.

> I don't mind having to release a new update to be compatible with a new
> format in /proc every once and a while, but if it's constantly a moving
> target I may want to look at alternatives.

You should also be able to get all the information you need from the tools.

> 
> I also have the same question with LVM related tools.  Is it sane to try
> to wrap a GUI around lvm tools or is there some API approach that would
> be better?

I would much rather see people wrapping the tools than using liblvm, in fact
liblvm will probably disappear in the future.

It would be really good to see a comprehensive LVM gui tool.  You
should certainly dig up Jorg de Jong's Java based gui to get ideas
(http://www.xs4all.nl/~mmj/lvm/), from the screen shots it looks nice.
Are you planning to implement in C and gtk ? or use some higher level
language ?

- Joe

Re: [linux-lvm] Writing forward compatible applications using /proc

[Steven Lembark]

> You should also be able to get all the information you need from the
> tools.

Problem with this is (a) having for fork/exec the tool at each
iteration and (b) having to capture/parse stdout/stderr and
check exit/signal status for each call.  Just the error checking
and validating stderr vs. stdout can double the code size for
getting data.

Another way out (which the /proc file system was intended to
avoid) is to put an add'l device file in the VG directory
(e.g., "./status") which delivers a struct or block text
with the same info as /proc has now.  Net result is that a
simple read or fgets will pull in the necessary info w/o
all the extra error checking.


--
Steven Lembark                                               2930 W. Palmer
Workhorse Computing                                       Chicago, IL 60647
                                                            +1 800 762 1582

Re: [linux-lvm] Writing forward compatible applications using /proc

[Joe Thornber]

On Sun, Aug 12, 2001 at 02:13:48PM -0500, Steven Lembark wrote:
> 
> 
> 
> > You should also be able to get all the information you need from the
> > tools.
> 
> Problem with this is (a) having for fork/exec the tool at each
> iteration

Look at the new tool in the experimental branch (which will turn into
2.0 at some point).  We've put in an interactive mode so you can run
the tool once, piping commands to it.

> and (b) having to capture/parse stdout/stderr and
> check exit/signal status for each call.  Just the error checking
> and validating stderr vs. stdout can double the code size for
> getting data.

Not sure I agree with this, the error checking code can be reused for
all your different LVM queries.

> Another way out (which the /proc file system was intended to
> avoid) is to put an add'l device file in the VG directory
> (e.g., "./status") which delivers a struct or block text
> with the same info as /proc has now.  Net result is that a
> simple read or fgets will pull in the necessary info w/o
> all the extra error checking.

This is unlikely to happen, it goes in /proc or not at all.  I do not
understand why people want to remove this sort of information from
/proc.

- Joe

Re: [linux-lvm] Writing forward compatible applications using /proc

[Jason Tackaberry]

> You should also be able to get all the information you need from the tools.

The problem I see with this is if the output generated by the tools
changes it could potentially break the GUI tool.  I guess no matter what
I do I'm picking a moving target.

> It would be really good to see a comprehensive LVM gui tool.  You

Actually my idea is for a comprehensive _disk_ administration tool, of
which LVM will be a part.  So you can, for example, query your
controllers, partition your disks, setup a RAID5 configuration with LVM
on top, and even tune and manage supported file systems.

My plan is to model it loosely from Veritas storage administrator.

> Are you planning to implement in C and gtk ? or use some higher level
> language ?

I've always wanted to learn Qt, so I'll likely use Qt, and build it in
such a way that the GUI is abstracted and can be ported to a different
toolkit.  Of course I'd distribute static binaries, since this sort of
thing should be able to be run on a server, many of which do not have X
libraries installed.

Cheers,
Jason.

Re: [linux-lvm] Writing forward compatible applications using /proc

[Ragnar Kjørstad]

On Sun, Aug 12, 2001 at 08:07:02PM +0100, Joe Thornber wrote:
> I would much rather see people wrapping the tools than using liblvm, in fact
> liblvm will probably disappear in the future.

Why will liblvm disappear? To me using a library interface seems much
nicer than wrapping applications.


-- 
Ragnar Kjorstad
Big Storage

Re: [linux-lvm] Writing forward compatible applications using /proc

[toon]

On Sun, Aug 12, 2001 at 01:54:02PM +0000, Jos� Luis Domingo L�pez wrote:
> On Saturday, 11 August 2001, at 23:57:18 -0400, Jason Tackaberry wrote:
> 
> > [...]
> > I am curious if it's a silly idea to write an application that relies on
> > the data in /proc?  Do the kernel developers attempt to keep the data
> > and format provided in /proc as stable as possible?  Or is it fair game
> > for trivial changes that could break applications that parse it?  Should
> > I use ioctl instead?
> > 
> As far as I know, there are plans to remove much of the thingss that
> currently appear under /proc. It seems that only information about
> processes currently running will remain under /proc.
> 
> Kernel parameters, system information, kernel variables and kernel
> tunables will probably disappear and will be replaced by some sort of
> user-space application that will allow query and modfication of kernel
> values and tunables. Something similar to Solaris' ndd, AFAIK.

Are you sure about the /proc filesystem disappearing?
Linus Torvalds likes it much more than ioctl's, as far as I know.
Besides, the story about the user-space application doesn't sound
very realistically to me.

I advise you to ask the question about the future of the /proc
filesystem on the linux-kernel mailing list.

Regards,
Toon.

Re: [linux-lvm] Writing forward compatible applications using /proc

[Ragnar Kjørstad]

On Sun, Aug 12, 2001 at 10:21:39PM +0000, Jos� Luis Domingo L�pez wrote:
> > On 12 Aug 2001 13:54:02 +0000, Jos� Luis Domingo L�pez wrote:
> > > As far as I know, there are plans to remove much of the thingss that
> > > currently appear under /proc. It seems that only information about
> > > processes currently running will remain under /proc.
> > > 
> >
> Check this linux kernel mailing list for the (still scarce) details about
> "removing" kernel tunables from /proc:
> http://uwsg.iu.edu/hypermail/linux/kernel/0105.3/1439.html
> 
> To be precise, dicussion about modifying existing /proc starts here:
> http://uwsg.iu.edu/hypermail/linux/kernel/0105.3/1469.html
> 
> There were not much discussion about the subject, but it seems that in
> 2.5.x /proc will just hold information about running processes. The rest
> is still to be defined :)

If I'm not mistaken, the idea is to move everything not process related
from procfs to a dedicated sysfs. 

I don't think it has ever been proposed to not have a fs interface to
the data, but some changes have been proposed:
* Some standard way of parsing the files (e.g. xml)
* One file pr value, rather than today's system where you have to parse 
  the files
* Changing the directory layout.


-- 
Ragnar Kjorstad
Big Storage

Re: [linux-lvm] Writing forward compatible applications using /proc

[José Luis Domingo López]

On Sunday, 12 August 2001, at 12:10:31 -0500,
Eric M. Hopper wrote:

> On 12 Aug 2001 13:54:02 +0000, Jos� Luis Domingo L�pez wrote:
> > As far as I know, there are plans to remove much of the thingss that
> > currently appear under /proc. It seems that only information about
> > processes currently running will remain under /proc.
> > 
>
Check this linux kernel mailing list for the (still scarce) details about
"removing" kernel tunables from /proc:
http://uwsg.iu.edu/hypermail/linux/kernel/0105.3/1439.html

To be precise, dicussion about modifying existing /proc starts here:
http://uwsg.iu.edu/hypermail/linux/kernel/0105.3/1469.html

There were not much discussion about the subject, but it seems that in
2.5.x /proc will just hold information about running processes. The rest
is still to be defined :)

Regards.

--
Jos� Luis Domingo L�pez
Linux Registered User #189436     Debian GNU/Linux Potato (P166 64 MB RAM)
 
jdomingo EN internautas PUNTO org  => � Spam ? Atente a las consecuencias
jdomingo AT internautas DOT   org  => Spam at your own risk

Re: [linux-lvm] Writing forward compatible applications using /proc

[Wichert Akkerman]

On 12 Aug 2001 13:54:02 +0000, Jos� Luis Domingo L�pez wrote:
> As far as I know, there are plans to remove much of the thingss that
> currently appear under /proc. It seems that only information about
> processes currently running will remain under /proc.

The name /proc already suggests it is a filesystem for processes,
not kernel configuration.

> Kernel parameters, system information, kernel variables and kernel
> tunables will probably disappear and will be replaced by some sort of
> user-space application that will allow query and modfication of kernel
> values and tunables. Something similar to Solaris' ndd, AFAIK.

From what I remember the idea is to move all the no-process stuff
from /proc to (an)other filesystem(s), /kernel or so. 

Wichert.

-- 
  _________________________________________________________________
 /       Nothing is fool-proof to a sufficiently talented fool     \
| wichert@wiggy.net                   http://www.liacs.nl/~wichert/ |
| 1024D/2FA3BC2D 576E 100B 518D 2F16 36B0  2805 3CB8 9250 2FA3 BC2D |

Re: [linux-lvm] Writing forward compatible applications using /proc

[Paul Jakma]

On Sun, 12 Aug 2001, Steven Lembark wrote:

> Any ides why they the /proc information is being dropped?

it isn't being dropped afaict...

if anything 2.5 will have /more/ things as files.

regards,
-- 
Paul Jakma	paul@clubi.ie	paul@jakma.org	Key ID: 64A2FF6A
Fortune:
Let's organize this thing and take all the fun out of it.

Re: [linux-lvm] Writing forward compatible applications using /proc

[Joe Thornber]

On Sun, Aug 12, 2001 at 10:16:50PM +0200, Ragnar Kj?rstad wrote:
> On Sun, Aug 12, 2001 at 08:07:02PM +0100, Joe Thornber wrote:
> > I would much rather see people wrapping the tools than using liblvm, in fact
> > liblvm will probably disappear in the future.
> 
> Why will liblvm disappear? To me using a library interface seems much
> nicer than wrapping applications.

Because it's means there's yet another interface (along with the
command line tools, and ioctl's) to constrain any implementation
changes.  Command line tool interface will not change.  liblvm (if it
still existed as a shared library) will change drastically between 1.0
and 2.0 - not least because it in turn reflects the driver ioctl
interface.

The experimental branch has a single lvm tool, with liblvm statically
linked into it.  There's no need for the outside world to know about
liblvm.

It may be a good idea for Sistina to produce a tiny 'wrap the tools'
library for use by gui implementors.

- Joe

Re: [linux-lvm] Writing forward compatible applications using /proc

[Terje Kvernes]

Jason Tackaberry <tack@linux.com> writes:

> I have some ideas for a comprehensive GUI disk administration tool,
> and I am first thinking about some of the issues I'll face regarding
> interfacing with controllers, disks, partitions, md, and LVM.

  cool. I've written a perl module to interface LVM via /proc. if
  you're interested I'll repost some of the code here. it's a backend,
  totally seperate from any representation of the data. 
 
> I am curious if it's a silly idea to write an application that
> relies on the data in /proc? 

  not really. some of the tools need special permissions to be run,
  and I _might_ want to have a quick peek at my LVM status without
  being root. besides, it's a lot cleaner (IMHO) than doing piping
  things to a process and catching the output, like the suggestion is
  atm. 

> I don't mind having to release a new update to be compatible with a
> new format in /proc every once and a while, but if it's constantly a
> moving target I may want to look at alternatives.

  you'll have to keep somethings in mind, before 0.9.1 beta 8 the PV's
  listed in /proc only had their basename listed. using devfs this
  could break badly, as you'd get easily get two identical
  identifiers. so you'd still have to check what data you're getting
  and see what you can, or more importantly can't, do.
 
> I also have the same question with LVM related tools. Is it sane to
> try to wrap a GUI around lvm tools or is there some API approach
> that would be better?

  I've wrapped the tools to do the dirtywork. as in, I use create a
  lvextend-command when someone does a "$lv->size('+1G')". 

-- 
Terje

Re: [linux-lvm] Writing forward compatible applications using /proc

[Wichert Akkerman]

Previously Joe Thornber wrote:
> It may be a good idea for Sistina to produce a tiny 'wrap the tools'
> library for use by gui implementors.

With python and perl bindings preferably.

Wichert.

-- 
  _________________________________________________________________
 /       Nothing is fool-proof to a sufficiently talented fool     \
| wichert@wiggy.net                   http://www.liacs.nl/~wichert/ |
| 1024D/2FA3BC2D 576E 100B 518D 2F16 36B0  2805 3CB8 9250 2FA3 BC2D |

Re: [linux-lvm] Writing forward compatible applications using /proc

[Terje Kvernes]

Wichert Akkerman <wichert@cistron.nl> writes:

> Previously Joe Thornber wrote:
>
> > It may be a good idea for Sistina to produce a tiny 'wrap the
> > tools' library for use by gui implementors.
> 
> With python and perl bindings preferably.

  *raises a hand* 

  I think I'll see about getting the perl-version into a releaseable
  state. ie: add documentation. of sorts. :)

-- 
Terje

Re: [linux-lvm] Writing forward compatible applications using /proc

[Alasdair G Kergon]

On Sun, Aug 12, 2001 at 10:16:50PM +0200, Ragnar Kj?rstad wrote:
> On Sun, Aug 12, 2001 at 08:07:02PM +0100, Joe Thornber wrote:
> > I would much rather see people wrapping the tools than using liblvm, in fact
> > liblvm will probably disappear in the future.
> Why will liblvm disappear? To me using a library interface seems much
> nicer than wrapping applications.

Let's put this another way.

In the 'experimental' branch each tool has been converted into a
function and they have all been linked into a single 'lvm'
binary.  This binary takes lvm commands from standard input
and processes them.  Or it can be invoked with argv[0] set to
the name of an existing tool (eg 'ln lvm pvcreate') and then it 
will behave as that tool.  There are no plans to change existing 
command line arguments etc.  However, in the interests of
achieving consistency between the tools and making it easy to
handle their output reliably, there are likely to be minor
changes to the output formats, error codes, signal-handling etc.  

The *existing* liblvm is very dependent on internal LVM structure.
Significant internal structural changes are on the cards, so the
existing liblvm is likely to disappear.  

My thinking is that, for future maintainability, the interface into
lvm should be at the level of the command-line tools, rather than
the level of the existing liblvm.  Where there are functions in
liblvm that people need to call independently (any examples?), then
we should either adapt existing tools or create new ones to provide
this.

I envisage any 'wrapper' application (eg. GUI) as simply opening a
pipe/socket to 'lvm --condensed-output' (say) and passing individual
lvm commands in and getting responses back in a consistent,
condensed text format.  Alternatively it could call the tools
through a new library - but the functionality this would expose
would be the same as that provided by the command line tools.

Alasdair
-- 
agk@uk.sistina.com

Re: [linux-lvm] Writing forward compatible applications using /proc

[Jason Tackaberry]

* Alasdair G Kergon (agk@uk.sistina.com) [010813 11:17]:
> I envisage any 'wrapper' application (eg. GUI) as simply opening a
> pipe/socket to 'lvm --condensed-output' (say) and passing individual
> lvm commands in and getting responses back in a consistent,

FWIW, this is precisely how Veritas Volume Manager's visual
administrator (vxva) works.  

It doesn't matter much to me how I interface with LVM and other
components, as long as there is a sane way to handle errors, and I'm
able to get all the same information vs. using ioctls.

So the general consensus seems to be to interface through the LVM tools.
That takes care of the lvm -- now the same question needs to be answered
for md and controller/disk/partition info.  This may be the wrong list.
:)

Cheers,
Jason.

Re: [linux-lvm] Writing forward compatible applications using /proc

[Michael Tokarev]

Alasdair G Kergon wrote:
> 
[]
> In the 'experimental' branch each tool has been converted into a
> function and they have all been linked into a single 'lvm'
> binary.  This binary takes lvm commands from standard input
> and processes them.  Or it can be invoked with argv[0] set to
> the name of an existing tool (eg 'ln lvm pvcreate') and then it
> will behave as that tool.  There are no plans to change existing
> command line arguments etc.  However, in the interests of
> achieving consistency between the tools and making it easy to
> handle their output reliably, there are likely to be minor
> changes to the output formats, error codes, signal-handling etc.

Wow, great!  No more tons of tiny executables!  Wow!  Looks
very good.  Just like `cvs' now -- "cvs co", "cvs add" etc...

Well, ok, but one little question/suggestion:  why not have
*some* tools instead of one (a step back? no): pvm for physycal
volumes, lvg for volume groups and lvm for logical volumes?
This seems to be logical...

Anyway, this is exactly a direction I always wanted to ask and/or
point to, but I remember some words somewhere on Sistina?  Or
old LVM homepage (where it was?) ? that such mergeing will not
happen because of something.  (It may be even lvm FAQ or HOWTO).
Unfortunately, I don't remember reasons behind that.

Regards,
 Michael.

Re: [linux-lvm] Writing forward compatible applications using /proc

[Wichert Akkerman]

Previously Michael Tokarev wrote:
> Wow, great!  No more tons of tiny executables!  Wow!  Looks
> very good.  Just like `cvs' now -- "cvs co", "cvs add" etc...

Compare with: wow! instead of a few simple modular applications we
now get a single one which tries to do everything. I'm not convinced
this is a step forward, the UNIX philosophy of having lots of small
single-purpose tools is a good one.

Wichert.

-- 
  _________________________________________________________________
 /       Nothing is fool-proof to a sufficiently talented fool     \
| wichert@wiggy.net                   http://www.liacs.nl/~wichert/ |
| 1024D/2FA3BC2D 576E 100B 518D 2F16 36B0  2805 3CB8 9250 2FA3 BC2D |

Re: [linux-lvm] Writing forward compatible applications using /proc

[Goetz Bock]

[-- Attachment #1: Type: text/plain, Size: 888 bytes --]


On Mon, Aug 13 '01 at 21:49, Wichert Akkerman wrote:
> Compare with: wow! instead of a few simple modular applications we
> now get a single one which tries to do everything. I'm not convinced
> this is a step forward, the UNIX philosophy of having lots of small
> single-purpose tools is a good one.
(I did not check the cvs or how it is done)
but if they did it right, they did it the same way busybox works:
independent code file that could work as independent executables (if
wrapped with some glue logic) but which work from one main executeable.
While it keeps the mantainability of lots of small tools, it is actually
better to manage for the user/admin (just one file), and the executable
might even be smaller (better for bootdisks or simmilar low diskspace
situations)
-- 
Goetz Bock                                              IT Consultant
Dipl.-Inf. Univ.

[-- Attachment #2: Type: application/pgp-signature, Size: 232 bytes --]

Re: [linux-lvm] Writing forward compatible applications using /proc

[Joe Thornber]

On Mon, Aug 13, 2001 at 09:49:43PM +0200, Wichert Akkerman wrote:
> Previously Michael Tokarev wrote:
> > Wow, great!  No more tons of tiny executables!  Wow!  Looks
> > very good.  Just like `cvs' now -- "cvs co", "cvs add" etc...
> 
> Compare with: wow! instead of a few simple modular applications we
> now get a single one which tries to do everything. I'm not convinced
> this is a step forward, the UNIX philosophy of having lots of small
> single-purpose tools is a good one.

Then split them out again, we'll make sure this can be configured at
the makefile level.  In particular we want to make sure that people
can make executables with the minimal set of tools necessary for
rescue disks etc.  If you want seperate binaries, you shall have them
:)

- Joe

Re: [linux-lvm] Writing forward compatible applications using /proc

[Joe Thornber]

On Mon, Aug 13, 2001 at 10:18:37PM +0200, Goetz Bock wrote:
> 
> On Mon, Aug 13 '01 at 21:49, Wichert Akkerman wrote:
> > Compare with: wow! instead of a few simple modular applications we
> > now get a single one which tries to do everything. I'm not convinced
> > this is a step forward, the UNIX philosophy of having lots of small
> > single-purpose tools is a good one.
> (I did not check the cvs or how it is done)
> but if they did it right, they did it the same way busybox works:
> independent code file that could work as independent executables (if
> wrapped with some glue logic) but which work from one main executeable.
> While it keeps the mantainability of lots of small tools, it is actually
> better to manage for the user/admin (just one file), and the executable
> might even be smaller (better for bootdisks or simmilar low diskspace
> situations)

Exactly.

experimental
wc -l tools/*.[hc]
9441

beta8
wc -l tools/*.[hc]
13566

and there's a lot left to do.

- Joe

Re: [linux-lvm] Writing forward compatible applications using /proc

[Ragnar Kjørstad]

On Mon, Aug 13, 2001 at 10:08:58AM +0100, Joe Thornber wrote:
> > Why will liblvm disappear? To me using a library interface seems much
> > nicer than wrapping applications.
> 
> Because it's means there's yet another interface (along with the
> command line tools, and ioctl's) to constrain any implementation
> changes.  Command line tool interface will not change.  liblvm (if it
> still existed as a shared library) will change drastically between 1.0
> and 2.0 - not least because it in turn reflects the driver ioctl
> interface.

I'm not complaining if it's changing, but if it's going away, that's bad
:(

> The experimental branch has a single lvm tool, with liblvm statically
> linked into it.  There's no need for the outside world to know about
> liblvm.
> 
> It may be a good idea for Sistina to produce a tiny 'wrap the tools'
> library for use by gui implementors.

yach! An uncomplete list of uglyness:
* Errorhandling
* fork/exec
* having to format input arguments as strings (and quote them)
* having to parse output data from strings 


-- 
Ragnar Kjorstad
Big Storage

Re: [linux-lvm] Writing forward compatible applications using /proc

[Nathan Scott]

hi,

On Mon, Aug 13, 2001 at 10:08:58AM +0100, Joe Thornber wrote:
> On Sun, Aug 12, 2001 at 10:16:50PM +0200, Ragnar Kj?rstad wrote:
> > On Sun, Aug 12, 2001 at 08:07:02PM +0100, Joe Thornber wrote:
> > > I would much rather see people wrapping the tools than using liblvm, in fact
> > > liblvm will probably disappear in the future.
> > 
> > Why will liblvm disappear? To me using a library interface seems much
> > nicer than wrapping applications.
> 
> Because it's means there's yet another interface (along with the
> command line tools, and ioctl's) to constrain any implementation
> changes.  Command line tool interface will not change.  liblvm (if it
> still existed as a shared library) will change drastically between 1.0
> and 2.0 - not least because it in turn reflects the driver ioctl
> interface.
> 
> The experimental branch has a single lvm tool, with liblvm statically
> linked into it.  There's no need for the outside world to know about
> liblvm.
> 

Just as another data point - the XFS mkfs program uses liblvm to
figure out the stripe width and stripe size of the underlying
volume, and then uses that knowledge to improve the layout of the
various ondisk data structures for the filesystem, if its sitting
on top of a striped LVM volume.

As a reference, the code in the XFS cvs tree on oss.sgi.com in
cmd/xfsprogs/libdisk/lvm.c contains this logic.

The equivalent md code (in that same directory) uses an ioctl for
this function and is much cleaner - I wouldn't mind liblvm going
away provided this information can be extracted just as easily as
the md case.

If you do decide to make liblvm go away, please make sure you let
us know the correct way to extract this information without using
liblvm.  My personal preference would be an ioctl interface like
the md code provides.

cheers.

-- 
Nathan

Re: [linux-lvm] Writing forward compatible applications using /proc

[Mark van Walraven]

On Sun, Aug 12, 2001 at 08:07:02PM +0100, Joe Thornber wrote:
> I would much rather see people wrapping the tools than using liblvm, in fact
> liblvm will probably disappear in the future.

I think it is a bad idea to eliminte liblvm.  Shared libraries are much
easier use to use from programs.  For example, it is much easier in a perl
script to use zlib than to run gzip as a subprocess in a bulletproof way.

Regards,

Mark.

Re: [linux-lvm] Writing forward compatible applications using /proc

[Joe Thornber]

On Tue, Aug 14, 2001 at 03:04:18PM +1200, Mark van Walraven wrote:
> On Sun, Aug 12, 2001 at 08:07:02PM +0100, Joe Thornber wrote:
> > I would much rather see people wrapping the tools than using liblvm, in fact
> > liblvm will probably disappear in the future.
> 
> I think it is a bad idea to eliminte liblvm.  Shared libraries are much
> easier use to use from programs.  For example, it is much easier in a perl
> script to use zlib than to run gzip as a subprocess in a bulletproof way.

I'm just worried about the interface to liblvm changing and breaking
tools.  Let's wait a bit to see where experimental goes, and then see
how much the library had to change.

- Joe

Re: [linux-lvm] Writing forward compatible applications using /proc

[Heinz J . Mauelshagen]

On Tue, Aug 14, 2001 at 10:31:12AM +1000, Nathan Scott wrote:
> hi,
> 
> On Mon, Aug 13, 2001 at 10:08:58AM +0100, Joe Thornber wrote:
> > On Sun, Aug 12, 2001 at 10:16:50PM +0200, Ragnar Kj?rstad wrote:
> > > On Sun, Aug 12, 2001 at 08:07:02PM +0100, Joe Thornber wrote:
> > > > I would much rather see people wrapping the tools than using liblvm, in fact
> > > > liblvm will probably disappear in the future.
> > > 
> > > Why will liblvm disappear? To me using a library interface seems much
> > > nicer than wrapping applications.
> > 
> > Because it's means there's yet another interface (along with the
> > command line tools, and ioctl's) to constrain any implementation
> > changes.  Command line tool interface will not change.  liblvm (if it
> > still existed as a shared library) will change drastically between 1.0
> > and 2.0 - not least because it in turn reflects the driver ioctl
> > interface.
> > 
> > The experimental branch has a single lvm tool, with liblvm statically
> > linked into it.  There's no need for the outside world to know about
> > liblvm.
> > 
> 
> Just as another data point - the XFS mkfs program uses liblvm to
> figure out the stripe width and stripe size of the underlying
> volume, and then uses that knowledge to improve the layout of the
> various ondisk data structures for the filesystem, if its sitting
> on top of a striped LVM volume.

Nathan,

as Joe et al. stated the liblvm API is subject to major changes in the future.
The command line interface is *not*.

Could you therefore please change mkfs to wrap lvdisplay(8) in order to
retrieve the onformation?

> 
> As a reference, the code in the XFS cvs tree on oss.sgi.com in
> cmd/xfsprogs/libdisk/lvm.c contains this logic.
> 
> The equivalent md code (in that same directory) uses an ioctl for
> this function and is much cleaner - I wouldn't mind liblvm going
> away provided this information can be extracted just as easily as
> the md case.
> 
> If you do decide to make liblvm go away, please make sure you let
> us know the correct way to extract this information without using
> liblvm.  My personal preference would be an ioctl interface like
> the md code provides.

As stated before: please don't use the ioctl interface directly because
it is subject to change!

We'll make sure, that the CLI stays compatible.


WRT the way how to retrieve the information using the CLI you could
go with a tiny perl script, check the exit code to be zero and scanf(3)
stdout (#stripes stripesize) in mkfs:

#!/usr/bin/perl
scalar @ARGV != 1 && exit 1;
open ( PIPE, "lvdisplay $ARGV[0] |") || exit 2;
$stripes=0;
foreach (<PIPE>) {
   if (/^Stripes\s+(\d+)/i) {$stripes=$1;}
   if (/^Stripe\s+size\s+\(KByte\)\s+(\d+)/i) {$stripesize=$1; last;}
}
close PIPE;
$stripes == 0 && exit 3;
print "$stripes $stripesize\n";
exit 0;


Regards,
Heinz    -- The LVM Guy --


> 
> cheers.
> 
> -- 
> Nathan
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

*** Software bugs are stupid.
    Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
Mauelshagen@Sistina.com                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Re: [linux-lvm] Writing forward compatible applications using /proc

[Heinz J . Mauelshagen]

On Tue, Aug 14, 2001 at 01:49:51AM +0200, Ragnar Kj�rstad wrote:
> On Mon, Aug 13, 2001 at 10:08:58AM +0100, Joe Thornber wrote:
> > > Why will liblvm disappear? To me using a library interface seems much
> > > nicer than wrapping applications.
> > 
> > Because it's means there's yet another interface (along with the
> > command line tools, and ioctl's) to constrain any implementation
> > changes.  Command line tool interface will not change.  liblvm (if it
> > still existed as a shared library) will change drastically between 1.0
> > and 2.0 - not least because it in turn reflects the driver ioctl
> > interface.
> 
> I'm not complaining if it's changing, but if it's going away, that's bad
> :(
> 
> > The experimental branch has a single lvm tool, with liblvm statically
> > linked into it.  There's no need for the outside world to know about
> > liblvm.
> > 
> > It may be a good idea for Sistina to produce a tiny 'wrap the tools'
> > library for use by gui implementors.
> 
> yach! An uncomplete list of uglyness:
> * Errorhandling

Not much of a difference to the existing library error handling anyway.

> * fork/exec

Well, that's what the UNIX tool box is all about.
No mass fork/execs necessary.


> * having to format input arguments as strings (and quote them)

printf doesn't seem to hard to use ;-)

> * having to parse output data from strings 

No big deal with for eg. perl.

> 
> 
> -- 
> Ragnar Kjorstad
> Big Storage
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

-- 

Regards,
Heinz    -- The LVM Guy --

*** Software bugs are stupid.
    Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
Mauelshagen@Sistina.com                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Re: [linux-lvm] Writing forward compatible applications using /proc

[Ragnar Kjørstad]

On Tue, Aug 14, 2001 at 09:14:02AM +0100, Joe Thornber wrote:
> On Tue, Aug 14, 2001 at 03:04:18PM +1200, Mark van Walraven wrote:
> > On Sun, Aug 12, 2001 at 08:07:02PM +0100, Joe Thornber wrote:
> > > I would much rather see people wrapping the tools than using liblvm, in fact
> > > liblvm will probably disappear in the future.
> > 
> > I think it is a bad idea to eliminte liblvm.  Shared libraries are much
> > easier use to use from programs.  For example, it is much easier in a perl
> > script to use zlib than to run gzip as a subprocess in a bulletproof way.
> 
> I'm just worried about the interface to liblvm changing and breaking
> tools.  Let's wait a bit to see where experimental goes, and then see
> how much the library had to change.

I don't understand why the library interface would need to change more often than the command
line interface?


-- 
Ragnar

Re: [linux-lvm] Writing forward compatible applications using /proc

[Heinz J . Mauelshagen]

On Tue, Aug 14, 2001 at 07:50:23PM +0200, Ragnar Kj�rstad wrote:
> On Tue, Aug 14, 2001 at 09:14:02AM +0100, Joe Thornber wrote:
> > On Tue, Aug 14, 2001 at 03:04:18PM +1200, Mark van Walraven wrote:
> > > On Sun, Aug 12, 2001 at 08:07:02PM +0100, Joe Thornber wrote:
> > > > I would much rather see people wrapping the tools than using liblvm, in fact
> > > > liblvm will probably disappear in the future.
> > > 
> > > I think it is a bad idea to eliminte liblvm.  Shared libraries are much
> > > easier use to use from programs.  For example, it is much easier in a perl
> > > script to use zlib than to run gzip as a subprocess in a bulletproof way.
> > 
> > I'm just worried about the interface to liblvm changing and breaking
> > tools.  Let's wait a bit to see where experimental goes, and then see
> > how much the library had to change.
> 
> I don't understand why the library interface would need to change more often than the command
> line interface?

Because we will change the ioctl interface to the kernel to be a lot simpler
and the library will reflect this.

The CLI will stay stable though.

> 
> 
> -- 
> Ragnar
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@sistina.com
> http://lists.sistina.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html

-- 

Regards,
Heinz    -- The LVM Guy --

*** Software bugs are stupid.
    Nevertheless it needs not so stupid people to solve them ***

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

Heinz Mauelshagen                                 Sistina Software Inc.
Senior Consultant/Developer                       Am Sonnenhang 11
                                                  56242 Marienrachdorf
                                                  Germany
Mauelshagen@Sistina.com                           +49 2626 141200
                                                       FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-