[PATCH] 2.6.x BSD Process Accounting w/High UID

[PATCH] 2.6.x BSD Process Accounting w/High UID

[Arthur Corliss]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 334 bytes --]

Greetings:

The patch only changes two lines which redefine the ac_uid/ac_gid fields as
uid_t/gid_t respectively.  Fixes accounting for high uid/gids.

	--Arthur Corliss
	  Bolverk's Lair -- http://arthur.corlissfamily.org/
	  Digital Mages -- http://www.digitalmages.com/
	  "Live Free or Die, the Only Way to Live" -- NH State Motto

[-- Attachment #2: Type: TEXT/PLAIN, Size: 837 bytes --]

--- linux-2.6.3/include/linux/acct.h    2004-02-17 18:57:18.000000000 -0900
+++ linux/include/linux/acct.h  2004-03-04 08:47:48.000000000 -0900
@@ -41,8 +41,8 @@
  *     No binary format break with 2.0 - but when we hit 32bit uid we'll
  *     have to bite one
  */
-       __u16           ac_uid;                 /* Accounting Real User ID */
-       __u16           ac_gid;                 /* Accounting Real Group ID */
+       uid_t           ac_uid;                 /* Accounting Real User ID */
+       gid_t           ac_gid;                 /* Accounting Real Group ID */
        __u16           ac_tty;                 /* Accounting Control Terminal */
        __u32           ac_btime;               /* Accounting Process Creation Time */
        comp_t          ac_utime;               /* Accounting User Time */

Re: [PATCH] 2.6.x BSD Process Accounting w/High UID

[Rik van Riel]

On Thu, 4 Mar 2004, Arthur Corliss wrote:

> The patch only changes two lines which redefine the ac_uid/ac_gid fields as
> uid_t/gid_t respectively.  Fixes accounting for high uid/gids.

Do the userspace commands that parse the acct files
know how to deal with this format change ?

-- 
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan

Re: [PATCH] 2.6.x BSD Process Accounting w/High UID

[Arthur Corliss]

On Thu, 4 Mar 2004, Rik van Riel wrote:

> On Thu, 4 Mar 2004, Arthur Corliss wrote:
>
> > The patch only changes two lines which redefine the ac_uid/ac_gid fields as
> > uid_t/gid_t respectively.  Fixes accounting for high uid/gids.
>
> Do the userspace commands that parse the acct files
> know how to deal with this format change ?

Well, that's the ugly part.  I had to munge glibc's sys/acct.h to reflect the
same changes as well to get the gnu tools to read the file correctly (the
accounting file does need to be started fresh with this struct, btw).

Here's the problem:  the kernel uses 32 bit uids internally, but external code
would default to 16 bit due to the ifdefs.

Unfortunately, I don't how else to support accounting with high uids.  All it
takes is two lines each in two files (linux/acct.h & sys/acct.h) to get
everything to work correctly.

So, what's the consensus:  are we going to leave accounting broken for high
uids, or support only low uids?  If I can get a patch accepted by the glibc
maintainers as well this will work for everyone.

I actually use high uids, so this is production issue for me.  :-P

	--Arthur Corliss
	  Bolverk's Lair -- http://arthur.corlissfamily.org/
	  Digital Mages -- http://www.digitalmages.com/
	  "Live Free or Die, the Only Way to Live" -- NH State Motto

Re: [PATCH] 2.6.x BSD Process Accounting w/High UID

[Tim Schmielau]

> > Do the userspace commands that parse the acct files
> > know how to deal with this format change ?
> 
> Well, that's the ugly part.  I had to munge glibc's sys/acct.h to reflect the
> same changes as well to get the gnu tools to read the file correctly (the
> accounting file does need to be started fresh with this struct, btw).

In a first step, we could map all high uids onto one 'reserved' uid to
avoid requiring new userspace tools.
This is along the lines what I did when jiffies became 64 bits - just map 
larger times onto the maximum time representable with 32 bits.

We didn't even exploit the fact that 34 bit times would be representable 
in the current format, because existing userspace tools would probably 
break then.
At the time I did that change, it seemed to be common consensus that too
few people cared about accurate BSD accounting to require new userspace 
tools.

btw: if you actually push an incompatible change, could we do something 
about large times as well?

Tim

Re: [PATCH] 2.6.x BSD Process Accounting w/High UID

[Arthur Corliss]

On Thu, 4 Mar 2004, Tim Schmielau wrote:

> In a first step, we could map all high uids onto one 'reserved' uid to
> avoid requiring new userspace tools.
> This is along the lines what I did when jiffies became 64 bits - just map
> larger times onto the maximum time representable with 32 bits.
>
> We didn't even exploit the fact that 34 bit times would be representable
> in the current format, because existing userspace tools would probably
> break then.
> At the time I did that change, it seemed to be common consensus that too
> few people cared about accurate BSD accounting to require new userspace
> tools.

First off:  there are no changes needed the current defacto accounting tools
on Linux (the GNU package).  As long as they can reference the correct size of
the struct (as defined in acct.h) they work just fine.  The GNU package
doesn't look like it's been updated since '98, and it works just fine with the
right header.  I mentioned that I did update glibc's sys/acct.h because it
uses that in preference over the kernel header.  Minor issue, but something
I'm looking at.

Second:  I don't want new userspace tools, either, but I do want the ones
I've got to work, which is what they don't do when it reports the lower bits
of the uid field on high uids.  In other words, the tools are *already*
broken.  I realise that I'm probably a corner case in that most admins will
never assign high uids, but that really doesn't make me feel better about
broken tools.  ;-)

> btw: if you actually push an incompatible change, could we do something
> about large times as well?

If the numbers we're logging are meaningless, then hell, yes, let's fix them
all!

	--Arthur Corliss
	  Bolverk's Lair -- http://arthur.corlissfamily.org/
	  Digital Mages -- http://www.digitalmages.com/
	  "Live Free or Die, the Only Way to Live" -- NH State Motto

Re: [PATCH] 2.6.x BSD Process Accounting w/High UID

[Jeremy Jackson]

Arthur Corliss wrote:

> 
> 
> If the numbers we're logging are meaningless, then hell, yes, let's fix them
> all!

I believe the answer (to seamless backwards compatibility) lies in 
struct acct's ac_pad[10] member.

3 options exist that I can see:

1) put the high 16 bits in there, with a magic # (at then end of?) 
ac_pad.  THe old tools will be none the wiser, the new tools will 
autodetect which format the acct file is in.  Ugly but easy.

2) just make the uid/gid 32bits, and put a magic#  (at the end of?) 
ac_pad.  The old tools will choke, but the new tools will autodetect. 
If you push the new tools out a couple of years ahead, then merge the 
fix, acceptance will be fairly smooth.  Clean but painful.

or
3) make the split of 16 bits interim with one magic#, make the tools 
detect 3 formats, and in a few years, switch from the bastard 32bit to 
the clean one (different magic #).  This will give tools time to become 
standard.
Combines best of both of the above.

You can do the above with the time stuff too, but 10 bytes spare might 
constrain things a bit.  Heck, make the struct bigger, as long as there 
is a magic #, userspace should be ok.  Right now, "file" command can't 
tell what the heck the file is.  Bit wasteful to put magic in every 
record though.

While you're at it, make a switch for a tool that prints out 
ac_exitcode, without reading the binary acct file (or it's dump).

Cheers,
-- 
Jeremy Jackson
Coplanar Networks
http://www.coplanar.net

Re: [PATCH] 2.6.x BSD Process Accounting w/High UID

[Tim Schmielau]

> I believe the answer (to seamless backwards compatibility) lies in 
> struct acct's ac_pad[10] member.

sure.

> 3 options exist that I can see:
> 
> 1) put the high 16 bits in there, with a magic # (at then end of?) 
> ac_pad.  THe old tools will be none the wiser, the new tools will 
> autodetect which format the acct file is in.  Ugly but easy.
> 
> 2) just make the uid/gid 32bits, and put a magic#  (at the end of?) 
> ac_pad.  The old tools will choke, but the new tools will autodetect. 
> If you push the new tools out a couple of years ahead, then merge the 
> fix, acceptance will be fairly smooth.  Clean but painful.
> 
> or
> 3) make the split of 16 bits interim with one magic#, make the tools 
> detect 3 formats, and in a few years, switch from the bastard 32bit to 
> the clean one (different magic #).  This will give tools time to become 
> standard.
> Combines best of both of the above.

4) make ac_uid and ac_gid 32 bits and move them into the place formerly
   occupied by ac_pad[10]. Establish comatibility fields ac_uid16 and
   ac_gid16 in the old places of ac_uid/ac/gid. Introduce a magic (or not,
   since checking for (ac_uid16==ac_uid && ac_gid16==ac_gid) will reveal
   whether 32bit uids are supported).
   Best of all worlds: old tools will keep working. Just recompiling them
   will yield 32 bit uid/gid support. New tools can detect whether 32 bit
   uid/gid are supported or not.
   Problem is, ac_pad[10] doesn't provide enough space for two aligned
   32 bit numbers _and_ a magic in ac_pad[10], which is the last byte of
   struct acct. If we put in only the two 32 bit numbers, we've lost the
   ability to establish a magic _in a prominent place_ for smooth 
   transitions in the future.
   We might, however, put the magic into the implicit padding of ac_flag
   (ugly, since this would require an arch dependant definition of stuct 
   acct), or put it into the uppermost three bits of ac_flag itself
   (7 seven incompatible changes to struct acct should suffice in the
   foreseeable future, but new flags would need to go into a different 
   field, breaking source compatibility with userspace acct code on 
   different archs, i.e. GNU acct tools).

I've posted a patch for 3) already. I'd be happy to go with 4) if we 
decide to reserve the remaining bits in ac_flag for a version number.
If any source incompatible change is foreseeable in the 2.7 timeframe, 
I'd prefer to stick with 3) though.

> You can do the above with the time stuff too, but 10 bytes spare might 
> constrain things a bit.  Heck, make the struct bigger, as long as there 
> is a magic #, userspace should be ok.  Right now, "file" command can't 
> tell what the heck the file is.  Bit wasteful to put magic in every 
> record though.

Yeah, we might want a larger comp_t not only for times, but also for
ac_io. Unfortunately, the layout of comp_t is hard-coded into GNU acct,
so no way of keeping source code compatibility (other than uncompressed 64
bit fields).

Worse, I have to revoke my previous statement of GNU acct being able to
read 34 bits comp_t values. Yes, it stores the into a double, but it
uses a long as temporary variable. Sigh.

So a new patch is due anyways. Any further comments for that?
What to do about comp_t in 2.7?

Tim

Re: [PATCH] 2.6.x BSD Process Accounting w/High UID

[Albert Cahalan]

> We might, however, put the magic into the implicit padding
> of ac_flag (ugly, since this would require an arch dependant
> definition of stuct acct), or put it into the uppermost
> three bits of ac_flag itself

First of all, none of this matters much if the format
is given a sysctl. The old format is default for now,
and the new one is default (only?) in a couple years.
Sun appears to have done something like this.

When fixing it, note that a 5-bit binary exponent
with denormals would beat the current float format.

Regarding the existing struct though...

Let's take a close look at this. I think there are 2 bytes
of padding on all Linux ports, and another 2 available
on everything except maybe m68k and/or arm. (that is, ports
that will put a u32 on any u16 boundry) Here is the current
struct, compactly formatted with 64-bit blocking:

struct linux_acct {
        char   ac_flag;        // Flags
// 1 pad byte
        __u16  ac_uid;         // Real User ID    
        __u16  ac_gid;         // Real Group ID      
        __u16  ac_tty;         // Control Terminal

        __u32  ac_btime;       // Process Creation Time
        comp_t ac_utime;       // User Time     
        comp_t ac_stime;       // System Time

        comp_t ac_etime;       // Elapsed Time
        comp_t ac_mem;         // Average Memory Usage
        comp_t ac_io;          // Chars Transferred
        comp_t ac_rw;          // Blocks Read or Written

        comp_t ac_minflt;      // Minor Pagefaults   
        comp_t ac_majflt;      // Major Pagefaults  
        comp_t ac_swaps;       // Number of Swaps
// 2 pad bytes (except m68k or arm maybe?)

        __u32  ac_exitcode;    // Exitcode
// hppa might pad this
        char   ac_comm[17];    // Command Name    
// hppa might pad this
        char   ac_pad[10];     // Padding Bytes
// 1 pad byte (maybe 6 for hppa)
};

Just a sec... ARRRGH WHY DO PEOPLE LEAVE PADDING AND
GENERALLY MIS-ALIGN THINGS ALL THE TIME??????

So there you go. You have a pad byte after ac_flag
at a known location, and a pad byte after ac_pad that
might move a bit due to mis-alignments above it.

Re: [PATCH] 2.6.x BSD Process Accounting w/High UID

[Tim Schmielau]

On Wed, 9 Mar 2004, Albert Cahalan wrote:

> First of all, none of this matters much if the format
> is given a sysctl. The old format is default for now,
> and the new one is default (only?) in a couple years.
> Sun appears to have done something like this.

Yeah, if we even want to introduce a new syscall for this...
But Documentation/Changes is not an empty file, i.e., we require
people to upgrade userspace anyways.

> When fixing it, note that a 5-bit binary exponent
> with denormals would beat the current float format.

Yes, but only by a short head. And, since comp_t is hard-coded into 
current GNU acct tools, we can't keep source compatibility
(not that this matters too much anyways...)
I'm open for suggestions in this direction. Any reasonable ideas to
get more precision? (e.g. 16 bit mantissa and 4 bit base-whatever 
exponent?) 

> Regarding the existing struct though...
> 
> Let's take a close look at this. I think there are 2 bytes
> of padding on all Linux ports, and another 2 available
> on everything except maybe m68k and/or arm. (that is, ports
> that will put a u32 on any u16 boundry) Here is the current
> struct, compactly formatted with 64-bit blocking:
> 
> struct linux_acct {
>         char   ac_flag;        // Flags
> // 1 pad byte

Yep, but depending on architecure I think the compiler is free to insert
the padding either before or after ac_flags.

[...]
> 
> Just a sec... ARRRGH WHY DO PEOPLE LEAVE PADDING AND
> GENERALLY MIS-ALIGN THINGS ALL THE TIME??????

Yupp, I changed this for the new suggested binary-incompatible layout.

> So there you go. You have a pad byte after ac_flag
> at a known location, and a pad byte after ac_pad that
> might move a bit due to mis-alignments above it.

Yes, but I'd prefer an arch-independent and prominent position
(i.e. one that doesn't perturb alignment at the beginning of the struct)

Tim

Re: [PATCH] 2.6.x BSD Process Accounting w/High UID

[Albert Cahalan]

On Wed, 2004-03-10 at 04:08, Tim Schmielau wrote:
> On Wed, 9 Mar 2004, Albert Cahalan wrote:
> 
>> First of all, none of this matters much if the format
>> is given a sysctl. The old format is default for now,
>> and the new one is default (only?) in a couple years.
>> Sun appears to have done something like this.
> 
> Yeah, if we even want to introduce a new syscall for this...
> But Documentation/Changes is not an empty file, i.e., we
> require people to upgrade userspace anyways.

In that case, don't forget to upgrade atop.
That's a version of top that uses BSD process
accounting to grab the transient processes
that wouldn't be seen in /proc before they die.

>> When fixing it, note that a 5-bit binary exponent
>> with denormals would beat the current float format.
> 
> Yes, but only by a short head.

It's by 8 bits, with a stable 11-bit fraction
instead of a 10-bit to 12-bit variable-size one.

old: 1xxxxxxxxxxyy000000000000000000000
new: 1xxxxxxxxxxx000000000000000000000000000000

That's a 42-bit number instead of a 36-bit one.
The old base-8 exponent is wasteful, plus the old
format stores the leading 1 instead of using an
implied 1 and special exponent for leading 0.

> And, since comp_t is hard-coded into 
> current GNU acct tools, we can't keep source compatibility
> (not that this matters too much anyways...)
> I'm open for suggestions in this direction. Any reasonable ideas to
> get more precision? (e.g. 16 bit mantissa and 4 bit base-whatever 
> exponent?) 

a. just what I said
b. 32-bit IEEE float (easy enough to encode by hand)
c. raw data -- is the space saving all that critical?
d. raw data with gzip-style compression

(note that gzip's deflate algorithm is in the kernel)

> > Regarding the existing struct though...
> > 
> > Let's take a close look at this. I think there are 2 bytes
> > of padding on all Linux ports, and another 2 available
> > on everything except maybe m68k and/or arm. (that is, ports
> > that will put a u32 on any u16 boundry) Here is the current
> > struct, compactly formatted with 64-bit blocking:
> > 
> > struct linux_acct {
> >         char   ac_flag;        // Flags
> > // 1 pad byte
> 
> Yep, but depending on architecure I think the compiler is free to insert
> the padding either before or after ac_flags.

I doubt it. I think the C standard has something to
say about this. In any case, I just checked a mix of
big-endian and little-endian boxes:

32-bit BE SPARC
64-bit BE SPARC
32-bit BE PowerPC
32-bit LE i386
64-bit LE x86-64
64-bit LE Alpha

In every case, 1-byte padding came after ac_flag.

I'm pretty sure ac_comm is too big as well. It has
room for 17 bytes. It needs room for 15, plus a '\0'
if you want one.

Re: [PATCH] 2.6.x BSD Process Accounting w/High UID

[Tim Schmielau]

On Wed, 10 Mar 2004, Albert Cahalan wrote:

> In that case, don't forget to upgrade atop.
> That's a version of top that uses BSD process
> accounting to grab the transient processes
> that wouldn't be seen in /proc before they die.

OK, thanks for the reference. I wonder what other packages use 
BSD accounting.

> 
> >> When fixing it, note that a 5-bit binary exponent
> >> with denormals would beat the current float format.
> > 
> > Yes, but only by a short head.
> 
> It's by 8 bits, with a stable 11-bit fraction
> instead of a 10-bit to 12-bit variable-size one.
> 
> old: 1xxxxxxxxxxyy000000000000000000000
> new: 1xxxxxxxxxxx000000000000000000000000000000
> 
> That's a 42-bit number instead of a 36-bit one.

Huh, are we talking about the same format? I think comp_t rather looks
like this:

  eeemmmmmmmmmmmmm

i.e., three bit (base 8) exponent, followed by 13 bits mantissa.
mantissa is right-aligned, so I don't understand why you are talking 
about denormals at all.
The base 8 exponent indeed wastes up to two bits of the mantissa, but it
saves two bits of the exponent, so it's as good in the worst case and
gains one bit of precision on average.
The largest representable number (as I see it) is only (2^13-1) * 8^7
= 0x3ffe00000, i.e. a 34 bit number.

What's wrong with my view?

> The old base-8 exponent is wasteful, plus the old
> format stores the leading 1 instead of using an
> implied 1 and special exponent for leading 0.

I really don't know which leading 1 you are talking about.

> > get more precision? (e.g. 16 bit mantissa and 4 bit base-whatever 
> > exponent?) 
> 
> a. just what I said
> b. 32-bit IEEE float (easy enough to encode by hand)

great idea - it even allows to keep source code compatibility with
(at least) the GNU acct tools!

> c. raw data -- is the space saving all that critical?

No, but I think it's just not worth expanding struct acct.

> d. raw data with gzip-style compression
> 
> (note that gzip's deflate algorithm is in the kernel)

too unpredictable, I think.

e. suddenly pops to my mind:  #define AHZ 10      /* ;-) */ 
   but b. seems superior to me as it increases precision as well and
   we can afford it.

> > Yep, but depending on architecure I think the compiler is free to insert
> > the padding either before or after ac_flags.
> 
> I doubt it. I think the C standard has something to
> say about this.

I'd guess I'll have to update my K&R to ANSI on that...

> In any case, I just checked a mix of
> big-endian and little-endian boxes:
> 
> 32-bit BE SPARC
> 64-bit BE SPARC
> 32-bit BE PowerPC
> 32-bit LE i386
> 64-bit LE x86-64
> 64-bit LE Alpha
> 
> In every case, 1-byte padding came after ac_flag.

That sounds good. A version magic at the beginning of the struct would
allow us to extend it more easily.

Tim

Re: [PATCH] 2.6.x BSD Process Accounting w/High UID

[Tim Schmielau]

On Wed, 10 Mar 2004, Albert Cahalan wrote:

> That's a 42-bit number instead of a 36-bit one.

OK, your format clearly wins. Especially since I think that comp_t can
only encode a 34-bit number.

But I favor your suggestion of 32-bit IEEE floats even more,
as it doesn't need a change to the GNU acct tools.

Sorry for the confusion

Tim

Re: [PATCH] 2.6.x BSD Process Accounting w/High UID

[Albert Cahalan]

On Wed, 2004-03-10 at 12:47, Tim Schmielau wrote:
> On Wed, 10 Mar 2004, Albert Cahalan wrote:
> 
> > That's a 42-bit number instead of a 36-bit one.
> 
> OK, your format clearly wins. Especially since I think that comp_t can
> only encode a 34-bit number.

That is correct. 42 - 8 != 36

My diagram was right; the math was not.

> But I favor your suggestion of 32-bit IEEE floats even more,
> as it doesn't need a change to the GNU acct tools.

I'm surprised. Do the tools rely on a #define for this?

Is there a reason to have the whole struct be a
power of two? If so, and you don't wish to expand
it to 128 bytes, consider packing 3 80-byte records
and a 16-byte header into 256 bytes.

Re: [PATCH] 2.6.x BSD Process Accounting w/High UID

[Tim Schmielau]

On Wed, 10 Mar 2004, Albert Cahalan wrote:

> > But I favor your suggestion of 32-bit IEEE floats even more,
> > as it doesn't need a change to the GNU acct tools.
> 
> I'm surprised. Do the tools rely on a #define for this?

They check the types with autoconf. In case of a comp_t, the 
hard-coded conversion routine is called, otherwise the value is
just cast to a double.

> Is there a reason to have the whole struct be a
> power of two?

Not that I'm aware of. But I hesitate to change the length of the
struct since seeking in a file with mixed field lengths would become
a nightmare. And why would we want to store a version number in each
acct record if we can't mix them afterwards?

So my current plot would be to expand the most interesting fields to
floats, using up the padding, and leave the others as comp_t.

Actually, the only field of interest here seems to be ac_etime where, as 
Arthur Corliss pointed out, also precision matters.
Less impotant candidates might be ac_io, ac_rw and/or ac_minflt.

For ac_utime and ac_stime, on the other hand, 497 days of cpu time for
each job seems rather plentyful. Even more, if we'd fix the internal
overflow in GNU acct's comp_t conversion routine and have 1988 days at
hand.


Tim