Re: [patch 1/2] fork_connector: add a fork connector

Jay Lan
2005-03-29T21:21:06Z

Paul,

The fork_connector is not designed to solve accounting data collection
problem.

The accounting data collection must be done via a hook from do_exit().
The acct_process() hook invokes do_acct_process() to write BSD
accounting data to disk. CSA needs a similar hook off do_exit() to
collect more accounting data and write to disk in different
accounting records format. This part is not part of fork_connector.

ELSA does not care how accounting data being written to disks. However,
it needs the accounting data being reliably and accurately collected
and written to disk by BSD and/or CSA and it needs the <ppid, pid>
information to aggregate processes. It was never the fork_connector's
intention to piggy back the data to the accounting file.

Thanks,
- jay

Paul Jackson wrote:
> Evgeniy writes:
>
>>Here forking connector module "exits" and can handle next fork() on the
>>same CPU.
>
>
> Fine ... but it's not about what the fork_connector does. It's about
> getting the accounting data to disk, if I understand correctly.
>
>
>>That is why it is very fast in "fast-path".
>
>
> I don't care how fast a tool is. I care how fast the job gets done. If
> a tool is only doing part of the job, then we can't decide whether to
> use that tool just based on how fast that part of the job gets done.
>
>
>>The most expensive part is cn_netlink_send()/netlink_broadcast(),
>>with CBUS it is deferred to the safe time,
>
>
> This is "safe time" for the immediate purpose of letting the forking
> process continue on its way. But the deferred work of buffering up the
> data and writing it to disk still needs to be done, pretty soon. When
> sizing a system to see how many users or jobs I can run on it at a time,
> I will have to include sufficient cpu, memory and disk i/o to handle
> getting this accounting data to disk, right?
>
>
>>> 2) Using a modified form of what BSD ACCOUNTING does now:
>>> - forking process appends single fork data to in-kernel buffer
>>
>>It is not as simple.
>>It takes global locks several times, it access bunch of shared between
>>CPU data.
>>It calls ->stat() and ->write() which may sleep.
>
>
> Hmmm ... good points. The mechanisms in the kernel now (and for the
> last 25 years) to write out BSD ACCOUNTING data may not be numa friendly.
>
> Perhaps there should be a per-cpu 512 byte buffer, which can gather up 8
> accounting records (64 bytes each) and only call the file system write
> once every 8 task exits. Or perhaps a per-node buffer, with a spinlock
> to serialize access by the CPUs on that node. Or perhaps per-node
> accounting files. Or something like that.
>
> Guillaume, Jay - do we (you ?) need to make classic BSD ACCOUNTING data
> collection numa friendly? Based on the various frustrated comments at
> the top of kernel/acct.c, this could be a non-trivial effort to get
> right. Maybe we need it, but can't afford it.
>
> And perhaps my proposed variable length records for supplementary
> accounting, such as <parent pid, child pid> from fork, need to allow
> for some way to pad out the rest of a buffer, when the next record
> won't fit entirely.
>
>
>>That work is deferred and does not affect in-kernel processes.
>
>
> The accounting data collection cannot be deferred for long, perhaps
> just a few minutes. Not until the data hits the disk can we rest
> indefinitely. Unless, that is, I don't understand what problem is
> being solved here (quite possible ;).
>
>
>>And why userspace fork connector should write data to the disk?
>
>
> I NEVER said it should. I am NOT trying to redesign fork_connector.
>
> Good grief ... how many times and ways do I have to say this ;)?
>
> I am asking what is the best tool for accounting data collection,
> which, if I understand correctly, does need to write to disk.
>