Using script(1) to log all user sessions

Using script(1) to log all user sessions

[Jesper Dahl Nyerup]

[-- Attachment #1: Type: text/plain, Size: 1578 bytes --]

Hi list,

We're toying with the idea of using script(1) to log all user sessions
towards a collection of systems, as one in a number of metheds to aid
debugging, education and auditing.

In order to do so, I have a couple of questions regarding a few
extensions we're considering to implement.

Firstly, script(1) is clearly and sanely designed to be invoked from the
command line to record a limited portion of a user's session. In order
for the user to have the logging started without manual invocation, it
may come in handy to support config files, to supply configurable
default values for some of the concepts normally passed in the
environment or as command line arguments.

Secondly, we're considering to add functionality for script(1) to
transmit the session transcript over the network to a collection daemon,
to be able to store transcripts from multiple machines on one or more
central systems.

This could involve:

1. Adding a daemon next to script(1) and scriptreplay(1), eg.
scriptcollect(1), to be in the receiving end of the traffic, optionally
handling the timing functionality, and finally storing data in the same
manner script(1) would.

2. Optionally linking against some crypto library to avoid putting
users' console data on the wire in clear text.

Ideally we'd like to upstream any changes we make, and this post serves
as an attempt to gauge the mood and ask for guidance as to which
direction you guys think we should take our efforts.

Yours,
-- 
Jesper Dahl Nyerup
Systems Engineer
One.com, nyerup@one.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Using script(1) to log all user sessions

[Dale R. Worley]

> From: Jesper Dahl Nyerup <nyerup@one.com>

> 1. Adding a daemon next to script(1) and scriptreplay(1), eg.
> scriptcollect(1), to be in the receiving end of the traffic, optionally
> handling the timing functionality, and finally storing data in the same
> manner script(1) would.

You might want to consider using syslog(3) for this.  Or conversely,
if syslog won't work in your environment, that might indicate
improvements that are needed in syslog.

Dale

Re: Using script(1) to log all user sessions

[Jesper Dahl Nyerup]

[-- Attachment #1: Type: text/plain, Size: 986 bytes --]

On Apr 25  10:27, Dale R. Worley wrote:
> > From: Jesper Dahl Nyerup <nyerup@one.com>
> >
> > 1. Adding a daemon next to script(1) and scriptreplay(1), eg.
> > scriptcollect(1), to be in the receiving end of the traffic, optionally
> > handling the timing functionality, and finally storing data in the same
> > manner script(1) would.
> 
> You might want to consider using syslog(3) for this.  Or conversely,
> if syslog won't work in your environment, that might indicate
> improvements that are needed in syslog.

We considered that, but kind of gave up on the idea.

Syslog is inherently newline delimited, where as the per-update payloads
from script(1) are pretty much the opposite - one payload will often
contain multiple newlines, others will consist of a single character.

Thus, (de)serializing to and from syslog will probably be more of a
hassle than using it will be a benifit.

J.
-- 
Jesper Dahl Nyerup
Systems Engineer
One.com, nyerup@one.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Using script(1) to log all user sessions

[Jesper Dahl Nyerup]

[-- Attachment #1: Type: text/plain, Size: 857 bytes --]

Karel,

On Apr 25  10:21, Jesper Dahl Nyerup wrote:
> This could involve:
> 
> 1. Adding a daemon next to script(1) and scriptreplay(1), eg.
> scriptcollect(1), to be in the receiving end of the traffic, optionally
> handling the timing functionality, and finally storing data in the same
> manner script(1) would.
> 
> 2. Optionally linking against some crypto library to avoid putting
> users' console data on the wire in clear text.
> 
> Ideally we'd like to upstream any changes we make, and this post serves
> as an attempt to gauge the mood and ask for guidance as to which
> direction you guys think we should take our efforts.

Do you think functionality like this belongs in script(1) et al., and do
you think you'd accept the patches, if they look sane?

J.
-- 
Jesper Dahl Nyerup
Systems Engineer
One.com, nyerup@one.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Using script(1) to log all user sessions

[Ángel González]

On 25/04/14 10:21, Jesper Dahl Nyerup wrote:
> Hi list,
>
> We're toying with the idea of using script(1) to log all user sessions
> towards a collection of systems, as one in a number of metheds to aid
> debugging, education and auditing.
>
> In order to do so, I have a couple of questions regarding a few
> extensions we're considering to implement.
>
> Firstly, script(1) is clearly and sanely designed to be invoked from the
> command line to record a limited portion of a user's session. In order
> for the user to have the logging started without manual invocation, it
> may come in handy to support config files, to supply configurable
> default values for some of the concepts normally passed in the
> environment or as command line arguments.

You can start it from a script acting as the user shell, through sshd 
config or one of the shell init scripts. As you already need to start 
script somehow, those defaults could be passed there, too (although I 
don't see a problem with supporting config files either).


> Secondly, we're considering to add functionality for script(1) to
> transmit the session transcript over the network to a collection daemon,
> to be able to store transcripts from multiple machines on one or more
> central systems.
>
> This could involve:
>
> 1. Adding a daemon next to script(1) and scriptreplay(1), eg.
> scriptcollect(1), to be in the receiving end of the traffic, optionally
> handling the timing functionality, and finally storing data in the same
> manner script(1) would.

I'm unsure about this bit. It may be needed. Perhaps a transfer after 
the session finishes also works.


> 2. Optionally linking against some crypto library to avoid putting
> users' console data on the wire in clear text.

Following unix philosophy, I would try to avoid reinventing crypto into 
the program, attempting instead to solve the issue by eg. using sftp to 
transfer the files and/or gpg to encrypt the data.



PS: I expect you are properly warning your users about the fascist-level 
logging done on your systems.

Re: Using script(1) to log all user sessions

[Karel Zak]

On Fri, Apr 25, 2014 at 10:21:34AM +0200, Jesper Dahl Nyerup wrote:
> Firstly, script(1) is clearly and sanely designed to be invoked from the

 Well, from my point of view it current code is a bit fragile regards
 to signals. Maybe it would be nice to use signalfd() and check for
 signals together with changes on master/slave file descriptors by
 select/poll. But this is not so important for your plans.

> command line to record a limited portion of a user's session. In order
> for the user to have the logging started without manual invocation, it
> may come in handy to support config files, to supply configurable
> default values for some of the concepts normally passed in the
> environment or as command line arguments.

 OK.

> Secondly, we're considering to add functionality for script(1) to
> transmit the session transcript over the network to a collection daemon,
> to be able to store transcripts from multiple machines on one or more
> central systems.

 Hmm.. "Every program attempts to expand until it can read mail. Those
 programs which cannot so expand are replaced by ones which can."

 I like git concept: here are files and it's your problem to transfer the
 staff over the network. You can use rsync, http, ssh, ...

 Maybe all you need is to store timing and typescript data to the one
 place (e.g. /var/log/typescripts/user/<timestamp>.{session,timing})
 and use ssh or so to transfer the data to another place.

> 1. Adding a daemon next to script(1) and scriptreplay(1), eg.
> scriptcollect(1), to be in the receiving end of the traffic, optionally
> handling the timing functionality, and finally storing data in the same
> manner script(1) would.

 Seem like overkill.

> 2. Optionally linking against some crypto library to avoid putting
> users' console data on the wire in clear text.

 Yep, but we have generic solutions like ssh, again see git.

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: Using script(1) to log all user sessions

[Jesper Dahl Nyerup]

[-- Attachment #1: Type: text/plain, Size: 2349 bytes --]

On Apr 27  21:53, Ángel González wrote:
> On 25/04/14 10:21, Jesper Dahl Nyerup wrote:
> >[...] support config files, to supply configurable
> >default values for some of the concepts normally passed in the
> >environment or as command line arguments.
> 
> You can start it from a script acting as the user shell, through
> sshd config or one of the shell init scripts. As you already need to
> start script somehow, those defaults could be passed there, too
> (although I don't see a problem with supporting config files
> either).

Yes, this is how we have it working in our current test deployments, but
for simplicity we'd like to leave out these steps, and ideally have
users' shell to be /usr/bin/script.

> >1. Adding a daemon next to script(1) and scriptreplay(1), eg.
> >scriptcollect(1), to be in the receiving end of the traffic, [...]
> 
> I'm unsure about this bit. It may be needed. Perhaps a transfer
> after the session finishes also works.

We also considered that, but we keep running in to theoretical corner
cases where this could end up being a problem - logging in on systems
with filled up mountpoints, securing the transcript even if the system
crashes, and so on.

> >2. Optionally linking against some crypto library to avoid putting
> >users' console data on the wire in clear text.
> 
> Following unix philosophy, I would try to avoid reinventing crypto
> into the program, attempting instead to solve the issue by eg. using
> sftp to transfer the files and/or gpg to encrypt the data.

I fully agree with this concern. This is just a necessary follow up, if
we want to natively enable network support in script(1).

However as both you and others have suggested us to reconsider this, and
as we also were pretty doubtful about this ourselves, we will probably
find an alternative transport method, one way or the other.

> PS: I expect you are properly warning your users about the
> fascist-level logging done on your systems.

I appreciate your concern for our users.

These users are myself as well as my colleagues, and we all have a
shared interest in maintaining audit trails and tracebacks of who did
what, when and where.

I can assure you that everyone are aware of these measures.

Yours,
-- 
Jesper Dahl Nyerup
Systems Engineer
One.com, nyerup@one.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Using script(1) to log all user sessions

[Karel Zak]

On Tue, Apr 29, 2014 at 12:00:28PM +0200, Jesper Dahl Nyerup wrote:
> > >1. Adding a daemon next to script(1) and scriptreplay(1), eg.
> > >scriptcollect(1), to be in the receiving end of the traffic, [...]
> > 
> > I'm unsure about this bit. It may be needed. Perhaps a transfer
> > after the session finishes also works.
> 
> We also considered that, but we keep running in to theoretical corner
> cases where this could end up being a problem - logging in on systems
> with filled up mountpoints, securing the transcript even if the system
> crashes, and so on.

You don't have to transfer the logs after finished session, but in
small segments (script(1) only add new entries to the file). The same
idea uses large databases for on-line backups
(http://www.postgresql.org/docs/8.1/static/backup-online.html)


The another (IMHO the best) possibility is to log into systemd journal, 
it supports all necessary advantages. See

  http://www.freedesktop.org/software/systemd/man/sd-journal.html

for example sd_journal_stream_fd(). I'm not sure, but it's probably
already possible to somehow mirror the journal over network or log to
remote machine.

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: Using script(1) to log all user sessions

[Jesper Dahl Nyerup]

[-- Attachment #1: Type: text/plain, Size: 1758 bytes --]

On Apr 28  09:13, Karel Zak wrote:
> On Fri, Apr 25, 2014 at 10:21:34AM +0200, Jesper Dahl Nyerup wrote:
> > [...] support config files, to supply configurable
> > default values for some of the concepts normally passed in the
> > environment or as command line arguments.
> 
>  OK.

We will produce a patch for this, and submit it fow review.

> > Secondly, we're considering to add functionality for script(1) to
> > transmit the session transcript over the network to a collection daemon,
> > to be able to store transcripts from multiple machines on one or more
> > central systems.
> 
>  Hmm.. "Every program attempts to expand until it can read mail. Those
>  programs which cannot so expand are replaced by ones which can."
> 
>  I like git concept: here are files and it's your problem to transfer the
>  staff over the network. You can use rsync, http, ssh, ...
> 
>  Maybe all you need is to store timing and typescript data to the one
>  place (e.g. /var/log/typescripts/user/<timestamp>.{session,timing})
>  and use ssh or so to transfer the data to another place.

I see your point.

Storing the files locally before transferring them away has a number of
drawbacks, for instance logging in on systems with filled up
mountpoints, data loss during crashes on systems without persistent
storage, tampering, and so on.

We'll probably end up piping the transcript data to ssh(1), and have
something collecting the data remotely. One of the challenges is to
ensure this transport mechanism doesn't end up blocking the user's
terminal session, in case of network problems or other offload
difficulties.

Thank you all for you input.

Yours,
-- 
Jesper Dahl Nyerup
Systems Engineer
One.com, nyerup@one.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: Using script(1) to log all user sessions

[Jesper Dahl Nyerup]

[-- Attachment #1: Type: text/plain, Size: 1616 bytes --]

On Apr 29  12:42, Karel Zak wrote:
> On Tue, Apr 29, 2014 at 12:00:28PM +0200, Jesper Dahl Nyerup wrote:
> > > >1. Adding a daemon next to script(1) and scriptreplay(1), eg.
> > > >scriptcollect(1), to be in the receiving end of the traffic, [...]
> > > 
> > > I'm unsure about this bit. It may be needed. Perhaps a transfer
> > > after the session finishes also works.
> > 
> > We also considered that, but we keep running in to theoretical corner
> > cases where this could end up being a problem - logging in on systems
> > with filled up mountpoints, securing the transcript even if the system
> > crashes, and so on.
> 
> You don't have to transfer the logs after finished session, but in
> small segments (script(1) only add new entries to the file). The same
> idea uses large databases for on-line backups
> (http://www.postgresql.org/docs/8.1/static/backup-online.html)

Good idea. But then we might as well pipe the output directly to some
rsh, avoiding the data to touch storage in the first place.

> The another (IMHO the best) possibility is to log into systemd journal, 
> it supports all necessary advantages. See
> 
>   http://www.freedesktop.org/software/systemd/man/sd-journal.html
> 
> for example sd_journal_stream_fd(). I'm not sure, but it's probably
> already possible to somehow mirror the journal over network or log to
> remote machine.

That's a good idea – we'll look into that. Sadly some of the systems
where we'd want this won't be running systemd for the next few years.

Yours,
-- 
Jesper Dahl Nyerup
Systems Engineer
One.com, nyerup@one.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]