RFC(V3): Audit Kernel Container IDs

RFC(V3): Audit Kernel Container IDs

[Richard Guy Briggs]

Containers are a userspace concept.  The kernel knows nothing of them.

The Linux audit system needs a way to be able to track the container
provenance of events and actions.  Audit needs the kernel's help to do
this.

Since the concept of a container is entirely a userspace concept, a
registration from the userspace container orchestration system initiates
this.  This will define a point in time and a set of resources
associated with a particular container with an audit container
identifier.

The registration is a u64 representing the audit container identifier
written to a special file in a pseudo filesystem (proc, since PID tree
already exists) representing a process that will become a parent process
in that container.  This write might place restrictions on mount
namespaces required to define a container, or at least careful checking
of namespaces in the kernel to verify permissions of the orchestrator so
it can't change its own container ID.  A bind mount of nsfs may be
necessary in the container orchestrator's mount namespace.  This write
can only happen once per process.

Note: The justification for using a u64 is that it minimizes the
information printed in every audit record, reducing bandwidth and limits
comparisons to a single u64 which will be faster and less error-prone.

Require CAP_AUDIT_CONTROL to be able to carry out the registration.  At
that time, record the target container's user-supplied audit container
identifier along with a target container's parent process (which may
become the target container's "init" process) process ID (referenced
from the initial PID namespace) in a new record AUDIT_CONTAINER with a
qualifying op=$action field.

Issue a new auxilliary record AUDIT_CONTAINER_INFO for each valid
container ID present on an auditable action or event.

Forked and cloned processes inherit their parent's audit container
identifier, referenced in the process' task_struct.  Since the audit
container identifier is inherited rather than written, it can still be
written once.  This will prevent tampering while allowing nesting.
(This can be implemented with an internal settable flag upon
registration that does not get copied across a fork/clone.)

Mimic setns(2) and return an error if the process has already initiated
threading or forked since this registration should happen before the
process execution is started by the orchestrator and hence should not
yet have any threads or children.  If this is deemed overly restrictive,
switch all of the target's threads and children to the new containerID.

Trust the orchestrator to judiciously use and restrict CAP_AUDIT_CONTROL.

When a container ceases to exist because the last process in that
container has exited log the fact to balance the registration action.  
(This is likely needed for certification accountability.)

At this point it appears unnecessary to add a container session
identifier since this is all tracked from loginuid and sessionid to
communicate with the container orchestrator to spawn an additional
session into an existing container which would be logged.  It can be
added at a later date without breaking API should it be deemed
necessary.

The following namespace logging actions are not needed for certification
purposes at this point, but are helpful for tracking namespace activity.
These are auxilliary records that are associated with namespace
manipulation syscalls unshare(2), clone(2) and setns(2), so the records
will only show up if explicit syscall rules have been added to document
this activity.

Log the creation of every namespace, inheriting/adding its spawning
process' audit container identifier(s), if applicable.  Include the
spawning and spawned namespace IDs (device and inode number tuples).
[AUDIT_NS_CREATE, AUDIT_NS_DESTROY] [clone(2), unshare(2), setns(2)]
Note: At this point it appears only network namespaces may need to track
container IDs apart from processes since incoming packets may cause an
auditable event before being associated with a process.  Since a
namespace can be shared by processes in different containers, the
namespace will need to track all containers to which it has been
assigned.

Upon registration, the target process' namespace IDs (in the form of a
nsfs device number and inode number tuple) will be recorded in an
AUDIT_NS_INFO auxilliary record.

Log the destruction of every namespace that is no longer used by any
process, including the namespace IDs (device and inode number tuples).
[AUDIT_NS_DESTROY] [process exit, unshare(2), setns(2)]

Issue a new auxilliary record AUDIT_NS_CHANGE listing (opt: op=$action)
the parent and child namespace IDs for any changes to a process'
namespaces. [setns(2)]
Note: It may be possible to combine AUDIT_NS_* record formats and
distinguish them with an op=$action field depending on the fields
required for each message type.

The audit container identifier will need to be reaped from all
implicated namespaces upon the destruction of a container.

This namespace information adds supporting information for tracking
events not attributable to specific processes.

Changelog:

(Upstream V3)
- switch back to u64 (from pmoore, can be expanded to u128 in future if
  need arises without breaking API.  u32 was originally proposed, up to
  c36 discussed)
- write-once, but children inherit audit container identifier and can
  then still be written once
- switch to CAP_AUDIT_CONTROL
- group namespace actions together, auxilliary records to namespace
  operations.

(Upstream V2)
- switch from u64 to u128 UUID
- switch from "signal" and "trigger" to "register"
- restrict registration to single process or force all threads and
  children into same container

- RGB

--
Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Sr. S/W Engineer, Kernel Security, Base Operating Systems
Remote, Ottawa, Red Hat Canada
IRC: rgb, SunRaycer
Voice: +1.647.777.2635, Internal: (81) 32635

RFC(V3): Audit Kernel Container IDs

[Richard Guy Briggs]

Containers are a userspace concept.  The kernel knows nothing of them.

The Linux audit system needs a way to be able to track the container
provenance of events and actions.  Audit needs the kernel's help to do
this.

Since the concept of a container is entirely a userspace concept, a
registration from the userspace container orchestration system initiates
this.  This will define a point in time and a set of resources
associated with a particular container with an audit container
identifier.

The registration is a u64 representing the audit container identifier
written to a special file in a pseudo filesystem (proc, since PID tree
already exists) representing a process that will become a parent process
in that container.  This write might place restrictions on mount
namespaces required to define a container, or at least careful checking
of namespaces in the kernel to verify permissions of the orchestrator so
it can't change its own container ID.  A bind mount of nsfs may be
necessary in the container orchestrator's mount namespace.  This write
can only happen once per process.

Note: The justification for using a u64 is that it minimizes the
information printed in every audit record, reducing bandwidth and limits
comparisons to a single u64 which will be faster and less error-prone.

Require CAP_AUDIT_CONTROL to be able to carry out the registration.  At
that time, record the target container's user-supplied audit container
identifier along with a target container's parent process (which may
become the target container's "init" process) process ID (referenced
from the initial PID namespace) in a new record AUDIT_CONTAINER with a
qualifying op=$action field.

Issue a new auxilliary record AUDIT_CONTAINER_INFO for each valid
container ID present on an auditable action or event.

Forked and cloned processes inherit their parent's audit container
identifier, referenced in the process' task_struct.  Since the audit
container identifier is inherited rather than written, it can still be
written once.  This will prevent tampering while allowing nesting.
(This can be implemented with an internal settable flag upon
registration that does not get copied across a fork/clone.)

Mimic setns(2) and return an error if the process has already initiated
threading or forked since this registration should happen before the
process execution is started by the orchestrator and hence should not
yet have any threads or children.  If this is deemed overly restrictive,
switch all of the target's threads and children to the new containerID.

Trust the orchestrator to judiciously use and restrict CAP_AUDIT_CONTROL.

When a container ceases to exist because the last process in that
container has exited log the fact to balance the registration action.  
(This is likely needed for certification accountability.)

At this point it appears unnecessary to add a container session
identifier since this is all tracked from loginuid and sessionid to
communicate with the container orchestrator to spawn an additional
session into an existing container which would be logged.  It can be
added at a later date without breaking API should it be deemed
necessary.

The following namespace logging actions are not needed for certification
purposes at this point, but are helpful for tracking namespace activity.
These are auxilliary records that are associated with namespace
manipulation syscalls unshare(2), clone(2) and setns(2), so the records
will only show up if explicit syscall rules have been added to document
this activity.

Log the creation of every namespace, inheriting/adding its spawning
process' audit container identifier(s), if applicable.  Include the
spawning and spawned namespace IDs (device and inode number tuples).
[AUDIT_NS_CREATE, AUDIT_NS_DESTROY] [clone(2), unshare(2), setns(2)]
Note: At this point it appears only network namespaces may need to track
container IDs apart from processes since incoming packets may cause an
auditable event before being associated with a process.  Since a
namespace can be shared by processes in different containers, the
namespace will need to track all containers to which it has been
assigned.

Upon registration, the target process' namespace IDs (in the form of a
nsfs device number and inode number tuple) will be recorded in an
AUDIT_NS_INFO auxilliary record.

Log the destruction of every namespace that is no longer used by any
process, including the namespace IDs (device and inode number tuples).
[AUDIT_NS_DESTROY] [process exit, unshare(2), setns(2)]

Issue a new auxilliary record AUDIT_NS_CHANGE listing (opt: op=$action)
the parent and child namespace IDs for any changes to a process'
namespaces. [setns(2)]
Note: It may be possible to combine AUDIT_NS_* record formats and
distinguish them with an op=$action field depending on the fields
required for each message type.

The audit container identifier will need to be reaped from all
implicated namespaces upon the destruction of a container.

This namespace information adds supporting information for tracking
events not attributable to specific processes.

Changelog:

(Upstream V3)
- switch back to u64 (from pmoore, can be expanded to u128 in future if
  need arises without breaking API.  u32 was originally proposed, up to
  c36 discussed)
- write-once, but children inherit audit container identifier and can
  then still be written once
- switch to CAP_AUDIT_CONTROL
- group namespace actions together, auxilliary records to namespace
  operations.

(Upstream V2)
- switch from u64 to u128 UUID
- switch from "signal" and "trigger" to "register"
- restrict registration to single process or force all threads and
  children into same container

- RGB

--
Richard Guy Briggs <rgb@redhat.com>
Sr. S/W Engineer, Kernel Security, Base Operating Systems
Remote, Ottawa, Red Hat Canada
IRC: rgb, SunRaycer
Voice: +1.647.777.2635, Internal: (81) 32635

Re: RFC(V3): Audit Kernel Container IDs

[Simo Sorce]

On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:
> Containers are a userspace concept.  The kernel knows nothing of them.
> 
> The Linux audit system needs a way to be able to track the container
> provenance of events and actions.  Audit needs the kernel's help to do
> this.
> 
> Since the concept of a container is entirely a userspace concept, a
> registration from the userspace container orchestration system initiates
> this.  This will define a point in time and a set of resources
> associated with a particular container with an audit container
> identifier.
> 
> The registration is a u64 representing the audit container identifier
> written to a special file in a pseudo filesystem (proc, since PID tree
> already exists) representing a process that will become a parent process
> in that container.  This write might place restrictions on mount
> namespaces required to define a container, or at least careful checking
> of namespaces in the kernel to verify permissions of the orchestrator so
> it can't change its own container ID.  A bind mount of nsfs may be
> necessary in the container orchestrator's mount namespace.  This write
> can only happen once per process.
> 
> Note: The justification for using a u64 is that it minimizes the
> information printed in every audit record, reducing bandwidth and limits
> comparisons to a single u64 which will be faster and less error-prone.
> 
> Require CAP_AUDIT_CONTROL to be able to carry out the registration.  At
> that time, record the target container's user-supplied audit container
> identifier along with a target container's parent process (which may
> become the target container's "init" process) process ID (referenced
> from the initial PID namespace) in a new record AUDIT_CONTAINER with a
> qualifying op=$action field.
> 
> Issue a new auxilliary record AUDIT_CONTAINER_INFO for each valid
> container ID present on an auditable action or event.
> 
> Forked and cloned processes inherit their parent's audit container
> identifier, referenced in the process' task_struct.  Since the audit
> container identifier is inherited rather than written, it can still be
> written once.  This will prevent tampering while allowing nesting.
> (This can be implemented with an internal settable flag upon
> registration that does not get copied across a fork/clone.)
> 
> Mimic setns(2) and return an error if the process has already initiated
> threading or forked since this registration should happen before the
> process execution is started by the orchestrator and hence should not
> yet have any threads or children.  If this is deemed overly restrictive,
> switch all of the target's threads and children to the new containerID.
> 
> Trust the orchestrator to judiciously use and restrict CAP_AUDIT_CONTROL.
> 
> When a container ceases to exist because the last process in that
> container has exited log the fact to balance the registration action.  
> (This is likely needed for certification accountability.)
> 
> At this point it appears unnecessary to add a container session
> identifier since this is all tracked from loginuid and sessionid to
> communicate with the container orchestrator to spawn an additional
> session into an existing container which would be logged.  It can be
> added at a later date without breaking API should it be deemed
> necessary.
> 
> The following namespace logging actions are not needed for certification
> purposes at this point, but are helpful for tracking namespace activity.
> These are auxilliary records that are associated with namespace
> manipulation syscalls unshare(2), clone(2) and setns(2), so the records
> will only show up if explicit syscall rules have been added to document
> this activity.
> 
> Log the creation of every namespace, inheriting/adding its spawning
> process' audit container identifier(s), if applicable.  Include the
> spawning and spawned namespace IDs (device and inode number tuples).
> [AUDIT_NS_CREATE, AUDIT_NS_DESTROY] [clone(2), unshare(2), setns(2)]
> Note: At this point it appears only network namespaces may need to track
> container IDs apart from processes since incoming packets may cause an
> auditable event before being associated with a process.  Since a
> namespace can be shared by processes in different containers, the
> namespace will need to track all containers to which it has been
> assigned.
> 
> Upon registration, the target process' namespace IDs (in the form of a
> nsfs device number and inode number tuple) will be recorded in an
> AUDIT_NS_INFO auxilliary record.
> 
> Log the destruction of every namespace that is no longer used by any
> process, including the namespace IDs (device and inode number tuples).
> [AUDIT_NS_DESTROY] [process exit, unshare(2), setns(2)]
> 
> Issue a new auxilliary record AUDIT_NS_CHANGE listing (opt: op=$action)
> the parent and child namespace IDs for any changes to a process'
> namespaces. [setns(2)]
> Note: It may be possible to combine AUDIT_NS_* record formats and
> distinguish them with an op=$action field depending on the fields
> required for each message type.
> 
> The audit container identifier will need to be reaped from all
> implicated namespaces upon the destruction of a container.
> 
> This namespace information adds supporting information for tracking
> events not attributable to specific processes.
> 
> Changelog:
> 
> (Upstream V3)
> - switch back to u64 (from pmoore, can be expanded to u128 in future if
>   need arises without breaking API.  u32 was originally proposed, up to
>   c36 discussed)
> - write-once, but children inherit audit container identifier and can
>   then still be written once
> - switch to CAP_AUDIT_CONTROL
> - group namespace actions together, auxilliary records to namespace
>   operations.
> 
> (Upstream V2)
> - switch from u64 to u128 UUID
> - switch from "signal" and "trigger" to "register"
> - restrict registration to single process or force all threads and
>   children into same container

I am trying to understand the back and forth on the ID size.

From an orchestrator POV anything that requires tracking a node
specific ID is not ideal.

Orchestrators tend to span many nodes, and containers tend to have IDs
that are either UUID or have a Hash (like SHA256) as identifier.

The problem here is two-fold:

a) Your auditing requires some mapping to be useful outside of the
system.
If you aggreggate audit logs outside of the system or you want to
correlate the system audit logs with other components dealing with
containers, now you need a place where you provide a mapping from your
audit u64 to the ID a container has in the rest of the system.

b) Now you need a mapping of some sort. The simplest way a container
orchestrator can go about this is to just use the UUID or Hash
representing their view of the container, truncate it to a u64 and use
that for Audit. This means there are some chances there will be a
collision and a duplicate u64 ID will be used by the orchestrator as
the container ID. What happen in that case ?

Simo.

-- 
Simo Sorce
Sr. Principal Software Engineer
Red Hat, Inc

Re: RFC(V3): Audit Kernel Container IDs

[Simo Sorce]

On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:
> Containers are a userspace concept.  The kernel knows nothing of them.
> 
> The Linux audit system needs a way to be able to track the container
> provenance of events and actions.  Audit needs the kernel's help to do
> this.
> 
> Since the concept of a container is entirely a userspace concept, a
> registration from the userspace container orchestration system initiates
> this.  This will define a point in time and a set of resources
> associated with a particular container with an audit container
> identifier.
> 
> The registration is a u64 representing the audit container identifier
> written to a special file in a pseudo filesystem (proc, since PID tree
> already exists) representing a process that will become a parent process
> in that container.  This write might place restrictions on mount
> namespaces required to define a container, or at least careful checking
> of namespaces in the kernel to verify permissions of the orchestrator so
> it can't change its own container ID.  A bind mount of nsfs may be
> necessary in the container orchestrator's mount namespace.  This write
> can only happen once per process.
> 
> Note: The justification for using a u64 is that it minimizes the
> information printed in every audit record, reducing bandwidth and limits
> comparisons to a single u64 which will be faster and less error-prone.
> 
> Require CAP_AUDIT_CONTROL to be able to carry out the registration.  At
> that time, record the target container's user-supplied audit container
> identifier along with a target container's parent process (which may
> become the target container's "init" process) process ID (referenced
> from the initial PID namespace) in a new record AUDIT_CONTAINER with a
> qualifying op=$action field.
> 
> Issue a new auxilliary record AUDIT_CONTAINER_INFO for each valid
> container ID present on an auditable action or event.
> 
> Forked and cloned processes inherit their parent's audit container
> identifier, referenced in the process' task_struct.  Since the audit
> container identifier is inherited rather than written, it can still be
> written once.  This will prevent tampering while allowing nesting.
> (This can be implemented with an internal settable flag upon
> registration that does not get copied across a fork/clone.)
> 
> Mimic setns(2) and return an error if the process has already initiated
> threading or forked since this registration should happen before the
> process execution is started by the orchestrator and hence should not
> yet have any threads or children.  If this is deemed overly restrictive,
> switch all of the target's threads and children to the new containerID.
> 
> Trust the orchestrator to judiciously use and restrict CAP_AUDIT_CONTROL.
> 
> When a container ceases to exist because the last process in that
> container has exited log the fact to balance the registration action.  
> (This is likely needed for certification accountability.)
> 
> At this point it appears unnecessary to add a container session
> identifier since this is all tracked from loginuid and sessionid to
> communicate with the container orchestrator to spawn an additional
> session into an existing container which would be logged.  It can be
> added at a later date without breaking API should it be deemed
> necessary.
> 
> The following namespace logging actions are not needed for certification
> purposes at this point, but are helpful for tracking namespace activity.
> These are auxilliary records that are associated with namespace
> manipulation syscalls unshare(2), clone(2) and setns(2), so the records
> will only show up if explicit syscall rules have been added to document
> this activity.
> 
> Log the creation of every namespace, inheriting/adding its spawning
> process' audit container identifier(s), if applicable.  Include the
> spawning and spawned namespace IDs (device and inode number tuples).
> [AUDIT_NS_CREATE, AUDIT_NS_DESTROY] [clone(2), unshare(2), setns(2)]
> Note: At this point it appears only network namespaces may need to track
> container IDs apart from processes since incoming packets may cause an
> auditable event before being associated with a process.  Since a
> namespace can be shared by processes in different containers, the
> namespace will need to track all containers to which it has been
> assigned.
> 
> Upon registration, the target process' namespace IDs (in the form of a
> nsfs device number and inode number tuple) will be recorded in an
> AUDIT_NS_INFO auxilliary record.
> 
> Log the destruction of every namespace that is no longer used by any
> process, including the namespace IDs (device and inode number tuples).
> [AUDIT_NS_DESTROY] [process exit, unshare(2), setns(2)]
> 
> Issue a new auxilliary record AUDIT_NS_CHANGE listing (opt: op=$action)
> the parent and child namespace IDs for any changes to a process'
> namespaces. [setns(2)]
> Note: It may be possible to combine AUDIT_NS_* record formats and
> distinguish them with an op=$action field depending on the fields
> required for each message type.
> 
> The audit container identifier will need to be reaped from all
> implicated namespaces upon the destruction of a container.
> 
> This namespace information adds supporting information for tracking
> events not attributable to specific processes.
> 
> Changelog:
> 
> (Upstream V3)
> - switch back to u64 (from pmoore, can be expanded to u128 in future if
>   need arises without breaking API.  u32 was originally proposed, up to
>   c36 discussed)
> - write-once, but children inherit audit container identifier and can
>   then still be written once
> - switch to CAP_AUDIT_CONTROL
> - group namespace actions together, auxilliary records to namespace
>   operations.
> 
> (Upstream V2)
> - switch from u64 to u128 UUID
> - switch from "signal" and "trigger" to "register"
> - restrict registration to single process or force all threads and
>   children into same container

I am trying to understand the back and forth on the ID size.

>From an orchestrator POV anything that requires tracking a node
specific ID is not ideal.

Orchestrators tend to span many nodes, and containers tend to have IDs
that are either UUID or have a Hash (like SHA256) as identifier.

The problem here is two-fold:

a) Your auditing requires some mapping to be useful outside of the
system.
If you aggreggate audit logs outside of the system or you want to
correlate the system audit logs with other components dealing with
containers, now you need a place where you provide a mapping from your
audit u64 to the ID a container has in the rest of the system.

b) Now you need a mapping of some sort. The simplest way a container
orchestrator can go about this is to just use the UUID or Hash
representing their view of the container, truncate it to a u64 and use
that for Audit. This means there are some chances there will be a
collision and a duplicate u64 ID will be used by the orchestrator as
the container ID. What happen in that case ?

Simo.

-- 
Simo Sorce
Sr. Principal Software Engineer
Red Hat, Inc

Re: RFC(V3): Audit Kernel Container IDs

[Simo Sorce]

On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:
> Containers are a userspace concept.  The kernel knows nothing of them.
> 
> The Linux audit system needs a way to be able to track the container
> provenance of events and actions.  Audit needs the kernel's help to do
> this.
> 
> Since the concept of a container is entirely a userspace concept, a
> registration from the userspace container orchestration system initiates
> this.  This will define a point in time and a set of resources
> associated with a particular container with an audit container
> identifier.
> 
> The registration is a u64 representing the audit container identifier
> written to a special file in a pseudo filesystem (proc, since PID tree
> already exists) representing a process that will become a parent process
> in that container.  This write might place restrictions on mount
> namespaces required to define a container, or at least careful checking
> of namespaces in the kernel to verify permissions of the orchestrator so
> it can't change its own container ID.  A bind mount of nsfs may be
> necessary in the container orchestrator's mount namespace.  This write
> can only happen once per process.
> 
> Note: The justification for using a u64 is that it minimizes the
> information printed in every audit record, reducing bandwidth and limits
> comparisons to a single u64 which will be faster and less error-prone.
> 
> Require CAP_AUDIT_CONTROL to be able to carry out the registration.  At
> that time, record the target container's user-supplied audit container
> identifier along with a target container's parent process (which may
> become the target container's "init" process) process ID (referenced
> from the initial PID namespace) in a new record AUDIT_CONTAINER with a
> qualifying op=$action field.
> 
> Issue a new auxilliary record AUDIT_CONTAINER_INFO for each valid
> container ID present on an auditable action or event.
> 
> Forked and cloned processes inherit their parent's audit container
> identifier, referenced in the process' task_struct.  Since the audit
> container identifier is inherited rather than written, it can still be
> written once.  This will prevent tampering while allowing nesting.
> (This can be implemented with an internal settable flag upon
> registration that does not get copied across a fork/clone.)
> 
> Mimic setns(2) and return an error if the process has already initiated
> threading or forked since this registration should happen before the
> process execution is started by the orchestrator and hence should not
> yet have any threads or children.  If this is deemed overly restrictive,
> switch all of the target's threads and children to the new containerID.
> 
> Trust the orchestrator to judiciously use and restrict CAP_AUDIT_CONTROL.
> 
> When a container ceases to exist because the last process in that
> container has exited log the fact to balance the registration action.  
> (This is likely needed for certification accountability.)
> 
> At this point it appears unnecessary to add a container session
> identifier since this is all tracked from loginuid and sessionid to
> communicate with the container orchestrator to spawn an additional
> session into an existing container which would be logged.  It can be
> added at a later date without breaking API should it be deemed
> necessary.
> 
> The following namespace logging actions are not needed for certification
> purposes at this point, but are helpful for tracking namespace activity.
> These are auxilliary records that are associated with namespace
> manipulation syscalls unshare(2), clone(2) and setns(2), so the records
> will only show up if explicit syscall rules have been added to document
> this activity.
> 
> Log the creation of every namespace, inheriting/adding its spawning
> process' audit container identifier(s), if applicable.  Include the
> spawning and spawned namespace IDs (device and inode number tuples).
> [AUDIT_NS_CREATE, AUDIT_NS_DESTROY] [clone(2), unshare(2), setns(2)]
> Note: At this point it appears only network namespaces may need to track
> container IDs apart from processes since incoming packets may cause an
> auditable event before being associated with a process.  Since a
> namespace can be shared by processes in different containers, the
> namespace will need to track all containers to which it has been
> assigned.
> 
> Upon registration, the target process' namespace IDs (in the form of a
> nsfs device number and inode number tuple) will be recorded in an
> AUDIT_NS_INFO auxilliary record.
> 
> Log the destruction of every namespace that is no longer used by any
> process, including the namespace IDs (device and inode number tuples).
> [AUDIT_NS_DESTROY] [process exit, unshare(2), setns(2)]
> 
> Issue a new auxilliary record AUDIT_NS_CHANGE listing (opt: op=$action)
> the parent and child namespace IDs for any changes to a process'
> namespaces. [setns(2)]
> Note: It may be possible to combine AUDIT_NS_* record formats and
> distinguish them with an op=$action field depending on the fields
> required for each message type.
> 
> The audit container identifier will need to be reaped from all
> implicated namespaces upon the destruction of a container.
> 
> This namespace information adds supporting information for tracking
> events not attributable to specific processes.
> 
> Changelog:
> 
> (Upstream V3)
> - switch back to u64 (from pmoore, can be expanded to u128 in future if
>   need arises without breaking API.  u32 was originally proposed, up to
>   c36 discussed)
> - write-once, but children inherit audit container identifier and can
>   then still be written once
> - switch to CAP_AUDIT_CONTROL
> - group namespace actions together, auxilliary records to namespace
>   operations.
> 
> (Upstream V2)
> - switch from u64 to u128 UUID
> - switch from "signal" and "trigger" to "register"
> - restrict registration to single process or force all threads and
>   children into same container

I am trying to understand the back and forth on the ID size.

>From an orchestrator POV anything that requires tracking a node
specific ID is not ideal.

Orchestrators tend to span many nodes, and containers tend to have IDs
that are either UUID or have a Hash (like SHA256) as identifier.

The problem here is two-fold:

a) Your auditing requires some mapping to be useful outside of the
system.
If you aggreggate audit logs outside of the system or you want to
correlate the system audit logs with other components dealing with
containers, now you need a place where you provide a mapping from your
audit u64 to the ID a container has in the rest of the system.

b) Now you need a mapping of some sort. The simplest way a container
orchestrator can go about this is to just use the UUID or Hash
representing their view of the container, truncate it to a u64 and use
that for Audit. This means there are some chances there will be a
collision and a duplicate u64 ID will be used by the orchestrator as
the container ID. What happen in that case ?

Simo.

-- 
Simo Sorce
Sr. Principal Software Engineer
Red Hat, Inc

Re: RFC(V3): Audit Kernel Container IDs

[Richard Guy Briggs]

On 2018-01-09 11:18, Simo Sorce wrote:
> On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:
> > Containers are a userspace concept.  The kernel knows nothing of them.
> > 
> > The Linux audit system needs a way to be able to track the container
> > provenance of events and actions.  Audit needs the kernel's help to do
> > this.
> > 
> > Since the concept of a container is entirely a userspace concept, a
> > registration from the userspace container orchestration system initiates
> > this.  This will define a point in time and a set of resources
> > associated with a particular container with an audit container
> > identifier.
> > 
> > The registration is a u64 representing the audit container identifier
> > written to a special file in a pseudo filesystem (proc, since PID tree
> > already exists) representing a process that will become a parent process
> > in that container.  This write might place restrictions on mount
> > namespaces required to define a container, or at least careful checking
> > of namespaces in the kernel to verify permissions of the orchestrator so
> > it can't change its own container ID.  A bind mount of nsfs may be
> > necessary in the container orchestrator's mount namespace.  This write
> > can only happen once per process.
> > 
> > Note: The justification for using a u64 is that it minimizes the
> > information printed in every audit record, reducing bandwidth and limits
> > comparisons to a single u64 which will be faster and less error-prone.
> > 
> > Require CAP_AUDIT_CONTROL to be able to carry out the registration.  At
> > that time, record the target container's user-supplied audit container
> > identifier along with a target container's parent process (which may
> > become the target container's "init" process) process ID (referenced
> > from the initial PID namespace) in a new record AUDIT_CONTAINER with a
> > qualifying op=$action field.
> > 
> > Issue a new auxilliary record AUDIT_CONTAINER_INFO for each valid
> > container ID present on an auditable action or event.
> > 
> > Forked and cloned processes inherit their parent's audit container
> > identifier, referenced in the process' task_struct.  Since the audit
> > container identifier is inherited rather than written, it can still be
> > written once.  This will prevent tampering while allowing nesting.
> > (This can be implemented with an internal settable flag upon
> > registration that does not get copied across a fork/clone.)
> > 
> > Mimic setns(2) and return an error if the process has already initiated
> > threading or forked since this registration should happen before the
> > process execution is started by the orchestrator and hence should not
> > yet have any threads or children.  If this is deemed overly restrictive,
> > switch all of the target's threads and children to the new containerID.
> > 
> > Trust the orchestrator to judiciously use and restrict CAP_AUDIT_CONTROL.
> > 
> > When a container ceases to exist because the last process in that
> > container has exited log the fact to balance the registration action.  
> > (This is likely needed for certification accountability.)
> > 
> > At this point it appears unnecessary to add a container session
> > identifier since this is all tracked from loginuid and sessionid to
> > communicate with the container orchestrator to spawn an additional
> > session into an existing container which would be logged.  It can be
> > added at a later date without breaking API should it be deemed
> > necessary.
> > 
> > The following namespace logging actions are not needed for certification
> > purposes at this point, but are helpful for tracking namespace activity.
> > These are auxilliary records that are associated with namespace
> > manipulation syscalls unshare(2), clone(2) and setns(2), so the records
> > will only show up if explicit syscall rules have been added to document
> > this activity.
> > 
> > Log the creation of every namespace, inheriting/adding its spawning
> > process' audit container identifier(s), if applicable.  Include the
> > spawning and spawned namespace IDs (device and inode number tuples).
> > [AUDIT_NS_CREATE, AUDIT_NS_DESTROY] [clone(2), unshare(2), setns(2)]
> > Note: At this point it appears only network namespaces may need to track
> > container IDs apart from processes since incoming packets may cause an
> > auditable event before being associated with a process.  Since a
> > namespace can be shared by processes in different containers, the
> > namespace will need to track all containers to which it has been
> > assigned.
> > 
> > Upon registration, the target process' namespace IDs (in the form of a
> > nsfs device number and inode number tuple) will be recorded in an
> > AUDIT_NS_INFO auxilliary record.
> > 
> > Log the destruction of every namespace that is no longer used by any
> > process, including the namespace IDs (device and inode number tuples).
> > [AUDIT_NS_DESTROY] [process exit, unshare(2), setns(2)]
> > 
> > Issue a new auxilliary record AUDIT_NS_CHANGE listing (opt: op=$action)
> > the parent and child namespace IDs for any changes to a process'
> > namespaces. [setns(2)]
> > Note: It may be possible to combine AUDIT_NS_* record formats and
> > distinguish them with an op=$action field depending on the fields
> > required for each message type.
> > 
> > The audit container identifier will need to be reaped from all
> > implicated namespaces upon the destruction of a container.
> > 
> > This namespace information adds supporting information for tracking
> > events not attributable to specific processes.
> > 
> > Changelog:
> > 
> > (Upstream V3)
> > - switch back to u64 (from pmoore, can be expanded to u128 in future if
> >   need arises without breaking API.  u32 was originally proposed, up to
> >   c36 discussed)
> > - write-once, but children inherit audit container identifier and can
> >   then still be written once
> > - switch to CAP_AUDIT_CONTROL
> > - group namespace actions together, auxilliary records to namespace
> >   operations.
> > 
> > (Upstream V2)
> > - switch from u64 to u128 UUID
> > - switch from "signal" and "trigger" to "register"
> > - restrict registration to single process or force all threads and
> >   children into same container
> 
> I am trying to understand the back and forth on the ID size.
> 
> From an orchestrator POV anything that requires tracking a node
> specific ID is not ideal.
> 
> Orchestrators tend to span many nodes, and containers tend to have IDs
> that are either UUID or have a Hash (like SHA256) as identifier.
> 
> The problem here is two-fold:
> 
> a) Your auditing requires some mapping to be useful outside of the
> system.
> If you aggreggate audit logs outside of the system or you want to
> correlate the system audit logs with other components dealing with
> containers, now you need a place where you provide a mapping from your
> audit u64 to the ID a container has in the rest of the system.
> 
> b) Now you need a mapping of some sort. The simplest way a container
> orchestrator can go about this is to just use the UUID or Hash
> representing their view of the container, truncate it to a u64 and use
> that for Audit. This means there are some chances there will be a
> collision and a duplicate u64 ID will be used by the orchestrator as
> the container ID. What happen in that case ?

Paul, can you justify this somewhat larger inconvenience for some
relatively minor convenience on our part?  u64 vs u128 is easy for us to
accomodate in terms of scalar comparisons.  It doubles the information
in every container id field we print in audit records.  A c36 is a
bigger step.

> Simo.

- RGB

--
Richard Guy Briggs <rgb@redhat.com>
Sr. S/W Engineer, Kernel Security, Base Operating Systems
Remote, Ottawa, Red Hat Canada
IRC: rgb, SunRaycer
Voice: +1.647.777.2635, Internal: (81) 32635

Re: RFC(V3): Audit Kernel Container IDs

[Richard Guy Briggs]

On 2018-01-09 11:18, Simo Sorce wrote:
> On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:
> > Containers are a userspace concept.  The kernel knows nothing of them.
> > 
> > The Linux audit system needs a way to be able to track the container
> > provenance of events and actions.  Audit needs the kernel's help to do
> > this.
> > 
> > Since the concept of a container is entirely a userspace concept, a
> > registration from the userspace container orchestration system initiates
> > this.  This will define a point in time and a set of resources
> > associated with a particular container with an audit container
> > identifier.
> > 
> > The registration is a u64 representing the audit container identifier
> > written to a special file in a pseudo filesystem (proc, since PID tree
> > already exists) representing a process that will become a parent process
> > in that container.  This write might place restrictions on mount
> > namespaces required to define a container, or at least careful checking
> > of namespaces in the kernel to verify permissions of the orchestrator so
> > it can't change its own container ID.  A bind mount of nsfs may be
> > necessary in the container orchestrator's mount namespace.  This write
> > can only happen once per process.
> > 
> > Note: The justification for using a u64 is that it minimizes the
> > information printed in every audit record, reducing bandwidth and limits
> > comparisons to a single u64 which will be faster and less error-prone.
> > 
> > Require CAP_AUDIT_CONTROL to be able to carry out the registration.  At
> > that time, record the target container's user-supplied audit container
> > identifier along with a target container's parent process (which may
> > become the target container's "init" process) process ID (referenced
> > from the initial PID namespace) in a new record AUDIT_CONTAINER with a
> > qualifying op=$action field.
> > 
> > Issue a new auxilliary record AUDIT_CONTAINER_INFO for each valid
> > container ID present on an auditable action or event.
> > 
> > Forked and cloned processes inherit their parent's audit container
> > identifier, referenced in the process' task_struct.  Since the audit
> > container identifier is inherited rather than written, it can still be
> > written once.  This will prevent tampering while allowing nesting.
> > (This can be implemented with an internal settable flag upon
> > registration that does not get copied across a fork/clone.)
> > 
> > Mimic setns(2) and return an error if the process has already initiated
> > threading or forked since this registration should happen before the
> > process execution is started by the orchestrator and hence should not
> > yet have any threads or children.  If this is deemed overly restrictive,
> > switch all of the target's threads and children to the new containerID.
> > 
> > Trust the orchestrator to judiciously use and restrict CAP_AUDIT_CONTROL.
> > 
> > When a container ceases to exist because the last process in that
> > container has exited log the fact to balance the registration action.  
> > (This is likely needed for certification accountability.)
> > 
> > At this point it appears unnecessary to add a container session
> > identifier since this is all tracked from loginuid and sessionid to
> > communicate with the container orchestrator to spawn an additional
> > session into an existing container which would be logged.  It can be
> > added at a later date without breaking API should it be deemed
> > necessary.
> > 
> > The following namespace logging actions are not needed for certification
> > purposes at this point, but are helpful for tracking namespace activity.
> > These are auxilliary records that are associated with namespace
> > manipulation syscalls unshare(2), clone(2) and setns(2), so the records
> > will only show up if explicit syscall rules have been added to document
> > this activity.
> > 
> > Log the creation of every namespace, inheriting/adding its spawning
> > process' audit container identifier(s), if applicable.  Include the
> > spawning and spawned namespace IDs (device and inode number tuples).
> > [AUDIT_NS_CREATE, AUDIT_NS_DESTROY] [clone(2), unshare(2), setns(2)]
> > Note: At this point it appears only network namespaces may need to track
> > container IDs apart from processes since incoming packets may cause an
> > auditable event before being associated with a process.  Since a
> > namespace can be shared by processes in different containers, the
> > namespace will need to track all containers to which it has been
> > assigned.
> > 
> > Upon registration, the target process' namespace IDs (in the form of a
> > nsfs device number and inode number tuple) will be recorded in an
> > AUDIT_NS_INFO auxilliary record.
> > 
> > Log the destruction of every namespace that is no longer used by any
> > process, including the namespace IDs (device and inode number tuples).
> > [AUDIT_NS_DESTROY] [process exit, unshare(2), setns(2)]
> > 
> > Issue a new auxilliary record AUDIT_NS_CHANGE listing (opt: op=$action)
> > the parent and child namespace IDs for any changes to a process'
> > namespaces. [setns(2)]
> > Note: It may be possible to combine AUDIT_NS_* record formats and
> > distinguish them with an op=$action field depending on the fields
> > required for each message type.
> > 
> > The audit container identifier will need to be reaped from all
> > implicated namespaces upon the destruction of a container.
> > 
> > This namespace information adds supporting information for tracking
> > events not attributable to specific processes.
> > 
> > Changelog:
> > 
> > (Upstream V3)
> > - switch back to u64 (from pmoore, can be expanded to u128 in future if
> >   need arises without breaking API.  u32 was originally proposed, up to
> >   c36 discussed)
> > - write-once, but children inherit audit container identifier and can
> >   then still be written once
> > - switch to CAP_AUDIT_CONTROL
> > - group namespace actions together, auxilliary records to namespace
> >   operations.
> > 
> > (Upstream V2)
> > - switch from u64 to u128 UUID
> > - switch from "signal" and "trigger" to "register"
> > - restrict registration to single process or force all threads and
> >   children into same container
> 
> I am trying to understand the back and forth on the ID size.
> 
> From an orchestrator POV anything that requires tracking a node
> specific ID is not ideal.
> 
> Orchestrators tend to span many nodes, and containers tend to have IDs
> that are either UUID or have a Hash (like SHA256) as identifier.
> 
> The problem here is two-fold:
> 
> a) Your auditing requires some mapping to be useful outside of the
> system.
> If you aggreggate audit logs outside of the system or you want to
> correlate the system audit logs with other components dealing with
> containers, now you need a place where you provide a mapping from your
> audit u64 to the ID a container has in the rest of the system.
> 
> b) Now you need a mapping of some sort. The simplest way a container
> orchestrator can go about this is to just use the UUID or Hash
> representing their view of the container, truncate it to a u64 and use
> that for Audit. This means there are some chances there will be a
> collision and a duplicate u64 ID will be used by the orchestrator as
> the container ID. What happen in that case ?

Paul, can you justify this somewhat larger inconvenience for some
relatively minor convenience on our part?  u64 vs u128 is easy for us to
accomodate in terms of scalar comparisons.  It doubles the information
in every container id field we print in audit records.  A c36 is a
bigger step.

> Simo.

- RGB

--
Richard Guy Briggs <rgb@redhat.com>
Sr. S/W Engineer, Kernel Security, Base Operating Systems
Remote, Ottawa, Red Hat Canada
IRC: rgb, SunRaycer
Voice: +1.647.777.2635, Internal: (81) 32635

Re: RFC(V3): Audit Kernel Container IDs

[Richard Guy Briggs]

On 2018-01-09 11:18, Simo Sorce wrote:
> On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:
> > Containers are a userspace concept.  The kernel knows nothing of them.
> > 
> > The Linux audit system needs a way to be able to track the container
> > provenance of events and actions.  Audit needs the kernel's help to do
> > this.
> > 
> > Since the concept of a container is entirely a userspace concept, a
> > registration from the userspace container orchestration system initiates
> > this.  This will define a point in time and a set of resources
> > associated with a particular container with an audit container
> > identifier.
> > 
> > The registration is a u64 representing the audit container identifier
> > written to a special file in a pseudo filesystem (proc, since PID tree
> > already exists) representing a process that will become a parent process
> > in that container.  This write might place restrictions on mount
> > namespaces required to define a container, or at least careful checking
> > of namespaces in the kernel to verify permissions of the orchestrator so
> > it can't change its own container ID.  A bind mount of nsfs may be
> > necessary in the container orchestrator's mount namespace.  This write
> > can only happen once per process.
> > 
> > Note: The justification for using a u64 is that it minimizes the
> > information printed in every audit record, reducing bandwidth and limits
> > comparisons to a single u64 which will be faster and less error-prone.
> > 
> > Require CAP_AUDIT_CONTROL to be able to carry out the registration.  At
> > that time, record the target container's user-supplied audit container
> > identifier along with a target container's parent process (which may
> > become the target container's "init" process) process ID (referenced
> > from the initial PID namespace) in a new record AUDIT_CONTAINER with a
> > qualifying op=$action field.
> > 
> > Issue a new auxilliary record AUDIT_CONTAINER_INFO for each valid
> > container ID present on an auditable action or event.
> > 
> > Forked and cloned processes inherit their parent's audit container
> > identifier, referenced in the process' task_struct.  Since the audit
> > container identifier is inherited rather than written, it can still be
> > written once.  This will prevent tampering while allowing nesting.
> > (This can be implemented with an internal settable flag upon
> > registration that does not get copied across a fork/clone.)
> > 
> > Mimic setns(2) and return an error if the process has already initiated
> > threading or forked since this registration should happen before the
> > process execution is started by the orchestrator and hence should not
> > yet have any threads or children.  If this is deemed overly restrictive,
> > switch all of the target's threads and children to the new containerID.
> > 
> > Trust the orchestrator to judiciously use and restrict CAP_AUDIT_CONTROL.
> > 
> > When a container ceases to exist because the last process in that
> > container has exited log the fact to balance the registration action.  
> > (This is likely needed for certification accountability.)
> > 
> > At this point it appears unnecessary to add a container session
> > identifier since this is all tracked from loginuid and sessionid to
> > communicate with the container orchestrator to spawn an additional
> > session into an existing container which would be logged.  It can be
> > added at a later date without breaking API should it be deemed
> > necessary.
> > 
> > The following namespace logging actions are not needed for certification
> > purposes at this point, but are helpful for tracking namespace activity.
> > These are auxilliary records that are associated with namespace
> > manipulation syscalls unshare(2), clone(2) and setns(2), so the records
> > will only show up if explicit syscall rules have been added to document
> > this activity.
> > 
> > Log the creation of every namespace, inheriting/adding its spawning
> > process' audit container identifier(s), if applicable.  Include the
> > spawning and spawned namespace IDs (device and inode number tuples).
> > [AUDIT_NS_CREATE, AUDIT_NS_DESTROY] [clone(2), unshare(2), setns(2)]
> > Note: At this point it appears only network namespaces may need to track
> > container IDs apart from processes since incoming packets may cause an
> > auditable event before being associated with a process.  Since a
> > namespace can be shared by processes in different containers, the
> > namespace will need to track all containers to which it has been
> > assigned.
> > 
> > Upon registration, the target process' namespace IDs (in the form of a
> > nsfs device number and inode number tuple) will be recorded in an
> > AUDIT_NS_INFO auxilliary record.
> > 
> > Log the destruction of every namespace that is no longer used by any
> > process, including the namespace IDs (device and inode number tuples).
> > [AUDIT_NS_DESTROY] [process exit, unshare(2), setns(2)]
> > 
> > Issue a new auxilliary record AUDIT_NS_CHANGE listing (opt: op=$action)
> > the parent and child namespace IDs for any changes to a process'
> > namespaces. [setns(2)]
> > Note: It may be possible to combine AUDIT_NS_* record formats and
> > distinguish them with an op=$action field depending on the fields
> > required for each message type.
> > 
> > The audit container identifier will need to be reaped from all
> > implicated namespaces upon the destruction of a container.
> > 
> > This namespace information adds supporting information for tracking
> > events not attributable to specific processes.
> > 
> > Changelog:
> > 
> > (Upstream V3)
> > - switch back to u64 (from pmoore, can be expanded to u128 in future if
> >   need arises without breaking API.  u32 was originally proposed, up to
> >   c36 discussed)
> > - write-once, but children inherit audit container identifier and can
> >   then still be written once
> > - switch to CAP_AUDIT_CONTROL
> > - group namespace actions together, auxilliary records to namespace
> >   operations.
> > 
> > (Upstream V2)
> > - switch from u64 to u128 UUID
> > - switch from "signal" and "trigger" to "register"
> > - restrict registration to single process or force all threads and
> >   children into same container
> 
> I am trying to understand the back and forth on the ID size.
> 
> From an orchestrator POV anything that requires tracking a node
> specific ID is not ideal.
> 
> Orchestrators tend to span many nodes, and containers tend to have IDs
> that are either UUID or have a Hash (like SHA256) as identifier.
> 
> The problem here is two-fold:
> 
> a) Your auditing requires some mapping to be useful outside of the
> system.
> If you aggreggate audit logs outside of the system or you want to
> correlate the system audit logs with other components dealing with
> containers, now you need a place where you provide a mapping from your
> audit u64 to the ID a container has in the rest of the system.
> 
> b) Now you need a mapping of some sort. The simplest way a container
> orchestrator can go about this is to just use the UUID or Hash
> representing their view of the container, truncate it to a u64 and use
> that for Audit. This means there are some chances there will be a
> collision and a duplicate u64 ID will be used by the orchestrator as
> the container ID. What happen in that case ?

Paul, can you justify this somewhat larger inconvenience for some
relatively minor convenience on our part?  u64 vs u128 is easy for us to
accomodate in terms of scalar comparisons.  It doubles the information
in every container id field we print in audit records.  A c36 is a
bigger step.

> Simo.

- RGB

--
Richard Guy Briggs <rgb@redhat.com>
Sr. S/W Engineer, Kernel Security, Base Operating Systems
Remote, Ottawa, Red Hat Canada
IRC: rgb, SunRaycer
Voice: +1.647.777.2635, Internal: (81) 32635

Re: RFC(V3): Audit Kernel Container IDs

[Paul Moore]

On Wed, Jan 10, 2018 at 2:00 AM, Richard Guy Briggs <rgb@redhat.com> wrote:
> On 2018-01-09 11:18, Simo Sorce wrote:
>> On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:
>> > Containers are a userspace concept.  The kernel knows nothing of them.
>> >
>> > The Linux audit system needs a way to be able to track the container
>> > provenance of events and actions.  Audit needs the kernel's help to do
>> > this.
>> >
>> > Since the concept of a container is entirely a userspace concept, a
>> > registration from the userspace container orchestration system initiates
>> > this.  This will define a point in time and a set of resources
>> > associated with a particular container with an audit container
>> > identifier.
>> >
>> > The registration is a u64 representing the audit container identifier
>> > written to a special file in a pseudo filesystem (proc, since PID tree
>> > already exists) representing a process that will become a parent process
>> > in that container.  This write might place restrictions on mount
>> > namespaces required to define a container, or at least careful checking
>> > of namespaces in the kernel to verify permissions of the orchestrator so
>> > it can't change its own container ID.  A bind mount of nsfs may be
>> > necessary in the container orchestrator's mount namespace.  This write
>> > can only happen once per process.
>> >
>> > Note: The justification for using a u64 is that it minimizes the
>> > information printed in every audit record, reducing bandwidth and limits
>> > comparisons to a single u64 which will be faster and less error-prone.
>> >
>> > Require CAP_AUDIT_CONTROL to be able to carry out the registration.  At
>> > that time, record the target container's user-supplied audit container
>> > identifier along with a target container's parent process (which may
>> > become the target container's "init" process) process ID (referenced
>> > from the initial PID namespace) in a new record AUDIT_CONTAINER with a
>> > qualifying op=$action field.
>> >
>> > Issue a new auxilliary record AUDIT_CONTAINER_INFO for each valid
>> > container ID present on an auditable action or event.
>> >
>> > Forked and cloned processes inherit their parent's audit container
>> > identifier, referenced in the process' task_struct.  Since the audit
>> > container identifier is inherited rather than written, it can still be
>> > written once.  This will prevent tampering while allowing nesting.
>> > (This can be implemented with an internal settable flag upon
>> > registration that does not get copied across a fork/clone.)
>> >
>> > Mimic setns(2) and return an error if the process has already initiated
>> > threading or forked since this registration should happen before the
>> > process execution is started by the orchestrator and hence should not
>> > yet have any threads or children.  If this is deemed overly restrictive,
>> > switch all of the target's threads and children to the new containerID.
>> >
>> > Trust the orchestrator to judiciously use and restrict CAP_AUDIT_CONTROL.
>> >
>> > When a container ceases to exist because the last process in that
>> > container has exited log the fact to balance the registration action.
>> > (This is likely needed for certification accountability.)
>> >
>> > At this point it appears unnecessary to add a container session
>> > identifier since this is all tracked from loginuid and sessionid to
>> > communicate with the container orchestrator to spawn an additional
>> > session into an existing container which would be logged.  It can be
>> > added at a later date without breaking API should it be deemed
>> > necessary.
>> >
>> > The following namespace logging actions are not needed for certification
>> > purposes at this point, but are helpful for tracking namespace activity.
>> > These are auxilliary records that are associated with namespace
>> > manipulation syscalls unshare(2), clone(2) and setns(2), so the records
>> > will only show up if explicit syscall rules have been added to document
>> > this activity.
>> >
>> > Log the creation of every namespace, inheriting/adding its spawning
>> > process' audit container identifier(s), if applicable.  Include the
>> > spawning and spawned namespace IDs (device and inode number tuples).
>> > [AUDIT_NS_CREATE, AUDIT_NS_DESTROY] [clone(2), unshare(2), setns(2)]
>> > Note: At this point it appears only network namespaces may need to track
>> > container IDs apart from processes since incoming packets may cause an
>> > auditable event before being associated with a process.  Since a
>> > namespace can be shared by processes in different containers, the
>> > namespace will need to track all containers to which it has been
>> > assigned.
>> >
>> > Upon registration, the target process' namespace IDs (in the form of a
>> > nsfs device number and inode number tuple) will be recorded in an
>> > AUDIT_NS_INFO auxilliary record.
>> >
>> > Log the destruction of every namespace that is no longer used by any
>> > process, including the namespace IDs (device and inode number tuples).
>> > [AUDIT_NS_DESTROY] [process exit, unshare(2), setns(2)]
>> >
>> > Issue a new auxilliary record AUDIT_NS_CHANGE listing (opt: op=$action)
>> > the parent and child namespace IDs for any changes to a process'
>> > namespaces. [setns(2)]
>> > Note: It may be possible to combine AUDIT_NS_* record formats and
>> > distinguish them with an op=$action field depending on the fields
>> > required for each message type.
>> >
>> > The audit container identifier will need to be reaped from all
>> > implicated namespaces upon the destruction of a container.
>> >
>> > This namespace information adds supporting information for tracking
>> > events not attributable to specific processes.
>> >
>> > Changelog:
>> >
>> > (Upstream V3)
>> > - switch back to u64 (from pmoore, can be expanded to u128 in future if
>> >   need arises without breaking API.  u32 was originally proposed, up to
>> >   c36 discussed)
>> > - write-once, but children inherit audit container identifier and can
>> >   then still be written once
>> > - switch to CAP_AUDIT_CONTROL
>> > - group namespace actions together, auxilliary records to namespace
>> >   operations.
>> >
>> > (Upstream V2)
>> > - switch from u64 to u128 UUID
>> > - switch from "signal" and "trigger" to "register"
>> > - restrict registration to single process or force all threads and
>> >   children into same container
>>
>> I am trying to understand the back and forth on the ID size.
>>
>> From an orchestrator POV anything that requires tracking a node
>> specific ID is not ideal.
>>
>> Orchestrators tend to span many nodes, and containers tend to have IDs
>> that are either UUID or have a Hash (like SHA256) as identifier.
>>
>> The problem here is two-fold:
>>
>> a) Your auditing requires some mapping to be useful outside of the
>> system.
>> If you aggreggate audit logs outside of the system or you want to
>> correlate the system audit logs with other components dealing with
>> containers, now you need a place where you provide a mapping from your
>> audit u64 to the ID a container has in the rest of the system.
>>
>> b) Now you need a mapping of some sort. The simplest way a container
>> orchestrator can go about this is to just use the UUID or Hash
>> representing their view of the container, truncate it to a u64 and use
>> that for Audit. This means there are some chances there will be a
>> collision and a duplicate u64 ID will be used by the orchestrator as
>> the container ID. What happen in that case ?
>
> Paul, can you justify this somewhat larger inconvenience for some
> relatively minor convenience on our part?

Done in direct response to Simo.

But to be clear Richard, we've talked about this a few times, it's not
a "minor convenience" on our part, it's a pretty big convenience once
we starting having to route audit events and make decisions based on
the audit container ID information.  Audit performance is less than
awesome now, I'm working hard to not make it worse.

> u64 vs u128 is easy for us to
> accomodate in terms of scalar comparisons.  It doubles the information
> in every container id field we print in audit records.

... and slows down audit container ID checks.

> A c36 is a bigger step.

Yeah, we're not doing that, no way.

-- 
paul moore
www.paul-moore.com

Re: RFC(V3): Audit Kernel Container IDs

[Simo Sorce]

On Fri, 2018-02-02 at 16:24 -0500, Paul Moore wrote:
> On Wed, Jan 10, 2018 at 2:00 AM, Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> > On 2018-01-09 11:18, Simo Sorce wrote:
> > > On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:
> > > > Containers are a userspace concept.  The kernel knows nothing of them.
> > > > 
> > > > The Linux audit system needs a way to be able to track the container
> > > > provenance of events and actions.  Audit needs the kernel's help to do
> > > > this.
> > > > 
> > > > Since the concept of a container is entirely a userspace concept, a
> > > > registration from the userspace container orchestration system initiates
> > > > this.  This will define a point in time and a set of resources
> > > > associated with a particular container with an audit container
> > > > identifier.
> > > > 
> > > > The registration is a u64 representing the audit container identifier
> > > > written to a special file in a pseudo filesystem (proc, since PID tree
> > > > already exists) representing a process that will become a parent process
> > > > in that container.  This write might place restrictions on mount
> > > > namespaces required to define a container, or at least careful checking
> > > > of namespaces in the kernel to verify permissions of the orchestrator so
> > > > it can't change its own container ID.  A bind mount of nsfs may be
> > > > necessary in the container orchestrator's mount namespace.  This write
> > > > can only happen once per process.
> > > > 
> > > > Note: The justification for using a u64 is that it minimizes the
> > > > information printed in every audit record, reducing bandwidth and limits
> > > > comparisons to a single u64 which will be faster and less error-prone.
> > > > 
> > > > Require CAP_AUDIT_CONTROL to be able to carry out the registration.  At
> > > > that time, record the target container's user-supplied audit container
> > > > identifier along with a target container's parent process (which may
> > > > become the target container's "init" process) process ID (referenced
> > > > from the initial PID namespace) in a new record AUDIT_CONTAINER with a
> > > > qualifying op=$action field.
> > > > 
> > > > Issue a new auxilliary record AUDIT_CONTAINER_INFO for each valid
> > > > container ID present on an auditable action or event.
> > > > 
> > > > Forked and cloned processes inherit their parent's audit container
> > > > identifier, referenced in the process' task_struct.  Since the audit
> > > > container identifier is inherited rather than written, it can still be
> > > > written once.  This will prevent tampering while allowing nesting.
> > > > (This can be implemented with an internal settable flag upon
> > > > registration that does not get copied across a fork/clone.)
> > > > 
> > > > Mimic setns(2) and return an error if the process has already initiated
> > > > threading or forked since this registration should happen before the
> > > > process execution is started by the orchestrator and hence should not
> > > > yet have any threads or children.  If this is deemed overly restrictive,
> > > > switch all of the target's threads and children to the new containerID.
> > > > 
> > > > Trust the orchestrator to judiciously use and restrict CAP_AUDIT_CONTROL.
> > > > 
> > > > When a container ceases to exist because the last process in that
> > > > container has exited log the fact to balance the registration action.
> > > > (This is likely needed for certification accountability.)
> > > > 
> > > > At this point it appears unnecessary to add a container session
> > > > identifier since this is all tracked from loginuid and sessionid to
> > > > communicate with the container orchestrator to spawn an additional
> > > > session into an existing container which would be logged.  It can be
> > > > added at a later date without breaking API should it be deemed
> > > > necessary.
> > > > 
> > > > The following namespace logging actions are not needed for certification
> > > > purposes at this point, but are helpful for tracking namespace activity.
> > > > These are auxilliary records that are associated with namespace
> > > > manipulation syscalls unshare(2), clone(2) and setns(2), so the records
> > > > will only show up if explicit syscall rules have been added to document
> > > > this activity.
> > > > 
> > > > Log the creation of every namespace, inheriting/adding its spawning
> > > > process' audit container identifier(s), if applicable.  Include the
> > > > spawning and spawned namespace IDs (device and inode number tuples).
> > > > [AUDIT_NS_CREATE, AUDIT_NS_DESTROY] [clone(2), unshare(2), setns(2)]
> > > > Note: At this point it appears only network namespaces may need to track
> > > > container IDs apart from processes since incoming packets may cause an
> > > > auditable event before being associated with a process.  Since a
> > > > namespace can be shared by processes in different containers, the
> > > > namespace will need to track all containers to which it has been
> > > > assigned.
> > > > 
> > > > Upon registration, the target process' namespace IDs (in the form of a
> > > > nsfs device number and inode number tuple) will be recorded in an
> > > > AUDIT_NS_INFO auxilliary record.
> > > > 
> > > > Log the destruction of every namespace that is no longer used by any
> > > > process, including the namespace IDs (device and inode number tuples).
> > > > [AUDIT_NS_DESTROY] [process exit, unshare(2), setns(2)]
> > > > 
> > > > Issue a new auxilliary record AUDIT_NS_CHANGE listing (opt: op=$action)
> > > > the parent and child namespace IDs for any changes to a process'
> > > > namespaces. [setns(2)]
> > > > Note: It may be possible to combine AUDIT_NS_* record formats and
> > > > distinguish them with an op=$action field depending on the fields
> > > > required for each message type.
> > > > 
> > > > The audit container identifier will need to be reaped from all
> > > > implicated namespaces upon the destruction of a container.
> > > > 
> > > > This namespace information adds supporting information for tracking
> > > > events not attributable to specific processes.
> > > > 
> > > > Changelog:
> > > > 
> > > > (Upstream V3)
> > > > - switch back to u64 (from pmoore, can be expanded to u128 in future if
> > > >   need arises without breaking API.  u32 was originally proposed, up to
> > > >   c36 discussed)
> > > > - write-once, but children inherit audit container identifier and can
> > > >   then still be written once
> > > > - switch to CAP_AUDIT_CONTROL
> > > > - group namespace actions together, auxilliary records to namespace
> > > >   operations.
> > > > 
> > > > (Upstream V2)
> > > > - switch from u64 to u128 UUID
> > > > - switch from "signal" and "trigger" to "register"
> > > > - restrict registration to single process or force all threads and
> > > >   children into same container
> > > 
> > > I am trying to understand the back and forth on the ID size.
> > > 
> > > From an orchestrator POV anything that requires tracking a node
> > > specific ID is not ideal.
> > > 
> > > Orchestrators tend to span many nodes, and containers tend to have IDs
> > > that are either UUID or have a Hash (like SHA256) as identifier.
> > > 
> > > The problem here is two-fold:
> > > 
> > > a) Your auditing requires some mapping to be useful outside of the
> > > system.
> > > If you aggreggate audit logs outside of the system or you want to
> > > correlate the system audit logs with other components dealing with
> > > containers, now you need a place where you provide a mapping from your
> > > audit u64 to the ID a container has in the rest of the system.
> > > 
> > > b) Now you need a mapping of some sort. The simplest way a container
> > > orchestrator can go about this is to just use the UUID or Hash
> > > representing their view of the container, truncate it to a u64 and use
> > > that for Audit. This means there are some chances there will be a
> > > collision and a duplicate u64 ID will be used by the orchestrator as
> > > the container ID. What happen in that case ?
> > 
> > Paul, can you justify this somewhat larger inconvenience for some
> > relatively minor convenience on our part?
> 
> Done in direct response to Simo.

Sorry but your response sounds more like waving away then addressing
them, the excuse being: we can't please everyone, so we are going to
please no one.

> But to be clear Richard, we've talked about this a few times, it's not
> a "minor convenience" on our part, it's a pretty big convenience once
> we starting having to route audit events and make decisions based on
> the audit container ID information.  Audit performance is less than
> awesome now, I'm working hard to not make it worse.

Sounds like a security vs performance trade off to me.

> > u64 vs u128 is easy for us to
> > accomodate in terms of scalar comparisons.  It doubles the information
> > in every container id field we print in audit records.
> 
> ... and slows down audit container ID checks.

Are you saying a cmp on a u128 is slower than a comparison on a u64 and
this is something that will be noticeable ?

> > A c36 is a bigger step.
> 
> Yeah, we're not doing that, no way.

Ok, I can see your point though I do not agree with it.

I can see why you do not want to have arbitrary length strings, but a
u128 sounded like a reasonable compromise to me as it has enough room
to be able to have unique cluster-wide IDs which a u64 definitely makes
a lot harder to provide w/o tight coordination.

Simo.

-- 
Simo Sorce
Sr. Principal Software Engineer
Red Hat, Inc

Re: RFC(V3): Audit Kernel Container IDs

[Simo Sorce]

On Fri, 2018-02-02 at 16:24 -0500, Paul Moore wrote:
> On Wed, Jan 10, 2018 at 2:00 AM, Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> > On 2018-01-09 11:18, Simo Sorce wrote:
> > > On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:
> > > > Containers are a userspace concept.  The kernel knows nothing of them.
> > > > 
> > > > The Linux audit system needs a way to be able to track the container
> > > > provenance of events and actions.  Audit needs the kernel's help to do
> > > > this.
> > > > 
> > > > Since the concept of a container is entirely a userspace concept, a
> > > > registration from the userspace container orchestration system initiates
> > > > this.  This will define a point in time and a set of resources
> > > > associated with a particular container with an audit container
> > > > identifier.
> > > > 
> > > > The registration is a u64 representing the audit container identifier
> > > > written to a special file in a pseudo filesystem (proc, since PID tree
> > > > already exists) representing a process that will become a parent process
> > > > in that container.  This write might place restrictions on mount
> > > > namespaces required to define a container, or at least careful checking
> > > > of namespaces in the kernel to verify permissions of the orchestrator so
> > > > it can't change its own container ID.  A bind mount of nsfs may be
> > > > necessary in the container orchestrator's mount namespace.  This write
> > > > can only happen once per process.
> > > > 
> > > > Note: The justification for using a u64 is that it minimizes the
> > > > information printed in every audit record, reducing bandwidth and limits
> > > > comparisons to a single u64 which will be faster and less error-prone.
> > > > 
> > > > Require CAP_AUDIT_CONTROL to be able to carry out the registration.  At
> > > > that time, record the target container's user-supplied audit container
> > > > identifier along with a target container's parent process (which may
> > > > become the target container's "init" process) process ID (referenced
> > > > from the initial PID namespace) in a new record AUDIT_CONTAINER with a
> > > > qualifying op=$action field.
> > > > 
> > > > Issue a new auxilliary record AUDIT_CONTAINER_INFO for each valid
> > > > container ID present on an auditable action or event.
> > > > 
> > > > Forked and cloned processes inherit their parent's audit container
> > > > identifier, referenced in the process' task_struct.  Since the audit
> > > > container identifier is inherited rather than written, it can still be
> > > > written once.  This will prevent tampering while allowing nesting.
> > > > (This can be implemented with an internal settable flag upon
> > > > registration that does not get copied across a fork/clone.)
> > > > 
> > > > Mimic setns(2) and return an error if the process has already initiated
> > > > threading or forked since this registration should happen before the
> > > > process execution is started by the orchestrator and hence should not
> > > > yet have any threads or children.  If this is deemed overly restrictive,
> > > > switch all of the target's threads and children to the new containerID.
> > > > 
> > > > Trust the orchestrator to judiciously use and restrict CAP_AUDIT_CONTROL.
> > > > 
> > > > When a container ceases to exist because the last process in that
> > > > container has exited log the fact to balance the registration action.
> > > > (This is likely needed for certification accountability.)
> > > > 
> > > > At this point it appears unnecessary to add a container session
> > > > identifier since this is all tracked from loginuid and sessionid to
> > > > communicate with the container orchestrator to spawn an additional
> > > > session into an existing container which would be logged.  It can be
> > > > added at a later date without breaking API should it be deemed
> > > > necessary.
> > > > 
> > > > The following namespace logging actions are not needed for certification
> > > > purposes at this point, but are helpful for tracking namespace activity.
> > > > These are auxilliary records that are associated with namespace
> > > > manipulation syscalls unshare(2), clone(2) and setns(2), so the records
> > > > will only show up if explicit syscall rules have been added to document
> > > > this activity.
> > > > 
> > > > Log the creation of every namespace, inheriting/adding its spawning
> > > > process' audit container identifier(s), if applicable.  Include the
> > > > spawning and spawned namespace IDs (device and inode number tuples).
> > > > [AUDIT_NS_CREATE, AUDIT_NS_DESTROY] [clone(2), unshare(2), setns(2)]
> > > > Note: At this point it appears only network namespaces may need to track
> > > > container IDs apart from processes since incoming packets may cause an
> > > > auditable event before being associated with a process.  Since a
> > > > namespace can be shared by processes in different containers, the
> > > > namespace will need to track all containers to which it has been
> > > > assigned.
> > > > 
> > > > Upon registration, the target process' namespace IDs (in the form of a
> > > > nsfs device number and inode number tuple) will be recorded in an
> > > > AUDIT_NS_INFO auxilliary record.
> > > > 
> > > > Log the destruction of every namespace that is no longer used by any
> > > > process, including the namespace IDs (device and inode number tuples).
> > > > [AUDIT_NS_DESTROY] [process exit, unshare(2), setns(2)]
> > > > 
> > > > Issue a new auxilliary record AUDIT_NS_CHANGE listing (opt: op=$action)
> > > > the parent and child namespace IDs for any changes to a process'
> > > > namespaces. [setns(2)]
> > > > Note: It may be possible to combine AUDIT_NS_* record formats and
> > > > distinguish them with an op=$action field depending on the fields
> > > > required for each message type.
> > > > 
> > > > The audit container identifier will need to be reaped from all
> > > > implicated namespaces upon the destruction of a container.
> > > > 
> > > > This namespace information adds supporting information for tracking
> > > > events not attributable to specific processes.
> > > > 
> > > > Changelog:
> > > > 
> > > > (Upstream V3)
> > > > - switch back to u64 (from pmoore, can be expanded to u128 in future if
> > > >   need arises without breaking API.  u32 was originally proposed, up to
> > > >   c36 discussed)
> > > > - write-once, but children inherit audit container identifier and can
> > > >   then still be written once
> > > > - switch to CAP_AUDIT_CONTROL
> > > > - group namespace actions together, auxilliary records to namespace
> > > >   operations.
> > > > 
> > > > (Upstream V2)
> > > > - switch from u64 to u128 UUID
> > > > - switch from "signal" and "trigger" to "register"
> > > > - restrict registration to single process or force all threads and
> > > >   children into same container
> > > 
> > > I am trying to understand the back and forth on the ID size.
> > > 
> > > From an orchestrator POV anything that requires tracking a node
> > > specific ID is not ideal.
> > > 
> > > Orchestrators tend to span many nodes, and containers tend to have IDs
> > > that are either UUID or have a Hash (like SHA256) as identifier.
> > > 
> > > The problem here is two-fold:
> > > 
> > > a) Your auditing requires some mapping to be useful outside of the
> > > system.
> > > If you aggreggate audit logs outside of the system or you want to
> > > correlate the system audit logs with other components dealing with
> > > containers, now you need a place where you provide a mapping from your
> > > audit u64 to the ID a container has in the rest of the system.
> > > 
> > > b) Now you need a mapping of some sort. The simplest way a container
> > > orchestrator can go about this is to just use the UUID or Hash
> > > representing their view of the container, truncate it to a u64 and use
> > > that for Audit. This means there are some chances there will be a
> > > collision and a duplicate u64 ID will be used by the orchestrator as
> > > the container ID. What happen in that case ?
> > 
> > Paul, can you justify this somewhat larger inconvenience for some
> > relatively minor convenience on our part?
> 
> Done in direct response to Simo.

Sorry but your response sounds more like waving away then addressing
them, the excuse being: we can't please everyone, so we are going to
please no one.

> But to be clear Richard, we've talked about this a few times, it's not
> a "minor convenience" on our part, it's a pretty big convenience once
> we starting having to route audit events and make decisions based on
> the audit container ID information.  Audit performance is less than
> awesome now, I'm working hard to not make it worse.

Sounds like a security vs performance trade off to me.

> > u64 vs u128 is easy for us to
> > accomodate in terms of scalar comparisons.  It doubles the information
> > in every container id field we print in audit records.
> 
> ... and slows down audit container ID checks.

Are you saying a cmp on a u128 is slower than a comparison on a u64 and
this is something that will be noticeable ?

> > A c36 is a bigger step.
> 
> Yeah, we're not doing that, no way.

Ok, I can see your point though I do not agree with it.

I can see why you do not want to have arbitrary length strings, but a
u128 sounded like a reasonable compromise to me as it has enough room
to be able to have unique cluster-wide IDs which a u64 definitely makes
a lot harder to provide w/o tight coordination.

Simo.

-- 
Simo Sorce
Sr. Principal Software Engineer
Red Hat, Inc

Re: RFC(V3): Audit Kernel Container IDs

[Simo Sorce]

On Fri, 2018-02-02 at 16:24 -0500, Paul Moore wrote:
> On Wed, Jan 10, 2018 at 2:00 AM, Richard Guy Briggs <rgb@redhat.com> wrote:
> > On 2018-01-09 11:18, Simo Sorce wrote:
> > > On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:
> > > > Containers are a userspace concept.  The kernel knows nothing of them.
> > > > 
> > > > The Linux audit system needs a way to be able to track the container
> > > > provenance of events and actions.  Audit needs the kernel's help to do
> > > > this.
> > > > 
> > > > Since the concept of a container is entirely a userspace concept, a
> > > > registration from the userspace container orchestration system initiates
> > > > this.  This will define a point in time and a set of resources
> > > > associated with a particular container with an audit container
> > > > identifier.
> > > > 
> > > > The registration is a u64 representing the audit container identifier
> > > > written to a special file in a pseudo filesystem (proc, since PID tree
> > > > already exists) representing a process that will become a parent process
> > > > in that container.  This write might place restrictions on mount
> > > > namespaces required to define a container, or at least careful checking
> > > > of namespaces in the kernel to verify permissions of the orchestrator so
> > > > it can't change its own container ID.  A bind mount of nsfs may be
> > > > necessary in the container orchestrator's mount namespace.  This write
> > > > can only happen once per process.
> > > > 
> > > > Note: The justification for using a u64 is that it minimizes the
> > > > information printed in every audit record, reducing bandwidth and limits
> > > > comparisons to a single u64 which will be faster and less error-prone.
> > > > 
> > > > Require CAP_AUDIT_CONTROL to be able to carry out the registration.  At
> > > > that time, record the target container's user-supplied audit container
> > > > identifier along with a target container's parent process (which may
> > > > become the target container's "init" process) process ID (referenced
> > > > from the initial PID namespace) in a new record AUDIT_CONTAINER with a
> > > > qualifying op=$action field.
> > > > 
> > > > Issue a new auxilliary record AUDIT_CONTAINER_INFO for each valid
> > > > container ID present on an auditable action or event.
> > > > 
> > > > Forked and cloned processes inherit their parent's audit container
> > > > identifier, referenced in the process' task_struct.  Since the audit
> > > > container identifier is inherited rather than written, it can still be
> > > > written once.  This will prevent tampering while allowing nesting.
> > > > (This can be implemented with an internal settable flag upon
> > > > registration that does not get copied across a fork/clone.)
> > > > 
> > > > Mimic setns(2) and return an error if the process has already initiated
> > > > threading or forked since this registration should happen before the
> > > > process execution is started by the orchestrator and hence should not
> > > > yet have any threads or children.  If this is deemed overly restrictive,
> > > > switch all of the target's threads and children to the new containerID.
> > > > 
> > > > Trust the orchestrator to judiciously use and restrict CAP_AUDIT_CONTROL.
> > > > 
> > > > When a container ceases to exist because the last process in that
> > > > container has exited log the fact to balance the registration action.
> > > > (This is likely needed for certification accountability.)
> > > > 
> > > > At this point it appears unnecessary to add a container session
> > > > identifier since this is all tracked from loginuid and sessionid to
> > > > communicate with the container orchestrator to spawn an additional
> > > > session into an existing container which would be logged.  It can be
> > > > added at a later date without breaking API should it be deemed
> > > > necessary.
> > > > 
> > > > The following namespace logging actions are not needed for certification
> > > > purposes at this point, but are helpful for tracking namespace activity.
> > > > These are auxilliary records that are associated with namespace
> > > > manipulation syscalls unshare(2), clone(2) and setns(2), so the records
> > > > will only show up if explicit syscall rules have been added to document
> > > > this activity.
> > > > 
> > > > Log the creation of every namespace, inheriting/adding its spawning
> > > > process' audit container identifier(s), if applicable.  Include the
> > > > spawning and spawned namespace IDs (device and inode number tuples).
> > > > [AUDIT_NS_CREATE, AUDIT_NS_DESTROY] [clone(2), unshare(2), setns(2)]
> > > > Note: At this point it appears only network namespaces may need to track
> > > > container IDs apart from processes since incoming packets may cause an
> > > > auditable event before being associated with a process.  Since a
> > > > namespace can be shared by processes in different containers, the
> > > > namespace will need to track all containers to which it has been
> > > > assigned.
> > > > 
> > > > Upon registration, the target process' namespace IDs (in the form of a
> > > > nsfs device number and inode number tuple) will be recorded in an
> > > > AUDIT_NS_INFO auxilliary record.
> > > > 
> > > > Log the destruction of every namespace that is no longer used by any
> > > > process, including the namespace IDs (device and inode number tuples).
> > > > [AUDIT_NS_DESTROY] [process exit, unshare(2), setns(2)]
> > > > 
> > > > Issue a new auxilliary record AUDIT_NS_CHANGE listing (opt: op=$action)
> > > > the parent and child namespace IDs for any changes to a process'
> > > > namespaces. [setns(2)]
> > > > Note: It may be possible to combine AUDIT_NS_* record formats and
> > > > distinguish them with an op=$action field depending on the fields
> > > > required for each message type.
> > > > 
> > > > The audit container identifier will need to be reaped from all
> > > > implicated namespaces upon the destruction of a container.
> > > > 
> > > > This namespace information adds supporting information for tracking
> > > > events not attributable to specific processes.
> > > > 
> > > > Changelog:
> > > > 
> > > > (Upstream V3)
> > > > - switch back to u64 (from pmoore, can be expanded to u128 in future if
> > > >   need arises without breaking API.  u32 was originally proposed, up to
> > > >   c36 discussed)
> > > > - write-once, but children inherit audit container identifier and can
> > > >   then still be written once
> > > > - switch to CAP_AUDIT_CONTROL
> > > > - group namespace actions together, auxilliary records to namespace
> > > >   operations.
> > > > 
> > > > (Upstream V2)
> > > > - switch from u64 to u128 UUID
> > > > - switch from "signal" and "trigger" to "register"
> > > > - restrict registration to single process or force all threads and
> > > >   children into same container
> > > 
> > > I am trying to understand the back and forth on the ID size.
> > > 
> > > From an orchestrator POV anything that requires tracking a node
> > > specific ID is not ideal.
> > > 
> > > Orchestrators tend to span many nodes, and containers tend to have IDs
> > > that are either UUID or have a Hash (like SHA256) as identifier.
> > > 
> > > The problem here is two-fold:
> > > 
> > > a) Your auditing requires some mapping to be useful outside of the
> > > system.
> > > If you aggreggate audit logs outside of the system or you want to
> > > correlate the system audit logs with other components dealing with
> > > containers, now you need a place where you provide a mapping from your
> > > audit u64 to the ID a container has in the rest of the system.
> > > 
> > > b) Now you need a mapping of some sort. The simplest way a container
> > > orchestrator can go about this is to just use the UUID or Hash
> > > representing their view of the container, truncate it to a u64 and use
> > > that for Audit. This means there are some chances there will be a
> > > collision and a duplicate u64 ID will be used by the orchestrator as
> > > the container ID. What happen in that case ?
> > 
> > Paul, can you justify this somewhat larger inconvenience for some
> > relatively minor convenience on our part?
> 
> Done in direct response to Simo.

Sorry but your response sounds more like waving away then addressing
them, the excuse being: we can't please everyone, so we are going to
please no one.

> But to be clear Richard, we've talked about this a few times, it's not
> a "minor convenience" on our part, it's a pretty big convenience once
> we starting having to route audit events and make decisions based on
> the audit container ID information.  Audit performance is less than
> awesome now, I'm working hard to not make it worse.

Sounds like a security vs performance trade off to me.

> > u64 vs u128 is easy for us to
> > accomodate in terms of scalar comparisons.  It doubles the information
> > in every container id field we print in audit records.
> 
> ... and slows down audit container ID checks.

Are you saying a cmp on a u128 is slower than a comparison on a u64 and
this is something that will be noticeable ?

> > A c36 is a bigger step.
> 
> Yeah, we're not doing that, no way.

Ok, I can see your point though I do not agree with it.

I can see why you do not want to have arbitrary length strings, but a
u128 sounded like a reasonable compromise to me as it has enough room
to be able to have unique cluster-wide IDs which a u64 definitely makes
a lot harder to provide w/o tight coordination.

Simo.

-- 
Simo Sorce
Sr. Principal Software Engineer
Red Hat, Inc

Re: RFC(V3): Audit Kernel Container IDs

[Paul Moore]

On Fri, Feb 2, 2018 at 5:19 PM, Simo Sorce <simo-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> On Fri, 2018-02-02 at 16:24 -0500, Paul Moore wrote:
>> On Wed, Jan 10, 2018 at 2:00 AM, Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
>> > On 2018-01-09 11:18, Simo Sorce wrote:
>> > > On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:

...

>> > Paul, can you justify this somewhat larger inconvenience for some
>> > relatively minor convenience on our part?
>>
>> Done in direct response to Simo.
>
> Sorry but your response sounds more like waving away then addressing
> them, the excuse being: we can't please everyone, so we are going to
> please no one.

I obviously disagree with the take on my comments but you're free to
your opinion.

I believe saying we are pleasing no one isn't really fair now is it?
Is there any type of audit container ID now?  How would you go about
associating audit events with containers now? (spoiler alert: it ain't
pretty, and there are gaps I don't believe you can cover)  This
proposal provides a mechanism to do this in a way that isn't tied to
any one particular concept of a container and is manageable inside the
kernel.

If you have a need to track audit events for containers, I find it
extremely hard to believe that you are not at least partially pleased
by the solutions presented here.  It may not be everything on your
wishlist, but when did you ever get *everything* on your wishlist?

>> But to be clear Richard, we've talked about this a few times, it's not
>> a "minor convenience" on our part, it's a pretty big convenience once
>> we starting having to route audit events and make decisions based on
>> the audit container ID information.  Audit performance is less than
>> awesome now, I'm working hard to not make it worse.
>
> Sounds like a security vs performance trade off to me.

Welcome to software development.  It's generally a pretty terrible
hobby and/or occupation, but we make up for it with long hours and
endless frustration.

>> > u64 vs u128 is easy for us to
>> > accomodate in terms of scalar comparisons.  It doubles the information
>> > in every container id field we print in audit records.
>>
>> ... and slows down audit container ID checks.
>
> Are you saying a cmp on a u128 is slower than a comparison on a u64 and
> this is something that will be noticeable ?

Do you have a 128 bit system?  I don't.  I've got a bunch of 64 bit
systems, and a couple of 32 bit systems too.  People that use audit
have a tendency to really hammer on it, to the point that we get
performance complaints on a not infrequent basis.  I don't know the
exact number of times we are going to need to check the audit
container ID, but it's reasonable to think that we'll expose it as a
filter-able field which adds a few checks, we'll use it for record
routing so that's a few more, and if we're running multiple audit
daemons we will probably want to include LSM checks which could result
in a few more audit container ID checks.  If it was one comparison I
wouldn't be too worried about it, but the point I'm trying to make is
that we don't know what the implementation is going to look like yet
and I suspect this ID is going to be leveraged in several places in
the audit subsystem and I would much rather start small to save
headaches later.

We can always expand the ID to a larger integer at a later date, but
we can't make it smaller.

>> > A c36 is a bigger step.
>>
>> Yeah, we're not doing that, no way.
>
> Ok, I can see your point though I do not agree with it.
>
> I can see why you do not want to have arbitrary length strings, but a
> u128 sounded like a reasonable compromise to me as it has enough room
> to be able to have unique cluster-wide IDs which a u64 definitely makes
> a lot harder to provide w/o tight coordination.

I originally wanted it to be a 32-bit integer, but Richard managed to
talk me into 64-bits, that was my compromise :)

As I said earlier, if you are doing container auditing you're going to
need coordination with the orchestrator, regardless of the audit
container ID size.

-- 
paul moore
www.paul-moore.com

Re: RFC(V3): Audit Kernel Container IDs

[Paul Moore]

On Fri, Feb 2, 2018 at 5:19 PM, Simo Sorce <simo@redhat.com> wrote:
> On Fri, 2018-02-02 at 16:24 -0500, Paul Moore wrote:
>> On Wed, Jan 10, 2018 at 2:00 AM, Richard Guy Briggs <rgb@redhat.com> wrote:
>> > On 2018-01-09 11:18, Simo Sorce wrote:
>> > > On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:

...

>> > Paul, can you justify this somewhat larger inconvenience for some
>> > relatively minor convenience on our part?
>>
>> Done in direct response to Simo.
>
> Sorry but your response sounds more like waving away then addressing
> them, the excuse being: we can't please everyone, so we are going to
> please no one.

I obviously disagree with the take on my comments but you're free to
your opinion.

I believe saying we are pleasing no one isn't really fair now is it?
Is there any type of audit container ID now?  How would you go about
associating audit events with containers now? (spoiler alert: it ain't
pretty, and there are gaps I don't believe you can cover)  This
proposal provides a mechanism to do this in a way that isn't tied to
any one particular concept of a container and is manageable inside the
kernel.

If you have a need to track audit events for containers, I find it
extremely hard to believe that you are not at least partially pleased
by the solutions presented here.  It may not be everything on your
wishlist, but when did you ever get *everything* on your wishlist?

>> But to be clear Richard, we've talked about this a few times, it's not
>> a "minor convenience" on our part, it's a pretty big convenience once
>> we starting having to route audit events and make decisions based on
>> the audit container ID information.  Audit performance is less than
>> awesome now, I'm working hard to not make it worse.
>
> Sounds like a security vs performance trade off to me.

Welcome to software development.  It's generally a pretty terrible
hobby and/or occupation, but we make up for it with long hours and
endless frustration.

>> > u64 vs u128 is easy for us to
>> > accomodate in terms of scalar comparisons.  It doubles the information
>> > in every container id field we print in audit records.
>>
>> ... and slows down audit container ID checks.
>
> Are you saying a cmp on a u128 is slower than a comparison on a u64 and
> this is something that will be noticeable ?

Do you have a 128 bit system?  I don't.  I've got a bunch of 64 bit
systems, and a couple of 32 bit systems too.  People that use audit
have a tendency to really hammer on it, to the point that we get
performance complaints on a not infrequent basis.  I don't know the
exact number of times we are going to need to check the audit
container ID, but it's reasonable to think that we'll expose it as a
filter-able field which adds a few checks, we'll use it for record
routing so that's a few more, and if we're running multiple audit
daemons we will probably want to include LSM checks which could result
in a few more audit container ID checks.  If it was one comparison I
wouldn't be too worried about it, but the point I'm trying to make is
that we don't know what the implementation is going to look like yet
and I suspect this ID is going to be leveraged in several places in
the audit subsystem and I would much rather start small to save
headaches later.

We can always expand the ID to a larger integer at a later date, but
we can't make it smaller.

>> > A c36 is a bigger step.
>>
>> Yeah, we're not doing that, no way.
>
> Ok, I can see your point though I do not agree with it.
>
> I can see why you do not want to have arbitrary length strings, but a
> u128 sounded like a reasonable compromise to me as it has enough room
> to be able to have unique cluster-wide IDs which a u64 definitely makes
> a lot harder to provide w/o tight coordination.

I originally wanted it to be a 32-bit integer, but Richard managed to
talk me into 64-bits, that was my compromise :)

As I said earlier, if you are doing container auditing you're going to
need coordination with the orchestrator, regardless of the audit
container ID size.

-- 
paul moore
www.paul-moore.com

Re: RFC(V3): Audit Kernel Container IDs

[Casey Schaufler]

On 2/2/2018 3:24 PM, Paul Moore wrote:
> On Fri, Feb 2, 2018 at 5:19 PM, Simo Sorce <simo@redhat.com> wrote:
>> On Fri, 2018-02-02 at 16:24 -0500, Paul Moore wrote:
>>> On Wed, Jan 10, 2018 at 2:00 AM, Richard Guy Briggs <rgb@redhat.com> wrote:
>>>> On 2018-01-09 11:18, Simo Sorce wrote:
>>>>> On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:
> ..
>
>>>> Paul, can you justify this somewhat larger inconvenience for some
>>>> relatively minor convenience on our part?
>>> Done in direct response to Simo.
>> Sorry but your response sounds more like waving away then addressing
>> them, the excuse being: we can't please everyone, so we are going to
>> please no one.
> I obviously disagree with the take on my comments but you're free to
> your opinion.
>
> I believe saying we are pleasing no one isn't really fair now is it?
> Is there any type of audit container ID now?  How would you go about
> associating audit events with containers now? (spoiler alert: it ain't
> pretty, and there are gaps I don't believe you can cover)  This
> proposal provides a mechanism to do this in a way that isn't tied to
> any one particular concept of a container and is manageable inside the
> kernel.
>
> If you have a need to track audit events for containers, I find it
> extremely hard to believe that you are not at least partially pleased
> by the solutions presented here.  It may not be everything on your
> wishlist, but when did you ever get *everything* on your wishlist?

I am going to back Paul 100% on this point. The container
community's emphatic position that containers are strictly
a user-space construct makes it impossible for the kernel
to provide any data more sophisticated than an integer, and
any processing based on that data cleverer than a check
for equality.


>>> But to be clear Richard, we've talked about this a few times, it's not
>>> a "minor convenience" on our part, it's a pretty big convenience once
>>> we starting having to route audit events and make decisions based on
>>> the audit container ID information.  Audit performance is less than
>>> awesome now, I'm working hard to not make it worse.
>> Sounds like a security vs performance trade off to me.

Without the kernel having a "container" policy to work with there
is no "security" it can possibly enforce.

> Welcome to software development.  It's generally a pretty terrible
> hobby and/or occupation, but we make up for it with long hours and
> endless frustration.
>
>>>> u64 vs u128 is easy for us to
>>>> accomodate in terms of scalar comparisons.  It doubles the information
>>>> in every container id field we print in audit records.
>>> ... and slows down audit container ID checks.
>> Are you saying a cmp on a u128 is slower than a comparison on a u64 and
>> this is something that will be noticeable ?
> Do you have a 128 bit system?  I don't.  I've got a bunch of 64 bit
> systems, and a couple of 32 bit systems too.  People that use audit
> have a tendency to really hammer on it, to the point that we get
> performance complaints on a not infrequent basis.  I don't know the
> exact number of times we are going to need to check the audit
> container ID, but it's reasonable to think that we'll expose it as a
> filter-able field which adds a few checks, we'll use it for record
> routing so that's a few more, and if we're running multiple audit
> daemons we will probably want to include LSM checks which could result
> in a few more audit container ID checks.  If it was one comparison I
> wouldn't be too worried about it, but the point I'm trying to make is
> that we don't know what the implementation is going to look like yet
> and I suspect this ID is going to be leveraged in several places in
> the audit subsystem and I would much rather start small to save
> headaches later.
>
> We can always expand the ID to a larger integer at a later date, but
> we can't make it smaller.
>
>>>> A c36 is a bigger step.
>>> Yeah, we're not doing that, no way.
>> Ok, I can see your point though I do not agree with it.
>>
>> I can see why you do not want to have arbitrary length strings, but a
>> u128 sounded like a reasonable compromise to me as it has enough room
>> to be able to have unique cluster-wide IDs which a u64 definitely makes
>> a lot harder to provide w/o tight coordination.
> I originally wanted it to be a 32-bit integer, but Richard managed to
> talk me into 64-bits, that was my compromise :)
>
> As I said earlier, if you are doing container auditing you're going to
> need coordination with the orchestrator, regardless of the audit
> container ID size.
>

Re: RFC(V3): Audit Kernel Container IDs

[Casey Schaufler]

On 2/2/2018 3:24 PM, Paul Moore wrote:
> On Fri, Feb 2, 2018 at 5:19 PM, Simo Sorce <simo-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
>> On Fri, 2018-02-02 at 16:24 -0500, Paul Moore wrote:
>>> On Wed, Jan 10, 2018 at 2:00 AM, Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
>>>> On 2018-01-09 11:18, Simo Sorce wrote:
>>>>> On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:
> ..
>
>>>> Paul, can you justify this somewhat larger inconvenience for some
>>>> relatively minor convenience on our part?
>>> Done in direct response to Simo.
>> Sorry but your response sounds more like waving away then addressing
>> them, the excuse being: we can't please everyone, so we are going to
>> please no one.
> I obviously disagree with the take on my comments but you're free to
> your opinion.
>
> I believe saying we are pleasing no one isn't really fair now is it?
> Is there any type of audit container ID now?  How would you go about
> associating audit events with containers now? (spoiler alert: it ain't
> pretty, and there are gaps I don't believe you can cover)  This
> proposal provides a mechanism to do this in a way that isn't tied to
> any one particular concept of a container and is manageable inside the
> kernel.
>
> If you have a need to track audit events for containers, I find it
> extremely hard to believe that you are not at least partially pleased
> by the solutions presented here.  It may not be everything on your
> wishlist, but when did you ever get *everything* on your wishlist?

I am going to back Paul 100% on this point. The container
community's emphatic position that containers are strictly
a user-space construct makes it impossible for the kernel
to provide any data more sophisticated than an integer, and
any processing based on that data cleverer than a check
for equality.


>>> But to be clear Richard, we've talked about this a few times, it's not
>>> a "minor convenience" on our part, it's a pretty big convenience once
>>> we starting having to route audit events and make decisions based on
>>> the audit container ID information.  Audit performance is less than
>>> awesome now, I'm working hard to not make it worse.
>> Sounds like a security vs performance trade off to me.

Without the kernel having a "container" policy to work with there
is no "security" it can possibly enforce.

> Welcome to software development.  It's generally a pretty terrible
> hobby and/or occupation, but we make up for it with long hours and
> endless frustration.
>
>>>> u64 vs u128 is easy for us to
>>>> accomodate in terms of scalar comparisons.  It doubles the information
>>>> in every container id field we print in audit records.
>>> ... and slows down audit container ID checks.
>> Are you saying a cmp on a u128 is slower than a comparison on a u64 and
>> this is something that will be noticeable ?
> Do you have a 128 bit system?  I don't.  I've got a bunch of 64 bit
> systems, and a couple of 32 bit systems too.  People that use audit
> have a tendency to really hammer on it, to the point that we get
> performance complaints on a not infrequent basis.  I don't know the
> exact number of times we are going to need to check the audit
> container ID, but it's reasonable to think that we'll expose it as a
> filter-able field which adds a few checks, we'll use it for record
> routing so that's a few more, and if we're running multiple audit
> daemons we will probably want to include LSM checks which could result
> in a few more audit container ID checks.  If it was one comparison I
> wouldn't be too worried about it, but the point I'm trying to make is
> that we don't know what the implementation is going to look like yet
> and I suspect this ID is going to be leveraged in several places in
> the audit subsystem and I would much rather start small to save
> headaches later.
>
> We can always expand the ID to a larger integer at a later date, but
> we can't make it smaller.
>
>>>> A c36 is a bigger step.
>>> Yeah, we're not doing that, no way.
>> Ok, I can see your point though I do not agree with it.
>>
>> I can see why you do not want to have arbitrary length strings, but a
>> u128 sounded like a reasonable compromise to me as it has enough room
>> to be able to have unique cluster-wide IDs which a u64 definitely makes
>> a lot harder to provide w/o tight coordination.
> I originally wanted it to be a 32-bit integer, but Richard managed to
> talk me into 64-bits, that was my compromise :)
>
> As I said earlier, if you are doing container auditing you're going to
> need coordination with the orchestrator, regardless of the audit
> container ID size.
>

Re: RFC(V3): Audit Kernel Container IDs

[Simo Sorce]

On Fri, 2018-02-02 at 18:24 -0500, Paul Moore wrote:
> On Fri, Feb 2, 2018 at 5:19 PM, Simo Sorce <simo-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> > On Fri, 2018-02-02 at 16:24 -0500, Paul Moore wrote:
> > > On Wed, Jan 10, 2018 at 2:00 AM, Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> > > > On 2018-01-09 11:18, Simo Sorce wrote:
> > > > > On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:
> 
> ...
> 
> > > > Paul, can you justify this somewhat larger inconvenience for some
> > > > relatively minor convenience on our part?
> > > 
> > > Done in direct response to Simo.
> > 
> > Sorry but your response sounds more like waving away then addressing
> > them, the excuse being: we can't please everyone, so we are going to
> > please no one.
> 
> I obviously disagree with the take on my comments but you're free to
> your opinion.

The I misunderstood your comments, I am not interested in putting words
in your mouth.

> I believe saying we are pleasing no one isn't really fair now is it?

Well, of course you are going to please the audit subsystem, I
understand that. I think there is a problem of expectations. Some
people, me included, hoped to have a way to identify a container with
the help of the kernel.

> Is there any type of audit container ID now?  How would you go about
> associating audit events with containers now?

We do not have a good way, there are some dirty tricks like inferring
the container identity via cgroup names, but that is ... eww.
This is why, given audit has the same need of user space, there was
some hope we could agree on an identifier that could be used by both.
It would make correlating audit logs and other cluster-wide events
simpler. That is all.

>  (spoiler alert: it ain't
> pretty, and there are gaps I don't believe you can cover)  This
> proposal provides a mechanism to do this in a way that isn't tied to
> any one particular concept of a container and is manageable inside the
> kernel.

I like the proposal for the most part, we are just discussing on the
nature of the identifier, which is a minor detail in the end.

> If you have a need to track audit events for containers, I find it
> extremely hard to believe that you are not at least partially pleased
> by the solutions presented here.  It may not be everything on your
> wishlist, but when did you ever get *everything* on your wishlist?

It is true, and I am sorry if I came out demanding or abrasive. It was
not my intention. Of course a u64 that has to be mapped is still better
than nothing. It does cause a lot more work in user space, but it is
not impossible to deal with.

> > > But to be clear Richard, we've talked about this a few times, it's not
> > > a "minor convenience" on our part, it's a pretty big convenience once
> > > we starting having to route audit events and make decisions based on
> > > the audit container ID information.  Audit performance is less than
> > > awesome now, I'm working hard to not make it worse.
> > 
> > Sounds like a security vs performance trade off to me.
> 
> Welcome to software development.  It's generally a pretty terrible
> hobby and/or occupation, but we make up for it with long hours and
> endless frustration.

Tell me more about that, not! ;-)

> > > > u64 vs u128 is easy for us to
> > > > accomodate in terms of scalar comparisons.  It doubles the information
> > > > in every container id field we print in audit records.
> > > 
> > > ... and slows down audit container ID checks.
> > 
> > Are you saying a cmp on a u128 is slower than a comparison on a u64 and
> > this is something that will be noticeable ?
> 
> Do you have a 128 bit system?

no, but all 64bit systems have an instruction that allow you to do
atomic 128 compare and swap (IIRC ?).

>   I don't.  I've got a bunch of 64 bit
> systems, and a couple of 32 bit systems too.  People that use audit
> have a tendency to really hammer on it, to the point that we get
> performance complaints on a not infrequent basis.  I don't know the
> exact number of times we are going to need to check the audit
> container ID, but it's reasonable to think that we'll expose it as a
> filter-able field which adds a few checks, we'll use it for record
> routing so that's a few more, and if we're running multiple audit
> daemons we will probably want to include LSM checks which could result
> in a few more audit container ID checks.  If it was one comparison I
> wouldn't be too worried about it, but the point I'm trying to make is
> that we don't know what the implementation is going to look like yet
> and I suspect this ID is going to be leveraged in several places in
> the audit subsystem and I would much rather start small to save
> headaches later.
> 
> We can always expand the ID to a larger integer at a later date, but
> we can't make it smaller.

Well looking through the history of in kernel identifiers I know it is
hard also to increase size, because userspace will end up depending on
a specific size ... and this is the only reason I am really debating
this. If it were really easy to change I wouldn't bother to do it now.

> > > > A c36 is a bigger step.
> > > 
> > > Yeah, we're not doing that, no way.
> > 
> > Ok, I can see your point though I do not agree with it.
> > 
> > I can see why you do not want to have arbitrary length strings, but a
> > u128 sounded like a reasonable compromise to me as it has enough room
> > to be able to have unique cluster-wide IDs which a u64 definitely makes
> > a lot harder to provide w/o tight coordination.
> 
> I originally wanted it to be a 32-bit integer, but Richard managed to
> talk me into 64-bits, that was my compromise :)
> 
> As I said earlier, if you are doing container auditing you're going to
> need coordination with the orchestrator, regardless of the audit
> container ID size.

Ok, I guess that's as good as I can get it for now, thank you for your
patient explanations.

Simo.

-- 
Simo Sorce
Sr. Principal Software Engineer
Red Hat, Inc

Re: RFC(V3): Audit Kernel Container IDs

[Simo Sorce]

On Fri, 2018-02-02 at 18:24 -0500, Paul Moore wrote:
> On Fri, Feb 2, 2018 at 5:19 PM, Simo Sorce <simo@redhat.com> wrote:
> > On Fri, 2018-02-02 at 16:24 -0500, Paul Moore wrote:
> > > On Wed, Jan 10, 2018 at 2:00 AM, Richard Guy Briggs <rgb@redhat.com> wrote:
> > > > On 2018-01-09 11:18, Simo Sorce wrote:
> > > > > On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:
> 
> ...
> 
> > > > Paul, can you justify this somewhat larger inconvenience for some
> > > > relatively minor convenience on our part?
> > > 
> > > Done in direct response to Simo.
> > 
> > Sorry but your response sounds more like waving away then addressing
> > them, the excuse being: we can't please everyone, so we are going to
> > please no one.
> 
> I obviously disagree with the take on my comments but you're free to
> your opinion.

The I misunderstood your comments, I am not interested in putting words
in your mouth.

> I believe saying we are pleasing no one isn't really fair now is it?

Well, of course you are going to please the audit subsystem, I
understand that. I think there is a problem of expectations. Some
people, me included, hoped to have a way to identify a container with
the help of the kernel.

> Is there any type of audit container ID now?  How would you go about
> associating audit events with containers now?

We do not have a good way, there are some dirty tricks like inferring
the container identity via cgroup names, but that is ... eww.
This is why, given audit has the same need of user space, there was
some hope we could agree on an identifier that could be used by both.
It would make correlating audit logs and other cluster-wide events
simpler. That is all.

>  (spoiler alert: it ain't
> pretty, and there are gaps I don't believe you can cover)  This
> proposal provides a mechanism to do this in a way that isn't tied to
> any one particular concept of a container and is manageable inside the
> kernel.

I like the proposal for the most part, we are just discussing on the
nature of the identifier, which is a minor detail in the end.

> If you have a need to track audit events for containers, I find it
> extremely hard to believe that you are not at least partially pleased
> by the solutions presented here.  It may not be everything on your
> wishlist, but when did you ever get *everything* on your wishlist?

It is true, and I am sorry if I came out demanding or abrasive. It was
not my intention. Of course a u64 that has to be mapped is still better
than nothing. It does cause a lot more work in user space, but it is
not impossible to deal with.

> > > But to be clear Richard, we've talked about this a few times, it's not
> > > a "minor convenience" on our part, it's a pretty big convenience once
> > > we starting having to route audit events and make decisions based on
> > > the audit container ID information.  Audit performance is less than
> > > awesome now, I'm working hard to not make it worse.
> > 
> > Sounds like a security vs performance trade off to me.
> 
> Welcome to software development.  It's generally a pretty terrible
> hobby and/or occupation, but we make up for it with long hours and
> endless frustration.

Tell me more about that, not! ;-)

> > > > u64 vs u128 is easy for us to
> > > > accomodate in terms of scalar comparisons.  It doubles the information
> > > > in every container id field we print in audit records.
> > > 
> > > ... and slows down audit container ID checks.
> > 
> > Are you saying a cmp on a u128 is slower than a comparison on a u64 and
> > this is something that will be noticeable ?
> 
> Do you have a 128 bit system?

no, but all 64bit systems have an instruction that allow you to do
atomic 128 compare and swap (IIRC ?).

>   I don't.  I've got a bunch of 64 bit
> systems, and a couple of 32 bit systems too.  People that use audit
> have a tendency to really hammer on it, to the point that we get
> performance complaints on a not infrequent basis.  I don't know the
> exact number of times we are going to need to check the audit
> container ID, but it's reasonable to think that we'll expose it as a
> filter-able field which adds a few checks, we'll use it for record
> routing so that's a few more, and if we're running multiple audit
> daemons we will probably want to include LSM checks which could result
> in a few more audit container ID checks.  If it was one comparison I
> wouldn't be too worried about it, but the point I'm trying to make is
> that we don't know what the implementation is going to look like yet
> and I suspect this ID is going to be leveraged in several places in
> the audit subsystem and I would much rather start small to save
> headaches later.
> 
> We can always expand the ID to a larger integer at a later date, but
> we can't make it smaller.

Well looking through the history of in kernel identifiers I know it is
hard also to increase size, because userspace will end up depending on
a specific size ... and this is the only reason I am really debating
this. If it were really easy to change I wouldn't bother to do it now.

> > > > A c36 is a bigger step.
> > > 
> > > Yeah, we're not doing that, no way.
> > 
> > Ok, I can see your point though I do not agree with it.
> > 
> > I can see why you do not want to have arbitrary length strings, but a
> > u128 sounded like a reasonable compromise to me as it has enough room
> > to be able to have unique cluster-wide IDs which a u64 definitely makes
> > a lot harder to provide w/o tight coordination.
> 
> I originally wanted it to be a 32-bit integer, but Richard managed to
> talk me into 64-bits, that was my compromise :)
> 
> As I said earlier, if you are doing container auditing you're going to
> need coordination with the orchestrator, regardless of the audit
> container ID size.

Ok, I guess that's as good as I can get it for now, thank you for your
patient explanations.

Simo.

-- 
Simo Sorce
Sr. Principal Software Engineer
Red Hat, Inc

Re: RFC(V3): Audit Kernel Container IDs

[Simo Sorce]

On Fri, 2018-02-02 at 18:24 -0500, Paul Moore wrote:
> On Fri, Feb 2, 2018 at 5:19 PM, Simo Sorce <simo-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> > On Fri, 2018-02-02 at 16:24 -0500, Paul Moore wrote:
> > > On Wed, Jan 10, 2018 at 2:00 AM, Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> > > > On 2018-01-09 11:18, Simo Sorce wrote:
> > > > > On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:
> 
> ...
> 
> > > > Paul, can you justify this somewhat larger inconvenience for some
> > > > relatively minor convenience on our part?
> > > 
> > > Done in direct response to Simo.
> > 
> > Sorry but your response sounds more like waving away then addressing
> > them, the excuse being: we can't please everyone, so we are going to
> > please no one.
> 
> I obviously disagree with the take on my comments but you're free to
> your opinion.

The I misunderstood your comments, I am not interested in putting words
in your mouth.

> I believe saying we are pleasing no one isn't really fair now is it?

Well, of course you are going to please the audit subsystem, I
understand that. I think there is a problem of expectations. Some
people, me included, hoped to have a way to identify a container with
the help of the kernel.

> Is there any type of audit container ID now?  How would you go about
> associating audit events with containers now?

We do not have a good way, there are some dirty tricks like inferring
the container identity via cgroup names, but that is ... eww.
This is why, given audit has the same need of user space, there was
some hope we could agree on an identifier that could be used by both.
It would make correlating audit logs and other cluster-wide events
simpler. That is all.

>  (spoiler alert: it ain't
> pretty, and there are gaps I don't believe you can cover)  This
> proposal provides a mechanism to do this in a way that isn't tied to
> any one particular concept of a container and is manageable inside the
> kernel.

I like the proposal for the most part, we are just discussing on the
nature of the identifier, which is a minor detail in the end.

> If you have a need to track audit events for containers, I find it
> extremely hard to believe that you are not at least partially pleased
> by the solutions presented here.  It may not be everything on your
> wishlist, but when did you ever get *everything* on your wishlist?

It is true, and I am sorry if I came out demanding or abrasive. It was
not my intention. Of course a u64 that has to be mapped is still better
than nothing. It does cause a lot more work in user space, but it is
not impossible to deal with.

> > > But to be clear Richard, we've talked about this a few times, it's not
> > > a "minor convenience" on our part, it's a pretty big convenience once
> > > we starting having to route audit events and make decisions based on
> > > the audit container ID information.  Audit performance is less than
> > > awesome now, I'm working hard to not make it worse.
> > 
> > Sounds like a security vs performance trade off to me.
> 
> Welcome to software development.  It's generally a pretty terrible
> hobby and/or occupation, but we make up for it with long hours and
> endless frustration.

Tell me more about that, not! ;-)

> > > > u64 vs u128 is easy for us to
> > > > accomodate in terms of scalar comparisons.  It doubles the information
> > > > in every container id field we print in audit records.
> > > 
> > > ... and slows down audit container ID checks.
> > 
> > Are you saying a cmp on a u128 is slower than a comparison on a u64 and
> > this is something that will be noticeable ?
> 
> Do you have a 128 bit system?

no, but all 64bit systems have an instruction that allow you to do
atomic 128 compare and swap (IIRC ?).

>   I don't.  I've got a bunch of 64 bit
> systems, and a couple of 32 bit systems too.  People that use audit
> have a tendency to really hammer on it, to the point that we get
> performance complaints on a not infrequent basis.  I don't know the
> exact number of times we are going to need to check the audit
> container ID, but it's reasonable to think that we'll expose it as a
> filter-able field which adds a few checks, we'll use it for record
> routing so that's a few more, and if we're running multiple audit
> daemons we will probably want to include LSM checks which could result
> in a few more audit container ID checks.  If it was one comparison I
> wouldn't be too worried about it, but the point I'm trying to make is
> that we don't know what the implementation is going to look like yet
> and I suspect this ID is going to be leveraged in several places in
> the audit subsystem and I would much rather start small to save
> headaches later.
> 
> We can always expand the ID to a larger integer at a later date, but
> we can't make it smaller.

Well looking through the history of in kernel identifiers I know it is
hard also to increase size, because userspace will end up depending on
a specific size ... and this is the only reason I am really debating
this. If it were really easy to change I wouldn't bother to do it now.

> > > > A c36 is a bigger step.
> > > 
> > > Yeah, we're not doing that, no way.
> > 
> > Ok, I can see your point though I do not agree with it.
> > 
> > I can see why you do not want to have arbitrary length strings, but a
> > u128 sounded like a reasonable compromise to me as it has enough room
> > to be able to have unique cluster-wide IDs which a u64 definitely makes
> > a lot harder to provide w/o tight coordination.
> 
> I originally wanted it to be a 32-bit integer, but Richard managed to
> talk me into 64-bits, that was my compromise :)
> 
> As I said earlier, if you are doing container auditing you're going to
> need coordination with the orchestrator, regardless of the audit
> container ID size.

Ok, I guess that's as good as I can get it for now, thank you for your
patient explanations.

Simo.

-- 
Simo Sorce
Sr. Principal Software Engineer
Red Hat, Inc

Re: RFC(V3): Audit Kernel Container IDs

[Paul Moore]

On Wed, Jan 10, 2018 at 2:00 AM, Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> On 2018-01-09 11:18, Simo Sorce wrote:
>> On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:
>> > Containers are a userspace concept.  The kernel knows nothing of them.
>> >
>> > The Linux audit system needs a way to be able to track the container
>> > provenance of events and actions.  Audit needs the kernel's help to do
>> > this.
>> >
>> > Since the concept of a container is entirely a userspace concept, a
>> > registration from the userspace container orchestration system initiates
>> > this.  This will define a point in time and a set of resources
>> > associated with a particular container with an audit container
>> > identifier.
>> >
>> > The registration is a u64 representing the audit container identifier
>> > written to a special file in a pseudo filesystem (proc, since PID tree
>> > already exists) representing a process that will become a parent process
>> > in that container.  This write might place restrictions on mount
>> > namespaces required to define a container, or at least careful checking
>> > of namespaces in the kernel to verify permissions of the orchestrator so
>> > it can't change its own container ID.  A bind mount of nsfs may be
>> > necessary in the container orchestrator's mount namespace.  This write
>> > can only happen once per process.
>> >
>> > Note: The justification for using a u64 is that it minimizes the
>> > information printed in every audit record, reducing bandwidth and limits
>> > comparisons to a single u64 which will be faster and less error-prone.
>> >
>> > Require CAP_AUDIT_CONTROL to be able to carry out the registration.  At
>> > that time, record the target container's user-supplied audit container
>> > identifier along with a target container's parent process (which may
>> > become the target container's "init" process) process ID (referenced
>> > from the initial PID namespace) in a new record AUDIT_CONTAINER with a
>> > qualifying op=$action field.
>> >
>> > Issue a new auxilliary record AUDIT_CONTAINER_INFO for each valid
>> > container ID present on an auditable action or event.
>> >
>> > Forked and cloned processes inherit their parent's audit container
>> > identifier, referenced in the process' task_struct.  Since the audit
>> > container identifier is inherited rather than written, it can still be
>> > written once.  This will prevent tampering while allowing nesting.
>> > (This can be implemented with an internal settable flag upon
>> > registration that does not get copied across a fork/clone.)
>> >
>> > Mimic setns(2) and return an error if the process has already initiated
>> > threading or forked since this registration should happen before the
>> > process execution is started by the orchestrator and hence should not
>> > yet have any threads or children.  If this is deemed overly restrictive,
>> > switch all of the target's threads and children to the new containerID.
>> >
>> > Trust the orchestrator to judiciously use and restrict CAP_AUDIT_CONTROL.
>> >
>> > When a container ceases to exist because the last process in that
>> > container has exited log the fact to balance the registration action.
>> > (This is likely needed for certification accountability.)
>> >
>> > At this point it appears unnecessary to add a container session
>> > identifier since this is all tracked from loginuid and sessionid to
>> > communicate with the container orchestrator to spawn an additional
>> > session into an existing container which would be logged.  It can be
>> > added at a later date without breaking API should it be deemed
>> > necessary.
>> >
>> > The following namespace logging actions are not needed for certification
>> > purposes at this point, but are helpful for tracking namespace activity.
>> > These are auxilliary records that are associated with namespace
>> > manipulation syscalls unshare(2), clone(2) and setns(2), so the records
>> > will only show up if explicit syscall rules have been added to document
>> > this activity.
>> >
>> > Log the creation of every namespace, inheriting/adding its spawning
>> > process' audit container identifier(s), if applicable.  Include the
>> > spawning and spawned namespace IDs (device and inode number tuples).
>> > [AUDIT_NS_CREATE, AUDIT_NS_DESTROY] [clone(2), unshare(2), setns(2)]
>> > Note: At this point it appears only network namespaces may need to track
>> > container IDs apart from processes since incoming packets may cause an
>> > auditable event before being associated with a process.  Since a
>> > namespace can be shared by processes in different containers, the
>> > namespace will need to track all containers to which it has been
>> > assigned.
>> >
>> > Upon registration, the target process' namespace IDs (in the form of a
>> > nsfs device number and inode number tuple) will be recorded in an
>> > AUDIT_NS_INFO auxilliary record.
>> >
>> > Log the destruction of every namespace that is no longer used by any
>> > process, including the namespace IDs (device and inode number tuples).
>> > [AUDIT_NS_DESTROY] [process exit, unshare(2), setns(2)]
>> >
>> > Issue a new auxilliary record AUDIT_NS_CHANGE listing (opt: op=$action)
>> > the parent and child namespace IDs for any changes to a process'
>> > namespaces. [setns(2)]
>> > Note: It may be possible to combine AUDIT_NS_* record formats and
>> > distinguish them with an op=$action field depending on the fields
>> > required for each message type.
>> >
>> > The audit container identifier will need to be reaped from all
>> > implicated namespaces upon the destruction of a container.
>> >
>> > This namespace information adds supporting information for tracking
>> > events not attributable to specific processes.
>> >
>> > Changelog:
>> >
>> > (Upstream V3)
>> > - switch back to u64 (from pmoore, can be expanded to u128 in future if
>> >   need arises without breaking API.  u32 was originally proposed, up to
>> >   c36 discussed)
>> > - write-once, but children inherit audit container identifier and can
>> >   then still be written once
>> > - switch to CAP_AUDIT_CONTROL
>> > - group namespace actions together, auxilliary records to namespace
>> >   operations.
>> >
>> > (Upstream V2)
>> > - switch from u64 to u128 UUID
>> > - switch from "signal" and "trigger" to "register"
>> > - restrict registration to single process or force all threads and
>> >   children into same container
>>
>> I am trying to understand the back and forth on the ID size.
>>
>> From an orchestrator POV anything that requires tracking a node
>> specific ID is not ideal.
>>
>> Orchestrators tend to span many nodes, and containers tend to have IDs
>> that are either UUID or have a Hash (like SHA256) as identifier.
>>
>> The problem here is two-fold:
>>
>> a) Your auditing requires some mapping to be useful outside of the
>> system.
>> If you aggreggate audit logs outside of the system or you want to
>> correlate the system audit logs with other components dealing with
>> containers, now you need a place where you provide a mapping from your
>> audit u64 to the ID a container has in the rest of the system.
>>
>> b) Now you need a mapping of some sort. The simplest way a container
>> orchestrator can go about this is to just use the UUID or Hash
>> representing their view of the container, truncate it to a u64 and use
>> that for Audit. This means there are some chances there will be a
>> collision and a duplicate u64 ID will be used by the orchestrator as
>> the container ID. What happen in that case ?
>
> Paul, can you justify this somewhat larger inconvenience for some
> relatively minor convenience on our part?

Done in direct response to Simo.

But to be clear Richard, we've talked about this a few times, it's not
a "minor convenience" on our part, it's a pretty big convenience once
we starting having to route audit events and make decisions based on
the audit container ID information.  Audit performance is less than
awesome now, I'm working hard to not make it worse.

> u64 vs u128 is easy for us to
> accomodate in terms of scalar comparisons.  It doubles the information
> in every container id field we print in audit records.

... and slows down audit container ID checks.

> A c36 is a bigger step.

Yeah, we're not doing that, no way.

-- 
paul moore
www.paul-moore.com

Re: RFC(V3): Audit Kernel Container IDs

[Richard Guy Briggs]

On 2018-01-09 11:18, Simo Sorce wrote:
> On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:
> > Containers are a userspace concept.  The kernel knows nothing of them.
> > 
> > The Linux audit system needs a way to be able to track the container
> > provenance of events and actions.  Audit needs the kernel's help to do
> > this.
> > 
> > Since the concept of a container is entirely a userspace concept, a
> > registration from the userspace container orchestration system initiates
> > this.  This will define a point in time and a set of resources
> > associated with a particular container with an audit container
> > identifier.
> > 
> > The registration is a u64 representing the audit container identifier
> > written to a special file in a pseudo filesystem (proc, since PID tree
> > already exists) representing a process that will become a parent process
> > in that container.  This write might place restrictions on mount
> > namespaces required to define a container, or at least careful checking
> > of namespaces in the kernel to verify permissions of the orchestrator so
> > it can't change its own container ID.  A bind mount of nsfs may be
> > necessary in the container orchestrator's mount namespace.  This write
> > can only happen once per process.
> > 
> > Note: The justification for using a u64 is that it minimizes the
> > information printed in every audit record, reducing bandwidth and limits
> > comparisons to a single u64 which will be faster and less error-prone.
> > 
> > Require CAP_AUDIT_CONTROL to be able to carry out the registration.  At
> > that time, record the target container's user-supplied audit container
> > identifier along with a target container's parent process (which may
> > become the target container's "init" process) process ID (referenced
> > from the initial PID namespace) in a new record AUDIT_CONTAINER with a
> > qualifying op=$action field.
> > 
> > Issue a new auxilliary record AUDIT_CONTAINER_INFO for each valid
> > container ID present on an auditable action or event.
> > 
> > Forked and cloned processes inherit their parent's audit container
> > identifier, referenced in the process' task_struct.  Since the audit
> > container identifier is inherited rather than written, it can still be
> > written once.  This will prevent tampering while allowing nesting.
> > (This can be implemented with an internal settable flag upon
> > registration that does not get copied across a fork/clone.)
> > 
> > Mimic setns(2) and return an error if the process has already initiated
> > threading or forked since this registration should happen before the
> > process execution is started by the orchestrator and hence should not
> > yet have any threads or children.  If this is deemed overly restrictive,
> > switch all of the target's threads and children to the new containerID.
> > 
> > Trust the orchestrator to judiciously use and restrict CAP_AUDIT_CONTROL.
> > 
> > When a container ceases to exist because the last process in that
> > container has exited log the fact to balance the registration action.  
> > (This is likely needed for certification accountability.)
> > 
> > At this point it appears unnecessary to add a container session
> > identifier since this is all tracked from loginuid and sessionid to
> > communicate with the container orchestrator to spawn an additional
> > session into an existing container which would be logged.  It can be
> > added at a later date without breaking API should it be deemed
> > necessary.
> > 
> > The following namespace logging actions are not needed for certification
> > purposes at this point, but are helpful for tracking namespace activity.
> > These are auxilliary records that are associated with namespace
> > manipulation syscalls unshare(2), clone(2) and setns(2), so the records
> > will only show up if explicit syscall rules have been added to document
> > this activity.
> > 
> > Log the creation of every namespace, inheriting/adding its spawning
> > process' audit container identifier(s), if applicable.  Include the
> > spawning and spawned namespace IDs (device and inode number tuples).
> > [AUDIT_NS_CREATE, AUDIT_NS_DESTROY] [clone(2), unshare(2), setns(2)]
> > Note: At this point it appears only network namespaces may need to track
> > container IDs apart from processes since incoming packets may cause an
> > auditable event before being associated with a process.  Since a
> > namespace can be shared by processes in different containers, the
> > namespace will need to track all containers to which it has been
> > assigned.
> > 
> > Upon registration, the target process' namespace IDs (in the form of a
> > nsfs device number and inode number tuple) will be recorded in an
> > AUDIT_NS_INFO auxilliary record.
> > 
> > Log the destruction of every namespace that is no longer used by any
> > process, including the namespace IDs (device and inode number tuples).
> > [AUDIT_NS_DESTROY] [process exit, unshare(2), setns(2)]
> > 
> > Issue a new auxilliary record AUDIT_NS_CHANGE listing (opt: op=$action)
> > the parent and child namespace IDs for any changes to a process'
> > namespaces. [setns(2)]
> > Note: It may be possible to combine AUDIT_NS_* record formats and
> > distinguish them with an op=$action field depending on the fields
> > required for each message type.
> > 
> > The audit container identifier will need to be reaped from all
> > implicated namespaces upon the destruction of a container.
> > 
> > This namespace information adds supporting information for tracking
> > events not attributable to specific processes.
> > 
> > Changelog:
> > 
> > (Upstream V3)
> > - switch back to u64 (from pmoore, can be expanded to u128 in future if
> >   need arises without breaking API.  u32 was originally proposed, up to
> >   c36 discussed)
> > - write-once, but children inherit audit container identifier and can
> >   then still be written once
> > - switch to CAP_AUDIT_CONTROL
> > - group namespace actions together, auxilliary records to namespace
> >   operations.
> > 
> > (Upstream V2)
> > - switch from u64 to u128 UUID
> > - switch from "signal" and "trigger" to "register"
> > - restrict registration to single process or force all threads and
> >   children into same container
> 
> I am trying to understand the back and forth on the ID size.
> 
> From an orchestrator POV anything that requires tracking a node
> specific ID is not ideal.
> 
> Orchestrators tend to span many nodes, and containers tend to have IDs
> that are either UUID or have a Hash (like SHA256) as identifier.
> 
> The problem here is two-fold:
> 
> a) Your auditing requires some mapping to be useful outside of the
> system.
> If you aggreggate audit logs outside of the system or you want to
> correlate the system audit logs with other components dealing with
> containers, now you need a place where you provide a mapping from your
> audit u64 to the ID a container has in the rest of the system.
> 
> b) Now you need a mapping of some sort. The simplest way a container
> orchestrator can go about this is to just use the UUID or Hash
> representing their view of the container, truncate it to a u64 and use
> that for Audit. This means there are some chances there will be a
> collision and a duplicate u64 ID will be used by the orchestrator as
> the container ID. What happen in that case ?

Paul, can you justify this somewhat larger inconvenience for some
relatively minor convenience on our part?  u64 vs u128 is easy for us to
accomodate in terms of scalar comparisons.  It doubles the information
in every container id field we print in audit records.  A c36 is a
bigger step.

> Simo.

- RGB

--
Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Sr. S/W Engineer, Kernel Security, Base Operating Systems
Remote, Ottawa, Red Hat Canada
IRC: rgb, SunRaycer
Voice: +1.647.777.2635, Internal: (81) 32635

Re: RFC(V3): Audit Kernel Container IDs

[Paul Moore]

On Tue, Jan 9, 2018 at 11:18 AM, Simo Sorce <simo-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:

...

>> Changelog:
>>
>> (Upstream V3)
>> - switch back to u64 (from pmoore, can be expanded to u128 in future if
>>   need arises without breaking API.  u32 was originally proposed, up to
>>   c36 discussed)
>> - write-once, but children inherit audit container identifier and can
>>   then still be written once
>> - switch to CAP_AUDIT_CONTROL
>> - group namespace actions together, auxilliary records to namespace
>>   operations.
>>
>> (Upstream V2)
>> - switch from u64 to u128 UUID
>> - switch from "signal" and "trigger" to "register"
>> - restrict registration to single process or force all threads and
>>   children into same container
>
> I am trying to understand the back and forth on the ID size.

I'm just now getting a chance to read Richard's latest draft, but I
wanted to comment on this quickly.

There are two main reasons for keeping this a 32 or 64 bit integer:

1) After the initial "be able to associate audit events with a
container" stage, we are going to look into supporting multiple audit
daemons on the system so that you could run an audit daemon inside a
container and it would collect events generated by the container
(we're tentatively calling this "phase 2", feel free to insert your
own "magic happens" joke).  There are a lot things that need to happen
in phase two, one of these things is the addition of an audit event
routing mechanism that will send audit records to the right audit
daemons (the "host" daemon will always see everything), in order to do
this we will need to be able to quickly compare audit container IDs,
this means an integer.

2) Whatever we pick for an audit container ID it is going to be wrong
for at least one container orchestrator.  There is no "one" solution
here, so we are providing a small and flexible mechanism that higher
level orchestrators can use to provide a more complete solution.

> >From an orchestrator POV anything that requires tracking a node
> specific ID is not ideal.
>
> Orchestrators tend to span many nodes, and containers tend to have IDs
> that are either UUID or have a Hash (like SHA256) as identifier.

You're helping me prove my reason #2.

> The problem here is two-fold:
>
> a) Your auditing requires some mapping to be useful outside of the
> system.
> If you aggreggate audit logs outside of the system or you want to
> correlate the system audit logs with other components dealing with
> containers, now you need a place where you provide a mapping from your
> audit u64 to the ID a container has in the rest of the system.

Yep, see my reason #2.  I want us to have something that "works" for a
single system as well as something that can be leveraged by higher
level tools for large networks of machines.

I realize it's easy, and tempting, to expand the scope of this effort;
but if we are to have any success it is only going to be through some
discipline.  We need to focus on a small solution which addresses the
basic needs and hopefully remains flexible enough for any potential
expansion while staying palatable to the audit folks and the general
kernel community.

> b) Now you need a mapping of some sort. The simplest way a container
> orchestrator can go about this is to just use the UUID or Hash
> representing their view of the container, truncate it to a u64 and use
> that for Audit. This means there are some chances there will be a
> collision and a duplicate u64 ID will be used by the orchestrator as
> the container ID. What happen in that case ?

That is a design decision left to the different container orchestrators.

-- 
paul moore
www.paul-moore.com

Re: RFC(V3): Audit Kernel Container IDs

[Paul Moore]

On Tue, Jan 9, 2018 at 11:18 AM, Simo Sorce <simo-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:

...

>> Changelog:
>>
>> (Upstream V3)
>> - switch back to u64 (from pmoore, can be expanded to u128 in future if
>>   need arises without breaking API.  u32 was originally proposed, up to
>>   c36 discussed)
>> - write-once, but children inherit audit container identifier and can
>>   then still be written once
>> - switch to CAP_AUDIT_CONTROL
>> - group namespace actions together, auxilliary records to namespace
>>   operations.
>>
>> (Upstream V2)
>> - switch from u64 to u128 UUID
>> - switch from "signal" and "trigger" to "register"
>> - restrict registration to single process or force all threads and
>>   children into same container
>
> I am trying to understand the back and forth on the ID size.

I'm just now getting a chance to read Richard's latest draft, but I
wanted to comment on this quickly.

There are two main reasons for keeping this a 32 or 64 bit integer:

1) After the initial "be able to associate audit events with a
container" stage, we are going to look into supporting multiple audit
daemons on the system so that you could run an audit daemon inside a
container and it would collect events generated by the container
(we're tentatively calling this "phase 2", feel free to insert your
own "magic happens" joke).  There are a lot things that need to happen
in phase two, one of these things is the addition of an audit event
routing mechanism that will send audit records to the right audit
daemons (the "host" daemon will always see everything), in order to do
this we will need to be able to quickly compare audit container IDs,
this means an integer.

2) Whatever we pick for an audit container ID it is going to be wrong
for at least one container orchestrator.  There is no "one" solution
here, so we are providing a small and flexible mechanism that higher
level orchestrators can use to provide a more complete solution.

> >From an orchestrator POV anything that requires tracking a node
> specific ID is not ideal.
>
> Orchestrators tend to span many nodes, and containers tend to have IDs
> that are either UUID or have a Hash (like SHA256) as identifier.

You're helping me prove my reason #2.

> The problem here is two-fold:
>
> a) Your auditing requires some mapping to be useful outside of the
> system.
> If you aggreggate audit logs outside of the system or you want to
> correlate the system audit logs with other components dealing with
> containers, now you need a place where you provide a mapping from your
> audit u64 to the ID a container has in the rest of the system.

Yep, see my reason #2.  I want us to have something that "works" for a
single system as well as something that can be leveraged by higher
level tools for large networks of machines.

I realize it's easy, and tempting, to expand the scope of this effort;
but if we are to have any success it is only going to be through some
discipline.  We need to focus on a small solution which addresses the
basic needs and hopefully remains flexible enough for any potential
expansion while staying palatable to the audit folks and the general
kernel community.

> b) Now you need a mapping of some sort. The simplest way a container
> orchestrator can go about this is to just use the UUID or Hash
> representing their view of the container, truncate it to a u64 and use
> that for Audit. This means there are some chances there will be a
> collision and a duplicate u64 ID will be used by the orchestrator as
> the container ID. What happen in that case ?

That is a design decision left to the different container orchestrators.

-- 
paul moore
www.paul-moore.com

Re: RFC(V3): Audit Kernel Container IDs

[Paul Moore]

On Tue, Jan 9, 2018 at 11:18 AM, Simo Sorce <simo@redhat.com> wrote:
> On Tue, 2018-01-09 at 07:16 -0500, Richard Guy Briggs wrote:

...

>> Changelog:
>>
>> (Upstream V3)
>> - switch back to u64 (from pmoore, can be expanded to u128 in future if
>>   need arises without breaking API.  u32 was originally proposed, up to
>>   c36 discussed)
>> - write-once, but children inherit audit container identifier and can
>>   then still be written once
>> - switch to CAP_AUDIT_CONTROL
>> - group namespace actions together, auxilliary records to namespace
>>   operations.
>>
>> (Upstream V2)
>> - switch from u64 to u128 UUID
>> - switch from "signal" and "trigger" to "register"
>> - restrict registration to single process or force all threads and
>>   children into same container
>
> I am trying to understand the back and forth on the ID size.

I'm just now getting a chance to read Richard's latest draft, but I
wanted to comment on this quickly.

There are two main reasons for keeping this a 32 or 64 bit integer:

1) After the initial "be able to associate audit events with a
container" stage, we are going to look into supporting multiple audit
daemons on the system so that you could run an audit daemon inside a
container and it would collect events generated by the container
(we're tentatively calling this "phase 2", feel free to insert your
own "magic happens" joke).  There are a lot things that need to happen
in phase two, one of these things is the addition of an audit event
routing mechanism that will send audit records to the right audit
daemons (the "host" daemon will always see everything), in order to do
this we will need to be able to quickly compare audit container IDs,
this means an integer.

2) Whatever we pick for an audit container ID it is going to be wrong
for at least one container orchestrator.  There is no "one" solution
here, so we are providing a small and flexible mechanism that higher
level orchestrators can use to provide a more complete solution.

> >From an orchestrator POV anything that requires tracking a node
> specific ID is not ideal.
>
> Orchestrators tend to span many nodes, and containers tend to have IDs
> that are either UUID or have a Hash (like SHA256) as identifier.

You're helping me prove my reason #2.

> The problem here is two-fold:
>
> a) Your auditing requires some mapping to be useful outside of the
> system.
> If you aggreggate audit logs outside of the system or you want to
> correlate the system audit logs with other components dealing with
> containers, now you need a place where you provide a mapping from your
> audit u64 to the ID a container has in the rest of the system.

Yep, see my reason #2.  I want us to have something that "works" for a
single system as well as something that can be leveraged by higher
level tools for large networks of machines.

I realize it's easy, and tempting, to expand the scope of this effort;
but if we are to have any success it is only going to be through some
discipline.  We need to focus on a small solution which addresses the
basic needs and hopefully remains flexible enough for any potential
expansion while staying palatable to the audit folks and the general
kernel community.

> b) Now you need a mapping of some sort. The simplest way a container
> orchestrator can go about this is to just use the UUID or Hash
> representing their view of the container, truncate it to a u64 and use
> that for Audit. This means there are some chances there will be a
> collision and a duplicate u64 ID will be used by the orchestrator as
> the container ID. What happen in that case ?

That is a design decision left to the different container orchestrators.

-- 
paul moore
www.paul-moore.com

Re: RFC(V3): Audit Kernel Container IDs

[Eric W. Biederman]

Please let's have a description of the problem you are trying to solve.

A proposed solution without talking about the problem space is useless.
Any proposed solution could potentially work.

I know to these exist.  There is motivation for your work.
What is the motivation?
What problem are you trying to solve?

In particular what information are you trying to get into logs that you
can not get into the logs today?

I am going to try to give this the attention it deserves but right now I
am having to deal with half thought out patches for information leaks
from speculative code paths, so I won't be able to give this much
attention for a little bit.

Eric

Re: RFC(V3): Audit Kernel Container IDs

[Eric W. Biederman]

Please let's have a description of the problem you are trying to solve.

A proposed solution without talking about the problem space is useless.
Any proposed solution could potentially work.

I know to these exist.  There is motivation for your work.
What is the motivation?
What problem are you trying to solve?

In particular what information are you trying to get into logs that you
can not get into the logs today?

I am going to try to give this the attention it deserves but right now I
am having to deal with half thought out patches for information leaks
from speculative code paths, so I won't be able to give this much
attention for a little bit.

Eric

Re: RFC(V3): Audit Kernel Container IDs

[Eric W. Biederman]

Please let's have a description of the problem you are trying to solve.

A proposed solution without talking about the problem space is useless.
Any proposed solution could potentially work.

I know to these exist.  There is motivation for your work.
What is the motivation?
What problem are you trying to solve?

In particular what information are you trying to get into logs that you
can not get into the logs today?

I am going to try to give this the attention it deserves but right now I
am having to deal with half thought out patches for information leaks
from speculative code paths, so I won't be able to give this much
attention for a little bit.

Eric

Re: RFC(V3): Audit Kernel Container IDs

[Richard Guy Briggs]

On 2018-01-09 19:05, Eric W. Biederman wrote:
> Please let's have a description of the problem you are trying to solve.

I thought the first sentence of the second paragraph summed it up rather
well.

Here are the elaborated motivations:

- Filter unwanted, irrelevant or unimportant messages before they fill
  queue so important messages don't get lost.  This is a certification
  requirement.

- Make security claims about containers, require tracking of actions
  within those containers to ensure compliance with established security
  policies.

- Route messages from events to local audit daemon instance or host
  audit daemon instance

- Tried nsIDs, but insufficient for efficient filtering, routing,
  tracking

> A proposed solution without talking about the problem space is useless.
> Any proposed solution could potentially work.
> 
> I know to these exist.  There is motivation for your work.
> What is the motivation?
> What problem are you trying to solve?
> 
> In particular what information are you trying to get into logs that you
> can not get into the logs today?
> 
> I am going to try to give this the attention it deserves but right now I
> am having to deal with half thought out patches for information leaks
> from speculative code paths, so I won't be able to give this much
> attention for a little bit.
> 
> Eric

- RGB

--
Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Sr. S/W Engineer, Kernel Security, Base Operating Systems
Remote, Ottawa, Red Hat Canada
IRC: rgb, SunRaycer
Voice: +1.647.777.2635, Internal: (81) 32635

Re: RFC(V3): Audit Kernel Container IDs

[Richard Guy Briggs]

On 2018-01-09 19:05, Eric W. Biederman wrote:
> Please let's have a description of the problem you are trying to solve.

I thought the first sentence of the second paragraph summed it up rather
well.

Here are the elaborated motivations:

- Filter unwanted, irrelevant or unimportant messages before they fill
  queue so important messages don't get lost.  This is a certification
  requirement.

- Make security claims about containers, require tracking of actions
  within those containers to ensure compliance with established security
  policies.

- Route messages from events to local audit daemon instance or host
  audit daemon instance

- Tried nsIDs, but insufficient for efficient filtering, routing,
  tracking

> A proposed solution without talking about the problem space is useless.
> Any proposed solution could potentially work.
> 
> I know to these exist.  There is motivation for your work.
> What is the motivation?
> What problem are you trying to solve?
> 
> In particular what information are you trying to get into logs that you
> can not get into the logs today?
> 
> I am going to try to give this the attention it deserves but right now I
> am having to deal with half thought out patches for information leaks
> from speculative code paths, so I won't be able to give this much
> attention for a little bit.
> 
> Eric

- RGB

--
Richard Guy Briggs <rgb@redhat.com>
Sr. S/W Engineer, Kernel Security, Base Operating Systems
Remote, Ottawa, Red Hat Canada
IRC: rgb, SunRaycer
Voice: +1.647.777.2635, Internal: (81) 32635

Re: RFC(V3): Audit Kernel Container IDs

[Richard Guy Briggs]

On 2018-01-09 19:05, Eric W. Biederman wrote:
> Please let's have a description of the problem you are trying to solve.

I thought the first sentence of the second paragraph summed it up rather
well.

Here are the elaborated motivations:

- Filter unwanted, irrelevant or unimportant messages before they fill
  queue so important messages don't get lost.  This is a certification
  requirement.

- Make security claims about containers, require tracking of actions
  within those containers to ensure compliance with established security
  policies.

- Route messages from events to local audit daemon instance or host
  audit daemon instance

- Tried nsIDs, but insufficient for efficient filtering, routing,
  tracking

> A proposed solution without talking about the problem space is useless.
> Any proposed solution could potentially work.
> 
> I know to these exist.  There is motivation for your work.
> What is the motivation?
> What problem are you trying to solve?
> 
> In particular what information are you trying to get into logs that you
> can not get into the logs today?
> 
> I am going to try to give this the attention it deserves but right now I
> am having to deal with half thought out patches for information leaks
> from speculative code paths, so I won't be able to give this much
> attention for a little bit.
> 
> Eric

- RGB

--
Richard Guy Briggs <rgb@redhat.com>
Sr. S/W Engineer, Kernel Security, Base Operating Systems
Remote, Ottawa, Red Hat Canada
IRC: rgb, SunRaycer
Voice: +1.647.777.2635, Internal: (81) 32635

Re: RFC(V3): Audit Kernel Container IDs

[Paul Moore]

On Tue, Jan 9, 2018 at 7:16 AM, Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> Containers are a userspace concept.  The kernel knows nothing of them.
>
> The Linux audit system needs a way to be able to track the container
> provenance of events and actions.  Audit needs the kernel's help to do
> this.

Two small comments below, but I tend to think we are at a point where
you can start cobbling together some prototype/RFC patches.  Surely
there are going to be a few changes, and new comments, that come out
once we see an initial implementation so let's see what those are.

> The registration is a u64 representing the audit container identifier
> written to a special file in a pseudo filesystem (proc, since PID tree
> already exists) representing a process that will become a parent process
> in that container.  This write might place restrictions on mount
> namespaces required to define a container, or at least careful checking
> of namespaces in the kernel to verify permissions of the orchestrator so
> it can't change its own container ID.  A bind mount of nsfs may be
> necessary in the container orchestrator's mount namespace.  This write
> can only happen once per process.
>
> Note: The justification for using a u64 is that it minimizes the
> information printed in every audit record, reducing bandwidth and limits
> comparisons to a single u64 which will be faster and less error-prone.

I know Steve generally worries about audit record size, which is a
perfectly valid concern in this case, I also worry about the
additional overhead when we start routing audit records to multiple
audit daemons (see my other emails in this thread).

> ...
> When a container ceases to exist because the last process in that
> container has exited log the fact to balance the registration action.
> (This is likely needed for certification accountability.)

On the "container ceases to exist" point, I expect this "container
dead" message to come from the orchestrator and not the kernel itself
(I don't want the kernel to have to handle that level of bookkeeping).
I imagine this should be similar to what is done for VM auditing with
libvirt.

-- 
paul moore
www.paul-moore.com

Re: RFC(V3): Audit Kernel Container IDs

[Paul Moore]

On Tue, Jan 9, 2018 at 7:16 AM, Richard Guy Briggs <rgb@redhat.com> wrote:
> Containers are a userspace concept.  The kernel knows nothing of them.
>
> The Linux audit system needs a way to be able to track the container
> provenance of events and actions.  Audit needs the kernel's help to do
> this.

Two small comments below, but I tend to think we are at a point where
you can start cobbling together some prototype/RFC patches.  Surely
there are going to be a few changes, and new comments, that come out
once we see an initial implementation so let's see what those are.

> The registration is a u64 representing the audit container identifier
> written to a special file in a pseudo filesystem (proc, since PID tree
> already exists) representing a process that will become a parent process
> in that container.  This write might place restrictions on mount
> namespaces required to define a container, or at least careful checking
> of namespaces in the kernel to verify permissions of the orchestrator so
> it can't change its own container ID.  A bind mount of nsfs may be
> necessary in the container orchestrator's mount namespace.  This write
> can only happen once per process.
>
> Note: The justification for using a u64 is that it minimizes the
> information printed in every audit record, reducing bandwidth and limits
> comparisons to a single u64 which will be faster and less error-prone.

I know Steve generally worries about audit record size, which is a
perfectly valid concern in this case, I also worry about the
additional overhead when we start routing audit records to multiple
audit daemons (see my other emails in this thread).

> ...
> When a container ceases to exist because the last process in that
> container has exited log the fact to balance the registration action.
> (This is likely needed for certification accountability.)

On the "container ceases to exist" point, I expect this "container
dead" message to come from the orchestrator and not the kernel itself
(I don't want the kernel to have to handle that level of bookkeeping).
I imagine this should be similar to what is done for VM auditing with
libvirt.

-- 
paul moore
www.paul-moore.com

Re: RFC(V3): Audit Kernel Container IDs

[Serge E. Hallyn]

On Fri, Feb 02, 2018 at 05:05:22PM -0500, Paul Moore wrote:
> On Tue, Jan 9, 2018 at 7:16 AM, Richard Guy Briggs <rgb@redhat.com> wrote:
> > Containers are a userspace concept.  The kernel knows nothing of them.
> >
> > The Linux audit system needs a way to be able to track the container
> > provenance of events and actions.  Audit needs the kernel's help to do
> > this.
> 
> Two small comments below, but I tend to think we are at a point where
> you can start cobbling together some prototype/RFC patches.  Surely

Agreed.

LGTM.

> there are going to be a few changes, and new comments, that come out
> once we see an initial implementation so let's see what those are.

thanks,
-serge

Re: RFC(V3): Audit Kernel Container IDs

[Serge E. Hallyn]

On Fri, Feb 02, 2018 at 05:05:22PM -0500, Paul Moore wrote:
> On Tue, Jan 9, 2018 at 7:16 AM, Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> > Containers are a userspace concept.  The kernel knows nothing of them.
> >
> > The Linux audit system needs a way to be able to track the container
> > provenance of events and actions.  Audit needs the kernel's help to do
> > this.
> 
> Two small comments below, but I tend to think we are at a point where
> you can start cobbling together some prototype/RFC patches.  Surely

Agreed.

LGTM.

> there are going to be a few changes, and new comments, that come out
> once we see an initial implementation so let's see what those are.

thanks,
-serge

RFC(V3): Audit Kernel Container IDs

[Richard Guy Briggs]

Containers are a userspace concept.  The kernel knows nothing of them.

The Linux audit system needs a way to be able to track the container
provenance of events and actions.  Audit needs the kernel's help to do
this.

Since the concept of a container is entirely a userspace concept, a
registration from the userspace container orchestration system initiates
this.  This will define a point in time and a set of resources
associated with a particular container with an audit container
identifier.

The registration is a u64 representing the audit container identifier
written to a special file in a pseudo filesystem (proc, since PID tree
already exists) representing a process that will become a parent process
in that container.  This write might place restrictions on mount
namespaces required to define a container, or at least careful checking
of namespaces in the kernel to verify permissions of the orchestrator so
it can't change its own container ID.  A bind mount of nsfs may be
necessary in the container orchestrator's mount namespace.  This write
can only happen once per process.

Note: The justification for using a u64 is that it minimizes the
information printed in every audit record, reducing bandwidth and limits
comparisons to a single u64 which will be faster and less error-prone.

Require CAP_AUDIT_CONTROL to be able to carry out the registration.  At
that time, record the target container's user-supplied audit container
identifier along with a target container's parent process (which may
become the target container's "init" process) process ID (referenced
from the initial PID namespace) in a new record AUDIT_CONTAINER with a
qualifying op=$action field.

Issue a new auxilliary record AUDIT_CONTAINER_INFO for each valid
container ID present on an auditable action or event.

Forked and cloned processes inherit their parent's audit container
identifier, referenced in the process' task_struct.  Since the audit
container identifier is inherited rather than written, it can still be
written once.  This will prevent tampering while allowing nesting.
(This can be implemented with an internal settable flag upon
registration that does not get copied across a fork/clone.)

Mimic setns(2) and return an error if the process has already initiated
threading or forked since this registration should happen before the
process execution is started by the orchestrator and hence should not
yet have any threads or children.  If this is deemed overly restrictive,
switch all of the target's threads and children to the new containerID.

Trust the orchestrator to judiciously use and restrict CAP_AUDIT_CONTROL.

When a container ceases to exist because the last process in that
container has exited log the fact to balance the registration action.  
(This is likely needed for certification accountability.)

At this point it appears unnecessary to add a container session
identifier since this is all tracked from loginuid and sessionid to
communicate with the container orchestrator to spawn an additional
session into an existing container which would be logged.  It can be
added at a later date without breaking API should it be deemed
necessary.

The following namespace logging actions are not needed for certification
purposes at this point, but are helpful for tracking namespace activity.
These are auxilliary records that are associated with namespace
manipulation syscalls unshare(2), clone(2) and setns(2), so the records
will only show up if explicit syscall rules have been added to document
this activity.

Log the creation of every namespace, inheriting/adding its spawning
process' audit container identifier(s), if applicable.  Include the
spawning and spawned namespace IDs (device and inode number tuples).
[AUDIT_NS_CREATE, AUDIT_NS_DESTROY] [clone(2), unshare(2), setns(2)]
Note: At this point it appears only network namespaces may need to track
container IDs apart from processes since incoming packets may cause an
auditable event before being associated with a process.  Since a
namespace can be shared by processes in different containers, the
namespace will need to track all containers to which it has been
assigned.

Upon registration, the target process' namespace IDs (in the form of a
nsfs device number and inode number tuple) will be recorded in an
AUDIT_NS_INFO auxilliary record.

Log the destruction of every namespace that is no longer used by any
process, including the namespace IDs (device and inode number tuples).
[AUDIT_NS_DESTROY] [process exit, unshare(2), setns(2)]

Issue a new auxilliary record AUDIT_NS_CHANGE listing (opt: op=$action)
the parent and child namespace IDs for any changes to a process'
namespaces. [setns(2)]
Note: It may be possible to combine AUDIT_NS_* record formats and
distinguish them with an op=$action field depending on the fields
required for each message type.

The audit container identifier will need to be reaped from all
implicated namespaces upon the destruction of a container.

This namespace information adds supporting information for tracking
events not attributable to specific processes.

Changelog:

(Upstream V3)
- switch back to u64 (from pmoore, can be expanded to u128 in future if
  need arises without breaking API.  u32 was originally proposed, up to
  c36 discussed)
- write-once, but children inherit audit container identifier and can
  then still be written once
- switch to CAP_AUDIT_CONTROL
- group namespace actions together, auxilliary records to namespace
  operations.

(Upstream V2)
- switch from u64 to u128 UUID
- switch from "signal" and "trigger" to "register"
- restrict registration to single process or force all threads and
  children into same container

- RGB

--
Richard Guy Briggs <rgb-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Sr. S/W Engineer, Kernel Security, Base Operating Systems
Remote, Ottawa, Red Hat Canada
IRC: rgb, SunRaycer
Voice: +1.647.777.2635, Internal: (81) 32635