Re: [PATCH 0/2] core dump: re-purpose VM_ALWAYSDUMP to user controlled VM_DONTDUMP

Re: [PATCH 0/2] core dump: re-purpose VM_ALWAYSDUMP to user controlled VM_DONTDUMP

[Michael Kerrisk]

Jason,

On Thu, Mar 8, 2012 at 6:00 AM, Jason Baron <jbaron-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> Hi,
>
> The motivation for this change was that I was looking at a way for a qemu-kvm
> process, to exclude the guest memory from its core dump, which can be quite
> large. There are already a number of filter flags in
> /proc/<pid>/coredump_filter, however, these allow one to specify 'types' of
> kernel memory, not specific address ranges (which is needed in this case).
>
> Since there are no more vma flags available, the first patch eliminates the
> need for the 'VM_ALWAYSDUMP' flag. The flag is used internally by the kernel to
> mark vdso and vsyscall pages. However, it is simple enough to check if a vma
> covers a vdso or vsyscall page without the need for this flag.
>
> The second patch then replaces the 'VM_ALWAYSDUMP' flag with a new
> 'VM_DONTDUMP' flag, which can be set by userspace using new madvise flags:
> 'MADV_DONTDUMP', and unset via 'MADV_DUMP'. The core dump filters continue to
> work the same as before unless 'MADV_DONTDUMP' is set on the region.
>
> The qemu code which implements this features is at:
> http://people.redhat.com/~jbaron/qemu-dump/qemu-dump.patch
>
> In my testing the qemu core dump shrunk from 383MB -> 13MB with this patch.
>
> I also believe that the 'MADV_DONTDUMP' flag might be useful for security
> sensitive apps, which might want to select which areas are dumped.

Since we have
MADV_DODUMP
MADV_DONTDUMP
MADV_NODUMP
heading for userspace in 3.4, would you be willing to write patches
for the madvise(2) man page to describe these flags?

See http://www.kernel.org/doc/man-pages/download.html for details on
accessing man-pages Git.

Cheers,

Michael

PS Please also CC linux-api@ when making API/ABI changes.


-- 
Michael Kerrisk Linux man-pages maintainer;
http://www.kernel.org/doc/man-pages/
Author of "The Linux Programming Interface", http://blog.man7.org/

Re: [PATCH 0/2] core dump: re-purpose VM_ALWAYSDUMP to user controlled VM_DONTDUMP

[Jason Baron]

On Tue, Apr 24, 2012 at 10:42:16AM +1200, Michael Kerrisk wrote:
> Jason,
> 
> On Thu, Mar 8, 2012 at 6:00 AM, Jason Baron <jbaron@redhat.com> wrote:
> > Hi,
> >
> > The motivation for this change was that I was looking at a way for a qemu-kvm
> > process, to exclude the guest memory from its core dump, which can be quite
> > large. There are already a number of filter flags in
> > /proc/<pid>/coredump_filter, however, these allow one to specify 'types' of
> > kernel memory, not specific address ranges (which is needed in this case).
> >
> > Since there are no more vma flags available, the first patch eliminates the
> > need for the 'VM_ALWAYSDUMP' flag. The flag is used internally by the kernel to
> > mark vdso and vsyscall pages. However, it is simple enough to check if a vma
> > covers a vdso or vsyscall page without the need for this flag.
> >
> > The second patch then replaces the 'VM_ALWAYSDUMP' flag with a new
> > 'VM_DONTDUMP' flag, which can be set by userspace using new madvise flags:
> > 'MADV_DONTDUMP', and unset via 'MADV_DUMP'. The core dump filters continue to
> > work the same as before unless 'MADV_DONTDUMP' is set on the region.
> >
> > The qemu code which implements this features is at:
> > http://people.redhat.com/~jbaron/qemu-dump/qemu-dump.patch
> >
> > In my testing the qemu core dump shrunk from 383MB -> 13MB with this patch.
> >
> > I also believe that the 'MADV_DONTDUMP' flag might be useful for security
> > sensitive apps, which might want to select which areas are dumped.
> 
> Since we have
> MADV_DODUMP
> MADV_DONTDUMP
> MADV_NODUMP
> heading for userspace in 3.4, would you be willing to write patches
> for the madvise(2) man page to describe these flags?
> 
> See http://www.kernel.org/doc/man-pages/download.html for details on
> accessing man-pages Git.
> 
> Cheers,
> 
> Michael
> 
> PS Please also CC linux-api@ when making API/ABI changes.
> 

Ok, here's a stab at manpage patch, let me know if I should send it as a
separate patch. Thanks.

-Jason


diff --git a/man2/madvise.2 b/man2/madvise.2
index 36f988a..472c23a 100644
--- a/man2/madvise.2
+++ b/man2/madvise.2
@@ -247,6 +247,22 @@ Ensures that memory in the address range specified by
 and
 .IR length
 will not be collapsed into huge pages.
+.TP
+.BR MADV_DONTDUMP " (since Linux 3.4)"
+Explicitly exclude from a core dump those pages in the range specified by
+.I addr
+and
+.IR length .
+Applications may have large areas of memory which are known not to be useful in
+diagnosing a core dump. This specification takes precedence over the bit mask that
+is set via the
+.I /proc/PID/coredump_filter
+file (see
+.BR core (5)).
+.TP
+.BR MADV_DODUMP " (since Linux 3.4)"
+Undo the effect of an earlier
+.BR MADV_DONTDUMP.
 .SH "RETURN VALUE"
 On success
 .BR madvise ()
@@ -356,4 +372,5 @@ from the system call, as it should).
 .BR mmap (2),
 .BR mprotect (2),
 .BR msync (2),
-.BR munmap (2)
+.BR munmap (2),
+.BR core (5)

Re: [PATCH 0/2] core dump: re-purpose VM_ALWAYSDUMP to user controlled VM_DONTDUMP

[Michael Kerrisk (man-pages)]

Hello Jason,

On Sat, Apr 28, 2012 at 6:43 AM, Jason Baron <jbaron-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
> On Tue, Apr 24, 2012 at 10:42:16AM +1200, Michael Kerrisk wrote:
>> Jason,
>>
>> On Thu, Mar 8, 2012 at 6:00 AM, Jason Baron <jbaron-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:
>> > Hi,
>> >
>> > The motivation for this change was that I was looking at a way for a qemu-kvm
>> > process, to exclude the guest memory from its core dump, which can be quite
>> > large. There are already a number of filter flags in
>> > /proc/<pid>/coredump_filter, however, these allow one to specify 'types' of
>> > kernel memory, not specific address ranges (which is needed in this case).
>> >
>> > Since there are no more vma flags available, the first patch eliminates the
>> > need for the 'VM_ALWAYSDUMP' flag. The flag is used internally by the kernel to
>> > mark vdso and vsyscall pages. However, it is simple enough to check if a vma
>> > covers a vdso or vsyscall page without the need for this flag.
>> >
>> > The second patch then replaces the 'VM_ALWAYSDUMP' flag with a new
>> > 'VM_DONTDUMP' flag, which can be set by userspace using new madvise flags:
>> > 'MADV_DONTDUMP', and unset via 'MADV_DUMP'. The core dump filters continue to
>> > work the same as before unless 'MADV_DONTDUMP' is set on the region.
>> >
>> > The qemu code which implements this features is at:
>> > http://people.redhat.com/~jbaron/qemu-dump/qemu-dump.patch
>> >
>> > In my testing the qemu core dump shrunk from 383MB -> 13MB with this patch.
>> >
>> > I also believe that the 'MADV_DONTDUMP' flag might be useful for security
>> > sensitive apps, which might want to select which areas are dumped.
>>
>> Since we have
>> MADV_DODUMP
>> MADV_DONTDUMP
>> MADV_NODUMP
>> heading for userspace in 3.4, would you be willing to write patches
>> for the madvise(2) man page to describe these flags?
>>
>> See http://www.kernel.org/doc/man-pages/download.html for details on
>> accessing man-pages Git.
>>
>> Cheers,
>>
>> Michael
>>
>> PS Please also CC linux-api@ when making API/ABI changes.
>>
>
> Ok, here's a stab at manpage patch, let me know if I should send it as a
> separate patch. Thanks.
>
> -Jason
>
>
> diff --git a/man2/madvise.2 b/man2/madvise.2
> index 36f988a..472c23a 100644
> --- a/man2/madvise.2
> +++ b/man2/madvise.2
> @@ -247,6 +247,22 @@ Ensures that memory in the address range specified by
>  and
>  .IR length
>  will not be collapsed into huge pages.
> +.TP
> +.BR MADV_DONTDUMP " (since Linux 3.4)"
> +Explicitly exclude from a core dump those pages in the range specified by
> +.I addr
> +and
> +.IR length .
> +Applications may have large areas of memory which are known not to be useful in
> +diagnosing a core dump. This specification takes precedence over the bit mask that
> +is set via the
> +.I /proc/PID/coredump_filter
> +file (see
> +.BR core (5)).
> +.TP
> +.BR MADV_DODUMP " (since Linux 3.4)"
> +Undo the effect of an earlier
> +.BR MADV_DONTDUMP.
>  .SH "RETURN VALUE"
>  On success
>  .BR madvise ()
> @@ -356,4 +372,5 @@ from the system call, as it should).
>  .BR mmap (2),
>  .BR mprotect (2),
>  .BR msync (2),
> -.BR munmap (2)
> +.BR munmap (2),
> +.BR core (5)

Thanks! I tweaked this patch a little and applied the version below.

Cheers,

Michael

--- a/man2/madvise.2
+++ b/man2/madvise.2
@@ -247,6 +247,24 @@ Ensures that memory in the address range specified by
 and
 .IR length
 will not be collapsed into huge pages.
+.TP
+.BR MADV_DONTDUMP " (since Linux 3.4)"
+Exclude from a core dump those pages in the range specified by
+.I addr
+and
+.IR length .
+This is useful in applications that have large areas of memory
+that are known not to be useful in a core dump.
+The effect of
+.BR MADV_DONTDUMP
+takes precedence over the bit mask that is set via the
+.I /proc/PID/coredump_filter
+file (see
+.BR core (5)).
+.TP
+.BR MADV_DODUMP " (since Linux 3.4)"
+Undo the effect of an earlier
+.BR MADV_DONTDUMP .
 .SH "RETURN VALUE"
 On success
 .BR madvise ()
@@ -356,4 +374,5 @@ from the system call, as it should).
 .BR mmap (2),
 .BR mprotect (2),
 .BR msync (2),
-.BR munmap (2)
+.BR munmap (2),
+.BR core (5)

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Author of "The Linux Programming Interface"; http://man7.org/tlpi/