Does sendfile() copy extended attributes?

Does sendfile() copy extended attributes?

[Joseph D. Wagner]

I want to use sendfile() to completely copy a file so that the copy is an 
EXACT, INDISTINGUISHABLE duplicate of the original.

Hence, I need to know if sendfile() copies the extended attributes in 
addition to the file itself.

While I have your attention, does sendfile() also copy the inode attributes 
(i.e. secure delete, compressed, etc.)?

TIA.

Joseph D. Wagner

Re: Does sendfile() copy extended attributes?

[Shaya Potter]

On Sun, 2003-12-14 at 12:39, Joseph D. Wagner wrote:
> I want to use sendfile() to completely copy a file so that the copy is an 
> EXACT, INDISTINGUISHABLE duplicate of the original.
> 
> Hence, I need to know if sendfile() copies the extended attributes in 
> addition to the file itself.
> 
> While I have your attention, does sendfile() also copy the inode attributes 
> (i.e. secure delete, compressed, etc.)?

sendfile acting as a sys_copy() is a 2.4 unsupported behavior.  It
doesn't work on 2.6.

With that proviso, even in 2.4 it just operates on the page data of an
inode, it has no concept of any of the meta data.

Re: Does sendfile() copy extended attributes?

[Jeff Garzik]

Shaya Potter wrote:
> On Sun, 2003-12-14 at 12:39, Joseph D. Wagner wrote:
> 
>>I want to use sendfile() to completely copy a file so that the copy is an 
>>EXACT, INDISTINGUISHABLE duplicate of the original.
>>
>>Hence, I need to know if sendfile() copies the extended attributes in 
>>addition to the file itself.
>>
>>While I have your attention, does sendfile() also copy the inode attributes 
>>(i.e. secure delete, compressed, etc.)?
> 
> 
> sendfile acting as a sys_copy() is a 2.4 unsupported behavior.  It
> doesn't work on 2.6.


Have you tested this assertion lately?  :)

AFAIK file-file sendfile(2) was fixed in 2.6.

	Jeff

Re: Does sendfile() copy extended attributes?

[Shaya Potter]

On Mon, 2003-12-15 at 00:46, Jeff Garzik wrote:
> Shaya Potter wrote:
> > On Sun, 2003-12-14 at 12:39, Joseph D. Wagner wrote:
> > 
> >>I want to use sendfile() to completely copy a file so that the copy is an 
> >>EXACT, INDISTINGUISHABLE duplicate of the original.
> >>
> >>Hence, I need to know if sendfile() copies the extended attributes in 
> >>addition to the file itself.
> >>
> >>While I have your attention, does sendfile() also copy the inode attributes 
> >>(i.e. secure delete, compressed, etc.)?
> > 
> > 
> > sendfile acting as a sys_copy() is a 2.4 unsupported behavior.  It
> > doesn't work on 2.6.
> 
> 
> Have you tested this assertion lately?  :)
> 
> AFAIK file-file sendfile(2) was fixed in 2.6.

didn't work on test6, haven't tried it since.

Linux zaphod 2.6.0-test6 #1 Sat Oct 4 22:35:19 EDT 2003 i686 GNU/Linux
spotter@zaphod:~$ touch a
spotter@zaphod:~$ touch b
spotter@zaphod:~$ ./a.out a b
error from sendfile: Invalid argument

Re: Does sendfile() copy extended attributes?

[Jeff Garzik]

Shaya Potter wrote:
> On Mon, 2003-12-15 at 00:46, Jeff Garzik wrote:
> 
>>Shaya Potter wrote:
>>
>>>On Sun, 2003-12-14 at 12:39, Joseph D. Wagner wrote:
>>>
>>>
>>>>I want to use sendfile() to completely copy a file so that the copy is an 
>>>>EXACT, INDISTINGUISHABLE duplicate of the original.
>>>>
>>>>Hence, I need to know if sendfile() copies the extended attributes in 
>>>>addition to the file itself.
>>>>
>>>>While I have your attention, does sendfile() also copy the inode attributes 
>>>>(i.e. secure delete, compressed, etc.)?
>>>
>>>
>>>sendfile acting as a sys_copy() is a 2.4 unsupported behavior.  It
>>>doesn't work on 2.6.
>>
>>
>>Have you tested this assertion lately?  :)
>>
>>AFAIK file-file sendfile(2) was fixed in 2.6.
> 
> 
> didn't work on test6, haven't tried it since.
> 
> Linux zaphod 2.6.0-test6 #1 Sat Oct 4 22:35:19 EDT 2003 i686 GNU/Linux
> spotter@zaphod:~$ touch a
> spotter@zaphod:~$ touch b
> spotter@zaphod:~$ ./a.out a b
> error from sendfile: Invalid argument


try it on a non-zero file size, there are special requirements IIRC...

Re: Does sendfile() copy extended attributes?

[Shaya Potter]

On Mon, 2003-12-15 at 00:55, Jeff Garzik wrote:
> Shaya Potter wrote:
> > On Mon, 2003-12-15 at 00:46, Jeff Garzik wrote:
> > 
> >>Shaya Potter wrote:
> >>
> >>>On Sun, 2003-12-14 at 12:39, Joseph D. Wagner wrote:
> >>>
> >>>
> >>>>I want to use sendfile() to completely copy a file so that the copy is an 
> >>>>EXACT, INDISTINGUISHABLE duplicate of the original.
> >>>>
> >>>>Hence, I need to know if sendfile() copies the extended attributes in 
> >>>>addition to the file itself.
> >>>>
> >>>>While I have your attention, does sendfile() also copy the inode attributes 
> >>>>(i.e. secure delete, compressed, etc.)?
> >>>
> >>>
> >>>sendfile acting as a sys_copy() is a 2.4 unsupported behavior.  It
> >>>doesn't work on 2.6.
> >>
> >>
> >>Have you tested this assertion lately?  :)
> >>
> >>AFAIK file-file sendfile(2) was fixed in 2.6.
> > 
> > 
> > didn't work on test6, haven't tried it since.
> > 
> > Linux zaphod 2.6.0-test6 #1 Sat Oct 4 22:35:19 EDT 2003 i686 GNU/Linux
> > spotter@zaphod:~$ touch a
> > spotter@zaphod:~$ touch b
> > spotter@zaphod:~$ ./a.out a b
> > error from sendfile: Invalid argument
> 
> 
> try it on a non-zero file size, there are special requirements IIRC...

no difference

spotter@zaphod:~$ dd if=/dev/zero of=a count=1
1+0 records in
1+0 records out
512 bytes transferred in 0.001311 seconds (390543 bytes/sec)
spotter@zaphod:~$ ls -l a 
-rw-r--r--    1 spotter  spotter       512 Dec 15 00:58 a
spotter@zaphod:~$ ./a.out a b
error from sendfile: Invalid argument
spotter@zaphod:~$ dd if=/dev/zero of=b count=1
1+0 records in
1+0 records out
512 bytes transferred in 0.000256 seconds (1999963 bytes/sec)
spotter@zaphod:~$ ./a.out a b
error from sendfile: Invalid argument

Re: Does sendfile() copy extended attributes?

[Bryan Henderson]

>I want to use sendfile() to completely copy a file so that the copy is an 

>EXACT, INDISTINGUISHABLE duplicate of the original.

Just so you know, that will never be possible using the file subsystem. 
You'd have to go in under that and write your own disk blocks.

Two ways one could distinguish the copy from the original that you can't 
do anything about via filesystem interfaces, by design: inode number, 
ctime.

Bryan Henderson                          IBM Almaden Research Center
San Jose CA                              Filesystems

RE: Does sendfile() copy extended attributes?

[Joseph D. Wagner]

> Two ways one could distinguish the copy from the original that you can't
> do anything about via filesystem interfaces, by design: inode number,
> ctime.

For my purposes, that's close enough.  I just wanted to make sure that everything else -- like owner, group, permissions, extended attributes, etc. -- is the same.

Joseph D. Wagner

Re: Does sendfile() copy extended attributes?

[Jamie Lokier]

Joseph D. Wagner wrote:
> For my purposes, that's close enough.  I just wanted to make sure
> that everything else -- like owner, group, permissions, extended
> attributes, etc. -- is the same.

There's no way to do it without copying all the properties you want
copied explicitly, or calling a program which does that.

-- Jamie

RE: Does sendfile() copy extended attributes?

[Joseph D. Wagner]

> Joseph D. Wagner wrote:
> > For my purposes, that's close enough.  I just wanted to make sure
> > that everything else -- like owner, group, permissions, extended
> > attributes, etc. -- is the same.
> 
> There's no way to do it without copying all the properties you want
> copied explicitly, or calling a program which does that.

Or writing a program which does that, which is exactly what I'm doing.  I'm just trying to make sure all my bases are covered.

Joseph D. Wagner

Re: Does sendfile() copy extended attributes?

[maximilian attems]

[-- Attachment #1: Type: text/plain, Size: 595 bytes --]

On Mon, 15 Dec 2003, Joseph D. Wagner wrote:

> > > For my purposes, that's close enough.  I just wanted to make sure
> > > that everything else -- like owner, group, permissions, extended
> > > attributes, etc. -- is the same.
> > 
> > There's no way to do it without copying all the properties you want
> > copied explicitly, or calling a program which does that.
> 
> Or writing a program which does that, which is exactly what I'm doing.  
> I'm just trying to make sure all my bases are covered.

just out of curiosity:
why not using rsync(1) or cp -a ??


greetings max


[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

RE: Does sendfile() copy extended attributes?

[Joseph D. Wagner]

> just out of curiosity:
> why not using rsync(1) or cp -a ??

Because that violates one of the Immutable Laws of Security -- "If you're running someone else's program, it's not your program anymore."

If my program executes another program like cp/rsync in your example, it would be vulnerable to a privilege elevation when run as root.  A hacker could replace cp/rsync, change the PATH, or a dozen other tricks to get it to execute his version of cp/rsync.

Now if there's a function call that will copy a file -- other than the sendfile function which is what this thread has been all about -- I'm all ears.  However, I am not going to execute another program.

Joseph D. Wagner

Re: Does sendfile() copy extended attributes?

[Jamie Lokier]

Joseph D. Wagner wrote:
> Because that violates one of the Immutable Laws of Security -- "If
> you're running someone else's program, it's not your program anymore."

That is equally the case if you're running someone else's function.

> If my program executes another program like cp/rsync in your
> example, it would be vulnerable to a privilege elevation when run as
> root.  A hacker could replace cp/rsync, change the PATH, or a dozen
> other tricks to get it to execute his version of cp/rsync.

A hacker could also change libc.so and thereby change the copy function.

> Now if there's a function call that will copy a file -- other than
> the sendfile function which is what this thread has been all about --
> I'm all ears.  However, I am not going to execute another program.

You are imagining a black box function which is specified to copy a
file and its attributes.  How can you know that function does not work
by calling an external program?

-- Jamie

RE: Does sendfile() copy extended attributes?

[Joseph D. Wagner]

>> Because that violates one of the Immutable Laws of Security -- "If
>> you're running someone else's program, it's not your program anymore."
 
> That is equally the case if you're running someone else's function.

Sendfile() is a KERNEL function.  If a hacker has compromised the kernel, what's the point of exploiting a PRIVILEGE ELEVATION vulnerability in a program?  You don't get any higher privileges than the kernel itself.

> A hacker could also change libc.so and thereby change the copy function.

Not without ALREADY compromising the root account.  Remember, the vulnerability I'm addressing is PRIVILEGE ELEVATION.  You can't elevate privileges any higher than root.

> You are imagining a black box function which is specified to copy a
> file and its attributes.  How can you know that function does not work
> by calling an external program?

I didn't say it doesn't work.  I just said that executing an external program is too much of a security risk.

Joseph D. Wagner

Re: Does sendfile() copy extended attributes?

[Jamie Lokier]

Joseph D. Wagner wrote:
> >> Because that violates one of the Immutable Laws of Security -- "If
> >> you're running someone else's program, it's not your program anymore."

> Not without ALREADY compromising the root account.  Remember, the
> vulnerability I'm addressing is PRIVILEGE ELEVATION.  You can't
> elevate privileges any higher than root.

Changing /bin/cp also requires a root compromise.

> > You are imagining a black box function which is specified to copy a
> > file and its attributes.  How can you know that function does not work
> > by calling an external program?
> 
> I didn't say it doesn't work.  I just said that executing an
> external program is too much of a security risk.

I think you read what I wrote the wrong way.  Let me rephrase it:
How can you know that function does not call an external program to
perform its action?

There actually are a few functions in the C library which work by
calling external programs - grantpt is one I think - and it's not
mentioned in the manual page (because it's an implementation detail).

Actually I agree with you that calling external programs is a big
risk.  It should be done carefully in security conscious code.
However you are deluded to imagine that calling functions in the C
library is automatically safe from the those risks.  That must be done
carefully as well.

-- Jamie

RE: Does sendfile() copy extended attributes?

[Joseph D. Wagner]

> I think you read what I wrote the wrong way.  Let me rephrase it:
> How can you know that function does not call an external program to
> perform its action?

Again, sendfile() is a KERNEL function.

> There actually are a few functions in the C library which work by
> calling external programs - grantpt is one I think - and it's not
> mentioned in the manual page (because it's an implementation detail).
> 
> Actually I agree with you that calling external programs is a big
> risk.  It should be done carefully in security conscious code.
> However you are deluded to imagine that calling functions in the C
> library is automatically safe from the those risks.  That must be done
> carefully as well.

Thanks for the heads up.  But a library function is at least SAFER -- not totally SAFE but SAFER than an external program -- right?

I view security as a process, not an end result.  Minimizing the risk, not eliminating the risk altogether, is my goal; hence, the question about a library function call being safer than an external program.

Joseph D. Wagner

RE: Does sendfile() copy extended attributes?

[Shaya Potter]

On Sun, 2003-12-21 at 14:31, Joseph D. Wagner wrote:
> > I think you read what I wrote the wrong way.  Let me rephrase it:
> > How can you know that function does not call an external program to
> > perform its action?
> 
> Again, sendfile() is a KERNEL function.

unless you use the syscall macros to do the syscall interrupt yourself,
you are calling the libc wrapper for the system call, not the system
call itself.

Re: Does sendfile() copy extended attributes?

[Jamie Lokier]

Shaya Potter wrote:
> On Sun, 2003-12-21 at 14:31, Joseph D. Wagner wrote:
> > > I think you read what I wrote the wrong way.  Let me rephrase it:
> > > How can you know that function does not call an external program to
> > > perform its action?
> > 
> > Again, sendfile() is a KERNEL function.
> 
> unless you use the syscall macros to do the syscall interrupt yourself,
> you are calling the libc wrapper for the system call, not the system
> call itself.

Right.

And even if you did use the syscall macros, if an attacker has root
it's just as easy for them to intercept the system call as it is to
modify libc or /bin/cp.

That _are_ very good reasons to be wary of calling external programs,
but "because an attacker may have changed the executable" isn't not
the most important.

-- Jamie