public inbox for linux-kernel@vger.kernel.org
 help / color / mirror / Atom feed
* splice/tee bugs?
@ 2006-07-07  7:07 Michael Kerrisk
  2006-07-07 11:07 ` Andrew Morton
  0 siblings, 1 reply; 34+ messages in thread
From: Michael Kerrisk @ 2006-07-07  7:07 UTC (permalink / raw)
  To: axboe; +Cc: linux-kernel, michael.kerrisk

Hello Jens,

While editing and extending your draft man pages for
tee(), splice(), vmsplice() I've been testing out 
the splice()/tee() calls using a modified version of 
the program you provided in the tee.2 manual page.

The most notable differences between my program and yours
are:

* I print some debugging info to stderr.

* I don't pass SPLICE_F_NONBLOCK to tee().

I'm running this on kernel 2.6.17, using the following 
command line:

$ ls *.c  | ktee r  | wc

On different runs I see:

a) No output from ls through the pipeline:

tee returned 0
      0       0       0


b) Very many instances of EAGAIN followed by expected results:

...
EAGAIN
EAGAIN
EAGAIN
EAGAIN
EAGAIN
EAGAIN
tee returned 19
splice returned 19
tee returned 0
      2       2      19

In some of these cases the elapsed time to run the command-line 
is 1 or 2 seconds in this case (instead of the more typical 
0.05 seconds).


c) Occasionally the command line just hangs, producing no output.
   In this case I can't kill it with ^C or ^\.  This is a 
   hard-to-reproduce behaviour on my (x86) system, but I have 
   seen it several times by now.

Assuming I'm not messing up with my test method, some 
observations:

Result a) seems to be occurring because tee() returns 0 if its
in_fd is not yet "ready" to deliver data.  Shouldn't tee() 
be blocking in this case?  And should not 0 only 
be returned for EOF? on the input file descriptor?

If I uncomment the usleep() line in the program, this behavior 
does not occur--the program always just produces the expected
output:

tee returned 19
splice returned 19
tee returned 0
      2       2      19

For behaviour b) -- why does tee() give EAGAIN when
I haven't specified SPLICE_F_NONBLOCK?  (This is a 
philosophical question; I can see that there are code paths
that lead to EAGAIN without SPLICE_F_NONBLOCK, but that
seems confusing behaviour for userland.)

Behaviour c) hints of a bug in tee().

Your thoughts?

Cheers,

Michael

====

/* ktee.c */

#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <assert.h>
#include <errno.h>
#include <limits.h>

#if defined(__i386__)
#define __NR_splice     313
#define __NR_tee        315
#else
#error unsupported arch
#endif

#define SPLICE_F_MOVE   (0x01)  /* move pages instead of copying */
#define SPLICE_F_NONBLOCK (0x02) /* don't block on the pipe splicing (but */
                                 /* we may still block on the fd we splice */
                                 /* from/to, of course */
#define SPLICE_F_MORE   (0x04)  /* expect more data */
#define SPLICE_F_GIFT   (0x08)  /* pages passed in are a gift */


static inline int splice(int fdin, loff_t *off_in, int fdout,
                         loff_t *off_out, size_t len, unsigned int flags)
{
    return syscall(__NR_splice, fdin, off_in, fdout, off_out, len, flags);
}

static inline int tee(int fdin, int fdout, size_t len, unsigned int flags)
{
    return syscall(__NR_tee, fdin, fdout, len, flags);
}

int
main(int argc, char *argv[])
{
    int fd;
    int len, slen;

    assert(argc == 2);

    fd = open(argv[1], O_WRONLY | O_CREAT | O_TRUNC, 0644);
    if (fd == -1) {
        perror("open");
        exit(EXIT_FAILURE);
    }

    //usleep(100000);
    do {
        /*
         * tee stdin to stdout.
         */
        len = tee(STDIN_FILENO, STDOUT_FILENO,
                  INT_MAX, 0);

        if (len < 0) {
            if (errno == EAGAIN) {
                fprintf(stderr, "EAGAIN\n");
                continue;
            }
            perror("tee");
            exit(EXIT_FAILURE);
        }
        fprintf(stderr, "tee returned %ld\n",  (long) len);
        if (len == 0)
            break;

        /*
         * Consume stdin by splicing it to a file.
         */
        while (len > 0) {
            slen = splice(STDIN_FILENO, NULL, fd, NULL,
                          len, SPLICE_F_MOVE);
            if (slen < 0) {
                perror("splice");
                break;
            }
            fprintf(stderr, "splice returned %ld\n", (long) slen);
            len -= slen;
        }
    } while (1);

    close(fd);
    exit(EXIT_SUCCESS);
}

-- 
Michael Kerrisk
maintainer of Linux man pages Sections 2, 3, 4, 5, and 7 

Want to help with man page maintenance?  
Grab the latest tarball at
ftp://ftp.win.tue.nl/pub/linux-local/manpages/, 
read the HOWTOHELP file and grep the source 
files for 'FIXME'.

^ permalink raw reply	[flat|nested] 34+ messages in thread
* Re: splice/tee bugs?
@ 2006-07-08  5:33 Chuck Ebbert
  0 siblings, 0 replies; 34+ messages in thread
From: Chuck Ebbert @ 2006-07-08  5:33 UTC (permalink / raw)
  To: Luiz Fernando N. Capitulino
  Cc: vendor-sec, Michael Kerrisk, linux-kernel, Jens Axboe,
	Andrew Morton

In-Reply-To: <20060707131310.0e382585@doriath.conectiva>

On Fri, 7 Jul 2006 13:13:10 -0300, Luiz Fernando N. Capitulino wrote:

> | > c) Occasionally the command line just hangs, producing no output.
> | >    In this case I can't kill it with ^C or ^\.  This is a 
> | >    hard-to-reproduce behaviour on my (x86) system, but I have 
> | >    seen it several times by now.
> | 
> | aka local DoS.  Please capture sysrq-T output next time.
> 
>  If I run lots of them in parallel, I get the following OOPs in a few
> seconds:
> 
> BUG: unable to handle kernel NULL pointer dereference at virtual address 00000018
>  printing eip:
> c01790c7
> *pde = 00000000
> Oops: 0000 [#1]
> Modules linked in: ipv6 capability commoncap snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq via_rhine mii snd_pcm_oss snd_mixer_oss af_packet snd_via82xx gameport snd_ac97_codec snd_ac97_bus snd_pcm snd_timer snd_page_alloc snd_mpu401_uart sn
> CPU:    0
> EIP:    0060:[sys_tee+371/924]    Not tainted VLI
> EIP:    0060:[<c01790c7>]    Not tainted VLI
> EFLAGS: 00010293   (2.6.18-rc1 #8) 
> EIP is at sys_tee+0x173/0x39c
> eax: d62bfa00   ebx: 00000000   ecx: 00000000   edx: d62bfa98
> esi: d7434800   edi: d62bfa98   ebp: d5d5cfb4   esp: d5d5cf84
> ds: 007b   es: 007b   ss: 0068
> Process ktee (pid: 12605, ti=d5d5c000 task=d9cce0b0 task.ti=d5d5c000)
> Stack: d5eede40 00000000 d827ac00 00000002 00000000 d62bfa00 00000000 00000000 
>        00000000 00000000 00000000 b7f72920 d5d5c000 c0102b7d 00000000 00000001 
>        7fffffff 00000000 b7f72920 bf8f37b8 0000013b 0000007b 0000007b 0000013b 
> Call Trace:
>  [<c010422c>] show_stack_log_lvl+0x8c/0x97
>  [<c0104397>] show_registers+0x124/0x191
>  [<c0104550>] die+0x14c/0x269
>  [<c02a6521>] do_page_fault+0x443/0x51e
>  [<c0103d49>] error_code+0x39/0x40
>  [<c0102b7d>] sysenter_past_esp+0x56/0x79
> Code: 00 00 00 89 d0 8b 55 e4 03 42 6c 83 e0 0f 6b c0 14 8d 7c 10 70 8b 46 68 89 45 e0 83 f8 0f 77 5c 8b 4f 0c 8b 5e 6c 89 fa 8b 45 e4 <ff> 51 18 03 5d e0 83 e3 0f 89 fa 6b db 14 b9 14 00 00 00 8d 5c 


ibuf->ops is NULL in the below code (fs/splice.c line 1355 in 2.6.18-rc1)


static int link_pipe(struct pipe_inode_info *ipipe,
                     struct pipe_inode_info *opipe,
                     size_t len, unsigned int flags)
{
        struct pipe_buffer *ibuf, *obuf;
        int ret, do_wakeup, i, ipipe_first;

        ret = do_wakeup = ipipe_first = 0;

        /*
         * Potential ABBA deadlock, work around it by ordering lock
         * grabbing by inode address. Otherwise two different processes
         * could deadlock (one doing tee from A -> B, the other from B -> A).
         */
        if (ipipe->inode < opipe->inode) {
                ipipe_first = 1;
                mutex_lock(&ipipe->inode->i_mutex);
                mutex_lock(&opipe->inode->i_mutex);
        } else {
                mutex_lock(&opipe->inode->i_mutex);
                mutex_lock(&ipipe->inode->i_mutex);
        }

        for (i = 0;; i++) {
                if (!opipe->readers) {
                        send_sig(SIGPIPE, current, 0);
                        if (!ret)
                                ret = -EPIPE;
                        break;
                }
                if (ipipe->nrbufs - i) {
                        ibuf = ipipe->bufs + ((ipipe->curbuf + i) & (PIPE_BUFFERS - 1));

                        /*
                         * If we have room, fill this buffer
                         */
                        if (opipe->nrbufs < PIPE_BUFFERS) {
                                int nbuf = (opipe->curbuf + opipe->nrbufs) & (PIPE_BUFFERS - 1);

                                /*
                                 * Get a reference to this pipe buffer,
                                 * so we can copy the contents over.
                                 */
========>                       ibuf->ops->get(ipipe, ibuf);

                                obuf = opipe->bufs + nbuf;
                                *obuf = *ibuf;

-- 
Chuck
 "You can't read a newspaper if you can't read."  --George W. Bush

^ permalink raw reply	[flat|nested] 34+ messages in thread
end of thread, other threads:[~2006-07-10  9:08 UTC | newest]

Thread overview: 34+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2006-07-07  7:07 splice/tee bugs? Michael Kerrisk
2006-07-07 11:07 ` Andrew Morton
2006-07-07 11:42   ` Michael Kerrisk
2006-07-07 12:03     ` Jens Axboe
2006-07-07 12:28       ` Jens Axboe
2006-07-07 12:31         ` Michael Kerrisk
2006-07-07 12:41           ` Jens Axboe
2006-07-07 13:12             ` Jens Axboe
2006-07-07 13:14               ` Jens Axboe
2006-07-07 13:21                 ` Arjan van de Ven
2006-07-07 13:26                   ` Jens Axboe
2006-07-07 13:54                     ` Paulo Marques
2006-07-07 14:02                       ` Jens Axboe
2006-07-07 14:05               ` Michael Kerrisk
2006-07-07 14:08                 ` Jens Axboe
2006-07-07 16:13   ` Luiz Fernando N. Capitulino
2006-07-07 21:43     ` Luiz Fernando N. Capitulino
2006-07-08  6:41     ` Jens Axboe
2006-07-08 21:09       ` Luiz Fernando N. Capitulino
2006-07-09 10:36         ` Jens Axboe
2006-07-09 11:16           ` Jens Axboe
2006-07-09 16:47             ` Luiz Fernando N. Capitulino
2006-07-09 17:57               ` Jens Axboe
2006-07-10  6:25                 ` Michael Kerrisk
2006-07-10  6:43                   ` Jens Axboe
2006-07-10  8:09                     ` Michael Kerrisk
2006-07-10  8:24                       ` Jens Axboe
2006-07-10  8:40                         ` Michael Kerrisk
2006-07-10  8:46                           ` Jens Axboe
2006-07-10  8:50                             ` Michael Kerrisk
2006-07-10  9:06                               ` Jens Axboe
2006-07-10  9:08                                 ` Michael Kerrisk
2006-07-10  8:50                             ` Jens Axboe
  -- strict thread matches above, loose matches on Subject: below --
2006-07-08  5:33 Chuck Ebbert

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox