Problem with read() and named pipe (FIFO)

Problem with read() and named pipe (FIFO)

[Hossein Mobahi]

Hello

There is an executable file, say PRG1, that measures
some quantities by sensors and writes the output to
stdout. I want to grab this output by my own program
PRG2, do some process on it and dump the result on the
screen. Since the whole system is real time, prg1 and
prg2 must run in parallel. The simplest answer could
be "prg1 | prg2", but for some reasons I have to use a
named pipe (FIFO), like "./prg1 > /tmp/mypipe &" and
"./prg2", where mypipe is a named pipe and prg2 reads
it by open() and read().

The problem is that prg1 does not produce any return
(I mean '\n'), and numbers are separated by space.
Therefore, read(fd,n,buf) blocks even if "n" is
smaller than what prg1 has actually written to the
pipe (no matter what n is, even if the pipe's buffer
contains more characters than n, read first waits for
return and onces gets it, it will copy n bytes of the
buffer to buf). On the other hand opening in
non-blocking fashion doesn't help, because I want my
program to wait for the output of prg1. Is there any
way to make read() return once the pipe has more
characters than n? What about defining another
delimiter such as "blank", instead of "return"? Let me
add that I do not want to touch kernel sources, I am
looking for a more portable solution using available
system calls or shell's capabilities.

Thnx

--Hossein



		
__________________________________
Do you Yahoo!?
New and Improved Yahoo! Mail - Send 10MB messages!
http://promotions.yahoo.com/new_mail 

Re: Problem with read() and named pipe (FIFO)

[Henry Margies]

Hi


Have you tried fflush or fsync or something like this?


Henry


-- 

Hi! I'm a .signature virus! Copy me into your
~/.signature to help me spread!

Re: Problem with read() and named pipe (FIFO)

[joy]

Hossein Mobahi wrote:

>
>The problem is that prg1 does not produce any return
>(I mean '\n'), and numbers are separated by space.
>Therefore, read(fd,n,buf) blocks even if "n" is
>smaller than what prg1 has actually written to the
>pipe (no matter what n is, even if the pipe's buffer
>contains more characters than n, read first waits for
>return and onces gets it, it will copy n bytes of the
>buffer to buf). On the other hand opening in
>  
>
Funny. I have used read() to get 2 characters out of a binary file with 
no "\n"'s
and it never behaved like this. Also the man page make sno mention of 
this either.
Did you try this out? How about fread/fwrite?

Joy.M.Monteiro

>non-blocking fashion doesn't help, because I want my
>program to wait for the output of prg1. Is there any
>way to make read() return once the pipe has more
>characters than n? What about defining another
>delimiter such as "blank", instead of "return"? Let me
>add that I do not want to touch kernel sources, I am
>looking for a more portable solution using available
>system calls or shell's capabilities.
>
>Thnx
>
>--Hossein
>
>
>  
>

Re: Problem with read() and named pipe (FIFO) (pipe not disk)

[Hossein Mobahi]

Hello

The problem is not "\n", but buffered I/O used by
standard IO library. I head that sometimes "\n"
flushes the buffer, but it did not (as I am
experiencing now).

You did not have any problem, because you were
directly reading from a file. If you were reading from
STDIN_FILENO, you would not get your characters when
using "./prg1 | ./prg2" where prg1 is:
main() { printf("TEST\n"); for (;;); }

and prg2 is:
main() { read(STDIN_FILENO, buf, 1000); printf
("%s\n",buf) ; }

The solution that I know is using
write(STDOUT_FILENO,....) instead of printf because it
does not buffer. However, I do not have access to the
source code of prg1, and I must somehow make prg1
flush its standard I/O buffer from outside.

--Hossein

--- joy <gracecott@sancharnet.in> wrote:

> Funny. I have used read() to get 2 characters out of
> a binary file with 
> no "\n"'s
> and it never behaved like this. Also the man page
> make sno mention of 
> this either.
> Did you try this out? How about fread/fwrite?



		
__________________________________
Do you Yahoo!?
Read only the mail you want - Yahoo! Mail SpamGuard.
http://promotions.yahoo.com/new_mail 

Re: Problem with read() and named pipe (FIFO) (fflush or fsync)

[Hossein Mobahi]

Hello

My program, prg2, is the reader, not writer. So making
the reader flush does not make sense. If I could make
prg1 flush (its writing buffer), it would help, but I
do not have access to the source code of prg1, it is
just an executable file. 

I was wondering if we can somehow force an executable
program not to use buffered output without access to
its source code? The program originally uses printf
for writing its output.

--Hossein

--- Henry Margies <henry.margies@gmx.de> wrote:

> Hi
> 
> 
> Have you tried fflush or fsync or something like
> this?
> 
> 
> Henry



		
__________________________________
Do you Yahoo!?
Yahoo! Mail Address AutoComplete - You start. We finish.
http://promotions.yahoo.com/new_mail 

Re: Problem with read() and named pipe (FIFO) (pipe not disk)

[joy]

Hossein Mobahi wrote:

>Hello
>
>The problem is not "\n", but buffered I/O used by
>standard IO library. I head that sometimes "\n"
>flushes the buffer, but it did not (as I am
>experiencing now).
>
>You did not have any problem, because you were
>directly reading from a file. If you were reading from
>STDIN_FILENO, you would not get your characters when
>using "./prg1 | ./prg2" where prg1 is:
>main() { printf("TEST\n"); for (;;); }
>
>and prg2 is:
>main() { read(STDIN_FILENO, buf, 1000); printf
>("%s\n",buf) ; }
>
>The solution that I know is using
>write(STDOUT_FILENO,....) instead of printf because it
>does not buffer. However, I do not have access to the
>source code of prg1, and I must somehow make prg1
>flush its standard I/O buffer from outside.
>
>  
>
The program cannot buffer data for all eternity and must eventually 
write it to the
pipe. So, even if read() blocks because  there is no data available, it 
must return as soon as data
is written. The real-time element will be lost though I doubt it will 
buffer for a long time.
Is this the behavior you are experiencing and is there a problem with this ?

Joy.M.Monteiro

Re: Problem with read() and named pipe (FIFO) (pipe not disk)

[Hossein Mobahi]

Sure, the stdio's buffer is limited. However, as you
said, the real-time constraint doesn't let me wait for
the buffer to become full. I need to read it as soon
as the data is written to the buffer. So I really need
to flush stdio's buffer out of that process.

--Hossein

--- joy <gracecott@sancharnet.in> wrote:
> The program cannot buffer data for all eternity and
> must eventually 
> write it to the
> pipe. So, even if read() blocks because  there is no
> data available, it 
> must return as soon as data
> is written. The real-time element will be lost
> though I doubt it will 
> buffer for a long time.
> Is this the behavior you are experiencing and is
> there a problem with this ?
>
> Hossein Mobahi wrote:
> >The solution that I know is using
> >write(STDOUT_FILENO,....) instead of printf because
> it
> >does not buffer. However, I do not have access to
> the
> >source code of prg1, and I must somehow make prg1
> >flush its standard I/O buffer from outside.



		
__________________________________
Do you Yahoo!?
New and Improved Yahoo! Mail - Send 10MB messages!
http://promotions.yahoo.com/new_mail 

Re: Problem with read() and named pipe (FIFO) (pipe not disk)

[Hossein Mobahi]

Sure, the stdio's buffer is limited. However, as you
said, the real-time constraint doesn't let me wait for
the buffer to become full. I need to read it as soon
as the data is written to the buffer. So I really need
to flush stdio's buffer out of that process.

--Hossein

--- joy <gracecott@sancharnet.in> wrote:
> The program cannot buffer data for all eternity and
> must eventually 
> write it to the
> pipe. So, even if read() blocks because  there is no
> data available, it 
> must return as soon as data
> is written. The real-time element will be lost
> though I doubt it will 
> buffer for a long time.
> Is this the behavior you are experiencing and is
> there a problem with this ?
>
> Hossein Mobahi wrote:
> >The solution that I know is using
> >write(STDOUT_FILENO,....) instead of printf because
> it
> >does not buffer. However, I do not have access to
> the
> >source code of prg1, and I must somehow make prg1
> >flush its standard I/O buffer from outside.



		
__________________________________
Do you Yahoo!?
Yahoo! Mail - 50x more storage than other providers!
http://promotions.yahoo.com/new_mail

Re: Problem with read() and named pipe (FIFO) (pipe not disk)

[Hossein Mobahi]

Sure, the stdio's buffer is limited. However, as you
said, the real-time constraint doesn't let me wait for
the buffer to become full. I need to read it as soon
as the data is written to the buffer. So I really need
to flush stdio's buffer out of that process.

--Hossein

--- joy <gracecott@sancharnet.in> wrote:
> The program cannot buffer data for all eternity and
> must eventually 
> write it to the
> pipe. So, even if read() blocks because  there is no
> data available, it 
> must return as soon as data
> is written. The real-time element will be lost
> though I doubt it will 
> buffer for a long time.
> Is this the behavior you are experiencing and is
> there a problem with this ?
>
> Hossein Mobahi wrote:
> >The solution that I know is using
> >write(STDOUT_FILENO,....) instead of printf because
> it
> >does not buffer. However, I do not have access to
> the
> >source code of prg1, and I must somehow make prg1
> >flush its standard I/O buffer from outside.



		
__________________________________
Do you Yahoo!?
Yahoo! Mail - 50x more storage than other providers!
http://promotions.yahoo.com/new_mail