Saving data from the serial port

Saving data from the serial port

[Alex Moen]

Hey all,

I am willing to bet that:
  a) this is so simple that I am overlooking it, or
  b) it has been covered a MILLION times already.

If b, please let me know where to find the answer, because I can't find it.
If a, tell me how blockheaded I am, laugh a little, and then tell me the
solution....

I want to take the data from a serial port from our PBX and log it to a
file.  Sounds simple, right?  I can (Winblows) hyperterm to the port and
gather the data.  I can connect the port to a serial printer and get the
data.  But, when I connect my linux box to the serial port, I am only
getting PART of the data.  For instance, I am only getting the first 14-16
characters of each line, with really no line breaks.  I also notice with
statserial that I am getting an overrun (wth is that???).  I have the port
settings in the software set at what Windows and the printer are set at
(9600-8-N-1).

I have read through the serial howto, tried a few programs that are
suggested (two that look promising but are giving the above results are
linbar and logserial), and am stumped.

I'd hate to have to set up a Win98 box running hyperterm logging to a file,
and then manually rotate that each day....


Thanks, and don't laugh too hard...

Alex

RE: Saving data from the serial port

[Ed Vance]

On Thu, Apr 17, 2003 at 11:44 AM, Alex Moen wrote:
> 
> I want to take the data from a serial port from our PBX and 
> log it to a file.  Sounds simple, right?  I can (Winblows) 
> hyperterm to the port and gather the data.  I can connect 
> the port to a serial printer and get the data.  But, when I 
> connect my linux box to the serial port, I am only getting 
> PART of the data.  For instance, I am only getting the 
> first 14-16 characters of each line, with really no line 
> breaks.  I also notice with statserial that I am getting an 
> overrun (wth is that???).  I have the port settings in the 
> software set at what Windows and the printer are set at
> (9600-8-N-1).
> 
> I have read through the serial howto, tried a few programs 
> that are suggested (two that look promising but are giving 
> the above results are linbar and logserial), and am stumped.
> 
> I'd hate to have to set up a Win98 box running hyperterm 
> logging to a file, and then manually rotate that each day....
> 
Overruns happen when the UART receiver FIFO buffer is completely
full when the next date character is received. The byte that did
not fit is discarded and the overrun status is posted. 

the root cause is either that the data is not being read fast 
enough by the program, or system interrupt latency is too long
to let the driver service the UART before an overrun occurs. 

If the data source supports XON/XOFF or hardware flow control, 
you could set that to hold off the data source when the system 
is not ready to receive more data. 

Does your UART have a FIFO buffer? 
What UART type is your port?
How fast is your system?
 
Cheers,
Ed

Re: Saving data from the serial port

[David Lawyer]

On Thu, Apr 17, 2003 at 12:10:39PM -0700, Ed Vance wrote:
> On Thu, Apr 17, 2003 at 11:44 AM, Alex Moen wrote:
> > 
> > I want to take the data from a serial port from our PBX and 
> > log it to a file.  Sounds simple, right?  I can (Winblows) 
> > hyperterm to the port and gather the data.  I can connect 
> > the port to a serial printer and get the data.  But, when I 
> > connect my linux box to the serial port, I am only getting 
> > PART of the data.  For instance, I am only getting the 
> > first 14-16 characters of each line, with really no line 
> > breaks.  I also notice with statserial that I am getting an 
> > overrun (wth is that???).  I have the port settings in the 
> > software set at what Windows and the printer are set at
> > (9600-8-N-1).
> > 
> > I have read through the serial howto, tried a few programs 
> > that are suggested (two that look promising but are giving 
> > the above results are linbar and logserial), and am stumped.
> > 
> > I'd hate to have to set up a Win98 box running hyperterm 
> > logging to a file, and then manually rotate that each day....
> > 
> Overruns happen when the UART receiver FIFO buffer is completely
> full when the next date character is received. The byte that did
> not fit is discarded and the overrun status is posted. 
> 
> the root cause is either that the data is not being read fast 
> enough by the program, or system interrupt latency is too long
> to let the driver service the UART before an overrun occurs. 

Or if the interrupt is misset.  This is what may be happening.  If the
interrupt is wrong, there are no interrupts but a sort of slow polling
takes place and fetches the contents of the FIFO.  Don't confuse this
with the fast polling you get if you set the IRQ to 0.  With the slow
polling there's a delay before the next poll and data is often lost due
to overruns of the FIFO.  The slow polling wasn't intended to be polling
but that's what it amounts to.  So check that your interrupt is working
OK by setting the IRQ to 0 with setserial to enable fast polling.  If
that seemingly fixes it, you had an interrupt problem.

> If the data source supports XON/XOFF or hardware flow control, 
> you could set that to hold off the data source when the system 
> is not ready to receive more data. 

I don't think so (but of course you should use flow control).  There
just isn't any way to use flow control to protect the FIFO buffers from
overruns in Linux.  Flow control (for the input flow into a PC) only
protects the serial 8K buffer in main memory.

			David Lawyer

RE: Saving data from the serial port

[Alex Moen]

Yep.  The BIOS of the computer had the serial ports set to "auto" rather
than an interrupt.  I set them, and everything works properly now.

Thanks all!!!

Alex

-----Original Message-----
From: David Lawyer [mailto:dave@lafn.org] 
Sent: Thursday, April 17, 2003 6:06 PM
To: Ed Vance
Cc: 'alexm@ndtel.com'; linux-serial@vger.kernel.org
Subject: Re: Saving data from the serial port


On Thu, Apr 17, 2003 at 12:10:39PM -0700, Ed Vance wrote:
> On Thu, Apr 17, 2003 at 11:44 AM, Alex Moen wrote:
> > 
> > I want to take the data from a serial port from our PBX and
> > log it to a file.  Sounds simple, right?  I can (Winblows) 
> > hyperterm to the port and gather the data.  I can connect 
> > the port to a serial printer and get the data.  But, when I 
> > connect my linux box to the serial port, I am only getting 
> > PART of the data.  For instance, I am only getting the 
> > first 14-16 characters of each line, with really no line 
> > breaks.  I also notice with statserial that I am getting an 
> > overrun (wth is that???).  I have the port settings in the 
> > software set at what Windows and the printer are set at
> > (9600-8-N-1).
> > 
> > I have read through the serial howto, tried a few programs
> > that are suggested (two that look promising but are giving 
> > the above results are linbar and logserial), and am stumped.
> > 
> > I'd hate to have to set up a Win98 box running hyperterm
> > logging to a file, and then manually rotate that each day....
> > 
> Overruns happen when the UART receiver FIFO buffer is completely full 
> when the next date character is received. The byte that did not fit is 
> discarded and the overrun status is posted.
> 
> the root cause is either that the data is not being read fast
> enough by the program, or system interrupt latency is too long
> to let the driver service the UART before an overrun occurs. 

Or if the interrupt is misset.  This is what may be happening.  If the
interrupt is wrong, there are no interrupts but a sort of slow polling takes
place and fetches the contents of the FIFO.  Don't confuse this with the
fast polling you get if you set the IRQ to 0.  With the slow polling there's
a delay before the next poll and data is often lost due to overruns of the
FIFO.  The slow polling wasn't intended to be polling but that's what it
amounts to.  So check that your interrupt is working OK by setting the IRQ
to 0 with setserial to enable fast polling.  If that seemingly fixes it, you
had an interrupt problem.

> If the data source supports XON/XOFF or hardware flow control,
> you could set that to hold off the data source when the system 
> is not ready to receive more data. 

I don't think so (but of course you should use flow control).  There just
isn't any way to use flow control to protect the FIFO buffers from overruns
in Linux.  Flow control (for the input flow into a PC) only protects the
serial 8K buffer in main memory.

			David Lawyer

Re: Saving data from the serial port

[Stuart MacDonald]

From: "David Lawyer" <dave@lafn.org>
> I don't think so (but of course you should use flow control).  There
> just isn't any way to use flow control to protect the FIFO buffers from
> overruns in Linux.  Flow control (for the input flow into a PC) only
> protects the serial 8K buffer in main memory.

Not true. Many modern uarts have on-board flow control both hardware
and software based. Turn it on and the hardware does the magic when it
finds its fifos getting full.

The caveat is that the remote side needs to support the same method of
flow control obviously.

..Stu

--
We make multiport serial boards.
<http://www.connecttech.com>
(800) 426-8979

Re: Saving data from the serial port

[David Lawyer]

On Fri, Apr 18, 2003 at 06:02:19PM -0400, Stuart MacDonald wrote:
> From: "David Lawyer" <dave@lafn.org>
> > I don't think so (but of course you should use flow control).  There
> > just isn't any way to use flow control to protect the FIFO buffers from
> > overruns in Linux.  Flow control (for the input flow into a PC) only
> > protects the serial 8K buffer in main memory.
> 
> Not true. Many modern uarts have on-board flow control both hardware
> and software based. Turn it on and the hardware does the magic when it
> finds its fifos getting full.

Does the Linux serial driver support this?  I didn't think that it did.
			David Lawyer

> The caveat is that the remote side needs to support the same method of
> flow control obviously.

Re: Saving data from the serial port

[Stuart MacDonald]

From: "David Lawyer" <dave@lafn.org>
> On Fri, Apr 18, 2003 at 06:02:19PM -0400, Stuart MacDonald wrote:
> > From: "David Lawyer" <dave@lafn.org>
> > > I don't think so (but of course you should use flow control).  There
> > > just isn't any way to use flow control to protect the FIFO buffers
from
> > > overruns in Linux.  Flow control (for the input flow into a PC) only
> > > protects the serial 8K buffer in main memory.
> >
> > Not true. Many modern uarts have on-board flow control both hardware
> > and software based. Turn it on and the hardware does the magic when it
> > finds its fifos getting full.
>
> Does the Linux serial driver support this?  I didn't think that it did.
> David Lawyer

linux-2.4.20/drivers/char/serial.c:1774

Although support seems to be only for auto-cts. I coulda sworn it was
for both. The hardware based soft-flow control isn't being used
though.

..Stu