Real time USB2Serial devices and behaivor

Real time USB2Serial devices and behaivor

[mark gross]

At home I'm playing with a real time amateur robotics application with a
micro controller talking over its serial port to a Linux box running an RT
kernel and some RT user mode software to suck in the serial port data
and do some processing in real time.  One example of what I want to do
with this type of configuration is to create a type of IMU thing and run
the data through a kalman filter to see how good I can do with my
hardware.

I'll be sending data regularly at data rates between 50 and 500 HZ from
the MCU (depending on mode)

Anyway, I have the RT code and the system prototyped using the hardware
serial port on my desktop and I'm getting rock solid RT delivery times
of the data +/- 30usec.  While stressing the system in CPU and disk. (I
still need to do the test while flood pinging the box)  However; when I
use my USB dongle instead of the hardware serial port the system falls
over WRT real time behavior.

I'm just starting to look into the behavior now but has anyone looked at
the RT'ness of USB2Serial + USB stack yet?  I would like to see what
other folks have done.  My googleing last night didn't get any useful
hits.

BTW Why does the PL2303 have a 1KB buffer for incoming data?

I'm a bit green at RT work at this level.  What are the best ways for
drilling down to figure out what the problems are so I can make a patch?

Thanks for any advice or pointers.  

--mgross

Re: Real time USB2Serial devices and behaivor

[Greg KH]

On Wed, Mar 26, 2008 at 08:25:59AM -0700, mark gross wrote:
> At home I'm playing with a real time amateur robotics application with a
> micro controller talking over its serial port to a Linux box running an RT
> kernel and some RT user mode software to suck in the serial port data
> and do some processing in real time.  One example of what I want to do
> with this type of configuration is to create a type of IMU thing and run
> the data through a kalman filter to see how good I can do with my
> hardware.
> 
> I'll be sending data regularly at data rates between 50 and 500 HZ from
> the MCU (depending on mode)
> 
> Anyway, I have the RT code and the system prototyped using the hardware
> serial port on my desktop and I'm getting rock solid RT delivery times
> of the data +/- 30usec.  While stressing the system in CPU and disk. (I
> still need to do the test while flood pinging the box)  However; when I
> use my USB dongle instead of the hardware serial port the system falls
> over WRT real time behavior.
> 
> I'm just starting to look into the behavior now but has anyone looked at
> the RT'ness of USB2Serial + USB stack yet?  I would like to see what
> other folks have done.  My googleing last night didn't get any useful
> hits.

I have, and others have in the past, and it's something that you can not
really do.  USB is not "deterministic", and these cheap usb to serial
devices introduce a very big lag that also is not deterministic.

I wouldn't recommend using USB for anything that you need to be able to
accuratly control the latency of, it's just not going to work as that is
not what it was designed for at all.

> BTW Why does the PL2303 have a 1KB buffer for incoming data?

The device or the driver?

> I'm a bit green at RT work at this level.  What are the best ways for
> drilling down to figure out what the problems are so I can make a patch?

Give up and use a "real" serial port, you'll have better luck :)

thanks,

greg k-h

Re: Real time USB2Serial devices and behaivor

[mark gross]

On Wed, Mar 26, 2008 at 09:27:28AM -0700, Greg KH wrote:
> On Wed, Mar 26, 2008 at 08:25:59AM -0700, mark gross wrote:
> > At home I'm playing with a real time amateur robotics application with a
> > micro controller talking over its serial port to a Linux box running an RT
> > kernel and some RT user mode software to suck in the serial port data
> > and do some processing in real time.  One example of what I want to do
> > with this type of configuration is to create a type of IMU thing and run
> > the data through a kalman filter to see how good I can do with my
> > hardware.
> > 
> > I'll be sending data regularly at data rates between 50 and 500 HZ from
> > the MCU (depending on mode)
> > 
> > Anyway, I have the RT code and the system prototyped using the hardware
> > serial port on my desktop and I'm getting rock solid RT delivery times
> > of the data +/- 30usec.  While stressing the system in CPU and disk. (I
> > still need to do the test while flood pinging the box)  However; when I
> > use my USB dongle instead of the hardware serial port the system falls
> > over WRT real time behavior.
> > 
> > I'm just starting to look into the behavior now but has anyone looked at
> > the RT'ness of USB2Serial + USB stack yet?  I would like to see what
> > other folks have done.  My googleing last night didn't get any useful
> > hits.
> 
> I have, and others have in the past, and it's something that you can not
> really do.  USB is not "deterministic", and these cheap usb to serial
> devices introduce a very big lag that also is not deterministic.
> 

I'm seeing some of that today :(

> I wouldn't recommend using USB for anything that you need to be able to
> accuratly control the latency of, it's just not going to work as that is
> not what it was designed for at all.

Oh, thats a drag.  I know bulk has retries, and that on a clean bus they
don't happen (well as far as my hobby use they shouldn't, real
applications should use isochronous streams to avoid them.)

What is the root cause for the non deterministic behavior?  HW or USB
stack?

Is there any reason to think that if I created my own isochronous
USB2Serial adapter and iso-usb-driver that I couldn't get determinism?

There are a few hackable devices I have (USB-Bitwacker and the LPC2148)
I could implement isoc-endpoint firmware and driver for to make a
ISO-USB2Serial device.  But I don't want to go down that path if its got
no chance of success.


> 
> > BTW Why does the PL2303 have a 1KB buffer for incoming data?
> 
> The device or the driver?

driver.

> 
> > I'm a bit green at RT work at this level.  What are the best ways for
> > drilling down to figure out what the problems are so I can make a patch?
> 
> Give up and use a "real" serial port, you'll have better luck :)

but its getting very hard to find real serial ports any more :(


Thanks,

--mgross

Re: Real time USB2Serial devices and behaivor

[Greg KH]

On Wed, Mar 26, 2008 at 09:49:46AM -0700, mark gross wrote:
> On Wed, Mar 26, 2008 at 09:27:28AM -0700, Greg KH wrote:
> > On Wed, Mar 26, 2008 at 08:25:59AM -0700, mark gross wrote:
> > > At home I'm playing with a real time amateur robotics application with a
> > > micro controller talking over its serial port to a Linux box running an RT
> > > kernel and some RT user mode software to suck in the serial port data
> > > and do some processing in real time.  One example of what I want to do
> > > with this type of configuration is to create a type of IMU thing and run
> > > the data through a kalman filter to see how good I can do with my
> > > hardware.
> > > 
> > > I'll be sending data regularly at data rates between 50 and 500 HZ from
> > > the MCU (depending on mode)
> > > 
> > > Anyway, I have the RT code and the system prototyped using the hardware
> > > serial port on my desktop and I'm getting rock solid RT delivery times
> > > of the data +/- 30usec.  While stressing the system in CPU and disk. (I
> > > still need to do the test while flood pinging the box)  However; when I
> > > use my USB dongle instead of the hardware serial port the system falls
> > > over WRT real time behavior.
> > > 
> > > I'm just starting to look into the behavior now but has anyone looked at
> > > the RT'ness of USB2Serial + USB stack yet?  I would like to see what
> > > other folks have done.  My googleing last night didn't get any useful
> > > hits.
> > 
> > I have, and others have in the past, and it's something that you can not
> > really do.  USB is not "deterministic", and these cheap usb to serial
> > devices introduce a very big lag that also is not deterministic.
> > 
> 
> I'm seeing some of that today :(
> 
> > I wouldn't recommend using USB for anything that you need to be able to
> > accuratly control the latency of, it's just not going to work as that is
> > not what it was designed for at all.
> 
> Oh, thats a drag.  I know bulk has retries, and that on a clean bus they
> don't happen (well as far as my hobby use they shouldn't, real
> applications should use isochronous streams to avoid them.)
> 
> What is the root cause for the non deterministic behavior?  HW or USB
> stack?

The USB hardware.  Go read the spec for the gory details, in short,
nothing is guaranteed :(

> Is there any reason to think that if I created my own isochronous
> USB2Serial adapter and iso-usb-driver that I couldn't get determinism?

I strongly doubt it as others have tried and failed in the past.

sorry,

greg k-h

Re: Real time USB2Serial devices and behaivor

[Alan Stern]

On Wed, 26 Mar 2008, Greg KH wrote:

> > Is there any reason to think that if I created my own isochronous
> > USB2Serial adapter and iso-usb-driver that I couldn't get determinism?
> 
> I strongly doubt it as others have tried and failed in the past.

I don't understand.  Isochronous transfers have pretty strict 
transfer-time guarantees.  Why wouldn't this work?

One reason I can think of is that Iso transfers aren't reliable.  But
then regular RS232-type serial transfers aren't reliable either.

The only other reason is that the USB stack itself has an unpredictable 
amount of overhead.  However I think it should fall within an 
acceptable range for RT applications.

Alan Stern

Re: Real time USB2Serial devices and behaivor

[Greg KH]

On Wed, Mar 26, 2008 at 10:58:37PM -0400, Alan Stern wrote:
> On Wed, 26 Mar 2008, Greg KH wrote:
> 
> > > Is there any reason to think that if I created my own isochronous
> > > USB2Serial adapter and iso-usb-driver that I couldn't get determinism?
> > 
> > I strongly doubt it as others have tried and failed in the past.
> 
> I don't understand.  Isochronous transfers have pretty strict 
> transfer-time guarantees.  Why wouldn't this work?

I don't know, but the person who tried this a while ago said it wasn't
really "real-time" enough for their application (robot arm movement).

> One reason I can think of is that Iso transfers aren't reliable.  But
> then regular RS232-type serial transfers aren't reliable either.
> 
> The only other reason is that the USB stack itself has an unpredictable 
> amount of overhead.  However I think it should fall within an 
> acceptable range for RT applications.

It's all about bounding the longest latency.  Sometimes, under heavy
loads, latency can be pretty big.  But now that we have the -rt kernel,
it might be a lot better than before, so that might be possible now,
haven't tried it...

good luck,

greg k-h

Re: Real time USB2Serial devices and behaivor

[David Brownell]

On Wednesday 26 March 2008, mark gross wrote:
> On Wed, Mar 26, 2008 at 09:27:28AM -0700, Greg KH wrote:
> > Give up and use a "real" serial port, you'll have better luck :)
> 
> but its getting very hard to find real serial ports any more :(

I happened across this the other day, and saved the URL since
the price was relatively sane ... $15 vs the $60 I saw at
another site, although ISTR Fry's used to have them at $5:

  http://www.wrighthobbies.net/catalog/index.php?cPath=47

Yeah, I know.  "It's getting very hard to find real PCI slots
any more."  :(

It seems like some companies (notably *cough* Intel) are
on a little jihad to get rid of serial ports...


I'd think that in a controlled environment (fixed set of
USB connections) USB should be able to meet fairly chosen
"real time" latency ceilings.  The stack probably needs a
few semantic updates to make it happen -- e.g. URB completions
are issued in_irq() -- but it shouldn't be insurmountable.

- Dave

Re: Real time USB2Serial devices and behaivor

[Lennart Sorensen]

On Thu, Mar 27, 2008 at 12:48:40PM -0700, David Brownell wrote:
> I happened across this the other day, and saved the URL since
> the price was relatively sane ... $15 vs the $60 I saw at
> another site, although ISTR Fry's used to have them at $5:
> 
>   http://www.wrighthobbies.net/catalog/index.php?cPath=47
> 
> Yeah, I know.  "It's getting very hard to find real PCI slots
> any more."  :(

Well digi (www.digi.com) makes some nice PCI serial cards, that work
with the jsm driver.  Not sure how many models are supported, but
certainly I have used the 2 port card with DE-9 connectors.

A little bit of patching will make the jsm driver run any exar
17[CD]15[248] based card, like the ones we use on the routers I work on.
The digi cards use a rebranded exar chip with a different PCI id.  Much
better than using the 8250 driver to run the exar's in PCI 16550A mode,
although it does work.

They have PCIe cards too although I have no idea if they use the same
interface or not (it would make sense to do so on the part of exar
though).

> It seems like some companies (notably *cough* Intel) are
> on a little jihad to get rid of serial ports...
> 
> I'd think that in a controlled environment (fixed set of
> USB connections) USB should be able to meet fairly chosen
> "real time" latency ceilings.  The stack probably needs a
> few semantic updates to make it happen -- e.g. URB completions
> are issued in_irq() -- but it shouldn't be insurmountable.

-- 
Len Sorensen

Re: Real time USB2Serial devices and behaivor

[David Brownell]

On Thursday 27 March 2008, Lennart Sorensen wrote:
> > I happened across this the other day, and saved the URL since
> > the price was relatively sane ... $15 vs the $60 I saw at
> > another site, although ISTR Fry's used to have them at $5:
> > 
> >   http://www.wrighthobbies.net/catalog/index.php?cPath=47
> > 
> > Yeah, I know.  "It's getting very hard to find real PCI slots
> > any more."  :(
> 
> Well digi (www.digi.com) makes some nice PCI serial cards, that work
> with the jsm driver.  Not sure how many models are supported, but
> certainly I have used the 2 port card with DE-9 connectors.

List price $117 ... 4-port for $202 ... you can buy entire
computers for less!!  ;)

I really should have stocked up on the $5 cards...

Re: Real time USB2Serial devices and behaivor

[Lennart Sorensen]

On Thu, Mar 27, 2008 at 02:08:55PM -0700, David Brownell wrote:
> List price $117 ... 4-port for $202 ... you can buy entire
> computers for less!!  ;)
> 
> I really should have stocked up on the $5 cards...

Well the digi cards can run 460Kbps, which the $5 cards could not, and
they have 64byte FIFOs, not 16, and are more efficient to manage from
software.

Basically plain serial is dying out for the majority of users
(unfortunate for those of us that use plain serial a lot).

I didn't realize the digi cards cost that much.  We get our own serial
PCI cards made at work for less than that and they are industrial
temperature and we don't exactly do large volumes.  Of course by the
time the sales people add markup and all that they would cost at least
what the digi cards do.  :)

-- 
Len Sorensen

Re: Real time USB2Serial devices and behaivor

[Ming Lei]

2008/3/27, Greg KH <greg@kroah.com>:
> On Wed, Mar 26, 2008 at 08:25:59AM -0700, mark gross wrote:
>  > At home I'm playing with a real time amateur robotics application with a
>  > micro controller talking over its serial port to a Linux box running an RT
>  > kernel and some RT user mode software to suck in the serial port data
>  > and do some processing in real time.  One example of what I want to do
>  > with this type of configuration is to create a type of IMU thing and run
>  > the data through a kalman filter to see how good I can do with my
>  > hardware.
>  >
>  > I'll be sending data regularly at data rates between 50 and 500 HZ from
>  > the MCU (depending on mode)
>  >
>  > Anyway, I have the RT code and the system prototyped using the hardware
>  > serial port on my desktop and I'm getting rock solid RT delivery times
>  > of the data +/- 30usec.  While stressing the system in CPU and disk. (I
>  > still need to do the test while flood pinging the box)  However; when I
>  > use my USB dongle instead of the hardware serial port the system falls
>  > over WRT real time behavior.
>  >
>  > I'm just starting to look into the behavior now but has anyone looked at
>  > the RT'ness of USB2Serial + USB stack yet?  I would like to see what
>  > other folks have done.  My googleing last night didn't get any useful
>  > hits.
>
>
> I have, and others have in the past, and it's something that you can not
>  really do.  USB is not "deterministic", and these cheap usb to serial
>  devices introduce a very big lag that also is not deterministic.
>

I had done some usb serial test about 2 weeks ago and found :
1),the max read thoughtput is 480bps,using the generic usb
 serial driver and a wireless cdma usb card.
2),if I increase the read urb transfer size from one wMaxPacketSize
to four wMaxPacketSize, the max read throughtput can be beyond 1440bps.
But maybe there is some timeout risk, isn't there?
3),The throughput still can be increased by submitting more urbs
 parallelly , and the latency can be decreased a bit by the way.

>  I wouldn't recommend using USB for anything that you need to be able to
>  accuratly control the latency of, it's just not going to work as that is
>  not what it was designed for at all.
>
>
>  > BTW Why does the PL2303 have a 1KB buffer for incoming data?
>
>
> The device or the driver?
>
>
>  > I'm a bit green at RT work at this level.  What are the best ways for
>  > drilling down to figure out what the problems are so I can make a patch?
>
>
> Give up and use a "real" serial port, you'll have better luck :)
>
>  thanks,
>
>  greg k-h
>
> --
>  To unsubscribe from this list: send the line "unsubscribe linux-usb" in
>  the body of a message to majordomo@vger.kernel.org
>  More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Lei Ming

Re: Real time USB2Serial devices and behaivor

[Peter Zijlstra]

On Thu, 2008-04-03 at 22:26 +0800, Ming Lei wrote:
> 2008/3/27, Greg KH <greg@kroah.com>:
> > On Wed, Mar 26, 2008 at 08:25:59AM -0700, mark gross wrote:
> >  > At home I'm playing with a real time amateur robotics application with a
> >  > micro controller talking over its serial port to a Linux box running an RT
> >  > kernel and some RT user mode software to suck in the serial port data
> >  > and do some processing in real time.  One example of what I want to do
> >  > with this type of configuration is to create a type of IMU thing and run
> >  > the data through a kalman filter to see how good I can do with my
> >  > hardware.
> >  >
> >  > I'll be sending data regularly at data rates between 50 and 500 HZ from
> >  > the MCU (depending on mode)
> >  >
> >  > Anyway, I have the RT code and the system prototyped using the hardware
> >  > serial port on my desktop and I'm getting rock solid RT delivery times
> >  > of the data +/- 30usec.  While stressing the system in CPU and disk. (I
> >  > still need to do the test while flood pinging the box)  However; when I
> >  > use my USB dongle instead of the hardware serial port the system falls
> >  > over WRT real time behavior.
> >  >
> >  > I'm just starting to look into the behavior now but has anyone looked at
> >  > the RT'ness of USB2Serial + USB stack yet?  I would like to see what
> >  > other folks have done.  My googleing last night didn't get any useful
> >  > hits.
> >
> >
> > I have, and others have in the past, and it's something that you can not
> >  really do.  USB is not "deterministic", and these cheap usb to serial
> >  devices introduce a very big lag that also is not deterministic.
> >
> 
> I had done some usb serial test about 2 weeks ago and found :
> 1),the max read thoughtput is 480bps,using the generic usb
>  serial driver and a wireless cdma usb card.
> 2),if I increase the read urb transfer size from one wMaxPacketSize
> to four wMaxPacketSize, the max read throughtput can be beyond 1440bps.
> But maybe there is some timeout risk, isn't there?
> 3),The throughput still can be increased by submitting more urbs
>  parallelly , and the latency can be decreased a bit by the way.

Non of which is relevant to realtime.

Realtime is about meeting deadlines, sod throughput (although if you can
get decent throughput while meeting the deadlines that's even better).

Realtime is about getting a signal on your serial port and reacting to
it before the $doom_scenario happens.

( I prefer the one where you break the laser guard and have to stop the
  saw blade before it cuts your fingers off - others prefer nuclear
  chain reactions )

Re: Real time USB2Serial devices and behaivor

[Greg KH]

On Thu, Apr 03, 2008 at 10:26:57PM +0800, Ming Lei wrote:
> 2008/3/27, Greg KH <greg@kroah.com>:
> > On Wed, Mar 26, 2008 at 08:25:59AM -0700, mark gross wrote:
> >  > At home I'm playing with a real time amateur robotics application with a
> >  > micro controller talking over its serial port to a Linux box running an RT
> >  > kernel and some RT user mode software to suck in the serial port data
> >  > and do some processing in real time.  One example of what I want to do
> >  > with this type of configuration is to create a type of IMU thing and run
> >  > the data through a kalman filter to see how good I can do with my
> >  > hardware.
> >  >
> >  > I'll be sending data regularly at data rates between 50 and 500 HZ from
> >  > the MCU (depending on mode)
> >  >
> >  > Anyway, I have the RT code and the system prototyped using the hardware
> >  > serial port on my desktop and I'm getting rock solid RT delivery times
> >  > of the data +/- 30usec.  While stressing the system in CPU and disk. (I
> >  > still need to do the test while flood pinging the box)  However; when I
> >  > use my USB dongle instead of the hardware serial port the system falls
> >  > over WRT real time behavior.
> >  >
> >  > I'm just starting to look into the behavior now but has anyone looked at
> >  > the RT'ness of USB2Serial + USB stack yet?  I would like to see what
> >  > other folks have done.  My googleing last night didn't get any useful
> >  > hits.
> >
> >
> > I have, and others have in the past, and it's something that you can not
> >  really do.  USB is not "deterministic", and these cheap usb to serial
> >  devices introduce a very big lag that also is not deterministic.
> >
> 
> I had done some usb serial test about 2 weeks ago and found :
> 1),the max read thoughtput is 480bps,using the generic usb
>  serial driver and a wireless cdma usb card.

The generic usb serial driver is KNOWN TO BE A VERY SLOW DRIVER!

I'm getting very tired of repeating myself about this one, don't do any
speed tests, or run any real hardware modems with this driver at all,
it's just not worth it.

The code was not designed to be fast, only get the job done.

> 2),if I increase the read urb transfer size from one wMaxPacketSize
> to four wMaxPacketSize, the max read throughtput can be beyond 1440bps.
> But maybe there is some timeout risk, isn't there?

That's one way to increase speed, but not the only way, it can be done a
whole lot better if you are worried about throughput.  See the sierra
driver for an implementation that does this correctly for maximum
througput.

> 3),The throughput still can be increased by submitting more urbs
>  parallelly , and the latency can be decreased a bit by the way.

But again, that has nothing to do with determinism, or latency, or real
time issues.

thanks,

greg k-h

Re: Real time USB2Serial devices and behaivor

[Jon Masters]

On Wed, 2008-03-26 at 08:25 -0700, mark gross wrote:

> BTW Why does the PL2303 have a 1KB buffer for incoming data?

Personally, RT or not, I really like to avoid using those PL230X parts.
They're everywhere, this is true, but also quite horrible sometimes.

Jon.