Linux tty layer hackery: Heads up and RFC

Linux tty layer hackery: Heads up and RFC

[Alan Cox]

At the moment tty buffers are attached directly to the tty. This is
causing a lot of the problems related to tty layer locking, also
problems at high speed and also with bursty data (such as occurs in
virtualised environments)

I'm working on ripping out the flip buffers and replacing them with a
pool of dynamically allocated buffers. This allows both for old style
"byte I/O" devices and also helps virtualisation and smart devices where
large blocks of data suddenely materialise and need storing.

So far so good. Lots of drivers reference tty->flip.*. Several of them
also call directly and unsafely into function pointers it provides. This
will all break. Most drivers can use tty_insert_flip_char which can be
kept as an API but others need more.

At the moment I've added the following interfaces, if people think more
will be needed now is a good time to say


int tty_buffer_request_room(tty, size)

Try and ensure at least size bytes are available, returns actual room
(may be zero). At the moment it just uses the flipbuf space but that
will change. Repeated calls without characters being added are not
cumulative. (ie if you call it with 1, 1, 1, and then 4 you'll have four
characters of space. The other functions will also try and grow buffers
in future but this will be a more efficient way when you know block
sizes.

int tty_insert_flip_char(tty, ch, flag)

As before insert a character if there is room. Now returns 1 for
success, 0 for failure.

int tty_insert_flip_string(tty, str, len)

Insert a block of non error characters. Returns the number inserted.

int tty_prepare_flip_string(tty, strptr, len)

Adjust the buffer to allow len characters to be added. Returns a buffer
pointer in strptr and the length available. This allows for hardware
that needs to use functions like insl or mencpy_fromio.


I've converted a fair number of drivers to this API ready and I'll post
some patches for review soon.

Alan

Re: Linux tty layer hackery: Heads up and RFC

[Sergei Organov]

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:
> At the moment tty buffers are attached directly to the tty. This is
> causing a lot of the problems related to tty layer locking, also
> problems at high speed and also with bursty data (such as occurs in
> virtualised environments)
> 
> I'm working on ripping out the flip buffers and replacing them with a
> pool of dynamically allocated buffers. This allows both for old style
> "byte I/O" devices and also helps virtualisation and smart devices where
> large blocks of data suddenely materialise and need storing.

Great! Really good news!

> 
> So far so good. Lots of drivers reference tty->flip.*. Several of them
> also call directly and unsafely into function pointers it provides. This
> will all break. Most drivers can use tty_insert_flip_char which can be
> kept as an API but others need more.
> 
> At the moment I've added the following interfaces, if people think more
> will be needed now is a good time to say
> 
> int tty_buffer_request_room(tty, size)
> 
> Try and ensure at least size bytes are available, returns actual room
> (may be zero). At the moment it just uses the flipbuf space but that
> will change. Repeated calls without characters being added are not
> cumulative. (ie if you call it with 1, 1, 1, and then 4 you'll have four
> characters of space. The other functions will also try and grow buffers
> in future but this will be a more efficient way when you know block
> sizes.
> 
> int tty_insert_flip_char(tty, ch, flag)
> 
> As before insert a character if there is room. Now returns 1 for
> success, 0 for failure.
> 
> int tty_insert_flip_string(tty, str, len)
> 
> Insert a block of non error characters. Returns the number inserted.
> 
> int tty_prepare_flip_string(tty, strptr, len)
> 
> Adjust the buffer to allow len characters to be added. Returns a buffer
> pointer in strptr and the length available. This allows for hardware
> that needs to use functions like insl or mencpy_fromio.

As you are going to replace flip buffers with different implementation
anyway, isn't it better to get rid of "flip" in the interface names as
well (maybe providing some synonyms for backward compatibility)? What I
mean is that names like

tty_buffer_insert_char()
tty_buffer_insert_string()

for the new interfaces would probably make more sense.

Otherwise I find the interfaces you suggest just fine and suitable for
the task I was unable to achieve without ugly hacks using current flip
buffers interfaces (reliable high-speed bulk USB tty driver).

-- 
Sergei.

Re: Linux tty layer hackery: Heads up and RFC

[Rogier Wolff]

On Thu, Jul 21, 2005 at 06:46:32PM +0100, Alan Cox wrote:
> int tty_prepare_flip_string(tty, strptr, len)
> 
> Adjust the buffer to allow len characters to be added. Returns a buffer
> pointer in strptr and the length available. This allows for hardware
> that needs to use functions like insl or mencpy_fromio.

Ok, So then I start copying characters into the flipstring, but how do
I say I'm done?

Or is there a race between that I call tty_prepare_flip_string, and
other processes start pulling my not-yet-filled string from the
buffer? (Surely not!)

	Roger. 

-- 
** R.E.Wolff@BitWizard.nl ** http://www.BitWizard.nl/ ** +31-15-2600998 **
*-- BitWizard writes Linux device drivers for any device you may have! --*
Q: It doesn't work. A: Look buddy, doesn't work is an ambiguous statement. 
Does it sit on the couch all day? Is it unemployed? Please be specific! 
Define 'it' and what it isn't doing. --------- Adapted from lxrbot FAQ

Re: Linux tty layer hackery: Heads up and RFC

[Alan Cox]

On Gwe, 2005-07-22 at 16:57 +0200, Rogier Wolff wrote:
> Ok, So then I start copying characters into the flipstring, but how do
> I say I'm done?

tty_flip_buffer_push() (or its possibly renamed equivalent). At that
point as now the buffer may get processed or queued to the ldisc. At
that point its now owned by the ldisc and the tty buffer layer will give
you new buffers.

Re: Linux tty layer hackery: Heads up and RFC

[Mark Underwood]

Hi Rogier,

Having just written a DMA UART driver I can say this
is good news :-). Here are some things to think about:

What my driver would like to do is to handle its own
input buffers. It would pass the buffer to the tty
layer when it is full and the tty layer would pass the
buffer back once it has drained the data from it.
The problem is that I don't always receive a block
worth of characters so I also need to pass the tty
layer a buffer (which I'm still DMAing into) with a
count of how many chars there are in the buffer and a
offset of where to start from. If I can't do this then
I have to stop the DMA transfer, pass you a mostly
empty buffer with a char count, and setup DMA transfer
into another buffer. If during this I get a burst of
data I could well lose it :-(.

Best Regards,

Mark

--- Rogier Wolff <R.E.Wolff@BitWizard.nl> wrote:

> On Thu, Jul 21, 2005 at 06:46:32PM +0100, Alan Cox
> wrote:
> > int tty_prepare_flip_string(tty, strptr, len)
> > 
> > Adjust the buffer to allow len characters to be
> added. Returns a buffer
> > pointer in strptr and the length available. This
> allows for hardware
> > that needs to use functions like insl or
> mencpy_fromio.
> 
> Ok, So then I start copying characters into the
> flipstring, but how do
> I say I'm done?
> 
> Or is there a race between that I call
> tty_prepare_flip_string, and
> other processes start pulling my not-yet-filled
> string from the
> buffer? (Surely not!)
> 
> 	Roger. 
> 
> -- 
> ** R.E.Wolff@BitWizard.nl **
> http://www.BitWizard.nl/ ** +31-15-2600998 **
> *-- BitWizard writes Linux device drivers for any
> device you may have! --*
> Q: It doesn't work. A: Look buddy, doesn't work is
> an ambiguous statement. 
> Does it sit on the couch all day? Is it unemployed?
> Please be specific! 
> Define 'it' and what it isn't doing. ---------
> Adapted from lxrbot FAQ
> -
> To unsubscribe from this list: send the line
> "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at 
> http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 



	
	
		
___________________________________________________________ 
Yahoo! Messenger - NEW crystal clear PC to PC calling worldwide with voicemail http://uk.messenger.yahoo.com

Re: Linux tty layer hackery: Heads up and RFC

[Alan Cox]

On Maw, 2005-07-26 at 10:55 +0100, Mark Underwood wrote:
> What my driver would like to do is to handle its own
> input buffers. It would pass the buffer to the tty
> layer when it is full and the tty layer would pass the

In theory you can do that already, although the locking is a bit screwed
up for it. Actually all the tty locking is broken for rx I believe.
Everyone should be holding the tty read lock when updating flip buffers
but right now we don't

> buffer back once it has drained the data from it.
> The problem is that I don't always receive a block
> worth of characters so I also need to pass the tty
> layer a buffer (which I'm still DMAing into) with a
> count of how many chars there are in the buffer and a
> offset of where to start from.

You can do this now providing you don't do it blindly from IRQ context.

>From a workqueue do

	struct tty_ldisc *ld = tty_ldisc_ref(tty);
	int space;

	if(ld == NULL)	/* Bin/defer */
		return;
	space = ld->receive_room(tty);
	if(count > space) count = space;

	ld->receive_buf(tty, charbuf, flagbuf, count);


There is a corner case if TTY_DONT_FLIP is set where you should queue
but not all drivers do this and the DONT_FLIP hack 'has to die' 

Re: Linux tty layer hackery: Heads up and RFC

[Mark Underwood]

Hi Alan,

Thanks for your help, I might give this ago once I've
fixed some flow control problems in my driver.

On a loosely related topic I have extended
serial_core.c to handle DMA UARTS (only the TX path is
effected). Once I'm happy with my changes I post a
patch.

Best Regards,

Mark

--- Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:

> On Maw, 2005-07-26 at 10:55 +0100, Mark Underwood
> wrote:
> > What my driver would like to do is to handle its
> own
> > input buffers. It would pass the buffer to the tty
> > layer when it is full and the tty layer would pass
> the
> 
> In theory you can do that already, although the
> locking is a bit screwed
> up for it. Actually all the tty locking is broken
> for rx I believe.
> Everyone should be holding the tty read lock when
> updating flip buffers
> but right now we don't
> 
> > buffer back once it has drained the data from it.
> > The problem is that I don't always receive a block
> > worth of characters so I also need to pass the tty
> > layer a buffer (which I'm still DMAing into) with
> a
> > count of how many chars there are in the buffer
> and a
> > offset of where to start from.
> 
> You can do this now providing you don't do it
> blindly from IRQ context.
> 
> >From a workqueue do
> 
> 	struct tty_ldisc *ld = tty_ldisc_ref(tty);
> 	int space;
> 
> 	if(ld == NULL)	/* Bin/defer */
> 		return;
> 	space = ld->receive_room(tty);
> 	if(count > space) count = space;
> 
> 	ld->receive_buf(tty, charbuf, flagbuf, count);
> 
> 
> There is a corner case if TTY_DONT_FLIP is set where
> you should queue
> but not all drivers do this and the DONT_FLIP hack
> 'has to die' 
> 
> -
> To unsubscribe from this list: send the line
> "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at 
> http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 



		
___________________________________________________________ 
How much free photo storage do you get? Store your holiday 
snaps for FREE with Yahoo! Photos http://uk.photos.yahoo.com