/sys/class/tty/tty0/active?

/sys/class/tty/tty0/active?

[richard -rw- weinberger]

Hi!

systemd goes nuts on UML.
While hunting down all issues I found out that systemd wants to read from
/sys/class/tty/tty0/active.

On UML this directory does not have this "active" file.
[1] indicates that it has something to do with /dev/tty0.
Of course, UML has a /dev/tty0, it's the default console.

But where comes "active" from?
Is there any black magic hidden register API to get this file? ;-)

[1]
http://lists.freedesktop.org/archives/systemd-devel/2011-December/004096.html

-- 
Thanks,
//richard

/sys/class/tty/tty0/active?

[richard -rw- weinberger]

Hi!

systemd goes nuts on UML.
While hunting down all issues I found out that systemd wants to read from
/sys/class/tty/tty0/active.

On UML this directory does not have this "active" file.
[1] indicates that it has something to do with /dev/tty0.
Of course, UML has a /dev/tty0, it's the default console.

But where comes "active" from?
Is there any black magic hidden register API to get this file? ;-)

[1]
http://lists.freedesktop.org/archives/systemd-devel/2011-December/004096.html

-- 
Thanks,
//richard

Re: [uml-devel] /sys/class/tty/tty0/active?

[stian]

> Hi!
>
> systemd goes nuts on UML.
> While hunting down all issues I found out that systemd wants to read from
> /sys/class/tty/tty0/active.

Kernel version?
http://comments.gmane.org/gmane.comp.sysutils.systemd.devel/3079 indicates
that the given file appears in 2.6.37

Stian Skjelstad

Re: [uml-devel] /sys/class/tty/tty0/active?

[richard -rw- weinberger]

On Thu, Jan 26, 2012 at 11:11 PM,  <stian@nixia.no> wrote:
>> Hi!
>>
>> systemd goes nuts on UML.
>> While hunting down all issues I found out that systemd wants to read from
>> /sys/class/tty/tty0/active.
>
> Kernel version?

Linus' tree as of today.

-- 
Thanks,
//richard

Re: [uml-devel] /sys/class/tty/tty0/active?

[richard -rw- weinberger]

On Thu, Jan 26, 2012 at 11:12 PM, richard -rw- weinberger
<richard.weinberger@gmail.com> wrote:
> On Thu, Jan 26, 2012 at 11:11 PM,  <stian@nixia.no> wrote:
>>> Hi!
>>>
>>> systemd goes nuts on UML.
>>> While hunting down all issues I found out that systemd wants to read from
>>> /sys/class/tty/tty0/active.
>>
>> Kernel version?
>
> Linus' tree as of today.

Hmm, fbc92a34 introduced it.
But it depends on CONFIG_VT. :-\

-- 
Thanks,
//richard

Re: [uml-devel] /sys/class/tty/tty0/active?

[Greg KH]

On Thu, Jan 26, 2012 at 11:56:50PM +0100, richard -rw- weinberger wrote:
> On Thu, Jan 26, 2012 at 11:12 PM, richard -rw- weinberger
> <richard.weinberger@gmail.com> wrote:
> > On Thu, Jan 26, 2012 at 11:11 PM,  <stian@nixia.no> wrote:
> >>> Hi!
> >>>
> >>> systemd goes nuts on UML.
> >>> While hunting down all issues I found out that systemd wants to read from
> >>> /sys/class/tty/tty0/active.
> >>
> >> Kernel version?
> >
> > Linus' tree as of today.
> 
> Hmm, fbc92a34 introduced it.
> But it depends on CONFIG_VT. :-\

Is that a problem for UML somehow?

Re: [uml-devel] /sys/class/tty/tty0/active?

[Kay Sievers]

On Fri, Jan 27, 2012 at 02:08, Greg KH <gregkh@suse.de> wrote:
> On Thu, Jan 26, 2012 at 11:56:50PM +0100, richard -rw- weinberger wrote:
>> On Thu, Jan 26, 2012 at 11:12 PM, richard -rw- weinberger
>> <richard.weinberger@gmail.com> wrote:
>> > On Thu, Jan 26, 2012 at 11:11 PM,  <stian@nixia.no> wrote:
>> >>> Hi!
>> >>>
>> >>> systemd goes nuts on UML.
>> >>> While hunting down all issues I found out that systemd wants to read from
>> >>> /sys/class/tty/tty0/active.
>> >>
>> >> Kernel version?
>> >
>> > Linus' tree as of today.
>>
>> Hmm, fbc92a34 introduced it.
>> But it depends on CONFIG_VT. :-\
>
> Is that a problem for UML somehow?

In general, the "crippled" environments might not really the right
thing too run full-blown user login managers like logind.

The main systemd 'init' will not have any problems with that, but
running logind might not really work out under UML.

Depending how far full-OS support in UML should go, logind should
either not run inside UML, or if, UML should provide the needed bits
for in virtual consoles.

If it is not to be solved otherwise, maybe logind needs more
VT-avoiding logic like this:
  http://cgit.freedesktop.org/systemd/systemd/commit/?id=addedec48ba0ffc4472ef6d3b5a45c9d4239f1cd

Kay

Re: [uml-devel] /sys/class/tty/tty0/active?

[Kay Sievers]

On Fri, Jan 27, 2012 at 02:08, Greg KH <gregkh@suse.de> wrote:
> On Thu, Jan 26, 2012 at 11:56:50PM +0100, richard -rw- weinberger wrote:
>> On Thu, Jan 26, 2012 at 11:12 PM, richard -rw- weinberger
>> <richard.weinberger@gmail.com> wrote:
>> > On Thu, Jan 26, 2012 at 11:11 PM,  <stian@nixia.no> wrote:
>> >>> Hi!
>> >>>
>> >>> systemd goes nuts on UML.
>> >>> While hunting down all issues I found out that systemd wants to read from
>> >>> /sys/class/tty/tty0/active.
>> >>
>> >> Kernel version?
>> >
>> > Linus' tree as of today.
>>
>> Hmm, fbc92a34 introduced it.
>> But it depends on CONFIG_VT. :-\
>
> Is that a problem for UML somehow?

In general, the "crippled" environments might not really the right
thing too run full-blown user login managers like logind.

The main systemd 'init' will not have any problems with that, but
running logind might not really work out under UML.

Depending how far full-OS support in UML should go, logind should
either not run inside UML, or if, UML should provide the needed bits
for in virtual consoles.

If it is not to be solved otherwise, maybe logind needs more
VT-avoiding logic like this:
  http://cgit.freedesktop.org/systemd/systemd/commit/?id=addedec48ba0ffc4472ef6d3b5a45c9d4239f1cd

Kay

Re: [uml-devel] /sys/class/tty/tty0/active?

[richard -rw- weinberger]

Greg, Kay,

>> Is that a problem for UML somehow?

The login on ttyX takes 30+ seconds.
This very ugly and mostly wrong hack makes the login fast again:
mount -o bind /sys/class/tty/console/ /sys/class/tty/tty0/

/sys/class/tty/console/active exists and contains "tty0"...

> In general, the "crippled" environments might not really the right
> thing too run full-blown user login managers like logind.
>
> The main systemd 'init' will not have any problems with that, but
> running logind might not really work out under UML.
>
> Depending how far full-OS support in UML should go, logind should
> either not run inside UML, or if, UML should provide the needed bits
> for in virtual consoles.

Any x86 distribution works fine in UML.
If this will be no longer true due to systemd then better no systemd developer
ever crosses my way. ;-)

> If it is not to be solved otherwise, maybe logind needs more
> VT-avoiding logic like this:
>  http://cgit.freedesktop.org/systemd/systemd/commit/?id=addedec48ba0ffc4472ef6d3b5a45c9d4239f1cd

Yeah, would make sense.
Many systems don't have CONFIG_VT set.

One good thing is that systemd seems to uncover a nasty bug in UML's
console driver.
A login on tty0 crashes UML.

UML's console driver (arch/um/drivers/line.c) implements tty_operations.
The crash happens because the tty subsystem calls the driver's close()
function and later
write_room() or chars_in_buffer().

write_room() and chars_in_buffer() fail badly because close() already
cleaned up the driver's private data...

Greg, is UML's assumption wrong that after closing the tty no call to
write_room() or chars_in_buffer() can happen?
I have no idea why systemd is able to trigger this, UML's console
driver is old and has always worked quite well.

-- 
Thanks,
//richard

Re: [uml-devel] /sys/class/tty/tty0/active?

[Alan Cox]

> UML's console driver (arch/um/drivers/line.c) implements tty_operations.
> The crash happens because the tty subsystem calls the driver's close()
> function and later
> write_room() or chars_in_buffer().
> 
> write_room() and chars_in_buffer() fail badly because close() already
> cleaned up the driver's private data...

You don't want to do that.

> Greg, is UML's assumption wrong that after closing the tty no call to
> write_room() or chars_in_buffer() can happen?
> I have no idea why systemd is able to trigger this, UML's console
> driver is old and has always worked quite well.

It's always been untrue but it's even more untrue nowdays. The tty layer
objects are refcounted, and the code has had significant rewrites. line.c
hidden away in uml hasn't been updated.

I added a comment about 3 years ago pointing out another older change
that was needed and that wasn't acted on either..

Take a look at how all the other tty drivers use tty_port, how the ioctls
have been supposed to work for the past few years and the callback
changes, then use them.

tty_port will actually do most of the hard work for you.

Alan

Re: [uml-devel] /sys/class/tty/tty0/active?

[richard -rw- weinberger]

On Fri, Jan 27, 2012 at 12:51 PM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>> UML's console driver (arch/um/drivers/line.c) implements tty_operations.
>> The crash happens because the tty subsystem calls the driver's close()
>> function and later
>> write_room() or chars_in_buffer().
>>
>> write_room() and chars_in_buffer() fail badly because close() already
>> cleaned up the driver's private data...
>
> You don't want to do that.

That's what i thought.

>> Greg, is UML's assumption wrong that after closing the tty no call to
>> write_room() or chars_in_buffer() can happen?
>> I have no idea why systemd is able to trigger this, UML's console
>> driver is old and has always worked quite well.
>
> It's always been untrue but it's even more untrue nowdays. The tty layer
> objects are refcounted, and the code has had significant rewrites. line.c
> hidden away in uml hasn't been updated.
>
> I added a comment about 3 years ago pointing out another older change
> that was needed and that wasn't acted on either..
>
> Take a look at how all the other tty drivers use tty_port, how the ioctls
> have been supposed to work for the past few years and the callback
> changes, then use them.

Can you recommend a well-written driver?

-- 
Thanks,
//richard

Re: [uml-devel] /sys/class/tty/tty0/active?

[Alan Cox]

On Fri, 27 Jan 2012 13:04:37 +0100
richard -rw- weinberger <richard.weinberger@gmail.com> wrote:

> On Fri, Jan 27, 2012 at 12:51 PM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> >> UML's console driver (arch/um/drivers/line.c) implements tty_operations.
> >> The crash happens because the tty subsystem calls the driver's close()
> >> function and later
> >> write_room() or chars_in_buffer().
> >>
> >> write_room() and chars_in_buffer() fail badly because close() already
> >> cleaned up the driver's private data...
> >
> > You don't want to do that.
> 
> That's what i thought.
> 
> >> Greg, is UML's assumption wrong that after closing the tty no call to
> >> write_room() or chars_in_buffer() can happen?
> >> I have no idea why systemd is able to trigger this, UML's console
> >> driver is old and has always worked quite well.
> >
> > It's always been untrue but it's even more untrue nowdays. The tty layer
> > objects are refcounted, and the code has had significant rewrites. line.c
> > hidden away in uml hasn't been updated.
> >
> > I added a comment about 3 years ago pointing out another older change
> > that was needed and that wasn't acted on either..
> >
> > Take a look at how all the other tty drivers use tty_port, how the ioctls
> > have been supposed to work for the past few years and the callback
> > changes, then use them.
> 
> Can you recommend a well-written driver?

drivers/mmc/card/sdio_uart.c

uses just about all the features including handling hotplug and stuff you
don't need.

drivers/usb/serial/usb-serial.c

may also be handy as it provides the interface but then calls into other
driver code to do the work.

Basically though you want a struct tty_port in your private data, either
created at open, or usually more cleanly for the physical port lifetime

	tty_port_init()

Sets it up, then set the port ops

	tty_port_open()
	tty_port_close()
	tty_port_hangup()

do almost all of the rest of the work for you. They call back to your
activate and shutdown port methods, they serialize them, they call them
on first open/last close in matching pairs.

For the tty itself

	tty_port_tty_get()

gets you a reference to the tty from the port (or NULL) - so handles a
close/hangup racing with data arrival

	tty_kref_put()

releases a reference

and 
	tty->ops->cleanup()

is called on the final destruction of the tty object (ie its where you
can free tty lifetime data in tty->private_data)

So for a simple non pluggable tty it tends to look like

	int my_tty_open(struct tty_struct *tty, struct file *filp)
	{
		tty->driver_data = &my_port;
		return tty_port_open(&my_port, tty, filp);
	}

	void my_tty_close(struct tty_struct *tty, struct file *filp)
	{
		struct my_tty *m = tty->driver_data;
		if (m == NULL)
			return;
		tty_port_close(&m->port, tty, filp);
	}

	void my_tty_hangup(struct tty_struct *tty)
	{
		struct my_tty *m = tty->driver_data;
		tty_port_hangup(&m->port);
	}

provide the needed callbacks and it'll do the locking and the like for
you.

On the ioctl side as far as I can see you should simply get rid of the
method entirely.

For buffer_data you might want to allocate the buffer sanely at open time
(tty_port has a function for this too) so it can't fail weirdly

And your termios method is a bit odd but makes sense if you are just
pretending anything works and is supported.

Alan

Re: [uml-devel] /sys/class/tty/tty0/active?

[richard -rw- weinberger]

On Fri, Jan 27, 2012 at 3:02 PM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
> On Fri, 27 Jan 2012 13:04:37 +0100
> richard -rw- weinberger <richard.weinberger@gmail.com> wrote:
>
>> On Fri, Jan 27, 2012 at 12:51 PM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>> >> UML's console driver (arch/um/drivers/line.c) implements tty_operations.
>> >> The crash happens because the tty subsystem calls the driver's close()
>> >> function and later
>> >> write_room() or chars_in_buffer().
>> >>
>> >> write_room() and chars_in_buffer() fail badly because close() already
>> >> cleaned up the driver's private data...
>> >
>> > You don't want to do that.
>>
>> That's what i thought.
>>
>> >> Greg, is UML's assumption wrong that after closing the tty no call to
>> >> write_room() or chars_in_buffer() can happen?
>> >> I have no idea why systemd is able to trigger this, UML's console
>> >> driver is old and has always worked quite well.
>> >
>> > It's always been untrue but it's even more untrue nowdays. The tty layer
>> > objects are refcounted, and the code has had significant rewrites. line.c
>> > hidden away in uml hasn't been updated.
>> >
>> > I added a comment about 3 years ago pointing out another older change
>> > that was needed and that wasn't acted on either..
>> >
>> > Take a look at how all the other tty drivers use tty_port, how the ioctls
>> > have been supposed to work for the past few years and the callback
>> > changes, then use them.
>>
>> Can you recommend a well-written driver?
>
> drivers/mmc/card/sdio_uart.c
>
> uses just about all the features including handling hotplug and stuff you
> don't need.
>
> drivers/usb/serial/usb-serial.c
>
> may also be handy as it provides the interface but then calls into other
> driver code to do the work.
>
> Basically though you want a struct tty_port in your private data, either
> created at open, or usually more cleanly for the physical port lifetime
>
>        tty_port_init()
>
> Sets it up, then set the port ops
>
>        tty_port_open()
>        tty_port_close()
>        tty_port_hangup()
>
> do almost all of the rest of the work for you. They call back to your
> activate and shutdown port methods, they serialize them, they call them
> on first open/last close in matching pairs.
>
> For the tty itself
>
>        tty_port_tty_get()
>
> gets you a reference to the tty from the port (or NULL) - so handles a
> close/hangup racing with data arrival
>
>        tty_kref_put()
>
> releases a reference
>
> and
>        tty->ops->cleanup()
>
> is called on the final destruction of the tty object (ie its where you
> can free tty lifetime data in tty->private_data)
>
> So for a simple non pluggable tty it tends to look like
>
>        int my_tty_open(struct tty_struct *tty, struct file *filp)
>        {
>                tty->driver_data = &my_port;
>                return tty_port_open(&my_port, tty, filp);
>        }
>
>        void my_tty_close(struct tty_struct *tty, struct file *filp)
>        {
>                struct my_tty *m = tty->driver_data;
>                if (m == NULL)
>                        return;
>                tty_port_close(&m->port, tty, filp);
>        }
>
>        void my_tty_hangup(struct tty_struct *tty)
>        {
>                struct my_tty *m = tty->driver_data;
>                tty_port_hangup(&m->port);
>        }
>
> provide the needed callbacks and it'll do the locking and the like for
> you.
>
> On the ioctl side as far as I can see you should simply get rid of the
> method entirely.
>
> For buffer_data you might want to allocate the buffer sanely at open time
> (tty_port has a function for this too) so it can't fail weirdly
>
> And your termios method is a bit odd but makes sense if you are just
> pretending anything works and is supported.

Thanks for your help!
tty_port makes sense and I think I understood it.

But after modifying the driver to use tty_port the kernel crashes in
tty_io.c:__tty_fasync() because filp->f_path.dentry is NULL.

Any idea which kind of error can cause this?

-- 
Thanks,
//richard

Re: /sys/class/tty/tty0/active?

[richard -rw- weinberger]

On Sat, Jan 28, 2012 at 12:55 AM, richard -rw- weinberger
<richard.weinberger@gmail.com> wrote:
> On Fri, Jan 27, 2012 at 3:02 PM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>> On Fri, 27 Jan 2012 13:04:37 +0100
>> richard -rw- weinberger <richard.weinberger@gmail.com> wrote:
>>
>>> On Fri, Jan 27, 2012 at 12:51 PM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>>> >> UML's console driver (arch/um/drivers/line.c) implements tty_operations.
>>> >> The crash happens because the tty subsystem calls the driver's close()
>>> >> function and later
>>> >> write_room() or chars_in_buffer().
>>> >>
>>> >> write_room() and chars_in_buffer() fail badly because close() already
>>> >> cleaned up the driver's private data...
>>> >
>>> > You don't want to do that.
>>>
>>> That's what i thought.
>>>
>>> >> Greg, is UML's assumption wrong that after closing the tty no call to
>>> >> write_room() or chars_in_buffer() can happen?
>>> >> I have no idea why systemd is able to trigger this, UML's console
>>> >> driver is old and has always worked quite well.
>>> >
>>> > It's always been untrue but it's even more untrue nowdays. The tty layer
>>> > objects are refcounted, and the code has had significant rewrites. line.c
>>> > hidden away in uml hasn't been updated.
>>> >
>>> > I added a comment about 3 years ago pointing out another older change
>>> > that was needed and that wasn't acted on either..
>>> >
>>> > Take a look at how all the other tty drivers use tty_port, how the ioctls
>>> > have been supposed to work for the past few years and the callback
>>> > changes, then use them.
>>>
>>> Can you recommend a well-written driver?
>>
>> drivers/mmc/card/sdio_uart.c
>>
>> uses just about all the features including handling hotplug and stuff you
>> don't need.
>>
>> drivers/usb/serial/usb-serial.c
>>
>> may also be handy as it provides the interface but then calls into other
>> driver code to do the work.
>>
>> Basically though you want a struct tty_port in your private data, either
>> created at open, or usually more cleanly for the physical port lifetime
>>
>>        tty_port_init()
>>
>> Sets it up, then set the port ops
>>
>>        tty_port_open()
>>        tty_port_close()
>>        tty_port_hangup()
>>
>> do almost all of the rest of the work for you. They call back to your
>> activate and shutdown port methods, they serialize them, they call them
>> on first open/last close in matching pairs.
>>
>> For the tty itself
>>
>>        tty_port_tty_get()
>>
>> gets you a reference to the tty from the port (or NULL) - so handles a
>> close/hangup racing with data arrival
>>
>>        tty_kref_put()
>>
>> releases a reference
>>
>> and
>>        tty->ops->cleanup()
>>
>> is called on the final destruction of the tty object (ie its where you
>> can free tty lifetime data in tty->private_data)
>>
>> So for a simple non pluggable tty it tends to look like
>>
>>        int my_tty_open(struct tty_struct *tty, struct file *filp)
>>        {
>>                tty->driver_data = &my_port;
>>                return tty_port_open(&my_port, tty, filp);
>>        }
>>
>>        void my_tty_close(struct tty_struct *tty, struct file *filp)
>>        {
>>                struct my_tty *m = tty->driver_data;
>>                if (m == NULL)
>>                        return;
>>                tty_port_close(&m->port, tty, filp);
>>        }
>>
>>        void my_tty_hangup(struct tty_struct *tty)
>>        {
>>                struct my_tty *m = tty->driver_data;
>>                tty_port_hangup(&m->port);
>>        }
>>
>> provide the needed callbacks and it'll do the locking and the like for
>> you.
>>
>> On the ioctl side as far as I can see you should simply get rid of the
>> method entirely.
>>
>> For buffer_data you might want to allocate the buffer sanely at open time
>> (tty_port has a function for this too) so it can't fail weirdly
>>
>> And your termios method is a bit odd but makes sense if you are just
>> pretending anything works and is supported.
>
> Thanks for your help!
> tty_port makes sense and I think I understood it.
>
> But after modifying the driver to use tty_port the kernel crashes in
> tty_io.c:__tty_fasync() because filp->f_path.dentry is NULL.
>
> Any idea which kind of error can cause this?

Never mind. :-)

UML's line driver works fine now.
But it needs some more cleanups.
Especially the serial line and management console code.

-- 
Thanks,
//richard

------------------------------------------------------------------------------
Try before you buy = See our experts in action!
The most comprehensive online learning library for Microsoft developers
is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
Metro Style Apps, more. Free future releases when you subscribe now!
http://p.sf.net/sfu/learndevnow-dev2

Re: [uml-devel] /sys/class/tty/tty0/active?

[richard -rw- weinberger]

On Sat, Jan 28, 2012 at 12:55 AM, richard -rw- weinberger
<richard.weinberger@gmail.com> wrote:
> On Fri, Jan 27, 2012 at 3:02 PM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>> On Fri, 27 Jan 2012 13:04:37 +0100
>> richard -rw- weinberger <richard.weinberger@gmail.com> wrote:
>>
>>> On Fri, Jan 27, 2012 at 12:51 PM, Alan Cox <alan@lxorguk.ukuu.org.uk> wrote:
>>> >> UML's console driver (arch/um/drivers/line.c) implements tty_operations.
>>> >> The crash happens because the tty subsystem calls the driver's close()
>>> >> function and later
>>> >> write_room() or chars_in_buffer().
>>> >>
>>> >> write_room() and chars_in_buffer() fail badly because close() already
>>> >> cleaned up the driver's private data...
>>> >
>>> > You don't want to do that.
>>>
>>> That's what i thought.
>>>
>>> >> Greg, is UML's assumption wrong that after closing the tty no call to
>>> >> write_room() or chars_in_buffer() can happen?
>>> >> I have no idea why systemd is able to trigger this, UML's console
>>> >> driver is old and has always worked quite well.
>>> >
>>> > It's always been untrue but it's even more untrue nowdays. The tty layer
>>> > objects are refcounted, and the code has had significant rewrites. line.c
>>> > hidden away in uml hasn't been updated.
>>> >
>>> > I added a comment about 3 years ago pointing out another older change
>>> > that was needed and that wasn't acted on either..
>>> >
>>> > Take a look at how all the other tty drivers use tty_port, how the ioctls
>>> > have been supposed to work for the past few years and the callback
>>> > changes, then use them.
>>>
>>> Can you recommend a well-written driver?
>>
>> drivers/mmc/card/sdio_uart.c
>>
>> uses just about all the features including handling hotplug and stuff you
>> don't need.
>>
>> drivers/usb/serial/usb-serial.c
>>
>> may also be handy as it provides the interface but then calls into other
>> driver code to do the work.
>>
>> Basically though you want a struct tty_port in your private data, either
>> created at open, or usually more cleanly for the physical port lifetime
>>
>>        tty_port_init()
>>
>> Sets it up, then set the port ops
>>
>>        tty_port_open()
>>        tty_port_close()
>>        tty_port_hangup()
>>
>> do almost all of the rest of the work for you. They call back to your
>> activate and shutdown port methods, they serialize them, they call them
>> on first open/last close in matching pairs.
>>
>> For the tty itself
>>
>>        tty_port_tty_get()
>>
>> gets you a reference to the tty from the port (or NULL) - so handles a
>> close/hangup racing with data arrival
>>
>>        tty_kref_put()
>>
>> releases a reference
>>
>> and
>>        tty->ops->cleanup()
>>
>> is called on the final destruction of the tty object (ie its where you
>> can free tty lifetime data in tty->private_data)
>>
>> So for a simple non pluggable tty it tends to look like
>>
>>        int my_tty_open(struct tty_struct *tty, struct file *filp)
>>        {
>>                tty->driver_data = &my_port;
>>                return tty_port_open(&my_port, tty, filp);
>>        }
>>
>>        void my_tty_close(struct tty_struct *tty, struct file *filp)
>>        {
>>                struct my_tty *m = tty->driver_data;
>>                if (m == NULL)
>>                        return;
>>                tty_port_close(&m->port, tty, filp);
>>        }
>>
>>        void my_tty_hangup(struct tty_struct *tty)
>>        {
>>                struct my_tty *m = tty->driver_data;
>>                tty_port_hangup(&m->port);
>>        }
>>
>> provide the needed callbacks and it'll do the locking and the like for
>> you.
>>
>> On the ioctl side as far as I can see you should simply get rid of the
>> method entirely.
>>
>> For buffer_data you might want to allocate the buffer sanely at open time
>> (tty_port has a function for this too) so it can't fail weirdly
>>
>> And your termios method is a bit odd but makes sense if you are just
>> pretending anything works and is supported.
>
> Thanks for your help!
> tty_port makes sense and I think I understood it.
>
> But after modifying the driver to use tty_port the kernel crashes in
> tty_io.c:__tty_fasync() because filp->f_path.dentry is NULL.
>
> Any idea which kind of error can cause this?

Never mind. :-)

UML's line driver works fine now.
But it needs some more cleanups.
Especially the serial line and management console code.

-- 
Thanks,
//richard