RFC: Serial port DTR/RTS - O_NRESETDEV

RFC: Serial port DTR/RTS - O_NRESETDEV

[H. Peter Anvin]

Hi,

I recently ran into a pretty serious issue due to the Unix/Linux (mis)behavior
of forcing DTR and RTS asserted when a serial port is set, losing the
pre-existing status in the process. Since it is impossible probe for the
current status or even if it is a functional serial port without a file
descriptor, this is very problematic. This came up in the context of probing
for serial ports from an application, so even if termios could be modified
without a file descriptor (which it can't) it would not be safe.

I noted there was a patchset for that on linux-serial from 2022 which
apparently got dropped and never merged, but I think it has a pretty serious
problem: it used a sysfs setting to control the behavior, which may be
reasonable for a default, but at the end of it this is really something that
is determined by the intent of the open() call, just like O_NONBLOCK replaced
the old callout devices we once had.

It seems to me that this may very well be a problem beyond ttys, in which case
a new open flag to request to a driver that the configuration and (observable)
state of the underlying hardware device -- whatever it may be -- should not be
disturbed by calling open(). This is of course already the case for many
devices, not to mention block and non-devices, in which case this flag is a
don't care.

The best name I came up with was O_NRESETDEV, but it's not something I'm
particularly attached to.

If the opinion is that this *doesn't* have a scope beyond ttys, then perhaps
abusing the O_DIRECT flag for this purpose would be an alternative.

Thoughts?

	-hpa

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[Theodore Ts'o]

On Thu, Nov 06, 2025 at 11:53:23PM -0800, H. Peter Anvin wrote:
> 
> I recently ran into a pretty serious issue due to the Unix/Linux
> (mis)behavior of forcing DTR and RTS asserted when a serial port is
> set, losing the pre-existing status in the process.

There's a hidden assumption in your problem statement which is that
DTR / RTS has a "state" which can be saved when the serial port is not
active, where active is one or more file descriptors holding the
serial port open.  There may be certain hardware or drivers where this
is just not possible, because nothing is defined if the serial port is
not active.  It might make sense if you are using a 8250 UART, but not
all the world is the National Semiconductor (or clones) UART.

Certainly the "state" will not be preserved across boots, since how we
autodetect the UART is going to mess with UART settings.  So
*presumably* what you are talking about is you want to be able to open
the serial port, mess with DTR / RTS, and then be able to close the
serial port, and then later on, re-open the serial port, have the DTR
/ RTS remain the same.  And it's Too Hard(tm) to have userspace
keeping a file descriptor open during the whole time?  (Which is
traditionally how Unix/Linux has required that applications do
things.)

Is that a fair summary of the requirements?

> It seems to me that this may very well be a problem beyond ttys, in
> which case a new open flag to request to a driver that the
> configuration and (observable) state of the underlying hardware
> device -- whatever it may be -- should not be disturbed by calling
> open(). This is of course already the case for many devices, not to
> mention block and non-devices, in which case this flag is a don't
> care.

I think it's going to be a lot simpler to keep this specific to serial
ports and DTR / RTS, because the concept that the hardware should not
be changed when the file descriptor is opened may simply not be
possible.  For example, it might be that until you open it, the there
might not even be power applied to the device.  The concept that all
hardware should burn battery power once the machine is booted may not
make sense, and the assumption that hardware has the extra
millicent(s) worth of silicon to maintain state when power is dropped
may again, not be something that we can assume as being possible for
all devices.

If that's the case, if you want to have something where DTR and RTS
stay the same, and for some reason we can't assume that userspace
can't just keep a process holding the tty device open, my suggestion is to use 

Given that DTR and RTS are secial port concepts, my suggesiton is to
set a serial port flag, using setserial(8).  It may be the case that
for certain types of serial device, the attempt to set the flag may be
rejected, but that's something which the ioctl used by setserial
already can do and which userspace applications such as setserial
understand may be the case.

Cheers,

						- Ted

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[H. Peter Anvin]

On 2025-11-07 09:37, Theodore Ts'o wrote:
> On Thu, Nov 06, 2025 at 11:53:23PM -0800, H. Peter Anvin wrote:
>>
>> I recently ran into a pretty serious issue due to the Unix/Linux
>> (mis)behavior of forcing DTR and RTS asserted when a serial port is
>> set, losing the pre-existing status in the process.
> 
> There's a hidden assumption in your problem statement which is that
> DTR / RTS has a "state" which can be saved when the serial port is not
> active, where active is one or more file descriptors holding the
> serial port open.  There may be certain hardware or drivers where this
> is just not possible, because nothing is defined if the serial port is
> not active.  It might make sense if you are using a 8250 UART, but not
> all the world is the National Semiconductor (or clones) UART.
> 
> Certainly the "state" will not be preserved across boots, since how we
> autodetect the UART is going to mess with UART settings.  So
> *presumably* what you are talking about is you want to be able to open
> the serial port, mess with DTR / RTS, and then be able to close the
> serial port, and then later on, re-open the serial port, have the DTR
> / RTS remain the same.  And it's Too Hard(tm) to have userspace
> keeping a file descriptor open during the whole time?  (Which is
> traditionally how Unix/Linux has required that applications do
> things.)
> 
> Is that a fair summary of the requirements?
> 

Not really.

First of all, obviously virtual serial connections that don't fully emulate
RS232/422/485 obviously may have other requirements, however, the ones that
*do* currently have a problem, see for example:

https://lore.kernel.org/linux-serial/20220531043356.8CAB637401A9@freecalypso.org/

RS232 is rarely used these days with its original purpose, modems (except as a
virtual port for things like GSM), but the ubiquitousness of the interface
means it is used for a ton of other things.

The standard ESP32 configuration for its serial port is that asserting RTS#
even for a moment will cause a device reset, and asserting DTR# during reset
forces the device into boot mode. So even if you execute TIOCMSET immediately
after opening the device, you will have glitched the output, and only the
capacitance of the output will save you, in the best case.

The use of RTS# and DTR# as a reset and/or debug mode entry for embedded
devices is, in fact, extremely common; another example is the Atmel single pin
reset/debug interface.

Another example is when the device is connected to an RS485 interface: in that
case asserting RTS# will activate the transmitter, disrupting traffic on the
bus. The kernel will manage RTS# *once it has been configured*, but until the
kernel has been told that the port is used to drive an RS485 port, it has no
way to know.

Furthermore, *even if* the kernel already knew the state and could have
reported it with TIOCMGET, that state is now lost.

It is not correct that the state cannot be maintained across system reboots
(without power loss.) The hardware may or may not allow the state to be read
back (notably, the USB CDC ACM specification, oddly enough, has a
GET_LINE_CODING command but no GET_CONTROL_LINE_STATE) but again, for *those
that can* it should be possible. For those that cannot, there won't be any way
to get valid data to TIOCMGET, but it is still possible to not send a change
command. Either way, the power up state of write-only devices can generally be
assumed to be RTS# and DTR# deasserted, not asserted.

(USB is also a bit special because it is normal for the USB host to power
cycle the device during bus initialization.)

>> It seems to me that this may very well be a problem beyond ttys, in
>> which case a new open flag to request to a driver that the
>> configuration and (observable) state of the underlying hardware
>> device -- whatever it may be -- should not be disturbed by calling
>> open(). This is of course already the case for many devices, not to
>> mention block and non-devices, in which case this flag is a don't
>> care.
> 
> I think it's going to be a lot simpler to keep this specific to serial
> ports and DTR / RTS, because the concept that the hardware should not
> be changed when the file descriptor is opened may simply not be
> possible.  For example, it might be that until you open it, the there
> might not even be power applied to the device.  The concept that all
> hardware should burn battery power once the machine is booted may not
> make sense, and the assumption that hardware has the extra
> millicent(s) worth of silicon to maintain state when power is dropped
> may again, not be something that we can assume as being possible for
> all devices.

This is actually a great example! One should be able to open a file descriptor
to such a device to configure the driver, without needing to power up the
physical hardware.

However, I intentionally defined this as a best-effort control for two reasons:

1. As you say, the hardware may not be able to do it;
2. It will take time until a significant set of drivers can implement this.

> If that's the case, if you want to have something where DTR and RTS
> stay the same, and for some reason we can't assume that userspace
> can't just keep a process holding the tty device open, my suggestion is to use 
> 
> Given that DTR and RTS are secial port concepts, my suggesiton is to
> set a serial port flag, using setserial(8).  It may be the case that
> for certain types of serial device, the attempt to set the flag may be
> rejected, but that's something which the ioctl used by setserial
> already can do and which userspace applications such as setserial
> understand may be the case.

setserial (TIOCSSERIAL) and termios (TCSETS*) both require file descriptors,
so that is not suitable. The 8250 driver, but *not* other serial drivers,
allows the setserial information to be accessed via sysfs; however, this
functionality is local to the 8250 driver.

(Incidentally: the only way to find out the type of a tty driver in the
current Linux kernel is to parse /proc/tty/drivers. This information is
neither available in sysfs nor via ioctl.

Consider the case of a terminal program wanting to display a list of serial
ports. Right now, some serial drivers -- notably the generic UART driver --
will create device nodes for all available minors, exactly so you would be
able to manually attach a device with TIOCSSERIAL. There is no driver-generic
way to find out if there is a hardware device configured other than opening
the device and calling TCGETS or TIOCMGET (depending on exactly what you are
looking for) and see if you get EIO back.

The problem here really isn't the need for a file descriptor -- file
descriptors are The Unix Way[TM] to refer to almost any kind of entity after
all -- but that the act of obtaining the file descriptor -- open -- causes a
direct action as well as loss of existing state.

Now, we obviously can't disable the classical terminal behavior
unconditionally -- that would break a whole lot of perfectly valid code.

Using a sysfs attribute is reasonable on the surface of it (and is what the
patchset linked to above implements) I believe this is the wrong approach,
because it is modal on the device level, and that makes it racy: one program
comes in, flips the attribute, then another program comes in and tries to open
the same device for whatever reason. This opens up at least three possible
race conditions:

- Process 1 sets the nreset bit;
- Process 2 opens the device, not expecting the nreset bit.

- Process 1 reads the nreset bit, trying to be a good citizen;
- Process 1 sets the nreset bit;
- Process 2 reads the nreset bit, ditto;
- ... other stuff happens ...
- Process 1 restores the nreset bit
- Process 2 restores the nreset bit, incorrectly setting it to 1

- Process 1 sets the nreset bit;
- Process 2 sets the nreset bit;
- Process 1 does its work;
- Process 1 clears the nreset bit;
- Process 2 opens the device.


Oh, yes, let's not forget: you need a file descriptor to lock the device for
exclusive use.

This is why I believe this:

1. Needs to be atomic with open(), as opposed to tied to per-device state;

2. Likely can have applications beyond the serial port space, and it would
   be a good idea to have a uniform interface. We can discuss the proper
   behavior for a device which cannot comply; the best way probably would be
   to refuse the open and return an error, which would also happen on older
   kernels without this functionality.

Does this make more sense?

	-hpa

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[Theodore Ts'o]

On Sat, Nov 08, 2025 at 06:25:20PM -0800, H. Peter Anvin wrote:
> 
> The standard ESP32 configuration for its serial port is that asserting RTS#
> even for a moment will cause a device reset, and asserting DTR# during reset
> forces the device into boot mode. So even if you execute TIOCMSET immediately
> after opening the device, you will have glitched the output, and only the
> capacitance of the output will save you, in the best case.

IMHO, these more esoteric use cases should involve a custom kernel
driver which replaces the generic serial driver.  In practice, these
things aren't really a tty, but somethiung else weird, and trying to
do this in userspace seems really awkward.

> setserial (TIOCSSERIAL) and termios (TCSETS*) both require file descriptors,
> so that is not suitable. The 8250 driver, but *not* other serial drivers,
> allows the setserial information to be accessed via sysfs; however, this
> functionality is local to the 8250 driver.

My suggestion of using setserial to turn on some "not really a tty;
but some weird networking / cheap debugging hack" flag should work,
because you would do this at boot up.  Note that the 8250
autoconfiguration code (see drivers/tty/serial/8250/8250_port.c) is
going to mess with DTR / RTS.  This is why I asserted that trying to
claim that you can preserve "state" across reboots is Just Not
Possible.

If you have some weird setup where DTR or RTS is wierd to the
"detonate the TNT" line, might I suggest that maybe we shouldn't be
using the tty / 8250 serial driver, but it should ***really*** be a
dedicated kernel driver?

					- Ted

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[H. Peter Anvin]

On November 9, 2025 7:35:56 PM PST, Theodore Ts'o <tytso@mit.edu> wrote:
>On Sat, Nov 08, 2025 at 06:25:20PM -0800, H. Peter Anvin wrote:
>> 
>> The standard ESP32 configuration for its serial port is that asserting RTS#
>> even for a moment will cause a device reset, and asserting DTR# during reset
>> forces the device into boot mode. So even if you execute TIOCMSET immediately
>> after opening the device, you will have glitched the output, and only the
>> capacitance of the output will save you, in the best case.
>
>IMHO, these more esoteric use cases should involve a custom kernel
>driver which replaces the generic serial driver.  In practice, these
>things aren't really a tty, but somethiung else weird, and trying to
>do this in userspace seems really awkward.
>
>> setserial (TIOCSSERIAL) and termios (TCSETS*) both require file descriptors,
>> so that is not suitable. The 8250 driver, but *not* other serial drivers,
>> allows the setserial information to be accessed via sysfs; however, this
>> functionality is local to the 8250 driver.
>
>My suggestion of using setserial to turn on some "not really a tty;
>but some weird networking / cheap debugging hack" flag should work,
>because you would do this at boot up.  Note that the 8250
>autoconfiguration code (see drivers/tty/serial/8250/8250_port.c) is
>going to mess with DTR / RTS.  This is why I asserted that trying to
>claim that you can preserve "state" across reboots is Just Not
>Possible.
>
>If you have some weird setup where DTR or RTS is wierd to the
>"detonate the TNT" line, might I suggest that maybe we shouldn't be
>using the tty / 8250 serial driver, but it should ***really*** be a
>dedicated kernel driver?
>
>					- Ted

That is a completely unrealistic idea. And you are hardly the first one to have it. Microsoft has been trying to get rid of serial and parallel ports since the 1990s for reasons like this. 

Microsoft even have had to back off the requirement of having .ini text file "drivers" for ACM serial ports 

Yet they probably will still be with us when the 22nd century dawns, exactly *because* they are ubiquitous, supported by everything, and require no separate kernel drivers.

And these days these aren't the "esoteric" use cases at all. They are the norm.

RE: RFC: Serial port DTR/RTS - O_NRESETDEV

[Maarten Brock]

> -----Original Message-----
> From: H. Peter Anvin <hpa@zytor.com>
> Sent: Monday, 10 November 2025 06:00
> To: Theodore Ts'o <tytso@mit.edu>
> Cc: linux-serial@vger.kernel.org; linux-api@vger.kernel.org; LKML <linux-
> kernel@vger.kernel.org>
> Subject: Re: RFC: Serial port DTR/RTS - O_NRESETDEV
> 
> On November 9, 2025 7:35:56 PM PST, Theodore Ts'o <tytso@mit.edu> wrote:
> >On Sat, Nov 08, 2025 at 06:25:20PM -0800, H. Peter Anvin wrote:
> >>
> >> The standard ESP32 configuration for its serial port is that asserting RTS#
> >> even for a moment will cause a device reset, and asserting DTR# during reset
> >> forces the device into boot mode. So even if you execute TIOCMSET
> immediately
> >> after opening the device, you will have glitched the output, and only the
> >> capacitance of the output will save you, in the best case.
> >
> >IMHO, these more esoteric use cases should involve a custom kernel
> >driver which replaces the generic serial driver.  In practice, these
> >things aren't really a tty, but somethiung else weird, and trying to
> >do this in userspace seems really awkward.
> >
> >> setserial (TIOCSSERIAL) and termios (TCSETS*) both require file descriptors,
> >> so that is not suitable. The 8250 driver, but *not* other serial drivers,
> >> allows the setserial information to be accessed via sysfs; however, this
> >> functionality is local to the 8250 driver.
> >
> >My suggestion of using setserial to turn on some "not really a tty;
> >but some weird networking / cheap debugging hack" flag should work,
> >because you would do this at boot up.  Note that the 8250
> >autoconfiguration code (see drivers/tty/serial/8250/8250_port.c) is
> >going to mess with DTR / RTS.  This is why I asserted that trying to
> >claim that you can preserve "state" across reboots is Just Not
> >Possible.
> >
> >If you have some weird setup where DTR or RTS is wierd to the
> >"detonate the TNT" line, might I suggest that maybe we shouldn't be
> >using the tty / 8250 serial driver, but it should ***really*** be a
> >dedicated kernel driver?
> >
> >					- Ted
> 
> That is a completely unrealistic idea. And you are hardly the first one to have it.
> Microsoft has been trying to get rid of serial and parallel ports since the 1990s for
> reasons like this.
> 
> Microsoft even have had to back off the requirement of having .ini text file
> "drivers" for ACM serial ports
> 
> Yet they probably will still be with us when the 22nd century dawns, exactly
> *because* they are ubiquitous, supported by everything, and require no separate
> kernel drivers.
> 
> And these days these aren't the "esoteric" use cases at all. They are the norm.

I fully agree that you cannot expect users that wired something like RS485 Driver
Enable or a microcontroller reset to RTS or DTR to write their own kernel driver.
And you need to open the port to make the appropriate settings. But opening a
port should not e.g. claim the RS485 bus and mess up whatever communication
was going on there.

Kind regards,
Maarten Brock

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[Theodore Ts'o]

On Mon, Nov 10, 2025 at 10:06:02AM +0000, Maarten Brock wrote:
> I fully agree that you cannot expect users that wired something like RS485 Driver
> Enable or a microcontroller reset to RTS or DTR to write their own kernel driver.
> And you need to open the port to make the appropriate settings. But opening a
> port should not e.g. claim the RS485 bus and mess up whatever communication
> was going on there.

Again, the existing seral driver code *will* mess with RTS and DTR at
boot up because that's part of the autoconfiuration code, and that was
added because it was needed for some number of serial ports.

If that's going to "mess up" the RS485 bus, maybe we need accept that
RS-232 != RS-485 and have a different driver for the two.  That's
going to be a lot simpler than trying to make the same code work for
both RS-232 and RS-485, and claiming that the existing RS-232 code is
"fundamentally buggy" when it interacts poorly with something that has
very different requirements than the historical RS-232 use cases.

              	  	     	    - Ted

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[H. Peter Anvin]

On November 10, 2025 12:19:33 PM PST, Theodore Ts'o <tytso@mit.edu> wrote:
>On Mon, Nov 10, 2025 at 10:06:02AM +0000, Maarten Brock wrote:
>> I fully agree that you cannot expect users that wired something like RS485 Driver
>> Enable or a microcontroller reset to RTS or DTR to write their own kernel driver.
>> And you need to open the port to make the appropriate settings. But opening a
>> port should not e.g. claim the RS485 bus and mess up whatever communication
>> was going on there.
>
>Again, the existing seral driver code *will* mess with RTS and DTR at
>boot up because that's part of the autoconfiuration code, and that was
>added because it was needed for some number of serial ports.
>
>If that's going to "mess up" the RS485 bus, maybe we need accept that
>RS-232 != RS-485 and have a different driver for the two.  That's
>going to be a lot simpler than trying to make the same code work for
>both RS-232 and RS-485, and claiming that the existing RS-232 code is
>"fundamentally buggy" when it interacts poorly with something that has
>very different requirements than the historical RS-232 use cases.
>
>              	  	     	    - Ted

I didn't say it was fundamentally buggy; if I did or implied it I apologize. It is most definitely not; however, in some cases it is undesired or undesirable *due to shifts in usage patterns.*

This isn't a bug at all but an enormous strength. That a 65-year-old standard — both hardware and software — designed for teletypes and dumb terminals can still be useful today is almost the definition of success. And, yes, some glitches in that process are going to be inevitable — like the mistake of retaining the termio Bxxx emumeration constants in the termios interface. But we deal with it by gradual evolution of interfaces. 

One such example is RTS itself: the RS485 definition is, in fact, the originally intended meaning of RTS: it is a request to the DCE to negotiate transmission privilege and activate transmission mode over a half duplex channel, after which it asserts CTS. RTS/CTS flow control was a nonstandard adaption to allow for binary transparent flow control over full duplex links — it wasn't formally standardized until 1991, and the signal is formally named RTR when used that way. However, RTR and RTS share hardware in nearly all existing implementations, and share pins in the standard — so whether or not you are using RTR or RTS is a property of the DCE, not DTE, and needs to be configured into the DTE.

Requiring new drivers for the gajillion different hardware devices already supported and then having the problem of which drivers claim it isn't really any better of a solution; one could in fact argue it is *exactly* equivalent to being able to indicate to the driver what mode one wants it to operate in before it does its configuration.

The parport driver layer is kind of similar to this, in some ways, but in the tty layer that is mostly handled by line disciplines instead. (The parport hardware was generally abused on a much lower level, as a substitute for GPIOs, so even the notion of a byte stream wasn't there.)

*If* I'm reading the code correctly – which is a little complicated due to the sheer number of abstraction layers – hardware initialization is already deferred until first open, which would mean that disabling autoconfiguration (one of the features in TIOCSSERIAL) would again be a valid reason for wanting to be able to communicate with a device driver before requiring that it puts the underlying hardware in the state expected for operation *in the mode configured* (catch-22).

As I stated, this is inherently going to be a best effort. For some devices that may mean simply leaving the power on default in place (in this specific case, presumably, DTR# and RTS# deasserted.) However, once the state is already known to the kernel then there is no such issue for any hardware. 

As far as naming is concerned: O_RAW is really suboptimal, as it has exactly the same implications as O_DIRECT (I/O goes straight to the device with no buffering.) I don't like the idea of abusing O_DIRECT at all; I only brought it up as a fallback alternative. I genuinely do believe that if we assign a new open flag it will find use cases outside the tty/serial port subsystems, and if there is anything Unix has done right it is to generalize interfaces as much as possible, case in point descriptors. 

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[Theodore Ts'o]

On Mon, Nov 10, 2025 at 01:05:55PM -0800, H. Peter Anvin wrote:
> 
> The parport driver layer is kind of similar to this, in some ways,
> but in the tty layer that is mostly handled by line disciplines
> instead. (The parport hardware was generally abused on a much lower
> level, as a substitute for GPIOs, so even the notion of a byte
> stream wasn't there.)

I'm not an RS-485 expert, but over the years, I've heard all sorts of
"interesting" timing requirements.  For example RTS can only be
dropped when the UART's shift register has completely drained.  That's
not when the OS has sent the characters to the FIFO (which is why
tcdrain isn't quite what you want); it's not when the UART has sent
the transmit interrupt, but when the UART's shift register is *empty*.
Doing this via the termios interface is decidedly non-optimal[1].

[1] https://www.moxa.com.cn/getmedia/a07808dd-f3d7-473b-9a2f-93b5ce673bb1/moxa-the-secrets-of-rs-485-half-duplex-communication-tech-note-v1.0.pdf

It *can* be done, sure, with some strategic usleep()'s, but not
necessarily optimally, and it is decidedly hacky.  From what I can
tell, it's actually not *that* different from treating RTS as a GPIO
pin, and really, this ought to be done in the kernel if you want
optimal control oer the timing of when RTS gets toggeled.

> *If* I'm reading the code correctly – which is a little complicated due to the sheer number of abstraction layers – hardware initialization is already deferred until first open, which would mean that disabling autoconfiguration (one of the features in TIOCSSERIAL) would again be a valid reason for wanting to be able to communicate with a device driver before requiring that it puts the underlying hardware in the state expected for operation *in the mode configured* (catch-22).

Well, part of the problem is that the PC's serial port predates PCI,
so there's no way we can tell whether or not there is a serial port at
a particular I/O port except by poking certain I/O ports, and seeing
if there is something that looks like a UART which responds.
Hopefully this won't accidentaly cause the NSA's thermite-charge
self-destruct charge from getting set off, and in practice, mainboard
designers who try to put things at the COM1/2/3/4 generally get weeded
out thanks to natural selection, hopefully without harming users along
the way.  :-)

Worse, we also wanted to support serial cards that were at
non-standard ports, or using non-standard interrupts, and so that's
one of the reason hardware initialization is deferred until after we
have a chance to configure the serial device's I/O port and interrupt.

Now, we are in the 21st century, and just as we are trying to declare
32-bit i386 dead (to the disappointment of legacy hardware owners
everywhere who are whining about Debian dropping installer support for
i386), we could try to declare the ISA bus as dead, and no longer
being supported, and we could probably drop a bunch of this
complexity.  We probably would need to support COM 1/2/3/4 ports since
these still tend to be hardwared without a way of detecting it their
existing via USB or PCI bus discovery mechanisms.  And for those, we
would still need to do UART autoconfiguration.  I suppose we could
just assume that all UART's are 16550A's, and that noone would
actually use older UART's type --- except (a) I bet there are some
cheap-skate embedded computer designers who decided to use a 8250
instead of 16550A for $REASONS$, and because of the extreme timing
requirements of some RS-485 use cases, I believe I have seen
recommendations to use setserial to actually force the UART into 8250
mode, to disable the TX FIFO --- all to satisfying the weird-shit RTS
timing requirements.  (This is where I get my beief that RS-485 !=
RS-232, and if you want to do all of this weird-shit timing
requirements because you are trying to implemet a multi-node bus which
is half-duplex, maybe this should be done in the kernel, damn it!)


How about this?  If you really don't want to open the device to
configure it not to assert DTR/RTS, perhaps we could specify on the
boot command line that one of TTYS[0123] has a different default line
discipline, and when tty's are opened when the RS-485 line displine,
the RS-485 rules apply, which would include not messing with DTR/RTS,
and if you want to play games with RTS getting dropped as soon as the
last bit has let the transmit shift register, we can get the kernel do
this in the line discpline, which arguably would be *way* more
reliable than playing GPIO-style timing games in userspace.

	      	      		 	      - Ted

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[H. Peter Anvin]

On November 10, 2025 7:51:43 PM PST, Theodore Ts'o <tytso@mit.edu> wrote:
>On Mon, Nov 10, 2025 at 01:05:55PM -0800, H. Peter Anvin wrote:
>> 
>> The parport driver layer is kind of similar to this, in some ways,
>> but in the tty layer that is mostly handled by line disciplines
>> instead. (The parport hardware was generally abused on a much lower
>> level, as a substitute for GPIOs, so even the notion of a byte
>> stream wasn't there.)
>
>I'm not an RS-485 expert, but over the years, I've heard all sorts of
>"interesting" timing requirements.  For example RTS can only be
>dropped when the UART's shift register has completely drained.  That's
>not when the OS has sent the characters to the FIFO (which is why
>tcdrain isn't quite what you want); it's not when the UART has sent
>the transmit interrupt, but when the UART's shift register is *empty*.
>Doing this via the termios interface is decidedly non-optimal[1].
>
>[1] https://www.moxa.com.cn/getmedia/a07808dd-f3d7-473b-9a2f-93b5ce673bb1/moxa-the-secrets-of-rs-485-half-duplex-communication-tech-note-v1.0.pdf
>
>It *can* be done, sure, with some strategic usleep()'s, but not
>necessarily optimally, and it is decidedly hacky.  From what I can
>tell, it's actually not *that* different from treating RTS as a GPIO
>pin, and really, this ought to be done in the kernel if you want
>optimal control oer the timing of when RTS gets toggeled.
>
>> *If* I'm reading the code correctly – which is a little complicated due to the sheer number of abstraction layers – hardware initialization is already deferred until first open, which would mean that disabling autoconfiguration (one of the features in TIOCSSERIAL) would again be a valid reason for wanting to be able to communicate with a device driver before requiring that it puts the underlying hardware in the state expected for operation *in the mode configured* (catch-22).
>
>Well, part of the problem is that the PC's serial port predates PCI,
>so there's no way we can tell whether or not there is a serial port at
>a particular I/O port except by poking certain I/O ports, and seeing
>if there is something that looks like a UART which responds.
>Hopefully this won't accidentaly cause the NSA's thermite-charge
>self-destruct charge from getting set off, and in practice, mainboard
>designers who try to put things at the COM1/2/3/4 generally get weeded
>out thanks to natural selection, hopefully without harming users along
>the way.  :-)
>
>Worse, we also wanted to support serial cards that were at
>non-standard ports, or using non-standard interrupts, and so that's
>one of the reason hardware initialization is deferred until after we
>have a chance to configure the serial device's I/O port and interrupt.
>
>Now, we are in the 21st century, and just as we are trying to declare
>32-bit i386 dead (to the disappointment of legacy hardware owners
>everywhere who are whining about Debian dropping installer support for
>i386), we could try to declare the ISA bus as dead, and no longer
>being supported, and we could probably drop a bunch of this
>complexity.  We probably would need to support COM 1/2/3/4 ports since
>these still tend to be hardwared without a way of detecting it their
>existing via USB or PCI bus discovery mechanisms.  And for those, we
>would still need to do UART autoconfiguration.  I suppose we could
>just assume that all UART's are 16550A's, and that noone would
>actually use older UART's type --- except (a) I bet there are some
>cheap-skate embedded computer designers who decided to use a 8250
>instead of 16550A for $REASONS$, and because of the extreme timing
>requirements of some RS-485 use cases, I believe I have seen
>recommendations to use setserial to actually force the UART into 8250
>mode, to disable the TX FIFO --- all to satisfying the weird-shit RTS
>timing requirements.  (This is where I get my beief that RS-485 !=
>RS-232, and if you want to do all of this weird-shit timing
>requirements because you are trying to implemet a multi-node bus which
>is half-duplex, maybe this should be done in the kernel, damn it!)
>
>
>How about this?  If you really don't want to open the device to
>configure it not to assert DTR/RTS, perhaps we could specify on the
>boot command line that one of TTYS[0123] has a different default line
>discipline, and when tty's are opened when the RS-485 line displine,
>the RS-485 rules apply, which would include not messing with DTR/RTS,
>and if you want to play games with RTS getting dropped as soon as the
>last bit has let the transmit shift register, we can get the kernel do
>this in the line discpline, which arguably would be *way* more
>reliable than playing GPIO-style timing games in userspace.
>
>	      	      		 	      - Ted

I really think you are looking at this from a very odd point of view, and you seem to be very inconsistent. Boot time setup? Isn't that what setserial is for? We have the ability to feed this configuration already, but you need a file descriptor. 

Honestly, though, I'm far less interested in what 8250-based hardware does than e.g. USB.

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[Theodore Ts'o]

On Mon, Nov 10, 2025 at 07:57:22PM -0800, H. Peter Anvin wrote:
> I really think you are looking at this from a very odd point of
> view, and you seem to be very inconsistent. Boot time setup? Isn't
> that what setserial is for? We have the ability to feed this
> configuration already, but you need a file descriptor.

I'm not really fond of adding some new open flag that to me seems
**very** serial / RS-485 specific, and so I'm trying to find some
way to avoid it.

I also think that that the GPIO style timing requirements of RTS
**really** should be done as a line discpline, and not in userspace.

> Honestly, though, I'm far less interested in what 8250-based hardware does than e.g. USB.

I'm quite confident that USB won't have "state" that will be preserved
across a reboot, because the device won't even get powered up until
the USB device is attached.  And part of the problem was that the
requirements weren't particularly clear, and given the insistence that
the "state" be preserved even across reboot, despite the serial port
autoconfiguration, I had assumed you were posting uing the COM 1/2/3/4
ports where autoconfiguration isn't stricty speaking necessary.

In some ways, USB ports might be easier, since it should be possible
to specify udev rules which get passed to the driver when the USB
serial device is inserted, and so *that* can easily be done without
needing a file descriptor.

And for this sort of thing, it seems perfectly fair to hard code some
specific behavior using either a boot command line or a udev rule,
since you seem to be positing that the serial port will be dedicated
to some kind of weird-shit RS-485 bus device, where any time RTS/DTR
gets raised, the bus will malfunction in weird and wondrous ways....

     	     	     	  	      	 - Ted

RE: RFC: Serial port DTR/RTS - O_NRESETDEV

[Maarten Brock]

Ted,

I don't understand why you drag userspace into this discussion.
The kernel drivers already support RS-485 mode for many/most
UARTs including the best timing they can give.

The reason I brought up RS-485 is because of the following situation:
Out of reset the UART has RTS# inactive. Any decent hardware design
will turn this into RS-485 Driver Enable (DE) inactive with the result that
it does not inadvertently claim the bus unnecessarily.

Now some userspace program wants to open the tty to start listening
and maybe later also to start transmitting. Opening the tty will activate
the RTS# line and thus DE and thereby claiming the bus. Any ongoing
communication will get disrupted. How's that for good behaviour?
Only after opening the port the userspace program can use the file
descriptor to configure the tty to use RS-485 mode and since it is not
transmitting it will now deactivate RTS# and DE again. But the possible
damage has been done.

Luckily if one is using a device tree one can already tell some drivers
that they need RS-485 mode and the glitch can be prevented.

But a flag to open a tty without touching the RTS# and DTR# lines would
bring the option of configuring the driver in RS-485 mode in a safe way.

And another reason why you should not handle this in a new driver is that
there are many RS-232 <-> RS-485 converters. Do you really expect users
to compile and install a new kernel just because they plugged in or out
some such converter?

Maarten

> -----Original Message-----
> From: Theodore Ts'o <tytso@mit.edu>
> Sent: Tuesday, 11 November 2025 05:38
> To: H. Peter Anvin <hpa@zytor.com>
> Cc: Maarten Brock <Maarten.Brock@sttls.nl>; linux-serial@vger.kernel.org; linux-
> api@vger.kernel.org; LKML <linux-kernel@vger.kernel.org>
> Subject: Re: RFC: Serial port DTR/RTS - O_NRESETDEV
> 
> On Mon, Nov 10, 2025 at 07:57:22PM -0800, H. Peter Anvin wrote:
> > I really think you are looking at this from a very odd point of
> > view, and you seem to be very inconsistent. Boot time setup? Isn't
> > that what setserial is for? We have the ability to feed this
> > configuration already, but you need a file descriptor.
> 
> I'm not really fond of adding some new open flag that to me seems
> **very** serial / RS-485 specific, and so I'm trying to find some
> way to avoid it.
> 
> I also think that that the GPIO style timing requirements of RTS
> **really** should be done as a line discpline, and not in userspace.
> 
> > Honestly, though, I'm far less interested in what 8250-based hardware does than
> e.g. USB.
> 
> I'm quite confident that USB won't have "state" that will be preserved
> across a reboot, because the device won't even get powered up until
> the USB device is attached.  And part of the problem was that the
> requirements weren't particularly clear, and given the insistence that
> the "state" be preserved even across reboot, despite the serial port
> autoconfiguration, I had assumed you were posting uing the COM 1/2/3/4
> ports where autoconfiguration isn't stricty speaking necessary.
> 
> In some ways, USB ports might be easier, since it should be possible
> to specify udev rules which get passed to the driver when the USB
> serial device is inserted, and so *that* can easily be done without
> needing a file descriptor.
> 
> And for this sort of thing, it seems perfectly fair to hard code some
> specific behavior using either a boot command line or a udev rule,
> since you seem to be positing that the serial port will be dedicated
> to some kind of weird-shit RS-485 bus device, where any time RTS/DTR
> gets raised, the bus will malfunction in weird and wondrous ways....
> 
>      	     	     	  	      	 - Ted

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[H. Peter Anvin]

On 2025-11-10 20:38, Theodore Ts'o wrote:
> On Mon, Nov 10, 2025 at 07:57:22PM -0800, H. Peter Anvin wrote:
>> I really think you are looking at this from a very odd point of
>> view, and you seem to be very inconsistent. Boot time setup? Isn't
>> that what setserial is for? We have the ability to feed this
>> configuration already, but you need a file descriptor.
> 
> I'm not really fond of adding some new open flag that to me seems
> **very** serial / RS-485 specific, and so I'm trying to find some
> way to avoid it.
> 

I don't think it is.  "Opening this device for configuration."

> I also think that that the GPIO style timing requirements of RTS
> **really** should be done as a line discpline, and not in userspace.
> 

No disagreement there -- and so it is. What I want to do is a way to *attach*
that line discipline without poking with the serial port itself.  That's what
I keep trying to get at.

>> Honestly, though, I'm far less interested in what 8250-based hardware does than e.g. USB.
> 
> I'm quite confident that USB won't have "state" that will be preserved
> across a reboot, because the device won't even get powered up until
> the USB device is attached.  And part of the problem was that the
> requirements weren't particularly clear, and given the insistence that
> the "state" be preserved even across reboot, despite the serial port
> autoconfiguration, I had assumed you were posting uing the COM 1/2/3/4
> ports where autoconfiguration isn't stricty speaking necessary.
> 
> In some ways, USB ports might be easier, since it should be possible
> to specify udev rules which get passed to the driver when the USB
> serial device is inserted, and so *that* can easily be done without
> needing a file descriptor.
> 
> And for this sort of thing, it seems perfectly fair to hard code some
> specific behavior using either a boot command line or a udev rule,
> since you seem to be positing that the serial port will be dedicated
> to some kind of weird-shit RS-485 bus device, where any time RTS/DTR
> gets raised, the bus will malfunction in weird and wondrous ways....

But again, it is very much a configuration property.  You don't know where
your dynamically assigned serial port will end up -- and you *can't*, because
it is a property of the DCE -- what is plugged *into* the device.

Now you have someone writing a terminal program or something like Arduino and
decide to enumerate serial ports (which, as I stated, you can't actually do
right now without opening the devices).  This is why it makes sense for the
open() caller to declare intent; this is similar to how O_NDELAY replaced
callout devices.

It would be lovely if we could do something like
open("/dev/ttyS0/option-string") and so on, but that is well and truly a far
bigger change to the whole driver API.

	-hpa

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[Greg KH]

On Mon, Nov 10, 2025 at 07:57:22PM -0800, H. Peter Anvin wrote:
> Honestly, though, I'm far less interested in what 8250-based hardware does than e.g. USB.

hahahahahahaha {snort}

Hah.  that's a good one.

Oh, you aren't kidding.

Wow, good luck with this.  USB-serial adaptors are all over the place,
some have real uarts in them (and so do buffering in the device, and
line handling in odd ways when powered up), and some are almost just a
straight pipe through to the USB host with control line handling ideas
tacked on to the side as an afterthought, if at all.

There is no standard here, they all work differently, and even work
differently across the same device type with just barely enough hints
for us to determine what is going on.

So don't worry about USB, if you throw that into the mix, all bets are
off and you should NEVER rely on that.

Remeber USB->serial was explicitly rejected by the USB standard group,
only to have it come back in the "side door" through the spec process
when it turned out that Microsoft hated having to write a zillion
different vendor-specific drivers because the vendor provided ones kept
crashing user's machines.  So what we ended up with was "just enough" to
make it through the spec process, and even then line signals are
probably never tested so you can't rely on them.

good luck!

greg "this brought up too many bad memories" k-h

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[H. Peter Anvin]

On November 12, 2025 3:22:56 AM PST, Greg KH <gregkh@linuxfoundation.org> wrote:
>On Mon, Nov 10, 2025 at 07:57:22PM -0800, H. Peter Anvin wrote:
>> Honestly, though, I'm far less interested in what 8250-based hardware does than e.g. USB.
>
>hahahahahahaha {snort}
>
>Hah.  that's a good one.
>
>Oh, you aren't kidding.
>
>Wow, good luck with this.  USB-serial adaptors are all over the place,
>some have real uarts in them (and so do buffering in the device, and
>line handling in odd ways when powered up), and some are almost just a
>straight pipe through to the USB host with control line handling ideas
>tacked on to the side as an afterthought, if at all.
>
>There is no standard here, they all work differently, and even work
>differently across the same device type with just barely enough hints
>for us to determine what is going on.
>
>So don't worry about USB, if you throw that into the mix, all bets are
>off and you should NEVER rely on that.
>
>Remeber USB->serial was explicitly rejected by the USB standard group,
>only to have it come back in the "side door" through the spec process
>when it turned out that Microsoft hated having to write a zillion
>different vendor-specific drivers because the vendor provided ones kept
>crashing user's machines.  So what we ended up with was "just enough" to
>make it through the spec process, and even then line signals are
>probably never tested so you can't rely on them.
>
>good luck!
>
>greg "this brought up too many bad memories" k-h

Ugh.

I have made it very clear that I am very aware that there is broken hardware. 

What I'm trying to do is to deal with the (occasional) case of *non*-broken hardware. Right now Linux breaks the non-broken hardware for it, and I don't think the existence of broken hardware is a good justification for that.

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[Greg KH]

On Wed, Nov 12, 2025 at 08:09:45AM -0800, H. Peter Anvin wrote:
> On November 12, 2025 3:22:56 AM PST, Greg KH <gregkh@linuxfoundation.org> wrote:
> >On Mon, Nov 10, 2025 at 07:57:22PM -0800, H. Peter Anvin wrote:
> >> Honestly, though, I'm far less interested in what 8250-based hardware does than e.g. USB.
> >
> >hahahahahahaha {snort}
> >
> >Hah.  that's a good one.
> >
> >Oh, you aren't kidding.
> >
> >Wow, good luck with this.  USB-serial adaptors are all over the place,
> >some have real uarts in them (and so do buffering in the device, and
> >line handling in odd ways when powered up), and some are almost just a
> >straight pipe through to the USB host with control line handling ideas
> >tacked on to the side as an afterthought, if at all.
> >
> >There is no standard here, they all work differently, and even work
> >differently across the same device type with just barely enough hints
> >for us to determine what is going on.
> >
> >So don't worry about USB, if you throw that into the mix, all bets are
> >off and you should NEVER rely on that.
> >
> >Remeber USB->serial was explicitly rejected by the USB standard group,
> >only to have it come back in the "side door" through the spec process
> >when it turned out that Microsoft hated having to write a zillion
> >different vendor-specific drivers because the vendor provided ones kept
> >crashing user's machines.  So what we ended up with was "just enough" to
> >make it through the spec process, and even then line signals are
> >probably never tested so you can't rely on them.
> >
> >good luck!
> >
> >greg "this brought up too many bad memories" k-h
> 
> Ugh.
> 
> I have made it very clear that I am very aware that there is broken hardware. 

I would posit that there is NO "non-broken" usb->serial devices out
there.  The closest I have seen was the old IO-Edgeport devices, but
they were expensive and got bought out by some other company and in the
end didn't succeed due to all of the "cheap" devices/chips out there
that just did dumb tx/rx transfers over a fake serial connection.

> What I'm trying to do is to deal with the (occasional) case of
> *non*-broken hardware. Right now Linux breaks the non-broken hardware
> for it, and I don't think the existence of broken hardware is a good
> justification for that.

No, but we have to handle both somehow.

And given that we still get brand-new UART drivers sent to use every few
months, there is just more and more "broken" hardware out there overall.

Anyway, good luck coming up with a scheme to handle your crazy
connections, I would push back and say "any device that treats a serial
control line as a power signal is broken to start with" :)

greg k-h

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[H. Peter Anvin]

On November 12, 2025 8:46:50 AM PST, Greg KH <gregkh@linuxfoundation.org> wrote:
>On Wed, Nov 12, 2025 at 08:09:45AM -0800, H. Peter Anvin wrote:
>> On November 12, 2025 3:22:56 AM PST, Greg KH <gregkh@linuxfoundation.org> wrote:
>> >On Mon, Nov 10, 2025 at 07:57:22PM -0800, H. Peter Anvin wrote:
>> >> Honestly, though, I'm far less interested in what 8250-based hardware does than e.g. USB.
>> >
>> >hahahahahahaha {snort}
>> >
>> >Hah.  that's a good one.
>> >
>> >Oh, you aren't kidding.
>> >
>> >Wow, good luck with this.  USB-serial adaptors are all over the place,
>> >some have real uarts in them (and so do buffering in the device, and
>> >line handling in odd ways when powered up), and some are almost just a
>> >straight pipe through to the USB host with control line handling ideas
>> >tacked on to the side as an afterthought, if at all.
>> >
>> >There is no standard here, they all work differently, and even work
>> >differently across the same device type with just barely enough hints
>> >for us to determine what is going on.
>> >
>> >So don't worry about USB, if you throw that into the mix, all bets are
>> >off and you should NEVER rely on that.
>> >
>> >Remeber USB->serial was explicitly rejected by the USB standard group,
>> >only to have it come back in the "side door" through the spec process
>> >when it turned out that Microsoft hated having to write a zillion
>> >different vendor-specific drivers because the vendor provided ones kept
>> >crashing user's machines.  So what we ended up with was "just enough" to
>> >make it through the spec process, and even then line signals are
>> >probably never tested so you can't rely on them.
>> >
>> >good luck!
>> >
>> >greg "this brought up too many bad memories" k-h
>> 
>> Ugh.
>> 
>> I have made it very clear that I am very aware that there is broken hardware. 
>
>I would posit that there is NO "non-broken" usb->serial devices out
>there.  The closest I have seen was the old IO-Edgeport devices, but
>they were expensive and got bought out by some other company and in the
>end didn't succeed due to all of the "cheap" devices/chips out there
>that just did dumb tx/rx transfers over a fake serial connection.
>
>> What I'm trying to do is to deal with the (occasional) case of
>> *non*-broken hardware. Right now Linux breaks the non-broken hardware
>> for it, and I don't think the existence of broken hardware is a good
>> justification for that.
>
>No, but we have to handle both somehow.
>
>And given that we still get brand-new UART drivers sent to use every few
>months, there is just more and more "broken" hardware out there overall.
>
>Anyway, good luck coming up with a scheme to handle your crazy
>connections, I would push back and say "any device that treats a serial
>control line as a power signal is broken to start with" :)
>
>greg k-h

Yeah, well, I will certainly *not* argue with that one! Quite the contrary... *shudder*.

There are enough facepalms to go around, both on the DTE and DCE sides, and I'm not in any shape, way, or form denying that.

Nor am I trying to boil the ocean here. I'm just trying to figure out how to make the situation a bit more flexible to try to at least reduce the amount of brokenness we throw back in the face of the user. 

The reason I brought up USB is that while RTS# briefly glitching on an actual RS232 line is unlikely to actually make it through the driver, which will have a cutoff of at most 10 MHz and usually much less, an ACM device receiving a SET_LINE_CONTROL message may react to it immediately, especially if it is an emulated port.(*)

The same thing is true, of course, for "uart" lines, a.k.a. TTL or CMOS level serial, which have much higher bandwidth than any physical RS232 or RS485 drivers.

Incidentally, I'm not looking at this because of a huge need on my own part; I'm doing it because it irked me to no end that glibc still hadn't implemented support for the almost two decades old support in the Linux kernel for arbitrary serial port speeds, and in the end I ended up Just Writing The Code; but there as well I ended up having several discussions with the glibc maintainers about how to deal with the unavoidable compromises involved in evolving one l the interfaces; the POSIX termios interface design from back in the '80s really caused some serious headaches. 

There was fallout, as it exposed bugs in a bunch of software which had implemented their own hacks during the 17 years that glibc didn't provide any support for handling this. It gave me a *lot* of new insights in how various applications in the field actually do things and what they have to do to work around limitations in both hardware and software at the moment, and so I'm feeling kind of motivated to try to make these real life use cases a little bit less obnoxious.

The thing with RTS# and DTR# came up when I started rooting around in gtkterm's serial code probing code, but I had myself run into it using ESP32 modules.

Had I personally designed the ESP32 interface I would have used BREAK to pulse reset instead of RTS#, but that would have required a capacitor and a diode, and omg that would have added *cost!*

(This is in fact exactly what I did when I implemented my own ACM device in an FPGA, but that's a different story.)

As far as actual USB-to-serial devices are concerned, my *personal* experience with quite a few of them is that the line control signals are generally quite reliable, and transmitting BREAK usually works OK; however, whether *receiving* BREAK works is a crapshoot at the very best.

Then there are of course the ones that just lock up randomly, or seriously glitch on the USB side, but that's an entirely different kettle of fish.

And you are most definitely right that not standardizing a USB to serial device class from the very beginning was a very bad mistake. At least now newer devices tend toward using ACM and advertise as "driverless". It also allows the rest of the USB descriptor to contain more useful information about the DCE, assuming it is a device that is physically bound to or integrated in the DCE (virtual.)

I'm not saying you and Ted are *wrong*; you are most certainly not. What I'm hoping for is a bit of pragmatism that would make at least some users' lives a little easier.

None of this will, of course, help when the hardware itself is buggered to the point that there is nothing we can do about it. I'm not trying to deal with anything like that.

Things that I have identified, at least in my opinion:

1. Opening a device for configuration as opposed to data streaming; in the tty case that doesn't just improve the DTR# and RTS# issue but allows setserial, configuring line disciplines and so on.

As I have said, this is application-specific intent, which is why I strongly believe that it needs to be part of the open system call. I furthermore believe that it would have use cases beyond ttys and serial ports, which is why I'm proposing a new open flag as opposed to a sysfs attribute, which actually was my initial approach (yes, I have already prototyped some of this, and as referenced before there is an existing patchset that was never merged.)

2. Currently the setserial configurables are available in sysfs, but *only* for UARTs, whereas TIOC[GS]SERIAL is at least available to all serial devices. That code should presumably be hoisted into a higher layer; this shouldn't be too difficult.

3. The only way to determine the type of a tty driver is reading and parsing /proc/tty/drivers; that information is exported neither through ioctl nor sysfs. Exporting *that* through sysfs is probably the easiest of all the improvements.

4. There isn't a device-independent way to determine if a device is "real" (configured for hardware) or not without opening it and executing one of the termios ioctls like TCGETS (returns -EIO if there isn't anything behind it.) For a UART port it is possible to come up with an educated guess based on the aforementioned sysfs properties (does it have any kind of address associated with it?), but seriously, should stty -a /dev/ttyS0 really glitch RTS# and DTR# even though there is no intent of using the port for communication? 

Let me make it very clear that I'm *not* criticizing neither you, Ted, Alan Cox nor anyone else who have been involved: on the contrary, you have done an absolutely fantastic job making Linux work with all these pieces of hardware, each with various "interesting" properties. Nor am I criticizing the tty interface as it is: it is designed to allow both interactive terminal use and use for other purposes under program control, which is really an astonishing level of flexibility. We have already shown that it can evolve to meet new needs, which sometimes requires interface extensions – like O_NOCTTY, O_NONBLOCK, CRTSCTS, termios2, and BOTHER. *And that is perfectly okay.* If anything it is a strength.

And please do recognize that I have stated from the beginning that I expect this to be a "best effort" on the part of kernel, not a guarantee. If the hardware is too broken, the user gets to keep both pieces – that's just reality.

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[Greg KH]

Trimming out stuff to get to the real questions:

On Wed, Nov 12, 2025 at 11:12:22AM -0800, H. Peter Anvin wrote:
> Things that I have identified, at least in my opinion:
> 
> 1. Opening a device for configuration as opposed to data streaming; in the tty case that doesn't just improve the DTR# and RTS# issue but allows setserial, configuring line disciplines and so on.
> 
> As I have said, this is application-specific intent, which is why I strongly believe that it needs to be part of the open system call. I furthermore believe that it would have use cases beyond ttys and serial ports, which is why I'm proposing a new open flag as opposed to a sysfs attribute, which actually was my initial approach (yes, I have already prototyped some of this, and as referenced before there is an existing patchset that was never merged.)

I think this is going to be the most difficult.  I don't remember why I
rejected the old submission, but maybe it would have modified the
existing behaviour?  A new open flag "O_DO_NOT_TOUCH_ANYTHING" might be
the simplest?

> 2. Currently the setserial configurables are available in sysfs, but *only* for UARTs, whereas TIOC[GS]SERIAL is at least available to all serial devices. That code should presumably be hoisted into a higher layer; this shouldn't be too difficult.

I agree, this shouldn't be hard, no reason to not do this.

> 3. The only way to determine the type of a tty driver is reading and parsing /proc/tty/drivers; that information is exported neither through ioctl nor sysfs. Exporting *that* through sysfs is probably the easiest of all the improvements.

The "type" is interesting.  We keep adding new "types" of serial ports
to the uapi list, and they don't really show up very well to userspace,
as you say.  Adding this export to sysfs is fine with me, but we should
make it a string somehow, and not just a random number like the current
types are listed as, to give people a chance to keep track of this.

So yes, this too should be done.

> 4. There isn't a device-independent way to determine if a device is "real" (configured for hardware) or not without opening it and executing one of the termios ioctls like TCGETS (returns -EIO if there isn't anything behind it.) For a UART port it is possible to come up with an educated guess based on the aforementioned sysfs properties (does it have any kind of address associated with it?), but seriously, should stty -a /dev/ttyS0 really glitch RTS# and DTR# even though there is no intent of using the port for communication? 

Determining "realness" is going to be hard I think (is a usb-serial
device real or not?  Some are, some are not, but how do we even know?)
Does a "real" uart mean that the device is real?  How do you define
that?  What about virtual ones?  Modem chips that do have full line
discipline support on USB connections?  There's a lot out there to deal
with here and I think some "fake" ones do pass TCGETS calls just because
they lie.)

And addresses are only the "very old" method, many "real" PCI uarts
don't have them, same for USB ones.

And changing 'stty -a' is going to be hard, unless you want to use the
new flag?

But yes, making this more sane is always good, 2 of your things here
should be pretty simple to knock up if someone wants to.  The others
might be more difficult just due to backwards compatibility issues.

thanks,

greg k-h

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[H. Peter Anvin]

On 2025-11-12 11:39, Greg KH wrote:
> 
>> 3. The only way to determine the type of a tty driver is reading and parsing /proc/tty/drivers; that information is exported neither through ioctl nor sysfs. Exporting *that* through sysfs is probably the easiest of all the improvements.
> 
> The "type" is interesting.  We keep adding new "types" of serial ports
> to the uapi list, and they don't really show up very well to userspace,
> as you say.  Adding this export to sysfs is fine with me, but we should
> make it a string somehow, and not just a random number like the current
> types are listed as, to give people a chance to keep track of this.
> 
> So yes, this too should be done.
> 

Yes, this one is pretty obvious:

>> 4. There isn't a device-independent way to determine if a device is "real" (configured for hardware) or not without opening it and executing one of the termios ioctls like TCGETS (returns -EIO if there isn't anything behind it.) For a UART port it is possible to come up with an educated guess based on the aforementioned sysfs properties (does it have any kind of address associated with it?), but seriously, should stty -a /dev/ttyS0 really glitch RTS# and DTR# even though there is no intent of using the port for communication? 
> 
> Determining "realness" is going to be hard I think (is a usb-serial
> device real or not?  Some are, some are not, but how do we even know?)
> Does a "real" uart mean that the device is real?  How do you define
> that?  What about virtual ones?  Modem chips that do have full line
> discipline support on USB connections?  There's a lot out there to deal
> with here and I think some "fake" ones do pass TCGETS calls just because
> they lie.)
> 

What I mean with "real" is that the device exists at all, unlike e.g.
/dev/ttyS* device nodes which are *only* available for the purpose of binding.

So "bound to a hardware device" is what I mean, not that it is a device with
RS232 drivers on it (which would be impossible to determine, as you very
correctly point out.)

> And addresses are only the "very old" method, many "real" PCI uarts
> don't have them, same for USB ones.
> 
> And changing 'stty -a' is going to be hard, unless you want to use the
> new flag?

That's exactly the idea: use the new open flag.

> But yes, making this more sane is always good, 2 of your things here
> should be pretty simple to knock up if someone wants to.  The others
> might be more difficult just due to backwards compatibility issues.


Indeed. Which is the whole reason for this RFC thread.

	-hpa

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[H. Peter Anvin]

On 2025-11-12 11:39, Greg KH wrote:
> Trimming out stuff to get to the real questions:
> 
> On Wed, Nov 12, 2025 at 11:12:22AM -0800, H. Peter Anvin wrote:
>> Things that I have identified, at least in my opinion:
>>
>> 1. Opening a device for configuration as opposed to data streaming; in the tty case that doesn't just improve the DTR# and RTS# issue but allows setserial, configuring line disciplines and so on.
>>
>> As I have said, this is application-specific intent, which is why I strongly believe that it needs to be part of the open system call. I furthermore believe that it would have use cases beyond ttys and serial ports, which is why I'm proposing a new open flag as opposed to a sysfs attribute, which actually was my initial approach (yes, I have already prototyped some of this, and as referenced before there is an existing patchset that was never merged.)
> 
> I think this is going to be the most difficult.  I don't remember why I
> rejected the old submission, but maybe it would have modified the
> existing behaviour?  A new open flag "O_DO_NOT_TOUCH_ANYTHING" might be
> the simplest?

That was exactly my proposal - see the header of this thread :)

>> 3. The only way to determine the type of a tty driver is reading and parsing /proc/tty/drivers; that information is exported neither through ioctl nor sysfs. Exporting *that* through sysfs is probably the easiest of all the improvements.
> 
> The "type" is interesting.  We keep adding new "types" of serial ports
> to the uapi list, and they don't really show up very well to userspace,
> as you say.  Adding this export to sysfs is fine with me, but we should
> make it a string somehow, and not just a random number like the current
> types are listed as, to give people a chance to keep track of this.
> 
> So yes, this too should be done.

I meant to add this to the previous email -- the obvious choice (and what is
in my prototype) is to use the same string as is currently exposed in
/proc/tty/drivers.

	-hpa

Re: RFC: Serial port DTR/RTS - O_<something>

[H. Peter Anvin]

On 2025-11-12 11:39, Greg KH wrote:
>>
>> 1. Opening a device for configuration as opposed to data streaming; in the tty case that doesn't just improve the DTR# and RTS# issue but allows setserial, configuring line disciplines and so on.
>>
>> As I have said, this is application-specific intent, which is why I strongly believe that it needs to be part of the open system call. I furthermore believe that it would have use cases beyond ttys and serial ports, which is why I'm proposing a new open flag as opposed to a sysfs attribute, which actually was my initial approach (yes, I have already prototyped some of this, and as referenced before there is an existing patchset that was never merged.)
> 
> I think this is going to be the most difficult.  I don't remember why I
> rejected the old submission, but maybe it would have modified the
> existing behaviour?  A new open flag "O_DO_NOT_TOUCH_ANYTHING" might be
> the simplest?
> 

Okay, to I'm going to toss out a couple suggestions for naming:

	O_(PRE|FOR|N|NO)?(INIT|CONFIG|START)(DEV|HW|IO)?
	O_(NO?RESET|PREPARE)(DEV|HW|IO)?
	O_NO?TOUCH
	O_NYET ("not yet")
	
I think my personal preference at the moment is either O_NYET or O_PRECONFIG
or O_NYET; although it is perhaps a bit more "use case centric" than "what
actual effect it has" I think it might be clearer.  A -DEV, -HW or -IO suffix
would seem to needlessly preclude it being used for future similar use cases
for files that are not device nodes.

O_NYET ("not yet") is kind of attractive because it has some geekish smirk
value, doesn't have "obvious enough" meaning that if you don't know what it
does you'll guess rather than looking it up, but once you know you are not
going to forget it!  There is even precedent: USB 2 already has the NYET
packet type meaning just "not yet".  The more I'm thinking about it the more
am starting to like it...

Many of the other combinations have the problem of seeming to do the opposite
of what the used wants in some use cases; it seems rather odd to open a device
node that you are intending to configure with "O_NOCONFIG".  On the other
hand, "O_CONFIG" might be a valid indication of the intent (like O_RDONLY or
O_RDWR are indicator of intent), but also has the implication that it *will*
cause the device to configure itself.  It also would seem to imply that the
resulting file descriptor can *only* be used for that purpose.

	-hpa

RE: RFC: Serial port DTR/RTS - O_<something>

[Maarten Brock]

> > A new open flag "O_DO_NOT_TOUCH_ANYTHING" might be
> > the simplest?
> >
> 
> Okay, to I'm going to toss out a couple suggestions for naming:
> 
> 	O_(PRE|FOR|N|NO)?(INIT|CONFIG|START)(DEV|HW|IO)?
> 	O_(NO?RESET|PREPARE)(DEV|HW|IO)?
> 	O_NO?TOUCH
> 	O_NYET ("not yet")
> 
> I think my personal preference at the moment is either O_NYET or O_PRECONFIG
> or O_NYET; although it is perhaps a bit more "use case centric" than "what
> actual effect it has" I think it might be clearer.  A -DEV, -HW or -IO suffix
> would seem to needlessly preclude it being used for future similar use cases
> for files that are not device nodes.
> 
> O_NYET ("not yet") is kind of attractive because it has some geekish smirk
> value, doesn't have "obvious enough" meaning that if you don't know what it
> does you'll guess rather than looking it up, but once you know you are not
> going to forget it!  There is even precedent: USB 2 already has the NYET
> packet type meaning just "not yet".  The more I'm thinking about it the more
> am starting to like it...

Personally, I don't much like the O_NYET as it seems to describe not to open
the device.

> Many of the other combinations have the problem of seeming to do the opposite
> of what the used wants in some use cases; it seems rather odd to open a device
> node that you are intending to configure with "O_NOCONFIG".

Don't like this one either.

> On the other
> hand, "O_CONFIG" might be a valid indication of the intent (like O_RDONLY or
> O_RDWR are indicator of intent), but also has the implication that it *will*
> cause the device to configure itself.  It also would seem to imply that the
> resulting file descriptor can *only* be used for that purpose.

I do like the O_CONFIG or O_FORCONFIG names.
I also like O_PREINIT or O_PRESTART.

Kind Regards,
Maarten

Re: RFC: Serial port DTR/RTS - O_<something>

[Maciej W. Rozycki]

On Thu, 13 Nov 2025, H. Peter Anvin wrote:

> > I think this is going to be the most difficult.  I don't remember why I
> > rejected the old submission, but maybe it would have modified the
> > existing behaviour?  A new open flag "O_DO_NOT_TOUCH_ANYTHING" might be
> > the simplest?
> > 
> 
> Okay, to I'm going to toss out a couple suggestions for naming:
> 
> 	O_(PRE|FOR|N|NO)?(INIT|CONFIG|START)(DEV|HW|IO)?
> 	O_(NO?RESET|PREPARE)(DEV|HW|IO)?
> 	O_NO?TOUCH
> 	O_NYET ("not yet")
> 	
> I think my personal preference at the moment is either O_NYET or O_PRECONFIG
> or O_NYET; although it is perhaps a bit more "use case centric" than "what
> actual effect it has" I think it might be clearer.  A -DEV, -HW or -IO suffix
> would seem to needlessly preclude it being used for future similar use cases
> for files that are not device nodes.

 Hmm, I'm inconvinced about any of these.

 How about O_FDONLY, to reflect that you are after a file descriptor only 
with no further actions at open time while avoiding the ambiguity of names 
such as CONFIG vs NOCONFIG or speaking more broadly implying any specific 
intent of use at all such as with CONFIG/INIT/PREPARE/RESET/whatever?

 I think O_FDONLY is concise, easy to spell/say/remember, and fits the 
purpose.  Your call!

  Maciej

Re: RFC: Serial port DTR/RTS - O_<something>

[H. Peter Anvin]

On November 14, 2025 10:49:09 AM PST, "Maciej W. Rozycki" <macro@orcam.me.uk> wrote:
>On Thu, 13 Nov 2025, H. Peter Anvin wrote:
>
>> > I think this is going to be the most difficult.  I don't remember why I
>> > rejected the old submission, but maybe it would have modified the
>> > existing behaviour?  A new open flag "O_DO_NOT_TOUCH_ANYTHING" might be
>> > the simplest?
>> > 
>> 
>> Okay, to I'm going to toss out a couple suggestions for naming:
>> 
>> 	O_(PRE|FOR|N|NO)?(INIT|CONFIG|START)(DEV|HW|IO)?
>> 	O_(NO?RESET|PREPARE)(DEV|HW|IO)?
>> 	O_NO?TOUCH
>> 	O_NYET ("not yet")
>> 	
>> I think my personal preference at the moment is either O_NYET or O_PRECONFIG
>> or O_NYET; although it is perhaps a bit more "use case centric" than "what
>> actual effect it has" I think it might be clearer.  A -DEV, -HW or -IO suffix
>> would seem to needlessly preclude it being used for future similar use cases
>> for files that are not device nodes.
>
> Hmm, I'm inconvinced about any of these.
>
> How about O_FDONLY, to reflect that you are after a file descriptor only 
>with no further actions at open time while avoiding the ambiguity of names 
>such as CONFIG vs NOCONFIG or speaking more broadly implying any specific 
>intent of use at all such as with CONFIG/INIT/PREPARE/RESET/whatever?
>
> I think O_FDONLY is concise, easy to spell/say/remember, and fits the 
>purpose.  Your call!
>
>  Maciej

Overlaps too much with O_PATH, and implies that communication isn't possible *after* device-dependent setup.

Re: RFC: Serial port DTR/RTS - O_<something>

[Ned Ulbricht]

On 11/14/25 10:53, H. Peter Anvin wrote:
> On November 14, 2025 10:49:09 AM PST, "Maciej W. Rozycki" <macro@orcam.me.uk> wrote:
>> On Thu, 13 Nov 2025, H. Peter Anvin wrote:
>>
>>>> I think this is going to be the most difficult.  I don't remember why I
>>>> rejected the old submission, but maybe it would have modified the
>>>> existing behaviour?  A new open flag "O_DO_NOT_TOUCH_ANYTHING" might be
>>>> the simplest?
>>>>
>>>
>>> Okay, to I'm going to toss out a couple suggestions for naming:
>>>
>>> 	O_(PRE|FOR|N|NO)?(INIT|CONFIG|START)(DEV|HW|IO)?
>>> 	O_(NO?RESET|PREPARE)(DEV|HW|IO)?
>>> 	O_NO?TOUCH
>>> 	O_NYET ("not yet")
>>> 	
>>> I think my personal preference at the moment is either O_NYET or O_PRECONFIG
>>> or O_NYET; although it is perhaps a bit more "use case centric" than "what
>>> actual effect it has" I think it might be clearer.  A -DEV, -HW or -IO suffix
>>> would seem to needlessly preclude it being used for future similar use cases
>>> for files that are not device nodes.
>>
>> Hmm, I'm inconvinced about any of these.
>>
>> How about O_FDONLY, to reflect that you are after a file descriptor only [snip]

Hi all,

Resurrecting a (private email) discussion from a few years back now, my
personal preferences are:
(1) O_KEEP
(2) O_TTY_KEEP
(3) O_TTY_NOINIT.

(Of course, naming an open() flag has got to be a paradigmatic
invitation for bike-shedding...)

It's worth pointing out, though, that even though O_TTY_INIT doesn't
generally appear in linux headers, that particular flag is documented in
POSIX to have at least incompatible --perhaps even strictly opposite--
behavior compared with this new proposed flag.

See The Open Group Base Specifications Issue 8 (IEEE Std 1003.1-2024):

| 11.1.1 Opening a Terminal Device File
|
| 3. ... The terminal parameters can be set to values that ensure the
| terminal behaves in a conforming manner by means of the O_TTY_INIT
| open flag....

https://pubs.opengroup.org/onlinepubs/9799919799/basedefs/V1_chap11.html

| open, openat — open file
|
| O_TTY_INIT

https://pubs.opengroup.org/onlinepubs/9799919799/

That's what motivates my first-glance preference to name this new flag,
which will have approximately opposite behavior, as O_TTY_NOINIT.

But as a generic abstraction, I more prefer O_KEEP.

Ned

Re: RFC: Serial port DTR/RTS - O_<something>

[H. Peter Anvin]

On 2025-11-15 13:29, Ned Ulbricht wrote:
> On 11/14/25 10:53, H. Peter Anvin wrote:
>> On November 14, 2025 10:49:09 AM PST, "Maciej W. Rozycki"
>> <macro@orcam.me.uk> wrote:
>>> On Thu, 13 Nov 2025, H. Peter Anvin wrote:
>>>
>>>>> I think this is going to be the most difficult.  I don't remember why I
>>>>> rejected the old submission, but maybe it would have modified the
>>>>> existing behaviour?  A new open flag "O_DO_NOT_TOUCH_ANYTHING" might be
>>>>> the simplest?
>>>>>
>>>>
>>>> Okay, to I'm going to toss out a couple suggestions for naming:
>>>>
>>>>     O_(PRE|FOR|N|NO)?(INIT|CONFIG|START)(DEV|HW|IO)?
>>>>     O_(NO?RESET|PREPARE)(DEV|HW|IO)?
>>>>     O_NO?TOUCH
>>>>     O_NYET ("not yet")
>>>>     
>>>> I think my personal preference at the moment is either O_NYET or O_PRECONFIG
>>>> or O_NYET; although it is perhaps a bit more "use case centric" than "what
>>>> actual effect it has" I think it might be clearer.  A -DEV, -HW or -IO suffix
>>>> would seem to needlessly preclude it being used for future similar use cases
>>>> for files that are not device nodes.
>>>
>>> Hmm, I'm inconvinced about any of these.
>>>
>>> How about O_FDONLY, to reflect that you are after a file descriptor only
>>> [snip]
> 
> Hi all,
> 
> Resurrecting a (private email) discussion from a few years back now, my
> personal preferences are:
> (1) O_KEEP
> (2) O_TTY_KEEP
> (3) O_TTY_NOINIT.
> 
> (Of course, naming an open() flag has got to be a paradigmatic
> invitation for bike-shedding...)
> 
> It's worth pointing out, though, that even though O_TTY_INIT doesn't
> generally appear in linux headers, that particular flag is documented in
> POSIX to have at least incompatible --perhaps even strictly opposite--
> behavior compared with this new proposed flag.
> 

I dislike O_TTY_* because restricts it to the TTY use case.

	-hpa

Re: RFC: Serial port DTR/RTS - O_<something>

[H. Peter Anvin]

On 2025-11-15 13:29, Ned Ulbricht wrote:
> |
> | O_TTY_INIT
> 
> https://pubs.opengroup.org/onlinepubs/9799919799/
> 
> That's what motivates my first-glance preference to name this new flag,
> which will have approximately opposite behavior, as O_TTY_NOINIT.
> 
> But as a generic abstraction, I more prefer O_KEEP.
> 

O_KEEP seems a little vague, but O_KEEPCONFIG seems like a decent name.

It seems like we don't have several new flags:

	O_EXEC
	O_SEARCH
	O_CLOFORK
	O_TTY_INIT
	O_RSYNC
	O_NOCLOBBER

Some of them *may* be possible to construct with existing Linux options, I'm
not 100% sure; in particular O_SEARCH might be the same as (O_DIRECTORY|O_PATH).

O_NOCLOBBER looks like an odd in-between between O_EXCL and
(O_EXCL|O_NOFOLLOW); stated to be specifically to implement the shell
"noclobber" semantic.

	-hpa

Re: RFC: Serial port DTR/RTS - O_<something>

[Ned Ulbricht]

On 11/15/25 16:47, H. Peter Anvin wrote:
> On 2025-11-15 13:29, Ned Ulbricht wrote:
>> |
>> | O_TTY_INIT
>>
>> https://pubs.opengroup.org/onlinepubs/9799919799/
>>
>> That's what motivates my first-glance preference to name this new flag,
>> which will have approximately opposite behavior, as O_TTY_NOINIT.
>>
>> But as a generic abstraction, I more prefer O_KEEP.
>>
> 
> O_KEEP seems a little vague, but O_KEEPCONFIG seems like a decent name.
> 
> It seems like we don't have several new flags:
> 
> 	O_EXEC
> 	O_SEARCH
> 	O_CLOFORK
> 	O_TTY_INIT
> 	O_RSYNC
> 	O_NOCLOBBER
> 
> Some of them *may* be possible to construct with existing Linux options, I'm
> not 100% sure; in particular O_SEARCH might be the same as (O_DIRECTORY|O_PATH).
> 
> O_NOCLOBBER looks like an odd in-between between O_EXCL and
> (O_EXCL|O_NOFOLLOW); stated to be specifically to implement the shell
> "noclobber" semantic.

"(O_EXCL|O_NOFOLLOW)" provokes a thought...

As essential context, fs/open.c build_open_flags() has:

if (flags & O_CREAT) {
	op->intent |= LOOKUP_CREATE;
	if (flags & O_EXCL) {
		op->intent |= LOOKUP_EXCL;
		flags |= O_NOFOLLOW;
	}
}

if (!(flags & O_NOFOLLOW))
	lookup_flags |= LOOKUP_FOLLOW;

So with that context, just imagine hypothetically implementing both a
non-zero O_TTY_INIT flag and an O_KEEPCONFIG flag. What would
build_open_flags() look like to handle the case where userspace
simultaneously asserts both flags?  Even if it's documented, specified
as unspecified behavior, what would the code actually do?

Or, alternatively, should an hypothetical standardization insist that in
any implementation, one of O_TTY_INIT, O_KEEPCONFIG must be #define'd 0?


Ned

Re: RFC: Serial port DTR/RTS - O_<something>

[H. Peter Anvin]

On November 18, 2025 8:33:07 AM PST, Ned Ulbricht <nedu@netscape.net> wrote:
>On 11/15/25 16:47, H. Peter Anvin wrote:
>> On 2025-11-15 13:29, Ned Ulbricht wrote:
>>> |
>>> | O_TTY_INIT
>>> 
>>> https://pubs.opengroup.org/onlinepubs/9799919799/
>>> 
>>> That's what motivates my first-glance preference to name this new flag,
>>> which will have approximately opposite behavior, as O_TTY_NOINIT.
>>> 
>>> But as a generic abstraction, I more prefer O_KEEP.
>>> 
>> 
>> O_KEEP seems a little vague, but O_KEEPCONFIG seems like a decent name.
>> 
>> It seems like we don't have several new flags:
>> 
>> 	O_EXEC
>> 	O_SEARCH
>> 	O_CLOFORK
>> 	O_TTY_INIT
>> 	O_RSYNC
>> 	O_NOCLOBBER
>> 
>> Some of them *may* be possible to construct with existing Linux options, I'm
>> not 100% sure; in particular O_SEARCH might be the same as (O_DIRECTORY|O_PATH).
>> 
>> O_NOCLOBBER looks like an odd in-between between O_EXCL and
>> (O_EXCL|O_NOFOLLOW); stated to be specifically to implement the shell
>> "noclobber" semantic.
>
>"(O_EXCL|O_NOFOLLOW)" provokes a thought...
>
>As essential context, fs/open.c build_open_flags() has:
>
>if (flags & O_CREAT) {
>	op->intent |= LOOKUP_CREATE;
>	if (flags & O_EXCL) {
>		op->intent |= LOOKUP_EXCL;
>		flags |= O_NOFOLLOW;
>	}
>}
>
>if (!(flags & O_NOFOLLOW))
>	lookup_flags |= LOOKUP_FOLLOW;
>
>So with that context, just imagine hypothetically implementing both a
>non-zero O_TTY_INIT flag and an O_KEEPCONFIG flag. What would
>build_open_flags() look like to handle the case where userspace
>simultaneously asserts both flags?  Even if it's documented, specified
>as unspecified behavior, what would the code actually do?
>
>Or, alternatively, should an hypothetical standardization insist that in
>any implementation, one of O_TTY_INIT, O_KEEPCONFIG must be #define'd 0?
>
>
>Ned

It's not actually a contradiction: it means preserve all configuration *except* the minimal termios tweaks required to bring it inside the POSIX compliant envelope, notably setting winsize to "appropriate default parameters."

Linux doesn't have a lot of such settings, but I can see at least one *very useful* one: bringing B19200 and B38400 (EXTA and EXTB), which can be tweaked by setserial, back to their proper baud rates.

There are even two ways to do that: either keep the B19200/B38400 setting and change the baud rate, or keep the baud rate and change termios to match (using BOTHER if necessary; after my changes to glibc 2.42+ BOTHER is a private interface between glibc and the kernel and thus doesn't break POSIX compliance.)

A fairly reasonable implementation would be the first with O_TTY_INIT and the second with O_TTY_INIT | O_KEEPCONFIG.

Flags in termios that probably should be cleared by O_TTY_INIT are CMSPAR, OLCUC, IUCLC, IMAXBEL, ECHOPRT, ECHOKE, FLUSHO, PENDIN, IEXTEN and EXTPROC; I'm not sure about ADDRB, CRTSCTS, IUTF8 or the line discipline; at least with O_KEEPCONFIG at least CRTSCTS ought to be kept I would think, as changing it would have immediate effect visible to 

Obviously an application that wants to absolutely minimize changes must not pass O_TTY_INIT.

The other (and simpler!) option is to simply declare O_KEEPCONFIG | O_TTY_INIT as a reserved bit combination and return -EINVAL until we find a good reason to do anything different.

Re: RFC: Serial port DTR/RTS - O_<something>

[H. Peter Anvin]

On 2025-11-18 08:33, Ned Ulbricht wrote:
>>
>> O_NOCLOBBER looks like an odd in-between between O_EXCL and
>> (O_EXCL|O_NOFOLLOW); stated to be specifically to implement the shell
>> "noclobber" semantic.
> 
> "(O_EXCL|O_NOFOLLOW)" provokes a thought...
> 
> As essential context, fs/open.c build_open_flags() has:
> 
> if (flags & O_CREAT) {
>     op->intent |= LOOKUP_CREATE;
>     if (flags & O_EXCL) {
>         op->intent |= LOOKUP_EXCL;
>         flags |= O_NOFOLLOW;
>     }
> }
> 
> if (!(flags & O_NOFOLLOW))
>     lookup_flags |= LOOKUP_FOLLOW;
> 

Interesting. As far as O_NOCLOBBER is concerned, that is an "O_EXCL unless the
output is a special file (device node, FIFO, etc)"; presumably to allow the
shell to not flip out when doing, say "foo > /dev/ttyS0" when in noclobber mode.

I had missed the bit in the spec that says that O_CREAT|O_EXCL is required to
imply O_NOFOLLOW (as Linux indeed does as seen above.)

O_NOCLOBBER emulation in user space would seem to be possible with a loop;
first try to open O_CREAT|O_EXCL and if that fails with EEXIST then open
without either; if that succeeds test with fstat() to see if it is a regular
file, and if it is, close it and error. However, it is hardly ideal, and I
might have overlooked some mechanism by which this may fail.

	-hpa

Re: RFC: Serial port DTR/RTS - O_<something>

[Ned Ulbricht]

On 11/18/25 10:05, H. Peter Anvin wrote:

>> "(O_EXCL|O_NOFOLLOW)" provokes a thought...
>>
>> As essential context, fs/open.c build_open_flags() has:
>>
>> if (flags & O_CREAT) {
>>      op->intent |= LOOKUP_CREATE;
>>      if (flags & O_EXCL) {
>>          op->intent |= LOOKUP_EXCL;
>>          flags |= O_NOFOLLOW;
>>      }
>> }

[snip]

> I had missed the bit in the spec that says that O_CREAT|O_EXCL is required to
> imply O_NOFOLLOW (as Linux indeed does as seen above.)

Fwiw, earlier today I had an ultimately unsuccessful series of searches
using the Austin Group Issue tracker at:

https://austingroupbugs.net/view_all_bug_page.php

Searched (serially): "O_EXCL", "O_NOFOLLOW", "EEXIST", "ELOOP"; all with
no other filter refinements.  Then searched filtering by 'Section'
(multiple adjacent selections): 'open' .. 'openat'. In all results,
simply eyeball-scanned 'Summary' (w/o opening most).

Apparent upshot, unless I'm mistaken, is that the exact error return is
a trivial conflict with no appreciable impact on higher levels.


In roughly same vein, FreeBSD open(2) man page, specifically at
"[EMLINK]" and "STANDARDS", might possibly be stretched to read as
implicitly encouraging that assessment.

https://man.freebsd.org/cgi/man.cgi?query=open&manpath=FreeBSD+16.0-CURRENT

Alhough I don't have a FreeBSD box available to actually test
(O_CREAT|O_EXCL|O_NOFOLLOW) symlink behavior on that platform.  (Maybe
that combo's wired to detonate tnt nasal daemons? Dunno;-)


Unfortunately, this does prompt a close re-scrutinization of linux's
open(2) man page. Notwithstanding the damn spec, the linux man page
should precisely and accurately reflect the observed error return?


Ned

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[H. Peter Anvin]

On November 7, 2025 9:37:43 AM PST, Theodore Ts'o <tytso@mit.edu> wrote:
>On Thu, Nov 06, 2025 at 11:53:23PM -0800, H. Peter Anvin wrote:
>> 
>> I recently ran into a pretty serious issue due to the Unix/Linux
>> (mis)behavior of forcing DTR and RTS asserted when a serial port is
>> set, losing the pre-existing status in the process.
>
>There's a hidden assumption in your problem statement which is that
>DTR / RTS has a "state" which can be saved when the serial port is not
>active, where active is one or more file descriptors holding the
>serial port open.  There may be certain hardware or drivers where this
>is just not possible, because nothing is defined if the serial port is
>not active.  It might make sense if you are using a 8250 UART, but not
>all the world is the National Semiconductor (or clones) UART.
>
>Certainly the "state" will not be preserved across boots, since how we
>autodetect the UART is going to mess with UART settings.  So
>*presumably* what you are talking about is you want to be able to open
>the serial port, mess with DTR / RTS, and then be able to close the
>serial port, and then later on, re-open the serial port, have the DTR
>/ RTS remain the same.  And it's Too Hard(tm) to have userspace
>keeping a file descriptor open during the whole time?  (Which is
>traditionally how Unix/Linux has required that applications do
>things.)
>
>Is that a fair summary of the requirements?
>
>> It seems to me that this may very well be a problem beyond ttys, in
>> which case a new open flag to request to a driver that the
>> configuration and (observable) state of the underlying hardware
>> device -- whatever it may be -- should not be disturbed by calling
>> open(). This is of course already the case for many devices, not to
>> mention block and non-devices, in which case this flag is a don't
>> care.
>
>I think it's going to be a lot simpler to keep this specific to serial
>ports and DTR / RTS, because the concept that the hardware should not
>be changed when the file descriptor is opened may simply not be
>possible.  For example, it might be that until you open it, the there
>might not even be power applied to the device.  The concept that all
>hardware should burn battery power once the machine is booted may not
>make sense, and the assumption that hardware has the extra
>millicent(s) worth of silicon to maintain state when power is dropped
>may again, not be something that we can assume as being possible for
>all devices.
>
>If that's the case, if you want to have something where DTR and RTS
>stay the same, and for some reason we can't assume that userspace
>can't just keep a process holding the tty device open, my suggestion is to use 
>
>Given that DTR and RTS are secial port concepts, my suggesiton is to
>set a serial port flag, using setserial(8).  It may be the case that
>for certain types of serial device, the attempt to set the flag may be
>rejected, but that's something which the ioctl used by setserial
>already can do and which userspace applications such as setserial
>understand may be the case.
>
>Cheers,
>
>						- Ted

So let's separate out a few things here:

1. You are taking about using setserial(8), which is really ioctl(TIOCSSERIAL), which requires a file descriptor. This is exactly why I believe there should be a mechanism for acquiring a file descriptor which *by that action itself* should not change whatever state is already available to the kernel.

2. What, if anything, can be done on a device by device basis to improve the situation beyond what currently exists. 

Re: RFC: Serial port DTR/RTS - O_NRESETDEV

[Maciej W. Rozycki]

On Thu, 6 Nov 2025, H. Peter Anvin wrote:

> It seems to me that this may very well be a problem beyond ttys, in which case
> a new open flag to request to a driver that the configuration and (observable)
> state of the underlying hardware device -- whatever it may be -- should not be
> disturbed by calling open(). This is of course already the case for many
> devices, not to mention block and non-devices, in which case this flag is a
> don't care.

 FWIW I find using an open flag the most natural way to solve this problem 
and I disagree with a view that a 50+ year old standard has to prevent us 
from handling new use cases found as the world has changed.  We do need to 
comply with the standard for the devices that use it, but I think a flag 
to opt out is a perfectly sane approach.

 Yes, some hardware has limitations and may have to conclude we can't do 
anything about it.  Just as, say, we can't choose an arbitrary baud rate 
with the dz.c driver, because the hardware handled has a 4-bit selector 
for a set of predefined rates (to stay remotely on topic).  That does not 
prevent us from handling more flexible hardware in a way that makes full 
use of its features.

> The best name I came up with was O_NRESETDEV, but it's not something I'm
> particularly attached to.

 I'd suggest a generic name such as O_RAW for an agnostic way to express a 
request not to fiddle with the device in any way regardless of its kind, 
i.e. for possible reuse with anything.

> If the opinion is that this *doesn't* have a scope beyond ttys, then perhaps
> abusing the O_DIRECT flag for this purpose would be an alternative.

 It seems like a hack to me, but if carefully evaluated we could reuse the 
bit encoding.  Either way I'd encourage defining a new meaningful name for 
the new application of the flag, such as one proposed above.

  Maciej