Re: [PATCH] termios.3: Document line length in canonical mode

[Peter Hurley <peter-WaGBZJeGNqdsbIuE7sb01tBPR1lH4CV8@public.gmane.org>]

Hi Michael,

On 02/17/2016 02:09 AM, Michael Kerrisk (man-pages) wrote:
> On 02/16/2016 11:30 PM, Peter Hurley wrote:
>> On 02/15/2016 07:28 AM, Michael Kerrisk (man-pages) wrote:
>>> On 02/15/2016 02:26 PM, Dr. Tobias Quathamer wrote:
>>>> See https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/tty/n_tty.c#n1673
>>>> See https://bugs.debian.org/797479
>>>> ---
>>>>  man3/termios.3 | 9 +++++++++
>>>>  1 file changed, 9 insertions(+)
>>>>
>>>> diff --git a/man3/termios.3 b/man3/termios.3
>>>> index 7d738d4..3f57607 100644
>>>> --- a/man3/termios.3
>>>> +++ b/man3/termios.3
>>>> @@ -728,11 +728,20 @@ requested fewer bytes than are available in the current line of input,
>>>>  then only as many bytes as requested are read,
>>>>  and the remaining characters will be available for a future
>>>>  .BR read (2).
>>>> +.IP * 2
>>>> +The maximum line length is 4096 chars (including the line termination
>>>> +char); lines longer than 4096 chars are truncated. After 4095 chars,
>>>> +input data is still processed but not stored. Overflow processing
>>>> +ensures the tty can always receive more input until at least one
>>>> +line can be read.
>>>>  .PP
>>>>  In noncanonical mode input is available immediately (without
>>>>  the user having to type a line-delimiter character),
>>>>  no input processing is performed,
>>>>  and line editing is disabled.
>>>> +The read buffer will only accept 4095 chars; this provides the
>>>> +necessary space for a newline char if the input mode is switched
>>>> +to canonical.
>>>>  The settings of MIN
>>>>  .RI ( c_cc[VMIN] )
>>>>  and TIME
>>>
>>> Thanks for crafting this. I've applied, and tweaked a little to clarify
>>> some details:
>>>
>>>        * The maximum line length is 4096 chars (including the  terminating
>>>          newline  character);  lines longer than 4096 chars are truncated.
>>>          After 4095 characters, input data up (but not including) any ter‐
>>>          minating  newline  is  discarded.  This ensures that the terminal
>>>          can always receive more input until at  least  one  line  can  be
>>>          read.
>>
>> Hmm. Neither description is accurate of the observable behavior from userspace.
>> For example, it's entirely possible to retrieve > 4096 bytes in non-canonical
>> mode, at least since 3.12
> 
> Note that the text I quoted applies just to canonical mode:
> 
>        In canonical mode:
> 
>        [...]
> 
>        * The maximum line length is 4096 chars (including the  terminat‐
>          ing  newline character); lines longer than 4096 chars are trun‐
>          cated.  After 4095 characters, input data up (but  not  includ‐
>          ing)  any  terminating newline is discarded.  This ensures that
>          the terminal can always receive more input until at  least  one
>          line can be read.
> 
> So, does that seem okay?

See below.

>> And input processing continues on the input, even past 4096 bytes.
>> Line editing, ISIG, ECHOxx processing still occurs.
> 
> Yes, but only in noncanonical mode, right?

No. Input processing continues in canonical mode, including echoing.
Only when bytes are to actually be stored for userspace to later read is
the input data discarded if the buffer is full (less the 1 byte required to
store the line termination).

Every input byte received in canonical mode is always processed normally
and only discarded if the buffer is full. IOW, a Ctrl+u (line kill) received
as the 6000th byte received w/o line termination will still cause all previous
data to be flushed and the line restarted when new input arrives.

Similarly, all data is still echoed regardless of how long the line is.

>> It is true that it is not possible to retrieve > 4096 char line in canonical
>> mode, and I don't see that ever changing via read() because userspace may
>> assume it has received a terminated line in 4096-byte read buffer.
> 
> Yep. So we all seem to agree.

The way I read your edits above is that all new input data is simply discarded
until line termination, which is not the case.

Dr. Quathamer's first diff hunk is accurate wrt canonical mode. However,
I think the 'Overflow processing ...' is overly specific; the tty core contains
many elements which prevent terminal lockup.


>> However, the noncanonical mode input buffer size may change in the near
>> future.
> 
> Can you say some more about that?

At very high line rates and with ptys, the size of the read buffer becomes a
bottleneck, while at the same time, an already-committed page of memory is
going unused because of the page order allocation used by vm area allocator
(where N_TTY data is stored).

The change is non-trivial however, since the maximum read size returned
by canonical mode cannot > 4096 bytes (because of the userspace assumptions
I referred to earlier).

> And what changed in 3.13 with respect to noncanonical mode, by the way?

In 3.12, the entire tty input path was parallelized.

So while data is being read by userspace, input processing continues
concurrently, adding new data for userspace to read (the actual
construct used is a lockless circular buffer). In non-canonical modes,
the userspace copy is always performed twice -- first, from beginning of
unread data (head) to the end of buffer and then, from the beginning of
buffer to last unread data (tail).

Since new data may be added to the circular buffer in parallel after the
first user copy has been performed, it's possible to retrieve up to 8192
bytes (assuming the user copy buffer is that large).

Regards,
Peter Hurley
--
To unsubscribe from this list: send the line "unsubscribe linux-man" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html