RFC: Illegal Characters in File Names

RFC: Illegal Characters in File Names

[Joseph Wagner]

IMHO, e2fsck is not sufficiently aggressive in examining a file name for 
illegal characters. While it is possible to place non-printing control 
characters in a file name, few (if any) programs support opening a file 
with control characters in the file name.  In fact, 'rm' doesn't even 
support control characters.  To remove such a file, one must substitute a
wildcard character and run rm interactively (i.e. 'rm -i').

Some of us in the e2fsprogs project are considering a change which would 
mark all non-printing control characters (i.e. ASCII <=31 and ASCII == 127) 
as illegal.

I decided not to flag characters > ASCII 127 as illegal in case some day in 
the future the encoding changes to UTF-8, in which case valid printing 
non-control characters exist > 127.  On the flip side, ASCII and UTF-8 are 
100% compatible when <= 127, so the changes I did make will be fine with 
both.

Can anyone suggest a good reason not to good forward with this?

If it is OK to go forward, should the kernel be changed to disallow a file 
name from having these same non-printing control characters?

Joseph D. Wagner

Re: RFC: Illegal Characters in File Names

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 2665 bytes --]

On Sun, Jul 18, 2004 at 19:41:32 -0500, Joseph Wagner wrote:
> IMHO, e2fsck is not sufficiently aggressive in examining a file name for 
> illegal characters. While it is possible to place non-printing control 

There are just two illegal characters. '\0' and '/'. All other
characters are *permited*.

> characters in a file name, few (if any) programs support opening a file 
> with control characters in the file name.  In fact, 'rm' doesn't even 
> support control characters.  To remove such a file, one must substitute a
> wildcard character and run rm interactively (i.e. 'rm -i').

rm does perfectly support control characters! As do ALL the standard
utilities. What rm especialy does NOT support is the wildcard! It's the
shell that expands these. It's the shell that has problems with these,
but it's not a show-stopper:

$ touch "$(echo foo\\nbar)"
$ ls
foo?bar
$ ls -Q
"foo\nbar"
$ rm 'foo
quote> bar'
$ ls
$

$ touch foo^Bbar
$ ls
foo?bar
$ ls -Q
"foo\002bar"
$ rm foo^Bbar
$ ls
$

(note, that the ^B was written as Ctrl-V Ctrl-B)

Perfectly legal. Everything is right. You can write each and every
control char this way, just some of them must be in single quotes.

> Some of us in the e2fsprogs project are considering a change which would 
> mark all non-printing control characters (i.e. ASCII <=31 and ASCII == 127) 
> as illegal.

They are not illegal. The shell has problems with them, and the problems
are not absolute. You CAN type them in if you try. The file managers
will operate on them without problems, too.

> I decided not to flag characters > ASCII 127 as illegal in case some day in 
> the future the encoding changes to UTF-8, in which case valid printing 
> non-control characters exist > 127.  On the flip side, ASCII and UTF-8 are 
> 100% compatible when <= 127, so the changes I did make will be fine with 
> both.

The system is 8-bit clean an is relied upon by many users. I have LOTS
of files with iso-8859-2 encoded names on my filesystem.

> Can anyone suggest a good reason not to good forward with this?
> 
> If it is OK to go forward, should the kernel be changed to disallow a file 
> name from having these same non-printing control characters?

No, it's not OK. These characters are perfectly legal. Actualy, more
programs will have problems with filename containing a ':', that with
a filename containing, say, a '\b'. (usualy because they accept URI
arguments and use presence of : to distinguish plain filename and an
URI).

-------------------------------------------------------------------------------
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: RFC: Illegal Characters in File Names

[Matthew Wilcox]

On Sun, Jul 18, 2004 at 07:41:32PM -0500, Joseph Wagner wrote:
> Some of us in the e2fsprogs project are considering a change which would 
> mark all non-printing control characters (i.e. ASCII <=31 and ASCII == 127) 
> as illegal.

Where are "we" considering this?  I didn't see any discussion on
ext2-devel.

Possibly it would be of interest to include a utf8 converter in e2fsck
like the -s option, but it'd have to be something like -u iso-8859-1 so
it knows what it's converting from.  and then it'd be wrong for various
directory hierarchies because the filenames in /home/dostoyevsy are in
8859-5 but those in /home/kant are in 8859-1.  Since this can be done by
a separate tool that just renames files (hmm, symlinks will be tricky ;-),
it probably shouldn't be done in e2fsck.

Maybe a -u option that checks all filenames are in utf8 would be useful,
but "repairing" them?  How would you do that?  I do see the utility of
checking for non-printing characters as they're normally only inserted
by accident.  But this could be done by a cronjob (like locatedb) rather
than putting more functionality into e2fsck.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

RE: RFC: Illegal Characters in File Names

[Joseph D. Wagner]

> Where are "we" considering this?  I didn't see any
> discussion on ext2-devel.

OK, "we" are mostly me.

> Possibly it would be of interest to include a utf8
> converter in e2fsck like the -s option... [trimmed]

That would be too much work.

One thing at a time...
Baby steps...
You have to crawl before you can walk...
... and all that jazz.

> I do see the utility of checking for non-printing
> characters as they're normally only inserted by
> accident.  But this could be done by a cronjob (like
> locatedb) rather than putting more functionality
> into e2fsck.

I'm not just talking about checking for non-printing characters; I'm talking about making those non-printing characters illegal.

According to ISO 8859, the lower 128 characters are all the same.  It's the upper 128 characters that differ with iso-8859-1, iso-8859-2, etc.  Hence, the proposed change should be OK regardless of the encoding mechanism.

Joseph D. Wagner

RE: RFC: Illegal Characters in File Names

[Joseph D. Wagner]

> There are just two illegal characters. '\0' and '/'. All other
> characters are *permited [sic]*.

By whose standard?  POSIX doesn't require non-printing control characters to be legal.  Linux PERMITS them, but there's no standard requiring them to be permitted.

> They are not illegal. The shell has problems with them, and the problems
> are not absolute. You CAN type them in if you try. The file managers
> will operate on them without problems, too.

I would argue that the shell having problems with them is the exact reason they should be made illegal.

Allowing them begs the question "how should they be handled?"  For example, if a file name contained a backspace, displaying the raw backspace would backup the character's position and result in two characters being overwritten: the backspace and the character immediately prior to the backspace.  Printing a substitute character instead of the raw character simply leads to more questions.  Does '\b' mean backspace or backslash and b?  How do you tell the difference?

Handling these characters has not been standardized on Linux.  Different applications handle the same characters differently.  For example, ls displays a backslash followed by the octal character number, while KDE's Konqueror automatically substitutes characters with their hexadecimal form prefixing them with '%' so that prefix and hexadecimal form is stored on disk and when displayed the hexadecimal character is automatically transformed into the actual character.  This can lead to problems when different applications need to access the same file.  How do you know which method the other application used in handling these characters?

Additionally, security vulnerabilities (now patched) have resulted from the allowed use of control characters.

I want to close this potential can-of-worms.

> The system is 8-bit clean an [sic] is relied upon by many users.
> I have LOTS of files with iso-8859-2 encoded names on my filesystem.

According to ISO 8859, the lower 128 characters are all the same.  It's the upper 128 characters that differ with iso-8859-1, iso-8859-2, etc.  Hence, the proposed change should be OK regardless of the encoding mechanism.

For iso-8859-2 specifically, see:
http://nl.ijs.si/gnusl/cee/charset.html

Joseph D. Wagner

Re: RFC: Illegal Characters in File Names

[Pat LaVarre]

> Additionally, security vulnerabilities (now patched) have resulted 
> from the allowed use of control characters.
>
> I want to close this potential can-of-worms.

ftp has issues with '\0' and '\r' and '\n' in file names because of its 
telnet legacy.  I'll not here now volunteer anything more specific, for 
fear of controversy.  I remember ftpext talk of that issue suggested 
classic Mac Finder (analogous to Windows Explorer) accepted '\r' = CR = 
EOL in file names.  Confusing the byte '\xFF' with the char xFF also 
matters in ftp by way of telnet, but less controversially so.

unicode.org speaks of the security issues implicit in multiple byte 
encodings of the same char.  Those remain even after a ban on '\b'.

Pat LaVarre

RE: RFC: Illegal Characters in File Names

[Joseph D. Wagner]

> unicode.org speaks of the security issues implicit in multiple byte
> encodings of the same char.  Those remain even after a ban on '\b'.

I can't find it.  Please provide a link.

Joseph D. Wagner

Re: RFC: Illegal Characters in File Names

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 4488 bytes --]

On Mon, Jul 19, 2004 at 14:21:33 -0500, Joseph D. Wagner wrote:
> > There are just two illegal characters. '\0' and '/'. All other
> > characters are *permited [sic]*.
> 
> By whose standard?  POSIX doesn't require non-printing control
> characters to be legal.  Linux PERMITS them, but there's no standard
> requiring them to be permitted.

I didn't say they are required to be permited. I said they are. Which is
currently true.

> > They are not illegal. The shell has problems with them, and the problems
> > are not absolute. You CAN type them in if you try. The file managers
> > will operate on them without problems, too.
> 
> I would argue that the shell having problems with them is the exact
> reason they should be made illegal.

The fact, that some program, even if it's a shell, has problem with
something, it's an argument for one and only one thing -- that the
program should be fixed.

As I have already shown, control chars are inconvenient to work with in
shell, but POSSIBLE.

> Allowing them begs the question "how should they be handled?"  For
> example, if a file name contained a backspace, displaying the raw
> backspace would backup the character's position and result in two
> characters being overwritten: the backspace and the character
> immediately prior to the backspace.  Printing a substitute character
> instead of the raw character simply leads to more questions.  Does
> '\b' mean backspace or backslash and b?  How do you tell the
> difference?

ls does handle this. You can choose how it will display them using
various options.

> Handling these characters has not been standardized on Linux.
> Different applications handle the same characters differently.  For
> example, ls displays a backslash followed by the octal character

In the good old UNIX tradition. All the "low level" tools use this same
style of escaping.

> number, while KDE's Konqueror automatically substitutes characters
> with their hexadecimal form prefixing them with '%' so that prefix and
> hexadecimal form is stored on disk and when displayed the hexadecimal
> character is automatically transformed into the actual character.

No. Just tried. It's NOT that way -- when I create file with % 'URL'
escapes, Konqueror, quite correctly, displays the names verbatim. With
the % characters... When the filename contains non-displayable
character, Konqueror draws a box instead of it.

The % conversion you talk about happens when it handles URIs. The URI
specification mandates such translation -- and it's undoing on the
server.

> This can lead to problems when different applications need to access
> the same file.  How do you know which method the other application
> used in handling these characters?

The application stored the characters on disk. How it got them has
nothing to do with the filesystem.

If you need to give the file to an application, that can't handle
characters in a name, don't do that then.

Make can't handle ':', ',' and ' ' -- go ahead, forbid those too!

The fact that an application can't handle something is none of kernel's
and filesystem's business. You don't have to use such characters. And
one day, you may have a good reason to do so.

> Additionally, security vulnerabilities (now patched) have resulted
> from the allowed use of control characters.

Filenames are input. They must be validated. Filesystem should not fix
broken applications.

Could you mention where the vulnereabilites were?

> I want to close this potential can-of-worms.

Why is it a can-of-worms? The kernel has no problem with those
characters. Neither does the filesystem. Neither does any of the
standard system utilities. Neither does most applications.

> > The system is 8-bit clean an [sic] is relied upon by many users.
> > I have LOTS of files with iso-8859-2 encoded names on my filesystem.
> 
> According to ISO 8859, the lower 128 characters are all the same.
> It's the upper 128 characters that differ with iso-8859-1, iso-8859-2,
> etc.  Hence, the proposed change should be OK regardless of the
> encoding mechanism.

I was comenting that the characters >= 128 are ok.

Oh...! The characters 128 -- 159 behave as control characters (in
iso-8859-x), but must be kept for utf-8. Only part of files might be
utf-8 encoded...

-------------------------------------------------------------------------------
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: RFC: Illegal Characters in File Names

[Matthew Wilcox]

Could you wrap lines <80 columns?

On Mon, Jul 19, 2004 at 02:21:08PM -0500, Joseph D. Wagner wrote:
> I'm not just talking about checking for non-printing characters;
> I'm talking about making those non-printing characters illegal.
> 
> According to ISO 8859, the lower 128 characters are all the same.
> It's the upper 128 characters that differ with iso-8859-1, iso-8859-2,
> etc.  Hence, the proposed change should be OK regardless of the encoding
> mechanism.

utf8, ascii and iso8859 are not the only encodings.  Compare and contrast
with EBCDIC, koi8, iso 9036, big5, iso 2022, iso 2033, iso 5427, etc.

There's nothing to stop people encoding filenames in something like a
carefully selected subset of ucs2 (to avoid 0x00 and 0x3a bytes).

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

RE: RFC: Illegal Characters in File Names

[Joseph D. Wagner]

> utf8, ascii and iso8859 are not the only encodings.  Compare and contrast
> with EBCDIC, koi8, iso 9036, big5, iso 2022, iso 2033, iso 5427, etc.

Same thing.  These encoding mechanisms preserve the lower 128 of ASCII.  There shouldn't be a problem.

> There's nothing to stop people encoding filenames in something like a
> carefully selected subset of ucs2 (to avoid 0x00 and 0x3a bytes).

Isn't that the problem?  A lack of standardization?  I'm thinking of the future of Linux here.  Isn't it easier for application developer's everywhere if there was some uniformity to the encoding mechanism?

But like I said, that's too much to address at once.  Gotta think baby steps.

Back to the topic, can you think of a GOOD reason that non-printing control characters in the lower 128 SHOULD be allowed?  I'm drawing a blank.  Hence, shouldn't we change it to disallow those characters?

Joseph D. Wagner

Re: RFC: Illegal Characters in File Names

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1568 bytes --]

On Mon, 2004-07-19 14:21:33 -0500, Joseph D. Wagner <theman@josephdwagner.info>
wrote in message <S265489AbUGSTVf/20040719192135Z+666@vger.kernel.org>:
> I would argue that the shell having problems with them is the exact
> reason they should be made illegal.

The shell does not have problems with strange filenames; it's typically
the user who doesn't really know how to enter them properly.

> Allowing them begs the question "how should they be handled?"  For
> example, if a file name contained a backspace, displaying the raw
> backspace would backup the character's position and result in two
> characters being overwritten: the backspace and the character
> immediately prior to the backspace.  Printing a substitute character
> instead of the raw character simply leads to more questions.  Does
> '\b' mean backspace or backslash and b?  How do you tell the difference?

See, it's only a question of *display*. Currently, a file name is
anything you'd place into an ASCIIZ C string. That is basically
"anything" except the terminating \0. ...and '/' is used to delimit
parts of the whole part, so it's used internally.

Eye-friendly file names are "nice to have", but from my point of view,
it's no more than that.

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
   ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: RFC: Illegal Characters in File Names

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 1840 bytes --]

On Mon, Jul 19, 2004 at 22:16:22 -0500, Joseph D. Wagner wrote:
> > utf8, ascii and iso8859 are not the only encodings.  Compare and contrast
> > with EBCDIC, koi8, iso 9036, big5, iso 2022, iso 2033, iso 5427, etc.
> 
> Same thing.  These encoding mechanisms preserve the lower 128 of ASCII.  There shouldn't be a problem.

Read the list again, please. Ebcdic is included there.

> > There's nothing to stop people encoding filenames in something like a
> > carefully selected subset of ucs2 (to avoid 0x00 and 0x3a bytes).
> 
> Isn't that the problem?

No. It's not a problem. The filesystem does not need to care -- thus it
shouldn't.

> A lack of standardization? I'm thinking of the future of Linux here.
> Isn't it easier for application developer's everywhere if there was
> some uniformity to the encoding mechanism?

Writing applications is easier when encoding is uniform. That's why Gtk2
enforces utf-8 internaly and why tcl and python only use utf-8 internaly
and so on and so on. Still the kernel shouldn't care. It's userland's
business.

> But like I said, that's too much to address at once.  Gotta think baby steps.
> 
> Back to the topic, can you think of a GOOD reason that non-printing
> control characters in the lower 128 SHOULD be allowed?  I'm drawing
> a blank.  Hence, shouldn't we change it to disallow those characters?

Things should be disallowed if there is a good reason to disallow them.
If there is not, they should be allowed.

It would add extraneous complexity to disallow them. The filesystem code
has no problem with those characters, so it should not enforce arbitrary
restrictions on them. Those who have problems should enforce solutions.

-------------------------------------------------------------------------------
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

RE: RFC: Illegal Characters in File Names

[Joseph D. Wagner]

> Read the list again, please. Ebcdic is included there.

OK, that was an oversimplification, but guess what?  They are all still non-printing control characters with the exception of 127.

Joseph D. Wagner

RE: RFC: Illegal Characters in File Names

[Joseph D. Wagner]

> See, it's only a question of *display*.

No, it's not.  Some programs use file names as input.  Placing a timestamp directly in the file name comes to mind, but I'm sure you can think of many other examples.  Control characters in these file names would screw-up these programs.  While you're going to argue that a program should anticipate malformed data, I'm going to argue that application development will be easier for Linux is programmers knew that those characters aren't going to be there.

Where do most Linux programmers come from?  Sure there's the same core of people who have been programming Linux for years, but the new ones are coming from Windows platforms.  Which is easier: to reeducate every single new Linux programmer that any character could appear in a file name, or to cut out some of the characters that don't make sense?

Joseph D. Wagner

Re: RFC: Illegal Characters in File Names

[Bryan Henderson]

>> Additionally, security vulnerabilities (now patched) have resulted
>> from the allowed use of control characters.
>
>Filenames are input.  They must be validated.  Filesystem should not fix
>broken applications.

This is the well known conflict between what's philosophically right and 
what's practical.  Same as when a browser maker refuses to change his 
browser so that his user can use his bank's web site because "The problem 
is the bank's, because the bank chose to run a web server that does the 
wrong thing."

In this case, we all know that people _are_ going to neglect to validate 
filenames.  They also will neglect to use the options on 'ls' that make it 
show you what you think it's showing you.  In some cases, these will cause 
security vulnerabilities.  We can't change that.

So even though filesystem design shouldn't have to solve that problem, 
it's a very practical place to solve it.  (Actually, not _solve_ it, 
because you can't force a user to use your filesystem type, but at least 
ameliorate it).

I am opposed to denying users the ability to use nonprinting characters in 
their filenames, because it is an unnecessary withholding of function. 
It's withholding function from the user because he might abuse it and hurt 
himself.  Plus, it's a compatibility issue.  BUT: I think it makes great 
sense as a filesystem option (I mean an option of the actual filesystem -- 
set with mkfs).  If someone wants his filesystem to police this sort of 
failure, that's a good service to provide.

--
Bryan Henderson                          IBM Almaden Research Center
San Jose CA                              Filesystems

RE: RFC: Illegal Characters in File Names

[Guy]

I vote you disallow no-printable characters!
0-31 and 127.
I have never seen a case where they are needed.  If some app wants to use
non-printable control characters, let the app change.

I have seen a case where the correct control characters cause the terminal
to change the cursor x,y position, then send the cursor x,y position back to
the computer and cause the user to log off.  The cursor was moved to the 4th
position, the response included a "ctrl D" (ascii 4).  Seems like a security
risk!

If someone used the above method, they may be able to cause a simple "ll"
command to cause another program to be run!

Maybe we should ask:
Is there a good reason to support these control characters?
For now, assume that "because we do now" is not a good reason.

Guy

-----Original Message-----
From: linux-fsdevel-owner@vger.kernel.org
[mailto:linux-fsdevel-owner@vger.kernel.org] On Behalf Of Joseph D. Wagner
Sent: Tuesday, July 20, 2004 12:25 PM
To: 'Jan Hudec'
Cc: linux-fsdevel@vger.kernel.org
Subject: RE: RFC: Illegal Characters in File Names

> Read the list again, please. Ebcdic is included there.

OK, that was an oversimplification, but guess what?  They are all still
non-printing control characters with the exception of 127.

Joseph D. Wagner

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RFC: Illegal Characters in File Names

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 2847 bytes --]

On Tue, 2004-07-20 11:25:48 -0500, Joseph D. Wagner <theman@josephdwagner.info>
wrote in message <20040720162549.857014B7E7@dvmwest.gt.owl.de>:
> > See, it's only a question of *display*.
> 
> No, it's not.  Some programs use file names as input.  Placing a
> timestamp directly in the file name comes to mind, but I'm sure you
> can think of many other examples.  Control characters in these file

What't wrong with creating a strftime()'d filename? That's still
permitted if you allow not-easily-displayable characters.

> names would screw-up these programs.  While you're going to argue

They need to properly check their input. If they don't, they're broken
by definition.

> that a program should anticipate malformed data, I'm going to argue
> that application development will be easier for Linux is programmers
> knew that those characters aren't going to be there.

Application development with C isn't easy. It needs quite some skills to
do The Right Thing (TM). I think that a good program is made up from
about some 20% of working code and 80% of error handling.

I've seen mission-critical apps that do near-no error checking
(including input checking and the like), but you get what you pay for.

What do you want to tell all those people that *rely* on being able to
store chars < 0x20 in filenames, or > 0x7f ?

> Where do most Linux programmers come from?  Sure there's the same

>From C land. And those writing good applications already learned
the basics.

> core of people who have been programming Linux for years, but the
> new ones are coming from Windows platforms.  Which is easier: to

Windows, and previously DOS, allowed for strange filenames, too.

> reeducate every single new Linux programmer that any character
> could appear in a file name, or to cut out some of the characters
> that don't make sense?

Ever tried to read a russion DOS floppy disk in a US-english Windows
PeeCee? Ever thought about UTF8 or UTF32?

These days, you can't even be sure that one char is one displayed
character *at all*. ...and while we're hard working (though slowly)
getting towards multi-character encodings, you try to propose not using
the full 8 bits available per byte?

Get real: Some lazy Ex-Windows programmers first need to properly
learn programming in the sense of not "make this work", but "make this
to never break". You can't trust in anything, except "input *can*
actually be even *worse* than this bullshit my customer just showed
me."

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
   ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

RE: RFC: Illegal Characters in File Names

[Guy]

You said:
"It's withholding function from the user because he might abuse it and hurt 
himself."

I say:
You say "user", you forget that Linux is a multi user OS.  What if another
user created a file in /tmp that effected another user, or root?

This is a security risk and maybe a bug.  This would not be the first bug
that someone depends on!  So we stop fixing bugs because of compatibility
issues?

You said:
"BUT: I think it makes great sense as a filesystem option"

I say:
I prefer it to be fixed, but would allow a filesystem option to enable the
control characters.  The default should be to not allow the control
characters.  Again, security risk!

Guy

-----Original Message-----
From: linux-fsdevel-owner@vger.kernel.org
[mailto:linux-fsdevel-owner@vger.kernel.org] On Behalf Of Bryan Henderson
Sent: Tuesday, July 20, 2004 12:41 PM
To: Jan Hudec
Cc: linux-fsdevel@vger.kernel.org; Joseph D. Wagner
Subject: Re: RFC: Illegal Characters in File Names

>> Additionally, security vulnerabilities (now patched) have resulted
>> from the allowed use of control characters.
>
>Filenames are input.  They must be validated.  Filesystem should not fix
>broken applications.

This is the well known conflict between what's philosophically right and 
what's practical.  Same as when a browser maker refuses to change his 
browser so that his user can use his bank's web site because "The problem 
is the bank's, because the bank chose to run a web server that does the 
wrong thing."

In this case, we all know that people _are_ going to neglect to validate 
filenames.  They also will neglect to use the options on 'ls' that make it 
show you what you think it's showing you.  In some cases, these will cause 
security vulnerabilities.  We can't change that.

So even though filesystem design shouldn't have to solve that problem, 
it's a very practical place to solve it.  (Actually, not _solve_ it, 
because you can't force a user to use your filesystem type, but at least 
ameliorate it).

I am opposed to denying users the ability to use nonprinting characters in 
their filenames, because it is an unnecessary withholding of function. 
It's withholding function from the user because he might abuse it and hurt 
himself.  Plus, it's a compatibility issue.  BUT: I think it makes great 
sense as a filesystem option (I mean an option of the actual filesystem -- 
set with mkfs).  If someone wants his filesystem to police this sort of 
failure, that's a good service to provide.

--
Bryan Henderson                          IBM Almaden Research Center
San Jose CA                              Filesystems
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RFC: Illegal Characters in File Names

[viro]

On Tue, Jul 20, 2004 at 12:54:15PM -0400, Guy wrote:
> You said:
> "It's withholding function from the user because he might abuse it and hurt 
> himself."
> 
> I say:
> You say "user", you forget that Linux is a multi user OS.  What if another
> user created a file in /tmp that effected another user, or root?
> 
> This is a security risk and maybe a bug.  This would not be the first bug
> that someone depends on!  So we stop fixing bugs because of compatibility
> issues?

What security risk?  You mean running applications written by lusers?
That's a serious one, of course, but how would anything short of rm(1)
and forced (re)education of idiots who wrote the crap in question help
it?

BTW, if your terminal is misconfigured badly enough to feed program output
back as input, you really ought to fix it, and not just because of ls(1)...

RE: RFC: Illegal Characters in File Names

[Bryan Henderson]

>>It's withholding function from the user because he might abuse it and 
hurt 
>>himself.
>
>You say "user", you forget that Linux is a multi user OS.  What if 
another
>user created a file in /tmp that affected another user, or root?

In this case, the one user is the owner of the system, and you're 
proposing to stop him from offering a dangerous system to _his_ users (the 
ones who would look at a file someone else created in /tmp without 
properly handling the filename).

I still can't support withholding from the system owner the ability to run 
a system with dangerous filenames.  Linux development is about providing 
tools to people who build computer systems, not controlling the way the 
world computes.

>The default should be to not allow the control characters.

I agree, only because the naive user implicitly assumes this problem 
doesn't exist and it would be misleading to give him a system where it 
does.  I'm against misleading.

Of course, if someone comes up with a significant compatibility issue (a 
case where people assume based on history that control characters in 
filenames are OK), I could go the other way.

--
Bryan Henderson                          IBM Almaden Research Center
San Jose CA                              Filesystems

Re: RFC: Illegal Characters in File Names

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 726 bytes --]

On Tue, 20 Jul 2004 11:25:48 -0500 "Joseph D. Wagner"
<theman@josephdwagner.info> wrote:
>
> Where do most Linux programmers come from?  Sure there's the same core
> of people who have been programming Linux for years, but the new ones
> are coming from Windows platforms.  Which is easier: to reeducate every
> single new Linux programmer that any character could appear in a file
> name, or to cut out some of the characters that don't make sense?

I suggest you find out what characters can appear in filenames on Windows
based files systems before you try to use that argument - you will obviously
be suprised ... :-)

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au
http://www.canb.auug.org.au/~sfr/

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

RE: RFC: Illegal Characters in File Names

[Guy]

I think you are confused!

I am not talking about lusers creating applications!  Just empty files with
very controlled filenames.  The file names would contain the control
characters to affect the terminal when anyone (including root) issued the
"ll" command in /tmp.

Many people use VT100 compatible terminal emulators.  If you send a
correctly formated string of control characters to a vt100, it will respond.
It must!  That's what it does!  I believe someone could send characters to
the VT100 that will cause it to respond with something that would be as if
the user had typed the text (or control codes).  If that text matched
another script in the /tmp dir then the root user may cause the script to
run just by typing "ll" in the /tmp dir.  This is similar to something I did
20+ years ago on a DG AOS system.

Have you ever run cat on a binary file and got trash on the screen, then a
few "command not found" errors?  This is similar to what I am saying.  But
in my case the command would be found!

I don't know the VT100 commands, so I can't create an example.  Maybe if I
get bored I will look into it.  But if I am successful, I would submit the
info to CERT.  Then Linux would have a published security risk!  I don't
want that to happen.  Fix it before CERT finds out about it!

You said:
" What security risk?  You mean running applications written by lusers?
That's a serious one, of course, but how would anything short of rm(1)
and forced (re)education of idiots who wrote the crap in question help
it?"

I said:
You lost me!  If someone creates a script or program and puts it in /tmp
anyone could run it, even root.  The script could do anything.

Guy

-----Original Message-----
From: viro@www.linux.org.uk [mailto:viro@www.linux.org.uk] On Behalf Of
viro@parcelfarce.linux.theplanet.co.uk
Sent: Tuesday, July 20, 2004 2:10 PM
To: Guy
Cc: 'Bryan Henderson'; 'Jan Hudec'; linux-fsdevel@vger.kernel.org; 'Joseph
D. Wagner'
Subject: Re: RFC: Illegal Characters in File Names

On Tue, Jul 20, 2004 at 12:54:15PM -0400, Guy wrote:
> You said:
> "It's withholding function from the user because he might abuse it and
hurt 
> himself."
> 
> I say:
> You say "user", you forget that Linux is a multi user OS.  What if another
> user created a file in /tmp that effected another user, or root?
> 
> This is a security risk and maybe a bug.  This would not be the first bug
> that someone depends on!  So we stop fixing bugs because of compatibility
> issues?

What security risk?  You mean running applications written by lusers?
That's a serious one, of course, but how would anything short of rm(1)
and forced (re)education of idiots who wrote the crap in question help
it?

BTW, if your terminal is misconfigured badly enough to feed program output
back as input, you really ought to fix it, and not just because of ls(1)...

Re: RFC: Illegal Characters in File Names

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 1261 bytes --]

On Tue, Jul 20, 2004 at 19:10:06 +0100, viro@parcelfarce.linux.theplanet.co.uk wrote:
> On Tue, Jul 20, 2004 at 12:54:15PM -0400, Guy wrote:
> > You said:
> > "It's withholding function from the user because he might abuse it and hurt 
> > himself."
> > 
> > I say:
> > You say "user", you forget that Linux is a multi user OS.  What if another
> > user created a file in /tmp that effected another user, or root?
> > 
> > This is a security risk and maybe a bug.  This would not be the first bug
> > that someone depends on!  So we stop fixing bugs because of compatibility
> > issues?
> 
> What security risk?  You mean running applications written by lusers?
> That's a serious one, of course, but how would anything short of rm(1)
> and forced (re)education of idiots who wrote the crap in question help
> it?

I agree with this.

> BTW, if your terminal is misconfigured badly enough to feed program output
> back as input, you really ought to fix it, and not just because of ls(1)...

That means, that the terminal should never respond to any escape
sequences sent to it. In reality, it responds to some...

-------------------------------------------------------------------------------
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

RE: RFC: Illegal Characters in File Names

[Guy]

-----Original Message-----
Cut---
>> BTW, if your terminal is misconfigured badly enough to feed program
output
>> back as input, you really ought to fix it, and not just because of
ls(1)...

>That means, that the terminal should never respond to any escape
>sequences sent to it. In reality, it responds to some...

So, I guess using a screen editor is out!  No cursor control!  Can't use vi
anymore.  Everyone must hack their terminals and/or terminal emulators to
play safe with Linux!  You got to be kidding!!!!!!  Wait until Microsoft
finds out about this!  Bill will laugh so hard he will shit!

Guy

There are 10 types of people in the world.
Those that know binary and those that don't.

Re: RFC: Illegal Characters in File Names

[Matthew Wilcox]

On Tue, Jul 20, 2004 at 04:44:08PM -0400, Guy wrote:
> I am not talking about lusers creating applications!  Just empty files with
> very controlled filenames.  The file names would contain the control
> characters to affect the terminal when anyone (including root) issued the
> "ll" command in /tmp.

Then your ls is broken.  Mine displays them:

$ touch ^D
$ ls -l
-rw-r--r--    1 willy    willy           0 Jul 20 17:26 ?

See the -N, -q, -b options to ls.  You're not the first person to think
about this problem.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

Re: RFC: Illegal Characters in File Names

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 1175 bytes --]

On Tue, Jul 20, 2004 at 17:09:41 -0400, Guy wrote:
> -----Original Message-----
> Cut---
> >> BTW, if your terminal is misconfigured badly enough to feed program
> output
> >> back as input, you really ought to fix it, and not just because of
> ls(1)...
> 
> >That means, that the terminal should never respond to any escape
> >sequences sent to it. In reality, it responds to some...
> 
> So, I guess using a screen editor is out!  No cursor control!  Can't use vi
> anymore.  Everyone must hack their terminals and/or terminal emulators to
> play safe with Linux!  You got to be kidding!!!!!!  Wait until Microsoft
> finds out about this!  Bill will laugh so hard he will shit!

But a curses application does not need the terminal to respond to
anything!!!! It only needst the terminal to obey the commands.

Actualy I failed to find any sequences that would cause response from
the terminal in termcap(5). Such sequences are not needed.
(With partial exception of mouse input -- but that is input, you can't
force particular).

-------------------------------------------------------------------------------
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: RFC: Illegal Characters in File Names

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 834 bytes --]

On Tue, Jul 20, 2004 at 22:27:53 +0100, Matthew Wilcox wrote:
> On Tue, Jul 20, 2004 at 04:44:08PM -0400, Guy wrote:
> > I am not talking about lusers creating applications!  Just empty files with
> > very controlled filenames.  The file names would contain the control
> > characters to affect the terminal when anyone (including root) issued the
> > "ll" command in /tmp.
> 
> Then your ls is broken.  Mine displays them:
> 
> $ touch ^D
> $ ls -l
> -rw-r--r--    1 willy    willy           0 Jul 20 17:26 ?
> 
> See the -N, -q, -b options to ls.  You're not the first person to think
> about this problem.

Try
ls -l | cat
;-)

Ls is clever. When it sees tty, it avoids control chars....

-------------------------------------------------------------------------------
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: RFC: Illegal Characters in File Names

[Matthew Wilcox]

On Tue, Jul 20, 2004 at 11:37:25PM +0200, Jan Hudec wrote:
> > See the -N, -q, -b options to ls.  You're not the first person to think
> > about this problem.
> 
> Try
> ls -l | cat
> ;-)
> 
> Ls is clever. When it sees tty, it avoids control chars....

Sure.  But that's not really relevant, is it?  If you say to CERT "Linux
sucks because you could type ls | cat and it'll fuck up your terminal",
they'll laugh at you.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

RE: RFC: Illegal Characters in File Names

[Bryan Henderson]

>I am not talking about lusers creating applications!  Just empty files 
with
>very controlled filenames.  The file names would contain the control
>characters to affect the terminal when anyone (including root) issued the
>"ll" command in /tmp.

Sure you are (talking about lusers creating applications).  You're talking 
about the luser that created an 'll' command that sends control characters 
that it reads from a directory to the terminal.

>Have you ever run cat on a binary file and got trash on the screen, then 
a
>few "command not found" errors?

So doesn't your logic demand that we restrict the contents of files to 
printable characters?

I think the only difference is one of numbers:  likelihood someone 
displays a raw filename with terminal control sequences vs likelihood that 
someone displays a raw file with terminal control sequences.  Pure 
principle argues for letting the filenames contain control characters and 
blaming the author of ls, creator of ll alias, teacher of the ll user, 
etc.

--
Bryan Henderson                          IBM Almaden Research Center
San Jose CA                              Filesystems

Re: RFC: Illegal Characters in File Names

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 806 bytes --]

On Tue, Jul 20, 2004 at 22:40:57 +0100, Matthew Wilcox wrote:
> On Tue, Jul 20, 2004 at 11:37:25PM +0200, Jan Hudec wrote:
> > > See the -N, -q, -b options to ls.  You're not the first person to think
> > > about this problem.
> > 
> > Try
> > ls -l | cat
> > ;-)
> > 
> > Ls is clever. When it sees tty, it avoids control chars....
> 
> Sure.  But that's not really relevant, is it?  If you say to CERT "Linux
> sucks because you could type ls | cat and it'll fuck up your terminal",
> they'll laugh at you.

"Linux sucks because you could type cat something and it'll fuck up your
terminal".

People don't type ls | cat by mistake. They do it on purpose, usualy.

-------------------------------------------------------------------------------
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

RE: RFC: Illegal Characters in File Names

[Guy]

What about:
ls|more

-----Original Message-----
From: Jan Hudec [mailto:bulb@vagabond.light.src] On Behalf Of Jan Hudec
Sent: Tuesday, July 20, 2004 5:45 PM
To: Matthew Wilcox
Cc: Guy; viro@parcelfarce.linux.theplanet.co.uk; 'Bryan Henderson';
linux-fsdevel@vger.kernel.org; 'Joseph D. Wagner'
Subject: Re: RFC: Illegal Characters in File Names

On Tue, Jul 20, 2004 at 22:40:57 +0100, Matthew Wilcox wrote:
> On Tue, Jul 20, 2004 at 11:37:25PM +0200, Jan Hudec wrote:
> > > See the -N, -q, -b options to ls.  You're not the first person to
think
> > > about this problem.
> > 
> > Try
> > ls -l | cat
> > ;-)
> > 
> > Ls is clever. When it sees tty, it avoids control chars....
> 
> Sure.  But that's not really relevant, is it?  If you say to CERT "Linux
> sucks because you could type ls | cat and it'll fuck up your terminal",
> they'll laugh at you.

"Linux sucks because you could type cat something and it'll fuck up your
terminal".

People don't type ls | cat by mistake. They do it on purpose, usualy.

----------------------------------------------------------------------------
---
						 Jan 'Bulb' Hudec
<bulb@ucw.cz>

Re: RFC: Illegal Characters in File Names

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 238 bytes --]

On Tue, Jul 20, 2004 at 17:49:07 -0400, Guy wrote:
> What about:
> ls|more

Hm, seems more is broken.
ls | less is OK.

-------------------------------------------------------------------------------
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: RFC: Illegal Characters in File Names

[Paul Stewart]

On 2004.07.20 14:49, Guy wrote:
> What about:
> ls|more

Hm.. I've tried this under rxvt, xterm and gnome-terminal and none of  
these respond to the "ls | cat", "ls" or "ls | more" scenarios in any  
unexpected way.  I think we're straying from the point.  If the  
argument is "the filesystem should change because some terminal  
applications might be coerced into responding in vulnerable ways", the  
answer is that the terminal application should be changed.

This is not just about philosophical purity or laziness about who  
should "protect the user".  There are various vectors for getting at  
the output to the terminal.  "ls | cat" is only one of them.  I could  
imagine mail content, and even file content being vulnerable.  If you  
saw a file in /tmp/ named "README_GUY", wouldn't you be the least bit  
tempted to have a look at what was inside?  If we're going to follow  
this branch of the argument, we've got to correctly determine the  
threat to terminals and consider changes to the terminal to truly have  
a story about preventing them.  Alas, this discussion is out of scope  
for fs-devel.

--
Paul

Re: RFC: Illegal Characters in File Names

[viro]

On Tue, Jul 20, 2004 at 05:09:41PM -0400, Guy wrote:
> -----Original Message-----
> Cut---
> >> BTW, if your terminal is misconfigured badly enough to feed program
> output
> >> back as input, you really ought to fix it, and not just because of
> ls(1)...
> 
> >That means, that the terminal should never respond to any escape
> >sequences sent to it.

No, it does not.
 
> So, I guess using a screen editor is out!  No cursor control!  Can't use vi
> anymore.  Everyone must hack their terminals and/or terminal emulators to
> play safe with Linux!  You got to be kidding!!!!!!  Wait until Microsoft
> finds out about this!  Bill will laugh so hard he will shit!

I'm not quite sure why the last one is of any interest for anybody except
aforementioned Bill and maybe his proctologist, but anyway.

You claimed the following:

"If someone used the above method, they may be able to cause a simple "ll"
command to cause another program to be run!"

Mind showing the sequence that would achieve that?

RE: RFC: Illegal Characters in File Names

[Joseph D. Wagner]

> People don't type ls | cat by mistake. They do it on purpose, usualy.

But they shouldn't be able to do it at all, which is the whole original point of this discussion.  In other words, people shouldn't be able to do things that harm the system.

Joseph D. Wagner

RE: RFC: Illegal Characters in File Names

[Joseph D. Wagner]

> What security risk?  You mean running applications written by lusers?

But that's the million dollar question, Regis.  How do you know if the application is any good or not?  Not every program that runs on Linux is open source.  Not every end user running Linux is skilled enough to read the source code.

And don't give me that crap about how a proprietary company's software is implied to be better.  Microsoft is a multi-billion dollar proprietary software company, and they can't even fix bugs in IE.  How can you place such demands on other programmers?

I'm afraid, however, that Bryan Henderson's analysis is correct when he said:

> This is the well known conflict between what's
> philosophically right and what's practical. [trimmed]
> So even though filesystem design shouldn't have to
> solve that problem, it's a very practical place to
> solve it.

I'm afraid this won't happen until it's a file system option.

Joseph D. Wagner

Re: RFC: Illegal Characters in File Names

[viro]

On Tue, Jul 20, 2004 at 05:16:57PM -0500, Joseph D. Wagner wrote:
 
> And don't give me that crap about how a proprietary company's software is implied to be better.  Microsoft is a multi-billion dollar proprietary software company, and they can't even fix bugs in IE.  How can you place such demands on other programmers?

Excuse me?  If your idea of decent programmer is "duhveloper employed by
MS and unable to find better job than working on IE", I *really* hope that
none of your code will ever get anywhere near my boxen.  I happen to have
somewhat higher standards than _that_.

Re: RFC: Illegal Characters in File Names

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 1718 bytes --]

On Tue, Jul 20, 2004 at 23:13:28 +0100, viro@parcelfarce.linux.theplanet.co.uk wrote:
> On Tue, Jul 20, 2004 at 05:09:41PM -0400, Guy wrote:
> > -----Original Message-----
> > Cut---
> > >> BTW, if your terminal is misconfigured badly enough to feed program
> > output
> > >> back as input, you really ought to fix it, and not just because of
> > ls(1)...
> > 
> > >That means, that the terminal should never respond to any escape
> > >sequences sent to it.
> 
> No, it does not.

Xterm does.

> > So, I guess using a screen editor is out!  No cursor control!  Can't use vi
> > anymore.  Everyone must hack their terminals and/or terminal emulators to
> > play safe with Linux!  You got to be kidding!!!!!!  Wait until Microsoft
> > finds out about this!  Bill will laugh so hard he will shit!
> 
> I'm not quite sure why the last one is of any interest for anybody except
> aforementioned Bill and maybe his proctologist, but anyway.
> 
> You claimed the following:
> 
> "If someone used the above method, they may be able to cause a simple "ll"
> command to cause another program to be run!"
> 
> Mind showing the sequence that would achieve that?

Xterm seems to have several escape sequences that cause data to be sent
back to the program. Only one that seems capable of sending data
affected by user seems to be 'get cursor position' though (\e[6n) and it
will only return an "\e[%i;%iR" (where %i get substitued by the cursor
position -- as decimal numbers).

If any terminal had "get title" sequence, it would be a problem. I think
none has, though.

-------------------------------------------------------------------------------
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: RFC: Illegal Characters in File Names

[viro]

On Wed, Jul 21, 2004 at 12:44:46AM +0200, Jan Hudec wrote:
> > "If someone used the above method, they may be able to cause a simple "ll"
> > command to cause another program to be run!"
> > 
> > Mind showing the sequence that would achieve that?
> 
> Xterm seems to have several escape sequences that cause data to be sent
> back to the program. Only one that seems capable of sending data
> affected by user seems to be 'get cursor position' though (\e[6n) and it
> will only return an "\e[%i;%iR" (where %i get substitued by the cursor
> position -- as decimal numbers).

Again, the claim was

"If someone used the above method, they may be able to cause a simple "ll"
command to cause another program to be run!"

Still waiting for the examples of that...

RE: RFC: Illegal Characters in File Names

[Guy]

>-----Original Message-----
>From: viro@www.linux.org.uk [mailto:viro@www.linux.org.uk] On Behalf Of
>viro@parcelfarce.linux.theplanet.co.uk
>Sent: Tuesday, July 20, 2004 6:13 PM
>You claimed the following:
Cut---
>"If someone used the above method, they may be able to cause a simple "ll"
>command to cause another program to be run!"

>Mind showing the sequence that would achieve that?

As I said, I don't know how to do it, but I believe it could be done.
I did look at the VT100 esc codes.  I saw nothing obvious.  Now I need to
look at VT102 and up, xterm and ANSI!  The list may be virtually endless.

I can get "nn" sent to the computer, but no <new line>, yet.

Guy

Re: RFC: Illegal Characters in File Names

[John Newbigin]

viro@parcelfarce.linux.theplanet.co.uk wrote:
...
> "If someone used the above method, they may be able to cause a simple "ll"
> command to cause another program to be run!"
> 
> Mind showing the sequence that would achieve that?

http://www.kb.cert.org/vuls/id/230561
http://www.digitaldefense.net/labs/papers/Termulation.txt
Read this page ^^^^ !!!!

These are poor protocols which were not designed for security :(

It is a real problem.  The solution is to fix the terminal though, not 
the filesystem.

Anyone with "mesy y" may be vulnerable to a host of malicious or just 
fun tricks.  (set font color to black on black is my fav.)

If you must filter your file names, why not build a filter mechanism 
which can do this?

John.

-- 
John Newbigin - Computer Systems Officer
School of Information Technology
Swinburne University of Technology
Melbourne, Australia
http://www.it.swin.edu.au/staff/jnewbigin

RE: RFC: Illegal Characters in File Names

[Joseph D. Wagner]

>> Mind showing the sequence that would achieve that?

> http://www.kb.cert.org/vuls/id/230561
> http://www.digitaldefense.net/labs/papers/Termulation.txt
> Read this page ^^^^ !!!!
> 
> These are poor protocols which were not designed for security :(
> 
> It is a real problem.  The solution is to fix the terminal though, not
> the filesystem.

The problem with fixing only the terminal is that there's more than one program that can be used for accessing a file name.  By your logic, we should also "fix" every single program that can access a file name.

I think it should be fixed in both places.

To use an analogy, I want to put a deadbolt on a door, but you don't want me to because the door is already locked.  Sure, the door is locked, but we'd all be better off if it had a deadbolt, too.

Joseph D. Wagner

Re: RFC: Illegal Characters in File Names

[viro]

On Tue, Jul 20, 2004 at 10:26:25PM -0500, Joseph D. Wagner wrote:
 
> The problem with fixing only the terminal is that there's more than one program that can be used for accessing a file name.  By your logic, we should also "fix" every single program that can access a file name.

What makes you think that "accessing a file name" is a problem and absense
of characters in range 1--31 is likely to solve that problem, whatever it
might be?

RE: RFC: Illegal Characters in File Names

[Guy]

>From what I have seen, all terminal control codes start with esc.  But I
have only looked at very few terminal types.  Maybe some other terminal
types don't use esc codes.

Maybe only the esc key 0x1b needs to be disallowed.

I would still prefer to disallow all ascii codes below 0x20.

Guy

-----Original Message-----
From: linux-fsdevel-owner@vger.kernel.org
[mailto:linux-fsdevel-owner@vger.kernel.org] On Behalf Of John Newbigin
Sent: Tuesday, July 20, 2004 7:52 PM
To: viro@parcelfarce.linux.theplanet.co.uk
Cc: Guy; 'Jan Hudec'; 'Bryan Henderson'; linux-fsdevel@vger.kernel.org;
'Joseph D. Wagner'
Subject: Re: RFC: Illegal Characters in File Names

viro@parcelfarce.linux.theplanet.co.uk wrote:
...
> "If someone used the above method, they may be able to cause a simple "ll"
> command to cause another program to be run!"
> 
> Mind showing the sequence that would achieve that?

http://www.kb.cert.org/vuls/id/230561
http://www.digitaldefense.net/labs/papers/Termulation.txt
Read this page ^^^^ !!!!

These are poor protocols which were not designed for security :(

It is a real problem.  The solution is to fix the terminal though, not 
the filesystem.

Anyone with "mesy y" may be vulnerable to a host of malicious or just 
fun tricks.  (set font color to black on black is my fav.)

If you must filter your file names, why not build a filter mechanism 
which can do this?

John.

-- 
John Newbigin - Computer Systems Officer
School of Information Technology
Swinburne University of Technology
Melbourne, Australia
http://www.it.swin.edu.au/staff/jnewbigin

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RFC: Illegal Characters in File Names

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1792 bytes --]

On Tue, 2004-07-20 16:44:08 -0400, Guy <bugzilla@watkins-home.com>
wrote in message <200407202044.i6KKi8318014@watkins-home.com>:
> I think you are confused!
> 
> I am not talking about lusers creating applications!  Just empty files with
> very controlled filenames.  The file names would contain the control
> characters to affect the terminal when anyone (including root) issued the
> "ll" command in /tmp.

Then, the local system's configuration is just missing a "-b" for ls.
...or you're using a broken terminal. In no way should ls be called with
"-N", except you ask for trouble...

> Have you ever run cat on a binary file and got trash on the screen, then a
> few "command not found" errors?  This is similar to what I am saying.  But
> in my case the command would be found!

Why should I be dumb enough to shoot my foot? Specifically "root"
shouldn't do such nonsense:)

> I don't know the VT100 commands, so I can't create an example.  Maybe if I

console_codes(4) may be a start.

> You said:
> " What security risk?  You mean running applications written by lusers?
> That's a serious one, of course, but how would anything short of rm(1)
> and forced (re)education of idiots who wrote the crap in question help
> it?"
> 
> I said:
> You lost me!  If someone creates a script or program and puts it in /tmp
> anyone could run it, even root.  The script could do anything.

How should "someone" make _me_ run his program?

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
   ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: RFC: Illegal Characters in File Names

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 675 bytes --]

On Tue, 2004-07-20 23:45:22 +0200, Jan Hudec <bulb@ucw.cz>
wrote in message <20040720214522.GQ3227@vagabond>:

> "Linux sucks because you could type cat something and it'll fuck up your
> terminal".

"Linux sucks because you could type su -c rm -rf / and it'll fuck up
your system after you enter root's password."

You get what you deserve...

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
   ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: RFC: Illegal Characters in File Names

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 858 bytes --]

On Tue, 2004-07-20 17:16:52 -0500, Joseph D. Wagner <theman@josephdwagner.info>
wrote in message <S266341AbUGTWQ7/20040720221700Z+361@vger.kernel.org>:
> > People don't type ls | cat by mistake. They do it on purpose, usualy.
> 
> But they shouldn't be able to do it at all, which is the whole original
> point of this discussion.  In other words, people shouldn't be able to
> do things that harm the system.

They *should* be able to do anything they want. It's Unix -- you're the
boss, system does what you request from it.

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
   ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: RFC: Illegal Characters in File Names

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1001 bytes --]

On Tue, 2004-07-20 17:09:41 -0400, Guy <bugzilla@watkins-home.com>
wrote in message <200407202109.i6KL9f318138@watkins-home.com>:
> >That means, that the terminal should never respond to any escape
> >sequences sent to it. In reality, it responds to some...
> 
> So, I guess using a screen editor is out!  No cursor control!  Can't use vi
> anymore.  Everyone must hack their terminals and/or terminal emulators to

In fact, you can use vi without cursor keys. Use h, j, k and l instead.

> play safe with Linux!  You got to be kidding!!!!!!  Wait until Microsoft
> finds out about this!  Bill will laugh so hard he will shit!

Care to take a digital photo of that?

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
   ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: RFC: Illegal Characters in File Names

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 2631 bytes --]

On Tue, 2004-07-20 17:16:57 -0500, Joseph D. Wagner <theman@josephdwagner.info>
wrote in message <S266335AbUGTWQ6/20040720221658Z+360@vger.kernel.org>:
> > What security risk?  You mean running applications written by lusers?
> 
> But that's the million dollar question, Regis.  How do you know if the
> application is any good or not?  Not every program that runs on Linux

When it's a FSF GNU application and some 10 years around, I tend to
think that it's old enough to have all major problems removed.

> is open source.  Not every end user running Linux is skilled enough to
> read the source code.

I think you'll face a hard time finding such badly written software on
most systems, and especially those ran (sp?) by root are not of that
slice. (Except root is incompetent and likes to start KDE or Gnome...)

> And don't give me that crap about how a proprietary company's software
> is implied to be better.  Microsoft is a multi-billion dollar
> proprietary software company, and they can't even fix bugs in IE.
> How can you place such demands on other programmers?

In Germany, we've got sentence, "Ist der Ruf erst ruiniert, lebt sich's
völlig ungeniert", meaning something like if you did some bad things,
nobody will mind if you do even more of them.

Open source programmers will get blamed *personally* for the bullshit
they produce. Nobody likes to write security advisories. It's *pain*. At
large software companies, there's no such blame, no such pain. I guess
commonly those people writing the advisories won't have seen the code at
all. Bugs just happen as systemic failures, nobody is personally blamed
for them. That's different for open-sources software, because you know
the author, sometimes even personally.

> I'm afraid, however, that Bryan Henderson's analysis is correct when he said:
> 
> > This is the well known conflict between what's
> > philosophically right and what's practical. [trimmed]
> > So even though filesystem design shouldn't have to
> > solve that problem, it's a very practical place to
> > solve it.
> 
> I'm afraid this won't happen until it's a file system option.

If you were to introduce an additional "check_filename()" system call,
you'd just put a function of the very same name into the application...

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
   ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

RE: RFC: Illegal Characters in File Names

[Guy]

?-----Original Message-----
?From: Jan-Benedict Glaw [mailto:jbglaw@lug-owl.de] 
?Sent: Wednesday, July 21, 2004 8:22 AM
?To: Guy
?Cc: viro@parcelfarce.linux.theplanet.co.uk; 'Bryan Henderson'; 'Jan Hudec';
?linux-fsdevel@vger.kernel.org; 'Joseph D. Wagner'
?Subject: Re: RFC: Illegal Characters in File Names

?On Tue, 2004-07-20 16:44:08 -0400, Guy <bugzilla@watkins-home.com>
?wrote in message <200407202044.i6KKi8318014@watkins-home.com>:
?> I think you are confused!
?> 
?> I am not talking about lusers creating applications!  Just empty files
?with
?> very controlled filenames.  The file names would contain the control
?> characters to affect the terminal when anyone (including root) issued the
?> "ll" command in /tmp.

?Then, the local system's configuration is just missing a "-b" for ls.
?...or you're using a broken terminal. In no way should ls be called with
?"-N", except you ask for trouble...

Maybe -b should be the default for ls.  Since ls|more does the same thing.

?> Have you ever run cat on a binary file and got trash on the screen, then
?a
?> few "command not found" errors?  This is similar to what I am saying.
?But
?> in my case the command would be found!

?Why should I be dumb enough to shoot my foot? Specifically "root"
?shouldn't do such nonsense:)

Goober!  I was giving an example!  If cat <binary> file can cause a script
to run, then "ls|more" could also, if the filename had the same binary data.
You can't shoot your foot, it's in your mouth!

?> I don't know the VT100 commands, so I can't create an example.  Maybe if
?I

?console_codes(4) may be a start.

?> You said:
?> " What security risk?  You mean running applications written by lusers?
?> That's a serious one, of course, but how would anything short of rm(1)
?> and forced (re)education of idiots who wrote the crap in question help
?> it?"
?> 
?> I said:
?> You lost me!  If someone creates a script or program and puts it in /tmp
?> anyone could run it, even root.  The script could do anything.

?How should "someone" make _me_ run his program?

One day when you do "ls|more" in a directory that they have created 2 or
more scripts.  You seem to have a hard time understanding this concept.

?MfG, JBG

?-- 
?   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
?   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen
?Krieg
?    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
?   ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM |
?TCPA));

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: RFC: Illegal Characters in File Names

[Guy]

Again, your missing the point.
Sure it I want to do "ls|cat" or "ls|more" I can.  But this simple, very
common set of commands should not be able to trash your system!

-----Original Message-----
From: Jan-Benedict Glaw [mailto:jbglaw@lug-owl.de] 
Sent: Wednesday, July 21, 2004 8:26 AM
To: Joseph D. Wagner
Cc: 'Jan Hudec'; 'Matthew Wilcox'; 'Guy';
viro@parcelfarce.linux.theplanet.co.uk; 'Bryan Henderson';
linux-fsdevel@vger.kernel.org
Subject: Re: RFC: Illegal Characters in File Names

On Tue, 2004-07-20 17:16:52 -0500, Joseph D. Wagner
<theman@josephdwagner.info>
wrote in message <S266341AbUGTWQ7/20040720221700Z+361@vger.kernel.org>:
> > People don't type ls | cat by mistake. They do it on purpose, usualy.
> 
> But they shouldn't be able to do it at all, which is the whole original
> point of this discussion.  In other words, people shouldn't be able to
> do things that harm the system.

They *should* be able to do anything they want. It's Unix -- you're the
boss, system does what you request from it.

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
   ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM |
TCPA));

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: RFC: Illegal Characters in File Names

[Guy]

What does cursor keys have to do with the conversation?
Please re-read the post this is based on.

-----Original Message-----
From: Jan-Benedict Glaw [mailto:jbglaw@lug-owl.de] 
Sent: Wednesday, July 21, 2004 8:28 AM
To: Guy
Cc: 'Jan Hudec'; viro@parcelfarce.linux.theplanet.co.uk; 'Bryan Henderson';
linux-fsdevel@vger.kernel.org; 'Joseph D. Wagner'
Subject: Re: RFC: Illegal Characters in File Names

On Tue, 2004-07-20 17:09:41 -0400, Guy <bugzilla@watkins-home.com>
wrote in message <200407202109.i6KL9f318138@watkins-home.com>:
> >That means, that the terminal should never respond to any escape
> >sequences sent to it. In reality, it responds to some...
> 
> So, I guess using a screen editor is out!  No cursor control!  Can't use
vi
> anymore.  Everyone must hack their terminals and/or terminal emulators to

In fact, you can use vi without cursor keys. Use h, j, k and l instead.

> play safe with Linux!  You got to be kidding!!!!!!  Wait until Microsoft
> finds out about this!  Bill will laugh so hard he will shit!

Care to take a digital photo of that?

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
   ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM |
TCPA));

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RFC: Illegal Characters in File Names

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1060 bytes --]

On Wed, 2004-07-21 11:28:36 -0400, Guy <bugzilla@watkins-home.com>
wrote in message <200407211528.i6LFSa322018@watkins-home.com>:
> Again, your missing the point.
> Sure it I want to do "ls|cat" or "ls|more" I can.  But this simple, very
> common set of commands should not be able to trash your system!

If you're a correct-paranoid user (or root), then you make sure that
your "ls" won't be ever show "raw" filenames, but use some kind of
hex-encoding or whatever. It's up to the user, and defaults seem to be
sensible. Plain ls (at least on my system) will not output control
sequences to a tty and fed-out raw data if it's output isn't a tty.

If you get input (eg. more, less etc.) you've got to await anything,
always.

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
   ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: RFC: Illegal Characters in File Names

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 793 bytes --]

On Wed, 2004-07-21 11:30:44 -0400, Guy <bugzilla@watkins-home.com>
wrote in message <200407211530.i6LFUi322033@watkins-home.com>:
> What does cursor keys have to do with the conversation?
> Please re-read the post this is based on.

The claim was that vi cannot be used without cursor keys. But that's not
true.

> > So, I guess using a screen editor is out!  No cursor control!  Can't use
> vi
> > anymore.  Everyone must hack their terminals and/or terminal emulators to

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
   ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: RFC: Illegal Characters in File Names

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 1158 bytes --]

On Wed, Jul 21, 2004 at 18:26:41 +0200, Jan-Benedict Glaw wrote:
> On Wed, 2004-07-21 11:30:44 -0400, Guy <bugzilla@watkins-home.com>
> wrote in message <200407211530.i6LFUi322033@watkins-home.com>:
> > What does cursor keys have to do with the conversation?
> > Please re-read the post this is based on.
> 
> The claim was that vi cannot be used without cursor keys. But that's not
> true.

The claim really was that vi cannot be used if the terminal can't report
back the cursor position or other things. That's not true either.

> > > So, I guess using a screen editor is out!  No cursor control!  Can't use
> > vi
> > > anymore.  Everyone must hack their terminals and/or terminal emulators to
> 
> -- 
>    Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
>    "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
>     fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
>    ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));


-------------------------------------------------------------------------------
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

RE: RFC: Illegal Characters in File Names

[Guy]

No, the claim was to disable control codes processed my the terminal.  That
would disable the cursor control and more.  Cursor control is the ability to
have the software control the screen and cursor.  Without cursor controls,
no screen editors.  No comments about disabling data from the terminal.  So
you can use all the function keys you want.

-----Original Message-----
From: Jan-Benedict Glaw [mailto:jbglaw@lug-owl.de] 
Sent: Wednesday, July 21, 2004 12:27 PM
To: Guy
Cc: 'Jan Hudec'; viro@parcelfarce.linux.theplanet.co.uk; 'Bryan Henderson';
linux-fsdevel@vger.kernel.org; 'Joseph D. Wagner'
Subject: Re: RFC: Illegal Characters in File Names

On Wed, 2004-07-21 11:30:44 -0400, Guy <bugzilla@watkins-home.com>
wrote in message <200407211530.i6LFUi322033@watkins-home.com>:
> What does cursor keys have to do with the conversation?
> Please re-read the post this is based on.

The claim was that vi cannot be used without cursor keys. But that's not
true.

> > So, I guess using a screen editor is out!  No cursor control!  Can't use
> vi
> > anymore.  Everyone must hack their terminals and/or terminal emulators
to

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
   ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM |
TCPA));

-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RFC: Illegal Characters in File Names

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 1894 bytes --]

On Wed, Jul 21, 2004 at 12:41:25 -0400, Guy wrote:
> No, the claim was to disable control codes processed my the terminal.  That
> would disable the cursor control and more.  Cursor control is the ability to
> have the software control the screen and cursor.  Without cursor controls,
> no screen editors.  No comments about disabling data from the terminal.  So
> you can use all the function keys you want.

No, it was not. The claim was to disable terminal reporting back it's
position on request. Such request-response escape sequences only exist
on some terminals and are not needed anyway.

> -----Original Message-----
> From: Jan-Benedict Glaw [mailto:jbglaw@lug-owl.de] 
> Sent: Wednesday, July 21, 2004 12:27 PM
> To: Guy
> Cc: 'Jan Hudec'; viro@parcelfarce.linux.theplanet.co.uk; 'Bryan Henderson';
> linux-fsdevel@vger.kernel.org; 'Joseph D. Wagner'
> Subject: Re: RFC: Illegal Characters in File Names
> 
> On Wed, 2004-07-21 11:30:44 -0400, Guy <bugzilla@watkins-home.com>
> wrote in message <200407211530.i6LFUi322033@watkins-home.com>:
> > What does cursor keys have to do with the conversation?
> > Please re-read the post this is based on.
> 
> The claim was that vi cannot be used without cursor keys. But that's not
> true.
> 
> > > So, I guess using a screen editor is out!  No cursor control!  Can't use
> > vi
> > > anymore.  Everyone must hack their terminals and/or terminal emulators
> to
> 
> -- 
>    Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
>    "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
>     fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
>    ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM |
> TCPA));
> 
-------------------------------------------------------------------------------
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

RE: RFC: Illegal Characters in File Names

[Bryan Henderson]

>>"If someone used the above method, they may be able to cause a simple 
"ll"
>>command to cause another program to be run!"
>
>>Mind showing the sequence that would achieve that?
>
>As I said, I don't know how to do it, but I believe it could be done.
>I did look at the VT100 esc codes.  I saw nothing obvious.

In the mid-80's I remember this kind of attack.  It was not via a 
directory listing, but via an email message.  I believe the terminal in 
question was a Heath/Zenith H19/Z19, attached to VMS.  The terminal had a 
"read the screen" command.  At the time the attack happened, that terminal 
feature was widely recognized as one of those "seemed like a good idea at 
the time."

I believe the system people solved the immediate problem in the mail 
displaying program. 

I also note that the GNU version of the primeval Unix instant messaging 
program 'write' specifically filters out anything that it thinks might 
perform a control function on the terminal.  Not just because of a 
security problem, but just because someone shouldn't be able to control 
someone else's terminal.  (Even well meaning people used to include Clear 
Screen commands in their instant messages).


But really, the idea of making a terminal send a command is a red herring. 
 The problems of non-text filenames are more basic:

I have personally lost a lot of time in cases where 'ls' misled me as to 
the contents of my directory.  (I eventually found the --almost-all and 
--human-readable options of GNU Ls and have been much happier since).

I wouldn't stop at nonprinting characters.  Punctuation, especially spaces 
and tabs, cause a lot of trouble too, for very little gain.

As I've said, I don't want to take away your weird filenames.  But I'd 
like to get rid of mine.  I'd rather have an untar fail than create a 
filename that's going to cause me grief.


In one of my earliest encounters with a computer, someone distributed a 
program that created a status file with a command line editing character 
in its name.  He did it to try to stop people from deleting it -- it was 
impossible to delete it with a typed command.  He didn't think through 
just what effect attempting to type such a command would have.  What it 
did was create a command that says, "delete the file I most recently 
worked on."  I deleted 3 important files before I realized what was going 
on.

--
Bryan Henderson                          IBM Almaden Research Center
San Jose CA                              Filesystems

Re: RFC: Illegal Characters in File Names

[John Newbigin]

Bryan Henderson wrote:
...
> But really, the idea of making a terminal send a command is a red herring. 
>  The problems of non-text filenames are more basic:
> 
> I have personally lost a lot of time in cases where 'ls' misled me as to 
> the contents of my directory.  (I eventually found the --almost-all and 
> --human-readable options of GNU Ls and have been much happier since).
So your problem is that ls does not do what you want by default.  The 
issue of 'control codes do bad things to terminals' is as you say a red 
herring.  I see a suspect file, even if the name does not contain 
control codes, the file it's self might. By catting the contents I am 
still vulnerable for the same reason.  My tools (in this case cat) don't 
do what I want (such as escape the control codes).

In the same way, if I don't know how to use any other tool, I might be 
misled by incorrect output.


> 
> I wouldn't stop at nonprinting characters.  Punctuation, especially spaces 
> and tabs, cause a lot of trouble too, for very little gain.
These all give me the shits too, but I don't use them.  I try to educate 
other users but I don't force them not use use them.  Particularly 
windows users using samba like to use spaces.

> 
> As I've said, I don't want to take away your weird filenames.  But I'd 
> like to get rid of mine.  I'd rather have an untar fail than create a 
> filename that's going to cause me grief.
Exactly.  You should have control over your environment.  A per user 
filename filter could do this.  You can turn it on and off, configure 
special encodings whatever.  If all your filenames turn to shit, the 
system still works and you don't impact other users.  If you don't like 
the files other people leave in /tmp then use a private /tmp.  (Why 
don't we have a vfs which can make my private tmp appear on /tmp?)

Preventing these characters at the filesystem is not good because it 
impacts on people who want (perhaps misguidedly) to use them.

> 
> 
> In one of my earliest encounters with a computer, someone distributed a 
> program that created a status file with a command line editing character 
> in its name.  He did it to try to stop people from deleting it -- it was 
> impossible to delete it with a typed command.  He didn't think through 
> just what effect attempting to type such a command would have.  What it 
> did was create a command that says, "delete the file I most recently 
> worked on."  I deleted 3 important files before I realized what was going 
> on.
We know that all valid filename characters can be entered into a bash 
shell if correctly formatted & escaped.  There are no impossible to 
delete file names (except for perhaps, impossible to create file names, 
which fsck should fix).

If people write software that does silly things then that software needs 
to be fixed.  Your own admission is that the author did not think about 
what they were doing.

The unix fs is a bit like C++.  You can do heaps of stupid things (c++ 
multiple inheritance, operator overloading) but you probably shouldn't. 
  Just because you don't however, does not mean that you should remove 
them from the language.

Re: RFC: Illegal Characters in File Names

[Matthew Wilcox]

On Thu, Jul 22, 2004 at 01:17:20PM +1000, John Newbigin wrote:
> system still works and you don't impact other users.  If you don't like 
> the files other people leave in /tmp then use a private /tmp.  (Why 
> don't we have a vfs which can make my private tmp appear on /tmp?)

We do.  mount --bind.  Yes, many problems, including that you don't have
write permission on /tmp so you can't bind-mount on top of it, but the
VFS is basically there.  We just need some HOWTO that shows how to set
all the private namespace up properly.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

Re: RFC: Illegal Characters in File Names

[viro]

On Thu, Jul 22, 2004 at 01:17:20PM +1000, John Newbigin wrote:
> Exactly.  You should have control over your environment.  A per user 
> filename filter could do this.  You can turn it on and off, configure 
> special encodings whatever.  If all your filenames turn to shit, the 
> system still works and you don't impact other users.  If you don't like 
> the files other people leave in /tmp then use a private /tmp.  (Why 
> don't we have a vfs which can make my private tmp appear on /tmp?)

We do - get a namespace of your own and bind whatever you want over /tmp.

Re: RFC: Illegal Characters in File Names

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1002 bytes --]

On Wed, 2004-07-21 13:25:30 -0700, Bryan Henderson <hbryan@us.ibm.com>
wrote in message <OFBA20EFDD.E3EC0CD9-ON88256ED8.006CC054-88256ED8.0070229F@us.ibm.com>:

> I wouldn't stop at nonprinting characters.  Punctuation, especially spaces 
> and tabs, cause a lot of trouble too, for very little gain.
> 
> As I've said, I don't want to take away your weird filenames.  But I'd 
> like to get rid of mine.  I'd rather have an untar fail than create a 
> filename that's going to cause me grief.

Where's the problem? That can (at least for dynamically linked
applications -- most of what you use is that:) be done with a little
LD_PRELOAD library.

MfG, JBG

-- 
Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481             _ O _
"Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg  _ _ O
 fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!   O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: RFC: Illegal Characters in File Names

[Matthew Wilcox]

On Wed, Jul 21, 2004 at 11:25:03AM -0400, Guy wrote:
> Goober!  I was giving an example!  If cat <binary> file can cause a script

Up to this point, I've been polite.  But now you've resorted to personal
insults, allow me to continue in this vein.

You are a moron.  You have no idea what you're talking about.  Your
incessant trolling in this thread causes me to believe you are the lowest
form of pond scum.  You think it's enough to have opinions and other people
should implement code for you.  You have no understanding or appreciation
for Unix's (and Linux's) history or philosophy.  You have no taste or
ability in design.

Go away.  And don't come back until you've read a book.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain

RE: RFC: Illegal Characters in File Names

[Guy]

Ouch!

He was the one going off topic!  No one ever said anything about function
keys!  And I think my example was clearly an example of what could be done,
not an example of how to do it.

I gave up on this issue.  Too many people on both sides.  Both sides have
very good reasons, and bad reasons!

I still vote for an option on the mount command.  But, default to disallow
all non-printable characters, and maybe a few printable that cause problems.

Oh, and some :) of your comments about me are obviously wrong, so you have
lost credibility.  "Read a book"?  I will wait for the movie! :)

Guy

-----Original Message-----
From: linux-fsdevel-owner@vger.kernel.org
[mailto:linux-fsdevel-owner@vger.kernel.org] On Behalf Of Matthew Wilcox
Sent: Thursday, July 22, 2004 2:04 PM
To: Guy
Cc: 'Jan-Benedict Glaw'; viro@parcelfarce.linux.theplanet.co.uk; 'Bryan
Henderson'; 'Jan Hudec'; linux-fsdevel@vger.kernel.org; 'Joseph D. Wagner'
Subject: Re: RFC: Illegal Characters in File Names

On Wed, Jul 21, 2004 at 11:25:03AM -0400, Guy wrote:
> Goober!  I was giving an example!  If cat <binary> file can cause a script

Up to this point, I've been polite.  But now you've resorted to personal
insults, allow me to continue in this vein.

You are a moron.  You have no idea what you're talking about.  Your
incessant trolling in this thread causes me to believe you are the lowest
form of pond scum.  You think it's enough to have opinions and other people
should implement code for you.  You have no understanding or appreciation
for Unix's (and Linux's) history or philosophy.  You have no taste or
ability in design.

Go away.  And don't come back until you've read a book.

-- 
"Next the statesmen will invent cheap lies, putting the blame upon 
the nation that is attacked, and every man will be glad of those
conscience-soothing falsities, and will diligently study them, and refuse
to examine any refutations of them; and thus he will by and by convince 
himself that the war is just, and will thank God for the better sleep 
he enjoys after this process of grotesque self-deception." -- Mark Twain
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: RFC: Illegal Characters in File Names

[Bryan Henderson]

>> I have personally lost a lot of time in cases where 'ls' misled me as 
to 
>> the contents of my directory.  (I eventually found the --almost-all and 

>> --human-readable options of GNU Ls and have been much happier since).
>So your problem is that ls does not do what you want by default.

Roughly, yes.  Only to be more precise, I would change "you" to "virtually 
everyone" and "want" to "expect."  And note that 'ls' is merely an example 
and my problem applies to just about every way people use filenames.

>We know that all valid filename characters can be entered into a bash 
>shell if correctly formatted & escaped.  There are no impossible to 
>delete file names (except for perhaps, impossible to create file names, 
>which fsck should fix).

In case it wasn't clear, my example (wherein I once deleted some valuable 
files due to a weird filename) was not one of a problem in Linux, but of 
how restricting filenames can in general be a benefit to users.  The 
system in question was not Unix.  What was weird about this particular 
filename, FWIW, is that it started with an underscore.

>If people write software that does silly things then that software needs 
>to be fixed.  Your own admission is that the author did not think about 
>what they were doing.

I think we've trod this ground (the blame game) before in this thread. 
There's plenty of blame to go around for my files getting improperly 
deleted.  The creator of the weird-named file, me, our teachers, and 
definitely the system designers.  Any one of these parties could have 
prevented the loss.

>The unix fs is a bit like C++.  You can do heaps of stupid things (c++ 
>multiple inheritance, operator overloading) but you probably shouldn't. 
>Just because you don't however, does not mean that you should remove 
>them from the language.

This is a good analogy, and note that I don't support removing from Linux 
the ability to have weird filenames.  The compiler has a bunch of options 
that say, "don't let me exploit the following features of the language, 
because it would not be intentional..."  I can make these a personal 
default, and if I own the system, even a system default.

Re: RFC: Illegal Characters in File Names

[Bryan Henderson]

>Where's the problem? That can (at least for dynamically linked
>applications -- most of what you use is that:) be done with a little
>LD_PRELOAD library.

Well, it's considerably harder and less robust than something implemented 
in the filesystem and its drivers; the people who need to do it don't know 
they need to; and it's not something the owner of a filesystem can force 
on his users (which he might want to do for the benefit of his other users 
-- naive, lazy, and stupid ones).

Re: RFC: Illegal Characters in File Names

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 753 bytes --]

On Thu, Jul 22, 2004 at 15:44:29 -0700, Bryan Henderson wrote:
> >Where's the problem? That can (at least for dynamically linked
> >applications -- most of what you use is that:) be done with a little
> >LD_PRELOAD library.
> 
> Well, it's considerably harder and less robust than something implemented 
> in the filesystem and its drivers; the people who need to do it don't know 
> they need to; and it's not something the owner of a filesystem can force 
> on his users (which he might want to do for the benefit of his other users 
> -- naive, lazy, and stupid ones).

He can. He can just link it into libc itself.

-------------------------------------------------------------------------------
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: RFC: Illegal Characters in File Names

[Bryan Henderson]

>it's not something the owner of a filesystem can force 
>> on his users (which he might want to do for the benefit of his other 
users 
>> -- naive, lazy, and stupid ones).
>
>He can. He can just link it into libc itself.

Users don't have to use his libc.  So that amounts to strongly encouraging 
users to keep the community filesystem clean of weird filenames, not 
forcing them to do so, as something built into the filesystem would do.