Re: [PATCH RFCv2 2/3] lib/vsprintf.c: make %pD print full path for file

Re: [PATCH RFCv2 2/3] lib/vsprintf.c: make %pD print full path for file

[Matthew Wilcox]

somehow the linux-fsdevel mailing list got dropped from this revision
of the patch set.  anyone who's following along may wish to refer to
the archives:
https://lore.kernel.org/linux-doc/20210528113951.6225-1-justin.he@arm.com/

On Tue, Jun 01, 2021 at 02:42:15PM +0000, Justin He wrote:
> > On Fri, May 28, 2021 at 03:09:28PM +0000, Justin He wrote:
> > > > I'm not sure why it's so complicated.  p->len records how many bytes
> > > > are needed for the entire path; can't you just return -p->len ?
> > >
> > > prepend_name() will return at the beginning if p->len is <0 in this case,
> > > we can't even get the correct full path size if keep __prepend_path
> > unchanged.
> > > We need another new helper __prepend_path_size() to get the full path
> > size
> > > regardless of the negative value p->len.
> >
> > It's a little hard to follow, based on just the patches.  Is there a
> > git tree somewhere of Al's patches that you're based on?
> >
> > Seems to me that prepend_name() is just fine because it updates p->len
> > before returning false:
> >
> >  static bool prepend_name(struct prepend_buffer *p, const struct qstr
> > *name)
> >  {
> >       const char *dname = smp_load_acquire(&name->name); /* ^^^ */
> >       u32 dlen = READ_ONCE(name->len);
> >       char *s;
> >
> >       p->len -= dlen + 1;
> >       if (unlikely(p->len < 0))
> >               return false;
> >
> > I think the only change you'd need to make for vsnprintf() is in
> > prepend_path():
> >
> > -             if (!prepend_name(&b, &dentry->d_name))
> > -                     break;
> > +             prepend_name(&b, &dentry->d_name);
> >
> > Would that hurt anything else?
> >
> 
> It almost works except the snprintf case,
> Consider,assuming filp path is 256 bytes, 2 dentries "/root/$long_string":
> snprintf(buffer, 128, "%pD", filp);
> p->len is positive at first, but negative after prepend_name loop.
> So, it will not fill any bytes in _buffer_.
> But in theory, it should fill the beginning 127 bytes and '\0'.

I have a few thoughts ...

1. Do we actually depend on that anywhere?
2. Is that something we should support?
3. We could print the start of the filename, if we do.  So something like
this ...

static void prepend(struct prepend_buffer *p, const char *str, int namelen)
{
	p->len -= namelen;
	if (likely(p->len >= 0)) {
		p->buf -= namelen;
		memcpy(p->buf, str, namelen);
	} else {
		char *s = p->buf;
		int buflen = strlen(p->buf);

		/* The first time we overflow the buffer */
		if (p->len + namelen > 0) {
			p->buf -= p->len + namelen;
			buflen += p->len + namelen;
		}

		if (buflen > namelen) {
			memmove(p->buf + namelen, s, buflen - namelen);
			memcpy(p->buf, str, namelen);
		} else {
			memcpy(p->buf, str, buflen);
		}
	}
}

I haven't tested this; it's probably full of confusion and off-by-one
errors.  But I hope you get the point -- we continue to accumulate
p->len to indicate how many characters we shifted off the right of the
buffer while adding the (start of) the filename on the left.

4. If we want the end of the filename instead, that looks easier:

static void prepend(struct prepend_buffer *p, const char *str, int namelen)
{
	p->len -= namelen;
	if (likely(p->len >= 0)) {
		p->buf -= namelen;
		memcpy(p->buf, str, namelen);
	} else if (p->len + namelen > 0) {
		p->buf -= p->len + namelen;
		memcpy(p->buf, str - p->len, p->len + namelen)
	}
}

But I don't think we want any of this at all.  Just don't put anything
in the buffer if the user didn't supply enough space.  As long as you
get the return value right, they know the string is bad (or they don't
care if the string is bad)

Re: [PATCH RFCv2 2/3] lib/vsprintf.c: make %pD print full path for file

[Andy Shevchenko]

On Tue, Jun 1, 2021 at 6:32 PM Matthew Wilcox <willy@infradead.org> wrote:
> On Tue, Jun 01, 2021 at 02:42:15PM +0000, Justin He wrote:

...

> Just don't put anything
> in the buffer if the user didn't supply enough space.  As long as you
> get the return value right, they know the string is bad (or they don't
> care if the string is bad)

It might be that I'm out of context here, but printf() functionality
in the kernel (vsprintf() if being precise)  and its users consider
that it should fill buffer up to the end of whatever space is
available.

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH RFCv2 2/3] lib/vsprintf.c: make %pD print full path for file

[Matthew Wilcox]

On Tue, Jun 01, 2021 at 06:36:41PM +0300, Andy Shevchenko wrote:
> On Tue, Jun 1, 2021 at 6:32 PM Matthew Wilcox <willy@infradead.org> wrote:
> > On Tue, Jun 01, 2021 at 02:42:15PM +0000, Justin He wrote:
> 
> ...
> 
> > Just don't put anything
> > in the buffer if the user didn't supply enough space.  As long as you
> > get the return value right, they know the string is bad (or they don't
> > care if the string is bad)
> 
> It might be that I'm out of context here, but printf() functionality
> in the kernel (vsprintf() if being precise)  and its users consider
> that it should fill buffer up to the end of whatever space is
> available.

Do they though?  What use is it to specify a small buffer, print a
large filename into it and then use that buffer, knowing that it wasn't
big enough?  That would help decide whether we should print the
start or the end of the filename.

Remember, we're going for usefulness here, not abiding by the letter of
the standard under all circumstances, no matter the cost.  At least
partially because we're far outside the standard here; POSIX does
not specify what %pD does.

"The argument shall be a pointer to void. The value of the
pointer is converted to a sequence of printable characters, in an
implementation-defined manner."

Re: [PATCH RFCv2 2/3] lib/vsprintf.c: make %pD print full path for file

[Andy Shevchenko]

On Tue, Jun 01, 2021 at 04:44:00PM +0100, Matthew Wilcox wrote:
> On Tue, Jun 01, 2021 at 06:36:41PM +0300, Andy Shevchenko wrote:
> > On Tue, Jun 1, 2021 at 6:32 PM Matthew Wilcox <willy@infradead.org> wrote:
> > > On Tue, Jun 01, 2021 at 02:42:15PM +0000, Justin He wrote:
> > 
> > ...
> > 
> > > Just don't put anything
> > > in the buffer if the user didn't supply enough space.  As long as you
> > > get the return value right, they know the string is bad (or they don't
> > > care if the string is bad)
> > 
> > It might be that I'm out of context here, but printf() functionality
> > in the kernel (vsprintf() if being precise)  and its users consider
> > that it should fill buffer up to the end of whatever space is
> > available.
> 
> Do they though?  What use is it to specify a small buffer, print a
> large filename into it and then use that buffer, knowing that it wasn't
> big enough?  That would help decide whether we should print the
> start or the end of the filename.
> 
> Remember, we're going for usefulness here, not abiding by the letter of
> the standard under all circumstances, no matter the cost.  At least
> partially because we're far outside the standard here; POSIX does
> not specify what %pD does.
> 
> "The argument shall be a pointer to void. The value of the
> pointer is converted to a sequence of printable characters, in an
> implementation-defined manner."

All nice words, but don't forget kasprintf() or other usages like this.
For the same input we have to have the same result independently on the room in
the buffer.

So, if I print "Hello, World" I should always get it, not "Monkey's Paw".
I.o.w.

 snprintf(10) ==> "Hello, Wor"
 snprintf(5)  ==> "Hello"
 snprintf(2)  !=> "Mo"
 snprintf(1)  !=> "M"
 snprintf(1)  ==> "H"

Inconsistency here is really not what we want.


-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH RFCv2 2/3] lib/vsprintf.c: make %pD print full path for file

[Andy Shevchenko]

On Tue, Jun 01, 2021 at 06:53:26PM +0300, Andy Shevchenko wrote:
> On Tue, Jun 01, 2021 at 04:44:00PM +0100, Matthew Wilcox wrote:
> > On Tue, Jun 01, 2021 at 06:36:41PM +0300, Andy Shevchenko wrote:
> > > On Tue, Jun 1, 2021 at 6:32 PM Matthew Wilcox <willy@infradead.org> wrote:
> > > > On Tue, Jun 01, 2021 at 02:42:15PM +0000, Justin He wrote:
> > > 
> > > ...
> > > 
> > > > Just don't put anything
> > > > in the buffer if the user didn't supply enough space.  As long as you
> > > > get the return value right, they know the string is bad (or they don't
> > > > care if the string is bad)
> > > 
> > > It might be that I'm out of context here, but printf() functionality
> > > in the kernel (vsprintf() if being precise)  and its users consider
> > > that it should fill buffer up to the end of whatever space is
> > > available.
> > 
> > Do they though?  What use is it to specify a small buffer, print a
> > large filename into it and then use that buffer, knowing that it wasn't
> > big enough?  That would help decide whether we should print the
> > start or the end of the filename.
> > 
> > Remember, we're going for usefulness here, not abiding by the letter of
> > the standard under all circumstances, no matter the cost.  At least
> > partially because we're far outside the standard here; POSIX does
> > not specify what %pD does.
> > 
> > "The argument shall be a pointer to void. The value of the
> > pointer is converted to a sequence of printable characters, in an
> > implementation-defined manner."
> 
> All nice words, but don't forget kasprintf() or other usages like this.
> For the same input we have to have the same result independently on the room in
> the buffer.
> 
> So, if I print "Hello, World" I should always get it, not "Monkey's Paw".
> I.o.w.
> 
>  snprintf(10) ==> "Hello, Wor"
>  snprintf(5)  ==> "Hello"
>  snprintf(2)  !=> "Mo"
>  snprintf(1)  !=> "M"
>  snprintf(1)  ==> "H"
> 
> Inconsistency here is really not what we want.

I have to add that in light of the topic those characters should be counted
from the end of the filename. So, we will give user as much as possible of useful
information. I.o.w. always print the last part of filename up to the buffer
size or if the filename is shorter than buffer we will have it in full.

-- 
With Best Regards,
Andy Shevchenko

Re: [PATCH RFCv2 2/3] lib/vsprintf.c: make %pD print full path for file

[Matthew Wilcox]

On Tue, Jun 01, 2021 at 07:10:41PM +0300, Andy Shevchenko wrote:
> On Tue, Jun 01, 2021 at 06:53:26PM +0300, Andy Shevchenko wrote:
> > On Tue, Jun 01, 2021 at 04:44:00PM +0100, Matthew Wilcox wrote:
> > > On Tue, Jun 01, 2021 at 06:36:41PM +0300, Andy Shevchenko wrote:
> > > > On Tue, Jun 1, 2021 at 6:32 PM Matthew Wilcox <willy@infradead.org> wrote:
> > > > > On Tue, Jun 01, 2021 at 02:42:15PM +0000, Justin He wrote:
> > > > 
> > > > ...
> > > > 
> > > > > Just don't put anything
> > > > > in the buffer if the user didn't supply enough space.  As long as you
> > > > > get the return value right, they know the string is bad (or they don't
> > > > > care if the string is bad)
> > > > 
> > > > It might be that I'm out of context here, but printf() functionality
> > > > in the kernel (vsprintf() if being precise)  and its users consider
> > > > that it should fill buffer up to the end of whatever space is
> > > > available.
> > > 
> > > Do they though?  What use is it to specify a small buffer, print a
> > > large filename into it and then use that buffer, knowing that it wasn't
> > > big enough?  That would help decide whether we should print the
> > > start or the end of the filename.
> > > 
> > > Remember, we're going for usefulness here, not abiding by the letter of
> > > the standard under all circumstances, no matter the cost.  At least
> > > partially because we're far outside the standard here; POSIX does
> > > not specify what %pD does.
> > > 
> > > "The argument shall be a pointer to void. The value of the
> > > pointer is converted to a sequence of printable characters, in an
> > > implementation-defined manner."
> > 
> > All nice words, but don't forget kasprintf() or other usages like this.
> > For the same input we have to have the same result independently on the room in
> > the buffer.
> > 
> > So, if I print "Hello, World" I should always get it, not "Monkey's Paw".
> > I.o.w.
> > 
> >  snprintf(10) ==> "Hello, Wor"
> >  snprintf(5)  ==> "Hello"
> >  snprintf(2)  !=> "Mo"
> >  snprintf(1)  !=> "M"
> >  snprintf(1)  ==> "H"
> > 
> > Inconsistency here is really not what we want.
> 
> I have to add that in light of the topic those characters should be counted
> from the end of the filename. So, we will give user as much as possible of useful
> information. I.o.w. always print the last part of filename up to the buffer
> size or if the filename is shorter than buffer we will have it in full.

Ah, not monkey's paw, but donkey hoof then ...

Here's some examples, what do you think makes sense?

snprintf(buf, 16, "bad file '%pD'\n", q);

what content do you want buf to have when q is variously:

1. /abcd/efgh
2. /a/bcdefgh.iso
3. /abcdef/gh

I would argue that
"bad file ''\n"
is actually a better string to have than any of (case 2)
"bad file '/a/bc"
"bad file 'bcdef"
"bad file 'h.iso"

Re: [PATCH RFCv2 2/3] lib/vsprintf.c: make %pD print full path for file

[Rasmus Villemoes]

On 01/06/2021 19.05, Matthew Wilcox wrote:

> Here's some examples, what do you think makes sense?
> 
> snprintf(buf, 16, "bad file '%pD'\n", q);
> 
> what content do you want buf to have when q is variously:
> 
> 1. /abcd/efgh
> 2. /a/bcdefgh.iso
> 3. /abcdef/gh
> 
> I would argue that
> "bad file ''\n"
> is actually a better string to have than any of (case 2)
> "bad file '/a/bc"
> "bad file 'bcdef"
> "bad file 'h.iso"
> 

Whatever ends up being decided, _please_ document that in
machine-readable and -verifiable form. I.e., update lib/test_printf.c
accordingly.

Currently (and originally) it only tests %pd because %pD is/was
essentially just %pd with an indirection to get the struct dentry* from
a struct file*.

The existing framework is strongly centered around expecting '/a/bc (see
all the logic where we do multiple checks with size 0, size random, size
plenty, and for the random case check that the buffer contents match the
complete output up till the randomly chosen size), so adding tests for
some other semantics would require a bit more juggling.

Not that that should be an argument in favor of that behaviour. But FWIW
that would be my preference.

Rasmus

RE: [PATCH RFCv2 2/3] lib/vsprintf.c: make %pD print full path for file

[Justin He]

Hi Rasmus

> -----Original Message-----
> From: Rasmus Villemoes <linux@rasmusvillemoes.dk>
> Sent: Wednesday, June 2, 2021 3:02 AM
> To: Matthew Wilcox <willy@infradead.org>; Andy Shevchenko
> <andy.shevchenko@gmail.com>
> Cc: Justin He <Justin.He@arm.com>; Linus Torvalds <torvalds@linux-
> foundation.org>; Petr Mladek <pmladek@suse.com>; Steven Rostedt
> <rostedt@goodmis.org>; Sergey Senozhatsky <senozhatsky@chromium.org>;
> Jonathan Corbet <corbet@lwn.net>; Alexander Viro <viro@zeniv.linux.org.uk>;
> Luca Coelho <luciano.coelho@intel.com>; Kalle Valo <kvalo@codeaurora.org>;
> David S. Miller <davem@davemloft.net>; Jakub Kicinski <kuba@kernel.org>;
> Heiko Carstens <hca@linux.ibm.com>; Vasily Gorbik <gor@linux.ibm.com>;
> Christian Borntraeger <borntraeger@de.ibm.com>; Johannes Berg
> <johannes.berg@intel.com>; linux-doc@vger.kernel.org; linux-
> kernel@vger.kernel.org; linux-wireless@vger.kernel.org;
> netdev@vger.kernel.org; linux-s390@vger.kernel.org; Linux FS Devel <linux-
> fsdevel@vger.kernel.org>
> Subject: Re: [PATCH RFCv2 2/3] lib/vsprintf.c: make %pD print full path for
> file
>
> On 01/06/2021 19.05, Matthew Wilcox wrote:
>
> > Here's some examples, what do you think makes sense?
> >
> > snprintf(buf, 16, "bad file '%pD'\n", q);
> >
> > what content do you want buf to have when q is variously:
> >
> > 1. /abcd/efgh
> > 2. /a/bcdefgh.iso
> > 3. /abcdef/gh
> >
> > I would argue that
> > "bad file ''\n"
> > is actually a better string to have than any of (case 2)
> > "bad file '/a/bc"
> > "bad file 'bcdef"
> > "bad file 'h.iso"
> >
>
> Whatever ends up being decided, _please_ document that in
> machine-readable and -verifiable form. I.e., update lib/test_printf.c
> accordingly.
>
> Currently (and originally) it only tests %pd because %pD is/was
> essentially just %pd with an indirection to get the struct dentry* from
> a struct file*.

Okay, I can add more test_printf cases for '%pD'

>
> The existing framework is strongly centered around expecting '/a/bc (see
> all the logic where we do multiple checks with size 0, size random, size
> plenty, and for the random case check that the buffer contents match the
> complete output up till the randomly chosen size), so adding tests for
> some other semantics would require a bit more juggling.
>

Yes, agree.
In other way, if the user:
char* full_path = d_path(...);
snprintf("%s", limited_size, full_path);

He/she will get the inconsistent result if we return "" for '%pD'.

--
Cheers,
Justin (Jia He)

> Not that that should be an argument in favor of that behaviour. But FWIW
> that would be my preference.
>
> Rasmus
>

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.