Orangefs, v4.5 and the merge window...

Orangefs, v4.5 and the merge window...

[Mike Marshall]

Greetings...

The Orangefs for-next tree is:

git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
for-next

I did a test merge (just locally, not pushed out) of Orangefs:for-next
and v4.5-rc7 so I could test out how I think I need to patch for
the follow_link -> get_link change, the diff is below.

On Monday next, assuming that v4.5 is finalized this weekend,
I plan to do a actual merge with v4.5, apply the get_link patch
and push that to Orangefs:for-next.

Hi Al <g>... might we get an ACK this time around?

-Mike

diff --git a/fs/orangefs/inode.c b/fs/orangefs/inode.c
index 4e923ec..50a2172 100644
--- a/fs/orangefs/inode.c
+++ b/fs/orangefs/inode.c
@@ -276,9 +276,12 @@ int orangefs_getattr(struct vfsmount *mnt,
  ret = orangefs_inode_getattr(inode, ORANGEFS_ATTR_SYS_ALL_NOHINT, 0);
  if (ret == 0) {
  generic_fillattr(inode, kstat);
+
  /* override block size reported to stat */
  orangefs_inode = ORANGEFS_I(inode);
  kstat->blksize = orangefs_inode->blksize;
+
+ inode->i_link = ORANGEFS_I(dentry->d_inode)->link_target;
  } else {
  /* assume an I/O error and flag inode as bad */
  gossip_debug(GOSSIP_INODE_DEBUG,
diff --git a/fs/orangefs/symlink.c b/fs/orangefs/symlink.c
index 2b8541a..6418dd6 100644
--- a/fs/orangefs/symlink.c
+++ b/fs/orangefs/symlink.c
@@ -8,22 +8,9 @@
 #include "orangefs-kernel.h"
 #include "orangefs-bufmap.h"

-static const char *orangefs_follow_link(struct dentry *dentry, void **cookie)
-{
- char *target =  ORANGEFS_I(dentry->d_inode)->link_target;
-
- gossip_debug(GOSSIP_INODE_DEBUG,
-     "%s: called on %s (target is %p)\n",
-     __func__, (char *)dentry->d_name.name, target);
-
- *cookie = target;
-
- return target;
-}
-
 struct inode_operations orangefs_symlink_inode_operations = {
  .readlink = generic_readlink,
- .follow_link = orangefs_follow_link,
+ .get_link = simple_get_link,
  .setattr = orangefs_setattr,
  .getattr = orangefs_getattr,
  .listxattr = orangefs_listxattr,

Re: Orangefs, v4.5 and the merge window...

[Al Viro]

On Fri, Mar 11, 2016 at 03:18:57PM -0500, Mike Marshall wrote:
> Greetings...
> 
> The Orangefs for-next tree is:
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
> for-next
> 
> I did a test merge (just locally, not pushed out) of Orangefs:for-next
> and v4.5-rc7 so I could test out how I think I need to patch for
> the follow_link -> get_link change, the diff is below.
> 
> On Monday next, assuming that v4.5 is finalized this weekend,
> I plan to do a actual merge with v4.5, apply the get_link patch
> and push that to Orangefs:for-next.
> 
> Hi Al <g>... might we get an ACK this time around?

You do realize that it will mean fun few weeks post-merge fixing the rest of
problems, right?  FWIW, I think that right now it *is* at the state where it
such fixing is feasible, so modulo that...

As far as I can see, waiting-related logics should be solid by now, ditto
for lifetime rules; sanitizing the input...  listxattr still does need fixing
(feed it a negative in ->downcall.resp.listxattr.lengths[0] and watch Bad
Things(tm) happen; no idea why would anyone go for
fs/orangefs/downcall.h:82:      __s32 lengths[ORANGEFS_MAX_XATTR_LISTLEN];
for representing string lengths in the first place, but that's what you've
got there and no sanity checks are done on it beyond
                if (total + new_op->downcall.resp.listxattr.lengths[i] > size)
                        goto done;
which is not enough - not with total and size being ssize_t and ...lengths[] -
signed 32bit).

The logics around maintaining the list of orangefs superblocks (add/remove/
traverse) needs fixing; right now ioctl(..., ORANGEFS_DEV_REMOUNT_ALL) will
walk through it with only request_mutex held.  Both insertion and removal
are protected only by orangefs_superblocks_lock, and removal is insane -
        struct list_head *tmp_safe = NULL;                              \
        struct orangefs_sb_info_s *orangefs_sb = NULL;                  \
                                                                        \
        spin_lock(&orangefs_superblocks_lock);                          \
        list_for_each_safe(tmp, tmp_safe, &orangefs_superblocks) {      \
                orangefs_sb = list_entry(tmp,                           \
                                      struct orangefs_sb_info_s,        \
                                      list);                            \
                if (orangefs_sb && (orangefs_sb->sb == sb)) {           \
                        gossip_debug(GOSSIP_SUPER_DEBUG,                \
                            "Removing SB %p from orangefs superblocks\n",      \
                        orangefs_sb);                                   \
                        list_del(&orangefs_sb->list);                   \
                        break;                                          \
                }                                                       \
        }                                                               \
        spin_unlock(&orangefs_superblocks_lock);                        \
list_entry is never NULL, for starters, and since there is a pointer back
from superblock to that orangefs_sb_info, there's no reason to walk the entire
list to find one.  BTW, both add_orangefs_sb() and remove_orangefs_sb() should
be taken to their sole users.

Sanity aside, there's really no lock in common for list modifiers and list
walker I'd mentioned above.  FWIW, I would make orangefs_remount()
take struct orangefs_sb_info instead of struct super_block and flipped the
order of operations in orangefs_kill_sb() - kill_anon_super() *first*, then
remove from the list, then tell the userland that it's going away (i.e.
call orangefs_unmount_sb()).  request_mutex in the last one would, at least,
prevent freeing the sucker before orangefs_remount() is done with it.

Walking the list and calling orangefs_remount() on everything would still need
care - you'd need to hold orangefs_superblocks_lock, drop it for actual calls
of orangefs_remount() and have list removal preserve the forward pointer.

That's probably the worst remaining locking issue I see in there.  Doable,
if not pleasant...

IIRC, there also had been some unpleasantness with getattr messing ->i_mode
halfway through... <checks>  Yes - copy_attributes_to_inode() will be called,
and do
        inode->i_mode = orangefs_inode_perms(attrs);
...
                inode->i_mode |= S_IFLNK;
...
                        strncpy(orangefs_inode->link_target,
                                symname,
                                ORANGEFS_NAME_MAX);
If nothing else, *another* stat(2) racing with this one could pick the
intermediate value of ->i_mode and proceed to report it to userland.
Another problem is overwriting the symlink body; that can get very
unpleasant, since it might be traversed by another syscall right at that
moment.  Any change of a symlink body means "we'd missed it going stale"; 
there is no way to change a symlink contents without removing it and
creating a new one.  Should anything other than orangefs_iget() even bother
copying it?  The same goes for inode type changes, of course (regular
vs. directory vs. symlink, etc.).

Speaking of orangefs_iget(), orangefs_set_inode() is pointlessly paranoid.
Not a bug per se, but
        struct orangefs_inode_s *orangefs_inode = NULL;

        /* Make sure that we have sane parameters */
        if (!data || !inode)  
                return 0;
        orangefs_inode = ORANGEFS_I(inode);
        if (!orangefs_inode)
                return 0;
is all wrong - 'data' is the last argument passed to iget5_locked (i.e. 'ref'
of orangefs_iget()) and that's always an address of either a local variable
or of a field in a large structure, and not even the first one; 'inode'
is never NULL - it's the address of struct inode the caller is about to
insert into the hash chain; ORANGEFS_I() is container_of(), so it's not
going to be NULL either.

I'll need to look through the archived threads to see if there's anything
else left; IIRC, debugfs-related code had seriously nasty issues in case of
allocation failures, but those were fairly isolated.  I'll read through the
archive tomorrow and see if there's anything else mentioned and not dealt
with; I don't remember anything really bad, but it had been well over
a hundred mails starting about half a year ago; I sure as hell do not
remember every tangential subthread in all of that, so I'll need to recheck.

I _think_ that all remaining issues can be quickly dealt with, and the code
has zero impact on the rest of the kernel.  I wouldn't risk putting it into
-final without fixups, but as for the merge schedule... either merge it
before -rc1 and fix it up by -rc3 or so, or fix it during the window and
merge at around -rc2 - I'm fine with either variant.

Re: Orangefs, v4.5 and the merge window...

[Mike Marshall]

> either merge it
> before -rc1 and fix it up by -rc3 or so, or fix it during the
> window and merge at around -rc2 - I'm fine with either
> variant.

We've kept a list we made from all those mail messages
so we could check off things we've tried to address, I
was looking at it yesterday and I know it is not up-to-date,
but we'll work to get it that way. The second option
might be safer unless you help us again, I don't want
to sign a rubber check to Linus.

-Mike

On Fri, Mar 11, 2016 at 4:47 PM, Al Viro <viro@zeniv.linux.org.uk> wrote:
> On Fri, Mar 11, 2016 at 03:18:57PM -0500, Mike Marshall wrote:
>> Greetings...
>>
>> The Orangefs for-next tree is:
>>
>> git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
>> for-next
>>
>> I did a test merge (just locally, not pushed out) of Orangefs:for-next
>> and v4.5-rc7 so I could test out how I think I need to patch for
>> the follow_link -> get_link change, the diff is below.
>>
>> On Monday next, assuming that v4.5 is finalized this weekend,
>> I plan to do a actual merge with v4.5, apply the get_link patch
>> and push that to Orangefs:for-next.
>>
>> Hi Al <g>... might we get an ACK this time around?
>
> You do realize that it will mean fun few weeks post-merge fixing the rest of
> problems, right?  FWIW, I think that right now it *is* at the state where it
> such fixing is feasible, so modulo that...
>
> As far as I can see, waiting-related logics should be solid by now, ditto
> for lifetime rules; sanitizing the input...  listxattr still does need fixing
> (feed it a negative in ->downcall.resp.listxattr.lengths[0] and watch Bad
> Things(tm) happen; no idea why would anyone go for
> fs/orangefs/downcall.h:82:      __s32 lengths[ORANGEFS_MAX_XATTR_LISTLEN];
> for representing string lengths in the first place, but that's what you've
> got there and no sanity checks are done on it beyond
>                 if (total + new_op->downcall.resp.listxattr.lengths[i] > size)
>                         goto done;
> which is not enough - not with total and size being ssize_t and ...lengths[] -
> signed 32bit).
>
> The logics around maintaining the list of orangefs superblocks (add/remove/
> traverse) needs fixing; right now ioctl(..., ORANGEFS_DEV_REMOUNT_ALL) will
> walk through it with only request_mutex held.  Both insertion and removal
> are protected only by orangefs_superblocks_lock, and removal is insane -
>         struct list_head *tmp_safe = NULL;                              \
>         struct orangefs_sb_info_s *orangefs_sb = NULL;                  \
>                                                                         \
>         spin_lock(&orangefs_superblocks_lock);                          \
>         list_for_each_safe(tmp, tmp_safe, &orangefs_superblocks) {      \
>                 orangefs_sb = list_entry(tmp,                           \
>                                       struct orangefs_sb_info_s,        \
>                                       list);                            \
>                 if (orangefs_sb && (orangefs_sb->sb == sb)) {           \
>                         gossip_debug(GOSSIP_SUPER_DEBUG,                \
>                             "Removing SB %p from orangefs superblocks\n",      \
>                         orangefs_sb);                                   \
>                         list_del(&orangefs_sb->list);                   \
>                         break;                                          \
>                 }                                                       \
>         }                                                               \
>         spin_unlock(&orangefs_superblocks_lock);                        \
> list_entry is never NULL, for starters, and since there is a pointer back
> from superblock to that orangefs_sb_info, there's no reason to walk the entire
> list to find one.  BTW, both add_orangefs_sb() and remove_orangefs_sb() should
> be taken to their sole users.
>
> Sanity aside, there's really no lock in common for list modifiers and list
> walker I'd mentioned above.  FWIW, I would make orangefs_remount()
> take struct orangefs_sb_info instead of struct super_block and flipped the
> order of operations in orangefs_kill_sb() - kill_anon_super() *first*, then
> remove from the list, then tell the userland that it's going away (i.e.
> call orangefs_unmount_sb()).  request_mutex in the last one would, at least,
> prevent freeing the sucker before orangefs_remount() is done with it.
>
> Walking the list and calling orangefs_remount() on everything would still need
> care - you'd need to hold orangefs_superblocks_lock, drop it for actual calls
> of orangefs_remount() and have list removal preserve the forward pointer.
>
> That's probably the worst remaining locking issue I see in there.  Doable,
> if not pleasant...
>
> IIRC, there also had been some unpleasantness with getattr messing ->i_mode
> halfway through... <checks>  Yes - copy_attributes_to_inode() will be called,
> and do
>         inode->i_mode = orangefs_inode_perms(attrs);
> ...
>                 inode->i_mode |= S_IFLNK;
> ...
>                         strncpy(orangefs_inode->link_target,
>                                 symname,
>                                 ORANGEFS_NAME_MAX);
> If nothing else, *another* stat(2) racing with this one could pick the
> intermediate value of ->i_mode and proceed to report it to userland.
> Another problem is overwriting the symlink body; that can get very
> unpleasant, since it might be traversed by another syscall right at that
> moment.  Any change of a symlink body means "we'd missed it going stale";
> there is no way to change a symlink contents without removing it and
> creating a new one.  Should anything other than orangefs_iget() even bother
> copying it?  The same goes for inode type changes, of course (regular
> vs. directory vs. symlink, etc.).
>
> Speaking of orangefs_iget(), orangefs_set_inode() is pointlessly paranoid.
> Not a bug per se, but
>         struct orangefs_inode_s *orangefs_inode = NULL;
>
>         /* Make sure that we have sane parameters */
>         if (!data || !inode)
>                 return 0;
>         orangefs_inode = ORANGEFS_I(inode);
>         if (!orangefs_inode)
>                 return 0;
> is all wrong - 'data' is the last argument passed to iget5_locked (i.e. 'ref'
> of orangefs_iget()) and that's always an address of either a local variable
> or of a field in a large structure, and not even the first one; 'inode'
> is never NULL - it's the address of struct inode the caller is about to
> insert into the hash chain; ORANGEFS_I() is container_of(), so it's not
> going to be NULL either.
>
> I'll need to look through the archived threads to see if there's anything
> else left; IIRC, debugfs-related code had seriously nasty issues in case of
> allocation failures, but those were fairly isolated.  I'll read through the
> archive tomorrow and see if there's anything else mentioned and not dealt
> with; I don't remember anything really bad, but it had been well over
> a hundred mails starting about half a year ago; I sure as hell do not
> remember every tangential subthread in all of that, so I'll need to recheck.
>
> I _think_ that all remaining issues can be quickly dealt with, and the code
> has zero impact on the rest of the kernel.  I wouldn't risk putting it into
> -final without fixups, but as for the merge schedule... either merge it
> before -rc1 and fix it up by -rc3 or so, or fix it during the window and
> merge at around -rc2 - I'm fine with either variant.

Re: Orangefs, v4.5 and the merge window...

[Mike Marshall]

Hi Al (and everyone)...

git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
for-next

... is updated to v4.5, it has the follow_link -> get_link change.

I worked today to clean up the debugfs (and sysfs) problems,
and we'll keep ticking things off the list, perhaps I should be
posting patches here for review instead automatically updating
for-next when we work on an issue...

We're working on putting "the list" in a place that can be
viewed by everyone and edited by both Martin and myself...

-Mike

On Fri, Mar 11, 2016 at 5:35 PM, Mike Marshall <hubcap@omnibond.com> wrote:
>> either merge it
>> before -rc1 and fix it up by -rc3 or so, or fix it during the
>> window and merge at around -rc2 - I'm fine with either
>> variant.
>
> We've kept a list we made from all those mail messages
> so we could check off things we've tried to address, I
> was looking at it yesterday and I know it is not up-to-date,
> but we'll work to get it that way. The second option
> might be safer unless you help us again, I don't want
> to sign a rubber check to Linus.
>
> -Mike
>
> On Fri, Mar 11, 2016 at 4:47 PM, Al Viro <viro@zeniv.linux.org.uk> wrote:
>> On Fri, Mar 11, 2016 at 03:18:57PM -0500, Mike Marshall wrote:
>>> Greetings...
>>>
>>> The Orangefs for-next tree is:
>>>
>>> git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
>>> for-next
>>>
>>> I did a test merge (just locally, not pushed out) of Orangefs:for-next
>>> and v4.5-rc7 so I could test out how I think I need to patch for
>>> the follow_link -> get_link change, the diff is below.
>>>
>>> On Monday next, assuming that v4.5 is finalized this weekend,
>>> I plan to do a actual merge with v4.5, apply the get_link patch
>>> and push that to Orangefs:for-next.
>>>
>>> Hi Al <g>... might we get an ACK this time around?
>>
>> You do realize that it will mean fun few weeks post-merge fixing the rest of
>> problems, right?  FWIW, I think that right now it *is* at the state where it
>> such fixing is feasible, so modulo that...
>>
>> As far as I can see, waiting-related logics should be solid by now, ditto
>> for lifetime rules; sanitizing the input...  listxattr still does need fixing
>> (feed it a negative in ->downcall.resp.listxattr.lengths[0] and watch Bad
>> Things(tm) happen; no idea why would anyone go for
>> fs/orangefs/downcall.h:82:      __s32 lengths[ORANGEFS_MAX_XATTR_LISTLEN];
>> for representing string lengths in the first place, but that's what you've
>> got there and no sanity checks are done on it beyond
>>                 if (total + new_op->downcall.resp.listxattr.lengths[i] > size)
>>                         goto done;
>> which is not enough - not with total and size being ssize_t and ...lengths[] -
>> signed 32bit).
>>
>> The logics around maintaining the list of orangefs superblocks (add/remove/
>> traverse) needs fixing; right now ioctl(..., ORANGEFS_DEV_REMOUNT_ALL) will
>> walk through it with only request_mutex held.  Both insertion and removal
>> are protected only by orangefs_superblocks_lock, and removal is insane -
>>         struct list_head *tmp_safe = NULL;                              \
>>         struct orangefs_sb_info_s *orangefs_sb = NULL;                  \
>>                                                                         \
>>         spin_lock(&orangefs_superblocks_lock);                          \
>>         list_for_each_safe(tmp, tmp_safe, &orangefs_superblocks) {      \
>>                 orangefs_sb = list_entry(tmp,                           \
>>                                       struct orangefs_sb_info_s,        \
>>                                       list);                            \
>>                 if (orangefs_sb && (orangefs_sb->sb == sb)) {           \
>>                         gossip_debug(GOSSIP_SUPER_DEBUG,                \
>>                             "Removing SB %p from orangefs superblocks\n",      \
>>                         orangefs_sb);                                   \
>>                         list_del(&orangefs_sb->list);                   \
>>                         break;                                          \
>>                 }                                                       \
>>         }                                                               \
>>         spin_unlock(&orangefs_superblocks_lock);                        \
>> list_entry is never NULL, for starters, and since there is a pointer back
>> from superblock to that orangefs_sb_info, there's no reason to walk the entire
>> list to find one.  BTW, both add_orangefs_sb() and remove_orangefs_sb() should
>> be taken to their sole users.
>>
>> Sanity aside, there's really no lock in common for list modifiers and list
>> walker I'd mentioned above.  FWIW, I would make orangefs_remount()
>> take struct orangefs_sb_info instead of struct super_block and flipped the
>> order of operations in orangefs_kill_sb() - kill_anon_super() *first*, then
>> remove from the list, then tell the userland that it's going away (i.e.
>> call orangefs_unmount_sb()).  request_mutex in the last one would, at least,
>> prevent freeing the sucker before orangefs_remount() is done with it.
>>
>> Walking the list and calling orangefs_remount() on everything would still need
>> care - you'd need to hold orangefs_superblocks_lock, drop it for actual calls
>> of orangefs_remount() and have list removal preserve the forward pointer.
>>
>> That's probably the worst remaining locking issue I see in there.  Doable,
>> if not pleasant...
>>
>> IIRC, there also had been some unpleasantness with getattr messing ->i_mode
>> halfway through... <checks>  Yes - copy_attributes_to_inode() will be called,
>> and do
>>         inode->i_mode = orangefs_inode_perms(attrs);
>> ...
>>                 inode->i_mode |= S_IFLNK;
>> ...
>>                         strncpy(orangefs_inode->link_target,
>>                                 symname,
>>                                 ORANGEFS_NAME_MAX);
>> If nothing else, *another* stat(2) racing with this one could pick the
>> intermediate value of ->i_mode and proceed to report it to userland.
>> Another problem is overwriting the symlink body; that can get very
>> unpleasant, since it might be traversed by another syscall right at that
>> moment.  Any change of a symlink body means "we'd missed it going stale";
>> there is no way to change a symlink contents without removing it and
>> creating a new one.  Should anything other than orangefs_iget() even bother
>> copying it?  The same goes for inode type changes, of course (regular
>> vs. directory vs. symlink, etc.).
>>
>> Speaking of orangefs_iget(), orangefs_set_inode() is pointlessly paranoid.
>> Not a bug per se, but
>>         struct orangefs_inode_s *orangefs_inode = NULL;
>>
>>         /* Make sure that we have sane parameters */
>>         if (!data || !inode)
>>                 return 0;
>>         orangefs_inode = ORANGEFS_I(inode);
>>         if (!orangefs_inode)
>>                 return 0;
>> is all wrong - 'data' is the last argument passed to iget5_locked (i.e. 'ref'
>> of orangefs_iget()) and that's always an address of either a local variable
>> or of a field in a large structure, and not even the first one; 'inode'
>> is never NULL - it's the address of struct inode the caller is about to
>> insert into the hash chain; ORANGEFS_I() is container_of(), so it's not
>> going to be NULL either.
>>
>> I'll need to look through the archived threads to see if there's anything
>> else left; IIRC, debugfs-related code had seriously nasty issues in case of
>> allocation failures, but those were fairly isolated.  I'll read through the
>> archive tomorrow and see if there's anything else mentioned and not dealt
>> with; I don't remember anything really bad, but it had been well over
>> a hundred mails starting about half a year ago; I sure as hell do not
>> remember every tangential subthread in all of that, so I'll need to recheck.
>>
>> I _think_ that all remaining issues can be quickly dealt with, and the code
>> has zero impact on the rest of the kernel.  I wouldn't risk putting it into
>> -final without fixups, but as for the merge schedule... either merge it
>> before -rc1 and fix it up by -rc3 or so, or fix it during the window and
>> merge at around -rc2 - I'm fine with either variant.

Re: Orangefs, v4.5 and the merge window...

[Al Viro]

On Mon, Mar 14, 2016 at 05:03:55PM -0400, Mike Marshall wrote:
> Hi Al (and everyone)...
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
> for-next
> 
> ... is updated to v4.5, it has the follow_link -> get_link change.
> 
> I worked today to clean up the debugfs (and sysfs) problems,
> and we'll keep ticking things off the list, perhaps I should be
> posting patches here for review instead automatically updating
> for-next when we work on an issue...
> 
> We're working on putting "the list" in a place that can be
> viewed by everyone and edited by both Martin and myself...

FWIW, I've ported several cleanups + short read/write on error + lseek
fixes + orangefs_superblocks locking fixes on top of that branch and pushed
into vfs.git#orangefs-untested.

I think it can be merged this cycle; if the fixes above really work, this
+ Martin's latest should leave only the handling of allocation failures in
debugfs-related code.

Linus, would you be OK with that thing going into -rc2?  It's self-contained
and I think it's in reasonably sane shape.  If not for the last locking fixes,
I would even suggest -rc1 with fixups of debugfs issues during the next week,
but that last commit got no testing whatsoever... ;-/

I would prefer if pull request went from Mike, once he's OK with the whole
set, but as far as I'm concerned the material in his #for-next + my pile above
+ Martin's branch (getattr fixes) should be OK for mainline merge.

Re: Orangefs, v4.5 and the merge window...

[Mike Marshall]

Wow... the superblock problem is something we haven't worked on yet...
thanks!

Martin's new code caused some xfstests regressions, but we got past
all but the generic/028 regression... I don't know if Martin worked on
that today, I wasn't in the office.

I had fixed up what I saw wrong in debugfs, so I'll have to look
again for the allocation failures...

I haven't pushed Martin's re-worked code to kernel.org yet, since
we were working through the couple of regressions, have you looked
at it on his github repo, and it doesn't conflict with your patches?

I'll look and see if I can get your patches to load on top of
Martin's patches and get that pushed to kernel.org.

I leave before dawn on Sunday morning for that Linux
Foundation summit in Lake Tahoe, so if we get this
thing merged now (I hope <g>) we'll have to continue
to polish it through the rc's, but that's what they're there
for, right?

-Mike

On Fri, Mar 25, 2016 at 8:21 PM, Al Viro <viro@zeniv.linux.org.uk> wrote:
> On Mon, Mar 14, 2016 at 05:03:55PM -0400, Mike Marshall wrote:
>> Hi Al (and everyone)...
>>
>> git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
>> for-next
>>
>> ... is updated to v4.5, it has the follow_link -> get_link change.
>>
>> I worked today to clean up the debugfs (and sysfs) problems,
>> and we'll keep ticking things off the list, perhaps I should be
>> posting patches here for review instead automatically updating
>> for-next when we work on an issue...
>>
>> We're working on putting "the list" in a place that can be
>> viewed by everyone and edited by both Martin and myself...
>
> FWIW, I've ported several cleanups + short read/write on error + lseek
> fixes + orangefs_superblocks locking fixes on top of that branch and pushed
> into vfs.git#orangefs-untested.
>
> I think it can be merged this cycle; if the fixes above really work, this
> + Martin's latest should leave only the handling of allocation failures in
> debugfs-related code.
>
> Linus, would you be OK with that thing going into -rc2?  It's self-contained
> and I think it's in reasonably sane shape.  If not for the last locking fixes,
> I would even suggest -rc1 with fixups of debugfs issues during the next week,
> but that last commit got no testing whatsoever... ;-/
>
> I would prefer if pull request went from Mike, once he's OK with the whole
> set, but as far as I'm concerned the material in his #for-next + my pile above
> + Martin's branch (getattr fixes) should be OK for mainline merge.

Re: Orangefs, v4.5 and the merge window...

[Al Viro]

On Fri, Mar 25, 2016 at 05:28:33PM -0700, Linus Torvalds wrote:
> Gaah. I'm going to be traveling starting tomorrow night (red-eye) and I was
> going to cut rc1 tomorrow and then really hope for a calm rc2 week.
> 
> At the same time, is live to get this finally merged, about a year after I
> originally thought were would. So I guess I'll take it, even if I hate the
> timing.

Umm... IIRC, there had been precedents when self-contained driver got merged
later than -rc1.  Might make more sense than rushing the merge in the last
day of the window, especially since you are travelling...

> There are zero changes outside orangefs itself, and obviously the
> Makefile/KConfig file to get it all enabled?

Yes.

Re: Orangefs, v4.5 and the merge window...

[Mike Marshall]

If Al's patches load on top of Martin's patches without conflict, I can probably
get it all merged and minimally tested before noon my time (eastern
US) tomorrow... that's probably late for where y'all are though...

-Mike

On Fri, Mar 25, 2016 at 9:01 PM, Al Viro <viro@zeniv.linux.org.uk> wrote:
> On Fri, Mar 25, 2016 at 05:28:33PM -0700, Linus Torvalds wrote:
>> Gaah. I'm going to be traveling starting tomorrow night (red-eye) and I was
>> going to cut rc1 tomorrow and then really hope for a calm rc2 week.
>>
>> At the same time, is live to get this finally merged, about a year after I
>> originally thought were would. So I guess I'll take it, even if I hate the
>> timing.
>
> Umm... IIRC, there had been precedents when self-contained driver got merged
> later than -rc1.  Might make more sense than rushing the merge in the last
> day of the window, especially since you are travelling...
>
>> There are zero changes outside orangefs itself, and obviously the
>> Makefile/KConfig file to get it all enabled?
>
> Yes.

Re: Orangefs, v4.5 and the merge window...

[Mike Marshall]

I got Al's patches merged with Martin's patches and only
one pretty simple conflict to fix.

In testing, most things seem to work right, but at least one
important thing is busted: umount...

unmounting the filesystem just gets the kernel into a forever loop,
"ERROR: fs_mount_pending 147936488" just goes on
forever until I interrupt the umount command:

Mar 25 23:42:00 be1 kernel: [  156.990737] Releasing OP (ffff8800189a0000: 103)
Mar 25 23:42:00 be1 kernel: [  156.991366] orangefs_kill_sb: called
Mar 25 23:42:00 be1 kernel: [  156.991401] orangefs_destroy_inode:
deallocated ffff880018998000 destroying inode
00001000-0000-0000-0000-000000000000
Mar 25 23:42:00 be1 kernel: [  156.991414] orangefs_unmount_sb called
on sb ffff880018810000
Mar 25 23:42:00 be1 kernel: [  156.991416] Alloced OP
(ffff8800189a0000: 104 OP_FS_UMOUNT)
Mar 25 23:42:00 be1 kernel: [  156.991417] Attempting ORANGEFS Unmount
via host tcp://be1:3334/orangefs
Mar 25 23:42:00 be1 kernel: [  156.991418] service_operation:
orangefs_fs_umount op:ffff8800189a0000: process:umount: pid:1124:
Mar 25 23:42:00 be1 kernel: [  156.991419] service_operation:
op:OP_FS_UMOUNT: op_state:1: process:umount:
Mar 25 23:42:00 be1 kernel: [  156.991435] orangefs: skipping op tag
104 OP_FS_UMOUNT
Mar 25 23:42:00 be1 kernel: [  156.991435] orangefs: ERROR:
fs_mount_pending 147936488
Mar 25 23:42:00 be1 kernel: [  156.991440] orangefs: skipping op tag
104 OP_FS_UMOUNT

The github repo has all the patches, I didn't think I
should push it to the orangefs for-next branch like this.

https://github.com/hubcapsc/linux/tree/current

I'll look to see if I can see it, I guess it has something to
do with Al's superblock re-do...

-Mike

On Fri, Mar 25, 2016 at 9:11 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Mar 25, 2016 18:01, "Al Viro" <viro@zeniv.linux.org.uk> wrote:
>>
>> Umm... IIRC, there had been precedents when self-contained driver got
>> merged
>> later than -rc1.
>
> I do it if there is some reason for it, yes.
>
> In particular, for things like consumer hardware that has not had a driver
> for it, I'll merge new drivers later, because missing one release nasty be a
> big deal if some average-Joe machine just doesn't work. But even then it's
> not just "random driver", but something that keeps that machine from working
> entirely - disk or network or GPU or USB or something core like that.
>
> So it's not just "new driver that cannot cause a regression".  It's also
> about the driver being important to people who can't reasonably be expected
> to compile their own kernel..
>
> There is little reason to do an expedited out-of-the-merge-window merge for
> something like orangefs, though.
>
>    Linus

Re: Orangefs, v4.5 and the merge window...

[Al Viro]

On Fri, Mar 25, 2016 at 11:55:31PM -0400, Mike Marshall wrote:

> I'll look to see if I can see it, I guess it has something to
> do with Al's superblock re-do...

It does; with this approach you would need to add ORANGEFS_VFS_OP_FS_UNMOUNT
to the whitelist in orangefs_devreq_read() - the
                                   !(op->upcall.type ==
                                        ORANGEFS_VFS_OP_FS_MOUNT ||
                                     op->upcall.type ==
                                        ORANGEFS_VFS_OP_GETATTR)) {
thing.  Sorry, should've thought about that...

Alternatively, we could do orangefs_unmount_sb(sb) before removing from
the list, and add mutex_lock/mutex_unlock of request_mutex right before
that kfree() in the very end.  Same effect in terms of list protection and
closer to your current logics.

Try this incremental (to be folded into "fix orangefs_superblock locking"):

diff --git a/fs/orangefs/super.c b/fs/orangefs/super.c
index bf78870..bb3dc14 100644
--- a/fs/orangefs/super.c
+++ b/fs/orangefs/super.c
@@ -514,6 +514,12 @@ void orangefs_kill_sb(struct super_block *sb)
 	/* provided sb cleanup */
 	kill_anon_super(sb);
 
+	/*
+	 * issue the unmount to userspace to tell it to remove the
+	 * dynamic mount info it has for this superblock
+	 */
+	orangefs_unmount_sb(sb);
+
 	/* remove the sb from our list of orangefs specific sb's */
 
 	spin_lock(&orangefs_superblocks_lock);
@@ -522,10 +528,11 @@ void orangefs_kill_sb(struct super_block *sb)
 	spin_unlock(&orangefs_superblocks_lock);
 
 	/*
-	 * issue the unmount to userspace to tell it to remove the
-	 * dynamic mount info it has for this superblock
+	 * make sure that ORANGEFS_DEV_REMOUNT_ALL loop that might've seen us
+	 * gets completed before we free the damn thing.
 	 */
-	orangefs_unmount_sb(sb);
+	mutex_lock(&request_mutex);
+	mutex_unlock(&request_mutex);
 
 	/* free the orangefs superblock private data */
 	kfree(ORANGEFS_SB(sb));

Re: Orangefs, v4.5 and the merge window...

[Mike Marshall]

Yay, Al, that did it, thanks!

I've tested, and updated the for-next branch at kernel.org...

Linus, if it is not too late, can you pull from:

git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
for-next

-Mike

On Sat, Mar 26, 2016 at 12:30 AM, Al Viro <viro@zeniv.linux.org.uk> wrote:
> On Fri, Mar 25, 2016 at 11:55:31PM -0400, Mike Marshall wrote:
>
>> I'll look to see if I can see it, I guess it has something to
>> do with Al's superblock re-do...
>
> It does; with this approach you would need to add ORANGEFS_VFS_OP_FS_UNMOUNT
> to the whitelist in orangefs_devreq_read() - the
>                                    !(op->upcall.type ==
>                                         ORANGEFS_VFS_OP_FS_MOUNT ||
>                                      op->upcall.type ==
>                                         ORANGEFS_VFS_OP_GETATTR)) {
> thing.  Sorry, should've thought about that...
>
> Alternatively, we could do orangefs_unmount_sb(sb) before removing from
> the list, and add mutex_lock/mutex_unlock of request_mutex right before
> that kfree() in the very end.  Same effect in terms of list protection and
> closer to your current logics.
>
> Try this incremental (to be folded into "fix orangefs_superblock locking"):
>
> diff --git a/fs/orangefs/super.c b/fs/orangefs/super.c
> index bf78870..bb3dc14 100644
> --- a/fs/orangefs/super.c
> +++ b/fs/orangefs/super.c
> @@ -514,6 +514,12 @@ void orangefs_kill_sb(struct super_block *sb)
>         /* provided sb cleanup */
>         kill_anon_super(sb);
>
> +       /*
> +        * issue the unmount to userspace to tell it to remove the
> +        * dynamic mount info it has for this superblock
> +        */
> +       orangefs_unmount_sb(sb);
> +
>         /* remove the sb from our list of orangefs specific sb's */
>
>         spin_lock(&orangefs_superblocks_lock);
> @@ -522,10 +528,11 @@ void orangefs_kill_sb(struct super_block *sb)
>         spin_unlock(&orangefs_superblocks_lock);
>
>         /*
> -        * issue the unmount to userspace to tell it to remove the
> -        * dynamic mount info it has for this superblock
> +        * make sure that ORANGEFS_DEV_REMOUNT_ALL loop that might've seen us
> +        * gets completed before we free the damn thing.
>          */
> -       orangefs_unmount_sb(sb);
> +       mutex_lock(&request_mutex);
> +       mutex_unlock(&request_mutex);
>
>         /* free the orangefs superblock private data */
>         kfree(ORANGEFS_SB(sb));

Re: Orangefs, v4.5 and the merge window...

[Al Viro]

On Sat, Mar 26, 2016 at 08:07:58AM -0400, Mike Marshall wrote:
> Yay, Al, that did it, thanks!
> 
> I've tested, and updated the for-next branch at kernel.org...
> 
> Linus, if it is not too late, can you pull from:
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
> for-next

Shortlog and diffstat would be a good idea...  Something like the output of
git request-pull origin git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git for-next

Longer term you want a remote to avoid typing that crap every time; just be
careful to set both url (git:// one) and pushurl (ssh:// equivalent) for
it.  The former will be usable for somebody to pull from you (and go into
the output of git request-pull), the latter is obviously for pushes...

Re: Orangefs, v4.5 and the merge window...

[Mike Marshall]

OK... I think this is what you're after:

$ git request-pull ko/for-next
git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
for-next

The following changes since commit 45996492e5c85aa0ac93a95d1b2d1ed56851c865:

  orangefs: fix orangefs_superblock locking (2016-03-26 07:22:00 -0400)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git for-next

for you to fetch changes up to 45996492e5c85aa0ac93a95d1b2d1ed56851c865:

  orangefs: fix orangefs_superblock locking (2016-03-26 07:22:00 -0400)

----------------------------------------------------------------

I understand about setting up URL types in your second paragraph,
and I have a github tree that I keep current, I thought we all sort of focused
on kernel.org being the "canonical place" around here... I'll pay more attention
to other pull/push requests on the mailing list...

-Mike

On Sat, Mar 26, 2016 at 10:47 AM, Al Viro <viro@zeniv.linux.org.uk> wrote:
> On Sat, Mar 26, 2016 at 08:07:58AM -0400, Mike Marshall wrote:
>> Yay, Al, that did it, thanks!
>>
>> I've tested, and updated the for-next branch at kernel.org...
>>
>> Linus, if it is not too late, can you pull from:
>>
>> git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
>> for-next
>
> Shortlog and diffstat would be a good idea...  Something like the output of
> git request-pull origin git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git for-next
>
> Longer term you want a remote to avoid typing that crap every time; just be
> careful to set both url (git:// one) and pushurl (ssh:// equivalent) for
> it.  The former will be usable for somebody to pull from you (and go into
> the output of git request-pull), the latter is obviously for pushes...

Re: Orangefs, v4.5 and the merge window...

[Al Viro]

On Sat, Mar 26, 2016 at 11:34:22AM -0400, Mike Marshall wrote:
> OK... I think this is what you're after:
> 
> $ git request-pull ko/for-next
> git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
> for-next
> 
> The following changes since commit 45996492e5c85aa0ac93a95d1b2d1ed56851c865:
> 
>   orangefs: fix orangefs_superblock locking (2016-03-26 07:22:00 -0400)
> 
> are available in the git repository at:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git for-next
> 
> for you to fetch changes up to 45996492e5c85aa0ac93a95d1b2d1ed56851c865:
> 
>   orangefs: fix orangefs_superblock locking (2016-03-26 07:22:00 -0400)

Er...  That ought to have generated shortlog and diffstat as well; the point
is mostly to see what's intended to be included into pull.

See "16) Sending "git pull" requests" in Documentation/SubmittingPatches:

| A pull request should also include an overall message saying what will be
| included in the request, a "git shortlog" listing of the patches
| themselves, and a diffstat showing the overall effect of the patch series.
| The easiest way to get all this information together is, of course, to let
| git do it for you with the "git request-pull" command.

Re: Orangefs, v4.5 and the merge window...

[Mike Marshall]

This is harder than going upstream <g>...

I think I'm doing the same thing greg kh did here, only he
figured it out (and didn't say how he did it right)...

https://lkml.org/lkml/2012/10/1/293

I made a tag, and...

$ git request-pull for-next
git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
ofs-pull-tag-1
The following changes since commit 45996492e5c85aa0ac93a95d1b2d1ed56851c865:

  orangefs: fix orangefs_superblock locking (2016-03-26 07:22:00 -0400)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
tags/ofs-pull-tag-1

for you to fetch changes up to 45996492e5c85aa0ac93a95d1b2d1ed56851c865:

  orangefs: fix orangefs_superblock locking (2016-03-26 07:22:00 -0400)

----------------------------------------------------------------
Orangefs: pull-tag-1

----------------------------------------------------------------

I think that when I made my tag I should have specified some particular
commit, instead of just:  "git tag -a ofs-pull-tag-1" ...
but which commit should I have specified, surely not 001bf455d?

-Mike

On Sat, Mar 26, 2016 at 11:50 AM, Al Viro <viro@zeniv.linux.org.uk> wrote:
> On Sat, Mar 26, 2016 at 11:34:22AM -0400, Mike Marshall wrote:
>> OK... I think this is what you're after:
>>
>> $ git request-pull ko/for-next
>> git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
>> for-next
>>
>> The following changes since commit 45996492e5c85aa0ac93a95d1b2d1ed56851c865:
>>
>>   orangefs: fix orangefs_superblock locking (2016-03-26 07:22:00 -0400)
>>
>> are available in the git repository at:
>>
>>   git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git for-next
>>
>> for you to fetch changes up to 45996492e5c85aa0ac93a95d1b2d1ed56851c865:
>>
>>   orangefs: fix orangefs_superblock locking (2016-03-26 07:22:00 -0400)
>
> Er...  That ought to have generated shortlog and diffstat as well; the point
> is mostly to see what's intended to be included into pull.
>
> See "16) Sending "git pull" requests" in Documentation/SubmittingPatches:
>
> | A pull request should also include an overall message saying what will be
> | included in the request, a "git shortlog" listing of the patches
> | themselves, and a diffstat showing the overall effect of the patch series.
> | The easiest way to get all this information together is, of course, to let
> | git do it for you with the "git request-pull" command.
>

Re: Orangefs, v4.5 and the merge window...

[Al Viro]

On Sat, Mar 26, 2016 at 01:36:04PM -0400, Mike Marshall wrote:
> This is harder than going upstream <g>...
> 
> I think I'm doing the same thing greg kh did here, only he
> figured it out (and didn't say how he did it right)...
> 
> https://lkml.org/lkml/2012/10/1/293
> 
> I made a tag, and...
> 
> $ git request-pull for-next
> git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
> ofs-pull-tag-1
> The following changes since commit 45996492e5c85aa0ac93a95d1b2d1ed56851c865:
> 
>   orangefs: fix orangefs_superblock locking (2016-03-26 07:22:00 -0400)
> 
> are available in the git repository at:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
> tags/ofs-pull-tag-1
> 
> for you to fetch changes up to 45996492e5c85aa0ac93a95d1b2d1ed56851c865:
> 
>   orangefs: fix orangefs_superblock locking (2016-03-26 07:22:00 -0400)

Oh, for...  The point is, *don't* *trim* *diffstat* *and* *shortlog*.  For
your branch that should've been as below and it belongs in pull request,
preferably - with [git pull] somewhere in the subject.  The lines from
dashes below to the end of this reply should have been produced by
git request-pull and should've been a part of your posting.

----------------------------------------------------------------
Al Viro (67):
      orangefs: explicitly pass the size to pvfs_bufmap_copy_to_iovec()
      pvfs_bufmap_copy_from_iovec(): don't rely upon size being equal to iov_iter_count(iter)
      orangefs: make postcopy_buffers() take iov_iter
      orangefs: make precopy_buffers() take iov_iter
      orangefs: make wait_for_direct_io() take iov_iter
      orangefs: don't bother with splitting iovecs
      orangefs: make do_readv_writev() take iov_iter
      orangefs: make pvfs2_inode_read() take iov_iter
      orangefs: kill kmap/kunmap wrappers
      orangefs: use get_user_pages_fast(), not get_user_pages()
      orangefs: double iput() in case of d_make_root() failure
      orangefs: kill struct pvfs2_mount_sb_info_s
      pvfs2_fill_sb(): use kzalloc()
      orangefs: kill pointless ->link() and ->mknod()
      orangefs: sanitize pvfs2_convert_time_field()
      orangefs: switch decode_dirents() to use of kcalloc()
      orangefs: get rid of dec_string and enc_string
      orangefs: don't leave uninitialized data in ->trailer_buf
      orangefs: validate the response in decode_dirents()
      fs: out of bounds on stack in iov_iter_advance
      orangefs: use DEFINE_MUTEX (and mutex_init() had been too late)
      orangefs: generic_file_open() is pointless for character devices
      orangefs: ->poll() is only called between successful ->open() and ->release()
      orangefs: kill ioctl32 rudiments
      orangefs: ->poll() doesn't need spinlock
      orangefs: get rid of <censored> macros
      orangefs: kill orangefs_inode_s ->list
      make orangefs_clean_up_interrupted_operation() static
      orangefs: make wait_for_...downcall() static
      orangefs: move wakeups into set_op_state_{serviced,purged}()
      orangefs: make wait_for_...downcall() static
      orangefs: move wakeups into set_op_state_{serviced,purged}()
      orangefs: nothing should remain in request list and in hash
      orangefs: make sure that reopening pvfs2-req won't overlap with the end of close
      orangefs: hopefully saner op refcounting and locking
      orangefs: remove cargo-culting spin_lock_irqsave() in service_operation()
      orangefs: reduce nesting in wait_for_matching_downcall()
      orangefs_clean_up_interrupted_operation: call with op->lock held
      orangefs: get rid of MSECS_TO_JIFFIES
      if ORANGEFS_VFS_OP_FILE_IO request had been given up, don't bother waiting
      orangefs: don't reinvent completion.h...
      orangefs: move handle_io_error() to file.c
      orangefs: clean up op_alloc()
      orangefs: avoid freeing a slot twice in wait_for_direct_io()
      orangefs: wait_for_direct_io(): restore the position in iter when restarting
      orangefs: get rid of handle_io_error()
      get rid of bufmap argument of orangefs_bufmap_put()
      orangefs: delay freeing slot until cancel completes
      orangefs: get rid of loop in wait_for_matching_downcall()
      orangefs: sanitize handling of request list
      service_operation(): don't block signals, just use ..._killable
      orangefs: lift handling of timeouts and attempts count to service_operation()
      orangefs_bufmap_..._query(): don't bother with refcounts
      orangefs: bufmap rewrite
      orangefs_readdir_index_put(): get rid of bufmap argument
      orangefs: get rid of op->done
      orangefs: set correct ->downcall.status on failing to copy reply from daemon
      orangefs: have ..._clean_interrupted_...() wait for copy to/from daemon
      orangefs: get rid of op refcounts
      orangefs: get rid of readdir_handle_s
      orangefs_copy_{to,from}_bufmap(): don't pass bufmap pointer
      orangefs: saner calling conventions for getting a slot
      orangefs-bufmap.h: trim unused junk
      orangefs: sanitize ->llseek()
      orangefs: have ->kill_sb() evict the VFS side of things first
      orangefs: fix do_readv_writev() handling of error halfway through
      orangefs: fix orangefs_superblock locking

Arnd Bergmann (3):
      orangefs: fix typo in ornagefs_inode_lock
      orangefs: avoid time conversion function
      orangefs: remove unused 'diff' function

Guenter Roeck (1):
      Orangefs: Swap order of include files

Julia Lawall (1):
      OrangeFS: constify export_operations structures

Martin Brandenburg (40):
      Orangefs: Use readonly mmap since writepage is not implemented.
      Orangefs: Clean up error decoding.
      Orangefs: Remove unused #defines from signal blocking code.
      Orangefs: Remove upcall trailers which are not used.
      Orangefs: Clean up pvfs2_devreq_read.
      Orangefs: do not finalize bufmap if it was never initialized.
      orangefs: Remove ``aligned'' upcall and downcall length macros.
      orangefs: Change visibility of several bufmap helpers to static.
      orangefs: Remove useless inline qualifier from bufmap functions.
      orangefs: Do not unref if there is no bufmap.
      orangefs: Fix some more global namespace pollution.
      orangefs: Util functions shouldn't operate on inode where it can be avoided.
      orangefs: Fix revalidate.
      orangefs: Only compare attributes specified in orangefs_inode_getattr.
      orangefs: Implement inode_operations->permission().
      orangefs: Do not retrieve size from servers unless it it necessary.
      orangefs: use S_ISREG(mode) and friends instead of mode & S_IFREG.
      orangefs: free readdir buffer index before the dir_emit loop
      orangefs: don't d_drop in d_revalidate since the caller will
      orangefs: use ORANGEFS_NAME_LEN everywhere; remove ORANGEFS_NAME_MAX
      orangefs: remove vestigial async io code
      orangefs: we never lookup with sym_follow set
      orangefs: clean up fill_default_sys_attrs
      orangefs: Avoid symlink upcall if target is too long.
      orangefs: make fs_mount_pending static
      orangefs: remove unused reference to xattr key length
      orangefs: sanitize listxattr and return EIO on impossible values
      orangefs: remove paranoia in orangefs_set_inode
      orangefs: put register_chrdev immediately before register_filesystem
      orangefs: remove inode->i_lock wrapper
      orangefs: rename orangefs_inode_getattr to orangefs_inode_old_getattr
      orangefs: use new orangefs_inode_getattr to create new inodes
      orangefs: use new orangefs_inode_getattr to get size in write and llseek
      orangefs: use new getattr in inode getattr and permission
      orangefs: use new getattr for revalidate and remove old getattr
      orangefs: refactor inode type or link_target change detection
      orangefs: remove wrapper around mutex_lock(&inode->i_mutex)
      orangefs: remove needless wrapper around GFP_KERNEL
      orangefs: move code which sets i_link to orangefs_inode_getattr
      ornagefs: ensure that truncate has an up to date inode size

Mike Marshall (61):
      Orangefs: kernel client part 1
      Orangefs: kernel client part 2
      Orangefs: kernel client part 3
      Orangefs: kernel client part 4
      Orangefs: kernel client part 5
      Orangefs: kernel client part 6
      Orangefs: kernel client part 7
      Orangefs: kernel client update 1.
      Orangefs: sooth most sparse complaints
      Orangefs: address problems found by static checker
      Orangefs: large integer implicitly truncated to unsigned type
      Orangefs: use inode_set_bytes for directories
      Orangefs: use iov_iter interface
      Orangefs: fix dir_emit code in pvfs2_readdir.
      Orangefs: put PVFS_util_min out of its misery.
      Orangefs: choose return codes from among the expected ones.
      Orangefs: Don't opencode memcpy.
      Orangefs: update signal blocking code before Oleg sees it.
      Orangefs: don't use mount_nodev, use sget directly.
      Orangefs: fix some checkpatch.pl complaints that had creeped in.
      Orangefs: set pos after generic_write_checks
      Orangefs: fix gossip statement
      Orangefs: Merge tag 'v4.4-rc1' into for-next
      Orangefs: change pvfs2 filenames to orangefs
      Orangefs: don't expose internal details of pathname resolution to userspace.
      Orangefs: don't keep checking stuff in on Friday afternoon.
      Orangefs: improve comments
      Orangef: remove overlooked old-style userspace debug parts
      Orangefs: de-uglify orangefs_devreq_writev, and devorangefs-req.c in general
      Orangefs: Don't wait the old-fashioned way.
      Orangefs: don't use deprecated xattr defines.
      Orangefs: validate resp.listxattr.returned_count
      Orangefs: don't change EXTRAVERSION
      Orangefs: don't trigger copy_attributes_to_inode from d_revalidate.
      Orangefs: add orangefs to MAINTAINERS
      Orangefs: implement .write_iter
      Orangefs: rename orangefs_kernel_op_s.aio_ref_count to just ref_count.
      Orangefs: change ORANGEFS_VERSION from "Unknown" to "upstream"
      Orangefs: define a minimum compatible userspace version.
      Orangefs: make .statfs gossip_debug more complete.
      Orangefs: add protocol information to Documentation/filesystems/orangefs.txt
      Orangefs: add verification to decode_dirents
      Orangefs: merge with V4.4
      Orangefs: make gossip statement more palatable to xtensa
      Orangefs: improve gossip statement
      Orangefs: clean up slab allocation.
      Orangefs: added a couple of WARN_ONs, perhaps just temporarily.
      Orangefs: make some gossip statements more helpful.
      Orangefs: remove vestigial ASYNC code
      Orangefs: clean up orangefs_kernel_op_s comments.
      Orangefs: code sanitation
      Orangefs: code sanitation.
      Orangefs: update orangefs.txt
      Orangefs: improve gossip statements
      Orangefs: add a new gossip statement
      Orangefs: improve the POSIXness of interrupted writes...
      Orangefs: merge to v4.5
      Orangefs: Extra sanity insurance on buffer before using string functions on it.
      Orangefs: follow_link -> get_link change
      Orangefs: fix sloppy cleanups of debugfs and sysfs init failures.
      Orangefs: adjust unwind on module init failure.

Nicholas Mc Guire (1):
      Orangefs: use kzalloc for kmalloc + memset 0

Richard Weinberger (1):
      orangefs: Don't pollute global namespace

Sasha Levin (1):
      fs: orangefs: remove execute priviliges from module params

Yi Liu (1):
      OrangeFS: Change almost all instances of the string PVFS2 to OrangeFS.

 Documentation/ABI/stable/sysfs-fs-orangefs |   87 ++
 Documentation/filesystems/orangefs.txt     |  406 +++++++
 MAINTAINERS                                |    8 +
 fs/Kconfig                                 |    1 +
 fs/Makefile                                |    1 +
 fs/orangefs/Kconfig                        |    6 +
 fs/orangefs/Makefile                       |   10 +
 fs/orangefs/acl.c                          |  175 +++
 fs/orangefs/dcache.c                       |  138 +++
 fs/orangefs/devorangefs-req.c              |  943 +++++++++++++++
 fs/orangefs/dir.c                          |  400 +++++++
 fs/orangefs/downcall.h                     |  133 +++
 fs/orangefs/file.c                         |  717 +++++++++++
 fs/orangefs/inode.c                        |  475 ++++++++
 fs/orangefs/namei.c                        |  462 ++++++++
 fs/orangefs/orangefs-bufmap.c              |  556 +++++++++
 fs/orangefs/orangefs-bufmap.h              |   36 +
 fs/orangefs/orangefs-cache.c               |  161 +++
 fs/orangefs/orangefs-debug.h               |   92 ++
 fs/orangefs/orangefs-debugfs.c             |  455 +++++++
 fs/orangefs/orangefs-debugfs.h             |    3 +
 fs/orangefs/orangefs-dev-proto.h           |   62 +
 fs/orangefs/orangefs-kernel.h              |  623 ++++++++++
 fs/orangefs/orangefs-mod.c                 |  293 +++++
 fs/orangefs/orangefs-sysfs.c               | 1772 ++++++++++++++++++++++++++++
 fs/orangefs/orangefs-sysfs.h               |    2 +
 fs/orangefs/orangefs-utils.c               | 1048 ++++++++++++++++
 fs/orangefs/protocol.h                     |  452 +++++++
 fs/orangefs/super.c                        |  559 +++++++++
 fs/orangefs/symlink.c                      |   19 +
 fs/orangefs/upcall.h                       |  246 ++++
 fs/orangefs/waitqueue.c                    |  357 ++++++
 fs/orangefs/xattr.c                        |  545 +++++++++
 33 files changed, 11243 insertions(+)
 create mode 100644 Documentation/ABI/stable/sysfs-fs-orangefs
 create mode 100644 Documentation/filesystems/orangefs.txt
 create mode 100644 fs/orangefs/Kconfig
 create mode 100644 fs/orangefs/Makefile
 create mode 100644 fs/orangefs/acl.c
 create mode 100644 fs/orangefs/dcache.c
 create mode 100644 fs/orangefs/devorangefs-req.c
 create mode 100644 fs/orangefs/dir.c
 create mode 100644 fs/orangefs/downcall.h
 create mode 100644 fs/orangefs/file.c
 create mode 100644 fs/orangefs/inode.c
 create mode 100644 fs/orangefs/namei.c
 create mode 100644 fs/orangefs/orangefs-bufmap.c
 create mode 100644 fs/orangefs/orangefs-bufmap.h
 create mode 100644 fs/orangefs/orangefs-cache.c
 create mode 100644 fs/orangefs/orangefs-debug.h
 create mode 100644 fs/orangefs/orangefs-debugfs.c
 create mode 100644 fs/orangefs/orangefs-debugfs.h
 create mode 100644 fs/orangefs/orangefs-dev-proto.h
 create mode 100644 fs/orangefs/orangefs-kernel.h
 create mode 100644 fs/orangefs/orangefs-mod.c
 create mode 100644 fs/orangefs/orangefs-sysfs.c
 create mode 100644 fs/orangefs/orangefs-sysfs.h
 create mode 100644 fs/orangefs/orangefs-utils.c
 create mode 100644 fs/orangefs/protocol.h
 create mode 100644 fs/orangefs/super.c
 create mode 100644 fs/orangefs/symlink.c
 create mode 100644 fs/orangefs/upcall.h
 create mode 100644 fs/orangefs/waitqueue.c
 create mode 100644 fs/orangefs/xattr.c

Re: Orangefs, v4.5 and the merge window...

[Al Viro]

On Sat, Mar 26, 2016 at 06:28:29PM +0000, Al Viro wrote:

> Oh, for...  The point is, *don't* *trim* *diffstat* *and* *shortlog*.  For
> your branch that should've been as below and it belongs in pull request,
> preferably - with [git pull] somewhere in the subject.  The lines from
> dashes below to the end of this reply should have been produced by
> git request-pull and should've been a part of your posting.

PS: I've pulled your branch into vfs.git#orangefs; pull request for that
one would look so:

Subject: [git pull] orangefs initial merge

	Orangefs client.  Note that the regular tree lives at
git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git and subsequent
pull requests will be from there (in fact, this branch is identical to
#for-next in there).

The following changes since commit b562e44f507e863c6792946e4e1b1449fbbac85d:

  Linux 4.5 (2016-03-13 21:28:54 -0700)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs.git orangefs

for you to fetch changes up to 45996492e5c85aa0ac93a95d1b2d1ed56851c865:

  orangefs: fix orangefs_superblock locking (2016-03-26 07:22:00 -0400)

----------------------------------------------------------------
Al Viro (67):
      orangefs: explicitly pass the size to pvfs_bufmap_copy_to_iovec()
      pvfs_bufmap_copy_from_iovec(): don't rely upon size being equal to iov_iter_count(iter)
      orangefs: make postcopy_buffers() take iov_iter
      orangefs: make precopy_buffers() take iov_iter
      orangefs: make wait_for_direct_io() take iov_iter
      orangefs: don't bother with splitting iovecs
      orangefs: make do_readv_writev() take iov_iter
      orangefs: make pvfs2_inode_read() take iov_iter
      orangefs: kill kmap/kunmap wrappers
      orangefs: use get_user_pages_fast(), not get_user_pages()
      orangefs: double iput() in case of d_make_root() failure
      orangefs: kill struct pvfs2_mount_sb_info_s
      pvfs2_fill_sb(): use kzalloc()
      orangefs: kill pointless ->link() and ->mknod()
      orangefs: sanitize pvfs2_convert_time_field()
      orangefs: switch decode_dirents() to use of kcalloc()
      orangefs: get rid of dec_string and enc_string
      orangefs: don't leave uninitialized data in ->trailer_buf
      orangefs: validate the response in decode_dirents()
      fs: out of bounds on stack in iov_iter_advance
      orangefs: use DEFINE_MUTEX (and mutex_init() had been too late)
      orangefs: generic_file_open() is pointless for character devices
      orangefs: ->poll() is only called between successful ->open() and ->release()
      orangefs: kill ioctl32 rudiments
      orangefs: ->poll() doesn't need spinlock
      orangefs: get rid of <censored> macros
      orangefs: kill orangefs_inode_s ->list
      make orangefs_clean_up_interrupted_operation() static
      orangefs: make wait_for_...downcall() static
      orangefs: move wakeups into set_op_state_{serviced,purged}()
      orangefs: make wait_for_...downcall() static
      orangefs: move wakeups into set_op_state_{serviced,purged}()
      orangefs: nothing should remain in request list and in hash
      orangefs: make sure that reopening pvfs2-req won't overlap with the end of close
      orangefs: hopefully saner op refcounting and locking
      orangefs: remove cargo-culting spin_lock_irqsave() in service_operation()
      orangefs: reduce nesting in wait_for_matching_downcall()
      orangefs_clean_up_interrupted_operation: call with op->lock held
      orangefs: get rid of MSECS_TO_JIFFIES
      if ORANGEFS_VFS_OP_FILE_IO request had been given up, don't bother waiting
      orangefs: don't reinvent completion.h...
      orangefs: move handle_io_error() to file.c
      orangefs: clean up op_alloc()
      orangefs: avoid freeing a slot twice in wait_for_direct_io()
      orangefs: wait_for_direct_io(): restore the position in iter when restarting
      orangefs: get rid of handle_io_error()
      get rid of bufmap argument of orangefs_bufmap_put()
      orangefs: delay freeing slot until cancel completes
      orangefs: get rid of loop in wait_for_matching_downcall()
      orangefs: sanitize handling of request list
      service_operation(): don't block signals, just use ..._killable
      orangefs: lift handling of timeouts and attempts count to service_operation()
      orangefs_bufmap_..._query(): don't bother with refcounts
      orangefs: bufmap rewrite
      orangefs_readdir_index_put(): get rid of bufmap argument
      orangefs: get rid of op->done
      orangefs: set correct ->downcall.status on failing to copy reply from daemon
      orangefs: have ..._clean_interrupted_...() wait for copy to/from daemon
      orangefs: get rid of op refcounts
      orangefs: get rid of readdir_handle_s
      orangefs_copy_{to,from}_bufmap(): don't pass bufmap pointer
      orangefs: saner calling conventions for getting a slot
      orangefs-bufmap.h: trim unused junk
      orangefs: sanitize ->llseek()
      orangefs: have ->kill_sb() evict the VFS side of things first
      orangefs: fix do_readv_writev() handling of error halfway through
      orangefs: fix orangefs_superblock locking

Arnd Bergmann (3):
      orangefs: fix typo in ornagefs_inode_lock
      orangefs: avoid time conversion function
      orangefs: remove unused 'diff' function

Guenter Roeck (1):
      Orangefs: Swap order of include files

Julia Lawall (1):
      OrangeFS: constify export_operations structures

Martin Brandenburg (40):
      Orangefs: Use readonly mmap since writepage is not implemented.
      Orangefs: Clean up error decoding.
      Orangefs: Remove unused #defines from signal blocking code.
      Orangefs: Remove upcall trailers which are not used.
      Orangefs: Clean up pvfs2_devreq_read.
      Orangefs: do not finalize bufmap if it was never initialized.
      orangefs: Remove ``aligned'' upcall and downcall length macros.
      orangefs: Change visibility of several bufmap helpers to static.
      orangefs: Remove useless inline qualifier from bufmap functions.
      orangefs: Do not unref if there is no bufmap.
      orangefs: Fix some more global namespace pollution.
      orangefs: Util functions shouldn't operate on inode where it can be avoided.
      orangefs: Fix revalidate.
      orangefs: Only compare attributes specified in orangefs_inode_getattr.
      orangefs: Implement inode_operations->permission().
      orangefs: Do not retrieve size from servers unless it it necessary.
      orangefs: use S_ISREG(mode) and friends instead of mode & S_IFREG.
      orangefs: free readdir buffer index before the dir_emit loop
      orangefs: don't d_drop in d_revalidate since the caller will
      orangefs: use ORANGEFS_NAME_LEN everywhere; remove ORANGEFS_NAME_MAX
      orangefs: remove vestigial async io code
      orangefs: we never lookup with sym_follow set
      orangefs: clean up fill_default_sys_attrs
      orangefs: Avoid symlink upcall if target is too long.
      orangefs: make fs_mount_pending static
      orangefs: remove unused reference to xattr key length
      orangefs: sanitize listxattr and return EIO on impossible values
      orangefs: remove paranoia in orangefs_set_inode
      orangefs: put register_chrdev immediately before register_filesystem
      orangefs: remove inode->i_lock wrapper
      orangefs: rename orangefs_inode_getattr to orangefs_inode_old_getattr
      orangefs: use new orangefs_inode_getattr to create new inodes
      orangefs: use new orangefs_inode_getattr to get size in write and llseek
      orangefs: use new getattr in inode getattr and permission
      orangefs: use new getattr for revalidate and remove old getattr
      orangefs: refactor inode type or link_target change detection
      orangefs: remove wrapper around mutex_lock(&inode->i_mutex)
      orangefs: remove needless wrapper around GFP_KERNEL
      orangefs: move code which sets i_link to orangefs_inode_getattr
      ornagefs: ensure that truncate has an up to date inode size

Mike Marshall (61):
      Orangefs: kernel client part 1
      Orangefs: kernel client part 2
      Orangefs: kernel client part 3
      Orangefs: kernel client part 4
      Orangefs: kernel client part 5
      Orangefs: kernel client part 6
      Orangefs: kernel client part 7
      Orangefs: kernel client update 1.
      Orangefs: sooth most sparse complaints
      Orangefs: address problems found by static checker
      Orangefs: large integer implicitly truncated to unsigned type
      Orangefs: use inode_set_bytes for directories
      Orangefs: use iov_iter interface
      Orangefs: fix dir_emit code in pvfs2_readdir.
      Orangefs: put PVFS_util_min out of its misery.
      Orangefs: choose return codes from among the expected ones.
      Orangefs: Don't opencode memcpy.
      Orangefs: update signal blocking code before Oleg sees it.
      Orangefs: don't use mount_nodev, use sget directly.
      Orangefs: fix some checkpatch.pl complaints that had creeped in.
      Orangefs: set pos after generic_write_checks
      Orangefs: fix gossip statement
      Orangefs: Merge tag 'v4.4-rc1' into for-next
      Orangefs: change pvfs2 filenames to orangefs
      Orangefs: don't expose internal details of pathname resolution to userspace.
      Orangefs: don't keep checking stuff in on Friday afternoon.
      Orangefs: improve comments
      Orangef: remove overlooked old-style userspace debug parts
      Orangefs: de-uglify orangefs_devreq_writev, and devorangefs-req.c in general
      Orangefs: Don't wait the old-fashioned way.
      Orangefs: don't use deprecated xattr defines.
      Orangefs: validate resp.listxattr.returned_count
      Orangefs: don't change EXTRAVERSION
      Orangefs: don't trigger copy_attributes_to_inode from d_revalidate.
      Orangefs: add orangefs to MAINTAINERS
      Orangefs: implement .write_iter
      Orangefs: rename orangefs_kernel_op_s.aio_ref_count to just ref_count.
      Orangefs: change ORANGEFS_VERSION from "Unknown" to "upstream"
      Orangefs: define a minimum compatible userspace version.
      Orangefs: make .statfs gossip_debug more complete.
      Orangefs: add protocol information to Documentation/filesystems/orangefs.txt
      Orangefs: add verification to decode_dirents
      Orangefs: merge with V4.4
      Orangefs: make gossip statement more palatable to xtensa
      Orangefs: improve gossip statement
      Orangefs: clean up slab allocation.
      Orangefs: added a couple of WARN_ONs, perhaps just temporarily.
      Orangefs: make some gossip statements more helpful.
      Orangefs: remove vestigial ASYNC code
      Orangefs: clean up orangefs_kernel_op_s comments.
      Orangefs: code sanitation
      Orangefs: code sanitation.
      Orangefs: update orangefs.txt
      Orangefs: improve gossip statements
      Orangefs: add a new gossip statement
      Orangefs: improve the POSIXness of interrupted writes...
      Orangefs: merge to v4.5
      Orangefs: Extra sanity insurance on buffer before using string functions on it.
      Orangefs: follow_link -> get_link change
      Orangefs: fix sloppy cleanups of debugfs and sysfs init failures.
      Orangefs: adjust unwind on module init failure.

Nicholas Mc Guire (1):
      Orangefs: use kzalloc for kmalloc + memset 0

Richard Weinberger (1):
      orangefs: Don't pollute global namespace

Sasha Levin (1):
      fs: orangefs: remove execute priviliges from module params

Yi Liu (1):
      OrangeFS: Change almost all instances of the string PVFS2 to OrangeFS.

 Documentation/ABI/stable/sysfs-fs-orangefs |   87 ++
 Documentation/filesystems/orangefs.txt     |  406 +++++++
 MAINTAINERS                                |    8 +
 fs/Kconfig                                 |    1 +
 fs/Makefile                                |    1 +
 fs/orangefs/Kconfig                        |    6 +
 fs/orangefs/Makefile                       |   10 +
 fs/orangefs/acl.c                          |  175 +++
 fs/orangefs/dcache.c                       |  138 +++
 fs/orangefs/devorangefs-req.c              |  943 +++++++++++++++
 fs/orangefs/dir.c                          |  400 +++++++
 fs/orangefs/downcall.h                     |  133 +++
 fs/orangefs/file.c                         |  717 +++++++++++
 fs/orangefs/inode.c                        |  475 ++++++++
 fs/orangefs/namei.c                        |  462 ++++++++
 fs/orangefs/orangefs-bufmap.c              |  556 +++++++++
 fs/orangefs/orangefs-bufmap.h              |   36 +
 fs/orangefs/orangefs-cache.c               |  161 +++
 fs/orangefs/orangefs-debug.h               |   92 ++
 fs/orangefs/orangefs-debugfs.c             |  455 +++++++
 fs/orangefs/orangefs-debugfs.h             |    3 +
 fs/orangefs/orangefs-dev-proto.h           |   62 +
 fs/orangefs/orangefs-kernel.h              |  623 ++++++++++
 fs/orangefs/orangefs-mod.c                 |  293 +++++
 fs/orangefs/orangefs-sysfs.c               | 1772 ++++++++++++++++++++++++++++
 fs/orangefs/orangefs-sysfs.h               |    2 +
 fs/orangefs/orangefs-utils.c               | 1048 ++++++++++++++++
 fs/orangefs/protocol.h                     |  452 +++++++
 fs/orangefs/super.c                        |  559 +++++++++
 fs/orangefs/symlink.c                      |   19 +
 fs/orangefs/upcall.h                       |  246 ++++
 fs/orangefs/waitqueue.c                    |  357 ++++++
 fs/orangefs/xattr.c                        |  545 +++++++++
 33 files changed, 11243 insertions(+)
 create mode 100644 Documentation/ABI/stable/sysfs-fs-orangefs
 create mode 100644 Documentation/filesystems/orangefs.txt
 create mode 100644 fs/orangefs/Kconfig
 create mode 100644 fs/orangefs/Makefile
 create mode 100644 fs/orangefs/acl.c
 create mode 100644 fs/orangefs/dcache.c
 create mode 100644 fs/orangefs/devorangefs-req.c
 create mode 100644 fs/orangefs/dir.c
 create mode 100644 fs/orangefs/downcall.h
 create mode 100644 fs/orangefs/file.c
 create mode 100644 fs/orangefs/inode.c
 create mode 100644 fs/orangefs/namei.c
 create mode 100644 fs/orangefs/orangefs-bufmap.c
 create mode 100644 fs/orangefs/orangefs-bufmap.h
 create mode 100644 fs/orangefs/orangefs-cache.c
 create mode 100644 fs/orangefs/orangefs-debug.h
 create mode 100644 fs/orangefs/orangefs-debugfs.c
 create mode 100644 fs/orangefs/orangefs-debugfs.h
 create mode 100644 fs/orangefs/orangefs-dev-proto.h
 create mode 100644 fs/orangefs/orangefs-kernel.h
 create mode 100644 fs/orangefs/orangefs-mod.c
 create mode 100644 fs/orangefs/orangefs-sysfs.c
 create mode 100644 fs/orangefs/orangefs-sysfs.h
 create mode 100644 fs/orangefs/orangefs-utils.c
 create mode 100644 fs/orangefs/protocol.h
 create mode 100644 fs/orangefs/super.c
 create mode 100644 fs/orangefs/symlink.c
 create mode 100644 fs/orangefs/upcall.h
 create mode 100644 fs/orangefs/waitqueue.c
 create mode 100644 fs/orangefs/xattr.c

Re: Orangefs, v4.5 and the merge window...

[Mike Marshall]

I think I've got it, Martin helped...

I made the tag right on top of the last thing in our branch that we
got from Linus: b562e44f5

pushed out the tag, and...

$ git request-pull ofs-pull-tag-2
git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git

I'll send out a message to Al, CC to Linus and fs-devel with
"[git pull] orangefs initial merge" as the subject,
and the body will be the below text generated by "git request-pull"

-Mike



The following changes since commit b562e44f507e863c6792946e4e1b1449fbbac85d:

  Linux 4.5 (2016-03-13 21:28:54 -0700)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
tags/ofs-pull-tag-1

for you to fetch changes up to 45996492e5c85aa0ac93a95d1b2d1ed56851c865:

  orangefs: fix orangefs_superblock locking (2016-03-26 07:22:00 -0400)

----------------------------------------------------------------
Orangefs: pull-tag-1

----------------------------------------------------------------
Al Viro (67):
      orangefs: explicitly pass the size to pvfs_bufmap_copy_to_iovec()
      pvfs_bufmap_copy_from_iovec(): don't rely upon size being equal
to iov_iter_count(iter)
      orangefs: make postcopy_buffers() take iov_iter
      orangefs: make precopy_buffers() take iov_iter
      orangefs: make wait_for_direct_io() take iov_iter
      orangefs: don't bother with splitting iovecs
      orangefs: make do_readv_writev() take iov_iter
      orangefs: make pvfs2_inode_read() take iov_iter
      orangefs: kill kmap/kunmap wrappers
      orangefs: use get_user_pages_fast(), not get_user_pages()
      orangefs: double iput() in case of d_make_root() failure
      orangefs: kill struct pvfs2_mount_sb_info_s
      pvfs2_fill_sb(): use kzalloc()
      orangefs: kill pointless ->link() and ->mknod()
      orangefs: sanitize pvfs2_convert_time_field()
      orangefs: switch decode_dirents() to use of kcalloc()
      orangefs: get rid of dec_string and enc_string
      orangefs: don't leave uninitialized data in ->trailer_buf
      orangefs: validate the response in decode_dirents()
      fs: out of bounds on stack in iov_iter_advance
      orangefs: use DEFINE_MUTEX (and mutex_init() had been too late)
      orangefs: generic_file_open() is pointless for character devices
      orangefs: ->poll() is only called between successful ->open()
and ->release()
      orangefs: kill ioctl32 rudiments
      orangefs: ->poll() doesn't need spinlock
      orangefs: get rid of <censored> macros
      orangefs: kill orangefs_inode_s ->list
      make orangefs_clean_up_interrupted_operation() static
      orangefs: make wait_for_...downcall() static
      orangefs: move wakeups into set_op_state_{serviced,purged}()
      orangefs: make wait_for_...downcall() static
      orangefs: move wakeups into set_op_state_{serviced,purged}()
      orangefs: nothing should remain in request list and in hash
      orangefs: make sure that reopening pvfs2-req won't overlap with
the end of close
      orangefs: hopefully saner op refcounting and locking
      orangefs: remove cargo-culting spin_lock_irqsave() in service_operation()
      orangefs: reduce nesting in wait_for_matching_downcall()
      orangefs_clean_up_interrupted_operation: call with op->lock held
      orangefs: get rid of MSECS_TO_JIFFIES
      if ORANGEFS_VFS_OP_FILE_IO request had been given up, don't bother waiting
      orangefs: don't reinvent completion.h...
      orangefs: move handle_io_error() to file.c
      orangefs: clean up op_alloc()
      orangefs: avoid freeing a slot twice in wait_for_direct_io()
      orangefs: wait_for_direct_io(): restore the position in iter
when restarting
      orangefs: get rid of handle_io_error()
      get rid of bufmap argument of orangefs_bufmap_put()
      orangefs: delay freeing slot until cancel completes
      orangefs: get rid of loop in wait_for_matching_downcall()
      orangefs: sanitize handling of request list
      service_operation(): don't block signals, just use ..._killable
      orangefs: lift handling of timeouts and attempts count to
service_operation()
      orangefs_bufmap_..._query(): don't bother with refcounts
      orangefs: bufmap rewrite
      orangefs_readdir_index_put(): get rid of bufmap argument
      orangefs: get rid of op->done
      orangefs: set correct ->downcall.status on failing to copy reply
from daemon
      orangefs: have ..._clean_interrupted_...() wait for copy to/from daemon
      orangefs: get rid of op refcounts
      orangefs: get rid of readdir_handle_s
      orangefs_copy_{to,from}_bufmap(): don't pass bufmap pointer
      orangefs: saner calling conventions for getting a slot
      orangefs-bufmap.h: trim unused junk
      orangefs: sanitize ->llseek()
      orangefs: have ->kill_sb() evict the VFS side of things first
      orangefs: fix do_readv_writev() handling of error halfway through
      orangefs: fix orangefs_superblock locking

Arnd Bergmann (3):
      orangefs: fix typo in ornagefs_inode_lock
      orangefs: avoid time conversion function
      orangefs: remove unused 'diff' function

Guenter Roeck (1):
      Orangefs: Swap order of include files

Julia Lawall (1):
      OrangeFS: constify export_operations structures

Martin Brandenburg (40):
      Orangefs: Use readonly mmap since writepage is not implemented.
      Orangefs: Clean up error decoding.
      Orangefs: Remove unused #defines from signal blocking code.
      Orangefs: Remove upcall trailers which are not used.
      Orangefs: Clean up pvfs2_devreq_read.
      Orangefs: do not finalize bufmap if it was never initialized.
      orangefs: Remove ``aligned'' upcall and downcall length macros.
      orangefs: Change visibility of several bufmap helpers to static.
      orangefs: Remove useless inline qualifier from bufmap functions.
      orangefs: Do not unref if there is no bufmap.
      orangefs: Fix some more global namespace pollution.
      orangefs: Util functions shouldn't operate on inode where it can
be avoided.
      orangefs: Fix revalidate.
      orangefs: Only compare attributes specified in orangefs_inode_getattr.
      orangefs: Implement inode_operations->permission().
      orangefs: Do not retrieve size from servers unless it it necessary.
      orangefs: use S_ISREG(mode) and friends instead of mode & S_IFREG.
      orangefs: free readdir buffer index before the dir_emit loop
      orangefs: don't d_drop in d_revalidate since the caller will
      orangefs: use ORANGEFS_NAME_LEN everywhere; remove ORANGEFS_NAME_MAX
      orangefs: remove vestigial async io code
      orangefs: we never lookup with sym_follow set
      orangefs: clean up fill_default_sys_attrs
      orangefs: Avoid symlink upcall if target is too long.
      orangefs: make fs_mount_pending static
      orangefs: remove unused reference to xattr key length
      orangefs: sanitize listxattr and return EIO on impossible values
      orangefs: remove paranoia in orangefs_set_inode
      orangefs: put register_chrdev immediately before register_filesystem
      orangefs: remove inode->i_lock wrapper
      orangefs: rename orangefs_inode_getattr to orangefs_inode_old_getattr
      orangefs: use new orangefs_inode_getattr to create new inodes
      orangefs: use new orangefs_inode_getattr to get size in write and llseek
      orangefs: use new getattr in inode getattr and permission
      orangefs: use new getattr for revalidate and remove old getattr
      orangefs: refactor inode type or link_target change detection
      orangefs: remove wrapper around mutex_lock(&inode->i_mutex)
      orangefs: remove needless wrapper around GFP_KERNEL
      orangefs: move code which sets i_link to orangefs_inode_getattr
      ornagefs: ensure that truncate has an up to date inode size

Mike Marshall (61):
      Orangefs: kernel client part 1
      Orangefs: kernel client part 2
      Orangefs: kernel client part 3
      Orangefs: kernel client part 4
      Orangefs: kernel client part 5
      Orangefs: kernel client part 6
      Orangefs: kernel client part 7
      Orangefs: kernel client update 1.
      Orangefs: sooth most sparse complaints
      Orangefs: address problems found by static checker
      Orangefs: large integer implicitly truncated to unsigned type
      Orangefs: use inode_set_bytes for directories
      Orangefs: use iov_iter interface
      Orangefs: fix dir_emit code in pvfs2_readdir.
      Orangefs: put PVFS_util_min out of its misery.
      Orangefs: choose return codes from among the expected ones.
      Orangefs: Don't opencode memcpy.
      Orangefs: update signal blocking code before Oleg sees it.
      Orangefs: don't use mount_nodev, use sget directly.
      Orangefs: fix some checkpatch.pl complaints that had creeped in.
      Orangefs: set pos after generic_write_checks
      Orangefs: fix gossip statement
      Orangefs: Merge tag 'v4.4-rc1' into for-next
      Orangefs: change pvfs2 filenames to orangefs
      Orangefs: don't expose internal details of pathname resolution
to userspace.
      Orangefs: don't keep checking stuff in on Friday afternoon.
      Orangefs: improve comments
      Orangef: remove overlooked old-style userspace debug parts
      Orangefs: de-uglify orangefs_devreq_writev, and
devorangefs-req.c in general
      Orangefs: Don't wait the old-fashioned way.
      Orangefs: don't use deprecated xattr defines.
      Orangefs: validate resp.listxattr.returned_count
      Orangefs: don't change EXTRAVERSION
      Orangefs: don't trigger copy_attributes_to_inode from d_revalidate.
      Orangefs: add orangefs to MAINTAINERS
      Orangefs: implement .write_iter
      Orangefs: rename orangefs_kernel_op_s.aio_ref_count to just ref_count.
      Orangefs: change ORANGEFS_VERSION from "Unknown" to "upstream"
      Orangefs: define a minimum compatible userspace version.
      Orangefs: make .statfs gossip_debug more complete.
      Orangefs: add protocol information to
Documentation/filesystems/orangefs.txt
      Orangefs: add verification to decode_dirents
      Orangefs: merge with V4.4
      Orangefs: make gossip statement more palatable to xtensa
      Orangefs: improve gossip statement
      Orangefs: clean up slab allocation.
      Orangefs: added a couple of WARN_ONs, perhaps just temporarily.
      Orangefs: make some gossip statements more helpful.
      Orangefs: remove vestigial ASYNC code
      Orangefs: clean up orangefs_kernel_op_s comments.
      Orangefs: code sanitation
      Orangefs: code sanitation.
      Orangefs: update orangefs.txt
      Orangefs: improve gossip statements
      Orangefs: add a new gossip statement
      Orangefs: improve the POSIXness of interrupted writes...
      Orangefs: merge to v4.5
      Orangefs: Extra sanity insurance on buffer before using string
functions on it.
      Orangefs: follow_link -> get_link change
      Orangefs: fix sloppy cleanups of debugfs and sysfs init failures.
      Orangefs: adjust unwind on module init failure.

Nicholas Mc Guire (1):
      Orangefs: use kzalloc for kmalloc + memset 0

Richard Weinberger (1):
      orangefs: Don't pollute global namespace

Sasha Levin (1):
      fs: orangefs: remove execute priviliges from module params

Yi Liu (1):
      OrangeFS: Change almost all instances of the string PVFS2 to OrangeFS.

 Documentation/ABI/stable/sysfs-fs-orangefs |   87 ++
 Documentation/filesystems/orangefs.txt     |  406 +++++++
 MAINTAINERS                                |    8 +
 fs/Kconfig                                 |    1 +
 fs/Makefile                                |    1 +
 fs/orangefs/Kconfig                        |    6 +
 fs/orangefs/Makefile                       |   10 +
 fs/orangefs/acl.c                          |  175 +++
 fs/orangefs/dcache.c                       |  138 +++
 fs/orangefs/devorangefs-req.c              |  943 +++++++++++++++
 fs/orangefs/dir.c                          |  400 +++++++
 fs/orangefs/downcall.h                     |  133 +++
 fs/orangefs/file.c                         |  717 +++++++++++
 fs/orangefs/inode.c                        |  475 ++++++++
 fs/orangefs/namei.c                        |  462 ++++++++
 fs/orangefs/orangefs-bufmap.c              |  556 +++++++++
 fs/orangefs/orangefs-bufmap.h              |   36 +
 fs/orangefs/orangefs-cache.c               |  161 +++
 fs/orangefs/orangefs-debug.h               |   92 ++
 fs/orangefs/orangefs-debugfs.c             |  455 +++++++
 fs/orangefs/orangefs-debugfs.h             |    3 +
 fs/orangefs/orangefs-dev-proto.h           |   62 +
 fs/orangefs/orangefs-kernel.h              |  623 ++++++++++
 fs/orangefs/orangefs-mod.c                 |  293 +++++
 fs/orangefs/orangefs-sysfs.c               | 1772 ++++++++++++++++++++++++++++
 fs/orangefs/orangefs-sysfs.h               |    2 +
 fs/orangefs/orangefs-utils.c               | 1048 ++++++++++++++++
 fs/orangefs/protocol.h                     |  452 +++++++
 fs/orangefs/super.c                        |  559 +++++++++
 fs/orangefs/symlink.c                      |   19 +
 fs/orangefs/upcall.h                       |  246 ++++
 fs/orangefs/waitqueue.c                    |  357 ++++++
 fs/orangefs/xattr.c                        |  545 +++++++++
 33 files changed, 11243 insertions(+)
 create mode 100644 Documentation/ABI/stable/sysfs-fs-orangefs
 create mode 100644 Documentation/filesystems/orangefs.txt
 create mode 100644 fs/orangefs/Kconfig
 create mode 100644 fs/orangefs/Makefile
 create mode 100644 fs/orangefs/acl.c
 create mode 100644 fs/orangefs/dcache.c
 create mode 100644 fs/orangefs/devorangefs-req.c
 create mode 100644 fs/orangefs/dir.c
 create mode 100644 fs/orangefs/downcall.h
 create mode 100644 fs/orangefs/file.c
 create mode 100644 fs/orangefs/inode.c
 create mode 100644 fs/orangefs/namei.c
 create mode 100644 fs/orangefs/orangefs-bufmap.c
 create mode 100644 fs/orangefs/orangefs-bufmap.h
 create mode 100644 fs/orangefs/orangefs-cache.c
 create mode 100644 fs/orangefs/orangefs-debug.h
 create mode 100644 fs/orangefs/orangefs-debugfs.c
 create mode 100644 fs/orangefs/orangefs-debugfs.h
 create mode 100644 fs/orangefs/orangefs-dev-proto.h
 create mode 100644 fs/orangefs/orangefs-kernel.h
 create mode 100644 fs/orangefs/orangefs-mod.c
 create mode 100644 fs/orangefs/orangefs-sysfs.c
 create mode 100644 fs/orangefs/orangefs-sysfs.h
 create mode 100644 fs/orangefs/orangefs-utils.c
 create mode 100644 fs/orangefs/protocol.h
 create mode 100644 fs/orangefs/super.c
 create mode 100644 fs/orangefs/symlink.c
 create mode 100644 fs/orangefs/upcall.h
 create mode 100644 fs/orangefs/waitqueue.c
 create mode 100644 fs/orangefs/xattr.c

On Sat, Mar 26, 2016 at 2:37 PM, Al Viro <viro@zeniv.linux.org.uk> wrote:
> On Sat, Mar 26, 2016 at 06:28:29PM +0000, Al Viro wrote:
>
>> Oh, for...  The point is, *don't* *trim* *diffstat* *and* *shortlog*.  For
>> your branch that should've been as below and it belongs in pull request,
>> preferably - with [git pull] somewhere in the subject.  The lines from
>> dashes below to the end of this reply should have been produced by
>> git request-pull and should've been a part of your posting.
>
> PS: I've pulled your branch into vfs.git#orangefs; pull request for that
> one would look so:
>
> Subject: [git pull] orangefs initial merge
>
>         Orangefs client.  Note that the regular tree lives at
> git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git and subsequent
> pull requests will be from there (in fact, this branch is identical to
> #for-next in there).
>
> The following changes since commit b562e44f507e863c6792946e4e1b1449fbbac85d:
>
>   Linux 4.5 (2016-03-13 21:28:54 -0700)
>
> are available in the git repository at:
>
>   git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs.git orangefs
>
> for you to fetch changes up to 45996492e5c85aa0ac93a95d1b2d1ed56851c865:
>
>   orangefs: fix orangefs_superblock locking (2016-03-26 07:22:00 -0400)
>
> ----------------------------------------------------------------
> Al Viro (67):
>       orangefs: explicitly pass the size to pvfs_bufmap_copy_to_iovec()
>       pvfs_bufmap_copy_from_iovec(): don't rely upon size being equal to iov_iter_count(iter)
>       orangefs: make postcopy_buffers() take iov_iter
>       orangefs: make precopy_buffers() take iov_iter
>       orangefs: make wait_for_direct_io() take iov_iter
>       orangefs: don't bother with splitting iovecs
>       orangefs: make do_readv_writev() take iov_iter
>       orangefs: make pvfs2_inode_read() take iov_iter
>       orangefs: kill kmap/kunmap wrappers
>       orangefs: use get_user_pages_fast(), not get_user_pages()
>       orangefs: double iput() in case of d_make_root() failure
>       orangefs: kill struct pvfs2_mount_sb_info_s
>       pvfs2_fill_sb(): use kzalloc()
>       orangefs: kill pointless ->link() and ->mknod()
>       orangefs: sanitize pvfs2_convert_time_field()
>       orangefs: switch decode_dirents() to use of kcalloc()
>       orangefs: get rid of dec_string and enc_string
>       orangefs: don't leave uninitialized data in ->trailer_buf
>       orangefs: validate the response in decode_dirents()
>       fs: out of bounds on stack in iov_iter_advance
>       orangefs: use DEFINE_MUTEX (and mutex_init() had been too late)
>       orangefs: generic_file_open() is pointless for character devices
>       orangefs: ->poll() is only called between successful ->open() and ->release()
>       orangefs: kill ioctl32 rudiments
>       orangefs: ->poll() doesn't need spinlock
>       orangefs: get rid of <censored> macros
>       orangefs: kill orangefs_inode_s ->list
>       make orangefs_clean_up_interrupted_operation() static
>       orangefs: make wait_for_...downcall() static
>       orangefs: move wakeups into set_op_state_{serviced,purged}()
>       orangefs: make wait_for_...downcall() static
>       orangefs: move wakeups into set_op_state_{serviced,purged}()
>       orangefs: nothing should remain in request list and in hash
>       orangefs: make sure that reopening pvfs2-req won't overlap with the end of close
>       orangefs: hopefully saner op refcounting and locking
>       orangefs: remove cargo-culting spin_lock_irqsave() in service_operation()
>       orangefs: reduce nesting in wait_for_matching_downcall()
>       orangefs_clean_up_interrupted_operation: call with op->lock held
>       orangefs: get rid of MSECS_TO_JIFFIES
>       if ORANGEFS_VFS_OP_FILE_IO request had been given up, don't bother waiting
>       orangefs: don't reinvent completion.h...
>       orangefs: move handle_io_error() to file.c
>       orangefs: clean up op_alloc()
>       orangefs: avoid freeing a slot twice in wait_for_direct_io()
>       orangefs: wait_for_direct_io(): restore the position in iter when restarting
>       orangefs: get rid of handle_io_error()
>       get rid of bufmap argument of orangefs_bufmap_put()
>       orangefs: delay freeing slot until cancel completes
>       orangefs: get rid of loop in wait_for_matching_downcall()
>       orangefs: sanitize handling of request list
>       service_operation(): don't block signals, just use ..._killable
>       orangefs: lift handling of timeouts and attempts count to service_operation()
>       orangefs_bufmap_..._query(): don't bother with refcounts
>       orangefs: bufmap rewrite
>       orangefs_readdir_index_put(): get rid of bufmap argument
>       orangefs: get rid of op->done
>       orangefs: set correct ->downcall.status on failing to copy reply from daemon
>       orangefs: have ..._clean_interrupted_...() wait for copy to/from daemon
>       orangefs: get rid of op refcounts
>       orangefs: get rid of readdir_handle_s
>       orangefs_copy_{to,from}_bufmap(): don't pass bufmap pointer
>       orangefs: saner calling conventions for getting a slot
>       orangefs-bufmap.h: trim unused junk
>       orangefs: sanitize ->llseek()
>       orangefs: have ->kill_sb() evict the VFS side of things first
>       orangefs: fix do_readv_writev() handling of error halfway through
>       orangefs: fix orangefs_superblock locking
>
> Arnd Bergmann (3):
>       orangefs: fix typo in ornagefs_inode_lock
>       orangefs: avoid time conversion function
>       orangefs: remove unused 'diff' function
>
> Guenter Roeck (1):
>       Orangefs: Swap order of include files
>
> Julia Lawall (1):
>       OrangeFS: constify export_operations structures
>
> Martin Brandenburg (40):
>       Orangefs: Use readonly mmap since writepage is not implemented.
>       Orangefs: Clean up error decoding.
>       Orangefs: Remove unused #defines from signal blocking code.
>       Orangefs: Remove upcall trailers which are not used.
>       Orangefs: Clean up pvfs2_devreq_read.
>       Orangefs: do not finalize bufmap if it was never initialized.
>       orangefs: Remove ``aligned'' upcall and downcall length macros.
>       orangefs: Change visibility of several bufmap helpers to static.
>       orangefs: Remove useless inline qualifier from bufmap functions.
>       orangefs: Do not unref if there is no bufmap.
>       orangefs: Fix some more global namespace pollution.
>       orangefs: Util functions shouldn't operate on inode where it can be avoided.
>       orangefs: Fix revalidate.
>       orangefs: Only compare attributes specified in orangefs_inode_getattr.
>       orangefs: Implement inode_operations->permission().
>       orangefs: Do not retrieve size from servers unless it it necessary.
>       orangefs: use S_ISREG(mode) and friends instead of mode & S_IFREG.
>       orangefs: free readdir buffer index before the dir_emit loop
>       orangefs: don't d_drop in d_revalidate since the caller will
>       orangefs: use ORANGEFS_NAME_LEN everywhere; remove ORANGEFS_NAME_MAX
>       orangefs: remove vestigial async io code
>       orangefs: we never lookup with sym_follow set
>       orangefs: clean up fill_default_sys_attrs
>       orangefs: Avoid symlink upcall if target is too long.
>       orangefs: make fs_mount_pending static
>       orangefs: remove unused reference to xattr key length
>       orangefs: sanitize listxattr and return EIO on impossible values
>       orangefs: remove paranoia in orangefs_set_inode
>       orangefs: put register_chrdev immediately before register_filesystem
>       orangefs: remove inode->i_lock wrapper
>       orangefs: rename orangefs_inode_getattr to orangefs_inode_old_getattr
>       orangefs: use new orangefs_inode_getattr to create new inodes
>       orangefs: use new orangefs_inode_getattr to get size in write and llseek
>       orangefs: use new getattr in inode getattr and permission
>       orangefs: use new getattr for revalidate and remove old getattr
>       orangefs: refactor inode type or link_target change detection
>       orangefs: remove wrapper around mutex_lock(&inode->i_mutex)
>       orangefs: remove needless wrapper around GFP_KERNEL
>       orangefs: move code which sets i_link to orangefs_inode_getattr
>       ornagefs: ensure that truncate has an up to date inode size
>
> Mike Marshall (61):
>       Orangefs: kernel client part 1
>       Orangefs: kernel client part 2
>       Orangefs: kernel client part 3
>       Orangefs: kernel client part 4
>       Orangefs: kernel client part 5
>       Orangefs: kernel client part 6
>       Orangefs: kernel client part 7
>       Orangefs: kernel client update 1.
>       Orangefs: sooth most sparse complaints
>       Orangefs: address problems found by static checker
>       Orangefs: large integer implicitly truncated to unsigned type
>       Orangefs: use inode_set_bytes for directories
>       Orangefs: use iov_iter interface
>       Orangefs: fix dir_emit code in pvfs2_readdir.
>       Orangefs: put PVFS_util_min out of its misery.
>       Orangefs: choose return codes from among the expected ones.
>       Orangefs: Don't opencode memcpy.
>       Orangefs: update signal blocking code before Oleg sees it.
>       Orangefs: don't use mount_nodev, use sget directly.
>       Orangefs: fix some checkpatch.pl complaints that had creeped in.
>       Orangefs: set pos after generic_write_checks
>       Orangefs: fix gossip statement
>       Orangefs: Merge tag 'v4.4-rc1' into for-next
>       Orangefs: change pvfs2 filenames to orangefs
>       Orangefs: don't expose internal details of pathname resolution to userspace.
>       Orangefs: don't keep checking stuff in on Friday afternoon.
>       Orangefs: improve comments
>       Orangef: remove overlooked old-style userspace debug parts
>       Orangefs: de-uglify orangefs_devreq_writev, and devorangefs-req.c in general
>       Orangefs: Don't wait the old-fashioned way.
>       Orangefs: don't use deprecated xattr defines.
>       Orangefs: validate resp.listxattr.returned_count
>       Orangefs: don't change EXTRAVERSION
>       Orangefs: don't trigger copy_attributes_to_inode from d_revalidate.
>       Orangefs: add orangefs to MAINTAINERS
>       Orangefs: implement .write_iter
>       Orangefs: rename orangefs_kernel_op_s.aio_ref_count to just ref_count.
>       Orangefs: change ORANGEFS_VERSION from "Unknown" to "upstream"
>       Orangefs: define a minimum compatible userspace version.
>       Orangefs: make .statfs gossip_debug more complete.
>       Orangefs: add protocol information to Documentation/filesystems/orangefs.txt
>       Orangefs: add verification to decode_dirents
>       Orangefs: merge with V4.4
>       Orangefs: make gossip statement more palatable to xtensa
>       Orangefs: improve gossip statement
>       Orangefs: clean up slab allocation.
>       Orangefs: added a couple of WARN_ONs, perhaps just temporarily.
>       Orangefs: make some gossip statements more helpful.
>       Orangefs: remove vestigial ASYNC code
>       Orangefs: clean up orangefs_kernel_op_s comments.
>       Orangefs: code sanitation
>       Orangefs: code sanitation.
>       Orangefs: update orangefs.txt
>       Orangefs: improve gossip statements
>       Orangefs: add a new gossip statement
>       Orangefs: improve the POSIXness of interrupted writes...
>       Orangefs: merge to v4.5
>       Orangefs: Extra sanity insurance on buffer before using string functions on it.
>       Orangefs: follow_link -> get_link change
>       Orangefs: fix sloppy cleanups of debugfs and sysfs init failures.
>       Orangefs: adjust unwind on module init failure.
>
> Nicholas Mc Guire (1):
>       Orangefs: use kzalloc for kmalloc + memset 0
>
> Richard Weinberger (1):
>       orangefs: Don't pollute global namespace
>
> Sasha Levin (1):
>       fs: orangefs: remove execute priviliges from module params
>
> Yi Liu (1):
>       OrangeFS: Change almost all instances of the string PVFS2 to OrangeFS.
>
>  Documentation/ABI/stable/sysfs-fs-orangefs |   87 ++
>  Documentation/filesystems/orangefs.txt     |  406 +++++++
>  MAINTAINERS                                |    8 +
>  fs/Kconfig                                 |    1 +
>  fs/Makefile                                |    1 +
>  fs/orangefs/Kconfig                        |    6 +
>  fs/orangefs/Makefile                       |   10 +
>  fs/orangefs/acl.c                          |  175 +++
>  fs/orangefs/dcache.c                       |  138 +++
>  fs/orangefs/devorangefs-req.c              |  943 +++++++++++++++
>  fs/orangefs/dir.c                          |  400 +++++++
>  fs/orangefs/downcall.h                     |  133 +++
>  fs/orangefs/file.c                         |  717 +++++++++++
>  fs/orangefs/inode.c                        |  475 ++++++++
>  fs/orangefs/namei.c                        |  462 ++++++++
>  fs/orangefs/orangefs-bufmap.c              |  556 +++++++++
>  fs/orangefs/orangefs-bufmap.h              |   36 +
>  fs/orangefs/orangefs-cache.c               |  161 +++
>  fs/orangefs/orangefs-debug.h               |   92 ++
>  fs/orangefs/orangefs-debugfs.c             |  455 +++++++
>  fs/orangefs/orangefs-debugfs.h             |    3 +
>  fs/orangefs/orangefs-dev-proto.h           |   62 +
>  fs/orangefs/orangefs-kernel.h              |  623 ++++++++++
>  fs/orangefs/orangefs-mod.c                 |  293 +++++
>  fs/orangefs/orangefs-sysfs.c               | 1772 ++++++++++++++++++++++++++++
>  fs/orangefs/orangefs-sysfs.h               |    2 +
>  fs/orangefs/orangefs-utils.c               | 1048 ++++++++++++++++
>  fs/orangefs/protocol.h                     |  452 +++++++
>  fs/orangefs/super.c                        |  559 +++++++++
>  fs/orangefs/symlink.c                      |   19 +
>  fs/orangefs/upcall.h                       |  246 ++++
>  fs/orangefs/waitqueue.c                    |  357 ++++++
>  fs/orangefs/xattr.c                        |  545 +++++++++
>  33 files changed, 11243 insertions(+)
>  create mode 100644 Documentation/ABI/stable/sysfs-fs-orangefs
>  create mode 100644 Documentation/filesystems/orangefs.txt
>  create mode 100644 fs/orangefs/Kconfig
>  create mode 100644 fs/orangefs/Makefile
>  create mode 100644 fs/orangefs/acl.c
>  create mode 100644 fs/orangefs/dcache.c
>  create mode 100644 fs/orangefs/devorangefs-req.c
>  create mode 100644 fs/orangefs/dir.c
>  create mode 100644 fs/orangefs/downcall.h
>  create mode 100644 fs/orangefs/file.c
>  create mode 100644 fs/orangefs/inode.c
>  create mode 100644 fs/orangefs/namei.c
>  create mode 100644 fs/orangefs/orangefs-bufmap.c
>  create mode 100644 fs/orangefs/orangefs-bufmap.h
>  create mode 100644 fs/orangefs/orangefs-cache.c
>  create mode 100644 fs/orangefs/orangefs-debug.h
>  create mode 100644 fs/orangefs/orangefs-debugfs.c
>  create mode 100644 fs/orangefs/orangefs-debugfs.h
>  create mode 100644 fs/orangefs/orangefs-dev-proto.h
>  create mode 100644 fs/orangefs/orangefs-kernel.h
>  create mode 100644 fs/orangefs/orangefs-mod.c
>  create mode 100644 fs/orangefs/orangefs-sysfs.c
>  create mode 100644 fs/orangefs/orangefs-sysfs.h
>  create mode 100644 fs/orangefs/orangefs-utils.c
>  create mode 100644 fs/orangefs/protocol.h
>  create mode 100644 fs/orangefs/super.c
>  create mode 100644 fs/orangefs/symlink.c
>  create mode 100644 fs/orangefs/upcall.h
>  create mode 100644 fs/orangefs/waitqueue.c
>  create mode 100644 fs/orangefs/xattr.c

Re: Orangefs, v4.5 and the merge window...

[Linus Torvalds]

On Sat, Mar 26, 2016 at 12:00 PM, Mike Marshall <hubcap@omnibond.com> wrote:
>
> I made the tag right on top of the last thing in our branch that we
> got from Linus: b562e44f5

So you really don't need to make *that* a tag.

The only tag you want to have is the tag that describes your own
top-of-tree if you want to send me a signed tag (which really is the
preferred mode).

Then, all you do is:

 - have some pointer to my tree - it's often just "origin", but you
can also literally just fetch my tree into a separate branch

   (note the *fetch* - not a pull. So you can do something like

 - give that origin pointer as the start

So something like

    git remote add linus
git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

would add the remote "linus" (unless you already have a remote
"origin" that points to that upstream repository), and then you can
just do

    git fetch linus

and that will get whatever state I have at the time into your
"linus/master" branch. Note that the "git remote add" you only need to
do once (and again, a "git clone" will automatically add an "origin"
remote, so you may have one already).

Then, the best practices is to make a signed tag of *your* work - you
could call it something like "for-linus-4.6", for example:

    git tag -s for-linus-4.6

and write a tag message and sign it with your key. I prefer that
signed tag to contain some useful description of what I'm actually
pulling, not just "Orangefs: pull-tag-1". And I do really want it to
be a *signed* tag for it to actually make sense.

After that, just do

    git request-pull remotes/linus/master \
        git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
tags/for-linus-4.6

which should do the right thing.

That said, the request-pull you sent out looks mostly fine - it's just
that you did an unnecessary tag there to create the base commit that
you really shouldn't have needed to do, and the tag you asked me to
pull was an unsigned one and didn't contain any useful information.

So I pulled it, but for next time:

 - please use a signed tag. I don't actually require it for kernel.org
pulls, but it's nice, and it's particularly nice if the signed tag
contents then describe what I'm pulling

 - don't do the extraneous tag that doesn't actually help or matter.

but it's in my tree now and I'll push it out after

            Linus

Re: Orangefs, v4.5 and the merge window...

[Mike Marshall]

 > it's in my tree now and I'll push it out after

That is beyond awesome!

I tried to make a signed tag, but never got past:

"You need a passphrase to unlock the secret key..."

it just sat there waiting for me to... do something?
I was ssh'd into my workstation at work, perhaps
the prompt came out there?

I config'd git with my public key as reported
by gpg --list-keys

"git config --global user.signingkey xyzzy"

Anyhow, I'll  work to make sure I can do it
properly next time, thanks so much for
taking Orangefs!

And, please enjoy this cartoon I drew in the spirit it
is offered <g>:

http://myweb.clemson.edu/~hubcap/toons/shipIt.jpg

-Mike


On Sat, Mar 26, 2016 at 3:51 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Sat, Mar 26, 2016 at 12:00 PM, Mike Marshall <hubcap@omnibond.com> wrote:
>>
>> I made the tag right on top of the last thing in our branch that we
>> got from Linus: b562e44f5
>
> So you really don't need to make *that* a tag.
>
> The only tag you want to have is the tag that describes your own
> top-of-tree if you want to send me a signed tag (which really is the
> preferred mode).
>
> Then, all you do is:
>
>  - have some pointer to my tree - it's often just "origin", but you
> can also literally just fetch my tree into a separate branch
>
>    (note the *fetch* - not a pull. So you can do something like
>
>  - give that origin pointer as the start
>
> So something like
>
>     git remote add linus
> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
>
> would add the remote "linus" (unless you already have a remote
> "origin" that points to that upstream repository), and then you can
> just do
>
>     git fetch linus
>
> and that will get whatever state I have at the time into your
> "linus/master" branch. Note that the "git remote add" you only need to
> do once (and again, a "git clone" will automatically add an "origin"
> remote, so you may have one already).
>
> Then, the best practices is to make a signed tag of *your* work - you
> could call it something like "for-linus-4.6", for example:
>
>     git tag -s for-linus-4.6
>
> and write a tag message and sign it with your key. I prefer that
> signed tag to contain some useful description of what I'm actually
> pulling, not just "Orangefs: pull-tag-1". And I do really want it to
> be a *signed* tag for it to actually make sense.
>
> After that, just do
>
>     git request-pull remotes/linus/master \
>         git://git.kernel.org/pub/scm/linux/kernel/git/hubcap/linux.git
> tags/for-linus-4.6
>
> which should do the right thing.
>
> That said, the request-pull you sent out looks mostly fine - it's just
> that you did an unnecessary tag there to create the base commit that
> you really shouldn't have needed to do, and the tag you asked me to
> pull was an unsigned one and didn't contain any useful information.
>
> So I pulled it, but for next time:
>
>  - please use a signed tag. I don't actually require it for kernel.org
> pulls, but it's nice, and it's particularly nice if the signed tag
> contents then describe what I'm pulling
>
>  - don't do the extraneous tag that doesn't actually help or matter.
>
> but it's in my tree now and I'll push it out after
>
>             Linus

Re: Orangefs, v4.5 and the merge window...

[Linus Torvalds]

On Sat, Mar 26, 2016 at 1:47 PM, Mike Marshall <hubcap@omnibond.com> wrote:
>  > it's in my tree now and I'll push it out after
>
> That is beyond awesome!
>
> I tried to make a signed tag, but never got past:
>
> "You need a passphrase to unlock the secret key..."
>
> it just sat there waiting for me to... do something?
> I was ssh'd into my workstation at work, perhaps
> the prompt came out there?

Hmm. Are you running gpg-agent? That apparently gets confused easily.

Normally "git tag -s" should ask you to write a tag message (using the
same editor you write your commit messages with).

It should just ask for your passphrase, looking something like this:

  You need a passphrase to unlock the secret key for
  user: "xyz"
  key-details-go-here

  Enter passphrase:

but I've seen gpg-agent get confused before. Are you sure you have a
working gpg setup on that machine? It might also be that there's a GUI
window open on your workstation that you can't see over ssh.

I love the convenience of gpg-agent, but quite frankly, I gave up on
it years ago because of how fragile it was (although honestly, I think
most of the fragility I saw was with the gnome keyring integration, so
I don't know who to blame - I just decided that it's easier to type my
passphrases than it is to try to figure out which part of the mess was
broken).

            Linus

Re: Orangefs, v4.5 and the merge window...

[Mike Marshall]

There's the orangefs.txt in Documentation.

And I'm listed as a maintainer, I forget what file that's in...

But you're right, we mostly don't touch anything out of fs/orangefs...

-Mike

On Fri, Mar 25, 2016 at 8:28 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
> On Mar 25, 2016 5:21 PM, "Al Viro" <viro@zeniv.linux.org.uk> wrote:
>>
>> Linus, would you be OK with that thing going into -rc2?  It's
>> self-contained
>> and I think it's in reasonably sane shape.  If not for the last locking
>> fixes,
>> I would even suggest -rc1 with fixups of debugfs issues during the next
>> week,
>> but that last commit got no testing whatsoever... ;-/
>
> Gaah. I'm going to be traveling starting tomorrow night (red-eye) and I was
> going to cut rc1 tomorrow and then really hope for a calm rc2 week.
>
> At the same time, is live to get this finally merged, about a year after I
> originally thought were would. So I guess I'll take it, even if I hate the
> timing.
>
> There are zero changes outside orangefs itself, and obviously the
> Makefile/KConfig file to get it all enabled?
>
>     Linus

Re: Orangefs, v4.5 and the merge window...

[Martin Brandenburg]

On Fri, 11 Mar 2016, Al Viro wrote:

> IIRC, there also had been some unpleasantness with getattr messing ->i_mode
> halfway through... <checks>  Yes - copy_attributes_to_inode() will be called,
> and do
>        inode->i_mode = orangefs_inode_perms(attrs);
> ...
>                inode->i_mode |= S_IFLNK;
> ...
>                        strncpy(orangefs_inode->link_target,
>                                symname,
>                                ORANGEFS_NAME_MAX);
> If nothing else, *another* stat(2) racing with this one could pick the
> intermediate value of ->i_mode and proceed to report it to userland.
> Another problem is overwriting the symlink body; that can get very
> unpleasant, since it might be traversed by another syscall right at that
> moment.  Any change of a symlink body means "we'd missed it going stale";
> there is no way to change a symlink contents without removing it and
> creating a new one.  Should anything other than orangefs_iget() even bother
> copying it?  The same goes for inode type changes, of course (regular
> vs. directory vs. symlink, etc.).

I thought I fixed most of that when I went over
d_revalidate. But now I see that there's the case that
the server attribute changes after revalidate and before
the operation e.g. chmod. What happens to the inode we
now know is stale? What if two processes notice it's
stale at the same time?

We can go ahead with the chmod even if the object type
has changed since the kernel last saw it. But OrangeFS
doesn't have NFS's sillyrename. So what do we do in the
middle of read when we get ENOENT? We ignore POSIX and
return it to userspace anyway and leave the bad inode
around now.

There's lots of unsanitized error codes from the server
going to userspace now.

Will writing i_mode in one shot fix it?

-- Martin

Re: Orangefs, v4.5 and the merge window...

[Martin Brandenburg]

On Tue, 15 Mar 2016, Martin Brandenburg wrote:

> On Fri, 11 Mar 2016, Al Viro wrote:
>
>> IIRC, there also had been some unpleasantness with getattr messing ->i_mode
>> halfway through... <checks>  Yes - copy_attributes_to_inode() will be 
>> called,
>> and do
>>        inode->i_mode = orangefs_inode_perms(attrs);
>> ...
>>                inode->i_mode |= S_IFLNK;
>> ...
>>                        strncpy(orangefs_inode->link_target,
>>                                symname,
>>                                ORANGEFS_NAME_MAX);
>> If nothing else, *another* stat(2) racing with this one could pick the
>> intermediate value of ->i_mode and proceed to report it to userland.
>> Another problem is overwriting the symlink body; that can get very
>> unpleasant, since it might be traversed by another syscall right at that
>> moment.  Any change of a symlink body means "we'd missed it going stale";
>> there is no way to change a symlink contents without removing it and
>> creating a new one.  Should anything other than orangefs_iget() even bother
>> copying it?  The same goes for inode type changes, of course (regular
>> vs. directory vs. symlink, etc.).
>
> I thought I fixed most of that when I went over
> d_revalidate. But now I see that there's the case that
> the server attribute changes after revalidate and before
> the operation e.g. chmod. What happens to the inode we
> now know is stale? What if two processes notice it's
> stale at the same time?
>
> We can go ahead with the chmod even if the object type
> has changed since the kernel last saw it. But OrangeFS
> doesn't have NFS's sillyrename. So what do we do in the
> middle of read when we get ENOENT? We ignore POSIX and
> return it to userspace anyway and leave the bad inode
> around now.
>
> There's lots of unsanitized error codes from the server
> going to userspace now.
>
> Will writing i_mode in one shot fix it?
>
> -- Martin
>

This is where I'm going with this. Not done yet, but you
can see where it's going hopefully.

https://github.com/martinbrandenburg/linux.git branch getattr

-- Martin

[PATCH] orangefs: getattr work (was: Re: Orangefs, v4.5 and the merge window...)

[Martin Brandenburg]

Mike,

The following changes since commit 2f83ace37181e445cab83d1d2a3a1dc88a36a814:

  orangefs: put register_chrdev immediately before register_filesystem (2016-03-17 14:34:10 -0400)

are available in the git repository at:

  https://github.com/martinbrandenburg/linux.git getattr

for you to fetch changes up to 6f0d78264c5c4e0e9da8f98e60da82f87b4a5a99:

  orangefs: remove wrapper around mutex_lock(&inode->i_mutex) (2016-03-17 16:33:08 -0400)

----------------------------------------------------------------
Martin Brandenburg (8):
      orangefs: remove inode->i_lock wrapper
      orangefs: rename orangefs_inode_getattr to orangefs_inode_old_getattr
      orangefs: use new orangefs_inode_getattr to create new inodes
      orangefs: use new orangefs_inode_getattr to get size in write and llseek
      orangefs: use new getattr in inode getattr and permission
      orangefs: add orangefs_inode_check_changed
      orangefs: refactor inode type or link_target change detection
      orangefs: remove wrapper around mutex_lock(&inode->i_mutex)

 fs/orangefs/dcache.c          |   3 +-
 fs/orangefs/file.c            |  13 +-
 fs/orangefs/inode.c           |  19 +-
 fs/orangefs/orangefs-kernel.h |  15 +-
 fs/orangefs/orangefs-utils.c  | 400 ++++++++++++-----------------------------
 5 files changed, 133 insertions(+), 317 deletions(-)

I hope this cleans up getattr significantly and fixes several
outstanding bugs. It is tested rather lightly as we have discussed.

-- Martin