[PATCH] binfmt_misc: Fix binfmt_misc priority

[PATCH] binfmt_misc: Fix binfmt_misc priority

[Jan Sembera]

Commit 74641f584da introduced a regression - binfmt_misc is now consulted
after binfmt_elf, which will unfortunately break ia32el. ia32 ELF binaries
on ia64 used to be matched using binfmt_misc and executed using wrapper. As
32bit binaries are now matched by binfmt_elf before bindmt_misc kicks in,
the wrapper is ignored.

The fix increases precedence of binfmt_misc to the original state.

Signed-off-by: Jan Sembera <jsembera@suse.cz>
---
 fs/binfmt_misc.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/fs/binfmt_misc.c b/fs/binfmt_misc.c
index a7528b9..fd0cc0b 100644
--- a/fs/binfmt_misc.c
+++ b/fs/binfmt_misc.c
@@ -724,7 +724,7 @@ static int __init init_misc_binfmt(void)
 {
 	int err = register_filesystem(&bm_fs_type);
 	if (!err) {
-		err = register_binfmt(&misc_format);
+		err = insert_binfmt(&misc_format);
 		if (err)
 			unregister_filesystem(&bm_fs_type);
 	}

Re: [PATCH] binfmt_misc: Fix binfmt_misc priority

[Andrew Morton]

(cc stable@kernel.org and the ia64 d00ds)

On Mon, 23 Aug 2010 16:15:20 +0200
Jan Sembera <jsembera@suse.cz> wrote:

> Commit 74641f584da introduced a regression - binfmt_misc is now consulted
> after binfmt_elf, which will unfortunately break ia32el. ia32 ELF binaries
> on ia64 used to be matched using binfmt_misc and executed using wrapper. As
> 32bit binaries are now matched by binfmt_elf before bindmt_misc kicks in,
> the wrapper is ignored.
> 
> The fix increases precedence of binfmt_misc to the original state.
> 
> Signed-off-by: Jan Sembera <jsembera@suse.cz>
> ---
>  fs/binfmt_misc.c |    2 +-
>  1 files changed, 1 insertions(+), 1 deletions(-)
> 
> diff --git a/fs/binfmt_misc.c b/fs/binfmt_misc.c
> index a7528b9..fd0cc0b 100644
> --- a/fs/binfmt_misc.c
> +++ b/fs/binfmt_misc.c
> @@ -724,7 +724,7 @@ static int __init init_misc_binfmt(void)
>  {
>  	int err = register_filesystem(&bm_fs_type);
>  	if (!err) {
> -		err = register_binfmt(&misc_format);
> +		err = insert_binfmt(&misc_format);
>  		if (err)
>  			unregister_filesystem(&bm_fs_type);
>  	}

Yikes, 74641f584da was merged in May 2009.  I wonder why this problem
wasn't noticed before now?

If we're going to change this then I guess we should change it in
2.6.36 and backport the fix as far as we can go.

Re: [PATCH] binfmt_misc: Fix binfmt_misc priority

[Jan Sembera]

On Fri, Sep 03, 2010 at 02:20:03PM -0700, Andrew Morton wrote:
> (cc stable@kernel.org and the ia64 d00ds)

Note that this isn't actually ia64 specific, ia32el is just where we
(actually our customer) noticed this first. It happens on any architectures,
if by any chance you would need to match something with binfmt_misc that
would be accepted by another binfmt...

> > Commit 74641f584da introduced a regression - binfmt_misc is now consulted
> > after binfmt_elf, which will unfortunately break ia32el. ia32 ELF binaries
> > on ia64 used to be matched using binfmt_misc and executed using wrapper. As
> > 32bit binaries are now matched by binfmt_elf before bindmt_misc kicks in,
> > the wrapper is ignored.
> > 
> > The fix increases precedence of binfmt_misc to the original state.
> > 
> > Signed-off-by: Jan Sembera <jsembera@suse.cz>
> > ---
> >  fs/binfmt_misc.c |    2 +-
> >  1 files changed, 1 insertions(+), 1 deletions(-)
> > 
> > diff --git a/fs/binfmt_misc.c b/fs/binfmt_misc.c
> > index a7528b9..fd0cc0b 100644
> > --- a/fs/binfmt_misc.c
> > +++ b/fs/binfmt_misc.c
> > @@ -724,7 +724,7 @@ static int __init init_misc_binfmt(void)
> >  {
> >  	int err = register_filesystem(&bm_fs_type);
> >  	if (!err) {
> > -		err = register_binfmt(&misc_format);
> > +		err = insert_binfmt(&misc_format);
> >  		if (err)
> >  			unregister_filesystem(&bm_fs_type);
> >  	}
>
> Yikes, 74641f584da was merged in May 2009.  I wonder why this problem
> wasn't noticed before now?

Not sure, probably not very common use case and it slipped through our
testing before release.

Thanks,

Jan

Re: [PATCH] binfmt_misc: Fix binfmt_misc priority

[Chuck Ebbert]

On Fri, 3 Sep 2010 14:20:03 -0700
Andrew Morton <akpm@linux-foundation.org> wrote:

> On Mon, 23 Aug 2010 16:15:20 +0200
> Jan Sembera <jsembera@suse.cz> wrote:
> 
> > Commit 74641f584da introduced a regression - binfmt_misc is now consulted
> > after binfmt_elf, which will unfortunately break ia32el. ia32 ELF binaries
> > on ia64 used to be matched using binfmt_misc and executed using wrapper. As
> > 32bit binaries are now matched by binfmt_elf before bindmt_misc kicks in,
> > the wrapper is ignored.

> Yikes, 74641f584da was merged in May 2009.  I wonder why this problem
> wasn't noticed before now?

I posted about this on Sep 15, 2009:

http://marc.info/?l=linux-kernel&m=125298891709482&w=4
"binfmt_misc takes precedence over binfmt_script in 2.6.30"

I also copied the author of the original patch, but there was no reply.

[PATCH] binfmt_misc: Fix binfmt_misc priority

[Jan Sembera]

This is resend of a patch that was sent couple of days ago, but I got
no response to it. No changes from previous version.

---
binfmt_misc: Fix binfmt_misc priority

Commit 74641f584da introduced a regression - binfmt_misc is now consulted
after binfmt_elf, which will unfortunately break ia32el. ia32 ELF binaries
on ia64 used to be matched using binfmt_misc and executed using wrapper. As
32bit binaries are now matched by binfmt_elf before bindmt_misc kicks in,
the wrapper is ignored.

The fix increases precedence of binfmt_misc to the original state.

Signed-off-by: Jan Sembera <jsembera@suse.cz>
---
 fs/binfmt_misc.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/fs/binfmt_misc.c b/fs/binfmt_misc.c
index a7528b9..fd0cc0b 100644
--- a/fs/binfmt_misc.c
+++ b/fs/binfmt_misc.c
@@ -724,7 +724,7 @@ static int __init init_misc_binfmt(void)
 {
 	int err = register_filesystem(&bm_fs_type);
 	if (!err) {
-		err = register_binfmt(&misc_format);
+		err = insert_binfmt(&misc_format);
 		if (err)
 			unregister_filesystem(&bm_fs_type);
 	}