grub-pe2elf

grub-pe2elf

[Robert Millan]

Hi,

On a recent discussion on IRC, Bean pointed out that grub-pe2elf is essentially
a workaround for a bug in objcopy.

Bean explained that this bug is quite obvious.  It seems Christian sent them
a patch, but it hasn't been applied (I don't know if it was rejected, or just
ignored, Christian could you explain?).

Back in 2007, Christian proposed an ad-hoc solution for converting from PE to
ELF:

  http://lists.gnu.org/archive/html/grub-devel/2007-11/msg00174.html

which was rejected (see my comment and Marco's reply).

Later in July 2008, Bean added grub-pe2elf to the repository.

I keep thinking the same I thought in 2007.  I don't agree that we should
add such workarounds just because objcopy has a bug.  binutils is a GNU
program.  It is free software and community-maintained.  If it has bugs,
we should be working with them instead of reimplementing their work.

First of all, I'm worried that we have discussions to decide things and they
are later forgotten or ignored.  Bean, please can you provide some explanation
on what happened?

Also, I think this commit should be reverted (at least the part that adds
grub-pe2elf).  Also, I'd like to hear input on what happened with Christian's
binutils patch, why it wasn't applied, etc.

Nevertheless, it's not an obligation for us to support win32.  As long as we
can do it sanely, I'm fine with it, but I don't want to see GRUB dragged into
a situation in which we need to preserve win32 support at all costs.

Our primary target are GNU-based systems, as well as other free software
systems, specially those that rely on GRUB as their default bootloader.

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: grub-pe2elf

[Christian Franke]

Hi Robert,

Robert Millan wrote:
> On a recent discussion on IRC, Bean pointed out that grub-pe2elf is essentially
> a workaround for a bug in objcopy.
>
>   

It is essentially a limitation of BFD: The object format specific 
backends are apparently not implemented such that conversion between 
different relocation formats is supported.

In particular, a relocation 'addend' (-4 in this case) is lost during 
conversion if source or target relocation format do not support it as 
separate field. Some conversions may work. PE -> ELF does not work, the 
problem is in both backends in this case.


> Bean explained that this bug is quite obvious.  It seems Christian sent them
> a patch, but it hasn't been applied (I don't know if it was rejected, or just
> ignored, Christian could you explain?).
>
>   

The patch was actually an ad-hoc approach to fix it in objcopy itself 
for the PE->ELF case only:
http://sourceware.org/ml/binutils/2007-10/msg00302.html

A generic solution which would have more chances to get accepted would 
require substantial work in BFD itself. Conversion between relocation 
formats is apparently rarely needed. A related bugzilla entry was 
reopened in 2005 and is idle since then:
http://sourceware.org/bugzilla/show_bug.cgi?id=970


> Later in July 2008, Bean added grub-pe2elf to the repository.
>
>   

Yes, and I really appreciated this :-)


> First of all, I'm worried that we have discussions to decide things and they
> are later forgotten or ignored.  Bean, please can you provide some explanation
> on what happened?
>
> Also, I think this commit should be reverted (at least the part that adds
> grub-pe2elf).  

I would regret this. Then you could also revert all my Cygwin related 
additions (some also included in Bean's r1726) because these no longer 
make much sense unless someone fixes BFD or maintains a ELF toolchain 
for Cygwin.


> Nevertheless, it's not an obligation for us to support win32.  As long as we
> can do it sanely, I'm fine with it, but I don't want to see GRUB dragged into
> a situation in which we need to preserve win32 support at all costs.
>
>   

$ find grub2 -type f -name '*.[ch]' | xargs cat | wc -l
128175

$ cat grub2/util/grub-pe2elf.c | wc -l
521

0.4% of the total LoC is IMO less than 'all costs' :-)

-- 
Regards
Christian Franke

Re: grub-pe2elf

[Robert Millan]

On Fri, Jul 31, 2009 at 08:27:58PM +0200, Christian Franke wrote:
>> First of all, I'm worried that we have discussions to decide things and they
>> are later forgotten or ignored.  Bean, please can you provide some explanation
>> on what happened?
>>
>> Also, I think this commit should be reverted (at least the part that adds
>> grub-pe2elf).  
>
> I would regret this. Then you could also revert all my Cygwin related  
> additions (some also included in Bean's r1726) because these no longer  
> make much sense unless someone fixes BFD or maintains a ELF toolchain  
> for Cygwin.

But we supported Cygwin before grub-pe2elf.  What has changed?

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: grub-pe2elf

[Christian Franke]

Robert Millan wrote:
> On Fri, Jul 31, 2009 at 08:27:58PM +0200, Christian Franke wrote:
>   
>>> First of all, I'm worried that we have discussions to decide things and they
>>> are later forgotten or ignored.  Bean, please can you provide some explanation
>>> on what happened?
>>>
>>> Also, I think this commit should be reverted (at least the part that adds
>>> grub-pe2elf).  
>>>       
>> I would regret this. Then you could also revert all my Cygwin related  
>> additions (some also included in Bean's r1726) because these no longer  
>> make much sense unless someone fixes BFD or maintains a ELF toolchain  
>> for Cygwin.
>>     
>
> But we supported Cygwin before grub-pe2elf.  What has changed?
>
>   

Since ~r1584, util/grub-* could be build and run on Cygwin. Commit r1726 
added support to build kernel and modules on Cygwin.

-- 
Regards,
Christian Franke

Re: grub-pe2elf

[Robert Millan]

On Sat, Aug 01, 2009 at 05:06:46PM +0200, Christian Franke wrote:
> Robert Millan wrote:
>> On Fri, Jul 31, 2009 at 08:27:58PM +0200, Christian Franke wrote:
>>   
>>>> First of all, I'm worried that we have discussions to decide things and they
>>>> are later forgotten or ignored.  Bean, please can you provide some explanation
>>>> on what happened?
>>>>
>>>> Also, I think this commit should be reverted (at least the part that adds
>>>> grub-pe2elf).        
>>> I would regret this. Then you could also revert all my Cygwin related 
>>>  additions (some also included in Bean's r1726) because these no 
>>> longer  make much sense unless someone fixes BFD or maintains a ELF 
>>> toolchain  for Cygwin.
>>>     
>>
>> But we supported Cygwin before grub-pe2elf.  What has changed?
>>
>>   
>
> Since ~r1584, util/grub-* could be build and run on Cygwin. Commit r1726  
> added support to build kernel and modules on Cygwin.

There has to be more to it.  I know you were using GRUB on Cygwin before
that commit.  Was this kludge being used since the beginning?  What are
the other options?

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: grub-pe2elf

[Christian Franke]

Robert Millan wrote:
> On Sat, Aug 01, 2009 at 05:06:46PM +0200, Christian Franke wrote:
> > 
> > Since ~r1584, util/grub-* could be build and run on Cygwin. Commit
> > r1726 added support to build kernel and modules on Cygwin.
> > 
> 
> There has to be more to it.  I know you were using GRUB on Cygwin
> before that commit.  Was this kludge being used since the beginning? 

I'm using GRUB on Cygwin since 'somewhat' before I announced the first
patch 2007-10-16:
http://lists.gnu.org/archive/html/grub-devel/2007-10/msg00071.html

This version used the hack in grub's ELF-loader already discussed. It
also contained some non Cygwin related fixes found during testing.

The grub2 package in the Cygwin distro from 2008-03-30 was based on this
private branch merged to 1.96+epsilon.


> What are the other options?
> 

I don't know.

Waiting until someone fixes BFD or maintains a Linux cross-compiler
package for Cygwin (which cannot be build OOTB, it requires build steps
on Linux) may last until time_t wraps around and is IMO not an option
:-)

Even the isolation of grub-pe2elf in a separate package would not help
much, because the linker script and some support in the build system
would be still needed in the grub2 tree.

-- 
Regards,
Christian Franke

Re: grub-pe2elf

[Vladimir 'phcoder' Serbinenko]

On Mon, Aug 3, 2009 at 11:33 AM, Christian
Franke<Christian.Franke@t-online.de> wrote:
> Robert Millan wrote:
>> On Sat, Aug 01, 2009 at 05:06:46PM +0200, Christian Franke wrote:
>> >
>> > Since ~r1584, util/grub-* could be build and run on Cygwin. Commit
>> > r1726 added support to build kernel and modules on Cygwin.
>> >
>>
>> There has to be more to it.  I know you were using GRUB on Cygwin
>> before that commit.  Was this kludge being used since the beginning?
>
> I'm using GRUB on Cygwin since 'somewhat' before I announced the first
> patch 2007-10-16:
> http://lists.gnu.org/archive/html/grub-devel/2007-10/msg00071.html
>
> This version used the hack in grub's ELF-loader already discussed. It
> also contained some non Cygwin related fixes found during testing.
>
> The grub2 package in the Cygwin distro from 2008-03-30 was based on this
> private branch merged to 1.96+epsilon.
>
>
>> What are the other options?
>>
>
> I don't know.
>
> Waiting until someone fixes BFD or maintains a Linux cross-compiler
> package for Cygwin (which cannot be build OOTB, it requires build steps
> on Linux) may last until time_t wraps around and is IMO not an option
> :-)
>
> Even the isolation of grub-pe2elf in a separate package would not help
> much, because the linker script and some support in the build system
> would be still needed in the grub2 tree.
>
I personally think this is a hassle for nothing: we already have a
working cygwin toolchain. While I acknowledge it's not perfect it
works. And I propose not to touch it until it gives any maintainance
or technical problem. When it does then we may consider dropping or
limiting cygwin support. Until then this discussion is a waste of time
(IMO)
> --
> Regards,
> Christian Franke
>
>
>
>
>
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel
>



-- 
Regards
Vladimir 'phcoder' Serbinenko

Personal git repository: http://repo.or.cz/w/grub2/phcoder.git

Re: grub-pe2elf

[Pavel Roskin]

On Mon, 2009-08-03 at 11:53 +0200, Vladimir 'phcoder' Serbinenko wrote:

> I personally think this is a hassle for nothing: we already have a
> working cygwin toolchain. While I acknowledge it's not perfect it
> works. And I propose not to touch it until it gives any maintainance
> or technical problem. When it does then we may consider dropping or
> limiting cygwin support. Until then this discussion is a waste of time
> (IMO)

I agree that there is no urgent need to remove grub-pe2elf (although the
issue might come up essentially).  However, it's not so much a technical
problem as an issue of trust.

Committing code against the wish of maintainers should not be permitted.
Those who do it should lose commit access.  They are welcome to
contribute, but their patches will need to be committed by others.

I haven't seen any apology or explanation in this thread.

-- 
Regards,
Pavel Roskin

Re: grub-pe2elf

[Robert Millan]

On Mon, Aug 03, 2009 at 11:33:23AM +0200, Christian Franke wrote:
> > 
> > There has to be more to it.  I know you were using GRUB on Cygwin
> > before that commit.  Was this kludge being used since the beginning? 
> 
> I'm using GRUB on Cygwin since 'somewhat' before I announced the first
> patch 2007-10-16:
> http://lists.gnu.org/archive/html/grub-devel/2007-10/msg00071.html
> 
> This version used the hack in grub's ELF-loader already discussed. It
> also contained some non Cygwin related fixes found during testing.
> 
> The grub2 package in the Cygwin distro from 2008-03-30 was based on this
> private branch merged to 1.96+epsilon.

Understood.

> > What are the other options?
> 
> I don't know.
> 
> Waiting until someone fixes BFD or maintains a Linux cross-compiler
> package for Cygwin

I don't understand what you mean about a "Linux cross-compiler":

  - Why a compiler?  AFAICT, the problem here is assembly/linkage
    rather than compiling.  binutils should suffice.

  - It's not really "Linux", just generic ELF support (i.e.
    i386-pc-elf rather than i386-pc-linux-gnu).

  - It's not exactly "cross".  I think something like
    --enable-targets=i386-pc-elf when building binutils should
    work just fine.  This produces an "ld" that understands multiple
    formats (-m switch).  Have you tried this?

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: grub-pe2elf

[Robert Millan]

On Mon, Aug 03, 2009 at 04:29:18PM -0400, Pavel Roskin wrote:
> On Mon, 2009-08-03 at 11:53 +0200, Vladimir 'phcoder' Serbinenko wrote:
> 
> > I personally think this is a hassle for nothing: we already have a
> > working cygwin toolchain. While I acknowledge it's not perfect it
> > works. And I propose not to touch it until it gives any maintainance
> > or technical problem. When it does then we may consider dropping or
> > limiting cygwin support. Until then this discussion is a waste of time
> > (IMO)
> 
> I agree that there is no urgent need to remove grub-pe2elf (although the
> issue might come up essentially).

Indeed, now that we're in this situation, we can't revert this without
causing major inconvenience on Cygwin users.  We might still do it, but
I'd rather explore the other possibilities first.

> However, it's not so much a technical
> problem as an issue of trust.
> 
> Committing code against the wish of maintainers should not be permitted.
> Those who do it should lose commit access.  They are welcome to
> contribute, but their patches will need to be committed by others.
> 
> I haven't seen any apology or explanation in this thread.

Yes, this is what worries me the most.

Anyhow, now that we've made our point, and everyone knows the drill, I hope
that this kind of things don't happen again.

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: grub-pe2elf

[Christian Franke]

Robert Millan wrote:
> I don't understand what you mean about a "Linux cross-compiler":
>
>   - Why a compiler?  AFAICT, the problem here is assembly/linkage
>     rather than compiling.  binutils should suffice.
>
>   

Unfortunately this is not true:

Some assembly language directives generated by gcc differ between PE and 
ELF. An ELF 'gas' does not accept assembly output from a PE 'gcc' and 
vice versa. The whole toolchain must support ELF.


>   - It's not really "Linux", just generic ELF support (i.e.
>     i386-pc-elf rather than i386-pc-linux-gnu).
>
>   

OK :-)


>   - It's not exactly "cross".  I think something like
>     --enable-targets=i386-pc-elf when building binutils should
>     work just fine.  This produces an "ld" that understands multiple
>     formats (-m switch).  Have you tried this?
>
>   

Cygwin binutils are build with this option:

$ ld --help
...
ld: supported targets: pe-i386 pei-i386 elf32-i386 elf32-little 
elf32-big srec symbolsrec tekhex binary ihex

But this does not help because ld refuses to build ELF output from PE 
input. So you need an ELF gas. Which requires an ELF gcc...

I already described that here:
http://lists.gnu.org/archive/html/grub-devel/2007-11/msg00233.html


-- 
Regards,
Christian Franke

Re: grub-pe2elf

[Christian Franke]

Robert Millan wrote:
> On Mon, Aug 03, 2009 at 04:29:18PM -0400, Pavel Roskin wrote:
>   
>> Committing code against the wish of maintainers should not be permitted.
>> Those who do it should lose commit access.  They are welcome to
>> contribute, but their patches will need to be committed by others.
>>
>>     

I agree...


>> I haven't seen any apology or explanation in this thread.
>>     
>
>   

... but I will not apologize simply because I did not commit the 
grub-pe2elf patch.


> Yes, this is what worries me the most.
>
>   

I'm somewhat worried that these complaints are delayed by more than a year
Commit 1726 is from 2008-07-24, start was here:
http://lists.gnu.org/archive/html/grub-devel/2008-07/msg00452.html


-- 
Regards,
Christian Franke

Re: grub-pe2elf

[Robert Millan]

On Tue, Aug 04, 2009 at 11:45:45PM +0200, Christian Franke wrote:
>
> I'm somewhat worried that these complaints are delayed by more than a year

I noticed by chance.  We were talking about something else on IRC, when
Bean mentioned that objcopy has bug when converting from PE to ELF.  Later
he pointed at the thread in which this was discussed.

We can't expect _anyone_ to be continously watching every commit and checking
the corresponding discussions related to it that happened in the few months
that precede the commit.  If someone had to do that, that person wouldn't
have time to do anything else.

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."