[Buildroot] ssh-keygen seg fault

[Buildroot] ssh-keygen seg fault

[Kyle Hayes]

I am getting a seg fault when ssh-keygen tries to run and when sshd
tries to start.  Here is my set up:

Buildroot: GIT checkout from 1/11/11
Kernel: 2.6.37
Initrd type: CPIO/gzip

machine: DMP eBox 3100 (PC-compatible x86 machine w/256MB of RAM)

I've got extlinux installed, the kernel boots fine and I get a login
prompt.  However, I see errors that ssh-keygen and sshd are seg
faulting.

This is my first attempt at using Buildroot, so I have not done any
tweaking at all to the templates.   The only changes I have made were
to the Buildroot configuration to specify the CPIO/gzip format for the
initrd and to turn off nearly everything (I want a very small build).
I am using uLibc and busybox with shared libraries.  I selected
openssh and at as new packages.  at seems to run fine.

I've configured Buildroot for i486 processors (that is close enough to
the DMP's Vortext86DX in the machine).  The tool chain appears to work
fine.

The kernel is standard or whatever Buildroot loads when you select
2.6.37.  I have turned off loadable modules and turned on devtmpfs and
selected mounting devtmpfs at boot.

When I run mdev -s manually all the right devices are created and I
can mount the CF card that the thing booted from, so I think I've got
most things working (no sign of the Ethernet device though).

I would appreciate any tips on how to debug/fix this.  Buildroot is
exactly the system I was looking for.

Best,
Kyle

[Buildroot] ssh-keygen seg fault

[Thomas Petazzoni]

Hello Kyle,

On Thu, 13 Jan 2011 13:29:42 -0800
Kyle Hayes <kyle@greenmonitor.com> wrote:

> I am getting a seg fault when ssh-keygen tries to run and when sshd
> tries to start.  Here is my set up:
> 
> Buildroot: GIT checkout from 1/11/11
> Kernel: 2.6.37
> Initrd type: CPIO/gzip
> 
> machine: DMP eBox 3100 (PC-compatible x86 machine w/256MB of RAM)
> 
> I've got extlinux installed, the kernel boots fine and I get a login
> prompt.  However, I see errors that ssh-keygen and sshd are seg
> faulting.
> 
> I would appreciate any tips on how to debug/fix this.  Buildroot is
> exactly the system I was looking for.

Hard to say since we can't reproduce without having the hardware. I
suspect that on a "normal" x86 CPU, sshd and ssh-keygen are working
fine.

So the best you could do is use gdbserver on the target and a
cross-gdb (both can be built by Buildroot) and start ssh-keygen without
gdbserver in order to find the location of the segfault in ssh-keygen.
That would really in understanding the issue.

Are you sure that the Vortex 86DX supports all instructions of i486 ?
Are you sure that the FPU is compatible ? The documentation of the
Vortex 86DX states that it has a FPU, but I don't know if it's the same
as the one in normal x86 CPUs. Maybe you could try to build Buildroot
with BR2_SOFT_FLOAT=y (config options: "Use software floating point by
default").

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] ssh-keygen seg fault

[Will Moore]

> -----Original Message-----
> From: buildroot-bounces at busybox.net [mailto:buildroot-bounces at busybox.net]
> On Behalf Of Thomas Petazzoni
> Sent: 14 January 2011 08:08
> Hello Kyle,
> 
> On Thu, 13 Jan 2011 13:29:42 -0800
> Kyle Hayes <kyle@greenmonitor.com> wrote:
> 
> > I am getting a seg fault when ssh-keygen tries to run and when sshd
> > tries to start.  Here is my set up:
> >
> > Buildroot: GIT checkout from 1/11/11
> > Kernel: 2.6.37
> > Initrd type: CPIO/gzip
> >
> > machine: DMP eBox 3100 (PC-compatible x86 machine w/256MB of RAM)
> >
> > I've got extlinux installed, the kernel boots fine and I get a login
> > prompt.  However, I see errors that ssh-keygen and sshd are seg
> > faulting.
> >
> > I would appreciate any tips on how to debug/fix this.  Buildroot is
> > exactly the system I was looking for.
> 
> Hard to say since we can't reproduce without having the hardware. I
> suspect that on a "normal" x86 CPU, sshd and ssh-keygen are working
> fine.

I am working on a *similar* platform (vortex86dx VDX6354 embedded PC104)
with buildroot for some time.  I have dropbear working without problems.

> 
> So the best you could do is use gdbserver on the target and a
> cross-gdb (both can be built by Buildroot) and start ssh-keygen without
> gdbserver in order to find the location of the segfault in ssh-keygen.
> That would really in understanding the issue.
> 
> Are you sure that the Vortex 86DX supports all instructions of i486 ?

I think it does.

> Are you sure that the FPU is compatible ?

I think it is.

Certainly I have had no problems associated with running a buildroot
generated i486 toolchain.

 The documentation of the
> Vortex 86DX states that it has a FPU, but I don't know if it's the same
> as the one in normal x86 CPUs. Maybe you could try to build Buildroot
> with BR2_SOFT_FLOAT=y (config options: "Use software floating point by
> default").

I think your problem may be the kernel config.  Can I suggest that you
download the X-Linux sources from http://www.dmp.com.tw/tech/os-xlinux/ then
use the kernel configuration for the vortex86dx as your kernel
configuration?  X-Linux is currently stuck at 2.6.29.6 and I have had
problems moving to more recent kernels (gdbserver no longer stopped at
breakpoints, kernel panics at boot).  YMMV.  As X-Linux is provided by DM&P
who make the vortex86dx I have taken the "safe" approach and stuck with
2.6.29.6 too and used their kernel configuration verbatim.  Please note: I
am unclear what is in the eBOX 3100 and so what the differences between this
and the VDX6354 are.  If the eBOX is actually a vortex86sx (no FPU) then I
have also previously used the VSX6154 which contains the vortex86sx with
buildroot using the X-Linux kernel configuration for vortex86sx.

> 
> Regards,
> 
> Thomas
> --
> Thomas Petazzoni, Free Electrons
> Kernel, drivers, real-time and embedded Linux
> development, consulting, training and support.
> http://free-electrons.com
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] ssh-keygen seg fault

[Thomas Petazzoni]

Hello,

On Fri, 14 Jan 2011 09:24:19 -0000
"Will Moore" <will.moore@beraninstruments.com> wrote:

> > Hard to say since we can't reproduce without having the hardware. I
> > suspect that on a "normal" x86 CPU, sshd and ssh-keygen are working
> > fine.
> 
> I am working on a *similar* platform (vortex86dx VDX6354 embedded PC104)
> with buildroot for some time.  I have dropbear working without problems.

Ok, good to know. We don't know if Kyle is using dropbear of the
full-fledged openssh. That might make a difference.

> > Are you sure that the Vortex 86DX supports all instructions of i486 ?
> 
> I think it does.
> 
> > Are you sure that the FPU is compatible ?
> 
> I think it is.
> 
> Certainly I have had no problems associated with running a buildroot
> generated i486 toolchain.

Ok, great.

> I think your problem may be the kernel config.  Can I suggest that you
> download the X-Linux sources from http://www.dmp.com.tw/tech/os-xlinux/ then
> use the kernel configuration for the vortex86dx as your kernel
> configuration?  X-Linux is currently stuck at 2.6.29.6 and I have had
> problems moving to more recent kernels (gdbserver no longer stopped at
> breakpoints, kernel panics at boot).  YMMV.  As X-Linux is provided by DM&P
> who make the vortex86dx I have taken the "safe" approach and stuck with
> 2.6.29.6 too and used their kernel configuration verbatim.  Please note: I
> am unclear what is in the eBOX 3100 and so what the differences between this
> and the VDX6354 are.  If the eBOX is actually a vortex86sx (no FPU) then I
> have also previously used the VSX6154 which contains the vortex86sx with
> buildroot using the X-Linux kernel configuration for vortex86sx.

You could probaby diff their 2.6.29.6 kernel with the official
2.6.29.6, and see what their changes were, port them over to a new
kernel release and even get them merged so that you can benefit from
newer kernel versions.

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] ssh-keygen seg fault

[Kyle Hayes]

Hi Thomas, Will,

I'll update in-line below.

On Fri, Jan 14, 2011 at 1:40 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Hello,
>
> On Fri, 14 Jan 2011 09:24:19 -0000
> "Will Moore" <will.moore@beraninstruments.com> wrote:
>
>> > Hard to say since we can't reproduce without having the hardware. I
>> > suspect that on a "normal" x86 CPU, sshd and ssh-keygen are working
>> > fine.
>>
>> I am working on a *similar* platform (vortex86dx VDX6354 embedded PC104)
>> with buildroot for some time. ?I have dropbear working without problems.
>
> Ok, good to know. We don't know if Kyle is using dropbear of the
> full-fledged openssh. That might make a difference.

Sorry, it is the full OpenSSH version.  I need to use OpenVPN too, so
I thought I would try to use the same tools I am currently using.

>> > Are you sure that the Vortex 86DX supports all instructions of i486 ?
>>
>> I think it does.

I am sure of it.  I can run Ubuntu 8.04 on the box with only a custom
kernel.  Everything else is completely stock from Ubuntu 8.04.  As far
as I know, Ubuntu is configured (or was) for either a 486 or a Pentium
when you run it as stock for x86.

>> > Are you sure that the FPU is compatible ?
>>
>> I think it is.
>>
>> Certainly I have had no problems associated with running a buildroot
>> generated i486 toolchain.
>
> Ok, great.

See above.  I am not sure what Ubuntu 8.04 assumes, but I'm fairly
sure it is not using software FPU.

>> I think your problem may be the kernel config. ?Can I suggest that you
>> download the X-Linux sources from http://www.dmp.com.tw/tech/os-xlinux/ then
>> use the kernel configuration for the vortex86dx as your kernel
>> configuration? ?X-Linux is currently stuck at 2.6.29.6 and I have had
>> problems moving to more recent kernels (gdbserver no longer stopped at
>> breakpoints, kernel panics at boot). ?YMMV. ?As X-Linux is provided by DM&P
>> who make the vortex86dx I have taken the "safe" approach and stuck with
>> 2.6.29.6 too and used their kernel configuration verbatim. ?Please note: I
>> am unclear what is in the eBOX 3100 and so what the differences between this
>> and the VDX6354 are. ?If the eBOX is actually a vortex86sx (no FPU) then I
>> have also previously used the VSX6154 which contains the vortex86sx with
>> buildroot using the X-Linux kernel configuration for vortex86sx.
>
> You could probaby diff their 2.6.29.6 kernel with the official
> 2.6.29.6, and see what their changes were, port them over to a new
> kernel release and even get them merged so that you can benefit from
> newer kernel versions.

I had some problems with their kernel when I started and was able to
get in contact with their developers and get a custom patch.  The
problem I was seeing was that one of the three USB ports did not have
an interrupt.  There was an interrupt sharing problem with their
kernel.  I got a patch from them and then ported it forward for both
the Vortex86DX and the Vortex86MX until about 2.6.33:

Linux test-host 2.6.33.2-vortex86mx #1 Fri Apr 16 08:46:19 PDT 2010
i586 GNU/Linux

The next time I checked the kernel, all the patches were already in
it.  I am now using 2.6.37 without any patches and it seems to work
absolutely fine.  Note that I selected _Pentium_ for the kernel and it
is working fine.  Of all the code that will exercise the CPU, the
kernel is probably the most demanding.  I may try selecting that for
the Buildroot build and see what happens.

For anyone using this SoC, I found references on the net that the EHCI
hardware is broken on the DX.  I do not know about the MX.  I have had
to enable the watchdog hardware to catch hangs related to this.  At
least I believe the hangs are related to this.  The boxes sometimes
hang when there is a USB serial device plugged in (for our proprietary
hardware) and seem to be completely stable when the USB serial device
is not plugged in.  Even though the USB serial device is not using the
EHCI controller, the machines seem more stable with the EHCI driver
completely removed from the kernel.  However, it seems to be vaguely
power-quality related.  Machines on cheap UPSes almost never (or
never) hang and machines on flaky local power hang more often.

I had to make custom kernels because I needed a specific, FTDI, USB
serial driver that is not included in the standard kernel builds for
some reason.  It is one of the more popular USB/serial chips, so I'm
not sure why the stock kernel does not include it.  I also needed to
enable the IDE and Ethernet drivers.  Both work fine in my 2.6.37
kernel.  I can post the .config file if someone wants it.

I am currently using a custom distro based on Ubuntu 8.04 with as many
packages trimmed as I can easily do.  This still makes the base CF
card image more than 400MB.  Buildroot is making a base image of about
6MB.  I intend to use extlinux' ability to concatenate multiple
initramfs files to keep our software (which revs fairly often)
separate from OpenVPN config (which doesn't change too often) separate
from the core root filesystem.

I will try dropbear and see if that works any better.  OpenSSH works
fine out-of-the box for the Ubuntu 8.04 version so I'm suspicious that
it is something about the way it is built or with the current version
of OpenSSH.

Best,
Kyle

[Buildroot] ssh-keygen seg fault

[Thomas Petazzoni]

Hi Kyle,

On Fri, 14 Jan 2011 09:00:16 -0800
Kyle Hayes <kyle@greenmonitor.com> wrote:

> > Ok, good to know. We don't know if Kyle is using dropbear of the
> > full-fledged openssh. That might make a difference.
> 
> Sorry, it is the full OpenSSH version.  I need to use OpenVPN too, so
> I thought I would try to use the same tools I am currently using.

Ok, good idea.

> >> I think it does.
> 
> I am sure of it.  I can run Ubuntu 8.04 on the box with only a custom
> kernel.  Everything else is completely stock from Ubuntu 8.04.  As far
> as I know, Ubuntu is configured (or was) for either a 486 or a Pentium
> when you run it as stock for x86.

Ok.

> > Ok, great.
> 
> See above.  I am not sure what Ubuntu 8.04 assumes, but I'm fairly
> sure it is not using software FPU.

Yes, definitely.

> I am currently using a custom distro based on Ubuntu 8.04 with as many
> packages trimmed as I can easily do.  This still makes the base CF
> card image more than 400MB.  Buildroot is making a base image of about
> 6MB.  I intend to use extlinux' ability to concatenate multiple
> initramfs files to keep our software (which revs fairly often)
> separate from OpenVPN config (which doesn't change too often) separate
> from the core root filesystem.

Sounds good. Didn't know that extlinux was able to do this kind of
things. By the way, do you build extlinux with Buildroot or separatly ?
If built with Buildroot, does it work ? Is there anything we should
improve on that side ?

> I will try dropbear and see if that works any better.  OpenSSH works
> fine out-of-the box for the Ubuntu 8.04 version so I'm suspicious that
> it is something about the way it is built or with the current version
> of OpenSSH.

Yes, it's definitely something wrong in the build process, and it'd be
great to sort it out.

Would you mind running ssh-keygen under gdbserver+cross-gdb as
suggested in my first e-mail ? Or run "ulimit -c unlimited" on your
board, get ssh-keygen to crash, and then transfer the core file to your
development workstation, and get a backstrace with the cross-gdb. That
would immediatly point us to the segfault location, which would really,
really help in understanding what the issue is.

Regards,

Thomas
-- 
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com

[Buildroot] ssh-keygen seg fault

[Kyle Hayes]

OK, now back to this...

On Fri, Jan 14, 2011 at 9:19 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Hi Kyle,
>
> On Fri, 14 Jan 2011 09:00:16 -0800
> Kyle Hayes <kyle@greenmonitor.com> wrote:
>> I am currently using a custom distro based on Ubuntu 8.04 with as many
>> packages trimmed as I can easily do. ?This still makes the base CF
>> card image more than 400MB. ?Buildroot is making a base image of about
>> 6MB. ?I intend to use extlinux' ability to concatenate multiple
>> initramfs files to keep our software (which revs fairly often)
>> separate from OpenVPN config (which doesn't change too often) separate
>> from the core root filesystem.
>
> Sounds good. Didn't know that extlinux was able to do this kind of
> things. By the way, do you build extlinux with Buildroot or separatly ?
> If built with Buildroot, does it work ? Is there anything we should
> improve on that side ?

I am using extlinux from the syslinux distribution included with
Ubuntu 10.04 (my build workstation). I have not tried to build it with
Buildroot as it is not a tool for the target, but something that I
used when setting up the CF card.  It isn't the easiest thing to
figure out (the config file for extlinux has a different _mandatory_
file extension than the config file for syslinux!), but once you get
it working, it is pretty slick.

>> I will try dropbear and see if that works any better. ?OpenSSH works
>> fine out-of-the box for the Ubuntu 8.04 version so I'm suspicious that
>> it is something about the way it is built or with the current version
>> of OpenSSH.
>
> Yes, it's definitely something wrong in the build process, and it'd be
> great to sort it out.

Dropbear appears to work.  It built RSA and DSA keys without barfing.
However, there was something else I found, see at the bottom.

> Would you mind running ssh-keygen under gdbserver+cross-gdb as
> suggested in my first e-mail ? Or run "ulimit -c unlimited" on your
> board, get ssh-keygen to crash, and then transfer the core file to your
> development workstation, and get a backstrace with the cross-gdb. That
> would immediatly point us to the segfault location, which would really,
> really help in understanding what the issue is.

I'll do this when I regress my build options back to where I had them.
 I have something working and I want to save this configuration so
that I can come back to it if I need to.

I was looking for clues in the fact that my previous custom kernel was
built for a 586, not a 486.  I checked in my old .config files and the
586 options, NOT Pentium was checked.  This support AMD K6-2
processors etc.  I changed Buildroot's configuration to select the
i386 processor family and Pentium processors as there was not an
option that matched the kernel exactly.  I tried to build the system
and then installed it.  The kernel failed to boot saying that it was
the wrong processor version?!

I checked the .config file for the kernel and found that Pentium-Pro
was selected! Huh?  I ran make menuconfig in the kernel build
directory and reset everything back the way I thought it should be and
rebuilt.  That worked.  I'm still confused as to how the kernel came
to be configured for the wrong processor.  I now have a fully working
base system.  At least as far as I have tested it.  I have not tested
logging into dropbear remotely.

I will now start undoing my changes and test to find out where things
go wrong.  I will first make the selected Buildroot CPU type = i486
and see what that does to the kernel CPU type.  I will then set it
back to Pentium to see if that is where the Pentium-Pro selection came
from.

Best,
Kyle

[Buildroot] ssh-keygen seg fault

[Michael S. Zick]

On Fri January 14 2011, Kyle Hayes wrote:
> OK, now back to this...
> 
> On Fri, Jan 14, 2011 at 9:19 AM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
> > Hi Kyle,
> >
> > On Fri, 14 Jan 2011 09:00:16 -0800
> > Kyle Hayes <kyle@greenmonitor.com> wrote:
> >> I am currently using a custom distro based on Ubuntu 8.04 with as many
> >> packages trimmed as I can easily do. ?This still makes the base CF
> >> card image more than 400MB. ?Buildroot is making a base image of about
> >> 6MB. ?I intend to use extlinux' ability to concatenate multiple
> >> initramfs files to keep our software (which revs fairly often)
> >> separate from OpenVPN config (which doesn't change too often) separate
> >> from the core root filesystem.
> >
> > Sounds good. Didn't know that extlinux was able to do this kind of
> > things. By the way, do you build extlinux with Buildroot or separatly ?
> > If built with Buildroot, does it work ? Is there anything we should
> > improve on that side ?
> 
> I am using extlinux from the syslinux distribution included with
> Ubuntu 10.04 (my build workstation). I have not tried to build it with
> Buildroot as it is not a tool for the target, but something that I
> used when setting up the CF card.  It isn't the easiest thing to
> figure out (the config file for extlinux has a different _mandatory_
> file extension than the config file for syslinux!), but once you get
> it working, it is pretty slick.
> 
> >> I will try dropbear and see if that works any better. ?OpenSSH works
> >> fine out-of-the box for the Ubuntu 8.04 version so I'm suspicious that
> >> it is something about the way it is built or with the current version
> >> of OpenSSH.
> >
> > Yes, it's definitely something wrong in the build process, and it'd be
> > great to sort it out.
> 
> Dropbear appears to work.  It built RSA and DSA keys without barfing.
> However, there was something else I found, see at the bottom.
> 
> > Would you mind running ssh-keygen under gdbserver+cross-gdb as
> > suggested in my first e-mail ? Or run "ulimit -c unlimited" on your
> > board, get ssh-keygen to crash, and then transfer the core file to your
> > development workstation, and get a backstrace with the cross-gdb. That
> > would immediatly point us to the segfault location, which would really,
> > really help in understanding what the issue is.
> 
> I'll do this when I regress my build options back to where I had them.
>  I have something working and I want to save this configuration so
> that I can come back to it if I need to.
> 
> I was looking for clues in the fact that my previous custom kernel was
> built for a 586, not a 486.  I checked in my old .config files and the
> 586 options, NOT Pentium was checked.  This support AMD K6-2
> processors etc.  I changed Buildroot's configuration to select the
> i386 processor family and Pentium processors as there was not an
> option that matched the kernel exactly.  I tried to build the system
> and then installed it.  The kernel failed to boot saying that it was
> the wrong processor version?!
> 
> I checked the .config file for the kernel and found that Pentium-Pro
> was selected! Huh?  I ran make menuconfig in the kernel build
> directory and reset everything back the way I thought it should be and
> rebuilt.  That worked.  I'm still confused as to how the kernel came
> to be configured for the wrong processor.  I now have a fully working
> base system.  At least as far as I have tested it.  I have not tested
> logging into dropbear remotely.
> 
> I will now start undoing my changes and test to find out where things
> go wrong.  I will first make the selected Buildroot CPU type = i486
> and see what that does to the kernel CPU type.  I will then set it
> back to Pentium to see if that is where the Pentium-Pro selection came
> from.
> 

It seems you are working in the area of processors that do/do not
support sse instructions.

You might check for a "-no-asm" build option on your crypto (sshd) library.
That might fix your ssh-keygen seg faults.

Note: if your using the vortex86dx SoC - the documents I found on-line
for that says it supports the full instruction set of the 80486SX (I.E:
none of the instructions that would be based in the fp co-processor).

PS: How old a kernel version?  Linux dropped support for i386 a long time ago.

Mike
> Best,
> Kyle
> _______________________________________________
> buildroot mailing list
> buildroot at busybox.net
> http://lists.busybox.net/mailman/listinfo/buildroot

[Buildroot] ssh-keygen seg fault

[Kyle Hayes]

Hi Michael,

The differences between the 486 and Pentium at the user level were not
that large.   For the OS level, there were a lot more.  This is from
memory of the time when the Pentium came out.  I remember it being a
big deal to have a dual processor P90 on my desk :-)  the PPro was a
completely different animal.  See more below.

On Fri, Jan 14, 2011 at 1:04 PM, Michael S. Zick <minimod@morethan.org> wrote:
>
> It seems you are working in the area of processors that do/do not
> support sse instructions.

Definitely not.  But, neither did the PPro which is what showed up in
the kernel configuration file.   SSE was not until the Pentium-III.
Even if the rest of the code was misconfigured for PPro, it would not
be using SSE.  It might use MMX though, so this is something to check.
 OpenSSH should be using openssl libraries I would think...

> You might check for a "-no-asm" build option on your crypto (sshd) library.
> That might fix your ssh-keygen seg faults.

Ah, good idea.  I'll see if that exists if the problem persists.

> Note: if your using the vortex86dx SoC - the documents I found on-line
> for that says it supports the full instruction set of the 80486SX (I.E:
> none of the instructions that would be based in the fp co-processor).

The Vortex86SX were that way.  The DX is actually pretty close to a
Pentium.  The MX is too (just more things on the SoC).  Support for
AMD K6-2 should be nearly identical to this.  The online documentation
is... not great :-(

> PS: How old a kernel version? ?Linux dropped support for i386 a long time ago.

I am currently using 2.6.37 which came out just a few weeks ago.  It
works fine.  The kernel will exercise more of the differences between
the processors than user code.  I would not be surprised at all to
find that the user code for the latest i7 Core processor is basically
identical (SSE excepted) as the code for the 486 except for
compile-time instruction scheduling.  There are a few new instructions
outside of the SSE set, but not very many.

Once I get a working kernel and can get a core dump, I'll see what I
find.  Maybe strace will help?

Best,
Kyle

[Buildroot] ssh-keygen seg fault

[Kyle Hayes]

I know, bad form to post a response to yourself.  I don't think I hit
a bug in Buildroot with the Linux version changing.  It was a user bug
:-)


On Fri, Jan 14, 2011 at 11:29 AM, Kyle Hayes <kyle@greenmonitor.com> wrote:
> OK, now back to this...
> [snip]
>
> I was looking for clues in the fact that my previous custom kernel was
> built for a 586, not a 486. ?I checked in my old .config files and the
> 586 options, NOT Pentium was checked. ?This support AMD K6-2
> processors etc. ?I changed Buildroot's configuration to select the
> i386 processor family and Pentium processors as there was not an
> option that matched the kernel exactly. ?I tried to build the system
> and then installed it. ?The kernel failed to boot saying that it was
> the wrong processor version?!
>
> I checked the .config file for the kernel and found that Pentium-Pro
> was selected! Huh? ?I ran make menuconfig in the kernel build
> directory and reset everything back the way I thought it should be and
> rebuilt. ?That worked. ?I'm still confused as to how the kernel came
> to be configured for the wrong processor. ?I now have a fully working
> base system. ?At least as far as I have tested it. ?I have not tested
> logging into dropbear remotely.
>
> I will now start undoing my changes and test to find out where things
> go wrong. ?I will first make the selected Buildroot CPU type = i486
> and see what that does to the kernel CPU type. ?I will then set it
> back to Pentium to see if that is where the Pentium-Pro selection came
> from.

I did a make clean between changes and that wipes the kernel config
file.  Luckily I saved it...

Off to build another kernel with the right CPU and settings.

Best,
Kyle



> Best,
> Kyle
>

[Buildroot] ssh-keygen seg fault

[Kyle Hayes]

Hi Thomas,

See below...

On Fri, Jan 14, 2011 at 9:19 AM, Thomas Petazzoni
<thomas.petazzoni@free-electrons.com> wrote:
> Yes, it's definitely something wrong in the build process, and it'd be
> great to sort it out.
>
> Would you mind running ssh-keygen under gdbserver+cross-gdb as
> suggested in my first e-mail ? Or run "ulimit -c unlimited" on your
> board, get ssh-keygen to crash, and then transfer the core file to your
> development workstation, and get a backstrace with the cross-gdb. That
> would immediatly point us to the segfault location, which would really,
> really help in understanding what the issue is.

I got a core file.  I have a target GDB for use with host programs.  I
turned off stripping.  But...

sudo /home/kyle/buildroot/buildroot/output/toolchain/gdbhost-7.1/gdb/gdb
/home/kyle/buildroot/buildroot/output/target/usr/bin/ssh-keygen
/media/HC/core
GNU gdb (GDB) 7.1
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=i686-pc-linux-gnu
--target=i486-unknown-linux-uclibc".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from
/home/kyle/buildroot/buildroot/output/target/usr/bin/ssh-keygen...(no
debugging symbols found)...done.
[New Thread 549]

warning: .dynamic section for "/lib/libgcc_s.so.1" is not at the
expected address (wrong library or version mismatch?)
Reading symbols from
/home/kyle/buildroot/buildroot/output/target/usr/lib/libcrypto.so.1.0.0...done.
Loaded symbols for
/home/kyle/buildroot/buildroot/output/target/usr/lib/libcrypto.so.1.0.0
Reading symbols from
/home/kyle/buildroot/buildroot/output/target/lib/libdl.so.0...(no
debugging symbols found)...done.
Loaded symbols for /home/kyle/buildroot/buildroot/output/target/lib/libdl.so.0
Reading symbols from
/home/kyle/buildroot/buildroot/output/target/lib/libutil.so.0...(no
debugging symbols found)...done.
Loaded symbols for /home/kyle/buildroot/buildroot/output/target/lib/libutil.so.0
Reading symbols from
/home/kyle/buildroot/buildroot/output/target/usr/lib/libz.so.1...done.
Loaded symbols for
/home/kyle/buildroot/buildroot/output/target/usr/lib/libz.so.1
Reading symbols from
/home/kyle/buildroot/buildroot/output/target/lib/libcrypt.so.0...(no
debugging symbols found)...done.
Loaded symbols for
/home/kyle/buildroot/buildroot/output/target/lib/libcrypt.so.0
Reading symbols from /lib/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libgcc_s.so.1
Reading symbols from
/home/kyle/buildroot/buildroot/output/target/lib/libc.so.0...(no
debugging symbols found)...done.
Loaded symbols for /home/kyle/buildroot/buildroot/output/target/lib/libc.so.0
Reading symbols from
/home/kyle/buildroot/buildroot/output/target/lib/ld-uClibc.so.0...(no
debugging symbols found)...done.
Loaded symbols for
/home/kyle/buildroot/buildroot/output/target/lib/ld-uClibc.so.0
Core was generated by `ssh-keygen -t rsa'.
Program terminated with signal 11, Segmentation fault.
#0  0x0804da1a in main ()
(gdb) backtrace
#0  0x0804da1a in main ()
(gdb) up
Initial frame selected; you cannot go up.

Uh, so somewhere I managed to build everything without any debugging
symbols.  I'll try again.  Sigh.  The compile/edit/retry cycle on this
is quite long.  How "smart" is Builtroot when I change configuration?
Will it just rebuild the things I need to rebuild?  I've been doing a
"make clean" before every attempt.

I'm trying!

Best,
Kyle

[Buildroot] ssh-keygen seg fault

[Kyle Hayes]

Sigh.  Same sort of problems.  I'm clearly missing something.  At
least this time I got a file name and a line number.

sudo /home/kyle/buildroot/buildroot/output/toolchain/gdbhost-7.1/gdb/gdb
/home/kyle/buildroot/buildroot/output/target/usr/bin/ssh-keygen
/media/HC/core
GNU gdb (GDB) 7.1
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=i686-pc-linux-gnu
--target=i486-unknown-linux-uclibc".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from
/home/kyle/buildroot/buildroot/output/target/usr/bin/ssh-keygen...done.
[New Thread 550]
Error while mapping shared library sections:
/usr/lib/libcrypto.so.1.0.0: No such file or directory.
Error while mapping shared library sections:
/lib/libdl.so.0: No such file or directory.
Error while mapping shared library sections:
/lib/libutil.so.0: No such file or directory.
Error while mapping shared library sections:
/usr/lib/libz.so.1: No such file or directory.
Error while mapping shared library sections:
/lib/libcrypt.so.0: No such file or directory.

warning: .dynamic section for "/lib/libgcc_s.so.1" is not at the
expected address (wrong library or version mismatch?)
Error while mapping shared library sections:
/lib/libc.so.0: No such file or directory.
Error while mapping shared library sections:
/lib/ld-uClibc.so.0: No such file or directory.
Symbol file not found for /usr/lib/libcrypto.so.1.0.0
Symbol file not found for /lib/libdl.so.0
Symbol file not found for /lib/libutil.so.0
Symbol file not found for /usr/lib/libz.so.1
Symbol file not found for /lib/libcrypt.so.0
Reading symbols from /lib/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib/libgcc_s.so.1
Symbol file not found for /lib/libc.so.0
Symbol file not found for /lib/ld-uClibc.so.0

warning: Unable to find dynamic linker breakpoint function.
GDB will be unable to debug shared library initializers
and track explicitly loaded dynamic code.
Core was generated by `ssh-keygen -t rsa'.
Program terminated with signal 11, Segmentation fault.
#0  main (argc=-1079009626, argv=0xbfaf9eb1) at ssh-keygen.c:1762
1762	{
(gdb)


This is useless information because 1762 is the first line of main() :-(

I'm now rebuilding with a version of gdb for the target...

Huh, did that and got the same thing.

How do I do this?  I've never tried to debug anything like this.  I'm
clearly doing something wrong with the debugging.  It only gives me
the initial stack frame, so I think that I'm missing something
important somewhere.

I've built everything with debugging symbols on.  I've build with
strip turned off.  I've built gdb for both host and target.  The above
is about all I can get.

Can someone explain what to do in short sentences using very small
words? I need to be able to get more of a stack trace than this.  I
tried running ssh-keygen inside gdb on the target system and this is
all I get.  So, it is as if the debugging info is still not there.
I'm doing yet another make clean and remake and I hope that will work.

Best,
Kyle



On Mon, Jan 17, 2011 at 2:23 PM, Kyle Hayes <kyle@greenmonitor.com> wrote:
> Hi Thomas,
>
> See below...
>
> On Fri, Jan 14, 2011 at 9:19 AM, Thomas Petazzoni
> <thomas.petazzoni@free-electrons.com> wrote:
>> Yes, it's definitely something wrong in the build process, and it'd be
>> great to sort it out.
>>
>> Would you mind running ssh-keygen under gdbserver+cross-gdb as
>> suggested in my first e-mail ? Or run "ulimit -c unlimited" on your
>> board, get ssh-keygen to crash, and then transfer the core file to your
>> development workstation, and get a backstrace with the cross-gdb. That
>> would immediatly point us to the segfault location, which would really,
>> really help in understanding what the issue is.
>
> I got a core file. ?I have a target GDB for use with host programs. ?I
> turned off stripping. ?But...
>
> sudo /home/kyle/buildroot/buildroot/output/toolchain/gdbhost-7.1/gdb/gdb
> /home/kyle/buildroot/buildroot/output/target/usr/bin/ssh-keygen
> /media/HC/core
> GNU gdb (GDB) 7.1
> Copyright (C) 2010 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. ?Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "--host=i686-pc-linux-gnu
> --target=i486-unknown-linux-uclibc".
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>...
> Reading symbols from
> /home/kyle/buildroot/buildroot/output/target/usr/bin/ssh-keygen...(no
> debugging symbols found)...done.
> [New Thread 549]
>
> warning: .dynamic section for "/lib/libgcc_s.so.1" is not at the
> expected address (wrong library or version mismatch?)
> Reading symbols from
> /home/kyle/buildroot/buildroot/output/target/usr/lib/libcrypto.so.1.0.0...done.
> Loaded symbols for
> /home/kyle/buildroot/buildroot/output/target/usr/lib/libcrypto.so.1.0.0
> Reading symbols from
> /home/kyle/buildroot/buildroot/output/target/lib/libdl.so.0...(no
> debugging symbols found)...done.
> Loaded symbols for /home/kyle/buildroot/buildroot/output/target/lib/libdl.so.0
> Reading symbols from
> /home/kyle/buildroot/buildroot/output/target/lib/libutil.so.0...(no
> debugging symbols found)...done.
> Loaded symbols for /home/kyle/buildroot/buildroot/output/target/lib/libutil.so.0
> Reading symbols from
> /home/kyle/buildroot/buildroot/output/target/usr/lib/libz.so.1...done.
> Loaded symbols for
> /home/kyle/buildroot/buildroot/output/target/usr/lib/libz.so.1
> Reading symbols from
> /home/kyle/buildroot/buildroot/output/target/lib/libcrypt.so.0...(no
> debugging symbols found)...done.
> Loaded symbols for
> /home/kyle/buildroot/buildroot/output/target/lib/libcrypt.so.0
> Reading symbols from /lib/libgcc_s.so.1...(no debugging symbols found)...done.
> Loaded symbols for /lib/libgcc_s.so.1
> Reading symbols from
> /home/kyle/buildroot/buildroot/output/target/lib/libc.so.0...(no
> debugging symbols found)...done.
> Loaded symbols for /home/kyle/buildroot/buildroot/output/target/lib/libc.so.0
> Reading symbols from
> /home/kyle/buildroot/buildroot/output/target/lib/ld-uClibc.so.0...(no
> debugging symbols found)...done.
> Loaded symbols for
> /home/kyle/buildroot/buildroot/output/target/lib/ld-uClibc.so.0
> Core was generated by `ssh-keygen -t rsa'.
> Program terminated with signal 11, Segmentation fault.
> #0 ?0x0804da1a in main ()
> (gdb) backtrace
> #0 ?0x0804da1a in main ()
> (gdb) up
> Initial frame selected; you cannot go up.
>
> Uh, so somewhere I managed to build everything without any debugging
> symbols. ?I'll try again. ?Sigh. ?The compile/edit/retry cycle on this
> is quite long. ?How "smart" is Builtroot when I change configuration?
> Will it just rebuild the things I need to rebuild? ?I've been doing a
> "make clean" before every attempt.
>
> I'm trying!
>
> Best,
> Kyle
>

[Buildroot] ssh-keygen seg fault

[Peter Korsgaard]

>>>>> "Kyle" == Kyle Hayes <kyle@greenmonitor.com> writes:

Hi,

 Kyle> I got a core file.  I have a target GDB for use with host programs.  I
 Kyle> turned off stripping.  But...

 Kyle> sudo /home/kyle/buildroot/buildroot/output/toolchain/gdbhost-7.1/gdb/gdb
 Kyle> /home/kyle/buildroot/buildroot/output/target/usr/bin/ssh-keygen
 Kyle> /media/HC/core

You don't need sudo.

You can do this even with stripping, if you point gdb at the unstripped
version in output/build/...

To get gdb to find the correct shared libs, you'll need to do
set solib-absolute-prefix path/to/output/staging

 Kyle> Uh, so somewhere I managed to build everything without any debugging
 Kyle> symbols.  I'll try again.  Sigh.  The compile/edit/retry cycle on this
 Kyle> is quite long.  How "smart" is Builtroot when I change configuration?
 Kyle> Will it just rebuild the things I need to rebuild?  I've been doing a
 Kyle> "make clean" before every attempt.

Buildroot handles small changes like adding a package quite well, but
for global changes like toolchain options or strip/no-strip you'll need
to make clean; make

-- 
Bye, Peter Korsgaard

[Buildroot] ssh-keygen seg fault

[Kyle Hayes]

Thanks for the tips.  See below:

On Wed, Jan 19, 2011 at 2:29 AM, Peter Korsgaard <jacmet@uclibc.org> wrote:
>>>>>> "Kyle" == Kyle Hayes <kyle@greenmonitor.com> writes:
>
> Hi,
>
> ?Kyle> I got a core file. ?I have a target GDB for use with host programs. ?I
> ?Kyle> turned off stripping. ?But...
>
> ?Kyle> sudo /home/kyle/buildroot/buildroot/output/toolchain/gdbhost-7.1/gdb/gdb
> ?Kyle> /home/kyle/buildroot/buildroot/output/target/usr/bin/ssh-keygen
> ?Kyle> /media/HC/core
>
> You don't need sudo.

I needed sudo because the core file was not readable by a normal user.

> You can do this even with stripping, if you point gdb at the unstripped
> version in output/build/...
>
> To get gdb to find the correct shared libs, you'll need to do
> set solib-absolute-prefix path/to/output/staging

Ah, OK.

> ?Kyle> Uh, so somewhere I managed to build everything without any debugging
> ?Kyle> symbols. ?I'll try again. ?Sigh. ?The compile/edit/retry cycle on this
> ?Kyle> is quite long. ?How "smart" is Builtroot when I change configuration?
> ?Kyle> Will it just rebuild the things I need to rebuild? ?I've been doing a
> ?Kyle> "make clean" before every attempt.
>
> Buildroot handles small changes like adding a package quite well, but
> for global changes like toolchain options or strip/no-strip you'll need
> to make clean; make

I think something I'm doing is still incorrect:

kyle at hp-workstation:~/buildroot/buildroot$
./output/toolchain/gdbhost-7.1/gdb/gdb
GNU gdb (GDB) 7.1
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=i686-pc-linux-gnu
--target=i486-unknown-linux-uclibc".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
(gdb) set solib-absolute-prefix
/home/kyle/buildroot/buildroot/output/staging/lib/
(gdb) file output/target/usr/bin/ssh-keygen
Reading symbols from
/home/kyle/buildroot/buildroot/output/target/usr/bin/ssh-keygen...done.
(gdb) target remote 10.206.1.102:4567
Remote debugging using 10.206.1.102:4567
warning: Unable to find dynamic linker breakpoint function.
GDB will be unable to debug shared library initializers
and track explicitly loaded dynamic code.
0xb787d870 in ?? ()
(gdb) continue
Continuing.
Error while mapping shared library sections:
/usr/lib/libcrypto.so.1.0.0: No such file or directory.
Error while mapping shared library sections:
/lib/libdl.so.0: No such file or directory.
Error while mapping shared library sections:
/lib/libutil.so.0: No such file or directory.
Error while mapping shared library sections:
/usr/lib/libz.so.1: No such file or directory.
Error while mapping shared library sections:
/lib/libcrypt.so.0: No such file or directory.
Error while mapping shared library sections:
/lib/libgcc_s.so.1: No such file or directory.
Error while mapping shared library sections:
/lib/libc.so.0: No such file or directory.
Error while mapping shared library sections:
/lib/ld-uClibc.so.0: No such file or directory.

Program received signal SIGSEGV, Segmentation fault.
main (argc=-1078251855, argv=0x0) at ssh-keygen.c:1762
1762	{
(gdb)


If I run ssh-keygen on the command line on the target machine, I get a
seg fault and I get "ssh-keygen used greatest stack depth: 6460 bytes
left"  I have stack smashing protection turned on.  Is this an
infinite recursion problem or just a normal output from the system
about this? Other commands from busybox do not print out this error
about the stack.

I tried adding 'no-asm' to the list of options for openssl and
rebuilding that (just a make at the top level does not rebuild
openssl, is that correct behavior?).  I get the same behavior.
However, I am not sure that I'm getting the right version of openssl.
 If I just change the Makefile for openssl to add the option, then a
remake with just "make" at the top level does NOT rebuild openssl.
How do I force this?  If I do a make clean, all the output/target
stuff is gone and I have nothing to change.

I tried with dropbear and it works.  I will probably go with that for now.

Best,
Kyle

[Buildroot] ssh-keygen seg fault

[Peter Korsgaard]

>>>>> "Kyle" == Kyle Hayes <kyle@greenmonitor.com> writes:

Hi,

 >> Buildroot handles small changes like adding a package quite well, but
 >> for global changes like toolchain options or strip/no-strip you'll need
 >> to make clean; make

 Kyle> I think something I'm doing is still incorrect:

 Kyle> kyle at hp-workstation:~/buildroot/buildroot$
 Kyle> ./output/toolchain/gdbhost-7.1/gdb/gdb

Normally you would use output/staging/usr/bin/$PREFIX-gdb, but OK.

 Kyle> GNU gdb (GDB) 7.1
 Kyle> Copyright (C) 2010 Free Software Foundation, Inc.
 Kyle> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
 Kyle> This is free software: you are free to change and redistribute it.
 Kyle> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
 Kyle> and "show warranty" for details.
 Kyle> This GDB was configured as "--host=i686-pc-linux-gnu
 Kyle> --target=i486-unknown-linux-uclibc".
 Kyle> For bug reporting instructions, please see:
 Kyle> <http://www.gnu.org/software/gdb/bugs/>.
 Kyle> (gdb) set solib-absolute-prefix
 Kyle> /home/kyle/buildroot/buildroot/output/staging/lib/

Don't add lib at the end. solib-absolute-prefix is a PREFIX, not the
location of the library files.

 Kyle> (gdb) file output/target/usr/bin/ssh-keygen
 Kyle> Reading symbols from
 Kyle> /home/kyle/buildroot/buildroot/output/target/usr/bin/ssh-keygen...done.

Again, please point gdb at the unstripped version in
output/build/openssh*/ssh-keygen instead of the (potentially) stripped
version in target.

 Kyle> If I run ssh-keygen on the command line on the target machine, I get a
 Kyle> seg fault and I get "ssh-keygen used greatest stack depth: 6460 bytes
 Kyle> left"  I have stack smashing protection turned on.  Is this an
 Kyle> infinite recursion problem or just a normal output from the system
 Kyle> about this? Other commands from busybox do not print out this error
 Kyle> about the stack.

Sorry. don't know. I've never uses the SSP stuff.

 Kyle> I tried adding 'no-asm' to the list of options for openssl and
 Kyle> rebuilding that (just a make at the top level does not rebuild
 Kyle> openssl, is that correct behavior?).  I get the same behavior.

Rebuild is controlled by the .stamp_* files inside build/openssl*/, so
it only gets rebuilt if you remove those (or remove the entire openssl
dir). Notice that dependent packages don't get rebuilt automatically.

-- 
Bye, Peter Korsgaard

[Buildroot] ssh-keygen seg fault

[Kyle Hayes]

Hi Peter,

Thanks for the tips, I'll try this and report back!

Note that I have stripping turned off, so the binaries should be the
same.  I'll check though.

Best,
Kyle

On Wed, Jan 19, 2011 at 11:14 AM, Peter Korsgaard <jacmet@uclibc.org> wrote:
>>>>>> "Kyle" == Kyle Hayes <kyle@greenmonitor.com> writes:
>
> Hi,
>
> ?>> Buildroot handles small changes like adding a package quite well, but
> ?>> for global changes like toolchain options or strip/no-strip you'll need
> ?>> to make clean; make
>
> ?Kyle> I think something I'm doing is still incorrect:
>
> ?Kyle> kyle at hp-workstation:~/buildroot/buildroot$
> ?Kyle> ./output/toolchain/gdbhost-7.1/gdb/gdb
>
> Normally you would use output/staging/usr/bin/$PREFIX-gdb, but OK.
>
> ?Kyle> GNU gdb (GDB) 7.1
> ?Kyle> Copyright (C) 2010 Free Software Foundation, Inc.
> ?Kyle> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
> ?Kyle> This is free software: you are free to change and redistribute it.
> ?Kyle> There is NO WARRANTY, to the extent permitted by law. ?Type "show copying"
> ?Kyle> and "show warranty" for details.
> ?Kyle> This GDB was configured as "--host=i686-pc-linux-gnu
> ?Kyle> --target=i486-unknown-linux-uclibc".
> ?Kyle> For bug reporting instructions, please see:
> ?Kyle> <http://www.gnu.org/software/gdb/bugs/>.
> ?Kyle> (gdb) set solib-absolute-prefix
> ?Kyle> /home/kyle/buildroot/buildroot/output/staging/lib/
>
> Don't add lib at the end. solib-absolute-prefix is a PREFIX, not the
> location of the library files.
>
> ?Kyle> (gdb) file output/target/usr/bin/ssh-keygen
> ?Kyle> Reading symbols from
> ?Kyle> /home/kyle/buildroot/buildroot/output/target/usr/bin/ssh-keygen...done.
>
> Again, please point gdb at the unstripped version in
> output/build/openssh*/ssh-keygen instead of the (potentially) stripped
> version in target.
>
> ?Kyle> If I run ssh-keygen on the command line on the target machine, I get a
> ?Kyle> seg fault and I get "ssh-keygen used greatest stack depth: 6460 bytes
> ?Kyle> left" ?I have stack smashing protection turned on. ?Is this an
> ?Kyle> infinite recursion problem or just a normal output from the system
> ?Kyle> about this? Other commands from busybox do not print out this error
> ?Kyle> about the stack.
>
> Sorry. don't know. I've never uses the SSP stuff.
>
> ?Kyle> I tried adding 'no-asm' to the list of options for openssl and
> ?Kyle> rebuilding that (just a make at the top level does not rebuild
> ?Kyle> openssl, is that correct behavior?). ?I get the same behavior.
>
> Rebuild is controlled by the .stamp_* files inside build/openssl*/, so
> it only gets rebuilt if you remove those (or remove the entire openssl
> dir). Notice that dependent packages don't get rebuilt automatically.
>
> --
> Bye, Peter Korsgaard
>

[Buildroot] ssh-keygen seg fault

[Kyle Hayes]

Hi Peter, Thomas,

See below.

On Wed, Jan 19, 2011 at 11:14 AM, Peter Korsgaard <jacmet@uclibc.org> wrote:
>>>>>> "Kyle" == Kyle Hayes <kyle@greenmonitor.com> writes:
>
> Hi,
>
>  >> Buildroot handles small changes like adding a package quite well, but
>  >> for global changes like toolchain options or strip/no-strip you'll need
>  >> to make clean; make
>
>  Kyle> I think something I'm doing is still incorrect:
>
>  Kyle> kyle at hp-workstation:~/buildroot/buildroot$
>  Kyle> ./output/toolchain/gdbhost-7.1/gdb/gdb
>
> Normally you would use output/staging/usr/bin/$PREFIX-gdb, but OK.
>
>  Kyle> GNU gdb (GDB) 7.1
>  Kyle> Copyright (C) 2010 Free Software Foundation, Inc.
>  Kyle> License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
>  Kyle> This is free software: you are free to change and redistribute it.
>  Kyle> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
>  Kyle> and "show warranty" for details.
>  Kyle> This GDB was configured as "--host=i686-pc-linux-gnu
>  Kyle> --target=i486-unknown-linux-uclibc".
>  Kyle> For bug reporting instructions, please see:
>  Kyle> <http://www.gnu.org/software/gdb/bugs/>.
>  Kyle> (gdb) set solib-absolute-prefix
>  Kyle> /home/kyle/buildroot/buildroot/output/staging/lib/
>
> Don't add lib at the end. solib-absolute-prefix is a PREFIX, not the
> location of the library files.
>
>  Kyle> (gdb) file output/target/usr/bin/ssh-keygen
>  Kyle> Reading symbols from
>  Kyle> /home/kyle/buildroot/buildroot/output/target/usr/bin/ssh-keygen...done.
>
> Again, please point gdb at the unstripped version in
> output/build/openssh*/ssh-keygen instead of the (potentially) stripped
> version in target.
>
>  Kyle> If I run ssh-keygen on the command line on the target machine, I get a
>  Kyle> seg fault and I get "ssh-keygen used greatest stack depth: 6460 bytes
>  Kyle> left"  I have stack smashing protection turned on.  Is this an
>  Kyle> infinite recursion problem or just a normal output from the system
>  Kyle> about this? Other commands from busybox do not print out this error
>  Kyle> about the stack.
>
> Sorry. don't know. I've never uses the SSP stuff.
>
>  Kyle> I tried adding 'no-asm' to the list of options for openssl and
>  Kyle> rebuilding that (just a make at the top level does not rebuild
>  Kyle> openssl, is that correct behavior?).  I get the same behavior.
>
> Rebuild is controlled by the .stamp_* files inside build/openssl*/, so
> it only gets rebuilt if you remove those (or remove the entire openssl
> dir). Notice that dependent packages don't get rebuilt automatically.
>

Something I'm doing is still incorrect I think:

(gdb) set solib-absolute-prefix /home/kyle/buildroot/buildroot/output/staging/
(gdb) file output/build/openssh-5.6p1/ssh-keygen
Reading symbols from
/home/kyle/buildroot/buildroot/output/build/openssh-5.6p1/ssh-keygen...done.
(gdb) target remote 10.206.1.102:4567
Remote debugging using 10.206.1.102:4567
Reading symbols from
/home/kyle/buildroot/buildroot/output/staging/lib/ld-uClibc.so.0...(no
debugging symbols found)...done.
Loaded symbols for
/home/kyle/buildroot/buildroot/output/staging/lib/ld-uClibc.so.0
0xb7792870 in _start () from
/home/kyle/buildroot/buildroot/output/staging/lib/ld-uClibc.so.0
(gdb) run
The "remote" target does not support "run".  Try "help target" or "continue".
(gdb) set args -t rsa
(gdb) continue
Continuing.

Program received signal SIGSEGV, Segmentation fault.
main (argc=-1081803087, argv=0x0) at ssh-keygen.c:1762
1762	{
(gdb)

I do not understand why ld-uClibc has no debugging symbols.  I have
enabled debug in the Buildroot config and done (several) make clean;
make.  This time has fewer errors.  I will try to make openssh
independently with no-asm specified.

The failure point is always the same: the first line of main().  Is
this perhaps a kernel problem?

Best,
Kyle