* Hidden symbol when debugging hypervisor
@ 2014-04-30 9:02 Dietmar Hahn
2014-04-30 9:26 ` George Dunlap
0 siblings, 1 reply; 15+ messages in thread
From: Dietmar Hahn @ 2014-04-30 9:02 UTC (permalink / raw)
To: xen-devel@lists.xen.org; +Cc: George Dunlap
Hi,
while debugging a vmcore with the crash tool I stumpled over a little problem.
I wanted to look at the "struct csched_private" of credit scheduler and got
the contents of the "struct csched_private" of credit2.
The debug informations of the hypervisor contain 2 entries
<1><b185e>: Abbrev Number: 8 (DW_TAG_structure_type)
<b185f> DW_AT_name : (indirect string, offset: 0x9d8d): csched_private
and
<1><c0677>: Abbrev Number: 25 (DW_TAG_structure_type)
<c0678> DW_AT_name : (indirect string, offset: 0x9d8d): csched_private
The first is credit and the second credit2. It seems in the crash command the
second entry wins :-(.
Maybe crash has the possibility somewhere to get access to the second structure
(I couldn't find it) but for simplicity it would be better to have different names
I think.
Are there any reasons not to rename
struct csched_private -> struct c2sched_private
or whatever?
Thanks.
Dietmar.
--
Company details: http://ts.fujitsu.com/imprint.html
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hidden symbol when debugging hypervisor
2014-04-30 9:02 Hidden symbol when debugging hypervisor Dietmar Hahn
@ 2014-04-30 9:26 ` George Dunlap
2014-04-30 9:39 ` Juergen Gross
0 siblings, 1 reply; 15+ messages in thread
From: George Dunlap @ 2014-04-30 9:26 UTC (permalink / raw)
To: Dietmar Hahn, xen-devel@lists.xen.org
On 04/30/2014 10:02 AM, Dietmar Hahn wrote:
> Hi,
>
> while debugging a vmcore with the crash tool I stumpled over a little problem.
> I wanted to look at the "struct csched_private" of credit scheduler and got
> the contents of the "struct csched_private" of credit2.
> The debug informations of the hypervisor contain 2 entries
>
> <1><b185e>: Abbrev Number: 8 (DW_TAG_structure_type)
> <b185f> DW_AT_name : (indirect string, offset: 0x9d8d): csched_private
> and
> <1><c0677>: Abbrev Number: 25 (DW_TAG_structure_type)
> <c0678> DW_AT_name : (indirect string, offset: 0x9d8d): csched_private
>
> The first is credit and the second credit2. It seems in the crash command the
> second entry wins :-(.
>
> Maybe crash has the possibility somewhere to get access to the second structure
> (I couldn't find it) but for simplicity it would be better to have different names
> I think.
> Are there any reasons not to rename
> struct csched_private -> struct c2sched_private
> or whatever?
No reasons at all -- the naming is an artifact of development. Feel
free to send a patch renaming it.
-George
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hidden symbol when debugging hypervisor
2014-04-30 9:26 ` George Dunlap
@ 2014-04-30 9:39 ` Juergen Gross
2014-04-30 10:07 ` George Dunlap
2014-04-30 10:10 ` Jan Beulich
0 siblings, 2 replies; 15+ messages in thread
From: Juergen Gross @ 2014-04-30 9:39 UTC (permalink / raw)
To: George Dunlap, Dietmar Hahn, xen-devel@lists.xen.org
On 30.04.2014 11:26, George Dunlap wrote:
> On 04/30/2014 10:02 AM, Dietmar Hahn wrote:
>> Hi,
>>
>> while debugging a vmcore with the crash tool I stumpled over a little problem.
>> I wanted to look at the "struct csched_private" of credit scheduler and got
>> the contents of the "struct csched_private" of credit2.
>> The debug informations of the hypervisor contain 2 entries
>> <1><b185e>: Abbrev Number: 8 (DW_TAG_structure_type)
>> <b185f> DW_AT_name : (indirect string, offset: 0x9d8d):
>> csched_private
>> and
>> <1><c0677>: Abbrev Number: 25 (DW_TAG_structure_type)
>> <c0678> DW_AT_name : (indirect string, offset: 0x9d8d):
>> csched_private
>>
>> The first is credit and the second credit2. It seems in the crash command the
>> second entry wins :-(.
>>
>> Maybe crash has the possibility somewhere to get access to the second structure
>> (I couldn't find it) but for simplicity it would be better to have different
>> names
>> I think.
>> Are there any reasons not to rename
>> struct csched_private -> struct c2sched_private
>> or whatever?
>
> No reasons at all -- the naming is an artifact of development. Feel free to
> send a patch renaming it.
I tried it once: http://lists.xen.org/archives/html/xen-devel/2013-02/msg02255.html
Jan didn't like it.
Jürgen
--
Juergen Gross Principal Developer Operating Systems
PSO PM&D ES&S SWE OS6 Telephone: +49 (0) 89 62060 2932
Fujitsu e-mail: juergen.gross@ts.fujitsu.com
Mies-van-der-Rohe-Str. 8 Internet: ts.fujitsu.com
D-80807 Muenchen Company details: ts.fujitsu.com/imprint.html
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hidden symbol when debugging hypervisor
2014-04-30 9:39 ` Juergen Gross
@ 2014-04-30 10:07 ` George Dunlap
2014-04-30 10:21 ` Jan Beulich
2014-04-30 10:10 ` Jan Beulich
1 sibling, 1 reply; 15+ messages in thread
From: George Dunlap @ 2014-04-30 10:07 UTC (permalink / raw)
To: Juergen Gross, Dietmar Hahn, xen-devel@lists.xen.org, Jan Beulich
On 04/30/2014 10:39 AM, Juergen Gross wrote:
> On 30.04.2014 11:26, George Dunlap wrote:
>> On 04/30/2014 10:02 AM, Dietmar Hahn wrote:
>>> Hi,
>>>
>>> while debugging a vmcore with the crash tool I stumpled over a
>>> little problem.
>>> I wanted to look at the "struct csched_private" of credit scheduler
>>> and got
>>> the contents of the "struct csched_private" of credit2.
>>> The debug informations of the hypervisor contain 2 entries
>>> <1><b185e>: Abbrev Number: 8 (DW_TAG_structure_type)
>>> <b185f> DW_AT_name : (indirect string, offset: 0x9d8d):
>>> csched_private
>>> and
>>> <1><c0677>: Abbrev Number: 25 (DW_TAG_structure_type)
>>> <c0678> DW_AT_name : (indirect string, offset: 0x9d8d):
>>> csched_private
>>>
>>> The first is credit and the second credit2. It seems in the crash
>>> command the
>>> second entry wins :-(.
>>>
>>> Maybe crash has the possibility somewhere to get access to the
>>> second structure
>>> (I couldn't find it) but for simplicity it would be better to have
>>> different
>>> names
>>> I think.
>>> Are there any reasons not to rename
>>> struct csched_private -> struct c2sched_private
>>> or whatever?
>>
>> No reasons at all -- the naming is an artifact of development. Feel
>> free to
>> send a patch renaming it.
>
> I tried it once:
> http://lists.xen.org/archives/html/xen-devel/2013-02/msg02255.html
>
> Jan didn't like it.
It looks like he was only thinking about backtraces, not about cscope or
about debugging core dumps. Jan, do you have another solution for
those, or shall we go ahead and change the names?
-George
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hidden symbol when debugging hypervisor
2014-04-30 9:39 ` Juergen Gross
2014-04-30 10:07 ` George Dunlap
@ 2014-04-30 10:10 ` Jan Beulich
1 sibling, 0 replies; 15+ messages in thread
From: Jan Beulich @ 2014-04-30 10:10 UTC (permalink / raw)
To: Dietmar Hahn, Juergen Gross; +Cc: George Dunlap, xen-devel@lists.xen.org
>>> On 30.04.14 at 11:39, <juergen.gross@ts.fujitsu.com> wrote:
> On 30.04.2014 11:26, George Dunlap wrote:
>> On 04/30/2014 10:02 AM, Dietmar Hahn wrote:
>>> Hi,
>>>
>>> while debugging a vmcore with the crash tool I stumpled over a little
> problem.
>>> I wanted to look at the "struct csched_private" of credit scheduler and got
>>> the contents of the "struct csched_private" of credit2.
>>> The debug informations of the hypervisor contain 2 entries
>>> <1><b185e>: Abbrev Number: 8 (DW_TAG_structure_type)
>>> <b185f> DW_AT_name : (indirect string, offset: 0x9d8d):
>>> csched_private
>>> and
>>> <1><c0677>: Abbrev Number: 25 (DW_TAG_structure_type)
>>> <c0678> DW_AT_name : (indirect string, offset: 0x9d8d):
>>> csched_private
>>>
>>> The first is credit and the second credit2. It seems in the crash command
> the
>>> second entry wins :-(.
>>>
>>> Maybe crash has the possibility somewhere to get access to the second
> structure
>>> (I couldn't find it) but for simplicity it would be better to have different
>>> names
>>> I think.
>>> Are there any reasons not to rename
>>> struct csched_private -> struct c2sched_private
>>> or whatever?
>>
>> No reasons at all -- the naming is an artifact of development. Feel free to
>> send a patch renaming it.
>
> I tried it once:
> http://lists.xen.org/archives/html/xen-devel/2013-02/msg02255.html
>
> Jan didn't like it.
And I still don't like it, but if George is okay with taking such a change,
it'll go in, as he's the maintainer of that code.
Jan
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hidden symbol when debugging hypervisor
2014-04-30 10:07 ` George Dunlap
@ 2014-04-30 10:21 ` Jan Beulich
2014-04-30 11:28 ` Andrew Cooper
0 siblings, 1 reply; 15+ messages in thread
From: Jan Beulich @ 2014-04-30 10:21 UTC (permalink / raw)
To: George Dunlap; +Cc: Juergen Gross, Dietmar Hahn, xen-devel@lists.xen.org
>>> On 30.04.14 at 12:07, <george.dunlap@eu.citrix.com> wrote:
> On 04/30/2014 10:39 AM, Juergen Gross wrote:
>> On 30.04.2014 11:26, George Dunlap wrote:
>>> On 04/30/2014 10:02 AM, Dietmar Hahn wrote:
>>>> Hi,
>>>>
>>>> while debugging a vmcore with the crash tool I stumpled over a
>>>> little problem.
>>>> I wanted to look at the "struct csched_private" of credit scheduler
>>>> and got
>>>> the contents of the "struct csched_private" of credit2.
>>>> The debug informations of the hypervisor contain 2 entries
>>>> <1><b185e>: Abbrev Number: 8 (DW_TAG_structure_type)
>>>> <b185f> DW_AT_name : (indirect string, offset: 0x9d8d):
>>>> csched_private
>>>> and
>>>> <1><c0677>: Abbrev Number: 25 (DW_TAG_structure_type)
>>>> <c0678> DW_AT_name : (indirect string, offset: 0x9d8d):
>>>> csched_private
>>>>
>>>> The first is credit and the second credit2. It seems in the crash
>>>> command the
>>>> second entry wins :-(.
>>>>
>>>> Maybe crash has the possibility somewhere to get access to the
>>>> second structure
>>>> (I couldn't find it) but for simplicity it would be better to have
>>>> different
>>>> names
>>>> I think.
>>>> Are there any reasons not to rename
>>>> struct csched_private -> struct c2sched_private
>>>> or whatever?
>>>
>>> No reasons at all -- the naming is an artifact of development. Feel
>>> free to
>>> send a patch renaming it.
>>
>> I tried it once:
>> http://lists.xen.org/archives/html/xen-devel/2013-02/msg02255.html
>>
>> Jan didn't like it.
>
> It looks like he was only thinking about backtraces, not about cscope or
> about debugging core dumps. Jan, do you have another solution for
> those, or shall we go ahead and change the names?
If you're happy with the names getting changed, so be it. I have no
alternative suggestion. I simply would have expected that in year
2014 tools can deal with situations like this (i.e. find the applicable
type rather than the first, last, or a random one). Dwarf debug info
certainly has all the necessary information for that to happen.
Jan
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hidden symbol when debugging hypervisor
2014-04-30 10:21 ` Jan Beulich
@ 2014-04-30 11:28 ` Andrew Cooper
2014-04-30 13:16 ` Jan Beulich
0 siblings, 1 reply; 15+ messages in thread
From: Andrew Cooper @ 2014-04-30 11:28 UTC (permalink / raw)
To: Jan Beulich
Cc: George Dunlap, Juergen Gross, Dietmar Hahn,
xen-devel@lists.xen.org
On 30/04/14 11:21, Jan Beulich wrote:
>>>> On 30.04.14 at 12:07, <george.dunlap@eu.citrix.com> wrote:
>> On 04/30/2014 10:39 AM, Juergen Gross wrote:
>>> On 30.04.2014 11:26, George Dunlap wrote:
>>>> On 04/30/2014 10:02 AM, Dietmar Hahn wrote:
>>>>> Hi,
>>>>>
>>>>> while debugging a vmcore with the crash tool I stumpled over a
>>>>> little problem.
>>>>> I wanted to look at the "struct csched_private" of credit scheduler
>>>>> and got
>>>>> the contents of the "struct csched_private" of credit2.
>>>>> The debug informations of the hypervisor contain 2 entries
>>>>> <1><b185e>: Abbrev Number: 8 (DW_TAG_structure_type)
>>>>> <b185f> DW_AT_name : (indirect string, offset: 0x9d8d):
>>>>> csched_private
>>>>> and
>>>>> <1><c0677>: Abbrev Number: 25 (DW_TAG_structure_type)
>>>>> <c0678> DW_AT_name : (indirect string, offset: 0x9d8d):
>>>>> csched_private
>>>>>
>>>>> The first is credit and the second credit2. It seems in the crash
>>>>> command the
>>>>> second entry wins :-(.
>>>>>
>>>>> Maybe crash has the possibility somewhere to get access to the
>>>>> second structure
>>>>> (I couldn't find it) but for simplicity it would be better to have
>>>>> different
>>>>> names
>>>>> I think.
>>>>> Are there any reasons not to rename
>>>>> struct csched_private -> struct c2sched_private
>>>>> or whatever?
>>>> No reasons at all -- the naming is an artifact of development. Feel
>>>> free to
>>>> send a patch renaming it.
>>> I tried it once:
>>> http://lists.xen.org/archives/html/xen-devel/2013-02/msg02255.html
>>>
>>> Jan didn't like it.
>> It looks like he was only thinking about backtraces, not about cscope or
>> about debugging core dumps. Jan, do you have another solution for
>> those, or shall we go ahead and change the names?
> If you're happy with the names getting changed, so be it. I have no
> alternative suggestion. I simply would have expected that in year
> 2014 tools can deal with situations like this (i.e. find the applicable
> type rather than the first, last, or a random one). Dwarf debug info
> certainly has all the necessary information for that to happen.
>
> Jan
That is impossible in this situation (if I have understood it
correctly). In memory, sched_priv is a void pointer, and Dietmar has
asked crash to interpret it as a 'struct csched_private'. It is
plausible that crash could ask "do you mean a struct csched_private from
sched_credit.c or from sched_credit2.c".
I have encountered similar problems generating stack traces with the Xen
Crashdump Analyser, which only has System.map available.
xen.git/xen$ cat System.map | cut -d ' ' -f 3 | sort | uniq -d | wc -l
78
Having duplicate symbol names for different symbols is confusing at the
very least, and trivial to avoid. I reckon that most if not all of
those 78 duplicate symbols can, and should be, deduplicated. Renaming
credit -> credit2 will amend about 1/4 of that list.
~Andrew
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hidden symbol when debugging hypervisor
2014-04-30 11:28 ` Andrew Cooper
@ 2014-04-30 13:16 ` Jan Beulich
2014-04-30 14:08 ` Andrew Cooper
0 siblings, 1 reply; 15+ messages in thread
From: Jan Beulich @ 2014-04-30 13:16 UTC (permalink / raw)
To: Andrew Cooper
Cc: George Dunlap, Juergen Gross, Dietmar Hahn,
xen-devel@lists.xen.org
>>> On 30.04.14 at 13:28, <andrew.cooper3@citrix.com> wrote:
> On 30/04/14 11:21, Jan Beulich wrote:
> I have encountered similar problems generating stack traces with the Xen
> Crashdump Analyser, which only has System.map available.
>
> xen.git/xen$ cat System.map | cut -d ' ' -f 3 | sort | uniq -d | wc -l
> 78
>
> Having duplicate symbol names for different symbols is confusing at the
> very least, and trivial to avoid. I reckon that most if not all of
> those 78 duplicate symbols can, and should be, deduplicated. Renaming
> credit -> credit2 will amend about 1/4 of that list.
For the crash dump analyzer, I can't see why it shouldn't be able to
consume the symbol table from elf-syms or elf.efi instead of the
(reduced) System.map.
And for Xen generated stack traces I think I already said that this has
been on my todo list for quite some time, pending no more important
things to deal with, yet not to follow what you suggest, but to make
Xen consume its own ELF/COFF symbol table instead of the (again
reduced) one generated by tools/symbols.
My main rationale here is that within a source file having prefix-less
names is not only fine, but preferable (less typing, less needless line
wrapping), and hence only global symbols need to be fully
disambiguated.
Jan
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hidden symbol when debugging hypervisor
2014-04-30 13:16 ` Jan Beulich
@ 2014-04-30 14:08 ` Andrew Cooper
2014-04-30 14:16 ` Jan Beulich
0 siblings, 1 reply; 15+ messages in thread
From: Andrew Cooper @ 2014-04-30 14:08 UTC (permalink / raw)
To: Jan Beulich
Cc: George Dunlap, Juergen Gross, Dietmar Hahn,
xen-devel@lists.xen.org
On 30/04/14 14:16, Jan Beulich wrote:
>>>> On 30.04.14 at 13:28, <andrew.cooper3@citrix.com> wrote:
>> On 30/04/14 11:21, Jan Beulich wrote:
>> I have encountered similar problems generating stack traces with the Xen
>> Crashdump Analyser, which only has System.map available.
>>
>> xen.git/xen$ cat System.map | cut -d ' ' -f 3 | sort | uniq -d | wc -l
>> 78
>>
>> Having duplicate symbol names for different symbols is confusing at the
>> very least, and trivial to avoid. I reckon that most if not all of
>> those 78 duplicate symbols can, and should be, deduplicated. Renaming
>> credit -> credit2 will amend about 1/4 of that list.
> For the crash dump analyzer, I can't see why it shouldn't be able to
> consume the symbol table from elf-syms or elf.efi instead of the
> (reduced) System.map.
Because it mostly runs on a systems without the debuginfo rpms installed.
Furthermore, it needs to fit in a 64MB crash region with the crash
kernel and initrd as well (although this is more flexible).
>
> And for Xen generated stack traces I think I already said that this has
> been on my todo list for quite some time, pending no more important
> things to deal with, yet not to follow what you suggest, but to make
> Xen consume its own ELF/COFF symbol table instead of the (again
> reduced) one generated by tools/symbols.
>
> My main rationale here is that within a source file having prefix-less
> names is not only fine, but preferable (less typing, less needless line
> wrapping), and hence only global symbols need to be fully
> disambiguated.
>
> Jan
>From a coding point of view, certainly.
>From a debugging point of view, I completely disagree. From a stack
trace, you want to be able to identify the function absolutely.
Currently, finding "csched_schedule()" in a stack trace still means that
I have to work out which scheduler is actually in use. With a cpupool
using credit1 and a cpupool using credit2, this can be very difficult
after-the-fact.
When stack traces have access to static symbol names, the only concept
of 'non-global' which actually exists are inlined functions.
~Andrew
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hidden symbol when debugging hypervisor
2014-04-30 14:08 ` Andrew Cooper
@ 2014-04-30 14:16 ` Jan Beulich
2014-04-30 14:32 ` Andrew Cooper
0 siblings, 1 reply; 15+ messages in thread
From: Jan Beulich @ 2014-04-30 14:16 UTC (permalink / raw)
To: Andrew Cooper
Cc: George Dunlap, Juergen Gross, Dietmar Hahn,
xen-devel@lists.xen.org
>>> On 30.04.14 at 16:08, <andrew.cooper3@citrix.com> wrote:
> On 30/04/14 14:16, Jan Beulich wrote:
>>>>> On 30.04.14 at 13:28, <andrew.cooper3@citrix.com> wrote:
>>> On 30/04/14 11:21, Jan Beulich wrote:
>>> I have encountered similar problems generating stack traces with the Xen
>>> Crashdump Analyser, which only has System.map available.
>>>
>>> xen.git/xen$ cat System.map | cut -d ' ' -f 3 | sort | uniq -d | wc -l
>>> 78
>>>
>>> Having duplicate symbol names for different symbols is confusing at the
>>> very least, and trivial to avoid. I reckon that most if not all of
>>> those 78 duplicate symbols can, and should be, deduplicated. Renaming
>>> credit -> credit2 will amend about 1/4 of that list.
>> For the crash dump analyzer, I can't see why it shouldn't be able to
>> consume the symbol table from elf-syms or elf.efi instead of the
>> (reduced) System.map.
>
> Because it mostly runs on a systems without the debuginfo rpms installed.
But crash dump analysis wouldn't normally be done on the crashing
system, would it?
> Furthermore, it needs to fit in a 64MB crash region with the crash
> kernel and initrd as well (although this is more flexible).
Why would the symbol table need to be in the crash region?
>> And for Xen generated stack traces I think I already said that this has
>> been on my todo list for quite some time, pending no more important
>> things to deal with, yet not to follow what you suggest, but to make
>> Xen consume its own ELF/COFF symbol table instead of the (again
>> reduced) one generated by tools/symbols.
>>
>> My main rationale here is that within a source file having prefix-less
>> names is not only fine, but preferable (less typing, less needless line
>> wrapping), and hence only global symbols need to be fully
>> disambiguated.
>
> From a coding point of view, certainly.
>
> From a debugging point of view, I completely disagree. From a stack
> trace, you want to be able to identify the function absolutely.
> Currently, finding "csched_schedule()" in a stack trace still means that
> I have to work out which scheduler is actually in use. With a cpupool
> using credit1 and a cpupool using credit2, this can be very difficult
> after-the-fact.
How would "common/sched_credit.c:schedule" (with the pointless
prefix already dropped) be ambiguous?
> When stack traces have access to static symbol names, the only concept
> of 'non-global' which actually exists are inlined functions.
No, because there might be multiple instances of them if the compiler
chose to not inlining them. They, just like other static functions, can
only be fully disambiguated by stating their object file name, which
ought to be easy to imply knowing the source file one.
Jan
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hidden symbol when debugging hypervisor
2014-04-30 14:16 ` Jan Beulich
@ 2014-04-30 14:32 ` Andrew Cooper
2014-04-30 14:49 ` Jan Beulich
0 siblings, 1 reply; 15+ messages in thread
From: Andrew Cooper @ 2014-04-30 14:32 UTC (permalink / raw)
To: Jan Beulich
Cc: George Dunlap, Juergen Gross, Dietmar Hahn,
xen-devel@lists.xen.org
On 30/04/14 15:16, Jan Beulich wrote:
>>>> On 30.04.14 at 16:08, <andrew.cooper3@citrix.com> wrote:
>> On 30/04/14 14:16, Jan Beulich wrote:
>>>>>> On 30.04.14 at 13:28, <andrew.cooper3@citrix.com> wrote:
>>>> On 30/04/14 11:21, Jan Beulich wrote:
>>>> I have encountered similar problems generating stack traces with the Xen
>>>> Crashdump Analyser, which only has System.map available.
>>>>
>>>> xen.git/xen$ cat System.map | cut -d ' ' -f 3 | sort | uniq -d | wc -l
>>>> 78
>>>>
>>>> Having duplicate symbol names for different symbols is confusing at the
>>>> very least, and trivial to avoid. I reckon that most if not all of
>>>> those 78 duplicate symbols can, and should be, deduplicated. Renaming
>>>> credit -> credit2 will amend about 1/4 of that list.
>>> For the crash dump analyzer, I can't see why it shouldn't be able to
>>> consume the symbol table from elf-syms or elf.efi instead of the
>>> (reduced) System.map.
>> Because it mostly runs on a systems without the debuginfo rpms installed.
> But crash dump analysis wouldn't normally be done on the crashing
> system, would it?
Large numbers of XenServer customers have servers with more RAM than
local hard drive space. Storing the crash ram image is not possible.
Analysis gets done on /proc/vmcore in the crash environment, with logs
written into the dom0 root filesystem.
In some copious free time I am looking to extend this to be able to
specify network locations to put the logs, and pack enough into the
initrd so the root filesystem doesn't need mounting. This will help
with issues caused by the root filesystem driver locking up (although in
general the crash environment 'reset_devices' kernel parameter is
usually good enough to get enough of a filesystem working).
>
>> Furthermore, it needs to fit in a 64MB crash region with the crash
>> kernel and initrd as well (although this is more flexible).
> Why would the symbol table need to be in the crash region?
It wouldn't (necessarily), but is certainly less overhead to have the
text symbol table in memory than all of the debugging symbols.
>
>>> And for Xen generated stack traces I think I already said that this has
>>> been on my todo list for quite some time, pending no more important
>>> things to deal with, yet not to follow what you suggest, but to make
>>> Xen consume its own ELF/COFF symbol table instead of the (again
>>> reduced) one generated by tools/symbols.
>>>
>>> My main rationale here is that within a source file having prefix-less
>>> names is not only fine, but preferable (less typing, less needless line
>>> wrapping), and hence only global symbols need to be fully
>>> disambiguated.
>> From a coding point of view, certainly.
>>
>> From a debugging point of view, I completely disagree. From a stack
>> trace, you want to be able to identify the function absolutely.
>> Currently, finding "csched_schedule()" in a stack trace still means that
>> I have to work out which scheduler is actually in use. With a cpupool
>> using credit1 and a cpupool using credit2, this can be very difficult
>> after-the-fact.
> How would "common/sched_credit.c:schedule" (with the pointless
> prefix already dropped) be ambiguous?
That wouldn't, but would we really want full paths in stack traces?
~Andrew
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hidden symbol when debugging hypervisor
2014-04-30 14:32 ` Andrew Cooper
@ 2014-04-30 14:49 ` Jan Beulich
2014-04-30 14:57 ` George Dunlap
2014-04-30 14:58 ` Andrew Cooper
0 siblings, 2 replies; 15+ messages in thread
From: Jan Beulich @ 2014-04-30 14:49 UTC (permalink / raw)
To: Andrew Cooper
Cc: George Dunlap, Juergen Gross, Dietmar Hahn,
xen-devel@lists.xen.org
>>> On 30.04.14 at 16:32, <andrew.cooper3@citrix.com> wrote:
> On 30/04/14 15:16, Jan Beulich wrote:
>>>>> On 30.04.14 at 16:08, <andrew.cooper3@citrix.com> wrote:
>>> Furthermore, it needs to fit in a 64MB crash region with the crash
>>> kernel and initrd as well (although this is more flexible).
>> Why would the symbol table need to be in the crash region?
>
> It wouldn't (necessarily), but is certainly less overhead to have the
> text symbol table in memory than all of the debugging symbols.
I never talked about debug info, but just about the ELF/COFF
symbol table. I'm not sure that's much larger than the nm output,
especially when first tidied of useless (e.g. machine generated)
symbols.
>>> Currently, finding "csched_schedule()" in a stack trace still means that
>>> I have to work out which scheduler is actually in use. With a cpupool
>>> using credit1 and a cpupool using credit2, this can be very difficult
>>> after-the-fact.
>> How would "common/sched_credit.c:schedule" (with the pointless
>> prefix already dropped) be ambiguous?
>
> That wouldn't, but would we really want full paths in stack traces?
I'm feeling confused - first you ask for names to be distinguishable,
and then you ask whether we would want this? Rather than having
every contributor take care for local symbols to be unique across the
tree, we _should_ leverage information that is available to achieve
the same goal without impact on source size and readability.
And no, I definitely don't want full paths, I want relative (to
$(BASEDIR)) ones.
Jan
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hidden symbol when debugging hypervisor
2014-04-30 14:49 ` Jan Beulich
@ 2014-04-30 14:57 ` George Dunlap
2014-04-30 15:23 ` Jan Beulich
2014-04-30 14:58 ` Andrew Cooper
1 sibling, 1 reply; 15+ messages in thread
From: George Dunlap @ 2014-04-30 14:57 UTC (permalink / raw)
To: Jan Beulich, Andrew Cooper
Cc: Juergen Gross, Dietmar Hahn, xen-devel@lists.xen.org
On 04/30/2014 03:49 PM, Jan Beulich wrote:
>>>> On 30.04.14 at 16:32, <andrew.cooper3@citrix.com> wrote:
>> On 30/04/14 15:16, Jan Beulich wrote:
>>>>>> On 30.04.14 at 16:08, <andrew.cooper3@citrix.com> wrote:
>>>> Furthermore, it needs to fit in a 64MB crash region with the crash
>>>> kernel and initrd as well (although this is more flexible).
>>> Why would the symbol table need to be in the crash region?
>> It wouldn't (necessarily), but is certainly less overhead to have the
>> text symbol table in memory than all of the debugging symbols.
> I never talked about debug info, but just about the ELF/COFF
> symbol table. I'm not sure that's much larger than the nm output,
> especially when first tidied of useless (e.g. machine generated)
> symbols.
>
>>>> Currently, finding "csched_schedule()" in a stack trace still means that
>>>> I have to work out which scheduler is actually in use. With a cpupool
>>>> using credit1 and a cpupool using credit2, this can be very difficult
>>>> after-the-fact.
>>> How would "common/sched_credit.c:schedule" (with the pointless
>>> prefix already dropped) be ambiguous?
>> That wouldn't, but would we really want full paths in stack traces?
> I'm feeling confused - first you ask for names to be distinguishable,
> and then you ask whether we would want this? Rather than having
> every contributor take care for local symbols to be unique across the
> tree, we _should_ leverage information that is available to achieve
> the same goal without impact on source size and readability.
>
> And no, I definitely don't want full paths, I want relative (to
> $(BASEDIR)) ones.
I think "relative to $(BASEDIR)" is what Andrew meant. In any case,
you're adding typically a minimum of 10 characters, worst case a lot
more, to *every single* line in the stack trace, just so that you can
disambiguate a handful of cases where there may be a name collision.
Doestn't it make more sense for the programmers to detect when there's a
name collision and add a sensibly small prefix to the symbol?
It would certainly be nice to be able to remove all the "csched_"
prefixes from the credit and credit2 codebases; but we do a lot more
debugging than we do coding, so that has to take priority.
-George
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hidden symbol when debugging hypervisor
2014-04-30 14:49 ` Jan Beulich
2014-04-30 14:57 ` George Dunlap
@ 2014-04-30 14:58 ` Andrew Cooper
1 sibling, 0 replies; 15+ messages in thread
From: Andrew Cooper @ 2014-04-30 14:58 UTC (permalink / raw)
To: Jan Beulich
Cc: George Dunlap, Juergen Gross, Dietmar Hahn,
xen-devel@lists.xen.org
On 30/04/14 15:49, Jan Beulich wrote:
>>>> On 30.04.14 at 16:32, <andrew.cooper3@citrix.com> wrote:
>> On 30/04/14 15:16, Jan Beulich wrote:
>>>>>> On 30.04.14 at 16:08, <andrew.cooper3@citrix.com> wrote:
>>>> Furthermore, it needs to fit in a 64MB crash region with the crash
>>>> kernel and initrd as well (although this is more flexible).
>>> Why would the symbol table need to be in the crash region?
>> It wouldn't (necessarily), but is certainly less overhead to have the
>> text symbol table in memory than all of the debugging symbols.
> I never talked about debug info, but just about the ELF/COFF
> symbol table. I'm not sure that's much larger than the nm output,
> especially when first tidied of useless (e.g. machine generated)
> symbols.
Ah ok - It would certainly be interesting to see.
>
>>>> Currently, finding "csched_schedule()" in a stack trace still means that
>>>> I have to work out which scheduler is actually in use. With a cpupool
>>>> using credit1 and a cpupool using credit2, this can be very difficult
>>>> after-the-fact.
>>> How would "common/sched_credit.c:schedule" (with the pointless
>>> prefix already dropped) be ambiguous?
>> That wouldn't, but would we really want full paths in stack traces?
> I'm feeling confused - first you ask for names to be distinguishable,
> and then you ask whether we would want this? Rather than having
> every contributor take care for local symbols to be unique across the
> tree, we _should_ leverage information that is available to achieve
> the same goal without impact on source size and readability.
>
> And no, I definitely don't want full paths, I want relative (to
> $(BASEDIR)) ones.
>
> Jan
>
I am just concerned that long paths and long function names might start
exceeding the width of a console, even if the path has $(XEN_ROOT)/xen/
stripped off the front.
~Andrew
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Hidden symbol when debugging hypervisor
2014-04-30 14:57 ` George Dunlap
@ 2014-04-30 15:23 ` Jan Beulich
0 siblings, 0 replies; 15+ messages in thread
From: Jan Beulich @ 2014-04-30 15:23 UTC (permalink / raw)
To: Andrew Cooper, George Dunlap
Cc: Juergen Gross, Dietmar Hahn, xen-devel@lists.xen.org
>>> On 30.04.14 at 16:57, <george.dunlap@eu.citrix.com> wrote:
> On 04/30/2014 03:49 PM, Jan Beulich wrote:
> In any case,
> you're adding typically a minimum of 10 characters, worst case a lot
> more, to *every single* line in the stack trace,
Only to those lines referring to non-global symbols. And I think it's
worth it (if I ever get to code this up).
Jan
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2014-04-30 15:23 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-04-30 9:02 Hidden symbol when debugging hypervisor Dietmar Hahn
2014-04-30 9:26 ` George Dunlap
2014-04-30 9:39 ` Juergen Gross
2014-04-30 10:07 ` George Dunlap
2014-04-30 10:21 ` Jan Beulich
2014-04-30 11:28 ` Andrew Cooper
2014-04-30 13:16 ` Jan Beulich
2014-04-30 14:08 ` Andrew Cooper
2014-04-30 14:16 ` Jan Beulich
2014-04-30 14:32 ` Andrew Cooper
2014-04-30 14:49 ` Jan Beulich
2014-04-30 14:57 ` George Dunlap
2014-04-30 15:23 ` Jan Beulich
2014-04-30 14:58 ` Andrew Cooper
2014-04-30 10:10 ` Jan Beulich
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.