32-bit Thumb-2 breakpoints

32-bit Thumb-2 breakpoints

[Daniel Jacobowitz]

I have been working the past week on GDB support for Thumb-2.  It
turns out that there's a case where we need a 32-bit software
breakpoint instruction to get things right (explanation below).

The kernel currently reserves two architecturally undefined
instructions for breakpoints, one ARM and one Thumb.  I suggest
another for Thumb-2, provisionally 0xfff7 0xffcf, but I'm open
to any other suggestion.  Any comments?  If that sounds OK,
I can put together a patch.

The problem I'm solving is that of stepping through IT blocks.  We
shouldn't use the BKPT instruction, which would interfere with an
attached hardware debugger.  The undefined instruction pattern
currently used as a breakpoint will not trigger an exception if the IT
block flags prevent its execution.  And without adding most of a
Thumb-2 simulator to GDB, we can't statically predict what the
next instruction will do to the CPSR flags, so we don't know which
of the following instructions in the IT block will be executed.
For example:

   cmp   r0, r0
   itt   eq
A: cmpeq r0, r1
B: moveq r0, r2
C: nop

At A, we try to single-step.  Unless we simulate the cmp instruction,
we do not know whether B or C will be executed next.  So we have to
set a breakpoint at each of the two possibilities.  If we set a 16-bit
breakpoint on a 32-bit instruction and it is not taken, then the
second half of that 32-bit instruction will be treated as the next
instruction.

This approach keeps the single-stepping code considerably simpler than
if it had to recognize and simulate every flag-setting instruction (~
40 instruction forms).  It's not the only approach, obviously; if
someone has a good reason why I ought to simulate every possible
flag-setting instruction, I can write the code to do that instead.

GDB can work around receiving a SIGILL so a patched GDB will mostly
work on existing kernels; for best results and abstract correctness,
though, there should be a reserved instruction pattern for SIGTRAP.

-- 
Daniel Jacobowitz
CodeSourcery

32-bit Thumb-2 breakpoints

[Russell King - ARM Linux]

On Mon, Jan 11, 2010 at 04:58:16PM -0500, Daniel Jacobowitz wrote:
> The problem I'm solving is that of stepping through IT blocks.  We
> shouldn't use the BKPT instruction, which would interfere with an
> attached hardware debugger.  The undefined instruction pattern
> currently used as a breakpoint will not trigger an exception if the IT
> block flags prevent its execution.  And without adding most of a
> Thumb-2 simulator to GDB, we can't statically predict what the
> next instruction will do to the CPSR flags, so we don't know which
> of the following instructions in the IT block will be executed.

Can't you read the CPSR in gdb, and look at the IT state bits?  See
A2.5.2 and CPSR register format in the ARM ARM.

32-bit Thumb-2 breakpoints

[Daniel Jacobowitz]

On Mon, Jan 11, 2010 at 10:35:03PM +0000, Russell King - ARM Linux wrote:
> On Mon, Jan 11, 2010 at 04:58:16PM -0500, Daniel Jacobowitz wrote:
> > The problem I'm solving is that of stepping through IT blocks.  We
> > shouldn't use the BKPT instruction, which would interfere with an
> > attached hardware debugger.  The undefined instruction pattern
> > currently used as a breakpoint will not trigger an exception if the IT
> > block flags prevent its execution.  And without adding most of a
> > Thumb-2 simulator to GDB, we can't statically predict what the
> > next instruction will do to the CPSR flags, so we don't know which
> > of the following instructions in the IT block will be executed.
> 
> Can't you read the CPSR in gdb, and look at the IT state bits?  See
> A2.5.2 and CPSR register format in the ARM ARM.

I already have code to do that, but it's not enough on its own.

It's easy to tell whether the instruction at the current pc will be
executed, but hard to tell whether the following instruction will be
executed.  Software single step works by computing the pc after
execution of the next instruction, and setting a breakpoint there; but
to get this right GDB would have to simulate the flag setting effects
of every possible 32-bit Thumb instruction (the 16-bit ones mostly
don't set flags inside the IT block).

In the example from my mail:

   cmp   r0, r0
   itt   eq
A: cmpeq r0, r1
B: moveq r0, r2
C: nop

The cmpeq can be an adcs.w, eors.w, et cetera.  We're at A, trying to
figure out whether to put a breakpoint on B or C (both of which could
validly be branches).

It's possible I've missed something obvious, of course!

-- 
Daniel Jacobowitz
CodeSourcery

32-bit Thumb-2 breakpoints

[Jamie Lokier]

Daniel Jacobowitz wrote:
> In the example from my mail:
> 
>    cmp   r0, r0
>    itt   eq
> A: cmpeq r0, r1
> B: moveq r0, r2
> C: nop
> 
> The cmpeq can be an adcs.w, eors.w, et cetera.  We're at A, trying to
> figure out whether to put a breakpoint on B or C (both of which could
> validly be branches).
>
> It's possible I've missed something obvious, of course!

I couldn't really tell from a quick glance at the net, which only
half-explained itt, so I'll ask some newbie-sounding questions just in
case it is something obvious.

Isn't the itt sequence for Thumb-2 supposed to be consistent with the
condition sequence for ARM, so that it should be:

    cmp   r0, r0
    itt   eq
 A: cmpeq r0, r1
 B: movne r0, r2   <- that is, ne not eq, because it's the "then" branch
 C: nop

And does the cmp at A really affect execution of the instructions at B
and C in a Thumb-2 itt sequence, rather than the condition being
checked once@the time of the itt instruction and played out for the
following 3 instructions?

Thanks,
-- Jamie

32-bit Thumb-2 breakpoints

[Russell King - ARM Linux]

On Mon, Jan 11, 2010 at 11:10:27PM +0000, Jamie Lokier wrote:
> Daniel Jacobowitz wrote:
> > In the example from my mail:
> > 
> >    cmp   r0, r0
> >    itt   eq
> > A: cmpeq r0, r1
> > B: moveq r0, r2
> > C: nop
> > 
> > The cmpeq can be an adcs.w, eors.w, et cetera.  We're at A, trying to
> > figure out whether to put a breakpoint on B or C (both of which could
> > validly be branches).
> >
> > It's possible I've missed something obvious, of course!
> 
> I couldn't really tell from a quick glance at the net, which only
> half-explained itt, so I'll ask some newbie-sounding questions just in
> case it is something obvious.

itt = if then then. (if condition met, execute following two instructions)
ite = if then else. (if condition met, execute following instruction,
  otherwise execute the instruction following _that_ instruction.)

The CPSR records the IT state.

32-bit Thumb-2 breakpoints

[Daniel Jacobowitz]

On Mon, Jan 11, 2010 at 11:10:27PM +0000, Jamie Lokier wrote:
> Isn't the itt sequence for Thumb-2 supposed to be consistent with the
> condition sequence for ARM, so that it should be:
> 
>     cmp   r0, r0
>     itt   eq
>  A: cmpeq r0, r1
>  B: movne r0, r2   <- that is, ne not eq, because it's the "then" branch
>  C: nop

No, that'd be if you wrote "ite eq".  This is a more complicated
sequence; you can get the ARM equivalent by ignoring the itt
instruction (the assembler will just use it for a consistency check).

This block is "(r0 == r0 && r0 == r1) -> r0 = r2".

> And does the cmp at A really affect execution of the instructions at B
> and C in a Thumb-2 itt sequence, rather than the condition being
> checked once at the time of the itt instruction and played out for the
> following 3 instructions?

Yes, it really does.  Useful for compound conditions, and for maximal
confusion.  Thankfully, both branches into IT blocks and branches
leaving IT blocks not at the last instruction are forbidden.

-- 
Daniel Jacobowitz
CodeSourcery

32-bit Thumb-2 breakpoints

[Russell King - ARM Linux]

On Mon, Jan 11, 2010 at 05:54:36PM -0500, Daniel Jacobowitz wrote:
> On Mon, Jan 11, 2010 at 10:35:03PM +0000, Russell King - ARM Linux wrote:
> > On Mon, Jan 11, 2010 at 04:58:16PM -0500, Daniel Jacobowitz wrote:
> > > The problem I'm solving is that of stepping through IT blocks.  We
> > > shouldn't use the BKPT instruction, which would interfere with an
> > > attached hardware debugger.  The undefined instruction pattern
> > > currently used as a breakpoint will not trigger an exception if the IT
> > > block flags prevent its execution.  And without adding most of a
> > > Thumb-2 simulator to GDB, we can't statically predict what the
> > > next instruction will do to the CPSR flags, so we don't know which
> > > of the following instructions in the IT block will be executed.
> > 
> > Can't you read the CPSR in gdb, and look at the IT state bits?  See
> > A2.5.2 and CPSR register format in the ARM ARM.
> 
> I already have code to do that, but it's not enough on its own.
> 
> It's easy to tell whether the instruction at the current pc will be
> executed, but hard to tell whether the following instruction will be
> executed.  Software single step works by computing the pc after
> execution of the next instruction, and setting a breakpoint there; but
> to get this right GDB would have to simulate the flag setting effects
> of every possible 32-bit Thumb instruction (the 16-bit ones mostly
> don't set flags inside the IT block).

It sounds rather evil - maybe the first question that should be asked
is whether it is legal to change the PSR flags in the middle of an IT
block.

32-bit Thumb-2 breakpoints

[Daniel Jacobowitz]

On Mon, Jan 11, 2010 at 11:31:43PM +0000, Russell King - ARM Linux wrote:
> It sounds rather evil - maybe the first question that should be asked
> is whether it is legal to change the PSR flags in the middle of an IT
> block.

I'm pretty sure it is.  Catalin, do you know for sure?

-- 
Daniel Jacobowitz
CodeSourcery

32-bit Thumb-2 breakpoints

[Jamie Lokier]

Russell King - ARM Linux wrote:
> itt = if then then. (if condition met, execute following two instructions)
> ite = if then else. (if condition met, execute following instruction,
>   otherwise execute the instruction following _that_ instruction.)

Yes, thinko.  Thanks all :-) Blame Wikipedia for teaching me about
those instructions; they sure were a surprise to see in regular ARM
kernel code the first time.

-- Jamie

32-bit Thumb-2 breakpoints

[Jamie Lokier]

Daniel Jacobowitz wrote:
> On Mon, Jan 11, 2010 at 11:10:27PM +0000, Jamie Lokier wrote:
> > And does the cmp at A really affect execution of the instructions at B
> > and C in a Thumb-2 itt sequence, rather than the condition being
> > checked once at the time of the itt instruction and played out for the
> > following 3 instructions?
> 
> Yes, it really does.  Useful for compound conditions, and for maximal
> confusion.  Thankfully, both branches into IT blocks and branches
> leaving IT blocks not at the last instruction are forbidden.

Russel King wrote:
> The CPSR records the IT state.

And therefore, would it be possible to modify the CPSR when
single-stepping to change the IT state so that the instruction
following the next one, i.e. the breakpoint, is unconditional?
So you only need one breakpoint after all.

You'd need to check for particular instructions being single-stepped
and probably emulate these: single-step into another 'it' :-), into
something which reads or writes CPSR directly so they see the correct
values, and jump instructions.

-- Jamie

32-bit Thumb-2 breakpoints

[Daniel Jacobowitz]

On Tue, Jan 12, 2010 at 12:17:28AM +0000, Jamie Lokier wrote:
> And therefore, would it be possible to modify the CPSR when
> single-stepping to change the IT state so that the instruction
> following the next one, i.e. the breakpoint, is unconditional?
> So you only need one breakpoint after all.

Yes, I thought about doing this.  However, the many combinations of
ways you'd have to restore the modified CPSR gave me a fright.  I
don't think it's a robust solution.  It's just different enough from
out-of-line stepping to be unable to share the code.

-- 
Daniel Jacobowitz
CodeSourcery

32-bit Thumb-2 breakpoints

[Catalin Marinas]

On Mon, 2010-01-11 at 23:51 +0000, Daniel Jacobowitz wrote:
> On Mon, Jan 11, 2010 at 11:31:43PM +0000, Russell King - ARM Linux wrote:
> > It sounds rather evil - maybe the first question that should be asked
> > is whether it is legal to change the PSR flags in the middle of an IT
> > block.
> 
> I'm pretty sure it is.  Catalin, do you know for sure?

16-bit instructions in the IT block other than CMN, CMP and TST do not
set the condition flags. You can use 32-bit instructions for setting the
condition flags inside an IT block.

AFAICT, the ARM ARM doesn't say anything that wouldn't allow changing
the condition flags.

-- 
Catalin

32-bit Thumb-2 breakpoints

[Catalin Marinas]

On Mon, 2010-01-11 at 21:58 +0000, Daniel Jacobowitz wrote:
> The kernel currently reserves two architecturally undefined
> instructions for breakpoints, one ARM and one Thumb.  I suggest
> another for Thumb-2, provisionally 0xfff7 0xffcf, but I'm open
> to any other suggestion.  Any comments?  If that sounds OK,
> I can put together a patch.

I find the reasons good enough to add a 32-bit Thumb-2 breakpoint
instruction.

Regarding the opcode, can we not use a permanently undefined one? There
is A6.3.4 in the ARM ARM defining such encoding. Maybe something like
0xf7f0 a000 (or 0xa000 f7f0 if you write it as a 32-bit word).

-- 
Catalin

32-bit Thumb-2 breakpoints

[Daniel Jacobowitz]

On Tue, Jan 12, 2010 at 10:34:58AM +0000, Catalin Marinas wrote:
> On Mon, 2010-01-11 at 21:58 +0000, Daniel Jacobowitz wrote:
> > The kernel currently reserves two architecturally undefined
> > instructions for breakpoints, one ARM and one Thumb.  I suggest
> > another for Thumb-2, provisionally 0xfff7 0xffcf, but I'm open
> > to any other suggestion.  Any comments?  If that sounds OK,
> > I can put together a patch.
> 
> I find the reasons good enough to add a 32-bit Thumb-2 breakpoint
> instruction.
> 
> Regarding the opcode, can we not use a permanently undefined one? There
> is A6.3.4 in the ARM ARM defining such encoding. Maybe something like
> 0xf7f0 a000 (or 0xa000 f7f0 if you write it as a 32-bit word).

Thanks, I meant to but messed up the byte ordering (and more).  I got
something currently undefined, and didn't notice :-(

I'll plan on 0xf7f0 0xa000.

-- 
Daniel Jacobowitz
CodeSourcery

32-bit Thumb-2 breakpoints

[Daniel Jacobowitz]

On Tue, Jan 12, 2010 at 09:25:23AM -0500, Daniel Jacobowitz wrote:
> I'll plan on 0xf7f0 0xa000.

Like so.  Shall I send this to the patch system?

From: Daniel Jacobowitz <dan@codesourcery.com>

Recognize 0xf7f0 0xa000 as a 32-bit breakpoint instruction for
Thumb-2.

Signed-off-by: Daniel Jacobowitz <dan@codesourcery.com>

diff --git a/arch/arm/kernel/ptrace.c b/arch/arm/kernel/ptrace.c
index a2ea385..bd56673 100644
--- a/arch/arm/kernel/ptrace.c
+++ b/arch/arm/kernel/ptrace.c
@@ -499,10 +499,41 @@ static struct undef_hook thumb_break_hook = {
 	.fn		= break_trap,
 };
 
+static int thumb2_break_trap(struct pt_regs *regs, unsigned int instr)
+{
+	unsigned int instr2;
+	void __user *pc;
+
+	/* Check the second half of the instruction.  */
+	pc = (void __user *)(instruction_pointer(regs) + 2);
+
+	if (processor_mode(regs) == SVC_MODE) {
+		instr2 = *(u16 *) pc;
+	} else {
+		get_user(instr2, (u16 __user *)pc);
+	}
+
+	if (instr2 == 0xa000) {
+		ptrace_break(current, regs);
+		return 0;
+	} else {
+		return 1;
+	}
+}
+
+static struct undef_hook thumb2_break_hook = {
+	.instr_mask	= 0xffff,
+	.instr_val	= 0xf7f0,
+	.cpsr_mask	= PSR_T_BIT,
+	.cpsr_val	= PSR_T_BIT,
+	.fn		= thumb2_break_trap,
+};
+
 static int __init ptrace_break_init(void)
 {
 	register_undef_hook(&arm_break_hook);
 	register_undef_hook(&thumb_break_hook);
+	register_undef_hook(&thumb2_break_hook);
 	return 0;
 }
 


-- 
Daniel Jacobowitz
CodeSourcery

32-bit Thumb-2 breakpoints

[Russell King - ARM Linux]

On Thu, Jan 28, 2010 at 03:21:24PM -0500, Daniel Jacobowitz wrote:
> On Tue, Jan 12, 2010 at 09:25:23AM -0500, Daniel Jacobowitz wrote:
> > I'll plan on 0xf7f0 0xa000.
> 
> Like so.  Shall I send this to the patch system?
> 
> From: Daniel Jacobowitz <dan@codesourcery.com>
> 
> Recognize 0xf7f0 0xa000 as a 32-bit breakpoint instruction for
> Thumb-2.

Umm, today there were patches posted using hardware support for
breakpoints / watchpoints.  I've not read through those patches
yet, but in light of hardware support, do we really need this patch
anymore?

32-bit Thumb-2 breakpoints

[Daniel Jacobowitz]

On Tue, Feb 02, 2010 at 10:43:22PM +0000, Russell King - ARM Linux wrote:
> Umm, today there were patches posted using hardware support for
> breakpoints / watchpoints.  I've not read through those patches
> yet, but in light of hardware support, do we really need this patch
> anymore?

Yes, it's unrelated.  Hardware breakpoints are a constrained resource,
but we can insert unlimited software breakpoints (and often need to
exceed the hardware breakpoint limit).

-- 
Daniel Jacobowitz
CodeSourcery

32-bit Thumb-2 breakpoints

[Catalin Marinas]

On Wed, 2010-02-03 at 00:50 +0000, Daniel Jacobowitz wrote:
> On Tue, Feb 02, 2010 at 10:43:22PM +0000, Russell King - ARM Linux
> wrote:
> > Umm, today there were patches posted using hardware support for
> > breakpoints / watchpoints.  I've not read through those patches
> > yet, but in light of hardware support, do we really need this patch
> > anymore?
> 
> Yes, it's unrelated.  Hardware breakpoints are a constrained resource,
> but we can insert unlimited software breakpoints (and often need to
> exceed the hardware breakpoint limit).

I agree, we still need support for software breakpoints.

The main benefit of hardware debugging support is for watchpoints.

-- 
Catalin

32-bit Thumb-2 breakpoints

[Russell King - ARM Linux]

On Wed, Feb 03, 2010 at 11:52:22AM +0000, Catalin Marinas wrote:
> On Wed, 2010-02-03 at 00:50 +0000, Daniel Jacobowitz wrote:
> > On Tue, Feb 02, 2010 at 10:43:22PM +0000, Russell King - ARM Linux
> > wrote:
> > > Umm, today there were patches posted using hardware support for
> > > breakpoints / watchpoints.  I've not read through those patches
> > > yet, but in light of hardware support, do we really need this patch
> > > anymore?
> > 
> > Yes, it's unrelated.  Hardware breakpoints are a constrained resource,
> > but we can insert unlimited software breakpoints (and often need to
> > exceed the hardware breakpoint limit).
> 
> I agree, we still need support for software breakpoints.
> 
> The main benefit of hardware debugging support is for watchpoints.

Software breakpoints are a pain in the backside if you have threaded
programs, because when you insert a breakpoint into one thread, it's
active in all threads - you can't insert a breakpoint into only one
thread.

If we have the ability to set hardware breakpoints, I'd like to see
that used in preference to software breakpoints, and only resort to
software breakpoints if there's no other alternative.

However, I don't believe the current hardware breakpoint/watchpoint
support is anywhere near up to scratch at present - there seems to be
nothing there which deals with tagging these points with a process ID,
nor does it seem to enable/disable them on any thread switching.  So,
(eg) if you install a watchpoint on a virtual address that is used by
the thread being debugged _and_ the debugger (even though they may be
different physical pages), you're going to have the debugger also
trapping.

It might even be the same physical page if it's inside a shared library.

32-bit Thumb-2 breakpoints

[Daniel Jacobowitz]

On Wed, Feb 03, 2010 at 01:28:24PM +0000, Russell King - ARM Linux wrote:
> Software breakpoints are a pain in the backside if you have threaded
> programs, because when you insert a breakpoint into one thread, it's
> active in all threads - you can't insert a breakpoint into only one
> thread.
> 
> If we have the ability to set hardware breakpoints, I'd like to see
> that used in preference to software breakpoints, and only resort to
> software breakpoints if there's no other alternative.

GDB has quite good support for this platform-independent problem,
not to mention that it's sometimes a feature.  I don't think it's a
problem.  The latest addition is displaced stepping support,
available for ARM, which is used to bypass software breakpoints
hit by an uninteresting thread without having to stop the rest
of the program.

There's a lot of reasons to avoid using hardware breakpoints; for
instance, software breakpoints are faster when installing the
breakpoint in every thread, and there's usually contention
with watchpoint resources where the hardware-ness makes a
bigger impact.

-- 
Daniel Jacobowitz
CodeSourcery

32-bit Thumb-2 breakpoints

[Jamie Iles]

On Wed, Feb 03, 2010 at 01:28:24PM +0000, Russell King - ARM Linux wrote:
> On Wed, Feb 03, 2010 at 11:52:22AM +0000, Catalin Marinas wrote:
> > On Wed, 2010-02-03 at 00:50 +0000, Daniel Jacobowitz wrote:
> > > On Tue, Feb 02, 2010 at 10:43:22PM +0000, Russell King - ARM Linux
> > > wrote:
> > > > Umm, today there were patches posted using hardware support for
> > > > breakpoints / watchpoints.  I've not read through those patches
> > > > yet, but in light of hardware support, do we really need this patch
> > > > anymore?
> > > 
> > > Yes, it's unrelated.  Hardware breakpoints are a constrained resource,
> > > but we can insert unlimited software breakpoints (and often need to
> > > exceed the hardware breakpoint limit).
> > 
> > I agree, we still need support for software breakpoints.
> > 
> > The main benefit of hardware debugging support is for watchpoints.
> 
> Software breakpoints are a pain in the backside if you have threaded
> programs, because when you insert a breakpoint into one thread, it's
> active in all threads - you can't insert a breakpoint into only one
> thread.
> 
> If we have the ability to set hardware breakpoints, I'd like to see
> that used in preference to software breakpoints, and only resort to
> software breakpoints if there's no other alternative.
> 
> However, I don't believe the current hardware breakpoint/watchpoint
> support is anywhere near up to scratch at present - there seems to be
> nothing there which deals with tagging these points with a process ID,
> nor does it seem to enable/disable them on any thread switching.  So,
> (eg) if you install a watchpoint on a virtual address that is used by
> the thread being debugged _and_ the debugger (even though they may be
> different physical pages), you're going to have the debugger also
> trapping.
Well it looks like the hardware breakpoint layer is on top of the perf_events
subsystem and the breakpoint becomes a perf event. In this case the breakpoint
should be scheduled in and out by perf on context switches if targetting a
specific PID or could be left in the whole time if desired.

Jamie

32-bit Thumb-2 breakpoints

[Russell King - ARM Linux]

On Wed, Feb 03, 2010 at 01:59:14PM +0000, Jamie Iles wrote:
> Well it looks like the hardware breakpoint layer is on top of the perf_events
> subsystem and the breakpoint becomes a perf event. In this case the breakpoint
> should be scheduled in and out by perf on context switches if targetting a
> specific PID or could be left in the whole time if desired.

Unfortunately, we're drifting from the original topic...

This starts worrying me more.  Is execution stopped (as in actually
stopped, not just switched away from leaving the thread runnable) in
the target thread when one of these 'perf' breakpoints is hit?  If
not, it's completely unsuitable for debuggers to use, and raises the
question of why it's being interfaced with the ptrace code.

Yes, x86 seems to use this method, but it doesn't send signals in the
counter overflow function, so things must be working differently there.

32-bit Thumb-2 breakpoints

[Russell King - ARM Linux]

On Wed, Feb 03, 2010 at 08:48:22AM -0500, Daniel Jacobowitz wrote:
> On Wed, Feb 03, 2010 at 01:28:24PM +0000, Russell King - ARM Linux wrote:
> > Software breakpoints are a pain in the backside if you have threaded
> > programs, because when you insert a breakpoint into one thread, it's
> > active in all threads - you can't insert a breakpoint into only one
> > thread.
> > 
> > If we have the ability to set hardware breakpoints, I'd like to see
> > that used in preference to software breakpoints, and only resort to
> > software breakpoints if there's no other alternative.
> 
> GDB has quite good support for this platform-independent problem,
> not to mention that it's sometimes a feature.  I don't think it's a
> problem.  The latest addition is displaced stepping support,
> available for ARM, which is used to bypass software breakpoints
> hit by an uninteresting thread without having to stop the rest
> of the program.

You should be aware that there has been a move to rid the kernel of
software-based breakpointing implementations through 'utrace' which
I've been more or less ignoring because of that (because it means
that we lose the ability to set any breakpoints.)

Just to be clear, I'm not objecting to your patch (which I think is
fine) - I'm just trying to make sure that the same ground isn't being
covered in multiple ways by different people.

32-bit Thumb-2 breakpoints

[Daniel Jacobowitz]

On Wed, Feb 03, 2010 at 02:43:20PM +0000, Russell King - ARM Linux wrote:
> You should be aware that there has been a move to rid the kernel of
> software-based breakpointing implementations through 'utrace' which
> I've been more or less ignoring because of that (because it means
> that we lose the ability to set any breakpoints.)

You've lost me.  I know that utrace has a ptrace emulation mode, and
memory read/write via ptrace is the only support GDB needs to set and
remove software breakpoints.  The kind of breakpoint support I'm
talking about here is just for generating the SIGTRAP instead of
SIGILL on particular instructions, as you know.  What is it that
utrace removes?  The breakpoint-based PTRACE_SINGLESTEP?

I think we've talked about PTRACE_SINGLESTEP in the past; GDB doesn't
currently use it, because of some of the tricky multi-threading issues
you mentioned a few messages back.  The only way to resolve them is to
have the execution control engine (which is in GDB) have explicit
knowledge of where breakpoints are inserted, so GDB inserts and
removes its own breakpoints.

There's some signal handler corner cases that I'd like to return to
someday, but I've never found the time :-( It's going to take some
serious thought to avoid race conditions in this hybrid model where
GDB does some of the single-step control.

> Just to be clear, I'm not objecting to your patch (which I think is
> fine) - I'm just trying to make sure that the same ground isn't being
> covered in multiple ways by different people.

Thanks.  It's 5912/1.

-- 
Daniel Jacobowitz
CodeSourcery

32-bit Thumb-2 breakpoints

[Matthieu CASTET]

Russell King - ARM Linux a ?crit :
> On Wed, Feb 03, 2010 at 11:52:22AM +0000, Catalin Marinas wrote:
>> On Wed, 2010-02-03 at 00:50 +0000, Daniel Jacobowitz wrote:
>>> On Tue, Feb 02, 2010 at 10:43:22PM +0000, Russell King - ARM Linux
>>> wrote:
>>>> Umm, today there were patches posted using hardware support for
>>>> breakpoints / watchpoints.  I've not read through those patches
>>>> yet, but in light of hardware support, do we really need this patch
>>>> anymore?
>>> Yes, it's unrelated.  Hardware breakpoints are a constrained resource,
>>> but we can insert unlimited software breakpoints (and often need to
>>> exceed the hardware breakpoint limit).
>> I agree, we still need support for software breakpoints.
>>
>> The main benefit of hardware debugging support is for watchpoints.
> 
> Software breakpoints are a pain in the backside if you have threaded
> programs, because when you insert a breakpoint into one thread, it's
> active in all threads - you can't insert a breakpoint into only one
> thread.
> 
An annoying things about software breakpoints is that gdb doesn't
understand arm kernel helper (for atomic operation/tls). And when it try
to set breakpoint here it fails...

Matthieu

32-bit Thumb-2 breakpoints

[Catalin Marinas]

On Wed, 2010-02-03 at 15:02 +0000, Matthieu CASTET wrote:
> Russell King - ARM Linux a ?crit :
> > On Wed, Feb 03, 2010 at 11:52:22AM +0000, Catalin Marinas wrote:
> >> On Wed, 2010-02-03 at 00:50 +0000, Daniel Jacobowitz wrote:
> >>> On Tue, Feb 02, 2010 at 10:43:22PM +0000, Russell King - ARM Linux
> >>> wrote:
> >>>> Umm, today there were patches posted using hardware support for
> >>>> breakpoints / watchpoints.  I've not read through those patches
> >>>> yet, but in light of hardware support, do we really need this patch
> >>>> anymore?
> >>> Yes, it's unrelated.  Hardware breakpoints are a constrained resource,
> >>> but we can insert unlimited software breakpoints (and often need to
> >>> exceed the hardware breakpoint limit).
> >> I agree, we still need support for software breakpoints.
> >>
> >> The main benefit of hardware debugging support is for watchpoints.
> >
> > Software breakpoints are a pain in the backside if you have threaded
> > programs, because when you insert a breakpoint into one thread, it's
> > active in all threads - you can't insert a breakpoint into only one
> > thread.
> 
> An annoying things about software breakpoints is that gdb doesn't
> understand arm kernel helper (for atomic operation/tls). And when it try
> to set breakpoint here it fails...

That kernel helpers are in a page is not writable (or COW-able). Maybe
with hardware breakpoints this could be solved.

-- 
Catalin

32-bit Thumb-2 breakpoints

[Daniel Jacobowitz]

On Wed, Feb 03, 2010 at 04:02:40PM +0100, Matthieu CASTET wrote:
> An annoying things about software breakpoints is that gdb doesn't
> understand arm kernel helper (for atomic operation/tls). And when it try
> to set breakpoint here it fails...

This is fixed in current GDB; it knows how to step out of the helper
page automatically.  If you're still seeing a problem with GDB 7.0,
could you file a bug?

On a related note, it would be Very Helpful(TM) if someone would
implement read-only access to the helper page via ptrace!  Not as
helpful as a visible vDSO with symbol names would be, but it would
still make things a lot easier.

-- 
Daniel Jacobowitz
CodeSourcery

32-bit Thumb-2 breakpoints

[Nicolas Pitre]

On Wed, 3 Feb 2010, Catalin Marinas wrote:

> On Wed, 2010-02-03 at 15:02 +0000, Matthieu CASTET wrote:
> > Russell King - ARM Linux a ?crit :
> > > On Wed, Feb 03, 2010 at 11:52:22AM +0000, Catalin Marinas wrote:
> > >> On Wed, 2010-02-03 at 00:50 +0000, Daniel Jacobowitz wrote:
> > >>> On Tue, Feb 02, 2010 at 10:43:22PM +0000, Russell King - ARM Linux
> > >>> wrote:
> > >>>> Umm, today there were patches posted using hardware support for
> > >>>> breakpoints / watchpoints.  I've not read through those patches
> > >>>> yet, but in light of hardware support, do we really need this patch
> > >>>> anymore?
> > >>> Yes, it's unrelated.  Hardware breakpoints are a constrained resource,
> > >>> but we can insert unlimited software breakpoints (and often need to
> > >>> exceed the hardware breakpoint limit).
> > >> I agree, we still need support for software breakpoints.
> > >>
> > >> The main benefit of hardware debugging support is for watchpoints.
> > >
> > > Software breakpoints are a pain in the backside if you have threaded
> > > programs, because when you insert a breakpoint into one thread, it's
> > > active in all threads - you can't insert a breakpoint into only one
> > > thread.
> > 
> > An annoying things about software breakpoints is that gdb doesn't
> > understand arm kernel helper (for atomic operation/tls). And when it try
> > to set breakpoint here it fails...
> 
> That kernel helpers are in a page is not writable (or COW-able). Maybe
> with hardware breakpoints this could be solved.

This is a well known ABI on ARM, so IMHO gdb should simply manage to put 
a breakpoint right before or right after those helpers are invoked.


Nicolas

32-bit Thumb-2 breakpoints

[Russell King - ARM Linux]

On Wed, Feb 03, 2010 at 10:19:31AM -0500, Daniel Jacobowitz wrote:
> On a related note, it would be Very Helpful(TM) if someone would
> implement read-only access to the helper page via ptrace!  Not as
> helpful as a visible vDSO with symbol names would be, but it would
> still make things a lot easier.

Apart from the TLS value which can be obtained by PTRACE_GET_THREAD_AREA,
the contents of the page is constant across all processes on all CPUs.

32-bit Thumb-2 breakpoints

[Jamie Iles]

On Wed, Feb 03, 2010 at 02:40:22PM +0000, Russell King - ARM Linux wrote:
> On Wed, Feb 03, 2010 at 01:59:14PM +0000, Jamie Iles wrote:
> > Well it looks like the hardware breakpoint layer is on top of the perf_events
> > subsystem and the breakpoint becomes a perf event. In this case the breakpoint
> > should be scheduled in and out by perf on context switches if targetting a
> > specific PID or could be left in the whole time if desired.
> 
> Unfortunately, we're drifting from the original topic...
> 
> This starts worrying me more.  Is execution stopped (as in actually
> stopped, not just switched away from leaving the thread runnable) in
> the target thread when one of these 'perf' breakpoints is hit?  If
> not, it's completely unsuitable for debuggers to use, and raises the
> question of why it's being interfaced with the ptrace code.
Will should be able to give a better anwer but my understanding is that at in
the core hw_breakpoint and perf code, the event is simply logged and the pc
recorded. The ptrace integration allows the processed to have SIGTRAP raised.

Jamie
> Yes, x86 seems to use this method, but it doesn't send signals in the
> counter overflow function, so things must be working differently there.

32-bit Thumb-2 breakpoints

[Daniel Jacobowitz]

On Wed, Feb 03, 2010 at 03:30:25PM +0000, Russell King - ARM Linux wrote:
> On Wed, Feb 03, 2010 at 10:19:31AM -0500, Daniel Jacobowitz wrote:
> > On a related note, it would be Very Helpful(TM) if someone would
> > implement read-only access to the helper page via ptrace!  Not as
> > helpful as a visible vDSO with symbol names would be, but it would
> > still make things a lot easier.
> 
> Apart from the TLS value which can be obtained by PTRACE_GET_THREAD_AREA,
> the contents of the page is constant across all processes on all CPUs.

Yes, I'm aware of that - but having it unreadable means special cases
all through GDB, gdbserver, and anything else that wants to read
memory.

If you really think it's the right choice, I can see about hacking
gdb/gdbserver up for this.

-- 
Daniel Jacobowitz
CodeSourcery

32-bit Thumb-2 breakpoints

[Nicolas Pitre]

On Wed, 3 Feb 2010, Daniel Jacobowitz wrote:

> On Wed, Feb 03, 2010 at 04:02:40PM +0100, Matthieu CASTET wrote:
> > An annoying things about software breakpoints is that gdb doesn't
> > understand arm kernel helper (for atomic operation/tls). And when it try
> > to set breakpoint here it fails...
> 
> This is fixed in current GDB; it knows how to step out of the helper
> page automatically.  If you're still seeing a problem with GDB 7.0,
> could you file a bug?
> 
> On a related note, it would be Very Helpful(TM) if someone would
> implement read-only access to the helper page via ptrace!  Not as
> helpful as a visible vDSO with symbol names would be, but it would
> still make things a lot easier.

Why?  Can't you just access it directly straight from gdb's own address 
space?  This page is the same for all tasks on the system.


Nicolas

32-bit Thumb-2 breakpoints

[Will Deacon]

Hi Russell,

* Jamie Iles wrote:

> On Wed, Feb 03, 2010 at 02:40:22PM +0000, Russell King - ARM Linux wrote:
> > On Wed, Feb 03, 2010 at 01:59:14PM +0000, Jamie Iles wrote:
> > > Well it looks like the hardware breakpoint layer is on top of the perf_events
> > > subsystem and the breakpoint becomes a perf event. In this case the breakpoint
> > > should be scheduled in and out by perf on context switches if targetting a
> > > specific PID or could be left in the whole time if desired.

That's correct. The perf framework adds and removes the breakpoint when the
task to which it is pinned is scheduled in and out. Adding/removing a breakpoint
has the consequence of enabling/disabling it in hardware.

> > Unfortunately, we're drifting from the original topic...
> >
> > This starts worrying me more.  Is execution stopped (as in actually
> > stopped, not just switched away from leaving the thread runnable) in
> > the target thread when one of these 'perf' breakpoints is hit?  If
> > not, it's completely unsuitable for debuggers to use, and raises the
> > question of why it's being interfaced with the ptrace code.
> Will should be able to give a better anwer but my understanding is that at in
> the core hw_breakpoint and perf code, the event is simply logged and the pc
> recorded. The ptrace integration allows the processed to have SIGTRAP raised.

When a breakpoint is hit, a prefetch abort occurs. The hw-breakpoint code will
have installed a handler for the abort provided that the FSR shows that it is 
a debug exception. The handler checks whether a breakpoint or a watchpoint occurred
and signals the event to perf using perf_bp_event. This then calls the callback
function that was registered when register_user_hw_breakpoint was called. In the case
of ptrace, the callback is ptrace_hwbreak_triggered, which sends a SIGTRAP to the current
task.

Why are you concerned about the state of execution? Even if the child is rescheduled
before the debugger takes the trap [is this possible?], it will immediately trap again
when it attempts to re-execute the faulting instruction.

As for x86, it looks like it does send the signal, but it's hidden in the do_debug
routine which is called when the hardware signals the breakpoint. This then uses
notifiers to call back into the hw-breakpoint code which will finally call back to ptrace.

Cheers,

Will

32-bit Thumb-2 breakpoints

[Russell King - ARM Linux]

On Wed, Feb 03, 2010 at 10:35:01AM -0500, Daniel Jacobowitz wrote:
> On Wed, Feb 03, 2010 at 03:30:25PM +0000, Russell King - ARM Linux wrote:
> > On Wed, Feb 03, 2010 at 10:19:31AM -0500, Daniel Jacobowitz wrote:
> > > On a related note, it would be Very Helpful(TM) if someone would
> > > implement read-only access to the helper page via ptrace!  Not as
> > > helpful as a visible vDSO with symbol names would be, but it would
> > > still make things a lot easier.
> > 
> > Apart from the TLS value which can be obtained by PTRACE_GET_THREAD_AREA,
> > the contents of the page is constant across all processes on all CPUs.
> 
> Yes, I'm aware of that - but having it unreadable means special cases
> all through GDB, gdbserver, and anything else that wants to read
> memory.

Well, we could siphon off the PEEK calls, and special case them at
the arch layer - it'll basically be the same hack as what you're
suggesting, only in the kernel instead.

So really the question is whether we want a hack for this in the
kernel or in userspace.

32-bit Thumb-2 breakpoints

[Jamie Lokier]

Daniel Jacobowitz wrote:
> Thankfully, both branches into IT blocks and branches
> leaving IT blocks not at the last instruction are forbidden.

Forbidden as in throws a fault, or forbidden as in "can we use this to
break out of GDB single-stepping to give third-parties a hard time
tracing my secret code"?

-- Jamie

32-bit Thumb-2 breakpoints

[Daniel Jacobowitz]

On Wed, Feb 03, 2010 at 05:23:01PM +0000, Jamie Lokier wrote:
> Daniel Jacobowitz wrote:
> > Thankfully, both branches into IT blocks and branches
> > leaving IT blocks not at the last instruction are forbidden.
> 
> Forbidden as in throws a fault, or forbidden as in "can we use this to
> break out of GDB single-stepping to give third-parties a hard time
> tracing my secret code"?

Forbidden as in "the architecture documentation says it has
unpredictable behavior".

That's generally where I draw the line at trying to handle bad code.
In practice, I will make a wild guess that current processors fault
branches during an IT block and don't do anything fancy for branches
into an IT block, or else it messes up in some hard-to-reproduce way.

-- 
Daniel Jacobowitz
CodeSourcery

32-bit Thumb-2 breakpoints

[Daniel Jacobowitz]

On Wed, Feb 03, 2010 at 04:35:22PM +0000, Russell King - ARM Linux wrote:
> Well, we could siphon off the PEEK calls, and special case them at
> the arch layer - it'll basically be the same hack as what you're
> suggesting, only in the kernel instead.
> 
> So really the question is whether we want a hack for this in the
> kernel or in userspace.

Agreed.  In my opinion, this is a useful thing to have in the kernel,
in order to keep the existing PTRACE_PEEKTEXT API more consistent
between architectures.  Arguments in the other direction?

-- 
Daniel Jacobowitz
CodeSourcery

32-bit Thumb-2 breakpoints

[Pavel Machek]

On Wed 2010-02-03 12:44:26, Daniel Jacobowitz wrote:
> On Wed, Feb 03, 2010 at 05:23:01PM +0000, Jamie Lokier wrote:
> > Daniel Jacobowitz wrote:
> > > Thankfully, both branches into IT blocks and branches
> > > leaving IT blocks not at the last instruction are forbidden.
> > 
> > Forbidden as in throws a fault, or forbidden as in "can we use this to
> > break out of GDB single-stepping to give third-parties a hard time
> > tracing my secret code"?
> 
> Forbidden as in "the architecture documentation says it has
> unpredictable behavior".
> 
> That's generally where I draw the line at trying to handle bad code.
> In practice, I will make a wild guess that current processors fault
> branches during an IT block and don't do anything fancy for branches
> into an IT block, or else it messes up in some hard-to-reproduce way.

So it seems the second kind of forbidden :-(.
									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html