[Qemu-devel] Memory Traces for System Simulation

[Qemu-devel] Memory Traces for System Simulation

[Luis Useche]

Hi Guys,

I have been trying the last two days to find the correct places in the
qemu code to trace the memory accesses in the running system. The qemu
code is somewhat confusing and there are plenty of places that look like
the correct trace point.

I know that there exist plenty of threads about this matter in the list
but non of them solve my problem.

Some people suggested to use Argos (http://www.few.vu.nl/argos/) to get
this information. There are two problems with this: (1) There is no clear
documentation on how to do this in Argos (2) The code seems outdated and
does not compile in systems with gcc>=4.

There is also a patch for qemu 0.8 but it does not apply anymore.
http://www.csl.cornell.edu/~vince/projects/qemu-trace/old/.

At the moment, I am instrumenting several functions in exec.c as my trace
points:
	ldl_phys
	ldq_phys
	ldup_phys
	lduw_phys
	stl_phys_notdirty
	stq_phys_notdirty
	stl_phys
	stb_phys
	stw_phys
	stq_phys

I would really appreciate any suggestion you have in order to solve my
problem. If you have any insights in the solutions I explained above I
would be very thankful.

Given that many people seems to be having the same problem than I, it
would be nice to have an actual framework that add this functionality to
qemu. I can offer myself to do that as long as I have enough help.

As a parallel question: Does qemu simulate CPU cache? i.e. There is always
memory access even when this would not happen in a real system due to CPU
cache?

Thanks in advance,
-- 
Luis Useche <luis@cs.fiu.edu>
Ph.D. Student
Florida International University

Re: [Qemu-devel] Memory Traces for System Simulation

[Luis Useche]

Nobody? Please somebody help me. I would really appreciate it.

On Wed, Jun 17, 2009 at 05:50:18PM -0400, Luis Useche wrote:
> Hi Guys,
> 
> I have been trying the last two days to find the correct places in the
> qemu code to trace the memory accesses in the running system. The qemu
> code is somewhat confusing and there are plenty of places that look like
> the correct trace point.
> 
> I know that there exist plenty of threads about this matter in the list
> but non of them solve my problem.
> 
> Some people suggested to use Argos (http://www.few.vu.nl/argos/) to get
> this information. There are two problems with this: (1) There is no clear
> documentation on how to do this in Argos (2) The code seems outdated and
> does not compile in systems with gcc>=4.
> 
> There is also a patch for qemu 0.8 but it does not apply anymore.
> http://www.csl.cornell.edu/~vince/projects/qemu-trace/old/.
> 
> At the moment, I am instrumenting several functions in exec.c as my trace
> points:
> 	ldl_phys
> 	ldq_phys
> 	ldup_phys
> 	lduw_phys
> 	stl_phys_notdirty
> 	stq_phys_notdirty
> 	stl_phys
> 	stb_phys
> 	stw_phys
> 	stq_phys
> 
> I would really appreciate any suggestion you have in order to solve my
> problem. If you have any insights in the solutions I explained above I
> would be very thankful.
> 
> Given that many people seems to be having the same problem than I, it
> would be nice to have an actual framework that add this functionality to
> qemu. I can offer myself to do that as long as I have enough help.
> 
> As a parallel question: Does qemu simulate CPU cache? i.e. There is always
> memory access even when this would not happen in a real system due to CPU
> cache?
> 
> Thanks in advance,

-- 
Luis Useche <luis@cs.fiu.edu>
Ph.D. Student
Florida International University

Re: [Qemu-devel] Memory Traces for System Simulation

[Edgar E. Iglesias]

On Wed, Jun 17, 2009 at 05:50:18PM -0400, Luis Useche wrote:
> Hi Guys,

Hello Luis,

> I have been trying the last two days to find the correct places in the
> qemu code to trace the memory accesses in the running system. The qemu
> code is somewhat confusing and there are plenty of places that look like
> the correct trace point.
> 
> I know that there exist plenty of threads about this matter in the list
> but non of them solve my problem.
> 
> Some people suggested to use Argos (http://www.few.vu.nl/argos/) to get
> this information. There are two problems with this: (1) There is no clear
> documentation on how to do this in Argos (2) The code seems outdated and
> does not compile in systems with gcc>=4.
> 
> There is also a patch for qemu 0.8 but it does not apply anymore.
> http://www.csl.cornell.edu/~vince/projects/qemu-trace/old/.
> 
> At the moment, I am instrumenting several functions in exec.c as my trace
> points:
> 	ldl_phys
> 	ldq_phys
> 	ldup_phys
> 	lduw_phys
> 	stl_phys_notdirty
> 	stq_phys_notdirty
> 	stl_phys
> 	stb_phys
> 	stw_phys
> 	stq_phys

Depending on what you want to do and on your cache arch you might need
to log both virtual and physical addresses. I don't know what the best
place to hook in would be though.

> I would really appreciate any suggestion you have in order to solve my
> problem. If you have any insights in the solutions I explained above I
> would be very thankful.
> 
> Given that many people seems to be having the same problem than I, it
> would be nice to have an actual framework that add this functionality to
> qemu. I can offer myself to do that as long as I have enough help.
> 
> As a parallel question: Does qemu simulate CPU cache? i.e. There is always

Nope.

> memory access even when this would not happen in a real system due to CPU
> cache?

Yes correct.

I've got a pretty nasty hack that online emulates controller and tag
memories. I find it useful for profiling and for some debugging. The data
memories are not emulated so accesses are never absorbed. I don't think
it's very useful for scientific purposes..

Anyway, it kind of works for CRIS and MicroBlaze. Feel free to look around
on the cris git at your own risk :)
git://repo.or.cz/qemu/cris-port.git

Cheers

Re: [Qemu-devel] Memory Traces for System Simulation

[Luis Useche]

On Thu, Jun 18, 2009 at 05:20:05PM +0200, Edgar E. Iglesias wrote:
> On Wed, Jun 17, 2009 at 05:50:18PM -0400, Luis Useche wrote:
> > Hi Guys,
> 
> Hello Luis,

Thanks for the response.

> > At the moment, I am instrumenting several functions in exec.c as my trace
> > points:
> > 	ldl_phys
> > 	ldq_phys
> > 	ldup_phys
> > 	lduw_phys
> > 	stl_phys_notdirty
> > 	stq_phys_notdirty
> > 	stl_phys
> > 	stb_phys
> > 	stw_phys
> > 	stq_phys
> 
> Depending on what you want to do and on your cache arch you might need
> to log both virtual and physical addresses. I don't know what the best
> place to hook in would be though.

If I can find the physical addresses that would be really great. And if I
can find both physical and virtual it would be even better. In any case, I
need to know what section of the code is the most likely to have this
memory operations to trace them.

> > I would really appreciate any suggestion you have in order to solve my
> > problem. If you have any insights in the solutions I explained above I
> > would be very thankful.
> > 
> > Given that many people seems to be having the same problem than I, it
> > would be nice to have an actual framework that add this functionality to
> > qemu. I can offer myself to do that as long as I have enough help.
> > 
> > As a parallel question: Does qemu simulate CPU cache? i.e. There is always
> 
> Nope.
> 
> > memory access even when this would not happen in a real system due to CPU
> > cache?
> 
> Yes correct.
> 
> I've got a pretty nasty hack that online emulates controller and tag
> memories. I find it useful for profiling and for some debugging. The data
> memories are not emulated so accesses are never absorbed. I don't think
> it's very useful for scientific purposes..
> 
> Anyway, it kind of works for CRIS and MicroBlaze. Feel free to look around
> on the cris git at your own risk :)
> git://repo.or.cz/qemu/cris-port.git

This can probably helps. Can you give me more information of what this
does.

Best,
-- 
Luis Useche <luis@cs.fiu.edu>
Ph.D. Student
Florida International University

Re: [Qemu-devel] Memory Traces for System Simulation

[Edgar E. Iglesias]

On Thu, Jun 18, 2009 at 03:48:20PM -0400, Luis Useche wrote:
> On Thu, Jun 18, 2009 at 05:20:05PM +0200, Edgar E. Iglesias wrote:
> > On Wed, Jun 17, 2009 at 05:50:18PM -0400, Luis Useche wrote:

...

> > > As a parallel question: Does qemu simulate CPU cache? i.e. There is always
> > 
> > Nope.
> > 
> > > memory access even when this would not happen in a real system due to CPU
> > > cache?
> > 
> > Yes correct.
> > 
> > I've got a pretty nasty hack that online emulates controller and tag
> > memories. I find it useful for profiling and for some debugging. The data
> > memories are not emulated so accesses are never absorbed. I don't think
> > it's very useful for scientific purposes..
> > 
> > Anyway, it kind of works for CRIS and MicroBlaze. Feel free to look around
> > on the cris git at your own risk :)
> > git://repo.or.cz/qemu/cris-port.git
> 
> This can probably helps. Can you give me more information of what this
> does.

Sure, I'm attaching relevant parts of the README.
Regarding details of the implementation you'll have to check the code.
I suggest looking at l1cache.c and greping for TARGET_HAS_CACHE.

Cheers

--

CRIS
----
For the CRIS based machines there is an L1 cache model to help debugging
and profiling cache related issues. It can:

* Model the I or the D cache or both. Modeling the I cache slows things
  down pretty significantly due to the large amount of cache misses.
* Output a miss-trace containing detalied info about every miss.
* Find cache coherence issues with the DMA. CRIS caches are coherent with
  other bus masters but unfortunately buggy. The model barfs whenever
  the DMA loads a value that is dirty in the dcache. It also barfs when
  the DMA stores memory that is valid in the dcache.
* Inform GDB for misses so that GDB stops the program (useless).
* Output cachegrind compatible profiling info suitable for kcachegrind.

To use, add -d icache,dcache,cachegrind (or any combination of the 3).
kcachegrind cachegrind.qemu.xx

For the CRIS machines there is also an interrupt latency profiler. It
outputs a trace of long code paths that run with interrupts masked off.
It only keeps track of the CRIS core's interrupt request line, not the
when interrupts get masked off by the interrupt controller nor at the
peripherals. To use this add, -d int.

The CRIS translator now also has a partial model of the CRIS core's
pipeline. The model can find instruction sequences that lead to
locks in the CRIS pipeline and possibly unnecessary cycles. A trace
is created marking every instruction that caused a lock. If cachegrind
was enabled, the profile info will contain more accurate cycle estimations.
It should be noted that not all interlocks are detected. To speed things
up, the pipeline model only does things at translation time, so inter
translation block code sequences that lead to locks will not be found.
The interblock issue might get fixed at some point.
There is also no branch prediction profiler yet.