Re: Help with ppc e500 tlb usage

Re: Help with ppc e500 tlb usage

[Cédric Le Goater]

Hello Jesse,

( For some reason, your email never reached my mailbox. In the future,
   could please also Cc: qemu-devel@nongnu.org. )

On 1/18/22 20:40, Jesse Millwood wrote:
> Hello,
> 
> I'm trying to get the e500 to set the initial tlb entry to mirror what the platform expects. I have made some changes that seem to follow the e500 Core Reference Manual better, but they do not seem to work at runtime. I am wondering if someone can point me to where the tlb is actually used to reserve memory.
> 
> Currently, the page size is set relative to the device tree size from the boot info[0]. However, different platforms that use the e500 have different default settings.
> 
> I have added a new field to the PPCE500MachineClass and the CPUPPCState, allowing the platform author to set the initial_tlb_size_kib from the platform's machine class init function. The value then can be accessed in the booke206_initial_map_tsize function that sets returns the page size for the TSIZE field of the MAS1 register. I have kept the original functionality in case the platform doesn't set the field. My new method for computing the TSIZE field from a requested page size takes advantage of the page size being defined as 4^TSIZE KiB and being equal to 2^(2*TSIZE) KiB. So with the page size requested, you can find the bit index that is set with clz64 and then divide by 2. This will give the bit field as defined in the e500 core reference manual.

A new MMU V2 was architected when e5500 was introduced and the TSIZE field
was extended (addition of bit 56) for v2 :

(MMU V1)
52–55 TSIZE Translation size. Defines the page size of the TLB
       	    entry. For TLB arrays that contain fixed-sized TLB
       	    entries, TSIZE is ignored. For variable page-size TLB
       	    arrays, the page size is 4 TSIZE Kbytes.  TSIZE must be
       	    between TLBnCFG[MINSIZE] and TLBnCFG[MAXSIZE]. Valid TSIZE
       	    values for TLB page sizes are described in Section
       	    7.5.3.2, “TLB Entry Page Size.”

(MMU V2)
52–56 TSIZE Translation size. Defines the page size of the TLB
       	    entry. (or the LRAT entry <E.HV.LRAT>) For TLB arrays that
       	    contain fixed-sized TLB entries, TSIZE is ignored. For
       	    variable page-size TLB arrays, the page size is 2 TSIZE
       	    Kbytes. Bit 63 - TSIZE of TLBnPS (or LRATPS <E.HV.LRAT>)
       	    is set if the page size is allowed for the given TLB array
       	    n (or the LRAT <E.HV.LRAT>). Valid TSIZE values for TLB
       	    (or LRAT <E.HV.LRAT>) page sizes are described in Section
       	    7.5.3.2, “TLB Entry Page Size.”
	
The e500 only supports the following page sizes:

	    0b0001 4KB
	    0b0010 16KB
	    0b0011 64KB
	    0b0100 256KB
	    0b1011 4GB
	    0b0101 1MB
	    0b0110 4MB
	    0b0111 16MB
	    0b1000 64MB
	    0b1001 256MB
	    0b1010 1GB

and MMU v2 as found on some e5500 and e6500, these :

	    0b00010 4KB
	    0b00011 8KB
	    0b00100 16KB
	    0b00101 32KB
	    0b00110 64KB
	    0b00111 128KB
	    0b01000 256KB
	    0b01001 512KB
	    0b01010 1MB
	    0b01011 2MB
	    0b01100 4MB
	    0b01101 8MB
	    0b01110 16MB
	    0b01111 32MB
	    0b10000 64MB
	    0b10001 128MB
	    0b10010 256MB
	    0b10011 512MB
	    0b10100 1GB
	    0b10101 2GB

but the values are compatible for both TSIZE field definitions.

Thanks,

C.


> Using my method of setting the TSIZE bits in booke206_initial_map_tsize, mmubooke_create_initial_mapping shifts the returned value and uses it to set the initial MAS1 register.
> 
> I found one other issue that doesn't match up with the e500 Core Reference Manual [1]:
> 
> The shift for the TSIZE is off by 1 [2]. The TSIZE field is bits 52-55 with a reserved field taking up 56-63. The reserved space should be 8 bits not 7.
> 
> My issue comes at runtime. If I compile qemu with my changes and tell it to have a 4GiB initial tlb map size, the monitor command "info tlb" only reports a 2MiB entry. This also makes it so my binary can't run because the window isn't big enough.
>
> I can see that the tlb mas regsiters get set in the mmubooke_create_initial_mapping[3] but I am having trouble finding where that tlb struct is used afterwards. If anyone has any pointers, I'd appreciate it.
>
> I have attached my patches in case anyone is interested in what I've done.
>
> 
> Thanks,
> Jesse
> 
> 0: https://gitlab.com/qemu-project/qemu/-/blob/master/hw/ppc/e500.c#L662
> 1: E500 CORE REFERENCE MANUAL: 2.12.5.2 MAS Register 1 (MAS1) (also attached)
> 2: https://gitlab.com/qemu-project/qemu/-/blob/master/target/ppc/cpu.h#L847
> 3: https://gitlab.com/qemu-project/qemu/-/blob/master/hw/ppc/e500.c#L687
> 
>  From 899aa05a05d4b0598c2637e08c31167e61c8db32 Mon Sep 17 00:00:00 2001
> From: Jesse Millwood <jmillwood@masten.aero>
> Date: Tue, 18 Jan 2022 11:44:20 -0500
> Subject: [PATCH 1/4] e500: Fix TSIZE shift constant
> 
> ---
>   target/ppc/cpu.h | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
> index f20d4ffa6d..ecf0279d08 100644
> --- a/target/ppc/cpu.h
> +++ b/target/ppc/cpu.h
> @@ -844,7 +844,7 @@ enum {
>   #define MAS0_ATSEL_TLB     0
>   #define MAS0_ATSEL_LRAT    MAS0_ATSEL
>   
> -#define MAS1_TSIZE_SHIFT   7
> +#define MAS1_TSIZE_SHIFT   8
>   #define MAS1_TSIZE_MASK    (0x1f << MAS1_TSIZE_SHIFT)
>   
>   #define MAS1_TS_SHIFT      12
> 

Re: Help with ppc e500 tlb usage

[Jesse Millwood]

On Fri, 2022-01-21 at 13:17 +0100, Cédric Le Goater wrote:
> Hello Jesse,
> 
> ( For some reason, your email never reached my mailbox. In the
> future,
>    could please also Cc: mailto:qemu-devel@nongnu.org. )
> 

Absolutely, I wasn't sure how much posting to two mailing lists was accepted. 

> On 1/18/22 20:40, Jesse Millwood wrote:
> > Hello,
> > 
> > I'm trying to get the e500 to set the initial tlb entry to mirror
> > what the platform expects. I have made some changes that seem to
> > follow the e500 Core Reference Manual better, but they do not seem
> > to work at runtime. I am wondering if someone can point me to where
> > the tlb is actually used to reserve memory.
> > 
> > Currently, the page size is set relative to the device tree size
> > from the boot info[0]. However, different platforms that use the
> > e500 have different default settings.
> > 
> > I have added a new field to the PPCE500MachineClass and the
> > CPUPPCState, allowing the platform author to set the
> > initial_tlb_size_kib from the platform's machine class init
> > function. The value then can be accessed in the
> > booke206_initial_map_tsize function that sets returns the page size
> > for the TSIZE field of the MAS1 register. I have kept the original
> > functionality in case the platform doesn't set the field. My new
> > method for computing the TSIZE field from a requested page size
> > takes advantage of the page size being defined as 4^TSIZE KiB and
> > being equal to 2^(2*TSIZE) KiB. So with the page size requested,
> > you can find the bit index that is set with clz64 and then divide
> > by 2. This will give the bit field as defined in the e500 core
> > reference manual.
> 
> A new MMU V2 was architected when e5500 was introduced and the TSIZE
> field
> was extended (addition of bit 56) for v2 :
> 
> (MMU V1)
> 52–55 TSIZE Translation size. Defines the page size of the TLB
>             entry. For TLB arrays that contain fixed-sized TLB
>             entries, TSIZE is ignored. For variable page-size TLB
>             arrays, the page size is 4 TSIZE Kbytes.  TSIZE must be
>             between TLBnCFG[MINSIZE] and TLBnCFG[MAXSIZE]. Valid
> TSIZE
>             values for TLB page sizes are described in Section
>             7.5.3.2, “TLB Entry Page Size.”
> 
> (MMU V2)
> 52–56 TSIZE Translation size. Defines the page size of the TLB
>             entry. (or the LRAT entry <E.HV.LRAT>) For TLB arrays
> that
>             contain fixed-sized TLB entries, TSIZE is ignored. For
>             variable page-size TLB arrays, the page size is 2 TSIZE
>             Kbytes. Bit 63 - TSIZE of TLBnPS (or LRATPS <E.HV.LRAT>)
>             is set if the page size is allowed for the given TLB
> array
>             n (or the LRAT <E.HV.LRAT>). Valid TSIZE values for TLB
>             (or LRAT <E.HV.LRAT>) page sizes are described in Section
>             7.5.3.2, “TLB Entry Page Size.”

Thanks for the background info Cédric, I wasn't aware of the MMUv2. I see now that in target/ppc/cpu_init.c the e500v2 and e5500 have the same mmu_model, the POWREPC_MMU_BOOKE206. 
I'd like to be able to set the initial tlb entry size from the platform definition by indicating the size in kib. I'll just use the 2^TSIZE then. 


>         
> The e500 only supports the following page sizes:
> 
>             0b0001 4KB
>             0b0010 16KB
>             0b0011 64KB
>             0b0100 256KB
>             0b1011 4GB
>             0b0101 1MB
>             0b0110 4MB
>             0b0111 16MB
>             0b1000 64MB
>             0b1001 256MB
>             0b1010 1GB
> 

Is that how much the emulated e500 supports? The E500 Core Reference manual states that it supports up to 4GiB.

> and MMU v2 as found on some e5500 and e6500, these :
> 
>             0b00010 4KB
>             0b00011 8KB
>             0b00100 16KB
>             0b00101 32KB
>             0b00110 64KB
>             0b00111 128KB
>             0b01000 256KB
>             0b01001 512KB
>             0b01010 1MB
>             0b01011 2MB
>             0b01100 4MB
>             0b01101 8MB
>             0b01110 16MB
>             0b01111 32MB
>             0b10000 64MB
>             0b10001 128MB
>             0b10010 256MB
>             0b10011 512MB
>             0b10100 1GB
>             0b10101 2GB
> 
> but the values are compatible for both TSIZE field definitions.
> 
> Thanks,
> 
> C.
> 
> 
> > Using my method of setting the TSIZE bits in
> > booke206_initial_map_tsize, mmubooke_create_initial_mapping shifts
> > the returned value and uses it to set the initial MAS1 register.
> > 
> > I found one other issue that doesn't match up with the e500 Core
> > Reference Manual [1]:
> > 
> > The shift for the TSIZE is off by 1 [2]. The TSIZE field is bits
> > 52-55 with a reserved field taking up 56-63. The reserved space
> > should be 8 bits not 7.
> > 
> > My issue comes at runtime. If I compile qemu with my changes and
> > tell it to have a 4GiB initial tlb map size, the monitor command
> > "info tlb" only reports a 2MiB entry. This also makes it so my
> > binary can't run because the window isn't big enough.
> > 
> > I can see that the tlb mas regsiters get set in the
> > mmubooke_create_initial_mapping[3] but I am having trouble finding
> > where that tlb struct is used afterwards. If anyone has any
> > pointers, I'd appreciate it.
> > 
> > I have attached my patches in case anyone is interested in what
> > I've done.
> > 
> > 
> > Thanks,
> > Jesse
> > 
> > 0: 
> > https://gitlab.com/qemu-project/qemu/-/blob/master/hw/ppc/e500.c#L662
> > 1: E500 CORE REFERENCE MANUAL: 2.12.5.2 MAS Register 1 (MAS1) (also
> > attached)
> > 2: 
> > https://gitlab.com/qemu-project/qemu/-/blob/master/target/ppc/cpu.h#L847
> > 3: 
> > https://gitlab.com/qemu-project/qemu/-/blob/master/hw/ppc/e500.c#L687
> > 
> >  From 899aa05a05d4b0598c2637e08c31167e61c8db32 Mon Sep 17 00:00:00
> > 2001
> > From: Jesse Millwood <mailto:jmillwood@masten.aero>
> > Date: Tue, 18 Jan 2022 11:44:20 -0500
> > Subject: [PATCH 1/4] e500: Fix TSIZE shift constant
> > 
> > ---
> >   target/ppc/cpu.h | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h
> > index f20d4ffa6d..ecf0279d08 100644
> > --- a/target/ppc/cpu.h
> > +++ b/target/ppc/cpu.h
> > @@ -844,7 +844,7 @@ enum {
> >   #define MAS0_ATSEL_TLB     0
> >   #define MAS0_ATSEL_LRAT    MAS0_ATSEL
> >   
> > -#define MAS1_TSIZE_SHIFT   7
> > +#define MAS1_TSIZE_SHIFT   8
> >   #define MAS1_TSIZE_MASK    (0x1f << MAS1_TSIZE_SHIFT)
> >   
> >   #define MAS1_TS_SHIFT      12
> > 
> 

Re: Help with ppc e500 tlb usage

[Cédric Le Goater]

> Thanks for the background info Cédric, I wasn't aware of the MMUv2. I see now that in target/ppc/cpu_init.c the e500v2 and e5500 have the same mmu_model, the POWREPC_MMU_BOOKE206.
> I'd like to be able to set the initial tlb entry size from the platform definition by indicating the size in kib. I'll just use the 2^TSIZE then.

Well, the approach of adding a class attribute and for a new machine
is the right one.
>> The e500 only supports the following page sizes:
>>
>>              0b0001 4KB
>>              0b0010 16KB
>>              0b0011 64KB
>>              0b0100 256KB
>>              0b1011 4GB
>>              0b0101 1MB
>>              0b0110 4MB
>>              0b0111 16MB
>>              0b1000 64MB
>>              0b1001 256MB
>>              0b1010 1GB
>>
> 
> Is that how much the emulated e500 supports? The E500 Core Reference manual states that it supports up to 4GiB.

Yes the list is out of order ! :)

Thanks,

C.

Re: [EXT] - Re: Help with ppc e500 tlb usage

[Jesse Millwood]

On Fri, 2022-01-21 at 18:57 +0100, Cédric Le Goater wrote:
> CAUTION: This email originated from outside of the organization.
> Please do not click links or open attachments unless you recognize
> the sender's email address and know the content is safe.
> 
> 
> > Thanks for the background info Cédric, I wasn't aware of the MMUv2.
> > I see now that in target/ppc/cpu_init.c the e500v2 and e5500 have
> > the same mmu_model, the POWREPC_MMU_BOOKE206.
> > I'd like to be able to set the initial tlb entry size from the
> > platform definition by indicating the size in kib. I'll just use
> > the 2^TSIZE then.
> 
> Well, the approach of adding a class attribute and for a new machine
> is the right one.

Are you saying this so that I can conditionally compute the accurate TSIZE value that matches the real world machine? 

I'm still trying to get my head wrapped around the QOM. Would you advise adding a class attribute to the CPUPPCState or the PPCE500MachineClass? The CPUPPCState has the mmu_model field. However, that is the POWERPC_MMU_BOOKE206. Is there somewhere else you had in mind adding an attribute? 

> > > The e500 only supports the following page sizes:
> > > 
> > >              0b0001 4KB
> > >              0b0010 16KB
> > >              0b0011 64KB
> > >              0b0100 256KB
> > >              0b1011 4GB
> > >              0b0101 1MB
> > >              0b0110 4MB
> > >              0b0111 16MB
> > >              0b1000 64MB
> > >              0b1001 256MB
> > >              0b1010 1GB
> > > 
> > 
> > Is that how much the emulated e500 supports? The E500 Core
> > Reference manual states that it supports up to 4GiB.
> 
> Yes the list is out of order ! :)

Oh shoot, yeah I missed that.

> 
> Thanks,
> 
> C.