[PATCH] PCI: dwc: rcar-gen4: Use 4K EPC BAR alignment

[PATCH] PCI: dwc: rcar-gen4: Use 4K EPC BAR alignment

[Koichiro Den]

R-Car S4 Series (R8A779F[4-7]*) uses a 4K minimum iATU region size
(CX_ATU_MIN_REGION_SIZE = 4K) as per R19UH0161EJ0130 Rev.1.30.

Update the advertised alignment to 4K, as described in
commit 2a9a801620ef ("PCI: endpoint: Add support to specify alignment
for buffers allocated to BARs").

With the previous 1MB alignment requirement, iATU programming for BAR4
on this platform often cannot be performed, since a 1MB-aligned target
address may fall outside the tiny 256B BAR4 window.

Signed-off-by: Koichiro Den <den@valinux.co.jp>
---
 drivers/pci/controller/dwc/pcie-rcar-gen4.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/pci/controller/dwc/pcie-rcar-gen4.c b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
index 3d4a889e38cc..396ef9432299 100644
--- a/drivers/pci/controller/dwc/pcie-rcar-gen4.c
+++ b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
@@ -428,7 +428,7 @@ static const struct pci_epc_features rcar_gen4_pcie_epc_features = {
 	.bar[BAR_3] = { .type = BAR_RESERVED, },
 	.bar[BAR_4] = { .type = BAR_FIXED, .fixed_size = 256 },
 	.bar[BAR_5] = { .type = BAR_RESERVED, },
-	.align = SZ_1M,
+	.align = SZ_4K,
 };
 
 static const struct pci_epc_features*
-- 
2.51.0

Re: [PATCH] PCI: dwc: rcar-gen4: Use 4K EPC BAR alignment

[Manivannan Sadhasivam]

On Thu, Mar 05, 2026 at 10:54:39AM +0900, Koichiro Den wrote:
> R-Car S4 Series (R8A779F[4-7]*) uses a 4K minimum iATU region size
> (CX_ATU_MIN_REGION_SIZE = 4K) as per R19UH0161EJ0130 Rev.1.30.
> 
> Update the advertised alignment to 4K, as described in
> commit 2a9a801620ef ("PCI: endpoint: Add support to specify alignment
> for buffers allocated to BARs").
> 
> With the previous 1MB alignment requirement, iATU programming for BAR4
> on this platform often cannot be performed, since a 1MB-aligned target
> address may fall outside the tiny 256B BAR4 window.
> 

Can you clarify this part? What do you mean by 'falling outside of the 256B
BAR4 window'? Where does the failure happen exactly?

> Signed-off-by: Koichiro Den <den@valinux.co.jp>

Fixes tag?

- Mani

> ---
>  drivers/pci/controller/dwc/pcie-rcar-gen4.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-rcar-gen4.c b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
> index 3d4a889e38cc..396ef9432299 100644
> --- a/drivers/pci/controller/dwc/pcie-rcar-gen4.c
> +++ b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
> @@ -428,7 +428,7 @@ static const struct pci_epc_features rcar_gen4_pcie_epc_features = {
>  	.bar[BAR_3] = { .type = BAR_RESERVED, },
>  	.bar[BAR_4] = { .type = BAR_FIXED, .fixed_size = 256 },
>  	.bar[BAR_5] = { .type = BAR_RESERVED, },
> -	.align = SZ_1M,
> +	.align = SZ_4K,
>  };
>  
>  static const struct pci_epc_features*
> -- 
> 2.51.0
> 

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH] PCI: dwc: rcar-gen4: Use 4K EPC BAR alignment

[Koichiro Den]

On Thu, Mar 05, 2026 at 11:27:42AM +0530, Manivannan Sadhasivam wrote:
> On Thu, Mar 05, 2026 at 10:54:39AM +0900, Koichiro Den wrote:
> > R-Car S4 Series (R8A779F[4-7]*) uses a 4K minimum iATU region size
> > (CX_ATU_MIN_REGION_SIZE = 4K) as per R19UH0161EJ0130 Rev.1.30.
> > 
> > Update the advertised alignment to 4K, as described in
> > commit 2a9a801620ef ("PCI: endpoint: Add support to specify alignment
> > for buffers allocated to BARs").
> > 
> > With the previous 1MB alignment requirement, iATU programming for BAR4
> > on this platform often cannot be performed, since a 1MB-aligned target
> > address may fall outside the tiny 256B BAR4 window.
> > 
> 
> Can you clarify this part? What do you mean by 'falling outside of the 256B
> BAR4 window'? Where does the failure happen exactly?

My primary motivation is to use MSI doorbell [1] for epf-vntb.

epf_ntb_db_bar_init_msi_doorbell() passes message address 'low' to
pci_epf_assign_bar_space(). The message address is a fixed physical address,
or IOVA if EPC is attached to an IOMMU domain.

Even though the doorbell window size (the difference between the 'high' and
'low' message addresses) is typically small, forcing the base address to be
aligned to a 1MB boundary may push the mapping base much lower than necessary.

For example:

  When the doorbell message address 'low' is 0xffdff0a0,

  - With the previous 1MB alignment, it is aligned down to 0xffd00000.
    The offset becomes: 0xffdff0a0 - 0xffd00000 = 0xff0a0, which is far larger
    than the 256B BAR4 window, so the mapping cannot be programmed.

  - With the correct 4K alignment, it is aligned down to 0xffdff000.
    The offset becomes: 0xffdff0a0 - 0xffdff000 = 0xa0, which fits within 256B
    BAR4 window.

  Note: if the address were e.g. 0xffdff1a0 instead of 0xffdff0a0, the
  4K-aligned offset would become 0x1a0 (416 bytes), which still exceeds the 256B
  window. (For simplicity, the 32-bit write width is ignored here.) In such a
  case, programming the mapping would still not be possible.

  Also note that I used the term 'aligned_mem_size' above, which is the local
  variable name in pci_epf_assign_bar_space(). The corresponding struct
  pci_epf_bar field was renamed from 'aligned_size' to 'mem_size' by commit
  483768846d66 ("PCI: endpoint: Rename 'epf_bar::aligned_size' to
  'epf_bar:mem_size'").

[1] Precisely speaking, the "embedded" doorbell fallback:
    https://lore.kernel.org/linux-pci/20260302071427.534158-1-den@valinux.co.jp/

> 
> > Signed-off-by: Koichiro Den <den@valinux.co.jp>
> 
> Fixes tag?

Commit e311b3834dfa ("PCI: rcar-gen4: Add endpoint mode support") is much later
than 2a9a801620ef ("PCI: endpoint: Add support to specify alignment for buffers
allocated to BARs"), so I believe it makes sense to add:

  Fixes: e311b3834dfa ("PCI: rcar-gen4: Add endpoint mode support")

Best regards,
Koichiro

> 
> - Mani
> 
> > ---
> >  drivers/pci/controller/dwc/pcie-rcar-gen4.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/drivers/pci/controller/dwc/pcie-rcar-gen4.c b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
> > index 3d4a889e38cc..396ef9432299 100644
> > --- a/drivers/pci/controller/dwc/pcie-rcar-gen4.c
> > +++ b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
> > @@ -428,7 +428,7 @@ static const struct pci_epc_features rcar_gen4_pcie_epc_features = {
> >  	.bar[BAR_3] = { .type = BAR_RESERVED, },
> >  	.bar[BAR_4] = { .type = BAR_FIXED, .fixed_size = 256 },
> >  	.bar[BAR_5] = { .type = BAR_RESERVED, },
> > -	.align = SZ_1M,
> > +	.align = SZ_4K,
> >  };
> >  
> >  static const struct pci_epc_features*
> > -- 
> > 2.51.0
> > 
> 
> -- 
> மணிவண்ணன் சதாசிவம்

Re: [PATCH] PCI: dwc: rcar-gen4: Use 4K EPC BAR alignment

[Manivannan Sadhasivam]

On Thu, Mar 05, 2026 at 04:03:03PM +0900, Koichiro Den wrote:
> On Thu, Mar 05, 2026 at 11:27:42AM +0530, Manivannan Sadhasivam wrote:
> > On Thu, Mar 05, 2026 at 10:54:39AM +0900, Koichiro Den wrote:
> > > R-Car S4 Series (R8A779F[4-7]*) uses a 4K minimum iATU region size
> > > (CX_ATU_MIN_REGION_SIZE = 4K) as per R19UH0161EJ0130 Rev.1.30.
> > > 
> > > Update the advertised alignment to 4K, as described in
> > > commit 2a9a801620ef ("PCI: endpoint: Add support to specify alignment
> > > for buffers allocated to BARs").
> > > 
> > > With the previous 1MB alignment requirement, iATU programming for BAR4
> > > on this platform often cannot be performed, since a 1MB-aligned target
> > > address may fall outside the tiny 256B BAR4 window.
> > > 
> > 
> > Can you clarify this part? What do you mean by 'falling outside of the 256B
> > BAR4 window'? Where does the failure happen exactly?
> 
> My primary motivation is to use MSI doorbell [1] for epf-vntb.
> 
> epf_ntb_db_bar_init_msi_doorbell() passes message address 'low' to
> pci_epf_assign_bar_space(). The message address is a fixed physical address,
> or IOVA if EPC is attached to an IOMMU domain.
> 
> Even though the doorbell window size (the difference between the 'high' and
> 'low' message addresses) is typically small, forcing the base address to be
> aligned to a 1MB boundary may push the mapping base much lower than necessary.
> 
> For example:
> 
>   When the doorbell message address 'low' is 0xffdff0a0,
> 
>   - With the previous 1MB alignment, it is aligned down to 0xffd00000.
>     The offset becomes: 0xffdff0a0 - 0xffd00000 = 0xff0a0, which is far larger
>     than the 256B BAR4 window, so the mapping cannot be programmed.
> 
>   - With the correct 4K alignment, it is aligned down to 0xffdff000.
>     The offset becomes: 0xffdff0a0 - 0xffdff000 = 0xa0, which fits within 256B
>     BAR4 window.
> 

Okay, thanks for clarifying. This information should've been present in the
commit message. Though, the top motivation is to comply with the reference
manual.

>   Note: if the address were e.g. 0xffdff1a0 instead of 0xffdff0a0, the
>   4K-aligned offset would become 0x1a0 (416 bytes), which still exceeds the 256B
>   window. (For simplicity, the 32-bit write width is ignored here.) In such a
>   case, programming the mapping would still not be possible.
> 

Hmm, that's probably fine as we would be running into hardware limitation.

>   Also note that I used the term 'aligned_mem_size' above, which is the local
>   variable name in pci_epf_assign_bar_space(). The corresponding struct
>   pci_epf_bar field was renamed from 'aligned_size' to 'mem_size' by commit
>   483768846d66 ("PCI: endpoint: Rename 'epf_bar::aligned_size' to
>   'epf_bar:mem_size'").
> 
> [1] Precisely speaking, the "embedded" doorbell fallback:
>     https://lore.kernel.org/linux-pci/20260302071427.534158-1-den@valinux.co.jp/
> 
> > 
> > > Signed-off-by: Koichiro Den <den@valinux.co.jp>
> > 
> > Fixes tag?
> 
> Commit e311b3834dfa ("PCI: rcar-gen4: Add endpoint mode support") is much later
> than 2a9a801620ef ("PCI: endpoint: Add support to specify alignment for buffers
> allocated to BARs"), so I believe it makes sense to add:
> 
>   Fixes: e311b3834dfa ("PCI: rcar-gen4: Add endpoint mode support")
> 

Please add it in next version.

- Mani

-- 
மணிவண்ணன் சதாசிவம்

Re: [PATCH] PCI: dwc: rcar-gen4: Use 4K EPC BAR alignment

[Koichiro Den]

On Thu, Mar 05, 2026 at 01:44:17PM +0530, Manivannan Sadhasivam wrote:
> On Thu, Mar 05, 2026 at 04:03:03PM +0900, Koichiro Den wrote:
> > On Thu, Mar 05, 2026 at 11:27:42AM +0530, Manivannan Sadhasivam wrote:
> > > On Thu, Mar 05, 2026 at 10:54:39AM +0900, Koichiro Den wrote:
> > > > R-Car S4 Series (R8A779F[4-7]*) uses a 4K minimum iATU region size
> > > > (CX_ATU_MIN_REGION_SIZE = 4K) as per R19UH0161EJ0130 Rev.1.30.
> > > > 
> > > > Update the advertised alignment to 4K, as described in
> > > > commit 2a9a801620ef ("PCI: endpoint: Add support to specify alignment
> > > > for buffers allocated to BARs").
> > > > 
> > > > With the previous 1MB alignment requirement, iATU programming for BAR4
> > > > on this platform often cannot be performed, since a 1MB-aligned target
> > > > address may fall outside the tiny 256B BAR4 window.
> > > > 
> > > 
> > > Can you clarify this part? What do you mean by 'falling outside of the 256B
> > > BAR4 window'? Where does the failure happen exactly?
> > 
> > My primary motivation is to use MSI doorbell [1] for epf-vntb.
> > 
> > epf_ntb_db_bar_init_msi_doorbell() passes message address 'low' to
> > pci_epf_assign_bar_space(). The message address is a fixed physical address,
> > or IOVA if EPC is attached to an IOMMU domain.
> > 
> > Even though the doorbell window size (the difference between the 'high' and
> > 'low' message addresses) is typically small, forcing the base address to be
> > aligned to a 1MB boundary may push the mapping base much lower than necessary.
> > 
> > For example:
> > 
> >   When the doorbell message address 'low' is 0xffdff0a0,
> > 
> >   - With the previous 1MB alignment, it is aligned down to 0xffd00000.
> >     The offset becomes: 0xffdff0a0 - 0xffd00000 = 0xff0a0, which is far larger
> >     than the 256B BAR4 window, so the mapping cannot be programmed.
> > 
> >   - With the correct 4K alignment, it is aligned down to 0xffdff000.
> >     The offset becomes: 0xffdff0a0 - 0xffdff000 = 0xa0, which fits within 256B
> >     BAR4 window.
> > 
> 
> Okay, thanks for clarifying. This information should've been present in the
> commit message. Though, the top motivation is to comply with the reference
> manual.

That makes sense.

> 
> >   Note: if the address were e.g. 0xffdff1a0 instead of 0xffdff0a0, the
> >   4K-aligned offset would become 0x1a0 (416 bytes), which still exceeds the 256B
> >   window. (For simplicity, the 32-bit write width is ignored here.) In such a
> >   case, programming the mapping would still not be possible.
> > 
> 
> Hmm, that's probably fine as we would be running into hardware limitation.

I have the same feeling.

> 
> >   Also note that I used the term 'aligned_mem_size' above, which is the local
> >   variable name in pci_epf_assign_bar_space(). The corresponding struct
> >   pci_epf_bar field was renamed from 'aligned_size' to 'mem_size' by commit
> >   483768846d66 ("PCI: endpoint: Rename 'epf_bar::aligned_size' to
> >   'epf_bar:mem_size'").
> > 
> > [1] Precisely speaking, the "embedded" doorbell fallback:
> >     https://lore.kernel.org/linux-pci/20260302071427.534158-1-den@valinux.co.jp/
> > 
> > > 
> > > > Signed-off-by: Koichiro Den <den@valinux.co.jp>
> > > 
> > > Fixes tag?
> > 
> > Commit e311b3834dfa ("PCI: rcar-gen4: Add endpoint mode support") is much later
> > than 2a9a801620ef ("PCI: endpoint: Add support to specify alignment for buffers
> > allocated to BARs"), so I believe it makes sense to add:
> > 
> >   Fixes: e311b3834dfa ("PCI: rcar-gen4: Add endpoint mode support")
> > 
> 
> Please add it in next version.

I'll add the Fixes tag and send v2 shortly.

Thanks for reviewing,
Koichiro

> 
> - Mani
> 
> -- 
> மணிவண்ணன் சதாசிவம்

Re: [PATCH] PCI: dwc: rcar-gen4: Use 4K EPC BAR alignment

[Niklas Cassel]

On Thu, Mar 05, 2026 at 10:54:39AM +0900, Koichiro Den wrote:
> R-Car S4 Series (R8A779F[4-7]*) uses a 4K minimum iATU region size
> (CX_ATU_MIN_REGION_SIZE = 4K) as per R19UH0161EJ0130 Rev.1.30.
> 
> Update the advertised alignment to 4K, as described in
> commit 2a9a801620ef ("PCI: endpoint: Add support to specify alignment
> for buffers allocated to BARs").
> 
> With the previous 1MB alignment requirement, iATU programming for BAR4
> on this platform often cannot be performed, since a 1MB-aligned target
> address may fall outside the tiny 256B BAR4 window.
> 
> Signed-off-by: Koichiro Den <den@valinux.co.jp>
> ---
>  drivers/pci/controller/dwc/pcie-rcar-gen4.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/controller/dwc/pcie-rcar-gen4.c b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
> index 3d4a889e38cc..396ef9432299 100644
> --- a/drivers/pci/controller/dwc/pcie-rcar-gen4.c
> +++ b/drivers/pci/controller/dwc/pcie-rcar-gen4.c
> @@ -428,7 +428,7 @@ static const struct pci_epc_features rcar_gen4_pcie_epc_features = {
>  	.bar[BAR_3] = { .type = BAR_RESERVED, },
>  	.bar[BAR_4] = { .type = BAR_FIXED, .fixed_size = 256 },
>  	.bar[BAR_5] = { .type = BAR_RESERVED, },
> -	.align = SZ_1M,
> +	.align = SZ_4K,
>  };
>  
>  static const struct pci_epc_features*
> -- 
> 2.51.0
> 

CX_ATU_MIN_REGION_SIZE has
Label: Minimum Size of Translation Region
Value Range: 4, 8, 16, 32, 64 kB
Default: 64 kB

1 MB is larger than 64 kB, so it is clearly wrong.


Kind regards,
Niklas

Re: [PATCH] PCI: dwc: rcar-gen4: Use 4K EPC BAR alignment

[Niklas Cassel]

On Thu, Mar 05, 2026 at 10:54:39AM +0900, Koichiro Den wrote:
> R-Car S4 Series (R8A779F[4-7]*) uses a 4K minimum iATU region size
> (CX_ATU_MIN_REGION_SIZE = 4K) as per R19UH0161EJ0130 Rev.1.30.
> 
> Update the advertised alignment to 4K, as described in
> commit 2a9a801620ef ("PCI: endpoint: Add support to specify alignment
> for buffers allocated to BARs").
> 
> With the previous 1MB alignment requirement, iATU programming for BAR4
> on this platform often cannot be performed, since a 1MB-aligned target
> address may fall outside the tiny 256B BAR4 window.

You could also mention that CX_ATU_MIN_REGION_SIZE can configured to
a value in range 4 kB to 64 kB, so 1 MB is clearly bogus.

Reviewed-by: Niklas Cassel <cassel@kernel.org>

Re: [PATCH] PCI: dwc: rcar-gen4: Use 4K EPC BAR alignment

[Koichiro Den]

On Thu, Mar 05, 2026 at 08:37:35AM +0100, Niklas Cassel wrote:
> On Thu, Mar 05, 2026 at 10:54:39AM +0900, Koichiro Den wrote:
> > R-Car S4 Series (R8A779F[4-7]*) uses a 4K minimum iATU region size
> > (CX_ATU_MIN_REGION_SIZE = 4K) as per R19UH0161EJ0130 Rev.1.30.
> > 
> > Update the advertised alignment to 4K, as described in
> > commit 2a9a801620ef ("PCI: endpoint: Add support to specify alignment
> > for buffers allocated to BARs").
> > 
> > With the previous 1MB alignment requirement, iATU programming for BAR4
> > on this platform often cannot be performed, since a 1MB-aligned target
> > address may fall outside the tiny 256B BAR4 window.
> 
> You could also mention that CX_ATU_MIN_REGION_SIZE can configured to
> a value in range 4 kB to 64 kB, so 1 MB is clearly bogus.
> 
> Reviewed-by: Niklas Cassel <cassel@kernel.org>

Thanks for the review and follow-up, Niklas.

(I initially wondered whether the 1MB alignment had some historical reason
behind it, i.e. an intentional or unavoidable hack, which is why I hesitated to
add a Fixes tag. But I agree that it is bogus today.)

Best regards,
Koichiro

Re: [PATCH] PCI: dwc: rcar-gen4: Use 4K EPC BAR alignment

[Niklas Cassel]

On Thu, Mar 05, 2026 at 11:13:14PM +0900, Koichiro Den wrote:
> On Thu, Mar 05, 2026 at 08:37:35AM +0100, Niklas Cassel wrote:
> > On Thu, Mar 05, 2026 at 10:54:39AM +0900, Koichiro Den wrote:
> > > R-Car S4 Series (R8A779F[4-7]*) uses a 4K minimum iATU region size
> > > (CX_ATU_MIN_REGION_SIZE = 4K) as per R19UH0161EJ0130 Rev.1.30.
> > > 
> > > Update the advertised alignment to 4K, as described in
> > > commit 2a9a801620ef ("PCI: endpoint: Add support to specify alignment
> > > for buffers allocated to BARs").
> > > 
> > > With the previous 1MB alignment requirement, iATU programming for BAR4
> > > on this platform often cannot be performed, since a 1MB-aligned target
> > > address may fall outside the tiny 256B BAR4 window.
> > 
> > You could also mention that CX_ATU_MIN_REGION_SIZE can configured to
> > a value in range 4 kB to 64 kB, so 1 MB is clearly bogus.
> > 
> > Reviewed-by: Niklas Cassel <cassel@kernel.org>
> 
> Thanks for the review and follow-up, Niklas.
> 
> (I initially wondered whether the 1MB alignment had some historical reason
> behind it, i.e. an intentional or unavoidable hack, which is why I hesitated to
> add a Fixes tag. But I agree that it is bogus today.)

My best guess:
Since rcar-gen4 previously incorrectly marked the Resizable BARs as
Programmable, and we know that the minumum size for a Resizable BAR is 1 MB...

pci_epf_alloc_space() and pci_epf_align_inbound_addr() would have done the
"right thing" for these incorrectly configured BARs, when putting an incorrect
alignment of 1 MB. (Even if this would result in excessive alignment for the
non-Resizable BARs, e.g. BAR4.)

Now when rcar-gen4 have marked the resizable BARs as resizable,
pci_epf_alloc_space() and pci_epf_align_inbound_addr() will still do the right
thing (ensure 1 MB alignment for the resizable BARs), but will not result in
excessive alignment for the non-Resizable BARs, e.g. BAR4.


Kind regards,
Niklas