Re: [PATCH v13 1/2] PCI: endpoint: pci-epf-vntb: change doorbell register offset calc mathod

From: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
To: Frank Li <Frank.Li@nxp.com>
Cc: lpieralisi@kernel.org, aisheng.dong@nxp.com, bhelgaas@google.com,
	devicetree@vger.kernel.org, festevam@gmail.com,
	imx@lists.linux.dev, jdmason@kudzu.us, kernel@pengutronix.de,
	kishon@ti.com, krzysztof.kozlowski+dt@linaro.org, kw@linux.com,
	linux-arm-kernel@lists.infradead.org, linux-imx@nxp.com,
	linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org,
	lorenzo.pieralisi@arm.com, lznuaa@gmail.com, maz@kernel.org,
	ntb@lists.linux.dev, peng.fan@nxp.com, robh+dt@kernel.org,
	s.hauer@pengutronix.de, shawnguo@kernel.org, tglx@linutronix.de
Subject: Re: [PATCH v13 1/2] PCI: endpoint: pci-epf-vntb: change doorbell register offset calc mathod
Date: Thu, 24 Nov 2022 14:49:21 +0530	[thread overview]
Message-ID: <20221124091921.GD5119@thinkpad> (raw)
In-Reply-To: <20221124055036.1630573-2-Frank.Li@nxp.com>

On Thu, Nov 24, 2022 at 12:50:35AM -0500, Frank Li wrote:
> In drivers/ntb/hw/epf/ntb_hw_epf.c
> ntb_epf_peer_db_set()
> {
>    ...
>    db_offset = readl(ndev->ctrl_reg + NTB_EPF_DB_OFFSET(interrupt_num));
>    writel(db_data, ndev->db_reg + (db_entry_size * interrupt_num) +
>                db_offset);
>    ...
> }
> 
> The door register offset's formular is
> 	offset = db_entry_size * interrupt_num + db_offset[interrupt_number]

You did not mention the DB BAR here. Without that, this calculation doesn't
make sense.

> 
> Previous db_entry_size is 4, all db_offset is 0.

s/Previous/Previously

> 	irq | offset
>        --------------
>          0     0
>          1     4
>          2     8
>         ...
> 
> Change to db_entry_size is 0 and db_offset is 0, 4, 8, ...
> So we can get the same map value between irq and offset. This will be
> convenience for hardware doorbell register memory map.
> 

In your irq-imx-mu-msi.c driver, the msi_address is calculated as:

```
u64 addr = msi_data->msiir_addr + 4 * data->hwirq;
```

So the MSI addresses itself are of 4 bytes width. So the offsets will be
separated by 8 bytes like, 0, 8, 16,... and this won't match the MSI addresses
as they are 4 bytes apart.

So you want to change the offset to 0, 4, 8,... by zeroing db_entry_size,
right?

> Signed-off-by: Frank Li <Frank.Li@nxp.com>
> ---
>  drivers/pci/endpoint/functions/pci-epf-vntb.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/pci/endpoint/functions/pci-epf-vntb.c b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> index 04698e7995a5..0d744975f815 100644
> --- a/drivers/pci/endpoint/functions/pci-epf-vntb.c
> +++ b/drivers/pci/endpoint/functions/pci-epf-vntb.c
> @@ -461,11 +461,11 @@ static int epf_ntb_config_spad_bar_alloc(struct epf_ntb *ntb)
>  	ctrl->num_mws = ntb->num_mws;
>  	ntb->spad_size = spad_size;
>  
> -	ctrl->db_entry_size = sizeof(u32);
> +	ctrl->db_entry_size = 0;
>  
>  	for (i = 0; i < ntb->db_count; i++) {
>  		ntb->reg->db_data[i] = 1 + i;
> -		ntb->reg->db_offset[i] = 0;
> +		ntb->reg->db_offset[i] = sizeof(u32) * i;

If my above understanding is correct, then you could just reassign
"db_entry_size" in epf_ntb_epc_msi_init().

Thanks,
Mani

>  	}
>  
>  	return 0;
> -- 
> 2.34.1
> 

-- 
மணிவண்ணன் சதாசிவம்