[PATCH v1 0/2] soc: aspeed: Add BMC and host driver for PCIe BMC device

[PATCH v1 0/2] soc: aspeed: Add BMC and host driver for PCIe BMC device

[Grégoire Layet]

This series upstreams the PCIe BMC device driver from the ASPEED kernel SDK (branch master-v6.18)  [1].
There are two drivers: a BMC-side driver and a host-side driver.
Together they enable host<->BMC communication via PCIe. 
This includes : shared memory, doorbell interrupt communication and two VUARTs.

This driver can be used on AST2600/AST2700 based PCIe extension cards, such as Asus IPMI Kommando Card.
The virtual UART support can be used for Serial over LAN.
The shared memory can be claimed by userspace programs.

Tested on :
BMC :
- Asus IPMI Kommando Card R1.01, AST2600 A3.
- OpenBMC
Host:
- Linux kernel v7.0.0

I tested VUART communication with interrupt.
No driver modifications were required.

Not tested on AST2700.

Provenance:

I did zero modifications to the ASPEED code, this is only an upstream patch.
The driver files match the ASPEED SDK tree at :
  drivers/soc/aspeed/aspeed-bmc-dev.c
    1815546a54f5f89bc9d3bd8f5658f0a573927509 [2]
    "soc: aspeed: remove MMBI implement in host bmc dev"

  drivers/soc/aspeed/aspeed-host-bmc-dev.c
    7217e3c872166d56389a97e1b81996f73a3e76d5 [3]
    "soc: aspeed: remove iounmap"

The driver code build cleanly against Linux 7.0 without any modification.
No fix were required. 
I built both the BMC side and Host side with KCFLAGS=-W and no warning apperead.
Original authors are credited via Signed-off-by in the individual patches.

checkpatch.pl --strict reports 1 warning for BMC side driver
and 2 warnings for host side driver. 
These are unchanged to keep zero diff against the ASPEED SDK tree.
I can clean them in a v2 if needed.

The host side driver had a submission from Ninad Palsule : 
"soc/aspeed: Add host side BMC device driver" in August 2023 [4].
This submission stalled and did not make it into the mainline kernel.
Since then, the host drivers have changed a lot on the ASPEED SDK. 
I decided not to base this patch on Ninad's one 
but rather to simply upstream the new host driver from ASPEED.

[1]: https://github.com/AspeedTech-BMC/linux/tree/aspeed-master-v6.18/drivers/soc/aspeed
[2]: https://github.com/AspeedTech-BMC/linux/commit/1815546a54f5f89bc9d3bd8f5658f0a573927509
[3]: https://github.com/AspeedTech-BMC/linux/commit/7217e3c872166d56389a97e1b81996f73a3e76d5
[4]: https://lore.kernel.org/linux-aspeed/20230821183525.3427144-1-ninad@linux.ibm.com/

Grégoire Layet (2):
  soc: aspeed: add BMC-side PCIe BMC device driver
  soc: aspeed: add host-side PCIe BMC device driver

 drivers/soc/aspeed/Kconfig               |  22 +
 drivers/soc/aspeed/Makefile              |   2 +
 drivers/soc/aspeed/aspeed-bmc-dev.c      | 701 +++++++++++++++++++++++
 drivers/soc/aspeed/aspeed-host-bmc-dev.c | 664 +++++++++++++++++++++
 4 files changed, 1389 insertions(+)
 create mode 100644 drivers/soc/aspeed/aspeed-bmc-dev.c
 create mode 100644 drivers/soc/aspeed/aspeed-host-bmc-dev.c

-- 
2.51.2

[PATCH v1 1/2] soc: aspeed: add BMC-side PCIe BMC device driver

[Grégoire Layet]

Taken from ASPEED 6.18 Kernel SDK

Add support for PCIe communication BMC<->host.
This add BMC side driver.

Signed-off-by: Jacky Chou <jacky_chou@aspeedtech.com>
Signed-off-by: aspeedyh <yh_chung@aspeedtech.com>
Signed-off-by: Grégoire Layet <gregoire.layet@9elements.com>
Tested-by: Grégoire Layet <gregoire.layet@9elements.com>
---
 drivers/soc/aspeed/Kconfig          |   9 +
 drivers/soc/aspeed/Makefile         |   1 +
 drivers/soc/aspeed/aspeed-bmc-dev.c | 701 ++++++++++++++++++++++++++++
 3 files changed, 711 insertions(+)
 create mode 100644 drivers/soc/aspeed/aspeed-bmc-dev.c

diff --git a/drivers/soc/aspeed/Kconfig b/drivers/soc/aspeed/Kconfig
index f579ee0b5afa..341728df07b3 100644
--- a/drivers/soc/aspeed/Kconfig
+++ b/drivers/soc/aspeed/Kconfig
@@ -4,6 +4,15 @@ if ARCH_ASPEED || COMPILE_TEST
 
 menu "ASPEED SoC drivers"
 
+config ASPEED_BMC_DEV
+	tristate "ASPEED BMC Device"
+	default n
+	help
+	  Enable support for the ASPEED AST2600/AST2700 BMC Device.
+	  This exposes the BMC to the host over PCIe,
+	  providing a shared-memory BAR, host-to-BMC and BMC-to-host
+	  message queues with doorbell interrupts and PCIe-to-LPC bridge.
+
 config ASPEED_LPC_CTRL
 	tristate "ASPEED LPC firmware cycle control"
 	select REGMAP
diff --git a/drivers/soc/aspeed/Makefile b/drivers/soc/aspeed/Makefile
index b35d74592964..fab0d247df66 100644
--- a/drivers/soc/aspeed/Makefile
+++ b/drivers/soc/aspeed/Makefile
@@ -1,4 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_ASPEED_BMC_DEV)		+= aspeed-bmc-dev.o
 obj-$(CONFIG_ASPEED_LPC_CTRL)		+= aspeed-lpc-ctrl.o
 obj-$(CONFIG_ASPEED_LPC_SNOOP)		+= aspeed-lpc-snoop.o
 obj-$(CONFIG_ASPEED_UART_ROUTING)	+= aspeed-uart-routing.o
diff --git a/drivers/soc/aspeed/aspeed-bmc-dev.c b/drivers/soc/aspeed/aspeed-bmc-dev.c
new file mode 100644
index 000000000000..06005fa41d2a
--- /dev/null
+++ b/drivers/soc/aspeed/aspeed-bmc-dev.c
@@ -0,0 +1,701 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+// Copyright (C) ASPEED Technology Inc.
+
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+
+#include <linux/of_address.h>
+#include <linux/of_irq.h>
+#include <linux/of.h>
+#include <linux/of_platform.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/platform_device.h>
+
+#include <linux/wait.h>
+#include <linux/workqueue.h>
+
+#include <linux/regmap.h>
+#include <linux/interrupt.h>
+#include <linux/mfd/syscon.h>
+#include <linux/dma-mapping.h>
+#include <linux/miscdevice.h>
+
+#define SCU_TRIGGER_MSI
+
+/* AST2600 SCU */
+#define ASPEED_SCU04			0x04
+#define AST2600A3_SCU04				0x05030303
+#define ASPEED_SCUC20			0xC20
+#define ASPEED_SCUC24			0xC24
+#define MSI_ROUTING_MASK			GENMASK(11, 10)
+#define PCIDEV1_INTX_MSI_HOST2BMC_EN		BIT(18)
+#define MSI_ROUTING_PCIe2LPC_PCIDEV0		(0x1 << 10)
+#define MSI_ROUTING_PCIe2LPC_PCIDEV1		(0x2 << 10)
+/* AST2700 SCU */
+#define SCU0_REVISION_ID		0x0
+#define REVISION_ID				GENMASK(23, 16)
+#define SCU0_PCIE_CONF_CTRL		0x970
+/* Host2BMC */
+#define ASPEED_BMC_MEM_BAR			0xF10
+#define  PCIE2PCI_MEM_BAR_ENABLE		BIT(1)
+#define  HOST2BMC_MEM_BAR_ENABLE		BIT(0)
+#define ASPEED_BMC_MEM_BAR_REMAP	0xF18
+
+#define ASPEED_BMC_SHADOW_CTRL		0xF50
+#define  READ_ONLY_MASK				BIT(31)
+#define  MASK_BAR1					BIT(2)
+#define  MASK_BAR0					BIT(1)
+#define  SHADOW_CFG					BIT(0)
+
+#define ASPEED_BMC_HOST2BMC_Q1		0xA000
+#define ASPEED_BMC_HOST2BMC_Q2		0xA010
+#define ASPEED_BMC_BMC2HOST_Q1		0xA020
+#define ASPEED_BMC_BMC2HOST_Q2		0xA030
+#define ASPEED_BMC_BMC2HOST_STS		0xA040
+#define	 BMC2HOST_INT_STS_DOORBELL		BIT(31)
+#define	 BMC2HOST_ENABLE_INTB			BIT(30)
+#define	 BMC2HOST_Q1_FULL				BIT(27)
+#define	 BMC2HOST_Q1_EMPTY				BIT(26)
+#define	 BMC2HOST_Q2_FULL				BIT(25)
+#define	 BMC2HOST_Q2_EMPTY				BIT(24)
+#define	 BMC2HOST_Q1_FULL_UNMASK		BIT(23)
+#define	 BMC2HOST_Q1_EMPTY_UNMASK		BIT(22)
+#define	 BMC2HOST_Q2_FULL_UNMASK		BIT(21)
+#define	 BMC2HOST_Q2_EMPTY_UNMASK		BIT(20)
+
+#define ASPEED_BMC_HOST2BMC_STS		0xA044
+#define	 HOST2BMC_INT_STS_DOORBELL		BIT(31)
+#define	 HOST2BMC_ENABLE_INTB			BIT(30)
+#define	 HOST2BMC_Q1_FULL				BIT(27)
+#define	 HOST2BMC_Q1_EMPTY				BIT(26)
+#define	 HOST2BMC_Q2_FULL				BIT(25)
+#define	 HOST2BMC_Q2_EMPTY				BIT(24)
+#define	 HOST2BMC_Q1_FULL_UNMASK		BIT(23)
+#define	 HOST2BMC_Q1_EMPTY_UNMASK		BIT(22)
+#define	 HOST2BMC_Q2_FULL_UNMASK		BIT(21)
+#define	 HOST2BMC_Q2_EMPTY_UNMASK		BIT(20)
+
+#define ASPEED_SCU_PCIE_CONF_CTRL	0xC20
+#define  SCU_PCIE_CONF_BMC_DEV_EN			 BIT(8)
+#define  SCU_PCIE_CONF_BMC_DEV_EN_MMIO		 BIT(9)
+#define  SCU_PCIE_CONF_BMC_DEV_EN_MSI		 BIT(11)
+#define  SCU_PCIE_CONF_BMC_DEV_EN_IRQ		 BIT(13)
+#define  SCU_PCIE_CONF_BMC_DEV_EN_DMA		 BIT(14)
+#define  SCU_PCIE_CONF_BMC_DEV_EN_E2L		 BIT(15)
+#define  SCU_PCIE_CONF_BMC_DEV_EN_LPC_DECODE BIT(21)
+
+#define ASPEED_SCU_BMC_DEV_CLASS	0xC68
+
+#define ASPEED_QUEUE_NUM 2
+enum queue_index {
+	QUEUE1 = 0,
+	QUEUE2,
+};
+
+struct aspeed_platform {
+	int (*init)(struct platform_device *pdev);
+	ssize_t (*queue_rx)(struct file *filp, struct kobject *kobj,
+			    const struct bin_attribute *attr, char *buf, loff_t off, size_t count);
+	ssize_t (*queue_tx)(struct file *filp, struct kobject *kobj,
+			    const struct bin_attribute *attr, char *buf, loff_t off, size_t count);
+};
+
+struct aspeed_queue_message {
+	/* Queue waiters for idle engine */
+	wait_queue_head_t tx_wait;
+	wait_queue_head_t rx_wait;
+	struct kernfs_node *kn;
+	struct bin_attribute bin;
+	int index;
+	struct aspeed_bmc_device *bmc_device;
+};
+
+struct aspeed_bmc_device {
+	unsigned char *host2bmc_base_virt;
+	struct device *dev;
+	struct miscdevice miscdev;
+	int id;
+	void __iomem *reg_base;
+	dma_addr_t bmc_mem_phy;
+	phys_addr_t bmc_mem_size;
+	void *bmc_mem_cpu;
+
+	int pcie2lpc;
+	int irq;
+
+	struct aspeed_queue_message queue[ASPEED_QUEUE_NUM];
+
+	const struct aspeed_platform *platform;
+
+	/* AST2700 */
+	struct regmap *device;
+	struct regmap *e2m;
+
+	struct regmap *scu;
+	int pcie_irq;
+};
+
+static struct aspeed_bmc_device *file_aspeed_bmc_device(struct file *file)
+{
+	return container_of(file->private_data, struct aspeed_bmc_device,
+			miscdev);
+}
+
+static int aspeed_bmc_device_mmap(struct file *file, struct vm_area_struct *vma)
+{
+	struct aspeed_bmc_device *bmc_device = file_aspeed_bmc_device(file);
+	unsigned long vsize = vma->vm_end - vma->vm_start;
+
+	if (vsize > bmc_device->bmc_mem_size)
+		return -EINVAL;
+
+	return dma_mmap_coherent(bmc_device->dev, vma,
+				 bmc_device->bmc_mem_cpu,
+				 bmc_device->bmc_mem_phy,
+				 bmc_device->bmc_mem_size);
+
+}
+
+static const struct file_operations aspeed_bmc_device_fops = {
+	.owner		= THIS_MODULE,
+	.mmap		= aspeed_bmc_device_mmap,
+};
+
+static ssize_t aspeed_ast2600_queue_rx(struct file *filp, struct kobject *kobj,
+				       const struct bin_attribute *attr, char *buf, loff_t off,
+				       size_t count)
+{
+	struct aspeed_queue_message *queue = attr->private;
+	struct aspeed_bmc_device *bmc_device = queue->bmc_device;
+	int index = queue->index;
+	u32 *data = (u32 *)buf;
+	u32 scu_id;
+	int ret;
+
+	ret = wait_event_interruptible(queue->rx_wait,
+				       !(readl(bmc_device->reg_base + ASPEED_BMC_HOST2BMC_STS) &
+				       ((index == QUEUE1) ? HOST2BMC_Q1_EMPTY : HOST2BMC_Q2_EMPTY)));
+	if (ret)
+		return -EINTR;
+
+	data[0] = readl(bmc_device->reg_base +
+			((index == QUEUE1) ? ASPEED_BMC_HOST2BMC_Q1 : ASPEED_BMC_HOST2BMC_Q2));
+
+	regmap_read(bmc_device->scu, ASPEED_SCU04, &scu_id);
+	if (scu_id == AST2600A3_SCU04) {
+		writel(BMC2HOST_INT_STS_DOORBELL | BMC2HOST_ENABLE_INTB,
+		       bmc_device->reg_base + ASPEED_BMC_BMC2HOST_STS);
+	} else {
+		//A0 : BIT(12) A1 : BIT(15)
+		regmap_update_bits(bmc_device->scu, 0x560, BIT(15), BIT(15));
+		regmap_update_bits(bmc_device->scu, 0x560, BIT(15), 0);
+	}
+
+	return sizeof(u32);
+}
+
+static ssize_t aspeed_ast2600_queue_tx(struct file *filp, struct kobject *kobj,
+				       const struct bin_attribute *attr, char *buf, loff_t off,
+				       size_t count)
+{
+	struct aspeed_queue_message *queue = attr->private;
+	struct aspeed_bmc_device *bmc_device = queue->bmc_device;
+	int index = queue->index;
+	u32 tx_buff;
+	u32 scu_id;
+	int ret;
+
+	if (count != sizeof(u32))
+		return -EINVAL;
+
+	ret = wait_event_interruptible(queue->tx_wait,
+				       !(readl(bmc_device->reg_base + ASPEED_BMC_BMC2HOST_STS) &
+				       ((index == QUEUE1) ? BMC2HOST_Q1_FULL : BMC2HOST_Q2_FULL)));
+	if (ret)
+		return -EINTR;
+
+	memcpy(&tx_buff, buf, 4);
+	writel(tx_buff, bmc_device->reg_base + ((index == QUEUE1) ? ASPEED_BMC_BMC2HOST_Q1 :
+								    ASPEED_BMC_BMC2HOST_Q2));
+
+	/* trigger to host
+	 * Only After AST2600A3 support DoorBell MSI
+	 */
+	regmap_read(bmc_device->scu, ASPEED_SCU04, &scu_id);
+	if (scu_id == AST2600A3_SCU04) {
+		writel(BMC2HOST_INT_STS_DOORBELL | BMC2HOST_ENABLE_INTB,
+		       bmc_device->reg_base + ASPEED_BMC_BMC2HOST_STS);
+	} else {
+		//A0 : BIT(12) A1 : BIT(15)
+		regmap_update_bits(bmc_device->scu, 0x560, BIT(15), BIT(15));
+		regmap_update_bits(bmc_device->scu, 0x560, BIT(15), 0);
+	}
+
+	return sizeof(u32);
+}
+
+static ssize_t aspeed_ast2700_queue_rx(struct file *filp, struct kobject *kobj,
+				       const struct bin_attribute *attr, char *buf, loff_t off,
+				       size_t count)
+{
+	struct aspeed_queue_message *queue = attr->private;
+	struct aspeed_bmc_device *bmc_device = queue->bmc_device;
+	int index = queue->index;
+	u32 *data = (u32 *)buf;
+	int ret;
+
+	ret = wait_event_interruptible(queue->rx_wait,
+				       !(readl(bmc_device->reg_base + ASPEED_BMC_HOST2BMC_STS) &
+				       ((index == QUEUE1) ? HOST2BMC_Q1_EMPTY : HOST2BMC_Q2_EMPTY)));
+	if (ret)
+		return -EINTR;
+
+	data[0] = readl(bmc_device->reg_base +
+			((index == QUEUE1) ? ASPEED_BMC_HOST2BMC_Q1 : ASPEED_BMC_HOST2BMC_Q2));
+
+	writel(BMC2HOST_INT_STS_DOORBELL | BMC2HOST_ENABLE_INTB,
+	       bmc_device->reg_base + ASPEED_BMC_BMC2HOST_STS);
+
+	return sizeof(u32);
+}
+
+static ssize_t aspeed_ast2700_queue_tx(struct file *filp, struct kobject *kobj,
+				       const struct bin_attribute *attr, char *buf, loff_t off,
+				       size_t count)
+{
+	struct aspeed_queue_message *queue = attr->private;
+	struct aspeed_bmc_device *bmc_device = queue->bmc_device;
+	int index = queue->index;
+	u32 tx_buff;
+	int ret;
+
+	if (count != sizeof(u32))
+		return -EINVAL;
+
+	ret = wait_event_interruptible(queue->tx_wait,
+				       !(readl(bmc_device->reg_base + ASPEED_BMC_BMC2HOST_STS) &
+				       ((index == QUEUE1) ? BMC2HOST_Q1_FULL : BMC2HOST_Q2_FULL)));
+	if (ret)
+		return -EINTR;
+
+	memcpy(&tx_buff, buf, 4);
+	writel(tx_buff, bmc_device->reg_base + ((index == QUEUE1) ? ASPEED_BMC_BMC2HOST_Q1 :
+								    ASPEED_BMC_BMC2HOST_Q2));
+
+	writel(BMC2HOST_INT_STS_DOORBELL | BMC2HOST_ENABLE_INTB,
+	       bmc_device->reg_base + ASPEED_BMC_BMC2HOST_STS);
+
+	return sizeof(u32);
+}
+
+/* AST2600 */
+static irqreturn_t aspeed_bmc_dev_pcie_isr(int irq, void *dev_id)
+{
+	struct aspeed_bmc_device *bmc_device = dev_id;
+
+	while (!(readl(bmc_device->reg_base + ASPEED_BMC_HOST2BMC_STS) & HOST2BMC_Q1_EMPTY))
+		readl(bmc_device->reg_base + ASPEED_BMC_HOST2BMC_Q1);
+
+	while (!(readl(bmc_device->reg_base + ASPEED_BMC_HOST2BMC_STS) & HOST2BMC_Q2_EMPTY))
+		readl(bmc_device->reg_base + ASPEED_BMC_HOST2BMC_Q2);
+
+	return IRQ_HANDLED;
+}
+
+static irqreturn_t aspeed_bmc_dev_isr(int irq, void *dev_id)
+{
+	struct aspeed_bmc_device *bmc_device = dev_id;
+	u32 host2bmc_q_sts = readl(bmc_device->reg_base + ASPEED_BMC_HOST2BMC_STS);
+
+	if (host2bmc_q_sts & HOST2BMC_INT_STS_DOORBELL)
+		writel(HOST2BMC_INT_STS_DOORBELL, bmc_device->reg_base + ASPEED_BMC_HOST2BMC_STS);
+
+	if (host2bmc_q_sts & HOST2BMC_ENABLE_INTB)
+		writel(HOST2BMC_ENABLE_INTB, bmc_device->reg_base + ASPEED_BMC_HOST2BMC_STS);
+
+	if (host2bmc_q_sts & HOST2BMC_Q1_FULL)
+		dev_info(bmc_device->dev, "Q1 Full\n");
+
+	if (host2bmc_q_sts & HOST2BMC_Q2_FULL)
+		dev_info(bmc_device->dev, "Q2 Full\n");
+
+	if (!(readl(bmc_device->reg_base + ASPEED_BMC_BMC2HOST_STS) & BMC2HOST_Q1_FULL))
+		wake_up_interruptible(&bmc_device->queue[QUEUE1].tx_wait);
+
+	if (!(readl(bmc_device->reg_base + ASPEED_BMC_HOST2BMC_STS) & HOST2BMC_Q1_EMPTY))
+		wake_up_interruptible(&bmc_device->queue[QUEUE1].rx_wait);
+
+	if (!(readl(bmc_device->reg_base + ASPEED_BMC_BMC2HOST_STS) & BMC2HOST_Q2_FULL))
+		wake_up_interruptible(&bmc_device->queue[QUEUE2].tx_wait);
+
+	if (!(readl(bmc_device->reg_base + ASPEED_BMC_HOST2BMC_STS) & HOST2BMC_Q2_EMPTY))
+		wake_up_interruptible(&bmc_device->queue[QUEUE2].rx_wait);
+
+	return IRQ_HANDLED;
+}
+
+static int aspeed_ast2600_init(struct platform_device *pdev)
+{
+	struct aspeed_bmc_device *bmc_device = platform_get_drvdata(pdev);
+	struct device *dev = &pdev->dev;
+	u32 pcie_config_ctl = SCU_PCIE_CONF_BMC_DEV_EN_IRQ |
+			      SCU_PCIE_CONF_BMC_DEV_EN_MMIO | SCU_PCIE_CONF_BMC_DEV_EN;
+	u32 scu_id;
+
+	bmc_device->scu = syscon_regmap_lookup_by_phandle(dev->of_node, "aspeed,scu");
+	if (IS_ERR(bmc_device->scu)) {
+		dev_err(&pdev->dev, "failed to find SCU regmap\n");
+		return PTR_ERR(bmc_device->scu);
+	}
+
+	if (bmc_device->pcie2lpc)
+		pcie_config_ctl |= SCU_PCIE_CONF_BMC_DEV_EN_E2L |
+				   SCU_PCIE_CONF_BMC_DEV_EN_LPC_DECODE;
+
+	regmap_update_bits(bmc_device->scu, ASPEED_SCU_PCIE_CONF_CTRL,
+			   pcie_config_ctl, pcie_config_ctl);
+
+	/* update class code to others as it is a MFD device */
+	regmap_write(bmc_device->scu, ASPEED_SCU_BMC_DEV_CLASS, 0xff000000);
+
+#ifdef SCU_TRIGGER_MSI
+	//SCUC24[17]: Enable PCI device 1 INTx/MSI from SCU560[15]. Will be added in next version
+	regmap_update_bits(bmc_device->scu, ASPEED_SCUC20, BIT(11) | BIT(14), BIT(11) | BIT(14));
+
+	regmap_read(bmc_device->scu, ASPEED_SCU04, &scu_id);
+	if (scu_id == AST2600A3_SCU04)
+		regmap_update_bits(bmc_device->scu, ASPEED_SCUC24,
+				   PCIDEV1_INTX_MSI_HOST2BMC_EN | MSI_ROUTING_MASK,
+				   PCIDEV1_INTX_MSI_HOST2BMC_EN | MSI_ROUTING_PCIe2LPC_PCIDEV1);
+	else
+		regmap_update_bits(bmc_device->scu, ASPEED_SCUC24,
+				   BIT(17) | BIT(14) | BIT(11), BIT(17) | BIT(14) | BIT(11));
+#else
+	//SCUC24[18]: Enable PCI device 1 INTx/MSI from Host-to-BMC controller.
+	regmap_update_bits(bmc_device->scu, 0xc24, BIT(18) | BIT(14), BIT(18) | BIT(14));
+#endif
+
+	writel((~(bmc_device->bmc_mem_size - 1) & 0xFFFFFFFF) | HOST2BMC_MEM_BAR_ENABLE,
+	       bmc_device->reg_base + ASPEED_BMC_MEM_BAR);
+	writel(bmc_device->bmc_mem_phy, bmc_device->reg_base + ASPEED_BMC_MEM_BAR_REMAP);
+
+	//Setting BMC to Host Q register
+	writel(BMC2HOST_Q2_FULL_UNMASK | BMC2HOST_Q1_FULL_UNMASK | BMC2HOST_ENABLE_INTB,
+	       bmc_device->reg_base + ASPEED_BMC_BMC2HOST_STS);
+	writel(HOST2BMC_Q2_FULL_UNMASK |  HOST2BMC_Q1_FULL_UNMASK | HOST2BMC_ENABLE_INTB,
+	       bmc_device->reg_base + ASPEED_BMC_HOST2BMC_STS);
+
+	return 0;
+}
+
+static int aspeed_ast2700_init(struct platform_device *pdev)
+{
+	struct aspeed_bmc_device *bmc_device = platform_get_drvdata(pdev);
+	struct device *dev = &pdev->dev;
+	u32 pcie_config_ctl;
+	u32 scu_id;
+	int i;
+
+	bmc_device->device = syscon_regmap_lookup_by_phandle(dev->of_node, "aspeed,device");
+	if (IS_ERR(bmc_device->device)) {
+		dev_err(&pdev->dev, "failed to find device regmap\n");
+		return PTR_ERR(bmc_device->device);
+	}
+
+	bmc_device->e2m = syscon_regmap_lookup_by_phandle(dev->of_node, "aspeed,e2m");
+	if (IS_ERR(bmc_device->e2m)) {
+		dev_err(&pdev->dev, "failed to find e2m regmap\n");
+		return PTR_ERR(bmc_device->e2m);
+	}
+
+	bmc_device->scu = syscon_regmap_lookup_by_phandle(dev->of_node, "aspeed,scu");
+	if (IS_ERR(bmc_device->scu)) {
+		dev_err(&pdev->dev, "failed to find SCU regmap\n");
+		return PTR_ERR(bmc_device->scu);
+	}
+
+	if (bmc_device->pcie2lpc) {
+		pcie_config_ctl = SCU_PCIE_CONF_BMC_DEV_EN_E2L |
+				  SCU_PCIE_CONF_BMC_DEV_EN_LPC_DECODE;
+		regmap_update_bits(bmc_device->scu, SCU0_PCIE_CONF_CTRL,
+				   pcie_config_ctl, pcie_config_ctl);
+	}
+
+	/* update class code to others as it is a MFD device */
+	regmap_write(bmc_device->device, 0x18, 0xff000027);
+
+	/* MSI */
+	regmap_update_bits(bmc_device->device, 0x74, GENMASK(7, 4), BIT(7) | (5 << 4));
+	/* EnPCIaMSI:BIT(25), EnPCIaIntA:BIT(17), EnPCIaMst:BIT(9), EnPCIaDev:BIT(1) */
+	regmap_read(bmc_device->scu, SCU0_REVISION_ID, &scu_id);
+	if (scu_id & REVISION_ID)
+		regmap_update_bits(bmc_device->device, 0x70,
+				   BIT(25) | BIT(17) | BIT(9) | BIT(1),
+				   BIT(25) | BIT(17) | BIT(9) | BIT(1));
+	else
+		/* Disable MSI[bit25] in ast2700A0 int only */
+		regmap_update_bits(bmc_device->device, 0x70,
+				   BIT(17) | BIT(9) | BIT(1),
+				   BIT(25) | BIT(17) | BIT(9) | BIT(1));
+
+	/* bar size check for 4k align */
+	for (i = 1; i < 16; i++) {
+		if ((bmc_device->bmc_mem_size / 4096) == (1 << (i - 1)))
+			break;
+	}
+	if (i == 16) {
+		i = 0;
+		dev_warn(bmc_device->dev,
+			 "Bar size not align for 4K : %dK\n", (u32)bmc_device->bmc_mem_size / 1024);
+	}
+
+	/*
+	 * BAR assign in scu
+	 * ((bar_mem / 4k) << 8) | per_size
+	 */
+	regmap_write(bmc_device->device, 0x1c, ((bmc_device->bmc_mem_phy) >> 4) | i);
+
+	if (bmc_device->id == 0)
+		/* Node 0 Bar 0 */
+		regmap_write(bmc_device->e2m, 0x108, ((bmc_device->bmc_mem_phy) >> 4) | i);
+	else
+		/* Node 1 Bar 0 */
+		regmap_write(bmc_device->e2m, 0x128, ((bmc_device->bmc_mem_phy) >> 4) | i);
+
+	/* Setting BMC to Host Q register */
+	writel(BMC2HOST_Q2_FULL_UNMASK | BMC2HOST_Q1_FULL_UNMASK | BMC2HOST_ENABLE_INTB,
+	       bmc_device->reg_base + ASPEED_BMC_BMC2HOST_STS);
+	writel(HOST2BMC_Q2_FULL_UNMASK | HOST2BMC_Q1_FULL_UNMASK | HOST2BMC_ENABLE_INTB,
+	       bmc_device->reg_base + ASPEED_BMC_HOST2BMC_STS);
+
+	return 0;
+}
+
+static int aspeed_bmc_device_setup_queue(struct platform_device *pdev)
+{
+	struct aspeed_bmc_device *bmc_device = platform_get_drvdata(pdev);
+	struct device *dev = &pdev->dev;
+	int ret, i;
+
+	for (i = 0; i < ASPEED_QUEUE_NUM; i++) {
+		struct aspeed_queue_message *queue = &bmc_device->queue[i];
+
+		init_waitqueue_head(&queue->tx_wait);
+		init_waitqueue_head(&queue->rx_wait);
+
+		sysfs_bin_attr_init(&queue->bin);
+
+		/* Queue name index starts from 1 */
+		queue->bin.attr.name =
+			devm_kasprintf(dev, GFP_KERNEL, "bmc-dev-queue%d", (i + 1));
+		queue->bin.attr.mode = 0600;
+		queue->bin.read = bmc_device->platform->queue_rx;
+		queue->bin.write = bmc_device->platform->queue_tx;
+		queue->bin.size = 4;
+		queue->bin.private = queue;
+
+		ret = sysfs_create_bin_file(&pdev->dev.kobj, &queue->bin);
+		if (ret) {
+			dev_err(dev, "error for bin%d file\n", i);
+			return ret;
+		}
+
+		queue->kn = kernfs_find_and_get(dev->kobj.sd, queue->bin.attr.name);
+		if (!queue->kn) {
+			sysfs_remove_bin_file(&dev->kobj, &queue->bin);
+			return ret;
+		}
+
+		queue->index = i;
+		queue->bmc_device = bmc_device;
+	}
+
+	return 0;
+}
+
+static int aspeed_bmc_device_setup_memory_mapping(struct platform_device *pdev)
+{
+	struct aspeed_bmc_device *bmc_device = platform_get_drvdata(pdev);
+	struct device *dev = &pdev->dev;
+	int ret;
+
+	bmc_device->miscdev.minor = MISC_DYNAMIC_MINOR;
+	bmc_device->miscdev.name = devm_kasprintf(dev, GFP_KERNEL, "bmc-device%d", bmc_device->id);
+	bmc_device->miscdev.fops = &aspeed_bmc_device_fops;
+	bmc_device->miscdev.parent = dev;
+	ret = misc_register(&bmc_device->miscdev);
+	if (ret) {
+		dev_err(dev, "Unable to register device\n");
+		return ret;
+	}
+
+	return 0;
+}
+
+static struct aspeed_platform ast2600_plaform = {
+	.init = aspeed_ast2600_init,
+	.queue_rx = aspeed_ast2600_queue_rx,
+	.queue_tx = aspeed_ast2600_queue_tx
+};
+
+static struct aspeed_platform ast2700_plaform = {
+	.init = aspeed_ast2700_init,
+	.queue_rx = aspeed_ast2700_queue_rx,
+	.queue_tx = aspeed_ast2700_queue_tx
+};
+
+static const struct of_device_id aspeed_bmc_device_of_matches[] = {
+	{ .compatible = "aspeed,ast2600-bmc-device", .data = &ast2600_plaform },
+	{ .compatible = "aspeed,ast2700-bmc-device", .data = &ast2700_plaform },
+	{},
+};
+MODULE_DEVICE_TABLE(of, aspeed_bmc_device_of_matches);
+
+static int aspeed_bmc_device_probe(struct platform_device *pdev)
+{
+	struct aspeed_bmc_device *bmc_device;
+	struct device *dev = &pdev->dev;
+	struct resource res;
+	const void *md = of_device_get_match_data(dev);
+	struct device_node *np;
+	int ret = 0, i;
+
+	if (!md)
+		return -ENODEV;
+
+	bmc_device = devm_kzalloc(&pdev->dev, sizeof(struct aspeed_bmc_device), GFP_KERNEL);
+	if (!bmc_device)
+		return -ENOMEM;
+	dev_set_drvdata(dev, bmc_device);
+
+	bmc_device->platform = md;
+
+	bmc_device->id = of_alias_get_id(dev->of_node, "bmcdev");
+	if (bmc_device->id < 0)
+		bmc_device->id = 0;
+
+	bmc_device->dev = dev;
+	bmc_device->reg_base = devm_platform_ioremap_resource(pdev, 0);
+	if (IS_ERR(bmc_device->reg_base))
+		return PTR_ERR(bmc_device->reg_base);
+
+	ret = of_reserved_mem_device_init(dev);
+	if (ret) {
+		dev_err(dev, "of_reserved_mem_device_init failed: %d\n", ret);
+		return ret;
+	}
+
+	ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64));
+	if (ret) {
+		dev_err(dev, "cannot set 64-bits DMA mask\n");
+		return ret;
+	}
+
+	np = of_parse_phandle(dev->of_node, "memory-region", 0);
+	if (!np || of_address_to_resource(np, 0, &res)) {
+		dev_err(dev, "Failed to find memory-region.\n");
+		return -ENOMEM;
+	}
+
+	of_node_put(np);
+
+	bmc_device->bmc_mem_size = resource_size(&res);
+	bmc_device->bmc_mem_cpu = dmam_alloc_coherent(dev, bmc_device->bmc_mem_size,
+						      &bmc_device->bmc_mem_phy, GFP_KERNEL);
+	if (!bmc_device->bmc_mem_cpu) {
+		dev_err(dev, "Failed to allocate BMC memory.\n");
+		return -ENOMEM;
+	}
+
+	bmc_device->irq = platform_get_irq(pdev, 0);
+	if (bmc_device->irq < 0) {
+		dev_err(&pdev->dev, "platform get of irq[=%d] failed!\n", bmc_device->irq);
+		return bmc_device->irq;
+	}
+
+	ret = devm_request_irq(&pdev->dev, bmc_device->irq, aspeed_bmc_dev_isr, 0,
+			       dev_name(&pdev->dev), bmc_device);
+	if (ret) {
+		dev_err(dev, "aspeed bmc device Unable to get IRQ");
+		return ret;
+	}
+
+	ret = aspeed_bmc_device_setup_queue(pdev);
+	if (ret) {
+		dev_err(dev, "Cannot setup queue message");
+		goto out;
+	}
+
+	ret = aspeed_bmc_device_setup_memory_mapping(pdev);
+	if (ret) {
+		dev_err(dev, "Cannot setup memory mapping misc");
+		goto out_free_queue;
+	}
+
+	if (of_property_read_bool(dev->of_node, "pcie2lpc"))
+		bmc_device->pcie2lpc = 1;
+
+	ret = bmc_device->platform->init(pdev);
+	if (ret) {
+		dev_err(dev, "Initialize bmc device failed\n");
+		goto out_free_misc;
+	}
+
+	bmc_device->pcie_irq =  platform_get_irq(pdev, 1);
+	if (bmc_device->pcie_irq < 0) {
+		dev_warn(&pdev->dev,
+			 "platform get of pcie irq[=%d] failed!\n", bmc_device->pcie_irq);
+	} else {
+		ret = devm_request_irq(&pdev->dev, bmc_device->pcie_irq,
+				       aspeed_bmc_dev_pcie_isr, IRQF_SHARED,
+				       dev_name(&pdev->dev), bmc_device);
+		if (ret < 0) {
+			dev_warn(dev, "Failed to request PCI-E IRQ %d.\n", ret);
+			bmc_device->pcie_irq = -1;
+		}
+	}
+
+	dev_info(dev, "aspeed bmc device: driver successfully loaded.\n");
+
+	return 0;
+
+out_free_misc:
+	misc_deregister(&bmc_device->miscdev);
+out_free_queue:
+	for (i = 0; i < ASPEED_QUEUE_NUM; i++)
+		sysfs_remove_bin_file(&pdev->dev.kobj, &bmc_device->queue[i].bin);
+out:
+	dev_warn(dev, "aspeed bmc device: driver init failed (ret=%d)!\n", ret);
+	return ret;
+}
+
+static void aspeed_bmc_device_remove(struct platform_device *pdev)
+{
+	struct aspeed_bmc_device *bmc_device = platform_get_drvdata(pdev);
+	int i;
+
+	for (i = 0; i < ASPEED_QUEUE_NUM; i++)
+		sysfs_remove_bin_file(&pdev->dev.kobj, &bmc_device->queue[i].bin);
+	misc_deregister(&bmc_device->miscdev);
+	devm_free_irq(&pdev->dev, bmc_device->irq, bmc_device);
+	devm_free_irq(&pdev->dev, bmc_device->pcie_irq, bmc_device);
+
+	devm_kfree(&pdev->dev, bmc_device);
+}
+
+static struct platform_driver aspeed_bmc_device_driver = {
+	.probe		= aspeed_bmc_device_probe,
+	.remove		= aspeed_bmc_device_remove,
+	.driver		= {
+		.name	= KBUILD_MODNAME,
+		.of_match_table = aspeed_bmc_device_of_matches,
+	},
+};
+
+module_platform_driver(aspeed_bmc_device_driver);
+
+MODULE_AUTHOR("Ryan Chen <ryan_chen@aspeedtech.com>");
+MODULE_DESCRIPTION("ASPEED BMC DEVICE Driver");
+MODULE_LICENSE("GPL");
\ No newline at end of file
-- 
2.51.2

[PATCH v1 2/2] soc: aspeed: add host-side PCIe BMC device driver

[Grégoire Layet]

Taken from ASPEED 6.18 Kernel SDK

Add support for PCIe communication BMC<->host.
This add Host side driver.

Signed-off-by: Jacky Chou <jacky_chou@aspeedtech.com>
Signed-off-by: aspeedyh <yh_chung@aspeedtech.com>
Signed-off-by: Grégoire Layet <gregoire.layet@9elements.com>
Tested-by: Grégoire Layet <gregoire.layet@9elements.com>
---
 drivers/soc/aspeed/Kconfig               |  13 +
 drivers/soc/aspeed/Makefile              |   1 +
 drivers/soc/aspeed/aspeed-host-bmc-dev.c | 664 +++++++++++++++++++++++
 3 files changed, 678 insertions(+)
 create mode 100644 drivers/soc/aspeed/aspeed-host-bmc-dev.c

diff --git a/drivers/soc/aspeed/Kconfig b/drivers/soc/aspeed/Kconfig
index 341728df07b3..e8670dfef1e6 100644
--- a/drivers/soc/aspeed/Kconfig
+++ b/drivers/soc/aspeed/Kconfig
@@ -13,6 +13,19 @@ config ASPEED_BMC_DEV
 	  providing a shared-memory BAR, host-to-BMC and BMC-to-host
 	  message queues with doorbell interrupts and PCIe-to-LPC bridge.
 
+config ASPEED_HOST_BMC_DEV
+	tristate "ASPEED Host BMC Device"
+	depends on PCI
+	depends on SERIAL_8250
+	help
+	  Enable support for the ASPEED AST2600/AST2700 BMC Device on the Host.
+	  This configure the PCIe and setup:
+	  - Two 8250 compatible VUART ports.
+	  - A character device exposing the BMC's shared memory
+	    region for host<->BMC data exchange.
+	  - A mailbox interrupt path and BMC message queue handler for
+		doorbell-style host<->BMC signaling.
+
 config ASPEED_LPC_CTRL
 	tristate "ASPEED LPC firmware cycle control"
 	select REGMAP
diff --git a/drivers/soc/aspeed/Makefile b/drivers/soc/aspeed/Makefile
index fab0d247df66..3fd3f6d8d36e 100644
--- a/drivers/soc/aspeed/Makefile
+++ b/drivers/soc/aspeed/Makefile
@@ -1,5 +1,6 @@
 # SPDX-License-Identifier: GPL-2.0-only
 obj-$(CONFIG_ASPEED_BMC_DEV)		+= aspeed-bmc-dev.o
+obj-$(CONFIG_ASPEED_HOST_BMC_DEV)	+= aspeed-host-bmc-dev.o
 obj-$(CONFIG_ASPEED_LPC_CTRL)		+= aspeed-lpc-ctrl.o
 obj-$(CONFIG_ASPEED_LPC_SNOOP)		+= aspeed-lpc-snoop.o
 obj-$(CONFIG_ASPEED_UART_ROUTING)	+= aspeed-uart-routing.o
diff --git a/drivers/soc/aspeed/aspeed-host-bmc-dev.c b/drivers/soc/aspeed/aspeed-host-bmc-dev.c
new file mode 100644
index 000000000000..9e6f1d39f18a
--- /dev/null
+++ b/drivers/soc/aspeed/aspeed-host-bmc-dev.c
@@ -0,0 +1,664 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+// Copyright (C) ASPEED Technology Inc.
+
+#include <linux/init.h>
+#include <linux/version.h>
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/errno.h>
+#include <linux/pci.h>
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <linux/interrupt.h>
+#include <linux/wait.h>
+#include <linux/workqueue.h>
+#include <linux/miscdevice.h>
+#include <linux/module.h>
+#include <linux/serial_core.h>
+#include <linux/serial_8250.h>
+#include <linux/poll.h>
+#include <linux/bitfield.h>
+
+#include <linux/if_arp.h>
+#include <linux/skbuff.h>
+#include <linux/mctp.h>
+#include <net/mctp.h>
+#include <net/pkt_sched.h>
+
+#define PCI_BMC_HOST2BMC_Q1 0x30000
+#define PCI_BMC_HOST2BMC_Q2 0x30010
+#define PCI_BMC_BMC2HOST_Q1 0x30020
+#define PCI_BMC_BMC2HOST_Q2 0x30030
+#define PCI_BMC_BMC2HOST_STS 0x30040
+#define BMC2HOST_INT_STS_DOORBELL BIT(31)
+#define BMC2HOST_ENABLE_INTB BIT(30)
+
+#define BMC2HOST_Q1_FULL BIT(27)
+#define BMC2HOST_Q1_EMPTY BIT(26)
+#define BMC2HOST_Q2_FULL BIT(25)
+#define BMC2HOST_Q2_EMPTY BIT(24)
+#define BMC2HOST_Q1_FULL_UNMASK BIT(23)
+#define BMC2HOST_Q1_EMPTY_UNMASK BIT(22)
+#define BMC2HOST_Q2_FULL_UNMASK BIT(21)
+#define BMC2HOST_Q2_EMPTY_UNMASK BIT(20)
+
+#define PCI_BMC_HOST2BMC_STS 0x30044
+#define HOST2BMC_INT_STS_DOORBELL BIT(31)
+#define HOST2BMC_ENABLE_INTB BIT(30)
+
+#define HOST2BMC_Q1_FULL BIT(27)
+#define HOST2BMC_Q1_EMPTY BIT(26)
+#define HOST2BMC_Q2_FULL BIT(25)
+#define HOST2BMC_Q2_EMPTY BIT(24)
+#define HOST2BMC_Q1_FULL_UNMASK BIT(23)
+#define HOST2BMC_Q1_EMPTY_UNMASK BIT(22)
+#define HOST2BMC_Q2_FULL_UNMASK BIT(21)
+#define HOST2BMC_Q2_EMPTY_UNMASK BIT(20)
+
+static DEFINE_IDA(bmc_device_ida);
+
+#define VUART_MAX_PARMS 2
+#define ASPEED_QUEUE_NUM 2
+#define MAX_MSI_NUM 8
+
+enum aspeed_platform_id {
+	ASPEED,
+};
+
+enum queue_index {
+	QUEUE1 = 0,
+	QUEUE2,
+};
+
+enum msi_index {
+	BMC_MSI,
+	MBX_MSI,
+	VUART0_MSI,
+	VUART1_MSI,
+};
+
+/* Match msi_index */
+static int ast2600_msi_idx_table[MAX_MSI_NUM] = { 4, 21, 16, 15 };
+static int ast2700_soc0_msi_idx_table[MAX_MSI_NUM] = { 0, 11, 6, 5 };
+
+struct aspeed_platform {
+	int (*setup)(struct pci_dev *pdev);
+};
+
+struct aspeed_queue_message {
+	/* Queue waiters for idle engine */
+	wait_queue_head_t tx_wait;
+	wait_queue_head_t rx_wait;
+	struct kernfs_node *kn;
+	struct bin_attribute bin;
+	int index;
+	struct aspeed_pci_bmc_dev *pci_bmc_device;
+};
+
+struct aspeed_pci_bmc_dev {
+	struct device *dev;
+	struct miscdevice miscdev;
+	struct aspeed_platform *platform;
+	kernel_ulong_t driver_data;
+	int id;
+
+	unsigned long mem_bar_base;
+	unsigned long mem_bar_size;
+	void __iomem *mem_bar_reg;
+
+	unsigned long message_bar_base;
+	unsigned long message_bar_size;
+	void __iomem *msg_bar_reg;
+
+	void __iomem *pcie_sio_decode_addr;
+
+	struct aspeed_queue_message queue[ASPEED_QUEUE_NUM];
+
+	void __iomem *sio_mbox_reg;
+	struct uart_8250_port uart[VUART_MAX_PARMS];
+	int uart_line[VUART_MAX_PARMS];
+
+	/* Interrupt
+	 * The index of array is using to enum msi_index
+	 */
+	int *msi_idx_table;
+};
+
+#define PCIE_DEVICE_SIO_ADDR (0x2E * 4)
+#define BMC_MULTI_MSI 32
+
+#define DRIVER_NAME "aspeed-host-bmc-dev"
+
+static struct aspeed_pci_bmc_dev *file_aspeed_bmc_device(struct file *file)
+{
+	return container_of(file->private_data, struct aspeed_pci_bmc_dev,
+			    miscdev);
+}
+
+static int aspeed_pci_bmc_dev_mmap(struct file *file,
+				   struct vm_area_struct *vma)
+{
+	struct aspeed_pci_bmc_dev *pci_bmc_dev = file_aspeed_bmc_device(file);
+	unsigned long vsize = vma->vm_end - vma->vm_start;
+	pgprot_t prot = vma->vm_page_prot;
+
+	if (vma->vm_pgoff + vsize > pci_bmc_dev->mem_bar_base + 0x100000)
+		return -EINVAL;
+
+	prot = pgprot_noncached(prot);
+
+	if (remap_pfn_range(vma, vma->vm_start,
+			    (pci_bmc_dev->mem_bar_base >> PAGE_SHIFT) +
+				    vma->vm_pgoff,
+			    vsize, prot))
+		return -EAGAIN;
+
+	return 0;
+}
+
+static const struct file_operations aspeed_pci_bmc_dev_fops = {
+	.owner = THIS_MODULE,
+	.mmap = aspeed_pci_bmc_dev_mmap,
+};
+
+static ssize_t aspeed_queue_rx(struct file *filp, struct kobject *kobj,
+			       const struct bin_attribute *attr, char *buf,
+			       loff_t off, size_t count)
+{
+	struct aspeed_queue_message *queue = attr->private;
+	struct aspeed_pci_bmc_dev *pci_bmc_device = queue->pci_bmc_device;
+	int index = queue->index;
+	u32 *data = (u32 *)buf;
+	int ret;
+
+	ret = wait_event_interruptible(
+		queue->rx_wait,
+		!(readl(pci_bmc_device->msg_bar_reg + PCI_BMC_BMC2HOST_STS) &
+		  ((index == QUEUE1) ? BMC2HOST_Q1_EMPTY : BMC2HOST_Q2_EMPTY)));
+	if (ret)
+		return -EINTR;
+
+	data[0] = readl(pci_bmc_device->msg_bar_reg +
+			((index == QUEUE1) ? PCI_BMC_BMC2HOST_Q1 :
+					     PCI_BMC_BMC2HOST_Q2));
+
+	writel(HOST2BMC_INT_STS_DOORBELL | HOST2BMC_ENABLE_INTB,
+	       pci_bmc_device->msg_bar_reg + PCI_BMC_HOST2BMC_STS);
+
+	return sizeof(u32);
+}
+
+static ssize_t aspeed_queue_tx(struct file *filp, struct kobject *kobj,
+			       const struct bin_attribute *attr, char *buf,
+			       loff_t off, size_t count)
+{
+	struct aspeed_queue_message *queue = attr->private;
+	struct aspeed_pci_bmc_dev *pci_bmc_device = queue->pci_bmc_device;
+	int index = queue->index;
+	u32 tx_buff;
+	int ret;
+
+	if (count != sizeof(u32))
+		return -EINVAL;
+
+	ret = wait_event_interruptible(
+		queue->tx_wait,
+		!(readl(pci_bmc_device->msg_bar_reg + PCI_BMC_HOST2BMC_STS) &
+		  ((index == QUEUE1) ? HOST2BMC_Q1_FULL : HOST2BMC_Q2_FULL)));
+	if (ret)
+		return -EINTR;
+
+	memcpy(&tx_buff, buf, 4);
+	writel(tx_buff, pci_bmc_device->msg_bar_reg +
+				((index == QUEUE1) ? PCI_BMC_HOST2BMC_Q1 :
+						     PCI_BMC_HOST2BMC_Q2));
+	//trigger to host
+	writel(HOST2BMC_INT_STS_DOORBELL | HOST2BMC_ENABLE_INTB,
+	       pci_bmc_device->msg_bar_reg + PCI_BMC_HOST2BMC_STS);
+
+	return sizeof(u32);
+}
+
+static irqreturn_t aspeed_pci_host_bmc_device_interrupt(int irq, void *dev_id)
+{
+	struct aspeed_pci_bmc_dev *pci_bmc_device = dev_id;
+	u32 bmc2host_q_sts =
+		readl(pci_bmc_device->msg_bar_reg + PCI_BMC_BMC2HOST_STS);
+
+	if (bmc2host_q_sts & BMC2HOST_INT_STS_DOORBELL)
+		writel(BMC2HOST_INT_STS_DOORBELL,
+		       pci_bmc_device->msg_bar_reg + PCI_BMC_BMC2HOST_STS);
+
+	if (bmc2host_q_sts & BMC2HOST_ENABLE_INTB)
+		writel(BMC2HOST_ENABLE_INTB,
+		       pci_bmc_device->msg_bar_reg + PCI_BMC_BMC2HOST_STS);
+
+	if (bmc2host_q_sts & BMC2HOST_Q1_FULL)
+		dev_info(pci_bmc_device->dev, "Q1 Full\n");
+
+	if (bmc2host_q_sts & BMC2HOST_Q2_FULL)
+		dev_info(pci_bmc_device->dev, "Q2 Full\n");
+
+	//check q1
+	if (!(readl(pci_bmc_device->msg_bar_reg + PCI_BMC_HOST2BMC_STS) &
+	      HOST2BMC_Q1_FULL))
+		wake_up_interruptible(&pci_bmc_device->queue[QUEUE1].tx_wait);
+
+	if (!(readl(pci_bmc_device->msg_bar_reg + PCI_BMC_BMC2HOST_STS) &
+	      BMC2HOST_Q1_EMPTY))
+		wake_up_interruptible(&pci_bmc_device->queue[QUEUE1].rx_wait);
+	//chech q2
+	if (!(readl(pci_bmc_device->msg_bar_reg + PCI_BMC_HOST2BMC_STS) &
+	      HOST2BMC_Q2_FULL))
+		wake_up_interruptible(&pci_bmc_device->queue[QUEUE2].tx_wait);
+
+	if (!(readl(pci_bmc_device->msg_bar_reg + PCI_BMC_BMC2HOST_STS) &
+	      BMC2HOST_Q2_EMPTY))
+		wake_up_interruptible(&pci_bmc_device->queue[QUEUE2].rx_wait);
+
+	return IRQ_HANDLED;
+}
+
+static irqreturn_t aspeed_pci_host_mbox_interrupt(int irq, void *dev_id)
+{
+	struct aspeed_pci_bmc_dev *pci_bmc_device = dev_id;
+	u32 isr = readl(pci_bmc_device->sio_mbox_reg + 0x94);
+
+	if (isr & BIT(7))
+		writel(BIT(7), pci_bmc_device->sio_mbox_reg + 0x94);
+
+	return IRQ_HANDLED;
+}
+
+static void aspeed_pci_setup_irq_resource(struct pci_dev *pdev)
+{
+	struct aspeed_pci_bmc_dev *pci_bmc_dev = pci_get_drvdata(pdev);
+
+	/* Assign static msi index table by platform */
+	if (pdev->revision == 0x27) {
+		pci_bmc_dev->msi_idx_table = ast2700_soc0_msi_idx_table;
+	} else {
+		pci_bmc_dev->msi_idx_table = ast2600_msi_idx_table;
+	}
+
+	if (pci_alloc_irq_vectors(pdev, 1, BMC_MULTI_MSI,
+				  PCI_IRQ_INTX | PCI_IRQ_MSI) <= 1)
+		/* Set all msi index to the first vector */
+		memset(pci_bmc_dev->msi_idx_table, 0,
+		       sizeof(int) * MAX_MSI_NUM);
+}
+
+static int aspeed_pci_bmc_device_setup_queue(struct pci_dev *pdev)
+{
+	struct aspeed_pci_bmc_dev *pci_bmc_device = pci_get_drvdata(pdev);
+	struct device *dev = &pdev->dev;
+	int ret, i;
+
+	for (i = 0; i < ASPEED_QUEUE_NUM; i++) {
+		struct aspeed_queue_message *queue = &pci_bmc_device->queue[i];
+
+		init_waitqueue_head(&queue->tx_wait);
+		init_waitqueue_head(&queue->rx_wait);
+
+		sysfs_bin_attr_init(&queue->bin);
+
+		/* Queue name index starts from 1 */
+		queue->bin.attr.name = devm_kasprintf(
+			dev, GFP_KERNEL, "pci-bmc-dev-queue%d", (i + 1));
+		queue->bin.attr.mode = 0600;
+		queue->bin.read = aspeed_queue_rx;
+		queue->bin.write = aspeed_queue_tx;
+		queue->bin.size = 4;
+		queue->bin.private = queue;
+
+		ret = sysfs_create_bin_file(&pdev->dev.kobj, &queue->bin);
+		if (ret) {
+			dev_err(dev, "error for bin%d file\n", i);
+			return ret;
+		}
+
+		queue->kn =
+			kernfs_find_and_get(dev->kobj.sd, queue->bin.attr.name);
+		if (!queue->kn) {
+			sysfs_remove_bin_file(&dev->kobj, &queue->bin);
+			return ret;
+		}
+
+		queue->index = i;
+		queue->pci_bmc_device = pci_bmc_device;
+	}
+
+	return 0;
+}
+
+static int aspeed_pci_bmc_device_setup_vuart(struct pci_dev *pdev)
+{
+	struct aspeed_pci_bmc_dev *pci_bmc_dev = pci_get_drvdata(pdev);
+	struct device *dev = &pdev->dev;
+	u16 vuart_ioport;
+	int ret, i;
+
+	for (i = 0; i < VUART_MAX_PARMS; i++) {
+		/* Assign the line to non-exist device */
+		pci_bmc_dev->uart_line[i] = -ENOENT;
+		vuart_ioport = 0x3F8 - (i * 0x100);
+		pci_bmc_dev->uart[i].port.flags =
+			UPF_SKIP_TEST | UPF_BOOT_AUTOCONF | UPF_SHARE_IRQ;
+		pci_bmc_dev->uart[i].port.uartclk = 115200 * 16;
+		pci_bmc_dev->uart[i].port.irq = pci_irq_vector(
+			pdev, pci_bmc_dev->msi_idx_table[VUART0_MSI + i]);
+		pci_bmc_dev->uart[i].port.dev = dev;
+		pci_bmc_dev->uart[i].port.iotype = UPIO_MEM32;
+		pci_bmc_dev->uart[i].port.iobase = 0;
+		pci_bmc_dev->uart[i].port.mapbase =
+			pci_bmc_dev->message_bar_base + (vuart_ioport << 2);
+		pci_bmc_dev->uart[i].port.membase = 0;
+		pci_bmc_dev->uart[i].port.type = PORT_16550A;
+		pci_bmc_dev->uart[i].port.flags |=
+			(UPF_IOREMAP | UPF_FIXED_PORT | UPF_FIXED_TYPE);
+		pci_bmc_dev->uart[i].port.regshift = 2;
+		ret = serial8250_register_8250_port(&pci_bmc_dev->uart[i]);
+		if (ret < 0) {
+			dev_err_probe(dev, ret, "Can't setup PCIe VUART\n");
+			return ret;
+		}
+		pci_bmc_dev->uart_line[i] = ret;
+	}
+	return 0;
+}
+
+static int aspeed_pci_bmc_device_setup_memory_mapping(struct pci_dev *pdev)
+{
+	struct aspeed_pci_bmc_dev *pci_bmc_dev = pci_get_drvdata(pdev);
+	struct device *dev = &pdev->dev;
+	int ret;
+
+	pci_bmc_dev->miscdev.minor = MISC_DYNAMIC_MINOR;
+	pci_bmc_dev->miscdev.name = devm_kasprintf(
+		dev, GFP_KERNEL, "%s%d", DRIVER_NAME, pci_bmc_dev->id);
+	pci_bmc_dev->miscdev.fops = &aspeed_pci_bmc_dev_fops;
+	pci_bmc_dev->miscdev.parent = dev;
+
+	ret = misc_register(&pci_bmc_dev->miscdev);
+	if (ret) {
+		pr_err("host bmc register fail %d\n", ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+static int aspeed_pci_bmc_device_setup_mbox(struct pci_dev *pdev)
+{
+	struct aspeed_pci_bmc_dev *pci_bmc_dev = pci_get_drvdata(pdev);
+	struct device *dev = &pdev->dev;
+	int ret;
+
+	/* setup mbox */
+	pci_bmc_dev->pcie_sio_decode_addr =
+		pci_bmc_dev->msg_bar_reg + PCIE_DEVICE_SIO_ADDR;
+	writel(0xaa, pci_bmc_dev->pcie_sio_decode_addr);
+	writel(0xa5, pci_bmc_dev->pcie_sio_decode_addr);
+	writel(0xa5, pci_bmc_dev->pcie_sio_decode_addr);
+	writel(0x07, pci_bmc_dev->pcie_sio_decode_addr);
+	writel(0x0e, pci_bmc_dev->pcie_sio_decode_addr + 0x04);
+	/* disable */
+	writel(0x30, pci_bmc_dev->pcie_sio_decode_addr);
+	writel(0x00, pci_bmc_dev->pcie_sio_decode_addr + 0x04);
+	/* set decode address 0x100 */
+	writel(0x60, pci_bmc_dev->pcie_sio_decode_addr);
+	writel(0x01, pci_bmc_dev->pcie_sio_decode_addr + 0x04);
+	writel(0x61, pci_bmc_dev->pcie_sio_decode_addr);
+	writel(0x00, pci_bmc_dev->pcie_sio_decode_addr + 0x04);
+	/* enable */
+	writel(0x30, pci_bmc_dev->pcie_sio_decode_addr);
+	writel(0x01, pci_bmc_dev->pcie_sio_decode_addr + 0x04);
+	pci_bmc_dev->sio_mbox_reg = pci_bmc_dev->msg_bar_reg + 0x400;
+
+	ret = devm_request_irq(
+		dev, pci_irq_vector(pdev, pci_bmc_dev->msi_idx_table[MBX_MSI]),
+		aspeed_pci_host_mbox_interrupt, IRQF_SHARED,
+		devm_kasprintf(dev, GFP_KERNEL, "aspeed-sio-mbox%d",
+			       pci_bmc_dev->id),
+		pci_bmc_dev);
+	if (ret) {
+		pr_err("host bmc device Unable to get IRQ %d\n", ret);
+		return ret;
+	}
+
+	return 0;
+}
+
+static void aspeed_pci_host_bmc_device_release_queue(struct pci_dev *pdev)
+{
+	struct aspeed_pci_bmc_dev *pci_bmc_dev = pci_get_drvdata(pdev);
+	int i;
+
+	for (i = 0; i < ASPEED_QUEUE_NUM; i++)
+		sysfs_remove_bin_file(&pdev->dev.kobj,
+				      &pci_bmc_dev->queue[i].bin);
+}
+
+static void aspeed_pci_host_bmc_device_release_vuart(struct pci_dev *pdev)
+{
+	struct aspeed_pci_bmc_dev *pci_bmc_dev = pci_get_drvdata(pdev);
+	int i;
+
+	for (i = 0; i < VUART_MAX_PARMS; i++) {
+		if (pci_bmc_dev->uart_line[i] >= 0)
+			serial8250_unregister_port(pci_bmc_dev->uart_line[i]);
+	}
+}
+
+static void
+aspeed_pci_host_bmc_device_release_memory_mapping(struct pci_dev *pdev)
+{
+	struct aspeed_pci_bmc_dev *pci_bmc_dev = pci_get_drvdata(pdev);
+
+	if (!list_empty(&pci_bmc_dev->miscdev.list))
+		misc_deregister(&pci_bmc_dev->miscdev);
+}
+
+static int aspeed_pci_host_setup(struct pci_dev *pdev)
+{
+	struct aspeed_pci_bmc_dev *pci_bmc_dev = pci_get_drvdata(pdev);
+	int rc = 0;
+
+	/* Get share memory BAR */
+	pci_bmc_dev->mem_bar_base = pci_resource_start(pdev, 0);
+	pci_bmc_dev->mem_bar_size = pci_resource_len(pdev, 0);
+	pci_bmc_dev->mem_bar_reg = pci_ioremap_bar(pdev, 0);
+	if (!pci_bmc_dev->mem_bar_reg)
+		return -ENOMEM;
+
+	/* Get Message BAR */
+	pci_bmc_dev->message_bar_base = pci_resource_start(pdev, 1);
+	pci_bmc_dev->message_bar_size = pci_resource_len(pdev, 1);
+	pci_bmc_dev->msg_bar_reg = pci_ioremap_bar(pdev, 1);
+	if (!pci_bmc_dev->msg_bar_reg) {
+		rc = -ENOMEM;
+		goto out_free0;
+	}
+
+	/* AST2600 ERRTA40: dummy read */
+	if (pdev->revision < 0x27)
+		(void)__raw_readl((void __iomem *)pci_bmc_dev->msg_bar_reg);
+
+	rc = aspeed_pci_bmc_device_setup_queue(pdev);
+	if (rc) {
+		pr_err("Cannot setup Queue Message");
+		goto out_free1;
+	}
+
+	rc = aspeed_pci_bmc_device_setup_memory_mapping(pdev);
+	if (rc) {
+		pr_err("Cannot setup Memory Mapping");
+		goto out_free_queue;
+	}
+
+	rc = aspeed_pci_bmc_device_setup_mbox(pdev);
+	if (rc) {
+		pr_err("Cannot setup Mailnbox");
+		goto out_free_mmapping;
+	}
+
+	rc = aspeed_pci_bmc_device_setup_vuart(pdev);
+	if (rc) {
+		pr_err("Cannot setup Virtual UART");
+		goto out_free_mbox;
+	}
+
+	rc = devm_request_irq(
+		&pdev->dev,
+		pci_irq_vector(pdev, pci_bmc_dev->msi_idx_table[BMC_MSI]),
+		aspeed_pci_host_bmc_device_interrupt, IRQF_SHARED,
+		pci_bmc_dev->miscdev.name, pci_bmc_dev);
+	if (rc) {
+		pr_err("Get BMC DEVICE IRQ failed. (err=%d)\n", rc);
+		goto out_free_uart;
+	}
+
+	return 0;
+
+out_free_uart:
+	aspeed_pci_host_bmc_device_release_vuart(pdev);
+out_free_mbox:
+	devm_free_irq(&pdev->dev,
+		      pci_irq_vector(pdev, pci_bmc_dev->msi_idx_table[MBX_MSI]),
+		      pci_bmc_dev);
+out_free_mmapping:
+	aspeed_pci_host_bmc_device_release_memory_mapping(pdev);
+out_free_queue:
+	aspeed_pci_host_bmc_device_release_queue(pdev);
+out_free1:
+	pci_iounmap(pdev, pci_bmc_dev->msg_bar_reg);
+out_free0:
+	pci_iounmap(pdev, pci_bmc_dev->mem_bar_reg);
+
+	pci_release_regions(pdev);
+	return rc;
+}
+
+static struct aspeed_platform aspeed_pcie_host[] = {
+	{ .setup = aspeed_pci_host_setup },
+	{ 0 }
+};
+
+static int aspeed_pci_host_bmc_device_probe(struct pci_dev *pdev,
+					    const struct pci_device_id *ent)
+{
+	struct aspeed_pci_bmc_dev *pci_bmc_dev;
+	int rc = 0;
+
+	pr_info("ASPEED BMC PCI ID %04x:%04x, IRQ=%u\n", pdev->vendor,
+		pdev->device, pdev->irq);
+
+	pci_bmc_dev =
+		devm_kzalloc(&pdev->dev, sizeof(*pci_bmc_dev), GFP_KERNEL);
+	if (!pci_bmc_dev)
+		return -ENOMEM;
+
+	/* Get platform id */
+	pci_bmc_dev->driver_data = ent->driver_data;
+	pci_bmc_dev->platform = &aspeed_pcie_host[ent->driver_data];
+
+	pci_bmc_dev->id = ida_alloc(&bmc_device_ida, GFP_KERNEL);
+	if (pci_bmc_dev->id < 0)
+		return pci_bmc_dev->id;
+
+	rc = pci_enable_device(pdev);
+	if (rc) {
+		dev_err(&pdev->dev, "pci_enable_device() returned error %d\n",
+			rc);
+		return rc;
+	}
+
+	pci_set_master(pdev);
+	pci_set_drvdata(pdev, pci_bmc_dev);
+
+	/* Prepare IRQ resource */
+	aspeed_pci_setup_irq_resource(pdev);
+
+	/* Setup BMC PCI device */
+	rc = pci_bmc_dev->platform->setup(pdev);
+	if (rc) {
+		dev_err(&pdev->dev,
+			"ASPEED PCIe Host device returned error %d\n", rc);
+		pci_free_irq_vectors(pdev);
+		pci_disable_device(pdev);
+		return rc;
+	}
+
+	return 0;
+}
+
+static void aspeed_pci_host_bmc_device_remove(struct pci_dev *pdev)
+{
+	struct aspeed_pci_bmc_dev *pci_bmc_dev = pci_get_drvdata(pdev);
+
+	if (pci_bmc_dev->driver_data == ASPEED) {
+		aspeed_pci_host_bmc_device_release_queue(pdev);
+		aspeed_pci_host_bmc_device_release_memory_mapping(pdev);
+		aspeed_pci_host_bmc_device_release_vuart(pdev);
+
+		devm_free_irq(
+			&pdev->dev,
+			pci_irq_vector(pdev,
+				       pci_bmc_dev->msi_idx_table[BMC_MSI]),
+			pci_bmc_dev);
+		devm_free_irq(
+			&pdev->dev,
+			pci_irq_vector(pdev,
+				       pci_bmc_dev->msi_idx_table[MBX_MSI]),
+			pci_bmc_dev);
+	}
+
+	ida_free(&bmc_device_ida, pci_bmc_dev->id);
+
+	pci_iounmap(pdev, pci_bmc_dev->msg_bar_reg);
+	pci_iounmap(pdev, pci_bmc_dev->mem_bar_reg);
+
+	pci_free_irq_vectors(pdev);
+	pci_release_regions(pdev);
+	pci_disable_device(pdev);
+}
+
+/**
+ * This table holds the list of (VendorID,DeviceID) supported by this driver
+ *
+ */
+static struct pci_device_id aspeed_host_bmc_dev_pci_ids[] = {
+	/* ASPEED BMC Device */
+	{ PCI_DEVICE(0x1A03, 0x2402), .class = 0xFF0000, .class_mask = 0xFFFF00,
+	  .driver_data = ASPEED },
+	{
+		0,
+	}
+};
+
+MODULE_DEVICE_TABLE(pci, aspeed_host_bmc_dev_pci_ids);
+
+static struct pci_driver aspeed_host_bmc_dev_driver = {
+	.name = DRIVER_NAME,
+	.id_table = aspeed_host_bmc_dev_pci_ids,
+	.probe = aspeed_pci_host_bmc_device_probe,
+	.remove = aspeed_pci_host_bmc_device_remove,
+};
+
+static int __init aspeed_host_bmc_device_init(void)
+{
+	return pci_register_driver(&aspeed_host_bmc_dev_driver);
+}
+
+static void aspeed_host_bmc_device_exit(void)
+{
+	/* unregister pci driver */
+	pci_unregister_driver(&aspeed_host_bmc_dev_driver);
+}
+
+late_initcall(aspeed_host_bmc_device_init);
+module_exit(aspeed_host_bmc_device_exit);
+
+MODULE_AUTHOR("Ryan Chen <ryan_chen@aspeedtech.com>");
+MODULE_DESCRIPTION("ASPEED Host BMC DEVICE Driver");
+MODULE_LICENSE("GPL");
\ No newline at end of file
-- 
2.51.2

Re: [PATCH v1 2/2] soc: aspeed: add host-side PCIe BMC device driver

[Andrew Lunn]

> +static int aspeed_pci_bmc_device_setup_vuart(struct pci_dev *pdev)
> +{
> +	struct aspeed_pci_bmc_dev *pci_bmc_dev = pci_get_drvdata(pdev);
> +	struct device *dev = &pdev->dev;
> +	u16 vuart_ioport;
> +	int ret, i;
> +
> +	for (i = 0; i < VUART_MAX_PARMS; i++) {
> +		/* Assign the line to non-exist device */
> +		pci_bmc_dev->uart_line[i] = -ENOENT;
> +		vuart_ioport = 0x3F8 - (i * 0x100);
> +		pci_bmc_dev->uart[i].port.flags =
> +			UPF_SKIP_TEST | UPF_BOOT_AUTOCONF | UPF_SHARE_IRQ;
> +		pci_bmc_dev->uart[i].port.uartclk = 115200 * 16;
> +		pci_bmc_dev->uart[i].port.irq = pci_irq_vector(
> +			pdev, pci_bmc_dev->msi_idx_table[VUART0_MSI + i]);
> +		pci_bmc_dev->uart[i].port.dev = dev;
> +		pci_bmc_dev->uart[i].port.iotype = UPIO_MEM32;
> +		pci_bmc_dev->uart[i].port.iobase = 0;

How virtual is this? Is this directly accessing the hardware via
shared memory? Or is there software on the BMC which traps these
reads/writes and responds?

But first we need to decide if this is the correct architecture. The
alternative is rpmsg or virtio. There is already tty/rpmsg_tty.c. A
gpio driver over rpmsg is on the way. I2C has been discussed. There
are plenty of virtio drivers, console, i2c, spi, gpio. virtio has
higher performance, but also requires more memory. rpmsg is more
suited to slower devices where memory is tight.

Either of these seem like a better way to expose resources of the BMC
to the host.

       Andrew