From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ale.deltatee.com ([207.54.116.67]:40064 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727345AbeH3W5l (ORCPT ); Thu, 30 Aug 2018 18:57:41 -0400 From: Logan Gunthorpe To: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org, linux-nvdimm@lists.01.org, linux-block@vger.kernel.org Cc: Stephen Bates , Christoph Hellwig , Keith Busch , Sagi Grimberg , Bjorn Helgaas , Jason Gunthorpe , Max Gurtovoy , Dan Williams , =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Benjamin Herrenschmidt , Alex Williamson , =?UTF-8?q?Christian=20K=C3=B6nig?= , Logan Gunthorpe Date: Thu, 30 Aug 2018 12:53:40 -0600 Message-Id: <20180830185352.3369-2-logang@deltatee.com> In-Reply-To: <20180830185352.3369-1-logang@deltatee.com> References: <20180830185352.3369-1-logang@deltatee.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Subject: [PATCH v5 01/13] PCI/P2PDMA: Support peer-to-peer memory Sender: linux-block-owner@vger.kernel.org List-Id: linux-block@vger.kernel.org Some PCI devices may have memory mapped in a BAR space that's intended for use in peer-to-peer transactions. In order to enable such transactions the memory must be registered with ZONE_DEVICE pages so it can be used by DMA interfaces in existing drivers. Add an interface for other subsystems to find and allocate chunks of P2P memory as necessary to facilitate transfers between two PCI peers: int pci_p2pdma_add_client(); struct pci_dev *pci_p2pmem_find(); void *pci_alloc_p2pmem(); The new interface requires a driver to collect a list of client devices involved in the transaction with the pci_p2pmem_add_client*() functions then call pci_p2pmem_find() to obtain any suitable P2P memory. Once this is done the list is bound to the memory and the calling driver is free to add and remove clients as necessary (adding incompatible clients will fail). With a suitable p2pmem device, memory can then be allocated with pci_alloc_p2pmem() for use in DMA transactions. Depending on hardware, using peer-to-peer memory may reduce the bandwidth of the transfer but can significantly reduce pressure on system memory. This may be desirable in many cases: for example a system could be designed with a small CPU connected to a PCIe switch by a small number of lanes which would maximize the number of lanes available to connect to NVMe devices. The code is designed to only utilize the p2pmem device if all the devices involved in a transfer are behind the same PCI bridge. This is because we have no way of knowing whether peer-to-peer routing between PCIe Root Ports is supported (PCIe r4.0, sec 1.3.1). Additionally, the benefits of P2P transfers that go through the RC is limited to only reducing DRAM usage and, in some cases, coding convenience. The PCI-SIG may be exploring adding a new capability bit to advertise whether this is possible for future hardware. This commit includes significant rework and feedback from Christoph Hellwig. Signed-off-by: Christoph Hellwig Signed-off-by: Logan Gunthorpe --- drivers/pci/Kconfig | 17 + drivers/pci/Makefile | 1 + drivers/pci/p2pdma.c | 761 +++++++++++++++++++++++++++++++++++++++++++++ include/linux/memremap.h | 5 + include/linux/mm.h | 18 ++ include/linux/pci-p2pdma.h | 102 ++++++ include/linux/pci.h | 4 + 7 files changed, 908 insertions(+) create mode 100644 drivers/pci/p2pdma.c create mode 100644 include/linux/pci-p2pdma.h diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig index 56ff8f6d31fc..deb68be4fdac 100644 --- a/drivers/pci/Kconfig +++ b/drivers/pci/Kconfig @@ -132,6 +132,23 @@ config PCI_PASID If unsure, say N. +config PCI_P2PDMA + bool "PCI peer-to-peer transfer support" + depends on PCI && ZONE_DEVICE + select GENERIC_ALLOCATOR + help + Enableѕ drivers to do PCI peer-to-peer transactions to and from + BARs that are exposed in other devices that are the part of + the hierarchy where peer-to-peer DMA is guaranteed by the PCI + specification to work (ie. anything below a single PCI bridge). + + Many PCIe root complexes do not support P2P transactions and + it's hard to tell which support it at all, so at this time, + P2P DMA transations must be between devices behind the same root + port. + + If unsure, say N. + config PCI_LABEL def_bool y if (DMI || ACPI) depends on PCI diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile index 1b2cfe51e8d7..85f4a703b2be 100644 --- a/drivers/pci/Makefile +++ b/drivers/pci/Makefile @@ -26,6 +26,7 @@ obj-$(CONFIG_PCI_SYSCALL) += syscall.o obj-$(CONFIG_PCI_STUB) += pci-stub.o obj-$(CONFIG_PCI_PF_STUB) += pci-pf-stub.o obj-$(CONFIG_PCI_ECAM) += ecam.o +obj-$(CONFIG_PCI_P2PDMA) += p2pdma.o obj-$(CONFIG_XEN_PCIDEV_FRONTEND) += xen-pcifront.o # Endpoint library must be initialized before its users diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c new file mode 100644 index 000000000000..88aaec5351cd --- /dev/null +++ b/drivers/pci/p2pdma.c @@ -0,0 +1,761 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * PCI Peer 2 Peer DMA support. + * + * Copyright (c) 2016-2018, Logan Gunthorpe + * Copyright (c) 2016-2017, Microsemi Corporation + * Copyright (c) 2017, Christoph Hellwig + * Copyright (c) 2018, Eideticom Inc. + */ + +#define pr_fmt(fmt) "pci-p2pdma: " fmt +#include +#include +#include +#include +#include +#include +#include +#include + +struct pci_p2pdma { + struct percpu_ref devmap_ref; + struct completion devmap_ref_done; + struct gen_pool *pool; + bool p2pmem_published; +}; + +static void pci_p2pdma_percpu_release(struct percpu_ref *ref) +{ + struct pci_p2pdma *p2p = + container_of(ref, struct pci_p2pdma, devmap_ref); + + complete_all(&p2p->devmap_ref_done); +} + +static void pci_p2pdma_percpu_kill(void *data) +{ + struct percpu_ref *ref = data; + + if (percpu_ref_is_dying(ref)) + return; + + percpu_ref_kill(ref); +} + +static void pci_p2pdma_release(void *data) +{ + struct pci_dev *pdev = data; + + if (!pdev->p2pdma) + return; + + wait_for_completion(&pdev->p2pdma->devmap_ref_done); + percpu_ref_exit(&pdev->p2pdma->devmap_ref); + + gen_pool_destroy(pdev->p2pdma->pool); + pdev->p2pdma = NULL; +} + +static int pci_p2pdma_setup(struct pci_dev *pdev) +{ + int error = -ENOMEM; + struct pci_p2pdma *p2p; + + p2p = devm_kzalloc(&pdev->dev, sizeof(*p2p), GFP_KERNEL); + if (!p2p) + return -ENOMEM; + + p2p->pool = gen_pool_create(PAGE_SHIFT, dev_to_node(&pdev->dev)); + if (!p2p->pool) + goto out; + + init_completion(&p2p->devmap_ref_done); + error = percpu_ref_init(&p2p->devmap_ref, + pci_p2pdma_percpu_release, 0, GFP_KERNEL); + if (error) + goto out_pool_destroy; + + percpu_ref_switch_to_atomic_sync(&p2p->devmap_ref); + + error = devm_add_action_or_reset(&pdev->dev, pci_p2pdma_release, pdev); + if (error) + goto out_pool_destroy; + + pdev->p2pdma = p2p; + + return 0; + +out_pool_destroy: + gen_pool_destroy(p2p->pool); +out: + devm_kfree(&pdev->dev, p2p); + return error; +} + +/** + * pci_p2pdma_add_resource - add memory for use as p2p memory + * @pdev: the device to add the memory to + * @bar: PCI BAR to add + * @size: size of the memory to add, may be zero to use the whole BAR + * @offset: offset into the PCI BAR + * + * The memory will be given ZONE_DEVICE struct pages so that it may + * be used with any DMA request. + */ +int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, + u64 offset) +{ + struct dev_pagemap *pgmap; + void *addr; + int error; + + if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM)) + return -EINVAL; + + if (offset >= pci_resource_len(pdev, bar)) + return -EINVAL; + + if (!size) + size = pci_resource_len(pdev, bar) - offset; + + if (size + offset > pci_resource_len(pdev, bar)) + return -EINVAL; + + if (!pdev->p2pdma) { + error = pci_p2pdma_setup(pdev); + if (error) + return error; + } + + pgmap = devm_kzalloc(&pdev->dev, sizeof(*pgmap), GFP_KERNEL); + if (!pgmap) + return -ENOMEM; + + pgmap->res.start = pci_resource_start(pdev, bar) + offset; + pgmap->res.end = pgmap->res.start + size - 1; + pgmap->res.flags = pci_resource_flags(pdev, bar); + pgmap->ref = &pdev->p2pdma->devmap_ref; + pgmap->type = MEMORY_DEVICE_PCI_P2PDMA; + + addr = devm_memremap_pages(&pdev->dev, pgmap); + if (IS_ERR(addr)) { + error = PTR_ERR(addr); + goto pgmap_free; + } + + error = gen_pool_add_virt(pdev->p2pdma->pool, (unsigned long)addr, + pci_bus_address(pdev, bar) + offset, + resource_size(&pgmap->res), dev_to_node(&pdev->dev)); + if (error) + goto pgmap_free; + + error = devm_add_action_or_reset(&pdev->dev, pci_p2pdma_percpu_kill, + &pdev->p2pdma->devmap_ref); + if (error) + goto pgmap_free; + + pci_info(pdev, "added peer-to-peer DMA memory %pR\n", + &pgmap->res); + + return 0; + +pgmap_free: + devres_free(pgmap); + return error; +} +EXPORT_SYMBOL_GPL(pci_p2pdma_add_resource); + +static struct pci_dev *find_parent_pci_dev(struct device *dev) +{ + struct device *parent; + + dev = get_device(dev); + + while (dev) { + if (dev_is_pci(dev)) + return to_pci_dev(dev); + + parent = get_device(dev->parent); + put_device(dev); + dev = parent; + } + + return NULL; +} + +/* + * Check if a PCI bridge has it's ACS redirection bits set to redirect P2P + * TLPs upstream via ACS. Returns 1 if the packets will be redirected + * upstream, 0 otherwise. + */ +static int pci_bridge_has_acs_redir(struct pci_dev *dev) +{ + int pos; + u16 ctrl; + + pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ACS); + if (!pos) + return 0; + + pci_read_config_word(dev, pos + PCI_ACS_CTRL, &ctrl); + + if (ctrl & (PCI_ACS_RR | PCI_ACS_CR | PCI_ACS_EC)) + return 1; + + return 0; +} + +static void seq_buf_print_bus_devfn(struct seq_buf *buf, struct pci_dev *dev) +{ + if (!buf) + return; + + seq_buf_printf(buf, "%04x:%02x:%02x.%x;", pci_domain_nr(dev->bus), + dev->bus->number, PCI_SLOT(dev->devfn), + PCI_FUNC(dev->devfn)); +} + +/* + * Find the distance through the nearest common upstream bridge between + * two PCI devices. + * + * If the two devices are the same device then 0 will be returned. + * + * If there are two virtual functions of the same device behind the same + * bridge port then 2 will be returned (one step down to the PCIe switch, + * then one step back to the same device). + * + * In the case where two devices are connected to the same PCIe switch, the + * value 4 will be returned. This corresponds to the following PCI tree: + * + * -+ Root Port + * \+ Switch Upstream Port + * +-+ Switch Downstream Port + * + \- Device A + * \-+ Switch Downstream Port + * \- Device B + * + * The distance is 4 because we traverse from Device A through the downstream + * port of the switch, to the common upstream port, back up to the second + * downstream port and then to Device B. + * + * Any two devices that don't have a common upstream bridge will return -1. + * In this way devices on separate PCIe root ports will be rejected, which + * is what we want for peer-to-peer seeing each PCIe root port defines a + * separate hierarchy domain and there's no way to determine whether the root + * complex supports forwarding between them. + * + * In the case where two devices are connected to different PCIe switches, + * this function will still return a positive distance as long as both + * switches evenutally have a common upstream bridge. Note this covers + * the case of using multiple PCIe switches to achieve a desired level of + * fan-out from a root port. The exact distance will be a function of the + * number of switches between Device A and Device B. + * + * If a bridge which has any ACS redirection bits set is in the path + * then this functions will return -2. This is so we reject any + * cases where the TLPs are forwarded up into the root complex. + * In this case, a list of all infringing bridge addresses will be + * populated in acs_list (assuming it's non-null) for printk purposes. + */ +static int upstream_bridge_distance(struct pci_dev *a, + struct pci_dev *b, + struct seq_buf *acs_list) +{ + int dist_a = 0; + int dist_b = 0; + struct pci_dev *bb = NULL; + int acs_cnt = 0; + + /* + * Note, we don't need to take references to devices returned by + * pci_upstream_bridge() seeing we hold a reference to a child + * device which will already hold a reference to the upstream bridge. + */ + + while (a) { + dist_b = 0; + + if (pci_bridge_has_acs_redir(a)) { + seq_buf_print_bus_devfn(acs_list, a); + acs_cnt++; + } + + bb = b; + + while (bb) { + if (a == bb) + goto check_b_path_acs; + + bb = pci_upstream_bridge(bb); + dist_b++; + } + + a = pci_upstream_bridge(a); + dist_a++; + } + + return -1; + +check_b_path_acs: + bb = b; + + while (bb) { + if (a == bb) + break; + + if (pci_bridge_has_acs_redir(bb)) { + seq_buf_print_bus_devfn(acs_list, bb); + acs_cnt++; + } + + bb = pci_upstream_bridge(bb); + } + + if (acs_cnt) + return -2; + + return dist_a + dist_b; +} + +static int upstream_bridge_distance_warn(struct pci_dev *provider, + struct pci_dev *client) +{ + struct seq_buf acs_list; + int ret; + + seq_buf_init(&acs_list, kmalloc(PAGE_SIZE, GFP_KERNEL), PAGE_SIZE); + + ret = upstream_bridge_distance(provider, client, &acs_list); + if (ret == -2) { + pci_warn(client, "cannot be used for peer-to-peer DMA as ACS redirect is set between the client and provider\n"); + /* Drop final semicolon */ + acs_list.buffer[acs_list.len-1] = 0; + pci_warn(client, "to disable ACS redirect for this path, add the kernel parameter: pci=disable_acs_redir=%s\n", + acs_list.buffer); + + } else if (ret < 0) { + pci_warn(client, "cannot be used for peer-to-peer DMA as the client and provider do not share an upstream bridge\n"); + } + + kfree(acs_list.buffer); + + return ret; +} + +struct pci_p2pdma_client { + struct list_head list; + struct pci_dev *client; + struct pci_dev *provider; +}; + +/** + * pci_p2pdma_add_client - allocate a new element in a client device list + * @head: list head of p2pdma clients + * @dev: device to add to the list + * + * This adds @dev to a list of clients used by a p2pdma device. + * This list should be passed to pci_p2pmem_find(). Once pci_p2pmem_find() has + * been called successfully, the list will be bound to a specific p2pdma + * device and new clients can only be added to the list if they are + * supported by that p2pdma device. + * + * The caller is expected to have a lock which protects @head as necessary + * so that none of the pci_p2p functions can be called concurrently + * on that list. + * + * Returns 0 if the client was successfully added. + */ +int pci_p2pdma_add_client(struct list_head *head, struct device *dev) +{ + struct pci_p2pdma_client *item, *new_item; + struct pci_dev *provider = NULL; + struct pci_dev *client; + int ret; + + if (IS_ENABLED(CONFIG_DMA_VIRT_OPS) && dev->dma_ops == &dma_virt_ops) { + dev_warn(dev, "cannot be used for peer-to-peer DMA because the driver makes use of dma_virt_ops\n"); + return -ENODEV; + } + + client = find_parent_pci_dev(dev); + if (!client) { + dev_warn(dev, "cannot be used for peer-to-peer DMA as it is not a PCI device\n"); + return -ENODEV; + } + + item = list_first_entry_or_null(head, struct pci_p2pdma_client, list); + if (item && item->provider) { + provider = item->provider; + + ret = upstream_bridge_distance_warn(provider, client); + if (ret < 0) { + ret = -EXDEV; + goto put_client; + } + } + + new_item = kzalloc(sizeof(*new_item), GFP_KERNEL); + if (!new_item) { + ret = -ENOMEM; + goto put_client; + } + + new_item->client = client; + new_item->provider = pci_dev_get(provider); + + list_add_tail(&new_item->list, head); + + return 0; + +put_client: + pci_dev_put(client); + return ret; +} +EXPORT_SYMBOL_GPL(pci_p2pdma_add_client); + +static void pci_p2pdma_client_free(struct pci_p2pdma_client *item) +{ + list_del(&item->list); + pci_dev_put(item->client); + pci_dev_put(item->provider); + kfree(item); +} + +/** + * pci_p2pdma_remove_client - remove and free a p2pdma client + * @head: list head of p2pdma clients + * @dev: device to remove from the list + * + * This removes @dev from a list of clients used by a p2pdma device. + * The caller is expected to have a lock which protects @head as necessary + * so that none of the pci_p2p functions can be called concurrently + * on that list. + */ +void pci_p2pdma_remove_client(struct list_head *head, struct device *dev) +{ + struct pci_p2pdma_client *pos, *tmp; + struct pci_dev *pdev; + + pdev = find_parent_pci_dev(dev); + if (!pdev) + return; + + list_for_each_entry_safe(pos, tmp, head, list) { + if (pos->client != pdev) + continue; + + pci_p2pdma_client_free(pos); + } + + pci_dev_put(pdev); +} +EXPORT_SYMBOL_GPL(pci_p2pdma_remove_client); + +/** + * pci_p2pdma_client_list_free - free an entire list of p2pdma clients + * @head: list head of p2pdma clients + * + * This removes all devices in a list of clients used by a p2pdma device. + * The caller is expected to have a lock which protects @head as necessary + * so that none of the pci_p2pdma functions can be called concurrently + * on that list. + */ +void pci_p2pdma_client_list_free(struct list_head *head) +{ + struct pci_p2pdma_client *pos, *tmp; + + list_for_each_entry_safe(pos, tmp, head, list) + pci_p2pdma_client_free(pos); +} +EXPORT_SYMBOL_GPL(pci_p2pdma_client_list_free); + +/** + * pci_p2pdma_distance - Determive the cumulative distance between + * a p2pdma provider and the clients in use. + * @provider: p2pdma provider to check against the client list + * @clients: list of devices to check (NULL-terminated) + * @verbose: if true, print warnings for devices when we return -1 + * + * Returns -1 if any of the clients are not compatible (behind the same + * root port as the provider), otherwise returns a positive number where + * the lower number is the preferrable choice. (If there's one client + * that's the same as the provider it will return 0, which is best choice). + * + * For now, "compatible" means the provider and the clients are all behind + * the same PCI root port. This cuts out cases that may work but is safest + * for the user. Future work can expand this to white-list root complexes that + * can safely forward between each ports. + */ +int pci_p2pdma_distance(struct pci_dev *provider, struct list_head *clients, + bool verbose) +{ + struct pci_p2pdma_client *pos; + int ret; + int distance = 0; + bool not_supported = false; + + if (list_empty(clients)) + return -1; + + list_for_each_entry(pos, clients, list) { + if (verbose) + ret = upstream_bridge_distance_warn(provider, + pos->client); + else + ret = upstream_bridge_distance(provider, pos->client, + NULL); + + if (ret < 0) + not_supported = true; + + if (not_supported && !verbose) + break; + + distance += ret; + } + + if (not_supported) + return -1; + + return distance; +} +EXPORT_SYMBOL_GPL(pci_p2pdma_distance); + +/** + * pci_p2pdma_assign_provider - Check compatibily (as per pci_p2pdma_distance) + * and assign a provider to a list of clients + * @provider: p2pdma provider to assign to the client list + * @clients: list of devices to check (NULL-terminated) + * + * Returns false if any of the clients are not compatible, true if the + * provider was successfully assigned to the clients. + */ +bool pci_p2pdma_assign_provider(struct pci_dev *provider, + struct list_head *clients) +{ + struct pci_p2pdma_client *pos; + + if (pci_p2pdma_distance(provider, clients, true) < 0) + return false; + + list_for_each_entry(pos, clients, list) + pos->provider = provider; + + return true; +} +EXPORT_SYMBOL_GPL(pci_p2pdma_assign_provider); + +/** + * pci_has_p2pmem - check if a given PCI device has published any p2pmem + * @pdev: PCI device to check + */ +bool pci_has_p2pmem(struct pci_dev *pdev) +{ + return pdev->p2pdma && pdev->p2pdma->p2pmem_published; +} +EXPORT_SYMBOL_GPL(pci_has_p2pmem); + +/** + * pci_p2pmem_find - find a peer-to-peer DMA memory device compatible with + * the specified list of clients and shortest distance (as determined + * by pci_p2pmem_dma()) + * @clients: list of devices to check (NULL-terminated) + * + * If multiple devices are behind the same switch, the one "closest" to the + * client devices in use will be chosen first. (So if one of the providers are + * the same as one of the clients, that provider will be used ahead of any + * other providers that are unrelated). If multiple providers are an equal + * distance away, one will be chosen at random. + * + * Returns a pointer to the PCI device with a reference taken (use pci_dev_put + * to return the reference) or NULL if no compatible device is found. The + * found provider will also be assigned to the client list. + */ +struct pci_dev *pci_p2pmem_find(struct list_head *clients) +{ + struct pci_dev *pdev = NULL; + struct pci_p2pdma_client *pos; + int distance; + int closest_distance = INT_MAX; + struct pci_dev **closest_pdevs; + int dev_cnt = 0; + const int max_devs = PAGE_SIZE / sizeof(*closest_pdevs); + int i; + + closest_pdevs = kmalloc(PAGE_SIZE, GFP_KERNEL); + + while ((pdev = pci_get_device(PCI_ANY_ID, PCI_ANY_ID, pdev))) { + if (!pci_has_p2pmem(pdev)) + continue; + + distance = pci_p2pdma_distance(pdev, clients, false); + if (distance < 0 || distance > closest_distance) + continue; + + if (distance == closest_distance && dev_cnt >= max_devs) + continue; + + if (distance < closest_distance) { + for (i = 0; i < dev_cnt; i++) + pci_dev_put(closest_pdevs[i]); + + dev_cnt = 0; + closest_distance = distance; + } + + closest_pdevs[dev_cnt++] = pci_dev_get(pdev); + } + + if (dev_cnt) + pdev = pci_dev_get(closest_pdevs[prandom_u32_max(dev_cnt)]); + + for (i = 0; i < dev_cnt; i++) + pci_dev_put(closest_pdevs[i]); + + if (pdev) + list_for_each_entry(pos, clients, list) + pos->provider = pdev; + + kfree(closest_pdevs); + return pdev; +} +EXPORT_SYMBOL_GPL(pci_p2pmem_find); + +/** + * pci_alloc_p2p_mem - allocate peer-to-peer DMA memory + * @pdev: the device to allocate memory from + * @size: number of bytes to allocate + * + * Returns the allocated memory or NULL on error. + */ +void *pci_alloc_p2pmem(struct pci_dev *pdev, size_t size) +{ + void *ret; + + if (unlikely(!pdev->p2pdma)) + return NULL; + + if (unlikely(!percpu_ref_tryget_live(&pdev->p2pdma->devmap_ref))) + return NULL; + + ret = (void *)gen_pool_alloc(pdev->p2pdma->pool, size); + + if (unlikely(!ret)) + percpu_ref_put(&pdev->p2pdma->devmap_ref); + + return ret; +} +EXPORT_SYMBOL_GPL(pci_alloc_p2pmem); + +/** + * pci_free_p2pmem - allocate peer-to-peer DMA memory + * @pdev: the device the memory was allocated from + * @addr: address of the memory that was allocated + * @size: number of bytes that was allocated + */ +void pci_free_p2pmem(struct pci_dev *pdev, void *addr, size_t size) +{ + gen_pool_free(pdev->p2pdma->pool, (uintptr_t)addr, size); + percpu_ref_put(&pdev->p2pdma->devmap_ref); +} +EXPORT_SYMBOL_GPL(pci_free_p2pmem); + +/** + * pci_virt_to_bus - return the PCI bus address for a given virtual + * address obtained with pci_alloc_p2pmem() + * @pdev: the device the memory was allocated from + * @addr: address of the memory that was allocated + */ +pci_bus_addr_t pci_p2pmem_virt_to_bus(struct pci_dev *pdev, void *addr) +{ + if (!addr) + return 0; + if (!pdev->p2pdma) + return 0; + + /* + * Note: when we added the memory to the pool we used the PCI + * bus address as the physical address. So gen_pool_virt_to_phys() + * actually returns the bus address despite the misleading name. + */ + return gen_pool_virt_to_phys(pdev->p2pdma->pool, (unsigned long)addr); +} +EXPORT_SYMBOL_GPL(pci_p2pmem_virt_to_bus); + +/** + * pci_p2pmem_alloc_sgl - allocate peer-to-peer DMA memory in a scatterlist + * @pdev: the device to allocate memory from + * @sgl: the allocated scatterlist + * @nents: the number of SG entries in the list + * @length: number of bytes to allocate + * + * Returns 0 on success + */ +struct scatterlist *pci_p2pmem_alloc_sgl(struct pci_dev *pdev, + unsigned int *nents, u32 length) +{ + struct scatterlist *sg; + void *addr; + + sg = kzalloc(sizeof(*sg), GFP_KERNEL); + if (!sg) + return NULL; + + sg_init_table(sg, 1); + + addr = pci_alloc_p2pmem(pdev, length); + if (!addr) + goto out_free_sg; + + sg_set_buf(sg, addr, length); + *nents = 1; + return sg; + +out_free_sg: + kfree(sg); + return NULL; +} +EXPORT_SYMBOL_GPL(pci_p2pmem_alloc_sgl); + +/** + * pci_p2pmem_free_sgl - free a scatterlist allocated by pci_p2pmem_alloc_sgl() + * @pdev: the device to allocate memory from + * @sgl: the allocated scatterlist + * @nents: the number of SG entries in the list + */ +void pci_p2pmem_free_sgl(struct pci_dev *pdev, struct scatterlist *sgl) +{ + struct scatterlist *sg; + int count; + + for_each_sg(sgl, sg, INT_MAX, count) { + if (!sg) + break; + + pci_free_p2pmem(pdev, sg_virt(sg), sg->length); + } + kfree(sgl); +} +EXPORT_SYMBOL_GPL(pci_p2pmem_free_sgl); + +/** + * pci_p2pmem_publish - publish the peer-to-peer DMA memory for use by + * other devices with pci_p2pmem_find() + * @pdev: the device with peer-to-peer DMA memory to publish + * @publish: set to true to publish the memory, false to unpublish it + * + * Published memory can be used by other PCI device drivers for + * peer-2-peer DMA operations. Non-published memory is reserved for + * exlusive use of the device driver that registers the peer-to-peer + * memory. + */ +void pci_p2pmem_publish(struct pci_dev *pdev, bool publish) +{ + if (publish && !pdev->p2pdma) + return; + + pdev->p2pdma->p2pmem_published = publish; +} +EXPORT_SYMBOL_GPL(pci_p2pmem_publish); diff --git a/include/linux/memremap.h b/include/linux/memremap.h index f91f9e763557..9553370ebdad 100644 --- a/include/linux/memremap.h +++ b/include/linux/memremap.h @@ -53,11 +53,16 @@ struct vmem_altmap { * wakeup event whenever a page is unpinned and becomes idle. This * wakeup is used to coordinate physical address space management (ex: * fs truncate/hole punch) vs pinned pages (ex: device dma). + * + * MEMORY_DEVICE_PCI_P2PDMA: + * Device memory residing in a PCI BAR intended for use with Peer-to-Peer + * transactions. */ enum memory_type { MEMORY_DEVICE_PRIVATE = 1, MEMORY_DEVICE_PUBLIC, MEMORY_DEVICE_FS_DAX, + MEMORY_DEVICE_PCI_P2PDMA, }; /* diff --git a/include/linux/mm.h b/include/linux/mm.h index a61ebe8ad4ca..2055df412a77 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -890,6 +890,19 @@ static inline bool is_device_public_page(const struct page *page) page->pgmap->type == MEMORY_DEVICE_PUBLIC; } +#ifdef CONFIG_PCI_P2PDMA +static inline bool is_pci_p2pdma_page(const struct page *page) +{ + return is_zone_device_page(page) && + page->pgmap->type == MEMORY_DEVICE_PCI_P2PDMA; +} +#else /* CONFIG_PCI_P2PDMA */ +static inline bool is_pci_p2pdma_page(const struct page *page) +{ + return false; +} +#endif /* CONFIG_PCI_P2PDMA */ + #else /* CONFIG_DEV_PAGEMAP_OPS */ static inline void dev_pagemap_get_ops(void) { @@ -913,6 +926,11 @@ static inline bool is_device_public_page(const struct page *page) { return false; } + +static inline bool is_pci_p2pdma_page(const struct page *page) +{ + return false; +} #endif /* CONFIG_DEV_PAGEMAP_OPS */ static inline void get_page(struct page *page) diff --git a/include/linux/pci-p2pdma.h b/include/linux/pci-p2pdma.h new file mode 100644 index 000000000000..7b2b0f547528 --- /dev/null +++ b/include/linux/pci-p2pdma.h @@ -0,0 +1,102 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * PCI Peer 2 Peer DMA support. + * + * Copyright (c) 2016-2018, Logan Gunthorpe + * Copyright (c) 2016-2017, Microsemi Corporation + * Copyright (c) 2017, Christoph Hellwig + * Copyright (c) 2018, Eideticom Inc. + * + */ + +#ifndef _LINUX_PCI_P2PDMA_H +#define _LINUX_PCI_P2PDMA_H + +#include + +struct block_device; +struct scatterlist; + +#ifdef CONFIG_PCI_P2PDMA +int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, + u64 offset); +int pci_p2pdma_add_client(struct list_head *head, struct device *dev); +void pci_p2pdma_remove_client(struct list_head *head, struct device *dev); +void pci_p2pdma_client_list_free(struct list_head *head); +int pci_p2pdma_distance(struct pci_dev *provider, struct list_head *clients, + bool verbose); +bool pci_p2pdma_assign_provider(struct pci_dev *provider, + struct list_head *clients); +bool pci_has_p2pmem(struct pci_dev *pdev); +struct pci_dev *pci_p2pmem_find(struct list_head *clients); +void *pci_alloc_p2pmem(struct pci_dev *pdev, size_t size); +void pci_free_p2pmem(struct pci_dev *pdev, void *addr, size_t size); +pci_bus_addr_t pci_p2pmem_virt_to_bus(struct pci_dev *pdev, void *addr); +struct scatterlist *pci_p2pmem_alloc_sgl(struct pci_dev *pdev, + unsigned int *nents, u32 length); +void pci_p2pmem_free_sgl(struct pci_dev *pdev, struct scatterlist *sgl); +void pci_p2pmem_publish(struct pci_dev *pdev, bool publish); +#else /* CONFIG_PCI_P2PDMA */ +static inline int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, + size_t size, u64 offset) +{ + return -EOPNOTSUPP; +} +static inline int pci_p2pdma_add_client(struct list_head *head, + struct device *dev) +{ + return 0; +} +static inline void pci_p2pdma_remove_client(struct list_head *head, + struct device *dev) +{ +} +static inline void pci_p2pdma_client_list_free(struct list_head *head) +{ +} +static inline int pci_p2pdma_distance(struct pci_dev *provider, + struct list_head *clients, + bool verbose) +{ + return -1; +} +static inline bool pci_p2pdma_assign_provider(struct pci_dev *provider, + struct list_head *clients) +{ + return false; +} +static inline bool pci_has_p2pmem(struct pci_dev *pdev) +{ + return false; +} +static inline struct pci_dev *pci_p2pmem_find(struct list_head *clients) +{ + return NULL; +} +static inline void *pci_alloc_p2pmem(struct pci_dev *pdev, size_t size) +{ + return NULL; +} +static inline void pci_free_p2pmem(struct pci_dev *pdev, void *addr, + size_t size) +{ +} +static inline pci_bus_addr_t pci_p2pmem_virt_to_bus(struct pci_dev *pdev, + void *addr) +{ + return 0; +} +static inline struct scatterlist *pci_p2pmem_alloc_sgl(struct pci_dev *pdev, + unsigned int *nents, u32 length) +{ + return NULL; +} +static inline void pci_p2pmem_free_sgl(struct pci_dev *pdev, + struct scatterlist *sgl) +{ +} +static inline void pci_p2pmem_publish(struct pci_dev *pdev, bool publish) +{ +} +#endif /* CONFIG_PCI_P2PDMA */ +#endif /* _LINUX_PCI_P2P_H */ diff --git a/include/linux/pci.h b/include/linux/pci.h index e72ca8dd6241..5d95dbf21f4a 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -281,6 +281,7 @@ struct pcie_link_state; struct pci_vpd; struct pci_sriov; struct pci_ats; +struct pci_p2pdma; /* The pci_dev structure describes PCI devices */ struct pci_dev { @@ -439,6 +440,9 @@ struct pci_dev { #ifdef CONFIG_PCI_PASID u16 pasid_features; #endif +#ifdef CONFIG_PCI_P2PDMA + struct pci_p2pdma *p2pdma; +#endif phys_addr_t rom; /* Physical address if not from BAR */ size_t romlen; /* Length if not from BAR */ char *driver_override; /* Driver name to force a match */ -- 2.11.0 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ale.deltatee.com (ale.deltatee.com [207.54.116.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id D31312110747B for ; Thu, 30 Aug 2018 11:54:04 -0700 (PDT) From: Logan Gunthorpe Date: Thu, 30 Aug 2018 12:53:40 -0600 Message-Id: <20180830185352.3369-2-logang@deltatee.com> In-Reply-To: <20180830185352.3369-1-logang@deltatee.com> References: <20180830185352.3369-1-logang@deltatee.com> MIME-Version: 1.0 Subject: [PATCH v5 01/13] PCI/P2PDMA: Support peer-to-peer memory List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" To: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org, linux-nvdimm@lists.01.org, linux-block@vger.kernel.org Cc: =?UTF-8?q?Christian=20K=C3=B6nig?= , Benjamin Herrenschmidt , Alex Williamson , =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Jason Gunthorpe , Bjorn Helgaas , Max Gurtovoy , Christoph Hellwig List-ID: U29tZSBQQ0kgZGV2aWNlcyBtYXkgaGF2ZSBtZW1vcnkgbWFwcGVkIGluIGEgQkFSIHNwYWNlIHRo YXQncwppbnRlbmRlZCBmb3IgdXNlIGluIHBlZXItdG8tcGVlciB0cmFuc2FjdGlvbnMuIEluIG9y ZGVyIHRvIGVuYWJsZQpzdWNoIHRyYW5zYWN0aW9ucyB0aGUgbWVtb3J5IG11c3QgYmUgcmVnaXN0 ZXJlZCB3aXRoIFpPTkVfREVWSUNFIHBhZ2VzCnNvIGl0IGNhbiBiZSB1c2VkIGJ5IERNQSBpbnRl cmZhY2VzIGluIGV4aXN0aW5nIGRyaXZlcnMuCgpBZGQgYW4gaW50ZXJmYWNlIGZvciBvdGhlciBz dWJzeXN0ZW1zIHRvIGZpbmQgYW5kIGFsbG9jYXRlIGNodW5rcyBvZiBQMlAKbWVtb3J5IGFzIG5l Y2Vzc2FyeSB0byBmYWNpbGl0YXRlIHRyYW5zZmVycyBiZXR3ZWVuIHR3byBQQ0kgcGVlcnM6Cgpp bnQgcGNpX3AycGRtYV9hZGRfY2xpZW50KCk7CnN0cnVjdCBwY2lfZGV2ICpwY2lfcDJwbWVtX2Zp bmQoKTsKdm9pZCAqcGNpX2FsbG9jX3AycG1lbSgpOwoKVGhlIG5ldyBpbnRlcmZhY2UgcmVxdWly ZXMgYSBkcml2ZXIgdG8gY29sbGVjdCBhIGxpc3Qgb2YgY2xpZW50IGRldmljZXMKaW52b2x2ZWQg aW4gdGhlIHRyYW5zYWN0aW9uIHdpdGggdGhlIHBjaV9wMnBtZW1fYWRkX2NsaWVudCooKSBmdW5j dGlvbnMKdGhlbiBjYWxsIHBjaV9wMnBtZW1fZmluZCgpIHRvIG9idGFpbiBhbnkgc3VpdGFibGUg UDJQIG1lbW9yeS4gT25jZQp0aGlzIGlzIGRvbmUgdGhlIGxpc3QgaXMgYm91bmQgdG8gdGhlIG1l bW9yeSBhbmQgdGhlIGNhbGxpbmcgZHJpdmVyIGlzCmZyZWUgdG8gYWRkIGFuZCByZW1vdmUgY2xp ZW50cyBhcyBuZWNlc3NhcnkgKGFkZGluZyBpbmNvbXBhdGlibGUgY2xpZW50cwp3aWxsIGZhaWwp LiBXaXRoIGEgc3VpdGFibGUgcDJwbWVtIGRldmljZSwgbWVtb3J5IGNhbiB0aGVuIGJlCmFsbG9j YXRlZCB3aXRoIHBjaV9hbGxvY19wMnBtZW0oKSBmb3IgdXNlIGluIERNQSB0cmFuc2FjdGlvbnMu CgpEZXBlbmRpbmcgb24gaGFyZHdhcmUsIHVzaW5nIHBlZXItdG8tcGVlciBtZW1vcnkgbWF5IHJl ZHVjZSB0aGUgYmFuZHdpZHRoCm9mIHRoZSB0cmFuc2ZlciBidXQgY2FuIHNpZ25pZmljYW50bHkg cmVkdWNlIHByZXNzdXJlIG9uIHN5c3RlbSBtZW1vcnkuClRoaXMgbWF5IGJlIGRlc2lyYWJsZSBp biBtYW55IGNhc2VzOiBmb3IgZXhhbXBsZSBhIHN5c3RlbSBjb3VsZCBiZSBkZXNpZ25lZAp3aXRo IGEgc21hbGwgQ1BVIGNvbm5lY3RlZCB0byBhIFBDSWUgc3dpdGNoIGJ5IGEgc21hbGwgbnVtYmVy IG9mIGxhbmVzCndoaWNoIHdvdWxkIG1heGltaXplIHRoZSBudW1iZXIgb2YgbGFuZXMgYXZhaWxh YmxlIHRvIGNvbm5lY3QgdG8gTlZNZQpkZXZpY2VzLgoKVGhlIGNvZGUgaXMgZGVzaWduZWQgdG8g b25seSB1dGlsaXplIHRoZSBwMnBtZW0gZGV2aWNlIGlmIGFsbCB0aGUgZGV2aWNlcwppbnZvbHZl ZCBpbiBhIHRyYW5zZmVyIGFyZSBiZWhpbmQgdGhlIHNhbWUgUENJIGJyaWRnZS4gVGhpcyBpcyBi ZWNhdXNlIHdlCmhhdmUgbm8gd2F5IG9mIGtub3dpbmcgd2hldGhlciBwZWVyLXRvLXBlZXIgcm91 dGluZyBiZXR3ZWVuIFBDSWUgUm9vdCBQb3J0cwppcyBzdXBwb3J0ZWQgKFBDSWUgcjQuMCwgc2Vj IDEuMy4xKS4gQWRkaXRpb25hbGx5LCB0aGUgYmVuZWZpdHMgb2YgUDJQCnRyYW5zZmVycyB0aGF0 IGdvIHRocm91Z2ggdGhlIFJDIGlzIGxpbWl0ZWQgdG8gb25seSByZWR1Y2luZyBEUkFNIHVzYWdl CmFuZCwgaW4gc29tZSBjYXNlcywgY29kaW5nIGNvbnZlbmllbmNlLiBUaGUgUENJLVNJRyBtYXkg YmUgZXhwbG9yaW5nCmFkZGluZyBhIG5ldyBjYXBhYmlsaXR5IGJpdCB0byBhZHZlcnRpc2Ugd2hl dGhlciB0aGlzIGlzIHBvc3NpYmxlIGZvcgpmdXR1cmUgaGFyZHdhcmUuCgpUaGlzIGNvbW1pdCBp bmNsdWRlcyBzaWduaWZpY2FudCByZXdvcmsgYW5kIGZlZWRiYWNrIGZyb20gQ2hyaXN0b3BoCkhl bGx3aWcuCgpTaWduZWQtb2ZmLWJ5OiBDaHJpc3RvcGggSGVsbHdpZyA8aGNoQGxzdC5kZT4KU2ln bmVkLW9mZi1ieTogTG9nYW4gR3VudGhvcnBlIDxsb2dhbmdAZGVsdGF0ZWUuY29tPgotLS0KIGRy aXZlcnMvcGNpL0tjb25maWcgICAgICAgIHwgIDE3ICsKIGRyaXZlcnMvcGNpL01ha2VmaWxlICAg ICAgIHwgICAxICsKIGRyaXZlcnMvcGNpL3AycGRtYS5jICAgICAgIHwgNzYxICsrKysrKysrKysr KysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKwogaW5jbHVkZS9saW51eC9tZW1yZW1h cC5oICAgfCAgIDUgKwogaW5jbHVkZS9saW51eC9tbS5oICAgICAgICAgfCAgMTggKysKIGluY2x1 ZGUvbGludXgvcGNpLXAycGRtYS5oIHwgMTAyICsrKysrKwogaW5jbHVkZS9saW51eC9wY2kuaCAg ICAgICAgfCAgIDQgKwogNyBmaWxlcyBjaGFuZ2VkLCA5MDggaW5zZXJ0aW9ucygrKQogY3JlYXRl IG1vZGUgMTAwNjQ0IGRyaXZlcnMvcGNpL3AycGRtYS5jCiBjcmVhdGUgbW9kZSAxMDA2NDQgaW5j bHVkZS9saW51eC9wY2ktcDJwZG1hLmgKCmRpZmYgLS1naXQgYS9kcml2ZXJzL3BjaS9LY29uZmln IGIvZHJpdmVycy9wY2kvS2NvbmZpZwppbmRleCA1NmZmOGY2ZDMxZmMuLmRlYjY4YmU0ZmRhYyAx MDA2NDQKLS0tIGEvZHJpdmVycy9wY2kvS2NvbmZpZworKysgYi9kcml2ZXJzL3BjaS9LY29uZmln CkBAIC0xMzIsNiArMTMyLDIzIEBAIGNvbmZpZyBQQ0lfUEFTSUQKIAogCSAgSWYgdW5zdXJlLCBz YXkgTi4KIAorY29uZmlnIFBDSV9QMlBETUEKKwlib29sICJQQ0kgcGVlci10by1wZWVyIHRyYW5z ZmVyIHN1cHBvcnQiCisJZGVwZW5kcyBvbiBQQ0kgJiYgWk9ORV9ERVZJQ0UKKwlzZWxlY3QgR0VO RVJJQ19BTExPQ0FUT1IKKwloZWxwCisJICBFbmFibGXRlSBkcml2ZXJzIHRvIGRvIFBDSSBwZWVy LXRvLXBlZXIgdHJhbnNhY3Rpb25zIHRvIGFuZCBmcm9tCisJICBCQVJzIHRoYXQgYXJlIGV4cG9z ZWQgaW4gb3RoZXIgZGV2aWNlcyB0aGF0IGFyZSB0aGUgcGFydCBvZgorCSAgdGhlIGhpZXJhcmNo eSB3aGVyZSBwZWVyLXRvLXBlZXIgRE1BIGlzIGd1YXJhbnRlZWQgYnkgdGhlIFBDSQorCSAgc3Bl Y2lmaWNhdGlvbiB0byB3b3JrIChpZS4gYW55dGhpbmcgYmVsb3cgYSBzaW5nbGUgUENJIGJyaWRn ZSkuCisKKwkgIE1hbnkgUENJZSByb290IGNvbXBsZXhlcyBkbyBub3Qgc3VwcG9ydCBQMlAgdHJh bnNhY3Rpb25zIGFuZAorCSAgaXQncyBoYXJkIHRvIHRlbGwgd2hpY2ggc3VwcG9ydCBpdCBhdCBh bGwsIHNvIGF0IHRoaXMgdGltZSwKKwkgIFAyUCBETUEgdHJhbnNhdGlvbnMgbXVzdCBiZSBiZXR3 ZWVuIGRldmljZXMgYmVoaW5kIHRoZSBzYW1lIHJvb3QKKwkgIHBvcnQuCisKKwkgIElmIHVuc3Vy ZSwgc2F5IE4uCisKIGNvbmZpZyBQQ0lfTEFCRUwKIAlkZWZfYm9vbCB5IGlmIChETUkgfHwgQUNQ SSkKIAlkZXBlbmRzIG9uIFBDSQpkaWZmIC0tZ2l0IGEvZHJpdmVycy9wY2kvTWFrZWZpbGUgYi9k cml2ZXJzL3BjaS9NYWtlZmlsZQppbmRleCAxYjJjZmU1MWU4ZDcuLjg1ZjRhNzAzYjJiZSAxMDA2 NDQKLS0tIGEvZHJpdmVycy9wY2kvTWFrZWZpbGUKKysrIGIvZHJpdmVycy9wY2kvTWFrZWZpbGUK QEAgLTI2LDYgKzI2LDcgQEAgb2JqLSQoQ09ORklHX1BDSV9TWVNDQUxMKQkrPSBzeXNjYWxsLm8K IG9iai0kKENPTkZJR19QQ0lfU1RVQikJCSs9IHBjaS1zdHViLm8KIG9iai0kKENPTkZJR19QQ0lf UEZfU1RVQikJKz0gcGNpLXBmLXN0dWIubwogb2JqLSQoQ09ORklHX1BDSV9FQ0FNKQkJKz0gZWNh bS5vCitvYmotJChDT05GSUdfUENJX1AyUERNQSkJKz0gcDJwZG1hLm8KIG9iai0kKENPTkZJR19Y RU5fUENJREVWX0ZST05URU5EKSArPSB4ZW4tcGNpZnJvbnQubwogCiAjIEVuZHBvaW50IGxpYnJh cnkgbXVzdCBiZSBpbml0aWFsaXplZCBiZWZvcmUgaXRzIHVzZXJzCmRpZmYgLS1naXQgYS9kcml2 ZXJzL3BjaS9wMnBkbWEuYyBiL2RyaXZlcnMvcGNpL3AycGRtYS5jCm5ldyBmaWxlIG1vZGUgMTAw NjQ0CmluZGV4IDAwMDAwMDAwMDAwMC4uODhhYWVjNTM1MWNkCi0tLSAvZGV2L251bGwKKysrIGIv ZHJpdmVycy9wY2kvcDJwZG1hLmMKQEAgLTAsMCArMSw3NjEgQEAKKy8vIFNQRFgtTGljZW5zZS1J ZGVudGlmaWVyOiBHUEwtMi4wCisvKgorICogUENJIFBlZXIgMiBQZWVyIERNQSBzdXBwb3J0Lgor ICoKKyAqIENvcHlyaWdodCAoYykgMjAxNi0yMDE4LCBMb2dhbiBHdW50aG9ycGUKKyAqIENvcHly aWdodCAoYykgMjAxNi0yMDE3LCBNaWNyb3NlbWkgQ29ycG9yYXRpb24KKyAqIENvcHlyaWdodCAo YykgMjAxNywgQ2hyaXN0b3BoIEhlbGx3aWcKKyAqIENvcHlyaWdodCAoYykgMjAxOCwgRWlkZXRp Y29tIEluYy4KKyAqLworCisjZGVmaW5lIHByX2ZtdChmbXQpICJwY2ktcDJwZG1hOiAiIGZtdAor I2luY2x1ZGUgPGxpbnV4L3BjaS1wMnBkbWEuaD4KKyNpbmNsdWRlIDxsaW51eC9tb2R1bGUuaD4K KyNpbmNsdWRlIDxsaW51eC9zbGFiLmg+CisjaW5jbHVkZSA8bGludXgvZ2VuYWxsb2MuaD4KKyNp bmNsdWRlIDxsaW51eC9tZW1yZW1hcC5oPgorI2luY2x1ZGUgPGxpbnV4L3BlcmNwdS1yZWZjb3Vu dC5oPgorI2luY2x1ZGUgPGxpbnV4L3JhbmRvbS5oPgorI2luY2x1ZGUgPGxpbnV4L3NlcV9idWYu aD4KKworc3RydWN0IHBjaV9wMnBkbWEgeworCXN0cnVjdCBwZXJjcHVfcmVmIGRldm1hcF9yZWY7 CisJc3RydWN0IGNvbXBsZXRpb24gZGV2bWFwX3JlZl9kb25lOworCXN0cnVjdCBnZW5fcG9vbCAq cG9vbDsKKwlib29sIHAycG1lbV9wdWJsaXNoZWQ7Cit9OworCitzdGF0aWMgdm9pZCBwY2lfcDJw ZG1hX3BlcmNwdV9yZWxlYXNlKHN0cnVjdCBwZXJjcHVfcmVmICpyZWYpCit7CisJc3RydWN0IHBj aV9wMnBkbWEgKnAycCA9CisJCWNvbnRhaW5lcl9vZihyZWYsIHN0cnVjdCBwY2lfcDJwZG1hLCBk ZXZtYXBfcmVmKTsKKworCWNvbXBsZXRlX2FsbCgmcDJwLT5kZXZtYXBfcmVmX2RvbmUpOworfQor CitzdGF0aWMgdm9pZCBwY2lfcDJwZG1hX3BlcmNwdV9raWxsKHZvaWQgKmRhdGEpCit7CisJc3Ry dWN0IHBlcmNwdV9yZWYgKnJlZiA9IGRhdGE7CisKKwlpZiAocGVyY3B1X3JlZl9pc19keWluZyhy ZWYpKQorCQlyZXR1cm47CisKKwlwZXJjcHVfcmVmX2tpbGwocmVmKTsKK30KKworc3RhdGljIHZv aWQgcGNpX3AycGRtYV9yZWxlYXNlKHZvaWQgKmRhdGEpCit7CisJc3RydWN0IHBjaV9kZXYgKnBk ZXYgPSBkYXRhOworCisJaWYgKCFwZGV2LT5wMnBkbWEpCisJCXJldHVybjsKKworCXdhaXRfZm9y X2NvbXBsZXRpb24oJnBkZXYtPnAycGRtYS0+ZGV2bWFwX3JlZl9kb25lKTsKKwlwZXJjcHVfcmVm X2V4aXQoJnBkZXYtPnAycGRtYS0+ZGV2bWFwX3JlZik7CisKKwlnZW5fcG9vbF9kZXN0cm95KHBk ZXYtPnAycGRtYS0+cG9vbCk7CisJcGRldi0+cDJwZG1hID0gTlVMTDsKK30KKworc3RhdGljIGlu dCBwY2lfcDJwZG1hX3NldHVwKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQoreworCWludCBlcnJvciA9 IC1FTk9NRU07CisJc3RydWN0IHBjaV9wMnBkbWEgKnAycDsKKworCXAycCA9IGRldm1fa3phbGxv YygmcGRldi0+ZGV2LCBzaXplb2YoKnAycCksIEdGUF9LRVJORUwpOworCWlmICghcDJwKQorCQly ZXR1cm4gLUVOT01FTTsKKworCXAycC0+cG9vbCA9IGdlbl9wb29sX2NyZWF0ZShQQUdFX1NISUZU LCBkZXZfdG9fbm9kZSgmcGRldi0+ZGV2KSk7CisJaWYgKCFwMnAtPnBvb2wpCisJCWdvdG8gb3V0 OworCisJaW5pdF9jb21wbGV0aW9uKCZwMnAtPmRldm1hcF9yZWZfZG9uZSk7CisJZXJyb3IgPSBw ZXJjcHVfcmVmX2luaXQoJnAycC0+ZGV2bWFwX3JlZiwKKwkJCXBjaV9wMnBkbWFfcGVyY3B1X3Jl bGVhc2UsIDAsIEdGUF9LRVJORUwpOworCWlmIChlcnJvcikKKwkJZ290byBvdXRfcG9vbF9kZXN0 cm95OworCisJcGVyY3B1X3JlZl9zd2l0Y2hfdG9fYXRvbWljX3N5bmMoJnAycC0+ZGV2bWFwX3Jl Zik7CisKKwllcnJvciA9IGRldm1fYWRkX2FjdGlvbl9vcl9yZXNldCgmcGRldi0+ZGV2LCBwY2lf cDJwZG1hX3JlbGVhc2UsIHBkZXYpOworCWlmIChlcnJvcikKKwkJZ290byBvdXRfcG9vbF9kZXN0 cm95OworCisJcGRldi0+cDJwZG1hID0gcDJwOworCisJcmV0dXJuIDA7CisKK291dF9wb29sX2Rl c3Ryb3k6CisJZ2VuX3Bvb2xfZGVzdHJveShwMnAtPnBvb2wpOworb3V0OgorCWRldm1fa2ZyZWUo JnBkZXYtPmRldiwgcDJwKTsKKwlyZXR1cm4gZXJyb3I7Cit9CisKKy8qKgorICogcGNpX3AycGRt YV9hZGRfcmVzb3VyY2UgLSBhZGQgbWVtb3J5IGZvciB1c2UgYXMgcDJwIG1lbW9yeQorICogQHBk ZXY6IHRoZSBkZXZpY2UgdG8gYWRkIHRoZSBtZW1vcnkgdG8KKyAqIEBiYXI6IFBDSSBCQVIgdG8g YWRkCisgKiBAc2l6ZTogc2l6ZSBvZiB0aGUgbWVtb3J5IHRvIGFkZCwgbWF5IGJlIHplcm8gdG8g dXNlIHRoZSB3aG9sZSBCQVIKKyAqIEBvZmZzZXQ6IG9mZnNldCBpbnRvIHRoZSBQQ0kgQkFSCisg KgorICogVGhlIG1lbW9yeSB3aWxsIGJlIGdpdmVuIFpPTkVfREVWSUNFIHN0cnVjdCBwYWdlcyBz byB0aGF0IGl0IG1heQorICogYmUgdXNlZCB3aXRoIGFueSBETUEgcmVxdWVzdC4KKyAqLworaW50 IHBjaV9wMnBkbWFfYWRkX3Jlc291cmNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2LCBpbnQgYmFyLCBz aXplX3Qgc2l6ZSwKKwkJCSAgICB1NjQgb2Zmc2V0KQoreworCXN0cnVjdCBkZXZfcGFnZW1hcCAq cGdtYXA7CisJdm9pZCAqYWRkcjsKKwlpbnQgZXJyb3I7CisKKwlpZiAoIShwY2lfcmVzb3VyY2Vf ZmxhZ3MocGRldiwgYmFyKSAmIElPUkVTT1VSQ0VfTUVNKSkKKwkJcmV0dXJuIC1FSU5WQUw7CisK KwlpZiAob2Zmc2V0ID49IHBjaV9yZXNvdXJjZV9sZW4ocGRldiwgYmFyKSkKKwkJcmV0dXJuIC1F SU5WQUw7CisKKwlpZiAoIXNpemUpCisJCXNpemUgPSBwY2lfcmVzb3VyY2VfbGVuKHBkZXYsIGJh cikgLSBvZmZzZXQ7CisKKwlpZiAoc2l6ZSArIG9mZnNldCA+IHBjaV9yZXNvdXJjZV9sZW4ocGRl diwgYmFyKSkKKwkJcmV0dXJuIC1FSU5WQUw7CisKKwlpZiAoIXBkZXYtPnAycGRtYSkgeworCQll cnJvciA9IHBjaV9wMnBkbWFfc2V0dXAocGRldik7CisJCWlmIChlcnJvcikKKwkJCXJldHVybiBl cnJvcjsKKwl9CisKKwlwZ21hcCA9IGRldm1fa3phbGxvYygmcGRldi0+ZGV2LCBzaXplb2YoKnBn bWFwKSwgR0ZQX0tFUk5FTCk7CisJaWYgKCFwZ21hcCkKKwkJcmV0dXJuIC1FTk9NRU07CisKKwlw Z21hcC0+cmVzLnN0YXJ0ID0gcGNpX3Jlc291cmNlX3N0YXJ0KHBkZXYsIGJhcikgKyBvZmZzZXQ7 CisJcGdtYXAtPnJlcy5lbmQgPSBwZ21hcC0+cmVzLnN0YXJ0ICsgc2l6ZSAtIDE7CisJcGdtYXAt PnJlcy5mbGFncyA9IHBjaV9yZXNvdXJjZV9mbGFncyhwZGV2LCBiYXIpOworCXBnbWFwLT5yZWYg PSAmcGRldi0+cDJwZG1hLT5kZXZtYXBfcmVmOworCXBnbWFwLT50eXBlID0gTUVNT1JZX0RFVklD RV9QQ0lfUDJQRE1BOworCisJYWRkciA9IGRldm1fbWVtcmVtYXBfcGFnZXMoJnBkZXYtPmRldiwg cGdtYXApOworCWlmIChJU19FUlIoYWRkcikpIHsKKwkJZXJyb3IgPSBQVFJfRVJSKGFkZHIpOwor CQlnb3RvIHBnbWFwX2ZyZWU7CisJfQorCisJZXJyb3IgPSBnZW5fcG9vbF9hZGRfdmlydChwZGV2 LT5wMnBkbWEtPnBvb2wsICh1bnNpZ25lZCBsb25nKWFkZHIsCisJCQlwY2lfYnVzX2FkZHJlc3Mo cGRldiwgYmFyKSArIG9mZnNldCwKKwkJCXJlc291cmNlX3NpemUoJnBnbWFwLT5yZXMpLCBkZXZf dG9fbm9kZSgmcGRldi0+ZGV2KSk7CisJaWYgKGVycm9yKQorCQlnb3RvIHBnbWFwX2ZyZWU7CisK KwllcnJvciA9IGRldm1fYWRkX2FjdGlvbl9vcl9yZXNldCgmcGRldi0+ZGV2LCBwY2lfcDJwZG1h X3BlcmNwdV9raWxsLAorCQkJCQkgICZwZGV2LT5wMnBkbWEtPmRldm1hcF9yZWYpOworCWlmIChl cnJvcikKKwkJZ290byBwZ21hcF9mcmVlOworCisJcGNpX2luZm8ocGRldiwgImFkZGVkIHBlZXIt dG8tcGVlciBETUEgbWVtb3J5ICVwUlxuIiwKKwkJICZwZ21hcC0+cmVzKTsKKworCXJldHVybiAw OworCitwZ21hcF9mcmVlOgorCWRldnJlc19mcmVlKHBnbWFwKTsKKwlyZXR1cm4gZXJyb3I7Cit9 CitFWFBPUlRfU1lNQk9MX0dQTChwY2lfcDJwZG1hX2FkZF9yZXNvdXJjZSk7CisKK3N0YXRpYyBz dHJ1Y3QgcGNpX2RldiAqZmluZF9wYXJlbnRfcGNpX2RldihzdHJ1Y3QgZGV2aWNlICpkZXYpCit7 CisJc3RydWN0IGRldmljZSAqcGFyZW50OworCisJZGV2ID0gZ2V0X2RldmljZShkZXYpOworCisJ d2hpbGUgKGRldikgeworCQlpZiAoZGV2X2lzX3BjaShkZXYpKQorCQkJcmV0dXJuIHRvX3BjaV9k ZXYoZGV2KTsKKworCQlwYXJlbnQgPSBnZXRfZGV2aWNlKGRldi0+cGFyZW50KTsKKwkJcHV0X2Rl dmljZShkZXYpOworCQlkZXYgPSBwYXJlbnQ7CisJfQorCisJcmV0dXJuIE5VTEw7Cit9CisKKy8q CisgKiBDaGVjayBpZiBhIFBDSSBicmlkZ2UgaGFzIGl0J3MgQUNTIHJlZGlyZWN0aW9uIGJpdHMg c2V0IHRvIHJlZGlyZWN0IFAyUAorICogVExQcyB1cHN0cmVhbSB2aWEgQUNTLiBSZXR1cm5zIDEg aWYgdGhlIHBhY2tldHMgd2lsbCBiZSByZWRpcmVjdGVkCisgKiB1cHN0cmVhbSwgMCBvdGhlcndp c2UuCisgKi8KK3N0YXRpYyBpbnQgcGNpX2JyaWRnZV9oYXNfYWNzX3JlZGlyKHN0cnVjdCBwY2lf ZGV2ICpkZXYpCit7CisJaW50IHBvczsKKwl1MTYgY3RybDsKKworCXBvcyA9IHBjaV9maW5kX2V4 dF9jYXBhYmlsaXR5KGRldiwgUENJX0VYVF9DQVBfSURfQUNTKTsKKwlpZiAoIXBvcykKKwkJcmV0 dXJuIDA7CisKKwlwY2lfcmVhZF9jb25maWdfd29yZChkZXYsIHBvcyArIFBDSV9BQ1NfQ1RSTCwg JmN0cmwpOworCisJaWYgKGN0cmwgJiAoUENJX0FDU19SUiB8IFBDSV9BQ1NfQ1IgfCBQQ0lfQUNT X0VDKSkKKwkJcmV0dXJuIDE7CisKKwlyZXR1cm4gMDsKK30KKworc3RhdGljIHZvaWQgc2VxX2J1 Zl9wcmludF9idXNfZGV2Zm4oc3RydWN0IHNlcV9idWYgKmJ1Ziwgc3RydWN0IHBjaV9kZXYgKmRl dikKK3sKKwlpZiAoIWJ1ZikKKwkJcmV0dXJuOworCisJc2VxX2J1Zl9wcmludGYoYnVmLCAiJTA0 eDolMDJ4OiUwMnguJXg7IiwgcGNpX2RvbWFpbl9ucihkZXYtPmJ1cyksCisJCSAgICAgICBkZXYt PmJ1cy0+bnVtYmVyLCBQQ0lfU0xPVChkZXYtPmRldmZuKSwKKwkJICAgICAgIFBDSV9GVU5DKGRl di0+ZGV2Zm4pKTsKK30KKworLyoKKyAqIEZpbmQgdGhlIGRpc3RhbmNlIHRocm91Z2ggdGhlIG5l YXJlc3QgY29tbW9uIHVwc3RyZWFtIGJyaWRnZSBiZXR3ZWVuCisgKiB0d28gUENJIGRldmljZXMu CisgKgorICogSWYgdGhlIHR3byBkZXZpY2VzIGFyZSB0aGUgc2FtZSBkZXZpY2UgdGhlbiAwIHdp bGwgYmUgcmV0dXJuZWQuCisgKgorICogSWYgdGhlcmUgYXJlIHR3byB2aXJ0dWFsIGZ1bmN0aW9u cyBvZiB0aGUgc2FtZSBkZXZpY2UgYmVoaW5kIHRoZSBzYW1lCisgKiBicmlkZ2UgcG9ydCB0aGVu IDIgd2lsbCBiZSByZXR1cm5lZCAob25lIHN0ZXAgZG93biB0byB0aGUgUENJZSBzd2l0Y2gsCisg KiB0aGVuIG9uZSBzdGVwIGJhY2sgdG8gdGhlIHNhbWUgZGV2aWNlKS4KKyAqCisgKiBJbiB0aGUg Y2FzZSB3aGVyZSB0d28gZGV2aWNlcyBhcmUgY29ubmVjdGVkIHRvIHRoZSBzYW1lIFBDSWUgc3dp dGNoLCB0aGUKKyAqIHZhbHVlIDQgd2lsbCBiZSByZXR1cm5lZC4gVGhpcyBjb3JyZXNwb25kcyB0 byB0aGUgZm9sbG93aW5nIFBDSSB0cmVlOgorICoKKyAqICAgICAtKyAgUm9vdCBQb3J0CisgKiAg ICAgIFwrIFN3aXRjaCBVcHN0cmVhbSBQb3J0CisgKiAgICAgICArLSsgU3dpdGNoIERvd25zdHJl YW0gUG9ydAorICogICAgICAgKyBcLSBEZXZpY2UgQQorICogICAgICAgXC0rIFN3aXRjaCBEb3du c3RyZWFtIFBvcnQKKyAqICAgICAgICAgXC0gRGV2aWNlIEIKKyAqCisgKiBUaGUgZGlzdGFuY2Ug aXMgNCBiZWNhdXNlIHdlIHRyYXZlcnNlIGZyb20gRGV2aWNlIEEgdGhyb3VnaCB0aGUgZG93bnN0 cmVhbQorICogcG9ydCBvZiB0aGUgc3dpdGNoLCB0byB0aGUgY29tbW9uIHVwc3RyZWFtIHBvcnQs IGJhY2sgdXAgdG8gdGhlIHNlY29uZAorICogZG93bnN0cmVhbSBwb3J0IGFuZCB0aGVuIHRvIERl dmljZSBCLgorICoKKyAqIEFueSB0d28gZGV2aWNlcyB0aGF0IGRvbid0IGhhdmUgYSBjb21tb24g dXBzdHJlYW0gYnJpZGdlIHdpbGwgcmV0dXJuIC0xLgorICogSW4gdGhpcyB3YXkgZGV2aWNlcyBv biBzZXBhcmF0ZSBQQ0llIHJvb3QgcG9ydHMgd2lsbCBiZSByZWplY3RlZCwgd2hpY2gKKyAqIGlz IHdoYXQgd2Ugd2FudCBmb3IgcGVlci10by1wZWVyIHNlZWluZyBlYWNoIFBDSWUgcm9vdCBwb3J0 IGRlZmluZXMgYQorICogc2VwYXJhdGUgaGllcmFyY2h5IGRvbWFpbiBhbmQgdGhlcmUncyBubyB3 YXkgdG8gZGV0ZXJtaW5lIHdoZXRoZXIgdGhlIHJvb3QKKyAqIGNvbXBsZXggc3VwcG9ydHMgZm9y d2FyZGluZyBiZXR3ZWVuIHRoZW0uCisgKgorICogSW4gdGhlIGNhc2Ugd2hlcmUgdHdvIGRldmlj ZXMgYXJlIGNvbm5lY3RlZCB0byBkaWZmZXJlbnQgUENJZSBzd2l0Y2hlcywKKyAqIHRoaXMgZnVu Y3Rpb24gd2lsbCBzdGlsbCByZXR1cm4gYSBwb3NpdGl2ZSBkaXN0YW5jZSBhcyBsb25nIGFzIGJv dGgKKyAqIHN3aXRjaGVzIGV2ZW51dGFsbHkgaGF2ZSBhIGNvbW1vbiB1cHN0cmVhbSBicmlkZ2Uu IE5vdGUgdGhpcyBjb3ZlcnMKKyAqIHRoZSBjYXNlIG9mIHVzaW5nIG11bHRpcGxlIFBDSWUgc3dp dGNoZXMgdG8gYWNoaWV2ZSBhIGRlc2lyZWQgbGV2ZWwgb2YKKyAqIGZhbi1vdXQgZnJvbSBhIHJv b3QgcG9ydC4gVGhlIGV4YWN0IGRpc3RhbmNlIHdpbGwgYmUgYSBmdW5jdGlvbiBvZiB0aGUKKyAq IG51bWJlciBvZiBzd2l0Y2hlcyBiZXR3ZWVuIERldmljZSBBIGFuZCBEZXZpY2UgQi4KKyAqCisg KiBJZiBhIGJyaWRnZSB3aGljaCBoYXMgYW55IEFDUyByZWRpcmVjdGlvbiBiaXRzIHNldCBpcyBp biB0aGUgcGF0aAorICogdGhlbiB0aGlzIGZ1bmN0aW9ucyB3aWxsIHJldHVybiAtMi4gVGhpcyBp cyBzbyB3ZSByZWplY3QgYW55CisgKiBjYXNlcyB3aGVyZSB0aGUgVExQcyBhcmUgZm9yd2FyZGVk IHVwIGludG8gdGhlIHJvb3QgY29tcGxleC4KKyAqIEluIHRoaXMgY2FzZSwgYSBsaXN0IG9mIGFs bCBpbmZyaW5naW5nIGJyaWRnZSBhZGRyZXNzZXMgd2lsbCBiZQorICogcG9wdWxhdGVkIGluIGFj c19saXN0IChhc3N1bWluZyBpdCdzIG5vbi1udWxsKSBmb3IgcHJpbnRrIHB1cnBvc2VzLgorICov CitzdGF0aWMgaW50IHVwc3RyZWFtX2JyaWRnZV9kaXN0YW5jZShzdHJ1Y3QgcGNpX2RldiAqYSwK KwkJCQkgICAgc3RydWN0IHBjaV9kZXYgKmIsCisJCQkJICAgIHN0cnVjdCBzZXFfYnVmICphY3Nf bGlzdCkKK3sKKwlpbnQgZGlzdF9hID0gMDsKKwlpbnQgZGlzdF9iID0gMDsKKwlzdHJ1Y3QgcGNp X2RldiAqYmIgPSBOVUxMOworCWludCBhY3NfY250ID0gMDsKKworCS8qCisJICogTm90ZSwgd2Ug ZG9uJ3QgbmVlZCB0byB0YWtlIHJlZmVyZW5jZXMgdG8gZGV2aWNlcyByZXR1cm5lZCBieQorCSAq IHBjaV91cHN0cmVhbV9icmlkZ2UoKSBzZWVpbmcgd2UgaG9sZCBhIHJlZmVyZW5jZSB0byBhIGNo aWxkCisJICogZGV2aWNlIHdoaWNoIHdpbGwgYWxyZWFkeSBob2xkIGEgcmVmZXJlbmNlIHRvIHRo ZSB1cHN0cmVhbSBicmlkZ2UuCisJICovCisKKwl3aGlsZSAoYSkgeworCQlkaXN0X2IgPSAwOwor CisJCWlmIChwY2lfYnJpZGdlX2hhc19hY3NfcmVkaXIoYSkpIHsKKwkJCXNlcV9idWZfcHJpbnRf YnVzX2RldmZuKGFjc19saXN0LCBhKTsKKwkJCWFjc19jbnQrKzsKKwkJfQorCisJCWJiID0gYjsK KworCQl3aGlsZSAoYmIpIHsKKwkJCWlmIChhID09IGJiKQorCQkJCWdvdG8gY2hlY2tfYl9wYXRo X2FjczsKKworCQkJYmIgPSBwY2lfdXBzdHJlYW1fYnJpZGdlKGJiKTsKKwkJCWRpc3RfYisrOwor CQl9CisKKwkJYSA9IHBjaV91cHN0cmVhbV9icmlkZ2UoYSk7CisJCWRpc3RfYSsrOworCX0KKwor CXJldHVybiAtMTsKKworY2hlY2tfYl9wYXRoX2FjczoKKwliYiA9IGI7CisKKwl3aGlsZSAoYmIp IHsKKwkJaWYgKGEgPT0gYmIpCisJCQlicmVhazsKKworCQlpZiAocGNpX2JyaWRnZV9oYXNfYWNz X3JlZGlyKGJiKSkgeworCQkJc2VxX2J1Zl9wcmludF9idXNfZGV2Zm4oYWNzX2xpc3QsIGJiKTsK KwkJCWFjc19jbnQrKzsKKwkJfQorCisJCWJiID0gcGNpX3Vwc3RyZWFtX2JyaWRnZShiYik7CisJ fQorCisJaWYgKGFjc19jbnQpCisJCXJldHVybiAtMjsKKworCXJldHVybiBkaXN0X2EgKyBkaXN0 X2I7Cit9CisKK3N0YXRpYyBpbnQgdXBzdHJlYW1fYnJpZGdlX2Rpc3RhbmNlX3dhcm4oc3RydWN0 IHBjaV9kZXYgKnByb3ZpZGVyLAorCQkJCQkgc3RydWN0IHBjaV9kZXYgKmNsaWVudCkKK3sKKwlz dHJ1Y3Qgc2VxX2J1ZiBhY3NfbGlzdDsKKwlpbnQgcmV0OworCisJc2VxX2J1Zl9pbml0KCZhY3Nf bGlzdCwga21hbGxvYyhQQUdFX1NJWkUsIEdGUF9LRVJORUwpLCBQQUdFX1NJWkUpOworCisJcmV0 ID0gdXBzdHJlYW1fYnJpZGdlX2Rpc3RhbmNlKHByb3ZpZGVyLCBjbGllbnQsICZhY3NfbGlzdCk7 CisJaWYgKHJldCA9PSAtMikgeworCQlwY2lfd2FybihjbGllbnQsICJjYW5ub3QgYmUgdXNlZCBm b3IgcGVlci10by1wZWVyIERNQSBhcyBBQ1MgcmVkaXJlY3QgaXMgc2V0IGJldHdlZW4gdGhlIGNs aWVudCBhbmQgcHJvdmlkZXJcbiIpOworCQkvKiBEcm9wIGZpbmFsIHNlbWljb2xvbiAqLworCQlh Y3NfbGlzdC5idWZmZXJbYWNzX2xpc3QubGVuLTFdID0gMDsKKwkJcGNpX3dhcm4oY2xpZW50LCAi dG8gZGlzYWJsZSBBQ1MgcmVkaXJlY3QgZm9yIHRoaXMgcGF0aCwgYWRkIHRoZSBrZXJuZWwgcGFy YW1ldGVyOiBwY2k9ZGlzYWJsZV9hY3NfcmVkaXI9JXNcbiIsCisJCQkgYWNzX2xpc3QuYnVmZmVy KTsKKworCX0gZWxzZSBpZiAocmV0IDwgMCkgeworCQlwY2lfd2FybihjbGllbnQsICJjYW5ub3Qg YmUgdXNlZCBmb3IgcGVlci10by1wZWVyIERNQSBhcyB0aGUgY2xpZW50IGFuZCBwcm92aWRlciBk byBub3Qgc2hhcmUgYW4gdXBzdHJlYW0gYnJpZGdlXG4iKTsKKwl9CisKKwlrZnJlZShhY3NfbGlz dC5idWZmZXIpOworCisJcmV0dXJuIHJldDsKK30KKworc3RydWN0IHBjaV9wMnBkbWFfY2xpZW50 IHsKKwlzdHJ1Y3QgbGlzdF9oZWFkIGxpc3Q7CisJc3RydWN0IHBjaV9kZXYgKmNsaWVudDsKKwlz dHJ1Y3QgcGNpX2RldiAqcHJvdmlkZXI7Cit9OworCisvKioKKyAqIHBjaV9wMnBkbWFfYWRkX2Ns aWVudCAtIGFsbG9jYXRlIGEgbmV3IGVsZW1lbnQgaW4gYSBjbGllbnQgZGV2aWNlIGxpc3QKKyAq IEBoZWFkOiBsaXN0IGhlYWQgb2YgcDJwZG1hIGNsaWVudHMKKyAqIEBkZXY6IGRldmljZSB0byBh ZGQgdG8gdGhlIGxpc3QKKyAqCisgKiBUaGlzIGFkZHMgQGRldiB0byBhIGxpc3Qgb2YgY2xpZW50 cyB1c2VkIGJ5IGEgcDJwZG1hIGRldmljZS4KKyAqIFRoaXMgbGlzdCBzaG91bGQgYmUgcGFzc2Vk IHRvIHBjaV9wMnBtZW1fZmluZCgpLiBPbmNlIHBjaV9wMnBtZW1fZmluZCgpIGhhcworICogYmVl biBjYWxsZWQgc3VjY2Vzc2Z1bGx5LCB0aGUgbGlzdCB3aWxsIGJlIGJvdW5kIHRvIGEgc3BlY2lm aWMgcDJwZG1hCisgKiBkZXZpY2UgYW5kIG5ldyBjbGllbnRzIGNhbiBvbmx5IGJlIGFkZGVkIHRv IHRoZSBsaXN0IGlmIHRoZXkgYXJlCisgKiBzdXBwb3J0ZWQgYnkgdGhhdCBwMnBkbWEgZGV2aWNl LgorICoKKyAqIFRoZSBjYWxsZXIgaXMgZXhwZWN0ZWQgdG8gaGF2ZSBhIGxvY2sgd2hpY2ggcHJv dGVjdHMgQGhlYWQgYXMgbmVjZXNzYXJ5CisgKiBzbyB0aGF0IG5vbmUgb2YgdGhlIHBjaV9wMnAg ZnVuY3Rpb25zIGNhbiBiZSBjYWxsZWQgY29uY3VycmVudGx5CisgKiBvbiB0aGF0IGxpc3QuCisg KgorICogUmV0dXJucyAwIGlmIHRoZSBjbGllbnQgd2FzIHN1Y2Nlc3NmdWxseSBhZGRlZC4KKyAq LworaW50IHBjaV9wMnBkbWFfYWRkX2NsaWVudChzdHJ1Y3QgbGlzdF9oZWFkICpoZWFkLCBzdHJ1 Y3QgZGV2aWNlICpkZXYpCit7CisJc3RydWN0IHBjaV9wMnBkbWFfY2xpZW50ICppdGVtLCAqbmV3 X2l0ZW07CisJc3RydWN0IHBjaV9kZXYgKnByb3ZpZGVyID0gTlVMTDsKKwlzdHJ1Y3QgcGNpX2Rl diAqY2xpZW50OworCWludCByZXQ7CisKKwlpZiAoSVNfRU5BQkxFRChDT05GSUdfRE1BX1ZJUlRf T1BTKSAmJiBkZXYtPmRtYV9vcHMgPT0gJmRtYV92aXJ0X29wcykgeworCQlkZXZfd2FybihkZXYs ICJjYW5ub3QgYmUgdXNlZCBmb3IgcGVlci10by1wZWVyIERNQSBiZWNhdXNlIHRoZSBkcml2ZXIg bWFrZXMgdXNlIG9mIGRtYV92aXJ0X29wc1xuIik7CisJCXJldHVybiAtRU5PREVWOworCX0KKwor CWNsaWVudCA9IGZpbmRfcGFyZW50X3BjaV9kZXYoZGV2KTsKKwlpZiAoIWNsaWVudCkgeworCQlk ZXZfd2FybihkZXYsICJjYW5ub3QgYmUgdXNlZCBmb3IgcGVlci10by1wZWVyIERNQSBhcyBpdCBp cyBub3QgYSBQQ0kgZGV2aWNlXG4iKTsKKwkJcmV0dXJuIC1FTk9ERVY7CisJfQorCisJaXRlbSA9 IGxpc3RfZmlyc3RfZW50cnlfb3JfbnVsbChoZWFkLCBzdHJ1Y3QgcGNpX3AycGRtYV9jbGllbnQs IGxpc3QpOworCWlmIChpdGVtICYmIGl0ZW0tPnByb3ZpZGVyKSB7CisJCXByb3ZpZGVyID0gaXRl bS0+cHJvdmlkZXI7CisKKwkJcmV0ID0gdXBzdHJlYW1fYnJpZGdlX2Rpc3RhbmNlX3dhcm4ocHJv dmlkZXIsIGNsaWVudCk7CisJCWlmIChyZXQgPCAwKSB7CisJCQlyZXQgPSAtRVhERVY7CisJCQln b3RvIHB1dF9jbGllbnQ7CisJCX0KKwl9CisKKwluZXdfaXRlbSA9IGt6YWxsb2Moc2l6ZW9mKCpu ZXdfaXRlbSksIEdGUF9LRVJORUwpOworCWlmICghbmV3X2l0ZW0pIHsKKwkJcmV0ID0gLUVOT01F TTsKKwkJZ290byBwdXRfY2xpZW50OworCX0KKworCW5ld19pdGVtLT5jbGllbnQgPSBjbGllbnQ7 CisJbmV3X2l0ZW0tPnByb3ZpZGVyID0gcGNpX2Rldl9nZXQocHJvdmlkZXIpOworCisJbGlzdF9h ZGRfdGFpbCgmbmV3X2l0ZW0tPmxpc3QsIGhlYWQpOworCisJcmV0dXJuIDA7CisKK3B1dF9jbGll bnQ6CisJcGNpX2Rldl9wdXQoY2xpZW50KTsKKwlyZXR1cm4gcmV0OworfQorRVhQT1JUX1NZTUJP TF9HUEwocGNpX3AycGRtYV9hZGRfY2xpZW50KTsKKworc3RhdGljIHZvaWQgcGNpX3AycGRtYV9j bGllbnRfZnJlZShzdHJ1Y3QgcGNpX3AycGRtYV9jbGllbnQgKml0ZW0pCit7CisJbGlzdF9kZWwo Jml0ZW0tPmxpc3QpOworCXBjaV9kZXZfcHV0KGl0ZW0tPmNsaWVudCk7CisJcGNpX2Rldl9wdXQo aXRlbS0+cHJvdmlkZXIpOworCWtmcmVlKGl0ZW0pOworfQorCisvKioKKyAqIHBjaV9wMnBkbWFf cmVtb3ZlX2NsaWVudCAtIHJlbW92ZSBhbmQgZnJlZSBhIHAycGRtYSBjbGllbnQKKyAqIEBoZWFk OiBsaXN0IGhlYWQgb2YgcDJwZG1hIGNsaWVudHMKKyAqIEBkZXY6IGRldmljZSB0byByZW1vdmUg ZnJvbSB0aGUgbGlzdAorICoKKyAqIFRoaXMgcmVtb3ZlcyBAZGV2IGZyb20gYSBsaXN0IG9mIGNs aWVudHMgdXNlZCBieSBhIHAycGRtYSBkZXZpY2UuCisgKiBUaGUgY2FsbGVyIGlzIGV4cGVjdGVk IHRvIGhhdmUgYSBsb2NrIHdoaWNoIHByb3RlY3RzIEBoZWFkIGFzIG5lY2Vzc2FyeQorICogc28g dGhhdCBub25lIG9mIHRoZSBwY2lfcDJwIGZ1bmN0aW9ucyBjYW4gYmUgY2FsbGVkIGNvbmN1cnJl bnRseQorICogb24gdGhhdCBsaXN0LgorICovCit2b2lkIHBjaV9wMnBkbWFfcmVtb3ZlX2NsaWVu dChzdHJ1Y3QgbGlzdF9oZWFkICpoZWFkLCBzdHJ1Y3QgZGV2aWNlICpkZXYpCit7CisJc3RydWN0 IHBjaV9wMnBkbWFfY2xpZW50ICpwb3MsICp0bXA7CisJc3RydWN0IHBjaV9kZXYgKnBkZXY7CisK KwlwZGV2ID0gZmluZF9wYXJlbnRfcGNpX2RldihkZXYpOworCWlmICghcGRldikKKwkJcmV0dXJu OworCisJbGlzdF9mb3JfZWFjaF9lbnRyeV9zYWZlKHBvcywgdG1wLCBoZWFkLCBsaXN0KSB7CisJ CWlmIChwb3MtPmNsaWVudCAhPSBwZGV2KQorCQkJY29udGludWU7CisKKwkJcGNpX3AycGRtYV9j bGllbnRfZnJlZShwb3MpOworCX0KKworCXBjaV9kZXZfcHV0KHBkZXYpOworfQorRVhQT1JUX1NZ TUJPTF9HUEwocGNpX3AycGRtYV9yZW1vdmVfY2xpZW50KTsKKworLyoqCisgKiBwY2lfcDJwZG1h X2NsaWVudF9saXN0X2ZyZWUgLSBmcmVlIGFuIGVudGlyZSBsaXN0IG9mIHAycGRtYSBjbGllbnRz CisgKiBAaGVhZDogbGlzdCBoZWFkIG9mIHAycGRtYSBjbGllbnRzCisgKgorICogVGhpcyByZW1v dmVzIGFsbCBkZXZpY2VzIGluIGEgbGlzdCBvZiBjbGllbnRzIHVzZWQgYnkgYSBwMnBkbWEgZGV2 aWNlLgorICogVGhlIGNhbGxlciBpcyBleHBlY3RlZCB0byBoYXZlIGEgbG9jayB3aGljaCBwcm90 ZWN0cyBAaGVhZCBhcyBuZWNlc3NhcnkKKyAqIHNvIHRoYXQgbm9uZSBvZiB0aGUgcGNpX3AycGRt YSBmdW5jdGlvbnMgY2FuIGJlIGNhbGxlZCBjb25jdXJyZW50bHkKKyAqIG9uIHRoYXQgbGlzdC4K KyAqLwordm9pZCBwY2lfcDJwZG1hX2NsaWVudF9saXN0X2ZyZWUoc3RydWN0IGxpc3RfaGVhZCAq aGVhZCkKK3sKKwlzdHJ1Y3QgcGNpX3AycGRtYV9jbGllbnQgKnBvcywgKnRtcDsKKworCWxpc3Rf Zm9yX2VhY2hfZW50cnlfc2FmZShwb3MsIHRtcCwgaGVhZCwgbGlzdCkKKwkJcGNpX3AycGRtYV9j bGllbnRfZnJlZShwb3MpOworfQorRVhQT1JUX1NZTUJPTF9HUEwocGNpX3AycGRtYV9jbGllbnRf bGlzdF9mcmVlKTsKKworLyoqCisgKiBwY2lfcDJwZG1hX2Rpc3RhbmNlIC0gRGV0ZXJtaXZlIHRo ZSBjdW11bGF0aXZlIGRpc3RhbmNlIGJldHdlZW4KKyAqCWEgcDJwZG1hIHByb3ZpZGVyIGFuZCB0 aGUgY2xpZW50cyBpbiB1c2UuCisgKiBAcHJvdmlkZXI6IHAycGRtYSBwcm92aWRlciB0byBjaGVj ayBhZ2FpbnN0IHRoZSBjbGllbnQgbGlzdAorICogQGNsaWVudHM6IGxpc3Qgb2YgZGV2aWNlcyB0 byBjaGVjayAoTlVMTC10ZXJtaW5hdGVkKQorICogQHZlcmJvc2U6IGlmIHRydWUsIHByaW50IHdh cm5pbmdzIGZvciBkZXZpY2VzIHdoZW4gd2UgcmV0dXJuIC0xCisgKgorICogUmV0dXJucyAtMSBp ZiBhbnkgb2YgdGhlIGNsaWVudHMgYXJlIG5vdCBjb21wYXRpYmxlIChiZWhpbmQgdGhlIHNhbWUK KyAqIHJvb3QgcG9ydCBhcyB0aGUgcHJvdmlkZXIpLCBvdGhlcndpc2UgcmV0dXJucyBhIHBvc2l0 aXZlIG51bWJlciB3aGVyZQorICogdGhlIGxvd2VyIG51bWJlciBpcyB0aGUgcHJlZmVycmFibGUg Y2hvaWNlLiAoSWYgdGhlcmUncyBvbmUgY2xpZW50CisgKiB0aGF0J3MgdGhlIHNhbWUgYXMgdGhl IHByb3ZpZGVyIGl0IHdpbGwgcmV0dXJuIDAsIHdoaWNoIGlzIGJlc3QgY2hvaWNlKS4KKyAqCisg KiBGb3Igbm93LCAiY29tcGF0aWJsZSIgbWVhbnMgdGhlIHByb3ZpZGVyIGFuZCB0aGUgY2xpZW50 cyBhcmUgYWxsIGJlaGluZAorICogdGhlIHNhbWUgUENJIHJvb3QgcG9ydC4gVGhpcyBjdXRzIG91 dCBjYXNlcyB0aGF0IG1heSB3b3JrIGJ1dCBpcyBzYWZlc3QKKyAqIGZvciB0aGUgdXNlci4gRnV0 dXJlIHdvcmsgY2FuIGV4cGFuZCB0aGlzIHRvIHdoaXRlLWxpc3Qgcm9vdCBjb21wbGV4ZXMgdGhh dAorICogY2FuIHNhZmVseSBmb3J3YXJkIGJldHdlZW4gZWFjaCBwb3J0cy4KKyAqLworaW50IHBj aV9wMnBkbWFfZGlzdGFuY2Uoc3RydWN0IHBjaV9kZXYgKnByb3ZpZGVyLCBzdHJ1Y3QgbGlzdF9o ZWFkICpjbGllbnRzLAorCQkJYm9vbCB2ZXJib3NlKQoreworCXN0cnVjdCBwY2lfcDJwZG1hX2Ns aWVudCAqcG9zOworCWludCByZXQ7CisJaW50IGRpc3RhbmNlID0gMDsKKwlib29sIG5vdF9zdXBw b3J0ZWQgPSBmYWxzZTsKKworCWlmIChsaXN0X2VtcHR5KGNsaWVudHMpKQorCQlyZXR1cm4gLTE7 CisKKwlsaXN0X2Zvcl9lYWNoX2VudHJ5KHBvcywgY2xpZW50cywgbGlzdCkgeworCQlpZiAodmVy Ym9zZSkKKwkJCXJldCA9IHVwc3RyZWFtX2JyaWRnZV9kaXN0YW5jZV93YXJuKHByb3ZpZGVyLAor CQkJCQkJCSAgICBwb3MtPmNsaWVudCk7CisJCWVsc2UKKwkJCXJldCA9IHVwc3RyZWFtX2JyaWRn ZV9kaXN0YW5jZShwcm92aWRlciwgcG9zLT5jbGllbnQsCisJCQkJCQkgICAgICAgTlVMTCk7CisK KwkJaWYgKHJldCA8IDApCisJCQlub3Rfc3VwcG9ydGVkID0gdHJ1ZTsKKworCQlpZiAobm90X3N1 cHBvcnRlZCAmJiAhdmVyYm9zZSkKKwkJCWJyZWFrOworCisJCWRpc3RhbmNlICs9IHJldDsKKwl9 CisKKwlpZiAobm90X3N1cHBvcnRlZCkKKwkJcmV0dXJuIC0xOworCisJcmV0dXJuIGRpc3RhbmNl OworfQorRVhQT1JUX1NZTUJPTF9HUEwocGNpX3AycGRtYV9kaXN0YW5jZSk7CisKKy8qKgorICog cGNpX3AycGRtYV9hc3NpZ25fcHJvdmlkZXIgLSBDaGVjayBjb21wYXRpYmlseSAoYXMgcGVyIHBj aV9wMnBkbWFfZGlzdGFuY2UpCisgKglhbmQgYXNzaWduIGEgcHJvdmlkZXIgdG8gYSBsaXN0IG9m IGNsaWVudHMKKyAqIEBwcm92aWRlcjogcDJwZG1hIHByb3ZpZGVyIHRvIGFzc2lnbiB0byB0aGUg Y2xpZW50IGxpc3QKKyAqIEBjbGllbnRzOiBsaXN0IG9mIGRldmljZXMgdG8gY2hlY2sgKE5VTEwt dGVybWluYXRlZCkKKyAqCisgKiBSZXR1cm5zIGZhbHNlIGlmIGFueSBvZiB0aGUgY2xpZW50cyBh cmUgbm90IGNvbXBhdGlibGUsIHRydWUgaWYgdGhlCisgKiBwcm92aWRlciB3YXMgc3VjY2Vzc2Z1 bGx5IGFzc2lnbmVkIHRvIHRoZSBjbGllbnRzLgorICovCitib29sIHBjaV9wMnBkbWFfYXNzaWdu X3Byb3ZpZGVyKHN0cnVjdCBwY2lfZGV2ICpwcm92aWRlciwKKwkJCQlzdHJ1Y3QgbGlzdF9oZWFk ICpjbGllbnRzKQoreworCXN0cnVjdCBwY2lfcDJwZG1hX2NsaWVudCAqcG9zOworCisJaWYgKHBj aV9wMnBkbWFfZGlzdGFuY2UocHJvdmlkZXIsIGNsaWVudHMsIHRydWUpIDwgMCkKKwkJcmV0dXJu IGZhbHNlOworCisJbGlzdF9mb3JfZWFjaF9lbnRyeShwb3MsIGNsaWVudHMsIGxpc3QpCisJCXBv cy0+cHJvdmlkZXIgPSBwcm92aWRlcjsKKworCXJldHVybiB0cnVlOworfQorRVhQT1JUX1NZTUJP TF9HUEwocGNpX3AycGRtYV9hc3NpZ25fcHJvdmlkZXIpOworCisvKioKKyAqIHBjaV9oYXNfcDJw bWVtIC0gY2hlY2sgaWYgYSBnaXZlbiBQQ0kgZGV2aWNlIGhhcyBwdWJsaXNoZWQgYW55IHAycG1l bQorICogQHBkZXY6IFBDSSBkZXZpY2UgdG8gY2hlY2sKKyAqLworYm9vbCBwY2lfaGFzX3AycG1l bShzdHJ1Y3QgcGNpX2RldiAqcGRldikKK3sKKwlyZXR1cm4gcGRldi0+cDJwZG1hICYmIHBkZXYt PnAycGRtYS0+cDJwbWVtX3B1Ymxpc2hlZDsKK30KK0VYUE9SVF9TWU1CT0xfR1BMKHBjaV9oYXNf cDJwbWVtKTsKKworLyoqCisgKiBwY2lfcDJwbWVtX2ZpbmQgLSBmaW5kIGEgcGVlci10by1wZWVy IERNQSBtZW1vcnkgZGV2aWNlIGNvbXBhdGlibGUgd2l0aAorICoJdGhlIHNwZWNpZmllZCBsaXN0 IG9mIGNsaWVudHMgYW5kIHNob3J0ZXN0IGRpc3RhbmNlIChhcyBkZXRlcm1pbmVkCisgKglieSBw Y2lfcDJwbWVtX2RtYSgpKQorICogQGNsaWVudHM6IGxpc3Qgb2YgZGV2aWNlcyB0byBjaGVjayAo TlVMTC10ZXJtaW5hdGVkKQorICoKKyAqIElmIG11bHRpcGxlIGRldmljZXMgYXJlIGJlaGluZCB0 aGUgc2FtZSBzd2l0Y2gsIHRoZSBvbmUgImNsb3Nlc3QiIHRvIHRoZQorICogY2xpZW50IGRldmlj ZXMgaW4gdXNlIHdpbGwgYmUgY2hvc2VuIGZpcnN0LiAoU28gaWYgb25lIG9mIHRoZSBwcm92aWRl cnMgYXJlCisgKiB0aGUgc2FtZSBhcyBvbmUgb2YgdGhlIGNsaWVudHMsIHRoYXQgcHJvdmlkZXIg d2lsbCBiZSB1c2VkIGFoZWFkIG9mIGFueQorICogb3RoZXIgcHJvdmlkZXJzIHRoYXQgYXJlIHVu cmVsYXRlZCkuIElmIG11bHRpcGxlIHByb3ZpZGVycyBhcmUgYW4gZXF1YWwKKyAqIGRpc3RhbmNl IGF3YXksIG9uZSB3aWxsIGJlIGNob3NlbiBhdCByYW5kb20uCisgKgorICogUmV0dXJucyBhIHBv aW50ZXIgdG8gdGhlIFBDSSBkZXZpY2Ugd2l0aCBhIHJlZmVyZW5jZSB0YWtlbiAodXNlIHBjaV9k ZXZfcHV0CisgKiB0byByZXR1cm4gdGhlIHJlZmVyZW5jZSkgb3IgTlVMTCBpZiBubyBjb21wYXRp YmxlIGRldmljZSBpcyBmb3VuZC4gVGhlCisgKiBmb3VuZCBwcm92aWRlciB3aWxsIGFsc28gYmUg YXNzaWduZWQgdG8gdGhlIGNsaWVudCBsaXN0LgorICovCitzdHJ1Y3QgcGNpX2RldiAqcGNpX3Ay cG1lbV9maW5kKHN0cnVjdCBsaXN0X2hlYWQgKmNsaWVudHMpCit7CisJc3RydWN0IHBjaV9kZXYg KnBkZXYgPSBOVUxMOworCXN0cnVjdCBwY2lfcDJwZG1hX2NsaWVudCAqcG9zOworCWludCBkaXN0 YW5jZTsKKwlpbnQgY2xvc2VzdF9kaXN0YW5jZSA9IElOVF9NQVg7CisJc3RydWN0IHBjaV9kZXYg KipjbG9zZXN0X3BkZXZzOworCWludCBkZXZfY250ID0gMDsKKwljb25zdCBpbnQgbWF4X2RldnMg PSBQQUdFX1NJWkUgLyBzaXplb2YoKmNsb3Nlc3RfcGRldnMpOworCWludCBpOworCisJY2xvc2Vz dF9wZGV2cyA9IGttYWxsb2MoUEFHRV9TSVpFLCBHRlBfS0VSTkVMKTsKKworCXdoaWxlICgocGRl diA9IHBjaV9nZXRfZGV2aWNlKFBDSV9BTllfSUQsIFBDSV9BTllfSUQsIHBkZXYpKSkgeworCQlp ZiAoIXBjaV9oYXNfcDJwbWVtKHBkZXYpKQorCQkJY29udGludWU7CisKKwkJZGlzdGFuY2UgPSBw Y2lfcDJwZG1hX2Rpc3RhbmNlKHBkZXYsIGNsaWVudHMsIGZhbHNlKTsKKwkJaWYgKGRpc3RhbmNl IDwgMCB8fCBkaXN0YW5jZSA+IGNsb3Nlc3RfZGlzdGFuY2UpCisJCQljb250aW51ZTsKKworCQlp ZiAoZGlzdGFuY2UgPT0gY2xvc2VzdF9kaXN0YW5jZSAmJiBkZXZfY250ID49IG1heF9kZXZzKQor CQkJY29udGludWU7CisKKwkJaWYgKGRpc3RhbmNlIDwgY2xvc2VzdF9kaXN0YW5jZSkgeworCQkJ Zm9yIChpID0gMDsgaSA8IGRldl9jbnQ7IGkrKykKKwkJCQlwY2lfZGV2X3B1dChjbG9zZXN0X3Bk ZXZzW2ldKTsKKworCQkJZGV2X2NudCA9IDA7CisJCQljbG9zZXN0X2Rpc3RhbmNlID0gZGlzdGFu Y2U7CisJCX0KKworCQljbG9zZXN0X3BkZXZzW2Rldl9jbnQrK10gPSBwY2lfZGV2X2dldChwZGV2 KTsKKwl9CisKKwlpZiAoZGV2X2NudCkKKwkJcGRldiA9IHBjaV9kZXZfZ2V0KGNsb3Nlc3RfcGRl dnNbcHJhbmRvbV91MzJfbWF4KGRldl9jbnQpXSk7CisKKwlmb3IgKGkgPSAwOyBpIDwgZGV2X2Nu dDsgaSsrKQorCQlwY2lfZGV2X3B1dChjbG9zZXN0X3BkZXZzW2ldKTsKKworCWlmIChwZGV2KQor CQlsaXN0X2Zvcl9lYWNoX2VudHJ5KHBvcywgY2xpZW50cywgbGlzdCkKKwkJCXBvcy0+cHJvdmlk ZXIgPSBwZGV2OworCisJa2ZyZWUoY2xvc2VzdF9wZGV2cyk7CisJcmV0dXJuIHBkZXY7Cit9CitF WFBPUlRfU1lNQk9MX0dQTChwY2lfcDJwbWVtX2ZpbmQpOworCisvKioKKyAqIHBjaV9hbGxvY19w MnBfbWVtIC0gYWxsb2NhdGUgcGVlci10by1wZWVyIERNQSBtZW1vcnkKKyAqIEBwZGV2OiB0aGUg ZGV2aWNlIHRvIGFsbG9jYXRlIG1lbW9yeSBmcm9tCisgKiBAc2l6ZTogbnVtYmVyIG9mIGJ5dGVz IHRvIGFsbG9jYXRlCisgKgorICogUmV0dXJucyB0aGUgYWxsb2NhdGVkIG1lbW9yeSBvciBOVUxM IG9uIGVycm9yLgorICovCit2b2lkICpwY2lfYWxsb2NfcDJwbWVtKHN0cnVjdCBwY2lfZGV2ICpw ZGV2LCBzaXplX3Qgc2l6ZSkKK3sKKwl2b2lkICpyZXQ7CisKKwlpZiAodW5saWtlbHkoIXBkZXYt PnAycGRtYSkpCisJCXJldHVybiBOVUxMOworCisJaWYgKHVubGlrZWx5KCFwZXJjcHVfcmVmX3Ry eWdldF9saXZlKCZwZGV2LT5wMnBkbWEtPmRldm1hcF9yZWYpKSkKKwkJcmV0dXJuIE5VTEw7CisK KwlyZXQgPSAodm9pZCAqKWdlbl9wb29sX2FsbG9jKHBkZXYtPnAycGRtYS0+cG9vbCwgc2l6ZSk7 CisKKwlpZiAodW5saWtlbHkoIXJldCkpCisJCXBlcmNwdV9yZWZfcHV0KCZwZGV2LT5wMnBkbWEt PmRldm1hcF9yZWYpOworCisJcmV0dXJuIHJldDsKK30KK0VYUE9SVF9TWU1CT0xfR1BMKHBjaV9h bGxvY19wMnBtZW0pOworCisvKioKKyAqIHBjaV9mcmVlX3AycG1lbSAtIGFsbG9jYXRlIHBlZXIt dG8tcGVlciBETUEgbWVtb3J5CisgKiBAcGRldjogdGhlIGRldmljZSB0aGUgbWVtb3J5IHdhcyBh bGxvY2F0ZWQgZnJvbQorICogQGFkZHI6IGFkZHJlc3Mgb2YgdGhlIG1lbW9yeSB0aGF0IHdhcyBh bGxvY2F0ZWQKKyAqIEBzaXplOiBudW1iZXIgb2YgYnl0ZXMgdGhhdCB3YXMgYWxsb2NhdGVkCisg Ki8KK3ZvaWQgcGNpX2ZyZWVfcDJwbWVtKHN0cnVjdCBwY2lfZGV2ICpwZGV2LCB2b2lkICphZGRy LCBzaXplX3Qgc2l6ZSkKK3sKKwlnZW5fcG9vbF9mcmVlKHBkZXYtPnAycGRtYS0+cG9vbCwgKHVp bnRwdHJfdClhZGRyLCBzaXplKTsKKwlwZXJjcHVfcmVmX3B1dCgmcGRldi0+cDJwZG1hLT5kZXZt YXBfcmVmKTsKK30KK0VYUE9SVF9TWU1CT0xfR1BMKHBjaV9mcmVlX3AycG1lbSk7CisKKy8qKgor ICogcGNpX3ZpcnRfdG9fYnVzIC0gcmV0dXJuIHRoZSBQQ0kgYnVzIGFkZHJlc3MgZm9yIGEgZ2l2 ZW4gdmlydHVhbAorICoJYWRkcmVzcyBvYnRhaW5lZCB3aXRoIHBjaV9hbGxvY19wMnBtZW0oKQor ICogQHBkZXY6IHRoZSBkZXZpY2UgdGhlIG1lbW9yeSB3YXMgYWxsb2NhdGVkIGZyb20KKyAqIEBh ZGRyOiBhZGRyZXNzIG9mIHRoZSBtZW1vcnkgdGhhdCB3YXMgYWxsb2NhdGVkCisgKi8KK3BjaV9i dXNfYWRkcl90IHBjaV9wMnBtZW1fdmlydF90b19idXMoc3RydWN0IHBjaV9kZXYgKnBkZXYsIHZv aWQgKmFkZHIpCit7CisJaWYgKCFhZGRyKQorCQlyZXR1cm4gMDsKKwlpZiAoIXBkZXYtPnAycGRt YSkKKwkJcmV0dXJuIDA7CisKKwkvKgorCSAqIE5vdGU6IHdoZW4gd2UgYWRkZWQgdGhlIG1lbW9y eSB0byB0aGUgcG9vbCB3ZSB1c2VkIHRoZSBQQ0kKKwkgKiBidXMgYWRkcmVzcyBhcyB0aGUgcGh5 c2ljYWwgYWRkcmVzcy4gU28gZ2VuX3Bvb2xfdmlydF90b19waHlzKCkKKwkgKiBhY3R1YWxseSBy ZXR1cm5zIHRoZSBidXMgYWRkcmVzcyBkZXNwaXRlIHRoZSBtaXNsZWFkaW5nIG5hbWUuCisJICov CisJcmV0dXJuIGdlbl9wb29sX3ZpcnRfdG9fcGh5cyhwZGV2LT5wMnBkbWEtPnBvb2wsICh1bnNp Z25lZCBsb25nKWFkZHIpOworfQorRVhQT1JUX1NZTUJPTF9HUEwocGNpX3AycG1lbV92aXJ0X3Rv X2J1cyk7CisKKy8qKgorICogcGNpX3AycG1lbV9hbGxvY19zZ2wgLSBhbGxvY2F0ZSBwZWVyLXRv LXBlZXIgRE1BIG1lbW9yeSBpbiBhIHNjYXR0ZXJsaXN0CisgKiBAcGRldjogdGhlIGRldmljZSB0 byBhbGxvY2F0ZSBtZW1vcnkgZnJvbQorICogQHNnbDogdGhlIGFsbG9jYXRlZCBzY2F0dGVybGlz dAorICogQG5lbnRzOiB0aGUgbnVtYmVyIG9mIFNHIGVudHJpZXMgaW4gdGhlIGxpc3QKKyAqIEBs ZW5ndGg6IG51bWJlciBvZiBieXRlcyB0byBhbGxvY2F0ZQorICoKKyAqIFJldHVybnMgMCBvbiBz dWNjZXNzCisgKi8KK3N0cnVjdCBzY2F0dGVybGlzdCAqcGNpX3AycG1lbV9hbGxvY19zZ2woc3Ry dWN0IHBjaV9kZXYgKnBkZXYsCisJCQkJCSB1bnNpZ25lZCBpbnQgKm5lbnRzLCB1MzIgbGVuZ3Ro KQoreworCXN0cnVjdCBzY2F0dGVybGlzdCAqc2c7CisJdm9pZCAqYWRkcjsKKworCXNnID0ga3ph bGxvYyhzaXplb2YoKnNnKSwgR0ZQX0tFUk5FTCk7CisJaWYgKCFzZykKKwkJcmV0dXJuIE5VTEw7 CisKKwlzZ19pbml0X3RhYmxlKHNnLCAxKTsKKworCWFkZHIgPSBwY2lfYWxsb2NfcDJwbWVtKHBk ZXYsIGxlbmd0aCk7CisJaWYgKCFhZGRyKQorCQlnb3RvIG91dF9mcmVlX3NnOworCisJc2dfc2V0 X2J1ZihzZywgYWRkciwgbGVuZ3RoKTsKKwkqbmVudHMgPSAxOworCXJldHVybiBzZzsKKworb3V0 X2ZyZWVfc2c6CisJa2ZyZWUoc2cpOworCXJldHVybiBOVUxMOworfQorRVhQT1JUX1NZTUJPTF9H UEwocGNpX3AycG1lbV9hbGxvY19zZ2wpOworCisvKioKKyAqIHBjaV9wMnBtZW1fZnJlZV9zZ2wg LSBmcmVlIGEgc2NhdHRlcmxpc3QgYWxsb2NhdGVkIGJ5IHBjaV9wMnBtZW1fYWxsb2Nfc2dsKCkK KyAqIEBwZGV2OiB0aGUgZGV2aWNlIHRvIGFsbG9jYXRlIG1lbW9yeSBmcm9tCisgKiBAc2dsOiB0 aGUgYWxsb2NhdGVkIHNjYXR0ZXJsaXN0CisgKiBAbmVudHM6IHRoZSBudW1iZXIgb2YgU0cgZW50 cmllcyBpbiB0aGUgbGlzdAorICovCit2b2lkIHBjaV9wMnBtZW1fZnJlZV9zZ2woc3RydWN0IHBj aV9kZXYgKnBkZXYsIHN0cnVjdCBzY2F0dGVybGlzdCAqc2dsKQoreworCXN0cnVjdCBzY2F0dGVy bGlzdCAqc2c7CisJaW50IGNvdW50OworCisJZm9yX2VhY2hfc2coc2dsLCBzZywgSU5UX01BWCwg Y291bnQpIHsKKwkJaWYgKCFzZykKKwkJCWJyZWFrOworCisJCXBjaV9mcmVlX3AycG1lbShwZGV2 LCBzZ192aXJ0KHNnKSwgc2ctPmxlbmd0aCk7CisJfQorCWtmcmVlKHNnbCk7Cit9CitFWFBPUlRf U1lNQk9MX0dQTChwY2lfcDJwbWVtX2ZyZWVfc2dsKTsKKworLyoqCisgKiBwY2lfcDJwbWVtX3B1 Ymxpc2ggLSBwdWJsaXNoIHRoZSBwZWVyLXRvLXBlZXIgRE1BIG1lbW9yeSBmb3IgdXNlIGJ5Cisg KglvdGhlciBkZXZpY2VzIHdpdGggcGNpX3AycG1lbV9maW5kKCkKKyAqIEBwZGV2OiB0aGUgZGV2 aWNlIHdpdGggcGVlci10by1wZWVyIERNQSBtZW1vcnkgdG8gcHVibGlzaAorICogQHB1Ymxpc2g6 IHNldCB0byB0cnVlIHRvIHB1Ymxpc2ggdGhlIG1lbW9yeSwgZmFsc2UgdG8gdW5wdWJsaXNoIGl0 CisgKgorICogUHVibGlzaGVkIG1lbW9yeSBjYW4gYmUgdXNlZCBieSBvdGhlciBQQ0kgZGV2aWNl IGRyaXZlcnMgZm9yCisgKiBwZWVyLTItcGVlciBETUEgb3BlcmF0aW9ucy4gTm9uLXB1Ymxpc2hl ZCBtZW1vcnkgaXMgcmVzZXJ2ZWQgZm9yCisgKiBleGx1c2l2ZSB1c2Ugb2YgdGhlIGRldmljZSBk cml2ZXIgdGhhdCByZWdpc3RlcnMgdGhlIHBlZXItdG8tcGVlcgorICogbWVtb3J5LgorICovCit2 b2lkIHBjaV9wMnBtZW1fcHVibGlzaChzdHJ1Y3QgcGNpX2RldiAqcGRldiwgYm9vbCBwdWJsaXNo KQoreworCWlmIChwdWJsaXNoICYmICFwZGV2LT5wMnBkbWEpCisJCXJldHVybjsKKworCXBkZXYt PnAycGRtYS0+cDJwbWVtX3B1Ymxpc2hlZCA9IHB1Ymxpc2g7Cit9CitFWFBPUlRfU1lNQk9MX0dQ TChwY2lfcDJwbWVtX3B1Ymxpc2gpOwpkaWZmIC0tZ2l0IGEvaW5jbHVkZS9saW51eC9tZW1yZW1h cC5oIGIvaW5jbHVkZS9saW51eC9tZW1yZW1hcC5oCmluZGV4IGY5MWY5ZTc2MzU1Ny4uOTU1MzM3 MGViZGFkIDEwMDY0NAotLS0gYS9pbmNsdWRlL2xpbnV4L21lbXJlbWFwLmgKKysrIGIvaW5jbHVk ZS9saW51eC9tZW1yZW1hcC5oCkBAIC01MywxMSArNTMsMTYgQEAgc3RydWN0IHZtZW1fYWx0bWFw IHsKICAqIHdha2V1cCBldmVudCB3aGVuZXZlciBhIHBhZ2UgaXMgdW5waW5uZWQgYW5kIGJlY29t ZXMgaWRsZS4gVGhpcwogICogd2FrZXVwIGlzIHVzZWQgdG8gY29vcmRpbmF0ZSBwaHlzaWNhbCBh ZGRyZXNzIHNwYWNlIG1hbmFnZW1lbnQgKGV4OgogICogZnMgdHJ1bmNhdGUvaG9sZSBwdW5jaCkg dnMgcGlubmVkIHBhZ2VzIChleDogZGV2aWNlIGRtYSkuCisgKgorICogTUVNT1JZX0RFVklDRV9Q Q0lfUDJQRE1BOgorICogRGV2aWNlIG1lbW9yeSByZXNpZGluZyBpbiBhIFBDSSBCQVIgaW50ZW5k ZWQgZm9yIHVzZSB3aXRoIFBlZXItdG8tUGVlcgorICogdHJhbnNhY3Rpb25zLgogICovCiBlbnVt IG1lbW9yeV90eXBlIHsKIAlNRU1PUllfREVWSUNFX1BSSVZBVEUgPSAxLAogCU1FTU9SWV9ERVZJ Q0VfUFVCTElDLAogCU1FTU9SWV9ERVZJQ0VfRlNfREFYLAorCU1FTU9SWV9ERVZJQ0VfUENJX1Ay UERNQSwKIH07CiAKIC8qCmRpZmYgLS1naXQgYS9pbmNsdWRlL2xpbnV4L21tLmggYi9pbmNsdWRl L2xpbnV4L21tLmgKaW5kZXggYTYxZWJlOGFkNGNhLi4yMDU1ZGY0MTJhNzcgMTAwNjQ0Ci0tLSBh L2luY2x1ZGUvbGludXgvbW0uaAorKysgYi9pbmNsdWRlL2xpbnV4L21tLmgKQEAgLTg5MCw2ICs4 OTAsMTkgQEAgc3RhdGljIGlubGluZSBib29sIGlzX2RldmljZV9wdWJsaWNfcGFnZShjb25zdCBz dHJ1Y3QgcGFnZSAqcGFnZSkKIAkJcGFnZS0+cGdtYXAtPnR5cGUgPT0gTUVNT1JZX0RFVklDRV9Q VUJMSUM7CiB9CiAKKyNpZmRlZiBDT05GSUdfUENJX1AyUERNQQorc3RhdGljIGlubGluZSBib29s IGlzX3BjaV9wMnBkbWFfcGFnZShjb25zdCBzdHJ1Y3QgcGFnZSAqcGFnZSkKK3sKKwlyZXR1cm4g aXNfem9uZV9kZXZpY2VfcGFnZShwYWdlKSAmJgorCQlwYWdlLT5wZ21hcC0+dHlwZSA9PSBNRU1P UllfREVWSUNFX1BDSV9QMlBETUE7Cit9CisjZWxzZSAvKiBDT05GSUdfUENJX1AyUERNQSAqLwor c3RhdGljIGlubGluZSBib29sIGlzX3BjaV9wMnBkbWFfcGFnZShjb25zdCBzdHJ1Y3QgcGFnZSAq cGFnZSkKK3sKKwlyZXR1cm4gZmFsc2U7Cit9CisjZW5kaWYgLyogQ09ORklHX1BDSV9QMlBETUEg Ki8KKwogI2Vsc2UgLyogQ09ORklHX0RFVl9QQUdFTUFQX09QUyAqLwogc3RhdGljIGlubGluZSB2 b2lkIGRldl9wYWdlbWFwX2dldF9vcHModm9pZCkKIHsKQEAgLTkxMyw2ICs5MjYsMTEgQEAgc3Rh dGljIGlubGluZSBib29sIGlzX2RldmljZV9wdWJsaWNfcGFnZShjb25zdCBzdHJ1Y3QgcGFnZSAq cGFnZSkKIHsKIAlyZXR1cm4gZmFsc2U7CiB9CisKK3N0YXRpYyBpbmxpbmUgYm9vbCBpc19wY2lf cDJwZG1hX3BhZ2UoY29uc3Qgc3RydWN0IHBhZ2UgKnBhZ2UpCit7CisJcmV0dXJuIGZhbHNlOwor fQogI2VuZGlmIC8qIENPTkZJR19ERVZfUEFHRU1BUF9PUFMgKi8KIAogc3RhdGljIGlubGluZSB2 b2lkIGdldF9wYWdlKHN0cnVjdCBwYWdlICpwYWdlKQpkaWZmIC0tZ2l0IGEvaW5jbHVkZS9saW51 eC9wY2ktcDJwZG1hLmggYi9pbmNsdWRlL2xpbnV4L3BjaS1wMnBkbWEuaApuZXcgZmlsZSBtb2Rl IDEwMDY0NAppbmRleCAwMDAwMDAwMDAwMDAuLjdiMmIwZjU0NzUyOAotLS0gL2Rldi9udWxsCisr KyBiL2luY2x1ZGUvbGludXgvcGNpLXAycGRtYS5oCkBAIC0wLDAgKzEsMTAyIEBACisvKiBTUERY LUxpY2Vuc2UtSWRlbnRpZmllcjogR1BMLTIuMCAqLworLyoKKyAqIFBDSSBQZWVyIDIgUGVlciBE TUEgc3VwcG9ydC4KKyAqCisgKiBDb3B5cmlnaHQgKGMpIDIwMTYtMjAxOCwgTG9nYW4gR3VudGhv cnBlCisgKiBDb3B5cmlnaHQgKGMpIDIwMTYtMjAxNywgTWljcm9zZW1pIENvcnBvcmF0aW9uCisg KiBDb3B5cmlnaHQgKGMpIDIwMTcsIENocmlzdG9waCBIZWxsd2lnCisgKiBDb3B5cmlnaHQgKGMp IDIwMTgsIEVpZGV0aWNvbSBJbmMuCisgKgorICovCisKKyNpZm5kZWYgX0xJTlVYX1BDSV9QMlBE TUFfSAorI2RlZmluZSBfTElOVVhfUENJX1AyUERNQV9ICisKKyNpbmNsdWRlIDxsaW51eC9wY2ku aD4KKworc3RydWN0IGJsb2NrX2RldmljZTsKK3N0cnVjdCBzY2F0dGVybGlzdDsKKworI2lmZGVm IENPTkZJR19QQ0lfUDJQRE1BCitpbnQgcGNpX3AycGRtYV9hZGRfcmVzb3VyY2Uoc3RydWN0IHBj aV9kZXYgKnBkZXYsIGludCBiYXIsIHNpemVfdCBzaXplLAorCQl1NjQgb2Zmc2V0KTsKK2ludCBw Y2lfcDJwZG1hX2FkZF9jbGllbnQoc3RydWN0IGxpc3RfaGVhZCAqaGVhZCwgc3RydWN0IGRldmlj ZSAqZGV2KTsKK3ZvaWQgcGNpX3AycGRtYV9yZW1vdmVfY2xpZW50KHN0cnVjdCBsaXN0X2hlYWQg KmhlYWQsIHN0cnVjdCBkZXZpY2UgKmRldik7Cit2b2lkIHBjaV9wMnBkbWFfY2xpZW50X2xpc3Rf ZnJlZShzdHJ1Y3QgbGlzdF9oZWFkICpoZWFkKTsKK2ludCBwY2lfcDJwZG1hX2Rpc3RhbmNlKHN0 cnVjdCBwY2lfZGV2ICpwcm92aWRlciwgc3RydWN0IGxpc3RfaGVhZCAqY2xpZW50cywKKwkJCWJv b2wgdmVyYm9zZSk7Citib29sIHBjaV9wMnBkbWFfYXNzaWduX3Byb3ZpZGVyKHN0cnVjdCBwY2lf ZGV2ICpwcm92aWRlciwKKwkJCQlzdHJ1Y3QgbGlzdF9oZWFkICpjbGllbnRzKTsKK2Jvb2wgcGNp X2hhc19wMnBtZW0oc3RydWN0IHBjaV9kZXYgKnBkZXYpOworc3RydWN0IHBjaV9kZXYgKnBjaV9w MnBtZW1fZmluZChzdHJ1Y3QgbGlzdF9oZWFkICpjbGllbnRzKTsKK3ZvaWQgKnBjaV9hbGxvY19w MnBtZW0oc3RydWN0IHBjaV9kZXYgKnBkZXYsIHNpemVfdCBzaXplKTsKK3ZvaWQgcGNpX2ZyZWVf cDJwbWVtKHN0cnVjdCBwY2lfZGV2ICpwZGV2LCB2b2lkICphZGRyLCBzaXplX3Qgc2l6ZSk7Citw Y2lfYnVzX2FkZHJfdCBwY2lfcDJwbWVtX3ZpcnRfdG9fYnVzKHN0cnVjdCBwY2lfZGV2ICpwZGV2 LCB2b2lkICphZGRyKTsKK3N0cnVjdCBzY2F0dGVybGlzdCAqcGNpX3AycG1lbV9hbGxvY19zZ2wo c3RydWN0IHBjaV9kZXYgKnBkZXYsCisJCQkJCSB1bnNpZ25lZCBpbnQgKm5lbnRzLCB1MzIgbGVu Z3RoKTsKK3ZvaWQgcGNpX3AycG1lbV9mcmVlX3NnbChzdHJ1Y3QgcGNpX2RldiAqcGRldiwgc3Ry dWN0IHNjYXR0ZXJsaXN0ICpzZ2wpOwordm9pZCBwY2lfcDJwbWVtX3B1Ymxpc2goc3RydWN0IHBj aV9kZXYgKnBkZXYsIGJvb2wgcHVibGlzaCk7CisjZWxzZSAvKiBDT05GSUdfUENJX1AyUERNQSAq Lworc3RhdGljIGlubGluZSBpbnQgcGNpX3AycGRtYV9hZGRfcmVzb3VyY2Uoc3RydWN0IHBjaV9k ZXYgKnBkZXYsIGludCBiYXIsCisJCXNpemVfdCBzaXplLCB1NjQgb2Zmc2V0KQoreworCXJldHVy biAtRU9QTk9UU1VQUDsKK30KK3N0YXRpYyBpbmxpbmUgaW50IHBjaV9wMnBkbWFfYWRkX2NsaWVu dChzdHJ1Y3QgbGlzdF9oZWFkICpoZWFkLAorCQlzdHJ1Y3QgZGV2aWNlICpkZXYpCit7CisJcmV0 dXJuIDA7Cit9CitzdGF0aWMgaW5saW5lIHZvaWQgcGNpX3AycGRtYV9yZW1vdmVfY2xpZW50KHN0 cnVjdCBsaXN0X2hlYWQgKmhlYWQsCisJCXN0cnVjdCBkZXZpY2UgKmRldikKK3sKK30KK3N0YXRp YyBpbmxpbmUgdm9pZCBwY2lfcDJwZG1hX2NsaWVudF9saXN0X2ZyZWUoc3RydWN0IGxpc3RfaGVh ZCAqaGVhZCkKK3sKK30KK3N0YXRpYyBpbmxpbmUgaW50IHBjaV9wMnBkbWFfZGlzdGFuY2Uoc3Ry dWN0IHBjaV9kZXYgKnByb3ZpZGVyLAorCQkJCSAgICAgIHN0cnVjdCBsaXN0X2hlYWQgKmNsaWVu dHMsCisJCQkJICAgICAgYm9vbCB2ZXJib3NlKQoreworCXJldHVybiAtMTsKK30KK3N0YXRpYyBp bmxpbmUgYm9vbCBwY2lfcDJwZG1hX2Fzc2lnbl9wcm92aWRlcihzdHJ1Y3QgcGNpX2RldiAqcHJv dmlkZXIsCisJCQkJCSAgICAgIHN0cnVjdCBsaXN0X2hlYWQgKmNsaWVudHMpCit7CisJcmV0dXJu IGZhbHNlOworfQorc3RhdGljIGlubGluZSBib29sIHBjaV9oYXNfcDJwbWVtKHN0cnVjdCBwY2lf ZGV2ICpwZGV2KQoreworCXJldHVybiBmYWxzZTsKK30KK3N0YXRpYyBpbmxpbmUgc3RydWN0IHBj aV9kZXYgKnBjaV9wMnBtZW1fZmluZChzdHJ1Y3QgbGlzdF9oZWFkICpjbGllbnRzKQoreworCXJl dHVybiBOVUxMOworfQorc3RhdGljIGlubGluZSB2b2lkICpwY2lfYWxsb2NfcDJwbWVtKHN0cnVj dCBwY2lfZGV2ICpwZGV2LCBzaXplX3Qgc2l6ZSkKK3sKKwlyZXR1cm4gTlVMTDsKK30KK3N0YXRp YyBpbmxpbmUgdm9pZCBwY2lfZnJlZV9wMnBtZW0oc3RydWN0IHBjaV9kZXYgKnBkZXYsIHZvaWQg KmFkZHIsCisJCXNpemVfdCBzaXplKQoreworfQorc3RhdGljIGlubGluZSBwY2lfYnVzX2FkZHJf dCBwY2lfcDJwbWVtX3ZpcnRfdG9fYnVzKHN0cnVjdCBwY2lfZGV2ICpwZGV2LAorCQkJCQkJICAg IHZvaWQgKmFkZHIpCit7CisJcmV0dXJuIDA7Cit9CitzdGF0aWMgaW5saW5lIHN0cnVjdCBzY2F0 dGVybGlzdCAqcGNpX3AycG1lbV9hbGxvY19zZ2woc3RydWN0IHBjaV9kZXYgKnBkZXYsCisJCXVu c2lnbmVkIGludCAqbmVudHMsIHUzMiBsZW5ndGgpCit7CisJcmV0dXJuIE5VTEw7Cit9CitzdGF0 aWMgaW5saW5lIHZvaWQgcGNpX3AycG1lbV9mcmVlX3NnbChzdHJ1Y3QgcGNpX2RldiAqcGRldiwK KwkJc3RydWN0IHNjYXR0ZXJsaXN0ICpzZ2wpCit7Cit9CitzdGF0aWMgaW5saW5lIHZvaWQgcGNp X3AycG1lbV9wdWJsaXNoKHN0cnVjdCBwY2lfZGV2ICpwZGV2LCBib29sIHB1Ymxpc2gpCit7Cit9 CisjZW5kaWYgLyogQ09ORklHX1BDSV9QMlBETUEgKi8KKyNlbmRpZiAvKiBfTElOVVhfUENJX1Ay UF9IICovCmRpZmYgLS1naXQgYS9pbmNsdWRlL2xpbnV4L3BjaS5oIGIvaW5jbHVkZS9saW51eC9w Y2kuaAppbmRleCBlNzJjYThkZDYyNDEuLjVkOTVkYmYyMWY0YSAxMDA2NDQKLS0tIGEvaW5jbHVk ZS9saW51eC9wY2kuaAorKysgYi9pbmNsdWRlL2xpbnV4L3BjaS5oCkBAIC0yODEsNiArMjgxLDcg QEAgc3RydWN0IHBjaWVfbGlua19zdGF0ZTsKIHN0cnVjdCBwY2lfdnBkOwogc3RydWN0IHBjaV9z cmlvdjsKIHN0cnVjdCBwY2lfYXRzOworc3RydWN0IHBjaV9wMnBkbWE7CiAKIC8qIFRoZSBwY2lf ZGV2IHN0cnVjdHVyZSBkZXNjcmliZXMgUENJIGRldmljZXMgKi8KIHN0cnVjdCBwY2lfZGV2IHsK QEAgLTQzOSw2ICs0NDAsOSBAQCBzdHJ1Y3QgcGNpX2RldiB7CiAjaWZkZWYgQ09ORklHX1BDSV9Q QVNJRAogCXUxNgkJcGFzaWRfZmVhdHVyZXM7CiAjZW5kaWYKKyNpZmRlZiBDT05GSUdfUENJX1Ay UERNQQorCXN0cnVjdCBwY2lfcDJwZG1hICpwMnBkbWE7CisjZW5kaWYKIAlwaHlzX2FkZHJfdAly b207CQkvKiBQaHlzaWNhbCBhZGRyZXNzIGlmIG5vdCBmcm9tIEJBUiAqLwogCXNpemVfdAkJcm9t bGVuOwkJLyogTGVuZ3RoIGlmIG5vdCBmcm9tIEJBUiAqLwogCWNoYXIJCSpkcml2ZXJfb3ZlcnJp ZGU7IC8qIERyaXZlciBuYW1lIHRvIGZvcmNlIGEgbWF0Y2ggKi8KLS0gCjIuMTEuMAoKX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KTGludXgtbnZkaW1tIG1h aWxpbmcgbGlzdApMaW51eC1udmRpbW1AbGlzdHMuMDEub3JnCmh0dHBzOi8vbGlzdHMuMDEub3Jn L21haWxtYW4vbGlzdGluZm8vbGludXgtbnZkaW1tCg== From mboxrd@z Thu Jan 1 00:00:00 1970 From: logang@deltatee.com (Logan Gunthorpe) Date: Thu, 30 Aug 2018 12:53:40 -0600 Subject: [PATCH v5 01/13] PCI/P2PDMA: Support peer-to-peer memory In-Reply-To: <20180830185352.3369-1-logang@deltatee.com> References: <20180830185352.3369-1-logang@deltatee.com> Message-ID: <20180830185352.3369-2-logang@deltatee.com> Some PCI devices may have memory mapped in a BAR space that's intended for use in peer-to-peer transactions. In order to enable such transactions the memory must be registered with ZONE_DEVICE pages so it can be used by DMA interfaces in existing drivers. Add an interface for other subsystems to find and allocate chunks of P2P memory as necessary to facilitate transfers between two PCI peers: int pci_p2pdma_add_client(); struct pci_dev *pci_p2pmem_find(); void *pci_alloc_p2pmem(); The new interface requires a driver to collect a list of client devices involved in the transaction with the pci_p2pmem_add_client*() functions then call pci_p2pmem_find() to obtain any suitable P2P memory. Once this is done the list is bound to the memory and the calling driver is free to add and remove clients as necessary (adding incompatible clients will fail). With a suitable p2pmem device, memory can then be allocated with pci_alloc_p2pmem() for use in DMA transactions. Depending on hardware, using peer-to-peer memory may reduce the bandwidth of the transfer but can significantly reduce pressure on system memory. This may be desirable in many cases: for example a system could be designed with a small CPU connected to a PCIe switch by a small number of lanes which would maximize the number of lanes available to connect to NVMe devices. The code is designed to only utilize the p2pmem device if all the devices involved in a transfer are behind the same PCI bridge. This is because we have no way of knowing whether peer-to-peer routing between PCIe Root Ports is supported (PCIe r4.0, sec 1.3.1). Additionally, the benefits of P2P transfers that go through the RC is limited to only reducing DRAM usage and, in some cases, coding convenience. The PCI-SIG may be exploring adding a new capability bit to advertise whether this is possible for future hardware. This commit includes significant rework and feedback from Christoph Hellwig. Signed-off-by: Christoph Hellwig Signed-off-by: Logan Gunthorpe --- drivers/pci/Kconfig | 17 + drivers/pci/Makefile | 1 + drivers/pci/p2pdma.c | 761 +++++++++++++++++++++++++++++++++++++++++++++ include/linux/memremap.h | 5 + include/linux/mm.h | 18 ++ include/linux/pci-p2pdma.h | 102 ++++++ include/linux/pci.h | 4 + 7 files changed, 908 insertions(+) create mode 100644 drivers/pci/p2pdma.c create mode 100644 include/linux/pci-p2pdma.h diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig index 56ff8f6d31fc..deb68be4fdac 100644 --- a/drivers/pci/Kconfig +++ b/drivers/pci/Kconfig @@ -132,6 +132,23 @@ config PCI_PASID If unsure, say N. +config PCI_P2PDMA + bool "PCI peer-to-peer transfer support" + depends on PCI && ZONE_DEVICE + select GENERIC_ALLOCATOR + help + Enable? drivers to do PCI peer-to-peer transactions to and from + BARs that are exposed in other devices that are the part of + the hierarchy where peer-to-peer DMA is guaranteed by the PCI + specification to work (ie. anything below a single PCI bridge). + + Many PCIe root complexes do not support P2P transactions and + it's hard to tell which support it at all, so at this time, + P2P DMA transations must be between devices behind the same root + port. + + If unsure, say N. + config PCI_LABEL def_bool y if (DMI || ACPI) depends on PCI diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile index 1b2cfe51e8d7..85f4a703b2be 100644 --- a/drivers/pci/Makefile +++ b/drivers/pci/Makefile @@ -26,6 +26,7 @@ obj-$(CONFIG_PCI_SYSCALL) += syscall.o obj-$(CONFIG_PCI_STUB) += pci-stub.o obj-$(CONFIG_PCI_PF_STUB) += pci-pf-stub.o obj-$(CONFIG_PCI_ECAM) += ecam.o +obj-$(CONFIG_PCI_P2PDMA) += p2pdma.o obj-$(CONFIG_XEN_PCIDEV_FRONTEND) += xen-pcifront.o # Endpoint library must be initialized before its users diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c new file mode 100644 index 000000000000..88aaec5351cd --- /dev/null +++ b/drivers/pci/p2pdma.c @@ -0,0 +1,761 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * PCI Peer 2 Peer DMA support. + * + * Copyright (c) 2016-2018, Logan Gunthorpe + * Copyright (c) 2016-2017, Microsemi Corporation + * Copyright (c) 2017, Christoph Hellwig + * Copyright (c) 2018, Eideticom Inc. + */ + +#define pr_fmt(fmt) "pci-p2pdma: " fmt +#include +#include +#include +#include +#include +#include +#include +#include + +struct pci_p2pdma { + struct percpu_ref devmap_ref; + struct completion devmap_ref_done; + struct gen_pool *pool; + bool p2pmem_published; +}; + +static void pci_p2pdma_percpu_release(struct percpu_ref *ref) +{ + struct pci_p2pdma *p2p = + container_of(ref, struct pci_p2pdma, devmap_ref); + + complete_all(&p2p->devmap_ref_done); +} + +static void pci_p2pdma_percpu_kill(void *data) +{ + struct percpu_ref *ref = data; + + if (percpu_ref_is_dying(ref)) + return; + + percpu_ref_kill(ref); +} + +static void pci_p2pdma_release(void *data) +{ + struct pci_dev *pdev = data; + + if (!pdev->p2pdma) + return; + + wait_for_completion(&pdev->p2pdma->devmap_ref_done); + percpu_ref_exit(&pdev->p2pdma->devmap_ref); + + gen_pool_destroy(pdev->p2pdma->pool); + pdev->p2pdma = NULL; +} + +static int pci_p2pdma_setup(struct pci_dev *pdev) +{ + int error = -ENOMEM; + struct pci_p2pdma *p2p; + + p2p = devm_kzalloc(&pdev->dev, sizeof(*p2p), GFP_KERNEL); + if (!p2p) + return -ENOMEM; + + p2p->pool = gen_pool_create(PAGE_SHIFT, dev_to_node(&pdev->dev)); + if (!p2p->pool) + goto out; + + init_completion(&p2p->devmap_ref_done); + error = percpu_ref_init(&p2p->devmap_ref, + pci_p2pdma_percpu_release, 0, GFP_KERNEL); + if (error) + goto out_pool_destroy; + + percpu_ref_switch_to_atomic_sync(&p2p->devmap_ref); + + error = devm_add_action_or_reset(&pdev->dev, pci_p2pdma_release, pdev); + if (error) + goto out_pool_destroy; + + pdev->p2pdma = p2p; + + return 0; + +out_pool_destroy: + gen_pool_destroy(p2p->pool); +out: + devm_kfree(&pdev->dev, p2p); + return error; +} + +/** + * pci_p2pdma_add_resource - add memory for use as p2p memory + * @pdev: the device to add the memory to + * @bar: PCI BAR to add + * @size: size of the memory to add, may be zero to use the whole BAR + * @offset: offset into the PCI BAR + * + * The memory will be given ZONE_DEVICE struct pages so that it may + * be used with any DMA request. + */ +int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, + u64 offset) +{ + struct dev_pagemap *pgmap; + void *addr; + int error; + + if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM)) + return -EINVAL; + + if (offset >= pci_resource_len(pdev, bar)) + return -EINVAL; + + if (!size) + size = pci_resource_len(pdev, bar) - offset; + + if (size + offset > pci_resource_len(pdev, bar)) + return -EINVAL; + + if (!pdev->p2pdma) { + error = pci_p2pdma_setup(pdev); + if (error) + return error; + } + + pgmap = devm_kzalloc(&pdev->dev, sizeof(*pgmap), GFP_KERNEL); + if (!pgmap) + return -ENOMEM; + + pgmap->res.start = pci_resource_start(pdev, bar) + offset; + pgmap->res.end = pgmap->res.start + size - 1; + pgmap->res.flags = pci_resource_flags(pdev, bar); + pgmap->ref = &pdev->p2pdma->devmap_ref; + pgmap->type = MEMORY_DEVICE_PCI_P2PDMA; + + addr = devm_memremap_pages(&pdev->dev, pgmap); + if (IS_ERR(addr)) { + error = PTR_ERR(addr); + goto pgmap_free; + } + + error = gen_pool_add_virt(pdev->p2pdma->pool, (unsigned long)addr, + pci_bus_address(pdev, bar) + offset, + resource_size(&pgmap->res), dev_to_node(&pdev->dev)); + if (error) + goto pgmap_free; + + error = devm_add_action_or_reset(&pdev->dev, pci_p2pdma_percpu_kill, + &pdev->p2pdma->devmap_ref); + if (error) + goto pgmap_free; + + pci_info(pdev, "added peer-to-peer DMA memory %pR\n", + &pgmap->res); + + return 0; + +pgmap_free: + devres_free(pgmap); + return error; +} +EXPORT_SYMBOL_GPL(pci_p2pdma_add_resource); + +static struct pci_dev *find_parent_pci_dev(struct device *dev) +{ + struct device *parent; + + dev = get_device(dev); + + while (dev) { + if (dev_is_pci(dev)) + return to_pci_dev(dev); + + parent = get_device(dev->parent); + put_device(dev); + dev = parent; + } + + return NULL; +} + +/* + * Check if a PCI bridge has it's ACS redirection bits set to redirect P2P + * TLPs upstream via ACS. Returns 1 if the packets will be redirected + * upstream, 0 otherwise. + */ +static int pci_bridge_has_acs_redir(struct pci_dev *dev) +{ + int pos; + u16 ctrl; + + pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_ACS); + if (!pos) + return 0; + + pci_read_config_word(dev, pos + PCI_ACS_CTRL, &ctrl); + + if (ctrl & (PCI_ACS_RR | PCI_ACS_CR | PCI_ACS_EC)) + return 1; + + return 0; +} + +static void seq_buf_print_bus_devfn(struct seq_buf *buf, struct pci_dev *dev) +{ + if (!buf) + return; + + seq_buf_printf(buf, "%04x:%02x:%02x.%x;", pci_domain_nr(dev->bus), + dev->bus->number, PCI_SLOT(dev->devfn), + PCI_FUNC(dev->devfn)); +} + +/* + * Find the distance through the nearest common upstream bridge between + * two PCI devices. + * + * If the two devices are the same device then 0 will be returned. + * + * If there are two virtual functions of the same device behind the same + * bridge port then 2 will be returned (one step down to the PCIe switch, + * then one step back to the same device). + * + * In the case where two devices are connected to the same PCIe switch, the + * value 4 will be returned. This corresponds to the following PCI tree: + * + * -+ Root Port + * \+ Switch Upstream Port + * +-+ Switch Downstream Port + * + \- Device A + * \-+ Switch Downstream Port + * \- Device B + * + * The distance is 4 because we traverse from Device A through the downstream + * port of the switch, to the common upstream port, back up to the second + * downstream port and then to Device B. + * + * Any two devices that don't have a common upstream bridge will return -1. + * In this way devices on separate PCIe root ports will be rejected, which + * is what we want for peer-to-peer seeing each PCIe root port defines a + * separate hierarchy domain and there's no way to determine whether the root + * complex supports forwarding between them. + * + * In the case where two devices are connected to different PCIe switches, + * this function will still return a positive distance as long as both + * switches evenutally have a common upstream bridge. Note this covers + * the case of using multiple PCIe switches to achieve a desired level of + * fan-out from a root port. The exact distance will be a function of the + * number of switches between Device A and Device B. + * + * If a bridge which has any ACS redirection bits set is in the path + * then this functions will return -2. This is so we reject any + * cases where the TLPs are forwarded up into the root complex. + * In this case, a list of all infringing bridge addresses will be + * populated in acs_list (assuming it's non-null) for printk purposes. + */ +static int upstream_bridge_distance(struct pci_dev *a, + struct pci_dev *b, + struct seq_buf *acs_list) +{ + int dist_a = 0; + int dist_b = 0; + struct pci_dev *bb = NULL; + int acs_cnt = 0; + + /* + * Note, we don't need to take references to devices returned by + * pci_upstream_bridge() seeing we hold a reference to a child + * device which will already hold a reference to the upstream bridge. + */ + + while (a) { + dist_b = 0; + + if (pci_bridge_has_acs_redir(a)) { + seq_buf_print_bus_devfn(acs_list, a); + acs_cnt++; + } + + bb = b; + + while (bb) { + if (a == bb) + goto check_b_path_acs; + + bb = pci_upstream_bridge(bb); + dist_b++; + } + + a = pci_upstream_bridge(a); + dist_a++; + } + + return -1; + +check_b_path_acs: + bb = b; + + while (bb) { + if (a == bb) + break; + + if (pci_bridge_has_acs_redir(bb)) { + seq_buf_print_bus_devfn(acs_list, bb); + acs_cnt++; + } + + bb = pci_upstream_bridge(bb); + } + + if (acs_cnt) + return -2; + + return dist_a + dist_b; +} + +static int upstream_bridge_distance_warn(struct pci_dev *provider, + struct pci_dev *client) +{ + struct seq_buf acs_list; + int ret; + + seq_buf_init(&acs_list, kmalloc(PAGE_SIZE, GFP_KERNEL), PAGE_SIZE); + + ret = upstream_bridge_distance(provider, client, &acs_list); + if (ret == -2) { + pci_warn(client, "cannot be used for peer-to-peer DMA as ACS redirect is set between the client and provider\n"); + /* Drop final semicolon */ + acs_list.buffer[acs_list.len-1] = 0; + pci_warn(client, "to disable ACS redirect for this path, add the kernel parameter: pci=disable_acs_redir=%s\n", + acs_list.buffer); + + } else if (ret < 0) { + pci_warn(client, "cannot be used for peer-to-peer DMA as the client and provider do not share an upstream bridge\n"); + } + + kfree(acs_list.buffer); + + return ret; +} + +struct pci_p2pdma_client { + struct list_head list; + struct pci_dev *client; + struct pci_dev *provider; +}; + +/** + * pci_p2pdma_add_client - allocate a new element in a client device list + * @head: list head of p2pdma clients + * @dev: device to add to the list + * + * This adds @dev to a list of clients used by a p2pdma device. + * This list should be passed to pci_p2pmem_find(). Once pci_p2pmem_find() has + * been called successfully, the list will be bound to a specific p2pdma + * device and new clients can only be added to the list if they are + * supported by that p2pdma device. + * + * The caller is expected to have a lock which protects @head as necessary + * so that none of the pci_p2p functions can be called concurrently + * on that list. + * + * Returns 0 if the client was successfully added. + */ +int pci_p2pdma_add_client(struct list_head *head, struct device *dev) +{ + struct pci_p2pdma_client *item, *new_item; + struct pci_dev *provider = NULL; + struct pci_dev *client; + int ret; + + if (IS_ENABLED(CONFIG_DMA_VIRT_OPS) && dev->dma_ops == &dma_virt_ops) { + dev_warn(dev, "cannot be used for peer-to-peer DMA because the driver makes use of dma_virt_ops\n"); + return -ENODEV; + } + + client = find_parent_pci_dev(dev); + if (!client) { + dev_warn(dev, "cannot be used for peer-to-peer DMA as it is not a PCI device\n"); + return -ENODEV; + } + + item = list_first_entry_or_null(head, struct pci_p2pdma_client, list); + if (item && item->provider) { + provider = item->provider; + + ret = upstream_bridge_distance_warn(provider, client); + if (ret < 0) { + ret = -EXDEV; + goto put_client; + } + } + + new_item = kzalloc(sizeof(*new_item), GFP_KERNEL); + if (!new_item) { + ret = -ENOMEM; + goto put_client; + } + + new_item->client = client; + new_item->provider = pci_dev_get(provider); + + list_add_tail(&new_item->list, head); + + return 0; + +put_client: + pci_dev_put(client); + return ret; +} +EXPORT_SYMBOL_GPL(pci_p2pdma_add_client); + +static void pci_p2pdma_client_free(struct pci_p2pdma_client *item) +{ + list_del(&item->list); + pci_dev_put(item->client); + pci_dev_put(item->provider); + kfree(item); +} + +/** + * pci_p2pdma_remove_client - remove and free a p2pdma client + * @head: list head of p2pdma clients + * @dev: device to remove from the list + * + * This removes @dev from a list of clients used by a p2pdma device. + * The caller is expected to have a lock which protects @head as necessary + * so that none of the pci_p2p functions can be called concurrently + * on that list. + */ +void pci_p2pdma_remove_client(struct list_head *head, struct device *dev) +{ + struct pci_p2pdma_client *pos, *tmp; + struct pci_dev *pdev; + + pdev = find_parent_pci_dev(dev); + if (!pdev) + return; + + list_for_each_entry_safe(pos, tmp, head, list) { + if (pos->client != pdev) + continue; + + pci_p2pdma_client_free(pos); + } + + pci_dev_put(pdev); +} +EXPORT_SYMBOL_GPL(pci_p2pdma_remove_client); + +/** + * pci_p2pdma_client_list_free - free an entire list of p2pdma clients + * @head: list head of p2pdma clients + * + * This removes all devices in a list of clients used by a p2pdma device. + * The caller is expected to have a lock which protects @head as necessary + * so that none of the pci_p2pdma functions can be called concurrently + * on that list. + */ +void pci_p2pdma_client_list_free(struct list_head *head) +{ + struct pci_p2pdma_client *pos, *tmp; + + list_for_each_entry_safe(pos, tmp, head, list) + pci_p2pdma_client_free(pos); +} +EXPORT_SYMBOL_GPL(pci_p2pdma_client_list_free); + +/** + * pci_p2pdma_distance - Determive the cumulative distance between + * a p2pdma provider and the clients in use. + * @provider: p2pdma provider to check against the client list + * @clients: list of devices to check (NULL-terminated) + * @verbose: if true, print warnings for devices when we return -1 + * + * Returns -1 if any of the clients are not compatible (behind the same + * root port as the provider), otherwise returns a positive number where + * the lower number is the preferrable choice. (If there's one client + * that's the same as the provider it will return 0, which is best choice). + * + * For now, "compatible" means the provider and the clients are all behind + * the same PCI root port. This cuts out cases that may work but is safest + * for the user. Future work can expand this to white-list root complexes that + * can safely forward between each ports. + */ +int pci_p2pdma_distance(struct pci_dev *provider, struct list_head *clients, + bool verbose) +{ + struct pci_p2pdma_client *pos; + int ret; + int distance = 0; + bool not_supported = false; + + if (list_empty(clients)) + return -1; + + list_for_each_entry(pos, clients, list) { + if (verbose) + ret = upstream_bridge_distance_warn(provider, + pos->client); + else + ret = upstream_bridge_distance(provider, pos->client, + NULL); + + if (ret < 0) + not_supported = true; + + if (not_supported && !verbose) + break; + + distance += ret; + } + + if (not_supported) + return -1; + + return distance; +} +EXPORT_SYMBOL_GPL(pci_p2pdma_distance); + +/** + * pci_p2pdma_assign_provider - Check compatibily (as per pci_p2pdma_distance) + * and assign a provider to a list of clients + * @provider: p2pdma provider to assign to the client list + * @clients: list of devices to check (NULL-terminated) + * + * Returns false if any of the clients are not compatible, true if the + * provider was successfully assigned to the clients. + */ +bool pci_p2pdma_assign_provider(struct pci_dev *provider, + struct list_head *clients) +{ + struct pci_p2pdma_client *pos; + + if (pci_p2pdma_distance(provider, clients, true) < 0) + return false; + + list_for_each_entry(pos, clients, list) + pos->provider = provider; + + return true; +} +EXPORT_SYMBOL_GPL(pci_p2pdma_assign_provider); + +/** + * pci_has_p2pmem - check if a given PCI device has published any p2pmem + * @pdev: PCI device to check + */ +bool pci_has_p2pmem(struct pci_dev *pdev) +{ + return pdev->p2pdma && pdev->p2pdma->p2pmem_published; +} +EXPORT_SYMBOL_GPL(pci_has_p2pmem); + +/** + * pci_p2pmem_find - find a peer-to-peer DMA memory device compatible with + * the specified list of clients and shortest distance (as determined + * by pci_p2pmem_dma()) + * @clients: list of devices to check (NULL-terminated) + * + * If multiple devices are behind the same switch, the one "closest" to the + * client devices in use will be chosen first. (So if one of the providers are + * the same as one of the clients, that provider will be used ahead of any + * other providers that are unrelated). If multiple providers are an equal + * distance away, one will be chosen at random. + * + * Returns a pointer to the PCI device with a reference taken (use pci_dev_put + * to return the reference) or NULL if no compatible device is found. The + * found provider will also be assigned to the client list. + */ +struct pci_dev *pci_p2pmem_find(struct list_head *clients) +{ + struct pci_dev *pdev = NULL; + struct pci_p2pdma_client *pos; + int distance; + int closest_distance = INT_MAX; + struct pci_dev **closest_pdevs; + int dev_cnt = 0; + const int max_devs = PAGE_SIZE / sizeof(*closest_pdevs); + int i; + + closest_pdevs = kmalloc(PAGE_SIZE, GFP_KERNEL); + + while ((pdev = pci_get_device(PCI_ANY_ID, PCI_ANY_ID, pdev))) { + if (!pci_has_p2pmem(pdev)) + continue; + + distance = pci_p2pdma_distance(pdev, clients, false); + if (distance < 0 || distance > closest_distance) + continue; + + if (distance == closest_distance && dev_cnt >= max_devs) + continue; + + if (distance < closest_distance) { + for (i = 0; i < dev_cnt; i++) + pci_dev_put(closest_pdevs[i]); + + dev_cnt = 0; + closest_distance = distance; + } + + closest_pdevs[dev_cnt++] = pci_dev_get(pdev); + } + + if (dev_cnt) + pdev = pci_dev_get(closest_pdevs[prandom_u32_max(dev_cnt)]); + + for (i = 0; i < dev_cnt; i++) + pci_dev_put(closest_pdevs[i]); + + if (pdev) + list_for_each_entry(pos, clients, list) + pos->provider = pdev; + + kfree(closest_pdevs); + return pdev; +} +EXPORT_SYMBOL_GPL(pci_p2pmem_find); + +/** + * pci_alloc_p2p_mem - allocate peer-to-peer DMA memory + * @pdev: the device to allocate memory from + * @size: number of bytes to allocate + * + * Returns the allocated memory or NULL on error. + */ +void *pci_alloc_p2pmem(struct pci_dev *pdev, size_t size) +{ + void *ret; + + if (unlikely(!pdev->p2pdma)) + return NULL; + + if (unlikely(!percpu_ref_tryget_live(&pdev->p2pdma->devmap_ref))) + return NULL; + + ret = (void *)gen_pool_alloc(pdev->p2pdma->pool, size); + + if (unlikely(!ret)) + percpu_ref_put(&pdev->p2pdma->devmap_ref); + + return ret; +} +EXPORT_SYMBOL_GPL(pci_alloc_p2pmem); + +/** + * pci_free_p2pmem - allocate peer-to-peer DMA memory + * @pdev: the device the memory was allocated from + * @addr: address of the memory that was allocated + * @size: number of bytes that was allocated + */ +void pci_free_p2pmem(struct pci_dev *pdev, void *addr, size_t size) +{ + gen_pool_free(pdev->p2pdma->pool, (uintptr_t)addr, size); + percpu_ref_put(&pdev->p2pdma->devmap_ref); +} +EXPORT_SYMBOL_GPL(pci_free_p2pmem); + +/** + * pci_virt_to_bus - return the PCI bus address for a given virtual + * address obtained with pci_alloc_p2pmem() + * @pdev: the device the memory was allocated from + * @addr: address of the memory that was allocated + */ +pci_bus_addr_t pci_p2pmem_virt_to_bus(struct pci_dev *pdev, void *addr) +{ + if (!addr) + return 0; + if (!pdev->p2pdma) + return 0; + + /* + * Note: when we added the memory to the pool we used the PCI + * bus address as the physical address. So gen_pool_virt_to_phys() + * actually returns the bus address despite the misleading name. + */ + return gen_pool_virt_to_phys(pdev->p2pdma->pool, (unsigned long)addr); +} +EXPORT_SYMBOL_GPL(pci_p2pmem_virt_to_bus); + +/** + * pci_p2pmem_alloc_sgl - allocate peer-to-peer DMA memory in a scatterlist + * @pdev: the device to allocate memory from + * @sgl: the allocated scatterlist + * @nents: the number of SG entries in the list + * @length: number of bytes to allocate + * + * Returns 0 on success + */ +struct scatterlist *pci_p2pmem_alloc_sgl(struct pci_dev *pdev, + unsigned int *nents, u32 length) +{ + struct scatterlist *sg; + void *addr; + + sg = kzalloc(sizeof(*sg), GFP_KERNEL); + if (!sg) + return NULL; + + sg_init_table(sg, 1); + + addr = pci_alloc_p2pmem(pdev, length); + if (!addr) + goto out_free_sg; + + sg_set_buf(sg, addr, length); + *nents = 1; + return sg; + +out_free_sg: + kfree(sg); + return NULL; +} +EXPORT_SYMBOL_GPL(pci_p2pmem_alloc_sgl); + +/** + * pci_p2pmem_free_sgl - free a scatterlist allocated by pci_p2pmem_alloc_sgl() + * @pdev: the device to allocate memory from + * @sgl: the allocated scatterlist + * @nents: the number of SG entries in the list + */ +void pci_p2pmem_free_sgl(struct pci_dev *pdev, struct scatterlist *sgl) +{ + struct scatterlist *sg; + int count; + + for_each_sg(sgl, sg, INT_MAX, count) { + if (!sg) + break; + + pci_free_p2pmem(pdev, sg_virt(sg), sg->length); + } + kfree(sgl); +} +EXPORT_SYMBOL_GPL(pci_p2pmem_free_sgl); + +/** + * pci_p2pmem_publish - publish the peer-to-peer DMA memory for use by + * other devices with pci_p2pmem_find() + * @pdev: the device with peer-to-peer DMA memory to publish + * @publish: set to true to publish the memory, false to unpublish it + * + * Published memory can be used by other PCI device drivers for + * peer-2-peer DMA operations. Non-published memory is reserved for + * exlusive use of the device driver that registers the peer-to-peer + * memory. + */ +void pci_p2pmem_publish(struct pci_dev *pdev, bool publish) +{ + if (publish && !pdev->p2pdma) + return; + + pdev->p2pdma->p2pmem_published = publish; +} +EXPORT_SYMBOL_GPL(pci_p2pmem_publish); diff --git a/include/linux/memremap.h b/include/linux/memremap.h index f91f9e763557..9553370ebdad 100644 --- a/include/linux/memremap.h +++ b/include/linux/memremap.h @@ -53,11 +53,16 @@ struct vmem_altmap { * wakeup event whenever a page is unpinned and becomes idle. This * wakeup is used to coordinate physical address space management (ex: * fs truncate/hole punch) vs pinned pages (ex: device dma). + * + * MEMORY_DEVICE_PCI_P2PDMA: + * Device memory residing in a PCI BAR intended for use with Peer-to-Peer + * transactions. */ enum memory_type { MEMORY_DEVICE_PRIVATE = 1, MEMORY_DEVICE_PUBLIC, MEMORY_DEVICE_FS_DAX, + MEMORY_DEVICE_PCI_P2PDMA, }; /* diff --git a/include/linux/mm.h b/include/linux/mm.h index a61ebe8ad4ca..2055df412a77 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -890,6 +890,19 @@ static inline bool is_device_public_page(const struct page *page) page->pgmap->type == MEMORY_DEVICE_PUBLIC; } +#ifdef CONFIG_PCI_P2PDMA +static inline bool is_pci_p2pdma_page(const struct page *page) +{ + return is_zone_device_page(page) && + page->pgmap->type == MEMORY_DEVICE_PCI_P2PDMA; +} +#else /* CONFIG_PCI_P2PDMA */ +static inline bool is_pci_p2pdma_page(const struct page *page) +{ + return false; +} +#endif /* CONFIG_PCI_P2PDMA */ + #else /* CONFIG_DEV_PAGEMAP_OPS */ static inline void dev_pagemap_get_ops(void) { @@ -913,6 +926,11 @@ static inline bool is_device_public_page(const struct page *page) { return false; } + +static inline bool is_pci_p2pdma_page(const struct page *page) +{ + return false; +} #endif /* CONFIG_DEV_PAGEMAP_OPS */ static inline void get_page(struct page *page) diff --git a/include/linux/pci-p2pdma.h b/include/linux/pci-p2pdma.h new file mode 100644 index 000000000000..7b2b0f547528 --- /dev/null +++ b/include/linux/pci-p2pdma.h @@ -0,0 +1,102 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * PCI Peer 2 Peer DMA support. + * + * Copyright (c) 2016-2018, Logan Gunthorpe + * Copyright (c) 2016-2017, Microsemi Corporation + * Copyright (c) 2017, Christoph Hellwig + * Copyright (c) 2018, Eideticom Inc. + * + */ + +#ifndef _LINUX_PCI_P2PDMA_H +#define _LINUX_PCI_P2PDMA_H + +#include + +struct block_device; +struct scatterlist; + +#ifdef CONFIG_PCI_P2PDMA +int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, size_t size, + u64 offset); +int pci_p2pdma_add_client(struct list_head *head, struct device *dev); +void pci_p2pdma_remove_client(struct list_head *head, struct device *dev); +void pci_p2pdma_client_list_free(struct list_head *head); +int pci_p2pdma_distance(struct pci_dev *provider, struct list_head *clients, + bool verbose); +bool pci_p2pdma_assign_provider(struct pci_dev *provider, + struct list_head *clients); +bool pci_has_p2pmem(struct pci_dev *pdev); +struct pci_dev *pci_p2pmem_find(struct list_head *clients); +void *pci_alloc_p2pmem(struct pci_dev *pdev, size_t size); +void pci_free_p2pmem(struct pci_dev *pdev, void *addr, size_t size); +pci_bus_addr_t pci_p2pmem_virt_to_bus(struct pci_dev *pdev, void *addr); +struct scatterlist *pci_p2pmem_alloc_sgl(struct pci_dev *pdev, + unsigned int *nents, u32 length); +void pci_p2pmem_free_sgl(struct pci_dev *pdev, struct scatterlist *sgl); +void pci_p2pmem_publish(struct pci_dev *pdev, bool publish); +#else /* CONFIG_PCI_P2PDMA */ +static inline int pci_p2pdma_add_resource(struct pci_dev *pdev, int bar, + size_t size, u64 offset) +{ + return -EOPNOTSUPP; +} +static inline int pci_p2pdma_add_client(struct list_head *head, + struct device *dev) +{ + return 0; +} +static inline void pci_p2pdma_remove_client(struct list_head *head, + struct device *dev) +{ +} +static inline void pci_p2pdma_client_list_free(struct list_head *head) +{ +} +static inline int pci_p2pdma_distance(struct pci_dev *provider, + struct list_head *clients, + bool verbose) +{ + return -1; +} +static inline bool pci_p2pdma_assign_provider(struct pci_dev *provider, + struct list_head *clients) +{ + return false; +} +static inline bool pci_has_p2pmem(struct pci_dev *pdev) +{ + return false; +} +static inline struct pci_dev *pci_p2pmem_find(struct list_head *clients) +{ + return NULL; +} +static inline void *pci_alloc_p2pmem(struct pci_dev *pdev, size_t size) +{ + return NULL; +} +static inline void pci_free_p2pmem(struct pci_dev *pdev, void *addr, + size_t size) +{ +} +static inline pci_bus_addr_t pci_p2pmem_virt_to_bus(struct pci_dev *pdev, + void *addr) +{ + return 0; +} +static inline struct scatterlist *pci_p2pmem_alloc_sgl(struct pci_dev *pdev, + unsigned int *nents, u32 length) +{ + return NULL; +} +static inline void pci_p2pmem_free_sgl(struct pci_dev *pdev, + struct scatterlist *sgl) +{ +} +static inline void pci_p2pmem_publish(struct pci_dev *pdev, bool publish) +{ +} +#endif /* CONFIG_PCI_P2PDMA */ +#endif /* _LINUX_PCI_P2P_H */ diff --git a/include/linux/pci.h b/include/linux/pci.h index e72ca8dd6241..5d95dbf21f4a 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -281,6 +281,7 @@ struct pcie_link_state; struct pci_vpd; struct pci_sriov; struct pci_ats; +struct pci_p2pdma; /* The pci_dev structure describes PCI devices */ struct pci_dev { @@ -439,6 +440,9 @@ struct pci_dev { #ifdef CONFIG_PCI_PASID u16 pasid_features; #endif +#ifdef CONFIG_PCI_P2PDMA + struct pci_p2pdma *p2pdma; +#endif phys_addr_t rom; /* Physical address if not from BAR */ size_t romlen; /* Length if not from BAR */ char *driver_override; /* Driver name to force a match */ -- 2.11.0 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Logan Gunthorpe Subject: [PATCH v5 01/13] PCI/P2PDMA: Support peer-to-peer memory Date: Thu, 30 Aug 2018 12:53:40 -0600 Message-ID: <20180830185352.3369-2-logang@deltatee.com> References: <20180830185352.3369-1-logang@deltatee.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <20180830185352.3369-1-logang-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linux-nvdimm-bounces-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org Sender: "Linux-nvdimm" To: linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-nvme-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org, linux-block-u79uwXL29TY76Z2rM5mHXA@public.gmane.org Cc: =?UTF-8?q?Christian=20K=C3=B6nig?= , Benjamin Herrenschmidt , Alex Williamson , =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Jason Gunthorpe , Bjorn Helgaas , Max Gurtovoy , Christoph Hellwig List-Id: linux-rdma@vger.kernel.org U29tZSBQQ0kgZGV2aWNlcyBtYXkgaGF2ZSBtZW1vcnkgbWFwcGVkIGluIGEgQkFSIHNwYWNlIHRo YXQncwppbnRlbmRlZCBmb3IgdXNlIGluIHBlZXItdG8tcGVlciB0cmFuc2FjdGlvbnMuIEluIG9y ZGVyIHRvIGVuYWJsZQpzdWNoIHRyYW5zYWN0aW9ucyB0aGUgbWVtb3J5IG11c3QgYmUgcmVnaXN0 ZXJlZCB3aXRoIFpPTkVfREVWSUNFIHBhZ2VzCnNvIGl0IGNhbiBiZSB1c2VkIGJ5IERNQSBpbnRl cmZhY2VzIGluIGV4aXN0aW5nIGRyaXZlcnMuCgpBZGQgYW4gaW50ZXJmYWNlIGZvciBvdGhlciBz dWJzeXN0ZW1zIHRvIGZpbmQgYW5kIGFsbG9jYXRlIGNodW5rcyBvZiBQMlAKbWVtb3J5IGFzIG5l Y2Vzc2FyeSB0byBmYWNpbGl0YXRlIHRyYW5zZmVycyBiZXR3ZWVuIHR3byBQQ0kgcGVlcnM6Cgpp bnQgcGNpX3AycGRtYV9hZGRfY2xpZW50KCk7CnN0cnVjdCBwY2lfZGV2ICpwY2lfcDJwbWVtX2Zp bmQoKTsKdm9pZCAqcGNpX2FsbG9jX3AycG1lbSgpOwoKVGhlIG5ldyBpbnRlcmZhY2UgcmVxdWly ZXMgYSBkcml2ZXIgdG8gY29sbGVjdCBhIGxpc3Qgb2YgY2xpZW50IGRldmljZXMKaW52b2x2ZWQg aW4gdGhlIHRyYW5zYWN0aW9uIHdpdGggdGhlIHBjaV9wMnBtZW1fYWRkX2NsaWVudCooKSBmdW5j dGlvbnMKdGhlbiBjYWxsIHBjaV9wMnBtZW1fZmluZCgpIHRvIG9idGFpbiBhbnkgc3VpdGFibGUg UDJQIG1lbW9yeS4gT25jZQp0aGlzIGlzIGRvbmUgdGhlIGxpc3QgaXMgYm91bmQgdG8gdGhlIG1l bW9yeSBhbmQgdGhlIGNhbGxpbmcgZHJpdmVyIGlzCmZyZWUgdG8gYWRkIGFuZCByZW1vdmUgY2xp ZW50cyBhcyBuZWNlc3NhcnkgKGFkZGluZyBpbmNvbXBhdGlibGUgY2xpZW50cwp3aWxsIGZhaWwp LiBXaXRoIGEgc3VpdGFibGUgcDJwbWVtIGRldmljZSwgbWVtb3J5IGNhbiB0aGVuIGJlCmFsbG9j YXRlZCB3aXRoIHBjaV9hbGxvY19wMnBtZW0oKSBmb3IgdXNlIGluIERNQSB0cmFuc2FjdGlvbnMu CgpEZXBlbmRpbmcgb24gaGFyZHdhcmUsIHVzaW5nIHBlZXItdG8tcGVlciBtZW1vcnkgbWF5IHJl ZHVjZSB0aGUgYmFuZHdpZHRoCm9mIHRoZSB0cmFuc2ZlciBidXQgY2FuIHNpZ25pZmljYW50bHkg cmVkdWNlIHByZXNzdXJlIG9uIHN5c3RlbSBtZW1vcnkuClRoaXMgbWF5IGJlIGRlc2lyYWJsZSBp biBtYW55IGNhc2VzOiBmb3IgZXhhbXBsZSBhIHN5c3RlbSBjb3VsZCBiZSBkZXNpZ25lZAp3aXRo IGEgc21hbGwgQ1BVIGNvbm5lY3RlZCB0byBhIFBDSWUgc3dpdGNoIGJ5IGEgc21hbGwgbnVtYmVy IG9mIGxhbmVzCndoaWNoIHdvdWxkIG1heGltaXplIHRoZSBudW1iZXIgb2YgbGFuZXMgYXZhaWxh YmxlIHRvIGNvbm5lY3QgdG8gTlZNZQpkZXZpY2VzLgoKVGhlIGNvZGUgaXMgZGVzaWduZWQgdG8g b25seSB1dGlsaXplIHRoZSBwMnBtZW0gZGV2aWNlIGlmIGFsbCB0aGUgZGV2aWNlcwppbnZvbHZl ZCBpbiBhIHRyYW5zZmVyIGFyZSBiZWhpbmQgdGhlIHNhbWUgUENJIGJyaWRnZS4gVGhpcyBpcyBi ZWNhdXNlIHdlCmhhdmUgbm8gd2F5IG9mIGtub3dpbmcgd2hldGhlciBwZWVyLXRvLXBlZXIgcm91 dGluZyBiZXR3ZWVuIFBDSWUgUm9vdCBQb3J0cwppcyBzdXBwb3J0ZWQgKFBDSWUgcjQuMCwgc2Vj IDEuMy4xKS4gQWRkaXRpb25hbGx5LCB0aGUgYmVuZWZpdHMgb2YgUDJQCnRyYW5zZmVycyB0aGF0 IGdvIHRocm91Z2ggdGhlIFJDIGlzIGxpbWl0ZWQgdG8gb25seSByZWR1Y2luZyBEUkFNIHVzYWdl CmFuZCwgaW4gc29tZSBjYXNlcywgY29kaW5nIGNvbnZlbmllbmNlLiBUaGUgUENJLVNJRyBtYXkg YmUgZXhwbG9yaW5nCmFkZGluZyBhIG5ldyBjYXBhYmlsaXR5IGJpdCB0byBhZHZlcnRpc2Ugd2hl dGhlciB0aGlzIGlzIHBvc3NpYmxlIGZvcgpmdXR1cmUgaGFyZHdhcmUuCgpUaGlzIGNvbW1pdCBp bmNsdWRlcyBzaWduaWZpY2FudCByZXdvcmsgYW5kIGZlZWRiYWNrIGZyb20gQ2hyaXN0b3BoCkhl bGx3aWcuCgpTaWduZWQtb2ZmLWJ5OiBDaHJpc3RvcGggSGVsbHdpZyA8aGNoQGxzdC5kZT4KU2ln bmVkLW9mZi1ieTogTG9nYW4gR3VudGhvcnBlIDxsb2dhbmdAZGVsdGF0ZWUuY29tPgotLS0KIGRy aXZlcnMvcGNpL0tjb25maWcgICAgICAgIHwgIDE3ICsKIGRyaXZlcnMvcGNpL01ha2VmaWxlICAg ICAgIHwgICAxICsKIGRyaXZlcnMvcGNpL3AycGRtYS5jICAgICAgIHwgNzYxICsrKysrKysrKysr KysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKwogaW5jbHVkZS9saW51eC9tZW1yZW1h cC5oICAgfCAgIDUgKwogaW5jbHVkZS9saW51eC9tbS5oICAgICAgICAgfCAgMTggKysKIGluY2x1 ZGUvbGludXgvcGNpLXAycGRtYS5oIHwgMTAyICsrKysrKwogaW5jbHVkZS9saW51eC9wY2kuaCAg ICAgICAgfCAgIDQgKwogNyBmaWxlcyBjaGFuZ2VkLCA5MDggaW5zZXJ0aW9ucygrKQogY3JlYXRl IG1vZGUgMTAwNjQ0IGRyaXZlcnMvcGNpL3AycGRtYS5jCiBjcmVhdGUgbW9kZSAxMDA2NDQgaW5j bHVkZS9saW51eC9wY2ktcDJwZG1hLmgKCmRpZmYgLS1naXQgYS9kcml2ZXJzL3BjaS9LY29uZmln IGIvZHJpdmVycy9wY2kvS2NvbmZpZwppbmRleCA1NmZmOGY2ZDMxZmMuLmRlYjY4YmU0ZmRhYyAx MDA2NDQKLS0tIGEvZHJpdmVycy9wY2kvS2NvbmZpZworKysgYi9kcml2ZXJzL3BjaS9LY29uZmln CkBAIC0xMzIsNiArMTMyLDIzIEBAIGNvbmZpZyBQQ0lfUEFTSUQKIAogCSAgSWYgdW5zdXJlLCBz YXkgTi4KIAorY29uZmlnIFBDSV9QMlBETUEKKwlib29sICJQQ0kgcGVlci10by1wZWVyIHRyYW5z ZmVyIHN1cHBvcnQiCisJZGVwZW5kcyBvbiBQQ0kgJiYgWk9ORV9ERVZJQ0UKKwlzZWxlY3QgR0VO RVJJQ19BTExPQ0FUT1IKKwloZWxwCisJICBFbmFibGXRlSBkcml2ZXJzIHRvIGRvIFBDSSBwZWVy LXRvLXBlZXIgdHJhbnNhY3Rpb25zIHRvIGFuZCBmcm9tCisJICBCQVJzIHRoYXQgYXJlIGV4cG9z ZWQgaW4gb3RoZXIgZGV2aWNlcyB0aGF0IGFyZSB0aGUgcGFydCBvZgorCSAgdGhlIGhpZXJhcmNo eSB3aGVyZSBwZWVyLXRvLXBlZXIgRE1BIGlzIGd1YXJhbnRlZWQgYnkgdGhlIFBDSQorCSAgc3Bl Y2lmaWNhdGlvbiB0byB3b3JrIChpZS4gYW55dGhpbmcgYmVsb3cgYSBzaW5nbGUgUENJIGJyaWRn ZSkuCisKKwkgIE1hbnkgUENJZSByb290IGNvbXBsZXhlcyBkbyBub3Qgc3VwcG9ydCBQMlAgdHJh bnNhY3Rpb25zIGFuZAorCSAgaXQncyBoYXJkIHRvIHRlbGwgd2hpY2ggc3VwcG9ydCBpdCBhdCBh bGwsIHNvIGF0IHRoaXMgdGltZSwKKwkgIFAyUCBETUEgdHJhbnNhdGlvbnMgbXVzdCBiZSBiZXR3 ZWVuIGRldmljZXMgYmVoaW5kIHRoZSBzYW1lIHJvb3QKKwkgIHBvcnQuCisKKwkgIElmIHVuc3Vy ZSwgc2F5IE4uCisKIGNvbmZpZyBQQ0lfTEFCRUwKIAlkZWZfYm9vbCB5IGlmIChETUkgfHwgQUNQ SSkKIAlkZXBlbmRzIG9uIFBDSQpkaWZmIC0tZ2l0IGEvZHJpdmVycy9wY2kvTWFrZWZpbGUgYi9k cml2ZXJzL3BjaS9NYWtlZmlsZQppbmRleCAxYjJjZmU1MWU4ZDcuLjg1ZjRhNzAzYjJiZSAxMDA2 NDQKLS0tIGEvZHJpdmVycy9wY2kvTWFrZWZpbGUKKysrIGIvZHJpdmVycy9wY2kvTWFrZWZpbGUK QEAgLTI2LDYgKzI2LDcgQEAgb2JqLSQoQ09ORklHX1BDSV9TWVNDQUxMKQkrPSBzeXNjYWxsLm8K IG9iai0kKENPTkZJR19QQ0lfU1RVQikJCSs9IHBjaS1zdHViLm8KIG9iai0kKENPTkZJR19QQ0lf UEZfU1RVQikJKz0gcGNpLXBmLXN0dWIubwogb2JqLSQoQ09ORklHX1BDSV9FQ0FNKQkJKz0gZWNh bS5vCitvYmotJChDT05GSUdfUENJX1AyUERNQSkJKz0gcDJwZG1hLm8KIG9iai0kKENPTkZJR19Y RU5fUENJREVWX0ZST05URU5EKSArPSB4ZW4tcGNpZnJvbnQubwogCiAjIEVuZHBvaW50IGxpYnJh cnkgbXVzdCBiZSBpbml0aWFsaXplZCBiZWZvcmUgaXRzIHVzZXJzCmRpZmYgLS1naXQgYS9kcml2 ZXJzL3BjaS9wMnBkbWEuYyBiL2RyaXZlcnMvcGNpL3AycGRtYS5jCm5ldyBmaWxlIG1vZGUgMTAw NjQ0CmluZGV4IDAwMDAwMDAwMDAwMC4uODhhYWVjNTM1MWNkCi0tLSAvZGV2L251bGwKKysrIGIv ZHJpdmVycy9wY2kvcDJwZG1hLmMKQEAgLTAsMCArMSw3NjEgQEAKKy8vIFNQRFgtTGljZW5zZS1J ZGVudGlmaWVyOiBHUEwtMi4wCisvKgorICogUENJIFBlZXIgMiBQZWVyIERNQSBzdXBwb3J0Lgor ICoKKyAqIENvcHlyaWdodCAoYykgMjAxNi0yMDE4LCBMb2dhbiBHdW50aG9ycGUKKyAqIENvcHly aWdodCAoYykgMjAxNi0yMDE3LCBNaWNyb3NlbWkgQ29ycG9yYXRpb24KKyAqIENvcHlyaWdodCAo YykgMjAxNywgQ2hyaXN0b3BoIEhlbGx3aWcKKyAqIENvcHlyaWdodCAoYykgMjAxOCwgRWlkZXRp Y29tIEluYy4KKyAqLworCisjZGVmaW5lIHByX2ZtdChmbXQpICJwY2ktcDJwZG1hOiAiIGZtdAor I2luY2x1ZGUgPGxpbnV4L3BjaS1wMnBkbWEuaD4KKyNpbmNsdWRlIDxsaW51eC9tb2R1bGUuaD4K KyNpbmNsdWRlIDxsaW51eC9zbGFiLmg+CisjaW5jbHVkZSA8bGludXgvZ2VuYWxsb2MuaD4KKyNp bmNsdWRlIDxsaW51eC9tZW1yZW1hcC5oPgorI2luY2x1ZGUgPGxpbnV4L3BlcmNwdS1yZWZjb3Vu dC5oPgorI2luY2x1ZGUgPGxpbnV4L3JhbmRvbS5oPgorI2luY2x1ZGUgPGxpbnV4L3NlcV9idWYu aD4KKworc3RydWN0IHBjaV9wMnBkbWEgeworCXN0cnVjdCBwZXJjcHVfcmVmIGRldm1hcF9yZWY7 CisJc3RydWN0IGNvbXBsZXRpb24gZGV2bWFwX3JlZl9kb25lOworCXN0cnVjdCBnZW5fcG9vbCAq cG9vbDsKKwlib29sIHAycG1lbV9wdWJsaXNoZWQ7Cit9OworCitzdGF0aWMgdm9pZCBwY2lfcDJw ZG1hX3BlcmNwdV9yZWxlYXNlKHN0cnVjdCBwZXJjcHVfcmVmICpyZWYpCit7CisJc3RydWN0IHBj aV9wMnBkbWEgKnAycCA9CisJCWNvbnRhaW5lcl9vZihyZWYsIHN0cnVjdCBwY2lfcDJwZG1hLCBk ZXZtYXBfcmVmKTsKKworCWNvbXBsZXRlX2FsbCgmcDJwLT5kZXZtYXBfcmVmX2RvbmUpOworfQor CitzdGF0aWMgdm9pZCBwY2lfcDJwZG1hX3BlcmNwdV9raWxsKHZvaWQgKmRhdGEpCit7CisJc3Ry dWN0IHBlcmNwdV9yZWYgKnJlZiA9IGRhdGE7CisKKwlpZiAocGVyY3B1X3JlZl9pc19keWluZyhy ZWYpKQorCQlyZXR1cm47CisKKwlwZXJjcHVfcmVmX2tpbGwocmVmKTsKK30KKworc3RhdGljIHZv aWQgcGNpX3AycGRtYV9yZWxlYXNlKHZvaWQgKmRhdGEpCit7CisJc3RydWN0IHBjaV9kZXYgKnBk ZXYgPSBkYXRhOworCisJaWYgKCFwZGV2LT5wMnBkbWEpCisJCXJldHVybjsKKworCXdhaXRfZm9y X2NvbXBsZXRpb24oJnBkZXYtPnAycGRtYS0+ZGV2bWFwX3JlZl9kb25lKTsKKwlwZXJjcHVfcmVm X2V4aXQoJnBkZXYtPnAycGRtYS0+ZGV2bWFwX3JlZik7CisKKwlnZW5fcG9vbF9kZXN0cm95KHBk ZXYtPnAycGRtYS0+cG9vbCk7CisJcGRldi0+cDJwZG1hID0gTlVMTDsKK30KKworc3RhdGljIGlu dCBwY2lfcDJwZG1hX3NldHVwKHN0cnVjdCBwY2lfZGV2ICpwZGV2KQoreworCWludCBlcnJvciA9 IC1FTk9NRU07CisJc3RydWN0IHBjaV9wMnBkbWEgKnAycDsKKworCXAycCA9IGRldm1fa3phbGxv YygmcGRldi0+ZGV2LCBzaXplb2YoKnAycCksIEdGUF9LRVJORUwpOworCWlmICghcDJwKQorCQly ZXR1cm4gLUVOT01FTTsKKworCXAycC0+cG9vbCA9IGdlbl9wb29sX2NyZWF0ZShQQUdFX1NISUZU LCBkZXZfdG9fbm9kZSgmcGRldi0+ZGV2KSk7CisJaWYgKCFwMnAtPnBvb2wpCisJCWdvdG8gb3V0 OworCisJaW5pdF9jb21wbGV0aW9uKCZwMnAtPmRldm1hcF9yZWZfZG9uZSk7CisJZXJyb3IgPSBw ZXJjcHVfcmVmX2luaXQoJnAycC0+ZGV2bWFwX3JlZiwKKwkJCXBjaV9wMnBkbWFfcGVyY3B1X3Jl bGVhc2UsIDAsIEdGUF9LRVJORUwpOworCWlmIChlcnJvcikKKwkJZ290byBvdXRfcG9vbF9kZXN0 cm95OworCisJcGVyY3B1X3JlZl9zd2l0Y2hfdG9fYXRvbWljX3N5bmMoJnAycC0+ZGV2bWFwX3Jl Zik7CisKKwllcnJvciA9IGRldm1fYWRkX2FjdGlvbl9vcl9yZXNldCgmcGRldi0+ZGV2LCBwY2lf cDJwZG1hX3JlbGVhc2UsIHBkZXYpOworCWlmIChlcnJvcikKKwkJZ290byBvdXRfcG9vbF9kZXN0 cm95OworCisJcGRldi0+cDJwZG1hID0gcDJwOworCisJcmV0dXJuIDA7CisKK291dF9wb29sX2Rl c3Ryb3k6CisJZ2VuX3Bvb2xfZGVzdHJveShwMnAtPnBvb2wpOworb3V0OgorCWRldm1fa2ZyZWUo JnBkZXYtPmRldiwgcDJwKTsKKwlyZXR1cm4gZXJyb3I7Cit9CisKKy8qKgorICogcGNpX3AycGRt YV9hZGRfcmVzb3VyY2UgLSBhZGQgbWVtb3J5IGZvciB1c2UgYXMgcDJwIG1lbW9yeQorICogQHBk ZXY6IHRoZSBkZXZpY2UgdG8gYWRkIHRoZSBtZW1vcnkgdG8KKyAqIEBiYXI6IFBDSSBCQVIgdG8g YWRkCisgKiBAc2l6ZTogc2l6ZSBvZiB0aGUgbWVtb3J5IHRvIGFkZCwgbWF5IGJlIHplcm8gdG8g dXNlIHRoZSB3aG9sZSBCQVIKKyAqIEBvZmZzZXQ6IG9mZnNldCBpbnRvIHRoZSBQQ0kgQkFSCisg KgorICogVGhlIG1lbW9yeSB3aWxsIGJlIGdpdmVuIFpPTkVfREVWSUNFIHN0cnVjdCBwYWdlcyBz byB0aGF0IGl0IG1heQorICogYmUgdXNlZCB3aXRoIGFueSBETUEgcmVxdWVzdC4KKyAqLworaW50 IHBjaV9wMnBkbWFfYWRkX3Jlc291cmNlKHN0cnVjdCBwY2lfZGV2ICpwZGV2LCBpbnQgYmFyLCBz aXplX3Qgc2l6ZSwKKwkJCSAgICB1NjQgb2Zmc2V0KQoreworCXN0cnVjdCBkZXZfcGFnZW1hcCAq cGdtYXA7CisJdm9pZCAqYWRkcjsKKwlpbnQgZXJyb3I7CisKKwlpZiAoIShwY2lfcmVzb3VyY2Vf ZmxhZ3MocGRldiwgYmFyKSAmIElPUkVTT1VSQ0VfTUVNKSkKKwkJcmV0dXJuIC1FSU5WQUw7CisK KwlpZiAob2Zmc2V0ID49IHBjaV9yZXNvdXJjZV9sZW4ocGRldiwgYmFyKSkKKwkJcmV0dXJuIC1F SU5WQUw7CisKKwlpZiAoIXNpemUpCisJCXNpemUgPSBwY2lfcmVzb3VyY2VfbGVuKHBkZXYsIGJh cikgLSBvZmZzZXQ7CisKKwlpZiAoc2l6ZSArIG9mZnNldCA+IHBjaV9yZXNvdXJjZV9sZW4ocGRl diwgYmFyKSkKKwkJcmV0dXJuIC1FSU5WQUw7CisKKwlpZiAoIXBkZXYtPnAycGRtYSkgeworCQll cnJvciA9IHBjaV9wMnBkbWFfc2V0dXAocGRldik7CisJCWlmIChlcnJvcikKKwkJCXJldHVybiBl cnJvcjsKKwl9CisKKwlwZ21hcCA9IGRldm1fa3phbGxvYygmcGRldi0+ZGV2LCBzaXplb2YoKnBn bWFwKSwgR0ZQX0tFUk5FTCk7CisJaWYgKCFwZ21hcCkKKwkJcmV0dXJuIC1FTk9NRU07CisKKwlw Z21hcC0+cmVzLnN0YXJ0ID0gcGNpX3Jlc291cmNlX3N0YXJ0KHBkZXYsIGJhcikgKyBvZmZzZXQ7 CisJcGdtYXAtPnJlcy5lbmQgPSBwZ21hcC0+cmVzLnN0YXJ0ICsgc2l6ZSAtIDE7CisJcGdtYXAt PnJlcy5mbGFncyA9IHBjaV9yZXNvdXJjZV9mbGFncyhwZGV2LCBiYXIpOworCXBnbWFwLT5yZWYg PSAmcGRldi0+cDJwZG1hLT5kZXZtYXBfcmVmOworCXBnbWFwLT50eXBlID0gTUVNT1JZX0RFVklD RV9QQ0lfUDJQRE1BOworCisJYWRkciA9IGRldm1fbWVtcmVtYXBfcGFnZXMoJnBkZXYtPmRldiwg cGdtYXApOworCWlmIChJU19FUlIoYWRkcikpIHsKKwkJZXJyb3IgPSBQVFJfRVJSKGFkZHIpOwor CQlnb3RvIHBnbWFwX2ZyZWU7CisJfQorCisJZXJyb3IgPSBnZW5fcG9vbF9hZGRfdmlydChwZGV2 LT5wMnBkbWEtPnBvb2wsICh1bnNpZ25lZCBsb25nKWFkZHIsCisJCQlwY2lfYnVzX2FkZHJlc3Mo cGRldiwgYmFyKSArIG9mZnNldCwKKwkJCXJlc291cmNlX3NpemUoJnBnbWFwLT5yZXMpLCBkZXZf dG9fbm9kZSgmcGRldi0+ZGV2KSk7CisJaWYgKGVycm9yKQorCQlnb3RvIHBnbWFwX2ZyZWU7CisK KwllcnJvciA9IGRldm1fYWRkX2FjdGlvbl9vcl9yZXNldCgmcGRldi0+ZGV2LCBwY2lfcDJwZG1h X3BlcmNwdV9raWxsLAorCQkJCQkgICZwZGV2LT5wMnBkbWEtPmRldm1hcF9yZWYpOworCWlmIChl cnJvcikKKwkJZ290byBwZ21hcF9mcmVlOworCisJcGNpX2luZm8ocGRldiwgImFkZGVkIHBlZXIt dG8tcGVlciBETUEgbWVtb3J5ICVwUlxuIiwKKwkJICZwZ21hcC0+cmVzKTsKKworCXJldHVybiAw OworCitwZ21hcF9mcmVlOgorCWRldnJlc19mcmVlKHBnbWFwKTsKKwlyZXR1cm4gZXJyb3I7Cit9 CitFWFBPUlRfU1lNQk9MX0dQTChwY2lfcDJwZG1hX2FkZF9yZXNvdXJjZSk7CisKK3N0YXRpYyBz dHJ1Y3QgcGNpX2RldiAqZmluZF9wYXJlbnRfcGNpX2RldihzdHJ1Y3QgZGV2aWNlICpkZXYpCit7 CisJc3RydWN0IGRldmljZSAqcGFyZW50OworCisJZGV2ID0gZ2V0X2RldmljZShkZXYpOworCisJ d2hpbGUgKGRldikgeworCQlpZiAoZGV2X2lzX3BjaShkZXYpKQorCQkJcmV0dXJuIHRvX3BjaV9k ZXYoZGV2KTsKKworCQlwYXJlbnQgPSBnZXRfZGV2aWNlKGRldi0+cGFyZW50KTsKKwkJcHV0X2Rl dmljZShkZXYpOworCQlkZXYgPSBwYXJlbnQ7CisJfQorCisJcmV0dXJuIE5VTEw7Cit9CisKKy8q CisgKiBDaGVjayBpZiBhIFBDSSBicmlkZ2UgaGFzIGl0J3MgQUNTIHJlZGlyZWN0aW9uIGJpdHMg c2V0IHRvIHJlZGlyZWN0IFAyUAorICogVExQcyB1cHN0cmVhbSB2aWEgQUNTLiBSZXR1cm5zIDEg aWYgdGhlIHBhY2tldHMgd2lsbCBiZSByZWRpcmVjdGVkCisgKiB1cHN0cmVhbSwgMCBvdGhlcndp c2UuCisgKi8KK3N0YXRpYyBpbnQgcGNpX2JyaWRnZV9oYXNfYWNzX3JlZGlyKHN0cnVjdCBwY2lf ZGV2ICpkZXYpCit7CisJaW50IHBvczsKKwl1MTYgY3RybDsKKworCXBvcyA9IHBjaV9maW5kX2V4 dF9jYXBhYmlsaXR5KGRldiwgUENJX0VYVF9DQVBfSURfQUNTKTsKKwlpZiAoIXBvcykKKwkJcmV0 dXJuIDA7CisKKwlwY2lfcmVhZF9jb25maWdfd29yZChkZXYsIHBvcyArIFBDSV9BQ1NfQ1RSTCwg JmN0cmwpOworCisJaWYgKGN0cmwgJiAoUENJX0FDU19SUiB8IFBDSV9BQ1NfQ1IgfCBQQ0lfQUNT X0VDKSkKKwkJcmV0dXJuIDE7CisKKwlyZXR1cm4gMDsKK30KKworc3RhdGljIHZvaWQgc2VxX2J1 Zl9wcmludF9idXNfZGV2Zm4oc3RydWN0IHNlcV9idWYgKmJ1Ziwgc3RydWN0IHBjaV9kZXYgKmRl dikKK3sKKwlpZiAoIWJ1ZikKKwkJcmV0dXJuOworCisJc2VxX2J1Zl9wcmludGYoYnVmLCAiJTA0 eDolMDJ4OiUwMnguJXg7IiwgcGNpX2RvbWFpbl9ucihkZXYtPmJ1cyksCisJCSAgICAgICBkZXYt PmJ1cy0+bnVtYmVyLCBQQ0lfU0xPVChkZXYtPmRldmZuKSwKKwkJICAgICAgIFBDSV9GVU5DKGRl di0+ZGV2Zm4pKTsKK30KKworLyoKKyAqIEZpbmQgdGhlIGRpc3RhbmNlIHRocm91Z2ggdGhlIG5l YXJlc3QgY29tbW9uIHVwc3RyZWFtIGJyaWRnZSBiZXR3ZWVuCisgKiB0d28gUENJIGRldmljZXMu CisgKgorICogSWYgdGhlIHR3byBkZXZpY2VzIGFyZSB0aGUgc2FtZSBkZXZpY2UgdGhlbiAwIHdp bGwgYmUgcmV0dXJuZWQuCisgKgorICogSWYgdGhlcmUgYXJlIHR3byB2aXJ0dWFsIGZ1bmN0aW9u cyBvZiB0aGUgc2FtZSBkZXZpY2UgYmVoaW5kIHRoZSBzYW1lCisgKiBicmlkZ2UgcG9ydCB0aGVu IDIgd2lsbCBiZSByZXR1cm5lZCAob25lIHN0ZXAgZG93biB0byB0aGUgUENJZSBzd2l0Y2gsCisg KiB0aGVuIG9uZSBzdGVwIGJhY2sgdG8gdGhlIHNhbWUgZGV2aWNlKS4KKyAqCisgKiBJbiB0aGUg Y2FzZSB3aGVyZSB0d28gZGV2aWNlcyBhcmUgY29ubmVjdGVkIHRvIHRoZSBzYW1lIFBDSWUgc3dp dGNoLCB0aGUKKyAqIHZhbHVlIDQgd2lsbCBiZSByZXR1cm5lZC4gVGhpcyBjb3JyZXNwb25kcyB0 byB0aGUgZm9sbG93aW5nIFBDSSB0cmVlOgorICoKKyAqICAgICAtKyAgUm9vdCBQb3J0CisgKiAg ICAgIFwrIFN3aXRjaCBVcHN0cmVhbSBQb3J0CisgKiAgICAgICArLSsgU3dpdGNoIERvd25zdHJl YW0gUG9ydAorICogICAgICAgKyBcLSBEZXZpY2UgQQorICogICAgICAgXC0rIFN3aXRjaCBEb3du c3RyZWFtIFBvcnQKKyAqICAgICAgICAgXC0gRGV2aWNlIEIKKyAqCisgKiBUaGUgZGlzdGFuY2Ug aXMgNCBiZWNhdXNlIHdlIHRyYXZlcnNlIGZyb20gRGV2aWNlIEEgdGhyb3VnaCB0aGUgZG93bnN0 cmVhbQorICogcG9ydCBvZiB0aGUgc3dpdGNoLCB0byB0aGUgY29tbW9uIHVwc3RyZWFtIHBvcnQs IGJhY2sgdXAgdG8gdGhlIHNlY29uZAorICogZG93bnN0cmVhbSBwb3J0IGFuZCB0aGVuIHRvIERl dmljZSBCLgorICoKKyAqIEFueSB0d28gZGV2aWNlcyB0aGF0IGRvbid0IGhhdmUgYSBjb21tb24g dXBzdHJlYW0gYnJpZGdlIHdpbGwgcmV0dXJuIC0xLgorICogSW4gdGhpcyB3YXkgZGV2aWNlcyBv biBzZXBhcmF0ZSBQQ0llIHJvb3QgcG9ydHMgd2lsbCBiZSByZWplY3RlZCwgd2hpY2gKKyAqIGlz IHdoYXQgd2Ugd2FudCBmb3IgcGVlci10by1wZWVyIHNlZWluZyBlYWNoIFBDSWUgcm9vdCBwb3J0 IGRlZmluZXMgYQorICogc2VwYXJhdGUgaGllcmFyY2h5IGRvbWFpbiBhbmQgdGhlcmUncyBubyB3 YXkgdG8gZGV0ZXJtaW5lIHdoZXRoZXIgdGhlIHJvb3QKKyAqIGNvbXBsZXggc3VwcG9ydHMgZm9y d2FyZGluZyBiZXR3ZWVuIHRoZW0uCisgKgorICogSW4gdGhlIGNhc2Ugd2hlcmUgdHdvIGRldmlj ZXMgYXJlIGNvbm5lY3RlZCB0byBkaWZmZXJlbnQgUENJZSBzd2l0Y2hlcywKKyAqIHRoaXMgZnVu Y3Rpb24gd2lsbCBzdGlsbCByZXR1cm4gYSBwb3NpdGl2ZSBkaXN0YW5jZSBhcyBsb25nIGFzIGJv dGgKKyAqIHN3aXRjaGVzIGV2ZW51dGFsbHkgaGF2ZSBhIGNvbW1vbiB1cHN0cmVhbSBicmlkZ2Uu IE5vdGUgdGhpcyBjb3ZlcnMKKyAqIHRoZSBjYXNlIG9mIHVzaW5nIG11bHRpcGxlIFBDSWUgc3dp dGNoZXMgdG8gYWNoaWV2ZSBhIGRlc2lyZWQgbGV2ZWwgb2YKKyAqIGZhbi1vdXQgZnJvbSBhIHJv b3QgcG9ydC4gVGhlIGV4YWN0IGRpc3RhbmNlIHdpbGwgYmUgYSBmdW5jdGlvbiBvZiB0aGUKKyAq IG51bWJlciBvZiBzd2l0Y2hlcyBiZXR3ZWVuIERldmljZSBBIGFuZCBEZXZpY2UgQi4KKyAqCisg KiBJZiBhIGJyaWRnZSB3aGljaCBoYXMgYW55IEFDUyByZWRpcmVjdGlvbiBiaXRzIHNldCBpcyBp biB0aGUgcGF0aAorICogdGhlbiB0aGlzIGZ1bmN0aW9ucyB3aWxsIHJldHVybiAtMi4gVGhpcyBp cyBzbyB3ZSByZWplY3QgYW55CisgKiBjYXNlcyB3aGVyZSB0aGUgVExQcyBhcmUgZm9yd2FyZGVk IHVwIGludG8gdGhlIHJvb3QgY29tcGxleC4KKyAqIEluIHRoaXMgY2FzZSwgYSBsaXN0IG9mIGFs bCBpbmZyaW5naW5nIGJyaWRnZSBhZGRyZXNzZXMgd2lsbCBiZQorICogcG9wdWxhdGVkIGluIGFj c19saXN0IChhc3N1bWluZyBpdCdzIG5vbi1udWxsKSBmb3IgcHJpbnRrIHB1cnBvc2VzLgorICov CitzdGF0aWMgaW50IHVwc3RyZWFtX2JyaWRnZV9kaXN0YW5jZShzdHJ1Y3QgcGNpX2RldiAqYSwK KwkJCQkgICAgc3RydWN0IHBjaV9kZXYgKmIsCisJCQkJICAgIHN0cnVjdCBzZXFfYnVmICphY3Nf bGlzdCkKK3sKKwlpbnQgZGlzdF9hID0gMDsKKwlpbnQgZGlzdF9iID0gMDsKKwlzdHJ1Y3QgcGNp X2RldiAqYmIgPSBOVUxMOworCWludCBhY3NfY250ID0gMDsKKworCS8qCisJICogTm90ZSwgd2Ug ZG9uJ3QgbmVlZCB0byB0YWtlIHJlZmVyZW5jZXMgdG8gZGV2aWNlcyByZXR1cm5lZCBieQorCSAq IHBjaV91cHN0cmVhbV9icmlkZ2UoKSBzZWVpbmcgd2UgaG9sZCBhIHJlZmVyZW5jZSB0byBhIGNo aWxkCisJICogZGV2aWNlIHdoaWNoIHdpbGwgYWxyZWFkeSBob2xkIGEgcmVmZXJlbmNlIHRvIHRo ZSB1cHN0cmVhbSBicmlkZ2UuCisJICovCisKKwl3aGlsZSAoYSkgeworCQlkaXN0X2IgPSAwOwor CisJCWlmIChwY2lfYnJpZGdlX2hhc19hY3NfcmVkaXIoYSkpIHsKKwkJCXNlcV9idWZfcHJpbnRf YnVzX2RldmZuKGFjc19saXN0LCBhKTsKKwkJCWFjc19jbnQrKzsKKwkJfQorCisJCWJiID0gYjsK KworCQl3aGlsZSAoYmIpIHsKKwkJCWlmIChhID09IGJiKQorCQkJCWdvdG8gY2hlY2tfYl9wYXRo X2FjczsKKworCQkJYmIgPSBwY2lfdXBzdHJlYW1fYnJpZGdlKGJiKTsKKwkJCWRpc3RfYisrOwor CQl9CisKKwkJYSA9IHBjaV91cHN0cmVhbV9icmlkZ2UoYSk7CisJCWRpc3RfYSsrOworCX0KKwor CXJldHVybiAtMTsKKworY2hlY2tfYl9wYXRoX2FjczoKKwliYiA9IGI7CisKKwl3aGlsZSAoYmIp IHsKKwkJaWYgKGEgPT0gYmIpCisJCQlicmVhazsKKworCQlpZiAocGNpX2JyaWRnZV9oYXNfYWNz X3JlZGlyKGJiKSkgeworCQkJc2VxX2J1Zl9wcmludF9idXNfZGV2Zm4oYWNzX2xpc3QsIGJiKTsK KwkJCWFjc19jbnQrKzsKKwkJfQorCisJCWJiID0gcGNpX3Vwc3RyZWFtX2JyaWRnZShiYik7CisJ fQorCisJaWYgKGFjc19jbnQpCisJCXJldHVybiAtMjsKKworCXJldHVybiBkaXN0X2EgKyBkaXN0 X2I7Cit9CisKK3N0YXRpYyBpbnQgdXBzdHJlYW1fYnJpZGdlX2Rpc3RhbmNlX3dhcm4oc3RydWN0 IHBjaV9kZXYgKnByb3ZpZGVyLAorCQkJCQkgc3RydWN0IHBjaV9kZXYgKmNsaWVudCkKK3sKKwlz dHJ1Y3Qgc2VxX2J1ZiBhY3NfbGlzdDsKKwlpbnQgcmV0OworCisJc2VxX2J1Zl9pbml0KCZhY3Nf bGlzdCwga21hbGxvYyhQQUdFX1NJWkUsIEdGUF9LRVJORUwpLCBQQUdFX1NJWkUpOworCisJcmV0 ID0gdXBzdHJlYW1fYnJpZGdlX2Rpc3RhbmNlKHByb3ZpZGVyLCBjbGllbnQsICZhY3NfbGlzdCk7 CisJaWYgKHJldCA9PSAtMikgeworCQlwY2lfd2FybihjbGllbnQsICJjYW5ub3QgYmUgdXNlZCBm b3IgcGVlci10by1wZWVyIERNQSBhcyBBQ1MgcmVkaXJlY3QgaXMgc2V0IGJldHdlZW4gdGhlIGNs aWVudCBhbmQgcHJvdmlkZXJcbiIpOworCQkvKiBEcm9wIGZpbmFsIHNlbWljb2xvbiAqLworCQlh Y3NfbGlzdC5idWZmZXJbYWNzX2xpc3QubGVuLTFdID0gMDsKKwkJcGNpX3dhcm4oY2xpZW50LCAi dG8gZGlzYWJsZSBBQ1MgcmVkaXJlY3QgZm9yIHRoaXMgcGF0aCwgYWRkIHRoZSBrZXJuZWwgcGFy YW1ldGVyOiBwY2k9ZGlzYWJsZV9hY3NfcmVkaXI9JXNcbiIsCisJCQkgYWNzX2xpc3QuYnVmZmVy KTsKKworCX0gZWxzZSBpZiAocmV0IDwgMCkgeworCQlwY2lfd2FybihjbGllbnQsICJjYW5ub3Qg YmUgdXNlZCBmb3IgcGVlci10by1wZWVyIERNQSBhcyB0aGUgY2xpZW50IGFuZCBwcm92aWRlciBk byBub3Qgc2hhcmUgYW4gdXBzdHJlYW0gYnJpZGdlXG4iKTsKKwl9CisKKwlrZnJlZShhY3NfbGlz dC5idWZmZXIpOworCisJcmV0dXJuIHJldDsKK30KKworc3RydWN0IHBjaV9wMnBkbWFfY2xpZW50 IHsKKwlzdHJ1Y3QgbGlzdF9oZWFkIGxpc3Q7CisJc3RydWN0IHBjaV9kZXYgKmNsaWVudDsKKwlz dHJ1Y3QgcGNpX2RldiAqcHJvdmlkZXI7Cit9OworCisvKioKKyAqIHBjaV9wMnBkbWFfYWRkX2Ns aWVudCAtIGFsbG9jYXRlIGEgbmV3IGVsZW1lbnQgaW4gYSBjbGllbnQgZGV2aWNlIGxpc3QKKyAq IEBoZWFkOiBsaXN0IGhlYWQgb2YgcDJwZG1hIGNsaWVudHMKKyAqIEBkZXY6IGRldmljZSB0byBh ZGQgdG8gdGhlIGxpc3QKKyAqCisgKiBUaGlzIGFkZHMgQGRldiB0byBhIGxpc3Qgb2YgY2xpZW50 cyB1c2VkIGJ5IGEgcDJwZG1hIGRldmljZS4KKyAqIFRoaXMgbGlzdCBzaG91bGQgYmUgcGFzc2Vk IHRvIHBjaV9wMnBtZW1fZmluZCgpLiBPbmNlIHBjaV9wMnBtZW1fZmluZCgpIGhhcworICogYmVl biBjYWxsZWQgc3VjY2Vzc2Z1bGx5LCB0aGUgbGlzdCB3aWxsIGJlIGJvdW5kIHRvIGEgc3BlY2lm aWMgcDJwZG1hCisgKiBkZXZpY2UgYW5kIG5ldyBjbGllbnRzIGNhbiBvbmx5IGJlIGFkZGVkIHRv IHRoZSBsaXN0IGlmIHRoZXkgYXJlCisgKiBzdXBwb3J0ZWQgYnkgdGhhdCBwMnBkbWEgZGV2aWNl LgorICoKKyAqIFRoZSBjYWxsZXIgaXMgZXhwZWN0ZWQgdG8gaGF2ZSBhIGxvY2sgd2hpY2ggcHJv dGVjdHMgQGhlYWQgYXMgbmVjZXNzYXJ5CisgKiBzbyB0aGF0IG5vbmUgb2YgdGhlIHBjaV9wMnAg ZnVuY3Rpb25zIGNhbiBiZSBjYWxsZWQgY29uY3VycmVudGx5CisgKiBvbiB0aGF0IGxpc3QuCisg KgorICogUmV0dXJucyAwIGlmIHRoZSBjbGllbnQgd2FzIHN1Y2Nlc3NmdWxseSBhZGRlZC4KKyAq LworaW50IHBjaV9wMnBkbWFfYWRkX2NsaWVudChzdHJ1Y3QgbGlzdF9oZWFkICpoZWFkLCBzdHJ1 Y3QgZGV2aWNlICpkZXYpCit7CisJc3RydWN0IHBjaV9wMnBkbWFfY2xpZW50ICppdGVtLCAqbmV3 X2l0ZW07CisJc3RydWN0IHBjaV9kZXYgKnByb3ZpZGVyID0gTlVMTDsKKwlzdHJ1Y3QgcGNpX2Rl diAqY2xpZW50OworCWludCByZXQ7CisKKwlpZiAoSVNfRU5BQkxFRChDT05GSUdfRE1BX1ZJUlRf T1BTKSAmJiBkZXYtPmRtYV9vcHMgPT0gJmRtYV92aXJ0X29wcykgeworCQlkZXZfd2FybihkZXYs ICJjYW5ub3QgYmUgdXNlZCBmb3IgcGVlci10by1wZWVyIERNQSBiZWNhdXNlIHRoZSBkcml2ZXIg bWFrZXMgdXNlIG9mIGRtYV92aXJ0X29wc1xuIik7CisJCXJldHVybiAtRU5PREVWOworCX0KKwor CWNsaWVudCA9IGZpbmRfcGFyZW50X3BjaV9kZXYoZGV2KTsKKwlpZiAoIWNsaWVudCkgeworCQlk ZXZfd2FybihkZXYsICJjYW5ub3QgYmUgdXNlZCBmb3IgcGVlci10by1wZWVyIERNQSBhcyBpdCBp cyBub3QgYSBQQ0kgZGV2aWNlXG4iKTsKKwkJcmV0dXJuIC1FTk9ERVY7CisJfQorCisJaXRlbSA9 IGxpc3RfZmlyc3RfZW50cnlfb3JfbnVsbChoZWFkLCBzdHJ1Y3QgcGNpX3AycGRtYV9jbGllbnQs IGxpc3QpOworCWlmIChpdGVtICYmIGl0ZW0tPnByb3ZpZGVyKSB7CisJCXByb3ZpZGVyID0gaXRl bS0+cHJvdmlkZXI7CisKKwkJcmV0ID0gdXBzdHJlYW1fYnJpZGdlX2Rpc3RhbmNlX3dhcm4ocHJv dmlkZXIsIGNsaWVudCk7CisJCWlmIChyZXQgPCAwKSB7CisJCQlyZXQgPSAtRVhERVY7CisJCQln b3RvIHB1dF9jbGllbnQ7CisJCX0KKwl9CisKKwluZXdfaXRlbSA9IGt6YWxsb2Moc2l6ZW9mKCpu ZXdfaXRlbSksIEdGUF9LRVJORUwpOworCWlmICghbmV3X2l0ZW0pIHsKKwkJcmV0ID0gLUVOT01F TTsKKwkJZ290byBwdXRfY2xpZW50OworCX0KKworCW5ld19pdGVtLT5jbGllbnQgPSBjbGllbnQ7 CisJbmV3X2l0ZW0tPnByb3ZpZGVyID0gcGNpX2Rldl9nZXQocHJvdmlkZXIpOworCisJbGlzdF9h ZGRfdGFpbCgmbmV3X2l0ZW0tPmxpc3QsIGhlYWQpOworCisJcmV0dXJuIDA7CisKK3B1dF9jbGll bnQ6CisJcGNpX2Rldl9wdXQoY2xpZW50KTsKKwlyZXR1cm4gcmV0OworfQorRVhQT1JUX1NZTUJP TF9HUEwocGNpX3AycGRtYV9hZGRfY2xpZW50KTsKKworc3RhdGljIHZvaWQgcGNpX3AycGRtYV9j bGllbnRfZnJlZShzdHJ1Y3QgcGNpX3AycGRtYV9jbGllbnQgKml0ZW0pCit7CisJbGlzdF9kZWwo Jml0ZW0tPmxpc3QpOworCXBjaV9kZXZfcHV0KGl0ZW0tPmNsaWVudCk7CisJcGNpX2Rldl9wdXQo aXRlbS0+cHJvdmlkZXIpOworCWtmcmVlKGl0ZW0pOworfQorCisvKioKKyAqIHBjaV9wMnBkbWFf cmVtb3ZlX2NsaWVudCAtIHJlbW92ZSBhbmQgZnJlZSBhIHAycGRtYSBjbGllbnQKKyAqIEBoZWFk OiBsaXN0IGhlYWQgb2YgcDJwZG1hIGNsaWVudHMKKyAqIEBkZXY6IGRldmljZSB0byByZW1vdmUg ZnJvbSB0aGUgbGlzdAorICoKKyAqIFRoaXMgcmVtb3ZlcyBAZGV2IGZyb20gYSBsaXN0IG9mIGNs aWVudHMgdXNlZCBieSBhIHAycGRtYSBkZXZpY2UuCisgKiBUaGUgY2FsbGVyIGlzIGV4cGVjdGVk IHRvIGhhdmUgYSBsb2NrIHdoaWNoIHByb3RlY3RzIEBoZWFkIGFzIG5lY2Vzc2FyeQorICogc28g dGhhdCBub25lIG9mIHRoZSBwY2lfcDJwIGZ1bmN0aW9ucyBjYW4gYmUgY2FsbGVkIGNvbmN1cnJl bnRseQorICogb24gdGhhdCBsaXN0LgorICovCit2b2lkIHBjaV9wMnBkbWFfcmVtb3ZlX2NsaWVu dChzdHJ1Y3QgbGlzdF9oZWFkICpoZWFkLCBzdHJ1Y3QgZGV2aWNlICpkZXYpCit7CisJc3RydWN0 IHBjaV9wMnBkbWFfY2xpZW50ICpwb3MsICp0bXA7CisJc3RydWN0IHBjaV9kZXYgKnBkZXY7CisK KwlwZGV2ID0gZmluZF9wYXJlbnRfcGNpX2RldihkZXYpOworCWlmICghcGRldikKKwkJcmV0dXJu OworCisJbGlzdF9mb3JfZWFjaF9lbnRyeV9zYWZlKHBvcywgdG1wLCBoZWFkLCBsaXN0KSB7CisJ CWlmIChwb3MtPmNsaWVudCAhPSBwZGV2KQorCQkJY29udGludWU7CisKKwkJcGNpX3AycGRtYV9j bGllbnRfZnJlZShwb3MpOworCX0KKworCXBjaV9kZXZfcHV0KHBkZXYpOworfQorRVhQT1JUX1NZ TUJPTF9HUEwocGNpX3AycGRtYV9yZW1vdmVfY2xpZW50KTsKKworLyoqCisgKiBwY2lfcDJwZG1h X2NsaWVudF9saXN0X2ZyZWUgLSBmcmVlIGFuIGVudGlyZSBsaXN0IG9mIHAycGRtYSBjbGllbnRz CisgKiBAaGVhZDogbGlzdCBoZWFkIG9mIHAycGRtYSBjbGllbnRzCisgKgorICogVGhpcyByZW1v dmVzIGFsbCBkZXZpY2VzIGluIGEgbGlzdCBvZiBjbGllbnRzIHVzZWQgYnkgYSBwMnBkbWEgZGV2 aWNlLgorICogVGhlIGNhbGxlciBpcyBleHBlY3RlZCB0byBoYXZlIGEgbG9jayB3aGljaCBwcm90 ZWN0cyBAaGVhZCBhcyBuZWNlc3NhcnkKKyAqIHNvIHRoYXQgbm9uZSBvZiB0aGUgcGNpX3AycGRt YSBmdW5jdGlvbnMgY2FuIGJlIGNhbGxlZCBjb25jdXJyZW50bHkKKyAqIG9uIHRoYXQgbGlzdC4K KyAqLwordm9pZCBwY2lfcDJwZG1hX2NsaWVudF9saXN0X2ZyZWUoc3RydWN0IGxpc3RfaGVhZCAq aGVhZCkKK3sKKwlzdHJ1Y3QgcGNpX3AycGRtYV9jbGllbnQgKnBvcywgKnRtcDsKKworCWxpc3Rf Zm9yX2VhY2hfZW50cnlfc2FmZShwb3MsIHRtcCwgaGVhZCwgbGlzdCkKKwkJcGNpX3AycGRtYV9j bGllbnRfZnJlZShwb3MpOworfQorRVhQT1JUX1NZTUJPTF9HUEwocGNpX3AycGRtYV9jbGllbnRf bGlzdF9mcmVlKTsKKworLyoqCisgKiBwY2lfcDJwZG1hX2Rpc3RhbmNlIC0gRGV0ZXJtaXZlIHRo ZSBjdW11bGF0aXZlIGRpc3RhbmNlIGJldHdlZW4KKyAqCWEgcDJwZG1hIHByb3ZpZGVyIGFuZCB0 aGUgY2xpZW50cyBpbiB1c2UuCisgKiBAcHJvdmlkZXI6IHAycGRtYSBwcm92aWRlciB0byBjaGVj ayBhZ2FpbnN0IHRoZSBjbGllbnQgbGlzdAorICogQGNsaWVudHM6IGxpc3Qgb2YgZGV2aWNlcyB0 byBjaGVjayAoTlVMTC10ZXJtaW5hdGVkKQorICogQHZlcmJvc2U6IGlmIHRydWUsIHByaW50IHdh cm5pbmdzIGZvciBkZXZpY2VzIHdoZW4gd2UgcmV0dXJuIC0xCisgKgorICogUmV0dXJucyAtMSBp ZiBhbnkgb2YgdGhlIGNsaWVudHMgYXJlIG5vdCBjb21wYXRpYmxlIChiZWhpbmQgdGhlIHNhbWUK KyAqIHJvb3QgcG9ydCBhcyB0aGUgcHJvdmlkZXIpLCBvdGhlcndpc2UgcmV0dXJucyBhIHBvc2l0 aXZlIG51bWJlciB3aGVyZQorICogdGhlIGxvd2VyIG51bWJlciBpcyB0aGUgcHJlZmVycmFibGUg Y2hvaWNlLiAoSWYgdGhlcmUncyBvbmUgY2xpZW50CisgKiB0aGF0J3MgdGhlIHNhbWUgYXMgdGhl IHByb3ZpZGVyIGl0IHdpbGwgcmV0dXJuIDAsIHdoaWNoIGlzIGJlc3QgY2hvaWNlKS4KKyAqCisg KiBGb3Igbm93LCAiY29tcGF0aWJsZSIgbWVhbnMgdGhlIHByb3ZpZGVyIGFuZCB0aGUgY2xpZW50 cyBhcmUgYWxsIGJlaGluZAorICogdGhlIHNhbWUgUENJIHJvb3QgcG9ydC4gVGhpcyBjdXRzIG91 dCBjYXNlcyB0aGF0IG1heSB3b3JrIGJ1dCBpcyBzYWZlc3QKKyAqIGZvciB0aGUgdXNlci4gRnV0 dXJlIHdvcmsgY2FuIGV4cGFuZCB0aGlzIHRvIHdoaXRlLWxpc3Qgcm9vdCBjb21wbGV4ZXMgdGhh dAorICogY2FuIHNhZmVseSBmb3J3YXJkIGJldHdlZW4gZWFjaCBwb3J0cy4KKyAqLworaW50IHBj aV9wMnBkbWFfZGlzdGFuY2Uoc3RydWN0IHBjaV9kZXYgKnByb3ZpZGVyLCBzdHJ1Y3QgbGlzdF9o ZWFkICpjbGllbnRzLAorCQkJYm9vbCB2ZXJib3NlKQoreworCXN0cnVjdCBwY2lfcDJwZG1hX2Ns aWVudCAqcG9zOworCWludCByZXQ7CisJaW50IGRpc3RhbmNlID0gMDsKKwlib29sIG5vdF9zdXBw b3J0ZWQgPSBmYWxzZTsKKworCWlmIChsaXN0X2VtcHR5KGNsaWVudHMpKQorCQlyZXR1cm4gLTE7 CisKKwlsaXN0X2Zvcl9lYWNoX2VudHJ5KHBvcywgY2xpZW50cywgbGlzdCkgeworCQlpZiAodmVy Ym9zZSkKKwkJCXJldCA9IHVwc3RyZWFtX2JyaWRnZV9kaXN0YW5jZV93YXJuKHByb3ZpZGVyLAor CQkJCQkJCSAgICBwb3MtPmNsaWVudCk7CisJCWVsc2UKKwkJCXJldCA9IHVwc3RyZWFtX2JyaWRn ZV9kaXN0YW5jZShwcm92aWRlciwgcG9zLT5jbGllbnQsCisJCQkJCQkgICAgICAgTlVMTCk7CisK KwkJaWYgKHJldCA8IDApCisJCQlub3Rfc3VwcG9ydGVkID0gdHJ1ZTsKKworCQlpZiAobm90X3N1 cHBvcnRlZCAmJiAhdmVyYm9zZSkKKwkJCWJyZWFrOworCisJCWRpc3RhbmNlICs9IHJldDsKKwl9 CisKKwlpZiAobm90X3N1cHBvcnRlZCkKKwkJcmV0dXJuIC0xOworCisJcmV0dXJuIGRpc3RhbmNl OworfQorRVhQT1JUX1NZTUJPTF9HUEwocGNpX3AycGRtYV9kaXN0YW5jZSk7CisKKy8qKgorICog cGNpX3AycGRtYV9hc3NpZ25fcHJvdmlkZXIgLSBDaGVjayBjb21wYXRpYmlseSAoYXMgcGVyIHBj aV9wMnBkbWFfZGlzdGFuY2UpCisgKglhbmQgYXNzaWduIGEgcHJvdmlkZXIgdG8gYSBsaXN0IG9m IGNsaWVudHMKKyAqIEBwcm92aWRlcjogcDJwZG1hIHByb3ZpZGVyIHRvIGFzc2lnbiB0byB0aGUg Y2xpZW50IGxpc3QKKyAqIEBjbGllbnRzOiBsaXN0IG9mIGRldmljZXMgdG8gY2hlY2sgKE5VTEwt dGVybWluYXRlZCkKKyAqCisgKiBSZXR1cm5zIGZhbHNlIGlmIGFueSBvZiB0aGUgY2xpZW50cyBh cmUgbm90IGNvbXBhdGlibGUsIHRydWUgaWYgdGhlCisgKiBwcm92aWRlciB3YXMgc3VjY2Vzc2Z1 bGx5IGFzc2lnbmVkIHRvIHRoZSBjbGllbnRzLgorICovCitib29sIHBjaV9wMnBkbWFfYXNzaWdu X3Byb3ZpZGVyKHN0cnVjdCBwY2lfZGV2ICpwcm92aWRlciwKKwkJCQlzdHJ1Y3QgbGlzdF9oZWFk ICpjbGllbnRzKQoreworCXN0cnVjdCBwY2lfcDJwZG1hX2NsaWVudCAqcG9zOworCisJaWYgKHBj aV9wMnBkbWFfZGlzdGFuY2UocHJvdmlkZXIsIGNsaWVudHMsIHRydWUpIDwgMCkKKwkJcmV0dXJu IGZhbHNlOworCisJbGlzdF9mb3JfZWFjaF9lbnRyeShwb3MsIGNsaWVudHMsIGxpc3QpCisJCXBv cy0+cHJvdmlkZXIgPSBwcm92aWRlcjsKKworCXJldHVybiB0cnVlOworfQorRVhQT1JUX1NZTUJP TF9HUEwocGNpX3AycGRtYV9hc3NpZ25fcHJvdmlkZXIpOworCisvKioKKyAqIHBjaV9oYXNfcDJw bWVtIC0gY2hlY2sgaWYgYSBnaXZlbiBQQ0kgZGV2aWNlIGhhcyBwdWJsaXNoZWQgYW55IHAycG1l bQorICogQHBkZXY6IFBDSSBkZXZpY2UgdG8gY2hlY2sKKyAqLworYm9vbCBwY2lfaGFzX3AycG1l bShzdHJ1Y3QgcGNpX2RldiAqcGRldikKK3sKKwlyZXR1cm4gcGRldi0+cDJwZG1hICYmIHBkZXYt PnAycGRtYS0+cDJwbWVtX3B1Ymxpc2hlZDsKK30KK0VYUE9SVF9TWU1CT0xfR1BMKHBjaV9oYXNf cDJwbWVtKTsKKworLyoqCisgKiBwY2lfcDJwbWVtX2ZpbmQgLSBmaW5kIGEgcGVlci10by1wZWVy IERNQSBtZW1vcnkgZGV2aWNlIGNvbXBhdGlibGUgd2l0aAorICoJdGhlIHNwZWNpZmllZCBsaXN0 IG9mIGNsaWVudHMgYW5kIHNob3J0ZXN0IGRpc3RhbmNlIChhcyBkZXRlcm1pbmVkCisgKglieSBw Y2lfcDJwbWVtX2RtYSgpKQorICogQGNsaWVudHM6IGxpc3Qgb2YgZGV2aWNlcyB0byBjaGVjayAo TlVMTC10ZXJtaW5hdGVkKQorICoKKyAqIElmIG11bHRpcGxlIGRldmljZXMgYXJlIGJlaGluZCB0 aGUgc2FtZSBzd2l0Y2gsIHRoZSBvbmUgImNsb3Nlc3QiIHRvIHRoZQorICogY2xpZW50IGRldmlj ZXMgaW4gdXNlIHdpbGwgYmUgY2hvc2VuIGZpcnN0LiAoU28gaWYgb25lIG9mIHRoZSBwcm92aWRl cnMgYXJlCisgKiB0aGUgc2FtZSBhcyBvbmUgb2YgdGhlIGNsaWVudHMsIHRoYXQgcHJvdmlkZXIg d2lsbCBiZSB1c2VkIGFoZWFkIG9mIGFueQorICogb3RoZXIgcHJvdmlkZXJzIHRoYXQgYXJlIHVu cmVsYXRlZCkuIElmIG11bHRpcGxlIHByb3ZpZGVycyBhcmUgYW4gZXF1YWwKKyAqIGRpc3RhbmNl IGF3YXksIG9uZSB3aWxsIGJlIGNob3NlbiBhdCByYW5kb20uCisgKgorICogUmV0dXJucyBhIHBv aW50ZXIgdG8gdGhlIFBDSSBkZXZpY2Ugd2l0aCBhIHJlZmVyZW5jZSB0YWtlbiAodXNlIHBjaV9k ZXZfcHV0CisgKiB0byByZXR1cm4gdGhlIHJlZmVyZW5jZSkgb3IgTlVMTCBpZiBubyBjb21wYXRp YmxlIGRldmljZSBpcyBmb3VuZC4gVGhlCisgKiBmb3VuZCBwcm92aWRlciB3aWxsIGFsc28gYmUg YXNzaWduZWQgdG8gdGhlIGNsaWVudCBsaXN0LgorICovCitzdHJ1Y3QgcGNpX2RldiAqcGNpX3Ay cG1lbV9maW5kKHN0cnVjdCBsaXN0X2hlYWQgKmNsaWVudHMpCit7CisJc3RydWN0IHBjaV9kZXYg KnBkZXYgPSBOVUxMOworCXN0cnVjdCBwY2lfcDJwZG1hX2NsaWVudCAqcG9zOworCWludCBkaXN0 YW5jZTsKKwlpbnQgY2xvc2VzdF9kaXN0YW5jZSA9IElOVF9NQVg7CisJc3RydWN0IHBjaV9kZXYg KipjbG9zZXN0X3BkZXZzOworCWludCBkZXZfY250ID0gMDsKKwljb25zdCBpbnQgbWF4X2RldnMg PSBQQUdFX1NJWkUgLyBzaXplb2YoKmNsb3Nlc3RfcGRldnMpOworCWludCBpOworCisJY2xvc2Vz dF9wZGV2cyA9IGttYWxsb2MoUEFHRV9TSVpFLCBHRlBfS0VSTkVMKTsKKworCXdoaWxlICgocGRl diA9IHBjaV9nZXRfZGV2aWNlKFBDSV9BTllfSUQsIFBDSV9BTllfSUQsIHBkZXYpKSkgeworCQlp ZiAoIXBjaV9oYXNfcDJwbWVtKHBkZXYpKQorCQkJY29udGludWU7CisKKwkJZGlzdGFuY2UgPSBw Y2lfcDJwZG1hX2Rpc3RhbmNlKHBkZXYsIGNsaWVudHMsIGZhbHNlKTsKKwkJaWYgKGRpc3RhbmNl IDwgMCB8fCBkaXN0YW5jZSA+IGNsb3Nlc3RfZGlzdGFuY2UpCisJCQljb250aW51ZTsKKworCQlp ZiAoZGlzdGFuY2UgPT0gY2xvc2VzdF9kaXN0YW5jZSAmJiBkZXZfY250ID49IG1heF9kZXZzKQor CQkJY29udGludWU7CisKKwkJaWYgKGRpc3RhbmNlIDwgY2xvc2VzdF9kaXN0YW5jZSkgeworCQkJ Zm9yIChpID0gMDsgaSA8IGRldl9jbnQ7IGkrKykKKwkJCQlwY2lfZGV2X3B1dChjbG9zZXN0X3Bk ZXZzW2ldKTsKKworCQkJZGV2X2NudCA9IDA7CisJCQljbG9zZXN0X2Rpc3RhbmNlID0gZGlzdGFu Y2U7CisJCX0KKworCQljbG9zZXN0X3BkZXZzW2Rldl9jbnQrK10gPSBwY2lfZGV2X2dldChwZGV2 KTsKKwl9CisKKwlpZiAoZGV2X2NudCkKKwkJcGRldiA9IHBjaV9kZXZfZ2V0KGNsb3Nlc3RfcGRl dnNbcHJhbmRvbV91MzJfbWF4KGRldl9jbnQpXSk7CisKKwlmb3IgKGkgPSAwOyBpIDwgZGV2X2Nu dDsgaSsrKQorCQlwY2lfZGV2X3B1dChjbG9zZXN0X3BkZXZzW2ldKTsKKworCWlmIChwZGV2KQor CQlsaXN0X2Zvcl9lYWNoX2VudHJ5KHBvcywgY2xpZW50cywgbGlzdCkKKwkJCXBvcy0+cHJvdmlk ZXIgPSBwZGV2OworCisJa2ZyZWUoY2xvc2VzdF9wZGV2cyk7CisJcmV0dXJuIHBkZXY7Cit9CitF WFBPUlRfU1lNQk9MX0dQTChwY2lfcDJwbWVtX2ZpbmQpOworCisvKioKKyAqIHBjaV9hbGxvY19w MnBfbWVtIC0gYWxsb2NhdGUgcGVlci10by1wZWVyIERNQSBtZW1vcnkKKyAqIEBwZGV2OiB0aGUg ZGV2aWNlIHRvIGFsbG9jYXRlIG1lbW9yeSBmcm9tCisgKiBAc2l6ZTogbnVtYmVyIG9mIGJ5dGVz IHRvIGFsbG9jYXRlCisgKgorICogUmV0dXJucyB0aGUgYWxsb2NhdGVkIG1lbW9yeSBvciBOVUxM IG9uIGVycm9yLgorICovCit2b2lkICpwY2lfYWxsb2NfcDJwbWVtKHN0cnVjdCBwY2lfZGV2ICpw ZGV2LCBzaXplX3Qgc2l6ZSkKK3sKKwl2b2lkICpyZXQ7CisKKwlpZiAodW5saWtlbHkoIXBkZXYt PnAycGRtYSkpCisJCXJldHVybiBOVUxMOworCisJaWYgKHVubGlrZWx5KCFwZXJjcHVfcmVmX3Ry eWdldF9saXZlKCZwZGV2LT5wMnBkbWEtPmRldm1hcF9yZWYpKSkKKwkJcmV0dXJuIE5VTEw7CisK KwlyZXQgPSAodm9pZCAqKWdlbl9wb29sX2FsbG9jKHBkZXYtPnAycGRtYS0+cG9vbCwgc2l6ZSk7 CisKKwlpZiAodW5saWtlbHkoIXJldCkpCisJCXBlcmNwdV9yZWZfcHV0KCZwZGV2LT5wMnBkbWEt PmRldm1hcF9yZWYpOworCisJcmV0dXJuIHJldDsKK30KK0VYUE9SVF9TWU1CT0xfR1BMKHBjaV9h bGxvY19wMnBtZW0pOworCisvKioKKyAqIHBjaV9mcmVlX3AycG1lbSAtIGFsbG9jYXRlIHBlZXIt dG8tcGVlciBETUEgbWVtb3J5CisgKiBAcGRldjogdGhlIGRldmljZSB0aGUgbWVtb3J5IHdhcyBh bGxvY2F0ZWQgZnJvbQorICogQGFkZHI6IGFkZHJlc3Mgb2YgdGhlIG1lbW9yeSB0aGF0IHdhcyBh bGxvY2F0ZWQKKyAqIEBzaXplOiBudW1iZXIgb2YgYnl0ZXMgdGhhdCB3YXMgYWxsb2NhdGVkCisg Ki8KK3ZvaWQgcGNpX2ZyZWVfcDJwbWVtKHN0cnVjdCBwY2lfZGV2ICpwZGV2LCB2b2lkICphZGRy LCBzaXplX3Qgc2l6ZSkKK3sKKwlnZW5fcG9vbF9mcmVlKHBkZXYtPnAycGRtYS0+cG9vbCwgKHVp bnRwdHJfdClhZGRyLCBzaXplKTsKKwlwZXJjcHVfcmVmX3B1dCgmcGRldi0+cDJwZG1hLT5kZXZt YXBfcmVmKTsKK30KK0VYUE9SVF9TWU1CT0xfR1BMKHBjaV9mcmVlX3AycG1lbSk7CisKKy8qKgor ICogcGNpX3ZpcnRfdG9fYnVzIC0gcmV0dXJuIHRoZSBQQ0kgYnVzIGFkZHJlc3MgZm9yIGEgZ2l2 ZW4gdmlydHVhbAorICoJYWRkcmVzcyBvYnRhaW5lZCB3aXRoIHBjaV9hbGxvY19wMnBtZW0oKQor ICogQHBkZXY6IHRoZSBkZXZpY2UgdGhlIG1lbW9yeSB3YXMgYWxsb2NhdGVkIGZyb20KKyAqIEBh ZGRyOiBhZGRyZXNzIG9mIHRoZSBtZW1vcnkgdGhhdCB3YXMgYWxsb2NhdGVkCisgKi8KK3BjaV9i dXNfYWRkcl90IHBjaV9wMnBtZW1fdmlydF90b19idXMoc3RydWN0IHBjaV9kZXYgKnBkZXYsIHZv aWQgKmFkZHIpCit7CisJaWYgKCFhZGRyKQorCQlyZXR1cm4gMDsKKwlpZiAoIXBkZXYtPnAycGRt YSkKKwkJcmV0dXJuIDA7CisKKwkvKgorCSAqIE5vdGU6IHdoZW4gd2UgYWRkZWQgdGhlIG1lbW9y eSB0byB0aGUgcG9vbCB3ZSB1c2VkIHRoZSBQQ0kKKwkgKiBidXMgYWRkcmVzcyBhcyB0aGUgcGh5 c2ljYWwgYWRkcmVzcy4gU28gZ2VuX3Bvb2xfdmlydF90b19waHlzKCkKKwkgKiBhY3R1YWxseSBy ZXR1cm5zIHRoZSBidXMgYWRkcmVzcyBkZXNwaXRlIHRoZSBtaXNsZWFkaW5nIG5hbWUuCisJICov CisJcmV0dXJuIGdlbl9wb29sX3ZpcnRfdG9fcGh5cyhwZGV2LT5wMnBkbWEtPnBvb2wsICh1bnNp Z25lZCBsb25nKWFkZHIpOworfQorRVhQT1JUX1NZTUJPTF9HUEwocGNpX3AycG1lbV92aXJ0X3Rv X2J1cyk7CisKKy8qKgorICogcGNpX3AycG1lbV9hbGxvY19zZ2wgLSBhbGxvY2F0ZSBwZWVyLXRv LXBlZXIgRE1BIG1lbW9yeSBpbiBhIHNjYXR0ZXJsaXN0CisgKiBAcGRldjogdGhlIGRldmljZSB0 byBhbGxvY2F0ZSBtZW1vcnkgZnJvbQorICogQHNnbDogdGhlIGFsbG9jYXRlZCBzY2F0dGVybGlz dAorICogQG5lbnRzOiB0aGUgbnVtYmVyIG9mIFNHIGVudHJpZXMgaW4gdGhlIGxpc3QKKyAqIEBs ZW5ndGg6IG51bWJlciBvZiBieXRlcyB0byBhbGxvY2F0ZQorICoKKyAqIFJldHVybnMgMCBvbiBz dWNjZXNzCisgKi8KK3N0cnVjdCBzY2F0dGVybGlzdCAqcGNpX3AycG1lbV9hbGxvY19zZ2woc3Ry dWN0IHBjaV9kZXYgKnBkZXYsCisJCQkJCSB1bnNpZ25lZCBpbnQgKm5lbnRzLCB1MzIgbGVuZ3Ro KQoreworCXN0cnVjdCBzY2F0dGVybGlzdCAqc2c7CisJdm9pZCAqYWRkcjsKKworCXNnID0ga3ph bGxvYyhzaXplb2YoKnNnKSwgR0ZQX0tFUk5FTCk7CisJaWYgKCFzZykKKwkJcmV0dXJuIE5VTEw7 CisKKwlzZ19pbml0X3RhYmxlKHNnLCAxKTsKKworCWFkZHIgPSBwY2lfYWxsb2NfcDJwbWVtKHBk ZXYsIGxlbmd0aCk7CisJaWYgKCFhZGRyKQorCQlnb3RvIG91dF9mcmVlX3NnOworCisJc2dfc2V0 X2J1ZihzZywgYWRkciwgbGVuZ3RoKTsKKwkqbmVudHMgPSAxOworCXJldHVybiBzZzsKKworb3V0 X2ZyZWVfc2c6CisJa2ZyZWUoc2cpOworCXJldHVybiBOVUxMOworfQorRVhQT1JUX1NZTUJPTF9H UEwocGNpX3AycG1lbV9hbGxvY19zZ2wpOworCisvKioKKyAqIHBjaV9wMnBtZW1fZnJlZV9zZ2wg LSBmcmVlIGEgc2NhdHRlcmxpc3QgYWxsb2NhdGVkIGJ5IHBjaV9wMnBtZW1fYWxsb2Nfc2dsKCkK KyAqIEBwZGV2OiB0aGUgZGV2aWNlIHRvIGFsbG9jYXRlIG1lbW9yeSBmcm9tCisgKiBAc2dsOiB0 aGUgYWxsb2NhdGVkIHNjYXR0ZXJsaXN0CisgKiBAbmVudHM6IHRoZSBudW1iZXIgb2YgU0cgZW50 cmllcyBpbiB0aGUgbGlzdAorICovCit2b2lkIHBjaV9wMnBtZW1fZnJlZV9zZ2woc3RydWN0IHBj aV9kZXYgKnBkZXYsIHN0cnVjdCBzY2F0dGVybGlzdCAqc2dsKQoreworCXN0cnVjdCBzY2F0dGVy bGlzdCAqc2c7CisJaW50IGNvdW50OworCisJZm9yX2VhY2hfc2coc2dsLCBzZywgSU5UX01BWCwg Y291bnQpIHsKKwkJaWYgKCFzZykKKwkJCWJyZWFrOworCisJCXBjaV9mcmVlX3AycG1lbShwZGV2 LCBzZ192aXJ0KHNnKSwgc2ctPmxlbmd0aCk7CisJfQorCWtmcmVlKHNnbCk7Cit9CitFWFBPUlRf U1lNQk9MX0dQTChwY2lfcDJwbWVtX2ZyZWVfc2dsKTsKKworLyoqCisgKiBwY2lfcDJwbWVtX3B1 Ymxpc2ggLSBwdWJsaXNoIHRoZSBwZWVyLXRvLXBlZXIgRE1BIG1lbW9yeSBmb3IgdXNlIGJ5Cisg KglvdGhlciBkZXZpY2VzIHdpdGggcGNpX3AycG1lbV9maW5kKCkKKyAqIEBwZGV2OiB0aGUgZGV2 aWNlIHdpdGggcGVlci10by1wZWVyIERNQSBtZW1vcnkgdG8gcHVibGlzaAorICogQHB1Ymxpc2g6 IHNldCB0byB0cnVlIHRvIHB1Ymxpc2ggdGhlIG1lbW9yeSwgZmFsc2UgdG8gdW5wdWJsaXNoIGl0 CisgKgorICogUHVibGlzaGVkIG1lbW9yeSBjYW4gYmUgdXNlZCBieSBvdGhlciBQQ0kgZGV2aWNl IGRyaXZlcnMgZm9yCisgKiBwZWVyLTItcGVlciBETUEgb3BlcmF0aW9ucy4gTm9uLXB1Ymxpc2hl ZCBtZW1vcnkgaXMgcmVzZXJ2ZWQgZm9yCisgKiBleGx1c2l2ZSB1c2Ugb2YgdGhlIGRldmljZSBk cml2ZXIgdGhhdCByZWdpc3RlcnMgdGhlIHBlZXItdG8tcGVlcgorICogbWVtb3J5LgorICovCit2 b2lkIHBjaV9wMnBtZW1fcHVibGlzaChzdHJ1Y3QgcGNpX2RldiAqcGRldiwgYm9vbCBwdWJsaXNo KQoreworCWlmIChwdWJsaXNoICYmICFwZGV2LT5wMnBkbWEpCisJCXJldHVybjsKKworCXBkZXYt PnAycGRtYS0+cDJwbWVtX3B1Ymxpc2hlZCA9IHB1Ymxpc2g7Cit9CitFWFBPUlRfU1lNQk9MX0dQ TChwY2lfcDJwbWVtX3B1Ymxpc2gpOwpkaWZmIC0tZ2l0IGEvaW5jbHVkZS9saW51eC9tZW1yZW1h cC5oIGIvaW5jbHVkZS9saW51eC9tZW1yZW1hcC5oCmluZGV4IGY5MWY5ZTc2MzU1Ny4uOTU1MzM3 MGViZGFkIDEwMDY0NAotLS0gYS9pbmNsdWRlL2xpbnV4L21lbXJlbWFwLmgKKysrIGIvaW5jbHVk ZS9saW51eC9tZW1yZW1hcC5oCkBAIC01MywxMSArNTMsMTYgQEAgc3RydWN0IHZtZW1fYWx0bWFw IHsKICAqIHdha2V1cCBldmVudCB3aGVuZXZlciBhIHBhZ2UgaXMgdW5waW5uZWQgYW5kIGJlY29t ZXMgaWRsZS4gVGhpcwogICogd2FrZXVwIGlzIHVzZWQgdG8gY29vcmRpbmF0ZSBwaHlzaWNhbCBh ZGRyZXNzIHNwYWNlIG1hbmFnZW1lbnQgKGV4OgogICogZnMgdHJ1bmNhdGUvaG9sZSBwdW5jaCkg dnMgcGlubmVkIHBhZ2VzIChleDogZGV2aWNlIGRtYSkuCisgKgorICogTUVNT1JZX0RFVklDRV9Q Q0lfUDJQRE1BOgorICogRGV2aWNlIG1lbW9yeSByZXNpZGluZyBpbiBhIFBDSSBCQVIgaW50ZW5k ZWQgZm9yIHVzZSB3aXRoIFBlZXItdG8tUGVlcgorICogdHJhbnNhY3Rpb25zLgogICovCiBlbnVt IG1lbW9yeV90eXBlIHsKIAlNRU1PUllfREVWSUNFX1BSSVZBVEUgPSAxLAogCU1FTU9SWV9ERVZJ Q0VfUFVCTElDLAogCU1FTU9SWV9ERVZJQ0VfRlNfREFYLAorCU1FTU9SWV9ERVZJQ0VfUENJX1Ay UERNQSwKIH07CiAKIC8qCmRpZmYgLS1naXQgYS9pbmNsdWRlL2xpbnV4L21tLmggYi9pbmNsdWRl L2xpbnV4L21tLmgKaW5kZXggYTYxZWJlOGFkNGNhLi4yMDU1ZGY0MTJhNzcgMTAwNjQ0Ci0tLSBh L2luY2x1ZGUvbGludXgvbW0uaAorKysgYi9pbmNsdWRlL2xpbnV4L21tLmgKQEAgLTg5MCw2ICs4 OTAsMTkgQEAgc3RhdGljIGlubGluZSBib29sIGlzX2RldmljZV9wdWJsaWNfcGFnZShjb25zdCBz dHJ1Y3QgcGFnZSAqcGFnZSkKIAkJcGFnZS0+cGdtYXAtPnR5cGUgPT0gTUVNT1JZX0RFVklDRV9Q VUJMSUM7CiB9CiAKKyNpZmRlZiBDT05GSUdfUENJX1AyUERNQQorc3RhdGljIGlubGluZSBib29s IGlzX3BjaV9wMnBkbWFfcGFnZShjb25zdCBzdHJ1Y3QgcGFnZSAqcGFnZSkKK3sKKwlyZXR1cm4g aXNfem9uZV9kZXZpY2VfcGFnZShwYWdlKSAmJgorCQlwYWdlLT5wZ21hcC0+dHlwZSA9PSBNRU1P UllfREVWSUNFX1BDSV9QMlBETUE7Cit9CisjZWxzZSAvKiBDT05GSUdfUENJX1AyUERNQSAqLwor c3RhdGljIGlubGluZSBib29sIGlzX3BjaV9wMnBkbWFfcGFnZShjb25zdCBzdHJ1Y3QgcGFnZSAq cGFnZSkKK3sKKwlyZXR1cm4gZmFsc2U7Cit9CisjZW5kaWYgLyogQ09ORklHX1BDSV9QMlBETUEg Ki8KKwogI2Vsc2UgLyogQ09ORklHX0RFVl9QQUdFTUFQX09QUyAqLwogc3RhdGljIGlubGluZSB2 b2lkIGRldl9wYWdlbWFwX2dldF9vcHModm9pZCkKIHsKQEAgLTkxMyw2ICs5MjYsMTEgQEAgc3Rh dGljIGlubGluZSBib29sIGlzX2RldmljZV9wdWJsaWNfcGFnZShjb25zdCBzdHJ1Y3QgcGFnZSAq cGFnZSkKIHsKIAlyZXR1cm4gZmFsc2U7CiB9CisKK3N0YXRpYyBpbmxpbmUgYm9vbCBpc19wY2lf cDJwZG1hX3BhZ2UoY29uc3Qgc3RydWN0IHBhZ2UgKnBhZ2UpCit7CisJcmV0dXJuIGZhbHNlOwor fQogI2VuZGlmIC8qIENPTkZJR19ERVZfUEFHRU1BUF9PUFMgKi8KIAogc3RhdGljIGlubGluZSB2 b2lkIGdldF9wYWdlKHN0cnVjdCBwYWdlICpwYWdlKQpkaWZmIC0tZ2l0IGEvaW5jbHVkZS9saW51 eC9wY2ktcDJwZG1hLmggYi9pbmNsdWRlL2xpbnV4L3BjaS1wMnBkbWEuaApuZXcgZmlsZSBtb2Rl IDEwMDY0NAppbmRleCAwMDAwMDAwMDAwMDAuLjdiMmIwZjU0NzUyOAotLS0gL2Rldi9udWxsCisr KyBiL2luY2x1ZGUvbGludXgvcGNpLXAycGRtYS5oCkBAIC0wLDAgKzEsMTAyIEBACisvKiBTUERY LUxpY2Vuc2UtSWRlbnRpZmllcjogR1BMLTIuMCAqLworLyoKKyAqIFBDSSBQZWVyIDIgUGVlciBE TUEgc3VwcG9ydC4KKyAqCisgKiBDb3B5cmlnaHQgKGMpIDIwMTYtMjAxOCwgTG9nYW4gR3VudGhv cnBlCisgKiBDb3B5cmlnaHQgKGMpIDIwMTYtMjAxNywgTWljcm9zZW1pIENvcnBvcmF0aW9uCisg KiBDb3B5cmlnaHQgKGMpIDIwMTcsIENocmlzdG9waCBIZWxsd2lnCisgKiBDb3B5cmlnaHQgKGMp IDIwMTgsIEVpZGV0aWNvbSBJbmMuCisgKgorICovCisKKyNpZm5kZWYgX0xJTlVYX1BDSV9QMlBE TUFfSAorI2RlZmluZSBfTElOVVhfUENJX1AyUERNQV9ICisKKyNpbmNsdWRlIDxsaW51eC9wY2ku aD4KKworc3RydWN0IGJsb2NrX2RldmljZTsKK3N0cnVjdCBzY2F0dGVybGlzdDsKKworI2lmZGVm IENPTkZJR19QQ0lfUDJQRE1BCitpbnQgcGNpX3AycGRtYV9hZGRfcmVzb3VyY2Uoc3RydWN0IHBj aV9kZXYgKnBkZXYsIGludCBiYXIsIHNpemVfdCBzaXplLAorCQl1NjQgb2Zmc2V0KTsKK2ludCBw Y2lfcDJwZG1hX2FkZF9jbGllbnQoc3RydWN0IGxpc3RfaGVhZCAqaGVhZCwgc3RydWN0IGRldmlj ZSAqZGV2KTsKK3ZvaWQgcGNpX3AycGRtYV9yZW1vdmVfY2xpZW50KHN0cnVjdCBsaXN0X2hlYWQg KmhlYWQsIHN0cnVjdCBkZXZpY2UgKmRldik7Cit2b2lkIHBjaV9wMnBkbWFfY2xpZW50X2xpc3Rf ZnJlZShzdHJ1Y3QgbGlzdF9oZWFkICpoZWFkKTsKK2ludCBwY2lfcDJwZG1hX2Rpc3RhbmNlKHN0 cnVjdCBwY2lfZGV2ICpwcm92aWRlciwgc3RydWN0IGxpc3RfaGVhZCAqY2xpZW50cywKKwkJCWJv b2wgdmVyYm9zZSk7Citib29sIHBjaV9wMnBkbWFfYXNzaWduX3Byb3ZpZGVyKHN0cnVjdCBwY2lf ZGV2ICpwcm92aWRlciwKKwkJCQlzdHJ1Y3QgbGlzdF9oZWFkICpjbGllbnRzKTsKK2Jvb2wgcGNp X2hhc19wMnBtZW0oc3RydWN0IHBjaV9kZXYgKnBkZXYpOworc3RydWN0IHBjaV9kZXYgKnBjaV9w MnBtZW1fZmluZChzdHJ1Y3QgbGlzdF9oZWFkICpjbGllbnRzKTsKK3ZvaWQgKnBjaV9hbGxvY19w MnBtZW0oc3RydWN0IHBjaV9kZXYgKnBkZXYsIHNpemVfdCBzaXplKTsKK3ZvaWQgcGNpX2ZyZWVf cDJwbWVtKHN0cnVjdCBwY2lfZGV2ICpwZGV2LCB2b2lkICphZGRyLCBzaXplX3Qgc2l6ZSk7Citw Y2lfYnVzX2FkZHJfdCBwY2lfcDJwbWVtX3ZpcnRfdG9fYnVzKHN0cnVjdCBwY2lfZGV2ICpwZGV2 LCB2b2lkICphZGRyKTsKK3N0cnVjdCBzY2F0dGVybGlzdCAqcGNpX3AycG1lbV9hbGxvY19zZ2wo c3RydWN0IHBjaV9kZXYgKnBkZXYsCisJCQkJCSB1bnNpZ25lZCBpbnQgKm5lbnRzLCB1MzIgbGVu Z3RoKTsKK3ZvaWQgcGNpX3AycG1lbV9mcmVlX3NnbChzdHJ1Y3QgcGNpX2RldiAqcGRldiwgc3Ry dWN0IHNjYXR0ZXJsaXN0ICpzZ2wpOwordm9pZCBwY2lfcDJwbWVtX3B1Ymxpc2goc3RydWN0IHBj aV9kZXYgKnBkZXYsIGJvb2wgcHVibGlzaCk7CisjZWxzZSAvKiBDT05GSUdfUENJX1AyUERNQSAq Lworc3RhdGljIGlubGluZSBpbnQgcGNpX3AycGRtYV9hZGRfcmVzb3VyY2Uoc3RydWN0IHBjaV9k ZXYgKnBkZXYsIGludCBiYXIsCisJCXNpemVfdCBzaXplLCB1NjQgb2Zmc2V0KQoreworCXJldHVy biAtRU9QTk9UU1VQUDsKK30KK3N0YXRpYyBpbmxpbmUgaW50IHBjaV9wMnBkbWFfYWRkX2NsaWVu dChzdHJ1Y3QgbGlzdF9oZWFkICpoZWFkLAorCQlzdHJ1Y3QgZGV2aWNlICpkZXYpCit7CisJcmV0 dXJuIDA7Cit9CitzdGF0aWMgaW5saW5lIHZvaWQgcGNpX3AycGRtYV9yZW1vdmVfY2xpZW50KHN0 cnVjdCBsaXN0X2hlYWQgKmhlYWQsCisJCXN0cnVjdCBkZXZpY2UgKmRldikKK3sKK30KK3N0YXRp YyBpbmxpbmUgdm9pZCBwY2lfcDJwZG1hX2NsaWVudF9saXN0X2ZyZWUoc3RydWN0IGxpc3RfaGVh ZCAqaGVhZCkKK3sKK30KK3N0YXRpYyBpbmxpbmUgaW50IHBjaV9wMnBkbWFfZGlzdGFuY2Uoc3Ry dWN0IHBjaV9kZXYgKnByb3ZpZGVyLAorCQkJCSAgICAgIHN0cnVjdCBsaXN0X2hlYWQgKmNsaWVu dHMsCisJCQkJICAgICAgYm9vbCB2ZXJib3NlKQoreworCXJldHVybiAtMTsKK30KK3N0YXRpYyBp bmxpbmUgYm9vbCBwY2lfcDJwZG1hX2Fzc2lnbl9wcm92aWRlcihzdHJ1Y3QgcGNpX2RldiAqcHJv dmlkZXIsCisJCQkJCSAgICAgIHN0cnVjdCBsaXN0X2hlYWQgKmNsaWVudHMpCit7CisJcmV0dXJu IGZhbHNlOworfQorc3RhdGljIGlubGluZSBib29sIHBjaV9oYXNfcDJwbWVtKHN0cnVjdCBwY2lf ZGV2ICpwZGV2KQoreworCXJldHVybiBmYWxzZTsKK30KK3N0YXRpYyBpbmxpbmUgc3RydWN0IHBj aV9kZXYgKnBjaV9wMnBtZW1fZmluZChzdHJ1Y3QgbGlzdF9oZWFkICpjbGllbnRzKQoreworCXJl dHVybiBOVUxMOworfQorc3RhdGljIGlubGluZSB2b2lkICpwY2lfYWxsb2NfcDJwbWVtKHN0cnVj dCBwY2lfZGV2ICpwZGV2LCBzaXplX3Qgc2l6ZSkKK3sKKwlyZXR1cm4gTlVMTDsKK30KK3N0YXRp YyBpbmxpbmUgdm9pZCBwY2lfZnJlZV9wMnBtZW0oc3RydWN0IHBjaV9kZXYgKnBkZXYsIHZvaWQg KmFkZHIsCisJCXNpemVfdCBzaXplKQoreworfQorc3RhdGljIGlubGluZSBwY2lfYnVzX2FkZHJf dCBwY2lfcDJwbWVtX3ZpcnRfdG9fYnVzKHN0cnVjdCBwY2lfZGV2ICpwZGV2LAorCQkJCQkJICAg IHZvaWQgKmFkZHIpCit7CisJcmV0dXJuIDA7Cit9CitzdGF0aWMgaW5saW5lIHN0cnVjdCBzY2F0 dGVybGlzdCAqcGNpX3AycG1lbV9hbGxvY19zZ2woc3RydWN0IHBjaV9kZXYgKnBkZXYsCisJCXVu c2lnbmVkIGludCAqbmVudHMsIHUzMiBsZW5ndGgpCit7CisJcmV0dXJuIE5VTEw7Cit9CitzdGF0 aWMgaW5saW5lIHZvaWQgcGNpX3AycG1lbV9mcmVlX3NnbChzdHJ1Y3QgcGNpX2RldiAqcGRldiwK KwkJc3RydWN0IHNjYXR0ZXJsaXN0ICpzZ2wpCit7Cit9CitzdGF0aWMgaW5saW5lIHZvaWQgcGNp X3AycG1lbV9wdWJsaXNoKHN0cnVjdCBwY2lfZGV2ICpwZGV2LCBib29sIHB1Ymxpc2gpCit7Cit9 CisjZW5kaWYgLyogQ09ORklHX1BDSV9QMlBETUEgKi8KKyNlbmRpZiAvKiBfTElOVVhfUENJX1Ay UF9IICovCmRpZmYgLS1naXQgYS9pbmNsdWRlL2xpbnV4L3BjaS5oIGIvaW5jbHVkZS9saW51eC9w Y2kuaAppbmRleCBlNzJjYThkZDYyNDEuLjVkOTVkYmYyMWY0YSAxMDA2NDQKLS0tIGEvaW5jbHVk ZS9saW51eC9wY2kuaAorKysgYi9pbmNsdWRlL2xpbnV4L3BjaS5oCkBAIC0yODEsNiArMjgxLDcg QEAgc3RydWN0IHBjaWVfbGlua19zdGF0ZTsKIHN0cnVjdCBwY2lfdnBkOwogc3RydWN0IHBjaV9z cmlvdjsKIHN0cnVjdCBwY2lfYXRzOworc3RydWN0IHBjaV9wMnBkbWE7CiAKIC8qIFRoZSBwY2lf ZGV2IHN0cnVjdHVyZSBkZXNjcmliZXMgUENJIGRldmljZXMgKi8KIHN0cnVjdCBwY2lfZGV2IHsK QEAgLTQzOSw2ICs0NDAsOSBAQCBzdHJ1Y3QgcGNpX2RldiB7CiAjaWZkZWYgQ09ORklHX1BDSV9Q QVNJRAogCXUxNgkJcGFzaWRfZmVhdHVyZXM7CiAjZW5kaWYKKyNpZmRlZiBDT05GSUdfUENJX1Ay UERNQQorCXN0cnVjdCBwY2lfcDJwZG1hICpwMnBkbWE7CisjZW5kaWYKIAlwaHlzX2FkZHJfdAly b207CQkvKiBQaHlzaWNhbCBhZGRyZXNzIGlmIG5vdCBmcm9tIEJBUiAqLwogCXNpemVfdAkJcm9t bGVuOwkJLyogTGVuZ3RoIGlmIG5vdCBmcm9tIEJBUiAqLwogCWNoYXIJCSpkcml2ZXJfb3ZlcnJp ZGU7IC8qIERyaXZlciBuYW1lIHRvIGZvcmNlIGEgbWF0Y2ggKi8KLS0gCjIuMTEuMAoKX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KTGludXgtbnZkaW1tIG1h aWxpbmcgbGlzdApMaW51eC1udmRpbW1AbGlzdHMuMDEub3JnCmh0dHBzOi8vbGlzdHMuMDEub3Jn L21haWxtYW4vbGlzdGluZm8vbGludXgtbnZkaW1tCg==