* RE: [PATCH 03/22] NTB: Alter NTB API to support both inbound and outbound MW based interfaces
@ 2016-12-04 0:04 ` Allen Hubbe
0 siblings, 0 replies; 5+ messages in thread
From: Allen Hubbe @ 2016-12-04 0:04 UTC (permalink / raw)
To: 'Serge Semin', jdmason, dave.jiang, Xiangliang.Yu
Cc: Sergey.Semin, linux-ntb, linux-kernel
From: Serge Semin
> Alter NTB API to support inbound and outbound MW based interfaces.
> Additionally I made it supporting multi-port devices as well. Useful
> infographics is added right before MW API is declared. It shall help to
> better understand how the new API really works and how it can be utilized
> within client drivers.
>
This looks good. I plan to ack.
See comments below on documentation.
> Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
>
> ---
> include/linux/ntb.h | 290 ++++++++++++++++++++++++++++++++++++++++++++--------
> 1 file changed, 245 insertions(+), 45 deletions(-)
>
> diff --git a/include/linux/ntb.h b/include/linux/ntb.h
> index 0941a43..4a150b5 100644
> --- a/include/linux/ntb.h
> +++ b/include/linux/ntb.h
> @@ -171,9 +171,13 @@ static inline int ntb_ctx_ops_is_valid(const struct ntb_ctx_ops *ops)
> * @link_enable: See ntb_link_enable().
> * @link_disable: See ntb_link_disable().
> * @mw_count: See ntb_mw_count().
> - * @mw_get_range: See ntb_mw_get_range().
> + * @mw_get_align: See ntb_mw_get_align().
> * @mw_set_trans: See ntb_mw_set_trans().
> * @mw_clear_trans: See ntb_mw_clear_trans().
> + * @peer_mw_count: See ntb_peer_mw_count().
> + * @peer_mw_get_addr: See ntb_peer_mw_get_addr().
> + * @peer_mw_set_trans: See ntb_peer_mw_set_trans().
> + * @peer_mw_clear_trans:See ntb_peer_mw_clear_trans().
> * @db_is_unsafe: See ntb_db_is_unsafe().
> * @db_valid_mask: See ntb_db_valid_mask().
> * @db_vector_count: See ntb_db_vector_count().
> @@ -211,13 +215,20 @@ struct ntb_dev_ops {
> enum ntb_speed max_speed, enum ntb_width max_width);
> int (*link_disable)(struct ntb_dev *ntb);
>
> - int (*mw_count)(struct ntb_dev *ntb);
> - int (*mw_get_range)(struct ntb_dev *ntb, int idx,
> - phys_addr_t *base, resource_size_t *size,
> - resource_size_t *align, resource_size_t *align_size);
> - int (*mw_set_trans)(struct ntb_dev *ntb, int idx,
> + int (*mw_count)(struct ntb_dev *ntb, int pidx);
> + int (*mw_get_align)(struct ntb_dev *ntb, int pidx, int widx,
> + resource_size_t *addr_align,
> + resource_size_t *size_align,
> + resource_size_t *size_max);
> + int (*mw_set_trans)(struct ntb_dev *ntb, int pidx, int widx,
> dma_addr_t addr, resource_size_t size);
> - int (*mw_clear_trans)(struct ntb_dev *ntb, int idx);
> + int (*mw_clear_trans)(struct ntb_dev *ntb, int pidx, int widx);
> + int (*peer_mw_count)(struct ntb_dev *ntb);
> + int (*peer_mw_get_addr)(struct ntb_dev *ntb, int widx,
> + phys_addr_t *base, resource_size_t *size);
> + int (*peer_mw_set_trans)(struct ntb_dev *ntb, int pidx, int widx,
> + u64 addr, resource_size_t size);
> + int (*peer_mw_clear_trans)(struct ntb_dev *ntb, int pidx, int widx);
>
> int (*db_is_unsafe)(struct ntb_dev *ntb);
> u64 (*db_valid_mask)(struct ntb_dev *ntb);
> @@ -266,9 +277,13 @@ static inline int ntb_dev_ops_is_valid(const struct ntb_dev_ops *ops)
> ops->link_enable &&
> ops->link_disable &&
> ops->mw_count &&
> - ops->mw_get_range &&
> - ops->mw_set_trans &&
> + ops->mw_get_align &&
> + (ops->mw_set_trans ||
> + ops->peer_mw_set_trans) &&
> /* ops->mw_clear_trans && */
> + ops->peer_mw_count &&
> + ops->peer_mw_get_addr &&
> + /* ops->peer_mw_clear_trans && */
>
> /* ops->db_is_unsafe && */
> ops->db_valid_mask &&
> @@ -555,79 +570,264 @@ static inline int ntb_link_disable(struct ntb_dev *ntb)
> }
>
> /**
> - * ntb_mw_count() - get the number of memory windows
> + * NTB Memory Windows description
The two variants could be more succintly described as "inbound translation configured on the local ntb port" and "outbound translation configured by the peer, on the peer ntb port" for a locally allocated dma-mapped range of memory.
Please avoid confusing these concepts in the documentation:
- "Memory" on the system vs. "Memory Window" on the ntb
- "Physical" address vs. "dma-mapped" address of memory
- "Base Address" vs. "Translation Address"
Inbound translation:
Memory: Local NTB Port: Peer NTB Port: Peer MMIO:
____________
| dma-mapped |-ntb_set_xlat_addr(addr) |
| memory | _v____________ | ______________
| (addr) |<======| MW xlat addr |<====| MW base addr |<==== memory-mapped IO
|------------| |--------------| | |--------------|
Outbound translation:
Memory: Local NTB Port: Peer NTB Port: Peer MMIO:
____________ ______________
| dma-mapped | | | MW base addr |<==== memory-mapped IO
| memory | | |--------------|
| (addr) |<===========================| MW xlat addr |<-ntb_peer_set_xlat_addr(addr)
|------------| | |--------------|
> + * There are two types of memory window interfaces supported by the NTB API:
> + * local and peer side initialization of memory sharing. The first type is
> + * depicted on the next figure:
> + *
> + * Local device: | Peer device:
> + * NTB config |
> + * Physical memory (RAM) __________ | Memory mapped IO
> + * ____________ +-->| addr | | _____________
> + * | | | |----------| | | |
> + * |------------|addr--+ | |-------------|
> + * | Inbound MW | PCI Express + NTB | Outbound MW |
> + * | |<=====================================| |
> + * |------------| |-------------|
> + *
> + * So typical scenario of the first type memory window initialization looks:
> + * 1) allocate a memory region, 2) put translated base address to NTB config,
> + * 3) somehow notify a peer device of performed initialization, 4) peer device
> + * maps corresponding outbound memory window so to have access to the shared
> + * memory region.
> + *
> + * The second type of interface, that implies the shared windows being
> + * initialized by a peer device, is depicted on the figure:
> + *
> + * Local device: | Peer device:
> + * | NTB config
> + * Physical memory (RAM) | __________ Memory mapped IO
> + * ____________ +-------------->| addr | _____________
> + * | | | | |----------| | |
> + * |------------|addr---+ | |-------------|
> + * | Inbound MW | PCI Express + NTB | Outbound MW |
> + * | |<=====================================| |
> + * |------------| |-------------|
> + *
> + * Typical scenario of the second type initialization would be:
> + * 1) allocate a memory region, 2) somehow deliver a translated base address
> + * to a peer device, 3) peer puts the translated base address to NTB config,
> + * 4) peer device maps outbound memory window so to have access to the shared
> + * memory region.
> + *
> + * As one can see the described scenarios can be combined in one portable
> + * algorithm.
> + * Local device:
> + * 1) Allocate memory for a shared window
> + * 2) Initialize memory window by base address of the allocated region
> + * (it may fail if local memory window initialzation is unsupported)
> + * 3) Send translated base address and memory window index to a peer device
> + * Peer device:
> + * 1) Initialize memory window by retrieved base address of the allocated
> + * by another device memory region (it may fail if peer memory window
> + * initialization is unsupported)
> + * 2) Map outbound memory window
> + * 3) Done
> + * In accordance with this scenario, the NTB Memory Window API can be used as
> + * follows:
> + * Local device:
> + * 1) ntb_mw_count(pidx) - retrieve number of memory ranges, which can
> + * be allocated for memory windows between local device and peer device
> + * of port with specified index.
> + * 2) ntb_get_align(pidx, midx) - retrieve parameters restricting the
> + * shared memory region alignment and size. Then memory can be properly
> + * allocated.
> + * 3) Allocate physically contiguous memory region in complience with
> + * restrictions retrieved in 2).
> + * 4) ntb_mw_set_trans(pidx, midx) - try to set translation address of
> + * the memory window with specified index for the defined peer device
> + * (it may fail if local translated address setting is not supported)
> + * 5) Send translated base address (usually together with memory window
> + * number) to the peer device using, for instance, scratchpad or message
> + * registers.
> + * Peer device:
> + * 1) ntb_peer_mw_set_trans(pidx, midx) - try to set received from other
> + * device (related to pidx) translated base address for specified memory
> + * window. It may fail if retrieved address, for instance, exceeds
> + * maximum possible address or isn't properly aligned.
> + * 2) ntb_peer_mw_get_addr(widx) - retrieve MMIO address to map the memory
> + * window so to have an access to the shared memory.
The above section belongs in Documentation/ntb.txt. Thanks for describing how to use the api so that portable applications can work with either variant of memory window configuration.
> + *
> + * Also it is worth to note, that method ntb_mw_count(pidx) should return the
> + * same value as ntb_peer_mw_count() of the peer with port index - pidx.
> + */
> +
> +/**
> + * ntb_mw_count() - get the number of inbound memory windows, which could
> + * be created for a specified peer device
> * @ntb: NTB device context.
> + * @pidx: Port index of peer device.
> *
> * Hardware and topology may support a different number of memory windows.
> + * Moreover different peer devices can support different number of memory
> + * windows. Simply speaking this method returns the number of possible inbound
> + * memory windows to share with specified peer device.
> *
> * Return: the number of memory windows.
> */
> -static inline int ntb_mw_count(struct ntb_dev *ntb)
> +static inline int ntb_mw_count(struct ntb_dev *ntb, int pidx)
> {
> - return ntb->ops->mw_count(ntb);
> + return ntb->ops->mw_count(ntb, pidx);
> }
>
> /**
> - * ntb_mw_get_range() - get the range of a memory window
> + * ntb_mw_get_align() - get the restriction parameters of inbound memory window
> * @ntb: NTB device context.
> - * @idx: Memory window number.
> - * @base: OUT - the base address for mapping the memory window
> - * @size: OUT - the size for mapping the memory window
> - * @align: OUT - the base alignment for translating the memory window
> - * @align_size: OUT - the size alignment for translating the memory window
> - *
> - * Get the range of a memory window. NULL may be given for any output
> - * parameter if the value is not needed. The base and size may be used for
> - * mapping the memory window, to access the peer memory. The alignment and
> - * size may be used for translating the memory window, for the peer to access
> - * memory on the local system.
> - *
> - * Return: Zero on success, otherwise an error number.
> + * @pidx: Port index of peer device.
> + * @widx: Memory window index.
> + * @addr_align: OUT - the base alignment for translating the memory window
> + * @size_align: OUT - the size alignment for translating the memory window
> + * @size_max: OUT - the maximum size of the memory window
> + *
> + * Get the alignments of an inbound memory window with specified index.
> + * NULL may be given for any output parameter if the value is not needed.
> + * The alignment and size parameters may be used for allocation of proper
> + * shared memory.
> + *
> + * Return: Zero on success, otherwise a negative error number.
> */
> -static inline int ntb_mw_get_range(struct ntb_dev *ntb, int idx,
> - phys_addr_t *base, resource_size_t *size,
> - resource_size_t *align, resource_size_t *align_size)
> +static inline int ntb_mw_get_align(struct ntb_dev *ntb, int pidx, int widx,
> + resource_size_t *addr_align,
> + resource_size_t *size_align,
> + resource_size_t *size_max)
> {
> - return ntb->ops->mw_get_range(ntb, idx, base, size,
> - align, align_size);
> + return ntb->ops->mw_get_align(ntb, pidx, widx, addr_align, size_align,
> + size_max);
> }
>
> /**
> - * ntb_mw_set_trans() - set the translation of a memory window
> + * ntb_mw_set_trans() - set the translation of an inbound memory window
> * @ntb: NTB device context.
> - * @idx: Memory window number.
> - * @addr: The dma address local memory to expose to the peer.
> + * @pidx: Port index of peer device.
> + * @widx: Memory window index.
> + * @addr: The dma address of local memory to expose to the peer.
> * @size: The size of the local memory to expose to the peer.
> *
> * Set the translation of a memory window. The peer may access local memory
> * through the window starting at the address, up to the size. The address
> - * must be aligned to the alignment specified by ntb_mw_get_range(). The size
> - * must be aligned to the size alignment specified by ntb_mw_get_range().
> + * and size must be aligned in complience with restrictions of
> + * ntb_mw_get_align(). The region size should not exceed the size_max parameter
> + * of that method.
> + *
> + * This method may not be implemented due to the hardware specific memory
> + * windows interface.
> *
> * Return: Zero on success, otherwise an error number.
> */
> -static inline int ntb_mw_set_trans(struct ntb_dev *ntb, int idx,
> +static inline int ntb_mw_set_trans(struct ntb_dev *ntb, int pidx, int widx,
> dma_addr_t addr, resource_size_t size)
> {
> - return ntb->ops->mw_set_trans(ntb, idx, addr, size);
> + if (!ntb->ops->mw_set_trans)
> + return -EINVAL;
> +
> + return ntb->ops->mw_set_trans(ntb, pidx, widx, addr, size);
> }
>
> /**
> - * ntb_mw_clear_trans() - clear the translation of a memory window
> + * ntb_mw_clear_trans() - clear the translation address of an inbound memory
> + * window
> * @ntb: NTB device context.
> - * @idx: Memory window number.
> + * @pidx: Port index of peer device.
> + * @widx: Memory window index.
> *
> - * Clear the translation of a memory window. The peer may no longer access
> - * local memory through the window.
> + * Clear the translation of an inbound memory window. The peer may no longer
> + * access local memory through the window.
> *
> * Return: Zero on success, otherwise an error number.
> */
> -static inline int ntb_mw_clear_trans(struct ntb_dev *ntb, int idx)
> +static inline int ntb_mw_clear_trans(struct ntb_dev *ntb, int pidx, int widx)
> {
> if (!ntb->ops->mw_clear_trans)
> - return ntb->ops->mw_set_trans(ntb, idx, 0, 0);
> + return ntb_mw_set_trans(ntb, pidx, widx, 0, 0);
> +
> + return ntb->ops->mw_clear_trans(ntb, pidx, widx);
> +}
> +
> +/**
> + * ntb_peer_mw_count() - get the number of outbound memory windows, which could
> + * be mapped to access a shared memory
> + * @ntb: NTB device context.
> + *
> + * Hardware and topology may support a different number of memory windows.
> + * This method returns the number of outbound memory windows supported by
> + * local device.
> + *
> + * Return: the number of memory windows.
> + */
> +static inline int ntb_peer_mw_count(struct ntb_dev *ntb)
> +{
> + return ntb->ops->peer_mw_count(ntb);
> +}
> +
> +/**
> + * ntb_peer_mw_get_addr() - get map address of an outbound memory window
> + * @ntb: NTB device context.
> + * @widx: Memory window index (within ntb_peer_mw_count() return value).
> + * @base: OUT - the base address of mapping region.
> + * @size: OUT - the size of mapping region.
> + *
> + * Get base and size of memory region to map. NULL may be given for any output
> + * parameter if the value is not needed. The base and size may be used for
> + * mapping the memory window, to access the peer memory.
> + *
> + * Return: Zero on success, otherwise a negative error number.
> + */
> +static inline int ntb_peer_mw_get_addr(struct ntb_dev *ntb, int widx,
> + phys_addr_t *base, resource_size_t *size)
> +{
> + return ntb->ops->peer_mw_get_addr(ntb, widx, base, size);
> +}
> +
> +/**
> + * ntb_peer_mw_set_trans() - set a translation address of a memory window
> + * retrieved from a peer device
> + * @ntb: NTB device context.
> + * @pidx: Port index of peer device the translation address received from.
> + * @widx: Memory window index.
> + * @addr: The dma address of the shared memory to access.
> + * @size: The size of the shared memory to access.
> + *
> + * Set the translation of an outbound memory window. The local device may
> + * access shared memory allocated by a peer device sent the address.
> + *
> + * This method may not be implemented due to the hardware specific memory
> + * windows interface, so a translation address can be only set on the side,
> + * where shared memory (inbound memory windows) is allocated.
> + *
> + * Return: Zero on success, otherwise an error number.
> + */
> +static inline int ntb_peer_mw_set_trans(struct ntb_dev *ntb, int pidx, int widx,
> + u64 addr, resource_size_t size)
> +{
> + if (!ntb->ops->peer_mw_set_trans)
> + return -EINVAL;
> +
> + return ntb->ops->peer_mw_set_trans(ntb, pidx, widx, addr, size);
> +}
> +
> +/**
> + * ntb_peer_mw_clear_trans() - clear the translation address of an outbound
> + * memory window
> + * @ntb: NTB device context.
> + * @pidx: Port index of peer device.
> + * @widx: Memory window index.
> + *
> + * Clear the translation of a outbound memory window. The local device may no
> + * longer access a shared memory through the window.
> + *
> + * This method may not be implemented due to the hardware specific memory
> + * windows interface.
> + *
> + * Return: Zero on success, otherwise an error number.
> + */
> +static inline int ntb_peer_mw_clear_trans(struct ntb_dev *ntb, int pidx,
> + int widx)
> +{
> + if (!ntb->ops->peer_mw_clear_trans)
> + return ntb_peer_mw_set_trans(ntb, pidx, widx, 0, 0);
>
> - return ntb->ops->mw_clear_trans(ntb, idx);
> + return ntb->ops->peer_mw_clear_trans(ntb, pidx, widx);
> }
>
> /**
> --
> 2.6.6
^ permalink raw reply [flat|nested] 5+ messages in thread* RE: [PATCH 03/22] NTB: Alter NTB API to support both inbound and outbound MW based interfaces
@ 2016-12-04 0:04 ` Allen Hubbe
0 siblings, 0 replies; 5+ messages in thread
From: Allen Hubbe @ 2016-12-04 0:04 UTC (permalink / raw)
To: 'Serge Semin', jdmason, dave.jiang, Xiangliang.Yu
Cc: Sergey.Semin, linux-ntb, linux-kernel
From: Serge Semin
> Alter NTB API to support inbound and outbound MW based interfaces.
> Additionally I made it supporting multi-port devices as well. Useful
> infographics is added right before MW API is declared. It shall help to
> better understand how the new API really works and how it can be utilized
> within client drivers.
>
This looks good. I plan to ack.
See comments below on documentation.
> Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
>
> ---
> include/linux/ntb.h | 290 ++++++++++++++++++++++++++++++++++++++++++++--------
> 1 file changed, 245 insertions(+), 45 deletions(-)
>
> diff --git a/include/linux/ntb.h b/include/linux/ntb.h
> index 0941a43..4a150b5 100644
> --- a/include/linux/ntb.h
> +++ b/include/linux/ntb.h
> @@ -171,9 +171,13 @@ static inline int ntb_ctx_ops_is_valid(const struct ntb_ctx_ops *ops)
> * @link_enable: See ntb_link_enable().
> * @link_disable: See ntb_link_disable().
> * @mw_count: See ntb_mw_count().
> - * @mw_get_range: See ntb_mw_get_range().
> + * @mw_get_align: See ntb_mw_get_align().
> * @mw_set_trans: See ntb_mw_set_trans().
> * @mw_clear_trans: See ntb_mw_clear_trans().
> + * @peer_mw_count: See ntb_peer_mw_count().
> + * @peer_mw_get_addr: See ntb_peer_mw_get_addr().
> + * @peer_mw_set_trans: See ntb_peer_mw_set_trans().
> + * @peer_mw_clear_trans:See ntb_peer_mw_clear_trans().
> * @db_is_unsafe: See ntb_db_is_unsafe().
> * @db_valid_mask: See ntb_db_valid_mask().
> * @db_vector_count: See ntb_db_vector_count().
> @@ -211,13 +215,20 @@ struct ntb_dev_ops {
> enum ntb_speed max_speed, enum ntb_width max_width);
> int (*link_disable)(struct ntb_dev *ntb);
>
> - int (*mw_count)(struct ntb_dev *ntb);
> - int (*mw_get_range)(struct ntb_dev *ntb, int idx,
> - phys_addr_t *base, resource_size_t *size,
> - resource_size_t *align, resource_size_t *align_size);
> - int (*mw_set_trans)(struct ntb_dev *ntb, int idx,
> + int (*mw_count)(struct ntb_dev *ntb, int pidx);
> + int (*mw_get_align)(struct ntb_dev *ntb, int pidx, int widx,
> + resource_size_t *addr_align,
> + resource_size_t *size_align,
> + resource_size_t *size_max);
> + int (*mw_set_trans)(struct ntb_dev *ntb, int pidx, int widx,
> dma_addr_t addr, resource_size_t size);
> - int (*mw_clear_trans)(struct ntb_dev *ntb, int idx);
> + int (*mw_clear_trans)(struct ntb_dev *ntb, int pidx, int widx);
> + int (*peer_mw_count)(struct ntb_dev *ntb);
> + int (*peer_mw_get_addr)(struct ntb_dev *ntb, int widx,
> + phys_addr_t *base, resource_size_t *size);
> + int (*peer_mw_set_trans)(struct ntb_dev *ntb, int pidx, int widx,
> + u64 addr, resource_size_t size);
> + int (*peer_mw_clear_trans)(struct ntb_dev *ntb, int pidx, int widx);
>
> int (*db_is_unsafe)(struct ntb_dev *ntb);
> u64 (*db_valid_mask)(struct ntb_dev *ntb);
> @@ -266,9 +277,13 @@ static inline int ntb_dev_ops_is_valid(const struct ntb_dev_ops *ops)
> ops->link_enable &&
> ops->link_disable &&
> ops->mw_count &&
> - ops->mw_get_range &&
> - ops->mw_set_trans &&
> + ops->mw_get_align &&
> + (ops->mw_set_trans ||
> + ops->peer_mw_set_trans) &&
> /* ops->mw_clear_trans && */
> + ops->peer_mw_count &&
> + ops->peer_mw_get_addr &&
> + /* ops->peer_mw_clear_trans && */
>
> /* ops->db_is_unsafe && */
> ops->db_valid_mask &&
> @@ -555,79 +570,264 @@ static inline int ntb_link_disable(struct ntb_dev *ntb)
> }
>
> /**
> - * ntb_mw_count() - get the number of memory windows
> + * NTB Memory Windows description
The two variants could be more succintly described as "inbound translation configured on the local ntb port" and "outbound translation configured by the peer, on the peer ntb port" for a locally allocated dma-mapped range of memory.
Please avoid confusing these concepts in the documentation:
- "Memory" on the system vs. "Memory Window" on the ntb
- "Physical" address vs. "dma-mapped" address of memory
- "Base Address" vs. "Translation Address"
Inbound translation:
Memory: Local NTB Port: Peer NTB Port: Peer MMIO:
____________
| dma-mapped |-ntb_set_xlat_addr(addr) |
| memory | _v____________ | ______________
| (addr) |<======| MW xlat addr |<====| MW base addr |<==== memory-mapped IO
|------------| |--------------| | |--------------|
Outbound translation:
Memory: Local NTB Port: Peer NTB Port: Peer MMIO:
____________ ______________
| dma-mapped | | | MW base addr |<==== memory-mapped IO
| memory | | |--------------|
| (addr) |<===========================| MW xlat addr |<-ntb_peer_set_xlat_addr(addr)
|------------| | |--------------|
> + * There are two types of memory window interfaces supported by the NTB API:
> + * local and peer side initialization of memory sharing. The first type is
> + * depicted on the next figure:
> + *
> + * Local device: | Peer device:
> + * NTB config |
> + * Physical memory (RAM) __________ | Memory mapped IO
> + * ____________ +-->| addr | | _____________
> + * | | | |----------| | | |
> + * |------------|addr--+ | |-------------|
> + * | Inbound MW | PCI Express + NTB | Outbound MW |
> + * | |<=====================================| |
> + * |------------| |-------------|
> + *
> + * So typical scenario of the first type memory window initialization looks:
> + * 1) allocate a memory region, 2) put translated base address to NTB config,
> + * 3) somehow notify a peer device of performed initialization, 4) peer device
> + * maps corresponding outbound memory window so to have access to the shared
> + * memory region.
> + *
> + * The second type of interface, that implies the shared windows being
> + * initialized by a peer device, is depicted on the figure:
> + *
> + * Local device: | Peer device:
> + * | NTB config
> + * Physical memory (RAM) | __________ Memory mapped IO
> + * ____________ +-------------->| addr | _____________
> + * | | | | |----------| | |
> + * |------------|addr---+ | |-------------|
> + * | Inbound MW | PCI Express + NTB | Outbound MW |
> + * | |<=====================================| |
> + * |------------| |-------------|
> + *
> + * Typical scenario of the second type initialization would be:
> + * 1) allocate a memory region, 2) somehow deliver a translated base address
> + * to a peer device, 3) peer puts the translated base address to NTB config,
> + * 4) peer device maps outbound memory window so to have access to the shared
> + * memory region.
> + *
> + * As one can see the described scenarios can be combined in one portable
> + * algorithm.
> + * Local device:
> + * 1) Allocate memory for a shared window
> + * 2) Initialize memory window by base address of the allocated region
> + * (it may fail if local memory window initialzation is unsupported)
> + * 3) Send translated base address and memory window index to a peer device
> + * Peer device:
> + * 1) Initialize memory window by retrieved base address of the allocated
> + * by another device memory region (it may fail if peer memory window
> + * initialization is unsupported)
> + * 2) Map outbound memory window
> + * 3) Done
> + * In accordance with this scenario, the NTB Memory Window API can be used as
> + * follows:
> + * Local device:
> + * 1) ntb_mw_count(pidx) - retrieve number of memory ranges, which can
> + * be allocated for memory windows between local device and peer device
> + * of port with specified index.
> + * 2) ntb_get_align(pidx, midx) - retrieve parameters restricting the
> + * shared memory region alignment and size. Then memory can be properly
> + * allocated.
> + * 3) Allocate physically contiguous memory region in complience with
> + * restrictions retrieved in 2).
> + * 4) ntb_mw_set_trans(pidx, midx) - try to set translation address of
> + * the memory window with specified index for the defined peer device
> + * (it may fail if local translated address setting is not supported)
> + * 5) Send translated base address (usually together with memory window
> + * number) to the peer device using, for instance, scratchpad or message
> + * registers.
> + * Peer device:
> + * 1) ntb_peer_mw_set_trans(pidx, midx) - try to set received from other
> + * device (related to pidx) translated base address for specified memory
> + * window. It may fail if retrieved address, for instance, exceeds
> + * maximum possible address or isn't properly aligned.
> + * 2) ntb_peer_mw_get_addr(widx) - retrieve MMIO address to map the memory
> + * window so to have an access to the shared memory.
The above section belongs in Documentation/ntb.txt. Thanks for describing how to use the api so that portable applications can work with either variant of memory window configuration.
> + *
> + * Also it is worth to note, that method ntb_mw_count(pidx) should return the
> + * same value as ntb_peer_mw_count() of the peer with port index - pidx.
> + */
> +
> +/**
> + * ntb_mw_count() - get the number of inbound memory windows, which could
> + * be created for a specified peer device
> * @ntb: NTB device context.
> + * @pidx: Port index of peer device.
> *
> * Hardware and topology may support a different number of memory windows.
> + * Moreover different peer devices can support different number of memory
> + * windows. Simply speaking this method returns the number of possible inbound
> + * memory windows to share with specified peer device.
> *
> * Return: the number of memory windows.
> */
> -static inline int ntb_mw_count(struct ntb_dev *ntb)
> +static inline int ntb_mw_count(struct ntb_dev *ntb, int pidx)
> {
> - return ntb->ops->mw_count(ntb);
> + return ntb->ops->mw_count(ntb, pidx);
> }
>
> /**
> - * ntb_mw_get_range() - get the range of a memory window
> + * ntb_mw_get_align() - get the restriction parameters of inbound memory window
> * @ntb: NTB device context.
> - * @idx: Memory window number.
> - * @base: OUT - the base address for mapping the memory window
> - * @size: OUT - the size for mapping the memory window
> - * @align: OUT - the base alignment for translating the memory window
> - * @align_size: OUT - the size alignment for translating the memory window
> - *
> - * Get the range of a memory window. NULL may be given for any output
> - * parameter if the value is not needed. The base and size may be used for
> - * mapping the memory window, to access the peer memory. The alignment and
> - * size may be used for translating the memory window, for the peer to access
> - * memory on the local system.
> - *
> - * Return: Zero on success, otherwise an error number.
> + * @pidx: Port index of peer device.
> + * @widx: Memory window index.
> + * @addr_align: OUT - the base alignment for translating the memory window
> + * @size_align: OUT - the size alignment for translating the memory window
> + * @size_max: OUT - the maximum size of the memory window
> + *
> + * Get the alignments of an inbound memory window with specified index.
> + * NULL may be given for any output parameter if the value is not needed.
> + * The alignment and size parameters may be used for allocation of proper
> + * shared memory.
> + *
> + * Return: Zero on success, otherwise a negative error number.
> */
> -static inline int ntb_mw_get_range(struct ntb_dev *ntb, int idx,
> - phys_addr_t *base, resource_size_t *size,
> - resource_size_t *align, resource_size_t *align_size)
> +static inline int ntb_mw_get_align(struct ntb_dev *ntb, int pidx, int widx,
> + resource_size_t *addr_align,
> + resource_size_t *size_align,
> + resource_size_t *size_max)
> {
> - return ntb->ops->mw_get_range(ntb, idx, base, size,
> - align, align_size);
> + return ntb->ops->mw_get_align(ntb, pidx, widx, addr_align, size_align,
> + size_max);
> }
>
> /**
> - * ntb_mw_set_trans() - set the translation of a memory window
> + * ntb_mw_set_trans() - set the translation of an inbound memory window
> * @ntb: NTB device context.
> - * @idx: Memory window number.
> - * @addr: The dma address local memory to expose to the peer.
> + * @pidx: Port index of peer device.
> + * @widx: Memory window index.
> + * @addr: The dma address of local memory to expose to the peer.
> * @size: The size of the local memory to expose to the peer.
> *
> * Set the translation of a memory window. The peer may access local memory
> * through the window starting at the address, up to the size. The address
> - * must be aligned to the alignment specified by ntb_mw_get_range(). The size
> - * must be aligned to the size alignment specified by ntb_mw_get_range().
> + * and size must be aligned in complience with restrictions of
> + * ntb_mw_get_align(). The region size should not exceed the size_max parameter
> + * of that method.
> + *
> + * This method may not be implemented due to the hardware specific memory
> + * windows interface.
> *
> * Return: Zero on success, otherwise an error number.
> */
> -static inline int ntb_mw_set_trans(struct ntb_dev *ntb, int idx,
> +static inline int ntb_mw_set_trans(struct ntb_dev *ntb, int pidx, int widx,
> dma_addr_t addr, resource_size_t size)
> {
> - return ntb->ops->mw_set_trans(ntb, idx, addr, size);
> + if (!ntb->ops->mw_set_trans)
> + return -EINVAL;
> +
> + return ntb->ops->mw_set_trans(ntb, pidx, widx, addr, size);
> }
>
> /**
> - * ntb_mw_clear_trans() - clear the translation of a memory window
> + * ntb_mw_clear_trans() - clear the translation address of an inbound memory
> + * window
> * @ntb: NTB device context.
> - * @idx: Memory window number.
> + * @pidx: Port index of peer device.
> + * @widx: Memory window index.
> *
> - * Clear the translation of a memory window. The peer may no longer access
> - * local memory through the window.
> + * Clear the translation of an inbound memory window. The peer may no longer
> + * access local memory through the window.
> *
> * Return: Zero on success, otherwise an error number.
> */
> -static inline int ntb_mw_clear_trans(struct ntb_dev *ntb, int idx)
> +static inline int ntb_mw_clear_trans(struct ntb_dev *ntb, int pidx, int widx)
> {
> if (!ntb->ops->mw_clear_trans)
> - return ntb->ops->mw_set_trans(ntb, idx, 0, 0);
> + return ntb_mw_set_trans(ntb, pidx, widx, 0, 0);
> +
> + return ntb->ops->mw_clear_trans(ntb, pidx, widx);
> +}
> +
> +/**
> + * ntb_peer_mw_count() - get the number of outbound memory windows, which could
> + * be mapped to access a shared memory
> + * @ntb: NTB device context.
> + *
> + * Hardware and topology may support a different number of memory windows.
> + * This method returns the number of outbound memory windows supported by
> + * local device.
> + *
> + * Return: the number of memory windows.
> + */
> +static inline int ntb_peer_mw_count(struct ntb_dev *ntb)
> +{
> + return ntb->ops->peer_mw_count(ntb);
> +}
> +
> +/**
> + * ntb_peer_mw_get_addr() - get map address of an outbound memory window
> + * @ntb: NTB device context.
> + * @widx: Memory window index (within ntb_peer_mw_count() return value).
> + * @base: OUT - the base address of mapping region.
> + * @size: OUT - the size of mapping region.
> + *
> + * Get base and size of memory region to map. NULL may be given for any output
> + * parameter if the value is not needed. The base and size may be used for
> + * mapping the memory window, to access the peer memory.
> + *
> + * Return: Zero on success, otherwise a negative error number.
> + */
> +static inline int ntb_peer_mw_get_addr(struct ntb_dev *ntb, int widx,
> + phys_addr_t *base, resource_size_t *size)
> +{
> + return ntb->ops->peer_mw_get_addr(ntb, widx, base, size);
> +}
> +
> +/**
> + * ntb_peer_mw_set_trans() - set a translation address of a memory window
> + * retrieved from a peer device
> + * @ntb: NTB device context.
> + * @pidx: Port index of peer device the translation address received from.
> + * @widx: Memory window index.
> + * @addr: The dma address of the shared memory to access.
> + * @size: The size of the shared memory to access.
> + *
> + * Set the translation of an outbound memory window. The local device may
> + * access shared memory allocated by a peer device sent the address.
> + *
> + * This method may not be implemented due to the hardware specific memory
> + * windows interface, so a translation address can be only set on the side,
> + * where shared memory (inbound memory windows) is allocated.
> + *
> + * Return: Zero on success, otherwise an error number.
> + */
> +static inline int ntb_peer_mw_set_trans(struct ntb_dev *ntb, int pidx, int widx,
> + u64 addr, resource_size_t size)
> +{
> + if (!ntb->ops->peer_mw_set_trans)
> + return -EINVAL;
> +
> + return ntb->ops->peer_mw_set_trans(ntb, pidx, widx, addr, size);
> +}
> +
> +/**
> + * ntb_peer_mw_clear_trans() - clear the translation address of an outbound
> + * memory window
> + * @ntb: NTB device context.
> + * @pidx: Port index of peer device.
> + * @widx: Memory window index.
> + *
> + * Clear the translation of a outbound memory window. The local device may no
> + * longer access a shared memory through the window.
> + *
> + * This method may not be implemented due to the hardware specific memory
> + * windows interface.
> + *
> + * Return: Zero on success, otherwise an error number.
> + */
> +static inline int ntb_peer_mw_clear_trans(struct ntb_dev *ntb, int pidx,
> + int widx)
> +{
> + if (!ntb->ops->peer_mw_clear_trans)
> + return ntb_peer_mw_set_trans(ntb, pidx, widx, 0, 0);
>
> - return ntb->ops->mw_clear_trans(ntb, idx);
> + return ntb->ops->peer_mw_clear_trans(ntb, pidx, widx);
> }
>
> /**
> --
> 2.6.6
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 00/22] NTB: Alter kernel API to support multi-port devices
@ 2016-11-29 17:15 Serge Semin
2016-11-29 17:15 ` [PATCH 03/22] NTB: Alter NTB API to support both inbound and outbound MW based interfaces Serge Semin
0 siblings, 1 reply; 5+ messages in thread
From: Serge Semin @ 2016-11-29 17:15 UTC (permalink / raw)
To: jdmason, dave.jiang, Allen.Hubbe, Xiangliang.Yu
Cc: Sergey.Semin, linux-ntb, linux-kernel, Serge Semin
There are devices, like IDT PCIe-switches, which have more than just two ports.
Particularly one device can have up to eight ports with NTB-function activated.
In order to support such devices, NTB kernel API should be altered since
currently it's optimized to work with two-ports devices only.
Here are the changes I made to conform the design we discussed a few months ago:
1) Port-index-related methods are added to KAPI
ntb_port_number();
ntb_peer_port_count();
ntb_peer_port_number(pdix);
ntb_peer_port_idx(port);
2) Link state method returns bitfield of link states for each reachable port
u64 ntb_link_is_up();
3) Link enable/disable methods work with local link side of NTB
ntb_link_enable()/ntb_link_disable();
4) NTB memory window related interface does the following things
ntb_mw_count(pidx); - number of inbound memory windows, which can be allocated
for shared buffer with specified peer device.
ntb_mw_get_align(pidx, widx); - get alignment and size restrition parameters
to properly allocate inbound memory region.
ntb_peer_mw_count(); - get number of outbound memory windows.
ntb_peer_mw_get_addr(widx); - get mapping address of an outbound memory window
Inbound MW based hardware:
ntb_mw_set_trans(pidx, widx); - set translation address of allocated inbound
memory window so a peer device could access it.
ntb_mw_clear_trans(pidx, widx); - clear the translation address of an inbound
memory window.
Outbound MW based hardware:
ntb_peer_mw_set_trans(pidx, widx); - set translation address of a memory
window retrieved from a peer device
ntb_peer_mw_clear_trans(pidx, widx); - clear the translation address of an
outbound memory window
5) Scratchpad interface needs to support multi-port devices as well
ntb_spad_count() - return number of Scratchpad per each port
ntb_peer_spad_addr(pidx, sidx) - address of Scratchpad register of the
peer device with pidx-index
ntb_peer_spad_read(pidx, sidx) - read specified Scratchpad register of the
peer with pidx-index
ntb_peer_spad_write(pidx, sidx) - write data to Scratchpad register of the
peer with pidx-index
6) Introduce new messaging interface of NTB KAPI
ntb_msg_count(); - get number of message registers
ntb_msg_inbits(); - get bitfield of inbound message registers status
ntb_msg_outbits(); - get bitfield of outbound message registers status
ntb_msg_read_sts(); - read the inbound and outbound message registers status
ntb_msg_clear_sts(); - clear status bits of message registers
ntb_msg_set_mask(); - mask interrupts raised by status bits of message
registers.
ntb_msg_clear_mask(); - clear interrupts mask bits of message registers
ntb_msg_recv(midx, *pidx); - read message register with specified index,
additionally getting peer port index which data received from
ntb_msg_send(midx, pidx); - write data to the specified message register
sending it to the passed peer device connected over a pidx port
ntb_msg_event(); - notify driver context of a new message event
7) Topology reduced to be either P2P (port-to-port) or B2B (bridge-to-bridge).
Since there is port number introduced to be part of ntb_dev structure, real
port number can be used to determine Primary and Secondary sides. Intel and AMD
driver are altered to support this novation.
8) Standard test drivers: PingPong, Debugging tool and Raw Perf as well as NTB
Transport drivers don't support multi-port devices at the moment.
Since we haven't got any real multi-port hadrware driver, it's dangerous to
make any serious alterations in the debugging tools. So I have made those
drivers to work the way they wokred before, but using the new NTB API.
The situation will change when I finish porting my current IDT NTB driver to
support new API. Then I'll be able to refactor the tools and test them using
real multi-port hardware.
The changes are made on top of the NTB-fork of the kernel:
https://github.com/jonmason/ntb
of "ntb"-branch with last commit:
9c763584b7c8911106bb77af7e648bef09af9d80 Linux 4.9-rc6
Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
Serge Semin (22):
NTB: Move link state API being first in sources
NTB: Add peer indexed ports NTB API
NTB: Alter NTB API to support both inbound and outbound MW based
interfaces
NTB: Add messaging NTB API
NTB: Alter Scratchpads NTB API to support multi-ports interface
NTB: Slightly alter link state NTB API
NTB: Fix a few ntb.h issues
NTB: Add T-Platforms copyrights to NTB API
NTB Intel: Move link-related methods being first in the driver
NTB Intel: Add port-related NTB API callback methods
NTB Intel: Alter MW interface to fit new NTB API
NTB Intel: Alter Scratchpads interface to fit new NTB API
NTB Intel: Add T-Platforms copyrights to Intel NTB driver
NTB AMD: Move link-related methods being first in the driver
NTB AMD: Add port-related NTB API callback methods
NTB AMD: Alter MW interface to fit new NTB API
NTB AMD: Alter Scratchpads interface to fit new NTB API
NTB AMD: Add T-Platforms copyrights to AMD NTB driver
NTB PingPong: Alter driver to work with two-ports NTB API
NTB Tool: Alter driver to work with two-ports NTB API
NTB Perf: Alter driver to work with two-ports NTB API
NTB Transport: Alter driver to work with two-ports NTB API
drivers/ntb/hw/amd/ntb_hw_amd.c | 310 +++++++++------
drivers/ntb/hw/amd/ntb_hw_amd.h | 12 +
drivers/ntb/hw/intel/ntb_hw_intel.c | 420 ++++++++++++--------
drivers/ntb/hw/intel/ntb_hw_intel.h | 12 +
drivers/ntb/ntb.c | 15 +
drivers/ntb/ntb_transport.c | 43 +-
drivers/ntb/test/ntb_perf.c | 27 +-
drivers/ntb/test/ntb_pingpong.c | 11 +-
drivers/ntb/test/ntb_tool.c | 87 +++--
include/linux/ntb.h | 753 +++++++++++++++++++++++++++++-------
10 files changed, 1221 insertions(+), 469 deletions(-)
--
2.6.6
^ permalink raw reply [flat|nested] 5+ messages in thread* [PATCH 03/22] NTB: Alter NTB API to support both inbound and outbound MW based interfaces
2016-11-29 17:15 [PATCH 00/22] NTB: Alter kernel API to support multi-port devices Serge Semin
@ 2016-11-29 17:15 ` Serge Semin
2016-11-30 18:54 ` kbuild test robot
2016-11-30 19:46 ` kbuild test robot
0 siblings, 2 replies; 5+ messages in thread
From: Serge Semin @ 2016-11-29 17:15 UTC (permalink / raw)
To: jdmason, dave.jiang, Allen.Hubbe, Xiangliang.Yu
Cc: Sergey.Semin, linux-ntb, linux-kernel, Serge Semin
Alter NTB API to support inbound and outbound MW based interfaces.
Additionally I made it supporting multi-port devices as well. Useful
infographics is added right before MW API is declared. It shall help to
better understand how the new API really works and how it can be utilized
within client drivers.
Signed-off-by: Serge Semin <fancer.lancer@gmail.com>
---
include/linux/ntb.h | 290 ++++++++++++++++++++++++++++++++++++++++++++--------
1 file changed, 245 insertions(+), 45 deletions(-)
diff --git a/include/linux/ntb.h b/include/linux/ntb.h
index 0941a43..4a150b5 100644
--- a/include/linux/ntb.h
+++ b/include/linux/ntb.h
@@ -171,9 +171,13 @@ static inline int ntb_ctx_ops_is_valid(const struct ntb_ctx_ops *ops)
* @link_enable: See ntb_link_enable().
* @link_disable: See ntb_link_disable().
* @mw_count: See ntb_mw_count().
- * @mw_get_range: See ntb_mw_get_range().
+ * @mw_get_align: See ntb_mw_get_align().
* @mw_set_trans: See ntb_mw_set_trans().
* @mw_clear_trans: See ntb_mw_clear_trans().
+ * @peer_mw_count: See ntb_peer_mw_count().
+ * @peer_mw_get_addr: See ntb_peer_mw_get_addr().
+ * @peer_mw_set_trans: See ntb_peer_mw_set_trans().
+ * @peer_mw_clear_trans:See ntb_peer_mw_clear_trans().
* @db_is_unsafe: See ntb_db_is_unsafe().
* @db_valid_mask: See ntb_db_valid_mask().
* @db_vector_count: See ntb_db_vector_count().
@@ -211,13 +215,20 @@ struct ntb_dev_ops {
enum ntb_speed max_speed, enum ntb_width max_width);
int (*link_disable)(struct ntb_dev *ntb);
- int (*mw_count)(struct ntb_dev *ntb);
- int (*mw_get_range)(struct ntb_dev *ntb, int idx,
- phys_addr_t *base, resource_size_t *size,
- resource_size_t *align, resource_size_t *align_size);
- int (*mw_set_trans)(struct ntb_dev *ntb, int idx,
+ int (*mw_count)(struct ntb_dev *ntb, int pidx);
+ int (*mw_get_align)(struct ntb_dev *ntb, int pidx, int widx,
+ resource_size_t *addr_align,
+ resource_size_t *size_align,
+ resource_size_t *size_max);
+ int (*mw_set_trans)(struct ntb_dev *ntb, int pidx, int widx,
dma_addr_t addr, resource_size_t size);
- int (*mw_clear_trans)(struct ntb_dev *ntb, int idx);
+ int (*mw_clear_trans)(struct ntb_dev *ntb, int pidx, int widx);
+ int (*peer_mw_count)(struct ntb_dev *ntb);
+ int (*peer_mw_get_addr)(struct ntb_dev *ntb, int widx,
+ phys_addr_t *base, resource_size_t *size);
+ int (*peer_mw_set_trans)(struct ntb_dev *ntb, int pidx, int widx,
+ u64 addr, resource_size_t size);
+ int (*peer_mw_clear_trans)(struct ntb_dev *ntb, int pidx, int widx);
int (*db_is_unsafe)(struct ntb_dev *ntb);
u64 (*db_valid_mask)(struct ntb_dev *ntb);
@@ -266,9 +277,13 @@ static inline int ntb_dev_ops_is_valid(const struct ntb_dev_ops *ops)
ops->link_enable &&
ops->link_disable &&
ops->mw_count &&
- ops->mw_get_range &&
- ops->mw_set_trans &&
+ ops->mw_get_align &&
+ (ops->mw_set_trans ||
+ ops->peer_mw_set_trans) &&
/* ops->mw_clear_trans && */
+ ops->peer_mw_count &&
+ ops->peer_mw_get_addr &&
+ /* ops->peer_mw_clear_trans && */
/* ops->db_is_unsafe && */
ops->db_valid_mask &&
@@ -555,79 +570,264 @@ static inline int ntb_link_disable(struct ntb_dev *ntb)
}
/**
- * ntb_mw_count() - get the number of memory windows
+ * NTB Memory Windows description
+ *
+ * There are two types of memory window interfaces supported by the NTB API:
+ * local and peer side initialization of memory sharing. The first type is
+ * depicted on the next figure:
+ *
+ * Local device: | Peer device:
+ * NTB config |
+ * Physical memory (RAM) __________ | Memory mapped IO
+ * ____________ +-->| addr | | _____________
+ * | | | |----------| | | |
+ * |------------|addr--+ | |-------------|
+ * | Inbound MW | PCI Express + NTB | Outbound MW |
+ * | |<=====================================| |
+ * |------------| |-------------|
+ *
+ * So typical scenario of the first type memory window initialization looks:
+ * 1) allocate a memory region, 2) put translated base address to NTB config,
+ * 3) somehow notify a peer device of performed initialization, 4) peer device
+ * maps corresponding outbound memory window so to have access to the shared
+ * memory region.
+ *
+ * The second type of interface, that implies the shared windows being
+ * initialized by a peer device, is depicted on the figure:
+ *
+ * Local device: | Peer device:
+ * | NTB config
+ * Physical memory (RAM) | __________ Memory mapped IO
+ * ____________ +-------------->| addr | _____________
+ * | | | | |----------| | |
+ * |------------|addr---+ | |-------------|
+ * | Inbound MW | PCI Express + NTB | Outbound MW |
+ * | |<=====================================| |
+ * |------------| |-------------|
+ *
+ * Typical scenario of the second type initialization would be:
+ * 1) allocate a memory region, 2) somehow deliver a translated base address
+ * to a peer device, 3) peer puts the translated base address to NTB config,
+ * 4) peer device maps outbound memory window so to have access to the shared
+ * memory region.
+ *
+ * As one can see the described scenarios can be combined in one portable
+ * algorithm.
+ * Local device:
+ * 1) Allocate memory for a shared window
+ * 2) Initialize memory window by base address of the allocated region
+ * (it may fail if local memory window initialzation is unsupported)
+ * 3) Send translated base address and memory window index to a peer device
+ * Peer device:
+ * 1) Initialize memory window by retrieved base address of the allocated
+ * by another device memory region (it may fail if peer memory window
+ * initialization is unsupported)
+ * 2) Map outbound memory window
+ * 3) Done
+ * In accordance with this scenario, the NTB Memory Window API can be used as
+ * follows:
+ * Local device:
+ * 1) ntb_mw_count(pidx) - retrieve number of memory ranges, which can
+ * be allocated for memory windows between local device and peer device
+ * of port with specified index.
+ * 2) ntb_get_align(pidx, midx) - retrieve parameters restricting the
+ * shared memory region alignment and size. Then memory can be properly
+ * allocated.
+ * 3) Allocate physically contiguous memory region in complience with
+ * restrictions retrieved in 2).
+ * 4) ntb_mw_set_trans(pidx, midx) - try to set translation address of
+ * the memory window with specified index for the defined peer device
+ * (it may fail if local translated address setting is not supported)
+ * 5) Send translated base address (usually together with memory window
+ * number) to the peer device using, for instance, scratchpad or message
+ * registers.
+ * Peer device:
+ * 1) ntb_peer_mw_set_trans(pidx, midx) - try to set received from other
+ * device (related to pidx) translated base address for specified memory
+ * window. It may fail if retrieved address, for instance, exceeds
+ * maximum possible address or isn't properly aligned.
+ * 2) ntb_peer_mw_get_addr(widx) - retrieve MMIO address to map the memory
+ * window so to have an access to the shared memory.
+ *
+ * Also it is worth to note, that method ntb_mw_count(pidx) should return the
+ * same value as ntb_peer_mw_count() of the peer with port index - pidx.
+ */
+
+/**
+ * ntb_mw_count() - get the number of inbound memory windows, which could
+ * be created for a specified peer device
* @ntb: NTB device context.
+ * @pidx: Port index of peer device.
*
* Hardware and topology may support a different number of memory windows.
+ * Moreover different peer devices can support different number of memory
+ * windows. Simply speaking this method returns the number of possible inbound
+ * memory windows to share with specified peer device.
*
* Return: the number of memory windows.
*/
-static inline int ntb_mw_count(struct ntb_dev *ntb)
+static inline int ntb_mw_count(struct ntb_dev *ntb, int pidx)
{
- return ntb->ops->mw_count(ntb);
+ return ntb->ops->mw_count(ntb, pidx);
}
/**
- * ntb_mw_get_range() - get the range of a memory window
+ * ntb_mw_get_align() - get the restriction parameters of inbound memory window
* @ntb: NTB device context.
- * @idx: Memory window number.
- * @base: OUT - the base address for mapping the memory window
- * @size: OUT - the size for mapping the memory window
- * @align: OUT - the base alignment for translating the memory window
- * @align_size: OUT - the size alignment for translating the memory window
- *
- * Get the range of a memory window. NULL may be given for any output
- * parameter if the value is not needed. The base and size may be used for
- * mapping the memory window, to access the peer memory. The alignment and
- * size may be used for translating the memory window, for the peer to access
- * memory on the local system.
- *
- * Return: Zero on success, otherwise an error number.
+ * @pidx: Port index of peer device.
+ * @widx: Memory window index.
+ * @addr_align: OUT - the base alignment for translating the memory window
+ * @size_align: OUT - the size alignment for translating the memory window
+ * @size_max: OUT - the maximum size of the memory window
+ *
+ * Get the alignments of an inbound memory window with specified index.
+ * NULL may be given for any output parameter if the value is not needed.
+ * The alignment and size parameters may be used for allocation of proper
+ * shared memory.
+ *
+ * Return: Zero on success, otherwise a negative error number.
*/
-static inline int ntb_mw_get_range(struct ntb_dev *ntb, int idx,
- phys_addr_t *base, resource_size_t *size,
- resource_size_t *align, resource_size_t *align_size)
+static inline int ntb_mw_get_align(struct ntb_dev *ntb, int pidx, int widx,
+ resource_size_t *addr_align,
+ resource_size_t *size_align,
+ resource_size_t *size_max)
{
- return ntb->ops->mw_get_range(ntb, idx, base, size,
- align, align_size);
+ return ntb->ops->mw_get_align(ntb, pidx, widx, addr_align, size_align,
+ size_max);
}
/**
- * ntb_mw_set_trans() - set the translation of a memory window
+ * ntb_mw_set_trans() - set the translation of an inbound memory window
* @ntb: NTB device context.
- * @idx: Memory window number.
- * @addr: The dma address local memory to expose to the peer.
+ * @pidx: Port index of peer device.
+ * @widx: Memory window index.
+ * @addr: The dma address of local memory to expose to the peer.
* @size: The size of the local memory to expose to the peer.
*
* Set the translation of a memory window. The peer may access local memory
* through the window starting at the address, up to the size. The address
- * must be aligned to the alignment specified by ntb_mw_get_range(). The size
- * must be aligned to the size alignment specified by ntb_mw_get_range().
+ * and size must be aligned in complience with restrictions of
+ * ntb_mw_get_align(). The region size should not exceed the size_max parameter
+ * of that method.
+ *
+ * This method may not be implemented due to the hardware specific memory
+ * windows interface.
*
* Return: Zero on success, otherwise an error number.
*/
-static inline int ntb_mw_set_trans(struct ntb_dev *ntb, int idx,
+static inline int ntb_mw_set_trans(struct ntb_dev *ntb, int pidx, int widx,
dma_addr_t addr, resource_size_t size)
{
- return ntb->ops->mw_set_trans(ntb, idx, addr, size);
+ if (!ntb->ops->mw_set_trans)
+ return -EINVAL;
+
+ return ntb->ops->mw_set_trans(ntb, pidx, widx, addr, size);
}
/**
- * ntb_mw_clear_trans() - clear the translation of a memory window
+ * ntb_mw_clear_trans() - clear the translation address of an inbound memory
+ * window
* @ntb: NTB device context.
- * @idx: Memory window number.
+ * @pidx: Port index of peer device.
+ * @widx: Memory window index.
*
- * Clear the translation of a memory window. The peer may no longer access
- * local memory through the window.
+ * Clear the translation of an inbound memory window. The peer may no longer
+ * access local memory through the window.
*
* Return: Zero on success, otherwise an error number.
*/
-static inline int ntb_mw_clear_trans(struct ntb_dev *ntb, int idx)
+static inline int ntb_mw_clear_trans(struct ntb_dev *ntb, int pidx, int widx)
{
if (!ntb->ops->mw_clear_trans)
- return ntb->ops->mw_set_trans(ntb, idx, 0, 0);
+ return ntb_mw_set_trans(ntb, pidx, widx, 0, 0);
+
+ return ntb->ops->mw_clear_trans(ntb, pidx, widx);
+}
+
+/**
+ * ntb_peer_mw_count() - get the number of outbound memory windows, which could
+ * be mapped to access a shared memory
+ * @ntb: NTB device context.
+ *
+ * Hardware and topology may support a different number of memory windows.
+ * This method returns the number of outbound memory windows supported by
+ * local device.
+ *
+ * Return: the number of memory windows.
+ */
+static inline int ntb_peer_mw_count(struct ntb_dev *ntb)
+{
+ return ntb->ops->peer_mw_count(ntb);
+}
+
+/**
+ * ntb_peer_mw_get_addr() - get map address of an outbound memory window
+ * @ntb: NTB device context.
+ * @widx: Memory window index (within ntb_peer_mw_count() return value).
+ * @base: OUT - the base address of mapping region.
+ * @size: OUT - the size of mapping region.
+ *
+ * Get base and size of memory region to map. NULL may be given for any output
+ * parameter if the value is not needed. The base and size may be used for
+ * mapping the memory window, to access the peer memory.
+ *
+ * Return: Zero on success, otherwise a negative error number.
+ */
+static inline int ntb_peer_mw_get_addr(struct ntb_dev *ntb, int widx,
+ phys_addr_t *base, resource_size_t *size)
+{
+ return ntb->ops->peer_mw_get_addr(ntb, widx, base, size);
+}
+
+/**
+ * ntb_peer_mw_set_trans() - set a translation address of a memory window
+ * retrieved from a peer device
+ * @ntb: NTB device context.
+ * @pidx: Port index of peer device the translation address received from.
+ * @widx: Memory window index.
+ * @addr: The dma address of the shared memory to access.
+ * @size: The size of the shared memory to access.
+ *
+ * Set the translation of an outbound memory window. The local device may
+ * access shared memory allocated by a peer device sent the address.
+ *
+ * This method may not be implemented due to the hardware specific memory
+ * windows interface, so a translation address can be only set on the side,
+ * where shared memory (inbound memory windows) is allocated.
+ *
+ * Return: Zero on success, otherwise an error number.
+ */
+static inline int ntb_peer_mw_set_trans(struct ntb_dev *ntb, int pidx, int widx,
+ u64 addr, resource_size_t size)
+{
+ if (!ntb->ops->peer_mw_set_trans)
+ return -EINVAL;
+
+ return ntb->ops->peer_mw_set_trans(ntb, pidx, widx, addr, size);
+}
+
+/**
+ * ntb_peer_mw_clear_trans() - clear the translation address of an outbound
+ * memory window
+ * @ntb: NTB device context.
+ * @pidx: Port index of peer device.
+ * @widx: Memory window index.
+ *
+ * Clear the translation of a outbound memory window. The local device may no
+ * longer access a shared memory through the window.
+ *
+ * This method may not be implemented due to the hardware specific memory
+ * windows interface.
+ *
+ * Return: Zero on success, otherwise an error number.
+ */
+static inline int ntb_peer_mw_clear_trans(struct ntb_dev *ntb, int pidx,
+ int widx)
+{
+ if (!ntb->ops->peer_mw_clear_trans)
+ return ntb_peer_mw_set_trans(ntb, pidx, widx, 0, 0);
- return ntb->ops->mw_clear_trans(ntb, idx);
+ return ntb->ops->peer_mw_clear_trans(ntb, pidx, widx);
}
/**
--
2.6.6
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PATCH 03/22] NTB: Alter NTB API to support both inbound and outbound MW based interfaces
2016-11-29 17:15 ` [PATCH 03/22] NTB: Alter NTB API to support both inbound and outbound MW based interfaces Serge Semin
@ 2016-11-30 18:54 ` kbuild test robot
2016-11-30 19:46 ` kbuild test robot
1 sibling, 0 replies; 5+ messages in thread
From: kbuild test robot @ 2016-11-30 18:54 UTC (permalink / raw)
To: Serge Semin
Cc: kbuild-all, jdmason, dave.jiang, Allen.Hubbe, Xiangliang.Yu,
Sergey.Semin, linux-ntb, linux-kernel, Serge Semin
[-- Attachment #1: Type: text/plain, Size: 8739 bytes --]
Hi Serge,
[auto build test ERROR on ntb/ntb-next]
[also build test ERROR on v4.9-rc7 next-20161130]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Serge-Semin/NTB-Alter-kernel-API-to-support-multi-port-devices/20161201-014939
base: https://github.com/jonmason/ntb ntb-next
config: x86_64-allmodconfig (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
# save the attached .config to linux build tree
make ARCH=x86_64
All errors (new ones prefixed by >>):
drivers/ntb/hw/intel/ntb_hw_intel.c: In function 'ndev_reset_unsafe_flags':
drivers/ntb/hw/intel/ntb_hw_intel.c:232:8: error: implicit declaration of function 'ntb_topo_is_b2b' [-Werror=implicit-function-declaration]
if (!ntb_topo_is_b2b(ndev->ntb.topo))
^~~~~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c: In function 'intel_ntb_link_enable':
drivers/ntb/hw/intel/ntb_hw_intel.c:1206:24: error: 'NTB_TOPO_SEC' undeclared (first use in this function)
if (ndev->ntb.topo == NTB_TOPO_SEC)
^~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c:1206:24: note: each undeclared identifier is reported only once for each function it appears in
drivers/ntb/hw/intel/ntb_hw_intel.c: In function 'intel_ntb_link_disable':
drivers/ntb/hw/intel/ntb_hw_intel.c:1235:24: error: 'NTB_TOPO_SEC' undeclared (first use in this function)
if (ndev->ntb.topo == NTB_TOPO_SEC)
^~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c: In function 'atom_ppd_topo':
drivers/ntb/hw/intel/ntb_hw_intel.c:1448:10: error: 'NTB_TOPO_B2B_USD' undeclared (first use in this function)
return NTB_TOPO_B2B_USD;
^~~~~~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c:1452:10: error: 'NTB_TOPO_B2B_DSD' undeclared (first use in this function)
return NTB_TOPO_B2B_DSD;
^~~~~~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c: In function 'atom_init_ntb':
drivers/ntb/hw/intel/ntb_hw_intel.c:1572:7: error: 'NTB_TOPO_B2B_USD' undeclared (first use in this function)
case NTB_TOPO_B2B_USD:
^~~~~~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c:1573:7: error: 'NTB_TOPO_B2B_DSD' undeclared (first use in this function)
case NTB_TOPO_B2B_DSD:
^~~~~~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c: In function 'atom_init_dev':
drivers/ntb/hw/intel/ntb_hw_intel.c:1614:24: error: 'NTB_TOPO_SEC' undeclared (first use in this function)
if (ndev->ntb.topo != NTB_TOPO_SEC) {
^~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c: In function 'skx_init_ntb':
drivers/ntb/hw/intel/ntb_hw_intel.c:1789:7: error: 'NTB_TOPO_B2B_USD' undeclared (first use in this function)
case NTB_TOPO_B2B_USD:
^~~~~~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c:1790:7: error: 'NTB_TOPO_B2B_DSD' undeclared (first use in this function)
case NTB_TOPO_B2B_DSD:
^~~~~~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c: In function 'xeon_link_is_up':
drivers/ntb/hw/intel/ntb_hw_intel.c:2043:24: error: 'NTB_TOPO_SEC' undeclared (first use in this function)
if (ndev->ntb.topo == NTB_TOPO_SEC)
^~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c: In function 'xeon_ppd_topo':
drivers/ntb/hw/intel/ntb_hw_intel.c:2053:10: error: 'NTB_TOPO_B2B_USD' undeclared (first use in this function)
return NTB_TOPO_B2B_USD;
^~~~~~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c:2056:10: error: 'NTB_TOPO_B2B_DSD' undeclared (first use in this function)
return NTB_TOPO_B2B_DSD;
^~~~~~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c:2060:10: error: 'NTB_TOPO_PRI' undeclared (first use in this function)
return NTB_TOPO_PRI;
^~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c:2064:10: error: 'NTB_TOPO_SEC' undeclared (first use in this function)
return NTB_TOPO_SEC;
^~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c: In function 'xeon_init_ntb':
drivers/ntb/hw/intel/ntb_hw_intel.c:2356:7: error: 'NTB_TOPO_PRI' undeclared (first use in this function)
case NTB_TOPO_PRI:
^~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c:2374:7: error: 'NTB_TOPO_SEC' undeclared (first use in this function)
case NTB_TOPO_SEC:
^~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c:2386:7: error: 'NTB_TOPO_B2B_USD' undeclared (first use in this function)
case NTB_TOPO_B2B_USD:
^~~~~~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c:2387:7: error: 'NTB_TOPO_B2B_DSD' undeclared (first use in this function)
case NTB_TOPO_B2B_DSD:
^~~~~~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c: In function 'xeon_init_dev':
drivers/ntb/hw/intel/ntb_hw_intel.c:2534:24: error: 'NTB_TOPO_SEC' undeclared (first use in this function)
if (ndev->ntb.topo != NTB_TOPO_SEC) {
^~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c: At top level:
>> drivers/ntb/hw/intel/ntb_hw_intel.c:2886:15: error: initialization from incompatible pointer type [-Werror=incompatible-pointer-types]
.mw_count = intel_ntb_mw_count,
^~~~~~~~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c:2886:15: note: (near initialization for 'intel_ntb_ops.mw_count')
>> drivers/ntb/hw/intel/ntb_hw_intel.c:2887:2: error: unknown field 'mw_get_range' specified in initializer
.mw_get_range = intel_ntb_mw_get_range,
^
drivers/ntb/hw/intel/ntb_hw_intel.c:2887:19: error: initialization from incompatible pointer type [-Werror=incompatible-pointer-types]
.mw_get_range = intel_ntb_mw_get_range,
^~~~~~~~~~~~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c:2887:19: note: (near initialization for 'intel_ntb_ops.mw_get_align')
drivers/ntb/hw/intel/ntb_hw_intel.c:2888:19: error: initialization from incompatible pointer type [-Werror=incompatible-pointer-types]
.mw_set_trans = intel_ntb_mw_set_trans,
^~~~~~~~~~~~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c:2888:19: note: (near initialization for 'intel_ntb_ops.mw_set_trans')
drivers/ntb/hw/intel/ntb_hw_intel.c:2912:15: error: initialization from incompatible pointer type [-Werror=incompatible-pointer-types]
.mw_count = intel_ntb_mw_count,
^~~~~~~~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c:2912:15: note: (near initialization for 'intel_ntb3_ops.mw_count')
drivers/ntb/hw/intel/ntb_hw_intel.c:2913:2: error: unknown field 'mw_get_range' specified in initializer
.mw_get_range = intel_ntb_mw_get_range,
^
drivers/ntb/hw/intel/ntb_hw_intel.c:2913:19: error: initialization from incompatible pointer type [-Werror=incompatible-pointer-types]
.mw_get_range = intel_ntb_mw_get_range,
^~~~~~~~~~~~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c:2913:19: note: (near initialization for 'intel_ntb3_ops.mw_get_align')
drivers/ntb/hw/intel/ntb_hw_intel.c:2914:19: error: initialization from incompatible pointer type [-Werror=incompatible-pointer-types]
.mw_set_trans = intel_ntb3_mw_set_trans,
^~~~~~~~~~~~~~~~~~~~~~~
drivers/ntb/hw/intel/ntb_hw_intel.c:2914:19: note: (near initialization for 'intel_ntb3_ops.mw_set_trans')
cc1: some warnings being treated as errors
vim +/mw_get_range +2887 drivers/ntb/hw/intel/ntb_hw_intel.c
89a85cde Dave Jiang 2016-11-16 2880 .bar2_limit = SKX_IMBAR1XLMT_OFFSET,
89a85cde Dave Jiang 2016-11-16 2881 .bar2_xlat = SKX_IMBAR1XBASE_OFFSET,
89a85cde Dave Jiang 2016-11-16 2882 };
89a85cde Dave Jiang 2016-11-16 2883
e26a5843 Allen Hubbe 2015-04-09 2884 /* operations for primary side of local ntb */
e26a5843 Allen Hubbe 2015-04-09 2885 static const struct ntb_dev_ops intel_ntb_ops = {
e26a5843 Allen Hubbe 2015-04-09 @2886 .mw_count = intel_ntb_mw_count,
e26a5843 Allen Hubbe 2015-04-09 @2887 .mw_get_range = intel_ntb_mw_get_range,
e26a5843 Allen Hubbe 2015-04-09 2888 .mw_set_trans = intel_ntb_mw_set_trans,
e26a5843 Allen Hubbe 2015-04-09 2889 .link_is_up = intel_ntb_link_is_up,
e26a5843 Allen Hubbe 2015-04-09 2890 .link_enable = intel_ntb_link_enable,
:::::: The code at line 2887 was first introduced by commit
:::::: e26a5843f7f5014ae4460030ca4de029a3ac35d3 NTB: Split ntb_hw_intel and ntb_transport drivers
:::::: TO: Allen Hubbe <Allen.Hubbe@emc.com>
:::::: CC: Jon Mason <jdmason@kudzu.us>
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 56642 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: [PATCH 03/22] NTB: Alter NTB API to support both inbound and outbound MW based interfaces
2016-11-29 17:15 ` [PATCH 03/22] NTB: Alter NTB API to support both inbound and outbound MW based interfaces Serge Semin
2016-11-30 18:54 ` kbuild test robot
@ 2016-11-30 19:46 ` kbuild test robot
1 sibling, 0 replies; 5+ messages in thread
From: kbuild test robot @ 2016-11-30 19:46 UTC (permalink / raw)
To: Serge Semin
Cc: kbuild-all, jdmason, dave.jiang, Allen.Hubbe, Xiangliang.Yu,
Sergey.Semin, linux-ntb, linux-kernel, Serge Semin
[-- Attachment #1: Type: text/plain, Size: 11289 bytes --]
Hi Serge,
[auto build test ERROR on ntb/ntb-next]
[also build test ERROR on v4.9-rc7 next-20161130]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Serge-Semin/NTB-Alter-kernel-API-to-support-multi-port-devices/20161201-014939
base: https://github.com/jonmason/ntb ntb-next
config: x86_64-randconfig-s5-12010242 (attached as .config)
compiler: gcc-6 (Debian 6.2.0-3) 6.2.0 20160901
reproduce:
# save the attached .config to linux build tree
make ARCH=x86_64
Note: the linux-review/Serge-Semin/NTB-Alter-kernel-API-to-support-multi-port-devices/20161201-014939 HEAD ced946cf007084caf9a2ec237c898bbf1940b440 builds fine.
It only hurts bisectibility.
All errors (new ones prefixed by >>):
drivers/ntb/ntb_transport.c: In function 'ntb_free_mw':
>> drivers/ntb/ntb_transport.c:685:2: error: too few arguments to function 'ntb_mw_clear_trans'
ntb_mw_clear_trans(nt->ndev, num_mw);
^~~~~~~~~~~~~~~~~~
In file included from drivers/ntb/ntb_transport.c:62:0:
include/linux/ntb.h:739:19: note: declared here
static inline int ntb_mw_clear_trans(struct ntb_dev *ntb, int pidx, int widx)
^~~~~~~~~~~~~~~~~~
drivers/ntb/ntb_transport.c: In function 'ntb_set_mw':
>> drivers/ntb/ntb_transport.c:742:7: error: too few arguments to function 'ntb_mw_set_trans'
rc = ntb_mw_set_trans(nt->ndev, num_mw, mw->dma_addr, mw->xlat_size);
^~~~~~~~~~~~~~~~
In file included from drivers/ntb/ntb_transport.c:62:0:
include/linux/ntb.h:718:19: note: declared here
static inline int ntb_mw_set_trans(struct ntb_dev *ntb, int pidx, int widx,
^~~~~~~~~~~~~~~~
drivers/ntb/ntb_transport.c: In function 'ntb_transport_probe':
>> drivers/ntb/ntb_transport.c:1072:13: error: too few arguments to function 'ntb_mw_count'
mw_count = ntb_mw_count(ndev);
^~~~~~~~~~~~
In file included from drivers/ntb/ntb_transport.c:62:0:
include/linux/ntb.h:669:19: note: declared here
static inline int ntb_mw_count(struct ntb_dev *ntb, int pidx)
^~~~~~~~~~~~
>> drivers/ntb/ntb_transport.c:1106:8: error: implicit declaration of function 'ntb_mw_get_range' [-Werror=implicit-function-declaration]
rc = ntb_mw_get_range(ndev, i, &mw->phys_addr, &mw->phys_size,
^~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
--
drivers/ntb/test/ntb_perf.c: In function 'perf_free_mw':
>> drivers/ntb/test/ntb_perf.c:455:2: error: too few arguments to function 'ntb_mw_clear_trans'
ntb_mw_clear_trans(perf->ntb, 0);
^~~~~~~~~~~~~~~~~~
In file included from drivers/ntb/test/ntb_perf.c:60:0:
include/linux/ntb.h:739:19: note: declared here
static inline int ntb_mw_clear_trans(struct ntb_dev *ntb, int pidx, int widx)
^~~~~~~~~~~~~~~~~~
drivers/ntb/test/ntb_perf.c: In function 'perf_set_mw':
>> drivers/ntb/test/ntb_perf.c:491:7: error: too few arguments to function 'ntb_mw_set_trans'
rc = ntb_mw_set_trans(perf->ntb, 0, mw->dma_addr, mw->xlat_size);
^~~~~~~~~~~~~~~~
In file included from drivers/ntb/test/ntb_perf.c:60:0:
include/linux/ntb.h:718:19: note: declared here
static inline int ntb_mw_set_trans(struct ntb_dev *ntb, int pidx, int widx,
^~~~~~~~~~~~~~~~
drivers/ntb/test/ntb_perf.c: In function 'perf_setup_mw':
>> drivers/ntb/test/ntb_perf.c:562:7: error: implicit declaration of function 'ntb_mw_get_range' [-Werror=implicit-function-declaration]
rc = ntb_mw_get_range(ntb, 0, &mw->phys_addr, &mw->phys_size,
^~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
--
drivers/ntb/hw/amd/ntb_hw_amd.c: In function 'amd_ntb_link_enable':
drivers/ntb/hw/amd/ntb_hw_amd.c:256:24: error: 'NTB_TOPO_SEC' undeclared (first use in this function)
if (ndev->ntb.topo == NTB_TOPO_SEC)
^~~~~~~~~~~~
drivers/ntb/hw/amd/ntb_hw_amd.c:256:24: note: each undeclared identifier is reported only once for each function it appears in
drivers/ntb/hw/amd/ntb_hw_amd.c: In function 'amd_ntb_link_disable':
drivers/ntb/hw/amd/ntb_hw_amd.c:277:24: error: 'NTB_TOPO_SEC' undeclared (first use in this function)
if (ndev->ntb.topo == NTB_TOPO_SEC)
^~~~~~~~~~~~
drivers/ntb/hw/amd/ntb_hw_amd.c: At top level:
>> drivers/ntb/hw/amd/ntb_hw_amd.c:434:15: error: initialization from incompatible pointer type [-Werror=incompatible-pointer-types]
.mw_count = amd_ntb_mw_count,
^~~~~~~~~~~~~~~~
drivers/ntb/hw/amd/ntb_hw_amd.c:434:15: note: (near initialization for 'amd_ntb_ops.mw_count')
>> drivers/ntb/hw/amd/ntb_hw_amd.c:435:2: error: unknown field 'mw_get_range' specified in initializer
.mw_get_range = amd_ntb_mw_get_range,
^
drivers/ntb/hw/amd/ntb_hw_amd.c:435:19: error: initialization from incompatible pointer type [-Werror=incompatible-pointer-types]
.mw_get_range = amd_ntb_mw_get_range,
^~~~~~~~~~~~~~~~~~~~
drivers/ntb/hw/amd/ntb_hw_amd.c:435:19: note: (near initialization for 'amd_ntb_ops.mw_get_align')
drivers/ntb/hw/amd/ntb_hw_amd.c:436:19: error: initialization from incompatible pointer type [-Werror=incompatible-pointer-types]
.mw_set_trans = amd_ntb_mw_set_trans,
^~~~~~~~~~~~~~~~~~~~
drivers/ntb/hw/amd/ntb_hw_amd.c:436:19: note: (near initialization for 'amd_ntb_ops.mw_set_trans')
drivers/ntb/hw/amd/ntb_hw_amd.c: In function 'amd_init_ntb':
drivers/ntb/hw/amd/ntb_hw_amd.c:880:7: error: 'NTB_TOPO_PRI' undeclared (first use in this function)
case NTB_TOPO_PRI:
^~~~~~~~~~~~
drivers/ntb/hw/amd/ntb_hw_amd.c:881:7: error: 'NTB_TOPO_SEC' undeclared (first use in this function)
case NTB_TOPO_SEC:
^~~~~~~~~~~~
drivers/ntb/hw/amd/ntb_hw_amd.c: In function 'amd_get_topo':
drivers/ntb/hw/amd/ntb_hw_amd.c:915:10: error: 'NTB_TOPO_SEC' undeclared (first use in this function)
return NTB_TOPO_SEC;
^~~~~~~~~~~~
drivers/ntb/hw/amd/ntb_hw_amd.c:917:10: error: 'NTB_TOPO_PRI' undeclared (first use in this function)
return NTB_TOPO_PRI;
^~~~~~~~~~~~
drivers/ntb/hw/amd/ntb_hw_amd.c:918:1: warning: control reaches end of non-void function [-Wreturn-type]
}
^
cc1: some warnings being treated as errors
vim +/ntb_mw_clear_trans +685 drivers/ntb/ntb_transport.c
e26a5843 Allen Hubbe 2015-04-09 679 struct ntb_transport_mw *mw = &nt->mw_vec[num_mw];
e26a5843 Allen Hubbe 2015-04-09 680 struct pci_dev *pdev = nt->ndev->pdev;
b77b2637 Jon Mason 2013-02-01 681
b77b2637 Jon Mason 2013-02-01 682 if (!mw->virt_addr)
b77b2637 Jon Mason 2013-02-01 683 return;
b77b2637 Jon Mason 2013-02-01 684
e26a5843 Allen Hubbe 2015-04-09 @685 ntb_mw_clear_trans(nt->ndev, num_mw);
e26a5843 Allen Hubbe 2015-04-09 686 dma_free_coherent(&pdev->dev, mw->buff_size,
e26a5843 Allen Hubbe 2015-04-09 687 mw->virt_addr, mw->dma_addr);
e26a5843 Allen Hubbe 2015-04-09 688 mw->xlat_size = 0;
e26a5843 Allen Hubbe 2015-04-09 689 mw->buff_size = 0;
b77b2637 Jon Mason 2013-02-01 690 mw->virt_addr = NULL;
b77b2637 Jon Mason 2013-02-01 691 }
b77b2637 Jon Mason 2013-02-01 692
e26a5843 Allen Hubbe 2015-04-09 693 static int ntb_set_mw(struct ntb_transport_ctx *nt, int num_mw,
8c9edf63 Allen Hubbe 2015-07-13 694 resource_size_t size)
fce8a7bb Jon Mason 2012-11-16 695 {
e26a5843 Allen Hubbe 2015-04-09 696 struct ntb_transport_mw *mw = &nt->mw_vec[num_mw];
e26a5843 Allen Hubbe 2015-04-09 697 struct pci_dev *pdev = nt->ndev->pdev;
8c9edf63 Allen Hubbe 2015-07-13 698 size_t xlat_size, buff_size;
e26a5843 Allen Hubbe 2015-04-09 699 int rc;
e26a5843 Allen Hubbe 2015-04-09 700
8c9edf63 Allen Hubbe 2015-07-13 701 if (!size)
8c9edf63 Allen Hubbe 2015-07-13 702 return -EINVAL;
8c9edf63 Allen Hubbe 2015-07-13 703
e26a5843 Allen Hubbe 2015-04-09 704 xlat_size = round_up(size, mw->xlat_align_size);
e26a5843 Allen Hubbe 2015-04-09 705 buff_size = round_up(size, mw->xlat_align);
fce8a7bb Jon Mason 2012-11-16 706
b77b2637 Jon Mason 2013-02-01 707 /* No need to re-setup */
e26a5843 Allen Hubbe 2015-04-09 708 if (mw->xlat_size == xlat_size)
b77b2637 Jon Mason 2013-02-01 709 return 0;
b77b2637 Jon Mason 2013-02-01 710
e26a5843 Allen Hubbe 2015-04-09 711 if (mw->buff_size)
b77b2637 Jon Mason 2013-02-01 712 ntb_free_mw(nt, num_mw);
b77b2637 Jon Mason 2013-02-01 713
e26a5843 Allen Hubbe 2015-04-09 714 /* Alloc memory for receiving data. Must be aligned */
e26a5843 Allen Hubbe 2015-04-09 715 mw->xlat_size = xlat_size;
e26a5843 Allen Hubbe 2015-04-09 716 mw->buff_size = buff_size;
fce8a7bb Jon Mason 2012-11-16 717
e26a5843 Allen Hubbe 2015-04-09 718 mw->virt_addr = dma_alloc_coherent(&pdev->dev, buff_size,
e26a5843 Allen Hubbe 2015-04-09 719 &mw->dma_addr, GFP_KERNEL);
fce8a7bb Jon Mason 2012-11-16 720 if (!mw->virt_addr) {
e26a5843 Allen Hubbe 2015-04-09 721 mw->xlat_size = 0;
e26a5843 Allen Hubbe 2015-04-09 722 mw->buff_size = 0;
8c9edf63 Allen Hubbe 2015-07-13 723 dev_err(&pdev->dev, "Unable to alloc MW buff of size %zu\n",
e26a5843 Allen Hubbe 2015-04-09 724 buff_size);
fce8a7bb Jon Mason 2012-11-16 725 return -ENOMEM;
fce8a7bb Jon Mason 2012-11-16 726 }
fce8a7bb Jon Mason 2012-11-16 727
3cc5ba19 Dave Jiang 2014-08-28 728 /*
3cc5ba19 Dave Jiang 2014-08-28 729 * we must ensure that the memory address allocated is BAR size
3cc5ba19 Dave Jiang 2014-08-28 730 * aligned in order for the XLAT register to take the value. This
3cc5ba19 Dave Jiang 2014-08-28 731 * is a requirement of the hardware. It is recommended to setup CMA
3cc5ba19 Dave Jiang 2014-08-28 732 * for BAR sizes equal or greater than 4MB.
3cc5ba19 Dave Jiang 2014-08-28 733 */
e26a5843 Allen Hubbe 2015-04-09 734 if (!IS_ALIGNED(mw->dma_addr, mw->xlat_align)) {
e26a5843 Allen Hubbe 2015-04-09 735 dev_err(&pdev->dev, "DMA memory %pad is not aligned\n",
3cc5ba19 Dave Jiang 2014-08-28 736 &mw->dma_addr);
3cc5ba19 Dave Jiang 2014-08-28 737 ntb_free_mw(nt, num_mw);
3cc5ba19 Dave Jiang 2014-08-28 738 return -ENOMEM;
3cc5ba19 Dave Jiang 2014-08-28 739 }
3cc5ba19 Dave Jiang 2014-08-28 740
fce8a7bb Jon Mason 2012-11-16 741 /* Notify HW the memory location of the receive buffer */
e26a5843 Allen Hubbe 2015-04-09 @742 rc = ntb_mw_set_trans(nt->ndev, num_mw, mw->dma_addr, mw->xlat_size);
e26a5843 Allen Hubbe 2015-04-09 743 if (rc) {
e26a5843 Allen Hubbe 2015-04-09 744 dev_err(&pdev->dev, "Unable to set mw%d translation", num_mw);
e26a5843 Allen Hubbe 2015-04-09 745 ntb_free_mw(nt, num_mw);
:::::: The code at line 685 was first introduced by commit
:::::: e26a5843f7f5014ae4460030ca4de029a3ac35d3 NTB: Split ntb_hw_intel and ntb_transport drivers
:::::: TO: Allen Hubbe <Allen.Hubbe@emc.com>
:::::: CC: Jon Mason <jdmason@kudzu.us>
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 23264 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2016-12-04 0:13 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-12-04 0:04 [PATCH 03/22] NTB: Alter NTB API to support both inbound and outbound MW based interfaces Allen Hubbe
2016-12-04 0:04 ` Allen Hubbe
-- strict thread matches above, loose matches on Subject: below --
2016-11-29 17:15 [PATCH 00/22] NTB: Alter kernel API to support multi-port devices Serge Semin
2016-11-29 17:15 ` [PATCH 03/22] NTB: Alter NTB API to support both inbound and outbound MW based interfaces Serge Semin
2016-11-30 18:54 ` kbuild test robot
2016-11-30 19:46 ` kbuild test robot
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.