Relationship between H/W ring and S/W ring

Relationship between H/W ring and S/W ring

[Gyumin]

Hi

I`m reading the ixgbe code especially about H/W ring and S/W ring. Is 
the relationship between H/W ring and S/W ring one-to-one mapping?
As far as I know, H/W ring size is determined in the code(hard coded) 
while S/W ring size is determined in port configuration time.
In the ixgbe_rx_alloc_bufs function, H/W ring header address and packet 
address indicate the DMA address of S/W ring's mbuf. I understand it 
means that the relationship between the H/W ring and S/W ring is 
one-to-one mapping. For example, if the size of H/W ring is greater than 
the size of  S/W ring then some portion of H/W ring is unused. Is it 
correct?

Thanks

Re: Relationship between H/W ring and S/W ring

[Bruce Richardson]

On Thu, Oct 30, 2014 at 04:32:16PM +0900, Gyumin wrote:
> Hi
> 
> I`m reading the ixgbe code especially about H/W ring and S/W ring. Is the
> relationship between H/W ring and S/W ring one-to-one mapping?
> As far as I know, H/W ring size is determined in the code(hard coded) while
> S/W ring size is determined in port configuration time.
> In the ixgbe_rx_alloc_bufs function, H/W ring header address and packet
> address indicate the DMA address of S/W ring's mbuf. I understand it means
> that the relationship between the H/W ring and S/W ring is one-to-one
> mapping. For example, if the size of H/W ring is greater than the size of
> S/W ring then some portion of H/W ring is unused. Is it correct?
> 
> Thanks

Hi,

Yes, there is a 1:1 mapping between the hardware and software ring entries, and both are sized depending on the configuration parameters passed to the ring setup APIs. As you state, the HW ring contains the DMA addresses of the packet buffers, while the sw_ring contains the pointers to the original mbufs. The two rings are always kept in sync in the code.

/Bruce

Re: Relationship between H/W ring and S/W ring

[Gyumin]

Thanks Bruce.

I also agree with that the size of the S/W ring depends on the 
configuration parameters because the size of the S/W ring is 
/sizeof(struct igb_rx_entry) * len/ in the ixgbe_dev_rx_queue_setup 
function. H/W ring is also allocated in the same function by using the 
ring_dma_zone_reserve function, and its size is RX_RING_SZ. I don't 
think the RX_RING_SZ is configurable but it is fixed value. Is there any 
other code configuring the size of H/W ring?

2014-10-30 오후 6:55에 Bruce Richardson 이(가) 쓴 글:
> On Thu, Oct 30, 2014 at 04:32:16PM +0900, Gyumin wrote:
>> Hi
>>
>> I`m reading the ixgbe code especially about H/W ring and S/W ring. Is the
>> relationship between H/W ring and S/W ring one-to-one mapping?
>> As far as I know, H/W ring size is determined in the code(hard coded) while
>> S/W ring size is determined in port configuration time.
>> In the ixgbe_rx_alloc_bufs function, H/W ring header address and packet
>> address indicate the DMA address of S/W ring's mbuf. I understand it means
>> that the relationship between the H/W ring and S/W ring is one-to-one
>> mapping. For example, if the size of H/W ring is greater than the size of
>> S/W ring then some portion of H/W ring is unused. Is it correct?
>>
>> Thanks
> Hi,
>
> Yes, there is a 1:1 mapping between the hardware and software ring entries, and both are sized depending on the configuration parameters passed to the ring setup APIs. As you state, the HW ring contains the DMA addresses of the packet buffers, while the sw_ring contains the pointers to the original mbufs. The two rings are always kept in sync in the code.
>
> /Bruce
>

Re: Relationship between H/W ring and S/W ring

[Bruce Richardson]

On Fri, Oct 31, 2014 at 09:51:56AM +0900, Gyumin wrote:
> Thanks Bruce.
> 
> I also agree with that the size of the S/W ring depends on the configuration
> parameters because the size of the S/W ring is /sizeof(struct igb_rx_entry)
> * len/ in the ixgbe_dev_rx_queue_setup function. H/W ring is also allocated
> in the same function by using the ring_dma_zone_reserve function, and its
> size is RX_RING_SZ. I don't think the RX_RING_SZ is configurable but it is
> fixed value. Is there any other code configuring the size of H/W ring?
> 

Indeed you are right, my mistake. The comment indicates that we always reserve
the memory to be the maximum size so that we can resize the rings easier later
on.
In terms of runtime usage, though, if you look a the RX functions, you can see
that the two rings are always kept in sync. For example, looking at 
ixgbe_rxq_rearm in ixgbe_rxtx_vec.c, you will see that rxdp and rxep values
both start at offset "rxq->rxrearm_start" at the top of the function, and that
in the main rearm loop, both are incremented twice each iteration (rxep += 2 in
the for statment itself, and two rxdp++'s are used in the last two lines of the
loop body).

Regards,
/Bruce

> 2014-10-30 오후 6:55에 Bruce Richardson 이(가) 쓴 글:
> >On Thu, Oct 30, 2014 at 04:32:16PM +0900, Gyumin wrote:
> >>Hi
> >>
> >>I`m reading the ixgbe code especially about H/W ring and S/W ring. Is the
> >>relationship between H/W ring and S/W ring one-to-one mapping?
> >>As far as I know, H/W ring size is determined in the code(hard coded) while
> >>S/W ring size is determined in port configuration time.
> >>In the ixgbe_rx_alloc_bufs function, H/W ring header address and packet
> >>address indicate the DMA address of S/W ring's mbuf. I understand it means
> >>that the relationship between the H/W ring and S/W ring is one-to-one
> >>mapping. For example, if the size of H/W ring is greater than the size of
> >>S/W ring then some portion of H/W ring is unused. Is it correct?
> >>
> >>Thanks
> >Hi,
> >
> >Yes, there is a 1:1 mapping between the hardware and software ring entries, and both are sized depending on the configuration parameters passed to the ring setup APIs. As you state, the HW ring contains the DMA addresses of the packet buffers, while the sw_ring contains the pointers to the original mbufs. The two rings are always kept in sync in the code.
> >
> >/Bruce
> >
> 

Re: Relationship between H/W ring and S/W ring

[GyuminHwang]

Now I understand.
Thanks Bruce.

2014년 10월 31일 19:08에 Bruce Richardson 이(가) 쓴 글:
> On Fri, Oct 31, 2014 at 09:51:56AM +0900, Gyumin wrote:
>> Thanks Bruce.
>>
>> I also agree with that the size of the S/W ring depends on the configuration
>> parameters because the size of the S/W ring is /sizeof(struct igb_rx_entry)
>> * len/ in the ixgbe_dev_rx_queue_setup function. H/W ring is also allocated
>> in the same function by using the ring_dma_zone_reserve function, and its
>> size is RX_RING_SZ. I don't think the RX_RING_SZ is configurable but it is
>> fixed value. Is there any other code configuring the size of H/W ring?
>>
> Indeed you are right, my mistake. The comment indicates that we always reserve
> the memory to be the maximum size so that we can resize the rings easier later
> on.
> In terms of runtime usage, though, if you look a the RX functions, you can see
> that the two rings are always kept in sync. For example, looking at
> ixgbe_rxq_rearm in ixgbe_rxtx_vec.c, you will see that rxdp and rxep values
> both start at offset "rxq->rxrearm_start" at the top of the function, and that
> in the main rearm loop, both are incremented twice each iteration (rxep += 2 in
> the for statment itself, and two rxdp++'s are used in the last two lines of the
> loop body).
>
> Regards,
> /Bruce
>
>> 2014-10-30 오후 6:55에 Bruce Richardson 이(가) 쓴 글:
>>> On Thu, Oct 30, 2014 at 04:32:16PM +0900, Gyumin wrote:
>>>> Hi
>>>>
>>>> I`m reading the ixgbe code especially about H/W ring and S/W ring. Is the
>>>> relationship between H/W ring and S/W ring one-to-one mapping?
>>>> As far as I know, H/W ring size is determined in the code(hard coded) while
>>>> S/W ring size is determined in port configuration time.
>>>> In the ixgbe_rx_alloc_bufs function, H/W ring header address and packet
>>>> address indicate the DMA address of S/W ring's mbuf. I understand it means
>>>> that the relationship between the H/W ring and S/W ring is one-to-one
>>>> mapping. For example, if the size of H/W ring is greater than the size of
>>>> S/W ring then some portion of H/W ring is unused. Is it correct?
>>>>
>>>> Thanks
>>> Hi,
>>>
>>> Yes, there is a 1:1 mapping between the hardware and software ring entries, and both are sized depending on the configuration parameters passed to the ring setup APIs. As you state, the HW ring contains the DMA addresses of the packet buffers, while the sw_ring contains the pointers to the original mbufs. The two rings are always kept in sync in the code.
>>>
>>> /Bruce
>>>