* Re: modify srq pyverbs test (new?) failing for the rxe driver [not found] <401221fd-b41d-4db9-be22-b1af17b0d456@gmail.com> @ 2023-10-24 17:47 ` Bob Pearson 2023-10-25 11:29 ` Edward Srouji 0 siblings, 1 reply; 4+ messages in thread From: Bob Pearson @ 2023-10-24 17:47 UTC (permalink / raw) To: edwards, jgg, linux-rdma On 10/23/23 17:47, Bob Pearson wrote: > Edward, > > There is a test which may be new since this failure didn't happen > before. Modify srq tries to set max_wr to 4094 but the rxe driver > rounds this up to 2^n-1 or 4095. My understanding of the IBA > spec is that queue sizes can be set larger than requested. > Also this same test tries to change max_sge using the same > MAX_WR mask bit. There is no mention (as far as I can recall) > in the IBA spec of being able to change the max_sge setting. > > Is this really the correct behavior? > > Bob Pearson Just for reference 11.2.3.3 MODIFY SHARED RECEIVE QUEUE Description: Modifies the attributes of an SRQ for the specified HCA. If any of the modify attributes are invalid, none of the attributes shall be modified. Input Modifiers: •HCA handle. •SRQ handle. •The SRQ attributes to modify and their new values. The SRQ at- tributes that can be modified after the SRQ has been created are: •The maximum number of outstanding Work Requests the Consumer expects to submit to the Shared Receive Queue, if resizing of the SRQ is supported by the HCA. [It does *not* include max_sge.] •SRQ Limit. If the SRQ Limit is greater than zero, then it shall be armed upon returning from this verb. Output Modifiers: •The actual number of outstanding Work Requests supported on the Shared Receive Queue. If an error is not returned, this is guaranteed to be greater than or equal to the number requested. (This may require the Consumer to increase the size of the CQ.) [Unfortunately the rdma verbs API does not support returning the new value of max_wr. You have to call ib_query_srq to get the new value which does *not* have to equal the requested size.] •Verb Results: •Operation completed successfully. •Insufficient resources to complete request. •Invalid HCA handle. •Invalid SRQ handle. •SRQ is in the Error State. •HCA does not support resizing SRQ. •Maximum number of Work Requests requested exceeds HCA capability. •SRQ Limit exceeds maximum number of Work Requests al- lowed on the SRQ. •More outstanding entries on WQ than size specified. •HCA does not support SRQ. [The test_modify_srq just seems incorrect.] ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: modify srq pyverbs test (new?) failing for the rxe driver 2023-10-24 17:47 ` modify srq pyverbs test (new?) failing for the rxe driver Bob Pearson @ 2023-10-25 11:29 ` Edward Srouji 2023-10-25 15:49 ` Bob Pearson 2023-10-26 2:12 ` Bob Pearson 0 siblings, 2 replies; 4+ messages in thread From: Edward Srouji @ 2023-10-25 11:29 UTC (permalink / raw) To: Bob Pearson, jgg, linux-rdma You're right. I've patched the test and pushed it Upstream (see PR link below [1]). Please notify me if you're still having an issue after applying the patch. On 10/24/2023 8:47 PM, Bob Pearson wrote: > External email: Use caution opening links or attachments > > > On 10/23/23 17:47, Bob Pearson wrote: >> Edward, >> >> There is a test which may be new since this failure didn't happen >> before. Modify srq tries to set max_wr to 4094 but the rxe driver >> rounds this up to 2^n-1 or 4095. My understanding of the IBA >> spec is that queue sizes can be set larger than requested. >> Also this same test tries to change max_sge using the same >> MAX_WR mask bit. There is no mention (as far as I can recall) >> in the IBA spec of being able to change the max_sge setting. >> >> Is this really the correct behavior? >> >> Bob Pearson > > Just for reference > > 11.2.3.3 MODIFY SHARED RECEIVE QUEUE > Description: > Modifies the attributes of an SRQ for the specified HCA. > If any of the modify attributes are invalid, none of the attributes > shall be modified. > Input Modifiers: > •HCA handle. > •SRQ handle. > •The SRQ attributes to modify and their new values. The SRQ at- > tributes that can be modified after the SRQ has been created are: > •The maximum number of outstanding Work Requests the > Consumer expects to submit to the Shared Receive Queue, if > resizing of the SRQ is supported by the HCA. > > [It does *not* include max_sge.] > > •SRQ Limit. If the SRQ Limit is greater than zero, then it shall > be armed upon returning from this verb. > Output Modifiers: > •The actual number of outstanding Work Requests supported on > the Shared Receive Queue. If an error is not returned, this is > guaranteed to be greater than or equal to the number requested. > (This may require the Consumer to increase the size of the CQ.) > > [Unfortunately the rdma verbs API does not support returning the > new value of max_wr. You have to call ib_query_srq to get the new > value which does *not* have to equal the requested size.] > > •Verb Results: > •Operation completed successfully. > •Insufficient resources to complete request. > •Invalid HCA handle. > •Invalid SRQ handle. > •SRQ is in the Error State. > •HCA does not support resizing SRQ. > •Maximum number of Work Requests requested exceeds HCA > capability. > •SRQ Limit exceeds maximum number of Work Requests al- > lowed on the SRQ. > •More outstanding entries on WQ than size specified. > •HCA does not support SRQ. > > [The test_modify_srq just seems incorrect.] [1] https://github.com/linux-rdma/rdma-core/pull/1402 Thanks. ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: modify srq pyverbs test (new?) failing for the rxe driver 2023-10-25 11:29 ` Edward Srouji @ 2023-10-25 15:49 ` Bob Pearson 2023-10-26 2:12 ` Bob Pearson 1 sibling, 0 replies; 4+ messages in thread From: Bob Pearson @ 2023-10-25 15:49 UTC (permalink / raw) To: Edward Srouji, jgg, linux-rdma On 10/25/23 06:29, Edward Srouji wrote: > You're right. > I've patched the test and pushed it Upstream (see PR link below [1]). > Please notify me if you're still having an issue after applying the patch. Will do. Thanks! Bob > > On 10/24/2023 8:47 PM, Bob Pearson wrote: >> External email: Use caution opening links or attachments >> >> >> On 10/23/23 17:47, Bob Pearson wrote: >>> Edward, >>> >>> There is a test which may be new since this failure didn't happen >>> before. Modify srq tries to set max_wr to 4094 but the rxe driver >>> rounds this up to 2^n-1 or 4095. My understanding of the IBA >>> spec is that queue sizes can be set larger than requested. >>> Also this same test tries to change max_sge using the same >>> MAX_WR mask bit. There is no mention (as far as I can recall) >>> in the IBA spec of being able to change the max_sge setting. >>> >>> Is this really the correct behavior? >>> >>> Bob Pearson ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: modify srq pyverbs test (new?) failing for the rxe driver 2023-10-25 11:29 ` Edward Srouji 2023-10-25 15:49 ` Bob Pearson @ 2023-10-26 2:12 ` Bob Pearson 1 sibling, 0 replies; 4+ messages in thread From: Bob Pearson @ 2023-10-26 2:12 UTC (permalink / raw) To: Edward Srouji, jgg, linux-rdma On 10/25/23 06:29, Edward Srouji wrote: > You're right. > I've patched the test and pushed it Upstream (see PR link below [1]). > Please notify me if you're still having an issue after applying the patch. > Works fine now. Thanks. Bob ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2023-10-26 2:12 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
[not found] <401221fd-b41d-4db9-be22-b1af17b0d456@gmail.com>
2023-10-24 17:47 ` modify srq pyverbs test (new?) failing for the rxe driver Bob Pearson
2023-10-25 11:29 ` Edward Srouji
2023-10-25 15:49 ` Bob Pearson
2023-10-26 2:12 ` Bob Pearson
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.