[ISSUE] The time cost of IOSQE_IO_LINK

Carter Li 李通洲
2020-02-12T17:05:04Z

Hi everyone,

IOSQE_IO_LINK seems to have very high cost, even greater then io_uring_enter syscall.

Test code attached below. The program completes after getting 100000000 cqes.

$ gcc test.c -luring -o test0 -g -O3 -DUSE_LINK=0
$ time ./test0
USE_LINK: 0, count: 100000000, submit_count: 1562500
0.99user 9.99system 0:11.02elapsed 99%CPU (0avgtext+0avgdata 1608maxresident)k
0inputs+0outputs (0major+72minor)pagefaults 0swaps

$ gcc test.c -luring -o test1 -g -O3 -DUSE_LINK=1
$ time ./test1
USE_LINK: 1, count: 100000110, submit_count: 799584
0.83user 19.21system 0:20.90elapsed 95%CPU (0avgtext+0avgdata 1632maxresident)k
0inputs+0outputs (0major+72minor)pagefaults 0swaps

As you can see, the `-DUSE_LINK=1` version emits only about half io_uring_submit calls
of the other version, but takes twice as long. That makes IOSQE_IO_LINK almost useless,
please have a check.

Thanks,
Carter Li

---

#include <liburing.h>
#include <stdio.h>

#define ENTRY_SIZE 128
#define MAX_COUNT 100000000

#ifndef USE_LINK
# define USE_LINK 0
#endif

int main(void) {
struct io_uring ring;
unsigned i, head;
unsigned count = 0, cqe_count = 0, submit_count = 0;
struct io_uring_sqe *sqe;
struct io_uring_cqe *cqe;

if (io_uring_queue_init(ENTRY_SIZE, &ring, 0) < 0) {
return -1;
}

for (i = 0; i < ENTRY_SIZE / 2; ++i) {
sqe = io_uring_get_sqe(&ring);
io_uring_prep_nop(sqe);
#if USE_LINK
io_uring_sqe_set_flags(sqe, IOSQE_IO_LINK);
sqe = io_uring_get_sqe(&ring);
io_uring_prep_nop(sqe);
sqe->user_data = 1;
#endif
}

while (count < MAX_COUNT) {
io_uring_submit_and_wait(&ring, 1);
++submit_count;

io_uring_for_each_cqe(&ring, head, cqe) {
++cqe_count;
++count;
#if USE_LINK
if (cqe->user_data) {
sqe = io_uring_get_sqe(&ring);
io_uring_prep_nop(sqe);
io_uring_sqe_set_flags(sqe, IOSQE_IO_LINK);
sqe = io_uring_get_sqe(&ring);
io_uring_prep_nop(sqe);
sqe->user_data = 1;
}
#else
sqe = io_uring_get_sqe(&ring);
io_uring_prep_nop(sqe);
sqe->user_data = !cqe->user_data;
#endif
}
io_uring_cq_advance(&ring, cqe_count);
cqe_count = 0;
}

printf("USE_LINK: %d, count: %u, submit_count: %u\n\n",
USE_LINK, count, submit_count);
}