[PATCH] KVM: selftests: Make sure kvm_create_max_vcpus test won't hit RLIMIT_NOFILE

[PATCH] KVM: selftests: Make sure kvm_create_max_vcpus test won't hit RLIMIT_NOFILE

[Vitaly Kuznetsov]

With the elevated 'KVM_CAP_MAX_VCPUS' value kvm_create_max_vcpus test
may hit RLIMIT_NOFILE limits:

 # ./kvm_create_max_vcpus
 KVM_CAP_MAX_VCPU_ID: 4096
 KVM_CAP_MAX_VCPUS: 1024
 Testing creating 1024 vCPUs, with IDs 0...1023.
 /dev/kvm not available (errno: 24), skipping test

Adjust RLIMIT_NOFILE limits to make sure KVM_CAP_MAX_VCPUS fds can be
opened. Note, raising hard limit ('rlim_max') requires CAP_SYS_RESOURCE
capability which is generally not needed to run kvm selftests (but without
raising the limit the test is doomed to fail anyway).

Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
---
 .../selftests/kvm/kvm_create_max_vcpus.c      | 22 +++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/tools/testing/selftests/kvm/kvm_create_max_vcpus.c b/tools/testing/selftests/kvm/kvm_create_max_vcpus.c
index f968dfd4ee88..19198477a10e 100644
--- a/tools/testing/selftests/kvm/kvm_create_max_vcpus.c
+++ b/tools/testing/selftests/kvm/kvm_create_max_vcpus.c
@@ -12,6 +12,7 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
+#include <sys/resource.h>
 
 #include "test_util.h"
 
@@ -19,6 +20,9 @@
 #include "asm/kvm.h"
 #include "linux/kvm.h"
 
+/* 'Safe' number of open file descriptors in addition to vCPU fds needed */
+#define NOFD 16
+
 void test_vcpu_creation(int first_vcpu_id, int num_vcpus)
 {
 	struct kvm_vm *vm;
@@ -40,10 +44,28 @@ int main(int argc, char *argv[])
 {
 	int kvm_max_vcpu_id = kvm_check_cap(KVM_CAP_MAX_VCPU_ID);
 	int kvm_max_vcpus = kvm_check_cap(KVM_CAP_MAX_VCPUS);
+	struct rlimit rl;
 
 	pr_info("KVM_CAP_MAX_VCPU_ID: %d\n", kvm_max_vcpu_id);
 	pr_info("KVM_CAP_MAX_VCPUS: %d\n", kvm_max_vcpus);
 
+	/*
+	 * Creating KVM_CAP_MAX_VCPUS vCPUs require KVM_CAP_MAX_VCPUS open
+	 * file decriptors.
+	 */
+	TEST_ASSERT(!getrlimit(RLIMIT_NOFILE, &rl),
+		    "getrlimit() failed (errno: %d)", errno);
+
+	if (kvm_max_vcpus > rl.rlim_cur - NOFD) {
+		rl.rlim_cur = kvm_max_vcpus + NOFD;
+
+		if (kvm_max_vcpus > rl.rlim_max - NOFD)
+			rl.rlim_max = kvm_max_vcpus + NOFD;
+
+		TEST_ASSERT(!setrlimit(RLIMIT_NOFILE, &rl),
+			    "setrlimit() failed (errno: %d)", errno);
+	}
+
 	/*
 	 * Upstream KVM prior to 4.8 does not support KVM_CAP_MAX_VCPU_ID.
 	 * Userspace is supposed to use KVM_CAP_MAX_VCPUS as the maximum ID
-- 
2.33.1

Re: [PATCH] KVM: selftests: Make sure kvm_create_max_vcpus test won't hit RLIMIT_NOFILE

[Sean Christopherson]

On Mon, Nov 22, 2021, Vitaly Kuznetsov wrote:
> With the elevated 'KVM_CAP_MAX_VCPUS' value kvm_create_max_vcpus test
> may hit RLIMIT_NOFILE limits:
> 
>  # ./kvm_create_max_vcpus
>  KVM_CAP_MAX_VCPU_ID: 4096
>  KVM_CAP_MAX_VCPUS: 1024
>  Testing creating 1024 vCPUs, with IDs 0...1023.
>  /dev/kvm not available (errno: 24), skipping test
> 
> Adjust RLIMIT_NOFILE limits to make sure KVM_CAP_MAX_VCPUS fds can be
> opened. Note, raising hard limit ('rlim_max') requires CAP_SYS_RESOURCE
> capability which is generally not needed to run kvm selftests (but without
> raising the limit the test is doomed to fail anyway).
> 
> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> ---
>  .../selftests/kvm/kvm_create_max_vcpus.c      | 22 +++++++++++++++++++
>  1 file changed, 22 insertions(+)
> 
> diff --git a/tools/testing/selftests/kvm/kvm_create_max_vcpus.c b/tools/testing/selftests/kvm/kvm_create_max_vcpus.c
> index f968dfd4ee88..19198477a10e 100644
> --- a/tools/testing/selftests/kvm/kvm_create_max_vcpus.c
> +++ b/tools/testing/selftests/kvm/kvm_create_max_vcpus.c
> @@ -12,6 +12,7 @@
>  #include <stdio.h>
>  #include <stdlib.h>
>  #include <string.h>
> +#include <sys/resource.h>
>  
>  #include "test_util.h"
>  
> @@ -19,6 +20,9 @@
>  #include "asm/kvm.h"
>  #include "linux/kvm.h"
>  
> +/* 'Safe' number of open file descriptors in addition to vCPU fds needed */
> +#define NOFD 16

Any reason not to make this "buffer" extra large, e.g. 100+ to avoid having to
debug this issue again in the future?

> +
>  void test_vcpu_creation(int first_vcpu_id, int num_vcpus)
>  {
>  	struct kvm_vm *vm;
> @@ -40,10 +44,28 @@ int main(int argc, char *argv[])
>  {
>  	int kvm_max_vcpu_id = kvm_check_cap(KVM_CAP_MAX_VCPU_ID);
>  	int kvm_max_vcpus = kvm_check_cap(KVM_CAP_MAX_VCPUS);

Rather than a separate define that's hard to describe succintly, what about:

	int nr_fds_wanted = kvm_max_vcpus + <arbitrary number>

and then the body becomes

	if (nr_fds_wanted > rl.rlim_cur) {
		rl.rlim_cur = nr_fds_wanted;
		rl.rlim_max = max(rl.rlim_max, nr_fds_wanted);

		...
	}

> +	struct rlimit rl;
>  
>  	pr_info("KVM_CAP_MAX_VCPU_ID: %d\n", kvm_max_vcpu_id);
>  	pr_info("KVM_CAP_MAX_VCPUS: %d\n", kvm_max_vcpus);
>  
> +	/*
> +	 * Creating KVM_CAP_MAX_VCPUS vCPUs require KVM_CAP_MAX_VCPUS open
> +	 * file decriptors.
> +	 */
> +	TEST_ASSERT(!getrlimit(RLIMIT_NOFILE, &rl),
> +		    "getrlimit() failed (errno: %d)", errno);

And strerror() output too?

> +
> +	if (kvm_max_vcpus > rl.rlim_cur - NOFD) {
> +		rl.rlim_cur = kvm_max_vcpus + NOFD;
> +
> +		if (kvm_max_vcpus > rl.rlim_max - NOFD)
> +			rl.rlim_max = kvm_max_vcpus + NOFD;
> +
> +		TEST_ASSERT(!setrlimit(RLIMIT_NOFILE, &rl),
> +			    "setrlimit() failed (errno: %d)", errno);
> +	}
> +
>  	/*
>  	 * Upstream KVM prior to 4.8 does not support KVM_CAP_MAX_VCPU_ID.
>  	 * Userspace is supposed to use KVM_CAP_MAX_VCPUS as the maximum ID
> -- 
> 2.33.1
> 

Re: [PATCH] KVM: selftests: Make sure kvm_create_max_vcpus test won't hit RLIMIT_NOFILE

[Vitaly Kuznetsov]

Sean Christopherson <seanjc@google.com> writes:

> On Mon, Nov 22, 2021, Vitaly Kuznetsov wrote:
>> With the elevated 'KVM_CAP_MAX_VCPUS' value kvm_create_max_vcpus test
>> may hit RLIMIT_NOFILE limits:
>> 
>>  # ./kvm_create_max_vcpus
>>  KVM_CAP_MAX_VCPU_ID: 4096
>>  KVM_CAP_MAX_VCPUS: 1024
>>  Testing creating 1024 vCPUs, with IDs 0...1023.
>>  /dev/kvm not available (errno: 24), skipping test
>> 
>> Adjust RLIMIT_NOFILE limits to make sure KVM_CAP_MAX_VCPUS fds can be
>> opened. Note, raising hard limit ('rlim_max') requires CAP_SYS_RESOURCE
>> capability which is generally not needed to run kvm selftests (but without
>> raising the limit the test is doomed to fail anyway).
>> 
>> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>> ---
>>  .../selftests/kvm/kvm_create_max_vcpus.c      | 22 +++++++++++++++++++
>>  1 file changed, 22 insertions(+)
>> 
>> diff --git a/tools/testing/selftests/kvm/kvm_create_max_vcpus.c b/tools/testing/selftests/kvm/kvm_create_max_vcpus.c
>> index f968dfd4ee88..19198477a10e 100644
>> --- a/tools/testing/selftests/kvm/kvm_create_max_vcpus.c
>> +++ b/tools/testing/selftests/kvm/kvm_create_max_vcpus.c
>> @@ -12,6 +12,7 @@
>>  #include <stdio.h>
>>  #include <stdlib.h>
>>  #include <string.h>
>> +#include <sys/resource.h>
>>  
>>  #include "test_util.h"
>>  
>> @@ -19,6 +20,9 @@
>>  #include "asm/kvm.h"
>>  #include "linux/kvm.h"
>>  
>> +/* 'Safe' number of open file descriptors in addition to vCPU fds needed */
>> +#define NOFD 16
>
> Any reason not to make this "buffer" extra large, e.g. 100+ to avoid having to
> debug this issue again in the future?
>

No, not really. We could've avoided this ambiguity completely by
checking how many fds are already open but all methods I can think of
are 'too much'. In my testing I needed around 10 so I put '16' but '100'
is even better.

>> +
>>  void test_vcpu_creation(int first_vcpu_id, int num_vcpus)
>>  {
>>  	struct kvm_vm *vm;
>> @@ -40,10 +44,28 @@ int main(int argc, char *argv[])
>>  {
>>  	int kvm_max_vcpu_id = kvm_check_cap(KVM_CAP_MAX_VCPU_ID);
>>  	int kvm_max_vcpus = kvm_check_cap(KVM_CAP_MAX_VCPUS);
>
> Rather than a separate define that's hard to describe succintly, what about:
>
> 	int nr_fds_wanted = kvm_max_vcpus + <arbitrary number>
>
> and then the body becomes
>
> 	if (nr_fds_wanted > rl.rlim_cur) {
> 		rl.rlim_cur = nr_fds_wanted;
> 		rl.rlim_max = max(rl.rlim_max, nr_fds_wanted);
>
> 		...
> 	}

Sure but a "succinct" comment will still be needed, either near the
'NOFD' define or above 'int nr_fds_wanted' :-)

>
>> +	struct rlimit rl;
>>  
>>  	pr_info("KVM_CAP_MAX_VCPU_ID: %d\n", kvm_max_vcpu_id);
>>  	pr_info("KVM_CAP_MAX_VCPUS: %d\n", kvm_max_vcpus);
>>  
>> +	/*
>> +	 * Creating KVM_CAP_MAX_VCPUS vCPUs require KVM_CAP_MAX_VCPUS open
>> +	 * file decriptors.
>> +	 */
>> +	TEST_ASSERT(!getrlimit(RLIMIT_NOFILE, &rl),
>> +		    "getrlimit() failed (errno: %d)", errno);
>
> And strerror() output too?
>

Sure, will add in v2.

>> +
>> +	if (kvm_max_vcpus > rl.rlim_cur - NOFD) {
>> +		rl.rlim_cur = kvm_max_vcpus + NOFD;
>> +
>> +		if (kvm_max_vcpus > rl.rlim_max - NOFD)
>> +			rl.rlim_max = kvm_max_vcpus + NOFD;
>> +
>> +		TEST_ASSERT(!setrlimit(RLIMIT_NOFILE, &rl),
>> +			    "setrlimit() failed (errno: %d)", errno);
>> +	}
>> +
>>  	/*
>>  	 * Upstream KVM prior to 4.8 does not support KVM_CAP_MAX_VCPU_ID.
>>  	 * Userspace is supposed to use KVM_CAP_MAX_VCPUS as the maximum ID
>> -- 
>> 2.33.1
>> 
>

-- 
Vitaly

Re: [PATCH] KVM: selftests: Make sure kvm_create_max_vcpus test won't hit RLIMIT_NOFILE

[Vitaly Kuznetsov]

Vitaly Kuznetsov <vkuznets@redhat.com> writes:

>>>  
>>> +	/*
>>> +	 * Creating KVM_CAP_MAX_VCPUS vCPUs require KVM_CAP_MAX_VCPUS open
>>> +	 * file decriptors.
>>> +	 */
>>> +	TEST_ASSERT(!getrlimit(RLIMIT_NOFILE, &rl),
>>> +		    "getrlimit() failed (errno: %d)", errno);
>>
>> And strerror() output too?
>>
>
> Sure, will add in v2.
>

Actually, there are two issues with the code above. First, TEST_ASSERT()
already prints both errno and strerror() (setrlimit() counterpart which
is easier to make fail):

KVM_CAP_MAX_VCPU_ID: 4096
KVM_CAP_MAX_VCPUS: 1024
==== Test Assertion Failure ====
  kvm_create_max_vcpus.c:68: !setrlimit(RLIMIT_NOFILE, &rl)
  pid=344504 tid=344504 errno=1 - Operation not permitted
     1	0x0000000000402485: main at kvm_create_max_vcpus.c:68
     2	0x00007fcb2e8b4041: ?? ??:0
     3	0x000000000040254d: _start at ??:?
  setrlimit() failed, errno: 0

Second, note "errno: 0" above. There's no guarantee that getrlimit()
will be executed before evaluating 'errno' in C. I think I'll just drop
redundant errno printout then.

-- 
Vitaly