[PATCH 0/2] gpudev: annotate memory allocation

[PATCH 0/2] gpudev: annotate memory allocation

[Stephen Hemminger]

Use function attributes to catch misuse of GPU memory
at compile time.

Stephen Hemminger (2):
  test-gpudev: avoid use-after-free and free-non-heap warnings
  gpudev: add malloc annotations to rte_gpu_mem_alloc

 app/test-gpudev/main.c  | 10 ++++++++-
 lib/gpudev/rte_gpudev.h | 46 +++++++++++++++++++++--------------------
 2 files changed, 33 insertions(+), 23 deletions(-)

-- 
2.45.2

[PATCH 1/2] test-gpudev: avoid use-after-free and free-non-heap warnings

[Stephen Hemminger]

This test intentionally frees a bad pointer in GPU memory,
and GCC sees that at compile time. Suppress the warnings to allow
the test to run.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
 app/test-gpudev/main.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/app/test-gpudev/main.c b/app/test-gpudev/main.c
index f065e6cd81..14df14d528 100644
--- a/app/test-gpudev/main.c
+++ b/app/test-gpudev/main.c
@@ -96,6 +96,13 @@ alloc_gpu_memory(uint16_t gpu_id)
 		goto error;
 	}
 
+	/* GCC 11 or later, is able to detect use-after-free and calling free on no-pointer
+	 * at compilation, but want to allow these tests to still work.
+	 */
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wuse-after-free"
+#pragma GCC diagnostic ignored "-Wfree-nonheap-object"
+
 	ret = rte_gpu_mem_free(gpu_id, (uint8_t *)(ptr_1)+0x700);
 	if (ret < 0) {
 		printf("GPU memory 0x%p NOT freed: GPU driver didn't find this memory address internally.\n",
@@ -128,6 +135,7 @@ alloc_gpu_memory(uint16_t gpu_id)
 	rte_gpu_mem_free(gpu_id, ptr_1);
 	rte_gpu_mem_free(gpu_id, ptr_2);
 
+#pragma GCC diagnostic pop
 	printf("\n=======> TEST: FAILED\n");
 	return -1;
 }
@@ -228,12 +236,12 @@ gpu_mem_cpu_map(uint16_t gpu_id)
 	}
 	printf("GPU memory CPU unmapped, 0x%p not valid anymore\n", ptr_cpu);
 
+	printf("GPU memory 0x%p freeing\n", ptr_gpu);
 	ret = rte_gpu_mem_free(gpu_id, ptr_gpu);
 	if (ret < 0) {
 		fprintf(stderr, "rte_gpu_mem_free returned error %d\n", ret);
 		goto error;
 	}
-	printf("GPU memory 0x%p freed\n", ptr_gpu);
 
 	printf("\n=======> TEST: PASSED\n");
 	return 0;
-- 
2.45.2

[PATCH 2/2] gpudev: add malloc annotations to rte_gpu_mem_alloc

[Stephen Hemminger]

Add function attributes that allow detecting use after free,
and calling free on bad pointer at compile time.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
 lib/gpudev/rte_gpudev.h | 46 +++++++++++++++++++++--------------------
 1 file changed, 24 insertions(+), 22 deletions(-)

diff --git a/lib/gpudev/rte_gpudev.h b/lib/gpudev/rte_gpudev.h
index 0a94a6abc4..7a35cf85d1 100644
--- a/lib/gpudev/rte_gpudev.h
+++ b/lib/gpudev/rte_gpudev.h
@@ -357,6 +357,28 @@ int rte_gpu_callback_unregister(int16_t dev_id, enum rte_gpu_event event,
 __rte_experimental
 int rte_gpu_info_get(int16_t dev_id, struct rte_gpu_info *info);
 
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Deallocate a chunk of memory allocated with rte_gpu_mem_alloc().
+ *
+ * @param dev_id
+ *   Reference device ID.
+ * @param ptr
+ *   Pointer to the memory area to be deallocated.
+ *   NULL is a no-op accepted value.
+ *
+ * @return
+ *   0 on success, -rte_errno otherwise:
+ *   - ENODEV if invalid dev_id
+ *   - ENOTSUP if operation not supported by the driver
+ *   - EPERM if driver error
+ */
+__rte_experimental
+int rte_gpu_mem_free(int16_t dev_id, void *ptr);
+
 /**
  * @warning
  * @b EXPERIMENTAL: this API may change without prior notice.
@@ -385,28 +407,8 @@ int rte_gpu_info_get(int16_t dev_id, struct rte_gpu_info *info);
  */
 __rte_experimental
 void *rte_gpu_mem_alloc(int16_t dev_id, size_t size, unsigned int align)
-__rte_alloc_size(2);
-
-/**
- * @warning
- * @b EXPERIMENTAL: this API may change without prior notice.
- *
- * Deallocate a chunk of memory allocated with rte_gpu_mem_alloc().
- *
- * @param dev_id
- *   Reference device ID.
- * @param ptr
- *   Pointer to the memory area to be deallocated.
- *   NULL is a no-op accepted value.
- *
- * @return
- *   0 on success, -rte_errno otherwise:
- *   - ENODEV if invalid dev_id
- *   - ENOTSUP if operation not supported by the driver
- *   - EPERM if driver error
- */
-__rte_experimental
-int rte_gpu_mem_free(int16_t dev_id, void *ptr);
+	__rte_alloc_size(2) __rte_alloc_align(3)
+	__rte_malloc __rte_dealloc(rte_gpu_mem_free, 2);
 
 /**
  * @warning
-- 
2.45.2

[PATCH v2 0/2] gpudev: annotate memory allocation functions

[Stephen Hemminger]

Use function attributes to catch misuse of GPU memory
at compile time.

v2 - remove test case where invalid pointer was being passed.
     The test is now caught at compile time, making a runtime test
     no longer necessary.

Stephen Hemminger (2):
  test-gpudev: avoid use-after-free and free-non-heap warnings
  gpudev: add malloc annotations to rte_gpu_mem_alloc

 app/test-gpudev/main.c  | 16 +++-----------
 lib/gpudev/rte_gpudev.h | 46 +++++++++++++++++++++--------------------
 2 files changed, 27 insertions(+), 35 deletions(-)

-- 
2.45.2

[PATCH v2 1/2] test-gpudev: avoid use-after-free and free-non-heap warnings

[Stephen Hemminger]

Remove the test intentionally frees a bad pointer in GPU memory,
and GCC sees that at compile time now.

Fix use after free in printf.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
 app/test-gpudev/main.c | 16 +++-------------
 1 file changed, 3 insertions(+), 13 deletions(-)

diff --git a/app/test-gpudev/main.c b/app/test-gpudev/main.c
index f065e6cd81..30fe573bf9 100644
--- a/app/test-gpudev/main.c
+++ b/app/test-gpudev/main.c
@@ -96,29 +96,19 @@ alloc_gpu_memory(uint16_t gpu_id)
 		goto error;
 	}
 
-	ret = rte_gpu_mem_free(gpu_id, (uint8_t *)(ptr_1)+0x700);
-	if (ret < 0) {
-		printf("GPU memory 0x%p NOT freed: GPU driver didn't find this memory address internally.\n",
-				(uint8_t *)(ptr_1)+0x700);
-	} else {
-		fprintf(stderr, "ERROR: rte_gpu_mem_free freed GPU memory 0x%p\n",
-				(uint8_t *)(ptr_1)+0x700);
-		goto error;
-	}
-
+	printf("GPU memory 0x%p free\n", ptr_2);
 	ret = rte_gpu_mem_free(gpu_id, ptr_2);
 	if (ret < 0) {
 		fprintf(stderr, "rte_gpu_mem_free returned error %d\n", ret);
 		goto error;
 	}
-	printf("GPU memory 0x%p freed\n", ptr_2);
 
+	printf("GPU memory 0x%p free\n", ptr_1);
 	ret = rte_gpu_mem_free(gpu_id, ptr_1);
 	if (ret < 0) {
 		fprintf(stderr, "rte_gpu_mem_free returned error %d\n", ret);
 		goto error;
 	}
-	printf("GPU memory 0x%p freed\n", ptr_1);
 
 	printf("\n=======> TEST: PASSED\n");
 	return 0;
@@ -228,12 +218,12 @@ gpu_mem_cpu_map(uint16_t gpu_id)
 	}
 	printf("GPU memory CPU unmapped, 0x%p not valid anymore\n", ptr_cpu);
 
+	printf("GPU memory 0x%p freeing\n", ptr_gpu);
 	ret = rte_gpu_mem_free(gpu_id, ptr_gpu);
 	if (ret < 0) {
 		fprintf(stderr, "rte_gpu_mem_free returned error %d\n", ret);
 		goto error;
 	}
-	printf("GPU memory 0x%p freed\n", ptr_gpu);
 
 	printf("\n=======> TEST: PASSED\n");
 	return 0;
-- 
2.45.2

[PATCH v2 2/2] gpudev: add malloc annotations to rte_gpu_mem_alloc

[Stephen Hemminger]

Add function attributes that allow detecting use after free,
and calling free on bad pointer at compile time.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
---
 lib/gpudev/rte_gpudev.h | 46 +++++++++++++++++++++--------------------
 1 file changed, 24 insertions(+), 22 deletions(-)

diff --git a/lib/gpudev/rte_gpudev.h b/lib/gpudev/rte_gpudev.h
index 0a94a6abc4..7a35cf85d1 100644
--- a/lib/gpudev/rte_gpudev.h
+++ b/lib/gpudev/rte_gpudev.h
@@ -357,6 +357,28 @@ int rte_gpu_callback_unregister(int16_t dev_id, enum rte_gpu_event event,
 __rte_experimental
 int rte_gpu_info_get(int16_t dev_id, struct rte_gpu_info *info);
 
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Deallocate a chunk of memory allocated with rte_gpu_mem_alloc().
+ *
+ * @param dev_id
+ *   Reference device ID.
+ * @param ptr
+ *   Pointer to the memory area to be deallocated.
+ *   NULL is a no-op accepted value.
+ *
+ * @return
+ *   0 on success, -rte_errno otherwise:
+ *   - ENODEV if invalid dev_id
+ *   - ENOTSUP if operation not supported by the driver
+ *   - EPERM if driver error
+ */
+__rte_experimental
+int rte_gpu_mem_free(int16_t dev_id, void *ptr);
+
 /**
  * @warning
  * @b EXPERIMENTAL: this API may change without prior notice.
@@ -385,28 +407,8 @@ int rte_gpu_info_get(int16_t dev_id, struct rte_gpu_info *info);
  */
 __rte_experimental
 void *rte_gpu_mem_alloc(int16_t dev_id, size_t size, unsigned int align)
-__rte_alloc_size(2);
-
-/**
- * @warning
- * @b EXPERIMENTAL: this API may change without prior notice.
- *
- * Deallocate a chunk of memory allocated with rte_gpu_mem_alloc().
- *
- * @param dev_id
- *   Reference device ID.
- * @param ptr
- *   Pointer to the memory area to be deallocated.
- *   NULL is a no-op accepted value.
- *
- * @return
- *   0 on success, -rte_errno otherwise:
- *   - ENODEV if invalid dev_id
- *   - ENOTSUP if operation not supported by the driver
- *   - EPERM if driver error
- */
-__rte_experimental
-int rte_gpu_mem_free(int16_t dev_id, void *ptr);
+	__rte_alloc_size(2) __rte_alloc_align(3)
+	__rte_malloc __rte_dealloc(rte_gpu_mem_free, 2);
 
 /**
  * @warning
-- 
2.45.2

Re: [PATCH v2 0/2] gpudev: annotate memory allocation functions

[David Marchand]

On Mon, Oct 28, 2024 at 8:03 PM Stephen Hemminger
<stephen@networkplumber.org> wrote:
>
> Use function attributes to catch misuse of GPU memory
> at compile time.
>
> v2 - remove test case where invalid pointer was being passed.
>      The test is now caught at compile time, making a runtime test
>      no longer necessary.
>
> Stephen Hemminger (2):
>   test-gpudev: avoid use-after-free and free-non-heap warnings
>   gpudev: add malloc annotations to rte_gpu_mem_alloc
>
>  app/test-gpudev/main.c  | 16 +++-----------
>  lib/gpudev/rte_gpudev.h | 46 +++++++++++++++++++++--------------------
>  2 files changed, 27 insertions(+), 35 deletions(-)

There is a build error due to referencing experimental symbol in annotation.


-- 
David Marchand

Re: [PATCH 0/2] gpudev: annotate memory allocation

[Stephen Hemminger]

On Thu, 17 Oct 2024 15:58:02 -0700
Stephen Hemminger <stephen@networkplumber.org> wrote:

> Use function attributes to catch misuse of GPU memory
> at compile time.
> 
> Stephen Hemminger (2):
>   test-gpudev: avoid use-after-free and free-non-heap warnings
>   gpudev: add malloc annotations to rte_gpu_mem_alloc
> 
>  app/test-gpudev/main.c  | 10 ++++++++-
>  lib/gpudev/rte_gpudev.h | 46 +++++++++++++++++++++--------------------
>  2 files changed, 33 insertions(+), 23 deletions(-)
> 


The problem is that include checker can't handle this.
####################################################################################
#### [Begin job log] "ubuntu-22.04-gcc-debug+doc+examples+tests" at step Build and test
####################################################################################
                 from buildtools/chkincs/chkincs.p/gpudev_driver.c:1:
/home/runner/work/dpdk/dpdk/lib/gpudev/rte_gpudev.h:411:9: error: ‘rte_gpu_mem_free’ is deprecated: Symbol is not yet part of stable ABI [-Werror=deprecated-declarations]
  411 |         __rte_malloc __rte_dealloc(rte_gpu_mem_free, 2);
      |         ^~~~~~~~~~~~
/home/runner/work/dpdk/dpdk/lib/gpudev/rte_gpudev.h:380:5: note: declared here
  380 | int rte_gpu_mem_free(int16_t dev_id, void *ptr);
      |     ^~~~~~~~~~~~~~~~

Either include checker needs to be able to handle experimental symbols
or maybe it is time for gpudev to be moved out experimental status for 25.03?