[PATCH v2 0/2] cfi: free old cfi shadow asynchronously

[PATCH v2 0/2] cfi: free old cfi shadow asynchronously

[Haibo Li]

This patchset changes synchronize_rcu to call_rcu in update_shadow.

Patch 1 enables the CFI sanitization for cfi.c and 
bypass __cfi_slowpath_diag.
It is the prerequisite for using call_rcu in cfi.c.

Patch 2 changes synchronize_rcu to call_rcu and
free old cfi shadow in rcu cb

Changes in v2:
 - Remove _run_cfi_check,add __nocfi to __cfi_slowpath_diag directly
 - Rename _cfi_shadow_free_rcu to free_shadow

Haibo Li (2):
  cfi: enable sanitize for cfi.c
  cfi: free old cfi shadow asynchronously

 kernel/Makefile |  3 ---
 kernel/cfi.c    | 14 +++++++++++---
 2 files changed, 11 insertions(+), 6 deletions(-)

-- 
2.25.1

[PATCH v2 0/2] cfi: free old cfi shadow asynchronously

[Haibo Li]

This patchset changes synchronize_rcu to call_rcu in update_shadow.

Patch 1 enables the CFI sanitization for cfi.c and 
bypass __cfi_slowpath_diag.
It is the prerequisite for using call_rcu in cfi.c.

Patch 2 changes synchronize_rcu to call_rcu and
free old cfi shadow in rcu cb

Changes in v2:
 - Remove _run_cfi_check,add __nocfi to __cfi_slowpath_diag directly
 - Rename _cfi_shadow_free_rcu to free_shadow

Haibo Li (2):
  cfi: enable sanitize for cfi.c
  cfi: free old cfi shadow asynchronously

 kernel/Makefile |  3 ---
 kernel/cfi.c    | 14 +++++++++++---
 2 files changed, 11 insertions(+), 6 deletions(-)

-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v2 1/2] cfi: enable sanitize for cfi.c

[Haibo Li]

currenly,cfi.c is excluded from cfi sanitize because of cfi handler.
The side effect is that we can not transfer function pointer to
other files which enable cfi sanitize.

Enable cfi sanitize for cfi.c and bypass cfi check for __cfi_slowpath_diag

Signed-off-by: Haibo Li <haibo.li@mediatek.com>
Signed-off-by: Lecopzer Chen <lecopzer.chen@mediatek.com>
---
 kernel/Makefile | 3 ---
 kernel/cfi.c    | 2 +-
 2 files changed, 1 insertion(+), 4 deletions(-)

diff --git a/kernel/Makefile b/kernel/Makefile
index a7e1f49ab2b3..a997bef1a200 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -40,9 +40,6 @@ KCSAN_SANITIZE_kcov.o := n
 UBSAN_SANITIZE_kcov.o := n
 CFLAGS_kcov.o := $(call cc-option, -fno-conserve-stack) -fno-stack-protector
 
-# Don't instrument error handlers
-CFLAGS_REMOVE_cfi.o := $(CC_FLAGS_CFI)
-
 obj-y += sched/
 obj-y += locking/
 obj-y += power/
diff --git a/kernel/cfi.c b/kernel/cfi.c
index 08102d19ec15..6c8ae07b5835 100644
--- a/kernel/cfi.c
+++ b/kernel/cfi.c
@@ -311,7 +311,7 @@ static inline cfi_check_fn find_check_fn(unsigned long ptr)
 	return fn;
 }
 
-void __cfi_slowpath_diag(uint64_t id, void *ptr, void *diag)
+void __nocfi __cfi_slowpath_diag(u64 id, void *ptr, void *diag)
 {
 	cfi_check_fn fn = find_check_fn((unsigned long)ptr);
 
-- 
2.25.1

[PATCH v2 1/2] cfi: enable sanitize for cfi.c

[Haibo Li]

currenly,cfi.c is excluded from cfi sanitize because of cfi handler.
The side effect is that we can not transfer function pointer to
other files which enable cfi sanitize.

Enable cfi sanitize for cfi.c and bypass cfi check for __cfi_slowpath_diag

Signed-off-by: Haibo Li <haibo.li@mediatek.com>
Signed-off-by: Lecopzer Chen <lecopzer.chen@mediatek.com>
---
 kernel/Makefile | 3 ---
 kernel/cfi.c    | 2 +-
 2 files changed, 1 insertion(+), 4 deletions(-)

diff --git a/kernel/Makefile b/kernel/Makefile
index a7e1f49ab2b3..a997bef1a200 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -40,9 +40,6 @@ KCSAN_SANITIZE_kcov.o := n
 UBSAN_SANITIZE_kcov.o := n
 CFLAGS_kcov.o := $(call cc-option, -fno-conserve-stack) -fno-stack-protector
 
-# Don't instrument error handlers
-CFLAGS_REMOVE_cfi.o := $(CC_FLAGS_CFI)
-
 obj-y += sched/
 obj-y += locking/
 obj-y += power/
diff --git a/kernel/cfi.c b/kernel/cfi.c
index 08102d19ec15..6c8ae07b5835 100644
--- a/kernel/cfi.c
+++ b/kernel/cfi.c
@@ -311,7 +311,7 @@ static inline cfi_check_fn find_check_fn(unsigned long ptr)
 	return fn;
 }
 
-void __cfi_slowpath_diag(uint64_t id, void *ptr, void *diag)
+void __nocfi __cfi_slowpath_diag(u64 id, void *ptr, void *diag)
 {
 	cfi_check_fn fn = find_check_fn((unsigned long)ptr);
 
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v2 2/2] cfi: free old cfi shadow asynchronously

[Haibo Li]

Currenly, it uses synchronize_rcu() to wait old rcu reader to go away
in update_shadow.In embedded platform like ARM CA7X,
load_module blocks 40~50ms in update_shadow.
When there are more than one hundred kernel modules,
it blocks several seconds.

To accelerate load_module,change synchronize_rcu to call_rcu.

Signed-off-by: Haibo Li <haibo.li@mediatek.com>
Signed-off-by: Lecopzer Chen <lecopzer.chen@mediatek.com>
---
 kernel/cfi.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/kernel/cfi.c b/kernel/cfi.c
index 6c8ae07b5835..f61439392bf8 100644
--- a/kernel/cfi.c
+++ b/kernel/cfi.c
@@ -43,6 +43,8 @@ typedef u16 shadow_t;
 struct cfi_shadow {
 	/* Page index for the beginning of the shadow */
 	unsigned long base;
+	/* rcu to free old cfi_shadow asynchronously */
+	struct rcu_head rcu;
 	/* An array of __cfi_check locations (as indices to the shadow) */
 	shadow_t shadow[1];
 } __packed;
@@ -182,6 +184,13 @@ static void remove_module_from_shadow(struct cfi_shadow *s, struct module *mod,
 	}
 }
 
+static void free_shadow(struct rcu_head *rcu)
+{
+	struct cfi_shadow *old = container_of(rcu, struct cfi_shadow, rcu);
+
+	vfree(old);
+}
+
 typedef void (*update_shadow_fn)(struct cfi_shadow *, struct module *,
 			unsigned long min_addr, unsigned long max_addr);
 
@@ -211,11 +220,10 @@ static void update_shadow(struct module *mod, unsigned long base_addr,
 
 	rcu_assign_pointer(cfi_shadow, next);
 	mutex_unlock(&shadow_update_lock);
-	synchronize_rcu();
 
 	if (prev) {
 		set_memory_rw((unsigned long)prev, SHADOW_PAGES);
-		vfree(prev);
+		call_rcu(&prev->rcu, free_shadow);
 	}
 }
 
-- 
2.25.1

[PATCH v2 2/2] cfi: free old cfi shadow asynchronously

[Haibo Li]

Currenly, it uses synchronize_rcu() to wait old rcu reader to go away
in update_shadow.In embedded platform like ARM CA7X,
load_module blocks 40~50ms in update_shadow.
When there are more than one hundred kernel modules,
it blocks several seconds.

To accelerate load_module,change synchronize_rcu to call_rcu.

Signed-off-by: Haibo Li <haibo.li@mediatek.com>
Signed-off-by: Lecopzer Chen <lecopzer.chen@mediatek.com>
---
 kernel/cfi.c | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)

diff --git a/kernel/cfi.c b/kernel/cfi.c
index 6c8ae07b5835..f61439392bf8 100644
--- a/kernel/cfi.c
+++ b/kernel/cfi.c
@@ -43,6 +43,8 @@ typedef u16 shadow_t;
 struct cfi_shadow {
 	/* Page index for the beginning of the shadow */
 	unsigned long base;
+	/* rcu to free old cfi_shadow asynchronously */
+	struct rcu_head rcu;
 	/* An array of __cfi_check locations (as indices to the shadow) */
 	shadow_t shadow[1];
 } __packed;
@@ -182,6 +184,13 @@ static void remove_module_from_shadow(struct cfi_shadow *s, struct module *mod,
 	}
 }
 
+static void free_shadow(struct rcu_head *rcu)
+{
+	struct cfi_shadow *old = container_of(rcu, struct cfi_shadow, rcu);
+
+	vfree(old);
+}
+
 typedef void (*update_shadow_fn)(struct cfi_shadow *, struct module *,
 			unsigned long min_addr, unsigned long max_addr);
 
@@ -211,11 +220,10 @@ static void update_shadow(struct module *mod, unsigned long base_addr,
 
 	rcu_assign_pointer(cfi_shadow, next);
 	mutex_unlock(&shadow_update_lock);
-	synchronize_rcu();
 
 	if (prev) {
 		set_memory_rw((unsigned long)prev, SHADOW_PAGES);
-		vfree(prev);
+		call_rcu(&prev->rcu, free_shadow);
 	}
 }
 
-- 
2.25.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 0/2] cfi: free old cfi shadow asynchronously

[Sami Tolvanen]

On Mon, Jul 04, 2022 at 09:40:44AM +0800, Haibo Li wrote:
> This patchset changes synchronize_rcu to call_rcu in update_shadow.
> 
> Patch 1 enables the CFI sanitization for cfi.c and 
> bypass __cfi_slowpath_diag.
> It is the prerequisite for using call_rcu in cfi.c.
> 
> Patch 2 changes synchronize_rcu to call_rcu and
> free old cfi shadow in rcu cb
> 
> Changes in v2:
>  - Remove _run_cfi_check,add __nocfi to __cfi_slowpath_diag directly
>  - Rename _cfi_shadow_free_rcu to free_shadow
> 
> Haibo Li (2):
>   cfi: enable sanitize for cfi.c
>   cfi: free old cfi shadow asynchronously
> 
>  kernel/Makefile |  3 ---
>  kernel/cfi.c    | 14 +++++++++++---
>  2 files changed, 11 insertions(+), 6 deletions(-)

Thanks for sending v2! For the series:

Reviewed-by: Sami Tolvanen <samitolvanen@google.com>

Sami

Re: [PATCH v2 0/2] cfi: free old cfi shadow asynchronously

[Sami Tolvanen]

On Mon, Jul 04, 2022 at 09:40:44AM +0800, Haibo Li wrote:
> This patchset changes synchronize_rcu to call_rcu in update_shadow.
> 
> Patch 1 enables the CFI sanitization for cfi.c and 
> bypass __cfi_slowpath_diag.
> It is the prerequisite for using call_rcu in cfi.c.
> 
> Patch 2 changes synchronize_rcu to call_rcu and
> free old cfi shadow in rcu cb
> 
> Changes in v2:
>  - Remove _run_cfi_check,add __nocfi to __cfi_slowpath_diag directly
>  - Rename _cfi_shadow_free_rcu to free_shadow
> 
> Haibo Li (2):
>   cfi: enable sanitize for cfi.c
>   cfi: free old cfi shadow asynchronously
> 
>  kernel/Makefile |  3 ---
>  kernel/cfi.c    | 14 +++++++++++---
>  2 files changed, 11 insertions(+), 6 deletions(-)

Thanks for sending v2! For the series:

Reviewed-by: Sami Tolvanen <samitolvanen@google.com>

Sami

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v2 0/2] cfi: free old cfi shadow asynchronously

[Yongqin Liu]

Hi, Haibo, Sami

Thanks for the changes here!

With the Linaro LKFT project[1], we noticed there is about 5 seconds
boottime improvement for the 5.15 kernel Android builds.
And after some investigation, we found it's these changes
made the improvement.
And I'd like to share one of the test results here for your information:

====>>> data started here
 Linux 5.15: 5.15-gki-android13-aosp-master-rb5-boottime
    boottime:
    Index Build Number boottime-fresh-install#KERNEL_BOOT_TIME_avg
boottime-fresh-install#ANDROID_BOOT_TIME_avg
boottime-fresh-install#TOTAL_BOOT_TIME_avg
boottime-reboot#KERNEL_BOOT_TIME_avg
boottime-reboot#ANDROID_BOOT_TIME_avg
boottime-reboot#TOTAL_BOOT_TIME_avg
    1 5.15.41-aec224bb9ebc 1.6 37.19 38.79 1.61 30.14 31.75
    2 5.15.41-c2162eca3f66 1.62 36.92 38.53 1.62 30.04 31.66
    3 5.15.41-e796ee569574 1.62 37.27 38.89 1.62 30.01 31.62
    4 5.15.41-432cf292eb20 1.62 41.64 43.26 1.61 35.16 36.76
    5 5.15.41-a3829998513a 1.62 42.2 43.82 1.61 35.4 37.01
<<<==== data ended here

# the improvements started from build#5.15.41-e796ee569574 since these
changes were submitted to the ACK[3]
# the unit of the data is seconds
# boottime-fresh items are data for the first time boot after deployment
# boottime-reboot items are the average data for the 2nd ~ 7th data
with the minimum and maximum excluded.
# for the meaning of
KERNEL_BOOT_TIME/ANDROID_BOOT_TIME/TOTAL_BOOT_TIME please see the
description here[2]

[1]: https://source.android.com/docs/core/architecture/kernel/android-common?hl=en#linaro-kernel-functional-testing
[2]: https://github.com/Linaro/test-definitions/blob/master/automated/android/boottime/device-script.sh
[3]: https://android-review.googlesource.com/q/I08dcb29ef97453b836efd0d64286196600be8cce

Thanks,
Yongqin Liu

On Tue, 12 Jul 2022 at 03:20, Sami Tolvanen <samitolvanen@google.com> wrote:
>
> On Mon, Jul 04, 2022 at 09:40:44AM +0800, Haibo Li wrote:
> > This patchset changes synchronize_rcu to call_rcu in update_shadow.
> >
> > Patch 1 enables the CFI sanitization for cfi.c and
> > bypass __cfi_slowpath_diag.
> > It is the prerequisite for using call_rcu in cfi.c.
> >
> > Patch 2 changes synchronize_rcu to call_rcu and
> > free old cfi shadow in rcu cb
> >
> > Changes in v2:
> >  - Remove _run_cfi_check,add __nocfi to __cfi_slowpath_diag directly
> >  - Rename _cfi_shadow_free_rcu to free_shadow
> >
> > Haibo Li (2):
> >   cfi: enable sanitize for cfi.c
> >   cfi: free old cfi shadow asynchronously
> >
> >  kernel/Makefile |  3 ---
> >  kernel/cfi.c    | 14 +++++++++++---
> >  2 files changed, 11 insertions(+), 6 deletions(-)
>
> Thanks for sending v2! For the series:
>
> Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
>
> Sami



-- 
Best Regards,
Yongqin Liu
---------------------------------------------------------------
#mailing list
linaro-android@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-android

Re: [PATCH v2 0/2] cfi: free old cfi shadow asynchronously

[Yongqin Liu]

Hi, Haibo, Sami

Thanks for the changes here!

With the Linaro LKFT project[1], we noticed there is about 5 seconds
boottime improvement for the 5.15 kernel Android builds.
And after some investigation, we found it's these changes
made the improvement.
And I'd like to share one of the test results here for your information:

====>>> data started here
 Linux 5.15: 5.15-gki-android13-aosp-master-rb5-boottime
    boottime:
    Index Build Number boottime-fresh-install#KERNEL_BOOT_TIME_avg
boottime-fresh-install#ANDROID_BOOT_TIME_avg
boottime-fresh-install#TOTAL_BOOT_TIME_avg
boottime-reboot#KERNEL_BOOT_TIME_avg
boottime-reboot#ANDROID_BOOT_TIME_avg
boottime-reboot#TOTAL_BOOT_TIME_avg
    1 5.15.41-aec224bb9ebc 1.6 37.19 38.79 1.61 30.14 31.75
    2 5.15.41-c2162eca3f66 1.62 36.92 38.53 1.62 30.04 31.66
    3 5.15.41-e796ee569574 1.62 37.27 38.89 1.62 30.01 31.62
    4 5.15.41-432cf292eb20 1.62 41.64 43.26 1.61 35.16 36.76
    5 5.15.41-a3829998513a 1.62 42.2 43.82 1.61 35.4 37.01
<<<==== data ended here

# the improvements started from build#5.15.41-e796ee569574 since these
changes were submitted to the ACK[3]
# the unit of the data is seconds
# boottime-fresh items are data for the first time boot after deployment
# boottime-reboot items are the average data for the 2nd ~ 7th data
with the minimum and maximum excluded.
# for the meaning of
KERNEL_BOOT_TIME/ANDROID_BOOT_TIME/TOTAL_BOOT_TIME please see the
description here[2]

[1]: https://source.android.com/docs/core/architecture/kernel/android-common?hl=en#linaro-kernel-functional-testing
[2]: https://github.com/Linaro/test-definitions/blob/master/automated/android/boottime/device-script.sh
[3]: https://android-review.googlesource.com/q/I08dcb29ef97453b836efd0d64286196600be8cce

Thanks,
Yongqin Liu

On Tue, 12 Jul 2022 at 03:20, Sami Tolvanen <samitolvanen@google.com> wrote:
>
> On Mon, Jul 04, 2022 at 09:40:44AM +0800, Haibo Li wrote:
> > This patchset changes synchronize_rcu to call_rcu in update_shadow.
> >
> > Patch 1 enables the CFI sanitization for cfi.c and
> > bypass __cfi_slowpath_diag.
> > It is the prerequisite for using call_rcu in cfi.c.
> >
> > Patch 2 changes synchronize_rcu to call_rcu and
> > free old cfi shadow in rcu cb
> >
> > Changes in v2:
> >  - Remove _run_cfi_check,add __nocfi to __cfi_slowpath_diag directly
> >  - Rename _cfi_shadow_free_rcu to free_shadow
> >
> > Haibo Li (2):
> >   cfi: enable sanitize for cfi.c
> >   cfi: free old cfi shadow asynchronously
> >
> >  kernel/Makefile |  3 ---
> >  kernel/cfi.c    | 14 +++++++++++---
> >  2 files changed, 11 insertions(+), 6 deletions(-)
>
> Thanks for sending v2! For the series:
>
> Reviewed-by: Sami Tolvanen <samitolvanen@google.com>
>
> Sami



-- 
Best Regards,
Yongqin Liu
---------------------------------------------------------------
#mailing list
linaro-android@lists.linaro.org
http://lists.linaro.org/mailman/listinfo/linaro-android

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel