[PATCH 0/2] zs_object_copy() micro-optimizations

[PATCH 0/2] zs_object_copy() micro-optimizations

[Sergey Senozhatsky]

Hello,

two small patches two micro-optimize zs_object_copy().

The first one removes unneeded kunmap_atomic/kmap_atomic of dst page,
when object that we copy belongs to two source pages.

The seconds one is also trivial -- removes branching and (a bit)
reduses the amount of work done by the function (double offsets
calculations).

Sergey Senozhatsky (2):
  zsmalloc: do not remap dst page while prepare next src page
  zsmalloc: micro-optimize zs_object_copy()

 mm/zsmalloc.c | 17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

-- 
2.3.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 0/2] zs_object_copy() micro-optimizations

[Sergey Senozhatsky]

Hello,

two small patches two micro-optimize zs_object_copy().

The first one removes unneeded kunmap_atomic/kmap_atomic of dst page,
when object that we copy belongs to two source pages.

The seconds one is also trivial -- removes branching and (a bit)
reduses the amount of work done by the function (double offsets
calculations).

Sergey Senozhatsky (2):
  zsmalloc: do not remap dst page while prepare next src page
  zsmalloc: micro-optimize zs_object_copy()

 mm/zsmalloc.c | 17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

-- 
2.3.4

[PATCH 1/2] zsmalloc: do not remap dst page while prepare next src page

[Sergey Senozhatsky]

object may belong to different pages. zs_object_copy() handles
this case and maps a new source page (get_next_page() and
kmap_atomic()) when object crosses boundaries of the current
source page. But it also performs unnecessary kunmap/kmap_atomic
of the destination page (it remains unchanged), which can be
avoided.

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
 mm/zsmalloc.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index d920e8b..7af4456 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -1536,12 +1536,10 @@ static void zs_object_copy(unsigned long src, unsigned long dst,
 			break;
 
 		if (s_off + size >= PAGE_SIZE) {
-			kunmap_atomic(d_addr);
 			kunmap_atomic(s_addr);
 			s_page = get_next_page(s_page);
 			BUG_ON(!s_page);
 			s_addr = kmap_atomic(s_page);
-			d_addr = kmap_atomic(d_page);
 			s_size = class->size - written;
 			s_off = 0;
 		} else {
-- 
2.3.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 1/2] zsmalloc: do not remap dst page while prepare next src page

[Sergey Senozhatsky]

object may belong to different pages. zs_object_copy() handles
this case and maps a new source page (get_next_page() and
kmap_atomic()) when object crosses boundaries of the current
source page. But it also performs unnecessary kunmap/kmap_atomic
of the destination page (it remains unchanged), which can be
avoided.

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
 mm/zsmalloc.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index d920e8b..7af4456 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -1536,12 +1536,10 @@ static void zs_object_copy(unsigned long src, unsigned long dst,
 			break;
 
 		if (s_off + size >= PAGE_SIZE) {
-			kunmap_atomic(d_addr);
 			kunmap_atomic(s_addr);
 			s_page = get_next_page(s_page);
 			BUG_ON(!s_page);
 			s_addr = kmap_atomic(s_page);
-			d_addr = kmap_atomic(d_page);
 			s_size = class->size - written;
 			s_off = 0;
 		} else {
-- 
2.3.4

Re: [PATCH 1/2] zsmalloc: do not remap dst page while prepare next src page

[Heesub Shin]

Hello,

On 03/25/2015 12:24 AM, Sergey Senozhatsky wrote:
> object may belong to different pages. zs_object_copy() handles
> this case and maps a new source page (get_next_page() and
> kmap_atomic()) when object crosses boundaries of the current
> source page. But it also performs unnecessary kunmap/kmap_atomic
> of the destination page (it remains unchanged), which can be
> avoided.

No, it's not unnecessary. We should do kunmap_atomic() in the reverse
order of kmap_atomic(), so unfortunately it's inevitable to
kunmap_atomic() both on d_addr and s_addr.

> 
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> ---
>  mm/zsmalloc.c | 2 --
>  1 file changed, 2 deletions(-)
> 
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index d920e8b..7af4456 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -1536,12 +1536,10 @@ static void zs_object_copy(unsigned long src, unsigned long dst,
>  			break;
>  
>  		if (s_off + size >= PAGE_SIZE) {
> -			kunmap_atomic(d_addr);
>  			kunmap_atomic(s_addr);

Removing kunmap_atomic(d_addr) here may cause BUG_ON() at __kunmap_atomic().

I tried yours to see it really happens:
> kernel BUG at arch/arm/mm/highmem.c:113!
> Internal error: Oops - BUG: 0 [#1] SMP ARM
> Modules linked in:
> CPU: 2 PID: 1774 Comm: bash Not tainted 4.0.0-rc2-mm1+ #105
> Hardware name: ARM-Versatile Express
> task: ee971300 ti: e8a26000 task.ti: e8a26000
> PC is at __kunmap_atomic+0x144/0x14c
> LR is at zs_object_copy+0x19c/0x2dc

regards
heesub

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/2] zsmalloc: do not remap dst page while prepare next src page

[Heesub Shin]

Hello,

On 03/25/2015 12:24 AM, Sergey Senozhatsky wrote:
> object may belong to different pages. zs_object_copy() handles
> this case and maps a new source page (get_next_page() and
> kmap_atomic()) when object crosses boundaries of the current
> source page. But it also performs unnecessary kunmap/kmap_atomic
> of the destination page (it remains unchanged), which can be
> avoided.

No, it's not unnecessary. We should do kunmap_atomic() in the reverse
order of kmap_atomic(), so unfortunately it's inevitable to
kunmap_atomic() both on d_addr and s_addr.

> 
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> ---
>  mm/zsmalloc.c | 2 --
>  1 file changed, 2 deletions(-)
> 
> diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> index d920e8b..7af4456 100644
> --- a/mm/zsmalloc.c
> +++ b/mm/zsmalloc.c
> @@ -1536,12 +1536,10 @@ static void zs_object_copy(unsigned long src, unsigned long dst,
>  			break;
>  
>  		if (s_off + size >= PAGE_SIZE) {
> -			kunmap_atomic(d_addr);
>  			kunmap_atomic(s_addr);

Removing kunmap_atomic(d_addr) here may cause BUG_ON() at __kunmap_atomic().

I tried yours to see it really happens:
> kernel BUG at arch/arm/mm/highmem.c:113!
> Internal error: Oops - BUG: 0 [#1] SMP ARM
> Modules linked in:
> CPU: 2 PID: 1774 Comm: bash Not tainted 4.0.0-rc2-mm1+ #105
> Hardware name: ARM-Versatile Express
> task: ee971300 ti: e8a26000 task.ti: e8a26000
> PC is at __kunmap_atomic+0x144/0x14c
> LR is at zs_object_copy+0x19c/0x2dc

regards
heesub

Re: [PATCH 1/2] zsmalloc: do not remap dst page while prepare next src page

[Sergey Senozhatsky]

On (03/25/15 14:05), Heesub Shin wrote:
> No, it's not unnecessary. We should do kunmap_atomic() in the reverse
> order of kmap_atomic(), so unfortunately it's inevitable to
> kunmap_atomic() both on d_addr and s_addr.
> 

Andrew, can you please drop this patch?


> > 
> > Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> > ---
> >  mm/zsmalloc.c | 2 --
> >  1 file changed, 2 deletions(-)
> > 
> > diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> > index d920e8b..7af4456 100644
> > --- a/mm/zsmalloc.c
> > +++ b/mm/zsmalloc.c
> > @@ -1536,12 +1536,10 @@ static void zs_object_copy(unsigned long src, unsigned long dst,
> >  			break;
> >  
> >  		if (s_off + size >= PAGE_SIZE) {
> > -			kunmap_atomic(d_addr);
> >  			kunmap_atomic(s_addr);
> 
> Removing kunmap_atomic(d_addr) here may cause BUG_ON() at __kunmap_atomic().
> 
> I tried yours to see it really happens:
> > kernel BUG at arch/arm/mm/highmem.c:113!

oh, arm. tested on x86_64 only. I see why it happens there. thanks for reporting.


sorry, should have checked.

> > Internal error: Oops - BUG: 0 [#1] SMP ARM
> > Modules linked in:
> > CPU: 2 PID: 1774 Comm: bash Not tainted 4.0.0-rc2-mm1+ #105
> > Hardware name: ARM-Versatile Express
> > task: ee971300 ti: e8a26000 task.ti: e8a26000
> > PC is at __kunmap_atomic+0x144/0x14c
> > LR is at zs_object_copy+0x19c/0x2dc

	-ss

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 1/2] zsmalloc: do not remap dst page while prepare next src page

[Sergey Senozhatsky]

On (03/25/15 14:05), Heesub Shin wrote:
> No, it's not unnecessary. We should do kunmap_atomic() in the reverse
> order of kmap_atomic(), so unfortunately it's inevitable to
> kunmap_atomic() both on d_addr and s_addr.
> 

Andrew, can you please drop this patch?


> > 
> > Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
> > ---
> >  mm/zsmalloc.c | 2 --
> >  1 file changed, 2 deletions(-)
> > 
> > diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
> > index d920e8b..7af4456 100644
> > --- a/mm/zsmalloc.c
> > +++ b/mm/zsmalloc.c
> > @@ -1536,12 +1536,10 @@ static void zs_object_copy(unsigned long src, unsigned long dst,
> >  			break;
> >  
> >  		if (s_off + size >= PAGE_SIZE) {
> > -			kunmap_atomic(d_addr);
> >  			kunmap_atomic(s_addr);
> 
> Removing kunmap_atomic(d_addr) here may cause BUG_ON() at __kunmap_atomic().
> 
> I tried yours to see it really happens:
> > kernel BUG at arch/arm/mm/highmem.c:113!

oh, arm. tested on x86_64 only. I see why it happens there. thanks for reporting.


sorry, should have checked.

> > Internal error: Oops - BUG: 0 [#1] SMP ARM
> > Modules linked in:
> > CPU: 2 PID: 1774 Comm: bash Not tainted 4.0.0-rc2-mm1+ #105
> > Hardware name: ARM-Versatile Express
> > task: ee971300 ti: e8a26000 task.ti: e8a26000
> > PC is at __kunmap_atomic+0x144/0x14c
> > LR is at zs_object_copy+0x19c/0x2dc

	-ss

[PATCH 2/2] zsmalloc: micro-optimize zs_object_copy()

[Sergey Senozhatsky]

A micro-optimization. Avoid additional branching and reduce
(a bit) registry pressure (f.e. s_off += size; d_off += size;
may be calculated twise: first for >= PAGE_SIZE check and later
for offset update in "else" clause).

/scripts/bloat-o-meter shows some improvement

add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-10 (-10)
function                          old     new   delta
zs_object_copy                    550     540     -10

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
 mm/zsmalloc.c | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 7af4456..dc35328 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -1535,28 +1535,27 @@ static void zs_object_copy(unsigned long src, unsigned long dst,
 		if (written == class->size)
 			break;
 
-		if (s_off + size >= PAGE_SIZE) {
+		s_off += size;
+		s_size -= size;
+		d_off += size;
+		d_size -= size;
+
+		if (s_off >= PAGE_SIZE) {
 			kunmap_atomic(s_addr);
 			s_page = get_next_page(s_page);
 			BUG_ON(!s_page);
 			s_addr = kmap_atomic(s_page);
 			s_size = class->size - written;
 			s_off = 0;
-		} else {
-			s_off += size;
-			s_size -= size;
 		}
 
-		if (d_off + size >= PAGE_SIZE) {
+		if (d_off >= PAGE_SIZE) {
 			kunmap_atomic(d_addr);
 			d_page = get_next_page(d_page);
 			BUG_ON(!d_page);
 			d_addr = kmap_atomic(d_page);
 			d_size = class->size - written;
 			d_off = 0;
-		} else {
-			d_off += size;
-			d_size -= size;
 		}
 	}
 
-- 
2.3.4

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

[PATCH 2/2] zsmalloc: micro-optimize zs_object_copy()

[Sergey Senozhatsky]

A micro-optimization. Avoid additional branching and reduce
(a bit) registry pressure (f.e. s_off += size; d_off += size;
may be calculated twise: first for >= PAGE_SIZE check and later
for offset update in "else" clause).

/scripts/bloat-o-meter shows some improvement

add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-10 (-10)
function                          old     new   delta
zs_object_copy                    550     540     -10

Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
---
 mm/zsmalloc.c | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)

diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index 7af4456..dc35328 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -1535,28 +1535,27 @@ static void zs_object_copy(unsigned long src, unsigned long dst,
 		if (written == class->size)
 			break;
 
-		if (s_off + size >= PAGE_SIZE) {
+		s_off += size;
+		s_size -= size;
+		d_off += size;
+		d_size -= size;
+
+		if (s_off >= PAGE_SIZE) {
 			kunmap_atomic(s_addr);
 			s_page = get_next_page(s_page);
 			BUG_ON(!s_page);
 			s_addr = kmap_atomic(s_page);
 			s_size = class->size - written;
 			s_off = 0;
-		} else {
-			s_off += size;
-			s_size -= size;
 		}
 
-		if (d_off + size >= PAGE_SIZE) {
+		if (d_off >= PAGE_SIZE) {
 			kunmap_atomic(d_addr);
 			d_page = get_next_page(d_page);
 			BUG_ON(!d_page);
 			d_addr = kmap_atomic(d_page);
 			d_size = class->size - written;
 			d_off = 0;
-		} else {
-			d_off += size;
-			d_size -= size;
 		}
 	}
 
-- 
2.3.4

Re: [PATCH 2/2] zsmalloc: micro-optimize zs_object_copy()

[Minchan Kim]

On Wed, Mar 25, 2015 at 12:24:47AM +0900, Sergey Senozhatsky wrote:
> A micro-optimization. Avoid additional branching and reduce
> (a bit) registry pressure (f.e. s_off += size; d_off += size;
> may be calculated twise: first for >= PAGE_SIZE check and later
> for offset update in "else" clause).
> 
> /scripts/bloat-o-meter shows some improvement
> 
> add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-10 (-10)
> function                          old     new   delta
> zs_object_copy                    550     540     -10
> 
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>

Acked-by: Minchan Kim <minchan@kernel.org>

Thanks.

-- 
Kind regards,
Minchan Kim

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH 2/2] zsmalloc: micro-optimize zs_object_copy()

[Minchan Kim]

On Wed, Mar 25, 2015 at 12:24:47AM +0900, Sergey Senozhatsky wrote:
> A micro-optimization. Avoid additional branching and reduce
> (a bit) registry pressure (f.e. s_off += size; d_off += size;
> may be calculated twise: first for >= PAGE_SIZE check and later
> for offset update in "else" clause).
> 
> /scripts/bloat-o-meter shows some improvement
> 
> add/remove: 0/0 grow/shrink: 0/1 up/down: 0/-10 (-10)
> function                          old     new   delta
> zs_object_copy                    550     540     -10
> 
> Signed-off-by: Sergey Senozhatsky <sergey.senozhatsky@gmail.com>

Acked-by: Minchan Kim <minchan@kernel.org>

Thanks.

-- 
Kind regards,
Minchan Kim