* Re: [PATCH] printk-formats.txt documentation update
From: Namhyung Kim @ 2011-06-09 18:20 UTC (permalink / raw)
To: Andrew Murray
Cc: joe, w.sang, geert, linux-embedded, linux-kernel, trivial,
rdunlap, udknight, Andrew Murray
In-Reply-To: <1307642692-23556-1-git-send-email-amurray@mpcdata.com>
2011-06-09 (목), 19:04 +0100, Andrew Murray:
> From: Andrew Murray <amurray@mpc-data.co.uk>
>
> This patch updates the incomplete documentation concerning the printk
> extended format specifiers.
>
> Signed-off-by: Andrew Murray <amurray@mpc-data.co.uk>
> ---
> Documentation/printk-formats.txt | 119 +++++++++++++++++++++++++++++++++++++-
> 1 files changed, 117 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/printk-formats.txt b/Documentation/printk-formats.txt
> index 1b5a5dd..69713d6 100644
> --- a/Documentation/printk-formats.txt
> +++ b/Documentation/printk-formats.txt
> @@ -9,7 +9,121 @@ If variable is of Type, use printk format specifier:
> size_t %zu or %zx
> ssize_t %zd or %zx
>
> -Raw pointer value SHOULD be printed with %p.
> +Raw pointer value SHOULD be printed with %p. The kernel supports
> +the following extended format specifiers for pointer types:
> +
> +Symbols/Function Pointers:
> +
> + %pF versatile_init+0x0/0x110
> + %pf versatile_init
> + %pS versatile_init+0x0/0x110
> + %ps versatile_init
> + %pB versatile_init+0x0/0x110
> +
-EBADEXAMPLE :)
If %pB is used, the offset part in the output could not be 0. That's the
key (and the only) difference of the %pB specifier than others. Thus the
output would be something like:
%pB prev_fn_of_versatile_init+0x88/0x88
Thanks.
--
Regards,
Namhyung Kim
^ permalink raw reply
* [PATCH] printk-formats.txt documentation update
From: Andrew Murray @ 2011-06-09 18:04 UTC (permalink / raw)
To: joe
Cc: w.sang, geert, linux-embedded, linux-kernel, trivial, rdunlap,
udknight, namhyung, Andrew Murray
In-Reply-To: <BANLkTik14_Zbfb3e9FXx_yUs35kBh6FqFg@mail.gmail.com>
From: Andrew Murray <amurray@mpc-data.co.uk>
This patch updates the incomplete documentation concerning the printk
extended format specifiers.
Signed-off-by: Andrew Murray <amurray@mpc-data.co.uk>
---
Documentation/printk-formats.txt | 119 +++++++++++++++++++++++++++++++++++++-
1 files changed, 117 insertions(+), 2 deletions(-)
diff --git a/Documentation/printk-formats.txt b/Documentation/printk-formats.txt
index 1b5a5dd..69713d6 100644
--- a/Documentation/printk-formats.txt
+++ b/Documentation/printk-formats.txt
@@ -9,7 +9,121 @@ If variable is of Type, use printk format specifier:
size_t %zu or %zx
ssize_t %zd or %zx
-Raw pointer value SHOULD be printed with %p.
+Raw pointer value SHOULD be printed with %p. The kernel supports
+the following extended format specifiers for pointer types:
+
+Symbols/Function Pointers:
+
+ %pF versatile_init+0x0/0x110
+ %pf versatile_init
+ %pS versatile_init+0x0/0x110
+ %ps versatile_init
+ %pB versatile_init+0x0/0x110
+
+ For printing symbols and function pointers. The 'S' and 's' specifiers
+ result in the symbol name with ('S') or without ('s') offsets. Where
+ this is used on a kernel without KALLSYMS - the symbol address is
+ printed instead.
+
+ The 'B' specifier results in the symbol name with offsets and should be
+ used when printing stack backtraces. The specifier takes into
+ consideration the effect of compiler optimisations which may occur
+ when tail-call's are used and marked with the noreturn GCC attribute.
+
+ On ia64, ppc64 and parisc64 architectures function pointers are
+ actually function descriptors which must first be resolved. The 'F' and
+ 'f' specifiers perform this resolution and then provide the same
+ functionality as the 'S' and 's' specifiers.
+
+Kernel Pointers:
+
+ %pK 0x01234567 or 0x0123456789abcdef
+
+ For printing kernel pointers which should be hidden from unprivileged
+ users. The behaviour of %pK depends on the kptr_restrict sysctl - see
+ Documentation/sysctl/kernel.txt for more details.
+
+Struct Resources:
+
+ %pr [mem 0x60000000-0x6fffffff flags 0x2200] or
+ [mem 0x0000000060000000-0x000000006fffffff flags 0x2200]
+ %pR [mem 0x60000000-0x6fffffff pref] or
+ [mem 0x0000000060000000-0x000000006fffffff pref]
+
+ For printing struct resources. The 'R' and 'r' specifiers result in a
+ printed resource with ('R') or without ('r') a decoded flags member.
+
+MAC/FDDI addresses:
+
+ %pM 00:01:02:03:04:05
+ %pMF 00-01-02-03-04-05
+ %pm 000102030405
+
+ For printing 6-byte MAC/FDDI addresses in hex notation. The 'M' and 'm'
+ specifiers result in a printed address with ('M') or without ('m') byte
+ separators. The default byte separator is the colon (':').
+
+ Where FDDI addresses are concerned the 'F' specifier can be used after
+ the 'M' specifier to use dash ('-') separators instead of the default
+ separator.
+
+IPv4 addresses:
+
+ %pI4 1.2.3.4
+ %pi4 001.002.003.004
+ %p[Ii][hnbl]
+
+ For printing IPv4 dot-separated decimal addresses. The 'I4' and 'i4'
+ specifiers result in a printed address with ('i4') or without ('I4')
+ leading zeros.
+
+ The additional 'h', 'n', 'b', and 'l' specifiers are used to specify
+ host, network, big or little endian order addresses respectively. Where
+ no specifier is provided the default network/big endian order is used.
+
+IPv6 addresses:
+
+ %pI6 0001:0002:0003:0004:0005:0006:0007:0008
+ %pi6 00010002000300040005000600070008
+ %pI6c 1:2:3:4:5:6:7:8
+
+ For printing IPv6 network-order 16-bit hex addresses. The 'I6' and 'i6'
+ specifiers result in a printed address with ('I6') or without ('i6')
+ colon-separators. Leading zeros are always used.
+
+ The additional 'c' specifier can be used with the 'I' specifier to
+ print a compressed IPv6 address as described by
+ http://tools.ietf.org/html/rfc5952
+
+UUID/GUID addresses:
+
+ %pUb 00010203-0405-0607-0809-0a0b0c0d0e0f
+ %pUB 00010203-0405-0607-0809-0A0B0C0D0E0F
+ %pUl 03020100-0504-0706-0809-0a0b0c0e0e0f
+ %pUL 03020100-0504-0706-0809-0A0B0C0E0E0F
+
+ For printing 16-byte UUID/GUIDs addresses. The additional 'l', 'L',
+ 'b' and 'B' specifiers are used to specify a little endian order in
+ lower ('l') or upper case ('L') hex characters - and big endian order
+ in lower ('b') or upper case ('B') hex characters.
+
+ Where no additional specifiers are used the default little endian
+ order with lower case hex characters will be printed.
+
+struct va_format:
+
+ %pV
+
+ For printing struct va_format structures. These contain a format string
+ and va_list as follows:
+
+ struct va_format {
+ const char *fmt;
+ va_list *va;
+ };
+
+ Do not use this feature without some mechanism to verify the
+ correctness of the format string and va_list arguments.
u64 SHOULD be printed with %llu/%llx, (unsigned long long):
@@ -32,4 +146,5 @@ Reminder: sizeof() result is of type size_t.
Thank you for your cooperation and attention.
-By Randy Dunlap <rdunlap@xenotime.net>
+By Randy Dunlap <rdunlap@xenotime.net> and
+Andrew Murray <amurray@mpc-data.co.uk>
--
1.7.4.1
^ permalink raw reply related
* Re: [PATCH] Revised patch
From: Joe Perches @ 2011-06-09 17:53 UTC (permalink / raw)
To: Randy Dunlap
Cc: Andrew Murray, w.sang, geert, linux-embedded, linux-kernel,
trivial, udknight, namhyung, Andrew Murray
In-Reply-To: <20110609104716.c8827a51.rdunlap@xenotime.net>
On Thu, 2011-06-09 at 10:47 -0700, Randy Dunlap wrote:
> On Thu, 9 Jun 2011 18:33:46 +0100 Andrew Murray wrote:
> > + For printing IPv6 network-order 16 bit hex addresses. The 'I6' and 'i6'
> 16-bit
byte. octet if pedantry is desired.
^ permalink raw reply
* Re: [PATCH] Revised patch
From: Andrew Murray @ 2011-06-09 17:48 UTC (permalink / raw)
To: Namhyung Kim
Cc: Joe Perches, w.sang, geert, linux-embedded, linux-kernel, trivial,
rdunlap, udknight
In-Reply-To: <1307641645.1326.1.camel@leonhard>
Thanks - I've actually written the %pB documentation but just getting
in a mess with Git and my work flow - standby...
On 9 June 2011 18:47, Namhyung Kim <namhyung@gmail.com> wrote:
> 2011-06-09 (목), 10:40 -0700, Joe Perches:
>> On Thu, 2011-06-09 at 18:33 +0100, Andrew Murray wrote:
>> > Hi,
>> > I've updated this patch based on your feedback. This covers all the recent feedback.
>>
>> Hey Andrew.
>>
>> You're missing the "%pB" backtrace extension.
>>
>
> http://marc.info/?l=linux-kernel&m=130093458112438&w=2
>
> Just for your information. :)
> Thanks.
>
>
> --
> Regards,
> Namhyung Kim
>
>
>
--
Andrew Murray, Embedded Linux Group
MPC Data Limited
e-mail: amurray@mpc-data.co.uk web: www.mpc-data.co.uk
tel: +44 (0) 1225 710600 fax: +44 (0) 1225 710601
ddi: +44 (0) 1225 710665
MPC Data Limited is a company registered in England and Wales with
company number 05507446
Registered Address: County Gate, County Way, Trowbridge, Wiltshire, BA14 7FJ
VAT no: 850625238
The information in this email and in the attached documents is
confidential and may be
legally privileged. Any unauthorized review, copying, disclosure or
distribution is
prohibited and may be unlawful. It is intended solely for the
addressee. Access to this
email by anyone else is unauthorized. If you are not the intended
recipient, please
contact the sender by reply email and destroy all copies of the
original message. When
addressed to our clients any opinions or advice contained in this
email is subject to
the terms and conditions expressed in the governing contract.
^ permalink raw reply
* Re: [PATCH] Revised patch
From: Namhyung Kim @ 2011-06-09 17:47 UTC (permalink / raw)
To: Joe Perches
Cc: Andrew Murray, w.sang, geert, linux-embedded, linux-kernel,
trivial, rdunlap, udknight, Andrew Murray
In-Reply-To: <1307641202.13329.22.camel@Joe-Laptop>
2011-06-09 (목), 10:40 -0700, Joe Perches:
> On Thu, 2011-06-09 at 18:33 +0100, Andrew Murray wrote:
> > Hi,
> > I've updated this patch based on your feedback. This covers all the recent feedback.
>
> Hey Andrew.
>
> You're missing the "%pB" backtrace extension.
>
http://marc.info/?l=linux-kernel&m=130093458112438&w=2
Just for your information. :)
Thanks.
--
Regards,
Namhyung Kim
^ permalink raw reply
* Re: [PATCH] Revised patch
From: Randy Dunlap @ 2011-06-09 17:47 UTC (permalink / raw)
To: Andrew Murray
Cc: joe, w.sang, geert, linux-embedded, linux-kernel, trivial,
udknight, namhyung, Andrew Murray
In-Reply-To: <1307640826-22618-1-git-send-email-amurray@mpcdata.com>
On Thu, 9 Jun 2011 18:33:46 +0100 Andrew Murray wrote:
> Hi,
>
> I've updated this patch based on your feedback. This covers all the recent feedback.
>
> From: Andrew Murray <amurray@mpc-data.co.uk>
>
> This patch updates the incomplete documentation concerning the printk
> extended format specifiers.
Thanks. A few comments below...
> Signed-off-by: Andrew Murray <amurray@mpc-data.co.uk>
>
> ---
> Documentation/printk-formats.txt | 113 +++++++++++++++++++++++++++++++++++++-
> 1 files changed, 111 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/printk-formats.txt b/Documentation/printk-formats.txt
> index 1b5a5dd..2968cbf 100644
> --- a/Documentation/printk-formats.txt
> +++ b/Documentation/printk-formats.txt
> @@ -9,7 +9,115 @@ If variable is of Type, use printk format specifier:
> size_t %zu or %zx
> ssize_t %zd or %zx
>
> -Raw pointer value SHOULD be printed with %p.
> +Raw pointer value SHOULD be printed with %p. The kernel supports
> +the following extended format specifiers for pointer types:
> +
> +Symbols/Function Pointers:
> +
> + %pF versatile_init+0x0/0x110
> + %pf versatile_init
> + %pS versatile_init+0x0/0x110
> + %ps versatile_init
> +
> + For printing symbols and function pointers. The 'S' and 's' specifiers
> + result in the symbol name with ('S') or without ('s') offsets. Where
> + this is used on a kernel without KALLSYMS - the symbol address is
> + printed instead.
> +
> + On ia64, ppc64 and parisc64 architectures function pointers are
> + actually function descriptors which must first be resolved. The 'F' and
> + 'f' specifiers perform this resolution and then provide the same
> + functionality as the 'S' and 's' specifiers.
> +
> +Kernel Pointers:
> +
> + %pK 0x01234567 or 0x0123456789abcdef
> +
> + For printing kernel pointers which should be hidden from unprivileged
> + users. The behaviour of %pK depends on the kptr_restrict sysctl - see
> + Documentation/sysctl/kernel.txt for more details.
> +
> +Struct Resources:
> +
> + %pr [mem 0x60000000-0x6fffffff flags 0x2200] or
> + [mem 0x0000000060000000-0x000000006fffffff flags 0x2200]
> + %pR [mem 0x60000000-0x6fffffff pref] or
> + [mem 0x0000000060000000-0x000000006fffffff pref]
> +
> + For printing struct resources. The 'R' and 'r' specifiers result in a
> + printed resource with ('R') or without ('r') a decoded flags member.
> +
> +MAC/FDDI addresses:
> +
> + %pM 00:01:02:03:04:05
> + %pMF 00-01-02-03-04-05
> + %pm 000102030405
> +
> + For printing 6-byte MAC/FDDI addresses in hex notation. The 'M' and 'm'
> + specifiers result in a printed address with ('M') or without ('m') byte
> + separators. The default byte separator is the colon (':').
> +
> + Where FDDI addresses are concerned the 'F' specifier can be used after
> + the 'M' specifier to use dash ('-') separators instead of the default
> + separator.
> +
> +IPv4 addresses:
> +
> + %pI4 1.2.3.4
> + %pi4 001.002.003.004
> + %p[Ii][hnbl]
> +
> + For printing IPv4 dot-separated decimal addresses. The 'I4' and 'i4'
> + specifiers result in a printed address with ('i4') or without ('I4')
> + leading zeros.
> +
> + The additional 'h', 'n', 'b', and 'l' specifiers are used to specify
> + host, network, big or little endian order addresses respectively. Where
> + no specifier is provided the default network/big endian order is used.
> +
I would hyphenate the -endian types, but won't insist on that.
> +IPv6 addresses:
> +
> + %pI6 0001:0002:0003:0004:0005:0006:0007:0008
> + %pi6 00010002000300040005000600070008
> + %pI6c 1:2:3:4:5:6:7:8
> +
> + For printing IPv6 network-order 16 bit hex addresses. The 'I6' and 'i6'
16-bit
> + specifiers result in a printed address with ('I6') or without ('i6')
> + colon-separators. Leading zeros are always used.
> +
> + The additional 'c' specifier can be used with the 'I' specifier to
> + print a compressed IPv6 address as described by
> + http://tools.ietf.org/html/rfc5952
> +
> +UUID/GUID addresses:
> +
> + %pUb 00010203-0405-0607-0809-0a0b0c0d0e0f
> + %pUB 00010203-0405-0607-0809-0A0B0C0D0E0F
> + %pUl 03020100-0504-0706-0809-0a0b0c0e0e0f
> + %pUL 03020100-0504-0706-0809-0A0B0C0E0E0F
> +
> + For printing 16 byte UUID/GUIDs addresses. The additional 'l', 'L',
16-byte
> + 'b' and 'B' specifiers are used to specify a little endian order in
> + lower ('l') or upper case ('L') hex characters - and big endian order
> + in lower ('b') or upper case ('B') hex characters.
> +
> + Where no additional specifiers are used the default little endian
> + order with lower case hex characters will be printed.
> +
> +struct va_format:
> +
> + %pV
> +
> + For printing struct va_format structures. These contain a format string
> + and va_list as follows:
> +
> + struct va_format {
> + const char *fmt;
> + va_list *va;
> + };
> +
> + Do not use this feature without some mechanism to verify the
> + correctness of the format string and va_list arguments.
>
> u64 SHOULD be printed with %llu/%llx, (unsigned long long):
%pB is missing?
* - 'B' For backtraced symbolic direct pointers with offset
> @@ -32,4 +140,5 @@ Reminder: sizeof() result is of type size_t.
> Thank you for your cooperation and attention.
>
>
> -By Randy Dunlap <rdunlap@xenotime.net>
> +By Randy Dunlap <rdunlap@xenotime.net> and
> +Andrew Murray <amurray@mpc-data.co.uk>
> --
---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***
^ permalink raw reply
* Re: [PATCH] Revised patch
From: Joe Perches @ 2011-06-09 17:40 UTC (permalink / raw)
To: Andrew Murray
Cc: w.sang, geert, linux-embedded, linux-kernel, trivial, rdunlap,
udknight, namhyung, Andrew Murray
In-Reply-To: <1307640826-22618-1-git-send-email-amurray@mpcdata.com>
On Thu, 2011-06-09 at 18:33 +0100, Andrew Murray wrote:
> Hi,
> I've updated this patch based on your feedback. This covers all the recent feedback.
Hey Andrew.
You're missing the "%pB" backtrace extension.
It's be better if the "Hi, I've updated" comment stuff
came after the --- content separator in the patch so that
the commit log is a bit neater.
Otherwise, looks great, thanks.
cheers, Joe
^ permalink raw reply
* [PATCH] Revised patch
From: Andrew Murray @ 2011-06-09 17:33 UTC (permalink / raw)
To: joe
Cc: w.sang, geert, linux-embedded, linux-kernel, trivial, rdunlap,
udknight, namhyung, Andrew Murray, Andrew Murray
In-Reply-To: <AANLkTikbPwMS0VGsMH64CGhDUEmvOwUzYnXSV0q3eW9_@mail.gmail.com>
Hi,
I've updated this patch based on your feedback. This covers all the recent feedback.
From: Andrew Murray <amurray@mpc-data.co.uk>
This patch updates the incomplete documentation concerning the printk
extended format specifiers.
Signed-off-by: Andrew Murray <amurray@mpc-data.co.uk>
---
Documentation/printk-formats.txt | 113 +++++++++++++++++++++++++++++++++++++-
1 files changed, 111 insertions(+), 2 deletions(-)
diff --git a/Documentation/printk-formats.txt b/Documentation/printk-formats.txt
index 1b5a5dd..2968cbf 100644
--- a/Documentation/printk-formats.txt
+++ b/Documentation/printk-formats.txt
@@ -9,7 +9,115 @@ If variable is of Type, use printk format specifier:
size_t %zu or %zx
ssize_t %zd or %zx
-Raw pointer value SHOULD be printed with %p.
+Raw pointer value SHOULD be printed with %p. The kernel supports
+the following extended format specifiers for pointer types:
+
+Symbols/Function Pointers:
+
+ %pF versatile_init+0x0/0x110
+ %pf versatile_init
+ %pS versatile_init+0x0/0x110
+ %ps versatile_init
+
+ For printing symbols and function pointers. The 'S' and 's' specifiers
+ result in the symbol name with ('S') or without ('s') offsets. Where
+ this is used on a kernel without KALLSYMS - the symbol address is
+ printed instead.
+
+ On ia64, ppc64 and parisc64 architectures function pointers are
+ actually function descriptors which must first be resolved. The 'F' and
+ 'f' specifiers perform this resolution and then provide the same
+ functionality as the 'S' and 's' specifiers.
+
+Kernel Pointers:
+
+ %pK 0x01234567 or 0x0123456789abcdef
+
+ For printing kernel pointers which should be hidden from unprivileged
+ users. The behaviour of %pK depends on the kptr_restrict sysctl - see
+ Documentation/sysctl/kernel.txt for more details.
+
+Struct Resources:
+
+ %pr [mem 0x60000000-0x6fffffff flags 0x2200] or
+ [mem 0x0000000060000000-0x000000006fffffff flags 0x2200]
+ %pR [mem 0x60000000-0x6fffffff pref] or
+ [mem 0x0000000060000000-0x000000006fffffff pref]
+
+ For printing struct resources. The 'R' and 'r' specifiers result in a
+ printed resource with ('R') or without ('r') a decoded flags member.
+
+MAC/FDDI addresses:
+
+ %pM 00:01:02:03:04:05
+ %pMF 00-01-02-03-04-05
+ %pm 000102030405
+
+ For printing 6-byte MAC/FDDI addresses in hex notation. The 'M' and 'm'
+ specifiers result in a printed address with ('M') or without ('m') byte
+ separators. The default byte separator is the colon (':').
+
+ Where FDDI addresses are concerned the 'F' specifier can be used after
+ the 'M' specifier to use dash ('-') separators instead of the default
+ separator.
+
+IPv4 addresses:
+
+ %pI4 1.2.3.4
+ %pi4 001.002.003.004
+ %p[Ii][hnbl]
+
+ For printing IPv4 dot-separated decimal addresses. The 'I4' and 'i4'
+ specifiers result in a printed address with ('i4') or without ('I4')
+ leading zeros.
+
+ The additional 'h', 'n', 'b', and 'l' specifiers are used to specify
+ host, network, big or little endian order addresses respectively. Where
+ no specifier is provided the default network/big endian order is used.
+
+IPv6 addresses:
+
+ %pI6 0001:0002:0003:0004:0005:0006:0007:0008
+ %pi6 00010002000300040005000600070008
+ %pI6c 1:2:3:4:5:6:7:8
+
+ For printing IPv6 network-order 16 bit hex addresses. The 'I6' and 'i6'
+ specifiers result in a printed address with ('I6') or without ('i6')
+ colon-separators. Leading zeros are always used.
+
+ The additional 'c' specifier can be used with the 'I' specifier to
+ print a compressed IPv6 address as described by
+ http://tools.ietf.org/html/rfc5952
+
+UUID/GUID addresses:
+
+ %pUb 00010203-0405-0607-0809-0a0b0c0d0e0f
+ %pUB 00010203-0405-0607-0809-0A0B0C0D0E0F
+ %pUl 03020100-0504-0706-0809-0a0b0c0e0e0f
+ %pUL 03020100-0504-0706-0809-0A0B0C0E0E0F
+
+ For printing 16 byte UUID/GUIDs addresses. The additional 'l', 'L',
+ 'b' and 'B' specifiers are used to specify a little endian order in
+ lower ('l') or upper case ('L') hex characters - and big endian order
+ in lower ('b') or upper case ('B') hex characters.
+
+ Where no additional specifiers are used the default little endian
+ order with lower case hex characters will be printed.
+
+struct va_format:
+
+ %pV
+
+ For printing struct va_format structures. These contain a format string
+ and va_list as follows:
+
+ struct va_format {
+ const char *fmt;
+ va_list *va;
+ };
+
+ Do not use this feature without some mechanism to verify the
+ correctness of the format string and va_list arguments.
u64 SHOULD be printed with %llu/%llx, (unsigned long long):
@@ -32,4 +140,5 @@ Reminder: sizeof() result is of type size_t.
Thank you for your cooperation and attention.
-By Randy Dunlap <rdunlap@xenotime.net>
+By Randy Dunlap <rdunlap@xenotime.net> and
+Andrew Murray <amurray@mpc-data.co.uk>
--
1.7.4.1
^ permalink raw reply related
* [ANNOUNCE] memedit-0.8
From: Robert Schwebel @ 2011-06-06 10:46 UTC (permalink / raw)
To: linux-embedded
After 5 years of development, there is a new memedit release out :-)
http://www.pengutronix.de/software/memedit/index_en.html
Memedit is a small tool that can be used to map & access physical memory
on embedded devices.
The new feature is that we have memory fill support now.
rsc
--
Pengutronix e.K. | |
Industrial Linux Solutions | http://www.pengutronix.de/ |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 |
Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
^ permalink raw reply
* Re: usefullness of a read-only block UBI interface ?
From: Ivan Djelic @ 2011-05-24 17:12 UTC (permalink / raw)
To: David Wagner; +Cc: linux-fsdevel, linux-mtd@lists.infradead.org, Linux Embedded
In-Reply-To: <4DDBC64F.5070409@free-electrons.com>
On Tue, May 24, 2011 at 03:53:03PM +0100, David Wagner wrote:
> Hello linux-mtd, -embedded and -fsdevel,
>
> There are a lot of actively developed block filesystems out there, more
> than flash filesystems. Read-only block FS can run with great perfs on
> flash supports with the mtdblock interface (eg. SquashFS) but since it
> doesn't handle bad blocks, read will fail when you hit one.
>
> That's why we are considering the pros and cons of having a block
> interface on top of UBI: UBI takes care of bad blocks and filesystems
> above it don't have to worry about them.
>
> An option could be to implement bad block handling in mtdblock but
> then, there wouldn't be any wear-leveling.
Hello David,
Handling bad blocks is one thing, but it would not be enough. I assume you want
to use a nand device (bad blocks ?). Reading nand pages over and over will
result in bitflips (so-called "read disturbs"). Those bitflips are corrected by
ecc in mtd, but they must also be taken care of at a higher level, by
(atomically) moving faulty data to another block and erasing the original
block (which is enough to clear read disturbs). UBI does that in its block
scrubbing operation.
> In case of read-only filesystems, wear-leveling is not so important but
> when read-only and read-write filesystems coexist, static wear-leveling
> is important. And I understand that UBI implements static
> wear-leveling. So it would make sense to have a block read-only
> filesystem on top of UBI along with a ubifs read-write filesystem.
Yes, especially if your read-only filesystem is very large; you need to spread
wear-levelling across the entire device in order to maximize its lifetime.
Regards,
Ivan
______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/
^ permalink raw reply
* usefullness of a read-only block UBI interface ?
From: David Wagner @ 2011-05-24 14:53 UTC (permalink / raw)
To: linux-mtd@lists.infradead.org, linux-fsdevel, Linux Embedded
Hello linux-mtd, -embedded and -fsdevel,
There are a lot of actively developed block filesystems out there, more
than flash filesystems. Read-only block FS can run with great perfs on
flash supports with the mtdblock interface (eg. SquashFS) but since it
doesn't handle bad blocks, read will fail when you hit one.
That's why we are considering the pros and cons of having a block
interface on top of UBI: UBI takes care of bad blocks and filesystems
above it don't have to worry about them.
An option could be to implement bad block handling in mtdblock but
then, there wouldn't be any wear-leveling.
In case of read-only filesystems, wear-leveling is not so important but
when read-only and read-write filesystems coexist, static wear-leveling
is important. And I understand that UBI implements static
wear-leveling. So it would make sense to have a block read-only
filesystem on top of UBI along with a ubifs read-write filesystem.
So, what do you think about that possibility ? Do you see alternative
approaches or other ways to address the problem of using read-only
oriented filesystems on flash (w/o reinventing the wheel) ?
Regards,
David.
--
David Wagner, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/
^ permalink raw reply
* Re: [PATCH 00/11] RFC: KBUS messaging subsystem
From: Tony Ibbs @ 2011-05-22 19:58 UTC (permalink / raw)
To: Florian Fainelli
Cc: Jonathan Corbet, Grant Likely, lkml, Linux-embedded,
Tibs at Kynesim, Richard Watts
In-Reply-To: <201105171050.38903.florian@openwrt.org>
On 17 May 2011, at 09:50, Florian Fainelli wrote:
> Hello,
>
> Sorry for this late answer.
Not a problem from here, all responses are helpful. In, turn, apologies
for taking so long to reply.
> Most implementations (if not all) involving system-wide message
> delivery for other daemons are running in user-space.
OK. Although I certainly wouldn't claim to have anywhere near a complete
list of such (an annotated list of all the messaging systems on Linux
would be rather interesting, though!).
> If you had in mind that this daemon might be killed under OOM
> conditions, then maybe your whole system has an issue, which
> could be circumvented by making sure the messaging process gets
> respawned when possible (upstart like mechanism or such).
OOM isn't particularly an issue I'd worried about for any part of the
system. Other things tend to cause user processes to crash - using
ffmpeg on random video data, for instance. Of course, that is clearly
not a problem for KBUS itself.
Respawning itself isn't directly a problem, but getting everyone talking
to everyone else again is typically a nasty pain (and one users don't
want to think about), so one tends to want one's messaging handler to be
*very* robust. I think the discipline of working in-kernel helps with
that, although I'd be surprised if that were considered enough reason to
add a new kernel module!
> From: Jonathan Corbet <corbet@lwn.net>
> Date: 22 March 2011 19:36:40 GMT
>
> > Even better might be to just use the socket API.
>
> Indeed, I would also suggest having a look at what generic netlink already
> provides like messages per application PID, multicasting and marshaling.
As I said in an earlier message, I'd ignored netlink because it sounded
as if were intrinsically losssy (no way of not losing messages if a
queue got full) which is a problem for KBUS requests/replies.
On the other hand, understanding netlink from scratch is somewhat
difficult (I've just spent some hours doing more research, and don't
feel like I've begun to get a good idea of its boundaries yet).
I have also been reading the libnl documentation, which seems to make
the userspace end somewhat less complex, and looks like a good thing.
> If you intend to keep a part of it in the kernel, you should have a
> look at this, because from my experience with generic netlink, most of
> the hard job you are re-doing here, has already been done in a generic
> manner.
It looks interesting, but the worrying part of statements like this is
always the "most of".
Is your suggestion that netlink would be a better API than the current
"creating" use of a file API for communicating from user space to the
KBUS kernel module, and then back?
The LWN article http://lwn.net/Articles/131802/ makes that sound
plausible (assuming one can still detect "release" events for netlink
sockets - I assume one can). At first glance I'm not sure how much
harder it is to program such a netlink interface "bare" (without a
userspace library such as libnl) than it is to use the current KBUS
interface in such a manner.
(Aside: a quick look at my current KBUS build shows kbus.ko as 60KB,
libkbus.so (the C userspace library on top of the "raw" usage) as 54KB,
and libnl.so as 277KB - although I don't know how Ubuntu build the
latter, and it obviously also includes all sorts of data description
handling which KBUS deliberately does not. So netlink smaller if "bare",
and bigger, but not a huge amount, if used with its library.)
I'm not entirely sure what happens if either end of the netlink API
doesn't respond in a timely manner - is netlink allowed to throw things
away?
Or did you mean that netlink is appropriate to replace some/much of the
KBUS kernel module as well? In that case I'd have to think about it a
lot more to have an informed opinion.
Anyway.
What I'm working on at the moment is an email in which I try to restate
what we are/were trying to do with KBUS, with simple examples of the
sorts of call we're talking about, and ask if that is a sensible thing
to have in the kernel, emphasising that we are more worried about the
functionality than the API.
If the concept is a good thing but our implementation of it is
objectionable (e.g., we need to rewrite to a less "creative" interface,
be more sockety, or whatever), then so be it, we'll need to rewrite.
If you'd be willing to look at that email when it is posted, I hope it
will be easier to point at specific things and say "yes, that would be
better done with netlink" or, perhaps, "netlink would not address this,
but one might attack it in this way".
Thanks,
Tibs
^ permalink raw reply
* Re: module boot time (was Re: [PATCH] module: Use binary search in lookup_symbol())
From: Jeff Mahoney @ 2011-05-21 14:23 UTC (permalink / raw)
To: Tim Bird; +Cc: Greg KH, LKML, Linux Embedded
In-Reply-To: <4DD6DD39.2090302@am.sony.com>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 05/20/2011 05:29 PM, Tim Bird wrote:
> On 05/19/2011 12:56 PM, Jeff Mahoney wrote:
>> On 05/18/2011 05:34 PM, Greg KH wrote:
>>> I don't think that's worth it, there has been talk, and some initial
>>> code, about adding kernel modules to the kernel image itself, which
>>> would solve a lot of the i/o time of loading modules, and solves some
>>> other boot speed problems. That work was done by Jeff Mahoney, but I
>>> think he ran into some "interesting" linker issues and had to shelve it
>>> due to a lack of time :(
>>
>> I had a few attempts at this before I had to move on to other things. I
>> haven't gotten a chance to take another look.
>>
>> I had two approaches:
>>
>> 1) Statically link in modules post-build. This actually worked but had
>> some large caveats. The first was that an un-relocated image (vmlinux.o)
>> was needed in order to make it work and a ~ 200 MB footprint to gain a
>> fairly small win in boot time didn't seem like a good tradeoff. The
>> other issue was more important and is what made me abandon this
>> approach: If the entire image is re-linked then the debuginfo package
>> that we as a distributor offer but don't typically install becomes
>> invalid. Our support teams would not be too thrilled with the idea of
>> crash dumps that can't be used.
>>
>> 2) Build a "megamodule" that is loaded like an initramfs but is already
>> internally linked and requires no additional userspace. I got the
>> megamodule creation working but didn't get the loading aspect of it done
>> yet.
>>
>> In both cases, I added the regular initcall sections to the modules in
>> addition to the module sections so they'd be loaded in the order they
>> would have been if they were actually statically linked.
>>
>> I hadn't thought about it until now and it may not actually work, but it
>> could be possible to use the megamodule approach *and* link it into a
>> static vmlinux image as an appended section that's optionally used.
>
>
> What was the use case for this? My use case is that I want
> to use all the modules compiled into the kernel, but I don't
> want to run some modules' initcalls until well after kernel
> and user-space startup.
>
> My solution is very simple - create a new initcall macro for
> the initcalls I want to defer, along with a new 'deferred' initcall
> section where the function entries can be accumulated. Then,
> I avoid freeing init memory at standard initcall time. Once
> the main user-space has initialized, it echos to a /proc file
> to cause the deferred initcalls to be called, and the init memory
> to be freed.
>
> I'm attaching the patch below, because it's short enough to
> see what's going on without a lot of digging.
>
> This method eliminates the linking cost for module loading,
> saves the memory overhead of the ELF module format, and gives
> me control over when the deferred modules will be initialized.
> The big downside is that you have to manually change the definition
> for the initcall from 'module_init' to 'deferred_module_init'
> for the modules you want to defer. Maybe there's a simple way
> to control this with a kernel config? That would make this a pretty
> nice, generic, system for deferring module initialization, IMHO.
>
> If your use case is that you want all the modules present, but
> want to initialize only some of them later, then maybe a list of
> module names could be passed into the /proc interface, and the
> routine could selectively initialize the deferred modules.
Yep, it seems like we were going after different use cases. Mine was to
eliminate the initramfs except where userspace portions of it are needed.
We improved the boot time of the openSUSE kernel by statically linking
the most popular drivers into the kernel. I'm not a fan of that approach
for a generic distribution kernel but you can't argue with the numbers.
In the ideal case, the initcalls would happen in the same order they
would if they were statically linked in. Wed get the boot speed of a
static kernel with the flexibility advantages of shipping modules.
Your approach might ultimately dovetail nicely into that but since I
didn't have the time to get something functional, I don't have numbers
of my own.
- -Jeff
- --
Jeff Mahoney
SuSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.16 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org/
iEYEARECAAYFAk3XyvYACgkQLPWxlyuTD7LlYwCgmQv86e0lxu6e5mfvwcevAIDb
7nwAn1jb6694mQ+vkcVD0bDsrKvQXOV1
=Q5xL
-----END PGP SIGNATURE-----
^ permalink raw reply
* Re: module boot time (was Re: [PATCH] module: Use binary search in lookup_symbol())
From: Tim Bird @ 2011-05-20 21:29 UTC (permalink / raw)
To: Jeff Mahoney; +Cc: Greg KH, LKML, Linux Embedded
In-Reply-To: <4DD575EF.2060008@suse.de>
On 05/19/2011 12:56 PM, Jeff Mahoney wrote:
> On 05/18/2011 05:34 PM, Greg KH wrote:
>> I don't think that's worth it, there has been talk, and some initial
>> code, about adding kernel modules to the kernel image itself, which
>> would solve a lot of the i/o time of loading modules, and solves some
>> other boot speed problems. That work was done by Jeff Mahoney, but I
>> think he ran into some "interesting" linker issues and had to shelve it
>> due to a lack of time :(
>
> I had a few attempts at this before I had to move on to other things. I
> haven't gotten a chance to take another look.
>
> I had two approaches:
>
> 1) Statically link in modules post-build. This actually worked but had
> some large caveats. The first was that an un-relocated image (vmlinux.o)
> was needed in order to make it work and a ~ 200 MB footprint to gain a
> fairly small win in boot time didn't seem like a good tradeoff. The
> other issue was more important and is what made me abandon this
> approach: If the entire image is re-linked then the debuginfo package
> that we as a distributor offer but don't typically install becomes
> invalid. Our support teams would not be too thrilled with the idea of
> crash dumps that can't be used.
>
> 2) Build a "megamodule" that is loaded like an initramfs but is already
> internally linked and requires no additional userspace. I got the
> megamodule creation working but didn't get the loading aspect of it done
> yet.
>
> In both cases, I added the regular initcall sections to the modules in
> addition to the module sections so they'd be loaded in the order they
> would have been if they were actually statically linked.
>
> I hadn't thought about it until now and it may not actually work, but it
> could be possible to use the megamodule approach *and* link it into a
> static vmlinux image as an appended section that's optionally used.
What was the use case for this? My use case is that I want
to use all the modules compiled into the kernel, but I don't
want to run some modules' initcalls until well after kernel
and user-space startup.
My solution is very simple - create a new initcall macro for
the initcalls I want to defer, along with a new 'deferred' initcall
section where the function entries can be accumulated. Then,
I avoid freeing init memory at standard initcall time. Once
the main user-space has initialized, it echos to a /proc file
to cause the deferred initcalls to be called, and the init memory
to be freed.
I'm attaching the patch below, because it's short enough to
see what's going on without a lot of digging.
This method eliminates the linking cost for module loading,
saves the memory overhead of the ELF module format, and gives
me control over when the deferred modules will be initialized.
The big downside is that you have to manually change the definition
for the initcall from 'module_init' to 'deferred_module_init'
for the modules you want to defer. Maybe there's a simple way
to control this with a kernel config? That would make this a pretty
nice, generic, system for deferring module initialization, IMHO.
If your use case is that you want all the modules present, but
want to initialize only some of them later, then maybe a list of
module names could be passed into the /proc interface, and the
routine could selectively initialize the deferred modules.
Patch (for 2.6.27 I believe) follows. This is for discussion
only, I wouldn't expect it to apply to mainline.
commit 1fab0d6a932d000780cd232b7d10ebfbe69f477c
Author: Tim Bird <tim.bird@am.sony.com>
Date: Fri Sep 12 11:31:52 2008 -0700
Add deferred_module_init
This allows statically linked modules to be initialized sometime after
the initial bootstrap. To do this, change the module_init() macro
to deferred_module_init(), for those init routines you want to defer.
Signed-off-by: Tim Bird <tim.bird@am.sony.com>
diff --git a/arch/x86/kernel/vmlinux_32.lds.S b/arch/x86/kernel/vmlinux_32.lds.S
index a9b8560..f5bdfc4 100644
--- a/arch/x86/kernel/vmlinux_32.lds.S
+++ b/arch/x86/kernel/vmlinux_32.lds.S
@@ -140,11 +140,21 @@ SECTIONS
*(.con_initcall.init)
__con_initcall_end = .;
}
+ .deferred_initcall.init : AT(ADDR(.deferred_initcall.init) - LOAD_OFFSET) {
+ __def_initcall_start = .;
+ *(.deferred_initcall.init)
+ __def_initcall_end = .;
+ }
.x86_cpu_dev.init : AT(ADDR(.x86_cpu_dev.init) - LOAD_OFFSET) {
__x86_cpu_dev_start = .;
*(.x86_cpu_dev.init)
__x86_cpu_dev_end = .;
}
+ .x86cpuvendor.init : AT(ADDR(.x86cpuvendor.init) - LOAD_OFFSET) {
+ __x86cpuvendor_start = .;
+ *(.x86cpuvendor.init)
+ __x86cpuvendor_end = .;
+ }
SECURITY_INIT
. = ALIGN(4);
.altinstructions : AT(ADDR(.altinstructions) - LOAD_OFFSET) {
diff --git a/fs/proc/proc_misc.c b/fs/proc/proc_misc.c
index 59ea42e..a247a8e 100644
--- a/fs/proc/proc_misc.c
+++ b/fs/proc/proc_misc.c
@@ -703,6 +703,22 @@ static int execdomains_read_proc(char *page, char **start, off_t off,
return proc_calc_metrics(page, start, off, count, eof, len);
}
+extern void do_deferred_initcalls(void);
+
+static int deferred_initcalls_read_proc(char *page, char **start, off_t off,
+ int count, int *eof, void *data)
+{
+ static int deferred_initcalls_done = 0;
+ int len;
+
+ len = sprintf(page, "%d\n", deferred_initcalls_done);
+ if (! deferred_initcalls_done) {
+ do_deferred_initcalls();
+ deferred_initcalls_done = 1;
+ }
+ return proc_calc_metrics(page, start, off, count, eof, len);
+}
+
#ifdef CONFIG_PROC_PAGE_MONITOR
#define KPMSIZE sizeof(u64)
#define KPMMASK (KPMSIZE - 1)
@@ -855,6 +871,7 @@ void __init proc_misc_init(void)
{"filesystems", filesystems_read_proc},
{"cmdline", cmdline_read_proc},
{"execdomains", execdomains_read_proc},
+ {"deferred_initcalls", deferred_initcalls_read_proc},
{NULL,}
};
for (p = simple_ones; p->name; p++)
diff --git a/include/linux/init.h b/include/linux/init.h
index ad63824..ef61767 100644
--- a/include/linux/init.h
+++ b/include/linux/init.h
@@ -200,6 +200,7 @@ extern void (*late_time_init)(void);
#define device_initcall_sync(fn) __define_initcall("6s",fn,6s)
#define late_initcall(fn) __define_initcall("7",fn,7)
#define late_initcall_sync(fn) __define_initcall("7s",fn,7s)
+//#define deferred_initcall(fn) __define_initcall("8",fn,8)
#define __initcall(fn) device_initcall(fn)
@@ -214,6 +215,10 @@ extern void (*late_time_init)(void);
static initcall_t __initcall_##fn \
__used __section(.security_initcall.init) = fn
+#define deferred_initcall(fn) \
+ static initcall_t __initcall_##fn \
+ __used __section(.deferred_initcall.init) = fn
+
struct obs_kernel_param {
const char *str;
int (*setup_func)(char *);
@@ -254,6 +259,7 @@ void __init parse_early_param(void);
* be one per module.
*/
#define module_init(x) __initcall(x);
+#define deferred_module_init(x) deferred_initcall(x);
/**
* module_exit() - driver exit entry point
diff --git a/init/main.c b/init/main.c
index 27f6bf6..e4bbdb2 100644
--- a/init/main.c
+++ b/init/main.c
@@ -789,12 +789,40 @@ static void run_init_process(char *init_filename)
kernel_execve(init_filename, argv_init, envp_init);
}
+extern initcall_t __def_initcall_start[], __def_initcall_end[];
+
+/* call deferred init routines */
+void do_deferred_initcalls(void)
+{
+ initcall_t *call;
+ static int already_run=0;
+
+ if (already_run) {
+ printk("do_deferred_initcalls() has already run\n");
+ return;
+ }
+
+ already_run=1;
+
+ printk("Running do_deferred_initcalls()\n");
+
+ lock_kernel(); /* make environment similar to early boot */
+
+ for(call = __def_initcall_start; call < __def_initcall_end; call++)
+ do_one_initcall(*call);
+
+ flush_scheduled_work();
+
+ free_initmem();
+ unlock_kernel();
+}
+
/* This is a non __init function. Force it to be noinline otherwise gcc
* makes it inline to init() and it becomes part of init.text section
*/
static int noinline init_post(void)
{
- free_initmem();
+ //free_initmem();
unlock_kernel();
mark_rodata_ro();
system_state = SYSTEM_RUNNING;
=============================
Tim Bird
Architecture Group Chair, CE Workgroup of the Linux Foundation
Senior Staff Engineer, Sony Network Entertainment
=============================
^ permalink raw reply related
* Re: module boot time (was Re: [PATCH] module: Use binary search in lookup_symbol())
From: Jeff Mahoney @ 2011-05-19 19:56 UTC (permalink / raw)
To: Greg KH
Cc: Tim Bird, Christoph Hellwig, Alessio Igor Bogani, Rusty Russell,
Anders Kaseorg, Tim Abbott, LKML, Linux Embedded, Jason Wessel,
Dirk Behme
In-Reply-To: <20110518213451.GA23702@kroah.com>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 05/18/2011 05:34 PM, Greg KH wrote:
> On Wed, May 18, 2011 at 02:10:15PM -0700, Tim Bird wrote:
>>>>> And why do people overly care for the load time?
>>>>
>>>> To reduce overall boot time.
>>>
>>> To reduce it even more, build the modules into the kernel :)
>>
>> That's what I do most of the time. For some projects,
>> it is useful to build certain things as modules so you can
>> defer initializing them until later in the boot sequence.
>> You can get some critical user-space tasks running, then
>> come back later to initialize USB and other drivers.
>> On cameras, it's not uncommon to want to get to user
>> space in the first 500 milliseconds.
>
> That's common even on desktops and servers, and using the bootchart code
> in the kernel helps find those bottlenecks.
>
>> Sony has some code which allows us to both statically link
>> drivers and defer their initialization, but it's kind of
>> kludgy and requires modifying the module declarations
>> for the code you want to defer. Let me know if you think
>> this is worth doing an RFC about.
>
> I don't think that's worth it, there has been talk, and some initial
> code, about adding kernel modules to the kernel image itself, which
> would solve a lot of the i/o time of loading modules, and solves some
> other boot speed problems. That work was done by Jeff Mahoney, but I
> think he ran into some "interesting" linker issues and had to shelve it
> due to a lack of time :(
I had a few attempts at this before I had to move on to other things. I
haven't gotten a chance to take another look.
I had two approaches:
1) Statically link in modules post-build. This actually worked but had
some large caveats. The first was that an un-relocated image (vmlinux.o)
was needed in order to make it work and a ~ 200 MB footprint to gain a
fairly small win in boot time didn't seem like a good tradeoff. The
other issue was more important and is what made me abandon this
approach: If the entire image is re-linked then the debuginfo package
that we as a distributor offer but don't typically install becomes
invalid. Our support teams would not be too thrilled with the idea of
crash dumps that can't be used.
2) Build a "megamodule" that is loaded like an initramfs but is already
internally linked and requires no additional userspace. I got the
megamodule creation working but didn't get the loading aspect of it done
yet.
In both cases, I added the regular initcall sections to the modules in
addition to the module sections so they'd be loaded in the order they
would have been if they were actually statically linked.
I hadn't thought about it until now and it may not actually work, but it
could be possible to use the megamodule approach *and* link it into a
static vmlinux image as an appended section that's optionally used.
- -Jeff
- --
Jeff Mahoney
SUSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org/
iEYEARECAAYFAk3Vde8ACgkQLPWxlyuTD7LxPgCeIuBHX+KnedsMzBz3H8JrwsGG
etgAn0J5tEwjnTFuIWFLdhzR9QHpSmq5
=Jtc/
-----END PGP SIGNATURE-----
^ permalink raw reply
* Re: [PATCH] module: Use binary search in lookup_symbol()
From: Rusty Russell @ 2011-05-19 7:26 UTC (permalink / raw)
To: Dirk Behme, Alessio Igor Bogani
Cc: Anders Kaseorg, Tim Abbott, Tim Bird, LKML, Linux Embedded,
Jason Wessel
In-Reply-To: <4DD3E540.8000205@googlemail.com>
On Wed, 18 May 2011 17:26:56 +0200, Dirk Behme <dirk.behme@googlemail.com> wrote:
> The old version with the warning is in linux-next now
>
> http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git;a=commitdiff;h=903996de9b35213aaa4162c24351a2cb2931d9ac
Yep, I switched to the new one after sfr's pull.
Then I switched to the enhanced-comments one just now, but if Linus was
quick to pull he'll have the undercommented version rather than the
overcommented one.
Cheers,
Rusty.
^ permalink raw reply
* Re: module boot time (was Re: [PATCH] module: Use binary search in lookup_symbol())
From: Greg KH @ 2011-05-18 21:34 UTC (permalink / raw)
To: Tim Bird, Jeff Mahoney
Cc: Christoph Hellwig, Alessio Igor Bogani, Rusty Russell,
Anders Kaseorg, Tim Abbott, LKML, Linux Embedded, Jason Wessel,
Dirk Behme
In-Reply-To: <4DD435B7.9090702@am.sony.com>
On Wed, May 18, 2011 at 02:10:15PM -0700, Tim Bird wrote:
> >>> And why do people overly care for the load time?
> >>
> >> To reduce overall boot time.
> >
> > To reduce it even more, build the modules into the kernel :)
>
> That's what I do most of the time. For some projects,
> it is useful to build certain things as modules so you can
> defer initializing them until later in the boot sequence.
> You can get some critical user-space tasks running, then
> come back later to initialize USB and other drivers.
> On cameras, it's not uncommon to want to get to user
> space in the first 500 milliseconds.
That's common even on desktops and servers, and using the bootchart code
in the kernel helps find those bottlenecks.
> Sony has some code which allows us to both statically link
> drivers and defer their initialization, but it's kind of
> kludgy and requires modifying the module declarations
> for the code you want to defer. Let me know if you think
> this is worth doing an RFC about.
I don't think that's worth it, there has been talk, and some initial
code, about adding kernel modules to the kernel image itself, which
would solve a lot of the i/o time of loading modules, and solves some
other boot speed problems. That work was done by Jeff Mahoney, but I
think he ran into some "interesting" linker issues and had to shelve it
due to a lack of time :(
thanks,
greg k-h
^ permalink raw reply
* module boot time (was Re: [PATCH] module: Use binary search in lookup_symbol())
From: Tim Bird @ 2011-05-18 21:10 UTC (permalink / raw)
To: Greg KH
Cc: Christoph Hellwig, Alessio Igor Bogani, Rusty Russell,
Anders Kaseorg, Tim Abbott, LKML, Linux Embedded, Jason Wessel,
Dirk Behme
In-Reply-To: <20110518192110.GB26945@kroah.com>
On 05/18/2011 12:21 PM, Greg KH wrote:
> On Wed, May 18, 2011 at 10:00:12AM -0700, Tim Bird wrote:
>> Carmelo Amoroso reported some good performance gains
>> in this presentation:
>> http://elinux.org/images/1/18/C_AMOROSO_Fast_lkm_loader_ELC-E_2009.pdf
>> (See slide 22).
>>
>> He doesn't report the overall time savings, and
>> he was using a different method (hash tables as opposed to
>> binary search), but I believe the results are comparable
>> to what the binary search enhancement provides.
>>
>> The biggest offenders in his testing were usbcore,
>> ehci_hcd and ohci_hcd.
>
> Why those? The size of them, or something else? They don't seem to
> have very many symbols they need to look up compared to anything else
> that I can tell.
>
> Is something else going on here due to the serialization of the USB
> drivers themselves?
I don't think there's anything wrong with these, compared to
other kernel modules. I just think they stood out (probably
because of size) from the other modules in the small set that
Carmelo tested. In his tests, usbcore was the largest module
by an order of magnitude.
>>> And why do people overly care for the load time?
>>
>> To reduce overall boot time.
>
> To reduce it even more, build the modules into the kernel :)
That's what I do most of the time. For some projects,
it is useful to build certain things as modules so you can
defer initializing them until later in the boot sequence.
You can get some critical user-space tasks running, then
come back later to initialize USB and other drivers.
On cameras, it's not uncommon to want to get to user
space in the first 500 milliseconds.
Sony has some code which allows us to both statically link
drivers and defer their initialization, but it's kind of
kludgy and requires modifying the module declarations
for the code you want to defer. Let me know if you think
this is worth doing an RFC about.
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Workgroup of the Linux Foundation
Senior Staff Engineer, Sony Network Entertainment
=============================
^ permalink raw reply
* [PATCH] module: Use binary search in lookup_symbol()
From: Alessio Igor Bogani @ 2011-05-18 20:35 UTC (permalink / raw)
To: Rusty Russell
Cc: Greg KH, Anders Kaseorg, Tim Abbott, Tim Bird, LKML,
Linux Embedded, Jason Wessel, Dirk Behme, Christoph Hellwig,
Alessio Igor Bogani
In-Reply-To: <BANLkTim2DBkrFbdR=0a=wGqFPX1O7d+C7g@mail.gmail.com>
The function is_exported() with its helper function lookup_symbol() are used to
verify if a provided symbol is effectively exported by the kernel or by the
modules. Now that both have their symbols sorted we can replace a linear search
with a binary search which provide a considerably speed-up.
This work was supported by a hardware donation from the CE Linux Forum.
Signed-off-by: Alessio Igor Bogani <abogani@kernel.org>
Acked-by: Greg Kroah-Hartman <gregkh@suse.de>
---
kernel/module.c | 7 ++-----
1 files changed, 2 insertions(+), 5 deletions(-)
diff --git a/kernel/module.c b/kernel/module.c
index 1e2b657..795bdc7 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -2055,11 +2055,8 @@ static const struct kernel_symbol *lookup_symbol(const char *name,
const struct kernel_symbol *start,
const struct kernel_symbol *stop)
{
- const struct kernel_symbol *ks = start;
- for (; ks < stop; ks++)
- if (strcmp(ks->name, name) == 0)
- return ks;
- return NULL;
+ return bsearch(name, start, stop - start,
+ sizeof(struct kernel_symbol), cmp_name);
}
static int is_exported(const char *name, unsigned long value,
--
1.7.4.1
^ permalink raw reply related
* Re: your mail
From: Alessio Igor Bogani @ 2011-05-18 20:35 UTC (permalink / raw)
To: Greg KH
Cc: Rusty Russell, Tim Bird, Christoph Hellwig, Anders Kaseorg,
Tim Abbott, LKML, Linux Embedded, Jason Wessel, Dirk Behme
In-Reply-To: <20110518192212.GC26945@kroah.com>
Dear Mr. Kroah-Hartman,
2011/5/18 Greg KH <greg@kroah.com>:
[...]
> Care to resend it without all the stuff above so someone (Rusty I guess)
> can apply it?
Sure! It'll follow in few minutes.
Thank you very much!
Ciao,
Alessio
^ permalink raw reply
* Re: your mail
From: Greg KH @ 2011-05-18 19:22 UTC (permalink / raw)
To: Alessio Igor Bogani
Cc: Rusty Russell, Tim Bird, Christoph Hellwig, Anders Kaseorg,
Tim Abbott, LKML, Linux Embedded, Jason Wessel, Dirk Behme
In-Reply-To: <1305744925-8162-1-git-send-email-abogani@kernel.org>
On Wed, May 18, 2011 at 08:55:25PM +0200, Alessio Igor Bogani wrote:
> Dear Mr. Bird, Dear Mr. Kroah-Hartman,
>
> Sorry for my very bad English.
>
> 2011/5/18 Tim Bird <tim.bird@am.sony.com>:
> [...]
> > Alessio - do you have any timings you can share for the speedup?
>
> You can find a little benchmark using ftrace at end of this email:
> https://lkml.org/lkml/2011/4/5/341
>
> > On 05/17/2011 04:22 PM, Greg KH wrote:
> >> On Tue, May 17, 2011 at 10:56:03PM +0200, Alessio Igor Bogani wrote:
> >>> This work was supported by a hardware donation from the CE Linux Forum.
> [...]
> >> Please explain why you make a change, not just who sponsored the change,
> >> that's not very interesting to developers.
>
> You are right. I apologize.
>
> This patch is a missing piece (not essential it is only a further little
> optimization) of this little patchset:
> https://lkml.org/lkml/2011/4/16/48
>
> Unfortunately I forgot to include this patch in the series (my first error)
> then I avoided explaining the changes because I had thought that those were
> already enough explained in the cover-letter of the patchset (my second error).
>
> Sorry for my mistakes.
>
> Is this better?
>
> Subject: [PATCH] module: Use binary search in lookup_symbol()
>
> The function is_exported() with its helper function lookup_symbol() are used to
> verify if a provided symbol is effectively exported by the kernel or by the
> modules. Now that both have their symbols sorted we can replace a linear search
> with a binary search which provide a considerably speed-up.
>
> This work was supported by a hardware donation from the CE Linux Forum.
>
> Signed-off-by: Alessio Igor Bogani <abogani@kernel.org>
Much better, I have no objection to this at all.
Acked-by: Greg Kroah-Hartman <gregkh@suse.de>
Care to resend it without all the stuff above so someone (Rusty I guess)
can apply it?
thanks,
greg k-h
^ permalink raw reply
* Re: [PATCH] module: Use binary search in lookup_symbol()
From: Greg KH @ 2011-05-18 19:21 UTC (permalink / raw)
To: Tim Bird
Cc: Christoph Hellwig, Alessio Igor Bogani, Rusty Russell,
Anders Kaseorg, Tim Abbott, LKML, Linux Embedded, Jason Wessel,
Dirk Behme
In-Reply-To: <4DD3FB1C.3040103@am.sony.com>
On Wed, May 18, 2011 at 10:00:12AM -0700, Tim Bird wrote:
> On 05/18/2011 12:54 AM, Christoph Hellwig wrote:
> > On Tue, May 17, 2011 at 04:33:07PM -0700, Tim Bird wrote:
> >> That said, I can answer Greg's question. This is to speed up
> >> the symbol resolution on module loading. The last numbers I
> >> saw showed a reduction of about 15-20% for the module load
> >> time, for large-ish modules. Of course this is highly dependent
> >> on the size of the modules, what they do at load time, and how many
> >> symbols are looked up to link them into the kernel.
> >
> > How large are these very large modules, and what are good examples for
> > that?
>
> usbcore seems to be a large-ish module whose
> load time is improved by this. More details follow:
Then add the module to the kernel image, that's what a lot of distros do
now to solve this issue.
> I don't know the exact modules, but Alan Jenkins reported a .3
> second reduction in overall boot time, on a EEE PC, presumably
> running a stock Linux distribution, and loading 41 modules.
>
> See http://lkml.org/lkml/2009/11/3/93
That's good to know.
> Carmelo Amoroso reported some good performance gains
> in this presentation:
> http://elinux.org/images/1/18/C_AMOROSO_Fast_lkm_loader_ELC-E_2009.pdf
> (See slide 22).
>
> He doesn't report the overall time savings, and
> he was using a different method (hash tables as opposed to
> binary search), but I believe the results are comparable
> to what the binary search enhancement provides.
>
> The biggest offenders in his testing were usbcore,
> ehci_hcd and ohci_hcd.
Why those? The size of them, or something else? They don't seem to
have very many symbols they need to look up compared to anything else
that I can tell.
Is something else going on here due to the serialization of the USB
drivers themselves?
> > And why do people overly care for the load time?
>
> To reduce overall boot time.
To reduce it even more, build the modules into the kernel :)
I'm not saying I object to this patch, I just want a whole lot more
information in it when submitted as currently there was no justification
for the change at all.
thanks,
greg k-h
^ permalink raw reply
* (unknown),
From: Alessio Igor Bogani @ 2011-05-18 18:55 UTC (permalink / raw)
To: Greg KH, Rusty Russell, Tim Bird, Christoph Hellwig
Cc: Anders Kaseorg, Tim Abbott, LKML, Linux Embedded, Jason Wessel,
Dirk Behme, Alessio Igor Bogani
In-Reply-To: <4DD305B3.3000707@am.sony.com>
Dear Mr. Bird, Dear Mr. Kroah-Hartman,
Sorry for my very bad English.
2011/5/18 Tim Bird <tim.bird@am.sony.com>:
[...]
> Alessio - do you have any timings you can share for the speedup?
You can find a little benchmark using ftrace at end of this email:
https://lkml.org/lkml/2011/4/5/341
> On 05/17/2011 04:22 PM, Greg KH wrote:
>> On Tue, May 17, 2011 at 10:56:03PM +0200, Alessio Igor Bogani wrote:
>>> This work was supported by a hardware donation from the CE Linux Forum.
[...]
>> Please explain why you make a change, not just who sponsored the change,
>> that's not very interesting to developers.
You are right. I apologize.
This patch is a missing piece (not essential it is only a further little
optimization) of this little patchset:
https://lkml.org/lkml/2011/4/16/48
Unfortunately I forgot to include this patch in the series (my first error)
then I avoided explaining the changes because I had thought that those were
already enough explained in the cover-letter of the patchset (my second error).
Sorry for my mistakes.
Is this better?
Subject: [PATCH] module: Use binary search in lookup_symbol()
The function is_exported() with its helper function lookup_symbol() are used to
verify if a provided symbol is effectively exported by the kernel or by the
modules. Now that both have their symbols sorted we can replace a linear search
with a binary search which provide a considerably speed-up.
This work was supported by a hardware donation from the CE Linux Forum.
Signed-off-by: Alessio Igor Bogani <abogani@kernel.org>
---
kernel/module.c | 7 ++-----
1 files changed, 2 insertions(+), 5 deletions(-)
diff --git a/kernel/module.c b/kernel/module.c
index 1e2b657..795bdc7 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -2055,11 +2055,8 @@ static const struct kernel_symbol *lookup_symbol(const char *name,
const struct kernel_symbol *start,
const struct kernel_symbol *stop)
{
- const struct kernel_symbol *ks = start;
- for (; ks < stop; ks++)
- if (strcmp(ks->name, name) == 0)
- return ks;
- return NULL;
+ return bsearch(name, start, stop - start,
+ sizeof(struct kernel_symbol), cmp_name);
}
static int is_exported(const char *name, unsigned long value,
--
Thank you very much!
Ciao,
Alessio
^ permalink raw reply related
* Re: [PATCH] module: Use binary search in lookup_symbol()
From: Tim Bird @ 2011-05-18 17:00 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Greg KH, Alessio Igor Bogani, Rusty Russell, Anders Kaseorg,
Tim Abbott, LKML, Linux Embedded, Jason Wessel, Dirk Behme
In-Reply-To: <20110518075428.GA29998@infradead.org>
On 05/18/2011 12:54 AM, Christoph Hellwig wrote:
> On Tue, May 17, 2011 at 04:33:07PM -0700, Tim Bird wrote:
>> That said, I can answer Greg's question. This is to speed up
>> the symbol resolution on module loading. The last numbers I
>> saw showed a reduction of about 15-20% for the module load
>> time, for large-ish modules. Of course this is highly dependent
>> on the size of the modules, what they do at load time, and how many
>> symbols are looked up to link them into the kernel.
>
> How large are these very large modules, and what are good examples for
> that?
usbcore seems to be a large-ish module whose
load time is improved by this. More details follow:
I don't know the exact modules, but Alan Jenkins reported a .3
second reduction in overall boot time, on a EEE PC, presumably
running a stock Linux distribution, and loading 41 modules.
See http://lkml.org/lkml/2009/11/3/93
Carmelo Amoroso reported some good performance gains
in this presentation:
http://elinux.org/images/1/18/C_AMOROSO_Fast_lkm_loader_ELC-E_2009.pdf
(See slide 22).
He doesn't report the overall time savings, and
he was using a different method (hash tables as opposed to
binary search), but I believe the results are comparable
to what the binary search enhancement provides.
The biggest offenders in his testing were usbcore,
ehci_hcd and ohci_hcd.
> And why do people overly care for the load time?
To reduce overall boot time.
-- Tim
=============================
Tim Bird
Architecture Group Chair, CE Workgroup of the Linux Foundation
Senior Staff Engineer, Sony Network Entertainment
=============================
^ permalink raw reply
* Re: [PATCH] module: Use binary search in lookup_symbol()
From: Dirk Behme @ 2011-05-18 15:26 UTC (permalink / raw)
To: Alessio Igor Bogani
Cc: Rusty Russell, Anders Kaseorg, Tim Abbott, Tim Bird, LKML,
Linux Embedded, Jason Wessel
In-Reply-To: <1305665763-3988-1-git-send-email-abogani@kernel.org>
On 17.05.2011 22:56, Alessio Igor Bogani wrote:
> This work was supported by a hardware donation from the CE Linux Forum.
>
> Signed-off-by: Alessio Igor Bogani<abogani@kernel.org>
> ---
> kernel/module.c | 7 ++-----
> 1 files changed, 2 insertions(+), 5 deletions(-)
>
> diff --git a/kernel/module.c b/kernel/module.c
> index 1e2b657..795bdc7 100644
> --- a/kernel/module.c
> +++ b/kernel/module.c
> @@ -2055,11 +2055,8 @@ static const struct kernel_symbol *lookup_symbol(const char *name,
> const struct kernel_symbol *start,
> const struct kernel_symbol *stop)
> {
> - const struct kernel_symbol *ks = start;
> - for (; ks< stop; ks++)
> - if (strcmp(ks->name, name) == 0)
> - return ks;
> - return NULL;
> + return bsearch(name, start, stop - start,
> + sizeof(struct kernel_symbol), cmp_name);
> }
>
> static int is_exported(const char *name, unsigned long value,
The old version with the warning is in linux-next now
http://git.kernel.org/?p=linux/kernel/git/next/linux-next.git;a=commitdiff;h=903996de9b35213aaa4162c24351a2cb2931d9ac
Best regards
Dirk
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox