* [RFC] gelf - user space probe debugger
From: Jiri Olsa @ 2011-01-24 16:11 UTC (permalink / raw)
To: linux-embedded; +Cc: jolsa
hi,
I started a debugger project. It's based on the user space backport
of optimized kprobes (djprobes) and RSP stub implementation.
The debugger is implemented as a preloaded shared library,
which handles all the debugger work, hence no kernel ptrace
is needed.
It just reached the state where it can handle simple gdb session
of setting/reaching a breakpoint and continue the execution.
Details can be found on http://people.redhat.com/jolsa/gelf/
I'd like to ask for opinions if there'd be users for such thing,
and if it's worth to continue on this..
thanks for any ideas,
jirka
ps. I'm not subscribed to the list, so please include my email
in the answer
^ permalink raw reply
* Re: determine boot device in initrd
From: Rob Landley @ 2011-01-22 2:48 UTC (permalink / raw)
To: Jacob Avraham; +Cc: linux-embedded@vger.kernel.org
In-Reply-To: <A4B10BB8CE8F86468ABF12A5E4A5CC9920F919A3C4@mail>
On 01/20/2011 03:57 AM, Jacob Avraham wrote:
> Hi,
>
> I have a system with several USB disks, each of them is boot-capable.
> Each such device has a kernel and initrd in its filesystem.
> I want to be able to determine in initrd from which USB device it (and the kernel) were
> loaded from, so I can mount the root filesystem from that device.
> Is that doable?
> I don't want to use a static "root=" entry in grub, since I don't know which sd device it will be.
Your bootloader loads the kernel and hands off control to it. Unless
the bootloader informs the kernel where it loaded it from, the kernel
has know way of knowing since it was a separate program that did it.
(Who says it was a local device? It could have been a PXE boot. It may
have been loadlin from Windows that got the kernel off of a network
share. It could be the qemu -kernel option that manually populates the
virtual system's DRAM with a kernel image and jumps to the start of it
when it initializes the virtual CPU.)
One of the motivations behind the "device tree" stuff is that the
bootloader hands the kernel a standardized data structure telling it
where all the hardware is in the system, and one of the things it can
annotate this tree with is "and _this_ one was the boot device". Of
course both the x86 guys and the ARM maintainer have come out against
cross-platform standardization that would require them to change what
they're already doing in any way.
In general, unless the bootloader tells you where it loaded stuff from,
the kernel has no way of knowing. Once upon a time PC hardware had
standardized layouts for stuff and the kernel could use heuristics based
on knowledge of that to go "ah, first hard drive on the first IDE
controller, this partition has the boot flag set, we came from HERE".
Then IBM went "this doesn't help us enumerate hardware in our mainframes
with 30,000 disks: Linux on the dekstop must accept a standardized
mechanism that will inflict the full pain of mainframe device
enumeration on the Linux desktop community or this will never be fixed
for us!" There hasn't really been much of a Linux desktop community
ever since that I've noticed, and device enumeration still sucks, but
that's udev for you.
Anyway, the kernel can try to guess this info by looking at various
devices and seeing if it recognizes any of them. But it's up to you to
annotate your devices with with something it can recognize, and
programming your initrd to look for that annotation. UUID partition
labels are fairly popular.
Rob
^ permalink raw reply
* determine boot device in initrd
From: Jacob Avraham @ 2011-01-20 9:57 UTC (permalink / raw)
To: linux-embedded@vger.kernel.org
Hi,
I have a system with several USB disks, each of them is boot-capable.
Each such device has a kernel and initrd in its filesystem.
I want to be able to determine in initrd from which USB device it (and the kernel) were
loaded from, so I can mount the root filesystem from that device.
Is that doable?
I don't want to use a static "root=" entry in grub, since I don't know which sd device it will be.
Thanks,
Jacob Avraham
^ permalink raw reply
* Re: Re: Boot time: Kernel start parallelization issue?
From: Martin Mueller @ 2011-01-17 12:33 UTC (permalink / raw)
To: Arjan van de Ven, Dirk Behme; +Cc: linux-embedded, Greg KH
[-- Attachment #1: Type: text/plain, Size: 3794 bytes --]
Sent: Mon, 17 Jan 2011
From: Arjan van de Ven<arjan@linux.intel.com>
> On 1/15/2011 11:33 PM, Dirk Behme wrote:
> >
> > Do you talk about
> >
> >
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdif
> f;h=22a9d645677feefd402befd02edd59b122289ef1
> >
> >
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=
> 4ace92fc112c6069b4fcb95a31d3142d4a43ff2a
> >
> >
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=
> 793180570ff2530d133343ceea85648de5f01b02
> >
> >
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=
> f29d3b23238e1955a8094e038c72546e99308e61
> >
> >
> > http://lwn.net/Articles/314808/
> >
> > merged with 2.6.29?
> >
> > With a recent .37 kernel the usage of async_schedule() seems to be
> > implemented for
>
> yep
> >
> > ./drivers/acpi/battery.c
> > ./drivers/s390/block/dasd.c
> > ./drivers/base/power/main.c
> > ./drivers/ata/libata-core.c
> > ./drivers/scsi/sd.c
> > ./drivers/md/raid5.c
> >
> > With this, on an embedded system using none of this, the completely
> > single-threaded start up reported in [2] seems to be reasonable, then?
>
> quite likely.
>
> >
> > This would mean that to improve the issues reported in [2],
> > async_schedule() has to be implemented for the sub systems used,
> > there, too? I.e. the USB init?
>
> USB init is asynchronous internal to USB; this was one of the outcomes
> of a side discussion at the Kernel Summit a year and a half ago or so
> where Alan Stern visualized the problem... and solved it internal to USB.
>
> Now, there is one 100 msec delay that is inherent to USB (mandatory
> delay required by the spec for voltage to stabilize) but that shouldn't
> be in the synchronous path anymore
There is a 100ms electrical/mechanical root debounce delay from USB
specification that is indeed avoided by [1]. What we see is that
there are two (EHCI + OHCI) additional 50ms root port reset delays
required by USB spec resulting in an 100ms delay. Seems that these
two are synchronous(?) See bootgraph pictures attached.
> >>> what kernel are you seeing issues on?
> >>
> >> [1] talks about 2.6.28, [2] talks about 2.6.34.
>
> yeah 2.6.28 is a lost cause ;-) 2.6.34 should be not too bad.
>
> Now, the one thing that's critical for all of this is that we need to do
> this data driven. Using async init is not always as easy as it sounds;
> there are device ordering things that need to be dealt with, and failure
> cases get much more complex as well.
>
> So the first order of business ALWAYS is to get a bootgraph (from
> scripts/bootgraph.pl)... to see which things are big on the one hand,
> but also to see if there is gains to be had. There is absolutely no gain
> possible (in terms of making something async) if it is pretty much the
> last big thing in the boot that you are making asynchronous (since the
> end of kernel boot has to wait for it all anyway); there is only gains
> for async if the expensive operation, when made async, can run in
> parallel with something else expensive that comes later in the kernel init.
>
> The bootgraph will show this opportunity / pitfall very clearly......
Please find in the attachment two bootgraphs from the 2.6.34 system
discussed in [2].
The first is from the unmodified boot. The second with having
"fastboot: turn the USB hostcontroller initcalls into async
initcalls" applied.
Does this make sense?
Many thanks and best regards
Martin
[1] http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=8520f38099ccfdac2147a0852f84ee7a8ee5e197
[2] http://thread.gmane.org/gmane.linux.usb.general/41181
[-- Attachment #2: bootgraph_2_6_34_arjan_usb_fastboot.png --]
[-- Type: image/png, Size: 14773 bytes --]
[-- Attachment #3: bootgraph_2_6_34_umodified.png --]
[-- Type: image/png, Size: 15427 bytes --]
^ permalink raw reply
* Re: Boot time: Kernel start parallelization issue?
From: Arjan van de Ven @ 2011-01-17 0:13 UTC (permalink / raw)
To: Dirk Behme; +Cc: linux-embedded, Greg KH, Martin Mueller
In-Reply-To: <4D329F62.1010505@googlemail.com>
On 1/15/2011 11:33 PM, Dirk Behme wrote:
>
> Do you talk about
>
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=22a9d645677feefd402befd02edd59b122289ef1
>
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=4ace92fc112c6069b4fcb95a31d3142d4a43ff2a
>
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=793180570ff2530d133343ceea85648de5f01b02
>
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=f29d3b23238e1955a8094e038c72546e99308e61
>
>
> http://lwn.net/Articles/314808/
>
> merged with 2.6.29?
>
> With a recent .37 kernel the usage of async_schedule() seems to be
> implemented for
yep
>
> ./drivers/acpi/battery.c
> ./drivers/s390/block/dasd.c
> ./drivers/base/power/main.c
> ./drivers/ata/libata-core.c
> ./drivers/scsi/sd.c
> ./drivers/md/raid5.c
>
> With this, on an embedded system using none of this, the completely
> single-threaded start up reported in [2] seems to be reasonable, then?
quite likely.
>
> This would mean that to improve the issues reported in [2],
> async_schedule() has to be implemented for the sub systems used,
> there, too? I.e. the USB init?
USB init is asynchronous internal to USB; this was one of the outcomes
of a side discussion at the Kernel Summit a year and a half ago or so
where Alan Stern visualized the problem... and solved it internal to USB.
Now, there is one 100 msec delay that is inherent to USB (mandatory
delay required by the spec for voltage to stabilize) but that shouldn't
be in the synchronous path anymore
>
>>> what kernel are you seeing issues on?
>>
>> [1] talks about 2.6.28, [2] talks about 2.6.34.
yeah 2.6.28 is a lost cause ;-) 2.6.34 should be not too bad.
Now, the one thing that's critical for all of this is that we need to do
this data driven. Using async init is not always as easy as it sounds;
there are device ordering things that need to be dealt with, and failure
cases get much more complex as well.
So the first order of business ALWAYS is to get a bootgraph (from
scripts/bootgraph.pl)... to see which things are big on the one hand,
but also to see if there is gains to be had. There is absolutely no gain
possible (in terms of making something async) if it is pretty much the
last big thing in the boot that you are making asynchronous (since the
end of kernel boot has to wait for it all anyway); there is only gains
for async if the expensive operation, when made async, can run in
parallel with something else expensive that comes later in the kernel init.
The bootgraph will show this opportunity / pitfall very clearly......
^ permalink raw reply
* Re: [PATCH 2/2] Squashfs: Add XZ compression configuration option
From: Geert Uytterhoeven @ 2011-01-16 18:56 UTC (permalink / raw)
To: Phillip Lougher
Cc: Linux Kernel Development, linux-fsdevel, Linux Embedded Maillist,
Lasse Collin, Andy Whitcroft
In-Reply-To: <4D01C770.1080301@lougher.demon.co.uk>
On Fri, Dec 10, 2010 at 07:23, Phillip Lougher
<phillip@lougher.demon.co.uk> wrote:
> Geert Uytterhoeven wrote:
>> On Thu, Dec 9, 2010 at 07:11, Phillip Lougher
>> <phillip@lougher.demon.co.uk> wrote:
>>>
>>> --- a/fs/squashfs/Kconfig
>>> +++ b/fs/squashfs/Kconfig
>>> @@ -53,6 +53,22 @@ config SQUASHFS_LZO
>>>
>>> If unsure, say N.
>>>
>>> +config SQUASHFS_XZ
>>> + bool "Include support for XZ compressed file systems"
>>> + depends on SQUASHFS
>>> + default n
>>
>> "default n" is the default, no reason to specify it.
>>
>
> Yes, thanks for pointing that out.
>
> It seems there's a lot of "default n"s in the mainline kernel
>
> % find . -name "Kconfig"| xargs grep "default n" | wc
> 485 1535 18753
>
> including two in arch/m68k ;-)
Thanks, fixed.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
^ permalink raw reply
* Re: Boot time: Kernel start parallelization issue?
From: Dirk Behme @ 2011-01-16 7:33 UTC (permalink / raw)
To: Arjan van de Ven; +Cc: linux-embedded, Greg KH, Martin Mueller
In-Reply-To: <4D31C8C5.6010306@googlemail.com>
On 15.01.2011 17:18, Dirk Behme wrote:
> On 15.01.2011 16:41, Arjan van de Ven wrote:
>> On 1/15/2011 12:40 AM, Dirk Behme wrote:
>>>
>>> There are some reports [1] [2] looking at the boot time of embedded
>>> (ARM?) systems using initcall debug [3]. Both reports seem to show
>>> that they have issues with the start up sequence of the kernel being
>>> completely single-threaded. In [2] Greg mentions that on a x86 box
>>> multi-threads are happening and that there he doesn't see this
>>> issue. On the other hand, both reports mention Arjan's async
>>> initcall patches [4] to help against the issue. I.e. introducing
>>> some parallelization (on ARM) does help, too.
>>>
>>> With this, I wonder
>>>
>>> - if anybody faces similar issues with single-threaded only kernel
>>> start on embedded (ARM?) systems? Or if this is known? Or if there
>>> are fixes for this?
>>>
>>> - if we somehow should try to 're-activate' Arjan's async initcall
>>> patches?
>>
>> those patches are obsoleted by the merged async_schedule() calls.
>
> Any link (patch name, git link) to be able to check for
> async_schedule() calls?
Do you talk about
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=22a9d645677feefd402befd02edd59b122289ef1
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=4ace92fc112c6069b4fcb95a31d3142d4a43ff2a
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=793180570ff2530d133343ceea85648de5f01b02
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=f29d3b23238e1955a8094e038c72546e99308e61
http://lwn.net/Articles/314808/
merged with 2.6.29?
With a recent .37 kernel the usage of async_schedule() seems to be
implemented for
./drivers/acpi/battery.c
./drivers/s390/block/dasd.c
./drivers/base/power/main.c
./drivers/ata/libata-core.c
./drivers/scsi/sd.c
./drivers/md/raid5.c
With this, on an embedded system using none of this, the completely
single-threaded start up reported in [2] seems to be reasonable, then?
This would mean that to improve the issues reported in [2],
async_schedule() has to be implemented for the sub systems used,
there, too? I.e. the USB init?
>> what kernel are you seeing issues on?
>
> [1] talks about 2.6.28, [2] talks about 2.6.34.
Thanks
Dirk
> [1]
> http://www.lindusembedded.com/blog/2010/06/02/measuring-the-boot-time-of-an-embedded-linux-device/
>
>
> [2] http://thread.gmane.org/gmane.linux.usb.general/41181 (mainly the
> last mail of this thread:
> http://article.gmane.org/gmane.linux.usb.general/41619)
^ permalink raw reply
* Re: Boot time: Kernel start parallelization issue?
From: Dirk Behme @ 2011-01-15 16:18 UTC (permalink / raw)
To: Arjan van de Ven; +Cc: linux-embedded, Greg KH, Martin Mueller
In-Reply-To: <4D31C034.8040105@linux.intel.com>
On 15.01.2011 16:41, Arjan van de Ven wrote:
> On 1/15/2011 12:40 AM, Dirk Behme wrote:
>>
>> There are some reports [1] [2] looking at the boot time of embedded
>> (ARM?) systems using initcall debug [3]. Both reports seem to show
>> that they have issues with the start up sequence of the kernel being
>> completely single-threaded. In [2] Greg mentions that on a x86 box
>> multi-threads are happening and that there he doesn't see this
>> issue. On the other hand, both reports mention Arjan's async
>> initcall patches [4] to help against the issue. I.e. introducing
>> some parallelization (on ARM) does help, too.
>>
>> With this, I wonder
>>
>> - if anybody faces similar issues with single-threaded only kernel
>> start on embedded (ARM?) systems? Or if this is known? Or if there
>> are fixes for this?
>>
>> - if we somehow should try to 're-activate' Arjan's async initcall
>> patches?
>
> those patches are obsoleted by the merged async_schedule() calls.
Any link (patch name, git link) to be able to check for
async_schedule() calls?
> what kernel are you seeing issues on?
[1] talks about 2.6.28, [2] talks about 2.6.34.
Best regards
Dirk
[1]
http://www.lindusembedded.com/blog/2010/06/02/measuring-the-boot-time-of-an-embedded-linux-device/
[2] http://thread.gmane.org/gmane.linux.usb.general/41181 (mainly the
last mail of this thread:
http://article.gmane.org/gmane.linux.usb.general/41619)
^ permalink raw reply
* Re: Boot time: Kernel start parallelization issue?
From: Arjan van de Ven @ 2011-01-15 15:41 UTC (permalink / raw)
To: Dirk Behme; +Cc: linux-embedded, Greg KH, Martin Mueller
In-Reply-To: <4D315D86.40401@googlemail.com>
On 1/15/2011 12:40 AM, Dirk Behme wrote:
>
> There are some reports [1] [2] looking at the boot time of embedded
> (ARM?) systems using initcall debug [3]. Both reports seem to show
> that they have issues with the start up sequence of the kernel being
> completely single-threaded. In [2] Greg mentions that on a x86 box
> multi-threads are happening and that there he doesn't see this issue.
> On the other hand, both reports mention Arjan's async initcall patches
> [4] to help against the issue. I.e. introducing some parallelization
> (on ARM) does help, too.
>
> With this, I wonder
>
> - if anybody faces similar issues with single-threaded only kernel
> start on embedded (ARM?) systems? Or if this is known? Or if there are
> fixes for this?
>
> - if we somehow should try to 're-activate' Arjan's async initcall
> patches?
those patches are obsoleted by the merged async_schedule() calls.
what kernel are you seeing issues on?
if it's something ancient, you should update first.
^ permalink raw reply
* Boot time: Kernel start parallelization issue?
From: Dirk Behme @ 2011-01-15 8:40 UTC (permalink / raw)
To: linux-embedded; +Cc: Arjan van de Ven, Greg KH, Martin Mueller
There are some reports [1] [2] looking at the boot time of embedded
(ARM?) systems using initcall debug [3]. Both reports seem to show
that they have issues with the start up sequence of the kernel being
completely single-threaded. In [2] Greg mentions that on a x86 box
multi-threads are happening and that there he doesn't see this issue.
On the other hand, both reports mention Arjan's async initcall patches
[4] to help against the issue. I.e. introducing some parallelization
(on ARM) does help, too.
With this, I wonder
- if anybody faces similar issues with single-threaded only kernel
start on embedded (ARM?) systems? Or if this is known? Or if there are
fixes for this?
- if we somehow should try to 're-activate' Arjan's async initcall
patches?
Any ideas?
Many thanks and best regards
Dirk
P.S.: I couldn't find a mail address of Alex Gonzalez, the author of
[1], to put him into CC. If anybody knows him, please feel free to add
him to this thread. Thanks!
[1]
http://www.lindusembedded.com/blog/2010/06/02/measuring-the-boot-time-of-an-embedded-linux-device/
[2] http://thread.gmane.org/gmane.linux.usb.general/41181 (mainly the
last mail of this thread:
http://article.gmane.org/gmane.linux.usb.general/41619)
[3] http://www.elinux.org/Initcall_Debug
[4] http://lwn.net/Articles/299591/
^ permalink raw reply
* Re: [PATCH 00/17] pramfs: persistent and protected RAM filesystem
From: Marco Stornelli @ 2011-01-11 19:01 UTC (permalink / raw)
To: sfr, Andrew Morton, dwmw2, mpm
Cc: Linux Kernel, Linux FS Devel, Linux Embedded, Tim Bird
In-Reply-To: <4D25AEEE.1050401@gmail.com>
Hi all,
On 06/01/2011 13:00, Marco Stornelli wrote:
> Hi all,
>
> after several reviews is time to submit the code for mainline. Thanks to
> CELF to believe and support actively the project and thanks to Tim Bird.
>
> Here the stats:
>
> Documentation/filesystems/pramfs.txt | 179 ++++++
> Documentation/filesystems/xip.txt | 2 +
> arch/Kconfig | 3 +
> arch/x86/Kconfig | 1 +
> fs/Kconfig | 8 +-
> fs/Makefile | 1 +
> fs/pramfs/Kconfig | 72 +++
> fs/pramfs/Makefile | 14 +
> fs/pramfs/acl.c | 433 +++++++++++++
> fs/pramfs/acl.h | 86 +++
> fs/pramfs/balloc.c | 147 +++++
> fs/pramfs/desctree.c | 181 ++++++
> fs/pramfs/desctree.h | 44 ++
> fs/pramfs/dir.c | 208 +++++++
> fs/pramfs/file.c | 326 ++++++++++
> fs/pramfs/inode.c | 848 ++++++++++++++++++++++++++
> fs/pramfs/ioctl.c | 121 ++++
> fs/pramfs/namei.c | 371 ++++++++++++
> fs/pramfs/pram.h | 269 +++++++++
> fs/pramfs/pramfs_test.c | 47 ++
> fs/pramfs/super.c | 940 +++++++++++++++++++++++++++++
> fs/pramfs/symlink.c | 76 +++
> fs/pramfs/wprotect.c | 41 ++
> fs/pramfs/wprotect.h | 151 +++++
> fs/pramfs/xattr.c | 1104 ++++++++++++++++++++++++++++++++++
> fs/pramfs/xattr.h | 131 ++++
> fs/pramfs/xattr_security.c | 78 +++
> fs/pramfs/xattr_trusted.c | 65 ++
> fs/pramfs/xattr_user.c | 68 +++
> fs/pramfs/xip.c | 83 +++
> fs/pramfs/xip.h | 28 +
> include/linux/magic.h | 1 +
> include/linux/pram_fs.h | 130 ++++
> include/linux/pram_fs_sb.h | 45 ++
> 34 files changed, 6299 insertions(+), 3 deletions(-)
>
Can I have a feedback to insert this fs in the linux-next? From this
latest review I received some comments about how to fix some fs problems
on-the-fly without to clean all and restart. My next step will be to
write a simple fsck.pram to allow to fix these situations. However I
think the kernel code it's mature to be inserted in a pre-mainline
container. If I'll receive a positive feedback to my question, I'll
repost the patches to the last version (I updated the fs to be RCU aware
according to the work of Nick).
Marco
^ permalink raw reply
* Re: LinuxCon Europe 2011 == ELCE 2011?
From: Eric Bénard @ 2011-01-11 17:33 UTC (permalink / raw)
To: linux-embedded; +Cc: Dirk Behme
In-Reply-To: <4D2C8E7B.9050600@googlemail.com>
Hi Dirk,
On 11/01/2011 18:08, Dirk Behme wrote:
>
> Is the LinuxCon Europe October 26 - 28, 2011, Prague
>
> http://events.linuxfoundation.org/events/linuxcon-europe
>
> now the same as the Embedded Linux Conference Europe (ELCE) 2011?
>
> There is some rumor that these were merged. But the above page doesn't even
> mention the string 'embedded'.
>
it's mentioned in the sponsorship document :
http://events.linuxfoundation.org/_sponsorship/lf_2011_sponsorship.pdf
Eric
^ permalink raw reply
* LinuxCon Europe 2011 == ELCE 2011?
From: Dirk Behme @ 2011-01-11 17:08 UTC (permalink / raw)
To: linux-embedded
Is the LinuxCon Europe October 26 - 28, 2011, Prague
http://events.linuxfoundation.org/events/linuxcon-europe
now the same as the Embedded Linux Conference Europe (ELCE) 2011?
There is some rumor that these were merged. But the above page doesn't
even mention the string 'embedded'.
Thanks
Dirk
^ permalink raw reply
* Re: [PATCH 01/17] pramfs: documentation
From: Roberto A. Foglietta @ 2011-01-11 15:42 UTC (permalink / raw)
To: Marco Stornelli
Cc: Tony Luck, Linux Kernel, Linux Embedded, Linux FS Devel, Tim Bird
In-Reply-To: <4D281D65.1070102@gmail.com>
2011/1/8 Marco Stornelli <marco.stornelli@gmail.com>:
> On 07/01/2011 22:59, Tony Luck wrote:
>> On Fri, Jan 7, 2011 at 12:30 PM, Marco Stornelli
>> <marco.stornelli@gmail.com> wrote:
>>> constraint). About the errors: pramfs does not maintain file data in the
>>> page caches for normal file I/O, so no writeback, the read/write
>>> operation are done with direct io and they are always sync. The data are
>>> write protected in hw when the arch provide this facility (x86 does).
>>> Inode contains a checksum and when there are problems they are marked as
>>> bad. Superblock contains checksum and there is a redundant superblock.
>>
>> But you can still get pramfs inconsistencies if the system crashes at an
>> inopportune moment. E.g. when making files you write the new inode to
>> pramfs, and then you insert the entry into the directory. A crash between
>> these two operations leaves an allocated inode that doesn't appear in
>> any directory. Without a fsck option, it will be hard to see that you have
>> this problem, and your only recovery option is to wipe *all* files by making
>> a new filesystem.
>
> Is it a problem if you lost some logs? However do you expect that fsck
> in this case will drop the inode?
IF there could be some inconsistencies in the file-system AND as long
as there is no way to fixup these inconsistencies than purging their
allocated space THEN I think the best approach would be clearing these
inconsistencies at the mount time and printing a WARNING message for
debug/stats purpose. Otherwise a user-space tool would be better
because it could be used in interactive mode, also.
Obviously the best would be to not have any inconsistencies at all.
However, in a real world, the thread-off between a journaling fs and a
simpler one in terms of code and memory usage could make acceptable
adopting a simpler fs than a journaled one. Kernel documentation
should inform clearly the user about pro/cons of adopting a simpler fs
especially about data loss conditions.
-RAF
^ permalink raw reply
* Re: [PATCH 00/17] pramfs: persistent and protected RAM filesystem
From: Pavel Machek @ 2011-01-11 13:33 UTC (permalink / raw)
To: Marco Stornelli
Cc: Luck, Tony, Peter Zijlstra, Linux Kernel, Linux FS Devel,
Linux Embedded, Tim Bird
In-Reply-To: <4D260A96.4020400@gmail.com>
> Il 06/01/2011 19:22, Luck, Tony ha scritto:
> >> Errata corrige: maybe I used the wrong term, I meant "volatile" instead
> >> of "temporary" information, i.e. I'd like to save this info to re-read
> >> it later but I don't want to store it in flash, a simple log, run-time
> >> information for debug like a flight-recorder or whatever you want.
> >
> > I'm puzzled by the use of "a generic piece of memory" to store "persistent"
> > things (Perhaps this is made clear in the 17 parts of the patch? I haven't
> > read them yet). On x86 f/w typically clears all of memory on reset ... so
> > you only get persistence if you use kexec to get from the old kernel to
> > the new one.
> >
> > -Tony
> >
>
> First of all, you can find a lot of information on the web site where
> there is an overview and a page with implementation details, benchmark
> and so on. With "a generic piece of memory" I mean a generic memory
> device directly addressable. Usually this generic device is an NVRAM, so
> we have a persistent store. If you haven't got this hw you can use other
> devices or the classic RAM, in this case you have a fs persistent only
> over reboot. The use of this fs is mainly for embedded systems, fw can
> be configured to not clear *all* the memory. Pramfs is indeed supported
> by U-Boot, you can see CONFIG_PRAM in the Das U-Boot manual. x86 in this
> case can be a "strange" world for this fs, but however if the user wants
> it can be used without problems because there aren't neither strict arch
> or hw dependency.
Actually you probably want to state u-boot compatibility in the commit
message. AFAICT, that is quite good argument _for_ inclusion.
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
^ permalink raw reply
* Re: [PATCH 01/17] pramfs: documentation
From: Marco Stornelli @ 2011-01-10 18:17 UTC (permalink / raw)
To: Luck, Tony
Cc: Pavel Machek, Linux Kernel, Linux Embedded, Linux FS Devel,
Tim Bird
In-Reply-To: <987664A83D2D224EAE907B061CE93D5301940801B2@orsmsx505.amr.corp.intel.com>
Il 10/01/2011 18:35, Luck, Tony ha scritto:
>> You'd be better running ext2 over special block device,
>> it is quite simple.
>
> Marco,
>
> You might want to spend some more time answering this question
> (it is a particularly good one). What are the reasons to use
> pramfs, rather than a ext2 over a mem<->block driver. You covered
> some in your part 0 patch (like ext2 wastes time getting optimal
> block placement for rotating media). But it might be a good idea
> to go back over them here. From my (lightweight) reading of your
> code, it looks like the biggest benefit is avoiding duplicating
> the data in the pramfs memory region and the VM page cache ...
> which is a big deal for your target audience of hand held devices
> where memory is a somewhat scarce resource. But you probably
> have other goodness in there too.
>
> -Tony
>
I can add that you can "place" the fs wherever you want, ext2 not
without to build something "special" as Pavel said. Sincerely I don't
know what other add. I think documentation, web site information and
benchmark say all. You have got a fs that it's simple, it doesn't
consume a lot of resources (you can do a fine tuning via N and bpi
options for the metadata space for example), better in performance in
this "environment", with the memory protection feature when
available....other? I could write a piece of code that it turn on your
coffee machine at morning, what do you think? :)
Marco
^ permalink raw reply
* RE: [PATCH 01/17] pramfs: documentation
From: Luck, Tony @ 2011-01-10 17:35 UTC (permalink / raw)
To: Pavel Machek, Marco Stornelli
Cc: Linux Kernel, Linux Embedded, Linux FS Devel, Tim Bird
In-Reply-To: <20110110080839.GA16066@atrey.karlin.mff.cuni.cz>
> You'd be better running ext2 over special block device,
> it is quite simple.
Marco,
You might want to spend some more time answering this question
(it is a particularly good one). What are the reasons to use
pramfs, rather than a ext2 over a mem<->block driver. You covered
some in your part 0 patch (like ext2 wastes time getting optimal
block placement for rotating media). But it might be a good idea
to go back over them here. From my (lightweight) reading of your
code, it looks like the biggest benefit is avoiding duplicating
the data in the pramfs memory region and the VM page cache ...
which is a big deal for your target audience of hand held devices
where memory is a somewhat scarce resource. But you probably
have other goodness in there too.
-Tony
^ permalink raw reply
* Re: [PATCH] Move an assert under DEBUG_KERNEL. (attempt 2)
From: Randy Dunlap @ 2011-01-10 17:19 UTC (permalink / raw)
To: Rob Landley; +Cc: Andrew Morton, trivial, linux-kernel, linux-embedded, netdev
In-Reply-To: <4D2AC4D5.5070503@parallels.com>
On Mon, 10 Jan 2011 02:35:33 -0600 Rob Landley wrote:
> On 01/06/2011 05:41 PM, Andrew Morton wrote:
> > Probably a worthwhile thing to do, IMO. If there's some net-specific
> > CONFIG_DEBUG_ setting then that wold be a better thing to use.
> >
> > However the patch was a) wordwrapped, b) space-stuffed and c) not cc'ed
> > to the networking list. So its prospects are dim.
>
> Ok, either I've beaten thunderbird into submission, or I'll be submitting a patch to Documentation/email-clients.txt. (Whether or not I need to find a different smtp server to send this through remains an open question.)
>
> (Confirming: I looked for a more specific DEBUG symbol, but couldn't find one. I can add one, but this seems a bit small to have its own symbol, and DEBUG_KERNEL is already used in a few *.c files.)
>
> From: Rob Landley <rlandley@parallels.com>
>
> Move an assert under DEBUG_KERNEL. (Minor cleanup to save a few bytes.)
>
> Signed-off-by: Rob Landley <rlandley@parallels.com>
> ---
>
> include/linux/rtnetlink.h | 4 ++++
> 1 file changed, 4 insertions(+)
Hm, cc to linux-net should be to netdev instead (so I changed it).
> diff --git a/include/linux/rtnetlink.h b/include/linux/rtnetlink.h
> index bbad657..28c4025 100644
> --- a/include/linux/rtnetlink.h
> +++ b/include/linux/rtnetlink.h
> @@ -782,6 +782,7 @@ extern struct netdev_queue *dev_ingress_queue_create(struct net_device *dev);
> extern void rtnetlink_init(void);
> extern void __rtnl_unlock(void);
>
> +#ifdef CONFIG_DEBUG_KERNEL
> #define ASSERT_RTNL() do { \
> if (unlikely(!rtnl_is_locked())) { \
> printk(KERN_ERR "RTNL: assertion failed at %s (%d)\n", \
> @@ -789,6 +790,9 @@ extern void __rtnl_unlock(void);
> dump_stack(); \
> } \
> } while(0)
> +#else
> +#define ASSERT_RTNL()
> +#endif
Empty macros in Linux usually take the (preferred) form of this one from kernel.h:
#define might_resched() do { } while (0)
although it's up to DaveM in this case.
>
> static inline u32 rtm_get_table(struct rtattr **rta, u8 table)
> {
> --
---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***
^ permalink raw reply
* Re: [PATCH] Move an assert under DEBUG_KERNEL. (attempt 2)
From: Randy Dunlap @ 2011-01-10 17:15 UTC (permalink / raw)
To: Rob Landley
Cc: Andrew Morton, trivial, linux-net, linux-kernel, linux-embedded
In-Reply-To: <4D2AC4D5.5070503@parallels.com>
On Mon, 10 Jan 2011 02:35:33 -0600 Rob Landley wrote:
> On 01/06/2011 05:41 PM, Andrew Morton wrote:
> > Probably a worthwhile thing to do, IMO. If there's some net-specific
> > CONFIG_DEBUG_ setting then that wold be a better thing to use.
> >
> > However the patch was a) wordwrapped, b) space-stuffed and c) not cc'ed
> > to the networking list. So its prospects are dim.
>
> Ok, either I've beaten thunderbird into submission, or I'll be submitting a patch to Documentation/email-clients.txt. (Whether or not I need to find a different smtp server to send this through remains an open question.)
Hi Rob,
The patch applies cleanly. However, if you have any additions/updates to
Documentation/email-clients.txt, please send them along.
and please do hard CR/LF every 70-72 characters or so, instead of very_long_lines.
> (Confirming: I looked for a more specific DEBUG symbol, but couldn't find one. I can add one, but this seems a bit small to have its own symbol, and DEBUG_KERNEL is already used in a few *.c files.)
>
> From: Rob Landley <rlandley@parallels.com>
>
> Move an assert under DEBUG_KERNEL. (Minor cleanup to save a few bytes.)
>
> Signed-off-by: Rob Landley <rlandley@parallels.com>
> ---
>
> include/linux/rtnetlink.h | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/include/linux/rtnetlink.h b/include/linux/rtnetlink.h
> index bbad657..28c4025 100644
> --- a/include/linux/rtnetlink.h
> +++ b/include/linux/rtnetlink.h
> @@ -782,6 +782,7 @@ extern struct netdev_queue *dev_ingress_queue_create(struct net_device *dev);
> extern void rtnetlink_init(void);
> extern void __rtnl_unlock(void);
>
> +#ifdef CONFIG_DEBUG_KERNEL
> #define ASSERT_RTNL() do { \
> if (unlikely(!rtnl_is_locked())) { \
> printk(KERN_ERR "RTNL: assertion failed at %s (%d)\n", \
> @@ -789,6 +790,9 @@ extern void __rtnl_unlock(void);
> dump_stack(); \
> } \
> } while(0)
> +#else
> +#define ASSERT_RTNL()
> +#endif
>
> static inline u32 rtm_get_table(struct rtattr **rta, u8 table)
> {
> --
---
~Randy
*** Remember to use Documentation/SubmitChecklist when testing your code ***
^ permalink raw reply
* [PATCH] Move an assert under DEBUG_KERNEL. (attempt 2)
From: Rob Landley @ 2011-01-10 8:35 UTC (permalink / raw)
To: Andrew Morton; +Cc: trivial, linux-net, linux-kernel, linux-embedded
In-Reply-To: <20110106154120.b69118c9.akpm@linux-foundation.org>
On 01/06/2011 05:41 PM, Andrew Morton wrote:
> Probably a worthwhile thing to do, IMO. If there's some net-specific
> CONFIG_DEBUG_ setting then that wold be a better thing to use.
>
> However the patch was a) wordwrapped, b) space-stuffed and c) not cc'ed
> to the networking list. So its prospects are dim.
Ok, either I've beaten thunderbird into submission, or I'll be submitting a patch to Documentation/email-clients.txt. (Whether or not I need to find a different smtp server to send this through remains an open question.)
(Confirming: I looked for a more specific DEBUG symbol, but couldn't find one. I can add one, but this seems a bit small to have its own symbol, and DEBUG_KERNEL is already used in a few *.c files.)
From: Rob Landley <rlandley@parallels.com>
Move an assert under DEBUG_KERNEL. (Minor cleanup to save a few bytes.)
Signed-off-by: Rob Landley <rlandley@parallels.com>
---
include/linux/rtnetlink.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/include/linux/rtnetlink.h b/include/linux/rtnetlink.h
index bbad657..28c4025 100644
--- a/include/linux/rtnetlink.h
+++ b/include/linux/rtnetlink.h
@@ -782,6 +782,7 @@ extern struct netdev_queue *dev_ingress_queue_create(struct net_device *dev);
extern void rtnetlink_init(void);
extern void __rtnl_unlock(void);
+#ifdef CONFIG_DEBUG_KERNEL
#define ASSERT_RTNL() do { \
if (unlikely(!rtnl_is_locked())) { \
printk(KERN_ERR "RTNL: assertion failed at %s (%d)\n", \
@@ -789,6 +790,9 @@ extern void __rtnl_unlock(void);
dump_stack(); \
} \
} while(0)
+#else
+#define ASSERT_RTNL()
+#endif
static inline u32 rtm_get_table(struct rtattr **rta, u8 table)
{
^ permalink raw reply related
* Re: [PATCH 01/17] pramfs: documentation
From: Marco Stornelli @ 2011-01-10 8:14 UTC (permalink / raw)
To: Pavel Machek
Cc: Tony Luck, Linux Kernel, Linux Embedded, Linux FS Devel, Tim Bird
In-Reply-To: <20110110080839.GA16066@atrey.karlin.mff.cuni.cz>
2011/1/10 Pavel Machek <pavel@ucw.cz>:
>> On 07/01/2011 22:59, Tony Luck wrote:
>> > On Fri, Jan 7, 2011 at 12:30 PM, Marco Stornelli
>> > <marco.stornelli@gmail.com> wrote:
>> >> constraint). About the errors: pramfs does not maintain file data in the
>> >> page caches for normal file I/O, so no writeback, the read/write
>> >> operation are done with direct io and they are always sync. The data are
>> >> write protected in hw when the arch provide this facility (x86 does).
>> >> Inode contains a checksum and when there are problems they are marked as
>> >> bad. Superblock contains checksum and there is a redundant superblock.
>> >
>> > But you can still get pramfs inconsistencies if the system crashes at an
>> > inopportune moment. E.g. when making files you write the new inode to
>> > pramfs, and then you insert the entry into the directory. A crash between
>> > these two operations leaves an allocated inode that doesn't appear in
>> > any directory. Without a fsck option, it will be hard to see that you have
>> > this problem, and your only recovery option is to wipe *all* files by making
>> > a new filesystem.
>>
>> Is it a problem if you lost some logs? However do you expect that fsck
>> in this case will drop the inode?
>
> Ask it the other way around.
>
> What is persistent filesystem good for when it is only persistent
> sometimes?
>
> You'd be better running ext2 over special block device, it is quite simple.
>
Ok I can work on it. However can an userspace tool prevent the insert
of fs in linux next?
Marco
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 01/17] pramfs: documentation
From: Pavel Machek @ 2011-01-10 8:08 UTC (permalink / raw)
To: Marco Stornelli
Cc: Tony Luck, Linux Kernel, Linux Embedded, Linux FS Devel, Tim Bird
In-Reply-To: <4D281D65.1070102@gmail.com>
> On 07/01/2011 22:59, Tony Luck wrote:
> > On Fri, Jan 7, 2011 at 12:30 PM, Marco Stornelli
> > <marco.stornelli@gmail.com> wrote:
> >> constraint). About the errors: pramfs does not maintain file data in the
> >> page caches for normal file I/O, so no writeback, the read/write
> >> operation are done with direct io and they are always sync. The data are
> >> write protected in hw when the arch provide this facility (x86 does).
> >> Inode contains a checksum and when there are problems they are marked as
> >> bad. Superblock contains checksum and there is a redundant superblock.
> >
> > But you can still get pramfs inconsistencies if the system crashes at an
> > inopportune moment. E.g. when making files you write the new inode to
> > pramfs, and then you insert the entry into the directory. A crash between
> > these two operations leaves an allocated inode that doesn't appear in
> > any directory. Without a fsck option, it will be hard to see that you have
> > this problem, and your only recovery option is to wipe *all* files by making
> > a new filesystem.
>
> Is it a problem if you lost some logs? However do you expect that fsck
> in this case will drop the inode?
Ask it the other way around.
What is persistent filesystem good for when it is only persistent
sometimes?
You'd be better running ext2 over special block device, it is quite simple.
Pavel
--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
^ permalink raw reply
* Re: [PATCH 01/17] pramfs: documentation
From: Marco Stornelli @ 2011-01-08 8:16 UTC (permalink / raw)
To: Tony Luck; +Cc: Linux Kernel, Linux Embedded, Linux FS Devel, Tim Bird
In-Reply-To: <AANLkTimtvYfHXG=rMLbFLzVaKbfB6nrj_ZE1AH4quRG2@mail.gmail.com>
On 07/01/2011 22:59, Tony Luck wrote:
> On Fri, Jan 7, 2011 at 12:30 PM, Marco Stornelli
> <marco.stornelli@gmail.com> wrote:
>> constraint). About the errors: pramfs does not maintain file data in the
>> page caches for normal file I/O, so no writeback, the read/write
>> operation are done with direct io and they are always sync. The data are
>> write protected in hw when the arch provide this facility (x86 does).
>> Inode contains a checksum and when there are problems they are marked as
>> bad. Superblock contains checksum and there is a redundant superblock.
>
> But you can still get pramfs inconsistencies if the system crashes at an
> inopportune moment. E.g. when making files you write the new inode to
> pramfs, and then you insert the entry into the directory. A crash between
> these two operations leaves an allocated inode that doesn't appear in
> any directory. Without a fsck option, it will be hard to see that you have
> this problem, and your only recovery option is to wipe *all* files by making
> a new filesystem.
Is it a problem if you lost some logs? However do you expect that fsck
in this case will drop the inode?
^ permalink raw reply
* Re: [PATCH 01/17] pramfs: documentation
From: Tony Luck @ 2011-01-07 21:59 UTC (permalink / raw)
To: Marco Stornelli; +Cc: Linux Kernel, Linux Embedded, Linux FS Devel, Tim Bird
In-Reply-To: <4D2777FC.6040509@gmail.com>
On Fri, Jan 7, 2011 at 12:30 PM, Marco Stornelli
<marco.stornelli@gmail.com> wrote:
> constraint). About the errors: pramfs does not maintain file data in the
> page caches for normal file I/O, so no writeback, the read/write
> operation are done with direct io and they are always sync. The data are
> write protected in hw when the arch provide this facility (x86 does).
> Inode contains a checksum and when there are problems they are marked as
> bad. Superblock contains checksum and there is a redundant superblock.
But you can still get pramfs inconsistencies if the system crashes at an
inopportune moment. E.g. when making files you write the new inode to
pramfs, and then you insert the entry into the directory. A crash between
these two operations leaves an allocated inode that doesn't appear in
any directory. Without a fsck option, it will be hard to see that you have
this problem, and your only recovery option is to wipe *all* files by making
a new filesystem.
-Tony
^ permalink raw reply
* Re: [PATCH 01/17] pramfs: documentation
From: Marco Stornelli @ 2011-01-07 20:30 UTC (permalink / raw)
To: Tony Luck; +Cc: Linux Kernel, Linux Embedded, Linux FS Devel, Tim Bird
In-Reply-To: <AANLkTimGt5hi+TpFHWxktJEg3tcDQmmARdqFqzAt++OQ@mail.gmail.com>
Il 07/01/2011 19:42, Tony Luck ha scritto:
> On Thu, Jan 6, 2011 at 4:01 AM, Marco Stornelli
> <marco.stornelli@gmail.com> wrote:
>> +accessed data that must survive system reboots and power cycles. An
>> +example usage might be system logs under /var/log, or a user address
>> +book in a cell phone or PDA.
>
> Some usage model questions:
>
> How do you handle errors? I see that there are a few sanity checks in the
> "mount" path ... but there would seem to be several opportunities for the
> file system to get corrupted in other ways. Since you don't have a block
> device, a standard "fsck" program looks challenging (though I guess you
> could mmap("/dev/mem") to peek & poke at the filesystem before trying
> to mount it).
Actually not (at least when strict devmem options is turned on) because
the memory region is marked exclusive at the moment (only a design
constraint). About the errors: pramfs does not maintain file data in the
page caches for normal file I/O, so no writeback, the read/write
operation are done with direct io and they are always sync. The data are
write protected in hw when the arch provide this facility (x86 does).
Inode contains a checksum and when there are problems they are marked as
bad. Superblock contains checksum and there is a redundant superblock.
> Some sort of recovery path would seem useful for the "address
> book" use model ... or do you just expect users to back their address book
> up (to the cloud?) and have the phone just make a clean filesystem if any
> errors are found?
Yeah maybe the address book can be a case not perfectly suitable, but it
was only an example. I thought about the fs as a "cache" in this use
case. However the designer can use this area whatever he wants,
recently I saw in a project this fs used as a system cache for decrypted
files where the files were stored in flash encrypted, so I think it's
flexible.
> What about quotas? You have a fixed amount of persistent space, and
> presumably a number of apps that the user installs on their device that
> may like to use pramfs to store data. Do you need some kernel enforcement
> to stop one rogue application from using up all the space? Or do you expect that
> this would be handled in some library level interface that applications will
> use to access pramfs?
Sincerely in my embedded systems I've never used quotas even to save
footprint (for the kernel support I mean). I don't think it's an hot
feature in this case and other fs for embedded use as ubifs, jffs2 etc.
don't support it.
Marco
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox