since when does ARM map the kernel memory in sections?

since when does ARM map the kernel memory in sections?

[Peter Wächtler]

Hello Linux ARM developers,

did the ARM Linux 2.6 kernel map the kernel memory in pages in the past?
Or was the memory always mapped in sections?

I still have to chase a potential memory corruption. The rootfs is located on 
a SDcard and gets corrupted even when the filesystem test programs write to 
different partitions.
The test scenario includes several dozen or even hundreds of warm and cold 
boot sequences, file system write tests with sudden soft resets. It's a large 
embedded project with a lot of drivers and the fact that always the rootfs and 
often the superblock gets damaged let me think of a memory corruption.

  Peter

since when does ARM map the kernel memory in sections?

[Colin Cross]

On Tue, Apr 12, 2011 at 11:52 AM, Peter W?chtler <pwaechtler@mac.com> wrote:
> Hello Linux ARM developers,
>
> did the ARM Linux 2.6 kernel map the kernel memory in pages in the past?
> Or was the memory always mapped in sections?
>
> I still have to chase a potential memory corruption. The rootfs is located on
> a SDcard and gets corrupted even when the filesystem test programs write to
> different partitions.
> The test scenario includes several dozen or even hundreds of warm and cold
> boot sequences, file system write tests with sudden soft resets. It's a large
> embedded project with a lot of drivers and the fact that always the rootfs and
> often the superblock gets damaged let me think of a memory corruption.

Gary King posted some patches a while ago that switched the kernel
back to page mappings, so he could modify attributes on some pages to
be non-cacheable.  The patches were not accepted, but you can probably
dig them up for testing.

since when does ARM map the kernel memory in sections?

[Andrei Warkentin]

Hi Peter,

2011/4/12 Peter W?chtler <pwaechtler@mac.com>:
> Hello Linux ARM developers,
>
> did the ARM Linux 2.6 kernel map the kernel memory in pages in the past?
> Or was the memory always mapped in sections?
>
> I still have to chase a potential memory corruption. The rootfs is located on
> a SDcard and gets corrupted even when the filesystem test programs write to
> different partitions.
> The test scenario includes several dozen or even hundreds of warm and cold
> boot sequences, file system write tests with sudden soft resets. It's a large
> embedded project with a lot of drivers and the fact that always the rootfs and
> often the superblock gets damaged let me think of a memory corruption.
>

Sorry, I don't want to be obvious, but you mentioned sudden resets
while writing, which is almost always going to wind
up as fs corruptions, with the severity depending on the level of
caching the system is doing to the writes.
How are you mounting your rootfs and what file system are you using?
What sort of corruptions to the super block are you seeing?

A

since when does ARM map the kernel memory in sections?

[Russell King - ARM Linux]

On Tue, Apr 12, 2011 at 08:52:17PM +0200, Peter W?chtler wrote:
> did the ARM Linux 2.6 kernel map the kernel memory in pages in the past?
> Or was the memory always mapped in sections?

Mainline kernels have never mapped memory using anything but sections.

since when does ARM map the kernel memory in sections?

[Jamie Lokier]

Andrei Warkentin wrote:
> Hi Peter,
> 
> 2011/4/12 Peter W?chtler <pwaechtler@mac.com>:
> > Hello Linux ARM developers,
> >
> > did the ARM Linux 2.6 kernel map the kernel memory in pages in the past?
> > Or was the memory always mapped in sections?
> >
> > I still have to chase a potential memory corruption. The rootfs is located on
> > a SDcard and gets corrupted even when the filesystem test programs write to
> > different partitions.
> > The test scenario includes several dozen or even hundreds of warm and cold
> > boot sequences, file system write tests with sudden soft resets. It's a large
> > embedded project with a lot of drivers and the fact that always the rootfs and
> > often the superblock gets damaged let me think of a memory corruption.
> >
> 
> Sorry, I don't want to be obvious, but you mentioned sudden resets
> while writing, which is almost always going to wind
> up as fs corruptions, with the severity depending on the level of
> caching the system is doing to the writes.
> How are you mounting your rootfs and what file system are you using?
> What sort of corruptions to the super block are you seeing?

If everything is implemented correctly, that depends on the type of
filesystem, block layer and storage.  Some are explicitly designed to
be safe against sudden reboots and power failure - which is an
important feature of systems where removing the power is how they are
turned off at night.

-- Jamie

since when does ARM map the kernel memory in sections?

[Peter Wächtler]

Am Dienstag, 12. April 2011, 21:20:14 schrieb Andrei Warkentin:
> Hi Peter,
> 
> 2011/4/12 Peter W?chtler <pwaechtler@mac.com>:
> > Hello Linux ARM developers,
> > 
> > did the ARM Linux 2.6 kernel map the kernel memory in pages in the past?
> > Or was the memory always mapped in sections?
> > 
> > I still have to chase a potential memory corruption. The rootfs is
> > located on a SDcard and gets corrupted even when the filesystem test
> > programs write to different partitions.
> > The test scenario includes several dozen or even hundreds of warm and
> > cold boot sequences, file system write tests with sudden soft resets.
> > It's a large embedded project with a lot of drivers and the fact that
> > always the rootfs and often the superblock gets damaged let me think of
> > a memory corruption.
> 
> Sorry, I don't want to be obvious, but you mentioned sudden resets
> while writing, which is almost always going to wind
> up as fs corruptions, with the severity depending on the level of
> caching the system is doing to the writes.
> How are you mounting your rootfs and what file system are you using?
> What sort of corruptions to the super block are you seeing?
> 

It's using ext4 with metadata journalling in ordered mode.
I had to check "if it's the FS" - the test programs create lots of directories 
and files while a timer is armed to issue a soft reset.
The partitions where the "stress tests" run on - survive it happily - just the 
rootfs where almost nothing gets written is severly damaged so that fsck.ext4 
will not repair it automatically.

I experiment with mounting rootfs as readonly, without a journal - and get 
hard to interpret results. The superblock gets written on a "mount -o 
remount,ro", the superblock of the journal gets hit and directories, inode 
bitmaps gets "thrashed". All on the file system where the actual tests do NOT 
run.
In the past I tried to write patterns onto the blocks without file system to 
check if the sdio device mangles some blocks - no it didn't.

I created a slab to move the superblock away from it's "old" memory location - 
the super block was not damaged anymore. Now I want to stop experimenting and 
want to get a nice panic where I see the root cause.
But it can be still devices issuing wrong dma transfers ;(

  Peter  

since when does ARM map the kernel memory in sections?

[Nicolas Pitre]

On Tue, 12 Apr 2011, Jamie Lokier wrote:

> Andrei Warkentin wrote:
> > Hi Peter,
> > 
> > 2011/4/12 Peter W?chtler <pwaechtler@mac.com>:
> > > Hello Linux ARM developers,
> > >
> > > did the ARM Linux 2.6 kernel map the kernel memory in pages in the past?
> > > Or was the memory always mapped in sections?
> > >
> > > I still have to chase a potential memory corruption. The rootfs is located on
> > > a SDcard and gets corrupted even when the filesystem test programs write to
> > > different partitions.
> > > The test scenario includes several dozen or even hundreds of warm and cold
> > > boot sequences, file system write tests with sudden soft resets. It's a large
> > > embedded project with a lot of drivers and the fact that always the rootfs and
> > > often the superblock gets damaged let me think of a memory corruption.
> > >
> > 
> > Sorry, I don't want to be obvious, but you mentioned sudden resets
> > while writing, which is almost always going to wind
> > up as fs corruptions, with the severity depending on the level of
> > caching the system is doing to the writes.
> > How are you mounting your rootfs and what file system are you using?
> > What sort of corruptions to the super block are you seeing?
> 
> If everything is implemented correctly, that depends on the type of
> filesystem, block layer and storage.  Some are explicitly designed to
> be safe against sudden reboots and power failure - which is an
> important feature of systems where removing the power is how they are
> turned off at night.

SD was mentioned as being the storage medium in this thread. I really 
doubt SD cards are designed to be safe against sudden power outages.


Nicolas

since when does ARM map the kernel memory in sections?

[Nicolas Pitre]

On Wed, 13 Apr 2011, Peter W?chtler wrote:

> Am Dienstag, 12. April 2011, 21:20:14 schrieb Andrei Warkentin:
> > How are you mounting your rootfs and what file system are you using?
> > What sort of corruptions to the super block are you seeing?
> > 
> 
> It's using ext4 with metadata journalling in ordered mode.
> I had to check "if it's the FS" - the test programs create lots of directories 
> and files while a timer is armed to issue a soft reset.
> The partitions where the "stress tests" run on - survive it happily - just the 
> rootfs where almost nothing gets written is severly damaged so that fsck.ext4 
> will not repair it automatically.

SD cards are doing their own wear leveling internally and you have no 
control over it.  Some blocks of data may be moved around, affecting a 
separate logical partition, even if you are not actively writing to that 
partition.  Now if you cut power or reset the card while this is 
happening you'll certainly end up with data loss.  Those SD cards are 
made to be both cheap and fast, meaning they're certainly not reliable 
with regards to unexpected interruptions.

Furthermore, you should have a look at this article and referenced 
material: http://lwn.net/Articles/428584/.


Nicolas

since when does ARM map the kernel memory in sections?

[Peter Wächtler]

Am Dienstag, 12. April 2011, 21:11:59 schrieb Colin Cross:
> On Tue, Apr 12, 2011 at 11:52 AM, Peter W?chtler <pwaechtler@mac.com> wrote:
> > Hello Linux ARM developers,
> > 
> > did the ARM Linux 2.6 kernel map the kernel memory in pages in the past?
> > Or was the memory always mapped in sections?
> > 
> > I still have to chase a potential memory corruption. The rootfs is
> > located on a SDcard and gets corrupted even when the filesystem test
> > programs write to different partitions.
> > The test scenario includes several dozen or even hundreds of warm and
> > cold boot sequences, file system write tests with sudden soft resets.
> > It's a large embedded project with a lot of drivers and the fact that
> > always the rootfs and often the superblock gets damaged let me think of
> > a memory corruption.
> 
> Gary King posted some patches a while ago that switched the kernel
> back to page mappings, so he could modify attributes on some pages to
> be non-cacheable.  The patches were not accepted, but you can probably
> dig them up for testing.

Thanx for the tip. I already found the patch and applied it. Some probs on 
config but not tested yet...

   Peter

since when does ARM map the kernel memory in sections?

[Peter Wächtler]

Am Mittwoch, 13. April 2011, 17:44:09 schrieb Nicolas Pitre:
> On Wed, 13 Apr 2011, Peter W?chtler wrote:
> > Am Dienstag, 12. April 2011, 21:20:14 schrieb Andrei Warkentin:
> > > How are you mounting your rootfs and what file system are you using?
> > > What sort of corruptions to the super block are you seeing?
> > 
> > It's using ext4 with metadata journalling in ordered mode.
> > I had to check "if it's the FS" - the test programs create lots of
> > directories and files while a timer is armed to issue a soft reset.
> > The partitions where the "stress tests" run on - survive it happily -
> > just the rootfs where almost nothing gets written is severly damaged so
> > that fsck.ext4 will not repair it automatically.
> 
> SD cards are doing their own wear leveling internally and you have no
> control over it.  Some blocks of data may be moved around, affecting a
> separate logical partition, even if you are not actively writing to that
> partition.  Now if you cut power or reset the card while this is
> happening you'll certainly end up with data loss.  Those SD cards are
> made to be both cheap and fast, meaning they're certainly not reliable
> with regards to unexpected interruptions.
> 
> Furthermore, you should have a look at this article and referenced
> material: http://lwn.net/Articles/428584/.
> 
> 

Yes, I know this article - and I know about wear levelling etc.

Data loss is not the biggest problem. The capacity is huge (>>4GB) - therefore 
I wouldn't want to miss journalling - but I was not involved on the decision. 
Ten years ago I carefully ordered the fsync, rename on ext2 on a 64MiB 
CompactFlash - worked well enough.

The vendor knows about the requirements - perhaps learned it with another 
customer ;)

I tried to "smash" the FS without success in the past - but the rootfs was 
mounted via NFS ;) And the partitions that get tortured stay intact - of 
course you can say that the wear levelling switches some blocks - but I don't 
buy it.

Thanks for all the nice work on Linux - Russell included ;)

  Peter

since when does ARM map the kernel memory in sections?

[Jamie Lokier]

Nicolas Pitre wrote:
> On Tue, 12 Apr 2011, Jamie Lokier wrote:
> 
> > Andrei Warkentin wrote:
> > > Hi Peter,
> > > 
> > > 2011/4/12 Peter W?chtler <pwaechtler@mac.com>:
> > > > Hello Linux ARM developers,
> > > >
> > > > did the ARM Linux 2.6 kernel map the kernel memory in pages in the past?
> > > > Or was the memory always mapped in sections?
> > > >
> > > > I still have to chase a potential memory corruption. The rootfs is located on
> > > > a SDcard and gets corrupted even when the filesystem test programs write to
> > > > different partitions.
> > > > The test scenario includes several dozen or even hundreds of warm and cold
> > > > boot sequences, file system write tests with sudden soft resets. It's a large
> > > > embedded project with a lot of drivers and the fact that always the rootfs and
> > > > often the superblock gets damaged let me think of a memory corruption.
> > > >
> > > 
> > > Sorry, I don't want to be obvious, but you mentioned sudden resets
> > > while writing, which is almost always going to wind
> > > up as fs corruptions, with the severity depending on the level of
> > > caching the system is doing to the writes.
> > > How are you mounting your rootfs and what file system are you using?
> > > What sort of corruptions to the super block are you seeing?
> > 
> > If everything is implemented correctly, that depends on the type of
> > filesystem, block layer and storage.  Some are explicitly designed to
> > be safe against sudden reboots and power failure - which is an
> > important feature of systems where removing the power is how they are
> > turned off at night.
> 
> SD was mentioned as being the storage medium in this thread. I really 
> doubt SD cards are designed to be safe against sudden power outages.

Ah, fair enough, you are sadly almost certainly right.

(Some flash even corrupts itself on power removal during reads too,
due to internal reorganisation.  I have no idea if SD is in that
category.  It is quite a problem when searching for something to
replace an old-fashioned hard disk in an application where removing
the power is normal daily usage, and traditional journalling
filesystems used to be effective.)

-- Jamie

since when does ARM map the kernel memory in sections?

[Pavel Machek]

Hi!

> > > did the ARM Linux 2.6 kernel map the kernel memory in pages in the past?
> > > Or was the memory always mapped in sections?
> > >
> > > I still have to chase a potential memory corruption. The rootfs is located on
> > > a SDcard and gets corrupted even when the filesystem test programs write to
> > > different partitions.
> > > The test scenario includes several dozen or even hundreds of warm and cold
> > > boot sequences, file system write tests with sudden soft resets. It's a large
> > > embedded project with a lot of drivers and the fact that always the rootfs and
> > > often the superblock gets damaged let me think of a memory corruption.
> > >
> > 
> > Sorry, I don't want to be obvious, but you mentioned sudden resets
> > while writing, which is almost always going to wind
> > up as fs corruptions, with the severity depending on the level of
> > caching the system is doing to the writes.
> > How are you mounting your rootfs and what file system are you using?
> > What sort of corruptions to the super block are you seeing?
> 
> If everything is implemented correctly, that depends on the type of
> filesystem, block layer and storage.  Some are explicitly designed to
> be safe against sudden reboots and power failure - which is an
> important feature of systems where removing the power is how they are
> turned off at night.

...but note that no existing  filesystem is safe on media such
as usb sticks, SD and CF cards...

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

since when does ARM map the kernel memory in sections?

[Jamie Lokier]

Pavel Machek wrote:
> ...but note that no existing  filesystem is safe on media such
> as usb sticks, SD and CF cards...

Sadly.  Do you know by now if enough information is available now to
make it possible even in principle?

Alternatively, do you know if there's been much progress
characterising particular manufacturers/parts/brands or identifiable
characteristics etc. to find ones on which a power fail safe
filesystem is possible, even if it's just in principle for the moment?

Sadly I know of a large number of systems using CF or cheap ATA SSDs
that were installed assuming ext3 is reliable when they are powered
off at the wall every day.  That's because a lot of people assume they
are solid-state drop-in replacements for hard disks, as that is how
they are advertised.

-- Jamie

since when does ARM map the kernel memory in sections?

[Nicolas Pitre]

On Mon, 18 Apr 2011, Jamie Lokier wrote:

> Pavel Machek wrote:
> > ...but note that no existing  filesystem is safe on media such
> > as usb sticks, SD and CF cards...
> 
> Sadly.  Do you know by now if enough information is available now to
> make it possible even in principle?
> 
> Alternatively, do you know if there's been much progress
> characterising particular manufacturers/parts/brands or identifiable
> characteristics etc. to find ones on which a power fail safe
> filesystem is possible, even if it's just in principle for the moment?

Have a look at:

https://wiki.linaro.org/WorkingGroups/Kernel/Projects/FlashCardSurvey


Nicolas

since when does ARM map the kernel memory in sections?

[Pavel Machek]

Hi!

(don't top post)

> I'm not 100% sure, but a lot of people made good experience with JFFS2.

jffs2 needs raw flash. CF & SD look like block devices.
 
> Is it that hard to get a log structured file system power fail safe?

Not that bad if you can access raw flash. Very bad if there's layer
that hides flash from you and corrupts data in between...

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

since when does ARM map the kernel memory in sections?

[Peter Waechtler]

Hmh,

I'm not 100% sure, but a lot of people made good experience with JFFS2.

Is it that hard to get a log structured file system power fail safe?
Either the end or "commit" block was written or not?

??Peter
?

On 18 Apr, 2011,at 03:52 PM, Pavel Machek <pavel@ucw.cz> wrote:


Hi!

> > > did the ARM Linux 2.6 kernel map the kernel memory in pages in the past?
> > > Or was the memory always mapped in sections?
> > >
> > > I still have to chase a potential memory corruption. The rootfs is located on
> > > a SDcard and gets corrupted even when the filesystem test programs write to
> > > different partitions.
> > > The test scenario includes several dozen or even hundreds of warm and cold
> > > boot sequences, file system write tests with sudden soft resets It's a large
> > > embedded project with a lot of drivers and the fact that always the rootfs and
> > > often the superblock gets damaged let me think of a memory corruption.
> > >
> > 
> > Sorry, I don't want to be obvious, but you mentioned sudden resets
> > while writing, which is almost always going to wind
> > up as fs corruptions, with the severity depending on the level of
> > caching the system is doing to the writes.
> > How are you mounting your rootfs and what file system are you using?
> > What sort of corruptions to the super block are you seeing?
> 
> If everything is implemented correctly, that depends on the type of
> filesystem, block layer and storage. Some are explicitly designed to
> be safe against sudden reboots and power failure - which is an
> important feature of systems where removing the power is how they are
> turned off at night.

...but note that no existing filesystem is safe on media such
as usb sticks, SD and CF cards...

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mffcuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20110418/bff5d2f9/attachment.html>

since when does ARM map the kernel memory in sections?

[Jamie Lokier]

Peter Waechtler wrote:
> Hmh,
> 
> I'm not 100% sure, but a lot of people made good experience with JFFS2.
> 
> Is it that hard to get a log structured file system power fail safe?
> Either the end or "commit" block was written or not?

Yes, it's very hard, maybe impossible, on the types of flash media
being discussed,

For a useful filesystem, it's not enough to know that a single
"commit" block is written or not.  The hard part is when the media
writes things out of order, so at the next mount, even though you
confirm the last "commit" is valid, any earlier blocks might not be
valid.  The only way to be sure of the last coherent state that you
still have, is to read everything, and even then a valid filesystem
where random files have acquired holes they didn't have before is not
very useful.

A purely log structured filesystem should at least look like a valid
filesystem after power failure and reboot, but:

- I am not sure that JFFS2 is purely log structured any more, with the
  compact summary information in each block about what's written
  elsewhere in the block.  That's not reliable on media where you
  can't guarantee anything about the order of writes.  If the summary
  information were ignored, it would be reliable, but slower to read.

- A valid filesystem, and valid files on it are a bit different.
  fsync needs to work for applications to have some sanity in what
  they can depend on.  JFFS2 is fine with that on NOR/NAND directly,
  but if the media doesn't guarantee order of writes...

- Write disturb effects, where writing something corrupts data stored
  elsewhere, in other blocks that the filesystem thought was stable.

- On some media, blocks far apart are less likely to disturb each
  other (for example some RAIDs), but not on flash translation media
  where the physical layout and logical layout may be completely
  different.

- Read disturb effects, where reading triggers flash reorganisation
  too, and then you pull the power while the flash is writing
  internally to reorganise, and on the next boot something's gone
  missing that was stable before.

All the problems are avoidable on devices designed to avoid them.
Problem is devices aren't sold on the basis of these
characteristics, and it's not something the next layer up can
workaround reliably, though it might be able to be more robust.

-- Jamie

since when does ARM map the kernel memory in sections?

[Pavel Machek]

Hi!

> > ...but note that no existing  filesystem is safe on media such
> > as usb sticks, SD and CF cards...
> 
> Sadly.  Do you know by now if enough information is available now to
> make it possible even in principle?

> Alternatively, do you know if there's been much progress
> characterising particular manufacturers/parts/brands or identifiable
> characteristics etc. to find ones on which a power fail safe
> filesystem is possible, even if it's just in principle for the moment?

There's nice table somewhere. If at least size of "big block" (usually
4MB) is known, safe fs should be possible. OTOH 4MB commit block is
not going to be too efficient.

Plus, I was told new MMC standard has "write reliable" option...

> Sadly I know of a large number of systems using CF or cheap ATA SSDs
> that were installed assuming ext3 is reliable when they are powered
> off at the wall every day.  That's because a lot of people assume they
> are solid-state drop-in replacements for hard disks, as that is how
> they are advertised.

False advertising, I'm afraid. I wanter if SATA SSDs do better...?
								       Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

since when does ARM map the kernel memory in sections?

[Linus Walleij]

2011/4/22 Pavel Machek <pavel@ucw.cz>:

> Plus, I was told new MMC standard has "write reliable" option...

I think Per F?rlin looked into reliable write. The latest eMMC cards
has this, but OTOMH it was too darn slow to be used on current
chips/"cards".

Per, Sebastian: any details?

Yours,
Linus Walleij

since when does ARM map the kernel memory in sections?

[Per Forlin]

On 23 April 2011 11:23, Linus Walleij <linus.walleij@linaro.org> wrote:
> 2011/4/22 Pavel Machek <pavel@ucw.cz>:
>
>> Plus, I was told new MMC standard has "write reliable" option...
>
> I think Per F?rlin looked into reliable write. The latest eMMC cards
> has this, but OTOMH it was too darn slow to be used on current
> chips/"cards".
>
> Per, Sebastian: any details?
I had plans to add reliable writes and do benchmarking but I never got
to it. Right now I have no plans to pick it up.

>
> Yours,
> Linus Walleij
>
Regards,
Per

since when does ARM map the kernel memory in sections?

[Peter Waechtler]

Am Dienstag, 26. April 2011, 12:33:29 schrieb Per Forlin:
> On 23 April 2011 11:23, Linus Walleij <linus.walleij@linaro.org> wrote:
> > 2011/4/22 Pavel Machek <pavel@ucw.cz>:
> >> Plus, I was told new MMC standard has "write reliable" option...
> > 
> > I think Per F?rlin looked into reliable write. The latest eMMC cards
> > has this, but OTOMH it was too darn slow to be used on current
> > chips/"cards".
> > 
> > Per, Sebastian: any details?
> 
> I had plans to add reliable writes and do benchmarking but I never got
> to it. Right now I have no plans to pick it up.
> 
> > Yours,
> > Linus Walleij
> 
> Regards,
> Per

As far as I understood the spec, reliable write only makes statements like
either the old data is still intact - or the new data was written 
(completely).

	Peter

since when does ARM map the kernel memory in sections?

[Jamie Lokier]

Peter Waechtler wrote:
> Am Dienstag, 26. April 2011, 12:33:29 schrieb Per Forlin:
> > On 23 April 2011 11:23, Linus Walleij <linus.walleij@linaro.org> wrote:
> > > 2011/4/22 Pavel Machek <pavel@ucw.cz>:
> > >> Plus, I was told new MMC standard has "write reliable" option...
> > > 
> > > I think Per F?rlin looked into reliable write. The latest eMMC cards
> > > has this, but OTOMH it was too darn slow to be used on current
> > > chips/"cards".
> > > 
> > > Per, Sebastian: any details?
> > 
> > I had plans to add reliable writes and do benchmarking but I never got
> > to it. Right now I have no plans to pick it up.
> > 
> > > Yours,
> > > Linus Walleij
> > 
> > Regards,
> > Per
> 
> As far as I understood the spec, reliable write only makes statements like
> either the old data is still intact - or the new data was written 
> (completely).

Hmm, if that's a correct understanding, it's not very useful for
fsync() or journal barriers (unless the spec implies something
barrier-like), and it would be nice if there were guarantees about the
_other_ data (that isn't being written at all) not getting corrupted
as a side effect.

-- Jamie

since when does ARM map the kernel memory in sections?

[Andrei Warkentin]

Hi Per,

On Tue, Apr 26, 2011 at 5:33 AM, Per Forlin <per.forlin@linaro.org> wrote:
> On 23 April 2011 11:23, Linus Walleij <linus.walleij@linaro.org> wrote:
>> 2011/4/22 Pavel Machek <pavel@ucw.cz>:
>>
>>> Plus, I was told new MMC standard has "write reliable" option...
>>
>> I think Per F?rlin looked into reliable write. The latest eMMC cards
>> has this, but OTOMH it was too darn slow to be used on current
>> chips/"cards".
>>
>> Per, Sebastian: any details?
> I had plans to add reliable writes and do benchmarking but I never got
> to it. Right now I have no plans to pick it up.
>
>>

Reliable writes are in mmc-next already. As an improvement to that
path, I have a CMD23-bounded request support patch set which is
pending.

Reliable writes are exposed via REQ_FUA.

Keep in mind that flash cards don't have a volatile cache, so once an
MMC transaction goes through the data is in flash. All reliable writes
guarantee is flash state if an MMC transaction is interrupted in the
middle. Additionally, the "new" reliable write (as opposed to legacy)
is even less useful, since it only provides that guarantee at a sector
boundary.

A

since when does ARM map the kernel memory in sections?

[Jamie Lokier]

Andrei Warkentin wrote:
> Hi Per,
> 
> On Tue, Apr 26, 2011 at 5:33 AM, Per Forlin <per.forlin@linaro.org> wrote:
> > On 23 April 2011 11:23, Linus Walleij <linus.walleij@linaro.org> wrote:
> >> 2011/4/22 Pavel Machek <pavel@ucw.cz>:
> >>
> >>> Plus, I was told new MMC standard has "write reliable" option...
> >>
> >> I think Per F?rlin looked into reliable write. The latest eMMC cards
> >> has this, but OTOMH it was too darn slow to be used on current
> >> chips/"cards".
> >>
> >> Per, Sebastian: any details?
> > I had plans to add reliable writes and do benchmarking but I never got
> > to it. Right now I have no plans to pick it up.
> >
> >>
> 
> Reliable writes are in mmc-next already. As an improvement to that
> path, I have a CMD23-bounded request support patch set which is
> pending.
> 
> Reliable writes are exposed via REQ_FUA.

Are you sure that's appropriate?

Unless I have misunderstood (very possible), REQ_FUA means writes hit
non-volatile storage before acknowledgement, not that they are atomic.
I think the normal users of REQ_FUA don't require or expect large
atomic writes; they use it as a shortcut for (write & flush this
write) without implying anything else is flushed.

> Keep in mind that flash cards don't have a volatile cache, so once an
> MMC transaction goes through the data is in flash.

Does that not mean MMC already provides REQ_FUA semantics on every
transactions?

I don't know much about MMC, but the problems reported with other
flash devices are either volatile cache (so may not apply to
conformant MMCs), or random corruption of data that was supposed to be
stored long ago, even data quite far from the locations being written
at the time, because the flash is presumably reorganising itself.
There are even reports of data loss resulting from power removal while
reading.

> All reliable writes guarantee is flash state if an MMC transaction
> is interrupted in the middle. Additionally, the "new" reliable write
> (as opposed to legacy) is even less useful, since it only provides
> that guarantee at a sector boundary.

Perhaps the sector bondary limitation makes it faster and/or limits
the amount of buffer required, and/or allows the device to accept
larger write transactions.  Which is good if it means reliability
doesn't get switched off or faked.  Or perhaps it's just to align, a
little, with perceived behaviour of hard disks.

Hard disks don't guarantee large atomic writes as far as I know, so
filesystems & databases generally don't assume it, and it's not really
a problem.  Some people say you can rely on a single 512-byte sector
being atomically updated or not on a hard disk, but some don't; I'm
siding with the latter.  (SQLite has a storage flag you can set if you
know the storage has that property, to tweak its commit strategy.)

-- Jamie

since when does ARM map the kernel memory in sections?

[Andrei Warkentin]

On Tue, Apr 26, 2011 at 5:58 PM, Jamie Lokier <jamie@shareable.org> wrote:
>>
>> Reliable writes are exposed via REQ_FUA.
>
> Are you sure that's appropriate?
>
> Unless I have misunderstood (very possible), REQ_FUA means writes hit
> non-volatile storage before acknowledgement, not that they are atomic.
> I think the normal users of REQ_FUA don't require or expect large
> atomic writes; they use it as a shortcut for (write & flush this
> write) without implying anything else is flushed.

I would agree with you that it's not the best mapping. However, a failed
MMC write transaction has other properties. If I understand correctly,
depending on mode of failure (say pulling power), you might wind up
with extra data getting erased (because erase
happens at erase unit boundary), and erase can be done before all the
data was transferred from host to card.

The connection I made between FUA and reliable writes, is that you
were guaranteed "physical presence" of the written data on
storage medium as long as the transaction went through successfully. I
can see where I assumed more than I should have.... If that's not the
correct interpretation I will change it.

REQ_META doesn't sound like the right candidate, because it's
enforcing policy. Should there be a REQ_ATOMIC request type?

A

since when does ARM map the kernel memory in sections?

[Jamie Lokier]

Andrei Warkentin wrote:
> The connection I made between FUA and reliable writes, is that you
> were guaranteed "physical presence" of the written data on
> storage medium as long as the transaction went through successfully. I
> can see where I assumed more than I should have.... If that's not the
> correct interpretation I will change it.

Well it would seem like a reasonable thing to *want* from the storage
medium. :-) Maybe that's intended but not stated clearly in that
excerpted bit of the MMC spec.

With SATA, you can do FUA, or you can write non-FUA and to a FLUSH
later to get the same level of durability (I presume it's the same
after that) -- or you can turn off the write cache (which is ok with
some disks/interfaces and wrecks the performance of others.)

The fact that ordinary writes can be commited too, and you can wait
for that, is quite important in practice.  It's the basis for all
reliable journalling and efficiently log-structured storage.

> REQ_META doesn't sound like the right candidate, because it's
> enforcing policy. Should there be a REQ_ATOMIC request type?

Imho, only if there's a use for it.  If this is about whole partitions
picking up random data corruption, versus not doing so, then I suggest
the choice of "Reliable Write" vs. "Unreliable Write" be a mount
option or hdparm-style block device option.

If there are tighter guarantees, such as "Unreliable Write" corruption
being limited to the written naturally aligned 1MB blocks (say), and
it was genuinely faster, that would be really valuable information to
pass up to filesystems - and to userspace - as you can structure
reliability around that in lots of ways.

-- Jamie

since when does ARM map the kernel memory in sections?

[Arnd Bergmann]

On Wednesday 27 April 2011, Jamie Lokier wrote:
> Imho, only if there's a use for it.  If this is about whole partitions
> picking up random data corruption, versus not doing so, then I suggest
> the choice of "Reliable Write" vs. "Unreliable Write" be a mount
> option or hdparm-style block device option.
> 
> If there are tighter guarantees, such as "Unreliable Write" corruption
> being limited to the written naturally aligned 1MB blocks (say), and
> it was genuinely faster, that would be really valuable information to
> pass up to filesystems - and to userspace - as you can structure
> reliability around that in lots of ways.

In all the SDHC cards that I have seen, the corruption should be local to
an erase block of the size that is supposedly found in
/sys/block/mmcblk*/device/preferred_erase_size, which is typically 4 MB.

However, I don't think that the standard actually guarantees this and,
worse, some cards that I have seen actually lie about the erase block
size and claim that it is 4 MB when it is actually 1.5, 2, 3 or 8 MB.

For eMMC devices, I don't think we can read the erase block size.

	Arnd

since when does ARM map the kernel memory in sections?

[Peter Waechtler]

Am Mittwoch, 27. April 2011, 15:32:24 schrieb Arnd Bergmann:
> On Wednesday 27 April 2011, Jamie Lokier wrote:
> > Imho, only if there's a use for it.  If this is about whole partitions
> > picking up random data corruption, versus not doing so, then I suggest
> > the choice of "Reliable Write" vs. "Unreliable Write" be a mount
> > option or hdparm-style block device option.
> > 
> > If there are tighter guarantees, such as "Unreliable Write" corruption
> > being limited to the written naturally aligned 1MB blocks (say), and
> > it was genuinely faster, that would be really valuable information to
> > pass up to filesystems - and to userspace - as you can structure
> > reliability around that in lots of ways.
> 
> In all the SDHC cards that I have seen, the corruption should be local to
> an erase block of the size that is supposedly found in
> /sys/block/mmcblk*/device/preferred_erase_size, which is typically 4 MB.
> 
> However, I don't think that the standard actually guarantees this and,
> worse, some cards that I have seen actually lie about the erase block
> size and claim that it is 4 MB when it is actually 1.5, 2, 3 or 8 MB.
> 
> For eMMC devices, I don't think we can read the erase block size.
> 
I have to check, but I think to remember that it can be calculated by values
provided in CSD/ ex CSD or whatever that acronym was... 4 or 8MB sounds 
familiar to me (and my problem).

	Peter

since when does ARM map the kernel memory in sections?

[Andrei Warkentin]

On Wed, Apr 27, 2011 at 1:50 PM, Peter Waechtler <pwaechtler@mac.com> wrote:
> Am Mittwoch, 27. April 2011, 15:32:24 schrieb Arnd Bergmann:
>> On Wednesday 27 April 2011, Jamie Lokier wrote:
>> > Imho, only if there's a use for it. ?If this is about whole partitions
>> > picking up random data corruption, versus not doing so, then I suggest
>> > the choice of "Reliable Write" vs. "Unreliable Write" be a mount
>> > option or hdparm-style block device option.
>> >
>> > If there are tighter guarantees, such as "Unreliable Write" corruption
>> > being limited to the written naturally aligned 1MB blocks (say), and
>> > it was genuinely faster, that would be really valuable information to
>> > pass up to filesystems - and to userspace - as you can structure
>> > reliability around that in lots of ways.
>>
>> In all the SDHC cards that I have seen, the corruption should be local to
>> an erase block of the size that is supposedly found in
>> /sys/block/mmcblk*/device/preferred_erase_size, which is typically 4 MB.
>>
>> However, I don't think that the standard actually guarantees this and,
>> worse, some cards that I have seen actually lie about the erase block
>> size and claim that it is 4 MB when it is actually 1.5, 2, 3 or 8 MB.
>>
>> For eMMC devices, I don't think we can read the erase block size.
>>
> I have to check, but I think to remember that it can be calculated by values
> provided in CSD/ ex CSD or whatever that acronym was... 4 or 8MB sounds
> familiar to me (and my problem).
>

Yep, there are ERASE_GRP_SIZE and ERASE_GRP_MULT in the CSD, and then
their high-capacity
variant in EXT_CSD (HC_ERASE_GRP_SIZE).

A

since when does ARM map the kernel memory in sections?

[Peter Waechtler]

Am Mittwoch, 27. April 2011, 02:27:57 schrieben Sie:
> On Tue, Apr 26, 2011 at 5:58 PM, Jamie Lokier <jamie@shareable.org> wrote:
> >> Reliable writes are exposed via REQ_FUA.
> > 
> > Are you sure that's appropriate?
> > 
> > Unless I have misunderstood (very possible), REQ_FUA means writes hit
> > non-volatile storage before acknowledgement, not that they are atomic.
> > I think the normal users of REQ_FUA don't require or expect large
> > atomic writes; they use it as a shortcut for (write & flush this
> > write) without implying anything else is flushed.
> 
> I would agree with you that it's not the best mapping. However, a failed
> MMC write transaction has other properties. If I understand correctly,
> depending on mode of failure (say pulling power), you might wind up
> with extra data getting erased (because erase
> happens at erase unit boundary), and erase can be done before all the
> data was transferred from host to card.
> 

No, it's not because of the erase unit size larger than write size.
An "erase unit" is cleared by garbage collection only IFF the complete unit is 
unused - asynchronously by the ongoing write transaction. The problem with 
damaging neighboring data is: density. No beer (today), i'm drinking wine ;)

The cells storing the "bits" gets smaller and smaller and with multi level 
cells even more bits are represented by "fewer electrons". Programming the 
neighbor cell disturbs the fragile cells. If the error correction is disturbed 
by power loss: off we go... 

Someone from the flash vendors eavesdropping? Please, pretty please ... :)

	Peter

P.S. Andrei: sorry for reposting, but Kmail confuses me with "Reply to all"

since when does ARM map the kernel memory in sections?

[Andrei Warkentin]

On Thu, Apr 28, 2011 at 3:26 PM, Peter Waechtler <pwaechtler@mac.com> wrote:
> Am Mittwoch, 27. April 2011, 02:27:57 schrieben Sie:
>> On Tue, Apr 26, 2011 at 5:58 PM, Jamie Lokier <jamie@shareable.org> wrote:
>> >> Reliable writes are exposed via REQ_FUA.
>> >
>> > Are you sure that's appropriate?
>> >
>> > Unless I have misunderstood (very possible), REQ_FUA means writes hit
>> > non-volatile storage before acknowledgement, not that they are atomic.
>> > I think the normal users of REQ_FUA don't require or expect large
>> > atomic writes; they use it as a shortcut for (write & flush this
>> > write) without implying anything else is flushed.
>>
>> I would agree with you that it's not the best mapping. However, a failed
>> MMC write transaction has other properties. If I understand correctly,
>> depending on mode of failure (say pulling power), you might wind up
>> with extra data getting erased (because erase
>> happens at erase unit boundary), and erase can be done before all the
>> data was transferred from host to card.
>>
>
> No, it's not because of the erase unit size larger than write size.
> An "erase unit" is cleared by garbage collection only IFF the complete unit is
> unused - asynchronously by the ongoing write transaction. The problem with
> damaging neighboring data is: density. No beer (today), i'm drinking wine ;)
>
> The cells storing the "bits" gets smaller and smaller and with multi level
> cells even more bits are represented by "fewer electrons". Programming the
> neighbor cell disturbs the fragile cells. If the error correction is disturbed
> by power loss: off we go...
>
> Someone from the flash vendors eavesdropping? Please, pretty please ... :)
>
> ? ? ? ?Peter
>
> P.S. Andrei: sorry for reposting, but Kmail confuses me with "Reply to all"
>

No problem, I'm confused actually if my mails got through... Here and
in the other mmc flash reliability thread.

A