Future Linux on Bistable Storage

Future Linux on Bistable Storage

[Rick van Rein]

Hello,

Future generations of Linux are likely to run on machines with non-volatile
memories based on bistable technologies.  This will save the energy of DRAM
refresh cycles and avoid the mechanical problems related to hard disks.  The
result is probably a computer with no distinction between disk and RAM.

Such computer architectures can conserve a lot of energy, as it is very
simple to suspend and resume them: no data loss means no time needed to
resume the system -- except perhaps for I/O initialisation.  Conserving
energy means (1) having more fun|hours per kg of batteries, and (2) saving
the planet.


I wonder inhowfar Linux is (already) capable of dealing with such hardware.
Several core concepts suddenly become meaningless if disk and RAM are merged
into one storage device:
 * swapping out data
 * lazy loading of programs from disk to RAM
 * buffering disk blocks
 * mapping and unmapping disk onto RAM


Regarding I/O init: Current Linux boots while initiating one device at a
time.  Some degree of serialism must exist, but after scanning a bus would
it not be possible to fork threads to initiate each of the devices found
on it?  Am I overlooking details or could this be a useful change?
If it is possible, it would make Linux _much_ quicker to boot (also on
current hardware), and it could even make it acceptable to do a suspend
after a _very_ short time (say, a minute) of non-activity.  It'd be an
easy way to save some trees :)

In the long run, it seems fair to assume that all devices, including
processors, would use non-volatile buffers based on one of the bistable
semiconductor devices currently being developed; but I would not be
surprised if at first we will be working with computers that have older
parts mixed in; boards that still forget their registers when switched off.
But in the end I expect the entire system to switch itself off in between
every two keystrokes (!) simply because loosing no data anywhere in the
system means being able to resume operations at the next whim/interrupt.


I have seen Linux make quite impressive changes to its fundamental
architecture.  Could this be the Next Big Challenge to the flexibility
of our favourite OS?  What would have to change to make it happen, and
is it doable?


Finally, a few background links:

* WikiPedia on Non-volatile memory; especially review the "Upcoming"
  technologies FeRAM, MRAM, NRAM, PRAM, and so on:
  http://en.wikipedia.org/wiki/Non-volatile_memory

* "Towards Zero-power Computing" at GroenGemak:
  http://groengemak.nl/en/article/2008-05-19-10-05.toekomst-computer.html


Yours truly,

Rick van Rein
GroenGemak
http://groengemak.nl/

Re: Future Linux on Bistable Storage

[david]

On Mon, 2 Jun 2008, Rick van Rein wrote:

> Hello,
>
> Future generations of Linux are likely to run on machines with non-volatile
> memories based on bistable technologies.  This will save the energy of DRAM
> refresh cycles and avoid the mechanical problems related to hard disks.  The
> result is probably a computer with no distinction between disk and RAM.

unless there has been a breakthrough that I haven't heard about (always 
possible) I seriously doubt that this is the case.

the alternate technologies that I have heard about are either _far_ less 
dense then DRAM (similar to static ram) or require erasing in blocks 
(similar to flash). niether one is appropriate for a large, flat 
addressed memory architecture as you list below.

David Lang

Re: Future Linux on Bistable Storage

[Jared Hulbert]

> the alternate technologies that I have heard about are either _far_ less
> dense then DRAM (similar to static ram) or require erasing in blocks
> (similar to flash).

I will agree with this.

> niether one is appropriate for a large, flat addressed
> memory architecture as you list below.

I don't agree with this one.   Erase blocks don't automatically
disqualify a memory as a fit into a flat addressed arch.   You need to
play games at the memory management layer to make it work, but it's
not impossible.

Re: Future Linux on Bistable Storage

[Daniel Barkalow]

On Mon, 2 Jun 2008, Rick van Rein wrote:

> Hello,
> 
> Future generations of Linux are likely to run on machines with non-volatile
> memories based on bistable technologies.  This will save the energy of DRAM
> refresh cycles and avoid the mechanical problems related to hard disks.  The
> result is probably a computer with no distinction between disk and RAM.

One major difference between disk and RAM is the tradeoffs between size, 
speed, and price. It's highly unlikely that any one technology will ever 
beat all others in both of the usage patterns which are normal for RAM and 
disk in devices considered by their users to be "computers".

> 
> Such computer architectures can conserve a lot of energy, as it is very
> simple to suspend and resume them: no data loss means no time needed to
> resume the system -- except perhaps for I/O initialisation.  Conserving
> energy means (1) having more fun|hours per kg of batteries, and (2) saving
> the planet.
> 
> 
> I wonder inhowfar Linux is (already) capable of dealing with such hardware.
> Several core concepts suddenly become meaningless if disk and RAM are merged
> into one storage device:
>  * swapping out data

CONFIG_SWAP=n

>  * buffering disk blocks

I think current ramdisk code skips it.

>  * lazy loading of programs from disk to RAM
>  * mapping and unmapping disk onto RAM

You still need the distinction between what's in a process's address space 
and what's in the filesystem, in order to have a system that feels like a 
computer.

> Regarding I/O init: Current Linux boots while initiating one device at a
> time.  Some degree of serialism must exist, but after scanning a bus would
> it not be possible to fork threads to initiate each of the devices found
> on it?  Am I overlooking details or could this be a useful change?

Userspace setup stuff hugely dominates boot time. Some drivers, and it's 
not known which, only successfully initialize on the assumption that some 
other drivers are not initializing hardware at the same time. This has 
been tried, and it (a) had some hard-to-track bugs and (b) didn't help 
enough to be worth dealing with.

> If it is possible, it would make Linux _much_ quicker to boot (also on
> current hardware), and it could even make it acceptable to do a suspend
> after a _very_ short time (say, a minute) of non-activity.

Writing all of RAM to disk burns more power than powering RAM for several 
minutes, even assuming you don't have to spin up the hard drive. For that 
matter, restarting the system will power up all of the devices on the bus 
that you've shut down because you're not using them.

> It'd be an easy way to save some trees :)

You have a wood-fired power plant?

> In the long run, it seems fair to assume that all devices, including
> processors, would use non-volatile buffers based on one of the bistable
> semiconductor devices currently being developed; but I would not be
> surprised if at first we will be working with computers that have older
> parts mixed in; boards that still forget their registers when switched off.

In general, any time you switch drivers or kernel versions, you'll want to 
have the device return all of its registers to a default state and take it 
through an initialization process. For that matter, even if the memory 
keeps its state, on power failure some calcuations will have been done and 
some will not, so the resulting storage will be in some unknown state and 
need to be reinitialized from storage that wasn't changing.

> But in the end I expect the entire system to switch itself off in between
> every two keystrokes (!) simply because loosing no data anywhere in the
> system means being able to resume operations at the next whim/interrupt.

It's likely that the power cost of getting a consistant state to the 
bistable memory will be more than the idle power required for a couple of 
seconds.

	-Daniel
*This .sig left intentionally blabk*

Re: Future Linux on Bistable Storage

[Andi Kleen]

Rick van Rein <rick@groengemak.nl> writes:

> Some degree of serialism must exist, but after scanning a bus would
> it not be possible to fork threads to initiate each of the devices found
> on it?

Multithreaded driver initialization has been tried, but reverted
because it triggered too many driver bugs.

-Andi

Re: Future Linux on Bistable Storage

[Rick van Rein]

Hello,

Thanks for your responses.  You convinced me I/O initiatilisation cannot
be improved upon.

> unless there has been a breakthrough that I haven't heard about (always
> possible) I seriously doubt that this is the case.

Oh wait... I'm talking of upcoming technologies, not established ones!

> the alternate technologies that I have heard about are either _far_ less
> dense then DRAM (similar to static ram) or require erasing in blocks
> (similar to flash).

NanoRAM for one sounds like it can achieve very high densities and high
speeds in coming years.  A slideshow from a researcher gives numbers:

http://www.imechanica.org/files/Proposal-MNRAM.pdf

100 GHz speeds, 100 T/cm2 -- and it's all bistable so no power is consumed
to keep it going, as is a limitation with DRAM.  With a cm2 of that I don't
think we'd care for a HDD or for DRAM anymore.

> Perhaps one of the more key stepping stones here is execution in place
> (XIP) support [...] Maybe [...] we should consider XIP-for-ramdisk
> (rather than ROM/flash).

That's the sort of thing I was hoping to spark -- I hadn't seen XIP but
it does sound like a limited form of the kind of thing that would integrate
memory/disk into one large bowl of bits.  Thanks for the pointer!

> Anyway, from a mmap() perspective, we'd be logically merging the
> filesystem and pagecache layers and losing a layer of physical
> indirection.

Something like that... a smooth exchange between memory pages and disk
pages, where it would even be possible for a page to be part of both,
a bit like XIP indeed.

> One major difference between disk and RAM is the tradeoffs between size,
> speed, and price. It's highly unlikely that any one technology will ever
> beat all others in both of the usage patterns which are normal for RAM and
> disk in devices considered by their users to be "computers".

I beg to differ.  If the numbers above are anything to go on, that is.
We have a tendency to think of DRAM as something that can grow without
bounds, but its power consumption and consequential dissipation due to
refreshing do limit it, for example when applied in a mobile device.

Refreshes occur ~1000 times a second for all cells in DRAM, AFAIK.
That's rather a lot of data being read and rewritten each second!
No wonder the chips heat up, and consequently cannot be stacked to
form a cubic memory structure.

> I think current ramdisk code skips [buffering disk blocks].

That'd help, but I'm not sure the concepts of memory and ramdisk should
be kept separate if this architecture I am thinking of would come through.


You have helped to answer my question, whether Linux was ready for the
architecture that I talked about: it already contains seedlings for its
support, so it does not seem to conflict with Linux' major architecture.
That's good news!


Thanks,

Rick van Rein
GroenGemak

Re: Future Linux on Bistable Storage

[Andi Kleen]

Rick van Rein wrote:

> Thanks for your responses.  You convinced me I/O initiatilisation cannot
> be improved upon.

I'm sure it can. It's just significant work to do it and would need
to be done step by step. It's a bit like the "remove BKL everywhere"
project.

-Andi

Re: Future Linux on Bistable Storage

[Stefan Richter]

Andi Kleen wrote:
> Rick van Rein wrote:
>>> Rick van Rein <rick@groengemak.nl> writes:
>>>> Regarding I/O init: Current Linux boots while initiating one device at a
>>>> time.  Some degree of serialism must exist, but after scanning a bus would
>>>> it not be possible to fork threads to initiate each of the devices found
>>>> on it?
>>> 
>>> Multithreaded driver initialization has been tried, but reverted
>>> because it triggered too many driver bugs.
>>> 
>> Thanks for your responses.  You convinced me I/O initiatilisation cannot
>> be improved upon.
> 
> I'm sure it can. It's just significant work to do it and would need
> to be done step by step. It's a bit like the "remove BKL everywhere"
> project.

Several bus subsystems actually probe asynchronously (i.e. in parallel
to other bus subsystems) or even with some degree of internal parallelism.

Of course there are notable benefits in this only if the devices are
very slow to start up --- e.g. because they contain moving parts (this
kind of devices is not present in Rick's setup) or because they are
connected via high latency interconnects --- or if there is a massive
number of devices.
-- 
Stefan Richter
-=====-==--- -==- ---==
http://arcgraph.de/sr/

Re: Future Linux on Bistable Storage

[Pavel Machek]

Hi!

> But in the end I expect the entire system to switch itself off in between
> every two keystrokes (!) simply because loosing no data anywhere in the
> system means being able to resume operations at the next whim/interrupt.

Some of todays devices are designed to sleep between keystrokes,
including cellphones and olpc notebook. It is doable in todays
hardware... and my sleepy linux patches aim to do something like that.

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

Re: Future Linux on Bistable Storage

[Daniel J Blueman]

On 2 Jun, 14:40, Rick van Rein <rick@groengemak.nl> wrote:
> Hello,
>
> Future generations of Linux are likely to run on machines with non-volatile
> memories based on bistable technologies.  This will save the energy of DRAM
> refresh cycles and avoid the mechanical problems related to hard disks.  The
> result is probably a computer with no distinction between disk and RAM.
>
> Such computer architectures can conserve a lot of energy, as it is very
> simple to suspend and resume them: no data loss means no time needed to
> resume the system -- except perhaps for I/O initialisation.  Conserving
> energy means (1) having more fun|hours per kg of batteries, and (2) saving
> the planet.
>
> I wonder inhowfar Linux is (already) capable of dealing with such hardware.
> Several core concepts suddenly become meaningless if disk and RAM are merged
> into one storage device:
>  * swapping out data
>  * lazy loading of programs from disk to RAM
>  * buffering disk blocks
>  * mapping and unmapping disk onto RAM
[snip]

Perhaps one of the more key stepping stones here is execution in place
(XIP) support, which at present works for a particular combination of
embedded architecture, block device and filesystem. Maybe that's the
wrong way of looking at it, and we should consider XIP-for-ramdisk
(rather than ROM/flash).

Anyway, from a mmap() perspective, we'd be logically merging the
filesystem and pagecache layers and losing a layer of physical
indirection.
-- 
Daniel J Blueman