Re: garbage collect

Re: garbage collect

[David Woodhouse]

jgg@ualberta.ca said:
>  Well, we don't have any partitioning ability in the MTD stuff yet.

Not difficult to add, at least the simple and ugly way.

For each 'partition' define a new MTD device, for which all the methods 
simply add an offset and pass control to the method for the 'original' MTD 
device.

Ugly, but simple and effective.


--
dwmw2




To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: garbage collect

[Jason Gunthorpe]

On Wed, 28 Jun 2000, David Woodhouse wrote:

> >  Well, we don't have any partitioning ability in the MTD stuff yet.
> 
> Not difficult to add, at least the simple and ugly way.

We could put a partition going up backwards from the bottom of the first
erase block, that would probably not interfere with any boot loader
action. Any boot loaders that are larger than a single erase block can
integrate the ptable into their image somehow. I'd like to advoid using
any space at the start, often very specific things have to be put there,
and sometimes that window is very small..

Something like:

struct PartitionEntry
{
   __u32 Signature;
   __u32 StartEraseBlock;
   __u32 LengthInBlocks;
   __u32 Type;          
};

struct PartitionHeader
{
   struct PartitionEntry[PartitionCount];
   __u32 PartitionCount;
   __u32 Signature;
};

Which is rather alot like how FFS2 works.

> For each 'partition' define a new MTD device, for which all the methods 
> simply add an offset and pass control to the method for the 'original' MTD 
> device.

Yes, that is how other block devices work? Can we somehow reuse that
mechanism, whatever it is..

Jason



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: garbage collect

[David Woodhouse]

jgg@ualberta.ca said:
>  Yes, that is how other block devices work? Can we somehow reuse that
> mechanism, whatever it is.. 

You don't want to know. The existing partitioning stuff in Linux is 
absolutely disgusting, and it's likely to be replaced fairly early in 2.5



--
dwmw2




To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: garbage collect

[Nicolas Pitre]

On Wed, 28 Jun 2000, David Woodhouse wrote:

> 
> jgg@ualberta.ca said:
> >  Well, we don't have any partitioning ability in the MTD stuff yet.
> 
> Not difficult to add, at least the simple and ugly way.
> 
> For each 'partition' define a new MTD device, for which all the methods 
> simply add an offset and pass control to the method for the 'original' MTD 
> device.
> 
> Ugly, but simple and effective.

However it will byte you if you use the master device and partitioned
devices at the same time.  You'll end up with coherency problems in the
buffer cache.



Nicolas



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: garbage collect

[David Woodhouse]

On Wed, 28 Jun 2000, Nicolas Pitre wrote:

> However it will byte you if you use the master device and partitioned
> devices at the same time.  You'll end up with coherency problems in the
> buffer cache.

MTD devices are not even vaguely related to block devices, and don't go
anywhere near the buffer cache.

As with block devices, though, using both the whole device and subsets of
it simultaneously can lead to confusion. Dont Do That Then.

-- 
dwmw2




To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: garbage collect

[Nicolas Pitre]

On Wed, 28 Jun 2000, David Woodhouse wrote:

> On Wed, 28 Jun 2000, Nicolas Pitre wrote:
> 
> > However it will byte you if you use the master device and partitioned
> > devices at the same time.  You'll end up with coherency problems in the
> > buffer cache.
> 
> MTD devices are not even vaguely related to block devices, and don't go
> anywhere near the buffer cache.
> 
> As with block devices, though, using both the whole device and subsets of
> it simultaneously can lead to confusion. Dont Do That Then.

I was thinking about the MTD block interface of course.  That's the only
method to use conventional filesystems with.


Nicolas



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: garbage collect

[David Woodhouse]

nico@cam.org said:
>  I was thinking about the MTD block interface of course.  That's the
> only method to use conventional filesystems with. 

The only time the mtdblock interface is useful is when the underlying 
memory device is RAM. AFAIK Linux doesn't allow you to register a block 
device with a blocksize as large as the erase blocks of most flash chips we 
use.

So it's not really likely to be used that often, and especially not with 
partitioning. JFFS only uses it to find a handle to the real MTD device, 
and I'm hoping to change that fairly shortly.

--
dwmw2




To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: mtdblock interface

[Nicolas Pitre]

On Thu, 29 Jun 2000, David Woodhouse wrote:

> 
> nico@cam.org said:
> >  I was thinking about the MTD block interface of course.  That's the
> > only method to use conventional filesystems with. 
> 
> The only time the mtdblock interface is useful is when the underlying 
> memory device is RAM. AFAIK Linux doesn't allow you to register a block 
> device with a blocksize as large as the erase blocks of most flash chips we 
> use.
> 
> So it's not really likely to be used that often, and especially not with 
> partitioning. 

For those who are interested:

The SA1100 patches I maintain at
ftp://ftp.arm.linux.org.uk/pub/linux/arm/people/nico/ contains a block
driver for Intel flash parts i.e. Strataflash and friends. Since the erase
block size is at least 64k, the driver implements a local cache to allow
the use of small block with the kernel block device interface for write
access.

This driver is currently in use by many embedded projects so it has a
rough idea of different flash configurations, addressing methods and
partitions. This was developed in order to use generic filesystems like
ext2, cramfs, etc.  Of course, jffs could be a much better choice for
Flash storage, but it wasn't available at the time and read-only
filesystems like cramfs still require a block device.

The major item on the wish list is to port this driver (at least the
low-level flash access) into the MTD framework, but nobody currently using
it found the time to do so yet.  I wanted to mention it here just in case
someone else might be interested to look at it.


Nicolas



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: garbage collect

[Alan Cox]

> MTD devices are not even vaguely related to block devices, and don't go
> anywhere near the buffer cache.

The interesting question is should they. Withouth them going via the block
layer you wont be able to mirror them or potentially run a loopback fs over
them soon

> 



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: garbage collect

[David Woodhouse]

alan@lxorguk.ukuu.org.uk said:
> The interesting question is should they.

I don't (currently) think so. It's fairly difficult to fit flash into the
block device model.

JFFS is the primary user of MTD devices now - everything else (i.e. FTL,
NFTL) is a second class citizen. JFFS writes variable-size nodes, and doing
that on a block device would mean wasting RAM on grouping writes into
blocksized chunks.

And what do you set your blocksize to? On most NOR flash devices, it's 
something like 128Kb. We need to support sub-blocksize writes, and we need 
that to be explicitly controlled by the user, so it can optimally balance the 
journalling requirements with the desire to have as few write cycles as 
possible. 

> Withouth them going via the block layer you wont be able to mirror 
> them 

Mirrored flash? That's just sick. And anyway, it needs to be done at a 
higher level than the individual flash chips - on which you may have bad 
blocks in different places, which need to be mapped round differently on 
each chip.

Currently, if you want mirrored flash, you have to use {N,}FTL to provide
emulated block devices on top of which you stick your RAID array.

It's feasible to add direct mirroring support to JFFS, though, if it's 
really a necessary feature.

> or potentially run a loopback fs over them soon 

That's even sicker, and should arguably be done in userspace with nbd.


--
dwmw2




To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: garbage collect

[David Woodhouse]

alan@lxorguk.ukuu.org.uk said:
> > MTD devices are not even vaguely related to block devices, and don't
> > go anywhere near the buffer cache.

> The interesting question is should they. Withouth them going via the
> block layer you wont be able to mirror them or potentially run a
> loopback fs over them soon 

I had a chat with sct about this. For mirroring the answer is definitely 
"Do it in the VFS". 

Loopback I still maintain should be done via nbd.

--
dwmw2




To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: garbage collect

[dwmw2]

On Tue, 20 Jun 2000, Nick Ivanter wrote:

> I've compiled JFFS and MTD into the kernel and tried to play with it on
> the mtdram device. It seems to work, but when I tried to mount device
> without manually specifying a filesystem type, I got error: "Unable to
> handle kernel NULL pointer dereference at virtual address ...........
> Process mount........... Segmentation fault". When specifying -t jffs,
> everything seems to be ok.
> Does anyone have ideas about what may cause that?

Probably the mtdblock device falling over when a 'real' filesystem
actually causes a block read request, which jffs will never do because it
goes straight to the underlying MTD device.

What kernel version? Can you give me a backtrace of the oops?

--
dwmw2



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: garbage collect

[Nick Ivanter]

dwmw2@infradead.org wrote:

> On Tue, 20 Jun 2000, Nick Ivanter wrote:
>
> > I've compiled JFFS and MTD into the kernel and tried to play with it on
> > the mtdram device. It seems to work, but when I tried to mount device
> > without manually specifying a filesystem type, I got error: "Unable to
> > handle kernel NULL pointer dereference at virtual address ...........
> > Process mount........... Segmentation fault". When specifying -t jffs,
> > everything seems to be ok.
> > Does anyone have ideas about what may cause that?
>
> Probably the mtdblock device falling over when a 'real' filesystem
> actually causes a block read request, which jffs will never do because it
> goes straight to the underlying MTD device.
>
> What kernel version? Can you give me a backtrace of the oops?

Kernel 2.2.12. The exact error message is as following:
bash# mount /dev/mtd0 /mnt
Unable to handle kernel NULL pointer dereference at virtual address 0000000c

current->tss.cr3 = 01676000, %cr3 = 01676000
*pde = 00000000
Oops: 0002
CPU:    0
EIP:    0010:[<c014cd43>]
EFLAGS: 00010092
eax: c019549c   ebx: 00000000   ecx: 00000000   edx: 00000003
esi: 00000000   edi: 00000001   ebp: c15ba000   esp: c15bbcc8
ds: 0018   es: 0018   ss: 0018
Process mount (pid: 10, process nr: 7, stackpage=c15bb000)
Stack: 00000001 c0131c62 00000000 00000001 c0174880 c01b252c c0131f3f
00000001
       c01b252c 00000001 00000246 c15bbd28 c168f4d0 00000400 c014c190
00000000
       c012165a c01bb02c 00000001 c168f4d0 c168f4d0 c15bbe9c c15bbd28
c15bbd28
Call Trace: [<c0131c62>] [<c0174880>] [<c0131f3f>] [<c014c190>] [<c012165a>]
[<c01255ea>] [<c0121e2b>]
       [<c012208b>] [<c013ad74>] [<c0121e2b>] [<c012208b>] [<c013ad74>]
[<c0121e2b>] [<c012208b>] [<c012768c>]
       [<c01748a0>] [<c0111fa9>] [<c0131d0b>] [<c017489b>] [<c012126d>]
[<c01206e0>] [<c0120935>] [<c01079c8>]
Code: c7 46 0c 00 00 00 00 85 ff 75 44 ff 76 10 ff 74 24 1c 0f b7
Segmentation fault


Nick.



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: garbage collect

[David Woodhouse]

nick@auriga.ru said:
> bash# mount /dev/mtd0 /mnt 
> Unable to handle kernel NULL pointer dereference at virtual address 0000000c 

Cvs update and try again. Version 1.16 of mtdblock.c should fix this. I 
haven't tested it yet, but it was 'obviously broken'.

--
dwmw2




To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: garbage collect

[Nick Ivanter]

David Woodhouse wrote:

> nick@auriga.ru said:
> > bash# mount /dev/mtd0 /mnt
> > Unable to handle kernel NULL pointer dereference at virtual address 0000000c
>
> Cvs update and try again. Version 1.16 of mtdblock.c should fix this. I
> haven't tested it yet, but it was 'obviously broken'.
>
> --
> dwmw2

yes, now it works ok!

Nick.



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: garbage collect

[Nick Ivanter]

SИbastien CТtИ wrote:

> Nick Ivanter wrote:
>
> > I've compiled JFFS and MTD into the kernel and tried to play with it on
> > the mtdram device. It seems to work, but when I tried to mount device
> > without manually specifying a filesystem type, I got error: "Unable to
> > handle kernel NULL pointer dereference at virtual address ...........
> > Process mount........... Segmentation fault". When specifying -t jffs,
> > everything seems to be ok.
> > Does anyone have ideas about what may cause that?
>
> hmm.... when I try the same thing I get :
> mount: you must specify the filesystem type
>
> I get the same output with -t auto.....  try using -t auto and see if it
> segfaults.
>
> --
> SИbastien CТtИ

I've tried it. mount with -t auto segfaults in the same way.

But I noticed interesting thing: when I then try to mount it for the first
time either with -t auto or not, I get the error immediately after entering
the command. But when I try it for the second time (again doesn't matter
specifying -t auto or not), I get debugging output from the mtdblock before
the error message:

bash# mount /dev/mtd0 /mnt -t auto
mtdblock_open
ok
Unable to handle kernel NULL pointer dereference at virtual address 0000000c

current->tss.cr3 = 01676000, %cr3 = 01676000
*pde = 00000000
Oops: 0002
CPU:    0
EIP:    0010:[<c014cd43>]
EFLAGS: 00010092
..................
.................

Starting the third time and till reset, mounting /dev/mtd0 without
specifying filesystem type or with -t auto doesn't cause segfault:

bash# mount /dev/mtd0 /mnt
mtdblock_open
ok
mtdblock_release
ok
mount: you must specify the filesystem type



Nick.



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org

Re: garbage collect

[Nick Ivanter]

Nick Ivanter wrote:

> SИbastien CТtИ wrote:
>
> > Nick Ivanter wrote:
> >
> > > I've compiled JFFS and MTD into the kernel and tried to play with it on
> > > the mtdram device. It seems to work, but when I tried to mount device
> > > without manually specifying a filesystem type, I got error: "Unable to
> > > handle kernel NULL pointer dereference at virtual address ...........
> > > Process mount........... Segmentation fault". When specifying -t jffs,
> > > everything seems to be ok.
> > > Does anyone have ideas about what may cause that?
> >
> > hmm.... when I try the same thing I get :
> > mount: you must specify the filesystem type
> >
> > I get the same output with -t auto.....  try using -t auto and see if it
> > segfaults.
> >
> > --
> > SИbastien CТtИ
>
> I've tried it. mount with -t auto segfaults in the same way.
>
> But I noticed interesting thing: when I then try to mount it for the first

                                          ^^^^ sorry shouldn't be "then" here

>
> time either with -t auto or not, I get the error immediately after entering
> the command. But when I try it for the second time (again doesn't matter
> specifying -t auto or not), I get debugging output from the mtdblock before
> the error message:
>
> bash# mount /dev/mtd0 /mnt -t auto
> mtdblock_open
> ok
> Unable to handle kernel NULL pointer dereference at virtual address 0000000c
>
> current->tss.cr3 = 01676000, %cr3 = 01676000
> *pde = 00000000
> Oops: 0002
> CPU:    0
> EIP:    0010:[<c014cd43>]
> EFLAGS: 00010092
> ..................
> .................
>
> Starting the third time and till reset, mounting /dev/mtd0 without
> specifying filesystem type or with -t auto doesn't cause segfault:
>
> bash# mount /dev/mtd0 /mnt
> mtdblock_open
> ok
> mtdblock_release
> ok
> mount: you must specify the filesystem type
>
> Nick.



To unsubscribe, send "unsubscribe mtd" to majordomo@infradead.org