duplicate DoC millenium with dd

duplicate DoC millenium with dd

[Nikolai Vladychevski]

Hi, 

for some reason I can not duplicate Disc on chip Millenium (8MB) with
dd command. I can build it using dd, fdisk, nftla_format, erase.... etc, but 
not only dd. 


This is how I do it: 


#! /bin/bash 

flash_on 

insmod docecc
insmod doc2001
insmod docprobe 

eraseall /dev/mtd0 

dd conv=notrunc if=docipl of=/dev/mtd0
dd conv=notrunc if=docipl of=/dev/mtd0 seek=1
dd conv=notrunc if=linuxbios.block of=/dev/mtd0 seek=2
dd conv=notrunc if=vmlinux.bin.gz.block of=/dev/mtd0 seek=128 

nftl_format /dev/mtd0 0x100000
sfdisk /dev/nftla <<EOF
0,,83
EOF 

mkcramfs /q3 /dev/nftla1 

rmmod docprobe
rmmod doc2001
rmmod docecc 


I shutdown, I boot, the image is working ok. 

Now I copy mtd image to a file: 

[root@qis1 ecs]# dd conv=notrunc if=/dev/mtd0 of=rom.img
16384+0 records in
16384+0 records out
[root@qis1 ecs]# 

now I copy the the _same_ image to the _same_ chip again: 

[root@qis1 /root]# flash_on
[root@qis1 /root]# cd /home/niko/qisbuild/freebios/ecs
[root@qis1 ecs]# insmod docecc
Using /lib/modules/2.4.7-normal/kernel/drivers/mtd/devices/docecc.o
[root@qis1 ecs]# insmod doc2001
Using /lib/modules/2.4.7-normal/kernel/drivers/mtd/devices/doc2001.o
[root@qis1 ecs]# insmod docprobe
Using /lib/modules/2.4.7-normal/kernel/drivers/mtd/devices/docprobe.o
[root@qis1 ecs]# dd conv=notrunc if=rom.img of=/dev/mtd0
16384+0 records in
16384+0 records out
[root@qis1 ecs]# 

I shutdown, I boot again , and I get this errors: 

M-Systems DiskOnChip driver. (C) 1999 Machine Vision Holdings, Inc.
Using configured probe address 0xfffc8000
DiskOnChip Millennium found at address 0xFFFC8000
Flash chip found: Manufacturer ID: 98, Chip ID: E6 (Toshiba TC58V64AFT/DC)
1 flash chips found. Total DiskOnChip size: 8 MiB
NFTL driver: nftlcore.c $Revision: 1.78 $, nftlmount.c $Revision: 1.22 $
Cannot calculate an NFTL geometry to match size of 0x37c0.
Using C:1019 H:14 S:1 (== 0x37ba sects)
nftla: unknown partition table
usb.c: registered new driver hub
NET4: Linux TCP/IP 1.0 for NET4.0
IP Protocols: ICMP, UDP, TCP, IGMP
IP: routing cache hash table of 512 buckets, 4Kbytes
TCP: Hash tables configured (established 8192 bind 8192)
NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
nftla1: bad access: block = 2, count = 2
end_request: I/O error, dev 5d:01 (unknown), sector 2
EXT3-fs: unable to read superblock
nftla1: bad access: block = 2, count = 2
end_request: I/O error, dev 5d:01 (unknown), sector 2
EXT2-fs: unable to read superblock
cramfs: wrong magic
nftla1: bad access: block = 0, count = 2
end_request: I/O error, dev 5d:01
(unknown), sector 0
romfs: unable to read superblock
Kernel panic: VFS: Unable to mount root fs on 5d:01 


If I add eraseall /dev/mtd0 just before using dd command I get this errors: 

Flash chip found: Manufacturer ID: 98, Chip ID: E6 (Toshiba TC58V64AFT/DC)
1 flash chips found. Total DiskOnChip size: 8 MiB
NFTL driver: nftlcore.c $Revision: 1.78 $, nftlmount.c $Revision: 1.22 $
Formatting block 130
Formatting block 131
Formatting block 132
Formatting block 133
Formatting block 134
.....
Formatting block 1023
bad access: block = 2, count =2
EXT2-fs: unable to read superblock 

I check again to make sure the image is the same and is not corrupted: 

[root@qis1 ecs]# insmod docecc
Using /lib/modules/2.4.7-normal/kernel/drivers/mtd/devices/docecc.o
[root@qis1 ecs]# insmod doc2001
Using /lib/modules/2.4.7-normal/kernel/drivers/mtd/devices/doc2001.o
[root@qis1 ecs]# flash_on
[root@qis1 ecs]# insmod docprobe
Using /lib/modules/2.4.7-normal/kernel/drivers/mtd/devices/docprobe.o
[root@qis1 ecs]# dd conv=notrunc if=/dev/mtd0 of=rom2.img
16384+0 records in
16384+0 records out
[root@qis1 ecs]# md5sum rom2.img rom.img
bf7d9fcaaba3d05245c17b67a738b9a1  rom2.img
bf7d9fcaaba3d05245c17b67a738b9a1  rom.img
[root@qis1 ecs]# 

the images are the same.... what I am missing? I thought that accessing
the DoC trough /dev/mtd0 I was accessing the whole chip, is therere
any other place that data is stored? Do I have to nftla_format again?
How to duplicate the Disk on Chip Millenium only with dd and no ohter tool? 


Nikolai

Re: duplicate DoC millenium with dd

[Nikolai Vladychevski]

David Woodhouse writes: 

> 
> niko@isl.net.mx said:
>>  for some reason I can not duplicate Disc on chip Millenium (8MB) with
>> dd command. I can build it using dd, fdisk, nftla_format, erase....
>> etc, but  not only dd.  
> 
> The DiskOnChip uses NAND flash, which has 'out of band' data areas. A 
> simple dd won't read or write them. Neither would it be sensible to use the 
> nanddump and nandwrite utilities for doing so, although they can - you want 
> to reformat each device individually, with its own bad block map, not the 
> bad block map of your master.

ohh, so bad.... I will have to nftl_format every time I update my flash.... 

> 
> nftl_format, mkfs and tar or dump/restore are sensible. 
> 
> What are you partitioning it for, btw?

It's a firewall. I place linuxbios & kernel first, then I nftl_format 
/dev/mtd0 0xFIRST_FREE_BLOCK and place a minimal filesystem (cramfs) on the 
rest of the chip..... since you say nftl_format removes original badblocks 
table M-Systems put on the chip at fabrication time, I kinda dislike this 
util..... 

is there any way to avoid nftl_format and boot my cramfs from /dev/mtdblockX 
or /dev/mtd0 with an 0xXXXX offset or something like this? 


Nikolai 

Re: duplicate DoC millenium with dd

[David Woodhouse]

niko@isl.net.mx said:
>  for some reason I can not duplicate Disc on chip Millenium (8MB) with
> dd command. I can build it using dd, fdisk, nftla_format, erase....
> etc, but  not only dd.  

The DiskOnChip uses NAND flash, which has 'out of band' data areas. A 
simple dd won't read or write them. Neither would it be sensible to use the 
nanddump and nandwrite utilities for doing so, although they can - you want 
to reformat each device individually, with its own bad block map, not the 
bad block map of your master.

nftl_format, mkfs and tar or dump/restore are sensible.

What are you partitioning it for, btw?

--
dwmw2

Re: duplicate DoC millenium with dd

[Nikolai Vladychevski]

David Woodhouse writes: 


> 
>> > What are you partitioning it for, btw?
>> It's a firewall. I place linuxbios & kernel first, then I nftl_format
>> /dev/mtd0 0xFIRST_FREE_BLOCK and place a minimal filesystem (cramfs)
>> on the  rest of the chip.....
> 
> You nftl_format it with the offset, which gives you a block device /dev/
> nftla. Then you use sfdisk rather than mkcramfs -o /dev/nftla. If you only 
> have one partition, then a partition table is just a waste of space.

right! thanks, didn't note it.... 

>  
> 
>> since you say nftl_format removes
>> original badblocks  table M-Systems put on the chip at fabrication
>> time, I kinda dislike this  util..... 
> 
> It's now better than it was. And you shouldn't need to use it more than 
> once, if the total size of your LinuxBIOS and kernel don't change (or if 
> you left enough space for a little bit of expansion).

yes, this is the problem... right now I'm leaving 1 Meg for the linuxBIOS 
and kernel and right now there is left as little as 40 K bytes free space of 
that meg. 

I beleive the kernel will grow very soon and 40 K will not be enough. 
Allocating more space, 1.5Meg for example seems to be a huge waste of space.
I think I will have to nftl_format with a variable offset every time I do an 
update..... you say nftl_format is better, but how much? If I update the 
software on the chip once a week (it's an automatic update via remote 
server), will it survive for 2 years without errors, for example ? 

> 
>>  is there any way to avoid nftl_format and boot my cramfs from /dev/
>> mtdblockX  or /dev/mtd0 with an 0xXXXX offset or something like this? 
> 
> Not out of the box. It's trivial to do that though, by using the mtd
> version of partitions, which makes it look like two (or more) separate MTD
> devices. 

Wow! that sounds cool! I have to try it, but would linuxBIOS conflict with 
it? 


Nikolai 

Re: duplicate DoC millenium with dd

[David Woodhouse]

niko@isl.net.mx said:
>  ohh, so bad.... I will have to nftl_format every time I update my
> flash....  

Not at all - no more than you need to do a low level format of your hard 
drive every time you put a new filesystem on it. Just do the mkcramfs step 
again.

> > What are you partitioning it for, btw?
> It's a firewall. I place linuxbios & kernel first, then I nftl_format
> /dev/mtd0 0xFIRST_FREE_BLOCK and place a minimal filesystem (cramfs)
> on the  rest of the chip.....

You nftl_format it with the offset, which gives you a block device /dev/
nftla. Then you use sfdisk rather than mkcramfs -o /dev/nftla. If you only 
have one partition, then a partition table is just a waste of space.


> since you say nftl_format removes
> original badblocks  table M-Systems put on the chip at fabrication
> time, I kinda dislike this  util..... 

It's now better than it was. And you shouldn't need to use it more than 
once, if the total size of your LinuxBIOS and kernel don't change (or if 
you left enough space for a little bit of expansion).

>  is there any way to avoid nftl_format and boot my cramfs from /dev/
> mtdblockX  or /dev/mtd0 with an 0xXXXX offset or something like this? 

Not out of the box. It's trivial to do that though, by using the mtd
version of partitions, which makes it look like two (or more) separate MTD
devices. 

--
dwmw2

Re: duplicate DoC millenium with dd

[David Woodhouse]

niko@isl.net.mx said:
>  yes, this is the problem... right now I'm leaving 1 Meg for the
> linuxBIOS  and kernel and right now there is left as little as 40 K
> bytes free space of  that meg. 

Use modules?


>  If I update the  software on the chip once a week (it's an automatic
> update via remote  server), will it survive for 2 years without
> errors, for example ?  

It doesn't degrade over time. Using it the first time may lose the 
bad-block information which is put on it by the NAND flash chip 
manufacturer (Toshiba or Samsung, not M-Systems). After that there's no 
difference.

>  Wow! that sounds cool! I have to try it, but would linuxBIOS conflict
> with  it?  

No, should be fine. You could have it appear as three MTD devices - one for 
LinuxBIOS, one for the kernel, and one for cramfs.

Using cramfs directly on the NAND flash gives you no error correction and 
no facility to work around bad blocks. I wouldn't wan to do it like that in 
production, although in practice it'll work unless you have a chip with bad 
blocks.

--
dwmw2

Re: duplicate DoC millenium with dd

[Nikolai Vladychevski]

David Woodhouse writes: 

> 
> niko@isl.net.mx said:
>>  yes, this is the problem... right now I'm leaving 1 Meg for the
>> linuxBIOS  and kernel and right now there is left as little as 40 K
>> bytes free space of  that meg. 
> 
> Use modules?

yes, I do, but it still big, its a 2.4.7 

>  
> 
>>  If I update the  software on the chip once a week (it's an automatic
>> update via remote  server), will it survive for 2 years without
>> errors, for example ?  
> 
> It doesn't degrade over time. Using it the first time may lose the 
> bad-block information which is put on it by the NAND flash chip 
> manufacturer (Toshiba or Samsung, not M-Systems). After that there's no 
> difference. 
> 
>>  Wow! that sounds cool! I have to try it, but would linuxBIOS conflict
>> with  it?  
> 
> No, should be fine. You could have it appear as three MTD devices - one for 
> LinuxBIOS, one for the kernel, and one for cramfs.

I'm sorry, I can't find any docs about creating partitions, mtd0, mtd1, 
mtd2, etc..? list archives dont help either, how it is done? 


>
> Using cramfs directly on the NAND flash gives you no error correction and 

but wait a minute, if I enable these options: 

   <*> NAND Device support
     [*] Enable ECC correction algorithm
     [*] Verify NAND page writes 

won't it help me? 

> no facility to work around bad blocks. I wouldn't wan to do it like that in 
> production, although in practice it'll work unless you have a chip with bad 
> blocks.

well, my environment will be production,so I guess I'm out of luck because 
if I take your precautions: 

1) I can't use nftl_format
2) I can't use mtd partitioning because I will write directly on the NAND 
flash..... 

this is kinda bad news....... 


nikolai