Ramdisks and tmpfs problems

Ramdisks and tmpfs problems

[Geoffrey Gallaway]

Hello,

I am attempting to create a central NFS server with a single slackware 8
installation that many boxes can use as their root disks. I got bootp kernel
level autoconfiguration working and the test box sucessfully mounts the root
(/) NFS share. I'm using floppy disks with kernels on diskless machines.

The problem occurs for /var, /tmp and /etc. Because each machine will need
it's own /var, /tmp and /etc I've been trying to create a ramdisk or tmpfs
filesystem for those partitions on each box. I've been using the system
initialization scripts to setup these directories and dynamically rewrite
important files (HOSTNAME, etc) in /etc.

Originally I started playing with ram disks but when I try to create a new
ramdisk with "mke2fs /dev/ram0 16384" mke2fs says:
mke2fs: Filesystem larger then apparent filesystem size.
Proceed anyway? (y,n) y
Warning: could not erase sector 2: Invalid arguement
Warning: could not erase sector 0: Attempt to write block from filesystem
resulted in short write
mke2fs: Invalid arguement zeroing block 16320 at end of filesystem

So no go with ram disks (this is kernel 2.4.18 on a 3 gig RAM dual PIII
1gig, BTW). So now to try tmpfs. Since I need to copy the existing files in
/etc off to tmpfs I have to create a "temporary" tmpfs, copy /etc off to it
then create another tmpfs on top of the existing /etc and copy from the
"temporary" tempfs back to the new /etc. I came up with the following 
commands:
mount -w -n -t tmpfs -o defaults tmpfs /mnt
cp -axf /etc /mnt
mount -w -t tmpfs -o defaults tmpfs /etc
cp -axf /mnt/etc/* /etc/
umount /mnt
# -- Reapeat for /var and /tmp --

Again, I put these commands in slackware's init scripts and it looks like
everything is working fine until the login prompt appears, at which time the
machine immediatly uncleanly reboots, eveytime without fail.

Anyone know what could be going on? I'm out of options as far as RAM-based
filesystems. :)

Thanks,
Geoffeg

Re: Ramdisks and tmpfs problems

[Andrew Morton]

Geoffrey Gallaway wrote:
> 
> ...
> Originally I started playing with ram disks but when I try to create a new
> ramdisk with "mke2fs /dev/ram0 16384" mke2fs says:
> mke2fs: Filesystem larger then apparent filesystem size.
> Proceed anyway? (y,n) y
> Warning: could not erase sector 2: Invalid arguement
> Warning: could not erase sector 0: Attempt to write block from filesystem
> resulted in short write
> mke2fs: Invalid arguement zeroing block 16320 at end of filesystem

Try omitting the `16384' option - let mke2fs work out the
size.

Re: Ramdisks and tmpfs problems

[Geoffrey Gallaway]

I tried that too, mke2fs says:
mke2fs: Device size reported to be zero. Invalid partition specified, or a
partition table wasn't reread after running fdisk, due to a modified
partition being busy and in use. You may need to reboot to re-read your
partition table.

This one time, at band camp, Andrew Morton wrote:
> Geoffrey Gallaway wrote:
> > 
> > ...
> > Originally I started playing with ram disks but when I try to create a new
> > ramdisk with "mke2fs /dev/ram0 16384" mke2fs says:
> > mke2fs: Filesystem larger then apparent filesystem size.
> > Proceed anyway? (y,n) y
> > Warning: could not erase sector 2: Invalid arguement
> > Warning: could not erase sector 0: Attempt to write block from filesystem
> > resulted in short write
> > mke2fs: Invalid arguement zeroing block 16320 at end of filesystem
> 
> Try omitting the `16384' option - let mke2fs work out the
> size.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: Ramdisks and tmpfs problems

[Sean Hunter]

On Tue, Apr 09, 2002 at 02:46:39PM -0400, Geoffrey Gallaway wrote:
> So no go with ram disks (this is kernel 2.4.18 on a 3 gig RAM dual PIII
> 1gig, BTW). So now to try tmpfs. Since I need to copy the existing files in
> /etc off to tmpfs I have to create a "temporary" tmpfs, copy /etc off to it
> then create another tmpfs on top of the existing /etc and copy from the
> "temporary" tempfs back to the new /etc. I came up with the following 
> commands:
> mount -w -n -t tmpfs -o defaults tmpfs /mnt
> cp -axf /etc /mnt
> mount -w -t tmpfs -o defaults tmpfs /etc
> cp -axf /mnt/etc/* /etc/
> umount /mnt
> # -- Reapeat for /var and /tmp --

Wouldn't this be easier?

mount -t tmpfs none /dev/shm
cp -axf /etc/* !$
mount --bind /dev/shm /etc

Re: Ramdisks and tmpfs problems

[Denis Vlasenko]

On 10 April 2002 05:45, Sean Hunter wrote:
> On Tue, Apr 09, 2002 at 02:46:39PM -0400, Geoffrey Gallaway wrote:
> > So no go with ram disks (this is kernel 2.4.18 on a 3 gig RAM dual PIII
> > 1gig, BTW). So now to try tmpfs. Since I need to copy the existing files
> > in /etc off to tmpfs I have to create a "temporary" tmpfs, copy /etc off
> > to it then create another tmpfs on top of the existing /etc and copy from
> > the "temporary" tempfs back to the new /etc. I came up with the following
> > commands:
> > mount -w -n -t tmpfs -o defaults tmpfs /mnt
> > cp -axf /etc /mnt
> > mount -w -t tmpfs -o defaults tmpfs /etc
> > cp -axf /mnt/etc/* /etc/
> > umount /mnt
> > # -- Reapeat for /var and /tmp --
>
> Wouldn't this be easier?
>
> mount -t tmpfs none /dev/shm
> cp -axf /etc/* !$
> mount --bind /dev/shm /etc

/dev is for devices, why do you use it for mounting filesystems?
I use /mnt/xxx or custom top-level dirs for mounts.

For example, I have /.share mounted to common NFS storage,
/.local mount --bind'ed to local (i.e. per-workstation) storage,
/mnt/auto is for automounter (local disk access, SMB etc)
and symlinks:
/usr -> /.share/usr
/home -> /.share/home
...
/tmp -> /.local/tmp
/var -> /.local/var
...

This works just fine for me.
--
vda

Re: Ramdisks and tmpfs problems

[Sean Hunter]

On Wed, Apr 10, 2002 at 01:31:14PM -0200, Denis Vlasenko wrote:
> On 10 April 2002 05:45, Sean Hunter wrote:
> > Wouldn't this be easier?
> >
> > mount -t tmpfs none /dev/shm
> > cp -axf /etc/* !$
> > mount --bind /dev/shm /etc
> 
> /dev is for devices, why do you use it for mounting filesystems?

Normally yes, but the tmpfs provides posix shared memory semantics and thus
/dev/shm is the "normal" place to mount it.  Don't blame me.

Sean

Re: Ramdisks and tmpfs problems

[Christoph Rohland]

Hi Sean,

On Wed, 10 Apr 2002, Sean Hunter wrote:
>> /dev is for devices, why do you use it for mounting filesystems?
> 
> Normally yes, but the tmpfs provides posix shared memory semantics
> and thus /dev/shm is the "normal" place to mount it.  Don't blame
> me.

Yes, and he does not want to use it for POSIX shared mem, but as a
local filesystem. So he should mount it where he needs it and
definitely not misunse the posix mount for different things.

Greetings
		Christoph

Re: Ramdisks and tmpfs problems

[Sean Hunter]

On Wed, Apr 10, 2002 at 01:52:37PM +0200, Christoph Rohland wrote:
> Hi Sean,
> 
> On Wed, 10 Apr 2002, Sean Hunter wrote:
> >> /dev is for devices, why do you use it for mounting filesystems?
> > 
> > Normally yes, but the tmpfs provides posix shared memory semantics
> > and thus /dev/shm is the "normal" place to mount it.  Don't blame
> > me.
> 
> Yes, and he does not want to use it for POSIX shared mem, but as a
> local filesystem. So he should mount it where he needs it and
> definitely not misunse the posix mount for different things.

The whole point was that he was doing extra copies and mount/unmounts that he
didn't need.   He couldn't just mount it in /etc/ in the first place because he
needed to copy stuff from the underlying fs that was there onto the tmpfs.

point -------------------------->
<------------- Christoph Rohland

Which is why I proposed two mounts:

(1) A mount under /dev/shm reflecting its nature and role as providing posix
shared mem (convenient because its not /etc where he already _has_ files)

(2) A bind mount under /etc reflecting its nature and role as providing a
ram-based file system (convenient because that's where he actually wants the fs
to be)

I just suggested that by mounting it what has been established as the canonical
place for mounting tmpfs and using a bind mount he doesn't need the extra
copies/mounts.

Sheesh.  Next thing you'll be asking if a filesystem can have buddha nature.

Sean "Mu" Hunter

Re: Ramdisks and tmpfs problems

[Denis Vlasenko]

On 9 April 2002 16:46, Geoffrey Gallaway wrote:
> I am attempting to create a central NFS server with a single slackware 8
> installation that many boxes can use as their root disks. I got bootp
> kernel level autoconfiguration working and the test box sucessfully mounts
> the root (/) NFS share. I'm using floppy disks with kernels on diskless
> machines.

Hi there! I've done this. I'm sitting on one of such diskless boxen now.
(actually, it has 2 disks, one is a 1.2G IDE turned into "diskette"
for home <-> work data transfer, other is *ugh* an NT installation).

> The problem occurs for /var, /tmp and /etc. Because each machine will need
> it's own /var, /tmp and /etc I've been trying to create a ramdisk or tmpfs
> filesystem for those partitions on each box. I've been using the system
> initialization scripts to setup these directories and dynamically rewrite
> important files (HOSTNAME, etc) in /etc.
[snip]

Looks like stuff for Al Viro. Consider mailing him with a report
(kernel/GCC version, symptoms etc).

> mount -w -n -t tmpfs -o defaults tmpfs /mnt
> cp -axf /etc /mnt
> mount -w -t tmpfs -o defaults tmpfs /etc
> cp -axf /mnt/etc/* /etc/
> umount /mnt
> # -- Reapeat for /var and /tmp --

Erm... just make a /etc_template, and you'll eliminate one cp.
--
vda

Update - Ramdisks and tmpfs problems

[Geoffrey Gallaway]

I finally found the problem, which appears to be a combination of things:

Multiple tmpfs mounts and SMP.

I am using a Dual Intel PIII 1Ghz box. When I use a SMP kernel AND do
multiple tmpfs mounts (mount --bind /dev/shm/etc /etc; mount --bind
/dev/shm/var /var) the machine goes into a reset loop. HOWEVER, when I use a
non-SMP kernel and still do multiple tmpfs mounts OR when I use a SMP kernel
and do only one tmpfs mount, the machine boots fine. Every once in a while
(1 out of 20 times?) the machine would boot fine with a SMP kernel and
multiple tmpfs mounts. Is this a timing issue?

If I can help to nail down this (apparent) bug more, please let me know.

Thanks,
Geoffeg

This one time, at band camp, Geoffrey Gallaway wrote:
> Hello,
> 
> I am attempting to create a central NFS server with a single slackware 8
> installation that many boxes can use as their root disks. I got bootp kernel
> level autoconfiguration working and the test box sucessfully mounts the root
> (/) NFS share. I'm using floppy disks with kernels on diskless machines.
> 
> The problem occurs for /var, /tmp and /etc. Because each machine will need
> it's own /var, /tmp and /etc I've been trying to create a ramdisk or tmpfs
> filesystem for those partitions on each box. I've been using the system
> initialization scripts to setup these directories and dynamically rewrite
> important files (HOSTNAME, etc) in /etc.
> 
> Originally I started playing with ram disks but when I try to create a new
> ramdisk with "mke2fs /dev/ram0 16384" mke2fs says:
> mke2fs: Filesystem larger then apparent filesystem size.
> Proceed anyway? (y,n) y
> Warning: could not erase sector 2: Invalid arguement
> Warning: could not erase sector 0: Attempt to write block from filesystem
> resulted in short write
> mke2fs: Invalid arguement zeroing block 16320 at end of filesystem
> 
> So no go with ram disks (this is kernel 2.4.18 on a 3 gig RAM dual PIII
> 1gig, BTW). So now to try tmpfs. Since I need to copy the existing files in
> /etc off to tmpfs I have to create a "temporary" tmpfs, copy /etc off to it
> then create another tmpfs on top of the existing /etc and copy from the
> "temporary" tempfs back to the new /etc. I came up with the following 
> commands:
> mount -w -n -t tmpfs -o defaults tmpfs /mnt
> cp -axf /etc /mnt
> mount -w -t tmpfs -o defaults tmpfs /etc
> cp -axf /mnt/etc/* /etc/
> umount /mnt
> # -- Reapeat for /var and /tmp --
> 
> Again, I put these commands in slackware's init scripts and it looks like
> everything is working fine until the login prompt appears, at which time the
> machine immediatly uncleanly reboots, eveytime without fail.
> 
> Anyone know what could be going on? I'm out of options as far as RAM-based
> filesystems. :)
> 
> Thanks,
> Geoffeg
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: Update - Ramdisks and tmpfs problems

[Sean Hunter]

Just a data point:  I have been using multiple tmpfs bind mounts for some time
on smp without this problem.

none on /dev/shm type tmpfs (rw,nosuid,nodev,mode=1777,size=256M)
/dev/shm on /tmp type none (rw,bind)
/dev/shm on /usr/tmp type none (rw,bind)

Never had a reboot fail for this reason.

Sean


On Wed, Apr 10, 2002 at 10:23:43AM -0400, Geoffrey Gallaway wrote:
> I finally found the problem, which appears to be a combination of things:
> 
> Multiple tmpfs mounts and SMP.
> 
> I am using a Dual Intel PIII 1Ghz box. When I use a SMP kernel AND do
> multiple tmpfs mounts (mount --bind /dev/shm/etc /etc; mount --bind
> /dev/shm/var /var) the machine goes into a reset loop. HOWEVER, when I use a
> non-SMP kernel and still do multiple tmpfs mounts OR when I use a SMP kernel
> and do only one tmpfs mount, the machine boots fine. Every once in a while
> (1 out of 20 times?) the machine would boot fine with a SMP kernel and
> multiple tmpfs mounts. Is this a timing issue?
> 
> If I can help to nail down this (apparent) bug more, please let me know.
> 
> Thanks,
> Geoffeg
> 

Re: Update - Ramdisks and tmpfs problems

[Geoffrey Gallaway]

What kernel version are you using on those smp+multi tmpfs boxes? What
hardware?

Geoffeg

This one time, at band camp, Sean Hunter wrote:
> Just a data point:  I have been using multiple tmpfs bind mounts for some time
> on smp without this problem.
> 
> none on /dev/shm type tmpfs (rw,nosuid,nodev,mode=1777,size=256M)
> /dev/shm on /tmp type none (rw,bind)
> /dev/shm on /usr/tmp type none (rw,bind)
> 
> Never had a reboot fail for this reason.
> 
> Sean
> 
> 
> On Wed, Apr 10, 2002 at 10:23:43AM -0400, Geoffrey Gallaway wrote:
> > I finally found the problem, which appears to be a combination of things:
> > 
> > Multiple tmpfs mounts and SMP.
> > 
> > I am using a Dual Intel PIII 1Ghz box. When I use a SMP kernel AND do
> > multiple tmpfs mounts (mount --bind /dev/shm/etc /etc; mount --bind
> > /dev/shm/var /var) the machine goes into a reset loop. HOWEVER, when I use a
> > non-SMP kernel and still do multiple tmpfs mounts OR when I use a SMP kernel
> > and do only one tmpfs mount, the machine boots fine. Every once in a while
> > (1 out of 20 times?) the machine would boot fine with a SMP kernel and
> > multiple tmpfs mounts. Is this a timing issue?
> > 
> > If I can help to nail down this (apparent) bug more, please let me know.
> > 
> > Thanks,
> > Geoffeg
> > 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: Update - Ramdisks and tmpfs problems

[Sean Hunter]

On Wed, Apr 10, 2002 at 11:01:07AM -0400, Geoffrey Gallaway wrote:
> What kernel version are you using on those smp+multi tmpfs boxes? 

As of this morning, 2.5.7.  Before that 2.4.{17,18,19-pre*ac*} for some months.


> What
> hardware?

Dual pentium III (Katmai) 

sean@henry:~$ lspci
00:00.0 Host bridge: Intel Corporation 440BX/ZX - 82443BX/ZX Host bridge (rev 02)
00:01.0 PCI bridge: Intel Corporation 440BX/ZX - 82443BX/ZX AGP bridge (rev 02)
00:04.0 ISA bridge: Intel Corporation 82371AB PIIX4 ISA (rev 02)
00:04.1 IDE interface: Intel Corporation 82371AB PIIX4 IDE (rev 01)
00:04.2 USB Controller: Intel Corporation 82371AB PIIX4 USB (rev 01)
00:04.3 Bridge: Intel Corporation 82371AB PIIX4 ACPI (rev 02)
00:06.0 SCSI storage controller: Adaptec AHA-2940U2/W / 7890
00:09.0 Multimedia audio controller: Creative Labs SB Live! EMU10000 (rev 04)
00:09.1 Input device controller: Creative Labs SB Live! (rev 01)
00:0b.0 Ethernet controller: Intel Corporation 82557 [Ethernet Pro 100] (rev 05)
00:0c.0 Ethernet controller: VIA Technologies, Inc. VT86C100A [Rhine 10/100] (rev 06)
01:00.0 VGA compatible controller: 3Dfx Interactive, Inc. Voodoo 3 (rev 01)

All scsi disks.  IDE cd-writer.  More-or-less redhat 7.x distro with most of
the interesting daemons replaced, and some legacy cruft still intact.

I don't use ACPI or devfs or framebuffer console, not that any of those should
affect this.

Sean

> 
> Geoffeg
> 
> This one time, at band camp, Sean Hunter wrote:
> > Just a data point:  I have been using multiple tmpfs bind mounts for some time
> > on smp without this problem.
> > 
> > none on /dev/shm type tmpfs (rw,nosuid,nodev,mode=1777,size=256M)
> > /dev/shm on /tmp type none (rw,bind)
> > /dev/shm on /usr/tmp type none (rw,bind)
> > 
> > Never had a reboot fail for this reason.
> > 
> > Sean
> > 
> > 
> > On Wed, Apr 10, 2002 at 10:23:43AM -0400, Geoffrey Gallaway wrote:
> > > I finally found the problem, which appears to be a combination of things:
> > > 
> > > Multiple tmpfs mounts and SMP.
> > > 
> > > I am using a Dual Intel PIII 1Ghz box. When I use a SMP kernel AND do
> > > multiple tmpfs mounts (mount --bind /dev/shm/etc /etc; mount --bind
> > > /dev/shm/var /var) the machine goes into a reset loop. HOWEVER, when I use a
> > > non-SMP kernel and still do multiple tmpfs mounts OR when I use a SMP kernel
> > > and do only one tmpfs mount, the machine boots fine. Every once in a while
> > > (1 out of 20 times?) the machine would boot fine with a SMP kernel and
> > > multiple tmpfs mounts. Is this a timing issue?
> > > 
> > > If I can help to nail down this (apparent) bug more, please let me know.
> > > 
> > > Thanks,
> > > Geoffeg
> > > 
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at  http://www.tux.org/lkml/
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: Update - Ramdisks and tmpfs problems

[Denis Vlasenko]

On 10 April 2002 12:23, Geoffrey Gallaway wrote:
> I finally found the problem, which appears to be a combination of things:
>
> Multiple tmpfs mounts and SMP.
>
> I am using a Dual Intel PIII 1Ghz box. When I use a SMP kernel AND do
> multiple tmpfs mounts (mount --bind /dev/shm/etc /etc; mount --bind
> /dev/shm/var /var) the machine goes into a reset loop. HOWEVER, when I use
> a non-SMP kernel and still do multiple tmpfs mounts OR when I use a SMP
> kernel and do only one tmpfs mount, the machine boots fine. Every once in a
> while (1 out of 20 times?) the machine would boot fine with a SMP kernel
> and multiple tmpfs mounts. Is this a timing issue?

Yes, sounds like race. It seems locking isn't quite right in tmpfs.

BTW, I'd like to know why do you want to use temporary storage
(tmpfs/shm/ramdisk) for /var and /etc? In 'normal' Unix (i.e. not a special
setup like computing farm) they are expected to survive reboot. Only /tmp
is volatile. You want to keep per-workstation config files (/etc) and data
(/var) across reboot, right?

Since my workstations aren't overstuffed with RAM, I use server supplied
storage for everything. If I will have a multigig RAM box someday, I'll
use tmpfs *only* for /tmp.
--
vda

Re: Update - Ramdisks and tmpfs problems

[Christoph Rohland]

Hi Denis,

On Thu, 11 Apr 2002, Denis Vlasenko wrote:
>> I am using a Dual Intel PIII 1Ghz box. When I use a SMP kernel AND
>> do multiple tmpfs mounts (mount --bind /dev/shm/etc /etc; mount
>> --bind /dev/shm/var /var) the machine goes into a reset
>> loop. HOWEVER, when I use a non-SMP kernel and still do multiple
>> tmpfs mounts OR when I use a SMP kernel and do only one tmpfs
>> mount, the machine boots fine. Every once in a while (1 out of 20
>> times?) the machine would boot fine with a SMP kernel and multiple
>> tmpfs mounts. Is this a timing issue?
> 
> Yes, sounds like race. It seems locking isn't quite right in tmpfs.

I am not sure. Could also be bad ram which only triggers under higher
load. I never had instant reboots on my machines due to races, but
only when I had bad DIMMs.

I also have multiple tmpfs instances on my 8-way test box where I
developed and tested tmpfs.

But you could be right. Unfortunately I do not have the time to
investigate further.

Greetings
		Christoph