OOM while deleting very large files

OOM while deleting very large files

[Alexander Schier]

Hi!
I had a large Nilfs2 (about 250 GB), and 2 files with about 100 GB in
it. When i tried to delete the file, either
rm bigfile1

or after it failed

echo '' >bigfile1

the pc worked a long time, and then nilfs2 used so much ram, that the
kernel killed all programs, and finally the pc was unusable. It was not
possible to delete the files, other than reformatting the partition.

At the moment, i can not do any further tests, because the partition is
now filled with useful files, and i do not want to fill the rest with
undeletable files. But maybe you can find the memleak(?) in the Source.

Alex

Re: OOM while deleting very large files

[Ryusuke Konishi]

Hi Alex,

Alexander Schier wrote
Date: Sun, 4 May 2008 18:54:26 +0200
> Hi!
> I had a large Nilfs2 (about 250 GB), and 2 files with about 100 GB in
> it. When i tried to delete the file, either
> rm bigfile1
> 
> or after it failed
> 
> echo '' >bigfile1
> 
> the pc worked a long time, and then nilfs2 used so much ram, that the
> kernel killed all programs, and finally the pc was unusable. It was not
> possible to delete the files, other than reformatting the partition.
> 
> At the moment, i can not do any further tests, because the partition is
> now filled with useful files, and i do not want to fill the rest with
> undeletable files. But maybe you can find the memleak(?) in the Source.
> 
> Alex

Thanks for reporting the above problem!
It seems to be a likely problem.

Unfortunately we are now in holiday periods in Japan (Golden Week
holidays).  So, I can't test it.
I'd like to look into the problem after the holidays, maybe on Wednesday
or Thursday.

Before that, could you tell me some information on your PC ?
e.g. NILFS version, kernel version, memory amount, and so on.

Cheers,
Ryusuke Konishi

Re: OOM while deleting very large files

[Alexander Schier]

Hi!
On Tue, May 06, 2008 at 01:18:37AM +0900, Ryusuke Konishi wrote:
It is not so important, as i do not need to use such big files further,
so you can have holydays ;).

For the fixing later:
PC: 
CPU: AMD Athlon XP 1,6+
RAM: 512 MB DDR RAM
Nilfs2 Version (Module and Tools): 2.0.2
Partition: 251 GB dmcrypt
Files: 2x approx. 100 GB created with dd if=/dev/zero of=bigfile, to
erase the old content on the cryptopartition (the zeros are written
encrypted to the real partition)

Problem while deleting them as described:
> > rm bigfile1
> > 
> > or after it failed
> > 
> > echo '' >bigfile1
> > 
> > the pc worked a long time, and then nilfs2 used so much ram, that the
> > kernel killed all programs, and finally the pc was unusable. It was not
> > possible to delete the files, other than reformatting the partition.

Now i'm using the same setup with a nilfs2 with "normal" files on it,
filled to 11 GB (df -h), without any further problems. 

I do not want to
experiment in a way, that the partition needs to be reformatted later
again :/.

Alex

Re: OOM while deleting very large files

[Ryusuke Konishi]

Hi,

Alexander Schier wrote:
Date: Mon, 5 May 2008 18:53:31 +0200
> For the fixing later:
> PC: 
> CPU: AMD Athlon XP 1,6+
> RAM: 512 MB DDR RAM
> Nilfs2 Version (Module and Tools): 2.0.2
> Partition: 251 GB dmcrypt
> Files: 2x approx. 100 GB created with dd if=/dev/zero of=bigfile, to
> erase the old content on the cryptopartition (the zeros are written
> encrypted to the real partition)

Thanks!

NILFS on dm-crypted partition!  Interesting.

> It is not so important, as i do not need to use such big files further,
> so you can have holydays ;).

Yeah, I will!

Thanks,
Ryusuke Konishi.

> Problem while deleting them as described:
> > > rm bigfile1
> > > 
> > > or after it failed
> > > 
> > > echo '' >bigfile1
> > > 
> > > the pc worked a long time, and then nilfs2 used so much ram, that the
> > > kernel killed all programs, and finally the pc was unusable. It was not
> > > possible to delete the files, other than reformatting the partition.
> 
> Now i'm using the same setup with a nilfs2 with "normal" files on it,
> filled to 11 GB (df -h), without any further problems. 
> 
> I do not want to
> experiment in a way, that the partition needs to be reformatted later
> again :/.
> 
> Alex