No space left on device after many files creation

No space left on device after many files creation

[cy6erGn0m]

[-- Attachment #1: Type: text/plain, Size: 938 bytes --]

I have found issue with ext4 partition.
I created many many empty files with the following way:
[cy6ergn0m@cgmachine heap]$ seq 1 10000000 | xargs touch

But this command crashes asfter a while with a error 'no space left on
device'. At the same time, df -h shows that I have 7Gb free on this file
system. So, I can't create any file in any directory at this filesystem.

[cy6ergn0m@cgmachine heap]$ pwd
/e4/sandbox/heap
[cy6ergn0m@cgmachine heap]$ ls -1 | wc -l
1261144
[cy6ergn0m@cgmachine heap]$ df -h /e4
Файловая система      Разм  Исп  Дост  Исп% смонтирована на
(filesystem/size/used/available/used%/mounted)
/dev/sda4              28G  7.2G   19G  28% /e4
[cy6ergn0m@cgmachine heap]$ touch ttt
touch: невозможно выполнить touch для `ttt': На устройстве кончилось место
(touch: failed to execute touch for 'ttt': No space left on device)



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 3650 bytes --]

Re: No space left on device after many files creation

[Eric Sandeen]

cy6erGn0m wrote:
> I have found issue with ext4 partition.
> I created many many empty files with the following way:
> [cy6ergn0m@cgmachine heap]$ seq 1 10000000 | xargs touch
> 
> But this command crashes asfter a while with a error 'no space left on
> device'. At the same time, df -h shows that I have 7Gb free on this file
> system. So, I can't create any file in any directory at this filesystem.
> 
> [cy6ergn0m@cgmachine heap]$ pwd
> /e4/sandbox/heap
> [cy6ergn0m@cgmachine heap]$ ls -1 | wc -l
> 1261144
> [cy6ergn0m@cgmachine heap]$ df -h /e4
> Файловая система      Разм  Исп  Дост  Исп% смонтирована на
> (filesystem/size/used/available/used%/mounted)
> /dev/sda4              28G  7.2G   19G  28% /e4

try df -i

There is a limited number of inodes that you can create, as set
at mkfs time.

-Eric

> [cy6ergn0m@cgmachine heap]$ touch ttt
> touch: невозможно выполнить touch для `ttt': На устройстве кончилось место
> (touch: failed to execute touch for 'ttt': No space left on device)
> 
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: No space left on device after many files creation

[cy6erGn0m]

[-- Attachment #1: Type: text/plain, Size: 1753 bytes --]

In the past I have done the same test for ext3 and it was created ~10M
files in one firectory at working filesystem (/home) and it was no
issues reached.

On ext4 i found inodes max size is limited as you wrote:
[root@cgmachine sandbox]# df -i /e4
Файловая система      Инодов   Испол   Своб  Исп % смонтирована на
(file system/inodes/used/free/used%/mounted on)
/dev/sda4               1,8M    1,8M       0  100% /e4
[root@cgmachine sandbox]#

Why inodes count limit is so small by default? Is it depends on
partition size?

Eric Sandeen wrote:
> cy6erGn0m wrote:
>   
>> I have found issue with ext4 partition.
>> I created many many empty files with the following way:
>> [cy6ergn0m@cgmachine heap]$ seq 1 10000000 | xargs touch
>>
>> But this command crashes asfter a while with a error 'no space left on
>> device'. At the same time, df -h shows that I have 7Gb free on this file
>> system. So, I can't create any file in any directory at this filesystem.
>>
>> [cy6ergn0m@cgmachine heap]$ pwd
>> /e4/sandbox/heap
>> [cy6ergn0m@cgmachine heap]$ ls -1 | wc -l
>> 1261144
>> [cy6ergn0m@cgmachine heap]$ df -h /e4
>> Файловая система      Разм  Исп  Дост  Исп% смонтирована на
>> (filesystem/size/used/available/used%/mounted)
>> /dev/sda4              28G  7.2G   19G  28% /e4
>>     
>
> try df -i
>
> There is a limited number of inodes that you can create, as set
> at mkfs time.
>
> -Eric
>
>   
>> [cy6ergn0m@cgmachine heap]$ touch ttt
>> touch: невозможно выполнить touch для `ttt': На устройстве кончилось место
>> (touch: failed to execute touch for 'ttt': No space left on device)
>>
>>
>>     
>
>   


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 3650 bytes --]

Re: No space left on device after many files creation

[Eric Sandeen]

cy6erGn0m wrote:
> In the past I have done the same test for ext3 and it was created ~10M
> files in one firectory at working filesystem (/home) and it was no
> issues reached.
> 
> On ext4 i found inodes max size is limited as you wrote:
> [root@cgmachine sandbox]# df -i /e4
> Файловая система      Инодов   Испол   Своб  Исп % смонтирована на
> (file system/inodes/used/free/used%/mounted on)
> /dev/sda4               1,8M    1,8M       0  100% /e4
> [root@cgmachine sandbox]#
> 
> Why inodes count limit is so small by default? Is it depends on
> partition size?

it does depend on partition size; the inode_ratio = 16384 in mke2fs.conf
says 1 inode per 16k of filesystem space.

the inode size was also increased a couple years ago, and the default
inode count was reduced by half so that the total size remained the same.

-Eric

> Eric Sandeen wrote:
>> cy6erGn0m wrote:
>>   
>>> I have found issue with ext4 partition.
>>> I created many many empty files with the following way:
>>> [cy6ergn0m@cgmachine heap]$ seq 1 10000000 | xargs touch
>>>
>>> But this command crashes asfter a while with a error 'no space left on
>>> device'. At the same time, df -h shows that I have 7Gb free on this file
>>> system. So, I can't create any file in any directory at this filesystem.
>>>
>>> [cy6ergn0m@cgmachine heap]$ pwd
>>> /e4/sandbox/heap
>>> [cy6ergn0m@cgmachine heap]$ ls -1 | wc -l
>>> 1261144
>>> [cy6ergn0m@cgmachine heap]$ df -h /e4
>>> Файловая система      Разм  Исп  Дост  Исп% смонтирована на
>>> (filesystem/size/used/available/used%/mounted)
>>> /dev/sda4              28G  7.2G   19G  28% /e4
>>>     
>> try df -i
>>
>> There is a limited number of inodes that you can create, as set
>> at mkfs time.
>>
>> -Eric
>>
>>   
>>> [cy6ergn0m@cgmachine heap]$ touch ttt
>>> touch: невозможно выполнить touch для `ttt': На устройстве кончилось место
>>> (touch: failed to execute touch for 'ttt': No space left on device)
>>>
>>>
>>>     
>>   
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: No space left on device after many files creation

[tytso]

On Mon, Apr 12, 2010 at 05:00:37PM -0500, Eric Sandeen wrote:
> cy6erGn0m wrote:
> > In the past I have done the same test for ext3 and it was created ~10M
> > files in one firectory at working filesystem (/home) and it was no
> > issues reached.
> > 
> > On ext4 i found inodes max size is limited as you wrote:
> > [root@cgmachine sandbox]# df -i /e4
> > Файловая система      Инодов   Испол   Своб  Исп % смонтирована на
> > (file system/inodes/used/free/used%/mounted on)
> > /dev/sda4               1,8M    1,8M       0  100% /e4
> > [root@cgmachine sandbox]#
> > 
> > Why inodes count limit is so small by default? Is it depends on
> > partition size?
> 
> it does depend on partition size; the inode_ratio = 16384 in mke2fs.conf
> says 1 inode per 16k of filesystem space.

1.8 million inodes is hardly small.....  most people complain that
we're reserving too much space for inodes.  

Cy6erGn0m, What is your use case where the average file size for your
20G partition is smaller that 16k?

						- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: No space left on device after many files creation

[cy6erGn0m]

[-- Attachment #1: Type: text/plain, Size: 2599 bytes --]

tytso@mit.edu wrote:
> On Mon, Apr 12, 2010 at 05:00:37PM -0500, Eric Sandeen wrote:
>   
>> cy6erGn0m wrote:
>>     
>>> In the past I have done the same test for ext3 and it was created ~10M
>>> files in one firectory at working filesystem (/home) and it was no
>>> issues reached.
>>>
>>> On ext4 i found inodes max size is limited as you wrote:
>>> [root@cgmachine sandbox]# df -i /e4
>>> Файловая система      Инодов   Испол   Своб  Исп % смонтирована на
>>> (file system/inodes/used/free/used%/mounted on)
>>> /dev/sda4               1,8M    1,8M       0  100% /e4
>>> [root@cgmachine sandbox]#
>>>
>>> Why inodes count limit is so small by default? Is it depends on
>>> partition size?
>>>       
>> it does depend on partition size; the inode_ratio = 16384 in mke2fs.conf
>> says 1 inode per 16k of filesystem space.
>>     
>
> 1.8 million inodes is hardly small.....  most people complain that
> we're reserving too much space for inodes.  
>
> Cy6erGn0m, What is your use case where the average file size for your
> 20G partition is smaller that 16k?
>
> 						- Ted
>   
I use filysystem as Set data structure to make fast checks from scripts.
Of course I can make long long file and test it with grep but it's will
have linear performance degradation (List), at the same time file system
has efficient tree binary structures (smth. like TreeSet). So, using
filesystem is the simplest way to do this efficient. Also I always have
timestamps for every entry. In this usecase all files are always empty.

Here is simple performance comparison: list vs tree:
cy6ergn0m@cgmachine /e4/sandbox/heap $ time cat ../x | grep ^610845$ ../x
610845
cat ../x  0,00s user 0,00s system 2% cpu 0,127 total
grep --color ^610845$ ../x  0,08s user 0,03s system 87% cpu 0,136 total
cy6ergn0m@cgmachine /e4/sandbox/heap $ time stat 610845            
  File: `610845'
  Size: 0               Blocks: 0          IO Block: 4096   пустой
обычный файл
Device: 804h/2052d      Inode: 1520242     Links: 1
Access: (0644/-rw-r--r--)  Uid: (  500/cy6ergn0m)   Gid: (  500/cy6ergn0m)
Access: 2010-04-13 10:17:58.119477521 +0400
Modify: 2010-04-13 10:17:58.119477521 +0400
Change: 2010-04-13 10:17:58.119477521 +0400
stat 610845  0,00s user 0,00s system 75% cpu 0,005 total
cy6ergn0m@cgmachine /e4/sandbox/heap $


I know that filesystems is not targeted for this usecase, but inodes
limit looks strange.. why they are can't be allocated dinamically? Can I
enlarge this limit on the fly?

 


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 3650 bytes --]

Re: No space left on device after many files creation

[Eric Sandeen]

On 04/13/2010 01:35 AM, cy6erGn0m wrote:

> I know that filesystems is not targeted for this usecase, but inodes
> limit looks strange.. why they are can't be allocated dinamically? Can I

ext2/3/4 don't allocate dynamically; there was some talk of trying to
find a way to do this but it's not really very high on the list at all.

xfs does do this, not sure about other filesystems.

> enlarge this limit on the fly?

No, but since you know your usecase, you can change it at mkfs time to
match what you need.

-Eric

Re: No space left on device after many files creation

[Andi Kleen]

cy6erGn0m <cy6ergn0m@gmail.com> writes:
> I use filysystem as Set data structure to make fast checks from scripts.
> Of course I can make long long file and test it with grep but it's will
> have linear performance degradation (List), at the same time file system
> has efficient tree binary structures (smth. like TreeSet). So, using
> filesystem is the simplest way to do this efficient. Also I always have
> timestamps for every entry. In this usecase all files are always empty.

If the files are always empty you could just create hard links: 
ln file number instead of touch.
Then you'll only need a single inode for your complete set
and also avoid a lot of seeking.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.