From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marcus Kool Subject: Re: mdadm raid5 array - 0 space available but usage is less than capacity Date: Thu, 23 Sep 2010 17:18:23 -0300 Message-ID: <4C9BB60F.5050000@familiakool.net> References: Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: Sender: linux-raid-owner@vger.kernel.org To: Robin Doherty Cc: linux-raid@vger.kernel.org List-Id: linux-raid.ids Robin, this is normal file system behaviour: File systems reserve 5-10% for reasons of efficiency. If 95% of the capacity is used, df will report 'file system full'. and *only* root can write new files in the remaining 5%, regular users cannot. You need to clean up or insert more disks :-) Marcus Robin Doherty wrote: > I have a RAID5 array of 5 1TB disks that has worked fine for 2 years > but now says that it has 0 space available (even though it does have > space available). It will allow me to read from it but not write. I > can delete things, and the usage goes down but the space stays at 0. > > I can touch but not mkdir: > > rob@cholera ~ $ mkdir /share/test > mkdir: cannot create directory `/share/test': No space left on device > rob@cholera ~ $ touch /share/test > rob@cholera ~ $ rm /share/test > rob@cholera ~ $ > > Output from df -h (/dev/md2 is the problem array): > > Filesystem Size Used Avail Use% Mounted on > /dev/md1 23G 15G 6.1G 72% / > varrun 1008M 328K 1007M 1% /var/run > varlock 1008M 0 1008M 0% /var/lock > udev 1008M 140K 1008M 1% /dev > devshm 1008M 0 1008M 0% /dev/shm > /dev/md0 183M 43M 131M 25% /boot > /dev/md2 3.6T 3.5T 0 100% /share > > and without the -h: > > Filesystem 1K-blocks Used Available Use% Mounted on > /dev/md1 23261796 15696564 6392900 72% / > varrun 1031412 328 1031084 1% /var/run > varlock 1031412 0 1031412 0% /var/lock > udev 1031412 140 1031272 1% /dev > devshm 1031412 0 1031412 0% /dev/shm > /dev/md0 186555 43532 133391 25% /boot > /dev/md2 3843709832 3705379188 0 100% /share > > Everything looks fine with the mdadm array as far as I can tell from > the following: > > rob@cholera /share $ cat /proc/mdstat > Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] > [raid4] [raid10] > md2 : active raid5 sda4[0] sde4[4] sdd4[3] sdc4[2] sdb4[1] > 3874235136 blocks level 5, 64k chunk, algorithm 2 [5/5] [UUUUU] > > md1 : active raid5 sda3[0] sde3[4] sdd3[3] sdc3[2] sdb3[1] > 31262208 blocks level 5, 64k chunk, algorithm 2 [5/5] [UUUUU] > > md0 : active raid1 sda1[0] sde1[4](S) sdd1[3] sdc1[2] sdb1[1] > 192640 blocks [4/4] [UUUU] > > unused devices: > > > rob@cholera /share $ sudo mdadm -D /dev/md2 > /dev/md2: > Version : 00.90.03 > Creation Time : Sat May 3 13:45:54 2008 > Raid Level : raid5 > Array Size : 3874235136 (3694.76 GiB 3967.22 GB) > Used Dev Size : 968558784 (923.69 GiB 991.80 GB) > Raid Devices : 5 > Total Devices : 5 > Preferred Minor : 2 > Persistence : Superblock is persistent > > Update Time : Wed Sep 22 23:16:06 2010 > State : clean > Active Devices : 5 > Working Devices : 5 > Failed Devices : 0 > Spare Devices : 0 > > Layout : left-symmetric > Chunk Size : 64K > > UUID : 4387b8c0:21551766:ed750333:824b67f8 > Events : 0.651050 > > Number Major Minor RaidDevice State > 0 8 4 0 active sync /dev/sda4 > 1 8 20 1 active sync /dev/sdb4 > 2 8 36 2 active sync /dev/sdc4 > 3 8 52 3 active sync /dev/sdd4 > 4 8 68 4 active sync /dev/sde4 > > > rob@cholera /share $ cat /etc/mdadm/mdadm.conf > # mdadm.conf > # > # Please refer to mdadm.conf(5) for information about this file. > # > > # by default, scan all partitions (/proc/partitions) for MD superblocks. > # alternatively, specify devices to scan, using wildcards if desired. > DEVICE partitions > > # auto-create devices with Debian standard permissions > CREATE owner=root group=disk mode=0660 auto=yes > > # automatically tag new arrays as belonging to the local system > HOMEHOST > > # instruct the monitoring daemon where to send mail alerts > MAILADDR root > > # definitions of existing MD arrays > ARRAY /dev/md0 level=raid1 num-devices=4 > UUID=a761c788:81771ba6:c983b0fe:7dba32e6 > ARRAY /dev/md1 level=raid5 num-devices=5 > UUID=291649db:9f874a3c:def17491:656cf263 > ARRAY /dev/md2 level=raid5 num-devices=5 > UUID=4387b8c0:21551766:ed750333:824b67f8 > > # This file was auto-generated on Sun, 04 May 2008 14:57:35 +0000 > # by mkconf $Id$ > > So maybe this is a file system problem rather than an mdadm problem? > Either way I've already bashed my head against a brick wall for a few > weeks and I don't know where to go from here so any advice would be > appreciated. > > Thanks > Rob > -- > To unsubscribe from this list: send the line "unsubscribe linux-raid" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > >