* irregular mkfs.xfs results on identical HW
@ 2014-09-25 20:01 Gamel Anton J.
2014-09-25 21:05 ` Eric Sandeen
2014-09-25 21:19 ` Dave Chinner
0 siblings, 2 replies; 4+ messages in thread
From: Gamel Anton J. @ 2014-09-25 20:01 UTC (permalink / raw)
To: xfs
[-- Attachment #1.1: Type: text/plain, Size: 2728 bytes --]
Dear all,
servers with identical disk setup (HW RAID0 H310a):
Disk /dev/sda: 598.9 GB, 598879502336 bytes
/dev/sda1 1 49152 394813439+ 82 Linux swap / Solaris
/dev/sda2 49153 50176 8225280 83 Linux
/dev/sda3 6399 72809 533446357+ 44 Unknown
mkfs.xfs creates on 28 of them:
meta-data=/dev/sda3 isize=256 agcount=16, agsize=8335099 blks
= sectsz=512 attr=1, projid32bit=0
data = bsize=4096 blocks=133361584, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal bsize=4096 blocks=32768, version=1
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
but on four out of them:
meta-data=/dev/sda3 isize=256 agcount=4, agsize=33340398 blks
= sectsz=512 attr=2, projid32bit=0
data = bsize=4096 blocks=133361589, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal bsize=4096 blocks=65117, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
I did not find a way to add mkfs.xfs options to get the identical
filesystem setup on all nodes, e.g.
# mkfs.xfs -d agcount=16 -b size=4096 /dev/sda3 -f
meta-data=/dev/sda3 isize=256 agcount=16, agsize=8335100 blks
= sectsz=512 attr=2, projid32bit=0
data = bsize=4096 blocks=133361589, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal log bsize=4096 blocks=65117, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
The only way it worked was to dump nodeA:/dev/sda3 to nodeB:/dev/sda3
Is there an explanation? May be I missed something ... hints?
Cheers and thanks in advance
Anton
--
Beste Gruesse
Anton J. Gamel
HPC und GRID-Computing
Physikalisches Institut
Abteilung Professor Schumacher
c/o Rechenzentrum der Universität Freiburg
Arbeitsgruppe Dr. Winterer
Hermann-Herder-Straße 10
79104 Freiburg
Tel.: ++49 (0)761 203 -4670
--
Es bleibt immer ein Rest - und ein Rest vom Rest.
[-- Attachment #1.2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 1870 bytes --]
[-- Attachment #2: Type: text/plain, Size: 121 bytes --]
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: irregular mkfs.xfs results on identical HW
2014-09-25 20:01 irregular mkfs.xfs results on identical HW Gamel Anton J.
@ 2014-09-25 21:05 ` Eric Sandeen
2014-09-25 21:19 ` Dave Chinner
1 sibling, 0 replies; 4+ messages in thread
From: Eric Sandeen @ 2014-09-25 21:05 UTC (permalink / raw)
To: Gamel Anton J., xfs
On 9/25/14 3:01 PM, Gamel Anton J. wrote:
> Dear all,
>
> servers with identical disk setup (HW RAID0 H310a):
and identical xfsprogs versions? Are these linux boxes?
> Disk /dev/sda: 598.9 GB, 598879502336 bytes
> /dev/sda1 1 49152 394813439+ 82 Linux swap / Solaris
> /dev/sda2 49153 50176 8225280 83 Linux
> /dev/sda3 6399 72809 533446357+ 44 Unknown
>
> mkfs.xfs creates on 28 of them:
> meta-data=/dev/sda3 isize=256 agcount=16, agsize=8335099 blks
agcount was higher on older mkfs's...
> = sectsz=512 attr=1, projid32bit=0
attr=1? that looks ancient. The only way to override the default (2) is with
a commandline argument, unless your mkfs is so old that it has different defaults.
check your xfsprogs versions on both hosts.
-Eric
> data = bsize=4096 blocks=133361584, imaxpct=25
> = sunit=0 swidth=0 blks
> naming =version 2 bsize=4096 ascii-ci=0
> log =internal bsize=4096 blocks=32768, version=1
> = sectsz=512 sunit=0 blks, lazy-count=1
> realtime =none extsz=4096 blocks=0, rtextents=0
>
> but on four out of them:
> meta-data=/dev/sda3 isize=256 agcount=4, agsize=33340398 blks
> = sectsz=512 attr=2, projid32bit=0
> data = bsize=4096 blocks=133361589, imaxpct=25
> = sunit=0 swidth=0 blks
> naming =version 2 bsize=4096 ascii-ci=0
> log =internal bsize=4096 blocks=65117, version=2
> = sectsz=512 sunit=0 blks, lazy-count=1
> realtime =none extsz=4096 blocks=0, rtextents=0
>
> I did not find a way to add mkfs.xfs options to get the identical
> filesystem setup on all nodes, e.g.
> # mkfs.xfs -d agcount=16 -b size=4096 /dev/sda3 -f
> meta-data=/dev/sda3 isize=256 agcount=16, agsize=8335100 blks
> = sectsz=512 attr=2, projid32bit=0
> data = bsize=4096 blocks=133361589, imaxpct=25
> = sunit=0 swidth=0 blks
> naming =version 2 bsize=4096 ascii-ci=0
> log =internal log bsize=4096 blocks=65117, version=2
> = sectsz=512 sunit=0 blks, lazy-count=1
> realtime =none extsz=4096 blocks=0, rtextents=0
>
> The only way it worked was to dump nodeA:/dev/sda3 to nodeB:/dev/sda3
> Is there an explanation? May be I missed something ... hints?
>
> Cheers and thanks in advance
>
> Anton
>
>
>
>
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: irregular mkfs.xfs results on identical HW
2014-09-25 20:01 irregular mkfs.xfs results on identical HW Gamel Anton J.
2014-09-25 21:05 ` Eric Sandeen
@ 2014-09-25 21:19 ` Dave Chinner
2014-09-29 16:47 ` Gamel Anton J.
1 sibling, 1 reply; 4+ messages in thread
From: Dave Chinner @ 2014-09-25 21:19 UTC (permalink / raw)
To: Gamel Anton J.; +Cc: xfs
On Thu, Sep 25, 2014 at 10:01:08PM +0200, Gamel Anton J. wrote:
> Dear all,
>
> servers with identical disk setup (HW RAID0 H310a):
>
> Disk /dev/sda: 598.9 GB, 598879502336 bytes
> /dev/sda1 1 49152 394813439+ 82 Linux swap / Solaris
> /dev/sda2 49153 50176 8225280 83 Linux
> /dev/sda3 6399 72809 533446357+ 44 Unknown
>
> mkfs.xfs creates on 28 of them:
> meta-data=/dev/sda3 isize=256 agcount=16, agsize=8335099 blks
> = sectsz=512 attr=1, projid32bit=0
> data = bsize=4096 blocks=133361584, imaxpct=25
> = sunit=0 swidth=0 blks
> naming =version 2 bsize=4096 ascii-ci=0
> log =internal bsize=4096 blocks=32768, version=1
> = sectsz=512 sunit=0 blks, lazy-count=1
> realtime =none extsz=4096 blocks=0, rtextents=0
THat's clearly an old version of mkfs - it's selected version 1 logs
and attr1 by default and a log size of only 128MB. mkfs.xfs has
defaulted to v2 logs since 3.0.0 (2007).
> but on four out of them:
> meta-data=/dev/sda3 isize=256 agcount=4, agsize=33340398 blks
> = sectsz=512 attr=2, projid32bit=0
> data = bsize=4096 blocks=133361589, imaxpct=25
> = sunit=0 swidth=0 blks
> naming =version 2 bsize=4096 ascii-ci=0
> log =internal bsize=4096 blocks=65117, version=2
> = sectsz=512 sunit=0 blks, lazy-count=1
> realtime =none extsz=4096 blocks=0, rtextents=0
Clearly much newer - attr2, log v2, log larger than 128MB...
mkfs.xfs -V on each of the nodes will tell you that they are running
different versions of mkfs, I think.
> The only way it worked was to dump nodeA:/dev/sda3 to nodeB:/dev/sda3
> Is there an explanation? May be I missed something ... hints?
If you are building a new storage system, then I'd highly recommend
all the nodes run the same software and that software is the newest
possible release you can get....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: irregular mkfs.xfs results on identical HW
2014-09-25 21:19 ` Dave Chinner
@ 2014-09-29 16:47 ` Gamel Anton J.
0 siblings, 0 replies; 4+ messages in thread
From: Gamel Anton J. @ 2014-09-29 16:47 UTC (permalink / raw)
To: sandeen, Dave Chinner; +Cc: xfs
[-- Attachment #1.1: Type: text/plain, Size: 3414 bytes --]
Re Dave, Eric and all,
thanks a lot for the quick answers. This is definitely a version
issue I did not take into account at first. We ran into it when
slurm found small differences in scratch space. All (diskless)
SL6.5 computenodes have xfsprogs-3.1.1-10.el6.x86_64 installed,
but creation of xfs was done with older versions on SL5.
Recreation of xfs after disk problems changed the xfs outcome
as seen.
From the beginning the plan was to recreate xfs at every boot
time. For some reason this did not work for (all) the diskless
machines and we dropped that in favour of a find+rm.
Now I reconsider a format at boot time again.
Cheers
Anton
On 09/25/2014 11:19 PM, Dave Chinner wrote:
> On Thu, Sep 25, 2014 at 10:01:08PM +0200, Gamel Anton J. wrote:
>> Dear all,
>>
>> servers with identical disk setup (HW RAID0 H310a):
>>
>> Disk /dev/sda: 598.9 GB, 598879502336 bytes
>> /dev/sda1 1 49152 394813439+ 82 Linux swap / Solaris
>> /dev/sda2 49153 50176 8225280 83 Linux
>> /dev/sda3 6399 72809 533446357+ 44 Unknown
>>
>> mkfs.xfs creates on 28 of them:
>> meta-data=/dev/sda3 isize=256 agcount=16, agsize=8335099 blks
>> = sectsz=512 attr=1, projid32bit=0
>> data = bsize=4096 blocks=133361584, imaxpct=25
>> = sunit=0 swidth=0 blks
>> naming =version 2 bsize=4096 ascii-ci=0
>> log =internal bsize=4096 blocks=32768, version=1
>> = sectsz=512 sunit=0 blks, lazy-count=1
>> realtime =none extsz=4096 blocks=0, rtextents=0
>
> THat's clearly an old version of mkfs - it's selected version 1 logs
> and attr1 by default and a log size of only 128MB. mkfs.xfs has
> defaulted to v2 logs since 3.0.0 (2007).
>
>> but on four out of them:
>> meta-data=/dev/sda3 isize=256 agcount=4, agsize=33340398 blks
>> = sectsz=512 attr=2, projid32bit=0
>> data = bsize=4096 blocks=133361589, imaxpct=25
>> = sunit=0 swidth=0 blks
>> naming =version 2 bsize=4096 ascii-ci=0
>> log =internal bsize=4096 blocks=65117, version=2
>> = sectsz=512 sunit=0 blks, lazy-count=1
>> realtime =none extsz=4096 blocks=0, rtextents=0
>
> Clearly much newer - attr2, log v2, log larger than 128MB...
>
> mkfs.xfs -V on each of the nodes will tell you that they are running
> different versions of mkfs, I think.
>
>> The only way it worked was to dump nodeA:/dev/sda3 to nodeB:/dev/sda3
>> Is there an explanation? May be I missed something ... hints?
>
> If you are building a new storage system, then I'd highly recommend
> all the nodes run the same software and that software is the newest
> possible release you can get....
>
> Cheers,
>
> Dave.
>
--
Beste Gruesse
Anton J. Gamel
HPC und GRID-Computing
Physikalisches Institut
Abteilung Professor Schumacher
c/o Rechenzentrum der Universität Freiburg
Arbeitsgruppe Dr. Winterer
Hermann-Herder-Straße 10
79104 Freiburg
Tel.: ++49 (0)761 203 -4670
--
Es bleibt immer ein Rest - und ein Rest vom Rest.
[-- Attachment #1.2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 1870 bytes --]
[-- Attachment #2: Type: text/plain, Size: 121 bytes --]
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2014-09-29 16:47 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-09-25 20:01 irregular mkfs.xfs results on identical HW Gamel Anton J.
2014-09-25 21:05 ` Eric Sandeen
2014-09-25 21:19 ` Dave Chinner
2014-09-29 16:47 ` Gamel Anton J.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox