Problem with file system on iSCSI FileIO

Problem with file system on iSCSI FileIO

[Slawomir Nowakowski]

Dear Sir,

We are using iSCSI FileIO on the LVM. The file (called lun) has been 
initialized with:
dd if=/dev/zero of=/dev/vg+vg00/lv+i+lv0000 bs=1M conv=notrunc

on the XFS formatted logical volume.

Additionally, we use 2.6.27-39 kernel  and xfs-tools 2.10.1

After some time, without any additional errors, we are not able mount 
the volume. After mount command we get:


"mount: Structure needs cleaning"

in dmesg we can see:

"XFS: failed to read root inode"

If we run the xfs_repair, the lun file will be removed and all data on 
the iSCSI FileIO volume is lost.

I have found in the FAQ that this can be everything:

http://xfs.org/index.php/XFS_FAQ#Q:_I_see_applications_returning_error_990_or_.22Structure_needs_cleaning.22.2C_what_is_wrong.3F

But do you know if it is possible to recover the data somehow?

Thanks in advance for response

-- 

If you have any more questions, please do not hesitate to contact me.

Best Regards

Slawomir Nowakowski
Technical Support Engineer
-------------------------------------------------------
Open-E GmbH
e-mail: slawomir.nowakowski@open-e.com
http://www.open-e.com/
-------------------------------------------------------

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Christoph Hellwig]

On Thu, Sep 23, 2010 at 03:35:02PM +0200, Slawomir Nowakowski wrote:
> Dear Sir,
> 
> We are using iSCSI FileIO on the LVM. The file (called lun) has been

What is iSCSI FileIO?

> initialized with:
> dd if=/dev/zero of=/dev/vg+vg00/lv+i+lv0000 bs=1M conv=notrunc
> 
> on the XFS formatted logical volume.
> 
> Additionally, we use 2.6.27-39 kernel  and xfs-tools 2.10.1
> 
> After some time, without any additional errors, we are not able
> mount the volume. After mount command we get:

Please describe the full storage topology.  But most likely it's an
issue with your underlying storage.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Slawomir Nowakowski]

If you have any more questions, please do not hesitate to contact me.

Best Regards

Slawomir Nowakowski
Technical Support Engineer
-------------------------------------------------------
Open-E GmbH
e-mail: slawomir.nowakowski@open-e.com
http://www.open-e.com/
-------------------------------------------------------


W dniu 23.09.2010 16:32, Christoph Hellwig pisze:
> On Thu, Sep 23, 2010 at 03:35:02PM +0200, Slawomir Nowakowski wrote:
>    
>> Dear Sir,
>>
>> We are using iSCSI FileIO on the LVM. The file (called lun) has been
>>      
> What is iSCSI FileIO?
>
>    
iSCSI FileIO is using struct file that serves it as block level device 
via iSCSI technology:

http://en.wikipedia.org/wiki/ISCSI

>> initialized with:
>> dd if=/dev/zero of=/dev/vg+vg00/lv+i+lv0000 bs=1M conv=notrunc
>>
>> on the XFS formatted logical volume.
>>
>> Additionally, we use 2.6.27-39 kernel  and xfs-tools 2.10.1
>>
>> After some time, without any additional errors, we are not able
>> mount the volume. After mount command we get:
>>      
> Please describe the full storage topology.  But most likely it's an
> issue with your underlying storage.
>
>    
There are RAID level 6 unit on the Areca RAID controller (1680). On the 
unit is created a volume group. In the volume group we have several 
logical volumes. The iSCSI FileIO volume is mounted and the file (lun) 
is served via SCST target as iSCSI LUN.

We have checked RAM with memtest+ and verified RAID unit health and no 
issues were found.

Cheers

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Christoph Hellwig]

On Thu, Sep 23, 2010 at 04:58:47PM +0200, Slawomir Nowakowski wrote:
> iSCSI FileIO is using struct file that serves it as block level
> device via iSCSI technology:
> 
> http://en.wikipedia.org/wiki/ISCSI

Thanks, I know ISCSI very well.  But what is "iSCSI FileIO" ?  The above
sounds like it's an iscsi target, is that correct?

> There are RAID level 6 unit on the Areca RAID controller (1680). On
> the unit is created a volume group. In the volume group we have
> several logical volumes. The iSCSI FileIO volume is mounted and the
> file (lun) is served via SCST target as iSCSI LUN.
> 
> We have checked RAM with memtest+ and verified RAID unit health and
> no issues were found.

I still can't make any sense of the actual setups.

The above seems to be the backend storage.  Then there's SCST somewhere
in which is in a out of tree kernel module.  And then you use XFS
somewhere.  Please provide a full description of the setup.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Slawomir Nowakowski]

>> iSCSI FileIO is using struct file that serves it as block level
>> device via iSCSI technology:
>>
>> http://en.wikipedia.org/wiki/ISCSI
>>      
> Thanks, I know ISCSI very well.  But what is "iSCSI FileIO" ?  The above
> sounds like it's an iscsi target, is that correct?
>
>    
Yes, it is iSCSI target that is served for another server. The FileIO 
serves a file as iSCSI target.
>> There are RAID level 6 unit on the Areca RAID controller (1680). On
>> the unit is created a volume group. In the volume group we have
>> several logical volumes. The iSCSI FileIO volume is mounted and the
>> file (lun) is served via SCST target as iSCSI LUN.
>>
>> We have checked RAM with memtest+ and verified RAID unit health and
>> no issues were found.
>>      
> I still can't make any sense of the actual setups.
>
> The above seems to be the backend storage.  Then there's SCST somewhere
> in which is in a out of tree kernel module.  And then you use XFS
> somewhere.  Please provide a full description of the setup.
>
>    
So once again. We have created a RAID unit level 6. On the top of the 
unit there is an LVM architecture, I mean a volume group that contains 
logical volumes. The logical volume is formatted with XFS and it 
contains one big file that takes almost all of the space on the LV. 
There is some free space left in order to be able expand the LV and FS 
in the future. The LV is mounted and the file is served as iSCSI target. 
The iSCSI Initiator (MS Initiator from Windows 2k3) connects to iSCSI 
target. The iSCSI disk is formatted with the NTFS.


But we believe the problem is with the XFS. With unknown reason we are 
not able to mount the LV and after running xfs_repair the file is 
missing from the LV. Do you have any ideas how we can try to fix the 
broken XFS?

Cheers
Slawek

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Emmanuel Florac]

Le Fri, 24 Sep 2010 03:55:05 -0400
Christoph Hellwig <hch@infradead.org> écrivait:

> Thanks, I know ISCSI very well.  But what is "iSCSI FileIO" ?  The
> above sounds like it's an iscsi target, is that correct?

It's an IO Mode that goes through the kernel VFS cache, as opposed to
blockIO that does direct IO.

> I still can't make any sense of the actual setups.
> 
> The above seems to be the backend storage.  Then there's SCST
> somewhere in which is in a out of tree kernel module.  And then you
> use XFS somewhere.  Please provide a full description of the setup.

If the iSCSI targets are actually block devices (lvm lvs, disk
partitions, etc), than using FileIO is a mistake, it may bring up all
kind of weird behaviours, though normally no real errors - though I
don't really know how scst fares in this regard.

My understanding : he planned to create a file on the mounted XFS
volume with dd but instead he dd'ed the lv itself, which obviously
destroyed the filesystem. Or something else, I don't really know :)

Slawomir, please show us the scst config file. Did you use mkfs
and dd on the target or the initiator? This isn't clear.

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac@intellique.com>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Emmanuel Florac]

Le Fri, 24 Sep 2010 13:11:25 +0200
Slawomir Nowakowski <slawomir.nowakowski@open-e.com> écrivait:

> But we believe the problem is with the XFS. With unknown reason we
> are not able to mount the LV and after running xfs_repair the file is 
> missing from the LV. Do you have any ideas how we can try to fix the 
> broken XFS?

This doesn't really make much sense to me. What target are you using?
scst, tgt, lio or iet? What looks weird to me is that the dd command
example you gave writes over the xfs filesystem :

dd if=/dev/zero of=/dev/vg+vg00/lv+i+lv0000 bs=1M conv=notrunc

This is definitely incorrect. Given that /dev/vg+vg00/lv+i+lv0000 is
your xfs formatted lv, you must mount it somewhere :

mount /dev/vg+vg00/lv+i+lv0000 /mnt/whatever

Then you dd the file on /mnt/whatever :

dd if=/dev/zero of=/mnt/whatever/lunfile bs=1M

Lastly, you declare /mnt/whatever/lunfile as the lun in your target
(method varies depending upon the target used), something like (ietd
example):

Target iqn.2010-09:com.whatever.host.target1
	Alias host.target1
	Lun 0 Type=fileio,Path=mnt/whatever/lunfile,IOMode=wback

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac@intellique.com>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Slawomir Nowakowski]

Dear Emmanuel,


> Le Fri, 24 Sep 2010 13:11:25 +0200
> Slawomir Nowakowski<slawomir.nowakowski@open-e.com>  écrivait:
>
>    
>> But we believe the problem is with the XFS. With unknown reason we
>> are not able to mount the LV and after running xfs_repair the file is
>> missing from the LV. Do you have any ideas how we can try to fix the
>> broken XFS?
>>      
> This doesn't really make much sense to me. What target are you using?
> scst, tgt, lio or iet? What looks weird to me is that the dd command
> example you gave writes over the xfs filesystem :
>
> dd if=/dev/zero of=/dev/vg+vg00/lv+i+lv0000 bs=1M conv=notrunc
>
> This is definitely incorrect. Given that /dev/vg+vg00/lv+i+lv0000 is
> your xfs formatted lv, you must mount it somewhere :
>
> mount /dev/vg+vg00/lv+i+lv0000 /mnt/whatever
>
> Then you dd the file on /mnt/whatever :
>
> dd if=/dev/zero of=/mnt/whatever/lunfile bs=1M
>
> Lastly, you declare /mnt/whatever/lunfile as the lun in your target
> (method varies depending upon the target used), something like (ietd
> example):
>
> Target iqn.2010-09:com.whatever.host.target1
> 	Alias host.target1
> 	Lun 0 Type=fileio,Path=mnt/whatever/lunfile,IOMode=wback
>
>    
It's my mistake in dd command, sorry for that.

First we mount the LV:

mount
[...]
/dev/vg+vg00/lv+i+lv0000 on /mnt/point type xfs 
(rw,nouuid,attr2,nobarrier,noquota)
[...]

then we run dd to file

dd if=/dev/zero of=/mnt/point/lun bs=1M conv=notrunc count=$size

$size is counted to leave some free space on the device

As the iSCSI target we use SCST 1.0.1.2. The scst.conf looks likes like 
this:

[HANDLER vdisk]
DEVICE 0QSP199WJI1yKOPj,/mnt/point/lun,WT,512,0QSP199WJI1yKOPj

[GROUP Default_iqn.2010-03:sn1.target0]
[GROUP Default]

[ASSIGNMENT Default_iqn.2010-03:sn1.target0]
DEVICE 0QSP199WJI1yKOPj,0
[ASSIGNMENT Default]

[TARGETS enable]

[TARGETS disable]

The problem is that we were able to use this LUN in the target, but 
suddenly after a reboot we are not.

Cheers
Slawek

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Slawomir Nowakowski]

Dear Sirs,
> Le Fri, 24 Sep 2010 03:55:05 -0400
> Christoph Hellwig<hch@infradead.org>  écrivait:
>
>    
>> Thanks, I know ISCSI very well.  But what is "iSCSI FileIO" ?  The
>> above sounds like it's an iscsi target, is that correct?
>>      
> It's an IO Mode that goes through the kernel VFS cache, as opposed to
> blockIO that does direct IO.
>
>    
>> I still can't make any sense of the actual setups.
>>
>> The above seems to be the backend storage.  Then there's SCST
>> somewhere in which is in a out of tree kernel module.  And then you
>> use XFS somewhere.  Please provide a full description of the setup.
>>      
> If the iSCSI targets are actually block devices (lvm lvs, disk
> partitions, etc), than using FileIO is a mistake, it may bring up all
> kind of weird behaviours, though normally no real errors - though I
> don't really know how scst fares in this regard.
>
> My understanding : he planned to create a file on the mounted XFS
> volume with dd but instead he dd'ed the lv itself, which obviously
> destroyed the filesystem. Or something else, I don't really know :)
>
> Slawomir, please show us the scst config file. Did you use mkfs
> and dd on the target or the initiator? This isn't clear.
>
>    
It's my mistake in dd command, sorry for that.

First we mount the LV:

mount
[...]
/dev/vg+vg00/lv+i+lv0000 on /mnt/point type xfs 
(rw,nouuid,attr2,nobarrier,noquota)
[...]

then we run dd to file

dd if=/dev/zero of=/mnt/point/lun bs=1M conv=notrunc count=$size

$size is counted to leave some free space on the device

As the iSCSI target we use SCST 1.0.1.2. The scst.conf looks likes like 
this:

[HANDLER vdisk]
DEVICE 0QSP199WJI1yKOPj,/mnt/point/lun,WT,512,0QSP199WJI1yKOPj

[GROUP Default_iqn.2010-03:sn1.target0]
[GROUP Default]

[ASSIGNMENT Default_iqn.2010-03:sn1.target0]
DEVICE 0QSP199WJI1yKOPj,0
[ASSIGNMENT Default]

[TARGETS enable]

[TARGETS disable]

The problem is that we were able to use this LUN in the target, but 
suddenly after a reboot we are not.

Cheers
Slawek

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Emmanuel Florac]

Le Fri, 24 Sep 2010 15:49:10 +0200
Slawomir Nowakowski <slawomir.nowakowski@open-e.com> écrivait:

> The problem is that we were able to use this LUN in the target, but 
> suddenly after a reboot we are not.
> 

That's strange indeed. I don't see why the target IO would cause this,
but I don't know much scst. Did you try this several times? Did you try
creating the file, unmounting and remounting without starting the scsi
target, to see it it's something scst does that kills the filesystem?

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac@intellique.com>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Richard Sharpe]

On Fri, Sep 24, 2010 at 12:55 AM, Christoph Hellwig <hch@infradead.org> wrote:
> On Thu, Sep 23, 2010 at 04:58:47PM +0200, Slawomir Nowakowski wrote:
>> iSCSI FileIO is using struct file that serves it as block level
>> device via iSCSI technology:
>>
>> http://en.wikipedia.org/wiki/ISCSI
>
> Thanks, I know ISCSI very well.  But what is "iSCSI FileIO" ?  The above
> sounds like it's an iscsi target, is that correct?
>
>> There are RAID level 6 unit on the Areca RAID controller (1680). On
>> the unit is created a volume group. In the volume group we have
>> several logical volumes. The iSCSI FileIO volume is mounted and the
>> file (lun) is served via SCST target as iSCSI LUN.
>>
>> We have checked RAM with memtest+ and verified RAID unit health and
>> no issues were found.
>
> I still can't make any sense of the actual setups.
>
> The above seems to be the backend storage.  Then there's SCST somewhere
> in which is in a out of tree kernel module.  And then you use XFS
> somewhere.  Please provide a full description of the setup.

Sounds like iSCSI-SCST to a vdisk_fileio on a file under xfs on a LUN
on the Areca RAID controller ...

Does sound awfully complicated. Why not just serve the LUNs from the
Areca directly? SCST allows you to do that. xfs is awesomely fast, but
it is just adding overhead here.


-- 
Regards,
Richard Sharpe

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Emmanuel Florac]

Le Fri, 24 Sep 2010 08:10:51 -0700 vous écriviez:

> Sounds like iSCSI-SCST to a vdisk_fileio on a file under xfs on a LUN
> on the Areca RAID controller ...

I use iet and it works fine. It has the beneit of allowing easy copy,
backup, etc of the virtual lun; and it allows thin provisioning too.

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac@intellique.com>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Ryszard Stawiarski]

Hi Emmanuel!

I C that there is no problem with IETD ;-)
But we need to stay focused on SCST in this case :-)

On 24.09.2010 20:18, Emmanuel Florac wrote:
> Le Fri, 24 Sep 2010 08:10:51 -0700 vous écriviez:
>
>    
>> Sounds like iSCSI-SCST to a vdisk_fileio on a file under xfs on a LUN
>> on the Areca RAID controller ...
>>      
> I use iet and it works fine. It has the beneit of allowing easy copy,
> backup, etc of the virtual lun; and it allows thin provisioning too.
>
>    

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Emmanuel Florac]

Le Fri, 24 Sep 2010 22:46:51 +0200 vous écriviez:

> I C that there is no problem with IETD ;-)
> But we need to stay focused on SCST in this case :-)
> 

Yes but you'd need to get sure that it's not scst that breaks the
filesystem. My guess : it actually is. I personnally didn't find SCST
of acceptable production quality.

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac@intellique.com>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Richard Sharpe]

On Sat, Sep 25, 2010 at 7:13 AM, Emmanuel Florac <eflorac@intellique.com> wrote:
> Le Fri, 24 Sep 2010 22:46:51 +0200 vous écriviez:
>
>> I C that there is no problem with IETD ;-)
>> But we need to stay focused on SCST in this case :-)
>>
>
> Yes but you'd need to get sure that it's not scst that breaks the
> filesystem. My guess : it actually is. *I personnally didn't find SCST
> of acceptable production quality.*

Great call there ... I see that Data Domain, Fusion IO and others have
ignored your sage advice and suffered the financial consequences.

-- 
Regards,
Richard Sharpe

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Christoph Hellwig]

> So once again. We have created a RAID unit level 6. On the top of
> the unit there is an LVM architecture, I mean a volume group that
> contains logical volumes. The logical volume is formatted with XFS
> and it contains one big file that takes almost all of the space on
> the LV. There is some free space left in order to be able expand the
> LV and FS in the future. The LV is mounted and the file is served as
> iSCSI target. The iSCSI Initiator (MS Initiator from Windows 2k3)
> connects to iSCSI target. The iSCSI disk is formatted with the NTFS.

ok, so we have:

Linux Server

+----------------------+
|   hardware raid 6    |
+----------------------+
| lvm2 - linear volume |
+----------------------+
|          XFS         |
+----------------------+
|    iSCSI target      |
+----------------------+

Windows client:


+----------------------+
|    iSCSI initiator   |
+----------------------+
|        NTFS          |
+----------------------+

> But we believe the problem is with the XFS. With unknown reason we
> are not able to mount the LV and after running xfs_repair the file
> is missing from the LV. Do you have any ideas how we can try to fix
> the broken XFS?

This does not sound like a plain XFS issue to me, but an interaction
between components going completely wrong.  Normal I/O to a file
should never corrupt the filesystem around it to the point where
it's unusable, and so far I never heard reports about that.  The hint
that this doesn't happen with another purely userspace target is
interesting.  I wonder if SCST that you use does any sort of in-kernel
block I/O after using bmap or similar?  I've not seen that for iscsi
targets yet but for other kernel modules, and that kind of I/O
can cause massive corruption on a filesystem with delayed allocation
and unwritten extents.

Can any of the SCST experts on the list here track down how I/O for this
configuration will be issued?

What does happen if you try the same setup with say jfs or ext4 instead
of xfs?

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Richard Sharpe]

On Sat, Sep 25, 2010 at 8:56 AM, Christoph Hellwig <hch@infradead.org> wrote:
>> So once again. We have created a RAID unit level 6. On the top of
>> the unit there is an LVM architecture, I mean a volume group that
>> contains logical volumes. The logical volume is formatted with XFS
>> and it contains one big file that takes almost all of the space on
>> the LV. There is some free space left in order to be able expand the
>> LV and FS in the future. The LV is mounted and the file is served as
>> iSCSI target. The iSCSI Initiator (MS Initiator from Windows 2k3)
>> connects to iSCSI target. The iSCSI disk is formatted with the NTFS.
>
> ok, so we have:
>
> Linux Server
>
> +----------------------+
> |   hardware raid 6    |
> +----------------------+
> | lvm2 - linear volume |
> +----------------------+
> |          XFS         |
> +----------------------+
> |    iSCSI target      |
> +----------------------+
>
> Windows client:
>
>
> +----------------------+
> |    iSCSI initiator   |
> +----------------------+
> |        NTFS          |
> +----------------------+
>
>> But we believe the problem is with the XFS. With unknown reason we
>> are not able to mount the LV and after running xfs_repair the file
>> is missing from the LV. Do you have any ideas how we can try to fix
>> the broken XFS?
>
> This does not sound like a plain XFS issue to me, but an interaction
> between components going completely wrong.  Normal I/O to a file
> should never corrupt the filesystem around it to the point where
> it's unusable, and so far I never heard reports about that.  The hint
> that this doesn't happen with another purely userspace target is
> interesting.  I wonder if SCST that you use does any sort of in-kernel
> block I/O after using bmap or similar?  I've not seen that for iscsi
> targets yet but for other kernel modules, and that kind of I/O
> can cause massive corruption on a filesystem with delayed allocation
> and unwritten extents.
>
> Can any of the SCST experts on the list here track down how I/O for this
> configuration will be issued?
>
> What does happen if you try the same setup with say jfs or ext4 instead
> of xfs?

I saw references to vdisk fileio in there and wondered why this was
being done rather than simply exporting the hardware raid 6 device?
Ie, why are all those other layers in there?

fileio uses submit_bio to submit the data and it defaults to
WRITE_THROUGH, NV_CACHE and DIRECT_IO (at least in the trunk, but I
suspect this has been the case for a long while) however, the person
making the complaint might have switched off WRITE_THROUGH in the
pursuit of performance, in which case a crash could corrupt things
badly but it would depend on whether or not clearing WRITE_THROUGH
also clears NV_CACHE and what the code assembling the caching mode
page does (and I have only had a cursory glance at the vdisk code).

What is needed here is the parameters used in configuring the vdisk
and the version of SCST in use.

-- 
Regards,
Richard Sharpe

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Christoph Hellwig]

On Sat, Sep 25, 2010 at 09:54:46AM -0700, Richard Sharpe wrote:
> fileio uses submit_bio to submit the data and it defaults to
> WRITE_THROUGH, NV_CACHE and DIRECT_IO (at least in the trunk, but I
> suspect this has been the case for a long while) however, the person
> making the complaint might have switched off WRITE_THROUGH in the
> pursuit of performance, in which case a crash could corrupt things
> badly but it would depend on whether or not clearing WRITE_THROUGH
> also clears NV_CACHE and what the code assembling the caching mode
> page does (and I have only had a cursory glance at the vdisk code).

If the target uses submit_bio for logical files inside a filesystems
there are hundreds of ways to get exactly the corruption that Slawomir
sees.  How does it obtain the logical to physical mapping?  What locking
does it use agains other access to the file?

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Richard Sharpe]

On Sat, Sep 25, 2010 at 10:01 AM, Christoph Hellwig <hch@infradead.org> wrote:
> On Sat, Sep 25, 2010 at 09:54:46AM -0700, Richard Sharpe wrote:
>> fileio uses submit_bio to submit the data and it defaults to
>> WRITE_THROUGH, NV_CACHE and DIRECT_IO (at least in the trunk, but I
>> suspect this has been the case for a long while) however, the person
>> making the complaint might have switched off WRITE_THROUGH in the
>> pursuit of performance, in which case a crash could corrupt things
>> badly but it would depend on whether or not clearing WRITE_THROUGH
>> also clears NV_CACHE and what the code assembling the caching mode
>> page does (and I have only had a cursory glance at the vdisk code).
>
> If the target uses submit_bio for logical files inside a filesystems
> there are hundreds of ways to get exactly the corruption that Slawomir
> sees.  How does it obtain the logical to physical mapping?  What locking
> does it use agains other access to the file?

Actually, I was wrong. for fileio it does vfs_writev. I started
wondering the same questions and went back through the code and
noticed that I had missed a test.

We really need to know what parameters the person making the complaint
is using, and the version of SCST.

In addition, I don't understand when this corruption occurred ...

It really might be a case of don't do that.

-- 
Regards,
Richard Sharpe

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Vladislav Bolkhovitin]

Richard Sharpe, on 09/25/2010 06:18 PM wrote:
> On Sat, Sep 25, 2010 at 7:13 AM, Emmanuel Florac <eflorac@intellique.com> wrote:
>> Le Fri, 24 Sep 2010 22:46:51 +0200 vous écriviez:
>>
>>> I C that there is no problem with IETD ;-)
>>> But we need to stay focused on SCST in this case :-)
>>>
>>
>> Yes but you'd need to get sure that it's not scst that breaks the
>> filesystem. My guess : it actually is. *I personnally didn't find SCST
>> of acceptable production quality.*

When did you try the last time?

I guess, all you need is the latest 2.0.0.x SCST SVN branch? Or wait a
bit for official 2.0 release.

Vlad

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Problem with file system on iSCSI FileIO

[Emmanuel Florac]

Le Sat, 25 Sep 2010 23:01:37 +0400 vous écriviez:

> When did you try the last time?

Not very recently, I confess :) Furthermore, as there was another
discussion on the ML about someone using tgt, I mixed it up.
 
> I guess, all you need is the latest 2.0.0.x SCST SVN branch? Or wait a
> bit for official 2.0 release.

Yes, I want to give it a try.

-- 
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<eflorac@intellique.com>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs