* Problem with file system on iSCSI FileIO @ 2010-09-23 13:35 Slawomir Nowakowski 2010-09-23 14:32 ` Christoph Hellwig 0 siblings, 1 reply; 21+ messages in thread From: Slawomir Nowakowski @ 2010-09-23 13:35 UTC (permalink / raw) To: xfs Cc: =?UTF-8?B?xYF1aw==?=, 'Ryszard Stawiarski', Artur Piechocki, asz Wittig Dear Sir, We are using iSCSI FileIO on the LVM. The file (called lun) has been initialized with: dd if=/dev/zero of=/dev/vg+vg00/lv+i+lv0000 bs=1M conv=notrunc on the XFS formatted logical volume. Additionally, we use 2.6.27-39 kernel and xfs-tools 2.10.1 After some time, without any additional errors, we are not able mount the volume. After mount command we get: "mount: Structure needs cleaning" in dmesg we can see: "XFS: failed to read root inode" If we run the xfs_repair, the lun file will be removed and all data on the iSCSI FileIO volume is lost. I have found in the FAQ that this can be everything: http://xfs.org/index.php/XFS_FAQ#Q:_I_see_applications_returning_error_990_or_.22Structure_needs_cleaning.22.2C_what_is_wrong.3F But do you know if it is possible to recover the data somehow? Thanks in advance for response -- If you have any more questions, please do not hesitate to contact me. Best Regards Slawomir Nowakowski Technical Support Engineer ------------------------------------------------------- Open-E GmbH e-mail: slawomir.nowakowski@open-e.com http://www.open-e.com/ ------------------------------------------------------- _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-23 13:35 Problem with file system on iSCSI FileIO Slawomir Nowakowski @ 2010-09-23 14:32 ` Christoph Hellwig 2010-09-23 14:58 ` Slawomir Nowakowski 0 siblings, 1 reply; 21+ messages in thread From: Christoph Hellwig @ 2010-09-23 14:32 UTC (permalink / raw) To: Slawomir Nowakowski Cc: Artur Piechocki, =?UTF-8?B?xYF1aw==?=, asz Wittig, 'Ryszard Stawiarski', xfs On Thu, Sep 23, 2010 at 03:35:02PM +0200, Slawomir Nowakowski wrote: > Dear Sir, > > We are using iSCSI FileIO on the LVM. The file (called lun) has been What is iSCSI FileIO? > initialized with: > dd if=/dev/zero of=/dev/vg+vg00/lv+i+lv0000 bs=1M conv=notrunc > > on the XFS formatted logical volume. > > Additionally, we use 2.6.27-39 kernel and xfs-tools 2.10.1 > > After some time, without any additional errors, we are not able > mount the volume. After mount command we get: Please describe the full storage topology. But most likely it's an issue with your underlying storage. _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-23 14:32 ` Christoph Hellwig @ 2010-09-23 14:58 ` Slawomir Nowakowski 2010-09-24 7:55 ` Christoph Hellwig 0 siblings, 1 reply; 21+ messages in thread From: Slawomir Nowakowski @ 2010-09-23 14:58 UTC (permalink / raw) To: Christoph Hellwig Cc: Artur Piechocki, Łukasz Wittig, 'Ryszard Stawiarski', xfs If you have any more questions, please do not hesitate to contact me. Best Regards Slawomir Nowakowski Technical Support Engineer ------------------------------------------------------- Open-E GmbH e-mail: slawomir.nowakowski@open-e.com http://www.open-e.com/ ------------------------------------------------------- W dniu 23.09.2010 16:32, Christoph Hellwig pisze: > On Thu, Sep 23, 2010 at 03:35:02PM +0200, Slawomir Nowakowski wrote: > >> Dear Sir, >> >> We are using iSCSI FileIO on the LVM. The file (called lun) has been >> > What is iSCSI FileIO? > > iSCSI FileIO is using struct file that serves it as block level device via iSCSI technology: http://en.wikipedia.org/wiki/ISCSI >> initialized with: >> dd if=/dev/zero of=/dev/vg+vg00/lv+i+lv0000 bs=1M conv=notrunc >> >> on the XFS formatted logical volume. >> >> Additionally, we use 2.6.27-39 kernel and xfs-tools 2.10.1 >> >> After some time, without any additional errors, we are not able >> mount the volume. After mount command we get: >> > Please describe the full storage topology. But most likely it's an > issue with your underlying storage. > > There are RAID level 6 unit on the Areca RAID controller (1680). On the unit is created a volume group. In the volume group we have several logical volumes. The iSCSI FileIO volume is mounted and the file (lun) is served via SCST target as iSCSI LUN. We have checked RAM with memtest+ and verified RAID unit health and no issues were found. Cheers _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-23 14:58 ` Slawomir Nowakowski @ 2010-09-24 7:55 ` Christoph Hellwig 2010-09-24 11:11 ` Slawomir Nowakowski ` (2 more replies) 0 siblings, 3 replies; 21+ messages in thread From: Christoph Hellwig @ 2010-09-24 7:55 UTC (permalink / raw) To: Slawomir Nowakowski Cc: Christoph Hellwig, Artur Piechocki, xfs, 'Ryszard Stawiarski', ??ukasz Wittig On Thu, Sep 23, 2010 at 04:58:47PM +0200, Slawomir Nowakowski wrote: > iSCSI FileIO is using struct file that serves it as block level > device via iSCSI technology: > > http://en.wikipedia.org/wiki/ISCSI Thanks, I know ISCSI very well. But what is "iSCSI FileIO" ? The above sounds like it's an iscsi target, is that correct? > There are RAID level 6 unit on the Areca RAID controller (1680). On > the unit is created a volume group. In the volume group we have > several logical volumes. The iSCSI FileIO volume is mounted and the > file (lun) is served via SCST target as iSCSI LUN. > > We have checked RAM with memtest+ and verified RAID unit health and > no issues were found. I still can't make any sense of the actual setups. The above seems to be the backend storage. Then there's SCST somewhere in which is in a out of tree kernel module. And then you use XFS somewhere. Please provide a full description of the setup. _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-24 7:55 ` Christoph Hellwig @ 2010-09-24 11:11 ` Slawomir Nowakowski 2010-09-24 13:18 ` Emmanuel Florac 2010-09-25 15:56 ` Christoph Hellwig 2010-09-24 13:18 ` Emmanuel Florac 2010-09-24 15:10 ` Richard Sharpe 2 siblings, 2 replies; 21+ messages in thread From: Slawomir Nowakowski @ 2010-09-24 11:11 UTC (permalink / raw) To: Christoph Hellwig; +Cc: xfs >> iSCSI FileIO is using struct file that serves it as block level >> device via iSCSI technology: >> >> http://en.wikipedia.org/wiki/ISCSI >> > Thanks, I know ISCSI very well. But what is "iSCSI FileIO" ? The above > sounds like it's an iscsi target, is that correct? > > Yes, it is iSCSI target that is served for another server. The FileIO serves a file as iSCSI target. >> There are RAID level 6 unit on the Areca RAID controller (1680). On >> the unit is created a volume group. In the volume group we have >> several logical volumes. The iSCSI FileIO volume is mounted and the >> file (lun) is served via SCST target as iSCSI LUN. >> >> We have checked RAM with memtest+ and verified RAID unit health and >> no issues were found. >> > I still can't make any sense of the actual setups. > > The above seems to be the backend storage. Then there's SCST somewhere > in which is in a out of tree kernel module. And then you use XFS > somewhere. Please provide a full description of the setup. > > So once again. We have created a RAID unit level 6. On the top of the unit there is an LVM architecture, I mean a volume group that contains logical volumes. The logical volume is formatted with XFS and it contains one big file that takes almost all of the space on the LV. There is some free space left in order to be able expand the LV and FS in the future. The LV is mounted and the file is served as iSCSI target. The iSCSI Initiator (MS Initiator from Windows 2k3) connects to iSCSI target. The iSCSI disk is formatted with the NTFS. But we believe the problem is with the XFS. With unknown reason we are not able to mount the LV and after running xfs_repair the file is missing from the LV. Do you have any ideas how we can try to fix the broken XFS? Cheers Slawek _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-24 11:11 ` Slawomir Nowakowski @ 2010-09-24 13:18 ` Emmanuel Florac 2010-09-24 13:46 ` Slawomir Nowakowski 2010-09-25 15:56 ` Christoph Hellwig 1 sibling, 1 reply; 21+ messages in thread From: Emmanuel Florac @ 2010-09-24 13:18 UTC (permalink / raw) To: slawomir.nowakowski; +Cc: xfs Le Fri, 24 Sep 2010 13:11:25 +0200 Slawomir Nowakowski <slawomir.nowakowski@open-e.com> écrivait: > But we believe the problem is with the XFS. With unknown reason we > are not able to mount the LV and after running xfs_repair the file is > missing from the LV. Do you have any ideas how we can try to fix the > broken XFS? This doesn't really make much sense to me. What target are you using? scst, tgt, lio or iet? What looks weird to me is that the dd command example you gave writes over the xfs filesystem : dd if=/dev/zero of=/dev/vg+vg00/lv+i+lv0000 bs=1M conv=notrunc This is definitely incorrect. Given that /dev/vg+vg00/lv+i+lv0000 is your xfs formatted lv, you must mount it somewhere : mount /dev/vg+vg00/lv+i+lv0000 /mnt/whatever Then you dd the file on /mnt/whatever : dd if=/dev/zero of=/mnt/whatever/lunfile bs=1M Lastly, you declare /mnt/whatever/lunfile as the lun in your target (method varies depending upon the target used), something like (ietd example): Target iqn.2010-09:com.whatever.host.target1 Alias host.target1 Lun 0 Type=fileio,Path=mnt/whatever/lunfile,IOMode=wback -- ------------------------------------------------------------------------ Emmanuel Florac | Direction technique | Intellique | <eflorac@intellique.com> | +33 1 78 94 84 02 ------------------------------------------------------------------------ _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-24 13:18 ` Emmanuel Florac @ 2010-09-24 13:46 ` Slawomir Nowakowski 0 siblings, 0 replies; 21+ messages in thread From: Slawomir Nowakowski @ 2010-09-24 13:46 UTC (permalink / raw) To: Emmanuel Florac; +Cc: xfs Dear Emmanuel, > Le Fri, 24 Sep 2010 13:11:25 +0200 > Slawomir Nowakowski<slawomir.nowakowski@open-e.com> écrivait: > > >> But we believe the problem is with the XFS. With unknown reason we >> are not able to mount the LV and after running xfs_repair the file is >> missing from the LV. Do you have any ideas how we can try to fix the >> broken XFS? >> > This doesn't really make much sense to me. What target are you using? > scst, tgt, lio or iet? What looks weird to me is that the dd command > example you gave writes over the xfs filesystem : > > dd if=/dev/zero of=/dev/vg+vg00/lv+i+lv0000 bs=1M conv=notrunc > > This is definitely incorrect. Given that /dev/vg+vg00/lv+i+lv0000 is > your xfs formatted lv, you must mount it somewhere : > > mount /dev/vg+vg00/lv+i+lv0000 /mnt/whatever > > Then you dd the file on /mnt/whatever : > > dd if=/dev/zero of=/mnt/whatever/lunfile bs=1M > > Lastly, you declare /mnt/whatever/lunfile as the lun in your target > (method varies depending upon the target used), something like (ietd > example): > > Target iqn.2010-09:com.whatever.host.target1 > Alias host.target1 > Lun 0 Type=fileio,Path=mnt/whatever/lunfile,IOMode=wback > > It's my mistake in dd command, sorry for that. First we mount the LV: mount [...] /dev/vg+vg00/lv+i+lv0000 on /mnt/point type xfs (rw,nouuid,attr2,nobarrier,noquota) [...] then we run dd to file dd if=/dev/zero of=/mnt/point/lun bs=1M conv=notrunc count=$size $size is counted to leave some free space on the device As the iSCSI target we use SCST 1.0.1.2. The scst.conf looks likes like this: [HANDLER vdisk] DEVICE 0QSP199WJI1yKOPj,/mnt/point/lun,WT,512,0QSP199WJI1yKOPj [GROUP Default_iqn.2010-03:sn1.target0] [GROUP Default] [ASSIGNMENT Default_iqn.2010-03:sn1.target0] DEVICE 0QSP199WJI1yKOPj,0 [ASSIGNMENT Default] [TARGETS enable] [TARGETS disable] The problem is that we were able to use this LUN in the target, but suddenly after a reboot we are not. Cheers Slawek _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-24 11:11 ` Slawomir Nowakowski 2010-09-24 13:18 ` Emmanuel Florac @ 2010-09-25 15:56 ` Christoph Hellwig 2010-09-25 16:54 ` Richard Sharpe 1 sibling, 1 reply; 21+ messages in thread From: Christoph Hellwig @ 2010-09-25 15:56 UTC (permalink / raw) To: Slawomir Nowakowski; +Cc: Christoph Hellwig, xfs > So once again. We have created a RAID unit level 6. On the top of > the unit there is an LVM architecture, I mean a volume group that > contains logical volumes. The logical volume is formatted with XFS > and it contains one big file that takes almost all of the space on > the LV. There is some free space left in order to be able expand the > LV and FS in the future. The LV is mounted and the file is served as > iSCSI target. The iSCSI Initiator (MS Initiator from Windows 2k3) > connects to iSCSI target. The iSCSI disk is formatted with the NTFS. ok, so we have: Linux Server +----------------------+ | hardware raid 6 | +----------------------+ | lvm2 - linear volume | +----------------------+ | XFS | +----------------------+ | iSCSI target | +----------------------+ Windows client: +----------------------+ | iSCSI initiator | +----------------------+ | NTFS | +----------------------+ > But we believe the problem is with the XFS. With unknown reason we > are not able to mount the LV and after running xfs_repair the file > is missing from the LV. Do you have any ideas how we can try to fix > the broken XFS? This does not sound like a plain XFS issue to me, but an interaction between components going completely wrong. Normal I/O to a file should never corrupt the filesystem around it to the point where it's unusable, and so far I never heard reports about that. The hint that this doesn't happen with another purely userspace target is interesting. I wonder if SCST that you use does any sort of in-kernel block I/O after using bmap or similar? I've not seen that for iscsi targets yet but for other kernel modules, and that kind of I/O can cause massive corruption on a filesystem with delayed allocation and unwritten extents. Can any of the SCST experts on the list here track down how I/O for this configuration will be issued? What does happen if you try the same setup with say jfs or ext4 instead of xfs? _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-25 15:56 ` Christoph Hellwig @ 2010-09-25 16:54 ` Richard Sharpe 2010-09-25 17:01 ` Christoph Hellwig 0 siblings, 1 reply; 21+ messages in thread From: Richard Sharpe @ 2010-09-25 16:54 UTC (permalink / raw) To: Christoph Hellwig; +Cc: Slawomir Nowakowski, xfs On Sat, Sep 25, 2010 at 8:56 AM, Christoph Hellwig <hch@infradead.org> wrote: >> So once again. We have created a RAID unit level 6. On the top of >> the unit there is an LVM architecture, I mean a volume group that >> contains logical volumes. The logical volume is formatted with XFS >> and it contains one big file that takes almost all of the space on >> the LV. There is some free space left in order to be able expand the >> LV and FS in the future. The LV is mounted and the file is served as >> iSCSI target. The iSCSI Initiator (MS Initiator from Windows 2k3) >> connects to iSCSI target. The iSCSI disk is formatted with the NTFS. > > ok, so we have: > > Linux Server > > +----------------------+ > | hardware raid 6 | > +----------------------+ > | lvm2 - linear volume | > +----------------------+ > | XFS | > +----------------------+ > | iSCSI target | > +----------------------+ > > Windows client: > > > +----------------------+ > | iSCSI initiator | > +----------------------+ > | NTFS | > +----------------------+ > >> But we believe the problem is with the XFS. With unknown reason we >> are not able to mount the LV and after running xfs_repair the file >> is missing from the LV. Do you have any ideas how we can try to fix >> the broken XFS? > > This does not sound like a plain XFS issue to me, but an interaction > between components going completely wrong. Normal I/O to a file > should never corrupt the filesystem around it to the point where > it's unusable, and so far I never heard reports about that. The hint > that this doesn't happen with another purely userspace target is > interesting. I wonder if SCST that you use does any sort of in-kernel > block I/O after using bmap or similar? I've not seen that for iscsi > targets yet but for other kernel modules, and that kind of I/O > can cause massive corruption on a filesystem with delayed allocation > and unwritten extents. > > Can any of the SCST experts on the list here track down how I/O for this > configuration will be issued? > > What does happen if you try the same setup with say jfs or ext4 instead > of xfs? I saw references to vdisk fileio in there and wondered why this was being done rather than simply exporting the hardware raid 6 device? Ie, why are all those other layers in there? fileio uses submit_bio to submit the data and it defaults to WRITE_THROUGH, NV_CACHE and DIRECT_IO (at least in the trunk, but I suspect this has been the case for a long while) however, the person making the complaint might have switched off WRITE_THROUGH in the pursuit of performance, in which case a crash could corrupt things badly but it would depend on whether or not clearing WRITE_THROUGH also clears NV_CACHE and what the code assembling the caching mode page does (and I have only had a cursory glance at the vdisk code). What is needed here is the parameters used in configuring the vdisk and the version of SCST in use. -- Regards, Richard Sharpe _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-25 16:54 ` Richard Sharpe @ 2010-09-25 17:01 ` Christoph Hellwig 2010-09-25 17:14 ` Richard Sharpe 0 siblings, 1 reply; 21+ messages in thread From: Christoph Hellwig @ 2010-09-25 17:01 UTC (permalink / raw) To: Richard Sharpe; +Cc: Christoph Hellwig, Slawomir Nowakowski, xfs On Sat, Sep 25, 2010 at 09:54:46AM -0700, Richard Sharpe wrote: > fileio uses submit_bio to submit the data and it defaults to > WRITE_THROUGH, NV_CACHE and DIRECT_IO (at least in the trunk, but I > suspect this has been the case for a long while) however, the person > making the complaint might have switched off WRITE_THROUGH in the > pursuit of performance, in which case a crash could corrupt things > badly but it would depend on whether or not clearing WRITE_THROUGH > also clears NV_CACHE and what the code assembling the caching mode > page does (and I have only had a cursory glance at the vdisk code). If the target uses submit_bio for logical files inside a filesystems there are hundreds of ways to get exactly the corruption that Slawomir sees. How does it obtain the logical to physical mapping? What locking does it use agains other access to the file? _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-25 17:01 ` Christoph Hellwig @ 2010-09-25 17:14 ` Richard Sharpe 0 siblings, 0 replies; 21+ messages in thread From: Richard Sharpe @ 2010-09-25 17:14 UTC (permalink / raw) To: Christoph Hellwig; +Cc: Slawomir Nowakowski, xfs On Sat, Sep 25, 2010 at 10:01 AM, Christoph Hellwig <hch@infradead.org> wrote: > On Sat, Sep 25, 2010 at 09:54:46AM -0700, Richard Sharpe wrote: >> fileio uses submit_bio to submit the data and it defaults to >> WRITE_THROUGH, NV_CACHE and DIRECT_IO (at least in the trunk, but I >> suspect this has been the case for a long while) however, the person >> making the complaint might have switched off WRITE_THROUGH in the >> pursuit of performance, in which case a crash could corrupt things >> badly but it would depend on whether or not clearing WRITE_THROUGH >> also clears NV_CACHE and what the code assembling the caching mode >> page does (and I have only had a cursory glance at the vdisk code). > > If the target uses submit_bio for logical files inside a filesystems > there are hundreds of ways to get exactly the corruption that Slawomir > sees. How does it obtain the logical to physical mapping? What locking > does it use agains other access to the file? Actually, I was wrong. for fileio it does vfs_writev. I started wondering the same questions and went back through the code and noticed that I had missed a test. We really need to know what parameters the person making the complaint is using, and the version of SCST. In addition, I don't understand when this corruption occurred ... It really might be a case of don't do that. -- Regards, Richard Sharpe _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-24 7:55 ` Christoph Hellwig 2010-09-24 11:11 ` Slawomir Nowakowski @ 2010-09-24 13:18 ` Emmanuel Florac 2010-09-24 13:49 ` Slawomir Nowakowski 2010-09-24 15:10 ` Richard Sharpe 2 siblings, 1 reply; 21+ messages in thread From: Emmanuel Florac @ 2010-09-24 13:18 UTC (permalink / raw) To: Christoph Hellwig Cc: 'Ryszard Stawiarski', ??ukasz Wittig, Artur Piechocki, xfs, Slawomir Nowakowski Le Fri, 24 Sep 2010 03:55:05 -0400 Christoph Hellwig <hch@infradead.org> écrivait: > Thanks, I know ISCSI very well. But what is "iSCSI FileIO" ? The > above sounds like it's an iscsi target, is that correct? It's an IO Mode that goes through the kernel VFS cache, as opposed to blockIO that does direct IO. > I still can't make any sense of the actual setups. > > The above seems to be the backend storage. Then there's SCST > somewhere in which is in a out of tree kernel module. And then you > use XFS somewhere. Please provide a full description of the setup. If the iSCSI targets are actually block devices (lvm lvs, disk partitions, etc), than using FileIO is a mistake, it may bring up all kind of weird behaviours, though normally no real errors - though I don't really know how scst fares in this regard. My understanding : he planned to create a file on the mounted XFS volume with dd but instead he dd'ed the lv itself, which obviously destroyed the filesystem. Or something else, I don't really know :) Slawomir, please show us the scst config file. Did you use mkfs and dd on the target or the initiator? This isn't clear. -- ------------------------------------------------------------------------ Emmanuel Florac | Direction technique | Intellique | <eflorac@intellique.com> | +33 1 78 94 84 02 ------------------------------------------------------------------------ _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-24 13:18 ` Emmanuel Florac @ 2010-09-24 13:49 ` Slawomir Nowakowski 2010-09-24 14:04 ` Emmanuel Florac 0 siblings, 1 reply; 21+ messages in thread From: Slawomir Nowakowski @ 2010-09-24 13:49 UTC (permalink / raw) To: Emmanuel Florac, Christoph Hellwig Cc: Artur Piechocki, Łukasz Wittig, 'Ryszard Stawiarski', xfs Dear Sirs, > Le Fri, 24 Sep 2010 03:55:05 -0400 > Christoph Hellwig<hch@infradead.org> écrivait: > > >> Thanks, I know ISCSI very well. But what is "iSCSI FileIO" ? The >> above sounds like it's an iscsi target, is that correct? >> > It's an IO Mode that goes through the kernel VFS cache, as opposed to > blockIO that does direct IO. > > >> I still can't make any sense of the actual setups. >> >> The above seems to be the backend storage. Then there's SCST >> somewhere in which is in a out of tree kernel module. And then you >> use XFS somewhere. Please provide a full description of the setup. >> > If the iSCSI targets are actually block devices (lvm lvs, disk > partitions, etc), than using FileIO is a mistake, it may bring up all > kind of weird behaviours, though normally no real errors - though I > don't really know how scst fares in this regard. > > My understanding : he planned to create a file on the mounted XFS > volume with dd but instead he dd'ed the lv itself, which obviously > destroyed the filesystem. Or something else, I don't really know :) > > Slawomir, please show us the scst config file. Did you use mkfs > and dd on the target or the initiator? This isn't clear. > > It's my mistake in dd command, sorry for that. First we mount the LV: mount [...] /dev/vg+vg00/lv+i+lv0000 on /mnt/point type xfs (rw,nouuid,attr2,nobarrier,noquota) [...] then we run dd to file dd if=/dev/zero of=/mnt/point/lun bs=1M conv=notrunc count=$size $size is counted to leave some free space on the device As the iSCSI target we use SCST 1.0.1.2. The scst.conf looks likes like this: [HANDLER vdisk] DEVICE 0QSP199WJI1yKOPj,/mnt/point/lun,WT,512,0QSP199WJI1yKOPj [GROUP Default_iqn.2010-03:sn1.target0] [GROUP Default] [ASSIGNMENT Default_iqn.2010-03:sn1.target0] DEVICE 0QSP199WJI1yKOPj,0 [ASSIGNMENT Default] [TARGETS enable] [TARGETS disable] The problem is that we were able to use this LUN in the target, but suddenly after a reboot we are not. Cheers Slawek _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-24 13:49 ` Slawomir Nowakowski @ 2010-09-24 14:04 ` Emmanuel Florac 0 siblings, 0 replies; 21+ messages in thread From: Emmanuel Florac @ 2010-09-24 14:04 UTC (permalink / raw) To: slawomir.nowakowski Cc: Christoph Hellwig, Artur Piechocki, Łukasz Wittig, 'Ryszard Stawiarski', xfs Le Fri, 24 Sep 2010 15:49:10 +0200 Slawomir Nowakowski <slawomir.nowakowski@open-e.com> écrivait: > The problem is that we were able to use this LUN in the target, but > suddenly after a reboot we are not. > That's strange indeed. I don't see why the target IO would cause this, but I don't know much scst. Did you try this several times? Did you try creating the file, unmounting and remounting without starting the scsi target, to see it it's something scst does that kills the filesystem? -- ------------------------------------------------------------------------ Emmanuel Florac | Direction technique | Intellique | <eflorac@intellique.com> | +33 1 78 94 84 02 ------------------------------------------------------------------------ _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-24 7:55 ` Christoph Hellwig 2010-09-24 11:11 ` Slawomir Nowakowski 2010-09-24 13:18 ` Emmanuel Florac @ 2010-09-24 15:10 ` Richard Sharpe 2010-09-24 18:18 ` Emmanuel Florac 2 siblings, 1 reply; 21+ messages in thread From: Richard Sharpe @ 2010-09-24 15:10 UTC (permalink / raw) To: Christoph Hellwig Cc: Artur Piechocki, ??ukasz Wittig, Slawomir Nowakowski, Ryszard Stawiarski, xfs On Fri, Sep 24, 2010 at 12:55 AM, Christoph Hellwig <hch@infradead.org> wrote: > On Thu, Sep 23, 2010 at 04:58:47PM +0200, Slawomir Nowakowski wrote: >> iSCSI FileIO is using struct file that serves it as block level >> device via iSCSI technology: >> >> http://en.wikipedia.org/wiki/ISCSI > > Thanks, I know ISCSI very well. But what is "iSCSI FileIO" ? The above > sounds like it's an iscsi target, is that correct? > >> There are RAID level 6 unit on the Areca RAID controller (1680). On >> the unit is created a volume group. In the volume group we have >> several logical volumes. The iSCSI FileIO volume is mounted and the >> file (lun) is served via SCST target as iSCSI LUN. >> >> We have checked RAM with memtest+ and verified RAID unit health and >> no issues were found. > > I still can't make any sense of the actual setups. > > The above seems to be the backend storage. Then there's SCST somewhere > in which is in a out of tree kernel module. And then you use XFS > somewhere. Please provide a full description of the setup. Sounds like iSCSI-SCST to a vdisk_fileio on a file under xfs on a LUN on the Areca RAID controller ... Does sound awfully complicated. Why not just serve the LUNs from the Areca directly? SCST allows you to do that. xfs is awesomely fast, but it is just adding overhead here. -- Regards, Richard Sharpe _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-24 15:10 ` Richard Sharpe @ 2010-09-24 18:18 ` Emmanuel Florac 2010-09-24 20:46 ` Ryszard Stawiarski 0 siblings, 1 reply; 21+ messages in thread From: Emmanuel Florac @ 2010-09-24 18:18 UTC (permalink / raw) To: Richard Sharpe Cc: Ryszard Stawiarski, ??ukasz Wittig, Artur Piechocki, xfs, Christoph Hellwig, Slawomir Nowakowski Le Fri, 24 Sep 2010 08:10:51 -0700 vous écriviez: > Sounds like iSCSI-SCST to a vdisk_fileio on a file under xfs on a LUN > on the Areca RAID controller ... I use iet and it works fine. It has the beneit of allowing easy copy, backup, etc of the virtual lun; and it allows thin provisioning too. -- ------------------------------------------------------------------------ Emmanuel Florac | Direction technique | Intellique | <eflorac@intellique.com> | +33 1 78 94 84 02 ------------------------------------------------------------------------ _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-24 18:18 ` Emmanuel Florac @ 2010-09-24 20:46 ` Ryszard Stawiarski 2010-09-25 14:13 ` Emmanuel Florac 0 siblings, 1 reply; 21+ messages in thread From: Ryszard Stawiarski @ 2010-09-24 20:46 UTC (permalink / raw) To: Emmanuel Florac Cc: Richard Sharpe, ig, Artur Piechocki, xfs, Christoph Hellwig, =?UTF-8?B?xYF1a2FzeiBXaXR0?=, Slawomir Nowakowski Hi Emmanuel! I C that there is no problem with IETD ;-) But we need to stay focused on SCST in this case :-) On 24.09.2010 20:18, Emmanuel Florac wrote: > Le Fri, 24 Sep 2010 08:10:51 -0700 vous écriviez: > > >> Sounds like iSCSI-SCST to a vdisk_fileio on a file under xfs on a LUN >> on the Areca RAID controller ... >> > I use iet and it works fine. It has the beneit of allowing easy copy, > backup, etc of the virtual lun; and it allows thin provisioning too. > > _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-24 20:46 ` Ryszard Stawiarski @ 2010-09-25 14:13 ` Emmanuel Florac 2010-09-25 14:18 ` Richard Sharpe 0 siblings, 1 reply; 21+ messages in thread From: Emmanuel Florac @ 2010-09-25 14:13 UTC (permalink / raw) To: ryszard.stawiarski Cc: Richard Sharpe, Łukasz Wittig, Artur Piechocki, xfs, Christoph Hellwig, Nowakowski, Slawomir Le Fri, 24 Sep 2010 22:46:51 +0200 vous écriviez: > I C that there is no problem with IETD ;-) > But we need to stay focused on SCST in this case :-) > Yes but you'd need to get sure that it's not scst that breaks the filesystem. My guess : it actually is. I personnally didn't find SCST of acceptable production quality. -- ------------------------------------------------------------------------ Emmanuel Florac | Direction technique | Intellique | <eflorac@intellique.com> | +33 1 78 94 84 02 ------------------------------------------------------------------------ _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-25 14:13 ` Emmanuel Florac @ 2010-09-25 14:18 ` Richard Sharpe 2010-09-25 19:01 ` Vladislav Bolkhovitin 0 siblings, 1 reply; 21+ messages in thread From: Richard Sharpe @ 2010-09-25 14:18 UTC (permalink / raw) To: Emmanuel Florac Cc: Vladislav Bolkhovitin, ryszard.stawiarski, Łukasz Wittig, Artur Piechocki, xfs, Christoph Hellwig, Slawomir Nowakowski On Sat, Sep 25, 2010 at 7:13 AM, Emmanuel Florac <eflorac@intellique.com> wrote: > Le Fri, 24 Sep 2010 22:46:51 +0200 vous écriviez: > >> I C that there is no problem with IETD ;-) >> But we need to stay focused on SCST in this case :-) >> > > Yes but you'd need to get sure that it's not scst that breaks the > filesystem. My guess : it actually is. *I personnally didn't find SCST > of acceptable production quality.* Great call there ... I see that Data Domain, Fusion IO and others have ignored your sage advice and suffered the financial consequences. -- Regards, Richard Sharpe _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-25 14:18 ` Richard Sharpe @ 2010-09-25 19:01 ` Vladislav Bolkhovitin 2010-09-25 21:23 ` Emmanuel Florac 0 siblings, 1 reply; 21+ messages in thread From: Vladislav Bolkhovitin @ 2010-09-25 19:01 UTC (permalink / raw) To: Emmanuel Florac Cc: Richard Sharpe, ryszard.stawiarski, Łukasz Wittig, Artur Piechocki, xfs, Christoph Hellwig, Slawomir Nowakowski Richard Sharpe, on 09/25/2010 06:18 PM wrote: > On Sat, Sep 25, 2010 at 7:13 AM, Emmanuel Florac <eflorac@intellique.com> wrote: >> Le Fri, 24 Sep 2010 22:46:51 +0200 vous écriviez: >> >>> I C that there is no problem with IETD ;-) >>> But we need to stay focused on SCST in this case :-) >>> >> >> Yes but you'd need to get sure that it's not scst that breaks the >> filesystem. My guess : it actually is. *I personnally didn't find SCST >> of acceptable production quality.* When did you try the last time? I guess, all you need is the latest 2.0.0.x SCST SVN branch? Or wait a bit for official 2.0 release. Vlad _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: Problem with file system on iSCSI FileIO 2010-09-25 19:01 ` Vladislav Bolkhovitin @ 2010-09-25 21:23 ` Emmanuel Florac 0 siblings, 0 replies; 21+ messages in thread From: Emmanuel Florac @ 2010-09-25 21:23 UTC (permalink / raw) To: Vladislav Bolkhovitin Cc: Richard Sharpe, ryszard.stawiarski, Łukasz Wittig, Piechocki, xfs, Christoph Hellwig, Slawomir Nowakowski, Artur Le Sat, 25 Sep 2010 23:01:37 +0400 vous écriviez: > When did you try the last time? Not very recently, I confess :) Furthermore, as there was another discussion on the ML about someone using tgt, I mixed it up. > I guess, all you need is the latest 2.0.0.x SCST SVN branch? Or wait a > bit for official 2.0 release. Yes, I want to give it a try. -- ------------------------------------------------------------------------ Emmanuel Florac | Direction technique | Intellique | <eflorac@intellique.com> | +33 1 78 94 84 02 ------------------------------------------------------------------------ _______________________________________________ xfs mailing list xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs ^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2010-09-25 21:22 UTC | newest] Thread overview: 21+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2010-09-23 13:35 Problem with file system on iSCSI FileIO Slawomir Nowakowski 2010-09-23 14:32 ` Christoph Hellwig 2010-09-23 14:58 ` Slawomir Nowakowski 2010-09-24 7:55 ` Christoph Hellwig 2010-09-24 11:11 ` Slawomir Nowakowski 2010-09-24 13:18 ` Emmanuel Florac 2010-09-24 13:46 ` Slawomir Nowakowski 2010-09-25 15:56 ` Christoph Hellwig 2010-09-25 16:54 ` Richard Sharpe 2010-09-25 17:01 ` Christoph Hellwig 2010-09-25 17:14 ` Richard Sharpe 2010-09-24 13:18 ` Emmanuel Florac 2010-09-24 13:49 ` Slawomir Nowakowski 2010-09-24 14:04 ` Emmanuel Florac 2010-09-24 15:10 ` Richard Sharpe 2010-09-24 18:18 ` Emmanuel Florac 2010-09-24 20:46 ` Ryszard Stawiarski 2010-09-25 14:13 ` Emmanuel Florac 2010-09-25 14:18 ` Richard Sharpe 2010-09-25 19:01 ` Vladislav Bolkhovitin 2010-09-25 21:23 ` Emmanuel Florac
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox