* Import/recover RAID6 created with SAS2108?
From: Dan Russell @ 2016-03-06 21:37 UTC (permalink / raw)
To: linux-raid
I have a RAID6 array which was created on a Supermicro card built around the LSI 2108. Two drives failed, I replaced one and started a rebuild, then two more failed, making my array go offline.
It looks like the failure was due to a cabling or backplane problem, and I suspect the data on the drives is good. However, the Supermicro card is unable to read the foreign configs (it thinks there are 3 foreign configs, none of which are importable into a working array).
I would like to attempt recovery per the Wiki’s instructions (overlay files, force assembly, etc); can md read a RAID6 created by the 2108, or would I be wasting my time?
I have moved the drives from the hardware RAID card to a non-RAID SAS card and can see all drives. However, mdadm -E on these drives seems to show the RAID controller’s field of vision (how many defined arrays, total number of physical disks in the system), not any insight into this drive’s role as a member of an array. I can paste or gist the output of madam -E (304 lines) if it would help.
Thanks in advance-
Dan
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH RESEND v2 11/18] fs: Ensure the mounter of a filesystem is privileged towards its inodes
From: Seth Forshee @ 2016-03-06 15:48 UTC (permalink / raw)
To: Eric W. Biederman
Cc: Alexander Viro, Serge Hallyn, Richard Weinberger,
Austin S Hemmelgarn, Miklos Szeredi, linux-kernel, linux-bcache,
dm-devel, linux-raid, linux-mtd, linux-fsdevel, fuse-devel,
linux-security-module, selinux
In-Reply-To: <87twkl50g5.fsf@x220.int.ebiederm.org>
On Fri, Mar 04, 2016 at 04:43:06PM -0600, Eric W. Biederman wrote:
> Seth Forshee <seth.forshee@canonical.com> writes:
>
> > On Mon, Jan 04, 2016 at 12:03:50PM -0600, Seth Forshee wrote:
> >> The mounter of a filesystem should be privileged towards the
> >> inodes of that filesystem. Extend the checks in
> >> inode_owner_or_capable() and capable_wrt_inode_uidgid() to
> >> permit access by users priviliged in the user namespace of the
> >> inode's superblock.
> >
> > Eric - I've discovered a problem related to this patch. The patches
> > you've already applied to your testing branch make it so that s_user_ns
> > can be an unprivileged user for proc and kernfs-based mounts. In some
> > cases DAC is the only thing protecting files in these mounts (ignoring
> > MAC), and with this patch an unprivileged user could bypass DAC.
> >
> > There's a simple solution - always set s_user_ns to &init_user_ns for
> > those filesystems. I think this is the right thing to do, since the
> > backing store behind these filesystems are really kernel objects. But
> > this would break the assumption behind your patch "userns: Simpilify
> > MNT_NODEV handling" and cause a regression in mounting behavior.
> >
> > I've come up with several possible solutions for this conflict.
> >
> > 1. Drop this patch and keep on setting s_user_ns to unprivilged users.
> > This would be unfortunate because I think this patch does make sense
> > for most filesystems.
> > 2. Restrict this patch so that a user privileged towards s_user_ns is
> > only privileged towards the super blocks inodes if s_user_ns has a
> > mapping for both i_uid and i_gid. This is better than (1) but still
> > not ideal in my mind.
> > 3. Drop your patch and maintain the current MNT_NODEV behavior.
> > 4. Add a new s_iflags flag to indicate a super block is from an
> > unprivileged mount, and use this in your patch instead of s_user_ns.
> >
> > Any preference, or any other ideas?
>
> In general this is only an issue if uids and gids on the filesystem
> do not map into the user namespace.
Yes, both capable_wrt_inode_uidgid and inode_owner_or_capable will
return true for a privileged user in the current namespace if the ids
map into that namespace.
> Therefore the general fix is to limit the logic of checking for
> capabilities in s_user_ns if we are dealing with INVALID_UID and
> INVALID_GID. For proc and kernfs that should never be the case
> so the problem becomes a non-issue.
>
> Further I would look at limiting that relaxation to just
> inode_change_ok. So that we can easily wrap that check per filesystem
> and deny the relaxation for proc and kernfs. proc and kernfs already
> have wrappers for .setattr so denying changes when !uid_vaid and
> !gid_valid would be a trivial addition, and ensure calamity does
> not ensure.
>
> Furthmore by limiting any additional to inode_change_ok we keep
> the work of the additional tests off of the fast paths.
So then the inode would need to be chowned before a privileged user in a
non-init namespace would be capable towards it. That seems workable. It
looks like INVALID_UID and INVALID_GID do map into init_user_ns (which
seems a bit odd) so real root remains capable towards those indoes.
That seems okay to me then.
Thanks,
Seth
^ permalink raw reply
* Re: multipath: I/O hanging forever
From: Kent Overstreet @ 2016-03-06 5:31 UTC (permalink / raw)
To: Andrea Righi; +Cc: Shaohua Li, linux-raid, linux-kernel
In-Reply-To: <20160304173044.GA2636@Dell>
On Fri, Mar 04, 2016 at 10:30:44AM -0700, Andrea Righi wrote:
> On Sun, Feb 28, 2016 at 08:46:16PM -0700, Andrea Righi wrote:
> > On Sun, Feb 28, 2016 at 06:53:33PM -0700, Andrea Righi wrote:
> > ...
> > > I'm using 4.5.0-rc5+, from Linus' git. I'll try to do a git bisect
> > > later, I'm pretty sure this problem has been introduced recently (i.e.,
> > > I've never seen this issue with 4.1.x).
> >
> > I confirm, just tested kernel 4.1 and this problem doesn't happen.
>
> Alright, I had some spare time to bisect this problem and I found that
> the commit that introduced this issue is c66a14d.
>
> So, I tried to revert the commit (with some changes to fix conflicts and
> ABI changes) and now multipath seems to work fine for me (no hung task).
Is it hanging on first IO, first large IO, or just randomly?
^ permalink raw reply
* Caching raid with SSD.
From: Ram Ramesh @ 2016-03-05 21:06 UTC (permalink / raw)
To: Linux Raid
Any one here actually use SSD caches for RAID arrays? Can you share your
experience and let me know your choice of the type of cache methods your
tired/used and why you think one is better or worse than other? If it is
possible, please provide raid type/size and ssd size used.
Thanks and Regards
Ramesh
^ permalink raw reply
* When do you replace old hard drives in a raid6?
From: Ram Ramesh @ 2016-03-05 20:49 UTC (permalink / raw)
To: Linux Raid
I am curious if people actually replace hard drives periodically because
they are old or out of warranty. My 5 device raid6 has several older
drives (3/5 are 3+ years old and out of warranty) They seem fine with
SMART and raid scrubs. However, it makes me wonder when they will die.
What is the best policy in such situations? More importantly, do people
wait for disks to die and then replace or have some ad hoc schedule of
replacing (like every 6mo replace oldest) to keep things safe?
Regards
Ramesh
^ permalink raw reply
* Re: RAID6 Array crash during reshape.....now will not re-assemble.
From: Andreas Klauer @ 2016-03-05 10:47 UTC (permalink / raw)
To: Another Sillyname; +Cc: Linux-RAID
In-Reply-To: <CAOS+5GEd4E2wzYhw0rPdrZnjK6UwAUdKCQLLTxXrdPO8d7hS7w@mail.gmail.com>
On Wed, Mar 02, 2016 at 03:46:48AM +0000, Another Sillyname wrote:
> As all the drives are showing Reshape pos'n 0 I'm assuming the reshape
> never got started (even though cat /proc/mdstat showed the array
> reshaping)?
I fixed such a thing by editing RAID metadata to no longer be in
reshape state... incomplete instructions:
https://bpaste.net/show/2231e697431d
The example above used 1.0 superblock I think; in order to adapt this
successfully to your situation you should refer to
https://raid.wiki.kernel.org/index.php/RAID_superblock_formats#The_version-1_superblock_format_on-disk_layout
If current mdadm/kernel has a way to get out of this ditch directly of
course that would be so much better...
Apart from that, the other alternative that comes to mind is using
--create but for that to be successful, you have to be sure to get all
variables right (superblock version, data offset, raid level, chunk
size, disk order, ...) and use --assume-clean and/or missing disks
to prevent resyncs and verify the results in read-only mode.
Regards
Andreas
^ permalink raw reply
* Re: Fwd: RAID6 Array crash during reshape.....now will not re-assemble.
From: Another Sillyname @ 2016-03-05 10:28 UTC (permalink / raw)
To: John Stoffel; +Cc: Alireza Haghdoost, Linux-RAID
In-Reply-To: <22234.1802.863099.33875@quad.stoffel.home>
John
As I said in a previous reply I'm not willing to just 'try' things
(such as using a later mdadm) as in my opinion that's not an
analytical approach and nothing will be learnt from a success. I want
to understand both why this happened and also what specifically needs
to be done to recover it (if it is a later version of mdadm what in
that later version addesses this problem), only then can any
subsequent user with a similar problem be able to to follow this
example to fix their array.
I'd already posted the mdadm examine in the OP, I've copied the
original OP below again for completeness.
Thanks for your thoughts.
The original post.
-----------------------------------------------------------------------------
I have a 30TB RAID6 array using 7 x 6TB drives that I wanted to
migrate to RAID5 to take one of the drives offline and use in a new
array for a migration.
sudo mdadm --grow /dev/md127 --level=raid5 --raid-device=6
--backup-file=mdadm_backupfile
I watched this using cat /proc/mdstat and even after an hour the
percentage of the reshape was still 0.0%.
I know from previous experience that reshaping can be slow, but did
not expect it to be this slow frankly. But erring on the side of
caution I decided to leave the array for 12 hours and see what was
happening then.
Sure enough, 12 hours later cat /proc/mdstat still shows reshape at 0.0%
Looking at CPU usage the reshape process is using 0% of the CPU.
So reading a bit more......if you reboot a server the reshape should continue.
Reboot.....
Array will not come back online at all.
Bring the server up without the array trying to automount.
cat /proc/mdstat shows the array offline.
Personalities :
md127 : inactive sdf1[2](S) sde1[3](S) sdg1[0](S) sdb1[8](S)
sdh1[7](S) sdc1[1](S) sdd1[6](S)
41022733300 blocks super 1.2
unused devices: <none>
Try to reassemble the array.
>sudo mdadm --assemble /dev/md127 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1
mdadm: /dev/sdg1 is busy - skipping
mdadm: /dev/sdh1 is busy - skipping
mdadm: Merging with already-assembled /dev/md/server187.internallan.com:1
mdadm: Failed to restore critical section for reshape, sorry.
Possibly you needed to specify the --backup-file
Have no idea where the server187 stuff has come from.
stop the array.
>sudo mdadm --stop /dev/md127
try to re-assemble
>sudo mdadm --assemble /dev/md127 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1
mdadm: Failed to restore critical section for reshape, sorry.
Possibly you needed to specify the --backup-file
try to re-assemble using the backup file
>sudo mdadm --assemble /dev/md127 /dev/sdb1 /dev/sdc1 /dev/sdd1 /dev/sde1 /dev/sdf1 /dev/sdg1 /dev/sdh1 --backup-file=mdadm_backupfile
mdadm: Failed to restore critical section for reshape, sorry.
have a look at the individual drives
>sudo mdadm --examine /dev/sd[b-h]1
/dev/sdb1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x5
Array UUID : da29a06f:f8cf1409:bc52afb2:6945ba08
Name : server187.internallan.com:1
Creation Time : Sun May 10 14:47:51 2015
Raid Level : raid6
Raid Devices : 7
Avail Dev Size : 11720780943 (5588.90 GiB 6001.04 GB)
Array Size : 29301952000 (27944.52 GiB 30005.20 GB)
Used Dev Size : 11720780800 (5588.90 GiB 6001.04 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=143 sectors
State : clean
Device UUID : 1152bdeb:15546156:1918b67d:37d68b1f
Internal Bitmap : 8 sectors from superblock
Reshape pos'n : 0
New Layout : left-symmetric-6
Update Time : Wed Mar 2 01:19:42 2016
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 3a66db58 - correct
Events : 369282
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 4
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdc1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x5
Array UUID : da29a06f:f8cf1409:bc52afb2:6945ba08
Name : server187.internallan.com:1
Creation Time : Sun May 10 14:47:51 2015
Raid Level : raid6
Raid Devices : 7
Avail Dev Size : 11720780943 (5588.90 GiB 6001.04 GB)
Array Size : 29301952000 (27944.52 GiB 30005.20 GB)
Used Dev Size : 11720780800 (5588.90 GiB 6001.04 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=143 sectors
State : clean
Device UUID : 140e09af:56e14b4e:5035d724:c2005f0b
Internal Bitmap : 8 sectors from superblock
Reshape pos'n : 0
New Layout : left-symmetric-6
Update Time : Wed Mar 2 01:19:42 2016
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 88916c56 - correct
Events : 369282
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 1
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdd1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x5
Array UUID : da29a06f:f8cf1409:bc52afb2:6945ba08
Name : server187.internallan.com:1
Creation Time : Sun May 10 14:47:51 2015
Raid Level : raid6
Raid Devices : 7
Avail Dev Size : 11720780943 (5588.90 GiB 6001.04 GB)
Array Size : 29301952000 (27944.52 GiB 30005.20 GB)
Used Dev Size : 11720780800 (5588.90 GiB 6001.04 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=143 sectors
State : clean
Device UUID : a50dd0a1:eeb0b3df:76200476:818e004d
Internal Bitmap : 8 sectors from superblock
Reshape pos'n : 0
New Layout : left-symmetric-6
Update Time : Wed Mar 2 01:19:42 2016
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 9f8eb46a - correct
Events : 369282
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 6
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sde1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x5
Array UUID : da29a06f:f8cf1409:bc52afb2:6945ba08
Name : server187.internallan.com:1
Creation Time : Sun May 10 14:47:51 2015
Raid Level : raid6
Raid Devices : 7
Avail Dev Size : 11720780943 (5588.90 GiB 6001.04 GB)
Array Size : 29301952000 (27944.52 GiB 30005.20 GB)
Used Dev Size : 11720780800 (5588.90 GiB 6001.04 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=143 sectors
State : clean
Device UUID : 7d0b65b3:d2ba2023:4625c287:1db2de9b
Internal Bitmap : 8 sectors from superblock
Reshape pos'n : 0
New Layout : left-symmetric-6
Update Time : Wed Mar 2 01:19:42 2016
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 552ce48f - correct
Events : 369282
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 3
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdf1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x5
Array UUID : da29a06f:f8cf1409:bc52afb2:6945ba08
Name : server187.internallan.com:1
Creation Time : Sun May 10 14:47:51 2015
Raid Level : raid6
Raid Devices : 7
Avail Dev Size : 11720780943 (5588.90 GiB 6001.04 GB)
Array Size : 29301952000 (27944.52 GiB 30005.20 GB)
Used Dev Size : 11720780800 (5588.90 GiB 6001.04 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=143 sectors
State : clean
Device UUID : cda4f5e5:a489dbb9:5c1ab6a0:b257c984
Internal Bitmap : 8 sectors from superblock
Reshape pos'n : 0
New Layout : left-symmetric-6
Update Time : Wed Mar 2 01:19:42 2016
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 2056e75c - correct
Events : 369282
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 2
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdg1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x5
Array UUID : da29a06f:f8cf1409:bc52afb2:6945ba08
Name : server187.internallan.com:1
Creation Time : Sun May 10 14:47:51 2015
Raid Level : raid6
Raid Devices : 7
Avail Dev Size : 11720780943 (5588.90 GiB 6001.04 GB)
Array Size : 29301952000 (27944.52 GiB 30005.20 GB)
Used Dev Size : 11720780800 (5588.90 GiB 6001.04 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=143 sectors
State : clean
Device UUID : df5af6ce:9017c863:697da267:046c9709
Internal Bitmap : 8 sectors from superblock
Reshape pos'n : 0
New Layout : left-symmetric-6
Update Time : Wed Mar 2 01:19:42 2016
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : fefea2b5 - correct
Events : 369282
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 0
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdh1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x5
Array UUID : da29a06f:f8cf1409:bc52afb2:6945ba08
Name : server187.internallan.com:1
Creation Time : Sun May 10 14:47:51 2015
Raid Level : raid6
Raid Devices : 7
Avail Dev Size : 11720780943 (5588.90 GiB 6001.04 GB)
Array Size : 29301952000 (27944.52 GiB 30005.20 GB)
Used Dev Size : 11720780800 (5588.90 GiB 6001.04 GB)
Data Offset : 262144 sectors
Super Offset : 8 sectors
Unused Space : before=262056 sectors, after=143 sectors
State : clean
Device UUID : 9d98af83:243c3e02:94de20c7:293de111
Internal Bitmap : 8 sectors from superblock
Reshape pos'n : 0
New Layout : left-symmetric-6
Update Time : Wed Mar 2 01:19:42 2016
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : b9f6375e - correct
Events : 369282
Layout : left-symmetric
Chunk Size : 512K
Device Role : Active device 5
Array State : AAAAAAA ('A' == active, '.' == missing, 'R' == replacing)
As all the drives are showing Reshape pos'n 0 I'm assuming the reshape
never got started (even though cat /proc/mdstat showed the array
reshaping)?
So now I'm well out of my comfort zone so instead of flapping around
have decided to sleep for a few hours before revisiting this.
Any help and guidance would be appreciated, the drives showing clean
gives me comfort that the data is likely intact and complete (crossed
fingers) however I can't re-assemble the array as I keep getting the
'critical information for reshape, sorry' warning.
Help???
--------------------------------------------------------------------------------------------------------
On 4 March 2016 at 22:07, John Stoffel <john@stoffel.org> wrote:
>
> Can you post the output of mdadm -E /dev/sd?1 for all your drives?
> And did you pull down the latest version of mdadm from neil's repo and
> build it and use that to undo the re-shape?
>
> John
>
>
> Another> I have no clue, they were used in a temporary system for 10 days about
> Another> 8 months ago, they were then used in the new array that was built back
> Another> in August.
>
> Another> Even if the metadata was removed from those two drives the 'merge'
> Another> that happened, without warning or requiring verification, seems to now
> Another> have 'contaminated' all the drives possibly.
>
> Another> I'm still reasonably convinced the data is there and intact, just need
> Another> an analytical approach to how to recover it.
>
>
>
> Another> On 4 March 2016 at 21:02, Alireza Haghdoost <alireza@cs.umn.edu> wrote:
>>> On Fri, Mar 4, 2016 at 2:30 PM, Another Sillyname
>>> <anothersname@googlemail.com> wrote:
>>>> That's possibly true, however there are lessons to be learnt here even
>>>> if my array is not recoverable.
>>>>
>>>> I don't know the process order of doing a reshape....but I would
>>>> suspect it's something along the lines of.
>>>>
>>>> Examine existing array.
>>>> Confirm command can be run against existing array configuration (i.e.
>>>> It's a valid command for this array setup).
>>>> Do backup file (if specified)
>>>> Set reshape flag high
>>>> Start reshape
>>>>
>>>> I would suggest....
>>>>
>>>> There needs to be another step in the process
>>>>
>>>> Before 'Set reshape flag high' that the backup file needs to be
>>>> checked for consistency.
>>>>
>>>> My backup file appears to be just full of EOLs (now for all I know the
>>>> backup file actually gets 'created' during the process and therefore
>>>> starts out as EOLs). But once the flag is set high you are then
>>>> committing the array before you know if the backup is good.
>>>>
>>>> Also
>>>>
>>>> The drives in this array had been working correctly for 6 months and
>>>> undergone a number of reboots.
>>>>
>>>> If, as we are theorising, there was some metadata from a previous
>>>> array setup on two of the drives that as a result of the reshape
>>>> somehow became the 'valid' metadata regarding those two drives RAID
>>>> status then I would suggest that during any mdadm raid create process
>>>> there is an extensive and thorough check of any drives being used to
>>>> identify and remove any possible previously existing RAID metadata
>>>> information...thus making the drives 'clean'.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On 4 March 2016 at 19:11, Alireza Haghdoost <alireza@cs.umn.edu> wrote:
>>>>> On Fri, Mar 4, 2016 at 1:01 PM, Another Sillyname
>>>>> <anothersname@googlemail.com> wrote:
>>>>>>
>>>>>>
>>>>>> Thanks for the suggestion but I'm still stuck and there is no bug
>>>>>> tracker on the mdadm git website so I have to wait here.
>>>>>>
>>>>>> Ho Huum
>>>>>>
>>>>>>
>>>>>
>>>>> Looks like it is going to be a long wait. I think you are waiting to
>>>>> do something that might not be inplace/available at all. That thing is
>>>>> the capability to reset reshape flag when the array metadata is not
>>>>> consistent. You had an old array in two of these drives and it seems
>>>>> mdadm confused when it observes the drives metadata are not
>>>>> consistent.
>>>>>
>>>>> Hope someone chip in some tricks to do so without a need to develop
>>>>> such a functionality in mdadm.
>>>
>>> Do you know the metadata version that is used on those two drives ?
>>> For example, if the version is < 1.0 then we could easily erase the
>>> old metadata since it has been recorded in the end of the drive. Newer
>>> metada versions after 1.0 are stored in the beginning of the drive.
>>>
>>> Therefore, there is no risk to erase your current array metadata !
> Another> --
> Another> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> Another> the body of a message to majordomo@vger.kernel.org
> Another> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH RESEND v2 11/18] fs: Ensure the mounter of a filesystem is privileged towards its inodes
From: Eric W. Biederman @ 2016-03-04 22:43 UTC (permalink / raw)
To: Seth Forshee
Cc: Alexander Viro, Serge Hallyn, Richard Weinberger,
Austin S Hemmelgarn, Miklos Szeredi, linux-kernel, linux-bcache,
dm-devel, linux-raid, linux-mtd, linux-fsdevel, fuse-devel,
linux-security-module, selinux
In-Reply-To: <20160303170201.GA30224@ubuntu-hedt>
Seth Forshee <seth.forshee@canonical.com> writes:
> On Mon, Jan 04, 2016 at 12:03:50PM -0600, Seth Forshee wrote:
>> The mounter of a filesystem should be privileged towards the
>> inodes of that filesystem. Extend the checks in
>> inode_owner_or_capable() and capable_wrt_inode_uidgid() to
>> permit access by users priviliged in the user namespace of the
>> inode's superblock.
>
> Eric - I've discovered a problem related to this patch. The patches
> you've already applied to your testing branch make it so that s_user_ns
> can be an unprivileged user for proc and kernfs-based mounts. In some
> cases DAC is the only thing protecting files in these mounts (ignoring
> MAC), and with this patch an unprivileged user could bypass DAC.
>
> There's a simple solution - always set s_user_ns to &init_user_ns for
> those filesystems. I think this is the right thing to do, since the
> backing store behind these filesystems are really kernel objects. But
> this would break the assumption behind your patch "userns: Simpilify
> MNT_NODEV handling" and cause a regression in mounting behavior.
>
> I've come up with several possible solutions for this conflict.
>
> 1. Drop this patch and keep on setting s_user_ns to unprivilged users.
> This would be unfortunate because I think this patch does make sense
> for most filesystems.
> 2. Restrict this patch so that a user privileged towards s_user_ns is
> only privileged towards the super blocks inodes if s_user_ns has a
> mapping for both i_uid and i_gid. This is better than (1) but still
> not ideal in my mind.
> 3. Drop your patch and maintain the current MNT_NODEV behavior.
> 4. Add a new s_iflags flag to indicate a super block is from an
> unprivileged mount, and use this in your patch instead of s_user_ns.
>
> Any preference, or any other ideas?
In general this is only an issue if uids and gids on the filesystem
do not map into the user namespace.
Therefore the general fix is to limit the logic of checking for
capabilities in s_user_ns if we are dealing with INVALID_UID and
INVALID_GID. For proc and kernfs that should never be the case
so the problem becomes a non-issue.
Further I would look at limiting that relaxation to just
inode_change_ok. So that we can easily wrap that check per filesystem
and deny the relaxation for proc and kernfs. proc and kernfs already
have wrappers for .setattr so denying changes when !uid_vaid and
!gid_valid would be a trivial addition, and ensure calamity does
not ensure.
Furthmore by limiting any additional to inode_change_ok we keep
the work of the additional tests off of the fast paths.
Eric
^ permalink raw reply
* Re: Fwd: RAID6 Array crash during reshape.....now will not re-assemble.
From: John Stoffel @ 2016-03-04 22:07 UTC (permalink / raw)
To: Another Sillyname; +Cc: Alireza Haghdoost, Linux-RAID
In-Reply-To: <CAOS+5GEr2p9nOcxWJxnX_u3_2PGyTfEPXnnzWYsMBWuXxSy-4Q@mail.gmail.com>
Can you post the output of mdadm -E /dev/sd?1 for all your drives?
And did you pull down the latest version of mdadm from neil's repo and
build it and use that to undo the re-shape?
John
Another> I have no clue, they were used in a temporary system for 10 days about
Another> 8 months ago, they were then used in the new array that was built back
Another> in August.
Another> Even if the metadata was removed from those two drives the 'merge'
Another> that happened, without warning or requiring verification, seems to now
Another> have 'contaminated' all the drives possibly.
Another> I'm still reasonably convinced the data is there and intact, just need
Another> an analytical approach to how to recover it.
Another> On 4 March 2016 at 21:02, Alireza Haghdoost <alireza@cs.umn.edu> wrote:
>> On Fri, Mar 4, 2016 at 2:30 PM, Another Sillyname
>> <anothersname@googlemail.com> wrote:
>>> That's possibly true, however there are lessons to be learnt here even
>>> if my array is not recoverable.
>>>
>>> I don't know the process order of doing a reshape....but I would
>>> suspect it's something along the lines of.
>>>
>>> Examine existing array.
>>> Confirm command can be run against existing array configuration (i.e.
>>> It's a valid command for this array setup).
>>> Do backup file (if specified)
>>> Set reshape flag high
>>> Start reshape
>>>
>>> I would suggest....
>>>
>>> There needs to be another step in the process
>>>
>>> Before 'Set reshape flag high' that the backup file needs to be
>>> checked for consistency.
>>>
>>> My backup file appears to be just full of EOLs (now for all I know the
>>> backup file actually gets 'created' during the process and therefore
>>> starts out as EOLs). But once the flag is set high you are then
>>> committing the array before you know if the backup is good.
>>>
>>> Also
>>>
>>> The drives in this array had been working correctly for 6 months and
>>> undergone a number of reboots.
>>>
>>> If, as we are theorising, there was some metadata from a previous
>>> array setup on two of the drives that as a result of the reshape
>>> somehow became the 'valid' metadata regarding those two drives RAID
>>> status then I would suggest that during any mdadm raid create process
>>> there is an extensive and thorough check of any drives being used to
>>> identify and remove any possible previously existing RAID metadata
>>> information...thus making the drives 'clean'.
>>>
>>>
>>>
>>>
>>>
>>>
>>> On 4 March 2016 at 19:11, Alireza Haghdoost <alireza@cs.umn.edu> wrote:
>>>> On Fri, Mar 4, 2016 at 1:01 PM, Another Sillyname
>>>> <anothersname@googlemail.com> wrote:
>>>>>
>>>>>
>>>>> Thanks for the suggestion but I'm still stuck and there is no bug
>>>>> tracker on the mdadm git website so I have to wait here.
>>>>>
>>>>> Ho Huum
>>>>>
>>>>>
>>>>
>>>> Looks like it is going to be a long wait. I think you are waiting to
>>>> do something that might not be inplace/available at all. That thing is
>>>> the capability to reset reshape flag when the array metadata is not
>>>> consistent. You had an old array in two of these drives and it seems
>>>> mdadm confused when it observes the drives metadata are not
>>>> consistent.
>>>>
>>>> Hope someone chip in some tricks to do so without a need to develop
>>>> such a functionality in mdadm.
>>
>> Do you know the metadata version that is used on those two drives ?
>> For example, if the version is < 1.0 then we could easily erase the
>> old metadata since it has been recorded in the end of the drive. Newer
>> metada versions after 1.0 are stored in the beginning of the drive.
>>
>> Therefore, there is no risk to erase your current array metadata !
Another> --
Another> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
Another> the body of a message to majordomo@vger.kernel.org
Another> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Fwd: RAID6 Array crash during reshape.....now will not re-assemble.
From: Another Sillyname @ 2016-03-04 21:52 UTC (permalink / raw)
To: Alireza Haghdoost, Linux-RAID
In-Reply-To: <CAB-428nDwNOuFe5Bqh=LXD5Vw8MxzKzpP6ELuAqen1wD54urmA@mail.gmail.com>
I have no clue, they were used in a temporary system for 10 days about
8 months ago, they were then used in the new array that was built back
in August.
Even if the metadata was removed from those two drives the 'merge'
that happened, without warning or requiring verification, seems to now
have 'contaminated' all the drives possibly.
I'm still reasonably convinced the data is there and intact, just need
an analytical approach to how to recover it.
On 4 March 2016 at 21:02, Alireza Haghdoost <alireza@cs.umn.edu> wrote:
> On Fri, Mar 4, 2016 at 2:30 PM, Another Sillyname
> <anothersname@googlemail.com> wrote:
>> That's possibly true, however there are lessons to be learnt here even
>> if my array is not recoverable.
>>
>> I don't know the process order of doing a reshape....but I would
>> suspect it's something along the lines of.
>>
>> Examine existing array.
>> Confirm command can be run against existing array configuration (i.e.
>> It's a valid command for this array setup).
>> Do backup file (if specified)
>> Set reshape flag high
>> Start reshape
>>
>> I would suggest....
>>
>> There needs to be another step in the process
>>
>> Before 'Set reshape flag high' that the backup file needs to be
>> checked for consistency.
>>
>> My backup file appears to be just full of EOLs (now for all I know the
>> backup file actually gets 'created' during the process and therefore
>> starts out as EOLs). But once the flag is set high you are then
>> committing the array before you know if the backup is good.
>>
>> Also
>>
>> The drives in this array had been working correctly for 6 months and
>> undergone a number of reboots.
>>
>> If, as we are theorising, there was some metadata from a previous
>> array setup on two of the drives that as a result of the reshape
>> somehow became the 'valid' metadata regarding those two drives RAID
>> status then I would suggest that during any mdadm raid create process
>> there is an extensive and thorough check of any drives being used to
>> identify and remove any possible previously existing RAID metadata
>> information...thus making the drives 'clean'.
>>
>>
>>
>>
>>
>>
>> On 4 March 2016 at 19:11, Alireza Haghdoost <alireza@cs.umn.edu> wrote:
>>> On Fri, Mar 4, 2016 at 1:01 PM, Another Sillyname
>>> <anothersname@googlemail.com> wrote:
>>>>
>>>>
>>>> Thanks for the suggestion but I'm still stuck and there is no bug
>>>> tracker on the mdadm git website so I have to wait here.
>>>>
>>>> Ho Huum
>>>>
>>>>
>>>
>>> Looks like it is going to be a long wait. I think you are waiting to
>>> do something that might not be inplace/available at all. That thing is
>>> the capability to reset reshape flag when the array metadata is not
>>> consistent. You had an old array in two of these drives and it seems
>>> mdadm confused when it observes the drives metadata are not
>>> consistent.
>>>
>>> Hope someone chip in some tricks to do so without a need to develop
>>> such a functionality in mdadm.
>
> Do you know the metadata version that is used on those two drives ?
> For example, if the version is < 1.0 then we could easily erase the
> old metadata since it has been recorded in the end of the drive. Newer
> metada versions after 1.0 are stored in the beginning of the drive.
>
> Therefore, there is no risk to erase your current array metadata !
^ permalink raw reply
* dead code question
From: Jes Sorensen @ 2016-03-04 21:27 UTC (permalink / raw)
To: gqjiang; +Cc: linux-raid, NeilBrown, Goldwyn Rodrigues
Hi,
Another question regarding the cluster code. This one was introduced
with:
commit d15a1f72bd92bc4724ee94b2ae8132633ffeb72b
Author: Guoqing Jiang <gqjiang@suse.com>
Date: Mon Oct 19 16:03:19 2015 +0800
Safeguard against writing to an active device of another node
Modifying an exiting device's superblock or creating a new superblock
on an existing device needs to be checked because the device could be
in use by another node in another array. So, we check this by taking
all superblock locks in userspace so that we don't step onto an active
device used by another node and safeguard against accidental edits.
After the edit is complete, we release all locks and the lockspace so
that it can be used by the kernel space.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
Signed-off-by: NeilBrown <neilb@suse.com>
[snip]
+/* Create the lockspace, take bitmapXXX locks on all the bitmaps. */
+int cluster_get_dlmlock(struct supertype *st, int *lockid)
+{
+ int ret = -1;
+ char str[64];
+ int flags = LKF_NOQUEUE;
+
+ dlm_lock_res = xmalloc(sizeof(struct dlm_lock_resource));
+ dlm_lock_res->ls = dlm_hooks->create_lockspace(st->cluster_name, O_RDWR);
+ if (!dlm_lock_res->ls) {
+ pr_err("%s failed to create lockspace\n", st->cluster_name);
+ goto out;
+ }
+
+ /* Conversions need the lockid in the LKSB */
+ if (flags & LKF_CONVERT)
+ dlm_lock_res->lksb.sb_lkid = *lockid;
You set flags = LKF_NOQUEUE but later check (flags & LKF_CONVERT), which
cannot be true given that:
mdadm.h:#define LKF_NOQUEUE 0x00000001
mdadm.h:#define LKF_CONVERT 0x00000004
Is this a bug, or do you have pending patches for this?
Cheers,
Jes
^ permalink raw reply
* use after free bug in b98043a2f8e7bb5b1918e2e02778f822f9dd4d3a
From: Jes Sorensen @ 2016-03-04 21:19 UTC (permalink / raw)
To: Guoqing Jiang; +Cc: linux-raid, Goldwyn Rodrigues, NeilBrown
Hi,
I was looking at ExamineBitmap() and noticed that your patch below,
seems to introduce a use after free bug.
commit b98043a2f8e7bb5b1918e2e02778f822f9dd4d3a
Author: Guoqing Jiang <gqjiang@suse.com>
Date: Wed Jun 10 13:42:07 2015 +0800
Show all bitmaps while examining bitmap
This adds capability of exmining bitmaps corresponding to all
nodes/slots on the device.
Signed-off-by: Goldwyn Rodrigues <rgoldwyn@suse.com>
Signed-off-by: Guoqing Jiang <gqjiang@suse.com>
Signed-off-by: NeilBrown <neilb@suse.de>
Line 284 does a close(fd), but further down the code your patch
introduces:
+ } else {
+ printf(" Cluster nodes : %d\n", sb->nodes);
+ printf(" Cluster name : %64s\n", sb->cluster_name);
+ for (i = 0; i < (int)sb->nodes; i++) {
+ if (i) {
+ free(info);
+ info = bitmap_fd_read(fd, brief);
+ sb = &info->sb;
+ }
It's not totally obvious to me here what the intention is for clustered
md here. Are you intending to use the same fd as was used at the top of
ExamineBitmap() or is it meant to open a new fd based on each clustered
entry?
In either case, the code as it is right now is certainly not going to
work :(
Cheers,
Jes
^ permalink raw reply
* Re: Fwd: RAID6 Array crash during reshape.....now will not re-assemble.
From: Alireza Haghdoost @ 2016-03-04 21:02 UTC (permalink / raw)
To: Another Sillyname; +Cc: John Stoffel, Linux-RAID
In-Reply-To: <CAOS+5GEZxF6B7f9W3eQg07fahYf02EGStu2whzWGTP4+p71fPQ@mail.gmail.com>
On Fri, Mar 4, 2016 at 2:30 PM, Another Sillyname
<anothersname@googlemail.com> wrote:
> That's possibly true, however there are lessons to be learnt here even
> if my array is not recoverable.
>
> I don't know the process order of doing a reshape....but I would
> suspect it's something along the lines of.
>
> Examine existing array.
> Confirm command can be run against existing array configuration (i.e.
> It's a valid command for this array setup).
> Do backup file (if specified)
> Set reshape flag high
> Start reshape
>
> I would suggest....
>
> There needs to be another step in the process
>
> Before 'Set reshape flag high' that the backup file needs to be
> checked for consistency.
>
> My backup file appears to be just full of EOLs (now for all I know the
> backup file actually gets 'created' during the process and therefore
> starts out as EOLs). But once the flag is set high you are then
> committing the array before you know if the backup is good.
>
> Also
>
> The drives in this array had been working correctly for 6 months and
> undergone a number of reboots.
>
> If, as we are theorising, there was some metadata from a previous
> array setup on two of the drives that as a result of the reshape
> somehow became the 'valid' metadata regarding those two drives RAID
> status then I would suggest that during any mdadm raid create process
> there is an extensive and thorough check of any drives being used to
> identify and remove any possible previously existing RAID metadata
> information...thus making the drives 'clean'.
>
>
>
>
>
>
> On 4 March 2016 at 19:11, Alireza Haghdoost <alireza@cs.umn.edu> wrote:
>> On Fri, Mar 4, 2016 at 1:01 PM, Another Sillyname
>> <anothersname@googlemail.com> wrote:
>>>
>>>
>>> Thanks for the suggestion but I'm still stuck and there is no bug
>>> tracker on the mdadm git website so I have to wait here.
>>>
>>> Ho Huum
>>>
>>>
>>
>> Looks like it is going to be a long wait. I think you are waiting to
>> do something that might not be inplace/available at all. That thing is
>> the capability to reset reshape flag when the array metadata is not
>> consistent. You had an old array in two of these drives and it seems
>> mdadm confused when it observes the drives metadata are not
>> consistent.
>>
>> Hope someone chip in some tricks to do so without a need to develop
>> such a functionality in mdadm.
Do you know the metadata version that is used on those two drives ?
For example, if the version is < 1.0 then we could easily erase the
old metadata since it has been recorded in the end of the drive. Newer
metada versions after 1.0 are stored in the beginning of the drive.
Therefore, there is no risk to erase your current array metadata !
^ permalink raw reply
* Re: Fwd: RAID6 Array crash during reshape.....now will not re-assemble.
From: Another Sillyname @ 2016-03-04 20:30 UTC (permalink / raw)
To: Alireza Haghdoost; +Cc: John Stoffel, Linux-RAID
In-Reply-To: <CAB-428mE21BZSvNCjyygQ4AWvRfRTyu7ZFfYZ8RYnEtbw+cMew@mail.gmail.com>
That's possibly true, however there are lessons to be learnt here even
if my array is not recoverable.
I don't know the process order of doing a reshape....but I would
suspect it's something along the lines of.
Examine existing array.
Confirm command can be run against existing array configuration (i.e.
It's a valid command for this array setup).
Do backup file (if specified)
Set reshape flag high
Start reshape
I would suggest....
There needs to be another step in the process
Before 'Set reshape flag high' that the backup file needs to be
checked for consistency.
My backup file appears to be just full of EOLs (now for all I know the
backup file actually gets 'created' during the process and therefore
starts out as EOLs). But once the flag is set high you are then
committing the array before you know if the backup is good.
Also
The drives in this array had been working correctly for 6 months and
undergone a number of reboots.
If, as we are theorising, there was some metadata from a previous
array setup on two of the drives that as a result of the reshape
somehow became the 'valid' metadata regarding those two drives RAID
status then I would suggest that during any mdadm raid create process
there is an extensive and thorough check of any drives being used to
identify and remove any possible previously existing RAID metadata
information...thus making the drives 'clean'.
On 4 March 2016 at 19:11, Alireza Haghdoost <alireza@cs.umn.edu> wrote:
> On Fri, Mar 4, 2016 at 1:01 PM, Another Sillyname
> <anothersname@googlemail.com> wrote:
>>
>>
>> Thanks for the suggestion but I'm still stuck and there is no bug
>> tracker on the mdadm git website so I have to wait here.
>>
>> Ho Huum
>>
>>
>
> Looks like it is going to be a long wait. I think you are waiting to
> do something that might not be inplace/available at all. That thing is
> the capability to reset reshape flag when the array metadata is not
> consistent. You had an old array in two of these drives and it seems
> mdadm confused when it observes the drives metadata are not
> consistent.
>
> Hope someone chip in some tricks to do so without a need to develop
> such a functionality in mdadm.
^ permalink raw reply
* Re: Fwd: RAID6 Array crash during reshape.....now will not re-assemble.
From: Alireza Haghdoost @ 2016-03-04 19:11 UTC (permalink / raw)
To: Another Sillyname; +Cc: John Stoffel, Linux-RAID
In-Reply-To: <CAOS+5GEeGapxE1_GUhxD6F2BCabso-M=h-HQYjZ0hvNxh04Hug@mail.gmail.com>
On Fri, Mar 4, 2016 at 1:01 PM, Another Sillyname
<anothersname@googlemail.com> wrote:
>
>
> Thanks for the suggestion but I'm still stuck and there is no bug
> tracker on the mdadm git website so I have to wait here.
>
> Ho Huum
>
>
Looks like it is going to be a long wait. I think you are waiting to
do something that might not be inplace/available at all. That thing is
the capability to reset reshape flag when the array metadata is not
consistent. You had an old array in two of these drives and it seems
mdadm confused when it observes the drives metadata are not
consistent.
Hope someone chip in some tricks to do so without a need to develop
such a functionality in mdadm.
^ permalink raw reply
* Re: Fwd: RAID6 Array crash during reshape.....now will not re-assemble.
From: Another Sillyname @ 2016-03-04 19:01 UTC (permalink / raw)
To: John Stoffel; +Cc: Linux-RAID
In-Reply-To: <22232.48582.65930.822795@quad.stoffel.home>
Hi John
Yes I had already tried the revert-reshape option with no effect. It
was when I found that option I also found the comment suggesting it
only applied to reshapes that are growing rather then shrinking.
Thanks for the suggestion but I'm still stuck and there is no bug
tracker on the mdadm git website so I have to wait here.
Ho Huum
On 3 March 2016 at 22:42, John Stoffel <john@stoffel.org> wrote:
>
> Another> Thanks for the suggestion but that's still 'trying' things
> Another> rather then an analytical approach.
>
> Well... since Neil is the guy who knows the code, and I've been
> several emails in the past about re-shapes gone wrong, and pulling
> down Neil's latest version was the solution. So that's what I'd go
> with.
>
> Another> I also do not want to reboot this machine until I absolutely
> Another> have to incase I need to capture any data needed to identify
> Another> and thereby resolve the problem.
>
> Reboot won't make a difference, all the data is on the disks.
>
> Another> Given I'm not getting much joy here I think I'll have to post
> Another> a bug tomorrow and see where that goes.
>
> I'd also argue that removing a disk from a RAID6 of 30Tb in size is
> crazy, but you know the risks I'm sure.
>
> It might have been better to just fail one disk, then zero it's
> super-block and use that new disk formatted by hand into a plain xfs
> or ext4 filesystem for you travels. Then when done, you'd just re-add
> the disk into the array and let it rebuild the second parity stripes.
>
> Also, I jsut dug into my archives, have you tried:
>
> --assemble --update=revert-reshape
>
> on your array?
>
> John
^ permalink raw reply
* Re: multipath: I/O hanging forever
From: Andrea Righi @ 2016-03-04 17:30 UTC (permalink / raw)
To: Shaohua Li, Kent Overstreet; +Cc: linux-raid, linux-kernel
In-Reply-To: <20160229034616.GA2682@Dell>
On Sun, Feb 28, 2016 at 08:46:16PM -0700, Andrea Righi wrote:
> On Sun, Feb 28, 2016 at 06:53:33PM -0700, Andrea Righi wrote:
> ...
> > I'm using 4.5.0-rc5+, from Linus' git. I'll try to do a git bisect
> > later, I'm pretty sure this problem has been introduced recently (i.e.,
> > I've never seen this issue with 4.1.x).
>
> I confirm, just tested kernel 4.1 and this problem doesn't happen.
Alright, I had some spare time to bisect this problem and I found that
the commit that introduced this issue is c66a14d.
So, I tried to revert the commit (with some changes to fix conflicts and
ABI changes) and now multipath seems to work fine for me (no hung task).
I'm not suggesting to apply the following patch, because if everthing
else is working (except multipath) probably the problem is in multipath
itself (any suggestion?). Anyway, for those who wanna test it, this is a
possible fix.
Thanks,
-Andrea
---
Subject: block: revert c66a14d (simplify bio_add_page())
Revert this change to avoid breaking the multipath (md) module.
After this commit doing I/O on a multipath volume makes tasks to hang
forever.
Example:
# mdadm -C /dev/md0 --level=multipath --raid-devices=2 /dev/sdb
# /dev/sdc
# cat /proc/mdstat
Personalities : [multipath]
md0 : active multipath sdb[0] sdc[1]
4042740 blocks super 1.2 [2/2] [UU]
# mkfs.xfs /dev/md0
meta-data=/dev/md0 isize=256 agcount=4, agsize=252672
blks
= sectsz=512 attr=2, projid32bit=0
data = bsize=4096 blocks=1010685, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
^C^C^C
# cat /proc/`pidof mkfs.xfs`/stack
[<ffffffff8126f53c>] do_blockdev_direct_IO+0x1adc/0x2300
[<ffffffff8126fda3>] __blockdev_direct_IO+0x43/0x50
[<ffffffff8126accc>] blkdev_direct_IO+0x4c/0x50
[<ffffffff811a2014>] generic_file_direct_write+0xa4/0x160
[<ffffffff811a2190>] __generic_file_write_iter+0xc0/0x1e0
[<ffffffff8126afc0>] blkdev_write_iter+0x80/0x100
[<ffffffff81228c3d>] __vfs_write+0xad/0xe0
[<ffffffff81229a85>] vfs_write+0xa5/0x1a0
[<ffffffff8122aacc>] SyS_pwrite64+0x6c/0xa0
[<ffffffff818281f2>] entry_SYSCALL_64_fastpath+0x12/0x76
[<ffffffffffffffff>] 0xffffffffffffffff
Signed-off-by: Andrea Righi <andrea@betterlinux.com>
---
block/bio.c | 110 ++++++++++++++++++++++++++++++++----------------------------
1 file changed, 58 insertions(+), 52 deletions(-)
diff --git a/block/bio.c b/block/bio.c
index cf75915..3cfcd9d 100644
--- a/block/bio.c
+++ b/block/bio.c
@@ -694,22 +694,31 @@ integrity_clone:
EXPORT_SYMBOL(bio_clone_bioset);
/**
- * bio_add_pc_page - attempt to add page to bio
- * @q: the target queue
- * @bio: destination bio
- * @page: page to add
- * @len: vec entry length
- * @offset: vec entry offset
- *
- * Attempt to add a page to the bio_vec maplist. This can fail for a
- * number of reasons, such as the bio being full or target block device
- * limitations. The target block device must allow bio's up to PAGE_SIZE,
- * so it is always possible to add a single page to an empty bio.
+ * bio_get_nr_vecs - return approx number of vecs
+ * @bdev: I/O target
*
- * This should only be used by REQ_PC bios.
+ * Return the approximate number of pages we can send to this target.
+ * There's no guarantee that you will be able to fit this number of pages
+ * into a bio, it does not account for dynamic restrictions that vary
+ * on offset.
*/
-int bio_add_pc_page(struct request_queue *q, struct bio *bio, struct page
- *page, unsigned int len, unsigned int offset)
+int bio_get_nr_vecs(struct block_device *bdev)
+{
+ struct request_queue *q = bdev_get_queue(bdev);
+ int nr_pages;
+
+ nr_pages = min_t(unsigned,
+ queue_max_segments(q),
+ queue_max_sectors(q) / (PAGE_SIZE >> 9) + 1);
+
+ return min_t(unsigned, nr_pages, BIO_MAX_PAGES);
+
+}
+EXPORT_SYMBOL(bio_get_nr_vecs);
+
+static int __bio_add_page(struct request_queue *q, struct bio *bio, struct page
+ *page, unsigned int len, unsigned int offset,
+ unsigned int max_sectors)
{
int retried_segments = 0;
struct bio_vec *bvec;
@@ -720,7 +729,7 @@ int bio_add_pc_page(struct request_queue *q, struct bio *bio, struct page
if (unlikely(bio_flagged(bio, BIO_CLONED)))
return 0;
- if (((bio->bi_iter.bi_size + len) >> 9) > queue_max_hw_sectors(q))
+ if (((bio->bi_iter.bi_size + len) >> 9) > max_sectors)
return 0;
/*
@@ -791,6 +800,28 @@ int bio_add_pc_page(struct request_queue *q, struct bio *bio, struct page
blk_recount_segments(q, bio);
return 0;
}
+
+/**
+ * bio_add_pc_page - attempt to add page to bio
+ * @q: the target queue
+ * @bio: destination bio
+ * @page: page to add
+ * @len: vec entry length
+ * @offset: vec entry offset
+ *
+ * Attempt to add a page to the bio_vec maplist. This can fail for a
+ * number of reasons, such as the bio being full or target block device
+ * limitations. The target block device must allow bio's up to PAGE_SIZE,
+ * so it is always possible to add a single page to an empty bio.
+ *
+ * This should only be used by REQ_PC bios.
+ */
+int bio_add_pc_page(struct request_queue *q, struct bio *bio, struct page *page,
+ unsigned int len, unsigned int offset)
+{
+ return __bio_add_page(q, bio, page, len, offset,
+ queue_max_hw_sectors(q));
+}
EXPORT_SYMBOL(bio_add_pc_page);
/**
@@ -800,47 +831,22 @@ EXPORT_SYMBOL(bio_add_pc_page);
* @len: vec entry length
* @offset: vec entry offset
*
- * Attempt to add a page to the bio_vec maplist. This will only fail
- * if either bio->bi_vcnt == bio->bi_max_vecs or it's a cloned bio.
+ * Attempt to add a page to the bio_vec maplist. This can fail for a
+ * number of reasons, such as the bio being full or target block device
+ * limitations. The target block device must allow bio's up to PAGE_SIZE,
+ * so it is always possible to add a single page to an empty bio.
*/
-int bio_add_page(struct bio *bio, struct page *page,
- unsigned int len, unsigned int offset)
+int bio_add_page(struct bio *bio, struct page *page, unsigned int len,
+ unsigned int offset)
{
- struct bio_vec *bv;
+ struct request_queue *q = bdev_get_queue(bio->bi_bdev);
+ unsigned int max_sectors;
- /*
- * cloned bio must not modify vec list
- */
- if (WARN_ON_ONCE(bio_flagged(bio, BIO_CLONED)))
- return 0;
+ max_sectors = blk_max_size_offset(q, bio->bi_iter.bi_sector);
+ if ((max_sectors < (len >> 9)) && !bio->bi_iter.bi_size)
+ max_sectors = len >> 9;
- /*
- * For filesystems with a blocksize smaller than the pagesize
- * we will often be called with the same page as last time and
- * a consecutive offset. Optimize this special case.
- */
- if (bio->bi_vcnt > 0) {
- bv = &bio->bi_io_vec[bio->bi_vcnt - 1];
-
- if (page == bv->bv_page &&
- offset == bv->bv_offset + bv->bv_len) {
- bv->bv_len += len;
- goto done;
- }
- }
-
- if (bio->bi_vcnt >= bio->bi_max_vecs)
- return 0;
-
- bv = &bio->bi_io_vec[bio->bi_vcnt];
- bv->bv_page = page;
- bv->bv_len = len;
- bv->bv_offset = offset;
-
- bio->bi_vcnt++;
-done:
- bio->bi_iter.bi_size += len;
- return len;
+ return __bio_add_page(q, bio, page, len, offset, max_sectors);
}
EXPORT_SYMBOL(bio_add_page);
^ permalink raw reply related
* Re: [PATCH 1/4] scatterlist: Introduce some helper functions
From: Baolin Wang @ 2016-03-04 6:53 UTC (permalink / raw)
To: Robert Jarzmik
Cc: Herbert Xu, David Miller, Alasdair G Kergon, Mike Snitzer, axboe,
dm-devel, akpm, david.s.gordon, Tom Lendacky, Masahiro Yamada,
smueller, tadeusz.struk, Masanari Iida, shli, Mark Brown,
Linus Walleij, Arnd Bergmann, LKML, linux-crypto, linux-raid
In-Reply-To: <CAMz4kuKtwCkgG-3GeCtfnAX5zN8MkwyVr2XVYzaV3Z6SFFcZww@mail.gmail.com>
>>> + **/
>>> +static inline bool sg_is_contiguous(struct scatterlist *sga,
>>> + struct scatterlist *sgb)
>>> +{
>>> + return ((sga->page_link & ~0x3UL) + sga->offset + sga->length ==
>>> + (sgb->page_link & ~0x3UL));
>>> +}
>> I don't understand that one.
>> sga->page_link is a pointer to a "struct page *". How can it be added to an
>> offset within a page ???
>
>
> Ah, sorry that's a mistake. It should check as below:
> static inline bool sg_is_contiguous(struct scatterlist *sga, struct
> scatterlist *sgb)
> {
> return (unsigned int)sg_virt(sga) + sga->length == (unsigned
> int)sg_virt(sgb);
> }
sorry, it should be:
static inline bool sg_is_contiguous(struct scatterlist *sga,
struct scatterlist *sgb)
{
return (unsigned long)sg_virt(sga) + sga->length ==
(unsigned long)sg_virt(sgb);
}
--
Baolin.wang
Best Regards
^ permalink raw reply
* Re: [PATCH 1/4] scatterlist: Introduce some helper functions
From: Baolin Wang @ 2016-03-04 6:29 UTC (permalink / raw)
To: Robert Jarzmik
Cc: Herbert Xu, David Miller, Alasdair G Kergon, Mike Snitzer, axboe,
dm-devel, akpm, david.s.gordon, Tom Lendacky, yamada.masahiro,
smueller, tadeusz.struk, standby24x7, shli, Mark Brown,
Linus Walleij, Arnd Bergmann, LKML, linux-crypto, linux-raid
In-Reply-To: <87povbpe3l.fsf@belgarion.home>
Hi Robert,
On 4 March 2016 at 03:15, Robert Jarzmik <robert.jarzmik@free.fr> wrote:
> Baolin Wang <baolin.wang@linaro.org> writes:
>
>> @@ -212,6 +212,37 @@ static inline void sg_unmark_end(struct scatterlist *sg)
>> }
>>
>> /**
>> + * sg_is_contiguous - Check if the scatterlists are contiguous
>> + * @sga: SG entry
>> + * @sgb: SG entry
>> + *
>> + * Description:
>> + * If the sga scatterlist is contiguous with the sgb scatterlist,
>> + * that means they can be merged together.
>> + *
>> + **/
>> +static inline bool sg_is_contiguous(struct scatterlist *sga,
>> + struct scatterlist *sgb)
>> +{
>> + return ((sga->page_link & ~0x3UL) + sga->offset + sga->length ==
>> + (sgb->page_link & ~0x3UL));
>> +}
> I don't understand that one.
> sga->page_link is a pointer to a "struct page *". How can it be added to an
> offset within a page ???
Ah, sorry that's a mistake. It should check as below:
static inline bool sg_is_contiguous(struct scatterlist *sga, struct
scatterlist *sgb)
{
return (unsigned int)sg_virt(sga) + sga->length == (unsigned
int)sg_virt(sgb);
}
>
>> @@ -370,6 +370,65 @@ int sg_alloc_table(struct sg_table *table, unsigned int
>> nents, gfp_t gfp_mask)
> ...
>> /**
>> + * sg_add_sg_to_table - Add one scatterlist into sg table
>> + * @sgt: The sg table header to use
>> + * @src: The sg need to be added into sg table
>> + *
>> + * Description:
>> + * The 'nents' member indicates how many scatterlists added in the sg table.
>> + * Copy the @src@ scatterlist into sg table and increase 'nents' member.
>> + *
>> + **/
>> +int sg_add_sg_to_table(struct sg_table *sgt, struct scatterlist *src)
>> +{
>> + unsigned int i = 0, orig_nents = sgt->orig_nents;
>> + struct scatterlist *sgl = sgt->sgl;
>> + struct scatterlist *sg;
>> +
>> + /* Check if there are enough space for the new sg to be added */
>> + if (sgt->nents >= sgt->orig_nents)
>> + return -EINVAL;
> I must admit I don't understand that one either : how do comparing the number of
> "mapped" entries against the number of "allocated" entries determines if there
> is enough room ?
That's for a dynamic sg table. If there is one sg table allocated
'orig_nents' scatterlists, and we need copy another mapped scatterlist
into the sg table if there are some requirements. So we use 'nents' to
record how many scatterlists have been copied into the sg table.
>
>> +/**
>> + * sg_alloc_empty_table - Allocate one empty sg table
>> + * @sgt: The sg table header to use
>> + * @nents: Number of entries in sg list
>> + * @gfp_mask: GFP allocation mask
>> + *
>> + * Description:
>> + * Allocate and initialize an sg table. The 'nents' member of sg_table
>> + * indicates how many scatterlists added in the sg table. It should set
>> + * 0 which means there are no scatterlists added in this sg table now.
>> + *
>> + **/
>> +int sg_alloc_empty_table(struct sg_table *sgt, unsigned int nents,
>> + gfp_t gfp_mask)
> As for this one, there has to be a purpose for it I fail to see. From far away
> it looks exactly like sg_alloc_table(), excepting it "works around" the nents >
> 0 protection of __sg_alloc_table().
> What is exactly the need for this one, and if it's usefull why not simply
> changing the __sg_alloc_table() "nents > 0" test and see what the outcome of the
> review will be ?
Like I said above. If we want to copy some mapped scatterlists into
one sg table, we should set the 'nents' to 0 to indicates how many
scatterlists coppied in the sg table.
Thanks for your comments.
>
> Cheers.
>
> --
> Robert
--
Baolin.wang
Best Regards
^ permalink raw reply
* Re: Fwd: RAID6 Array crash during reshape.....now will not re-assemble.
From: John Stoffel @ 2016-03-03 22:42 UTC (permalink / raw)
To: Another Sillyname; +Cc: John Stoffel, Linux-RAID
In-Reply-To: <CAOS+5GFfNdeYrfw6j_LowkW+g69h2fEOikHN1=-VQ98ZF9Jtpg@mail.gmail.com>
Another> Thanks for the suggestion but that's still 'trying' things
Another> rather then an analytical approach.
Well... since Neil is the guy who knows the code, and I've been
several emails in the past about re-shapes gone wrong, and pulling
down Neil's latest version was the solution. So that's what I'd go
with.
Another> I also do not want to reboot this machine until I absolutely
Another> have to incase I need to capture any data needed to identify
Another> and thereby resolve the problem.
Reboot won't make a difference, all the data is on the disks.
Another> Given I'm not getting much joy here I think I'll have to post
Another> a bug tomorrow and see where that goes.
I'd also argue that removing a disk from a RAID6 of 30Tb in size is
crazy, but you know the risks I'm sure.
It might have been better to just fail one disk, then zero it's
super-block and use that new disk formatted by hand into a plain xfs
or ext4 filesystem for you travels. Then when done, you'd just re-add
the disk into the array and let it rebuild the second parity stripes.
Also, I jsut dug into my archives, have you tried:
--assemble --update=revert-reshape
on your array?
John
^ permalink raw reply
* Re: Fwd: RAID6 Array crash during reshape.....now will not re-assemble.
From: Another Sillyname @ 2016-03-03 22:19 UTC (permalink / raw)
To: John Stoffel; +Cc: Linux-RAID
In-Reply-To: <22232.41693.691818.451316@quad.stoffel.home>
John
Thanks for the suggestion but that's still 'trying' things rather then
an analytical approach.
I also do not want to reboot this machine until I absolutely have to
incase I need to capture any data needed to identify and thereby
resolve the problem.
Given I'm not getting much joy here I think I'll have to post a bug
tomorrow and see where that goes.
Thanks again.
On 3 March 2016 at 20:47, John Stoffel <john@stoffel.org> wrote:
>
> Have you tried pulling down the latest version of mdadm from Neil's
> site with:
>
> git clone git://neil.brown.name/mdadm/ mdadm
> cd mdadm
> ./configure
> make
>
> and seeing if that custom build does the trick for you? I know he's
> done some newer patches which might help in this case.
>
>
> Another> I'd read that a couple of days back and while it's an interesting idea
> Another> I don't believe it will address my specific issue of the array not
> Another> allowing re-assembly (even with force and read only flags set) as
> Another> mdadm reports the 'need backup file for reshape, sorry' error no
> Another> matter what I've tried.
>
> Another> Even trying the above flags with invalid-backup does not work so I
> Another> need someone to have a eureka moment and say "....it's this....".
>
> Another> I believe that mdadm 3.3 incorporated the recover during reshape
> Another> functionality, however I've read elsewhere it only applies to
> Another> expansion into a new drive...I was going RAID6 to RAID5 (even though
> Another> it never really got started) and the backup file looks like 20mb of
> Another> EOLs.
>
> Another> So at he moment I'm pretty much stuck unless someone can tell me how
> Another> to clear down the reshape flag, even in read only mode so I can copy
> Another> the data off.
>
>
> Another> On 3 March 2016 at 17:48, Sarah Newman <srn@prgmr.com> wrote:
>>> On 03/03/2016 06:07 AM, Another Sillyname wrote:
>>>> Plenty of hits....but no clear fixes and as I said I'm not willing to
>>>> 'try' things from google hits until someone with better insight can
>>>> give me a view.
>>>>
>>>> Trying things is fine in many instances, but not with a 20+TB data set
>>>> as I'm sure you can understand.
>>>>
>>>
>>> I have not tried this personally, but it may be of interest:
>>> https://raid.wiki.kernel.org/index.php/Recovering_a_failed_software_RAID#Making_the_harddisks_read-only_using_an_overlay_file I found this explanation
>>> of the dmsetup commands to be more easy to follow: http://www.flaterco.com/kb/sandbox.html
>>>
> Another> --
> Another> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> Another> the body of a message to majordomo@vger.kernel.org
> Another> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: Fwd: RAID6 Array crash during reshape.....now will not re-assemble.
From: John Stoffel @ 2016-03-03 20:47 UTC (permalink / raw)
To: Another Sillyname; +Cc: Linux-RAID
In-Reply-To: <CAOS+5GEnvtPMUqorddDNUvkNr2+K2S=0N76i3f0AkAOE-7U4Pg@mail.gmail.com>
Have you tried pulling down the latest version of mdadm from Neil's
site with:
git clone git://neil.brown.name/mdadm/ mdadm
cd mdadm
./configure
make
and seeing if that custom build does the trick for you? I know he's
done some newer patches which might help in this case.
Another> I'd read that a couple of days back and while it's an interesting idea
Another> I don't believe it will address my specific issue of the array not
Another> allowing re-assembly (even with force and read only flags set) as
Another> mdadm reports the 'need backup file for reshape, sorry' error no
Another> matter what I've tried.
Another> Even trying the above flags with invalid-backup does not work so I
Another> need someone to have a eureka moment and say "....it's this....".
Another> I believe that mdadm 3.3 incorporated the recover during reshape
Another> functionality, however I've read elsewhere it only applies to
Another> expansion into a new drive...I was going RAID6 to RAID5 (even though
Another> it never really got started) and the backup file looks like 20mb of
Another> EOLs.
Another> So at he moment I'm pretty much stuck unless someone can tell me how
Another> to clear down the reshape flag, even in read only mode so I can copy
Another> the data off.
Another> On 3 March 2016 at 17:48, Sarah Newman <srn@prgmr.com> wrote:
>> On 03/03/2016 06:07 AM, Another Sillyname wrote:
>>> Plenty of hits....but no clear fixes and as I said I'm not willing to
>>> 'try' things from google hits until someone with better insight can
>>> give me a view.
>>>
>>> Trying things is fine in many instances, but not with a 20+TB data set
>>> as I'm sure you can understand.
>>>
>>
>> I have not tried this personally, but it may be of interest:
>> https://raid.wiki.kernel.org/index.php/Recovering_a_failed_software_RAID#Making_the_harddisks_read-only_using_an_overlay_file I found this explanation
>> of the dmsetup commands to be more easy to follow: http://www.flaterco.com/kb/sandbox.html
>>
Another> --
Another> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
Another> the body of a message to majordomo@vger.kernel.org
Another> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply
* Re: [PATCH 1/4] scatterlist: Introduce some helper functions
From: Robert Jarzmik @ 2016-03-03 19:15 UTC (permalink / raw)
To: Baolin Wang
Cc: herbert, davem, agk, snitzer, axboe, dm-devel, akpm,
david.s.gordon, thomas.lendacky, yamada.masahiro, smueller,
tadeusz.struk, standby24x7, shli, broonie, linus.walleij, arnd,
linux-kernel, linux-crypto, linux-raid
In-Reply-To: <3bff3e286d3ee01ebb7e26d7233075054c42a7a9.1456981314.git.baolin.wang@linaro.org>
Baolin Wang <baolin.wang@linaro.org> writes:
> @@ -212,6 +212,37 @@ static inline void sg_unmark_end(struct scatterlist *sg)
> }
>
> /**
> + * sg_is_contiguous - Check if the scatterlists are contiguous
> + * @sga: SG entry
> + * @sgb: SG entry
> + *
> + * Description:
> + * If the sga scatterlist is contiguous with the sgb scatterlist,
> + * that means they can be merged together.
> + *
> + **/
> +static inline bool sg_is_contiguous(struct scatterlist *sga,
> + struct scatterlist *sgb)
> +{
> + return ((sga->page_link & ~0x3UL) + sga->offset + sga->length ==
> + (sgb->page_link & ~0x3UL));
> +}
I don't understand that one.
sga->page_link is a pointer to a "struct page *". How can it be added to an
offset within a page ???
> @@ -370,6 +370,65 @@ int sg_alloc_table(struct sg_table *table, unsigned int
> nents, gfp_t gfp_mask)
...
> /**
> + * sg_add_sg_to_table - Add one scatterlist into sg table
> + * @sgt: The sg table header to use
> + * @src: The sg need to be added into sg table
> + *
> + * Description:
> + * The 'nents' member indicates how many scatterlists added in the sg table.
> + * Copy the @src@ scatterlist into sg table and increase 'nents' member.
> + *
> + **/
> +int sg_add_sg_to_table(struct sg_table *sgt, struct scatterlist *src)
> +{
> + unsigned int i = 0, orig_nents = sgt->orig_nents;
> + struct scatterlist *sgl = sgt->sgl;
> + struct scatterlist *sg;
> +
> + /* Check if there are enough space for the new sg to be added */
> + if (sgt->nents >= sgt->orig_nents)
> + return -EINVAL;
I must admit I don't understand that one either : how do comparing the number of
"mapped" entries against the number of "allocated" entries determines if there
is enough room ?
> +/**
> + * sg_alloc_empty_table - Allocate one empty sg table
> + * @sgt: The sg table header to use
> + * @nents: Number of entries in sg list
> + * @gfp_mask: GFP allocation mask
> + *
> + * Description:
> + * Allocate and initialize an sg table. The 'nents' member of sg_table
> + * indicates how many scatterlists added in the sg table. It should set
> + * 0 which means there are no scatterlists added in this sg table now.
> + *
> + **/
> +int sg_alloc_empty_table(struct sg_table *sgt, unsigned int nents,
> + gfp_t gfp_mask)
As for this one, there has to be a purpose for it I fail to see. From far away
it looks exactly like sg_alloc_table(), excepting it "works around" the nents >
0 protection of __sg_alloc_table().
What is exactly the need for this one, and if it's usefull why not simply
changing the __sg_alloc_table() "nents > 0" test and see what the outcome of the
review will be ?
Cheers.
--
Robert
^ permalink raw reply
* Re: Fwd: RAID6 Array crash during reshape.....now will not re-assemble.
From: Another Sillyname @ 2016-03-03 17:59 UTC (permalink / raw)
To: Linux-RAID
In-Reply-To: <56D878E8.1060409@prgmr.com>
Sarah
Thanks for the suggestion.
I'd read that a couple of days back and while it's an interesting idea
I don't believe it will address my specific issue of the array not
allowing re-assembly (even with force and read only flags set) as
mdadm reports the 'need backup file for reshape, sorry' error no
matter what I've tried.
Even trying the above flags with invalid-backup does not work so I
need someone to have a eureka moment and say "....it's this....".
I believe that mdadm 3.3 incorporated the recover during reshape
functionality, however I've read elsewhere it only applies to
expansion into a new drive...I was going RAID6 to RAID5 (even though
it never really got started) and the backup file looks like 20mb of
EOLs.
So at he moment I'm pretty much stuck unless someone can tell me how
to clear down the reshape flag, even in read only mode so I can copy
the data off.
On 3 March 2016 at 17:48, Sarah Newman <srn@prgmr.com> wrote:
> On 03/03/2016 06:07 AM, Another Sillyname wrote:
>> Plenty of hits....but no clear fixes and as I said I'm not willing to
>> 'try' things from google hits until someone with better insight can
>> give me a view.
>>
>> Trying things is fine in many instances, but not with a 20+TB data set
>> as I'm sure you can understand.
>>
>
> I have not tried this personally, but it may be of interest:
> https://raid.wiki.kernel.org/index.php/Recovering_a_failed_software_RAID#Making_the_harddisks_read-only_using_an_overlay_file I found this explanation
> of the dmsetup commands to be more easy to follow: http://www.flaterco.com/kb/sandbox.html
>
^ permalink raw reply
* Re: Fwd: RAID6 Array crash during reshape.....now will not re-assemble.
From: Sarah Newman @ 2016-03-03 17:48 UTC (permalink / raw)
To: Another Sillyname, Linux-RAID
In-Reply-To: <CAOS+5GHUKD63M90zWfiv14QkJunj2QF+gYqDMrg208-S4oXE2Q@mail.gmail.com>
On 03/03/2016 06:07 AM, Another Sillyname wrote:
> Plenty of hits....but no clear fixes and as I said I'm not willing to
> 'try' things from google hits until someone with better insight can
> give me a view.
>
> Trying things is fine in many instances, but not with a 20+TB data set
> as I'm sure you can understand.
>
I have not tried this personally, but it may be of interest:
https://raid.wiki.kernel.org/index.php/Recovering_a_failed_software_RAID#Making_the_harddisks_read-only_using_an_overlay_file I found this explanation
of the dmsetup commands to be more easy to follow: http://www.flaterco.com/kb/sandbox.html
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox