Re: xfs_repair of critical volume

Eli Morris
2010-12-04T00:41:42Z

[-- Attachment #1.1: Type: text/plain, Size: 5562 bytes --]

On Dec 2, 2010, at 3:33 AM, Michael Monnerie wrote:

> On Dienstag, 30. November 2010 Eli Morris wrote:
>> Thanks for your help with this. I wrote the program and ran it
>> through and it looks like we have we able to preserve 44 TB of valid
>> data, while removing the corrupted files, which is a great result,
>> considering the circumstances.
>
> Eli, could you post the relevant program here so others can use it if
> needed? There are requests from time to time, and it would be good if
> such a program were available (like I'm sure you'd been happy if it
> already existed the time you needed it).
>
> Thanks, and wow: what an amazing filesystem can recover such an event!
>
> --
> mit freundlichen Grüssen,
> Michael Monnerie, Ing. BSc
>
> it-management Internet Services: Protéger
> http://proteger.at [gesprochen: Prot-e-schee]
> Tel: +43 660 / 415 6531
>
> // ****** Radiointerview zum Thema Spam ******
> // http://www.it-podcast.at/archiv.html#podcast-100716
> //
> // Haus zu verkaufen: http://zmi.at/langegg/

Good idea, here is the program:

Eli

#!/bin/bash
#
# Copyright 2010 Eli Morris, Travis O'Brien, University of California
#
# remove_bad.sh is free software: you can redistribute it under the terms
# of the GNU General Public License as published by the Free Software
# Foundation, either version 3 of the License, or (at your option) any later
# version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program. If not, see <http://www.gnu.org/licenses/>.
#
#
#remove_bad.sh: A script to determine whether any part of a file falls within a
#set of blocks (indicated by arguments 1 and 2). This script is
#originally written with the intent to find files on a file system that
#exist(ed) on a corrupt section of the file system. It generates a list of files
#that are potentially bad, so that they can be removed by another script.
#

#Check command line arguments; grab arguments 1 and 2
if [ $# -eq 2 ]; then
BAD_BLOCK_BEGINNING=$1
BAD_BLOCK_END=$2
echo "bad block beginning $BAD_BLOCK_BEGINNING"
echo "bad block ending $BAD_BLOCK_END"
#if there aren't exactly 2 arguments then print the usage to the user
else
echo "usage: remove_bad.sh beginning_block ending_block"
exit
fi

remove file from last run
if ( test -e "./naughty_list.txt")
then
echo "removing the previous naughty list"
rm "./naughty_list.txt"
fi

IFS=$'\n' #set the field separator to the carriage return character
ALL_FILES=(`find /export/vol5 -type f`) #A list of all files on the volume, SUBSTITUTE NAME OF YOUR VOLUME
NUM_FILES=${#ALL_FILES[@]} #The number of files on the volume
echo "number of files is $NUM_FILES" #Report the number of files to the user

# for each of the file in vol5
for (( COUNT=0; COUNT<$NUM_FILES; COUNT++))
do
#Report which file is being worked on
echo "file number: $COUNT is ${ALL_FILES[$COUNT]}"

# report number of files to go
FILES_TO_GO=$((NUM_FILES-COUNT))
echo "files left: $FILES_TO_GO"

#Run xfs_bmap to get the blocks that the file lives within
OUTPUT=(`xfs_bmap ${ALL_FILES[$COUNT]}`)
# output looks like this
# vol5dump:
# 0: [0..1053271]: 5200578944..5201632215

BAD_FILE=0 #Initialize the bad file flag
NUM_LINES=${#OUTPUT[@]} #The number of lines from xfs_bmap

# echo "number of lines for file: $NUM_LINES" #Report the number of lines to the user
#Loop through each line
for (( LINE=1; LINE < $NUM_LINES; LINE++))
do
# echo "line number $LINE: output: ${OUTPUT[$LINE]}" #Report the current working line

# get the block range from the line
BLOCKS=`echo ${OUTPUT[$LINE]} | cut -d':' -f3`

#Report the number of blocks occupied
# echo "blocks after cut: '$BLOCKS'"
#Use cut to get the first and last block for the file
FIRST_BLOCK=`echo $BLOCKS | cut -d'.' -f1`
LAST_BLOCK=`echo $BLOCKS | cut -d'.' -f3`

#Report these to the user
# echo "beginning block: $FIRST_BLOCK"
# echo "ending block: $LAST_BLOCK"

#TODO: I'm not sure what exactly 'hole' means, but I get the impression that it has something
#to do with XFS's way of avoiding file fragmentation. TAO
if [ "$BLOCKS" != " hole" ]; then #Don't deal with lines that report 'hole'
# compare to bad block region
#For now, check whether the blocks for the file fall within the user-given block range
#if any of the blocks do, then mark this file as bad.

if ( (( "$BAD_BLOCK_BEGINNING" <= "$FIRST_BLOCK")) && (( "$FIRST_BLOCK" <= "$BAD_BLOCK_END")) ); then
# echo "hit first criterium"
BAD_FILE=1
break
elif ( (( "$BAD_BLOCK_BEGINNING" <= "$LAST_BLOCK")) && (( "$LAST_BLOCK" <= "$BAD_BLOCK_END")) ); then
# echo "hit second criterium"
BAD_FILE=1
break
fi
fi
done
# add the file to the list of bad files
if (($BAD_FILE == 1)); then
#Report to the user that the current file is bad
echo "putting file: ${ALL_FILES[$COUNT]} on the naughty list"
#Write the file's name to the list
echo "${ALL_FILES[$COUNT]}" >> naughty_list.txt
fi
done
echo "program_ended_succesfully" >> naughty_list.txt

[-- Attachment #1.2: Type: text/html, Size: 32446 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]