From mboxrd@z Thu Jan 1 00:00:00 1970 Message-Id: <002401c1ea10$873d91a0$6b01a8c0@PC07> From: "Ralf Zerres" MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0020_01C1EA21.4ABA2CA0" Subject: [linux-lvm] Desater Recovery Sender: linux-lvm-admin@sistina.com Errors-To: linux-lvm-admin@sistina.com Reply-To: linux-lvm@sistina.com List-Help: List-Post: List-Subscribe: , List-Unsubscribe: , List-Archive: Date: Mon Apr 22 10:14:02 2002 List-Id: To: linux-lvm@sistina.com This is a multi-part message in MIME format. ------=_NextPart_000_0020_01C1EA21.4ABA2CA0 Content-Type: multipart/alternative; boundary="----=_NextPart_001_0021_01C1EA21.4ABBB340" ------=_NextPart_001_0021_01C1EA21.4ABBB340 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable hello everybody, on a production system we do have a serious data destruction problem.=20 Any help leading us to restore the filesystem will be honored. We can = dicuss financial details later. First I'd like to describe the situation giving you a picture wether it = is possible to restore the data. Please answer as soon as possible to give me a hint wether I should wait = for a detailed analysis, if it make sense to dig deeper or if it is more likely that people have = to reinsert the data from printed=20 information. 0. PROBLEM: =3D=3D=3D=3D=3D=3D=3D=3D=3D Destructed Filesystem after resizing LVG running a Postgres Database.=20 ### # There is no actual backup, which is the mad point!!!!=20 ###=20 24 days of DATA updates are lost right now! I. Environment =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D System: Dual Prozessor PIII (2 CPU's 1Mhz) OS: Linux 2.4.9=20 Harddisk: SCSI-3 (as Raid-5) Controller: GDT=20 LVM version: 1.0.1-rc2(30/08/2001) Kernel sees one harddrive (/dev/scsi/host2/bus0/target0) Root is on standard ext3 fs (partition 5) LVM has a Physical Volumegroup (KI) on partition 7 The system runs a PostgreSQL Database (version 7.1.3). The = Data-Structures live on a Logical-Volume-Group which was mounted as /var/lib/postgres as a ext3 filesystem. Everything was working until 4 hours ago, before LVG was resized. II. Changes The LVG /dev/ki/postgres was reduced from 8 GB to 800MB. Before we did = that a df has shown a used size of 366 MB data. Postgres was shut down. No other process was writing to = /dev/ki/postgres. # e2fsadm --size - 4,5G=20 Worked out, but remounting has shown incorrect structure. A new LVG was created=20 # lvcreate --size 2,5G --name opt /dev/ki/opt Here the dump-information from the actual Volume-Group after resizing = was successful. ----------------------------------------------schnipp / schnapp = ------------------------------------------------ #vgdisplay -v /dev/ki --- Volume group --- VG Name ki VG Access read/write VG Status available/resizable VG # 0 MAX LV 256 Cur LV 7 Open LV 6 MAX LV Size 255.99 GB Max PV 256 Cur PV 1 Act PV 1 VG Size 48.77 GB PE Size 4 MB Total PE 12484 Alloc PE / Size 11968 / 46.75 GB Free PE / Size 516 / 2.02 GB VG UUID P6LodN-bQUg-u5WF-03Cu-A22a-UUhE-NsKhvZ --- Logical volume --- LV Name /dev/ki/export VG Name ki LV Write Access read/write LV Status available LV # 1 # open 1 LV Size 6 GB Current LE 1536 Allocated LE 1536 Allocation next free Read ahead sectors 120 Block device 58:0 --- Logical volume --- LV Name /dev/ki/var VG Name ki LV Write Access read/write LV Status available LV # 2 # open 1 LV Size 1 GB Current LE 256 Allocated LE 256 Allocation next free Read ahead sectors 120 Block device 58:1 --- Logical volume --- LV Name /dev/ki/tmp VG Name ki LV Write Access read/write LV Status available LV # 3 # open 1 LV Size 1 GB Current LE 256 Allocated LE 256 Allocation next free Read ahead sectors 120 Block device 58:2 --- Logical volume --- LV Name /dev/ki/home VG Name ki LV Write Access read/write LV Status available LV # 4 # open 1 LV Size 30 GB Current LE 7680 Allocated LE 7680 Allocation next free Read ahead sectors 120 Block device 58:3 --- Logical volume --- LV Name /dev/ki/postgres VG Name ki LV Write Access read/write LV Status available LV # 5 # open 0 LV Size 3.78 GB Current LE 968 Allocated LE 968 Allocation next free Read ahead sectors 120 Block device 58:4 --- Logical volume --- LV Name /dev/ki/www VG Name ki LV Write Access read/write LV Status available LV # 6 # open 1 LV Size 2.44 GB Current LE 625 Allocated LE 625 Allocation next free Read ahead sectors 120 Block device 58:5 --- Logical volume --- LV Name /dev/ki/opt VG Name ki LV Write Access read/write LV Status available LV # 7 # open 1 LV Size 2.53 GB Current LE 647 Allocated LE 647 Allocation next free Read ahead sectors 120 Block device 58:6 --- Physical volumes --- PV Name (#) /dev/scsi/host2/bus0/target0/lun0/part7 (1) PV Status available / allocatable Total PE / Free PE 12484 / 516 Volume-Group: KI ----------------------------------------------schnipp / schnapp = ------------------------------------------------ The sysadmin was running a vgscan an lvscan. Afterwards he runs=20 # e2fsck -y -f /dev/ki/postgres now the mount on /var/lib/postgres was successful, but all date were = gone! I found a long list of dir-entries in the lost+found subdir. No = file-entries. What can we do? Ralf ------=_NextPart_001_0021_01C1EA21.4ABBB340 Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
hello everybody,
 
on a production system we do have a = serious data=20 destruction problem.
Any help leading us to restore the = filesystem will=20 be honored. We can dicuss financial details later.
First I'd like to describe the = situation giving you=20 a picture wether it is possible to restore the data.
 
Please answer as soon as possible to = give me a hint=20 wether I should wait for a detailed analysis,
if it make sense to dig deeper or if it = is more=20 likely that people have to reinsert the data from printed
information.
 
0. PROBLEM:
=3D=3D=3D=3D=3D=3D=3D=3D=3D
Destructed Filesystem after resizing = LVG running a=20 Postgres Database.
 
###
# There is no actual backup, which is = the mad=20 point!!!!
###
 
24 days of DATA updates are lost right=20 now!
 
I. Environment
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
 
System:    Dual = Prozessor PIII (2=20 CPU's 1Mhz)
OS:    Linux 2.4.9 =
 
Harddisk:    SCSI-3 (as=20 Raid-5)
Controller:    GDT =
LVM version:    =20 1.0.1-rc2(30/08/2001)
 
Kernel sees one harddrive=20 (/dev/scsi/host2/bus0/target0)
 
Root is on standard ext3 fs (partition=20 5)
 
LVM has a Physical Volumegroup (KI)=20 on partition 7
The system runs a PostgreSQL Database = (version=20 7.1.3). The Data-Structures live on a Logical-Volume-Group
which was mounted as = /var/lib/postgres as a=20 ext3 filesystem.
Everything was working until 4 = hours ago,=20 before LVG was resized.
 
II. Changes
 
The LVG /dev/ki/postgres was reduced = from 8 GB to=20 800MB. Before we did that a df has shown a used size
of 366 MB data.
Postgres was shut down. No other = process was=20 writing to /dev/ki/postgres.
 
     # e2fsadm = --size - 4,5G=20
 
Worked out, but remounting has shown = incorrect=20 structure.
 
A new LVG was created
 
    # = lvcreate   =20 --size 2,5G --name opt /dev/ki/opt
 
Here the dump-information from the = actual=20 Volume-Group after resizing was successful.
 
----------------------------------------------schnipp / schnapp = ------------------------------------------------
 
 
#vgdisplay -v /dev/ki
--- = Volume group=20 ---
VG=20 Name           &nb= sp;  =20 ki
VG=20 Access           &= nbsp;=20 read/write
VG=20 Status           &= nbsp;=20 available/resizable
VG=20 #            =      =20 0
MAX=20 LV            = ;   =20 256
Cur=20 LV            = ;   =20 7
Open=20 LV            = ;  =20 6
MAX LV = Size          =20 255.99 GB
Max=20 PV            = ;   =20 256
Cur=20 PV            = ;   =20 1
Act=20 PV            = ;   =20 1
VG=20 Size           &nb= sp;  =20 48.77 GB
PE=20 Size           &nb= sp;  =20 4 MB
Total=20 PE            = ; =20 12484
Alloc PE / Size       11968 / = 46.75=20 GB
Free  PE / Size       516 / = 2.02=20 GB
VG=20 UUID           &nb= sp;  =20 P6LodN-bQUg-u5WF-03Cu-A22a-UUhE-NsKhvZ
 
--- Logical volume ---
LV=20 Name           &nb= sp;   =20 /dev/ki/export
VG=20 Name           &nb= sp;   =20 ki
LV Write Access        = read/write
LV=20 Status           &= nbsp; =20 available
LV=20 #            =       =20 1
#=20 open           &nb= sp;    =20 1
LV=20 Size           &nb= sp;   =20 6 GB
Current=20 LE            = ;=20 1536
Allocated = LE          =20 1536
Allocation         &= nbsp;  =20 next free
Read ahead sectors     120
Block=20 device          =20 58:0
 
--- Logical volume ---
LV=20 Name           &nb= sp;   =20 /dev/ki/var
VG=20 Name           &nb= sp;   =20 ki
LV Write Access        = read/write
LV=20 Status           &= nbsp; =20 available
LV=20 #            =       =20 2
#=20 open           &nb= sp;    =20 1
LV=20 Size           &nb= sp;   =20 1 GB
Current=20 LE            = ;=20 256
Allocated = LE          =20 256
Allocation         &n= bsp;  =20 next free
Read ahead sectors     120
Block=20 device          =20 58:1
 
--- Logical volume ---
LV=20 Name           &nb= sp;   =20 /dev/ki/tmp
VG=20 Name           &nb= sp;   =20 ki
LV Write Access        = read/write
LV=20 Status           &= nbsp; =20 available
LV=20 #            =       =20 3
#=20 open           &nb= sp;    =20 1
LV=20 Size           &nb= sp;   =20 1 GB
Current=20 LE            = ;=20 256
Allocated = LE          =20 256
Allocation         &n= bsp;  =20 next free
Read ahead sectors     120
Block=20 device          =20 58:2
 
--- Logical volume ---
LV=20 Name           &nb= sp;   =20 /dev/ki/home
VG=20 Name           &nb= sp;   =20 ki
LV Write Access        = read/write
LV=20 Status           &= nbsp; =20 available
LV=20 #            =       =20 4
#=20 open           &nb= sp;    =20 1
LV=20 Size           &nb= sp;   =20 30 GB
Current=20 LE            = ;=20 7680
Allocated = LE          =20 7680
Allocation         &= nbsp;  =20 next free
Read ahead sectors     120
Block=20 device          =20 58:3
 
--- Logical volume ---
LV=20 Name           &nb= sp;   =20 /dev/ki/postgres
VG=20 Name           &nb= sp;   =20 ki
LV Write Access        = read/write
LV=20 Status           &= nbsp; =20 available
LV=20 #            =       =20 5
#=20 open           &nb= sp;    =20 0
LV=20 Size           &nb= sp;   =20 3.78 GB
Current=20 LE            = ;=20 968
Allocated = LE          =20 968
Allocation         &n= bsp;  =20 next free
Read ahead sectors     120
Block=20 device          =20 58:4
 
--- Logical volume ---
LV=20 Name           &nb= sp;   =20 /dev/ki/www
VG=20 Name           &nb= sp;   =20 ki
LV Write Access        = read/write
LV=20 Status           &= nbsp; =20 available
LV=20 #            =       =20 6
#=20 open           &nb= sp;    =20 1
LV=20 Size           &nb= sp;   =20 2.44 GB
Current=20 LE            = ;=20 625
Allocated = LE          =20 625
Allocation         &n= bsp;  =20 next free
Read ahead sectors     120
Block=20 device          =20 58:5
 
--- Logical volume ---
LV=20 Name           &nb= sp;   =20 /dev/ki/opt
VG=20 Name           &nb= sp;   =20 ki
LV Write Access        = read/write
LV=20 Status           &= nbsp; =20 available
LV=20 #            =       =20 7
#=20 open           &nb= sp;    =20 1
LV=20 Size           &nb= sp;   =20 2.53 GB
Current=20 LE            = ;=20 647
Allocated = LE          =20 647
Allocation         &n= bsp;  =20 next free
Read ahead sectors     120
Block=20 device          =20 58:6
 

--- Physical volumes ---
PV Name = (#)          =20 /dev/scsi/host2/bus0/target0/lun0/part7 (1)
PV=20 Status           &= nbsp;=20 available / allocatable
Total PE / Free PE    12484 /=20 516
Volume-Group: KI
 
----------------------------------------------schnipp / schnapp = ------------------------------------------------
 
 
The sysadmin was running a vgscan an lvscan. Afterwards he runs =
 
    # e2fsck -y -f /dev/ki/postgres
 
now the mount on /var/lib/postgres was successful, but all date = were=20 gone!
I found a long list of dir-entries in the lost+found subdir. No=20 file-entries.
 
What can we do?
 
Ralf
 
------=_NextPart_001_0021_01C1EA21.4ABBB340-- ------=_NextPart_000_0020_01C1EA21.4ABA2CA0 Content-Type: text/x-vcard; name="Ralf, Zerres.vcf" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="Ralf, Zerres.vcf" BEGIN:VCARD VERSION:2.1 N:Ralf;Zerres FN:Ralf, Zerres ORG:Networkx GmbH TEL;WORK;VOICE:+49 221 937725 - 0 TEL;WORK;FAX:+49 221 937725 - 18 ADR;WORK:;;Marktstr. 8;K=F6ln;;50968 LABEL;WORK;ENCODING=3DQUOTED-PRINTABLE:Marktstr. 8=3D0D=3D0AK=3DF6ln = 50968 EMAIL;PREF;INTERNET:rzerres@networkx.de REV:20020422T151525Z END:VCARD ------=_NextPart_000_0020_01C1EA21.4ABA2CA0--