Recovering folder and files from ext4 partition

Recovering folder and files from ext4 partition

[Bhatia Amit]

Hi

I have a WD Live duo 2*3TB drives, which was setup in RAID1 mirror configuration. The unit failed with one drive totally inaccessible. I pulled the second drive out and connected it to a Linux laptop via a standalone enclosure via sata.

Using a r-linux scan tool on fourth partition sdc4, it shows me files that are relevant to me, but has no folder structure information. So, getting folder information out from the disk would be extremely valuable. Also, the files seem to have lost their original name.

When trying to start the array on the disk, I was unable to start it, and described the whole problem on linux-raid mailing list.
http://marc.info/?l=linux-raid&m=146878562322359&w=2

I got lot of help from Phil on the raid list, to decipher the issue. The last update today was that apparently the disk has some very old (not relevant to me) data and some new (relevant) data. The superblocks in start of the disk seem to refer to 2012 data while that last superblock refers to 2016 data. Phil thinks that folder information for the new relevant data is in the first half of the disk, but that folder information is missing. He suggested describe my problem on this list, to get any advice. I have repeated the superblock information here to start this discussion. Out of the many superblocks that r-linux found on disk, I have pasted the first, middle'ish and last superblock content here. I can paste other superblocks too based on any suggestion.

Any suggestion on getting file and folder information from this disk would be very helpful.


Thanks
Amit

$ cat sblock.txt

Name: Ext2/Ext3/Ext4 SuperBlock
Offset (in sectors): 8387584
Size (in sectors): 2


Sector 8387584 (Parent: WDC WD30EZRX-00D8PB0 80.00A80 Record: 17419264)

FFF80000:  00 46 B7 02 5F 8C B9 02 - 00 00 00 00 62 25 B6 02  .F.._.......b%..
FFF80010:  1A 3B B7 02 00 00 00 00 - 06 00 00 00 06 00 00 00  .;..............
FFF80020:  F8 FF 00 00 F8 FF 00 00 - 00 FF 00 00 43 3E B3 4F  ............C>.O
FFF80030:  47 9D BA 50 03 00 1B 00 - 53 EF 00 00 01 00 00 00  G..P....S.......
FFF80040:  3B B5 02 00 00 4E ED 00 - 00 00 00 00 01 00 00 00  ;....N..........
FFF80050:  00 00 00 00 0B 00 00 00 - 00 01 01 00 3C 00 00 00  ............<...
FFF80060:  42 02 00 00 7B 00 00 00 - B6 7F 9F FD 12 BF 4D 57  B...{.........MW
FFF80070:  A8 34 DD 09 E9 E8 BA C0 - 00 00 00 00 00 00 00 00  .4..............
FFF80080:  00 00 00 00 00 00 00 00 - 2F 43 61 63 68 65 56 6F  ......../CacheVo
FFF80090:  6C 75 6D 65 00 C8 24 9E - 20 22 82 24 22 C8 1B 60  lume..$. ".$"..`
FFF800A0:  60 FF FF FF 9C CA D0 E4 - 5C C8 13 41 80 C0 10 E6  `.......\..A....
FFF800B0:  68 C8 08 21 60 CA D0 E4 - 5C 00 00 00 00 CD AC 46  h..!`...\......F
FFF800C0:  80 C8 1C 5A 60 C8 24 9E - 00 00 00 00 00 00 20 00  ...Z`.$....... .
FFF800D0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
FFF800E0:  08 00 00 00 00 00 00 00 - 00 00 00 00 FC ED 10 2F  .............../
FFF800F0:  38 44 66 85 D0 AC 7C A3 - FF F3 B0 AF 01 01 00 00  8Df...|.........
FFF80100:  00 00 00 00 00 00 00 00 - 3B B5 02 00 0A F3 02 00  ........;.......
FFF80110:  04 00 00 00 00 00 00 00 - 00 00 00 00 FF 7F 00 00  ................
FFF80120:  78 EA B0 02 FF 7F 00 00 - 01 00 00 00 77 6A B1 02  x...........wj..
FFF80130:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
FFF80140:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 80  ................
FFF80150:  00 00 00 00 00 00 00 00 - 00 00 00 00 1C 00 1C 00  ................
FFF80160:  02 00 00 00 FF 78 00 00 - 00 00 00 00 00 00 00 00  .....x..........
FFF80170:  00 00 00 00 04 00 00 00 - 94 F3 87 01 00 00 00 00  ................
FFF80180:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
FFF80190:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
FFF801A0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
FFF801B0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
FFF801C0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
FFF801D0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
FFF801E0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
FFF801F0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................



Name: Ext2/Ext3/Ext4 SuperBlock
Offset (in sectors): 2038182912
Size (in sectors): 2


Sector 2038182912 (Parent: WDC WD30EZRX-00D8PB0 80.00A80 Record: 2047214592)


F2F8680000:  00 46 B7 02 5F 8C B9 02 - 00 00 00 00 62 25 B6 02  .F.._.......b%..
F2F8680010:  1A 3B B7 02 00 00 00 00 - 06 00 00 00 06 00 00 00  .;..............
F2F8680020:  F8 FF 00 00 F8 FF 00 00 - 00 FF 00 00 43 3E B3 4F  ............C>.O
F2F8680030:  47 9D BA 50 03 00 1B 00 - 53 EF 00 00 01 00 00 00  G..P....S.......
F2F8680040:  3B B5 02 00 00 4E ED 00 - 00 00 00 00 01 00 00 00  ;....N..........
F2F8680050:  00 00 00 00 0B 00 00 00 - 00 01 F3 00 3C 00 00 00  ............<...
F2F8680060:  42 02 00 00 7B 00 00 00 - B6 7F 9F FD 12 BF 4D 57  B...{.........MW
F2F8680070:  A8 34 DD 09 E9 E8 BA C0 - 00 00 00 00 00 00 00 00  .4..............
F2F8680080:  00 00 00 00 00 00 00 00 - 2F 43 61 63 68 65 56 6F  ......../CacheVo
F2F8680090:  6C 75 6D 65 00 C8 24 9E - 20 22 82 24 22 C8 1B 60  lume..$. ".$"..`
F2F86800A0:  60 FF FF FF 9C CA D0 E4 - 5C C8 13 41 80 C0 10 E6  `.......\..A....
F2F86800B0:  68 C8 08 21 60 CA D0 E4 - 5C 00 00 00 00 CD AC 46  h..!`...\......F
F2F86800C0:  80 C8 1C 5A 60 C8 24 9E - 00 00 00 00 00 00 20 00  ...Z`.$....... .
F2F86800D0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
F2F86800E0:  08 00 00 00 00 00 00 00 - 00 00 00 00 FC ED 10 2F  .............../
F2F86800F0:  38 44 66 85 D0 AC 7C A3 - FF F3 B0 AF 01 01 00 00  8Df...|.........
F2F8680100:  00 00 00 00 00 00 00 00 - 3B B5 02 00 0A F3 02 00  ........;.......
F2F8680110:  04 00 00 00 00 00 00 00 - 00 00 00 00 FF 7F 00 00  ................
F2F8680120:  78 EA B0 02 FF 7F 00 00 - 01 00 00 00 77 6A B1 02  x...........wj..
F2F8680130:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
F2F8680140:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 80  ................
F2F8680150:  00 00 00 00 00 00 00 00 - 00 00 00 00 1C 00 1C 00  ................
F2F8680160:  02 00 00 00 FF 78 00 00 - 00 00 00 00 00 00 00 00  .....x..........
F2F8680170:  00 00 00 00 04 00 00 00 - 94 F3 87 01 00 00 00 00  ................
F2F8680180:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
F2F8680190:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
F2F86801A0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
F2F86801B0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
F2F86801C0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
F2F86801D0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
F2F86801E0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
F2F86801F0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................



Name: Ext2/Ext3/Ext4 SuperBlock
Offset (in sectors): 5781334786
Size (in sectors): 2



Sector 5781334786 (Parent: WDC WD30EZRX-00D8PB0 80.00A80 Record: 5790366466)


2B130560400:  00 46 B7 02 5F 8C B9 02 - 00 00 00 00 25 7F 09 02  .F.._.......%...
2B130560410:  DC F8 99 02 00 00 00 00 - 06 00 00 00 06 00 00 00  ................
2B130560420:  F8 FF 00 00 F8 FF 00 00 - 00 FF 00 00 6D C8 02 57  ............m..W
2B130560430:  6D C8 02 57 2E 00 1B 00 - 53 EF 01 00 01 00 00 00  m..W....S.......
2B130560440:  3B B5 02 00 00 4E ED 00 - 00 00 00 00 01 00 00 00  ;....N..........
2B130560450:  00 00 00 00 0B 00 00 00 - 00 01 00 00 3C 00 00 00  ............<...
2B130560460:  46 02 00 00 7B 00 00 00 - B6 7F 9F FD 12 BF 4D 57  F...{.........MW
2B130560470:  A8 34 DD 09 E9 E8 BA C0 - 00 00 00 00 00 00 00 00  .4..............
2B130560480:  00 00 00 00 00 00 00 00 - 2F 43 61 63 68 65 56 6F  ......../CacheVo
2B130560490:  6C 75 6D 65 00 C7 4B BE - 20 24 82 24 82 C8 1B A0  lume..K. $.$....
2B1305604A0:  60 FF FF FF 9C C0 F1 11 - DC C8 13 2A A0 C0 10 E6  `..........*....
2B1305604B0:  68 C8 08 21 60 C0 F1 11 - DC 00 00 00 00 CF 3D 2A  h..!`.........=*
2B1305604C0:  E0 C2 94 C1 60 C7 4B BE - 00 00 00 00 00 00 20 00  ....`.K....... .
2B1305604D0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
2B1305604E0:  08 00 00 00 00 00 00 00 - 00 00 00 00 FC ED 10 2F  .............../
2B1305604F0:  38 44 66 85 D0 AC 7C A3 - FF F3 B0 AF 01 01 00 00  8Df...|.........
2B130560500:  00 00 00 00 00 00 00 00 - 3B B5 02 00 0A F3 02 00  ........;.......
2B130560510:  04 00 00 00 00 00 00 00 - 00 00 00 00 FF 7F 00 00  ................
2B130560520:  78 EA B0 02 FF 7F 00 00 - 01 00 00 00 77 6A B1 02  x...........wj..
2B130560530:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
2B130560540:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 80  ................
2B130560550:  00 00 00 00 00 00 00 00 - 00 00 00 00 1C 00 1C 00  ................
2B130560560:  02 00 00 00 FF 78 00 00 - 00 00 00 00 00 00 00 00  .....x..........
2B130560570:  00 00 00 00 04 00 00 00 - D4 DD EB 54 00 00 00 00  ...........T....
2B130560580:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
2B130560590:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
2B1305605A0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
2B1305605B0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
2B1305605C0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
2B1305605D0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
2B1305605E0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................
2B1305605F0:  00 00 00 00 00 00 00 00 - 00 00 00 00 00 00 00 00  ................

Re: Recovering folder and files from ext4 partition

[Theodore Ts'o]

So the thing about superblock write times is that the backup
superblocks are only written (a) at mkfs time, and (b) when the file
system has been damaged enough that e2fsck needs to do repair work.
So the fact that the last write times in the superblock are different
doesn't mean that they are from different file systems.

In fact, the UUID for the file system is located at offset 0x68 from
the beginning of the superblock, and these seem to be the same across
your three superblocks:

 FFF80060:  42 02 00 00 7B 00 00 00 - B6 7F 9F FD 12 BF 4D 57  B...{.........MW
 FFF80070:  A8 34 DD 09 E9 E8 BA C0 - 00 00 00 00 00 00 00 00  .4.............

So the UUID is b67f9ffd-12bf-4d57-a834-dd09e9e8bac0

The odd thing is that the file system creation time (located at
offset 0x108) is also the same across all three superblocks, but it's
Fri Jan 2 20:17:47 EST 1970.  The only thing I can think of is that
the file system was actually created in the factory, before the time
was properly set.

However, the *really* weird thing is that the superblock with the
recent last write time is at sector 5781334786 --- and normally the
superblock where the primary superblock is located is the first
superblock.  Fortunately, we can figure out where this is by looking
at s_block_group_nr, located at offset 0x5A.   Across your superblocks:

Sector #     Block group
==========================
8387584      1
2038182912   243
5781334786   0

This is consistent with the superblock from 5781334786 having the
updated timestamp.  Unfortunately, it also means that the partition
was *not* laid out sequentially across your disk.  I'm guessing that
WD Live Duo is using some kind of LVM scheme, and somehow the sectors
have gotten allocated in a very odd way indeed, perhaps because of how
disks were added and removed from your system.  (When you replaced the
disks, were they by any chance different sizes from the disk that they
replaced?)

I'm not sure what to tell you at this point.  If you want to try to
get at least some data off your drive, we can hope the files you care
about where laid out w/o sequentially, and maybe the photorec program
(originally designed to pull images off of failed SD cards, but it now
recognizes some 300+ different file formats) can pull some data off
the disk.

But unless you can figure out how to get the blocks reconstructed in
the correct order, you're not going to be able to mount it or
otherwise get any data off of it using file system recovery tools....

Good luck,

						- Ted

Re: Recovering folder and files from ext4 partition

[Bhatia Amit]

Hi Ted,


Pardon my ignorance here, in some replies.

> So the thing about superblock write times is that the backup

> superblocks are only written (a) at mkfs time, and (b) when the file

> system has been damaged enough that e2fsck needs to do repair work.
> So the fact that the last write times in the superblock are different

> doesn't mean that they are from different file systems.

I had in fact tried running e2fsck on sdc4 earlier this month, based on some forum post I was viewing, but obviously that didn't work:
$ sudo e2fsck /dev/sdc4
e2fsck 1.42.9 (4-Feb-2014)
ext2fs_open2: Bad magic number in super-block
e2fsck: Superblock invalid, trying backup blocks...
e2fsck: Bad magic number in super-block while trying to open /dev/sdc4

The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem.  If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>
or
e2fsck -b 32768 <device>

$ 


> In fact, the UUID for the file system is located at offset 0x68 from
> the beginning of the superblock, and these seem to be the same across
> your three superblocks:
> 
> FFF80060:  42 02 00 00 7B 00 00 00 - B6 7F 9F FD 12 BF 4D 57  B...{.........MW
> FFF80070:  A8 34 DD 09 E9 E8 BA C0 - 00 00 00 00 00 00 00 00  .4.............
> 
> So the UUID is b67f9ffd-12bf-4d57-a834-dd09e9e8bac0


I assume this is a good thing that UUID is same, i.e. implies all the 3 superblocks are from same filesystem.

> The odd thing is that the file system creation time (located at
> offset 0x108) is also the same across all three superblocks, but it's
> Fri Jan 2 20:17:47 EST 1970.  The only thing I can think of is that
> the file system was actually created in the factory, before the time

> was properly set.

This disk was a RMA disk that I had received from WDC, to replace a failed disk earlier this year (Feb 2016), although that might still not explain the odd filesystem creation time of Jan 2 1970.

> However, the *really* weird thing is that the superblock with the
> recent last write time is at sector 5781334786 --- and normally the
> superblock where the primary superblock is located is the first
> superblock.  Fortunately, we can figure out where this is by looking
> at s_block_group_nr, located at offset 0x5A.   Across your superblocks:
> 
> Sector #     Block group
> ==========================
> 8387584      1
> 2038182912   243

> 5781334786   0

So, is the superblock with smallest value of s_block_group_nr is the primary superblock, i.e. at sector 5781334786 in this case ?



> This is consistent with the superblock from 5781334786 having the
> updated timestamp.  Unfortunately, it also means that the partition
> was *not* laid out sequentially across your disk.  I'm guessing that
> WD Live Duo is using some kind of LVM scheme, and somehow the sectors
> have gotten allocated in a very odd way indeed, perhaps because of how
> disks were added and removed from your system.  (When you replaced the
> disks, were they by any chance different sizes from the disk that they

> replaced?)

The disks in the WD enclosure were always 3TB disks. So, when you say "not laid out sequentially", it implies you would have expected the first superblock to be the primary superblock and with latest write time, but this disk kind of shows them in reverse order?

> I'm not sure what to tell you at this point.  If you want to try to
> get at least some data off your drive, we can hope the files you care
> about where laid out w/o sequentially, and maybe the photorec program
> (originally designed to pull images off of failed SD cards, but it now
> recognizes some 300+ different file formats) can pull some data off

> the disk.

Yes, the files are laid out sequentially in the sense that when I clicked on one of the PNG files recovered by r-linux, it showed me the correct photo, i.e not corrupted or anything. The missing piece is the true file name and its folder information. Right now I have a very large number of files that possibly r-linux can recover, but I cannot have them sorted into folders.

> But unless you can figure out how to get the blocks reconstructed in
> the correct order, you're not going to be able to mount it or
> otherwise get any data off of it using file system recovery tools....


I am not sure I understand this. So, can I write a program to reconstruct blocks in the correct order, in memory, and then write these correct blocks to a new disk? If so, can you suggest some existing code on how to get started in this direction? Are there tools to scan a disk for superblock locations and give them in sorted order? Also, how do I confirm whether this is an ext4 or ext3 filesystem?




Thanks,
Amit

Re: Recovering folder and files from ext4 partition

[Theodore Ts'o]

On Sun, Jul 24, 2016 at 08:04:25AM +0000, Bhatia Amit wrote:
> > But unless you can figure out how to get the blocks reconstructed in
> > the correct order, you're not going to be able to mount it or
> > otherwise get any data off of it using file system recovery tools....
> 
> I am not sure I understand this. So, can I write a program to
> reconstruct blocks in the correct order, in memory, and then write
> these correct blocks to a new disk? If so, can you suggest some
> existing code on how to get started in this direction? Are there
> tools to scan a disk for superblock locations and give them in
> sorted order? Also, how do I confirm whether this is an ext4 or ext3
> filesystem?

Given that the first (primary) superblock is near the *end* of the
disk, it means that the file systems blocks are arranged in some
arbitrary order.  e.g., in the normal course of events, you might
expect the blocks to be:

AAAAAAAAAAAAABBBBBBBBBBBBBCCCCCCCCCCCCCDDDDDDDDDDEEEEEEEEEEFFFFFFFFFF

but instead, the blocks might be arranged something like this:

DDDDDDDDDDBBBBBBBBBBBBBEEEEEEEEEEFFFFFFFFFFCCCCCCCCCCCCCAAAAAAAAAAAAA

This is just an example, but this is what I meant by the file system
blocks are not located sequentially across the disk.  The block group
number in the superblocks can help you a little bit, but it's not
enough necessarily to figure out where the boundaries are between
BBBBBBBBB and EEEEEE (for example).  So no, strictly speaking, you
can't just "reconstruct the blocks in the correct order".  It might be
possible for a human, doing a lot of forensics, to make some guess
about where the boundaries are, but it would require a lot of expert
human intuition.  It would probably be easier to try to see if you can
somehow extract the LVM metadata (if you can find it) which gives you
the mapping between the logical block numbers and the physical blocks
on disk.

Here is an example of an LVM map which is laid out completely
sequentially:

# lvdisplay -m /dev/lambda/library
  --- Logical volume ---
  LV Path                /dev/lambda/library
  LV Name                library
  VG Name                lambda
  LV UUID                KvM2gK-k9dQ-iB9z-kzdc-DhYq-L02J-MWC02l
  LV Write Access        read/write
  LV Creation host, time closure, 2015-07-13 23:27:41 -0400
  LV Status              available
  # open                 0
  LV Size                20.00 GiB
  Current LE             5120
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           252:13
   
  --- Segments ---
  Logical extents 0 to 5119:
    Type		linear
    Physical volume	/dev/sda4
    Physical extents	103680 to 108799
   
Unfortunately I don't have a fragmented LVM volume to show you, but if
I did, there would be multiple segments listed below, and you would
see (for example) maybe something that looked like this:

  Logical extents 0 to 1000:
    Type		linear
    Physical volume	/dev/sda4
    Physical extents	10368 to 11368
  Logical extents 1001 to 5119:
    Type		linear
    Physical volume	/dev/sda4
    Physical extents	100 to 4218

In this example, see how the physical extents for logical extents 1001
-- 5119 are lower than the physical extents for 0 -- 1000?  In this
particular case, my volume group has a physical extent of 4MB:

# vgdisplay lambda
  --- Volume group ---
  VG Name               lambda
  System ID             
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  25
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                5
  Open LV               2
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               701.51 GiB
  PE Size               4.00 MiB   <----------------
  Total PE              179587
  Alloc PE / Size       108800 / 425.00 GiB
  Free  PE / Size       70787 / 276.51 GiB
  VG UUID               5VlKAW-Yahk-fqJz-1eK9-jQ8P-4xt7-gofvh6
   
And so to get from an extent number to an lba number, you'd take the
physical extent number, multiply by 4MB, and then divide by 512 (the
sector size) to get the lba number.

The problem is knowing the mapping from logical extent to physical
extent.  And figuring out what the physical extent size might be.

And all of this is assuming that the WD Duo Live is using the stock
standard LVM system, as opposed to some proprietary cr*p....

	     	     		   		    - Ted

Re: Recovering folder and files from ext4 partition

[Bhatia Amit]

> The problem is knowing the mapping from logical extent to physical
> extent.  And figuring out what the physical extent size might be.
> 
> And all of this is assuming that the WD Duo Live is using the stock
> standard LVM system, as opposed to some proprietary cr*p....



Hi Ted,

Looks like I am out of luck in getting folder information out of this disk. I will try figuring out if I can get any LVM information from disk. If so, I will get back on this post. Until then, thanks for all the help and knowledge.

Thanks,
Amit