files mapped funny? (related to online defragmentation)

files mapped funny? (related to online defragmentation)

[Eric]

[-- Attachment #1.1: Type: text/plain, Size: 861 bytes --]

Hi,

I'm getting strange results when I map out the blocks used in files
larger than a several thousand KB. I never seem to get any more than
1024 contiguous data blocks in a row. 

Here's a portion of the output of my script when I run it on a 176MB
file in my home directory:
...
Contiguous chunk 67: 2385568 - 2385591  (24 blocks)
Contiguous chunk 68: 2385608 - 2386448  (841 blocks)
Contiguous chunk 69: 2386450 - 2387473  (1024 blocks)
Contiguous chunk 70: 2387475 - 2388498  (1024 blocks)
Contiguous chunk 71: 2388500 - 2389523  (1024 blocks)
...

Maybe this is a bug in my script? Can anyone explain why this would
happen?

I'm attaching my script in case other ext2/3/4 newbies can get any use
out of it, and in case anyone needs to see it in order to answer my
question. It's pretty self-explanatory, though.

Cheers,

Eric


[-- Attachment #1.2: fibmap.py --]
[-- Type: text/x-python, Size: 1611 bytes --]

#!/usr/bin/python

# Eric's first python program :)

import sys #for sys.argv
import fcntl #for fcntl.ioctl
import os #for os.access
import struct # for struct.pack and struct.unpack

print "WARNING: This script may not work for files with holes."

if len(sys.argv) != 2:
	print "Usage: " + sys.argv[0] + " <filename>"
	print "Note: This program uses the FIBMAP ioctl, so you must be root."
	sys.exit(1)

if os.access(sys.argv[1], os.F_OK) == False:
	print "File " + sys.argv[1] + " doesn't exist."
	sys.exit(1)

f = open(sys.argv[1],'r')

fsBlockSize = struct.unpack('i',fcntl.ioctl(f,2,'    '))[0]
numFileBlocks = os.stat(sys.argv[1])[6] / fsBlockSize
if (os.stat(sys.argv[1])[6] % fsBlockSize) != 0:
	numFileBlocks += 1

blockIterator = 0
blockmap = []
for blockIterator in range(numFileBlocks):
	h=struct.pack('i',blockIterator)
	blockmap += struct.unpack('i',fcntl.ioctl(f,1,h))

print "Filesystem block size: " + str(fsBlockSize)
print "Number of filesystem blocks in file: " + str(numFileBlocks)

extentBegin = []
extentEnd = []
extentBegin += [blockmap[0]]
for blockIterator in range(1,len(blockmap)):
	if blockmap[blockIterator]-blockmap[blockIterator-1] == 1:
		blockIterator += 1
		continue
	else:
		extentEnd += [blockmap[blockIterator-1]]
		extentBegin += [blockmap[blockIterator]]
		blockIterator += 1
extentEnd += [blockmap[blockIterator-1]]

for n in range(0,len(extentBegin)):
	print "Contiguous chunk " + str(n) + ": " + str(extentBegin[n]) + " - " + str(extentEnd[n]) + "  (" + str(extentEnd[n] - extentBegin[n]+1) + " blocks)"
sys.exit(0)

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: files mapped funny? (related to online defragmentation)

[Mingming Cao]

On Mon, 2007-05-21 at 06:49 -0700, Eric wrote:
> Hi,
> 
> I'm getting strange results when I map out the blocks used in files
> larger than a several thousand KB. I never seem to get any more than
> 1024 contiguous data blocks in a row. 
> 
> Here's a portion of the output of my script when I run it on a 176MB
> file in my home directory:
> ...
> Contiguous chunk 67: 2385568 - 2385591  (24 blocks)
> Contiguous chunk 68: 2385608 - 2386448  (841 blocks)
> Contiguous chunk 69: 2386450 - 2387473  (1024 blocks)
> Contiguous chunk 70: 2387475 - 2388498  (1024 blocks)
> Contiguous chunk 71: 2388500 - 2389523  (1024 blocks)
> ...
>
> Maybe this is a bug in my script? Can anyone explain why this would
> happen?
> 
filefrag command comes with e2fsprogs will print the file fragmentation
info. I guess you can try filefrag -v command and see if that matches
what your scripts reported.

Mingming

> I'm attaching my script in case other ext2/3/4 newbies can get any use
> out of it, and in case anyone needs to see it in order to answer my
> question. It's pretty self-explanatory, though.
> 
> Cheers,
> 
> Eric
> 

SOLVED: Re: files mapped funny? (related to online defragmentation)

[Eric]

[-- Attachment #1: Type: text/plain, Size: 617 bytes --]

> > I'm getting strange results when I map out the blocks used in files
> > larger than a several thousand KB. I never seem to get any more than
> > 1024 contiguous data blocks in a row. 
> > 
> filefrag command comes with e2fsprogs will print the file fragmentation
> info. I guess you can try filefrag -v command and see if that matches
> what your scripts reported.

Thanks for the pointer. It seems my program only counts data blocks, so
it sees the indirect block for each 1024 data blocks as a space between
extents. I will have to read the filefrag source and account for that.

Cheers,

Eric


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]