Required help for understanding ext4 block allocation

Required help for understanding ext4 block allocation

[Akshay Nehe]

Hello,

I am trying to understand the block allocation in ext4 file sytem, I
have gone through all basic theory and browsing code to do it.

I am writing a module in which i got ext4 on disk inode, now according
to theory first 12 bytes of i_block points to ext4_extent_header, so
can i use this array of i_block[EXT4_N_BLOCKS].

Means how to fill objects of  struct ext4_extent_header (using first
12 byte of i_block) and  struct ext4_extent (using reminder bytes of
i_block).


-- 
Regards,
Akshay Nehe.

Re: Required help for understanding ext4 block allocation

[Jan Kara]

  Hello,

On Thu 15-03-12 18:48:56, Akshay Nehe wrote:
> I am trying to understand the block allocation in ext4 file sytem, I
> have gone through all basic theory and browsing code to do it.
> 
> I am writing a module in which i got ext4 on disk inode, now according
> to theory first 12 bytes of i_block points to ext4_extent_header, so
> can i use this array of i_block[EXT4_N_BLOCKS].
> 
> Means how to fill objects of  struct ext4_extent_header (using first
> 12 byte of i_block) and  struct ext4_extent (using reminder bytes of
> i_block).
  struct ext4_extent_header is just directly stored in the space occupied
by i_block[EXT4_N_BLOCKS]. So for example ext_inode_hdr() does:
  (struct ext4_extent_header *)EXT4_I(inode)->i_data
(where i_data is just one-to-one copy of i_blocks from on disk inode).

And it's the same with struct ext4_extent. See for example macro
EXT_FIRST_EXTENT.

								Honza

-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR

Re: Required help for understanding ext4 block allocation

[Akshay Nehe]

Yes, i got extents as you have suggested, but one thing i found that
each time modification occurs on any file from ext4 file system then
kernel deallocates all current blocks and allocate new blocks and
write modified data on newly allocated blocks.

Why kernel uses this policy? My main aim is to browse ext4 code and
try to get and understand functions which dose the job of allocation
and deallocation of blocks.

So by tracing kernel code i got some useful functions, for
deallocation: ext4_free_blocks, for allocation: ext4_map_blocks, but
found difficult to analyse each of them. Can anyone suggest correct
functions which actually dose allocation and deallocation?

On 3/16/12, Jan Kara <jack@suse.cz> wrote:
>   Hello,
>
> On Thu 15-03-12 18:48:56, Akshay Nehe wrote:
>> I am trying to understand the block allocation in ext4 file sytem, I
>> have gone through all basic theory and browsing code to do it.
>>
>> I am writing a module in which i got ext4 on disk inode, now according
>> to theory first 12 bytes of i_block points to ext4_extent_header, so
>> can i use this array of i_block[EXT4_N_BLOCKS].
>>
>> Means how to fill objects of  struct ext4_extent_header (using first
>> 12 byte of i_block) and  struct ext4_extent (using reminder bytes of
>> i_block).
>   struct ext4_extent_header is just directly stored in the space occupied
> by i_block[EXT4_N_BLOCKS]. So for example ext_inode_hdr() does:
>   (struct ext4_extent_header *)EXT4_I(inode)->i_data
> (where i_data is just one-to-one copy of i_blocks from on disk inode).
>
> And it's the same with struct ext4_extent. See for example macro
> EXT_FIRST_EXTENT.
>
> 								Honza
>
> --
> Jan Kara <jack@suse.cz>
> SUSE Labs, CR
>


-- 
Regards,
Akshay Nehe.

Re: Required help for understanding ext4 block allocation

[Jan Kara]

  Hello,

On Thu 22-03-12 18:54:26, Akshay Nehe wrote:
> Yes, i got extents as you have suggested, but one thing i found that
> each time modification occurs on any file from ext4 file system then
> kernel deallocates all current blocks and allocate new blocks and
> write modified data on newly allocated blocks.
  That is not true. Why do you think it is? Maybe you are modifying the
file with something like a text editor? That certainly truncates the old
file and writes a new one. But generally ext4 supports overwriting.

> So by tracing kernel code i got some useful functions, for
> deallocation: ext4_free_blocks, for allocation: ext4_map_blocks, but
> found difficult to analyse each of them. Can anyone suggest correct
> functions which actually dose allocation and deallocation?
  Ext4 uses multiblock allocator which is in fs/ext4/mballoc.c. In
particular function allocating blocks in ext4_mb_new_blocks(), function
freeing blocks is ext4_free_blocks(). But you will need to understand most
of mballoc.c to understand how mballoc works. You can read comments in the
beginning of the file to understand some basics about mballoc. You can
also read some basics from by presentation about ext4 at Linux Kongress in
2009 (http://www.linux-kongress.org/2009/slides/ext4+btrfs_jan_kara.pdf).

								Honza
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR

Re: Required help for understanding ext4 block allocation

[Akshay Nehe]

On 3/22/12, Jan Kara <jack@suse.cz> wrote:
>   Hello,
>
> On Thu 22-03-12 18:54:26, Akshay Nehe wrote:
>> Yes, i got extents as you have suggested, but one thing i found that
>> each time modification occurs on any file from ext4 file system then
>> kernel deallocates all current blocks and allocate new blocks and
>> write modified data on newly allocated blocks.
>   That is not true. Why do you think it is? Maybe you are modifying the
> file with something like a text editor? That certainly truncates the old
> file and writes a new one. But generally ext4 supports overwriting.
>
Yes, i m using vim editor to modify files. Actually due to this inode
number of file is changing each time after modification.
>> So by tracing kernel code i got some useful functions, for
>> deallocation: ext4_free_blocks, for allocation: ext4_map_blocks, but
>> found difficult to analyse each of them. Can anyone suggest correct
>> functions which actually dose allocation and deallocation?
>   Ext4 uses multiblock allocator which is in fs/ext4/mballoc.c. In
> particular function allocating blocks in ext4_mb_new_blocks(), function
> freeing blocks is ext4_free_blocks(). But you will need to understand most
> of mballoc.c to understand how mballoc works. You can read comments in the
> beginning of the file to understand some basics about mballoc. You can
> also read some basics from by presentation about ext4 at Linux Kongress in
> 2009 (http://www.linux-kongress.org/2009/slides/ext4+btrfs_jan_kara.pdf).
>
> 								Honza
> --
> Jan Kara <jack@suse.cz>
> SUSE Labs, CR
>
Thanks for help, I will go through both mballoc file and your presentation file.

-- 
Regards,
Akshay Nehe.