Creating mkfs for my custom filesystem

Creating mkfs for my custom filesystem

[Sankar P]

Hi,

I am trying to write a simple filesystem to learn the basics of it.

I have decided on a simple layout for my filesystem where the first
block will be the super block and   will contain the version
information etc. The second block will contain the list of inodes.
Third block onwards will be data blocks. Each file can grow only up to
a single block size. Thrid block will represent the first file, fourth
block for the second file and so on. Directories will not be
supported.

Now I want to create a mkfs for my filesystem as mentioned above. But
I am not able to find out how to do the mkfs for my filesystem such
that the generic mkfs utility will understand my filesystem. What APIs
should I be using ?

Any help is appreciated. Thanks.

--
Sankar P
http://psankar.blogspot.com

Creating mkfs for my custom filesystem

[Tobias Boege]

On Fri, 29 Mar 2013, Sankar P wrote:
> Hi,
> 
> I am trying to write a simple filesystem to learn the basics of it.
> 
> I have decided on a simple layout for my filesystem where the first
> block will be the super block and   will contain the version
> information etc. The second block will contain the list of inodes.
> Third block onwards will be data blocks. Each file can grow only up to
> a single block size. Thrid block will represent the first file, fourth
> block for the second file and so on. Directories will not be
> supported.
> 
> Now I want to create a mkfs for my filesystem as mentioned above. But
> I am not able to find out how to do the mkfs for my filesystem such
> that the generic mkfs utility will understand my filesystem. What APIs
> should I be using ?
> 
> Any help is appreciated. Thanks.

According to my copy of the mkfs sources, you just have to create a program
named "mkfs.ID" where ID identifies your filesystem. Then put that program
in a location that the generic mkfs can find, i.e. under $PATH (mkfs seems
to make some additions to PATH but you should figure this out yourself).

Finally, calling "mkfs -t ID" makes mkfs search for a program named
"mkfs.ID" - simple concatenation.

Regards,
Tobi

Creating mkfs for my custom filesystem

[Sankar P]

On Fri, Mar 29, 2013 at 3:28 PM, Tobias Boege <tobias@gambas-buch.de> wrote:
> On Fri, 29 Mar 2013, Sankar P wrote:
>> Hi,
>>
>> I am trying to write a simple filesystem to learn the basics of it.
>>
>> I have decided on a simple layout for my filesystem where the first
>> block will be the super block and   will contain the version
>> information etc. The second block will contain the list of inodes.
>> Third block onwards will be data blocks. Each file can grow only up to
>> a single block size. Thrid block will represent the first file, fourth
>> block for the second file and so on. Directories will not be
>> supported.
>>
>> Now I want to create a mkfs for my filesystem as mentioned above. But
>> I am not able to find out how to do the mkfs for my filesystem such
>> that the generic mkfs utility will understand my filesystem. What APIs
>> should I be using ?
>>
>> Any help is appreciated. Thanks.
>
> According to my copy of the mkfs sources, you just have to create a program
> named "mkfs.ID" where ID identifies your filesystem. Then put that program
> in a location that the generic mkfs can find, i.e. under $PATH (mkfs seems
> to make some additions to PATH but you should figure this out yourself).
>
> Finally, calling "mkfs -t ID" makes mkfs search for a program named
> "mkfs.ID" - simple concatenation.
>

oh okay. But how do I create the superblock ? What are the APIs
available to do these block level operations from a user space
application (my mkfs program ) ?

Thanks.


--
Sankar P
http://psankar.blogspot.com

Creating mkfs for my custom filesystem

[Valdis.Kletnieks at vt.edu]

On Fri, 29 Mar 2013 15:44:49 +0530, Sankar P said:

>> I have decided on a simple layout for my filesystem where the first
>> block will be the super block and   will contain the version
>> information etc. The second block will contain the list of inodes.
>> Third block onwards will be data blocks. Each file can grow only up to
>> a single block size. Thrid block will represent the first file, fourth
>> block for the second file and so on. Directories will not be
>> supported.

You *will* have to support at least the top-level directory, because
you'll need at least directory entries for "." and "..".  If your second
block is "list of inodes", then you have directories (and adding subdir
support isn't *that* hard).

You'll also want either a list or bitmap structure or some other way to
determine if a given block is allocated - trying to write a new file without
having a freelist to get blocks from is hard.  Oh, and don't forget to
add locking around the freelist operations and similar things - having
two processes both grab block 27 for the file they just created can suck :)

> oh okay. But how do I create the superblock ? What are the APIs
> available to do these block level operations from a user space
> application (my mkfs program ) ?

struct foobar_suoper {
	int version;
	int num_files;
	int free_blocks;
	char padding[512-3*sizeof(int)];
};

struct foobar_super sb;

int disk;

	bzero(sb, sizeof struct foobar_super);
	sb.version = 1;
	sb.num_files= 0;
	sb.free_blocks = 999; /* should probably set to actual size of partition/file */

	disk = open(*diskorfilename, ....);  /* testing on loop mounts is useful */
	lseek ( disk, 0);
	write (disk, &sb, sizeof(sb)); /* congrats, you just wrote a superblock */

Yes, it's that simple :)  You want to write some empty inodes, add a
'struct inode' variable, initialize it, lseek to were the inode goes and
write it out.

Just open, lseek, write, close.  ;)  And yes, those operations *do* work
just fine on both files you then use woth 'mount -o loop' and with /dev/sd*
or /dev/mapper/* LVM.

You might want to look at the source for mkfs.vfat (part of dosfstools package)
for additional details.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 865 bytes
Desc: not available
Url : http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130329/87bc7433/attachment.bin 

Creating mkfs for my custom filesystem

[Tobias Boege]

On Fri, 29 Mar 2013, Sankar P wrote:
> On Fri, Mar 29, 2013 at 3:28 PM, Tobias Boege <tobias@gambas-buch.de> wrote:
> > On Fri, 29 Mar 2013, Sankar P wrote:
> >> Hi,
> >>
> >> I am trying to write a simple filesystem to learn the basics of it.
> >>
> >> I have decided on a simple layout for my filesystem where the first
> >> block will be the super block and   will contain the version
> >> information etc. The second block will contain the list of inodes.
> >> Third block onwards will be data blocks. Each file can grow only up to
> >> a single block size. Thrid block will represent the first file, fourth
> >> block for the second file and so on. Directories will not be
> >> supported.
> >>
> >> Now I want to create a mkfs for my filesystem as mentioned above. But
> >> I am not able to find out how to do the mkfs for my filesystem such
> >> that the generic mkfs utility will understand my filesystem. What APIs
> >> should I be using ?
> >>
> >> Any help is appreciated. Thanks.
> >
> > According to my copy of the mkfs sources, you just have to create a program
> > named "mkfs.ID" where ID identifies your filesystem. Then put that program
> > in a location that the generic mkfs can find, i.e. under $PATH (mkfs seems
> > to make some additions to PATH but you should figure this out yourself).
> >
> > Finally, calling "mkfs -t ID" makes mkfs search for a program named
> > "mkfs.ID" - simple concatenation.
> >
> 
> oh okay. But how do I create the superblock ? What are the APIs
> available to do these block level operations from a user space
> application (my mkfs program ) ?
> 

Look at how other mkfs.* do it. In general, these programs seem to be fairly
complex. To interface with block devices from userspace: use the standard
system calls, open(), lseek(), write(), ...

For an example, you can look at the e2fsprogs source code. If you get
through the jungle of indirections you will eventually find just a plain
write() call in lib/ext2fs/unix_io.c:raw_write_blk() which is used for data
transfer.

Regards,
Tobi

Creating mkfs for my custom filesystem

[Sankar P]

Thank you everyone for the plenty of good pointers. I should be able
to proceed from here :-)

On Fri, Mar 29, 2013 at 7:31 PM, Tobias Boege <tobias@gambas-buch.de> wrote:
> On Fri, 29 Mar 2013, Sankar P wrote:
>> On Fri, Mar 29, 2013 at 3:28 PM, Tobias Boege <tobias@gambas-buch.de> wrote:
>> > On Fri, 29 Mar 2013, Sankar P wrote:
>> >> Hi,
>> >>
>> >> I am trying to write a simple filesystem to learn the basics of it.
>> >>
>> >> I have decided on a simple layout for my filesystem where the first
>> >> block will be the super block and   will contain the version
>> >> information etc. The second block will contain the list of inodes.
>> >> Third block onwards will be data blocks. Each file can grow only up to
>> >> a single block size. Thrid block will represent the first file, fourth
>> >> block for the second file and so on. Directories will not be
>> >> supported.
>> >>
>> >> Now I want to create a mkfs for my filesystem as mentioned above. But
>> >> I am not able to find out how to do the mkfs for my filesystem such
>> >> that the generic mkfs utility will understand my filesystem. What APIs
>> >> should I be using ?
>> >>
>> >> Any help is appreciated. Thanks.
>> >
>> > According to my copy of the mkfs sources, you just have to create a program
>> > named "mkfs.ID" where ID identifies your filesystem. Then put that program
>> > in a location that the generic mkfs can find, i.e. under $PATH (mkfs seems
>> > to make some additions to PATH but you should figure this out yourself).
>> >
>> > Finally, calling "mkfs -t ID" makes mkfs search for a program named
>> > "mkfs.ID" - simple concatenation.
>> >
>>
>> oh okay. But how do I create the superblock ? What are the APIs
>> available to do these block level operations from a user space
>> application (my mkfs program ) ?
>>
>
> Look at how other mkfs.* do it. In general, these programs seem to be fairly
> complex. To interface with block devices from userspace: use the standard
> system calls, open(), lseek(), write(), ...
>
> For an example, you can look at the e2fsprogs source code. If you get
> through the jungle of indirections you will eventually find just a plain
> write() call in lib/ext2fs/unix_io.c:raw_write_blk() which is used for data
> transfer.
>
> Regards,
> Tobi
>
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies



-- 
Sankar P
http://psankar.blogspot.com

Fwd: Creating mkfs for my custom filesystem

[Pranay Srivastava]

---------- Forwarded message ----------
From: Pranay Srivastava <pranjas@gmail.com>
Date: Fri, 29 Mar 2013 15:53:21 +0530
Subject: Re: Creating mkfs for my custom filesystem
To: Sankar P <sankar.curiosity@gmail.com>



On 3/29/13, Sankar P <sankar.curiosity@gmail.com> wrote:
> Hi,
>
> I am trying to write a simple filesystem to learn the basics of it.
>
> I have decided on a simple layout for my filesystem where the first
> block will be the super block and   will contain the version
> information etc. The second block will contain the list of inodes.
> Third block onwards will be data blocks. Each file can grow only up to
> a single block size. Thrid block will represent the first file, fourth
> block for the second file and so on. Directories will not be
> supported.
>
> Now I want to create a mkfs for my filesystem as mentioned above. But
> I am not able to find out how to do the mkfs for my filesystem such
> that the generic mkfs utility will understand my filesystem. What APIs
> should I be using ?
>
> Any help is appreciated. Thanks.
>
> --
> Sankar P
> http://psankar.blogspot.com
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>

Hi Sankar,

I started with something like that. Although I wrote just a bit more
complex but it really is quite simple. If you want you can use this
and I'll also like to contribute to it.

https://github.com/pranjas/psfs.git

Checkout psfs.h and psfs-format.c for complete details. Let me know if
you need any help.

         ---P.K.S---

Creating mkfs for my custom filesystem

[Sankar P]

On Fri, Mar 29, 2013 at 3:56 PM, Pranay Srivastava <pranjas@gmail.com> wrote:
> ---------- Forwarded message ----------
> From: Pranay Srivastava <pranjas@gmail.com>
> Date: Fri, 29 Mar 2013 15:53:21 +0530
> Subject: Re: Creating mkfs for my custom filesystem
> To: Sankar P <sankar.curiosity@gmail.com>
>
>
>
> On 3/29/13, Sankar P <sankar.curiosity@gmail.com> wrote:
>> Hi,
>>
>> I am trying to write a simple filesystem to learn the basics of it.
>>
>> I have decided on a simple layout for my filesystem where the first
>> block will be the super block and   will contain the version
>> information etc. The second block will contain the list of inodes.
>> Third block onwards will be data blocks. Each file can grow only up to
>> a single block size. Thrid block will represent the first file, fourth
>> block for the second file and so on. Directories will not be
>> supported.
>>
>> Now I want to create a mkfs for my filesystem as mentioned above. But
>> I am not able to find out how to do the mkfs for my filesystem such
>> that the generic mkfs utility will understand my filesystem. What APIs
>> should I be using ?
>>
>> Any help is appreciated. Thanks.
>>
>> --
>> Sankar P
>> http://psankar.blogspot.com
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.org
>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>
>
> Hi Sankar,
>
> I started with something like that. Although I wrote just a bit more
> complex but it really is quite simple. If you want you can use this
> and I'll also like to contribute to it.
>
> https://github.com/pranjas/psfs.git
>
> Checkout psfs.h and psfs-format.c for complete details. Let me know if
> you need any help.
>

Thank you.

Yours seems a bit complex with support for extents etc. I am planning
to start as simple as it could be. However, I believe your code will
help me to find the right APIs I need. I will let you know once I
publish my code. Your code should be very helpful to me. Thanks.


--
Sankar P
http://psankar.blogspot.com

Creating mkfs for my custom filesystem

[Madper]

On 03/29/2013 06:48 PM, Sankar P wrote:
> On Fri, Mar 29, 2013 at 3:56 PM, Pranay Srivastava <pranjas@gmail.com> wrote:
>> ---------- Forwarded message ----------
>> From: Pranay Srivastava <pranjas@gmail.com>
>> Date: Fri, 29 Mar 2013 15:53:21 +0530
>> Subject: Re: Creating mkfs for my custom filesystem
>> To: Sankar P <sankar.curiosity@gmail.com>
>>
>>
>>
>> On 3/29/13, Sankar P <sankar.curiosity@gmail.com> wrote:
>>> Hi,
>>>
>>> I am trying to write a simple filesystem to learn the basics of it.
>>>
>>> I have decided on a simple layout for my filesystem where the first
>>> block will be the super block and   will contain the version
>>> information etc. The second block will contain the list of inodes.
>>> Third block onwards will be data blocks. Each file can grow only up to
>>> a single block size. Thrid block will represent the first file, fourth
>>> block for the second file and so on. Directories will not be
>>> supported.
>>>
>>> Now I want to create a mkfs for my filesystem as mentioned above. But
>>> I am not able to find out how to do the mkfs for my filesystem such
>>> that the generic mkfs utility will understand my filesystem. What APIs
>>> should I be using ?
>>>
>>> Any help is appreciated. Thanks.
>>>
>>> --
>>> Sankar P
>>> http://psankar.blogspot.com
>>>
>>> _______________________________________________
>>> Kernelnewbies mailing list
>>> Kernelnewbies at kernelnewbies.org
>>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>>
>> Hi Sankar,
>>
>> I started with something like that. Although I wrote just a bit more
>> complex but it really is quite simple. If you want you can use this
>> and I'll also like to contribute to it.
>>
>> https://github.com/pranjas/psfs.git
>>
>> Checkout psfs.h and psfs-format.c for complete details. Let me know if
>> you need any help.
>>
> Thank you.
>
> Yours seems a bit complex with support for extents etc. I am planning
> to start as simple as it could be. However, I believe your code will
> help me to find the right APIs I need. I will let you know once I
> publish my code. Your code should be very helpful to me. Thanks.
Howdy Sankar,

I found a simple mkfs's code. Hope it's helpful to you.
     -- https://github.com/mkatiyar/testfs/blob/master/util/mktestfs.c

--
Best,
Madper Xie.
>
> --
> Sankar P
> http://psankar.blogspot.com
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Creating mkfs for my custom filesystem

[Manish Katiyar]

On Fri, Mar 29, 2013 at 1:27 AM, Sankar P <sankar.curiosity@gmail.com>wrote:

> Hi,
>
> I am trying to write a simple filesystem to learn the basics of it.
>
> I have decided on a simple layout for my filesystem where the first
> block will be the super block and   will contain the version
> information etc. The second block will contain the list of inodes.
> Third block onwards will be data blocks. Each file can grow only up to
> a single block size. Thrid block will represent the first file, fourth
> block for the second file and so on. Directories will not be
> supported.
>

You can have a look at
https://github.com/mkatiyar/testfs/blob/master/READMEwhich has a
similar layout.

-- 
Thanks -
Manish
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20130329/9719ee78/attachment.html 

Creating mkfs for my custom filesystem

[Sankar P]

On Fri, Mar 29, 2013 at 10:57 PM, Manish Katiyar <mkatiyar@gmail.com> wrote:
>
>
>
> On Fri, Mar 29, 2013 at 1:27 AM, Sankar P <sankar.curiosity@gmail.com>
> wrote:
>>
>> Hi,
>>
>> I am trying to write a simple filesystem to learn the basics of it.
>>
>> I have decided on a simple layout for my filesystem where the first
>> block will be the super block and   will contain the version
>> information etc. The second block will contain the list of inodes.
>> Third block onwards will be data blocks. Each file can grow only up to
>> a single block size. Thrid block will represent the first file, fourth
>> block for the second file and so on. Directories will not be
>> supported.
>
>
> You can have a look at https://github.com/mkatiyar/testfs/blob/master/README
> which has a similar layout.
>

ah, nice :)

I assumed that my layout should be patentably simple, rudimentary and
unique. Seems like it is not the case :)

Thanks.


--
Sankar P
http://psankar.blogspot.com