Re: [PATCH 00/22 take 3] UBI: Unsorted Block Images

[Theodore Tso <tytso@mit.edu>]

On Tue, Mar 20, 2007 at 02:25:49PM +0200, Artem Bityutskiy wrote:
> You failed to clearly define what is block until now, then you blame me
> that I do not understand you. So I see block = eraseblock, lets assume
> for further conversation.
> 
> OK. Suppose we have done what you say, although I _do not_ think it is
> makes a lot of sense. So, now we have a block device, with 128KiB block
> size. We have LVM, dm-wl or whatever stuff. Fine.
> 
> Do you realize that 128KiB is _huge_ block size, and performance will
> suck, and suck a lot if you utilize say, ext2 or whatever block device
> FS.

As a suggestion, let's stop right here and see if we can get both
sides talking in a more constructive fashion.  Maybe it's just me, but
I see both sides talking past each other in a rather dramatic fashion.

Linux seems to allow multiple implementations of things at the edge
(such as filesystems), but not at the core (devicemapper when in,
device mapper didn't; it's unlikely that we would have two competing
block device layers or two VFS layers, etc.).  The question then is
whether UBI and dm are close enough or not that should be one
subsystem or not.

There a number of red herrings that have been introduced in this
discussion; of *course* the existing block device layer can handle
FLASH devices; Matt is proposing that they be extended.  And of
*course* you woulnd't propose to use ext2 on top of an 128k blocksize,
anymore than you would force a flash filesystem to use a 4k or 512
byte blocksize; there are plenty of configurations that won't make
sense, and by itself this isn't an indictment of the core idea that
the block device layer and dm should be augmented to encompass flash
functionality.

As far as who gets to do the work, unfortunately sometimes the people
submitting the new code have to make the changes suggested by the
reviewer.  That's one of the prices that gets paid for mainline
inclusion.  It's different when someone asks for a completely new
feature, especially for code that is already in mainline; then, "feel
free to send a patch" is perfectly accepted.  But if it's a matter of
refactoring the code to fit in some other framework, that's often up
to the submitter to do, not the reviewer. 

Of course, it remains to be seen whether or not this is a good idea to
do in the first place; but some of the arguments being used to shoot
down Matt's suggestion aren't really good ones to begin with.

> To make it faster I have to have a way to do finer grained I/O:
> read/write to different positions of 128KiB block. Do you realize how
> much you will abuse all the generic block device infrastructure if you
> try to add this? Note, all levels up to LVM will need to have this. A
> believe it is braindead ((c) tglx) to add this feature.

OK, and this could be it.  But I suspect one of the things which may
be missing that make it easier for you to explain why what UBI is
doing is so different from the dm and block device stack is to include
the contents of:

	http://www.linux-mtd.infradead.org/faq/ubi.html
	http://www.linux-mtd.infradead.org/doc/ubi.html

and some system level documentation in a Documentation/ubi.txt file as
part of the patch set.  I don't think people completely understand the
high-level architecture of what UBI is trying to achieve.  What are
the interfaces at the top and the bottom of the stack?  For example,
the fact that UBI exports Logical Erase blocks that are not a
power-of-two (possibly 128k minus 128 bytes) means that it certainly
might not be a good match for the dm stack.  But why is that the case?
I can imagine good reasons for it, but a high-level description of the
design decisions would be very useful.

It would also help people understand why there are so many "units" in
UBI, since hopefully the high-level documentation would explain why
they fit together, and perhaps why some of the units weren't folded
together.  What value do they add as separate components?

There are hints of the overall system architecture in some of the
indivdual comments for data structures, but even reading all of those,
there isn't quite enough for people to figure out what it is; and that
may be causing some of these comments of people saying there's too
much code to evaluate, or why didn't you do it *this* way?   

Regards,

						- Ted