Why is SECTOR_SIZE = 512 inside kernel ?

Why is SECTOR_SIZE = 512 inside kernel ?

[Navin P]

Hi,

 Why is SECTOR_SIZE 512 ?

http://lxr.free-electrons.com/source/include/linux/ide.h#L118

http://lxr.free-electrons.com/source/include/linux/device-mapper.h#L548

548 #define SECTOR_SHIFT 9

I was looking at disks with hw_sector_size . Most of them i looked at
had 512 bytes except for one which had 4096 (virtual disk). The one
with AF format ie has logical sector size as 512 and hw_sector_size as
512 . So it is fine for my calculation from /proc/diskstats.

But the one with 4096 logical and 4096 physical i multiply
hw_sector_size with the sectors read and written but that is wrong
since the kernel always defines sectors in terms of 512.


Is it going to change or is it  cast in stone ?


Here is an example. Again this is a VM virtual disk. vdc is the
subject in interest.


[root@hphuge-049 ~]# fdisk -l /dev/vdb /dev/vdc

Disk /dev/vdb: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/vdc: 17.2 GB, 17179869184 bytes, 4194304 sectors
Units = sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

[root@hphuge-049 ~]#


[root@hphuge-049 ~]# cat /sys/block/vdc/queue/hw_sector_size && cat
/sys/block/vdc/queue/logical_block_size  && cat
/sys/block/vdc/queue/physical_block_size
4096
4096
4096
[root@hphuge-049 ~]# cat /sys/block/vdb/queue/hw_sector_size && cat
/sys/block/vdb/queue/logical_block_size  && cat
/sys/block/vdb/queue/physical_block_size
512
512
4096
[root@hphuge-049 ~]#



Regards,
Navin

Re: Why is SECTOR_SIZE = 512 inside kernel ?

[Theodore Ts'o]

On Mon, Aug 17, 2015 at 06:23:04PM +0530, Navin P wrote:
> 
>  Why is SECTOR_SIZE 512 ?
> 
> http://lxr.free-electrons.com/source/include/linux/ide.h#L118
> 
> http://lxr.free-electrons.com/source/include/linux/device-mapper.h#L548
> 
> 548 #define SECTOR_SHIFT 9
> 
> I was looking at disks with hw_sector_size . Most of them i looked at
> had 512 bytes except for one which had 4096 (virtual disk). The one
> with AF format ie has logical sector size as 512 and hw_sector_size as
> 512 . So it is fine for my calculation from /proc/diskstats.
> 
> But the one with 4096 logical and 4096 physical i multiply
> hw_sector_size with the sectors read and written but that is wrong
> since the kernel always defines sectors in terms of 512.
> 
> Is it going to change or is it  cast in stone ?

It's cast in stone.  There are too many places all over the kernel,
especially in a huge number of file systems, which assume that the
sector size is 512 bytes.  So above the block layer, the sector size
is always going to be 512.

This is actually *better* for user space programs using
/proc/diskstats, since they don't need to know whether a particular
underlying hardware is using 512, 4k, (or if the HDD manufacturers
fantasies become true 32k or 64k) sector sizes.

For similar reason, st_blocks in struct size is always in units of 512
bytes.  We don't want to force userspace to have to figure out whether
the underlying file system is using 1k, 2k, or 4k.  For that reason
the units of st_blocks is always going to be 512 bytes, and this is
hard-coded in the POSIX standard.

					- Ted

Re: Why is SECTOR_SIZE = 512 inside kernel ?

[Brice Goglin]

Le 17/08/2015 15:54, Theodore Ts'o a écrit :
> 
> It's cast in stone.  There are too many places all over the kernel,
> especially in a huge number of file systems, which assume that the
> sector size is 512 bytes.  So above the block layer, the sector size
> is always going to be 512.

Could this be a problem when using pmem/nvdimm devices with
byte-granularity (no BTT layer)? (hw_sector_size reports
512 in this case while we could expect 1 instead).
Or it just doesn't matter because BTT is the only way to use
these devices for filesystems like other block devices?

thanks
Brice

Re: Why is SECTOR_SIZE = 512 inside kernel ?

[tytso]

On Tue, Aug 18, 2015 at 11:06:47PM +0200, Brice Goglin wrote:
> Le 17/08/2015 15:54, Theodore Ts'o a écrit :
> > 
> > It's cast in stone.  There are too many places all over the kernel,
> > especially in a huge number of file systems, which assume that the
> > sector size is 512 bytes.  So above the block layer, the sector size
> > is always going to be 512.
> 
> Could this be a problem when using pmem/nvdimm devices with
> byte-granularity (no BTT layer)? (hw_sector_size reports
> 512 in this case while we could expect 1 instead).
> Or it just doesn't matter because BTT is the only way to use
> these devices for filesystems like other block devices?

Right now there are very few applications that understand how to use
pmem/nvdimm devices as memory.  And even where they do, they will need
some kind of file system to provide resource isolation in case more
than one application or more than one user wants to use the
pmem/nvdimm.  In that case, they will probably mmap a file and then
access the nvdimm directly.  In that case, the applications won't be
using the block device layer at all, so they won't care about the
advertised hw_sector_Size.

The challenge with pmem-aware applications is that they need to be
able to correctly update their in-memory data structures in such a way
that they can correctly recover after an arbitrary power failure.
That means they have to use atomic updates and/or copy-on-write update
schemes, and I suspect most application writes just aren't going to be
able to get this right.

So for many legacy applications, they will still read in the file
"foo", make changes in local memory, and then write the new contents
to the file "foo.new", and then rename "foo.new" on top of "foo".
These applications will effectively use nvdimm as super fast flash,
and so they will use file systems as file systems.  And since file
systems today all use block sizes which are multiples of the
traditional 512 byte sector size, again, changing something as
fundamental as the kernel's internal sector size doesn't have any real
value, at least not as far as pmem/nvdimm support is concerned.

       	         	  	       	   - Ted

Why is SECTOR_SIZE = 512 inside kernel ?

[Navin P]

Hi,
 Why is SECTOR_SIZE 512 ?

http://lxr.free-electrons.com/source/include/linux/ide.h#L118

http://lxr.free-electrons.com/source/include/linux/device-mapper.h#L548

548 #define SECTOR_SHIFT 9

I was looking at disks with hw_sector_size . Most of them i looked at
had 512 bytes except for one which had 4096 (virtual disk). The one
with AF format ie has logical sector size as 512 and hw_sector_size as
512 . So it is fine for my calculation from /proc/diskstats.

But the one with 4096 logical and 4096 physical i multiply
hw_sector_size with the sectors read and written but that is wrong
since the kernel always defines sectors in terms of 512.


Is it going to change or is it  cast in stone ?


Here is an example. Again this is a VM virtual disk. vdc is the
subject in interest.


[root at hphuge-049 ~]# fdisk -l /dev/vdb /dev/vdc

Disk /dev/vdb: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/vdc: 17.2 GB, 17179869184 bytes, 4194304 sectors
Units = sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

[root at hphuge-049 ~]#


[root at hphuge-049 ~]# cat /sys/block/vdc/queue/hw_sector_size && cat
/sys/block/vdc/queue/logical_block_size  && cat
/sys/block/vdc/queue/physical_block_size
4096
4096
4096
[root at hphuge-049 ~]# cat /sys/block/vdb/queue/hw_sector_size && cat
/sys/block/vdb/queue/logical_block_size  && cat
/sys/block/vdb/queue/physical_block_size
512
512
4096
[root at hphuge-049 ~]#



Regards,
Navin

Why is SECTOR_SIZE = 512 inside kernel ?

[Navin P]

On Mon, Aug 17, 2015 at 10:46 PM,  <Valdis.Kletnieks@vt.edu> wrote:
> On Mon, 17 Aug 2015 16:35:59 +0530, Navin P said:
>> Hi,
>>  Why is SECTOR_SIZE 512 ?
>
> Because most fixed-block devices presented a logical interface in terms
> of 512 byte sectors.  (IBM's count-key-data format is an example of a non-fixed
> block device).
>
>> Is it going to change or is it  cast in stone ?
>
> As you have probably figured out, high-capacity devices have often moved
> to 4K physical blocks to avoid wasting so much room between blocks (the hardware
> format includes a preamble to each block and a postamble, and dead space in
> between, it uses more space than you think).
>
> However, as long as there are still devices out there that have 512 byte
> blocks, it's not changing.  And it still won't change unless there's a
> good reason to change it (hint - how many places in the kernel have hidden
> assumptions about the size of SECTOR_SIZE being 9 bits worth?)

I posted this to
http://www.mail-archive.com/linux-kernel at vger.kernel.org/msg956881.html
avoided cross posting.

Looks like this is cast in stone at kernel . I was thinking of doing
sectors written  * hw_sector_size. But the sectors counters in
/proc/diskstats are actually 512 byte sections even for 4kN disks.
That way userspace programs are immune to underlying sector. Wish that
page_size was also same :)

There are lot of places like 9 bits .

http://lxr.free-electrons.com/source/block/blk-lib.c

 bio->bi_iter.bi_size = req_sects << 9;

http://lxr.free-electrons.com/source/include/linux/blkdev.h

sprinkled with 9 . I'm assuming the magic number 9 here is SECTOR_SIZE. :)

The hw_sector_size must be cosmetic in that sense or just to you tell
you that "your drive really is 4kn" or "512e".