size limit for backing store? block sizes? ("[sdx] Bad block number requested")

size limit for backing store? block sizes? ("[sdx] Bad block number requested")

[Jens-U. Mozdzen]

Hi *,

we're facing an obsure problem with a fresh bcache setup:

After creating a 8TB (netto) RAID5 device (hardware RAID controller),  
setting it up for bcache (using an existing cache set) and populating  
it with data, we got struck by massive dmesg reports of "[sdx] Bad  
block number requested" during writeback of dirty data. Both with our  
4.1.x kernel, as well as a 4.9.8 kernel.

After recreating the backing store with 3 TB (netto) and recreating  
the bcache setup, population went without any noticable errors.

While the 8TB device was populated with only the same amount of data  
(2.7 TB), block placement was probably across all of the 8TB space  
available.

Another parameter catching the eye is block sizes - the 8 TB backing  
store was created in a way such that 4k block size was exposed to the  
OS, while the 3 TB backing store was created so that 512b block size  
was reported. The caching set is on a PCI SSD with 512b block size.

So with backing:4k and cache:512b and 8 TB backing store size, bcache  
went mad during writeback ("echo 0 > writeback_running" immediately  
made the messages stop). With backing:512b and cache:512b and 3 TB  
backing store size, we had no error reports at all.

On a second  node, we have (had) a similar situation - backing:4k and  
cache:512b, but 4 TB backing store size. We've seen the errors there,  
too, when accessing an especially big logical volume that likely  
crossed some magic limit (block number on "physical volume"?). We  
still see the message there today, only much less frequent since we no  
longer use that large volume on the bcache device. Other volumes are  
there now, probably with a few data spaces at high block numbers,  
leading to the occasional error message (every few minutes) during  
writeback?

Even more puzzling, we have a third node, identical to the latter one  
- except that the bcache device is more filled with data and we see no  
such error (yet)...

So here we are - what are we facing? Is it a size limit regarding the  
backing store? Or does the error result from mixing block sizes, plus  
some other triggers?

If the former, where's the limit?

If it is about block sizes, questions pile up: Are the "dos" and  
"don'ts" documented anywhere? It's a rather common situation for us to  
run multiple backing devices on a single cache set, with both complete  
HDDs and logical volumes as backing stores. So it's very easy to come  
into a situation where we see either different block sizes between  
backing store and caching device or even differing block sizes between  
the various backing stores.

- using 512b for cache and 4k for backing device seems not to work,  
unless above is purely a size limit problem

- 512b for cache and 512b for backing store seems to work

- 4k for cache and 4k for backing store will probably work as well

- will 4k for cache and 512b for backing store work (sounds likely, as  
there will be no alignment problem in the backing store. OTOH, will  
bcache try to write 4k data (cache block) into 512b blocks (backing  
store) or will it write 8 blocks then, mapping the block size  
differences?)

- if the latter works, will using both 4k and 512b backing stores in  
parallel work if using a 4k cache?

Any insight and/or help tracking down the error are most welcome!

Regards,
Jens

Re: size limit for backing store? block sizes? ("[sdx] Bad block number requested")

[Kai Krakow]

Am Tue, 07 Feb 2017 13:35:58 +0100
schrieb "Jens-U. Mozdzen" <jmozdzen@nde.ag>:

> Hi *,
> 
> we're facing an obsure problem with a fresh bcache setup:
> 
> After creating a 8TB (netto) RAID5 device (hardware RAID
> controller), setting it up for bcache (using an existing cache set)
> and populating it with data, we got struck by massive dmesg reports
> of "[sdx] Bad block number requested" during writeback of dirty data.
> Both with our 4.1.x kernel, as well as a 4.9.8 kernel.
> 
> After recreating the backing store with 3 TB (netto) and recreating  
> the bcache setup, population went without any noticable errors.
> 
> While the 8TB device was populated with only the same amount of data  
> (2.7 TB), block placement was probably across all of the 8TB space  
> available.
> 
> Another parameter catching the eye is block sizes - the 8 TB backing  
> store was created in a way such that 4k block size was exposed to
> the OS, while the 3 TB backing store was created so that 512b block
> size was reported. The caching set is on a PCI SSD with 512b block
> size.
> 
> So with backing:4k and cache:512b and 8 TB backing store size,
> bcache went mad during writeback ("echo 0 > writeback_running"
> immediately made the messages stop). With backing:512b and cache:512b
> and 3 TB backing store size, we had no error reports at all.
> 
> On a second  node, we have (had) a similar situation - backing:4k
> and cache:512b, but 4 TB backing store size. We've seen the errors
> there, too, when accessing an especially big logical volume that
> likely crossed some magic limit (block number on "physical volume"?).
> We still see the message there today, only much less frequent since
> we no longer use that large volume on the bcache device. Other
> volumes are there now, probably with a few data spaces at high block
> numbers, leading to the occasional error message (every few minutes)
> during writeback?
> 
> Even more puzzling, we have a third node, identical to the latter
> one  
> - except that the bcache device is more filled with data and we see
> no such error (yet)...
> 
> So here we are - what are we facing? Is it a size limit regarding
> the backing store? Or does the error result from mixing block sizes,
> plus some other triggers?
> 
> If the former, where's the limit?
> 
> If it is about block sizes, questions pile up: Are the "dos" and  
> "don'ts" documented anywhere? It's a rather common situation for us
> to run multiple backing devices on a single cache set, with both
> complete HDDs and logical volumes as backing stores. So it's very
> easy to come into a situation where we see either different block
> sizes between backing store and caching device or even differing
> block sizes between the various backing stores.
> 
> - using 512b for cache and 4k for backing device seems not to work,  
> unless above is purely a size limit problem
> 
> - 512b for cache and 512b for backing store seems to work
> 
> - 4k for cache and 4k for backing store will probably work as well
> 
> - will 4k for cache and 512b for backing store work (sounds likely,
> as there will be no alignment problem in the backing store. OTOH,
> will bcache try to write 4k data (cache block) into 512b blocks
> (backing store) or will it write 8 blocks then, mapping the block
> size differences?)
> 
> - if the latter works, will using both 4k and 512b backing stores in  
> parallel work if using a 4k cache?
> 
> Any insight and/or help tracking down the error are most welcome!

Hmm, I think for me it refused to attach backend and cache if block
sizes differ. So I think the bug is there...

Once I created backing store and cache store in two separate steps.
During attaching, it complained that block sizes don't match and the
cacheset cannot be attached.

-- 
Regards,
Kai

Replies to list-only preferred.

Re: size limit for backing store? block sizes? ("[sdx] Bad block number requested")

[Jens-U. Mozdzen]

Hello Kai,

Zitat von Kai Krakow <hurikhan77@gmail.com>:
> Am Tue, 07 Feb 2017 13:35:58 +0100
> schrieb "Jens-U. Mozdzen" <jmozdzen@nde.ag>:
>
>> Hi *,
>> [...]
>> If it is about block sizes, questions pile up: Are the "dos" and
>> "don'ts" documented anywhere? It's a rather common situation for us
>> to run multiple backing devices on a single cache set, with both
>> complete HDDs and logical volumes as backing stores. So it's very
>> easy to come into a situation where we see either different block
>> sizes between backing store and caching device or even differing
>> block sizes between the various backing stores.
>>
>> - using 512b for cache and 4k for backing device seems not to work,
>> unless above is purely a size limit problem
>>
>> - 512b for cache and 512b for backing store seems to work
>>
>> - 4k for cache and 4k for backing store will probably work as well
>>
>> - will 4k for cache and 512b for backing store work (sounds likely,
>> as there will be no alignment problem in the backing store. OTOH,
>> will bcache try to write 4k data (cache block) into 512b blocks
>> (backing store) or will it write 8 blocks then, mapping the block
>> size differences?)
>>
>> - if the latter works, will using both 4k and 512b backing stores in
>> parallel work if using a 4k cache?
>>
>> Any insight and/or help tracking down the error are most welcome!
>
> Hmm, I think for me it refused to attach backend and cache if block
> sizes differ. So I think the bug is there...
>
> Once I created backing store and cache store in two separate steps.
> During attaching, it complained that block sizes don't match and the
> cacheset cannot be attached.

hm, I believe to have created the backing store and cache set in a  
common call to make-bcache for the mentioned server 2 and 3 - usually,  
I only attach separately when creating additional backing stores for  
an existing cache set. Currently I have no server to test, though.  
Attaching differingly block-sized cache sets definitely works for me,  
both kernel 4.1.36 and 4.9.8.

So you're saying that block sizes need to match? Then I'm in trouble,  
as I have SAS SSDs with 512b and HDDs with 4k block size that build  
the main storage for the environment in question.

Regards,
Jens

Re: size limit for backing store? block sizes? ("[sdx] Bad block number requested")

[Kai Krakow]

Am Wed, 08 Feb 2017 00:28:43 +0100
schrieb "Jens-U. Mozdzen" <jmozdzen@nde.ag>:

> Hello Kai,
> 
> Zitat von Kai Krakow <hurikhan77@gmail.com>:
> > Am Tue, 07 Feb 2017 13:35:58 +0100
> > schrieb "Jens-U. Mozdzen" <jmozdzen@nde.ag>:
> >  
> >> Hi *,
> >> [...]
> >> If it is about block sizes, questions pile up: Are the "dos" and
> >> "don'ts" documented anywhere? It's a rather common situation for us
> >> to run multiple backing devices on a single cache set, with both
> >> complete HDDs and logical volumes as backing stores. So it's very
> >> easy to come into a situation where we see either different block
> >> sizes between backing store and caching device or even differing
> >> block sizes between the various backing stores.
> >>
> >> - using 512b for cache and 4k for backing device seems not to work,
> >> unless above is purely a size limit problem
> >>
> >> - 512b for cache and 512b for backing store seems to work
> >>
> >> - 4k for cache and 4k for backing store will probably work as well
> >>
> >> - will 4k for cache and 512b for backing store work (sounds likely,
> >> as there will be no alignment problem in the backing store. OTOH,
> >> will bcache try to write 4k data (cache block) into 512b blocks
> >> (backing store) or will it write 8 blocks then, mapping the block
> >> size differences?)
> >>
> >> - if the latter works, will using both 4k and 512b backing stores
> >> in parallel work if using a 4k cache?
> >>
> >> Any insight and/or help tracking down the error are most welcome!  
> >
> > Hmm, I think for me it refused to attach backend and cache if block
> > sizes differ. So I think the bug is there...
> >
> > Once I created backing store and cache store in two separate steps.
> > During attaching, it complained that block sizes don't match and the
> > cacheset cannot be attached.  
> 
> hm, I believe to have created the backing store and cache set in a  
> common call to make-bcache for the mentioned server 2 and 3 -
> usually, I only attach separately when creating additional backing
> stores for an existing cache set. Currently I have no server to test,
> though. Attaching differingly block-sized cache sets definitely works
> for me, both kernel 4.1.36 and 4.9.8.
> 
> So you're saying that block sizes need to match? Then I'm in
> trouble, as I have SAS SSDs with 512b and HDDs with 4k block size
> that build the main storage for the environment in question.

The hardware block size (aka sector size) is not what is meant here...
You should use the biggest value. 8x512b fits into 4k. So bcache would
simply use 4k as the block size and it just works despite the sectors
being 512b.

What I meant is "make-bcacke --block X". It defaults to X=512b. I think
you may want to force it to 4k and see if your problem persists.

Have a look at bcache-super-show and compare backing and caching dev.
For all involved devices it shows for me:

dev.sectors_per_block   8

All my devices are 512b sector mode. So it is set to 4k for all my
devices (8x 512b = 4k). I did this to support migrating easily to 4k
sector devices later.

If you have different sector sizes (check information from [1]), it
should say 1 for 4k devices and 8 for 512b devices to use a common
block size of 4k for the complete set.

If this didn't match for me, I couldn't attach cache and backing.

Maybe we are talking different here. I understood you mean these
values. If blocksize differs (not sectorsize!), it shouldn't be
attachable. If it is, you're seeing a bug, I guess.

[1]: grep ^ /sys/block/sd?/queue/hw_sector_size

PS: Some devices emulate 512b sectors although they use 4k internally.
In that case you have to force 4k accesses for maximum performance! One
other reason why I forced 4k blocks on bcache - just to be sure. ;-)

-- 
Regards,
Kai

Replies to list-only preferred.

Re: size limit for backing store? block sizes? ("[sdx] Bad block number requested")

[Eric Wheeler]

On Wed, 8 Feb 2017, Kai Krakow wrote:

> Am Wed, 08 Feb 2017 00:28:43 +0100
> schrieb "Jens-U. Mozdzen" <jmozdzen@nde.ag>:
> 
> > Hello Kai,
> > 
> > Zitat von Kai Krakow <hurikhan77@gmail.com>:
> > > Am Tue, 07 Feb 2017 13:35:58 +0100
> > > schrieb "Jens-U. Mozdzen" <jmozdzen@nde.ag>:
> > >  
> > >> Hi *,
> > >> [...]
> > >> If it is about block sizes, questions pile up: Are the "dos" and
> > >> "don'ts" documented anywhere? It's a rather common situation for us
> > >> to run multiple backing devices on a single cache set, with both
> > >> complete HDDs and logical volumes as backing stores. So it's very
> > >> easy to come into a situation where we see either different block
> > >> sizes between backing store and caching device or even differing
> > >> block sizes between the various backing stores.
> > >>
> > >> - using 512b for cache and 4k for backing device seems not to work,
> > >> unless above is purely a size limit problem
> > >>
> > >> - 512b for cache and 512b for backing store seems to work
> > >>
> > >> - 4k for cache and 4k for backing store will probably work as well
> > >>
> > >> - will 4k for cache and 512b for backing store work (sounds likely,
> > >> as there will be no alignment problem in the backing store. OTOH,
> > >> will bcache try to write 4k data (cache block) into 512b blocks
> > >> (backing store) or will it write 8 blocks then, mapping the block
> > >> size differences?)
> > >>
> > >> - if the latter works, will using both 4k and 512b backing stores
> > >> in parallel work if using a 4k cache?
> > >>
> > >> Any insight and/or help tracking down the error are most welcome!  
> > >
> > > Hmm, I think for me it refused to attach backend and cache if block
> > > sizes differ. So I think the bug is there...
> > >
> > > Once I created backing store and cache store in two separate steps.
> > > During attaching, it complained that block sizes don't match and the
> > > cacheset cannot be attached.  
> > 
> > hm, I believe to have created the backing store and cache set in a  
> > common call to make-bcache for the mentioned server 2 and 3 -
> > usually, I only attach separately when creating additional backing
> > stores for an existing cache set. Currently I have no server to test,
> > though. Attaching differingly block-sized cache sets definitely works
> > for me, both kernel 4.1.36 and 4.9.8.
> > 
> > So you're saying that block sizes need to match? Then I'm in
> > trouble, as I have SAS SSDs with 512b and HDDs with 4k block size
> > that build the main storage for the environment in question.
> 
> The hardware block size (aka sector size) is not what is meant here...
> You should use the biggest value. 8x512b fits into 4k. So bcache would
> simply use 4k as the block size and it just works despite the sectors
> being 512b.
> 
> What I meant is "make-bcacke --block X". It defaults to X=512b. I think
> you may want to force it to 4k and see if your problem persists.

Maybe --block 4096 should be default in make-bcache?

Kent, is there any reason this wouldn't be a good idea?


--
Eric Wheeler




> 
> Have a look at bcache-super-show and compare backing and caching dev.
> For all involved devices it shows for me:
> 
> dev.sectors_per_block   8
> 
> All my devices are 512b sector mode. So it is set to 4k for all my
> devices (8x 512b = 4k). I did this to support migrating easily to 4k
> sector devices later.
> 
> If you have different sector sizes (check information from [1]), it
> should say 1 for 4k devices and 8 for 512b devices to use a common
> block size of 4k for the complete set.
> 
> If this didn't match for me, I couldn't attach cache and backing.
> 
> Maybe we are talking different here. I understood you mean these
> values. If blocksize differs (not sectorsize!), it shouldn't be
> attachable. If it is, you're seeing a bug, I guess.
> 
> [1]: grep ^ /sys/block/sd?/queue/hw_sector_size
> 
> PS: Some devices emulate 512b sectors although they use 4k internally.
> In that case you have to force 4k accesses for maximum performance! One
> other reason why I forced 4k blocks on bcache - just to be sure. ;-)
> 
> -- 
> Regards,
> Kai
> 
> Replies to list-only preferred.
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: size limit for backing store? block sizes? ("[sdx] Bad block number requested")

[Kent Overstreet]

On Thu, Feb 09, 2017 at 01:37:46PM -0800, Eric Wheeler wrote:
> Maybe --block 4096 should be default in make-bcache?
> 
> Kent, is there any reason this wouldn't be a good idea?

That'd break things for people who expect sub 4k DIO to work (and if your
devices all have 512 byte sectors, that's a perfectly reasonable thing to
expect).

Unfortunately, there's no easy solution here.

Re: size limit for backing store? block sizes? ("[sdx] Bad block number requested")

[Jens-U. Mozdzen]

Hi *,

Zitat von Eric Wheeler <bcache@lists.ewheeler.net>:
> On Wed, 8 Feb 2017, Kai Krakow wrote:
> [...]
>> What I meant is "make-bcacke --block X". It defaults to X=512b. I think
>> you may want to force it to 4k and see if your problem persists.
>
> Maybe --block 4096 should be default in make-bcache?
>
> Kent, is there any reason this wouldn't be a good idea?

actually, make-bcache seems not to default to 512 octets. If no  
specific block size is specified on the command line, it determines  
the maximum sector size of the devices involved:

--- cut here ---
if (!block_size) {
		for (i = 0; i < ncache_devices; i++)
			block_size = max(block_size,
					 get_blocksize(cache_devices[i]));

		for (i = 0; i < nbacking_devices; i++)
			block_size = max(block_size,
					 get_blocksize(backing_devices[i]));
}
--- cut here ---

get_blocksize() queries the logical block size as reported by the  
device ( ioctl(fd, BLKSSZGET, &logical_block_size) ).

At least that's what I see at  
https://github.com/g2p/bcache-tools/blob/master/make-bcache.c, said to  
be from Apr 16, 2014. And it seems to be the reason why invoking  
"make-bcache -B ... -C ..." did create a proper setup for me, while  
separating that to "make-bcache -B ..." and "make-bcache -C ..."  
created a wrongly set up caching device for me (backing device has 4k  
sector size, caching device 512b).

Regards,
J

Re: size limit for backing store? block sizes? ("[sdx] Bad block number requested")

[Kai Krakow]

Am Tue, 14 Feb 2017 12:05:48 +0100
schrieb "Jens-U. Mozdzen" <jmozdzen@nde.ag>:

> Hi *,
> 
> Zitat von Eric Wheeler <bcache@lists.ewheeler.net>:
> > On Wed, 8 Feb 2017, Kai Krakow wrote:
> > [...]  
> >> What I meant is "make-bcacke --block X". It defaults to X=512b. I
> >> think you may want to force it to 4k and see if your problem
> >> persists.  
> >
> > Maybe --block 4096 should be default in make-bcache?
> >
> > Kent, is there any reason this wouldn't be a good idea?  
> 
> actually, make-bcache seems not to default to 512 octets. If no  
> specific block size is specified on the command line, it determines  
> the maximum sector size of the devices involved:
> 
> --- cut here ---
> if (!block_size) {
> 		for (i = 0; i < ncache_devices; i++)
> 			block_size = max(block_size,
> 					 get_blocksize(cache_devices[i]));
> 
> 		for (i = 0; i < nbacking_devices; i++)
> 			block_size = max(block_size,
> 					 get_blocksize(backing_devices[i]));
> }
> --- cut here ---
> 
> get_blocksize() queries the logical block size as reported by the  
> device ( ioctl(fd, BLKSSZGET, &logical_block_size) ).
> 
> At least that's what I see at  
> https://github.com/g2p/bcache-tools/blob/master/make-bcache.c, said
> to be from Apr 16, 2014. And it seems to be the reason why invoking  
> "make-bcache -B ... -C ..." did create a proper setup for me, while  
> separating that to "make-bcache -B ..." and "make-bcache -C ..."  
> created a wrongly set up caching device for me (backing device has
> 4k sector size, caching device 512b).

The call to max() in this function clearly tells that both devices
(backing and cache) should use the same block size.

I still wonder why attaching different block-sized devices wasn't
denied for you. When I tried it once, it didn't work.

-- 
Regards,
Kai

Replies to list-only preferred.

Re: size limit for backing store? block sizes? ("[sdx] Bad block number requested")

[Jens-U. Mozdzen]

Hello *,

Zitat von Kai Krakow <hurikhan77@gmail.com>:
> [...]
> The hardware block size (aka sector size) is not what is meant here...
> You should use the biggest value. 8x512b fits into 4k. So bcache would
> simply use 4k as the block size and it just works despite the sectors
> being 512b.

sorry for using confusing terms - yes, I was talking about sector sizes.

> What I meant is "make-bcacke --block X". It defaults to X=512b. I think
> you may want to force it to 4k and see if your problem persists.

I checked my various bcache backing stores and cache devices and got  
pretty inconsistent values. Time to clean up.

So trying to straighten things out, I started with a fresh backing  
device and feel like something's broken here:

- created a new RAID device, which has a sector size of 4096 per  
underlying disks:

myserver:~ # blockdev --getss /dev/md125
4096

- created the bcache super block

myserver:~ # make-bcache -B --block 4096b /dev/md125
UUID:                   05d4d36a-ef17-4ecb-99d4-0a0eeec03941
Set UUID:               dd4165a9-47ca-4e35-9e7a-5968e1e40d92
version:                1
block_size:             8
data_offset:            16
myserver:~ # bcache-super-show /dev/md125
sb.magic                ok
sb.first_sector         8 [match]
sb.csum                 F4E616E067FA3C45 [match]
sb.version              1 [backing device]

dev.label               (empty)
dev.uuid                05d4d36a-ef17-4ecb-99d4-0a0eeec03941
dev.sectors_per_block   8
dev.sectors_per_bucket  1024
dev.data.first_sector   16
dev.data.cache_mode     0 [writethrough]
dev.data.cache_state    0 [detached]

cset.uuid               dd4165a9-47ca-4e35-9e7a-5968e1e40d92
myserver:~ # ll /sys/fs/bcache/
total 0
drwxr-xr-x 7 root root    0 Feb 10 11:37 32c8c1f9-268c-4405-8fd7-ab9f5acbe3fb
--w------- 1 root root 4096 Feb 10 11:35 register
--w------- 1 root root 4096 Feb 10 11:35 register_quiet
myserver:~ #

Why does it report 8 sectors per block? With 4k per sector, that'd be  
a block size of 32k?

"myserver" has a running bcache setup in parallel, with the cache  
device having a sector size of 512b (and bcache_super_show reports one  
sector per block). Does this influence "make-bcache", although no  
cache is attached (and the reported cset.uuid is different from the  
running bcache)?

Regards,
Jens

Re: size limit for backing store? block sizes? ("[sdx] Bad block number requested")

[Eric Wheeler]

On Fri, 10 Feb 2017, Jens-U. Mozdzen wrote:

> Hello *,
> 
> Zitat von Kai Krakow <hurikhan77@gmail.com>:
> >[...]
> >The hardware block size (aka sector size) is not what is meant here...
> >You should use the biggest value. 8x512b fits into 4k. So bcache would
> >simply use 4k as the block size and it just works despite the sectors
> >being 512b.
> 
> sorry for using confusing terms - yes, I was talking about sector sizes.
> 
> >What I meant is "make-bcacke --block X". It defaults to X=512b. I think
> >you may want to force it to 4k and see if your problem persists.
> 
> I checked my various bcache backing stores and cache devices and got pretty
> inconsistent values. Time to clean up.
> 
> So trying to straighten things out, I started with a fresh backing device and
> feel like something's broken here:
> 
> - created a new RAID device, which has a sector size of 4096 per underlying
> disks:
> 
> myserver:~ # blockdev --getss /dev/md125
> 4096
> 
> - created the bcache super block
> 
> myserver:~ # make-bcache -B --block 4096b /dev/md125
> UUID:                   05d4d36a-ef17-4ecb-99d4-0a0eeec03941
> Set UUID:               dd4165a9-47ca-4e35-9e7a-5968e1e40d92
> version:                1
> block_size:             8
> data_offset:            16
> myserver:~ # bcache-super-show /dev/md125
> sb.magic                ok
> sb.first_sector         8 [match]
> sb.csum                 F4E616E067FA3C45 [match]
> sb.version              1 [backing device]
> 
> dev.label               (empty)
> dev.uuid                05d4d36a-ef17-4ecb-99d4-0a0eeec03941
> dev.sectors_per_block   8
> dev.sectors_per_bucket  1024
> dev.data.first_sector   16
> dev.data.cache_mode     0 [writethrough]
> dev.data.cache_state    0 [detached]
> 
> cset.uuid               dd4165a9-47ca-4e35-9e7a-5968e1e40d92
> myserver:~ # ll /sys/fs/bcache/
> total 0
> drwxr-xr-x 7 root root    0 Feb 10 11:37 32c8c1f9-268c-4405-8fd7-ab9f5acbe3fb
> --w------- 1 root root 4096 Feb 10 11:35 register
> --w------- 1 root root 4096 Feb 10 11:35 register_quiet
> myserver:~ #
> 
> Why does it report 8 sectors per block? With 4k per sector, that'd be a block
> size of 32k?

Its in 512b units and 8*512=4096.  Lots of things are in units of 512 even 
though your block size is 4096.  Its correct, it just looks funny since 
you have 4k sectors.


--
Eric Wheeler


> 
> "myserver" has a running bcache setup in parallel, with the cache device
> having a sector size of 512b (and bcache_super_show reports one sector per
> block). Does this influence "make-bcache", although no cache is attached (and
> the reported cset.uuid is different from the running bcache)?
> 
> Regards,
> Jens
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: size limit for backing store? block sizes? ("[sdx] Bad block number requested")

[Jens-U. Mozdzen]

Hi Eric,

Zitat von Eric Wheeler <bcache@lists.ewheeler.net>:
> On Fri, 10 Feb 2017, Jens-U. Mozdzen wrote:
> [...]
>> myserver:~ # blockdev --getss /dev/md125
>> 4096
>>
>> - created the bcache super block
>>
>> myserver:~ # make-bcache -B --block 4096b /dev/md125
>> UUID:                   05d4d36a-ef17-4ecb-99d4-0a0eeec03941
>> Set UUID:               dd4165a9-47ca-4e35-9e7a-5968e1e40d92
>> version:                1
>> block_size:             8
>> data_offset:            16
>> myserver:~ # bcache-super-show /dev/md125
>> sb.magic                ok
>> sb.first_sector         8 [match]
>> sb.csum                 F4E616E067FA3C45 [match]
>> sb.version              1 [backing device]
>>
>> dev.label               (empty)
>> dev.uuid                05d4d36a-ef17-4ecb-99d4-0a0eeec03941
>> dev.sectors_per_block   8
>> dev.sectors_per_bucket  1024
>> dev.data.first_sector   16
>> dev.data.cache_mode     0 [writethrough]
>> dev.data.cache_state    0 [detached]
>>
>> cset.uuid               dd4165a9-47ca-4e35-9e7a-5968e1e40d92
>> myserver:~ # [...]
>>
>> Why does it report 8 sectors per block? With 4k per sector, that'd  
>> be a block
>> size of 32k?
>
> Its in 512b units and 8*512=4096.  Lots of things are in units of 512 even
> though your block size is 4096.  Its correct, it just looks funny since
> you have 4k sectors.

first of all, thank you for that information, I succeeded in setting  
up that specific bcache scenario, have populated it and found no  
problems so far.

Once I had a few spare minutes, I double-checked the source code of  
make-bcache and bcache-show-super and indeed, there's hardcoded the  
factor of 512 bytes. Nothing wrong with that, but I strongly suggest  
to change both tools to include some unit information:

--- cut here ---
myserver:~ # make-bcache -B --block 4096b /dev/md125
[...]
block_size:             8 (units of 512 octets)

myserver:~ # bcache-super-show /dev/md125
[...]
dev.sectors_per_block   8 (pseudo-sectors of 512 octets each)
--- cut here ---

dev.sectors_per_block is especially misleading in its current display,  
because that data element actually does *not* relate to native block  
device sectors.

Both changes would be purely cosmetic and not altering current functionality.

It might be favorable (AKA "better readable") to instead output the  
"block_size" in bytes/octets in both places, instead of just dumping  
the value of the super block element.

Regards,
J