* Can't resgister cache @ 2012-02-15 5:26 Franco 2012-02-15 6:24 ` Kent Overstreet 0 siblings, 1 reply; 5+ messages in thread From: Franco @ 2012-02-15 5:26 UTC (permalink / raw) To: linux-bcache-u79uwXL29TY76Z2rM5mHXA Hi I was in the process of testing linux-bcache but was seeing some strange behaviour so I thought I would remount my cached filesystem. I then noticed that the performance results started looking like the cache just wasn't there, so I unregistered it: bcache: Cache set be008612-38aa-4082-9f42-1bada25cb002 unregistered bcache: Caching disabled for sdc but now when I try to register it again, I get bcache: error opening /dev/sdb: device busy. I haven't tried rebooting yet but probably will do after I've sent this message. These were the strange results I was getting that prompted me to unmount and remount the filesystems. Setup: hardware RAID 5 16 disk array with 2 partitions, 1 Intel 40GB SSD for bcache. Filesystem Size Used Avail Use% Mounted on /dev/bcache0 1.9T 26G 1.8T 2% /data33 /dev/sdd 1.9T 23G 1.8T 2% /data34 Wrote an 8GB file to each filesystem, then tried reading it back. Machine has 8GB RAM. Speed for random reads from identical SSD = 181.766523 MB/Sec, random read test done in the following order with measured speed in MB/Sec. The random IO is actually just reading with a 4GB stride, wrapping around at the end of the file. /data33 49 /data34 114 /data33 114 /data33 and /data44 simultaneously 78 and 59 respectively /data33 76 /data33 68 /data34 114 /data34 96 /data34 69 /data34 74 /data33 118 /data33 40 /data34 91 After remounting both filesystems: /data34 148 /data33 120 /data34 120 /data33 82 After unregistering the cache /data33 65 /data34 83 /data33 99 /data34 113 Cheers, Franco ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Can't resgister cache 2012-02-15 5:26 Can't resgister cache Franco @ 2012-02-15 6:24 ` Kent Overstreet [not found] ` <CAH+dOx+g=eKDgTCnVdZYKrtn7tZjAnWeMdYP0KuZwD=rktsuJw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 0 siblings, 1 reply; 5+ messages in thread From: Kent Overstreet @ 2012-02-15 6:24 UTC (permalink / raw) To: Franco; +Cc: linux-bcache-u79uwXL29TY76Z2rM5mHXA Not sure what's going on with not being able to register again - not a lot of info to go off of. It was working last I checked in the bcache-3.2-dev branch, it might be buggy in the older bcache branch. From your numbers it looks like the 8 gb of data wasn't in the cache - bcache bypasses sequential IO by default. You can flip it off by echoing 0 to sequential_cutoff. On Tue, Feb 14, 2012 at 9:26 PM, Franco <franco-uJ2R6605E/fKnmoGZ802fQ@public.gmane.org> wrote: > Hi > > I was in the process of testing linux-bcache but was seeing some strange > behaviour so I thought I would remount my cached filesystem. I then > noticed that the performance results started looking like the cache just > wasn't there, so I unregistered it: > > bcache: Cache set be008612-38aa-4082-9f42-1bada25cb002 unregistered > bcache: Caching disabled for sdc > > but now when I try to register it again, I get bcache: error > opening /dev/sdb: device busy. > > I haven't tried rebooting yet but probably will do after I've sent this > message. > > These were the strange results I was getting that prompted me to unmount > and remount the filesystems. > > Setup: hardware RAID 5 16 disk array with 2 partitions, 1 Intel 40GB SSD > for bcache. > > Filesystem Size Used Avail Use% Mounted on > /dev/bcache0 1.9T 26G 1.8T 2% /data33 > /dev/sdd 1.9T 23G 1.8T 2% /data34 > > Wrote an 8GB file to each filesystem, then tried reading it back. > Machine has 8GB RAM. > > Speed for random reads from identical SSD = 181.766523 MB/Sec, random > read test done in the following order with measured speed in MB/Sec. The > random IO is actually just reading with a 4GB stride, wrapping around at > the end of the file. > > /data33 49 > /data34 114 > /data33 114 > /data33 and /data44 simultaneously 78 and 59 respectively > /data33 76 > /data33 68 > /data34 114 > /data34 96 > /data34 69 > /data34 74 > /data33 118 > /data33 40 > /data34 91 > > After remounting both filesystems: > > /data34 148 > /data33 120 > /data34 120 > /data33 82 > > > After unregistering the cache > > /data33 65 > /data34 83 > /data33 99 > /data34 113 > > Cheers, > Franco > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-bcache" in > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 5+ messages in thread
[parent not found: <CAH+dOx+g=eKDgTCnVdZYKrtn7tZjAnWeMdYP0KuZwD=rktsuJw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>]
* Re: Can't resgister cache [not found] ` <CAH+dOx+g=eKDgTCnVdZYKrtn7tZjAnWeMdYP0KuZwD=rktsuJw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> @ 2012-02-15 7:16 ` Franco 2012-02-16 1:09 ` Kent Overstreet 0 siblings, 1 reply; 5+ messages in thread From: Franco @ 2012-02-15 7:16 UTC (permalink / raw) To: Kent Overstreet; +Cc: linux-bcache-u79uwXL29TY76Z2rM5mHXA I did a reboot and got it registered again. bio: create slab <bio-1> at 1 bcache: invalidating existing data bcache: registered cache device sdb bcache: Caching sdc, inserted new UUID 87ba7904-7690-4985-b6a8-daeba41712a3 EXT4-fs (bcache0): mounted filesystem with ordered data mode. Opts: (null) First read went at 119 MB/Sec, second at 37MB/Sec Here's what's in stats_hour /sys/block/sdc/bcache/stats_hour/cache_bypass_hits :::::::::::::: 44 :::::::::::::: /sys/block/sdc/bcache/stats_hour/cache_bypass_misses :::::::::::::: 81172 :::::::::::::: /sys/block/sdc/bcache/stats_hour/cache_hit_ratio :::::::::::::: 0 :::::::::::::: /sys/block/sdc/bcache/stats_hour/cache_hits :::::::::::::: 24 :::::::::::::: /sys/block/sdc/bcache/stats_hour/cache_miss_collisions :::::::::::::: 14 :::::::::::::: /sys/block/sdc/bcache/stats_hour/cache_misses :::::::::::::: 407422 :::::::::::::: /sys/block/sdc/bcache/stats_hour/cache_readaheads :::::::::::::: 0 With echo 0 >/sys/block/sdc/bcache/sequential_cutoff 33 MB/Sec On Tue, 2012-02-14 at 22:24 -0800, Kent Overstreet wrote: > Not sure what's going on with not being able to register again - not a > lot of info to go off of. It was working last I checked in the > bcache-3.2-dev branch, it might be buggy in the older bcache branch. > > >From your numbers it looks like the 8 gb of data wasn't in the cache - > bcache bypasses sequential IO by default. You can flip it off by > echoing 0 to sequential_cutoff. > > On Tue, Feb 14, 2012 at 9:26 PM, Franco <franco-uJ2R6605E/fKnmoGZ802fQ@public.gmane.org> wrote: > > Hi > > > > I was in the process of testing linux-bcache but was seeing some strange > > behaviour so I thought I would remount my cached filesystem. I then > > noticed that the performance results started looking like the cache just > > wasn't there, so I unregistered it: > > > > bcache: Cache set be008612-38aa-4082-9f42-1bada25cb002 unregistered > > bcache: Caching disabled for sdc > > > > but now when I try to register it again, I get bcache: error > > opening /dev/sdb: device busy. > > > > I haven't tried rebooting yet but probably will do after I've sent this > > message. > > > > These were the strange results I was getting that prompted me to unmount > > and remount the filesystems. > > > > Setup: hardware RAID 5 16 disk array with 2 partitions, 1 Intel 40GB SSD > > for bcache. > > > > Filesystem Size Used Avail Use% Mounted on > > /dev/bcache0 1.9T 26G 1.8T 2% /data33 > > /dev/sdd 1.9T 23G 1.8T 2% /data34 > > > > Wrote an 8GB file to each filesystem, then tried reading it back. > > Machine has 8GB RAM. > > > > Speed for random reads from identical SSD = 181.766523 MB/Sec, random > > read test done in the following order with measured speed in MB/Sec. The > > random IO is actually just reading with a 4GB stride, wrapping around at > > the end of the file. > > > > /data33 49 > > /data34 114 > > /data33 114 > > /data33 and /data44 simultaneously 78 and 59 respectively > > /data33 76 > > /data33 68 > > /data34 114 > > /data34 96 > > /data34 69 > > /data34 74 > > /data33 118 > > /data33 40 > > /data34 91 > > > > After remounting both filesystems: > > > > /data34 148 > > /data33 120 > > /data34 120 > > /data33 82 > > > > > > After unregistering the cache > > > > /data33 65 > > /data34 83 > > /data33 99 > > /data34 113 > > > > Cheers, > > Franco > > > > > > -- > > To unsubscribe from this list: send the line "unsubscribe linux-bcache" in > > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Can't resgister cache 2012-02-15 7:16 ` Franco @ 2012-02-16 1:09 ` Kent Overstreet [not found] ` <CAC7rs0sadP553tU0foAgDMzWh9euk4LyUYFL0_13rdeD+bBdWA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> 0 siblings, 1 reply; 5+ messages in thread From: Kent Overstreet @ 2012-02-16 1:09 UTC (permalink / raw) To: Franco; +Cc: Kent Overstreet, linux-bcache-u79uwXL29TY76Z2rM5mHXA That sounds like your SSD is slow and bottlenecking you... What kind of SSD is it? Could be some strange plugging thing, maybe... On Tue, Feb 14, 2012 at 11:16 PM, Franco <franco-uJ2R6605E/fKnmoGZ802fQ@public.gmane.org> wrote: > > I did a reboot and got it registered again. > > bio: create slab <bio-1> at 1 > bcache: invalidating existing data > bcache: registered cache device sdb > bcache: Caching sdc, inserted new UUID 87ba7904-7690-4985-b6a8-daeba41712a3 > EXT4-fs (bcache0): mounted filesystem with ordered data mode. Opts: (null) > > > First read went at 119 MB/Sec, second at 37MB/Sec > > Here's what's in stats_hour > > /sys/block/sdc/bcache/stats_hour/cache_bypass_hits > :::::::::::::: > 44 > :::::::::::::: > /sys/block/sdc/bcache/stats_hour/cache_bypass_misses > :::::::::::::: > 81172 > :::::::::::::: > /sys/block/sdc/bcache/stats_hour/cache_hit_ratio > :::::::::::::: > 0 > :::::::::::::: > /sys/block/sdc/bcache/stats_hour/cache_hits > :::::::::::::: > 24 > :::::::::::::: > /sys/block/sdc/bcache/stats_hour/cache_miss_collisions > :::::::::::::: > 14 > :::::::::::::: > /sys/block/sdc/bcache/stats_hour/cache_misses > :::::::::::::: > 407422 > :::::::::::::: > /sys/block/sdc/bcache/stats_hour/cache_readaheads > :::::::::::::: > 0 > > With echo 0 >/sys/block/sdc/bcache/sequential_cutoff > > 33 MB/Sec > > > On Tue, 2012-02-14 at 22:24 -0800, Kent Overstreet wrote: >> Not sure what's going on with not being able to register again - not a >> lot of info to go off of. It was working last I checked in the >> bcache-3.2-dev branch, it might be buggy in the older bcache branch. >> >> >From your numbers it looks like the 8 gb of data wasn't in the cache - >> bcache bypasses sequential IO by default. You can flip it off by >> echoing 0 to sequential_cutoff. >> >> On Tue, Feb 14, 2012 at 9:26 PM, Franco <franco-uJ2R6605E/fKnmoGZ802fQ@public.gmane.org> wrote: >> > Hi >> > >> > I was in the process of testing linux-bcache but was seeing some strange >> > behaviour so I thought I would remount my cached filesystem. I then >> > noticed that the performance results started looking like the cache just >> > wasn't there, so I unregistered it: >> > >> > bcache: Cache set be008612-38aa-4082-9f42-1bada25cb002 unregistered >> > bcache: Caching disabled for sdc >> > >> > but now when I try to register it again, I get bcache: error >> > opening /dev/sdb: device busy. >> > >> > I haven't tried rebooting yet but probably will do after I've sent this >> > message. >> > >> > These were the strange results I was getting that prompted me to unmount >> > and remount the filesystems. >> > >> > Setup: hardware RAID 5 16 disk array with 2 partitions, 1 Intel 40GB SSD >> > for bcache. >> > >> > Filesystem Size Used Avail Use% Mounted on >> > /dev/bcache0 1.9T 26G 1.8T 2% /data33 >> > /dev/sdd 1.9T 23G 1.8T 2% /data34 >> > >> > Wrote an 8GB file to each filesystem, then tried reading it back. >> > Machine has 8GB RAM. >> > >> > Speed for random reads from identical SSD = 181.766523 MB/Sec, random >> > read test done in the following order with measured speed in MB/Sec. The >> > random IO is actually just reading with a 4GB stride, wrapping around at >> > the end of the file. >> > >> > /data33 49 >> > /data34 114 >> > /data33 114 >> > /data33 and /data44 simultaneously 78 and 59 respectively >> > /data33 76 >> > /data33 68 >> > /data34 114 >> > /data34 96 >> > /data34 69 >> > /data34 74 >> > /data33 118 >> > /data33 40 >> > /data34 91 >> > >> > After remounting both filesystems: >> > >> > /data34 148 >> > /data33 120 >> > /data34 120 >> > /data33 82 >> > >> > >> > After unregistering the cache >> > >> > /data33 65 >> > /data34 83 >> > /data33 99 >> > /data34 113 >> > >> > Cheers, >> > Franco >> > >> > >> > -- >> > To unsubscribe from this list: send the line "unsubscribe linux-bcache" in >> > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org >> > More majordomo info at http://vger.kernel.org/majordomo-info.html > > > -- > To unsubscribe from this list: send the line "unsubscribe linux-bcache" in > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 5+ messages in thread
[parent not found: <CAC7rs0sadP553tU0foAgDMzWh9euk4LyUYFL0_13rdeD+bBdWA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>]
* Re: Can't resgister cache [not found] ` <CAC7rs0sadP553tU0foAgDMzWh9euk4LyUYFL0_13rdeD+bBdWA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org> @ 2012-02-16 1:10 ` Kent Overstreet 0 siblings, 0 replies; 5+ messages in thread From: Kent Overstreet @ 2012-02-16 1:10 UTC (permalink / raw) To: Franco; +Cc: Kent Overstreet, linux-bcache-u79uwXL29TY76Z2rM5mHXA Also, IO is still getting bypassed. If sequential_cutoff is 0, then the only reason that'd happen is if the latency to the SSD is getting too high... On Wed, Feb 15, 2012 at 5:09 PM, Kent Overstreet <kent.overstreet-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote: > That sounds like your SSD is slow and bottlenecking you... What kind > of SSD is it? > > Could be some strange plugging thing, maybe... > > On Tue, Feb 14, 2012 at 11:16 PM, Franco <franco-uJ2R6605E/fKnmoGZ802fQ@public.gmane.org> wrote: >> >> I did a reboot and got it registered again. >> >> bio: create slab <bio-1> at 1 >> bcache: invalidating existing data >> bcache: registered cache device sdb >> bcache: Caching sdc, inserted new UUID 87ba7904-7690-4985-b6a8-daeba41712a3 >> EXT4-fs (bcache0): mounted filesystem with ordered data mode. Opts: (null) >> >> >> First read went at 119 MB/Sec, second at 37MB/Sec >> >> Here's what's in stats_hour >> >> /sys/block/sdc/bcache/stats_hour/cache_bypass_hits >> :::::::::::::: >> 44 >> :::::::::::::: >> /sys/block/sdc/bcache/stats_hour/cache_bypass_misses >> :::::::::::::: >> 81172 >> :::::::::::::: >> /sys/block/sdc/bcache/stats_hour/cache_hit_ratio >> :::::::::::::: >> 0 >> :::::::::::::: >> /sys/block/sdc/bcache/stats_hour/cache_hits >> :::::::::::::: >> 24 >> :::::::::::::: >> /sys/block/sdc/bcache/stats_hour/cache_miss_collisions >> :::::::::::::: >> 14 >> :::::::::::::: >> /sys/block/sdc/bcache/stats_hour/cache_misses >> :::::::::::::: >> 407422 >> :::::::::::::: >> /sys/block/sdc/bcache/stats_hour/cache_readaheads >> :::::::::::::: >> 0 >> >> With echo 0 >/sys/block/sdc/bcache/sequential_cutoff >> >> 33 MB/Sec >> >> >> On Tue, 2012-02-14 at 22:24 -0800, Kent Overstreet wrote: >>> Not sure what's going on with not being able to register again - not a >>> lot of info to go off of. It was working last I checked in the >>> bcache-3.2-dev branch, it might be buggy in the older bcache branch. >>> >>> >From your numbers it looks like the 8 gb of data wasn't in the cache - >>> bcache bypasses sequential IO by default. You can flip it off by >>> echoing 0 to sequential_cutoff. >>> >>> On Tue, Feb 14, 2012 at 9:26 PM, Franco <franco-uJ2R6605E/fKnmoGZ802fQ@public.gmane.org> wrote: >>> > Hi >>> > >>> > I was in the process of testing linux-bcache but was seeing some strange >>> > behaviour so I thought I would remount my cached filesystem. I then >>> > noticed that the performance results started looking like the cache just >>> > wasn't there, so I unregistered it: >>> > >>> > bcache: Cache set be008612-38aa-4082-9f42-1bada25cb002 unregistered >>> > bcache: Caching disabled for sdc >>> > >>> > but now when I try to register it again, I get bcache: error >>> > opening /dev/sdb: device busy. >>> > >>> > I haven't tried rebooting yet but probably will do after I've sent this >>> > message. >>> > >>> > These were the strange results I was getting that prompted me to unmount >>> > and remount the filesystems. >>> > >>> > Setup: hardware RAID 5 16 disk array with 2 partitions, 1 Intel 40GB SSD >>> > for bcache. >>> > >>> > Filesystem Size Used Avail Use% Mounted on >>> > /dev/bcache0 1.9T 26G 1.8T 2% /data33 >>> > /dev/sdd 1.9T 23G 1.8T 2% /data34 >>> > >>> > Wrote an 8GB file to each filesystem, then tried reading it back. >>> > Machine has 8GB RAM. >>> > >>> > Speed for random reads from identical SSD = 181.766523 MB/Sec, random >>> > read test done in the following order with measured speed in MB/Sec. The >>> > random IO is actually just reading with a 4GB stride, wrapping around at >>> > the end of the file. >>> > >>> > /data33 49 >>> > /data34 114 >>> > /data33 114 >>> > /data33 and /data44 simultaneously 78 and 59 respectively >>> > /data33 76 >>> > /data33 68 >>> > /data34 114 >>> > /data34 96 >>> > /data34 69 >>> > /data34 74 >>> > /data33 118 >>> > /data33 40 >>> > /data34 91 >>> > >>> > After remounting both filesystems: >>> > >>> > /data34 148 >>> > /data33 120 >>> > /data34 120 >>> > /data33 82 >>> > >>> > >>> > After unregistering the cache >>> > >>> > /data33 65 >>> > /data34 83 >>> > /data33 99 >>> > /data34 113 >>> > >>> > Cheers, >>> > Franco >>> > >>> > >>> > -- >>> > To unsubscribe from this list: send the line "unsubscribe linux-bcache" in >>> > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org >>> > More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in >> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2012-02-16 1:10 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-02-15 5:26 Can't resgister cache Franco
2012-02-15 6:24 ` Kent Overstreet
[not found] ` <CAH+dOx+g=eKDgTCnVdZYKrtn7tZjAnWeMdYP0KuZwD=rktsuJw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2012-02-15 7:16 ` Franco
2012-02-16 1:09 ` Kent Overstreet
[not found] ` <CAC7rs0sadP553tU0foAgDMzWh9euk4LyUYFL0_13rdeD+bBdWA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2012-02-16 1:10 ` Kent Overstreet
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.