Block Size for Windows

Block Size for Windows

[Jonathan Tripathy]

Hi Everyone,

I'm running bcache on my md raid device for use with my xen machine (via 
LVM). It works very well for Linux DomUs. However, I'm having some 
trouble installing Windows. When I enter the WIndows 2008 setup, it 
refuses to install to the LV volume. Reading the mailing lists, I feel 
this may have something to do with the block size that bcache is using.

Assuming that my cache is md0, and my spindles are md1, it is my 
understanding that to change the block size I would have to do:

make-bcache --block 512 --cache /dev/md0
and
make-bcache --block 512 --bdev /dev/md1

However, this doesn't appear to work is both of the above devices are 
already set for bcache (The block size seems to still be '1' according 
to the output that make-bcache gives).

However do I change the block size? I don't really mind if all data is 
lost, as this is just a test machine.

I am using the 3.2 branch of bcache

Many Thanks

Re: Block Size for Windows

[Kent Overstreet]

On Mon, Aug 27, 2012 at 12:14 PM, Jonathan Tripathy <jonnyt-Nf8S+5hNwl710XsdtD+oqA@public.gmane.org> wrote:
> Hi Everyone,
>
> I'm running bcache on my md raid device for use with my xen machine (via
> LVM). It works very well for Linux DomUs. However, I'm having some trouble
> installing Windows. When I enter the WIndows 2008 setup, it refuses to
> install to the LV volume. Reading the mailing lists, I feel this may have
> something to do with the block size that bcache is using.
>
> Assuming that my cache is md0, and my spindles are md1, it is my
> understanding that to change the block size I would have to do:
>
> make-bcache --block 512 --cache /dev/md0
> and
> make-bcache --block 512 --bdev /dev/md1
>
> However, this doesn't appear to work is both of the above devices are
> already set for bcache (The block size seems to still be '1' according to
> the output that make-bcache gives).

That's the output you want - it's displaying in units of 512 byte sectors

Re: Block Size for Windows

[Jonathan Tripathy]

On 27/08/2012 20:17, Kent Overstreet wrote:
> On Mon, Aug 27, 2012 at 12:14 PM, Jonathan Tripathy <jonnyt-Nf8S+5hNwl710XsdtD+oqA@public.gmane.org> wrote:
>> Hi Everyone,
>>
>> I'm running bcache on my md raid device for use with my xen machine (via
>> LVM). It works very well for Linux DomUs. However, I'm having some trouble
>> installing Windows. When I enter the WIndows 2008 setup, it refuses to
>> install to the LV volume. Reading the mailing lists, I feel this may have
>> something to do with the block size that bcache is using.
>>
>> Assuming that my cache is md0, and my spindles are md1, it is my
>> understanding that to change the block size I would have to do:
>>
>> make-bcache --block 512 --cache /dev/md0
>> and
>> make-bcache --block 512 --bdev /dev/md1
>>
>> However, this doesn't appear to work is both of the above devices are
>> already set for bcache (The block size seems to still be '1' according to
>> the output that make-bcache gives).
> That's the output you want - it's displaying in units of 512 byte sectors
Hi Kent,

Thanks for the reply. However, my device appears to still have a block 
size of 4.0k:

# cat /sys/fs/bcache/c215aa0a-c722-48e4-afcb-0ab8946d303c/block_size
4.0k

Maybe I have to deregister the devices first?

Thanks

Re: Block Size for Windows

[Kent Overstreet]

Yes - unregister, format, reregister

On Mon, Aug 27, 2012 at 12:19 PM, Jonathan Tripathy <jonnyt-Nf8S+5hNwl710XsdtD+oqA@public.gmane.org> wrote:
> On 27/08/2012 20:17, Kent Overstreet wrote:
>>
>> On Mon, Aug 27, 2012 at 12:14 PM, Jonathan Tripathy <jonnyt-Nf8S+5hNwl710XsdtD+oqA@public.gmane.org>
>> wrote:
>>>
>>> Hi Everyone,
>>>
>>> I'm running bcache on my md raid device for use with my xen machine (via
>>> LVM). It works very well for Linux DomUs. However, I'm having some
>>> trouble
>>> installing Windows. When I enter the WIndows 2008 setup, it refuses to
>>> install to the LV volume. Reading the mailing lists, I feel this may have
>>> something to do with the block size that bcache is using.
>>>
>>> Assuming that my cache is md0, and my spindles are md1, it is my
>>> understanding that to change the block size I would have to do:
>>>
>>> make-bcache --block 512 --cache /dev/md0
>>> and
>>> make-bcache --block 512 --bdev /dev/md1
>>>
>>> However, this doesn't appear to work is both of the above devices are
>>> already set for bcache (The block size seems to still be '1' according to
>>> the output that make-bcache gives).
>>
>> That's the output you want - it's displaying in units of 512 byte sectors
>
> Hi Kent,
>
> Thanks for the reply. However, my device appears to still have a block size
> of 4.0k:
>
> # cat /sys/fs/bcache/c215aa0a-c722-48e4-afcb-0ab8946d303c/block_size
> 4.0k
>
> Maybe I have to deregister the devices first?
>
>
> Thanks
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Block Size for Windows

[Jonathan Tripathy]

On 27/08/2012 20:19, Kent Overstreet wrote:
> Yes - unregister, format, reregister
>
>
What's the command to unregister?

Thanks

Re: Block Size for Windows

[Kent Overstreet]

echo 1 > /sys/fs/bcache/<uuid>/unregister

echo 1 > /sys/block/bcache0/bcache/stop

On Mon, Aug 27, 2012 at 12:20 PM, Jonathan Tripathy <jonnyt-Nf8S+5hNwl710XsdtD+oqA@public.gmane.org> wrote:
> On 27/08/2012 20:19, Kent Overstreet wrote:
>>
>> Yes - unregister, format, reregister
>>
>>
> What's the command to unregister?
>
> Thanks
>

Re: Block Size for Windows

[Jonathan Tripathy]

On 27/08/2012 20:22, Kent Overstreet wrote:
> echo 1 > /sys/fs/bcache/<uuid>/unregister
>
> echo 1 > /sys/block/bcache0/bcache/stop
Ok, so I managed to recreate everything with a block size of 512 bytes. 
However, 2 bad things have happened:

1) In my Linux DomU, my fio randomwrite test have gone from about 28k 
iops to about 800 iops. Can this be fixed? changing bs and ba to 512 in 
the fio config file didn't hep.

2) The windows setup didn't complain that it couldn't install on the LV, 
but once I clicked 'next', the Dom0 crashed and the server rebooted. A 
lot of output was displayed on screen but quickly vanished as the system 
rebooted. I'm trying to see if the output was saved anywhere. Any ideas 
why this could of happened and/or where the output might be saved?

Thanks

Re: Block Size for Windows

[Kent Overstreet]

On Mon, Aug 27, 2012 at 12:43 PM, Jonathan Tripathy <jonnyt-Nf8S+5hNwl710XsdtD+oqA@public.gmane.org> wrote:
> On 27/08/2012 20:22, Kent Overstreet wrote:
>>
>> echo 1 > /sys/fs/bcache/<uuid>/unregister
>>
>> echo 1 > /sys/block/bcache0/bcache/stop
>
> Ok, so I managed to recreate everything with a block size of 512 bytes.
> However, 2 bad things have happened:
>
> 1) In my Linux DomU, my fio randomwrite test have gone from about 28k iops
> to about 800 iops. Can this be fixed? changing bs and ba to 512 in the fio
> config file didn't hep.

Ouch!

Bet the ssd doesn't like those 512 byte journal writes. I'm going to
have to think about that...

Re: Block Size for Windows

[Jonathan Tripathy]

>
> 2) The windows setup didn't complain that it couldn't install on the 
> LV, but once I clicked 'next', the Dom0 crashed and the server 
> rebooted. A lot of output was displayed on screen but quickly vanished 
> as the system rebooted. I'm trying to see if the output was saved 
> anywhere. Any ideas why this could of happened and/or where the output 
> might be saved?
>
>
I'd also like to add that after the server came back up, the md raid 
array started rebuilding. I wondering if that's just a coincidence (due 
to the forced reboot), or a sign of something wrong with the md 
integration with bcache?

I'm going to see if Windows installs natively on the md array (it's RAID 
10 btw) and post back here.

Re: Block Size for Windows

[Jonathan Tripathy]

On 27/08/2012 21:00, Jonathan Tripathy wrote:
>
>>
>> 2) The windows setup didn't complain that it couldn't install on the 
>> LV, but once I clicked 'next', the Dom0 crashed and the server 
>> rebooted. A lot of output was displayed on screen but quickly 
>> vanished as the system rebooted. I'm trying to see if the output was 
>> saved anywhere. Any ideas why this could of happened and/or where the 
>> output might be saved?
>>
>>
> I'd also like to add that after the server came back up, the md raid 
> array started rebuilding. I wondering if that's just a coincidence 
> (due to the forced reboot), or a sign of something wrong with the md 
> integration with bcache?
>
> I'm going to see if Windows installs natively on the md array (it's 
> RAID 10 btw) and post back here.

Ok, so trying to install Windows directly onto the spindles causes the 
same thing to happen. I'm going to try and boot up into the non-bcache 
kernel (The default ubuntu one) and see if it works there. If it fails 
there, then this is clearly a xen and/or mdraid issue...

Thanks

Re: Block Size for Windows

[Jonathan Tripathy]

On 27/08/2012 21:07, Jonathan Tripathy wrote:
> On 27/08/2012 21:00, Jonathan Tripathy wrote:
>>
>>>
>>> 2) The windows setup didn't complain that it couldn't install on the 
>>> LV, but once I clicked 'next', the Dom0 crashed and the server 
>>> rebooted. A lot of output was displayed on screen but quickly 
>>> vanished as the system rebooted. I'm trying to see if the output was 
>>> saved anywhere. Any ideas why this could of happened and/or where 
>>> the output might be saved?
>>>
>>>
>> I'd also like to add that after the server came back up, the md raid 
>> array started rebuilding. I wondering if that's just a coincidence 
>> (due to the forced reboot), or a sign of something wrong with the md 
>> integration with bcache?
>>
>> I'm going to see if Windows installs natively on the md array (it's 
>> RAID 10 btw) and post back here.
>
> Ok, so trying to install Windows directly onto the spindles causes the 
> same thing to happen. I'm going to try and boot up into the non-bcache 
> kernel (The default ubuntu one) and see if it works there. If it fails 
> there, then this is clearly a xen and/or mdraid issue...
>
> Thanks
>
>
Ok, so booting into the default Ubuntu kernel, the windows installation 
seems to progress just fine.

Does this mean there is something wrong with the mdraid code in the 
bcache kernel?

Actually, I'm not telling the whole story. The kernel I'm using is the 
bcache-3.2 tree (from evilpriate.org) with changes merged in from 
kernel.org's 3.2.27 tree. There were no merge conflicts when I did the 
git merge.

What do you think I should do?

Thanks

Re: Block Size for Windows

[Joseph Glanville]

On 28 August 2012 06:15, Jonathan Tripathy <jonnyt-Nf8S+5hNwl710XsdtD+oqA@public.gmane.org> wrote:
> On 27/08/2012 21:07, Jonathan Tripathy wrote:
>>
>> On 27/08/2012 21:00, Jonathan Tripathy wrote:
>>>
>>>
>>>>
>>>> 2) The windows setup didn't complain that it couldn't install on the LV,
>>>> but once I clicked 'next', the Dom0 crashed and the server rebooted. A lot
>>>> of output was displayed on screen but quickly vanished as the system
>>>> rebooted. I'm trying to see if the output was saved anywhere. Any ideas why
>>>> this could of happened and/or where the output might be saved?
>>>>
>>>>
>>> I'd also like to add that after the server came back up, the md raid
>>> array started rebuilding. I wondering if that's just a coincidence (due to
>>> the forced reboot), or a sign of something wrong with the md integration
>>> with bcache?
>>>
>>> I'm going to see if Windows installs natively on the md array (it's RAID
>>> 10 btw) and post back here.
>>
>>
>> Ok, so trying to install Windows directly onto the spindles causes the
>> same thing to happen. I'm going to try and boot up into the non-bcache
>> kernel (The default ubuntu one) and see if it works there. If it fails
>> there, then this is clearly a xen and/or mdraid issue...
>>
>> Thanks
>>
>>
> Ok, so booting into the default Ubuntu kernel, the windows installation
> seems to progress just fine.
>
> Does this mean there is something wrong with the mdraid code in the bcache
> kernel?
>
> Actually, I'm not telling the whole story. The kernel I'm using is the
> bcache-3.2 tree (from evilpriate.org) with changes merged in from
> kernel.org's 3.2.27 tree. There were no merge conflicts when I did the git
> merge.
>
> What do you think I should do?
>
>
> Thanks
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

I would recommend booting with the raw bcache-3.2 branch before
applying the stable patches (even though they should be fine) and
trying to catch the panic.
This is easiest done with a serial port and setting it to the kernel
console on the kernel command line in grub.

Joseph.

-- 
CTO | Orion Virtualisation Solutions | www.orionvm.com.au
Phone: 1300 56 99 52 | Mobile: 0428 754 846

Re: Block Size for Windows

[Jonathan Tripathy]

On 27/08/2012 21:18, Joseph Glanville wrote:
> On 28 August 2012 06:15, Jonathan Tripathy <jonnyt-Nf8S+5hNwl710XsdtD+oqA@public.gmane.org> wrote:
>> On 27/08/2012 21:07, Jonathan Tripathy wrote:
>>> On 27/08/2012 21:00, Jonathan Tripathy wrote:
>>>>
>>>>> 2) The windows setup didn't complain that it couldn't install on the LV,
>>>>> but once I clicked 'next', the Dom0 crashed and the server rebooted. A lot
>>>>> of output was displayed on screen but quickly vanished as the system
>>>>> rebooted. I'm trying to see if the output was saved anywhere. Any ideas why
>>>>> this could of happened and/or where the output might be saved?
>>>>>
>>>>>
>>>> I'd also like to add that after the server came back up, the md raid
>>>> array started rebuilding. I wondering if that's just a coincidence (due to
>>>> the forced reboot), or a sign of something wrong with the md integration
>>>> with bcache?
>>>>
>>>> I'm going to see if Windows installs natively on the md array (it's RAID
>>>> 10 btw) and post back here.
>>>
>>> Ok, so trying to install Windows directly onto the spindles causes the
>>> same thing to happen. I'm going to try and boot up into the non-bcache
>>> kernel (The default ubuntu one) and see if it works there. If it fails
>>> there, then this is clearly a xen and/or mdraid issue...
>>>
>>> Thanks
>>>
>>>
>> Ok, so booting into the default Ubuntu kernel, the windows installation
>> seems to progress just fine.
>>
>> Does this mean there is something wrong with the mdraid code in the bcache
>> kernel?
>>
>> Actually, I'm not telling the whole story. The kernel I'm using is the
>> bcache-3.2 tree (from evilpriate.org) with changes merged in from
>> kernel.org's 3.2.27 tree. There were no merge conflicts when I did the git
>> merge.
>>
>> What do you think I should do?
>>
>>
>> Thanks
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> I would recommend booting with the raw bcache-3.2 branch before
> applying the stable patches (even though they should be fine) and
> trying to catch the panic.
> This is easiest done with a serial port and setting it to the kernel
> console on the kernel command line in grub.
>
> Joseph.
Thanks Joseph.

I'll do a clean build of the raw bcache-3.2 tree and try my windows 
install again. I'll also try and find a serial cable with the correct 
genders..

Will keep you updated

Thanks

Re: Block Size for Windows

[Jonathan Tripathy]

On 27/08/2012 21:18, Joseph Glanville wrote:
> On 28 August 2012 06:15, Jonathan Tripathy <jonnyt-Nf8S+5hNwl710XsdtD+oqA@public.gmane.org> wrote:
>> On 27/08/2012 21:07, Jonathan Tripathy wrote:
>>> On 27/08/2012 21:00, Jonathan Tripathy wrote:
>>>>
>>>>> 2) The windows setup didn't complain that it couldn't install on the LV,
>>>>> but once I clicked 'next', the Dom0 crashed and the server rebooted. A lot
>>>>> of output was displayed on screen but quickly vanished as the system
>>>>> rebooted. I'm trying to see if the output was saved anywhere. Any ideas why
>>>>> this could of happened and/or where the output might be saved?
>>>>>
>>>>>
>>>> I'd also like to add that after the server came back up, the md raid
>>>> array started rebuilding. I wondering if that's just a coincidence (due to
>>>> the forced reboot), or a sign of something wrong with the md integration
>>>> with bcache?
>>>>
>>>> I'm going to see if Windows installs natively on the md array (it's RAID
>>>> 10 btw) and post back here.
>>>
>>> Ok, so trying to install Windows directly onto the spindles causes the
>>> same thing to happen. I'm going to try and boot up into the non-bcache
>>> kernel (The default ubuntu one) and see if it works there. If it fails
>>> there, then this is clearly a xen and/or mdraid issue...
>>>
>>> Thanks
>>>
>>>
>> Ok, so booting into the default Ubuntu kernel, the windows installation
>> seems to progress just fine.
>>
>> Does this mean there is something wrong with the mdraid code in the bcache
>> kernel?
>>
>> Actually, I'm not telling the whole story. The kernel I'm using is the
>> bcache-3.2 tree (from evilpriate.org) with changes merged in from
>> kernel.org's 3.2.27 tree. There were no merge conflicts when I did the git
>> merge.
>>
>> What do you think I should do?
>>
>>
>> Thanks
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> I would recommend booting with the raw bcache-3.2 branch before
> applying the stable patches (even though they should be fine) and
> trying to catch the panic.
> This is easiest done with a serial port and setting it to the kernel
> console on the kernel command line in grub.
>
> Joseph.
Hi There,

I can confirm that the problem occurs even when using the raw bcache-3.2 
branch from evilpirate.org. Just to clarify, I am trying to install 
Windows Server 2008 in a Xen HVM DomU, onto an LV which is on top of a 
MDRAID 10 array. Using the bcache-3.2 kernel, the system reboots (after 
panicing) as soon as I click 'next' after selecting the drive to install 
windows onto. Using the standard Ubuntu kernel everything works as 
normal. This leads me to believe that there is an issue with the mdraid 
code inside the bcache-3.2 tree. I'd like to stress that I wasn't doing 
any bcaching during this test.

What should my next step be? Try and find a serial cable to capture the 
debug output?

Thanks

Re: Block Size for Windows

[Joseph Glanville]

On 28 August 2012 08:02, Jonathan Tripathy <jonnyt-Nf8S+5hNwl710XsdtD+oqA@public.gmane.org> wrote:
> On 27/08/2012 21:18, Joseph Glanville wrote:
>>
>> On 28 August 2012 06:15, Jonathan Tripathy <jonnyt-Nf8S+5hNwl710XsdtD+oqA@public.gmane.org> wrote:
>>>
>>> On 27/08/2012 21:07, Jonathan Tripathy wrote:
>>>>
>>>> On 27/08/2012 21:00, Jonathan Tripathy wrote:
>>>>>
>>>>>
>>>>>> 2) The windows setup didn't complain that it couldn't install on the
>>>>>> LV,
>>>>>> but once I clicked 'next', the Dom0 crashed and the server rebooted. A
>>>>>> lot
>>>>>> of output was displayed on screen but quickly vanished as the system
>>>>>> rebooted. I'm trying to see if the output was saved anywhere. Any
>>>>>> ideas why
>>>>>> this could of happened and/or where the output might be saved?
>>>>>>
>>>>>>
>>>>> I'd also like to add that after the server came back up, the md raid
>>>>> array started rebuilding. I wondering if that's just a coincidence (due
>>>>> to
>>>>> the forced reboot), or a sign of something wrong with the md
>>>>> integration
>>>>> with bcache?
>>>>>
>>>>> I'm going to see if Windows installs natively on the md array (it's
>>>>> RAID
>>>>> 10 btw) and post back here.
>>>>
>>>>
>>>> Ok, so trying to install Windows directly onto the spindles causes the
>>>> same thing to happen. I'm going to try and boot up into the non-bcache
>>>> kernel (The default ubuntu one) and see if it works there. If it fails
>>>> there, then this is clearly a xen and/or mdraid issue...
>>>>
>>>> Thanks
>>>>
>>>>
>>> Ok, so booting into the default Ubuntu kernel, the windows installation
>>> seems to progress just fine.
>>>
>>> Does this mean there is something wrong with the mdraid code in the
>>> bcache
>>> kernel?
>>>
>>> Actually, I'm not telling the whole story. The kernel I'm using is the
>>> bcache-3.2 tree (from evilpriate.org) with changes merged in from
>>> kernel.org's 3.2.27 tree. There were no merge conflicts when I did the
>>> git
>>> merge.
>>>
>>> What do you think I should do?
>>>
>>>
>>> Thanks
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-bcache"
>>> in
>>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>> I would recommend booting with the raw bcache-3.2 branch before
>> applying the stable patches (even though they should be fine) and
>> trying to catch the panic.
>> This is easiest done with a serial port and setting it to the kernel
>> console on the kernel command line in grub.
>>
>> Joseph.
>
> Hi There,
>
> I can confirm that the problem occurs even when using the raw bcache-3.2
> branch from evilpirate.org. Just to clarify, I am trying to install Windows
> Server 2008 in a Xen HVM DomU, onto an LV which is on top of a MDRAID 10
> array. Using the bcache-3.2 kernel, the system reboots (after panicing) as
> soon as I click 'next' after selecting the drive to install windows onto.
> Using the standard Ubuntu kernel everything works as normal. This leads me
> to believe that there is an issue with the mdraid code inside the bcache-3.2
> tree. I'd like to stress that I wasn't doing any bcaching during this test.
>
> What should my next step be? Try and find a serial cable to capture the
> debug output?
>
> Thanks
>

The issue is probably in fs/bio.c, some of the bio splitting changes
could effect md without even having bcache in use.

Kent, I will try get a test box up running more recent code then the
stuff we run (which doesn't have said issue).

Joseph.

-- 
CTO | Orion Virtualisation Solutions | www.orionvm.com.au
Phone: 1300 56 99 52 | Mobile: 0428 754 846

RE: Block Size for Windows

[James Harper]

> 
> On 27/08/2012 21:18, Joseph Glanville wrote:
> > On 28 August 2012 06:15, Jonathan Tripathy <jonnyt-Nf8S+5hNwl710XsdtD+oqA@public.gmane.org> wrote:
> >> On 27/08/2012 21:07, Jonathan Tripathy wrote:
> >>> On 27/08/2012 21:00, Jonathan Tripathy wrote:
> >>>>
> >>>>> 2) The windows setup didn't complain that it couldn't install on
> >>>>> the LV, but once I clicked 'next', the Dom0 crashed and the server
> >>>>> rebooted. A lot of output was displayed on screen but quickly
> >>>>> vanished as the system rebooted. I'm trying to see if the output
> >>>>> was saved anywhere. Any ideas why this could of happened and/or
> where the output might be saved?
> >>>>>
> >>>>>
> >>>> I'd also like to add that after the server came back up, the md
> >>>> raid array started rebuilding. I wondering if that's just a
> >>>> coincidence (due to the forced reboot), or a sign of something
> >>>> wrong with the md integration with bcache?
> >>>>
> >>>> I'm going to see if Windows installs natively on the md array (it's
> >>>> RAID
> >>>> 10 btw) and post back here.
> >>>
> >>> Ok, so trying to install Windows directly onto the spindles causes
> >>> the same thing to happen. I'm going to try and boot up into the
> >>> non-bcache kernel (The default ubuntu one) and see if it works
> >>> there. If it fails there, then this is clearly a xen and/or mdraid issue...
> >>>
> >>> Thanks
> >>>
> >>>
> >> Ok, so booting into the default Ubuntu kernel, the windows
> >> installation seems to progress just fine.
> >>
> >> Does this mean there is something wrong with the mdraid code in the
> >> bcache kernel?
> >>
> >> Actually, I'm not telling the whole story. The kernel I'm using is
> >> the
> >> bcache-3.2 tree (from evilpriate.org) with changes merged in from
> >> kernel.org's 3.2.27 tree. There were no merge conflicts when I did
> >> the git merge.
> >>
> >> What do you think I should do?
> >>
> >>
> >> Thanks
> >>
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe
> >> linux-bcache" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > I would recommend booting with the raw bcache-3.2 branch before
> > applying the stable patches (even though they should be fine) and
> > trying to catch the panic.
> > This is easiest done with a serial port and setting it to the kernel
> > console on the kernel command line in grub.
> >
> > Joseph.
> Hi There,
> 
> I can confirm that the problem occurs even when using the raw bcache-3.2
> branch from evilpirate.org. Just to clarify, I am trying to install Windows
> Server 2008 in a Xen HVM DomU, onto an LV which is on top of a MDRAID 10
> array. Using the bcache-3.2 kernel, the system reboots (after
> panicing) as soon as I click 'next' after selecting the drive to install windows
> onto. Using the standard Ubuntu kernel everything works as normal. This
> leads me to believe that there is an issue with the mdraid code inside the
> bcache-3.2 tree. I'd like to stress that I wasn't doing any bcaching during this
> test.
> 

FWIW, i'm using the 3.2 patches applied to a Debian kernel with lvm on raid1 (not raid10) on bcache and it's all working fine since I changed to a 512 byte block size. I haven't done an install of 2008, just 2003, but there doesn't seem to be any problems.

> What should my next step be? Try and find a serial cable to capture the
> debug output?
> 

Before tinkering with a serial cable, see if the system is alive enough to use netconsole - it can be a bit of a timesaver.

James

Re: Block Size for Windows

[Brad Campbell]

On 28/08/12 06:02, Jonathan Tripathy wrote:
>
> What should my next step be? Try and find a serial cable to capture the debug output?
>

If the machine has a network port and you have another machine nearby you could use netconsole. It's 
much quicker and easier than trying to cobble up a serial cable if you don't already have one handy.

You'll find the documentation in your kernel source tree here ---> 
Documentation/networking/netconsole.txt

I have my server set up to log to the rsyslog daemon on my desktop machine. This way if it panics 
and auto-reboots, I get a good copy of the oops.

Regards,
Brad

RE: Block Size for Windows

[Jonathan Tripathy]

On 28.08.2012 01:59, James Harper wrote:
>>
>> On 27/08/2012 21:18, Joseph Glanville wrote:
>> > On 28 August 2012 06:15, Jonathan Tripathy <jonnyt-Nf8S+5hNwl710XsdtD+oqA@public.gmane.org> 
>> wrote:
>> >> On 27/08/2012 21:07, Jonathan Tripathy wrote:
>> >>> On 27/08/2012 21:00, Jonathan Tripathy wrote:
>> >>>>
>> >>>>> 2) The windows setup didn't complain that it couldn't install 
>> on
>> >>>>> the LV, but once I clicked 'next', the Dom0 crashed and the 
>> server
>> >>>>> rebooted. A lot of output was displayed on screen but quickly
>> >>>>> vanished as the system rebooted. I'm trying to see if the 
>> output
>> >>>>> was saved anywhere. Any ideas why this could of happened 
>> and/or
>> where the output might be saved?
>> >>>>>
>> >>>>>
>> >>>> I'd also like to add that after the server came back up, the md
>> >>>> raid array started rebuilding. I wondering if that's just a
>> >>>> coincidence (due to the forced reboot), or a sign of something
>> >>>> wrong with the md integration with bcache?
>> >>>>
>> >>>> I'm going to see if Windows installs natively on the md array 
>> (it's
>> >>>> RAID
>> >>>> 10 btw) and post back here.
>> >>>
>> >>> Ok, so trying to install Windows directly onto the spindles 
>> causes
>> >>> the same thing to happen. I'm going to try and boot up into the
>> >>> non-bcache kernel (The default ubuntu one) and see if it works
>> >>> there. If it fails there, then this is clearly a xen and/or 
>> mdraid issue...
>> >>>
>> >>> Thanks
>> >>>
>> >>>
>> >> Ok, so booting into the default Ubuntu kernel, the windows
>> >> installation seems to progress just fine.
>> >>
>> >> Does this mean there is something wrong with the mdraid code in 
>> the
>> >> bcache kernel?
>> >>
>> >> Actually, I'm not telling the whole story. The kernel I'm using 
>> is
>> >> the
>> >> bcache-3.2 tree (from evilpriate.org) with changes merged in from
>> >> kernel.org's 3.2.27 tree. There were no merge conflicts when I 
>> did
>> >> the git merge.
>> >>
>> >> What do you think I should do?
>> >>
>> >>
>> >> Thanks
>> >>
>> >> --
>> >> To unsubscribe from this list: send the line "unsubscribe
>> >> linux-bcache" in the body of a message to 
>> majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> >> More majordomo info at  
>> http://vger.kernel.org/majordomo-info.html
>> > I would recommend booting with the raw bcache-3.2 branch before
>> > applying the stable patches (even though they should be fine) and
>> > trying to catch the panic.
>> > This is easiest done with a serial port and setting it to the 
>> kernel
>> > console on the kernel command line in grub.
>> >
>> > Joseph.
>> Hi There,
>>
>> I can confirm that the problem occurs even when using the raw 
>> bcache-3.2
>> branch from evilpirate.org. Just to clarify, I am trying to install 
>> Windows
>> Server 2008 in a Xen HVM DomU, onto an LV which is on top of a 
>> MDRAID 10
>> array. Using the bcache-3.2 kernel, the system reboots (after
>> panicing) as soon as I click 'next' after selecting the drive to 
>> install windows
>> onto. Using the standard Ubuntu kernel everything works as normal. 
>> This
>> leads me to believe that there is an issue with the mdraid code 
>> inside the
>> bcache-3.2 tree. I'd like to stress that I wasn't doing any bcaching 
>> during this
>> test.
>>
>
> FWIW, i'm using the 3.2 patches applied to a Debian kernel with lvm
> on raid1 (not raid10) on bcache and it's all working fine since I
> changed to a 512 byte block size. I haven't done an install of 2008,
> just 2003, but there doesn't seem to be any problems.
>
>

Hi James,

That's very interesting. However, my problem occurs regardless of 
whether I'm using a bcache or not. I will try to see if the problems 
happens with Windows 2003 and I'll also try RAID1.

That's a good idea about the netconsole, I'll give that a go!

Out of interest, do you see poor fio benchmark performance when using a 
512 byte block size?

Re: Block Size for Windows

[Jonathan Tripathy]

On 28/08/2012 01:59, James Harper wrote:
>> On 27/08/2012 21:18, Joseph Glanville wrote:
>>> On 28 August 2012 06:15, Jonathan Tripathy <jonnyt-Nf8S+5hNwl710XsdtD+oqA@public.gmane.org> wrote:
>>>> On 27/08/2012 21:07, Jonathan Tripathy wrote:
>>>>> On 27/08/2012 21:00, Jonathan Tripathy wrote:
>>>>>>> 2) The windows setup didn't complain that it couldn't install on
>>>>>>> the LV, but once I clicked 'next', the Dom0 crashed and the server
>>>>>>> rebooted. A lot of output was displayed on screen but quickly
>>>>>>> vanished as the system rebooted. I'm trying to see if the output
>>>>>>> was saved anywhere. Any ideas why this could of happened and/or
>> where the output might be saved?
>>>>>>>
>>>>>> I'd also like to add that after the server came back up, the md
>>>>>> raid array started rebuilding. I wondering if that's just a
>>>>>> coincidence (due to the forced reboot), or a sign of something
>>>>>> wrong with the md integration with bcache?
>>>>>>
>>>>>> I'm going to see if Windows installs natively on the md array (it's
>>>>>> RAID
>>>>>> 10 btw) and post back here.
>>>>> Ok, so trying to install Windows directly onto the spindles causes
>>>>> the same thing to happen. I'm going to try and boot up into the
>>>>> non-bcache kernel (The default ubuntu one) and see if it works
>>>>> there. If it fails there, then this is clearly a xen and/or mdraid issue...
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>> Ok, so booting into the default Ubuntu kernel, the windows
>>>> installation seems to progress just fine.
>>>>
>>>> Does this mean there is something wrong with the mdraid code in the
>>>> bcache kernel?
>>>>
>>>> Actually, I'm not telling the whole story. The kernel I'm using is
>>>> the
>>>> bcache-3.2 tree (from evilpriate.org) with changes merged in from
>>>> kernel.org's 3.2.27 tree. There were no merge conflicts when I did
>>>> the git merge.
>>>>
>>>> What do you think I should do?
>>>>
>>>>
>>>> Thanks
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe
>>>> linux-bcache" in the body of a message to majordomo-u79uwXL29TaqPxH82wqD4g@public.gmane.orgg
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> I would recommend booting with the raw bcache-3.2 branch before
>>> applying the stable patches (even though they should be fine) and
>>> trying to catch the panic.
>>> This is easiest done with a serial port and setting it to the kernel
>>> console on the kernel command line in grub.
>>>
>>> Joseph.
>> Hi There,
>>
>> I can confirm that the problem occurs even when using the raw bcache-3.2
>> branch from evilpirate.org. Just to clarify, I am trying to install Windows
>> Server 2008 in a Xen HVM DomU, onto an LV which is on top of a MDRAID 10
>> array. Using the bcache-3.2 kernel, the system reboots (after
>> panicing) as soon as I click 'next' after selecting the drive to install windows
>> onto. Using the standard Ubuntu kernel everything works as normal. This
>> leads me to believe that there is an issue with the mdraid code inside the
>> bcache-3.2 tree. I'd like to stress that I wasn't doing any bcaching during this
>> test.
>>
> FWIW, i'm using the 3.2 patches applied to a Debian kernel with lvm on raid1 (not raid10) on bcache and it's all working fine since I changed to a 512 byte block size. I haven't done an install of 2008, just 2003, but there doesn't seem to be any problems.
>
>> What should my next step be? Try and find a serial cable to capture the
>> debug output?
>>
> Before tinkering with a serial cable, see if the system is alive enough to use netconsole - it can be a bit of a timesaver.
>
> James
Hi Everyone,

Here is the trace as capture by netconsole:

[ 130.844069] ------------[ cut here ]------------
[ 130.844165] kernel BUG at fs/bio.c:420!
[ 130.844232] invalid opcode: 0000 [#1] SMP
[ 130.844404] CPU 4
[ 130.844448] Modules linked in: xen_netback xen_blkback 8021q garp 
xt_physdev bridge stp ebtable_filter ebtables ip6table_filter ip6_tables 
iptable_filter ip_tables x_tables xen_gntdev xen_evtchn xenfs 
nls_iso8859_1 nls_cp437 vfat fat netconsole configfs psmouse lp joydev 
parport video mac_hid serio_raw usb_storage uas raid456 async_pq 
async_xor xor async_memcpy async_raid6_recov usbhid hid raid10 e1000e 
raid6_pq async_tx raid1 raid0 multipath linear
[ 130.846688]
[ 130.846746] Pid: 0, comm: swapper/4 Not tainted 3.2.0+ #1 Supermicro 
X9SCI/X9SCA/X9SCI/X9SCA
[ 130.846956] RIP: e030:[<ffffffff811a6ef7>] [<ffffffff811a6ef7>] 
bio_put+0x27/0x30
[ 130.847089] RSP: e02b:ffff88005ff3cb80 EFLAGS: 00010246
[ 130.847155] RAX: 0000000000000000 RBX: 00000000fffffffb RCX: 
00000000000003a6
[ 130.847224] RDX: 00000000000003a5 RSI: 0000000000016c00 RDI: 
ffff880039b58918
[ 130.847293] RBP: ffff88005ff3cb80 R08: ffffffff81115e67 R09: 
0000000000000100
[ 130.847362] R10: ffff88001a16eea0 R11: 0000000000000000 R12: 
ffff880017fd4018
[ 130.847431] R13: ffff880039b58918 R14: ffff880017220028 R15: 
ffff88001a6dd400
[ 130.847504] FS: 00007f507e004700(0000) GS:ffff88005ff39000(0000) 
knlGS:0000000000000000
[ 130.847590] CS: e033 DS: 002b ES: 002b CR0: 000000008005003b
[ 130.847656] CR2: 00007f507d6910b0 CR3: 000000003989a000 CR4: 
0000000000002660
[ 130.847725] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
0000000000000000
[ 130.847794] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
0000000000000400
[ 130.847863] Process swapper/4 (pid: 0, threadinfo ffff88003d646000, 
task ffff88003d648000)
[ 130.847952] Stack:
[ 130.848012] ffff88005ff3cbc0 ffffffff8150238e ffff88003d6e40b0 
ffff880039b58900
[ 130.848258] ffff880039b58920 ffff88001a278100 0000000000000018 
0000000000000001
[ 130.848505] ffff88005ff3cbd0 ffffffff811a5f5d ffff88005ff3cbf0 
ffffffff811a6f3e
[ 130.848749] Call Trace:
[ 130.848810] <IRQ>
[ 130.848914] [<ffffffff8150238e>] clone_endio+0x8e/0xd0
[ 130.848979] [<ffffffff811a5f5d>] bio_endio+0x1d/0x40
[ 130.849047] [<ffffffff811a6f3e>] bio_pair_release+0x3e/0x50
[ 130.849113] [<ffffffff811a6f6f>] bio_pair_end+0x1f/0x30
[ 130.849180] [<ffffffff811a5f5d>] bio_endio+0x1d/0x40
[ 130.849248] [<ffffffffa00dd3d2>] raid_end_bio_io+0xf2/0x100 [raid10]
[ 130.849319] [<ffffffffa00ddf38>] one_write_done+0x38/0x50 [raid10]
[ 130.849390] [<ffffffffa00deec4>] raid10_end_write_request+0xc4/0x130 
[raid10]
[ 130.849476] [<ffffffff811a5f5d>] bio_endio+0x1d/0x40
[ 130.849543] [<ffffffff812e7a03>] req_bio_endio.isra.49+0xa3/0xe0
[ 130.849614] [<ffffffff812e839d>] blk_update_request+0xfd/0x480
[ 130.849681] [<ffffffff812e8751>] blk_update_bidi_request+0x31/0x90
[ 130.849751] [<ffffffff812e9a4c>] blk_end_bidi_request+0x2c/0x80
[ 130.849819] [<ffffffff812e9ae0>] blk_end_request+0x10/0x20
[ 130.849888] [<ffffffff8141d89f>] scsi_io_completion+0xaf/0x630
[ 130.849960] [<ffffffff8165c88e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
[ 130.850031] [<ffffffff81413d31>] scsi_finish_command+0xc1/0x120
[ 130.850098] [<ffffffff8141d6fe>] scsi_softirq_done+0x13e/0x150
[ 130.850167] [<ffffffff812ef9f3>] blk_done_softirq+0x83/0xa0
[ 130.850237] [<ffffffff8106d4d8>] __do_softirq+0xa8/0x210
[ 130.850304] [<ffffffff8139ddf7>] ? __xen_evtchn_do_upcall+0x207/0x250
[ 130.850373] [<ffffffff816669ac>] call_softirq+0x1c/0x30
[ 130.850442] [<ffffffff81015195>] do_softirq+0x65/0xa0
[ 130.850507] [<ffffffff8106d8be>] irq_exit+0x8e/0xb0
[ 130.850574] [<ffffffff8139fbb5>] xen_evtchn_do_upcall+0x35/0x50
[ 130.850643] [<ffffffff816669fe>] xen_do_hypervisor_callback+0x1e/0x30
[ 130.850711] <EOI>
[ 130.850810] [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
[ 130.850881] [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
[ 130.850950] [<ffffffff8100a170>] ? xen_safe_halt+0x10/0x20
[ 130.851017] [<ffffffff8101b5a3>] ? default_idle+0x53/0x1d0
[ 130.851085] [<ffffffff81012236>] ? cpu_idle+0xd6/0x120
[ 130.851153] [<ffffffff8100a9a9>] ? xen_irq_enable_direct_reloc+0x4/0x4
[ 130.851223] [<ffffffff81643992>] ? cpu_bringup_and_idle+0xe/0x10
[ 130.851291] Code: 00 00 00 00 55 48 89 e5 66 66 66 66 90 8b 47 40 85 
c0 74 17 f0 ff 4f 40 0f 94 c0 84 c0 75 05 5d c3 0f 1f 00 e8 2b ff ff ff 
5d c3 <0f> 0b 0f 1f 80 00 00 00 00 55 48 89 e5 53 48 83 ec 08 66 66 66
[ 130.854057] RIP [<ffffffff811a6ef7>] bio_put+0x27/0x30
[ 130.854163] RSP <ffff88005ff3cb80>
[ 130.854226] ---[ end trace fa3fbcc21926358a ]---
[ 130.855127] Kernel panic - not syncing: Fatal exception in interrupt
[ 130.855198] Pid: 0, comm: swapper/4 Tainted: G D 3.2.0+ #1
[ 130.855267] Call Trace:
[ 130.855327] <IRQ> [<ffffffff816516ef>] panic+0x91/0x1a7
[ 130.855438] [<ffffffff8165d85a>] oops_end+0xea/0xf0
[ 130.855505] [<ffffffff81016708>] die+0x58/0x90
[ 130.855571] [<ffffffff8165d194>] do_trap+0xc4/0x170
[ 130.855636] [<ffffffff81013e25>] do_invalid_op+0x95/0xb0
[ 130.855702] [<ffffffff811a6ef7>] ? bio_put+0x27/0x30
[ 130.855767] [<ffffffff8100a23d>] ? xen_force_evtchn_callback+0xd/0x10
[ 130.855838] [<ffffffff8100aa02>] ? check_events+0x12/0x20
[ 130.855906] [<ffffffff8166672b>] invalid_op+0x1b/0x20
[ 130.855974] [<ffffffff81115e67>] ? mempool_free_slab+0x17/0x20
[ 130.856041] [<ffffffff811a6ef7>] ? bio_put+0x27/0x30
[ 130.856109] [<ffffffff8150238e>] clone_endio+0x8e/0xd0
[ 130.856175] [<ffffffff811a5f5d>] bio_endio+0x1d/0x40
[ 130.856241] [<ffffffff811a6f3e>] bio_pair_release+0x3e/0x50
[ 130.856308] [<ffffffff811a6f6f>] bio_pair_end+0x1f/0x30
[ 130.856374] [<ffffffff811a5f5d>] bio_endio+0x1d/0x40
[ 130.856443] [<ffffffffa00dd3d2>] raid_end_bio_io+0xf2/0x100 [raid10]
[ 130.856513] [<ffffffffa00ddf38>] one_write_done+0x38/0x50 [raid10]
[ 130.856584] [<ffffffffa00deec4>] raid10_end_write_request+0xc4/0x130 
[raid10]
[ 130.856671] [<ffffffff811a5f5d>] bio_endio+0x1d/0x40
[ 130.856738] [<ffffffff812e7a03>] req_bio_endio.isra.49+0xa3/0xe0
[ 130.856807] [<ffffffff812e839d>] blk_update_request+0xfd/0x480
[ 130.856877] [<ffffffff812e8751>] blk_update_bidi_request+0x31/0x90
[ 130.856946] [<ffffffff812e9a4c>] blk_end_bidi_request+0x2c/0x80
[ 130.857014] [<ffffffff812e9ae0>] blk_end_request+0x10/0x20
[ 130.857081] [<ffffffff8141d89f>] scsi_io_completion+0xaf/0x630
[ 130.857150] [<ffffffff8165c88e>] ? _raw_spin_unlock_irqrestore+0x1e/0x30
[ 130.857219] [<ffffffff81413d31>] scsi_finish_command+0xc1/0x120
[ 130.857287] [<ffffffff8141d6fe>] scsi_softirq_done+0x13e/0x150
[ 130.857356] [<ffffffff812ef9f3>] blk_done_softirq+0x83/0xa0
[ 130.857423] [<ffffffff8106d4d8>] __do_softirq+0xa8/0x210
[ 130.857491] [<ffffffff8139ddf7>] ? __xen_evtchn_do_upcall+0x207/0x250
[ 130.857562] [<ffffffff816669ac>] call_softirq+0x1c/0x30
[ 130.857631] [<ffffffff81015195>] do_softirq+0x65/0xa0
[ 130.857697] [<ffffffff8106d8be>] irq_exit+0x8e/0xb0
[ 130.857761] [<ffffffff8139fbb5>] xen_evtchn_do_upcall+0x35/0x50
[ 130.857829] [<ffffffff816669fe>] xen_do_hypervisor_callback+0x1e/0x30
[ 130.857897] <EOI> [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
[ 130.858010] [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
[ 130.858080] [<ffffffff8100a170>] ? xen_safe_halt+0x10/0x20
[ 130.858146] [<ffffffff8101b5a3>] ? default_idle+0x53/0x1d0
[ 130.858214] [<ffffffff81012236>] ? cpu_idle+0xd6/0x120
[ 130.858282] [<ffffffff8100a9a9>] ? xen_irq_enable_direct_reloc+0x4/0x4
[ 130.858351] [<ffffffff81643992>] ? cpu_bringup_and_idle+0xe/0x10


I hope this helps. Let me know if you need me to do any further testing, 
or if you have any other questions regarding my environment.

Thanks


Re: Block Size for Windows

[Jonathan Tripathy]

On 28/08/2012 21:03, Jonathan Tripathy wrote:
> On 28/08/2012 01:59, James Harper wrote:
>>> On 27/08/2012 21:18, Joseph Glanville wrote:
>>>> On 28 August 2012 06:15, Jonathan Tripathy <jonnyt-Nf8S+5hNwl710XsdtD+oqA@public.gmane.org> wrote:
>>>>> On 27/08/2012 21:07, Jonathan Tripathy wrote:
>>>>>> On 27/08/2012 21:00, Jonathan Tripathy wrote:
>>>>>>>> 2) The windows setup didn't complain that it couldn't install on
>>>>>>>> the LV, but once I clicked 'next', the Dom0 crashed and the server
>>>>>>>> rebooted. A lot of output was displayed on screen but quickly
>>>>>>>> vanished as the system rebooted. I'm trying to see if the output
>>>>>>>> was saved anywhere. Any ideas why this could of happened and/or
>>> where the output might be saved?
>>>>>>>>
>>>>>>> I'd also like to add that after the server came back up, the md
>>>>>>> raid array started rebuilding. I wondering if that's just a
>>>>>>> coincidence (due to the forced reboot), or a sign of something
>>>>>>> wrong with the md integration with bcache?
>>>>>>>
>>>>>>> I'm going to see if Windows installs natively on the md array (it's
>>>>>>> RAID
>>>>>>> 10 btw) and post back here.
>>>>>> Ok, so trying to install Windows directly onto the spindles causes
>>>>>> the same thing to happen. I'm going to try and boot up into the
>>>>>> non-bcache kernel (The default ubuntu one) and see if it works
>>>>>> there. If it fails there, then this is clearly a xen and/or 
>>>>>> mdraid issue...
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>>
>>>>> Ok, so booting into the default Ubuntu kernel, the windows
>>>>> installation seems to progress just fine.
>>>>>
>>>>> Does this mean there is something wrong with the mdraid code in the
>>>>> bcache kernel?
>>>>>
>>>>> Actually, I'm not telling the whole story. The kernel I'm using is
>>>>> the
>>>>> bcache-3.2 tree (from evilpriate.org) with changes merged in from
>>>>> kernel.org's 3.2.27 tree. There were no merge conflicts when I did
>>>>> the git merge.
>>>>>
>>>>> What do you think I should do?
>>>>>
>>>>>
>>>>> Thanks
>>>>>
>>>>> -- 
>>>>> To unsubscribe from this list: send the line "unsubscribe
>>>>> linux-bcache" in the body of a message to majordomo-u79uwXL29TbrhsbdSgBK9A@public.gmane.orgrg
>>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>> I would recommend booting with the raw bcache-3.2 branch before
>>>> applying the stable patches (even though they should be fine) and
>>>> trying to catch the panic.
>>>> This is easiest done with a serial port and setting it to the kernel
>>>> console on the kernel command line in grub.
>>>>
>>>> Joseph.
>>> Hi There,
>>>
>>> I can confirm that the problem occurs even when using the raw 
>>> bcache-3.2
>>> branch from evilpirate.org. Just to clarify, I am trying to install 
>>> Windows
>>> Server 2008 in a Xen HVM DomU, onto an LV which is on top of a 
>>> MDRAID 10
>>> array. Using the bcache-3.2 kernel, the system reboots (after
>>> panicing) as soon as I click 'next' after selecting the drive to 
>>> install windows
>>> onto. Using the standard Ubuntu kernel everything works as normal. This
>>> leads me to believe that there is an issue with the mdraid code 
>>> inside the
>>> bcache-3.2 tree. I'd like to stress that I wasn't doing any bcaching 
>>> during this
>>> test.
>>>
>> FWIW, i'm using the 3.2 patches applied to a Debian kernel with lvm 
>> on raid1 (not raid10) on bcache and it's all working fine since I 
>> changed to a 512 byte block size. I haven't done an install of 2008, 
>> just 2003, but there doesn't seem to be any problems.
>>
>>> What should my next step be? Try and find a serial cable to capture the
>>> debug output?
>>>
>> Before tinkering with a serial cable, see if the system is alive 
>> enough to use netconsole - it can be a bit of a timesaver.
>>
>> James
> Hi Everyone,
>
> Here is the trace as capture by netconsole:
>
> [ 130.844069] ------------[ cut here ]------------
> [ 130.844165] kernel BUG at fs/bio.c:420!
> [ 130.844232] invalid opcode: 0000 [#1] SMP
> [ 130.844404] CPU 4
> [ 130.844448] Modules linked in: xen_netback xen_blkback 8021q garp 
> xt_physdev bridge stp ebtable_filter ebtables ip6table_filter 
> ip6_tables iptable_filter ip_tables x_tables xen_gntdev xen_evtchn 
> xenfs nls_iso8859_1 nls_cp437 vfat fat netconsole configfs psmouse lp 
> joydev parport video mac_hid serio_raw usb_storage uas raid456 
> async_pq async_xor xor async_memcpy async_raid6_recov usbhid hid 
> raid10 e1000e raid6_pq async_tx raid1 raid0 multipath linear
> [ 130.846688]
> [ 130.846746] Pid: 0, comm: swapper/4 Not tainted 3.2.0+ #1 Supermicro 
> X9SCI/X9SCA/X9SCI/X9SCA
> [ 130.846956] RIP: e030:[<ffffffff811a6ef7>] [<ffffffff811a6ef7>] 
> bio_put+0x27/0x30
> [ 130.847089] RSP: e02b:ffff88005ff3cb80 EFLAGS: 00010246
> [ 130.847155] RAX: 0000000000000000 RBX: 00000000fffffffb RCX: 
> 00000000000003a6
> [ 130.847224] RDX: 00000000000003a5 RSI: 0000000000016c00 RDI: 
> ffff880039b58918
> [ 130.847293] RBP: ffff88005ff3cb80 R08: ffffffff81115e67 R09: 
> 0000000000000100
> [ 130.847362] R10: ffff88001a16eea0 R11: 0000000000000000 R12: 
> ffff880017fd4018
> [ 130.847431] R13: ffff880039b58918 R14: ffff880017220028 R15: 
> ffff88001a6dd400
> [ 130.847504] FS: 00007f507e004700(0000) GS:ffff88005ff39000(0000) 
> knlGS:0000000000000000
> [ 130.847590] CS: e033 DS: 002b ES: 002b CR0: 000000008005003b
> [ 130.847656] CR2: 00007f507d6910b0 CR3: 000000003989a000 CR4: 
> 0000000000002660
> [ 130.847725] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 
> 0000000000000000
> [ 130.847794] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 
> 0000000000000400
> [ 130.847863] Process swapper/4 (pid: 0, threadinfo ffff88003d646000, 
> task ffff88003d648000)
> [ 130.847952] Stack:
> [ 130.848012] ffff88005ff3cbc0 ffffffff8150238e ffff88003d6e40b0 
> ffff880039b58900
> [ 130.848258] ffff880039b58920 ffff88001a278100 0000000000000018 
> 0000000000000001
> [ 130.848505] ffff88005ff3cbd0 ffffffff811a5f5d ffff88005ff3cbf0 
> ffffffff811a6f3e
> [ 130.848749] Call Trace:
> [ 130.848810] <IRQ>
> [ 130.848914] [<ffffffff8150238e>] clone_endio+0x8e/0xd0
> [ 130.848979] [<ffffffff811a5f5d>] bio_endio+0x1d/0x40
> [ 130.849047] [<ffffffff811a6f3e>] bio_pair_release+0x3e/0x50
> [ 130.849113] [<ffffffff811a6f6f>] bio_pair_end+0x1f/0x30
> [ 130.849180] [<ffffffff811a5f5d>] bio_endio+0x1d/0x40
> [ 130.849248] [<ffffffffa00dd3d2>] raid_end_bio_io+0xf2/0x100 [raid10]
> [ 130.849319] [<ffffffffa00ddf38>] one_write_done+0x38/0x50 [raid10]
> [ 130.849390] [<ffffffffa00deec4>] raid10_end_write_request+0xc4/0x130 
> [raid10]
> [ 130.849476] [<ffffffff811a5f5d>] bio_endio+0x1d/0x40
> [ 130.849543] [<ffffffff812e7a03>] req_bio_endio.isra.49+0xa3/0xe0
> [ 130.849614] [<ffffffff812e839d>] blk_update_request+0xfd/0x480
> [ 130.849681] [<ffffffff812e8751>] blk_update_bidi_request+0x31/0x90
> [ 130.849751] [<ffffffff812e9a4c>] blk_end_bidi_request+0x2c/0x80
> [ 130.849819] [<ffffffff812e9ae0>] blk_end_request+0x10/0x20
> [ 130.849888] [<ffffffff8141d89f>] scsi_io_completion+0xaf/0x630
> [ 130.849960] [<ffffffff8165c88e>] ? 
> _raw_spin_unlock_irqrestore+0x1e/0x30
> [ 130.850031] [<ffffffff81413d31>] scsi_finish_command+0xc1/0x120
> [ 130.850098] [<ffffffff8141d6fe>] scsi_softirq_done+0x13e/0x150
> [ 130.850167] [<ffffffff812ef9f3>] blk_done_softirq+0x83/0xa0
> [ 130.850237] [<ffffffff8106d4d8>] __do_softirq+0xa8/0x210
> [ 130.850304] [<ffffffff8139ddf7>] ? __xen_evtchn_do_upcall+0x207/0x250
> [ 130.850373] [<ffffffff816669ac>] call_softirq+0x1c/0x30
> [ 130.850442] [<ffffffff81015195>] do_softirq+0x65/0xa0
> [ 130.850507] [<ffffffff8106d8be>] irq_exit+0x8e/0xb0
> [ 130.850574] [<ffffffff8139fbb5>] xen_evtchn_do_upcall+0x35/0x50
> [ 130.850643] [<ffffffff816669fe>] xen_do_hypervisor_callback+0x1e/0x30
> [ 130.850711] <EOI>
> [ 130.850810] [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
> [ 130.850881] [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
> [ 130.850950] [<ffffffff8100a170>] ? xen_safe_halt+0x10/0x20
> [ 130.851017] [<ffffffff8101b5a3>] ? default_idle+0x53/0x1d0
> [ 130.851085] [<ffffffff81012236>] ? cpu_idle+0xd6/0x120
> [ 130.851153] [<ffffffff8100a9a9>] ? xen_irq_enable_direct_reloc+0x4/0x4
> [ 130.851223] [<ffffffff81643992>] ? cpu_bringup_and_idle+0xe/0x10
> [ 130.851291] Code: 00 00 00 00 55 48 89 e5 66 66 66 66 90 8b 47 40 85 
> c0 74 17 f0 ff 4f 40 0f 94 c0 84 c0 75 05 5d c3 0f 1f 00 e8 2b ff ff 
> ff 5d c3 <0f> 0b 0f 1f 80 00 00 00 00 55 48 89 e5 53 48 83 ec 08 66 66 66
> [ 130.854057] RIP [<ffffffff811a6ef7>] bio_put+0x27/0x30
> [ 130.854163] RSP <ffff88005ff3cb80>
> [ 130.854226] ---[ end trace fa3fbcc21926358a ]---
> [ 130.855127] Kernel panic - not syncing: Fatal exception in interrupt
> [ 130.855198] Pid: 0, comm: swapper/4 Tainted: G D 3.2.0+ #1
> [ 130.855267] Call Trace:
> [ 130.855327] <IRQ> [<ffffffff816516ef>] panic+0x91/0x1a7
> [ 130.855438] [<ffffffff8165d85a>] oops_end+0xea/0xf0
> [ 130.855505] [<ffffffff81016708>] die+0x58/0x90
> [ 130.855571] [<ffffffff8165d194>] do_trap+0xc4/0x170
> [ 130.855636] [<ffffffff81013e25>] do_invalid_op+0x95/0xb0
> [ 130.855702] [<ffffffff811a6ef7>] ? bio_put+0x27/0x30
> [ 130.855767] [<ffffffff8100a23d>] ? xen_force_evtchn_callback+0xd/0x10
> [ 130.855838] [<ffffffff8100aa02>] ? check_events+0x12/0x20
> [ 130.855906] [<ffffffff8166672b>] invalid_op+0x1b/0x20
> [ 130.855974] [<ffffffff81115e67>] ? mempool_free_slab+0x17/0x20
> [ 130.856041] [<ffffffff811a6ef7>] ? bio_put+0x27/0x30
> [ 130.856109] [<ffffffff8150238e>] clone_endio+0x8e/0xd0
> [ 130.856175] [<ffffffff811a5f5d>] bio_endio+0x1d/0x40
> [ 130.856241] [<ffffffff811a6f3e>] bio_pair_release+0x3e/0x50
> [ 130.856308] [<ffffffff811a6f6f>] bio_pair_end+0x1f/0x30
> [ 130.856374] [<ffffffff811a5f5d>] bio_endio+0x1d/0x40
> [ 130.856443] [<ffffffffa00dd3d2>] raid_end_bio_io+0xf2/0x100 [raid10]
> [ 130.856513] [<ffffffffa00ddf38>] one_write_done+0x38/0x50 [raid10]
> [ 130.856584] [<ffffffffa00deec4>] raid10_end_write_request+0xc4/0x130 
> [raid10]
> [ 130.856671] [<ffffffff811a5f5d>] bio_endio+0x1d/0x40
> [ 130.856738] [<ffffffff812e7a03>] req_bio_endio.isra.49+0xa3/0xe0
> [ 130.856807] [<ffffffff812e839d>] blk_update_request+0xfd/0x480
> [ 130.856877] [<ffffffff812e8751>] blk_update_bidi_request+0x31/0x90
> [ 130.856946] [<ffffffff812e9a4c>] blk_end_bidi_request+0x2c/0x80
> [ 130.857014] [<ffffffff812e9ae0>] blk_end_request+0x10/0x20
> [ 130.857081] [<ffffffff8141d89f>] scsi_io_completion+0xaf/0x630
> [ 130.857150] [<ffffffff8165c88e>] ? 
> _raw_spin_unlock_irqrestore+0x1e/0x30
> [ 130.857219] [<ffffffff81413d31>] scsi_finish_command+0xc1/0x120
> [ 130.857287] [<ffffffff8141d6fe>] scsi_softirq_done+0x13e/0x150
> [ 130.857356] [<ffffffff812ef9f3>] blk_done_softirq+0x83/0xa0
> [ 130.857423] [<ffffffff8106d4d8>] __do_softirq+0xa8/0x210
> [ 130.857491] [<ffffffff8139ddf7>] ? __xen_evtchn_do_upcall+0x207/0x250
> [ 130.857562] [<ffffffff816669ac>] call_softirq+0x1c/0x30
> [ 130.857631] [<ffffffff81015195>] do_softirq+0x65/0xa0
> [ 130.857697] [<ffffffff8106d8be>] irq_exit+0x8e/0xb0
> [ 130.857761] [<ffffffff8139fbb5>] xen_evtchn_do_upcall+0x35/0x50
> [ 130.857829] [<ffffffff816669fe>] xen_do_hypervisor_callback+0x1e/0x30
> [ 130.857897] <EOI> [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
> [ 130.858010] [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
> [ 130.858080] [<ffffffff8100a170>] ? xen_safe_halt+0x10/0x20
> [ 130.858146] [<ffffffff8101b5a3>] ? default_idle+0x53/0x1d0
> [ 130.858214] [<ffffffff81012236>] ? cpu_idle+0xd6/0x120
> [ 130.858282] [<ffffffff8100a9a9>] ? xen_irq_enable_direct_reloc+0x4/0x4
> [ 130.858351] [<ffffffff81643992>] ? cpu_bringup_and_idle+0xe/0x10
>
>
> I hope this helps. Let me know if you need me to do any further 
> testing, or if you have any other questions regarding my environment.
>
> Thanks
> 
Also, I'd like to add that this issue does not occur when using 
MD-RAID1. It only occurs when using MD-RAID10.

Thanks

Re: Block Size for Windows

[Jonathan Tripathy]

On 27/08/2012 20:55, Kent Overstreet wrote:
> On Mon, Aug 27, 2012 at 12:43 PM, Jonathan Tripathy <jonnyt-Nf8S+5hNwl710XsdtD+oqA@public.gmane.org> wrote:
>> On 27/08/2012 20:22, Kent Overstreet wrote:
>>> echo 1 > /sys/fs/bcache/<uuid>/unregister
>>>
>>> echo 1 > /sys/block/bcache0/bcache/stop
>> Ok, so I managed to recreate everything with a block size of 512 bytes.
>> However, 2 bad things have happened:
>>
>> 1) In my Linux DomU, my fio randomwrite test have gone from about 28k iops
>> to about 800 iops. Can this be fixed? changing bs and ba to 512 in the fio
>> config file didn't hep.
> Ouch!
>
> Bet the ssd doesn't like those 512 byte journal writes. I'm going to
> have to think about that...
Hi Kent,

Ok, a little good news. I managed to sort out the low IOPS issue. When I 
formatted bcache0 with a 512 block size, I forgot to set the caching 
mode back to writeback. Once I did that, the IOPS were nice and high 
again (circa 26k IOPS). Sorry about the noise regarding that non-issue.

I re-tried the windows setup with writeback enabled for the 512B 
formatted cache. It kernel paniced as per-usual, however for some 
reason, it's kernel panicking upon every boot now, whereas when it was 
formatted with 4k sectors, once the system came back up, it was stable 
(until I tried another windows setup). The debug trace appears to be the 
same as the one I posted to the list (kernel BUG at fs/bio.c:420! 
invalid opcode: 0000 [#1] SMP).

I appreciate your time.

Thanks

Re: Block Size for Windows

[Jonathan Tripathy]

On 28/08/2012 23:43, Jonathan Tripathy wrote:
> On 27/08/2012 20:55, Kent Overstreet wrote:
>> On Mon, Aug 27, 2012 at 12:43 PM, Jonathan Tripathy 
>> <jonnyt-Nf8S+5hNwl710XsdtD+oqA@public.gmane.org> wrote:
>>> On 27/08/2012 20:22, Kent Overstreet wrote:
>>>> echo 1 > /sys/fs/bcache/<uuid>/unregister
>>>>
>>>> echo 1 > /sys/block/bcache0/bcache/stop
>>> Ok, so I managed to recreate everything with a block size of 512 bytes.
>>> However, 2 bad things have happened:
>>>
>>> 1) In my Linux DomU, my fio randomwrite test have gone from about 
>>> 28k iops
>>> to about 800 iops. Can this be fixed? changing bs and ba to 512 in 
>>> the fio
>>> config file didn't hep.
>> Ouch!
>>
>> Bet the ssd doesn't like those 512 byte journal writes. I'm going to
>> have to think about that...
> Hi Kent,
>
> Ok, a little good news. I managed to sort out the low IOPS issue. When 
> I formatted bcache0 with a 512 block size, I forgot to set the caching 
> mode back to writeback. Once I did that, the IOPS were nice and high 
> again (circa 26k IOPS). Sorry about the noise regarding that non-issue.
>
> I re-tried the windows setup with writeback enabled for the 512B 
> formatted cache. It kernel paniced as per-usual, however for some 
> reason, it's kernel panicking upon every boot now, whereas when it was 
> formatted with 4k sectors, once the system came back up, it was stable 
> (until I tried another windows setup). The debug trace appears to be 
> the same as the one I posted to the list (kernel BUG at fs/bio.c:420! 
> invalid opcode: 0000 [#1] SMP).
>
> I appreciate your time.
>
> Thanks
>
I'm going to take a wild guess and say that my system is constantly 
throwing the same panic as there is some dirty data in the cache (that 
the windows setup has written) and when bcache tries to write this to 
the MD-RAID10 array, the kernel panics. Before when it only happened 
during windows setup, this was probably because I didn't have writeback 
enabled (or no cache setup at all), hence no dirty data.

To sum up my wild guess: The windows setup is writing certain data that 
is causing the MD-RAID10 array to panic the kernel.

But these are only guesses.

Re: Block Size for Windows

[Kent Overstreet]

On Mon, Aug 27, 2012 at 3:06 PM, Joseph Glanville
<joseph.glanville-2MxvZkOi9dvvnOemgxGiVw@public.gmane.org> wrote:
> The issue is probably in fs/bio.c, some of the bio splitting changes
> could effect md without even having bcache in use.
>
> Kent, I will try get a test box up running more recent code then the
> stuff we run (which doesn't have said issue).

Well, sounds like it's definitively just raid10 that triggers it -
that helps a lot. I think I know a sure workaround for it now, I'll
try and upload a new version tomorrow that should fix it.

Re: Block Size for Windows

[Kent Overstreet]

I just went ahead and backed out the problematic change. Haven't even
tried building the code yet - but if you feel lucky, it's up. I bet it
fixes it.

On Wed, Aug 29, 2012 at 1:56 AM, Kent Overstreet <koverstreet-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> wrote:
> On Mon, Aug 27, 2012 at 3:06 PM, Joseph Glanville
> <joseph.glanville-2MxvZkOi9dvvnOemgxGiVw@public.gmane.org> wrote:
>> The issue is probably in fs/bio.c, some of the bio splitting changes
>> could effect md without even having bcache in use.
>>
>> Kent, I will try get a test box up running more recent code then the
>> stuff we run (which doesn't have said issue).
>
> Well, sounds like it's definitively just raid10 that triggers it -
> that helps a lot. I think I know a sure workaround for it now, I'll
> try and upload a new version tomorrow that should fix it.

Re: Block Size for Windows

[Jonathan Tripathy]

On 29.08.2012 08:03, Kent Overstreet wrote:
> I just went ahead and backed out the problematic change. Haven't even
> tried building the code yet - but if you feel lucky, it's up. I bet 
> it
> fixes it.

Hi Kent,

I pull down your latest code and did a build. I'm pleased to say that 
Windows has now installed. I will do some further testing/benchmarking 
to make sure that the install is stable.

Many thanks for your quick help

Re: Block Size for Windows

[Jonathan Tripathy]

On 29/08/2012 13:12, Jonathan Tripathy wrote:
>
> On 29.08.2012 08:03, Kent Overstreet wrote:
>> I just went ahead and backed out the problematic change. Haven't even
>> tried building the code yet - but if you feel lucky, it's up. I bet it
>> fixes it.
>
> Hi Kent,
>
> I pull down your latest code and did a build. I'm pleased to say that 
> Windows has now installed. I will do some further testing/benchmarking 
> to make sure that the install is stable.
>
> Many thanks for your quick help
Hi Kent,

A little bad news. With this new build, the spindles are getting 
thrashed and this is constantly (never stopping) spewing out on my 
netconsole:

[23198.979215] md/raid10:md7: make_request bug: can't convert block 
across chunks or bigger than 512k 67230714 4
[23198.979304] md/raid10:md7: make_request bug: can't convert block 
across chunks or bigger than 512k 67234814 4
[23198.979393] md/raid10:md7: make_request bug: can't convert block 
across chunks or bigger than 512k 67235834 4
[23198.979595] md/raid10:md7: make_request bug: can't convert block 
across chunks or bigger than 512k 67276796 3
[23198.979696] md/raid10:md7: make_request bug: can't convert block 
across chunks or bigger than 512k 67297276 4
[23198.979791] md/raid10:md7: make_request bug: can't convert block 
across chunks or bigger than 512k 67248126 4
[23198.979886] md/raid10:md7: make_request bug: can't convert block 
across chunks or bigger than 512k 67284987 3
[23198.979966] md/raid10:md7: make_request bug: can't convert block 
across chunks or bigger than 512k 67305471 4
[23198.980215] md/raid10:md7: make_request bug: can't convert block 
across chunks or bigger than 512k 67329017 4
[23198.980337] md/raid10:md7: make_request bug: can't convert block 
across chunks or bigger than 512k 67323899 4
[23198.980542] md/raid10:md7: make_request bug: can't convert block 
across chunks or bigger than 512k 67330047 2
[23198.981707] md/raid10:md7: make_request bug: can't convert block 
across chunks or bigger than 512k 67335164 3
[23198.982164] md/raid10:md7: make_request bug: can't convert block 
across chunks or bigger than 512k 67346429 4

Any ideas?

Thanks

Re: Block Size for Windows

[Kent Overstreet]

On Wed, Aug 29, 2012 at 06:22:30PM +0100, Jonathan Tripathy wrote:
> On 29/08/2012 13:12, Jonathan Tripathy wrote:
> >
> >On 29.08.2012 08:03, Kent Overstreet wrote:
> >>I just went ahead and backed out the problematic change. Haven't even
> >>tried building the code yet - but if you feel lucky, it's up. I bet it
> >>fixes it.
> >
> >Hi Kent,
> >
> >I pull down your latest code and did a build. I'm pleased to say
> >that Windows has now installed. I will do some further
> >testing/benchmarking to make sure that the install is stable.
> >
> >Many thanks for your quick help
> Hi Kent,
> 
> A little bad news. With this new build, the spindles are getting
> thrashed and this is constantly (never stopping) spewing out on my
> netconsole:
> 
> [23198.979215] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67230714 4
> [23198.979304] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67234814 4
> [23198.979393] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67235834 4
> [23198.979595] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67276796 3
> [23198.979696] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67297276 4
> [23198.979791] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67248126 4
> [23198.979886] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67284987 3
> [23198.979966] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67305471 4
> [23198.980215] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67329017 4
> [23198.980337] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67323899 4
> [23198.980542] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67330047 2
> [23198.981707] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67335164 3
> [23198.982164] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67346429 4
> 
> Any ideas?

Oh hell, I see the problem. That one is not going to be such an easy
fix. I'm going to have to think about it.

Amusing part is it looks like blocksize = 4k was working around the
problem... :/

Re: Block Size for Windows

[Jonathan Tripathy]

On 29/08/2012 18:34, Kent Overstreet wrote:
> On Wed, Aug 29, 2012 at 06:22:30PM +0100, Jonathan Tripathy wrote:
>> On 29/08/2012 13:12, Jonathan Tripathy wrote:
>>> On 29.08.2012 08:03, Kent Overstreet wrote:
>>>> I just went ahead and backed out the problematic change. Haven't even
>>>> tried building the code yet - but if you feel lucky, it's up. I bet it
>>>> fixes it.
>>> Hi Kent,
>>>
>>> I pull down your latest code and did a build. I'm pleased to say
>>> that Windows has now installed. I will do some further
>>> testing/benchmarking to make sure that the install is stable.
>>>
>>> Many thanks for your quick help
>> Hi Kent,
>>
>> A little bad news. With this new build, the spindles are getting
>> thrashed and this is constantly (never stopping) spewing out on my
>> netconsole:
>>
>> [23198.979215] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67230714 4
>> [23198.979304] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67234814 4
>> [23198.979393] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67235834 4
>> [23198.979595] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67276796 3
>> [23198.979696] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67297276 4
>> [23198.979791] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67248126 4
>> [23198.979886] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67284987 3
>> [23198.979966] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67305471 4
>> [23198.980215] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67329017 4
>> [23198.980337] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67323899 4
>> [23198.980542] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67330047 2
>> [23198.981707] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67335164 3
>> [23198.982164] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67346429 4
>>
>> Any ideas?
> Oh hell, I see the problem. That one is not going to be such an easy
> fix. I'm going to have to think about it.
>
> Amusing part is it looks like blocksize = 4k was working around the
> problem... :/
Ah, ok :)

Please keep me posted when this is fixed, I'm very keen to see bcache 
deployed in our cloud infrastructure :)

If there is anything I can do to help from a testing standpoint, please 
let me know.

Cheers

Re: Block Size for Windows

[Kent Overstreet]

On Wed, Aug 29, 2012 at 06:42:43PM +0100, Jonathan Tripathy wrote:
> On 29/08/2012 18:34, Kent Overstreet wrote:
> >On Wed, Aug 29, 2012 at 06:22:30PM +0100, Jonathan Tripathy wrote:
> >>On 29/08/2012 13:12, Jonathan Tripathy wrote:
> >>>On 29.08.2012 08:03, Kent Overstreet wrote:
> >>>>I just went ahead and backed out the problematic change. Haven't even
> >>>>tried building the code yet - but if you feel lucky, it's up. I bet it
> >>>>fixes it.
> >>>Hi Kent,
> >>>
> >>>I pull down your latest code and did a build. I'm pleased to say
> >>>that Windows has now installed. I will do some further
> >>>testing/benchmarking to make sure that the install is stable.
> >>>
> >>>Many thanks for your quick help
> >>Hi Kent,
> >>
> >>A little bad news. With this new build, the spindles are getting
> >>thrashed and this is constantly (never stopping) spewing out on my
> >>netconsole:
> >>
> >>[23198.979215] md/raid10:md7: make_request bug: can't convert block
> >>across chunks or bigger than 512k 67230714 4
> >>[23198.979304] md/raid10:md7: make_request bug: can't convert block
> >>across chunks or bigger than 512k 67234814 4
> >>[23198.979393] md/raid10:md7: make_request bug: can't convert block
> >>across chunks or bigger than 512k 67235834 4
> >>[23198.979595] md/raid10:md7: make_request bug: can't convert block
> >>across chunks or bigger than 512k 67276796 3
> >>[23198.979696] md/raid10:md7: make_request bug: can't convert block
> >>across chunks or bigger than 512k 67297276 4
> >>[23198.979791] md/raid10:md7: make_request bug: can't convert block
> >>across chunks or bigger than 512k 67248126 4
> >>[23198.979886] md/raid10:md7: make_request bug: can't convert block
> >>across chunks or bigger than 512k 67284987 3
> >>[23198.979966] md/raid10:md7: make_request bug: can't convert block
> >>across chunks or bigger than 512k 67305471 4
> >>[23198.980215] md/raid10:md7: make_request bug: can't convert block
> >>across chunks or bigger than 512k 67329017 4
> >>[23198.980337] md/raid10:md7: make_request bug: can't convert block
> >>across chunks or bigger than 512k 67323899 4
> >>[23198.980542] md/raid10:md7: make_request bug: can't convert block
> >>across chunks or bigger than 512k 67330047 2
> >>[23198.981707] md/raid10:md7: make_request bug: can't convert block
> >>across chunks or bigger than 512k 67335164 3
> >>[23198.982164] md/raid10:md7: make_request bug: can't convert block
> >>across chunks or bigger than 512k 67346429 4
> >>
> >>Any ideas?
> >Oh hell, I see the problem. That one is not going to be such an easy
> >fix. I'm going to have to think about it.
> >
> >Amusing part is it looks like blocksize = 4k was working around the
> >problem... :/
> Ah, ok :)
> 
> Please keep me posted when this is fixed, I'm very keen to see
> bcache deployed in our cloud infrastructure :)
> 
> If there is anything I can do to help from a testing standpoint,
> please let me know.

I had an idea for how to work around it :) (and other potential issues
I recently became aware of).

More untested code up!

Re: Block Size for Windows

[Jonathan Tripathy]

On 29/08/2012 19:06, Kent Overstreet wrote:
> On Wed, Aug 29, 2012 at 06:42:43PM +0100, Jonathan Tripathy wrote:
>> On 29/08/2012 18:34, Kent Overstreet wrote:
>>> On Wed, Aug 29, 2012 at 06:22:30PM +0100, Jonathan Tripathy wrote:
>>>> On 29/08/2012 13:12, Jonathan Tripathy wrote:
>>>>> On 29.08.2012 08:03, Kent Overstreet wrote:
>>>>>> I just went ahead and backed out the problematic change. Haven't even
>>>>>> tried building the code yet - but if you feel lucky, it's up. I bet it
>>>>>> fixes it.
>>>>> Hi Kent,
>>>>>
>>>>> I pull down your latest code and did a build. I'm pleased to say
>>>>> that Windows has now installed. I will do some further
>>>>> testing/benchmarking to make sure that the install is stable.
>>>>>
>>>>> Many thanks for your quick help
>>>> Hi Kent,
>>>>
>>>> A little bad news. With this new build, the spindles are getting
>>>> thrashed and this is constantly (never stopping) spewing out on my
>>>> netconsole:
>>>>
>>>> [23198.979215] md/raid10:md7: make_request bug: can't convert block
>>>> across chunks or bigger than 512k 67230714 4
>>>> [23198.979304] md/raid10:md7: make_request bug: can't convert block
>>>> across chunks or bigger than 512k 67234814 4
>>>> [23198.979393] md/raid10:md7: make_request bug: can't convert block
>>>> across chunks or bigger than 512k 67235834 4
>>>> [23198.979595] md/raid10:md7: make_request bug: can't convert block
>>>> across chunks or bigger than 512k 67276796 3
>>>> [23198.979696] md/raid10:md7: make_request bug: can't convert block
>>>> across chunks or bigger than 512k 67297276 4
>>>> [23198.979791] md/raid10:md7: make_request bug: can't convert block
>>>> across chunks or bigger than 512k 67248126 4
>>>> [23198.979886] md/raid10:md7: make_request bug: can't convert block
>>>> across chunks or bigger than 512k 67284987 3
>>>> [23198.979966] md/raid10:md7: make_request bug: can't convert block
>>>> across chunks or bigger than 512k 67305471 4
>>>> [23198.980215] md/raid10:md7: make_request bug: can't convert block
>>>> across chunks or bigger than 512k 67329017 4
>>>> [23198.980337] md/raid10:md7: make_request bug: can't convert block
>>>> across chunks or bigger than 512k 67323899 4
>>>> [23198.980542] md/raid10:md7: make_request bug: can't convert block
>>>> across chunks or bigger than 512k 67330047 2
>>>> [23198.981707] md/raid10:md7: make_request bug: can't convert block
>>>> across chunks or bigger than 512k 67335164 3
>>>> [23198.982164] md/raid10:md7: make_request bug: can't convert block
>>>> across chunks or bigger than 512k 67346429 4
>>>>
>>>> Any ideas?
>>> Oh hell, I see the problem. That one is not going to be such an easy
>>> fix. I'm going to have to think about it.
>>>
>>> Amusing part is it looks like blocksize = 4k was working around the
>>> problem... :/
>> Ah, ok :)
>>
>> Please keep me posted when this is fixed, I'm very keen to see
>> bcache deployed in our cloud infrastructure :)
>>
>> If there is anything I can do to help from a testing standpoint,
>> please let me know.
> I had an idea for how to work around it :) (and other potential issues
> I recently became aware of).
>
> More untested code up!
Wow that was quick! I'll send this to the build server and let you know 
the results.

Cheers

Jonny

Re: Block Size for Windows

[Brad Campbell]

On 30/08/12 01:22, Jonathan Tripathy wrote:
> On 29/08/2012 13:12, Jonathan Tripathy wrote:
>>
>> On 29.08.2012 08:03, Kent Overstreet wrote:
>>> I just went ahead and backed out the problematic change. Haven't even
>>> tried building the code yet - but if you feel lucky, it's up. I bet it
>>> fixes it.
>>
>> Hi Kent,
>>
>> I pull down your latest code and did a build. I'm pleased to say that
>> Windows has now installed. I will do some further testing/benchmarking
>> to make sure that the install is stable.
>>
>> Many thanks for your quick help
> Hi Kent,
>
> A little bad news. With this new build, the spindles are getting
> thrashed and this is constantly (never stopping) spewing out on my
> netconsole:
>
> [23198.979215] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67230714 4
> [23198.979304] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67234814 4
> [23198.979393] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67235834 4
> [23198.979595] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67276796 3
> [23198.979696] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67297276 4
> [23198.979791] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67248126 4
> [23198.979886] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67284987 3
> [23198.979966] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67305471 4
> [23198.980215] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67329017 4
> [23198.980337] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67323899 4
> [23198.980542] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67330047 2
> [23198.981707] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67335164 3
> [23198.982164] md/raid10:md7: make_request bug: can't convert block
> across chunks or bigger than 512k 67346429 4
>
> Any ideas?

Looks like the same problem I reported in April.

http://www.spinics.net/lists/linux-bcache/msg00255.html

Re: Block Size for Windows

[Jonathan Tripathy]

On 30/08/2012 03:00, Brad Campbell wrote:
> On 30/08/12 01:22, Jonathan Tripathy wrote:
>> On 29/08/2012 13:12, Jonathan Tripathy wrote:
>>>
>>> On 29.08.2012 08:03, Kent Overstreet wrote:
>>>> I just went ahead and backed out the problematic change. Haven't even
>>>> tried building the code yet - but if you feel lucky, it's up. I bet it
>>>> fixes it.
>>>
>>> Hi Kent,
>>>
>>> I pull down your latest code and did a build. I'm pleased to say that
>>> Windows has now installed. I will do some further testing/benchmarking
>>> to make sure that the install is stable.
>>>
>>> Many thanks for your quick help
>> Hi Kent,
>>
>> A little bad news. With this new build, the spindles are getting
>> thrashed and this is constantly (never stopping) spewing out on my
>> netconsole:
>>
>> [23198.979215] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67230714 4
>> [23198.979304] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67234814 4
>> [23198.979393] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67235834 4
>> [23198.979595] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67276796 3
>> [23198.979696] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67297276 4
>> [23198.979791] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67248126 4
>> [23198.979886] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67284987 3
>> [23198.979966] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67305471 4
>> [23198.980215] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67329017 4
>> [23198.980337] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67323899 4
>> [23198.980542] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67330047 2
>> [23198.981707] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67335164 3
>> [23198.982164] md/raid10:md7: make_request bug: can't convert block
>> across chunks or bigger than 512k 67346429 4
>>
>> Any ideas?
>
> Looks like the same problem I reported in April.
>
Hi There,

I've built the latest head from the bcache-3.2 branch and now all seems 
fine. I currently have an IOMeter test that has been running for about 
90 minutes with an IO depth of 256 with 30 workers (a very heavy load) 
and so far not a single error in netconsole....

Cheers