From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: pci-passthrough in pvops causing offline raid Date: Thu, 11 Nov 2010 13:57:37 -0500 Message-ID: <20101111185737.GA15530@dumpdata.com> References: <20101111102416.GA32457@campbell-lange.net> <20101111165340.GB30006@dumpdata.com> <20101111173850.GA8756@campbell-lange.net> <20101111175809.GA30825@dumpdata.com> <20101111181329.GA9058@campbell-lange.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: <20101111181329.GA9058@campbell-lange.net> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Mark Adams Cc: xen-devel@lists.xensource.com, xen-users@lists.xensource.com List-Id: xen-devel@lists.xenproject.org > > > It is an Areca 1220. I can't see anything when the device goes offline > > > apart from > > > > > > [77324.264270] sd 0:0:0:1: rejecting I/O to offline device > > > [77334.005854] sd 0:0:0:0: rejecting I/O to offline device > > > > That is it? No other details from the driver? Did you poke at the driver (modinfo) > > to see if there are any options to increase its verbosity. > > I can't do anything once its happened, everything is offline so I have > no utils... An easy is to use netconsole. You can make all of the kernel log output got a different machine on your network. > > > > > > > > Unfortunately nothing get's logged because there is nothing to write to > > > anymore. I'm not sure how I can see the IRQs otherwise. There is no > > > > cat /proc/interrupts > > > > > pinning being done at all, and the machine was running for a few months > > > OK before the pciback was added. > > > > Ok, what about your NICs? Are they on-board? Are they sharing the IRQ > > with the card? You should be able to see this by looking at /proc/interrupts. > > Which NICs are they? lspci can you help you there. As of matter of fact, run > > lspci -vvv and send that. > > It is the onboard nics, they are Intel 82574L. I can see the arcmsr > line, but not anything for the NICS (because they are hidden?) Your lspci tells me it is on 16 and 17. You should see in /proc/interrupts on that line something about pciback? > > 39: 1126249 0 0 0 0 0 0 0 xen-pirq-ioapic-level arcmsr > > Nothing else is on 1126249 You mean IRQ 39. > > see lspci.txt attached. thanks. > > When this hang occurs, can you do 'xm debug-key Q', 'xm debug-key i', 'xm debug-key z'. > > Then run 'xm dmesg' and provide that to me? > > I can try this, but It probably won't work as the device is will not be > readable. Look on Google for 'Wiki PVOPS' and there is a section on how to connect a serial console. With the serial console we can send those commands to the hypervisor even if your box is hanged. http://wiki.xen.org/xenwiki/XenSerialConsole > > > > Is your boot disk on the same disk as the RAID? > > There are 2 raids, a Raid1 for the OS (/boot / /var /tmp /usr) and a > raid5 for VM's - They both dissapear at the same time so it appears the > card is dissapearing.. > I wonder if we have your IRQs confused. Can you provide the full cat /proc/interrupts and as well the serial bootup of the console? Or just the 'xm dmesg' and 'dmesg' output if you don't have the serial console hooked up yet.