Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Daniel Eisenbud]

On Fri, Jan 26, 2001 at 09:50:49PM +0100, Michael Schmitz <schmitz@mail.biophys.uni-duesseldorf.de> wrote:
> > > Though i cannot help you with your exact problem, some pointers.
> > > 1. The log should be typicall in /var/log/messages as long  as you
> > > have syslogd and klogd running.
> >
> > Unfortunately, since /var lives on the same bus, this doesn't help.  But
> > I'll try setting up a minimal linux installation on the other SCSI bus
> > and burning a CD, and will hopefully then find something out.
>
> Even with /var living on a different bus, locking the MESH bus solid
> probably means you have all of the SCSI midlevel and perhaps the whole VFS
> locked up.

Hmm, I wouldn't expect the VFS to be involved because this is using the
SCSI generic driver and presumably not going through the VFS at all.
Ah, I see, you mean if the SCSI subsystem locks, then the next time VFS
was waiting for the SCSI bus it might just stay locked forever.  Still,
if it turned out that the other SCSI bus was OK but the VFS had locked
up, I could avoid that by just mounting disks on the external bus.  But
I'll try to figure out other avenuse before going to the trouble of
doing another install on the external disk.

> I'm not sure about the precise granularity of locking (2.2 or
> 2.4 kernel? I forgot what you said about kernel versions) but I doubt
> there is a per-hostadapter lock even in 2.4.

This happens with both 2.2 and 2.4.

> So what I'm saying is: rather try logging to serial console or network in
> order to see if there's any output produced (it might help to stop syslogd
> and klogd altogether and only use serial console) before you go to the
> effort to set up a copy of the system on another bus.

The thing that makes me a little bit pessimistic about this is that I
don't get any log messages even when I'm logged in on the console.
Maybe I need to change my syslogd setup to log everything to the console
(which virtual console will it get logged to, or can I specify that?)
and not to any files so syslogd won't freeze when SCSI goes?  Is there
any reason to believe that I'd be able to get useful output over the
network or serial port when I can't on my screen?  (Note that the whole
system doesn't lock up immediately, especially with swap turned off, but
any attempts to access filesystems make things die quickly.)

-Daniel

--
Daniel E. Eisenbud
eisenbud@cs.swarthmore.edu

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Michael Schmitz]

> > > Unfortunately, since /var lives on the same bus, this doesn't help.  But
> > > I'll try setting up a minimal linux installation on the other SCSI bus
> > > and burning a CD, and will hopefully then find something out.
> >
> > Even with /var living on a different bus, locking the MESH bus solid
> > probably means you have all of the SCSI midlevel and perhaps the whole VFS
> > locked up.
>
> Hmm, I wouldn't expect the VFS to be involved because this is using the
> SCSI generic driver and presumably not going through the VFS at all.
> Ah, I see, you mean if the SCSI subsystem locks, then the next time VFS
> was waiting for the SCSI bus it might just stay locked forever.  Still,

Right. And are you sure the sg driver isn't going through VFS (sharing
buffers with the rest of the system)?

> if it turned out that the other SCSI bus was OK but the VFS had locked
> up, I could avoid that by just mounting disks on the external bus.  But

As long as the sg driver doesn't touch VFS buffers at all, not even for
copying data from user space, I think you should be safe.

> This happens with both 2.2 and 2.4.

So more lock granularity in 2.4 didn't help :-(


> > So what I'm saying is: rather try logging to serial console or network in
> > order to see if there's any output produced (it might help to stop syslogd
> > and klogd altogether and only use serial console) before you go to the
> > effort to set up a copy of the system on another bus.
>
> The thing that makes me a little bit pessimistic about this is that I
> don't get any log messages even when I'm logged in on the console.

I think that's a different story: these messages would be posted to the
console by syslogd which might already hang. Depends on your syslog.conf.

Serial console is different in that it doesn't just write log messages to
the kernel log buffer and wait for klogd/syslogd to pick them up, but
instead write the messages out to the serial port without delay.

> Maybe I need to change my syslogd setup to log everything to the console
> (which virtual console will it get logged to, or can I specify that?)

# same to a separate screen
*.info;mail.none;authpriv.none;local0.none		/dev/tty7

is what keeps a not-too-cluttered log (copy of /var/log/messages) on tty7
of our server. kern.notice to some tty would probably be enough for you.

> and not to any files so syslogd won't freeze when SCSI goes?  Is there

So put just that one line redirecting log output to /dev/tty7 into
syslog.conf temporarily and kill -1 syslogd before starting the test.

> any reason to believe that I'd be able to get useful output over the
> network or serial port when I can't on my screen?  (Note that the whole
> system doesn't lock up immediately, especially with swap turned off, but
> any attempts to access filesystems make things die quickly.)

The system not locking up immediately makes me hope serial console will
still work and show messages if there are any generated. At least
interrupts aren't blocked yet. syslog to a log host might also still work.
But output to console should also still work ...

Have you tried to increase the kernel log level (dmesg -n 7 should be the
max level you can set from user space) to make sure every kernel message
goes to the current console directly? If that doesn't work, but you can
still use the keyboard, it's time to hack a new sysrq option that dumps
the current MESH status and so on. Or toggles logging in the MESH driver.
You might be able to use xmon as well - I've never tried hard enough to
understand xmon.

	Michael


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Iain Sandoe]

> You might be able to use xmon as well - I've never tried hard enough to
> understand xmon.

under 2.4.x xmon isn't too difficult - it's saved my bacon a few times.

under 2.2.x it depends on what adb hardware you have (via-cuda didn't work -
but I hacked 2.2.17 to get it going - don't suppose it would be too
difficult to move that hack to 2.2.18).  I posted a patch for via-cuda &
xmon sometime last year (can drag 'em out if you like).

----
basic instructions:

go to arch/ppc/xmon and edit start.c to enable output to both bootx screen &
the console (and a serial port as well if you like).  It's kinda grape-shot
- but if there's any form of oops you will probably see it this way.

You have to do a little hack to get output on bootx screen & console
simultaneously.  I can post a patch if it's tricky.

----

I keep wondering if it's worth asking Paulus to consider making the
different xmon output options CONFIG_XXXX instead of a file edit.  It gets
tedious if you pull every other day.

hth,
Iain.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Daniel Eisenbud]

On Fri, Jan 26, 2001 at 10:32:35PM +0100, Michael Schmitz <schmitz@zirkon.biophys.uni-duesseldorf.de> wrote:
> > Hmm, I wouldn't expect the VFS to be involved because this is using the
> > SCSI generic driver and presumably not going through the VFS at all.
> > Ah, I see, you mean if the SCSI subsystem locks, then the next time VFS
> > was waiting for the SCSI bus it might just stay locked forever.  Still,
>
> Right. And are you sure the sg driver isn't going through VFS (sharing
> buffers with the rest of the system)?

Um, no idea.  I'd believe you if you said that it was.  :-)

> > The thing that makes me a little bit pessimistic about this is that I
> > don't get any log messages even when I'm logged in on the console.
>
> I think that's a different story: these messages would be posted to the
> console by syslogd which might already hang. Depends on your syslog.conf.
>
> Serial console is different in that it doesn't just write log messages to
> the kernel log buffer and wait for klogd/syslogd to pick them up, but
> instead write the messages out to the serial port without delay.

How do I set it up to log to a serial console?  (I may not be able to do
this for a few days until I borrow a friend's laptop, though.)

> > Maybe I need to change my syslogd setup to log everything to the console
> > (which virtual console will it get logged to, or can I specify that?)
>
> # same to a separate screen
> *.info;mail.none;authpriv.none;local0.none		/dev/tty7
>
> is what keeps a not-too-cluttered log (copy of /var/log/messages) on tty7
> of our server. kern.notice to some tty would probably be enough for you.
>
> > and not to any files so syslogd won't freeze when SCSI goes?  Is there
>
> So put just that one line redirecting log output to /dev/tty7 into
> syslog.conf temporarily and kill -1 syslogd before starting the test.

I'll try this variation in a few minutes.

> Have you tried to increase the kernel log level (dmesg -n 7 should be the
> max level you can set from user space) to make sure every kernel message
> goes to the current console directly?

I did "dmesg -n 8" since I forgot the number and didn't have your email
right in front of me, and it accepted that.  Does dmesg just tell the
kernel what to send to syslogd, or does it tell it what to send to the
current console directly?  If the former, how does this work on a
per-console basis?  If the latter, why would a serial console have a
better chance of working?

> If that doesn't work, but you can still use the keyboard, it's time to
> hack a new sysrq option that dumps the current MESH status and so on.
> Or toggles logging in the MESH driver.  You might be able to use xmon
> as well - I've never tried hard enough to understand xmon.

Hmm, I'll try to look into this.  If anyone has any pointers to
information about xmon, please let me know.  Do I need to use it over a
serial line?

-Daniel

--
Daniel E. Eisenbud
eisenbud@cs.swarthmore.edu

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Daniel Eisenbud]

On Sat, Jan 27, 2001 at 02:23:46AM +0000, Iain Sandoe <iain@sandoe.co.uk> wrote:
> > You might be able to use xmon as well - I've never tried hard enough to
> > understand xmon.
>
> under 2.4.x xmon isn't too difficult - it's saved my bacon a few times.

I can boot a 2.4.x kernel for this purpose.

> basic instructions:
>
> go to arch/ppc/xmon and edit start.c to enable output to both bootx screen &
> the console (and a serial port as well if you like).  It's kinda grape-shot
> - but if there's any form of oops you will probably see it this way.

I use quik, not bootx (though I could switch for debugging purposes.)
Do I just change the "console = 0" to "console = 1" to get it to work on
my console (I assume that you don't mean the serial console here, but
indeed my screen?)  Or do I need to to something else to the file that
I'm not seeing?

-Daniel

--
Daniel E. Eisenbud
eisenbud@cs.swarthmore.edu

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Daniel Eisenbud]

On Sat, Jan 27, 2001 at 04:31:15PM -0500, Daniel Eisenbud <eisenbud@cs.swarthmore.edu> wrote:
> On Fri, Jan 26, 2001 at 10:32:35PM +0100, Michael Schmitz <schmitz@zirkon.biophys.uni-duesseldorf.de> wrote:
> > # same to a separate screen
> > *.info;mail.none;authpriv.none;local0.none		/dev/tty7
> >
> > is what keeps a not-too-cluttered log (copy of /var/log/messages) on tty7
> > of our server. kern.notice to some tty would probably be enough for you.
> >
> > > and not to any files so syslogd won't freeze when SCSI goes?  Is there
> >
> > So put just that one line redirecting log output to /dev/tty7 into
> > syslog.conf temporarily and kill -1 syslogd before starting the test.
>
> I'll try this variation in a few minutes.

This worked, but I didn't get any useful log output until I started
getting error 600 from the SCSI midlevel and errors from ext2fs.  I'm
going to try again, making sure that I have dmesg set as high as
possible, and see if I can get anything else.  Otherwise, is it time to
start scattering printk()'s around the mesh driver?

-Daniel

--
Daniel E. Eisenbud
eisenbud@cs.swarthmore.edu

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Iain Sandoe]

> Do I just change the "console = 0" to "console = 1" to get it to work on
> my console

yes, assuming that console is is on your screen  ;-)

of course the console redirection of /proc kernel messages mechanisms  have
to be running - i.e. don't expect output when you're in 'single'

- although the info is indeed in the kernel /proc log. and you could tee it
to the screen and a file by hand...

Once klogd is going everything's fine.  (this is why I often enable the
grape-shot approach... )

>(I assume that you don't mean the serial console here, but indeed my screen?)

I'm fairly sure that as far as xmon is concerned - console is console - if
you set your console to a serial port elsewhere - I would expect the xmon
output to come out of that instead.

> Or do I need to to something else to the file that I'm not seeing?

nope, not that I know.

BTW: if you enable serial output in xmon - be aware you have to make sure
the baud rate is correct by hand as well.  By default it is set to 115 kb
(IIRC) correct for talking via modem - but pretty useless for most
"hardware" terminals

I wrote the scc formula down somewhere... (but you can find it in the serial
drivers with a little look).

good luck,
Iain.

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Daniel Eisenbud]

On Sat, Jan 27, 2001 at 07:59:24PM -0500, Daniel Eisenbud <eisenbud@cs.swarthmore.edu> wrote:
>
> On Sat, Jan 27, 2001 at 04:31:15PM -0500, Daniel Eisenbud <eisenbud@cs.swarthmore.edu> wrote:
> > On Fri, Jan 26, 2001 at 10:32:35PM +0100, Michael Schmitz <schmitz@zirkon.biophys.uni-duesseldorf.de> wrote:
> > > # same to a separate screen
> > > *.info;mail.none;authpriv.none;local0.none		/dev/tty7
> > >
> > > is what keeps a not-too-cluttered log (copy of /var/log/messages) on tty7
> > > of our server. kern.notice to some tty would probably be enough for you.
> > >
> > > > and not to any files so syslogd won't freeze when SCSI goes?  Is there
> > >
> > > So put just that one line redirecting log output to /dev/tty7 into
> > > syslog.conf temporarily and kill -1 syslogd before starting the test.
> >
> > I'll try this variation in a few minutes.
>
> This worked, but I didn't get any useful log output until I started
> getting error 600 from the SCSI midlevel and errors from ext2fs.  I'm
> going to try again, making sure that I have dmesg set as high as
> possible, and see if I can get anything else.  Otherwise, is it time to
> start scattering printk()'s around the mesh driver?

In fact, it looks like the mesh driver has significant debugging code.
My last attempt of this evening will be to turn that on and see if it
does anything.  If not, I'll look at this again tomorrow or the day
after.

-Daniel

--
Daniel E. Eisenbud
eisenbud@cs.swarthmore.edu

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Daniel Eisenbud]

On Sat, Jan 27, 2001 at 08:25:00PM -0500, Daniel Eisenbud <eisenbud@cs.swarthmore.edu> wrote:
> In fact, it looks like the mesh driver has significant debugging code.
> My last attempt of this evening will be to turn that on and see if it
> does anything.  If not, I'll look at this again tomorrow or the day
> after.

Well, the mesh driver debugging code did not trigger.  However, I have
some new information: cdrdao eventually exits with
"cdrdao: Device not configured.  Cannot set SG_SET_TIMEOUT"
I also get a message that the SCSI subsystem gave a return code of
6000000 (looking at the source, this is actually in hex) while trying to
access my internal disk.

There also appear to be reports of this happening on the single
(non-mesh) SCSI bus of a powermac 7200.  I just replied to one of them
on the mailing list.  In that case, it's not clear that this only dies
when trying to burn in disk-at-once mode.  There's another report of
trouble on the mailing list from someone who was trying to burn in
disk-at-once mode but didn't specify on what hardware.  I'm about to
respnd to that one to see if I can get a little bit closer to tracking
this down.  The non-mesh report makes me wonder if the problem is in
some other layer.  Maybe the sg driver has a subtle endianness issue on
PowerPC?  Has anyone ever successfully burned a disk in disk-at-once
mode with a SCSI CD-R or CD-RW on a powerppc machine?  On a powermac?

Maybe the next thing to do (why didn't I think of this sooner?) is to
strace cdrdao and cdrecord and see what system call they're really
hanging in, and what difference there is between what they do in dao
mode and what they do in tao mode.  More later!

-Daniel

--
Daniel E. Eisenbud
eisenbud@cs.swarthmore.edu

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Daniel Eisenbud]

On Sat, Jan 27, 2001 at 09:34:01PM -0500, Daniel Eisenbud <eisenbud@cs.swarthmore.edu> wrote:
> Maybe the next thing to do (why didn't I think of this sooner?) is to
> strace cdrdao and cdrecord and see what system call they're really
> hanging in, and what difference there is between what they do in dao
> mode and what they do in tao mode.  More later!

Attached is strace output from cdrdao.  I honsetly can't make much of
it, but maybe someone else will be able to?  I'm not sure if it hung in
the last read or the last write.  I think it was the last read, because
strace printed part of that line and then its output file (which I put
on a ramdisk, and tarred to a floppy from there, so I wouldn't have to
copy it by hand once my SCSI bus was useless) didn't change for a while,
and then the read line finished printing.  There's probably a way to
make strace print timestamps, and I'll do that tomorrow (as well as
doing this with cdrecord in both disk-at-once and track-at-once modes to
be able to compare them.

-Daniel

--
Daniel E. Eisenbud
eisenbud@cs.swarthmore.edu

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Daniel Eisenbud]

[-- Attachment #1: Type: text/plain, Size: 1082 bytes --]

Oops, I forgot to actually attach the file.  If anyone wants to see this
and has trouble with reading MIME attachments, just email me and I'll
send you a plaintext copy -- I figured I compress and attach it to spare
the whole list a 20K email.

-Daniel

On Sun, Jan 28, 2001 at 12:09:29AM -0500, Daniel Eisenbud <eisenbud@cs.swarthmore.edu> wrote:
> Attached is strace output from cdrdao.  I honsetly can't make much of
> it, but maybe someone else will be able to?  I'm not sure if it hung in
> the last read or the last write.  I think it was the last read, because
> strace printed part of that line and then its output file (which I put
> on a ramdisk, and tarred to a floppy from there, so I wouldn't have to
> copy it by hand once my SCSI bus was useless) didn't change for a while,
> and then the read line finished printing.  There's probably a way to
> make strace print timestamps, and I'll do that tomorrow (as well as
> doing this with cdrecord in both disk-at-once and track-at-once modes to
> be able to compare them.

--
Daniel E. Eisenbud
eisenbud@cs.swarthmore.edu

[-- Attachment #2: cdrdao.log.0.gz --]
[-- Type: application/x-gunzip, Size: 3016 bytes --]

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Daniel Eisenbud]

More output.  It looks like SCSI dies when cdrecord sends the cue sheet,
which it only does in DAO mode, the SCSI driver dies, as exemplified in
this portion of its output in verbose mode:

Executing 'send_cue_sheet' command on Bus 0 Target 3, Lun 0 timeout 200s
CDB:  5D 00 00 00 00 00 00 00 20 00
DMA addr: 0x100404F0 size: 32 - using copy buffer
pack_len: 36, reply_len: 36 pack_id: 70 result: 0 sense[0]: 00
cmd finished after 200.500s timeout 200s

As you can see, it times out.  Attached is a bzip2'ed tar file of the
strace output (with timings, this time) and output of burning the same
very short track (which happens to be the TOC of an audio CD, not that
it seems to matter) in TAO mode and DAO mode.

Now to get to the bottom of what about the "send_cue_sheet" command is
losing.  But I need to get some sleep first.  If anyone has any ideas
about what I might do to actually see the packet and actually watch what
happens as it goes through the SCSI subsystem, that would be great.  I
can see xmon looming in my future.  :-)

-Daniel

--
Daniel E. Eisenbud
eisenbud@cs.swarthmore.edu

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Daniel Eisenbud]

[-- Attachment #1: Type: text/plain, Size: 408 bytes --]

Aargh, forgot to attach the file again!  Here it is.

-Daniel

On Sun, Jan 28, 2001 at 02:16:58AM -0500, Daniel Eisenbud <eisenbud@cs.swarthmore.edu> wrote:
> More output.  It looks like SCSI dies when cdrecord sends the cue sheet,
> which it only does in DAO mode, the SCSI driver dies, as exemplified in
> this portion of its output in verbose mode:
[...]

--
Daniel E. Eisenbud
eisenbud@cs.swarthmore.edu

[-- Attachment #2: cdrecord-tests.tar.bz2 --]
[-- Type: application/octet-stream, Size: 13644 bytes --]

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Michael Schmitz]

> How do I set it up to log to a serial console?  (I may not be able to do
> this for a few days until I borrow a friend's laptop, though.)

console=/dev/ttyS0 in the kernel options.

> I did "dmesg -n 8" since I forgot the number and didn't have your email
> right in front of me, and it accepted that.  Does dmesg just tell the

Maybe 8 is OK as well.

> kernel what to send to syslogd, or does it tell it what to send to the
> current console directly?  If the former, how does this work on a

The debug level controls what sort of printk messages are sent to the
console device(s) directly. syslogd always gets a copy of everything
anyway.

> per-console basis?  If the latter, why would a serial console have a
> better chance of working?

Just in case the screen update is blocked by some interrupt disable stuff.
Serial console works without interrupts (at least it used to).

	Michael


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Michael Schmitz]

> > > So put just that one line redirecting log output to /dev/tty7 into
> > > syslog.conf temporarily and kill -1 syslogd before starting the test.
> >
> > I'll try this variation in a few minutes.
>
> This worked, but I didn't get any useful log output until I started
> getting error 600 from the SCSI midlevel and errors from ext2fs.  I'm
> going to try again, making sure that I have dmesg set as high as

Please report the precise message text.

	Michael


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Daniel Eisenbud]

On Mon, Jan 29, 2001 at 03:29:16PM +0100, Michael Schmitz <schmitz@mail.biophys.uni-duesseldorf.de> wrote:
> > > > So put just that one line redirecting log output to /dev/tty7 into
> > > > syslog.conf temporarily and kill -1 syslogd before starting the test.
> > >
> > > I'll try this variation in a few minutes.
> >
> > This worked, but I didn't get any useful log output until I started
> > getting error 600 from the SCSI midlevel and errors from ext2fs.  I'm
> > going to try again, making sure that I have dmesg set as high as
>
> Please report the precise message text.

Here is what appears to be the relevant part of the log.  I have more of
it, which I can send to anyone if they need more context.  Note that in
the full log, there are _no_ _more_ messages from the mesh driver ever
after the last one you see in this segment.  It just crawls into a
corner and dies.

Note that this is from turning mesh debugging on for all hosts in
mesh.c, and turning on SCSI debugging by compiling in the option and
then doing "echo scsi log all > /proc/scsi/scsi".

-Daniel

Jan 29 10:42:50 cumulonimbus kernel: sg_unlink_reserve: req->use_sg=0
Jan 29 10:42:50 cumulonimbus kernel: sg_write: dev=2, count=78
Jan 29 10:42:50 cumulonimbus kernel: Open returning 1
Jan 29 10:42:50 cumulonimbus kernel: sg_write:   scsi opcode=0x5d, cmd_size=10
Jan 29 10:42:50 cumulonimbus kernel: sg_start_req: max_buff_size=32
Jan 29 10:42:50 cumulonimbus kernel: sg_link_reserve: size=32
Jan 29 10:42:50 cumulonimbus kernel: sg_write_xfer: num_write_xfer=32, use_sg=0
Jan 29 10:42:50 cumulonimbus kernel: Activating command for device 3 (1)
Jan 29 10:42:50 cumulonimbus kernel: scsi_do_cmd (host = 0, channel = 0 target = 3, buffer =c3490000, bufflen = 32, done = c00e149c, timeout = 20050, retries = 1)
Jan 29 10:42:50 cumulonimbus kernel: command : 5d  00  00  00  00  00  00  00  20  00
Jan 29 10:42:50 cumulonimbus kernel: internal_cmnd (host = 0, channel = 0, target = 3, command = c033c245, buffer = c3490000,
Jan 29 10:42:50 cumulonimbus kernel: bufflen = 32, done = c00e149c)
Jan 29 10:42:50 cumulonimbus kernel: queuecommand : routine at c00e39f4
Jan 29 10:42:50 cumulonimbus kernel: mesh_start: c033c200 ser=1518 tgt=3 cmd= 5d 0 0 0 0 0 0 0 20 0 use_sg=0 buffer=c3490000 bufflen=32
Jan 29 10:42:50 cumulonimbus kernel: leaving internal_cmnd()
Jan 29 10:42:50 cumulonimbus kernel: Leaving scsi_do_cmd()
Jan 29 10:42:50 cumulonimbus kernel: mesh: sending 1 msg bytes: c0
Jan 29 10:42:50 cumulonimbus kernel: sg_read: dev=2, count=36
Jan 29 10:42:50 cumulonimbus kernel: Open returning 1
Jan 29 10:46:10 cumulonimbus kernel: Command timed out active=1 busy=1 failed=1
Jan 29 10:46:11 cumulonimbus kernel: Error handler waking up
Jan 29 10:46:11 cumulonimbus kernel: scsi_unjam_host: Checking to see if we need to request sense
Jan 29 10:46:11 cumulonimbus kernel: Command to ID 3 timedout
Jan 29 10:46:11 cumulonimbus kernel: Total of 0+1 commands on 1 devices require eh work
Jan 29 10:46:11 cumulonimbus kernel: scsi_unjam_host: Checking to see if we want to try abort
Jan 29 10:46:11 cumulonimbus kernel: scsi_unjam_host: Checking to see if we want to try BDR
Jan 29 10:46:11 cumulonimbus kernel: scsi_unjam_host: Try hard bus reset
Jan 29 10:46:11 cumulonimbus kernel: scsi_unjam_host: Try hard host reset
Jan 29 10:46:11 cumulonimbus kernel: scsi_unjam_host: Take device offline
Jan 29 10:46:11 cumulonimbus kernel: Finishing command for device 3 6000000
Jan 29 10:46:11 cumulonimbus kernel: scsi_unjam_host: Returning
Jan 29 10:46:11 cumulonimbus kernel: Clearing timer for command c033c200
Jan 29 10:46:11 cumulonimbus kernel: scsi_error.c: Waking up host to restart
Jan 29 10:46:11 cumulonimbus kernel: Calling request function to restart things...
Jan 29 10:46:11 cumulonimbus last message repeated 2 times
Jan 29 10:46:11 cumulonimbus kernel: scsi_error.c: device offline - report as SUCCESS
Jan 29 10:46:11 cumulonimbus kernel: Command finished 1 0 0x6000000
Jan 29 10:46:11 cumulonimbus kernel: Notifying upper driver of completion for device 3 6000000
Jan 29 10:46:11 cumulonimbus kernel: sg__done: dev=2, scsi_stat=0, res=0x6000000
Jan 29 10:46:11 cumulonimbus kernel: Deactivating command for device 3 (active=0, failed=0)
Jan 29 10:46:11 cumulonimbus kernel: Error handler sleeping
Jan 29 10:46:11 cumulonimbus kernel: sg_finish_rem_req: res_used=1
Jan 29 10:46:11 cumulonimbus kernel: sg_unlink_reserve: req->use_sg=0

--
Daniel E. Eisenbud
eisenbud@cs.swarthmore.edu

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Michael Schmitz]

> >
> > Please report the precise message text.
>
> Here is what appears to be the relevant part of the log.  I have more of
> it, which I can send to anyone if they need more context.  Note that in
> the full log, there are _no_ _more_ messages from the mesh driver ever
> after the last one you see in this segment.  It just crawls into a
> corner and dies.
> Jan 29 10:42:50 cumulonimbus kernel: mesh: sending 1 msg bytes: c0
> Jan 29 10:42:50 cumulonimbus kernel: sg_read: dev=2, count=36
> Jan 29 10:42:50 cumulonimbus kernel: Open returning 1
> Jan 29 10:46:10 cumulonimbus kernel: Command timed out active=1 busy=1 failed=1
> Jan 29 10:46:11 cumulonimbus kernel: Error handler waking up
> Jan 29 10:46:11 cumulonimbus kernel: scsi_unjam_host: Checking to see if we need to request sense
> Jan 29 10:46:11 cumulonimbus kernel: Command to ID 3 timedout

The command in question seems to time out (now does this message come from
the sg or midlevel timeout?).

> Jan 29 10:46:11 cumulonimbus kernel: Total of 0+1 commands on 1 devices require eh work
> Jan 29 10:46:11 cumulonimbus kernel: scsi_unjam_host: Checking to see if we want to try abort
> Jan 29 10:46:11 cumulonimbus kernel: scsi_unjam_host: Checking to see if we want to try BDR
> Jan 29 10:46:11 cumulonimbus kernel: scsi_unjam_host: Try hard bus reset
> Jan 29 10:46:11 cumulonimbus kernel: scsi_unjam_host: Try hard host reset
> Jan 29 10:46:11 cumulonimbus kernel: scsi_unjam_host: Take device offline
> Jan 29 10:46:11 cumulonimbus kernel: Finishing command for device 3 6000000
> Jan 29 10:46:11 cumulonimbus kernel: scsi_unjam_host: Returning

The error handling code can't cope with the situation. Either because
abort support isn't implemented, or because something in the low level
driver went FUBAR.

> Jan 29 10:46:11 cumulonimbus kernel: Clearing timer for command c033c200
> Jan 29 10:46:11 cumulonimbus kernel: scsi_error.c: Waking up host to restart
> Jan 29 10:46:11 cumulonimbus kernel: Calling request function to restart things...
> Jan 29 10:46:11 cumulonimbus last message repeated 2 times

That 'restart things' doesn't seem to do anything either.

The big question: why does this particular command time out in the first
place? Does the target expect more data to be sent? Does the target just
crash and fails to release the bus? It seems to work on other host
adapters, doesn't it?

	Michael


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Daniel Eisenbud]

On Tue, Jan 30, 2001 at 10:28:25PM +0100, Michael Schmitz <schmitz@zirkon.biophys.uni-duesseldorf.de> wrote:
> The big question: why does this particular command time out in the first
> place? Does the target expect more data to be sent? Does the target just
> crash and fails to release the bus? It seems to work on other host
> adapters, doesn't it?

There's a report of excatly the same failure mode on mac53c94 with a
different brand (Sony, as opposed to Yamaha on my MESH) of DVD drive.
So the only points in common seem to be that the DVD drives are running
on mac hardware and that the drivers are written by Paul Mackerras.  :-)
I don't at all mean to impugn Paul's coding ability, which is obviously
excellent, but the fact that he wrote both the drivers may mean that
they share lots of code and possibly share the same bug.

-Daniel

--
Daniel E. Eisenbud
eisenbud@cs.swarthmore.edu

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Michael Schmitz]

> > The big question: why does this particular command time out in the first
> > place? Does the target expect more data to be sent? Does the target just
> > crash and fails to release the bus? It seems to work on other host
> > adapters, doesn't it?
>
> There's a report of excatly the same failure mode on mac53c94 with a
> different brand (Sony, as opposed to Yamaha on my MESH) of DVD drive.

Please send a detailed description on how to reproduce this bug by PM.
I'll try to look into that on my Lombard, and in case I can get the
required binaries for m68k I'd also test on an old Quadra with 53c9x
SCSI.

Does the bug happen only with DVD writers, or regular CD writers?

> So the only points in common seem to be that the DVD drives are running
> on mac hardware and that the drivers are written by Paul Mackerras.  :-)
> I don't at all mean to impugn Paul's coding ability, which is obviously
> excellent, but the fact that he wrote both the drivers may mean that
> they share lots of code and possibly share the same bug.

If so, the bug should have propagated to the m68k 53c9x driver. Wouldn't
be the 53c9x first weirdness - I never got it to accept a DAT drive (old
DDS-1, possibly Apple) without blowing up.

It might be something PPC specific as well and have nothing to do with
Paul's drivers.

	Michael


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Daniel Eisenbud]

On Tue, Jan 30, 2001 at 10:46:06PM +0100, Michael Schmitz <schmitz@zirkon.biophys.uni-duesseldorf.de> wrote:
> > > The big question: why does this particular command time out in the first
> > > place? Does the target expect more data to be sent? Does the target just
> > > crash and fails to release the bus? It seems to work on other host
> > > adapters, doesn't it?
> >
> > There's a report of excatly the same failure mode on mac53c94 with a
> > different brand (Sony, as opposed to Yamaha on my MESH) of DVD drive.
>
> Please send a detailed description on how to reproduce this bug by PM.
> I'll try to look into that on my Lombard, and in case I can get the
> required binaries for m68k I'd also test on an old Quadra with 53c9x
> SCSI.

Basically, any time I have used cdrdao to write a disc, or cdrecord with
the -dao flag, the SCSI bus has locked up, under kernels from 2.2.15pre3
or so to a recent 2.4 from the bk tree.  I'm tempted to see whether I
can write a short program using the sg stuff to just send the
"send_cue_sheet" command to the CD-RW and see if it locks up, but I
haven't had a chance to look into it yet.

> Does the bug happen only with DVD writers, or regular CD writers?

Aargh, sorry, when I said "DVD" above it was a complete thinko.  I have
a Yamaha CD-RW drive (the model number is something wit 8424 in it, I
can get exact details when I get home.)  The report with the mac53c94
driver was a Sony CD-R or CD-RW drive -- I've exchanged email with the
guy who reported it and can ask for the details if you'd like.  I have
no idea if it would fail with a DVD writer, but all the cases I've seen
have been CD writers.  (This is why it's not a good idea to wait until
late afternoon to have coffee, and why if I do so, I should refrain from
writing email until after I have found some.)  :-)

> > So the only points in common seem to be that the DVD drives are running
> > on mac hardware and that the drivers are written by Paul Mackerras.  :-)
> > I don't at all mean to impugn Paul's coding ability, which is obviously
> > excellent, but the fact that he wrote both the drivers may mean that
> > they share lots of code and possibly share the same bug.
>
> If so, the bug should have propagated to the m68k 53c9x driver. Wouldn't
> be the 53c9x first weirdness - I never got it to accept a DAT drive (old
> DDS-1, possibly Apple) without blowing up.

It would be great if you could check this out.

> It might be something PPC specific as well and have nothing to do with
> Paul's drivers.

This is why I asked in a previous email whether anyone had ever burned
a disk in DAO mode successfully on a pmac or in fact on any powerpc
machine, but I've received no responses.  Hmm, maybe I should try to
conduct a quick survey of this on linuxppc-user and/or debian-powerpc.

-Daniel

--
Daniel E. Eisenbud
eisenbud@cs.swarthmore.edu

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Geert Uytterhoeven]

On Tue, 30 Jan 2001, Michael Schmitz wrote:
> > So the only points in common seem to be that the DVD drives are running
> > on mac hardware and that the drivers are written by Paul Mackerras.  :-)
> > I don't at all mean to impugn Paul's coding ability, which is obviously
> > excellent, but the fact that he wrote both the drivers may mean that
> > they share lots of code and possibly share the same bug.
>
> If so, the bug should have propagated to the m68k 53c9x driver. Wouldn't
> be the 53c9x first weirdness - I never got it to accept a DAT drive (old
> DDS-1, possibly Apple) without blowing up.

FYI, my DDS-1 worked with my MESH. Not perfectly, sometimes it gave a warning
(lost arbitration, IIRC), which aborted the current reading/writing.

Moving the DD-1 to the Sym53c875 solved the problem.

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Chris Boot]

>> It might be something PPC specific as well and have nothing to do with
>> Paul's drivers.
>
> This is why I asked in a previous email whether anyone had ever burned
> a disk in DAO mode successfully on a pmac or in fact on any powerpc
> machine, but I've received no responses.  Hmm, maybe I should try to
> conduct a quick survey of this on linuxppc-user and/or debian-powerpc.

Well, I can't write a dao disk on my Panasonic CW-7502.  cdrecord complains
that it can't send the cue sheet, but the SCSI bus doesn't die in any way, I
can still mount and use disks on it afterwards (I only have the built-in
CD-ROM and my CD-R on it).  I can provide exactly what cdrecord says and any
other info if you wish.

This is on a Performa 5400/180 with an early pull of the 2.4.0 kernel from
bk.  Everything seems to be working fine, except for this (which isn't too
important for me).

Good luck,

--
    .-.        Chris Boot
    /v\     bootc@worldnet.fr
   // \\
  /(   )\   L   I   N   U   X
   ^^-^^   >Phear the Penguin<


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: mesh SCSI bus locks hard on 7500 when burning a CD-R in dao mode

[Daniel Eisenbud]

On Wed, Jan 31, 2001 at 05:40:24PM +0100, Chris Boot <bootc@worldnet.fr> wrote:
>
> >> It might be something PPC specific as well and have nothing to do with
> >> Paul's drivers.
> >
> > This is why I asked in a previous email whether anyone had ever burned
> > a disk in DAO mode successfully on a pmac or in fact on any powerpc
> > machine, but I've received no responses.  Hmm, maybe I should try to
> > conduct a quick survey of this on linuxppc-user and/or debian-powerpc.
>
> Well, I can't write a dao disk on my Panasonic CW-7502.  cdrecord complains
> that it can't send the cue sheet, but the SCSI bus doesn't die in any way, I
> can still mount and use disks on it afterwards (I only have the built-in
> CD-ROM and my CD-R on it).  I can provide exactly what cdrecord says and any
> other info if you wish.

That would be great -- any additional info that we can get helps.  Is
there a 200 second wait before it fails, or does it fail immediately?

> This is on a Performa 5400/180 with an early pull of the 2.4.0 kernel from
> bk.  Everything seems to be working fine, except for this (which isn't too
> important for me).

What SCSI driver do you use?  (If you don't know offhand, you should be
able to tell from the kernel boot messages.)

If you're willing to do a little bit more work, could you compile a
kernel with all the SCSI debugging support (there are two config
options) enabled, and before you try to burn a disk in DAO mode, type
(as root) "echo scsi log all > /proc/scsi/scsi"?  (The logging will be
turned off again when you next reboot.)  And then send us the resultant
syslog?

My working hypothesis is that there is something powerpc specific that
derails the send_cue_sheet command, but the lockup is because of the
mesh driver not handling the error well.  If this is correct, these
would both be good problems to fix.

Thanks,
Daniel

--
Daniel E. Eisenbud
eisenbud@cs.swarthmore.edu

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/