Re: cfq: oops in __call_for_each_cic

[Jeff Layton <jlayton@redhat.com>]

On Tue, 10 Aug 2010 19:58:41 -0400
Jens Axboe <axboe@kernel.dk> wrote:

> On 08/10/2010 12:35 PM, Jeff Layton wrote:
> > On Tue, 10 Aug 2010 12:10:05 -0400
> > Jens Axboe <axboe@kernel.dk> wrote:
> > 
> >> On 08/10/2010 10:27 AM, Jeff Layton wrote:
> >>> On Tue, 10 Aug 2010 10:22:41 -0400
> >>> Jeff Moyer <jmoyer@redhat.com> wrote:
> >>>
> >>>> Jeff Layton <jlayton@redhat.com> writes:
> >>>>
> >>>>> Saw this oops on my test machine this morning. I rebooted the machine
> >>>>> last night and hadn't done anything on it other than log in this
> >>>>> morning. The kernel here is based on Steve French's git tree, which is
> >>>>> based on Linus' as of Sunday Aug 8th. Last non-cifs commit is:
> >>>>
> >>>> This looks a lot like this bug:
> >>>>   https://bugzilla.redhat.com/show_bug.cgi?id=577968
> >>>>
> >>>> See also:
> >>>>   http://kerneloops.org/guilty.php?guilty=cfq_free_io_context&version=2.6.34-rc&start=2228224&end=2260991&class=oops
> >>>>
> >>>> It's been around since 2.6.30.8 according to kerneloops.org.  If you
> >>>> find that you have a reliable way of reproducing the issue, that would
> >>>> be great.
> >>>>
> >>>
> >>> Ok, thanks -- no clear reproducer so far. This morning was the
> >>> first time I've seen it and it was on the console of my rawhide
> >>> machine. The last thing I did with it was reboot it last night. I
> >>> suspect that the gzip process came from a cron job or something.
> >>
> >> What version did you hit it on?
> >>
> > 
> > It was a kernel built out of git, based on Steve French's git tree. The
> > last commit from Linus in it was
> > 45d7f32c7a43cbb9592886d38190e379e2eb2226. Everything else on top of
> > that was patches that only touched cifs code. cifs.ko hadn't been
> > plugged in since it was rebooted.
> 
> OK. That bug is pretty elusive, so far I haven't been able to figure
> out what the heck is going on here and my attempts at reproducing
> have all failed. The reports so far seem to have the cron component
> in common. Does fedora ionice some cron jobs or anything like that?
> Or use CLONE_IO?
> 

Yes. I sort of doubt anything there would use CLONE_IO, but ionice is
definitely used. Fedora uses anacron. I don't see any explicit calls to
gzip in there, but it's possible something else is calling it:

# grep ionice /etc/cron.*/*
/etc/cron.daily/mlocate.cron:ionice -c2 -n7 -p $$ >/dev/null 2>&1
/etc/cron.daily/readahead.cron:ionice -c3 -p $$ >/dev/null 2>&1

# cat /etc/anacrontab 
# /etc/anacrontab: configuration file for anacron

# See anacron(8) and anacrontab(5) for details.

SHELL=/bin/sh
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
# the maximal random delay added to the base delay of the jobs
RANDOM_DELAY=45
# the jobs will be started during the following hours only
START_HOURS_RANGE=3-22

#period in days   delay in minutes   job-identifier   command
1	5	cron.daily		nice run-parts /etc/cron.daily
7	25	cron.weekly		nice run-parts /etc/cron.weekly
@monthly 45	cron.monthly		nice run-parts /etc/cron.monthly

-- 
Jeff Layton <jlayton@redhat.com>