Re: Another (ESP?) scsi blk-mq problem on sparc64

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Mon, Nov 24, 2014 at 10:33:37AM -0700, Jens Axboe wrote:
> On 11/24/2014 10:31 AM, Paul E. McKenney wrote:
> >On Mon, Nov 24, 2014 at 10:16:15AM -0700, Jens Axboe wrote:
> >>On 11/24/2014 09:22 AM, Paul E. McKenney wrote:
> >>>On Mon, Nov 24, 2014 at 08:35:46AM -0700, Jens Axboe wrote:
> >>>>On 11/24/2014 01:21 AM, Christoph Hellwig wrote:
> >>>>>On Fri, Nov 21, 2014 at 02:56:00PM -0500, David Miller wrote:
> >>>>>>I would suggest looking into the possibility that we allocate the memory
> >>>>>>using the count of valid cpus, rather than the largest cpu number.
> >>>>>>
> >>>>>>That's a common error that runs into problems with discontiguous
> >>>>>>cpu numbering like Sparc sometimes has.
> >>>>>
> >>>>>Yes, that does look like the case.  Do you have a good trick on how
> >>>>>to allocate a map for the highest possible cpu number without first
> >>>>>iterating the cpu map?  I couldn't find something that looks like a
> >>>>>highest_possible_cpu() helper.
> >>>>
> >>>>Honestly I think that num_posible_cpus() should return the max of
> >>>>number of CPUs (weigt), and the highest numbered CPU. It's a pain in
> >>>>the butt to handle this otherwise.
> >>>
> >>>Hear, hear!!!  That would make my life easier, and would make this sort
> >>>of problem much less likely to occur!
> >>
> >>How about this one?
> >
> >Works for me!
> 
> Thanks! I'll add an appropriate comment and send it out for review.
> 
> >(Just for the record, as far as I know, this doesn't matter for RCU,
> >which already uses nr_cpu_ids.)
> 
> Was that done after hitting something like this?

Nope.  It was done after people started griping about RCU's older habit
of sizing its data structures based on NR_CPUS.  Something about distros
cranking NR_CPUS up to 1024 and beyond.  ;-)

							Thanx, Paul

> Meelis, can you check if it fixes your issue?
> 
> 
> -- 
> Jens Axboe
>