[RFC] genirq: prevent allocated_irqs from being smaller than NR_IRQS

[RFC] genirq: prevent allocated_irqs from being smaller than NR_IRQS

[Marcelo Schmitt]

Hi,

I was trying to understand IRQ initialization when suddenly got
intrigued about the declaration of the "allocated_irqs" bitmap at
kernel/irq/irqdesc.c. The size of allocated_irqs is defined by
IRQ_BITMAP_BITS, which in turn is passed to BITS_TO_LONGS to calculate
the actual number of IRQs the system may have. If I got it right, there
should be one entry at allocated_irqs for each possible IRQ line. At
kernel/irq/internals.h, IRQ_BITMAP_BITS is defined to be NR_IRQS (or
NR_IRQS plus a high constant in the case of sparse IRQs), which most
architectures seem to define as being the actual number of IRQs a board
has.

#ifdef CONFIG_SPARSE_IRQ
# define IRQ_BITMAP_BITS (NR_IRQS + 8196)
#else
# define IRQ_BITMAP_BITS NR_IRQS
#endif

The thing I'm troubled about is that BITS_TO_LONGS divides
IRQ_BITMAP_BITS by sizeof(long) * 8, which makes it possible for the
size of allocated_irqs to be smaller than NR_IRQS.

For instance, if !CONFIG_SPARSE_IRQ, sizeof(long) == 8, and NR_IRQS is
defined as 16, then IRQ_BITMAP_BITS would be equal to 
(16 + 64 - 1)/64 = 1. Even if CONFIG_SPARSE_IRQ is defined, a device
with a large number of IRQ lines would end up with a small bitmap for
allocated_irqs.

I thought NR_IRQS would be multiplied by the number of bits it uses.
Something like:

#ifdef CONFIG_SPARSE_IRQ
# define IRQ_BITMAP_BITS (NR_IRQS * BITS_PER_TYPE(long) + 8196)
#else
# define IRQ_BITMAP_BITS (NR_IRQS * BITS_PER_TYPE(long))
#endif

Anyhow, IRQ_BITMAP_BITS is also used to limit the maximum number of IRQs
at irqdesc.c. If my understanding of nr_irqs is correct, it would make
sense to change some sanity checks at early_irq_init() too.

Does anyone mind giving me some advice on how allocated_irqs is
initialized with a suitable size to support the number of interrupt
lines a board may have?

By the way, is there any mailing list for IRQ related discussions?
I couldn't find one at vger.kernel.org.


Thanks,

Marcelo

Re: [RFC] genirq: prevent allocated_irqs from being smaller than NR_IRQS

[Lars-Peter Clausen]

On 4/2/20 5:08 PM, Marcelo Schmitt wrote:
> Hi,
>
> I was trying to understand IRQ initialization when suddenly got
> intrigued about the declaration of the "allocated_irqs" bitmap at
> kernel/irq/irqdesc.c. The size of allocated_irqs is defined by
> IRQ_BITMAP_BITS, which in turn is passed to BITS_TO_LONGS to calculate
> the actual number of IRQs the system may have. If I got it right, there
> should be one entry at allocated_irqs for each possible IRQ line. At
> kernel/irq/internals.h, IRQ_BITMAP_BITS is defined to be NR_IRQS (or
> NR_IRQS plus a high constant in the case of sparse IRQs), which most
> architectures seem to define as being the actual number of IRQs a board
> has.
>
> #ifdef CONFIG_SPARSE_IRQ
> # define IRQ_BITMAP_BITS (NR_IRQS + 8196)
> #else
> # define IRQ_BITMAP_BITS NR_IRQS
> #endif
>
> The thing I'm troubled about is that BITS_TO_LONGS divides
> IRQ_BITMAP_BITS by sizeof(long) * 8, which makes it possible for the
> size of allocated_irqs to be smaller than NR_IRQS.
>
> For instance, if !CONFIG_SPARSE_IRQ, sizeof(long) == 8, and NR_IRQS is
> defined as 16, then IRQ_BITMAP_BITS would be equal to
> (16 + 64 - 1)/64 = 1. Even if CONFIG_SPARSE_IRQ is defined, a device
> with a large number of IRQ lines would end up with a small bitmap for
> allocated_irqs.
>
> I thought NR_IRQS would be multiplied by the number of bits it uses.
> Something like:
>
> #ifdef CONFIG_SPARSE_IRQ
> # define IRQ_BITMAP_BITS (NR_IRQS * BITS_PER_TYPE(long) + 8196)
> #else
> # define IRQ_BITMAP_BITS (NR_IRQS * BITS_PER_TYPE(long))
> #endif
>
> Anyhow, IRQ_BITMAP_BITS is also used to limit the maximum number of IRQs
> at irqdesc.c. If my understanding of nr_irqs is correct, it would make
> sense to change some sanity checks at early_irq_init() too.
>
> Does anyone mind giving me some advice on how allocated_irqs is
> initialized with a suitable size to support the number of interrupt
> lines a board may have?

Maybe I'm missing something, but allocated_irqs is a bitmap. This means 
each bit corresponds to one IRQ. if sizeof(long) is 8 and allocated_irqs 
is sized to be one long that means it is large enough for 64 IRQs.

- Lars

Re: [RFC] genirq: prevent allocated_irqs from being smaller than NR_IRQS

[Marcelo Schmitt]

On 04/02, Lars-Peter Clausen wrote:
> On 4/2/20 5:08 PM, Marcelo Schmitt wrote:
> > Hi,
> > 
> > I was trying to understand IRQ initialization when suddenly got
> > intrigued about the declaration of the "allocated_irqs" bitmap at
> > kernel/irq/irqdesc.c. The size of allocated_irqs is defined by
> > IRQ_BITMAP_BITS, which in turn is passed to BITS_TO_LONGS to calculate
> > the actual number of IRQs the system may have. If I got it right, there
> > should be one entry at allocated_irqs for each possible IRQ line. At
> > kernel/irq/internals.h, IRQ_BITMAP_BITS is defined to be NR_IRQS (or
> > NR_IRQS plus a high constant in the case of sparse IRQs), which most
> > architectures seem to define as being the actual number of IRQs a board
> > has.
> > 
> > #ifdef CONFIG_SPARSE_IRQ
> > # define IRQ_BITMAP_BITS (NR_IRQS + 8196)
> > #else
> > # define IRQ_BITMAP_BITS NR_IRQS
> > #endif
> > 
> > The thing I'm troubled about is that BITS_TO_LONGS divides
> > IRQ_BITMAP_BITS by sizeof(long) * 8, which makes it possible for the
> > size of allocated_irqs to be smaller than NR_IRQS.
> > 
> > For instance, if !CONFIG_SPARSE_IRQ, sizeof(long) == 8, and NR_IRQS is
> > defined as 16, then IRQ_BITMAP_BITS would be equal to
> > (16 + 64 - 1)/64 = 1. Even if CONFIG_SPARSE_IRQ is defined, a device
> > with a large number of IRQ lines would end up with a small bitmap for
> > allocated_irqs.
> > 
> > I thought NR_IRQS would be multiplied by the number of bits it uses.
> > Something like:
> > 
> > #ifdef CONFIG_SPARSE_IRQ
> > # define IRQ_BITMAP_BITS (NR_IRQS * BITS_PER_TYPE(long) + 8196)
> > #else
> > # define IRQ_BITMAP_BITS (NR_IRQS * BITS_PER_TYPE(long))
> > #endif
> > 
> > Anyhow, IRQ_BITMAP_BITS is also used to limit the maximum number of IRQs
> > at irqdesc.c. If my understanding of nr_irqs is correct, it would make
> > sense to change some sanity checks at early_irq_init() too.
> > 
> > Does anyone mind giving me some advice on how allocated_irqs is
> > initialized with a suitable size to support the number of interrupt
> > lines a board may have?
> 
> Maybe I'm missing something, but allocated_irqs is a bitmap. This means each
> bit corresponds to one IRQ. if sizeof(long) is 8 and allocated_irqs is sized
> to be one long that means it is large enough for 64 IRQs.

Probably that's what I was missing about the allocated_irqs use.
This explains a lot.

Many thanks,

Marcelo

> 
> - Lars
> 

Re: [RFC] genirq: prevent allocated_irqs from being smaller than NR_IRQS

[Thomas Gleixner]

Marcelo,

Marcelo Schmitt <marcelo.schmitt1@gmail.com> writes:
> #ifdef CONFIG_SPARSE_IRQ
> # define IRQ_BITMAP_BITS (NR_IRQS + 8196)
> #else
> # define IRQ_BITMAP_BITS NR_IRQS
> #endif
>
> The thing I'm troubled about is that BITS_TO_LONGS divides
> IRQ_BITMAP_BITS by sizeof(long) * 8, which makes it possible for the
> size of allocated_irqs to be smaller than NR_IRQS.

No. It calculates the number of longs which are required to store
IRQ_BITMAP_BITS bits. And it does not only divide, it also takes the
reminder into account.

One byte fits 8 bits. Multiplied with sizeof(long) tells you how many
bits fit into a long: Unsurprisingly that 32 on 32bit and 64 on 64bit
systems.

> By the way, is there any mailing list for IRQ related discussions?
> I couldn't find one at vger.kernel.org.

The MAINTAINERS file tells you:

IRQ SUBSYSTEM
L:      linux-kernel@vger.kernel.org

So you picked the right one.

Thanks,

        tglx

Re: [RFC] genirq: prevent allocated_irqs from being smaller than NR_IRQS

[Marcelo Schmitt]

On 04/02, Thomas Gleixner wrote:
> Marcelo,
> 
> Marcelo Schmitt <marcelo.schmitt1@gmail.com> writes:
> > #ifdef CONFIG_SPARSE_IRQ
> > # define IRQ_BITMAP_BITS (NR_IRQS + 8196)
> > #else
> > # define IRQ_BITMAP_BITS NR_IRQS
> > #endif
> >
> > The thing I'm troubled about is that BITS_TO_LONGS divides
> > IRQ_BITMAP_BITS by sizeof(long) * 8, which makes it possible for the
> > size of allocated_irqs to be smaller than NR_IRQS.
> 
> No. It calculates the number of longs which are required to store
> IRQ_BITMAP_BITS bits. And it does not only divide, it also takes the
> reminder into account.
> 
> One byte fits 8 bits. Multiplied with sizeof(long) tells you how many
> bits fit into a long: Unsurprisingly that 32 on 32bit and 64 on 64bit
> systems.

I see. Given that a single bit is enough to mark whether interrupts were
allocated for an IRQ line, the allocated_irqs needs only IRQ_BITMAP_BITS
bits to track how many IRQs have been allocated.

Since BITS_TO_LONGS will calculate the number of longs required to store
at least IRQ_BITMAP_BIT bits, the bitmap will have enough room for the
tracking bits.

Naturally, the number of longs needed to store N bits will depend on the
sizeof(long) in each system. Probably this is why BITS_PER_TYPE(long) is
taken into account to find out how many longs are needed to house
IRQ_BITMAP_BIT bits. Therefore, the initialization of allocated_irqs
shall generalize well for systems with different word sizes. A clever
implementation.

> 
> > By the way, is there any mailing list for IRQ related discussions?
> > I couldn't find one at vger.kernel.org.
> 
> The MAINTAINERS file tells you:
> 
> IRQ SUBSYSTEM
> L:      linux-kernel@vger.kernel.org
> 
> So you picked the right one.
> 
> Thanks,
> 
>         tglx

Thanks,

Marcelo