Skip to content

IRQ

IRQ is majorly ported based on linux-4.4. The decision of porting of whole IRQ stack from linux was made at early stage of Lego, when I’m not so familiar with this stuff. This technique decision has pros and cons.

The whole thing is made complicated by having IRQ domain. IRQ domain is introduced to address the multiple interrupt controller issue. And in x86, we kind of have mutiple as well: IO-APIC, REMAP, LAPIC. Although we are not supporting IRQ remap now.

Init

  • The first part of initialization is trap_init() at early setup_arch().
  • The second major entry point is irq_init() at start_kernel(). This irq_init() is actually a combination of linux’s:
    • early_irq_init(): 1) setup irq_desc[] array, and then call arch_early_irq_init(), which will register two IRQ domains (x86_vector_domain, msi_domain).
    • init_IRQ(): is actually a callback to low-level x86 interrupt setup. It mainly setup the desc’s data/chip etc, and register all different handlers.
    • In Lego, you will be able to find all the functionalitis are moved into arch_irq_init(). And, to this point, we have a complete setup.
  • The third (and last) entry point is smp_prepare_cpus(): smp_prepare_cpus() -> apic_bsp_setup() -> setup_local_APIC() -> setup_IO_APIC() -> x86_init.timers.setup_percpu_clockev()

IRQ Domain

We should have at least 2 or 3 IRQ domains:

  • x86_vector
  • x86_msi
  • x86_ioapic-N (each ioapic has one)

The first two guys are created during arch_irq_init(). While the latter ioapic ones are created during setup_IO_APIC(). All of them are allocated eventually by __irq_domain_add(), and linked at LIST_HEAD(irq_domain_list).

So.... Lego or Linux maintains its own IRQ numbers, starting from 0 to NR_IRQs. However, this IRQ number MAY not have a identical mapping to hardware’s own IRQ number (let us call it hwirq). Given this, we want to know the mapping between IRQ and hwirq. That’s the purpose of having linear_revmap and revmap_tree within each domain, it is used to translate hwirq to IRQ.

Why two different data structures? linear_revmap is fairly simple, an array, which is indexed by hwirq. However, the hwirq maybe very large, we don’t want to waste memory, that’s how we want to use trees.

These two can be used together. If we fail to insert into linear_revmap, we insert into tree. During search time, we need to look up both.

By default, x86_vector and x86_msi use radix tree only. x86_ioapic-N uses a mix of linear and radix tree.

To dump all IRQ domains, call dump_irq_domain_list(), which give you something like this: c [ 118.308544] name mapped linear-max direct-max devtree-node [ 118.316114] x86_ioapic-2 24 24 0 [ 118.322707] x86_ioapic-1 24 24 0 [ 118.329299] x86_ioapic-0 24 24 0 [ 118.335893] x86_msi 25 0 0 [ 118.342486] *x86_vector 40 0 0 [ 118.349078] irq hwirq chip name chip data active type domain [ 118.358775] 1 0x00001 IO-APIC 0xffff88107fcae000 LINEAR x86_ioapic-0 [ 118.368858] 3 0x00003 IO-APIC 0xffff88107fc8f000 LINEAR x86_ioapic-0 [ 118.378940] 4 0x00004 IO-APIC 0xffff88107fc6e000 LINEAR x86_ioapic-0 [ 118.389025] 5 0x00005 IO-APIC 0xffff88107fc6f000 LINEAR x86_ioapic-0 [ 118.399109] 6 0x00006 IO-APIC 0xffff88107fc4e000 LINEAR x86_ioapic-0 [ 118.409192] 7 0x00007 IO-APIC 0xffff88107fc4f000 LINEAR x86_ioapic-0 [ 118.419276] 8 0x00008 IO-APIC 0xffff88107fc2e000 LINEAR x86_ioapic-0 [ 118.429358] 9 0x00009 IO-APIC 0xffff88107fc2f000 LINEAR x86_ioapic-0 [ 118.439442] 10 0x0000a IO-APIC 0xffff88107fc0e000 LINEAR x86_ioapic-0 [ 118.449525] 11 0x0000b IO-APIC 0xffff88107fc0f000 LINEAR x86_ioapic-0 [ 118.459609] 12 0x0000c IO-APIC 0xffff88107fff0000 LINEAR x86_ioapic-0 [ 118.469692] 13 0x0000d IO-APIC 0xffff88107fff1000 LINEAR x86_ioapic-0 [ 118.479776] 14 0x0000e IO-APIC 0xffff88107fff2000 LINEAR x86_ioapic-0 [ 118.489860] 15 0x0000f IO-APIC 0xffff88107fff3000 LINEAR x86_ioapic-0 [ 118.499943] 24 0x300000 PCI-MSI (null) * RADIX x86_msi [ 118.509833] 25 0x300001 PCI-MSI (null) * RADIX x86_msi [ 118.519722] 26 0x300002 PCI-MSI (null) * RADIX x86_msi [ 118.529612] 27 0x300003 PCI-MSI (null) * RADIX x86_msi [ 118.539501] 28 0x300004 PCI-MSI (null) RADIX x86_msi

Aug 20, 2018

Well, I’ve ported the IRQ stuff at early days of Lego. At that time, I mainly ported the low-level APIC, IO-APIC, and ACPI stuff, along with the upper layer irqchip, irqdesc stuff.

These days, I was verifying our IB code and tried to add back mlx4en’s interrupt handler, somehow, there is no interrupt after request_irq().

Two possible reasons: 1) I missed something during PCI setup, 2) underlying APIC and IO-APIC need more work.

– Last Updated: Aug 28, 2018

Last update: August 29, 2018

Comments

Back to top