gizmomouse
Posts: 11
Joined: Mon Jun 10, 2019 2:45 am

inv-mpu6050 trigger probe fail -22

Thu Oct 03, 2019 12:08 pm

Hello,

I'm trying to get my Invisense MPU9250 up and running on my board.

The MPU9250 is an IMU that is attached to the RPI using I2C, actually it's attached to the host using a GPIO based I2C Multiplexer (Channel 0 or i2c4) and the interrupt is attached to an SX1509 I2C GPIO Expander/Interrupt Controller on a different multiplexed I2C channel (Channel 3 or i2c6).

I can see the MPU9250 with I2C Address 0x68 when I run I2C detect, so the IMU is definitely there.

Code: Select all

$ i2cdetect -y 4
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- 42 -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- 68 -- -- -- -- -- -- --
70: -- -- -- -- -- -- -- --
Without going into details I've verified that the gpio expander/interrupt controller exists and is working as expected.

The SX1509 is used as an interrupt controller and the MPU9250 interrupt line that is attached to GPIO 5 where the SX1509 should propagate the interrupt through to the Raspberry Pi on pin 28

Here is a snippet of the overlay that defines this specific gpio expander

Code: Select all

...
          gpex1a: [email protected] {
            compatible = "semtech,sx1509q";
            reg = <0x3e>;
            gpio-controller;
            interrupt-controller;
            #gpio-cells = <2>;
            #interrupt-cells = <2>;
            interrupt-parent = <&gpio>;
            interrupts = <28 IRQ_TYPE_EDGE_FALLING>;
            status = "okay";
            ngpios = <16>;


            pinctrl-names = "default";
            pinctrl-0 = <&interrupt_pin28>;

            gpex1a_pins: gpx1a-cfg {
              pins =  "gpio0",
                      "gpio1",
                      "gpio2",
                      "gpio3",
                      "gpio4",
                      "gpio6",
                      "gpio7",
                      "gpio8",
                      "gpio9",
                      "gpio10",
                      "gpio11",
                      "gpio12",
                      "gpio14",
                      "gpio15";
              bias-pull-up;
            };
            gpex1a_gps_reset_n: gpx1a_gps_reset_n {
              pins = "gpio13";
              bias-pull-up;
            };
            gpex1a_mpu9250_int: gpx1a_mpu9250_int {
              pins = "gpio5";
              bias-pull-up;
            };
          };
...
Here is the overlay snippet of the MPU 9250

Code: Select all

...
          mpu9250_2_68: [email protected] {
            compatible = "invensense,mpu9250";
            reg = <0x68>;
            interrupt-parent = <&gpex1a>;
            interrupts = <5 IRQ_TYPE_EDGE_RISING>;
          };
...
If it helps I would be happy to provide my entire overlay.

When I boot up I get this error (NOTE: I've modified the kernel to output the 'iio iio:device0 message):

Code: Select all

...
Oct  3 00:10:33 rpcmexpress kernel: [    8.306870] inv-mpu6050-i2c 4-0068: mounting matrix not found: using identity...
Oct  3 00:10:33 rpcmexpress kernel: [    8.456930] iio iio:device0: devm_request_irq failed: IRQ # 184
Oct  3 00:10:33 rpcmexpress kernel: [    8.456949] inv-mpu6050-i2c 4-0068: trigger probe fail -22
Oct  3 00:10:33 rpcmexpress kernel: [    8.457092] inv-mpu6050-i2c: probe of 4-0068 failed with error -22
...

Looking into the driver for this, the error is from here: <kernel source>/drivers/iio/imu/inv_mpu6050/inv_mpu6050_core.c

Code: Select all

...
  result = inv_mpu6050_probe_trigger(indio_dev, irq_type);
  if (result) {
    dev_err(dev, "trigger probe fail %d\n", result);
    return result;
  }
...
-22 is EINVAL (invalid argument)
Tracing it down the call stack from "inv_mpu6050_probe_trigger" to "inv_mpu6050_probe_trigger" within the file <kernel source>/drivers/iio/imu/inv_mpu6050/inv_mpu6050_trigger.c

the failure happens in the following block:

Code: Select all

...
  ret = devm_request_irq(&indio_dev->dev, st->irq,
             &iio_trigger_generic_data_rdy_poll,
             irq_type,
             "inv_mpu",
             st->trig);
  if (ret){
    //Added debug message to help track down issue
    dev_err(&indio_dev->dev, "devm_request_irq failed: IRQ # %d\n", st->irq);
    return ret;
  }
...
This is strange because when I 'cat' out /proc/interrupts I get this:

Code: Select all

[email protected]:~$ cat /proc/interrupts
           CPU0       CPU1       CPU2       CPU3
 17:        115          0          0          0  ARMCTRL-level   1 Edge      3f00b880.mailbox
 18:         36          0          0          0  ARMCTRL-level   2 Edge      VCHIQ doorbell
 40:          0          0          0          0  ARMCTRL-level  48 Edge      bcm2708_fb DMA
 42:       5008          0          0          0  ARMCTRL-level  50 Edge      DMA IRQ
 56:      97692          0          0          0  ARMCTRL-level  64 Edge      dwc_otg, dwc_otg_pcd, dwc_otg_hcd:usb1
 77:        722          0          0          0  ARMCTRL-level  85 Edge      3f205000.i2c, 3f804000.i2c
 80:        202          0          0          0  ARMCTRL-level  88 Edge      mmc0
 81:        138          0          0          0  ARMCTRL-level  89 Edge      uart-pl011
161:          0          0          0          0  bcm2836-timer   0 Edge      arch_timer
162:       2482       2061       3320       1784  bcm2836-timer   1 Edge      arch_timer
165:          0          0          0          0  bcm2836-pmu   9 Edge      arm-pmu
166:          0          0          0          0  pinctrl-bcm2835  35 Edge      sx1509q
167:          0          0          0          0   sx1509q   6 Edge      select
168:          0          0          0          0   sx1509q   7 Edge      start
169:          0          0          0          0   sx1509q   1 Edge      button y
170:          0          0          0          0   sx1509q   0 Edge      button x
171:          0          0          0          0   sx1509q  15 Edge      button b
172:          0          0          0          0   sx1509q  14 Edge      button a
173:          0          0          0          0   sx1509q  13 Edge      right-shoulder-bottom
174:          0          0          0          0   sx1509q  11 Edge      left-shoulder-bottom
175:          0          0          0          0   sx1509q  12 Edge      right-shoulder-top
176:          0          0          0          0   sx1509q  10 Edge      left-shoulder-top
177:          0          0          0          0   sx1509q   8 Edge      volume-down
178:          0          0          0          0   sx1509q   9 Edge      volume-up
179:          0          0          0          0   sx1509q   4 Edge      right
180:          0          0          0          0   sx1509q   3 Edge      left
181:          0          0          0          0   sx1509q   5 Edge      down
182:          0          0          0          0   sx1509q   2 Edge      up
183:          0          0          0          0  pinctrl-bcm2835  28 Edge      sx1509q
FIQ:              usb_fiq
IPI0:          0          0          0          0  CPU wakeup interrupts
IPI1:          0          0          0          0  Timer broadcast interrupts
IPI2:       2520       3825       5564       5704  Rescheduling interrupts
IPI3:        622       1471        982        948  Function call interrupts
IPI4:          0          0          0          0  CPU stop interrupts
IPI5:        139        393        215         50  IRQ work interrupts
IPI6:          0          0          0          0  completion interrupts

As you can see all the interrupt based gpio-keys I've added above work as expected while '184', which would logically go after '183' doesn't work.

As an experiment I removed the dts entry for the MPU9250 and replaced it with another 'gpio-key', adding a key to the same pin that the MPU9250 interrupt would use and here is the output of the /sys/kernel/debug/gpio

Code: Select all

gpiochip0: GPIOs 0-53, parent: platform/3f200000.gpio, pinctrl-bcm2835:
 gpio-17  (                    |i2c-mux-gpio        ) out hi
 gpio-45  (                    |i2c-mux-gpio        ) out hi

gpiochip2: GPIOs 100-101, brcmvirt-gpio, can sleep:
 gpio-100 (                    |led0                ) out hi

gpiochip5: GPIOs 453-469, parent: i2c/6-003e, sx1509q, can sleep:
 gpio-458 (                    |coffee              ) in  lo IRQ

gpiochip4: GPIOs 470-486, parent: i2c/5-003e, sx1509q, can sleep:

gpiochip3: GPIOs 487-503, parent: i2c/0-003e, sx1509q, can sleep:
 gpio-487 (                    |button x            ) in  hi IRQ
 gpio-488 (                    |button y            ) in  hi IRQ
 gpio-489 (                    |up                  ) in  hi IRQ
 gpio-490 (                    |left                ) in  hi IRQ
 gpio-491 (                    |right               ) in  hi IRQ
 gpio-492 (                    |down                ) in  hi IRQ
 gpio-493 (                    |select              ) in  hi IRQ
 gpio-494 (                    |start               ) in  hi IRQ
 gpio-495 (                    |volume-down         ) in  hi IRQ
 gpio-496 (                    |volume-up           ) in  hi IRQ
 gpio-497 (                    |left-shoulder-top   ) in  hi IRQ
 gpio-498 (                    |left-shoulder-bottom) in  hi IRQ
 gpio-499 (                    |right-shoulder-top  ) in  hi IRQ
 gpio-500 (                    |right-shoulder-botto) in  hi IRQ
 gpio-501 (                    |button a            ) in  hi IRQ
 gpio-502 (                    |button b            ) in  hi IRQ
gpio-458 is there, and the interrupt is there as well:

Code: Select all

[email protected]:~$ cat /proc/interrupts
           CPU0       CPU1       CPU2       CPU3
 17:        158          0          0          0  ARMCTRL-level   1 Edge      3f00b880.mailbox
 18:         34          0          0          0  ARMCTRL-level   2 Edge      VCHIQ doorbell
 40:          0          0          0          0  ARMCTRL-level  48 Edge      bcm2708_fb DMA
 42:       5316          0          0          0  ARMCTRL-level  50 Edge      DMA IRQ
 56:     159840          0          0          0  ARMCTRL-level  64 Edge      dwc_otg, dwc_otg_pcd, dwc_otg_hcd:usb1
 77:        714          0          0          0  ARMCTRL-level  85 Edge      3f205000.i2c, 3f804000.i2c
 80:        219          0          0          0  ARMCTRL-level  88 Edge      mmc0
 81:        109          0          0          0  ARMCTRL-level  89 Edge      uart-pl011
161:          0          0          0          0  bcm2836-timer   0 Edge      arch_timer
162:       4443       2066       5836       4337  bcm2836-timer   1 Edge      arch_timer
165:          0          0          0          0  bcm2836-pmu   9 Edge      arm-pmu
166:          0          0          0          0  pinctrl-bcm2835  35 Edge      sx1509q
167:          0          0          0          0   sx1509q   6 Edge      select
168:          0          0          0          0   sx1509q   7 Edge      start
169:          0          0          0          0   sx1509q   1 Edge      button y
170:          0          0          0          0   sx1509q   0 Edge      button x
171:          0          0          0          0   sx1509q  15 Edge      button b
172:          0          0          0          0   sx1509q  14 Edge      button a
173:          0          0          0          0   sx1509q  13 Edge      right-shoulder-bottom
174:          0          0          0          0   sx1509q  11 Edge      left-shoulder-bottom
175:          0          0          0          0   sx1509q  12 Edge      right-shoulder-top
176:          0          0          0          0   sx1509q  10 Edge      left-shoulder-top
177:          0          0          0          0   sx1509q   8 Edge      volume-down
178:          0          0          0          0   sx1509q   9 Edge      volume-up
179:          0          0          0          0   sx1509q   4 Edge      right
180:          0          0          0          0   sx1509q   3 Edge      left
181:          0          0          0          0   sx1509q   5 Edge      down
182:          0          0          0          0   sx1509q   2 Edge      up
183:          0          0          0          0  pinctrl-bcm2835  28 Edge      sx1509q
184:          0          0          0          0   sx1509q   5 Edge      coffee
FIQ:              usb_fiq
IPI0:          0          0          0          0  CPU wakeup interrupts
IPI1:          0          0          0          0  Timer broadcast interrupts
IPI2:       4945       4638       7308      12893  Rescheduling interrupts
IPI3:        459       1526       1029        893  Function call interrupts
IPI4:          0          0          0          0  CPU stop interrupts
IPI5:        316         20        865        469  IRQ work interrupts
IPI6:          0          0          0          0  completion interrupts
interrupt 184. I've checked over my DTS entry, compared it with the beaglebone-blue, beaglebone-sm-red (there's a mistake in this one by the way, I've already notified them), and the overlay so many times I can recite it.

I thought that maybe the MPU was requesting the interrupt before the GPIO expander was ready but because the MPU9250 is a loadable module I used modprobe to remove and re-install in hopes that it would find it later and that didn't work, it gave the same error.

Any help or ideas to try would be welcomed.

Thanks,
Dave

PhilE
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2374
Joined: Mon Sep 29, 2014 1:07 pm
Location: Cambridge

Re: inv-mpu6050 trigger probe fail -22

Thu Oct 03, 2019 12:35 pm

I'm wondering if the sx1509 driver is rejecting the attempt to use an interrupt, or whether it's happening in one of the Linux frameworks. Since you're already building your own kernel, try adding some tracing to drivers/pinctrl/pinctrl-sx150x.c, concentrating initially on the places where it returns -EINVAL.

gizmomouse
Posts: 11
Joined: Mon Jun 10, 2019 2:45 am

Re: inv-mpu6050 trigger probe fail -22

Fri Oct 04, 2019 10:56 am

PhillE,

Thanks a lot for the feedback, I should have thought about throwing in 'printks'.

I checked the pinctrl-sx150x as well as associated files and everything returned with no errors.

Using tags I ended up in the file <kernel>/irq/manage.c

here is the sequence of functions that let to '__setup_irq' (can be read as file:function name->function called)

Code: Select all

<kernel>/dirvers/iio/imu/inv_mpu6050/inv_mpu_core.c:inv_mpu_core_probe->inv_mpu6050_probe_trigger
<kernel>/drivers/iio/imu/inv_mpu6050/inv_mpu_trigger.c:inv_mpu6050_probe_trigger->devm_request_irq
<kernel>/include/linux/interrupt.h:devm_request_irq->devm_request_threaded_irq (The 'irq_handler_t thread_fn' is set to 'NULL')
<kernel>/kernel/irq/devres.c:devm_request_threaded_irq->request_threaded_irq
<kernel>/kernel/irq/manage.c:request_threaded_irq->__setup_irq
<kernel>/kernel/irq/manage.c:__setup_irq
What is strange is that within <kernel>/include/linux/interrupt.h:devm_request_irq when it calls the function 'devm_request_threaded_irq' one of the function parameters 'thread_fn' is set to NULL

Then when the code reaches <kernel>/kernel/irq/manage.c:__setup_irq

  • There is a check to see if this irq is nested, which is what the sx150x does, it's a nested interrupt.
  • If it is nested then it checks the 'thread_fn' and because it's NULL it returns -EINVAL
Now I know where the issue is, I'm trying to track down what 'thread_fn' is for and why it's NULL?



If anyone has any ideas let me know.

Dave

[Edit]
Should have searched all leads before I posted this:

'thread_fn' is an interrupt handler function for threaded interrupts

Within 'devm_request_threaded_irq' there is a note: "'thread_fn' function to be called in a threaded interrupt context. NULL for devices which handle everything within in the 'handler'"

The 'handler' is the normal interrupt handler supplied by 'inv_mpu6050_probe_trigger' function, specifically 'iio_trigger_generic_data_rdy_poll' I don't see any place where the thread_fn is modified to fix this issue, as least in this chain of functions.

It seems as though the only way this would work is if the interrupt for mpu9250 was not nested.

PhilE
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2374
Joined: Mon Sep 29, 2014 1:07 pm
Location: Cambridge

Re: inv-mpu6050 trigger probe fail -22

Fri Oct 04, 2019 1:08 pm

It seems as though the only way this would work is if the interrupt for mpu9250 was not nested.
Or if it were threaded? I found this page explaining the difference between chained and threaded interrupts: https://stackoverflow.com/questions/343 ... ed-to-used

It's not clear to me how one would create an interrupt handler that is safe to be used in both/all contexts, since the helper functions required are different.

gizmomouse
Posts: 11
Joined: Mon Jun 10, 2019 2:45 am

Re: inv-mpu6050 trigger probe fail -22

Sat Oct 05, 2019 2:51 am

I looked at another industrial IO sensor, specifically: <kernel>/drivers/iio/gyro/bmg160_core.c

this driver uses the 'devm_request_threaded_irq' directly and I found that it used the 'thread_fn' only for events. I looked around the 'inv_mpu6050' driver and found no events to manage so I wrote a small irq handler that simply returned the IRQ_HANDLED and modified the the 'inv_mpu6050_probe_trigger' to use 'devm_request_threaded_irq'

Here is a diff of that file if anyone is running into this issue:

Code: Select all

136a137,141
> static irqreturn_t inv_mpu6050_event_handler(int irq, void *private)
> {
>   return IRQ_HANDLED;
> }
>
149c154
<       ret = devm_request_irq(&indio_dev->dev, st->irq,
---
>       ret = devm_request_threaded_irq(&indio_dev->dev, st->irq,
151c156,157
<                              irq_type,
---
>                              &inv_mpu6050_event_handler,
>                              irq_type,
I have it working, looking around the /sys/bus/iio/devices/iio:device0 everything looks good but I'll need to figure out a way to test it. Once I do I'll see about submitting a patch.

Thanks again for your help PhilE!

Dave

gizmomouse
Posts: 11
Joined: Mon Jun 10, 2019 2:45 am

Re: inv-mpu6050 trigger probe fail -22

Tue Oct 08, 2019 10:21 am

For anyone looking for a solution, I can't say that this is it. I need to research if this 'thread_fn' is correct. Once I bring up the rest of the board I can come back to this so, for the moment, this is on the backburner. So far the driver has been stable but be warned.

Return to “Device Tree”