snorremans
Posts: 12
Joined: Wed Jun 14, 2017 6:26 am

Loading of kernel module with overlay

Tue May 22, 2018 7:28 pm

Hello all,

I have been experimenting with overlays for the device tree for the rpi. I am using a rpi B+ V1.2 with buildroot and the kernel from the raspberry pi github (I've tried versions 4.14 through 4.16).

I am able to add the overlays to the device tree, for example the mcp3008-overlay by setting the value in config.txt. For example when I set

dtoverlay=mcp3008

I see the cmp3008 in the device tree through the /proc/ filesystem. But the kernel module is not loaded, even though the 'compatible' property in the device tree matches the one in the driver. Also the driver is installed because I can insert it manually with

modprobe mcp320x .

I can see the alias value in the /lib/modules/VERSION/modules.alias , so that means depmod has run. It is my understanding that the kernel should then load the module automatically. What part of the kernel/userland is responsible for loading the modules for nodes in the device tree? How can I debug this?

I've tried with other overlays, with the same results.

Many thanks!

PhilE
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1866
Joined: Mon Sep 29, 2014 1:07 pm
Location: Cambridge

Re: Loading of kernel module with overlay

Tue May 22, 2018 7:38 pm

The README entry for the mcp3008 overlay (which you can get by running "$ dtoverlay -h mcp3008" under Raspbian) says:

Code: Select all

Name:   mcp3008
Info:   Configures MCP3008 A/D converters
        For devices on spi1 or spi2, the interfaces should be enabled
        with one of the spi1-1/2/3cs and/or spi2-1/2/3cs overlays.
Load:   dtoverlay=mcp3008,<param>[=<val>]
Params: spi<n>-<m>-present      boolean, configure device at spi<n>, cs<m>
        spi<n>-<m>-speed        integer, set the spi bus speed for this device
You must tell the overlay which SPI bus and chip select to use - there is no default. To configure an MCP3008 on SPI0.0 you would use:

Code: Select all

dtoverlay=mcp3008,spi0-0-present

snorremans
Posts: 12
Joined: Wed Jun 14, 2017 6:26 am

Re: Loading of kernel module with overlay

Tue May 22, 2018 7:46 pm

Thanks,

Forgot to mention, I did that, for this particular overlay I've used the steps as in http://www.jumpnowtek.com/rpi/Using-mcp ... y-Pis.html .

So I've added the lines

dtparam=spi=on

and

dtoverlay=mcp3008:spi0-0-present,spi0-0-speed=1000000
.

Still the driver is not loaded though

PhilE
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1866
Joined: Mon Sep 29, 2014 1:07 pm
Location: Cambridge

Re: Loading of kernel module with overlay

Tue May 22, 2018 8:21 pm

It does sound like something is wrong with your system.

When the kernel encounters a device node it essentially sends a message containing an alias, causing a module to be loaded. You can monitor this by running:

Code: Select all

udevadm monitor > udev.txt &
(or run it without the redirection in another shell)

This will be started too late to capture the udev traffic during booting, but we can get round that by loading the overlay at runtime.

Remove or comment out the dtoverlay line in config.txt, reboot, start the udevadm logging as above, then:

Code: Select all

sudo dtoverlay mcp3008 spi0-0-present spi0-0-speed=1000000
udevadm should then show activity like this:

Code: Select all

KERNEL[44183.729688] add      /devices/platform/soc/3f204000.spi (platform)
ACTION=add
DEVPATH=/devices/platform/soc/3f204000.spi
MODALIAS=of:NspiT<NULL>Cbrcm,bcm2835-spi
OF_ALIAS_0=spi0
OF_COMPATIBLE_0=brcm,bcm2835-spi
OF_COMPATIBLE_N=1
OF_FULLNAME=/soc/[email protected]
OF_NAME=spi
SEQNUM=1330
SUBSYSTEM=platform

KERNEL[44183.729956] add      /devices/platform/soc/3f204000.spi/spi_master/spi0 (spi_master)
ACTION=add
DEVPATH=/devices/platform/soc/3f204000.spi/spi_master/spi0
OF_ALIAS_0=spi0
OF_COMPATIBLE_0=brcm,bcm2835-spi
OF_COMPATIBLE_N=1
OF_FULLNAME=/soc/[email protected]
OF_NAME=spi
SEQNUM=1331
SUBSYSTEM=spi_master

KERNEL[44183.731039] add      /devices/platform/soc/3f204000.spi/spi_master/spi0/spi0.0 (spi)
ACTION=add
DEVPATH=/devices/platform/soc/3f204000.spi/spi_master/spi0/spi0.0
MODALIAS=spi:mcp3008
OF_COMPATIBLE_0=mcp3008
OF_COMPATIBLE_N=1
OF_FULLNAME=/soc/[email protected]/[email protected]
OF_NAME=mcp3008
SEQNUM=1332
SUBSYSTEM=spi

KERNEL[44183.731541] add      /devices/platform/soc/3f204000.spi/spi_master/spi0/spi0.0/iio:device0 (iio)
ACTION=add
DEVNAME=/dev/iio:device0
DEVPATH=/devices/platform/soc/3f204000.spi/spi_master/spi0/spi0.0/iio:device0
DEVTYPE=iio_device
MAJOR=243
MINOR=0
OF_COMPATIBLE_0=mcp3008
OF_COMPATIBLE_N=1
OF_FULLNAME=/soc/[email protected]/[email protected]
OF_NAME=mcp3008
SEQNUM=1333
SUBSYSTEM=iio

KERNEL[44183.731772] add      /devices/platform/soc/3f204000.spi/spi_master/spi0/spi0.1 (spi)
ACTION=add
DEVPATH=/devices/platform/soc/3f204000.spi/spi_master/spi0/spi0.1
MODALIAS=spi:spidev
OF_COMPATIBLE_0=spidev
OF_COMPATIBLE_N=1
OF_FULLNAME=/soc/[email protected]/[email protected]
OF_NAME=spidev
SEQNUM=1335
SUBSYSTEM=spi

KERNEL[44183.731920] add      /devices/platform/soc/3f204000.spi/spi_master/spi0/spi0.1/spidev/spidev0.1 (spidev)
ACTION=add
DEVNAME=/dev/spidev0.1
DEVPATH=/devices/platform/soc/3f204000.spi/spi_master/spi0/spi0.1/spidev/spidev0.1
MAJOR=153
MINOR=0
SEQNUM=1336
SUBSYSTEM=spidev

UDEV  [44183.739662] add      /devices/platform/soc/3f204000.spi (platform)
ACTION=add
DEVPATH=/devices/platform/soc/3f204000.spi
DRIVER=spi-bcm2835
MODALIAS=of:NspiT<NULL>Cbrcm,bcm2835-spi
OF_ALIAS_0=spi0
OF_COMPATIBLE_0=brcm,bcm2835-spi
OF_COMPATIBLE_N=1
OF_FULLNAME=/soc/[email protected]
OF_NAME=spi
SEQNUM=1330
SUBSYSTEM=platform
USEC_INITIALIZED=44183736271

UDEV  [44183.741561] add      /devices/platform/soc/3f204000.spi/spi_master/spi0 (spi_master)
ACTION=add
DEVPATH=/devices/platform/soc/3f204000.spi/spi_master/spi0
OF_ALIAS_0=spi0
OF_COMPATIBLE_0=brcm,bcm2835-spi
OF_COMPATIBLE_N=1
OF_FULLNAME=/soc/[email protected]
OF_NAME=spi
SEQNUM=1331
SUBSYSTEM=spi_master
USEC_INITIALIZED=44183740916

UDEV  [44183.758377] add      /devices/platform/soc/3f204000.spi/spi_master/spi0/spi0.0 (spi)
ACTION=add
DEVPATH=/devices/platform/soc/3f204000.spi/spi_master/spi0/spi0.0
DRIVER=mcp320x
MODALIAS=spi:mcp3008
OF_COMPATIBLE_0=mcp3008
OF_COMPATIBLE_N=1
OF_FULLNAME=/soc/[email protected]/[email protected]
OF_NAME=mcp3008
SEQNUM=1332
SUBSYSTEM=spi
USEC_INITIALIZED=44183744427

UDEV  [44183.761766] add      /devices/platform/soc/3f204000.spi/spi_master/spi0/spi0.1 (spi)
ACTION=add
DEVPATH=/devices/platform/soc/3f204000.spi/spi_master/spi0/spi0.1
DRIVER=spidev
MODALIAS=spi:spidev
OF_COMPATIBLE_0=spidev
OF_COMPATIBLE_N=1
OF_FULLNAME=/soc/[email protected]/[email protected]
OF_NAME=spidev
SEQNUM=1335
SUBSYSTEM=spi
USEC_INITIALIZED=44183745020

UDEV  [44183.768564] add      /devices/platform/soc/3f204000.spi/spi_master/spi0/spi0.0/iio:device0 (iio)
ACTION=add
DEVNAME=/dev/iio:device0
DEVPATH=/devices/platform/soc/3f204000.spi/spi_master/spi0/spi0.0/iio:device0
DEVTYPE=iio_device
MAJOR=243
MINOR=0
OF_COMPATIBLE_0=mcp3008
OF_COMPATIBLE_N=1
OF_FULLNAME=/soc/[email protected]/[email protected]
OF_NAME=mcp3008
SEQNUM=1333
SUBSYSTEM=iio
USEC_INITIALIZED=44183768113

UDEV  [44183.769877] add      /devices/platform/soc/3f204000.spi/spi_master/spi0/spi0.1/spidev/spidev0.1 (spidev)
ACTION=add
DEVNAME=/dev/spidev0.1
DEVPATH=/devices/platform/soc/3f204000.spi/spi_master/spi0/spi0.1/spidev/spidev0.1
MAJOR=153
MINOR=0
SEQNUM=1336
SUBSYSTEM=spidev
USEC_INITIALIZED=44183769454
Of course, in a minimal buildroot system you may not have udevadm, but you also may not have a user-space process listening for udev messages, which would explain your problem.

snorremans
Posts: 12
Joined: Wed Jun 14, 2017 6:26 am

Re: Loading of kernel module with overlay

Tue May 22, 2018 8:28 pm

Thanks I'll try this.

You're saying the kernel sends a message when it encounters a node, so is it udev who receives this message and loads the module? In that case I might check out the udev configuration in buildroot. I will look into how to get the dtoverlay program in in my buildroot image.

I will update with results!

PhilE
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1866
Joined: Mon Sep 29, 2014 1:07 pm
Location: Cambridge

Re: Loading of kernel module with overlay

Tue May 22, 2018 8:34 pm

You can grab binaries of dtoverlay and libdtovl.so from opt/vc/bin and opt/vc/lib in the firmware repo.

epoch1970
Posts: 2019
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Loading of kernel module with overlay

Tue May 22, 2018 9:55 pm

I'm not sure this is relevant to the issue, but anyways.

Buildroot normally installs pre-built overlays it fetches from raspberrypi/firmware. So there is a chance of version mismatch between kernel and overlays.
The good Jumpnow has published how to patch your buildroot system so that it installs the overlays freshly compiled from the linux source tree.

The post is a bit old but this patch is still required AFAIK.
Or you can just get them from the builddir and fudge the generated boot partition with them.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

snorremans
Posts: 12
Joined: Wed Jun 14, 2017 6:26 am

Re: Loading of kernel module with overlay

Tue May 22, 2018 10:33 pm

Yup, that's also what I did. I'm testing some overlay I wrote myself. After I saw it wasn't loading the module I tried with an existing one (the mcp3008).

Currently rebuilding everything with buildroot, it seems udev is not included in the default config. But it still seems to me that the kernel would be responsible for loading the module, udev would be just for userspace programs to get information about plugging in devices, correct?

epoch1970
Posts: 2019
Joined: Thu May 05, 2016 9:33 am
Location: Paris, France

Re: Loading of kernel module with overlay

Wed May 23, 2018 8:11 am

To me, you're met with a buildroot-specific issue.
Jumpnow's configs/github repos build fine and produce a fully working system, IIRC. Maybe you could diff your defconfig files with his?

Also, the BR manual recommends
https://buildroot.org/downloads/manual/manual.html#_dev_management wrote:The Buildroot developers recommendation is to start with the Dynamic using devtmpfs only solution, until you have the need for userspace to be notified when devices are added/removed, or if firmwares are needed, in which case Dynamic using devtmpfs + mdev is usually a good solution.
"S'il n'y a pas de solution, c'est qu'il n'y a pas de problème." Les Shadoks, J. Rouxel

PhilE
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1866
Joined: Mon Sep 29, 2014 1:07 pm
Location: Cambridge

Re: Loading of kernel module with overlay

Wed May 23, 2018 9:09 am

The kernel has the ability to request that a specific named module is loaded, which it does by spawning an instance of /sbin/modprobe to load it; on Raspbian this is a symlink to /bin/kmod. The executable can also be changed by writing the preferred path to /proc/sys/kernel/modprobe.

You can debug whether modprobe is loading the module by inserting a man in the middle. Create a new script - /bin/mymodprobe:

Code: Select all

#!/bin/sh
echo modprobe $* >/dev/kmsg
exec /sbin/modprobe $*
Activate it by updating the /proc entry (this is safer than changing symlinks because it won't survive a reboot):

Code: Select all

$ sudo sh -c "echo /bin/mymodprobe > /proc/sys/kernel/modprobe"
After this, all kernel calls to request_module should result in a message being added to the kernel log, which you can read using dmesg.

I've found that I get lots of messages for the packet filter modules, but loading an overlay shows nothing, even though it has caused a module to be loaded. Because I was using a throw-away image I felt brave enough to modify the symlink to point to my script (and make a few other tweaks - don't try this at home), and reboot, at which point I was a bit surprised to see "modprobe spi:spidev" and "modprobe spi:mcp3008", so it seems that "hotplugging" at run-time uses a different mechanism to boot-time device instantiation.

snorremans
Posts: 12
Joined: Wed Jun 14, 2017 6:26 am

Re: Loading of kernel module with overlay

Wed May 23, 2018 3:05 pm

Thanks,

I've tried rebuilding the image with the udev config enabled, when doing this the module actually gets loaded, which confuses me even more as I thought udev would have nothing to do with this.

I will try editing the /proc/ entry next without udev to get an idea what is happening. Is this mechanism documented somewhere? So far I have not been able to find it.

PhilE
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1866
Joined: Mon Sep 29, 2014 1:07 pm
Location: Cambridge

Re: Loading of kernel module with overlay

Wed May 23, 2018 4:16 pm

At least we're on the right track.

Googling for "linux kernel udev modprobe kmod" (and subsets thereof) turns up quite a bit of useful information.

snorremans
Posts: 12
Joined: Wed Jun 14, 2017 6:26 am

Re: Loading of kernel module with overlay

Thu May 24, 2018 1:42 am

So I've tried the following steps:

Create script /bin/mymodprobe:

Code: Select all

#! /bin/sh

echo modprobe $*
exec /bin/busybox modprobe $*
Point symlink to the script:

Code: Select all

ln -s -f /bin/mymodprobe /sbin/modprobe 
When rebooting, the only time its called is:

Code: Select all

modprobe -q -- net-pf-10
which I think is from the init script.

So it seems that udev is needed to load drivers for device nodes that are present in the device tree? But why is this path /proc/sys/kernel/modprobe needed then if not for the kernel to load drivers?

The buildroot fork from jumpteknow has udev enabled also, so that might be another clue.

snorremans
Posts: 12
Joined: Wed Jun 14, 2017 6:26 am

Re: Loading of kernel module with overlay

Fri Jun 01, 2018 3:39 pm

So finally after some experimentation it seems that the modules are loaded by udev/eudev/mdev if the node is added by an overlay but they are added by the kernel(?) if the node is present in the original device tree.

If I want to have the module loaded in a system without udev, the node should be present in the device tree itself. Maybe an overlay can change the "status" property to "okay" though to make changes with an overlay?

PhilE
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1866
Joined: Mon Sep 29, 2014 1:07 pm
Location: Cambridge

Re: Loading of kernel module with overlay

Fri Jun 01, 2018 3:46 pm

Overlays can be applied before the kernel starts using "dtoverlay-..." in config.txt; as far as the kernel is concerned the result should be indistinguishable from a modified .dtb file. They can also be applied at runtime with the "dtoverlay" command - these are clearly different. Which type of overlay application are you referring to?

As far as Device Tree is concerned, if the status property is "disabled" then the node is effectively hidden and might as well not be there. There should be no advantage (at least as far as who loads the module goes) to be gained by moving nodes from an overlay into the base dtb and just using the overlay to set the status to "okay".

snorremans
Posts: 12
Joined: Wed Jun 14, 2017 6:26 am

Re: Loading of kernel module with overlay

Fri Jun 01, 2018 4:29 pm

Allright, I thought I finally got it but I guess not! You were right I made a mistake, even if I include the node in the original device tree the module is only loaded with udev included.

I'm using the method of writing "dtoverlay=myoverlay" in config.txt. So if this makes no difference to including it in the device tree, it seems like you need udev to load modules automatically even if there are nodes in the device tree with the corresponding "compatible" property? But I thought udev is only for notifying userspace if a device is hotplugged?

PhilE
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1866
Joined: Mon Sep 29, 2014 1:07 pm
Location: Cambridge

Re: Loading of kernel module with overlay

Fri Jun 01, 2018 4:47 pm

That behaviour (that the kernel always uses udev to load driver modules) does not match my experience, as I described earlier - for me, modprobe was used for the static device tree provided by the firmware (after applying config.txt overlays), and udev for devices hot-plugged with runtime overlays.

I don't really know what to suggest, other than adding lots of "pr_err()" calls to the kernel source for a custom build to get a feel for that paths taken.

snorremans
Posts: 12
Joined: Wed Jun 14, 2017 6:26 am

Re: Loading of kernel module with overlay

Fri Jun 01, 2018 7:07 pm

Thanks, maybe when I have time I'll dive a little deeper then :)

Btw I found this link : http://buildroot-busybox.2317881.n4.nab ... 53712.html , which contradicts your experience. But it looks like this is either a buildroot or busybox specific question, not so much rpi.

PhilE
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1866
Joined: Mon Sep 29, 2014 1:07 pm
Location: Cambridge

Re: Loading of kernel module with overlay

Fri Jun 01, 2018 7:23 pm

This comment is interesting:

In reply to this post by Bugzilla from [email protected]
https://bugs.busybox.net/show_bug.cgi?id=9541

--- Comment #4 from Artem Synytsyn <[hidden email]> ---
Now it works! Really, problem was in BR2_ROOTFS_DEVICE_CREATION_DYNAMIC_MDEV=y
option. Thank you very much!
And the reply:
Great, I'll close the issue then. Perhaps it makes sense to enable mdev for the
rpi*_defconfigs to make it easier to use?

Return to “Device Tree”

Who is online

Users browsing this forum: No registered users and 1 guest