Raspbian kernel headers missing from official image

[justme123]

Hello kernel developers/compilers!

I'd like to ask you, if you are so kind, to include the kernel headers with the official Raspbian image from now on. The entire folders structure is around 71MB:
du -s -h /usr/src/linux-headers-4.4.48-v7+/
71M /usr/src/linux-headers-4.4.48-v7+/

Raspberry Pi should be an open platform for developers and hacking enthusiasts, patching and compiling kernel modules should therefore be granted by default. To this date, the Raspbian official kernel is compiled with CONFIG_MODVERSIONS enabled (I'm always wondering why) and in order to be able to patch or compile a kernel module the afferent Module.symvers are always needed, hence at least the kernel headers are always requested.

Until now, every time I needed these kernel headers I was getting them from:
https://www.niksula.hut.fi/~mhiienka/Rp ... aders-rpi/
but with the kernel 4.4.50-v7+ included in the latest official Raspbian image from 10.04.2017 I got
some issues apparently because the Module.symvers from the official Raspbian kernel are different.

I know there are many ways to obtain them, discussed on forums, but all of them are pretty complicated and involve downloading some gigabytes of useless historical kernel data from git.

Again. If you'd like to make my (our) life happier, please include at least those kernel headers and the afferent Module.symvers with the official Raspbian image, as they only cost 71MB extra and it'll save a lot of bandwidth for the git servers where the kernel sources are located.

On this occasion, I'd also like to share the simple way I'm compiling modules on this forum
(prepare the environment):
Get the kernel headers for your running kernel from:
https://www.niksula.hut.fi/~mhiienka/Rp ... aders-rpi/
- get .config:
modprobe configs
config.gz is to be found in /proc
cp /proc/config.gz /usr/src/linux-headers-XXX
rmmod configs
cd /usr/src/linux-headers-XXX
gunzip config.gz
mv config .config

Prepare kernel headers / config:
make -j 4 oldconfig && make -j 4 prepare
make -j 4 modules_prepare

- compile your module(s) & install

Thanks!


P.S. Please do something with the authentication timeout on this forum. I had to rewrite the whole post again after I pressed submit and was redirected to the login page.

[DougieLawson]

https://github.com/notro/rpi-source/wiki

[justme123]

Thank you for your quick reply!

I was aware about that how-to and remember studying it some time ago.
Unfortunately, as I stated before, it implies downloading a lot of stuff from git, when only the headers are sufficient, and it's Raspbian (Debian) specific. Personally, for some specific and objective reasons, I'm not using Raspbian at all. Additionally, being a Linux veteran, using it for almost 2 decades, I never ever saw a serious Linux Distribution that was not containing the kernel sources in the official image!


But maybe it's easier to over-complicate things nowadays and write pages of instructions on how to obtain them, instead of just copying them over in the image together with the kernel, once compiled. (That's one of the reasons I'm not using Debian, Ubuntu, etc., together with apt-get, apt-unget, apt-you-never-know-what-you-get)

I still hope that one day we'll get the sources together with the image. Until then, I'll maybe address the guys from
https://www.niksula.hut.fi/~mhiienka/Rp ... aders-rpi/
and ask them what happened with the latest kernel symbols.

Me has left the building!

[DougieLawson]

It's currently the only supported option. The Raspbian *.deb package is too old to be useful.

[gregeric]

raspberrypi-kernel headers is available by apt-get & gives you the headers matching the installed (by apt-get) kernel.

[mfa298]

raspberrypi-kernel-headers is available by apt-get & gives you the headers matching the installed (by apt-get) kernel.

This! (added a dash in the package name)

Personally, for some specific and objective reasons, I'm not using Raspbian at all. Additionally, being a Linux veteran, using it for almost 2 decades, I never ever saw a serious Linux Distribution that was not containing the kernel sources in the official image!

I've seen very few that include the kernel sources / headers by default. It's almost always a separate install-able package. And that's been try for a decade or more.

If you're not using Raspbian at all then maybe it would make sense to aim this question at you're chosen distribution.

Hello kernel developers/compilers!

I'd like to ask you, if you are so kind, to include the kernel headers with the official Raspbian image from now on. The entire folders structure is around 71MB:
du -s -h /usr/src/linux-headers-4.4.48-v7+/
71M /usr/src/linux-headers-4.4.48-v7+/

This now makes little sense, If you're not using raspbian then why should they include the headers in the image?

The headers are easily there on raspbian if you need them:

Code: Select all

$ sudo apt-get install raspberrypi-kernel-headers

$ uname -a
Linux pi3.local 4.9.24-v7+ #993 SMP Wed Apr 26 18:01:23 BST 2017 armv7l GNU/Linux

$ du -chs /usr/src/linux-headers-4.9.24*
78M     /usr/src/linux-headers-4.9.24+
78M     /usr/src/linux-headers-4.9.24-v7+
156M    total

[MrEngman]

On my Pi 2B -

Code: Select all

$ uname -a
Linux Pi-64GB-1 4.9.25-v7+ #994 SMP Fri Apr 28 16:56:00 BST 2017 armv7l GNU/Linux

$ sudo apt-get install raspberrypi-kernel-headers
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  raspberrypi-kernel-headers
0 upgraded, 1 newly installed, 0 to remove and 2 not upgraded.
Need to get 15.3 MB of archives.
After this operation, 111 MB of additional disk space will be used.
Get:1 http://archive.raspberrypi.org/debian/ jessie/main raspberrypi-kernel-headers armhf 1.20170427-1 [15.3 MB]
Fetched 15.3 MB in 18s (851 kB/s)
Selecting previously unselected package raspberrypi-kernel-headers.
(Reading database ... 35028 files and directories currently installed.)
Preparing to unpack .../raspberrypi-kernel-headers_1.20170427-1_armhf.deb ...
Unpacking raspberrypi-kernel-headers (1.20170427-1) ...
Setting up raspberrypi-kernel-headers (1.20170427-1) ...
run-parts: executing /etc/kernel/header_postinst.d/dkms 4.9.24+
Error! Could not locate dkms.conf file.
File:  does not exist.
run-parts: executing /etc/kernel/header_postinst.d/dkms 4.9.24-v7+
Error! Could not locate dkms.conf file.
File:  does not exist.

du -chs /usr/src/linux-headers-*
78M     /usr/src/linux-headers-4.9.24+
78M     /usr/src/linux-headers-4.9.24-v7+
155M    total

Not very helpful. So what about the headers for 4.9.25-v7+?

And what's this issue with dkms.conf? Why does dkms.conf not exist "Error! Could not locate dkms.conf file."



MrEngman

[gregeric]

raspberrypi-kernel-headers gives you the version matching the kernel available on apt-get. I assume your 4.9.25 came from rpi-update, in which case rpi-source is your friend.

There is something amiss with dkms, my (esp8089) driver being uninstalled and not automatically rebuilt on apt-get kernel updates. Quickly resolved with a manual "dkms install...", haven't looked further.

[MrEngman]

raspberrypi-kernel-headers gives you the version matching the kernel available on apt-get. I assume your 4.9.25 came from rpi-update, in which case rpi-source is your friend.

Yes, but if you have a kernel version that doesn't match what is available or been installed using apt-get then you will have an issue as my example shows. I have a clone of the linux source which goes back to 2011 or so not long after the git repo was originally created and I update it as required with git pull and it doesn't require downloading GBytes to update as @justme123 suggests. Even doing a new clone you can reduce the amount to download with git clone --depth=xxx etc. @justme123 take a look at man git-clone.

There is something amiss with dkms, my (esp8089) driver being uninstalled and not automatically rebuilt on apt-get kernel updates. Quickly resolved with a manual "dkms install...", haven't looked further.

Not sure what this is all about with dkms. Never ever used it and have no real idea what it does. I cross-compile a few out of tree wifi drivers, using a script I've developed, which are available for anyone using Raspbian kernels to download and I usually recompile every time there is an update to hexxeh/rpi-firmware as I regularly update my Pis using rpi-update. It only takes me a few minutes to re-compile the drivers for Pi 1 and Pi 2, cross-compiling on an old PC I have.



MrEngman

[justme123]

I've asked to get my registration deleted but since I'm active I'm taking the opportunity to reply to some ineteresting and rather rigid answers:

gregeric:

- thanks for your inputs and hints, I'll try the apt-get (don't get) to obtain the kernel headers. Anyways I'm downloading the whole Raspbian image and taking the kernel and userland out of it. I think I would be able to mount it on a spare SDCard and boot it in order to nick the headers too.

mfa298:

- you're right and I was wrong. Raspbian isn't actually a distribution but a live image. And you were right again, the serious distributions have the kernel sources as an additional package, but you have the option to install them and they're always available as an easily obtainable archive, either on the distribution repository or in the ISO/Image. No apt-get (don't get) script or git cloning.
While git is a wonderful development management engine and handy for getting some patches or files, it has huge speed issues around the world, especially when you clone some tens/hundreds of MB.

- it doesn't make sense to aim my question to the folks at the distribution I'm using, which is Slackware by the way, because it's only Raspberry Foundation that compiles the kernel, which by the way contains some closed source Broadcom proprietary blobs, for the Raspberry Pi. Additionally, the folks at Raspberry Foundation are THE ONLY ONES that have the necessary closed-source knowledge to optimize the kernel for the Pi platform. Any other distribution that supports the Pi instructs you to take the kernel and userland from the official Raspbian image.

- Besides, with all this optimization knowledge, I'm considering the 4.4.48-v7+ kernel flawed as it has some weird softIRQ management - someone at Raspberry Foundation or kernel.org might have played with the deferred interrupts settings. Sometimes I just need to restart my Pi because the amount of softIRQs piles up to several thousand for a simple copy through ethernet operation. And there are lucky reboots and unlucky ones, sometimes I need to reboot it twice to have proper functionality restored.

- While I do respect Debian/Ubuntu, especially for the cutting edge development they're doing, I'm not using them because they're filled with convenient proprietary scripts and they're not following the all holy KISS principle. I'm more into learning and maintaining my general Linux/UNIX skills and be able to work on any *NIX platform with the default tools and get things done.


MrEngman:

- thanks for you help, I know how to use git but was never sure how to identify my running kernel
- I'm also seriously considering to compile the kernel on my own, especially after the experience with 4.4.48-v7+. Until now I was considering that the folks at Raspberry/Debian will know better than me how to optimize it.

General:

I still hope that 70MB are not that much to add to the 3.something gigabytes Raspbian image.


Thank you in advance!

[ghans]

You need to use apt-get at least once to get matching kernel headers from the internet on Debian. This has
always been the case on the PCs and Laptops i've used Debian on, even when installed via the official (net)installer.

The foundation is actually involved in upstreaming their modifications to the kernel. They are making progress on all fronts.

Raspbian runs on other machines too. If you want to set up a Debian-style system without the
foundation mods you might want to look into raspbian-ua-netinst , which many people prefer for staying closer to how Debian is installed anc configured.

https://github.com/debian-pi/raspbian-ua-netinst

[fruitoftheloom]

You need to use apt-get at least once to get matching kernel headers from the internet on Debian. This has
always been the case on the PCs and Laptops i've used Debian on, even when installed via the official (net)installer.

The foundation is actually involved in upstreaming their modifications to the kernel. They are making progress on all fronts.

Raspbian runs on other machines too. If you want to set up a Debian-style system without the
foundation mods you might want to look into raspbian-ua-netinst , which many people prefer for staying closer to how Debian is installed anc configured.

https://github.com/debian-pi/raspbian-ua-netinst

There is also Armbian Distro which is a Debian re-spin

[justme123]

The official supported way to obtain the kernel headers, although nowhere specified, looks like its only working on Raspbian:
https://github.com/notro/rpi-source/wiki

rpi-source
ERROR:
gcc version check: mismatch between gcc (5.4.0) and /proc/version (4.9.3)
Skip this check with --skip-gcc
Help: https://github.com/notro/rpi-source/wiki

rpi-source --skip-gcc
ERROR:
Can't find a source for this kernel
Help: https://github.com/notro/rpi-source/wiki

I was able to obtain the kernel headers by booting the Raspbian image and running:
apt-get install raspberrypi-kernel-headers

However, there are Raspbian/Debian ARM Compiler specific dependencies:

/usr/src/linux-headers-4.4.50-v7+# make -j 4 oldconfig && make -j 4 prepare
.....
make[1]: *** No rule to make target 'arch/arm/tools/gen-mach-types', needed by 'include/generated/mach-types.h'. Stop.
make: *** [arch/arm/Makefile:315: archprepare] Error 2

/usr/src/linux-headers-4.4.50-v7+# make -j 4 modules_prepare
.....
scripts/sortextable.c:31:32: fatal error: tools/be_byteshift.h: No such file or directory
compilation terminated.
make[1]: *** [scripts/Makefile.host:91: scripts/sortextable] Error 1
make[1]: *** Waiting for unfinished jobs....
HOSTCC scripts/dtc/util.o
HOSTCC scripts/dtc/dtc-lexer.lex.o
HOSTCC scripts/dtc/dtc-parser.tab.o
HOSTLD scripts/dtc/dtc
make: *** [Makefile:556: scripts] Error 2

This issue was freshly discussed here:
viewtopic.php?f=63&t=153166

Finally I was successful in fooling the Linux Media Drivers (Media Build) that it has a proper Kernel Source Environment.

It would be ideal to have a small archive containing what is needed for any Linux distro to support the Pi once a new Raspbian/Noobs image is released. A download section for Aliens/Martians/etc. could be easily added and it might make the Pi even more demanded and adopted, not to mention the smaller carbon footprint achieved with downloading only 110MB instead of 1.5 GB.

Here is what any other Linux distros usually need from Raspberry:

du -s -h raspbian_boot_stuff-4-4-50-v7-plus.tgz
88M raspbian_boot_stuff-4-4-50-v7-plus.tgz
Content:
boot/
lib/modules/4.4.50+/
lib/modules/4.4.50-v7+/
lib/firmware/
opt/vc/


du -s -h kernel-headers-4-4-50-v7-plus.tgz
19M kernel-headers-4-4-50-v7-plus.tgz
Content:
linux-headers-4.4.50+/
linux-headers-4.4.50-v7+/