Xenomai Patch for Raspberry Pi

[ian-cim]

I created a patch to get the Raspberry Pi to compile and work with the Xenomai 2.6.1 real-time framework with Linux 3.2.21, which I am sharing here with everybody.

After doing some minor testing, it seems to be running fine. Keep in mind that I've never done Linux kernel development before, so it could be that what the code does is wrong is some way, or could be improved. I welcome any suggestions or fixes!

Depending on the kernel configuration, the patch will use one of two clock sources for updating the timestamp counter. If power management is enabled, the 1MHz System Timer will be used. If power management is disabled, the ARM Timer, configured to run at 250MHz, will be used instead. Keep in mind that the ARM Timer’s clock is derived from the system clock. Since power management can dynamically change the system clock, the ARM Timer is only used if power management is disabled. See section 14 of the BCM2835 ARM Peripherals guide for more details.

To compile:

1. Get Chris Boot's rpi-3.2.21 branch (https://github.com/bootc/linux/tree/rpi-3.2.21).

2. Apply the Xenomai 2.6.1 patch:

Code: Select all

$xenomai_root/scripts/prepare-kernel.sh --arch=arm \
  --adeos=$xenomai_root/ksrc/arch/arm/patches/ipipe-core-3.2.21-arm-1.patch \
  --linux=$linux_tree

3. Apply the Raspberry Pi Xenomai patch (http://www.cim.mcgill.ca/~ian/rpi-linux ... .6.1.patch):

Code: Select all

cd $linux_tree
patch -p1 < rpi-linux-3.2.21-xenomai-2.6.1.patch

4. Compile and install!

I compiled it with the cross compilers from https://github.com/raspberrypi/tools.

Ian

P.S. Thanks to Juan for providing the hardware!

[Serac]

Looks like you took (most of) the ipipe modifications to timer-sp.c and pasted them in to bcm2708.c. Not that much different to the changes I was aiming for.

A couple of things to add to the build notes: CONFIG_CC_STACKPROTECTOR should not be enabled, and it is (currently) advisable to disable any KDBG support. Most of the kernel hacking options can safely be disabled, although the ipipe tracer is useful for debugging or fine tuning applications.

[ian-cim]

A couple of things to add to the build notes: CONFIG_CC_STACKPROTECTOR should not be enabled

If you're using bcmrpi_cutdown_defconfig, it is disabled by default.

Here's a couple of options that can reduce latency:

Disable: XENO_OPT_STATS
Disable: NO_HZ
Select "Preemptible Kernel (Low-Latency Desktop)" for "Kernel Features -> Preemption Model"

Ian

[ian-cim]

Looks like you took (most of) the ipipe modifications to timer-sp.c

Actually I didn't even see that file. I'll test a new implementation that uses the common code and post a patch once I get it working.

Ian

[Serac]

Gilles Chanteperdrix (one of the main Xenomai/ipipe developers) posted a timely note about Arm SoC porting today and points to: http://www.xenomai.org/index.php/I-pipe-core:ArmPorting for further reading. I think you have just about everything covered and it should migrate to the 3.4 series kernels when the next round of releases kicks off.

Unlike you, I won't be getting much free time until the weekend due to pressures at work.

[ian-cim]

Unlike you, I won't be getting much free time until the weekend due to pressures at work.

I should be working on other stuff too, but the rpi is just so fun! I'll take a look at Gilles' page this weekend. Looks like there's a lot of good information in there.

Btw, about the common/timer-sp.c code, the rpi has a modified version of the SP804 timer with some features removed, and some new ones. The timer-sp.c code might not be directly usable with the rpi in its current form. Though we could use it as a template. I was thinking of splitting up the bcm2708.c file into separate files to organize it a bit - would be nice to move all timer related code into one file.

Ian

[Serac]

A quick (very quick) test shows typical latencies in the order of 20-30uS and peaking at ~70uS for user space tasks. The kernel space latencies are even better and look pretty solid at around 15uS. Some fine tuning of config parameters are in order along with some more aggressive testing including running the xserver with some GPU intensive tasks.

Need to set up an environment to trace the slow execution path and see if the user space latencies can be improved upon.

Thanks for the patch Ian - You saved me a bit of time there that I can now spend on testing.

[ian-cim]

That is good to hear

I've also been getting similar latencies (20-30 us) as well in my tests.

[the_summer]

Same here.
I also compiled the bootc-kernel for Raspbian.
Works fine so far.
Without load the latency is even better (1-2 us).

Has anyone made even more progress yet (I have been on vacation the last 2 weeks)?

[Serac]

Works fine so far.
Without load the latency is even better (1-2 us).

To be honest (and the Xenomai developers will back this up), a no-load latency test is pretty much worthless - What you need to know is the worst case which can only be determined after prolonged testing under heavy load. With that, and the distribution of the samples (most will cluster around an "average"), you can then determine if performance is suitable for your task.

As for progress - I am awaiting some hardware to turn up so that I can track down a couple of fatal kernel bugs.

[mhaberler]

>> If power management is disabled, the ARM Timer, configured to run at 250MHz, will be used instead.

Is that a free running timer?
if so, is there a way to read it similar like the RDTSC read timestamp counter instruction?

- Michael

[Serac]

Whilst it is possible to read the timer counter directly in C, it is non-portable. The preferred method is to use library functions already provided rather than inventing your own.

Posix method:

Code: Select all

        struct timespec tp;
        clock_gettime(CLOCK_REALTIME, &tp)
        printf("Time = %i seconds, %i nanoseconds\n", tp.tv_sec, tp.tv_nsec);

Native method:

Code: Select all

        RTIME ticks;
        ticks = rt_timer_tsc();
        printf("Time = %i nanoseconds\n", rt_timer_tsc2ns(ticks));

This, and more, is all in the documentation that comes with the Xenomai sources.

[kinsa]

Here is the combined patch for 3.2.27:
http://dl.dropbox.com/u/17024524/ipipe- ... -rpi.patch

Step 3 needs to be skipped and Step 2 needs to be changed to this:

Code: Select all

$xenomai_root/scripts/prepare-kernel.sh --arch=arm \
  --adeos=ipipe-core-3.2.27-rpi.patch \
  --linux=$linux_tree

Cheers!