Real-time linux kernel

More thoughts, I think I may give up on this as I have no real time expertise. I imagine people that actually know anything about this stuff are intelligent enough to be silent.

But I have noticed as I booted the RPI this evening that the RPI showed something like the following:

Linux raspberrypi 3.1.9+ #125 PREEMPT ***** armv6

Maybe I noticed this due to the previous thread comment someone made about PREEMPT on the 2.6 kernel???

Unfortunatly I don't follow kernel devel and have no idea what the PREEMPT means for certain I am sure I have heard in the past that the kernel did have some sort of RT patch called RT_PREEMPT, but I get the feeling this would have stated RT_PREEMPT rather than PREEMPT. But maybe its worth checking???

From what I understood the PREEMPT patch maybe give soft realtime not hard realtime???

So I looked into the RT_PREEMPT possibilitys and found some links most of which I have not yet fully read, but I think possibly there may be a linuxcnc branch that uses only the RT_PREEMPT patch with no need for RTAI (I don't really follow linuxcnc and last time I looked it seemed RTAI was essential to run emc2).

So maybe there could be a simpler way to get emc2 running on the RPI by using RT_PREEMPT???

below are some more links I have found:
https://rt.wiki.kernel.org/index.php/CO ... T_RT_Patch
http://kerneltrap.org/node/2702
http://www.linuxfordevices.com/c/a/Linu ... es-Part-C/
http://wiki.linuxcnc.org/cgi-bin/wiki.p ... e_LinuxCNC

Just that I have seen the image of EMC2 on Raspberry Pi it's fantastic.
I am far to weak to help you in this endeavor but I will be your first tester !

Daniel

Just that I have seen the image of EMC2 on Raspberry Pi it's fantastic.
I am far to weak to help you in this endeavor but I will be your first tester !

Daniel

I thought I would just make an update of thoughts as I have really done very little more about getting real time linuxcnc running.


You may think that as the axis front end for emc2 (I think I should be refering to it as linuxcnc now ) is working there will soon be a working cnc controller available. Unfortunatly I think you are probably wrong .

The compilation I made was only of the simulator without real time kernel hooks, this was a fairly simple job only requiring a little research of apt-get install packages, and a short wrapper c file for emc2 headers and modification of 2 headers in /usr/include/

GETTING REAL TIME LINUX WORKING AND HAVING SUFFICIENTLY SMALL LATENCY AND JITTER COULD BE A HUGE PROBLEM!


I do not really have any low level real time kernel programming experience and I also have better things to do, when I was a young student maybe I would have dived in and spent many long evenings try to hack and debug things, unfortunately I cannot afford to now.

I think this would be a good project for a ee or compsci student to take up and would look very impressive on their resume, I hope someone will be interested enough to take up where I have left off.

As far as I can see the best way to go would be the raspbian 2.6 kernel and apply the i-pipe patches and use xenomai with the RTAI skin.

Unfortunatly I have become confused by the other options such as the RT_LINUX patches, but after looking into all options I still think 2.6 kernel and xenomai will be the quickest route(though really 3.2 kernel will be the future).

Unfortunatly I have seen that 3.1 kernels onward have a different scheduling system that xenomai and i-pipe do not yet support??? (apparently there are patches for the linux3.0 kernel for xenomai but I see no schedule for 3.1 or 3.2 kernel implementation)

It seems whatever the case there are things that need to be waited for, so I have stopped looking at this project, I will probably make a kernel compilation with ipipe patch once there is a reasonably stable release of the raspbian 2.6 kernel.

The general scheme I imagine would be to compile the default raspbian 2.6 kernel tree, apply the i-pipe patches, do the same for the i.MX3 realtime kernel tree from denx.de then make a diff between the important files and see how they differ, then use this information if necessary to modify the code for raspbian kernel.

Once a real time kernel is compiled there will probably be much testing and benchmarking required and then xenomai and RTAI skin need to be looked at.

So I guess what I am saying is the same as I said from the start, I doubt I will be the person that will get this working, but I hope others can find what I have done useful.

If anyone wants to see in more detail the very basic work I have done please let me know, I will try to make it available(though most is already posted on this thread).

Unfortunately I am not a real time kernel expert so I think this could take me far far longer than someone that already knows this stuff, I imagine that a xenomai expert maybe could do this in a couple of weeks hard work??

Would anyone be interested in starting a crowd source fund raiser to put some sort of price on getting this task done, maybe an expert would agree to undertake this for a few thousand?

I really don't like dealing with people so am not going to do it myself, but asking around on the relevant mailing lists maybe would get some experts interested?

I hope this will eventually all become reality with real time working flawlessly with low latency and jitter for linuxcnc on the RPI, but at present I am still unsure what the eventual outcome will be. Maybe the work required will be too grreat and we will have to wait for the next gen RPI??


Next gen RPI makes me think, would it be possible for the next broadcom ARM chip to have a few arduino AVR cores on the die with some sort of shared memory and multiplexing (transputer) and more IO pins, possible some additional very cheap custom support chip like mcp23017 i2c with DAC and ADC that can be plugged in an out of a DIL socket if its blown??.
I would have thought multiple additional AVR or PIC core would be very cheap and take tiny realestate compared with ARM cores, and could make handling real world interfacing without extra context switching and interupts easier(but if shared memory make it simple for the ARM to take over processing if necessary). I sort of see the way computers should be going as handling realtime real world interfacing (robots) and I think this needs large numbers of IO subsystems with fast inter system comms(I'm sure saying this last paragraph is pointless as I am no expert and others have a far better understanding of where things should be heading).


Anyways, I hope the future is a better place, but thats probably only true for the young .

Hello Mung,
Impressive amount of work and information !
Since I conclude that real time linux it's something far for Raspberry I was thinking to following solution :
viewtopic.php?f=63&t=9490&p=126225

Since raspberry has a serial connector on board it seems that it can be integrated easy with an GRBL/arduino board.

Could you please make a short description how to get LinuxCnc running on Raspberry without any realtime library ? It is possible something like that ?

Kind Regards,
DAniel

Quite a bit of misinformation in this thread:

* Xenomai is not, nor has it ever been "a fork of RTAI".
* There have been ipipe patches available for Arm on 3.x series kernels for some months.
* Xenomai's RTAI skin only supports (afaik) user space applications and is subject to substantial bit-rot.
* linuxcnc (a.k.a. emc2) will not work on anything other than an x86_32 without a lot of work - It is debatable wether the effort is worth expending on what will ultimately be nothing more than another series of ugly kludges.

Set up a cross-compile toolchain this afternoon and compiled a Xenomai enabled 3.2.21 kernel with the relevant ipipe patch - Need to run some latency tests over the weekend, but it looks good so far. Based on previous work with x86_64/RTAI, it would be a fairly short step to get RTAI to work over a 3.2.xx kernel. BUT... The Xenomai/ipipe team are deprecating floating point support in kernel space (and have actively discouraged it's use for a number of years). Without in-kernel FP support, emc2 is a dead end exercise period.

Well , it seems that a fellow made it work on and ARM processor :

http://www.youtube.com/watch?v=YSnNtgZrfyU

If could have the knowledge I would try it myself. His board it's running at 400mhz.

Set up a cross-compile toolchain this afternoon and compiled a Xenomai enabled 3.2.21 kernel with the relevant ipipe patch -

Hi Serac,

I'm trying to compile Xenomai with the ipipe core patches for 3.2.21 but I'm getting this error

Code: Select all

 
  CC      net/xfrm/xfrm_user.o
  LD      net/sunrpc/auth_gss/auth_rpcgss.o
  LD      net/sunrpc/auth_gss/built-in.o
  LD      net/sunrpc/sunrpc.o
  LD      net/sunrpc/built-in.o
  LD      net/xfrm/built-in.o
  LD      net/built-in.o
  LD      vmlinux.o
  MODPOST vmlinux.o
  GEN     .version
  CHK     include/generated/compile.h
  UPD     include/generated/compile.h
  CC      init/version.o
  LD      init/built-in.o
  LD      vmlinux
arch/arm/kernel/built-in.o: In function `ipipe_get_sysinfo':
ipipe.c:(.text+0x6110): undefined reference to `__ipipe_mach_get_tscinfo'
kernel/built-in.o: In function `xnintr_irq_handler':
proc.c:(.text+0x3e740): undefined reference to `__ipipe_mach_get_tsc'
proc.c:(.text+0x3e7f0): undefined reference to `__ipipe_mach_get_tsc'
proc.c:(.text+0x3e80c): undefined reference to `__ipipe_mach_get_tsc'
kernel/built-in.o: In function `xnintr_clock_handler':
proc.c:(.text+0x3e934): undefined reference to `__ipipe_mach_get_tsc'
proc.c:(.text+0x3e950): undefined reference to `__ipipe_mach_get_tsc'
kernel/built-in.o:proc.c:(.text+0x3e9a4): more undefined references to `__ipipe_mach_get_tsc' follow
make[1]: *** [vmlinux] Error 1
make[1]: Leaving directory `/localdata/raspbian_stuff/rpi-3.2.21'
make: *** [debian/stamp/build/kernel] Error 2


I compiled a cross compiler following Chris Boot's instructions here http://www.bootc.net/archives/2012/05/26/how-to-build-a-cross-compiler-for-your-raspberry-pi/ and patched the 3.2.21 kernel available here https://github.com/bootc/


Can you point me out how did you succesfully compile your patched kernel?


Thank you,

Juan Camilo

Well , it seems that a fellow made it work on and ARM processor :

http://www.youtube.com/watch?v=YSnNtgZrfyU

If could have the knowledge I would try it myself. His board it's running at 400mhz.

The key there appears to be miniemc2 http://code.google.com/p/miniemc2/ . You just need to port the source to R.Pi.

I have one of those mini2440 boards knocking around, I ported Rockbox to it.

Can you point me out how did you succesfully compile your patched kernel?

Been fighting with flaky SD cards, so haven't had the time to look at this in detail. The initial dry-run was with a vanilla kernel (kinda forgot about the Rpi specific patches ). Looks like the ipipe patch includes some stuff from the IXP4 code base - Need to look to see what is being included when CONFIG_IPIPE is enabled and "port" the necessary functionality across to the BCM2708 tree.

Hello Serac,

Would you please wrote a to do list in order to make the port of linuxCNC to R.Pi ?
I don't have necessary knowledge but If I would have a tutorial/guidance I would give it a try ...

Kind Regards,
Daniel

Would you please wrote a to do list in order to make the port of linuxCNC to R.Pi ?

First stage is to get the ipipe patch to play nice with the BCM2708 specific patches - This will have to be done without much in the way of support from the ipipe developers (they are not interested unless the code is in the mainline kernel tree).

Once the Rpi/ipipe patched kernel is up and running, there is one of two paths to follow to get emc2 (or linuxcnc if you prefer) running:

1) Get RTAI working on the arm platform - I'm really not interested in going down that road at present for a number of (undisclosed) reasons.

2) Modify emc2 to run under the Xenomai framework - Having seen one kludge layered on top of the current pile, I see little point in repeating the same mistakes. There are a number of fundamental flaws and bugs that need to be resolved before I waste any time going down that road.

Question: What do you need a CNC control system for, and why do you think that linuxCNC is the only (or best) solution ?

Althought I have said using a 2.6 kernel for rpi and patching with ipipe and xenomai looked like it maybe the easiest way, I have decided to look at RT_PREEMPT patches as it seems serac knows what he is doing and has the relevant patches.

I would like to apologise for any mistakes and misinformation that I may have posted (I am no real time expert and all of what I have posted is what I have found on google and I have not verified it).

As the raspbian image was released last week I was spurred to try a little more hacking over the weekend.

As others seem to be looking at xenomai I took a very quick look at RT_PREEMPT and grabbed the kernel source from http://www.bootc.net/projects/raspberry-pi-kernel/ (3.2.23 linux kernel actually downloaded as a zip from https://github.com/bootc/linux/tree/rpi-3.2.23), I then grabbed the linux RT patch from http://www.kernel.org/pub/linux/kernel/projects/rt/3.2/. applying the patch with patch -p1 < patchfilename, gave me what appeared to be a totally clean patch.

Compile without errors, I copied the kernel to /boot/ and then reboot...........

CRASH ((((((

The boot messages stop with:

[ 1.934170] Init: Power Port (0)

I have no idea what this means some sort of deadlock or what ever I do not know???

Anyways I really am not kernel hacker and have very very little kernel knowledge (last time I looked at kernel code was in 2.2 series).

So I moved on to trying the linuxcnc rt_preempt compilation to see what happens....

followed instructions at http://wiki.linuxcnc.org/cgi-bin/wiki.p ... e_LinuxCNC
I got as far as ./configure, but that fails as it cannot find some files in /usr/realtime???

I think maybe the realtime kernel must be running for it to work (maybe /usr/realtime is created when the real time kernel boots??)

I may have missed some required packages with apt-get, I have no idea, but will have a quick look at it again next weekend.

Most of what I am doing is unskilled guesswork, so please feel free to give advice.

Why do I think we should support LinuxCNC?

First, it works. If supported well on RaspberryPi then LinuxCNC can run on a lower cost platform (also lower power, and physically smaller than the historical platforms).

LinuxCNC is best known for it's being able to drive multi-axis CNC machines, and is not limited to 3 or even 6 DOF axese, using g-code.

It is used for SCADA robot programming (a particular flavor of pick and place robot arm), and PLC (Program Logic Controller) type Ladder Logic. PLC's tend to be the 'little black boxes' that run much of the industrial control systems in the world. Ladder Logic is a simple (but not simplistic) way of programming and documenting these devices without having to have a formal programmer or programming team available. Many 'plant operators' in industrial plants can program ladder logic, where C or Python would be lost on them (without significant additional training).

LinuxCNC also has some historical significance, at least for me, of being one of the first widely available control systems designed for public use. It was initially provided by NIST (US Goverments National Institute of Science and Technology, I believe) as EMC (Enhanced Machine Controller) to help jump start the US industrial complex into using computers for industrial control systems. EMC2 was an enhanced/redo of EMC, and the name change to LinuxCNC came from pressure from the EMC company.

servant74 - Your only two posts to date have pretty much made the same claims and just in case you missed my responses:

1) No documents have been presented to support the claim that the EMC Corporation had initiated legal action to force a name change. ergo: The rebranding is the result of an alleged and unsupported claim.

2) You state that the emc2 code is not pretty, ..... It is also not easy to get running outside of the many pre-configured options. - It is damned ugly, convoluted, and quite frankly, in a real mess and I have neither the time nor the inclination to clean up the puppy puke. However, if you want to do something, go right ahead. The first obvious task would be to get emc2 to build in a cross-compile environment without error.

3) The PLC section you refer to is ClassicLadder written by Marc Le Douarain back in 2001 and is still supporting/updating the code today. It has always been an independent project outside of NIST or LinuxCNC. It's integration in to emc2 has been poorly managed and provides little benefit to the majority of users.

4) Sure, EMC supports all kinds of parallel motion machines (e.g. stewart platforms) and SCARA robotic arms - Most, if not all, exist only inside a lab and I doubt if more than a handful of people are using them with emc2 out in the real world.

5) The original question was not "Why do I think we should support LinuxCNC?", but rather "What do you need a CNC control system for, and why do you think that linuxCNC is the only (or best) solution ?" The picture ten/fifteen years ago was limited to a few DOS/Windows programs, today there are a few more choices. However, for anyone wanting to use a Linux computer, the choices are very limited with just two or three main contenders. If you want to support emc2 or linuxcnc on a Raspberry Pi then get on and say it. Just don't include those around you in the we without asking first.<reaches for the clue bat>

(In an attempt to get this thread back on track)

Althought I have said using a 2.6 kernel for rpi and patching with ipipe and xenomai looked like it maybe the easiest way, I have decided to look at RT_PREEMPT patches...

Most of what I am doing is unskilled guesswork, so please feel free to give advice.

You probably need to look at some of the other ARMv6 specific code and see what changes have been made to (amongst other things) the timer support routines then port them to the BCM2708 BSP. Not an easy task unless you are familiar with kernel internals.

On the upside, made some changes to tsc emulation in my kernel tree and it now compiles without any significant error and boots to the login screen - Time is limited at the moment, so testing will have to wait for another day.

I posted in the Other Projects forum a patch I made to get Xenomai 2.6.1 working with the Linux 3.2.21 kernel if anyone is interested:

viewtopic.php?f=41&t=12368

Ian

I get the feeling there are more skilled people looking at this now so I doubt my hacking will be of much use, and also I feel like I maybe hijacked this thread and am taking it off topic with my constant references to linuxcnc .

I have found that maybe there will be problems with raspbian as I cannot run linuxcnc when compiled in sim mode (seems to compile with but does not run), I have noticed compile errors "swp(b) is deprecated for this architecture" but maybe I am just doing something wrong, as it worked okay on squeeze.

I am now wondering though if there are some serious problems with the raspberry pi kernel as I seem to get really bad freezing maybe due to viewtopic.php?f=28&t=7866&start=50

I think maybe there could be real problems running all peripherals and drivers and getting any kind decent real time performance especially as the drivers are closed source

Of course could run the kernel with minimal drivers.

I did see this thread http://www.xenomai.org/pipermail/xenoma ... 00283.html that suggests there could be good chances that xenomai has 3.2 ARM kernel support NOW. Of course I have been led to believe that xenomai RTAI skin only supports userland calls, what does linuxcnc use?

I took a look at the RT_PREEMPT route last weekend as I saw that linuxcnc had been ported to use RT_PREEMPT, but I could not get the 3.2 RPI kernel to boot with the RT_PREEMPT patches

I am not sure what I will be hacking this weekend, I think maybe I may just move on to another project and come back when someone more skilled has done all the hard work.

One thing about the Xenomai team is that they are quick off the mark to get their realtime framework running on current kernels on all the architectures they support. ARM support on 3.2 kernels has been available for some time and you can expect a 3.4 series release before too long. What will hold back most people with a Pi is the lack of support in the mainline kernel for the Broadcom SoC so you'd be reliant on others to provide the rpi specific updates. Once they are available, the additional work to get the Xenomai/ipipe stuff working is relatively trivial.

For linuxcnc support, I'd suggest asking servant74 as he seems happy to take on the task - Failing that, go to the developer's mailing list and ask there. But don't expect anything worthwhile unless you are running their chosen (modified) distribution (see http://thread.gmane.org/gmane.linux.distributions.emc.devel/5944/ as an example of a simple problem that gets a nasty response).

As for the RTAI skin in Xenomai, it is userspace only, the developers have admitted it was subject to bitrot and has been removed from current releases. For new applications the native & posix skins coupled with the RTDM api provides sufficient functionality for most. The other skins are only really intended to aid in the "porting" of legacy code.

In closing, I think you place too much on the notion of proprietary drives - Aside from the (XFree?)video, all the source code for Broadcom kernel drivers are available. The video driver [may] only be a problem when running a graphics intensive task (subject to testing).

I think I can say that the new RPI 3.2.27 kernel from git seems to patch with RT_PREEMPT with no problems and boots and runs X (X is noticabley slower than without RT, just hope the accelerated X server comes soon). I am hoping to try compiling and testing RT things this weekend.

I should qualify this by saying I have no idea about realtime kernel internal coding and just applied the patch to the new kernel and compiled it and installed it. I don't yet know how things will perform but hope to post some test results here after the weekend. Obviously some genius has been working hard behind the scenes and I would like to thank who ever is responsible.