User avatar
Grumpy Mike
Posts: 938
Joined: Sat Sep 10, 2011 7:49 pm
Location: Manchester (England England)
Contact: Website

Who knows where the time goes?

Thu May 31, 2012 11:20 am

Who knows where the time goes? Not a favorite song from Sandy Denny, but a question about using the GPIO pins.
I know that running under an operating system chunks of time disappear as ISRs are serviced but I was not prepared for what I found, being somewhat of a beginner at this Linux game. I modified the standard I/O code to output a binary count on all the GPIO pins brought out to the expansion header.
The relevant part is this:-

Code: Select all

// Set all exposed GPIO pins to output
  for (g=0; g<=32; g++)
  {
    if( ((1<<g) & 0x3e6cf93) != 0){
    INP_GPIO(g); // must use INP_GPIO before we can use OUT_GPIO
    OUT_GPIO(g);
   }
  }
for(; ;){
  for (rep=0; rep<0x3ffffff; rep++)
  { GPIO_SET = 0x3e6cf93 & (rep);
    GPIO_CLR = 0x3e6cf93 & (~rep); 
  }
}
  return 0;

} // main
So the count is going but it only changes the GPIO pins on the header (that's the 0x3e6cf93) bit.
I was checking on the scope to see that I had identified the pins correctly as there is a bit of a doubt with Wiki entries being wrong.
This is the normal trace of GPIO 11 & 15 and is what you would expect

Then I saw some time being stolen:-

I was expecting some but 10.2mS seems excessive. I had no network connection just a keyboard and mouse using debian6-19-04-2012
Anyone know how to disable interrupts to stop this?
Attachments
PI glitch.png
Glitch
PI glitch.png (7.76 KiB) Viewed 8671 times
Pi_Pin11_Pin15.png
Normal operation
Pi_Pin11_Pin15.png (8 KiB) Viewed 8671 times

kasperl
Posts: 90
Joined: Fri Jan 06, 2012 6:20 pm

Re: Who knows where the time goes?

Thu May 31, 2012 12:16 pm

I'm not sure you can easily disable the interrupts, but you might want to look into the RealTime Linux Kernel, it's made to use Linux and near-realtime GPIO.

andyl
Posts: 265
Joined: Tue Jan 10, 2012 11:05 am

Re: Who knows where the time goes?

Thu May 31, 2012 3:41 pm

I think that this depends on just how exact you want to time the pulses.

If you want them very exact then I think you will be out of luck.

Standard Linux just isn't very good at hard realtime.

Some things that may help. Run your program at a better priority. Try nice -20 to see if that helps.

As kasperi says you can try the linux realtime kernel, although I have to say that the current kernel is pretty low latency.

User avatar
gordon@drogon.net
Posts: 2022
Joined: Tue Feb 07, 2012 2:14 pm
Location: Devon, UK
Contact: Website Twitter

Re: Who knows where the time goes?

Sat Jun 02, 2012 10:47 am

This is where a multi-tasking (multi-user) operating system like Linux is really going to struggle... There are real-time variants of Linux though, and lots of patches to the kernel and so on, however from my initial (brief) checks on it, they're not going to give you the cycle-counting accuracy of an embedded microcontroller where you have control over everything.
Since your running the program as root, then it might be possible to disable interrupts - certianly at the device drvir (kernel) level there are ways to disable interrupts -however I'm sure many people will suggest it's really not recommended in user-land code whre you could probably execute a line or 2 of ARM assembler to disable all interrupts (as root) If you keep interrupts off for any length of time then Linux isn't going to be happy - the clocks will run slow, hardware might time-out and so-on.
As to what's causing your 10mS "gap", who knows. It might be some process accessing the disk (SD), so screen operation, X window operation, who knows...

What I do know is that trying to achieve a high-precision timed pulsed output is going to be nearly impossible in user-land by "bit banging". However there are hooks to do software PWM on GPIO pins built into the Linux kernel, so that might be worth investigating, however PWM won't control the likes of stepper motors and so on.

My view is that the GPIO on the Pi is fine for doing simple tasks like LEDs, switches and anything non timing critical (e.g. bit-banged IO to read things like the SHT15 temp/humidity sensor) Relatively simple motor control is possible too, but I'm not sure the Pi is ready to win the micromouse competition - yet! But in addition to the IO, there is I2C and SPI - and drivers are now working for these to communicate with a variety of other sensors and so on. I'm also working on making the Pi talk to (e.g.) Arduinos and have already extended "wiring" back into the Pi to control the local GPIO pins and a remote Arduino's IO pins.

Pi reading the SHT15 temp/humidity sensor and outputting the temperature in binary on 8 LEDs:

Image

-Gordon
--
Gordons projects: https://projects.drogon.net/

User avatar
Dave_G_2
Posts: 196
Joined: Sat Apr 14, 2012 7:04 pm

Re: Who knows where the time goes?

Sat Jun 02, 2012 10:59 am

@Grumpy Mike

What does your kernel report:

Code: Select all

strace -tt YourPgmName
It would also be very interesting to see if there is any significant time difference
if the app was coded using ASM and accessing the registers directly thus bypassing the drivers.

User avatar
gordon@drogon.net
Posts: 2022
Joined: Tue Feb 07, 2012 2:14 pm
Location: Devon, UK
Contact: Website Twitter

Re: Who knows where the time goes?

Sat Jun 02, 2012 12:31 pm

Dave_G_2 wrote:@Grumpy Mike

What does your kernel report:

Code: Select all

strace -tt YourPgmName
It would also be very interesting to see if there is any significant time difference
if the app was coded using ASM and accessing the registers directly thus bypassing the drivers.
His C program is directly accessing the registers - no device driver in there at all. There are 52 GPIO pins on the chip, accessed via 2 x 32-bit registers - GrumpyMikes code is poking the 1st register directly with a mask of 0x3e6cf93 which (assuming he's set all the output), will affect 15 of all the 17 usable bits in it. Re-coding in assembler might save a cycle or 2, but not 10mS.

I don't have a 'scope or I'd give it a go myself, but a C program ought to be able to toggle a pin in the MHz range. (until it gets interrupted)

-Gordon
--
Gordons projects: https://projects.drogon.net/

User avatar
Dave_G_2
Posts: 196
Joined: Sat Apr 14, 2012 7:04 pm

Re: Who knows where the time goes?

Sat Jun 02, 2012 2:24 pm

gordon@drogon.net wrote:
His C program is directly accessing the registers - no device driver in there at all.
Now that you mention it, I agree.
I just skimmed thru his code and forgot that GPIO_CLR and GPIO_SET are declared in some .h file.
gordon@drogon.net wrote: Re-coding in assembler might save a cycle or 2, but not 10mS.
It would still be an interesting exercise to disassemble the code produced by gcc and compare
that to one done in ASM.
gordon@drogon.net wrote:
I don't have a 'scope or I'd give it a go myself, but a C program ought to be able to toggle a pin in the MHz range. (until it gets interrupted)
Agreed.

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5658
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Who knows where the time goes?

Sat Jun 02, 2012 2:37 pm

There's lots of other userland processes running (try ps ax) that have equal rights to run as your program. I expect your program has just been timesliced away.

You could try playing with nice. E.g.
nice -n -20 ./myprog

should make myprog have more favourable scheduling. (But you still won't trump kernel code).

User avatar
Grumpy Mike
Posts: 938
Joined: Sat Sep 10, 2011 7:49 pm
Location: Manchester (England England)
Contact: Website

Re: Who knows where the time goes?

Sun Jun 03, 2012 8:43 am

gordon@drogon.net wrote:
Dave_G_2 wrote:@Grumpy Mike
GrumpyMikes code is poking the 1st register directly with a mask of 0x3e6cf93 which (assuming he's set all the output), will affect 15 of all the 17 usable bits in it. Re-coding in assembler might save a cycle or 2, but not 10mS.
-Gordon
No it affects all 17 usable pins not just 15 of them. Count the number of ones in 0x3e6cf93. I was also verifying the pinout of the header. This seems to have been screwed up by software types who don't know which end of a soldering iron is hot.

I am not interested in the speed, the least significant bit goes at 4.2MHz so there is little point in trying to poke the registers with assembler.
Yes there are other glitches as well as the massive 10mS one, some at 2mS and others around 5mS. It is just that I was not expecting such a massive glitch.
You could try playing with nice.
Thanks I will try that when I get on to my real application.

User avatar
gordon@drogon.net
Posts: 2022
Joined: Tue Feb 07, 2012 2:14 pm
Location: Devon, UK
Contact: Website Twitter

Re: Who knows where the time goes?

Sun Jun 03, 2012 9:57 am

Grumpy Mike wrote:
gordon@drogon.net wrote:
Dave_G_2 wrote:@Grumpy Mike
GrumpyMikes code is poking the 1st register directly with a mask of 0x3e6cf93 which (assuming he's set all the output), will affect 15 of all the 17 usable bits in it. Re-coding in assembler might save a cycle or 2, but not 10mS.
-Gordon
No it affects all 17 usable pins not just 15 of them. Count the number of ones in 0x3e6cf93. I was also verifying the pinout of the header. This seems to have been screwed up by software types who don't know which end of a soldering iron is hot.
From what I gather, it was laid out by a hardware engineer and one of the cost issues was - time vs. number of layers on the PCB - to get the routing from the BGA chip to the header (and to everything else). So AIUI, going from a 6 layer board to 8 layers plus the time would have pushed the price way over the target. So it's somewhat sub-optimal, but it's usable.
Grumpy Mike wrote: I am not interested in the speed, the least significant bit goes at 4.2MHz so there is little point in trying to poke the registers with assembler.
Yes there are other glitches as well as the massive 10mS one, some at 2mS and others around 5mS. It is just that I was not expecting such a massive glitch.
Grumpy Mike wrote:
dom wrote:You could try playing with nice.
Thanks I will try that when I get on to my real application.
I do not think you will ever get round things like this under Linux... Well, unles you go to the troubles of porting/adapting one of the real-time kernel variants and even then there are going to be other issues. Simply put you can not use a general purpose pre-emptive, multi-tasking, multi-user operating system from timing critical things like this.

However, you can improve things slightly using the sched_setscheduler(2) system call. (probably with a schedulling priority set to 10 and the SCHED_RR parameter) but that's only part of it - you may also want to look at mlock(2) to make sure your process never gets swapped out, however even with the real-time schedulling, it's still scedulling - your process will be de-schedulled when Linux needs to do someting else - for exameple background activity - network - any broadcast on your LAN will generate an interrupt (and your Pi needs to generate at least one of these every 30 seconds to process ARP requests to maintain the default gateway - and if it's a busy LAN, especially with Microsoft PCs broadbasing for domain controllers, etc. then there can easilly be more than one broadcast a second), then there's other more hidden things - cron jobs, log-file writes, (change the syslog config to not use fsync) and so on.

Probably far easier to use something more suited to this level of real-time control in the first place - e.g. an attached arduino/GertBoard an so on.. Do higher level control from the Pi and get a separate IO board to do more timing critical stuff.

The Pi is great for doing introductory stuff - lights, switches, simple sensors - and talking to more advanced sensors, etc. via the I2C and SPI, but proper "hard" real-time control? I really have my doubts as I suspect the first person to try to drive a stepper motor smoothly will find out.

Gordon
--
Gordons projects: https://projects.drogon.net/

User avatar
Walks42
Posts: 3
Joined: Fri Apr 27, 2012 6:09 am

Re: Who knows where the time goes?

Sun Jun 03, 2012 10:36 am

Sounds like a classic trade-off: Linux to get everyone access vs low-level control to REALLY be in control. I'm new to Linux but used to PICs (and earlier assembler on 8bit micros) and this is a pretty fundamental step. Is it realistic to EVER control the Broadcom/Pi in realtime?
I agree.. trying to control steppers without full control is reckless - better to devolve it (down I2C) to a PIC16F.

User avatar
gordon@drogon.net
Posts: 2022
Joined: Tue Feb 07, 2012 2:14 pm
Location: Devon, UK
Contact: Website Twitter

Re: Who knows where the time goes?

Sun Jun 03, 2012 11:05 am

Walks42 wrote:Sounds like a classic trade-off: Linux to get everyone access vs low-level control to REALLY be in control. I'm new to Linux but used to PICs (and earlier assembler on 8bit micros) and this is a pretty fundamental step. Is it realistic to EVER control the Broadcom/Pi in realtime?
Well, if you abandon Linux for something else, then yes :)

There are degrees of control avalable though - with the kernel scheduller set to pre-emptive and using the schedulling options then you can achieve what might be described as "soft" real-time control. Ie. you can make it so that your program does get a known percentage of the CPU, but you can never really get it to the state where you can accurately control hardware from userland in a timing-critical manner. However, with care you can get there - more or less. e.g. the Asterisk PBX uses a 1KHz timing source to synchronise all it's actions - and it has to clock packets out at a pre-set rate (50 packets per second) and it manages that surprisingly well - however there is a degree of tolerance to jitter in VoIP anyway, so the odd packet that's a few mS late isn't going to mange much difference, and our ears are good at making it up anyway - hardware isn't so good, so it might be wise to work out what the best schedulling interval is whee you can guarantee a set percentage of jitter in the output, but my suspicions are that its going to be under 100Hz. Input is going to be more of an issue - polling for a pulse on a pin is going to be hit or miss, however there are steps being made in the GPIO driver to connect a GPIO pin to an interrupt when you can effectively have a user process check for an interrupt, so you will get the signal (if a pulse), but you might not detect it for some time after the event. OK for a slow floor-crawler bumping into something where a few 10's of mS isn't going to hurt it, probably not OK for a balancing robot falling over..
Walks42 wrote:I agree.. trying to control steppers without full control is reckless - better to devolve it (down I2C) to a PIC16F.
Indeed - but AVR chips in my case ;-)

-Gordon
--
Gordons projects: https://projects.drogon.net/

User avatar
Grumpy Mike
Posts: 938
Joined: Sat Sep 10, 2011 7:49 pm
Location: Manchester (England England)
Contact: Website

Re: Who knows where the time goes?

Sun Jun 03, 2012 11:38 am

So say I want to do some audio sampling from an A/D, this means it is impossible to do such a thing without audio glitches. However, I am sure there are Linux sound samplers, how do they do it?

dom
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5658
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge

Re: Who knows where the time goes?

Sun Jun 03, 2012 11:46 am

Grumpy Mike wrote:So say I want to do some audio sampling from an A/D, this means it is impossible to do such a thing without audio glitches. However, I am sure there are Linux sound samplers, how do they do it?
You would need a small fifo between the ADC and ARM. Say enough for 1024 samples. You also have a kernel driver triggered from an interrupt which will have priority over user tasks. You can now tolerate 20ms of latenecy before losing any data.

User avatar
Grumpy Mike
Posts: 938
Joined: Sat Sep 10, 2011 7:49 pm
Location: Manchester (England England)
Contact: Website

Re: Who knows where the time goes?

Sun Jun 03, 2012 12:53 pm

dom wrote: You would need a small fifo between the ADC and ARM. Say enough for 1024 samples.
Extra hardware! :o - very unimpressed.
What makes you think 1024 X 16 is small for a FIFO?
Is that the only way to do it under Linux?

User avatar
gordon@drogon.net
Posts: 2022
Joined: Tue Feb 07, 2012 2:14 pm
Location: Devon, UK
Contact: Website Twitter

Re: Who knows where the time goes?

Sun Jun 03, 2012 1:03 pm

Grumpy Mike wrote:
dom wrote: You would need a small fifo between the ADC and ARM. Say enough for 1024 samples.
Extra hardware! :o - very unimpressed.
Why? You already want to add extra hardware - an ADC... So what's one more chip?
Grumpy Mike wrote: What makes you think 1024 X 16 is small for a FIFO?
Is that the only way to do it under Linux?
How do you think it's done with existing PC soundcards? This is what needs to be emulated. Alternatively just go out & buy a USB soundcard and use the standard drivers, however I guess that's not your aim.

-Gordon
--
Gordons projects: https://projects.drogon.net/

User avatar
Grumpy Mike
Posts: 938
Joined: Sat Sep 10, 2011 7:49 pm
Location: Manchester (England England)
Contact: Website

Re: Who knows where the time goes?

Sun Jun 03, 2012 1:26 pm

Why? You already want to add extra hardware - an ADC... So what's one more chip?
You can't get a FIFO like that in just one chip.

I never had to add a FIFO on a BBC model B, an Archimedes or a RISC PC, looks like the Pi is in the dark ages.

The more I find out about this board the less I like it.

pygmy_giant
Posts: 1562
Joined: Sun Mar 04, 2012 12:49 am

Re: Who knows where the time goes?

Sun Jun 03, 2012 1:46 pm

Well, I'm attempting a balancing robot with the Pi.

Technicaly speaking I beleive I need to be taking readings and updating wheel torque every 10ms in order to stay upright.

I'm intending to test the harware via linux and adapt / upgrade to bare metal as necessary.

I'll be sure to let you know when it falls over so you can all laugh.
Ostendo ignarus addo scientia.

hippy
Posts: 8520
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Who knows where the time goes?

Sun Jun 03, 2012 3:49 pm

Grumpy Mike wrote:
dom wrote: You would need a small fifo between the ADC and ARM. Say enough for 1024 samples.
Extra hardware! :o - very unimpressed.
What makes you think 1024 X 16 is small for a FIFO?
Is that the only way to do it under Linux?
You could perhaps use a software FIFO / circular buffer. Set up some timer that samples ADC at the desired rate and fills the buffer, and as long as you are emptying the buffer faster than it is filling you'll be okay.

Similar for DAC output; fill a buffer and use a timer tick set the rate of output and empties the buffer at a constant rate.

I don't know if you can do all that with Linux nor how but it's a generic problem when using any OS and sometimes when not.

For best throughput and minimum demands on user apps and OS, the buffering is usually handed-off to hardware which performs buffering itself. That's common for soundcards, network interfaces, even serial ports. The OS and/or user apps then only have to do anything when a new buffer-load arrives or the hardware wants a new buffer-load providing. It's quite common to have hardware and OS buffering allowing the hardware to work flat-out and the user app receiving or generating the data to work at its leisure without worrying about when or how long it's suspended for.

User avatar
gordon@drogon.net
Posts: 2022
Joined: Tue Feb 07, 2012 2:14 pm
Location: Devon, UK
Contact: Website Twitter

Re: Who knows where the time goes?

Sun Jun 03, 2012 6:03 pm

Grumpy Mike wrote:
Why? You already want to add extra hardware - an ADC... So what's one more chip?
You can't get a FIFO like that in just one chip.

I never had to add a FIFO on a BBC model B, an Archimedes or a RISC PC, looks like the Pi is in the dark ages.

The more I find out about this board the less I like it.
There's nothing wrong with the board, Mike. Nothing wrong with Linux either, but perhaps Linux is not the right tool for your job? Linux (and other unix OSs) are fantastic at what they achieve in terms of managing resources, hardware and so on, but it's really not the right tool for "hard" real-time control - unless you want to dive into kernel land and write device drivers to achieve your means.

-Gordon
--
Gordons projects: https://projects.drogon.net/

User avatar
Grumpy Mike
Posts: 938
Joined: Sat Sep 10, 2011 7:49 pm
Location: Manchester (England England)
Contact: Website

Re: Who knows where the time goes?

Sun Jun 03, 2012 6:37 pm

There's nothing wrong with the board,
I didn't say there was. I said I didn't like it.
Nothing wrong with Linux either,
Well I am beginning to think there is. Rather the way it is sold and evangelised. Or should I say over sold.

User avatar
gordon@drogon.net
Posts: 2022
Joined: Tue Feb 07, 2012 2:14 pm
Location: Devon, UK
Contact: Website Twitter

Re: Who knows where the time goes?

Sun Jun 03, 2012 6:58 pm

Grumpy Mike wrote:
There's nothing wrong with the board,
I didn't say there was. I said I didn't like it.
Nothing wrong with Linux either,
Well I am beginning to think there is. Rather the way it is sold and evangelised. Or should I say over sold.
Linux is free. So you get what you pay for.

However I don't know anyone pushing it for real-time control applications on general purpose systems. I know there are specialist versions for some very specific hardware, but I'd be really surprised if anyone was pushing it for general purpose real-time control for the sort of things you want to do. (Or what I think you're trying to do). Linux, and unix and unix-like system are just not suited for that sort of thing unless you're going to get right into the guts of it and write your own device drivers.

-Gordon
--
Gordons projects: https://projects.drogon.net/

User avatar
jbeale
Posts: 3716
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Who knows where the time goes?

Sun Jun 03, 2012 7:17 pm

Desktop Linux, just like Desktop Windows and every other general-purpose multitasking desktop system, does not suit hard-real-time control. I don't think you'll find any informed controls engineer who argues differently.

That said, people have done hard-real time control using software more suited to the job. See, for example
http://en.wikibooks.org/wiki/Embedded_Systems/Linux

By the way- little known fact- the R-Pi SoC actually contains a DSP core, and it is possible that it might be useful in real-time ADC sampling work, at some point in the future when an API becomes available.

User avatar
jbeale
Posts: 3716
Joined: Tue Nov 22, 2011 11:51 pm
Contact: Website

Re: Who knows where the time goes?

Sun Jun 03, 2012 7:43 pm

Grumpy Mike wrote:
Why? You already want to add extra hardware - an ADC... So what's one more chip?
You can't get a FIFO like that in just one chip.
Might not be exactly what you want, but you can get ADC and FIFO in a single chip. Here is a chip with a 10 channel 10-bit ADC, 3800 bytes RAM usable as a buffer, 2 SPI ports. Microchip PIC18F25J11. Costs under $3 in single-unit quantity.
http://www.microchip.com/wwwproducts/De ... e=en533682

If you want faster or higher-resolution ADCs, those are also available.
ADUC847: Precision Analog Microcontroller: 12MIPS 8052 Flash MCU + 10-Ch 24-Bit ADC + 12-Bit DAC
http://www.analog.com/en/processors-dsp ... oduct.html

Bakul Shah
Posts: 324
Joined: Sun Sep 25, 2011 1:25 am

Re: Who knows where the time goes?

Mon Jun 04, 2012 1:26 am

Grumpy Mike wrote:
Why? You already want to add extra hardware - an ADC... So what's one more chip?
You can't get a FIFO like that in just one chip.

I never had to add a FIFO on a BBC model B, an Archimedes or a RISC PC, looks like the Pi is in the dark ages.

The more I find out about this board the less I like it.
I think IDT still makes them if you need them. Such fifos even come with signals like almost empty, almost full etc. which help reduce s/w overhead and yet not lose any data. Or you can put a s/w fifo in an device driver. The problem is Linux, not the Raspi. It is not a hard realtime system.

Return to “C/C++”