Panalyzer - a RaspberryPi based Logic Analyzer


65 posts   Page 2 of 3   1, 2, 3
by higwoth1 » Mon Jun 18, 2012 8:24 pm
Compile kernel. Ah, in that case I might wait for the binary release. ;)
Posts: 26
Joined: Sat May 26, 2012 1:27 am
by secretagent » Thu Jun 21, 2012 4:18 am
There is another timer which can be made to run at up to 250 MHz and as far as I know is not used for anything else. It is described in chapter 14 of the datasheet. Here is some code I used for testing it:

Code: Select all
#include <linux/module.h>   
#include <linux/kernel.h>
#include <mach/platform.h>
#include <linux/io.h>

#define TIMER_CTRL (ARMCTRL_TIMER0_1_BASE+0x08)
#define TIMER_CTRL_EN_FREE_RUNNING (BIT(9))
#define TIMER_FREE_RUNNING (ARMCTRL_TIMER0_1_BASE+0x20)
#define ST_VALUE (ST_BASE+0x4)

int init_module(void) {
   u32 timer_start, timer_end, freq;
   u64 st_start, st_end;

   //ARM timer at 250MHz
   volatile u32 *arm_timer = (volatile uint32_t *)ioremap(TIMER_FREE_RUNNING, 4);

   //system timer at 1MHz
   volatile u64 *system_timer = (volatile uint64_t *)ioremap(ST_VALUE, 8);

   writel(TIMER_CTRL_EN_FREE_RUNNING, __io_address(TIMER_CTRL));
   
   local_irq_disable();
   local_fiq_disable();

   st_start = *system_timer+2;
   st_end = st_start+1000000; //run for 1 second
   while (*system_timer < st_start);
   timer_start = *arm_timer;
   while (*system_timer < st_end);
   timer_end = *arm_timer;
   freq = timer_end - timer_start;
   
   local_fiq_enable();
   local_irq_enable();

   printk("ARM timer freq = %u\n", freq);
   return 0;
}

void cleanup_module(void) {
   writel(0, __io_address(TIMER_CTRL));
}

MODULE_LICENSE("GPL");
Posts: 36
Joined: Wed Mar 07, 2012 10:09 am
by rgh » Thu Jun 21, 2012 6:14 am
secretagent wrote:There is another timer which can be made to run at up to 250 MHz and as far as I know is not used for anything else. It is described in chapter 14 of the datasheet. Here is some code I used for testing it:...[/code]


Oooh, thankyou! I'd obviously lost concentration by I got to page 196. Will play with this at the weekend. I'm also hoping that as it is 'arm side' I'll be able to read it in less than the 60ns it takes to read the system timer.
Posts: 220
Joined: Fri Nov 25, 2011 3:53 pm
by dom » Thu Jun 21, 2012 9:00 am
Be aware that the ARM timer has a "feature" that it runs off the GPU clock.
This is by default 250MHz, but may be reduced when GPU is idle, making it not useful for timing.
As a hack, "core_freq=250" in config.txt has the side effect of disabling this power management and should make that timer useful again.
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 4058
Joined: Wed Aug 17, 2011 7:41 pm
Location: Cambridge
by gordon@drogon.net » Sat Jun 23, 2012 3:02 pm
jbeale wrote:Amazing! The conventional wisdom was that this sort of thing was impossible on "standard Linux" given that it is not a real-time OS. So it is possible to disable ALL interrupts? It looks like you have at least several milliseconds of data shown, I'm curious how long you can hold off interrupts (10 ms? ...longer?) without ill effect to the system.


Almost anything is possible as root. The quesion being "Is it the 'done thing' on a multi-tasking OS?"... The answer being "Not really, but you can get away with it most of the time".

Some things may stop working, the time of day clock might start to go slow (if not being synced from NTP), device drivers waiting on an interrupt may time-out and report errors and so on.

So YMMV ...

-Gordon
--
Gordons projects: https://projects.drogon.net/
User avatar
Posts: 1543
Joined: Tue Feb 07, 2012 2:14 pm
Location: Devon, UK
by jbeale » Sat Jun 23, 2012 3:27 pm
secretagent wrote:There is another timer which can be made to run at up to 250 MHz and as far as I know is not used for anything else. It is described in chapter 14 of the datasheet.

Very interesting! I'd really like to use this timer to get the best possible resolution to time an external input pulse on a GPIO pin. The question is, can I make full use of that resolution? I don't expect the input pins have 250 MHz bandwidth, but my question is, let's say I have a set of input pulses, each pulse about 1 millisecond long, and one pulse per second. Each pulse has a very slightly different length. Can I use a GPIO interrupt, or any other means, along with the 250MHz R-Pi timer to measure that input pulse width with a resolution of 1/250E6 = 4 nanoseconds, or is there some discretization (internal synch across clock domains, etc.) that makes the effective resolution much lower? (Just wondering about resolution- not accuracy.)
User avatar
Posts: 2094
Joined: Tue Nov 22, 2011 11:51 pm
by rgh » Sat Jun 23, 2012 3:49 pm
jbeale wrote:Very interesting! I'd really like to use this timer to get the best possible resolution to time an external input pulse on a GPIO pin. The question is, can I make full use of that resolution? I don't expect the input pins have 250 MHz bandwidth, but my question is, let's say I have a set of input pulses, each pulse about 1 millisecond long, and one pulse per second. Each pulse has a very slightly different length. Can I use a GPIO interrupt, or any other means, along with the 250MHz R-Pi timer to measure that input pulse width with a resolution of 1/250E6 = 4 nanoseconds, or is there some discretization (internal synch across clock domains, etc.) that makes the effective resolution much lower? (Just wondering about resolution- not accuracy.)

It takes about 60 or 70ns to read the GPIO pins register, and a similar time to read the counter, so you are really not going to get 4ns resolution. I did a test that looped reading the GPIO and then reading the counter and wroting the counter to sequential memory locations. While the loop usually ran in 164ns, it did occasionally take up to 400ns - presumably because of bus contention, etc. If you want to do your timing via interrupts, then even if you investigate the preempt-rt kernel patches you'll probably still have 10's of microseconds jitter due to variable interrupt latencies. If you want to do these timings over a long period then disabling interrupts like Panalyzer does is probably not practical anyway as it effectively locks up the Pi for the duration.
Posts: 220
Joined: Fri Nov 25, 2011 3:53 pm
by jbeale » Sat Jun 23, 2012 3:58 pm
rgh wrote:It takes about 60 or 70ns to read the GPIO pins register, and a similar time to read the counter, so you are really not going to get 4ns resolution. I did a test that looped reading the GPIO and then reading the counter and wroting the counter to sequential memory locations. While the loop usually ran in 164ns, it did occasionally take up to 400ns - presumably because of bus contention, etc.


Ok, but the read latency doesn't prevent 4 ns resolution, if it is constant. You can just remove a fixed offset afterwards. I'm willing to turn off all other interrupts and processes for a few msec. The question is the "bus contention, etc." If you have already disabled everything and still get the 164-400 ns variability, then I guess that's the answer. Maybe I'm hoping Dom can suggest some other hidden control to turn something else off :-)
User avatar
Posts: 2094
Joined: Tue Nov 22, 2011 11:51 pm
by rgh » Sat Jun 23, 2012 5:42 pm
Actually my 164-400ns included writing sequentially up memory; you might just want to poll the gpio and then when you saw a change, read the counter. There could be less variation then. However, if you can only read every 70ns or so, and those reads are not synchronized to when the gpio signal happens to change state, you can't know to within better than 70ns when it actually changed. So no 4ns resolution.
Posts: 220
Joined: Fri Nov 25, 2011 3:53 pm
by jbeale » Sat Jun 23, 2012 6:56 pm
would not expect polled mode to have good resolution. but GPIO can trigger interrupt? or is there some granularity there?
User avatar
Posts: 2094
Joined: Tue Nov 22, 2011 11:51 pm
by rgh » Sun Jun 24, 2012 10:53 am
jbeale wrote:would not expect polled mode to have good resolution. but GPIO can trigger interrupt? or is there some granularity there?

I suppose that if you disabled all interrupts except your GPIO interrupt, then ensured that your interrupt handler was in cache already, then you might get fairly consistent results, but I'd still be quite impressed if you consistently got better than within 100ns of reality. I don't actually know anything about how the GPIO interrupt stuff works - there might even be some h/w state machine polling the pins for changes every few micoseconds which would make the idea of using interrupts a non-starter.
Posts: 220
Joined: Fri Nov 25, 2011 3:53 pm
by jbeale » Mon Jun 25, 2012 2:47 pm
rgh wrote:I don't actually know anything about how the GPIO interrupt stuff works - there might even be some h/w state machine polling the pins for changes every few micoseconds which would make the idea of using interrupts a non-starter.


When my RPi arrives I'll try to see if this will work. First I need to find out how to individually control all interrupts, as you mention- is there a complete list somewhere?
User avatar
Posts: 2094
Joined: Tue Nov 22, 2011 11:51 pm
by rgh » Mon Jun 25, 2012 5:50 pm
The only docs I'm aware of are the Broadcom peripherals datasheet on the wiki, and the source code :)
Posts: 220
Joined: Fri Nov 25, 2011 3:53 pm
by reggie » Mon Jun 25, 2012 7:27 pm
Fantastic work on the panalyzer code, I'm wondering if there is a pressing need to actually use linux as the interface to the gpio pins here, or whether you'd get more speed out of it accessing everything before linux gets it's grubby fingers all over the place. Something like uboot does (believe someone is working on uboot for the pi and had it downloading the kernel across the net, so there might be some useful code there to start with!). It's not like you need a whole operating system, just need a means to get at the data once you've collected it, whether that's out across the lan, usb, serial, spi to an lcd, file on an sd card etc.

Either way, keep up the good work, it'll be interesting to see how far you get with regard to sample rates :)
Posts: 151
Joined: Fri Aug 26, 2011 11:51 am
by Gert van Loo » Mon Jul 02, 2012 10:20 am
I have asked our DMA developer how he would make the DMA read data from the GPIO pins on a regular basis.
Basically we can't do it absolute regularly like a real analyzer as there is always the uncertainty that the bus may be busy.
But he came up with a scheme which should give you excellent results.
To exactly understand how it works you have to know how the AXI bus interface works.
I can't explain all the details but you can get the AXI specification from the ARM website. As far as I know it is free.

The principle is that you do two DMA transfers. A real DMA read from the GIO pins
followed by a dummy DMA transfer which takes a certain amount of time. Together these
two give you a reasonably regular interval. The dummy transfer should be as slow as possible as it is just a time-out.
Furthermore you do not want to put extra load on the internal DRAM bus as that will impact your performance.
Thus the transfer should be on the peripheral bus only.

For the dummy transfer we use some of the exotic features of the BMC2835:
1/ We tell the DMA controller to do the write only (We suppress the read of the data).
2/ We tell the DMA controller to write to the peripheral bus to a special non-existing device.
(A hardware equivalent of /dev/null) at address 0x7E20B000.
3/ The DMA controller should wait for the peripheral to acknowledge that it has received the data (that gives a little bit of extra delay)
4/ The DMA controller should wait between cycle, (the max. value of that is 32 clocks.)

This give rise to the following DMA transfer parameters.
src_addr = 0x0000_1000 // some unused space in ram
dest_addr = 0x7e20b000 // the testbus interface which is a dump peripheral that goes nowhere
txfr_len = 0x????? your delay value

and the following fields in the TI value
nowide_bursts = 0
waits = 0x1f // wait 32 clocks between each transfer
permap = 0 // look at the always on dreq
burst_length = 0
src_ignore = 1 // don’t bother to go and read the data that you are going to write
src_dreq = 0
src_width = 1
src_inc = 1
dest_ignore = 0
dest_dreq = 0
dest_width = 0
dest_inc = 0
wait_resp = 1 // wait for the bresp to come back before doing the next write
tdmode = 0
inten = 0

So this arrangement should do your real transfer.
It should then waste some time writing zeros to the testbus peripheral. It won't bother to go and read the data for this (src_ignore) so there should be no memory bus overhead
and no uncertainty caused by arbitrating for room on this bus.
It will write quite slowly to the testbus, as it has to wait for each bresponse to come back over the axi bus, and its adding a 32clock wait in as well
so the peripheral bus traffic should be quite small.

This will delay the DMA for a while, depending on how long you set the transfer length to. It should be reasonably predictable.
The down side is that the DMA will overwrite the data on each pass of the loop, as the first control block will be reloaded and write to the same destination address.
You can get round this by making a much longer chain of transfers, of read gpio to buf1, delay, read gpio to buf2 , delay etc etc but to a different buffer each time, and then do an interrupt at the end and go and clean out the destination data. You will have an extra delay between transfers when the DMA controller read the next configuration block from memory.

Good luck!

The Broadcom Weird Solutions Team.
User avatar
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2095
Joined: Tue Aug 02, 2011 7:27 am
by jbeale » Mon Jul 02, 2012 4:41 pm
Wow... I don't know enough to turn that description into code, but I'm willing to try it out if someone else can code it! Still wondering about that "bus may be busy" condition. Is there a list of everything that might be using the bus, and can we turn off the non-critical ones, and for the rest, accurately predict when the bus will be in use?
User avatar
Posts: 2094
Joined: Tue Nov 22, 2011 11:51 pm
by Gert van Loo » Mon Jul 02, 2012 6:40 pm
jbeale wrote:Wow... I don't know enough to turn that description into code, but I'm willing to try it out if someone else can code it! Still wondering about that "bus may be busy" condition. Is there a list of everything that might be using the bus, and can we turn off the non-critical ones, and for the rest, accurately predict when the bus will be in use?


EVERYTHING uses the main bus**. You con not really turn things off. But if your processors are idling they are not using the bus as they will get everything they need from/to their local caches. Unfortunately they are seldom completely idling. The DMA process described below uses the main memory bus on two occasions:
1/ After it has read the GPIO status the data has to be written to the SDRAM memory.
2/ When it fetches the next block descriptor
That is why I said, "We can't do it absolute regularly".

**The bus connects things on the chip like a railway. At any one time you can have only one train on a railway segment. If not you get a collision. The hardware prevents collisions which means that if a segment is busy (has a train) you have to wait for that to pass before you can get on the bus. Every time a devices reads or writes data parts of the bus are busy. Fortunately the peripherals have their own bus so most of the DMA read/writes do not have to wait for the main bus. (They still have to wait for the peripheral bus but that is used much, much less so the probability that it has to wait is much lower).
User avatar
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 2095
Joined: Tue Aug 02, 2011 7:27 am
by rgh » Mon Jul 02, 2012 6:48 pm
Gert, thanks for that! I'll have to experiment and see what range of sample rates this can generate at the higher end. I'm curious to see what impact the overhead of the DMA controller having to read two control blocks for each gpio pin read is. If only there were more days in a weekend :)
Posts: 220
Joined: Fri Nov 25, 2011 3:53 pm
by godFather89 » Thu Jul 05, 2012 7:51 am
The best performance would be achievable by running on the bare metal. Of course this would be much more complicated when it comes to saving the data (USB driver/stack or SD driver + file system).
I guess the simplest and fastest way to do this would be a simple asm program (the bare metal kernel) that loops and reads the GPIOs (or uses DMA or timers/interrupts) and stores the values in RAM. When it's done (memory full or time limit), it stores the data on an SPI memory chip (or more of them) or an SD card via the SPI bus.
There is no way you can get it faster than this.
Posts: 150
Joined: Fri May 18, 2012 9:40 am
Location: Timisoara, RO
by reggie » Thu Jul 05, 2012 1:44 pm
I also wonder whether this can be extended to a single/dual trace scope of some sort? I've seen circuits on diy boards that use an ADC connected to 8/16 digital pins, which are then read and converted. Probably a bit of a simplification but from memory it's not far off that. Not sure what speeds it would get to?
Posts: 151
Joined: Fri Aug 26, 2011 11:51 am
by secretagent » Sat Jul 07, 2012 6:37 am
godFather89 wrote:The best performance would be achievable by running on the bare metal. Of course this would be much more complicated when it comes to saving the data (USB driver/stack or SD driver + file system).
I guess the simplest and fastest way to do this would be a simple asm program (the bare metal kernel) that loops and reads the GPIOs (or uses DMA or timers/interrupts) and stores the values in RAM. When it's done (memory full or time limit), it stores the data on an SPI memory chip (or more of them) or an SD card via the SPI bus.
There is no way you can get it faster than this.


What exactly are you going to gain from running on bare metal versus in a kernel module with all interrupts disabled for the duration of the sampling?
Posts: 36
Joined: Wed Mar 07, 2012 10:09 am
by godFather89 » Sat Jul 07, 2012 8:18 am
Longer sampling time. If you keep the kernel with the interrupts off for a big period of time, all sort of problems will appear.
Posts: 150
Joined: Fri May 18, 2012 9:40 am
Location: Timisoara, RO
by rgh » Sun Jul 08, 2012 6:10 pm
godFather89 wrote:Longer sampling time. If you keep the kernel with the interrupts off for a big period of time, all sort of problems will appear.

What sort of problems do you have in mind? If the time of day goes adrift, well, you wouldn't have that anyway with a bare metal solution. If you get timeouts and retries on SD accesses, or if you get timeouts and dropped connections for ssh, or whatever, so what? I've not disabled interrupts for more than 2 seconds so far, but if doing it for longer causes a serious problem, I think that rates as a kernel bug :) I expect the kernel has got code that makes assumptions that jiffies doesn't wrap between interrupts, or whatever, but I don't think disabling for minutes should cause problems (time for me to put my money where my mouth is, perhaps). If you are sampling some low frequency thing for days, then you probably can just do if from user space without disabling interrupts anyway.
Posts: 220
Joined: Fri Nov 25, 2011 3:53 pm
by secretagent » Sun Jul 08, 2012 11:32 pm
I've had the kernel stopped at a breakpoint with interrupts disabled for at least 5-10 minutes at a time with no apparent problems. I agree with rgh that if something fails permanently just because you disabled interrupts for a few minutes then that would be a sign of a bug. And for any decent sampling rate your sampling time would be limited by the amount of RAM you have to store samples.
Posts: 36
Joined: Wed Mar 07, 2012 10:09 am
by MadCow42 » Mon Jul 09, 2012 4:22 am
secretagent wrote:I've had the kernel stopped at a breakpoint with interrupts disabled for at least 5-10 minutes at a time with no apparent problems. I agree with rgh that if something fails permanently just because you disabled interrupts for a few minutes then that would be a sign of a bug. And for any decent sampling rate your sampling time would be limited by the amount of RAM you have to store samples.


Interesting thread guys - I'm having similar issues with tiny hiccups induced while driving stepper motors, and wanted to try disabling interrupts to see if that helped. The problem is that I'm doing it all from Python (which may be part of the issue, I know). Is there an easy way to disable the interrupts from Python that you might suggest?

Thanks if you can help!
Kevin.
Posts: 86
Joined: Sun Jul 01, 2012 12:48 am