notro
Posts: 693
Joined: Tue Oct 16, 2012 6:21 pm
Location: Drammen, Norway

Re: SPI driver latency and a possible solution

Thu May 16, 2013 4:27 pm

jwasch wrote:Works as expected except that I expected to be able to have each item have a byte length of up to 4096 bytes, and instead, fails when SUM of individual transfers exceeds 4096. spii_test.c message is "can't send spi message: Message too long". Anybody find a way around this? Goal is to transfer as much as possible in single ioctl() call. Thanks.

Code: Select all

modprobe spidev bufsiz=65536
Details: http://www.raspberrypi.org/phpBB3/viewt ... 82#p309582

oak
Posts: 1
Joined: Fri May 31, 2013 3:13 pm

Re: SPI driver latency and a possible solution

Fri May 31, 2013 5:54 pm

Hi everyone and thank you for the enormous work you've put in to fixing this issue!!

I have been trying and failing at patching and/or compiling the kernel on my RPi for quite a while now. I'm still new to the RPi and Linux in general.
I was wondering, if one of you would be willing to share an image of their RPi with the working SPI patch for beginners like myself. I am trying to setup the RPi as an open platform for a research project but have to show that it is reading SPI at high frequencies (to collect EMG data).
So it is very frustrating to see that it works in theory but not being able to use it...
Any help would be greatly appreciated.

Thank you!
oak

WasimKh
Posts: 2
Joined: Wed Jun 05, 2013 3:23 pm

Re: SPI driver latency and a possible solution

Wed Jun 05, 2013 3:38 pm

Hi Everyone,

Thank you for all your enormous work! I have been trying to get the SPI to work but I am still getting the latency issues. Rather than using the patch, I simply copied the two files (spi_bcm2708.c and bcm2708.c) from Martin's repository, re-compiled the kernel and installed the modules. I must say that I am a beginner and do not have experience with the Kernel, however, using 'dmesg' and modinfo seemed to show that I compiled everything correctly.

In essence, I am trying to send one byte at a time and so I am using the interrupt mode as this should be sufficient (as opposed to DMA). However, there is a constant 60us delay between each transfer and as speed plays a large role in my project, this is detrimental. I am looking for transfers of at least 1MHz. I have a suspicion that I am not using the driver correctly though.

So if anyone has an example of how to use the driver or any idea of what could be wrong, I would really appreciate your help. Currently, I am using ioctl with macros defined in the spidev.h to write using the SPI interface but I am not sure whether that's correct.

Thank you for all your help in advance.

Wasim

WasimKh
Posts: 2
Joined: Wed Jun 05, 2013 3:23 pm

Re: SPI driver latency and a possible solution

Wed Jun 05, 2013 3:47 pm

oak wrote: I have been trying and failing at patching and/or compiling the kernel on my RPi for quite a while now. I'm still new to the RPi and Linux in general.
I was wondering, if one of you would be willing to share an image of their RPi with the working SPI patch for beginners like myself. I am trying to setup the RPi as an open platform for a research project but have to show that it is reading SPI at high frequencies (to collect EMG data).
oak
I don't know if you have managed to compile your kernel yet, however, I recommend using this http://bchavez.bitarmory.com/archive/20 ... ry-pi.aspx. I have very limited knowledge in programming but I have managed to follow it just fine so hopefully you should be OK. The only thing that you have to do extra is patching. Here is what I suggest. In terminal:

1) use wget https://raw.github.com/msperl/linux/73a ... -bcm2708.c
2) use wget https://raw.github.com/msperl/linux/73a ... /bcm2708.c

This will get you the two files that you need. Copy them into the correct folder: The first file goes into drivers/SPI and the second file goes into arch/arm/mach-bcm2708 (In the linux folder). Once you have copied these you can then start compiling i.e. step 6 in the given tutorial.

Hope this helps.

Jezmo
Posts: 34
Joined: Tue Jun 04, 2013 1:08 pm

Re: SPI driver latency and a possible solution

Thu Jun 06, 2013 7:05 pm

A very simple observation, if you want real time performance from your operating system, why the hell are you using linux?
If you used a realtime OS you would get realtime performance.

SnickerDudle
Posts: 1
Joined: Wed Jul 17, 2013 6:53 pm

Re: SPI driver latency and a possible solution

Thu Jul 18, 2013 8:39 pm

Hey guys,

I've been engaged in a glorious and rather fruitless battle with my Raspberry Pi for a couple of weeks now, and every time I try to compile, something happens. It is either missing files, or has huge chunks of code that are somehow wrong, or the stupid "fiq_fix_enable" problem - you get the point.

Now, I've tried compiling both the 3.2.27 and 3.6.y kernels, no luck with either. And since all I need to move my project forward is a simple 3MB kernel.img file, I was wondering if any of you awesome folks could drop off a link to a properly compiled kernel.img with the SPI issue fixed and end my (and I'm sure many other people's) epic, glorious battle with the RPi.

Pretty please?

Thanks

noafterglow
Posts: 5
Joined: Tue Jul 23, 2013 3:58 am

Re: SPI driver latency and a possible solution

Tue Jul 23, 2013 4:41 am

I have also been having problems getting the proper module built and running.

I started with Notro's quite handy instructions, which seem to indicate compiling on the pi itself. which takes FOREVER. I assumed that the instructions still held, but it turns out the new adafruit SD cards come with 3.6.11+ not 3.2.27.

WTF does + mean? Does it mean "we just put in whatever we wanted to" ? or this there some official release?

OK, so for the next try i loaded (cleanly) ubuntu 12.04 onto a vm, and followed the basic directions from http://bchavez.bitarmory.com/archive/20 ... ry-pi.aspx

Of course I had to substitute 3.6.y for 3.2.27. (WTF is y) is there a defined stopping point?

before compiling the kernel I took the two files pointed to by WasimKh » Wed Jun 05, 2013 3:38 pm
and put them over top of the two files which were in the kernel source tree.

I compiled and promptly got the fiq_fix_enable problem.

I modified the appropriate file to create fiq_fix_enable. The other file was already updated... SO i guess that fix went into the 3.6.11 kernel, but not the two spi driver files referenced by WasimKh... did it go into the + version? I have NO IDEA.

I was then able to compile completely, and got the resulting spi-bcm2708.ko file.

I was able to copy it over to the pi, and I have it sitting there. No matter how hard I tried I could not get the pi to load it.

the modinfo dumps on the two files were identical, EXCEPT:
mine was 3.6.11 vs the original 3.6.11+
mine had a line about parameters, ie. processmode, (which I assume is pretty good proof that the module compiled successfully, since it seemed to offer Interrupts and DMA, which is kind of the point of all of this...)

Finally I got annoyed and force unloaded the module.
insmod would NOT allow the new one to load... "No such device"

I copied the file over the old one and rebooted.

It still won't load. "No such device"

This is getting kind of comical. I'm not sure why this is so annoyingly bad, or more appropriately, why this driver didn't make it into 3.6.11(+) in the first place, but I would like to suggest that one of you guys see if we can come up with an actually coherent file set which actually works together.

Barring that, I'm happy to take suggestions. I'm probably at the point where I guess I should just copy the kernel and all of the modules to my SD card and push the button and see what happens.

Thoughts?

vchristo
Posts: 7
Joined: Wed Aug 28, 2013 12:23 am

Re: SPI driver latency and a possible solution

Mon Sep 23, 2013 1:55 am

Dear friends, I need your help about spi-bcm2708 driver, I'm trying to read the mcp3008, synchronized with my hardware developed to download 4Mpixel from a CCD. I was trying chibios-rpi, it's works fine but are so hard to develop another functionalities likes disk i/o, and display interface.
At now I'm trying to read it using spi-bcm2708 driver, my question are how can I modify the driver to set GPIO18, to starts my hardware upload, and check GPIO4 (if is set) to start conversion.
I'm using the arduino to generate the clock phases to CCD, the GPIO18 starts the arduino to begin clock phases to CCD.
The GPIO4 are ready to convert signal.
can you give to my some help, I never modify any module, I'm a beginner in this new world.
If anyone are interested in this hardware I can share, the schematics and pcb`s.
thank you,
Vitor.

msperl
Posts: 325
Joined: Thu Sep 20, 2012 3:40 pm

Re: SPI driver latency and a possible solution

Mon Sep 30, 2013 4:33 pm

Coming out of summer hibernation - I have created a port of the driver for the latest 3.11.y branch from github.

See: https://github.com/msperl/linux/commit/ ... 4ab25edfc1

Note that it does not (yet) include the latest changes to the original driver (9 bit and clock speed).

But my basic test - mcp2515 CAN bus controller - is working fine (again)...

Ciao, Martin

notro
Posts: 693
Joined: Tue Oct 16, 2012 6:21 pm
Location: Drammen, Norway

Re: SPI driver latency and a possible solution

Tue Oct 01, 2013 11:56 am

Thanks for your driver Martin. I'm using it with SPI driven LCD displays: https://github.com/notro/fbtft/wiki
I have made two important changes to the driver:
* DMA transfer longer than 36 bytes bug fixed
* increase DMA buffer limit to 64kB

I was the one who added 9-bit support to the vanilla driver, but I have since changed my mind. The problem is that certain values trigger a read, which makes it problematic in a general driver: https://github.com/notro/spi-bcm2708/wiki#lossi-mode
My framebuffer drivers does 9-bit emulation instead.

My intention is to add CAN support in my next prebuilt kernel release. I have seen talk about a patched mcp2515 driver. Do you use a patched version, or the vanilla driver?
The main firmware will most likely move to 3.10 shortly, so my next release will be based on that.

msperl
Posts: 325
Joined: Thu Sep 20, 2012 3:40 pm

Re: SPI driver latency and a possible solution

Tue Oct 01, 2013 12:18 pm

I personally still prefer the mcp2515.c driver that is out of tree...

There are some circumstances that the mcp251x.c in the default kernel is having a hickup and stops fetching received packets.
If I remember correctly the mcp251x.c does only trigger on interrupts for RX and the way it works it does not recognize that the IRQ line is still low (if a new message comes in in the meantime) and so no more RX interrupts are received (this is the case for CAN Buss speeds >100kbit - below it might not happen or only for specific "cases") .

I believe in 3.2.X we had the IRQ configured as triggering on low and not on edge. But something changed between Kernel 3.2 and 3.6 that made the IRQ_LOW triggering "complain" and thus getting ignored by the kernel...

So if you want to figure out how we set up the IRQ on low level correctly, then you can use mcp251x.c otherwise I would add the out of tree mcp2515.c driver to your build. But then there is a different problem anyway: You would need to move to deviceTree so that you can assign different "SPI" devices to the system, as otherwise we are still stuck with the fact that you have to recompile bcm2708.c with your "personal" SPI setup...

See: http://lnxpps.de/rpie/ for CAN details and where it all started... (there you also find the "link" to the mcp2515.c driver)

I will incorporate this one line patch of yours into my 3.6.11 tree...

Martin

msperl
Posts: 325
Joined: Thu Sep 20, 2012 3:40 pm

Re: SPI driver latency and a possible solution

Tue Oct 01, 2013 12:58 pm

Actually I see that you have done a lot of cleanup of my code in your branch in https://github.com/notro/spi-bcm2708/...

Nice work - I like the idea of moving it out of the tree for now - except for when we want to "upstream" the driver...

A few comments:

Are you sure that >4k pages DMA is REALLY working in all cases?

I ask, because there is NO guarantee that 2 adjacent pages in ARM memory space are actually 2 adjacent pages in the VideoCore memory - which is the DMA address range!

I also thought to change this in a future revision walking the pages and checking each one of those and scheduling extra DMA transfers to handle the situation correctly.

The other part I came to realize is that there IS an API that allows you to map a userspace-address to kernel address space.
That would allow you to make spidev work with DMA mode directly to userspace (in the end) without any modifications.

Also I have come to realize that any transfer < 32 bytes is equally well handled with interrupt mode as it is with DMA because of the FIFO nature (besides a bit more of explicit copying - versus more cycles needed for DMA setup), so the question is: is it sensible to use DMA in all cases.

My plan - at some point - was actually to make the (DMA) driver even more aware of the SPI pipeline and schedule more than one SPI transaction at a time and only trigger an IRQ on requested callbacks.

Right now if there is a sleep requested (spi_transfer.delay_usecs) - we literally have to make use of udelay, which means rescheduling the driver introducing additional scheduling latencies.
An idea of mine was to schedule some dummy SPI transfers with no CS asserted to pass the time in DMA mode instead - or schedule some DMA transfers of a different type (say transfer from NULL to NULL X bytes) instead to pass the required time.
This would make the "sampling" at "exact" times work quite will with minimal jitter.

One other pain that - I believe - we all feel (until we reach a kernel with device-tree-support) is the need to modify the kernel (arch/arm/mach-bcm2708/bcm2708.c) to set up the SPI config...
One idea here: create a separate module that "implements" just the mapping for SPI (say 2xSPIDEV,mcp2515+SPIDEV,...) and then load that one for the setup of your choice... - that module would only contain: the spi_register_board_info() initialization section...
This is not tested or anything, but if you are creating a distibution kernel, then you might as well help people with such a module, as recompiling the kernel for just that defeats the purpose of a precompiled kernel...

Martin

msperl
Posts: 325
Joined: Thu Sep 20, 2012 3:40 pm

Re: SPI driver latency and a possible solution

Tue Oct 01, 2013 12:59 pm

p.s: if you move to DMA and it works for all cases, then remove the non-DMA stuff... I just kept the old stuff for learning purposes and as a progression/fallback)

notro
Posts: 693
Joined: Tue Oct 16, 2012 6:21 pm
Location: Drammen, Norway

Re: SPI driver latency and a possible solution

Tue Oct 01, 2013 4:33 pm

msperl wrote: Actually I see that you have done a lot of cleanup of my code in your branch in https://github.com/notro/spi-bcm2708/...

Nice work - I like the idea of moving it out of the tree for now - except for when we want to "upstream" the driver...
My intention was to have the driver in my kernel for some time, and then try and get it upstream later.
Then I realized that in the torvalds/linux tree, the master driver is called spi-bcm2835.
So I see no point in working further on spi-bcm2708 with the purpose of upstreaming, only bug fixing and DMA features.
msperl wrote: Are you sure that >4k pages DMA is REALLY working in all cases?

I ask, because there is NO guarantee that 2 adjacent pages in ARM memory space are actually 2 adjacent pages in the VideoCore memory - which is the DMA address range!

I also thought to change this in a future revision walking the pages and checking each one of those and scheduling extra DMA transfers to handle the situation correctly.
No, I'm not sure :-) I wasn't aware of this limitation.
I have only done some very limited testing with 64k buffers. DMA support is experimental in my framebuffer drivers, and the default transfer buffer is 4096 bytes, so I'm not sure >4k gets much testing either.
msperl wrote: Also I have come to realize that any transfer < 32 bytes is equally well handled with interrupt mode as it is with DMA because of the FIFO nature (besides a bit more of explicit copying - versus more cycles needed for DMA setup), so the question is: is it sensible to use DMA in all cases.
My testing with your driver shows the FIFO to be around ~110 bytes: https://github.com/notro/spi-bcm2708/wiki#fifo-size. In my framebuffer drivers, I only use DMA for display memory, and irq for control transfers.
If I remember correctly, the BeagleBone Black SPI driver skips DMA on small transfers.
msperl wrote: One other pain that - I believe - we all feel (until we reach a kernel with device-tree-support) is the need to modify the kernel (arch/arm/mach-bcm2708/bcm2708.c) to set up the SPI config...
One idea here: create a separate module that "implements" just the mapping for SPI (say 2xSPIDEV,mcp2515+SPIDEV,...) and then load that one for the setup of your choice... - that module would only contain: the spi_register_board_info() initialization section...
This is not tested or anything, but if you are creating a distibution kernel, then you might as well help people with such a module, as recompiling the kernel for just that defeats the purpose of a precompiled kernel...
Your right, I completly forgot about that. In my drivers I have solved this by making a module as you suggests: https://github.com/notro/fbtft/blob/mas ... t_device.c
I'm currently working on getting Device Tree support working in my kernel to solve this type of problem: https://github.com/raspberrypi/linux/issues/381
I will have to wait with CAN support until DT is working then.

msperl
Posts: 325
Joined: Thu Sep 20, 2012 3:40 pm

Re: SPI driver latency and a possible solution

Tue Oct 01, 2013 4:37 pm

Actually I have started to work on a "stop-gap" module where you can define the spi devices when loading this module...
For how I intend to implement it - see the CAN thread...

Martin

msperl
Posts: 325
Joined: Thu Sep 20, 2012 3:40 pm

Re: SPI driver latency and a possible solution

Fri Oct 04, 2013 10:52 am

Here the version that can do already a lot of configurations of SPI devices via arguments to modprobe... (assuming non-too-complex structures to platform_data - pointers inside platform_data are not supported (yet)...)
https://github.com/msperl/spi-config

Here the relevant thread for this:
http://www.raspberrypi.org/phpBB3/viewt ... 44&t=57157

Ciao,
Martin

msperl
Posts: 325
Joined: Thu Sep 20, 2012 3:40 pm

Re: SPI driver latency and a possible solution

Thu Oct 10, 2013 3:23 pm

Unfortunately I found out that the DMA code has a bug when doing multiple SPI transfers in a single transaction.

So beware of this deficiency!

Anyway: I am in progress of rewriting the code to ONLY use DMA - and then in an efficient way!
So I will only update you on that code... I will have to restart from scratch so that we can get it upstream into the regular linux kernel as well, as there is some attribution/sign-off missing on the original SPI driver code on which mine is based. So all the changes I have done can not go upstream because of the "copyright" gaps before my time.

Ciao,
Martin

msperl
Posts: 325
Joined: Thu Sep 20, 2012 3:40 pm

Re: SPI driver latency and a possible solution

Thu Oct 17, 2013 7:14 pm

Hi!

I have started moving to a fully DMA version of the SPI driver (which is far from finished), but I found the following:
  • the SPI driver that is in the upstream kernel (spi-bcm2835.c) has now been moved to a fully interrupt driven design. So it is already in much better shape than the spi-bcm2708.c driver and should give better performance over all. Unfortunately it has been created with the focus of working with device-tree, so some modifications are needed to make it work on a Raspbian based kernel. I have to admit, that the coding-style is MUCH clearer than my driver instance, so I would recommend all of you to switching to that one...
  • there seems to be a bug in the drivers (all versions - old an new), that is related to SPI devices which have CS with different polarity (so active on high, not active on low). If it is a single device it does not seem a problem, but as soon as one tries to access 2 devices on the same bus, the SPI driver will set the CS-polarity of the currently "non-active" device to the default state (which is CS active on low) when a transfer is happening. This means that actually 2 devices will "feel" addressed at the same time , which will then mess up the SPI transfer. Obviously this is far from perfect, but nobody seems to have seen the issue (yet), so the devices that show this "abnormal" CS behaviour are rare - unless someone is having the SPI bus working via optocoupler, where you often would be inverting the signal to avoid the optocoupler using current unnecessarily when not active... Please note that this is based on logical analysis of the driver not based on "empirical" evidence...
I will try to create a set of 2 "patches", that will:
  • make the spi-bcm2835.c work also on the Raspbian kernel
  • solves this "CS" issue - there is already code there


Ciao,
Martin

msperl
Posts: 325
Joined: Thu Sep 20, 2012 3:40 pm

Re: SPI driver latency and a possible solution

Wed Oct 23, 2013 8:17 pm

Latest updates - the CS issue has been fixed in the bcm2835 driver - see also: http://www.raspberrypi.org/phpBB3/viewt ... 58#p439658

Also there is an initial working version of a DMA only driver, that schedules a FULL spi_message in one go to get executed via DMA.

It already includes:
  • transfer->cs_change support
  • transfer->delay_usec support via DMA. it is at least a good empirical approximation by wasting DMA cycles in a loop - this depends a bit on how many cycles are available on the AXI bus (=internal data bus)
  • transfer->speed_hz speed change during the transfer should work as well (not tested - except for the initial setup)
  • single Interrupt gets triggered per spi_message (if message->complete is set)
so all the basic transfers are done WITHOUT further CPU interaction.

The interface used for now is still based on the transfer_one_message, where we can only handle one transfer at a time and we have processing in between.

The idea going forward is to switch back to the "direct" scheduling interface for SPI-drivers using transfer.
But for that to work I will need to change:
  • the interface itself - this would mean that during the call spi_async the code fore scheduling would get called IMMEDIATELY and it would also schedule/chain the Message to the existing DMA chain. This means no more adding to a work-queue, waking up the worker thread and then starting the processing, which should reduce latencies immediately
  • need to check if the scheduling code (and especially the memory allocation paths for the DMA-control-structures) are interrupt-safe and making them safe if needed
  • implement locking - the transfer_one_message interface does not require any explicit locking making it easier to programm... This mostly applies to interrupt-handling and DMA structure dumping code (which is mostly used for debugging purposes)
  • unfortunately there is still the need for a worker thread for callback, as there is no requirement for Atomic mode for these callbacks - this means callbacks can NOT get used directly from the interrupt handler for the DMA - there might be an option to improve on the overall Linux SPI interface, which could allow for immediate wakup of the calling thread instead of having to do 2 context switches (and the corresponding delays) - alternatively maybe even a atomic callback handler - or doing this via a flag...
  • there is also a need to clean up memory after a transfer - so either a separate thread for that or merged with the above...
So there is some progress, and some "interrested" parties may want to give it a try already with the device of their choice....

You can find it at: https://github.com/msperl/spi-bcm2835/tree/dma_cleanup

Please give it a shot to see if the basic functionality works (before I move to fully pipelining it (But before doing that I need a driver that makes HEAVY use of the asyncronous spi message feature, so that I can see the advantage....

Thanks, Martin

msperl
Posts: 325
Joined: Thu Sep 20, 2012 3:40 pm

Re: SPI driver latency and a possible solution

Wed Oct 23, 2013 8:41 pm

p.s: theoretically it should be possible to run the following SPI_ASYNC chain with the future pipelined version of the DMA driver:
  • SPI Message
    • CS=0 - 24 Bit ADC connected
    • Read 3 Bytes
    • speed 10MHz (assuming the RPI could do that exactly)
    • this results in a transfer time of: 2.4us
  • SPI Message
    • CS=2 or 3 (unavailable CS on the RPI)
    • delay_usec 7us
  • continue this chain for however long you want - only set complete every so often, maybe every 1024 samples to keep interrupt rates low...
This could give you an effective sample rate for the ADC of 100kHz (ok - not 100% exact 100khz, but in the order of) with only a tiny bit of jitter and without any linux-scheduling jitter!

Could be a nice feature for some use-cases...

With that kind of timing you could even run 2 24-bit ADCs in (almost) parallel!!!

Obviously you would need some other infrastructure as well (say SPIDEV with DMA support or a dedicated driver) to make this work directly from user-land...

Martin

msperl
Posts: 325
Joined: Thu Sep 20, 2012 3:40 pm

Re: SPI driver latency and a possible solution

Wed Oct 23, 2013 8:49 pm

One other thing to mention to the interested minds:
The SOC document is wrong on the account of how to "best" do SPI-DMA. Especially ADCS is required to make it work, when using the approach they described (otherwise the SPI clock does not start and no transfer occurs).
The way around this "bug" was to actually program the SPI registers directly from DMA, setting up transfer length, enabling SPI-DMA and such...
Only with this approach it became possible to make all the described features (delay, change_cs,...) work only in DMA without software interaction.

I hope that this driver is now much more readable than my first DMA-driver incarnation from almost a year ago (which did not really work)...

User avatar
mikronauts
Posts: 2637
Joined: Sat Jan 05, 2013 7:28 pm
Contact: Website

Re: SPI driver latency and a possible solution

Fri Oct 25, 2013 12:21 am

This sounds great!

I must admit that I am new to low level programming on the Pi, so I will likely ask a lot of elementary questions in the near future.

Initially I will be using MCP3008/3208 and MCP23S17's, however other chips are on the horizon.

I will be keeping a careful eye on this thread - thanks for the great work.
msperl wrote:p.s: theoretically it should be possible to run the following SPI_ASYNC chain with the future pipelined version of the DMA driver:
  • SPI Message
    • CS=0 - 24 Bit ADC connected
    • Read 3 Bytes
    • speed 10MHz (assuming the RPI could do that exactly)
    • this results in a transfer time of: 2.4us
  • SPI Message
    • CS=2 or 3 (unavailable CS on the RPI)
    • delay_usec 7us
  • continue this chain for however long you want - only set complete every so often, maybe every 1024 samples to keep interrupt rates low...
This could give you an effective sample rate for the ADC of 100kHz (ok - not 100% exact 100khz, but in the order of) with only a tiny bit of jitter and without any linux-scheduling jitter!

Could be a nice feature for some use-cases...

With that kind of timing you could even run 2 24-bit ADCs in (almost) parallel!!!

Obviously you would need some other infrastructure as well (say SPIDEV with DMA support or a dedicated driver) to make this work directly from user-land...

Martin
http://Mikronauts.com - home of EZasPi, RoboPi, Pi Rtc Dio and Pi Jumper @Mikronauts on Twitter
Advanced Robotics, I/O expansion and prototyping boards for the Raspberry Pi

msperl
Posts: 325
Joined: Thu Sep 20, 2012 3:40 pm

Re: SPI driver latency and a possible solution

Fri Oct 25, 2013 10:54 pm

Hi!

Just to let you know the current measurements for an SPI transfer, where we simple configure a mcp2515 CAN controller with 6 messages and a total of 18 transfers (including CS changes)

Here the stats for the "central" configuration message with 11 transfers and a total of 46 bytes:
  • the "stock" SPI driver takes 0.80 ms (plus some jitter of +-0.02ms) using 36 interrupts (including some other transfers)
  • the new DMA only version takes 0.39 ms using 6 interrupts (for all transfers)
Obviously there is some overhead preparing the DMA chain, but that computation should be "cheaper" than those 35 extra interrupts that need to get handled...

Here what the transfer looks like for the DMA version:
Screen Shot 2013-10-26 at 00.33.50.0.png
"DMA" driver
Screen Shot 2013-10-26 at 00.33.50.0.png (36.49 KiB) Viewed 5054 times
And here the "stock" version (beware different scales on the x-axis compared to the above!):
Screen Shot 2013-10-26 at 00.49.17.0.png
"stock" driver
Screen Shot 2013-10-26 at 00.49.17.0.png (36.79 KiB) Viewed 5054 times
Ciao, Martin

P.s: there was a bug until today, which has blocked the transfer of more than 36 bytes at a time - this seems to be solved now - see code on GIT-hub...

alpage
Posts: 7
Joined: Sun Feb 17, 2013 11:13 pm

Re: SPI driver latency and a possible solution

Sun Nov 03, 2013 10:35 pm

I'm using SPI for my project at http://virtualfloppy.blogspot.ca/ and I need as much speed as possible so I would like to use DMA but my requirements are a bit different. I need to send or receive up to 120K or more in a single burst. Currently I can get about 3 MHZ by running a thread in high priority and directly accessing the SPI FIFO. No doubt the thread is being interrupted but the FIFO is large enough to cover that. If I send data I don't care about what is being received and if I receive I send zeros. I also don't care about CS - in fact CS isn't even connected!

It seems you need lots of small fast, small messages instead which is a very different requirement. Can the current state of your code run large transfers or does it choke on > 64K? Or if I chain two 64K is there a gap between them?

msperl
Posts: 325
Joined: Thu Sep 20, 2012 3:40 pm

Re: SPI driver latency and a possible solution

Mon Nov 04, 2013 5:15 am

There is a newer version that should be able to handle 4GB of transfers in one go - probably even more...

Still you will need to "chunk" the spi_message into blocks (chained spi_transfers of say up to 4k), which still are sent with CS enabled as one big sequence. (the old DMA driver does not support this "chunking").

But that is something you will need to implement in your driver first.

People are using the original patched SPI driver to drive 320x200 displays at up to 25 Hz SPI-Bus rate (and that requires quite some bandwidth already!)

So you can transfer more data at a time - even the "stock" spi-bus-driver can, but at the cost of high interrupt rates impacting responsiveness of the system. The DMA allows you to offload more of the transfer without impacting the system in other ways.

Hope this answers your question,

Martin

Return to “Interfacing (DSI, CSI, I2C, etc.)”

Who is online

Users browsing this forum: No registered users and 12 guests