Page 1 of 2

High bandwidth interface to FPGA?

Posted: Fri Mar 16, 2018 6:03 am
by kmuffsfpv
Hi, I was hoping someone can point me in the right direction.. I haven't been able to find whether or not it would be possible to use the SMI, SDIO, or another interface on the CM3 to get decent data rates between the CM3 and an FPGA (ideally >300Mb/s).. Considering that the broadcom chip is fairly closed as far as documentation, it would be very nice to know if someone has tried this before.. :P

Thanks
- Kmuffs

Re: High bandwidth interface to FPGA?

Posted: Fri Mar 16, 2018 8:27 am
by Gavinmc42
Gert posted some SMI-IDE code.
viewtopic.php?f=45&t=197875

SMI 16bit wide is only about 19MHz for 300Mbs.
I think SMI will do 40+MHz? But Gert is an expert ;)

A search has shown 25MHz has been done.
Get a bit more speed going 18bit wide?

Use a Pi 3B+ Ethernet?
Use one core for the DMA?

Re: High bandwidth interface to FPGA?

Posted: Mon Mar 19, 2018 11:13 pm
by Imre
Hi Gavinmc,

I've read the topic you linked, and there was a discussion about to make the SMI documentation public. I checked the Github repo, but could not find it. Maybe I did something wrong.
Do you know it was/still is available somewhere?
I am also interesed in this peripheral. ;)

Thanks,
Imre

Re: High bandwidth interface to FPGA?

Posted: Tue Mar 20, 2018 1:14 pm
by Gavinmc42
The code is the documentation?
https://github.com/fenlogic/IDE_trial
These is a simple FPGA in the schematic too.

Been a while, not sure if there was more docs.
The actual SMI registers and how to set them up is the main trick.

I have not tried it, I don't use C these days.
But it looks to be word read and write and block read and write code.

Code: Select all

 // TODO: switch to higher clock freq for fast PIO modes 
smi_clock_freq = 125000000;
Not sure if that smi_clk is a real 125MHz data clock or what fast PIO mode is?

Recent GPIO bit banging on the new Pi3B+ show software toggling at 60MHz compared to 40MHz on the BCM835 chips.
But software toggling is not DMA based word data dumping.

Re: High bandwidth interface to FPGA?

Posted: Tue Mar 20, 2018 2:40 pm
by 6by9
There's also a kernel driver for it - https://github.com/raspberrypi/linux/bl ... _smi_dev.c
I haven't checked on the apis for it.

Re: High bandwidth interface to FPGA?

Posted: Tue Mar 20, 2018 5:01 pm
by Imre
Thanks for the answers! These seem to be useful. :)
But I am affraid, I have to dig quite deep into linux. :roll:

Re: High bandwidth interface to FPGA?

Posted: Tue Mar 20, 2018 11:53 pm
by Gavinmc42
But I am afraid, I have to dig into linux quite deep
That scares me too ;)

So I was very happy to find something that is not Linux.
https://ultibo.org/

You have not said what you are using the FPGA for, do you need Linux?
If so, then that kernel driver will give you clues.
If you make your FPGA look like a NAND flash it might be one way.

Gert's IDE/SMI interface is similar to NAND flash/compact flash...
The Motorola 6800 interface or Intel 8080 interface are both old, well known to grumpy old guys like me.
These were common to some things like LCDs but at slower rates.

I really need to dig into that 125MHz clock option for pio, that sounds way too fast.
There must be wait and divide by clock cycles or something.

From the manual, the GPCLKs can go to 125MHz but at 1.2V, but I don't think GPIO can go that fast.
Two or three wait cycles?
62.5 and 41.67MHz do seem close to the bitbang results.
125MHz at 1.2V is consistent with 62.5MHz at 3.3V swing.
As far as I can tell the GPCLK pins are just normal GPIO with ALT functions.

New silicon chip in the Pi3B+, it is a flip chip with a mirrored die, what this means it there are no internal bond wires.
Shorter wires in RF terms means higher freqs.
It would be interesting to compare GPCLK0 output on new 3B+ to old 3B on a cro to see what the signal levels and waveshape looks like.
At 40um and flipped die the 3B+ should have the fastest I/O until 28um or less versions come online in 20xx?

Re: High bandwidth interface to FPGA?

Posted: Mon Apr 23, 2018 10:45 pm
by Imre
Thank you for linking ultibo. It seems to be a definitely useful thing. Especially if all else fails. :)
You have not said what you are using the FPGA for, do you need Linux?

I have not said that I use FPGA. It was said by the author of this thread. ;)

I am just also interested in the potential of SMI. It would be a very nice project for me, if I can make two or more RPIs working together, as one system (not in a cluster configuration). Some kind of SMP.

For this, I will need to use FPGA(s) for sure. So, I haven't told yet, but I will use FPGA(s) though. ;)

But at this moment, I am reading about Linux SMP in details. Not an easy topic for sure. At least for me. :)

Re: High bandwidth interface to FPGA?

Posted: Tue Apr 24, 2018 1:53 am
by Gavinmc42
But at this moment, I am reading about Linux SMP in details. Not an easy topic for sure. At least for me. :)
Not easy for me either else I would not be posting on a beginners computer forum :lol:

I have been thinking about a model on the Cray 1/2 made from Zero's.
Using the header in SMI mode as a high speed interface.

It might be faster if CM3's used, perhaps CM3+ when/if it comes out.
A flexible PCB with sockets could connect them all.

Other high speed interfaces are the CSI/DSI ports.
Low voltage differential signals can be quite fast ;)
Easier to get to these on the Compute Modules.
Not sure if ARM core can access those ports?

Now that Gentoo Aarch64 is out the full ARM8vA speed can be used.
So many things to do with Pi's.

Re: High bandwidth interface to FPGA?

Posted: Tue Apr 24, 2018 11:33 am
by Imre
I have been thinking about a model on the Cray 1/2 made from Zero's.
Using the header in SMI mode as a high speed interface.

It sounds an also very interesting project. ;)
Other high speed interfaces are the CSI/DSI ports.
Low voltage differential signals can be quite fast ;)
Easier to get to these on the Compute Modules.
Not sure if ARM core can access those ports?

I have also thought about them. I am not exactly sure, but I found that these peripherals are accessible directly by the VideoCore only, not the ARM cores. In the other hand, I don't know is it possilbe or not to use them for data transfer somehow.

Re: High bandwidth interface to FPGA?

Posted: Tue Apr 24, 2018 12:03 pm
by 6by9
Imre wrote:
Tue Apr 24, 2018 11:33 am
Other high speed interfaces are the CSI/DSI ports.
Low voltage differential signals can be quite fast ;)
Easier to get to these on the Compute Modules.
Not sure if ARM core can access those ports?
I have also thought about them. I am not exactly sure, but I found that these peripherals are accessible directly by the VideoCore only, not the ARM cores. In the other hand, I don't know is it possilbe or not to use them for data transfer somehow.
DSI https://github.com/raspberrypi/linux/bl ... /vc4_dsi.c
CSI https://github.com/raspberrypi/linux/pull/2513

Both are very much designed around transferring image frames, ie big buffers. Small buffers will mean lots of interrupts and may not be practical.

Re: High bandwidth interface to FPGA?

Posted: Tue Apr 24, 2018 12:37 pm
by Gavinmc42
Both are very much designed around transferring image frames, ie big buffers. Small buffers will mean lots of interrupts and may not be practical.
That could be perfect.
Not sure if a DSI display output can talk to a CSI camera input, both are LVDS.
But if they could, then a ring type high speed serial channel could be done.

Actually for CV/NN, a 32bit RGBA image format would be interesting to try.
I think it was Pete Warden who said 8bit data was fine for AI/ML/CV.
It is something I want to try and that is using RGBA image blending methods as NN filters.
Basically a stack of 2D array filters to get a mono (yes/no) output.

As the VC4 will be handling the DSI/CSI data moving, the Aarch64/NEON cores do the image processing.
Near total ignorance suggests it may be worth investigating further.

Some FPGA can do LVDS too :D
How many channels do they have?
Is there FPGA CSI/DSI libraries?
The FPGA could act as CSI/DSI converter?

Re: High bandwidth interface to FPGA?

Posted: Sat Apr 28, 2018 1:56 am
by Gavinmc42
Today's blog has an interesting device. HDMI to CSI.
https://www.raspberrypi.org/blog/tinker ... streaming/

I suppose if you organise your data in image format this is a fast way to move data round too.
That's 3 LVDS high speed serial data methods, HDMI, DSI, CSI.
Do they have lower layers for raw data transfer?

FPGA hat that uses those DSI/CSI connectors.

viewtopic.php?p=786997
https://www.cnx-software.com/2017/05/10 ... rm-factor/
https://www.parallella.org/2015/06/01/t ... ra-module/

Pack bandaids, this looks to be bleeding edge stuff :lol:

Raspiraw just grabs CSI input without going through bayer etc
viewtopic.php?f=43&t=109137

DSI is probably going to be easier as I think there is a Linux framebuffer driver.

Re: High bandwidth interface to FPGA?

Posted: Sat Apr 28, 2018 2:36 am
by Gavinmc42
// TODO: switch to higher clock freq for fast PIO modes
smi_clock_freq = 125000000;

Not sure if that smi_clk is a real 125MHz data clock or what fast PIO mode is?
Quoting myself, but pio??? is that a reference to parallel I/O
https://en.wikipedia.org/wiki/Parallel_I/O

Around the time this was designed-8 years ago?
Compact flash was still around so perhaps this is reference to CF interfaces?
https://en.wikipedia.org/wiki/CompactFlash
The iPod mini, Nokia N91, iriver H10 (5 or 6 GB model), PalmOne LifeDrive, and Rio Carbon used a Microdrive to store data.
Pi + IBM microdrive? Run FreeDOS on it, very retro :lol:

Re: High bandwidth interface to FPGA?

Posted: Sat Apr 28, 2018 7:41 am
by 6by9
Gavinmc42 wrote:
Sat Apr 28, 2018 1:56 am
Today's blog has an interesting device. HDMI to CSI.
https://www.raspberrypi.org/blog/tinker ... streaming/
It's the same Auvidea board that has been discussed so many times before.
Auvidea cloned the prototype developed at Pi Towers and shown at Embedded World in Nuremberg in 2014 http://blog.pi3g.com/2014/03/embedded-w ... bc-market/. The drivers using the firmware have always been a bit flaky, and are not officially supported (I wish our comms team would talk to us sometimes before publicising things!).
Lots of discussion at viewtopic.php?f=38&t=120702.

No, it doesn't add anything over doing an FPGA direct to CSI2, except making life more complicated (you have to worry about EDIDs and the like).

Re: High bandwidth interface to FPGA?

Posted: Sat Apr 28, 2018 8:06 am
by Gavinmc42
I thought that Toshiba part number looked familiar.

Might have to get some FPGAs to play with.
Wonder if a FPGA could emulate a CSI camera?

Stereo sensors to FPGA to CSI?

Hangon, does not the Google AIY Vision sensor use CSI?
https://blog.google/topics/machine-lear ... vices-see/
https://aiyprojects.withgoogle.com/visi ... e-hardware
Any source code for this?

Re: High bandwidth interface to FPGA?

Posted: Sat Apr 28, 2018 12:49 pm
by gsh
Yes, you can use an FPGA to do CSI, with just a couple of resistors, but it won't be able to run full Gbit speed. Instead you can use a CSI phy and connect to the FPGA

Not easy though

Re: High bandwidth interface to FPGA?

Posted: Sun Apr 29, 2018 1:01 am
by Gavinmc42
Yes, you can use an FPGA to do CSI, with just a couple of resistors, but it won't be able to run full Gbit speed. Instead you can use a CSI phy and connect to the FPGA
Not easy though
Serious investment in tools and time to do it.
But is there any faster way to get data into a Pi?
As a thought experiment, it should work, just lots of hard work.
You should have said it is impossible, someone would be out to prove you wrong :lol:

It would be nice to a have a small CPLD board that had LVDS IO and there was free software for designing/programming it.
Most of the good CPLD/FPGA come in packages with lots of pins.

SATA is LVDS? Some sort of converter? SATA has TX/RX so need DSI/CSI.
Starting now, might get it going about the time Pi5 comes out :lol:
So many ideas, so much lack of skill and knowledge :oops:

I would try SMI first, might not be as fast but it would probably work and could be tested in days ;)

Re: High bandwidth interface to FPGA?

Posted: Sun May 12, 2019 8:46 pm
by andrea.varesio
can anyone provide SMI (secondary mamory interface) hardware specification?

Re: High bandwidth interface to FPGA?

Posted: Mon May 13, 2019 12:45 am
by Gavinmc42
can anyone provide SMI (secondary mamory interface) hardware specification?
That's really old, try a search for ATA, PCMCIA.
Not really a spec, more an old Standard.

Somewhat related for the LCD DPI visa GPIO is the 8080/6800 interface modes.
You will find some source code in the Kernel SMI Nand driver.

Re: High bandwidth interface to FPGA?

Posted: Thu Oct 10, 2019 1:40 pm
by BEB
andrea.varesio wrote:
Sun May 12, 2019 8:46 pm
can anyone provide SMI (secondary mamory interface) hardware specification?
The problem is not to have access to the hardware specification. The SMI bus is nothing else than 68xx/808x style interface and there are plenty of references available everywhere on the web giving the description of the signals.

The biggest issue is that Raspberry team did not publish any datasheet describing exact configuration expected on the SMI. The only information available is the source code of the SMI driver and the excellent work from Gert Van Loo to make the IDE interface. There is simply nothing explained in the BCM2837 GPIO datasheet for the SMI configuration.

I have developed an FPGA based interface for the RPi (I called it RasPiBus) and I was planning originally to use SMI for high bandwidth parallel bus. For now, using the SMI is such a pain that I have created my own protocol based on GPIO. Even if it has lower performances than SMI, I am able to get more than 3MB/s on the parallel bus to the FPGA.

Re: High bandwidth interface to FPGA?

Posted: Mon Oct 21, 2019 12:46 am
by gizmomouse
@BEB Why was it challenging to use the SMI interface?

I'm curious because I designed a board with the SMI interface with an FPGA in mind. I'm a ways away from exercising it but one thing that concerned me was that there doesn't seem to be any dedicated SMI clock, perhaps I missed it. This would imply that the FPGA interface would need to be asynchronous, and we would need to over-sample the signals on the FPGA side to read the SMI data... Or some other scheme that I haven't thought of??

I did some research on this when I started working on this board last year and found this within the kernel documentation:

<kernel>/Documentation/devicetree/bindings/misc/brcm/bcm2835-smi.txt

The part that interested me was the 'brcm,smi-clock-source' and the 'brcm,smi-clock-divisor'

It allows the user to select the clock source and clock divider. I was thinking that if I create a GPCLK with the same settings I would then have a 'clocked' SMI interface (there would probably a phase delay but that can be solved with delay taps on an FPGA). Perhaps I'm just naive and this is what was intended or there is another way of doing this. If so please let me know.

As a side note I am hopeful that a throughput of 125MB/s is possible because the settings in the the above documentation are

Code: Select all

...
  brcm,smi-clock-source = <6>;
  brcm,smi-clock-divisor = <4>;
...
based on this page: Raspberry Pi GPCLK Page these settings are for a 125MHz clock.

Dave

Re: High bandwidth interface to FPGA?

Posted: Mon Oct 21, 2019 2:15 am
by Gavinmc42
I think BCM2835 can access GPIO at about 40MHz and the 3's at 60MHz.
Pi4's are faster again?
If you use a 16bit wide bus then 80MB/s would be more realistic?

It is hard to switch cmos 3V3 logic faster than 100-200MHz.
That's why LVDS is "low voltage" and DDRAMs have lower voltage I/O etc.

I have stuck a cro probe on that 125MHz GPCLK0, it was not a full voltage rail to rail swing.
It was only about 1V peak to peak.

You might get 125MB/s or close on a Pi4?
If you read up on 68/80 type bus interface there are CS, RD, WR lines.
RD/WR are clock type data latch signals.
Not sure about sychnnoous mode that has a clock?

Re: High bandwidth interface to FPGA?

Posted: Sun Oct 27, 2019 4:18 pm
by BEB
Gavinmc42 wrote:
Mon Oct 21, 2019 2:15 am
I think BCM2835 can access GPIO at about 40MHz and the 3's at 60MHz.
Pi4's are faster again?
If you use a 16bit wide bus then 80MB/s would be more realistic?

It is hard to switch cmos 3V3 logic faster than 100-200MHz.
That's why LVDS is "low voltage" and DDRAMs have lower voltage I/O etc.

I have stuck a cro probe on that 125MHz GPCLK0, it was not a full voltage rail to rail swing.
It was only about 1V peak to peak.

You might get 125MB/s or close on a Pi4?
If you read up on 68/80 type bus interface there are CS, RD, WR lines.
RD/WR are clock type data latch signals.
Not sure about sychnnoous mode that has a clock?
Hi Gavin,

apparently, the maximum GPIO speed depend on an external factor (I mean not related to core clock frequency) that I don't know for now.

I have made a test on a Pi3 and changing an I/O line takes roughly between 30 and 60ns (direct I/O access, not using the driver). The expansion system I have created requires four I/O cycle to perform a full cycle (since you can't write directly a byte on GPIO if you are not in SMI mode), so I am reaching a bandwidth of 4 to 6 MB/s depeding of optimization level and use/not use of memory barriers.

I have never been able to change GPIO as fast as 40MHz even with direct I/O access (that's why I was expecting to use the SMI mode to get higher bandwidths)

And yes, I agree with you : 150MHz is something I see as the higher limit for LVCMOS. And such a frequency requires EXTREMELY well designed PCB tracks as soon as you deal with a bus. I have made some designs involving 133MHz SDRAM and it becomes quickly a nightmare, believe me.
Just remember that PCI was running at 33MHz max, and it was using some signal tricks (reflection) for data transmission.

Re: High bandwidth interface to FPGA?

Posted: Sun Oct 27, 2019 4:25 pm
by BEB
gizmomouse wrote:
Mon Oct 21, 2019 12:46 am
@BEB Why was it challenging to use the SMI interface?


Dave
Hi Dave,

it is challenging simply because there is simply *no* exact datasheet available. If you look into the RPi driver's source code, there are plenty of constants coming from nowhere to adjust SMI parameters

The best code I have seen for now is the one written by Gert Van Loo for his Parallel ATA test, but it suffers from the same issue : Gert used the Broadcom docs he had at this time to adjust a lot of parameters, leading to a lot of constants you can't understand.

Simply said, I hate to use something on a chip without the original doc, just by reading existing code (except if code is perfectly documented). Each time I did that in the past, in the best case, I was missing a lot of features (you can't know what is not described). And in the worst case, when it does not work, you have no idea why (there is no way to find if the value in a register is correct or not when you debug the code)