kmuffsfpv
Posts: 2
Joined: Mon Mar 05, 2018 7:38 am

High bandwidth interface to FPGA?

Fri Mar 16, 2018 6:03 am

Hi, I was hoping someone can point me in the right direction.. I haven't been able to find whether or not it would be possible to use the SMI, SDIO, or another interface on the CM3 to get decent data rates between the CM3 and an FPGA (ideally >300Mb/s).. Considering that the broadcom chip is fairly closed as far as documentation, it would be very nice to know if someone has tried this before.. :P

Thanks
- Kmuffs

User avatar
Gavinmc42
Posts: 2108
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Fri Mar 16, 2018 8:27 am

Gert posted some SMI-IDE code.
viewtopic.php?f=45&t=197875

SMI 16bit wide is only about 19MHz for 300Mbs.
I think SMI will do 40+MHz? But Gert is an expert ;)

A search has shown 25MHz has been done.
Get a bit more speed going 18bit wide?

Use a Pi 3B+ Ethernet?
Use one core for the DMA?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

Imre
Posts: 15
Joined: Fri Jul 14, 2017 8:43 pm
Location: Hungary
Contact: Website

Re: High bandwidth interface to FPGA?

Mon Mar 19, 2018 11:13 pm

Hi Gavinmc,

I've read the topic you linked, and there was a discussion about to make the SMI documentation public. I checked the Github repo, but could not find it. Maybe I did something wrong.
Do you know it was/still is available somewhere?
I am also interesed in this peripheral. ;)

Thanks,
Imre
'Good luck, Mr. Gorsky!' - Neil Armstrong, July 21, 1969, Moon

User avatar
Gavinmc42
Posts: 2108
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Tue Mar 20, 2018 1:14 pm

The code is the documentation?
https://github.com/fenlogic/IDE_trial
These is a simple FPGA in the schematic too.

Been a while, not sure if there was more docs.
The actual SMI registers and how to set them up is the main trick.

I have not tried it, I don't use C these days.
But it looks to be word read and write and block read and write code.

Code: Select all

 // TODO: switch to higher clock freq for fast PIO modes 
smi_clock_freq = 125000000;
Not sure if that smi_clk is a real 125MHz data clock or what fast PIO mode is?

Recent GPIO bit banging on the new Pi3B+ show software toggling at 60MHz compared to 40MHz on the BCM835 chips.
But software toggling is not DMA based word data dumping.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5671
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: High bandwidth interface to FPGA?

Tue Mar 20, 2018 2:40 pm

There's also a kernel driver for it - https://github.com/raspberrypi/linux/bl ... _smi_dev.c
I haven't checked on the apis for it.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

Imre
Posts: 15
Joined: Fri Jul 14, 2017 8:43 pm
Location: Hungary
Contact: Website

Re: High bandwidth interface to FPGA?

Tue Mar 20, 2018 5:01 pm

Thanks for the answers! These seem to be useful. :)
But I am affraid, I have to dig quite deep into linux. :roll:
'Good luck, Mr. Gorsky!' - Neil Armstrong, July 21, 1969, Moon

User avatar
Gavinmc42
Posts: 2108
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Tue Mar 20, 2018 11:53 pm

But I am afraid, I have to dig into linux quite deep
That scares me too ;)

So I was very happy to find something that is not Linux.
https://ultibo.org/

You have not said what you are using the FPGA for, do you need Linux?
If so, then that kernel driver will give you clues.
If you make your FPGA look like a NAND flash it might be one way.

Gert's IDE/SMI interface is similar to NAND flash/compact flash...
The Motorola 6800 interface or Intel 8080 interface are both old, well known to grumpy old guys like me.
These were common to some things like LCDs but at slower rates.

I really need to dig into that 125MHz clock option for pio, that sounds way too fast.
There must be wait and divide by clock cycles or something.

From the manual, the GPCLKs can go to 125MHz but at 1.2V, but I don't think GPIO can go that fast.
Two or three wait cycles?
62.5 and 41.67MHz do seem close to the bitbang results.
125MHz at 1.2V is consistent with 62.5MHz at 3.3V swing.
As far as I can tell the GPCLK pins are just normal GPIO with ALT functions.

New silicon chip in the Pi3B+, it is a flip chip with a mirrored die, what this means it there are no internal bond wires.
Shorter wires in RF terms means higher freqs.
It would be interesting to compare GPCLK0 output on new 3B+ to old 3B on a cro to see what the signal levels and waveshape looks like.
At 40um and flipped die the 3B+ should have the fastest I/O until 28um or less versions come online in 20xx?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

Imre
Posts: 15
Joined: Fri Jul 14, 2017 8:43 pm
Location: Hungary
Contact: Website

Re: High bandwidth interface to FPGA?

Mon Apr 23, 2018 10:45 pm

Thank you for linking ultibo. It seems to be a definitely useful thing. Especially if all else fails. :)
You have not said what you are using the FPGA for, do you need Linux?

I have not said that I use FPGA. It was said by the author of this thread. ;)

I am just also interested in the potential of SMI. It would be a very nice project for me, if I can make two or more RPIs working together, as one system (not in a cluster configuration). Some kind of SMP.

For this, I will need to use FPGA(s) for sure. So, I haven't told yet, but I will use FPGA(s) though. ;)

But at this moment, I am reading about Linux SMP in details. Not an easy topic for sure. At least for me. :)
'Good luck, Mr. Gorsky!' - Neil Armstrong, July 21, 1969, Moon

User avatar
Gavinmc42
Posts: 2108
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Tue Apr 24, 2018 1:53 am

But at this moment, I am reading about Linux SMP in details. Not an easy topic for sure. At least for me. :)
Not easy for me either else I would not be posting on a beginners computer forum :lol:

I have been thinking about a model on the Cray 1/2 made from Zero's.
Using the header in SMI mode as a high speed interface.

It might be faster if CM3's used, perhaps CM3+ when/if it comes out.
A flexible PCB with sockets could connect them all.

Other high speed interfaces are the CSI/DSI ports.
Low voltage differential signals can be quite fast ;)
Easier to get to these on the Compute Modules.
Not sure if ARM core can access those ports?

Now that Gentoo Aarch64 is out the full ARM8vA speed can be used.
So many things to do with Pi's.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

Imre
Posts: 15
Joined: Fri Jul 14, 2017 8:43 pm
Location: Hungary
Contact: Website

Re: High bandwidth interface to FPGA?

Tue Apr 24, 2018 11:33 am

I have been thinking about a model on the Cray 1/2 made from Zero's.
Using the header in SMI mode as a high speed interface.

It sounds an also very interesting project. ;)
Other high speed interfaces are the CSI/DSI ports.
Low voltage differential signals can be quite fast ;)
Easier to get to these on the Compute Modules.
Not sure if ARM core can access those ports?

I have also thought about them. I am not exactly sure, but I found that these peripherals are accessible directly by the VideoCore only, not the ARM cores. In the other hand, I don't know is it possilbe or not to use them for data transfer somehow.
'Good luck, Mr. Gorsky!' - Neil Armstrong, July 21, 1969, Moon

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5671
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: High bandwidth interface to FPGA?

Tue Apr 24, 2018 12:03 pm

Imre wrote:
Tue Apr 24, 2018 11:33 am
Other high speed interfaces are the CSI/DSI ports.
Low voltage differential signals can be quite fast ;)
Easier to get to these on the Compute Modules.
Not sure if ARM core can access those ports?
I have also thought about them. I am not exactly sure, but I found that these peripherals are accessible directly by the VideoCore only, not the ARM cores. In the other hand, I don't know is it possilbe or not to use them for data transfer somehow.
DSI https://github.com/raspberrypi/linux/bl ... /vc4_dsi.c
CSI https://github.com/raspberrypi/linux/pull/2513

Both are very much designed around transferring image frames, ie big buffers. Small buffers will mean lots of interrupts and may not be practical.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
Gavinmc42
Posts: 2108
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Tue Apr 24, 2018 12:37 pm

Both are very much designed around transferring image frames, ie big buffers. Small buffers will mean lots of interrupts and may not be practical.
That could be perfect.
Not sure if a DSI display output can talk to a CSI camera input, both are LVDS.
But if they could, then a ring type high speed serial channel could be done.

Actually for CV/NN, a 32bit RGBA image format would be interesting to try.
I think it was Pete Warden who said 8bit data was fine for AI/ML/CV.
It is something I want to try and that is using RGBA image blending methods as NN filters.
Basically a stack of 2D array filters to get a mono (yes/no) output.

As the VC4 will be handling the DSI/CSI data moving, the Aarch64/NEON cores do the image processing.
Near total ignorance suggests it may be worth investigating further.

Some FPGA can do LVDS too :D
How many channels do they have?
Is there FPGA CSI/DSI libraries?
The FPGA could act as CSI/DSI converter?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
Gavinmc42
Posts: 2108
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Sat Apr 28, 2018 1:56 am

Today's blog has an interesting device. HDMI to CSI.
https://www.raspberrypi.org/blog/tinker ... streaming/

I suppose if you organise your data in image format this is a fast way to move data round too.
That's 3 LVDS high speed serial data methods, HDMI, DSI, CSI.
Do they have lower layers for raw data transfer?

FPGA hat that uses those DSI/CSI connectors.

viewtopic.php?p=786997
https://www.cnx-software.com/2017/05/10 ... rm-factor/
https://www.parallella.org/2015/06/01/t ... ra-module/

Pack bandaids, this looks to be bleeding edge stuff :lol:

Raspiraw just grabs CSI input without going through bayer etc
viewtopic.php?f=43&t=109137

DSI is probably going to be easier as I think there is a Linux framebuffer driver.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
Gavinmc42
Posts: 2108
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Sat Apr 28, 2018 2:36 am

// TODO: switch to higher clock freq for fast PIO modes
smi_clock_freq = 125000000;

Not sure if that smi_clk is a real 125MHz data clock or what fast PIO mode is?
Quoting myself, but pio??? is that a reference to parallel I/O
https://en.wikipedia.org/wiki/Parallel_I/O

Around the time this was designed-8 years ago?
Compact flash was still around so perhaps this is reference to CF interfaces?
https://en.wikipedia.org/wiki/CompactFlash
The iPod mini, Nokia N91, iriver H10 (5 or 6 GB model), PalmOne LifeDrive, and Rio Carbon used a Microdrive to store data.
Pi + IBM microdrive? Run FreeDOS on it, very retro :lol:
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5671
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: High bandwidth interface to FPGA?

Sat Apr 28, 2018 7:41 am

Gavinmc42 wrote:
Sat Apr 28, 2018 1:56 am
Today's blog has an interesting device. HDMI to CSI.
https://www.raspberrypi.org/blog/tinker ... streaming/
It's the same Auvidea board that has been discussed so many times before.
Auvidea cloned the prototype developed at Pi Towers and shown at Embedded World in Nuremberg in 2014 http://blog.pi3g.com/2014/03/embedded-w ... bc-market/. The drivers using the firmware have always been a bit flaky, and are not officially supported (I wish our comms team would talk to us sometimes before publicising things!).
Lots of discussion at viewtopic.php?f=38&t=120702.

No, it doesn't add anything over doing an FPGA direct to CSI2, except making life more complicated (you have to worry about EDIDs and the like).
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

User avatar
Gavinmc42
Posts: 2108
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Sat Apr 28, 2018 8:06 am

I thought that Toshiba part number looked familiar.

Might have to get some FPGAs to play with.
Wonder if a FPGA could emulate a CSI camera?

Stereo sensors to FPGA to CSI?

Hangon, does not the Google AIY Vision sensor use CSI?
https://blog.google/topics/machine-lear ... vices-see/
https://aiyprojects.withgoogle.com/visi ... e-hardware
Any source code for this?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

gsh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1317
Joined: Sat Sep 10, 2011 11:43 am

Re: High bandwidth interface to FPGA?

Sat Apr 28, 2018 12:49 pm

Yes, you can use an FPGA to do CSI, with just a couple of resistors, but it won't be able to run full Gbit speed. Instead you can use a CSI phy and connect to the FPGA

Not easy though
--
Gordon Hollingworth PhD
Raspberry Pi - Director of Software Engineering

User avatar
Gavinmc42
Posts: 2108
Joined: Wed Aug 28, 2013 3:31 am

Re: High bandwidth interface to FPGA?

Sun Apr 29, 2018 1:01 am

Yes, you can use an FPGA to do CSI, with just a couple of resistors, but it won't be able to run full Gbit speed. Instead you can use a CSI phy and connect to the FPGA
Not easy though
Serious investment in tools and time to do it.
But is there any faster way to get data into a Pi?
As a thought experiment, it should work, just lots of hard work.
You should have said it is impossible, someone would be out to prove you wrong :lol:

It would be nice to a have a small CPLD board that had LVDS IO and there was free software for designing/programming it.
Most of the good CPLD/FPGA come in packages with lots of pins.

SATA is LVDS? Some sort of converter? SATA has TX/RX so need DSI/CSI.
Starting now, might get it going about the time Pi5 comes out :lol:
So many ideas, so much lack of skill and knowledge :oops:

I would try SMI first, might not be as fast but it would probably work and could be tested in days ;)
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

Return to “Compute Module”

Who is online

Users browsing this forum: No registered users and 2 guests