User avatar
Gert van Loo
Posts: 2474
Joined: Tue Aug 02, 2011 7:27 am
Contact: Website

Anybody interested in talking direct! to IDE/SATA?

Thu Nov 16, 2017 8:03 pm

I recently found the remains of a project I started three years ago on my hard drive.
The idea was to use the Raspberr-Pi SMI (which stands for Secondary Memory Interface) to talk to
an IDE interface. Although IDE is old there are still IDE to SATA adapters so it might be used to talk to SATA as well.

I did get some of it working. I managed to use PIO mode to read and write sectors on an old Seagate IDE drive.
(40 pin interface, NOT the 80 pin) With a bit of C-code optimization I got to about 44Mbyte/sec.
However I never managed to get the disk run DMA cycles.
As far as I could tell my command was OK, the interface signals where OK but the disk would just not respond.
I send a support request to Seagate but of course never got a reply.
I also found the wave forms in the ATAPI standard horrific. Who ever made those has never, ever seen a real datasheet.
Maybe I did interpret them wrong.

I still should have a test PCB somewhere (if I can find it)
I have the C-code and the gerber files but I lost the drive to pursue this any further.
But maybe somebody else is interested.
After all the 44Mbyte/sec is independent of what comes over the USB.

fanoush
Posts: 325
Joined: Mon Feb 27, 2012 2:37 pm

Re: Anybody interested in talking direct! to IDE/SATA?

Mon Nov 20, 2017 9:33 pm

Hi Gert,

can you put up the source and schematics/gerbers somewhere like Github? Or at least attach to forum post here. It is hard to estimate how complicated/practical it is without seeing. Thanks a lot.

Btw when googling for SMI I see another post here viewtopic.php?f=31&t=92233&p=1008403#p1008403 and it even looks like there is a driver for this interface in standard kernel!

Can this SMI be also used to connect two Pis and have bidirectional fast data transfer between them? Or does the Pi always need to be a master for connecting slave devices like NAND memory etc? I am thinking how to put Pi Zero (or W) on top of Pi3 or Pi2 as something like accelerator card to get extra fast 512MB swap with possibly using the rest of its hw like usb, wifi etc.

User avatar
Gert van Loo
Posts: 2474
Joined: Tue Aug 02, 2011 7:27 am
Contact: Website

Re: Anybody interested in talking direct! to IDE/SATA?

Tue Nov 21, 2017 10:47 am

I will put it alongside the VGA on Github. (I never was good at working with Git so it might be a bit messy)

There is also a complete (free!) database for a 128 port I/O card there on Github but nobody seem to have discovered that yet!
Can this SMI be also used to connect two Pis and have bidirectional fast data transfer between them?
It is a CPU bus with read and write transactions. Thus always a master.
To simplest way to exchange information between Pi's using SMI would be a dual-ported
16 bit wide memory which is read and written from both sides or two FIFOs.
I have to look into that but what I recall it would need some logic. Maybe a small FPGA with
lots of internal (or external) memory would be required. If somebody is interested I
see if i can spend some time on that.
The C-code for the IDE interface has several routines to deal with the SMI interface and it's settings.
It is very well possible that somebody can reverse engineer from that how the SMI works. :shock:

Post edit: just pushed it out: https://github.com/fenlogic/IDE_trial

fanoush
Posts: 325
Joined: Mon Feb 27, 2012 2:37 pm

Re: Anybody interested in talking direct! to IDE/SATA?

Tue Nov 21, 2017 3:32 pm

Gert van Loo wrote:
Tue Nov 21, 2017 10:47 am
Post edit: just pushed it out: https://github.com/fenlogic/IDE_trial
Thanks a lot!

Looks like you are using CPLD in the schematics so the IDE interface cannot be driven directly by SMI? You need to transform the signals somehow?
Also you mentioned 44MB/s, how could that work with IDE PIO modes, those are supposed to be much slower(?) or was it with some IDE DMA mode?
Gert van Loo wrote:
Tue Nov 21, 2017 10:47 am
To simplest way to exchange information between Pi's using SMI would be a dual-ported
16 bit wide memory which is read and written from both sides or two FIFOs.
I have to look into that but what I recall it would need some logic. Maybe a small FPGA with
lots of internal (or external) memory would be required. If somebody is interested I
see if i can spend some time on that.
Well it would make sense only if it is relatively cheap and easy. Sticking $5 zero on top of $35 pi3 to get fast swap space may make sense. If the cost is much higher it may not be practical. BTW, few weeks ago I found cheap $8 FPGA board http://gnarlygrey.atspace.cc/developmen ... ml#upduino that could be useful for such stuff and does not cost arm and leg. I hope I can learn some FPGA basics with this board and possibly interface it with the Pi in some interesting ways.

User avatar
Gert van Loo
Posts: 2474
Joined: Tue Aug 02, 2011 7:27 am
Contact: Website

Re: Anybody interested in talking direct! to IDE/SATA?

Tue Nov 21, 2017 6:46 pm

Looks like you are using CPLD in the schematics so the IDE interface cannot be driven directly by SMI?
The CPLD was 'just in case'. The current code works without it. (Although maybe it is needed after all to get the DMA sorted out)
That is also in the read me file in the schematics.
16 bit wide memory which is read and written from both sides or two FIFOs.
I have to look into that but what I recall it would need some logic.
I had a look and the SMI offers at least 16 wide data bus with 5 bit address bus.
So that would be a very small memory or 32 FIFOs.
I don't have the specs here so I am not sure it can do address & data multiplexing to support more then 5 address bits.

The SMI header file might be interesting as it tells about how the SMI and it's registers work.
I was not aware I made such an extensive header file.

If you need a hand with the FPGA, I am an ASIC engineer after all, although I do write a lot of software.

User avatar
mikronauts
Posts: 2623
Joined: Sat Jan 05, 2013 7:28 pm
Contact: Website

Re: Anybody interested in talking direct! to IDE/SATA?

Wed Nov 22, 2017 12:52 am

Hi Gert,

Well done! 44MBytes/sec over a channel independent from USB is a nice achievement.

I was looking into doing the same thing, and have been trying to get actual register level documentation from RPF, Broadcom reps etc - no joy.

A couple of kind RPF employees pointed me at the NAND kernel driver example, but frankly I thought it too much effort to reverse engineer the existing drivers, and still no DMA transfers.

If we had full documentation, we should be able to get >100MBytes/sec with DMA, probably 200MBytes per sec.

I am looking forward to see how far you get with this.
http://Mikronauts.com - home of EZasPi, RoboPi, Pi Rtc Dio and Pi Jumper @Mikronauts on Twitter
Advanced Robotics, I/O expansion and prototyping boards for the Raspberry Pi

User avatar
Gavinmc42
Posts: 2042
Joined: Wed Aug 28, 2013 3:31 am

Re: Anybody interested in talking direct! to IDE/SATA?

Wed Nov 22, 2017 3:05 am

Cool, more clues to SMI.
It is not just IDE that can use SMI.
Will be interesting to look at the code, one day I might even understand it ;)
A nice SATA hat in the works?
Can Pi3's boot from SMI/SATA :o Add to list for Pi4 ;)

These look interesting as there are lots of old chips that still use these interfaces :D

Code: Select all

#define SMI_RW_MODE68           0x00800000  // Run cycle motorola mode 
#define SMI_RW_MODE80 0x00000000 // Run cycle intel mode 
Use this for those graphical mono LCDs

Code: Select all

#define SMI_RW_WID8 0x00000000 // Data width 8 bits
IRQPn on gpio27?
Is that using a GPIO interrupt or an undocumented <reserved> ALT mode?

Highspeed ADC's

Keeping that 128 I/O secret?
Someone was after 179 I/O the other day.
128 on one i2c is good, as long at it is not a Pi2 or 3, i2c0 can be used to get 256 I/O ;)
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
Gert van Loo
Posts: 2474
Joined: Tue Aug 02, 2011 7:27 am
Contact: Website

Re: Anybody interested in talking direct! to IDE/SATA?

Wed Nov 22, 2017 5:45 am

We did some I/O test on the pi many, many years ago. (My god, I am getting old...)
At 100 MHz the signal looked more like a sine wave. So my guess is that 50-75 MHz is about the limit.
At 16 bits wide that is 100-150 Mbyte/sec.
I am looking forward to see how far you get with this.
Uhhh.... I have given up on this, that is why started this thread.
I was hoping somebody else would have the enthusiasm I lack.
Keeping that 128 I/O secret?
It has been on Github for three years! You can hardly calling that 'keeping it a secret'
I think it has been shown working on a Raspberry-Pi twitter picture.
IRQPn on gpio27?
Can't remember, I don't think there is an ALT mode for that. It must have been a normal GPIO.
I was looking into doing the same thing, and have been trying to get actual register level documentation from RPF, Broadcom reps etc - no joy.
At the time I wrote that code, I had access to the documentation.
I don't understand why they keep the SMI a secret. The only reason why it is not in the published documentation is because I left it out.
At the time it was no use because there was no way you could get it working on a Pi with the 26 pin header.
Learned another lessen, I fell into the '640K ought to be enough for anybody' trap.
128 on one i2c is good, as long at it is not a Pi2 or 3, i2c0 can be used to get 256 I/O
It uses the 23017 I2C expander, which has 3 address bits and 16 I/O which gives 8x16=128 I/O bits.

User avatar
Gavinmc42
Posts: 2042
Joined: Wed Aug 28, 2013 3:31 am

Re: Anybody interested in talking direct! to IDE/SATA?

Wed Nov 22, 2017 10:49 am

I don't understand why they keep the SMI a secret. The only reason why it is not in the published documentation is because I left it out.
Now we know who to blame :lol:

Seriously SMI is old, my generation old ;)
Motorola 6800, Intel 8080 interfaces have been around for a long time.
Most of them ran 1-4MHz, might have problems slowing the Pi down.
Probably a lot of old code/hardware around that will just work with the info you have already put out there.
Probably some ADC/DACs that could be used too?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
mikronauts
Posts: 2623
Joined: Sat Jan 05, 2013 7:28 pm
Contact: Website

Re: Anybody interested in talking direct! to IDE/SATA?

Sat Nov 25, 2017 4:43 pm

Gert van Loo wrote:
Wed Nov 22, 2017 5:45 am
We did some I/O test on the pi many, many years ago. (My god, I am getting old...)
At 100 MHz the signal looked more like a sine wave. So my guess is that 50-75 MHz is about the limit.
At 16 bits wide that is 100-150 Mbyte/sec.
100MB-150MB/sec would still be sweet :)
Gert van Loo wrote:
Wed Nov 22, 2017 5:45 am
At the time I wrote that code, I had access to the documentation.
I don't understand why they keep the SMI a secret. The only reason why it is not in the published documentation is because I left it out.
At the time it was no use because there was no way you could get it working on a Pi with the 26 pin header.
Learned another lessen, I fell into the '640K ought to be enough for anybody' trap.
Hindsight is great :) I suspect Bill Gates now regrets saying that.

Frankly keeping the SMI bits undocumented now that the signals are available is just dumb, many more interesting add-on's could be built using it. (But I can understand why you did not document it when the signals were not available).

Heck, the Broadcom rep told me that "it is up to the RPF to release it".
http://Mikronauts.com - home of EZasPi, RoboPi, Pi Rtc Dio and Pi Jumper @Mikronauts on Twitter
Advanced Robotics, I/O expansion and prototyping boards for the Raspberry Pi

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 19836
Joined: Sat Jul 30, 2011 7:41 pm

Re: Anybody interested in talking direct! to IDE/SATA?

Sat Nov 25, 2017 8:25 pm

If I remember, I'll try and find out why we have never released that information. It might simply be lack of time.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Please direct all questions to the forum, I do not do support via PM.

User avatar
scruss
Posts: 1703
Joined: Sat Jun 09, 2012 12:25 pm
Location: Toronto, ON
Contact: Website

Re: Anybody interested in talking direct! to IDE/SATA?

Sun Nov 26, 2017 1:46 am

Gert van Loo wrote:
Tue Nov 21, 2017 10:47 am
The C-code for the IDE interface has several routines to deal with the SMI interface and its settings.
I'm no hardware guy, but isn't Compact Flash very close to a 40-pin IDE interface? Some of them can be pretty fast.
‘Remember the Golden Rule of Selling: “Do not resort to violence.”’ — McGlashan.

User avatar
rpdom
Posts: 12588
Joined: Sun May 06, 2012 5:17 am
Location: Essex, UK

Re: Anybody interested in talking direct! to IDE/SATA?

Sun Nov 26, 2017 4:25 am

scruss wrote:
Sun Nov 26, 2017 1:46 am
Gert van Loo wrote:
Tue Nov 21, 2017 10:47 am
The C-code for the IDE interface has several routines to deal with the SMI interface and its settings.
I'm no hardware guy, but isn't Compact Flash very close to a 40-pin IDE interface? Some of them can be pretty fast.
Almost identical. It can run from 3.3V IIRC, which should make things easier.

User avatar
Gavinmc42
Posts: 2042
Joined: Wed Aug 28, 2013 3:31 am

Re: Anybody interested in talking direct! to IDE/SATA?

Sun Nov 26, 2017 6:15 am

I'm no hardware guy, but isn't Compact Flash very close to a 40-pin IDE interface?
Wow I had forgotten that, I have used 64MB CF card for a FreeDOS CNC PC with a IDE to CF adapter which only had a few things on it.
Basically a PCB with IDE header and CF socket and maybe a few jumpers and power connector.

https://en.wikipedia.org/wiki/Parallel_ATA
Hmm, CDROM, Floppy drives?

Can Pi's boots from SMI?
IDE to SSD? yep you can still get them 32, 64, 128, 240GB.
Wow I just think we bypassed the no SATA on Pi issues.
Pi Optical disc -CDROM media player - CDROM Jukebox.

All we need is a BIOS :lol:
Forget those Mini Pi Arcades, I want a Jukebox :D
Dual IDE headers on a Hat?
Must be dreaming.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
Gert van Loo
Posts: 2474
Joined: Tue Aug 02, 2011 7:27 am
Contact: Website

Re: Anybody interested in talking direct! to IDE/SATA?

Sun Nov 26, 2017 3:21 pm

jamesh wrote:
Sat Nov 25, 2017 8:25 pm
If I remember, I'll try and find out why we have never released that information. It might simply be lack of time.
I have just created a stand-alone version of the SMI specs.
I COULD publish it right now but I am not sure what the legal side of this is.
On the one side, I was the one who decided what could and could not be published and the higher management agreed with every one of my choices. Also it is old and there is nothing really special or clever about it.
On the other hand I am no longer employed by Broadcom or RPi so I could also get into trouble.

Maybe I should publish it (it is easier to get forgiveness than permission) and at the same time open a kickstarter fund for the judicial costs... :lol:

hippy
Posts: 3169
Joined: Fri Sep 09, 2011 10:34 pm
Location: UK

Re: Anybody interested in talking direct! to IDE/SATA?

Sun Nov 26, 2017 4:52 pm

scruss wrote:
Sun Nov 26, 2017 1:46 am
I'm no hardware guy, but isn't Compact Flash very close to a 40-pin IDE interface?
It is. One could buy CF to IDE cable adapters, some embedded boards contained IDE buses with CF sockets, and there were even a couple of CF cards with IDE micro-HDD embedded within them.

I designed a CF interface for a PICAXE microcontroller which also worked with IDE HDD. The hard part is getting useful transfer speed. I lost interest in IDE and CF when MMC and then SD cards came along.

User avatar
Gavinmc42
Posts: 2042
Joined: Wed Aug 28, 2013 3:31 am

Re: Anybody interested in talking direct! to IDE/SATA?

Mon Nov 27, 2017 3:49 am

I have just created a stand-alone version of the SMI specs.
Gert, what a tease you are :D
Any patented stuff? All of those would have expired?

It looks like IDE only need two address pins, SA0 ,SA1?
This is great as it leaves SA2-5 as I2C0, I2C1.
A real IDE Hat plus i2c1 for sensors, a seriously large memory sensor datalogger?
Or a HD for a PiTop.

GPIO26, 27 are IRQ and?
Probably going to use all the pins like the DPI TFT interface :( :(

Somewhere, maybe a long time ago, someone may have released Android/Linux code that uses the SMI for DRAM/LCD/IDE/PATA/6800/8080....
Has 8, 9, 16, 18 modes, the 9, 18 bit modes scream DRAM.
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

User avatar
Gert van Loo
Posts: 2474
Joined: Tue Aug 02, 2011 7:27 am
Contact: Website

Re: Anybody interested in talking direct! to IDE/SATA?

Mon Nov 27, 2017 9:36 am

Any patented stuff?
No, as I said there is nothing special about the SMI.
For those interested a bit of history: The first Raspberry-Pi 2835 was originally a 2708.
The 2708 was the same die in a different package which had two 128Mbyte stacked SDRAM chips inside.
However there also was a 2707, the predecessor. A LOT of work was done to the 2707 to get to the 2708.
The GPU was made more powerful, the 3D engine was completely new, the ISP was improved
new peripherals (e.g. SLIM bus) and a lot more like the approval we got to add an ARM core.
The SMI however was the same as the chip BEFORE the 2707 and I think even the one before that.
A such I regard the SMI as old fashioned proven technology. Also there is little you can patent on the
old 8080/6800 bus cycles from the 1970's.
I know TI had a interface which could run 80/68 cycles, but was so infinitely programmable you could even talk to SDRAM with it.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 19836
Joined: Sat Jul 30, 2011 7:41 pm

Re: Anybody interested in talking direct! to IDE/SATA?

Mon Nov 27, 2017 1:24 pm

Gert van Loo wrote:
Mon Nov 27, 2017 9:36 am
Any patented stuff?
No, as I said there is nothing special about the SMI.
For those interested a bit of history: The first Raspberry-Pi 2835 was originally a 2708.
The 2708 was the same die in a different package which had two 128Mbyte stacked SDRAM chips inside.
However there also was a 2707, the predecessor. A LOT of work was done to the 2707 to get to the 2708.
The GPU was made more powerful, the 3D engine was completely new, the ISP was improved
new peripherals (e.g. SLIM bus) and a lot more like the approval we got to add an ARM core.
The SMI however was the same as the chip BEFORE the 2707 and I think even the one before that.
A such I regard the SMI as old fashioned proven technology. Also there is little you can patent on the
old 8080/6800 bus cycles from the 1970's.
I know TI had a interface which could run 80/68 cycles, but was so infinitely programmable you could even talk to SDRAM with it.
Hi Gert, feel free to release the documentation.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Please direct all questions to the forum, I do not do support via PM.

User avatar
Gert van Loo
Posts: 2474
Joined: Tue Aug 02, 2011 7:27 am
Contact: Website

Re: Anybody interested in talking direct! to IDE/SATA?

Mon Nov 27, 2017 3:24 pm

Issues!!!!
Tried to attached it to this post but the website does not accept .pdf file.
I uploaded it to scribd to have it exists alongside my other Pi documents.
Set it to public (lock is open) but it does NOT appear on the main page.
I have raised a question with Scribd.

Before you all start commenting: yes, I know, I also have issues with Scribd.
I used it when it was free but then they changed their police under my feet.
All my other Pi documents are there and I found many websites link to them so I can't change it.

Post edit: Just got an email that they respond to most issues in a week.....

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 19836
Joined: Sat Jul 30, 2011 7:41 pm

Re: Anybody interested in talking direct! to IDE/SATA?

Mon Nov 27, 2017 3:47 pm

You could add it to the Pi documentation archive, its all in github and links from the website https://www.raspberrypi.org/documentation/

And the github repo is

https://github.com/raspberrypi/documentation

Instructions on how to do it here

https://www.raspberrypi.org/documentati ... -github.md
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Please direct all questions to the forum, I do not do support via PM.

User avatar
Gert van Loo
Posts: 2474
Joined: Tue Aug 02, 2011 7:27 am
Contact: Website

Re: Anybody interested in talking direct! to IDE/SATA?

Mon Nov 27, 2017 5:34 pm

I never know exactly what to do with git so this is what I did:
- Forked the documentation.
- Uploaded the file to hardware/raspberrypi/bcm2835.
- Created a pull request.

If any of that needs changing let me know.

gsh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 1280
Joined: Sat Sep 10, 2011 11:43 am

Re: Anybody interested in talking direct! to IDE/SATA?

Mon Nov 27, 2017 7:15 pm

Most of the information is already in the code anyway, in the SMI driver...

Gordon
--
Gordon Hollingworth PhD
Raspberry Pi - Director of Software Engineering

User avatar
Gavinmc42
Posts: 2042
Joined: Wed Aug 28, 2013 3:31 am

Re: Anybody interested in talking direct! to IDE/SATA?

Tue Nov 28, 2017 3:14 am

Thanks guys.

This should be able to boost the speed for those smaller LCDs that have been using SPI mode.
I think some of those chips will also have 68/80 parallel interfaces?
HX8347, yep it has 8, 9, 16, 18 bit modes :D

Need to check Linus's fbtft folder for more.
https://github.com/torvalds/linux/tree/ ... ging/fbtft
Would the drivers already be there and working :shock:

Faster displays for Pi's 8-)

Wonder what the first Hat will be that uses SMI?
I'm dancing on Rainbows.
Raspberries are not Apples or Oranges

Return to “Add-ons”

Who is online

Users browsing this forum: timg236 and 10 guests