Circle - C++ bare metal environment (with USB)

[blippy]

Circle is based on Linux so the USB host driver has a lot of Linux system dependencies, as you mentioned (hence Circle OS). .

I didn't know that Circle borrowed a lot from Linux. I had assumed it was a ground-up implementation.

My own dream goal is to have a kind-of "unikernel" that makes few assumptions about how the host application should work. It would be a highly decoupled system.

I'm not sure if that is ultimately attainable. As more functionality is added, it becomes increasingly reliant on surrounding infrastructure. One therefore ends up with a "framework" rather than a library.

There is an interesting project out there called Ultibo, which is built using Pascal. I tried it, and Pascal is not too bad, really. My disappointment is that seemed to have a pre-emptive scheduler, which will of course thwart attempts at any real-time processing.

An interesting approach seems to be Rumpkernel, which is the idea of running a kernel on top of NetBSD's anykernel. The idea is that the anykernel is a set of driver software. You then layer any flavour of kernel on top of that. NetBSD does, of course, provide its own kernel. This separation of layers strikes me as the way of doing it. I'm a little surprised that the idea hasn't caught on more. I hear that Andy Tanenbaum is porting his Minix3 kernel to the anykernel, for example.

On a different note, I am interested in buying your book. I'm very leery about the publisher, though. Having to agree to T&C's just to read a book puts me off, and I'm generally skeptical about cloud-based platforms. In my view, the majority of people aren't skeptical enough

[rst]

Circle is based on Linux so the USB host driver has a lot of Linux system dependencies, as you mentioned (hence Circle OS). .

I didn't know that Circle borrowed a lot from Linux. I had assumed it was a ground-up implementation.

Indeed it is. Circle has a totally different concept, compared to Linux, with tightly coupled C++ classes. There are a number of drivers ported from Linux for Circle. The information for USB stack for the RPi 1-3 of course is widely based on the Linux driver, because there was only CSUD available as an alternative, when Circle started. The USB support for the RPi 4 has been developed from scratch. And yes, it is not that easy, to build Circle-based applications on non-Linux hosts, but it is possible with some adaptions to the Makefiles and scripts. Unfortunately I found no Make version 4.x for Windows, which is required to use all Circle functions.

I'm not sure if that is ultimately attainable. As more functionality is added, it becomes increasingly reliant on surrounding infrastructure. One therefore ends up with a "framework" rather than a library.

Indeed. That was the idea behind Circle, to provide a framework and the required infrastructure to develop bare-metal applications on the Raspberry Pi. BTW I wouldn't call it an OS, because it does not provide functions, which an OS normally has (e.g. loading programs). It's more like programming on a very fast micro-controller.

[sean.lawless]

On a different note, I am interested in buying your book. I'm very leery about the publisher, though. Having to agree to T&C's just to read a book puts me off, and I'm generally skeptical about cloud-based platforms. In my view, the majority of people aren't skeptical enough

I typically agree, although after years of working with the team at Leanpub they have earned my trust.

Use the links below to directly download the first half of the book (sample) as a PDF, without signing up on Leanpub.

http://samples.leanpub.com/computersystems-sample.pdf
http://samples.leanpub.com/computersyst ... sample.pdf

If you would like to purchase outside of Leanpub you can send me a $10 payment on Paypal ([removed for security; IM me for my email address]) and I will email you the PDFs. If you want any format besides PDF you will have to use Leanpub.

Cheers,
Sean

[rst]

With the new Circle release 43.2 it is now possible to build all Circle sample programs on Windows without modification on build scripts or make-files. Details are available in the file doc/windows-build.txt in the project.

[aj22r]

Thanks for this amazing project! I have already made a working shoutcast webradio player that outputs to I2S on a Pi Zero W. The only thing missing is bluetooth to use the Pi Zero W as an A2DP source. Have you looked at this https://github.com/bluekitchen/btstack? It already has a port for the raspberry pi but only linux, not bare metal. I've been trying to port it to circle for a while now, but it is a massive pain to get it all hooked up and working.

[rst]

Thanks for this amazing project! I have already made a working shoutcast webradio player that outputs to I2S on a Pi Zero W. The only thing missing is bluetooth to use the Pi Zero W as an A2DP source. Have you looked at this https://github.com/bluekitchen/btstack? It already has a port for the raspberry pi but only linux, not bare metal. I've been trying to port it to circle for a while now, but it is a massive pain to get it all hooked up and working.

Thanks! Your project sounds very interesting. There has been rudimentary BT support earlier in Circle, but it was stopped, because of legal problems with the BT specs, which can only be used by members of the BT SIG. Sorry, but currently I do not have any plans to support BT again in Circle.

[rst]

Thanks to @MikeDB, I was able to test the CM4 on a CM4IO board with Circle. Generally it is working well. But there is still some work to do. I tried the internal xHCI controller of the BCM2711 and it is detected with the "otg_mode=1" setting in config.txt, but unfortunately it does not found an USB hub. The Gigabit Ethernet controller does not connect to the PHY. Probably a problem with the Circle driver yet.

[cleverca22]

Thanks to @MikeDB, I was able to test the CM4 on a CM4IO board with Circle. Generally it is working well. But there is still some work to do. I tried the internal xHCI controller of the BCM2711 and it is detected with the "otg_mode=1" setting in config.txt, but unfortunately it does not found an USB hub. The Gigabit Ethernet controller does not connect to the PHY. Probably a problem with the Circle driver yet.

i believe the CM4 is using a different PHY from the rpi4b, so it will need new drivers

[aj22r]

Thanks for this amazing project! I have already made a working shoutcast webradio player that outputs to I2S on a Pi Zero W. The only thing missing is bluetooth to use the Pi Zero W as an A2DP source. Have you looked at this https://github.com/bluekitchen/btstack? It already has a port for the raspberry pi but only linux, not bare metal. I've been trying to port it to circle for a while now, but it is a massive pain to get it all hooked up and working.

Thanks! Your project sounds very interesting. There has been rudimentary BT support earlier in Circle, but it was stopped, because of legal problems with the BT specs, which can only be used by members of the BT SIG. Sorry, but currently I do not have any plans to support BT again in Circle.

I've abandoned bluetooth for now and instead decided to write a driver for the enc28j60 to be able to use ethernet on the pi zero for one of my projects. I thought it would be easy and i would be done in a few hours but instead i went through even more hours of debugging and reflashing the sd card than when i was trying to implement BT. For some reason commenting out or changing the order of CLogger::Get()->Write being called completely changes the behaviour of the program. And also when CNetDevice::SendFrame gets called too much(?) for example when the pi sends a file over tftp the entire CNetDevice hangs and doesn't receive/send any more packets (CNetDevice::ReceiveFrame is still being polled though). Have you experienced any of this when writing the wlan or usb ethernet driver?

[rst]

i believe the CM4 is using a different PHY from the rpi4b, so it will need new drivers

It must be something like that. I'm trying to get the internal xHCI controller of the CM4 running first and will come back to this later.

[rst]

For some reason commenting out or changing the order of CLogger::Get()->Write being called completely changes the behaviour of the program.

Yes, the logger changes the timing, especially when it writes to the screen and scrolls. You should enable the REALTIME system option in include/circle/sysconfig.h. This reduces this effect very much. You will not see logger messages, generated from IRQ handlers, then any more.

And also when CNetDevice::SendFrame gets called too much(?) for example when the pi sends a file over tftp the entire CNetDevice hangs and doesn't receive/send any more packets (CNetDevice::ReceiveFrame is still being polled though). Have you experienced any of this when writing the wlan or usb ethernet driver?

No, but I worked only with network drivers over USB high-speed (480 Mbps) or with internal queues, read and written using DMA. That's much quicker than SPI. Transferring a 1500 byte Ethernet frame at 12 Mbps SPI clock takes 1 millisecond, which is relatively long, compared to the other interfaces. You may need some mechanism, so that not many frames are transferred in a row to/from the controller. This is implemented in lib/net/netdevlayer.cpp. Perhaps it is already enough to add the following after each SendFrame() and ReceiveFrame() (in line 115 and 121):

Code: Select all

CScheduler::Get ()->Yield ();

[cleverca22]

For some reason commenting out or changing the order of CLogger::Get()->Write being called completely changes the behaviour of the program.

Yes, the logger changes the timing, especially when it writes to the screen and scrolls.

the impact of scrolling could be greatly reduced if you make use of the HVS

https://github.com/librerpi/rpi-open-fi ... ocs/hvs.md
https://blog.benjdoherty.com/2019/05/21 ... pberry-Pi/
https://github.com/librerpi/lk/blob/vc4 ... /hvs/hvs.c

though the mailbox framebuffer also allows the same tricks, but with less flexibility

basically, allocate a framebuffer that is maybe 2x the height of the screen, and just change the y offset to move the entire framebuffer up/down, without having to move the contents

using the HVS directly, would make the wrap-around simpler, because you can display the tail and head of the image as 2 layers
while the mailbox api doesnt allow such wrap-around at the tail of the buffer

HVS drivers would also open up the ability to do sprites: https://www.youtube.com/watch?v=JFmCin3EJIs
that demo video was done with baremetal arm based on LK

[rst]

the impact of scrolling could be greatly reduced if you make use of the HVS

That's interesting. Currently I have some other work items, but when this is done, I will have a closer look on HVS. Thanks for info!

[cleverca22]

the impact of scrolling could be greatly reduced if you make use of the HVS

That's interesting. Currently I have some other work items, but when this is done, I will have a closer look on HVS. Thanks for info!

when using 1:1 mode, you basically just generate a list of packets, each containing:

• width/height of the image
• x/y to render at
• pixel format
• if either axis is being flipped
• stride
• physical address (same dma rules as normal vc stuff, 0xc000_0000 based for pi2 and up)
• an optional alpha override

and then the HVS will render every image in the list
the only real limits are the total amount of dedicated ram the list must fit within, and the memory bandwidth of the HVS to fetch all the data, its being fetched on the fly, as it generates each scanline

[aj22r]

You may need some mechanism, so that not many frames are transferred in a row to/from the controller. This is implemented in lib/net/netdevlayer.cpp. Perhaps it is already enough to add the following after each SendFrame() and ReceiveFrame() (in line 115 and 121):

The problem was actually that the ENC28J60 was holding the INT line low which meant that the BCM2835 wasn't receiving any more gpio interrupts on that pin. The solution was to add a while(!m_IntPin.Read()) loop into the IRQ handler. Everything now seems relatively stable with a few dropped connections every now and then. A bigger problem however is apparently a race condition in CNetQueue. After some time of running, m_RxQueue.Enqueue in the IRQ handler seems to get called at the same time as m_RxQueue.Dequeue in ReceiveFrame which leads to this:

[rst]

A bigger problem however is apparently a race condition in CNetQueue. After some time of running, m_RxQueue.Enqueue in the IRQ handler seems to get called at the same time as m_RxQueue.Dequeue in ReceiveFrame which leads to this:
assertion failed in netqueue.cpp(129) ...

CNetQueue is normally not meant to be used from an IRQ handler. If you really need this, replace TASK_LEVEL with IRQ_LEVEL in this line.

[rst]

when using 1:1 mode, you basically just generate a list of packets, each containing:

• width/height of the image
• x/y to render at
• pixel format
• if either axis is being flipped
• stride
• physical address (same dma rules as normal vc stuff, 0xc000_0000 based for pi2 and up)
• an optional alpha override

and then the HVS will render every image in the list
the only real limits are the total amount of dedicated ram the list must fit within, and the memory bandwidth of the HVS to fetch all the data, its being fetched on the fly, as it generates each scanline

Thanks! I have to see, if this fits into Circle. I don't want to make the screen output too complicated, given that most applications use their own screen routines or GUIs and there is also accelerated graphics support for the RPi 1-3.

[rst]

i believe the CM4 is using a different PHY from the rpi4b, so it will need new drivers

It must be something like that. I'm trying to get the internal xHCI controller of the CM4 running first and will come back to this later.

The CM4 is using PHY address 0, instead of 1, which is used by RPi 4B and 400. The driver is the same. The internal xHCI controller is working now too. Remains the eMMC access, which is not working at the moment with the RPi 4B EMMC2 driver.