Looking for an Embedded Linux for the Pi4 with similar features to Nard

[MikeDB]

I found the Nard build http://www.arbetsmyra.dyndns.org/nard/ which does EXACTLY what I need, but the author confirms it won't currently work on the Pi4 and would take months of effort to retarget. So can anybody point me at anything similar that would work on the Pi4 ?

I'd really like all the things listed on the Nard website, but the key issues for me are :

Running in RAM - no write access to the SIM card except for updates
Runs 64 bit code
Constant running for months on end
Handles power-fail situations seamlessly, including in the middle of an update
Watchdog
Hotswap of USB memory sticks

Doesn't need any of the fancy Linux stuff - just needs to boot and launch a single application which just uses the memory mapping, multithreading, SIM, USB and WiFi APIs.

And I realise this is almost achievable with Bare Metal ... except the WiFi !

[hippy]

I don't believe there is anything similar to NARD and what it offers. The best bet may be PiCore or Ultibo but I don't know if they support the 4B, 64-bits, or have the robustness and capabilities of NARD. Otherwise it's use what there is or go the DIY route.

The Pi 3B/3B+ may end up marking the zenith of third-party alternatives for the Pi as independent developers struggle to migrate their offerings to the 4B, find they don't have the resources or can't afford the investment that requires.

Having to chase the coattails of an ever-changing targeted product has always been a problem for third-party developers. The further it gets away from them the more likely they are to drop out of the race.

[jamesh]

I don't believe there is anything similar to NARD and what it offers. The best bet may be PiCore or Ultibo but I don't know if they support the 4B, 64-bits, or have the robustness and capabilities of NARD. Otherwise it's use what there is or go the DIY route.

The Pi 3B/3B+ may end up marking the zenith of third-party alternatives for the Pi as independent developers struggle to migrate their offerings to the 4B, find they don't have the resources or can't afford the investment that requires.

Having to chase the coattails of an ever-changing targeted product has always been a problem for third-party developers. The further it gets away from them the more likely they are to drop out of the race.

I'm intrigued to know what these major changes are that are making the transition to the Pi4 so difficult. All the relevant changes are in the Linux kernel code (New ethernet driver, new USB driver, device tree) that we provide, so it's not like this stuff is hidden in the firmware, which as before should just be regarded as a black box.

[hippy]

I'm intrigued to know what these major changes are that are making the transition to the Pi4 so difficult.

I honestly don't know - You would have to talk to those developers to find out what their take on things is.

I can only judge by what I see in passing which doesn't support the 4B five months after launch, when developers say that's because work needs to be done to make it work. There doesn't seem to be reluctance to support the 4B but something appears to have prevented them delivering that in a timely manner.

Fedora's take on things is - https://fedoraproject.org/wiki/Architec ... spberry_Pi

"We do not (as of 30th October 2019) support the Raspberry Pi 4 any Fedora release. There is currently a lot of outstanding work to do upstream to enable basic core functionality of the hardware. Until a minimum viable amount of hardware is supported (storage, USB, display, network) in the upstream kernel to enable a useful end user experience this device will remain unsupported".

[MikeDB]

I don't believe there is anything similar to NARD and what it offers. The best bet may be PiCore or Ultibo but I don't know if they support the 4B, 64-bits, or have the robustness and capabilities of NARD. Otherwise it's use what there is or go the DIY route.

The Pi 3B/3B+ may end up marking the zenith of third-party alternatives for the Pi as independent developers struggle to migrate their offerings to the 4B, find they don't have the resources or can't afford the investment that requires.

Having to chase the coattails of an ever-changing targeted product has always been a problem for third-party developers. The further it gets away from them the more likely they are to drop out of the race.

I'm intrigued to know what these major changes are that are making the transition to the Pi4 so difficult. All the relevant changes are in the Linux kernel code (New ethernet driver, new USB driver, device tree) that we provide, so it's not like this stuff is hidden in the firmware, which as before should just be regarded as a black box.

The author of Nard told me he would need to totally update his kernel, and lots of other stuff has to be changed as well so it would take several months and I got the impression he wouldn't be making the effort. I'm sure you could email him for more details as this is well out of my area of expertise.

[MikeDB]

I don't believe there is anything similar to NARD and what it offers. The best bet may be PiCore or Ultibo but I don't know if they support the 4B, 64-bits, or have the robustness and capabilities of NARD. Otherwise it's use what there is or go the DIY route.

I looked at Ultibo a while back but it is a bit left-field of what I am after. But thanks for the link to PiCore which has a new Pi4 release. Unfortunately they are still arguing about getting the 32bit version going rather than accepting that 32 bits is for doorbells and other gadgets and moving to 64 bits straight away. But I'll give it a try and see how well it runs, and if any of the ways of running 64bit applications on Raspian work with it.

[jamesh]

Strange, especially in Fedora's case. We do now use MESA libraries for display, but again, they are completely standard. And you can turn it off and return to legacy if you want (losing 3D acceleration though), USB stack is standard, ethernet stack is standard. It should really just be a matter of enabling the right kernel drivers and setting up device tree, and grabbing the FKMS driver if you want that 3D acceleration.

For non-linux kernels (don't know about NARD's underlying architecture) things would be more difficult, but again, not impossible as the Linux source for the new drivers is available.

Shame people are not allocating resource for this, things won't be moving back, only forward.

[hippy]

Shame people are not allocating resource for this, things won't be moving back, only forward.

I would suppose many are ultimately 'one man projects' not obliged to keep up, don't have the resources to spare, see the effort as outweighing the rewards, or have the will but need to find the resources and make time to do it. It may be that the perception of effort required is greater than it actually is.

[MikeDB]

Strange, especially in Fedora's case. We do now use MESA libraries for display, but again, they are completely standard. And you can turn it off and return to legacy if you want (losing 3D acceleration though), USB stack is standard, ethernet stack is standard. It should really just be a matter of enabling the right kernel drivers and setting up device tree, and grabbing the FKMS driver if you want that 3D acceleration.

Shame people are not allocating resource for this, things won't be moving back, only forward.

I think one reason for their hesitation may be that the documentation for the Pi4 is shall we say somewhat lacking in places. Indeed whilst if may have been said before on this forum, is there any official document that says "We use MESA libraries for display .... they are completely standard ... USB stack is standard, ethernet stack is standard." ? I'm sure that would help overcome nerves. For example the USB page just says "It also means that almost all of the known issues with USB on previous models are no longer present." which just emphasises that the older Pis were imperfect but not how the Pi4 fixes them.

For another example this document hasn't been updated since June and is still woefully short on essential details like how does the new MMU work and what is the caching architecture.
https://www.raspberrypi.org/documentati ... /README.md
I've gained about a 20% speed improvement in the past week by trial and error (and help from this forum) but with a proper design document I would have got it right in the first place.

I'm sure it was the same at the beginning for the original Pi (I actually bought one from the very first batch but found it underpowered so gave to a local school) but on coming back to the Pi4 much of its usage has to be inferred from the documentation for the older Pis and by asking questions on this forum which is bound to put off some potential users.

[jamesh]

We are still waiting for the peripheral spec copy editing to be completed, but that doesn't cover some of the stuff you are requesting (MMU, Caching etc). I'll see if I can cobble something together, once I know what to cobble together.

Out of interest how did you get a 20% improvement on something?

[jamesh]

I think one reason for their hesitation may be that the documentation for the Pi4 is shall we say somewhat lacking in places. Indeed whilst if may have been said before on this forum, is there any official document that says "We use MESA libraries for display .... they are completely standard ... USB stack is standard, ethernet stack is standard." ? I'm sure that would help overcome nerves. For example the USB page just says "It also means that almost all of the known issues with USB on previous models are no longer present." which just emphasises that the older Pis were imperfect but not how the Pi4 fixes them.

The USB page seems mostly fine to me - specifies the chip that is used which is likely all that is needed when porting. Doesn't seem any need to state what is 'fixed' over earlier models - basically it's just a USB controller that actually works under all scenarios. AFAIK, it just uses the completely standard Linux drivers.

https://www.raspberrypi.org/documentati ... /README.md

[MikeDB]

We are still waiting for the peripheral spec copy editing to be completed, but that doesn't cover some of the stuff you are requesting (MMU, Caching etc). I'll see if I can cobble something together, once I know what to cobble together.

Out of interest how did you get a 20% improvement on something?

I was getting lots of cache misses so changed both the code and the data structures around so that I worked more linearly through memory. But I still haven't worked out exactly how much Icache each Pi4 CPU has - the ARM website just says 16-64kbytes. If I knew it exactly I could tune the code more. I tried a few Linux utilities but they all reported 0 bytes for some reason.

I also got the A72 out of order processing working better - it seems GCC is pretty poor at this and needed a lot of help. I had a tight inner loop doing most of the work so could swap instructions around quite easily without using assembly code.

As as example of what a cache document should look like, google for
st AN4839
and open the first pdf shown. This was the chip I used six of before replacing them with a Pi4. This document and the ones it linked allowed me to tune the cache and internal busses and get huge improvements in throughput.


ADDITION for anybody else optimising code : Each CPU has 48k Icache (3-way set associative) and 32k Dcache (2-way set associative). Common L2 cache is 1Mbyte. Found buried deep in the documentation rather than on the front page of the spec. Since these are some of the most important parameters for any processor this seems strange to say the least.