gtoal
Posts: 113
Joined: Sun Nov 18, 2012 12:02 am

Re: Reserving cores for bare-metal programs - interupts

Wed Aug 07, 2019 4:22 pm

Heater wrote:
Wed Aug 07, 2019 9:22 am
gtoal,

Wow, what an awesome project. I love those old vector graphic screens. Last one I programmed was a huge round display for Marconi Radar. But that had a hardware driver for the display, just build a list of objects to draw in memory and away it goes.

The Pi Zero is only a single core machine. Sadly what we have discussed here, isolating a core, is not applicable. Any particular reason you need to use a Pi Zero? I suspect it may work with an isolated core on a bigger Pi, even without delving into that pesky interrupt that is eating 5us occasionally.
As I mentioned above, we know that we couldn't do this on a Pi Zero, and for the moment a Pi Zero is a requirement because of the physical constraints, but we can use a ribbon cable or a 90-degree connector which would let us use a full sized pi at least for development experiments - have a look at the various mounting options in this wiki page: http://computernerdkev.heliohost.org/pi ... escription

I'm assuming at some point that the Raspberry Pi Corporation will bring out a multi-core in the Zero format, and even if they don't there's always the 4-core Banana Pi M2 Zero (if we're allowed to mention that here without getting crucified!)

So yes, I was planning to do some experiments in this area in the next day or two using a full sized pi. I've read the various posts about isolcpus etc but it sounds like you guys have already done a lot of testing and no doubt have found various details that may take us some time to rediscover, so I was hoping we could build on your experience rather than have to reinvent it...

regards,

Graham
PS Neighbour? I worked at GEC Borehamwood for years, in the floor below Marconi Radar. In fact I think you guys' radar used to cook our meals for us in the canteen across the street ;-) - no - wait, you're the guy from Great Baddow. I remember now, we chatted about Coral...

Heater
Posts: 14714
Joined: Tue Jul 17, 2012 3:02 pm

Re: Reserving cores for bare-metal programs - interupts

Wed Aug 07, 2019 8:31 pm

Yep. That's me. Marconi Radar at Great Baddow, then later Writtle Road Chelmsford. Hence the big vector graphic radar displays.

I think you have inspired me dig out my scope and try my isolated core experiments again. Got to get that display of your working...
Memory in C++ is a leaky abstraction .

gtoal
Posts: 113
Joined: Sun Nov 18, 2012 12:02 am

Re: Reserving cores for bare-metal programs - interupts

Wed Aug 07, 2019 9:47 pm

Well, I modified bcm2835.h rather trivially (since I'm only testing with 2 kinds of boards):

Code: Select all

#ifdef PIZERO
/*! Peripherals block base address on RPi 1 */
#define BCM2835_PERI_BASE               0x20000000
#else
#ifdef PI3B
#define BCM2835_PERI_BASE               0x3F000000
#else
#error Must compile bcm2835 with -DPIZERO or -DPI3B
#endif
#endif
added "isolcpus=3" to the end of cmdline.txt

and invoked my program as root with: taskset 0x08 ./vecx

It runs, not too much random crap happening on screen, but more than when run bare metal as a kernel image.

There are still several unix processes running on this core.

Code: Select all

[email protected]:~ $ ps -eLo ruser,pid,ppid,lwp,psr,args|awk '{if ($5==3) print $0}'
root        22     2    22   3 [cpuhp/3]
root        23     2    23   3 [migration/3]
root        24     2    24   3 [ksoftirqd/3]
root        25     2    25   3 [kworker/3:0]
root        26     2    26   3 [kworker/3:0H]
root       849     2   849   3 [kworker/3:1H]
root       850     2   850   3 [kworker/3:1]
root       922   857   922   3 ./vecx
so, any tricks for eliminating the remainder of the OS overhead?

User avatar
rpdom
Posts: 16329
Joined: Sun May 06, 2012 5:17 am
Location: Chelmsford, Essex, UK

Re: Reserving cores for bare-metal programs - interupts

Wed Aug 07, 2019 9:52 pm

Heater wrote:
Wed Aug 07, 2019 8:31 pm
Yep. That's me. Marconi Radar at Great Baddow, then later Writtle Road Chelmsford. Hence the big vector graphic radar displays.
It's a shame there's very little left of those places now. One of the towers still stands at Baddow, but most of the other Marconi buildings have either been demolished or turned into flats or offices.

LdB
Posts: 1478
Joined: Wed Dec 07, 2016 2:29 pm

Re: Reserving cores for bare-metal programs - interupts

Thu Aug 08, 2019 12:24 am

We did this dance above the main source of jitter you can't get rid of which is the 1Khz task switch interrupt because the core is not really isolated at all in a baremetal sense. There is a terminolgy problem that yes from a linux point of view you have isolated the core in that normal tasks wont run on it. However compared to a baremetal isolated core the core is not usable for chunks of time which is the timing jitter you notice.

What might work in your specific case since you are trying to generate signals is DMA. It is slightly slower than bashing the GPIO directly but the interrupt jitter will be much less.

Realistically with what you are trying to do a simple FPGA hat on the pi in which you setup a proper timing state machine would make what you are doing so much easier. However the obvious problem is it requires you have understanding of FPGA and Verilog or VHDL programming.

Heater
Posts: 14714
Joined: Tue Jul 17, 2012 3:02 pm

Re: Reserving cores for bare-metal programs - interupts

Thu Aug 08, 2019 5:58 am

But there is the thing. It has been suggested in this thread that it may well be possible to clobber an interrupt vector table some place and stop any 1KHz task switch interrupt or whatever it is. Or provide a "null" handler for it that takes only a couple of nats to return. I find that an intriguing prospect.
Memory in C++ is a leaky abstraction .

LdB
Posts: 1478
Joined: Wed Dec 07, 2016 2:29 pm

Re: Reserving cores for bare-metal programs - interupts

Thu Aug 08, 2019 12:26 pm

You don't have to clobber anything I told you how to turn it off, write zero to address 0x4000004C which turns the EL0 timer IRQ off from the core. What linux will make of that and whether it will give you the mapping I have no idea because there are still linux threads assigned to the core all be it unimportant threads.

Heater
Posts: 14714
Joined: Tue Jul 17, 2012 3:02 pm

Re: Reserving cores for bare-metal programs - interupts

Thu Aug 08, 2019 4:09 pm

LdB,

Sorry that was a bit vague of me. When I said "clobber" I simply meant turn off, disable, divert, or whatever it takes to stop that interrupt bothering us. Or any other.

Sure I have no idea what happens when writing to random registers and addresses. That is something I need to find out.

Having threads hanging around there may not be an issue if they are never woken up.

Thanks for the hints and tips. Hopefully I'll find time to look more seriously at it and try some experiments soon.
Memory in C++ is a leaky abstraction .

gtoal
Posts: 113
Joined: Sun Nov 18, 2012 12:02 am

Re: Reserving cores for bare-metal programs - interupts

Thu Aug 08, 2019 8:14 pm

Is this what you mean?

Code: Select all

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/mman.h>

/** Main function - we'll never return from here */
void main()
{
  static volatile uint32_t *regs = NULL;
  int   fd;
  cpu_set_t cpus;

  CPU_ZERO(&cpus);

  CPU_SET(3, &cpus);

  if (sched_setaffinity(0, sizeof(cpus), &cpus)) {
    perror("sched_setaffinity");
    exit(1);
  }

  if ((fd = open ("/dev/mem", O_RDWR | O_SYNC | O_CLOEXEC) ) < 0) exit(1);
  regs = (uint32_t *)mmap(0, 0x10000 /* BLOCK_SIZE */, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0x40000000);
  if ((int32_t)regs == -1) exit(1);

  regs[0x4C/sizeof(uint32_t)] = 0x00000000; // TURN OFF IRQs

  vectrexinit(1);
  initTailGunner();
It seems to run without crashing, though I can't be 100% sure it actually had any effect. But so far it does look like our graphics are running clean without the jitter you'd expect from interrupts... (but then it was only very occasional before I added this code, and maybe I just haven't noticed any yet)

pik33
Posts: 187
Joined: Thu Sep 10, 2015 4:26 pm

Re: Reserving cores for bare-metal programs - interupts

Mon Aug 12, 2019 6:42 pm

Isn't it possible that the reason of "interrupts" is the memory refresh and/or cpu freq switching thing? I had to add these lines to my baremetal (Ultibo) project's config.txt as I had clicks in my audio (driven by my own driver)

Code: Select all

#Without these setting there are clicks in the audio.

disable_pvt=1
force_turbo=1

LdB
Posts: 1478
Joined: Wed Dec 07, 2016 2:29 pm

Re: Reserving cores for bare-metal programs - interupts

Tue Aug 13, 2019 12:51 am

pik33 wrote:
Mon Aug 12, 2019 6:42 pm
Isn't it possible that the reason of "interrupts" is the memory refresh and/or cpu freq switching thing?
task switcher discussed above = your "memory refresh and/or cpu freq switching thing" ... so yes we are talking about same thing
The code above should stop the task switcher if linux allows access ... the poster said it didn't throw an error so it is likely it does.
pik33 wrote:
Mon Aug 12, 2019 6:42 pm
#Without these setting there are clicks in the audio.
disable_pvt=1
force_turbo=1
The first line is about DMA access which if isocpu is doing it's thing should not be on the core and the second line simply increases the speed of the cpu.

trejan
Posts: 1202
Joined: Tue Jul 02, 2019 2:28 pm

Re: Reserving cores for bare-metal programs - interupts

Tue Aug 13, 2019 1:13 am

disable_pvt does nothing now and isn't recognised as an option if you run "vcgencmd get_config disable_pvt". The firmware was changed ages ago so it doesn't have the pauses anymore.

Return to “Bare metal, Assembly language”