PWM peripheral bus error

[hooverphonique]

Hi all,

I'm aiming at using the pwm peripheral in FIFO serializer mode to generate a stepping pulse train for a stepper motor driver, where I have control over the number of steps/pulses generated. My project is a Raspbian userland process for the Pi 3B+/4B.

Generally it works, but I'm having trouble with bus errors (indicated by the BERR bit in the STA register), and as far as I can diagnose, this sometimes seems to cause my writes to pwm registers to be ignored/skipped (mainly the CTL register).

I have been unable to find any information on the BERR condition except for the sentence in the peripheral datasheet:

BERR sets to high when an error has occurred while writing to registers via APB. This may happen if the bus tries to write successively to same set of registers faster than the synchroniser block can cope with. Multiple switching may occur and contaminate the data during synchronisation. Software should clear this bit by writing 1. Writing 0 to this bit has no effect.

This, however, is not really helpful regarding how to avoid bus errors, or what mitigation to take if they occur. I see other pwm code/examples sprinkle small delays all over the code between register writes - is this related to BERR? Surely there must be an official method to this other than arbitrary delays here and there?

[joan]

It's not clear if you have added delays and the problem wasn't solved.

[hooverphonique]

It's not clear if you have added delays and the problem wasn't solved.

I tried adding delays after writing some registers, and it seems the bus errors get less frequent, but I was looking for a 'proper solution'. Is it really the case that there's no Broadcom description/code that deals with this?

If delays are the 'proper solution', what delay times do I need between which register accesses, to be sure it always works?

[joan]

I am not aware of any documented guidance. Personally I just search for similar code if I have a problem and see if others have a solution. My code seem to add delays after writes to the PWM control, data, and range registers.

[jdb]

What clock speed are you using to drive the PWM peripheral? I would imagine stepper motors are driven at kHz rates.

[hooverphonique]

Ok, thanks.

This simple code

Code: Select all

	// pwm_base mapped through mmap()
	uint32_t volatile * const PWMSTA = pwm_base + (0x04/4);
	uint32_t volatile * const PWMCTL = pwm_base + (0x00/4);

	*PWMSTA = 0x1FC; // clear error flags
	usleep(0); // sleep A
	printf("PWMSTA 0: 0x%08X\n", *PWMSTA); // no BERR here

	*PWMCTL = 0;
	usleep(400); // sleep B
	printf("PWMSTA 1: 0x%08X\n", *PWMSTA); // no BERR here

	*PWMCTL = (1<<5)|(1<<1)|(1<<0);	// USEF1,MODE1,PWEN1
	usleep(0); // sleep C

	printf("PWMSTA 2: 0x%08X\n", *PWMSTA); // BERR set here if sleep B < 350us

on a Pi 3B+ often shows BERR set in the "PWMSTA 2" statement, unless "sleep B" is >350 microseconds (the other two sleeps have no effect no matter the delay). A delay somewhere above 300 microseconds seems excessive though (and 3 times larger than what I've seen in other code), so maybe BERR sometimes sets without any actual write errors, but if that's the case, it's hard to know which delays, and where, makes it error proof.

[joan]

There won't be many people working with the PWM registers. It might be worth asking on the bare metal forum as they may have missed it here.

[jdb]

Please post the full code you are using to drive the PWM peripheral.

[hooverphonique]

Please post the full code you are using to drive the PWM peripheral.

This is the full code I'm using to explore this issue:

Code: Select all

#include <stdio.h>
#include <stdint.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/mman.h>
#include <string.h>
#include <bcm_host.h>

static __off_t peri_phys_base_;

#define PWM_BASE			(0x20C000)
#define MAP_BLOCK_SIZE		(4*1024)

typedef volatile uint32_t bcm_peri_reg_t;

bcm_peri_reg_t * pwm_base;

bcm_peri_reg_t * map(int fd, __off_t offset)
{
	void *map = mmap(
			NULL,
			MAP_BLOCK_SIZE,       //Map length
			PROT_READ|PROT_WRITE,// Enable reading & writting to mapped memory
			MAP_SHARED,       //Shared with other processes
			fd,           //File to map
			peri_phys_base_ + offset  //Offset to hardware  peripheral
	);

	if (map == MAP_FAILED) {
		printf("Failed mapping offset 0x%lX (%s).\n", offset, strerror(errno));
		close(fd);
		exit(-1);
	}

	return (bcm_peri_reg_t *)map;
}

void Setup()
{
	int mem_fd;
	if ((mem_fd = open("/dev/mem", O_RDWR | O_SYNC | O_CLOEXEC) ) < 0) {
		perror("Failed to open /dev/mem");
		exit(-1);
	}

	peri_phys_base_ = bcm_host_get_peripheral_address();

	pwm_base = map(mem_fd, PWM_BASE);

	close(mem_fd); //No need to keep mem_fd open after mmap
}

int main(int argc, char* argv[])
{
	Setup();

	uint32_t volatile * const PWMSTA = pwm_base + (0x04/4);
	uint32_t volatile * const PWMCTL = pwm_base + (0x00/4);

	*PWMSTA = 0x1FC; // clear error flags
	usleep(0); // sleep A
	printf("PWMSTA 0: 0x%08X\n", *PWMSTA); // no BERR here

	*PWMCTL = 0;
	usleep(400); // sleep B
	printf("PWMSTA 1: 0x%08X\n", *PWMSTA); // no BERR here

	*PWMCTL = (1<<5)|(1<<1)|(1<<0);	// USEF1,MODE1,PWEN1
	usleep(0); // sleep C

	printf("PWMSTA 2: 0x%08X\n", *PWMSTA); // BERR set here if sleep B < 350us

	return 0;
}

Buildable using "gcc -o pwmberr -I/opt/vc/include -L/opt/vc/lib -lbcm_host -Wall pwmberr.c"

[hooverphonique]

There won't be many people working with the PWM registers. It might be worth asking on the bare metal forum as they may have missed it here.

Maybe you're right.. It does seem like the "peripherals" in the forum title doesn't refer to the classic meaning (embedded peripheral), but external hardware.

[jdb]

That code doesn't set up the clock to the PWM peripheral. If the clock is not running, you won't get any output at all.

As far as I know, the firmware won't touch the PWM clock unless you request onboard audio output, at which point it would get initialised at 100MHz - far too fast for a stepper motor drive.

Synchronisation of register writes usually requires a handshake type interface, where a write in the system clock domain crosses into the PWM clock domain and is acknowledged. A 300uS sleep being neccessary between writes to PWMCTL implies you are driving a slow clock into PWM.

What's the output of "vcgencmd measure_clock pwm"?

[hooverphonique]

That code doesn't set up the clock to the PWM peripheral. If the clock is not running, you won't get any output at all.

As far as I know, the firmware won't touch the PWM clock unless you request onboard audio output, at which point it would get initialised at 100MHz - far too fast for a stepper motor drive.

Yes, when running the code posted above, the divisor is set to 4095 and clock src to oscillator (19.2/54MHz) by a different piece of software, I just didn't want to have a lot of "irrelevant" stuff in the bare bones BERR investigation. The "full" stepping application generally works ok, just not always when BERR occurs (because a write to PWM CTL is skipped, I suppose).
I have "dtparam=audio=off" in "/boot/config.txt", by the way.

Synchronisation of register writes usually requires a handshake type interface, where a write in the system clock domain crosses into the PWM clock domain and is acknowledged. A 300uS sleep being neccessary between writes to PWMCTL implies you are driving a slow clock into PWM.

This is interesting - where is this interface on the BCM283x then, if it even has one?
Does the necessary sleep length only depend on the pwm input clock src frequence, or also on the requested divisor?

What's the output of "vcgencmd measure_clock pwm"?

Pi 3B+: "frequency(25)=5000"
Pi 4B: "frequency(25)=0"

[jdb]

Synchronisation of register writes usually requires a handshake type interface, where a write in the system clock domain crosses into the PWM clock domain and is acknowledged. A 300uS sleep being neccessary between writes to PWMCTL implies you are driving a slow clock into PWM.

This is interesting - where is this interface on the BCM283x then, if it even has one?
Does the necessary sleep length only depend on the pwm input clock src frequence, or also on the requested divisor?

Clock-crossing logic is integral to any piece of hardware with multiple clock domains. It is usually automatic but with constraints on stability of signals held in one domain crossing to another. A typical constraint is that you must wait for 2 destination clock cycles + 1 source clock cycle for the destination to update successfully, but I don't know the particular method used in the PWM block so this may be inaccurate.

What's the output of "vcgencmd measure_clock pwm"?

Pi 3B+: "frequency(25)=5000"
Pi 4B: "frequency(25)=0"

A 5kHz pwm clock ties up well with the 300uS "wait" required in your program between two successive writes to PWMCTL (it's 1.5 destination clock cycles). Bus latencies and processing time will extend the second write after the 300uS delay such that you probably no longer hit the case where the clock-crossing logic reports an error.

For every non-fifo register write I would recommend waiting for 2,000,000/(pwm_clk_in_hz) microseconds, so it guarantees that two destination clock edges occur between register writes.

[hooverphonique]

For every non-fifo register write I would recommend waiting for 2,000,000/(pwm_clk_in_hz) microseconds, so it guarantees that two destination clock edges occur between register writes.

Thanks for that explanation - this also matches with my observation that I can mostly get away with waiting just 100+ us on the Pi4 at divisor 4095 (due to its faster oscillator).

Since I only write non-fifo registers while setting things up (i.e. not during stepping), I will probably just set a fixed delay of 427 (2,000,000*4095/19200000) microsecs to match the slowest case for the PI 3B+.

Just out of curiosity, how do pwm register reads work - they must be doing the same domain crossing, so do they just stall the cpu somehow?

[jdb]

Reads either stall the access bus or (more likely) just sample the PWM domain signal and provide a slightly delayed version of it. The consequence of the second implementation is that you can write a register and read it back immediately, but you will get stale data in the read.

If the bus is stalled on reads then reading in a loop for 1 second will complete a variable number of times, dependent on PWM clock speed.