rgbstar
Posts: 5
Joined: Fri Jul 19, 2019 3:22 pm

swp or ldrex/strex issue on RPI2

Mon Jul 22, 2019 10:01 pm

It is about a spin lock code for RPI2 (only one core is enabled), the lock pointer is passed in and swap with 0 and return the pointed value.
but the code using "swp" doesn't work on RPI2 (not supported?, compiler warning, etc),
so I am trying to use LDREX/STREX as replacement, it also doesn't work, seems a dead lock.

spin lock using swp:

Code: Select all

try_lock:
    mov     r1, #0
    swp     r2, r1, [r0]
    mov     r0, r2
    blx     lr 
spin lock using LDREX/STREX:

Code: Select all

try_lock:
try_swap:
	ldrex r2,[r0]
	mov   r1, #0
	strex r3,r1,[r0]
	cmp r3,#0
	bne try_swap

    mov     r0, r2
    blx     lr 

The code without using swp or ledrex/strex works, but has concurrency issue.

Code: Select all

try_lock:
    mov     r1, #0
    ldr   r2,[r0]
    str   r1,[r0]
    mov 	r0, r2
    blx     lr 
What I am missing here? Thanks in advance.

User avatar
DavidS
Posts: 4334
Joined: Thu Dec 15, 2011 6:39 am
Location: USA
Contact: Website

Re: swp or ldrex/strex issue on RPI2

Mon Jul 22, 2019 10:25 pm

rgbstar wrote:
Mon Jul 22, 2019 10:01 pm
It is about a spin lock code for RPI2 (only one core is enabled), the lock pointer is passed in and swap with 0 and return the pointed value.
but the code using "swp" doesn't work on RPI2 (not supported?, compiler warning, etc),
so I am trying to use LDREX/STREX as replacement, it also doesn't work, seems a dead lock.

spin lock using swp:

Code: Select all

try_lock:
    mov     r1, #0
    swp     r2, r1, [r0]
    mov     r0, r2
    blx     lr 
spin lock using LDREX/STREX:

Code: Select all

try_lock:
try_swap:
	ldrex r2,[r0]
	mov   r1, #0
	strex r3,r1,[r0]
	cmp r3,#0
	bne try_swap

    mov     r0, r2
    blx     lr 

The code without using swp or ledrex/strex works, but has concurrency issue.

Code: Select all

try_lock:
    mov     r1, #0
    ldr   r2,[r0]
    str   r1,[r0]
    mov 	r0, r2
    blx     lr 
What I am missing here? Thanks in advance.
Firstly unless you have an early model Raspberry Pi 2B the SWP instruction will not be present (it was dropped in the ARMv8, and broke a lot of software in the process).

For a try lock you may want to go with something like:

Code: Select all

TryLock:
TrySwap:
  MOV   R1,#0
  LDREX R2,[r0]
  CMP   R2, #0
  STREXEQ R3,R1,[r0]
  CMPEQ R3,#0
  BNE   TrySwap
  MOV R0,R2
  MOV   R15,R14
You may note that the STREX is only executed if the loaded value is 0 in this case, much better way around it.

Also note that I used a non-thumb releated return instruction.

BLX exchanges R15 (PC) with the specified register setting the Thumb flag based on the value of the least significant bit in the other register, where a MOV R15, R14 just puts the address in the Link Register into the Program Counter, causing a return.
RPi = The best ARM based RISC OS computer around
More than 95% of posts made from RISC OS on RPi 1B/1B+ computers. Most of the rest from RISC OS on RPi 2B/3B/3B+ computers

rgbstar
Posts: 5
Joined: Fri Jul 19, 2019 3:22 pm

Re: swp or ldrex/strex issue on RPI2

Tue Jul 23, 2019 12:15 am

Thanks, i got strange results, once TryLock(&lock) got called, it never returns anything. Then I remove the "BNE TrySwap", it still never return anything, then I replace the "MOV R15,R14" to "blx lr", it also never returns.

Then I tried to put a small blink helper function to blink LED at pin 6. The LED keep blinking, i.e.., the function keeps calling itself without loop. By the way, I print the value that goes into the function at [r0] is 1. The lock = 1 means available (so I changed CMP R2,#1, i also tried CMP R2 = #0, still a dead loop). The RPI 2 version is 1.1. Is it because only 1 core is running or newer raspian did something to the processor during start up?

EDIT 2: I did more test, if I put blink before LDREX, it blink once as expected. If I put blink between LDREX and STREX, it never blinks, it seems CPU did something to make sure the ldrex/strex is an atomic operation and doesn't allow blink to run. If I put blink behind STREX, it blinks forever, it is probably indicating the ldrex/strex has failed. I did some more search, it seems if there is no MMU to mark the memory as Normal, ldrex/strex will fail. https://www.raspberrypi.org/forums/view ... p?t=228444 and https://stackoverflow.com/questions/260 ... data-abort

Code: Select all

TryLock:
TrySwap:
  MOV   R1,#0
  LDREX R2,[r0]
  CMP   R2, #1
  STREXEQ R3,R1,[r0]
  CMPEQ R3,#0
  //BNE   TrySwap
  bl blink
  MOV R0,R2
  MOV   R15,R14
Here is my debugging helper function, the GPIO pin 6 is set as output mode before the spin lock is created.

Code: Select all

blink:
	ldr r0,=0x3F200000 
	mov r1,#1
	lsl r1,#18
	str r1,[r0,#0]

	// turn LED on
	ldr r0,=0x3F200000
	mov r1,#1
	lsl r1,#6
	str r1,[r0,#28]
	
	mov r2,#0x3F0000
_wait1:
	sub r2,#1
	cmp r2,#0
	bne _wait1

	// turn LED off
	ldr r0,=0x3F200000
	mov r1,#1
	lsl r1,#6
	str r1,[r0,#40]

	mov r2,#0x3F0000
_wait2:
	sub r2,#1
	cmp r2,#0
	bne _wait2

	// turn LED on
	ldr r0,=0x3F200000
	mov r1,#1
	lsl r1,#6
	str r1,[r0,#28]

	blx lr
Last edited by rgbstar on Tue Jul 23, 2019 1:11 am, edited 1 time in total.

User avatar
DavidS
Posts: 4334
Joined: Thu Dec 15, 2011 6:39 am
Location: USA
Contact: Website

Re: swp or ldrex/strex issue on RPI2

Tue Jul 23, 2019 1:07 am

I am a bit confused, what OS is this running on?

I had assumed bare metal. I may be wrong based on your talk about Raspbian.

Though if bare metal, how do you do your setup of the CP15 feature registers?
RPi = The best ARM based RISC OS computer around
More than 95% of posts made from RISC OS on RPi 1B/1B+ computers. Most of the rest from RISC OS on RPi 2B/3B/3B+ computers

rgbstar
Posts: 5
Joined: Fri Jul 19, 2019 3:22 pm

Re: swp or ldrex/strex issue on RPI2

Tue Jul 23, 2019 1:15 am

thanks, I used Raspian (2018 Nov) to create the booting SD card, the rest is bare metal (I was following the OS exercise: https://jsandler18.github.io/tutorial/locks.html). I may have found the issue, I edited my previous post, please see "EDIT 2" in my previous post, it may be related to that without MMU to mark the memory as Normal, LDREX/STREX will fail.

User avatar
DavidS
Posts: 4334
Joined: Thu Dec 15, 2011 6:39 am
Location: USA
Contact: Website

Re: swp or ldrex/strex issue on RPI2

Tue Jul 23, 2019 6:18 pm

rgbstar wrote:
Tue Jul 23, 2019 1:15 am
thanks, I used Raspian (2018 Nov) to create the booting SD card, the rest is bare metal (I was following the OS exercise: https://jsandler18.github.io/tutorial/locks.html). I may have found the issue, I edited my previous post, please see "EDIT 2" in my previous post, it may be related to that without MMU to mark the memory as Normal, LDREX/STREX will fail.
Well that likely gives a good explination. I have never tried to see what LDREX/STREX/CLREX would do on a system witout the MMU, as they are not so much use before getting to the point of having a working MMU. If multiple threads are running on a single core the MMU should be up and running ideally, if multiple cores are running the MMU should be up and running, if you are requiring mutual exclusion between an inturupt handler and something that can be inturupted by the handler the MMU should already be up and running.

Also for actual product implementation code it would be recommended to have an exit condition for failure to obtain the lock, regardless of why it failed (lock already set, or instruction failure), using CLREX before exiting the procedure.
RPi = The best ARM based RISC OS computer around
More than 95% of posts made from RISC OS on RPi 1B/1B+ computers. Most of the rest from RISC OS on RPi 2B/3B/3B+ computers

Thalhammer
Posts: 15
Joined: Fri Aug 03, 2012 6:34 pm

Re: swp or ldrex/strex issue on RPI2

Wed Jul 24, 2019 8:14 pm

You need to enable the MMU (or to be more precise the data and instruction caches, which in turn require the mmu).
What happens here is that if you call ldrex or strex without enabled caches it jumps to the data abort handler which unless you set one up correctly will result in a deadloop. This had me struggeling for quite some time as well and is the only reason I choose to enable the mmu (with a simple 1:1 mapping and only some protection magic).

* I am talking Zero W which is Armv6, but I assume yours is similar.

dwelch67
Posts: 961
Joined: Sat May 26, 2012 5:32 pm

Re: swp or ldrex/strex issue on RPI2

Tue Aug 13, 2019 8:25 pm

ldrex/strex are not meant to be a swp replacement. They are for shared access to a resource in a multi-processor configuration. Not required to be implemented in uniprocessor implementation, but you are not talking about the arm11/pi1. The software side of arms documentation implied this but the hardware side paints the picture. The cpuid registers tell you if swp is implemented for that core, if implemented use them and get buy for a while longer. (and on some of the mcu cortex-m cores they are implemented in a way that always passes for compatibiity, just like they have wfi/wfe that are nops)

cache on, which means mmu on, and that section enabled for caching, you are within ARM logic and their logic is implemented for their cache and does what one would hope despite being with L1 within a uniprocessor domain. it actually keeps track and provides status. once you are into broadcom's domain the its however they implemented it. their cache is not the only factor is the memory controller in front of it on the edge of the arm. if the cache/mmu are off then you go directly into broadcoms logic, easier to test the logic that way if you can get away with it but you may need the mmu on anyway for the exclusive accesses to pass through. I stopped researching this a number of cores ago.

ARM may have changed their definition for the use of ldrex/strex since I last researched it but I dont think the axi bus has changed. for it to work at all you have to have a separate ID go out on the bus for the logic to determine that between your instructions no other id has interfered with that address. if the same cpu core always generates the same ID then it simply wont work.

HIGHLY recommended, that you do your ldrex/strex research in baremetal and not modifying or working within an operating system. there is nothing an operating system can do that you cant do in baremetal you can configure what you need. these instructions should not cause an abort any more than an ldr/str would the whole point is to determine if someone else has accessed a specific address between your accesses of a specific address. you first have to get the ldrex/strex to go out as exclusive accesses which may or may not involve going through the mmu and for testing the (ARM L1) cache/logic you want the cache enabled, but it is possible to get an eviction and miss so you also need to know what the chip vendors logic does.

A chip I worked on followed the uniprocessor comment and we ran into ldrex/strex issues in linux code because of the assumption that ldrex/strex replaces swp. Further supported by finding a comment somewhere, page, blog, something from a ti engineer that they had done the same thing. The quick and dirty is if exclusive then return exokay else the normal okay. you cant tell one thread from another so there is no need for more logic. For a while then I would dabble on other systems, probably the pi1 as well, seeing what I can see.

It is trivial to test simply put a non exclusive in between or put an access from another id in between. and see if the logic catches it or not. From there build your lock system around what you learned FOR THAT PLATFORM. Repeat for every platform meaning every CHIP not necessarily core that you plan to support.

pica200
Posts: 152
Joined: Tue Aug 06, 2019 10:27 am

Re: swp or ldrex/strex issue on RPI2

Sat Aug 17, 2019 6:22 pm

It's noteworthy that you don't need ldrex/strex on a single core system. It's enough to temporarily disable interrupts (cpsid i) which is much more lightweight than a ldrex/strex loop aswell.

Return to “Bare metal, Assembly language”