Page 1 of 1

prevent crashes from unaligned memory accesses when porting code

Posted: Wed May 09, 2018 7:55 pm
by mattgrum
I'm porting some C code to the Pi 3 baremetal, and I have found that reading/writing structs from unaligned memory addresses causes a crash.

For example the following will crash:

Code: Select all

typedef struct {
   short a;
   short b;
   short c;
} S;

S array[2];
S s = {1,2,3};
array[1] = s;
This can be fixed by adding __attribute__((aligned(4)))

However this would require a lot of changes as there are many structs and they are read from binary files etc. If I don't care about performance is there a way to make the compiler generate extra code for safe unaligned memory accesses?

I'm using arm-none-eabi-gcc.exe (GNU Tools for Arm Embedded Processors 7-2017-q4-major) 7.2.1 20170904 with options:

-march=armv7-a -marm -mfpu=neon-vfpv4 -mfloat-abi=hard -fsigned-char


Re: prevent crashes from unaligned memory accesses when porting code

Posted: Thu May 10, 2018 8:59 am
by Schnoogle
Hi mattgrum,

you might want to tell the CPU that unaligned memory access is fine. This could be done in your boot asm code like this:

Code: Select all

	mrc 	p15, 0, r0, c1, c0, 0
	ldr		r1, =#0xfffffffd
	and		r0, r0, r1 					/* enable unaligned access */	
	mcr		p15, 0, r0, c1, c0, 0
Than everything should work fine :)

BR Schnoogle

Re: prevent crashes from unaligned memory accesses when porting code

Posted: Fri May 11, 2018 7:12 pm
by dwelch67
or fix the code such that it doesnt misuse structures like that.

Re: prevent crashes from unaligned memory accesses when porting code

Posted: Fri May 11, 2018 7:28 pm
by mattgrum
Thanks for your response. I've added that code to my startup but unfortunately the problem persists. I can read unaligned structs from an array but if I write one the system hangs.

Here is the disassembly of kernel.elf, the code appears as expected:

Code: Select all

kernel7.elf:     file format elf32-littlearm

Disassembly of section .text:

00008000 <_start>:
    8000:	ee110f10 	mrc	15, 0, r0, cr1, cr0, {0}
    8004:	e3e01002 	mvn	r1, #2
    8008:	e0000001 	and	r0, r0, r1
    800c:	ee010f10 	mcr	15, 0, r0, cr1, cr0, {0}
    8010:	f57ff06f 	isb	sy
Is there anywhere I can read more about what that piece of code is doing?

Re: prevent crashes from unaligned memory accesses when porting code

Posted: Fri May 11, 2018 8:38 pm
by LdB
Make sure you backup your code :-)

A brutal fix is to word substitute "typedef struct" with "typedef struct __attribute__((__packed__, aligned(1)))" in all your source code and live with the massive speed penalty. The compiler is forced to assume your structs are packed and can be byte aligned.

I know on Visual Studio you can do text substitutions on entire source code or directories and there are piles of command line programs that can do it.

Personally I wouldn't do it as it wouldn't take that long to go thru and do every struct properly and add proper alignments on them.

Re: prevent crashes from unaligned memory accesses when porting code

Posted: Sun Jul 01, 2018 8:49 am
by mattgrum
Thanks for the replies.

Firstly, I don't see how this constitutes "misusing structures" since this is perfectly valid C, and it's possible to generate correct machine code from it, which is the job of the compiler.

Unfortunately adding __attribute__((__packed__, aligned(1))) doesn't help, the code still crashes. Going through the code and making all the structs word aligned would also be a huge amount of work as a lot of the data is read directly from binary files in packed/unaligned form so I would have to rewrite a huge amount of code.

It really feels like the compiler has all the information it needs to generate correct code - it knows the [relative] address of a struct in memory and it knows how to read/write individual bytes of data so I'm sure I'm just missing an option, or need to configure the Pi for unaligned accesses.

Re: prevent crashes from unaligned memory accesses when porting code

Posted: Sun Jul 01, 2018 5:42 pm
by dwelch67
You are using structures across compile domains. Which lands in the "implementation defined" area of the language, which means there is no reason to expect it to work, nor continue to work (unless you stay within that domain). If you were using structs properly you wouldnt need to pack them, unless it is the 1970s and you are out of ram. Because the number of compilers (and targets) is decreasing this time bomb keeps being planted in more and more code, even library/chip vendor code. Because one interpretation of the standard happens to work by definition doesnt make it part of the standard, thats the problem. Anywhere you see implementation defined you should simply avoid that part of the language or add a fair amount of code to protect yourself from it, language portable code and/or code that fails to build or run during initial testing catching the problem very early in the development/porting cycle.

The processor has properly flagged the bug, making it easier to find, you should leave the exception enabled, put a handler there so that you can find the source of each instance, and repair the code.

Fixing the code and or making a tool to fix the binary files and put them in a portable format is the correct solution if this code is desired to have some longevity. The pain involved is a reminder to not do this again, ideally you want the person who created the pain to feel the pain, sadly that is not always the case.

Most folks just pass the buck onto someone else or save it for the future hoping the software will die and become obsolete by pack the structs and/or disabling unaligned accesses. And yes packing the structs doesnt always work, had to deal with someones code that had this exact problem recently. My hacked up solution was pure evil...We were literally down to a handful of bytes left for the program, to the point that when we added code we had to make the messages in the printfs smaller. Passing the buck to a re-write in the future.

Re: prevent crashes from unaligned memory accesses when porting code

Posted: Sun Jul 01, 2018 7:46 pm
by dwelch67
I assume your situation is worse than mine was. the person creating the pain was no longer with the company, and we were holding off a re-write.

The structure had pointers to other structures. The desire was to have a mechanism for an alternate boot config without re-compiling the bootloader every time. No more room for additional if-then-else structures. The structures were such that there were pointers to other structures thus the problem, re-arranging the items in the structure would not fix it, stdint would not fix it. the pointer from one structure to another so that the top level functions only needed to pass one structure (I wont get into that now) were the rub. 32 bit pointers on an ARM11. 64 bit pointers on the host computer needing to make an alternate structure. no more room for program space, got rid of one if-then-else we didnt use freeing up just enough bytes to add new-if-then-else code. can map the flash into address space so if the alternate was built such that the code could "simply" point at it then done. normal point at compiled in else point at alternate. So the crossing compile domains was to try to make an x86, 64 bit, host based tool. Fill in the desired structure, but the tool is to make the flash binary (think file) for the bootloader. On the x86 64 bit domain you get 64 bit pointers in the structure, arm 32 bit domain 32 bit, multiple layers of pointers to structs within the structs. My temp solution was not to compile the alternate struct using the x86 tools, compile and link with the 32 bit tools extract (or just build the whole binary) the relevant bytes and put them in flash, with the appropriate amount of size of struct, length, checksum, etc that you would do even if using the same tools on a different day against the same target or similar targets.

I suspect your issue is worse you are crossing perhaps 32 bit to 64 or vice versa and stdint tricks or other wont fix it? They are not there to fix such things btw. can have a struct with

uint8_t a;
uint16_t b;
uint32_t c;
uint64_t d;

and the compiler is perfectly free to allocate 4x64 bits and align each of those in a natural way within each 64 bit field, or even a 64 bit compiler can choose to say align those on 32 bit boundaries. so long as you stay within the compiler domain the language works you can pass structures around between functions, even ones in different source files, and it will work. Its when you cross to another compiler or version of compiler or same compiler (family) different target, where the implementation defined hits you. So I suspect you landed in one of these traps.

The ideal solution is to have a conversion tool in the original compile domain such that you can read the data into a struct then have an outgoing struct that is different, same element names perhaps but more portable

uint64_t a;
uint64_t b;
uint64_t c;
uint64_t d;

from which you could leave it like that in the final domain, or that domain could read it in with a higher chance of success into a portable struct then copy the items over into the desired struct to take advantage of clipping or memory savings or whatever.

if you cant do that then maybe the target domain needs padding added to the struct and maybe ifdefs in the shared source if any so that one domain adds padding for a short term hack to be able to read in the code without a complete re-write.

if packing worked then the quick fix would have been to disable alignment checking.

Re: prevent crashes from unaligned memory accesses when porting code

Posted: Sun Jul 01, 2018 7:48 pm
by dwelch67
so I didnt see my solution as bad actually I stayed within the toolchain, target, compile domain, etc...then the host program wrapped around that a checksum modification for validation of the data before saving it for the flash tool. someone else called it pure evil. when he had the same problem he chose to take the one specific structure and hand pick and place each item with a lot of code...functional, but painful and unmaintainable...

Re: prevent crashes from unaligned memory accesses when porting code

Posted: Wed Jul 04, 2018 10:04 am
by bzt
@dwelch67: you are absoutely right. I just want to add that in some cases avoiding packed struct is simply not an option. One I run into was the BPB structure of FAT. Without packed the compiler would allocate 64 bits for each field to satisfy alignment requirements. Needless to say that struct's fields wouldn't match the data read from the disk.

My solution is ugly as hell, but works, and does not need enabling unaligned access: I do an aligned read, shifted and masked the value to get the required byte. (Something I'd expect the compiler do for me in a strict-aligned domain. Maybe I expect too much...)