ARMv7 emulation on ARMv6: System Control Processor

[fd_]

Hi all,
I'm trying to write a kernel module that allows to run ARMv7 userspace code on an ARMv6 Raspberry Pi Zero. The kernel module registers an undefined instruction hook and parses ARMv7 instructions, manually executing the required register transfers etc. I'm hoping this would have as little overhead as possible, since I suspect using something like QEMU userspace emulation will be much too slow for any real use case.

For some instructions, such as MOVW and UBFX, this seems to work fine. However, I'm now seeing my hook being called for an MRRC instruction for coprocessor 15, which seems to be the System Control Coprocessor:

Code: Select all

ec510f1e = mrrc p15, #1, r0, r1, c14

According to the documentation of ARMv7 [1], register 14 of the System Control Coprocessor is for the Generic Timer. Interestingly, it seems like ARMv6's System Control Coprocessor [2] doesn't even have a register 14, and comparing all the other registers, they seem to largely differ from their ARMv7 counterparts. Yet, these differences are not mentioned in the "Migrating a software application from ARMv5 to ARMv7-A/R Application Note 425" [3] document.

Does anyone know how these System Control Coprocessor accesses could be emulated on ARMv6? Does ARMv6 have a Generic Timer? Why do userspace programs access the System Control Coprocessor anyway?

Thanks in advance!

[1]: http://infocenter.arm.com/help/index.js ... HBIDJ.html
[2]: http://infocenter.arm.com/help/index.js ... index.html
[3]: http://infocenter.arm.com/help/index.js ... GFCGC.html

[DougieLawson]

I don't have the first faintest idea why you think you'd want to do this. Why not simply buy a RPi4B 4GB and you can run ARM6, ARM7 and ARM8 code natively on the processor.

[fd_]

I don't have the first faintest idea why you think you'd want to do this. Why not simply buy a RPi4B 4GB and you can run ARM6, ARM7 and ARM8 code natively on the processor.

I have existing ARMv7 binaries (no source) that I'd like to run on a portable device with as little power as possible. The Pi Zero is the most portable and most low-power Pi, that's why using that would be nice. I did buy a Pi 3A+ that works fine, but seems to use about 2x as much power as the Zero.

Beside my specific application, given that there will likely never be a CPU upgrade to the Pi Zero, and the fact that ARMv6 support is getting dropped in major projects, I do think a performant ARMv7 emulation layer would be very useful. Like a hardware upgrade for the Pi Zero, but done in software.

[jamesh]

I don't have the first faintest idea why you think you'd want to do this. Why not simply buy a RPi4B 4GB and you can run ARM6, ARM7 and ARM8 code natively on the processor.

I have existing ARMv7 binaries (no source) that I'd like to run on a portable device with as little power as possible. The Pi Zero is the most portable and most low-power Pi, that's why using that would be nice. I did buy a Pi 3A+ that works fine, but seems to use about 2x as much power as the Zero.

Beside my specific application, given that there will likely never be a CPU upgrade to the Pi Zero, and the fact that ARMv6 support is getting dropped in major projects, I do think a performant ARMv7 emulation layer would be very useful. Like a hardware upgrade for the Pi Zero, but done in software.

Any CPU upgrade to the Zero would come with an increase in power consumption.....swings and roundabouts!

Interesting project, no idea if it's even possible.

[fd_]

Any CPU upgrade to the Zero would come with an increase in power consumption.....swings and roundabouts!

Interesting project, no idea if it's even possible.

I agree that even this "software CPU upgrade" would increase power consumption per executed program instruction, ie there'd be a performance decrease caused by the emulation. Still, if we can keep the performance at a reasonable level, this could make the Pi Zero useful for projects that have only modest requirements on the CPU speed, but require ARMv7.

AFAICT, the power required by the more capable Pi models doesn't just scale proportionally to the used processing power. They also need more power than Pi Zero even when just idling.

[jamesh]

AFAICT, the power required by the more capable Pi models doesn't just scale proportionally to the used processing power. They also need more power than Pi Zero even when just idling.

Depends on the peripherals attached. Put the Pi4 CPU on a Zero board and you still need a load more power though. IIRC, the CPU is the main draw, then ethernet etc.

[ejolson]

Hi all,
I'm trying to write a kernel module that allows to run ARMv7 userspace code on an ARMv6 Raspberry Pi Zero. The kernel module registers an undefined instruction hook and parses ARMv7 instructions, manually executing the required register transfers etc. I'm hoping this would have as little overhead as possible, since I suspect using something like QEMU userspace emulation will be much too slow for any real use case.

For some instructions, such as MOVW and UBFX, this seems to work fine. However, I'm now seeing my hook being called for an MRRC instruction for coprocessor 15, which seems to be the System Control Coprocessor:

Code: Select all

ec510f1e = mrrc p15, #1, r0, r1, c14

According to the documentation of ARMv7 [1], register 14 of the System Control Coprocessor is for the Generic Timer. Interestingly, it seems like ARMv6's System Control Coprocessor [2] doesn't even have a register 14, and comparing all the other registers, they seem to largely differ from their ARMv7 counterparts. Yet, these differences are not mentioned in the "Migrating a software application from ARMv5 to ARMv7-A/R Application Note 425" [3] document.

Does anyone know how these System Control Coprocessor accesses could be emulated on ARMv6? Does ARMv6 have a Generic Timer? Why do userspace programs access the System Control Coprocessor anyway?

Thanks in advance!

[1]: http://infocenter.arm.com/help/index.js ... HBIDJ.html
[2]: http://infocenter.arm.com/help/index.js ... index.html
[3]: http://infocenter.arm.com/help/index.js ... GFCGC.html

Thank you for working on this. It has the potential to increase the life expectancy of Zero hardware as software is tuned to run on the ARMv7. I'm sorry I don't have any insight on the control coprocessor, but wonder, what are such instructions doing in user-space code?

[fd_]

Thank you for working on this. It has the potential to increase the life expectancy of Zero hardware as software is tuned to run on the ARMv7. I'm sorry I don't have any insight on the control coprocessor, but wonder, what are such instructions doing in user-space code?

Thanks for the encouragement! However, I'm afraid without help from someone familiar with the ARM ISA, I won't get very far. I was hoping someone from the Foundation/Trading would be willing to help here. Probably I should ask in the bare-metal subforums?

ARMv7 is actually composed of multiple different instruction sets:

• The ARM ISA itself. The difference between ARMv7 and ARMv6 is minimal here, and I'm confident this would be something I'm capable of implementing.
• Thumb. ARMv6 only supports 16-bit Thumb-1 instructions, but ARMv7 added basically a 32-bit Thumb-2 equivalent for most ARM instructions, so that's a lot of Thumb-2 instructions to emulate. However, it seems gcc doesn't produce Thumb-2 instructions unless explicitly enabled, which could mean they aren't too common in the wild. I might be wrong about that.
• VFP. The ARM itself doesn't have instructions for floating point operations. These are done by a separate coprocessor, the VFP (Vector Floating Point). The ARMv6 has a VFPv2 coprocessor with 16 64-bit registers, while ARMv7's VFPv3 has 32 64-bit registers. Some binaries are compiled with VFPv3-d16 though, which only uses 16 of these registers. VFPv3 also has a few more instructions than VFPv2, which probably could be emulated.
• NEON. This is a coprocessor for SIMD operations. The ARMv6 in the Pi Zero doesn't have it, so these instructions would (IMHO) be the hardest to emulate. However, NEON has to be explicitly enabled on compilation, so should generally not be used in your average ARMv7 binary.

So, realistically, this is not something I could implement myself in its entirety. Also, as mentioned, I already purchased a Pi 3A+ essentially so I don't have to tackle this undertaking. Ideally, this would be a topic for a Bachelor's or Master's thesis for a computer science student (Unfortunately, I already started mine on a different topic a few months back).

Another (likely illusory) scenario would be the Pi Foundation/Trading tackling this project as a means to extend the lifespan of the Pi Zero. It's likely orders of magnitudes cheaper to implement than developing new silicon suitable for an updated Pi Zero.

[jamesh]

Another (likely illusory) scenario would be the Pi Foundation/Trading tackling this project as a means to extend the lifespan of the Pi Zero. It's likely orders of magnitudes cheaper to implement than developing new silicon suitable for an updated Pi Zero.

Sorry....not something we would have the time to do. Or, actually, the experience, I suspect. Zero's sell pretty well already, and most software that is run on them people have source code for so they can recompile if they need to.

I can move this entire thread to bare metal if you wish.

[ejolson]

Another (likely illusory) scenario would be the Pi Foundation/Trading tackling this project as a means to extend the lifespan of the Pi Zero. It's likely orders of magnitudes cheaper to implement than developing new silicon suitable for an updated Pi Zero.

Sorry....not something we would have the time to do. Or, actually, the experience, I suspect. Zero's sell pretty well already, and most software that is run on them people have source code for so they can recompile if they need to.

I can move this entire thread to bare metal if you wish.

I think the main difficulty with the Zero right now is recent Java VMs. If a kernel module worked, would the maintainers of Raspbian consider including it in the repository so some of the currently broken binaries would run on the Zero?

[plugwash]

[*] Thumb. ARMv6 only supports 16-bit Thumb-1 instructions, but ARMv7 added basically a 32-bit Thumb-2 equivalent for most ARM instructions, so that's a lot of Thumb-2 instructions to emulate. However, it seems gcc doesn't produce Thumb-2 instructions unless explicitly enabled, which could mean they aren't too common in the wild. I might be wrong about that.

Unfortunately for you many if not all major linux distros did explicitly enable it.

[trejan]

readelf on a Debian Buster armhf binary shows this:

Code: Select all

  Tag_CPU_name: "7-A"
  Tag_CPU_arch: v7
  Tag_CPU_arch_profile: Application
  Tag_ARM_ISA_use: Yes
  Tag_THUMB_ISA_use: Thumb-2
  Tag_FP_arch: VFPv3-D16
  Tag_ABI_PCS_wchar_t: 4
  Tag_ABI_FP_rounding: Needed
  Tag_ABI_FP_denormal: Needed
  Tag_ABI_FP_exceptions: Needed
  Tag_ABI_FP_number_model: IEEE 754
  Tag_ABI_align_needed: 8-byte
  Tag_ABI_align_preserved: 8-byte, except leaf SP
  Tag_ABI_enum_size: int
  Tag_ABI_VFP_args: VFP registers
  Tag_CPU_unaligned_access: v6

[jamesh]

Another (likely illusory) scenario would be the Pi Foundation/Trading tackling this project as a means to extend the lifespan of the Pi Zero. It's likely orders of magnitudes cheaper to implement than developing new silicon suitable for an updated Pi Zero.

Sorry....not something we would have the time to do. Or, actually, the experience, I suspect. Zero's sell pretty well already, and most software that is run on them people have source code for so they can recompile if they need to.

I can move this entire thread to bare metal if you wish.

I think the main difficulty with the Zero right now is recent Java VMs. If a kernel module worked, would the maintainers of Raspbian consider including it in the repository so some of the currently broken binaries would run on the Zero?

Java is likely too slow on a Zero to be of much use anyway.

If someone could get something like this to work, it's certainly possible it could be added. However, the fact that there isn't anything already like it might be an indicator is very difficult to make work. It wouldn't be a simple kernel module.

[dickon]

The performance hit is likely to be horrendous, too. Even excluding the decode and synthesis stages, you nobble the D-cache by polluting it with instruction data. It's one of the things ISTR Acorn muttered about when eventually implementing an FPU on whichever of the ARM chips it was they introduced the thing on.

[helo]

Um, get an armv6 toolchain and compile your binary.

Sounds like some bull that apple sould hire someone to work on for anti-competitive OSX-on-PI with a shaky and dysfunctional emulation scheme.

Compile gcc+binugils for armv6-linuz, its plumbing not rocket sciencs

[cleverca22]

Um, get an armv6 toolchain and compile your binary.

Sounds like some bull that apple sould hire someone to work on for anti-competitive OSX-on-PI with a shaky and dysfunctional emulation scheme.

Compile gcc+binugils for armv6-linuz, its plumbing not rocket sciencs

I have existing ARMv7 binaries (no source) that I'd like to run on a portable device with as little power as possible. The Pi Zero is the most portable and most low-power Pi, that's why using that would be nice. I did buy a Pi 3A+ that works fine, but seems to use about 2x as much power as the Zero.

not an option

Code: Select all

ec510f1e = mrrc p15, #1, r0, r1, c14

According to the documentation of ARMv7 [1], register 14 of the System Control Coprocessor is for the Generic Timer. Interestingly, it seems like ARMv6's System Control Coprocessor [2] doesn't even have a register 14, and comparing all the other registers, they seem to largely differ from their ARMv7 counterparts. Yet, these differences are not mentioned in the "Migrating a software application from ARMv5 to ARMv7-A/R Application Note 425" [3] document.

Does anyone know how these System Control Coprocessor accesses could be emulated on ARMv6? Does ARMv6 have a Generic Timer? Why do userspace programs access the System Control Coprocessor anyway?

the pi0/pi1 doesnt have a generic arm timer, so you have to emulate all of that based on the free-running timer (1mhz clock) on the VPU timer, at ST_CHI and ST_CLO, or the generic timer subsystem on linux