fd_
Posts: 93
Joined: Thu Oct 25, 2018 7:35 am

ARMv7 emulation on ARMv6: System Control Processor

Wed Apr 22, 2020 7:38 am

Hi all,
I'm trying to write a kernel module that allows to run ARMv7 userspace code on an ARMv6 Raspberry Pi Zero. The kernel module registers an undefined instruction hook and parses ARMv7 instructions, manually executing the required register transfers etc. I'm hoping this would have as little overhead as possible, since I suspect using something like QEMU userspace emulation will be much too slow for any real use case.

For some instructions, such as MOVW and UBFX, this seems to work fine. However, I'm now seeing my hook being called for an MRRC instruction for coprocessor 15, which seems to be the System Control Coprocessor:

Code: Select all

ec510f1e = mrrc p15, #1, r0, r1, c14
According to the documentation of ARMv7 [1], register 14 of the System Control Coprocessor is for the Generic Timer. Interestingly, it seems like ARMv6's System Control Coprocessor [2] doesn't even have a register 14, and comparing all the other registers, they seem to largely differ from their ARMv7 counterparts. Yet, these differences are not mentioned in the "Migrating a software application from ARMv5 to ARMv7-A/R Application Note 425" [3] document.

Does anyone know how these System Control Coprocessor accesses could be emulated on ARMv6? Does ARMv6 have a Generic Timer? Why do userspace programs access the System Control Coprocessor anyway?

Thanks in advance!

[1]: http://infocenter.arm.com/help/index.js ... HBIDJ.html
[2]: http://infocenter.arm.com/help/index.js ... index.html
[3]: http://infocenter.arm.com/help/index.js ... GFCGC.html

User avatar
DougieLawson
Posts: 40828
Joined: Sun Jun 16, 2013 11:19 pm
Location: A small cave in deepest darkest Basingstoke, UK
Contact: Website Twitter

Re: ARMv7 emulation on ARMv6: System Control Processor

Wed Apr 22, 2020 10:23 am

I don't have the first faintest idea why you think you'd want to do this. Why not simply buy a RPi4B 4GB and you can run ARM6, ARM7 and ARM8 code natively on the processor.
Any language using left-hand whitespace for syntax is ridiculous

Any DMs sent on Twitter will be answered next month.
Fake doctors - are all on my foes list.

Any requirement to use a crystal ball or mind reading will result in me ignoring your question.

fd_
Posts: 93
Joined: Thu Oct 25, 2018 7:35 am

Re: ARMv7 emulation on ARMv6: System Control Processor

Wed Apr 22, 2020 10:27 am

DougieLawson wrote:
Wed Apr 22, 2020 10:23 am
I don't have the first faintest idea why you think you'd want to do this. Why not simply buy a RPi4B 4GB and you can run ARM6, ARM7 and ARM8 code natively on the processor.
I have existing ARMv7 binaries (no source) that I'd like to run on a portable device with as little power as possible. The Pi Zero is the most portable and most low-power Pi, that's why using that would be nice. I did buy a Pi 3A+ that works fine, but seems to use about 2x as much power as the Zero.

Beside my specific application, given that there will likely never be a CPU upgrade to the Pi Zero, and the fact that ARMv6 support is getting dropped in major projects, I do think a performant ARMv7 emulation layer would be very useful. Like a hardware upgrade for the Pi Zero, but done in software.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 28358
Joined: Sat Jul 30, 2011 7:41 pm

Re: ARMv7 emulation on ARMv6: System Control Processor

Wed Apr 22, 2020 10:57 am

fd_ wrote:
Wed Apr 22, 2020 10:27 am
DougieLawson wrote:
Wed Apr 22, 2020 10:23 am
I don't have the first faintest idea why you think you'd want to do this. Why not simply buy a RPi4B 4GB and you can run ARM6, ARM7 and ARM8 code natively on the processor.
I have existing ARMv7 binaries (no source) that I'd like to run on a portable device with as little power as possible. The Pi Zero is the most portable and most low-power Pi, that's why using that would be nice. I did buy a Pi 3A+ that works fine, but seems to use about 2x as much power as the Zero.

Beside my specific application, given that there will likely never be a CPU upgrade to the Pi Zero, and the fact that ARMv6 support is getting dropped in major projects, I do think a performant ARMv7 emulation layer would be very useful. Like a hardware upgrade for the Pi Zero, but done in software.
Any CPU upgrade to the Zero would come with an increase in power consumption.....swings and roundabouts!

Interesting project, no idea if it's even possible.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

fd_
Posts: 93
Joined: Thu Oct 25, 2018 7:35 am

Re: ARMv7 emulation on ARMv6: System Control Processor

Wed Apr 22, 2020 11:12 am

jamesh wrote:
Wed Apr 22, 2020 10:57 am
Any CPU upgrade to the Zero would come with an increase in power consumption.....swings and roundabouts!

Interesting project, no idea if it's even possible.
I agree that even this "software CPU upgrade" would increase power consumption per executed program instruction, ie there'd be a performance decrease caused by the emulation. Still, if we can keep the performance at a reasonable level, this could make the Pi Zero useful for projects that have only modest requirements on the CPU speed, but require ARMv7.

AFAICT, the power required by the more capable Pi models doesn't just scale proportionally to the used processing power. They also need more power than Pi Zero even when just idling.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 28358
Joined: Sat Jul 30, 2011 7:41 pm

Re: ARMv7 emulation on ARMv6: System Control Processor

Wed Apr 22, 2020 12:28 pm

fd_ wrote:
Wed Apr 22, 2020 11:12 am
AFAICT, the power required by the more capable Pi models doesn't just scale proportionally to the used processing power. They also need more power than Pi Zero even when just idling.
Depends on the peripherals attached. Put the Pi4 CPU on a Zero board and you still need a load more power though. IIRC, the CPU is the main draw, then ethernet etc.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

ejolson
Posts: 6639
Joined: Tue Mar 18, 2014 11:47 am

Re: ARMv7 emulation on ARMv6: System Control Processor

Wed Apr 22, 2020 6:21 pm

fd_ wrote:
Wed Apr 22, 2020 7:38 am
Hi all,
I'm trying to write a kernel module that allows to run ARMv7 userspace code on an ARMv6 Raspberry Pi Zero. The kernel module registers an undefined instruction hook and parses ARMv7 instructions, manually executing the required register transfers etc. I'm hoping this would have as little overhead as possible, since I suspect using something like QEMU userspace emulation will be much too slow for any real use case.

For some instructions, such as MOVW and UBFX, this seems to work fine. However, I'm now seeing my hook being called for an MRRC instruction for coprocessor 15, which seems to be the System Control Coprocessor:

Code: Select all

ec510f1e = mrrc p15, #1, r0, r1, c14
According to the documentation of ARMv7 [1], register 14 of the System Control Coprocessor is for the Generic Timer. Interestingly, it seems like ARMv6's System Control Coprocessor [2] doesn't even have a register 14, and comparing all the other registers, they seem to largely differ from their ARMv7 counterparts. Yet, these differences are not mentioned in the "Migrating a software application from ARMv5 to ARMv7-A/R Application Note 425" [3] document.

Does anyone know how these System Control Coprocessor accesses could be emulated on ARMv6? Does ARMv6 have a Generic Timer? Why do userspace programs access the System Control Coprocessor anyway?

Thanks in advance!

[1]: http://infocenter.arm.com/help/index.js ... HBIDJ.html
[2]: http://infocenter.arm.com/help/index.js ... index.html
[3]: http://infocenter.arm.com/help/index.js ... GFCGC.html
Thank you for working on this. It has the potential to increase the life expectancy of Zero hardware as software is tuned to run on the ARMv7. I'm sorry I don't have any insight on the control coprocessor, but wonder, what are such instructions doing in user-space code?

fd_
Posts: 93
Joined: Thu Oct 25, 2018 7:35 am

Re: ARMv7 emulation on ARMv6: System Control Processor

Wed Apr 22, 2020 6:51 pm

ejolson wrote:
Wed Apr 22, 2020 6:21 pm
Thank you for working on this. It has the potential to increase the life expectancy of Zero hardware as software is tuned to run on the ARMv7. I'm sorry I don't have any insight on the control coprocessor, but wonder, what are such instructions doing in user-space code?
Thanks for the encouragement! However, I'm afraid without help from someone familiar with the ARM ISA, I won't get very far. I was hoping someone from the Foundation/Trading would be willing to help here. Probably I should ask in the bare-metal subforums?

ARMv7 is actually composed of multiple different instruction sets:
  • The ARM ISA itself. The difference between ARMv7 and ARMv6 is minimal here, and I'm confident this would be something I'm capable of implementing.
  • Thumb. ARMv6 only supports 16-bit Thumb-1 instructions, but ARMv7 added basically a 32-bit Thumb-2 equivalent for most ARM instructions, so that's a lot of Thumb-2 instructions to emulate. However, it seems gcc doesn't produce Thumb-2 instructions unless explicitly enabled, which could mean they aren't too common in the wild. I might be wrong about that.
  • VFP. The ARM itself doesn't have instructions for floating point operations. These are done by a separate coprocessor, the VFP (Vector Floating Point). The ARMv6 has a VFPv2 coprocessor with 16 64-bit registers, while ARMv7's VFPv3 has 32 64-bit registers. Some binaries are compiled with VFPv3-d16 though, which only uses 16 of these registers. VFPv3 also has a few more instructions than VFPv2, which probably could be emulated.
  • NEON. This is a coprocessor for SIMD operations. The ARMv6 in the Pi Zero doesn't have it, so these instructions would (IMHO) be the hardest to emulate. However, NEON has to be explicitly enabled on compilation, so should generally not be used in your average ARMv7 binary.
So, realistically, this is not something I could implement myself in its entirety. Also, as mentioned, I already purchased a Pi 3A+ essentially so I don't have to tackle this undertaking. Ideally, this would be a topic for a Bachelor's or Master's thesis for a computer science student (Unfortunately, I already started mine on a different topic a few months back).

Another (likely illusory) scenario would be the Pi Foundation/Trading tackling this project as a means to extend the lifespan of the Pi Zero. It's likely orders of magnitudes cheaper to implement than developing new silicon suitable for an updated Pi Zero.

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 28358
Joined: Sat Jul 30, 2011 7:41 pm

Re: ARMv7 emulation on ARMv6: System Control Processor

Wed Apr 22, 2020 7:00 pm

fd_ wrote:
Wed Apr 22, 2020 6:51 pm
Another (likely illusory) scenario would be the Pi Foundation/Trading tackling this project as a means to extend the lifespan of the Pi Zero. It's likely orders of magnitudes cheaper to implement than developing new silicon suitable for an updated Pi Zero.
Sorry....not something we would have the time to do. Or, actually, the experience, I suspect. Zero's sell pretty well already, and most software that is run on them people have source code for so they can recompile if they need to.

I can move this entire thread to bare metal if you wish.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

ejolson
Posts: 6639
Joined: Tue Mar 18, 2014 11:47 am

Re: ARMv7 emulation on ARMv6: System Control Processor

Wed Apr 22, 2020 7:56 pm

jamesh wrote:
Wed Apr 22, 2020 7:00 pm
fd_ wrote:
Wed Apr 22, 2020 6:51 pm
Another (likely illusory) scenario would be the Pi Foundation/Trading tackling this project as a means to extend the lifespan of the Pi Zero. It's likely orders of magnitudes cheaper to implement than developing new silicon suitable for an updated Pi Zero.
Sorry....not something we would have the time to do. Or, actually, the experience, I suspect. Zero's sell pretty well already, and most software that is run on them people have source code for so they can recompile if they need to.

I can move this entire thread to bare metal if you wish.
I think the main difficulty with the Zero right now is recent Java VMs. If a kernel module worked, would the maintainers of Raspbian consider including it in the repository so some of the currently broken binaries would run on the Zero?

plugwash
Forum Moderator
Forum Moderator
Posts: 3738
Joined: Wed Dec 28, 2011 11:45 pm

Re: ARMv7 emulation on ARMv6: System Control Processor

Wed Apr 22, 2020 8:34 pm

fd_ wrote:
Wed Apr 22, 2020 6:51 pm
[*] Thumb. ARMv6 only supports 16-bit Thumb-1 instructions, but ARMv7 added basically a 32-bit Thumb-2 equivalent for most ARM instructions, so that's a lot of Thumb-2 instructions to emulate. However, it seems gcc doesn't produce Thumb-2 instructions unless explicitly enabled, which could mean they aren't too common in the wild. I might be wrong about that.
Unfortunately for you many if not all major linux distros did explicitly enable it.

trejan
Posts: 3064
Joined: Tue Jul 02, 2019 2:28 pm

Re: ARMv7 emulation on ARMv6: System Control Processor

Wed Apr 22, 2020 9:00 pm

readelf on a Debian Buster armhf binary shows this:

Code: Select all

  Tag_CPU_name: "7-A"
  Tag_CPU_arch: v7
  Tag_CPU_arch_profile: Application
  Tag_ARM_ISA_use: Yes
  Tag_THUMB_ISA_use: Thumb-2
  Tag_FP_arch: VFPv3-D16
  Tag_ABI_PCS_wchar_t: 4
  Tag_ABI_FP_rounding: Needed
  Tag_ABI_FP_denormal: Needed
  Tag_ABI_FP_exceptions: Needed
  Tag_ABI_FP_number_model: IEEE 754
  Tag_ABI_align_needed: 8-byte
  Tag_ABI_align_preserved: 8-byte, except leaf SP
  Tag_ABI_enum_size: int
  Tag_ABI_VFP_args: VFP registers
  Tag_CPU_unaligned_access: v6

jamesh
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 28358
Joined: Sat Jul 30, 2011 7:41 pm

Re: ARMv7 emulation on ARMv6: System Control Processor

Wed Apr 22, 2020 9:37 pm

ejolson wrote:
Wed Apr 22, 2020 7:56 pm
jamesh wrote:
Wed Apr 22, 2020 7:00 pm
fd_ wrote:
Wed Apr 22, 2020 6:51 pm
Another (likely illusory) scenario would be the Pi Foundation/Trading tackling this project as a means to extend the lifespan of the Pi Zero. It's likely orders of magnitudes cheaper to implement than developing new silicon suitable for an updated Pi Zero.
Sorry....not something we would have the time to do. Or, actually, the experience, I suspect. Zero's sell pretty well already, and most software that is run on them people have source code for so they can recompile if they need to.

I can move this entire thread to bare metal if you wish.
I think the main difficulty with the Zero right now is recent Java VMs. If a kernel module worked, would the maintainers of Raspbian consider including it in the repository so some of the currently broken binaries would run on the Zero?
Java is likely too slow on a Zero to be of much use anyway.

If someone could get something like this to work, it's certainly possible it could be added. However, the fact that there isn't anything already like it might be an indicator is very difficult to make work. It wouldn't be a simple kernel module.
Principal Software Engineer at Raspberry Pi (Trading) Ltd.
Contrary to popular belief, humorous signatures are allowed.
I've been saying "Mucho" to my Spanish friend a lot more lately. It means a lot to him.

User avatar
dickon
Posts: 1886
Joined: Sun Dec 09, 2012 3:54 pm
Location: Home, just outside Reading

Re: ARMv7 emulation on ARMv6: System Control Processor

Wed Apr 22, 2020 9:50 pm

The performance hit is likely to be horrendous, too. Even excluding the decode and synthesis stages, you nobble the D-cache by polluting it with instruction data. It's one of the things ISTR Acorn muttered about when eventually implementing an FPU on whichever of the ARM chips it was they introduced the thing on.
As it is apparently board policy to disallow any criticism of anything, as it appears to criticise something is to criticise all the users of that something, I will no longer be commenting in threads which are not directly relevant to my uses of the Pi.

helo
Posts: 1
Joined: Mon Dec 28, 2020 9:47 pm

Re: ARMv7 emulation on ARMv6: System Control Processor

Mon Dec 28, 2020 9:51 pm

:?

Um, get an armv6 toolchain and compile your binary.

Sounds like some bull that apple sould hire someone to work on for anti-competitive OSX-on-PI with a shaky and dysfunctional emulation scheme.

Compile gcc+binugils for armv6-linuz, its plumbing not rocket sciencs

cleverca22
Posts: 2904
Joined: Sat Aug 18, 2012 2:33 pm

Re: ARMv7 emulation on ARMv6: System Control Processor

Tue Dec 29, 2020 4:54 am

helo wrote:
Mon Dec 28, 2020 9:51 pm
:?

Um, get an armv6 toolchain and compile your binary.

Sounds like some bull that apple sould hire someone to work on for anti-competitive OSX-on-PI with a shaky and dysfunctional emulation scheme.

Compile gcc+binugils for armv6-linuz, its plumbing not rocket sciencs
fd_ wrote:
Wed Apr 22, 2020 10:27 am
I have existing ARMv7 binaries (no source) that I'd like to run on a portable device with as little power as possible. The Pi Zero is the most portable and most low-power Pi, that's why using that would be nice. I did buy a Pi 3A+ that works fine, but seems to use about 2x as much power as the Zero.
not an option

fd_ wrote:
Wed Apr 22, 2020 7:38 am

Code: Select all

ec510f1e = mrrc p15, #1, r0, r1, c14
According to the documentation of ARMv7 [1], register 14 of the System Control Coprocessor is for the Generic Timer. Interestingly, it seems like ARMv6's System Control Coprocessor [2] doesn't even have a register 14, and comparing all the other registers, they seem to largely differ from their ARMv7 counterparts. Yet, these differences are not mentioned in the "Migrating a software application from ARMv5 to ARMv7-A/R Application Note 425" [3] document.

Does anyone know how these System Control Coprocessor accesses could be emulated on ARMv6? Does ARMv6 have a Generic Timer? Why do userspace programs access the System Control Coprocessor anyway?
the pi0/pi1 doesnt have a generic arm timer, so you have to emulate all of that based on the free-running timer (1mhz clock) on the VPU timer, at ST_CHI and ST_CLO, or the generic timer subsystem on linux

Return to “General discussion”