timanu90
Posts: 61
Joined: Sat Dec 24, 2016 11:54 am

RPI3 QEMU

Tue Oct 17, 2017 11:01 am

Hi guys, seems that some people could run aarch64 rpi3 code in qemu. Anyone can tell me the version of qemu needed and what command you use to launch the session?

bzt
Posts: 43
Joined: Sat Oct 14, 2017 9:57 pm

Re: RPI3 QEMU

Tue Oct 17, 2017 12:02 pm

Hi timanu90!

I've patched the qemu source to support Raspberry Pi 3 booting. The repository is here:
https://github.com/bztsrc/qemu-raspi3

I have configured it with

Code: Select all

./configure --target-list=aarch64-softmmu,aarch64-linux-user --enable-modules --enable-tcg-interpreter --enable-debug-tcg
After compilation, the command to start it:

Code: Select all

qemu-system-aarch64 -M raspi3 -kernel kernel8.img
But it has only limited support for now:
- booting procedure as on real hw without altering config.txt
- VideoCore mailboxes
- framebuffer
- UART0 serial
- UART1 (AUX) supported, but not sent to stdio

Devices are not really supported yet, for example it can't use sd card. Mostly because the sd card driver depends on microsec delay, and in qemu ARM_SYSTEM_TIMER at 0x3F003000 always returns zero (making the delay hang). According to the device tree, the sd device does exists though (haven't checked the EMMC registers existence yet). I made a workaround for AUX output: I turned off UART0 and when I write to AUX_MU_IO, I also write to UART0_DR (on real hardware it won't work as UART0 is disabled, but qemu doesn't care and displays my output on it's stdio). With this trick uart_putc() works on qemu and on real hw as well.

Because of the limited BCM2835 support, it's mainly good for testing AArch64 features (like exceptions, EL2, EL1 transitions, SVC calls, MMU etc.).

I have wrote an email to the qemu arm maintainer whether my code is okay for committing to the mainline qemu, but haven't got a response yet.

Cheers,
bzt

timanu90
Posts: 61
Joined: Sat Dec 24, 2016 11:54 am

Re: RPI3 QEMU

Tue Oct 17, 2017 12:28 pm

Hi bzt, thank you for sharing. For me it's perfect because my use case is for configuring the MMU, it's very hard to debug it in the real hardware.

I will give it a try :D .

timanu90
Posts: 61
Joined: Sat Dec 24, 2016 11:54 am

Re: RPI3 QEMU

Tue Oct 17, 2017 1:24 pm

I managed to put the qemu to work and print the content of the UART (PL011) on the terminal with the command

Code: Select all

qemu-system-aarch64 -M raspi3 -kernel multicos.elf -serial stdio
My question now is, did you managed to attach a gdb session to debug step by step?

bzt
Posts: 43
Joined: Sat Oct 14, 2017 9:57 pm

Re: RPI3 QEMU

Tue Oct 17, 2017 3:47 pm

timanu90 wrote:
Tue Oct 17, 2017 1:24 pm
I managed to put the qemu to work and print the content of the UART (PL011) on the terminal with the command
Yes, that works, but I'm using mini-UART (UART1), and I was unable to configure qemu to redirect UART1 to stdio instead of UART0. I've tried several -chardev options, and finally come up with that workaround.
timanu90 wrote:
Tue Oct 17, 2017 1:24 pm
My question now is, did you managed to attach a gdb session to debug step by step?
Yes. But I've used .img, not .elf as kernel, and added symbols from the original elf on gdb's command line. It was quite useless btw, as step by step execution failed with "not in a function" error message (as boot starts at offset 0 with a small stub jumping to 0x80000, not in any function). You can play with breakpoints though. I found it easier to add "-d in_asm,cpu,mmu,int -singlestep 2>run.log" arguments to qemu to see what's happening.

And good news everyone :-) I've managed to get SD card working under qemu! :-)

As I have said, in hldswrth's sdcard code, and in haribote sdcard driver there's a waitMicro() function that will freeze as unfortunately get_system_timer() returns constant zero on qemu (same problem with circle64's CTimer).

I've checked qemu's source, system timer is defined but unfortunately no region added at that location in bcm2835_peripherals.c, so there's no driver behind it.

But if you change that function to:

Code: Select all

// somewhere in initialization before the first waitMicro() call
uint64_t cntfrq;
asm volatile ("mrs %0, cntfrq_el0" : "=r" (cntfrq));

// delay cnt microsec
void waitMicro(uint32_t cnt)
{
  uint64_t t, r;
  asm volatile ("mrs %0, cntpct_el0" : "=r" (t));
  // careful with precision and overflow / underflow
  t += ((cntfrq/1000)*cnt)/1000;
  do{
    asm volatile ("mrs %0, cntpct_el0" : "=r" (r));
  }while(r < t);
}
Then you are good to go! :-)

Few differences though: real hardware only supports ACMD41_CCS mode (so READ_SINGLE and READ_MULTI commands require LBA address), whilst qemu does not support CCS at all (no READ_MULTI command, and READ_SINGLE requires absolute position, i.e. lba*512). But those differences are handled pretty well by hldswrth's code.

Cheers,
bzt

timanu90
Posts: 61
Joined: Sat Dec 24, 2016 11:54 am

Re: RPI3 QEMU

Wed Nov 29, 2017 10:17 am

Hi, I am trying qemu again, this time I am trying use GDB but in qemu the option -S seem not to wrok. Do you have any clue what could be?

-S flag should tell qemu to pause emulation right before the execution of the first instruction

cheers
Tiago

bzt
Posts: 43
Joined: Sat Oct 14, 2017 9:57 pm

Re: RPI3 QEMU

Wed Nov 29, 2017 3:18 pm

Hi,
timanu90 wrote:
Wed Nov 29, 2017 10:17 am
Hi, I am trying qemu again, this time I am trying use GDB but in qemu the option -S seem not to wrok. Do you have any clue what could be?

-S flag should tell qemu to pause emulation right before the execution of the first instruction

cheers
Tiago
Well, for one '-S' only stops the CPU, so you'll have to type 'c' in a debugger. That expected to be qemu's built in debugger. So if you want to use gdb instead, you must also enable the gdb server stub listening on localhost:1234 with the '-s' flag (or use '-gdb' and specify other host and/or port if needed).

The commands I've used:

Terminal #1:

Code: Select all

qemu-system-aarch64 -s -S -M raspi3 -kernel loader/bootboot.img

Terminal #2:
Start gdb and at gdb's command prompt:

Code: Select all

set architecture aarch64
set debug aarch64
target remote localhost:1234
symbol-file (your elf file here)
display/i $pc
The first command tells gdb that the target is an AArch64, important to set before you connect to the remote target. The 2nd line enables AArch64 specific commands and options in gdb. The 3rd line connects gdb to the stopped qemu. The 4th line loads symbols from an elf file (not loader/bootboot.img in my case, but for you it should be the same elf you're running with qemu) so that you can use function names for breakpoints and such. Finally the last command makes gdb to display where the hell is code execution wandering... It's not necessary, but very useful :-)
Anyway, after your have connected the gdb to qemu, you can issue a 'c' command to continue (more precisely start) the execution.

Oh, and one more thing. I'm assuming you're running qemu and gdb on a PC with Intel CPU. Most gdb supports only the native architecture as target, so you'll need a cross-compiled gdb. The standard gdb shipped with your linux won't work, and will complain about missing XMLs on "set architecture aarch64". Therefore configure and compile gdb with "--target=aarch64-elf". You don't have to worry about GDBserver, that's included in qemu.

Hope it helps,
bzt

timanu90
Posts: 61
Joined: Sat Dec 24, 2016 11:54 am

Re: RPI3 QEMU

Wed Nov 29, 2017 5:12 pm

Hello, I tried de -s -S but seems to not work for me either. About gdb I am using

Code: Select all

>$ aarch64-linux-gnu-gdb -v
NU gdb (GDB) 8.0.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-pc-linux-gnu --target=aarch64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
When I do the target remote the code is already traped in an infinite loop I have in address 0x8800.

Qemu start at 0x00 right???

bzt
Posts: 43
Joined: Sat Oct 14, 2017 9:57 pm

Re: RPI3 QEMU

Thu Nov 30, 2017 11:18 pm

timanu90 wrote:
Wed Nov 29, 2017 5:12 pm

Code: Select all

>$ aarch64-linux-gnu-gdb -v
This GDB was configured as "--host=x86_64-pc-linux-gnu --target=aarch64-linux-gnu".
When I do the target remote the code is already traped in an infinite loop I have in address 0x8800.
Yeah, that's the cross debugger you need. Everything is fine, you can connect and you see your code running.
Not sure about the address though. If the machine is not stopped at the smp boot up code, the PC should be above 0x80000, unless you have set up something explicitly. Also check your elf's program headers with readelf, there's a good chance it was linked at that offset, and if you do a normal linux boot in qemu it won't use FIRMWARE_ADDR_3.

Anyway it seems to me you managed to get a working cross gdb, congrats! :-) You can use "set $pc+=4" to jump over the infinite loop, and use step-by-step execution from there.

Good work!
bzt

vindy
Posts: 3
Joined: Tue Nov 28, 2017 12:48 pm

Re: RPI3 QEMU

Mon Dec 18, 2017 12:22 pm

Sorry for a noob questions, I am a total newbie in RPI world... I have a system running on RPI3, and do not have any access to it (flightaware.com/adsb/flightfeeder/). I was wondering if I can just make an .img and run it under QEMU on the same RPI, which will give me a chance to use the raspberry for something else at the same time?
So:
1. Considering that I can not change anything in the system I have, would it be possible to have a VM communicate with LAN interface and a radio board connected via USB?
2. If the answer is yes, would it be too much if I ask to share a pre-compiled QEMU for RPI3 target platform?

Thank you in advance!

bzt
Posts: 43
Joined: Sat Oct 14, 2017 9:57 pm

Re: RPI3 QEMU

Mon Dec 18, 2017 12:49 pm

vindy wrote:
Mon Dec 18, 2017 12:22 pm
Sorry for a noob questions, I am a total newbie in RPI world... I have a system running on RPI3, and do not have any access to it (flightaware.com/adsb/flightfeeder/). I was wondering if I can just make an .img and run it under QEMU on the same RPI, which will give me a chance to use the raspberry for something else at the same time?
First of all, I'm not sure this is the right forum for such a question. Anyway.

Yes, you can. I don't know that site, but you must have an SD card your RPi boots from. Simply get the SD card out from your RPi, put it in a Linux box as a data storage (so do not boot from it) and use 'dd' command to create an '.img' out of it. The 'dd' command is a standard UNIX command on every distribution, also available on raspbian. Read it's manual page on how to use it.
vindy wrote: So:
1. Considering that I can not change anything in the system I have, would it be possible to have a VM communicate with LAN interface and a radio board connected via USB?
As I've said, I do not know the image you're using. But if there's a driver in that image for any of the LAN devices qemu provides then it can be done easily. Now about USB radio, for that you'll need a driver for the host OS. So summarizing it up:
1. you'll need a qemu LAN driver for the guest running in VM
2. you'll need an USB radio driver for the host OS
3. you can configure qemu to connect host LAN interface to guest LAN interface (google it, there are many manuals on this topic)
vindy wrote: 2. If the answer is yes, would it be too much if I ask to share a pre-compiled QEMU for RPI3 target platform?
Yes too much, I definitely don't have the resources to provide pre-compiled qemu binaries. I've sent the rpi3 patch to qemu-devel, and they asked for some modifications. Now there's already a pending patch that implements some of those, and I'm awaiting for that patch. Once it's merged with the mainline, I'll resend my patch, so sooner or later rpi3 will make it through to the qemu mainline, and sooner or later you'll have it with the standard raspbian qemu package. Just matter of years, be patient. (Sorry for being sarcastic, no offense meant).

Good luck,
bzt

vindy
Posts: 3
Joined: Tue Nov 28, 2017 12:48 pm

Re: RPI3 QEMU

Mon Dec 18, 2017 1:02 pm

Thanks for sharing the knowledge! I am absolutely ok with going all the way myself with learning how to compile and setup QEMU from the scratch, just wanted to be sure that my plan is a doable thing. Appreciate your help with making it clear, and sorry if I started it in a wrong forum (no one seemed to be interested in beginners section when I asked the question there).
Huge thanks and regards again,
Vladimir

Return to “Bare metal”

Who is online

Users browsing this forum: No registered users and 4 guests