QEMU patches for RPi emulation - Initial release

[Torlus]

Hi,
I've been working for a few weeks on a QEMU backend for RPi emulation.
It's now advanced enough to boot an unmodified Linux kernel, along with its associated SD image.

Get it here : https://github.com/Torlus/qemu-rpi
Please read carefully the instructions.

It's far from being complete, but well, I'm releasing it anyway.
Hope it helps, and feel free to contribute.

Regards,
Greg

[ShiftPlusOne]

Haven't tried it out yet (github is down), but it sounds great. Will give it a go when I get the chance.
Thanks

[Torlus]

I've also set up a QEMU branch, that can be found here : https://github.com/Torlus/qemu/tree/rpi

[benzn]

Awesome! This is a huge asset moving forward, thanks so much.

I got this running on OSX and booting successfully, although running the latest Raspbian image (12-16-12) it consistently locks up (process not responding) after a couple minutes. I'll try some more bare metal things tomorrow.

For others looking to do the same, I'm running 10.8.2 with homebrew. I needed to run the following commands after checking out the rpi qemu branch:

Code: Select all

brew install glib
brew install pixman
export PKG_CONFIG_PATH=/usr/local/Cellar/pixman/0.26.2/lib/pkgconfig
./configure
make
sudo make install
cd [dir where kernel images are]
qemu-system-arm -kernel kernel.img -cpu arm1176 -m 512 -M raspi -serial stdio -append "rw dma.dmachans=0x7f35 bcm2708_fb.fbwidth=1024 bcm2708_fb.fbheight=768 bcm2708.boardrev=0xf bcm2708.serial=0xcad0eedf smsc95xx.macaddr=B8:27:EB:D0:EE:DF sdhci-bcm2708.emmc_clock_freq=100000000 vc_mem.mem_base=0x1c000000 vc_mem.mem_size=0x20000000 dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait" -snapshot -sd 2012-12-16-wheezy-raspbian.img -d guest_errors

[IntelMiner]

Would this QEMU fork/build/whathaveyou be able to "bypass" the 512MB RAM limit of the Pi?

OFC one would need a compatible kernel, but being able to use it as a full compile "sandbox" would be awesome for projects like Gentoo

[Torlus]

Awesome! This is a huge asset moving forward, thanks so much.

I got this running on OSX and booting successfully, although running the latest Raspbian image (12-16-12) it consistently locks up (process not responding) after a couple minutes. I'll try some more bare metal things tomorrow.

For others looking to do the same, I'm running 10.8.2 with homebrew. I needed to run the following commands after checking out the rpi qemu branch:

Thanks for your appreciation.
Can you provide more information about this lockup ? I've tested the latest Raspbian image, and I managed to go up to starting X-window (using the "stdio" console). USB emulation is one of the next things on my TODO list so testing should be easier that way.

[Torlus]

Would this QEMU fork/build/whathaveyou be able to "bypass" the 512MB RAM limit of the Pi?

OFC one would need a compatible kernel, but being able to use it as a full compile "sandbox" would be awesome for projects like Gentoo

Well (looking at the "ARM physical address" table in the datasheet), by passing some ARM boot tags, it should be possible to define a second physical memory range (after the I/O space). If the kernel can deal with it, that would increase the available memory to 0x3E000000 (i.e 992 MB). Not sure that's what you want/mean though.

[benzn]

Re: qemu hanging on OSX.

This post: http://www.rpiforum.net/forum/tutorials ... -osx-lion/ indicated that using qemu to compile LLVM may lead to hanging, so I downloaded gcc-4.2 and recompiled.

Unfortunately, the issue still persists. It doesn't seem to matter where or what you are doing (I can get through startx as well), but a few minutes into booting Linux, it will lock up, sometimes before you even get to a console, sometimes after you get to . On the other hand, I got some of dwelch's examples (namely uartx01) working, which don't hang at all.

Could it possibly be some unexpected interrupt (handling) which is locking up because of a partial implementation? I suppose I can build qemu with debug symbols and force kill it and share a trace next.

[Torlus]

Oh, I wasn't sure if it was Linux or QEMU that locked, but it seems that it's QEMU... There are many references about similar issues on the net, but indeed the real reason isn't very clear.
If you can provide a trace, that would surely help.
Meanwhile, I will check my code... If by chance QEMU is still running fine but the kernel hangs, it might be an issue with the system timer or some faulty interrupt generation/processing.

[cmason]

Oh, I wasn't sure if it was Linux or QEMU that locked, but it seems that it's QEMU... There are many references about similar issues on the net, but indeed the real reason isn't very clear.
If you can provide a trace, that would surely help.
Meanwhile, I will check my code... If by chance QEMU is still running fine but the kernel hangs, it might be an issue with the system timer or some faulty interrupt generation/processing.

I'm trying to use your qemu-rpi patches with the qemu tree you posted above. I'm doing this on MacOS 10.8.2, and I'm running in to a similar hang: white screen and no activity. This happens very early in the process, before I get any kernel messages. I'm using:

Code: Select all

qemu-system-arm -kernel kernel-qemu -cpu arm1176 -m 512 -M raspi -serial stdio -append "rw dma.dmachans=0x7f35 bcm2708_fb.fbwidth=1024 bcm2708_fb.fbheight=768 bcm2708.boardrev=0xf bcm2708.serial=0xcad0eedf smsc95xx.macaddr=B8:27:EB:D0:EE:DF sdhci-bcm2708.emmc_clock_freq=100000000 vc_mem.mem_base=0x1c000000 vc_mem.mem_size=0x20000000 dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait" -snapshot -sd 2012-12-16-wheezy-raspbian.img -d guest_errors

I configured qemu with:

Code: Select all

./configure --disable-sdl --disable-kvm --enable-cocoa --cc=gcc-4.2 --host-cc=gcc-4.2 --enable-debug   --extra-cflags=-g   --extra-ldflags=-g

I see a similar hang when using vanilla arm qemu without your patches, (see this post for more details), so I'd guess its not related to your code specifically. I'm wondering if you have any hints on debugging this.

Here's some debugger output:

Code: Select all

Program received signal SIGUSR1, User defined signal 1.
[Switching to process 44920 thread 0x604f]
0x00007fff9a5a9330 in strcmp ()
(gdb) bt
#0  0x00007fff9a5a9330 in strcmp ()
#1  0x0000000100fbdcb4 in g_str_equal ()
#2  0x0000000100fbe13f in g_hash_table_lookup ()
#3  0x00000001001b7503 in type_table_lookup (name=0x1003b940c "arm-cpu") at qom/object.c:86
#4  0x00000001001b77e7 in type_get_by_name (name=0x1003b940c "arm-cpu") at qom/object.c:144
#5  0x00000001001b8694 in object_class_dynamic_cast (class=0x102118f70, typename=0x1003b940c "arm-cpu") at qom/object.c:452
#6  0x00000001001b85b7 in object_dynamic_cast (obj=0x102992800, typename=0x1003b940c "arm-cpu") at qom/object.c:427
#7  0x00000001001b8616 in object_dynamic_cast_assert (obj=0x102992800, typename=0x1003b940c "arm-cpu") at qom/object.c:438
#8  0x000000010021247a in arm_env_get_cpu (env=0x102992860) at cpu-qom.h:105
#9  0x0000000100214067 in qemu_tcg_cpu_thread_fn (arg=0x102992800) at /usr/local/src/raspberry/qemu-rpi/cpus.c:836
#10 0x00007fff9a5bb742 in _pthread_start ()
#11 0x00007fff9a5a8181 in thread_start ()

(gdb) c
Continuing.

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x000000010147bc90
[Switching to process 44920 thread 0xf0b]
0x000000010147bc90 in ?? ()
(gdb) bt
#0  0x000000010147bc90 in ?? ()
#1  0x000000010016e209 in qemu_iohandler_poll (readfds=0x10097ca00, writefds=0x10097ca80, xfds=0x10097cb00, ret=3) at iohandler.c:124
#2  0x0000000100172acf in main_loop_wait (nonblocking=0) at main-loop.c:418
#3  0x0000000100207bbf in main_loop () at vl.c:1765
#4  0x000000010020e7b0 in qemu_main (argc=18, argv=0x7fff5fbff210, envp=0x7fff5fbff2a8) at vl.c:3992
#5  0x00000001001d5a79 in -[QemuCocoaAppController startEmulationWithArgc:argv:] (self=0x102308fb0, _cmd=0x1003ca438, argc=18, argv=0x7fff5fbff210) at ui/cocoa.m:798
#6  0x00000001001d5991 in -[QemuCocoaAppController applicationDidFinishLaunching:] (self=0x102308fb0, _cmd=0x7fff91517f79, note=0x10145fd50) at ui/cocoa.m:776
#7  0x00007fff950ca47a in _CFXNotificationPost ()
#8  0x00007fff8f376846 in -[NSNotificationCenter postNotificationName:object:userInfo:] ()
#9  0x00007fff8f7ee60d in -[NSApplication _postDidFinishNotification] ()
#10 0x00007fff8f7ee346 in -[NSApplication _sendFinishLaunchingNotification] ()
#11 0x00007fff8f7eb532 in -[NSApplication(NSAppleEventHandling) _handleAEOpenEvent:] ()
#12 0x00007fff8f7eb12c in -[NSApplication(NSAppleEventHandling) _handleCoreEvent:withReplyEvent:] ()
#13 0x00007fff8f39012b in -[NSAppleEventManager dispatchRawAppleEvent:withRawReply:handlerRefCon:] ()
#14 0x00007fff8f38ff8d in _NSAppleEventManagerGenericHandler ()
#15 0x00007fff8dc89b48 in aeDispatchAppleEvent ()
#16 0x00007fff8dc899a9 in dispatchEventAndSendReply ()
#17 0x00007fff8dc89869 in aeProcessAppleEvent ()
#18 0x00007fff8e5a98e9 in AEProcessAppleEvent ()
#19 0x00007fff8f7e7916 in _DPSNextEvent ()
#20 0x00007fff8f7e6ed2 in -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] ()
#21 0x00007fff8f7de283 in -[NSApplication run] ()
#22 0x00000001001d6966 in main (argc=18, argv=0x7fff5fbff210) at ui/cocoa.m:946

(gdb) p ioh
$4 = (IOHandlerRecord *) 0x10147bb60
(gdb) p *ioh
$5 = {
  fd_read_poll = 0, 
  fd_read = 0x10017212b <sigfd_handler>, 
  fd_write = 0, 
  opaque = 0x5, 
  next = {
    le_next = 0x0, 
    le_prev = 0x102118b40
  }, 
  fd = 5, 
  deleted = false
}





(gdb) p io_handlers
$1 = {
  lh_first = 0x102118b20
}
(gdb) p *io_handlers.lh_first
$2 = {
  fd_read_poll = 0x10019eeec <fd_chr_read_poll>, 
  fd_read = 0x10019ef4c <fd_chr_read>, 
  fd_write = 0, 
  opaque = 0x102118a40, 
  next = {
    le_next = 0x10147bb60, 
    le_prev = 0x10097c9c8
  }, 
  fd = 0, 
  deleted = false
}
(gdb) p *io_handlers.lh_first->next.le_next
$3 = {
  fd_read_poll = 0, 
  fd_read = 0x10017212b <sigfd_handler>, 
  fd_write = 0, 
  opaque = 0x5, 
  next = {
    le_next = 0x0, 
    le_prev = 0x102118b40
  }, 
  fd = 5, 
  deleted = false
}

[pointfree]

Has anyone used this to successfully boot a non-Linux kernel such as Plan9 or DexOS?

[Torlus]

Has anyone used this to successfully boot a non-Linux kernel such as Plan9 or DexOS?

Well I'm currently trying to get RiscOS to boot, but without success so far... I'll give a try to these OS, though.

[BrianW]

Hi,

I tried it with my bare metal kernel and have noticed a handful of things

• The kernel is booted at 0x00010000, not 0x00008000. This might affect you if your compiler is producing position-dependent code
• The framebuffer can only be set up using the framebuffer mailbox. The properties mailbox isn't implemented
• The ARM1176 emulation is incomplete. (I noticed this because I'm using the vector base address register in the system coprocessor. Any reads/writes to c12 give an undefined instruction exception)

[Torlus]

Hi,

I tried it with my bare metal kernel and have noticed a handful of things

• The kernel is booted at 0x00010000, not 0x00008000. This might affect you if your compiler is producing position-dependent code
• The framebuffer can only be set up using the framebuffer mailbox. The properties mailbox isn't implemented
• The ARM1176 emulation is incomplete. (I noticed this because I'm using the vector base address register in the system coprocessor. Any reads/writes to c12 give an undefined instruction exception)

For the two first points, these have been addressed, I will commit the changes soon.
The last point is interesting, as it seems that's what prevents RiscOS from booting (it encounters an undefined instruction exception as well). Thanks for pointing that out.

[tufty]

Hi,

I tried it with my bare metal kernel and have noticed a handful of things

• The kernel is booted at 0x00010000, not 0x00008000. This might affect you if your compiler is producing position-dependent code
• The framebuffer can only be set up using the framebuffer mailbox. The properties mailbox isn't implemented
• The ARM1176 emulation is incomplete. (I noticed this because I'm using the vector base address register in the system coprocessor. Any reads/writes to c12 give an undefined instruction exception)

For the two first points, these have been addressed, I will commit the changes soon.
The last point is interesting, as it seems that's what prevents RiscOS from booting (it encounters an undefined instruction exception as well). Thanks for pointing that out.

There's a few things qemu doesn't like that ought to be valid for an 1176. Another one that might be biting you is clrex.

[Torlus]

Hi,
I just commited my recent work on the framebuffer, VC->ARM property mailbox, eMMC host, as well as cp15-c12 register handling, as suggested by BrianW, and now RiscOS is booting fine.
I have yet to try other operating systems.
Get it here if you want to have a look: https://github.com/Torlus/qemu/tree/rpi

@tufty : could you provide an example ?

[benzn]

Just chiming in to say that I pulled from the latest branch and recompiled and raspbian no longer hangs! I haven't been pulling regularly so not sure when this was fixed, but it appears to be. Thanks again for all your hard work!

[Torlus]

Glad to hear it's working for you now... However I have no idea why
I hope my next commits won't break it again.

[ShiftPlusOne]

Finally got around to having a go at it. Has anyone had any luck with the kernel that comes with archlinux? I just get a black screen. However, switching to the raspbian kernel works.

Then when raspi-config is started, qemu seem to ignore all keyboard input. I killed raspi-config though serial, but still nothing. Same problem when running the arch image with the raspbian kernel.

Code: Select all

Configured with: './configure' '--prefix=/home/shift/usr' '--target-list=arm-linux-user armeb-linux-user arm-softmmu'  '--enable-sdl' '--audio-drv-list=oss alsa sdl' '--audio-card-list=ac97 es1370 sb16 cs4231a adlib gus hda'

Code: Select all

qemu-system-arm -kernel raspboot/kernel.img -cpu arm1176 -m 512 -M raspi -serial stdio -append "rw dma.dmachans=0x7f35 bcm2708_fb.fbwidth=1024 bcm2708_fb.fbheight=768 bcm2708.boardrev=0x2 bcm2708.serial=0xeditedout smsc95xx.macaddr=editedout sdhci-bcm2708.emmc_clock_freq=100000000 vc_mem.mem_base=0x1c000000 vc_mem.mem_size=0x20000000 dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait" -snapshot -d guest_errors -sd 2012-12-16-wheezy-raspbian.img

[Torlus]

Finally got around to having a go at it. Has anyone had any luck with the kernel that comes with archlinux? I just get a black screen. However, switching to the raspbian kernel works.

Then when raspi-config is started, qemu seem to ignore all keyboard input. I killed raspi-config though serial, but still nothing. Same problem when running the arch image with the raspbian kernel.

I will give a try to archlinux kernel. For the other issue, this is normal at the current stage of the emulation, as it lacks USB controller support. You can still run it from serial, though.

[ShiftPlusOne]

Makes sense, thanks.

[DavidS]

Beings that it will be a long time before it will be possible to get the same performance out of an emulation of the RPi as you get from a real RPi; I must how much longer you think it will be before this enulation reaches a point where it could run all the supported OS's on the RPi. I think that this may be a good way to get many people I know here in the USA that are running RISC OS on either older HW or under Emulation to look at buying an RPi, as some of them are to far away to show them my RPi. So if you can get it to run RISC OS, i will be seeing about getting them to look at it to see if they might be interested in purching some RPi's .

And of course beings that they use RISC OS, you probably already surmized that they are very much into low level hacking (bare metal programming and HW modding).

[BrianW]

Hi,
I just commited my recent work on the framebuffer, VC->ARM property mailbox, eMMC host, as well as cp15-c12 register handling, as suggested by BrianW, and now RiscOS is booting fine.
I have yet to try other operating systems.
Get it here if you want to have a look: https://github.com/Torlus/qemu/tree/rpi

I tried my homebrew kernel in it. It mostly works; the only minor difference I noticed was that on boot, a Raspberry Pi configures a framebuffer. If queried via the properties mailbox, it will return the physical size of this framebuffer (the virtual size is 2x2, which is used to display the 4-colour splash screen). I was using this to configure the framebuffer.

However, it's an easy matter to make it pick a screen size if the emulated Pi doesn't return one. My code now works on both a real Raspberry Pi and qemu.

Very impressive work.

[Torlus]

I added some basic USB emulation. Works on Linux, and provides keyboard support.

Mouse support exists, but is sadly awful in many cases (works quite fine on some setups, but is unusable in most cases).
The main reason for this is that scheduling is done in software with a combination of FIQ and IRQ, and there are a LOT of them (one FIQ per USB Start-of-frame basically).

There are some fixes in the eMMC controller as well, regarding interrupt generation.

[asb]

This looks great. Are you aware of the earlier work by ricky26? https://github.com/ricky26/QEMU/commits/master