Updated 64-bit Gentoo Image for RPi3 Released (now also for RPi3B+ and RPi4B)

[code_exec]

I tried Fedora 28 and 29 - both don't work on my B+
29 didn't reach the gui
28 started a frozen gui

Use one of the spins such as XFCE which use lighter desktop environments.

[sakaki]

Slight delay with the 1.3.1 image release, unfortunately. I've had to update the kernel due to an issue leading to the wrong (i.e. upstream rather than downstream) versions of the MMC drivers being used (thanks to GitHub user burnbabyburn for bringing this to my attention, and providing a PR to fix it). I've switched the image to using the patched kernel, and it seems to work OK, but I'm going to give it a few more days burn-in before release. Everything in userland is frozen, so it should be out shortly.

Best, sakaki ><

[sakaki]

Hello,

I've just posted a v1.3.1 release of my bootable 64-bit Gentoo image for the RPi3 (model B and B+) on GitHub (here, includes full download instructions).

A changelog from the prior release image (with upgrade instructions) may be viewed here, but in summary:

• Added the full-scale www-client/chromium web browser (in addition to www-client/firefox, which remains supported and included on the image). Note that the version on the image (71.0.3578.62-r2) uses a slightly modified ebuild, which adds the openh264 USE flag, and is built with "-proprietary-codecs -openh264" USE (this removes all the (patent encumbered) h264 code from the application, so it may be lawfully distributed in binary form). You can rebuild it yourself with modified USE flags yourself if you wish, of course.
• The minimum supported microSD card capacity has been increased to 16GB (up from 8GB). In reality, with recent versions of the image 8GB cards were only really useful for a quick tryout; they were too small to properly genup etc., and 16GB cards are now readily available and relatively inexpensive. The initial (uncompressed) image size is now 11,328MiB (of which the root partition is 11,264MiB (11GiB), which will be autoexpanded to fill any additional free space on first boot of course), so this provides reasonable headroom. The current image is ~1.32GiB in compressed form.
• Added the pyconfig_gen utility, for easy (GUI-based) update of (a subset of the features contained in) the /boot/config.txt file on the RPi3. This utility is autostarted on first boot for each user. Changes made to the /boot/config.txt file using this application are subject to ratification on reboot; if the user does not explicitly accept them before a short time limit expires, an automatic reboot under the prior, 'last known good' config is performed. This prevents users locking themselves out by specifying (e.g.) an HDMI mode that is unsupported by their display (as an additional precaution against this particular case, only modes reported as being supported by the currently connected display are presented for selection in the GUI). Currently, only a small (but hopefully, useful) subset of the options available may be edited via this application, but this will be increased in future. It is still possible to directly edit /boot/config.txt - the app ignores keys it does not manage. Tool screenshot may be seen here.
• As a bonus side-effect of the above, the PyQt5 framework - including the Qt5 Designer GUI editor - has been added to the image.
• Added an Xfce4 startup service, xfce-extra/xfce4-noblank, to prevent the user's xscreensaver blanking the display when notionally 'off', since this can cause problems with some displays (under vc4-{f,}kms-v3d graphics drivers, at least) where the display cannot be powered back up once blanked in this manner (by regular mouse or keyboard activity, anyhow). Installs an /etc/xdg/autostart/xfce4-noblank.desktop entry.
• Updated the kernel to sys-kernel/bcmrpi3-kernel-bis-bin-4.14.90.20181222 (and the boot firmware to sys-boot/rpi3-64bit-firmware-1.20181112).
• Various minor ebuild tidy-ups.
• All packages brought up-to-date against the Gentoo tree, as of 26 November 2018.

A variant image for the Pi-Top v1 (an RPi3-based DIY laptop) is also included.

Here's a screenshot of the (vanilla) image in use, on an RPi3B+:



Have fun ^-^

And, as always, any problems or comments, please post either in this thread, or in the project's (sticky) thread on the Gentoo forums (here).

PS: the isshoni.org binhost (and mirror) has been frozen vs the Gentoo tree since around a month ago, while this image has been in prep. Now the image is released, the autobuild process will be allowed to run again, so there should be an updated Portage tree plus updated binary package set available shortly, and the normal weekly-cadence service will resume thereafter. Apologies for any disruption this has caused ><

[sakaki]

If you'd like to try out .NET applications on your gentoo-on-rpi3-64bit system, you may be interested to know I've just pushed a build of dev-lang/mono-4.8.0.524 to the binhost (mono is a an open source implementation of Microsoft's .NET Framework).

To emerge it (assuming you're starting from a reasonably up-to-date version of the image), just issue:

Code: Select all

demouser@pi64 ~ $ emaint sync --repo rpi3
demouser@pi64 ~ $ emerge -av dev-lang/mono

This is a binary package, so shouldn't take too long to install.

Then you can try it. The bundled C# compiler is called mcs. So for a simple 'hello world' (following this post):

Code: Select all

demouser@pi64 ~ $ mkdir HelloWorld && cd HelloWorld
demouser@pi64 ~/HelloWorld $ nano -w HelloWorld.cs

Put the following text in that file:

Code: Select all

using System;

public class HelloWorld
{
    public static void Main()
    {
        Console.WriteLine("Hello World!");
    }
}

Save, and exit nano. Then:

Code: Select all

demouser@pi64 ~/HelloWorld $ TERM=xterm mcs HelloWorld.cs

(The TERM=xterm is to ensure you don't get hit by bug 6752; haven't got the 5.x series to build on aarch64 yet)

You should now have a HelloWorld.exe .NET program. To run it:

Code: Select all

demouser@pi64 ~/HelloWorld $ TERM=xterm mono HelloWorld.exe
Hello World!

best,

sakaki

[Gavinmc42]

.NET? wow.
Ok, now I want to know how you fit everything on your SD card.
What is your Pi setup?
My 32GB card is getting full and with so much stuff now working in Gentoo64 I need to upgrade storage.

Is there a simple way to assign the home demouser folder to USB stick?

I am trying to get Pony lang to compile but it needs LLVM version 3.9.1 and we have 7.0.0.

[Gavinmc42]

Sakaki, got a message trying to emerge mono.

"Unable to find kernel sources at /usr/src/linux" ...
see http://bugs.gentoo.org/261869

Type "mcs" and I get "Unhandled Exception:......

[sakaki]

Sakaki, got a message trying to emerge mono.

"Unable to find kernel sources at /usr/src/linux" ...
see http://bugs.gentoo.org/261869

Type "mcs" and I get "Unhandled Exception:......

The first message, about kernel sources, is a warning, not an error. The image, as shipped, uses a binary kernel package, so there are (by default) no sources at /usr/src/linux for the ebuild to check. It is a fairly common thing to see on Gentoo. The ebuild is trying, per bug 261869, to ensure your kernel has CONFIG_SYSVIPC enabled, as this is a build requirement. Fortunately, the kernel shipped with the image does have this set. The ebuild assumes you know what you are doing in this situation, outputs the message, and keeps going. The dev-lang/mono package did install successfully on your system - as otherwise there would be no "mcs" binary for you to experience your second issue with ^-^.

And as to that second issue (the exception), it is actually just due to mono not recognizing your terminal type (due to an underlying terminfo issue). It is the bug 6752 I mentioned in the above post. To work around it, either run mcs and mono in a vanilla xterm (Applications -> System -> XTerm) or, as suggested in my post above (and in the #6752 bug thread) prefix the command with "TERM=xterm". So, instead of

Code: Select all

demouser@pi64 ~ $ mcs Foo.cs
demouser@pi64 ~ $ mono Foo.exe

you would instead use:

Code: Select all

demouser@pi64 ~ $ TERM=xterm mcs Foo.cs
demouser@pi64 ~ $ TERM=xterm mono Foo.exe

If typing the TERM=xterm prefix gets annoying, you can always alias mcs and mono (and any other affected executables). To do so, as explained e.g. here, you could put the following at the bottom of your ~/.bashrc:

Code: Select all

alias mcs='TERM=xterm mcs'
alias mono='TERM=xterm mono'

Open a new terminal, and you should find that you can now simply type:

Code: Select all

demouser@pi64 ~ $ mcs Foo.cs
demouser@pi64 ~ $ mono Foo.exe

and everything should work (provided you have a Foo.cs of course!)

Ok, now I want to know how you fit everything on your SD card.
What is your Pi setup?
My 32GB card is getting full and with so much stuff now working in Gentoo64 I need to upgrade storage.

Mostly now for emerges etc. I work on a QEMU binfmt_misc aarch64 chroot on a PC workstation. As explained in my wiki article about it (here), this allows me to have a large tmpfs-based /var/tmp/portage, and sufficient memory that swapping is not necessary during builds. Root (and boot) for the build system is on a 250GB Samsung SSD connected via a Sabrent USB adaptor - when building on a PC this can take advantage of the USB3 interface speed. For odd packages that won't build in this way, I simply boot the same image on a regular RPi3B+, and, where possible, distribute the build using distcc (as explained here ff.)

For testing, I currently write the image to a 64GB Sandisk Extreme Pro class A2 microSD card, pretty happy with the speed of this.

Is there a simple way to assign the home demouser folder to USB stick?

You can do this simply by recursively copying the contents of (e.g.) /home to an ext4 partition on the USB drive, and then adding an entry to /etc/fstab to automount it on boot, and finally clearing the old /home to save space. Make sure to use UUID-based partition naming in /etc/fstab for robustness. See e.g. this post and this wiki entry.

hth, sakaki

[Gavinmc42]

Hi Sakaki,

I found the pre-built binaries for Clang/LLVM 3.9.1.
It looks like AArch64 has pre-builts start at version 3.5.

Is there a way that emerge can be used to install multiple ebuild versions?
To build Pony I have to set the compiler to 3.9.1, but I don't want to uninstalled 7.0.0

Distcc looks interesting, your wiki now has lots of useful info.
Thanks again for your help.

Regards
Gavin

[sakaki]

I found the pre-built binaries for Clang/LLVM 3.9.1.
It looks like AArch64 has pre-builts start at version 3.5.

Is there a way that emerge can be used to install multiple ebuild versions?
To build Pony I have to set the compiler to 3.9.1, but I don't want to uninstalled 7.0.0

Yes, you can, using a nice Portage feature called slotting. Not all packages support this, but, where they do, it is (usually, in the absence of masking conflicts) possible to have multiple versions of the same software installed simultaneously on your system. See e.g. here for further details. And fortunately, LLVM/Clang is slotted on Gentoo; you can see this if you use the (bundled) eix tool on your gentoo-on-rpi3-64bit system:

Code: Select all

demouser@pi64 ~ $ eix --exact sys-devel/llvm
[I] sys-devel/llvm
     Available versions:  
     (0)    ~*3.4.2-r100(0/3.4) ~*3.5.2-r100(0/3.5) ~*3.6.2-r100(0/3.6) ~*3.7.1-r3(0/3.7.1) ~*3.9.1-r1(0/3.9.1)
     (4)    (~)4.0.1-r1
     (5)    (~)5.0.2
     (6)    6.0.1^t{tbz2} [M](~)6.0.1-r2^t
     (7)    (~)7.0.1^t{tbz2} **7.0.9999^t
     (8)    **9999^t
       {clang debug default-compiler-rt default-libcxx doc exegesis gold libedit +libffi lldb multitarget ncurses ocaml python +sanitize +static-analyzer test xar xml ABI_MIPS="n32 n64 o32" ABI_PPC="32 64" ABI_S390="32 64" ABI_X86="32 64 x32" ELIBC="musl" KERNEL="Darwin FreeBSD linux" LLVM_TARGETS="AArch64 AMDGPU ARM AVR BPF Hexagon Lanai MSP430 Mips NVPTX Nios2 PowerPC RISCV Sparc SystemZ WebAssembly X86 XCore" PYTHON_TARGETS="python2_7" VIDEO_CARDS="radeon"}
     Installed versions:  6.0.1(6)^t{tbz2}(18:01:22 10/01/19)(libffi ncurses xml -debug -doc -gold -libedit -test -xar ABI_MIPS="-n32 -n64 -o32" ABI_PPC="-32 -64" ABI_S390="-32 -64" ABI_X86="-32 -64 -x32" KERNEL="-Darwin" LLVM_TARGETS="AArch64 BPF -AMDGPU -ARM -Hexagon -Lanai -MSP430 -Mips -NVPTX -PowerPC -Sparc -SystemZ -X86 -XCore")
     Homepage:            https://llvm.org/
     Description:         Low Level Virtual Machine

(Your output may vary slightly as to versions etc)

Those parenthesized leading numbers on each line are the (major) slots, so you could have e.g. llvm v4 thru v7 installed simultaneously if you wanted (the same is true for Clang, use "eix --exact sys-devel/clang" to check).

However, as Neddy notes here, (multi-)slotting for Clang/LLVM is only available from v4 onwards - while the version you say you need (3.9.1) is available, it's only available in the default (0) slot (see the eix output above), and cannot be simultaneously installed with other versions, which is a bit annoying.

But all is not lost: looking at the above eix output, you can see that binary packages are available (the 'tbz2' tag tells you this) on the binhost for slots 6 and 7 (and a slot 7 instance is currently installed). Might pony be buildable with one of these versions of LLVM?

Well, looking at the pony README (here), we see that stefantalpalaru maintains an overlay (user repository of ebuilds) with a pony-0.25.0 ebuild in there. Looking at the relevant ebuild in that repo, we can see the following dependencies listed:

Code: Select all

RDEPEND="dev-libs/libpcre2
	dev-libs/openssl:=
	>=sys-devel/llvm-3.9.1:=
	<sys-devel/llvm-7.0.0:=
	sys-libs/ncurses:=
	sys-libs/zlib
	vim-syntax? ( app-vim/pony-syntax )"
DEPEND="${RDEPEND}
	virtual/pkgconfig"

That's encouraging, looks like stefantalpalaru claims it will build for 3.9.1 <= version < 7.0.0, so 6.0.1 (for which a binary package is available, per the above eix output) should work.

At this stage, it would be possible to use this ebuild to install pony, by:

• emerging app-portage/layman (an overlay management tool), if not already installed
• adding stefantalpalaru's overlay (via "layman -a stefantalpalaru")
• masking all packages in it, so there are no conflicts with the base system (see e.g. my notes on overlay hygiene here)
• unmasking the package you want (dev-lang/pony-0.25.0)
• adding a package.accept_keywords entry for pony (the ebuild doesn't keyword ~arm64)
• emerging dev-lang/pony

There are quite a few steps to do that however, so I've copied across stefantalpalaru's ebuild into the rpi3 overlay already used by the image, and keyworded it. So, you can just follow the instructions below to install it.

Installing dev-lang/pony on a gentoo-on-rpi3-64bit System

I assume you're working from a reasonably up-to-date version of the image in what follows, ideally 1.3.1.

Open a terminal and issue:

Code: Select all

demouser@pi64 ~ $ sudo emaint sync --repo rpi3

to pick up the pony ebuild (and other changes), then:

Code: Select all

demouser@pi64 ~ $ sudo emerge -av dev-lang/pony

These are the packages that would be merged, in order:

Calculating dependencies... done!
[binary  NS    ] sys-devel/llvm-6.0.1:6::gentoo [7.0.1:7::gentoo] USE="libffi ncurses xml -debug -doc -gold -libedit -test -xar" LLVM_TARGETS="(AArch64) BPF -AMDGPU -ARM -Hexagon -Lanai -MSP430 -Mips -NVPTX -PowerPC -Sparc -SystemZ -X86 -XCore" 18,611 KiB
[ebuild  N     ] dev-lang/pony-0.25.0::rpi3  USE="-test (-vim-syntax)" 1,346 KiB

Total: 2 packages (1 new, 1 in new slot, 1 binary), Size of downloads: 19,957 KiB

Would you like to merge these packages? [Yes/No] 

As you can see (your output may vary slightly from the above), this is proposing to install llvm-6.0.1 from a binary into a new slot ("[binary NS]"). So you get to keep version 7 of LLVM! Start the above merge, by pressing Enter at the prompt. Once the slot-6 LLVM is installed, it will then build dev-lang/pony from source ("[ebuild N]") using it, but this shouldn't take too long (20 minutes or so - I'll look at adding a binary package for pony itself to the binhost at some point).

Once installed, if you run "eix --exact sys-devel/llvm", you should now see you have two versions of LLVM (in major slots 6 and 7, corresponding to the respective package major version numbers) installed side-by-side. Warning: features like this can make it hard to leave Gentoo once you start getting used to them ^-^

OK, so now you should be able to try out pony! Issue:

Code: Select all

demouser@pi64 ~ $ mkdir hello && cd hello
demouser@pi64 ~/hello $ nano -w main.pony

And put in that file (following this example):

Code: Select all

actor Main
  new create(env: Env) =>
    env.out.print("Hello, world!")

Save, and exit nano. Now build it using ponyc:

Code: Select all

demouser@pi64 ~/hello $ ponyc
Building builtin -> /usr/lib/pony/0.25.0/packages/builtin
Building . -> /home/demouser/hello
Generating
 Reachability
 Selector painting
 Data prototypes
 Data types
 Function prototypes
 Functions
 Descriptors
Optimising
Writing ./hello.o
Linking ./hello

And run the resulting executable:

Code: Select all

demouser@pi64 ~/hello $ ./hello
Hello, world!

Yay!

hth, sakaki

[Gavinmc42]

Thanks Sakaki, your instructions worked perfectly.

Could be a reason why they say 3.9.1.
The first examples I tried after HelloWorld were Mandelbrot and OpenGL-app from the wiki.

Not the only one with LLVM6 and Mandelbrot issues
https://github.com/ponylang/ponyc/issues/2918

I may just wait until 7.0.1 is sorted
https://github.com/ponylang/ponyc/pull/2976

Learning a new language on a new OS and debugging compilers might just be a bit much for me.
Not sure, but you might have been the first to get ponyc compiled on an Aarch64 Pi in any OS?

That's the thing with bleeding edge, you never know how close you are until you fall over it
That's also the fun with Pi's, pushing the boundaries.

[sakaki]

Gavinmc42,

so it looks as if using llvm:6 for this was a bit optimistic. I've just revbumped the pony-0.25.0 ebuild (so it now restricts the max LLVM slot to 5), built the package, and pushed the resulting binaries to the binhost (so as a result, sys-devel/llvm-5.0.2 is now available as a binary package too).

Built this way (i.e., under llvm:5), pony seems to compile the mandelbrot example ok. Steps to install and try it out for yourself:

Code: Select all

demouser@pi64 ~ $ sudo emaint sync --repo rpi3
demouser@pi64 ~ $ sudo emerge -avu dev-lang/pony

This will update pony to 0.25.0-r1 via a binary, pulling in the (also binary) llvm-5.0.2 in the process (your 'other' LLVMs in slots 6 and 7 will not be deleted). Once done, download the mandelbrot test program:

Code: Select all

demouser@pi64 ~ $ mkdir mandelbrot && cd mandelbrot
demouser@pi64 ~/mandelbrot $ wget -c https://raw.githubusercontent.com/ponylang/ponyc/0.25.0/examples/mandelbrot/mandelbrot.pony

And compile it:

Code: Select all

demouser@pi64 ~/mandelbrot $ ponyc
Building builtin -> /usr/lib/pony/0.25.0/packages/builtin
Building . -> /home/demouser/mandelbrot
Building cli -> /usr/lib/pony/0.25.0/packages/cli
Building collections -> /usr/lib/pony/0.25.0/packages/collections
Building ponytest -> /usr/lib/pony/0.25.0/packages/ponytest
Building time -> /usr/lib/pony/0.25.0/packages/time
Building collections/persistent -> /usr/lib/pony/0.25.0/packages/collections/persistent
Building random -> /usr/lib/pony/0.25.0/packages/random
Building buffered -> /usr/lib/pony/0.25.0/packages/buffered
Building files -> /usr/lib/pony/0.25.0/packages/files
Building term -> /usr/lib/pony/0.25.0/packages/term
Building promises -> /usr/lib/pony/0.25.0/packages/promises
Building strings -> /usr/lib/pony/0.25.0/packages/strings
Building signals -> /usr/lib/pony/0.25.0/packages/signals
Building capsicum -> /usr/lib/pony/0.25.0/packages/capsicum
Generating
 Reachability
 Selector painting
 Data prototypes
 Data types
 Function prototypes
 Functions
 Descriptors
Optimising
Writing ./mandelbrot.o
Linking ./mandelbrot

Now you can try running it, with the default parameters, saving the output bitmap to out.dat:

Code: Select all

demouser@pi64 ~/mandelbrot $ time ./mandelbrot --output=out.dat

real	0m17.197s
user	0m57.406s
sys	0m0.258s

demouser@pi64 ~/mandelbrot $ file out.dat
out.dat: Netpbm image data, size = 16000 x 16000, rawbits, bitmap

Warning! That's a huge bitmap - if you try loading it (on your RPi3 anyhow) into a program like Ristretto, it'll create a full-colour-depth canvas to hold it (even though the original is only 1 bit per pixel), run out of memory, and quit. So, to convert it into something more viewable on your RPi3, rescale the image (and convert to a png), using imagemagick's convert tool (available as a binary package):

Code: Select all

demouser@pi64 ~/mandelbrot $ sudo emerge -av virtual/imagemagick-tools
demouser@pi64 ~/mandelbrot $ time convert out.dat -resize 4000x4000 -depth 4 -colorspace Gray out.png

real	4m2.034s
user	1m31.596s
sys	0m15.878s

Conversion took longer than generation ^-^

You could of course build a smaller image in the first place (using e.g. the --width option to ./mandelbrot), but you may have wanted the original parameters for benchmarking etc., which is why I left it as above.

You can now view the resulting png, in e.g. gimp:

Code: Select all

demouser@pi64 ~/mandelbrot $ gimp out.png &>/dev/null&

Screenshot on one of my testing boxes:



hth, sakaki

PS: www-client/firefox-64.0.2 is now out of basic testing and pushed to the binhost. To get it:

Code: Select all

demouser@pi64 ~ $ emaint sync --repo rpi3
demouser@pi64 ~ $ emerge -avu www-client/firefox

[Gavinmc42]

Thanks again Sakaki,
You are like my own personal Gentoo Sensei.

My next requirement for a Gentoo64 Pi development OS was Arduino.
The IDE installed fine but the avr/arm cross compilers keep popping up errors.

missing /usr/bin/avr-g+=: no such file or directory
Arduino SAM boards- "Tool bossac is not available for your operating system"
Arduino SAMD boards- "Tool arduinoOTA is not available for your operating system"
Arduino nRF52- "Tool openocd is not available"

Getting the hang of emerge so openocd is now being installed, maybe

[sakaki]

missing /usr/bin/avr-g+=: no such file or directory

Been a while, but IIRC, you need to build a cross-toolchain separately from the dev-embedded/arduino IDE, via crossdev. A good reference is:
https://wiki.gentoo.org/wiki/Arduino#Pr ... _toolchain

hth, sakaki

[sakaki]

All,

A binary package for www-client/chromium-72.0.3626.28-r2 has just been pushed to the binhost.

To get it (assuming you are on a reasonably modern version of gentoo-on-rpi3-64bit), just issue:

Code: Select all

demouser@pi64 ~ $ sudo emaint sync --repo rpi3
demouser@pi64 ~ $ emerge -avu www-client/chromium

Seems to work reasonably well in basic testing. Please let me know of any issues.

Best, sakaki

PS some notes about the slightly-tweaked ebuild for this package (wrt the main tree, needed to get an arm64 compile) may be read here.

[Gavinmc42]

Both the new Firefox and Chromium seem to be fine up to 10 tabs, Chromium now using less memory?
Swap and zram changes improves things?

WebGLsamples Aquarium still the ultimate stress test.
I think they fail cleaner than the last time I tested, Gentoo still runs, last time i needed to pull power?

What else stress tests Browsers?
What needs to done for Aarch64 Gentoo to be declared stable?
Seems to me it is mostly OpenGL driver issues and low number of ebuilds?

Busy learning about compilers and paths, lots of manual reading, google not much help yet as Aarch64 is so new.
Some automatic install scripts have issues with anything not x86 and cross compiling means learning the build systems
The Free Pascal cross build tools seem simpler than the C/C++ ones.
Nearly got command line fpc working, once that's going then the Lazarus IDE is next

Since lots of languages like Rust/Pony are based on the LLVM tools perhaps check that first for cross compiling tools to Arm, Avr etc.
Wonder if anyone has cross compiled examples/demos on Aarch64 to x86?

Geany works well as a simple generic language editor/IDE
Time to try Codeblocks? Is it x86 only?
I have read Netbeans has a plugin for C++, so that's another IDE to try
Wonder if Visual Studio Code will run on Aarch64? It's mostly nodejs based?
I am going to avoid Eclipse for as long as possible
Might have go Vim/Emacs for Aarch64

[sakaki]

Both the new Firefox and Chromium seem to be fine up to 10 tabs, Chromium now using less memory?
Swap and zram changes improves things?

WebGLsamples Aquarium still the ultimate stress test.
I think they fail cleaner than the last time I tested, Gentoo still runs, last time i needed to pull power?

The aquarium test runs OK for me, doesn't crash on Chromium-72 even with 30,000 webgl fish ^-^ Not the world's fastest frame rate though (I think it is about 2s / frame on vc4-fkms-v3d, 256MiB CMA):



But the cube mapping test runs at a respectable 5fps:



Chromium does appear to use less memory per tab now than for the 5x/6x releases, yes. Additionally, the use of zswap (not zram incidentally, see e.g. this post for details of the differences) helps; mostly tbh when you are running e.g. the browser and another large app, like libreoffice, at the same time, and, say, alt-tabbing between them (a not unrealistic scenario).

best, sakaki

[Gavinmc42]

Yep zswap whatever that is, seems to help
If I read that explanation right, as well as being faster it could save some uSD card wear too?

I got up to 3fps for Aquarium, which is pretty amazing as it did not crash Gentoo.
If you think about it, this could be WebGL code (OpenGLES) converted to OpenGL and then Eric's driver converts it back to OpenGLES?
What do these Pi version browsers do?
Are they doing in all in software using 4 cores plus NEON with no hardware acceleration?
I know Android can run these WebGL demos a bit faster, so still room for tweaking.

Are these two browser versions WASM aware?
Both are usable now, YouTube tests next?
Android does have a separate standalone YouTube viewer, are they available for Pi Aarch64?

Stress test - give it to a kid, if they don't complain, it is good enough
Hmm, cannot play Android games, drawback or bonus for parents?
Android emulation layer?
If people can do x86 emulation why not Android on Gentoo64?
So much still to learn.

[sakaki]

Yep zswap whatever that is, seems to help
If I read that explanation right, as well as being faster it could save some uSD card wear too?

Yes, as in many cases it avoids hitting the backing-store swapfile at all. Not a magic bullet though - if your about-to-be-swapped-out pages are incompressible, it'll add a bit of overhead, and it evicts decompressed swap pages to backing store when the in-memory cache is exhausted. But overall, I find it useful to have turned on.

Are these two browser versions WASM aware?

Both browsers have some core WebAssembly support, yes, although Chromium-72 seems to have the edge over Firefox-64. I tried e.g. the WasmBoy[1] benchmark (using the Tobu Tobu Girl open-source ROM), and Chromium hit around 40fps, while Firefox was around 30fps:



Also interesting, the comparative baseline {Java,Type}script variant in Firefox was just awful, at around 2fps. Not sure why that is, something to look into; it appears to be just using one CPU core when running the benchmark, wheras Chromium has all 4 going; perhaps something I have messed up in about:config ><)

Also, post-MVP WASM browser integration is still work in progress. See for example this overview by Lin Clark et al (one of the nicest bits of technical writing I have ever read, incidentally):



hth, sakaki

PS I've just pushed dev-lang/rust-1.31.1 to the binhost (this package now installs cargo also). To update, simply:

Code: Select all

demouser@pi64 ~ $ sudo emaint sync --repo rpi3
demouser@pi64 ~ $ sudo emerge -av1u virtual/rust virtual/cargo

[1] WasmBoy is a Game Boy / Game Boy Color Emulator written for Web Assembly using AssemblyScript (repo on GitHub here). It has a nice "versus-Typescript" benchmarking tool.

[Gavinmc42]

Again a very informative post.
Yep Lin had very good explanation and it got me thinking and yes someone has already done it.

WASM as native code for FPGA CPU.
https://github.com/WebAssembly/design/issues/1050
https://github.com/lastmjs/wasm-metal

Buried in the Browsers is more Compilers, just when I was thinking I knew about most compilers, they invent new ones and hide them in plain sight.
Do I add Javascript compilers on my list of ones to learn?

Not a magic bullet though - if your about-to-be-swapped-out pages are incompressible, it'll add a bit of overhead, and it evicts decompressed swap pages to backing store when the in-memory cache is exhausted. But overall, I find it useful to have turned on.

It seems to make an improvement, but I don't know how to qualify it, except to say it feels a bit faster and perhaps more reliable?

That Wasmboy benchmark now makes sense, I had heard of it before but had no idea what it referred to.
It looks like Chromium has taken the lead for speed on Pi's, on my Linux Mint box Celeron duo Firefox 64.0.0., I get 188, 54, 55fps.
Is it because cranelift is still turned off in Firefox?
And you are right, those Javascript fps are terrible in Firefox, a quad core AArch64 cpu should be much better.
Even a single core should get better than 2fps?

From Lin's blog, I think browsers still can be better and faster, which will make them more usable on Pi's.
But I am glad I have no idea how to compile or debug them, they are becoming very, very big pieces of software.
Sure glad you know what you are doing

Yep with Rust2018 out, time to update to Rust 1.31.1
Was latest Firefox compiled in the new one? Something changed?
Was wasm in previous version the same fps?
Mozilla targeting x 86 first, ARM a few changes behind?
Check Wasm on Firefox in Android?

[sakaki]

FYI, I've just pushed www-client/firefox-65.0 to the binhost.

Users on the latest v1.3.1 gentoo-on-rpi3-64bit image just need to run genup (as root, or using sudo) to get the updated browser via binary package - or, if you prefer, simply wait for the system's weekly autoupdate to do this for you. (If you have a significant amount of work invested in an older release of this image, I have also provided manual upgrade instructions (from 1.0.0 thru 1.2.2 → 1.3.1) here.)

[Gavinmc42]

Haha, probably just after I genuped up last night?
Did you noticed any improvements with 65.0?

[sakaki]

Haha, probably just after I genuped up last night?
Did you noticed any improvements with 65.0?

Not particularly tbh. There are a few tweaks to webgl and tooling, plus support for AV1 and WebP: see this note and also this one.

Best, sakaki

[Gavinmc42]

What, WebP and AV1? two new codecs I have not heard of
Er looks kind of important, going to be fun to test.

The AV1 encoder rav1e is written in Rust and whoops x86 assembler?
Is it so new there is no Arm AV1 encoder?

I hope RPT have not gone too far down the H.265 licencing path for the next Pi's?
AV1 decoding on Arm probably going to be using all the cores and NEON like h.265?
And as it is open I might learn how codecs work.
Wonder if web cameras will adopt it?

WebP as JPEG replacement? VC4 does JPEG in hardware.
WebP could be slower on Pi's?

The learning never stops

[sakaki]

Hello,

I've just posted a v1.4.0 release of my bootable 64-bit Gentoo image for the RPi3 (model B and B+) on GitHub (here, includes full download instructions).

This release marks a significant milestone for 64-bit OSes on the RPi3, as, leveraging the hard work by 6by9 and others on the rpi-4.19.y kernel, use of the Pi's camera module and the Pi's hardware video codecs are now both supported! [1]

These new features have been requested for some time, and indeed their absence in 64-bit mode has often been cited as a reason to stick with a 32-bit OS. To help showcase them, I've provided two (trivial! [2]) bundled apps, pi-ffcam and pi-ffplay, so you can quickly try them out yourself! Hopefully, the v4l2 m2m codec data paths, on which these apps rely, will soon also be integrated into VLC, Firefox etc. ^-^

Here's a screenshot of the two apps running on an RPi3B+, with an attached Pi camera module v2 (Pi-Top v1 chassis):



A changelog from the prior release image (with upgrade instructions) may be viewed here, but in summary:

• Kernel migrated to rpi-4.19.y (sys-kernel/bcmrpi3-kernel-bis-bin-4.19.25.20190226). As just noted, this represents a significant upgrade, as a number of features (such as v4l2 m2m codec support) are only available in 4.19, and will not (aiui) be backported to 4.14. The boot firmware was also updated to sys-boot/rpi3-64bit-firmware-1.20190213, and the userland libraries to media-libs/raspberrypi-userland-1.20190114.
• Thanks to advice and patches from 6by9, added a version of ffmpeg that can both access the Pi's v4l2 camera (via /dev/video0) and access the Pi's hardware video codec capabilities, via v4l2 m2m (memory-to-memory) interfaces at /dev/video{10,11,12}. The h264_v4l2m2m, mpeg4_v4l2m2m, and (if you have purchased a license key) mpeg2_v4l2m2m codecs all currently work from 64-bit userland. (Note, I have primarily tested decoding, as that is relevant for video playback, but these endpoints should support hw-based encoding too (for h264 and mpeg4, anyhow).)
• Added (via USE flag on media-plugins/gst-plugins-meta) the necessary additional gstreamer library, to allow it to also access the v4l2 m2m endpoints (media-plugins/gst-plugins-v4l2-1.14.4).
• Added the pi-ffcam app, a trivial 'live viewer' for the Pi camera module v1/v2 (available under the "Multimedia" menu, as shown in the screenshot above). This streams an h264-encoded live video feed from the camera, decodes it using h264_v4l2m2m (i.e., a hardware codec, just for fun ^-^), and then displays the output in an SDL desktop window. It is built on ffmpeg / ffplay. Note that due to buffering, the stream has around a 1s lag. (Also note that you wouldn't build a 'real' live view app like this of course; it's main purpose is to provide confidence that the various interfaces are working.)
• The media-tv/v4l-utils package is also included; this provides an app (Multimedia -> Qt V4L2 test Utility) that may also be used (inter alia) to live-view the camera (although I have found its GL output to be unreliable).
• Added the pi-ffplay app, a trivial video player that can utilize the Pi's built-in hardware codecs just described, for h264, mpeg4, and - if you have purchased the license - mpeg2 (this may also be found under the "Multimedia" menu; the cursor is on it in the screenshot above). The app prompts for a video to play, attempts to probe its video stream type, and plays it, forcing (where possible) an appropriate v4l2 m2m hardware codec (stream decode only; actual blit of the rendered bitmaps to the window is still via the regular pipeline). This uses the standard ffplay underneath, so consult its manpage for details (double-click to toggle full screen etc.) If the video contains an audio track, this will be played also. Unfortunately, the other bundled video playback apps (VLC, Kodi, and SMPlayer) do not yet leverage the m2m codec paths by default - at least, not in the versions currently provided. If you know how to make them do this (either by preference file, USE flag or patching) please let me know! It is a similar case (at the moment) for the bundled Firefox and Chromium browser (the latter can use v4l2 m2m, but only in an ozone build context right now). That's a shame - files like http://jell.yfish.us/media/jellyfish-50 ... d-h264.mkv (1920x1080 50Mbps) choke the RPi3 (even B+) under software rendering, but display relatively smoothly when using the v4l2 m2m codecs (you can easily try for yourself, just download the file and try playing it with e.g. VLC or SMPlayer (the latter does the better job), then try again with ff-play). But as making the connection (particularly where ffmpeg or gstreamer are used 'under the hood') is a relatively straightforward matter, these facilities will hopefully be added in relatively short order.
• The default GPU memory setting in /boot/config.txt has been increased, to 128 MiB from the prior (de minimis) 16 MiB. Per 6by9 and my own empirical tests, this should be sufficient to permit both camera and v4l2 m2m codec use simultaneously, for most use cases (as in the above screenshot). The camera interface has also been selected by default (start_x=1). NB: if you do not need to use the codecs (or camera) please feel free to revert these settings (memory being a precious resource on the RPi3!).
• The pyconfig_gen application has been updated; it now allows setting of gpu_mem, and selection of the camera interface (start_x=1).
• Although I have elected to stay with fkms in this release, for regular RPi3 users, 'true' kms (vc4-kms-v3d) should prove an entirely viable option now (and this can easily be selected via the pyconfig_gen application); I have one system that has been running this for weeks now, and it seems very usable (and exhibits maybe 5-10% more GL bandwidth relative to an fkms system, when compared on a like-for-like, no-display-manager-compositing basis). Please note that under vc4-kms-v3d, window manager compositing should be turned off to prevent periodic 'stalling' of the GUI, and so I have modified the existing rpi3-safecompositor service to do this automatically. For avoidance of doubt, fkms - the shipped default - is unaffected.
• Various minor ebuild tidy-ups.
• All packages brought up-to-date against the Gentoo tree, as of 19 February 2019 (which means e.g., www-client/firefox-65.0.1, www-client/chromium-72.0.3626.96-r1, app-office/libreoffice-6.1.5.2 etc. are bundled; a full list of installed packages may be found here).

A variant image for the Pi-Top v1 (an RPi3-based DIY laptop) is also included.

Here's a screenshot of the (vanilla) image in use, on an RPi3B+:



Have fun ^-^

And, as always, any problems or comments, please post either in this thread, or in the project's (sticky) thread on the Gentoo forums (here).

Best,

sakaki

PS: the isshoni.org binhost (and mirror) has been frozen vs the Gentoo tree since 19 February, while this image has been in prep. Now the image is released, the autobuild process will be allowed to run again, so there should be an updated Portage tree plus updated binary package set available shortly, and the normal weekly-cadence service will resume thereafter. Apologies for any disruption this has caused ><

[1] There's still no 64-bit userland MMAL just yet, but things are getting closer; in any event, hardware video codec access is one of the big reasons people wanted MMAL ported to 64-bit in the first place, so getting access to this facility via the v4l2 m2m endpoints instead will obviate the need for it, in many cases ^-^

[2] Once ffmpeg has the necessary v4l2 m2m codec support built in (which the version on the image has) then exploiting these features from the command line is trivial - the provided demo apps are simply wrappers there to lower initial barrier to entry.

See for example the example CLI 'recipes' in this project's open wiki.

Erratum: I've just noticed there is an issue affecting video playback on sites like youtube with the version of chromium (72.0.3626.96-r1) supplied on the 1.4.0 image (the bundled firefox however works fine).

To fix this, by rolling back to the prior version (72.0.3626.28-r2), first ensure you have network connectivity, then open a terminal and issue:

Code: Select all

demouser@pi64 ~ $ sudo emaint sync --repo rpi3
demouser@pi64 ~ $ sudo emerge -vu www-client/chromium dev-embedded/rpi3-64bit-meta

This will install the prior (72.0.3626.28-r2) version from a binary package, so the process shouldn't take long (10 minutes or so).

Apologies, I should have caught that in testing ><

Update 10 March 2019: a new 1.4.1 release of the image is available, which fixes the issue in the above erratum; please see this post for more details.

[cjan]

1. chromium-ozone build does means ozone-wayland patch?
2. chromium is .121 now.