Documentation for VC6

[6by9]

Legacy = old framebuffer APIs, and firmware based GLES on Pi0-3 (no 3D on 4)

F(ake) KMS (vc4-fkms-v3d) = ARM side GL driver, firmware driven DRM/KMS for composition.
This is the default configuration on the Pi4.

(Full) KMS (vc4-kms-v3d) = ARM side GL drivers, and ARM side DRM?KMS drivers for composition.
Only currently available on Pi0-3, but unlikely to be usable on 0&1.
This also drops all support for DispmanX, MMAL, or IL being able to add anything to the output, so raspivid, omxplayer, and many other things will stop working.

[pik33]

This also drops all support for DispmanX, MMAL, or IL being able to add anything to the output, so raspivid, omxplayer, and many other things will stop working.

So let the KMS remains "fake" : these things defined the Raspberry Pi. Without them it will be "yet another (small slow) PC"

[6by9]

So let the KMS remains "fake" : these things defined the Raspberry Pi. Without them it will be "yet another (small slow) PC"

There are alternative apis for rendering, and they should be transferable to any mainline Linux machine too.

MMAL and IL remain for all functions other than rendering.

[gustafla]

Supporting DRM is just great in my opinion and I don't miss having to use dispmanx. With the new driver SDL2+GLES2.0 is a feasible game/demo development platform and it simplifies things. I no longer have to use SDL with dummy video in the background to play audio and capture input, it can also do output and works the same way on every machine I own. Including Wayland (I'm posting this from Sway on a Pi4).

[graphicw]

Development is very much active - https://gitlab.freedesktop.org/mesa/mes ... rivers/v3d

In all fairness, the majority of the most recent commits are of code written months ago. This driver is among one of the most important things to getting the Raspberry Pi 4 up to it's full capability. Does a binary blob with full functionality for the VC6 exist at this time? So many hate using them, but at times, it is the best option and anybody that has used Linux for any length of time should be used to binary blobs for GPU's. It could be a considerable while before the open source driver fully catches up should a blob exist. If no blob, then it will indeed be a while before we see what the VC6 is capable of.

[graphicw]

Checking the master branch shows much more recent progress as in the last few hours. I also have found me an IRC channel to check out. I rescind what I said in the previous post as there is indeed some work in progress and it appears there is a Mesa 19.2 RC 3 that was just released incorporating the driver.

[Gavinmc42]

Checking the master branch shows much more recent progress as in the last few hours.

Chasing a moving target these days.
But not every release has Pi VC6 stuff, lots of small print to read

Might not be any blobs any more for Broadcom's newest chips?
The industry seems to have accepted the open source ways.
The previous VC5 was a lot of Eric's Mesa driver work.
Or maybe BCM said "ok, we'll give you a discount if you don't need a blob"
Will it ever be revealed in Eben's Memoir or the Pi history books?

How much is still blobbed for the camera, that's gone Open?
These is a binary blob for the WiFi/BT chip?

[jamesh]

Checking the master branch shows much more recent progress as in the last few hours.

Chasing a moving target these days.
But not every release has Pi VC6 stuff, lots of small print to read

Might not be any blobs any more for Broadcom's newest chips?
The industry seems to have accepted the open source ways.
The previous VC5 was a lot of Eric's Mesa driver work.
Or maybe BCM said "ok, we'll give you a discount if you don't need a blob"
Will it ever be revealed in Eben's Memoir or the Pi history books?

How much is still blobbed for the camera, that's gone Open?
These is a binary blob for the WiFi/BT chip?

I've said it before - please re-read your own posts before posting to at least make sure they make some sense.


Binary Blobs on the Pi4, all closed source.

Bootloader
GPU
USB chip (VLI)
Wifi chip (Broadcom)


We are moving away from the GPU binary blob, which is the reason for moving to standards like Mesa/DRM. We are not developing the GPU 3D in any way for Pi4 , there is now ONLY the Mesa driver, which is under constant development (along with MESA/DRM/KMS themselves). The camera, H264 etc codecs still require the GPU blob. H265 is Linux side only so nothing to do with the GPU.

[Technocolour]

We are not developing the GPU 3D in any way for Pi4 , there is now ONLY the Mesa driver, which is under constant development (along with MESA/DRM/KMS themselves). The camera, H264 etc codecs still require the GPU blob. H265 is Linux side only so nothing to do with the GPU.

Ok.

So by "not developing GPU 3D" for the RPi4B. The RPi team does driver work to interface with, but doesn't contribute code to, MESA/DRM/KMS? Is this understanding correct, or am I being silly? (Well, even more silly than I already am)

[jamesh]

We are not developing the GPU 3D in any way for Pi4 , there is now ONLY the Mesa driver, which is under constant development (along with MESA/DRM/KMS themselves). The camera, H264 etc codecs still require the GPU blob. H265 is Linux side only so nothing to do with the GPU.

Ok.

So by "not developing GPU 3D" for the RPi4B. The RPi team does driver work to interface with, but doesn't contribute code to, MESA/DRM/KMS? Is this understanding correct, or am I being silly? (Well, even more silly than I already am)

Prior to the Pi4, the GPU could handle all the 3D, so you have the choice of GPU 3D or (F)KMS/DRM/MESA. On the Pi4 there is only (F)KMS/DRM/MESA, so the GPU code for driving the 3D is removed and no more work will be done on it.

So all work we do will now be on (F)KMS/DRM/MESA.

Note, FKMS uses the GPU for compositing (it uses dispmanx), so still involves use of the GPU in that respect , but the 3D is driven by the ARM.

[PeterO]

So all work we do will now be on (F)KMS/DRM/MESA.

I don't believe there is anything called "(F)KMS" ! Is it KMS or FKMS or Full KMS or Fake FMS you are talking about ? Someone needs to come up with better names !

And I thought the hardware still did openGLES 3.0 so saying "but the 3D is driven by the ARM" is confusing as well.

PeterO

[graphicw]

We are not developing the GPU 3D in any way for Pi4 , there is now ONLY the Mesa driver, which is under constant development (along with MESA/DRM/KMS themselves). The camera, H264 etc codecs still require the GPU blob. H265 is Linux side only so nothing to do with the GPU.

Ok.

So by "not developing GPU 3D" for the RPi4B. The RPi team does driver work to interface with, but doesn't contribute code to, MESA/DRM/KMS? Is this understanding correct, or am I being silly? (Well, even more silly than I already am)

Prior to the Pi4, the GPU could handle all the 3D, so you have the choice of GPU 3D or (F)KMS/DRM/MESA. On the Pi4 there is only (F)KMS/DRM/MESA, so the GPU code for driving the 3D is removed and no more work will be done on it.

So all work we do will now be on (F)KMS/DRM/MESA.

Note, FKMS uses the GPU for compositing (it uses dispmanx), so still involves use of the GPU in that respect , but the 3D is driven by the ARM.

Isn't the point of a GPU the handling of the 3D? Would there not be a performance penalty by handling 3D on the CPU (Arm)? This sort of defies the assumed logic of the GPU. Is this decision based on a limitation of the SOC or was this by design in the architecture?

[graphicw]

I have one other question as well. On other Linux computers, I have heard of DKMS (Dynamic Kernel Module Support) but not FKMS. FKMS seems to be a Pi specific animal. Is it a re-spin of DKMS specific to Broadcom? I have dealt primarily with Nvidia GPU's in Linux since the tend to have the best drivers when using the blobs. Nvidia GPU's also have a very good open source implementation that is quite mature as well and for most functions work as well as the blob. We cannot say the same for VC6 as of yet though it looks like there is a good progress.

[ShiftPlusOne]

I have one other question as well. On other Linux computers, I have heard of DKMS (Dynamic Kernel Module Support) but not FKMS. FKMS seems to be a Pi specific animal. Is it a re-spin of DKMS specific to Broadcom? I have dealt primarily with Nvidia GPU's in Linux since the tend to have the best drivers when using the blobs. Nvidia GPU's also have a very good open source implementation that is quite mature as well and for most functions work as well as the blob. We cannot say the same for VC6 as of yet though it looks like there is a good progress.

Not related to DKMS.

https://www.kernel.org/doc/html/v4.15/gpu/drm-kms.html

AIUI, F is for fake or firmware because it sits on top of the firmware rather than doing all the work itself.

[PeterO]

https://www.kernel.org/doc/html/v4.15/gpu/drm-kms.html

I'm not sure why you think that posting a link to that huge page of kernel developers documentation is supposed to help explain things to non-kernal developers ?

There is a growing problem here, that those in the know about the new graphics hardware/firmware/software seem to be unable to provide simple explanations of the features provided by each component, and which components are available on which versions of the Pi.

I've asked for a simple diagram or table on several occasions now but now has ever been produced. The acronym soup seems to be getting thicker.

You can see from the questions that are being asked that there is a level of confusion about how the whole graphics system works that was not present before the Pi4 appeared. The changes that come with the Pi4 have not been well explained as far as I can see.

PeterO

[Technocolour]

Isn't the point of a GPU the handling of the 3D? Would there not be a performance penalty by handling 3D on the CPU (Arm)? This sort of defies the assumed logic of the GPU. Is this decision based on a limitation of the SOC or was this by design in the architecture?

I think that refers to the VideoCores VPU "CPU" units handing the sending of geometry data/ data locations and instructions to the GPU part (VC4) vs the ARM A72 CPU (RPi4 & VC6). The VPU block still does compositioning on the RPi4.

This as the VPUs aren't publicly documented (relies on binary blobs), and having the main CPU do the above tasks are how the rest of the computing world generally works, so this reduces the support effort.

But I could be wrong.

[graphicw]

https://www.kernel.org/doc/html/v4.15/gpu/drm-kms.html

I'm not sure why you think that posting a link to that huge page of kernel developers documentation is supposed to help explain things to non-kernal developers ?

There is a growing problem here, that those in the know about the new graphics hardware/firmware/software seem to be unable to provide simple explanations of the features provided by each component, and which components are available on which versions of the Pi.

I've asked for a simple diagram or table on several occasions now but now has ever been produced. The acronym soup seems to be getting thicker.

You can see from the questions that are being asked that there is a level of confusion about how the whole graphics system works that was not present before the Pi4 appeared. The changes that come with the Pi4 have not been well explained as far as I can see.

PeterO

The linked document does provide some explanation as to why the change were made. This same thing is beginning to show up in desktop Linux as well. Just about all the Intel graphics stacks work in a similar manner (in Linux anyway). AMD is still along the more traditional route but Nvidia appears to be now heading in this new direction as well. This is probably going to be the future of Graphics computing. Much as it appears as though Arm will be the future of desktop computing. I have a whole new way of computing and coding to learn but this will bring back the fun of the good old days to me again.

[graphicw]

Isn't the point of a GPU the handling of the 3D? Would there not be a performance penalty by handling 3D on the CPU (Arm)? This sort of defies the assumed logic of the GPU. Is this decision based on a limitation of the SOC or was this by design in the architecture?

I think that refers to the VideoCores VPU "CPU" units handing the sending of geometry data/ data locations and instructions to the GPU part (VC4) vs the ARM A72 CPU (RPi4 & VC6). The VPU block still does compositioning on the RPi4.

This as the VPUs aren't publicly documented (relies on binary blobs), and having the main CPU do the above tasks are how the rest of the computing world generally works, so this reduces the support effort.

But I could be wrong.

That does indeed make sense. In essence, the VPUs function similar to CUDA if I am reading what you have posted correctly. Makes sense, but those are not the same as a CPU in a traditional sense of the what it used to be.

[ShiftPlusOne]

I'm not sure why you think that posting a link to that huge page of kernel developers documentation is supposed to help explain things to non-kernal developers ?

It was more of a "this is the KMS that's being referred to", not an attempt to explain what it is to non-kernel developers.

Kernel Mode Setting (KMS) is a method for setting display resolution and depth in the kernel space rather than user space.

The Linux kernel's implementation of KMS enables native resolution in the framebuffer and allows for instant console (tty) switching. KMS also enables newer technologies (such as DRI2) which will help reduce artifacts and increase 3D performance, even kernel space power-saving.
Note: The proprietary NVIDIA driver (since 364.12) also implements kernel mode-setting, but it does not use the built-in kernel implementation and it lacks an fbdev driver for the high-resolution console.

Background

Previously, setting up the video card was the job of the X server. Because of this, it was not easily possible to have fancy graphics in virtual consoles. Also, each time a switch from X to a virtual console was made (Ctrl+Alt+F2), the server had to give control over the video card to the kernel, which was slow and caused flickering. The same "painful" process happened when the control was given back to the X server (Alt+F7 when X runs in VT7).

With Kernel Mode Setting (KMS), the kernel is now able to set the mode of the video card. This makes fancy graphics during bootup, virtual console and X fast switching possible, among other things.

https://wiki.archlinux.org/index.php/ke ... de_setting

[jamesh]

I used (f)KMS to indicate I was talking about both KMS and FKMS. FKMS being the Fake/Firmware KMS, in that instead of talking direct to hardware, its talks to dispmanx on the Videocore, which then talks to the hardware.

It does appear some sort of documentation would be in order to explain this more fully (although its mostly of academic interest since I suspect it won;t help 'development'). I just need to find someone who understands it fully to explain it to me so I can write it...

[graphicw]

I used (f)KMS to indicate I was talking about both KMS and FKMS. FKMS being the Fake/Firmware KMS, in that instead of talking direct to hardware, its talks to dispmanx on the Videocore, which then talks to the hardware.

It does appear some sort of documentation would be in order to explain this more fully (although its mostly of academic interest since I suspect it won;t help 'development'). I just need to find someone who understands it fully to explain it to me so I can write it...

True it is of mostly academic interest but the information can help in setting a target as it would give one somewhat of an idea of hardware capabilities when coding. A block diagram showing how general workflow is handled on chip would help as well. Are we looking at a situation of VPUs directly assisting the CPU in many tasks?

[graphicw]

I'm not sure why you think that posting a link to that huge page of kernel developers documentation is supposed to help explain things to non-kernal developers ?

It was more of a "this is the KMS that's being referred to", not an attempt to explain what it is to non-kernel developers.

Kernel Mode Setting (KMS) is a method for setting display resolution and depth in the kernel space rather than user space.

The Linux kernel's implementation of KMS enables native resolution in the framebuffer and allows for instant console (tty) switching. KMS also enables newer technologies (such as DRI2) which will help reduce artifacts and increase 3D performance, even kernel space power-saving.
Note: The proprietary NVIDIA driver (since 364.12) also implements kernel mode-setting, but it does not use the built-in kernel implementation and it lacks an fbdev driver for the high-resolution console.

Background

Previously, setting up the video card was the job of the X server. Because of this, it was not easily possible to have fancy graphics in virtual consoles. Also, each time a switch from X to a virtual console was made (Ctrl+Alt+F2), the server had to give control over the video card to the kernel, which was slow and caused flickering. The same "painful" process happened when the control was given back to the X server (Alt+F7 when X runs in VT7).

With Kernel Mode Setting (KMS), the kernel is now able to set the mode of the video card. This makes fancy graphics during bootup, virtual console and X fast switching possible, among other things.

https://wiki.archlinux.org/index.php/ke ... de_setting

That link explains it much better and shows I was tracking in the right direction. Still don't have the whole picture yet, but at least I am sort of piecing it together. That link even explained what I had mentioned in regard to the AMD blob drivers and their lack of support. Thanks