Page 1 of 1

Use BFQ?

Posted: Fri Sep 15, 2017 6:24 pm
by Ollittm
I have kernel 4.12 working just fine compiled from source. However if I try to enable BFQ support, RPi fails to boot. In fact it does not seem to be able to grok the sdcard at all.

In the cmdline.txt I simply changed elevator=deadline to elevator=bfq and compiled the kernel with the BFQ options. There's also some other discussion about scsi option in cmdline and making UDEV rule but neither of these helped.

So, 4.12.11 kernel for RPi just doesn't work with BFQ?

Re: Use BFQ?

Posted: Fri Sep 15, 2017 6:54 pm
by DougieLawson
Since the number of folks on here that will know anything about BFQ can probably be counted on the fingers of one hand, you are much more likely to get the help you need at https://groups.google.com/forum/#!forum/bfq-iosched

Re: Use BFQ?

Posted: Fri Sep 15, 2017 9:24 pm
by asavah
Ollittm wrote:
Fri Sep 15, 2017 6:24 pm
I have kernel 4.12 working just fine compiled from source. However if I try to enable BFQ support, RPi fails to boot. In fact it does not seem to be able to grok the sdcard at all.

In the cmdline.txt I simply changed elevator=deadline to elevator=bfq and compiled the kernel with the BFQ options. There's also some other discussion about scsi option in cmdline and making UDEV rule but neither of these helped.

So, 4.12.11 kernel for RPi just doesn't work with BFQ?
AFAIK mmc driver currently does not support BFQ.
Trying to boot with elevator=bfq resulted in kernel panic for me.

Code: Select all

cat /sys/block/mmcblk0/queue/scheduler
[noop] deadline cfq
As you can see above bfq is not available for mmcblk0 (sdcard)

However for a usb flash I get

Code: Select all

 cat /sys/block/sda/queue/scheduler
[mq-deadline] kyber bfq none
This is from custom 4.13.x kernel built from raspberry pi github sources.

Edit: honestly I'm not sure if HUGE amount of work required to make mmc driver to use blk-mq/scsi-mq (which is required for bfq and friends) is worth it performance wise. There is no MQ support (yet) in mmc driver in kernel but it's being worked on https://www.spinics.net/lists/linux-mmc/msg46129.html

Re: Use BFQ?

Posted: Mon Sep 18, 2017 8:51 am
by Ollittm
Right, well, I already noticed it doesn't work ;-)

But good to know I was not doing anything wrong as such, it's just not supported. I'm not sure how much difference it would really do versus deadline anyways. The writeback throttling on the other hand makes a big difference although of course benchmark wise it makes things worse.

Re: Use BFQ?

Posted: Mon Sep 18, 2017 8:54 am
by Ollittm
asavah wrote:
Fri Sep 15, 2017 9:24 pm
AFAIK mmc driver currently does not support BFQ.
Trying to boot with elevator=bfq resulted in kernel panic for me.

Code: Select all

cat /sys/block/mmcblk0/queue/scheduler
[noop] deadline cfq
As you can see above bfq is not available for mmcblk0 (sdcard)
If you look at google results on getting BFQ to work, this is normal and it should be replaced by the MQ options including BFQ when you have the configuration options set up correctly. Which is what I spent some time struggling with.

OK, not for MMC specifically but you get that result for HDD/SDD as well.

Re: Use BFQ?

Posted: Mon Sep 18, 2017 2:30 pm
by asavah
Ollittm wrote:
Mon Sep 18, 2017 8:54 am
it should be replaced by the MQ options including BFQ when you have the configuration options set up correctly.
The underlying subsystem driver has to have support for CQ/MQ which is not the case for mmc and specifically MMC_BCM2835_MMC on the pi's.

Re: Use BFQ?

Posted: Mon Sep 18, 2017 3:12 pm
by Ollittm
Looks like this was moved to "other". Not quite right as I'm actually using the raspian kernel from Git.

Re: Use BFQ?

Posted: Tue Sep 19, 2017 12:11 am
by asavah
Ollittm wrote:
Mon Sep 18, 2017 3:12 pm
Looks like this was moved to "other". Not quite right as I'm actually using the raspian kernel from Git.
Agreed, at least kernel could get it's own subforum, having it buried in "Operating system distributions" - "other" seems wrong to me.
Linux kernel is not an "Operating system distribution" and dtbs which are actually a part of kernel get their own subforum "Device Tree" .

Something is wrong with mods' logic here.

Re: Use BFQ?

Posted: Tue Sep 19, 2017 8:53 am
by jamesh
There will be some major rearrangements of the forums when time allows, moving the kernel subforum may happen then.

Re: Use BFQ?

Posted: Tue Sep 19, 2017 12:09 pm
by Ollittm
I decided to go back to 4.9.y kernel. Given that the 4.10 and above lag thousands of commits behind 4.9.y, they're not working quite properly. Wifi does not work and neither does bluetooth. Desktop is misbehaving.. I'm sure there are ways to get around it, probably you can just tell git somehow to patch the missing commits in.

I decided to grab 4.9.y and patch the writeback throttling and BFQ in. It appears the "legacy" patches for <4.12 kernels do indeed support single queue operation as well. Theoretically having both enabled is redundant as they try to achieve the same thing in different ways.

On the writeback discussion it would seem that the problem cannot be fixed by IO scheduler as it's the disk buffer that gets overrun when you copy lots of stuff to a slow medium from a faster one. Or do something like install kernel headers..

Re: Use BFQ?

Posted: Tue Sep 19, 2017 2:24 pm
by asavah
jamesh wrote:
Tue Sep 19, 2017 8:53 am
There will be some major rearrangements of the forums when time allows, moving the kernel subforum may happen then.
Thanks

Re: Use BFQ?

Posted: Tue Sep 19, 2017 4:08 pm
by jamesh
Ollittm wrote:
Tue Sep 19, 2017 12:09 pm
I decided to go back to 4.9.y kernel. Given that the 4.10 and above lag thousands of commits behind 4.9.y, they're not working quite properly. Wifi does not work and neither does bluetooth. Desktop is misbehaving.. I'm sure there are ways to get around it, probably you can just tell git somehow to patch the missing commits in.

I decided to grab 4.9.y and patch the writeback throttling and BFQ in. It appears the "legacy" patches for <4.12 kernels do indeed support single queue operation as well. Theoretically having both enabled is redundant as they try to achieve the same thing in different ways.

On the writeback discussion it would seem that the problem cannot be fixed by IO scheduler as it's the disk buffer that gets overrun when you copy lots of stuff to a slow medium from a faster one. Or do something like install kernel headers..
I'm using 4.13 from the Raspberry Pi github, everything seems to work on that. Not seen a single problem. Is there a reason you cannot use that?

Re: Use BFQ?

Posted: Tue Sep 19, 2017 5:40 pm
by Ollittm
jamesh wrote:
Tue Sep 19, 2017 4:08 pm
I'm using 4.13 from the Raspberry Pi github, everything seems to work on that. Not seen a single problem. Is there a reason you cannot use that?
As I commented in the other thread, the problems were caused by "/lib/firmware" directory getting wiped out at some part from the RPi. Kernel installation does not recreate everything in that dir, is it supposed to?

4.9.50 kernel has the advantage that you can use BFQ scheduler in single queue mode. BFQ does not work with 4.12+ as the MMC driver does not do MQ as discussed above.

Primary reason to use rpi-4.9.y branch was that I was trying to backtrack to reasonably vanilla setup to see if the problem goes away but since that was not the root cause at all..

Re: Use BFQ?

Posted: Thu Sep 21, 2017 12:31 pm
by Ollittm
It looks like BFQ + throttle with 4.9 works pretty bad so I went with 4.13 kernel + throttle only.

Re: Use BFQ?

Posted: Mon Dec 03, 2018 2:19 am
by nairn62
Ollittm wrote:
Fri Sep 15, 2017 6:24 pm
I have kernel 4.12 working just fine compiled from source. However if I try to enable BFQ support, RPi fails to boot. In fact it does not seem to be able to grok the sdcard at all.

In the cmdline.txt I simply changed elevator=deadline to elevator=bfq and compiled the kernel with the BFQ options. There's also some other discussion about scsi option in cmdline and making UDEV rule but neither of these helped.

So, 4.12.11 kernel for RPi just doesn't work with BFQ?
Hi, I'm using the following Linux version on my Raspberry Pi 3:

Linux raspberrypi_septic_tank 4.18.20-v7 #1 SMP Sun Dec 2 17:48:40 GMT 2018 armv7l GNU/Linux

From the listing below, you can see that BFQ is enabled and working:

[email protected]_septic_tank:~ $ cat /sys/block/mmcblk0/queue/scheduler
mq-deadline kyber [bfq] none

Regards.