RinaMat
Posts: 5
Joined: Wed Jan 31, 2018 11:20 am

Problem using DPI + PWM + Videodecode

Tue Jun 12, 2018 9:07 am

During development on a CM3 I encountered a problem when using the DPI-LCD-interface during video-playback (with omxplayer) when ALSO using a PWM channel.
The DPI-interface works perfect for videos, but when I do an export of a PWM channel (to activate the userspace control in sysfs) the video becomes slow or even stops (showing the last decoded frame).
I first encountered this on my "own" Yocto-build but it's the same using Raspbian Strech lite (2018-4-18), so I think it is a general problem.

Configuration:
I am using DPI Mode4 (16bit RGB565) with my own overlay (I named it dpi16m4-overlay.dts), but this one is nothing special, it is derived from standard dpi18 overlay using some other GPIO pins:

Code: Select all

/dts-v1/;
/plugin/;

/{
	compatible = "brcm,bcm2708";

	// There is no DPI driver module, but we need a platform device
	// node (that doesn't already use pinctrl) to hang the pinctrl
	// reference on - leds will do

	fragment@0 {
		target = <&leds>;
		__overlay__ {
			pinctrl-names = "default";
			pinctrl-0 = <&dpi16_pins>;
		};
	};

	//Pins used for Mode 4, RGB565

	fragment@1 {
		target = <&gpio>;
		__overlay__ {
			dpi16_pins: dpi16_pins {
				brcm,pins = <0 1 2 3 5 6 7 8 9 12
					    13 14 15 16 17 21 22 
					    23 24 25>;
				brcm,function = <6>; /* alt2 */
				brcm,pull = <0>; /* no pull */
			};
		};
	};
};
I slightly changed my dt-blob in the CM3-section (moving uart0 and activating gp_clk on GPIO42):

Code: Select all

pins_cm3 { // Pi 3 CM3
         pin_config {
            pin@default {
               polarity = "active_high";
               termination = "pull_down";
               startup_state = "inactive";
               function = "input";
            }; // pin
            pin@p32 { function = "uart0";  termination = "no_pulling"; drive_strength_mA = < 8 >; }; // TX uart0
            pin@p33 { function = "uart0";  termination = "pull_up"; drive_strength_mA = < 8 >; }; // RX uart0
            pin@p42 { function = "gp_clk"; termination = "pull_down"; drive_strength_mA = < 8 >; }; // ETH_CLK - Ethernet 25MHz output
            pin@p46 { function = "input";   termination = "pull_up";    }; // SMPS_SCL
            pin@p47 { function = "input";   termination = "pull_up";    }; // SMPS_SDA
            pin@p48 { function = "sdcard"; termination = "pull_up";    drive_strength_mA = < 8 >; }; // SD CLK
            pin@p49 { function = "sdcard"; termination = "pull_up";    drive_strength_mA = < 8 >; }; // SD CMD
            pin@p50 { function = "sdcard"; termination = "pull_up";    drive_strength_mA = < 8 >; }; // SD D0
            pin@p51 { function = "sdcard"; termination = "pull_up";    drive_strength_mA = < 8 >; }; // SD D1
            pin@p52 { function = "sdcard"; termination = "pull_up";    drive_strength_mA = < 8 >; }; // SD D2
            pin@p53 { function = "sdcard"; termination = "pull_up";    drive_strength_mA = < 8 >; }; // SD D3
            pin@p128 { function = "input";  termination = "no_pulling"; polarity = "active_low"; }; // Hotplug
            pin@p129 { function = "output"; termination = "no_pulling"; polarity = "active_low"; }; // EMMC_ENABLE_N
         }; // pin_config

         pin_defines {
            pin_define@HDMI_CONTROL_ATTACHED {
               type = "external";
               number = <0>;
            };
            pin_define@EMMC_ENABLE {
               type = "external";
               number = <1>;
            };
            pin_define@NUM_CAMERAS {
               type = "internal";
               number = <0>;
            };
            pin_define@POWER_LOW {
               type = "absent";
            };
            pin_define@LEDS_DISK_ACTIVITY {
               type = "absent";
            };
            pin_define@LAN_RUN {
               type = "absent";
            };
            pin_define@SMPS_SDA {
               type = "internal";
               number = <46>;
            };
            pin_define@SMPS_SCL {
               type = "internal";
               number = <47>;
            };
            pin_define@ETH_CLK {
               type = "internal";
               number = <42>;
            };
            pin_define@WL_LPO_CLK {
               type = "absent";
            };
            pin_define@USB_LIMIT_1A2 {
               type = "absent";
            };
            pin_define@SIO_1V8_SEL {
               type = "absent";
            };
            pin_define@PWML {
               type = "absent";
            };
            pin_define@PWMR {
               type = "absent";
            };
            pin_define@SAFE_MODE {
               type = "absent";
            };
            pin_define@SD_CARD_DETECT {
               type = "absent";
            };
            pin_define@ID_SDA {
               type = "absent";
            };
            pin_define@ID_SCL {
               type = "absent";
            };
         }; // pin_defines
      }; // pins


I stripped down my config.txt to a minimum (for DPI and the PWM):

Code: Select all

#Parameters for parallel RGB LCD (RGB565, using BCM DPI-Mode 4 )
dtoverlay=dpi16m4
framebuffer_width=800
framebuffer_height=480
framebuffer_depth=16
enable_dpi_lcd=1
display_default_lcd=1
dpi_group=2
dpi_mode=87
dpi_output_format=0x70014
hdmi_timings=800 0 34 10 36 480 0 21 5 18 0 0 0 70 0 32000000 6

#PWM for backlight: at GPIO18
dtoverlay=pwm,pin=18,func=2
I can start watching videos (h264) with omxplayer without any issue (as long as lsmod says that the kernel module "pwm_bcm2835" is used by "0")!
But when I do a "echo 1 > /sys/class/pwm/pwmchip0/export" (now lsmod says that the kernel module "pwm_bcm2835" is used by "1") the video immediately becomes slow or, when alternatively doing a "echo 0 > /sys/class/pwm/pwmchip0/export" the video hangs after some seconds or at the very end.

NOTE that when embedding the video in my own Qt application this also happens, but I can see that the DPI-interface is still working properly, as my whole GUI still responds and changes the LCD-content properly - e.g. I can see a moving mouse-cursor when moving an attached mouse while the video still jerks or is stopped (only shows the last decoded frame) "behind" the mouse-cursor!

NOTE that when using HDMI as output this DOES NOT HAPPEN!

I do absolutely NOT know how activating (or even only exporting it to userspace) the PWM would affect video-playback at the DPI-interface, I can only think of a "shared" PLL (clock generator) causing this issue - but that's beyond my debugging skills on the pi ...

RinaMat
Posts: 5
Joined: Wed Jan 31, 2018 11:20 am

Re: Problem using DPI + PWM + Videodecode

Thu Jun 14, 2018 9:04 am

Hm, no one using DPI for (cheap) standard LCD modules and controlling the LCD backlight via PWM? This would show the problem.

As all this happens with using the standard software (Raspbian, overlays) I suppose I am doing nothing "wrong".
So should I post a BUG for this? And where would I do that - at Raspbian?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5141
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Problem using DPI + PWM + Videodecode

Thu Jun 14, 2018 9:29 am

RinaMat wrote:
Thu Jun 14, 2018 9:04 am
Hm, no one using DPI for (cheap) standard LCD modules and controlling the LCD backlight via PWM? This would show the problem.
Quite possibly not.

Having said that the PWM peripheral is also used for analogue audio, so that might also be an easy test case.

I assume you've disabled analogue audio on your CM3. Not doing so would fit with your description. With HDMI connected the audio would be routed through HDMI so no interaction with analogue audio. No HDMI and the audio would go via analogue. If you export/open the PWM peripheral then you've probably killed the sample speed for the audio, and it's likely to throw playback off.

Please also note that AIUI omxplayer uses OpenMAX for audio decode and render, and not Linux drivers. It also has the comment

Code: Select all

// By default audio_render is the clock master, and if output samples don't fit the timestamps, it will speed up/slow down the clock.
so that will slow down video playback if you slow the PWM peripheral.
RinaMat wrote:As all this happens with using the standard software (Raspbian, overlays) I suppose I am doing nothing "wrong".
So should I post a BUG for this? And where would I do that - at Raspbian?
https://github.com/raspberrypi/firmware/issues
Please provide as much detail as possible so that the problem can be replicated.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5141
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Problem using DPI + PWM + Videodecode

Thu Jun 14, 2018 9:33 am

I've just confirmed my suspicions by adding "-o local" to the omxplayer command line when on HDMI.
Again exporting the pwmchip slows audio right down, and disabling it again causes funny things to occur.

Simple is answer is that you need to avoid using analogue audio in your use case.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

RinaMat
Posts: 5
Joined: Wed Jan 31, 2018 11:20 am

Re: Problem using DPI + PWM + Videodecode

Thu Jun 14, 2018 10:13 am

6by9 wrote: Simple is answer is that you need to avoid using analogue audio in your use case.
Yeah, would be cool if that's the case.
I did not mention it in the first place - because my first posting is a little bit long and that often leads to not reading it - but your suspicion was also mine. Only thing is that this did not improve anything!

I added everything I found regarding audio and PWM (not using a PLL for audio PWM, disabling opening the PWM for audio and even switching audio off using the dtparam) - so that would be the config.txt STILL CAUSING THE PROBLEM:

Code: Select all

avoid_pwm_pll=1
force_pwm_open=0
dtparam=audio=off

#Parameters for parallel RGB LCD (RGB565, using BCM DPI-Mode 4 )
dtoverlay=dpi16m4
framebuffer_width=800
framebuffer_height=480
framebuffer_depth=16
enable_dpi_lcd=1
display_default_lcd=1
dpi_group=2
dpi_mode=87
dpi_output_format=0x70014
hdmi_timings=800 0 34 10 36 480 0 21 5 18 0 0 0 70 0 32000000 6

#PWM for backlight: at GPIO18
dtoverlay=pwm,pin=18,func=2
Any further idea before I do post it as a bug?

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5141
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Problem using DPI + PWM + Videodecode

Thu Jun 14, 2018 11:06 am

"dtparam=audio=off" will disable the Linux analogue audio driver only, whilst OMX player by default is using the GPU and going direct to the PWM peripheral.

If you have no audio output, then play a video file with no audio and confirm that that has no issues. Use ffmpeg or avconv, eg

Code: Select all

ffmpeg -i input.mp4 -c:v copy -an output.mp4
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

RinaMat
Posts: 5
Joined: Wed Jan 31, 2018 11:20 am

Re: Problem using DPI + PWM + Videodecode

Thu Jun 14, 2018 11:13 am

Seems like the 3 lines in config.txt

Code: Select all

avoid_pwm_pll=1
force_pwm_open=0
dtparam=audio=off
do not do what I expected.

According to 6by9 (and thinking that those 3 lines above might not do what I want them to do) I start the videoplayback with audio output of omxplayer disabled (by using omxplayer -n -1 videoname) - and now it works!

Due to my usecase this can't be the final solution, so the question might be:
How do I completely disable the onboard analogue audio, so that even on a compute module (CM3 in my case) no PWM is claimed for audio output?

Thanks and best regards,
Mat

6by9
Raspberry Pi Engineer & Forum Moderator
Raspberry Pi Engineer & Forum Moderator
Posts: 5141
Joined: Wed Dec 04, 2013 11:27 am
Location: ZZ9 Plural Z Alpha, aka just outside Cambridge.

Re: Problem using DPI + PWM + Videodecode

Thu Jun 14, 2018 11:58 am

RinaMat wrote:
Thu Jun 14, 2018 11:13 am
Due to my usecase this can't be the final solution.
Why? What audio output are you using that isn't using the PWM peripheral? Direct omxplayer at that via alsa or similar, or disable the omxplayer from decoding the audio.
omxplayer is the only player I'm aware of that goes direct to the audio hardware as it uses OpenMAX, almost all other apps will use ALSA, so perhaps you're better changing player. As you've said nothing about what is so special with your use case it's very hard to advise.
There's certainly little point in reporting a bug on the firmware as the issue is understood.
Software Engineer at Raspberry Pi Trading. Views expressed are still personal views.
Please don't send PMs asking for support - use the forum.
I'm not interested in doing contracts for bespoke functionality - please don't ask.

RinaMat
Posts: 5
Joined: Wed Jan 31, 2018 11:20 am

Re: Problem using DPI + PWM + Videodecode

Thu Jun 14, 2018 12:28 pm

6by9,I agree with you, in this case it is not a bug.
My usecase is a little complicated, that's why I do not want to explain it here in detail. But this doesn't matter, I will find a solution now that I know the behaviour of the openmax decoder ...

I'd see this topic as "solved" - many thanks for pointing me (more) into the right direction!

Return to “Interfacing (DSI, CSI, I2C, etc.)”

Who is online

Users browsing this forum: No registered users and 12 guests