meson-vdec: poor playback of h264 #8884
Comments
|
Does v4l2m2m even work on wayland? I have absolutely no idea. |
|
This thread reports success using mpv under Wayland / Weston with a similar hardware config to mine (meson vpu) although they don't give details on what format videos they are playing etc. I SHOULD be OK to play a 30 fps 4K h264 file, if all the many ducks are aligned and the kernel code isn't buggy. Of course, this could well be a kernel rather than an mpv bug? I prefer KDE to GNOME but if the mpv devs would prefer I test under Wayland GNOME instead that's fine by me. |
|
I have updated my bug report with a YT video of mpv trying to play the test video. I'm very impressed with the X96 Air Q1000 as a budget Linux box, even without a fully functional mpv. It has 4 GB of DDR 4 RAM, it can boot directly off a USB 3 SSD, it has gigabit ethernet, Wifi, a better GPU and faster CPU than the Rpi 4, 3 USB ports (which can be used to power the board) and you get a case and remote all for about £30! Amazing! Even moreso if we can get mpv working nicely on it! |
|
Reading through the mpv output, I noticed this line: Is that required for mpv v4l2m2m / meson to work? |
|
No it's not related. Some of the visual glitches in your video look quite strange but I suppose that's just hwdec at work. |
|
fwiw the visual glitches may also be related to panfrost enabling afbc in mesa 21.1 by default on some more platforms. |
|
@CounterPillow I presume afbc can't be disabled or worked around other than disabling it within mesa? I will open a bug against mesa if this is actually their problem. Maybe its a mesa, kernel and mpv problem? This is pretty virgin territory, I admit. |
|
there's an environment variable you can set in your environment to disable it ( |
|
I have tried exporting |
|
I have tried playing a h264 file using the same codec and transport as the test video above but using 1080p / 30 fps instead. It seems to drop less frames during playback but its still a jerky mess and with the same flicker/distortion you can see in the 4K playback video. |
|
I should note that I have seen a similar sort of flicker occur on this box under Wayland (both GNOME and KDE) when playing videos under FireFox, the difference being that YT videos play at a decent framerate, at least when not full screen with 4K as it would seem FF is not using any hw accel, and mpv flickers more frequently with my test video(s) than FF does playing videos. I can post a video of Firefox playing videos if required? |
|
My test 4K h264 video plays perfectly under the imaginatively titled |
|
I have also tried playing some HEVC files under mpv on this box with very similar results to h264 - very slow/jerky playback. |
|
My TV supports Freesync but not g-sync. Android can play this video fine on the same display. |
|
I was having trouble dumping the edid on my TV box so I did it on my laptop. Shouldn't matter how I get it, right? |
|
"Can you switch to 24/1.001 and check out how many zeroes it will print?" I don't understand what you are asking me to do? |
|
The display refresh hz being slightly weird or whatever shouldn't result in poor performance as shown above. Seems like a red herring. |
|
I'm still unsure what you want me to do. In my case, when connecting to my laptop running X the xrandr command would be: but what do you want me to check after running that? xrandr doen't change the edid does it? My TV is a LG 65NANO866. |
|
I'm trying to play this video on my ARM TV box running wayland tho. X runs like absolute s#it but Wayland runs pretty well. I am new to Wayland. Does xrandr still work with Wayland? I presumed it was X only. If so, what is the equivalent Wayland command? |
|
It turns out the display settings in wayland plasma are buggy. I am currently unable to change the resolution and refresh rate under wayKDE but it does work under wayland GNOME. I configured the GNOME display to 4K @ 23.98 Hz and now the flicker is gone and the video at least plays (the last time I tried playing this test video under wayland GNOME mpv crashed) and its a little less jerky but it still drops hundreds of frames in less than 30 seconds and is nowhere near as smooth as when the video is played under Android. Here's video of me playing the video under wayland GNOME: https://www.youtube.com/watch?v=86oCKnUzcW8 Here is the log: |
|
It's not working correctly. Under X, mpv is fine but has anyone got mpv working properly with Wayland under any Linux distro, on any architecture using any GPU or is mpv's wayland support just not there yet? Today I installed Manjaro unstable on an old i7 desktop with a nvidia GPU (GeForce 650 ti I think). My test video played fine with mpv under X11 plasma but I couldn't get it to play at all under plasma wayland nor under GNOME wayland, I tried different refresh rates but no joy. Under amd64 nvidia plasma, I have an empty mpv.conf but I've been using the command:
Might I need any more options or should I be using a different gpu-context? Funnily enough, I have had more success playing my test video on the £30 TV box than using an i7 with 12 GB RAM etc, at least I can see some of the frames on the Mali TV box. I did have a Radeon RX 580 GPU and I was going to use that to test mpv under wayland on amd64 but unfortunately it seems my Radeon is dead. I have read Wayland works better on modern Intel and AMD GPUs than NVIDIA but I'm using the latest NVIDIA driver I can get for my GPU - 465.xx . I've got a machine with an older Intel HD but its not quite up to the job of running Wayland plasma at 4K. I have also read online that the mpv devs don't support mpv under wayland GNOME? Is this true? Which Wayland compositor has had the most testing with mpv? Maybe I should open another ticket for mpv on Nvidia plasma or is this combo known not to work already? |
|
NVIDIA on wayland is a broken mess. Expect nothing good from it unless NVIDIA fixes their drivers one day (good luck).
That is not true. mpv intends to support all compositors that properly follow wayland protocols (which does include GNOME). |
|
Until they add GBM support or something similar that allows direct access to buffers, it's basically a big hack. |
It's not. Tying a display server to specifically EGL is actually insanity.
I'll believe it when it actually exists and works. |
|
Have any of you successfully used mpv under Wayland (on amd64 or ARM)? If so, what combo of OS, GPU and compositor are/were you using? Obviously not Nvidia! :) Did you test 4K or UHD decoding and playback (of h264 or HEVC)? EDIT Wayland, not plasma but I'm mainly interested in Plasma. I don't really like GNOME or sway. |
|
I use mpv on wayland everyday of course (amdgpu on sway). Everything works fine for me (of course), but I don't own any ARM machines so I can't say anything about their performance or lack thereof. |
|
I'd be happy to buy you a X96 Air if you think you could get mpv working on it @Dudemanguy. Might that be a challenge that would interest you? I was disappointed that the RPi 4 cannot decode h264 @ 4K in hardware, h264 still being a very widely used codec. These S905X3 based TV boxes can decode 4K h264 (and HEVC, VP 9 etc) in hardware so as far as mpv is concerned these meson TV boxes are arguably (could be) a superior platform for using mpv. On top of that you get a case, a remote and SPDIF for about half the price of an RPi 4. The Odroid C4 is almost identical in spec to the X96 Air Q1000, same SoC etc. The C4 has GPIO pins which the X96 Air does not but the C4 is also twice the price. The X96 Air has a serial port but you have to solder the headers on if you want it. I have got USB 3 boot, gigabit ethernet, wifi, blutooth, accelerated GLES 3 and HDMI audio all working on the X96 Air under Manjaro so its really just the hardware video decoding that isn't working under proper (non-Android) Linux now. I think it'd be very cool to have a ~£30 Linux box that could play 4K videos with mpv, don't you? |
|
I have already seen mpv working great with HW decoding of 4K videos on an ARM device on my Jetson Nano but that is under X11 and using all that Nvidia nastiness that no-one wants. It'd be much better to have it working on these dirt cheap meson boards that aren't tied to Nvidia's stuff. |
|
Performance issues are mostly down to drivers and hardware. There's nothing that can really be done from mpv's end. All of the tests I've done show that xorg and wayland on my hardware perform essentially identical which is expected. |
|
Today I updated Manjaro unstable on my X96 Air and I tested playing my test video under mpv at all of the refresh rates that the gnome display manager offers for 3840x2160, the res of my test clip.
When using 23 to 25 Hz, the video played 'jerkily' with about ~450 dropped frames. 29.97 Hz didn't work at all and nor did 60 Hz, 50 Hz performed about the same as 23-25 Hz but the interesting setting was at 59.94 Hz. The video played back even more jerky than at 23 to 25 etc but according to the mpv log it only dropped about 46 frames instead of 10x that amount at the other refresh rates. Here is the mpv log when using 59.94 Hz @ 3840x2160: |
|
I am currently running Linux 5.11 but I will be attempting to build a newer kernel soon. I will report back on this after upgrading to a newer kernel. If any mpv devs want to try and get mpv working properly on the X96 Air Q1000, you can use my dtb: |
|
@ValZapod 60/1.001 vs. 60 fps will not lead to massive framedrops or desync, it'll lead to one dropped/duplicated frame every few seconds. I don't think this is relevant to what OP is seeing, which is a lot of dropped frames. |
Where are you getting this from? I haven't read the full issue because there's a lot of unrelated noise about nvidia for some reason but decoder drops are shown as VO drops. What could also be causing it is simply the GPU not able to rescale the output with the full shading pipeline, or the cpu memory to gpu memory copy (if there is one with that hwdec) being slow. |
|
The null VO can be used to check whether decoding is the bottleneck, if I recall correctly. |
|
Something like this? (with no mpv.conf) Looks like its the decoding then, so is this a kernel rather than an mpv issue? |
|
I wouldn't necessarily say kernel, it could be in ffmpeg. There is some way to test hardware decoding in ffmpeg which @Akemi tells people to try with videotoolbox issues, but I forgot the precise command. But trying that and seeing what fps you get just decoding could turn this into an ffmpeg bug report instead, who can then decide whether it's a kernel issue or not. Of course, it's entirely possible your video decoding core simply does not support 4k H.264 and only does 4K HEVC or something. I know the hwdec on my old GTX 780 Ti drops many frames on 4k H.264 content, so try with a lower resolution file and double-check the datasheet for your SoC. (Everyone likes to claim "4K" when the display core is technically able to output a 4K video signal, but rarely do they mention in the marketing material which precise other components can do anything useful at that resolution) |
|
I've not looked at the datasheets for my SoC but as I mentioned earlier I have been able to successfully play the test video under Android on my X96 Air with one of the included video players so thats all the proof I need that the hardware is capable of decoding this file. It seems to be playing it back at UHD res although I don't know if the video player I'm using has a stats HUD feature to verify that. I tried mpv under Android recently but I get the impression its got a lot of catching up to do to match the regular Linux mpv. None of the videos I tried with Android mpv played properly. I have tried to play my test video with ffplay on my X96 Air and I'm getting pretty much exactly the same results as with mpv so I've opened a bug report with ffmpeg: |
|
My latest comment on the ffmpeg ticket: I successfully built and installed ffmpeg git under Manjaro ARM and I can run this command without any errors: As per the example given here: https://lwn.net/Articles/767126/ How might I modify that ffmpeg command to output a picture (ie to play the video under Wayland) and if decoding is working fine under ffmpeg, why does it not work under ffplay? |
|
Here is the datasheet for the SoC my TV box uses: https://dn.odroid.com/S905X3/ODROID-C4/Docs/S905X3_Public_Datasheet_Hardkernel.pdf |
|
Whilst it won't help fix this bug, I think its worth mentioning that 1920x1080 h264 videos using this same codec/transport etc play fine under Wayland with mpv with no hw acceleration (vo=gpu/ no options given) on my X96 Air. There are a few dropped frames if you are playing a 1080p h264 on a 4K/UHD desktop but that's only to be expected. If I use 1920x1080 for the desktop res then it can play Full HD/1080p h264 files full screen without dropping frames, without hw accel decoding. It definitely plays 1080p videos much better without hw acceleration than with. |
|
I have written a wiki page for installing and running Manjaro on Amlogic TV boxes. I added a mpv / hw accelerated video section to it today: |
|
The incorrect assumption(s) are that the current (staging) vdec driver is good, the handling of 8-bit/10-bit output in the DRM layer is good, the firmware blobs from Amlogic are good, and support for v4l2m2m in ffmpeg is both also good and aligned with the current vdec. All of the variables have some question marks against them. The vdec was developed by Maxime Jourdan in 2018/19 before the stateful UAPI started to mature and so v4l2m2m support in ffmpeg was also immature, but in combination H264 worked well and other codecs were being added - based on an 8-bit output pipeline that did not support Amlogic framebuffer compression used in Mali Utgard SoCs or ARM framebuffer compression (different, but similar) used in Mali Midgard and Bifrost SoCs. AFBC(s) are crucial for supporting 4K output else RAM speeds (esp. in older/cheaper devices) can't sustain the throughput the video pipeline needs. Once the initial stateful UAPI conformance tests were published in late summer 2019, some work to adapt the vdec to the tests was done by Maxime to meet them. This work focussed on the vdec itself and the conformance test-app, so didn't result in matching changes for userspace apps like ffmpeg and gstreamer. Around this time mesa has started to support AFBC in Panfrost, and the dw-hdmi DRM bridge layer has evolved support for 10-bit output (GXBB/GXL are 10-bit internally but output 8-bit, GXM and newer are 10-bit through the full pipeline). Maxime sadly had to stop work in late 2019 for personal reasons and has never resumed. Neil Armstrong (kernel maintainer and colleague to Maxime) was able to cleanup the remaining scraps of code Maxime had lying around in test branches to finish the kernel side of conformance work, and those went upstreaam in early 2020. Earlier this year some minor work to revive HEVC support was done and this works well (if you have the right ffmpeg source) on devices with 8-bit ouput. However 10-bit output (essential for 4K) results in hard crashes. One of Maximes original goals was to support all hardware in a single driver. We have learnt over time that HEVC/VP9 are handled quite differently in older and newer SoCs (different firmwares, different approaches to memory management, use of IOMMU, etc.) and to progress support the current vdec should be split-up to handle these differences. One example of bad is, the vdec currently needs a huge CMA reservation (896MB!) for 4K which doesn't play nice with older devices equipped with 1GB RAM. In the wider V4L2 world there has been great progress on the stateless UAPI (v4l2-request) but stateful is some way behind, and I perceive this is due to dependencies on closed-source firmwares (less attractive to the open-source community) and other devices that use the stateful UAPI, e.g. Exynos, iMX6, often being in the late stages of their lifecycle or with a much higher overall dependency on BSP code so there is less interest or less need for a mainline solution (Exynos is quite broken, iMX6 works okay but its world exists behind NXPs NDA-wall). The notable stateful exception is the H264 IP in the Raspberry Pi which is under current and very active development from the Pi Foundation developers. RPI4 is also unusual since the HEVC IP is stateless, so you have use of both v4l2-request and v4l2m2m in the same chip. In their ffmpeg sources v4l2m2m is now working very (very) well, but "it takes two to tango" and it works well due to an alignment of ffmpeg changes, vdec changes, and UAPI tweaks. At some point there will be a serious attempt to upstream everything, but for now we're still at the "make it all work" stage. "We" is the Pi devs working with Kodi (a fancy wrapper on ffmpeg) and LibreELEC (which I contribute to). Up to a specific git-commit point, the improvements Pi devs made in ffmpeg also benefitted the Amlogic vdec and H264 playback is generally great, and with some of the nip/tuck work done on HEVC this now plays (but not seeks) reliably on the 8-bit output devices. However, once Pi devs started to tweak the Pi vdec in combination with ffmpeg we started to see a large regression. From this point the Amlogic vdec needs matching changes .. and that leads into the redesign work previously mentioned. Why put all this in a long reply? .. because I've been looking for developer talent to work on the vdec since 2019 without much success. The gene-pool of people who can write kernel vdec code is small. The gene-pool of people who also understand ffmpeg/gstreamer etc. is smaller again. The main names on the list are all professional developeers who rightly expect to be paid for their work. It's not impossible for "community" developers to take on the task, but it's not a small task and thus needs a small group with a personal interest; else the commitment is too large. If anyone is interested .. RSVP! |
|
Thanks for giving all that gory technical detai @chewitt! I found it interesting but I don't have the necessary coding skills to make a real dent in any of that unfortinately. I'm going to repost that it a few places to see if it helps drum up any interest. Thanks |
|
@danboid Have you had any success so far? I am trying to get 1080p h264 videos playing in Armbian on the Le Potato and have been running into another set of interesting issues (video plays but does not end). I am wondering if this is related |
|
No, I've not tried at this for a while. 1080p videos actually play well enough without any h/w acceleration. I have heard of people getting h/w accel decoding working with 1080p h264 videos but I'm really only interested in UHD/4K hw decoding. |
|
kernel: https://github.com/chewitt/linux/commits/amlogic-5.18.y ^ that combo has working H264 @ 1080p with good seeking and HEVC/VP9 with 4K/HDR on GXL/GXM devices; seeking in HEVC isn't perfect but playback should be okay. If you run the same combo on G12 or newer hardware 8-bit HEVC will play but 10-bit will wedge the board. I haven't tested gstreamer in a while but it will probably give similar results; the codec drivers need effort to progress. I'm still trying to find people to work on them. |
|
Thanks @chewitt for the info. I am trying to play H264 instead of HEVC, but will try the mentioned ffmpeg and kernel to see if it will help. |
|
After upgrading my Armbian image to (https://redirect.armbian.com/region/EU/lepotato/Jammy_edge_xfce), apt-get update && apt-get upgrade, I am now running on kernel 5.17.5-meson64, ffmpeg 4.4.1-3ubuntu5, mpv mpv 0.34.1. The situation looks much better. Video actually plays to the end. However when playing 1080p videos, i get about 10 dropped frames every second. I reckon this is due to the vo/gpu not being able to keep up with the decoder. @chewitt would I have any luck trying to get PRIME working with meson-vdec? |
|
LE uses DRMPRIME with Kodi (EGL also works, as an option). I can't speak for other player apps as I don't use them. |
|
@cyph84 there was a bug in the ffmpeg sources; branch is updated with a fix (build HEAD) |
|
@chewitt Sorry was away from this project for a while. After building the folowing: drmprime works:
However trying to seek causes: Any clues on where I might start debugging this? When you say "good seeking" which player app are you using to test? |
and others
Important Information
Provide following Information:
X96 Air Q1000 Amlogic S905X3 based TV box.
0.33.1-dirty
Manjaro ARM unstable branch, kernel 5.12.1-1-MANJARO-ARM
manjaro arm unstable repo
KDE plasma wayland, which works better than GNOME wayland for mpv. Attempting to play this video with mpv under wayland GNOME crashed mpv.
mesa 21.1.1-1
https://youtu.be/r4lWJO35o00
Reproduction steps
I am trying to play this video:
https://drive.google.com/file/d/14z1Gd8zXbJ2XWvULvmWXeqLVRcgUHo2p/view?usp=sharing
Under Manjaro ARM unstable KDE wayland (what a mouthful!)
My
mpv.conflooks like this:Expected behavior
Smooth hw accelerated fullscreen video playback using the meson vpu.
Actual behavior
Very jerky video playback, less than 1 fps with 100's of dropped frames. I have tried experimenting with the plasma compositor settings but they don't seem to make any notable difference to mpv.
Log file
Sample files
https://drive.google.com/file/d/14z1Gd8zXbJ2XWvULvmWXeqLVRcgUHo2p/view?usp=sharing
The text was updated successfully, but these errors were encountered: