Tdesktop: Videos on 4K resolution have low FPS

[EugeneRymarev]

Steps to reproduce

1. Set resolution 4K (3840 x 2160)
2. Open any video (stream or saved)
3. Make it on fullscreen

Expected behaviour

Normal video playing

Actual behaviour

Low FPS video playing

Configuration

Operating system:
Windows 10
Version of Telegram Desktop:
Current last stable
Used theme:
Default

[undbsd]

Same thing happens with 1280x720 and 1360x768 videos

[EugeneRymarev]

Same thing happens with 1280x720 and 1360x768 videos

Any video resolution fullscreen on 4k monitor resolution.

I think problem in video render driver.

[turcandan]

on 4k monitor

I have the same problem with a Full HD video on a Full HD monitor(laptop), but not with all of them, tried many FHD videos.

[Aokromes]

very likely it needs hardware acceleration.

[turcandan]

very likely it needs hardware acceleration.

That's true, I'm encountering this only when I receive videos recorded in H265(HEVC).

[OctoNezd]

Any update on this issue? HEVC videos playback properly on mobile client.
ffmpeg args: -c:v hevc_qsv -q QUALITY -r 60 -acodec copy
Channel with sample videos playback of which is broken: https://t.me/tdesktophevcbroken

[turcandan]

No update, still the same lags when watching a video sent from a smartphone recorded in HEVC.

[Aokromes]

@john-preston maybe this can help you to close this issue :)
https://github.com/wang-bin/QtAV

[megapro17]

Where is any fix?? Actually telegram doesn't accelerate any videos AT ALL. It is huge issue!

[tsumeone]

This is the only app I'm aware of with such a bad video player in 2020. If my 3 year old phone can play back 4k60 HEVC video clips using the android Telegram app, then the desktop version with a high end CPU and GPU should be able to.

[megapro17]

On desktop macOS version it actually accelerates... On other platforms you have to use this trash client

[ilya-fedin]

If my 3 year old phone can play back 4k60 HEVC video clips using the android Telegram app, then the desktop version with a high end CPU and GPU should be able to.

On android when you playing video, acceleration is done automatically by the system. For cross-platform desktop client you need to write a lot of code that will interact with GPU and with system API of each supported system.

[ilya-fedin]

Bullshit, you just need to add ffmpeg, and it can deal with it easily

tdesktop already uses ffmpeg, I can't see hardware acceleration

[ilya-fedin]

firefox uses ffmpeg too and they added hardware acceleration on linux only a month ago and only on Wayland.
If ffmpeg deal with it easliy then this is strange.

[megapro17]

so where is any fix? it is also drains battery on notebooks very fast

[ilya-fedin]

so where is any fix?

well, let's implement hardware decoding (if you think it is very simple) and everyone say thank you for you :)

[megapro17]

@ilya-fedin telegram doesn't accept any pull requests from people which are not working in the company

[ilya-fedin]

telegram doesn't accept any pull requests from people which are not working in the company

that's false, I'm improving Linux support starting from January. You can make pull requests as soon as they doesn't add any new interface elements (like options in settings)
https://github.com/telegramdesktop/tdesktop/blob/dev/.github/CONTRIBUTING.md

[Aokromes]

@ilya-fedin telegram doesn't accept any pull requests from people which are not working in the company

lies, you only need to check the amount of pull requests from ilya (who is not telegram member)

[megapro17]

doesn't add any new interface elements (like options in settings)

so, implementing hw acceleration requires optional switch to disable it as done in any browser

[ilya-fedin]

so, implementing hw acceleration requires optional switch to disable it as done in any browser

It is not required by the algorithm :)

[EugeneRymarev]

I don't know from which version, but for me works fine at this moment. Trying on 1280x720 video.

[megapro17]

https://github.com/telegramdesktop/tdesktop/issues/7950 i'm stuck here...

[megapro17]

I don't know from which version, but for me works fine at this moment. Trying on 1280x720 video.

Nothing changed, it still doesn't use gpu at all, i checked in task manager. try to watch high bitrate high resolution video

[ashellunts]

@ilya-fedin @Aokromes is ffmpeg used for h265 decoding?

H265 hardware accelerated decoding is supported on Intel CPUs with Skylake or fresher architecture.

[ashellunts]

I don't know from which version, but for me works fine at this moment. Trying on 1280x720 video.

Nothing changed, it still doesn't use gpu at all, i checked in task manager. try to watch high bitrate high resolution video

I guess that the issue reported by @EugeneRymarev was because of rendering, not decoding. He didn't mention HEVC at all, and said the issue is reproduced in full screen mode.

[ashellunts]

I have tried playing videos from https://t.me/tdesktophevcbroken.

When video is played inside chat, it works much better, though still lags. In fullscreen -- 1fps or less. So that means there exist 2 issues:

• Slow software HEVC decoding.
• Some problem with rendering decoded HEVC video in full screen mode.

[ashellunts]

I have tried playing this HEVC video using ffplay. Both software and hardware decoders can play the video in full screen in 60 fps.

ffplay -vcodec hevc_qsv -i q35.mp4 -v verbose # intel qsv hw deocder
ffplay -vcodec hevc -i q35.mp4 -v verbose # sw deocder

[ashellunts]

7950 i'm stuck here...

Have you started working on the issue?

[ilya-fedin]

I guess that the issue reported by @EugeneRymarev was because of rendering, not decoding

Yeah, that very likely, since tdesktop uses only software rendering and tdesktop plays videos with slideshow even that are played smoothly by Firefox (which doesn't have hardware decoding on Linux)

I tried to activate QOpenGLWidget on Linux, but didn't notice any changes...

[ilya-fedin]

is ffmpeg used for h265 decoding?

it is used for decoding of everything

[ashellunts]

I can try to work on this issue.

[ilya-fedin]

I can try to work on this issue.

I think everyone here will be thankful to you, including me :)

[ashellunts]

Ok, let's try. 1st task - to build. Build instructions are quite long and complicated. Maybe someone already has scripts to automate build setup? Also what version of Visual Studio is recommended now? People were complaining about latest 16.6 version.

[EugeneRymarev]

16.6 version

i hearing about usage 16.7 preview

[ilya-fedin]

Build instructions are quite long and complicated.

I made a possibility to build with system packages on Linux and macOS (with brew). Don't know any way to make build simpler on Windows.

Also what version of Visual Studio is recommended now?

16.5 or 16.7 preview

[megapro17]

ffmpeg in telegram has enabled hw accel support, it seems that it only needed to be enabled in telegram, right before video initialization https://ffmpeg.org/doxygen/3.4/hw__decode_8c_source.html

[ashellunts]

ffmpeg in telegram has enabled hw accel support, it seems that it only needed to be enabled in telegram, right before video initialization https://ffmpeg.org/doxygen/3.4/hw__decode_8c_source.html

From my tests it looks like the problem (this issue) is not in decoding (or only decoding), it is something in rendering.

[ashellunts]

Trying to build Telegram. Now stuck with QT build. I would appreciate for help.

[ashellunts]

Finally built Telegram!

[megapro17]

Wtf incredible

[john-preston]

@ashellunts

I've tried this code from hw_decode.c example. I couldn't make it decode my sample 4K H264 video. It did init the hw decoder successfully and then it failed with "invalid data provided" on the first data packet that I've avcodec_send_packet-d.

I've tried linking with pre-built .dll-s from ffmpeg.org for 4.2.1 version and I got exactly the same - for both "d3d11va" and "dxva2" it initialized successfully and then failed on the first avcodec_send_packet call. Maybe my file is bad, but it works as expected without hw accelerated decoder, and I have no idea how to distinguish between those to at least provide a software-decoder fallback.

Then I've taken hw_decode.c example from 4.3.1 release tag and tried it (it had a few changes), but still I was not able to start decoding that 4K H264 video. If I comment two lines that attempt to enable the hardware decoding:

Then the first and the next avcodec_send_packet calls succeed. So I don't know how I can use that.

[ilya-fedin]

I've tried this code from hw_decode.c example.

I guess you wanted to mention @megapro17

[megapro17]

Noo I'm not a expert

[ilya-fedin]

Noo I'm not a expert

You linked the example :D

[ilya-fedin]

"Bullshit, you just need to add ffmpeg, and it can deal with it easily" :copyright: :registered: :tm:

[ashellunts]

I have also tried to enable hw decoder in Telegram and I saw that avcodec_receive_frame succeeded and I have added copying from hw surface to sw surface and it also succeeded. But due to some reason rendering still didn't work.

But I have not checked return code from avcodec_send_packet, I will check today.

[ashellunts]

Good news: with small changes hw decoder works in in-chat view.
Bad news: it doesn't work yet in full screen view.

[megapro17]

@ashellunts can you show what you changed?

[ashellunts]

@megapro17 Sure, will push branch today. Didn't have much time recently to work on the issue.

[ashellunts]

@megapro17 here is the branch https://github.com/ashellunts/tdesktop/commits/hw_decoder

[john-preston]

@ashellunts Btw to fix the compiler error (regression) you can uncomment the code and change

auto endpoint = TgVoipEndpoint{

To

TgVoipEndpoint endpoint = {

I'm tracking the issue in Visual Studio bugtracker about that regression.

[ashellunts]

@john-preston thank you.

[AetherEternity]

Current software implementation of HEVC decoding in tdesktop uses only one thread (which is a pain to see on a 24-thread cpu).
Is there an easy way to enable multithread support for it while @ashellunts is experimenting with hw?

[john-preston]

@AetherEternity It will be done in the next version.

[ashellunts]

@john-preston have you already got the fix for using more threads in sw HEVC? I can work on it.

[john-preston]

@ashellunts This should do it:

https://github.com/telegramdesktop/tdesktop/commit/361b99b0c9c9f69bc8479a4413dbe62113405734#diff-df2c819744fe7d25b5e212988c7e7bf9R178

[kabachook]

Any news on hw accel?
UPD: Animated stickers affected too

[Aokromes]

one fix was merged on jul 28

[ilya-fedin]

one fix was merged on jul 28

I don't see any PRs at Jul 28

Any news on hw accel?

No news, and they won't be most likely, implementing hw accel is a really hard task that require to work with GPU.

[kabachook]

So sad 😭
My CPU goes to 100% on HEVC and 50% on h264

[megapro17]

It's still not possible to build telegram under stable version of vs? 😂