Compile FFmpeg with CUDA 11.2

I’ve been experimenting GPU accelerated options to scale my videos using FFmpeg and CUDA. And found there are MANY MANY errors in the existing official documentation & forums. I’d like to summarize them here just in case you encounter them too.

My hardware, software & SDKs used:
Windows 10
x64 platform
NVIDIA GTX 1080 (compute capability 6.1 or 61)
Visual Studio 2019 Professional Version 16.8.3
CUDA v11.2
FFmpeg 4.3.1 source code: https://ffmpeg.org/download.html
Mysys2 From https://www.msys2.org/

Documentation and discussion reviewed: (Generally, none of them cover all with no issue!)
Official Doc: https://docs.nvidia.com/video-technologies/video-codec-sdk/ffmpeg-with-nvidia-gpu/index.html
Nvidia Forum: https://forums.developer.nvidia.com/t/problem-compiling-ffmpeg-with-nvenc-using-visual-studio-2015-community-edition/112540/2
CSDN: https://blog.csdn.net/qq_18998145/article/details/108244374

First, I got my Mysys2 and ran its console to install some required packages:

pacman -Syu
# restart mysys2 console after executing the first command


pacman -S make pkgconf diffutils
pacman -S make pkg-config diffutils yasm
pacman -S mingw-w64-x86_64-nasm mingw-w64-x86_64-gcc mingw-w64-x86_64-SDL2

After installing the packages I closed mysys2 and put the following command in a file named “mysys_vs2019.bat” to start mysys2:
You may need to change the highlighted Professional to Community depending on the Visual Studio you use or the location you install them.

set MSYS2_PATH_TYPE=inherit
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Auxiliary\Build\vcvars64.bat"
msys2_shell.cmd -mingw64

Then continue the steps in Nvidia’s official document.
Generally most of the steps in the official doc works, I changed a few steps to match up my environment:
I use CUDA v11.2 instead of V8.0. I have Visual Studio 2019 Professional instead of VS2013(Visual Studio 12.0).

I changed:

export PATH="/c/Program Files (x86)/Microsoft Visual Studio 12.0/VC/BIN/amd64/":$PATH
export PATH="/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v8.0/bin/":$PATH

to

export PATH="/c/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/TOOLS/MSVC/14.28.29333/bin/Hostx64/x64/":$PATH
export PATH="/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.2/bin/":$PATH

Keep in mind I am using Visual Studio 2019 Professional, most people use Community. And you need to adjust the highlighted part in the first path.

You can confirm the environment:

MATRIXDOGE@MATRIXDOGE-PC MINGW64 ~/ffmpeg-4.3.1
$ which nvcc
/c/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.2/bin/nvcc

MATRIXDOGE@MATRIXDOGE-PC MINGW64 ~/ffmpeg-4.3.1
$ which link
/c/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/MSVC/14.28.29333/bin/HostX64/x64/link

Here is what goes gravely wrong in the official doc:

./configure --enable-nonfree –disable-shared --enable-cuda-sdk --enable-libnpp –-toolchain=msvc --extra-cflags=-I../nv_sdk --extra-ldflags=-libpath:../nv_sdk

Problem 1: why there are --enable and -enable!? Such a careless typo! Fix them!
And you will see the following error.
Unknown option "–disable-shared".

Problem 2: --disable-shared never works. Remove it.
Problem 3: According to this forum link Problem compiling ffmpeg with nvenc using visual studio 2015 community edition, "--enable-cuda-sdk" is deprecated. We need to use “--enable-cuda-nvcc” instead. However, the author in the Nvidia forum also made a typo same as the one in Problem 1 in another spot. And you may see the following error:

Unknown option "-toolchain=msvc".


He also has weird unicode “…” character in the post. We need to use “..”. The guy in the CSDN post made the similar mistake.

Here is finally the right one worked for me:
./configure --enable-cuda-nvcc --enable-nonfree --enable-libnpp --toolchain=msvc --extra-cflags=-I../nv_sdk --extra-ldflags=-libpath:../nv_sdk

After this you can compile:
make -j 8

 

 

Enjoy your FFmpeg with CUDA. It didn’t improve the video scaling performance in my case :'(

 

None of the following runs made any difference:

 

ffmpeg -i input.mp4 -filter:v scale=720:-1 -c:a copy -vcodec libx265 -crf 28 output1.mp4

ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda –resize 720x400 -i "input.mp4" -c:a copy -c:v h264_nvenc -b:v 5M output2.mp4

ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -i input.mp4 -vf scale_cuda=720:400 -c:a copy -c:v h264_nvenc -b:v 5M output3.mp4