ffmpeg: enable libvmaf due to dependent ab-av1

[cho-m]

Fixes #274849

Maintainer feedback to make sure we want to enable. At least no recursive dependencies. License is BSD-2-Clause-Patent

---

• Have you followed the guidelines for contributing?
• Have you ensured that your commits follow the commit style guide?
• Have you checked that there aren't other open pull requests for the same formula update/change?
• Have you built your formula locally with HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source <formula>?
• Is your test running fine brew test <formula>?
• Does your build pass brew audit --strict <formula> (after doing HOMEBREW_NO_INSTALL_FROM_API=1 brew install --build-from-source <formula>)? If this is a new formula, does it pass brew audit --new <formula>?

---

• AI was used to generate or assist with generating this PR. Please specify below how you used AI to help you, and what steps you have taken to manually verify the changes.

---

[MikeMcQuaid]

I don't think we should add a ffmpeg dependency due to a dependent breaking. Can we not add libvmaf to ab-av1 instead?

[cho-m]

I don't think we should add a ffmpeg dependency due to a dependent breaking. Can we not add libvmaf to ab-av1 instead?

No as it doesn't use libvmaf directly. It needs the support in ffmpeg itself.

The current options are:

• depend on ffmpeg-full but we've blocked this in Homebrew/core
• enable libvmaf in ffmpeg.

This PR follows existing comment in ffmpeg, i.e. "required for dependents in homebrew-core":

homebrew-core/Formula/f/ffmpeg.rb

Lines 29 to 32 in 61faac4

	# Only add dependencies required for dependents in homebrew-core
	# or INCREDIBLY widely used and light codecs in the current year (2026).
	# Add other dependencies to ffmpeg-full formula or consider making
	# formulae dependent on ffmpeg-full.

[cho-m]

Snippet of error from following shows exact ffmpeg command that fails

• ab-av1: use ffmpeg-full #276061

  ----cmd-----
  ffmpeg -r 25 -i /private/tmp/ab-av1-test-20260404-12738-txlun6/.ab-av1-GroYSZv8Spmp/test.av1.crf37_5.8.mp4 -r 25 -i test.mp4 -filter_complex [0:v]format=yuv420p10le,scale=-1:1080:flags=bicubic,setpts=PTS-STARTPTS,settb=AVTB[dis];[1:v]format=yuv420p10le,scale=-1:1080:flags=bicubic,setpts=PTS-STARTPTS,settb=AVTB[ref];[dis][ref]libvmaf=shortest=true:ts_sync_mode=nearest:n_threads=4 -an -sn -dn -f null -
  ---stderr---
  ffmpeg version 8.1 Copyright (c) 2000-2026 the FFmpeg developers
    built with Apple clang version 17.0.0 (clang-1700.6.4.2)
    configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/8.1 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gpl --enable-libsvtav1 --enable-libopus --enable-libx264 --enable-libmp3lame --enable-libdav1d --enable-libvpx --enable-libx265 --enable-openssl --enable-videotoolbox --enable-audiotoolbox --enable-neon
    libavutil      60. 26.100 / 60. 26.100
    libavcodec     62. 28.100 / 62. 28.100
    libavformat    62. 12.100 / 62. 12.100
    libavdevice    62.  3.100 / 62.  3.100
    libavfilter    11. 14.100 / 11. 14.100
    libswscale      9.  5.100 /  9.  5.100
    libswresample   6.  3.100 /  6.  3.100
  [AVFilterGraph @ 0x849010200] No such filter: 'libvmaf'
  Error : Filter not found

---

EDIT: Other options are

• rubocops/lines: allowlist for -full dependency brew#21922 + ab-av1: use ffmpeg-full #276061
• Assume degraded usage and caveat so users have to manually install ffmpeg-full (overall worse user experience so don't see any advantage over previous options)
• Deprecate/remove ab-av1 (also don't see advantage over just using ffmpeg-full)