From e294b390a0783d45f0d29012cb46302c613e02c3 Mon Sep 17 00:00:00 2001 From: Nil Fons Miret Date: Fri, 24 Apr 2026 20:33:28 +0000 Subject: [PATCH] avfilter/vf_unsharp: fix amount scaling in the high-bit-depth path The 16-bit kernel is dispatched for every non-8-bit pixel format (9/10/12/16-bit content, all stored in uint16_t). It's supposed to undo the Q16 scaling that set_filter_param() applies to `amount`: fp->amount = amount * 65536.0; but the shift written in the kernel is `>> (8+nbits)`, which for the nbits=16 instantiation of the macro comes out to `>> 24` instead of `>> 16`. Because of this, on any non-8-bit input, unsharp applies ~1/256 of the user's requested strength and is effectively a no-op. The 8-bit kernel (nbits=8) happens to be correct because 8+8 == 16. This commit also widens the intermediate product to int64 before the shift, to avoid a potential overflow. Take a 16-bit pixel at the edge of a sharp white/black region, with the user-facing `amount` set to its declared maximum of 5.0. *srx = 65535 blur = 32768 diff = *srx - blur = 32767 amount_q16 = 5.0 * 65536 = 327680 Then the kernel computes: product = diff * amount_q16 = 32767 * 327680 = 10,737,090,560 (~1.07e10) which overflows INT32_MAX. Widening to int64 keeps the multiplication in range; the subsequent `>> 16` brings it back to sample range and the final cast to int32 is then safe. The widening is a semantic no-op for 8/9/10/12-bit content where the product always fits in int32 (worst case at 12-bit: 4095 * 327680 ~ 1.34e9). Introduced by ee792ebe08 (2019-11-08, "avfilter/vf_unsharp: add 10bit support"). The fate-filter-unsharp-yuv420p10 reference added in the same series was generated from the broken kernel and is regenerated here. fate-filter-unsharp (8-bit) is unaffected. Repro: python3 -c "import numpy as np; y=np.tile(np.where(np.arange(128)//8 & 1, 512, 256).astype('&1 | grep PSNR Before: `PSNR y:66.50 ...` -- the filter is effectively a no-op, so the sharpened output matches the input almost exactly. After: `PSNR y:28.27 ...` -- the filter actually sharpens, so output and input differ as expected. Signed-off-by: Nil Fons Miret Made-with: Cursor --- libavfilter/vf_unsharp.c | 6 ++-- tests/ref/fate/filter-unsharp-yuv420p10 | 40 ++++++++++++------------- 2 files changed, 23 insertions(+), 23 deletions(-) diff --git a/libavfilter/vf_unsharp.c b/libavfilter/vf_unsharp.c index 71d40447c0..57eccc76d7 100644 --- a/libavfilter/vf_unsharp.c +++ b/libavfilter/vf_unsharp.c @@ -157,9 +157,9 @@ static int name##_##nbits(AVFilterContext *ctx, void *arg, int jobnr, int nb_job const uint##nbits##_t *srx = src - steps_y * src_stride + x - steps_x; \ uint##nbits##_t *dsx = dst - steps_y * dst_stride + x - steps_x; \ \ - res = (int32_t)*srx + ((((int32_t) * srx - \ - (int32_t)((tmp1 + halfscale) >> scalebits)) * amount) >> (8+nbits)); \ - *dsx = av_clip_uint##nbits(res); \ + res = (int32_t)*srx + (int32_t)(((int64_t)((int32_t)*srx - \ + (int32_t)((tmp1 + halfscale) >> scalebits)) * amount) >> 16); \ + *dsx = av_clip_uintp2(res, s->bitdepth); \ } \ } \ if (y >= 0) { \ diff --git a/tests/ref/fate/filter-unsharp-yuv420p10 b/tests/ref/fate/filter-unsharp-yuv420p10 index 302becaa34..d4608558c3 100644 --- a/tests/ref/fate/filter-unsharp-yuv420p10 +++ b/tests/ref/fate/filter-unsharp-yuv420p10 @@ -3,23 +3,23 @@ #codec_id 0: rawvideo #dimensions 0: 320x240 #sar 0: 1/1 -0, 0, 0, 1, 230400, 0x5166b2b0 -0, 1, 1, 1, 230400, 0x1fcc8e0e -0, 2, 2, 1, 230400, 0xce04db47 -0, 3, 3, 1, 230400, 0xc25661af -0, 4, 4, 1, 230400, 0x727f06aa -0, 5, 5, 1, 230400, 0x2e0f9cc5 -0, 6, 6, 1, 230400, 0x407fb93b -0, 7, 7, 1, 230400, 0xc19c2087 -0, 8, 8, 1, 230400, 0xacacb223 -0, 9, 9, 1, 230400, 0x1277e355 -0, 10, 10, 1, 230400, 0xcd36c65d -0, 11, 11, 1, 230400, 0x6c530182 -0, 12, 12, 1, 230400, 0x803b2da3 -0, 13, 13, 1, 230400, 0x82638dc2 -0, 14, 14, 1, 230400, 0x2064904b -0, 15, 15, 1, 230400, 0xce3a7092 -0, 16, 16, 1, 230400, 0x374abb39 -0, 17, 17, 1, 230400, 0xf8b37d16 -0, 18, 18, 1, 230400, 0xb3d1c642 -0, 19, 19, 1, 230400, 0xc9b392d1 +0, 0, 0, 1, 230400, 0xccb5ffac +0, 1, 1, 1, 230400, 0x6aed4ef7 +0, 2, 2, 1, 230400, 0xc7bc3502 +0, 3, 3, 1, 230400, 0x2a7b9fa5 +0, 4, 4, 1, 230400, 0x29cf1fe7 +0, 5, 5, 1, 230400, 0x1b457849 +0, 6, 6, 1, 230400, 0x42fd4a9e +0, 7, 7, 1, 230400, 0xccec0881 +0, 8, 8, 1, 230400, 0x9b6cca80 +0, 9, 9, 1, 230400, 0xee3d5ff1 +0, 10, 10, 1, 230400, 0x794d9319 +0, 11, 11, 1, 230400, 0x5196b793 +0, 12, 12, 1, 230400, 0x239d7933 +0, 13, 13, 1, 230400, 0x02ea92ee +0, 14, 14, 1, 230400, 0x111b2c9b +0, 15, 15, 1, 230400, 0x5fbf8810 +0, 16, 16, 1, 230400, 0x96afa114 +0, 17, 17, 1, 230400, 0x3b587382 +0, 18, 18, 1, 230400, 0xb6aa9d42 +0, 19, 19, 1, 230400, 0xd463c17e