fopen() with "r" opens the file in text mode, which on Windows
translates \r\n to \n, corrupting raw NV12 pixel data. Use "rb"
to open in binary mode, matching the output file which already
uses "w+b".
Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
avformat_close_input() is designed for input format contexts only.
Using it on output contexts is API misuse — it accesses iformat
(which is NULL for output contexts) and does not follow the correct
output cleanup path.
Replace with the proper pattern already used in remux.c and
transcode.c: avio_closep() to close the IO handle, followed by
avformat_free_context() to free the format context.
Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
When encoding VP9 with a YUV pixel format (e.g. yuv420p) and
AVCOL_SPC_RGB colorspace metadata, libvpxenc unconditionally set
VPX_CS_SRGB. This produced a spec-violating bitstream: Profile 0
(4:2:0) with sRGB colorspace, which is only valid for Profile 1/3
(4:4:4). The resulting file is undecodable.
Fix this by setting ctx->vpx_cs to VPX_CS_SRGB in set_pix_fmt()
for 4:4:4 YUV formats when AVCOL_SPC_RGB is set, matching the
existing GBRP path. This covers the legitimate case of RGB data in
YUV444 containers (e.g. H.264 High 4:4:4 with identity matrix).
With this change, any AVCOL_SPC_RGB that reaches the switch in
set_colorspace() is guaranteed to be a subsampled format where
sRGB is invalid. Return an error so the user can fix their
pipeline rather than silently producing incorrect output.
To reproduce:
ffmpeg -f lavfi -i testsrc=s=64x64:d=1:r=1 \
-c:v libvpx-vp9 -pix_fmt yuv420p -colorspace rgb bad.webm
ffprobe bad.webm
# -> "vp9 (Profile 0), none(pc, gbr/...), 64x64"
ffmpeg -i bad.webm -f null -
# -> 0 frames decoded, error
See also:
https://issues.webmproject.org/487307225
Signed-off-by: Guangyu Sun <gsun@roblox.com>
Signed-off-by: James Zern <jzern@google.com>
Signal that our assembly is compliant with the GCS feature, if
the GCS feature is enabled in the compiler (available since Clang
18 and GCC 15) - this is enabled by -mbranch-protection=standard
with a new enough compiler.
GCS doesn't require any specific modifications to the assembly
code, but requires that all functions return to the expected call
address (checked through a shadow stack).
For cases when returning early without updating any pixels, we
previously returned to return address in the caller's scope,
bypassing one function entirely. While this may seem like a neat
optimization, it makes the return stack predictor mispredict
the returns - which potentially can cost more performance than
it gains.
Secondly, if the armv9.3 feature GCS (Guarded Control Stack) is
enabled, then returns _must_ match the expected value; this feature
is being enabled across linux distributions, and by fixing the
hevc assembly, we can enable the security feature on ffmpeg as well.
adapt_colors() allocates a SwsLut3D before calling add_convert_pass(). If add_convert_pass() fails, the function returns without freeing the previously allocated lut. Free lut on that error path.
Signed-off-by: Huihui_Huang <hhhuang@smu.edu.sg>
Passing a struct/union by value can generally be inefficient.
Additionally, when the struct/union is declared to be aligned,
whether it really stays aligned when passed as a parameter by
value is unclear.
This fixes build errors like this, with MSVC targeting 32 bit ARM:
libswscale/ops_chain.h(91): error C2719: 'unnamed-parameter': formal parameter with requested alignment of 16 won't be aligned
Cap ulNumDecodeSurfaces to 32 and ulNumOutputSurfaces to 64 to prevent
cuvidCreateDecoder from failing with CUDA_ERROR_INVALID_VALUE when
initial_pool_size exceeds the hardware limits.
Also cap the decoder index pool (dpb_size) to 32 so that indices
handed out via av_refstruct_pool_get stay within the valid range
for cuvidDecodePicture's CurrPicIdx.
When unsafe_output is enabled, stop holding idx_ref in the unmap
callback. Since cuvidMapVideoFrame copies decoded data into an
independent output mapping slot, the decode surface index can safely
be reused as soon as the DPB releases it, without waiting for the
downstream consumer to release the mapped frame. This decouples the
decode surface index lifetime (max 32) from the output mapping slot
lifetime (max 64), eliminating the "No decoder surfaces left" error
that occurred when downstream components like nvenc held too many
frames.
Signed-off-by: Diego de Souza <ddesouza@nvidia.com>
Attributes with the language-supported [[attr]] style are only supported
since C++11 and C23 respectively, so this needs to be accounted for in
these checks.
This solves a huge amount of warning spam of:
warning: [[]] attributes are a C23 extension [-Wc23-extensions]
when using --enable-extra-warnings.
The VVC probe only checked forbidden_zero_bit but not
nuh_layer_id range in the NAL unit header. This allowed
certain MP3 files to be misdetected as VVC streams because
their frame data coincidentally contained 00 00 01 start
code patterns that looked like valid NAL units.
Add a check for nuh_layer_id (must be <= 55). The existing
check_temporal_id() already validates nuh_temporal_id_plus1
is in [1, 7]. Together these two checks reject the bogus
NAL units produced by MP3 frame data.
Note: nuh_reserved_zero_bit is intentionally not checked
here, as it is reserved for future use by the spec and may
become non-zero in a later revision.
Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
OpenSSL DTLS can't retransmit lost packets in blocking mode.
Switch to non-blocking mode and use DTLSv1_handle_timeout()
to properly handle DTLS handshake retransmissions.
pmt_cb() passes mp4_descr + mp4_descr_count as the output base but
MAX_MP4_DESCR_COUNT (16) as the capacity, not the remaining capacity.
init_MP4DescrParseContext() resets d->descr_count to 0 on every call,
so the bounds check at parse_MP4ESDescrTag compares a fresh 0 against
16 regardless of the shifted base.
A PMT with two IOD descriptors of 16 ESDescrs each will crash. The first
fills the buffer mp4_descr[0..15], and then the second writes
mp4_descr[16..31] -- 1152 bytes past the end of the stack.
This change passes the remaining capacity instead of always passing 16.
The writeback in mp4_read_iods is incremented so the caller's running
count is preserved.
Fixes: stack-buffer-overflow
Found-by: Nicholas Carlini <nicholas@carlini.com>
Fixes ticket #22420.
When the first decoded frame is type 1, xan_decode_frame_type1() reads y_buffer as prior-frame state before any data has been written to it.
Since y_buffer is allocated with av_malloc(), this may propagate uninitialized heap data into the decoded luma output.
Allocate y_buffer with av_mallocz() instead.
Fixes UB in the form or adding a 0 offset to a NULL pointer, and substracting a
NULL pointer from another.
Signed-off-by: James Almer <jamrial@gmail.com>
The buffers are allocated using the worst case scenario of the entire NALU
being written, when this is in many times not the case.
Signed-off-by: James Almer <jamrial@gmail.com>
The specification for LCEVC states that start codes may be three or four bytes
long except for the first NALU in an AU, which must be four bytes long.
Signed-off-by: James Almer <jamrial@gmail.com>
The specification for H.26{4,5,6} states that start codes may be three or four
bytes long long except for the first NALU in an AU, and for NALUs of parameter
set types, which must be four bytes long.
This is checked by ff_cbs_h2645_unit_requires_zero_byte(), which is made
available outside of CBS for this change.
Signed-off-by: James Almer <jamrial@gmail.com>
Based on the behaviour from cbs_h2645, which removes actual
trailing_zero_8bits bytes and possibly also work arounds issues in
ff_h2645_extract_rbsp(). In this case, the same issue could be
present in ff_nal_find_startcode().
Signed-off-by: James Almer <jamrial@gmail.com>
The correct syntax after country_code is:
t35_uk_country_code_second_octet b(8)
t35_uk_manufacturer_code_first_octet b(8)
t35_uk_manufacturer_code_second_octet b(8)
Signed-off-by: James Almer <jamrial@gmail.com>
new_pes_packet() moves a buffer with pkt->buf = pes->buffer before
JPEG-XS validation. If header_size > pkt->size, an early return leaves
pes->buffer as a stale alias of pkt->buf with refcount 1. Later,
mpegts_read_packet() calls av_packet_unref(), freeing the buffer
through pkt->buf. The flush loop then re-enters new_pes_packet() and
dereferences the dangling pes->buffer; a second path hits it via
av_buffer_unref() in handle_packets() after a seek.
Drop the early return. The packet is delivered with AV_PKT_FLAG_CORRUPT
set, matching the PES-size-mismatch case above, and the function falls
through to the normal cleanup path. The else guards the header trim so
pkt->data/pkt->size stay valid for the memset.
Fixes: use after free
Fixes regression since 16f89d342e.
Found-by: Nicholas Carlini <nicholas@carlini.com>
This also reverts: c2364e9222
Fixes: out of array access (testcase exists but did not replicate for me)
Founbd-by: Gil Portnoy <dddhkts1@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
An H.264 picture with 65536 slices makes slice_num collide with the
slice_table sentinel. slice_table is uint16_t, initialized via
memset(..., -1, ...) so spare entries (one per row, mb_stride =
mb_width + 1) stay 0xFFFF. slice_num is an uncapped ++h->current_slice.
At slice 65535 the collision makes slice_table[spare] == slice_num
pass, defeating the deblock_topleft check in xchg_mb_border and the
top_type zeroing in fill_decode_caches.
With both guards bypassed at mb_x = 0, top_borders[top_idx][-1]
underflows 96 bytes and XCHG writes at -88 below the allocation
(plus -72 and -56 for chroma in the non-444 path).
Fixes: heap-buffer-overflow
Found-by: Nicholas Carlini <nicholas@carlini.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Jun Zhao