Commit Graph

44737 Commits

Author SHA1 Message Date
Andreas Rheinhardt
eca36784b8 avcodec/[e]ac3enc: Fix indentation
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2021-04-02 12:19:07 +02:00
Andreas Rheinhardt
5d4234b3ea avcodec/ac3enc: Avoid needlessly copying encoded packets around
AC-3 and EAC-3 are codecs whose packet sizes are known in advance,
so one can use the min_size parameter of ff_alloc_packet2() to
allocate exactly this amount. This avoids a memcpy later in
av_packet_make_refcounted() in encode_simple_internal().

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2021-04-02 12:06:56 +02:00
Andreas Rheinhardt
968c158abd avcodec/ac3enc: Use actual size of buffer in init_put_bits()
Since the very beginning (since de6d9b6404)
the AC-3 encoder used AC3_MAX_CODED_FRAME_SIZE (namely 3840) for the
size of the output buffer (without any check at all).
This causes problems when encoding EAC-3 for which the maximum is too small,
smaller than the actual size of the buffer: One can run into asserts used
by the PutBits API. Ticket #8513 is about such a case and this commit
fixes it by using the real size of the buffer.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2021-04-02 12:06:56 +02:00
Zane van Iperen
aa1cfe05a5 avcodec/adpcmenc: don't share a single AVClass between multiple AVCodecs.
Temporary fix until AVClass::child_class_next is gone.

Reviewed-By: James Almer <jamrial@gmail.com>
Signed-off-by: Zane van Iperen <zane@zanevaniperen.com>
2021-04-02 09:00:32 +10:00
Andreas Rheinhardt
b7b73e83e3 avcodec/flashsv2enc: Fix undefined NULL + 0
Affected the vsynth*-flashsv2 FATE-tests.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-04-01 14:52:43 +02:00
Andreas Rheinhardt
3ef65fd4d1 avcodec/motion_est: Fix invalid left shift of negative numbers
Affected many FATE-tests.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-04-01 14:50:25 +02:00
Andreas Rheinhardt
a86f3e983e avcodec/g2meet: Fix undefined NULL + 0
Affected the g2m4 FATE-test.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-04-01 14:48:01 +02:00
Andreas Rheinhardt
566bf56791 avcodec/vmdvideo: Fix NULL + 0
Affected the FATE tests filter-gradfun-sample and sierra-vmd-video.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-04-01 14:45:54 +02:00
Andreas Rheinhardt
8429661db8 avcodec/mss12: Don't apply non-zero offset to null pointer
Affected the FATE tests mss2-wmv and mss1-pal.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-04-01 14:44:40 +02:00
Andreas Rheinhardt
dd9cbd1cc3 avcodec/lcldec: Fix undefined NULL + 0
Affected the FATE tests vsynth*-zlib, mszh and zlib.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-04-01 14:41:31 +02:00
Andreas Rheinhardt
911fe69c5f avcodec/qtrleenc: Fix negative linesizes, don't use NULL + offset
Before commit f1e17eb446, the qtrle
encoder had undefined pointer arithmetic: Outside of a loop, two
pointers were set to point to the ith element (with index i-1) of
a line of a frame. At the end of each loop iteration, these pointers
were decremented, so that they pointed to the -1th element of the line
after the loop. Furthermore, one of these pointers can be NULL (in which
case all pointer arithmetic is automatically undefined behaviour).

Commit f1e17eb44 added a check in order to ensure that the elements
never point to the -1th element of the array: The pointers are only
decremented if they are bigger than the frame's base pointer
(i.e. AVFrame.data[0]). Yet this check does not work at all in case of
negative linesizes; furthermore in case the pointer that can be NULL is
NULL initializing it still involves undefined pointer arithmetic.

This commit fixes both of these issues: First, non-NULL pointers are
initialized to point to the element after the ith element and
decrementing is moved to the beginning of the loop. Second, if a pointer
is NULL, it is just made to point to the other pointer, as this allows
to avoid checks before decrementing it.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-04-01 14:41:20 +02:00
Andreas Rheinhardt
d5fc16a6a8 avcodec/qtrleenc: Use keyframe when no previous frame is available
If keeping a reference to an earlier frame failed, the next frame must
be an I frame for lack of reference frame. This commit implements this.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-04-01 14:15:52 +02:00
Andreas Rheinhardt
f83976344e avcodec/proresdec2: Don't apply non-zero offset to null pointer
Affected ProRes without alpha; affected 32 FATE tests, e.g. prores-422,
prores-422_proxy, prores-422_lt or matroska-prores-header-insertion-bz2.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-04-01 14:13:50 +02:00
Andreas Rheinhardt
4863671d88 avcodec/mpegvideo_enc: Don't apply non-zero offset to null pointer
Affected many FATE tests (mostly vsynth ones).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-04-01 14:06:33 +02:00
Andreas Rheinhardt
659a925939 avcodec/dcaenc: Fix undefined left shift of negative numbers
Affected the acodec-dca and acodec-dca2 FATE tests.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-04-01 14:01:47 +02:00
Michael Niedermayer
79ac8d5546 avcodec/pnm_parser: Check image size addition for overflow
Fixes: assertion failure
Fixes: out of array access
Fixes: 32664/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PGMYUV_fuzzer-6533642202513408.fuzz
Fixes: 32669/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PGMYUV_fuzzer-6001928875147264

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-04-01 10:10:21 +02:00
Michael Niedermayer
c01cd2a8b2 avcodec/lscrdec: Check length in decode_idat()
Fixes: out of array access
Fixes: 32264/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_LSCR_fuzzer-6684504010915840

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-04-01 10:10:21 +02:00
Michael Niedermayer
497ea04dbd avcodec/h265_metadata_bsf: Check nb_units before accessing the first in h265_metadata_update_fragment()
Fixes: null pointer dereference
Fixes: 32113/clusterfuzz-testcase-minimized-ffmpeg_BSF_HEVC_METADATA_fuzzer-4803262287052800

Same as 0c48c332ee

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-04-01 10:10:21 +02:00
Michael Niedermayer
9e8475c7c7 avcodec/exr: Check oe in huf_decode() before use
Fixes: out of array access
Fixes: 31386/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_EXR_fuzzer-5773234709594112

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-04-01 10:10:21 +02:00
Michael Niedermayer
ceae92cb29 avcodec/h264_slice: Check input SPS in ff_h264_update_thread_context()
Fixes: crash
Fixes: check_pkt.mp4

Found-by: Rafael Dutra <rafael.dutra@cispa.de>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-03-31 23:09:35 +02:00
Michael Niedermayer
76cea1d2ce avcodec/mpegpicture: Keep ff_mpeg_framesize_alloc() failure state consistent
Fixes: null pointer dereference
Fixes: ff_put_pixels16_sse2.mp4

Found-by: Rafael Dutra <rafael.dutra@cispa.de>
Regression-since: 4b2863ff01
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-03-31 23:09:35 +02:00
Michael Niedermayer
87d87e6587 avcodec/mpegvideo: Update chroma_?_shift in ff_mpv_common_frame_size_change()
Fixes: out of array access
Fixes: 31201/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MPEG4_fuzzer-4627865612189696.fuzz

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-03-31 23:09:35 +02:00
Andreas Rheinhardt
84ac35ecb8 avcodec/mjpegenc: Fix segfault when freeing incomplete context
When allocating the MJpegContext fails (or if the dimensions run afoul
of the 65500x65500 limit), an attempt to free a subbuffer of said
context leads to a segfault in ff_mjpeg_encode_close().
Seems to be a regression since 467d9e27e0.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2021-03-31 14:37:05 +02:00
Andreas Rheinhardt
e5d6af7b35 avcodec/tiff: Avoid forward declarations
In this case it also fixes a potential for compilation failures:
Not all compilers can handle the case in which a function with
a forward declaration declared with an attribute to always inline it
is called before the function body appears. E.g. GCC 4.2.1 on OS X 10.6
doesn't like it.

Reviewed-by: Pavel Koshevoy <pkoshevoy@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2021-03-31 14:08:38 +02:00
Mark Plomer
6ef5d8ca86 avcodec/dv_profile: PAL DV files with dsf flag 0 - detect via pal flag and buf_size
Some old DV AVI files have the DSF-Flag of frames set to 0, although it
is PAL (maybe rendered with an old Ulead Media Studio Pro) ... this causes
ffmpeg/VLC-player to produce/play corrupted video (other players/editors
like VirtualDub work fine).

Fixes ticket #8333 and replaces/extends hack for ticket #2177

Signed-off-by: Marton Balint <cus@passwd.hu>
2021-03-30 19:57:34 +02:00
Andreas Rheinhardt
e7cbbd9026 avcodec/put_bits: Don't set size_in_bits, fix overflow
A PutBitContext has a field called size_in_bits which is set to the
context's bitsize init_put_bits(); but it isn't used at all (the PutBits
API uses pointers directly and not bit indexes), so remove it (due to
ABI concerns the actual element is only removed at the next bump).

Furthermore, the multiplication inherent in setting this field can lead
to undefined integer overflows. This is particularly true for FFV1,
which uses a very big worst-case buffer (37*4*width*height; even
ordinary 1080p triggers an overflow). Ticket #8350 is about this
overflow which this commit fixes.

This means that the effective range of the PutBits API is no longer
restricted by the /8 as long as one isn't using put_bits_(count|left).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-03-30 12:36:32 +02:00
Andreas Rheinhardt
c81b8e04aa Avoid intermediate bitcount for number of bytes in PutBitContext
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-03-30 12:36:32 +02:00
Andreas Rheinhardt
73fb1b8a9b avcodec/dvenc: Avoid using PutBitContext fields directly
Also avoid using bitcounts in case one is actually byte-aligned.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-03-30 12:36:32 +02:00
Andreas Rheinhardt
f9d1528fc9 avcodec/proresenc_kostya: Factor flushing PutBitContext out
The function to write an ordinary (luma or chroma) plane as well as
the function for writing an alpha plane have some similarities:
They record the initial bitposition (despite said position always being
byte-aligned), flush the PutBitContext themselves and return the amount
of bytes they wrote.

This commit factors this out; it also replaces bitpositions by
bytepositions and it avoids recording the initial byteposition because
said information is already available from the position at the end of
the last plane.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-03-30 12:36:32 +02:00
Andreas Rheinhardt
df1c30f139 avcodec/utvideoenc: Don't use bitcounts when byte-aligned
Despite write_huff_codes() receiving an ordinary buffer (not a
PutBitContext), it returned the amount of data written in bits,
not in bytes. This has been changed: There is now no intermediate
bitcount any more.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-03-30 12:36:32 +02:00
Andreas Rheinhardt
67f6e7ed6d avcodec: Remove cumbersome way of checking for amount of bytes left
Several encoders used code like the following to check for the amount of
bytes left in a PutBitContext:
pb->buf_end - pb->buf - (put_bits_count(pb) >> 3)
Besides the fact that using the pointers directly might pose
a maintainence burden in the future this also leads to suboptimal code:
The above code reads all three pointers (buf, buf_ptr and buf_end), but
touching buf is unnecessary and switching to put_bytes_left()
automatically fixes this.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-03-30 12:36:32 +02:00
Andreas Rheinhardt
11ff9cb5e9 avcodec/put_bits: Add functions for amount of bytes written/left
Often a caller doesn't want the amount of bits written via a
PutBitContext, but the amount of bytes. This in particular happens
after one has flushed the PutBitContext (e.g. at the end of encoding,
when one wants to know the actual packet size). The current way of doing
this is with put_bits_count(pb)/8 (or (put_bits_count(pb) + 7)/8).

Yet this has some issues: It contains implicit multiplications and
divisions by 8 with a cast in between; it obscurs the intent; and
it restricts the size of the buffer to (currently) INT_MAX/8 (or
to 1/8 of the maximum of whatever put_bits_count() returns), although
said restriction is not really necessary for users that don't need
a bitcount.

Corresponding functions for the amount of bytes left have also been
addded.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-03-30 12:36:32 +02:00
Andreas Rheinhardt
e48f18e2d5 avcodec/vorbisenc, wmavoice: Use put_bits_left() where appropriate
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-03-30 12:36:32 +02:00
Andreas Rheinhardt
ddd0f5dbe9 avcodec/dvenc: Remove dead error message
The PutBits API checks the available space before every write,
so this check for overread is dead.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-03-30 12:36:31 +02:00
Andreas Rheinhardt
3183eda0fa avcodec/mlpenc: Avoid redundant temporary PutBitContext
We are already word-aligned here, so one can just as well flush the main
PutBitContext.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-03-30 12:36:31 +02:00
Andreas Rheinhardt
0ab70230f2 avcodec/sonic: Remove outdated outcommented line
Compilation would fail if it were outcommented as it refers to a
nonexistent PutBitContext.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-03-30 12:36:31 +02:00
Andreas Rheinhardt
e4ccd348c1 avcodec/vorbisenc: Remove always-false check
The PutBitContext is big enough: It has just been initialized to 8192B.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-03-30 12:36:31 +02:00
Michael Niedermayer
7fc8ba9068 avcodec/setts_bsf: Check timebase
Fixes: Division by 0
Fixes: 30952/clusterfuzz-testcase-minimized-ffmpeg_BSF_SETTS_fuzzer-6601016202100736

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-03-29 22:19:39 +02:00
Michael Niedermayer
7b4510d2a8 avcodec/dvbsubdec: Support computing clut only once
This avoids crafted files from consuming excessive resources recomputing the clut after each pixel change

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-03-29 22:19:39 +02:00
Michael Niedermayer
075d793ba8 avcodec/sonic: Use unsigned temporary in predictor_calc_error()
Fixes: signed integer overflow: -2147471366 - 18638 cannot be represented in type 'int'
Fixes: 30157/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SONIC_fuzzer-5171199746506752

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-03-28 14:20:00 +02:00
James Almer
cad3a5d715 avcodec/libxvid: remove unnecessary output packet data check
The user buffers passed to avcodec_encode_video2() haven't been propagated to
AVCodec.encode2 implementations since 93016f5d1d.
Also, the generic encode code already unrefs the packet if nothing was encoded.

Signed-off-by: James Almer <jamrial@gmail.com>
2021-03-26 18:36:28 -03:00
Michael Niedermayer
8047243899 avcodec/h264_slice: Check sps in h264_slice_header_init()
Fixes: null pointer dereference
Fixes: h264_slice_header_init.mp4

Found-by: Rafael Dutra <rafael.dutra@cispa.de>
Tested-by: Rafael Dutra <rafael.dutra@cispa.de>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-03-26 16:00:14 +01:00
Michael Niedermayer
28859d6e7c avcodec/aacpsy: Check model_priv_data before dereferencing in psy_3gpp_end()
Fixes: null pointer dereference
Fixes: av_freep.mp4

Found-by: Rafael Dutra <rafael.dutra@cispa.de>
Tested-by: Rafael Dutra <rafael.dutra@cispa.de>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-03-26 16:00:14 +01:00
Michael Niedermayer
e8bd34fe4f avcodec/sga: Check for array end in lzss_decompress()
Fixes: out of array access
Fixes: 31640/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SGA_fuzzer-5630883286614016
Fixes: 31619/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SGA_fuzzer-5176667708456960

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-03-26 16:00:14 +01:00
Michael Niedermayer
be08b84f8b avcodec/ffwavesynth: Avoid signed integer overflow in phi_at()
Fixes: signed integer overflow: 2314885530818453536 - -9070214327174160352 cannot be represented in type 'long'
Fixes: 31000/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FFWAVESYNTH_fuzzer-6558389742206976

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Nicolas George <george@nsup.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-03-26 15:53:30 +01:00
Andreas Rheinhardt
6599960940 avcodec/pthread_frame: Reindentation
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-03-26 03:04:28 +01:00
Andreas Rheinhardt
c85fcc96b7 avcodec/pthread_frame: Check initializing mutexes/condition variables
Up until now, initializing the mutexes/condition variables wasn't
checked by ff_frame_thread_init(). This commit changes this.

Given that it is not documented to be save to destroy a zeroed but
otherwise uninitialized mutex/condition variable, one has to choose
between two approaches: Either one duplicates the code to free them
in ff_frame_thread_init() in case of errors or one records which have
been successfully initialized. This commit takes the latter approach:
For each of the two structures with mutexes/condition variables
an array containing the offsets of the members to initialize is added.
Said array is used both for initializing and freeing and the only thing
that needs to be recorded is how many of these have been successfully
initialized.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-03-26 03:04:28 +01:00
Andreas Rheinhardt
e9b6617579 avcodec/pthread_frame: Fix cleanup during init
In case an error happened when setting up the child threads,
ff_frame_thread_init() would up until now call ff_frame_thread_free()
to clean up all threads set up so far, including the current, not
properly initialized one.
But a half-allocated context needs special handling which
ff_frame_thread_frame_free() doesn't provide.
Notably, if allocating the AVCodecInternal, the codec's private data
or setting the options fails, the codec's close function will be
called (if there is one); it will also be called if the codec's init
function fails, regardless of whether the FF_CODEC_CAP_INIT_CLEANUP
is set. This is not supported by all codecs; in ticket #9099 it led
to a crash.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-03-26 03:04:28 +01:00
Andreas Rheinhardt
24ee151402 avcodec/pthread_frame: Factor initializing single thread out
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-03-26 03:04:28 +01:00
Zane van Iperen
e550667f61 avcodec/adpcm_ima_cunning: reset state on flush
Signed-off-by: Zane van Iperen <zane@zanevaniperen.com>
2021-03-25 12:51:09 +10:00