Commit Graph

17363 Commits

Author SHA1 Message Date
Michael Niedermayer
789f433bc6 utvideodec: Handle slice_height being zero
Fixes out of array accesses.

CC: libav-stable@libav.org
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Bug-Id: CVE-2014-9604
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit 0ce3a0f9d9)
(cherry picked from commit 3a417a86b3)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
(cherry picked from commit e032e647dd)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2015-03-08 22:11:31 -04:00
Andreas Cadhalpun
ded9931d16 rv10: check size of s->mb_width * s->mb_height
If it doesn't fit into 12 bits it triggers an assertion.

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
2015-03-08 20:33:57 +00:00
Federico Tomassetti
c17da32ba2 eamad: check for out of bounds read
Bug-Id: CID 1257500
CC: libav-stable@libav.org

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2015-03-08 19:41:12 +00:00
Michael Niedermayer
eb1aa871d4 h264_cabac: Break infinite loops
This fixes out of array reads and/or infinite loops.

30 is the maximum number of bits that can be read into
coeff_abs below.

CC: libav-stable@libav.org
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Martin Storsjö <martin@martin.st>
2015-02-23 00:04:11 +00:00
Michael Niedermayer
306ee95088 smc: fix the bounds check
Fixes invalid writes when there are more blocks in a run than total
remaining blocks.

CC: libav-stable@libav.org
Bug-ID: CVE-2014-8548
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit d423dd72be)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 58dc526ebf)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit f249e98891)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-12-20 11:11:46 +01:00
Michael Niedermayer
02de44073a gifdec: refactor interleave end handling
Fixes invalid writes with very small image heights.

CC: libav-stable@libav.org
Bug-ID: CVE-2014-8547
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 0b39ac6f54)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit eac49477aa)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 92888e9ed4)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-12-20 11:11:42 +01:00
Anton Khirnov
03dba25a40 mmvideo: check frame dimensions
The frame size must be set by the caller and each dimension must be a
multiple of 2.

CC: libav-stable@libav.org
Bug-ID: CVE-2014-8543
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 17ba719d9b)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 69a930b988)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 3f10a779b4)
Signed-off-by: Anton Khirnov <anton@khirnov.net>

Conflicts:
	libavcodec/mmvideo.c
2014-12-20 11:11:26 +01:00
Anton Khirnov
50cb695bf1 jvdec: check frame dimensions
The frame size must be set by the caller and each dimension must be a
multiple of 8.

CC: libav-stable@libav.org
Bug-ID: CVE-2014-8542
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 88626e5af8)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 55788572ea)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 8f238dd9bd)
Signed-off-by: Anton Khirnov <anton@khirnov.net>

Conflicts:
	libavcodec/jvdec.c
2014-12-20 11:10:32 +01:00
Michael Niedermayer
e4fb53c73a ffv1dec: check that global parameters do not change in version 0/1
Such changes are neither allowed nor supported

Found-by: ami_stuff
Bug-Id: CVE-2013-7020
CC: libav-stable@libav.org
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit da7d839a0d)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-08-12 10:35:09 +00:00
Michael Niedermayer
437848e37a vp3: Copy all 3 frames for thread updates
Fixes a double release of the current frame on deinit.

Bug-Id: CVE-2011-3934
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
2014-08-08 14:57:01 +01:00
Michael Niedermayer
52254067b3 error_concealment: avoid using the picture if not fully setup
Fixes state becoming inconsistent and a null pointer dereference.

CC: libav-stable@libav.org
Bug-Id: CVE-2013-0860
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-08-06 19:25:56 +00:00
Anton Khirnov
af9b62654d svq1: do not modify the input packet
The input data must remain constant, make a copy instead. This is in
theory a performance hit, but since I failed to find any samples
using this feature, this should not matter in practice.

Also, check the size of the header, avoiding invalid reads on truncated
data.

CC:libav-stable@libav.org
(cherry picked from commit 7b588bb691)
Signed-off-by: Anton Khirnov <anton@khirnov.net>

Conflicts:
	libavcodec/svq1dec.c
2014-08-06 19:10:28 +00:00
Anton Khirnov
80c268eaae cdgraphics: do not return 0 from the decode function
0 means no data consumed, so it can trigger an infinite loop in the
caller.

CC:libav-stable@libav.org
(cherry picked from commit c7d9b473e2)
Signed-off-by: Anton Khirnov <anton@khirnov.net>

Conflicts:
	libavcodec/cdgraphics.c
2014-08-06 18:46:36 +00:00
Anton Khirnov
8cd67ddde4 cdgraphics: switch to bytestream2
Fixes possible invalid memory accesses on corrupted data.

CC:libav-stable@libav.org
Bug-ID: CVE-2013-3674
(cherry picked from commit a1599f3f7e)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-08-06 18:45:57 +00:00
Michael Niedermayer
c53effc41b huffyuvdec: check width size for yuv422p
Avoid out of array accesses.

CC: libav-stable@libav.org
Bug-Id: CVE-2013-0848
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit a7153444df)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-08-05 20:11:08 +00:00
Michael Niedermayer
ede7388800 mmvideo: check horizontal coordinate too
Fixes out of array accesses.

Bug-Id: CVE-2013-3672
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 70cd3b8e65)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-08-05 19:28:47 +00:00
Michael Niedermayer
36d8914f1b wmalosslessdec: fix mclms_coeffs* array size
Fixes corruption of context

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
Bug-Id: CVE-2014-2098
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 849b9d34c7)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-08-05 15:05:22 +00:00
Luca Barbato
146b187113 lavc: Check the image size before calling get_buffer
Bug-Id: CVE-2011-3935
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
2014-08-04 14:15:45 +02:00
Diego Biurrun
43d6764327 huffyuv: Check and propagate function return values
Bug-Id: CVE-2013-0868

inspired by a patch from Michael Niedermayer <michaelni@gmx.at>
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind

(cherry picked from commit 744b406ff3)
Signed-off-by: Diego Biurrun <diego@biurrun.de>

Conflicts:
	libavcodec/huffyuvdec.c
2014-08-03 16:08:59 -07:00
Vittorio Giovara
5123541913 h264: prevent theoretical infinite loop in SEI parsing
Properly address CVE-2011-3946 and parse bitstream as described in the spec.

CC: libav-stable@libav.org
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
2014-08-01 13:36:37 +01:00
Michael Niedermayer
01f9540320 h264_sei: check SEI size
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
2014-08-01 13:36:37 +01:00
Michael Niedermayer
00915d3cd2 pgssubdec: Check RLE size before copying
Make sure the buffer size does not exceed the expected
RLE size.

Prevent an out of array bound write.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Bug-Id: CVE-2013-0852

Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
(cherry picked from commit a1f7844a11)
Signed-off-by: Diego Biurrun <diego@biurrun.de>
2014-08-01 05:12:07 -07:00
Diego Biurrun
58d7b835e3 fate: Add dependencies for dct/fft/mdct/rdft tests
(cherry picked from commit d396987c30)
Signed-off-by: Diego Biurrun <diego@biurrun.de>

Conflicts:
	libavcodec/fft-test.c
2014-07-30 12:15:44 -07:00
Anton Khirnov
e8ff797206 eamad: use the bytestream2 API instead of AV_RL
This is safer and possibly fixes invalid reads on truncated data.
(cherry-picked from commit 541427ab4d)

CC:libav-stable@libav.org

Conflicts:
	libavcodec/eamad.c

(cherry picked from commit f9204ec56a)
Signed-off-by: Diego Biurrun <diego@biurrun.de>
2014-07-30 11:35:41 -07:00
Martin Storsjö
21d3e0ac9e adpcm: Write the proper predictor in trellis mode in IMA QT
The actual predictor value, set by the trellis code, never
was written back into the variable that was written into
the block header. This was accidentally removed in b304244b.

This significantly improves the audio quality of the trellis
case, which was plain broken since b304244b.

Encoding IMA QT with trellis still actually gives a slightly
worse quality than without trellis, since the trellis encoder
doesn't use the exact same way of rounding as in
adpcm_ima_qt_compress_sample and adpcm_ima_qt_expand_nibble.

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 0776e0ef6b)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-06-26 13:43:46 +02:00
Martin Storsjö
744e7eea5d adpcm: Avoid reading out of bounds in the IMA QT trellis encoder
This was broken in 095be4fb - samples+ch (for the previous
non-planar case) equals &samples_p[ch][0]. The confusion
probably stemmed from the IMA WAV case where it originally
was &samples[avctx->channels + ch], which was correctly
changed into &samples_p[ch][1].

CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit 3d79d0c93e)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-06-26 13:43:41 +02:00
Justin Ruggles
d7dbc687e3 Check mp3 header before calling avpriv_mpegaudio_decode_header().
As indicated in the function documentation, the header MUST be
checked prior to calling it because no consistency check is done
there.

CC:libav-stable@libav.org
(cherry picked from commit f2f2e7627f)
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2014-06-26 12:52:01 +02:00
Justin Ruggles
7997acee05 Check if an mp3 header is using a reserved sample rate.
Fixes an invalid read past the end of avpriv_mpa_freq_tab.
Fixes divide-by-zero due to sample_rate being set to 0.

Bug-Id: 705

CC:libav-stable@libav.org

Conflicts:
	libavcodec/mpegaudiodecheader.c
2014-06-26 12:51:04 +02:00
Sean McGovern
4310ba273d sgidec: fix an incorrect backport
Bug-Id: 691

Signed-off-by: Anton Khirnov <anton@khirnov.net>
2014-06-17 21:44:38 +02:00
Rafaël Carré
05de372350 apedec: do not buffer decoded samples over AVPackets
Only consume an AVPacket when all the samples have been read.

When the rate of samples output is limited (by the default value
of max_samples), consuming the first packet immediately will cause
timing problems:

- The first packet with PTS 0 will output 4608 samples and be
consumed entirely
- The second packet with PTS 64 will output the remaining samples
(typically, a lot, that's why max_samples exist) until the decoded
samples of the first packet have been exhausted, at which point the
samples of the second packet will be decoded and output when
av_decode_frame is called with the next packet).

That means there's a PTS jump since the first packet is 'decoded'
immediately, which can be seen with avplay or mplayer: the timing
jumps immediately to 6.2s (which is the size of a packet).

Sample: http://streams.videolan.org/issues/6348/Goldwave-MAClib.ape

Bug-Debian: http://bugs.debian.org/744901
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 91d4cfb812)
Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2014-05-03 12:06:08 -04:00
Anton Khirnov
945a1b7ab9 h264: reset next_output_pic earlier in start_frame()
In case start_frame() fails, this potentially invalid frame can still be
output to the caller.

Bug-Id: 672
Bug-Id: debian/741240
Bug-Id: ubuntu/1288206
2014-04-24 13:06:02 +02:00
Anton Khirnov
cf7bb6ceb1 Revert "pthread: flush all threads on flush, not just the first one"
This reverts commit 2eb15cdeef.

It does not work correctly in pre-refcounting threading code.
2014-04-24 07:46:48 +02:00
Justin Ruggles
85b8b16917 tiffdec: use bytestream2 to simplify overread/overwrite protection
Based on a patch by Paul B Mahol <onemda@gmail.com>

CC:libav-stable@libav.org
2014-04-15 00:45:06 -04:00
Justin Ruggles
d946e4c1bc bytestream: add bytestream2_copy_buffer() functions
This is basically an overread/overwrite-safe memcpy between a
GetByteContext and a PutByteContext.

CC:libav-stable@libav.org
(cherry picked from commit 5748faf291)
2014-04-14 23:19:18 -04:00
Paul B Mahol
0c5e6c542f bytestream: add functions for accessing size of buffer
Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>

CC:libav-stable@libav.org
(cherry picked from commit de9d2705f6)
2014-04-14 23:18:53 -04:00
Diego Biurrun
2eb15cdeef pthread: flush all threads on flush, not just the first one
avcodec_flush_buffers() must release all internally held references
according to its documentation, for which all the threads need to be
flushed.

CC:libav-stable@libav.org
Bug-Id: vlc/9665
2014-04-14 18:09:09 -04:00
Janne Grunau
b701e26a4e mpeg12: check scantable indices in all decode_block functions
Add checks to the fast functions used with CODEC_FLAGS2_FAST and move
the check for all other functions to before the invalid memory is
accessed. Fixes https://trac.videolan.org/vlc/ticket/9713 with
CODEC_FLAGS2_FAST.

CC: libav-stable@libav.org
2014-04-14 17:45:19 -04:00
Anton Khirnov
ea1806ce65 sgidec: fix buffer size check in expand_rle_row()
Right now it will spuriously fail if the linesize is exactly equal to
the data width.

CC:libav-stable@libav.org
2014-04-14 16:55:39 -04:00
Anton Khirnov
ff409c7d80 adx: check that the offset is not negative
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 5569146d48)
2014-04-14 16:55:39 -04:00
Anton Khirnov
c9be276f8b mpegvideo: set reference/pict_type on generated reference frames
Otherwise the generic code will unref them, which can then result in
last_picture_ptr == current_picture_ptr, which causes deadlocks at least
in rv40.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:39 -04:00
Anton Khirnov
d8e89a3726 h264: reset data partitioning at the beginning of each decode call
Prevents using GetBitContexts with data from previous calls.

Fixes access to freed memory.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:39 -04:00
Anton Khirnov
7e8d27c851 h264: reset ref count if decoding the slice header fails
Otherwise the ER code might try to use some already freed references.

Fixes possible access to freed memory.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:39 -04:00
Anton Khirnov
7fd6c9fb49 h264: reset first_field if frame_start() fails for missing refs
In this case we may not have a current frame, while first_field being
set implies we do.

Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:38 -04:00
Anton Khirnov
175b53d051 h264: limit allowed pred modes in ff_h264_check_intra_pred_mode() to 3
Higher modes are not allowed for 16x16/chroma, which is what this
function is used for. Otherwise this function would return 0 (vertical
prediction) for invalid higher modes, which could result in invalid
reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:38 -04:00
Anton Khirnov
7f604a048e h264: reject mismatching luma/chroma bit depths during sps parsing
There is no point in delaying the check and it avoids bugs with a
half-initialized context.

Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:38 -04:00
Anton Khirnov
7e513d85e8 h264: check that execute_decode_slices() is not called too many times
Fixes invalid reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 9eef9eb301)
2014-04-14 16:55:38 -04:00
Anton Khirnov
b6a0f5cde8 h264: do not use 422 functions for monochrome
Fixes invalid memory access.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:38 -04:00
Anton Khirnov
1147d39bca h264: reset data_partitioning if decoding the slice header for NAL_DPA fails
If it was set before then we can end up trying to decode a slice without
a valid slice header, which can lead to invalid memory access.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
(cherry picked from commit 58312b2472)
2014-04-14 16:55:38 -04:00
Anton Khirnov
30e58e65e5 h264_refs: make sure not to write over the bounds of the default ref list
Fixes invalid writes.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:38 -04:00
Anton Khirnov
af40847681 h264: check buffer size before accessing it
Fixes invalid reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
2014-04-14 16:55:37 -04:00