Commit Graph

23244 Commits

Author SHA1 Message Date
Michael Niedermayer
b92ccfefc3 avcodec/utils: Align dimensions by at least their chroma sub-sampling factors.
Fixes: out of array accesses
Fixes: asan_heap-oob_112c6b3_13_012.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit df74811cd5)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 11:14:25 +01:00
Michael Niedermayer
6e45a99a7c avcodec/dnxhddec: treat pix_fmt like width/height
Fixes out of array accesses
Fixes: asan_heap-oob_22c9a39_16_015.mxf
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f3c0e0bf6f)

Conflicts:

	libavcodec/dnxhddec.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 11:14:25 +01:00
Michael Niedermayer
d59804bafb avcodec/dxa: check dimensions
Fixes out of array access
Fixes: asan_heap-oob_11222fb_21_020.dxa
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e70312dfc2)

Conflicts:

	libavcodec/dxa.c
2014-11-02 11:14:25 +01:00
Michael Niedermayer
35cf24eee9 avcodec/dirac_arith: fix integer overflow
Fixes: asan_heap-oob_1078676_9_008.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 39680cacee)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 11:14:25 +01:00
Michael Niedermayer
341cf9ed81 avcodec/diracdec: Tighter checks on CODEBLOCKS_X/Y
Fixes very long but finite loop
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5145d22b88)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 11:14:25 +01:00
Michael Niedermayer
c36f5df34a avcodec/diracdec: Use 64bit in calculation of codeblock coordinates
Fixes integer overflow
Fixes out of array read
Fixes: asan_heap-oob_107866c_42_041.drc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 526886e606)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-11-02 11:14:25 +01:00
Michael Niedermayer
25a9823eb4 avcodec/h264_slice: Clear table pointers to avoid stale pointers
Might fix Ticket3889

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 547fce9585)

Conflicts:

	libavcodec/h264_slice.c

Conflicts:

	libavcodec/h264.c
2014-11-02 02:22:14 +01:00
Carl Eugen Hoyos
a8fd50bb4e Fix compilation after aa9d7058. 2014-11-02 00:43:50 +01:00
Michael Niedermayer
9282c96071 avcodec/h264_parser: rewrite the parse_nal_units() loop logic based on h264.c
Fixes Ticket4011

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 69a9a90d2e)

Conflicts:

	libavcodec/h264_parser.c
2014-11-01 20:27:01 +01:00
Michael Niedermayer
aa9d705871 avcodec/h264: simplify find_start_code()
this also uses avpriv_find_start_code(), though no speed change is expected as
the area searched is generally small

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3b678da5e3)

Conflicts:

	libavcodec/h264.h
2014-11-01 20:12:25 +01:00
Michael Niedermayer
7ddf252c7e Move get_avc_nalsize() and find_start_code() to h264.h
This allows sharing them with the h264 parser

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4898440f6b)

Conflicts:

	libavcodec/h264.c
	libavcodec/h264.h
2014-11-01 20:12:16 +01:00
Michael Niedermayer
7fc97160c2 avcodec/h264: Check mode before considering mixed mode intra prediction
Fixes out of array read
Fixes: asan_heap-oob_e476fc_2_asan_heap-oob_1333ec6_61_CAMACI3_Sony_C.jsv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9734a7a1de)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-04 15:04:43 +02:00
Michael Niedermayer
4865948d2e avcodec/smc: fix off by 1 error
Fixes out of array access
Fixes: asan_heap-oob_1685bf0_5_asan_heap-oob_1f35116_430_smc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c727401aa9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 23:24:27 +02:00
Michael Niedermayer
891376d261 avcodec/qpeg: fix off by 1 error in MV bounds check
Fixes out of array access
Fixes: asan_heap-oob_153760f_4_asan_heap-oob_1d7a4cf_164_VWbig6.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit dd3bfe3cc1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 21:32:15 +02:00
Michael Niedermayer
24a5cd720d avcodec/gifdec: factorize interleave end handling out
also change it to a loop
Fixes out of array access
Fixes: asan_heap-oob_ca5410_8_asan_heap-oob_ca5410_97_ID_LSD_Size_Less_Then_Data_Inter_3.gif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f1457864b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 20:28:52 +02:00
Michael Niedermayer
75f5fe165c avcodec/cinepak: fix integer underflow
Fixes out of array access
Fixes: asan_heap-oob_4da0ba_6_asan_heap-oob_4da0ba_241_cvid_crash.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e7e5114c50)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 19:47:43 +02:00
Michael Niedermayer
665f70209a avcodec/pngdec: Calculate MPNG bytewidth more defensively
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e830902934)

Conflicts:

	libavcodec/pngdec.c
2014-10-03 18:36:41 +02:00
Michael Niedermayer
12c8e4021c avcodec/pngdec: Check bits per pixel before setting monoblack pixel format
Fixes out of array accesses
Fixes: asan_heap-oob_14dbfcf_4_asan_heap-oob_1ce5767_179_add_method_small.png

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3e2b745020)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 18:35:38 +02:00
Michael Niedermayer
f56095c4d7 avcodec/tiff: more completely check bpp/bppcount
Fixes pixel format selection
Fixes out of array accesses
Fixes: asan_heap-oob_1766029_6_asan_heap-oob_20aa045_332_cov_1823216757_m2-d1d366d7965db766c19a66c7a2ccbb6b.tif

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e1c0cfaa41)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 16:48:23 +02:00
Michael Niedermayer
76601e4ab8 avcodec/mmvideo: Bounds check 2nd line of HHV Intra blocks
Fixes out of array access
Fixes: asan_heap-oob_4da4f3_8_asan_heap-oob_4da4f3_419_scene1a.mm

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8b0e96e1f2)

Conflicts:

	libavcodec/mmvideo.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 15:12:37 +02:00
Michael Niedermayer
e90d620cb9 avcodec/utils: Add case for jv to avcodec_align_dimensions2()
Fixes out of array accesses
Fixes: asan_heap-oob_12304aa_8_asan_heap-oob_4da4f3_300_intro.jv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 105654e376)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 05:07:33 +02:00
Michael Niedermayer
0ffa44340f avcodec/mjpegdec: check bits per pixel for changes similar to dimensions
Fixes out of array accesses
Fixes: asan_heap-oob_16668e9_2_asan_heap-oob_16668e9_346_miss_congeniality_pegasus_mjpg.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5c378d6a6d)

Conflicts:

	libavcodec/mjpegdec.c
(cherry picked from commit 94371a404c663c3dae3d542fa43951567ab67f82)

Conflicts:

	libavcodec/mjpegdec.c
2014-10-03 02:30:00 +02:00
Michael Niedermayer
0bd9b78fb6 avcodec/jpeglsdec: Check run value more completely in ls_decode_line()
previously it could have been by 1 too large
Fixes out of array access
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8c1e3.jls
Fixes: asan_heap-oob_12240f5_1_asan_heap-oob_12240f5_448_t8nde0.jls
Fixes: asan_heap-oob_12240fa_1_asan_heap-oob_12240fa_448_t16e3.jls

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 06e7d58410)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-10-03 02:29:04 +02:00
Michael Niedermayer
e9d85860d0 avcodec/ac3enc_template: fix out of array read
Found-by: Andreas Cadhalpun
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d85ebea3f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-27 20:58:44 +02:00
Gianluigi Tiesi
e28c27e25f avcodec/libilbc: support for latest git of libilbc
in the latest git commits of libilbc developers removed WebRtc_xxx typedefs

This commit uses int types instead,
it's safe to apply also for previous versions since
WebRtc_Word16 was always a typedef of int16_t and
WebRtc_UWord16 a typedef of uint16_t

Reviewed-by: Timothy Gu <timothygu99@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 59af5383c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-21 17:04:19 +02:00
James Almer
9b9048a3dd x86/dsputil: add emms to ff_scalarproduct_int16_mmxext()
Also undo the changes to ra144enc.c from previous commits.
Should fix ticket #3429

Signed-off-by: James Almer <jamrial@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9e0e1f9067)

Conflicts:

	libavcodec/ra144enc.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-20 14:14:30 +02:00
Michael Niedermayer
d49b57fe0f avcodec/mpegvideo: Set err on failure in ff_mpv_common_frame_size_change()
Found-by: ubitux
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfce6f7efd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 15:33:07 +02:00
Michael Niedermayer
0f1882b9b2 avcodec/mpegvideo: check that the context is initialized in ff_mpv_common_frame_size_change()
The function otherwise would initialize the context without setting context_initialized
alternatively we could set context_initialized

Fixes valgrind anomalies related to ticket 3928

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0d0f7f0ba4)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 15:33:06 +02:00
Michael Niedermayer
0fc229450f avcodec/mpegvideo: Use "goto fail" for all error paths in ff_mpv_common_frame_size_change()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 2762323c37)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 15:33:05 +02:00
Michael Niedermayer
1482253790 avcodec/snow: check coeffs for validity
Fixes deadlock
Fixes integer overflow
Fixes Ticket 3892

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 596636a474)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:41:13 +02:00
Michael Niedermayer
48443598ed avcodec/utils: add GBRP16 to avcodec_align_dimensions2()
Fixes Ticket3869

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3fe9e7be4c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:41:13 +02:00
Michael Niedermayer
30ffb80dca avcodec: fix aac/ac3 parser bitstream buffer size
Buffers containing copies of the AAC and AC3 header bits were not padded
before parsing, violating init_get_bits() buffer padding requirement,
leading to potential buffer read overflows.
This change adds FF_INPUT_BUFFER_PADDING_SIZE bytes to the bit buffer
for parsing the header in each of aac_parser.c and ac3_parser.c.

Based on patch by: Matt Wolenetz <wolenetz@chromium.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fccd85b9f3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-09-07 13:41:13 +02:00
Christophe Gisquet
18342848f7 proresenc_kostya: report buffer overflow
If the allocated size, despite best efforts, is too small, exit
with the appropriate error.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 52b81ff463)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-17 16:49:56 +02:00
Luca Barbato
61a51598e3 lavc: move put_bits_left in put_bits.h
(cherry picked from commit afe03092dd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-17 16:49:56 +02:00
Michael Niedermayer
abc1fa7c5a avcodec/iff: check pixfmt for rgb8 / rgbn
Fixes out of array access

Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3539d6c63a)

Conflicts:

	libavcodec/iff.c
(cherry picked from commit 656f930160)

Conflicts:

	libavcodec/iff.c
2014-08-10 22:31:02 +02:00
Anton Khirnov
f712d3669e cdgraphics: do not return 0 from the decode function
0 means no data consumed, so it can trigger an infinite loop in the
caller.

CC:libav-stable@libav.org
(cherry picked from commit c7d9b473e2)

Conflicts:

	libavcodec/cdgraphics.c

Merged-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-07 02:05:19 +02:00
Michael Niedermayer
b4632cec74 avcodec/dvdsub_parser: print message if packet is smaller than the packet size field
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bcc898dd26)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:37 +02:00
Michael Niedermayer
454f6c622c avcodec/dvdsub_parser: Check buf_size before reading 32bit packet size
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 81c1657a59)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:36 +02:00
Michael Niedermayer
f70ee7d14f avcodec/dvdsub_parser: never return 0 when the input isnt 0
Fixes a infinite loop
Fixes Ticket3804

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cfdb30d2f1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-08-02 01:27:34 +02:00
Michael Niedermayer
9d77a62936 avcodec/alsdec: Clear MPEG4AudioConfig so that no use of uninitialized memory is possible
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6e6bd5481c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:43 +02:00
Michael Niedermayer
d13cf32099 avcodec/mss4: Fix () in MKVAL() macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cf7ff0146c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
5626e11bd5 avcodec/mss34dsp: fix () in SOP* macros
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6e720c5c81)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
8d1f38c4b1 avcodec/mlpdec: fix () in MSB_MASK() macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fa160af08b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
ea7fbd0b16 avcodec/golomb-test: fix () in EXTEND() macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 97e6b5ee3a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
b2b4df8d51 avcodec/aac: fix () in IS_CODEBOOK_UNSIGNED macro
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fa915d4193)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
5691d9a6c1 avcodec/libvorbisenc: dont add the duration to AV_NOPTS_VALUE
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 19e66c7232)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
4b638ea42d avcodec/diracdec: move mc buffer allocation to per frame
Fixes out of array accesses for non default buffers with large strides

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4a30f08505)
(cherry picked from commit 9c9fc79d9237d28e33161cb2e75082d8ad232b2e)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
0607c8af2b avcodec/mjpegdec: Fix undefined shift
Fixes CID1194388

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b432960528)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:17:42 +02:00
Michael Niedermayer
85b00d798f iavcodec/vc1dec: Fix missing {}
Fixes part of Ticket3466
Found-by: Andrey_Karpov / PVS-Studio
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit cb53beb81a)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:21 +02:00
Michael Niedermayer
53c02d04e4 avcodec/g723_1: add assert to help static code analyzers
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1457f3fd90)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:21 +02:00