Commit Graph

23188 Commits

Author SHA1 Message Date
Michael Niedermayer
e897eb108b avcodec/libx264: move where x264opts is applied down so it isnt overridden by avctx & defaults
fixes x264opts opengop=1

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 64b79141bd)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Michael Niedermayer
2af3a04164 avcodec/h264_mp4toannexb_bsf: prepend global headers before any in stream parameter sets
Fixes h264_mp4toannexb_bsf_failure.mkv

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 289b149cec)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Michael Niedermayer
92ca3578bf avcodec/x86/mpegvideoenc_template: fix integer overflow
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-06-23 18:16:20 +02:00
Martin Storsjö
401b3dafd9 adpcm: Write the proper predictor in trellis mode in IMA QT
The actual predictor value, set by the trellis code, never
was written back into the variable that was written into
the block header. This was accidentally removed in b304244b.

This significantly improves the audio quality of the trellis
case, which was plain broken since b304244b.

Encoding IMA QT with trellis still actually gives a slightly
worse quality than without trellis, since the trellis encoder
doesn't use the exact same way of rounding as in
adpcm_ima_qt_compress_sample and adpcm_ima_qt_expand_nibble.

Fixes part of Ticket3701

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fa8f060b75)

Conflicts:
	tests/ref/acodec/adpcm-ima_qt-trellis
2014-06-10 00:24:30 +02:00
Martin Storsjö
71516ac175 adpcm: Fix trellis encoding of IMA QT
This was broken in 095be4fb - samples+ch (for the previous
non-planar case) equals &samples_p[ch][0]. The confusion
probably stemmed from the IMA WAV case where it originally
was &samples[avctx->channels + ch], which was correctly
changed into &samples_p[ch][1].

Fixes part of Ticket3701

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit a32765c425)
2014-06-10 00:24:22 +02:00
Carl Eugen Hoyos
0983720cf2 avcodec/utvideodec: Increase vlc len
Fixes a regression since fb3e380 similar to ticket #2661,
reported by fluffrabbit at aol dot com.

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 673716c54b)

Conflicts:
	libavcodec/utvideodec.c
2014-06-10 00:21:05 +02:00
Michael Niedermayer
2c71734d9e avcodec/vorbisdec: try to workaround libvorbisenc bug
Fixes Ticket3590

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0a266cb55a)

Conflicts:
	libavcodec/vorbisdec.c
2014-05-01 18:32:13 +02:00
Michael Niedermayer
f07cebcd91 avcodec/h263dec: Fix use of uninitialized memory from the bitstream buffer
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-18 18:06:17 +01:00
Peter Ross
b1a960cc83 avcodec/adpcm: squelch 'mismatch in coded sample count' warning for AV_CODEC_ID_ADPCM_EA_R2/3
These ADPCM codecs include a per-frame flag that enables a raw 16-bit mode. Therefore
the the number of samples returned by get_nb_samples() is only ever approximate.

Fixes ticket #3460.

Signed-off-by: Peter Ross <pross@xvid.org>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7380201451)
2014-03-17 16:06:20 +01:00
Peter Ross
620d80d572 avcodec/adpcm: ADPCM_IMA_DK3 packets are padded to 16-bit packet boundary
Fixes ticket #3461.

Signed-off-by: Peter Ross <pross@xvid.org>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d1bb17940d)
2014-03-17 16:06:09 +01:00
Michael Niedermayer
e3ce3d373c avcodec/h264: be more tolerant on what pixel format changes trigger reinits
Fixes Ticket3260

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8e92ff2546)

Conflicts:
	libavcodec/h264.c
2014-03-17 16:06:04 +01:00
Michael Niedermayer
fb44a3e4a3 avcodec/vorbisdec: use the stored previous window type only when the actual previous is not known
Fixes Ticket3432

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5171ae781a)
2014-03-13 07:35:17 +01:00
Michael Niedermayer
ddcccababe avcodec/utvideoenc: fix slice_bits size
Fixes assertion failure

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0a8c90202b)

Conflicts:

	libavcodec/utvideoenc.c
(cherry picked from commit 57522ca79cc38c279123596d3288ddbf56fa8903)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-05 03:06:10 +01:00
Michael Niedermayer
b580bae53a avcodec/snow: split block clipping checks
Fixes out of array read
Fixes: d4476f68ca1c1c57afbc45806f581963-asan_heap-oob_2266b27_8607_cov_4044577381_snow_chroma_bug.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 61d59703c9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-05 03:06:10 +01:00
Michael Niedermayer
9085cdd677 avcodec/ansi: fix integer overflow
Fixes out of array read
Fixes: 5f9698e86d92f19bb08d54ff0d57027f-signal_sigsegv_b30756_3795_cov_2693691257_ansi256.ans
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d42ec8433c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-05 03:06:10 +01:00
Michael Niedermayer
ab31a9ee4a avcodec/msrle: use av_image_get_linesize() to calculate the linesize
Fixes out of array access
Fixes: 14a74a0a2dc67ede543f0e35d834fbbe-asan_heap-oob_49572c_556_cov_215466444_44_001_engine_room.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c919e1ca2e)

Conflicts:

	libavcodec/msrle.c
(cherry picked from commit bc1c8ec5e6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-05 03:06:10 +01:00
Michael Niedermayer
11b14d0e63 avcodec/wmalosslessdec: fix mclms_coeffs* array size
Fixes corruption of context
Fixes: 8835659dde6a4f7dcdf341de6a45c6c8-signal_sigsegv_1dce67b_4564_cov_2504444599_classical_22_16_1_14000_v3c_0_extend_0_29.wma
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ec9578d54d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-05 03:06:10 +01:00
Michael Niedermayer
b3cc4bd18f avcodec/vc1: reset fcm/field_mode in non advanced header parsing
Fixes NULL pointer dereference
Fixes: signal_sigsegv_1ab8bf4_2847_cov_4254117347_SA10091.vc1
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b51e935477)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-05 03:06:10 +01:00
Michael Niedermayer
ca9d302519 avcodec/takdec: always check bits_per_raw_sample
Fixes out of array access
Fixes: asan_heap-oob_19c7a94_6470_cov_1453611734_luckynight-partial.tak
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f58eab1512)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-05 03:06:10 +01:00
Michael Niedermayer
e925fd215f avcodec/vmnc: Check that rectangles are within the picture
Prevents out of array accesses with CODEC_FLAG_EMU_EDGE

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6ba02602aa)

Conflicts:

	libavcodec/vmnc.c

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7c17207ab9)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-05 03:06:10 +01:00
Michael Niedermayer
5e01cd3b69 dnxhdenc: fix mb_rc size
Fixes out of array access with RC_VARIANCE set to 0

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f1caaa1c61)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-03-05 03:06:10 +01:00
Anton Khirnov
2256b2a3c1 pthread_frame: flush all threads on flush, not just the first one
avcodec_flush_buffers() must release all internally held references
according to its documentation, for which all the threads need to be
flushed.

Bug-Id: vlc/9665
(cherry picked from commit d1f9563d50)

Conflicts:
	libavcodec/pthread_frame.c
2014-03-04 01:39:11 +01:00
Reimar Döffinger
8a38deb789 Fix compilation on ARM with android gcc 4.7
With the current code it fails due to running out
of registers.
So code the store offsets manually into the assembler
instead.
Passes "make fate-dts".

Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
(cherry picked from commit 8067f55edf)

Fixes ticket #2756.
2014-01-26 16:23:58 +01:00
Michael Niedermayer
b902ba478c avcodec/mjpegdec: Dont skip picture allocation if theres no picture allocated
Fixes Ticket 3245
(cherry picked from commit ad8d063f23)
2014-01-20 22:39:50 +01:00
Michael Niedermayer
f0fabcc6ed avcodec/mjpegdec: Dont treat the lack of a startcode differently from end of the bitstream
Fixes Ticket3303

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 31e703e899)

Conflicts:
	libavcodec/mjpegdec.c
2014-01-19 14:43:47 +01:00
Michael Niedermayer
bd29764c61 avcodec/aacdec: Dont fail if channels arent known yet
Fixes Ticket3312

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 676a395ab9)

Conflicts:
	libavcodec/aacdec.c
2014-01-19 14:41:05 +01:00
Michael Niedermayer
d81ccf1fb2 avcodec/mjpegdec: only run EOI emulation code when there was a scan
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 361e27a3d8)
2014-01-19 14:38:03 +01:00
Michael Niedermayer
67d20495f5 avcodec/mjpegdec: update cur_scan also for non-LS jpeg
This should make no difference but the variable will be used in a subsequent commit

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8893f31e20)

Conflicts:
	libavcodec/mjpegdec.c
2014-01-19 14:37:32 +01:00
Michael Niedermayer
db20610c49 wmaprodec: Fix null pointer dereference in decode_frame()
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c7a7605656)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 01:58:13 +01:00
Michael Niedermayer
15a736483e h264: check mb_width/height
Fixes inconsistency that leads to out of array accesses with threads

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5a9e376049)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:52:56 +01:00
Michael Niedermayer
c8b90c7cd5 h264: Always decode MMCOs into temprary array
When decoding succeeded the array is copied into the permanent one.
This prevents inconsistencies
Fixes assertion failure

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit c40f51e15b)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:52:56 +01:00
Michael Niedermayer
97978b7ae8 h264: move the default_ref_list_done check down after its inputs have been written
Fixes out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 11c3381ce3)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:52:56 +01:00
Michael Niedermayer
3f2efac0fe h264: Move slice_table clean out of frame_start
Fixes inconsistency ultimately leading to an out of array read

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 746016598d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:52:56 +01:00
Michael Niedermayer
b9058f58bd h264_refs: Check for attempts to assign pictures to short & long.
Fixes null pointer dereference

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 92002db3eb)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:49:11 +01:00
Michael Niedermayer
47faf347a0 h264_cavlc: fix assertion failure due to reading too long vlc
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 10ece44d09)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:48:58 +01:00
Michael Niedermayer
3d1e4b7ca5 avcodec/aacps: fix number of bands used with ipd/opd
Fixes use of uninitialized memory
Fixes: msan_uninit-mem_7f861d16355f_1664_File1_fixed.aac
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6433b393ba)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:47:25 +01:00
Michael Niedermayer
a2bea0df0e aacps: correct opdipd code to match spec
This fixes out of array reads

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 66e9716a36)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:47:18 +01:00
Michael Niedermayer
483b2016bb avcodec/ansi: Fix right/bottom end checks
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6021615bbe)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:45:22 +01:00
Michael Niedermayer
271f5d68b9 avcodec/ansi: Check x/y
This prevents out of array accesses

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e91fd754c6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:45:07 +01:00
Michael Niedermayer
dee327b0e7 shorten: fix "off by padding" bug
Fixes array overread

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ad22767cb6)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:42:23 +01:00
Michael Niedermayer
e438fd3be9 avcodec/lcldec: Check that dimensions are a multiple of the subsample factors
Other dimensions would not work correctly currently,
also ask for a sample for files that fail this check.

This fixes an integer overflow leading to out of array
accesses.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 1e00bbb10c)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:35:12 +01:00
Michael Niedermayer
84b100396e dxa: check vectors of 2x2 motion blocks
Fixes out of array reads

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ead590c256)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:35:12 +01:00
Michael Niedermayer
7855083443 dxa: check vectors of 4x4 motion blocks
Fixes out of array reads

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f96e0eb238)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:35:12 +01:00
Michael Niedermayer
ca7f64e0ac shorten: allocate space for padding
Fixes array overread

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 89d998f1c1)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:35:12 +01:00
Michael Niedermayer
2b26f8c6bf evrcdec: fix sign error
The specification wants round(abs(x))) * sign(x) which is
equivakent to round(x)

Fixes out of array access

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit f18d2dff11)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:35:12 +01:00
Michael Niedermayer
6e3697b985 eamad: Check against minimum supported dimensions
Fixes out of array reads

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e756635964)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:35:12 +01:00
Michael Niedermayer
c93501687c xan: Check for overlapping copies
No valid samples i found use such copies

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bdfe60c769)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:35:12 +01:00
Michael Niedermayer
bb6f466794 avcodec/bitstream: check codes in ff_init_vlc_sparse()
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6998af4a40)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:35:12 +01:00
Michael Niedermayer
08a319549a avcodec/bitstream: Check bits in ff_init_vlc_sparse()
Fixes out of array reads

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit fb3e3808ae)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:35:12 +01:00
Michael Niedermayer
d1da1c8384 avcodec/mpegvideo_motion: Use a field from the current frame if the last is unavailable in DMV & 16x8
Fixes null pointer dereference

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 42bcc4082d)

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
2014-01-16 00:35:12 +01:00