Reimar Döffinger
de6d44829c
aacps.c: Move large arrays to context to reduce stack usage.
...
Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de >
2014-12-18 23:50:41 +01:00
Michael Niedermayer
3d8bedef45
Merge commit '91bfac759dfd536e439ad3e35964705012c6a5a7'
...
* commit '91bfac759dfd536e439ad3e35964705012c6a5a7':
h261enc: Disallow sliced encoding
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-18 21:10:56 +01:00
Michael Niedermayer
368642361f
avcodec/indeo3: ensure offsets are non negative
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-18 18:57:27 +01:00
Michael Niedermayer
e59c28b166
avcodec/adpcm: Check idelta
...
Fixes integer overflow
Fixes: signal_sigsegv_1b0a4da_1865_cov_2167818389_computer_anger.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-18 16:10:23 +01:00
Martin Storsjö
91bfac759d
h261enc: Disallow sliced encoding
...
This avoids trying to do sliced encoding, even if a slice/packet
size is requested (via the -ps option or the rtp_payload_size
field), since the encoder currently doesn't support it (or at least
our decoder can't decode it, even if the h261_encode_gob_header
function is hooked up to be called from the slicing part in
mpegvideo_enc.c).
Signed-off-by: Martin Storsjö <martin@martin.st >
2014-12-18 12:00:01 +02:00
Michael Niedermayer
61296d41e2
avcodec/h264: Check *log2_weight_denom
...
Fixes undefined behavior
Fixes: signal_sigsegv_14768d2_2248_cov_3629497219_h264_h264___pi_20070614T182942.h264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-18 03:45:59 +01:00
Michael Niedermayer
3281fa8925
avcodec/hevc_ps: Check diff_cu_qp_delta_depth
...
Fixes undefined behavior
Fixes: asan_static-oob_17aa046_582_cov_1577759978_DBLK_G_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-18 02:19:53 +01:00
Michael Niedermayer
99f8c9e4d1
avcodec/hevc: move qp_block_mask to where its used
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-18 01:55:02 +01:00
Michael Niedermayer
e8714f6f93
avcodec/h264: Clear delayed_pic on deallocation
...
Fixes use of freed memory
Fixes: case5_av_frame_copy_props.mp4
Found-by: Michal Zalewski <lcamtuf@coredump.cx >
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-17 21:55:45 +01:00
Michael Niedermayer
8aa8d12554
avcodec/hevc: clear filter_slice_edges() on allocation
...
This avoids use of uninitialized memory
Fixes: asan_static-oob_17aa046_582_cov_212287884_DBLK_G_VIXS_1.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-17 19:43:58 +01:00
Michael Niedermayer
7d593495e4
avcodec/dcadec: Check that the added xch channel isnt already there
...
Fixes null pointer dereference
Fixes: signal_sigsegv_369609d_623_cov_2008234281_ES_6.1_16bit.dts
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-17 15:33:59 +01:00
Michael Niedermayer
3ba1050292
avcodec/imc: dont read bits beyond the end
...
Fixes use of uninitialized memory
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-17 05:14:11 +01:00
Michael Niedermayer
ddd7dac7ae
avcodec/utils: check the private context class
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-17 05:14:11 +01:00
Michael Niedermayer
3305acdc92
avcodec/indeo3: use signed variables to avoid underflow
...
Fixes out of array read
Fixes: signal_sigsegv_1b0a4da_1865_cov_2167818389_computer_anger.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-17 05:14:11 +01:00
Reimar Döffinger
254da44bf9
flacenc: calculate lower sum levels in-place.
...
Should improve cache usage and reduces stack usage.
Also reduces number of copies in case many levels
have the same number of bits.
Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de >
2014-12-17 04:27:09 +01:00
Michael Niedermayer
f3b5b139ad
avcodec/h264: make the first field of H264Context an AVClass
...
Fixes use of freed memory
Fixes: asan_heap-uaf_3660f67_757_cov_1257014655_Hi422FR1_SONY_A.jsv
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-17 01:33:06 +01:00
Michael Niedermayer
3881606240
avcodec/utvideodec: Fix handling of slice_height=0
...
Fixes out of array accesses
Fixes: asan_heap-oob_25bcd7e_3783_cov_3553517262_utvideo_rgba_median.avi
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-16 23:00:04 +01:00
Michael Niedermayer
211200e0c0
avcodec/xface: Add asserts to limit nb_words from becoming too large
...
Approved-by: Stefano Sabatini <stefasab@gmail.com >
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-16 19:19:35 +01:00
Michael Niedermayer
93a5a16f13
avcodec/xface: correct the XFACE_MAX_* values
...
Fixes out of array access
Fixes: asan_stack-oob_32c12e5_2536_cov_2442316831_lena.xface
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-16 19:19:28 +01:00
Michael Niedermayer
3030fb7e0d
avcodec/vmdvideo: Check len before using it in method 3
...
Fixes out of array access
Fixes: asan_heap-oob_4d23ba_91_cov_3853393937_128.vmd
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-16 16:27:45 +01:00
Derek Buitenhuis
402a61c919
snow_dwt: Don't try and free members of non-existent arrays
...
If allocation fails earlier on, and the next frame is processed,
the slice buffer could be left in a state where line and data_stack
have already been freed, or are otherwise null pointers.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com >
2014-12-15 21:52:19 -05:00
Ronald S. Bultje
0a7964dca5
vp9/x86: save one register on 32bit idct32x32.
...
Fixes build on win32.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-16 02:51:26 +01:00
Michael Niedermayer
4f1923a91b
avcodec/tiffenc: Consistently use ADD_ENTRY1()
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-15 22:45:41 +01:00
Michael Niedermayer
4cffcbc809
Merge commit '10d4c5e55e5b23e165aa4c7723073ebe2c2e7da0'
...
* commit '10d4c5e55e5b23e165aa4c7723073ebe2c2e7da0':
tiff: set the correct return value when check_size() fails
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-15 22:14:14 +01:00
Michael Niedermayer
07b4d4b1a2
Merge commit '3c27275c1309190f2d6ed69140b67d014215b6c9'
...
* commit '3c27275c1309190f2d6ed69140b67d014215b6c9':
tiff: Check the check_size() return value and forward it
Conflicts:
libavcodec/tiffenc.c
See: d50aa006fb
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-15 22:00:52 +01:00
Michael Niedermayer
41915d0b86
Merge commit '4690e01c3aaf495c87127e5dc74aa347197dbc0b'
...
* commit '4690e01c3aaf495c87127e5dc74aa347197dbc0b':
prores: Evaluate all the quantizers
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-15 21:46:29 +01:00
Michael Niedermayer
38c3ffbd55
Merge commit 'd75190aa93206c02cb4f18a66d2c927d6a0198e1'
...
* commit 'd75190aa93206c02cb4f18a66d2c927d6a0198e1':
mpegvideo: move REBASE_PICTURE where it is used
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-15 21:27:10 +01:00
Michael Niedermayer
0a3ffcfc82
Merge commit '63fcedcf01247f15b72c0d1f959bb3b612853ba8'
...
* commit '63fcedcf01247f15b72c0d1f959bb3b612853ba8':
mpegvideo: remove unused function declaration
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-15 21:19:40 +01:00
Michael Niedermayer
b799ec7e0b
avcodec/hevc: Print an error if luma_log2_weight_denom is out of range
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-15 21:12:46 +01:00
Michael Niedermayer
241943d3a2
Merge commit 'b2bece5e9a4dd96142962c6798e187c77b88bbfa'
...
* commit 'b2bece5e9a4dd96142962c6798e187c77b88bbfa':
hevc: always clip luma_log2_weight_denom
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-15 21:06:20 +01:00
Michael Niedermayer
1307c11ce6
avcodec/avdct: support >8bit per sample dcts
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-15 18:25:00 +01:00
Vittorio Giovara
10d4c5e55e
tiff: set the correct return value when check_size() fails
...
Only one instance affected and solved as other occurences.
2014-12-15 15:46:34 +01:00
Luca Barbato
3c27275c13
tiff: Check the check_size() return value and forward it
...
Also use the same type for add_entry and check_size.
Bug-Id: CID 700699
CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org >
Signed-off-by: Vittorio Giovara <vittorio.giovarao@gmail.com >
2014-12-15 15:46:34 +01:00
Luca Barbato
4690e01c3a
prores: Evaluate all the quantizers
...
Prevent an uninitialized data access.
CC: libav-stable@libav.org
Bug-Id: CID 703824 / CID 703825
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com >
Signed-off-by: Luca Barbato <lu_zero@gentoo.org >
2014-12-15 15:46:34 +01:00
Vittorio Giovara
d75190aa93
mpegvideo: move REBASE_PICTURE where it is used
...
Drop an unused #undef from h264 decoder.
2014-12-15 15:46:34 +01:00
Vittorio Giovara
63fcedcf01
mpegvideo: remove unused function declaration
2014-12-15 15:46:34 +01:00
Vittorio Giovara
b2bece5e9a
hevc: always clip luma_log2_weight_denom
...
Its value shall be between 0 and 7 according to the specifications.
CC: libav-stable@libav.org
Bug-Id: CID 1257502
2014-12-15 15:46:34 +01:00
Michael Niedermayer
228fded036
Merge commit 'bab8d73765238bfeea7a8dfd454f40ac213b3d22'
...
* commit 'bab8d73765238bfeea7a8dfd454f40ac213b3d22':
vaapi: wrap codec specific functions in appropiate #ifs
Conflicts:
libavcodec/vaapi.c
See: f2118d44c9
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-15 13:03:01 +01:00
Michael Niedermayer
d2f9b9da03
Merge commit '5a82ad644f281701eb22e0359fd732fac67aafa4'
...
* commit '5a82ad644f281701eb22e0359fd732fac67aafa4':
svq1dec: Unbreak the scratch buffer allocation
Conflicts:
libavcodec/svq1dec.c
See: 4213fc5b9e
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-15 12:23:27 +01:00
Michael Niedermayer
0c7733904e
Merge commit 'bcbdeb318e0828cf00b458885ffa1e1596c85d06'
...
* commit 'bcbdeb318e0828cf00b458885ffa1e1596c85d06':
asvenc: Forward the error instead of asserting on it
Conflicts:
libavcodec/asvenc.c
Not merged, conditions which are impossible should be checked by assert*()
adding impossible to be true error checks has the potential to confuse
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-15 12:04:31 +01:00
Ronald S. Bultje
cae893f692
vp9/x86: sse2 MC assembly.
...
Also a slight change to the ssse3 code, which prevents a theoretical
overflow in the sharp filter.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-15 02:34:05 +01:00
Ronald S. Bultje
fd77fbb390
vp9/x86: 32bit and sse2 support for vp9 inverse transform assembly
...
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-15 00:38:05 +01:00
Michael Niedermayer
27f2e211aa
Merge commit '559fa0d41b5cf8f7e1ba89d1ac70e74712eb2816'
...
* commit '559fa0d41b5cf8f7e1ba89d1ac70e74712eb2816':
vdpau: add mapping for H.264 Constrained Baseline profile and fallback
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-14 22:23:53 +01:00
Michael Niedermayer
3f679f73a2
Merge commit '8502c1e9ff9c1dbb6e467630c048d098f4064021'
...
* commit '8502c1e9ff9c1dbb6e467630c048d098f4064021':
vdpau: add mapping for H.264 Extended profile
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-14 22:14:58 +01:00
Michael Niedermayer
0b30d186f1
Merge commit '8b51bcfed75721e0d20832e5a90e1d585318c9d9'
...
* commit '8b51bcfed75721e0d20832e5a90e1d585318c9d9':
vdpau: revector macro to reduce line span
Merged-by: Michael Niedermayer <michaelni@gmx.at >
2014-12-14 22:07:41 +01:00
Thiago Santos
bab8d73765
vaapi: wrap codec specific functions in appropiate #ifs
...
Fix linking when only a subset of vaapi decoders is enabled.
Bug-Id: 760
CC: libav-stable@libav.org
Signed-off-by: Rémi Denis-Courmont <remi@remlab.net >
Signed-off-by: Luca Barbato <lu_zero@gentoo.org >
2014-12-14 22:04:38 +01:00
Michael Niedermayer
5a82ad644f
svq1dec: Unbreak the scratch buffer allocation
...
The input packets are always assumed to be padded and
the av_fast_ family of function takes a pointer to a pointer.
Thanks to Nicolas Dufresne <nicolas.dufresne@collabora.com > for
a similar patch.
Introduced in 7b588bb691 .
Bug-Id: 766
CC: libav-stable@libav.org
Signed-off-by: Michael Niedermayer <michaelni@gmx.at >
Signed-off-by: Luca Barbato <lu_zero@gentoo.org >
2014-12-14 22:03:58 +01:00
Luca Barbato
bcbdeb318e
asvenc: Forward the error instead of asserting on it
...
Bug-Id: CID 732997
CC: libav-stable@libav.org
2014-12-14 22:03:22 +01:00
Rémi Denis-Courmont
559fa0d41b
vdpau: add mapping for H.264 Constrained Baseline profile and fallback
...
Old VDPAU drivers do not support this newly defined profile, so falling
back to Main profile is necessary for backward binary compatibility.
Signed-off-by: Anton Khirnov <anton@khirnov.net >
2014-12-14 18:30:03 +01:00
Rémi Denis-Courmont
8502c1e9ff
vdpau: add mapping for H.264 Extended profile
...
Signed-off-by: Anton Khirnov <anton@khirnov.net >
2014-12-14 18:29:27 +01:00