Commit Graph

15211 Commits

Author SHA1 Message Date
Mans Rullgard
d5207e2af8 vqavideo: return error if image size is not a multiple of block size
The decoder assumes in various places that the image size
is a multiple of the block size, and there is no obvious
way to support odd sizes.  Bailing out early if the header
specifies a bad size avoids various errors later on.

Fixes CVE-2012-0947.

Signed-off-by: Mans Rullgard <mans@mansr.com>
(cherry picked from commit 58b2e0f0f2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-05-04 22:14:26 +02:00
Alex Converse
9ea94c44b1 celp filters: Do not read earlier than the start of the 'out' vector.
CC: libav-stable@libav.org
(cherry picked from commit 37ddd38332)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-05-04 22:09:27 +02:00
Alex Converse
aaa6a66677 motionpixels: Clip YUV values after applying a gradient.
Prevents illegal reads on truncated and malformed input.

CC: libav-stable@libav.org
(cherry picked from commit b5da848fac)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-05-04 22:09:27 +02:00
Ronald S. Bultje
7240cc3f8b jpeg: handle progressive in second field of interlaced.
Progressive data is allocated later in decode_sof(), not allocating
that data leads to NULL dereferences.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 5eec5a79da)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-05-04 22:09:27 +02:00
Ronald S. Bultje
7fe4c8cb76 h263: more strictly forbid frame size changes with frame-mt.
Prevents crashes because the old check was incomplete.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 2d22d4307d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-05-04 22:09:27 +02:00
Ronald S. Bultje
746f1594d7 h264: additional protection against unsupported size/bitdepth changes.
Fixes crashes in codepaths not covered by original checks.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
(cherry picked from commit 732f9fcfe5)

Conflicts:

	libavcodec/h264.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-05-04 22:09:27 +02:00
Ronald S. Bultje
0e4bb0530f tta: prevents overflows for 32bit integers in header.
This prevents sample_rate/data_length from going negative, which
caused various crashes and undefined behaviour further down.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit ac80b812cd)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-05-04 21:28:45 +02:00
Paul B Mahol
994c0efcc7 ttadec: CRC checking
Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit 2af3dc8698)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-05-04 21:28:35 +02:00
Paul B Mahol
cf5e119d4a tta: use skip_bits_long()
Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
(cherry picked from commit 9aff2d1753)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-05-04 21:28:28 +02:00
Michael Niedermayer
e8050f313e apedec: check bits <= 32.
Fixes a floating-point exception further down.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
(cherry picked from commit 420d1df2e2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:03 +02:00
Ronald S. Bultje
be424d86a8 truemotion: forbid invalid VLC bitsizes and token values.
SHOW_UBITS() is only defined up to n_bits is 25, therefore forbid
values larger than this in get_vlc2() (max_bits). tokens[][] can be
used as an index in deltas[], which has a size of 64, so ensure the
values are smaller than that.

This prevents crashes on corrupt bitstreams.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit b7b1509d06)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:03 +02:00
Ronald S. Bultje
46f8bbfc6d truemotion2: handle out-of-frame motion vectors through edge extension.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit bf39d3b59d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:03 +02:00
Ronald S. Bultje
562c6a7bf1 lzw: prevent buffer overreads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit ddcf67c8a5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:03 +02:00
Ronald S. Bultje
e711ccee4d truemotion2: convert packet header reading to bytestream2.
Also use correct buffer sizes in calls to tm2_read_stream(). Together,
this prevents overreads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit bd508d435b)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:03 +02:00
Ronald S. Bultje
d6372e80fe lagarith: fix buffer overreads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 0a82f5275f)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:03 +02:00
Ronald S. Bultje
29d91e9161 raw: forward avpicture_fill() error code in raw_decode().
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 98df2e2414)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:02 +02:00
Mashiat Sarker Shakkhar
583f57f04a vc1: Do not read from array if index is invalid.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 95b192de5d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:02 +02:00
Ronald S. Bultje
f8f6c14f54 utvideo: port header reading to bytestream2.
Fixes crash during slice size reading if slice_end goes negative.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit ec0ed97b04)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:02 +02:00
Paul B Mahol
9e24f2a1f0 bytestream: add more unchecked variants for bytestream2 API
Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit f1ce053cd0)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:02 +02:00
Aneesh Dogra
e788c6e9cb bytestream: K&R formatting cosmetics
Signed-off-by: Diego Biurrun <diego@biurrun.de>
(cherry picked from commit ab9ae40152)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:02 +02:00
Aneesh Dogra
2e681cf50f bytestream: Add bytestream2 writing API.
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
(cherry picked from commit db7d45237a)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:02 +02:00
Alex Converse
9ddd3abe78 aac: Reset PS parameters on header decode failure.
If the next header frame codes zero envelopes the previous frame's
values will be used. Consequently the invalid values must be cleared.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit a237b38021)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:02 +02:00
Ronald S. Bultje
c21b858b27 vqa: check palette chunk size before reading data.
Prevents overreads beyond buffer boundaries.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 75d7975268)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:01 +02:00
Paul B Mahol
0b9bb581fd vqavideo: port to bytestream2 API
Protects against overreads.

Signed-off-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 5a3a906ba2)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:01 +02:00
Ronald S. Bultje
105601c151 wmavoice: fix stack overread.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 262196445c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:01 +02:00
Ronald S. Bultje
3a4949aa50 indeo4: fix out-of-bounds function call.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org

Signed-off-by: Kostya Shishkov <kostya.shishkov@gmail.com>
(cherry picked from commit 68fd077f68)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:01 +02:00
Ronald S. Bultje
bf3998d71e mimic: don't use self as reference, and report completion at end of decode().
Fixes hangs on corrupt samples that reference self-frames.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 80387f0e25)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:01 +02:00
Ronald S. Bultje
87208b8fc4 mpeg4: report frame decoding completion at ff_MPV_frame_end().
Prevents hangs on corrupt input.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit c6ccb96bc9)

Conflicts:

	libavcodec/mpegvideo.c

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-04-29 22:07:01 +02:00
Kostya Shishkov
1ee0cd1ad7 dca: include libavutil/mathematics.h for possibly missing M_SQRT1_2
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
2012-03-14 23:32:15 +01:00
Ronald S. Bultje
b594732475 dca: don't use av_clip_uintp2().
The argument is not a literal, thus causing the ARM v6 or later
builds to break.

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
2012-03-14 23:30:19 +01:00
Michael Niedermayer
ce15406e78 snow: check reference frame indices.
Fixes NULL ptr dereference

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 1f8ff2b13c)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:35:09 +01:00
Michael Niedermayer
c9e95636a8 snow: reject unsupported chroma shifts.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit c9837954e7)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:34:55 +01:00
Ronald S. Bultje
6e5c07f4c8 xa_adpcm: limit filter to prevent xa_adpcm_table[] array bounds overruns.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 86020073db)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:34:36 +01:00
Ronald S. Bultje
c999a8ed65 h264: increase reference poc list from 16 to 32.
Interlaced images can have 32 references (16 per field), so limiting the
array size to 16 leads to invalid writes.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 48cbe4b092)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:34:13 +01:00
Ronald S. Bultje
4d343a6f47 h264: stricter reference limit enforcement.
Progressive images can have only 16 references, error out if there are
more, since the data is almost certainly corrupt, and the invalid value
will lead to random crashes or invalid writes later on.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit e0febda22d)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:33:15 +01:00
Michael Niedermayer
a81a6d9c80 h264: improve parsing of broken AVC SPS
Parsing the entire NAL as SPS fixes decoding of some AVC bitstreams
with broken escaping. Since the size of the NAL unit is known and
checked against the buffer end we can parse it entirely without buffer
overreads.

Fixes playback of
http://streams.videolan.org/streams/mp4/Mr_MrsSmith-h264_aac.mp4

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
(cherry picked from commit 3aa661ec56)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:27:22 +01:00
Alex Converse
48f0eeb2e5 Replace computations of remaining bits with calls to get_bits_left().
(cherry picked from commit 3574a85ce5)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:27:16 +01:00
Ronald S. Bultje
d26e47bf6c png: convert to bytestream2 API.
Protects against overreads in the input buffer.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 4c25269ced)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:14:28 +01:00
Ronald S. Bultje
568a474a08 roqvideo: convert to bytestream2 API.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit cdf1577162)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:09:40 +01:00
Ronald S. Bultje
9a66cdbc16 smc: port to bytestream2 API.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 8febcb9fc1)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:09:28 +01:00
Ronald S. Bultje
ddb1149e25 tgq: convert to bytestream2 API.
This protects against input buffer overreads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 1255eed533)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:09:19 +01:00
Ronald S. Bultje
f6778f58d4 algmm: convert to bytestream2 API.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit a55d5bdc6e)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:09:19 +01:00
Paul B Mahol
e4e4d92641 jvdec: unbreak video decoding
The safe bitstream reader broke it since the buffer size was specified
in bytes instead of bits.

Signed-off-by: Janne Grunau <janne-libav@jannau.net>
CC: libav-stable@libav.org
(cherry picked from commit a1c036e961)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:02:23 +01:00
Michael Niedermayer
de0ff4ce69 h264: Fix invalid interlaced/progressive MB combinations for direct mode prediction.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
(cherry picked from commit 758ec11153)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:00:52 +01:00
Anton Khirnov
6548cb2578 libx264: add 'stats' private option for setting 2pass stats filename.
x264 always opens the file itself with fopen, so we cannot use the
standard lavc stats mechanism.

CC: libav-stable@libav.org
(cherry picked from commit d533e395e1)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:00:12 +01:00
Anton Khirnov
f6257cf4b7 libx264: fix help text for slice-max-size option.
CC: libav-stable@libav.org
(cherry picked from commit 9d5c131ece)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 21:00:02 +01:00
Janne Grunau
d94256d36c Revert "h264: clear trailing bits in partially parsed NAL units"
This reverts commit 729ebb2f18.

There was an off-by-one error in the bit mask calculation clearing
actually the last valid bit and causing
http://bugzilla.libav.org/show_bug.cgi?id=227

The broken sample (Mr_MrsSmith-h264_aac.mp4) the commit was fixing
does not work after correcting the off-by-one error.

CC: libav-stable@libav.org
(cherry picked from commit 8a6037c390)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 20:56:55 +01:00
Ronald S. Bultje
7bb97a61df mpc: pad mpc_CC/SCF[] tables to allow for negative indices.
MPC8 allows indices of mpc_CC up to -1, and mpc_SCF up to -6, thus pad
the tables by that much on the left end.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit d7eabd5042)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 20:48:29 +01:00
Ronald S. Bultje
c65eadee5d xxan: protect against chroma LUT overreads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit f77bfa8376)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 20:47:19 +01:00
Ronald S. Bultje
a43f4bd601 xxan: convert to bytestream2 API.
Protects against overreads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
(cherry picked from commit 5518827816)

Signed-off-by: Reinhard Tartler <siretart@tauware.de>
2012-03-14 20:47:19 +01:00