Fixes out of array read
Fixes: 544/clusterfuzz-testcase-5936536407244800.f8bd9b24_8ba77916_70c2c7be_3df6a2ea_96cd9f14
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This fixes ubsan warnings in non debug builds by using unsigned operations
in debug builds the correct signed operations are retained so that overflows
(which should not occur in valid files and may indicate problems in the DSP code
or decoder) can be detected.
Alternatively they can be changed to unsigned unconditionally, then its
not possible though to detect overflows easily if someone wants to test
the DSP code for overflows.
The 2nd alternative would be to leave the code as it is and accept that
there are undefined operations in the DSP code and that ubsan output is
full of them in some cases.
Similar changes would be needed in some other DSP routines
Suggested-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Change the encoding of the original developer name from ISO-8859-1 to UTF-8.
Remove the stale/completed TODO list.
Fix two small typos.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Certain alpha run lengths (for SHQ1/SHQ3/SHQ5) could be stored in
both long and short versions, and we would only accept the short version,
returning -1 (invalid code) for the others. This could cause an
out-of-bounds write on malicious input, as discovered by
Andreas Cadhalpun during fuzzing.
Fix by simply allowing both versions, leaving no invalid codes
in the alpha VLC.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Multichannel joint stereo simply interleaves stereo pairs (6ch: 2ch + 2ch + 2ch), so each pair is decoded separatedly.
***
To test my changes, I converted examples to wav with ffmpeg.exe (old and new), and compared them to see they are byte-exact.
Regular 2ch files (JS and normal) were straightforward to test.
For multichannel, to check each JS pair is correctly decoded separatedly I did:
- manually demux 6ch.msf into 3 pairs and convert them (2ch_1.wav + 2ch_2.wav + 2ch_3.wav)
- convert the 6ch.msf file to wav (with my changes)
- manually demux the 6ch.wav into 3 pairs (6ch_d1.wav + 6ch_d2.wav + 6ch_d3.wav)
- compare each pair (ex. 2ch_3.wav vs 6ch_d3.wav): all pairs are byte-exact.
The new code just processes each JS pair separatedly, there are no algorithm changes.
It could be improved a bit but I'm not sure about typical styles.
I've only seen 6ch .MSF (probably the AT3 spec only supports 2ch audio).
Signed-off-by: bnnm <bananaman255@gmail.com>
Fixes: u263_b-frames_1.avi
Fixes part of Ticket1536
return -1 is used here as it is used in similar code in this function, I intend
to replace it by proper error codes in the whole function.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* commit 'a1f6a2dfdaf9beb42ca66e49d10bfaf5905a0128':
ratecontrol: Reorder functions to avoid forward declarations
Merged, but this seems to break the clear separation of 1-pass vs
2-pass.
Merged-by: Clément Bœsch <u@pkh.me>
* commit 'd639dcdae022130078c9c84b7b691c5e9694786c':
ratecontrol: Move Xvid-related functions to the place they are actually used
Merged-by: Clément Bœsch <u@pkh.me>
The code relies on their validity and otherwise can try to access a NULL
object->rle pointer, causing segmentation faults.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The assumption this is based on is wrong, the code is not always run with bitexact flags
This reverts commit a956164e1e, reversing
changes made to f6005907fd.
Approved-by: James Almer <jamrial@gmail.com>
* commit 'd06dfaa5cbdd20acfd2364b16c0f4ae4ddb30a65':
x86: huffyuv: Use EXTERNAL_SSSE3_FAST convenience macro where appropriate
Merged-by: James Almer <jamrial@gmail.com>
* commit '4efab89332ea39a77145e8b15562b981d9dbde68':
x86: Use *_FAST/*_SLOW CPU feature detection macros where appropriate
Merged-by: James Almer <jamrial@gmail.com>
* commit '0a39c9ac0bfd7345fe676b4e2707d9cec3cbb553':
x86: hpeldsp: Don't check for bitexact flag when initializing VP3-specific code
Merged-by: James Almer <jamrial@gmail.com>
* commit '1dfc3cf89d0eb026af28be46294b85d79499ffb5':
x86: hpeldsp: Split off VP3-specific bits into a separate file
Merged-by: James Almer <jamrial@gmail.com>
* commit 'fca3c3b61952aacc45e9ca54d86a762946c21942':
hevc: Add AVX2 DC IDCT
Mostly noop as we already have that code.
In the ASM, code is merged with the exception of SECTION which is kept
uppercase for consistency with the rest of the codebase.
Still in the ASM, the prototype comment is fixed to honor the '_' added
from the original commit.
idct_dc_proto() is dropped as it's not used anymore here.
Merged-by: Clément Bœsch <cboesch@gopro.com>
* commit 'cc16da75c2f99d92f7a6461100f041352deb6d88':
hevc: Add coefficient limiting to speed up IDCT
Noop again as we have these changes already, only random spacing
changes.
Merged-by: Clément Bœsch <cboesch@gopro.com>
