Fixes: floating point division by 0
Fixes: -nan is outside the range of representable values of type 'int'
Fixes: Ticket8307
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4f49fa6abe)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: floating point division by 0
Fixes: undefined behavior in handling NaN
Fixes: Ticket 8268
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3d500e62f6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array access
Fixes: Ticket8240
Fixes: CVE-2020-22021
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7971f62120)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Even in this scenario, the frame still contains references to data that
won't be freed if the frame isn't unreferenced. And the AVFrame itself
will leak, too.
Fixes Coverity issue #1441422.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 212077eda4)
apply_palette() would free an AVFrame given to it only via an AVFrame *
(and not via AVFrame **) in three of its four exists (namely in the
normal path and in two error paths). So upon error the caller has no way
to know whether the frame has already been freed or not;
load_apply_palette(), the only caller, opted to free the frame in this
scenario.
This commit changes this by making apply_palette not freeing the frame
at all, which is left to load_apply_palette().
Fixes Coverity issue #1452434.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit adea33f465)
Affected every usage of this filter; in particular, it affected the
FATE-tests filter-2xbr, filter-3xbr and filter-4xbr.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit fa21194326)
The unsharp filter uses an array of arrays of uint32_t, each of which is
separately allocated. These arrays also need to freed separately; but
before doing so, one needs to check whether the array of arrays has
actually been allocated, otherwise one would dereference a NULL pointer.
This fixes#8408.
Furthermore, the array of arrays needs to be zero-initialized so that
no uninitialized pointer will be freed in case an allocation of one of
the individual arrays fails.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 710ab13693)
Fixes: signed integer overflow: 1562273630 * 17 cannot be represented in type 'int'
Fixes: Ticket8323
Found-by: Suhwan
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0c0ca0f244)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The linearize function (usually refered to EOTF) is the inverse of
delinearize function (usually referred to OETF). Demarcation point of
EOTF should be beta*delta, but the actual value used now in the source
code is beta.
For ITU Rec.709, they are 0.081 (0.018*4.5) and 0.018 respectively
(beta = 0.018 and delta = 4.5), and they correspond to pixel value 5
and 21 for an 8-bit image. Linearized result of pixel within that range
(5-21) will be different, but this commit will make linearize function
of the filter more accurate in the mathematical sense.
Signed-off-by: Yonglin Luo <vincenluo@tencent.com>
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
