This behavior is currently completely broken, leading to an abrupt end of the
first audio stream. I want to generalize this filter to multiple inputs, but
having too short input files will always represent a significant problem.
I considered a few approaches for how to handle this more gracefully, but
most of them come with their own problems; in particular when a short input
is sandwiched between two longer ones; or when there is a sequence of short
inputs. In the end, it's simplest to just shorten the crossfade window.
I also considered (and tested) padding the input with silence, but this also
has its own aesthetic implications and strange edge cases.
See-Also: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20388
There's no reason to use a completely separate graph just to process the
alpha plane in isolation - zimg supports native alpha handling as part of the
main image.
Fixes several issues with this filter when adding or removing alpha planes,
and also adds support for scaling premultiplied alpha (which reduces artefacts
near the image borders).
Fortunately, we only care if this flag is set - otherwise, this filter is
alpha mode agnostic (since it is purely scaling, etc).
That said, there is an argument to be made that we should prefer premul
alpha when scaling, because scaling in straight alpha will leak garbage
pixels; but I think that would be a too backwards-incompatible change to
be worth thinking about at this time.
The overwhelming majority of references to ff_draw_init2() just set the
colorspace properties from a filter link. This wrapper allows us to update
all such referencen automatically.
Conceptually, these are pretty simple to handle, since they are basically
equivalent to the case of alpha being absent, since the only thing the
destination alpha plane is used for is unpremultiplyng straight alpha.
For planar formats, the total number of cases doesn't change in principle,
since before this patch we have:
- alpha present, overlay straight
- alpha present, overlay premultiplied
- alpha absent, overlay straight
- alpha absent, overlay premultiplied
And now we have:
- main straight, overlay straight
- main straight, overlay premultiplied
- main premultiplied, overlay straight
- main premultiplied, overlay premultiplied
That said, we do gain some cases now that we used to (incorrectly) ignore,
like premultiplied yuva420p10.
Notably, we can skip defining separate functions for the case of main alpha
being absent, since that can be a single cheap branch inside the function
itself, on whether or not to also process the alpha plane. Otherwise, as long
as we treat "alpha absent" as "main premultiplied", the per-plane logic will
skip touching the nonexistent alpha plane.
The only format that actually needs extra cases is packed rgb, but that's only
two additional cases in total.
Also arguably simplifies the assignment logic massively.
Chooses the desired output alpha mode. Note that this depends on
an upstream version of libplacebo new enough to respect the corresponding
AVFrame field in pl_map_avframe_ex.
While vf_scale cannot directly convert between premultiplied and straight
alpha, the effective tagging can still change as a result of a change in
the pixel format (i.e. adding or removing an alpha channel).
We need a filter that can premultiply and unpremultiply the alpha channel
dynamically, on demand, in response to the negotiated alpha mode, analogous
to how vf_scale operates. Introduce a new filter "vf_premultiply_dynamic"
that accomplishes this.
Set the output link according to the chosen direction, and the input link
to its opposite (unless the input is not in-place, in which case there is no
alpha channel).
Also update the alpha mode on the output frame, since it may differ.
Instead of hard-coding the assumption that all video properties are handled
by vf_scale, generalize the struct slightly to allow different conversion
filters for each property being merged. The avfiltergraph code creates a list
of conversion filters needed and inserts all of them at once.
Because a conversion filter might itself e.g. not support all formats,
it's possible that we need to go through the whole process of negotiating
formats multiple times, and keep adding conversion filters until all of them
are settled. Do this by simply jumping back to the beginning of the loop
to ensure that the `convertor_count` field remains contiguous.
In constant FPS mode, it's possible for there no be *no* input active at
the current timestamp, even though there would be active inputs later in
the timeline (nb_active > 0). This would previously hit the AVERROR_BUG case.
With this change, we can instead output an empty frame.
Instead of having a single s->status field to track the most recently
EOF'd stream, track the number of active streams directly and only query
the most recent status at the end.
The reason this worked before is because we implicitly relied on the
assumption that `ok` would always be true until all streams are EOF, but
because it's possible to have gaps in the timeline when mixing multiple
streams, this is not always the case in principle.
In practice, this fixes a bug where the filter would early-exit (EOF)
too soon, when any input reached EOF and there is a gap in the timeline.
When using libplacebo to composite multiple streams with complex PTS
values, it's possible for there to be a "hole" in the output stream; in
particular in constant FPS mode. This commit refactors output_frame() to
allow it to handle the case of there being no active input.
This wrapping logic still considered any nonzero return from the ASM function
to be the overall result, but this is not true since the addition of
FF_ALPHA_TRANSPARENT.
Fix it by only early returning if FF_ALPHA_STRAIGHT is detected.
Fixes: 9b8b78a815
See-Also: https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/20301#issuecomment-4802
The project introduced API breaking changes to some of their public functions,
and given the library is relatively new, just bump the minimum supported
version instead of adding ifdeffery to the source files.
Signed-off-by: James Almer <jamrial@gmail.com>
It makes sense to treat the presence of a frame duration and the presence
of frame rate metadata identically - because both convey effectively the same
amount of information.
In f121d95 and fa110c3 respectively, this information was stripped by default,
originally to work-around bugs when changing the PTS information of a stream
being fed to some encoders. (See https://trac.ffmpeg.org/ticket/10886)
Later, commit 959b799c restored the ability to preserve the frame rate
medatata via the `strip_fps` option, but this option did not extend to also
include the frame duration.
This commit resolves the scenario by making `frame_rate` and `duration`
handled in a consistent manner, so that the frame rate information is
generally preserved unless explicitly stripped by the user.
While it does regress the exact invocation presented in the trac ticket unless
using `strip_fps=yes`, I consider this an acceptable trade-off, especially in
light of the fact that the `fps` filter also exists and is arguably the better
tool for the task at hand.
It can be useful to know if the alpha plane consists of fully opaque
pixels or not, in which case it can e.g. safely be stripped.
This only requires a very minor modification to the AVX2 routines, adding
an extra AND on the read alpha value with the reference alpha value, and a
single extra cheap test per line.
detect_alpha_8_full_c: 2849.1 ( 1.00x)
detect_alpha_8_full_avx2: 260.3 (10.95x)
detect_alpha_8_full_avx512icl: 130.2 (21.87x)
detect_alpha_8_limited_c: 8349.2 ( 1.00x)
detect_alpha_8_limited_avx2: 756.6 (11.04x)
detect_alpha_8_limited_avx512icl: 364.2 (22.93x)
detect_alpha_16_full_c: 1652.8 ( 1.00x)
detect_alpha_16_full_avx2: 236.5 ( 6.99x)
detect_alpha_16_full_avx512icl: 134.6 (12.28x)
detect_alpha_16_limited_c: 5263.1 ( 1.00x)
detect_alpha_16_limited_avx2: 797.4 ( 6.60x)
detect_alpha_16_limited_avx512icl: 400.3 (13.15x)
This safety margin was motivated by the fact that vf_premultiply sometimes
produces such illegally high values, but this has since been fixed by
603334a043, so there's no more reason to have this safety margin, at
least for our own code. (Of course, other sources may also produce such
broken files, but we shouldn't work around that - garbage in, garbage out.)
See-Also: 603334a043
Basically cosmetic.
I want to expand this to detect more than a single property about the alpha
channel at the same time; so we first need a way for this function to
return a more complex result.
Move the enum AlphaMode to the header and formally generalize the return
signature a bit to allow returning more than just one value.
