added audacity scripting

README.md

@@ -10,14 +10,15 @@ Script | Description
 ------ | -----------
 [Drop h264 I-Frames](drop-h264-iframes/) | Removes every reference frames from a video, except the first one.
 [Optical Flow Transfer](optical-flow-transfer/) | Transfer optical flow from one video to an image.
+[Audacity Scripting](audacity-scripting/) | Frame by frame datamoshing relying on Audacity.
 
 ## Examples
 
 Here are some videos made using those scripts (click on the thumbnails to see the videos):
 
-Drop h264 I-Frames | Optical Flow Transfer
------------------- | ---------------------
-[![](drop-h264-iframes/example.gif)](https://drive.chalier.fr/protected/datamoshing/sunrise-dive.mp4) | [![](optical-flow-transfer/example.gif)](https://drive.chalier.fr/protected/datamoshing/optical-flow-transfer-output.mp4)
+Drop h264 I-Frames | Optical Flow Transfer | Audacity Scripting
+------------------ | --------------------- | ------------------
+[![](drop-h264-iframes/example.gif)](https://drive.chalier.fr/protected/datamoshing/sunrise-dive.mp4) | [![](optical-flow-transfer/example.gif)](https://drive.chalier.fr/protected/datamoshing/optical-flow-transfer-output.mp4) | ![](audacity-scripting/example.gif)
 
 ## Demo
 

audacity-scripting/README.md

@@ -0,0 +1,55 @@
+# Audacity Script
+
+Frame by frame datamoshing relying on Audacity.
+
+## Getting Started
+
+### Prerequisites
+
+You'll need a working installation of [Python 3](https://www.python.org/), and [FFmpeg](https://ffmpeg.org/). Make sure they are in PATH.
+
+You'll also need [Audacity](https://www.audacityteam.org/), and to enable the [mod-script-pipe] module. According to [the documentation](https://manual.audacityteam.org/man/scripting.html):
+
+1. Run Audacity
+2. Go into **Edit** > **Preferences** > **Modules**
+3. Choose **mod-script-pipe** (which should show **New**) and change that to **Enabled**.
+4. Restart Audacity
+5. Check that it now does show **Enabled**.
+
+### Installation
+
+Download or clone this repository:
+
+```console
+git clone https://github.com/ychalier/datamoshing.git
+cd datamoshing/audacity-script/
+```
+
+Install the requirements:
+
+```console
+pip -m install requirements.txt
+```
+
+### Usage
+
+1. Run Audacity
+2. Run the [audacity.py](audacity.py) script with the following parameters:
+   ```console
+   python audacity.py <input-video> <input-filter> <output-video>
+   ```
+
+The input filter is a preset text file for the *Filter Curve* effect in Audacity. You'll find some in the [filters](filters) folder. You can create one by running Audacity, opening the *Filter Curve* effect, messing around with the EQ line, and exporting it as a preset .txt file.
+
+## Examples
+
+This grid shows examples for several filter presets.
+
+![](grid.gif)
+
+## References
+
+- [mod-script-pipe documentation](https://manual.audacityteam.org/man/scripting.html)
+- [mod-script-pipe test files](https://github.com/audacity/audacity/blob/master/au3/scripts/piped-work/) (location in docs is deprecated)
+- [Audacity Scripting Reference](https://manual.audacityteam.org/man/scripting_reference.html)
+- [OSError: [Errno 22] Invalid argument](https://forum.audacityteam.org/t/audacity-pipe-python-errno-22/57799/8) (grrr 🥸)

audacity-scripting/audacity.py

@@ -0,0 +1,144 @@
+import argparse
+import io
+import os
+import subprocess
+import sys
+import tempfile
+import wave
+
+import tqdm
+
+
+def get_video_framerate(video_path: str) -> int:
+    result = subprocess.run([
+        "ffprobe",
+        "-v", "error",
+        "-select_streams", "v:0",
+        "-show_entries", "stream=r_frame_rate",
+        "-of", "default=noprint_wrappers=1:nokey=1",
+        video_path,
+    ], stdout=subprocess.PIPE)
+    return int(result.stdout.decode().strip().split("/")[0])
+
+
+class Audacity:
+
+    def __init__(self):
+        self.tofile: io.TextIOWrapper
+        self.fromfile: io.TextIOWrapper
+        self.eol: str
+
+    def __enter__(self):
+        if sys.platform == 'win32':
+            toname = '\\\\.\\pipe\\ToSrvPipe'
+            fromname = '\\\\.\\pipe\\FromSrvPipe'
+            self.eol = '\r\n\0'
+        else:
+            toname = '/tmp/audacity_script_pipe.to.' + str(os.getuid())
+            fromname = '/tmp/audacity_script_pipe.from.' + str(os.getuid())
+            self.eol = '\n'
+        try:
+            self.tofile = open(toname, "w")
+        except FileNotFoundError:
+            sys.exit("Error: Audacity not running")
+        self.fromfile = open(fromname, "rt")
+        return self
+
+    def send_command(self, command):
+        """Send a single command."""
+        self.tofile.write(command + self.eol)
+        self.tofile.flush()
+
+    def get_response(self):
+        """Return the command response."""
+        result = ''
+        line = ''
+        while True:
+            result += line
+            line = self.fromfile.readline()
+            if line == '\n' and len(result) > 0:
+                break
+        return result
+
+    def do(self, command):
+        """Send one command, and return the response."""
+        self.send_command(command)
+        response = self.get_response()
+        return response
+    
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        self.tofile.close()
+        self.fromfile.close()
+
+
+def process(video_path: str, filter_path: str, output_path: str):
+    tempdir = tempfile.gettempdir()
+    os.makedirs(os.path.join(tempdir, "framesin"), exist_ok=True)
+    os.makedirs(os.path.join(tempdir, "framesout"), exist_ok=True)
+    subprocess.Popen([
+        "ffmpeg",
+        "-hide_banner",
+        "-loglevel", "error",
+        "-stats",
+        "-i", video_path,
+        os.path.join(tempdir, "framesin", "%09d.bmp"),
+    ]).wait()
+    with open(filter_path, "r") as file:
+        filter_command = file.read().strip()
+    with Audacity() as au3:
+        for image_name in tqdm.tqdm(os.listdir(os.path.join(tempdir, "framesin"))):
+            image_path = os.path.join(tempdir, "framesin", image_name)
+            with open(image_path, "rb") as file:
+                data = file.read()
+            with wave.open(os.path.join(tempdir, "input.wav"), "wb") as file:
+                file.setnchannels(1)
+                file.setsampwidth(1)
+                file.setframerate(44100)
+                file.writeframesraw(data)
+            au3.do(f"SelectAll: ")
+            au3.do(f"RemoveTracks: ")
+            au3.do(f"Import2: Filename={os.path.realpath(os.path.join(tempdir, 'input.wav'))}")
+            au3.do(f"SelectAll: ")
+            au3.do(filter_command)
+            au3.do(f"Export2: Filename={os.path.realpath(os.path.join(tempdir, 'output.wav'))}")
+            subprocess.Popen([
+                "ffmpeg",
+                "-hide_banner",
+                "-loglevel", "error",
+                "-i", os.path.join(tempdir, "output.wav"),
+                "-c:a", "pcm_mulaw",
+                "-f", "u8",
+                os.path.join(tempdir, "output.ulaw"),
+                "-y",
+            ]).wait()
+            with open(os.path.join(tempdir, "output.ulaw"), "rb") as f:
+                data = f.read()
+            with open(image_path, "rb") as f:
+                buffer = f.read(128)
+            with open(os.path.join(tempdir, "framesout", image_name), "wb") as f:
+                f.write(buffer)
+                f.write(data[len(buffer):])
+    subprocess.Popen([
+        "ffmpeg",
+        "-hide_banner",
+        "-loglevel", "error",
+        "-stats",
+        "-r", str(get_video_framerate(video_path)),
+        "-i", os.path.join(tempdir, "framesout", "%09d.bmp"),
+        "-pix_fmt", "yuv420p",
+        output_path,
+    ]).wait()
+    os.startfile(output_path)
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("video_path", help="Path to the video file")
+    parser.add_argument("filter_path", help="Path to the filter file")
+    parser.add_argument("output_path", help="Path to the output file")
+    args = parser.parse_args()
+    process(args.video_path, args.filter_path, args.output_path)
+
+
+if __name__ == "__main__":
+    main()

audacity-scripting/filters/cut_high.txt

@@ -0,0 +1 @@
+FilterCurve:f0="286,0868" f1="307,82203" FilterLength="8191" InterpolateLin="0" InterpolationMethod="B-spline" v0="-0,085960388" v1="-30,171921"

audacity-scripting/filters/cut_low.txt

@@ -0,0 +1 @@
+FilterCurve:f0="40,59271" f1="60,477181" FilterLength="8191" InterpolateLin="0" InterpolationMethod="B-spline" v0="-30,171921" v1="0,085960388"

audacity-scripting/filters/cut_mid.txt

@@ -0,0 +1 @@
+FilterCurve:f0="99,343058" f1="109,53175" f2="501,54758" f3="526,63944" FilterLength="8191" InterpolateLin="0" InterpolationMethod="B-spline" v0="0,25787926" v1="-30" v2="-29,656158" v3="-0,085960388"

audacity-scripting/filters/phone.txt

@@ -0,0 +1 @@
+FilterCurve:f0="20" f1="200" f10="48000" f2="250" f3="315" f4="400" f5="500" f6="2500" f7="3150" f8="4000" f9="5000" FilterLength="8191" InterpolateLin="0" InterpolationMethod="B-spline" v0="-94,087" v1="-14,254" v10="-88,117" v2="-7,243" v3="-2,245" v4="-0,414" v5="0" v6="0" v7="-0,874" v8="-3,992" v9="-9,993"

audacity-scripting/requirements.txt

@@ -0,0 +1 @@
+tqdm

drop-h264-iframes/README.md

@@ -81,7 +81,3 @@ There's more information on [FFmpeg's documentation](https://trac.ffmpeg.org/wik
 [![](example.gif)](https://drive.chalier.fr/protected/datamoshing/sunrise-dive.mp4)
 
 I wrote some details [on my blog](https://chalier.fr/blog/datamoshing#droppingreferenceframes).
-
-## Known Issues
-
-The resulting video may stutter if too many frames are dropped. It it less noticeable with high FPS videos. Unless I get to time travel, I will probably not address it.