19 KiB
Freej - free vision mixer
Introduction
FreeJ is a vision mixer: an instrument for realtime video manipulation used in the fields of dance teather, veejaying, medical visualisation and TV. Its development started in 2001 and continued actively for many years.
What we need to do
We need to interact with multiple layers of video, filtered by effect chains and then mixed together.
We want to write scripts to control video mixes with keyboards, midi signals, OSC messages, wiimotes, video mouse and joysticks; manipulating images, movies, live cameras, particle generators, text scrollers, flash animations and more.
We show the resulting video mix on multiple and remote screens, encode it into a movie and stream it live to the internet.
We control the vision mixer locally or remotely, also from multiple places at the same time; all functionalities are designed ad-hoc using javascript.
How FreeJ does it
FreeJ is a commandline application on GNU/Linux, a graphical application on Apple/OSX, a C++ library offering an API for a multimedia framework that relies on different native functions on the operating systems it is ported and, at last, bindings to languages like Python and Ruby (using Swig, more can be implemented as needed.
The code is fairly documented and usable in C++, with full bindings to python, while parsing scripts is done using Spidermonkey, the Mozilla interpreter (there are plans to use also Google V8 in the future).
file:images/mask-sospesa-psico.jpg
FreeJ has started being developed on a dual-core CPU already in 2001 and has grown with emphasis on multi-threading to run efficiently on modern multi-core computers.
Features
With FreeJ we can overlay, mask, transform and filter multiple layers on the screen. There is no limit to the number of layers that can be mixed. Each layer can be video taken from different sources: movie files, webcams, tv cards, images, rendered text, flash animations and more: even particle generators are there, and effects from the frei0r plugin collection.
Stream online video
FreeJ can produce an Ogg/Theora/Vorbis stream for broadcast on Icecast servers, mixing all the video and grabbing the audio from soundcard. The resulting video can be played on any computer connected via internet (list of compatible players).
long live pirate TV! scream your video to the masses! ;)
Instructions on how to stream with FreeJ.
Here are the recordings of a streaming veejay session by Kysucix, screened live at the Linux Audio Developer Conference, all done employing 100% free software: mixer, encoder, codec and players! download the free footage for a demo here ftp.dyne.org/performance/freej-lad2005
VeeJay over Ethernet
The console interface of FreeJ can be remotely operated using ssh with very good responsiveness even on an internet connection. It can be left running and reconnected later and, last but not least, operated from multiple places at the same time - all using screen.
Use all the power and speed of a text console in your video!
This FreeJ console tutorial explores the usage of the console controller (mostly based on the older 0.8 version) and shows you how to load in images and videos, blend them together and put effects on them: it is complete with screenshots and all it needs to start using FreeJ from a console.
This way FreeJ unleashes the power of some GNU/Linux old-school utilities to reach a unique grade of flexibility:
[ascii-box] $ ssh freej-box (authenticate) [freej-box] $ export DISPLAY=localhost:0 [freej-box] $ freej
the freej-box should already be running X on the :0 display, then this will launch a freej running on the remote machine connected to the projector: no more long expensive VGA coaxial cable hassle ;)
Then if you hook up a running freej with a screen -x you can control it from multiple connections at the same time!
Procedural Video Scripting
FreeJ is an asynchronous video rendering engine, it can be scripted using javascript syntax in an object oriented way, to control its operations thru a procedural list of commands and actions.
We can start talking about procedural video as an evolution of the current non/linear paradigm widely spread in video production. In fact, algorithmical approach to video has been widespread since the early ages of the demoscene ;)
A FreeJ scripting API reference is available.
More features in brief…
- 100% FREE GNU GPL Software
- live compositing of multiple webcams, TV signals, movie files, images, TXT files, particle generators and more..
- can be remotely controlled (<b>VJoE</b>)
- can be scripted in procedural object oriented language
- can playback flash vectorial animations
- no frame drop when looping movie clips
- Emacs/Vi style console with hotkeys (S-Lang)
- can accept asynchronous controllers at the same time (Midi, joystick and more coming..)
- very efficient video engine with multithreaded layers
- modular C/C++ code and flexible API
Requirements
- a GNU/Linux or Apple/OSX workstation (see README.OSX)
- Simple Directmedia Layer library
- S-Lang console library
Internals
FreeJ is written with efficiency in mind, benefits of a realtime object oriented and multithreaded architecture where layers and controllers all run independently, to take advantage of multiple CPUs and clustered systems.
The language employed in development is C/C++ respecting POSIX compliance and avoiding the computational bloat of some 'advanced' C++ functions, which makes it highly portable. The FreeJ Debian package for instance is also distributed in binaries for ARM and MIPS processors.
Its C++ programming API is fairly understandable, here you'll find a brief introduction to it.
Developers
The FreeJ team at Piksel 2005 in Norway, from left to right: Kysucix, Mr.Goil and Jaromil
file:images/freej-at-piksel2005.jpg
The FreeJ team together with developers from Engagemedia, Giss.tv and Mal de Ojos at Wintercamp 2009 in Amsterdam
file:images/dyne-at-wintercamp.jpg
The FreeJ source code is mostly written by Denis "Jaromil" Roio, Silvano "Kysucix" Galliani, Christoph "Mr.Goil" Rudorff, Andrea "Xant" Guzzo, Luca "Shammash" Bigliardi and Filippo "Godog" Giunchedi with contributions by Pablo "Caedes" Martines, Lluis Gomez I Bigorda, Ramiro Cosentino, Tatiana de la O and Andy Nicholson. Check the AUTHORS documentation for complete references.
Parts of libraries are written by Andreas Schiffler (sdl_gfx), Jan (theorautils), Dave Griffiths (audio bus), Nemosoft (ccvt), Charles Yates (yuv2rgb), Steve Harris (liblo), Sam Lantinga (sdl_*), Jean-Christophe Hoelt (goom), L. Donnie Smith (cwiid), Olivier Debon (flash).
Documentation, testing and user case studies have bee contributed by: Anne-Marie Skriver, Marloes de Valk, Robert de Geus, Piotr Sobolewski, Alejo Duque, Vladimir Flores Garcia and Gabriele Zaverio.
Besides the passionate commitment of its creators, FreeJ development is made possible also thanks to modest funding and infrastructure support by European institutions, organisations and individuals: among them and most importantly are NIMk, Digitale Pioniers and Open Source Alliance.
| file:images/NIMK.NL-Logo.png | file:images/digitalepioniers.png | file:images/osalliance.png |
Please do not hesitate to contact us if you can help to make this software better: code contributions, financial support, ad-hoc streaming TV setups, festivals and project collaborations are among the most welcome proposals.
Get FreeJ
This software is free and open source, you are free to download it, use it, study, modify it and redistribute it, even for commercial purposes, as long as you release your creations the same way, granting your "users" the same rights we grant to you. Share the Freedom! :)
For more information see the GNU General Public License.
Below a list of formats you can download this application: ready to be run with some of the interfaces developed, as a library you can use to build your own application and as source code you can study.
Source code
Latest stable release is 0.10 (30 May 2008) more about it in the NEWS and ChangeLog.
Source releases are checked and signed by Jaromil using GnuPG.
On ftp.dyne.org/freej you find all present and past FreeJ releases, source code for extra plugins and more binaries that we occasionally build for various architectures.
A mirror is kindly made available by the Ljudmila medialab.
The bleeding edge version is developed on our code repository using git, you can clone the repository free and anonymously
git clone git://code.dyne.org/freej.git
please use this version when reporting bugs and getting in contact with us.
Debian GNU/Linux
Here is an overview of packages and quality assurance provided by Debian. Using Synaptic, simply install freej (or apt-get install freej of course)
packages in Debian are updated frequently, to compile from source use the following dependencies:
sudo apt-get install c++-compiler libtool flex bison libsdl-dev \ libpng-dev libfreetype6-dev libfontconfig-dev dpatch libogg-dev \ libvorbis-dev libjpeg-dev libslang2-dev libtheora-dev \ libavcodec-dev libavformat-dev libbluetooth2-dev fftw3-dev \ libjack-dev libasound-dev liblo0-dev swig python-dev
Apple Darwin/OSX 10.5 / 10.6
Download either the i386/PPC or the x86_64 (snow leopard only) FreeJ with support for quicktime, freeframe and frei0r effect plugins here: this version includes an experimental graphical interface (Carbon/Cocoa) that shows functionalities.
We are looking for designers to help make it look better! please contact one if you like to be the one ;)
Ubuntu GNU/Linux
Here is an overview of packages hosted by Ubuntu. Using Synaptic, simply install freej (or apt-get install freej of course).
If you like, you can also download our packages:
or add "deb http://ppa.launchpad.net/jaromil/ubuntu hardy main" to your /etc/apt/sources
Arch GNU/Linux
FreeJ is packaged in Arch since long time and has been just updated recently, you can see the FreeJ package on AUR and install it from yaourt of just download the PKGBUILD
yaourt freej-git
OpenSuse
RPM Packages are available for all architectures for the Open Suse distribution, here the package information including the 1 click install.
Winslows
If you are looking for a winslows version, boot dyne:bolic GNU/Linux!
Documentation
User's Manual
Existing graphical interfaces can introduce you quickly to its usage. The FreeJ Manual (PDF) helps you getting started with installation and proceeds introducing you to the power-use of FreeJ via scripting. Of course an "Hello World" example is here
text = new TxtLayer();
text.print("Hello world!");
text.start();
add_layer(text);You can paste the code above into a file "hello.js" and execute it from commandline or using the "Load script" button from Apple/OSX.
freej -j hello_world.js
Here you find the full scripting API reference and the online collection of example scripts.
Along with the sourcecode, the following text files are also distributed: README, INSTALL, AUTHORS, ChangeLog, USAGE, TODO, NEWS, COPYING
Streaming online video
You can stream online your video: FreeJ encodes using the Ogg/Theora codec and broadcasts to an Icecast server.
And while streaming, is also possible to save a local copy of the video, all realtime. Here below a script example:
// Audio input is taken via Jack from other applications running
// port name buffer size samlerate
audio = new AudioJack("alsaplayer", 2048, 44100);
// tweak the values below accordingly, see Jack documentation
// Create a Video Encoder object
// V=video A=audio V quality V bitrate A quality A bitrate
encoder = new VideoEncoder(10, 64000, 0, 24000);
// Add the audio channel in the video encoded
encoder.add_audio(audio);
// Configure the encoder to stream over an Icecast server
encoder.stream_host("giss.tv");
encoder.stream_port(8000);
encoder.stream_title("testing new freej");
encoder.stream_username("source");
encoder.stream_password("2t645");
encoder.stream_mountpoint("freej-test.ogg");
// Register the encoder on the running FreeJ engine
register_encoder(encoder);
// Start a network stream
encoder.start_stream();
// Record the stream into a local file
encoder.start_filesave('Video/freej-test.ogg');You'll find more complete instructions on our wiki on lab.dyne.org/FreeJStreaming, please feel free to contribute more documentation to that wiki page.
Procedural Video Scripting
It is possible to script actions in FreeJ using an Object Oriented interface with Javascript procedural syntax. This approach discloses a new range of possibilities for manipulating video, while offering a familiar syntax for web developers.
Besides the User's Manual linked above, here below some sources of documentation:
Video manipulation theory
Some interesting links to online publications about video manipulation techniques:
API for C++ programmers
FreeJ is a library that can be linked shared and used by your application. It is fairly easy to be approached by programmers, documented in this simple text file as well in this doxygen code documentation.
Communication
If you are new to mailinglists, please take your time to read those Netiquette Guidelines (RFC1855) and in particular the chapter 3.0 about One-to-Many Communication.
The mailinglists we use for discussion are completely free, they are published on the web and can be searched in publicly available archives. Posting to them is limited to subscribers and subscription is free.
Please do not send bugreports on any mailinglist, use instead our bugtracker which helps tracking down the reports and keeping them in order.
Your email details will be never disclosed: neither in the public mail archives, neither will be sold to anyone. This service is provided "as is", in the hope it will be useful but without any other warranty.
Anyone posting is the sole responsible for any information shared.
Get in contact with us via our freej at lists.dyne.org mailinglist.
<table align="center" width="100%"><tr>
<td>
<font size="+1">
<a href="http://lists.dyne.org/lurker/list/freej.en.html">
<img src="
" alt="archive" align="middle" border="0">browse the archive</a>
</font>
</td>
<td> <form method="POST" action="http://lists.dyne.org/subscribe.php"> join: <input type="text" size="12" name="addr" value=""> <input type="hidden" name="ml" value="freej"> <input type="submit" name="submit" value=">>"> </form> insert your email<br> </td>
<td> post to <b>freej@lists.dyne.org</b><br> (only for subscribers)<br> go <a href="http://mailinglists.dyne.org/mailman/options/freej">here to unsubscribe</a> <!– <br>need help?<br> ask: <b>freej-help@dyne.org</b></p>–> </td>
<td> <a href="http://lists.dyne.org/stats/freej.html"> <img src="images/globe.png" border="0" align="left" alt="world"> subscribers<br> statistics</a> </td>
</tr></table>