FreeJ is a *vision mixer*: an instrument for realtime video
manipulation used in the fields of dance teather, veejaying, medical
visualisation and TV. Its development started in 2001 and continued
[[ftp://ftp.dyne.org/perform/freej_git_activity.ogm][actively for many years]].
** What we need to do
We need to interact with *multiple layers* of video, filtered by
*effect chains* and then *mixed together*.
We want to write scripts to control video mixes with keyboards, midi
signals, OSC messages, wiimotes, video mouse and joysticks;
manipulating images, movies, live cameras, particle generators, text
scrollers, flash animations and more.
We show the resulting video mix on *multiple and remote screens*,
*encode it into a movie* and *stream it live* to the internet.
We *control the vision mixer locally or remotely*, also from multiple
places at the same time; all functionalities are *designed ad-hoc
using javascript*.
** How FreeJ does it
FreeJ is a commandline application on GNU/Linux, a graphical
application on Apple/OSX, a C++ library offering an API for a
multimedia framework that relies on different native functions on the
operating systems it is ported and, at last, bindings to languages
like Python and Ruby (using Swig, more can be implemented as needed.
The code is fairly documented and usable in C++, with full *bindings
to python*, while parsing scripts is done using Spidermonkey, the
Mozilla interpreter (there are plans to use also Google V8 in the
future).
file:images/mask-sospesa-psico.jpg
FreeJ has started being developed on a dual-core CPU already in 2001
and has grown with emphasis on multi-threading to run efficiently on
modern multi-core computers.
** Features
With FreeJ we can overlay, mask, transform and filter multiple layers
on the screen. There is no limit to the number of layers that can be
mixed. Each layer can be video taken from different sources: *movie*
files, *webcams*, *tv cards*, *images*, *rendered text*, *flash
animations* and more: even particle generators are there, and effects
from the [[http://frei0r.dyne.org][frei0r]] plugin collection.
** Stream online video
FreeJ can produce an Ogg/Theora/Vorbis stream for broadcast on [[http://www.icecast.org][Icecast]]
servers, mixing all the video and grabbing the audio from
soundcard. The resulting video can be played on any computer connected
via internet ([[http://en.wikipedia.org/wiki/Theora#List_of_Theora_video_players][list of compatible players]]).
*long live pirate TV! scream your video to the masses! ;)*
Instructions on how to [[http://lab.dyne.org/FreejStreaming][stream with FreeJ]].
Here are the recordings of a streaming veejay session by Kysucix,
screened live at the [[http://lac.zkm.de][Linux Audio Developer Conference]], all done
employing 100% free software: mixer, encoder, codec and players!
download the free footage for a demo here
[[ftp://ftp.dyne.org/performance/freej-lad2005/][ftp.dyne.org/performance/freej-lad2005]]
** VeeJay over Ethernet
The console interface of FreeJ can be remotely operated using [[http://en.wikipedia.org/wiki/Ssh][ssh]] with
very good responsiveness even on an internet connection. It can be
left running and reconnected later and, last but not least, operated
from multiple places at the same time - all using [[http://en.wikipedia.org/wiki/GNU_screen][screen]].
*Use all the power and speed of a text console in your video!*
This [[http://lab.dyne.org/FreejTutorial][FreeJ console tutorial]] explores the usage of the console
controller (mostly based on the older *0.8 version*) and shows you how
to load in images and videos, blend them together and put effects on
them: it is complete with screenshots and all it needs to start using
FreeJ from a console.
This way FreeJ unleashes the power of some GNU/Linux old-school
utilities to reach a unique grade of flexibility:
: [ascii-box] $ ssh freej-box
: (authenticate)
: [freej-box] $ export DISPLAY=localhost:0
: [freej-box] $ freej
the freej-box should already be running X on the :0 display, then this
will launch a freej running on the remote machine connected to the
projector: no more long expensive VGA coaxial cable hassle ;)
Then if you hook up a running freej with a *screen -x* you can control
it from multiple connections at the same time!
** Procedural Video Scripting
FreeJ is an asynchronous video rendering engine, it can be scripted
using javascript syntax in an object oriented way, to control its
operations thru a procedural list of commands and actions.
We can start talking about *procedural video* as an evolution of the
current non/linear paradigm widely spread in video production. In
fact, *algorithmical approach to video* has been widespread since the
early ages of the demoscene ;)
A [[/docs/scripting][FreeJ scripting API reference]] is available.
** More features in brief...
* 100% FREE GNU GPL Software
* live compositing of multiple webcams, TV signals, movie files, images, TXT files, particle generators and more..
* can be remotely controlled (VJoE)
* can be scripted in procedural object oriented language
* can playback flash vectorial animations
* no frame drop when looping movie clips
* Emacs/Vi style console with hotkeys (*S-Lang*)
* can accept asynchronous controllers at the same time (Midi, joystick and more coming..)
* very efficient video engine with multithreaded layers
* modular C/C++ code and flexible API
** Requirements
* a GNU/Linux or Apple/OSX workstation (see [[ftp://ftp.dyne.org/freej/README.OSX][README.OSX]])
* [[http://www.libsdl.org][Simple Directmedia Layer]] library
* [[http://www.s-lang.org][S-Lang]] console library
** Internals
FreeJ is written with efficiency in mind, benefits of a realtime
object oriented and multithreaded architecture where layers and
controllers all run independently, to take advantage of multiple CPUs
and clustered systems.
The language employed in development is C/C++ respecting POSIX
compliance and avoiding the computational bloat of some 'advanced' C++
functions, which makes it highly portable. The FreeJ Debian package
for instance is also distributed in binaries for ARM and MIPS
processors.
Its C++ programming API is fairly understandable, here you'll find a
[[ftp://ftp.dyne.org/freej/API][brief introduction]] to it.
** Developers
The FreeJ team at Piksel 2005 in Norway, from left to right: Kysucix,
Mr.Goil and Jaromil
file:images/freej-at-piksel2005.jpg
The FreeJ team together with developers from Engagemedia, Giss.tv and
Mal de Ojos at Wintercamp 2009 in Amsterdam
file:images/dyne-at-wintercamp.jpg
The FreeJ source code is mostly written by Denis "Jaromil" Roio,
Silvano "Kysucix" Galliani, Christoph "Mr.Goil" Rudorff, Andrea "Xant"
Guzzo, Luca "Shammash" Bigliardi and Filippo "Godog" Giunchedi with
contributions by Pablo "Caedes" Martines, Lluis Gomez I Bigorda,
Ramiro Cosentino, Tatiana de la O and Andy Nicholson. Check the
[[ftp://ftp.dyne.org/freej/AUTHORS][AUTHORS]] documentation for complete references.
Parts of libraries are written by Andreas Schiffler (sdl_gfx), Jan
(theorautils), Dave Griffiths (audio bus), Nemosoft (ccvt), Charles
Yates (yuv2rgb), Steve Harris (liblo), Sam Lantinga (sdl_*),
Jean-Christophe Hoelt (goom), L. Donnie Smith (cwiid), Olivier Debon
(flash).
Documentation, testing and user case studies have bee contributed by:
Anne-Marie Skriver, Marloes de Valk, Robert de Geus, Piotr Sobolewski,
Alejo Duque, Vladimir Flores Garcia and Gabriele Zaverio.
Besides the passionate commitment of its creators, FreeJ development
is made possible also thanks to modest funding and infrastructure
support by European institutions, organisations and individuals: among
them and most importantly are [[http://nimk.nl/eng/research/freej-vision-mixer][NIMk]], [[http://www.digitalepioniers.nl/projecten/Freej-Vision-Mixer/146][Digitale Pioniers]] and [[http://osalliance.com/netculture/project/beTV/bericht/][Open Source
Alliance]].
| file:images/NIMK.NL-Logo.png | file:images/digitalepioniers.png | file:images/osalliance.png |
Please do not hesitate to contact us if you can help to make this
software better: code contributions, financial support, ad-hoc
streaming TV setups, festivals and project collaborations are among
the most welcome proposals.
* Get FreeJ
This software is free and open source, you are free to download it,
use it, study, modify it and redistribute it, even for commercial
purposes, as long as you release your creations the same way, granting
your "users" the same rights we grant to you. Share the Freedom! :)
For more information see the [[http://www.gnu.org/copyleft/gpl.html][GNU General Public License]].
Below a list of formats you can download this application: ready to be
run with some of the interfaces developed, as a library you can use to
build your own application and as source code you can study.
** Source code
Latest stable release is 0.10 (30 May 2008) more about it in the [[ftp://ftp.dyne.org/freej/NEWS][NEWS]]
and [[ftp://ftp.dyne.org/freej/ChangeLog][ChangeLog]].
Source releases are checked and signed by [[http://rastasoft.org][Jaromil]] using [[http://www.gnupg.org][GnuPG]].
On [[ftp://ftp.dyne.org/freej][ftp.dyne.org/freej]] you find all present and past FreeJ releases,
source code for extra plugins and more binaries that we occasionally
build for various architectures.
A mirror is kindly made available by the [[http://www.ljudmila.org/~jaromil/mirror/freej/][Ljudmila medialab]].
The bleeding edge version is developed on our [[http://git.dyne.org][code repository]] using *git*,
you can clone the repository free and anonymously
: git clone git://code.dyne.org/freej.git
please use this version when [[http://bugs.dyne.org][reporting bugs]] and getting in contact with us.
** Debian GNU/Linux
Here is an [[http://packages.debian.org/search?searchon=names&keywords=freej][overview of packages]] and [[http://packages.qa.debian.org/freej][quality assurance]] provided by
Debian. Using Synaptic, simply install freej (or *apt-get install freej* of course)
packages in Debian are updated frequently, to compile from source use
the following dependencies:
: sudo apt-get install c++-compiler libtool flex bison libsdl-dev \
: libpng-dev libfreetype6-dev libfontconfig-dev dpatch libogg-dev \
: libvorbis-dev libjpeg-dev libslang2-dev libtheora-dev \
: libavcodec-dev libavformat-dev libbluetooth2-dev fftw3-dev \
: libjack-dev libasound-dev liblo0-dev swig python-dev
** Apple Darwin/OSX 10.5 / 10.6
Download either the [[http://www.xant.net/freej.dmg][i386/PPC]] or the [[http://www.xant.net/freej64.dmg][x86_64 (snow leopard only)]] FreeJ with support for quicktime, [[http://www.troikatronix.com/freeframe.html][freeframe]] and
frei0r effect plugins here: this version includes an experimental
graphical interface (Carbon/Cocoa) that shows functionalities.
We are looking for designers to help make it look better! please
contact one if you like to be the one ;)
** Ubuntu GNU/Linux
Here is an [[http://packages.ubuntu.com/search?keywords=freej&searchon=names&suite=all§ion=all][overview of packages]] hosted by Ubuntu. Using Synaptic,
simply install freej (or *apt-get install freej* of course).
If you like, you can also download our packages:
* [[ftp://ftp.dyne.org/freej/binary/freej_0.10-ppa9~ubuntu8.10_i386.deb][i386 binary (PC x86)]]
* [[ftp://ftp.dyne.org/freej/binary/freej_0.10-ppa9~ubuntu8.10_amd64.deb][Amd-64 binary (PC 64bit)]]
or add "deb http://ppa.launchpad.net/jaromil/ubuntu hardy main" to
your /etc/apt/sources
** Arch GNU/Linux
FreeJ is packaged in Arch since long time and has been just updated
recently, you can [[http://aur.archlinux.org/packages.php?ID=34964][see the FreeJ package on AUR]] and install it from
*yaourt* of just download the PKGBUILD
: yaourt freej-git
** OpenSuse
RPM Packages are available for all architectures for the Open Suse
distribution, here the [[http://packman.links2linux.org/package/freej][package information]] including the [[http://packman.links2linux.org/install/freej][1 click
install]].
** Winslows
If you are looking for a winslows version, boot [[http://dynebolic.org][dyne:bolic GNU/Linux]]!
* Documentation
** User's Manual
Existing graphical interfaces can introduce you quickly to its usage.
The [[http://freej.dyne.org/docs/freej-manual-EN.pdf][FreeJ Manual (PDF)]] helps you getting started with installation and
proceeds introducing you to the power-use of FreeJ via scripting. Of
course an "Hello World" example is here
#+BEGIN_SRC java
text = new TxtLayer();
text.print("Hello world!");
text.start();
add_layer(text);
#+END_SRC
You can paste the code above into a file "hello.js" and execute it
from commandline or using the "Load script" button from Apple/OSX.
: freej -j hello_world.js
Here you find the full [[http://freej.dyne.org/docs/scripting][scripting API reference]] and the online
collection of [[http://git.dyne.org/index.cgi?url=freej/tree/scripts/javascript/examples][example scripts]].
Along with the sourcecode, the following text files are also
distributed: [[ftp://ftp.dyne.org/freej/README][README]], [[ftp://ftp.dyne.org/freej/][INSTALL]], [[ftp://ftp.dyne.org/freej/][AUTHORS]], [[ftp://ftp.dyne.org/freej/][ChangeLog]], [[ftp://ftp.dyne.org/freej/][USAGE]], [[ftp://ftp.dyne.org/freej/][TODO]], [[ftp://ftp.dyne.org/freej/][NEWS]],
[[ftp://ftp.dyne.org/freej/][COPYING]]
** Streaming online video
You can stream online your video: FreeJ encodes using the [[http://theora.org/][Ogg/Theora]]
codec and broadcasts to an [[http://icecast.org][Icecast]] server.
And while streaming, is also possible to save a local copy of the
video, all realtime. Here below a script example:
#+BEGIN_SRC java
// Audio input is taken via Jack from other applications running
// port name buffer size samlerate
audio = new AudioJack("alsaplayer", 2048, 44100);
// tweak the values below accordingly, see Jack documentation
// Create a Video Encoder object
// V=video A=audio V quality V bitrate A quality A bitrate
encoder = new VideoEncoder(10, 64000, 0, 24000);
// Add the audio channel in the video encoded
encoder.add_audio(audio);
// Configure the encoder to stream over an Icecast server
encoder.stream_host("giss.tv");
encoder.stream_port(8000);
encoder.stream_title("testing new freej");
encoder.stream_username("source");
encoder.stream_password("2t645");
encoder.stream_mountpoint("freej-test.ogg");
// Register the encoder on the running FreeJ engine
register_encoder(encoder);
// Start a network stream
encoder.start_stream();
// Record the stream into a local file
encoder.start_filesave('Video/freej-test.ogg');
#+END_SRC
You'll find more complete instructions on our wiki on
[[http://lab.dyne.org/FreejStreaming][lab.dyne.org/FreeJStreaming]], please feel free to contribute more
documentation to that wiki page.
** Procedural Video Scripting
It is possible to script actions in FreeJ using an Object Oriented
interface with *Javascript* procedural syntax. This approach discloses
a new range of possibilities for manipulating video, while offering a
familiar syntax for web developers.
Besides the User's Manual linked above, here below some sources of
documentation:
* [[ftp://ftp.dyne.org/freej/freej_scripting.txt][overview of scripting API]]
* [[http://lab.dyne.org/FreejScripting][wiki notes on scripting]]
* [[http://freej.dyne.org/docs/javascript-1.5-core-documentation.tar.gz][Javascript 1.5 core documentation]]
** Video manipulation theory
Some interesting links to online publications about video manipulation
techniques:
* [[http://www.gamedev.net/reference/articles/article2007.asp][Introduction to digital image processing]]
* [[http://homepages.inf.ed.ac.uk/rbf/HIPR2/][Image Processing Learning Resources]]
* [[http://www.engineering.uiowa.edu/~dip/LECTURE/lecture.html][Iowa engineering univ. lectures]]
* [[http://yov408.free.fr/][YOV 408 technologies]]
** API for C++ programmers
FreeJ is a library that can be linked shared and used by your
application. It is fairly easy to be approached by programmers,
documented in this [[ftp://ftp.dyne.org/freej/API][simple text file]] as well in this [[http://freej.dyne.org/codedoc][doxygen code
documentation]].
* Communication
If you are new to mailinglists, please take your time to read those
[[http://www.rfc-editor.org/rfc/rfc1855.txt][Netiquette Guidelines]] (RFC1855) and in particular the chapter 3.0
about One-to-Many Communication.
The mailinglists we use for discussion are *completely free*, they are
published on the web and can be searched in *publicly available
archives*. Posting to them is limited to subscribers and
*subscription is free*.
*Please do not send bugreports on any mailinglist*, use instead our
[[http://bugs.dyne.org][bugtracker]] which helps tracking down the reports and keeping them in
order.
*Your email details will be never disclosed*: neither in the public
mail archives, neither will be sold to anyone. This service is
provided "as is", in the hope it will be useful but without any other
warranty.
Anyone posting is the sole responsible for any information shared.
Get in contact with us via our *freej at lists.dyne.org* mailinglist.
#+BEGIN_HTML
browse the archive
subscribers