Commit 694db39e authored by Yann Mombrun's avatar Yann Mombrun
Browse files

New stable version 1.0.1 fixing blocking bug 180 and 182 of the 1.0.0

OPENAXES-180 - Include last version of ffmpeg 1.1.x and compile it. Update all the components that need it to work with that version (shot extractor, video normaliser, virtual cutter...). Re add the option to include GPL libraries at compile time.
OPENAXES-181 - Properly set version to 1.0.1.
OPENAXES-182 - Update the include/lib paths of caffe and cpuvisor due to a change in HDF5 library on latest Ubuntu distributions.
parent b5f77bff
......@@ -2,12 +2,13 @@ This is the Open AXES package for Linux 64 Debian/Ubuntu based.
Open AXES contains software that have already been compiled and that are available only on such a target platform.
It has been validated on Ubuntu 14.04 and 14.10 as well as on Mint 17.
Moreover, the installation and compilation script is bound on Ubuntu package delivery system and would have to be changed to work on other Linux distributions.
It has been validated on Ubuntu 14.04, 14.10 and 15.04 as well as on Mint 17.1.
The package is made of components with various licences. You will find more details in the data/licences folder.
To make it short, most of the system is delivered using free and open sources non contaminating licences (LGPL, MIT and Apache 2 licences).
Two components are releases with more restrictive licences and without the sources:
To make it short, most of the system is delivered using free and open source non contaminating licences (LGPL, MIT and Apache 2 licences).
Two components are releases with more restrictive licences and without the sources (but are free for research purposes):
- Technicolor shot detector and key frame extractor
- KU Leuven image classifier
......@@ -27,27 +28,30 @@ In order to install Open AXES on your system:
1) Unzip the package on your system.
2) Open a terminal and launch the the installation process by calling the install.sh script.
2) Open a terminal and launch the installation process by calling the install.sh script.
> ./install.sh
It has a few options (listed below) that enables to inclusion of optional functionality (like classifiers...).
It has a few options (listed below) that enables the inclusion of optional functionality (like classifiers, GPL H264 codec for FFMPEG...).
This script should be run by the normal system user.
During the installation you will have to confirm several steps, especially during the installation of the JDK, as well as authorise some installation parts as a super user.
It will install various libraries that are needed like (not exhaustive):
- apache2
- ffmpeg
- python and a lot of python modules
- an Oracle JDK 7,
- opencv, openblas, protobuf and a lot of development tools if OnTheFlyClassifier option is activated.
- opencv, openblas, protobuf and a lot of development tools if includeOnTheFlyClassifier option is activated,
- Matlab Compiler Runtime if the includeClassifier option is activated.
It will also link some folders and give read-write-execute access to others.
A specific version of FFmpeg is also installed locally (it will not overwrite any previously installed FFmpeg).
Options:
--includeClassifier activate the classifiers. It will download and install Matlab CR as well a set of classifiers for a total weight of more than 1Gb.
--includeOnTheFlyClassifier activate the on the fly classifiers (cpuvisor). It will download and compile a lot of libraries and data for a total weight of more than 3Gb.
--nonInteractiveMode to assume yes/true to all question asked during installation.
--includeOnTheFlyClassifier activate the on the fly classifiers (cpuvisor). It will download and compile a lot of libraries and data for a total weight of more than 2Gb.
--includeFFmpegGPL activate GPL and h264 compilation flags for FFmpeg.
--nonInteractiveMode to assume yes/true to all question asked during installation (but Java installation confirmation than cannot be skipped).
> ./install.sh --includeClassifier --includeOnTheFlyClassifier --nonInteractiveMode
> ./install.sh --includeClassifier --includeOnTheFlyClassifier --includeFFmpegGPL --nonInteractiveMode
3) Wait between 10 minutes to 3 hours depending on the bandwidth of your Internet, the power of your computer and the number of activated options.
3) Wait between 10 minutes to 3 hours depending on the bandwidth of your Internet connection, the performance of your computer and the activated options.
......@@ -82,25 +86,24 @@ Know issues and evolution requests at https://jira.ow2.org/issues/?jql=project%2
The version 1.0.0 of the Open AXES is available for download at http://forge.ow2.org/project/showfiles.php?group_id=436.
The version 1.0.1 of the Open AXES is available for download at http://forge.ow2.org/project/showfiles.php?group_id=436.
Open AXES is an output of the AXES European project.
The goal AXES (http://axes-project.eu) is to develop tools that provide various types of users with new engaging ways to interact with audiovisual libraries,
The goal AXES (http://axes-project.eu) was to develop tools that provide various types of users with new engaging ways to interact with audiovisual libraries,
helping them to discover, browse, search and enrich video archives. Based on the existing OW2 WebLab (http://weblab-project.org) integration platform for multimedia processing,
the "Open AXES" solution gathers innovative audiovisual content analyses technologies (shot and keyframes detection, image classification, speech transcription, large scale indexing, etc.)
as well as an ergonomic interface to navigate the video archive.
This version 1.0.0 contains:
- a folder gathering service, provided by Airbus Defence and Space, enabling to collect video files and metadata (in various format : json, properties, rdf)
in a set of relevant folders;
This version 1.0.1 contains:
- a folder gathering service, provided by Airbus Defence and Space, enabling to collect video files and metadata (in various format: json, properties, rdf) in a set of relevant folders;
- a shot-detector and keyframe-extraction service provided by Technicolor and enabling to select some keyframes representative of each shot of the video
(and prevent image based components to analyse every single frame of a video or picking random ones) which is available from binaries and only allowed for non commercial use;
- a video-normaliser service, provided by Airbus Defence and Space and based on FFmpeg, enabling to convert the original video in appropriate formats for other components and UI);
- an image-classifier service, provided by the Katholieke Universiteit Leuven in charge of classifing selected images over a set of more than 1000 classes
- a video-normaliser service, provided by Airbus Defence and Space and based on FFmpeg, enabling to convert the original video in appropriate formats for other components and UI;
- an image-classifier service, provided by the Katholieke Universiteit Leuven in charge of classifying selected images over a set of more than 1000 classes
which is available from binaries and allowed for non commercial use only;
- an automated speech recognition service, provided by Airbus Defence and Space and based on CMU Sphinx, extracting the text from English audio speech;
- a named entity detection service, provided by Airbus Defence and Space and based on GATE platform, extracting organisation, persons and locations from the text of the speeches.
- a named entity detection service, provided by Airbus Defence and Space and based on GATE platform, extracting organisation, persons and locations from the text of the speeches;
- a text and metadata indexing service as well as the search and fuse search interfaces, named LIMAS and provided by the University of Twente,
saving the results of the processes and being the backend server for the UI, also in charge of calling the other search engines and fusing their result.
It is provided with an Apache 2 licence. The codebase is available at: https://bitbucket.org/alyr/limas/;
......@@ -113,9 +116,9 @@ This version 1.0.0 contains:
(either using a pre-trained or by learning on the fly the concept). It enables various operations over the videos like virtual cutting, collection management
and social sharing (like/dislike). The codebase is available at https://github.com/kevinmcguinness/axes-research using an Apache 2 licence.
- two technical components are also needed to let Visor and Limas works together. Jcpuvisor is the Java interface to Visor.
It is available at https://bitbucket.org/alyr/jcpuvisor and provided as open source using an Apache 2 licence. Imsearch-tools is available at
https://github.com/kencoken/imsearch-tools and provided with an Apache 2 licence. Please note however that this later project might be enforcing Google Terms of Use
as well as authoring and image rights when gathering images on Google to send to Visor in order to learn its models.
It is available at https://bitbucket.org/alyr/jcpuvisor and provided as open source using an Apache 2 licence.
Imsearch-tools is available at https://github.com/kencoken/imsearch-tools and provided with an Apache 2 licence. Please note however that this later project
might be enforcing Google Terms of Use as well as authoring and image rights when gathering images on Google to send to Visor in order to learn its models.
All together are packaged into an OW2 WebLab application. When not specified, the licence that apply on the Open AXES code is the LGPL-V2.1.
#!/bin/sh
export LD_LIBRARY_PATH=.
export LD_LIBRARY_PATH=.:../ffmpeg-1.1.16/libavformat:../ffmpeg-1.1.16/libavutil:../ffmpeg-1.1.16/libavcodec:../ffmpeg-1.1.16/libswscale
./frame_extractor -M 1 $1 $2;
#!/bin/sh
export LD_LIBRARY_PATH=.
export LD_LIBRARY_PATH=.:../ffmpeg-1.1.16/libavformat:../ffmpeg-1.1.16/libavutil:../ffmpeg-1.1.16/libavcodec:../ffmpeg-1.1.16/libswscale
./VideoKFToolsFFMPEG -i $1 -o $2;
......@@ -39,8 +39,8 @@
<prop key="TEC.binaries">${weblab.components}/TEC</prop>
<prop key="TEC.cache">${weblab.data}cache/shot-extractor</prop>
<prop key="ffmpeg.path">ffmpeg</prop>
<prop key="normaliser.workingDirectory">${weblab.data}cache/video-normaliser</prop>
<prop key="ffmpeg.path">${weblab.components}/ffmpeg-1.1.16/ffmpeg.sh</prop>
<prop key="normaliser.workingDirectory">${weblab.components}/ffmpeg-1.1.16</prop>
<prop key="normaliser.cache">${weblab.data}cache/video-normaliser</prop>
<prop key="asr.cache">${weblab.data}cache/sphinx-recognition</prop>
......
......@@ -187,3 +187,4 @@ UI_TEXT_SEARCH_TYPES = ['meta', 'speech', 'meta speech']
UI_VISUAL_SEARCH_TYPES = [VISUAL_SEARCH_ACTIVATED]
UI_IMAGE_SEARCH_TYPES = ['instance-i']
FFMPEG = 'FFMPEG_PATH'
\ No newline at end of file
FFmpeg:
Most files in FFmpeg are under the GNU Lesser General Public License version 2.1
or later (LGPL v2.1+). Read the file COPYING.LGPLv2.1 for details. Some other
files have MIT/X11/BSD-style licenses. In combination the LGPL v2.1+ applies to
FFmpeg.
Some optional parts of FFmpeg are licensed under the GNU General Public License
version 2 or later (GPL v2+). See the file COPYING.GPLv2 for details. None of
these parts are used by default, you have to explicitly pass --enable-gpl to
configure to activate them. In this case, FFmpeg's license changes to GPL v2+.
Specifically, the GPL parts of FFmpeg are
- libpostproc
- libmpcodecs
- optional x86 optimizations in the files
libavcodec/x86/idct_mmx.c
- libutvideo encoding/decoding wrappers in
libavcodec/libutvideo*.cpp
- the X11 grabber in libavdevice/x11grab.c
- the swresample test app in
libswresample/swresample-test.c
- the texi2pod.pl tool
- the following filters in libavfilter:
- f_ebur128.c
- vf_blackframe.c
- vf_boxblur.c
- vf_colormatrix.c
- vf_cropdetect.c
- vf_decimate.c
- vf_delogo.c
- vf_geq.c
- vf_histeq.c
- vf_hqdn3d.c
- vf_hue.c
- vf_kerndeint.c
- vf_mp.c
- vf_pp.c
- vf_smartblur.c
- vf_super2xsai.c
- vf_tinterlace.c
- vf_yadif.c
- vsrc_mptestsrc.c
There are a handful of files under other licensing terms, namely:
* The files libavcodec/jfdctfst.c, libavcodec/jfdctint_template.c and
libavcodec/jrevdct.c are taken from libjpeg, see the top of the files for
licensing details. Specifically note that you must credit the IJG in the
documentation accompanying your program if you only distribute executables.
You must also indicate any changes including additions and deletions to
those three files in the documentation.
Should you, for whatever reason, prefer to use version 3 of the (L)GPL, then
the configure parameter --enable-version3 will activate this licensing option
for you. Read the file COPYING.LGPLv3 or, if you have enabled GPL parts,
COPYING.GPLv3 to learn the exact legal terms that apply in this case.
external libraries
==================
FFmpeg can be combined with a number of external libraries, which sometimes
affect the licensing of binaries resulting from the combination.
compatible libraries
--------------------
The libcdio, libx264, libxavs and libxvid libraries are under GPL. When
combining them with FFmpeg, FFmpeg needs to be licensed as GPL as well by
passing --enable-gpl to configure.
The OpenCORE and VisualOn libraries are under the Apache License 2.0. That
license is incompatible with the LGPL v2.1 and the GPL v2, but not with
version 3 of those licenses. So to combine these libraries with FFmpeg, the
license version needs to be upgraded by passing --enable-version3 to configure.
incompatible libraries
----------------------
The Fraunhofer AAC library, FAAC and aacplus are under licenses which
are incompatible with the GPLv2 and v3. We do not know for certain if their
licenses are compatible with the LGPL.
If you wish to enable these libraries, pass --enable-nonfree to configure.
But note that if you enable any of these libraries the resulting binary will
be under a complex license mix that is more restrictive than the LGPL and that
may result in additional obligations. It is possible that these
restrictions cause the resulting binary to be unredistributeable.
......@@ -5,8 +5,9 @@ set -e
###################################################################################################
# #
# This script is in charge of: #
# - installing required packages like Java, Python, Apache, FFmpeg... #
# - installing required packages like Java, Python, Apache... #
# - configuring the new instance of Apache w.r.t. the need for Open AXES GUI #
# - compiling a local version of FFmpeg since a specific version is needed for shot-detection #
# - downloading MCR and classifier as well as installing them if activated when calling script #
# #
# Options: #
......@@ -18,6 +19,9 @@ set -e
# Google or Bing image search. This requires more space, install a lot #
# of development libraries since compilation has to be done. As a #
# result installation time will be widely increased. #
# --includeFFmpegGPL, can be used to compile FFmpeg with --enable-gpl --enable-libx264 flags. #
# This is needed in order to process H264 videos. Due to licence issue, #
# this is not activated by default. #
# --nonInteractiveMode, can be used to assume the answer is yes to any question and to extend #
# sudo session in order to ease automatic installation using tools like #
# Vagrant, Puppet or Ansible (but the Java PPA based installation). #
......@@ -31,7 +35,7 @@ set -e
################################# First check options #############################################
###################################################################################################
ffmpegFlags="--disable-logging --disable-doc --enable-shared --extra-cflags=-w --enable-libvorbis --enable-libvpx --enable-libopus --enable-static --enable-libass --enable-libmp3lame"
aptGetFlags=
pipFlags=
addAptRepositoryFlags=
......@@ -47,6 +51,9 @@ while [ $# -gt 0 ]; do
'--includeOnTheFlyClassifier')
includeOnTheFlyClassifier=true;
;;
'--includeFFmpegGPL')
ffmpegFlags="$ffmpegFlags --enable-gpl --enable-libx264"
;;
'--nonInteractiveMode')
aptGetFlags="$aptGetFlags -q -y"
extendSudo=true
......@@ -77,11 +84,6 @@ if ! grep -q "^deb http://ppa.launchpad.net/webupd8team/java/ubuntu" /etc/apt/so
sudo add-apt-repository $addAptRepositoryFlags ppa:webupd8team/java
fi
# Add PPA for FFmpeg if not already there.
if ! grep -q "^deb http://ppa.launchpad.net/jon-severinsson/ffmpeg/ubuntu" /etc/apt/sources.list /etc/apt/sources.list.d/*.list; then
echo "Installing a personal package archive for FFmpeg (see http://doc.ubuntu-fr.org/ffmpeg for more details)."
sudo add-apt-repository $addAptRepositoryFlags ppa:jon-severinsson/ffmpeg
fi
sudo apt-get $aptGetFlags update
......@@ -93,8 +95,10 @@ if [[ -n "$extendSudo" ]] ; then sudo -v ; fi
# Python and Apache for the UI
sudo apt-get $aptGetFlags install python-pip python-dev apache2 libapache2-mod-wsgi
if [[ -n "$extendSudo" ]] ; then sudo -v ; fi
# FFmpeg needed for UI Virtual cutter, video-normaliser and shot extractor
sudo apt-get $aptGetFlags install ffmpeg
# Various lib needed for FFMPEG, UI Virtual cutter, video-normaliser and shot extractor
sudo apt-get $aptGetFlags install autoconf automake build-essential libass-dev libbz2-dev libfreetype6-dev libgpac-dev libjpeg8 libmp3lame-dev
sudo apt-get $aptGetFlags install libopus-dev libsdl1.2-dev libtheora-dev libtool libva-dev libvdpau-dev libvorbis-dev libvpx-dev libx264-dev
sudo apt-get $aptGetFlags install libxcb1-dev libxcb-shm0-dev libxcb-xfixes0-dev pkg-config texi2html yasm zlib1g-dev
if [[ -n "$extendSudo" ]] ; then sudo -v ; fi
# Various lib needed pastec
sudo apt-get $aptGetFlags install libopencv-dev libmicrohttpd-dev libjsoncpp-dev curl libyaml-dev
......@@ -133,7 +137,8 @@ if [[ -n "$extendSudo" ]] ; then sudo -v ; fi
sed 's?BUNDLELOCATION?'`pwd`'?' conf/gui/axes-research.conf.tmpl > conf/gui/axes-research.conf
sudo ln -sf "$(pwd)/conf/gui/axes-research.conf" /etc/apache2/sites-enabled/axes-research.conf
sed 's?BUNDLEVERSION?'1.0.0-SNAPSHOT'?' conf/gui/defaults.py.tmpl > conf/gui/defaults.py.tmpl2
sed 's?FFMPEG_PATH?'$(pwd)/components/ffmpeg-1.1.16/ffmpeg.sh'?' conf/gui/defaults.py.tmpl > conf/gui/defaults.py.tmpl1
sed 's?BUNDLEVERSION?'1.0.1'?' conf/gui/defaults.py.tmpl1 > conf/gui/defaults.py.tmpl2
if [[ -n "$includeOnTheFlyClassifier" ]] ; then
sed 's?VISUAL_SEARCH_ACTIVATED?'\'category-t\''?' conf/gui/defaults.py.tmpl2 > conf/gui/defaults.py
elif [[ -n "$includeClassifier" ]] ; then
......@@ -178,12 +183,47 @@ echo "GUI and LIMAS configuration finished successfully"
###################################################################################################
###################################### Installing FFMPEG ##########################################
###################################################################################################
cd components
echo "Installing FFMPEG with flags $ffmpegFlags"
# Version 1.1.x is needed to be compliant with ShotDetector and KeyFrameExtractor.
# Normaliser could use recent one but once this one is installed and needed, the normaliser also use this one.
tar -zxf ffmpeg-1.1.16.tar.gz
cd ffmpeg-1.1.16
./configure $ffmpegFlags
make -j -s
# Wrapping ffmpeg binary in order to set its library path
LD_LIBRARY_PATH=""
for dir in lib*/; do
cd $dir;
LD_LIBRARY_PATH=$LD_LIBRARY_PATH`pwd`:
cd ..
done
if [[ -f ffmpeg.sh ]]; then
rm ffmpeg.sh
fi
echo "#!/bin/bash" >> ffmpeg.sh
echo "export LD_LIBRARY_PATH=\"$LD_LIBRARY_PATH\"" >> ffmpeg.sh
echo "./ffmpeg \"\$@\"" >> ffmpeg.sh
chmod +x ffmpeg.sh
cd ..
echo "FFMPEG successfully installed"
###################################################################################################
###################################### Installing Pastec ##########################################
###################################################################################################
cd components/pastec
cd pastec
echo "Installing Pastec"
tar zxf visualWordsORB.tar.gz
chmod +x pastec
......@@ -239,7 +279,7 @@ if [[ -n "$includeOnTheFlyClassifier" ]] ; then
mkdir -p {lib,include,build,server_data,model_data}
echo "Downloading and unzpiping imsearch-tool"
echo "Downloading and unzipping imsearch-tool"
wget --continue http://maven.ercim.eu/axes/eu/axes/visor/imsearch-tools/1.2.1/imsearch-tools-1.2.1-8c43d9b643b5d8372be8e2147229388505528031.zip
unzip $unzipFlags imsearch-tools-1.2.1-8c43d9b643b5d8372be8e2147229388505528031.zip
mv imsearch-tools-master imsearch-tools
......@@ -283,8 +323,8 @@ if [[ -n "$includeOnTheFlyClassifier" ]] ; then
unzip $unzipFlags caffe-dev-8f0c71cb86daaf753ea812be267fbb95b69aeabe.zip
cd caffe-dev
cp Makefile.config.example Makefile.config
sed -i -e "s#INCLUDE_DIRS := \$(PYTHON_INCLUDE) /usr/local/include#INCLUDE_DIRS := \$(PYTHON_INCLUDE) /usr/local/include $PREFIX/include#" Makefile.config
sed -i -e "s#LIBRARY_DIRS := \$(PYTHON_LIB) /usr/local/lib /usr/lib#LIBRARY_DIRS := \$(PYTHON_LIB) /usr/local/lib /usr/lib $PREFIX/lib#" Makefile.config
sed -i -e "s#INCLUDE_DIRS := \$(PYTHON_INCLUDE) /usr/local/include#INCLUDE_DIRS := \$(PYTHON_INCLUDE) /usr/local/include $PREFIX/include /usr/include/hdf5/serial/#" Makefile.config
sed -i -e "s#LIBRARY_DIRS := \$(PYTHON_LIB) /usr/local/lib /usr/lib#LIBRARY_DIRS := \$(PYTHON_LIB) /usr/local/lib /usr/lib $PREFIX/lib /usr/lib/x86_64-linux-gnu/hdf5/serial#" Makefile.config
sed -i -e "s/# CPU_ONLY := 1/CPU_ONLY := 1/" Makefile.config
sed -i -e "s/BLAS := atlas/BLAS := open/" Makefile.config
make all -j
......@@ -316,8 +356,8 @@ if [[ -n "$includeOnTheFlyClassifier" ]] ; then
cat > setup_env.sh <<END_OF_MESSAGE_HERE
export PATH=$PREFIX/bin:\$PATH
export CPLUS_INCLUDE_PATH=$PREFIX/include:\$CPLUS_INCLUDE_PATH
export LD_LIBRARY_PATH=$PREFIX/lib:$PREFIX/lib64:\$LD_LIBRARY_PATH
export CPLUS_INCLUDE_PATH=$PREFIX/include:/usr/include/hdf5/serial/:\$CPLUS_INCLUDE_PATH
export LD_LIBRARY_PATH=$PREFIX/lib:$PREFIX/lib64:/usr/lib/x86_64-linux-gnu/hdf5/serial:\$LD_LIBRARY_PATH
export LIBRARY_PATH=$PREFIX/lib:$PREFIX/lib64:\$LIBRARY_PATH
export BOOST_ROOT=$PREFIX
export CMAKE_PREFIX_PATH=$PREFIX
......
......@@ -3,7 +3,7 @@
<groupId>eu.axes.bundle.servers</groupId>
<artifactId>celery-manager</artifactId>
<version>1.0.0</version>
<version>1.0.1</version>
<packaging>jar</packaging>
......
......@@ -3,7 +3,7 @@
<groupId>eu.axes.bundle.servers</groupId>
<artifactId>imsearch-manager</artifactId>
<version>1.0.0</version>
<version>1.0.1</version>
<packaging>jar</packaging>
......
......@@ -3,7 +3,7 @@
<groupId>eu.axes.bundle.servers</groupId>
<artifactId>jetty-manager</artifactId>
<version>1.0.0</version>
<version>1.0.1</version>
<packaging>jar</packaging>
......
......@@ -3,7 +3,7 @@
<groupId>eu.axes.bundle.servers</groupId>
<artifactId>mongodb-manager</artifactId>
<version>1.0.0</version>
<version>1.0.1</version>
<packaging>jar</packaging>
......
......@@ -4,7 +4,7 @@
<groupId>eu.axes.bundle.osgi</groupId>
<artifactId>openaxes-camel</artifactId>
<version>1.0.0</version>
<version>1.0.1</version>
<packaging>bundle</packaging>
<name>OpenAXES Camel Chains</name>
......
......@@ -3,7 +3,7 @@
<groupId>eu.axes.bundle.servers</groupId>
<artifactId>pastec-manager</artifactId>
<version>1.0.0</version>
<version>1.0.1</version>
<packaging>jar</packaging>
......
......@@ -3,7 +3,7 @@
<groupId>eu.axes.bundle.servers</groupId>
<artifactId>visor-manager</artifactId>
<version>1.0.0</version>
<version>1.0.1</version>
<packaging>jar</packaging>
......
......@@ -3,7 +3,7 @@
<groupId>eu.axes.bundle</groupId>
<artifactId>open-axes</artifactId>
<version>1.0.0</version>
<version>1.0.1</version>
<packaging>pom</packaging>
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment