paella/doc
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Initial commit

Browse Source
This commit is contained in:
Thomas FORGIONE
2015-04-21 16:47:12 +02:00
parent 4458dddae1
commit 8b08f66bff
141 changed files with 2612 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

139
report/annexes/skeletons.tex Normal file
View File

@@ -0,0 +1,139 @@
\chapter{\label{AppendixA}Skeleton examples}
A 2D-skeleton : list of vertices and edges.
\begin{lstlisting}
v 200 300 1
v 300 400 1
v 300 500 1
v 500 600 1
v 600 500 1
v 800 400 1
v 900 300 1
v 500 800 1
v 600 900 1
v 600 1000 1
v 400 1000 1
v 200 1100 1
v 800 1100 1
v 900 1300 1
v 500 1200 1
v 500 1300 1
v 400 1400 1
e 1 2
e 2 3
e 3 4
e 4 5
e 5 6
e 6 7
e 4 8
e 8 9
e 9 10
e 10 11
e 11 12
e 10 13
e 13 14
e 10 15
e 15 16
e 16 17
\end{lstlisting}
\newpage
\noindent A 3D-skeleton made with splines
\begin{lstlisting}
d 3
n 0 0 0 0 0.333333 0.666667 1 1 1 1
p 0.60 0.43 0 0.12
p 0.60 0.89 0 0.115
p 0.72 1.28 0 0.11
p 0.96 1.72 0 0.118
p 1.27 2.10 0 0.122
p 1.69 2.29 0 0.12
d 3
n 0 0 0 0 0.25 0.5 0.75 1 1 1 1
p 2.80 0.33 0 0.12
p 2.78 0.67 0 0.116
p 2.68 1.01 0 0.111
p 2.52 1.49 0 0.117
p 2.33 1.83 0 0.119
p 2.05 2.11 0 0.121
p 1.69 2.29 0 0.12
d 3
n 0 0 0 0 0.5 1 1 1 1
p 1.69 2.29 0 0.12
p 1.70 2.44 0 0.125
p 1.70 2.56 0 0.129
p 1.71 2.72 0 0.127
p 1.73 2.89 0 0.125
d 3
n 0 0 0 0 0.125 0.25 0.375 0.5 0.625 0.75 0.875 1 1 1 1
p 1.73 2.89 0 0.125
p 2.06 2.78 0 0.128
p 2.52 2.70 0 0.13
p 3.18 2.69 0 0.132
p 3.44 2.78 0 0.129
p 3.62 2.93 0 0.126
p 3.67 3.16 0 0.111
p 3.63 3.32 0 0.098
p 3.44 3.46 0 0.071
p 3.14 3.47 0 0.036
p 3.01 3.38 0 0.011
d 3
n 0 0 0 0 0.5 1 1 1 1
p 1.73 2.89 0 0.125
p 1.73 3.18 0 0.131
p 1.73 3.53 0 0.136
p 1.72 3.84 0 0.141
p 1.72 4.28 0 0.138
d 3
n 0 0 0 0 0.333333 0.666667 1 1 1 1
p 1.72 4.28 0 0.138
p 1.43 4.25 0 0.112
p 1.19 4.16 0 0.101
p 0.92 3.98 0 0.094
p 0.71 3.81 0 0.088
p 0.56 3.56 0 0.075
d 3
n 0 0 0 0 0.333333 0.666667 1 1 1 1
p 1.72 4.28 0 0.138
p 1.95 4.25 0 0.115
p 2.19 4.20 0 0.103
p 2.43 4.10 0 0.096
p 2.62 3.97 0 0.090
p 2.79 3.67 0 0.077
d 3
n 0 0 0 0 1 1 1 1
p 1.72 4.28 0 0.138
p 1.71 4.53 0 0.155
p 1.67 4.89 0 0.158
p 1.68 5.12 0 0.135
l 3 1 2 3 1.0 1.0 0.0
l 3 3 4 5 1.0 0.0 0.0
l 4 5 6 7 8 1.0 0.0 0.0 0.0
\end{lstlisting}

88
report/chapters/actions.tex Normal file
View File

@@ -0,0 +1,88 @@
\chapter{Actions}
In order to avoid the realization of the identified risks, we applied some of our preventive actions.
\section{Initial actions table}
\begin{table}[h!]
\begin{center}
\begin{tabular}{|p{0.5cm}|p{3cm}|p{3cm}|p{2cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}|p{1cm}|}
\hline
Id & Origin & Description & Accountable & Target date & Realisation & State \\
\hline
1 & Lack of knowledge from the developer & C++ training & Thomas & 22 Jan & 23 Jan & Closed \\
\hline
2 & Team work \& version management & Git repository creation & Thomas & 05 Feb & 05 Feb & Closed \\
\hline
3 & Input needed & Taking pictures & Thierry & 22 Jan & 22 Jan & Closed \\
\hline
4 & Report to write in english & Specifications translation & Emilie & 22 Jan & 22 Jan & Closed \\
\hline
5 & Set up the dev tools & Install Linux / OpenCV & Thomas & 01 Feb & 02 Feb & Closed \\
\hline
6 & Remember stuff to do & Actions table writing & Marion & 05 Feb & 04 Feb & Closed \\
\hline
7 & Git Repository not well organized & Git cleaning & Thomas & 05 Feb & & Open \\
\hline
8 & Compatibility with the other team & Defining subskeletons format & Thierry & 06 Feb & & Open \\
\hline
9 & We love sushi & Sushi eating with the client & Thierry & 13 Feb & & Open \\
\hline
\end{tabular}
\end{center}
\caption{Initial actions table}
\end{table}
\section{Final actions table}
\begin{table}[h!]
\begin{tabular}{|p{0.5cm}|p{3cm}|p{3cm}|p{2cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}|p{1cm}|}
\hline
Id & Origin & Description & Accountable & Target date & Realisation & State & Risk table\\
\hline
1 & Project management & Meeting with the industrial supervisor & Thomas & 15 Jan & 15 Jan & Closed & \\
\hline
2 & Need of specifications & Meeting with the client & Thierry & 22 Jan & 22 Jan & Closed & \\
\hline
3 & Lack of knowledge from the developer & C++ training & Thomas & 22 Jan & 23 Jan & Closed & \\
\hline
4 & Team work \& version management & Git repository creation & Thomas & 05 Feb & 05 Feb & Closed & \\
\hline
5 & Project management & Meeting with the industrial supervisor & Thierry & 21 Jan & 21 Jan & Closed & \\
\hline
6 & Input needed & Taking pictures & Thierry & 22 Jan & 22 Jan & Closed & \\
\hline
7 & Report to write in english & Specifications translation & Emilie & 22 Jan & 22 Jan & Closed & \\
\hline
8 & Set up the dev tools & Install Linux / OpenCV & Thomas & 01 Feb & 02 Feb & Closed & \\
\hline
9 & Need of specifications & Meeting with the client & Thierry & 04 Feb & 04 Feb & Closed & \\
\hline
10 & Remember stuff to do & Actions table writing & Marion & 05 Feb & 04 Feb & Closed & \\
\hline
11 & Project management & Meeting with the industrial supervisor & Marion & 5 Feb & 5 Feb & Closed & \\
\hline
12 & Git Repository not well organized & Git cleaning & Thomas & 05 Feb & 05 Feb & Closed & \\
\hline
13 & Compatibility with the other team & Defining subskeletons format & Thierry & 06 Feb & 09 Feb & CLosed & \\
\hline
14 & Need of specifications & Meeting with the client & Thierry & 11 Feb & 11 Feb & Closed & \\
\hline
15 & The other group did'nt deliver binaries in time & Reschedule planning & Marion & 11 Feb & 11 Feb & Closed & 2 \\
\hline
16 & Binaries provided by he client are not compatible & Speak with the client & Thomas & 11 Feb & & Open & 4 \\
\hline
17 & Client new need & Contact the client by mail to know what we need to do & Thierry & 12 Feb & 12 Feb & Closed & 11 \\
\hline
18 & Project management & Meeting with the industrial supervisor & Amandine & 12 Feb & 12 Feb & Closed & \\
\hline
19 & Integration to do again & discuss with the client & Thierry & 25 Feb & 25 Feb & Closed &\\
\hline
20 & Project management & Meeting with the industrial supervisor & Emilie & 24 Feb & 24 Feb & Closed & \\
\hline
21 & Need of specifications & Meeting with the client & Thierry & 25 Feb & 25 Feb & Closed & \\
\hline
22 & The client need CMake to compile the code & Use CMake compilation & Thomas & 27 Feb & 27 Feb & Closed & \\
\hline
\end{tabular}
\caption{Actions table on February 12th}
\end{table}

9
report/chapters/conclusion.tex Normal file
View File

@@ -0,0 +1,9 @@
\chapter*{Conclusion}
This project was a useful experience for us. It allowed us to better understand the usefulness and efficiency of project management. Having a schedule, and anticipating risks with our risks table allowed us to enter this project more serenely, and to finish it in time.
The main difficulty we had to face happened during the first part of our project. The first part was shared between us and an other "Projet long" team. Some parts were also provided by the client. Unfortunately problems happened with some of his binaries, and the integration had to be abandoned. However the client was still satisfied with the results on this first part, since as a researcher it showed him problems to work on.
The technical part of the project was very interesting, we learned a lot about C++ programming. We obtained good results on the second part of the project, which produces interesting animations.
To conclude, this project was an opportunity for us to apply management on a real project and an interesting technical experience.

47
report/chapters/frontPage.tex Normal file
View File

@@ -0,0 +1,47 @@
\begin{titlepage}
\begin{sffamily}
\begin{center}
~\\[2cm]
\textsc{\LARGE Projet long report}\\[3cm]
% Title
\HRule \\[0.4cm]
{ \huge \bfseries Segmentation, Skeleton and Animation\\[0.4cm] }
\HRule \\[3cm]
\includegraphics[scale=0.5]{img/n7.png}~\\[2cm]
\end{center}
% Author and supervisor
\begin{minipage}{0.5\textwidth}
\begin{flushleft} \large
Thomas \textsc{Forgione}\\
Emilie \textsc{Jalras}\\
Marion \textsc{Lenfant}\\
Thierry \textsc{Malon}\\
Amandine \textsc{Pailloux}\\
\end{flushleft}
\end{minipage}
\begin{minipage}{0.5\textwidth}
\begin{flushright} \large
\emph{Client :} \\
M. Bastien \textsc{Durix}\\
Ms G\'{e}raldine \textsc{Morin}\\
Ms Sylvie \textsc{Chambon}\\
\emph{Industrial supervisor :}\\
Ms R\'{e}gine \textsc{Nigris}
\end{flushright}
\end{minipage}
\vfill
% Bottom of the page
{\large January 19th 2015 -- March 13th 2015}
\end{sffamily}
\end{titlepage}
\tableofcontents
\newpage

10
report/chapters/introManagement.tex Normal file
View File

@@ -0,0 +1,10 @@
\chapter*{Management introduction}
The second important aspect of our project was its management.
Indeed project management will soon be part of our everyday life in our future work. This is why becoming familiar with the management tools studied in class is important and the "Projet Long" was a great opportunity to put it into practice.
We spent our first project week on this part. We tried to define clearly the client need, in order to achieve good product specification. Unfortunately it had to evolve during the first part of our project. We took decisions about the way we would work, for example we decided to use pair programming, and to spend time on our tests before developing. We also worked especially on our risks table to anticipate problems, on our actions table to prevent risks realization, and on our schedule to make sure we would finish on time. \\
We have been updating this documents during the project in order to always anticipate problems.

21
report/chapters/introduction.tex Normal file
View File

@@ -0,0 +1,21 @@
\chapter*{Introduction}
As a conclusion to our studies at the ENSEEIHT, we have been working on our Projet Long. Our project takes part in Bastien Durix thesis which focuses on the extraction of 2D skeletons from pictures and the computation of 3D skeletons from associated 2D skeletons of the same objects, and will also help him on other parts of his approach.
Our subject consisted in realizing animated 3D versions of real 3D objects. We based ourselves on different pictures of objects taken from different perspectives, and use them to reconstruct a 3D version of the object, which we animate.\\
Here is our pipeline :
\begin{itemize}
\item segmentation
\item intrinsic and extrinsic camera calibration
\item points detection and matching
\item skeletonization : cut and match skeleton pieces
\item computing the B-splines
\item meshing the 3D skeleton computed with the 2D skeletons
\item meshing of extremities and junctions
\item choosing connecting points to animate the object
\item animation
\end{itemize}
We were two Projet Long groups working on these subject. The first part was split into two subsections : the other group got to work on the segmentation and B-splines computation, while we were given the matching part and the camera calibration. The second part, concerning the meshing and the animation, was done completely by the two groups.

36
report/chapters/license.tex Normal file
View File

@@ -0,0 +1,36 @@
\chapter{License}
We decided to add a license to our project in order to allow the client using
our project without any problems of properties. We chose the Zlib license
which allow a free-use of our project. You can see our lisence in the listing \ref{license}.\\
\begin{lstlisting}[caption=Zlib license, label=license]
Paella - Copyright (C) 2015 - Thomas FORGIONE, Emilie JALRAS, Marion LENFANT, Thierry MALON, Amandine PAILLOUX
This software is provided 'as-is', without any express or implied warranty.
In no event will the authors be held liable for any damages arising from the use of this software.
Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it freely,
subject to the following restrictions:
1. The origin of this software must not be misrepresented;
you must not claim that you wrote the original software.
If you use this software in a product, an acknowledgment
in the product documentation would be appreciated but is not required.
2. Altered source versions must be plainly marked as such,
and must not be misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.
External-libraries
------------------
Boost is under the boost software license
Eigen is under the MPL2
OpenCV is under the BSD License
SFML is under the zlib/libpng license
\end{lstlisting}

22
report/chapters/productDescription.tex Normal file
View File

@@ -0,0 +1,22 @@
\chapter{Product description}
Here is the detailed description of the different steps of our project.
We spent time on finding tests before developing, which helped us to avoid some problems and find some errors earlier than we would have otherwise. You can find more information concerning our tests in the attached tests file.
\section{Segmentation, Camera calibration, skeletonization and branches matching}
\input{subsections/skeleton}
\input{subsections/segmentation}
\input{subsections/calibration}
\input{subsections/pointsDetectionAndMatching}
\input{subsections/branchesMatching}
\section{Meshing and animation}
\input{subsections/splines}
\input{subsections/circles}
\input{subsections/extremities}
\input{subsections/junctions}
\input{subsections/animation}

41
report/chapters/productSpecification.tex Normal file
View File

@@ -0,0 +1,41 @@
\chapter{Product specification}
During the first week of our project we tried to clearly define the need of the client and the specifications. In an industrial context it is really fundamental to describe the client expectations as well as possible. It protects us from the client if he suddenly wants something not intended at the beginning or if he is disappointed because he forgot to specify certain problems. But it also gives us some duties toward the client concerning the deliveries for instance. This is why it is really important to take it seriously.
Here is our product specification.
\section{General constraints}
\subsection{Expected deliverables and delivery delay}
We will deliver an archive containing:
\begin{itemize}
\item detailed documentation (commented code and a user manual)
\item source code : camera calibration and points matching, delivered on February 6th,
and then casing and animation on March 13th.
\end{itemize}
\subsection{Client supplies}
The client engages to supply the following elements :
\begin{itemize}
\item programs performing the skeleton extraction
\item mathematical formulas for skeleton casing
\item scientific articles concerning animation
\end{itemize}
\subsection{Imposed technical constraints}
Our technical constraints are as follow :
\begin{itemize}
\item C++ as programming language
\item it is suggested to use the openCV library
\item CMake is suggested as compilation utility
\item used libraries must be portable between Windows and Linux
\end{itemize}
\newpage
\section{Detailed Pipeline}
\begin{figure}[H]
\centering
\includegraphics[scale=0.56]{img/pipeline.png}
\caption{Detailled pipeline}
\end{figure}

69
report/chapters/risksManagement.tex Normal file
View File

@@ -0,0 +1,69 @@
\chapter{Risks management}
In order to anticipate problems and to set a right schedule, we tried to find as many risks as possible. We made our risks table evolve while the project progressed.
\section{Initial risks table}
\bottomcaption{Risks table on 25th February}
\begin{supertabular}{|p{0.5cm}|p{2cm}|p{2cm}|p{1cm}|p{2cm}|p{1.5cm}|p{2cm}|p{2cm}|p{2cm}|p{1.5cm}|}
\hline
Id & Description & risk cause & Proba (1-5) & Consequence & Seriousness (1-5) & Preventive actions & Corrective actions & Risk state \\
\hline
\cellcolor{green} 1 & Bad time estimation & Under estimation of the problem difficulty & 3 & Late delivery & 4 & Time margin of the planning, name a task advancement responsible & Raise the work amount, revise the planning & Open \\
\hline
\cellcolor{green} 2 & The other group does not deliver the deliverables on time & Under estimation of their problem difficulty, last time augmentation of the work amount & 2 & Delay for tests on the 3D part of the skeleton & 3 & Contemplate an other method to test on simple case & Implement ourselves a segmentation method based on simple threshold & Open \\
\hline
\cellcolor{green} 3 & The other group delivers wrong outputs & Bad comprehension of the subject & 3 & Tests do not work correctly & 1 & Find en agreement on the format & Develop a transition function & Open \\
\hline
\cellcolor{green} 4 & Binaries provided by the client are not compatible & Systems complexity & 4 & We can not carry on with the pipeline & 4 & Discuss with the client the provided binary & Emulate the system allowing the use of the binaries & Open \\
\hline
\cellcolor{green} 5 & Fragile harmony inside the group & Different uses for project management and programmation & 1 & Tensions, bad ambiance, intense stress & 5 & Name a cohesion responsible & Organize a mediation beetween the two parties & Open \\
\hline
\cellcolor{green} 6 & Bad understanding with the other group & A bancal division of the project might bring tensions & 2 & Difficulty to manage shared parts & 3 & Bring a present as a sign of sympathy & Come to a compromise & Open \\
\hline
\cellcolor{green} 7 & A group member is absent during a certain period of time & Illness, event & 5 & Someone's absence & 1 & Put in place teleworking solutions & Planning adaptation & Open \\
\hline
\cellcolor{green} 8 & Production of unreliable, unmaintainable and illegible code & Lack of skills with the used technology & 4 & Lack of time, bad quality, need to reprogram & 3 & C++ formation by a technical expert & Call of the expert & Open \\
\hline
\cellcolor{green} 9 & Need to access urgently to a deleted resource & Utilization of a suppression command, material crash & 2 & Lack of time, need to reprogram & 5 & Use github & Use of recuperation technology on long and complicated files & Open \\
\hline
\cellcolor{green} 10 & Pictures unadapted & Client not satisfied of the pictures & 4 & Wasting time & 2 & Defined the conditions of the shoot with the client more precisely & Be aware of client expectations and taking again the picture & Open \\
\hline
\end{supertabular}
\section{Final risks table}
In the table below you can find the risks state in the end of the project.\\
\bottomcaption{Risks table on 25th February}
\begin{supertabular}{|p{0.5cm}|p{2cm}|p{2cm}|p{1cm}|p{2cm}|p{1.5cm}|p{2cm}|p{2cm}|p{2cm}|p{1.5cm}|}
\hline
Id & Description & risk cause & Proba (1-5) & Consequence & Seriousness (1-5) & Preventive actions & Corrective actions & Risk state \\
\hline
\cellcolor{green} 1 & Bad time estimation & Under estimation of the problem difficulty & 3 & Late delivery & 4 & Time margin of the planning, name a task advancement responsible & Raise the work amount, revise the planning & Open \\
\hline
\cellcolor{yellow} 2 & The other group does not deliver the deliverables on time & Under estimation of their problem difficulty, last time augmentation of the work amount & 2 & Delay for tests on the 3D part of the skeleton & 3 & Contemplate an other method to test on simple case & Implement ourselves a segmentation method based on simple threshold & Open \\
\hline
\cellcolor{green} 3 & The other group delivers wrong outputs & Bad comprehension of the subject & 3 & Tests do not work correctly & 1 & Find en agreement on the format & Develop a transition function & Open \\
\hline
\cellcolor{yellow} 4 & Binaries provided by the client are not compatible & Systems complexity & 4 & We can not carry on with the pipeline & 4 & Discuss with the client the provided binary & Emulate the system allowing the use of the binaries & Open \\
\hline
\cellcolor{green} 5 & Fragile harmony inside the group & Different uses for project management and programmation & 1 & Tensions, bad ambiance, intense stress & 5 & Name a cohesion responsible & Organize a mediation beetween the two parties & Open \\
\hline
\cellcolor{green} 6 & Bad understanding with the other group & A bancal division of the project might bring tensions & 2 & Difficulty to manage shared parts & 3 & Bring a present as a sign of sympathy & Come to a compromise & Open \\
\hline
\cellcolor{green} 7 & A group member is absent during a certain period of time & Illness, event & 5 & Someone's absence & 1 & Put in place teleworking solutions & Planning adaptation & Open \\
\hline
\cellcolor{green} 8 & Production of unreliable, unmaintainable and illegible code & Lack of skills with the used technology & 4 & Lack of time, bad quality, need to reprogram & 3 & C++ formation by a technical expert & Call of the expert & Closed \\
\hline
\cellcolor{green} 9 & Need to access urgently to a deleted resource & Utilization of a suppression command, material crash & 2 & Lack of time, need to reprogram & 5 & Use github & Use of recuperation technology on long and complicated files & Open \\
\hline
\cellcolor{yellow} 10 & Pictures unadapted & Client not satisfied of the pictures & 4 & Wasting time & 2 & Defined the conditions of the shoot with the client more precisely & Be aware of client expectations and taking again the picture & Open \\
\hline
\cellcolor{green} 11 & New task to do & Client new needs & 4 & Late delivery & 4 & Add margin in planning & change planning and contact the client to negociate about this task & Closed \\
\hline
\cellcolor{green} 12 & Integration not cancelled & Client changes his mind and wants again the integration & 2 & Late delivery & 4 & discuss with the client & change planning & Open \\
\hline
\cellcolor{green} 13 & members of the group get remedial classes & bad results at the exams & 2 & member absent & 2 & add margin & change planning & Open \\
\hline
\end{supertabular}

29
report/chapters/schedule.tex Normal file
View File

@@ -0,0 +1,29 @@
\chapter{Schedule}
Thanks to our risks table we had anticipated the possibility of a schedule modification due to our interaction with the other group and with the client binaries. We had kept one week as security, and finally had to use it before the integration period.
\section{Initial schedule}
We spent our first week on the project management, like doing this schedule. \\
At the beginning we did a detailed schedule for the first part of the project, but not for the second part of the project. We just assigned 2 weeks and half to the meshing, and one week to the animation.
\begin{figure}[H]
\begin{center}
\includegraphics[scale=0.5]{img/initialSchedule}
\caption{\label{initialSchedule} Initial Schedule}
\end{center}
\end{figure}
\section{Final schedule}
Before starting the second part of the project we did the detailed schedule of this part, which you can see below.
The integration with the other group was abandoned, as you can see.
We also decided to finally start the report before finishing the second part.
\begin{figure}[H]
\begin{center}
\includegraphics[scale=0.5]{img/finalSchedule}
\caption{\label{finalSchedule} Final Schedule}
\end{center}
\end{figure}

6
report/chapters/technicalConclusion.tex Normal file
View File

@@ -0,0 +1,6 @@
\chapter*{Conclusion of the technical part}
The technical part of this project taught us a lot, especially in C++ programming.
Our knowledge from the multimedia specialization was very helpful, as well in the image analysis part as in the meshes and animation part.
During this project we also tried to think of tests before developing, which helped us in finding errors early, and anticipate problems.

4
report/chapters/tests.tex Normal file
View File

@@ -0,0 +1,4 @@
\chapter{Tests}
You can find our tests in the attached folder.
We reported the results obtained with different parameters or methods, and explained our choices. We also tackle the limits of our different programs.

BIN
report/img/0.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 8.3 KiB

BIN
report/img/0ed.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 43 KiB

BIN
report/img/1.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 8.9 KiB

BIN
report/img/10.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 8.9 KiB

BIN
report/img/10ed.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 36 KiB

BIN
report/img/1ed.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 44 KiB

BIN
report/img/2.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 9.0 KiB

BIN
report/img/2ed.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 51 KiB

BIN
report/img/JunctionTheory.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 76 KiB

BIN
report/img/Junctions1.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 20 KiB

BIN
report/img/Junctions10.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 23 KiB

BIN
report/img/Junctions11.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 56 KiB

BIN
report/img/Junctions2.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 24 KiB

BIN
report/img/Junctions3.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 28 KiB

BIN
report/img/Junctions4.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 32 KiB

BIN
report/img/Junctions5.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 23 KiB

BIN
report/img/Junctions9.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 6.0 KiB

BIN
report/img/LapinSiftSurf.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 740 KiB

BIN
report/img/LapinSymetricGeometric.PNG Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 987 KiB

BIN
report/img/Poupee.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 358 KiB

BIN
report/img/Poupeebin.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.4 KiB

BIN
report/img/Poupeeskl.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 5.7 KiB

BIN
report/img/SkeletonEx.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 866 KiB

BIN
report/img/Subdivision.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 115 KiB

BIN
report/img/brisk3415.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 534 KiB

BIN
report/img/characteristicCircle.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 5.7 KiB

BIN
report/img/finalSchedule.PNG Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 190 KiB

BIN
report/img/initialSchedule.PNG Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 307 KiB

BIN
report/img/meshExtremity.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 135 KiB

BIN
report/img/meshextremity.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 86 KiB

BIN
report/img/n7.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 11 KiB

BIN
report/img/pipeline.png Executable file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 193 KiB

BIN
report/img/projection.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 114 KiB

BIN
report/img/regularMesh.png Executable file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 18 KiB

BIN
report/img/spline.png Executable file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 7.0 KiB

34
report/references.bib Normal file
View File

@@ -0,0 +1,34 @@
@article{brisk,
author = "Stefan, Leutenegger and Margarita, Chli and Roland, Y.Siegwart",
title = "{BRISK} : Binary Robust Invariant Scalable Keypoints",
year = "2011",
}
@online{surf,
author = "OpenCV 3.0.0-dev documentation",
title = "Introduction to {SURF} (Speeded-Up Robust Features)",
url = "http://docs.opencv.org/trunk/doc/py_tutorials/py_feature2d/py_surf_intro/py_surf_intro.html",
urldate = "2015-03-12"
}
@online{sift,
author = "OpenCV 3.0.0-dev documentation",
title = "Introduction to {SIFT} (Scale-Invariant Feature Transform)",
url = "http://docs.opencv.org/trunk/doc/py_tutorials/py_feature2d/py_sift_intro/py_sift_intro.html",
urldate = "2015-03-12"
}
@online{license,
author = "Open Source Initiative",
title = "The zlib/libpng License (Zlib)",
url = "http://opensource.org/licenses/Zlib",
urldate = "2015-03-12"
}
@online{spline,
author="Jana Proch\'azkov\'a",
title="DERIVATIVE OF B-SPLINE FUNCTION",
url="http://mat.fsv.cvut.cz/gcg/sbornik/prochazkova.pdf",
urldate="2015-03-12"
}

127
report/report.tex Normal file
View File

@@ -0,0 +1,127 @@
\documentclass{report}
\usepackage[english]{babel}
\usepackage{amssymb}
\usepackage{lmodern}
\usepackage[T1]{fontenc}
\usepackage{verbatim}
\usepackage{ucs}
\usepackage{graphicx}
\usepackage{caption}
\usepackage{amsmath}
\usepackage{multicol}
\usepackage{hyperref}
\usepackage{tikz}
\usepackage{pgfplots}
\usepackage{subcaption}
\usepackage{supertabular}
\usepackage{colortbl}
\usepackage{lastpage}
\usepackage{listings}
\usepackage{xcolor}
\usepackage{graphicx}
\definecolor{colKeys}{rgb}{0,0.5,0}
\definecolor{colIdentifier}{rgb}{0,0,0}
\definecolor{colComments}{rgb}{0,0.5,1}
\definecolor{colString}{rgb}{0.6,0.1,0.1}
\definecolor{colBackground}{rgb}{0.95,0.95,1}
\lstset{%configuration de listings
float=hbp,%
basicstyle=\ttfamily\small, %
%
identifierstyle=\color{colIdentifier}, %
keywordstyle=\color{colKeys}, %
stringstyle=\color{colString}, %
commentstyle=\color{colComments}\textit, %
%
backgroundcolor=\color{colBackground},%
%
columns=flexible, %
tabsize=2, %
frame=trbl, %
%frameround=tttt,%
extendedchars=true, %
showspaces=false, %
showstringspaces=false, %
numbers=left, %
numberstyle=\tiny, %
breaklines=true, %
breakautoindent=true, %
captionpos=b,%
xrightmargin=0cm, %
xleftmargin=0cm%
}
\setlength{\hoffset}{-18pt}
\setlength{\oddsidemargin}{0pt} % Marge gauche sur pages impaires
\setlength{\evensidemargin}{9pt} % Marge gauche sur pages paires
\setlength{\marginparwidth}{54pt} % Largeur de note dans la marge
\setlength{\textwidth}{17cm} % Largeur de la zone de texte (17cm)
\setlength{\voffset}{-18pt} % Bon pour DOS
\setlength{\marginparsep}{7pt} % Séparation de la marge
\setlength{\topmargin}{-30pt} % Pas de marge en haut
\setlength{\headheight}{13pt} % Haut de page
\setlength{\headsep}{10pt} % Entre le haut de page et le texte
\setlength{\footskip}{0.8cm} % Bas de page + séparation
\setlength{\textheight}{24.5cm} % Hauteur de la zone de texte (25cm)
\usepackage{titling}
\usepackage{fancyhdr}
\pagestyle{fancy}
\renewcommand{\headrulewidth}{1pt}
\renewcommand{\footrulewidth}{\headrulewidth}
\usepackage{float}
\newcommand{\hsp}{\hspace{20pt}}
\newcommand{\HRule}{\rule{\linewidth}{0.5mm}}
\fancyfoot[RO]{\thepage/\pageref{LastPage}}
\fancyfoot[C]{January 2015 --- March 2015}
\fancyfoot[L]{\emph{Projet long report}}
% Redefine the plain page style (display the footer on chapter page)
\fancypagestyle{plain}{%
\fancyhf{}%
\fancyfoot[RO]{\thepage/\pageref{LastPage}}
\fancyfoot[C]{January 2015 --- March 2015}
\fancyfoot[L]{\emph{Projet long report}}
\renewcommand{\headrulewidth}{0pt}% Line at the header invisible
\renewcommand{\footrulewidth}{0.4pt}% Line at the footer visible
}
\makeatletter
\@addtoreset{chapter}{part}
\makeatother
\begin{document}
\input{chapters/frontPage}
\input{chapters/introduction}
\part{Technical Field}
\input{chapters/productSpecification}
\input{chapters/productDescription}
%\input{chapters/tests}
\input{chapters/technicalConclusion}
\part{Project Management}
\input{chapters/introManagement}
\input{chapters/risksManagement}
\input{chapters/actions}
\input{chapters/schedule}
\input{chapters/license}
\input{chapters/conclusion}
\bibliographystyle{plain}
\bibliography{references}
\appendix
\input{annexes/skeletons}
\end{document}

93
report/subsections/animation.tex Normal file
View File

@@ -0,0 +1,93 @@
\subsection{Animation}
The objective of this part is to animate the mesh we previously generated
around the skeleton. We have as an input segments of the skeleton, and its
mesh. The idea is to deform the segments of the skeleton, and to
automatically have the impact of this deformation on the vertices of the
mesh.
\subsubsection{Trees}
When we will animate our mesh, we want to keep it as one piece. For
example, when you move your arm, your elbow and your forearm follows
the movement of your arm. This means that the transformation that we
will apply after the shoulders must be kept on the arms and forearms.
To manage to do this, we structured everything in trees. The tree for a
person will be the following for example (to simplify, we will take the
hypothesis that this human has no knees).
\begin{figure}[H]
\centering
\begin{tikzpicture}
% Head
\draw (0,0) circle [radius=1];
\draw (0,0) node{Head};
% Left son of head
\draw (0,-1) -- (-3,-2);
\draw (-3,-3) circle [radius=1];
\draw (-3,-3) node{Left arm};
% Left son of left arm
\draw (-3,-4) -- (-6,-5);
\draw (-6,-6) circle [radius=1];
\draw (-6,-6) node{Left forearm};
% Middle son
\draw ( 0,-1) -- ( 0,-2);
\draw ( 0,-3) circle [radius=1];
\draw ( 0,-3) node{Body};
% Left leg
\draw (0, -4) -- (-2, -5);
\draw ( -2,-6) circle [radius=1];
\draw ( -2,-6) node{Left leg};
% Right leg
\draw (0, -4) -- (2, -5);
\draw ( 2,-6) circle [radius=1];
\draw ( 2,-6) node{Right leg};
% Right son
\draw (0,-1) -- (3,-2);
\draw (3,-3) circle [radius=1];
\draw (3,-3) node{Right arm};
% Left son of left arm
\draw (3,-4) -- (6,-5);
\draw (6,-6) circle [radius=1];
\draw (6,-6) node{Right forearm};
\end{tikzpicture}
\caption{Tree for a human being skeleton}
\end{figure}
\paragraph{}
At each node of this, there will be a rotation (with a center, and
angles), and we will draw the animated mesh by traversing the tree.
At each node, we will apply the transformation of the current node, and
then draw the subtree.
Of course, we also need to know what faces are in each nodes, so we
will have a tree of rotations, and a tree of faces.
\subsubsection{Junctions processing}
All we said before is valid only if the three vertices of a face are
mapped to the same segment. But some faces have vertices that are
mapped to different segments. For these faces, we need to apply a
different transformation for each vertex.
For this purpose, we created a hashtable between vertex numbers and
paths in trees so that we are able to find the transformation for these
vertices.
\subsubsection{Graphical User Interface}
In order to be able to manipulate the animated mesh easily, two graphical interfaces were made :
\begin{enumerate}
\item the first one allows the user to add rotation points to the
skeleton (by default, only the junction points are considered
as rotation points, and there are no knees and elbows for
example).
\item the second one has the OpenGL rendering and a menu controlled
with the keyboard, allowing the user to select rotation points
and to change the value of the angles.
\end{enumerate}

39
report/subsections/branchesMatching.tex Normal file
View File

@@ -0,0 +1,39 @@
\subsection{Branches Extraction and Matching}
The branches extraction part was allocated to both groups. The skeleton we have extracted are composed of points. A skeleton point can be of different types:
\begin{enumerate}
\item a extremity point which only have one neighbor, i.e. this point is involved in only one edge of the skeleton.
\item a regular point which have two neighbors, i.e. this point is involved in only two edges of the skeleton.
\item a junction point which have at least three neighbors, i.e. this point is involved in three edges or more than three edges.
\end{enumerate}
In the cases of extremity points and junction points, we consider that the point is irregular. To extract the branches of the skeleton, we start from each irregular point and we follow the edges starting from this irregular point until we find another irregular point (junction or extremity).
Let us consider two irregular points $P_1$ and $P_5$ such as there is a path $P_1$, $P_2$, $P_3$, $P_4$, $P_5$ linking these two irregular points together. As a result, $P_1$, $P_2$, $P_3$, $P_4$, $P_5$ is a branch of the skeleton. However, the algorithm will also find $P_5$, $P_4$, $P_3$, $P_2$, $P_1$ as a branch of the skeleton. To avoid having each branch two times, we added a new branch to the branches set only if the contrary branch is not already in the set.
Once we extract all the branches on two correspoding skeletons, we would like to match these branches together thanks to the keypoints found on each picture. For this purpose, we would like to assign to each keypoint a branch of the skeleton. We will consider that both skeletons have the same number of branches. If not, the matching between branches would not be perfectly possible.
To determine the correspoding branch of a keypoint, we use the euclidian distance. Each keypoint is assigned to the nearest branch of the skeleton. As a branch is composed of segments, we will define the distance between a point and a branch as the shortest distance between this point and all the segments composing the branch. Thus, the nearest branch of a point is the branch that contains the nearest segment to the point.
The distance between a point and a segment is defined as following:
\begin{enumerate}
\item if the projection of the point is on the line that contains the segment, then the distance between the point and the segment is the distance between the point and its projection.
\item if the projection of the point is not on the line that contains the segment, then the distance between the point and the segment is the shortest distance between the distance between the point and each extremity of the segment.
\end{enumerate}
Once each keypoint has been associated to a branch, we use the matching between keypoints from the two pictures and, for each picture, the matching between keypoints and branches to deduce the matchings between the branches from the two skeletons.
$$\begin{matrix}
\text{Keypoints 1} & \longleftrightarrow & \text{Keypoints 2} \\
\updownarrow & & \updownarrow \\
\text{Branches 1} & & \text{Branches 2}
\end{matrix}$$
To compute the matches between branches, we define a matching a square matrix $M$ which size is the number of branches of each skeleton.
Let $KP_1$ and $KP_2$ be two matching keypoints, respectively from $Picture_1$ and $Picture_2$. Let $Skeleton_1$ and $Skeleton_2$ be the extracted skeletons of $Picture_1$ and $Picture_2$. Let $Branch_1$ be a branch of $Skeleton_1$, $Branch_2$ a branch of $Skeleton_2$. We assume that $KP_1$ is associated to $Branch_1$ and $KP_2$ is associated to $Branch_2$. Since $KP_1$ and $KP_2$ are matching keypoints, a vote is given to a match between $Branch_1$ and $Branch_2$.
Each time a vote is given to a match between $Branch_i$ and $Branch_j$, the element $m_{i,j}$ of $M$ is incremented. Once each pair of matching keypoints has voted, the branches are matching using the matching matrix.
Let $Branch_i$ be a branch of $Skeleton_1$ and $M$ be the matching matrix between branches of $Skeleton_1$ and the branches of $Skeleton_2$. The index $j$ of the matching branch $Branch_j$ with $Branch_i$ is defined such as the element $m_{i,j}$ of $M_{i,\cdot}$ is maximal.
In pratice, the skeletons we try to match have never the same number of branches: some differences may the algorithm unable to give good results. The test report details the differences between skeletons. We thought about solutions, such as merging short branches with others. We could also ask the user to click on matching branches. In this case, there is no need to detect keypoints anymore. Another solution is to ask the user to click on noising branches that often appears on extremities of the skeleton.

21
report/subsections/calibration.tex Normal file
View File

@@ -0,0 +1,21 @@
% Here we go
\subsection{Camera calibration}
The camera calibration is needed by the client during the reconstruction of the 3D skeleton. The calibration consists in two steps :
\begin{enumerate}
\item the \textbf{internal calibration} which is the estimation of the instrinsic parameters of the camera (such as the focal length, or the
principal point)
\item the \textbf{external calibration} which is the estimation of the
extrinsic parameters (the position of the camera from the chessboard)
\end{enumerate}
\subsubsection{Internal calibration}
The internal calibration is based on a program that we completed during a lesson last semester. By detecting chessboards from numerous positions of the camera, we are able to estimate the intrinsic parameters. This program takes as input a video of a chessboard where the camera position changes, and it captures some frames on which the chessboard is detected. With these pictures, we estimate the intrinsic parameters.
\subsubsection{External calibration}
For the external calibration, we wrote a program that uses an
estimation of the homography to compute the extrinsical parameters of
the camera. One should note that what we call the \emph{extrinsical
parameters} are the ones that convert points from the reference of the
chessboard to the reference of the camera.

42
report/subsections/circles.tex Normal file
View File

@@ -0,0 +1,42 @@
\subsection{Regular mesh}
At the beginning of this part we have a set of 3D splines. We sample this splines and calculate the characteristic circle associated to each sampled point. Then we match the points between successive circles and use a triangular mesh.
\subsubsection{Characteristic circles computation}
We take a point sampled on the spline. It gives us information about the sphere located on this point : $\overrightarrow{C}(t)$ the sphere center coordinates and r(t) its radius. We also have their derivatives : $\overrightarrow{C'}(t)$ and r'(t). The idea is to find the intersection between this sphere and the characteristic plane, which would give us the characteristic circle.
If a point P is on the circle, then we can write the formula \ref{eq1} :
\begin{equation}
<\overrightarrow{C'}(t),\overrightarrow{PC}(t)> -r'(t)r(t) = 0
\label{eq1}
\end{equation}
and then we calculate the center of the characteristic circle with the formula \ref{eq2}:
\begin{equation}
\overrightarrow{Cp}(t) = \overrightarrow{C}(t) - \frac{r'(t)r(t)}{\| \overrightarrow{C'}(t)\| ^{2}} \times \overrightarrow{C'}(t)
\label{eq2}
\end{equation}
\begin{figure}[H]
\begin{center}
\includegraphics[scale=0.8]{img/characteristicCircle}
\caption{\label{initialSchedule} Characteristic circle calculation}
\end{center}
\end{figure}
\subsubsection{Meshing}
After obtaining characteristic circles we sample them.
Then we need to find matching points. We decided to make points sampling start in one direction of our mark.
When this is done we project the mark on the new circle and sample again. Then we link each point to the point of the same index on the next circle.
The final mesh can be seen Figure \ref{regularMesh}.
\begin{figure}[H]
\begin{center}
\includegraphics[scale=0.4]{img/regularMesh}
\caption{\label{regularMesh} The regular mesh}
\end{center}
\end{figure}
%TODO

42
report/subsections/extremities.tex Normal file
View File

@@ -0,0 +1,42 @@
\subsection{Extremities}
After computing the mesh around the splines we need to draw the extremities using subdivisions to obtain smoother extremities.
The first step is to calculate the projection of the extremity circle on the correspondent sphere. Then for a subdivision of depth 0 we link this projection with the mesh points along the extrem circle (see Figure \ref{projection}). For one subdivision we take the middle of each segments computed earlier and we project it on the sphere (see Figure \ref{sub}). Finally we made a triangular mesh the way you can see on Figure \ref{mesh}).
The rendering of our extremities on dino.skl for example can be seen on Figures \ref{extremity2} (2 subdivisions) and \ref{extremity10}
(10 subdivisions).
\begin{figure}[H]
\begin{center}
\includegraphics[scale=0.2]{img/projection}
\caption{\label{projection}Projection of the extrem circle center on the sphere}
\end{center}
\end{figure}
\begin{figure}[H]
\begin{center}
\includegraphics[scale=0.2]{img/Subdivision}
\caption{\label{sub}One subdivision}
\end{center}
\end{figure}
\begin{figure}[H]
\begin{center}
\includegraphics[scale=0.2]{img/meshExtremity}
\caption{\label{mesh}The final extremity mesh for one subdivision}
\end{center}
\end{figure}
\begin{figure}[H]
\begin{minipage}[b]{0.45\linewidth}
\centering
\includegraphics[scale=0.5]{img/2}
\caption{\label{extremity2}The final extremity mesh for 2 subdivisions}
\end{minipage}
\hspace{0.5cm}
\begin{minipage}[b]{0.45\linewidth}
\centering
\includegraphics[scale=0.5]{img/10}
\caption{\label{extremity10}The final extremity mesh for 10 subdivisions}
\end{minipage}
\end{figure}

93
report/subsections/junctions.tex Normal file
View File

@@ -0,0 +1,93 @@
\subsection{Junctions}
The last step to complete is to mesh the junctions.
A junction is a point of a skeleton where more than 2 branches join. The process for meshing this portion of the skeleton is complex. In fact it needs to be applicable on multiple cases, for instance 3 or 4 branches (see Figure \ref{junction}, and to take into account the fact that the perfect case will not always be there.
\begin{figure}[H]
\begin{center}
\includegraphics[scale=0.5]{img/Junctions9}
\caption{\label{junction}Three splines junction}
\end{center}
\end{figure}
In the figure \ref{junction1} you can see what we should theoretically obtain with a perfect skeleton. You can see the sphere shared between three splines joining in this junction, and the three characteristic circles associated. Those circles are tangent by pair in one point.
\begin{figure}[H]
\begin{center}
\includegraphics[scale=0.5]{img/JunctionTheory}
\caption{\label{junction1}Theoretical case with perfect characteristic circles}
\end{center}
\end{figure}
But in most of practical cases it is not like this. This is due to number's approximation on computers. The idea is then to look for the closest points of two consecutive circles and to join them. Then we join every point of the circles in one point upside and one downside.
We begin by identifying the last circles of each spline of the junction (see figure \ref{junction3}).
\begin{figure}[H]
\begin{center}
\includegraphics[scale=0.5]{img/Junctions3}
\caption{\label{junction3}3-splines junction}
\end{center}
\end{figure}
To be able to do it the first step is to cut the circles in two parts to find the upside and downside of each of them. This can be done by computing the best fitting plane for the set of circles' center points (figure \ref{junction2}).
The equation to find is easy when there is only 3 circles : first we compute the normal vector of the plane and then the constant element of the equation.
However when there is more than 3 circles to join, so more than three points to approximate, we need to use a PCA (Principal component analysis) decomposition to find the best fitting plane.
The first thing to do is to create X the matrix of the points coordinates.
\[
X =
\begin{pmatrix}
x_{1} & x_{2} & \cdots & x_{n} \\
y_{1} & y_{2} & \cdots & y_{n} \\
z_{1} & z_{2} & \cdots & z_{n}
\end{pmatrix}
\]
Then subtract it with the centroid matrix associated.
\[
X_{c} = X - X_{m}
\]
The normal vector $N$ of the best-fitting plane will be the cross product of the 2 eigenvectors $v_1,v_2$ corresponding to the two biggest singular values of the covariance matrix of $X_{c}$ :
\[
X_{c}X_{c}^T
\]
\[
N = v_1 \wedge v_2
\]
Then to compute the constant value of the equation we use the centroid.
\[
d = -N*X_m
\]
At the end we have the parameters of the plane in N and d.
\begin{figure}[H]
\begin{center}
\includegraphics[scale=0.35]{img/Junctions1}
\includegraphics[scale=0.35]{img/Junctions2}
\caption{\label{junction2}Best fitting plane for the set of circles' center points}
\end{center}
\end{figure}
Then when points are sorted we connect the up-points of each circle with the up-projection of the sphere's center on itself, and idem for the down-points with the down-projection of the sphere's center.
The result of this process is presented in figure \ref{junction4}.
\begin{figure}[H]
\begin{center}
\includegraphics[scale=0.5]{img/Junctions4}
\includegraphics[scale=0.5]{img/Junctions5}
\caption{\label{junction4}Edges added and mesh result}
\end{center}
\end{figure}
To easily see the result of the mesh on a junction, we have made some tests with only the extreme circles that we use. On real skeletons the junction are not easy to watch.
Here \ref{junction5} are two examples of a junction's mesh for 3 and 4-branches junctions.
\begin{figure}[H]
\begin{center}
\includegraphics[scale=0.5]{img/Junctions10}
\includegraphics[scale=0.4]{img/Junctions11}
\caption{\label{junction5}Mesh result on 3 and 4-branches junctions}
\end{center}
\end{figure}

62
report/subsections/pointsDetectionAndMatching.tex Normal file
View File

@@ -0,0 +1,62 @@
\subsection{Points detection and matching}
\label{detection}
\subsubsection{Points detection}
As a first step the detection of keypoints was made by using SURF (Speeded-Up Robust Features) algorithm (provided by
OpenCV) because of its good
speed but this algorithm is computed in nonfree module which is not free to use in commercial
application. That is why we decided to choose the BRISK (Binary Robust Invariant Scalable Keypoints)
algorithm (also provided by OpenCV) is pretty close to the SIFT (Scale-invariant feature transform) algorithm.
Therefore the tests were made using SURF and BRISK algorithm but the final version is with BRISK as SURF is patented.
As the aera of interest in the images is the object, we provide a binary mask (computed during the segmentation) in input of
SURF and BRISK detectors.
SURF is a speed version of SIFT (see Figure \ref{siftSurf}). Indeed instead of approximate Laplacian of Gaussian (LoG) with Difference
of Gaussian for finding scale-space. SURF goes a little further and approximates LoG with Box Filter.
\begin{figure}[H]
\centering
\includegraphics[scale=0.45]{img/LapinSiftSurf}
\caption{\label{siftSurf}Points detection for SIFT and SURF algorithm}
\end{figure}
For BRISK (see Figure \ref{brisk3415}) points of interest are identified across both the image and scale dimensions using a saliency criterion.
In order to increase the speed of computation, keypoints are detected in octave layers of the image pyramid as well as
in layers in-between. The location and the scale of each keypoint are obtained in the continuous domain via quadratic function fitting.
A sampling pattern consisting of points lying on appropriately scaled concentric circles is applied at the neighborhood of each keypoint to retrieve
gray values. Finally, the oriented BRISK sampling pattern is used to obtain pairwise brightness comparison results which are
assembled into the binary BRISK descriptor. The BRISK constructor take in input three parameters which modify the results :
the thresh, the octave and the patternScale.
A detailed explanation of these algorithms can be found with the references \cite{brisk} (BRISK), \cite{surf} (SURF) and
\cite{sift} (SIFT).
\begin{figure}[H]
\centering
\includegraphics[scale=0.45]{img/brisk3415}
\caption{\label{brisk3415}Detected points with BRISK algorithm}
\end{figure}
\subsubsection{Points matching}
We did the points matching, using a brute force matcher provided by OpenCV and then applied filters to get rid of inaccurate points.
For each descriptor in the first set, this matcher finds the closest descriptor in the second set by trying each one.
The filters used are :
\begin{itemize}
\item symmetric filter : the matches found when we take the image\_1 as base need to be found when we take the image\_2 as base also.
\item order constraint : the position of each point is compared to each other in image\_1 and image\_2, if there is too much error these points
are deleted.
\item threshold filter : filter on the distance between the descriptors of the matching points. This filter is not used with BRISK
detection because the results are quite good without it.
\item geometric filter : filter which use epipolar geometry, and the fundamental matrix to filter strange points.
\end{itemize}
\begin{figure}[H]
\centering
\includegraphics[scale=0.45]{img/LapinSymetricGeometric}
\caption{Points matching obtained after symmetric and geometric filtering}
\end{figure}

27
report/subsections/segmentation.tex Normal file
View File

@@ -0,0 +1,27 @@
\subsection{Segmentation}
This part was done by the other group. It consists in partitioning the orignal image \ref{img} into two regions : the background with the chessboard in one part and the plush in the other part. With the binary mask that the segmentation allows us to extract \ref{binarymask} we will be able to create the 2D skeleton with the client's supply \ref{skeleton} and to match the skeleton's branches (see \ref{detection}).
\begin{figure}[H]
\centering
\begin{subfigure}[b]{0.3\textwidth}
\includegraphics[width=\textwidth]{img/Poupee}
\caption{Original image}
\label{img}
\end{subfigure}%
~ %add desired spacing between images, e. g. ~, \quad, \qquad, \hfill etc.
%(or a blank line to force the subfigure onto a new line)
\begin{subfigure}[b]{0.3\textwidth}
\includegraphics[width=\textwidth]{img/Poupeebin}
\caption{Binary mask}
\label{binarymask}
\end{subfigure}
~ %add desired spacing between images, e. g. ~, \quad, \qquad, \hfill etc.
%(or a blank line to force the subfigure onto a new line)
\begin{subfigure}[b]{0.3\textwidth}
\includegraphics[width=\textwidth]{img/Poupeeskl}
\caption{Skeleton extract from (b)}
\label{skeleton}
\end{subfigure}
\caption{Pictures of animals}\label{segmentation}
\end{figure}

13
report/subsections/skeleton.tex Normal file
View File

@@ -0,0 +1,13 @@
\subsection{Skeleton}
Bastien, our client, is working on the extraction of skeleton from pictures. A skeleton of an object is the elemental structure of it. In this project 2D-skeletons extracted, like in figure \ref{skeletonex}, are defined by a set of vertices and edges. The other group compute with it 2D-skeleton in another format : a set of B-Splines, each of them representing a branch of the skeleton.
B-Splines are given by : the degree of the spline, their control points and the nodes vectors.
The 3D-skeletons are given with 3D B-Splines with the same structure except that the control point are in 3D.
You will find in appendix A \ref{AppendixA} some examples of 2D and 3D skeletons.
\begin{figure}[H]
\begin{center}
\includegraphics[scale=0.35]{img/SkeletonEx}
\caption{\label{skeletonex}Best fitting plane for the set of circles' center points}
\end{center}
\end{figure}

39
report/subsections/splines.tex Normal file
View File

@@ -0,0 +1,39 @@
The splines are a necessary element for everything that will follow. This is
why we tried to make it as generic as possible. For this, we used a C++11
feature called \emph{variadic templates}. A spline is composed of control
points (represented as C++ tuples), a nodes vector and degree. The class we
made looks like this
\begin{lstlisting}[language=c++]
template<typename... Types>
class Spline
{
public:
std::tuple<Types...> operator()(float t);
std::tuple<Types...> prime(float t);
private:
std::vector<std::tuple<Types...>> controlPoints;
std::vector<float> nodes;
int degree;
};
\end{lstlisting}
We redefined the \texttt{operator()} to be able to \emph{evaluate} a spline at
a certain time, and the result will be an object which is a barycenter of those
control points. We also made a member function \texttt{prime} to be able to
compute the derivative of a spline.
The \texttt{operator()} computes the barycent of the control points like this
$$C(t) = \sum_{i=0}^m P_i N_i^n(t)$$
where $(P_i)_{i\in [[0,m]]}$ are the $m+1$ control points, $n$ is the degree of
the spline and
$$N_i^0(t) = \left\{\begin{matrix}1&\text{if } t \in [t_i, t_{i+1}] \\ 0 & \text{otherwise} \end{matrix} \right.$$
$$ N_i^k(t) = \frac{t-t_i}{t_{i+k}-t_{i}} N_i^{k-1}(t) + \frac{t_{i+k+1} - t}{t_{i+k+1}-t_{i+1}} N_{i+1}^{k-1}(t) \quad\text{assuming} \quad \frac{0}{0} = 0$$
The \texttt{prime} member function computes the derivative like this
$$C'(t) = \sum_{i=0}^m N_i^{\prime n} (t) P_i $$
with
$$ N_i^{\prime n}(t) = \frac{n}{t_{i+n}-t_{i}}N_i^{n-1}(t) - \frac{n}{t_{i+n+1}-t_{i+1}}N_{i+1}^{n-1}(t)$$
These formulas have been taken from the reference \cite{spline}.