diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..e056704 --- /dev/null +++ b/.gitignore @@ -0,0 +1,23 @@ +*.aux +*.glo +*.idx +*.log +*.toc +*.ist +*.acn +*.acr +*.alg +*.bbl +*.blg +*.dvi +*.glg +*.gls +*.ilg +*.ind +*.lof +*.lot +*.maf +*.mtc +*.mtc1 +*.out +*.synctex.gz \ No newline at end of file diff --git a/Charges/cahier.mdown b/Charges/cahier.mdown new file mode 100644 index 0000000..d2e9ce1 --- /dev/null +++ b/Charges/cahier.mdown @@ -0,0 +1,158 @@ +# Cahier des charges + +## Description du besoin + +Dans le cadre du projet long de 3ème année et en étroite collaboration avec +Bastien Durix dont la thèse porte sur le sujet de l'extraction de squelettes, +notre rôle est de fournir un pipeline complet allant partant d'un ensemble +d'images en entrée dont on segmente les contours. On extrait ensuite le +squelette. En parallèle, on apparie les points. Une fois ces deux premières +étapes faites, on cherche à habiller et enfin à animer le squelette ainsi +habillé en définissant des points de rotation. + +Nous définissons ici un cahier des charges aussi précis que possible : dans un +premier temps, nous spécifions le besoin, puis nous dressons une liste +d'exigences. + +## Livrables attendus et dates de livraison souhaitées + +On s'appliquera à livrer une archive contenant : + - une documentation détaillée + - les codes sources : une partie calibrage et mise en correspondance dont la + date de livraison espérée est prévue pour le 6 février, puis une partie + habillage et animation dont la date de livraison est fixée au 13 mars + - une formation au C++ 11 + +## Fournitures du client + +Le client s'engage à fournir les éléments suivants : + - les programmes réalisant l'extraction de squelette + - les formules mathématiques permettant l'habillage du squelette + - les références aux articles sur lesquels s'appuyer pour l'animation + +## Contraintes techniques imposées + +Les contraintes techniques sont les suivantes : + - le langage de programmation sera le C++ + - il est suggéré de s'appuyer sur la librairie OpenCV + - l'utilitaire de compilation suggéré est CMake + - les librairies utilisées doivent toutes être portables entre Windows et + Linux + +## Contraintes de gestion de projet et de suivi qualité + +Les contraintes de gestion de projet sont les suivantes : + - la programmation sera effectuée par pair programming + - au moins une réunion par semaine sera effectuée avec le client + - une réunion par semaine est programmée chaque jeudi après-midi de 15h30 à + 16h30 avec l'industrielle + +## Organisation client +Les clients sont une équipe de chercheurs composée de : + - Bastien Durix, en thèse à l'IRIT + - Sylvie Chambon, maître de conférence à l'IRIT + - Géraldine Morin, maître de conférence à l'IRIT + +Au moins une réunion par semaine sera effectuée avec le client, deux en début +de projet pour la rédaction du cahier des charges. + +# Gestion des risques + +## Tableau des risques + +| Id | Description | Cause du risque | Proba(1-5) | Conséquence | Gravité(1-5) | Actions préventives | Actions correctives | Etat du risque | +| --- | --- | --- | :---: | --- | :---: | --- | --- | --- | +| 1 | Mauvaise estimation du temps | Sous-estimation de la difficulté du problème | 3 | Livraison en retard | 4 | Prendre de la marge sur le planning, nommer un responsable de l'avancement des différentes tâches | Augmenter la charge de travail, revoir le planning | Ouvert | +| 2 | L'autre groupe ne livre pas à temps | Sous-estimation de la difficulté de leur problème, augmentation de la charge de travail au dernier moment | 2 | Retard pour les tests de la partie mise en 3D du squelette | 3 | Envisager une autre méthode pour pouvoir tester sur des cas simples | Implémenter soi-même une méthode de segmentation plus simple se basant sur un simple seuil | Ouvert | +| 3 | L'autre groupe ne livre pas les bonnes sorties | Mauvaise compréhension des termes du sujet | 3 | Les tests ne fonctionneront pas correctement | 1 | Se mettre d'accord sur le format | Développement d'une fonction de transition | Ouvert | +| 4 | Les binaires fournis par le client ne sont pas compatibles | Complexité des systèmes | 2 | On ne peut pas continuer le pipeline | 3 | Discuter avec le client à propos du binaire fourni | Emuler le système permettant d'utiliser les binaires | Ouvert | +| 5 | L'entente au sein du groupe est fragile | Différentes habitudes de gestion de projet, de programmation | 1 | Tensions, ambiance accablante, stress intense | 5 | Nommer un responsable de la cohésion | Organiser une médiation entre les deux partis | Ouvert | +| 6 | L'entente avec l'autre groupe est périlleuse | Le découpage un peu bancal du projet pourrait amener des tensions | 2 | Difficultés à gérer les parties communes | 3 | Apporter une offrande en gage de bonne entente | Faire des compromis | Ouvert | +| 7 | L'un des membres est absent pendant une période donnée | Maladie, évènement | 5 | Absence d'une personne | 1 | Mettre en place des solutions de télétravail | Adaptation du planning | Ouvert | +| 8 | Production de code non fiable, non maintenable et non lisible | Manque de maîtrise des technologies utilisées | 4 | Pertes de temps, mauvaise qualité, devoir refaire des programmes | 3 | Formation au C++ par un expert technique | Appel de l'expert | Ouvert | +| 9 | Besoin d'accéder à une ressource supprimée de manière urgente | Utilisation d'une commande de suppression, crash du matériel | 2 | Pertes de temps, devoir refaire des programmes | 5 | Utiliser git | Utilisation de techniques de récupération de fichiers, longues et compliquées | Ouvert | + + +## Spécifications + +### Partie appariement + +- En entrée : $n$ images d'un même objet photographiées selon $n$ points de vue + différents sur un fond unis de même couleur (vert, rouge ou bleu) dont une + sera l'image de référence. Pour un même objet, les conditions de prises de + vue (éclairage, appareil photo, position de l'objet) doivent être les mêmes : + entre deux photos, l'objet ne doit pas bouger, c'est l'utilisateur qui se + déplace. Si le temps le permet, on pourra envisager de suspendre l'objet pour + les prises de vue. On définira une résolution minimale. Un marqueur (croix) + servant à calibrer par la suite la caméra sera visible sur le papier. On + définira précisément la façon dont ces marqueurs seront présents : un damier + risque de géner la segmentation. Des croix espacées régulièrement seraient + plus discrètes mais pourraient aussi géner. On se demande s'il serait + possible de n'afficher que 4 croix pour définir 4 coins. + +- Traitement : + 1. Appliquer SIFT sur chaque photo pour récupérer un ensemble de points + d'intérêt par photo. Les paramètres de l'algorithme SIFT seront à définir. + 1. Définir les paramètres intrinsèques de la caméra sous la forme d'un fichier + au format xml (matK). + 1. Définir les paramètres extrinsèques de la caméra à l'aide des marqueurs : + les détecter puis calculer l'homographie. + 1. Appariement des points d'intérêt. + +- En sortie : Des paires de correspondances de points d'intérêt entre l'image + de référence et une autre image. + +### Partie habillage + +- En entrée : un squelette 3D (B-Splines) et une fonction de rayon. + +- Traitement : Utilise les surfaces canales. + +- En sortie : Un maillage 3D (points et faces). + +### Partie animation + +- En entrée : le maillage 3D, les points d'articulations sur le squelette + et leur degré de liberté associé. + +- Traitement: à élucider ? + +- En sortie : l'affichage d'une animation. + +### Définition des tâches +| | Détection des points d'intérêts | Appariement | Calibrage interne de la caméra | Calibrage externe de la caméra | Reconstruction 3D du squelette | Habillage | Animation | +| --- | --- | --- | --- | --- | --- | --- | --- | +| Conception | | | | | | | | +| Rédaction de la documentation | | | | | | | | +| Préparation des tests | | | | | | | | +| Implémentation | | | | | | | | + + +### Matrice de Rasci +| | Groupe 8 | Chef de projet | Service qualité | Equipe 1 | Equipe 2 | Client | Régine Nigris| +| --- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | +| Prise des photos | R | I | I | A-R | R | C | | +| Détection des points d'intérêts | | C | I | A-R | | C | | +| Appariement | | C | I | A-R | | C | | +| Calibrage interne de la caméra | | C | I | | A-R | C | | +| Calibrage externe de la caméra | | C | I | | A-R | C | | +| Reconstruction 3D du squelette | | C | I | R | A-R | C | | +| Habillage | | C | I | | | C | | +| Animation | | C | I | | | C | | +| Gestion de projet | | A | R | R | R | | C | + +### Exigences + + 1. L'algorithme doit pouvoir évaluer les paramètres extrinsèques à partir des + marqueurs. + 1. L'algorithme doit pouvoir évaluer les paramètres intrinsèques de la caméra. + +### Rôles + +Afin de réaliser le projet dans de bonnes conditions, nous avons choisis de développer notre prototype en travaillant par *pair-programming*, c'est à dire travailler en parallèle par groupe de deux. La cinquième personne serait quand à elle mobile et servirait de support aux deux groupes en cas de blocage sur un aspect technique. Le poste de *support technique* pourra éventuellement être tournant. + +La répartition initiale des rôles est la suivante : + - Le chef de projet : Thomas Forgione + - Responsable qualité : Amandine Pailloux + - Expert algorithmique : Thierry Malon + - Gestion des ressources humaines : Marion Lenfant diff --git a/Charges/projectManagement.tex b/Charges/projectManagement.tex new file mode 100644 index 0000000..c88deb6 --- /dev/null +++ b/Charges/projectManagement.tex @@ -0,0 +1,335 @@ +\documentclass{article} + +\usepackage[top=2cm,bottom=2cm,left=2cm,right=2cm]{geometry} +\usepackage{array} +\usepackage{colortbl} +\usepackage{supertabular} + +\author{Thomas Forgione, Emilie Jalras, Marion Lenfant, Thierry Malon and Amandine Pailloux} +\date{\today} +\title{Project management} + +\begin{document} +\maketitle +\begin{center} + Version 3 +\end{center} + +\section{Need description} + +Our third year final project consists in producing a 3D skeleton, its casing, and its animation +from some images of an object. +This project is done in collaboration with Bastien Durix, whose thesis is about skeleton extraction. +The steps of the project development are as follow : +\begin{itemize} + \item segmentation of the image + \item skeleton extraction + \item points matching + \item camera calibration + \item skeleton branches matching + \item skeleton casing + \item animation (using rotation points) +\end{itemize} + +\subsection{Expected deliverables and delivery delay} + +We will deliver an archive containing: +\begin{itemize} + \item detailed documentation (commented code and a user manual) + \item source code : camera calibration and points matching, delivered on February 6th, + and then casing and animation on March 13th. +\end{itemize} + +\subsection{Client supplies} + +The client engages to supply the following elements : +\begin{itemize} + \item programs performing the skeleton extraction + \item mathematical formulas for skeleton casing + \item scientific articles concerning animation +\end{itemize} + +\subsection{Imposed technical constraints} + +Our technical constraints are as follow : +\begin{itemize} + \item C++ as programming language + \item it is suggested to use the openCV library + \item CMake is suggested as compilation utility + \item used libraries must be portable between Windows and Linux +\end{itemize} + +\subsection{Project management constraints and quality follow-up} + +The project management constraints are the following : +\begin{itemize} + \item programming is done by pair programming + \item at least one meeting a week with the client will be carried out + \item one meeting a week is set every Thursday from 3:30pm to 4:30pm with the industrial +\end{itemize} + +\subsection{Client organisation} + +Our clients are a team of researchers composed from : +\begin{itemize} + \item Bastien Durix, preparing a thesis at the IRIT + \item Sylvie Chambon, lecturer at the IRIT + \item Geraldine Morin, lecturer at the IRIT +\end{itemize} + +At least one meeting a week with the client will be carried out, and two meetings a week +at the beginning of the project to write the specifications. + +\section{Risks management} + +\subsection{Risks table} + + +\bottomcaption{Risks table} +\begin{supertabular}{|p{0.5cm}|p{2cm}|p{2cm}|p{1cm}|p{2cm}|p{1.5cm}|p{2cm}|p{2cm}|p{2cm}|p{1.5cm}|} + +\hline + Id & Description & risk cause & Proba (1-5) & Consequence & Seriousness (1-5) & Preventive actions & Corrective actions & Risk state \\ +\hline + \cellcolor{green} 1 & Bad time estimation & Under estimation of the problem difficulty & 3 & Late delivery & 4 & Time margin of the planning, name a task advancement responsible & Raise the work amount, revise the planning & Open \\ + \hline + \cellcolor{yellow} 2 & The other group does not deliver the deliverables on time & Under estimation of their problem difficulty, last time augmentation of the work amount & 2 & Delay for tests on the 3D part of the skeleton & 3 & Contemplate an other method to test on simple case & Implement ourselves a segmentation method based on simple threshold & Open \\ + \hline + \cellcolor{green} 3 & The other group delivers wrong outputs & Bad comprehension of the subject & 3 & Tests do not work correctly & 1 & Find en agreement on the format & Develop a transition function & Open \\ + \hline + \cellcolor{yellow} 4 & Binaries provided by the client are not compatible & Systems complexity & 4 & We can not carry on with the pipeline & 4 & Discuss with the client the provided binary & Emulate the system allowing the use of the binaries & Open \\ + \hline + \cellcolor{green} 5 & Fragile harmony inside the group & Different uses for project management and programmation & 1 & Tensions, bad ambiance, intense stress & 5 & Name a cohesion responsible & Organize a mediation beetween the two parties & Open \\ + \hline + \cellcolor{green} 6 & Bad understanding with the other group & A bancal division of the project might bring tensions & 2 & Difficulty to manage shared parts & 3 & Bring a present as a sign of sympathy & Come to a compromise & Open \\ + \hline + \cellcolor{green} 7 & A group member is absent during a certain period of time & Illness, event & 5 & Someone's absence & 1 & Put in place teleworking solutions & Planning adaptation & Open \\ + \hline + \cellcolor{green} 8 & Production of unreliable, unmaintainable and illegible code & Lack of skills with the used technology & 4 & Lack of time, bad quality, need to reprogram & 3 & C++ formation by a technical expert & Call of the expert & Closed \\ + \hline + \cellcolor{green} 9 & Need to access urgently to a deleted resource & Utilization of a suppression command, material crash & 2 & Lack of time, need to reprogram & 5 & Use github & Use of recuperation technology on long and complicated files & Open \\ + \hline + \cellcolor{yellow} 10 & Pictures unadapted & Client not satisfied of the pictures & 4 & Wasting time & 2 & Defined the conditions of the shoot with the client more precisely & Be aware of client expectations and taking again the picture & Open \\ +\hline + \cellcolor{green} 11 & New task to do & Client new needs & 4 & Late delivery & 4 & Add margin in planning & change planning and contact the client to negociate about this task & Closed \\ + \hline + \cellcolor{green} 12 & Integration not cancelled & Client changes his mind and wants again the integration & 2 & Late delivery & 4 & discuss with the client & change planning & Open \\ + \hline + \cellcolor{green} 13 & members of the group get remedial classes & bad results at the exams & 2 & member absent & 2 & add margin & change planning & Open \\ + \hline +\end{supertabular} + + +\subsection{Specifications} + +\subsubsection{Matching part} + +\begin{itemize} + \item Input : $n$ images from the same object photographed on a plain green background within $n$ different points of view + of which one will be the reference image. For the same object the shooting conditions must be the same : + \begin{itemize} + \item the light + \item the object position + \item the camera used + \end{itemize} + + A minimum resolution will be defined. + A target has to be printed and put on the image for calibration : we chose to use a chequerboard. + + \item Processing : + + \begin{enumerate} + \item Apply SURF on every picture to get a group of interest points for each picture. + The SURF algorithm parameters will be defined later + \item Define the camera intrinsic parameters as a .xml file. + \item Define the camera extrinsic parameters using the chequerboard. + \item Matching of the interest points. + \end{enumerate} + \item Output : correspondence pairs of interest points between the reference image and the other image. + \end{itemize} + +\subsubsection{Casing part} + +\begin{itemize} + \item Input : a 3D skeleton (B-Splines) and a rayon function. + + \item Processing : Use canal surfaces + + \item Output : a 3D mesh (edges and vertices) + \end{itemize} +\subsubsection{Animation part} +\begin{itemize} + \item Input : a 3D mesh, articulation points on the skeleton and their associated degree of freedom + + \item Processing : to do + + \item Output : animation display + \end{itemize} + +\subsubsection{Tasks definition} + + +\begin{table}[h] +\begin{center} +\begin{tabular}{|p{2.2cm}|p{2cm}|p{2cm}|p{2cm}|p{2cm}|p{2cm}|} +\hline + & Keypoints detection & Matching & Camera intern calibration & Camera extern calibration & skeleton matching \\ + \hline + Conception & Emilie and Amandine & Emilie and Amandine & Thomas & Thomas and Marion & Thierry and Thomas \\ + \hline + Redaction of documentation & Emilie and Amandine & Emilie and Amandine & Thomas & Thomas and Marion & Thierry and Thomas \\ + \hline + Tests & Emilie and Amandine & Emilie and Amandine & Thomas & Thomas, Marion and Thierry & Thierry and Thomas \\ + \hline + Implementation & Emilie and Amandine & Emilie and Amandine & Thomas & Thomas and Marion & Thierry and Thomas \\ + \hline +\end{tabular} +\caption{Tasks definition for part 1 : segmentation, camera calibration, skeletonization, matching} +\end{center} +\end{table} + + +\begin{table}[h] +\begin{center} +\begin{tabular}{|p{2.2cm}|p{2cm}|p{2cm}|p{2cm}|p{2cm}|p{2cm}|p{2cm}|p{2cm}|} +\hline + & Function B-Splines from controll points given & Compute caracteristic circle & Matching points on the circles & Quadrangular mesh computing & Ends processing & junctions processing \\ + \hline + Conception & Thomas and Thierry & Emilie and Thierry & Marion and Thomas & Amandine and Thierry & Emilie and Amandine & Marion and Thomas \\ + \hline + Redaction of documentation & Thomas and Thierry & Emilie and Thierry & Marion and Thomas & Amandine and Thierry & Emilie and Amandine & Marion and Thomas \\ + \hline + Tests & & Thierry & & & Emilie, Amandine and Thierry & \\ + \hline + Implementation & Thomas and Thierry & Emilie and Thierry & Marion and Thomas & Amandine and Thierry & Emilie and Amandine & Marion and Thomas \\ + \hline +\end{tabular} +\caption{Tasks definition for part 2.1 : meshing} +\end{center} +\end{table} + +\begin{table}[h] +\begin{center} +\begin{tabular}{|p{2.2cm}|p{2cm}|p{2cm}|p{2cm}|p{2cm}|} +\hline + & clicks on joints & create branches & mesh and branches matching & renderer \\ + \hline + Conception & & Emilie and Amandine & Thierry and Marion & Thomas\\ + \hline + Redaction of documentation & & Emilie and Amandine & Thierry and Marion & Thomas\\ + \hline + Tests & & & & \\ + \hline + Implementation & & Emilie and Amandine & + Thierry and Marion & Thomas\\ + \hline +\end{tabular} +\caption{Tasks definition for part 2.2 : animation} +\end{center} +\end{table} + +\subsubsection{Matrix of responsibilities} +\begin{table} +\begin{tabular}{|p{2cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}|} + \hline + & Group 8 & Project leader & Quality service & Team 1 & Team 2 & Client & Régine Nigris \\ + \hline + Pictures shooting & R & I & I & A-R & R & C & \\ + \hline + Interest points detection & & C & I & A-R & & C & \\ + \hline + Matching & & C & I & A-R & & C & \\ + \hline + Camera intern calibration & & C & I & & A-R & C & \\ + \hline + Camera extern calibration & & C & I & A-R & C & & \\ + \hline + Skeleton 3D reconstruction & & C & I & R & A-R & C & \\ + \hline + Casing & & C & I & & & C & \\ + \hline + Animation & & C & I & & & C & \\ + \hline + Project management & & A & R & R & R & & C \\ + \hline +\end{tabular} +\caption{Matrix of responsibilities} +\end{table} + +\subsubsection{Requirements} +\begin{enumerate} + \item The algorithm must evaluate extrinsic parameters using the chequerboar. + \item The algorithm must be able to evaluate intrinsic parameters. +\end{enumerate} + +\subsubsection{Roles} + +We decided to develop our prototype working in "pair programming", meaning we will work in parallel in teams of two. +The fifth person would be mobile and support the other groups in case of blockage on a technical aspect. +These "technical support" post would be moving. + +The initial role repartition is the following : +\begin{itemize} + \item Project leader : Thomas Forgione + \item Quality responsible : Amandine Pailloux + \item Tester : Thierry Malon + \item Task advancement responsible : Emilie Jalras + \item Integration responsible : Marion Lenfant +\end{itemize} + +\subsubsection{Actions table} +\begin{table} +\begin{tabular}{|p{0.5cm}|p{3cm}|p{3cm}|p{2cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}|p{1cm}|} +\hline + Id & Origin & Description & Accountable & Target date & Realisation & State & Risk table\\ +\hline + 1 & Project management & Meeting with the industrial supervisor & Thomas & 15 Jan & 15 Jan & Closed & \\ + \hline + 2 & Need of specifications & Meeting with the client & Thierry & 22 Jan & 22 Jan & Closed & \\ + \hline + 3 & Lack of knowledge from the developer & C++ training & Thomas & 22 Jan & 23 Jan & Closed & \\ + \hline + 4 & Team work \& version management & Git repository creation & Thomas & 05 Feb & 05 Feb & Closed & \\ + \hline + 5 & Project management & Meeting with the industrial supervisor & Thierry & 21 Jan & 21 Jan & Closed & \\ + \hline + 6 & Input needed & Taking pictures & Thierry & 22 Jan & 22 Jan & Closed & \\ + \hline + 7 & Report to write in english & Specifications translation & Emilie & 22 Jan & 22 Jan & Closed & \\ + \hline + 8 & Set up the dev tools & Install Linux / OpenCV & Thomas & 01 Feb & 02 Feb & Closed & \\ + \hline + 9 & Need of specifications & Meeting with the client & Thierry & 04 Feb & 04 Feb & Closed & \\ + \hline + 10 & Remember stuff to do & Actions table writing & Marion & 05 Feb & 04 Feb & Closed & \\ + \hline + 11 & Project management & Meeting with the industrial supervisor & Marion & 5 Feb & 5 Feb & Closed & \\ + \hline + 12 & Git Repository not well organized & Git cleaning & Thomas & 05 Feb & 05 Feb & Closed & \\ + \hline + 13 & Compatibility with the other team & Defining subskeletons format & Thierry & 06 Feb & 09 Feb & CLosed & \\ +\hline +14 & Need of specifications & Meeting with the client & Thierry & 11 Feb & 11 Feb & Closed & \\ + \hline + 15 & The other group did'nt deliver binaries in time & Reschedule planning & Marion & 11 Feb & 11 Feb & Closed & 2 \\ + \hline + 16 & Binaries provided by he client are not compatible & Speak with the client & Thomas & 11 Feb & & Open & 4 \\ + \hline + 17 & Client new need & Contact the client by mail to know what we need to do & Thierry & 12 Feb & 12 Feb & Closed & 11 \\ + \hline + 18 & Project management & Meeting with the industrial supervisor & Amandine & 12 Feb & 12 Feb & Closed & \\ + \hline + 19 & Integration to do again & discuss with the client & Thierry & 25 Feb & 25 Feb & Closed &\\ + \hline + 20 & Project management & Meeting with the industrial supervisor & Emilie & 24 Feb & 24 Feb & Closed & \\ + \hline + 21 & Need of specifications & Meeting with the client & Thierry & 25 Feb & 25 Feb & Closed & \\ + \hline + 22 & The client need CMake to compile the code & Use CMake compilation & Thomas & 27 Feb & 27 Feb & Closed & \\ + \hline +\end{tabular} +\caption{Actions table} +\end{table} + + +\end{document} diff --git a/Charges/specifications.mdown b/Charges/specifications.mdown new file mode 100644 index 0000000..8a11611 --- /dev/null +++ b/Charges/specifications.mdown @@ -0,0 +1,153 @@ +# Specifications + +## Need description + +Our third year final project consists in producing a 3D skeleton, its casing, and its animation +from some images of an object. +This project is done in collaboration with Bastien Durix, whose thesis is about skeleton extraction. +The steps of the project development are as follow : + - segmentation of the image + - skeleton extraction + - points matching + - camera calibration + - skeleton branches matching + - skeleton casing + - animation (using rotation points) + + +## Expected deliverables and delivery delay + +We will deliver an archive containing: + - detailed documentation (commented code and a user manual) + - source code : camera calibration and points matching, delivered on February 6th, + and then casing and animation on March 13th. + + +## Client supplies + +The client engages to supply the following elements : + - programs performing the skeleton extraction + - mathematical formulas for skeleton casing + - scientific articles concerning animation + + +## Contraintes techniques imposées +## Imposed technical constraints + +Our technical constraints are as follow : + - C++ as programming language + - it is suggested to use the openCV library + - CMake is suggested as compilation utility + - used libraries must be portable between Windows and Linux + + +## Project management constraints and quality follow-up + +The project management constraints are the following : + - programming is done by pair programming + - at least one meeting a week with the client will be carried out + - one meeting a week is set every Thursday from 3:30pm to 4:30pm with the industrial + + +## Client organisation + +Our clients are a team of researchers composed from : + - Bastien Durix, preparing a thesis at the IRIT + - Sylvie Chambon, lecturer at the IRIT + - Geraldine Morin, lecturer at the IRIT + +At least one meeting a week with the client will be carried out, and two meetings a week +at the beginning of the project to write the specifications. + +# Risks management + +## Risks table + +| Id | Description | risk cause | Proba(1-5) | Consequence | Seriousness(1-5) | Preventive actions | Corrective actions | Risk state | +| --- | --- | --- | :---: | --- | :---: | --- | --- | --- | +| 1 | Bad time estimation | Under estimation of the problem difficulty | 3 | Late delivery | 4 | Time margin of the planning, name a task advancement responsible | Raise the work amount, revise the planning | Open | +| 2 | The other group does not deliver the deliverables on time | Under estimation of their problem difficulty, last time augmentation of the work amount | 2 | Delay for tests on the 3D part of the skeleton | 3 | Contemplate an other method to test on simple case | Implement ourselves a segmentation method based on simple threshold | Open | +| 3 | The other group delivers wrong outputs | Bad comprehension of the subject | 3 | Tests do not work correctly | 1 | Find en agreement on the format | Develop a transition function | Open | +| 4 | Binaries provided by the client are not compatible | Systems complexity | 2 | We can not carry on with the pipeline | 3 | Discuss with the client the provided binary | Emulate the system allowing the use of the binaries | Open | +| 5 | Fragile harmony inside the group | Different uses for project management and programmation | 1 | Tensions, bad ambiance, intense stress | 5 | Name a cohesion responsible | Organize a mediation beetween the two parties | Open | +| 6 | Bad understanding with the other group | A bancal division of the project might bring tensions | 2 | Difficulty to manage shared parts | 3 | Bring a present as a sign of sympathy | Come to a compromise | Open | +| 7 | A group member is absent during a certain period of time | Illness, event | 5 | Someone's absence | 1 | Put in place teleworking solutions | Planning adaptation | Open | +| 8 | Production of unreliable, unmaintainable and illegible code | Lack of skills with the used technology | 4 | Lack of time, bad quality, need to reprogram | 3 | C++ formation by a technical expert | Call of the expert | Open | +| 9 | Need to access urgently to a deleted resource | Utilization of a suppression command, material crash | 2 | Lack of time, need to reprogram | 5 | Use github | Use of recuperation technology on long and complicated files | Open | + + +## Specifications + +### Matching part + + - Input : $n$ images from the same object photographed on a plain green background within $n$ different points of view + of which one will be the reference image. For the same object the shooting conditions must be the same : + - the light + - the object position + - the camera used + A minimum resolution will be defined. + A target has to be printed and put on the image for calibration : we chose to use a chequerboard. + + - Processing : + 1. Apply SIFT on every picture to get a group of interest points for each picture. + The SIFT algorithm parameters will be defined later. + 1. Define the camera intrinsic parameters as a .xml file. + 1. Define the camera extrinsic parameters using the chequerboard. + 1. Matching of the interest points. + + - Output : correspondence pairs of interest points between the reference image and the other image. + +### Casing part + + - Input : a 3D skeleton (B-Splines) and a rayon function. + + - Processing : Use canal surfaces + + - Output : a 3D mesh (edges and vertices) + +### Animation part + + - Input : a 3D mesh, articulation points on the skeleton and their associated degree of freedom + + - Processing : to do + + - Output : animation display + + +### Tasks definition +| | Interest points detection | Matching | Camera intern calibration | Camera extern calibration | skeleton 3D reconstruction | Casing | Animation | +| --- | --- | --- | --- | --- | --- | --- | --- | +| Conception | | | | | | | | +| Redaction of documentation | | | | | | | | +| Tests preparation | | | | | | | | +| Implementation | | | | | | | | + + +### Rasci matrix +| | Group 8 | Project leader | Quality service | Team 1 | Team 2 | Client | Régine Nigris| +| --- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | +| Pictures shooting | R | I | I | A-R | R | C | | +| Interest points detection | | C | I | A-R | | C | | +| Matching | | C | I | A-R | | C | | +| Camera intern calibration | | C | I | | A-R | C | | +| Camera extern calibration | | C | I | | A-R | C | | +| Skeleton 3D reconstruction | | C | I | R | A-R | C | | +| Casing | | C | I | | | C | | +| Animation | | C | I | | | C | | +| Project management | | A | R | R | R | | C | + +### Requirements + + 1. The algorithm must evaluate extrinsic parameters using the chequerboar. + 1. The algorithm must be able to evaluate intrinsic parameters. + +### Roles + +We decided to develop our prototype working in "pair programming", meaning we will work in parallel in teams of two. +The fifth person would be mobile and support the other groups in case of blockage on a technical aspect. +These "technical support" post would be moving. + +The initial role repartition is the following : + - Project leader : Thomas Forgione + - Quality responsible : Amandine Pailloux + diff --git a/Organisation/ReunionSuperviseur.ods b/Organisation/ReunionSuperviseur.ods new file mode 100644 index 0000000..6428f99 Binary files /dev/null and b/Organisation/ReunionSuperviseur.ods differ diff --git a/Rapports/.gitignore b/Rapports/.gitignore new file mode 100644 index 0000000..b276019 --- /dev/null +++ b/Rapports/.gitignore @@ -0,0 +1,25 @@ +*.tex +*.aux +*.glo +*.idx +*.log +*.toc +*.ist +*.acn +*.acr +*.alg +*.bbl +*.blg +*.dvi +*.glg +*.gls +*.ilg +*.ind +*.lof +*.lot +*.maf +*.mtc +*.mtc1 +*.out +*.synctex.gz +*.pdf diff --git a/Rapports/RapportReunion15012015.mdown b/Rapports/RapportReunion15012015.mdown new file mode 100644 index 0000000..9358091 --- /dev/null +++ b/Rapports/RapportReunion15012015.mdown @@ -0,0 +1,62 @@ +Compte-rendu de la réunion 1 du 15 janvier 2015 +========== +A l'approche du démarrage du projet, cette première réunion a été l'occasion : + +* dans un premier temps, **avec les deux groupes réunis**, de rencontrer notre superviseur industriel, Régine Nigris, d'Airbus. +* dans un second temps, **au sein de notre groupe**, de répondre à quelques questions fondamentales indispensables au bon démarrage du projet. + +##Rencontre avec notre superviseur industriel + +**Régine Nigris** (_[regine.nigris@airbus.com](regine.nigris@airbus.com)_), qui travaille chez Airbus en temps que project manager, supervisera les aspects gestion de projet et gestion de la qualité. Comme elle nous l'a rappelé, son rôle n'est pas de regarder les aspects techniques ; c'est au client et à nous-mêmes de gérer ces derniers. + +Un premier tour de table a permis à chacun de **se présenter** brièvement et de **donner son rôle au sein du projet**. L'ensemble des dix membres constituant les deux groupes était présent. + +###Les réunions à venir + +Nous avons ensuite convenu d'un créneau horaire fixe de réunion pour les semaines à venir : + +* exceptionnellement, **mercredi 21 janvier de 15h30 à 16h30 en B122** (une heure par groupe, notre groupe passant en deuxième). +* puis à partir du 5 février, tous les **jeudis de 15h30 à 16h30 en salle A201**. + +Nous avons choisi de ne **pas travailler la semaine du 26 au 30 janvier**. + +Au cours des prochaines réunions, il nous est conseillé de venir avec un rapport présentant les tâches effectuées dans la semaine et les difficultés rencontrées. **L'auteur du rapport de la semaine sera en charge d'animer la réunion** et de présenter le travail effectué. Ce rôle tournera chaque semaine. + +###Le rapport individuel + +Un rapport individuel devra être remis (en anglais ou en français) **mi-février**. Il devra présenter de manière précise les apports de la gestion de projet dans le cadre de notre projet long. Une version numérique de ce qui est attendu nous sera envoyée. L'ensemble des autres documents que l'on devra remettre étant en anglais, il nous est conseillé de rédiger ce rapport individuel en anglais également. + +###Les tâches à effectuer à court terme + +Nous avons ensuite donner une liste de tâches à effectuer en début de projet : + +* Le **cahier des charges** : définir clairement le besoin lors de la prochaine réunion avec le client. +* Un premier **planning** : assez vague pour le moment, il changera très probablement par la suite, mais l'on peut déjà y incorporer les réunions. +* Définir les **rôles** dans l'équipe. +* Initialiser la **table des risques** et des **actions préventives** pour les limites, ainsi que des **actions correctives** si l'un des risques se produit. +* Réfléchir au **WBS**. + +###Concernant la répartition des parties 1 et 2 + +Après le départ de l'industrielle, les deux groupes ayant a priori plutôt des attraits pour la partie de segmentation, une discussion a eu lieu pour décider auquel il serait attribué. **Nous avons finalement cédé la partie 1** (segmentation du contour et de la normale) à l'autre groupe et **nous traiterons donc la partie 2** (placement du squelette). + +##Réunion de groupe + +Une fois la réunion avec les deux groupes terminée, nous sommes restés afin de mener une première réunion entre membres du groupe. L'objectif a été de déterminer : + +* un lieu de travail : **chez Thomas** dans un premier temps, les salles de TP risquant d'être bruyantes et remplies en période de projet long. +* un **premier planning** de la première semaine pour fixer les heures de travail et placer les réunions avec le client et avec l'industrielle. +* un maximum d'éléments de spécification au client pour pouvoir formuler un cahier des charges aussi complet, détaillé et précis que possible. + * Qu'est-ce qui nous est **fourni** ? + * Quelles **entrées** ? Combien de photos en entrée ? + * Qu'y a-t-il en **sortie de la première partie** (c'est-à-dire en entrée de la deuxième partie) ? + * Sous quel environnement le produit final sera-t-il utilisé ? (**Windows ou Linux** ?) + * Y a-t-il des **méthodes particulières à utiliser** et, le cas échéant, quels noms portent-elles ? Y a-t-il des **articles à consulter** ? + * Y a-t-il des **IHM** à développer et éventuellement lesquelles ? Ou tout en ligne de commande ? + * A quelle date pourront-ils nous donner les **modalités exactes des parties 3 et 4** (habillage et animation) ? + * Comment est-on censé s'y prendre pour l'**appariement** ? + * Faut-il cliquer pour guider la **recherche de squelette** ? Comment s'intègre l'**algorithme de Ma** avec le squelette approximatif défini par l'utilisateur ? + * Quelle **documentation** est attendue ? + +_Rédigé par Thierry Malon_ + diff --git a/Rapports/conception.mdown b/Rapports/conception.mdown new file mode 100644 index 0000000..b96be19 --- /dev/null +++ b/Rapports/conception.mdown @@ -0,0 +1,56 @@ +# Conception + +## Intrinsic calibration + +cf TP Simone Gasparini + +## Extrinsic calibration + +cf TP Simone Gasparini + +## Skeletonization + + - Input : vector string + - Ouptut : skeleton + +### Points of interest detection + + - Input : string : pictures ().jpg), (int : nb features?) + - Output : keypoints vector ( std::vector) + - function : we chose to use function SIFT (not SURF or Harris + + - tests : + - with an image, verify the right number of points is detected + +### Matching points + + - Input : keypoints vectors, calibration + - Output : keypoints vector + - function : + - tests : + - use the function to match the points between a picture and herself + - use the function with different soft toys on the first and second picture + +### Binary filter ? + + - Input : binary filter, matched points + - Ouptut : filtered matched points + - tests : + - + +### Camera filter + + +### Skeleton division + + - Input : + - Output : + + +### Skeleton matching + + - Input : + - Output : + + + diff --git a/Tests/Test.tex b/Tests/Test.tex new file mode 100644 index 0000000..296d06e --- /dev/null +++ b/Tests/Test.tex @@ -0,0 +1,647 @@ +\documentclass[a4paper,10pt]{report} +\usepackage[utf8]{inputenc} +\usepackage[top=2cm,bottom=2cm,left=2cm,right=2cm]{geometry} +\usepackage{graphicx} +\usepackage{amsmath} +\usepackage{array} +\usepackage{float} +\usepackage{xcolor} + +% Title Page +\title{Tests report} +\author{Thomas Forgione, Emilie Jalras, Marion Lenfant, Thierry Malon, Amandine Pailloux} + +\makeatletter +\@addtoreset{chapter}{part} +\makeatother + +\begin{document} +\maketitle +\tableofcontents + +\part{Camera calibration, skeletonization, detection and matching of keypoints} +\chapter{Calibration} +\section{External calibration} + +To test the external calibration, the idea is to have two pictures of a scene +containing two chessboards : one blue and one red. We will compute the +homography that transforms the blue chessboard on the first picture to the blue +chessboard of the second picture. Then, we will apply this transformation to +the red chessboard on the first picture, and if the calibration has a good +quality, the points found should be on the red chessboard. +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/externalCalibration/result.png} + \caption{Test of the external calibration} +\end{figure} +As you can see, the points computed are slightly displaced from the corners of +the chessboard. We can explain this by the difficulty we had to have a very +plane chessboard : the ink from the printer humidify the paper, and we were not +able to have a really plane chessboard. + +\chapter{Detection and matching of keypoints} +\section{Detection of keypoints} +We tried three differents algorithm to find points of interest : +\begin{itemize} + \item surf algorithm + \item sift algorithm + \item brisk algorithm +\end{itemize} +The results were similar with the three algorithms. +We kept the surf algorithm for detection as it is the fatest one but finally we decided to use brisk algorithm because surf and sift +are patented algorithm and cannot be used for commercial use. In the following results you can see the differences between surf +and brisk. +With the surf algorithm there is a parameter named minHessian which can be modified, influencing the number of detected points. +The higher the minHessian, the fewer keypoints you will obtain, on the other hand, the lower the minHessian, the more keypoints you get, but they may be more noisy. +With the brisk algorithm some parameters can be changed like the FAST/AGAST detection threshold score (thresh), the detection octaves (octaves) and +the scale to the pattern used for sampling the neighbourhood of a keypoint (patternScale). + +\subsection{Test lapin} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/pointsDetection/LapinSift} + \caption{\label{LapinSift} Lapin, Sift algorithm} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/pointsDetection/LapinSurf500} + \caption{\label{LapinSurf500} Lapin, Surf algorithm, minHessian=500} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/pointsDetection/LapinSurf1000} + \caption{\label{LapinSurf1000} Lapin, Surf algorithm, minHessian=1000} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.49]{images/pointsDetection/brisk3415} + \caption{\label{brisk3415} Lapin, Brisk, thresh=3, octaves=4, patternScale=1.5} +\end{figure} + +\newpage +\subsection{Test minion} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/pointsDetection/MignonSift} + \caption{\label{MignonSift} Minion, Sift algorithm} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/pointsDetection/MignonSurf500} + \caption{\label{} Minion, Surf algorithm, minHessian=500} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/pointsDetection/MignonSurf1000} + \caption{\label{fig} Minion, Surf algorithm, minHessian=1000} +\end{figure} + + +\section{Matching of keypoints} +\subsection{Different filters on Surf } +Different kinds of filters : +\begin{itemize} + \item symmetric constraint + \item order constraint + \item match distance filter + \item geometric filter (with Ransac algorithm) +\end{itemize} + +\begin{table}[H] +\begin{center} +\begin{tabular}{|p{3.8cm}|p{3.8cm}|p{3.8cm}|} + \hline + Filters & Description & Parameter \\ + \hline + Symmetric constraint & The symmetric constraint delivers efficient results, and does not remove too many points. This is why we will keep this filter. & None\\ + \hline + Order constraint & The order constraint does not seem relevant when used with a too high threshold. The order constraint does not seem relevant when used with a too high threshold. & proportion: represent the rate of error accepted\\ + \hline + Match distance filter & The match distance criteria seems to remove wise points but the results are less effective when removing lots of points. & threshold : represent the distance between two keypoints, between 0 and 1. \\ + \hline + Geometric filter & The geometric filter, using Ransac to find outliers, gives us back really good points, but it seems too selective. & param RANSAC : It is the maximum distance from a point to an epipolar line in pixels, beyond which the point is considered an outlier and is not used for computing the final fundamental matrix. It can be set to something like 1-3, depending on the accuracy of the point localization, image resolution, and the image noise.\\ + \hline +\end{tabular} +\caption{\label{filters}Different kinds of filters} +\end{center} +\end{table} + + +\begin{table}[H] +\begin{center} +\begin{tabular}{|c|c|c|c|} +%begin{tabular}{|p{6cm}|p{2.8cm}|p{2.8cm}|p{3.8cm}|} + \hline + Type of filter & Parameter & Number of points & Figure\\ + \hline + no cache & -- & -- & fig \ref{LapinNoCache} \\ + \hline + no filter img1 - img2 & -- & 503 & fig \ref{LapinNoFilter1-2}\\ + \hline + no filter img2 - img1 & -- & 395 & fig \ref{LapinNoFilter2-1} \\ + \hline + symmetric constraint & -- & 174 & fig \ref{LapinSymmetric}\\ + \hline + order constraint & proportion : 1 & 503 & \\ + \hline + order constraint & proportion : 1/4 & 0 & \\ + \hline + distance criteria & threshold : 0.5 & 478 & \\ + \hline + distance criteria & threshold : 0.4 & 417 & fig \ref{LapinCorrected0-4} \\ + \hline + symmetric constraint / distance criteria & threshold : 0.3 & 124 & fig \ref{LapinSymmetricCorrected0-3} \\ + \hline + geometric constraint & RANSAC param : 3 & 73 & fig \ref{LapinGeometric}\\ + \hline + symmetric constraint / geometric constraint & RANSAC param : 3 & 104 & fig \ref{LapinSymmetricGeometric}\\ + \hline + +\end{tabular} +\caption{\label{fig2} Number of points per filter} +\end{center} +\end{table} +With the geometric constraint we obtain more points when limiting to the symmetric points. +This probably comes from the calculation of the fundamental matrix with the Ransac algorithm, which is less accurate when using wrong points as input. + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/pointsMatching/LapinNoCache} + \caption{\label{LapinNoCache} No cache} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/pointsMatching/LapinNoFilter1-2} + \caption{\label{LapinNoFilter1-2} Mask but no filter img1 - img2} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/pointsMatching/LapinNoFilter2-1} + \caption{\label{LapinNoFilter2-1} Mask but no filter img2 - img1} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/pointsMatching/LapinSymmetric} + \caption{\label{LapinSymmetric} Symmetric constraint} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/pointsMatching/LapinCorrected0-4} + \caption{\label{LapinCorrected0-4} Distance criteria 0.4} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/pointsMatching/LapinSymmetricCorrected0-3} + \caption{\label{LapinSymmetricCorrected0-3} Symmetric constraint and distance criteria 0.4} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/pointsMatching/LapinGeometric} + \caption{\label{LapinGeometric} Geometric constraint} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/pointsMatching/LapinSymmetricGeometric} + \caption{\label{LapinSymmetricGeometric} Symmetric constraint and geometric constraint} +\end{figure} + +\subsection{Tests with two times the same picture} + We seem to have good results when using only the surf matching algorithm, with or without mask. + At first we obtained no points at all with the geometric filter. This test allowed us to find an error in our file + (you can see the different results wihout mask, no filter, geometric filter with and without error Figures \ref{LapinAgain}, + \ref{LapinAgainMask}, \ref{LapinAgainWrongGeometric} and \ref{LapinAgainGeometric}) + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/pointsMatching/LapinAgain} + \caption{\label{LapinAgain} No mask no filter} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/pointsMatching/LapinAgainMask} + \caption{\label{LapinAgainMask}No filter} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/pointsMatching/LapinAgainWrongGeometric} + \caption{\label{LapinAgainWrongGeometric} Geometric filter with error} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{images/pointsMatching/LapinAgainGeometric} + \caption{\label{LapinAgainGeometric} Geometric filter} +\end{figure} + +\subsection{Test using BRISK algorithm} + +We changed the parameters of brisk algorithm and of the filters to find the best +values in order to find the best matching points. The different results are +summarise in the table \ref{briskTest}. + +\begin{table}[H] +\begin{center} +\begin{tabular}{|c|c|c|c|c|c|c|} +\hline +Figure & Thresh & Octaves & PatternScales & Proportion & RANSAC & Matches number\\ +\hline +\ref{B1} & 2 & 4 & 1.5 & 1 & 3 & 55\\ +\hline +\ref{B2} & 10 & 4 & 1.5 & 1 & 3 & 43\\ +\hline +\ref{B3} & 3 & 4 & 1.5 & 1 & 3 & 57\\ +\hline +\ref{B4} & 3 & 4 & 1.0 & 1 & 3 & 31\\ +\hline +\ref{B5} & 3 & 4 & 2.0 & 1 & 3 & 53\\ +\hline +\ref{B6} & 3 & 4 & 1.5 & 1 & 5 & 66\\ +\hline +\ref{B7} & 3 & 4 & 1.5 & 0.5 & 3 & 69\\ +\hline +\end{tabular} +\caption{\label{briskTest} Test on parameters of brisk algorithm and filters} +\end{center} +\end{table} + +\paragraph{Analyses} When we raise the thresh parameter over 3 the number of matches decrease, the same thing +happened when we decrease it below 3. That is why we kept the value 3 for thresh parameter. Then, the value of +PatternScales parameter is important to adjust the number of keypoints detected and so the number of matches, +the optimal value is 1.5. Using the order constraint filter allow us to find more and better matches because +the fundamental matrix is computed from all the keypoints so if there are a lot of errors this matrix is not +well approximated and there are less good matches at the end. Finally when we change the value of RANSAC parameter +around 3 there are more errors. The final values kept for the parameters are +those corresponding to the figure \ref{B7}. + + + +\begin{figure}[H] + \centering + \includegraphics[scale=0.55]{images/pointsMatching/1} + \caption{\label{B1} BRISK} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.55]{images/pointsMatching/2} + \caption{\label{B2} BRISK} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.55]{images/pointsMatching/3} + \caption{\label{B3} BRISK} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.55]{images/pointsMatching/4} + \caption{\label{B4} BRISK} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.55]{images/pointsMatching/5} + \caption{\label{B5} BRISK} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.55]{images/pointsMatching/6} + \caption{\label{B6} BRISK : one error when we change RANSAC parameter} +\end{figure} + +\begin{figure}[H] + \centering + \includegraphics[scale=0.55]{images/pointsMatching/7} + \caption{\label{B7} BRISK : final Result} +\end{figure} + +\chapter{Skeletonization} +\section{Cutting skeleton into pieces} +To test the cutting skeleton part, we use different files ".skl". We count ourselves the number of branches which has to be found. Then, we apply the algorithm and verify that the number of branches is the same as expected. We plotted the branches with different colors: it gives an easy way to verify that there is no error with branches. To be sure of the results, numerical verifications have been made concerning the indexes of the edges composing each branch. + +The following figure shows two skeletons just before branches identification. The skeletons that we used are untitled "skullTest1.skl" and "skullTest2.skl". There are very simple as they are composed of about twenty points. We build these skeletons ourselves to be able to make unitary tests. In addition, we will see in the matching branches section that these skeletons were the only ones on which we could execute the matching algorithm. + +\begin{figure}[H] + \centering + \includegraphics[scale=0.24]{images/branchesExtraction/unbranchedSkelsTest.png} + \caption{No color has been associated before the identification of the branches. We clearly see that six branches have to be found on each skeleton.} +\end{figure} + +Then, we identify the branches starting from each junction point and going on until we find an extremity point or another junction point. It delimitates a branch. +We used the six same colors for the two skeletons to highlight the fact that the branches are not matched yet. It will be the next step. + +\begin{figure}[H] + \centering + \includegraphics[scale=0.32]{images/branchesExtraction/branchedSkelsTest.png} + \caption{The skeletons have been splitted into branches. A color has been associated to each branch.} +\end{figure} + + +\section{Matching pieces of skeleton by pairs} +The matching part revealed unexpected issues. In particular, skeletons rarely have the same number of branches on two different views. Several ways of improvement to address this issue are provided in the report. + +After we took several pictures of each plushes and applied segmentation to each, we extracted the skeletons and splitted them into branches. + +Results are quite unexpected: for a same plush photographed in the same conditions and with the same angles, the results can be different. + +On the following figure, the two first pictures are almost identical. The segmented images are also almost identical. However, on the second picture, the left leg of the rabbit is composed of an extra branch, and the right leg is composed of a branch less. The right foreleg also contains more branches on the second picture than on the first. On the third picture, the photograph was taken closer, so the plush appears bigger. The skeleton is totally different from the two others. + +These main problems occur on extremeties, but there can also be problem outside the extremities. There is a small branch separating the two arms of the rabbit on the two first pictures, whereas on the third picture, the two arms branches join on a point. + +Branches extraction reveals 30 branches on the first picture, 31 on the second picture and 21 on the third one. + +\begin{figure}[H] + \centering + \includegraphics[scale=0.38]{images/skeletonsExtraction/TestSkel00.png} + \caption{Front rabbit pictures and skeletons} +\end{figure} + +With the same plush and an other angle, the same kind of issue occur: the plush skeleton is different on each picture. Even when there seems to be the same number of branches, for example the first and the second skeletons, these branches can be different, for example on the first skeleton, the foreleg branch continues until the outline, whereas it ends earlier on the second skeleton. Third and fourth skeletons show one more time the differences near the extremities with an extra branch on the ear or a single branch foreleg. + +Having different number of branches between different pictures of the same plush with the same angle is a first problem. It leads naturally to another problem: the number of branches is different between two different pictures of the same object with different angles. Even when, by chance, there is the same amount of branches of both, these branches do not really match together because there is probably an extra branch on a limb of the first picture, and an extra branch on another limb on the second picture. + +Branches extraction reveals 16 branches on the first picture, 15 on the second picture, 16 on the third picture and 14 on the fourth one. + +\begin{figure}[H] + \centering + \includegraphics[scale=0.37]{images/skeletonsExtraction/TestSkel01.png} + \caption{Side rabbit pictures and skeletons} +\end{figure} + +We tried two other plushes. For the teddybear, branches extraction reveals 17 branches on the first picture, 14 on the second, 15 on the third and 18 on the fourth. + +\begin{figure}[H] + \centering + \includegraphics[scale=0.37]{images/skeletonsExtraction/TestSkel02.png} + \caption{Teddy bear pictures and skeletons} +\end{figure} + +For the red doll, branches extraction reveals 15 branches on the first and the second pictures, and 10 branches on the third and the fourth pictures. Even so, the first and the second skeletons cannot be matched at all. + +These tests confirm how different skeletons are. The first picture never match with the other angles. The only correct matching is between the third and the forth pictures of the red doll, but they were taken with the same angle. + +\begin{figure}[H] + \centering + \includegraphics[scale=0.42]{images/skeletonsExtraction/TestSkel03.png} + \caption{Side rabbit pictures and skeletons} +\end{figure} + +As the two skeletons do not have the same number of branches, the tests do not succeed. However, we used the same two skeletons as in the cutting skeleton part to verify that the matching branches algorithm works when it is used on two corresponding skeletons which have the same number of branches. These skeletons have been imaginated, so there is no associated picture. That is why we defined keypoints ourselves. To test the robustness of the matching branches algorithm, one of the keypoint has been designed to give a vote to a branch match between branches. + +In particular, the matching matrix between the branches of the first skeleton and the branches of the second skeleton is the expected following one: + +$$ +\begin{pmatrix} + 0 & 5 & 0 & 0 & 0 & 0 \\ + 0 & 0 & 0 & 0 & 0 & 5 \\ + 0 & 0 & 0 & 3 & 0 & 0 \\ + 0 & 0 & 3 & 0 & 0 & 0 \\ + 1 & 0 & 0 & 0 & 3 & 0 \\ + 4 & 0 & 0 & 0 & 0 & 0 +\end{pmatrix} +$$ + +For the tests, the branches of the two skeletons have not been written in the same order. In the other case, we would have had a diagonal matching matrix. The first line can be read as "five pairs of matching keypoints vote for a match between the first branch of the first skeleton and the second branch of the second skeleton". The "1" in the matrix is the outlier we put to test the robustness of the algorithm. + +Finally, the test revealed the following matches between the indexes of the branches in the first skeleton and the indexes of the branches in the second skeleton: + +$$ +\begin{pmatrix} + 0 & 1 \\ + 1 & 5 \\ + 2 & 4 \\ + 3 & 2 \\ + 4 & 3 \\ + 5 & 0 +\end{pmatrix} +$$ + +\begin{figure}[H] + \centering + \includegraphics[scale=0.3]{images/branchesExtraction/pairedSkelsTest.png} + \caption{The two sets of branches have been matched together: the colors of corresponding branches are the same.} +\end{figure} + + + +\part{Meshing and animation} +\chapter{Meshing} +\section{Splines} +The spline is a mathematical tool to create a smooth curve from a set of +points. The following test illustrates the fact that our spline is smooth, and +that the derivative of the splines are computed right. +\begin{figure}[H] + \centering + \includegraphics[scale=0.4]{images/splines/spline.png} + \caption{A spline blue and its tangent in red} +\end{figure} + +\section{Meshing the extremities} +\paragraph{} In our mesh the extremities refer to the extrem points of splines which are not part of a junction. +Meshing the extremities consists in two steps : build each extremity and subdivise it. In order to build the extremities, +we chose to link the existing meshing points along the extrem circles of each extremity with the projection of the +circle's center on the corresponding sphere. Then the subdivisions are made by projecting the middle of the previous segments (links) +on the sphere. + +\paragraph{} Some results can be seen in the following Figures (All of these extremities were built on dino.skl). The deeper the subdivision +is, the smoother the extremity's surface is. We can easily choose the depth of the subdivision to adapt it according to the skeleton +in input. + +\begin{table}[H] +\begin{center} +\begin{tabular}{|c|c|c|} +%begin{tabular}{|p{6cm}|p{2.8cm}|p{2.8cm}|p{3.8cm}|} + \hline + Depth of subdivision & Figure & Description\\ + \hline + 0 & \ref{0} & No subdivision : filled mesh\\ + \hline + 0 & \ref{0ed} & No subdivision : edges\\ + \hline + 1 & \ref{1} & One subdivision : filled mesh\\ + \hline + 1 & \ref{1ed} & One subdivision : edges\\ + \hline + 2 & \ref{meshextremity} & Triangular faces for two subdivisions\\ + \hline + 2 & \ref{2} & Two subdivisions : filled mesh\\ + \hline + 2 & \ref{2ed} & Two subdivisions : edges\\ + \hline + 10 & \ref{10} & Ten subdivisions : filled mesh\\ + \hline + 10 & \ref{10ed} & Ten subdivisions : edges\\ + \hline +\end{tabular} +\caption{\label{tblfigures} Tests on extremities} +\end{center} +\end{table} + + +\begin{figure}[H] + \centering + \includegraphics[scale=0.4]{images/extremity/meshextremity} + \caption{\label{meshextremity} 2 subdivisions on an extremity} +\end{figure} + +\begin{figure}[H] +\begin{minipage}[b]{0.45\linewidth} + \centering + \includegraphics[scale=0.35]{images/extremity/0} + \caption{\label{0} No subdivision : filled mesh} + \end{minipage} +\hspace{0.5cm} +\begin{minipage}[b]{0.45\linewidth} + \centering + \includegraphics[scale=0.35]{images/extremity/0ed} + \caption{\label{0ed} No subdivision : edges} + \end{minipage} +\end{figure} + +\begin{figure}[H] +\begin{minipage}[b]{0.45\linewidth} + \centering + \includegraphics[scale=0.35]{images/extremity/1} + \caption{\label{1} One subdivision : filled mesh} + \end{minipage} +\hspace{0.5cm} +\begin{minipage}[b]{0.45\linewidth} + \centering + \includegraphics[scale=0.35]{images/extremity/1ed} + \caption{\label{1ed} One subdivision : edges} + \end{minipage} +\end{figure} + +\begin{figure}[H] +\begin{minipage}[b]{0.45\linewidth} + \centering + \includegraphics[scale=0.35]{images/extremity/2} + \caption{\label{2} Two subdivisions : filled mesh} + \end{minipage} +\hspace{0.5cm} +\begin{minipage}[b]{0.45\linewidth} + \centering + \includegraphics[scale=0.35]{images/extremity/2ed} + \caption{\label{2ed} Two subdivisions : edges} + \end{minipage} +\end{figure} + +\begin{figure}[H] +\begin{minipage}[b]{0.45\linewidth} + \centering + \includegraphics[scale=0.35]{images/extremity/10} + \caption{\label{10} Ten subdivisions : filled mesh} + \end{minipage} +\hspace{0.5cm} +\begin{minipage}[b]{0.45\linewidth} + \centering + \includegraphics[scale=0.35]{images/extremity/10ed} + \caption{\label{10ed} Ten subdivisions : edges} + \end{minipage} +\end{figure} + + + +\section{Meshing the junctions} + +\paragraph{} Meshing junctions was the most difficult part of the meshing. +It is due to the fact that it depends a lot on the 3D-skeleton given in input and particularly quality of it in line with the theory. +In fact to give a good result it needs to respect some rules that are not completely respected because of the numbers' precision possible on computers and the approximation that come with it. +In theory to mesh a junction we get the last circle of each spline that join on that point and then we cut the circles in two and mesh the upper points of the circles with the center of the sphere corresponding to the junction projected on the up side of the sphere, idem for the down points. This is possible because consecutive circles are tangent. In reality circles are not tangent there is a little shift or they crossed themselves and we need to link the circles between them. +: +\paragraph{} However the circles need to be apart to have a good looking junction and in the real skeletons that we have it is not the case. +That is why we have created some simple examples of junction with three or four circles to join to be sure that our algorithm is working, it is easier to see than in a complete skeleton with the rest of the mesh. + +Here \ref{3junction} you can see the result with 3 circles not tangent mesh with our algorithm. +\begin{figure}[H] + \centering + \includegraphics[scale=0.3]{images/junctions/img/test1/capture0.png} + \includegraphics[scale=0.3]{images/junctions/img/test1/capture1.png} + \includegraphics[scale=0.3]{images/junctions/img/test1/capture3.png} + \caption{\label{3junction}Junction with three parts to join show from different angles} +\end{figure} + +Because junctions are not always between three branches but can be between four or more branches, we need to have a algorithm that work for all possible cases. Here \ref{4junction} the result of a 4-branches' junction meshing. + +\begin{figure}[H] + \centering + \includegraphics[scale=0.25]{images/junctions/junction4/capture0.png} + \includegraphics[scale=0.25]{images/junctions/junction4/capture1.png} + \includegraphics[scale=0.15]{images/junctions/junction4/capture2.png} + \caption{\label{4junction}Junction with four parts to join show from different angles} +\end{figure} + +With the skeleton made by ourself the geometrical properties that allow the circles to join without crossing themselves is not respected that is why the result is not pretty good. So the results obtained are not particularly good looking but respect the algorithm. + +\begin{figure}[H] + \centering + \colorbox{black}{ + \includegraphics[scale=0.1]{images/junctions/test2/capture1.png} + \includegraphics[scale=0.1]{images/junctions/test2/capture2.png} + } + \caption{Result of meshing junction on Dino.skl} +\end{figure} + +And last, we have test our junction's algorithm with a 3D-skeleton computed from a real 2D skeleton. Because of the complexity of the 2D-skeleton it is not easy to see properly the junction. + +\begin{figure}[H] + \centering + \colorbox{black}{ + \includegraphics[scale=0.1]{images/junctions/test3/capture0.png} + \includegraphics[scale=0.1]{images/junctions/test3/capture1.png} + } + \caption{Result of meshing junction on a 3D skeleton computed from a 2D skeleton} +\end{figure} + + +\chapter{Animation} +\section{Clicks} +To test this part, we clicked around the skeleton. We can see that the point +added is the one which is the closest point of the spline. We can also see that +the points are in the correct order. +\begin{figure}[H] + \centering + \includegraphics[scale=0.3]{images/clicks/click1.png} + \includegraphics[scale=0.315]{images/clicks/click2.png} + \caption{Clicks} +\end{figure} + +\section{Match vertices of the mesh with the closest branch} +The test of the matching between the vertices and the segments of the skeleton +is tested by using colors. All the vertices associated to the same segment +share the same color, and the following image shows that it works correctly +(especially the gradient of colors near the rotation points). +\begin{figure}[H] + \centering + \includegraphics[scale=0.15]{images/colormesh/capture.png} + \caption{Matching of the vertices of the mesh to the segments} +\end{figure} + +\section{Renderer} +This part is only here to show the quality of the renderer and the animation, +and especially the junctions. +\begin{figure}[H] + \centering + \includegraphics[scale=0.15]{images/colormesh/animated.png} + \caption{Matching of the vertices of the mesh to the segments} +\end{figure} + +\end{document} diff --git a/Tests/images/branchesExtraction/branchedSkelsTest.png b/Tests/images/branchesExtraction/branchedSkelsTest.png new file mode 100644 index 0000000..acad1a6 Binary files /dev/null and b/Tests/images/branchesExtraction/branchedSkelsTest.png differ diff --git a/Tests/images/branchesExtraction/pairedSkelsTest.png b/Tests/images/branchesExtraction/pairedSkelsTest.png new file mode 100644 index 0000000..b96fb9a Binary files /dev/null and b/Tests/images/branchesExtraction/pairedSkelsTest.png differ diff --git a/Tests/images/branchesExtraction/unbranchedSkelsTest.png b/Tests/images/branchesExtraction/unbranchedSkelsTest.png new file mode 100644 index 0000000..06b1e61 Binary files /dev/null and b/Tests/images/branchesExtraction/unbranchedSkelsTest.png differ diff --git a/Tests/images/clicks/click1.png b/Tests/images/clicks/click1.png new file mode 100644 index 0000000..03379f5 Binary files /dev/null and b/Tests/images/clicks/click1.png differ diff --git a/Tests/images/clicks/click2.png b/Tests/images/clicks/click2.png new file mode 100644 index 0000000..ff34f68 Binary files /dev/null and b/Tests/images/clicks/click2.png differ diff --git a/Tests/images/colormesh/animated.png b/Tests/images/colormesh/animated.png new file mode 100644 index 0000000..eb4a2bd Binary files /dev/null and b/Tests/images/colormesh/animated.png differ diff --git a/Tests/images/colormesh/capture.png b/Tests/images/colormesh/capture.png new file mode 100644 index 0000000..bc02765 Binary files /dev/null and b/Tests/images/colormesh/capture.png differ diff --git a/Tests/images/colormesh/capture.png.bak b/Tests/images/colormesh/capture.png.bak new file mode 100644 index 0000000..bc02765 Binary files /dev/null and b/Tests/images/colormesh/capture.png.bak differ diff --git a/Tests/images/externalCalibration/result.png b/Tests/images/externalCalibration/result.png new file mode 100644 index 0000000..693ab17 Binary files /dev/null and b/Tests/images/externalCalibration/result.png differ diff --git a/Tests/images/extremity/0.jpg b/Tests/images/extremity/0.jpg new file mode 100644 index 0000000..4f2fabf Binary files /dev/null and b/Tests/images/extremity/0.jpg differ diff --git a/Tests/images/extremity/0ed.jpg b/Tests/images/extremity/0ed.jpg new file mode 100644 index 0000000..53b7a1b Binary files /dev/null and b/Tests/images/extremity/0ed.jpg differ diff --git a/Tests/images/extremity/1.jpg b/Tests/images/extremity/1.jpg new file mode 100644 index 0000000..ae13c54 Binary files /dev/null and b/Tests/images/extremity/1.jpg differ diff --git a/Tests/images/extremity/10.jpg b/Tests/images/extremity/10.jpg new file mode 100644 index 0000000..ec8f439 Binary files /dev/null and b/Tests/images/extremity/10.jpg differ diff --git a/Tests/images/extremity/10ed.jpg b/Tests/images/extremity/10ed.jpg new file mode 100644 index 0000000..02ebbf4 Binary files /dev/null and b/Tests/images/extremity/10ed.jpg differ diff --git a/Tests/images/extremity/1ed.jpg b/Tests/images/extremity/1ed.jpg new file mode 100644 index 0000000..29ab682 Binary files /dev/null and b/Tests/images/extremity/1ed.jpg differ diff --git a/Tests/images/extremity/2.jpg b/Tests/images/extremity/2.jpg new file mode 100644 index 0000000..1ae0df0 Binary files /dev/null and b/Tests/images/extremity/2.jpg differ diff --git a/Tests/images/extremity/2ed.jpg b/Tests/images/extremity/2ed.jpg new file mode 100644 index 0000000..d190e06 Binary files /dev/null and b/Tests/images/extremity/2ed.jpg differ diff --git a/Tests/images/extremity/meshextremity.jpg b/Tests/images/extremity/meshextremity.jpg new file mode 100644 index 0000000..13babf3 Binary files /dev/null and b/Tests/images/extremity/meshextremity.jpg differ diff --git a/Tests/images/junctions/img/capture1.png b/Tests/images/junctions/img/capture1.png new file mode 100644 index 0000000..5549aa0 Binary files /dev/null and b/Tests/images/junctions/img/capture1.png differ diff --git a/Tests/images/junctions/img/test1/capture0.png b/Tests/images/junctions/img/test1/capture0.png new file mode 100644 index 0000000..b8913a2 Binary files /dev/null and b/Tests/images/junctions/img/test1/capture0.png differ diff --git a/Tests/images/junctions/img/test1/capture1.png b/Tests/images/junctions/img/test1/capture1.png new file mode 100644 index 0000000..87f73b5 Binary files /dev/null and b/Tests/images/junctions/img/test1/capture1.png differ diff --git a/Tests/images/junctions/img/test1/capture3.png b/Tests/images/junctions/img/test1/capture3.png new file mode 100644 index 0000000..bcea776 Binary files /dev/null and b/Tests/images/junctions/img/test1/capture3.png differ diff --git a/Tests/images/junctions/junction4/capture0.png b/Tests/images/junctions/junction4/capture0.png new file mode 100644 index 0000000..1f3df5a Binary files /dev/null and b/Tests/images/junctions/junction4/capture0.png differ diff --git a/Tests/images/junctions/junction4/capture1.png b/Tests/images/junctions/junction4/capture1.png new file mode 100644 index 0000000..d1818ae Binary files /dev/null and b/Tests/images/junctions/junction4/capture1.png differ diff --git a/Tests/images/junctions/junction4/capture2.png b/Tests/images/junctions/junction4/capture2.png new file mode 100644 index 0000000..9474b52 Binary files /dev/null and b/Tests/images/junctions/junction4/capture2.png differ diff --git a/Tests/images/junctions/test1/capture0.png b/Tests/images/junctions/test1/capture0.png new file mode 100644 index 0000000..cf43ac6 Binary files /dev/null and b/Tests/images/junctions/test1/capture0.png differ diff --git a/Tests/images/junctions/test2/capture0.png b/Tests/images/junctions/test2/capture0.png new file mode 100644 index 0000000..a9bf6f3 Binary files /dev/null and b/Tests/images/junctions/test2/capture0.png differ diff --git a/Tests/images/junctions/test2/capture1.png b/Tests/images/junctions/test2/capture1.png new file mode 100644 index 0000000..c8db247 Binary files /dev/null and b/Tests/images/junctions/test2/capture1.png differ diff --git a/Tests/images/junctions/test2/capture2.png b/Tests/images/junctions/test2/capture2.png new file mode 100644 index 0000000..cc1e7fa Binary files /dev/null and b/Tests/images/junctions/test2/capture2.png differ diff --git a/Tests/images/junctions/test3/capture0.png b/Tests/images/junctions/test3/capture0.png new file mode 100644 index 0000000..26d7c5f Binary files /dev/null and b/Tests/images/junctions/test3/capture0.png differ diff --git a/Tests/images/junctions/test3/capture1.png b/Tests/images/junctions/test3/capture1.png new file mode 100644 index 0000000..85c8a27 Binary files /dev/null and b/Tests/images/junctions/test3/capture1.png differ diff --git a/Tests/images/pointsDetection/LapinSift.PNG b/Tests/images/pointsDetection/LapinSift.PNG new file mode 100644 index 0000000..99774b1 Binary files /dev/null and b/Tests/images/pointsDetection/LapinSift.PNG differ diff --git a/Tests/images/pointsDetection/LapinSurf100.PNG b/Tests/images/pointsDetection/LapinSurf100.PNG new file mode 100644 index 0000000..5f78cf2 Binary files /dev/null and b/Tests/images/pointsDetection/LapinSurf100.PNG differ diff --git a/Tests/images/pointsDetection/LapinSurf1000.PNG b/Tests/images/pointsDetection/LapinSurf1000.PNG new file mode 100644 index 0000000..46c1ce1 Binary files /dev/null and b/Tests/images/pointsDetection/LapinSurf1000.PNG differ diff --git a/Tests/images/pointsDetection/LapinSurf10000.PNG b/Tests/images/pointsDetection/LapinSurf10000.PNG new file mode 100644 index 0000000..f58967d Binary files /dev/null and b/Tests/images/pointsDetection/LapinSurf10000.PNG differ diff --git a/Tests/images/pointsDetection/LapinSurf500.PNG b/Tests/images/pointsDetection/LapinSurf500.PNG new file mode 100644 index 0000000..7aa1f66 Binary files /dev/null and b/Tests/images/pointsDetection/LapinSurf500.PNG differ diff --git a/Tests/images/pointsDetection/MignonSift.PNG b/Tests/images/pointsDetection/MignonSift.PNG new file mode 100644 index 0000000..3bd3938 Binary files /dev/null and b/Tests/images/pointsDetection/MignonSift.PNG differ diff --git a/Tests/images/pointsDetection/MignonSurf100.PNG b/Tests/images/pointsDetection/MignonSurf100.PNG new file mode 100644 index 0000000..dc73ac6 Binary files /dev/null and b/Tests/images/pointsDetection/MignonSurf100.PNG differ diff --git a/Tests/images/pointsDetection/MignonSurf1000.PNG b/Tests/images/pointsDetection/MignonSurf1000.PNG new file mode 100644 index 0000000..c05d596 Binary files /dev/null and b/Tests/images/pointsDetection/MignonSurf1000.PNG differ diff --git a/Tests/images/pointsDetection/MignonSurf10000.PNG b/Tests/images/pointsDetection/MignonSurf10000.PNG new file mode 100644 index 0000000..bde787f Binary files /dev/null and b/Tests/images/pointsDetection/MignonSurf10000.PNG differ diff --git a/Tests/images/pointsDetection/MignonSurf500.PNG b/Tests/images/pointsDetection/MignonSurf500.PNG new file mode 100644 index 0000000..f38b3ac Binary files /dev/null and b/Tests/images/pointsDetection/MignonSurf500.PNG differ diff --git a/Tests/images/pointsDetection/ThierrySift.PNG b/Tests/images/pointsDetection/ThierrySift.PNG new file mode 100644 index 0000000..de4699c Binary files /dev/null and b/Tests/images/pointsDetection/ThierrySift.PNG differ diff --git a/Tests/images/pointsDetection/ThierrySurf100.PNG b/Tests/images/pointsDetection/ThierrySurf100.PNG new file mode 100644 index 0000000..072c74f Binary files /dev/null and b/Tests/images/pointsDetection/ThierrySurf100.PNG differ diff --git a/Tests/images/pointsDetection/brisk3415.png b/Tests/images/pointsDetection/brisk3415.png new file mode 100644 index 0000000..0bee192 Binary files /dev/null and b/Tests/images/pointsDetection/brisk3415.png differ diff --git a/Tests/images/pointsMatching/1.png b/Tests/images/pointsMatching/1.png new file mode 100644 index 0000000..11dfa5c Binary files /dev/null and b/Tests/images/pointsMatching/1.png differ diff --git a/Tests/images/pointsMatching/2.png b/Tests/images/pointsMatching/2.png new file mode 100644 index 0000000..f6314d4 Binary files /dev/null and b/Tests/images/pointsMatching/2.png differ diff --git a/Tests/images/pointsMatching/3.png b/Tests/images/pointsMatching/3.png new file mode 100644 index 0000000..59e92b1 Binary files /dev/null and b/Tests/images/pointsMatching/3.png differ diff --git a/Tests/images/pointsMatching/4.png b/Tests/images/pointsMatching/4.png new file mode 100644 index 0000000..82369dd Binary files /dev/null and b/Tests/images/pointsMatching/4.png differ diff --git a/Tests/images/pointsMatching/5.png b/Tests/images/pointsMatching/5.png new file mode 100644 index 0000000..570a668 Binary files /dev/null and b/Tests/images/pointsMatching/5.png differ diff --git a/Tests/images/pointsMatching/6.png b/Tests/images/pointsMatching/6.png new file mode 100644 index 0000000..4df461d Binary files /dev/null and b/Tests/images/pointsMatching/6.png differ diff --git a/Tests/images/pointsMatching/7.png b/Tests/images/pointsMatching/7.png new file mode 100644 index 0000000..ee57c72 Binary files /dev/null and b/Tests/images/pointsMatching/7.png differ diff --git a/Tests/images/pointsMatching/LapinAgain.PNG b/Tests/images/pointsMatching/LapinAgain.PNG new file mode 100644 index 0000000..f902c21 Binary files /dev/null and b/Tests/images/pointsMatching/LapinAgain.PNG differ diff --git a/Tests/images/pointsMatching/LapinAgainGeometric.PNG b/Tests/images/pointsMatching/LapinAgainGeometric.PNG new file mode 100644 index 0000000..1f5c2bf Binary files /dev/null and b/Tests/images/pointsMatching/LapinAgainGeometric.PNG differ diff --git a/Tests/images/pointsMatching/LapinAgainMask.PNG b/Tests/images/pointsMatching/LapinAgainMask.PNG new file mode 100644 index 0000000..03f7385 Binary files /dev/null and b/Tests/images/pointsMatching/LapinAgainMask.PNG differ diff --git a/Tests/images/pointsMatching/LapinAgainWrongGeometric.PNG b/Tests/images/pointsMatching/LapinAgainWrongGeometric.PNG new file mode 100644 index 0000000..0893336 Binary files /dev/null and b/Tests/images/pointsMatching/LapinAgainWrongGeometric.PNG differ diff --git a/Tests/images/pointsMatching/LapinCorrected0-4.PNG b/Tests/images/pointsMatching/LapinCorrected0-4.PNG new file mode 100644 index 0000000..d6b9536 Binary files /dev/null and b/Tests/images/pointsMatching/LapinCorrected0-4.PNG differ diff --git a/Tests/images/pointsMatching/LapinCorrected0-5.PNG b/Tests/images/pointsMatching/LapinCorrected0-5.PNG new file mode 100644 index 0000000..b98fa4e Binary files /dev/null and b/Tests/images/pointsMatching/LapinCorrected0-5.PNG differ diff --git a/Tests/images/pointsMatching/LapinGeometric.PNG b/Tests/images/pointsMatching/LapinGeometric.PNG new file mode 100644 index 0000000..3e32400 Binary files /dev/null and b/Tests/images/pointsMatching/LapinGeometric.PNG differ diff --git a/Tests/images/pointsMatching/LapinNoCache.PNG b/Tests/images/pointsMatching/LapinNoCache.PNG new file mode 100644 index 0000000..26b0a8f Binary files /dev/null and b/Tests/images/pointsMatching/LapinNoCache.PNG differ diff --git a/Tests/images/pointsMatching/LapinNoFilter.PNG b/Tests/images/pointsMatching/LapinNoFilter.PNG new file mode 100644 index 0000000..a6226e6 Binary files /dev/null and b/Tests/images/pointsMatching/LapinNoFilter.PNG differ diff --git a/Tests/images/pointsMatching/LapinNoFilter1-2.PNG b/Tests/images/pointsMatching/LapinNoFilter1-2.PNG new file mode 100644 index 0000000..7f8b84b Binary files /dev/null and b/Tests/images/pointsMatching/LapinNoFilter1-2.PNG differ diff --git a/Tests/images/pointsMatching/LapinNoFilter2-1.PNG b/Tests/images/pointsMatching/LapinNoFilter2-1.PNG new file mode 100644 index 0000000..6e5bf03 Binary files /dev/null and b/Tests/images/pointsMatching/LapinNoFilter2-1.PNG differ diff --git a/Tests/images/pointsMatching/LapinOrder1.PNG b/Tests/images/pointsMatching/LapinOrder1.PNG new file mode 100644 index 0000000..1d02781 Binary files /dev/null and b/Tests/images/pointsMatching/LapinOrder1.PNG differ diff --git a/Tests/images/pointsMatching/LapinSymmetric.PNG b/Tests/images/pointsMatching/LapinSymmetric.PNG new file mode 100644 index 0000000..3bb5781 Binary files /dev/null and b/Tests/images/pointsMatching/LapinSymmetric.PNG differ diff --git a/Tests/images/pointsMatching/LapinSymmetricCorrected0-3.PNG b/Tests/images/pointsMatching/LapinSymmetricCorrected0-3.PNG new file mode 100644 index 0000000..c2ce319 Binary files /dev/null and b/Tests/images/pointsMatching/LapinSymmetricCorrected0-3.PNG differ diff --git a/Tests/images/pointsMatching/LapinSymmetricGeometric.PNG b/Tests/images/pointsMatching/LapinSymmetricGeometric.PNG new file mode 100644 index 0000000..7242a3d Binary files /dev/null and b/Tests/images/pointsMatching/LapinSymmetricGeometric.PNG differ diff --git a/Tests/images/pointsMatching/consideredKeypoints.PNG b/Tests/images/pointsMatching/consideredKeypoints.PNG new file mode 100644 index 0000000..f504b1e Binary files /dev/null and b/Tests/images/pointsMatching/consideredKeypoints.PNG differ diff --git a/Tests/images/skeletonsExtraction/TestSkel00.png b/Tests/images/skeletonsExtraction/TestSkel00.png new file mode 100644 index 0000000..e5ca139 Binary files /dev/null and b/Tests/images/skeletonsExtraction/TestSkel00.png differ diff --git a/Tests/images/skeletonsExtraction/TestSkel01.png b/Tests/images/skeletonsExtraction/TestSkel01.png new file mode 100644 index 0000000..bab6ec6 Binary files /dev/null and b/Tests/images/skeletonsExtraction/TestSkel01.png differ diff --git a/Tests/images/skeletonsExtraction/TestSkel02.png b/Tests/images/skeletonsExtraction/TestSkel02.png new file mode 100644 index 0000000..9a8e9d5 Binary files /dev/null and b/Tests/images/skeletonsExtraction/TestSkel02.png differ diff --git a/Tests/images/skeletonsExtraction/TestSkel03.png b/Tests/images/skeletonsExtraction/TestSkel03.png new file mode 100644 index 0000000..88f367b Binary files /dev/null and b/Tests/images/skeletonsExtraction/TestSkel03.png differ diff --git a/Tests/images/splines/spline.png b/Tests/images/splines/spline.png new file mode 100755 index 0000000..7097e0c Binary files /dev/null and b/Tests/images/splines/spline.png differ diff --git a/report/annexes/skeletons.tex b/report/annexes/skeletons.tex new file mode 100644 index 0000000..3048ae9 --- /dev/null +++ b/report/annexes/skeletons.tex @@ -0,0 +1,139 @@ +\chapter{\label{AppendixA}Skeleton examples} + +A 2D-skeleton : list of vertices and edges. +\begin{lstlisting} + v 200 300 1 + v 300 400 1 + v 300 500 1 + v 500 600 1 + v 600 500 1 + v 800 400 1 + v 900 300 1 + v 500 800 1 + v 600 900 1 + v 600 1000 1 + v 400 1000 1 + v 200 1100 1 + v 800 1100 1 + v 900 1300 1 + v 500 1200 1 + v 500 1300 1 + v 400 1400 1 + + e 1 2 + e 2 3 + e 3 4 + e 4 5 + e 5 6 + e 6 7 + e 4 8 + e 8 9 + e 9 10 + e 10 11 + e 11 12 + e 10 13 + e 13 14 + e 10 15 + e 15 16 + e 16 17 +\end{lstlisting} + +\newpage + +\noindent A 3D-skeleton made with splines +\begin{lstlisting} +d 3 + +n 0 0 0 0 0.333333 0.666667 1 1 1 1 + +p 0.60 0.43 0 0.12 +p 0.60 0.89 0 0.115 +p 0.72 1.28 0 0.11 +p 0.96 1.72 0 0.118 +p 1.27 2.10 0 0.122 +p 1.69 2.29 0 0.12 + +d 3 + +n 0 0 0 0 0.25 0.5 0.75 1 1 1 1 + +p 2.80 0.33 0 0.12 +p 2.78 0.67 0 0.116 +p 2.68 1.01 0 0.111 +p 2.52 1.49 0 0.117 +p 2.33 1.83 0 0.119 +p 2.05 2.11 0 0.121 +p 1.69 2.29 0 0.12 + +d 3 + +n 0 0 0 0 0.5 1 1 1 1 + +p 1.69 2.29 0 0.12 +p 1.70 2.44 0 0.125 +p 1.70 2.56 0 0.129 +p 1.71 2.72 0 0.127 +p 1.73 2.89 0 0.125 + +d 3 + +n 0 0 0 0 0.125 0.25 0.375 0.5 0.625 0.75 0.875 1 1 1 1 + +p 1.73 2.89 0 0.125 +p 2.06 2.78 0 0.128 +p 2.52 2.70 0 0.13 +p 3.18 2.69 0 0.132 +p 3.44 2.78 0 0.129 +p 3.62 2.93 0 0.126 +p 3.67 3.16 0 0.111 +p 3.63 3.32 0 0.098 +p 3.44 3.46 0 0.071 +p 3.14 3.47 0 0.036 +p 3.01 3.38 0 0.011 + +d 3 + +n 0 0 0 0 0.5 1 1 1 1 + +p 1.73 2.89 0 0.125 +p 1.73 3.18 0 0.131 +p 1.73 3.53 0 0.136 +p 1.72 3.84 0 0.141 +p 1.72 4.28 0 0.138 + +d 3 + +n 0 0 0 0 0.333333 0.666667 1 1 1 1 + +p 1.72 4.28 0 0.138 +p 1.43 4.25 0 0.112 +p 1.19 4.16 0 0.101 +p 0.92 3.98 0 0.094 +p 0.71 3.81 0 0.088 +p 0.56 3.56 0 0.075 + +d 3 + +n 0 0 0 0 0.333333 0.666667 1 1 1 1 + +p 1.72 4.28 0 0.138 +p 1.95 4.25 0 0.115 +p 2.19 4.20 0 0.103 +p 2.43 4.10 0 0.096 +p 2.62 3.97 0 0.090 +p 2.79 3.67 0 0.077 + +d 3 + +n 0 0 0 0 1 1 1 1 + +p 1.72 4.28 0 0.138 +p 1.71 4.53 0 0.155 +p 1.67 4.89 0 0.158 +p 1.68 5.12 0 0.135 + +l 3 1 2 3 1.0 1.0 0.0 +l 3 3 4 5 1.0 0.0 0.0 +l 4 5 6 7 8 1.0 0.0 0.0 0.0 +\end{lstlisting} + diff --git a/report/chapters/actions.tex b/report/chapters/actions.tex new file mode 100644 index 0000000..37f4bc4 --- /dev/null +++ b/report/chapters/actions.tex @@ -0,0 +1,88 @@ + +\chapter{Actions} + +In order to avoid the realization of the identified risks, we applied some of our preventive actions. + +\section{Initial actions table} +\begin{table}[h!] +\begin{center} +\begin{tabular}{|p{0.5cm}|p{3cm}|p{3cm}|p{2cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}|p{1cm}|} +\hline + Id & Origin & Description & Accountable & Target date & Realisation & State \\ + \hline + 1 & Lack of knowledge from the developer & C++ training & Thomas & 22 Jan & 23 Jan & Closed \\ + \hline + 2 & Team work \& version management & Git repository creation & Thomas & 05 Feb & 05 Feb & Closed \\ + \hline + 3 & Input needed & Taking pictures & Thierry & 22 Jan & 22 Jan & Closed \\ + \hline + 4 & Report to write in english & Specifications translation & Emilie & 22 Jan & 22 Jan & Closed \\ + \hline + 5 & Set up the dev tools & Install Linux / OpenCV & Thomas & 01 Feb & 02 Feb & Closed \\ + \hline + 6 & Remember stuff to do & Actions table writing & Marion & 05 Feb & 04 Feb & Closed \\ + \hline + 7 & Git Repository not well organized & Git cleaning & Thomas & 05 Feb & & Open \\ + \hline + 8 & Compatibility with the other team & Defining subskeletons format & Thierry & 06 Feb & & Open \\ + \hline + 9 & We love sushi & Sushi eating with the client & Thierry & 13 Feb & & Open \\ + \hline +\end{tabular} +\end{center} +\caption{Initial actions table} +\end{table} + +\section{Final actions table} +\begin{table}[h!] +\begin{tabular}{|p{0.5cm}|p{3cm}|p{3cm}|p{2cm}|p{1.5cm}|p{1.5cm}|p{1.5cm}|p{1cm}|} +\hline + Id & Origin & Description & Accountable & Target date & Realisation & State & Risk table\\ +\hline + 1 & Project management & Meeting with the industrial supervisor & Thomas & 15 Jan & 15 Jan & Closed & \\ + \hline + 2 & Need of specifications & Meeting with the client & Thierry & 22 Jan & 22 Jan & Closed & \\ + \hline + 3 & Lack of knowledge from the developer & C++ training & Thomas & 22 Jan & 23 Jan & Closed & \\ + \hline + 4 & Team work \& version management & Git repository creation & Thomas & 05 Feb & 05 Feb & Closed & \\ + \hline + 5 & Project management & Meeting with the industrial supervisor & Thierry & 21 Jan & 21 Jan & Closed & \\ + \hline + 6 & Input needed & Taking pictures & Thierry & 22 Jan & 22 Jan & Closed & \\ + \hline + 7 & Report to write in english & Specifications translation & Emilie & 22 Jan & 22 Jan & Closed & \\ + \hline + 8 & Set up the dev tools & Install Linux / OpenCV & Thomas & 01 Feb & 02 Feb & Closed & \\ + \hline + 9 & Need of specifications & Meeting with the client & Thierry & 04 Feb & 04 Feb & Closed & \\ + \hline + 10 & Remember stuff to do & Actions table writing & Marion & 05 Feb & 04 Feb & Closed & \\ + \hline + 11 & Project management & Meeting with the industrial supervisor & Marion & 5 Feb & 5 Feb & Closed & \\ + \hline + 12 & Git Repository not well organized & Git cleaning & Thomas & 05 Feb & 05 Feb & Closed & \\ + \hline + 13 & Compatibility with the other team & Defining subskeletons format & Thierry & 06 Feb & 09 Feb & CLosed & \\ +\hline +14 & Need of specifications & Meeting with the client & Thierry & 11 Feb & 11 Feb & Closed & \\ + \hline + 15 & The other group did'nt deliver binaries in time & Reschedule planning & Marion & 11 Feb & 11 Feb & Closed & 2 \\ + \hline + 16 & Binaries provided by he client are not compatible & Speak with the client & Thomas & 11 Feb & & Open & 4 \\ + \hline + 17 & Client new need & Contact the client by mail to know what we need to do & Thierry & 12 Feb & 12 Feb & Closed & 11 \\ + \hline + 18 & Project management & Meeting with the industrial supervisor & Amandine & 12 Feb & 12 Feb & Closed & \\ + \hline + 19 & Integration to do again & discuss with the client & Thierry & 25 Feb & 25 Feb & Closed &\\ + \hline + 20 & Project management & Meeting with the industrial supervisor & Emilie & 24 Feb & 24 Feb & Closed & \\ + \hline + 21 & Need of specifications & Meeting with the client & Thierry & 25 Feb & 25 Feb & Closed & \\ + \hline + 22 & The client need CMake to compile the code & Use CMake compilation & Thomas & 27 Feb & 27 Feb & Closed & \\ + \hline +\end{tabular} +\caption{Actions table on February 12th} +\end{table} \ No newline at end of file diff --git a/report/chapters/conclusion.tex b/report/chapters/conclusion.tex new file mode 100644 index 0000000..128d7a0 --- /dev/null +++ b/report/chapters/conclusion.tex @@ -0,0 +1,9 @@ +\chapter*{Conclusion} + + This project was a useful experience for us. It allowed us to better understand the usefulness and efficiency of project management. Having a schedule, and anticipating risks with our risks table allowed us to enter this project more serenely, and to finish it in time. + + The main difficulty we had to face happened during the first part of our project. The first part was shared between us and an other "Projet long" team. Some parts were also provided by the client. Unfortunately problems happened with some of his binaries, and the integration had to be abandoned. However the client was still satisfied with the results on this first part, since as a researcher it showed him problems to work on. + + The technical part of the project was very interesting, we learned a lot about C++ programming. We obtained good results on the second part of the project, which produces interesting animations. + + To conclude, this project was an opportunity for us to apply management on a real project and an interesting technical experience. \ No newline at end of file diff --git a/report/chapters/frontPage.tex b/report/chapters/frontPage.tex new file mode 100644 index 0000000..01d29a0 --- /dev/null +++ b/report/chapters/frontPage.tex @@ -0,0 +1,47 @@ +\begin{titlepage} + \begin{sffamily} + \begin{center} + ~\\[2cm] + \textsc{\LARGE Projet long report}\\[3cm] + + % Title + \HRule \\[0.4cm] + { \huge \bfseries Segmentation, Skeleton and Animation\\[0.4cm] } + + \HRule \\[3cm] + + \includegraphics[scale=0.5]{img/n7.png}~\\[2cm] + \end{center} + + + % Author and supervisor + \begin{minipage}{0.5\textwidth} + \begin{flushleft} \large + Thomas \textsc{Forgione}\\ + Emilie \textsc{Jalras}\\ + Marion \textsc{Lenfant}\\ + Thierry \textsc{Malon}\\ + Amandine \textsc{Pailloux}\\ + \end{flushleft} + \end{minipage} + \begin{minipage}{0.5\textwidth} + \begin{flushright} \large + \emph{Client :} \\ + M. Bastien \textsc{Durix}\\ + Ms G\'{e}raldine \textsc{Morin}\\ + Ms Sylvie \textsc{Chambon}\\ + \emph{Industrial supervisor :}\\ + Ms R\'{e}gine \textsc{Nigris} + + \end{flushright} + \end{minipage} + + \vfill + + % Bottom of the page + {\large January 19th 2015 -- March 13th 2015} + \end{sffamily} +\end{titlepage} + +\tableofcontents +\newpage \ No newline at end of file diff --git a/report/chapters/introManagement.tex b/report/chapters/introManagement.tex new file mode 100644 index 0000000..cd1b959 --- /dev/null +++ b/report/chapters/introManagement.tex @@ -0,0 +1,10 @@ +\chapter*{Management introduction} + + The second important aspect of our project was its management. + + Indeed project management will soon be part of our everyday life in our future work. This is why becoming familiar with the management tools studied in class is important and the "Projet Long" was a great opportunity to put it into practice. + + We spent our first project week on this part. We tried to define clearly the client need, in order to achieve good product specification. Unfortunately it had to evolve during the first part of our project. We took decisions about the way we would work, for example we decided to use pair programming, and to spend time on our tests before developing. We also worked especially on our risks table to anticipate problems, on our actions table to prevent risks realization, and on our schedule to make sure we would finish on time. \\ + + We have been updating this documents during the project in order to always anticipate problems. + diff --git a/report/chapters/introduction.tex b/report/chapters/introduction.tex new file mode 100644 index 0000000..5480b56 --- /dev/null +++ b/report/chapters/introduction.tex @@ -0,0 +1,21 @@ +\chapter*{Introduction} + + As a conclusion to our studies at the ENSEEIHT, we have been working on our Projet Long. Our project takes part in Bastien Durix thesis which focuses on the extraction of 2D skeletons from pictures and the computation of 3D skeletons from associated 2D skeletons of the same objects, and will also help him on other parts of his approach. + + Our subject consisted in realizing animated 3D versions of real 3D objects. We based ourselves on different pictures of objects taken from different perspectives, and use them to reconstruct a 3D version of the object, which we animate.\\ + +Here is our pipeline : + +\begin{itemize} + \item segmentation + \item intrinsic and extrinsic camera calibration + \item points detection and matching + \item skeletonization : cut and match skeleton pieces + \item computing the B-splines + \item meshing the 3D skeleton computed with the 2D skeletons + \item meshing of extremities and junctions + \item choosing connecting points to animate the object + \item animation +\end{itemize} + + We were two Projet Long groups working on these subject. The first part was split into two subsections : the other group got to work on the segmentation and B-splines computation, while we were given the matching part and the camera calibration. The second part, concerning the meshing and the animation, was done completely by the two groups. \ No newline at end of file diff --git a/report/chapters/license.tex b/report/chapters/license.tex new file mode 100644 index 0000000..012015e --- /dev/null +++ b/report/chapters/license.tex @@ -0,0 +1,36 @@ +\chapter{License} + + +We decided to add a license to our project in order to allow the client using +our project without any problems of properties. We chose the Zlib license +which allow a free-use of our project. You can see our lisence in the listing \ref{license}.\\ + + +\begin{lstlisting}[caption=Zlib license, label=license] +Paella - Copyright (C) 2015 - Thomas FORGIONE, Emilie JALRAS, Marion LENFANT, Thierry MALON, Amandine PAILLOUX + +This software is provided 'as-is', without any express or implied warranty. +In no event will the authors be held liable for any damages arising from the use of this software. + +Permission is granted to anyone to use this software for any purpose, +including commercial applications, and to alter it and redistribute it freely, +subject to the following restrictions: + +1. The origin of this software must not be misrepresented; + you must not claim that you wrote the original software. + If you use this software in a product, an acknowledgment + in the product documentation would be appreciated but is not required. + +2. Altered source versions must be plainly marked as such, + and must not be misrepresented as being the original software. + +3. This notice may not be removed or altered from any source distribution. + +External-libraries +------------------ +Boost is under the boost software license +Eigen is under the MPL2 +OpenCV is under the BSD License +SFML is under the zlib/libpng license + +\end{lstlisting} diff --git a/report/chapters/productDescription.tex b/report/chapters/productDescription.tex new file mode 100644 index 0000000..1c2584d --- /dev/null +++ b/report/chapters/productDescription.tex @@ -0,0 +1,22 @@ +\chapter{Product description} + + Here is the detailed description of the different steps of our project. + + We spent time on finding tests before developing, which helped us to avoid some problems and find some errors earlier than we would have otherwise. You can find more information concerning our tests in the attached tests file. + + + +\section{Segmentation, Camera calibration, skeletonization and branches matching} +\input{subsections/skeleton} +\input{subsections/segmentation} +\input{subsections/calibration} +\input{subsections/pointsDetectionAndMatching} +\input{subsections/branchesMatching} + + +\section{Meshing and animation} +\input{subsections/splines} +\input{subsections/circles} +\input{subsections/extremities} +\input{subsections/junctions} +\input{subsections/animation} diff --git a/report/chapters/productSpecification.tex b/report/chapters/productSpecification.tex new file mode 100644 index 0000000..184d0e3 --- /dev/null +++ b/report/chapters/productSpecification.tex @@ -0,0 +1,41 @@ +\chapter{Product specification} + +During the first week of our project we tried to clearly define the need of the client and the specifications. In an industrial context it is really fundamental to describe the client expectations as well as possible. It protects us from the client if he suddenly wants something not intended at the beginning or if he is disappointed because he forgot to specify certain problems. But it also gives us some duties toward the client concerning the deliveries for instance. This is why it is really important to take it seriously. +Here is our product specification. + +\section{General constraints} +\subsection{Expected deliverables and delivery delay} + +We will deliver an archive containing: +\begin{itemize} + \item detailed documentation (commented code and a user manual) + \item source code : camera calibration and points matching, delivered on February 6th, + and then casing and animation on March 13th. +\end{itemize} + +\subsection{Client supplies} + +The client engages to supply the following elements : +\begin{itemize} + \item programs performing the skeleton extraction + \item mathematical formulas for skeleton casing + \item scientific articles concerning animation +\end{itemize} + +\subsection{Imposed technical constraints} + +Our technical constraints are as follow : +\begin{itemize} + \item C++ as programming language + \item it is suggested to use the openCV library + \item CMake is suggested as compilation utility + \item used libraries must be portable between Windows and Linux +\end{itemize} + +\newpage +\section{Detailed Pipeline} +\begin{figure}[H] + \centering + \includegraphics[scale=0.56]{img/pipeline.png} + \caption{Detailled pipeline} +\end{figure} diff --git a/report/chapters/risksManagement.tex b/report/chapters/risksManagement.tex new file mode 100644 index 0000000..c55dfbb --- /dev/null +++ b/report/chapters/risksManagement.tex @@ -0,0 +1,69 @@ +\chapter{Risks management} + + In order to anticipate problems and to set a right schedule, we tried to find as many risks as possible. We made our risks table evolve while the project progressed. + +\section{Initial risks table} + +\bottomcaption{Risks table on 25th February} +\begin{supertabular}{|p{0.5cm}|p{2cm}|p{2cm}|p{1cm}|p{2cm}|p{1.5cm}|p{2cm}|p{2cm}|p{2cm}|p{1.5cm}|} +\hline + Id & Description & risk cause & Proba (1-5) & Consequence & Seriousness (1-5) & Preventive actions & Corrective actions & Risk state \\ +\hline + \cellcolor{green} 1 & Bad time estimation & Under estimation of the problem difficulty & 3 & Late delivery & 4 & Time margin of the planning, name a task advancement responsible & Raise the work amount, revise the planning & Open \\ + \hline + \cellcolor{green} 2 & The other group does not deliver the deliverables on time & Under estimation of their problem difficulty, last time augmentation of the work amount & 2 & Delay for tests on the 3D part of the skeleton & 3 & Contemplate an other method to test on simple case & Implement ourselves a segmentation method based on simple threshold & Open \\ + \hline + \cellcolor{green} 3 & The other group delivers wrong outputs & Bad comprehension of the subject & 3 & Tests do not work correctly & 1 & Find en agreement on the format & Develop a transition function & Open \\ + \hline + \cellcolor{green} 4 & Binaries provided by the client are not compatible & Systems complexity & 4 & We can not carry on with the pipeline & 4 & Discuss with the client the provided binary & Emulate the system allowing the use of the binaries & Open \\ + \hline + \cellcolor{green} 5 & Fragile harmony inside the group & Different uses for project management and programmation & 1 & Tensions, bad ambiance, intense stress & 5 & Name a cohesion responsible & Organize a mediation beetween the two parties & Open \\ + \hline + \cellcolor{green} 6 & Bad understanding with the other group & A bancal division of the project might bring tensions & 2 & Difficulty to manage shared parts & 3 & Bring a present as a sign of sympathy & Come to a compromise & Open \\ + \hline + \cellcolor{green} 7 & A group member is absent during a certain period of time & Illness, event & 5 & Someone's absence & 1 & Put in place teleworking solutions & Planning adaptation & Open \\ + \hline + \cellcolor{green} 8 & Production of unreliable, unmaintainable and illegible code & Lack of skills with the used technology & 4 & Lack of time, bad quality, need to reprogram & 3 & C++ formation by a technical expert & Call of the expert & Open \\ + \hline + \cellcolor{green} 9 & Need to access urgently to a deleted resource & Utilization of a suppression command, material crash & 2 & Lack of time, need to reprogram & 5 & Use github & Use of recuperation technology on long and complicated files & Open \\ + \hline + \cellcolor{green} 10 & Pictures unadapted & Client not satisfied of the pictures & 4 & Wasting time & 2 & Defined the conditions of the shoot with the client more precisely & Be aware of client expectations and taking again the picture & Open \\ +\hline +\end{supertabular} + +\section{Final risks table} + + In the table below you can find the risks state in the end of the project.\\ + +\bottomcaption{Risks table on 25th February} +\begin{supertabular}{|p{0.5cm}|p{2cm}|p{2cm}|p{1cm}|p{2cm}|p{1.5cm}|p{2cm}|p{2cm}|p{2cm}|p{1.5cm}|} +\hline + Id & Description & risk cause & Proba (1-5) & Consequence & Seriousness (1-5) & Preventive actions & Corrective actions & Risk state \\ +\hline + \cellcolor{green} 1 & Bad time estimation & Under estimation of the problem difficulty & 3 & Late delivery & 4 & Time margin of the planning, name a task advancement responsible & Raise the work amount, revise the planning & Open \\ + \hline + \cellcolor{yellow} 2 & The other group does not deliver the deliverables on time & Under estimation of their problem difficulty, last time augmentation of the work amount & 2 & Delay for tests on the 3D part of the skeleton & 3 & Contemplate an other method to test on simple case & Implement ourselves a segmentation method based on simple threshold & Open \\ + \hline + \cellcolor{green} 3 & The other group delivers wrong outputs & Bad comprehension of the subject & 3 & Tests do not work correctly & 1 & Find en agreement on the format & Develop a transition function & Open \\ + \hline + \cellcolor{yellow} 4 & Binaries provided by the client are not compatible & Systems complexity & 4 & We can not carry on with the pipeline & 4 & Discuss with the client the provided binary & Emulate the system allowing the use of the binaries & Open \\ + \hline + \cellcolor{green} 5 & Fragile harmony inside the group & Different uses for project management and programmation & 1 & Tensions, bad ambiance, intense stress & 5 & Name a cohesion responsible & Organize a mediation beetween the two parties & Open \\ + \hline + \cellcolor{green} 6 & Bad understanding with the other group & A bancal division of the project might bring tensions & 2 & Difficulty to manage shared parts & 3 & Bring a present as a sign of sympathy & Come to a compromise & Open \\ + \hline + \cellcolor{green} 7 & A group member is absent during a certain period of time & Illness, event & 5 & Someone's absence & 1 & Put in place teleworking solutions & Planning adaptation & Open \\ + \hline + \cellcolor{green} 8 & Production of unreliable, unmaintainable and illegible code & Lack of skills with the used technology & 4 & Lack of time, bad quality, need to reprogram & 3 & C++ formation by a technical expert & Call of the expert & Closed \\ + \hline + \cellcolor{green} 9 & Need to access urgently to a deleted resource & Utilization of a suppression command, material crash & 2 & Lack of time, need to reprogram & 5 & Use github & Use of recuperation technology on long and complicated files & Open \\ + \hline + \cellcolor{yellow} 10 & Pictures unadapted & Client not satisfied of the pictures & 4 & Wasting time & 2 & Defined the conditions of the shoot with the client more precisely & Be aware of client expectations and taking again the picture & Open \\ +\hline + \cellcolor{green} 11 & New task to do & Client new needs & 4 & Late delivery & 4 & Add margin in planning & change planning and contact the client to negociate about this task & Closed \\ + \hline + \cellcolor{green} 12 & Integration not cancelled & Client changes his mind and wants again the integration & 2 & Late delivery & 4 & discuss with the client & change planning & Open \\ + \hline + \cellcolor{green} 13 & members of the group get remedial classes & bad results at the exams & 2 & member absent & 2 & add margin & change planning & Open \\ + \hline +\end{supertabular} diff --git a/report/chapters/schedule.tex b/report/chapters/schedule.tex new file mode 100644 index 0000000..a3081a1 --- /dev/null +++ b/report/chapters/schedule.tex @@ -0,0 +1,29 @@ + +\chapter{Schedule} + + Thanks to our risks table we had anticipated the possibility of a schedule modification due to our interaction with the other group and with the client binaries. We had kept one week as security, and finally had to use it before the integration period. + +\section{Initial schedule} + + We spent our first week on the project management, like doing this schedule. \\ + At the beginning we did a detailed schedule for the first part of the project, but not for the second part of the project. We just assigned 2 weeks and half to the meshing, and one week to the animation. + +\begin{figure}[H] + \begin{center} + \includegraphics[scale=0.5]{img/initialSchedule} + \caption{\label{initialSchedule} Initial Schedule} + \end{center} +\end{figure} + +\section{Final schedule} + + Before starting the second part of the project we did the detailed schedule of this part, which you can see below. + The integration with the other group was abandoned, as you can see. + We also decided to finally start the report before finishing the second part. + +\begin{figure}[H] + \begin{center} + \includegraphics[scale=0.5]{img/finalSchedule} + \caption{\label{finalSchedule} Final Schedule} + \end{center} +\end{figure} \ No newline at end of file diff --git a/report/chapters/technicalConclusion.tex b/report/chapters/technicalConclusion.tex new file mode 100644 index 0000000..4e3cf0c --- /dev/null +++ b/report/chapters/technicalConclusion.tex @@ -0,0 +1,6 @@ +\chapter*{Conclusion of the technical part} + + The technical part of this project taught us a lot, especially in C++ programming. + Our knowledge from the multimedia specialization was very helpful, as well in the image analysis part as in the meshes and animation part. + During this project we also tried to think of tests before developing, which helped us in finding errors early, and anticipate problems. + \ No newline at end of file diff --git a/report/chapters/tests.tex b/report/chapters/tests.tex new file mode 100644 index 0000000..94e1bec --- /dev/null +++ b/report/chapters/tests.tex @@ -0,0 +1,4 @@ +\chapter{Tests} + + You can find our tests in the attached folder. + We reported the results obtained with different parameters or methods, and explained our choices. We also tackle the limits of our different programs. \ No newline at end of file diff --git a/report/img/0.jpg b/report/img/0.jpg new file mode 100644 index 0000000..4f2fabf Binary files /dev/null and b/report/img/0.jpg differ diff --git a/report/img/0ed.jpg b/report/img/0ed.jpg new file mode 100644 index 0000000..53b7a1b Binary files /dev/null and b/report/img/0ed.jpg differ diff --git a/report/img/1.jpg b/report/img/1.jpg new file mode 100644 index 0000000..ae13c54 Binary files /dev/null and b/report/img/1.jpg differ diff --git a/report/img/10.jpg b/report/img/10.jpg new file mode 100644 index 0000000..ec8f439 Binary files /dev/null and b/report/img/10.jpg differ diff --git a/report/img/10ed.jpg b/report/img/10ed.jpg new file mode 100644 index 0000000..02ebbf4 Binary files /dev/null and b/report/img/10ed.jpg differ diff --git a/report/img/1ed.jpg b/report/img/1ed.jpg new file mode 100644 index 0000000..29ab682 Binary files /dev/null and b/report/img/1ed.jpg differ diff --git a/report/img/2.jpg b/report/img/2.jpg new file mode 100644 index 0000000..1ae0df0 Binary files /dev/null and b/report/img/2.jpg differ diff --git a/report/img/2ed.jpg b/report/img/2ed.jpg new file mode 100644 index 0000000..d190e06 Binary files /dev/null and b/report/img/2ed.jpg differ diff --git a/report/img/JunctionTheory.png b/report/img/JunctionTheory.png new file mode 100644 index 0000000..400f8b6 Binary files /dev/null and b/report/img/JunctionTheory.png differ diff --git a/report/img/Junctions1.png b/report/img/Junctions1.png new file mode 100644 index 0000000..574f971 Binary files /dev/null and b/report/img/Junctions1.png differ diff --git a/report/img/Junctions10.png b/report/img/Junctions10.png new file mode 100644 index 0000000..6e625b7 Binary files /dev/null and b/report/img/Junctions10.png differ diff --git a/report/img/Junctions11.png b/report/img/Junctions11.png new file mode 100644 index 0000000..50f1599 Binary files /dev/null and b/report/img/Junctions11.png differ diff --git a/report/img/Junctions2.png b/report/img/Junctions2.png new file mode 100644 index 0000000..58c458b Binary files /dev/null and b/report/img/Junctions2.png differ diff --git a/report/img/Junctions3.png b/report/img/Junctions3.png new file mode 100644 index 0000000..7555370 Binary files /dev/null and b/report/img/Junctions3.png differ diff --git a/report/img/Junctions4.png b/report/img/Junctions4.png new file mode 100644 index 0000000..526ea48 Binary files /dev/null and b/report/img/Junctions4.png differ diff --git a/report/img/Junctions5.png b/report/img/Junctions5.png new file mode 100644 index 0000000..d5a5ec7 Binary files /dev/null and b/report/img/Junctions5.png differ diff --git a/report/img/Junctions9.png b/report/img/Junctions9.png new file mode 100644 index 0000000..269d036 Binary files /dev/null and b/report/img/Junctions9.png differ diff --git a/report/img/LapinSiftSurf.png b/report/img/LapinSiftSurf.png new file mode 100644 index 0000000..c99d47c Binary files /dev/null and b/report/img/LapinSiftSurf.png differ diff --git a/report/img/LapinSymetricGeometric.PNG b/report/img/LapinSymetricGeometric.PNG new file mode 100644 index 0000000..7242a3d Binary files /dev/null and b/report/img/LapinSymetricGeometric.PNG differ diff --git a/report/img/Poupee.png b/report/img/Poupee.png new file mode 100644 index 0000000..419b775 Binary files /dev/null and b/report/img/Poupee.png differ diff --git a/report/img/Poupeebin.png b/report/img/Poupeebin.png new file mode 100644 index 0000000..5d0dfe3 Binary files /dev/null and b/report/img/Poupeebin.png differ diff --git a/report/img/Poupeeskl.png b/report/img/Poupeeskl.png new file mode 100644 index 0000000..03793ff Binary files /dev/null and b/report/img/Poupeeskl.png differ diff --git a/report/img/SkeletonEx.png b/report/img/SkeletonEx.png new file mode 100644 index 0000000..9a8e9d5 Binary files /dev/null and b/report/img/SkeletonEx.png differ diff --git a/report/img/Subdivision.png b/report/img/Subdivision.png new file mode 100644 index 0000000..14b2731 Binary files /dev/null and b/report/img/Subdivision.png differ diff --git a/report/img/brisk3415.png b/report/img/brisk3415.png new file mode 100644 index 0000000..0bee192 Binary files /dev/null and b/report/img/brisk3415.png differ diff --git a/report/img/characteristicCircle.png b/report/img/characteristicCircle.png new file mode 100644 index 0000000..d9b6ce2 Binary files /dev/null and b/report/img/characteristicCircle.png differ diff --git a/report/img/finalSchedule.PNG b/report/img/finalSchedule.PNG new file mode 100644 index 0000000..5cf2301 Binary files /dev/null and b/report/img/finalSchedule.PNG differ diff --git a/report/img/initialSchedule.PNG b/report/img/initialSchedule.PNG new file mode 100644 index 0000000..66c0e24 Binary files /dev/null and b/report/img/initialSchedule.PNG differ diff --git a/report/img/meshExtremity.png b/report/img/meshExtremity.png new file mode 100644 index 0000000..32c03eb Binary files /dev/null and b/report/img/meshExtremity.png differ diff --git a/report/img/meshextremity.jpg b/report/img/meshextremity.jpg new file mode 100644 index 0000000..13babf3 Binary files /dev/null and b/report/img/meshextremity.jpg differ diff --git a/report/img/n7.png b/report/img/n7.png new file mode 100644 index 0000000..e1ce833 Binary files /dev/null and b/report/img/n7.png differ diff --git a/report/img/pipeline.png b/report/img/pipeline.png new file mode 100755 index 0000000..c462847 Binary files /dev/null and b/report/img/pipeline.png differ diff --git a/report/img/projection.png b/report/img/projection.png new file mode 100644 index 0000000..a844fef Binary files /dev/null and b/report/img/projection.png differ diff --git a/report/img/regularMesh.png b/report/img/regularMesh.png new file mode 100755 index 0000000..00d0d2d Binary files /dev/null and b/report/img/regularMesh.png differ diff --git a/report/img/spline.png b/report/img/spline.png new file mode 100755 index 0000000..7097e0c Binary files /dev/null and b/report/img/spline.png differ diff --git a/report/references.bib b/report/references.bib new file mode 100644 index 0000000..472849f --- /dev/null +++ b/report/references.bib @@ -0,0 +1,34 @@ +@article{brisk, + author = "Stefan, Leutenegger and Margarita, Chli and Roland, Y.Siegwart", + title = "{BRISK} : Binary Robust Invariant Scalable Keypoints", + year = "2011", +} + +@online{surf, + author = "OpenCV 3.0.0-dev documentation", + title = "Introduction to {SURF} (Speeded-Up Robust Features)", + url = "http://docs.opencv.org/trunk/doc/py_tutorials/py_feature2d/py_surf_intro/py_surf_intro.html", + urldate = "2015-03-12" +} + +@online{sift, + author = "OpenCV 3.0.0-dev documentation", + title = "Introduction to {SIFT} (Scale-Invariant Feature Transform)", + url = "http://docs.opencv.org/trunk/doc/py_tutorials/py_feature2d/py_sift_intro/py_sift_intro.html", + urldate = "2015-03-12" +} + +@online{license, + author = "Open Source Initiative", + title = "The zlib/libpng License (Zlib)", + url = "http://opensource.org/licenses/Zlib", + urldate = "2015-03-12" +} + +@online{spline, + author="Jana Proch\'azkov\'a", + title="DERIVATIVE OF B-SPLINE FUNCTION", + url="http://mat.fsv.cvut.cz/gcg/sbornik/prochazkova.pdf", + urldate="2015-03-12" +} + diff --git a/report/report.tex b/report/report.tex new file mode 100644 index 0000000..336787d --- /dev/null +++ b/report/report.tex @@ -0,0 +1,127 @@ + +\documentclass{report} +\usepackage[english]{babel} +\usepackage{amssymb} +\usepackage{lmodern} +\usepackage[T1]{fontenc} +\usepackage{verbatim} +\usepackage{ucs} +\usepackage{graphicx} +\usepackage{caption} +\usepackage{amsmath} +\usepackage{multicol} +\usepackage{hyperref} +\usepackage{tikz} +\usepackage{pgfplots} +\usepackage{subcaption} +\usepackage{supertabular} +\usepackage{colortbl} +\usepackage{lastpage} +\usepackage{listings} + +\usepackage{xcolor} +\usepackage{graphicx} +\definecolor{colKeys}{rgb}{0,0.5,0} +\definecolor{colIdentifier}{rgb}{0,0,0} +\definecolor{colComments}{rgb}{0,0.5,1} +\definecolor{colString}{rgb}{0.6,0.1,0.1} +\definecolor{colBackground}{rgb}{0.95,0.95,1} + +\lstset{%configuration de listings + float=hbp,% + basicstyle=\ttfamily\small, % + % + identifierstyle=\color{colIdentifier}, % + keywordstyle=\color{colKeys}, % + stringstyle=\color{colString}, % + commentstyle=\color{colComments}\textit, % + % + backgroundcolor=\color{colBackground},% + % + columns=flexible, % + tabsize=2, % + frame=trbl, % + %frameround=tttt,% + extendedchars=true, % + showspaces=false, % + showstringspaces=false, % + numbers=left, % + numberstyle=\tiny, % + breaklines=true, % + breakautoindent=true, % + captionpos=b,% + xrightmargin=0cm, % + xleftmargin=0cm% +} + +\setlength{\hoffset}{-18pt} +\setlength{\oddsidemargin}{0pt} % Marge gauche sur pages impaires +\setlength{\evensidemargin}{9pt} % Marge gauche sur pages paires +\setlength{\marginparwidth}{54pt} % Largeur de note dans la marge +\setlength{\textwidth}{17cm} % Largeur de la zone de texte (17cm) +\setlength{\voffset}{-18pt} % Bon pour DOS +\setlength{\marginparsep}{7pt} % Séparation de la marge +\setlength{\topmargin}{-30pt} % Pas de marge en haut +\setlength{\headheight}{13pt} % Haut de page +\setlength{\headsep}{10pt} % Entre le haut de page et le texte +\setlength{\footskip}{0.8cm} % Bas de page + séparation +\setlength{\textheight}{24.5cm} % Hauteur de la zone de texte (25cm) + + +\usepackage{titling} +\usepackage{fancyhdr} +\pagestyle{fancy} +\renewcommand{\headrulewidth}{1pt} +\renewcommand{\footrulewidth}{\headrulewidth} +\usepackage{float} +\newcommand{\hsp}{\hspace{20pt}} +\newcommand{\HRule}{\rule{\linewidth}{0.5mm}} +\fancyfoot[RO]{\thepage/\pageref{LastPage}} +\fancyfoot[C]{January 2015 --- March 2015} +\fancyfoot[L]{\emph{Projet long report}} + +% Redefine the plain page style (display the footer on chapter page) +\fancypagestyle{plain}{% + \fancyhf{}% + \fancyfoot[RO]{\thepage/\pageref{LastPage}} + \fancyfoot[C]{January 2015 --- March 2015} + \fancyfoot[L]{\emph{Projet long report}} + \renewcommand{\headrulewidth}{0pt}% Line at the header invisible + \renewcommand{\footrulewidth}{0.4pt}% Line at the footer visible +} + + +\makeatletter +\@addtoreset{chapter}{part} +\makeatother + +\begin{document} + +\input{chapters/frontPage} +\input{chapters/introduction} + +\part{Technical Field} + +\input{chapters/productSpecification} +\input{chapters/productDescription} +%\input{chapters/tests} +\input{chapters/technicalConclusion} + +\part{Project Management} + +\input{chapters/introManagement} +\input{chapters/risksManagement} +\input{chapters/actions} +\input{chapters/schedule} +\input{chapters/license} + +\input{chapters/conclusion} + +\bibliographystyle{plain} +\bibliography{references} + +\appendix +\input{annexes/skeletons} + +\end{document} + diff --git a/report/subsections/animation.tex b/report/subsections/animation.tex new file mode 100644 index 0000000..44fc4fc --- /dev/null +++ b/report/subsections/animation.tex @@ -0,0 +1,93 @@ +\subsection{Animation} + The objective of this part is to animate the mesh we previously generated + around the skeleton. We have as an input segments of the skeleton, and its + mesh. The idea is to deform the segments of the skeleton, and to + automatically have the impact of this deformation on the vertices of the + mesh. + + \subsubsection{Trees} + When we will animate our mesh, we want to keep it as one piece. For + example, when you move your arm, your elbow and your forearm follows + the movement of your arm. This means that the transformation that we + will apply after the shoulders must be kept on the arms and forearms. + + To manage to do this, we structured everything in trees. The tree for a + person will be the following for example (to simplify, we will take the + hypothesis that this human has no knees). + + \begin{figure}[H] + \centering + \begin{tikzpicture} + % Head + \draw (0,0) circle [radius=1]; + \draw (0,0) node{Head}; + + % Left son of head + \draw (0,-1) -- (-3,-2); + \draw (-3,-3) circle [radius=1]; + \draw (-3,-3) node{Left arm}; + + % Left son of left arm + \draw (-3,-4) -- (-6,-5); + \draw (-6,-6) circle [radius=1]; + \draw (-6,-6) node{Left forearm}; + + % Middle son + \draw ( 0,-1) -- ( 0,-2); + \draw ( 0,-3) circle [radius=1]; + \draw ( 0,-3) node{Body}; + + % Left leg + \draw (0, -4) -- (-2, -5); + \draw ( -2,-6) circle [radius=1]; + \draw ( -2,-6) node{Left leg}; + + % Right leg + \draw (0, -4) -- (2, -5); + \draw ( 2,-6) circle [radius=1]; + \draw ( 2,-6) node{Right leg}; + + % Right son + \draw (0,-1) -- (3,-2); + \draw (3,-3) circle [radius=1]; + \draw (3,-3) node{Right arm}; + + % Left son of left arm + \draw (3,-4) -- (6,-5); + \draw (6,-6) circle [radius=1]; + \draw (6,-6) node{Right forearm}; + + \end{tikzpicture} + \caption{Tree for a human being skeleton} + \end{figure} + + \paragraph{} + At each node of this, there will be a rotation (with a center, and + angles), and we will draw the animated mesh by traversing the tree. + At each node, we will apply the transformation of the current node, and + then draw the subtree. + + Of course, we also need to know what faces are in each nodes, so we + will have a tree of rotations, and a tree of faces. + + \subsubsection{Junctions processing} + All we said before is valid only if the three vertices of a face are + mapped to the same segment. But some faces have vertices that are + mapped to different segments. For these faces, we need to apply a + different transformation for each vertex. + + For this purpose, we created a hashtable between vertex numbers and + paths in trees so that we are able to find the transformation for these + vertices. + + \subsubsection{Graphical User Interface} + In order to be able to manipulate the animated mesh easily, two graphical interfaces were made : + \begin{enumerate} + \item the first one allows the user to add rotation points to the + skeleton (by default, only the junction points are considered + as rotation points, and there are no knees and elbows for + example). + \item the second one has the OpenGL rendering and a menu controlled + with the keyboard, allowing the user to select rotation points + and to change the value of the angles. + \end{enumerate} diff --git a/report/subsections/branchesMatching.tex b/report/subsections/branchesMatching.tex new file mode 100644 index 0000000..9bde76d --- /dev/null +++ b/report/subsections/branchesMatching.tex @@ -0,0 +1,39 @@ +\subsection{Branches Extraction and Matching} + The branches extraction part was allocated to both groups. The skeleton we have extracted are composed of points. A skeleton point can be of different types: + \begin{enumerate} + \item a extremity point which only have one neighbor, i.e. this point is involved in only one edge of the skeleton. + \item a regular point which have two neighbors, i.e. this point is involved in only two edges of the skeleton. + \item a junction point which have at least three neighbors, i.e. this point is involved in three edges or more than three edges. + \end{enumerate} + + In the cases of extremity points and junction points, we consider that the point is irregular. To extract the branches of the skeleton, we start from each irregular point and we follow the edges starting from this irregular point until we find another irregular point (junction or extremity). + + Let us consider two irregular points $P_1$ and $P_5$ such as there is a path $P_1$, $P_2$, $P_3$, $P_4$, $P_5$ linking these two irregular points together. As a result, $P_1$, $P_2$, $P_3$, $P_4$, $P_5$ is a branch of the skeleton. However, the algorithm will also find $P_5$, $P_4$, $P_3$, $P_2$, $P_1$ as a branch of the skeleton. To avoid having each branch two times, we added a new branch to the branches set only if the contrary branch is not already in the set. + + Once we extract all the branches on two correspoding skeletons, we would like to match these branches together thanks to the keypoints found on each picture. For this purpose, we would like to assign to each keypoint a branch of the skeleton. We will consider that both skeletons have the same number of branches. If not, the matching between branches would not be perfectly possible. + + To determine the correspoding branch of a keypoint, we use the euclidian distance. Each keypoint is assigned to the nearest branch of the skeleton. As a branch is composed of segments, we will define the distance between a point and a branch as the shortest distance between this point and all the segments composing the branch. Thus, the nearest branch of a point is the branch that contains the nearest segment to the point. + + The distance between a point and a segment is defined as following: + \begin{enumerate} + \item if the projection of the point is on the line that contains the segment, then the distance between the point and the segment is the distance between the point and its projection. + \item if the projection of the point is not on the line that contains the segment, then the distance between the point and the segment is the shortest distance between the distance between the point and each extremity of the segment. + \end{enumerate} + + Once each keypoint has been associated to a branch, we use the matching between keypoints from the two pictures and, for each picture, the matching between keypoints and branches to deduce the matchings between the branches from the two skeletons. + +$$\begin{matrix} +\text{Keypoints 1} & \longleftrightarrow & \text{Keypoints 2} \\ + \updownarrow & & \updownarrow \\ + \text{Branches 1} & & \text{Branches 2} +\end{matrix}$$ + + To compute the matches between branches, we define a matching a square matrix $M$ which size is the number of branches of each skeleton. + + Let $KP_1$ and $KP_2$ be two matching keypoints, respectively from $Picture_1$ and $Picture_2$. Let $Skeleton_1$ and $Skeleton_2$ be the extracted skeletons of $Picture_1$ and $Picture_2$. Let $Branch_1$ be a branch of $Skeleton_1$, $Branch_2$ a branch of $Skeleton_2$. We assume that $KP_1$ is associated to $Branch_1$ and $KP_2$ is associated to $Branch_2$. Since $KP_1$ and $KP_2$ are matching keypoints, a vote is given to a match between $Branch_1$ and $Branch_2$. + + Each time a vote is given to a match between $Branch_i$ and $Branch_j$, the element $m_{i,j}$ of $M$ is incremented. Once each pair of matching keypoints has voted, the branches are matching using the matching matrix. + + Let $Branch_i$ be a branch of $Skeleton_1$ and $M$ be the matching matrix between branches of $Skeleton_1$ and the branches of $Skeleton_2$. The index $j$ of the matching branch $Branch_j$ with $Branch_i$ is defined such as the element $m_{i,j}$ of $M_{i,\cdot}$ is maximal. + + In pratice, the skeletons we try to match have never the same number of branches: some differences may the algorithm unable to give good results. The test report details the differences between skeletons. We thought about solutions, such as merging short branches with others. We could also ask the user to click on matching branches. In this case, there is no need to detect keypoints anymore. Another solution is to ask the user to click on noising branches that often appears on extremities of the skeleton. diff --git a/report/subsections/calibration.tex b/report/subsections/calibration.tex new file mode 100644 index 0000000..6e1cff7 --- /dev/null +++ b/report/subsections/calibration.tex @@ -0,0 +1,21 @@ +% Here we go +\subsection{Camera calibration} + The camera calibration is needed by the client during the reconstruction of the 3D skeleton. The calibration consists in two steps : + + \begin{enumerate} + \item the \textbf{internal calibration} which is the estimation of the instrinsic parameters of the camera (such as the focal length, or the + principal point) + + \item the \textbf{external calibration} which is the estimation of the + extrinsic parameters (the position of the camera from the chessboard) + \end{enumerate} + + \subsubsection{Internal calibration} + The internal calibration is based on a program that we completed during a lesson last semester. By detecting chessboards from numerous positions of the camera, we are able to estimate the intrinsic parameters. This program takes as input a video of a chessboard where the camera position changes, and it captures some frames on which the chessboard is detected. With these pictures, we estimate the intrinsic parameters. + + \subsubsection{External calibration} + For the external calibration, we wrote a program that uses an + estimation of the homography to compute the extrinsical parameters of + the camera. One should note that what we call the \emph{extrinsical + parameters} are the ones that convert points from the reference of the + chessboard to the reference of the camera. diff --git a/report/subsections/circles.tex b/report/subsections/circles.tex new file mode 100644 index 0000000..a1d744c --- /dev/null +++ b/report/subsections/circles.tex @@ -0,0 +1,42 @@ +\subsection{Regular mesh} + + At the beginning of this part we have a set of 3D splines. We sample this splines and calculate the characteristic circle associated to each sampled point. Then we match the points between successive circles and use a triangular mesh. + +\subsubsection{Characteristic circles computation} + + We take a point sampled on the spline. It gives us information about the sphere located on this point : $\overrightarrow{C}(t)$ the sphere center coordinates and r(t) its radius. We also have their derivatives : $\overrightarrow{C'}(t)$ and r'(t). The idea is to find the intersection between this sphere and the characteristic plane, which would give us the characteristic circle. + If a point P is on the circle, then we can write the formula \ref{eq1} : +\begin{equation} +<\overrightarrow{C'}(t),\overrightarrow{PC}(t)> -r'(t)r(t) = 0 + \label{eq1} +\end{equation} + + +and then we calculate the center of the characteristic circle with the formula \ref{eq2}: +\begin{equation} +\overrightarrow{Cp}(t) = \overrightarrow{C}(t) - \frac{r'(t)r(t)}{\| \overrightarrow{C'}(t)\| ^{2}} \times \overrightarrow{C'}(t) +\label{eq2} +\end{equation} + +\begin{figure}[H] + \begin{center} + \includegraphics[scale=0.8]{img/characteristicCircle} + \caption{\label{initialSchedule} Characteristic circle calculation} + \end{center} +\end{figure} + +\subsubsection{Meshing} + +After obtaining characteristic circles we sample them. +Then we need to find matching points. We decided to make points sampling start in one direction of our mark. +When this is done we project the mark on the new circle and sample again. Then we link each point to the point of the same index on the next circle. +The final mesh can be seen Figure \ref{regularMesh}. + +\begin{figure}[H] + \begin{center} + \includegraphics[scale=0.4]{img/regularMesh} + \caption{\label{regularMesh} The regular mesh} + \end{center} +\end{figure} + +%TODO diff --git a/report/subsections/extremities.tex b/report/subsections/extremities.tex new file mode 100644 index 0000000..eb8ae92 --- /dev/null +++ b/report/subsections/extremities.tex @@ -0,0 +1,42 @@ +\subsection{Extremities} + +After computing the mesh around the splines we need to draw the extremities using subdivisions to obtain smoother extremities. +The first step is to calculate the projection of the extremity circle on the correspondent sphere. Then for a subdivision of depth 0 we link this projection with the mesh points along the extrem circle (see Figure \ref{projection}). For one subdivision we take the middle of each segments computed earlier and we project it on the sphere (see Figure \ref{sub}). Finally we made a triangular mesh the way you can see on Figure \ref{mesh}). + +The rendering of our extremities on dino.skl for example can be seen on Figures \ref{extremity2} (2 subdivisions) and \ref{extremity10} + (10 subdivisions). + +\begin{figure}[H] + \begin{center} + \includegraphics[scale=0.2]{img/projection} + \caption{\label{projection}Projection of the extrem circle center on the sphere} + \end{center} +\end{figure} + +\begin{figure}[H] + \begin{center} + \includegraphics[scale=0.2]{img/Subdivision} + \caption{\label{sub}One subdivision} + \end{center} +\end{figure} + +\begin{figure}[H] + \begin{center} + \includegraphics[scale=0.2]{img/meshExtremity} + \caption{\label{mesh}The final extremity mesh for one subdivision} + \end{center} +\end{figure} + +\begin{figure}[H] +\begin{minipage}[b]{0.45\linewidth} +\centering +\includegraphics[scale=0.5]{img/2} +\caption{\label{extremity2}The final extremity mesh for 2 subdivisions} +\end{minipage} +\hspace{0.5cm} +\begin{minipage}[b]{0.45\linewidth} +\centering +\includegraphics[scale=0.5]{img/10} +\caption{\label{extremity10}The final extremity mesh for 10 subdivisions} +\end{minipage} +\end{figure} diff --git a/report/subsections/junctions.tex b/report/subsections/junctions.tex new file mode 100644 index 0000000..0fd3638 --- /dev/null +++ b/report/subsections/junctions.tex @@ -0,0 +1,93 @@ +\subsection{Junctions} + +The last step to complete is to mesh the junctions. +A junction is a point of a skeleton where more than 2 branches join. The process for meshing this portion of the skeleton is complex. In fact it needs to be applicable on multiple cases, for instance 3 or 4 branches (see Figure \ref{junction}, and to take into account the fact that the perfect case will not always be there. + +\begin{figure}[H] + \begin{center} + \includegraphics[scale=0.5]{img/Junctions9} + \caption{\label{junction}Three splines junction} + \end{center} +\end{figure} + +In the figure \ref{junction1} you can see what we should theoretically obtain with a perfect skeleton. You can see the sphere shared between three splines joining in this junction, and the three characteristic circles associated. Those circles are tangent by pair in one point. + +\begin{figure}[H] + \begin{center} + \includegraphics[scale=0.5]{img/JunctionTheory} + \caption{\label{junction1}Theoretical case with perfect characteristic circles} + \end{center} +\end{figure} + +But in most of practical cases it is not like this. This is due to number's approximation on computers. The idea is then to look for the closest points of two consecutive circles and to join them. Then we join every point of the circles in one point upside and one downside. + +We begin by identifying the last circles of each spline of the junction (see figure \ref{junction3}). + +\begin{figure}[H] + \begin{center} + \includegraphics[scale=0.5]{img/Junctions3} + \caption{\label{junction3}3-splines junction} + \end{center} +\end{figure} + +To be able to do it the first step is to cut the circles in two parts to find the upside and downside of each of them. This can be done by computing the best fitting plane for the set of circles' center points (figure \ref{junction2}). +The equation to find is easy when there is only 3 circles : first we compute the normal vector of the plane and then the constant element of the equation. +However when there is more than 3 circles to join, so more than three points to approximate, we need to use a PCA (Principal component analysis) decomposition to find the best fitting plane. + +The first thing to do is to create X the matrix of the points coordinates. +\[ +X = + \begin{pmatrix} + x_{1} & x_{2} & \cdots & x_{n} \\ + y_{1} & y_{2} & \cdots & y_{n} \\ + z_{1} & z_{2} & \cdots & z_{n} + \end{pmatrix} + \] +Then subtract it with the centroid matrix associated. +\[ + X_{c} = X - X_{m} +\] +The normal vector $N$ of the best-fitting plane will be the cross product of the 2 eigenvectors $v_1,v_2$ corresponding to the two biggest singular values of the covariance matrix of $X_{c}$ : +\[ + X_{c}X_{c}^T + \] + \[ + N = v_1 \wedge v_2 + \] + +Then to compute the constant value of the equation we use the centroid. +\[ +d = -N*X_m +\] + +At the end we have the parameters of the plane in N and d. + +\begin{figure}[H] + \begin{center} + \includegraphics[scale=0.35]{img/Junctions1} + \includegraphics[scale=0.35]{img/Junctions2} + \caption{\label{junction2}Best fitting plane for the set of circles' center points} + \end{center} +\end{figure} + +Then when points are sorted we connect the up-points of each circle with the up-projection of the sphere's center on itself, and idem for the down-points with the down-projection of the sphere's center. +The result of this process is presented in figure \ref{junction4}. + +\begin{figure}[H] + \begin{center} + \includegraphics[scale=0.5]{img/Junctions4} + \includegraphics[scale=0.5]{img/Junctions5} + \caption{\label{junction4}Edges added and mesh result} + \end{center} +\end{figure} + +To easily see the result of the mesh on a junction, we have made some tests with only the extreme circles that we use. On real skeletons the junction are not easy to watch. +Here \ref{junction5} are two examples of a junction's mesh for 3 and 4-branches junctions. + +\begin{figure}[H] + \begin{center} + \includegraphics[scale=0.5]{img/Junctions10} + \includegraphics[scale=0.4]{img/Junctions11} + \caption{\label{junction5}Mesh result on 3 and 4-branches junctions} + \end{center} +\end{figure} diff --git a/report/subsections/pointsDetectionAndMatching.tex b/report/subsections/pointsDetectionAndMatching.tex new file mode 100644 index 0000000..5d9e69c --- /dev/null +++ b/report/subsections/pointsDetectionAndMatching.tex @@ -0,0 +1,62 @@ +\subsection{Points detection and matching} +\label{detection} +\subsubsection{Points detection} + +As a first step the detection of keypoints was made by using SURF (Speeded-Up Robust Features) algorithm (provided by +OpenCV) because of its good +speed but this algorithm is computed in nonfree module which is not free to use in commercial +application. That is why we decided to choose the BRISK (Binary Robust Invariant Scalable Keypoints) +algorithm (also provided by OpenCV) is pretty close to the SIFT (Scale-invariant feature transform) algorithm. +Therefore the tests were made using SURF and BRISK algorithm but the final version is with BRISK as SURF is patented. +As the aera of interest in the images is the object, we provide a binary mask (computed during the segmentation) in input of +SURF and BRISK detectors. + + +SURF is a speed version of SIFT (see Figure \ref{siftSurf}). Indeed instead of approximate Laplacian of Gaussian (LoG) with Difference +of Gaussian for finding scale-space. SURF goes a little further and approximates LoG with Box Filter. + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{img/LapinSiftSurf} + \caption{\label{siftSurf}Points detection for SIFT and SURF algorithm} +\end{figure} + +For BRISK (see Figure \ref{brisk3415}) points of interest are identified across both the image and scale dimensions using a saliency criterion. +In order to increase the speed of computation, keypoints are detected in octave layers of the image pyramid as well as +in layers in-between. The location and the scale of each keypoint are obtained in the continuous domain via quadratic function fitting. +A sampling pattern consisting of points lying on appropriately scaled concentric circles is applied at the neighborhood of each keypoint to retrieve +gray values. Finally, the oriented BRISK sampling pattern is used to obtain pairwise brightness comparison results which are +assembled into the binary BRISK descriptor. The BRISK constructor take in input three parameters which modify the results : +the thresh, the octave and the patternScale. + + +A detailed explanation of these algorithms can be found with the references \cite{brisk} (BRISK), \cite{surf} (SURF) and +\cite{sift} (SIFT). + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{img/brisk3415} + \caption{\label{brisk3415}Detected points with BRISK algorithm} +\end{figure} + + +\subsubsection{Points matching} + +We did the points matching, using a brute force matcher provided by OpenCV and then applied filters to get rid of inaccurate points. +For each descriptor in the first set, this matcher finds the closest descriptor in the second set by trying each one. +The filters used are : +\begin{itemize} + \item symmetric filter : the matches found when we take the image\_1 as base need to be found when we take the image\_2 as base also. + \item order constraint : the position of each point is compared to each other in image\_1 and image\_2, if there is too much error these points + are deleted. + \item threshold filter : filter on the distance between the descriptors of the matching points. This filter is not used with BRISK + detection because the results are quite good without it. + \item geometric filter : filter which use epipolar geometry, and the fundamental matrix to filter strange points. +\end{itemize} + + +\begin{figure}[H] + \centering + \includegraphics[scale=0.45]{img/LapinSymetricGeometric} + \caption{Points matching obtained after symmetric and geometric filtering} +\end{figure} diff --git a/report/subsections/segmentation.tex b/report/subsections/segmentation.tex new file mode 100644 index 0000000..aec33f4 --- /dev/null +++ b/report/subsections/segmentation.tex @@ -0,0 +1,27 @@ +\subsection{Segmentation} + +This part was done by the other group. It consists in partitioning the orignal image \ref{img} into two regions : the background with the chessboard in one part and the plush in the other part. With the binary mask that the segmentation allows us to extract \ref{binarymask} we will be able to create the 2D skeleton with the client's supply \ref{skeleton} and to match the skeleton's branches (see \ref{detection}). + +\begin{figure}[H] + \centering + \begin{subfigure}[b]{0.3\textwidth} + \includegraphics[width=\textwidth]{img/Poupee} + \caption{Original image} + \label{img} + \end{subfigure}% + ~ %add desired spacing between images, e. g. ~, \quad, \qquad, \hfill etc. + %(or a blank line to force the subfigure onto a new line) + \begin{subfigure}[b]{0.3\textwidth} + \includegraphics[width=\textwidth]{img/Poupeebin} + \caption{Binary mask} + \label{binarymask} + \end{subfigure} + ~ %add desired spacing between images, e. g. ~, \quad, \qquad, \hfill etc. + %(or a blank line to force the subfigure onto a new line) + \begin{subfigure}[b]{0.3\textwidth} + \includegraphics[width=\textwidth]{img/Poupeeskl} + \caption{Skeleton extract from (b)} + \label{skeleton} + \end{subfigure} + \caption{Pictures of animals}\label{segmentation} +\end{figure} \ No newline at end of file diff --git a/report/subsections/skeleton.tex b/report/subsections/skeleton.tex new file mode 100644 index 0000000..e70b5de --- /dev/null +++ b/report/subsections/skeleton.tex @@ -0,0 +1,13 @@ +\subsection{Skeleton} + +Bastien, our client, is working on the extraction of skeleton from pictures. A skeleton of an object is the elemental structure of it. In this project 2D-skeletons extracted, like in figure \ref{skeletonex}, are defined by a set of vertices and edges. The other group compute with it 2D-skeleton in another format : a set of B-Splines, each of them representing a branch of the skeleton. +B-Splines are given by : the degree of the spline, their control points and the nodes vectors. +The 3D-skeletons are given with 3D B-Splines with the same structure except that the control point are in 3D. +You will find in appendix A \ref{AppendixA} some examples of 2D and 3D skeletons. + +\begin{figure}[H] + \begin{center} + \includegraphics[scale=0.35]{img/SkeletonEx} + \caption{\label{skeletonex}Best fitting plane for the set of circles' center points} + \end{center} +\end{figure} \ No newline at end of file diff --git a/report/subsections/splines.tex b/report/subsections/splines.tex new file mode 100644 index 0000000..f0a79df --- /dev/null +++ b/report/subsections/splines.tex @@ -0,0 +1,39 @@ +The splines are a necessary element for everything that will follow. This is +why we tried to make it as generic as possible. For this, we used a C++11 +feature called \emph{variadic templates}. A spline is composed of control +points (represented as C++ tuples), a nodes vector and degree. The class we +made looks like this + +\begin{lstlisting}[language=c++] +template +class Spline +{ + public: + std::tuple operator()(float t); + std::tuple prime(float t); + + private: + std::vector> controlPoints; + std::vector nodes; + int degree; +}; +\end{lstlisting} + +We redefined the \texttt{operator()} to be able to \emph{evaluate} a spline at +a certain time, and the result will be an object which is a barycenter of those +control points. We also made a member function \texttt{prime} to be able to +compute the derivative of a spline. + +The \texttt{operator()} computes the barycent of the control points like this +$$C(t) = \sum_{i=0}^m P_i N_i^n(t)$$ +where $(P_i)_{i\in [[0,m]]}$ are the $m+1$ control points, $n$ is the degree of +the spline and +$$N_i^0(t) = \left\{\begin{matrix}1&\text{if } t \in [t_i, t_{i+1}] \\ 0 & \text{otherwise} \end{matrix} \right.$$ +$$ N_i^k(t) = \frac{t-t_i}{t_{i+k}-t_{i}} N_i^{k-1}(t) + \frac{t_{i+k+1} - t}{t_{i+k+1}-t_{i+1}} N_{i+1}^{k-1}(t) \quad\text{assuming} \quad \frac{0}{0} = 0$$ + +The \texttt{prime} member function computes the derivative like this +$$C'(t) = \sum_{i=0}^m N_i^{\prime n} (t) P_i $$ +with +$$ N_i^{\prime n}(t) = \frac{n}{t_{i+n}-t_{i}}N_i^{n-1}(t) - \frac{n}{t_{i+n+1}-t_{i+1}}N_{i+1}^{n-1}(t)$$ + +These formulas have been taken from the reference \cite{spline}.