More impl, more diargams

This commit is contained in:
Thomas Forgione 2019-09-23 12:04:36 +02:00
parent 1c294cd4fb
commit 53b7f43f10
No known key found for this signature in database
GPG Key ID: 203DAEA747F48F41
5 changed files with 160 additions and 55 deletions

1
.gitignore vendored
View File

@ -1,4 +1,5 @@
build
src/plan-chapter.tex
# Created by https://www.gitignore.io/api/osx,vim,latex,linux,windows
# Edit at https://www.gitignore.io/?templates=osx,vim,latex,linux,windows

View File

@ -0,0 +1,51 @@
let camera, scene, renderer;
let geometry, material, mesh;
init();
animate();
function init() {
// Computes the aspect ratio of the window.
let aspectRatio = window.innerWidth / window.innerHeight;
// Creates a camera and sets its parameters and position.
camera = new THREE.PerspectiveCamera(70, aspectRatio, 0.01, 10);
camera.position.z = 1;
// Creates the scene that contains our objects.
scene = new THREE.Scene();
// Creates a geometry (vertices and faces) corresponding to a cube.
geometry = new THREE.BoxGeometry(0.2, 0.2, 0.2);
// Creates a material that paints the faces depending on their normal.
material = new THREE.MeshNormalMaterial();
// Creates a mesh that associates the geometry with the material.
mesh = new THREE.Mesh(geometry, material);
// Adds the mesh to the scene.
scene.add(mesh);
// Creates the renderer and append its canvas to the DOM.
renderer = new THREE.WebGLRenderer({ antialias: true });
renderer.setSize(window.innerWidth, window.innerHeight);
document.body.appendChild(renderer.domElement);
}
// This function will be called at each frame.
function animate() {
// Makes the mesh rotate to have a nice animation
mesh.rotation.x += 0.01;
mesh.rotation.y += 0.02;
// Renders the scene with the camera.
renderer.render(scene, camera);
// Re-triggers animate() when the moment has come.
requestAnimationFrame(animate);
}

View File

@ -80,6 +80,15 @@ anchorcolor = blue]{hyperref}
\makesomeone{judge}{4}{Quatri\`eme MEMBRE}{Charg\'e de Recherche}{Membre du Jury}
\makesomeone{judge}{5}{Cinqui\`eme MEMBRE}{Charg\'e de Recherche}{Membre du Jury}
\lstdefinelanguage{JavaScript}{
keywords={break, case, catch, continue, debugger, default, delete, do, else, finally, for, function, if, in, instanceof, new, return, switch, this, throw, try, typeof, var, void, while, with, let},
morecomment=[l]{//},
morecomment=[s]{/*}{*/},
morestring=[b]',
morestring=[b]",
sensitive=true
}
\definecolor{mygreen}{RGB}{0,139,0}
\definecolor{mycolor1}{RGB}{148,0,211}
\definecolor{mycolor2}{RGB}{0,158,115}

View File

@ -1,6 +1,105 @@
\fresh{}
\section{JavaScript implementation}
\subsection{Media engine}
\subsubsection{THREE.js}
On the web browser, the best way to perform 3D rendering is to use WebGL\@.
However, WebGL is very low level and it can be really painful to write code, even to render a simple triangle.
For example, \href{https://www.tutorialspoint.com/webgl/webgl_drawing_a_triangle.htm}{this tutorial}'s code contains 121 lines of javascript, 46 being code (not comments or empty lines) to render a simple, non-textured triangle.
For this reason, it seems unreasonable to build a system like the one we are describing in raw WebGL\@.
There are many libraires that wrap WebGL code and that help people building 3D interfaces, and \href{https://threejs.org}{THREE.js} is probably one of the most popular.
THREE.js acts as a 3D engine built on WebGL\@.
It provides classes to deal with everything we need:
\begin{itemize}
\item the \textbf{Renderer} class contains all the WebGL code needed to render a scene on the web page;
\item the \textbf{Object} class contain all the boilerplate needed to manage the tree structure of the content, it contains a transform and it can have children that are other objects;
\item the \textbf{Scene} class is the root object, it contains all of the objects we want to render and it is passed as argument to the render function;
\item the \textbf{Geometry} and \textbf{BufferGeometry} classes are the classes that hold the vertices buffers, we will discuss that more in Section~\ref{d3i:geometries};
\item the \textbf{Material} class is the class that holds the properties used to render geometry (the most important information being the texture), there are many classes derived from Material, and the developper can choose what material he wants for its objects;
\item the \textbf{Mesh} class is the class that links the geometry and the material, it derives the Object class and can thus be added to a scene and renderer.
\end{itemize}
A snippet of the basic usage of these classes is given in Listing~\ref{d3i:three-hello-world}.
\begin{figure}[th]
\lstinputlisting[%
language=javascript,
caption={A THREE.js \emph{hello world}},
label=d3i:three-hello-world,
]{assets/dash-3d-implementation/base.js}
\end{figure}
\subsubsection{Geometries\label{d3i:geometries}}
Geometries are the classes that hold the vertices, texture coordinates, normals and faces.
There are two most important geometry classes in THREE.js:
\begin{itemize}
\item the \textbf{Geometry} class, which is made to be developper friendly and allows easy editing but can suffer issues of performance;
\item the \textbf{BufferGeometry} class, which is harder to use for a developper, but allows better performance since the developper controls and data is transmitted to the GPU\@.
\end{itemize}
Of course, in this work, we are concerned about performance of our system, and we will not be able to use normal geometries.
However, the way our system works, the way changes happen to the 3D content is always the same: we only add faces and textures to the model.
Therefore, we made a class that derives BufferGeometry, and that makes it more convenient for us.
\begin{itemize}
\item It has a constructor that takes as parameter the number of faces: it allocates all the memory needed for our buffers so we do not have to reallocate it which would be inefficient.
\item It keeps track of the number of faces it is currently holding: it can then avoid rendering faces that have not been filled and knows where to put new faces.
\item It provides a method that adds a face to the geometry.
\item It also keeps track of what part of the buffers has been transmitted to the GPU\@: THREE.js allows us to set the range of the buffer that we want to update and we are able to update only what is necessary.
\end{itemize}
\subsubsection{Our 3D model class}
As said in the previous sections, a geometry and a material a bound together in a mesh.
This means that we are forced to have has many meshes as there are materials in our model.
To make this easy to manage, we made a \textbf{Model} class, that holds everything we need.
We can add vertices, faces, and materials to this model, and it will internally deal with the right geometries, materials and meshes.
\subsection{Access client}
In order to be able to implement our DASH-3D client, we need to implement the access client, which is responsible for deciding what to download and download it.
\begin{figure}[ht]
\centering
\begin{tikzpicture}[scale=0.8]
\draw (-1, 0) rectangle (4, -4);
\draw (-1, -1) -- (4, -1);
\node at (1.5, -0.5) {\large DashClient};
\node[right] at (-1, -1.5) {loadNextSegment()};
\draw (4, -2) -- (6, -2);
\draw (6, 0) rectangle (10, -4);
\draw (6, -1) -- (10, -1);
\node at (8, -0.5) {\large LoadingPolicy};
\node[right] at (6, -1.5) {nextSegment()};
\draw (3, -6) rectangle (7, -10);
\draw (3, -7) -- (7, -7);
\node at (5, -6.5) {\large Greedy};
\node[right] at (3, -7.5) {nextSegment()};
\draw (9, -6) rectangle (13, -10);
\draw (9, -7) -- (13, -7);
\node at (11, -6.5) {\large GreedyPredictive};
\node[right] at (9, -7.5) {nextSegment()};
\draw (15, -6) rectangle (19, -10);
\draw (15, -7) -- (19, -7);
\node at (17, -6.5) {\large Proposed};
\node[right] at (15, -7.5) {nextSegment()};
\draw[-{Triangle[open, length=3mm, width=3mm]}] (5, -6) -- (5, -5) -- (8, -5) -- (8, -4);
\draw (11, -6) -- (11, -5) -- (8, -5);
\draw (17, -6) -- (17, -5) -- (8, -5);
\end{tikzpicture}
\caption{Class diagram of our DASH client\label{d3i:dash-loader}}
\end{figure}
\subsection{Performance}
In JavaScript, there is no way of doing parallel computing without using \emph{web workers}.
A web worker is a script in JavaScript that runs in the background, on a separate thread and that can communicate with the main script by sending and receiving messages.
Since our system has many tasks to do, it seems natural to use workers to manage the streaming without impacting the framerate of the renderer.

View File

@ -1,55 +0,0 @@
% This file is temporary. It's meant to be here while we are working on the
% plan, to make changing it easier. Once it has more or less converged, this
% file will be deleted, leaving place of a tree structure to make editing to
% content easier.
\part{3D Content Preparation}
\chapter{The challenges of managing 3D content}
\section{State of the art}
\begin{itemize}
\item Google Maps
\item Sketchfab
\end{itemize}
\section{System needs}
\subsection{Streaming}
Correctly managing the bandwidth to download the right content
\written{MMSys 16, ACMMM 18}
\subsection{Rendering}
Correctly managing the data on the client to perform an efficient rendering
\missing{everything}
\subsection{Server}
Avoiding as much as possible server-side computations
\written{MMSys 16, ACMMM 18}
\input{dash-3d/main.tex}
\part{3D Interaction}
\chapter{The challenges of 3D interaction}
\section{Degrees of freedom}
\section{Desktop / Mobile}
\written{MMSys 16, ACMMM 18, ACMMM 19 Demo}
\section{SoA}
\section{QoE / Interaction}
\written{MMSys 16}
\input{system-bookmarks/main}