This commit is contained in:
2019-10-16 17:01:56 +02:00
parent 8b9303b1a5
commit 8e686dc040
8 changed files with 40 additions and 42 deletions

View File

@@ -1,6 +1,6 @@
A 3D streaming system is a system that dynamically collects 3D data.
The previous chapter voluntarily remained vague about what \emph{3D data} actually is.
This chapter presents in detail the 3D data we consider and how it is renderer.
This chapter presents in detail the 3D data we consider and how it is rendered.
We also give insights about interaction and streaming by comparing the 3D case to the video one.
\section{What is a 3D model?\label{f:3d}}
@@ -13,7 +13,7 @@ Such a model can typically contain the following:
\item \textbf{Vertices} are simply 3D points;
\item \textbf{Faces} are polygons defined from vertices (most of the time, they are triangles);
\item \textbf{Textures} are images that can be used for painting faces, to add visual richness;
\item \textbf{Texture coordinates} are information added to a face, describing how the texture should be applied on a face;
\item \textbf{Texture coordinates} are information added to a face, describing how the texture should be painted over faces;
\item \textbf{Normals} are 3D vectors that can give information about light behaviour on a face.
\end{itemize}
@@ -23,7 +23,7 @@ A 3D model encoded in the OBJ format typically consists in two files: the materi
\paragraph{}
The materials file declares all the materials that the object file will reference.
A material consists in name, and other photometric properties such as ambient, diffuse and specular colors, as well as texture maps.
Each face correspond to a material and a renderer can use the material's information to render the faces.
Each face corresponds to a material and a renderer can use the material's information to render the faces.
A simple material file is visible on Listing~\ref{i:mtl}.
\paragraph{}

View File

@@ -1,10 +1,10 @@
\fresh{}
\section{Implementation details}
During this thesis, a lot of software has been developed, and for this software to be successful and efficient, we took care of choosing the appropriate languages.
During this thesis, a lot of software has been developed, and for this software to be successful and efficient, we chose the appropriate languages.
When it comes to 3D streaming systems, there are two kind of software that we need.
\begin{itemize}
\item \textbf{Interactive applications} that can run on as many devices as possible whether it be desktop or mobile in order to try and to conduct user studies. For this context, we chose the \textbf{JavaScript language}, since it can run on many devices and it has great support for WebGL\@.
\item \textbf{Interactive applications} that can run on as many devices as possible whether it be desktop or mobile in order to conduct user studies. For this context, we chose the \textbf{JavaScript language}, since it can run on many devices and it has great support for WebGL\@.
\item \textbf{Native applications} that can run fast on desktop devices, in order to run simulations and evaluate our ideas. For this context, we chose the \textbf{Rust} language, which is a somewhat recent language that provides both the efficiency of C and C++ and the safety of functional languages.
\end{itemize}
@@ -21,9 +21,9 @@ THREE.js acts as a 3D engine built on WebGL\@.
It provides classes to deal with everything we need:
\begin{itemize}
\item the \textbf{Renderer} class contains all the WebGL code needed to render a scene on the web page;
\item the \textbf{Object} class contains all the boilerplate needed to manage the tree structure of the content, it contains a transform and it can have children that are other objects;
\item the \textbf{Object} class contains all the boilerplate needed to manage the tree structure of the content, it contains a transform (translation and rotation) and it can have children that are other objects;
\item the \textbf{Scene} class is the root object, it contains all of the objects we want to render and it is passed as argument to the render function;
\item the \textbf{Geometry} and \textbf{BufferGeometry} classes are the classes that hold the vertices buffers, we will discuss that more in Section~\ref{f:geometries};
\item the \textbf{Geometry} and \textbf{BufferGeometry} classes are the classes that hold the vertices buffers, we will discuss it more in Section~\ref{f:geometries};
\item the \textbf{Material} class is the class that holds the properties used to render geometry (the most important information being the texture), there are many classes derived from Material, and the developer can choose what material he wants for its objects;
\item the \textbf{Mesh} class is the class that links the geometry and the material, it derives the Object class and can thus be added to a scene and renderer.
\end{itemize}
@@ -41,7 +41,7 @@ A snippet of the basic usage of these classes is given in Listing~\ref{f:three-h
Geometries are the classes that hold the vertices, texture coordinates, normals and faces.
There are two most important geometry classes in THREE.js:
\begin{itemize}
\item the \textbf{Geometry} class, which is made to be developer friendly and allows easy editing but can suffer from issues of performance;
\item the \textbf{Geometry} class, which is made to be developer friendly and allows easy editing but can suffer from performance issues;
\item the \textbf{BufferGeometry} class, which is harder to use for a developer, but allows better performance since the developer controls how data is transmitted to the GPU\@.
\end{itemize}
@@ -52,12 +52,12 @@ In this section, we explain the specificities of Rust and why it is a great lang
\subsubsection{Specificity of Rust}
Rust is a system programming language focused on safety.
It is made to be efficient (and effectively has performances comparable to C or C++) but with some nice features.
Developers coming from C++ (as myself) might see it as a language like C++ but that forbids undefined behaviours.\footnote{in Rust, when you need to execute code that might lead to undefined behaviours, you need to put it inside an \texttt{unsafe} block. Many operations will not be available outside an \texttt{unsafe} block (e.g., dereferencing a pointer, or mutating a static variable). The idea is that you can use \texttt{unsafe} blocks when you require it, but you should avoid it as much as possible and when you do it, you must be particularly careful.}
It is made to be efficient (and effectively has performances comparable to C or C++) but with some extra features.
C++ users might see it as a language like C++ but that forbids undefined behaviours.\footnote{in Rust, when you need to execute code that might lead to undefined behaviours, you need to put it inside an \texttt{unsafe} block. Many operations will not be available outside an \texttt{unsafe} block (e.g., dereferencing a pointer, or mutating a static variable). The idea is that you can use \texttt{unsafe} blocks when you require it, but you should avoid it as much as possible and when you do it, you must be particularly careful.}
The most powerful concept from Rust is \emph{ownership}.
Basically, every value has a variable that we call its \emph{owner}.
To be able to use a value, you must either be its owner or borrow it.
There are two types of borrow, the immutable borrow and the mutable borrow (people from C++ can see them as having a const reference or a reference to a variable).
There are two types of borrow, the immutable borrow and the mutable borrow (people from C++ can see them as having a reference to a variable).
The compiler comes with the \emph{borrow checker} which makes sure you only use variables that you are allowed to.
For example, the owner can only use the value if it is not being borrowed, and it is only possible to either borrow mutably a value once, or immutably borrow a value as many times as you want.
At first, the borrow checker seems particularly efficient to detect bugs in concurrent software, but in fact, it is also decisive in non concurrent code.
@@ -74,7 +74,7 @@ Consider the piece of C++ code in Listings~\ref{f:undefined-behaviour-cpp} and~\
\lstinputlisting[
language=c++,
label={f:undefined-behaviour-cpp-it},
caption={Undefined beheaviour with iterator syntax}
caption={Undefined behaviour with iterator syntax}
]{assets/dash-3d-implementation/undefined-behaviour-it.cpp}
\end{minipage}
\end{figure}

View File

@@ -36,13 +36,13 @@ It can be chosen directly by the user or automatically determined by analysing t
\caption{The different resolutions available for a Youtube video}
\end{figure}
Similarly, recent work in 3D streaming have proposed different ways to progressively streaming 3D models, displaying a low resolution to the user without latency, and supporting interaction with the model while the details are being downloaded.
Similarly, recent work in 3D streaming have proposed different ways to progressively stream 3D models, displaying a low resolution to the user without latency, and supporting interaction with the model while details are being downloaded.
Such strategies are reviewed in Section~\ref{sote:3d-streaming}.
\subsection{Media types}
Just like a video, a 3D scene is composed of different types of media.
In video, those media typically are images, sounds, and eventually subtitles, whereas in 3D, those media typically are geometry or textures.
In video, those media are mostly images, sounds, and eventually subtitles, whereas in 3D, those media are geometry or textures.
In both cases, an algorithm for content streaming has to acknowledge those different media types and manage them correctly.
In video streaming, most of the data (in terms of bytes) is used for images.
@@ -52,9 +52,9 @@ This is one of the main differences between video and 3D streaming: in a 3D scen
\subsection{Interaction}
The ways of interacting with the content is probably the most important difference between video and 3D.
The ways of interacting with content is probably the most important difference between video and 3D.
In a video interface, there is only one degree of freedom: time.
The only thing a user can do is letting the video play, pausing, resuming, or jumping to another time in the video.
The only things a user can do is letting the video play, pausing, resuming, or jumping to another time in the video.
Even though these interactions seem easy to handle, giving the best possible experience to the user is already challenging. For example, to perform these few actions, Youtube provides the user with multiple options.
\begin{itemize}
@@ -301,7 +301,7 @@ When it comes to 3D, there are many approaches to manage user interaction.
Some interfaces mimic the video scenario, where the only variable is the time and the camera follows a predetermined path on which the user has no control.
These interfaces are not interactive, and can be frustrating to the user who might feel constrained.
Some other interfaces add 2 degrees of freedom to the timeline: the user does not control the position of the camera but they can control the angle. This mimics the scenario of the 360 video.
Some other interfaces add 2 degrees of freedom to the timeline: the user does not control the position of the camera but can control the angle. This mimics the scenario of the 360 video.
This is typically the case of the video game \emph{nolimits 2: roller coaster simulator} which works with VR devices (oculus rift, HTC vive, etc.) where the only interaction the user has is turning their head.
Finally, most of the other interfaces give at least 5 degrees of freedom to the user: 3 being the coordinates of the position of the camera, and 2 being the angle (assuming the up vector is unchangeable, some interfaces might allow that, giving a sixth degree of freedom).