This commit is contained in:
Thomas Forgione 2019-09-25 17:18:35 +02:00
parent 47b0ddf540
commit 77e737dfa2
No known key found for this signature in database
GPG Key ID: 203DAEA747F48F41
14 changed files with 233 additions and 111 deletions

View File

@ -0,0 +1,10 @@
error[E0502]: cannot borrow `vec` as mutable because it is also borrowed as immutable
--> src/main.rs:4:9
|
3 | for value in &vec {
| ----
| |
| immutable borrow occurs here
| immutable borrow later used here
4 | vec.push(*value);
| ^^^^^^^^^^^^^^^^ mutable borrow occurs here

View File

@ -0,0 +1,3 @@
auto vec = std::vector<int> {1, 2, 3};
for (auto it = std::begin(vec); it < std::end(vec); it++)
vec.push_back(*it);

View File

@ -0,0 +1,10 @@
fn func() {
let mut vec = vec![1, 2, 3];
let iter = vec.iter();
loop {
match iter.next() {
Some(x) => vec.push(x),
None => break,
}
}
}

View File

@ -0,0 +1,3 @@
auto vec = std::vector<int> {1, 2, 3};
for (auto value: vec)
vec.push_back(value);

View File

@ -0,0 +1,6 @@
fn func() {
let mut vec = vec![1, 2, 3];
for value in &vec {
vec.push(value);
}
}

View File

@ -105,6 +105,15 @@ anchorcolor = blue]{hyperref}
sensitive=true
}
\lstdefinelanguage{Rust}{
keywords={break, continue, else, for, fn, if, in, return, self, while, let, loop, match, mut},
morecomment=[l]{//},
morecomment=[s]{/*}{*/},
morestring=[b]',
morestring=[b]",
sensitive=true
}
\definecolor{mygreen}{RGB}{0,139,0}
\definecolor{mycolor1}{RGB}{148,0,211}
\definecolor{mycolor2}{RGB}{0,158,115}

View File

@ -0,0 +1,72 @@
\fresh{}
\section{Clients}
All DASH clients are built from the same basic bricks, as shown in Figure~\ref{d3i:dash-scheme}:
\begin{itemize}
\item the \emph{access client}, which is the module that deals with making requests and receiving responses;
\item the \emph{segment parsers}, which decodes the data downloaded by the access client, whether it be materials, geometry or textures;
\item the \emph{control engine}, which analyses the bandwidth to dynamically adapt to it;
\item the \emph{media engine}, which renders the multimedia content and the user interface to the screen.
\end{itemize}
\begin{figure}[ht]
\centering
\begin{tikzpicture}
% Server
\draw[rounded corners=5pt,fill=Pink] (-10, 0) rectangle (-3, 7.5);
\node at (-9, 7) {Server};
% Segments
\begin{scope}[shift={(0.5,0.5)}]
\foreach \x in {0,...,3}
{
\draw [fill=Bisque](\x/2-7.5, 1.5-\x/2) rectangle (\x/2-5.5, 6-\x/2);
\node at (\x/2-6.5, 5.5-\x/2) {\fcolorbox{black}{LightBlue}{Segment}};
\node at (\x/2-6.5, 4.75-\x/2) {\fcolorbox{black}{LightBlue}{Segment}};
\draw [fill=LightBlue] (\x/2-6.5, 3.825-\x/2) circle (2pt) {};
\draw [fill=LightBlue] (\x/2-6.5, 3.325 -\x/2) circle (2pt) {};
\draw [fill=LightBlue] (\x/2-6.5, 2.825 -\x/2) circle (2pt) {};
\node at (\x/2-6.5, 2-\x/2) {\fcolorbox{black}{LightBlue}{Segment}};
}
\end{scope}
% MPD
\draw[fill=LightBlue] (-9.5, 6.5) rectangle (-7.5, 0.5);
\node at(-8.5, 3.5) {MPD};
% Client
\draw[rounded corners=5pt, fill=LemonChiffon] (-2, 0) rectangle (3, 7.5);
\node at (-0.5, 7) {DASH client};
% Access client
\draw[fill=PaleGreen] (-1.5, 0.5) rectangle (2.5, 1.5);
\node at (0.5, 1) {Access Client};
% Media engine
\draw[fill=PaleGreen] (-1.5, 5.5) rectangle (2.5, 6.5);
\node at (0.5, 6) {Media Engine};
% Control engine
\draw[fill=PaleGreen] (-1.5, 2) rectangle (0.25, 5);
\node[align=center] at (-0.625, 3.5) {Control \\ Engine};
% Segment parser
\draw[fill=PaleGreen] (0.75, 2) rectangle (2.5, 5);
\node[align=center] at (1.625, 3.5) {Segment \\ Parser};
% Access client to server
\draw[double arrow=5pt colored by RoyalBlue and white] (-3.25, 1.0) -- (-1.0, 1.0);
% Access client to control engine
\draw[double ended double arrow=5pt colored by RoyalBlue and white] (-0.625, 1.25) -- (-0.625, 2.5);
% Acces client to segment parser
\draw[double arrow=5pt colored by RoyalBlue and white] (1.625, 1.25) -- (1.625, 2.5);
% Segment parser to media engine
\draw[double arrow=5pt colored by RoyalBlue and white] (1.625, 4.5) -- (1.625, 5.75);
\end{tikzpicture}
\caption{DASH client-server architecture\label{d3i:dash-scheme}}
\end{figure}

View File

@ -1,86 +1,93 @@
\fresh{}
\section{Introduction}
In the previous chapter, we discussed the theoritical aspects of DASH based 3D streaming\@.
We showed different ways of structuring and downloading content, and we evaluated the parameters.
Implementation was noticeably absent from the previous chapter, and this is why, in this chapter, we detail every aspect of the implementation of the DASH-3D client, from the way segments are downloaded to how they are rendered.
Implementation was noticeably absent from the previous chapter, and this is why, in this chapter, we detail every aspect of the implementation of the DASH-3D client, from the way segments are built to how they are rendered through how they are downloaded.
All DASH clients are built from the same basic bricks, as shown in Figure~\ref{d3i:dash-scheme}:
\begin{itemize}
\item the \emph{access client}, which is the module that deals with making requests and receiving responses;
\item the \emph{segment parsers}, which decodes the data downloaded by the access client, whether it be materials, geometry or textures;
\item the \emph{control engine}, which analyses the bandwidth to dynamically adapt to it;
\item the \emph{media engine}, which renders the multimedia content and the user interface to the screen.
\end{itemize}
\begin{figure}[ht]
\centering
\begin{tikzpicture}
% Server
\draw[rounded corners=5pt,fill=Pink] (-10, 0) rectangle (-3, 7.5);
\node at (-9, 7) {Server};
% Segments
\begin{scope}[shift={(0.5,0.5)}]
\foreach \x in {0,...,3}
{
\draw [fill=Bisque](\x/2-7.5, 1.5-\x/2) rectangle (\x/2-5.5, 6-\x/2);
\node at (\x/2-6.5, 5.5-\x/2) {\fcolorbox{black}{LightBlue}{Segment}};
\node at (\x/2-6.5, 4.75-\x/2) {\fcolorbox{black}{LightBlue}{Segment}};
\draw [fill=LightBlue] (\x/2-6.5, 3.825-\x/2) circle (2pt) {};
\draw [fill=LightBlue] (\x/2-6.5, 3.325 -\x/2) circle (2pt) {};
\draw [fill=LightBlue] (\x/2-6.5, 2.825 -\x/2) circle (2pt) {};
\node at (\x/2-6.5, 2-\x/2) {\fcolorbox{black}{LightBlue}{Segment}};
}
\end{scope}
% MPD
\draw[fill=LightBlue] (-9.5, 6.5) rectangle (-7.5, 0.5);
\node at(-8.5, 3.5) {MPD};
% Client
\draw[rounded corners=5pt, fill=LemonChiffon] (-2, 0) rectangle (3, 7.5);
\node at (-0.5, 7) {DASH client};
% Access client
\draw[fill=PaleGreen] (-1.5, 0.5) rectangle (2.5, 1.5);
\node at (0.5, 1) {Access Client};
% Media engine
\draw[fill=PaleGreen] (-1.5, 5.5) rectangle (2.5, 6.5);
\node at (0.5, 6) {Media Engine};
% Control engine
\draw[fill=PaleGreen] (-1.5, 2) rectangle (0.25, 5);
\node[align=center] at (-0.625, 3.5) {Control \\ Engine};
% Segment parser
\draw[fill=PaleGreen] (0.75, 2) rectangle (2.5, 5);
\node[align=center] at (1.625, 3.5) {Segment \\ Parser};
% Access client to server
\draw[double arrow=5pt colored by RoyalBlue and white] (-3.25, 1.0) -- (-1.0, 1.0);
% Access client to control engine
\draw[double ended double arrow=5pt colored by RoyalBlue and white] (-0.625, 1.25) -- (-0.625, 2.5);
% Acces client to segment parser
\draw[double arrow=5pt colored by RoyalBlue and white] (1.625, 1.25) -- (1.625, 2.5);
% Segment parser to media engine
\draw[double arrow=5pt colored by RoyalBlue and white] (1.625, 4.5) -- (1.625, 5.75);
\end{tikzpicture}
\caption{DASH client-server architecture\label{d3i:dash-scheme}}
\end{figure}
We want to have two implementations of such a client.
All in all, we want three pieces of software.
\begin{itemize}
\item \textbf{We need a preprocessor}, that can take as input our models (we use the Wavefront OBJ format) and that prepares and structures the content and generates an MPD\@. This program is written in Rust.
\item \textbf{We need a client available on the web}, so we can easily have demos and conduct user-studies with real users trying the real interface on desktop or mobile devices (this implementation is detailed in Section~\ref{d3i:js-implementation}). This client is written in JavaScript.
\item \textbf{We need a client that compiles to native code}, so we can easily run simulations with maximum performance to be able to compare different setups or parameters with more precision (this implementation is detailed in Section~\ref{d3i:rust-implementation}). This client is written in Rust.
\end{itemize}
Our implementation also contains the software that preprocess the model and structures it in a DASH manner.
This piece of software was also implemented in Rust, and is detailed in Section~\ref{d3i:rust-exporter}.
\section{A note about Rust language}
During our work, we implemented the preprocessor as well as a client in Rust.
In this section, we explain the specifities of Rust and why it is a great language for those tasks.
\subsection{Specifity of Rust}
Rust is a system programming language focused on safety.
It is made to be performant (and effectively has performances comparable to C or C++) but with some nice features.
Developpers coming from C++ (as myself) might see it as a language like C++ but that forbids undefined behaviours.\footnote{in Rust, when you need to execute code that might lead to undefined behaviours, you need to put it inside an \texttt{unsafe} block. Many operations will not be available outside an \texttt{unsafe} block (e.g., dereferencing a pointer, or mutating a static variable). The idea is that you can use \texttt{unsafe} blocks when you require it, but you should avoid it as much as possible and when you do it, you must be particularly careful.}
The most powerful concept from Rust is \emph{ownership}.
Basically, every value has a variable that we call its \emph{owner}.
To be able to use a value, you must either be its owner or borrow it.
There are two types of borrow, the immutable borrow and the mutable borrow (people from C++ can see them as having a const reference or a reference to a variable).
The compiler comes with the \emph{borrow checker} which makes sure you only use variables that you are allowed to.
For example, the owner can only use the value if it is not being borrowed, and it is only possible to either borrow mutably a value once, or immutably borrow a value as many times as you want.
At first, the borrow checker seems particularly efficient to detect bugs in concurrent software, but in fact, it is also decisive in non concurrent code.
Consider the piece of C++ code in Listings~\ref{d3i:undefined-behaviour-cpp} and~\ref{d3i:undefined-behaviour-cpp-it}.
\begin{figure}[ht]
\centering
\begin{minipage}[c]{0.75\textwidth}
\lstinputlisting[
language=c++,
label={d3i:undefined-behaviour-cpp},
caption={Undefined behaviour: with for each syntax}
]{assets/dash-3d-implementation/undefined-behaviour.cpp}
\lstinputlisting[
language=c++,
label={d3i:undefined-behaviour-cpp-it},
caption={Undefined beheaviour with iterator syntax}
]{assets/dash-3d-implementation/undefined-behaviour-it.cpp}
\end{minipage}
\end{figure}
Of course, this loop should go undefintely because the vector grows in size as we add elements in the loop.
But the most important thing here is that since we add elements to the vector, it will eventually need to be reallocated, and that reallocation will invalidate the iterator, meaning that the following iterator will provoke an undefined behaviour.
The equivalent code in Rust is in Listings~\ref{d3i:undefined-behaviour-rs} and~\ref{d3i:undefined-behaviour-rs-it}.
\begin{figure}[ht]
\centering
\begin{minipage}[b]{0.45\textwidth}
\lstinputlisting[
language=rust,
caption={Rust version of Listing~\rawref{d3i:undefined-behaviour-cpp}},
label={d3i:undefined-behaviour-rs}
]{assets/dash-3d-implementation/undefined-behaviour.rs}
\end{minipage}
\quad\quad\quad
\begin{minipage}[b]{0.45\textwidth}
\lstinputlisting[
language=rust,
caption={Rust version of Listing~\rawref{d3i:undefined-behaviour-cpp-it}},
label={d3i:undefined-behaviour-rs-it}
]{assets/dash-3d-implementation/undefined-behaviour-it.rs}
\end{minipage}
\end{figure}
What happens is that the iterator needs to borrow of the vector.
Since it is borrowed, it can no longer be borrowed as mutable since mutating it could invalidate the other borrowers.
And effectively, the borrow checker will crash the compiler with the error in Listing~\ref{d3i:undefined-behaviour-rs-error}.
\begin{figure}[ht]
\lstinputlisting[
language=XML,
caption={Error given by the compiler on Listing~\ref{d3i:undefined-behaviour-rs}},
label={d3i:undefined-behaviour-rs-error}
]{assets/dash-3d-implementation/undefined-behaviour-error.txt}
\end{figure}
This example is one of the many examples of how powerful the borrow checker is: in Rust code, there can be no dangling reference, and all the segmentation faults coming from them are detected by the compiler.
The borrow checker may seem like an enemy to newcomers because it often rejects code that seem correct, but once you get used to it, you understand what is the problem with the code and either fix the problem easily, or realise that the whole architecture is wrong and understand why.
\subsection{Tooling}
Even better, Rust comes with great tooling.
\begin{itemize}
\item \href{https://github.com/rust-lang/rust}{\textbf{\texttt{rustc}}} is the Rust compiler. It is very confortable due to the nice error messages it displays.
\item \href{https://github.com/rust-lang/cargo}{\textbf{\texttt{cargo}}} is the official Rust's project and package manager. It manages compilation, dependencies, documentation, tests, etc\ldots
\item \href{https://github.com/racer-rust/racer}{\textbf{\texttt{racer}}}, \href{https://github.com/rust-lang/rls}{\textbf{\texttt{rls}} (Rust Language Server)} and \href{https://github.com/rust-analyzer/rust-analyzer}{\textbf{\texttt{rust-analyzer}}} are software that manage automatic compilation to display errors in code editors as well as providing semantic code completion.
\item \href{https://github.com/rust-lang/rustfmt}{\textbf{\texttt{rustfmt}}} auto formats code.
\item \href{https://github.com/rust-lang/rust-clippy}{\textbf{\texttt{clippy}}} is a linter that detects unidiomatic code and suggests modifications.
\end{itemize}
It is probably for those reasons that Rust is the \emph{most loved programming language} according to the Stack Overflow Developper Survey in~\citeyear{so-survey-2016,so-survey-2017,so-survey-2018} and~\citeyear{so-survey-2019}.

View File

@ -1,10 +1,9 @@
\fresh{}
\section{JavaScript client\label{d3i:js-implementation}}
\subsection{JavaScript client\label{d3i:js-implementation}}
\subsection{Media engine}
\subsubsection{THREE.js}
\subsubsection{Media engine}
\paragraph{THREE.js.}
On the web browser, the best way to perform 3D rendering is to use WebGL\@.
However, WebGL is very low level and it can be really painful to write code, even to render a simple triangle.
For example, \href{https://www.tutorialspoint.com/webgl/webgl_drawing_a_triangle.htm}{this tutorial}'s code contains 121 lines of javascript, 46 being code (not comments or empty lines) to render a simple, non-textured triangle.
@ -31,8 +30,7 @@ A snippet of the basic usage of these classes is given in Listing~\ref{d3i:three
]{assets/dash-3d-implementation/base.js}
\end{figure}
\subsubsection{Geometries\label{d3i:geometries}}
\paragraph{Geometries.\label{d3i:geometries}}
Geometries are the classes that hold the vertices, texture coordinates, normals and faces.
There are two most important geometry classes in THREE.js:
\begin{itemize}
@ -50,9 +48,8 @@ Therefore, we made a class that derives BufferGeometry, and that makes it more c
\item It also keeps track of what part of the buffers has been transmitted to the GPU\@: THREE.js allows us to set the range of the buffer that we want to update and we are able to update only what is necessary.
\end{itemize}
\subsubsection{Our 3D model class\label{d3i:model-class}}
As said in the previous sections, a geometry and a material a bound together in a mesh.
\paragraph{Our 3D model class.\label{d3i:model-class}}
As said in the previous subsections, a geometry and a material a bound together in a mesh.
This means that we are forced to have has many meshes as there are materials in our model.
To make this easy to manage, we made a \textbf{Model} class, that holds everything we need.
We can add vertices, faces, and materials to this model, and it will internally deal with the right geometries, materials and meshes.
@ -113,7 +110,7 @@ In order to avoid having many models that have the same material (which would ha
\caption{Restructuration of the content on the renderer\label{d3i:render-structure}}
\end{figure}
\subsection{Access client}
\subsubsection{Access client}
In order to be able to implement our DASH-3D client, we need to implement the access client, which is responsible for deciding what to download and download it.
To do so, we use the strategy pattern, as shown in Figure~\ref{d3i:dash-loader}.
@ -165,7 +162,7 @@ The \texttt{DashLoader} class accepts as parameter a function that will be calle
\caption{Class diagram of our DASH client\label{d3i:dash-loader}}
\end{figure}
\subsection{Performance}
\subsubsection{Performance}
In JavaScript, there is no way of doing parallel computing without using \emph{web workers}.
A web worker is a script in JavaScript that runs in the background, on a separate thread and that can communicate with the main script by sending and receiving messages.

View File

@ -6,11 +6,15 @@
\input{dash-3d-implementation/introduction}
\resetstyle{}
\input{dash-3d-implementation/rust-exporter}
\resetstyle{}
\input{dash-3d-implementation/clients}
\resetstyle{}
\input{dash-3d-implementation/js-implementation}
\resetstyle{}
\input{dash-3d-implementation/rust-implementation}
\resetstyle{}
\input{dash-3d-implementation/rust-exporter}
\resetstyle{}

View File

@ -1,6 +1,6 @@
\fresh{}
\section{Preprocessing\label{d3i:rust-exporter}}
\section{Content preparation\label{d3i:rust-exporter}}
The piece of software that does the preprocessing of the model mostly consists in file manipulation and is written is Rust as well.
It successively preprocess the geometry and then the textures.
@ -12,3 +12,17 @@ The MPD is generated by a library named \href{https://github.com/netvl/xml-rs}{x
\end{itemize}
This structure is passed along with our geometry and texture preprocessors that can add elements to the XML file as they are generating the corresponding files.
The other details of the preprocessing is available in Section~\ref{d3:dash-3d}.
\subsection{Geometry preparation}
The content preparation starts with the geometry.
It structures the faces of the model into a $k$-d tree, and then shrink the bounding boxes to avoid having empty space.
The cells of our $k$-d tree structure are our adaptation sets, so we traverse them and sort the faces by area and make segments, recording all the metadata in the MPD\@.
\subsection{Materials and texture preparation}
Then, we prepare the materials and the textures.
Exporting the materials files is quite straightforward: we simply add an attribute to each material specifying the number of faces painted with this materials in the model so we know the size of the buffers we need to allocate.
We then prepare the textures.
In OpenGL, texture sizes must be power of two.
So the first thing we do is resize the input texture to the next power of two.
We save it, add its metadata to the MPD, divide the size by two and keep doing this process until one of the dimension is less than 32 pixels.

View File

@ -1,25 +1,9 @@
\fresh{}
\section{Rust client\label{d3i:rust-implementation}}
\subsection{Rust client\label{d3i:rust-implementation}}
In order to run our simulation, we want to have a native implementation of our DASH client that is performant and reliable.
We chose to use Rust to accomplish this.
\subsection{Rust language}
Rust is a system programming language focused on safety.
Developpers coming from C++ (as myself) might see it as a language like C++ but that forbids undefined behaviours.\footnote{in Rust, when you need to execute code that might lead to undefined behaviours, you need to put it inside an \texttt{unsafe} block. Many operations will not be available outside an \texttt{unsafe} block (e.g., dereferencing a pointer, or mutating a static variable). The idea is that you can use \texttt{unsafe} blocks when you require it, but you should avoid it as much as possible.}
This makes segmentation fault extremely rare, and it also makes the compiler giving comprehensible error messages instead of letting code compile that would crash on the first run.
Other concepts (such as concepts from functional programming) make it very confortable to write Rust code.
Furthermore, Rust comes with great tooling.
\begin{itemize}
\item \href{https://github.com/rust-lang/rust}{\textbf{\texttt{rustc}}} is the Rust compiler. It is very confortable due to the nice error messages it displays.
\item \href{https://github.com/rust-lang/cargo}{\textbf{\texttt{cargo}}} is the official Rust's project and package manager. It manages compilation, dependencies, documentation, tests, etc\ldots
\item \href{https://github.com/racer-rust/racer}{\textbf{\texttt{racer}}}, \href{https://github.com/rust-lang/rls}{\textbf{\texttt{rls}} (Rust Language Server)} and \href{https://github.com/rust-analyzer/rust-analyzer}{\textbf{\texttt{rust-analyzer}}} are software that manage automatic compilation to display errors in code editors as well as providing semantic code completion.
\item \href{https://github.com/rust-lang/rustfmt}{\textbf{\texttt{rustfmt}}} auto formats code.
\item \href{https://github.com/rust-lang/rust-clippy}{\textbf{\texttt{clippy}}} is a linter that detects unidiomatic code and suggests modifications.
\end{itemize}
It is probably for those reasons that Rust is the \emph{most loved programming language} according to the Stack Overflow Developper Survey in~\citeyear{so-survey-2016,so-survey-2017,so-survey-2018} and~\citeyear{so-survey-2019}.
\subsection{Specific requirements}
Our requirements are quite different that the ones we had to deal with in our JavaScript implementation.
In this setup, we want to build a system that is the closest to our theoritical concepts.
Therefore, we do not have a full client in Rust (meaning an application to which you would give the URL to an MPD file and that would allow you to navigate in the scene while it is being downloaded).

View File

@ -1,8 +1,11 @@
\documentclass[
paper=a4,
pagesize,
bibliography=totoc,
headings=big,
captions=tableheading,
chapterprefix=true% like in standard class "report"
chapterprefix=true,% like in standard class "report"
BCOR=2cm,
]{scrbook}
\usepackage{commands}

View File

@ -16,7 +16,7 @@ However, the system described in this chapter has some drawbacks and fails to an
\begin{itemize}
\item \textbf{The content preparation and chunk utility is almost inexistant}: the server knows all the data and simply determines what the client needs, he prepares the content and builds chunk on the go. Furthermore, it has no support for material or textures: in our setup, they are downloaded before the streaming starts.
\item \textbf{The streaming policy is basic}: the server traverses all the polygons and determines which polygons should be sent to the client.
\item \textbf{The implementation has sloppy\note{maybe \emph{sloppy} is too strong} performances}: since the content preparation and the streaming policy is made on the fly, the server has to keep track of what the client already has (which will eat memory) and has to compute what should be sent next (which will eat CPU). The scalability of such a server is therefore not possible. Moreover, no client performance has been taken into account since the client used in the user-study did not have to perform streaming.
\item \textbf{The implementation has sloppy\todo{maybe \emph{sloppy} is too strong} performances}: since the content preparation and the streaming policy is made on the fly, the server has to keep track of what the client already has (which will eat memory) and has to compute what should be sent next (which will eat CPU). The scalability of such a server is therefore not possible. Moreover, no client performance has been taken into account since the client used in the user-study did not have to perform streaming.
\end{itemize}
After learning these lessons, we show, in the next chapter, what is possible to do in order to alleviate these issues.