217 lines
12 KiB
TeX
217 lines
12 KiB
TeX
\fresh{}
|
|
\section{JavaScript implementation}
|
|
|
|
\subsection{Media engine}
|
|
|
|
\subsubsection{THREE.js}
|
|
|
|
On the web browser, the best way to perform 3D rendering is to use WebGL\@.
|
|
However, WebGL is very low level and it can be really painful to write code, even to render a simple triangle.
|
|
For example, \href{https://www.tutorialspoint.com/webgl/webgl_drawing_a_triangle.htm}{this tutorial}'s code contains 121 lines of javascript, 46 being code (not comments or empty lines) to render a simple, non-textured triangle.
|
|
For this reason, it seems unreasonable to build a system like the one we are describing in raw WebGL\@.
|
|
There are many libraires that wrap WebGL code and that help people building 3D interfaces, and \href{https://threejs.org}{THREE.js} is probably one of the most popular.
|
|
|
|
THREE.js acts as a 3D engine built on WebGL\@.
|
|
It provides classes to deal with everything we need:
|
|
\begin{itemize}
|
|
\item the \textbf{Renderer} class contains all the WebGL code needed to render a scene on the web page;
|
|
\item the \textbf{Object} class contain all the boilerplate needed to manage the tree structure of the content, it contains a transform and it can have children that are other objects;
|
|
\item the \textbf{Scene} class is the root object, it contains all of the objects we want to render and it is passed as argument to the render function;
|
|
\item the \textbf{Geometry} and \textbf{BufferGeometry} classes are the classes that hold the vertices buffers, we will discuss that more in Section~\ref{d3i:geometries};
|
|
\item the \textbf{Material} class is the class that holds the properties used to render geometry (the most important information being the texture), there are many classes derived from Material, and the developper can choose what material he wants for its objects;
|
|
\item the \textbf{Mesh} class is the class that links the geometry and the material, it derives the Object class and can thus be added to a scene and renderer.
|
|
\end{itemize}
|
|
A snippet of the basic usage of these classes is given in Listing~\ref{d3i:three-hello-world}.
|
|
|
|
\begin{figure}[th]
|
|
\lstinputlisting[%
|
|
language=javascript,
|
|
caption={A THREE.js \emph{hello world}},
|
|
label=d3i:three-hello-world,
|
|
]{assets/dash-3d-implementation/base.js}
|
|
\end{figure}
|
|
|
|
\subsubsection{Geometries\label{d3i:geometries}}
|
|
|
|
Geometries are the classes that hold the vertices, texture coordinates, normals and faces.
|
|
There are two most important geometry classes in THREE.js:
|
|
\begin{itemize}
|
|
\item the \textbf{Geometry} class, which is made to be developper friendly and allows easy editing but can suffer issues of performance;
|
|
\item the \textbf{BufferGeometry} class, which is harder to use for a developper, but allows better performance since the developper controls and data is transmitted to the GPU\@.
|
|
\end{itemize}
|
|
|
|
Of course, in this work, we are concerned about performance of our system, and we will not be able to use normal geometries.
|
|
However, the way our system works, the way changes happen to the 3D content is always the same: we only add faces and textures to the model.
|
|
Therefore, we made a class that derives BufferGeometry, and that makes it more convenient for us.
|
|
\begin{itemize}
|
|
\item It has a constructor that takes as parameter the number of faces: it allocates all the memory needed for our buffers so we do not have to reallocate it which would be inefficient.
|
|
\item It keeps track of the number of faces it is currently holding: it can then avoid rendering faces that have not been filled and knows where to put new faces.
|
|
\item It provides a method that adds a face to the geometry.
|
|
\item It also keeps track of what part of the buffers has been transmitted to the GPU\@: THREE.js allows us to set the range of the buffer that we want to update and we are able to update only what is necessary.
|
|
\end{itemize}
|
|
|
|
\subsubsection{Our 3D model class}
|
|
|
|
As said in the previous sections, a geometry and a material a bound together in a mesh.
|
|
This means that we are forced to have has many meshes as there are materials in our model.
|
|
To make this easy to manage, we made a \textbf{Model} class, that holds everything we need.
|
|
We can add vertices, faces, and materials to this model, and it will internally deal with the right geometries, materials and meshes.
|
|
|
|
\subsection{Access client}
|
|
|
|
In order to be able to implement our DASH-3D client, we need to implement the access client, which is responsible for deciding what to download and download it.
|
|
|
|
\begin{figure}[ht]
|
|
\centering
|
|
\begin{tikzpicture}[scale=0.8]
|
|
\draw (-1, 0) rectangle (4, -4);
|
|
\draw (-1, -1) -- (4, -1);
|
|
\node at (1.5, -0.5) {\large DashClient};
|
|
\node[right] at (-1, -1.5) {loadNextSegment()};
|
|
|
|
\draw (4, -2) -- (6, -2);
|
|
|
|
\draw (6, 0) rectangle (10, -4);
|
|
\draw (6, -1) -- (10, -1);
|
|
\node at (8, -0.5) {\large LoadingPolicy};
|
|
\node[right] at (6, -1.5) {nextSegment()};
|
|
|
|
\draw (3, -6) rectangle (7, -10);
|
|
\draw (3, -7) -- (7, -7);
|
|
\node at (5, -6.5) {\large Greedy};
|
|
\node[right] at (3, -7.5) {nextSegment()};
|
|
|
|
\draw (9, -6) rectangle (13, -10);
|
|
\draw (9, -7) -- (13, -7);
|
|
\node at (11, -6.5) {\large GreedyPredictive};
|
|
\node[right] at (9, -7.5) {nextSegment()};
|
|
|
|
\draw (15, -6) rectangle (19, -10);
|
|
\draw (15, -7) -- (19, -7);
|
|
\node at (17, -6.5) {\large Proposed};
|
|
\node[right] at (15, -7.5) {nextSegment()};
|
|
|
|
\draw[-{Triangle[open, length=3mm, width=3mm]}] (5, -6) -- (5, -5) -- (8, -5) -- (8, -4);
|
|
\draw (11, -6) -- (11, -5) -- (8, -5);
|
|
\draw (17, -6) -- (17, -5) -- (8, -5);
|
|
\end{tikzpicture}
|
|
\caption{Class diagram of our DASH client\label{d3i:dash-loader}}
|
|
\end{figure}
|
|
|
|
\subsection{Performance}
|
|
|
|
In JavaScript, there is no way of doing parallel computing without using \emph{web workers}.
|
|
A web worker is a script in JavaScript that runs in the background, on a separate thread and that can communicate with the main script by sending and receiving messages.
|
|
Since our system has many tasks to do, it seems natural to use workers to manage the streaming without impacting the framerate of the renderer.
|
|
However, what a worker can do is very limited, since it cannot access the variables of the main script.
|
|
Because of this, we are forced to run the renderer on the main script, where it can access the HTML page, and we move all the other tasks to the worker (the access client, the control engine and the segment parsers), and since the main script is the one communicating with the GPU, it will still have to update the model with the parsed content it receives from the worker.
|
|
Using the worker does not so much improve the framerate of the system, but it reduces the latency that occurs when receiving a new segment, which can be very frustrating since in a single thread scenario, each time a segment is received, the interface freezes for around half a second.
|
|
A sequence diagram of what happens when downloading, parsing and rendering content is shown in Figure~\ref{d3i:sequence}.
|
|
|
|
\begin{figure}[ht]
|
|
|
|
\centering
|
|
|
|
\begin{tikzpicture}
|
|
\node at(0, 1) {Main script};
|
|
\draw[->, color=LightGray] (0, 0.5) -- (0, -17.5);
|
|
|
|
\node at(2.5, 1) {Worker};
|
|
\draw[->, color=LightGray] (2.5, 0.5) -- (2.5, -17.5);
|
|
|
|
\node at(10, 1) {Server};
|
|
\draw[->, color=LightGray] (10, 0.5) -- (10, -17.5);
|
|
|
|
% MPD
|
|
\draw[color=blue] (0, 0) -- (2.5, -0.1) -- (10, -0.5);
|
|
\draw[color=blue, fill=PaleLightBlue] (10, -0.5) -- (10, -1.5) -- (2.5, -2) -- (2.5, -1) -- cycle;
|
|
\node[color=blue, rotate=5] at (6.25, -1.25) {Download};
|
|
\node[color=blue, above] at(1.25, 0.0) {Ask MPD};
|
|
\draw[color=blue, fill=LightBlue] (2.375, -2) rectangle (2.625, -2.9);
|
|
\node[color=blue, right=0.2cm] at(2.5, -2.45) {Parse MPD};
|
|
\draw[color=blue] (2.5, -2.9) -- (0, -3);
|
|
\draw[color=blue, fill=LightBlue] (-0.125, -3.0) rectangle(0.125, -3.5);
|
|
\node[color=blue, left=0.2cm] at (0.0, -3.25) {Update model};
|
|
|
|
% Ask segments
|
|
\begin{scope}[shift={(0, -3.5)}]
|
|
\draw[color=red] (0, 0) -- (2.5, -0.1);
|
|
\node[color=red, above] at(1.25, 0.0) {Ask segment};
|
|
|
|
\draw[color=red, fill=Pink] (2.375, -0.1) rectangle (2.625, -1);
|
|
\node[color=red, right=0.2cm] at (2.5, -0.55) {Compute utilities};
|
|
|
|
\draw[color=red] (2.5, -1) -- (10, -1.5);
|
|
\draw[color=red, fill=PalePink] (10, -1.5) -- (10, -2.5) -- (2.5, -3) -- (2.5, -2) -- cycle;
|
|
\node[color=red, rotate=5] at (6.25, -2.25) {Download};
|
|
|
|
\draw[color=red, fill=Pink] (2.375, -3) rectangle (2.625, -3.9);
|
|
\node[color=red, right=0.2cm] at(2.5, -3.45) {Parse segment};
|
|
|
|
\draw[color=red] (2.5, -3.9) -- (0, -4);
|
|
\draw[color=red, fill=Pink] (-0.125, -4.0) rectangle(0.125, -4.5);
|
|
\node[color=red, left=0.2cm] at (0.0, -4.25) {Update model};
|
|
\end{scope}
|
|
|
|
% Ask more segments
|
|
\begin{scope}[shift={(0, -8)}]
|
|
\draw[color=DarkGreen] (0, 0) -- (2.5, -0.1);
|
|
\node[color=DarkGreen, above] at(1.25, 0.0) {Ask segment};
|
|
|
|
\draw[color=DarkGreen, fill=PaleGreen] (2.375, -0.1) rectangle (2.625, -1);
|
|
\node[color=DarkGreen, right=0.2cm] at (2.5, -0.55) {Compute utilities};
|
|
|
|
\draw[color=DarkGreen] (2.5, -1) -- (10, -1.5);
|
|
\draw[color=DarkGreen, fill=PalePaleGreen] (10, -1.5) -- (10, -2.5) -- (2.5, -3) -- (2.5, -2) -- cycle;
|
|
\node[color=DarkGreen, rotate=5] at (6.25, -2.25) {Download};
|
|
|
|
\draw[color=DarkGreen, fill=PaleGreen] (2.375, -3) rectangle (2.625, -3.9);
|
|
\node[color=DarkGreen, right=0.2cm] at(2.5, -3.45) {Parse segment};
|
|
|
|
\draw[color=DarkGreen] (2.5, -3.9) -- (0, -4);
|
|
\draw[color=DarkGreen, fill=PaleGreen] (-0.125, -4.0) rectangle(0.125, -4.5);
|
|
\node[color=DarkGreen, left=0.2cm] at (0.0, -4.25) {Update model};
|
|
\end{scope}
|
|
|
|
% Ask even more segments
|
|
\begin{scope}[shift={(0, -12.5)}]
|
|
\draw[color=purple] (0, 0) -- (2.5, -0.1);
|
|
\node[color=purple, above] at(1.25, 0.0) {Ask segment};
|
|
|
|
\draw[color=purple, fill=Plum] (2.375, -0.1) rectangle (2.625, -1);
|
|
\node[color=purple, right=0.2cm] at (2.5, -0.55) {Compute utilities};
|
|
|
|
\draw[color=purple] (2.5, -1) -- (10, -1.5);
|
|
\draw[color=purple, fill=PalePlum] (10, -1.5) -- (10, -2.5) -- (2.5, -3) -- (2.5, -2) -- cycle;
|
|
\node[color=purple, rotate=5] at (6.25, -2.25) {Download};
|
|
|
|
\draw[color=purple, fill=Plum] (2.375, -3) rectangle (2.625, -3.9);
|
|
\node[color=purple, right=0.2cm] at(2.5, -3.45) {Parse segment};
|
|
|
|
\draw[color=purple] (2.5, -3.9) -- (0, -4);
|
|
\draw[color=purple, fill=Plum] (-0.125, -4.0) rectangle(0.125, -4.5);
|
|
\node[color=purple, left=0.2cm] at (0.0, -4.25) {Update model};
|
|
\end{scope}
|
|
|
|
\foreach \x in {0,...,5}
|
|
{
|
|
\draw[color=Goldenrod, fill=LemonChiffon] (-0.125, -\x/2) rectangle (0.125, -\x/2-0.5);
|
|
}
|
|
\node[color=Goldenrod, left=0.2cm] at (0.0, -1.5) {Render};
|
|
|
|
\foreach \x in {0,...,7}
|
|
{
|
|
\draw[color=Goldenrod, fill=LemonChiffon] (-0.125, -\x/2-3.5) rectangle (0.125, -\x/2-4);
|
|
\draw[color=Goldenrod, fill=LemonChiffon] (-0.125, -\x/2-8) rectangle (0.125, -\x/2-8.5);
|
|
\draw[color=Goldenrod, fill=LemonChiffon] (-0.125, -\x/2-12.5) rectangle (0.125, -\x/2-13);
|
|
}
|
|
\node[color=Goldenrod, left=0.2cm] at (0.0, -5.5) {Render};
|
|
\node[color=Goldenrod, left=0.2cm] at (0.0, -10) {Render};
|
|
\node[color=Goldenrod, left=0.2cm] at (0.0, -14.5) {Render};
|
|
|
|
\end{tikzpicture}
|
|
|
|
\caption{Repartition of the tasks on the main script and the worker\label{d3i:sequence}}
|
|
|
|
\end{figure}
|