A 3D streaming system is a system that dynamically collects 3D data.
The previous chapter voluntarily remained vague about what \emph{3D data} actually is.
This chapter presents in detail the 3D data we consider and how it is rendered.
We also give insights about interaction and streaming by comparing the 3D case to the video one.

\section{What is a 3D model?\label{f:3d}}

\subsection{3D data}
Most classical 3D models are set of mesh and textures, that can potentially be arranged in a scene graph.
Such a model can typically contain the following:

\begin{itemize}
    \item \textbf{Vertices} are simply 3D points;
    \item \textbf{Faces} are polygons defined from vertices (most of the time, they are triangles);
    \item \textbf{Textures} are images that can be used for painting faces, to add visual richness;
    \item \textbf{Texture coordinates} are information added to a face, describing how the texture should be painted over faces;
    \item \textbf{Normals} are 3D vectors that can give information about light behaviour on a face.
\end{itemize}

The Wavefront OBJ is one of the most popular format that describes all these elements in text format.
A 3D model encoded in the OBJ format typically consists in two files: the materials file (\texttt{.mtl}) and the object file (\texttt{.obj}).

\paragraph{}
The materials file declares all the materials that the object file will reference.
A material consists in name, and other photometric properties such as ambient, diffuse and specular colors, as well as texture maps.
Each face corresponds to a material and a renderer can use the material's information to render the faces.
A simple material file is visible on Listing~\ref{i:mtl}.

\paragraph{}
The object file declares the 3D content of the objects.
It declares vertices, texture coordinates and normals from coordinates (e.g.\ \texttt{v 1.0 2.0 3.0} for a vertex, \texttt{vt 1.0 2.0} for a texture coordinate, \texttt{vn 1.0 2.0 3.0} for a normal).
These elements are numbered starting from 1.
Faces are declared by using the indices of these elements. A face is a polygon with an arbitrary number of vertices and can be declared in multiple manners:

\begin{itemize}
    \item \texttt{f 1 2 3} defines a triangle face that joins the first, the second and the third vertex declared;
    \item \texttt{f 1/1 2/3 3/4} defines a similar triangle but with texture coordinates, the first texture coordinate is associated to the first vertex, the third texture coordinate is associated to the second vertex, and the fourth texture coordinate is associated with the third vertex;
    \item \texttt{f 1//1 2//3 3//4} defines a similar triangle but referencing normals instead of texture coordinates;
    \item \texttt{f 1/1/1 2/3/3 3/4/4} defines a triangle with both texture coordinates and normals.
\end{itemize}

An object file can include materials from a material file (\texttt{mtllib path.mtl}) and apply the materials that it declares to faces.
A material is applied by using the \texttt{usemtl} keyword, followed by the name of the material to use.
The faces declared after a \texttt{usemtl} are painted using the material in question.
An example of object file is visible on Listing~\ref{i:obj}.

\begin{figure}[th]
    \centering
    \begin{subfigure}[c]{0.4\textwidth}
        \lstinputlisting[
            language=XML,
            caption={An object file describing a cube},
            label=i:obj,
        ]{assets/introduction/cube.obj}
    \end{subfigure}\quad%
    \begin{subfigure}[c]{0.4\textwidth}
        \lstinputlisting[
            language=XML,
            caption={A material file describing a material},
            label=i:mtl,
        ]{assets/introduction/materials.mtl}
        \includegraphics[width=\textwidth]{assets/introduction/cube.png}
        \captionof{figure}{A rendering of the cube}
    \end{subfigure}
    \caption{The OBJ representation of a cube and its render\label{i:cube}}
\end{figure}

\subsection{Rendering a 3D model\label{i:rendering}}

A typical 3D renderer follows Algorithm~\ref{f:renderer}.

\begin{algorithm}[th]
    \SetKwData{Material}{material}
    \SetKwData{Object}{object}
    \SetKwData{Geometry}{geometry}
    \SetKwData{Materials}{all\_materials}
    \SetKwData{Object}{object}
    \SetKwData{Scene}{scene}
    \SetKwData{True}{true}
    \SetKwFunction{LoadGeometry}{load\_geometry}
    \SetKwFunction{LoadMaterial}{load\_material}
    \SetKwFunction{BindMaterial}{bind\_material}
    \SetKwFunction{Draw}{draw}

    \tcc{Initialization}
    \For{$\Object\in\Scene$}{%
        \LoadGeometry{\Object.\Geometry}\;
        \LoadMaterial{\Object.\Material}\;
    }
    \BlankLine%
    \BlankLine%
    \tcc{Render loop}
    \While{\True}{%
        \For{$\Object\in\Scene$}{%
            \BindMaterial{\Object.\Material}\;
            \Draw{\Object.\Geometry}\;
        }
    }

    \caption{A rendering algorithm\label{f:renderer}}
\end{algorithm}

The first task the renderer needs to perform is sending the data to the GPU\@: this is done in the loading loop at the beginning.
This step can be slow, but it is generally acceptable since it only occurs once at the beginning of the program.
Then, the renderer starts the rendering loop: at each frame, it renders the whole scene: for each object, it binds the corresponding material to the GPU and then renders the object.
During the rendering loop, there are two things to consider regarding performances:
\begin{itemize}
    \item obviously, the more faces a geometry contains, the slower the \texttt{draw} call is;
    \item the more objects in the scene, the more overhead cause by the CPU/GPU communication at each step of the loop.
\end{itemize}

The way the loop works forces objects with different textures to be rendered separately.
An efficient renderer keeps the number of objects in a scene low to avoid introducing overhead.

However, an important feature of 3D engine regarding  performance is frustum culling.
The frustum is the viewing volume of the camera.
Frustum culling consists in avoiding rendering objects that are outside the viewing volume of the camera.
Algorithm~\ref{f:frustum-culling} is a variation of Algorithm~\ref{f:renderer} with frustum culling.

\begin{algorithm}[th]
    \SetKwData{Texture}{texture}
    \SetKwData{Object}{object}
    \SetKwData{Geometry}{geometry}
    \SetKwData{Textures}{all\_textures}
    \SetKwData{Object}{object}
    \SetKwData{Scene}{scene}
    \SetKwData{True}{true}
    \SetKwData{CameraFrustum}{camera\_frustum}
    \SetKwFunction{LoadGeometry}{load\_geometry}
    \SetKwFunction{LoadTexture}{load\_texture}
    \SetKwFunction{BindTexture}{bind\_texture}
    \SetKwFunction{Draw}{draw}

    \tcc{Initialization}
    \For{$\Object\in\Scene$}{%
        \LoadGeometry{\Object.\Geometry}\;
        \LoadTexture{\Object.\Texture}\;
    }
    \BlankLine%
    \BlankLine%
    \tcc{Render loop}
    \While{\True}{%
        \For{$\Object\in\Scene$}{%
            \If{$\Object\cap\CameraFrustum\neq\emptyset$}{%
                \BindTexture{\Object.\Texture}\;
                \Draw{\Object.\Geometry}\;
            }
        }
    }

    \caption{A rendering algorithm with frustum culling\label{f:frustum-culling}}
\end{algorithm}

A renderer that uses a single object avoids the overhead, but fails to benefit from frustum culling.
An optimized renderer needs to find a compromise between a too fine partition of the scene, which introduces overhead, and a too coarse partition which introduces useless rendering.

% ensures to have objects that do not spread across the whole scene, since that would lead to a useless frustum culling, and many objects to avoid rendering the whole scene at each frame.

% but not too many objects to avoid suffering from the overhead.

