158 lines
7.6 KiB
TeX
158 lines
7.6 KiB
TeX
A 3D streaming system is a system that progressively collects 3D data.
|
|
The previous chapter voluntarily remained vague about what \emph{3D data} actually are.
|
|
This chapter presents in detail the 3D data we consider and how they are rendered.
|
|
We also give insights about interaction and streaming by comparing the 3D setting to the video one.
|
|
|
|
\section{What is a 3D model?\label{f:3d}}
|
|
|
|
\subsection{3D data}
|
|
The 3D models we are interested in are sets of textured meshes, which can potentially be arranged in a scene graph.
|
|
Such models can typically contain the following:
|
|
|
|
\begin{itemize}
|
|
\item \textbf{Vertices}, which are 3D points,
|
|
\item \textbf{Faces}, which are polygons defined from vertices (most of the time, they are triangles),
|
|
\item \textbf{Textures}, which are images that can be used to paint faces in order to add visual richness,
|
|
\item \textbf{Texture coordinates}, which are information added to a face, describing how the texture should be painted over it,
|
|
\item \textbf{Normals}, which are 3D vectors that can give information about light behaviour on a face.
|
|
\end{itemize}
|
|
|
|
The Wavefront OBJ is a format that describes all these elements in text format.
|
|
A 3D model encoded in the OBJ format typically consists in two files: the material file (\texttt{.mtl}) and the object file (\texttt{.obj}).
|
|
|
|
\paragraph{}
|
|
The material file declares all the materials that the object file will reference.
|
|
A material consists in name, and other photometric properties such as ambient, diffuse and specular colors, as well as texture maps, which are images that are painted on faces.
|
|
Each face corresponds to a material.
|
|
A simple material file is visible on Snippet~\ref{i:mtl}.
|
|
|
|
\paragraph{}
|
|
The object file declares the 3D content of the objects.
|
|
It declares vertices, texture coordinates and normals from coordinates (e.g.\ \texttt{v 1.0 2.0 3.0} for a vertex, \texttt{vt 1.0 2.0} for a texture coordinate, \texttt{vn 1.0 2.0 3.0} for a normal).
|
|
These elements are numbered starting from 1.
|
|
Faces are declared by using the indices of these elements. A face is a polygon with an arbitrary number of vertices and can be declared in multiple manners:
|
|
|
|
\begin{itemize}
|
|
\item \texttt{f 1 2 3} defines a triangle face that joins the first, the second and the third declared vertices;
|
|
\item \texttt{f 1/1 2/3 3/4} defines a similar triangle but with texture coordinates, the first texture coordinate is associated to the first vertex, the third texture coordinate is associated to the second vertex, and the fourth texture coordinate is associated with the third vertex;
|
|
\item \texttt{f 1//1 2//3 3//4} defines a similar triangle but referencing normals instead of texture coordinates;
|
|
\item \texttt{f 1/1/1 2/3/3 3/4/4} defines a triangle with both texture coordinates and normals.
|
|
\end{itemize}
|
|
|
|
An object file can include materials from a material file (\texttt{mtllib path.mtl}) and apply the materials that it declares to faces.
|
|
A material is applied by using the \texttt{usemtl} keyword, followed by the name of the material to use.
|
|
The faces declared after a \texttt{usemtl} are painted using the material in question.
|
|
An example of object file is visible on Snippet~\ref{i:obj}.
|
|
|
|
\begin{figure}[th]
|
|
\centering
|
|
\begin{subfigure}[c]{0.4\textwidth}
|
|
\lstinputlisting[
|
|
language=XML,
|
|
caption={An object file describing a cube},
|
|
label=i:obj,
|
|
]{assets/introduction/cube.obj}
|
|
\end{subfigure}\quad%
|
|
\begin{subfigure}[c]{0.4\textwidth}
|
|
\lstinputlisting[
|
|
language=XML,
|
|
caption={A material file describing a material},
|
|
label=i:mtl,
|
|
]{assets/introduction/materials.mtl}
|
|
\includegraphics[width=\textwidth]{assets/introduction/cube.png}
|
|
% \captionof{figure}{A rendering of the cube}
|
|
\end{subfigure}
|
|
\caption{The OBJ representation of a cube and its render\label{i:cube}}
|
|
\end{figure}
|
|
|
|
\subsection{Rendering a 3D model\label{i:rendering}}
|
|
|
|
A typical 3D renderer follows Algorithm~\ref{f:renderer}.
|
|
\begin{algorithm}[th]
|
|
\SetKwData{Material}{material}
|
|
\SetKwData{Object}{object}
|
|
\SetKwData{Geometry}{geometry}
|
|
\SetKwData{Materials}{all\_materials}
|
|
\SetKwData{Object}{object}
|
|
\SetKwData{Scene}{scene}
|
|
\SetKwData{True}{true}
|
|
\SetKwFunction{LoadGeometry}{load\_geometry}
|
|
\SetKwFunction{LoadMaterial}{load\_material}
|
|
\SetKwFunction{BindMaterial}{bind\_material}
|
|
\SetKwFunction{Draw}{draw}
|
|
|
|
\tcc{Initialization}
|
|
\For{$\Object\in\Scene$}{%
|
|
\LoadGeometry{\Object.\Geometry}\;
|
|
\LoadMaterial{\Object.\Material}\;
|
|
}
|
|
\BlankLine%
|
|
\BlankLine%
|
|
\tcc{Render loop}
|
|
\While{\True}{%
|
|
\For{$\Object\in\Scene$}{%
|
|
\BindMaterial{\Object.\Material}\;
|
|
\Draw{\Object.\Geometry}\;
|
|
}
|
|
}
|
|
|
|
\caption{A rendering algorithm\label{f:renderer}}
|
|
\end{algorithm}
|
|
The first task the renderer needs to perform is sending the data to the GPU\@: this is done in the loading loop during an initialization step.
|
|
This step can be slow, but it is generally acceptable since it only occurs once at the beginning of the program.
|
|
Then, the renderer starts the rendering loop: at each frame, it renders the whole scene: for each object, it binds the corresponding material to the GPU and then renders the object.
|
|
During the rendering loop, there are two things to consider regarding performances:
|
|
\begin{itemize}
|
|
\item the more faces in a geometry, the slower the \texttt{draw} call;
|
|
\item the more objects in the scene, the more overhead caused by the CPU/GPU communication at each step of the loop.
|
|
\end{itemize}
|
|
|
|
The way the loop works forces objects with different materials to be rendered separately.
|
|
An efficient renderer keeps the number of objects in a scene low to avoid introducing overhead.
|
|
However, an important feature of 3D engines regarding performance is frustum culling.
|
|
The frustum is the viewing volume of the camera.
|
|
Frustum culling consists in skipping the objects that are outside the viewing volume of the camera in the rendering loop.
|
|
Algorithm~\ref{f:frustum-culling} is a variation of Algorithm~\ref{f:renderer} with frustum culling.
|
|
|
|
\begin{algorithm}[th]
|
|
\SetKwData{Texture}{texture}
|
|
\SetKwData{Object}{object}
|
|
\SetKwData{Geometry}{geometry}
|
|
\SetKwData{Textures}{all\_textures}
|
|
\SetKwData{Object}{object}
|
|
\SetKwData{Scene}{scene}
|
|
\SetKwData{True}{true}
|
|
\SetKwData{CameraFrustum}{camera\_frustum}
|
|
\SetKwFunction{LoadGeometry}{load\_geometry}
|
|
\SetKwFunction{LoadTexture}{load\_texture}
|
|
\SetKwFunction{BindTexture}{bind\_texture}
|
|
\SetKwFunction{Draw}{draw}
|
|
|
|
\tcc{Initialization}
|
|
\For{$\Object\in\Scene$}{%
|
|
\LoadGeometry{\Object.\Geometry}\;
|
|
\LoadTexture{\Object.\Texture}\;
|
|
}
|
|
\BlankLine%
|
|
\BlankLine%
|
|
\tcc{Render loop}
|
|
\While{\True}{%
|
|
\For{$\Object\in\Scene$}{%
|
|
\If{$\Object\cap\CameraFrustum\neq\emptyset$}{%
|
|
\BindTexture{\Object.\Texture}\;
|
|
\Draw{\Object.\Geometry}\;
|
|
}
|
|
}
|
|
}
|
|
|
|
\caption{A rendering algorithm with frustum culling\label{f:frustum-culling}}
|
|
\end{algorithm}
|
|
|
|
A renderer that uses a single object avoids the overhead, but fails to benefit from frustum culling.
|
|
An optimized renderer needs to find a compromise between a too fine partition of the scene, which introduces overhead, and a too coarse partition, which introduces useless rendering.
|
|
|
|
% ensures to have objects that do not spread across the whole scene, since that would lead to a useless frustum culling, and many objects to avoid rendering the whole scene at each frame.
|
|
|
|
% but not too many objects to avoid suffering from the overhead.
|
|
|