48 lines
3.5 KiB
TeX
48 lines
3.5 KiB
TeX
|
\fresh{}
|
||
|
|
|
||
|
|
\section{Challenges}
|
||
|
|
|
||
|
|
The objective of our work is to design a system that allows a user to access remote 3D content that guarantees good quality of service and quality of experience.
|
||
|
|
A 3D streaming client has lots of tasks to accomplish:
|
||
|
|
|
||
|
|
\begin{itemize}
|
||
|
|
\item render a scene;
|
||
|
|
\item find out what to download and download it;
|
||
|
|
\item parse the result of the download;
|
||
|
|
\item add the parsed result to the scene;
|
||
|
|
\item manage the interaction with the user.
|
||
|
|
\end{itemize}
|
||
|
|
|
||
|
|
This brings multiple issues that we need to take care of.
|
||
|
|
|
||
|
|
To add to the challenge, we hope to have a client that would be portable between devices, and that would work easily on both desktop and mobile setups.
|
||
|
|
For this reason, we decided to implement a client in JavaScript, that uses THREE.js for rendering (which is based on WebGL) and even with the recent progress of browser based rendering, the fact that the client needs to be written in JavaScript already harms its performance.
|
||
|
|
|
||
|
|
\subsection{Streaming aspects}
|
||
|
|
|
||
|
|
Of course, such a system can exist only if it does streaming.
|
||
|
|
3D models grow in size as the 3D reconstruction tools advance, and even though network systems also advance, it is still unacceptable for a user to have to wait until all the model is downloaded to be able to visualise and interact with it.
|
||
|
|
Therefore, we have to face the standard issues that come naturally when talking about streaming, we need to find out what data to download and when to download it. This raises many questions, especially in 3D, since we have to deal with two main types of data: geometry and texture. A model can have a high resolution by having a high resolution geometry, with many small faces, or by having a high resolution texture, mapping a high resolution image on basic geometry. A client has to decide whether to download geometry or texture, and in which proportions.
|
||
|
|
|
||
|
|
\paragraph{}
|
||
|
|
We have to answer those questions while respecting constraints required for performant software:
|
||
|
|
|
||
|
|
\begin{itemize}
|
||
|
|
\item \textbf{our system must keep the computational load on the server low}, since a server must serve many clients, and a solution that requires even low computation on the server will scale difficultly;
|
||
|
|
\item \textbf{our system must keep the computational load on the client low}, since the end user will use his own device, whether it be an old computer or a mobile device, and has many other things to do than simply downloading content.
|
||
|
|
\end{itemize}
|
||
|
|
|
||
|
|
\subsection{Rendering aspects}
|
||
|
|
|
||
|
|
When rendering a 3D scene, the first step is to build vertex buffers and textures that are sent to the GPU\@.
|
||
|
|
Communicating between the CPU and the GPU is usually slow, but in most cases, it is not that big of a deal since once the data has been sent to the GPU, the rendering part is fast, and no more communication is needed: in old 3D video games (and even in some recent ones), we had a loading screen while the CPU was transmitting the information to the GPU\@.
|
||
|
|
|
||
|
|
In the streaming context however, this is an important concern.
|
||
|
|
Each time a portion of the model is downloaded, we need to send it to the GPU while at the same time rendering the scene.
|
||
|
|
|
||
|
|
|
||
|
|
\subsection{Interaction aspects}
|
||
|
|
|
||
|
|
We also have to provide the user with an interface that allows him to navigate in the scene.
|
||
|
|
This brings some extra work the client: it needs to manage input (for a desktop interface, this typically consist in keyboard and mouse inputs, for a mobile device, the touchscreen and the gyroscope), and add elements to the display that need to be updated and renderer.
|