dash-3d-tmp
/
phd
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Did stuff

This commit is contained in:
Thomas Forgione 2019-10-15 11:55:15 +02:00
parent 4cc06d47de
commit 2b0aae3430
No known key found for this signature in database
GPG Key ID: 203DAEA747F48F41
4 changed files with 42 additions and 40 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
src/dash-3d/main.tex
View File

@ -17,9 +17,12 @@ In this chapter, we present the most important contribution of this thesis: adap
We start by showing how we prepare 3D data into a format that complies with DASH and that stores enough metadata to enable a client to perform efficient streaming: we partition the scene into a $k$-d tree and we further segment each cell into chunks with a fixed number of faces, which are sorted by area so that faces of a different level of detail are not grouped together. We start by showing how we prepare 3D data into a format that complies with DASH and that stores enough metadata to enable a client to perform efficient streaming: we partition the scene into a $k$-d tree and we further segment each cell into chunks with a fixed number of faces, which are sorted by area so that faces of a different level of detail are not grouped together.
We also export each texture at different resolution, and we encode all the acquired metadata into a 3D version of the Media Presentation Description (MPD) that DASH uses for video. We also export each texture at different resolution, and we encode all the acquired metadata into a 3D version of the Media Presentation Description (MPD) that DASH uses for video.
Namely, we store in the metadata the coordinates of the cells of the $k$-d tree, the areas of geometry chunks, and the average colors of textures. All this prepared content is then stored on a simple static HTTP server: a clients can request the content without any need for computation on the server side, allowing a server to support an arbitrary number of clients.
% Namely, we store in the metadata the coordinates of the cells of the $k$-d tree, the areas of geometry chunks, and the average colors of textures.
We then propose a standard client that can perform frustum culling to eliminate cells outside the viewing volume of the camera (as shown in Figure~\ref{d3:big-picture}), and we define a few utility metrics to give scores to each chunk of data, and a few streaming policies that rely on those utilities to determine which chunks need to be downloaded. We then propose a DASH-3D client that performs frustum culling to eliminate cells outside the viewing volume of the camera (as shown in Figure~\ref{d3:big-picture}).
We define utility metrics to give scores to each chunk of data, be it geometry or texture, based on offline information that is given in the MPD, and online information that the client is able to compute, such as user interaction or bandwidth measurements.
We also define streaming policies that rely on those utilities in order for the client to determine which chunks need to be downloaded.
We finally evaluate all those parameters under different bandwidth setups and compare our streaming policies. We finally evaluate all those parameters under different bandwidth setups and compare our streaming policies.
\newpage \newpage

23
src/preliminary-work/bookmarks-impact.tex
View File

@ -22,29 +22,6 @@ The mouse movement controls the camera rotation.
The user can always choose to lock the pointer, or unlock it using the escape key. The user can always choose to lock the pointer, or unlock it using the escape key.
The interface also includes a button to reset the camera back to the starting position in the scene. The interface also includes a button to reset the camera back to the starting position in the scene.
\begin{figure}[th]
\centering
\begin{subfigure}[b]{0.45\textwidth}
\includegraphics[width=\textwidth]{assets/preliminary-work/bookmarks/camera-bookmark.png}
\caption{Viewport display of a bookmark}
\end{subfigure}
\begin{subfigure}[b]{0.45\textwidth}
\includegraphics[width=\textwidth]{assets/preliminary-work/bookmarks/arrow-bookmark.png}
\caption{Arrow display of a bookmark}
\end{subfigure}
\begin{subfigure}[b]{0.45\textwidth}
\includegraphics[width=\textwidth]{assets/preliminary-work/bookmarks/arrow-bookmark-with-preview.png}
\caption{A preview is shown when the mouse hovers over a bookmark}
\end{subfigure}
\begin{subfigure}[b]{0.45\textwidth}
\includegraphics[width=\textwidth]{assets/preliminary-work/bookmarks/arrow-bookmark.png}
\caption{A coin is hidden behind the curtain\newline}
\end{subfigure}
\caption{3D bookmarks propose to move to a new viewpoint; when the user clicks on the bookmark, his viewpoint moves to the indicated viewpoint.\label{bi:bookmark}}
\end{figure}
\subsection{3D Bookmarks\label{bi:3d-bookmarks}} \subsection{3D Bookmarks\label{bi:3d-bookmarks}}
Our NVE supports 3D bookmarks. Our NVE supports 3D bookmarks.
A 3D bookmark, or bookmark for short, is simply a fixed camera location (in 3D space), a view direction, and a focal. A 3D bookmark, or bookmark for short, is simply a fixed camera location (in 3D space), a view direction, and a focal.

23
src/preliminary-work/conclusion.tex
View File

@ -2,21 +2,20 @@
\section{Conclusion\label{bi:conclusion}} \section{Conclusion\label{bi:conclusion}}
In this chapter, we have described a basic interface that allows a user to navigate in a scene that is being streamed. In this chapter, we have described an interface that allows a user to navigate in a scene that is being streamed.
It allowed us to understand the problems linked to the dynamics of both the user behaviour and the 3D content: We identified and addressed the problems linked to the dynamics of both the user behaviour and the 3D content:
\begin{itemize} \begin{itemize}
\item Navigating in a 3D scene can be complex, due to the many degrees of freedom, and tweaking the interface can increase the user's Quality of Experience. \item Navigating in a 3D scene can be complex, due to the many degrees of freedom, and tweaking the interface can increase the user's Quality of Experience.
\item Adding bookmarks to the interface can have a drawback on the quality of service of the system. \item Adding bookmarks to the interface increases the quality of experience of the users and makes them visiting more data in the same amount of time.
\item Having bookmarks in the scene biases the users navigation and make them more predictable: it is then possible to precompute data from bookmarks in order to benefit from this predictability. \item This increase in speed of navigation has a negative impact on the quality of service of the system.
\item Having bookmarks in the scene biases the users navigation and makes the navigation more predictable: it is possible to link data utility to bookmarks in order to benefit from this predictability.
\end{itemize} \end{itemize}
However, the system described in this chapter has some drawbacks and fails to answer some of the problems we mentioned in Section~\ref{i:challenges}. However, the system described in this chapter has some drawbacks and fails to answer the problems we mentioned in Section~\ref{i:challenges}, and all these problems come from the fact that \textbf{the content preparation is inexistent}.
The server knows all the data and simply determines what the client needs, it prepares the content and builds chunks on the go.
Thus, he server has to keep track of what the client already has (which will eat memory) and has to compute what should be sent next (which will eat CPU). The scalability of such a server is therefore not possible.
Furthermore, we only considered geometry streaming: materials and textures are downloaded before the streaming starts, which causes great latency at the start of the streaming and harms the quality of experience.
\begin{itemize} After learning these lessons, we describe, in the next chapter, what is possible to do in order to alleviate these issues.
\item \textbf{The content preparation and chunk utility is almost inexistent}: the server knows all the data and simply determines what the client needs, it prepares the content and builds chunk on the go. Furthermore, it has no support for material or textures: in our setup, they are downloaded before the streaming starts. We show how the standard for video steaming, DASH, teaches us to prepare 3D content in order to remove all server side computations, to elaborate great streaming policies, and to support both geometry and texture chunks.
\item \textbf{The streaming policy is basic}: the server traverses all the polygons and determines which polygons should be sent to the client.
\item \textbf{The implementation has sloppy\todo{maybe \emph{sloppy} is too strong} performances}: since the content preparation and the streaming policy is made on the fly, the server has to keep track of what the client already has (which will eat memory) and has to compute what should be sent next (which will eat CPU). The scalability of such a server is therefore not possible. Moreover, no client performance has been taken into account since the client used in the user-study did not have to perform streaming.
\end{itemize}
After learning these lessons, we show, in the next chapter, what is possible to do in order to alleviate these issues.

29
src/preliminary-work/main.tex
View File

@ -5,16 +5,39 @@
\fresh{} \fresh{}
Bookmarks are virtual objects that represent a point of view, that are rendered with the scene and that can help user navigation. \begin{figure}[th]
\centering
\begin{subfigure}[b]{0.45\textwidth}
\includegraphics[width=\textwidth]{assets/preliminary-work/bookmarks/camera-bookmark.png}
\caption{Viewport display of a bookmark}
\end{subfigure}
\begin{subfigure}[b]{0.45\textwidth}
\includegraphics[width=\textwidth]{assets/preliminary-work/bookmarks/arrow-bookmark.png}
\caption{Arrow display of a bookmark}
\end{subfigure}
\begin{subfigure}[b]{0.45\textwidth}
\includegraphics[width=\textwidth]{assets/preliminary-work/bookmarks/arrow-bookmark-with-preview.png}
\caption{A preview is shown when the mouse hovers over a bookmark}
\end{subfigure}
\begin{subfigure}[b]{0.45\textwidth}
\includegraphics[width=\textwidth]{assets/preliminary-work/bookmarks/arrow-bookmark.png}
\caption{A coin is hidden behind the curtain\newline}
\end{subfigure}
\caption{3D bookmarks propose to move to a new viewpoint; when the user clicks on the bookmark, his viewpoint moves to the indicated viewpoint.\label{bi:bookmark}}
\end{figure}
% Bookmarks are virtual objects that represent a point of view, that are rendered with the scene and that can help user navigation.
In this chapter, we present our first contribution: an analysis of the impact of bookmarks on navigation and streaming. In this chapter, we present our first contribution: an analysis of the impact of bookmarks on navigation and streaming.
We implement a simple 3D navigation interface that we augment with 3D bookmarks. We implement a 3D navigation interface where the keyboard translates the camera and the mouse rotates it.
We augment this interface with 3D bookmarks.
When the user's cursor hovers over a bookmark, a preview of the point of view is displayed to the user, and when the user clicks on a bookmark, the camera smoothly moves from its current position to the bookmarked point of view. When the user's cursor hovers over a bookmark, a preview of the point of view is displayed to the user, and when the user clicks on a bookmark, the camera smoothly moves from its current position to the bookmarked point of view.
We conduct a within-subject user-study on 51 participants, where each user starts with a tutorial to get used to the 3D navigation controls, and then tries successively to perform a task with and without bookmarks. We conduct a within-subject user-study on 51 participants, where each user starts with a tutorial to get used to the 3D navigation controls, and then tries successively to perform a task with and without bookmarks.
We show that not only the presence of bookmarks causes a faster task completion, but also that it allows users to see a larger part of the scene during the same time span. We show that not only the presence of bookmarks causes a faster task completion, but also that it allows users to see a larger part of the scene during the same time span.
However, in a streaming scenario, this phenomenon leads to higher network requirements to maintain the same quality of service\todo{not sure if service or experience}. However, in a streaming scenario, this phenomenon leads to higher network requirements to maintain the same quality of service\todo{not sure if service or experience}.
In the last part of this chapter, we simulate streaming by replaying the traces collected during the user study, and we show that knowing the positions of the bookmarks beforehand allows us to pre-compute information that we can reuse during streaming to compensate for the harm caused by the faster navigation with bookmarks. In the last part of this chapter, we simulate a streaming setup and we show that knowing the positions of the bookmarks beforehand allows us to pre-compute information that we can reuse during streaming to compensate for the harm caused by the faster navigation with bookmarks.
\newpage \newpage