dash-3d-tmp
/
phd
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Lowercase titles

This commit is contained in:
Thomas Forgione 2020-02-05 16:39:21 +01:00
parent 0e812abafa
commit 9d003d4544
9 changed files with 27 additions and 27 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
src/dash-3d/client.tex
View File

@ -80,7 +80,7 @@ At time instance $t_i$, the DASH client decides to download the appropriate segm
Starting from $t_1$, the camera continuously follows a camera path $C=\{v(t_i), t_i \in [t_1,t_{end}]\}$, along which downloading opportunities are strategically exploited to sequentially query the most useful segments. Starting from $t_1$, the camera continuously follows a camera path $C=\{v(t_i), t_i \in [t_1,t_{end}]\}$, along which downloading opportunities are strategically exploited to sequentially query the most useful segments.
\subsection{Segment Utility\label{d3:utility}} \subsection{Segment utility\label{d3:utility}}
Unlike video streaming, where the bitrate of each segment correlates with the quality of the video received, for 3D content, the size (in bytes) of the content does not necessarily correlate well to its contribution to visual quality. Unlike video streaming, where the bitrate of each segment correlates with the quality of the video received, for 3D content, the size (in bytes) of the content does not necessarily correlate well to its contribution to visual quality.
A large polygon with huge visual impact takes the same number of bytes as a tiny polygon. A large polygon with huge visual impact takes the same number of bytes as a tiny polygon.
@ -140,7 +140,7 @@ We do this to acknowledge the fact that a texture at a greater resolution has a
The equivalent term for geometry is 1 (and does not appear). The equivalent term for geometry is 1 (and does not appear).
Having defined a utility on both geometry and texture segments, the client uses it next for its streaming strategy. Having defined a utility on both geometry and texture segments, the client uses it next for its streaming strategy.
\subsection{DASH Adaptation Logic\label{d3:dash-adaptation}} \subsection{DASH adaptation logic\label{d3:dash-adaptation}}
Along the camera path $C=\{v(t_i)\}$, viewpoints are indexed by a continuous time interval $t_i \in [t_1,t_{end}]$. Along the camera path $C=\{v(t_i)\}$, viewpoints are indexed by a continuous time interval $t_i \in [t_1,t_{end}]$.
Contrastingly, the DASH adaptation logic proceeds sequentially along a discrete time line. Contrastingly, the DASH adaptation logic proceeds sequentially along a discrete time line.

8
src/dash-3d/content-preparation.tex
View File

@ -23,11 +23,11 @@ The MPD is generated by a library named \href{https://github.com/netvl/xml-rs}{x
\end{itemize} \end{itemize}
This structure is passed along with our geometry and texture preprocessors that can add elements to the XML file as they are generating the corresponding data chunks. This structure is passed along with our geometry and texture preprocessors that can add elements to the XML file as they are generating the corresponding data chunks.
\subsection{Adaptation Sets} \subsection{Adaptation sets}
When the user navigates freely within an NVE, the frustum at given time almost always contains a limited part of the 3D scene. When the user navigates freely within an NVE, the frustum at given time almost always contains a limited part of the 3D scene.
Similar to how DASH for video streaming partitions a video clip into temporal chunks, we segment the polygons into spatial chunks, such that the DASH client can request only the relevant chunks. Similar to how DASH for video streaming partitions a video clip into temporal chunks, we segment the polygons into spatial chunks, such that the DASH client can request only the relevant chunks.
\subsubsection{Geometry Management\label{d3:geometry}} \subsubsection{Geometry management\label{d3:geometry}}
We use a space partitioning tree to organize the faces into cells. We use a space partitioning tree to organize the faces into cells.
A face belongs to a cell if its barycenter falls inside the corresponding bounding box. A face belongs to a cell if its barycenter falls inside the corresponding bounding box.
Each cell corresponds to an adaptation set. Each cell corresponds to an adaptation set.
@ -43,14 +43,14 @@ We thus obtain a decomposition of the NVE into adaptation sets that partitions t
We store the spatial location of each adaptation set, characterized by the coordinates of its bounding box, in the MPD file as the supplementary property of the adaptation set in the form of ``\textit{$x_{\min}$, width, $y_{\min}$, height, $z_{\min}$, depth}'' (as shown in Listing~\ref{d3:mpd}). We store the spatial location of each adaptation set, characterized by the coordinates of its bounding box, in the MPD file as the supplementary property of the adaptation set in the form of ``\textit{$x_{\min}$, width, $y_{\min}$, height, $z_{\min}$, depth}'' (as shown in Listing~\ref{d3:mpd}).
This information is used by the client to implement a view-dependent streaming (Section~\ref{d3:dash-client}). This information is used by the client to implement a view-dependent streaming (Section~\ref{d3:dash-client}).
\subsubsection{Texture Management} \subsubsection{Texture management}
As with geometry data, we handle textures using adaptation sets but separate from geometry. As with geometry data, we handle textures using adaptation sets but separate from geometry.
Each texture file is contained in a different adaptation set, with multiple representations providing different image resolutions (see Section~\ref{d3:representation}). Each texture file is contained in a different adaptation set, with multiple representations providing different image resolutions (see Section~\ref{d3:representation}).
We add an attribute to each adaptation set that contains texture, describing the average color of the texture. We add an attribute to each adaptation set that contains texture, describing the average color of the texture.
The client can use this attribute to render a face for which the corresponding texture has not been loaded yet, so that most objects appear, at least, with a uniform natural color (see Figure~\ref{d3:textures}). The client can use this attribute to render a face for which the corresponding texture has not been loaded yet, so that most objects appear, at least, with a uniform natural color (see Figure~\ref{d3:textures}).
\subsubsection{Material Management} \subsubsection{Material management}
The material \texttt{.mtl} file is a text file that describes all materials used in the \texttt{.obj} files for the entire 3D model. The material \texttt{.mtl} file is a text file that describes all materials used in the \texttt{.obj} files for the entire 3D model.
A material has a name, properties such as specular parameters, and, most importantly, a path to a texture file. A material has a name, properties such as specular parameters, and, most importantly, a path to a texture file.
The \texttt{.mtl} file maps each face of the \texttt{.obj} to a material. The \texttt{.mtl} file maps each face of the \texttt{.obj} to a material.

10
src/dash-3d/evaluation.tex
View File

@ -2,7 +2,7 @@
We now describe our setup and the data we use in our experiments. We present an evaluation of our system and a comparison of the impact of the design choices we introduced in the previous sections. We now describe our setup and the data we use in our experiments. We present an evaluation of our system and a comparison of the impact of the design choices we introduced in the previous sections.
\subsection{Experimental Setup} \subsection{Experimental setup}
\subsubsection{Model} \subsubsection{Model}
We use a city model of the Marina Bay area in Singapore in our experiments. We use a city model of the Marina Bay area in Singapore in our experiments.
@ -49,7 +49,7 @@ We partition the geometry into a k-$d$ tree until the leafs have less than 10000
\caption{Sizes of the different files of the model\label{d3:size}} \caption{Sizes of the different files of the model\label{d3:size}}
\end{table} \end{table}
\subsubsection{User Navigations} \subsubsection{User navigations}
To evaluate our system, we collected realistic user navigation traces which we can replay in our experiments. To evaluate our system, we collected realistic user navigation traces which we can replay in our experiments.
We presented six users with a web interface, on which the model was loaded progressively as the user could interact with it. We presented six users with a web interface, on which the model was loaded progressively as the user could interact with it.
The available interactions were inspired by traditional first-person interactions in video games, i.e., W, A, S, and D keys to translate the camera, and mouse to rotate the camera. The available interactions were inspired by traditional first-person interactions in video games, i.e., W, A, S, and D keys to translate the camera, and mouse to rotate the camera.
@ -60,7 +60,7 @@ Every 100ms, the position, viewing angle of the camera and look-at point are sav
The recorded camera trace allows us to replay each camera path to perform our simulations and evaluate our system. The recorded camera trace allows us to replay each camera path to perform our simulations and evaluate our system.
We collected 13 camera paths this way. We collected 13 camera paths this way.
\subsubsection{Network Setup} \subsubsection{Network setup}
We tested our implementation under three network bandwidth of 2.5 Mbps, 5 Mbps, and 10 Mbps with an RTT of 38 ms, following the settings from DASH-IF~\citep{dash-network-profiles}. We tested our implementation under three network bandwidth of 2.5 Mbps, 5 Mbps, and 10 Mbps with an RTT of 38 ms, following the settings from DASH-IF~\citep{dash-network-profiles}.
The values are kept constant during the entire client session to analyze the difference in magnitude of performance by increasing the bandwidth. The values are kept constant during the entire client session to analyze the difference in magnitude of performance by increasing the bandwidth.
@ -68,7 +68,7 @@ In our experiments, we set up a virtual camera that moves along a navigation pat
We log in a JSON file the time when a segment is requested and when it is received. We log in a JSON file the time when a segment is requested and when it is received.
By doing so, we avoid wasting time and resources to evaluate our system while downloading segments and store all the information necessary to plot the figures introduced in the subsequent sections. By doing so, we avoid wasting time and resources to evaluate our system while downloading segments and store all the information necessary to plot the figures introduced in the subsequent sections.
\subsubsection{Hardware and Software} \subsubsection{Hardware and software}
The experiments were run on an Acer Aspire V3 with an Intel Core i7 3632QM processor and an NVIDIA GeForce GT 740M graphics card. The experiments were run on an Acer Aspire V3 with an Intel Core i7 3632QM processor and an NVIDIA GeForce GT 740M graphics card.
The DASH client is written in Rust\footnote{\url{https://www.rust-lang.org/}}, using Glium\footnote{\url{https://github.com/glium/glium}} for rendering, and reqwest\footnote{\url{https://github.com/seanmonstar/reqwest/}} to load the segments. The DASH client is written in Rust\footnote{\url{https://www.rust-lang.org/}}, using Glium\footnote{\url{https://github.com/glium/glium}} for rendering, and reqwest\footnote{\url{https://github.com/seanmonstar/reqwest/}} to load the segments.
@ -105,7 +105,7 @@ Finally, we try several bandwidth parameters to study how our system can adapt t
\caption{Different parameters in our experiments\label{d3:experiments}} \caption{Different parameters in our experiments\label{d3:experiments}}
\end{table} \end{table}
\subsection{Experimental Results} \subsection{Experimental results}
\begin{figure}[th] \begin{figure}[th]
\centering \centering
\begin{tikzpicture} \begin{tikzpicture}

2
src/french/client.tex
View File

@ -5,7 +5,7 @@ Le client DASH commence par télécharger le MPD, avant de commencer à téléch
Nous considérons une caméra virtuelle qui suit continuellement un chemin $C=\{v(t_i), t_i \in [t_1, t_{final}]\}$, où $t_i$ est l'instant où le segment $i$ est demandé, et au cours duquel les opportunités de téléchargement sont stratégiquement exploitées pour télécharger séquentiellement les segments les plus utiles. Nous considérons une caméra virtuelle qui suit continuellement un chemin $C=\{v(t_i), t_i \in [t_1, t_{final}]\}$, où $t_i$ est l'instant où le segment $i$ est demandé, et au cours duquel les opportunités de téléchargement sont stratégiquement exploitées pour télécharger séquentiellement les segments les plus utiles.
\subsection{Utilité des Segments}\label{fr:utility} \subsection{Utilité des segments}\label{fr:utility}
Contrairement au streaming vidéo, où la taille (en octets) de chaque segment est corrélée à la qualité de la vidéo reçue, pour le contenu 3D, la taille du contenu n'est pas nécessairement corrélée à sa contribution en terme de qualité du rendu. Un grand polygone qui aura un grand impact visuel occupera à peu près autant d'octets qu'un petit polygone. De plus, l'impact visuel dépend du point de vue --- un grand polygone lointain ne contribuera pas autant qu'un petit polygone plus près de l'utilisateur. Ainsi, il est important pour un client DASH-NVE d'estimer ce que nous appelons \emph{l'utilité d'un segment}, de sorte à prendre les bonnes décisions de téléchargement. Contrairement au streaming vidéo, où la taille (en octets) de chaque segment est corrélée à la qualité de la vidéo reçue, pour le contenu 3D, la taille du contenu n'est pas nécessairement corrélée à sa contribution en terme de qualité du rendu. Un grand polygone qui aura un grand impact visuel occupera à peu près autant d'octets qu'un petit polygone. De plus, l'impact visuel dépend du point de vue --- un grand polygone lointain ne contribuera pas autant qu'un petit polygone plus près de l'utilisateur. Ainsi, il est important pour un client DASH-NVE d'estimer ce que nous appelons \emph{l'utilité d'un segment}, de sorte à prendre les bonnes décisions de téléchargement.

2
src/french/dash-3d.tex
View File

@ -12,7 +12,7 @@ Les \emph{periods} sont le premier niveau, qui dans le cas de la vidéo, indique
Chaque \emph{period} contient un ou plusieurs \emph{adaptation sets} qui décrivent des versions alternatives, formats et types de contenu. Nous utilisons les \emph{adaptation sets} pour organiser la géométrie et les textures de la scène. Chaque \emph{period} contient un ou plusieurs \emph{adaptation sets} qui décrivent des versions alternatives, formats et types de contenu. Nous utilisons les \emph{adaptation sets} pour organiser la géométrie et les textures de la scène.
\subsection{Adaptation Sets} \subsection{Adaptation sets}
Quand un utilisateur navigue librement dans un NVE, le champ de vision à un moment donné ne contient qu'une partie limitée de la scène. De la même façon que DASH-vidéo partitionne une vidéo en blocs temporels, nous partitionnons les polygones en blocs spatiaux, de sorte que notre client puisse ne télécharger que les blocs nécessaires. Quand un utilisateur navigue librement dans un NVE, le champ de vision à un moment donné ne contient qu'une partie limitée de la scène. De la même façon que DASH-vidéo partitionne une vidéo en blocs temporels, nous partitionnons les polygones en blocs spatiaux, de sorte que notre client puisse ne télécharger que les blocs nécessaires.

10
src/preliminary-work/bookmarks-impact.tex
View File

@ -1,4 +1,4 @@
\section{Impact of 3D Bookmarks on Navigation\label{bi:3dnavigation}} \section{Impact of 3D bookmarks on navigation\label{bi:3dnavigation}}
We now describe an experiment that we conducted on 51 participants, with two goals in mind. We now describe an experiment that we conducted on 51 participants, with two goals in mind.
First, we want to measure the impact of 3D bookmarks on navigation within an NVE\@. First, we want to measure the impact of 3D bookmarks on navigation within an NVE\@.
@ -20,7 +20,7 @@ The mouse movement controls the camera rotation.
The user can always choose to lock the pointer, or unlock it using the escape key. The user can always choose to lock the pointer, or unlock it using the escape key.
The interface also includes a button to reset the camera back to the starting position in the scene. The interface also includes a button to reset the camera back to the starting position in the scene.
\subsection{3D Bookmarks\label{bi:3d-bookmarks}} \subsection{3D bookmarks\label{bi:3d-bookmarks}}
Our NVE supports 3D bookmarks. Our NVE supports 3D bookmarks.
A 3D bookmark, or bookmark for short, is simply a fixed camera location (in 3D space), a view direction, and a focal. A 3D bookmark, or bookmark for short, is simply a fixed camera location (in 3D space), a view direction, and a focal.
Bookmarks visible from the user's current viewpoint are shown as 3D objects in the scene. Bookmarks visible from the user's current viewpoint are shown as 3D objects in the scene.
@ -42,7 +42,7 @@ Since bookmarks are part of the scene, they are visible only when not hidden by
We chose size and colors that are salient enough to be easily seen, but not too large to limit the occlusion of regions within the scene. We chose size and colors that are salient enough to be easily seen, but not too large to limit the occlusion of regions within the scene.
When reaching the bookmark, the corresponding arrow or viewport is not visible anymore, and subsequently will appear in a different color, to indicate that it has been clicked (similar to Web links). When reaching the bookmark, the corresponding arrow or viewport is not visible anymore, and subsequently will appear in a different color, to indicate that it has been clicked (similar to Web links).
\subsection{User Study\label{bi:user-study}} \subsection{User study\label{bi:user-study}}
We now describe in details our experimental setup and the user study that we conducted on 3D navigation. We now describe in details our experimental setup and the user study that we conducted on 3D navigation.
@ -102,7 +102,7 @@ Table~\ref{bi:questions} shows the list of questions.
The participants were recruited on microworkers.com, a crowdsourcing website. The participants were recruited on microworkers.com, a crowdsourcing website.
There were 51 participants (36 men and 15 women), who are in average 30.44 years old. There were 51 participants (36 men and 15 women), who are in average 30.44 years old.
\subsection{Experimental Results}\label{bi::qoe-results} \subsection{Experimental results}\label{bi::qoe-results}
We now present the results from our user study, focusing on whether bookmarks help users navigating the 3D scene. We now present the results from our user study, focusing on whether bookmarks help users navigating the 3D scene.
@ -120,7 +120,7 @@ This is slightly in contradiction with our setup; even if coins may appear in so
The strongest result is that almost all users (49 out of 51) find bookmarks to be helpful. The strongest result is that almost all users (49 out of 51) find bookmarks to be helpful.
In addition, users seem to have a preference for \Arrows{} against \Viewports{} (32 against 7). In addition, users seem to have a preference for \Arrows{} against \Viewports{} (32 against 7).
\subsubsection{Analysis of Interactions} \subsubsection{Analysis of interactions}
\begin{table}[th] \begin{table}[th]
\centering \centering

14
src/preliminary-work/streaming.tex
View File

@ -1,6 +1,6 @@
\section{Impact of 3D Bookmarks on Streaming\label{bi:system}} \section{Impact of 3D bookmarks on streaming\label{bi:system}}
\subsection{3D Model Streaming} \subsection{3D model streaming}
In this section, we describe our implementation of a 3D model streaming policy in our simulation. In this section, we describe our implementation of a 3D model streaming policy in our simulation.
A summary of the streaming policies we designed is give in Table~\ref{bi:streaming-policies}. A summary of the streaming policies we designed is give in Table~\ref{bi:streaming-policies}.
@ -68,12 +68,12 @@ Note that the server may send faces that are occluded and not visible to the cli
In the following, we shall denote this streaming policy \textsf{culling}; in Figures~\ref{bi:click-1250} and~\ref{bi:click-625} streaming using \textsf{culling} only is denoted \textsf{C-only}. In the following, we shall denote this streaming policy \textsf{culling}; in Figures~\ref{bi:click-1250} and~\ref{bi:click-625} streaming using \textsf{culling} only is denoted \textsf{C-only}.
\subsection{3D Bookmarks} \subsection{3D bookmarks}
We have seen (Figure~\ref{bi:triangles-curve}) that navigation with bookmarks is more demanding on the bandwidth. We have seen (Figure~\ref{bi:triangles-curve}) that navigation with bookmarks is more demanding on the bandwidth.
We want to exploit bookmarks to improve the user's quality of experience. For this purpose, we propose two streaming policies based on offline computation of the relevance of 3D content to bookmarked viewpoints. We want to exploit bookmarks to improve the user's quality of experience. For this purpose, we propose two streaming policies based on offline computation of the relevance of 3D content to bookmarked viewpoints.
\subsubsection{Visibility Determination for 3D Bookmarks} \subsubsection{Visibility determination for 3D bookmarks}
A bookmarked viewpoint is more likely to be accessed, compared to other arbitrary viewpoint in the 3D scene. A bookmarked viewpoint is more likely to be accessed, compared to other arbitrary viewpoint in the 3D scene.
We exploit this fact to perform some pre-computation on the 3D content visible from the bookmarked viewpoint. We exploit this fact to perform some pre-computation on the 3D content visible from the bookmarked viewpoint.
@ -128,7 +128,7 @@ In what follows, we will refer to this streaming policy as \textsf{visible}.
\caption{Comparison of rendered image quality (average on all bookmarks and starting position): the triangles are sorted offline (green curve), or sorted online by distance to the viewpoint (blue curve).}\label{bi:sorted-tri} \caption{Comparison of rendered image quality (average on all bookmarks and starting position): the triangles are sorted offline (green curve), or sorted online by distance to the viewpoint (blue curve).}\label{bi:sorted-tri}
\end{figure} \end{figure}
\subsubsection{Prefetching by Predicting the Next Bookmark Clicked} \subsubsection{Prefetching by predicting the next bookmark clicked}
We can now use the precomputed, visibility-based streaming of 3D content for the bookmarks to reduce the amount of traffic needed. We can now use the precomputed, visibility-based streaming of 3D content for the bookmarks to reduce the amount of traffic needed.
Next, we propose to prefetch the 3D content from the bookmarks. Next, we propose to prefetch the 3D content from the bookmarks.
@ -185,7 +185,7 @@ We denote this combination as \textsf{V-PP}, for Prefetching based on Prediction
\caption{Example of how a chunk can be divided into fetching what is needed to display the current viewport (culling), and prefetching three recommendations according to their probability of being visited next.\label{bi:prefetched-chunk}} \caption{Example of how a chunk can be divided into fetching what is needed to display the current viewport (culling), and prefetching three recommendations according to their probability of being visited next.\label{bi:prefetched-chunk}}
\end{figure} \end{figure}
\subsubsection{Fetching Destination Bookmark} \subsubsection{Fetching destination bookmark}
An alternate method to benefit from the precomputing visible triangles at the bookmark, is to fetch 3D content during the ``fly-to'' transition to reach the destination. An alternate method to benefit from the precomputing visible triangles at the bookmark, is to fetch 3D content during the ``fly-to'' transition to reach the destination.
Indeed, as specified in Section~\ref{bi:3dnavigation}, moving to a bookmarked viewpoint is not instantaneous, but rather takes a small amount of time to smoothly move the user camera from its initial position towards the bookmark. Indeed, as specified in Section~\ref{bi:3dnavigation}, moving to a bookmarked viewpoint is not instantaneous, but rather takes a small amount of time to smoothly move the user camera from its initial position towards the bookmark.
@ -211,7 +211,7 @@ We call this method \textsf{V-FD}, since we are Fetching the 3D data from the De
\caption{Summary of the streaming policies\label{bi:streaming-policies}} \caption{Summary of the streaming policies\label{bi:streaming-policies}}
\end{table} \end{table}
\subsection{Comparing Streaming Policies} \subsection{Comparing streaming policies}
In order to determine which policy to use, we replay the traces from the user study while simulating different streaming policies. In order to determine which policy to use, we replay the traces from the user study while simulating different streaming policies.
The first point we are interested in is which streaming policy leads to the lower discovery latency and better image quality for the user: \textsf{culling} (no prefetching), \textsf{V-PP} (prefetching based on probability of accessing bookmarks), or \textsf{V-FD} (no prefetching, but fetch the destination during fly-to transition) or combining both \textsf{V-PP} and \textsf{V-FD} (\textsf{V-PP+FD}). The first point we are interested in is which streaming policy leads to the lower discovery latency and better image quality for the user: \textsf{culling} (no prefetching), \textsf{V-PP} (prefetching based on probability of accessing bookmarks), or \textsf{V-FD} (no prefetching, but fetch the destination during fly-to transition) or combining both \textsf{V-PP} and \textsf{V-FD} (\textsf{V-PP+FD}).

2
src/state-of-the-art/3d-interaction.tex
View File

@ -1,4 +1,4 @@
\section{3D Bookmarks and Navigation Aids} \section{3D bookmarks and navigation aids}
One of the uses for 3D streaming is to allow users interacting with the content while it is being downloaded. One of the uses for 3D streaming is to allow users interacting with the content while it is being downloaded.
However, devising an ergonomic technique for browsing 3D environments through a 2D interface is difficult. However, devising an ergonomic technique for browsing 3D environments through a 2D interface is difficult.

2
src/state-of-the-art/3d-streaming.tex
View File

@ -1,4 +1,4 @@
\section{3D Streaming\label{sote:3d-streaming}} \section{3D streaming\label{sote:3d-streaming}}
In this thesis, we focus on the objective of delivering large, massive 3D scenes over the network. In this thesis, we focus on the objective of delivering large, massive 3D scenes over the network.
While 3D streaming is not the most standard field of research, there has been a special attention around 3D content compression, in particular progressive compression which can be considered a premise for 3D streaming. While 3D streaming is not the most standard field of research, there has been a special attention around 3D content compression, in particular progressive compression which can be considered a premise for 3D streaming.