dash-3d-tmp
/
phd
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Did stuff

This commit is contained in:
Thomas Forgione 2019-10-09 15:00:01 +02:00
parent 0f11fe5f45
commit 05db88ddb0
No known key found for this signature in database
GPG Key ID: 203DAEA747F48F41
2 changed files with 20 additions and 13 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
src/system-bookmarks/bookmark.tex
View File

@ -21,7 +21,7 @@ We use the gyroscope to enable a user to rotate his device to rotate the virtual
We also add the possibility to rotate the camera by using touch controls. We also add the possibility to rotate the camera by using touch controls.
This way, the user is not forced to perform a real-world half-turn to be able to look behind or to keep its device pointing to the sky (which can quickly become tiring) to look up. This way, the user is not forced to perform a real-world half-turn to be able to look behind or to keep its device pointing to the sky (which can quickly become tiring) to look up.
These interactions, however, do not allow the user to move the camera: he can rotate it but not translate it. These interactions, however, do not allow the user to move the camera: he can rotate it but not translate it.
For this reason, we display a small joystick on the bottom left corner of the screen that mimics the first person video games interactions and allow the user translating the camera: For this reason, we display a small joystick on the bottom left corner of the screen that mimics the first person video games interactions and allows the user translating the camera:
\begin{itemize} \begin{itemize}
\item moving the joystick up makes the camera move forward; \item moving the joystick up makes the camera move forward;
\item moving the joystick down makes the camera move backwards; \item moving the joystick down makes the camera move backwards;
@ -75,18 +75,18 @@ Indeed, since they are emphasized and, in a way, recommended viewpoints, bookmar
As such, bookmarks can be used as a way to optimize streaming by downloading segments in an optimal, pre-computed order. As such, bookmarks can be used as a way to optimize streaming by downloading segments in an optimal, pre-computed order.
More specifically, segment utility as introduced in Section~\ref{d3:utility} is only an approximation of the segment's true contribution to the current viewpoint rendering. More specifically, segment utility as introduced in Section~\ref{d3:utility} is only an approximation of the segment's true contribution to the current viewpoint rendering.
When bookmarks are defined, it is possible to obtain a perfect measure of segment utility by performing an offline rendering at each bookmark's viewpoint. When bookmarks are defined, it is possible to obtain a better measure of segment utility by performing an offline rendering at each bookmark's viewpoint.
% Then, by simply counting the number of pixels that are rendered using each segment, we can rank the segments by order of importance in the rendering. % Then, by simply counting the number of pixels that are rendered using each segment, we can rank the segments by order of importance in the rendering.
We define $\mathcal{U}^{*} (s,B_i)$ as being the true utility of a segment $s$ in a viewpoint defined at bookmark $B_i$. We define $\mathcal{U}^{*} (s,B_i)$ as being the optimized utility of a segment $s$ in a viewpoint defined at bookmark $B_i$.
\fresh{} \fresh{}
In order to know the true utility of a segment, we developed Algorithm~\ref{sb:algo-optimal-order}, that sorts segments according to their true utility. In order to know the optimized utility of a segment, we developed Algorithm~\ref{sb:algo-optimal-order}, that sorts segments according to their optimized utility.
It takes as input the considered viewpoint, the ground truth from this viewpoint and the set of segments to sort. It takes as input the considered viewpoint, the ground truth from this viewpoint and the set of segments to sort.
It starts with an empty model, and tries all the segments from the set of candidates, and computes the PSNR between the corresponding render and the ground truth render. It starts with an empty model, individually tries adding each segment from the set of candidates, and computes the PSNR between the corresponding render and the ground truth render.
With all those PSNRs, it is able to determine which segment is the one that brings the best $\Delta\text{PSNR} / s$, $s$ being the size of the segment in bytes. We can thus determine which segment brings the highest $\Delta\text{PSNR} / s$, $s$ being the size of the segment in bytes.
Once it found the best segment, it registers it, and starts again. Once the best segment is found, it is registered, and a new iteration begins.
That way, it is able to generate an order of segments sorted by $\Delta\text{PSNR} / s$. That way, it is able to generate an order of segments sorted by $\Delta\text{PSNR} / s$.
This order is then saved as a JSON file that a client can download it to know which segments are more appropriate from a certain viewpoint. This order is then saved as a JSON file that a client can download in order to know which segments contribute the most to a certain viewpoint.
\begin{algorithm}[th] \begin{algorithm}[th]
\SetKwInOut{Input}{input} \SetKwInOut{Input}{input}
@ -145,7 +145,7 @@ This order is then saved as a JSON file that a client can download it to know wh
\end{algorithm} \end{algorithm}
Sorting all the segments from the model would be an excessively time consuming computation. Sorting all the segments from the model would be an excessively time consuming computation.
To speed up this algorithm, we only sort the 200 best segments, and we choose these segments among a filtered set of candidates. To speed up this algorithm, we only sort the 200 first best segments, and we choose these segments among a filtered set of candidates.
To find those candidates, we reuse the ideas developed in Chapter~\ref{bi}. To find those candidates, we reuse the ideas developed in Chapter~\ref{bi}.
We render the ``pixel to geometry segment'' and ``pixel to texture'' maps, as shown in Figure~\ref{sb:bookmarks-utility}. We render the ``pixel to geometry segment'' and ``pixel to texture'' maps, as shown in Figure~\ref{sb:bookmarks-utility}.
These renderings allow us to know what geometry segment and what texture correspond to each pixel, and filter out useless candidates. These renderings allow us to know what geometry segment and what texture correspond to each pixel, and filter out useless candidates.
@ -225,7 +225,7 @@ The three first values in the supplemental property are the camera position coor
\subsection{Loader modifications} \subsection{Loader modifications}
We build on the loader introduced in Algorithm~\ref{d3:next-segment} to implement a client adaptation logic. We build on the loader introduced in Algorithm~\ref{d3:next-segment} to implement a client adaptation logic.
We include a bookmark adaptation logic such that (i) when a bookmark is hovered for the first time, the corresponding images (see Listing~\ref{sb:bookmark-as}) are downloaded, and (ii) when a bookmark is clicked, we switch from utility $\mathcal{U}$ to true utility $\mathcal{U}^*$ to determine which segments to download next. We include a bookmark adaptation logic such that (i) when a bookmark is hovered for the first time, the corresponding images (see Listing~\ref{sb:bookmark-as}) are downloaded, and (ii) when a bookmark is clicked, we switch from utility $\mathcal{U}$ to optimized utility $\mathcal{U}^*$ to determine which segments to download next.
\begin{algorithm}[th] \begin{algorithm}[th]
\SetKwInOut{Input}{input} \SetKwInOut{Input}{input}

13
src/system-bookmarks/user-study.tex
View File

@ -1,10 +1,17 @@
\fresh{} \fresh{}
\section{Evaluation}\label{sb:evaluation} \section{Evaluation}\label{sb:evaluation}
To evaluate the impact of the modifications made in the previous section, we conducted a user study. \subsection{Preliminary user study}
Since we already conduct a user study on desktop devices, we decide to conduct this experiment exclusively on mobile devices.
\subsection{Setup} Before conducting the user study on mobile devices, we designed a user study for desktop devices.
This experiment was conducted on a little more than a dozen of people, with the model described in the previous chapter.
Bookmarks were positioned from user-generated panoramic picture available on Google Maps, and the task consisted in retrieving spots on the 3D model from a picture: users were presented with an image coming from Google Street View and user had to find the corresponding spot in the 3D model.
Due to the fact the task was hard, and that our users were familiar with 3D navigation, they preferred navigating slowly in the scene, and did not use bookmarks as much as they did during the experiment we ran in Chapter~\ref{bi}.
For these reasons, we decided to setup a new experiment, on a model a little larger, with a less complex task, and we decided to conduct this experiment on mobile device exclusively, to see how bookmarks help people navigate in a scene when controls are more cumbersome.
\subsection{Mobile navigation user study}
\subsubsection{Models} \subsubsection{Models}