Merge branch 'axel'

This commit is contained in:
Thomas Forgione 2019-10-18 16:09:10 +02:00
commit 10238c1e2b
No known key found for this signature in database
GPG Key ID: 203DAEA747F48F41
5 changed files with 113 additions and 109 deletions

View File

@ -1,20 +1,20 @@
\frontmatter{} \frontmatter{}
\input{introduction/main} %\input{introduction/main}
\resetstyle{} \resetstyle{}
\mainmatter{} \mainmatter{}
\input{foreword/main} %\input{foreword/main}
\resetstyle{} \resetstyle{}
\input{state-of-the-art/main} %\input{state-of-the-art/main}
\resetstyle{} \resetstyle{}
\input{preliminary-work/main} %\input{preliminary-work/main}
\resetstyle{} \resetstyle{}
\input{dash-3d/main} %\input{dash-3d/main}
\resetstyle{} \resetstyle{}
\input{system-bookmarks/main} \input{system-bookmarks/main}
@ -22,5 +22,5 @@
\backmatter{} \backmatter{}
\input{conclusion/main} %\input{conclusion/main}

View File

@ -11,88 +11,69 @@ Regarding desktop interaction, we keep the interaction we described in Section~\
\end{itemize} \end{itemize}
A screenshot of this interface is displayed in Figure~\ref{sb:desktop}. A screenshot of this interface is displayed in Figure~\ref{sb:desktop}.
\subsection{Mobile interaction}
\copied{}
Mobile interactions are more complex because the user does not have neither keyboard nor mouse to interact with.
However, there are some other sensors on most mobile devices that can help interaction.
One useful sensor for 3D interaction on mobile devices is definitely the gyroscope.
We use the gyroscope to enable a user to rotate his device to rotate the virtual camera.
We also add the possibility to rotate the camera by using touch controls.
This way, the user is not forced to perform a real-world half-turn to be able to look behind or to keep its device pointing to the sky (which can quickly become tiring) to look up.
These interactions, however, do not allow the user to move the camera: he can rotate it but not translate it.
For this reason, we display a small joystick on the bottom left corner of the screen that mimics the first person video games interactions and allows the user translating the camera:
\begin{itemize}
\item moving the joystick up makes the camera move forward;
\item moving the joystick down makes the camera move backwards;
\item moving the joystick sideways makes the camera move sideways.
\end{itemize}
A screenshot of this interface is displayed in Figure~\ref{sb:mobile}.
\section{Adding bookmarks into DASH NVE framework\label{sb:bookmarks}}
\copied{}
In this section, we explain how to include a new interaction in the system described in the previous chapter.
\fresh{}
\subsection{Interaction --- Visual}
In Chapter~\ref{bi} Section~\ref{bi:3d-bookmarks}, we described two 3D widgets that we used to display bookmarks to users.
One of the conclusions of the user-study, described in Section~\ref{bi:user-study}, was that the impact of the way we display bookmark was not significant.
In this work, we chose a slightly different way of representing bookmarks due to some concerns with our original representations:
\begin{itemize}
\item viewport bookmarks are simple, but people who are not computer vision scientists are not familiar with this representation;
\item arrow bookmarks are complex, and need to be regenerated when the camera moves, which can harm the framerate of the rendering.
\end{itemize}
For these reasons, we changed the display to a vertical bar with a 2D sprite of a pictorial representation of an eye.
This 2D sprite is always facing the camera to prevent it from being invisible when the camera would be on the side of it.
Screenshots of user interfaces with bookmarks are available in Figures~\ref{sb:desktop} and~\ref{sb:mobile}.
The size of the sprite changes over time following a sine function to help the user distinguish what is part of the scene and what is extra widgets.
Since our scene is static, a user knows that a changing object is not part of the scene, but part of the UI\@.
The other bookmark parameters remain unchanged since Chapter~\ref{bi}: in order to avoid users to lose context, clicking on a bookmark triggers an automatic, smooth, camera displacement that ends up at the bookmark.
We also display a thumbnail of the bookmark's viewpoint when the mouse hovers a bookmark.
Such thumbnail is displayed in Figure~\ref{sb:desktop}.
Note that since on mobile, there is no mouse and thus no pointer, thumbnails are never downloaded nor displayed.
\begin{figure}[ht] \begin{figure}[ht]
\centering \centering
\includegraphics[width=\textwidth]{assets/system-bookmarks/screenshots/desktop.png} \includegraphics[width=\textwidth]{assets/system-bookmarks/screenshots/desktop.png}
\caption{Screenshot of the desktop version, with a bookmark and its thumbnail on the bottom left corner\label{sb:desktop}} \caption{Screenshot of the desktop version, with a bookmark and its thumbnail on the bottom left corner\label{sb:desktop}}
\end{figure} \end{figure}
\subsection{Mobile interaction}
\copied{}
Mobile interactions are more complex because the user does not have the keyboard and mouse to interact with.
However, there are some other sensors on most mobile devices that can help interaction.
One useful sensor for 3D interaction on mobile devices is definitely the gyroscope.
We use the gyroscope to enable a user to rotate his device to rotate the virtual camera.
We also add the possibility to rotate the camera by using touch controls. The user can touch a part of the screen to get a hold at the virtual camera, and drag the camera direction along the two screen axis.
This way, the user is not forced to perform a real-world half-turn to be able to look behind or to point the device towards the sky (which can quickly become tiring) to look up.
These interactions, however, only allow the user to rotate the camera but not translate it.
For this reason, we display a small joystick on the bottom-left corner of the screen that mimics the first person video games interactions and allows the user translating the camera:
\begin{itemize}
\item moving the joystick up makes the camera move forward;
\item moving the joystick down makes the camera move backwards;
\item moving the joystick on the sides makes the camera move sideways.
\end{itemize}
A screenshot of this interface is displayed in Figure~\ref{sb:mobile}. The virtual joystick is rendered as a black circle inside a larger semi-transparent white circle. The black circle can be moved up, down, and sideways to define the direction in which the camera is translated.
\begin{figure}[ht] \begin{figure}[ht]
\centering \centering
\includegraphics[width=\textwidth]{assets/system-bookmarks/screenshots/mobile.png} \includegraphics[width=\textwidth]{assets/system-bookmarks/screenshots/mobile.png}
\caption{Screenshot of the mobile version, with its joystick on the bottom left corner\label{sb:mobile}} \caption{Screenshot of the mobile version, with its joystick on the bottom left corner\label{sb:mobile}}
\end{figure} \end{figure}
\subsection{Segments utility at bookmarked viewpoint\label{sb:utility}} \section{Adding bookmarks into DASH NVE framework\label{sb:bookmarks}}
\copied{}
Introducing bookmarks is a way to make users navigation more predictable.
Indeed, since they are emphasized and, in a way, recommended viewpoints, bookmarks are more likely to be visited by a significant portion of users than any other viewpoint on the scene.
As such, bookmarks can be used as a way to optimize streaming by downloading segments in an optimal, pre-computed order.
More specifically, segment utility as introduced in Section~\ref{d3:utility} is only an approximation of the segment's true contribution to the current viewpoint rendering.
When bookmarks are defined, it is possible to obtain a better measure of segment utility by performing an offline rendering at each bookmark's viewpoint.
% Then, by simply counting the number of pixels that are rendered using each segment, we can rank the segments by order of importance in the rendering.
We define $\mathcal{U}^{*} (s,B_i)$ as being the optimized utility of a segment $s$ in a viewpoint defined at bookmark $B_i$.
\fresh{} \fresh{}
In order to know the optimized utility of a segment, we developed Algorithm~\ref{sb:algo-optimal-order}, that sorts segments according to their optimized utility. While the previously defined interactions allow users to navigate freely throughout the scene, controlling such a high number of degrees of freedom can feel overwhelming to some users. That is why we introduce bookmarks, i.e. widgets that help the users reach a distant part of the scene using only a single, simple, interaction.
It takes as input the considered viewpoint, the ground truth from this viewpoint and the set of segments to sort.
It starts with an empty model, individually tries adding each segment from the set of candidates, and computes the PSNR between the corresponding render and the ground truth render.
We can thus determine which segment brings the highest $\Delta\text{PSNR} / s$, $s$ being the size of the segment in bytes. \subsection{Bookmark interaction and visual aspect}
Once the best segment is found, it is registered, and a new iteration begins.
That way, it is able to generate an order of segments sorted by $\Delta\text{PSNR} / s$. In Chapter~\ref{bi} Section~\ref{bi:3d-bookmarks}, we described two 3D widgets that we use to display bookmarks to users.
This order is then saved as a JSON file that a client can download in order to know which segments contribute the most to a certain viewpoint. One of the conclusions of the user-study, described in Section~\ref{bi:user-study}, was that the impact of the way we display bookmark was not significant.
In this work, we chose a slightly different way of representing bookmarks due to some concerns with our original representations:
\begin{itemize}
\item viewport bookmarks are simple, but non computer vision experts may not be familiar with this type of representation;
\item arrow bookmarks are more intuitive to most users, but need to be regenerated when the camera moves, which can harm the rendering framerate.
\end{itemize}
For these reasons, we changed the bookmarks display to a vertical bar textured with a 2D sprite of a pictorial representation of an eye. The use of such symbol is partly inspired by the cartographic pictograms used to showcase a worthwhile panorama.
This 2D sprite is always facing the camera to prevent it from being invisible when the camera would be on the side of it.
Screenshots of user interfaces with bookmarks are available in Figures~\ref{sb:desktop} and~\ref{sb:mobile}.
The size of the sprite changes over time following a sine function to help the user distinguish what is part of the scene and what is extra widgets.
Since our scene is static, a user knows that a changing object is not part of the scene, but part of the UI\@.
The other bookmark parameters remain unchanged since Chapter~\ref{bi}: in order to avoid users to lose context, clicking on a bookmark triggers an automatic, smooth, camera displacement that ends up at the bookmarked camera position.
We also display a thumbnail of the bookmark's viewpoint when the mouse hovers a bookmark.
Such thumbnail is displayed in Figure~\ref{sb:desktop}.
Note that since on mobile, there is no mouse and thus no pointer, thumbnails are never downloaded nor displayed.
\begin{algorithm}[th] \begin{algorithm}[th]
\SetKwInOut{Input}{input} \SetKwInOut{Input}{input}
\SetKwInOut{Output}{output} \SetKwInOut{Output}{output}
\SetKwData{BookmarkViewpoint}{bookmark\_viewpoint} \SetKwData{BookmarkViewpoint}{bookmarked\_viewpoint}
\SetKwData{OptimalOrder}{optimal\_order} \SetKwData{OptimalOrder}{optimal\_order}
\SetKwData{TotalModel}{total\_model} \SetKwData{TotalModel}{total\_model}
\SetKwData{EmptyModel}{empty\_model} \SetKwData{EmptyModel}{empty\_model}
@ -112,7 +93,7 @@ This order is then saved as a JSON file that a client can download in order to k
\SetKwFunction{Render}{render} \SetKwFunction{Render}{render}
\SetKwFunction{Psnr}{psnr} \SetKwFunction{Psnr}{psnr}
\Input{The bookmark viewpoint, the ground truth render at this viewpoint, the candidate segments} \Input{The bookmarked viewpoint, the ground truth render at this viewpoint, the candidate segments}
\Output{The optimal order of the segments} \Output{The optimal order of the segments}
\OptimalOrder\leftarrow{} []\; \OptimalOrder\leftarrow{} []\;
@ -144,6 +125,28 @@ This order is then saved as a JSON file that a client can download in order to k
\caption{Computation of the optimal order of segments from a bookmark\label{sb:algo-optimal-order}} \caption{Computation of the optimal order of segments from a bookmark\label{sb:algo-optimal-order}}
\end{algorithm} \end{algorithm}
\subsection{Segments utility at bookmarked viewpoint\label{sb:utility}}
\copied{}
Introducing bookmarks is a way to make users navigation more predictable.
Indeed, since they are emphasized and, in a way, recommended viewpoints, bookmarks are more likely to be visited by a significant portion of users than any other viewpoint on the scene.
As such, bookmarks can be used as a way to optimize streaming by downloading segments in an optimal, pre-computed order.
More specifically, segment utility as introduced in Section~\ref{d3:utility} is only an approximation of the segment's true contribution to the current viewpoint rendering.
When bookmarks are defined, it is possible to obtain a better measure of segment utility by performing an offline rendering at each bookmark's viewpoint.
% Then, by simply counting the number of pixels that are rendered using each segment, we can rank the segments by order of importance in the rendering.
We define $\mathcal{U}^{*} (s,B_i)$ as being the optimized utility of a segment $s$ in a viewpoint defined at bookmark $B_i$.
\fresh{}
In order to compute the optimized utility of a segment, we developed Algorithm~\ref{sb:algo-optimal-order}, that sorts segments according to their optimized utility.
This algorithm takes as input the considered viewpoint, the ground truth rendering from this viewpoint and the set of segments (both geometry and texture) to sort.
Starting from an empty model, each segment from the set of candidates is independently added to the scene, and the PSNR between the corresponding render and the ground truth render is computed.
We can thus determine which segment brings the highest $\Delta\text{PSNR} / s$, $s$ being the size of the segment in bytes.
Once the best segment is found, it is registered, and a new iteration begins.
That way, we are able to generate an order of segments sorted by $\Delta\text{PSNR} / s$.
This order is then saved as a JSON file that a client can download in order to know which segments contribute the most to a certain viewpoint.
Sorting all the segments from the model would be an excessively time consuming computation. Sorting all the segments from the model would be an excessively time consuming computation.
To speed up this algorithm, we only sort the 200 first best segments, and we choose these segments among a filtered set of candidates. To speed up this algorithm, we only sort the 200 first best segments, and we choose these segments among a filtered set of candidates.
To find those candidates, we reuse the ideas developed in Chapter~\ref{bi}. To find those candidates, we reuse the ideas developed in Chapter~\ref{bi}.
@ -159,8 +162,8 @@ These renderings allow us to know which geometry segment and which texture corre
\end{figure} \end{figure}
Figure~\ref{sb:precomputation} shows how this precomputation improves the quality of rendering. Figure~\ref{sb:precomputation} shows how this precomputation improves the quality of rendering.
Each curve represents the PSNR one can obtain by downloading a certain amount of data, and they show that, for the same amount of data downloaded, the optimized order reaches a higher PSNR than the greedy order, which means that its utility metric is more accurate. Each curve represents the PSNR one can obtain by downloading a certain amount of data following the greedy policy introduced in Section~\ref{d3:dash-adaptation}. The blue curve, labelled "Default order", is obtained by optimizing the utilities as defined in Section~\ref{d3:utility}, whereas the green curve labelled "Proposed order" uses the sorting computed in Algorithm~\ref{sb:algo-optimal-order}. We can observe that for the same amount of data downloaded, the optimized order reaches a higher PSNR which means that its utility metric is more accurate.
This curve is averaged on all the 9 bookmarks of the scene: we decided the locations of the bookmarks and each bookmark faces an interesting object in the scene. Note that this curve is averaged over all the 9 bookmarks of the scene. These bookmarks are chosen to cover the widest area in the scene, and each one faces a particular object-of-interest.
\begin{figure}[th] \begin{figure}[th]
\centering \centering
@ -192,7 +195,7 @@ This curve is averaged on all the 9 bookmarks of the scene: we decided the locat
\copied{} \copied{}
We now present how to include bookmarks information in the Media Presentation Description (MPD) file. We now present how to include bookmarks information in the Media Presentation Description (MPD) file.
Bookmarks are fully defined by a position, a direction, and the additional content needed to properly render and use a bookmark in a system consists in two files: a thumbnail of the point of view at the bookmark, along with the JSON file giving the optimal segment order for this viewpoint. Bookmarks are fully defined by a position, a direction, and the additional content needed to properly render and use a bookmark in a system consists in two files: a thumbnail of the point of view at the bookmark, along with the JSON file giving the optimal segment order for this viewpoint, as computed by Algorithm~\ref{sb:algo-optimal-order}.
For this reason, for each bookmark, we create a separate adaptation set in the MPD\@. For this reason, for each bookmark, we create a separate adaptation set in the MPD\@.
The bookmarked viewpoint information is stored as a supplemental property. The bookmarked viewpoint information is stored as a supplemental property.
Bookmarks adaptation set only contain one representation, composed of two segments: the thumbnail used as a preview for the desktop interface and the JSON file. Bookmarks adaptation set only contain one representation, composed of two segments: the thumbnail used as a preview for the desktop interface and the JSON file.
@ -225,7 +228,7 @@ The three first values in the supplemental property are the camera position coor
\subsection{Loader modifications} \subsection{Loader modifications}
We build on the loader introduced in Algorithm~\ref{d3:next-segment} to implement a client adaptation logic. We build on the loader introduced in Algorithm~\ref{d3:next-segment} to implement a client adaptation logic.
We include a bookmark adaptation logic such that (i) when a bookmark is hovered for the first time, the corresponding images (see Listing~\ref{sb:bookmark-as}) are downloaded, and (ii) when a bookmark is clicked, we switch from utility $\mathcal{U}$ to optimized utility $\mathcal{U}^*$ to determine which segments to download next. We include a bookmark adaptation logic such that (i) when a bookmark is hovered for the first time, the corresponding thumbnail image as well as the JSON file containing the optimal order of the segments (see Listing~\ref{sb:bookmark-as}) are downloaded, and (ii) when a bookmark is clicked, we switch from utility $\mathcal{U}$ to optimized utility $\mathcal{U}^*$ to determine which segments to download next.
\begin{algorithm}[th] \begin{algorithm}[th]
\SetKwInOut{Input}{input} \SetKwInOut{Input}{input}

View File

@ -3,8 +3,8 @@
\section{Introduction} \section{Introduction}
In Chapter~\ref{bi}, we described how it is possible to modify a user interface to ease user navigation in a 3D scene, and how the system can benefit from it. In Chapter~\ref{bi}, we described how it is possible to modify a user interface to ease user navigation in a 3D scene, and how the system can benefit from it.
In Chapter~\ref{d3}, we presented a streaming system that takes neither the interface nor the user interaction into account. In Chapter~\ref{d3}, we presented the DASH-3D streaming system, which does not depend on the interface nor on the user interaction.
Hence, it is natural to study how the user interaction can impact performances of DASH-3D. In this chapter, we will analyze how the user interaction can impact performances of DASH-3D.
In order to do so, we followed these two steps: In order to do so, we followed these two steps:
\begin{itemize} \begin{itemize}

View File

@ -3,18 +3,18 @@
\minitoc{} \minitoc{}
\newpage \newpage
Nowadays, smartphones are more and more powerful, and people slowly move their applications from their computers to smartphones or tablets. The growing capabilities and usage of mobile devices, especially smartphones, nowadays incur a progressive shift of many applications from desktop to mobile devices. In order to be made available and usable by the greater audience, 3D streaming and visualization should also be possible on mobile devices.
This is why we decided to port our interface to mobile devices.
Desktop devices and mobile devices are very different. However, desktop devices tend to be much more powerful, have a larger memory and better network connections than mobile devices.
There are many differences in terms of performance: desktop devices tend to be much more powerful and have much better memory network connection than mobile devices. In addition, the interactive modalities of these two types of devices are not comparable in any way: the desktop mostly uses keyboard and mouse, whereas most of the mobile devices only have a touchscreen, as well as various additional sensors (accelerometer, gyroscope, GPS, etc.).
Also, the interaction is not comparable in any way: the desktop mostly uses keyboard and mouse, whereas most of the mobile devices only have a touchscreen, as well as many sensors (accelerometer, gyroscope, GPS, etc.). For these reasons, using DASH to stream 3D on mobile devices requires specific adaptations, that we describe in this chapter.
This is why porting our DASH-3D client to mobile is not an easy task.
To do so, we add some widgets on the screen to support touch interaction: a virtual joystick is displayed on the screen and the user can touch it to translate the camera, instead of using the W, A, S and D keys on a computer.
We add some widgets on the screen to support touch interactions: a virtual joystick is displayed on the screen and the user can touch it to translate the camera, instead of using the W, A, S and D keys on a computer keyboard.
Since most mobile devices embed a gyroscope, we allow users to rotate the camera by physically rotating the device. Since most mobile devices embed a gyroscope, we allow users to rotate the camera by physically rotating the device.
This interaction is more precise and intuitive to the user, but it is also more tiring, this is why we also added a touch interaction to rotate the screen: a user can also touch any place on the screen that does not correspond to the joystick to rotate the camera by moving the scene. This interaction is more precise and intuitive to the user, but it is also more tiring, this is why we also added a touch interaction to rotate the screen: a user can also "touch and drag" at any point on the screen that does not correspond to the joystick to rotate the camera.
In order to ease navigation, we integrate bookmarks back, and we enhance the precomputations explained in Chapter~\ref{sb}. In order to ease navigation, we integrate bookmarks back, and we include an enhanced version of the precomputations explained in Chapter~\ref{sb} in the DASH Media Presentation Description.
We then conducted a user study on 18 participants, to test both the interaction and the streaming aspect of the bookmarks. We then present a user study on 18 participants, that evaluate how users perceive the visual quality of the scene, and how their interactions affect it.
\newpage \newpage

View File

@ -3,65 +3,66 @@
\subsection{Preliminary user study} \subsection{Preliminary user study}
Before conducting the user study on mobile devices, we designed a user study for desktop devices. Before conducting the user study on mobile devices, we designed a preliminary user study for desktop devices.
This experiment was conducted on a little more than a dozen of people, with the model described in the previous chapter. This experiment was conducted on twelve users, using the 3D model described in the previous chapter (i.e. the Marina Bay district in Singapore).
Bookmarks were positioned from user-generated panoramic picture available on Google Maps, and the task consisted in retrieving spots on the 3D model from a picture: users were presented with an image coming from Google Street View and they had to find the corresponding spot in the 3D model. Bookmarks were sampled from the set of locations of user-uploaded panoramic pictures available on Google Maps, and the task consisted in matching real-world pictures to their virtual location on the 3D model: users were presented with an image coming from Google Street View and they were asked to find the exact same location in the 3D model.
Due to the fact the task was hard, and that our users were familiar with 3D navigation, they preferred navigating slowly in the scene, and did not use bookmarks as much as they did during the experiment we ran in Chapter~\ref{bi}. Due to the great difficulty of the task was, as well as the relative familiarity of the users with 3D navigation, the user behaviour was biased towards navigating slowly in the scene. The users almost never clicked the bookmarks, much less as they did during the experiment we ran in Chapter~\ref{bi}.
For these reasons, we decided to setup a new experiment, on a model a little larger, with a less complex task, and we decided to conduct this experiment on mobile device exclusively, to see how bookmarks help people navigate in a scene when controls are more cumbersome. For these reasons, we decided to setup a new experiment, with a less complex task, and we decided to conduct this experiment on mobile device exclusively, to see how bookmarks help people navigate in a scene when controls are more cumbersome.
\subsection{Mobile navigation user study} \subsection{Mobile navigation user study}
\subsubsection{Models} \subsubsection{Models}
In this user study, we use two models. In this user study, we display two successive 3D models to the users:
\textbf{A FIGURE WOULD BE NICE !!!}
\begin{itemize} \begin{itemize}
\item For the tutorial, we use a first model from a video game, representing a small scene, to maintain a good framerate and to prevent users from getting lost in the scene. \item For the tutorial phase, we use a model derived from a video game, representing a small scene, in order to prevent users from getting lost in the scene.
\item For all the other parts of the experiment, we used an extended version of the model used in the previous chapter. \item For all the other parts of the experiment, we use a larger version of the Singaporean district 3D model, that include neighbouring districts such as Central Business District.
\end{itemize} \end{itemize}
\subsubsection{Experiment} \subsubsection{Experiment}
The experiment consists in 4 phases: a tutorial, a comparison between interfaces with and without bookmarks, a comparison between two streaming policies, and a final navigation during which the user is looking for objects in the scene. The experiment is articulated into four phases: a tutorial, a comparison between interfaces with and without bookmarks, a comparison between two streaming policies, and a final navigation during which the user is looking for objects in the scene.
\paragraph{Tutorial} \paragraph{Tutorial}
The experiment starts with a tutorial, to get the users accustomed to our interface. The experiment starts with a tutorial, to get the users accustomed to the controls and the interface.
This tutorial shows the different types of interactions available and explains how to use them. This tutorial showcases the different types of interactions available, including bookmarks, and explains how to use them.
It then presents bookmarks to the users.
\paragraph{Bookmark} \paragraph{Bookmark}
This part of the experiment consists in two 1 minute long sessions: the first one has a naked interface where the only available interactions are translations and rotations of the camera, and the second one augments the interface with bookmarks. The second part of the experiment consists in two 1 minute long sessions: the first session displays a bookmarks-free interface where the only available interactions are translations and rotations of the camera, and the second one augments the interface with bookmarks.
There are no special tasks other than to take a walk around the model. There are no special tasks other than to navigate around the model.
This part ends with a small questionnaire where users are asked whether they prefer navigating with bookmarks, and they can use a text field to describe their reasons. The part ends with a small questionnaire where users are asked whether they prefer navigating with or without bookmarks, along with a text field to explain their answer.
The main objective of this part of the experiment is not really to know whether people like using the bookmarks or not: we already know from our previous work and from the other parts of this experiment that they do like using the bookmarks. The main objective of this part of the experiment is not really to know whether people like using the bookmarks or not: we already know from our previous work and from the other parts of this experiment that they do like using the bookmarks.
This part most importantly acts as an extended tutorial: the first half trains the users with the controls, and the second half trains them with the bookmarks, and this is why we decided not to randomize those two halves. This part most importantly acts as an extended tutorial: the first half trains the users with the basic controls, and the second half trains them to specifically use the bookmarks. This is why we decided not to randomize those two steps at this point.
\paragraph{Streaming} \paragraph{Streaming}
This part of the experiment also consists in two 1 minute long sessions that use different streaming policies. This part of the experiment also consists in two 1 minute long sessions that use different streaming policies.
One of those experiment has the default greedy policy described in~\ref{d3:dash-adaptation}, and the other one has the enhanced policy for bookmarks. One of those experiment has the default greedy policy described in~\ref{d3:dash-adaptation}, and the other one has the enhanced policy for bookmarks described in the previous section.
The order of those two sessions is randomized to avoid biases. The order of those two sessions is randomized to avoid biases.
% Since we know that the difference between our streaming policies is subtle, we designed a task a little more complex in order to highlight the differences so that the user can see it. % Since we know that the difference between our streaming policies is subtle, we designed a task a little more complex in order to highlight the differences so that the user can see it.
Since the behaviours of our streaming policy only differ when the user clicks a bookmark, we design a task where the users have to perform a guided tour of the scene, where each bookmark is a step of the tour. Since the behaviours of our streaming policy only differ when the user clicks a bookmark, we design a task where the users have to perform a guided tour of the scene, where each bookmark is a step of the tour.
The user starts in the scene, and one of the bookmarks is blinking. The user starts from anywhere in the scene, and one of the bookmarks is blinking.
The user has to touch the bookmark, and wait a little when he arrives at the destination. The user has to touch the bookmark, and observe the recommended viewpoint for a while when arriving at destination.
Once some data has been downloaded, and the user is satisfied with the data downloaded, they can look for the next blinking bookmarks. Once some data has been downloaded and the user could get a feeling about the quality of the streaming, another bookmark starts blinking to move one with the tour.
This setup is repeated for each streaming policy, and after the two sessions, the users have to answer a questionnaire asking the question \emph{In what session did you find the streaming the smoothest?} This setup is repeated for each streaming policy, and after the two sessions, the users have to answer a questionnaire asking the question \emph{In what session did you find the streaming the smoothest?}
The questionnaire also has a text field for users to explain their answer if they wish. The questionnaire also has a text field for users to explain their answer if they wish.
\paragraph{Free navigation} \paragraph{Free navigation}
The last part of the experiment is a free navigation. The last part of the experiment is a free navigation with an object-finding task.
Diamonds are hidden in the scene, and are invisible until the user is close enough. Diamonds are hidden in the scene, and are invisible until the user is close enough.
The users have to find the diamonds, and they can navigate by using indifferently the controls and the bookmarks. The users have to find the diamonds, and they can navigate by using indifferently the controls and the bookmarks.
The loading policy is the default greedy policy for half of the users, and the enhanced policy for bookmarks for the other half, and this order has been randomized. The loading policy is the default greedy policy for half of the users, and the enhanced policy for bookmarks for the other half, and this order has been randomized.
With this part of the experiment, we hope to see differences in terms of PSNR for the two policies, when users are not forced to click on bookmarks.
This is the most important part of the study, as we aim at observing several aspects. First, we hope that users navigate using the bookmarks. Since no guideline has been given to them as to how to interact, we want to observe whether they naturally use the bookmarks or not.
In addition, we want to prove the superiority of our bookmark-optimized streaming policy by observing that users tend to perceive a better visual quality (as measured by the PSNR).
\subsubsection{Setup} \subsubsection{Setup}
During these experiments, we need a server and a client. During these experiments, we need a server and a client.