tforgione
/
phd-typst
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Adds subsection

This commit is contained in:
Thomas Forgione 2023-04-27 17:40:57 +02:00
parent 77dfebde7d
commit 5b07cf4f8f
3 changed files with 285 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

197
preliminary-work/bookmarks-impact.typ Normal file
View File

@ -0,0 +1,197 @@
= Impact of 3D bookmarks on navigation
We now describe an experiment that we conducted on 51 participants, with two goals in mind.
First, we want to measure the impact of 3D bookmarks on navigation within an NVE\@.
Second, we want to collect traces from the users so that we can replay them for reproducible experiments for comparing streaming strategies in Section~\ref{bi:system}.
== Our NVE
To ease the deployment of our experiments to users in distributed locations on a crowdsourcing platform, we implement a simple web-based NVE client using THREE.js// \footnote{http://threejs.org}.
The NVE server is implemented with node.js. // \footnote{http://nodejs.org}.
The NVE server streams a 3D scene to the client; the client renders the scene as the 3D content is received.
The user can navigate within the NVE in the following way; the camera can be translated using the arrow keys along four directions: forward, backward, to the left, and to the right.
Alternatively, the keys W, A, S and D can also be used for the same actions.
This choice was inspired by 3D video games, which often use these keys in conjunction with the mouse to move an avatar.
The virtual camera can rotate in four different directions using the keys I, K, J and L.
The user can also rotate the camera by dragging the mouse in the desired direction.
Finally, following the UI of popular 3D games, we also give users the possibility to lock their pointer and use their mouse as a virtual camera.
The mouse movement controls the camera rotation.
The user can always choose to lock the pointer, or unlock it using the escape key.
The interface also includes a button to reset the camera back to the starting position in the scene.
== 3D bookmarks
Our NVE supports 3D bookmarks.
A 3D bookmark, or bookmark for short, is simply a fixed camera location (in 3D space), a view direction, and a focal.
Bookmarks visible from the user's current viewpoint are shown as 3D objects in the scene.
Figure X depicts some bookmarks from our NVE.
The user can click on a bookmark object to automatically move and align its viewpoint to that of the bookmark.
The movement follows a Hermite curve joining the current viewpoint to the viewpoint of the bookmark.
The tangent of the curve is the view direction.
The user can hover the mouse pointer over a bookmark object to see a thumbnail view of the 3D scene as seen from the bookmark. (Figure X, bottom left).
In our work, we consider two different possibilities for displaying bookmarks: viewports (Figure X top left) and arrows (Figure X top right).
A viewport is displayed as a pyramid where the top corresponds to the optical center of its viewpoint and the base corresponds to its image plane.
The arrows are view dependent.
The bottom of the arrow turns towards the current position, to better visualize the relative position of the bookmark.
Bookmarks allow the user to achieve a large movement within the 3D environment using a single action (a mouse click).
Since bookmarks are part of the scene, they are visible only when not hidden by other objects from the scene.
We chose size and colors that are salient enough to be easily seen, but not too large to limit the occlusion of regions within the scene.
When reaching the bookmark, the corresponding arrow or viewport is not visible anymore, and subsequently will appear in a different color, to indicate that it has been clicked (similar to web links).
== User study
We now describe in details our experimental setup and the user study that we conducted on 3D navigation.
#heading(level: 3, numbering: none)[Models]
We use four 3D scenes (one for the tutorial and three for the actual experiments) which represent recreated scenes from a famous video game.
Those models are light (a few thousand of triangles per model) and are sent before the experiment starts.
We keep the models small so that users can perform the task with acceptable latency from any country using a decent internet connection.
Our NVE does not stream the 3D content for these experiments, in order to avoid unreliable conditions caused by the network bandwidth variation, which might affect how the users interact.
#heading(level: 3, numbering: none)[Task design]
Since we are interested in studying how efficiently users navigate in the 3D scene, we ask our participants to complete a task which forces them to visit, at least partially, various regions in the scene.
To this end, we hide a set of 8 coins on the scene: participants are asked to collect the coins by clicking on them.
In order to avoid any bias due to the coins position, we predefined 50 possible coin locations per scene, and randomly select 8 out of these 50 positions each time a new participant starts the experiment.
#heading(level: 3, numbering: none)[Experiment]
Participants are first presented with an initial screen to collect some preliminary information: age, gender, the last time they played 3D video games, and self-rated 3D gaming skills. We ask those questions because we believe that someone who is used to playing 3D video games should browse the scene more easily, and thus, may not need to use our bookmarks.
Then, the participants go through a tutorial to learn how the UI works, and how to complete the task.
The different interactions (keyboard navigation, mouse navigation, bookmarks interaction) are progressively introduced to participants, and the tutorial ends once the participant completes an easy version of the task.
The tutorial is always performed on the same scene.
Then, each participant has to complete the task three times.
Each task is performed on a different scene, with a different interface.
Three interfaces are used.
A \NoReco{} interface lets the participant navigates without any bookmarks.
The other two interfaces allow a participant to move using bookmarks displayed as viewports (denoted as \Viewports) and arrows (denoted as \Arrows) respectively.
The coins are chosen randomly, based on the coin configurations that were used by previous participants: if another participant has done an experiment with a certain set of coins, on a certain scene, with a certain type of bookmarks, the current participant will do the experiment with the same set of coins, on the same scene, but with a different type of bookmarks.
This policy allows us to limit the bias that could be caused by coin locations.
Once a participant has found all coins, a button is shown on the interface to let the participant move to the next step.
Alternatively, this button may appear one minute after the sixth coin was found.
This means that a user is authorized to move on without completing the task, in order to avoid potential frustration caused by not finding the remaining two coins.
After completing the three tasks, the participants have to answer a set of questions about their experience with the bookmarks (we refer to the bookmarks as _recommendations_ in the experiments).
@bi:questions shows the list of questions.
#figure(
table(
columns: (auto, auto, auto),
align: left,
[], [*Questions*], [*Answers*],
[1], [What was the difficulty level WITHOUT recommendation?], [3.04 / 5 $plus.minus 0.31$],
[2], [What was the difficulty level WITH recommendation?], [2.15 / 5, $plus.minus 0.30$],
[3], [Did the recommendations help you to find the coins?], [42 Yes, 5 No],
[4], [Did the recommendations help you to browse the scene?], [49 Yes, 2 No],
[5], [Do you think recommendations can be helpful?], [49 Yes, 2 No],
[6], [Which recommendation style do you prefer and why?], [32 \Arrows, 7 \Viewports],
[7], [Did you enjoy this?], [36 Yes, 3 No],
),
caption: [List of questions in the questionnaire and summary of answers. Questions 1 and 2 have a 99% confidence interval.],
)<bi:questions>
#heading(level: 3, numbering: none)[Participants]
The participants were recruited on microworkers.com, a crowdsourcing website.
There were 51 participants (36 men and 15 women), who are in average 30.44 years old.
== Experimental results
We now present the results from our user study, focusing on whether bookmarks help users navigating the 3D scene.
#heading(level: 3, numbering: none)[Questionnaire]
We had 51 responses to the questionnaire.
The answers are summarized in Table~\ref{bi:questions}.
Note that not all questions were answered by all participants.
The participants seem to find the task to be of average difficulty (3.04/5) when they have no bookmarks to help their navigation.
They judge the task to be easier in average (2.15/5) with bookmarks, which indicates that bookmarks ease the completion of the task.
Almost all users (49 out of 51) think the bookmarks are useful for browsing the scene, and most users (42 out of 51) think bookmarks are also useful to complete the given task.
This is slightly in contradiction with our setup; even if coins may appear in some bookmarked viewpoints (which is normal since the viewpoints have been chosen to get the most complete coverage of the scene), most of the time no coin is visible in a given bookmark, and there are always coins that are invisible from all bookmarks.
The strongest result is that almost all users (49 out of 51) find bookmarks to be helpful.
In addition, users seem to have a preference for \Arrows{} against \Viewports{} (32 against 7).
#heading(level: 3, numbering: none)[Analysis of interactions]
#figure(
table(
columns: (auto, auto, auto, auto, auto),
[*BM type*], [*\#Exp*], [*Mean \# coins*], [*\# completed*], [*Mean time*],
[\NoReco], [51], [7.08], [18], [4 min 16 s],
[\Arrows], [51], [7.39], [27], [2 min 33 s],
[\Viewports], [51], [7.51], [30], [2 min 16 s],
),
caption: [Analysis of the sessions length and users success by type of bookmarks],
)<bi:sessions>
@bi:sessions shows basic statistics on task completion given the type of bookmarks that were provided to the participants.
First, we can see that without bookmarks, only a little bit more than a third of the users are able to complete the task, i.e.\ find all 8 coins.
In average, these users find just above 7 coins, and spend 4 minutes and 16 seconds to do it.
Interestingly, and regardless of the bookmark type, users who have bookmarks complete the task more than half of the time, and spend in average significantly less time to complete the task: 2 minutes and 16 seconds using \Viewports{} and 2 minutes and 33 seconds using \Arrows.
Although \Viewports{} seem to help users a little bit more in completing the task than \Arrows{}, the performance difference between both types of bookmarks is not significant enough to conclude on which type of bookmarks is best.
The difference between an interface with bookmarks and without bookmarks, however, is very clear.
Users tend to complete the task more efficiently using bookmarks: more users actually finish the task, and it takes them half the time to do so.
We computed 99\% confidence intervals on the results introduced in Table~\ref{bi:sessions}.
We found that the difference in mean number of coins collected with and without bookmarks is not high enough to be statistically significant: we would need more experiments to reach the significance.
The mean time spent on the task however is statistically significant.
#figure(
table(
columns: (auto, auto, auto, auto),
[*BM type*], [*Total distance*], [*Distance to a bookmark*], [*Ratio*],
[\NoReco], [610.80], [0], [0%],
[\Arrows], [586.30], [369.77], [63%],
[\Viewports], [546.96], [332.72], [61%],
),
caption: [Analysis of the length of the paths by type of bookmarks],
)<bi:paths-lengths>
@bi:paths-lengths presents the length of the paths traveled by users in the scenes.
Although users tend to spend less time on the tasks when they do not have bookmarks, they travel pretty much the same distance as without bookmarks.
As a consequence, they visit the scene faster in average with bookmarks, than without bookmarks.
The table shows that this higher speed is due to the bookmarks, as more than 60\% of the distance traveled by users with bookmarks happens when users click on bookmarks and fly to the destination.
#heading(level: 3, numbering: none)[Discussion]
In the previous paragraphs, we have shown how bookmarks are well perceived by users (looking at the questionnaire answers).
We also showed that users tend to be more efficient in completing the task when they have bookmarks than when they do not.
We can say that bookmarks have a positive impact on navigation within the 3D scene, but since users move, on average, twice as fast, it might have a negative impact on the streaming of objects to the client.
// \begin{figure}[th]
// \centering
// \begin{tikzpicture}
// \begin{axis}[
// xlabel=Time (in s),
// ylabel=Percentage of the scene queried,
// no markers,
// width=\tikzwidth,
// height=\tikzheight,
// cycle list name=mystyle,
// legend pos=south east,
// xmin=0,
// ymin=0,
// ]
//
// \addplot table [y=y1, x=x]{assets/preliminary-work/discovery.dat};
// \addlegendentry{Without bookmarks}
// \addplot table [y=y2, x=x]{assets/preliminary-work/discovery.dat};
// \addlegendentry{With bookmarks}
//
// \end{axis}
// \end{tikzpicture}
// \caption{Comparison of the triangles queried after a certain time}\label{bi:triangles-curve}
// \end{figure}
Figure X shows a CDF of the percentage of 3D mesh triangles in the scene that have been queried by users after a certain time. We plotted this same curve for users with and without bookmarks.
As expected, the fact that the users can browse the scene significantly quicker with bookmarks reflects on the demand on the 3D content.
Users need more triangles more quickly, which either leads to more demand on network bandwidth, or if the bandwidth is kept constant, leads to fewer objects being displayed.
In the next section, we introduce experiments based on our user study traces that show how the rendering is affected by the presence of bookmarks and how to improve it.

31
preliminary-work/intro.typ Normal file
View File

@ -0,0 +1,31 @@
= Introduction
Navigating in NVE with a large virtual space (most times through a 2D interface) is sometimes cumbersome.
In particular, a user may have difficulties reaching the right place to find information.
The content provider of the NVE may want to highlight certain interesting features for the users to view and experience, such as a vantage point in a city, an excavation at an archaeological site, or an exhibit in a museum.
To allow users to easily find these interesting locations within the NVE, _3D bookmarks_ or _bookmarks_ for short, can be provided.
A bookmark is simply a 3D virtual camera (with position and camera parameters) predefined by the content provider, and can be presented to users in different ways, including as a text link (URL), a thumbnail image, or a 3D object embedded within the NVE itself.
When users click on a bookmark, NVEs commonly provide a "fly-to" animation to transit the camera from the current
viewpoint to the destination #cite("controlled-movement-virtual-3d", "browsing-3d-bookmarks") to help orient the users within the 3D space.
Clicking on a bookmark to fly to another viewpoint leads to reduced data locality.
The 3D content at the bookmarked destination viewpoint may overlap less with the current viewpoint.
In the worst case, the 3D objects corresponding to the current and destination viewpoints can be completely disjoint.
Such movement to a bookmark may lead to a _discovery latency_ #cite("second-life"), in which users have to wait for the 3D content for the new viewpoint to be loaded and displayed.
An analogy for this situation, in the context of video streaming, is seeking into a segment of video that has not been prefetched yet.
In this chapter, we explore the impact of bookmarks on NVE navigation and streaming, and make several contributions.
First, we conducted a crowdsourcing experiment where 51 participants navigated in 3 virtual scenes to complete a task.
This experiment serves two purposes: (i) it validates our intuition that bookmarks significantly reduce the number of interactions and navigation time (in average the time needed to complete the task
for users with bookmarks is half the time for users without bookmarks); (ii) it produces a set of user interaction traces that we use for subsequent simulation experiments.
Second, we quantified the effect of bookmarking on prefetching and visual quality in our experiments.
We showed that, without prefetching, the number of correctly rendered pixels right after clicking on bookmarks can drop up to 10% on average.
If we prefetch the 3D content from the bookmarks according to the probability of access, we do not limit this drop by more than 5%.
Finally, we proposed a method to improve the visual quality after clicking on bookmarks, by exploiting the fact that the visible faces at the bookmark can be precomputed, and by fetching the visible faces only after a bookmark is clicked.
We showed that, if the fetching is done during the 1 or 2 seconds of the "fly-to" camera movement from the current viewpoint to the bookmarked viewpoint, it suffices to increase the number of correctly rendered pixels to more than 20%, without wasting bandwidth on prefetching.
Our key message is that, _in addition to easing navigation, bookmarking allows precomputation of visible faces and can significantly reduce interaction latency, without resorting to prefetching_, which may waste bandwidth by prefetching 3D data that will not be needed.
The rest of the chapter consists of the following sections.
Section X describes the 3D bookmarks that we use in our work, along with our experiments to validate the usefulness of bookmarking.
Section X describes the streaming and prefetching mechanisms that we used to simulate our experiments as well as our main findings.
Finally, we conclude in Section X.

57
preliminary-work/main.typ Normal file
View File

@ -0,0 +1,57 @@
#import "../chapter.typ"
#chapter.chapter[Bookmarks, navigation and streaming]
#figure(
grid(
columns: (1fr, 0.2fr, 1fr),
align(center + top)[
#figure(
image("../assets/preliminary-work/bookmarks/camera-bookmark.png", width: 100%),
caption: [Viewport display of a bookmark]
)
],
[],
align(center + top)[
#figure(
image("../assets/preliminary-work/bookmarks/arrow-bookmark.png", width: 100%),
caption: [Arrow display of a bookmark]
)
],
align(center + top)[
#figure(
image("../assets/preliminary-work/bookmarks/arrow-bookmark-with-preview.png", width: 100%),
caption: [A preview is shown when the mouse hovers a bookmark]
)
],
[],
align(center + top)[
#figure(
image("../assets/preliminary-work/bookmarks/coin.png", width: 100%),
caption: [A coin is hidden behind the curtain]
)
],
),
caption: [3D bookmarks propose to move to a new viewpoint; when the user clicks on the bookmark, his viewpoint moves to the indicated viewpoint.]
)
In this chapter, we present our first contribution: an analysis of the impact of bookmarks on navigation and streaming.
We implement a 3D navigation interface where the keyboard translates the camera and the mouse rotates it.
We augment this interface with 3D bookmarks.
When the user's cursor hovers a bookmark, a preview of the point of view is displayed to the user, and when the user clicks on a bookmark, the camera smoothly moves from its current position to the bookmarked point of view.
We conduct a within-subject user-study on 51 participants, where each user starts with a tutorial to get used to the 3D navigation controls, and then tries successively to perform a task with and without bookmarks.
We show that not only the presence of bookmarks causes a faster task completion, but also that it allows users to see a larger part of the scene during the same time span.
However, in a streaming scenario, this phenomenon leads to higher network requirements to maintain the same quality of service.
In the last part of this chapter, we simulate a streaming setup and we show that knowing the positions of the bookmarks beforehand allows us to precompute information that we can reuse during streaming to compensate for the harm caused by the faster navigation with bookmarks.
#pagebreak()
#include "intro.typ"
#include "bookmarks-impact.typ"