tforgione/phd-typst
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Cleaning

Browse Source
This commit is contained in:
Thomas Forgione
2023-05-02 23:55:48 +02:00
parent eb3afbee34
commit fcc78d0c77
13 changed files with 119 additions and 78 deletions
Download Patch File Download Diff File Expand all files Collapse all files
preliminary-work/bookmarks-impact.typ
+19 -15
View File
@@ -1,3 +1,7 @@
#show "NoReco": set text(weight: "bold");
#show "Viewports": set text(weight: "bold");
#show "Arrows": set text(weight: "bold");
== Impact of 3D bookmarks on navigation
We now describe an experiment that we conducted on 51 participants, with two goals in mind.
@@ -65,8 +69,8 @@ The tutorial is always performed on the same scene.
Then, each participant has to complete the task three times.
Each task is performed on a different scene, with a different interface.
Three interfaces are used.
A \NoReco{} interface lets the participant navigates without any bookmarks.
The other two interfaces allow a participant to move using bookmarks displayed as viewports (denoted as \Viewports) and arrows (denoted as \Arrows) respectively.
A NoReco interface lets the participant navigates without any bookmarks.
The other two interfaces allow a participant to move using bookmarks displayed as viewports (denoted as Viewports) and arrows (denoted as Arrows) respectively.
The coins are chosen randomly, based on the coin configurations that were used by previous participants: if another participant has done an experiment with a certain set of coins, on a certain scene, with a certain type of bookmarks, the current participant will do the experiment with the same set of coins, on the same scene, but with a different type of bookmarks.
This policy allows us to limit the bias that could be caused by coin locations.
@@ -88,7 +92,7 @@ After completing the three tasks, the participants have to answer a set of quest
[3], [Did the recommendations help you to find the coins?], [42 Yes, 5 No],
[4], [Did the recommendations help you to browse the scene?], [49 Yes, 2 No],
[5], [Do you think recommendations can be helpful?], [49 Yes, 2 No],
[6], [Which recommendation style do you prefer and why?], [32 \Arrows, 7 \Viewports],
[6], [Which recommendation style do you prefer and why?], [32 Arrows, 7 Viewports],
[7], [Did you enjoy this?], [36 Yes, 3 No],
),
caption: [List of questions in the questionnaire and summary of answers. Questions 1 and 2 have a 99% confidence interval.],
@@ -114,7 +118,7 @@ Almost all users (49 out of 51) think the bookmarks are useful for browsing the
This is slightly in contradiction with our setup; even if coins may appear in some bookmarked viewpoints (which is normal since the viewpoints have been chosen to get the most complete coverage of the scene), most of the time no coin is visible in a given bookmark, and there are always coins that are invisible from all bookmarks.
The strongest result is that almost all users (49 out of 51) find bookmarks to be helpful.
In addition, users seem to have a preference for \Arrows{} against \Viewports{} (32 against 7).
In addition, users seem to have a preference for Arrows against Viewports (32 against 7).
#heading(level: 4, numbering: none)[Analysis of interactions]
@@ -122,24 +126,24 @@ In addition, users seem to have a preference for \Arrows{} against \Viewports{}
table(
columns: (auto, auto, auto, auto, auto),
[*BM type*], [*\#Exp*], [*Mean \# coins*], [*\# completed*], [*Mean time*],
[\NoReco], [51], [7.08], [18], [4 min 16 s],
[\Arrows], [51], [7.39], [27], [2 min 33 s],
[\Viewports], [51], [7.51], [30], [2 min 16 s],
[NoReco], [51], [7.08], [18], [4 min 16 s],
[Arrows], [51], [7.39], [27], [2 min 33 s],
[Viewports], [51], [7.51], [30], [2 min 16 s],
),
caption: [Analysis of the sessions length and users success by type of bookmarks],
)<bi:sessions>
@bi:sessions shows basic statistics on task completion given the type of bookmarks that were provided to the participants.
First, we can see that without bookmarks, only a little bit more than a third of the users are able to complete the task, i.e.\ find all 8 coins.
First, we can see that without bookmarks, only a little bit more than a third of the users are able to complete the task, i.e. find all 8 coins.
In average, these users find just above 7 coins, and spend 4 minutes and 16 seconds to do it.
Interestingly, and regardless of the bookmark type, users who have bookmarks complete the task more than half of the time, and spend in average significantly less time to complete the task: 2 minutes and 16 seconds using \Viewports{} and 2 minutes and 33 seconds using \Arrows.
Although \Viewports{} seem to help users a little bit more in completing the task than \Arrows{}, the performance difference between both types of bookmarks is not significant enough to conclude on which type of bookmarks is best.
Interestingly, and regardless of the bookmark type, users who have bookmarks complete the task more than half of the time, and spend in average significantly less time to complete the task: 2 minutes and 16 seconds using Viewports and 2 minutes and 33 seconds using Arrows.
Although Viewports seem to help users a little bit more in completing the task than Arrows, the performance difference between both types of bookmarks is not significant enough to conclude on which type of bookmarks is best.
The difference between an interface with bookmarks and without bookmarks, however, is very clear.
Users tend to complete the task more efficiently using bookmarks: more users actually finish the task, and it takes them half the time to do so.
We computed 99\% confidence intervals on the results introduced in Table~\ref{bi:sessions}.
We computed 99% confidence intervals on the results introduced in Table~\ref{bi:sessions}.
We found that the difference in mean number of coins collected with and without bookmarks is not high enough to be statistically significant: we would need more experiments to reach the significance.
The mean time spent on the task however is statistically significant.
@@ -147,9 +151,9 @@ The mean time spent on the task however is statistically significant.
table(
columns: (auto, auto, auto, auto),
[*BM type*], [*Total distance*], [*Distance to a bookmark*], [*Ratio*],
[\NoReco], [610.80], [0], [0%],
[\Arrows], [586.30], [369.77], [63%],
[\Viewports], [546.96], [332.72], [61%],
[NoReco], [610.80], [0], [0%],
[Arrows], [586.30], [369.77], [63%],
[Viewports], [546.96], [332.72], [61%],
),
caption: [Analysis of the length of the paths by type of bookmarks],
)<bi:paths-lengths>
@@ -157,7 +161,7 @@ The mean time spent on the task however is statistically significant.
@bi:paths-lengths presents the length of the paths traveled by users in the scenes.
Although users tend to spend less time on the tasks when they do not have bookmarks, they travel pretty much the same distance as without bookmarks.
As a consequence, they visit the scene faster in average with bookmarks, than without bookmarks.
The table shows that this higher speed is due to the bookmarks, as more than 60\% of the distance traveled by users with bookmarks happens when users click on bookmarks and fly to the destination.
The table shows that this higher speed is due to the bookmarks, as more than 60% of the distance traveled by users with bookmarks happens when users click on bookmarks and fly to the destination.
#heading(level: 4, numbering: none)[Discussion]
preliminary-work/main.typ
+1 -1
View File
@@ -1,4 +1,4 @@
= Bookmarks, navigation and streaming
= Bookmarks, navigation and streaming<bi>
#figure(
grid(
preliminary-work/streaming.typ
+32 -25
View File
@@ -1,3 +1,10 @@
#show "V-FD": set text(weight: "bold")
#show "V-PP": set text(weight: "bold")
#show "V-PP+FD": set text(weight: "bold")
#show "C-only": set text(weight: "bold")
#show "culling": set text(weight: "bold")
#show "visible": set text(weight: "bold")
== Impact of 3D bookmarks on streaming
=== 3D model streaming
@@ -64,7 +71,7 @@ Note that the server may send faces that are occluded and not visible to the cli
// \caption{Server side algorithm\label{bi:streaming-algorithm-server}}
// \end{algorithm}
In the following, we shall denote this streaming policy \textsf{culling}; in Figures~\ref{bi:click-1250} and~\ref{bi:click-625} streaming using \textsf{culling} only is denoted \textsf{C-only}.
In the following, we shall denote this streaming policy culling; in Figures~\ref{bi:click-1250} and~\ref{bi:click-625} streaming using culling only is denoted C-only.
=== 3D bookmarks
@@ -76,7 +83,7 @@ We want to exploit bookmarks to improve the user's quality of experience. For th
A bookmarked viewpoint is more likely to be accessed, compared to other arbitrary viewpoint in the 3D scene.
We exploit this fact to perform some precomputation on the 3D content visible from the bookmarked viewpoint.
Recall that \textsf{culling} does not consider occlusion of the faces.
Recall that culling does not consider occlusion of the faces.
Furthermore, it prioritizes the faces according to distance from the camera, and does not consider the actual contribution of the faces to the rendered 2D images.
Ideally, we should prioritize the faces that occupy a bigger area in the 2D rendered images.
Computing this, however, requires rendering the scene at the server, and measuring the area of each face.
@@ -88,16 +95,16 @@ Once rendered, we scan the output image to find the visible triangles (based on
This technique is also used by~\citep{view-dependent-progressive-mesh}.
Thus, when the user clicks on a 3D bookmark, this precomputed list of faces is used by the server, and only visible faces are sent in decreasing order of contributions to the rendered image.
For the three scenes that we used in the experiment, we can reduce the number of triangles sent by 60\% (over all bookmarks).
This reduction is as high as 85.7\% for one particular bookmark (from 26,886 culled triangles to 3,853 culled and visible triangles).
For the three scenes that we used in the experiment, we can reduce the number of triangles sent by 60% (over all bookmarks).
This reduction is as high as 85.7% for one particular bookmark (from 26,886 culled triangles to 3,853 culled and visible triangles).
To illustrate the impact of sorting by projected area of faces, Figure~\ref{bi:sorted-tri} shows the quality improvement gained by sending the precomputed visible triangles prioritized by projected areas, compared to using culling only prioritized by distance.
The curve shows the average quality over all bookmarks over all scenes, for a given number of triangles received.
The quality is measured by the ratio of correctly rendered pixels, comparing the fully and correctly rendered image (when all 3D content is available) and the rendered image (when content is partially available).
We sample one pixel every 100 rows and every 100 columns to compute this value.
The figure shows that, to obtain 90\% of correctly displayed samples, we require 1904 triangles instead of 5752 triangles, about 1/3 savings.
The figure shows that, to obtain 90% of correctly displayed samples, we require 1904 triangles instead of 5752 triangles, about 1/3 savings.
In what follows, we will refer to this streaming policy as \textsf{visible}.
In what follows, we will refer to this streaming policy as visible.
// \begin{figure}[th]
// \centering
@@ -159,14 +166,14 @@ Thus, we hypothesize that prefetching along those paths would lead to better ima
The policy used is the following.
We divide each chunk sent by the server into two parts.
The first part is used to fetch the content from the current viewpoint, using the \textsf{culling} streaming policy.
The first part is used to fetch the content from the current viewpoint, using the culling streaming policy.
The second part is used to prefetch content from the bookmarks, according to their likelihood of being clicked next.
We use the probabilities displayed in Figure~\ref{bi:mat1} to determine the size of each part.
Each bookmark $B$ has a probability $p(B|B_"prev")$ of being clicked next, considering that $B_"prev"$ was the last clicked bookmark.
We assign to each bookmark a certain portion of the chunk to prefetch the corresponding data proportionally to the probability of it being clicked.
We use the \textsf{visible} policy to determine which data should be sent for a bookmark.
We use the visible policy to determine which data should be sent for a bookmark.
We denote this combination as \textsf{V-PP}, for Prefetching based on Prediction using \textsf{visible} policy.
We denote this combination as V-PP, for Prefetching based on Prediction using visible policy.
// \begin{figure}[th]
// \centering
@@ -190,18 +197,18 @@ Indeed, as specified in Section~\ref{bi:3dnavigation}, moving to a bookmarked vi
This transition usually takes from 1 to 2 seconds, depending on how far the current user camera position is from the bookmark.
When the user clicks on the bookmark, the client fetches the visible vertices from the destination viewpoint, with all the available bandwidth.
So, during the transition time, the server no longer does \textsf{culling}, but the whole chunk is used for fetching following \textsf{visible} policy.
So, during the transition time, the server no longer does culling, but the whole chunk is used for fetching following visible policy.
The immediate drawback of this policy is that on the way to the bookmark, the user perception of the scene will be degraded because of the lack of data for the viewpoints in transition.
On the bright side, no time is lost to prefetch bookmarks that will never be consumed, because we fetch only when we are sure that the user has clicked on a bookmark.
This way, when the user is not clicking on bookmarks, we can use the entire bandwidth for the current viewpoint and get as many triangles as possible to improve the current viewpoint.
We call this method \textsf{V-FD}, since we are Fetching the 3D data from the Destination using \textsf{visible} policy.
We call this method V-FD, since we are Fetching the 3D data from the Destination using visible policy.
// \begin{table}
// \centering
// \begin{tabular}{ccccc}
// \toprule
// & \textsf{Visible} & \textsf{V-FD} & \textsf{V-PP} & \textsf{V-PP+FD} \\ \midrule
// & \textsf{Visible} & V-FD & V-PP & V-PP+FD \\ \midrule
// \textbf{Frustum culling} &\cmark&\cmark&\cmark&\cmark\\
// \textbf{Fetch destination} &\xmark&\cmark&\xmark&\cmark\\
// \textbf{Prefetch predicted} &\xmark&\xmark&\cmark&\cmark\\\bottomrule
@@ -213,7 +220,7 @@ We call this method \textsf{V-FD}, since we are Fetching the 3D data from the De
In order to determine which policy to use, we replay the traces from the user study while simulating different streaming policies.
The first point we are interested in is which streaming policy leads to the lower discovery latency and better image
quality for the user: \textsf{culling} (no prefetching), \textsf{V-PP} (prefetching based on probability of accessing bookmarks), or \textsf{V-FD} (no prefetching, but fetch the destination during "fly-to" transition) or combining both \textsf{V-PP} and \textsf{V-FD} (\textsf{V-PP+FD}).
quality for the user: culling (no prefetching), V-PP (prefetching based on probability of accessing bookmarks), or V-FD (no prefetching, but fetch the destination during "fly-to" transition) or combining both V-PP and V-FD (V-PP+FD).
// \begin{figure}[th]
// \centering
@@ -247,14 +254,14 @@ quality for the user: \textsf{culling} (no prefetching), \textsf{V-PP} (prefetch
Figure~\ref{bi:click-1250} compares the quality of the view of a user after their first click on a bookmark.
The ratio of pixels correctly displayed is computed in the client algorithm, see also algorithm~\ref{bi:streaming-algorithm-client}.
In this figure we use a bandwidth of 1 Mbps.
The blue curve corresponds to the \textsf{culling} policy.
The blue curve corresponds to the culling policy.
Clicking on a bookmark generates a user path with less spatial locality, causing a large drop in visual quality that is only compensated after 4 seconds.
During the first second, the camera moves from the current viewport to the bookmarked viewport.
When the data has been prefetched according to the probability of the bookmark to be clicked, the drop in quality is less visible (\textsf{V-PP} curve).
However, by benefiting from the precomputation of visible triangles and ordering of the important triangles in a bookmark (\textsf{V-FD}) the drop in quality is still there, but is very short (approximately four times shorter than for \textsf{culling}).
When the data has been prefetched according to the probability of the bookmark to be clicked, the drop in quality is less visible (V-PP curve).
However, by benefiting from the precomputation of visible triangles and ordering of the important triangles in a bookmark (V-FD) the drop in quality is still there, but is very short (approximately four times shorter than for culling).
This drop in quality is happening during the transition on the path.
More quantitatively, with a $1$ Mbps bandwidth, 3 seconds are necessary after the click to recover $90\%$ of correct pixels.
More quantitatively, with a $1$ Mbps bandwidth, 3 seconds are necessary after the click to recover $90%$ of correct pixels.
// \begin{figure}[th]
// \centering
@@ -304,21 +311,21 @@ More quantitatively, with a $1$ Mbps bandwidth, 3 seconds are necessary after th
// \addplot table [y=y1, x=x]{assets/preliminary-work/evaluation/click-curves-local-2500.dat};
// \addlegendentry{V-FD}
// \addplot table [y=y2, x=x]{assets/preliminary-work/evaluation/click-curves-local-2500.dat};
// \addlegendentry{V-PP-FD}
// \addlegendentry{V-PP+FD}
// \end{axis}
// \end{tikzpicture}
// \caption{Same curve as Figures~\ref{bi:click-1250} and~\ref{bi:click-625}, for comparing streaming policies \textsf{V-FD} alone and \textsf{V-PP+FD}. BW=2Mbps\label{bi:2MB}}
// \caption{Same curve as Figures~\ref{bi:click-1250} and~\ref{bi:click-625}, for comparing streaming policies V-FD alone and V-PP+FD. BW=2Mbps\label{bi:2MB}}
// \end{figure}
Figure~\ref{bi:click-625} showed the results of the same experiment with 0.5 Mbps bandwidth. Here, it takes 4 to 5 seconds to recover $85%$ of the pixels with \textsf{culling} and \textsf{V-PP}, against 1.5 second for recovering $90%$ with \textsf{V-FD}.
Combining both strategies (\textsf{V-PP+FD}) leads to the best quality.
Figure~\ref{bi:click-625} showed the results of the same experiment with 0.5 Mbps bandwidth. Here, it takes 4 to 5 seconds to recover $85%$ of the pixels with culling and V-PP, against 1.5 second for recovering $90%$ with V-FD.
Combining both strategies (V-PP+FD) leads to the best quality.
At 1 Mbps bandwidth, \textsf{V-PP} penalizes the quality, as the curve \textsf{V-PP-FD} leads to a lower quality image than \textsf{V-FD} alone.
At 1 Mbps bandwidth, V-PP penalizes the quality, as the curve V-PP+FD leads to a lower quality image than V-FD alone.
This effect is even stronger when the bandwidth is set to 2 Mbps (Figure~\ref{bi:2MB}).
Both streaming strategies based on the precomputation of the ordering improves the image quality.
We see here, that \textsf{V-FD} has a greater impact than \textsf{V-PP}.
Here, \textsf{V-PP} may prefetch content that eventually may not be used, whereas \textsf{V-FD} only sends relevant 3D content (knowing which bookmark has been just clicked).
We see here, that V-FD has a greater impact than V-PP.
Here, V-PP may prefetch content that eventually may not be used, whereas V-FD only sends relevant 3D content (knowing which bookmark has been just clicked).
We present only the results after the first click.
For subsequent clicks, we found that other factors came into play and thus, it is hard to analyze the impact of the various streaming policies.
@@ -326,7 +333,7 @@ For instance, a user may revisit a previously visited bookmark, or the bookmarks
If the users click on a subsequent bookmark after a long period, then more content would have been fetched for this user, making comparisons difficult.
To summarize, we found that exploiting the fact that bookmarked viewpoints are frequently visited to precompute the visible faces and sort them according to projected areas can lead to significant improvement in image quality after a user interaction (clicking on a bookmark).
This alone can lead to 60\% less triangles being sent, with 1/3 of the triangles sufficient to ensure 90% of pixels correctly rendered, compared to doing frustum/backface culling.
This alone can lead to 60% less triangles being sent, with 1/3 of the triangles sufficient to ensure 90% of pixels correctly rendered, compared to doing frustum/backface culling.
If we fetch these precomputed faces of the destination viewpoint this way immediately after the click, during the "fly-to" transition, then we can already significantly improve the quality without any prefetching.
Prefetching helps if the bandwidth is low, and fewer triangles can be downloaded during this transition.
The network conditions play a minimum role in this key message --- bookmarking allows precomputation of an ordered list of visible faces, and this holds regardless of the underlying network condition (except for non-interesting extreme cases, such as negligible bandwidth or abundance of bandwidth).