Cleaning
This commit is contained in:
parent
eb3afbee34
commit
fcc78d0c77
|
|
@ -1,12 +1,12 @@
|
|||
#heading(level: 2, numbering: none)[Acknowledgments]
|
||||
|
||||
// Directeurs de thèse / WTO
|
||||
First of all, I would like to thank my advisors, Vincent CHARVILLAT, Axel CARLIER, and Géraldine MORIN for luring me into doing a PhD (which was a lot of work), for the support, and for the fun (and beers). // TODO \footnote{drink responsibly}
|
||||
First of all, I would like to thank my advisors, Vincent #smallcaps("Charvillat"), Axel #smallcaps("Carlier"), and Géraldine #smallcaps("Morin") for luring me into doing a PhD (which was a lot of work), for the support, and for the fun (and beers). // TODO \footnote{drink responsibly}
|
||||
I took _a little time_ to take the decision of starting a PhD, so I also want to thank them for their patience.
|
||||
I also want to thank Wei Tsang OOI, for the ideas, the discussions and for the polish during the deadlines.
|
||||
|
||||
// Reviewers / membres du jury
|
||||
Then, I want to thank Sidonie CHRISTOPHE and Gwendal SIMON for reviewing this manuscript: I appreciated the feedback and constructive comments.
|
||||
Then, I want to thank Sidonie #smallcaps("Christophe") and Gwendal #smallcaps("Simon") for reviewing this manuscript: I appreciated the feedback and constructive comments.
|
||||
I also want to thank all the members of the jury, for their attention and the interesting discussions during the defense.
|
||||
|
||||
// Potes
|
||||
|
|
|
|||
24
cover.typ
24
cover.typ
|
|
@ -1,29 +1,33 @@
|
|||
#set page(background: image("assets/background.png", width: 100%))
|
||||
#set text(fill: white)
|
||||
#set par(first-line-indent: 0em, leading: 0.65em)
|
||||
|
||||
#align(center + bottom)[
|
||||
#rect(width: 120%, fill: rgb(0, 0, 0))[
|
||||
#pad(5pt, text(weight: "bold", size: 20pt)[Dynamic Adaptive 3D Streaming over HTTP])
|
||||
#v(0.5em)
|
||||
#text(weight: "bold")[For the University of Toulouse PhD granted by the INP Toulouse]\
|
||||
#text(weight: "bold")[Presented and defended on Friday 29th November, 2019 by Thomas Forgione]
|
||||
#v(0.5em)
|
||||
|
||||
*Gilles GESQUIÈRE*, president\
|
||||
*Sidonie CHRISTOPHE*, reviewer \
|
||||
*Gwendal SIMON*, reviewer\
|
||||
*Maarten WIJNANTS*, examiner\
|
||||
*Wei Tsang OOI*, examiner\
|
||||
*Vincent CHARVILLAT*, thesis supervisor\
|
||||
*Axel CARLIER*, thesis co-supervisor\
|
||||
*Géraldine MORIN*, thesis co-supervisor
|
||||
*Gilles #smallcaps("Gesquière")*, president\
|
||||
*Sidonie #smallcaps("Christophe")*, reviewer \
|
||||
*Gwendal #smallcaps("Simon")*, reviewer\
|
||||
*Maarten #smallcaps("Wijnants")*, examiner\
|
||||
*Wei Tsang #smallcaps("Ooi")*, examiner\
|
||||
*Vincent #smallcaps("Charvillat")*, thesis supervisor\
|
||||
*Axel #smallcaps("Carlier")*, thesis co-supervisor\
|
||||
*Géraldine #smallcaps("Morin")*, thesis co-supervisor
|
||||
|
||||
#set text(size: 10pt)
|
||||
|
||||
#v(0.5em)
|
||||
#align(left, [*Doctoral school and field*: EDMITT: École Doctorale de Mathématiques, Informatiques et Télécommunications de
|
||||
Toulouse\
|
||||
*Field*: Computer science and telecommunication\
|
||||
*Research unit*: IRIT (5505)\
|
||||
*Thesis supervisors*: Vincent CHARVILLAT, Axel CARLIER and Géraldine MORIN\
|
||||
*Reviewers*: Sidonie CHRISTOPHE and Gwendal SIMON
|
||||
*Thesis supervisors*: Vincent #smallcaps("Charvillat"), Axel #smallcaps("Carlier") and Géraldine #smallcaps("Morin")\
|
||||
*Reviewers*: Sidonie #smallcaps("Christophe") and Gwendal #smallcaps("Simon")
|
||||
])
|
||||
]
|
||||
]
|
||||
|
|
|
|||
|
|
@ -1,9 +1,9 @@
|
|||
A 3D streaming system is a system that progressively collects 3D data.
|
||||
The previous chapter voluntarily remained vague about what \emph{3D data} actually are.
|
||||
The previous chapter voluntarily remained vague about what _3D data_ actually are.
|
||||
This chapter presents in detail the 3D data we consider and how they are rendered.
|
||||
We also give insights about interaction and streaming by comparing the 3D setting to the video one.
|
||||
|
||||
== What is a 3D model?
|
||||
== What is a 3D model?<f:3d>
|
||||
|
||||
The 3D models we are interested in are sets of textured meshes, which can potentially be arranged in a scene graph.
|
||||
Such models can typically contain the following:
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ When it comes to 3D streaming systems, we need two kind of software.
|
|||
|
||||
=== JavaScript
|
||||
|
||||
#heading(level: 4, numbering: none)[THREE.js]
|
||||
==== THREE.js
|
||||
|
||||
On the web browser, it is now possible to perform 3D rendering by using WebGL.
|
||||
However, WebGL is very low level and it can be painful to write code, even to render a simple triangle.
|
||||
|
|
@ -37,7 +37,7 @@ A snippet of the basic usage of these classes is given in @three-hello-world.
|
|||
caption: [A THREE.js _hello world_]
|
||||
)<three-hello-world>
|
||||
|
||||
#heading(level: 4, numbering: none)[Geometries]
|
||||
==== Geometries
|
||||
|
||||
Geometries are the classes that hold the vertices, texture coordinates, normals and faces.
|
||||
THREE.js proposes two classes for handling geometries:
|
||||
|
|
@ -49,7 +49,7 @@ THREE.js proposes two classes for handling geometries:
|
|||
|
||||
In this section, we explain the specificities of Rust and why it is an adequate language for writing efficient native software safely.
|
||||
|
||||
#heading(level: 4, numbering: none)[Borrow checker]
|
||||
==== Borrow checker
|
||||
|
||||
Rust is a system programming language focused on safety.
|
||||
It is made to be efficient (and effectively has performances comparable to C // TODO \footnote{\url{https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/rust.html}} or C++\footnote{\url{https://benchmarksgame-team.pages.debian.net/benchmarksgame/fastest/rust-gpp.html}})
|
||||
|
|
@ -142,7 +142,7 @@ The borrow checker may seem like an enemy to newcomers because it often rejects
|
|||
It is probably for those reasons that Rust is the _most loved programming language_ according to the Stack Overflow
|
||||
Developer Survey // TODO in~\citeyear{so-survey-2016}, \citeyear{so-survey-2017}, \citeyear{so-survey-2018} and~\citeyear{so-survey-2019}.
|
||||
|
||||
#heading(level: 4, numbering: none)[Tooling]
|
||||
==== Tooling
|
||||
|
||||
Moreover, Rust comes with many programs that help developers.
|
||||
- #link("https://github.com/rust-lang/rust")[*`rustc`*] is the Rust compiler. It is comfortable due to the clarity and precise explanations of its error messages.
|
||||
|
|
@ -152,7 +152,7 @@ Moreover, Rust comes with many programs that help developers.
|
|||
- #link("https://github.com/rust-lang/rustfmt")[*`rustfmt`*] auto formats code.
|
||||
- #link("https://github.com/rust-lang/rust-clippy")[*`clippy`*] is a linter that detects unidiomatic code and suggests modifications.
|
||||
|
||||
#heading(level: 4, numbering: none)[Glium]
|
||||
==== Glium
|
||||
|
||||
When we need to perform rendering for 3D content analysis or for evaluation, we use the #link("https://github.com/glium/glium")[*`glium`*] library.
|
||||
Glium has many advantages over using raw OpenGL calls.
|
||||
|
|
@ -163,7 +163,7 @@ Its objectives are:
|
|||
- to be fast: the binary produced use optimized OpenGL functions calls;
|
||||
- to be compatible: glium seeks to support the latest versions of OpenGL functions and falls back to older functions if the most recent ones are not supported on the device.
|
||||
|
||||
#heading(level: 4, numbering: none)[Conclusion]
|
||||
==== Conclusion
|
||||
|
||||
In our work, many tasks will consist in 3D content analysis, reorganization, rendering and evaluation.
|
||||
Many of these tasks require long computations, lasting from hours to entire days.
|
||||
|
|
|
|||
|
|
@ -1,5 +1,7 @@
|
|||
== Open problems
|
||||
|
||||
#set heading(numbering: none, outlined: false)
|
||||
|
||||
The objective of our work is to design a system which allows a user to access remote 3D content.
|
||||
A 3D streaming client has lots of tasks to accomplish:
|
||||
|
||||
|
|
@ -12,25 +14,30 @@ A 3D streaming client has lots of tasks to accomplish:
|
|||
|
||||
This opens multiple problems which need to be considered and will be studied in this thesis.
|
||||
|
||||
#heading(level: 4, numbering: none)[Content preparation]
|
||||
|
||||
=== Content preparation
|
||||
|
||||
Before streaming content, it needs to be prepared.
|
||||
The segmentation of the content into chunks is particularly important for streaming since it allows transmitting only a portion of the data to the client.
|
||||
The downloaded chunks can be rendered while more chunks are being downloaded.
|
||||
Content preparation also includes compression.
|
||||
One of the questions this thesis has to answer is: _what is the best way to prepare 3D content so that a streaming client can progressively download and render the 3D model?_
|
||||
|
||||
#heading(level: 4, numbering: none)[Streaming policies]
|
||||
=== Streaming policies
|
||||
|
||||
Once our content is prepared and split in chunks, a client needs to determine which chunks should be downloaded first.
|
||||
A chunk that contains data in the field of view of the user is more relevant than a chunk that is not inside; a chunk that is close to the camera is more relevant than a chunk far away from the camera.
|
||||
This should also include other contextual parameters, such as the size of a chunk, the bandwidth and the user's behaviour.
|
||||
In order to propose efficient streaming policies, we need to know _how to estimate a chunk utility, and how to determine which chunks need to be downloaded depending the user's interactions?_
|
||||
|
||||
#heading(level: 4, numbering: none)[Evaluation]
|
||||
=== Evaluation
|
||||
|
||||
In such systems, two commonly used criteria for evaluation are quality of service, and quality of experience.
|
||||
The quality of service is a network-centric metric, which considers values such as throughput and measures how well the content is served to the client.
|
||||
The quality of experience is a user-centric metric: it relies on user perception and can only be measured by asking how users feel about a system.
|
||||
To be able to know which streaming policies are best, one needs to know _how to compare streaming policies and evaluate the impact of their parameters on the quality of service of the streaming system and on the quality of experience of the final user?_
|
||||
|
||||
#heading(level: 4, numbering: none)[Implementation]
|
||||
=== Implementation
|
||||
|
||||
The objective of our work is to setup a client-server architecture that answers the above problems: content preparation, chunk utility, streaming policies.
|
||||
In this regard, we have to find out _how do we build this architecture that keeps a low computational load on the server so it scales up and on the client so that it has enough resources to perform the tasks described above?_
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
#heading(level: 1, numbering: none)[Introduction]
|
||||
|
||||
#set heading(numbering: (..nums) => {
|
||||
nums.pos().slice(1).join(".")
|
||||
nums.pos().slice(1).map(str).join(".") + " "
|
||||
})
|
||||
|
||||
During the last years, 3D acquisition and modeling techniques have made tremendous progress.
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ This chapter starts with an analysis of the video streaming standards.
|
|||
Then it reviews the different 3D streaming approaches.
|
||||
The last section of this chapter focuses on 3D interaction.
|
||||
|
||||
Then, in Chapter X, we present our first contribution: an in-depth analysis of the impact of the UI on navigation and streaming in a 3D scene.
|
||||
Then, in @bi, we present our first contribution: an in-depth analysis of the impact of the UI on navigation and streaming in a 3D scene.
|
||||
We first develop a basic interface for navigating in 3D and then, we introduce 3D objects called _bookmarks_ that help users navigating in the scene.
|
||||
We then present a user study that we conducted on 51 people which shows that bookmarks ease user navigation: they improve performance at tasks such as finding objects.
|
||||
We analyze how the presence of bookmarks impacts the streaming: we propose and evaluate streaming policies based on precomputations relying on bookmarks and that measurably increase the quality of experience.
|
||||
|
|
|
|||
10
main.typ
10
main.typ
|
|
@ -3,9 +3,19 @@
|
|||
#show: doc => template.phd(doc)
|
||||
|
||||
#include "cover.typ"
|
||||
|
||||
#include "abstracts/main.typ"
|
||||
|
||||
#pagebreak()
|
||||
#include "acknowledgments.typ"
|
||||
|
||||
#pagebreak()
|
||||
|
||||
#[
|
||||
#set par(first-line-indent: 0em, justify: true, leading: 1em)
|
||||
#outline(indent: true, depth: 3)
|
||||
]
|
||||
|
||||
#pagebreak()
|
||||
#include "introduction/main.typ"
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,7 @@
|
|||
#show "NoReco": set text(weight: "bold");
|
||||
#show "Viewports": set text(weight: "bold");
|
||||
#show "Arrows": set text(weight: "bold");
|
||||
|
||||
== Impact of 3D bookmarks on navigation
|
||||
|
||||
We now describe an experiment that we conducted on 51 participants, with two goals in mind.
|
||||
|
|
@ -65,8 +69,8 @@ The tutorial is always performed on the same scene.
|
|||
Then, each participant has to complete the task three times.
|
||||
Each task is performed on a different scene, with a different interface.
|
||||
Three interfaces are used.
|
||||
A \NoReco{} interface lets the participant navigates without any bookmarks.
|
||||
The other two interfaces allow a participant to move using bookmarks displayed as viewports (denoted as \Viewports) and arrows (denoted as \Arrows) respectively.
|
||||
A NoReco interface lets the participant navigates without any bookmarks.
|
||||
The other two interfaces allow a participant to move using bookmarks displayed as viewports (denoted as Viewports) and arrows (denoted as Arrows) respectively.
|
||||
|
||||
The coins are chosen randomly, based on the coin configurations that were used by previous participants: if another participant has done an experiment with a certain set of coins, on a certain scene, with a certain type of bookmarks, the current participant will do the experiment with the same set of coins, on the same scene, but with a different type of bookmarks.
|
||||
This policy allows us to limit the bias that could be caused by coin locations.
|
||||
|
|
@ -88,7 +92,7 @@ After completing the three tasks, the participants have to answer a set of quest
|
|||
[3], [Did the recommendations help you to find the coins?], [42 Yes, 5 No],
|
||||
[4], [Did the recommendations help you to browse the scene?], [49 Yes, 2 No],
|
||||
[5], [Do you think recommendations can be helpful?], [49 Yes, 2 No],
|
||||
[6], [Which recommendation style do you prefer and why?], [32 \Arrows, 7 \Viewports],
|
||||
[6], [Which recommendation style do you prefer and why?], [32 Arrows, 7 Viewports],
|
||||
[7], [Did you enjoy this?], [36 Yes, 3 No],
|
||||
),
|
||||
caption: [List of questions in the questionnaire and summary of answers. Questions 1 and 2 have a 99% confidence interval.],
|
||||
|
|
@ -114,7 +118,7 @@ Almost all users (49 out of 51) think the bookmarks are useful for browsing the
|
|||
This is slightly in contradiction with our setup; even if coins may appear in some bookmarked viewpoints (which is normal since the viewpoints have been chosen to get the most complete coverage of the scene), most of the time no coin is visible in a given bookmark, and there are always coins that are invisible from all bookmarks.
|
||||
|
||||
The strongest result is that almost all users (49 out of 51) find bookmarks to be helpful.
|
||||
In addition, users seem to have a preference for \Arrows{} against \Viewports{} (32 against 7).
|
||||
In addition, users seem to have a preference for Arrows against Viewports (32 against 7).
|
||||
|
||||
#heading(level: 4, numbering: none)[Analysis of interactions]
|
||||
|
||||
|
|
@ -122,24 +126,24 @@ In addition, users seem to have a preference for \Arrows{} against \Viewports{}
|
|||
table(
|
||||
columns: (auto, auto, auto, auto, auto),
|
||||
[*BM type*], [*\#Exp*], [*Mean \# coins*], [*\# completed*], [*Mean time*],
|
||||
[\NoReco], [51], [7.08], [18], [4 min 16 s],
|
||||
[\Arrows], [51], [7.39], [27], [2 min 33 s],
|
||||
[\Viewports], [51], [7.51], [30], [2 min 16 s],
|
||||
[NoReco], [51], [7.08], [18], [4 min 16 s],
|
||||
[Arrows], [51], [7.39], [27], [2 min 33 s],
|
||||
[Viewports], [51], [7.51], [30], [2 min 16 s],
|
||||
),
|
||||
caption: [Analysis of the sessions length and users success by type of bookmarks],
|
||||
)<bi:sessions>
|
||||
|
||||
@bi:sessions shows basic statistics on task completion given the type of bookmarks that were provided to the participants.
|
||||
|
||||
First, we can see that without bookmarks, only a little bit more than a third of the users are able to complete the task, i.e.\ find all 8 coins.
|
||||
First, we can see that without bookmarks, only a little bit more than a third of the users are able to complete the task, i.e. find all 8 coins.
|
||||
In average, these users find just above 7 coins, and spend 4 minutes and 16 seconds to do it.
|
||||
|
||||
Interestingly, and regardless of the bookmark type, users who have bookmarks complete the task more than half of the time, and spend in average significantly less time to complete the task: 2 minutes and 16 seconds using \Viewports{} and 2 minutes and 33 seconds using \Arrows.
|
||||
Although \Viewports{} seem to help users a little bit more in completing the task than \Arrows{}, the performance difference between both types of bookmarks is not significant enough to conclude on which type of bookmarks is best.
|
||||
Interestingly, and regardless of the bookmark type, users who have bookmarks complete the task more than half of the time, and spend in average significantly less time to complete the task: 2 minutes and 16 seconds using Viewports and 2 minutes and 33 seconds using Arrows.
|
||||
Although Viewports seem to help users a little bit more in completing the task than Arrows, the performance difference between both types of bookmarks is not significant enough to conclude on which type of bookmarks is best.
|
||||
|
||||
The difference between an interface with bookmarks and without bookmarks, however, is very clear.
|
||||
Users tend to complete the task more efficiently using bookmarks: more users actually finish the task, and it takes them half the time to do so.
|
||||
We computed 99\% confidence intervals on the results introduced in Table~\ref{bi:sessions}.
|
||||
We computed 99% confidence intervals on the results introduced in Table~\ref{bi:sessions}.
|
||||
We found that the difference in mean number of coins collected with and without bookmarks is not high enough to be statistically significant: we would need more experiments to reach the significance.
|
||||
The mean time spent on the task however is statistically significant.
|
||||
|
||||
|
|
@ -147,9 +151,9 @@ The mean time spent on the task however is statistically significant.
|
|||
table(
|
||||
columns: (auto, auto, auto, auto),
|
||||
[*BM type*], [*Total distance*], [*Distance to a bookmark*], [*Ratio*],
|
||||
[\NoReco], [610.80], [0], [0%],
|
||||
[\Arrows], [586.30], [369.77], [63%],
|
||||
[\Viewports], [546.96], [332.72], [61%],
|
||||
[NoReco], [610.80], [0], [0%],
|
||||
[Arrows], [586.30], [369.77], [63%],
|
||||
[Viewports], [546.96], [332.72], [61%],
|
||||
),
|
||||
caption: [Analysis of the length of the paths by type of bookmarks],
|
||||
)<bi:paths-lengths>
|
||||
|
|
@ -157,7 +161,7 @@ The mean time spent on the task however is statistically significant.
|
|||
@bi:paths-lengths presents the length of the paths traveled by users in the scenes.
|
||||
Although users tend to spend less time on the tasks when they do not have bookmarks, they travel pretty much the same distance as without bookmarks.
|
||||
As a consequence, they visit the scene faster in average with bookmarks, than without bookmarks.
|
||||
The table shows that this higher speed is due to the bookmarks, as more than 60\% of the distance traveled by users with bookmarks happens when users click on bookmarks and fly to the destination.
|
||||
The table shows that this higher speed is due to the bookmarks, as more than 60% of the distance traveled by users with bookmarks happens when users click on bookmarks and fly to the destination.
|
||||
|
||||
#heading(level: 4, numbering: none)[Discussion]
|
||||
|
||||
|
|
|
|||
|
|
@ -1,4 +1,4 @@
|
|||
= Bookmarks, navigation and streaming
|
||||
= Bookmarks, navigation and streaming<bi>
|
||||
|
||||
#figure(
|
||||
grid(
|
||||
|
|
|
|||
|
|
@ -1,3 +1,10 @@
|
|||
#show "V-FD": set text(weight: "bold")
|
||||
#show "V-PP": set text(weight: "bold")
|
||||
#show "V-PP+FD": set text(weight: "bold")
|
||||
#show "C-only": set text(weight: "bold")
|
||||
#show "culling": set text(weight: "bold")
|
||||
#show "visible": set text(weight: "bold")
|
||||
|
||||
== Impact of 3D bookmarks on streaming
|
||||
|
||||
=== 3D model streaming
|
||||
|
|
@ -64,7 +71,7 @@ Note that the server may send faces that are occluded and not visible to the cli
|
|||
// \caption{Server side algorithm\label{bi:streaming-algorithm-server}}
|
||||
// \end{algorithm}
|
||||
|
||||
In the following, we shall denote this streaming policy \textsf{culling}; in Figures~\ref{bi:click-1250} and~\ref{bi:click-625} streaming using \textsf{culling} only is denoted \textsf{C-only}.
|
||||
In the following, we shall denote this streaming policy culling; in Figures~\ref{bi:click-1250} and~\ref{bi:click-625} streaming using culling only is denoted C-only.
|
||||
|
||||
=== 3D bookmarks
|
||||
|
||||
|
|
@ -76,7 +83,7 @@ We want to exploit bookmarks to improve the user's quality of experience. For th
|
|||
A bookmarked viewpoint is more likely to be accessed, compared to other arbitrary viewpoint in the 3D scene.
|
||||
We exploit this fact to perform some precomputation on the 3D content visible from the bookmarked viewpoint.
|
||||
|
||||
Recall that \textsf{culling} does not consider occlusion of the faces.
|
||||
Recall that culling does not consider occlusion of the faces.
|
||||
Furthermore, it prioritizes the faces according to distance from the camera, and does not consider the actual contribution of the faces to the rendered 2D images.
|
||||
Ideally, we should prioritize the faces that occupy a bigger area in the 2D rendered images.
|
||||
Computing this, however, requires rendering the scene at the server, and measuring the area of each face.
|
||||
|
|
@ -88,16 +95,16 @@ Once rendered, we scan the output image to find the visible triangles (based on
|
|||
This technique is also used by~\citep{view-dependent-progressive-mesh}.
|
||||
Thus, when the user clicks on a 3D bookmark, this precomputed list of faces is used by the server, and only visible faces are sent in decreasing order of contributions to the rendered image.
|
||||
|
||||
For the three scenes that we used in the experiment, we can reduce the number of triangles sent by 60\% (over all bookmarks).
|
||||
This reduction is as high as 85.7\% for one particular bookmark (from 26,886 culled triangles to 3,853 culled and visible triangles).
|
||||
For the three scenes that we used in the experiment, we can reduce the number of triangles sent by 60% (over all bookmarks).
|
||||
This reduction is as high as 85.7% for one particular bookmark (from 26,886 culled triangles to 3,853 culled and visible triangles).
|
||||
|
||||
To illustrate the impact of sorting by projected area of faces, Figure~\ref{bi:sorted-tri} shows the quality improvement gained by sending the precomputed visible triangles prioritized by projected areas, compared to using culling only prioritized by distance.
|
||||
The curve shows the average quality over all bookmarks over all scenes, for a given number of triangles received.
|
||||
The quality is measured by the ratio of correctly rendered pixels, comparing the fully and correctly rendered image (when all 3D content is available) and the rendered image (when content is partially available).
|
||||
We sample one pixel every 100 rows and every 100 columns to compute this value.
|
||||
The figure shows that, to obtain 90\% of correctly displayed samples, we require 1904 triangles instead of 5752 triangles, about 1/3 savings.
|
||||
The figure shows that, to obtain 90% of correctly displayed samples, we require 1904 triangles instead of 5752 triangles, about 1/3 savings.
|
||||
|
||||
In what follows, we will refer to this streaming policy as \textsf{visible}.
|
||||
In what follows, we will refer to this streaming policy as visible.
|
||||
|
||||
// \begin{figure}[th]
|
||||
// \centering
|
||||
|
|
@ -159,14 +166,14 @@ Thus, we hypothesize that prefetching along those paths would lead to better ima
|
|||
|
||||
The policy used is the following.
|
||||
We divide each chunk sent by the server into two parts.
|
||||
The first part is used to fetch the content from the current viewpoint, using the \textsf{culling} streaming policy.
|
||||
The first part is used to fetch the content from the current viewpoint, using the culling streaming policy.
|
||||
The second part is used to prefetch content from the bookmarks, according to their likelihood of being clicked next.
|
||||
We use the probabilities displayed in Figure~\ref{bi:mat1} to determine the size of each part.
|
||||
Each bookmark $B$ has a probability $p(B|B_"prev")$ of being clicked next, considering that $B_"prev"$ was the last clicked bookmark.
|
||||
We assign to each bookmark a certain portion of the chunk to prefetch the corresponding data proportionally to the probability of it being clicked.
|
||||
We use the \textsf{visible} policy to determine which data should be sent for a bookmark.
|
||||
We use the visible policy to determine which data should be sent for a bookmark.
|
||||
|
||||
We denote this combination as \textsf{V-PP}, for Prefetching based on Prediction using \textsf{visible} policy.
|
||||
We denote this combination as V-PP, for Prefetching based on Prediction using visible policy.
|
||||
|
||||
// \begin{figure}[th]
|
||||
// \centering
|
||||
|
|
@ -190,18 +197,18 @@ Indeed, as specified in Section~\ref{bi:3dnavigation}, moving to a bookmarked vi
|
|||
This transition usually takes from 1 to 2 seconds, depending on how far the current user camera position is from the bookmark.
|
||||
|
||||
When the user clicks on the bookmark, the client fetches the visible vertices from the destination viewpoint, with all the available bandwidth.
|
||||
So, during the transition time, the server no longer does \textsf{culling}, but the whole chunk is used for fetching following \textsf{visible} policy.
|
||||
So, during the transition time, the server no longer does culling, but the whole chunk is used for fetching following visible policy.
|
||||
|
||||
The immediate drawback of this policy is that on the way to the bookmark, the user perception of the scene will be degraded because of the lack of data for the viewpoints in transition.
|
||||
On the bright side, no time is lost to prefetch bookmarks that will never be consumed, because we fetch only when we are sure that the user has clicked on a bookmark.
|
||||
This way, when the user is not clicking on bookmarks, we can use the entire bandwidth for the current viewpoint and get as many triangles as possible to improve the current viewpoint.
|
||||
We call this method \textsf{V-FD}, since we are Fetching the 3D data from the Destination using \textsf{visible} policy.
|
||||
We call this method V-FD, since we are Fetching the 3D data from the Destination using visible policy.
|
||||
|
||||
// \begin{table}
|
||||
// \centering
|
||||
// \begin{tabular}{ccccc}
|
||||
// \toprule
|
||||
// & \textsf{Visible} & \textsf{V-FD} & \textsf{V-PP} & \textsf{V-PP+FD} \\ \midrule
|
||||
// & \textsf{Visible} & V-FD & V-PP & V-PP+FD \\ \midrule
|
||||
// \textbf{Frustum culling} &\cmark&\cmark&\cmark&\cmark\\
|
||||
// \textbf{Fetch destination} &\xmark&\cmark&\xmark&\cmark\\
|
||||
// \textbf{Prefetch predicted} &\xmark&\xmark&\cmark&\cmark\\\bottomrule
|
||||
|
|
@ -213,7 +220,7 @@ We call this method \textsf{V-FD}, since we are Fetching the 3D data from the De
|
|||
|
||||
In order to determine which policy to use, we replay the traces from the user study while simulating different streaming policies.
|
||||
The first point we are interested in is which streaming policy leads to the lower discovery latency and better image
|
||||
quality for the user: \textsf{culling} (no prefetching), \textsf{V-PP} (prefetching based on probability of accessing bookmarks), or \textsf{V-FD} (no prefetching, but fetch the destination during "fly-to" transition) or combining both \textsf{V-PP} and \textsf{V-FD} (\textsf{V-PP+FD}).
|
||||
quality for the user: culling (no prefetching), V-PP (prefetching based on probability of accessing bookmarks), or V-FD (no prefetching, but fetch the destination during "fly-to" transition) or combining both V-PP and V-FD (V-PP+FD).
|
||||
|
||||
// \begin{figure}[th]
|
||||
// \centering
|
||||
|
|
@ -247,14 +254,14 @@ quality for the user: \textsf{culling} (no prefetching), \textsf{V-PP} (prefetch
|
|||
Figure~\ref{bi:click-1250} compares the quality of the view of a user after their first click on a bookmark.
|
||||
The ratio of pixels correctly displayed is computed in the client algorithm, see also algorithm~\ref{bi:streaming-algorithm-client}.
|
||||
In this figure we use a bandwidth of 1 Mbps.
|
||||
The blue curve corresponds to the \textsf{culling} policy.
|
||||
The blue curve corresponds to the culling policy.
|
||||
Clicking on a bookmark generates a user path with less spatial locality, causing a large drop in visual quality that is only compensated after 4 seconds.
|
||||
During the first second, the camera moves from the current viewport to the bookmarked viewport.
|
||||
|
||||
When the data has been prefetched according to the probability of the bookmark to be clicked, the drop in quality is less visible (\textsf{V-PP} curve).
|
||||
However, by benefiting from the precomputation of visible triangles and ordering of the important triangles in a bookmark (\textsf{V-FD}) the drop in quality is still there, but is very short (approximately four times shorter than for \textsf{culling}).
|
||||
When the data has been prefetched according to the probability of the bookmark to be clicked, the drop in quality is less visible (V-PP curve).
|
||||
However, by benefiting from the precomputation of visible triangles and ordering of the important triangles in a bookmark (V-FD) the drop in quality is still there, but is very short (approximately four times shorter than for culling).
|
||||
This drop in quality is happening during the transition on the path.
|
||||
More quantitatively, with a $1$ Mbps bandwidth, 3 seconds are necessary after the click to recover $90\%$ of correct pixels.
|
||||
More quantitatively, with a $1$ Mbps bandwidth, 3 seconds are necessary after the click to recover $90%$ of correct pixels.
|
||||
|
||||
// \begin{figure}[th]
|
||||
// \centering
|
||||
|
|
@ -304,21 +311,21 @@ More quantitatively, with a $1$ Mbps bandwidth, 3 seconds are necessary after th
|
|||
// \addplot table [y=y1, x=x]{assets/preliminary-work/evaluation/click-curves-local-2500.dat};
|
||||
// \addlegendentry{V-FD}
|
||||
// \addplot table [y=y2, x=x]{assets/preliminary-work/evaluation/click-curves-local-2500.dat};
|
||||
// \addlegendentry{V-PP-FD}
|
||||
// \addlegendentry{V-PP+FD}
|
||||
// \end{axis}
|
||||
// \end{tikzpicture}
|
||||
// \caption{Same curve as Figures~\ref{bi:click-1250} and~\ref{bi:click-625}, for comparing streaming policies \textsf{V-FD} alone and \textsf{V-PP+FD}. BW=2Mbps\label{bi:2MB}}
|
||||
// \caption{Same curve as Figures~\ref{bi:click-1250} and~\ref{bi:click-625}, for comparing streaming policies V-FD alone and V-PP+FD. BW=2Mbps\label{bi:2MB}}
|
||||
// \end{figure}
|
||||
|
||||
|
||||
Figure~\ref{bi:click-625} showed the results of the same experiment with 0.5 Mbps bandwidth. Here, it takes 4 to 5 seconds to recover $85%$ of the pixels with \textsf{culling} and \textsf{V-PP}, against 1.5 second for recovering $90%$ with \textsf{V-FD}.
|
||||
Combining both strategies (\textsf{V-PP+FD}) leads to the best quality.
|
||||
Figure~\ref{bi:click-625} showed the results of the same experiment with 0.5 Mbps bandwidth. Here, it takes 4 to 5 seconds to recover $85%$ of the pixels with culling and V-PP, against 1.5 second for recovering $90%$ with V-FD.
|
||||
Combining both strategies (V-PP+FD) leads to the best quality.
|
||||
|
||||
At 1 Mbps bandwidth, \textsf{V-PP} penalizes the quality, as the curve \textsf{V-PP-FD} leads to a lower quality image than \textsf{V-FD} alone.
|
||||
At 1 Mbps bandwidth, V-PP penalizes the quality, as the curve V-PP+FD leads to a lower quality image than V-FD alone.
|
||||
This effect is even stronger when the bandwidth is set to 2 Mbps (Figure~\ref{bi:2MB}).
|
||||
Both streaming strategies based on the precomputation of the ordering improves the image quality.
|
||||
We see here, that \textsf{V-FD} has a greater impact than \textsf{V-PP}.
|
||||
Here, \textsf{V-PP} may prefetch content that eventually may not be used, whereas \textsf{V-FD} only sends relevant 3D content (knowing which bookmark has been just clicked).
|
||||
We see here, that V-FD has a greater impact than V-PP.
|
||||
Here, V-PP may prefetch content that eventually may not be used, whereas V-FD only sends relevant 3D content (knowing which bookmark has been just clicked).
|
||||
|
||||
We present only the results after the first click.
|
||||
For subsequent clicks, we found that other factors came into play and thus, it is hard to analyze the impact of the various streaming policies.
|
||||
|
|
@ -326,7 +333,7 @@ For instance, a user may revisit a previously visited bookmark, or the bookmarks
|
|||
If the users click on a subsequent bookmark after a long period, then more content would have been fetched for this user, making comparisons difficult.
|
||||
|
||||
To summarize, we found that exploiting the fact that bookmarked viewpoints are frequently visited to precompute the visible faces and sort them according to projected areas can lead to significant improvement in image quality after a user interaction (clicking on a bookmark).
|
||||
This alone can lead to 60\% less triangles being sent, with 1/3 of the triangles sufficient to ensure 90% of pixels correctly rendered, compared to doing frustum/backface culling.
|
||||
This alone can lead to 60% less triangles being sent, with 1/3 of the triangles sufficient to ensure 90% of pixels correctly rendered, compared to doing frustum/backface culling.
|
||||
If we fetch these precomputed faces of the destination viewpoint this way immediately after the click, during the "fly-to" transition, then we can already significantly improve the quality without any prefetching.
|
||||
Prefetching helps if the bandwidth is low, and fewer triangles can be downloaded during this transition.
|
||||
The network conditions play a minimum role in this key message --- bookmarking allows precomputation of an ordered list of visible faces, and this holds regardless of the underlying network condition (except for non-interesting extreme cases, such as negligible bandwidth or abundance of bandwidth).
|
||||
|
|
|
|||
|
|
@ -83,8 +83,6 @@ This process reduces the time a user has to wait before seeing a downloaded 3D o
|
|||
caption: [Four levels of resolution of a mesh]
|
||||
)
|
||||
|
||||
%These methods have been vastly researched #cite("bayazit20093", "mamou2010shape"), but very few of these methods can handle meshes with attributes, such as texture coordinates.
|
||||
|
||||
#cite("streaming-compressed-webgl") develop a dedicated progressive compression algorithm based on iterative decimation, for efficient decoding, in order to be usable on web clients.
|
||||
With the same objective, #cite("pop-buffer") proposes pop buffer, a progressive compression method based on quantization that allows efficient decoding.
|
||||
|
||||
|
|
@ -151,7 +149,7 @@ By benefiting from the video compression techniques, the authors are able to rea
|
|||
|
||||
=== Geometry and textures
|
||||
|
||||
As discussed in Chapter~\ref{f:3d}, most 3D scenes consist in two main types of data: geometry and textures.
|
||||
As discussed in @f:3d, most 3D scenes consist in two main types of data: geometry and textures.
|
||||
When addressing 3D streaming, one must handle the concurrency between geometry and textures, and the system needs to address this compromise.
|
||||
|
||||
Balancing between streaming of geometry and texture data is addressed by #cite("batex3"), #cite("visual-quality-assessment"), and #cite("mesh-texture-multiplexing").
|
||||
|
|
@ -161,8 +159,6 @@ This work designs a cost driven framework for 3D data compression, both in terms
|
|||
The authors generate an atlas for textures that enables efficient compression and multi-resolution scheme.
|
||||
All four works considered a single mesh, and have constraints on the types of meshes that they are able to compress.
|
||||
Since the 3D scenes we are interested in in our work consist in soups of textured polygons, those constraints are not satisfied and we cannot use those techniques.
|
||||
% All four works considered a single, manifold textured mesh model with progressive meshes, and are not applicable in our work since we deal with large and potentially non-manifold scenes.
|
||||
|
||||
|
||||
=== Streaming in game engines
|
||||
|
||||
|
|
|
|||
25
template.typ
25
template.typ
|
|
@ -1,7 +1,7 @@
|
|||
#let phd(doc) = {
|
||||
set page(paper: "a4")
|
||||
set par(first-line-indent: 1em, justify: true, leading: 1em)
|
||||
set page(paper: "a4", numbering: "1")
|
||||
|
||||
set par(first-line-indent: 1em, justify: true, leading: 1em)
|
||||
|
||||
// Code formatting
|
||||
show raw.where(block: true): it => {
|
||||
|
|
@ -23,8 +23,9 @@
|
|||
}
|
||||
|
||||
show heading: content => {
|
||||
v(0.5em)
|
||||
content
|
||||
v(1em)
|
||||
v(0.38em)
|
||||
}
|
||||
|
||||
show figure: content => {
|
||||
|
|
@ -55,17 +56,26 @@
|
|||
}
|
||||
)
|
||||
|
||||
set heading(numbering: "1.1")
|
||||
set heading(numbering: (..nums) => {
|
||||
if nums.pos().len() >= 4 {
|
||||
none
|
||||
} else {
|
||||
nums.pos().map(str).join(".") + " "
|
||||
}
|
||||
})
|
||||
|
||||
show heading.where(level: 1): it => {
|
||||
set text(size: 11pt, weight: "regular")
|
||||
align(right, {
|
||||
pagebreak();
|
||||
v(100pt)
|
||||
if it.numbering != none {
|
||||
text(size: 50pt)[Chapter ]
|
||||
text(size: 50pt, weight: "bold")[Chapter ]
|
||||
set heading(numbering: (..nums) => str(nums.pos().at(0)))
|
||||
text(counter(heading).display(), size: 150pt, fill: rgb(173, 216, 230))
|
||||
}
|
||||
v(50pt)
|
||||
text(it.body, size: 40pt)
|
||||
text(it.body, size: 40pt, weight: "bold")
|
||||
if it.numbering != none {
|
||||
pagebreak()
|
||||
} else {
|
||||
|
|
@ -74,5 +84,8 @@
|
|||
})
|
||||
}
|
||||
|
||||
show heading.where(level: 2): set text(size: 15pt)
|
||||
show heading.where(level: 3): set text(size: 13pt)
|
||||
|
||||
doc
|
||||
}
|
||||
|
|
|
|||
Loading…
Reference in New Issue