dash-3d-tmp/phd
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Fixes

Browse Source
This commit is contained in:
Thomas Forgione
2019-10-16 17:01:56 +02:00
parent 8b9303b1a5
commit 8e686dc040
8 changed files with 40 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
src/introduction/challenges.tex
View File

@@ -7,39 +7,39 @@ A 3D streaming client has lots of tasks to accomplish:
\begin{itemize}
\item render a scene;
\item find out what part of the model to download next;
\item decide what part of the model to download next;
\item download the next part;
\item parse the downloaded content;
\item add the parsed result to the scene;
\item manage the interaction with the user.
\end{itemize}
This opens multiple problems that we need to take care of.
This opens multiple problems that need to be considered and will be studied in this thesis.
\paragraph{Content preparation.}
% Any preprocessing that can be done on our 3D data gives us a strategical advantage since it consists in computations that will not be needed live, neither for the server nor for the client.
% Furthermore, for streaming, data needs to be split into chunks that are requested separately, so perparing those chunks in advance can also help the streaming.
Before streaming content, it needs to be prepared.
The segmentation of the content into chunks is particularly important for streaming since it allows transmitting only a portion of the data to the client.
A partial model consisting in the downloaded content, it can be rendered before downloading more chunks.
A partial model consisting in the downloaded content, it can be rendered while downloading more chunks.
Content preparation also includes compression.
One of the questions this thesis has to answer is: \emph{what is the best way to prepare 3D content so that a client can progressively download and render the 3D model?}
One of the questions this thesis has to answer is: \emph{what is the best way to prepare 3D content so that a streaming client can progressively download and render the 3D model?}
\paragraph{Streaming policies.}
Once our content is prepared and split in chunks, a client needs to determine which chunks should be downloaded first.
A chunk that contains data in the field of view of the user is more relevant than a chunk that is not inside; a chunk that is close to the camera is more relevant than a chunk far away from the camera, etc.
This should also include other contextual parameters, such as the size of a chunk, the bandwidth, the user's behaviour, etc.
The most important questions we have to answer are: \emph{how to estimate a chunk utility, and how to determine which chunks need to be downloaded depending on the chunks themselves and the user's interactions?}
The most important questions we have to answer are: \emph{how to estimate a chunk utility, and how to determine which chunks need to be downloaded depending the user's interactions?}
\paragraph{Evaluation.}
In such systems, the two most important criteria for evaluation are quality of service, and quality of experience.
The quality of service is a network-centric metric, which considers values such as throughput.
The quality of experience is a user-centric metric, and can only be measured by asking how users feel about a system.
The quality of service is a network-centric metric, which considers values such as throughput and measures how well the content is served to the client.
The quality of experience is a user-centric metric: it relies on user perception and can only be measured by asking how users feel about a system.
To be able to know which streaming policies are best, one needs to know \emph{how to compare streaming policies and evaluate the impact of their parameters in terms of quality of service and quality of experience?}
\paragraph{Implementation.}
The objective of our work is to setup a client-server architecture that answers the above problems: content preparation, chunk utility, streaming policies.
In this regard, we have to find out \emph{how do we build this architecture that keeps the computational load for the server low so it scales up and for the client so that it has enough resources to perform the tasks described above?}
In this regard, we have to find out \emph{how do we build this architecture that keeps a low computational load on the server so it scales up and on the client so that it has enough resources to perform the tasks described above?}
% This implementation must respect constraints required for performant software:
%

10
src/introduction/main.tex
View File

@@ -2,17 +2,17 @@
\fresh{}
During the last years, 3D acquisition and modeling techniques have progressed a lot.
Recent software such as \href{https://alicevision.org/\#meshroom}{Meshroom} use \emph{structure from motion} and \emph{multi view stereo} to infer a 3D model from a set of photographs.
There are more and more devices that are specifically built to obtain 3D data: some are more expensive and provide with very precise information such as LIDAR (Light Detection And Ranging, as in RADAR but with light instead of radio waves), and some cheaper devices can obtain coarse data such as the Kinect.
During the last years, 3D acquisition and modeling techniques have made tremendous progress.
Recent software such as \href{https://alicevision.org/\#meshroom}{Meshroom} use \emph{structure-from-motion} and \emph{multi-view-stereo} to infer a 3D model from a set of photographs.
There are more and more devices that are specifically built to harvest 3D data: some still very expensive and provide precise information such as LIDAR (Light Detection And Ranging, as in RADAR but with light instead of radio waves), while some cheaper devices can obtain coarse data such as the Kinect.
Thanks to these techniques, more and more 3D data become available.
These models have potential for multiple purposes, for example, they can be 3D printed, which can reduce the production cost of some pieces of hardware or enable the creation of new objects, but most uses will consist in visualisation.
These models have potential for multiple purposes, for example, they can be printed, which can reduce the production cost of some pieces of hardware or enable the creation of new objects, but most uses are based on visualisation.
For example, they can be used for augmented reality, to provide user with feedback that can be useful to help worker with complex tasks, but also for fashion (for example, \emph{Fitting Box} is a company that develops software to virtually try glasses).
3D acquisition and visualisation is also useful to preserve cultural heritage, and software such as Google Heritage or 3DHop are such examples, or to allow users navigating in a city (as in Google Earth or Google Maps in 3D).
\href{https://sketchfab.com}{Sketchfab} is an example of a website allowing users to share their 3D models and visualise models from other users.
In most 3D visualisation systems, the 3D data is stored on a server and needs to be transmitted to a terminal before the user can visualise it.
The improvements in the acquisition setups we described lead to an increasing quality of the 3D models, thus an increasing size in bytes as well.
Simply downloading 3D content and waiting until the content is fully downloaded to let the user visualise it is no longer a satisfactory solution, so streaming needs to be performed.
Simply downloading 3D content and waiting until the content is fully downloaded to let the user visualise it is no longer a satisfactory solution, so adaptive streaming is needed.
In this thesis, we propose a full framework for the navigation and the streaming of large 3D scenes, such as districts or whole cities.
% With the progress in data acquisition and modeling techniques, networked virtual environments, or NVE, are increasing in scale.

4
src/introduction/outline.tex
View File

@@ -3,7 +3,7 @@
First, in Chapter~\ref{f}, we give some preliminary information required to understand the types of objects we are manipulating in this thesis.
We then proceed to compare 3D and video content: surprisingly, video and 3D share many features, and analysing video setting gives inspiration for building a 3D streaming system.
In Chapter~\ref{sote}, we present a review of the state of the art in the multimedia interaction and streaming.
In Chapter~\ref{sote}, we present a review of the state of the art in multimedia interaction and streaming.
This chapter starts with an analysis of the video streaming standards.
Then it reviews the different 3D streaming approaches.
The last section of this chapter focuses on 3D interaction.
@@ -24,4 +24,4 @@ We finally evaluate the different parameters of our client.
In Chapter~\ref{sb}, we present our last contribution: the integration of the interaction ideas that we developed in Chapter~\ref{bi} into DASH-3D.
We first develop an interface that allows desktop as well as mobile devices to navigate in a 3D scene being streamed, and that introduces a new style of bookmarks.
We then explain why simply applying the ideas developed in Chapter~\ref{bi} is not sufficient and we propose more efficient pre-computations that can enhance the streaming.
Finally, we present a user study that provides us with traces on which we can perform simulations, and we evaluate the impact of our extension of DASH-3D on the quality of service and on the quality of experience.\todo{maybe only qos here}
Finally, we present a user study that provides us with traces on which we evaluate the impact of our extension of DASH-3D on the quality of service and on the quality of experience.