dash-3d-tmp
/
phd
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Stuff

This commit is contained in:
Thomas Forgione 2019-10-20 14:38:18 +02:00
parent 010dc70497
commit 8a8e03f9d0
No known key found for this signature in database
GPG Key ID: BFD17A2D71B3B5E7
2 changed files with 45 additions and 33 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

46
src/conclusion/contributions.tex
View File

@ -3,30 +3,32 @@
In this thesis, we have presented three main contributions.
\paragraph{}
First, we set up a basic system allowing 3D navigation and 3D content streaming.
We developped a navigation aid in the form of 3D bookmarks, and we conducted a user study to analyse its impact on navigation and streaming.
On one hand, we concluded that navigation aid helps people navigating in a scene, they can perform tasks faster and more easily.
On the other hand, we showed that this help in 3D navigation comes at the cost of a negative impact for the quality of service: since users navigate faster, they require more data to perform in the same way.
However, we also showed that this cost is not a fatality.
Due to the prior knowledge we have about bookmarks, we are able to precompute data offline that we are then able to use when users click on bookmarks to improve the quality of service.
We then ran simulations on the traces we collected during the user study to show how these precomputations increase the quality of service.
First, we set up a basic system allowing 3D navigation and 3D content streaming in the context of streaming online 3D content consisting in textured meshes.
We developed a navigation aid in the form of \textbf{3D bookmarks}, and we conducted a user study to analyse its impact on navigation and streaming.
On one hand, consistently with the state of the art, we observed that navigation aid \textbf{helps people navigating in a scene}, since they perform tasks faster and more easily.
On the other hand, we showed that benefiting from bookmarks in 3D navigation comes at the cost of a negative impact for the quality of service (QoS): since users navigate faster, they require more data during the same time span.
However, we also showed that this cost is not a fatality: using prior knowledge we have about bookmarks, we are able to \textbf{precompute data ordering offline} so that when users click on bookmark, their QoS improves.
Simulations on the traces we collected during the user study quantify how these precomputations \textbf{improve the QoS}.
This work has been published at the conference MMSys in 2016~\citep{bookmarks-impact}.
\paragraph{}
Then, we put the focus on the streaming aspect of the system.
The objective of this contribution was to introduce a system able of performing view-dependent 3D streaming, without having the drawbacks of the basic system described in our first contribution (no support for texture, heavy computations on the server, no support for multi-resolution, no rendering performances considerations, \ldots).
We took massive inspiration from the DASH technology, that has become a standard for video streaming thanks to its solutions for all the issues described before.
We exploited the fact that DASH is made to be content agnostic to fit 3D content into its structure.
We used DASH-SRD extension to cut our 3D content into a $k$-d tree and profit from this structure to perform view-dependant streaming, without having any computation to run on the server side at all.
We implemented a few loading policies based on a utility metric that gives a score for each portion of the model.
We compared different values for a set of parameters, as well as our different loading policies by running simulations.
This work has been published at the conference ACMMM in 2018~\citep{dash-3d}. A demo paper was also published~\citep{dash-3d-demo}.
Then, we focus on the streaming aspect of the system.
The objective of this contribution is to introduce a system able to perform \textbf{scalable, view-dependent 3D streaming}.
This new framework proposes many improvements upon the basic system described in our first contribution: support for texture, moving computation from the server to the clients, support for multi-resolution textures, rendering performances considerations.
We took massive inspiration from the DASH technology, a standard for video streaming thanks to its scalability and its adaptability.
We exploit the fact that DASH is made to be content agnostic to fit 3D content into its structure.
We used DASH-SRD extension to partition our 3D content tree and profit from this partition to perform view-dependant streaming, without having any computation to run on the server side.
For the clients, we implement loading policies based on a utility metric that gives a score for both geometry and texture portion of the model.
We throughly tested our solutions for different values of parameters, as well as for our different loading policies by running simulations, to propose an efficient framework that we name DASH-3D.
This work has been published as a full paper at the conference ACMMM in 2018~\citep{dash-3d}.
A demo paper on the DASH-3D implementation was also published~\citep{dash-3d-demo}.
\paragraph{}
Finally, we brought back the 3D navigation aspect in DASH-3D.
We developped interfaces that allow navigating in 3D scenes for both desktop and mobile devices we reintroduced bookmarks in these interfaces.
The setup of our first contribution had some simplifications that made precomputations very easy to implement and efficient, particularly due to the fact that the server is able to know exactly what the client needs, and can thus create chunks adapted to the client's requirements.
In DASH-3D, the data is structured and chunks are precomputed and do not depend on the client's need.
However, this does not mean that all hope is lost: we showed that we are still able to precompute an optimal order for chunks from each bookmark, and keep using the policies from the previous contribution, switching to this optimal order when a user clicks a bookmark.
We then ran simulations to show how the quality of service is impacted by those techniques.
A demo paper was published at the conference ACMMM in 2019~\citep{dash-3d-bookmarks-demo} showing the interfaces for desktop and mobile clients with bookmarks, but without any streaming aspect.
Finally, we brought back the \textbf{3D navigation bookmark within DASH-3D}.
We developed interfaces that allow navigating in 3D scenes for both \textbf{desktop and mobile devices} and we reintroduced bookmarks in these interfaces.
The setup of our first contribution considered only geometry, triangle by triangle, which made precomputations and ordering straightforward.
Moreover, the server knows exactly the client needs, and thus create chunks adapted to the client's requirements.
In DASH-3D, the data is structured a priori (offline), so that chunks are grouped independently of a client's need.
However, this does not mean that all hope is lost: we are still able to precompute an optimal order for chunks from each bookmark, and, using the policies from our first contribution, switch to this optimal order when a user clicks a bookmark.
Simulations show that the QoS is positively impacted by those policies.
A demo paper was published at the conference ACMMM in 2019~\citep{dash-3d-bookmarks-demo} showing the interfaces for desktop and mobile clients with bookmarks, but without the streaming aspect.

32
src/conclusion/future-work.tex
View File

@ -1,23 +1,33 @@
\fresh{}
\section{Future work}
After all this, we have three major perspectives for future work.
Successfully using the DASH framework for 3D content is a significant step that naturally opens many exciting perspectives.
In this section, we shall detail three major perspectives for future work.
\subsection{Semantic information}
In all of this work, no attention has been paid towards semantic.
Our content preparation does not take anything like that into account, and our adaptation sets and segments surely will cut data that could be grouped semantically.
Having semantic support can help us having a better structure for our content: we know that displaying half a building will lead to low quality of experience.
Moreover, semantic data could also change the utilities we have been defining for our segments: some data can be marked as more important than other and this can be taken into account in our utilities.
In this thesis, no attention has been given to semantic.
Our content preparation considers only spatial information both for 3D content and clients so our adaptation sets and segments may separate data that could be grouped semantically.
Having semantic information could help us derive a better structure for our content: we know that displaying half a building will lead to low quality of experience.
In order to account for semantic besides partitioning, we could also adapt the utilities we have been defining for our segments: some semantically significant data can be considered as more important than other by taking it into account in our utilities.
\subsection{Compression / multi resolution for geometry}
In all of this work, no attention has been paid towards geometry compression or multi-resolution.
Geometry data is transmitted as OBJ files which mostly consist in ASCII encoded numbers, which is terrible for transmission, and compression could drastically increase the quality of experience.
Being able to support multi resolution geometry would be even better, and even if performing multi resolution on a large and heterogeneous scene just like ours is difficult, we have no doubt that semantic information can help this task.
In this thesis, we considered different resolutions for textures, but for geometry, neither compression nor multi-resolution.
Geometry data is transmitted as OBJ files which mostly consist in ASCII encoded numbers, which is terrible for transmission, and compression could reduce size, thus increase the quality of experience.
Being able to support multi resolution geometry would be even better, and even if performing multi resolution on a large and heterogeneous scene just like ours is difficult.
Moreover, only few work have considered multi-resolution for textured geometry~\citep{maglo20153d}, and only for objects.
There, using semantic information would also benefit compression.
% we have no doubt that semantic information can help this task.
\subsection{Performance optimization}
Performance has already been discussed in Chapter~\ref{d3}.
However, in this work, we never tried to remove data from the media engine when it is no longer useful.
This means that on a really large scene, performance is bound to become damaged due to the growing amount of data to render, and the saturation of memory.
For our system to be able to support such scenes, it needs to unload data.
The utility measures that we described in Section~\ref{d3:utility} seem to be good candidates to determine what to unload.
This means that on a really large scene, performance is bound to become damaged due to the growing amount of data to render, and the saturation of GPU memory.
For our system, in order for a client (even more for a mobile client), to be able to support such scenes, it needs to unload data.
The utility measures that we described in Section~\ref{d3:utility} is a good candidate to determine what to unload.
We could estimate the performance of our system by measuring variables such as memory used or framerate and use those values to discard data with low enough utility.
Finally, this thesis has proposed a first, but ambitious, complete development of a DASH-3D framework, providing a scalable, efficient, streaming for textured meshes.
This framework can be directly used to adapt to semantic or compressed geometry.
As the client, especially for mobile clients, special attention is needed for handling locally the large amount of received data for a controlled framerate.