We study performance characteristics of convolutional neural networks (CNN) for mobile computer vision systems. CNNs have proven to be a powerful and efficient approach implement such However, the system depends largely on utilization hardware accelerators, which are able speed up execution underlying mathematical operations tremendously through massive parallelism. Our contribution is characterization multiple CNN-based models object recognition detection with several different platforms...

The availability of high bandwidth with low-latency communication in 5G mobile networks enables remote rendered real-time virtual reality (VR) applications. Remote rendering VR graphics a cloud removes the need for local personal computer and augments weak processing unit capacity stand-alone headsets. However, to prevent added network latency from ruining user experience, locally navigable viewport that is larger than field view HMD necessary. size required depends on latency: Longer...

High quality immersive Virtual Reality experience currently requires a PC setup with cable connected head mounted display, which is expensive and restricts user mobility. This paper presents CloudVR system for cloud accelerated interactive mobile VR. It designed to provide short rotation interaction latencies through panoramic rendering dynamic object placement. also includes optimizations reduce server-side computational load bandwidth requirements between the server client. Performance...

Live multimedia streaming from mobile devices is rapidly gaining popularity but little known about the QoE they provide. In this paper, we examine Periscope service. We first crawl service in order to understand its usage patterns. Then, study protocols used, typical quality of experience indicators, such as playback smoothness and latency, video quality, energy consumption Android application.

Cloud gaming is a relatively new paradigm in which the game rendered cloud and streamed to an end-user device through thin client. Latency key challenge for gaming. In order optimize end-to-end latency, it first necessary understand how latency builds up from mobile server. this paper we dissect delays occurring measure access various networks network conditions. We also perform Europe-wide measurement study find optimal server locations see number of affects delay. The results are compared...

In cloud gaming the game is rendered on a distant server and resulting video stream sent back to user who controls via thin client. The high resource usage of servers challenge. Expensive hardware including GPUs have be efficiently shared among multiple simultaneous users. use virtualization techniques isolate users share resources dedicated servers. traditional can however inflict notable performance overhead limiting count for single server. Operating-system-level instances known as...

Cloud gaming, where the game is rendered in cloud and streamed to an end-user device through a thin client, rapidly gaining ground. Latency still key challenge gaming: highly interactive games can become unplayable even with response delays below 100 ms. To overcome this issue, we propose deploy gaming services on more distributed infrastructure, instantiate servers close proximity of user when necessary order shorten delay. Our prototype platform also allows flexible configuration controls...

In this paper we measure the step-wise latency in pipeline of three kinds interactive mobile video applications that are rapidly gaining popularity, namely Remote Graphics Rendering (RGR), Mobile Augmented Reality (MAR), and Virtual (MVR). The differ from each other by way which user interacts with application, i.e., I/O controls, however all experience is highly sensitive to end-to-end latency. Long between a control event display update renders application unusable. Hence, understanding...

In cloud gaming the game is rendered on a distant server and resulting video stream sent back to user who controls via thin client. The high resource usage of servers challenge. Expensive hardware including GPUs have be efficiently shared among multiple simultaneous users. use virtualization techniques isolate users share resources dedicated servers. traditional can however inflict notable performance overhead limiting count for single server. Operating-system-level instances known as...

Broadcasting live video directly from mobile devices is rapidly gaining popularity with applications like Periscope and Facebook Live. The quality of experience (QoE) provided by these services comprises many factors, such as transmitted video, playback stalling, end-to-end latency, impact on battery life, they are not yet well understood. In this article, we examine mainly the service through a comprehensive measurement study compare it in some aspects to We shed light usage analysis...

Cloud gaming, where the game is rendered in cloud and streamed to an end-user device through a thin client, rapidly gaining ground. Latency still key challenge gaming: highly interactive games can become unplayable even with response delays below 100 ms. To overcome this issue, we propose deploy gaming services on more distributed infrastructure, instantiate servers close proximity of user when necessary order shorten delay. Our prototype platform also allows flexible configuration controls...

Recent advances from wearables have significantly changed the way how humans communicate with surrounding environment. To some extent, they extended and augmented capability of humans. For example, a Google Glass, people can take pictures simply by winking eyes twice, which releases human hands cumbersome image-taking process. Thus, it enables new application scenarios that were not possible before. In this paper, we investigate utilizing vision-based techniques to provide wearable...

Real time interactive cloud-based mobile applications such as augmented reality and cloud gaming require low stable latency, especially in urban areas. These conditions are difficult to meet with the traditional single carrier LTE network access consolidated server deployment a cloud. Yet, multiple SIM/multiple radio devices, latency can be kept under given threshold through dynamic selection among carriers at edge. To this end, it is necessary understand how changes over during session...

Gaze is an important indicator of visual attention and knowledge gaze location can be used to improve augment Virtual Reality (VR) experiences. This has led the development VR Head Mounted Displays (HMD) with inbuilt trackers. Given latency constraints VR, foreknowledge gaze, i.e., before it reported by tracker, similarly leveraged preemptively apply gaze-based improvements augmentations a experience, especially in distributed architectures. In this paper, we propose light weight neural...

Accurate characterization of Head Mounted Display (HMD) pose in a virtual scene is essential for rendering immersive graphics Extended Reality (XR). Remote employs servers the cloud or at edge network to overcome computational limitations either standalone tethered HMDs. Unfortunately, it increases latency experienced by user; this reason, predicting HMD advance highly beneficial, as long achieves high accuracy. This work provides thorough solutions that forecast remotely-rendered reality...

Remote rendering systems comprise powerful servers that render graphics on behalf of low-end client devices and stream the as compressed video, enabling high end gaming Virtual Reality those devices. One key challenge with them is amount bandwidth required for streaming quality video. Humans have spatially non-uniform visual acuity: We sharp central vision but our ability to discern details rapidly decreases angular distance from point gaze. This phenomenon called foveation can be taken...

Cloud Virtual Reality (VR) gaming offloads the computationally-intensive rendering tasks from resource-limited Head-Mounted Displays (HMDs) to cloud servers, which consume a staggering amount of bandwidth for high-quality experiences. One way cope with such high demands is capitalize on human vision systems by allocating higher bitrate foveal region HMD viewport, known as foveation in literature. Although was employed remote VR gaming, existing open-source projects all adopt static...

In remote rendered virtual reality (VR), the rendering of application is moved to cloud enabling high quality real-time content be consumed on low-powered standalone head mounted displays (HMDs). The frames are encoded a video stream and streamed thin client which relays user's input server decodes incoming video. Latency bandwidth requirements key challenges for graphics. Foveation can used optimize transmitted in line with human visual system (HVS). this paper we evaluate multiple...

Depth maps are needed by various graphics rendering and processing operations. map streaming is often necessary when such operations performed in a distributed system it requires most cases fast performing compression, which why video codecs used. Hardware implementations of standard enable relatively high resolution frame rate combinations, even on resource constrained devices, but unfortunately those do not currently support RGB+depth extensions. However, they can be used for depth...

Humans have sharp central vision but low peripheral visual acuity. Prior work has taken advantage of this phenomenon in two ways: foveated rendering (FR) reduces the computational workload by producing lower quality for regions and video encoding (FVE) bitrate streamed through heavier compression regions. Remote systems require both techniques can be combined to reduce computing bandwidth consumption. We report early results from such a combination with remote VR rendering. The highlight...

We study performance characteristics of convolutional neural networks (CNN) for mobile computer vision systems. CNNs have proven to be a powerful and efficient approach implement such However, the system depends largely on utilization hardware accelerators, which are able speed up execution underlying mathematical operations tremendously through massive parallelism. Our contribution is characterization multiple CNN-based models object recognition detection with several different platforms...

Depth maps are needed by various graphics rendering and processing operations. map streaming is often necessary when such operations performed in a distributed system it requires most cases fast performing compression, which why video codecs used. Hardware implementations of standard enable relatively high resolution framerate combinations, even on resource constrained devices, but unfortunately those do not currently support RGB+depth extensions. However, they can be used for depth...