This site is closed and please have a look at https://athena.itec.aau.at/ for more recent updates… thank you!

We are happy to announce that the successor of the WESP, namely WESP+, has been released as open source under the LGPLv3 license. Check out our Software&Services section or just follow this link: https://github.com/maxkr/qoe_evaluation_platform.

We are happy to announce that we are going to attend this year’s Workshop on Quality of Multimedia Experience (QoMEX), Greece, (May 26-29, 2015.) with the following contribution:

Full Paper
Title: Is One Second Enough?
Evaluating QoE for Inter-Destination Multimedia Synchronization using Human Computation and Crowdsourcing
Authors: B. Rainer, S. Petscharnig, and C. Timmerer
Abstract:
Modern-age technology enables us to consume multimedia for enjoyment and as a social experience. The traditional way to consume multimedia together (e.g., with family or friends in the living room) is being superseded by a location-independent scenario where geographically distributed users consume the same content while having a real-time communication channel among each other. Inter-Destination Multimedia Synchronization (IDMS) is the tool of choice in order to enable users a high-quality multimedia experience. In this paper, we investigate the influence of asynchronism when consuming multimedia content together while being geographically distributed. In particular, we adopt the concept of human computation and developed a reaction game which we used to conduct a crowdsourced subjective quality assessment in order to evaluate a threshold for multimedia synchronization within an IDMS scenario. Our results show a significant decrease in overall Quality of Experience (QOE) at an asynchronism level of 750ms. At the same time, we were able to show that asynchronism at a level of 400ms does not have significant differences regarding the QoE when compared to the synchronous reference case.

We are happy to announce that we are going to attend this year’s Workshop on Quality of Multimedia Experience (QoMEX), Singapore, (Sep 18-20, 2014.) with the following contributions:

Full Paper
Title: A Quality of Experience Model for Adaptive Media Playout
Authors: B. Rainer, and C. Timmerer
Abstract:
In the past decade Adaptive Media Playout (AMP) has been intensively studied with respect to the detection of when to increase or decrease the playback rate in order to maintain a certain buffer fill state. In this paper we subjectively assess the QoE of AMP with respect to non-periodically and randomly selected content sections of a video sequence by using crowdsourcing. Furthermore, we introduce metrics that allow to quantify the distortion for audio and video that are caused by increasing or decreasing the playback rate. With these preliminaries we study the correlation between the introduced metrics and the subjectively assessed QoE. Therefore, we derive a utility model that allows estimating the QoE with the introduced metrics. We instantiate and validate the model using the data gathered from the conducted study.

Tutorial
Title: Best Practices in Quality Assessment
Authors: Touradj Ebrahimi, EPFL; Andrew Perkis, NTNU; Sebastian Möller, TU Berlin; Christian Timmerer, AAU

Journal Paper
Title: Mulsemedia: State-of-the-Art, Perspectives and Challenges
Authors: G. Ghinea, C. Timmerer, W. Lin, S. Gulliver

We are happy to announce that we are going to attend this year’s IEEE/IFIP International Workshop on Quality of Experience Centric Management (QCMan), Krakow, Poland (May 9, 2014.).

The title and the abstract of the QCMAN paper are:

Title:A Subjective Evaluation using Crowdsourcing of Adaptive Media Playout utilizing Audio-Visual Content Features
Authors: B. Rainer, and C. Timmerer,
Abstract:
Inter-Destination Multimedia Synchronization (IDMS) pushes social interactions to a new level. IDMS allows the users to experience multimedia together with friends, colleagues, or the family while having a real-time communication at the same time. The actual challenge of synchronizing the playout of each participant to a reference playout time is a tough task in terms of Quality of Experience (QoE). A possible solution for carrying out the synchronization is Adaptive Media Playout (AMP) where the playout speed of the multimedia is increased or decreased. In this paper we evaluate the impact of the playout variations on the QoE by adopting a crowdsourcing approach. In particular, we investigate the impact of randomly selecting content sections for adapting the playout rate compared to our approach that exploits audio-visual features of the content in order to minimize the impact on the QoE.

Looking forward to seeing you in Krakow.

ACM Multimedia 2014

ACM Multimedia 2014 calls for full papers presenting novel theoretical and algorithmic solutions addressing problems across the domain of multimedia and related applications.The conference also calls for short papers presenting novel, thought-provoking ideasand promising (preliminary) results in realizing these ideas.

Submissions are invited in the following 14 areas:

1. Media Transport and Delivery
2. Multimedia Systems and Middleware
3. Big and Broad Multimedia
4. Deep Learning for Multimedia
5. Emotional and Social Signals in Multimedia
6. Multimedia and Society
7. Multimedia and the Crowd
8. Multimedia Art, Entertainment and Culture
9. Multimedia HCI and QoE
10. Multimedia Search and Recommendation
11. Multimedia Security, Privacy and Forensics
12. Multimodal Analysis and Description
13. Music, Speech and Audio Processing in Multimedia
14. Social Media and Collective Online Presence

Papers submitted to this track must conform with a “Double-blind Review” process, such that authors should not know the names of the reviewers of their papers, and reviewers should not know the names of the authors. Please prepare your paper in a way that preserves anonymity of the authors. Do not put your names under the title.

• Avoid using phrases such as “our previous work” when referring to earlier publications by the authors.
• Remove information that may identify the authors in the acknowledgments (e.g., co-workers and grant IDs).
• Check supplemental material (e.g., titles in the video clips, or supplementary documents) for information that may identify the authors identity.
• Avoid providing links to websites that identify the authors.

Please check this site regularly for various updates.

Important dates:

Manuscript for Full Papers: March 31, 2014
Manuscript for Short papers: April 14, 2014
Initial reviews to authors (only for Full Papers): May 19, 2014
Rebuttal period (only for Full Papers): May 19 – May 26, 2014
Author-to-Author’s Advocate contact period: May 19 – May 23, 2014
Notification of Acceptance: July 7, 2014
Camera-ready submission: August 11, 2014

Sixth International Workshop on Quality of Multimedia Experience
QoMEX 2014
18-20 September 2014
Singapore
http://www.qomex2014.org/
——————————————————————-

The sixth International Workshop on Quality of Multimedia Experience (QoMEX) will bring together leading experts from academia and industry interested in evaluating multimedia quality and user experience. Coming to Asia for the first time, QoMEX 2014 will be held in Singapore, conveniently located adjacent to InterSpeech and MMSP. Prospective organizers of special sessions are invited to submit proposals as per the guidelines on the web site. Prospective authors are invited to prepare full (six pages) and/or short (two pages) papers. Furthermore, authors of recent journal papers will have the opportunity to present their work as a poster. Please refer to the workshop web site for more details. Proceedings will be published in the IEEE Xplore digital library (pending).

Topics of interest include:

• QoE Fundamentals: Understanding experience and quality formation; alternatives to MOS; quality vs. user satisfaction vs. acceptance; crowdsourcing studies; long-term quality measurement; physiological QoE assessment; emotions & QoE.
• Audio/Visual User Experience: Objective/subjective quality evaluation of speech, audio, video, multimedia; spatial/3D audio and stereo/multi-view video quality; psycho-physical modeling; quality-centered processing, compression and transmission; perceptual optimization.
• QoE in Graphics & Rendering: High-dynamic range imaging; (non-)photorealistic rendering; object/texture/motion modeling; special effects; virtual/augmented/mixed realities, games.
• Sensory User Experiences: Methods for sensory user feedback; quality metrics for ambient and sensory experiences; novel sensory interfaces; multisensory interaction.
• Interactive Systems QoE: Tele-conferencing/tele-presence; multimedia-based group inter-action; gaming; multimedia Web search/browsing; e-commerce; social media applications.
• QoE for Mobile Devices: Evaluating multimedia applications and interactive experiences on mobile devices; impact of viewing conditions and device properties; adaptive user interfaces.
• QoE-centric Network and Application Management: Distributed and central management approaches; cloud-based multimedia services; adaptive media streaming; link between QoS and QoE; interoperability; performance optimization; pricing and service-level agreements.
• Reproducible QoE Research: Multimedia databases/datasets; benchmarking and certification; testing conditions and methods; standardization efforts; open-source QoE tools.

Important Dates (tentative)

• Submission deadline: May 4, 2014 (full papers); June 29, 2014 (short paper)
• Notification of acceptance: June 15, 2014 (full papers); July 13, 2014 (short paper)
• Camera-­ready submission: July 13, 2013 (full papers)

General Chair
Stefan Winkler, ADSC/UIUC

General Co-Chair
Susanto Rahardja, NUS

Technical Program Co­‐Chairs
Weisi Lin, NTU
Alexander Raake, TU Berlin 
Zhou Wang, Univ. Waterloo

Special Session Chair
Luigi Atzori, Univ. Cagliari

Steering Committee
Loretta Anania, EC 
Ian Burnett, RMIT 
Touradj Ebrahimi, EPFL/NTNU 
Khaled El-Maleh, Qualcomm 
Lina Karam, ASU 
Patrick Le Callet, Univ. Nantes 
Sebastian Möller, TU Berlin 
Fernando Pereira, IST-IT 
Andrew Perkis, NTNU 
Amy Reibman, AT&T Labs 
Peter Schelkens, iMinds/VUB 
Christian Timmerer, AAU

Publicity Chairs
Guangtao Zhai, SJTU 
Margaret Pinson, NTIA

Finance Chair
Pina Marziliano, NTU

Local Arrangements Chairs
Lai-Tee Cheok, SMU
Ngai-Man Cheung, SUTD

Special thanks to the organizers of the 4th Workshop on Perceptual Quality (PQS). We attended the event with a paper. With this post we provide you with the slides of our presentation given during the workshop.

Recommendations for the Subjective Evaluation of Sensory Experience

We are happy to announce that we are going to attend this year’s International workshop on Perceptual Quality of System (PQS 2013) , Vienna, Austria (2. Sept. – 4. Sept.).

At the PQS 2013, we will present our paper on the recommendation for subjectively assessing sensory experience. The title and the abstract of the PQS paper are:

Title: Recommendations for the Subjective Evaluation of Sensory Experience
Authors: B. Rainer, C. Timmerer, and M. Waltl
Abstract:
Selecting and adopting the appropriate assessment method for conducting subjective quality assessments is a challenging task. The method decides whether the assessment is successful in delivering the correct answers to previously set up hypotheses. Therefore, in this paper we provide recommendations on test methods used in the domain of Sensory Experience. The proposed test methods comprise single stimulus and double stimulus methods. These test methods were used in previous studies and are presented in combination with the results of the subjective quality assessments with which they were used. Furthermore, we briefly outline our test setup, test design, and test content for assessing Sensory Experience which have been validated through conducted assessments.

Looking forward to seeing you in Vienna.

The fifth international workshop on Quality of Multimedia Experience (QoMEX) will bring together leading professionals and scientists interested in evaluating multimedia quality and user experience. QoMEX is the flagship workshop of theEuropean COST action Qualinet, which serves as its technical and financial sponsor. After San Diego (2009), Trondheim (2010), Mechelen (2011), and Yarra Valley (2012), QoMEX 2013 will be held in Klagenfurt am Wörthersee, Austria.

Continue reading →