We have collected a large dataset of subjective image quality “*nesses,” such as sharpness or colorfulness. The dataset comes from seven studies and contains 39,415 quotations from 146 observers who have evaluated 62 scenes either in print images or on display. We analyzed the subjective evaluations and formed a hierarchical image quality attribute lexicon for *nesses, which is visualized as image quality wheel (IQ-Wheel). Similar wheel diagrams for attributes have become industry standards in other sensory experience fields such as flavor and fragrance sciences. The IQ-Wheel contains the frequency information of 68 attributes relating to image quality. Only 20% of the attributes were positive, which agrees with previous findings showing a preference for negative attributes in image quality evaluation. Our results also show that excluding physical attributes of paper gloss, observers then use similar terminology when evaluating images with printed images or images viewed on a display. IQ-Wheel can be used to guide the selection of scenes and distortions when designing subjective experimental setups and creating image databases.
Evaluating algorithms used to assess image and video quality requires performance measures. Traditional performance measures (e.g., Pearson’s linear correlation coefficient, Spearman’s rank-order correlation coefficient, and root mean square error) compare quality predictions of algorithms to subjective mean opinion scores (mean opinion score/differential mean opinion score). We propose a subjective root-mean-square error (SRMSE) performance measure for evaluating the accuracy of algorithms used to assess image and video quality. The SRMSE performance measure takes into account dispersion between observers. The other important property of the SRMSE performance measure is its measurement scale, which is calibrated to units of the number of average observers. The results of the SRMSE performance measure indicate the extent to which the algorithm can replace the subjective experiment (as the number of observers). Furthermore, we have presented the concept of target values, which define the performance level of the ideal algorithm. We have calculated the target values for all sample sets of the CID2013, CVD2014, and LIVE multiply distorted image quality databases.The target values and MATLAB implementation of the SRMSE performance measure are available on the project page of this study.
KEYWORDS: 3D image processing, 3D displays, 3D modeling, Analytical research, Information technology, Optical filters, Glasses, Behavioral sciences, Safety, Photography
This paper describes an experiment that focuses on disparity changes in emotional scenes of stereoscopic (3D) images, in which an examination of the effects on pleasant and arousal was carried out by adding binocular disparity to 2D images that evoke specific emotions, and applying disparity modification based on the disparity analysis of prominent 3D movies. From the results of the experiment, it was found that pleasant and arousal was increased by expanding 3D space to a certain level. In addition, pleasant gradually decreased and arousal gradually increased by expansion of 3D space above a certain level.
An established way of validating and testing new image quality assessment (IQA) algorithms have been to compare how
well they correlate with subjective data on various image databases. One of the most common measures is to calculate
linear correlation coefficient (LCC) and Spearman’s rank order correlation coefficient (SROCC) against the subjective
mean opinion score (MOS). Recently, databases with multiply distorted images have emerged 1,2. However with
multidimensional stimuli, there is more disagreement between observers as the task is more preferential than that of
distortion detection. This reduces the statistical differences between image pairs. If the subjects cannot distinguish a
difference between some of the image pairs, should we demand any better performance with IQA algorithms? This paper
proposes alternative performance measures for the evaluation of IQA’s for the CID2013 database. One proposed
alternative performance measure is root-mean-square-error (RMSE) value for the subjective data as a function of the
number of observers. The other alternative performance measure is the number of statistical differences between image
pairs. This study shows that after 12 subjects the RMSE value saturates around the level of three, meaning that a target
RMSE value for an IQA algorithm for CID2013 database should be three. In addition, this study shows that the state-of-the-art IQA algorithms found the better image from the image pairs with a probability of 0.85 when the image pairs with
statistically significant differences were taken into account.
To understand the viewing strategies employed in a quality estimation task, we compared two visual tasks—quality estimation and difference estimation. The estimation was done for a pair of natural images having small global changes in quality. Two groups of observers estimated the same set of images, but with different instructions. One group estimated the difference in quality and the other the difference between image pairs. The results demonstrated the use of different visual strategies in the tasks. The quality estimation was found to include more visual planning during the first fixation than the difference estimation, but afterward needed only a few long fixations on the semantically important areas of the image. The difference estimation used many short fixations. Salient image areas were mainly attended to when these areas were also semantically important. The results support the hypothesis that these tasks’ general characteristics (evaluation time, number of fixations, area fixated on) show differences in processing, but also suggest that examining only single fixations when comparing tasks is too narrow a view. When planning a subjective experiment, one must remember that a small change in the instructions might lead to a noticeable change in viewing strategy.
This paper describes a study that focuses on disparity changes in emotional scenes of stereoscopic (3D) images, in which
an examination of the effects on pleasant and arousal was carried out by adding binocular disparity to 2D images that
evoke specific emotions, and applying disparity modification based on the disparity analysis of famous 3D movies.
From the results of the experiment, for pleasant, a significant difference was found only for the main effect of the
emotions. On the other hand, for arousal, there was a trend of increasing the evaluation values in the order 2D condition,
3D condition and 3D condition applied the disparity modification for happiness, surprise, and fear. This suggests the
possibility that binocular disparity and the modification affect arousal.
The authors have analyzed binocular disparity included in stereoscopic (3D) images from the perspective of producing depth sensation. This paper described the disparity analysis conducted by the authors for well-known 3D movies. Two types of disparity analysis were performed; full-length analysis of four 3D movies and analysis of emotional scenes from them. This paper reports an overview of the authors’ approaches and the results obtained from their analysis.
This study presents a geometric and subjective analysis of typical mobile stereoscopic 3-D images. The geometry of the stereoscopic pipeline from the scene to the eyes of the viewer is a highly relevant issue in stereoscopic media. One important factor is camera separation, because it can be used to control the perceived depth of stereoscopic images. The geometric analysis included consideration of disparity and roundness factor within typical mobile stereoscopic imaging scenes. These geometric properties of stereoscopic 3-D images were compared to subjective evaluations by varying camera separation in different scenes. The participants in this study evaluated the strength and naturalness of depth sensation and the overall viewing experience from still images with the single-stimulus method. The results showed that participants were able to perceive the change of depth range even though the images were shown in random order without a reference depth scale. The highest naturalness of depth sensation and viewing experience were achieved with 2 cm to 6 cm camera separation in every content. With these preferred camera separations, the disparity range was less than ±1 deg and cardboard effect (quantified with roundness factor) did not negatively affect the naturalness of depth sensation.
A stereoscopic 3-D version of the film Avatar was shown to 85 people who subsequently answered questions related to sickness, visual strain, stereoscopic image quality, and sense of presence. Viewing Avatar for 165 min induced some symptoms of visual strain and sickness, but the symptom levels remained low. A comparison between Avatar and previously published results for the film U2 3D showed that sickness and visual strain levels were similar despite the films' runtimes. The genre of the film had a significant effect on the viewers' opinions and sense of presence. Avatar, which has been described as a combination of action, adventure, and sci-fi genres, was experienced as more immersive and engaging than the music documentary U2 3D. However, participants in both studies were immersed, focused, and absorbed in watching the stereoscopic 3-D (S3-D) film and were pleased with the film environments. The results also showed that previous stereoscopic 3-D experience significantly reduced the amount of reported eye strain and complaints about the weight of the viewing glasses.
Camera separation affects the perceived depth in stereoscopic movies. Through control of the separation and thereby the
depth magnitudes, the movie can be kept comfortable but interesting. In addition, the viewing context has a significant
effect on the perceived depth, as a larger display and longer viewing distances also contribute to an increase in depth.
Thus, if the content is to be viewed in multiple viewing contexts, the depth magnitudes should be carefully planned so
that the content always looks acceptable. Alternatively, the content can be modified for each viewing situation. To
identify the significance of changes due to the viewing context, we studied the effect of stereoscopic camera base
distance on the viewer experience in three different situations: 1) small sized video and a viewing distance of 38 cm, 2)
television and a viewing distance of 158 cm, and 3) cinema and a viewing distance of 6-19 meters. We examined three
different animations with positive parallax. The results showed that the camera distance had a significant effect on the
viewing experience in small display/short viewing distance situations, in which the experience ratings increased until the
maximum disparity in the scene was 0.34 - 0.45 degrees of visual angle. After 0.45 degrees, increasing the depth
magnitude did not affect the experienced quality ratings. Interestingly, changes in the camera distance did not affect the
experience ratings in the case of television or cinema if the depth magnitudes were below one degree of visual angle.
When the depth was greater than one degree, the experience ratings began to drop significantly. These results indicate
that depth magnitudes have a larger effect on the viewing experience with a small display. When a stereoscopic movie is
viewed from a larger display, other experiences might override the effect of depth magnitudes.
In this study, two experiments were conducted to evaluate the psycho-physiological effects by practical use of monocular
head-mounted display (HMD) in a real-world environment, based on the assumption of consumer-level applications as
viewing video content and receiving navigation information while walking. In the experiment 1, the workload was
examined for different types of presenting stimuli using an HMD (monocular or binocular, see-through or non-see-through).
The experiment 2 focused on the relationship between the real-world environment and the visual information
presented using a monocular HMD. The workload was compared between a case where participants walked while
viewing video content without relation to the real-world environment, and a case where participants walked while
viewing visual information to augment the real-world environment as navigations.
We measured the eye movements of participants who watched a 6-minute movie in stereoscopic and non-stereoscopic
form. We analyzed four shots of the movie. The results indicate that in a 2D movie viewers tended to look at the actors,
as most of the eye movements were clustered there. The significance of the actors started at the beginning of a shot, as
the eyes of the viewer focused almost immediately to them. In S3D movie the eye movement patterns were more widely
distributed to other targets. For example, complex stereoscopic structures and objects nearer than the actor captured the
interest and eye movements of the participants. Also, the tendency to first look at the actors was diminished in the S3D
shots. The results suggests that in a S3D movie there are more eye movements which are directed to wider array of objects
than in a 2D movie.
In this paper, the authors conducted an experiment to evaluate the UX in an actual outdoor environment, assuming the
casual use of monocular HMD to view video content while short walking. In conducting the experiment, eight subjects
were asked to view news videos on a monocular HMD while walking through a large shopping mall. Two types of
monocular HMDs and a hand-held media player were used, and the
psycho-physiological responses of the subjects were
measured before, during, and after the experiment. The VSQ, SSQ and NASA-TLX were used to assess the subjective
workloads and symptoms. The objective indexes were heart rate and stride and a video recording of the environment in
front of the subject's face. The results revealed differences between the two types of monocular HMDs as well as
between the monocular HMDs and other conditions. Differences between the types of monocular HMDs may have been
due to screen vibration during walking, and it was considered as a major factor in the UX in terms of the workload.
Future experiments to be conducted in other locations will have higher cognitive loads in order to study the performance
and the situation awareness to actual and media environments.
Subjective image quality data for 9 image processing pipes and 8 image contents (taken with mobile phone
camera, 72 natural scene test images altogether) from 14 test subjects were collected. A triplet comparison setup
and a hybrid qualitative/quantitative methodology were applied. MOS data and spontaneous, subjective
image quality attributes to each test image were recorded. The use of positive and negative image quality
attributes by the experimental subjects suggested a significant difference between the subjective spaces of low
and high image quality. The robustness of the attribute data was shown by correlating DMOS data of the test
images against their corresponding, average subjective attribute vector length data. The findings demonstrate
the information value of spontaneous, subjective image quality attributes in evaluating image quality at variable
quality levels. We discuss the implications of these findings for the development of sensitive performance
measures and methods in profiling image processing systems and their components, especially at high image
quality levels.
KEYWORDS: 3D displays, 3D image processing, Mobile devices, Image quality, Visualization, 3D vision, Glasses, Large screens, Small screens, 3D visualizations
High-quality stereoscopic image content must be viewable in a variety of visual environments, from 3-D theaters to 3-D
mobile devices. Stereoscopic effects, however, are affected by screen size, viewing distance, and other parameters. In
this study, the authors focus on the stereoscopic image quality experience of viewing 3-D content on a mobile device in
order to compare it with that of viewing 3-D content on a large screen. The stereoscopic image quality experience was
evaluated using Interpretation Based Quality (IBQ) methodology, which combines existing approaches to image quality
evaluation, such as the paired comparison and interview, and assesses the viewer experience using both quantitative and
qualitative data. Five stereoscopic images were used in the experiment. The results of the experiment suggest that the
discomfort felt while viewing stereoscopic images on a 3-D mobile device arise from not only visual fatigue but also the
effects of the smaller screen size. The study also revealed the types of stereoscopic images that are suitable for viewing
on 3-D mobile devices.
KEYWORDS: 3D image processing, 3D displays, Visualization, Safety, Prototyping, Image analysis, 3D imaging standards, Software development, Standards development, Algorithm development
Digital 3D cinema has recently become popular and a number of high-quality 3D films have been produced. However, in
contrast with advances in 3D display technology, it has been pointed out that there is a lack of suitable 3D content and
content creators. Since 3D display methods and viewing environments vary widely, there is expectation that high-quality
content will be multi-purposed. On the other hand, there is increasing interest in the bio-medical effects of image content
of various types and there are moves toward international standardization, so 3D content production needs to take into
consideration safety and conformity with international guidelines. The aim of the authors' research is to contribute to the
production and application of 3D content that is safe and comfortable to watch by developing a scalable 3D conversion technology. In this paper, the authors focus on the process of changing the screen size, examining a conversion algorithm and its effectiveness. The authors evaluated the visual load imposed during the viewing of various 3D content converted by the prototype algorithm as compared with ideal conditions and with content expanded without conversion. Sheffe's paired comparison method was used for evaluation. To examine the effects of screen size reduction on viewers, changes in user impression and experience were elucidated using the IBQ methodology. The results of the evaluation are presented along with a discussion of the effectiveness and potential of the developed scalable 3D conversion algorithm
and future research tasks.
KEYWORDS: Video, Televisions, Mobile devices, Multimedia, Video processing, Visualization, Information technology, Sensors, Signal processing, Diffusion
In the product development of services it is important to adjust mobile video quality according to the quality
requirements of potential users. Therefore, a careful participant selection is very important. However, in the literature the
details of participant selection are often handled without great detail. This is also reflected in the handling of
experimental results, where the impact of psychographic factors on quality is rarely reported. As the user attributes
potentially have a large effect to the results, we investigated the role of various psychographical variables on the
subjective evaluation of audiovisual video quality in two different experiments. The studied variables were age, gender,
education, professionalism, television consumption, experiences of different digital video qualities, and attitude towards
technology. The results showed that quality evaluations were affected by almost all background factors. The most
significant variables were age, professionalism, knowledge of digital quality features and attitude towards technology.
The knowledge of these factors can be exploited in careful participant selection, which will in turn increase the validity
of results as the subjective evaluations reflect better the requirements of potential users.
Stereoscopic technologies have developed significantly in recent years. These advances require also more understanding
of the experiental dimensions of stereoscopic contents. In this article we describe experiments in which we explore the
experiences that viewers have when they view stereoscopic contents. We used eight different contents that were shown
to the participants in a paired comparison experiment where the task of the participants was to compare the same content
in stereoscopic and non-stereoscopic form. The participants indicated their preference but were also interviewed about
the arguments they used when making the decision. By conducting a qualitative analysis of the interview texts we
categorized the significant experiental factors related to viewing stereoscopic material. Our results indicate that reality-likeness
as well as artificiality were often used as arguments in comparing the stereoscopic materials. Also, there were
more emotional terms in the descriptions of the stereoscopic films, which might indicate that the stereoscopic projection
technique enhances the emotions conveyed by the film material. Finally, the participants indicated that the three-dimensional
material required longer presentation time, as there were more interesting details to see.
Subjective evaluation is used to identify impairment factors of multimedia quality. The final quality is often formulated
via quantitative experiments, but this approach has its constraints, as subject's quality interpretations, experiences and
quality evaluation criteria are disregarded. To identify these quality evaluation factors, this study examined qualitatively
the criteria participants used to evaluate audiovisual video quality. A semi-structured interview was conducted with 60
participants after a subjective audiovisual quality evaluation experiment. The assessment compared several, relatively
low audio-video bitrate ratios with five different television contents on mobile device. In the analysis, methodological
triangulation (grounded theory, Bayesian networks and correspondence analysis) was applied to approach the qualitative
quality. The results showed that the most important evaluation criteria were the factors of visual quality, contents,
factors of audio quality, usefulness - followability and audiovisual interaction. Several relations between the quality
factors and the similarities between the contents were identified. As a research methodological recommendation, the
focus on content and usage related factors need to be further examined to improve the quality evaluation experiments.
There are innumerable concepts, terms and definitions for user experience. Few of them have a solid empirical foundation.
In trying to understand user experience in interactive technologies such as computer games and virtual environments,
reliable and valid concepts are needed for measuring relevant user reactions and experiences. Here we present
our approach to create both theoretically and methodologically sound methods for quantification of the rich user experience
in different digital environments. Our approach is based on the idea that the experience received from a content
presented with a specific technology is always a result of a complex psychological interpretation process, which components
should be understood. The main aim of our approach is to grasp the complex and multivariate nature of the experience
and make it measurable. We will present our two basic measurement frameworks, which have been developed
and tested in large data set (n=2182). The 15 measurement scales extracted from these models are applied to digital
gaming with a head-mounted display and a table-top display. The results show how it is possible to map between experience,
technology variables and the background of the user (e.g., gender). This approach can help to optimize, for
example, the contents for specific viewing devices or viewing situations.
Stereoscopic display produces enhanced game playing experience for the user. However, this experience might be affected by eye strain symptoms produced by the convergence-accommodation conflict in the visual system. In this study we measured the level of sickness symptoms in mobile stereoscopic game playing situation. Our results showed that playing a mobile game with an autostereoscopic display did not cause eye strain that differed from eye strain caused by ordinary mobile device usage. The results suggest that with sufficiently small disparities a mobile stereoscopic display can be used to achieve a comfortable user experience. We also found links between experienced sickness symptoms and background variables. Firstly, our results indicated that females reported higher symptom levels than males. Secondly, we showed that the participants with higher susceptibility to motion sickness reported higher sickness
levels in the experiment. Thirdly, we showed that participants with less computes skills or with less enthusiastic attitude towards new technology had significantly more sickness symptoms than the other participants.
Image evaluation schemes must fulfill both objective and subjective requirements. Objective image quality evaluation models are often preferred over subjective quality evaluation, because of their fastness and cost-effectiveness. However, the correlation between subjective and objective estimations is often poor. One of the key reasons for this is that it is not known what image features subjects use when they evaluate image quality. We have studied subjective image quality evaluation in the case of image sharpness. We used an Interpretation-based Quality (IBQ) approach, which combines both qualitative and quantitative approaches to probe the observer's quality experience. Here we examine how naive subjects experienced and classified natural images, whose sharpness was changing. Together the psychometric and qualitative information obtained allows the correlation of quantitative evaluation data with its underlying subjective attribute sets. This offers guidelines to product designers and developers who are responsible for image quality. Combining these methods makes the end-user experience approachable and offers new ways to improve objective image quality evaluation schemes.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.