Introduction

Smartphone cameras have turned everyone into a photographer. It has become very easy to document life’s moments in a photo or video because a camera is always at hand. Reliance on a smartphone camera means that image quality must be good in various usage conditions, from the easiest to the most challenging one, helping to capture the moment without any difficulty.  

Introduced in 2012, the DXOMARK Camera protocol, has been built to provide the casual or professional photographer a comprehensive overview of smartphone camera performance, with the right indicators to help them choose the right product for their needs. By combining measurements reproducing real-life usages run in our labs as well as tests in natural scenes, the overall score is composed of a total of five sub-scores: Photo, Zoom, Bokeh, Preview, and Video, allowing for a quick comparison among tested smartphone cameras.

To keep up with the latest technology trends and usages, DXOMARK regularly updates its protocols and continues to offer exhaustive and meaningful evaluations.  

Camera Score Structure

3000

Photos shot

and analyzed

150

Minutes of video

recorded

Learn more about our Use Cases

 

Photo 50%

We evaluate the performance of the smartphone when taking still images in a large number of use cases such as friends & family moments, landscape, close ups, cityscape, architecture, etc. 

 

Our tests are performed :

    • Using the default camera app and its default mode, at 1x zoom mode
    • In an extended range of lighting conditions from 1 lux for lowlight to 1000 lux for bright light conditions and even higher for actual outdoor scenes
    • With a tripod or through real or simulated handheld conditions.

To evaluate photo, assessments are made on a total of six technical quality attributes :

Exposure

How well the camera properly adjusts to and captures the brightness of the subject and the background. Face exposure and contrast are also evaluated.

Color

How accurately the camera can reproduce color in a variety of lighting conditions and use cases, as well as how pleasing its color rendering is to its viewers. We particularly focus on white balance, color rendering, color shading, as well as skin tone rendering, which are tested on a large spectrum, from deep to fair to light.

Autofocus

How quickly and accurately the camera can focus on a subject in a variety of lighting conditions. Besides speed, we also evaluate the depth of field of the camera and its capability to capture the instant

Texture

How well the camera can preserve small and fine details such as those found on object surfaces. A large set of details type is evaluated in all lighting conditions. The level of sharpness is also measured on a moving object, evaluating if the camera freezes well the motion while capturing a high level of details.

Noise

How much noise is present in an image, which is particularly sensitive in low light captures

Artifacts

Any effect appearing in an image which is not present in the original imaged object (distortion, halo effect…)

Important note:

Our range of acceptable renderings for all attributes in various light conditions are based on the findings gathered from our multi-cultural focus groups. We believe that within this inclusive scope, the risk of bias is eliminated when evaluating a smartphone camera for a particular attribute. In keeping with this approach to testing, we do not evaluate manufacturers’ signature renderings.

Video 28.5%

We evaluate the performance of the smartphone while recording videos in a large number of use cases, various lighting conditions and environments from natural low light to indoor and outdoor conditions.  

Our tests are performed:

    • Using the resolution setting and frame rate that provides the best video quality. If a main camera offers 4K video mode but uses 1080p Full HD by default, we’ll select 4K if 4K provides a better image quality (sometimes 4K is not stabilized which lead to a lower image quality).
    • In an extended range of lighting conditions from 1 lux for lowlight to 1000 lux for bright light conditions and even higher for actual outdoor scenes
    • With a tripod or through real or simulated handheld conditions from steady shot to some very challenging walking or even running scenarios.

To evaluate video, we evaluate seven quality attributes, the same as the ones applied for photo, with the addition of stabilization:

Exposure

How well the camera properly adjusts to and captures the brightness of the subject and the background. We also evaluate its adaptation and stability over time.

Color

How accurately the camera can reproduce color in a variety of lighting conditions, as well as how pleasing its color rendering is to its viewers. White Balance adaptation is also measured and evaluated on challenging use cases.

Autofocus

How quickly and accurately the camera can focus on a subject in a variety of lighting conditions. Face tracking capabilities are evaluated in various lighting conditions.

Texture

How well the camera can preserve small details such as those found on object surfaces

Noise

How much noise is present in an image, which is particularly sensitive in low light captures

Artifacts

Any effect appearing in an image which is not present in the original imaged object (distortion, halo effect…)

Stabilization

How well the camera eliminates motions that occur while capturing video walking, running, etc…

Important note:

Our range of acceptable renderings for all attributes in various light conditions are based on the findings gathered from our multi-cultural focus groups. We believe that within this inclusive scope, the risk of bias is eliminated when evaluating a smartphone camera for a particular attribute. In keeping with this approach to testing, we do not evaluate manufacturers’ signature renderings.

Zoom 14%

We evaluate the performance of the camera while zooming in and zooming out for still images and videos, from ultra-wide to very long-range zoom.

 

The Zoom evaluation breaks down into two key parts:

Tele

How well the device zooms in, looking at a telephoto zoom (to 350mm or 14x)

Wide

How well the device performs at the widest field of view, zooming out from the standard focal length (from 12mm or 0.5x)

We evaluate all image quality attributes, including artifacts. Every attribute is key in the experience, and we do a homogenous image quality evaluation for the complete zoom range. Some criteria may be more challenging than others. For example:

    • For Wide: distortion (straight lines appearing curved in an image), face deformation or chromatic aberration, level of detail and level of noise in the field of view 
    • For Tele: Texture preservation at various distances whether the device keeps a high level of details at different zoom distances 

Tests are performed for more than 10 zoom distances, for photo and video. In video, the smoothness of the zoom process during video recording is also evaluated.  

 

Preview 4.5%

The camera app preview has a huge impact on user experience and gives you a first impression of image quality. A photographer might simply choose not to capture an image at all if the preview looks like it’s not worth pressing the shutter button, even though the recorded image would have been absolutely fine.

 

We evaluate the consistency between the image seen on the screen before the shutter is pressed and the final image rendering.

 

To evaluate preview, we particularly check:

    • Default mode: Every image quality attribute is evaluated with a particular attention to overall exposure and HDR capabilities.
    • Bokeh and Portrait mode: Simulation of the bokeh effect is properly applied to the preview
    • Zooming photo: zoom smoothness of the exposure, color and focus adaptation when zooming from the minimal to the maximal zoom factor available

Bokeh 3%

This sub-score covers the popular yet non default mode often called Portrait or Bokeh in modern smartphones. We evaluate the overall image quality with a simulated depth of field effect. As smartphone cameras do not provide a natural lens effect, they need to simulate the depth of field.

 

We check how well the camera separates the subject from the background when taking portraits without accidental blur on the focus plane, as well as the quality of the gradation and naturalness of background blur, which makes the subject stand out. In addition, we evaluate the traditional portrait image quality attributes (such as exposure, color, texture, etc.)

 

Tests are performed with the bokeh dedicated mode within the camera application 

 

What we also Test