Introducing our new Wide and Night scores

DXOMARK started testing smartphone cameras back in 2012, which to many people may not seem too long ago, but could easily be called an eternity in terms of mobile technology. Mobile imaging technology in particular continues to move forward at warp speed, far outpacing the traditional camera sector; and compared to the first smartphone cameras we tested, such as the Apple iPhone 4, the Samsung Galaxy S2, and the Nokia 808 PureView, the latest generation of flagship smartphones are veritable mobile imaging juggernauts.

We are continuously monitoring the advancements in technology in order to adapt, upgrade, and improve our testing methodologies and protocols. In 2017, we launched the first update to our DXOMARK Mobile smartphone camera testing protocol, adding dedicated tests for simulated bokeh modes and zoom quality, as well as for low-light testing down to 1 lux and motion-based test scenes. In January 2019, we introduced the DXOMARK Selfie test protocol to cover the technological advancements and growing popularity of smartphone front cameras.

Now it’s time for another update to the DXOMARK smartphone main camera protocol: We are adding new Wide and Night scores—Wide to accommodate the popularity of ultra-wide cameras, and Night to evaluate the improved low-light capabilities and dedicated night modes of smartphone cameras.

LG started the trend towards ultra-wide cameras with its G5 model in 2016, and since then most manufacturers have adopted the concept, with pretty much all 2019 flagship phones offering the capability to squeeze more scene into your pictures. For our Wide protocol, we undertake “mini-tests” that cover all the same image attributes as our main camera test, including exposure and dynamic range, color rendering, texture and detail, image noise, and autofocus performance and accuracy. We test wide-angle cameras in real-life scenes with light levels ranging from approximately 1000 to 50 lux. We not only test at the widest focal length, but repeat all tests at 2mm focal-length steps up to 20mm equivalent. (For more details about our new Wide test protocol, see “Expanding the DXOMARK Camera protocol: The Wide score.”)

Thanks to improved image sensors, faster lenses, and computational imaging methods (among other advances in technology), current smartphones are much more capable tools for low-light photography than previous generations. To reflect this in our testing, we have developed the new Night testing protocol, for which we capture a variety of real-life low-light scenes, including night cityscapes, low-light portraits, and shots in extreme low light—in flash-on, flash-off, and flash-auto modes. If a smartphone has a dedicated night mode, we test it as well.

We have integrated into our Night protocol our previous lab-based test results, and they feed into the Night score. (For more details about our new Night test protocol, see “Expanding the DXOMARK Camera protocol: The Night score.”)

We have also decided to rename our smartphone camera testing protocol. From now, on DXOMARK Mobile is DXOMARK Camera.

Review updates

We have subjected a large proportion of the devices in our current rankings to our new Wide and Night tests and have updated the reviews and scores accordingly. You will currently find three types of reviews in DXOMARK Camera:

• Fully-updated reviews: We have fully updated these reviews with modified scores and new Wide and Night sections. The new and updated sections are right at the top of the article; if you scroll down, you can still read the complete original review. We have fully-updated reviews for the following smartphones: Samsung Galaxy S10 5G, Google Pixel 3, Huawei Mate 20, Lenovo Z6 Pro, Honor 20 Pro, Xiaomi Mi 9, Huawei P30 Pro, and Samsung Galaxy Note 10+ 5G.
• Reviews with updated scores only: We have fully tested these smartphones using the new protocols, but we have (so far) only added the new Wide and Night scores and recalculated the Overall score, but not updated the review text. The following smartphones have updated scores: Apple iPhone XS Max, LG G8, Sony Xperia 1, and Nokia 9.
• Reviews not updated: There are a number of smartphones (mainly older and/or less popular) that we have not yet tested with the new protocols. Their DXOMARK Camera scores are therefore not directly comparable to those found in the updated reviews. Nevertheless, reading these reviews will give you a very good understanding of the devices’ camera performance. (We have marked with an asterisk these in our smartphone camera rankings.)

We are still working on additional tests and are planning to update reviews not yet updated, and add new test sections to those that underwent only a score update. Please also note that for consistency of testing, we undertook all Wide and Night tests with the same device firmware we used for the original testing; this means that there may be devices using a more recent firmware version.

We have also taken this opportunity to make some changes to our review format. You will now find at the top of every image quality attribute section the score of the device being tested, displayed next to the score of the best device in the category  for reference.

The new DXOMARK Camera ranking

Just like Exposure, Texture, Zoom, and Bokeh, the Wide and Night scores are sub-scores of the Photo score, which in turn is a sub-score of the DXOMARK Camera Overall score.

The Wide score works like a bonus: with a good Wide score, wide-angle-equipped devices can improve their Photo and Overall scores, but a bad result won’t reduce them. So even a terrible wide-angle camera is not any worse than no wide-angle camera at all.

The Night score replaces the previous Flash score in our scoring system and can increase or decrease the Photo and Overall scores. Within the Night test protocol, the results count as a bonus in a similar way as the Wide score does for the Photo and Overall scores—if Night mode results are better than images captured at default camera settings, they can improve the overall Night score; but bad image quality in night mode, or the lack of a night node, won’t reduce the Night score. (Please also note that the new Night sub-score is not comparable to the previous Flash score.)

Most of the retested devices in our rankings increased their Photo and DXOMARK Camera Overall scores. The score changes are fairly small, but vary from device to device.

Our current number one, the Samsung Galaxy Note 10+ 5G, firmly defended its top spot by securing the best results for both Wide at 42 and Night at 59. This pushed its Photo score up by 8 points, the largest increase for all devices, resulting in a new Overall score of 117, up from 113.

The Samsung Galaxy S10 5G and Huawei P30 also maintain their podium positions. Both achieved the same Night score as the Note 10+, but scored slightly lower for Wide (40 for the S10 5G, 34 for the P30 Pro). They both score 116 points overall in the updated ranking.

In contrast to the three devices mentioned above, the Apple iPhone XS Max and Google Pixel 3 have to make do without a wide-angle camera. The Pixel scores a respectable 49 for Night; but at 37, the Apple is one of the weaker devices in this category. Without the wide-angle bonus, both devices increased their Overall score by only one point.

Further down in the rankings, both the LG G8 and Sony Xperia 1 were able to increase their Photo scores by a more substantial 5 points and their Overall scores by 3, thanks to a decent Night performance (44) for the LG and a respectable Wide score (38) for the Sony. Nonetheless, they have not been able to close the gap with the front runners.

The Nokia 9 PureView is the only retested device that was not able to increase its Photo and Overall scores. It does not come with a wide-angle option and comes in dead last in our Night ranking (31).

We hope you found this brief overview of our new Wide and Night scores useful. For more details about testing and scoring, please read the following articles:

• Expanding the DXOMARK Camera protocol: The Wide score
• Expanding the DXOMARK Camera protocol: The Night score

Click through to our smartphone section to see all the updated DXOMARK Camera reviews and test results.