Halide Does Deep Dive Into iPhone SE's Software-Based Portrait Mode

Though it is a budget device with a single-lens camera, the iPhone SE features support for Portrait Mode, enabled through the powerful A13 chip in the smartphone.

iphonesehandson
It is the first of Apple's smartphones to offer Portrait Mode photos created entirely with software techniques rather than hardware, which prompted the developers behind popular iOS camera app Halide to take a deep dive into how it works.

The ‌iPhone SE‌ is equipped the same camera sensor as the iPhone 8, based on a recent teardown done by iFixit, but its camera can do more because it's using "Single Image Monocular Depth Estimation," aka generating Portrait Mode effects using a 2D image.

As Halide developer Ben Sandofsky points out, the ‌iPhone‌ XR is also a single-lens camera with Portrait Mode support, but the ‌iPhone‌ XR gets depth information through hardware. That's not possible on the ‌iPhone SE‌ because the older camera sensor doesn't support the feature.

Halide has discovered that unlike other iPhones, the ‌iPhone SE‌ can take a picture of another picture to attempt to develop a depth map. The app was even able to take a photo of an old slide film, adding depth effects to a 50 year old photo.

halideoldphoto

A picture of a picture and the resulting depth map from the ‌iPhone SE‌

The ‌iPhone SE‌'s Portrait Mode is somewhat limited because it only works with people, which is due to the neural network that powers the feature. When a Portrait Mode image without a person is captured, it fails in various ways because it can't create an accurate estimated depth map.

The ‌iPhone‌ XR also limited Portrait Mode to people alone, and using Portrait Mode with other objects requires upgrading to one of Apple's more expensive phones.

According to Halide, depth maps on the ‌iPhone SE‌ (or any phone with Portrait Mode) can be viewed by using the Halide app and then shooting in Depth mode. Halide's full breakdown of the ‌iPhone SE‌'s Portrait Mode can be read over on the Halide website.

Related Roundup: iPhone SE
Tag: Halide
Buyer's Guide: iPhone SE (Don't Buy)
Related Forum: iPhone

Popular Stories

Generic iOS 18

Apple Announces iOS 18.2 Launching Today With These New Features

Wednesday December 11, 2024 5:23 am PST by
Apple has announced that iOS 18.2, iPadOS 18.2, and macOS Sequoia 15.2 will be released today following more than six weeks of beta testing. For the iPhone 15 Pro and iPhone 16 models, the update introduces additional Apple Intelligence features, including Genmoji for creating custom emoji, Image Playground and Image Wand for generating images, and ChatGPT integration for Siri. There is also ...
iphone 17 pro concept render cameras

Major iPhone 17 Pro Redesign Backed by Supply Chain Info, Claims Leaker

Thursday December 12, 2024 4:36 am PST by
Next year's iPhone 17 Pro models will reportedly feature a major redesign, specifically centering around changes to the rear camera module, and now new supply chain information appears to confirm the striking change, according to a Chinese leaker. iPhone 17 Pro concept render Late last month, The Information's Wayne Ma claimed that the rear of the ‌iPhone 17‌ Pro and ‌iPhone 17‌ Pro...
m4 mac mini hands on

Cloud-Based M4 and M4 Pro Mac Mini Models Now Available

Wednesday December 11, 2024 7:34 am PST by
Developers now have access to cloud-based M4 and M4 Pro Mac mini units via MacWeb, a Silicon Valley-based provider of cloud services. The company has launched three configurations of the new Mac mini, powered by Apple's M4 and M4 Pro chips. Developers and IT teams can rent these machines for tasks ranging from basic development to advanced artificial intelligence modeling, providing an...
macOS Sequoia Night Feature

Apple Releases macOS Sequoia 15.2 With New Apple Intelligence Features

Wednesday December 11, 2024 10:02 am PST by
Apple today released macOS Sequoia 15.2, the second update to the macOS Sequoia operating system that was released in September. macOS Sequoia 15.2 comes over a month after the release of macOS Sequoia 15.1. Mac users can download the ‌macOS Sequoia‌ update through the Software Update section of System Settings. macOS Sequoia 15.2 adds Image Playground, an app that lets you create...
macbook air m2 13 inch

macOS Sequoia 15.2 Confirms New M4 MacBook Air Models Are Coming

Wednesday December 11, 2024 10:54 am PST by
Apple today made a mistake with its macOS Sequoia 15.2 update, releasing the software for two Macs that have yet to be launched. There is a software file for "Mac16,12" and "Mac16,13," which are upcoming MacBook Air models. The leaked software references the "MacBook Air (13-inch, M4, 2025)" and the "MacBook Air (15-inch, M4, 2025)," confirming that new M4 MacBook Air models are in...
maxresdefault

Apple Releases iOS 18.2 and iPadOS 18.2 With Genmoji, Image Playground, Siri ChatGPT and More

Wednesday December 11, 2024 10:03 am PST by
Apple today released iOS 18.2 and iPadOS 18.2, the second major updates to the iOS 18 and iPadOS 18 updates that came out in September. The new updates come over a month after Apple released iOS 18.1 and iPadOS 18.1. Subscribe to the MacRumors YouTube channel for more videos. The new software can be downloaded on eligible iPhones and iPads over-the-air by going to Settings > General >...
Generic iOS 18

Apple Seeds Second Release Candidate Versions of iOS 18.2 and More With Genmoji, Image Playground and ChatGPT Integration

Monday December 9, 2024 10:06 am PST by
Apple today seeded the second release candidate versions of upcoming iOS 18.2, iPadOS 18.2, and macOS 15.2 updates to developers and public beta testers for testing purposes, a week after releasing the first RCs. The first iOS 18.2 RC had a build number of 22C150, while the second RC's build number is 22C151. Release candidates represent the final version of beta software that's expected to see a ...

Top Rated Comments

luvbug Avatar
61 months ago

Could you have used a nicer pic? lol
She's probably the author's (Halide's) grandmother. Take a break, you'll be old too, sooner than you can imagine.
Score: 21 Votes (Like | Disagree)
AngerDanger Avatar
60 months ago

Could you have used a nicer pic? lol
As you wish. :)



Attachment Image
Score: 18 Votes (Like | Disagree)
NickName99 Avatar
61 months ago

This is wildly impressive.
Agreed - this is far more impressive than most people realize. The Pixel phones and the iPhone XR achieve this with a single lens by leveraging focus pixels to generate a depth map from available 3D information. The SE doesn’t have enough focus pixels for that, so it’s relying purely on machine learning, building a depth map from 2D information.

I’m surprised it was more cost effective to do it this way rather than just make the changes necessary to fit the XR rear camera module. That really speaks to the expense of retooling manufacturing.
Score: 10 Votes (Like | Disagree)
farewelwilliams Avatar
61 months ago
Theoretically, you should be able to extract frames from a video, apply neural network from Portrait Mode on to each frame, and stitch it back into a video.

Highly likely iOS 14 will have Portrait Mode for video. Would be only enhanced with Lidar too. This will take cinematography on a smartphone to a whole new level.
Score: 9 Votes (Like | Disagree)
NickName99 Avatar
60 months ago

I had assumed the criticism was because the image is low resolution and blurry, making it a poor test image.
I‘d say the worse the quality of the photo (resolution and sharpness), the more impressive that it works.
Score: 8 Votes (Like | Disagree)
JSRinUK Avatar
60 months ago

wonder if it works on pets... or if it will in the future
Hoping that no one minds a photo of me here, but I did a test photo when playing with my dog yesterday.

I'm not sure how the portrait mode works, but I was quite impressed that both myself and my dog are in focus.



Attachment Image
Score: 6 Votes (Like | Disagree)