Pixel Counting in 360 Degree Videos: Reinventing Resolution

Since digital cameras arrived (like, really arrived, commercially speaking), we’ve enjoyed a very convenient method of determining their achievable level of detail; we count their pixels, and call it “resolution”.

But as virtual reality enthusiasts, developers, deep thinkers and content creators are aware, digital video and images are changing. We can no longer consider them to be “flat”, and yet we still don’t have a reasonable alternative than chopping up the captured image into a nice, neat grid.

So what does resolution really mean any more as we sprint headlong toward immersive video formats?

What’s the Pixel Problem?

Resolution was actually never a great way to measure a digital camera’s quality. A high pixel density CCD, for example, could still have a rubbish lense to look through, and its image processing could be woefully underpowered and put your photos through a digital meat grinder.

But it was undeniably convenient to compress a camera’s technological prowess into one, neat number, and resolution is the number that the marketing department landed upon. Also, the number is usually pretty high, and sounds impressive.

It’s all a little easier when you picture it in terms of a TV, rather than a camera sensor, and the two devices are divided up and measured just the same way.

Let’s consider what “4K” actually means.

It refers to your TV (or camera’s CCD) having a horizontal pixel count of 3,840 (note that vertical count isn’t standardized for 4K, due to different cinematic widescreen ratios). What’s confusing about this is that, A) It’s not actually 4,000 (or 4K) pixels, although it’s close enough that we can forgive it, and, B) The previous HD resolution was counted along the vertical instead, being 1,080 pixels from top to bottom. 8K is counted on the horizontal too, and is just 4K resolutions doubled.

So if we were to follow the previous convention, 4K should really be called 2160, since we never chose to refer to 1080 TVs as being “2K”. Which is what they are. Clear so far?

No, me either. The real point here is demonstrating how resolution is more about marketing than it is about quality.

Even more so when it comes to spherical video.

Here’s a comparison of the various resolutions as they look when flattened out, as they are on your TV.

So, What Resolution Are Spherical Videos?

Ah, now you’re asking all the right questions!

Firstly, consider this: There aren’t any single camera lenses that can capture a spherical video, and probably never will be. Instead, 360 degree videos are achieved using multiple cameras pointing in different directions.
Does that mean that we simple add up their individual pixel counts to determine the resolution of a spherical camera cluster, and thereby determine its quality of detail? Sadly, no.

The images from those individual cameras are “stitched” together afterwards, and an overlap is required to make that stitching as seamless as possible. The greater the overlap, the better the stitching but the lower the overall resolution of the spherical video.

Note: It’s worth drawing a comparison back to TVs here. The image quality of high definition TVs benefited more from progressive scan than it did from resolution (as opposed to 1080i, which meant the images was still interlaced, despite being HD resolution). Similarly, high quality stitching is vitally important in the quality of your spherical videos, regardless of resolution. Keep this in mind when purchasing your gear.

It might actually prove to be beneficial that those marketing people opted for horizontal pixel count with the 4K standard as we create more and more spherical content.

After processing, we can still measure the horizontal pixel count in 360 degree videos quite easily (and in 180 degree, half-spherical content, too), stopping when the image repeats itself. But this is highly unlikely to actually hit 4K standards on the dot, because everyone’s processing their stitching differently, even if the cameras are a fixed resolution.

Neither does it take into account the vertical, which is admittedly less important than horizontal since that’s how our peripheral vision works. But let’s say you have eight cameras around the horizontal circumference but doesn’t have any pointing up or down, you’re video is still going to suffer from a bit of a fisheye effect due to the “flattening” of your sphere.

So this type of lens distortion also messes with our convenient resolution-based quality quantification by requiring us to factor in not only the number of cameras used in a 360 degree capture rig, but also their positioning and allocation.

The GoPro 360 camera rig. Lots of cameras means lots of stitching, but smoother overlap and a better post-production resolution.

So How Do I Tell if a 360 Degree Camera is Any Good?

The truth is, manufacturers aren’t likely to stop using resolution as the catch-all yard stick of quality. So rather than fight it, we just need to re-learn how we think about it in terms of spherical content, rather than “flat” images.
One standard that’s beginning to emerge in the 360 degree video arena is the rather obscure sounding 5.2K. Seems like an odd number, right, given our proclivity for 4K?

This small jump up from 4K offers overhead for stitching losses and is actually quite useful when talking about spherical resolutions. It’s ultimate purpose is to take us back to good old, recognizable 1080 HD standards as a minimum base resolution for a 360 degree video.

In order for a camera rig to consider itself as being 5.2K, the post-production image will work out as being the equivalent of 1080p no matter which direction you’re looking in. So you can see how a 4K spherical image actually comes out as being lower resolution than your old 1080 TV. With industry leaders like GoPro getting behind a minimum acceptable spherical resolution of 5.2K, it’s likely to help drive our much-needed industry standards.

In the end, just like everything, the bigger the number the better the camera. Just remember that resolution should not be your only deciding factor when it comes to a spherical camera; even less so than when you choose a TV.

And we haven’t even started on stereoscopic recording, which requires two cameras pointing in each direction (and could halve the number of cameras you think your rig is using — after all, eight 1080P cameras on a stereoscopic rig is actually capturing less than 4K, post stitching, on the horizontal plane).

But that’s a dilemma for another day.

DIY, but still works!

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.