Sony Launches World’s First Image Sensor With Built-in AI

Today, Sony has unveiled two 12.3MP image sensors — IMX500 and IMX501 that have AI capability onboard.

That means the revolutionary AI image sensors will be able to handle “light” machine learning tasks without depending on secondary AI hardware. This is an important feat, especially for the surveillance camera industry.

The connection between AI and camera systems has never been direct. If a camera were to recognize an object using machine learning capabilities, it will either rely on an external AI chip on the device or ask for assistance from the cloud.

An example of machine learning processing on the device would be the Pixel Visual Core chip in Google Pixel smartphones. Its job is to improve performance when capturing photos in HDR+ mode. In a nutshell, cameras with AI capabilities can be used in a number of areas.

Now, with AI embedded in Sony’s image sensors, the machine processing tasks will get faster and will be more secure, since the part that includes sending data to the cloud for analysis has been eliminated. On-device AI processing will also help in safeguarding users’ privacy.

As Sony’s VP of business and innovation, Mark Hanson, tells Verge, the new image sensor can only work with “basic” algorithms. In other words, tasks like identifying objects in real-time aren’t something it can achieve right now. At the same time, it’s unwise of us to expect so much from the first generation.

Nevertheless, Hanson says that samples of IMX500 have already started shipping at a price of ¥10,000 ($93). Meanwhile, Sony products that will use the new AI image sensor will arrive in the first quarter of 2021.

It appears that Sony will keep on pushing the boundaries of the camera technology. The company has a 60% market share and sold about 1.6 billion image sensors last year.