This site may earn affiliate commissions from the links on this folio. Terms of use.

Ambarella EVA by David Cardinal

It'due south conventional wisdom in the self-driving industry that lidar is a must-take. From time to time some accept argued otherwise, but almost every serious endeavour currently underway includes at least i lidar — until now. Flake maker Ambarella, best known for creating the epitome processing systems found in many cameras, has introduced its EVA (Embedded Vehicle Autonomy) evolution vehicle built on its CVFlow architecture and a phalanx of 20 cameras powered by 16 of its CV1 image and vision processing SoCs. I was lucky plenty to be one of the first reporters to ride in one, as the firm is now doing limited testing on public roads in Santa Clara, California.

Ambarella isn't trying to go head to head with the car companies and planning to launch its own cocky-driving vehicles. Instead, EVA, which is built on a modified Lincoln MKZ, is a development and design showcase for what Ambarella tin bring to those companies in the way of an democratic vehicle solution based almost entirely on vision, though it does take a single front-facing radar to help with tricky weather.

All In With Long- And Brusk-Range Cameras

The EVA uses six powerful stereo cameras to see and recognize objects at quite a distanceFor long range detection, EVA relies on six of Ambarella's SuperCam3 4K stereo cameras, each with a 75-degree field of view. They are bundled in a hexagon, providing total 360-degree coverage. Ambarella claims that the system tin can detect and identify a pedestrian out to 150 meters, and every bit far every bit 180 meters if coupled with boosted neural network software.

Each stereo camera is powered past its own pair of Ambarella'due south CV1 vision SoCs. The cameras use Sony IMX317 sensors, which are well-nigh the same size every bit a typical smartphone sensor, simply have lower resolution to give them bigger pixels and improved low-calorie-free and high-dynamic range performance. General Manager of Ambarella'southward subsidiary VisLab Alberto Broggi emphasized to me the importance of integrating the vision modules with the ISP (Image Signal Processor) in the camera. That'southward possible because Ambarella as a company makes much of its living developing ISP systems for a diversity of applications.

The short-range stereo cameras are identify in the forepart, back, and both sides of the car. Utilizing fisheye lenses, this provides the EVA with a total 360-caste view of the area immediately effectually the automobile. The iv stereo cameras are supported by a defended pair of CV1 SoCs, and use an assortment of Sony's 2MP IMX290, which characteristic very large pixels.

Rounding out the system are a forepart-facing radar for redundancy and driving in poor visibility, and a PC to run college-level sensor fusion, localization, and path planning tasks. In this demo video, you tin can meet the overall sensor layout on the EVA, along with what it sees (starting about twoscore seconds in), the 3D indicate cloud information technology generates, and the objects it recognizes while driving down a suburban street. The short- and long-range systems are fused so that the auto has a view all the way from the nearby curb to objects hundreds of meters away.

The CV1 and Now the CV2 Provide Both Paradigm and Vision Processing

Photo of Ambarella's newly-announced CV2 SoC -- 20x the performance of CV1I consequence with vehicle camera systems is that they tend to be optimized either for man viewing (dash cams, backup cameras, and bird'due south-heart view cameras) or computer vision (safety systems and autonomous driving systems). The CV1 and new CV2 SoCs is designed to allow cameras to perform both tasks, so only a single integrated system of cameras is required. In the EVA's photographic camera, it does this through defended Ethernet interfaces for object recognition information that goes to the vehicle systems, and more than-traditional video output via a companion H2 chip used for encoding.

The EVA exam vehicle was equipped with CV1 chips, simply the company has just announced a successor, the CV2, with up to 20 times more processing power, at around the same amazingly low power envelope of 4 watts. The visitor hopes that with the CV2 and some additional evolution, it will exist able to run the entire autonomous vehicle stack solely on its own fries, not even needing the PC currently running some of the organisation'southward functions.

Why Ambarella?

Ambarella is not a household proper noun to many, even those who follow tech. But you have almost certainly either used a camera organisation they designed, or been photographed by one. The company specializes in providing a broad range of silicon-based imaging and vision solutions for a number of industries, including drones, wearables, security, automotive, and VR. So it was logical for it to showcase the applications of its technology for cocky-driving cars. The attempt was jumpstarted with its acquisition of VisLab in 2022. VisLab and its founder Broggi were early pioneers in autonomous vehicles starting in the 1990s through their work at the University of Parma, and were engineering providers to the Terramax squad in the famous 2005 DARPA claiming before spinning VisLab out every bit a divide company in 2009.

The CV1 processors do object recognition and motion tracking in addition to creating a 3D point cloud as you can see from this real time view of the car's perceptual system

The CV1 processors do object recognition and move tracking in addition to creating a 3D point cloud every bit you tin see from this real time view of the machine's perceptual system

EVA Road Exam

The team from Ambarella and its subsidiary VisLab took me out for a test drive in the EVA demo automobile. I had a nice view of what the system was seeing as we drove via a defended monitor in front of my seat. It showed both the motorcar's own guess of its location along with GPS data for starters. More than interestingly, it also showed objects as they were discovered and identified. The existent-time visual computing was impressive. Stationary objects, moving objects including cars and people, and semantically important objects like stop lights were all quickly identified and reported.

Our EVA demo ride was designed to highlight several of the capabilities of its vision system

Our EVA demo ride was designed to highlight several of the capabilities of its vision system

The localization and path planning wasn't quite every bit polished. The GPS information wandered (despite the motorcar's utilize of high-fidelity GPS) and started to confuse the motorcar when information technology conflicted with its own data (based on placing itself visually on known maps). That acquired some weaving in our lane. That, plus some jitters in path planning, also caused us to have some trouble with tricky lane lines.

On the other hand, the vehicle'due south collision avoidance and safety systems worked actually well. We were able to merge safely onto a busy highway at a well-nigh blind on ramp, and when surprised by a car coming out of a parking lot our machine braked essentially instantly. Overall, since the core technology that Ambarella and VisLab specialize in is the vision system and processing, I was pretty impressed with that portion of the system.

I asked Broggi most night driving without lidar, and he made the reasonable point that just as with a human driver, the automobile'south headlights are designed to permit prophylactic forrard visibility, and most other objects moving towards the auto take their own lights. Subsequently all, humans are an existence proof of the possibility of nighttime driving without lidar or IR illumination. However, it could all the same exist a claiming to convince partners and ultimately safe regulators that autonomous vehicles can follow suit.