The Russian policeman waved at the orange van zigzagging around the empty plaza, ordering it to stop. The van didn’t, so the officer stepped closer to address the driver. But there was no driver.
The van is an autonomous vehicle developed at the University of Parma’s Artificial Vision and Intelligent Systems Laboratory, known as VisLab. Crammed with computers, cameras, and sensors, the vehicle is capable of detecting cars, lanes, and obstacles — and drive itself.
The VisLab researchers, after getting tired of testing their vehicles in laboratory conditions, decided to set out on a real-world test drive: a 13,000-kilometer, three-month intercontinental journey from Parma to Shanghai. The group is now about halfway through their trip, which started in July and will end in late October, at the 2010 World Expo in China.
The autonomous vehicle Grand Challenges organized by the Defense Advanced Research Projects Agency, or Darpa, in the United States, popularized the idea of cars that drive themselves. After participating twice in the Darpa competitions, VisLab was seeking new ways of testing their ideas.
“When you do things in the lab, it all really works. But when you go out in a real road, with real traffic, real weather, it’s another story,” says Alberto Broggi, VisLab’s director and an engineering professor at Parma University.
That’s how the idea for their Parma-Shanghai trip, which is partially funded by the European Research Council, came about. The goal is to test, and later perfect, their vision and navigation systems, which the researchers hope to one day deploy on commercial vehicles.
Unlike the Darpa vehicles, the VisLab van is not driving fully autonomously from start to finish. That approach would require at least a rough map of the complete route, Broggi says, adding that they lack maps for locations like Mongolia and Kazakhstan. So instead of programming the trajectory ahead of time, the Italians adopted a simpler, though still technically challenging, approach.
Two vans travel in line. The first uses maps and GPS to drive itself whenever possible, but a human driver is in control most of the time. The second van uses its cameras and navigation system to follow the first; it visually tracks the lead van, plans a trajectory in real time, and generates controls for steering and accelerating or braking. (If a car gets in between the two vans, the second van guides itself using GPS instructions it receives from the leader.) Watch:
[youtube]http://www.youtube.com/watch?v=cIjR06TjaNg[/youtube]
And whereas the Darpa vehicles ran unmanned, the VisLab vans have people on board (including, in one occasion, hitchhikers). Inside, researchers monitor the systems and are ready to take control of the vehicles if necessary. The vans only drive with no one inside during demonstrations. That was the case when it was stopped by the police officer in Yekaterinburg, in central Russia.
Broggi insists they are not attempting to prove anything, just trying to “stress our systems.” And he acknowledges that even the second van is not driving autonomously the entire time. There were situations when they had to take control because of maniac drivers or distracted pedestrians, or because they had to drive faster to compensate for delays. Still, he estimates that on average the van is in autonomous mode more than 90 percent of the time.
The compact vehicles carry a lot of equipment. Each has seven cameras, four laser scanners, GPS unit, and inertial sensor suite. Two cameras hanging above the windshield provide stereo vision, used for identifying lane markings and the terrain slope. Three synchronized cameras behind the windshield stitch their images into a 180-degree panoramic frontal view. The laser scanners — three mono-beam and one four-plane laser beam — detect obstacles, pedestrians, and other vehicles, as well as ditches and bumps on the road.
Each vehicle also carries three computers. One computer is responsible for processing images and data from the front of the vehicle. Another computer handles data for the sides. The third integrates all the data and plans a path, which in turn triggers low-level controls for steering, accelerating, and braking the vehicle. Solar panels on top of the vans power all the electronics.
A key piece of software is the one that processes the 180-degree frontal view. This component takes the large panoramic image and identifies the lead van, even when approaching a tight turn or steep hill. It can also detect road markings, pedestrians, and obstacles in general. And though it can recognize Italian traffic signs, it can’t do the same for those in Russian or Chinese.
The vans are manufactured by Piaggio, better known for its Vespa scooters. The vans are fully electric, and the researchers drive them in the morning, recharge in the afternoon, drive some more, and recharge again overnight. They use power outlets along the way or, when none is available, diesel generators.
A group of 20 staff researchers and students travel in a convoy that includes four vans (two pairs of leader-follower vehicles) and six support trucks, which provide a mechanic shop, storage, accommodation, and satellite communications. Every three weeks a few team members return to Italy to rest and others join the convoy.
The experiment will produce a mountain of data. Whenever the vehicles are running, the computers are recording data from the cameras, laser scanners, inertial sensors, GPS, vehicle actuators, batteries, and other systems. Every night the team replaces a set of hard drives with empty ones. They estimate they’ll end up with between 50 and 100 terabytes of data.
The idea is that after the test is over, the researchers can use the data to study every instance when things didn’t work, such as when the vehicle failed to detect lanes or misidentified an obstacle. Moreover, they can design new vision and navigation systems and use the data set to test them, recreating their journey.
So the end of the trip is not the end of the work for the VisLab team. “It will be the beginning of many new projects,” Broggi says. Indeed, they’ve already done one major upgrade during the trip. At the beginning, there were problems with the second van, which was failing to follow the lead van, and they had to tweak their software and re-upload it to the vehicles.
Even with all the planning, some problems are unpredictable. During a recent stretch of the trip, the convoy found itself in the middle of the notoriously bad traffic of Moscow. Because of the congestion, a two-lane road had three rows of cars. The van’s system insisted in staying on its lane, so the researchers had to turn to manual driving.
“It was too dangerous to drive lawfully!” Broggi says with a laugh.
And if the technology is challenging, the bureaucracy can be even more overwhelming. VisLab, a top research group in autonomous vehicles and intelligent transportation systems, first proposed the project to the Italian government years ago; the researchers never heard back. The project moved forward when they teamed up with Overland, an expedition organization, which handles logistics, including obtaining permissions to enter Russia and China carrying high-tech cameras and satellite equipment.
In fact, to cross the Russian border the group was held for 22 hours by custom officers, who took a huge number of pictures of the vehicles and the equipment and demanded a pile of paperwork. The Russians also insisted in having access to every data packet sent over a satellite connection.
The VisLab group is currently crossing Siberia, facing rain, snow, and icy roads. Next they will encounter even tougher conditions, a stretch of mountainous driving in Kazakhstan, with treacherous hills and roads, which will last for several days.
As for the Russian police office who stopped — or tried to stop — the driverless van, the researchers say they approached him and explained the situation. The officer wanted to see documents, though he didn’t ask for anyone’s driver license. In the end, he didn’t look amused, but he left — without issuing a ticket.
