How Waymo Trains Driverless Cars: Virtual World + False City

Netease Technology News August 25, according to Atlantic Monthly reported that in the corner of Google's parent company Alphabet, a team is developing a software, it is essential for driverless cars. So far, no reporter has seen it. The R&D team called the software Carcraft, apparently inspired by the popular video game World of Warcraft.

Figure 1: Alphabet's ambitious driverless project

The developer of this software is a young engineer with hairy hair and a baby face. He is James Stout. He and I (the author of this article, Alexis C. Madrigal, deputy editor in charge of the science and technology column for the Atlantic Monthly) sit quietly in an open-plan office. A circular junction is displayed on the screen. For the human eye, there is nothing to see on these maps. Only simple line drawings are presented on the road texture background. We saw the driverless Chrysler Pacifica at medium resolution, with a simple wireframe next to it showing the presence of another car.

Figure 2: Carcraft, a simulation building software for Alphabet's unmanned subsidiary Waymo

A few months ago, a driverless car team encountered such a roundabout in Texas. In this case, the high speed and complexity puzzled the driverless car, so they decided to build similar physical channels on the test equipment. What I see is the third step in this learning process, namely the digitization of driving in the real world. Here, a real-world driving action (like a car at a roundabout) can be magnified thousands of times into a simulated scene to detect the edge of the car's capabilities.

This scenario provides the foundation for the company's powerful simulation tests. Stott told me: "The vast majority of the work we see is driven by simulations." It's Waymo's tool to accelerate the development of driverless cars. In December 2016, Alphabet separated the driverless project from the research organization X and became an independent business. If Waymo can launch fully-autonomous cars in the next few years, Carcraft, which aims to reshape the real world and create a virtual world, will do much.

The original intention of the development of Carcraft was as a "scene replay" means of car driving on the road. Now Carcraft plays an increasingly important role in driverless planning. At any time, 25,000 virtual unmanned cars are in motion in the city of Austin, Mountain View, Phoenix and other test sites rebuilt through Carcraft.

In just one day, Waymo could simulate thousands of driving in areas where road conditions are particularly complex. Today, the Waymo car travels more than 12.87 million kilometers in the virtual world every day. In 2016, their virtual total mileage reached 4 billion kilometers, while Google’s driverless cars traveling on real roads traveled 4.83 million kilometers. Crucially, virtual miles are concentrated in what Waymo calls "interesting places," and they may learn a lot from it.

These simulations are part of an intricate system developed by Waymo. Their driverless cars traveled millions of kilometers on public roads while undergoing "structured tests" in a secret base in the central valley called "Castle." Waymo has never disclosed this system. Their mileage on regular roads shows that in some places they need extra practice. They carved these places into the castle in order to experience thousands of different scenes in situ.

In two real-world tests, their cars capture enough data to create complete digital entertainment at any time in the future. In the virtual space, they can break away from the limits of real life, create thousands of variants of a single scene, and drive virtual cars through all scenes. With the improvement of the driver software, it is downloaded back to the physical car, which can drive more and more mileage. This process is repeated and repeated.

To get to the "Castle," you need to drive east from San Francisco Bay, but turn south to Route 99 and drive south along the Central Valley Highway to the small town of Atwater in Fresno. It is 30 degrees Celsius hotter than San Francisco. It used to be the Castle Air Force Base. At its most prosperous time, it employed 6,000 people to participate in the B-52 project. It is now located on the northern edge of the Merced metropolitan area, where the unemployment rate broke through 20% in the early 2010s and still rarely falls below 10%. Here 40% of people speak Spanish.

Leaving the town of Atwater, we crossed some of the railways and turned to the old bases that had been left. Now it has been transformed into the Merced Animal Control Center and the Atwater Prison. My phone did not point to a specific address but provided GPS coordinates. We walk along tall, opaque green fences until Google Maps tells us to stop. There does not seem to be anything here, and even the gate looks like another fence, but my Waymo master is confident. Sure enough, a security guard appeared, then slipped out of the crack and checked our documents.

Figure 3: Fences outside the castle

Through the fence part, we drove into a bustling small campus. Many young people in shorts and hats walk around. There is also a mobile building, a dome garage, and parking for driverless cars, the main goal of our trip. There are several types of driverless cars here, including the Lexus models you are most likely to see on the road, the retired Prius, and the new Chrysler Pacifica.

Driverless cars are easy to tell because they are full of sensors. The most prominent of these are laser scanners (often called LIDARs) on the top of cars. But near the side mirrors of Chrysler Pacifica, there are also smaller rotating LIDARs. Behind them are radars that look like Shrek's disturbing white ears.

When the car's sensors are put into use, rotating LIDARs can make strange noises even when they are parked. It is somewhere between mourning and thumping, simply because it is so novel that my ears can't automatically filter out the usual sounds that cars make. In the street opposite the main building, there is a more special car parked. It is entangled with different sizes of red tape printed with the X logo, which is a sign of the Class IV car.

The classification of autonomy in driverless cars was developed by the Society of Automotive Engineers. Most of the cars we see on the road belong to class one or two cars, which means that they can perform intelligent cruise control on the freeway. However, the red X car is completely different. It is not only fully automated, it cannot be driven internally, so Waymo does not want to mix it with other cars.

When we drove into the parking lot, we could not help but have the impression of the “Manhattan Project”, which is the frontier of technology startups. In a classroom-sized room in the main building, I saw the driving force behind this magical place, Steph Villegas. Villegos is wearing a long, very fit white T-shirt, sturdy jeans, gray knitted sneakers, still as fashionable as she did before she joined Google in the boutique boutique Azalea in San Francisco. Villegos grew up in the suburbs of East Bay near Berkeley, California, studied fine arts at the University of California, Berkeley, and joined Google’s driverless car project in 2011.

I asked, "Are you a driver?" Villegos responded by saying, "I will always be a driver." She spent countless hours between Highways 101 and 280. These highways lead to San Francisco and the mountains. King City. Like other drivers, she began to feel a sense of the cars driving on the open road, which was considered very important in the driverless plan because they had a very keen sense of what the car might encounter. Villegos told me: "After doing some testing on newer software and staying in the team for a long time, I started thinking about ways to challenge the existing system."

To this end, Villgues and some engineers began to prepare to try to find a controlled way to test new behavior. They began to occupy the Shoreline Amphitheater car park and all entrances were staffed to ensure only authorized Googlers entered. Villegos said: "This is where we started. I and several drivers try each week. We will come up with a set of things we want to test, fill the truck with supplies, and then drive the truck to Parking lot for testing."

These became the first structured tests for driverless projects. Facts have proved that the most difficult part is actually not the kind of "zombie doomsday" scenario that people think. It is like human drivers experiencing endless changes in normal traffic. Villigos began collecting props from anywhere she could find: dummy, cones, botanicals, children's toys, skateboards, tricycles, dolls, balls, and other gadgets, and put them all in the prop box. The props were first stored in tents, then in the castle. Now there is a full storage room.

Figure 4: "props garage" in the castle

However, there are many problems in this process. They want to drive the car faster and use street lights and stop signs. The Shoreline Amphitheater often organizes concerts and often disrupts their plans. For this they need a base and a secret base. This is what the castle can provide. They signed a lease and started building their dream city. Villegos said: "We decided to design residential streets, highways, cul-de-sacs, car parks and other infrastructure, so we are like driving in a real city."

We walked to her car from the main trailer office. When we were about to leave, she handed me a map and said: "Like Disneyland, you can walk along the map." The map is also well-drawn. In one corner, there is a Vegas-style sign that says: "Welcome to the mythical castle in California." Different parts of the park even have their own naming conventions. In the place we are passing through, each road is named after a famous car.

We passed through several pink buildings. They are old military quarters and one of them has been transformed. When Waymo employees cannot return to the Bay Area, they can rest here. In addition, there are no other buildings in the test area. It is indeed a city of robot cars.

Figure 5: "Neighbors" in the castle

As an outsider, it feels like a video game scene without players. From the boulevards to the streets of neighboring communities, from the cement driveway to the suburban crossroads, and minus the buildings we associate with these places, all this seems incredible. I continued to glimpse the road I had walked through and ended up on the huge two-lane circular road. In the middle, there is a white fence. Villegos said: “We had a multi-lane roundabout in Austin, Texas, and we installed this kind of roundabout here. Initially there was only a single-lane roundabout. At the intersection, it looks like a horse of different colors, thanks to Texas for inspiration."

Figure 6: Two-lane roundabout

When Villegos stared at the new facility – two lanes and a bicycle lane built along a parallel parking lot adjacent to the turf, we stopped. She said: “I really like to build new facilities along parallel parking lots. Similar scenes have already appeared in the suburbs of the commercial area, such as Walnut Creek, Mountain View, and Palo Alto. People from the store or park Come out and walk between cars, and you can also carry things across the road.” This path is very much like the debris in the memory of Villguez, especially the memory embedded in asphalt and concrete, which will become more abstract. Forms help robot cars improve their ability to adapt to the terrain of the home.

Villegos drove me back to the main office. We hopped into a driverless car and it was modified by Chrysler Pacificas. Our "Left Seat" driver is Brandon Cain, who tracks the car's performance through the XView software on the laptop. There are other test assistants who are called "foxes" and evolved from the word "man-made." They drive cars, create traffic, act like pedestrians, ride bicycles and carry parking signs. They can be more or less called actors, and their audience is cars.

The first test we have to do is "simple overtaking and merging," but it only needs to be completed at high speeds, at a speed of about 72 kilometers per hour. We go straight on the wide road called Autobahn. When the "fox" stopped us, the Waymo car braked, and the team started the critical data point, the deceleration process. They are trying to create a scene that allows the car to brake in time. How hard is it? It's like stopping the light, stopping my armpits from sweating, or stopping the phone from falling to the floor.

Let me say something funny: This is not my first time driving a driverless car. In the past, I chose two different drones: the first time I drove a Lexus buggy through the streets of Mountain View, and the second time I drove Google Firefly to “dance” on Google’s roof. They are all inconspicuous rides, but this time it is different. This time involved two fast-moving cars, one of which needed to be stopped by what the Waymo team called “spicy”.

The test began, Kane started the car and whispered to enter the "autopilot" mode. Another car approaches us and tries to block us. Our car braked quickly and steadily, which left a deep impression on me. Subsequently, Kane et al. examined the deceleration figures and realized that our brakes were not doing well enough. We must do it again and repeat it again and again. Another car uses different methods to block us from different angles. They call this kind of test “coverage”.

Figure 7: Two cars drive at high speed and one of them is a driverless car

We have experienced three other tests: high-speed parallel roads, encountering cars that are retreating in the driveway, and the third one is dominated by the sight of the driverless car. When the pedestrians throw basketball on the road, the car smoothly. Slow down the brake to stop. Each test was impressive in its own way, but the blocking test was the most shocking to me.

When we decided to continue experiencing the driverlessness, Kane changed his seat. He asked me, "Have you ever seen Pacific Rim?" It was Guillermo del Toro's movie. People inside were fighting monsters by synchronizing with giant robots. He said: "I try to keep pace with the car and we share some ideas."

I hope Kane can explain what it means to "synchronize with the car." He said: "I'm trying to adjust the difference in people's weight in the car. I often stay in the car and can feel what the car is doing. It sounds weird, but I can really feel it on the hip and I know it wants to do it. what."

Stay away from the dusty castle and come to Mountain View’s comfortable Google headquarters. I visited Waymo's engineers. From a technical point of view, they belong to the X department, Google's long-term, high-risk research department. In 2015, Google was removed from its name when Google reorganized itself into the holding company Alphabet. In the year following the reorganization, X and Alphabet decided to split the driverless car project and establish an independent company, just as Google’s other previous projects also became independent businesses.

Waymo is like a Google kid, and its office is still inside the mothership. Although this sector consists of two small groups, they are slowly becoming integrated. The X/Waymo building is large and well-ventilated, with Project Wing's drone prototype hanging. Here, I caught the clues of the company's development of the Firefly car.牬 牬 牬 梢 梢 梢 梢 梢 绗 绗 绗 绗 模拟 模拟 模拟 模拟 模拟 模拟 模拟 模拟 模拟 模拟 模拟 模拟 的 的. Here, everyone seems to have Carcraft and XView on the screen, and black background polygons abound. These people created the virtual world that Waymo cars passed through.

Figure 8: What does the laser scanner on the Waymo car show when four people are pushing a car?

The person waiting for me is Carst creator James Stout. He never talked about his project publicly, but his enthusiasm never faded. Carcraft is like his child. He said: "I was browsing recruitment information at the time and saw that the driverless car team was recruiting people. I couldn't believe that they were recruiting people." Stott successfully entered the team and immediately began to develop this tool, now It supports driverless cars driving in the virtual world every day.

At that time, they mainly used this tool to observe what the car would do under difficult circumstances. In similar circumstances, human drivers had already controlled the car and they began to make various assumptions. Stott said: "Obviously, this is a very useful thing, we can do a lot of things in this area." Carcraft's space is expanding, even covering the entire city, the number of virtual cars is also increasing.

Stott finds Elena Kolarov, who is the person in charge of the "Situation Maintenance" team responsible for managing the controls. There are two screens in front of her. The left screen runs XView, showing what the car "sees". The car uses cameras, radar and laser scanning to identify objects in its field of vision. These objects are represented in the software by a simple wireframe shape and are easy to recognize. The green lines extending from the shape of the object represent possible ways of moving. At the bottom, there is an image bar showing the scene captured by a regular camera on the car. Kolalov can also turn on the data returned by the laser scanner, which is displayed in orange and purple dots.

We saw the replay of the true parallel scene at the roundabout of the castle. Kolalov converted to an analog version, they look almost no difference, but it is no longer a data log, but a new situation that the car must solve. The only difference is that at the top of the XView screen, it shows "simulation" with red letters. Stott said that they must give a supplementary explanation because it is difficult for people to distinguish the difference between simulation and reality.

Figure 9: Castle roundabout seen in XView simulation

They loaded another scene, this time in Phoenix. Kolalov enlarged the simulation to show that it is a city. Stout said that there are all kinds of lanes inside, there are connections between lanes, parking signs, traffic lights, restricted areas, lane center locations, and everything you need to know.

Figure 10: Waymo's car model travels in the virtual city of Chandler, which is located near Phoenix, Arizona

We magnified a 4-car intersection somewhere in the vicinity of Phoenix. Then Kolalov started putting on synthetic cars, pedestrians and bicycles.

Figure 11: Composite scene created in Carcraft

Click on the hotkey and the object on the screen will begin to move. The car moves like a car, driving in the lane, turning. Cyclists are like real cyclists. After the driverless car team manipulated the driverless cars and traveled millions of kilometers in the real world, their logic had formed a fixed pattern. Behind all this, there is an ultra-detailed map of the world, as well as different physical models at the site, rubber and roads are modeled.

Figure 12: Scene being simulated in Xview

Not surprisingly, the hardest thing to simulate is actually the behavior of others. Waymetric's software director Dmitri Dolgov told me: "Our car sees the world and we understand the world. Then, for any object that plays a dynamic role in the environment, whether it is a car or not Pedestrians, cyclists, or motorcycles, our cars can understand their intentions. It's not enough to track an object through a space, but you must understand what it is doing. This is to build a competent, safe The key to driving a car. This kind of modeling, this understanding of the behavior of other participants in the world, is very similar to the simulation of them in this task."

There is a key difference: in the real world, they must accept fresh, real-time data about the environment, translate it into understanding the scene, and then navigate. But now, after years of research, Stout and others believe that they can do this because they have run "a series of tests" and proved that they can identify a variety of pedestrians.

Therefore, in most simulations, they skipped the object recognition step, did not provide the car with the original data to identify pedestrians, but directly told the car that these were pedestrians. At the intersection, Kolalov was even more difficult for driverless cars. She clicks V, which is a hot key for the vehicle. A new object appears in Carcraft. Then she moved her mouse to the drop-down menu on the right hand side, where there are many different models, including my favorite bird_squirrel.

Different objects are told to follow the logic that Waymo has modeled them and need to move in a precise way in the Carcraft-built scene to test specific behaviors. Stott said: "There is a good spectrum, you can control a scene, then put objects in and let them go." Once they have the basic structure of the scene, they can test all the important variables it contains . So, imagine that for the intersection, you might want to test the arrival time of various vehicles, pedestrians and cyclists, the time they stopped, the speed they moved, and everything else. ”

They call this model "fuzzing." In this case, this intersection can produce 800 possible situations. It creates beautiful lace charts, and engineers can go in and see how the combination of different variables has changed the car's decision to take the path.

Figure 13: Carcraft's "fuzzing" chart

This problem has really turned into an analysis of all these scenarios and simulations to find interesting data that can guide engineers to better drive. The first step may be: Will the car be hit? If it is, it will be an interesting scene. The following figure shows this situation. In the real life of Mountain View, the intersection scene is more complicated. When the car was driving to the left, a bicycle approached, causing the car to stop on the road.

The engineers solved this problem and redesigned the software to correct it. The picture shows the real situation, and then simulate the operation. This means that in two different situations, you will see the simulated car continue to drive, and then the dotted box that says "shadow_vehicle_pose" appears. This dotted box shows what happens in real life. For the Waymo team, this is the clearest visual progress.

Figure 14: Waymo simulation shows improvements in vehicle navigation systems

But they are not just looking for details when the car is hit. They may also want to look beyond the correct range for long-term decisions or the effects of sluggish brake response. If engineers want to learn from them, they will look for problems through simulation. Doergof, director of software for Stott and Waymo Software, emphasized that there are three core aspects of simulation. First, they drive far more mileage than physical teams in the real world, and get more and better experience. Second, these cars focus on miles that are interesting and still interactive, and do not pay attention to boring miles. Third, the software development cycle can be faster.

Dolgov said: "This iterative cycle is very important to us and all the work we did during the simulation allowed us to dramatically reduce the time. In the initial phase of the project, this cycle may take several weeks, but now only It will take a few minutes.” I continued to ask him, are there any oil films on the road or punctures, strange birds, tiankeng and other crazy things that they can all simulate? Dolgov is optimistic about this, saying It's already possible to simulate, but the key is how high the fidelity of these simulations is? Maybe there are issues where you get better value, or you need to run a series of tests to confirm the scene in the physical world in the simulation.

The power of Carcraft's virtual world is not that they are real-world beautiful, perfect, realistic renderings, but that they reflect the real world in many ways. This is very important for driverless cars. It can Gain billions of kilometers more than physical tests. For driving software running in simulation, although it is determined differently from the real world, it is the same decision made in the real world.

This method is effective. The California Motor Vehicle Authority asked the company to report their annual unmanned mileage and automated tests that were carried out by the driver. Waymo not only has three orders of magnitude more mileage than anyone else, but the number of human driver interventions is also rapidly declining. From December 2015 to November 2016, the Waymo car drove 1.02 million kilometers in unmanned mode. In all mileage, the driver only participated in 124 times, and once every 8000 kilometers. In the previous year, these cars had traveled 680,000 kilometers automatically, but human drivers intervened 272 times, an average of 1,432 kilometers.

Although everyone has painstakingly noticed that these are not the same digital comparisons between Apple and Apple, let's take a look at the reality: these are the best comparisons we have, at least in California. In total, other vehicles traveled approximately 32,000 kilometers in automatic driving mode. Waymo's strategy is not surprising for outside experts. Chris Dixon, head of investment at venture capital firm Andreessen Horowitz, said: “Now, you can measure the maturity of an unmanned team almost by treating the seriousness of the simulation. Waymo is at the top of the list. It is also the most mature."

I asked Sunil Chintakindi, innovation director of Allstate Insurance Insurance, about the Waymo project. He said: "If there is no strong simulation infrastructure, we cannot build (higher-level) automated vehicles." Researchers who drive cars are also looking for similar paths. Huei Peng, director of Mcity, an automated and networked automotive laboratory at the University of Michigan, said that any system used for driverless cars will be "more than 99% simulation + well-designed structured testing + highway testing."

Peng Hui and a graduate student proposed a system that can speed up testing through simulation, which is not different from Waymo's implementation. Peng Hui said: "So what we're debating is just to cut off the boring part of driving and focus on the interesting part. This can make you speed up hundreds of times, and 1,000 kilometers into millions of kilometers."

What is surprising is the size, organization and intensity of the Waymo project. I described to Peng Hui the structured tests conducted by Google, including the 20,000 simulation scenarios designed by the Castle Structured Test Team. But he got it wrong and said: "The 2,000 scenes are impressive." When I interjected and corrected him, he replied, "Indeed, 20,000 scenes. It's really impressive." 20,000 scenes represent only a small part of all scenes tested by Waymo. They are all created by structured tests. There are more scenes from public driving and imagination. Peng Hui said: "They have done really good. They are far ahead of everyone else in the four-class field."

However, Peng Hui also emphasized the position of traditional car manufacturers. He said that they are trying to do something completely different, rather than aiming for a fully automated “moon shot (crazy and hard to achieve project)”, they are trying to increase driver assistive technology, “make money,” and then gradually move toward full automation. . Comparing with Waymo is unfair. It is rich in resources, installing a $70,000 laser rangefinder in a car, and car manufacturers like Chevrolet may see that $40,000 may already be mass market. The price cap used.

Peng Hui said: "GM, Ford, Toyota and other auto companies are saying: 'Let me reduce the number of car accidents and fatalities and increase the safety of the mass market.' They are completely different from Waymo's goal and we need to consider millions of vehicles. Cars, not just a few thousand.” Even in a fully automated race, Waymo now has more challengers than ever, especially Tesla.

18 months ago, Chris Gerdes, director of the Stanford University Automotive Research Center, once said that Waymo has a deeper understanding of the depth of the problem. Compared with other people, we are closer to the solution to the problem. When I asked him last week if he still thinks so, he said: “A lot of things have changed. Automakers such as Ford and General Motors have been developing their own cars and have built road data sets. Tesla has now passed Deploying Autopilot collects large amounts of data to understand how the system operates under its customer experience.They can test algorithms in silent mode and quickly expand the number of test vehicles, making them an astonishing testbed.

In the simulation field, Geddes said: “I have seen many competitors with substantial projects. I am sure there is quite a lot of simulation capabilities, but I have seen many things that seem trustworthy. In this regard, Waymo It doesn't look so unique anymore. Of course they started very early, but now many groups are using similar methods. So the most important question now is who can do the best."

This is not a low-risk demonstration of the “brain type” capabilities of neural networks, but a giant leap forward in the field of artificial intelligence, even for intra-company companies that have always been very active in adopting artificial intelligence. This is not Google Photos. If you make mistake here, the consequences are not too serious. And for an active and fully interactive system in the human world, it will understand our rules, convey its cravings, and let our eyes see them.

Waymo seems to want to use driving as a skill, speed and direction are some of them. They are now integrating driving into human social activities. Usually, how to drive is considered “legal”? How does humanity teach artificial intelligence what it stands for? It turns out that the development of such artificial intelligence does not only require endless data and engineering techniques. These are necessary but not enough. On the contrary, the development of this artificial intelligence requires humans to synchronize with the car and understand the world. Although anyone can do it, the drivers in the castle know that as a car, they must observe and make decisions like humans. Perhaps there are also two ways: The deeper the human understanding of the car, the deeper the car's understanding of mankind.

The memory of Austin's roundabout became part of the castle, becoming a data log of driverless cars, turning into a scene in Carcraft, becoming a simulation network, and eventually returning to the Texas driverless car. New software. Even in the simulation, the AI ​​is used to understand the polygon abstract image of the world, there are traces of the human dream, fragments of memories, the feeling of the driver. These elements are not mistakes, nor are they tainted by humans. They are the necessary components of systems that can radically change traffic, cities, and everything else. (small)