“Here it comes,” I call out to photographer William Walker. “Yeah, there’s no one driving.”
As Walker’s shutter rapid-fires and the upfitted Chrysler Pacifica plug-in hybrid (PHEV) pulls up to where I’m standing on a residential cul-de-sac in Chandler, Arizona, I realize my mistake. There’s no one in the driver’s seat, but someone—something—is definitely driving.
We’ve traveled from Los Angeles to the Phoenix suburbs to experience the type of vehicle that might just put us automotive journalists out of business: the Waymo One. You might remember it as the Google Self-Driving Car Project we first experienced in 2015, but it’s now an independent company wholly owned by Google’s parent company, Alphabet. We’re here to answer two fundamental questions: How well does it work, and how the hell do we review cars we can’t drive?
The Chrysler Pacifica-based Waymo autonomous taxi vans you can hail today in Arizona are fourth-generation vehicles. The fifth generation is based on the battery electric Jaguar I-Pace, and its hardware and software are more sophisticated and the hardware is downsized. But fifth-generation vehicles are presently only open to Waymo employees.
Each Pacifica is outfitted with 19 cameras, one long-range and one midrange lidar sensor mounted on the roof, four short-range lidar sensors, and six radar sensors. (You can order a Pacifica from the factory with radar, camera, and ultrasonic sensors that provide advanced driver aids, but Waymo elects not to use any of that equipment.) All the data from the sensors is fused together into one stream along with high-definition map data, and it’s processed by a computer stack in the van’s cargo area.
Critically, Waymo doesn’t buy hardware or software from suppliers. The company builds its own sensors, computers, and software. It produces its own high-definition maps of roads, parking lots, buildings, and more in its service area; it does not rely on Google Maps.
Inside, Waymo repurposes the Pacifica’s rear-seat entertainment system to display the route, a bird’s-eye view of what the vehicle “sees” around it, a way to contact Rider Services with questions, and to start or end the ride (which can also be done from a phone app). Hard buttons on the ceiling offer redundancy, and a ceiling-mounted camera is used only between rides to check for lost items or a mess that needs cleaning.
Waymo won’t put a price tag on one of its vehicles, but it says the fifth-gen version costs about half as much as gen four. Following Tesla CEO Elon Musk’s advice that “building cars is hard,” Waymo no longer has any intention of building its own as it once suggested it would. Whether building Waymo One vehicles or potentially licensing its technology in the future, the company will upfit vehicles from existing automakers.
Because the Pacificas retain their steering wheels and pedals, they’re considered Level 4 autonomous vehicles. The Society of Automotive Engineers, SAE, defines autonomous driving on a six-point scale, with Level 0 being a car with no driver aids whatsoever and a Level 5 vehicle having no steering wheel, pedals, or any other means of manual human driver control.
This will sound anticlimactic, but using the Waymo One taxi service is barely different than using Uber, Lyft, or any other ride-hailing option. That’s by design. Since the early days, Waymo’s goal was to make its service and vehicles as approachable and unintimidating as possible.
Like any other service, you download the app, give it your payment info, and hail a ride. Unlike the rest, Waymo’s only works if you’re within the roughly 50-square-mile territory covering most of Tempe, Chandler, and Mesa, Arizona (expanding soon to downtown Phoenix). Also unique to Waymo is the ability to link your Google account and play music, podcasts, and books through the car using the Google Assistant app.
Otherwise, you punch in where you want to go, agree to the price, and wait for the van to arrive. You can adjust your pickup and drop-off points, and the One will do its best to get close to them, but there’s an ever-present warning you may have to walk a bit. You can add or change destinations during the ride, which may increase the price. If you enter multiple destinations, the vehicle will either wait for you nearby or another vehicle will be sent when you tell the app you’re ready to be picked up.
In our experience, the app was easy to use but a bit buggy running on an older iPhone. The map, in particular, tended to get stuck, especially when trying to change the pickup or drop-off point. Adding and changing destinations mid-ride also caused the app to get stuck in a loop on several occasions, with the updated itinerary never making it to the vehicle, causing us to continue to the original destination.
The Waymo One isn’t the first fully autonomous vehicle I’ve ridden in personally, but it’s the first on public streets with no human in the front seat. I share this because my experience might make me more comfortable with the technology than someone who’s never experienced it, though Waymo Group product manager Chris Ludwick says my reaction was typical of most riders regardless of prior experience.
That reaction, in short, was temporary bewilderment followed by a surprisingly quick onset of complacency. The Waymo One drives itself well enough that the novelty wears off in a matter of minutes and you’re soon tempted to look at your phone or out the window, or do anything besides scrutinize the autonomous driving.
We’re here to scrutinize, though. What we discovered is, even though the vehicles are all ostensibly the same, they didn’t all drive identically. Some drove in a manner best described as casual, while others drove like a cab or rideshare driver in a bit of a hurry to get you out and get to their next fare.
Going in, we expected the vehicles would drive conservatively and cautiously, timidly even. Instead, the driving was more humanlike than predicted, though with a strict regard for the posted speed limit. Acceleration ranged from casual to a bit aggressive, as did steering inputs. Some lane changes were casual, and some were a bit abrupt, particularly when entering dedicated turn lanes. Ludwick says the fourth-generation vans like to get over into the lane immediately while the fifth-generation SUVs are programmed to do so a little more gradually and smoothly.
Braking was almost always early enough to slow gradually and smoothly. The only exception was when the system thought a pedestrian or cyclist might be in the way. Photographer Walker triggered one van walking out behind our parallel-parked truck to get to the driver door, and a cyclist clearly in their lane and not deviating concerned the same van enough to slow down and follow the cyclist for several seconds before eventually passing.
On other occasions, the Waymos were surprisingly assertive. One saw the light change as we were approaching the intersection at 45 mph, hesitated a moment, then accelerated to make the light. Another followed a city bus that began to pull off into a stop and the van cheated to the left side of the lane to go around it rather than wait for it to pull over all the way.
Even unexpected obstacles didn’t faze it. Rounding the corner in one neighborhood, we came upon a pile of gravel dumped in the street for someone’s yard project. The van went right around. Late in the day, we were driving directly into the setting sun, and the Waymo was slowing for speed bumps we couldn’t see through the glare with human eyes until we were right on top of them.
To really test it, though, we needed a tougher environment than suburban-grid streets. We needed as many crash dangers as possible. We needed the Costco parking lot. Cars zooming around frantically searching for open spots, people with and without cars wandering into the lane without looking, vehicles stopping in the middle of the lane to pick up bulky items, you name it. The Waymo One handled it all like a pro. It was certainly slower and less assertive than you or I might be in the same situation, but we had no emergency stops or questionable moves.
After seven rides spanning 12 hours (many of them making the poor van go in circles by adding stops), our Waymo rides cost an average of $2.06 per mile. Trips ranged from less than a mile to just more than 15, taking as little as 1 minute and as many as 43, and cost anywhere from $4.99 for a 1-mile, 6-minute trip to $23.14 for a 15.5-mile, 43-minute trip two towns over.
It occurred to me this type of scrutiny wouldn’t be the future of the automotive evaluation business. It’s a bit like looking over the shoulder of a teen driver: You know some situations might catch it off guard, and were it a person, we’d be far more forgiving. If someone made a tally of every driving decision any one of us has made and they disagreed with, objectively or subjectively, we’d have an equally long list.
Sufficiently advanced autonomous driving, such as in Waymo’s case, renders driving suburban grids unilluminating. As challenging an environment as urban areas are for autonomous vehicles (especially compared to relatively controlled and predictable freeways), the company seems to have the suburbs mostly in hand.
It isn’t difficult to identify more difficult environments, though. Waymo One runs 24 hours a day without human drivers, but as soon as it rains, the humans are back in the front seat keeping a watchful eye and ready to intervene. Snow? Yeah, right: Even the fifth-generation vehicles are still learning to cope with heavy fog in San Francisco.
For as much a hurdle as weather presents, the seemingly forever problem is edge cases. These are what programmers call rare, unpredictable events the software has never seen before and thus doesn’t know how to handle. Waymo claims millions of miles of driving on public roads and billions of miles via simulators, but the real world constantly creates new edge cases, and the simulators can only simulate what they’re programmed with. One workaround: When a Waymo One vehicle isn’t sure what to do, such as in the case of construction or lane closures or a person directing traffic or where the map isn’t accurate, it automatically connects with a human on the Fleet Response team who provides guidance to help it make the right decision.
Naturally, Waymo’s working on as many edge cases as it can think of. The vehicles can recognize and follow the commands of a construction worker or police officer giving hand signals. They know how to deal with construction zones and lane closures. The company is working on enhancing body language detection to better predict what pedestrians and cyclists are going to do. All of it, though, is a moving target.
A future challenge for us at MotorTrend will be designing an autonomous vehicle test regimen. We tricked the Waymo into standing still by keeping a door open (though we had to explain ourselves to the helpful people at Rider Services more than once), but getting it to stop exactly where we wanted for a photo was challenging at best. Creating a test loop an autonomous vehicle is guaranteed to follow exactly is nearly impossible because the rider doesn’t choose the actual route to a destination. Attempting an autonomous vehicle comparison test looks even more difficult, as repeatability is crucial to guarantee each vehicle is evaluated in the same conditions by every one of our judges. These robots, though—they only listen when we tell them where we want to end up, not how we want to get there.
We’ll be driven across that bridge when we get to it. To get a better sense of what we might look for when evaluating future autonomous vehicles, we asked the guy who runs the program how he does it.
“Keeping in mind that it’s still fairly early days for autonomous vehicles,” Ludwick said, “we tend to evaluate our fully autonomous Waymo One service in four categories: safety, comfort, convenience, and delight. So if you boil that down to one question: Is it safe, comfortable, convenient, and fun?”
Ludwick defined safety in terms of not only the raw number of crashes (47 during 10 years, most of which involved mistakes by other human drivers and road users in some way) but also the company’s approach to safety and transparency. Even California, which has far stricter regulations than Arizona, only asks for voluntary reporting. Waymo, for its part, releases both crash data and white papers explaining how it approaches safety.
“In terms of comfort,” he said, “this is where you evaluate driving behavior. How does the consumer feel while riding in one of our cars? We use a number of metrics to evaluate what we think will be comfortable, receive feedback from our autonomous specialists during testing, and ultimately seek feedback from our public customers on areas related to smooth driving, such as acceleration and deceleration, turning, lane changing, and more. Riders can leave us star ratings at the end of every trip and provide freeform feedback on what went well and what could be improved.”
The company doesn’t always get it right. In the early days, Waymo programmers quickly found that what looked like smooth and comfortable braking on a spreadsheet did not translate at all to the real world. Human feedback from riders was critical.
“For utility,” Ludwick said, “we measure things like how intuitive our app is to use, how is the quality of our pickups and drop-offs, how are our ETAs and routing times, how quickly our Rider Support team responds to inquiries, and more, to evaluate the overall usefulness of an AV service.
“Lastly, a fully autonomous ride should be fun,” he continued. “Consumers should be excited to get inside one of these vehicles and enjoy the ride. We’ve incorporated a number of features to bring this type of delight to riders, whether it’s letting them cast their favorite playlist or podcast directly on the in-car screens or adding customizable ID to each car so riders can find their specific Waymo ride.”
With vehicles that literally drive themselves, you’d expect there to be fewer things to manage, but that’s not the case at all.
First and foremost is fueling. Gas pumps and EV chargers don’t plug themselves in. Right now, Waymo’s Fleet Response team handles that, as well as rescuing any vehicle that gets itself into a situation it can’t get out of or if it breaks down. As the company grows, Waymo will need to expand that team alone to thousands across the country and beyond, or contract with local third-party companies.
Fleet Response is also responsible for cleaning vehicles in the field. Using the cabin camera, fleet dispatchers can see if someone’s left a mess, but a human still must deal with it.
Then there’s the matter of what to do with Waymo Ones when there are no customers. Sometimes they’ll find a place to park on the side of the road, but they can also be directed to drive a test route or collect map data to send back to the operations center. Waymo fleet dispatchers work to match the supply and demand curves so as few vehicles as possible are waiting around—and so half the fleet isn’t off getting fueled during a high-demand period.
What about service? Waymo needs a facility in every market where it operates where technicians can maintain and repair the vehicles. This becomes increasingly expensive the farther you get from major metro areas where there are fewer riders to pay for all this infrastructure.
More than anything, this test of ours revealed a chasm in the autonomous vehicle world. Fully autonomous taxi services like Waymo are miles ahead of companies like Tesla, Mercedes-Benz, Audi, General Motors, and Ford. We need to differentiate between an autonomous vehicle you hail and one you own, because the former is coming far sooner. If you live in Phoenix or San Francisco, it’s already here.
The distinction especially matters due to liability. A pervasive unanswered question has been hanging over the industry since the first computer grabbed a steering wheel: Who’s liable when a crash happens? If you, a human, crash a car, it’s on you. If the computer that came with the car crashes it, is the manufacturer liable? Is any automaker on the planet willing to take full or even partial liability for every one of the hundreds of thousands or even millions of vehicles it builds each year? Until recently, the answer was no, though Mercedes-Benz has recently broken new ground with its upcoming Level 3 partially autonomous S-Class.
Services like Waymo, though, sidestep that question. If you’re riding in a conventional taxi cab that gets in a crash, it’s on the cab driver and the taxi company that employs them. Waymo and its future competitors assume the same liability, which is why the rollout is so much slower and more meticulous. Waymo has outlined a four-step process for entering a new market, from manually driving and mapping the area to autonomous driving with human safety drivers to the Trusted Testers program—Waymo employees and local residents—to public operation. The company is trying to get the time it all takes down to 18 months. Once a passenger is aboard, though, Waymo is the legally responsible party. (Don’t think you can hit a payday by yanking on the wheel to cause a crash and sue—in Rider Only operation, the Waymo One is programmed to resist steering inputs and quickly and safely pull over and park.)
While we ponder these many questions, the public is already settling in slowly. While we wrestle with our future and that of the industry, one anonymous Waymo One customer has already made it a part of his daily routine: He uses the service and its vehicles to get to the bar and back home safely every night.
Looks good! More details?