Level 5 autonomy corner cases with GM Cruise Automation

Thanks for sharing.

 

There there is big discussion on use of LIDAR technology. Elon claims all other competitors are morons (a little more direct than that) for using LIDAR technology, though WYAMO and others continue to use it. I am sure the others have analyzed the pros and cons of LIDAR vs the Tesla approach and made a decision. They are not that foolish to go for something if they did not have some hard analysis.

Tesla definitely has a lot of real data from thousands of cars on the street, but so do others. In the usual Elon arrogance, he dismisses anyone who is a competitor. While he has been pretty successful with his approach and bravado, there may be better ideas elsewhere from which he can learn from. As they say pride comes before a fall.

 

Disclaimer: Let me say right up front that all that I know about driving autonomy, and systems such as radar, lidar, and using software for optical object recognition in video images, is book-learning, rather than any experience I've had. So take my assertions on the subject with a pinch of salt. Now, with that being said...

I think Elon had a very good argument against lidar in Tesla's recent "Full Self Driving" presentation. I think he had some very good points about there being too much overlap between lidar and video cameras; they both depend on the visible light wavelength of the electromagnetic spectrum, and they both can be blocked by such things as fog and heavy rain.

What Elon seemed to be purposefully eliding, though, was the advantage of active scanning for making a 3D map of the environment. Using either lidar or radar to "ping" an object, detecting its shape and exact distance, requires far less computer processing and is far more reliable -- and faster -- than trying to use software to laboriously use number-crunching to match a grayscale image, pixel by pixel, in the hope that the software will be able to identify an object in the frame of video.

They spent a lot of time during the presentation talking about how much more advanced their new computer motherboard is, and how much more advanced the chips on it are; and completely ignored the fact that if they were using active scanning for the primary sensors, instead of cameras, then they wouldn't need such a powerful, expensive computer motherboard.

I'll repeat my claim that phased-array radar, so-called "high-res" radar, would be best for most of the sensor needs of a fully autonomous car. Yes, Elon is correct to say that all of the traffic signs, lights, lane markings, etc. are designed for human sight, and thus cameras are the best system to use for an autonomous car to "see" them. So yes, autonomous or even semi-autonomous cars do need cameras.

The fallacy is in trying to use them for everything. As I understand it, Elon's argument boils to this: Because autonomous cars need cameras to "see" certain things, they have to have cameras. But we shouldn't need a second primary/ long-range sensor system, since humans can drive cars using sight alone.

Yeah, well there's a pretty big fallacy there: If humans were so great a driving safely, then why do we need autonomous cars? The fact is that humans are not such safe drivers. And part of the reason we're not, is that we are almost entirely dependent on human eyesight. We're blind at night, and we can't see thru fog or smoke. Furthermore, we don't have the ability to build up in our minds an accurate 3D map of the environment. We can only look in one direction at a time, and altho we can gauge distance to some extent, we can't do it very accurately.

Our eyesight is also degraded in certain conditions such as fog, heavy rain and heavy snow.

There is a reason why Waymo's self-driving cars use three sensor systems: Cameras, lidar, and some form of radar. What kind of radar is Waymo using? Well, here's how Waymo describes it:

Conventional automotive radars have a narrow field of view and typically only track forward moving cars. In contrast, Waymo’s custom radar system has a continuous, 360 degree view, so it can track objects and vehicles usually hidden from the human eye. Our radars are capable of monitoring a vehicle as it comes up from behind and as it moves out in front of us without any interruption.

They also complement our other sensors by being highly effective in rain, fog or snow. Conventional radars have been designed to focus solely on vehicle movements. However, our fully self-driving cars need to safely navigate around all road users, from cars to pedestrians to cyclists. So, we’ve configured our radars so that they’re much more sensitive to these slower moving objects.​

Hmmm, I see Waymo isn't claiming their radar system can detect stationary obstacles. So my guess is that they're just uisng Doppar radar, which only "sees" moving objects and not stationary ones. If Waymo's cars are is using an active sensor system to create a 3D map of the environment, they must be using lidar for that.

* * * * *

My armchair engineer conclusion is that fully self-driving, Level 4/5 autonomy, cars will need phased-array radar to create a SLAM system; that is, a 3D map of the environment around the car made, in real time, using active scanning. Phased-array radar is needed for seeing stationary obstacles; seeing only moving obstacles with Doppler radar quite clearly isn't sufficient. If it was sufficient, then cars using GM Super Cruise and Tesla Autopilot wouldn't keep running into stationary obstacles such as fire trucks parked in a highway lane.

And cameras aren't sufficient for reliably detecting obstacles, either. Neither moving nor stationary obstacles. According to one report I read recently, 50% of accidents happen at night, and 25% of them on unlit roads. We need fully autonomous cars to be able to operate just as safely on unlit roads at night as they do during the day, or on well-lit ones. Phased-array radar works just as well day and night, and doesn't need headlights lighting up the landscape to be able to "see" things. Phased-array radar also sees thru fog and smoke just fine, and does better than lidar or cameras at penetrating heavy rain or snow.

Nor is using cameras with headlights at night on unlit roads sufficient. It wouldn't be sufficient even if headlights were adequate, and according to the IIHS, most of them aren't. Lights pointed in the direction the car is driving won't help the car "see" vehicles approaching from other directions. Autonomous cars need to be able to prevent T-bone collisions and being hit from behind by cars without functioning headlights. Just preventing head-on collisions at night, on unlit roads, isn't sufficient for truly safe autonomous cars.

 

Level 4? Let's see; quoting from Wikipedia, regarding the SAE driving autonomy levels:

Level 3 – Conditional automation
The system is in complete control of vehicle functions such as speed, steering, and monitoring the environment under specific conditions. Such specific conditions may be fulfilled while on fenced-off highway with no intersections, limited driving speed, boxed-in driving situation etc. A human driver must be ready to intervene when requested by the system to do so. If the driver does not respond within a predefined time or if a failure occurs in the system, the system needs to do a safety stop in ego lane (no lane change allowed). The driver is only allowed to be partially distracted, such as checking text messages, but taking a nap is not allowed.

Level 4 – High automation
The system is in complete control of the vehicle and human presence is no longer needed, but its applications are limited to specific conditions. An example of a system being developed that falls into this category is the Waymo self driving cab service. If the actual motoring condition exceeds the performance boundaries, the system does not have to ask the human to intervene but can choose to abort the trip in a safe manner, e.g. park the car.
​

Hmmm, okay, it looks like they consider the Waymo self-driving taxi fleet to be Level 4. I thought the only difference between Level 4 and Level 5 was that Level 5 autonomous cars don't even have driver controls (steering wheel, foot pedals, etc.), but looking at SAE's website for autonomous driving, I see that's not true.

From the SAE website, Level 3/4 autonomy is described as "These features can drive the vehicle under limited conditions and will not operate unless all required conditions are met", whereas Level 5 autonomy is described as "This feature can drive the car under all conditions".

* * * * *

I guess that the reason Waymo's fleet of self-driving taxis is only Level 4, and not Level 5, is because they are limited to a certain range from the central point in their area, and thus drive only within a geo-fenced region. I guess that qualifies as what SAE describes as "under limited conditions".

Waymo is actually using its self-driving taxis in a suburb of Phoenix, and clearly it must be mostly successful, or we would have heard stories about a massive failure there. My assessment is that Waymo's system is more advanced than Tesla's; perhaps significantly more advanced. And I strongly suspect that Waymo will continue to improve their system faster than Tesla does, so long as Tesla remains committed to relying on cameras for nearly everything.

I think Tesla is at Level 3 automation. Elon said during the FSD presentation that when Tesla's "FSD" features are complete, you'd be able to safely fall asleep in the car. That pretty clearly indicates Tesla's not there yet. Tesla's recent self-driving demo video showed the car driving for at least several minutes, possibly a half-hour or so, without the "driver" ever touching the steering wheel. During that time, the car drove on both surface streets and freeways, and showed the ability to respond properly to traffic lights and stop signs. Altho of course Tesla was able to cherry-pick its best example of driving autonomy, it seems pretty clear that Tesla has reached Level 3 autonomy in its advanced testing vehicles.

GM? I have no idea how far advanced GM is or isn't. If they are as advanced as either Waymo or Tesla, they certainly are hiding their light under a bushel basket! So, I'm rather skeptical about Mr. Green's claims, especially since they're coming from a serial Tesla basher who has every motive to belittle and denigrate Tesla's accomplishments.

​

 

In general I agree with most of your post, but you are suggesting here that if Tesla should spend a few hundred dollars more on LiDAR so then they wouldn't need to have the FSD computer. There is no evidence that the FSD computer is more expensive than LiDAR, in fact I suggest it is much cheaper. What we do know is that the new HW3 FSD board is 20% cheaper than the old one.

 

No fallacy. Your assertion that the reason humans have accidents is because we depend on eyesight is false. We have accidents when we do not pay enough attention to driving. We get distracted or tired or drunk etc.

 

Although this is one of my chief complaints about Autopilot, I haven't seen reports of these recently and wonder if newer updates have resolved these issues. As you suggested before Doppler radar cannot detect non-moving objects, but I think I saw evidence in Tesla's presentation that they are using radar for distance measurement, not just speed so this suggests to me that they must be using pulsed doppler. Could this have been a capability not used in earlier software versions?

 

Waymo Announces It's Opening an Autonomous Car Factory in Detroit

(https://gizmodo.com/waymo-announces-its-opening-an-autonomous-car-factory-i-1834253156 )

It’s official: Waymo is opening an autonomous vehicle factory in Motor City.

Or to be totally accurate, Waymo is building a factory to turn regular ol’ dumb cars into smart autonomous cars. Months after announcing that it was bringing its self-driving car operation to Michigan, the company said on Tuesday that it would be opening a repurposed factory in Detroit. In selecting the facility, at which Waymo will mass manufacture level 4 autonomous vehicles, the company said it had sought out a location that would not only allow for future growth but “would allow us to quickly get up and running by mid-2019.”

From what little knowledge I have, Waymo is the closet to level 5, though GM claims to be at par or better (Tesla/Elon of course claims to be ahead of both of them). I would doubt if the bean counters at Alphabet/Google would really invest in a factory if they did not feel they were pretty confident. Wyamo is buying about 20,000 I-Pace's, which even at a deeply discounted price is a fair chunk of money, even for Alphabet Inc. So I am still on sidelines on believing Elon's claims that he has better technology.

 

I don't know what I said which suggested to you that I think Tesla should be putting lidar into its cars. In fact, I said:

I thought I made it pretty clear that I think they should use phased-array radar in preference to lidar.

The new one, with the fancy new microprocessors which process video frames and perform integer math at like 10x or 20x faster than the old one, is actually cheaper? Well, if that's true, then it's a remarkable achievement for Tesla!

​

 

If the only reason humans have driving accidents is because of distracted/ tired/ drunk driving, then why do 50% of accidents happen at night, despite there being so much less traffic at night? And why do 25% of accidents happen on unlit roads, which -- obviously -- carry a much, much smaller fraction of all traffic than the more well-traveled roads which are lit at night?

Nope, sorry; logic, facts, and statistics are sharply contrary to your argument here.

Here's a real-world example: A relative of mine, who is in general a pretty safe driver, has twice hit a deer when driving on the back roads to her house. Of course, that happened at night, when a deer suddenly charged out of the darkness in front of her car. A car equipped with a SLAM system which could "see" in the dark would have avoided both those accidents.

​

 

I certainly am willing to be told that my info is outdated or that I missed something in Tesla's FSD presentation, but I did listen to very nearly every minute of that presentation (I did take a couple of short breaks to grab something to eat while watching), and I saw/ heard almost no mention of radar at all. In fact, at one point when responding to a question from the audience, Elon did mention Tesla's forward-facing radar, but he gave it only a passing mention and didn't say anything about any improvement. Certainly he made it quite clear that the only long-range radar in Tesla's production cars is forward-facing only, which of course isn't adequate for 3D mapping.

There was an awful lot in the FSD presentation about processing video images, and pretty much nothing whatsoever about radar. That very strongly suggests to me that Tesla is trying to depend almost entirely on software interpretation of video images. Furthermore, the only 3D maps they showed at the presentation -- and those were woefully sparse and incomplete, certainly not sufficient to spot adult pedestrians reliably, let alone small children or pets -- were based on data pulled from video images, and not from active scanning such as phased-array radar or lidar.

So my assessment -- not necessarily correct, but based on the evidence I've seen -- is that the only long-range radar Tesla is using, is still a front-mounted low-res Doppler radar.

​

 

It is a remarkable achievement, but quite reasonable when you consider if you buy your chips from Nvidia that Nvidia has to put in a mark up to cover their development costs and make a profit. Musk's statement about 20% cheaper certainly did not take into account their own development costs.

 

IMHO that's one of the most stupid business plans I've ever seen. Waymo should license its tech to auto makers, so they can integrate the tech into their production cars. Or at worst, Waymo should build the hardware to retrofit cars for self-driving, and sell those systems to aftermarket suppliers and/or customizing shops.

Buying new cars (or rather, light trucks) to refit them and sell them... that's what Via Motors is trying to do. How is that working for them? Not well!

I predict a monumental disaster, like (Project) Better Place's high-profile attempt to set up a BEV battery swapping subscription service in Israel. I could see from the start that was guaranteed to fail, and the only surprise was it happened even sooner than I expected.

This venture by Waymo may not be quite as inevitably doomed to failure as (Project) Better Place, but I'll be very surprised if it turns out to be a financial success for Waymo.

Please note I'm not claiming Waymo's tech isn't good enough for true self-driving. I think their driving autonomy tech is significantly in advance of any other company's. I just think this isn't the way to profit on what they've achieved.

 

Well now, with that caveat, it's a lot less surprising. I was wondering if claiming lower cost was just more of Elon's spin, and now that you've explained that, it certainly looks like it was partly or mostly spin.

​

 

Here is the difference for Waymo. One, they can retrofit them, then operate it as a taxi service, then depreciate them using the 200% declining balance method (which is 40% of the cost), and possibly sell them after a year at great profit (just like the car rental companies, who make a lot of money on the resale of cars). Second, now all of these cars they put back in the market have Google technology, they have to use Google maps etc. It is like buying an Android phone, Google tracks you in a hundred ways and profits from the information. Three, they will have then the same ability to get information under real life conditions like Tesla, more cars on the road, more information. Four, if they want to get into production of cars themselves, what they can do is have the base assembled elsewhere (Magna is a partner) and use their new factory to complete the work. (Just like Android technology is licensed, but Google sells their own pixel phones). They are in much better position than Via, they have deeper pockets, proprietary expertise in autonomous driving, ability to profit off information gathered etc.

 

Personally, I don't care about LIDAR since I have basic AutoPilot and come Tuesday, a month of experience, over 2,800 miles. The cost of LIDAR versus the $3k I spent on basic AutoPilot has decided the problem.

I'm reminded of the old joke:

• First prize is a LIDAR equipped car
• Second prize is a car with two LIDARs

Bob Wilson