Behavior Prediction and Decision-Making in Self-Driving Cars Using Deep Learning — Part 2
Let’s continue from the point we left last time.
Behavior Prediction + Planner or Mid-to-Mid Driving
The next approach is to combine the Behavior Prediction and Planner modules.
ChauffeurNet
The next work is from Waymo. They tried to do prediction and planning together using one single neural network using Imitation Learning (IL).
They decided to use mid-level information from the Perception module and HDMap to create BEV images as the input for their model. You can see the different inputs they used as input:
It is easy to augment this type of representation and create some fake data for some corner cases like collisions, going off the road, …. You can see one example of creating a fake trajectory to teach the car to come back to the road when it is going off the road:
Using these augmented that’s their model is able to handle these cases. It also learns to avoid a parked car and nudge as you can see in the following gif:
