We don't have a standardized way to test self driving yet. If we do we probably need to spec how the road should be, traffic sign should be, road construction cones and signs should be, how pedestrians should behave, etc. Other than freeway / highway I haven't seen human follow the rules to the dot with construction and street crossing. We probably need to tighten some rules over time to make self driving easier. But then you can't prevent people trying to taunt self driving like you can have a human driver calling cops on those guys.
One thing I think self driving can do better than human is using more than human eyes' spectrum for driving. Maybe infra red? maybe UV? Maybe network connection to a local traffic camera to view from the next intersection?