As OEMs gradually roll-out their self-driving technology, they need a way to communicate to consumers exactly what their cars do and what they don’t do. Does this car enable pre-teen children to have mobility? Does the occupant need a driver’s license because they may still be expected to drive under certain circumstances? What does “SuperCruise” and “Autopilot” really mean? This is what autonomous classification is all about. Without a clear understanding of the division of labor between car and driver, we’ll end up with more cars driving into white semis.
It is, however, a devilish problem. SAE’s J3016 system, which has become the de facto industry-standard autonomous classification and certainly has proven itself useful, is unfortunately too vague to make conclusive comparisons and understand explicit technology limitations. Yet on the other hand, SAE’s system is still too technical to clearly communicate driver responsibility. As we refer to in our e-book, many other organizations have tried their hand at developing classification systems to address different aspects of the problem. Unfortunately there is little consistency and still much work to do on these classifications.
Driver-facing classification
I think that there must be (at least) two independent classification systems. One must be consumer-facing, simply and clearly explaining to drivers and/or occupants their responsibility. While I find both Alex Roy’s Geotonomy and CARMERA have proposals that have much to recommend, I might suggest a scheme that corrects minor oversights with both:
Manual – The human controls all aspects of the vehicle’s behavior. This is “legacy” driving and requires a trained human driver.
Assisted – The human and vehicle collaborate in the driving task. The vehicle takes over some aspects of piloting (adaptive cruise, lane change, highway self-driving, etc) but the human is still fundamentally expected to drive. This is trickier than manual because the human driver must be aware of the flexible boundary of the car’s limitations. While the car may take over some aspects of driving under certain circumstances, the person using the car needs to know what systems are automated and the limits of that automation.
Automated – The car is expected to drive itself without human intervention. The car’s performance is not guaranteed on every road surface, so the car may need to exclude areas of travel where it is not tested or certified to perform acceptably. The occupants of the car need not know anything about the driving task – they may be vision impaired, children, pets, etc – and as a result, must abide by the car’s limitations.
In-between the simplicity of manual and automated lies the morass of complexity implied by assisted. Many vehicles that offer assisted capability today do not clearly communicate to their driver what assorted beeps, chimes, or flashes mean. To be truly useful, assisted vehicles need to have the driver’s responsibilities clearly spelled out as simply and concisely as possible by the automaker to the driver through human-factor techniques using text, audio, and/or iconic representations that are OEM consistent.
This is our “driver-centric” classification, and we’ve included it in our comparison of various different autonomous classifications.
Capability classification
Notably while the above scheme allows the consumer to understand their responsibility, it does not allow granular comparisons in the ever-increasing self-driving technology arms race. For this, we suggest a second capability-oriented finer scale that can be used by reviewers, OEMs, industry insiders, and anyone else who is attempting to compare different vehicle performances. It might also include consumers wanting to compare technologies during the purchasing process.
This uses a scale from 0-1000, with higher numbers indicating better overall self-driving performance. A score would be a composite of many individual tests ranked from 0 (no ability) to 10 (equivalent or exceeding perfect human performance), scaled and summed for a final score, but with all detailed rankings available in a publicly shared test result. The many individual tests would need to incorporate things like:
- Driving on a highway with clear lane markings
- Driving on a highway with lane markings that are not visible (worn off paint)
- Driving in dense traffic moving at least 100 km/h
- Changing lanes in dense highway traffic
- Safely braking at 100 km/h with a sudden stop from a leading vehicle five car lengths ahead
- Driving in fog conditions that limit visibility to three car lengths
- Driving in heavy rain conditions that might cause hydroplaning
- Driving when snow obscures the roadway
- Using highway on-ramps
- Using highway off-ramps
- Merging/yielding on a highway on-ramp into heavy traffic
- Properly obeying a signal-gated on-ramp
- Properly using HOV lanes
- Navigating construction areas without human workers present
- Navigating construction areas with human workers present
- Slowing and stopping for pedestrians stepping into the roadway in daylight
- Slowing and stopping for pedestrians stepping into the roadway at night
- Avoiding large animals (deer) leaping across roadway at night
- Slowing and stopping for a toy (ball) bouncing into the roadway
- Avoiding a disabled car in the middle of a roadway
- Slowing beside a disabled car on the shoulder of a roadway
- Slowing and switching lanes beside police or an ambulance in the right-most lane of travel
- Slowing and pulling over when encountering a passing emergency vehicle
- Driving on urban streets
- Driving on urban streets with stop signs partially obscured by vegetation
- Negotiating urban streets with a series of single column traffic lights
- Making a left turn on an urban street with a dedicated left arrow signal
- Performing an all-way stop for a flashing red traffic light
- Proceeding with caution through a flashing yellow traffic light
- Performing an all-way stop for traffic lights that are non-operational
- Performing a legal U-turn at a signal
- Performing a legal U-turn with a dedicated turning lane on a divided highway
- Not performing a U-turn when signage prohibits
- Navigating traffic circles (outer lane)
- Navigating traffic circles (inner lane)
- Navigating roundabouts (several lanes, multiple entry points)
- Slowing and navigating crowds of people walking into the street
- Safely avoiding cyclists in the same lane of travel
- Locating an empty street-parking space and parallel parking in it
- Locating an empty garage-parking space and parking in it
- Locating a space in a parking lot and parking in it
- Signaling when changing lanes
- Signaling when turning left and right
- Performing a left turn at a light when the opposing traffic does not stop
- Stopping at a rail-road crossing and proceeding when clear
- Operating on a heavily pot-holed road
- Operating on a dirt road
- etc
This long list is meant to give us an idea of the type of categorization that could be substantially more informative in comparing ADAS and autonomous vehicle attributes than existing systems. Such a scoring system would need to be standardized to allow independent bodies to arrive at the same score and comprehensively test the vehicle’s performance limits. It would require significant contributions from engineers and experts to build into a feasible metric. It’s also a continually moving target since self-driving cars are still under development, which probably explains why something that would be this useful hasn’t yet been codified to this level of detail.
Nevertheless, this prototypical capability classification might serve as another contribution towards the measurement of autonomous progress and we’ve added it to our autonomous classification chart.
Sources: SAE, The Drive, Brad Templeton (EFF), Carmera, Inagaki and Sheridan, Toyota Research Institute.
DownloadAbout the author
