It's a lot of work to demonstrate that self-driving cars will actually work properly. Testing alone is probably not going to be enough, and probably there will need to be some clever architectural approaches as well. Last week I gave a talk at the SAE World Congress in Detroit about this and related challenges.
Link to paper
Link to slides (also see slideshare version by scrolling down).
Challenges in Autonomous Vehicle Testing and Validation
Philip Koopman & Michael Wagner
Carnegie Mellon University; Edge Case Research LLC
SAE World Congress, April 14, 2016
Abstract:
Software testing is all too often simply a bug hunt rather than a well considered
exercise in ensuring quality. A more methodical approach
than a simple cycle of system-level test-fail-patch-test will be
required to deploy safe autonomous vehicles at scale. The ISO 26262
development V process sets up a framework that ties each type of
testing to a corresponding design or requirement document, but
presents challenges when adapted to deal with the sorts of novel
testing problems that face autonomous vehicles. This paper identifies
five major challenge areas in testing according to the V model for
autonomous vehicles: driver out of the loop, complex requirements,
non-deterministic algorithms, inductive learning algorithms, and fail operational
systems. General solution approaches that seem
promising across these different challenge areas include: phased
deployment using successively relaxed operational scenarios, use of a
monitor/actuator pair architecture to separate the most complex
autonomy functions from simpler safety functions, and fault injection
as a way to perform more efficient edge case testing. While
significant challenges remain in safety-certifying the type of
algorithms that provide high-level autonomy themselves, it seems
within reach to instead architect the system and its accompanying
design process to be able to employ existing software safety
approaches.
Subscribe to:
Post Comments (Atom)
Static Analysis Ranked Defect List
Crazy idea of the day: Static Analysis Ranked Defect List. Here is a software analysis tool feature request/product idea: So many times we...
-
It is common to see small helper functions implemented as macros, especially in older C code. Everyone seems to do it. But you should ... -
(If you want to know more, see my Webinar on CRCs and checksums based on work sponsored by the FAA.) If you are looking for a lightwei... -
Oct 3, 2014: updated with video of the lecture Here is my case study talk on the Toyota unintended acceleration cases that have been in ...
No comments:
Post a Comment
Please send me your comments. I read all of them, and I appreciate them. To control spam I manually approve comments before they show up. It might take a while to respond. I appreciate generic "I like this post" comments, but I don't publish non-substantive comments like that.
If you prefer, or want a personal response, you can send e-mail to comments@koopman.us.
If you want a personal response please make sure to include your e-mail reply address. Thanks!