Directory

Safety Verification of Neural Networks

  • Palo Alto, United States of America
  • September 2018
  • Stanford University
For my thesis, I developed open-source algorithms that can be used to verify the safety of nonlinear systems containing neural network components. Any developer working on machine learning for a safety-critical system (cars, planes, etc.) is welcome to use my code and algorithms.
  • Open source
  • 60/40/0 (% m/f/d)
  • Private

Project stage (in ):

Implemented/Running Project

Implemented by:

Inhouse

Industrial Sectors:

Professional, scientific and technical activities

Usage of AI:

Robotics

Generation of AI:

Deep Learning (CNN, Transformers, etc.)

Model training:

Supervised Learning

Contact

Responsible Person

Chelsea Sidrane

Motivation and values

What in your view defines the public interest and how does your project meet this purpose?

Even though my open source software is for developers rather than consumers, developing safe machine learning systems is in the public interest. Safety validation is a critical part of product deployment, especially for safety critical cyber-physical systems like autonomous cars or other transportation systems. I have developed open source algorithms that can verify the safety of machine learning based systems. This protects the user of such a system from undiscovered bugs.

How did the idea of your project come about?

Internal in our organization

What is the goal of your project in relation to the public interest?

Safer machine learning systems through more thorough safety validation.

Guiding Values

What are your top 5 guiding-values for the project? Top 1

Validity

What measures do you use to implement this value? Top 1

Our system should produce sound claims.

What are your top 5 guiding-values for the project? Top 2

Clarity

What measures do you use to implement this value? Top 2

Our algorithms and software should be easy to use, understand, and build upon.

Design & Safeguards

Which stakeholders were involved in the process of development and implementation of the project?

Academic researcher, Industry partners

How did you engage relevant stakeholders?

Workshops

Have the project results been validated by third parties?

Yes

Have the design and the results of the project been made transparent to the public?

Yes

In what way have the design and the results of the project been made transparent to the public?

By academic researchers, By the open-source community

Involving the people who will be affected by the project is a necessary part of the project design

5

Which direct channels for feedback exist?

Email address is on all papers.

How did you verify the accuracy and robustness of your system?

Mathematical reasoning and numerical experiments.