AI Testing Talks

Anomaly Detection is a mini-course, hosted as part of Exactpro’s AI Testing Talks events series. The mini-course consists of 3 lectures presented by Rostislav Yavorski, Head of Research, Exactpro. Rostislav reviewed the basics of anomaly detection, as well as the application of statistical and Machine Learning (ML) methods in software testing.

Videos

Anomaly Detection. Part 1 – Basics

On 20 May, we held the first lecture of the Anomaly Detection mini-course from Rostislav Yavorski, Head of Research, Exactpro. In his lecture on “Anomaly Detection. Part 1 – Basics”, Rostislav reviewed the definitions and practical examples of outliers and anomalies in different domains: financial fraud detection, medical diagnosis, fault identification, and others.

Anomaly Detection. Part 2 – Statistical Methods

In Lecture 2, we discussed the graphical methods: histogram, box plot, and scatter plot, as well as interquartile range, Tukey's fences, and null hypothesis, t-statistic, p-value.

Anomaly Detection. Part 3 – Machine Learning Methods

In Lecture 3, we discussed the unsupervised, supervised, and semi-supervised methods, as well as review dimensionality reduction and clustering.

Test Design for AI Systems

“Machine Learning (ML) has achieved remarkable progress over the past decade and has been widely applied to many industry domains, including safety-critical ones. With the expansion of ML, the risks related to correctness and robustness are also evolving. Businesses and governments are mitigating the risks with regulatory activities. Since software testing is an important aspect of monitoring and control processes, its applications in ML-based systems (MLS) are also evolving.”

Machine Learning for Software Testing

Rostislav reviewed the current research areas in software testing and the ongoing AI Testing challenges, as well as suggested topics for further research in this area.

Autonomous Vehicles (AVs): Basics and Testing Challenges

Julia Emelianova, PhD, Researcher, Exactpro, discussed the process of developing an autonomous vehicle, the key principles of automated driving and the existing navigation challenges, the ways in which AVs drive and the data that supports their movement, the architecture of automated and connected driving, testing approaches, and and the main testing objectives for AVs.

Negative Testing

In this session, Julia gave an overview of the characteristics of negative testing and its scenarios, and discussed how negative testing can help identify the defects that can potentially result in major failures or outages, data corruption or security violations, while also improving the quality and stability of software applications. Julia also reviewed the benefits and disadvantages of negative testing.