Today we are happy to announce the Kick-Off of a new and exiting joint research project with ScaDS.AI collaboration. The project focusses on Cyber Security. Thanks to all the project partners ITPower Solutions GmbH, quapona technologies GmbH, Fraunhofer Institute for open communication systems FOKUS and Leipzig University. Our fellow researcher Martin Grimmer wrote a short sketch about his interesting new project which is attached below. For now, we wish you a successful time. 

Attack-based automation of security testing for IoT applications

A project from the ZIM network “Successful large-scale IT projects: With System to Success”, with ScaDS.AI participation.

Author: Martin Grimmer

The extensive interconnection of electronic devices offers industry and economy various opportunities for new, innovative products and services. Those include predictive maintenance or asset tracking in a wide range of economic sectors, such as agriculture, production or vehicle networking. At the same time, the extensive networking of devices poses new risks, as previous attack surfaces of systems are enlarged and completely new ones arise.

The increasing complexity of both the systems (e.g. due to more extensive functionality) and the application scenarios (e.g. with regard to the number of systems and users involved) that are controlled in use cases make it increasingly difficult and costly to protect these systems against security-relevant weaknesses in implementation. At the same time, the greater integration of development and operation of systems, known as DevOps, and shorter release cycles reduce the time available for adequate security tests. A continuous automation of security tests from the identification of test objectives and the derivation of test cases to test evaluation is therefore necessary. 

Since no complex, networked system is completely secure, vulnerabilities and attacks that exploit them are unavoidable. The project therefore develops an automated procedure to derive a test case from a successful attack against an IoT application. This represents the actual core of a successful attack. This test case can support the creation of a patch against the security hole. Techniques of sensor-based attack detection using data analysis techniques and techniques for test case reduction such as delta debugging will be used.  

One technique often used for such purposes, the measurement of code coverage is based on instrumentation and makes the recompilation of source code necessary. This is not always possible, for example, if a component is only available in compiled form. In addition, code coverage as the sole criterion cannot satisfy the needs of security testing with regard to the detection of vulnerabilities. Therefore, the sensors in the project are not introduced into the IoT application to be tested, but observe the input and output data streams of the microservices that make up the IoT application. In this way, the behavior of the IoT application is comprehensively recorded.

The collected data is analyzed, clustered and evaluated using modern classification methods such as machine learning. The process thus has a broad technological basis and leads to a universally applicable test evaluation function. This sensor-based approach reduces the effort that would otherwise be required by weak-point-specific detection mechanisms, and any necessary instrumentation and recompilation of the code is eliminated. In addition, this increases the detection rate of the system, as the number of false-positive and false-negative findings is reduced, thus increasing the number of true positive findings. 

Even if a completely secure system is not technically feasible or not feasible in a meaningful way, the goal during development and maintenance of applications is to detect and close as many security gaps before delivery or before their active exploitation. Methods such as fuzzing use extensive knowledge of the communication protocol and security gaps to efficiently uncover weaknesses. In addition, fuzzing procedures usually work in such a way that they attempt to cover the entire input data space achievable with fuzzing heuristics.

With the help of protocol models this can be limited. Nevertheless, fuzzing remains a test technique that requires the execution of a very large number of test cases. In addition, the information generated by the test object at runtime is usually hardly ever used to generate further test cases. Typically, only code coverage metrics are used. Search-based methods use a quality function that converts the algorithm to one or more (global or local) optimal solutions efficiently. This means that you can efficiently find a solution even without specific knowledge about a system or weaknesses.

Genetic methods in particular imitate natural evolution with operators such as selection, mutation and recombination using a quality function called fitness function, and develop a solution step by step over several iterations, so-called generations. In combination with fuzzing techniques for mutating test cases, a considerable reduction of the search space can be achieved, since the fitness function considerably limits the search space and thus the application of fuzzing heuristics. Since the fitness function is the test evaluation, which evaluates the system behavior as reactions to a security test case, the search-based fuzzing is achieved without the need for further modifications to the IoT application or its source code.

Usually, an attack-based test case is used as a starting point. Taking into account the coupling effect, similar stored vulnerabilities can be found. Search-based techniques such as restarting (e.g. starting from functional test cases) and larger amplitudes of mutation allow the search-based method to detect vulnerabilities in other parts of an IoT application. Functional test cases can also serve as a starting point if no attacks have been recorded yet. 

In the project, a completely automated solution is being developed. This includes the identification of test objectives, test case generation and test evaluation including the generation of reports. The starting point are attacks on IoT applications identified in the field, which are varied with the help of genetic algorithms in order to identify similar weak points. This not only supports the efficient development of security patches, but also provides protection against previously unknown attacks. The automation provides an enormous time and cost advantage compared to manual examinations.