Fingerprinting Network Protocol Implementations

Abstract

With the rise of network security threats, quickly knowing whether the underlying system has known vulnerabilities is crucial for proactively preventing attacks. In this thesis, we examine the Adaptive Distinguishing Graph (ADG) algorithm, which matches black-box systems with Mealy Machine models of known implementations, formally known as fingerprinting. We extend the algorithm to work with partial Mealy Machine models and apply our method to the DTLS protocol, one of the most important security protocols. To our knowledge, we are the first ones to test this method on DTLS, and one of the few works in fingerprinting to explicitly test using partial Mealy Machines. Our results showcase that the underlying DTLS implementation can be successfully identified, making fingerprinting a useful tool for security analysis. Finally, we propose two ways to modify the ADG algorithm to work with Extended Finite State Machines, allowing for more descriptive models and thus better identification.