2/5
--- ### Understanding Fuzz Testing and Formal Verification #### The Limits of Fuzz Testing Fuzz testing, as demonstrated in the example of the math masters, can initially provide a false sense of security by indicating no bugs in the code. It was only after increasing the fuzz testing iterations that a counterexample was found. This example illustrates that even extensive fuzz testing might not always be sufficient to detect all potential issues. #### Exploring a Complex Codebase In our case study, we delve into a specific `Sol` file within the `src` directory. The codebase is intentionally convoluted to challenge manual auditing processes: - **Function Mapping Confusion:** Common mathematical operations are intentionally mislabeled: - `add` performs subtraction. - `div` performs addition. - `mole` (presumably meant to be `mul`) performs division. - `sub` performs multiplication. This approach creates a nonsensical set of functions that defy standard mathematical logic, complicating the audit process. #### The Cursed Code Example The example further complicates the already chaotic environment by misusing these operations. For instance, using `add` not just for subtraction but also wrapping and dividing integers, creating a highly irregular and error-prone scenario. This kind of coding practice is referred to as "cursed code," designed to be difficult to audit manually. #### Real-World Coding Practices Despite the exaggerated example, it reflects real challenges in software development where complex or unconventional coding can exist, sometimes as seen in other notorious codebases like "make or die." #### Formal Verification and Its Necessity The session then shifts focus to formal verification, an essential method to ensure code correctness, especially in complex or critical systems. This method is applied to a specific function: - **Function Name:** `hellfunk` - **Invariant:** This function must never revert. - **Behavior:** The function's behavior is illogical and complex, making it a prime candidate for formal verification rather than traditional auditing. #### Testing the Codebase The demonstration concludes with a test setup: - **Test Objective:** Ensure that `hellfunk` never reverts. - **Method:** A stateless fuzz test is utilized, as the function does not interact with or alter any state. - **Outcome:** Initial tests pass, suggesting the function upholds its invariant under the current testing conditions. In a hypothetical scenario, even escalating the number of fuzz test runs extraordinarily high fails to identify further issues, emphasizing the limits and challenges of relying solely on fuzz testing for certain types of code verification. By dissecting this intricate example, the lesson aims to highlight the significance of robust testing and verification techniques in software development, particularly when dealing with complex or potentially misleading codebases.
A deep dive into where fuzzing fails - This lesson looks at an example of a code base where a fuzzer would give the false impression that there are no bugs. It then goes on to explore some of the potential reasons why fuzzing might fail to find bugs, including the possibility that a fuzzer might not be run for long enough. The lesson also considers the idea that, while fuzzing might be a powerful tool, some codebases might be so complex that they require manual auditing instead of fuzzing.
Previous lesson
Previous
Next lesson
Next
Give us feedback
Course Overview
About the course
Assembly
Writing smart contracts using Huff and Yul
Ethereum Virtual Machine OPCodes
Formal verification testing
Smart contract invariant testing
Halmos, Certora, Kontrol
Security researcher
$49,999 - $120,000 (avg. salary)
Smart Contract Auditor
$100,000 - $200,000 (avg. salary)
Guest lecturers:
Josselin Feist
Head of Blockchain at Trail of Bits
Last updated on January 17, 2025
Solidity Developer
Assembly and Formal VerificationDuration: 30min
Duration: 4h 38min
Duration: 3h 57min
Duration: 1h 56min
Course Overview
About the course
Assembly
Writing smart contracts using Huff and Yul
Ethereum Virtual Machine OPCodes
Formal verification testing
Smart contract invariant testing
Halmos, Certora, Kontrol
Security researcher
$49,999 - $120,000 (avg. salary)
Smart Contract Auditor
$100,000 - $200,000 (avg. salary)
Guest lecturers:
Josselin Feist
Head of Blockchain at Trail of Bits
Last updated on January 17, 2025
Testimonials
Read what our students have to say about this course.
Chainlink
Chainlink
Gustavo Gonzalez
Solutions Engineer at OpenZeppelin
Francesco Andreoli
Lead Devrel at Metamask
Albert Hu
DeForm Founding Engineer
Radek
Senior Developer Advocate at Ceramic
Boidushya
WalletConnect
Idris
Developer Relations Engineer at Axelar