In the last few weeks
#OpenZFS has grown a unit test framework!
This is different to the existing "ZTS" integration/functional test suite. That's running the zpool and zfs commands, creating and destroying pools, writing to files, and all sorts of "user-facing" stuff to confirm operation.
This is more like what you'd expect - targeting specific internal functions and APIs, mocking objects and behaviours that sit "under" those, to confirm that they do what they should.
Unit testing in C, especially very old C software, is traditionally quite complicated, and it will be for some of the very complicated and entangled parts of OpenZFS, but there's a huge amount of code that is behind pretty mundane and boring and narrow APIs that are easy to test.
A few months ago I was looking into the very old normalization and casesensitivity properties, which control how filenames are compared and matched. We have some ancient tests in ZTS (the casenorm group) that have never passed cleanly, and have long been on the ignore list (see
#7633). I started looking into them, and followed path into the ZAP subsystem, which is our on-disk K/V data structure that we use for tons of stuff - filesystem directories are the thing you see, but internally, dataset and pool property lists, dedup and clone tables, and tons of other metadata). Its a critical component, and also very old, very quirky, and very hard to understand.
I've long wanted to do experiments with different kinds of on-disk K/V formats, but haven't because messing in there is intimidating and the potential blast radius is extreme. For the same reason, I haven't spent much time looking at adding ZAP "housekeeping" operations, like defragement, rebalance, etc. Now I was faced with the same issue again, this time to try to understand and possibly fix a bug in the name lookup machinery.
ZTS tests OpenZFS from "outside". It makes sure directories work, for example, but it mostly can't tell what's happening under the hood. There are some things that can be inferred, eg by checking counters and metrics, looking for evidence that certain things have happened. That's good for confirming that you haven't broken an existing mechanism, but doesn't help if your whole purpose is to change the existing mechanism.
Of course, I've long known that unit tests were what I wanted here, but we didn't really have that in any general way (there are a small handful of unit-test-like drivers in ZTS, but they're ad-hoc and don't really fit inside an integration test suite). So I finally took the hit and got on with it, making a unit test framework as small and focused as I possibly could, that was quick and easy to run and easy to iterate on and just pleasant to use! I made a particular effort to show a short and sharp coverage summary right there in the output. I even wrote some docs!
https://github.com/openzfs/zfs/tree/master/tests/unit
While building the framework, I started writing the ZAP tests alongside it. Those are making tremendous progress.What you see in the above link and the images below is where we're at today. I've still got another set to land which tests some of the existing internal housekeeping functions, but more important than that, I already have good progress on a substantial refactor, trying to clear out some of the quirks and modernise the structure. Very nice with a safety net underneath.
Even more exciting to me though, is that it hasn't taken long for another contributor to start building out test suites for other areas of the codebase. namecheck is the set of functions used to validate names of things - pool, dataset, mountpoint, etc. Important, low hanging fruit, easy to write a test case for. And got to 100% coverage before I did, boooo but yaaay!
Anyway, I'm pretty excited about this. Making OpenZFS easy for new developers to immediately start doing useful work has been a long-term project for me for years, and this is just another thing that will definitely make my life easier and hopefully a bit easier for others too! If nothing else, there's lots of tests to be written!
