LeafData and the Typo from Heck!

The pending release of LeafData v1.37.5 in Washington is about like their previous releases. It's late, it's still got bugs and the Washington State government is still wasting money on a system that is missing critical functionality. Let's look at example of how this dumpster-fire is wasting tax-payer money.

Trival Typos

In the LeafData system there is a definition of a type of material called a clone. It represents small cuttings of a plant, and by law must be under eight inches.

In many parts of the system, the value is the word "clones". However, in this one part of the system they are using the word "clone". Slightly inconsistent -- like all high quality software.

But, how does this affect the seed-to-sale (aka: track-and-trace) software provider such as ourselves?

Identifying Bugs

First, we, along with our competitors in this space all had to identify this bug. Each of us runs our test cases, and discovers that, oops! it's not always the same, something is off, lets investigate. Then, each of us identifies a work-around, to patch this invalid data we receive from the system.

There are at least 10 vendors integrated with LeafData. It's reasonable to assume that each of us spent at least two hours looking into this bug. What with finding it, cross-referencing with the past data, reviewing are schema validators, double checking the documentation, implementing and validating a workaround with test case(s). Each of those steps could be 15 to 30 minutes, easy longer -- it adds up quick. This could rough in at least $2400 dollars of waste (10-companies 2-hours-each 120-dollars-per-hour).

Issue Breakdown

LeafData requires that we create propagation material through a object called a Batch. When creating this new Batch and associated piece of data is created, an Inventory Type. This Inventory Type has two properties: type and intermediate_type The Type is set to immature_plant and the intermediate_type is set to clone. However, and this is a step towards the root-cause: clone is not a valid value. The correct value is clones -- with an s at the end, a single byte difference.

How can this software be more than a year old, nearing this "major milestone" release and still have sloppy bugs like this?

Oh, and also, you cannot re-save or update this item because the intermediate_type is set to an invalid value. It's down to external software providers systems to repair data that comes directly from LeafData in a broken way.

Attempted Resolution

During a meeting 30 May, 2019 this issue was raised. Something like

Hey, you guys are sending "clone" when it should be "clones", could you fix that one byte typo?

So, then, on this we web-conference, for the next 20 minutes, with at least 20 participants (from various software vendors, the LCB and MJ Freeway). We discussed how the fix would impact the rollout. Does the Business side of the project need to review the change; how is it going to affect integrators (!?) what it if introduces others (!?). Around and around we go, many integrators are saying: "just fix it, so easy". "it would allow us to remove some silly/jank work-around". One freaking byte, just wasted another $800 - (20-humans 20-minutes 2-dollars-per-minute).

It's not a joke. So many people wasted so much time. It could have been already resolved, by one human, in 15 minutes (cost:$30)

Integrity Issues

These kinds of bugs highlight some of the ongoing complaints users and integrators have had with LeafData.

There are definitive ways to prevent typos like this in software. That they occur in this system is an indicator that MJ Freeway is not following best practices, is using ad-hoc string constants.

For those of you at home wondering, here's the mind-blowing magic trick (I'll even use PHP so MJ Freeway can copy our work ;p ).

define('TYPE_CLONES', 'clones');
// Now, everywhere use use that constant and not the word.
// If you typo to 'TYPE_CLONE' you'll get a nice complier warning
// And save at least $3200, now you are 1337 hAx0r

Another neat trick, that relational database systems have had since the 80s is another piece of magic called a constraint. This permits your high-quality, core-data provider to enforce the word 'clones' and never 'clone'. Constraints make out-of-band data impossible (and if you know how to make them not-impossible then this article is likely boring for you)

Conclusion

The lack of quality as demonstrated by this trival bug (one of 1000s) highlights problems with these closed systems. These are the kinds of bugs that are easily caught by the many-eyeballs principle of free and open software In fact, LeafData didn't even know about this issue until the many-eyeballs event occurred (they've had months to review!) Any competent developer could have submitted a PR to the code-base to a) use constants/enums/nearly-anything-is-better-than-ad-hoc-fixed-strings or b) simply fixed this one typo in an ad-hoc string.

How was this not caught in their Unit Testing? Do they even have Unit Testing? I've asked, giving me the answer is a security risk.

What happend to data validation? Constants? Enum? How can the system make this record one way, and then not accept a re-save of the same values? How can your own system generate data this is not valid for your own system!?

This contract was valued at $3 million USD -- and we can't even get the spelling of one of the core data items correct.

Are there other bugs in this system that could cost more than $3200? Where else are we wasting two orders of magnitude more money than necessary on this system?