LeafData Data Anomaly: Negative Values

Somehow, in some cases, the LeafData system changes, or permits to change inventory quantity to a negative value. Naturally, one cannot have a negative quantity of physical goods. Some times the system blocks this effect, many times it does not. Other cases the items are affected due to data-sync-lag type of issues.

Success Mode

One case that does work, from Lot to Plant conversion. That is, taking Seeds/Clones or other propagation type materials into Crop items. Here, the LeafData system warns if the quantity of source inventory (Clone/Seed) is below zero.

Failure Modes

Other Lot quantities are adjusted or modified in numerous ways. And each can create a situation where the resulting lot quantity becomes negative.

B2B Sales Case A

Here, a supply side license puts 10 units on a B2B sale (aka: Manifest, Transfer) They release the sale, the receiver has a complaint on the transfer, which must be fixed and re-filed. The void process isn't restoring inventory, the second sale record reduces the quantity again.

B2B Case Case B

Here, while completing the paperwork the same source lot is added to a second sale. Both of these sales items are released.

Conversion Case A

Two cases exist for these negative values to be created on convert.

One method, when coverting it's possible to consume imbalanced materials. So, one could consume 200 g of Flower when the quantity is only 100g. And it's possible to record that as producing 300 g of packaged goods!

This could happen from a typo perhaps, but in some cases there there is a reasonable situation for larger output. Edibles for example, have an additive.

Conversion Case B

The second case is a conversion failure in LeafData system.

From a source item with 1000 quantity, attempt to convert all of it to joints (1000 x 1g). LeafData however issues an error response, and it appears no changes have been made. To complete the process, the request is re-submitted -- resulting in a double count.

Sometimes these errors occur in close time-proximity to each other. It's common to see three or four conversion requests in a short window.

Conversion Case C

Reverting Returned Materials.

Returned materials come back under new IDs, however the label is from when the original material was sent. So, for the LeafData system to reflect the physical world those received units must be merged back into the proper lot. During this process, if one encounters a system error, the count can slip.

Additionally, due to some caching issues / data-lag one may end up processing the returns and not seeing those values properly reflected. The user would then attempt to make the correction a second time which results in a negative values.

Waste Case A

Another odd case for negative quantities are phantom inventory items, of type Waste that are created by LeafData. Items such as these appear to occur with random quantitys and becuase they are classified as Waste type it's not even possible for users to correct them.

Pattern Observations

In many of these cases the process that landed at the negative quantity is clear, others of course are not. So, each would be need to evaluated case by case (that's millions of records). The LeafData system also keeps "adjustment" records, so those should be consulted as well.

Multiple Same Q Down

Here the pattern is something like NEW » 20 then rapid -20 » -20 » -20. A user creates the new inventory, with a quatity of 20. Then, the record changes down by the same quantity two or more times in quick succession.

Created on Convert, or Process or Recieve operations, then trying to correct the value. Correction to undo the convert or merge the received materials back to their origin lot. It's repeated because of data-sync-lag type issues.

Double Q to Negative

This one seems to be the result of some conversion error. An existing Lot with quantity 200 has a proper -150 event but some error occurs and the operation is only partially finished however the lot now has a quantity of 50. A second attempt is made by the user, again for -150 which succeeds all the way. Now the original lot is -100, from 200 - 150 - 150 process.

The negative value is the result of an operation that was repeated.

Data Fidelity Issues

One of the issues with discovering which pattern (if any) have resulted in these negative values is tracking the changes. Not all of the adjustments to quantity values are recorded in LeafData. The licensees own tracking solution should have logs of these change to each of the records however. Those, along with the adjustments that ARE recorded in LeafData can provide a more complete picture.

Data Analysis

Now we know how odd values can enter the system we can look at some of the data. We can see how negative values have appeared in the system.

Some quick notes.

• 15276508 records were sampled, from 2019-01-01 forward.
• 8165305 records (53.5 %) have been zeroed out, this is a normal situation
• 6936987 records (45.4 %) have a positive quantity, this represents active inventory.
• 174216 records (1.1 %) have a negative quantity.

More details have been posted at https://data.openthc.org/lot/qty

Summary

Due to bugs in the system (all components: LeafData, Integrators, Licensees) can create a situation where some quantity values can become negative. In many cases this can be tracked directly to technology-layer bugs generally related to caching, transactions and error-state handling. System wide it only affects a small portion of the records however the distribution across licenses is very wide.