“There doesn't seem to be a very good accountability mechanism here,” said Victoria resident and frequent police critic Stephen Harrison, who obtained the documents. “Literally nobody other than the individual officer has access — you wouldn't even know that they're using it.”
If the output is different every time, even subtly, then the overarching system is non-deterministic. If you need to rely on the output to be accurate either for safety, reliability, liability or any combination of the three then it must be verified by every time and your benefit in not using the human that can actually be held accountable for the output is severely limited.