Artificial Intelligence Cautionary Tales

The technology landscape is getting so big, so fast even the most connected among us can feel like we’re racing against time. Staying on top of all the breaking developments taking place today seems impossible, and it can likely lead to the fear we’re falling behind. Nowhere is that felt more than in the realm of artificial intelligence (AI).

Along with its promises of efficiency and innovation comes the looming threat of persistent disruption. Tune into any media channel and you’ll be bombarded with messages about AI disrupting your job, your role, possibly your entire industry. On a scale from 0 to Terminator, many of us are hovering around a solid 6 (Robocop). 

Whether you’re deploying AI as an attempt to avoid a dystopian future or aiming to roll out AI in a rush, both are just as likely to backfire as they are to work out well. So let us share a few tales of warning regarding our experiences, as well as those of a few of our friends, in testing and deploying AI.

“Three may keep a secret, if two of them are dead.” – Benjamin Franklin

In our experience, machine learning models are terrible secret keepers, and for a myriad of reasons. We’ll share the three we encountered while testing a single AI application –

one that integrated our knowledge base solution with our team chat platform. The possible upside was huge. If it worked right, team members could simply ask the AI, and it would fetch information from across our prior conversations, knowledge base, and potentially our document library as well. If this worked half as well as we’d hoped, it would save hundreds of human-hours per month.

Wanting to be intentional about how we deploy such things, we started with a small test. Within 30 minutes we encountered numerous issues with the platform.

Third Party Sharing

Our knowledge base’s solution was built off of the OpenAI API, which means all of the indexed content was shared with both Open AI and Microsoft, something not made immediately clear in the language promoting the AI tool and only discovered after we did some deep digging. 

In other words, in order to use these tools, we’d need to find and remove any sensitive or confidential information from both our knowledge base and our team chat system, illustrating how easy it is to accidentally provide these tools access to confidential or proprietary information. Once done, it’s a genie you can’t put back in the bottle.

Private Files Aren’t So Private

We also created a number of private and secret documents that, for testing purposes, simply stored comical information. Some files were only accessible by a single person, others by small discrete groups, and others by the whole team. 

In our testing, it was ridiculously easy for team members who did not have proper access to documents to query the system in such a way that it readily disclosed the private information.  Further compounding the already-confounding situation, the AI had been instructed not to show private files as one of the sources it used. And yet, the information was used since it was the only file to store this type of information. 

(For those curious, the information was a table comparing each team members’ running speed to the speed of an animal. Someone without access to this file was able to get the AI to share almost the whole table by asking “who are the fastest people at Yoko Co?” and it spit out the first five rows of the table. But, it cited our “team bios” as the information source. Our team bios do not, unfortunately, include peoples running speed comparable to various land animals… though perhaps it will in the future.)

Hallucinations Galore

Finally, we tested the AI to see if it could help us quickly reference various contracts and deliverables to answer relatively simple and straightforward questions related to projects. “How many wireframes are we supposed to create for this project?” “What are the payment terms for project X?” “What link structure did we decide to use for client Z’s resource library?”

In the majority of these cases, the AI quickly and confidently shared an answer. 

Unfortunately, those answers were wrong. It often cited the wrong document, or said it was citing the document. Upon checking, it had simply made up (technically “hallucinated”) the answer, which, if we had taken it at face value, could have been disastrous. Besides, if we have to go back and double-check the documents to see if the answer is correct, isn’t it faster to just check the documents in the first place?

“It was nice while it lasted.” – Marie ‘Slim’ Browning

Thus the experiment concluded, just thirty minutes after it started, and we found that a widespread rollout of this AI would have been nothing short of a catastrophe. 

Regrettably, in this high-pressure technological arms race, many feel pressured to roll out AI sans much testing and naively assume the tools have been rigorously vetted by their makers. Why would AI do something inappropriate let alone illegal?

“It may be that your sole purpose in life is to simply serve as a warning to others.” – Jill Shalvis

AI implementations gone wrong could be a new streaming series like Help! I Wrecked My House or Nailed It! We’ve shared some examples here in the hopes that we can learn from these mistakes rather than repeating them.

Copyrights Gone Wrong

AI image generators, from MidJourney and Dalle3 to ArtFlow and Character AI, some incredible tools exist out there, with no shortage of ways to use them. But these platforms are trained on a ton of copyrighted or trademarked work. 

While we can’t share specifics, we have heard of clients who wanted a quick and easy logo update whipped up by one of these platforms. They really liked it, slapped it on their website and email signatures, as well as their building signage, shirts, and other swag only to receive a cease and desist letter a few months later because their new logo was strikingly similar to a large and aggressively litigious enterprise. This “free” logo had now cost them tens of thousands of dollars in work to replace the offending logo.

You Got Served

You might’ve heard the hilarious horror stories of AI-enabled “customer service” chatbots going rogue. From the funny ones like the delivery company’s chatbot swearing at customers and the Chevy dealership bot who sold a Chevy Tahoe for $1 to more extreme examples like AirCanada’s chatbot lying to a grieving passenger about a discount and a medical advice chatbot suggesting a patient should off themselves.

Outsourcing Biases

Why should people waste time applying biases like racism, sexism, agism and others when AI can do it for them? </sarcasm> Like Amazon’s now discontinued AI recruiting tool which was found to be highly discriminatory against women, AI in general seems to be highly biased. That’s what happens when a machine has been training on large volumes of information that are inherently biased. (A quick shout out to ChatBlackGPT by Erin Reddick, for bringing both awareness and solutions to this enormous problem.)

Cybersecurity Nightmares

AI provides a number of concerns for anyone who considers themselves security conscious. AI is largely a black box we don’t understand. You can’t trust what you don’t understand. Which means AI is unlikely to be used in highly secure environments anytime soon.

Beyond that, AI opens up a number of potential vectors of exploitation: sloppy code that has vulnerabilities itself, being engineered via prompt injections, giving up information it shouldn’t, or simply by being used by bad actors to purport themselves as others, like this example of an employee who was tricked into wiring $25 Million on a call with a deepfake of their CEO.

“It’s hard to see things when you’re too close. Take a step back, and look.” – Bob Ross

Perspective can be hard to maintain when you’re in the tall weeds, rushing to be as productive as possible. So take a bit of advice from Bob Ross. Step back and gain some perspective before deciding where, when, and how to implement AI.

Start with Why

Before you make any technology decision (or really any decision), understand why you’re doing it. As you pause and look at your problems with newfound perspective, consider where your organization aims to go and what role AI might play in helping you get there. What are its use cases? What information does it need? What are the risks? Know your why before doing anything else.

Prioritize Impact Over Speed

Once you know why you want to use AI and where it may be applicable, strive to maximize impact rather than speed. The tech startup adage, “Move fast and break things,” is not applicable to everyone. It’s probably not applicable to you. Instead of hastily jumping into the deep end, take a deliberate and discerning approach. In many cases it is better to be right than to be first. 

Our Approach @ Yoko Co

If you’re like 97% of the companies Cisco recently surveyed, you understand the urgency to deploy AI technologies. Yet 86% of those same companies say they are not fully prepared to use AI to its full potential. So how should you adapt to AI? Perhaps we can best answer that question by offering our own approach as a possible path forward. 

We’re no strangers to the complexities of technological change. Our experience and expertise spans a wide range of software stacks and tech trends, and for years we’ve helped clients navigate them all with objectivity and purpose. We explore their unique needs and recommend tailored strategies that align with their mission and values. Our emphasis on long-term planning and strategic adaptability will continue, regardless of what’s popular or trending.

In a world where shiny new tech is flying at us at rapid pace, we recognize the value of human-centered design and ethical management. We reject the idea that quality must be sacrificed for convenience and innovation for integrity. We believe, long term, AI will be an incredible force for good only when we establish comprehensive governance and appropriate guardrails.

Ultimately, our role is to help clients approach artificial intelligence with curiosity and intention, remaining mindful of making incremental progress in pursuit of the long view, ongoing humane improvement.