In the past three or four years, our concept of what’s “possible” for AI has shifted. Innovations we thought to be Hollywood science fiction have become, well, reality. For instance, Google recently released an API that can read handwriting within images, an unsolvable problem four years ago.

The market has matured, too. We’ve seen the rise of AI in specialized business applications like detecting credit card fraud, identifying smoking-gun documents in lawsuits, categorizing business contracts, and tuning supply chains. Tasks that previously would have taken teams of specialists weeks or months of work, AI now completes in minutes or seconds.

All of a sudden AI is everywhere. It’s a part of our daily lives that we already take for granted, like the facial recognition that unlocks your phone, or the voice commands that trigger digital assistants like Siri and Alexa. No matter the user, business line, or industry, there’s an AI vendor for you.

But the commoditization of AI has created a new problem. When academics talk about AI, they’re usually referring to “deep learning,” an advanced form of neural network shaped by big data. In the software industry, however, it’s an open secret that “AI” by itself means nothing.

Marketers eager to cash in on AI mania have applied the term to everything from Amazon-style recommendation systems (“If you liked this, you might also like this”) to rule-based products that are little more than an extended series of if/then decision trees, the kind novice coders learn in their first weeks of class.

Associating deep learning with such basic technology has made it difficult for AI buyers to know what they’re actually getting and what the solution can actually do. Even with a product demo, cutting through the marketing noise to see what’s under the hood can be downright confusing.

Since the market has gotten much savvier, you as a buyer should get savvy, too. You need to be diligent, and you need to be on your guard. You can go a long way by keeping these three principles in mind.

  1. Know what you want: What do you need AI to do that your current software can’t? What key business outcomes are you looking for? Don’t invest in AI with vague objectives, out of fear of falling behind. Focus on the outcome you want and then back into how to get there. Decide exactly what you want AI to do before you buy: come up with specific requirements and have vendors prove they can meet them. Defining your success criteria before evaluating vendors not only levels the playing field, it also reduces scope creep and gives your team a common selection framework.
  2. Trust, but verify: Once you’ve decided what you want AI to do, make vendors demo their products in front of you. Literally right in front of you. Using your own data. The goal is to figure out how much their software relies on manual training, and how much on automation. To that end, have vendors analyze data you give them on the spot, sight unseen. If they can’t, it’s not necessarily a deal breaker. It doesn’t mean the product isn’t “true AI.” But it is a yellow flag. At minimum, it suggests the AI will require frequent tuning. As a rule, the more an AI can do on the spot, the less maintenance it’s likely to need and the more flexible it’s likely to be.
  3. Watch out for consultants: The point of AI is automation, so take note of any mention of consulting firms or professional services teams. Some AI frameworks need developers to get up and running. But many so-called AIs are actually rule-based systems that need constant attention. One Fortune 500 company I know of spent $1 million on a system that claimed to have added AI, then had to spend 10 times that on professional services teams to build the product. If a system needs regular updates from large teams of consultants, it’s a strong signal you’re not dealing with AI in any meaningful sense.


In my experience, if you scratch someone who has had a bad AI experience, you’ll find someone who was sold a bill of goods—either basic technology posing as something else, or an unrealistic vision of a sci-fi supermachine.

My advice? Be practical. There’s no room for “maybe,” or “possibly,” especially not in times like these. Focus on what you need. Buy only what you’ve seen work. And leave the fantasy to Hollywood.

For guidance on understanding true AI in contract management software, read my company’s enterprise buyer’s guide.