This content is locked. Please login or become a member.
Accuracy
When we’re talking about predictive AI or or you want to call it predictive analytics, predictive basically means generating a probability. So accuracy is the metric you always hear about. How well does it predict? Well, it’s highly accurate. It’s got an accuracy of ninety percent of ninety-eight percent. This is amazing. Usually, the thing we’re trying to predict, what are called the positive rather than negative cases, is the thing that happens less often. In the case of fraud, it happens very infrequently. If point one percent of transactions are fraudulent, that means that nine hundred ninety-nine out of a thousand are legitimate.
If I have a dumb predictive model and it just always says “not fraudulent, not fraudulent;” Of course, it never correctly identifies a fraudulent transaction, but it’s literally ninety-nine point nine percent accurate. This incredibly dumb model is incredibly accurate. Accuracy is just how often am I correct for both positive and negative. There’s no differentiation.
Lift
A metric called lift, which is a predictive multiplier, differentiates between positive and negative. And what it says is like, look. This model can identify a pocket of cases that are three times more likely, five times more likely than the average, to be positive. So for example, fraud detection. You identify a small pocket that have six times the prevalence of fraud. Well, if fraud was only one in a thousand, now within this pocket, it’s still only six in a thousand, but it’s six times more often. That’s actually really, really valuable. Essentially, you’re looking for a needle in a haystack, and you just made the haystack substantially smaller. That turns out to be really, really valuable for the business.