Deploying AI instantly brought value and growth to many businesses. However, it is well established that sustaining the value over time, not to mention maximizing it, could be quite challenging. Continuous optimization is the key to successful AI deployments. Begin with a product that’s good enough, learn from how it performs in the real world (especially as the data environment changes), and then improve; then learn and improve again, and so on. It’s a bit of an obvious insight, but it is rare for AI-driven products to be perfect from day one.
To accomplish continuous optimization you need continuous feedback. You need “eyes and ears” observing your data and models and telling you whether they’re performing well. That’s easier said than done, for various reasons. These reasons are outlined below.