When creating predictive models, it's important to measure accuracy to be able to clearly articulate how good the model is. This article talks about two mistakes that are commonly made when measuring these accuracy values.
1. Measuring Accuracy on the Same Data Used for Training
One common mistake that gets made is measuring the accuracy of the same data that was trained. For example, say you have data from 2017 and 2018 for customer churn. Say you feed all that data to train the model and subsequently use the same data to predict and compare the predictions with the actual results. That is like you are given a question paper before the exam to study at home and the exact same question paper was given to you the next day in the exam. Obviously, that person is going to do great in the exam.