On monitoring of machine learning models

. In this article, we want to focus on monitoring machine learning models. The practical use of machine learning systems includes several fairly standard steps that do not depend on the subject area. These stages of the pipeline are described in sufficient detail, all descriptions of machine learning systems begin with them and include steps such as preparing the model, selecting training data, training the model, and testing it (putting it into operation). At the same time, such a stage as monitoring during the operation phase is almost always excluded from consideration. Meanwhile, this moment plays a key role, for example, to ensure the stability of the machine learning system. This article discusses the practical issues of building monitoring for machine learning systems.