AXA Direct Assurance Increases Its Growth Rate and Employee Productivity With MLOps

1. Code development and quality – to improve code robustness with best coding practices and tools.
   
   Moving from Jupyter notebooks to Python code that can be tested, and that people can understand and execute themselves, increased productivity and minimized handover issues. Quality is ensured by setting up a CI/CD pipeline that executes unit and integration tests and formatting and linting checks. 
2. Code execution automation – to allow automatic code execution with improved traceability, transparency, and lineage of the process. 
   
   The team focused on automating the process, all the way from collecting data to the final production model.  All data extraction and model training pipelines are implemented as Databricks jobs and run on designated job clusters. The pipeline jobs themselves are defined using versioned JSON files, allowing automatic replication/repair when accidentally destroyed, and environment management.
   
3. Data, experiment, and model tracking – to enable automatic tracking of data and model versions to facilitate reusability and reproducibility of results.
   
   Nelli and Jeroen helped Direct Assurance set up MLflow tracking and a data versioning workflow. Each Databricks job is given a unique identifier that is propagated to data sources as version ID, stored as metadata in the MLflow experiment, and connected to the model version. Each model is logged to the model registry in MLflow, which allows metrics across different models to be easily compared and the best model promoted to the production environment. This allows the data team to track how different changes in the model parameters or in the data affected performance, and to reproduce past results.  

Apart from the points mentioned above, Direct Assurance was explicitly interested in measuring potential business gains that automatic and more frequent retraining of the models would bring.