The Solution
Bas Harenslak, Data Engineer at Xebia Data and AI, and Laurens Knoll, Cloud Engineer at Xebia, started to develop the infrastructure, the data platform, and the data processes. Bas explains: “We started from scratch with AWS. In addition to security, we paid particular attention to scalability and the greatest possible simplicity of use.” Pathé’s data is made available on the data platform via the AWS service RDS. For the notebooks and training of the models, Xebia chose to implement AWS Sagemaker, and ECS hosts the models. “Every day, we load the data of the latest movies to train the recommender. Then we deploy this model to Sagemaker again. Via the ETL pipeline, we ensure the data arrives in the right form at the right place. This is how we make data available to the rest of the organization,” Bas explains.
Security
The new platform uses managed services from AWS for the most part. However, to control the replication of both static and streaming data from the POS systems, the Xebia engineers wrote a custom solution.
“Looking at the structure of the data platform as a cloud engineer, you get a very familiar feeling. In addition to the standard components, the way we created the connections stands out,” says Laurens. “To reduce the dependency on third parties, we wrote the coupling ourselves. We wanted to ensure that Pathé could always retrieve the data. When usage increases in the future, it will still be able to access data on demand. This gives Pathé maximum control over all its processes,” Bas adds.
Pathé’s data platform has been implemented according to the standards of the AWS Well-Architected Framework.” In this framework, security is about compliance and safe daily use. To achieve the latter, we focused on ‘keeping people away.’ Only the data engineers have access to the data, and processing is automated as much as possible,” says Bas.
Applications
“From a list of possible applications, we started with a recommender and a visitor forecast,” Rick says. The recommender was a real high potential for Pathé. “If you have seen movies one, two, three, you will probably like movie four. By comparing your viewing history with people who have a similar search or viewing history or making selections based on your favorite actors or genres, we can recommend new movies. I’ve made sure these models go to production in an automated pipeline,” Bas explains.
Another high potential was predicting the number of future visitors. Pathé was familiar with analyzing the number of visitors per theater for purchasing and planning purposes. The Xebia consultants started working with the available (historical) data to develop a model to help them do this with AWS Sagemaker. “We want to predict the number of visitors per day and theater for six weeks, for staff planning, among other things. In addition, we want to use a long-term forecast to respond to trends together with distributors,” Rick adds.
The Data Squad
An (external) Data Squad worked on Pathé’s new data goals: Bas and Laurens took responsibility for the infrastructure and bringing models to production, were joined by two other Data Scientists. Rick supported the Squad by helping them find data or write queries together, with Bas and Diederik often challenging them by asking how they could do things faster or smarter.
To achieve the goals, the Data Squad also worked very closely with the content team (film programming) and the e-commerce department, and there was a lot of testing. Pathé has 30 theaters, but also 2 million visitors in its systems. “We were already sending these movie tips by email, but we see, after a lot of testing, that the conversion rates have improved significantly by using the recommender,” adds Rik.
Finally, Pathé uses Cloud Control. “This is incredibly valuable to them because they are still building the knowledge and skills of working with the cloud. Xebia, as a partner, can supplement and guide them in building up this know-how properly and ultimately continuing to run this platform safely and securely. Pathé has a great love for movies but is just as passionate about making this platform a success,” says Laurens.
