Wrap-up of the RecSys Summer School

From 12 to 16 June, several researchers from the Media Economics & Policy Unit took part in the Recommender Systems Summer School in Copenhagen. During this Summer School, academics and industry leaders lectured on the practice, research, and state of the art in recommender systems. The lectures covered a broad range of topics from an algorithmic as well as a methodological perspective, including hands-on sessions. This week brought many interesting insights for our strategic research program on recommender systems and this short report highlights some of the key takeaways.

Not an algorithm, but a system

Recommender systems encompass more than just the algorithm itself. They require the thoughtful implementation of (a combination of several) algorithms that align with specific domain objectives, account for optimal user experience in different contexts, and seek to create mutual value for various stakeholders

It is crucial to recognize that there is no one-size-fits-all approach to recommender systems. The strategy employed must be tailored to the specific domain in which the system operates, considering the unique objectives, economic factors, values and challenges inherent to that domain. For instance, in the realm of news, evaluating the system's quality necessitates considerations such as diversity and serendipity, speed and coverage.

The recommender system design must consider the overall user experience. This involves understanding the various features and styles influencing users’ interaction with the platform, as well as analyzing the user journey and recognizing the impact of contextual dependencies on their preferences and needs.

In both academic literature and practical applications, the primary goal of recommendation systems is to create value. This entails a reciprocal relationship between user values and business values. By addressing user needs and providing quality personalized recommendations that go beyond mere accuracy the system increases short-term engagement. Furthermore, these systems aim to cultivate long-term loyalty and build strong relationships with the target audience. In summary, well-designed recommendation systems enhance user engagement by offering personalized recommendations, optimizing the user experience, reducing information overload, fostering serendipitous discovery, continuously learning and improving, integrating social features, and thereby generate added value for business through increased user satisfaction, retention, and potential revenue growth.

End of the artificial clean cut between content?

Media mergers are changing the industry. One of the many examples is how RTL XL will become a part of Videoland. The question rises how the video-on demand platforms can blend different content types such as movies, series, TV programs, short clips and livestreams in an appealing way. An answer to that question could be answered by formulating the optimal user experience through recommender systems.

Currently, the company is analyzing user behaviour in order to develop models that can be used in online user experiments. A next step will be to start A-B testing to create the optimal recommendation model.

The goal of VOD platforms is to have loyal visitors, but it is a metric that moves very slowly. Currently, the recommendation system of the RTL is built up on three different types of recommendations: content-based, popular within the genre and collaborative filtering. Interesting fact: Personalized swimming lanes compared to editorials swimming lanes generate 30 min more viewing time per active user per month.

The main goal of “this recommender optimizing project” is to work towards continuous loops. Sequential recommender systems are different in that sense that they convert user’s behavior trajectory into recommended items or services. It takes into account the current and recent preferences of a user for a more accurate recommendations.

Implementing this new form of recommender system will be one of the key factors to generate a user interface with “blended content” that answers the customer’s needs.  

A critical stance in the evaluation of recommenders 

Being grounded in dominantly quantitative forms of assessment, recommender system evaluation needs to pay sufficient attention to real-world significance of numerical results and to whether outcomes actually make sense in applied cases. For instance, is an overall increase in prediction accuracy of a recommendation an accomplishment when large parts of the user base still receive bad recommendations and their preferences remain ill-defined? A call for qualitative sense-making of quantitative evaluation outcomes was certainly made at the Summer School.

Also, research papers that seek to evaluate recommender systems often focus more on (incremental) increases in performance percentages than on a solid basis for their actual evaluation. In terms of relevance for the academic field, consequently, little contributions are made. Papers that are characterised by vagueness and technical complexity thus prevent real progress and cannot form the basis for further research. The lack of valuable longitudinal research in the domain of recommender systems can also be related to this.

Lastly, by acknowledging that recommender systems impact not only the intended end-user and that their complex nature implies influences also on those not directly involved, we argue that multiple-stakeholder considerations should be the norm. We realize that the inherent complexity of recommender systems makes this a difficult endeavour. But in our attempts to evade the McNamara fallacy and to research in the most holistic way possible, the aim should always be to keep a multi-stakeholder involvement top-of-mind. Not solely in phases of design or evaluation, but as a constant reflective thought from the outset and throughout.