未加星标

Developing A High Performance Recommendations Engine with Redis and Go

字体大小 | |
[数据库(综合) 所属分类 数据库(综合) | 发布者 店小二03 | 时间 2016 | 作者 红领巾 ] 0人收藏点击收藏

Next generation user facing applications are expected to include a built-in recommendations engine that tells the user what he or she’s likely to “like,” “purchase”, “read” or “listen to” next. Redis , the popular open source, in-memory database known for its in-database analytics capability, and Go , an open source programming language that makes it easy to build reliable and efficient software, combine to deliver a simple, high performance recommendations engine that doesn’t require many system resources. This paper outlines the algorithm and code necessary to implement a collaborative filtering approach to generating recommendations.

The supporting code can be found at https://github.com/RedisLabs/redis-recommend .

What is a recommendations engine?

A recommendations engine is an application or micro-service that presents users with the choices they are most likely to make next. Recommendations could include the next music track a user is likely to want to hear, the next movie that they might watch or the next step they’ll choose while making a reservation.

At a system level, recommendations engines match users with items they are most likely to be interested in. By pushing relevant, tailor-made items to users, application developers can encourage users to purchase relevant items, increase their time spent on a site or in the app, or click on the right ads ultimately helping maximize revenues, usage or eyeballs.

Effective recommendation engines need to meet the following criteria:

Generate the right and relevant choices for their users (this usually depends on the algorithm chosen) Provide high performance, with choices presented to users in real-time Be efficient with system resources, as with any well-written application Approaches to building a recommendations engine

There are two basic approaches for building recommendation engines:

Content-based classification:

This approach relies on classification by a large number of item and user attributes, assigning each user to possible classes of items.

Pros : Can be very targeted Allows detailed control by the system owner Does not require user history Cons: Requires deep knowledge of items Complicated data model A lot of manual work entering items Usually requires users to enter a lot of details Best for: dating, restaurant recommendations, etc.

Collaborative filtering:

This approach taps into user behavior and makes recommendations based on actions made by other users with similar behavior.

Pros: Very generic, content of the item is irrelevant Can generate surprisingly interesting results Cons: Requires a significant level of user history before recommendations are viable an be computationally heavy Best for: movie and music recommendations

Both options are easily implemented with Redis, but the collaborative filtering approach fits real-time use cases best, so we chose this as the basis of our simple Redis recommendation engine.

A Simple Redis recommendation engine written in Go: redis-recommend

This project demonstrates how to build a recommendation engine with Redis, using code written in Go and the Redigo client library.

Redis is an open source (BSD licensed), in-memory database platform store, which can be used as a database, cache and message broker. Redis data structures are like “Lego” building blocks they simplify the implementation of complex functionality, and are extremely efficient because data operations are performed in-memory, right next to where the data is stored, which conserves cpu and network resources.

We will demonstrate how some Redis data structures can tremendously reduce application complexity, while delivering very high performance at high scale. For this engine, we mainly use Redis sorted sets and the associated operations.

Why Use Redis for Recommendations?

If you look at the approaches above, both choices need set operations and sorting, and require that each be done very quickly. With Redis data structures like sorted sets, a solution is extremely easy to implement. Also, with Redis running extremely efficiently in-memory, you don’t have to worry about performance under any load conditions. Compared to a disk-based or RDBMS solution, Redis provides orders of magnitude higher throughput at much lower latencies (<1ms), with very little hardware.

The Chosen Approach

With my collaborative filtering example, the algorithm is simple:

For a given user, find the top similar users by:

Find all users that rated at least one (or N) common items as the user, and use them as candidates For each candidate, calculate a score using the Root Mean Square (RMS) of the difference between their mutual item ratings Store the top similar users for each individual user

Now find the top item recommendations by:

Find all the items that were rated by the user’s top similars, but *have not* yet been rated by the individual user Calculate the average rating for each item Store the top items The Redis Implementation

The main Redis objects in use will be sorted sets. For example, intersect functionality will let us easily find users who rated the same items (zinterstore):


Developing A High Performance Recommendations Engine with Redis and Go

And if we want all the users who rated a group of items (zunionstore):


Developing A High Performance Recommendations Engine with Redis and Go

Let’s follow the logic step by step:

Step 1 insert rating events:

ZADDuser:U:itemsR I ZADDitem:I:scoresR U

Note that for our algorithm, this is all the input data we need!

To get items for a user:

ZRANGEuser:U:items0-1

Step 2 get candidates for similarity:

In order to get the similarity candidates for user (U), we need the union of all the users that

本文数据库(综合)相关术语:系统安全软件

主题: Redis
分页:12
转载请注明
本文标题:Developing A High Performance Recommendations Engine with Redis and Go
本站链接:http://www.codesec.net/view/479686.html
分享请点击:


1.凡CodeSecTeam转载的文章,均出自其它媒体或其他官网介绍,目的在于传递更多的信息,并不代表本站赞同其观点和其真实性负责;
2.转载的文章仅代表原创作者观点,与本站无关。其原创性以及文中陈述文字和内容未经本站证实,本站对该文以及其中全部或者部分内容、文字的真实性、完整性、及时性,不作出任何保证或承若;
3.如本站转载稿涉及版权等问题,请作者及时联系本站,我们会及时处理。
登录后可拥有收藏文章、关注作者等权限...
技术大类 技术大类 | 数据库(综合) | 评论(0) | 阅读(65)