未加星标

Messaging with RabbitMQ in Node.js

字体大小 | |
[前端(javascript) 所属分类 前端(javascript) | 发布者 店小二04 | 时间 2017 | 作者 红领巾 ] 0人收藏点击收藏

If you are a software developer, you already know what Node.js is. You know how to create an API and may feel like Superman while implementing new features. However, as you attract more users, response time gets slower. With workloads increasing, your application may start to fail. For those with existing applications, this article explores the most likely issues and how message queues, specifically with RabbitMQ in Node.js, effectively helps applications connect and scale. For others planning to build a new application, I incorporate useful tips to help you avoid issues up front.

The Pros and Cons of Node.js

If you have spent significant time working with Node.js or javascript in general, you know that it is single threaded. This means that while executing synchronous operations nothing else is happening. Node.js is particularly well suited for applications that have numerous concurrent connections (up toone million concurrent connections), and each request requires very few CPU cycles. This allows applications to support thousands of concurrent connections as long as very little CPU is used. Other benefits of using JavaScript across a technology stack include:

The same language on the backend and frontend A thriving open source community Relatively fast (mainly due to the fact that V8 compiles it into native machine code) Module-driven development

Additionally, because JavaScript is an extremely popular programming language primarily used in web development, there are numerous readily available libraries. In fact, there are more than 400,000 libraries available onNPM.

The downside of Node.js is that CPU intensive work could slow down or make Node.js completely unresponsive. Although there are plenty of examples on the web with tutorials on how to implement API’s using Node.js, very few address how to deal with making sure Node.js can scale and handle large loads.What approach should be used when you already made significant investments into using Node.js and need to be able to process compute intensive tasks?

The good news is that there are tons of libraries available that can be readily used. The bad news is that there are a lot of cases when third-party code would process data synchronously and could potentially slow down your API’s or make them unusable. Let's explore how message queues come into play.

Message Queue Overview

Offloading work to a separate process is key to building scalable web applications and APIs. You need to be able to transfer compute intensive tasks from the request/response lifecycle. When building applications you can use one of two interaction patterns:

Request-Response (synchronous) : An answer is returned with a response. For example, in e-commerce applications, a user is notified immediately if a submitted order has been processed or if there are any issues. Fire-and-forget (asynchronous) : A request has been received and an alternative means to get the response is provided. For example, when users import a large amount of data that needs to be processed, they receive acknowledgement that the data has been received and instructed to check an import queue to view the status. Or, a message is sent upon import completion.

In both synchronous and asynchronous operations, you can offload processing and free up resources so that the application can handle other requests. This is done using message queue. Message queues come in many shapes and sizes, and can organize the flow of messages in different ways. A very important advantage of using message queues in your project is feature growth, because what may start as a simple project can easily grow into a monster if not planned properly. We suggest considering RabbitMQ .

RabbitMQ Benefits

RabbitMQ is an open source message broker software that implements the Advanced Message Queuing Protocol (AMQP). A message broker is an architectural pattern for message validation, transformation and routing. It gives applications a common platform to send and receive messages and a safe place for messages to live until received.Some of the benefits we've seen from using RabbitMQ include:

Connectability and scalability through messaging (i.e., applications can connect to each other as components of a larger application) Asynchronous messaging (decoupling applications by separating sending and receiving data) Robust messaging for applications Runs on all major operating systems Supports a huge number of developer platforms Open source and commercially supported RabbitMQ Main Concepts

RabbitMQ is a very powerful concept when building applications that need to scale. It is a common pattern when building distributed systems to have a work queue that separates work producers from consumers (workers).

Messages are sent by Producers Messages are delivered to Consumers Messages go through a Channel
Messaging with RabbitMQ in Node.js

There are many messaging patterns, which have been written about ad nauseum. Two noteworthy messaging patterns in the context of RabbitMQ are Competing Consumers and Remote Procedure Call.

Using Rabbit MQ with a competing consumers messaging pattern

To illustrate, consider a user who needs to import a large amount of data. Processing this data is a CPU-intensive task. A naive implementation could be:

User uploads data (request) Server processes the data Respond to the user with a result (reply)

If you try to process this data directly using a Node.js process, you will be blocking event loop. Any requests that come in during data processing will have to wait, which is not good. The system is unresponsive. A better approach would be:

User uploads data (request) Server sends a message to a queue A response is sent to the user with a message that data has been received successfully and will be processed (reply).

The message that was sent to a queue contains all required information (user information, file location,etc.), and will be picked up and processed by an available consumer (worker). The user will be notified after the data has been processed. This approach (fire-and-forget) has several advantages:

The server is able to handle other requests immediately Processing is delayed if there are no resources available at that moment Processing is retried if it fails the first time

In the above example we applied a competing consumer pattern. When there is a message in a queue, any of the consumers could potentially receive it. Simply speaking, consumers compete with each other to be the message receiver.


Messaging with RabbitMQ in Node.js
Using Rabbit MQ with a remote procedure call pattern

In instances when you have to respond synchronously (request/reply) within a reasonable amount of time, but need to do relatively expensive computations, you can use a remote procedure messaging pattern commonly known asRemote Procedure Call (RPC). This entails a producer sending a message and a server replying with a response message as demonstrated in the following figure.


Messaging with RabbitMQ in Node.js
When a producer connects to a RPC queue, it creates an anonymous exclusive callback queue. It then sends a message with a replyTo property, so that the worker knows where to send a response. The worker is waiting for messages from the RPC queue. After the worker is done processing the message it sends a message back to theReply Queue.

本文前端(javascript)相关术语:javascript是什么意思 javascript下载 javascript权威指南 javascript基础教程 javascript 正则表达式 javascript设计模式 javascript高级程序设计 精通javascript javascript教程

主题: Node.jsRabbitMQCPURPCJavaJavaScript
分页:12
转载请注明
本文标题:Messaging with RabbitMQ in Node.js
本站链接:http://www.codesec.net/view/533769.html
分享请点击:


1.凡CodeSecTeam转载的文章,均出自其它媒体或其他官网介绍,目的在于传递更多的信息,并不代表本站赞同其观点和其真实性负责;
2.转载的文章仅代表原创作者观点,与本站无关。其原创性以及文中陈述文字和内容未经本站证实,本站对该文以及其中全部或者部分内容、文字的真实性、完整性、及时性,不作出任何保证或承若;
3.如本站转载稿涉及版权等问题,请作者及时联系本站,我们会及时处理。
登录后可拥有收藏文章、关注作者等权限...
技术大类 技术大类 | 前端(javascript) | 评论(0) | 阅读(89)