Lately I have been learning about CPU-intensive operations and I/O-intensive operations. The problem I have run into has been: how do I handle long-running tasks in the routes/endpoints. Let’s say that you have a million items you want to show the user, so and you have to iterate over all the items and edit them in some way. Wait, that a crappy example, let’s instead say that you have a ton of statistics and you want to give the user a real-time analysis of that statistics. That is going to take a lot of time. If that happens right in the endpoint that request is going to block the node-server until the process is finished. That’s because of the single-threadedness of node (Yada yada this presentation is still the best thing on the internet to explain nodes event-loop.). This means that all other users have to wait for that request to finish.
While I read about this I came across a few concepts that are useful to know.
This can sometimes be called: long running CPU tasks, CPU intensive tasks or
A program/application/process that is CPU-bound is something that requires a lot of CPU-power. CPU-power is usually needed when doing a lot of calculations.
An application that is I/O-bound is mostly wait for network or database. This is something that Nodejs does really good. Since it can process several requests while it is waiting for a response from the database. That’s why node can be so fast in servering many requests.
So there are conceptually some different servers. One being a web-server.
A web-server is basically a I/O-bound server. The main focus is to server requests as fast as possible. That mean that if you have an operation that is CPU-bound is does not belong in the Web-server. That process probably belongs better in a worker-server.
Possible solutions for CPU-bound operations in a NodeJs-server
Okay, so how to we solve the problem of doing CPU-bound operations on a NodeJS application?
Here are some of the alternatives I have run into while researching this. This list is not exhaustive, and it might not even be correct (I have just been googeling around a bit, I’m no expert).
1. “Offline”-server dedicated to processing data
You can have a server that process the data offline, in the sense that it is not in real-time.
Having a cron-job that calculates on a schedule. For example every half hour or something like that. This is of course very resource consuming. What if nothing has happened and the server process the data again? That would be very unnecessary.
B. Only calculate when an update in the database is done. This could be achieved by having a message sent every time the database gets updated.
C. Do part of the calculation when an update to the database is done. But only process the data since the last process. If this is possible.
There is a pretty good answer on SO about this.
2. Separate server do the CPU-crunching.
Have a separte server do the number crunching. And then just connect the web-server to that endpoint. Since the request is async, other requests will work meanwhile. Since node can work while waiting for a callback.
Here is a blog-post discussing this.
- 3. Using Nodes ClusterModule — Clusters – LoadBalancer with separate processes
- 4. Rabbit queue message
If you know what you are doing and you really want the operation to happen on your web-server. This can be a way.
This trick can make synchronous operations asynchronous. So that the server can attend other easier tasks meanwhile. This is blog exemplified it pretty well.