Node 9.X IPC Performance

-
I wanted to perform image analysis on a public camera overlooking one of the busiest boulevards in Sofia, Bulgaria:

The overall goal is to clip just the part of the image for a single direction of the boulevard and count the number of vehicles passing by in the field of view. Then index the counts aggregated on minute, hour and daily intervals to get a nice overview of the traffic ebbs and dips.

I chose to do it in Node.js and Typescript. The architecture of the components is:

The "Image Downloader & JPEG decoder" component has the following performance markers:
  • around 200ms to download around 60KB JPEG image on a public WiFi in a cafe. 
  • around 83ms on a MacBookPro (2017) to unpack the JPEG into Buffer of bytes. The image has resolution 1280px by 720px. Each pixel is represented with 4 bytes, for a total of 3.5MB of memory of unpacked Uint8Array.
This gives total time for download and decompression between 400 and 500 ms, and a new image ready for analysis roughly every half second:
Sat Apr 14 2018 11:22:19 GMT+0300 (EEST): Image(1280 x 720) processed in 403.811349 ms
Sat Apr 14 2018 11:22:20 GMT+0300 (EEST): Image(1280 x 720) processed in 305.733611 ms
Sat Apr 14 2018 11:22:21 GMT+0300 (EEST): Image(1280 x 720) processed in 484.65647 ms
Sat Apr 14 2018 11:22:21 GMT+0300 (EEST): Image(1280 x 720) processed in 306.6703 ms
Sat Apr 14 2018 11:22:21 GMT+0300 (EEST): Image(1280 x 720) processed in 283.651022 ms
Sat Apr 14 2018 11:22:22 GMT+0300 (EEST): Image(1280 x 720) processed in 310.54076 ms
Sat Apr 14 2018 11:22:22 GMT+0300 (EEST): Image(1280 x 720) processed in 301.422415 ms
Sat Apr 14 2018 11:22:23 GMT+0300 (EEST): Image(1280 x 720) processed in 267.434492 ms
Sat Apr 14 2018 11:22:23 GMT+0300 (EEST): Image(1280 x 720) processed in 277.57939 ms
Sat Apr 14 2018 11:22:24 GMT+0300 (EEST): Image(1280 x 720) processed in 280.239104 ms
Sat Apr 14 2018 11:22:24 GMT+0300 (EEST): Image(1280 x 720) processed in 277.993232 ms
Sat Apr 14 2018 11:22:25 GMT+0300 (EEST): Image(1280 x 720) processed in 263.314174 ms


Obviously Node.js is single threaded but very efficient in downloading resources with its event based model. Practically the thread is free to perform any other task for those 200ms when the image is being transmitted over.

But the JPEG decoding (see jpeg-js) is CPU intensive. In this case Node.js single thread is completely blocked. So I decided to experiment with Node.js Cluster API and to spawn a separate process to host the "Image Downloader & JPEG decoder" component. In this way the CPU on the main thread will be completely free in multi CPU hosts to perform the image analysis.

So I created a Cluster with the master process just waiting for messages and printing performance timings when such messages arrives. The spawned process will download, JPEG decode and send an IPC message with process.send(). 

I got the following timings:
Master 10167 is running
Image Downloader process(PID=10168) started
2018-04-14T07:51:13.145Z: Image(1280 x 720) processed in 386.500649 ms
2018-04-14T07:51:16.523Z: Image(1280 x 720) processed in 5.429576 ms
2018-04-14T07:51:21.078Z: Image(1280 x 720) processed in 713.327255 ms
2018-04-14T07:51:24.215Z: Image(1280 x 720) processed in 207.49122 ms
2018-04-14T07:51:27.152Z: Image(1280 x 720) processed in 229.344572 ms
2018-04-14T07:51:30.090Z: Image(1280 x 720) processed in 848.952361 ms

The image processing (download & decode) is roughly the same. But.. hh.. wait.. what.. why the main thread gets a new image only once per 3 seconds? It turns out that IPC does not come for free. The spawned process must encode, transmit and then decode those 3.5MB of data to the master process. 

This result defeats the purpose of having a separate thread/process handle the image decoding. Since the IPC communication on a big file is roughly order of magnitude more expensive. When using Cluster API in Node.js for your architecture have in mind that "heavy" IPC message will actually downgrade the performance. You should use it for pooling sockets and exchanging light messages only.

Comments

Post a Comment

Popular posts from this blog

Data types: Backend DB architecture

-
Image
In this post, the data types that represent the entities mentioned in " Backend DB architecture " would be presented. The language is Typescript running in Node. Lets start with the Client: export class Client { public id : number ; public name : string ; labels : string []; constructor () { } } The `labels` array contains all labels that could happen on any of the `Post`s. A `Client` could have any number of `Channel`s associated with it. The `_client_ref` points to the `Client`:: import { Client } from "./client" ; export class Channel { public client : number ; public id : number ; public name : string ; public _client_ref : Client ; constructor ( client : Client ) { this . _client_ref = client ; } } Each `Channel` could have any number of `Post`s: import { Client } from "./client" ; import { Channel } from "./channel" ; export class Post { public client : number ; public chann...
Read more

Node.js: Optimisations on parsing large JSON file

-
Image
What if you have 14GB of JSON data, and you want to read it in Node.js and perform a filtering operation. How would you do it? Type of the data For the Backend DB architecture  project, with a friend we generated 15,930,001 posts for a total of 14GB of data. Each post has the following type: export class Post { public client : number ; //ID of the Client public channel : number ; //ID of the Channel public id : number public created_date : number ; public text : string ; public post_type : "TEXT" | "IMAGE" | "VIDEO" | "LINK" ; public labels : string []; //max 10 public insights : number []; //array of 100 integers } Baseline Good benchmark for the minimum time to scan the file is to compare with  the "wc" utility. It yields 12.1 seconds scan time for "\n": time wc -l ../posts.json 15930000 ../posts.json wc -l ../posts.json 8.23 s user 3.32 s system 94 % cpu 12 . 191 total Filtering for...
Read more

Back to teaching

-
When I was still a student at Sofia University, Bulgaria back in 1999 I got an assistant position within the Faculty of Mathematics and Informatics. It was so much involving and enjoyable for me that I went on and did it for 6 semesters up till I finished with my bachelor degree. This is not uncommon practice since the “students as teachers” tend to bring a lot of enthusiasm to the classroom. This was a very positive experience. It shaped and improved skills like talking in front of an audience, showing leadership, reaching out and inspiring the inner curiosity, feeling the audience with the most appropriate pace for delivering the content. Of course back then I did not do it with the aim of self improving. I did it for the fun and the prestige in the eyes of friends, colleagues and family that comes with it. I realize the above mentioned added values involved with such activity only later, when looking back and musing over it. And it was a lot of fun, especially after the kind of ...
Read more