About

Test

Popular posts from this blog

Back to teaching

When I was still a student at Sofia University, Bulgaria back in 1999 I got an assistant position within the Faculty of Mathematics and Informatics. It was so much involving and enjoyable for me that I went on and did it for 6 semesters up till I finished with my bachelor degree. This is not uncommon practice since the “students as teachers” tend to bring a lot of enthusiasm to the classroom. This was a very positive experience. It shaped and improved skills like talking in front of an audience, showing leadership, reaching out and inspiring the inner curiosity, feeling the audience with the most appropriate pace for delivering the content. Of course back then I did not do it with the aim of self improving. I did it for the fun and the prestige in the eyes of friends, colleagues and family that comes with it. I realize the above mentioned added values involved with such activity only later, when looking back and musing over it. And it was a lot of fun, especially after the kind of ...
Read more

Node.js: Optimisations on parsing large JSON file

Image
What if you have 14GB of JSON data, and you want to read it in Node.js and perform a filtering operation. How would you do it? Type of the data For the Backend DB architecture  project, with a friend we generated 15,930,001 posts for a total of 14GB of data. Each post has the following type: export class Post { public client : number ; //ID of the Client public channel : number ; //ID of the Channel public id : number public created_date : number ; public text : string ; public post_type : "TEXT" | "IMAGE" | "VIDEO" | "LINK" ; public labels : string []; //max 10 public insights : number []; //array of 100 integers } Baseline Good benchmark for the minimum time to scan the file is to compare with  the "wc" utility. It yields 12.1 seconds scan time for "\n": time wc -l ../posts.json 15930000 ../posts.json wc -l ../posts.json 8.23 s user 3.32 s system 94 % cpu 12 . 191 total Filtering for...
Read more

Data types: Backend DB architecture

Image
In this post, the data types that represent the entities mentioned in " Backend DB architecture " would be presented. The language is Typescript running in Node. Lets start with the Client: export class Client { public id : number ; public name : string ; labels : string []; constructor () { } } The `labels` array contains all labels that could happen on any of the `Post`s. A `Client` could have any number of `Channel`s associated with it. The `_client_ref` points to the `Client`:: import { Client } from "./client" ; export class Channel { public client : number ; public id : number ; public name : string ; public _client_ref : Client ; constructor ( client : Client ) { this . _client_ref = client ; } } Each `Channel` could have any number of `Post`s: import { Client } from "./client" ; import { Channel } from "./channel" ; export class Post { public client : number ; public chann...
Read more