About

Test

Popular posts from this blog

Node.js: Optimisations on parsing large JSON file

Image
What if you have 14GB of JSON data, and you want to read it in Node.js and perform a filtering operation. How would you do it? Type of the data For the Backend DB architecture  project, with a friend we generated 15,930,001 posts for a total of 14GB of data. Each post has the following type: export class Post { public client : number ; //ID of the Client public channel : number ; //ID of the Channel public id : number public created_date : number ; public text : string ; public post_type : "TEXT" | "IMAGE" | "VIDEO" | "LINK" ; public labels : string []; //max 10 public insights : number []; //array of 100 integers } Baseline Good benchmark for the minimum time to scan the file is to compare with  the "wc" utility. It yields 12.1 seconds scan time for "\n": time wc -l ../posts.json 15930000 ../posts.json wc -l ../posts.json 8.23 s user 3.32 s system 94 % cpu 12 . 191 total Filtering for...
Read more

Backend DB architecture

Image
With a friend, we were trying to prepare for a Facebook general architecture interview question. I proposed we try to solve the following problem (very keen to my heart): Backend architecture for massive post analytics  So basically - selecting a DB that can efficiently query and scale well with the number of users. Data representation of a post Each post has the following meta-data: Customer ID Channel ID (i.e. the Facebook page ID, or Twitter account ID, etc) Created date - date & time when posted Post type - one of "TEXT", "IMAGE", "VIDEO", "LINK" Text Labels Has no labels Has ALL of the specified labels Has ANY of the specified labels The analytics-data is just: 100 integers Sample queries A list of possible queries All posts for a "Channel ID" for the last 30 days All posts for all "Channel IDs" in a "Customer ID" for a given "Post type" for the last 60 days All...
Read more

Back to teaching

When I was still a student at Sofia University, Bulgaria back in 1999 I got an assistant position within the Faculty of Mathematics and Informatics. It was so much involving and enjoyable for me that I went on and did it for 6 semesters up till I finished with my bachelor degree. This is not uncommon practice since the “students as teachers” tend to bring a lot of enthusiasm to the classroom. This was a very positive experience. It shaped and improved skills like talking in front of an audience, showing leadership, reaching out and inspiring the inner curiosity, feeling the audience with the most appropriate pace for delivering the content. Of course back then I did not do it with the aim of self improving. I did it for the fun and the prestige in the eyes of friends, colleagues and family that comes with it. I realize the above mentioned added values involved with such activity only later, when looking back and musing over it. And it was a lot of fun, especially after the kind of ...
Read more