Posts

Tesla investment

Image
I've never bought a stock before. My background is that I am naturally drawn to popular science and engineering. And among many others I follow Elon Musk's SpaceX and Tesla development. This blog post is intended for my future self, to note down the reasons why I am committing to buying Tesla stocks. Working patent model of alternating current induction motor invented by Nikola Tesla in 1887-1888. source:  MediaWiki Please take those notes with skepticism and critical thinking. I am biased and newbie. Tesla strong points This section list the strong points of Tesla as a longer term investment from my point of view. Elon Musk Obviously mr. Musk is a quite strong character and driver for investing in his company. With his strong innovative mind, charming in his own way personality, powerful leader, obsessive workaholic and ludicrous marvel in marketing. He likes to create things, "engineer" and "designer" is his primary focus. I feel that he ...

Node 9.X IPC Performance

Image
I wanted to perform image analysis on a public camera overlooking one of the busiest boulevards in Sofia, Bulgaria: The overall goal is to clip just the part of the image for a single direction of the boulevard and count the number of vehicles passing by in the field of view. Then index the counts aggregated on minute, hour and daily intervals to get a nice overview of the traffic ebbs and dips. I chose to do it in Node.js and Typescript. The architecture of the components is: The "Image Downloader & JPEG decoder" component has the following performance markers: around 200ms to download around 60KB JPEG image on a public WiFi in a cafe.  around 83ms on a MacBookPro (2017) to unpack the JPEG into Buffer of bytes. The image has resolution 1280px by 720px. Each pixel is represented with 4 bytes, for a total of 3.5MB of memory of unpacked Uint8Array. This gives total time for download and decompression between 400 and 500 ms, and a new image ready for analys...

Bulk delete users in Fireabse

Firebase  is nice google owned service that lets you use database right from the browser, not much need for creating your own APIs. It is NoSQL type of DBs and you are responsible for building your indexes .. but this is not the point of this topic. It turns out that from the Authentication section it is not possible to bulk delete users. I have let the signInAnonymosly  while testing - and only to realize after some months that there are more than 800 users.. most of them just empty. It would be crazy to manually click and delete each of this users. A hackish way to do this is via the node firebase-admin package . After installing and configuring the private key, I wrote the following script: var admin = require ( " firebase-admin " ) ; var serviceAccount = require ( " ./admin_private_key/racket-tournaments-firebase-adminsdk-fheco-949023672f.json " ) ; admin . initializeApp ( { credential : admin . credential . cert ( serviceAccount ) , databaseU...

Automating page scraping with Selenium and Chrome

Image
Sometimes you are interested in information on some page, be it on some social media, or on a news site, or a blog. But you do not have an automated way via API to access this data. I.e. you want to stock the public FB posts on the girl that did not accept your friend request. Or you want to monitor a site for product availabily or price range. Or you might want to get notified if the university web site pushes a new update. Be warned that it might be agains the respective sites policy to make such automated scannings, or you might risk suspending your account or legal proceedings. But lately I have the need to monitor a web site for auto parts. I was interested in the availability of a certain part for my car. I wanted to get notified when there is something new. The vendor does not have API or RSS feeds, but claims that the site is updated the moment there is a new part available and the moment when an old part is sold. Fortunatelly the filter for figuring out the car, model, yea...

Upload file to Google Storage from memory with Node.js gcloud library

These days was interacting with the Google Cloud Storage (GCS) with the gcloud nodejs library(version 0.31.0) and was surprised that there is no API method to directly upload file from memory. So if you are looking for it - the shorthand is just to use the write stream returned from createWriteStream() on file object: var gcloud = require( "gcloud" ); var storage = gcloud.storage({projectId: "dir-bg-scraper" }); var sample_TXT_file = "Hello there, \r \n " + "This is sample text file with important info \r \n \r \n " + "Best Regards!" ; storage.createBucket( "dir-bg-scraper" , function (err, bucket, apiResponse) { if (err === null ) { // bucket is the newly created "Test-Bucket" console .log ( "'dir-bg-scraper.appspot.com' successfully created!" ); // create 'README.md' file into this bucket var new_file = bucket.file(...

Google Datastore Namespaces

Image
The Google Datastore is one of the Google publicly available scallable No-SQL solution. The other is the BigTable  - which can be installed on a swarm of Compute Engine virtual machines. The Datastore is build on top of the BigTable. So with the BigTable you could get the raw performance, but you miss some goodies, like transactions and SQL like query language available with the Datastore. The Datastore terminology gets me confused. They introduce "Dataset" and "Namespace" - which is not immediately clear, and had me some issues till I figure it out. That is why started this post. From the welcome page of the Datastore project the terminology is defined as Concept Datastore Relational database Category of object Kind Table One object Entity Row Individual data for an object Property Field Unique ID for an object Key Primary key But the Dataset is not defined what it means in the documentation. Through trial and error I found out that the Dataset is the ...

Google Datastore intro

Image
I have been getting updated on the Google Datastore for several months now. It seems the major promotions for it started back in 2008, or at least that is when the most videos on Google IO started to appear. It got some hype until 2012, and then no more promotions from Google - they focus now on mobile, web and material designs. Anyway, I've watched multiple time the following videos - and recommend them if you want to know more about scallable, sorted, distributed, persisted nosql solution "Datastore under the covers", 2008 talk.  The slides are a mess, but Ryann Barret makes a good introduction  to how the entities and inded are layed out, and how transactions work. "Scalable, Complex Apps on App Engine" 2009 talk Brett Slatkin presents solutions for complex datastructures  having in mind the limitations of the Datastore Other notables videos to checkot are: Google I/O 2010 - Next gen queries Google I/O 2011: More 9s Please: Under Th...