Shawn Ng

Shawn Ng

Data Engineer

I'm Shawn Ng. I am a Data Engineer from Singapore.

I start this website to keep track of my data and software projects. I hope that you find something useful here.

My main image

What I do

Here is an overview of my technical skill sets

Data Engineering / Science

  • Python
  • Big data (Spark, Hadoop)
  • Data pipeline (Airflow)
  • Database (SQL, NoSQL)
  • API
  • Web scraping
  • Data visualization
  • Machine learning
  • Natural language processing

Cloud Engineering

  • Linux (Bash)
  • Amazon Web Service
  • Google Cloud Platform
  • Heroku

Backend Engineering

  • Django
  • Ruby on Rails
  • Express.js

Frontend Engineering

  • Bootstrap
  • JavaScript


Latest Blog Posts

What I am looking for

What I am looking for

Read more →

Migrate to Unraid from Synology

How I migrate to Unraid from Synology

Read more →

Apache Spark VS Pandas VS Koalas

Apache Spark is an open-source unified analytics engine for large-scale data processing. Pandas is a fast, powerful, flexible and easy to use open source data analysis and manipulation tool, built on top of the Python programming language. Koalas is Pandas API on Apache Spark.

Read more →