And just like that, 2017 is almost gone. Last year, around this time, I wrote a post through which I reflected what I did or did not accomplish in 2016. It is now time to do the same for 2017. A lot happened in 2017 and I would like to take a step back and …
Monthly Archives: December 2017
Implementing a Polynomial Regression Model in Python
So far, we have looked at two types of linear regression models and how to implement them in python using scikit-learn. To recap, we began with a simple linear regression (SLR) model where we have one independent variable (feature) and one dependent variable (label). We then expended it slightly to a more general use case …
Continue reading “Implementing a Polynomial Regression Model in Python”
Analyzing NYC motor vehicle data in Spark
A while back I wrote about analyzing NYC’s traffic (motor vehicle) data in q/kdb+. Then, soon afterwards, I showed how to analyze that data in python using pandas library. Now, I would like to again analyze the same dataset but this time, in Apache Spark. As I mentioned in my last post, I am currently …
Continue reading “Analyzing NYC motor vehicle data in Spark”
Setting up Apache Spark on an AWS EC2 instance
I am currently learning Apache Spark and how to use it for in-memory analytics as well as machine learning (ML). Scikit-learn is a great library for ML but when you want to deploy an ML model in prod to analyze billions of rows (‘big data’), you want to be working with some technology or framework …
Continue reading “Setting up Apache Spark on an AWS EC2 instance”