Understanding sets in python

As I learn more and more about python’s different data types, I find myself surprised that not enough people use (or even know) sets. At my job, I am often taking some data and transforming it. Once transformed, I have to do analysis on how the data may have changed and sets are great for such comparisons.

In this post, I will cover how to create sets and show some examples on how to use them.

What is a set?
A set is an unordered collection of unique items in python. They are sort-of like lists but they only contain unique items and don’t maintain order. They also have a lot of helpful unique operations.


Book Review: The WSJ Complete Money and Investing Guidebook

Note: You can now subscribe to my blog updates here to receive latest updates.

If you had asked me to discuss bonds on the first day of my first job, I would have probably started talking about ionic or covalent bonds that I had learnt in  my high school chemistry class. I knew close to nothing about finance and financial securities. Derivatives only reminded me of derivation and integration. Options were of no significance and I was clueless about trade and quote data. Now that I look back, I wish someone had given me The WSJ Complete Money and Investing Guidebook (referred to as CM&IG from henceforth) as the first thing to read.

(Side note: I was given a book to read by my boss and it was q for mortals, so I could learn q.)

This ~200 page book is written by Dave Kansas and covers every asset class and major investment vehicles. The book never goes in-depth into any of the topics which I really like. It also assumes that you have barely any knowledge of the financial markets and investing.


Understanding list, set and dict comprehensions

Just few days ago, you were having a good time with your friends and counting down to 2017. Few days have passed and you are left with a typical cold snowy day in January. You are busy writing code for a high profile project at work. Suddenly, a situation arises where you need to create a new list from an existing one. You code it like you have always been coding:

>>> old = ['adam', 'mike', 'olga']
>>> for name in old:
        new.append(name+ ' last')
>>> new
['adam last', 'mike last', 'olga last']

But then you realize that one of your 5 new year’s resolutions is to start using list comprehensions! You have heard about them but were always a little intimidated by them. You were also not really sure of their point.

This post will help you with your new year’s resolution. However, it won’t help you with the other one about going to gym 4 times a week.


2016: Year in Review

I started EnlistQ in December 2014 as a way for me to express my thoughts and to reinforce technical concepts that I was learning (on my job or elsewhere). This is the first time I have thought of doing a review of my year. It probably has to do with the fact that a lot has changed this year – EnlistQ, my personal and professional life, and the World! All these changes convinced me to take a step back and look at what I have accomplished this year.

Let’s begin!


10 python idioms to help you improve your code

If you have ever tried to learn a new language (not a programming language), you know that we always think in our native language before we translate it to the new language. This can lead to you forming some sentences that don’t make sense in the new language but are perfectly normal in your native language. For example, in a lot of languages, you ‘open’ an electronic gadget such as fan, AC or cell phone. When you say that in English, it means to literally open the gadget instead of turning it on.

The same is true for programming languages. As we pick up new languages, such as python, we are using our prior knowledge of programming in another language (q, java, c++ etc) and translating that to python. Many times, your code will work but it won’t be ‘pretty’ or fast. In python terms, your code won’t be ‘pythonic’.

In this post, I would like to cover some python idioms that can be very helpful. These idioms will:

  1. Help your code look better,
  2. Speed up your code, and
  3. Set you apart from beginners

Let’s begin!

Note: All examples are written in python 2.

Update: Thanks to Diane and my other readers for pointing out some errors in my examples!


A brief history of ECNs

What comes to your mind when someone mentions ‘ECNs’? Stock exchanges? NYSE and NASDAQ? Prior to reading Scott Patterson’s Dark Pools, I knew very little about Alternate Trading Systems (ATS) and how famous public exchanges such as NYSE and NASDAQ came to be.  This post is not meant to be a detailed history of (US) trading venues but instead to provide you with a brief overview of different ECNs that exist or existed and how they led to today’s behemoth trading venues.

What are ECNs?

ECN stands for electronic communication network. An ECN is an automated system where client orders are matched. ECNs eliminate the need to interact with a middleman, such as an exchange market maker, to buy or sell securities. ECNs earn money by charging users a fee for each transaction. While public exchanges have fixed hours dedicated to trading, ECNs are usually open 24 hours and hence, facilitate after-hours trading.


AquaQ launch event on Nov 17th

Over the past few years, I have worked with many consultants. Some can be very easy to work with while others can be a little challenging. Some of the consultants I worked with were from a company called AquaQ Analytics and they were probably the best ones I have worked with.

For those who don’t know, AquaQ is a consulting company that specializes in data management, data analytics and data mining services (mainly through kdb+). They are also very good with sharing what they have been working on. For example, few years ago, they released TorQ which is a framework for setting up kdb+ stack easily (RDB, HDB etc).


Book Review: Dark Pools by Scott Patterson

Every day, I work on a platform that captures intraday and end-of-day tick data from multiple exchanges. Some of them are very popular like NYSE and NASDAQ but they themselves have several different feeds such as NYSE ARCA, NYSE MKT and NASDAQ TOTALVIEW-ITCH. Then, there are some other ones that are not as popular but still very significant in terms of the daily market volume they handle. For example, BATS has two exchanges BATS BYX and BZX and Direct Edge had Direct EDGA and Direct EDGX.

Before I read Scott Patterson’s Dark Pools, I knew very little about these exchanges and their history. How did they come into existence? What is NYSE Arca? What does Arca stand for? What about NASDAQ TOTALVIEW-ITCH?

When my colleague recommended the book to me, I thought the book would be about, well, dark pools. As in, these mysterious, secretive pools created by companies (mainly investment banks) for trading. They are considered ‘dark’ because they don’t disclose what transactions take place in the pool. Instead the book was about ‘lit pools’ or rather, the quest of a man called, Josh Levine, to make the US exchanges as transparent as possible.


Book Review: Too Big To Fail by Andrew Ross Sorkin

For anyone working in the financial industry, the events that unfolded in 2007/2008 were nothing but extraordinary. During that time, I had just started my first semester in college and barely knew anything about the financial industry. Every day, I would catch the express train and read about another bank either being acquired or going bankrupt. Soon, it wasn’t just banks in US but globally. After reading about all these banks biting the dust, I would dispose of the newspaper and go back to my day as if nothing had happened.

Things are different now. I have worked at a sell side and a buy side and I can’t afford to not know the details of the financial crisis. I knew bits and pieces after reading so many articles over the years but I needed one detailed source that would walk me through all the major events of the crisis. And this is where Too Big To Fail (TBTF) by Andrew Ross Sorkin comes in. TBTF is an amazing book which covers the financial crisis in extraordinary amount of detail. Sorkin does a wonderful job by giving his readers insights into all the meetings that were held at the Fed, Treasury and investment banks by some very powerful people. Sorkin provides his readers with a great opportunity to know the details of the conference calls that were being held to decide whether or not to takeover a bank, whether or not a company needs to declare bankruptcy before market opens the next day, and whether or not Congress will pass a new bill that can potentially put an end to the crisis.


Getting started with regex in python

I have been wanting to learn regex. Not just because it reminds me of a bit but because it’s actually very useful and can be used within numerous languages including python. In case you don’t know, regex stands for regular expression. A regex is “a sequence of characters that define a search pattern.” Most popular use case is to search strings for a pattern.

I was looking at videos from this year’s PyCon and stumbled up on a video of Trey Hunner conducting regex workshop at PyCon 2016. If you are looking to learn regex, I encourage you to watch the video. You can also find most of the stuff mentioned in the video on this website.

After I watched the video, I did some practice on my own which I wanted to share here.