10 python idioms to help you improve your code

If you have ever tried to learn a new language (not a programming language), you know that we always think in our native language before we translate it to the new language. This can lead to you forming some sentences that don’t make sense in the new language but are perfectly normal in your native language. For example, in a lot of languages, you ‘open’ an electronic gadget such as fan, AC or cell phone. When you say that in English, it means to literally open the gadget instead of turning it on.

The same is true for programming languages. As we pick up new languages, such as python, we are using our prior knowledge of programming in another language (q, java, c++ etc) and translating that to python. Many times, your code will work but it won’t be ‘pretty’ or fast. In python terms, your code won’t be ‘pythonic’.

In this post, I would like to cover some python idioms that can be very helpful. These idioms will:

  1. Help your code look better,
  2. Speed up your code, and
  3. Set you apart from beginners

Let’s begin!

Note: All examples are written in python 2.

Update: Thanks to Diane and my other readers for pointing out some errors in my examples!


A brief history of ECNs

What comes to your mind when someone mentions ‘ECNs’? Stock exchanges? NYSE and NASDAQ? Prior to reading Scott Patterson’s Dark Pools, I knew very little about Alternate Trading Systems (ATS) and how famous public exchanges such as NYSE and NASDAQ came to be.  This post is not meant to be a detailed history of (US) trading venues but instead to provide you with a brief overview of different ECNs that exist or existed and how they led to today’s behemoth trading venues.

What are ECNs?

ECN stands for electronic communication network. An ECN is an automated system where client orders are matched. ECNs eliminate the need to interact with a middleman, such as an exchange market maker, to buy or sell securities. ECNs earn money by charging users a fee for each transaction. While public exchanges have fixed hours dedicated to trading, ECNs are usually open 24 hours and hence, facilitate after-hours trading.


AquaQ launch event on Nov 17th

Over the past few years, I have worked with many consultants. Some can be very easy to work with while others can be a little challenging. Some of the consultants I worked with were from a company called AquaQ Analytics and they were probably the best ones I have worked with.

For those who don’t know, AquaQ is a consulting company that specializes in data management, data analytics and data mining services (mainly through kdb+). They are also very good with sharing what they have been working on. For example, few years ago, they released TorQ which is a framework for setting up kdb+ stack easily (RDB, HDB etc).


Book Review: Dark Pools by Scott Patterson

Every day, I work on a platform that captures intraday and end-of-day tick data from multiple exchanges. Some of them are very popular like NYSE and NASDAQ but they themselves have several different feeds such as NYSE ARCA, NYSE MKT and NASDAQ TOTALVIEW-ITCH. Then, there are some other ones that are not as popular but still very significant in terms of the daily market volume they handle. For example, BATS has two exchanges BATS BYX and BZX and Direct Edge had Direct EDGA and Direct EDGX.

Before I read Scott Patterson’s Dark Pools, I knew very little about these exchanges and their history. How did they come into existence? What is NYSE Arca? What does Arca stand for? What about NASDAQ TOTALVIEW-ITCH?

When my colleague recommended the book to me, I thought the book would be about, well, dark pools. As in, these mysterious, secretive pools created by companies (mainly investment banks) for trading. They are considered ‘dark’ because they don’t disclose what transactions take place in the pool. Instead the book was about ‘lit pools’ or rather, the quest of a man called, Josh Levine, to make the US exchanges as transparent as possible.


Book Review: Too Big To Fail by Andrew Ross Sorkin

For anyone working in the financial industry, the events that unfolded in 2007/2008 were nothing but extraordinary. During that time, I had just started my first semester in college and barely knew anything about the financial industry. Every day, I would catch the express train and read about another bank either being acquired or going bankrupt. Soon, it wasn’t just banks in US but globally. After reading about all these banks biting the dust, I would dispose of the newspaper and go back to my day as if nothing had happened.

Things are different now. I have worked at a sell side and a buy side and I can’t afford to not know the details of the financial crisis. I knew bits and pieces after reading so many articles over the years but I needed one detailed source that would walk me through all the major events of the crisis. And this is where Too Big To Fail (TBTF) by Andrew Ross Sorkin comes in. TBTF is an amazing book which covers the financial crisis in extraordinary amount of detail. Sorkin does a wonderful job by giving his readers insights into all the meetings that were held at the Fed, Treasury and investment banks by some very powerful people. Sorkin provides his readers with a great opportunity to know the details of the conference calls that were being held to decide whether or not to takeover a bank, whether or not a company needs to declare bankruptcy before market opens the next day, and whether or not Congress will pass a new bill that can potentially put an end to the crisis.


Getting started with regex in python

I have been wanting to learn regex. Not just because it reminds me of a bit but because it’s actually very useful and can be used within numerous languages including python. In case you don’t know, regex stands for regular expression. A regex is “a sequence of characters that define a search pattern.” Most popular use case is to search strings for a pattern.

I was looking at videos from this year’s PyCon and stumbled up on a video of Trey Hunner conducting regex workshop at PyCon 2016. If you are looking to learn regex, I encourage you to watch the video. You can also find most of the stuff mentioned in the video on this website.

After I watched the video, I did some practice on my own which I wanted to share here.


Book Review: Flash Boys by Michael Lewis

I remember what a big deal Flash Boys (amazon) was when it was released in 2014. Everyone I knew was talking about it (I was working at an investment bank back then). I am two years late but I finally finished reading it and have to say, it’s a really interesting book.

There is no denying that Michael Lewis (author of Liar’s Poker and The Big Short among others) is probably the best financial author of our times. In this book, he does a great job explaining the inner workings of high frequency trading and how high frequency trading firms have been ‘stealing’ from investors for the last few years. He explains different ways through which high frequency traders make money at the expense of ‘slow’ investors.


Everything I have learnt about trading oil through ETFs

When I started trading independently few months ago, I began with stocks. I picked some popular stocks such as Apple and Amazon and some not so popular ones such as Extra Space Storage and Public Storage. I would look at my P&L every few minutes to see how my portfolio was doing. However, there was not much I could do if it was doing poorly because since I work at a hedge fund, I am not allowed to sell my stocks within 30 days of buying them. This made me feel a little helpless and to be a honest, it’s boring. This is why I started trading ETFs.

ETFs are great. Not only do they allow me to trade intraday but they also, indirectly, allow me to short. Again, due to the industry I work in, I am not allowed to trade futures and engage in short sales. ETFs allow me to go long and short! A colleague of mine recommended I check out an ETF called FAZ. This ETF is meant to go against the Russell 1000 Financial Services Index. Without knowing much about how 3x leveraged ETFs work, I decided to start trading FAZ to hedge against the market. This was around February when the market was going down. Turns out this wasn’t a good investment because as soon as I invested in FAZ, the economy started recovering and FAZ declined. Of course, I could have switched to FAZ’s counterpart FAS which goes in the direction of Russell 1000 Financial Services Index but I didn’t and had to accept some loss. As I learnt more about trading ETFs, I shifted my attention to trading oil.


Developers should be practical, not perfectionists

There is barely any time when I can say I learnt a particular skill in college which I use in my daily professional life. Being able to build circuits? Nope. Triple integration? Nope. Calculating poles and zeros of a transfer equation to be able to optimize your circuit? Nope. However, one skill I do use is being practical. As an (electrical) engineer, I learnt to build stuff good enough to function. I learnt to focus more on practical aspect of things rather than purely theoretical. This concept was reinforced when I started my first job as a programmer. It was more important to have a functional code that could be delivered on time than to have the perfect code that took months, if not years, to write. My clients cared about results and budget. This is not to say they didn’t care about quality but they cared about it only to the extent that was necessary. But how do you know what level of quality is good enough?

As a programmer, many times, you will have to decide whether to make your code ‘better’ (and sophisticated) or to leave it in its current functional state. You wrote the code that meets your client’s requirements and everyone’s happy. Do you go the extra mile and make sure all the boundary cases are accounted for? Even the ones that you think have fewer chances of occurring than you winning a million dollar lottery tomorrow? It’s because of such scenarios that I am able to proudly say I am not a perfectionist. I am a practical guy who assesses each situation and determines what level of quality is required. I do not write code that will last for two decades because I know that due to the dynamic nature of the industry I work in, no code will survive that long. Why bother wasting my team and my company’s resources for something that will never be utilized?


My experience at the QuantCon 2016

quantcon_1Most of you have probably heard of a startup called Quantopian which was launched in 2011. Quantopian is a crowd-sourced hedge fund that allows its users to develop and back-test strategies using its platform. Last year, Quantopian started an annual conference for quants called QuantCon. It has been gaining a lot of popularity in the quant world which I find quite astonishing for a conference that is only a year old. Maybe it has to do with the lack of conferences targeted at quants. I attended the conference yesterday on Apr 09th, 2016 at Convene Conference Center in Midtown Manhattan. From what I could tell and I might be wrong about this, there were about 300-400 people in attendance. The day consisted of various lectures from 40 different speakers from fields such as quantitative finance, machine learning and data science. At any given time, there were about 5 sessions going on which meant that you had to pick your favorite and attend that one only.