q/kdb+ api for getting market and financial data from IEX

Few months ago, I wrote an api for getting market and financial data from IEX in python. As discussed earlier, IEX makes a lot of its data available to the public through its webservice api (link).

In this post, I will show you how to use the api I wrote in q/kdb+. Let’s get started.

You can find the code here.

get_last_trade

Use this function to get last trade data (price and size) for one or more securities.

q)get_last_trade`aapl`ibm
sym  price  size time
----------------------------------------------
AAPL 174.66 100  2017.11.10D20:59:58.008999936
IBM  149.18 300  2017.11.10D20:59:59.724999936 

Continue…

Listing alphabets and numbers using predefined variables

When I was trying to solve Challenge#4, I was trying to remember the variables that contains string of all the alphabets such as “ABCDEFGHIJKLMNOPQRSTUVWXYZ”. After some digging, I remembered that it was .Q.A. The only reason I knew about it was because I had used it earlier at my job. If you are new to kdb, then you probably didn’t know about it. I, also, couldn’t find it on Kx reference wiki page about DotQ namespace. In fact, there are quite a few others that are not listed there but can be very handy.

Continue…

Challenge#4 What’s in a name?

I will admit…I have been having a tough time coming up with challenges lately. So, if you have any ideas please contact me! 🙂

This week’s challenge has been borrowed from Project Euler (Problem#22). Here is what the challenge is:

Begin by sorting the names list into alphabetical order. Then working out the alphabetical value for each name, multiply this value by its alphabetical position in the list to obtain a name score. For example, when the list is sorted into alphabetical order, ELIA, which is worth 5 + 12 + 9 + 1 = 27, is the 5th name in the list. So, ELIA would obtain a score of 27 × 5 = 135. What is the total of all the name scores in the file?

Here is the list of names:

`GUS`SHAWN`LASSI`JULES`HENRY`GALINA`ERIN`EMMETT`ONITA`LORNA`ROSALINE`ELIA`BRITNI`NARCISA
`SHANNA`STEPHANY`SIOBHAN`ANNABELLE`KARLY`LASHAUN`MICHELINA`REA`SUZAN`BLAINE`LORILEE
`GABRIELLA`JOANIE`MARIELA`DIANNE`KAROL

The score should be 30822.

Continue…

Analyzing NYC traffic data using q

In my previous post, I analyzed some NYC traffic data using Pandas. In this post, I would like to perform the same analysis using q. As far as graphing is concerned, I won’t be showing that. q is not really used for graphing. You can use GUIs like qPad or qStudio to chart the data on your own.

This post will help you see how one can achieve same results using different methods. It’s nice to have these kind of options. Earlier a major disadvantage for not using q used to be its cost but now you can just use the 32 bit version that is now available for free!

One thing to note is that the analysis done in this post is very straight foreword. Both Pandas and q are capable of handing much more complex analysis. The point of these posts is to show you the different tools available to you.

For my analysis, I will be using the same data set that I used for the earlier post.

Let’s begin!

Continue…

Manipulating columns in a table

Some time back, when I was still getting familiar with qsql, I went for an interview at an investment bank. I was interviewed by 4 different people for a total of 2 hours. 3 of the interviewers asked theoretical architecture related questions but the last one was all about writing code. The last interviewer gave me a piece of paper and asked me to write a bunch of qsql statements for different scenarios. One of the questions had to do with parsing two columns that contained first and last name and then joining them together. At that time, I didn’t know how to do that. I didn’t get the job but I realized the importance of qsql and started learning. Later, I would come to realize the importance of functional queries.

Anyways, the point of the story is that you will face many situations during the course of your career where you will have to dissect columns to get your desired output. Yesterday, I had a text file with some data that I needed to dissect and put into a table so I can properly analyze it. In this post, I will discuss how I did that.

Continue…

Hidden arguments of functional select

In one of my previous posts, I discussed functional form of select, update, exec and delete. If you remember, general form of a functional select is ?[t;c;b;a] where t is the table, c is for constraints, b for by clause and a for aggregation. Most of the times you will see only these 4 arguments but functional select can actually have up to 6 arguments!

The full form looks like this:

?[t;c;b;a;n;i]

where n stands for number of rows and i for specific rows you want to extract.

Lets look at some examples!
Continue…

q chat client

Check out this great chat client written in q/html with less than 50 lines of code. The author, Stephen Peck, describes it as “Persistent group chat in <50 lines using kdb+/k/q web sockets and JS”.  You only need a working version of q/kdb and a browser to get it started. You can find the code here.

Here is a screenshot of what it looks like in action:

chat demo

My favorite thing about this project is not how simple it is but how it uses q to do something that’s not related to data/databases. Any project I have worked on has had to do with databases and why shouldn’t it? This is what the language was designed for. Seeing q being used to do something different is very interesting and really makes you think what else you can use q for (I really shouldn’t be ending my sentences with prepositions).

Anyways, check out the code. Full credit goes to Stephen Peck! Good job!

 

Challenge#2 Reverse words in a phrase

Sorry for the delay in coming up with a new challenge. It’s been a busy few weeks. Last challenge was about partial sort. This week’s challenge will be about reversing words in a phrase. It’s a common challenge and many of you have probably seen it already.

Task:
Given a string with some words, produce a string in reverse order.

Example:

q)f "you must be outta your damn mind"
"mind damn your outta be must you"

P.S. Bonus points for getting the reference!

Post your answers as comments.

Continue…

Using functional forms of amend

In one of my previous posts, I talked about functional forms of select, exec, update and delete. I also briefly touched on the importance of functional queries and why you should get used to them as much as possible if you are serious about q.

In this post, I will talk about functional forms of amend (@ and .) which I think are even more important than select/update. You can use amend to modify lists/dictionaries/tables via a custom or built-in function. A major advantage of amend is its ability to modify both in memory and on-disk tables.

There are two versions of amend: @ and .
@ is for higher level modification where as . modifies nested elements.

Continue…

Challenge#1 – Partial sort

One of my colleagues recently introduced me to Project Euler. Project Euler is a great way for you to solve mathematical problems using your preferred language and then compare your solution with others. However, I noticed that most of the problems there were not practical for q. Many were designed to use loops which are frowned up on in q. This is why I am going to start posting challenges here that are q/kdb+ related.

Our first challenge is about partial sort.

A developer asked on q/kdb+ google groups page about how to partially sort a list. Lets say I have a list:

53 66 59 30 85 89 23 60 6 52 39

I want to get:

30 53 59 66 85 89 23 60 6 52 39

Note that it found the middle number (89) and then sorted the list up to that point.

Post your solutions in the comments section.

Continue…