This is a guest post by Mark Street. If you like it be sure to check out his other posts, or find him on LinkedIn. If you are interested in being a guest blogger on enlist[q], please contact me.
As we previously learnt, q/kdb+ has a callback function .z.ts which fires an event every x milliseconds, where x is the precision applied to the time via the \t command.
q)\t 100 / set .z.ts to fire every 100 millisecondsThis works just fine if we want to fire a single function at the same period (e.g. when running the TickerPlant in batch mode), but leaves us a little stuck if we want to execute different functions at different intervals.
An approach to solving this, is to create and maintain a list of functions we want to call, along with the interval that they should be triggered, and, when the .z.ts callback is fired, trigger the relevant ones. We will refer to this combination of a function and interval as a ‘job’.
As not to pollute the global namespace, we will create a new namespace, .timer and work inside it:
q)\d .timer
q.timer)First, we will create a table to hold 4 pieces of information:
- The unique id of the job; long type
- The interval that the job should be run; timespan type
- The time that the job should next run; timestamp type
- The function itself; “any” type
q.timer)flip `id`interval`nextRun`function!"jnp*"$\:() / my favourite way of creating tables
id interval nextRun function
----------------------------We can key the table on `id to enforce uniqueness, and assign it to variable Jobs
q.timer)Jobs:`id xkey flip `id`interval`nextRun`function!"jnp*"$\:()
q.timer)Jobs
id| interval nextRun function
--| -------------------------We want to add a dummy row to the table to allow us to support triggering named functions (e.g. foo) as well as anonymous lambdas (e.g. { 1+1 }).
q.timer)Jobs[0N]:(0Nn;0Wp;::) / null id, null timespan, infinity nextRun, identity function
q.timer)Jobs
id| interval nextRun function
--| -------------------------
| 0W ::Now for a function to Add a new job to our Jobs. We will need to give the job a unique id, and will take the function and the interval as arguments.
First, let’s construct the Add function:
Add:{[FUNCTION;INTERVAL]
/ insert values, increment id
Jobs[id+:1]:(INTERVAL;.z.p;FUNCTION);
/ return id of the newly added job
id
}Now for the brains, we need a function that:
- Discovers what jobs need to be run
- Executes each of them
- Updates their next run time
/ Note that the argument that is (automagically) fed to .z.ts is the current timestamp (.z.p).
ts:{[TS]
/ select jobs where nextRun time has passed
jobs:select from Jobs where nextRun < TS;
/ execute each function
{ x[] } each exec function from jobs;
/ update nextRun time for executed jobs
update nextRun:.z.p+interval from `.timer.Jobs where id in exec id from jobs
}Let’s try it out. Drop out of the .timer namespace:
q.timer)\d .
q)And create an example function, foo that prints “foo” and the current time, .z.p:
q)foo:{0N!"foo ",string .z.p;}
q)foo[]
"foo 2020.06.18D11:43:20.917728000"Let’s add a job that fires function foo every 5 seconds:
q).timer.Add[`foo;0D00:00:05]
1You’ll notice that nothing happens!
We need to assign our .timer.ts function to .z.ts so that it triggered on the timer:
q).z.ts:.timer.ts… and we need to set the interval for the timer via \t:
q)\t 100You’ll see that foo is triggered immediately:
q)"foo 2020.06.18D11:49:45.073923000"Let’s stop the timer with \t 0 and add a new job that prints out “bar” and the current timestamp every 10 seconds:
q)\t 0 / stop timer
q).timer.Add[{0N!"bar ",string .z.p;};0D00:00:10]If we look at .timer.Jobs we can see all the jobs:
q).timer.Jobs
id| interval nextRun function
--| ---------------------------------------------------------------------------
| 0W ::
1 | 0D00:00:05.000000000 2020.06.18D11:50:56.774279000 `foo
2 | 0D00:00:10.000000000 2020.06.18D11:51:52.610280000 {0N!"bar ",string .z.p;}Re-enable the timer, and watch as both jobs are now triggered at their respective intervals:
q)\t 100
q)"foo 2020.06.18D11:52:54.124075000"
"bar 2020.06.18D11:52:54.124195000"
"foo 2020.06.18D11:52:59.223174000"
"bar 2020.06.18D11:53:04.223076000"
"foo 2020.06.18D11:53:04.323046000"If we ever wanted to remove a running job, we can simply delete it from the table:
q)delete from `.timer.Jobs where id=1 / delete 'foo' job
`.timer.Jobs
q).timer.Jobs
id| interval nextRun function
--| ---------------------------------------------------------------------------
| 0W ::
2 | 0D00:00:10.000000000 2020.06.18D11:55:15.423363000 {0N!"bar ",string .z.p;}Extending .timer functionality
There are a number of ways that the .timer functionality could be extended, the below is a non-exhaustive list:
- Adding the
INTERVALto the firstnextRunso that jobs do not trigger immediately after being added - Error-trapping the function execution
- Adding support for Jobs that are only executed as a ‘one-off’ (i.e. null
INTERVAL) - Wrap the calls to
.z.pas e.g..timer.getTime[]to support mocking
These are left as exercises for the reader.
Full timer.q snippet
\d .timer
/ table to hold all jobs
Jobs:`id xkey flip `id`interval`nextRun`function!"jnp*"$\:()
/ dummy row to allow various function types
Jobs[0N]:(0Nn;0Wp;::)
Add:{[FUNCTION;INTERVAL]
/ insert values, increment id
Jobs[id+:1]:(INTERVAL;.z.p;FUNCTION);
/ return id of the newly added job
id
}
ts:{[TS]
/ select jobs where nextRun time has passed
jobs:select from Jobs where nextRun < TS;
/ execute each function
{ x[] } each exec function from jobs;
/ update nextRun time for executed jobs
update nextRun:.z.p+interval from `.timer.Jobs where id in exec id from jobs
}
\d .
.z.ts:.timer.ts
system"t 100"
/
foo:{0N!"foo ",string .z.p;}
.timer.Add[`foo;0D00:00:05]
.timer.Add[{0N!"bar ",string .z.p;};0D00:00:10]
\