TimescaleDB: efficiently select last row
postgresql, psycopg2, sql, timescaledb
Solution
An efficient way to get last / first record in TimescaleDB:
First record:
SELECT <COLUMN>, time FROM <TABLE_NAME> ORDER BY time ASC LIMIT 1 ;
Last record:
SELECT <COLUMN>, time FROM <TABLE_NAME> ORDER BY time DESC LIMIT 1 ;
The question has already answered but I believe it might be useful if people will get here. Using first() and last() in TimescaleDB takes much longer.
Problem
I have a postgres database with the timescaledb extension. My primary index is a timestamp, and I would like to select the latest row. If I happen to know the latest row happened after a certain time, then I can use a query such as: ``` query = 'select * from prices where time > %(dt)s' ``` Here I specify a datetime, and execute the query using psycopg2: ``` # 2018-01-10 11:15:00 dt = datetime.datetime(2018,1,10,11,15,0) with psycopg2.connect(**params) as conn: cur = conn.cursor() # start timing beg = datetime.datetime.now() # execute query cur.execute(query, {'dt':dt}) rows = cur.fetchall() # stop timing end = datetime.datetime.now() print('took {} ms'.format((end-beg).total_seconds() * 1e3)) ``` The timing output: ``` took 2.296 ms ``` If, however, I don't know the time to input into the above query, I can use a query such as: ``` query = 'select * from prices order by time desc limit 1' ``` I execute the query in a similar fashion ``` with psycopg2.connect(**params) as conn: cur = conn.cursor() # start timing beg = datetime.datetime.now() # execute query cur.execute(query) rows = cur.fetchall() # stop timing end = datetime.datetime.now() print('took {} ms'.format((end-beg).total_seconds() * 1e3)) ``` The timing output: ``` took 19.173 ms ``` So that's more than 8 times slower. I'm no expert in SQL, but I would have thought the query planner would figure out that "limit 1" and "order by primary index" equates to an O(1) operation. Question: Is there a more efficient way to select the last row in my table? In case it is useful, here is the description of my table: ``` # \d+ prices Table "public.prices" Column | Type | Collation | Nullable | Default | Storage | Stats target | Description --------+-----------------------------+-----------+----------+---------+---------+--------------+------------- time | timestamp without time zone | | not null | | plain | | AAPL | double precision | | | | plain | | GOOG | double precision | | | | plain | | MSFT | double precision | | | | plain | | Indexes: "prices_time_idx" btree ("time" DESC) Child tables: _timescaledb_internal._hyper_12_100_chunk, _timescaledb_internal._hyper_12_101_chunk, _timescaledb_internal._hyper_12_102_chunk, ... ```