Wednesday, November 25, 2009

Aggregate function MEDIAN in PostgreSQL

Searching and calculating Median in databases was terrible. Still median isn't ANSI SQL aggregate function. There are two commons method how to calculate median of some column. First - very old, and very slow based on self join alchemy, second - new - based on analytic function. Now, I will be test some newer methods on one million rows large table:
postgres=# create table milrows(a real);
CREATE TABLE
Time: 7,975 ms
postgres=# insert into milrows select random()*1000000 from generate_series(1,1000000);
INSERT 0 1000000
Time: 6863,575 ms
simple scan on this table takes 200ms
Frompostgres=# select avg(a) from milrows ;
       avg        
──────────────────
 499515.883033113
(1 row)

Time: 200,176 ms
In 8.4 we can use analytic functions. These functions uses TupleStore - internal store feature - it allows work with very large tables - limit is free space on disc.

Analytic methods

--Joe Celko's method
postgres=# SELECT avg(a)::float
              FROM (SELECT a, row_number() OVER (ORDER BY a asc) AS hi,
                              count(*) OVER () + 1 - row_number() OVER (ORDER BY a) AS lo
                       FROM milrows) qs
             WHERE hi IN (lo-1,lo,lo+1);
      avg      
───────────────
 499188.546875
(1 row)

Time: 4922,678 ms

-- Andrew Gierth's method 
postgres=# select avg(a)
   from ( select a, row_number() over (order by a),count(*) over () from milrows ) s
  where row_number between floor((count::float8-1)/2+1) and ceil((count::float8-1)/2+1)
;
      avg      
───────────────
 499188.546875
(1 row)

Time: 5021,001 ms

-- modified Andrew's method (count(*) over () is slow)
postgres=# select avg(a)
   from ( select a, row_number() over (order by a),(select count(*) from milrows) as count from milrows ) s
  where row_number between floor((count::float8-1)/2+1) and ceil((count::float8-1)/2+1)
;
      avg      
───────────────
 499188.546875
(1 row)

Time: 3931,922 ms

Array based methods

Next methods are based on using an arrays. These methods are fast, but limit for this methods is size of operation memory. For very very large tables could to take all application memory.
--Regina's method -- it's not 100% correct http://www.postgresonline.com/journal/index.php?/archives/67-Build-Median-Aggregate-Function-in-SQL.html#extended

CREATE OR REPLACE FUNCTION array_median(double precision[])
  RETURNS double precision AS
$$
    SELECT CASE WHEN array_upper($1,1) = 0 THEN null 
                ELSE asorted[ceiling(array_upper(asorted,1)/2.0)]::double precision END
       FROM (SELECT ARRAY(SELECT $1[n] 
                FROM generate_series(1, array_upper($1, 1)) AS n
               WHERE $1[n] IS NOT NULL
               ORDER BY $1[n]) As asorted) As foo 
$$ LANGUAGE 'sql' IMMUTABLE;

CREATE AGGREGATE median(double precision) (
  SFUNC=array_append,
  STYPE=double precision[],
  FINALFUNC=array_median
);

postgres=# select median(a) from milrows ;
^CCancel request sent
ERROR:  canceling statement due to user request -- killed 5 minutes !don't use array_append for bigger arrays (length > 10000)
postgres=# 

--My method
postgres=# create or replace function median(anyarray) 
returns double precision as $$
  select ($1[array_upper($1,1)/2+1]::double precision + $1[(array_upper($1,1)+1) / 2]::double precision) / 2.0; 
$$ language sql immutable strict;
CREATE FUNCTION
Time: 1,557 ms

Time: 2574,677 ms
postgres=# select median(array(select a from milrows where a is not null order by a));
    median     
───────────────
 499188.546875
(1 row)

Time: 2555,342 ms
This week I added support for median aggregate to orafce package. You can download it from url http://pgfoundry.org/frs/download.php/2472/orafce-3.0.2-devel.tar.gz . Function median use some fetures 8.4 and needs 8.4 - it isn't supported on PostgreSQL 8.3 and older.
-- orafce 3.0.2 median (needs PostgreSQL 8.4 and higher)

postgres=# select median(a::float8) from milrows;
    median     
───────────────
 499188.546875
(1 row)

Time: 687,577 ms
It's very fast - if your table has about one million rows (1000000) you can use it (for this table size takes max. 15MB RAM (for one column)).

Saturday, November 14, 2009

longtime plpgsql misfeature removed

Tom Lane did refactoring of plpgsql source code. These changes are very very important. plpgsql is good language - simple, robust with good error diagnostic. But it had one bizarre behave. plpgsql connects two worlds - procedural ADA like code and SQL statements. Usually there are not problems. But there are one exception - collision of identifiers. Older behave was too simply. Plpgsql identifiers win every-time. It was a source of some bizarre bugs. Look on code: 
postgres=# select * from omega;
 a  
────
 10
 20
 30
(3 rows)

create or replace function foo() 
returns void as $$
#variable_conflict use_variable -- compatible with 8.4 and older
declare a integer; 
begin 
  for a in select a from omega 
  loop 
    raise notice '%', a;
  end loop; 
end; 
$$ language plpgsql;
This code is very simple. Just show content of table omega.
postgres=# select foo();
NOTICE:  <null>
NOTICE:  <null>
NOTICE:  <null>
 foo 
─────
 
(1 row)
or not? Why we don't see values 10,20,30? Because interpret prefer plpgsql identifier against to sql identifier omega.a. This bug is very strange and some time is very difficult to find it. But it is a history. plpgsql 8.5 is much more cleaner. Wrong code raises en exception:
postgres=# 
create or replace function foo() 
returns void as $$
declare a integer;                                              
begin              
  for a in select a from omega 
  loop                         
    raise notice '%', a;
  end loop; 
end; 
$$ language plpgsql;
CREATE FUNCTION
Time: 3,501 ms
postgres=# select foo();
ERROR:  column reference "a" is ambiguous
LINE 1: select a from omega
               ^
DETAIL:  It could refer to either a PL/pgSQL variable or a table column.
QUERY:  select a from omega
CONTEXT:  PL/pgSQL function "foo" line 3 at FOR over SELECT rows
We could to fix this problem and we get a good answer:
postgres=# 
create or replace function foo() 
returns void as $$
declare a integer; 
begin 
  for a in select omega.a from omega 
  loop 
    raise notice '%', a;
  end loop; 
end; 
$$ language plpgsql;
CREATE FUNCTION
Time: 2,289 ms
postgres=# select foo();
NOTICE:  10
NOTICE:  20
NOTICE:  30
 foo 
─────
 
(1 row)
I am very happy from this changes. Thanks Tom.

Thursday, October 29, 2009

Named function parameters - a feature of PostgreSQL 8.5

Hello

New PostgreSQL 8.5 alfa 2 allows to use named parameters. You may to know this feature from Oracle or MS SQL server. This feature is possible in PostgreSQL now too. Following function add n working days to specified date. What is new? You can specify a free days (default is based on europen traditions).

CREATE TYPE dayname AS ENUM ('sun','mon','tue','wed','thu','fri','sat');

CREATE OR REPLACE FUNCTION dow(dayname) 
RETURNS int AS $$ 
SELECT CASE $1 
           WHEN 'sun' THEN 0 
           WHEN 'mon' THEN 1 
           WHEN 'tue' THEN 2 
           WHEN 'wed' THEN 3 
           WHEN 'thu' THEN 4 
           WHEN 'fri' THEN 5 
           WHEN 'sat' THEN 6 END; 
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION dows(dayname[]) 
RETURNS int[] AS $$ 
SELECT ARRAY(SELECT dow(d) FROM UNNEST($1) d(d))
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION addwdays(date, int, freedays text[] = ARRAY['sat','sun']) 
RETURNS date AS $$
SELECT ad 
   FROM (SELECT $1 + i as ad, 
                extract (dow from $1 + i), 
                sum(CASE WHEN EXTRACT(dow FROM $1 + i) = ANY(dows($3::dayname[])) THEN 0 ELSE 1 END) OVER (ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS x 
            FROM generate_series(1,1000) g(i)) s 
  WHERE x = $2 AND date_part <> ALL(dows($3::dayname[]));
$$ LANGUAGE sql;

postgres=# select current_date, addwdays(current_date,1);
    date    │  addwdays  
────────────┼────────────
 2009-10-29 │ 2009-10-30
(1 row)

Time: 1,768 ms
postgres=# select current_date, addwdays(current_date,1, ARRAY['fri'] AS freedays);
    date    │  addwdays  
────────────┼────────────
 2009-10-29 │ 2009-10-31
(1 row)

Please, test it.
see http://www.postgresql.org/about/news.1152

Thursday, October 1, 2009

Dynamic access to record fields in PL/pgSQL

PL/pgSQL isn't dynamic language. Usually it isn't problem. This language is best as glue of SQL statements. One typical feature is strongly static access to record or row fields. But sometime we need dynamic access. This is very hard task in older version. Some better situation is in 8.4, and maybe in 8.5, this problem is definitely out. New an enhancing of hstore module allows conversion from ROW type to hstore type (this is similar to hash table in Perl). We could to use this functionality:

postgres=# SELECT * FROM foo;
 a  | b  |   c    
----+----+--------
 10 | 20 | Pavel
 30 | 40 | Zdenek
(2 rows)

postgres=# select (each(hstore(foo))).* from foo;
 key | value  
-----+--------
 a   | 10
 b   | 20
 c   | Pavel
 a   | 30
 b   | 40
 c   | Zdenek
(6 rows)

CREATE OR REPLACE FUNCTION trgfce() 
RETURNS trigger AS $$
DECLARE r record; 
BEGIN 
  FOR r IN SELECT (each(hstore(NEW))).* 
  LOOP 
    RAISE NOTICE 'key:%, value: %', r.key, r.value; 
  END LOOP; 
  RETURN new; 
END
$$ LANGUAGE plpgsql;

CREATE TRIGGER trg BEFORE INSERT ON foo FOR EACH ROW EXECUTE PROCEDURE trgfce();

postgres=# INSERT INTO foo VALUES(80,90,'Zbynek');
NOTICE:  key:a, value: 80
NOTICE:  key:b, value: 90
NOTICE:  key:c, value: Zbynek
INSERT 0 1

Saturday, August 22, 2009

MySQL functions for PostgreSQL

Hello

I found article about MySQL function field. PostgreSQL doesn't support similar function (it is nice together with ORDER BY expression clause), but with new variadic parameters is easy to develop it in pg:

CREATE OR REPLACE FUNCTION field(text, variadic text[])
RETURNS int AS $$
  SELECT i
     FROM generate_subscripts($2,1) g(i)
    WHERE $1 = $2[i]
    UNION ALL
    SELECT 0
    LIMIT 1
$$ LANGUAGE sql STRICT;
An using looks like:
select * from pet order by field(species, 'cat', 'dog', 'bird');

+----------+--------+---------+------+------------+------------+
| name     | owner  | species | sex  | birthday   | death      |
+----------+--------+---------+------+------------+------------+
| Fluffy   | Harold | cat     | f    | 1993-02-04 | NULL       |
| Claws    | Gwen   | cat     | m    | 1994-03-17 | NULL       |
| Buffy    | Harold | dog     | f    | 1989-05-13 | NULL       |
| Fang     | Benny  | dog     | m    | 1990-08-27 | NULL       |
| Bowser   | Diane  | dog     | m    | 1989-08-31 | 1995-07-29 |
| Chirpy   | Gwen   | bird    | f    | 1998-09-11 | NULL       |
| Whistler | Gwen   | bird    | NULL | 1997-12-09 | NULL       |
+----------+--------+---------+------+------------+------------+
When I played with it, I ported mostly used MySQL functions to PostgreSQL. I hope, so this file can help somebody with porting applications to PostgreSQL or with better orientation in PostgreSQL. Lot of thing with strings or date PostgreSQL is done little bit different than MySQL - these functions can help.

String Functions

CREATE OR REPLACE FUNCTION concat(variadic str text[])
RETURNS text AS $$
SELECT array_to_string($1, '');
$$ LANGUAGE sql

CREATE OR REPLACE FUNCTION concat_ws(separator text, variadic str text[])
RETURNS text as $$
SELECT array_to_string($2, $1);
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION elt(int, VARIADIC text[])
RETURNS text AS $$
SELECT $2[$1];
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION field(varchar, VARIADIC text[])
RETURNS int AS $$
SELECT i
FROM generate_subscripts($2,1) g(i)
WHERE $1 = $2[i]
UNION ALL
SELECT 0
LIMIT 1
$$ LANGUAGE sql STRICT;

CREATE OR REPLACE FUNCTION find_in_set(str text, strlist text)
RETURNS int AS $$
SELECT i
   FROM generate_subscripts(string_to_array($2,','),1) g(i)
  WHERE (string_to_array($2, ','))[i] = $1
  UNION ALL
  SELECT 0
  LIMIT 1
$$ LANGUAGE sql STRICT;

CREATE OR REPLACE FUNCTION hex(int)
RETURNS text AS $$
SELECT upper(to_hex($1));
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION hex(bigint)
RETURNS text AS $$
SELECT upper(to_hex($1));
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION hex(text)
RETURNS text AS $$
SELECT upper(encode($1::bytea, 'hex'))
$$ LANGUAGE sql;

/*
* char is keyword, double quotes are necessary.
*
*  postgres=# select "char"(77,121,83,81,'76');
*  char
* -------
*  MySQL
*/
CREATE OR REPLACE FUNCTION "char"(VARIADIC int[])
RETURNS text AS $$
SELECT array_to_string(ARRAY(SELECT chr(unnest($1))),'')
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION lcase(str text)
RETURNS text AS $$
SELECT lower($1)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION left(str text, len int)
RETURNS text AS $$
SELECT substring($1 FROM 1 FOR $2)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION locate(substr text, str text)
RETURNS int AS $$
SELECT position($1 in $2)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION reverse(str text)
RETURNS text AS $$
SELECT array_to_string(ARRAY(SELECT substring($1 FROM i FOR 1)
                                FROM generate_series(length($1),1,-1) g(i)),
                       '')
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION right(str text, len int)
RETURNS text AS $$
SELECT substring($1 FROM length($1) - $2 FOR $2)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION space(n int)
RETURNS text AS $$
SELECT repeat(' ', $1)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION strcmp(text, text)
RETURNS int AS $$
SELECT CASE WHEN $1 < $2 THEN -1
WHEN $1 > $2 THEN 1
ELSE 0 END;
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION substring_index(str text, delim text, count int)
RETURNS text AS $$
SELECT CASE WHEN $3 > 0 
THEN array_to_string((string_to_array($1, $2))[1:$3], $2)
ELSE array_to_string(ARRAY(SELECT unnest(string_to_array($1,$2))
                             OFFSET array_upper(string_to_array($1,$2),1) + $3),
                     $2)
END
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION ucase(str text)
RETURNS text AS $$
SELECT upper($1)
$$ LANGUAGE sql;

CREATE CAST (bytea AS text) WITHOUT FUNCTION AS ASSIGNMENT;

/*
* SELECT hex('žlutý kůň'), unhex(hex('žlutý kůň'))
*/
CREATE OR REPLACE FUNCTION unhex(text)
RETURNS text AS $$
SELECT decode($1, 'hex')::text;
$$ LANGUAGE sql;

Date, time Functions

/*
* postgres=# select adddate('2008-01-02','31 day');
*   adddate
* -----------
*  2008-02-02
*/
CREATE OR REPLACE FUNCTION adddate(date, interval)
RETURNS date AS $$
SELECT ($1 + $2)::date; $$
LANGUAGE sql;

CREATE OR REPLACE FUNCTION curdate()
RETURNS date AS $$
SELECT CURRENT_DATE
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION convert_tz(dt timestamp, from_tz text, to_tz text)
RETURNS timestamp AS $$
SELECT ($1 AT TIME ZONE $2) AT TIME ZONE $3;
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION date(anyelement)
RETURNS date AS $$
SELECT $1::date;
$$ LANGUAGE sql;

SELECT OR REPLACE FUNCTION datediff(date, date)
RETURNS int AS $$
SELECT $1 - $2
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION date_add(date, interval)
RETURNS date AS $$
SELECT adddate($1, $2)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION date_format(date, text)
RETURNS text AS $$
SELECT to_char($1, _mysqlf_pgsql($2))
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION date_format(timestamp, text)
RETURNS text AS $$
SELECT to_char($1, _mysqlf_pgsql($2))
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION date_sub(date, interval)
RETURNS date AS $$
SELECT ($1 - $2)::date;
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION dayofmonth(date)
RETURNS int AS $$
SELECT EXTRACT(day from $1)::int
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION day(date)
RETURNS int AS $$
SELECT dayofmonth($1)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION dayname(date)
RETURNS text AS $$
SELECT to_char($1, 'TMDay')
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION dayofweek(date)
RETURNS int AS $$
SELECT EXTRACT(dow FROM $1)::int
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION dayofyear(date)
RETURNS int AS $$
SELECT EXTRACT(doy FROM $1)::int
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION from_days(int)
RETURNS date AS $$
SELECT date '0001-01-01bc' + $1
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION from_unixtime(double precision)
RETURNS timestamp AS $$
SELECT to_timestamp($1)::timestamp
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION _mysqlf_pgsql(text)
RETURNS text AS $$
SELECT array_to_string(ARRAY(SELECT s
FROM (SELECT CASE WHEN substring($1 FROM i FOR 1) <> '%'
AND substring($1 FROM i-1 FOR 1) <> '%'
THEN substring($1 FROM i for 1)
ELSE CASE substring($1 FROM i FOR 2)
WHEN '%H' THEN 'HH24'
WHEN '%p' THEN 'am'
WHEN '%Y' THEN 'YYYY'
WHEN '%m' THEN 'MM'
WHEN '%d' THEN 'DD'
WHEN '%i' THEN 'MI'
WHEN '%s' THEN 'SS'
WHEN '%a' THEN 'Dy'
WHEN '%b' THEN 'Mon'
WHEN '%W' THEN 'Day'
WHEN '%M' THEN 'Month'
END
END s
FROM generate_series(1,length($1)) g(i)) g
WHERE s IS NOT NULL),
'')
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION get_format(text, text)
RETURNS text AS $$
SELECT CASE lower($1)
WHEN 'date' THEN
CASE lower($2)
WHEN 'usa' THEN '%m.%d.%Y'
WHEN 'jis' THEN '%Y-%m-%d'
WHEN 'iso' THEN '%Y-%m-%d'
WHEN 'eur' THEN '%d.%m.%Y'
WHEN 'internal' THEN '%Y%m%d'
END
WHEN 'datetime' THEN
CASE lower($2)
WHEN 'usa' THEN '%Y-%m-%d %H-.%i.%s'
WHEN 'jis' THEN '%Y-%m-%d %H:%i:%s'
WHEN 'iso' THEN '%Y-%m-%d %H:%i:%s'
WHEN 'eur' THEN '%Y-%m-%d %H.%i.%s'
WHEN 'internal' THEN '%Y%m%d%H%i%s'
END
WHEN 'time' THEN
CASE lower($2)
WHEN 'usa' THEN '%h:%i:%s %p'
WHEN 'jis' THEN '%H:%i:%s'
WHEN 'iso' THEN '%H:%i:%s'
WHEN 'eur' THEN '%H.%i.%s'
WHEN 'internal' THEN '%H%i%s'
END
END;
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION hour(time)
RETURNS int AS $$
SELECT EXTRACT(hour FROM $1)::int;
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION hour(timestamp)
RETURNS int AS $$
SELECT EXTRACT(hour FROM $1)::int;
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION last_day(date)
RETURNS date AS $$
SELECT (date_trunc('month',$1 + interval '1 month'))::date - 1
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION makedate(year int, dayofyear int)
RETURNS date AS $$
SELECT (date '0001-01-01' + ($1 - 1) * interval '1 year' + ($2 - 1) * interval '1 day'):: date
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION maketime(int, int, double precision)
RETURNS time AS $$
SELECT time '00:00:00' + $1 * interval '1 hour' + $2 * interval '1 min'
+ $3 * interval '1 sec'
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION minute(timestamp)
RETURNS int AS $$
SELECT EXTRACT(minute FROM $1)::int
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION month(date)
RETURNS int AS $$
SELECT EXTRACT(month FROM $1)::int
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION monthname(date)
RETURNS text AS $$
SELECT to_char($1, 'TMMonth')
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION str_to_date(text, text)
RETURNS date AS $$
SELECT to_date($1, _mysqlf_pgsql($2))
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION time(timestamp)
RETURNS time AS $$
SELECT $1::time
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION to_days(date)
RETURNS int AS $$
SELECT $1 - '0001-01-01bc'
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION unix_timestamp()
RETURNS double precision AS $$
SELECT EXTRACT(epoch FROM current_timestamp)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION unix_timestamp(timestamp)
RETURNS double precision AS $$
SELECT EXTRACT(epoch FROM $1)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION year(date)
RETURNS int AS $$
SELECT EXTRACT(year FROM $1)
$$ LANGUAGE sql;

CREATE OR REPLACE FUNCTION week(date)
RETURNS int AS $$
SELECT EXTRACT(week FROM $1)::int;
$$ LANGUAGE sql;

GROUP_CONCAT

PostgreSQL doesn't support aggregate function group_concat. But you can use aggregate array_agg:
postgres=# select * from omega;
+---+
| x |
+---+
| 1 |
| 3 |
| 6 |
+---+
(3 rows)

postgres=# select array_to_string(array_agg(x),',') from omega;
+-----------------+
| array_to_string |
+-----------------+
| 1,3,6           |
+-----------------+
(1 row)
If you search more complete MySQL API, look on mysqlcompat project. Regards Pavel

Thursday, August 13, 2009

epsql available

Hello

Today I uploaded patch enhancing psql on pgfoundry. You can download it from url http://pgfoundry.org/frs/download.php/2335/epsql.diff. This patch should be applied on postgresql 8.4 source:
cd postgresql-8.4.0/src/bin/psql
patch -p1 < epsql.diff
make clean
make all
su
make install
Enhanced psql has new metacommand \lf and new formating option linestyle (+2 new border styles).

\lf

\lf print function's source code (with and without row numbers):
postgres=# \lf foo(int)
**** CREATE OR REPLACE FUNCTION public.foo(_a integer)
**** RETURNS integer
**** LANGUAGE plpgsql
**** AS $function$
1 BEGIN
2 RETURN _a + 1;
3 END;
**** $function$

postgres=# \lf- foo(int)
CREATE OR REPLACE FUNCTION public.foo(_a integer)
RETURNS integer
LANGUAGE plpgsql
AS $function$
BEGIN
RETURN _a + 1;
END;
$function$

smart wrap mode

original wrap mode:
postgres=# \pset format wrapped
Output format is wrapped.
postgres=# select * from test;
+--------------------------------------------------------------------------------------+
| a |
+--------------------------------------------------------------------------------------+
| Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor inci |
| didunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exerc |
| itation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dol |
| or in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. |
| Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt m |
| ollit anim id est laborum. |
+--------------------------------------------------------------------------------------+
(1 row)
modified wrap mode:
postgres=# select * from test;
+--------------------------------------------------------------------------------------+
| a |
+--------------------------------------------------------------------------------------+
| Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor |
| incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud |
| exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute |
| irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla |
| pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia |
| deserunt mollit anim id est laborum. |
+--------------------------------------------------------------------------------------+
(1 row)

Border styles

postgres=# \pset border 0
Border style is 0.
postgres=# \d pg_database
Table "pg_catalog.pg_database"
Column Type Modifiers
------------- --------- ---------
datname name not null
datdba oid not null
encoding integer not null
datcollate name not null
datctype name not null
datistemplate boolean not null
datallowconn boolean not null
datconnlimit integer not null
datlastsysoid oid not null
datfrozenxid xid not null
dattablespace oid not null
datconfig text[]
datacl aclitem[]

postgres=# \pset border 1
Border style is 1.
postgres=# \d pg_database
Table "pg_catalog.pg_database"
Column | Type | Modifiers
---------------+-----------+-----------
datname | name | not null
datdba | oid | not null
encoding | integer | not null
datcollate | name | not null
datctype | name | not null
datistemplate | boolean | not null
datallowconn | boolean | not null
datconnlimit | integer | not null
datlastsysoid | oid | not null
datfrozenxid | xid | not null
dattablespace | oid | not null
datconfig | text[] |
datacl | aclitem[] |

postgres=# \pset border 2
Border style is 2.
postgres=# \d pg_database
Table "pg_catalog.pg_database"
+---------------+-----------+-----------+
| Column | Type | Modifiers |
+---------------+-----------+-----------+
| datname | name | not null |
| datdba | oid | not null |
| encoding | integer | not null |
| datcollate | name | not null |
| datctype | name | not null |
| datistemplate | boolean | not null |
| datallowconn | boolean | not null |
| datconnlimit | integer | not null |
| datlastsysoid | oid | not null |
| datfrozenxid | xid | not null |
| dattablespace | oid | not null |
| datconfig | text[] | |
| datacl | aclitem[] | |
+---------------+-----------+-----------+

postgres=# \pset border 3
Border style is 3.
postgres=# \d pg_database
Table "pg_catalog.pg_database"
+---------------+-----------+-----------+
| Column | Type | Modifiers |
+---------------+-----------+-----------+
| datname | name | not null |
+---------------+-----------+-----------+
| datdba | oid | not null |
+---------------+-----------+-----------+
| encoding | integer | not null |
+---------------+-----------+-----------+
| datcollate | name | not null |
+---------------+-----------+-----------+
| datctype | name | not null |
+---------------+-----------+-----------+
| datistemplate | boolean | not null |
+---------------+-----------+-----------+
| datallowconn | boolean | not null |
+---------------+-----------+-----------+
| datconnlimit | integer | not null |
+---------------+-----------+-----------+
| datlastsysoid | oid | not null |
+---------------+-----------+-----------+
| datfrozenxid | xid | not null |
+---------------+-----------+-----------+
| dattablespace | oid | not null |
+---------------+-----------+-----------+
| datconfig | text[] | |
+---------------+-----------+-----------+
| datacl | aclitem[] | |
+---------------+-----------+-----------+

postgres=# \pset border 4
Border style is 4.
postgres=# \d pg_database
Table "pg_catalog.pg_database"
Column | Type | Modifiers
---------------+-----------+----------
datname | name | not null
---------------+-----------+----------
datdba | oid | not null
---------------+-----------+----------
encoding | integer | not null
---------------+-----------+----------
datcollate | name | not null
---------------+-----------+----------
datctype | name | not null
---------------+-----------+----------
datistemplate | boolean | not null
---------------+-----------+----------
datallowconn | boolean | not null
---------------+-----------+----------
datconnlimit | integer | not null
---------------+-----------+----------
datlastsysoid | oid | not null
---------------+-----------+----------
datfrozenxid | xid | not null
---------------+-----------+----------
dattablespace | oid | not null
---------------+-----------+----------
datconfig | text[] |
---------------+-----------+----------
datacl | aclitem[] |

Linestyles

postgres=# \pset linestyle 0
Border line style is 0.
postgres=# \d pg_user_mapping
Table "pg_catalog.pg_user_mapping"
+-----------+--------+-----------+
| Column | Type | Modifiers |
+-----------+--------+-----------+
| umuser | oid | not null |
+-----------+--------+-----------+
| umserver | oid | not null |
+-----------+--------+-----------+
| umoptions | text[] | |
+-----------+--------+-----------+

postgres=# \pset linestyle 1
Border line style is 1.
postgres=# \d pg_user_mapping
Table "pg_catalog.pg_user_mapping"
┌───────────┬────────┬───────────┐
│ Column │ Type │ Modifiers │
├───────────┼────────┼───────────┤
│ umuser │ oid │ not null │
├───────────┼────────┼───────────┤
│ umserver │ oid │ not null │
├───────────┼────────┼───────────┤
│ umoptions │ text[] │ │
└───────────┴────────┴───────────┘

postgres=# \pset linestyle 2
Border line style is 2.
postgres=# \d pg_user_mapping
Table "pg_catalog.pg_user_mapping"
┌───────────┬────────┬───────────┐
│ Column │ Type │ Modifiers │
├───────────┴────────┴───────────┤
│ umuser │ oid │ not null │
├───────────┼────────┼───────────┤
│ umserver │ oid │ not null │
├───────────┼────────┼───────────┤
│ umoptions │ text[] │ │
└────────────────────────────────┘

postgres=# \pset linestyle 3
Border line style is 3.
postgres=# \d pg_user_mapping
Table "pg_catalog.pg_user_mapping"
╔═══════════╤════════╤═══════════╗
║ Column │ Type │ Modifiers ║
╟───────────┼────────┼───────────╢
║ umuser │ oid │ not null ║
╟───────────┼────────┼───────────╢
║ umserver │ oid │ not null ║
╟───────────┼────────┼───────────╢
║ umoptions │ text[] │ ║
╚═══════════╧════════╧═══════════╝

postgres=# \pset linestyle 4
Border line style is 4.
postgres=# \d pg_user_mapping
Table "pg_catalog.pg_user_mapping"
╔═══════════╤════════╤═══════════╗
║ Column │ Type │ Modifiers ║
╠═══════════╪════════╪═══════════╣
║ umuser │ oid │ not null ║
╟───────────┼────────┼───────────╢
║ umserver │ oid │ not null ║
╟───────────┼────────┼───────────╢
║ umoptions │ text[] │ ║
╚═══════════╧════════╧═══════════╝

postgres=# \pset linestyle 5
Border line style is 5.
postgres=# \d pg_user_mapping
Table "pg_catalog.pg_user_mapping"
╔═══════════╦════════╦═══════════╗
║ Column ║ Type ║ Modifiers ║
╠═══════════╩════════╩═══════════╣
║ umuser │ oid │ not null ║
╟───────────┼────────┼───────────╢
║ umserver │ oid │ not null ║
╟───────────┼────────┼───────────╢
║ umoptions │ text[] │ ║
╚════════════════════════════════╝

postgres=# \pset linestyle 6
Border line style is 6.
postgres=# \d pg_user_mapping
Table "pg_catalog.pg_user_mapping"
╔═══════════╦════════╦═══════════╗
║ Column ║ Type ║ Modifiers ║
╠═══════════╬════════╬═══════════╣
║ umuser ║ oid ║ not null ║
╟───────────╫────────╫───────────╢
║ umserver ║ oid ║ not null ║
╟───────────╫────────╫───────────╢
║ umoptions ║ text[] ║ ║
╚═══════════╩════════╩═══════════╝
Please, test it, use it.
Regards
Pavel Stehule

Tuesday, March 10, 2009

Experimental psql

Hello

I spent some on psql's face lifting. Some new features are really experimental. I don't expect early adopting it for core. It's more game for now. epsql will be available with release 8.4. What epsql can do?

UTF8 boxes

  • one new border style, five new border lines styles

Stored function's listing

  • should help with debuging

    postgres=# select test(10);
    ERROR: division by zero
    CONTEXT: PL/pgSQL function "test" line 3 at assignment

    postgres=# \lf test
    *** CREATE OR REPLACE FUNCTION public.test(a integer)
    *** RETURNS integer
    *** LANGUAGE plpgsql
    *** AS $function$
    1 declare b int;
    2 begin
    3 b := a/0;
    4 return a;
    5 end;
    *** $function$

Cursor's support

  • allows using data from database for statement parameters
    • \fetch
      postgres=# BEGIN;
      BEGIN
      postgres=# DECLARE c CURSOR FOR SELECT * FROM pg_database;
      DECLARE CURSOR
      postgres=# \fetch c \echo :datname :datcollate
      template1 cs_CZ.UTF-8
      postgres=# \fetch c \echo :datname :datcollate
      template0 cs_CZ.UTF-8
      postgres=# \fetch c \echo :datname :datcollate
      postgres cs_CZ.UTF-8
      postgres=# \fetch c \echo :datname :datcollate
      ERROR: (not available)
      postgres=# COMMIT;
      COMMIT
    • \fetchall
      postgres=# CREATE TABLE test1(a int); CREATE TABLE test2(a int);
      CREATE TABLE
      CREATE TABLE
      postgres=# BEGIN;
      BEGIN
      postgres=# DECLARE c CURSOR FOR SELECT tablename FROM pg_tables WHERE tablename LIKE 'test%';
      DECLARE CURSOR
      postgres=# \fetchall c DROP TABLE :tablename;
      DROP TABLE
      DROP TABLE
      postgres=# COMMIT;
      COMMIT
      postgres=#
I invite any ideas, comments.

Pavel

Labels: