Using non ACID storage as workaround instead missing autonomous transactions
When I was younger, the culture war (in my bubble) was about transactional versus non-transactional engines, Postgres versus MySQL (MyISAM). Surely, I preferred the transactional concept. Data integrity and crash safety is super important. But it is not without costs. It was visible 30 years ago, when MySQL was a super fast and PostgreSQL super slow database. Today on more powerful computers it is visible too, not too strong, but still it is visible. And we still use non-transactional storages a lot of - applications logs.
There are some cases when performance wins over consistency, and it can be acceptable. When I thought about non-transactional storages, I got one idea. It can be great replacement for missing autonomous transactions. But how to test it. Fortunately I found a csv_tam storage implemented by Alexey Gordeev. This storage is mostly a concept with a lot of limits. But the idea is great - csv is a strong protocol - it is not block based, it has no row headers - so it can be very hard to support transactions. On second hand, it is primitive, and without any buffering and with forcing syncing after any row, it is mostly crash safe (against Postgres crash). Sure - it is not as safe as block storage ensured by WAL, but can be safe enough - billions applications use this safety for logging today.
I did fork and fixed build on pg 17+. Now all types are supported and writing from parallel writes should be safe. It doesn't write to WAL, so these tables cannot be backuped and cannot be replicated - what can be a nice game to support it. It is not easy to do that in a non-block format. But for testing it is enough, and I believe so this extension is very simple, so it is enough for non critical environments. It is really very very simple.
Postgres has not autonomous transactions. There are some workarounds like using dblink or pg_background. As usual any workaround has some disadvantages and limits. pg_background looks good, but at the end, it doesn't ensure 100% success in write (under high load) - although there will be space on IO. So I wrote another workaround - using a transactional engine. Not all transactional engines are not same. If I remember well, MyISAM is non-transactional and non crash safe. Aria engine is non-transactional, but crash safe. csv_tam storage is non transactional and mostly crash safe. For fully crash safety it needs fault tolerant reading (which is now possible, and should not be too hard to implement). csv_tam supports only inserts, and truncating. Nothing more. Thanks to this it is mostly crash safe.
(2026-04-03 07:44:37) postgres=# create extension csv_tam ; CREATE EXTENSION (2026-04-03 07:47:10) postgres=# create table log(ts timestamp with time zone, message varchar); CREATE TABLE (2026-04-03 07:52:22) postgres=# \sf foo CREATE OR REPLACE FUNCTION public.foo(integer) RETURNS integer LANGUAGE plpgsql AS $function$ begin return 0/$1; exception when others then insert into log values(current_timestamp, sqlerrm); raise; -- reraise error end; $function$ (2026-04-03 07:49:15) postgres=# select foo(0); ERROR: division by zero CONTEXT: PL/pgSQL function foo(integer) line 3 at RETURN (2026-04-03 07:49:20) postgres=# select * from log; ┌────┬─────────┐ │ ts │ message │ ╞════╪═════════╡ └────┴─────────┘ (0 rows) -- it doesn't work because we used classic heap (transactional) storage (2026-04-03 07:49:51) postgres=# create table log(ts timestamp with time zone, message varchar) using csv_tam; CREATE TABLE (2026-04-03 07:49:58) postgres=# select foo(0); ERROR: division by zero CONTEXT: PL/pgSQL function foo(integer) line 3 at RETURN (2026-04-03 07:50:01) postgres=# select * from log; ┌───────────────────────────────┬──────────────────┐ │ ts │ message │ ╞═══════════════════════════════╪══════════════════╡ │ 2026-04-03 07:50:01.437296+02 │ division by zero │ └───────────────────────────────┴──────────────────┘ (1 row)

0 Comments:
Post a Comment
Subscribe to Post Comments [Atom]
<< Home