Q:
Hi Pg Hackers,

XID wraparound seems to be quite a big concern and we introduce changes like “adding another frozen bit to each page” [http://rhaas.blogspot.com/2016/03/no-more-full-table-vacuums.html  to tackle this. I am just wondering what’s the challenges preventing us from moving to 64 bit xid?  This is the previous thread I find https://www.postgresql.org/message-id/CAEYLb_UfC%2BHZ4RAP7XuoFZr%2B2_ktQmS9xqcQgE-rNf5UCqEt5A%40mail.gmail.com , the only answer there is:

“
The most obvious reason for not using 64-bit xid values is that they
require more storage than 32-bit values. There is a patch floating
around that makes it safe to not forcibly safety shutdown the server
where currently it is necessary, but it doesn’t work by making xids
64-bit.

“

I am personally not quite convinced that is the main reason, since I feel for database hitting this issue, the schema is mostly non-trivial and doesn’t matter so much with 8 more bytes. Could some postgres experts share more insights about the challenges?

Thanks
Tianzhou

A:
Thank you for raising this question. There was a discussion about 64-bit
xids during PGCon 2017. Couple ways to provide pg_upgrade were discussed.

1) We’ve page layout version in the page (current is number 4). So, we can
define new page layout version 5. Pages with new layout version would
contain 64-bit base values for xid and multixact. The question is how to
deal with page of layout version 4. If this page have enough of free space
to fit extra 16 bytes, then it could be upgraded on the fly. If it doesn’t
contains enough of space for than then things becomes more complicated: we
can’t upgrade it to new format, but we still need to fit new xmax value
there in the case tuple being updated or deleted. pg_upgrade requires
server restart. Thus, once we set hint bits, pre-pg_upgrade xmin is not
really meaningful – corresponding xid is visible for every post-pg_upgrade
snapshot. So, idea is to use both xmin and xmax tuple fields on such
unupgradable page to store 64-bit xmax. This idea was proposed by me, but
was criticized by some session attendees (sorry, but I don’t recall who
were them) for its complexity and suspected overhead.

2) Alternative idea was to use unused bits in page header. Naturally, if
we would look for unused bits in pd_flags (3 bits of 16 is
used), pd_pagesize_version (we can left 1 bit of 16 to distinguish between
old and new format) and pd_special (we can leave 1 bit to distinguish
sequence pages), we can scrape together 43 bits. That would be far enough
for single base value, because we definitely don’t need all lower 32-bits
of base value (21 bits is more than enough). But I’m not sure about two
base values: if we would live 2 bits for lower part of base value, than it
leaves us 19 bits for high part of base value. This solution would give us
2^51 maximum values for xids and multixacts. I’m not sure if it’s enough
to assume these counters infinite. AFAIK, there are products on the market
whose have 48-bit transaction identifiers and don’t care about wraparound
or something…

New heap AM for 64-bit xids is an interesting idea too. I would even say
that pluggable storage API being discussed now is excessive for this
particular purpose (but still can fit!), because in most of aspects heap
with 64-bit xids is absolutely same as current heap (in contrast to heap
with undo log, for example). Best fit API for heap with 64-bit xid support
would be pluggable heap page format. But I don’t think it deserves
separate API though.

Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company

Re: Challenges preventing us moving to 64 bit transaction id (XID)?

重点 : 为了减少空间占用,会在page header中新增字段用以扩展xid