PostgreSQL 14在MVCC上面有所增强，以保证在大批量客户端连接场景下性能不会出现显著下降（详细可参见参考资料中的链接）。本节介绍了提交记录“snapshot scalability: Introduce dense array of in-progress xids.”中的修改以及涉及的相关数据结构。

一、数据结构

PROC_HDR

/*
 * There is one ProcGlobal struct for the whole database cluster.
 * 全局的ProcGlobal(PROC_HDR)结构体
 *
 * Adding/Removing an entry into the procarray requires holding *both*
 * ProcArrayLock and XidGenLock in exclusive mode (in that order). Both are
 * needed because the dense arrays (see below) are accessed from
 * GetNewTransactionId() and GetSnapshotData(), and we don't want to add
 * further contention by both using the same lock. Adding/Removing a procarray
 * entry is much less frequent.
 * 对proc数组的插入和删除,需要以独占的模式持有ProcArrayLock和XidGenLock,
 * 原因是GetNewTransactionId()和GetSnapshotData()函数需要访问,并且我们不希望使用同样的锁来增加争用.
 * 其实,插入和删除proc数组并不频繁.
 *
 * Some fields in PGPROC are mirrored into more densely packed arrays (e.g.
 * xids), with one entry for each backend. These arrays only contain entries
 * for PGPROCs that have been added to the shared array with ProcArrayAdd()
 * (in contrast to PGPROC array which has unused PGPROCs interspersed).
 * PGPROC中的一些字段镜像到更稠密的数组中(如xids),每一个后台进程都有一项.
 * 这些数组只包含使用ProcArrayAdd()加入到共享数组的PGPROCs.
 *
 * The dense arrays are indexed by PGPROC->pgxactoff. Any concurrent
 * ProcArrayAdd() / ProcArrayRemove() can lead to pgxactoff of a procarray
 * member to change.  Therefore it is only safe to use PGPROC->pgxactoff to
 * access the dense array while holding either ProcArrayLock or XidGenLock.
 * 稠密数组通过PGPROC->pgxactoff字段进行索引,所有同步ProcArrayAdd和ProcArrayRemove操作
 * 会改变procarray中的pgxactoff成员变量.
 *
 * As long as a PGPROC is in the procarray, the mirrored values need to be
 * maintained in both places in a coherent manner.
 * 由于PGPROC在proc数组中,镜像值需要以一致的方式进行运维.
 *
 * The denser separate arrays are beneficial for three main reasons: First, to
 * allow for as tight loops accessing the data as possible. Second, to prevent
 * updates of frequently changing data (e.g. xmin) from invalidating
 * cachelines also containing less frequently changing data (e.g. xid,
 * statusFlags). Third to condense frequently accessed data into as few
 * cachelines as possible.
 * 基于以下3点原因,更稠密单独的数组会带来不少好处.
 * 1.紧密的循环访问数据成为可能;
 * 2.避免了对修改频度不高的数据进行修改导致CPU缓存无效的情况出现;
 * 3.使频繁访问的数据放在小量的CPU缓存中成为可能.
 *
 * There are two main reasons to have the data mirrored between these dense
 * arrays and PGPROC. First, as explained above, a PGPROC's array entries can
 * only be accessed with either ProcArrayLock or XidGenLock held, whereas the
 * PGPROC entries do not require that (obviously there may still be locking
 * requirements around the individual field, separate from the concerns
 * here). That is particularly important for a backend to efficiently checks
 * it own values, which it often can safely do without locking.  Second, the
 * PGPROC fields allow to avoid unnecessary accesses and modification to the
 * dense arrays. A backend's own PGPROC is more likely to be in a local cache,
 * whereas the cachelines for the dense array will be modified by other
 * backends (often removing it from the cache for other cores/sockets). At
 * commit/abort time a check of the PGPROC value can avoid accessing/dirtying
 * the corresponding array value.
 * 让这些稠密数组和PGPROC之间生成数据镜像,有两个主要原因:
 * 1.如上所述,PGPROC数组条目需要在持有ProcArrayLock和XidGenLock的前提下才能访问,但实际上并不需要.
 *   在没有锁的情况下,后台进程有效且安全的检查自己的值是十分重要的.
 * 2.PGPROC中的字段避免不必要的访问和修改.后台进程自己的PGPROC更应该在自己的本地缓存中,
 *   而无需理会数组会被其他进程修改.在事务提交或回滚时,检查PGPROC值可以避免访问或者弄脏相应的数组值
 *
 * Basically it makes sense to access the PGPROC variable when checking a
 * single backend's data, especially when already looking at the PGPROC for
 * other reasons already.  It makes sense to look at the "dense" arrays if we
 * need to look at many / most entries, because we then benefit from the
 * reduced indirection and better cross-process cache-ability.
 * 检查单个后台进程数据时,访问PGPROC值是比较敏感的,特别是在已经因为其他原因检索过PGPROC的情况下
 *
 * When entering a PGPROC for 2PC transactions with ProcArrayAdd(), the data
 * in the dense arrays is initialized from the PGPROC while it already holds
 * ProcArrayLock.
 * 在2PC事务的情况下,通过ProcArrayAdd方法进入PGPROC时,
 *   数组中的数据在持有ProcArrayLock锁的情况下初始化数组.
 */
typedef struct PROC_HDR
{
    /* Array of PGPROC structures (not including dummies for prepared txns) */
    //PGPROC结构体数组
    PGPROC       *allProcs;
    /* Array mirroring PGPROC.xid for each PGPROC currently in the procarray */
    // 镜像每一个PGPROC.xid
    TransactionId *xids;
    /*
     * Array mirroring PGPROC.subxidStatus for each PGPROC currently in the
     * procarray.
     */
    //镜像PGPROC.subxidStatus
    XidCacheStatus *subxidStates;
    /*
     * Array mirroring PGPROC.statusFlags for each PGPROC currently in the
     * procarray.
     */
    //镜像PGPROC.statusFlags
    uint8       *statusFlags;
    /* Length of allProcs array */
    //allProcs数组的大小
    uint32        allProcCount;
    /* Head of list of free PGPROC structures */
    //空闲PGPROC结构体链表头
    PGPROC       *freeProcs;
    /* Head of list of autovacuum's free PGPROC structures */
    //autovacuum进程空闲PGPROC结构体链表头
    PGPROC       *autovacFreeProcs;
    /* Head of list of bgworker free PGPROC structures */
    //bgworker进程
    PGPROC       *bgworkerFreeProcs;
    /* Head of list of walsender free PGPROC structures */
    //walsender
    PGPROC       *walsenderFreeProcs;
    /* First pgproc waiting for group XID clear */
    //等待XID组清理的第一个pgproc
    pg_atomic_uint32 procArrayGroupFirst;
    /* First pgproc waiting for group transaction status update */
    //等待事务状态组更新的首个pgproc
    pg_atomic_uint32 clogGroupFirst;
    /* WALWriter process's latch */
    //WALWriter进程latch
    Latch       *walwriterLatch;
    /* Checkpointer process's latch */
    //检查点进程latch
    Latch       *checkpointerLatch;
    /* Current shared estimate of appropriate spins_per_delay value */
    //spins_per_delay
    int            spins_per_delay;
    /* The proc of the Startup process, since not in ProcArray */
    //Startup进程的proc结构体,不再Proc数组中
    PGPROC       *startupProc;
    int            startupProcPid;
    /* Buffer id of the buffer that Startup process waits for pin on, or -1 */
    //Startup进程等待pin的缓存ID
    int            startupBufferPinWaitBufId;
} PROC_HDR;
extern PGDLLIMPORT PROC_HDR *ProcGlobal;

二、源码解读

提交记录和相关解析如下:

The new array contains the xids for all connected backends / in-use
PGPROC entries in a dense manner (in contrast to the PGPROC/PGXACT
arrays which can have unused entries interspersed).
主要的改动是对于处理中的xids使用稠密数组存储(相对于稀疏数组而言,每个数组项都是有效的).
This improves performance because GetSnapshotData() always needs to
scan the xids of all live procarray entries and now there's no need to
go through the procArray->pgprocnos indirection anymore.
这样的做法之所以有助于提升性能,原因是GetSnapshotData()函数经常需要扫描所有活动procarray条目中的xids,
修改之后不再需要通过procArray->pgprocnos进行间接访问.
As the set of running top-level xids changes rarely, compared to the
number of snapshots taken, this substantially increases the likelihood
of most data required for a snapshot being in l2 cache.  In
read-mostly workloads scanning the xids[] array will sufficient to
build a snapshot, as most backends will not have an xid assigned.
同时,相对于快照的数目,由于正在运行的顶层xids很少改动,
这样的处理可以提高快照需要的数据更多的留存在CPU L2缓存中.
在读为主的场景中,由于大多数后台进程不会分配xid,因此扫描xids[]数组对于构建快照已经足够了
To keep the xid array dense ProcArrayRemove() needs to move entries
behind the to-be-removed proc's one further up in the array. Obviously
moving array entries cannot happen while a backend sets it
xid. I.e. locking needs to prevent that array entries are moved while
a backend modifies its xid.
为了保持xid数组的稠密,函数ProcArrayRemove()需要移动在待移除的proc之后的条目,
显然,在设置xid时是不能移动数组条目的,因此需要锁定机制.
To avoid locking ProcArrayLock in GetNewTransactionId() - a fairly hot
spot already - ProcArrayAdd() / ProcArrayRemove() now needs to hold
XidGenLock in addition to ProcArrayLock. Adding / Removing a procarray
entry is not a very frequent operation, even taking 2PC into account.
为了避免在GetNewTransactionId()函数中锁定热点ProcArrayLock,
ProcArrayAdd()和ProcArrayRemove除了ProcArrayLock之外需要持有XidGenLock.
就算考虑到2PC,新增和移除procarray条目也不会频繁发生.
Due to the above, the dense array entries can only be read or modified
while holding ProcArrayLock and/or XidGenLock. This prevents a
concurrent ProcArrayRemove() from shifting the dense array while it is
accessed concurrently.
综上,稠密数组中的条目在持有ProcArrayLock和/或XidGenLock时才能读和修改.
这可以阻止在同步访问时ProcArrayRemove()函数同步移动稠密数组
While the new dense array is very good when needing to look at all
xids it is less suitable when accessing a single backend's xid. In
particular it would be problematic to have to acquire a lock to access
a backend's own xid. Therefore a backend's xid is not just stored in
the dense array, but also in PGPROC. This also allows a backend to
only access the shared xid value when the backend had acquired an
xid.
新的稠密数组在需要检索所有xids时表现得很好,在访问单个进程xid时则一般.
特别在不得不请求锁来访问后台进程自己的xid时表现尤其不好.
因此后台进程的xid不单单存储在稠密数组中,还存储在PGPROC中.
这允许在请求了事务号后台进程只需要访问共享的xid.
The infrastructure added in this commit will be used for the remaining
PGXACT fields in subsequent commits. They are kept separate to make
review easier.
这次提交增加的"基建"在后续的提交中用于PGXACT仍遗留的字段,
之所以还保存在这里是因为便于复审.

源码可参见参考资料中的链接。

三、跟踪分析

N/A

四、参考资料

1. Improving connection scalability: GetSnapshotData()
2. snapshot scalability: Introduce dense array of in-progress xids.