PostgreSQL 14在MVCC上面有所增强，以保证在大批量客户端连接场景下性能不会出现显著下降（详细可参见参考资料中的链接）。本节介绍了提交记录“snapshot scalability: cache snapshots using a xact completion counter.”和“Fix race condition in snapshot caching when 2PC is used.”中的修改以及涉及的相关数据结构。
一、数据结构
PROC_HDR
/*
* There is one ProcGlobal struct for the whole database cluster.
* 全局的ProcGlobal(PROC_HDR)结构体
*
* Adding/Removing an entry into the procarray requires holding *both*
* ProcArrayLock and XidGenLock in exclusive mode (in that order). Both are
* needed because the dense arrays (see below) are accessed from
* GetNewTransactionId() and GetSnapshotData(), and we don't want to add
* further contention by both using the same lock. Adding/Removing a procarray
* entry is much less frequent.
* 对proc数组的插入和删除,需要以独占的模式持有ProcArrayLock和XidGenLock,
* 原因是GetNewTransactionId()和GetSnapshotData()函数需要访问,并且我们不希望使用同样的锁来增加争用.
* 其实,插入和删除proc数组并不频繁.
*
* Some fields in PGPROC are mirrored into more densely packed arrays (e.g.
* xids), with one entry for each backend. These arrays only contain entries
* for PGPROCs that have been added to the shared array with ProcArrayAdd()
* (in contrast to PGPROC array which has unused PGPROCs interspersed).
* PGPROC中的一些字段镜像到更稠密的数组中(如xids),每一个后台进程都有一项.
* 这些数组只包含使用ProcArrayAdd()加入到共享数组的PGPROCs.
*
* The dense arrays are indexed by PGPROC->pgxactoff. Any concurrent
* ProcArrayAdd() / ProcArrayRemove() can lead to pgxactoff of a procarray
* member to change. Therefore it is only safe to use PGPROC->pgxactoff to
* access the dense array while holding either ProcArrayLock or XidGenLock.
* 稠密数组通过PGPROC->pgxactoff字段进行索引,所有同步ProcArrayAdd和ProcArrayRemove操作
* 会改变procarray中的pgxactoff成员变量.
*
* As long as a PGPROC is in the procarray, the mirrored values need to be
* maintained in both places in a coherent manner.
* 由于PGPROC在proc数组中,镜像值需要以一致的方式进行运维.
*
* The denser separate arrays are beneficial for three main reasons: First, to
* allow for as tight loops accessing the data as possible. Second, to prevent
* updates of frequently changing data (e.g. xmin) from invalidating
* cachelines also containing less frequently changing data (e.g. xid,
* statusFlags). Third to condense frequently accessed data into as few
* cachelines as possible.
* 基于以下3点原因,更稠密单独的数组会带来不少好处.
* 1.紧密的循环访问数据成为可能;
* 2.避免了对修改频度不高的数据进行修改导致CPU缓存无效的情况出现;
* 3.使频繁访问的数据放在小量的CPU缓存中成为可能.
*
* There are two main reasons to have the data mirrored between these dense
* arrays and PGPROC. First, as explained above, a PGPROC's array entries can
* only be accessed with either ProcArrayLock or XidGenLock held, whereas the
* PGPROC entries do not require that (obviously there may still be locking
* requirements around the individual field, separate from the concerns
* here). That is particularly important for a backend to efficiently checks
* it own values, which it often can safely do without locking. Second, the
* PGPROC fields allow to avoid unnecessary accesses and modification to the
* dense arrays. A backend's own PGPROC is more likely to be in a local cache,
* whereas the cachelines for the dense array will be modified by other
* backends (often removing it from the cache for other cores/sockets). At
* commit/abort time a check of the PGPROC value can avoid accessing/dirtying
* the corresponding array value.
* 让这些稠密数组和PGPROC之间生成数据镜像,有两个主要原因:
* 1.如上所述,PGPROC数组条目需要在持有ProcArrayLock和XidGenLock的前提下才能访问,但实际上并不需要.
* 在没有锁的情况下,后台进程有效且安全的检查自己的值是十分重要的.
* 2.PGPROC中的字段避免不必要的访问和修改.后台进程自己的PGPROC更应该在自己的本地缓存中,
* 而无需理会数组会被其他进程修改.在事务提交或回滚时,检查PGPROC值可以避免访问或者弄脏相应的数组值
*
* Basically it makes sense to access the PGPROC variable when checking a
* single backend's data, especially when already looking at the PGPROC for
* other reasons already. It makes sense to look at the "dense" arrays if we
* need to look at many / most entries, because we then benefit from the
* reduced indirection and better cross-process cache-ability.
* 检查单个后台进程数据时,访问PGPROC值是比较敏感的,特别是在已经因为其他原因检索过PGPROC的情况下
*
* When entering a PGPROC for 2PC transactions with ProcArrayAdd(), the data
* in the dense arrays is initialized from the PGPROC while it already holds
* ProcArrayLock.
* 在2PC事务的情况下,通过ProcArrayAdd方法进入PGPROC时,
* 数组中的数据在持有ProcArrayLock锁的情况下初始化数组.
*/
typedef struct PROC_HDR
{
/* Array of PGPROC structures (not including dummies for prepared txns) */
//PGPROC结构体数组
PGPROC *allProcs;
/* Array mirroring PGPROC.xid for each PGPROC currently in the procarray */
// 镜像每一个PGPROC.xid
TransactionId *xids;
/*
* Array mirroring PGPROC.subxidStatus for each PGPROC currently in the
* procarray.
*/
//镜像PGPROC.subxidStatus
XidCacheStatus *subxidStates;
/*
* Array mirroring PGPROC.statusFlags for each PGPROC currently in the
* procarray.
*/
//镜像PGPROC.statusFlags
uint8 *statusFlags;
/* Length of allProcs array */
//allProcs数组的大小
uint32 allProcCount;
/* Head of list of free PGPROC structures */
//空闲PGPROC结构体链表头
PGPROC *freeProcs;
/* Head of list of autovacuum's free PGPROC structures */
//autovacuum进程空闲PGPROC结构体链表头
PGPROC *autovacFreeProcs;
/* Head of list of bgworker free PGPROC structures */
//bgworker进程
PGPROC *bgworkerFreeProcs;
/* Head of list of walsender free PGPROC structures */
//walsender
PGPROC *walsenderFreeProcs;
/* First pgproc waiting for group XID clear */
//等待XID组清理的第一个pgproc
pg_atomic_uint32 procArrayGroupFirst;
/* First pgproc waiting for group transaction status update */
//等待事务状态组更新的首个pgproc
pg_atomic_uint32 clogGroupFirst;
/* WALWriter process's latch */
//WALWriter进程latch
Latch *walwriterLatch;
/* Checkpointer process's latch */
//检查点进程latch
Latch *checkpointerLatch;
/* Current shared estimate of appropriate spins_per_delay value */
//spins_per_delay
int spins_per_delay;
/* The proc of the Startup process, since not in ProcArray */
//Startup进程的proc结构体,不再Proc数组中
PGPROC *startupProc;
int startupProcPid;
/* Buffer id of the buffer that Startup process waits for pin on, or -1 */
//Startup进程等待pin的缓存ID
int startupBufferPinWaitBufId;
} PROC_HDR;
extern PGDLLIMPORT PROC_HDR *ProcGlobal;
二、源码解读
提交记录和相关解析如下:
Previous commits made it faster/more scalable to compute snapshots. But not
building a snapshot is still faster. Now that GetSnapshotData() does not
maintain RecentGlobal* anymore, that is actually not too hard:
先前的提交使计算快照变得更快和更有扩展性。但构建快照并不见得更快。现在GetSnapshotData
不再维护RecentGlobal*相关的数据结构，实际上实现起来也不麻烦：
This commit introduces xactCompletionCount, which tracks the number of
top-level transactions with xids (i.e. which may have modified the database)
that completed in some form since the start of the server.
这次提交介绍了xactCompletionCount，该变量跟踪具备xids的顶层事务（比如可能会更新数据库）。
We can avoid rebuilding the snapshot's contents whenever the current
xactCompletionCount is the same as it was when the snapshot was
originally built. Currently this check happens while holding
ProcArrayLock. While it's likely possible to perform the check without
acquiring ProcArrayLock, it seems better to do that separately /
later, some careful analysis is required. Even with the lock this is a
significant win on its own.
如果当前的xactCompletionCount与快照初始构建时一样，那么不再重新构建快照内容。
在持有ProcArrayLock锁时才执行该检查。由于没有该锁时并不会执行该检查，
因此单独或者延后执行检查看起来会更好，因为某些更仔细的分析是需要的。
On a smaller two socket machine this gains another ~1.03x, on a larger
machine the effect is roughly double (earlier patch version tested
though). If we were able to safely avoid the lock there'd be another
significant gain on top of that.
在小型的两路机器上，这样的修改可以获得1.03x的收益，在更大的机器上效果显著达到了原来的2倍。
如果我们可以安全的避免锁，那么会有更显著的收益。
源码解读
diff --git a/src/backend/replication/logical/snapbuild.c b/src/backend/replication/logical/snapbuild.c
index e9701ea7221549f02fb683766cbf024577066ca7..9d5d68f3fa7855daa4782cb162c7620ddb85bfc0 100644 (file)
--- a/src/backend/replication/logical/snapbuild.c
+++ b/src/backend/replication/logical/snapbuild.c
@@ -524,6 +524,7 @@ SnapBuildBuildSnapshot(SnapBuild *builder)
snapshot->curcid = FirstCommandId;
snapshot->active_count = 0;
snapshot->regd_count = 0;
//初始化变量snapXactCompletionCount
+ snapshot->snapXactCompletionCount = 0;
return snapshot;
}
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 96e4a87857602f03f9c856edefaa16af5e22e636..e687cde6f176fcf955df020ca29f3e1489dad0a0 100644 (file)
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -407,6 +407,7 @@ CreateSharedProcArray(void)
procArray->lastOverflowedXid = InvalidTransactionId;
procArray->replication_slot_xmin = InvalidTransactionId;
procArray->replication_slot_catalog_xmin = InvalidTransactionId;
//CreateSharedProcArray函数中,在初始化数组时,设置xactCompletionCount为1
+ ShmemVariableCache->xactCompletionCount = 1;
}
allProcs = ProcGlobal->allProcs;
@@ -534,6 +535,9 @@ ProcArrayRemove(PGPROC *proc, TransactionId latestXid)
/* Advance global latestCompletedXid while holding the lock */
MaintainLatestCompletedXid(latestXid);
+ /* Same with xactCompletionCount */
//增加1(ProcArrayRemove函数)
+ ShmemVariableCache->xactCompletionCount++;
+
ProcGlobal->xids[proc->pgxactoff] = 0;
ProcGlobal->subxidStates[proc->pgxactoff].overflowed = false;
ProcGlobal->subxidStates[proc->pgxactoff].count = 0;
@@ -667,6 +671,7 @@ ProcArrayEndTransactionInternal(PGPROC *proc, TransactionId latestXid)
{
size_t pgxactoff = proc->pgxactoff;
//验证是否持有锁
+ Assert(LWLockHeldByMe(ProcArrayLock));
Assert(TransactionIdIsValid(ProcGlobal->xids[pgxactoff]));
Assert(ProcGlobal->xids[pgxactoff] == proc->xid);
@@ -698,6 +703,9 @@ ProcArrayEndTransactionInternal(PGPROC *proc, TransactionId latestXid)
/* Also advance global latestCompletedXid while holding the lock */
MaintainLatestCompletedXid(latestXid);
+
+ /* Same with xactCompletionCount */
//增加1(ProcArrayEndTransactionInternal函数)
+ ShmemVariableCache->xactCompletionCount++;
}
/*
@@ -1916,6 +1924,93 @@ GetMaxSnapshotSubxidCount(void)
return TOTAL_MAX_CACHED_SUBXIDS;
}
+/*
+ * Initialize old_snapshot_threshold specific parts of a newly build snapshot.
初始化新构建快照中old_snapshot_threshold的指定域(启用了快照过旧特性)
+ */
+static void
+GetSnapshotDataInitOldSnapshot(Snapshot snapshot)
+{
+ if (!OldSnapshotThresholdActive())
+ {
+ /*
+ * If not using "snapshot too old" feature, fill related fields with
+ * dummy values that don't require any locking.
+ */
//没有启用快照过旧特性,填充相关域值
+ snapshot->lsn = InvalidXLogRecPtr;
+ snapshot->whenTaken = 0;
+ }
+ else
+ {
+ /*
+ * Capture the current time and WAL stream location in case this
+ * snapshot becomes old enough to need to fall back on the special
+ * "old snapshot" logic.
启用了快照过旧特性,收集当前时间和WAL位置并填充到相应的位置中
+ */
+ snapshot->lsn = GetXLogInsertRecPtr();
+ snapshot->whenTaken = GetSnapshotCurrentTimestamp();
+ MaintainOldSnapshotTimeMapping(snapshot->whenTaken, snapshot->xmin);
+ }
+}
+
+/*
+ * Helper function for GetSnapshotData() that checks if the bulk of the
+ * visibility information in the snapshot is still valid. If so, it updates
+ * the fields that need to change and returns true. Otherwise it returns
+ * false.
GetSnapshotData的助手函数,用于检查快照中可见性信息是否仍然有效.
如有效,则更新需要更新的域并返回true,否则返回false.
+ *
+ * This very likely can be evolved to not need ProcArrayLock held (at very
+ * least in the case we already hold a snapshot), but that's for another day.
+ */
+static bool
+GetSnapshotDataReuse(Snapshot snapshot)
+{
+ uint64 curXactCompletionCount;
+
+ Assert(LWLockHeldByMe(ProcArrayLock));
+
+ if (unlikely(snapshot->snapXactCompletionCount == 0))
+ return false;
+
//从共享内存中获取事务计数
+ curXactCompletionCount = ShmemVariableCache->xactCompletionCount;
//不相等,则返回false
+ if (curXactCompletionCount != snapshot->snapXactCompletionCount)
+ return false;
+
+ /*
+ * If the current xactCompletionCount is still the same as it was at the
+ * time the snapshot was built, we can be sure that rebuilding the
+ * contents of the snapshot the hard way would result in the same snapshot
+ * contents:
如果当前的xactCompletionCount与快照建立时的计数一样,那我们可以断定重新构建快照内容
其实跟原来是一样的.
+ *
+ * As explained in transam/README, the set of xids considered running by
+ * GetSnapshotData() cannot change while ProcArrayLock is held. Snapshot
+ * contents only depend on transactions with xids and xactCompletionCount
+ * is incremented whenever a transaction with an xid finishes (while
+ * holding ProcArrayLock) exclusively). Thus the xactCompletionCount check
+ * ensures we would detect if the snapshot would have changed.
就如在transam/README中已解释过的,
GetSnapshotData认为正在运行的xids集合在持有ProcArrayLock锁时是不能改变的.
快照内容仅仅依赖于已分配xids的事务,而在事务结束时xactCompletionCount会加一.
因此xactCompletionCount的检查确保可以侦测到快照的变化.
+ *
+ * As the snapshot contents are the same as it was before, it is is safe
+ * to re-enter the snapshot's xmin into the PGPROC array. None of the rows
+ * visible under the snapshot could already have been removed (that'd
+ * require the set of running transactions to change) and it fulfills the
+ * requirement that concurrent GetSnapshotData() calls yield the same
+ * xmin.
由于快照内容与先前的一样,重新把快照的xmin放到PGPROC数组中是安全的.
+ */
+ if (!TransactionIdIsValid(MyProc->xmin))
+ MyProc->xmin = TransactionXmin = snapshot->xmin;//更新xmin
+
+ RecentXmin = snapshot->xmin;
+ Assert(TransactionIdPrecedesOrEquals(TransactionXmin, RecentXmin));
+
//更新snapshot的相关信息
+ snapshot->curcid = GetCurrentCommandId(false);
+ snapshot->active_count = 0;
+ snapshot->regd_count = 0;
+ snapshot->copied = false;
+
//初始化快照过旧的信息
+ GetSnapshotDataInitOldSnapshot(snapshot);
+
//返回true
+ return true;
+}
+
/*
* GetSnapshotData -- returns information about running transactions.
*
@@ -1963,6 +2058,7 @@ GetSnapshotData(Snapshot snapshot)
TransactionId oldestxid;
int mypgxactoff;
TransactionId myxid;
+ uint64 curXactCompletionCount;
TransactionId replication_slot_xmin = InvalidTransactionId;
TransactionId replication_slot_catalog_xmin = InvalidTransactionId;
@@ -2007,12 +2103,19 @@ GetSnapshotData(Snapshot snapshot)
*/
LWLockAcquire(ProcArrayLock, LW_SHARED);
//快照是否可重用?如可以,则直接返回
+ if (GetSnapshotDataReuse(snapshot))
+ {
+ LWLockRelease(ProcArrayLock);
+ return snapshot;
+ }
+
latest_completed = ShmemVariableCache->latestCompletedXid;
mypgxactoff = MyProc->pgxactoff;
myxid = other_xids[mypgxactoff];
Assert(myxid == MyProc->xid);
oldestxid = ShmemVariableCache->oldestXid;
//更新事务计数
+ curXactCompletionCount = ShmemVariableCache->xactCompletionCount;
/* xmax is always latestCompletedXid + 1 */
xmax = XidFromFullTransactionId(latest_completed);
@@ -2266,6 +2369,7 @@ GetSnapshotData(Snapshot snapshot)
snapshot->xcnt = count;
snapshot->subxcnt = subcount;
snapshot->suboverflowed = suboverflowed;
//更新快照事务计数
+ snapshot->snapXactCompletionCount = curXactCompletionCount;
snapshot->curcid = GetCurrentCommandId(false);
@@ -2277,26 +2381,7 @@ GetSnapshotData(Snapshot snapshot)
snapshot->regd_count = 0;
snapshot->copied = false;
//下面的代码已移入GetSnapshotDataReuse函数
- if (old_snapshot_threshold < 0)
- {
- /*
- * If not using "snapshot too old" feature, fill related fields with
- * dummy values that don't require any locking.
- */
- snapshot->lsn = InvalidXLogRecPtr;
- snapshot->whenTaken = 0;
- }
- else
- {
- /*
- * Capture the current time and WAL stream location in case this
- * snapshot becomes old enough to need to fall back on the special
- * "old snapshot" logic.
- */
- snapshot->lsn = GetXLogInsertRecPtr();
- snapshot->whenTaken = GetSnapshotCurrentTimestamp();
- MaintainOldSnapshotTimeMapping(snapshot->whenTaken, xmin);
- }
//改为调用函数
+ GetSnapshotDataInitOldSnapshot(snapshot);
return snapshot;
}
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index c208538e2e5ca32d6e9ce1d8bb61fe56d54f704e..22cf3ebaf4728d0295fbed11dec3fe00e7e81d7a 100644 (file)
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -597,6 +597,8 @@ SetTransactionSnapshot(Snapshot sourcesnap, VirtualTransactionId *sourcevxid,
CurrentSnapshot->takenDuringRecovery = sourcesnap->takenDuringRecovery;
/* NB: curcid should NOT be copied, it's a local matter */
//设置快照计数
+ CurrentSnapshot->snapXactCompletionCount = 0;
+
/*
* Now we have to fix what GetSnapshotData did with MyProc->xmin and
* TransactionXmin. There is a race condition: to make sure we are not
@@ -672,6 +674,7 @@ CopySnapshot(Snapshot snapshot)
newsnap->regd_count = 0;
newsnap->active_count = 0;
newsnap->copied = true;
//设置快照计数
+ newsnap->snapXactCompletionCount = 0;
/* setup XID array */
if (snapshot->xcnt > 0)
@@ -2209,6 +2212,7 @@ RestoreSnapshot(char *start_address)
snapshot->curcid = serialized_snapshot.curcid;
snapshot->whenTaken = serialized_snapshot.whenTaken;
snapshot->lsn = serialized_snapshot.lsn;
//设置快照计数
+ snapshot->snapXactCompletionCount = 0;
/* Copy XIDs, if present. */
if (serialized_snapshot.xcnt > 0)
diff --git a/src/include/access/transam.h b/src/include/access/transam.h
index b32044153b09d9fac6264c4812427294d624360f..2f1f144db4d062a374104bc62d382d0d5c7c4de7 100644 (file)
--- a/src/include/access/transam.h
+++ b/src/include/access/transam.h
@@ -231,6 +231,15 @@ typedef struct VariableCacheData
FullTransactionId latestCompletedXid; /* newest full XID that has
* committed or aborted */
+ /*
+ * Number of top-level transactions with xids (i.e. which may have
+ * modified the database) that completed in some form since the start of
+ * the server. This currently is solely used to check whether
+ * GetSnapshotData() needs to recompute the contents of the snapshot, or
+ * not. There are likely other users of this. Always above 1.
+ */
//VariableCacheData数据结构,新增xactCompletionCount
+ uint64 xactCompletionCount;
+
/*
* These fields are protected by XactTruncationLock
*/
diff --git a/src/include/utils/snapshot.h b/src/include/utils/snapshot.h
index 35b1f05bea6593c6c5df28b3fb3bbf51b132c6a0..dea072e5edf5e881e172cebb81a5c8c29398e65c 100644 (file)
--- a/src/include/utils/snapshot.h
+++ b/src/include/utils/snapshot.h
@@ -207,6 +207,13 @@ typedef struct SnapshotData
TimestampTz whenTaken; /* timestamp when snapshot was taken */
XLogRecPtr lsn; /* position in the WAL stream when taken */
+
+ /*
+ * The transaction completion count at the time GetSnapshotData() built
+ * this snapshot. Allows to avoid re-computing static snapshots when no
+ * transactions completed since the last GetSnapshotData().
+ */
//SnapshotData数据结构,新增xactCompletionCount
+ uint64 snapXactCompletionCount;
} SnapshotData;
#endif /* SNAPSHOT_H */
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index e687cde6f176fcf955df020ca29f3e1489dad0a0..51f8099cad2ca43d7be1c5b5f66426d282fdbcb1 100644 (file)
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -860,6 +860,15 @@ ProcArrayClearTransaction(PGPROC *proc)
Assert(!(proc->vacuumFlags & PROC_VACUUM_STATE_MASK));
Assert(!proc->delayChkpt);
+ /*
+ * Need to increment completion count even though transaction hasn't
+ * really committed yet. The reason for that is that GetSnapshotData()
+ * omits the xid of the current transaction, thus without the increment we
+ * otherwise could end up reusing the snapshot later. Which would be bad,
+ * because it might not count the prepared transaction as running.
+ */
//增加xactCompletionCount计数
+ ShmemVariableCache->xactCompletionCount++;
+
/* Clear the subtransaction-XID cache too */
Assert(ProcGlobal->subxidStates[pgxactoff].count == proc->subxidStatus.count &&
ProcGlobal->subxidStates[pgxactoff].overflowed == proc->subxidStatus.overflowed);
三、跟踪分析
N/A
四、参考资料
1.
Improving connection scalability: GetSnapshotData()
2.
snapshot scalability: cache snapshots using a xact completion counter.
3.
Fix race condition in snapshot caching when 2PC is used.