PostgreSQL 14在MVCC上面有所增强，以保证在大批量客户端连接场景下性能不会出现显著下降（详细可参见参考资料中的链接）。本节介绍了提交记录“Don’t compute global horizons while building snapshots.”中procarray.c中的修改以及涉及的相关数据结构。
一、数据结构
N/A
二、源码解读
回顾这次代码commit的说明,会有助于理解committer为什么这样修改:
To make GetSnapshotData() more scalable, it cannot not look at at each proc's
xmin: While snapshot contents do not need to change whenever a read-only
transaction commits or a snapshot is released, a proc's xmin is modified in
those cases. The frequency of xmin modifications leads to, particularly on
higher core count systems, many cache misses inside GetSnapshotData(), despite
the data underlying a snapshot not changing. That is the most
significant source of GetSnapshotData() scaling poorly on larger systems.
GetSnapshotData的性能随着连接的增长会出现明显的下降,其根源是xmin的频繁更新导致该函数
在执行过程中出现许多缓存没有命中的情况,导致效率低下.
对于只读事务或者快照释放xmin无需更新的情况,内核也更新了xmin.
Without accessing xmins, GetSnapshotData() cannot calculate accurate horizons /
thresholds as it has so far. But we don't really have to: The horizons don't
actually change that much between GetSnapshotData() calls. Nor are the horizons
actually used every time a snapshot is built.
如果不访问PROC的xmins,GetSnapshotData不能计算精确的基准线/阈值.
但实际上在GetSnapshotData调用时并不需要更新基准线,在快照建立时也不需要使用基准线.
The trick this commit introduces is to delay computation of accurate horizons
until there use and using horizon boundaries to determine whether accurate
horizons need to be computed.
因此,在需要的时候才计算精确的基准线.
The use of RecentGlobal[Data]Xmin to decide whether a row version could be
removed has been replaces with new GlobalVisTest* functions. These use two
thresholds to determine whether a row can be pruned:
1) definitely_needed, indicating that rows deleted by XIDs >= definitely_needed
are definitely still visible.
2) maybe_needed, indicating that rows deleted by XIDs < maybe_needed can
definitely be removed
GetSnapshotData() updates definitely_needed to be the xmin of the computed
snapshot.
When testing whether a row can be removed (with GlobalVisTestIsRemovableXid())
and the tested XID falls in between the two (i.e. XID >= maybe_needed && XID <
definitely_needed) the boundaries can be recomputed to be more accurate. As it
is not cheap to compute accurate boundaries, we limit the number of times that
happens in short succession. As the boundaries used by
GlobalVisTestIsRemovableXid() are never reset (with maybe_needed updated by
GetSnapshotData()), it is likely that further test can benefit from an earlier
computation of accurate horizons.
To avoid regressing performance when old_snapshot_threshold is set (as that
requires an accurate horizon to be computed), heap_page_prune_opt() doesn't
unconditionally call TransactionIdLimitedForOldSnapshots() anymore. Both the
computation of the limited horizon, and the triggering of errors (with
SetOldSnapshotThresholdTimestamp()) is now only done when necessary to remove
tuples.
This commit just removes the accesses to PGXACT->xmin from
GetSnapshotData(), but other members of PGXACT residing in the same
cache line are accessed. Therefore this in itself does not result in a
significant improvement. Subsequent commits will take advantage of the
fact that GetSnapshotData() now does not need to access xmins anymore.
本次提交只是移除了GetSnapshotData函数对PGXACT->xmin的访问,因此本次提交并没有带来明显的性能提升.
Note: This contains a workaround in heap_page_prune_opt() to keep the
snapshot_too_old tests working. While that workaround is ugly, the tests
currently are not meaningful, and it seems best to address them separately.
procarray.c(第一部分)
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -99,6 +99,142 @@ typedef struct ProcArrayStruct
int pgprocnos[FLEXIBLE_ARRAY_MEMBER];
} ProcArrayStruct;
+/*
+ * State for the GlobalVisTest* family of functions. Those functions can
+ * e.g. be used to decide if a deleted row can be removed without violating
+ * MVCC semantics: If the deleted row's xmax is not considered to be running
+ * by anyone, the row can be removed.
/*
用于GlobalVisTest函数族的状态。这些函数的作用包括用于确定删除的行是否在不违反MVCC语义的情况下清除：
如果被删除行的的xmax不会被其他正在运行的事务所看到，那么可以被清除。
*/
+ *
+ * To avoid slowing down GetSnapshotData(), we don't calculate a precise
+ * cutoff XID while building a snapshot (looking at the frequently changing
+ * xmins scales badly). Instead we compute two boundaries while building the
+ * snapshot:
+ *
+ * 1) definitely_needed, indicating that rows deleted by XIDs >=
+ * definitely_needed are definitely still visible.
+ *
+ * 2) maybe_needed, indicating that rows deleted by XIDs < maybe_needed can
+ * definitely be removed
/*
为了避免降低GetSnapshotData的性能，在构建快照时，我们不会计算精确的cutoff XID，
相反，在构建快照时，我们会计算两个边界：
1)definitely_needed，满足XIDs >= definitely_needed的事务，仍是可见的
2)maybe_needed，满足XIDs < maybe_needed，可以被清除
*/
+ *
+ * When testing an XID that falls in between the two (i.e. XID >= maybe_needed
+ * && XID < definitely_needed), the boundaries can be recomputed (using
+ * ComputeXidHorizons()) to get a more accurate answer. This is cheaper than
+ * maintaining an accurate value all the time.
/*
如果XID在两者之间（XID >= maybe_needed && XID < definitely_needed）,
边界会使用函数ComputeXidHorizons重新计算来获得更精确的结果。
这个操作的成本会比全程维护一个精确值要低得多。
*/
+ *
+ * As it is not cheap to compute accurate boundaries, we limit the number of
+ * times that happens in short succession. See GlobalVisTestShouldUpdate().
/*
计算精确的边界成本不低，需要控制次数，详见GlobalVisTestShouldUpdate。
*/
+ *
+ *
+ * There are three backend lifetime instances of this struct, optimized for
+ * different types of relations. As e.g. a normal user defined table in one
+ * database is inaccessible to backends connected to another database, a test
+ * specific to a relation can be more aggressive than a test for a shared
+ * relation. Currently we track three different states:
+ *
+ * 1) GlobalVisSharedRels, which only considers an XID's
+ * effects visible-to-everyone if neither snapshots in any database, nor a
+ * replication slot's xmin, nor a replication slot's catalog_xmin might
+ * still consider XID as running.
+ *
+ * 2) GlobalVisCatalogRels, which only considers an XID's
+ * effects visible-to-everyone if neither snapshots in the current
+ * database, nor a replication slot's xmin, nor a replication slot's
+ * catalog_xmin might still consider XID as running.
+ *
+ * I.e. the difference to GlobalVisSharedRels is that
+ * snapshot in other databases are ignored.
+ *
+ * 3) GlobalVisCatalogRels, which only considers an XID's
+ * effects visible-to-everyone if neither snapshots in the current
+ * database, nor a replication slot's xmin consider XID as running.
+ *
+ * I.e. the difference to GlobalVisCatalogRels is that
+ * replication slot's catalog_xmin is not taken into account.
/*
参见结构体GlobalVisState说明
*/
+ *
+ * GlobalVisTestFor(relation) returns the appropriate state
+ * for the relation.
+ *
+ * The boundaries are FullTransactionIds instead of TransactionIds to avoid
+ * wraparound dangers. There e.g. would otherwise exist no procarray state to
+ * prevent maybe_needed to become old enough after the GetSnapshotData()
+ * call.
+ *
+ * The typedef is in the header.
+ */
/*
参见结构体GlobalVisState说明
*/
+struct GlobalVisState
+{
+ /* XIDs >= are considered running by some backend */
+ FullTransactionId definitely_needed;
+
+ /* XIDs < are not considered to be running by any backend */
+ FullTransactionId maybe_needed;
+};
+
+/*
+ * Result of ComputeXidHorizons().
+ */
/*
参见结构体ComputeXidHorizonsResult说明
*/
+typedef struct ComputeXidHorizonsResult
+{
+ /*
+ * The value of ShmemVariableCache->latestCompletedXid when
+ * ComputeXidHorizons() held ProcArrayLock.
+ */
+ FullTransactionId latest_completed;
+
+ /*
+ * The same for procArray->replication_slot_xmin and.
+ * procArray->replication_slot_catalog_xmin.
+ */
+ TransactionId slot_xmin;
+ TransactionId slot_catalog_xmin;
+
+ /*
+ * Oldest xid that any backend might still consider running. This needs to
+ * include processes running VACUUM, in contrast to the normal visibility
+ * cutoffs, as vacuum needs to be able to perform pg_subtrans lookups when
+ * determining visibility, but doesn't care about rows above its xmin to
+ * be removed.
+ *
+ * This likely should only be needed to determine whether pg_subtrans can
+ * be truncated. It currently includes the effects of replications slots,
+ * for historical reasons. But that could likely be changed.
+ */
+ TransactionId oldest_considered_running;
+
+ /*
+ * Oldest xid for which deleted tuples need to be retained in shared
+ * tables.
+ *
+ * This includes the effects of replications lots. If that's not desired,
+ * look at shared_oldest_nonremovable_raw;
+ */
+ TransactionId shared_oldest_nonremovable;
+
+ /*
+ * Oldest xid that may be necessary to retain in shared tables. This is
+ * the same as shared_oldest_nonremovable, except that is not affected by
+ * replication slot's catalog_xmin.
+ *
+ * This is mainly useful to be able to send the catalog_xmin to upstream
+ * streaming replication servers via hot_standby_feedback, so they can
+ * apply the limit only when accessing catalog tables.
+ */
+ TransactionId shared_oldest_nonremovable_raw;
+
+ /*
+ * Oldest xid for which deleted tuples need to be retained in non-shared
+ * catalog tables.
+ */
+ TransactionId catalog_oldest_nonremovable;
+
+ /*
+ * Oldest xid for which deleted tuples need to be retained in normal user
+ * defined tables.
+ */
+ TransactionId data_oldest_nonremovable;
+} ComputeXidHorizonsResult;
+
+
static ProcArrayStruct *procArray;
static PGPROC *allProcs;
@@ -118,6 +254,22 @@ static TransactionId latestObservedXid = InvalidTransactionId;
*/
static TransactionId standbySnapshotPendingXmin;
+/*
+ * State for visibility checks on different types of relations. See struct
+ * GlobalVisState for details. As shared, catalog, and user defined
+ * relations can have different horizons, one such state exists for each.
+ */
/*
不同的relation有不同的可见性检查状态，详细参见GlobalVisState
共享、字典和用户定义relation有不同的边界，因此每种relation有一个结构体与其对应
*/
+static GlobalVisState GlobalVisSharedRels;
+static GlobalVisState GlobalVisCatalogRels;
+static GlobalVisState GlobalVisDataRels;
+
+/*
+ * This backend's RecentXmin at the last time the accurate xmin horizon was
+ * recomputed, or InvalidTransactionId if it has not. Used to limit how many
+ * times accurate horizons are recomputed. See GlobalVisTestShouldUpdate().
+ */
/*
精确的xmin边界在重新计算时最后时刻后台进程的RecentXmin，如没有计算则该值为InvalidTransactionId。
用于限制精确边界计算的次数，详细参见GlobalVisTestShouldUpdate
*/
+static TransactionId ComputeXidHorizonsResultLastXmin;
+
#ifdef XIDCACHE_DEBUG
/* counters for XidCache measurement */
@@ -180,6 +332,7 @@ static void MaintainLatestCompletedXidRecovery(TransactionId latestXid);
static inline FullTransactionId FullXidRelativeTo(FullTransactionId rel,
TransactionId xid);
+static void GlobalVisUpdateApply(ComputeXidHorizonsResult *horizons);
/*
* Report shared-memory space needed by CreateSharedProcArray.
@@ -1302,159 +1455,191 @@ TransactionIdIsActive(TransactionId xid)
/*
TransactionIdIsActive : 判断xid是否活跃后台进程最高层的XID
*/
/*
- * GetOldestXmin -- returns oldest transaction that was running
- * when any current transaction was started.
+ * Determine XID horizons.
/*
ComputeXidHorizons :
确定XID水平线（也就是MVCC所需要的边界），原来使用GetOldestXmin，现在不需要了
*/
*
- * If rel is NULL or a shared relation, all backends are considered, otherwise
- * only backends running in this database are considered.
+ * This is used by wrapper functions like GetOldestNonRemovableTransactionId()
+ * (for VACUUM), GetReplicationHorizons() (for hot_standby_feedback), etc as
+ * well as "internally" by GlobalVisUpdate() (see comment above struct
+ * GlobalVisState).
/*
ComputeXidHorizons函数会被其他函数如GetOldestNonRemovableTransactionId()（用于VACUUM）、
GetReplicationHorizons（用于hot_standby_feedback）、GlobalVisUpdate（内部函数）等函数调用
详细可参见GlobalVisState
*/
*
- * The flags are used to ignore the backends in calculation when any of the
- * corresponding flags is set. Typically, if you want to ignore ones with
- * PROC_IN_VACUUM flag, you can use PROCARRAY_FLAGS_VACUUM.
+ * See the definition of ComputedXidHorizonsResult for the various computed
+ * horizons.
/*
对于返回的结果，参见ComputeXidHorizonsResult（不是ComputedXidHorizonsResult）
*/
*
- * PROCARRAY_SLOTS_XMIN causes GetOldestXmin to ignore the xmin and
- * catalog_xmin of any replication slots that exist in the system when
- * calculating the oldest xmin.
+ * For VACUUM separate horizons (used to to decide which deleted tuples must
+ * be preserved), for shared and non-shared tables are computed. For shared
+ * relations backends in all databases must be considered, but for non-shared
+ * relations that's not required, since only backends in my own database could
+ * ever see the tuples in them. Also, we can ignore concurrently running lazy
+ * VACUUMs because (a) they must be working on other tables, and (b) they
+ * don't need to do snapshot-based lookups.
/*
对于VACUUM的水平线（用于确定哪些被删除的元组需要保留），共享&非共享表都会计算。
对于共享关系，所有数据库都必须考虑到，但对于非共享关系并不需要，
因为只有自己数据库中的后台进程才能看到这些元组。
同时，我们可以忽略并发执行的lazy VACUUMs，因为这些进程比如会处理其他表并且不需要执行基于快照的检索
*/
*
- * This is used by VACUUM to decide which deleted tuples must be preserved in
- * the passed in table. For shared relations backends in all databases must be
- * considered, but for non-shared relations that's not required, since only
- * backends in my own database could ever see the tuples in them. Also, we can
- * ignore concurrently running lazy VACUUMs because (a) they must be working
- * on other tables, and (b) they don't need to do snapshot-based lookups.
- *
- * This is also used to determine where to truncate pg_subtrans. For that
- * backends in all databases have to be considered, so rel = NULL has to be
- * passed in.
+ * This also computes a horizon used to truncate pg_subtrans. For that
+ * backends in all databases have to be considered, and concurrently running
+ * lazy VACUUMs cannot be ignored, as they still may perform pg_subtrans
+ * accesses.
/*
计算用于截断pg_subtrans的水平线。
为此，需要考虑所有数据库中的后台进程，而且不能忽略lazy VACUUMs，
因为这些进程在访问pg_subtrans
*/
*
* Note: we include all currently running xids in the set of considered xids.
* This ensures that if a just-started xact has not yet set its snapshot,
* when it does set the snapshot it cannot set xmin less than what we compute.
* See notes in src/backend/access/transam/README.
*
- * Note: despite the above, it's possible for the calculated value to move
- * backwards on repeated calls. The calculated value is conservative, so that
- * anything older is definitely not considered as running by anyone anymore,
- * but the exact value calculated depends on a number of things. For example,
- * if rel = NULL and there are no transactions running in the current
- * database, GetOldestXmin() returns latestCompletedXid. If a transaction
+ * Note: despite the above, it's possible for the calculated values to move
+ * backwards on repeated calls. The calculated values are conservative, so
+ * that anything older is definitely not considered as running by anyone
+ * anymore, but the exact values calculated depend on a number of things. For
+ * example, if there are no transactions running in the current database, the
+ * horizon for normal tables will be latestCompletedXid. If a transaction
* begins after that, its xmin will include in-progress transactions in other
* databases that started earlier, so another call will return a lower value.
* Nonetheless it is safe to vacuum a table in the current database with the
* first result. There are also replication-related effects: a walsender
* process can set its xmin based on transactions that are no longer running
* on the primary but are still being replayed on the standby, thus possibly
- * making the GetOldestXmin reading go backwards. In this case there is a
- * possibility that we lose data that the standby would like to have, but
- * unless the standby uses a replication slot to make its xmin persistent
- * there is little we can do about that --- data is only protected if the
- * walsender runs continuously while queries are executed on the standby.
- * (The Hot Standby code deals with such cases by failing standby queries
- * that needed to access already-removed data, so there's no integrity bug.)
- * The return value is also adjusted with vacuum_defer_cleanup_age, so
- * increasing that setting on the fly is another easy way to make
- * GetOldestXmin() move backwards, with no consequences for data integrity.
+ * making the values go backwards. In this case there is a possibility that
+ * we lose data that the standby would like to have, but unless the standby
+ * uses a replication slot to make its xmin persistent there is little we can
+ * do about that --- data is only protected if the walsender runs continuously
+ * while queries are executed on the standby. (The Hot Standby code deals
+ * with such cases by failing standby queries that needed to access
+ * already-removed data, so there's no integrity bug.) The computed values
+ * are also adjusted with vacuum_defer_cleanup_age, so increasing that setting
+ * on the fly is another easy way to make horizons move backwards, with no
+ * consequences for data integrity.
/*
注意：尽管存在上述情况，但计算值在重复调用时仍有后移的可能。计算值是“保守派”，
任何比此值要旧的都确定不会被运行的事务所看到，但正确的值的计算需要依赖很多东西。
比如，如果在当前数据库中没有正在运行中的事务，普通表的边界会是latestCompletedXid。
如果事务的事务号比此事务号要大，xmin会包含在其他数据库中比此事务要早的正在处理的事务，
因此另外的调用会返回更小的值。
尽管如此，在当前数据库中使用上一个结果来vacuum一张表仍然是安全的。
另外，存在复制相关的影响：walsender进程基于不在主节点运行但仍在备节点运行的事务来设置它自己的xmin，
因此构造该值会出现后移的情况。
在这种情况下，存在备节点丢失数据的可能，除非备节点使用复制槽来构造它的xmin从而使其持久化。
数据只有在walsender在备节点查询期间持续运行时才受保护。
（备节点通过废弃备节点查询来因应对这样的情况，备节点可能会查询已经被清除的数据，因此不存在一致性的缺陷）
计算值通过vacuum_defer_cleanup_age来调整，因此增加vacuum_defer_cleanup_age是另外一种容易实现的方法，
从而使水平线后移而不需要影响数据一致性。
*/
+ *
+ * Note: the approximate horizons (see definition of GlobalVisState) are
+ * updated by the computations done here. That's currently required for
+ * correctness and a small optimization. Without doing so it's possible that
+ * heap vacuum's call to heap_page_prune() uses a more conservative horizon
+ * than later when deciding which tuples can be removed - which the code
+ * doesn't expect (breaking HOT).
*/
-TransactionId
-GetOldestXmin(Relation rel, int flags)
+static void
+ComputeXidHorizons(ComputeXidHorizonsResult *h)
{
ProcArrayStruct *arrayP = procArray;
- TransactionId result;
- int index;
- bool allDbs;
-
- TransactionId replication_slot_xmin = InvalidTransactionId;
- TransactionId replication_slot_catalog_xmin = InvalidTransactionId;
-
- /*
- * If we're not computing a relation specific limit, or if a shared
- * relation has been passed in, backends in all databases have to be
- * considered.
- */
- allDbs = rel == NULL || rel->rd_rel->relisshared;
+ TransactionId kaxmin;//KnownAssignedMin
+ bool in_recovery = RecoveryInProgress();//判断是否处于恢复过程
- /* Cannot look for individual databases during recovery */
- Assert(allDbs || !RecoveryInProgress());
+ /* inferred after ProcArrayLock is released */
+ h->catalog_oldest_nonremovable = InvalidTransactionId;//非共享字典表不可清除xmin
LWLockAcquire(ProcArrayLock, LW_SHARED);
+ h->latest_completed = ShmemVariableCache->latestCompletedXid;//最后提交的XID
+
/*
* We initialize the MIN() calculation with latestCompletedXid + 1. This
* is a lower bound for the XIDs that might appear in the ProcArray later,
* and so protects us against overestimating the result due to future
* additions.
* latestCompletedXid + 1其实就是xmax，这是后续出现在ProcArray中XIDs的下界
*/
- result = XidFromFullTransactionId(ShmemVariableCache->latestCompletedXid);
- TransactionIdAdvance(result);
- Assert(TransactionIdIsNormal(result));
+ {
+ TransactionId initial;//初始的事务ID
- for (index = 0; index < arrayP->numProcs; index++)
+ initial = XidFromFullTransactionId(h->latest_completed);//初始化为latestCompletedXid
+ Assert(TransactionIdIsValid(initial));
+ TransactionIdAdvance(initial);//initial++，即latestCompletedXid + 1
+
+ h->oldest_considered_running = initial;
+ h->shared_oldest_nonremovable = initial;
+ h->data_oldest_nonremovable = initial;
+ }
+
+ /*
+ * Fetch slot horizons while ProcArrayLock is held - the
+ * LWLockAcquire/LWLockRelease are a barrier, ensuring this happens inside
+ * the lock.
/*
在持有ProcArrayLock的前提下，提取槽边界（LWLockAcquire/LWLockRelease是围栏，确保在持有锁的情况才能处理）
*/
+ */
+ h->slot_xmin = procArray->replication_slot_xmin;// 直接赋值
+ h->slot_catalog_xmin = procArray->replication_slot_catalog_xmin;// 直接赋值
+
+ for (int index = 0; index < arrayP->numProcs; index++)//遍历ProcArrayStruct
{
int pgprocno = arrayP->pgprocnos[index];
PGPROC *proc = &allProcs[pgprocno];// allProcs : static PGPROC *allProcs;
PGXACT *pgxact = &allPgXact[pgprocno];// 该Procs中进程对应的事务信息（后面会废弃该全局变量allPgXact）
+ TransactionId xid;
+ TransactionId xmin;
+
+ /* Fetch xid just once - see GetNewTransactionId */
// #define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
+ xid = UINT32_ACCESS_ONCE(pgxact->xid);
+ xmin = UINT32_ACCESS_ONCE(pgxact->xmin);
+
+ /*
+ * Consider both the transaction's Xmin, and its Xid.
+ *
+ * We must check both because a transaction might have an Xmin but not
+ * (yet) an Xid; conversely, if it has an Xid, that could determine
+ * some not-yet-set Xmin.
+ */
//比较事务xmin和xid,取其中最旧的那个
+ xmin = TransactionIdOlder(xmin, xid);
- if (pgxact->vacuumFlags & (flags & PROCARRAY_PROC_FLAGS_MASK))
+ /* if neither is set, this proc doesn't influence the horizon */
+ if (!TransactionIdIsValid(xmin))
continue;//事务启动但没有分配xmin,则继续下一个PGPROC
- if (allDbs ||
+ /*
+ * Don't ignore any procs when determining which transactions might be
+ * considered running. While slots should ensure logical decoding
+ * backends are protected even without this check, it can't hurt to
+ * include them here as well..
+ */
/*
在确定哪个事务正在运行时,需要遍历所有的后台进程.
由于没有改检查复制槽也必须确保逻辑解码后台进程受到保护,因此不会有影响
*/
//取xmin和h->oldest_considered_running中最旧的一个,
//遍历完之后,其实就是所有后台进程中最旧的那个min(pgproc->xmin,pgproc->xid)
+ h->oldest_considered_running =
+ TransactionIdOlder(h->oldest_considered_running, xmin);
+
+ /*
+ * Skip over backends either vacuuming (which is ok with rows being
+ * removed, as long as pg_subtrans is not truncated) or doing logical
+ * decoding (which manages xmin separately, check below).
+ */
/*
跳过vacuum和逻辑解码后台进程
*/
+ if (pgxact->vacuumFlags & (PROC_IN_VACUUM | PROC_IN_LOGICAL_DECODING))
+ continue;
+
+ /* shared tables need to take backends in all database into account */
/*
共享数据表需要考虑所有数据库上的后台进程,取所有进程中xmin最小的那个
实际的比较函数是:TransactionIdPrecedes(h->shared_oldest_nonremovable, xmin)
*/
+ h->shared_oldest_nonremovable =
+ TransactionIdOlder(h->shared_oldest_nonremovable, xmin);
+
+ /*
+ * Normally queries in other databases are ignored for anything but
+ * the shared horizon. But in recovery we cannot compute an accurate
+ * per-database horizon as all xids are managed via the
+ * KnownAssignedXids machinery.
+ */
/*
除了共享的xid边界外,其他数据库中的常规查询会被忽略
但在恢复状态下,还不能精确的计算每个数据库的边界,因为所有xids通过KnownAssignedXids机制来维护
*/
+ if (in_recovery ||
proc->databaseId == MyDatabaseId ||
proc->databaseId == 0) /* always include WalSender */
{
- /* Fetch xid just once - see GetNewTransactionId */
- TransactionId xid = UINT32_ACCESS_ONCE(pgxact->xid);
-
- /* First consider the transaction's own Xid, if any */
- if (TransactionIdIsNormal(xid) &&
- TransactionIdPrecedes(xid, result))
- result = xid;
-
- /*
- * Also consider the transaction's Xmin, if set.
- *
- * We must check both Xid and Xmin because a transaction might
- * have an Xmin but not (yet) an Xid; conversely, if it has an
- * Xid, that could determine some not-yet-set Xmin.
- */
- xid = UINT32_ACCESS_ONCE(pgxact->xmin);
- if (TransactionIdIsNormal(xid) &&
- TransactionIdPrecedes(xid, result))
- result = xid;
//常规数据表,shared_oldest_nonremovable面向全局共享的数据表
+ h->data_oldest_nonremovable =
+ TransactionIdOlder(h->data_oldest_nonremovable, xmin);
}
}
/*
- * Fetch into local variable while ProcArrayLock is held - the
- * LWLockRelease below is a barrier, ensuring this happens inside the
- * lock.
+ * If in recovery fetch oldest xid in KnownAssignedXids, will be applied
+ * after lock is released.
*/
- replication_slot_xmin = procArray->replication_slot_xmin;
- replication_slot_catalog_xmin = procArray->replication_slot_catalog_xmin;
/*
处于恢复状态,提前最旧的xid到kaxmin中,在锁释放后再行应用
KnownAssignedXidsGetOldestXmin的执行逻辑是从tailKnownAssignedXids遍历至headKnownAssignedXids,
通过标志数据KnownAssignedXidsValid找到KnownAssignedXids数组中第一个有效的xid
*/
+ if (in_recovery)
+ kaxmin = KnownAssignedXidsGetOldestXmin();
- if (RecoveryInProgress())
- {
- /*
- * Check to see whether KnownAssignedXids contains an xid value older
- * than the main procarray.
- */
- TransactionId kaxmin = KnownAssignedXidsGetOldestXmin();
-
- LWLockRelease(ProcArrayLock);
+ /*
+ * No other information from shared state is needed, release the lock
+ * immediately. The rest of the computations can be done without a lock.
+ */
+ LWLockRelease(ProcArrayLock);
- if (TransactionIdIsNormal(kaxmin) &&
- TransactionIdPrecedes(kaxmin, result))
- result = kaxmin;
/*
释放锁后,如处于恢复状态,则更新oldest_considered_running/shared_oldest_nonremovable/data_oldest_nonremovable
*/
+ if (in_recovery)
+ {
+ h->oldest_considered_running =
+ TransactionIdOlder(h->oldest_considered_running, kaxmin);
+ h->shared_oldest_nonremovable =
+ TransactionIdOlder(h->shared_oldest_nonremovable, kaxmin);
+ h->data_oldest_nonremovable =
+ TransactionIdOlder(h->data_oldest_nonremovable, kaxmin);
}
else
{
/*
- * No other information needed, so release the lock immediately.
- */
- LWLockRelease(ProcArrayLock);
-
- /*
- * Compute the cutoff XID by subtracting vacuum_defer_cleanup_age,
- * being careful not to generate a "permanent" XID.
+ * Compute the cutoff XID by subtracting vacuum_defer_cleanup_age.
*
* vacuum_defer_cleanup_age provides some additional "slop" for the
* benefit of hot standby queries on standby servers. This is quick
@@ -1466,34 +1651,146 @@ GetOldestXmin(Relation rel, int flags)
* in varsup.c. Also note that we intentionally don't apply
* vacuum_defer_cleanup_age on standby servers.
*/
- result -= vacuum_defer_cleanup_age;
- if (!TransactionIdIsNormal(result))
- result = FirstNormalTransactionId;
/*
正常状态,则根据vacuum_defer_cleanup_age修正oldest_considered_running/shared_oldest_nonremovable/data_oldest_nonremovable
*/
/*
/* return transaction ID backed up by amount, handling wraparound correctly */
static inline TransactionId
TransactionIdRetreatedBy(TransactionId xid, uint32 amount)
{
xid -= amount;
//xid的类型是 : uint32
while (xid < FirstNormalTransactionId)//FirstNormalTransactionId = 3
xid--;
return xid;
}
*/
+ h->oldest_considered_running =
+ TransactionIdRetreatedBy(h->oldest_considered_running,
+ vacuum_defer_cleanup_age);
+ h->shared_oldest_nonremovable =
+ TransactionIdRetreatedBy(h->shared_oldest_nonremovable,
+ vacuum_defer_cleanup_age);
+ h->data_oldest_nonremovable =
+ TransactionIdRetreatedBy(h->data_oldest_nonremovable,
+ vacuum_defer_cleanup_age);
}
/*
* Check whether there are replication slots requiring an older xmin.
*/
- if (!(flags & PROCARRAY_SLOTS_XMIN) &&
- TransactionIdIsValid(replication_slot_xmin) &&
- NormalTransactionIdPrecedes(replication_slot_xmin, result))
- result = replication_slot_xmin;
/*
检查是否存在复制槽需要更旧的xmin
*/
+ h->shared_oldest_nonremovable =
+ TransactionIdOlder(h->shared_oldest_nonremovable, h->slot_xmin);
+ h->data_oldest_nonremovable =
+ TransactionIdOlder(h->data_oldest_nonremovable, h->slot_xmin);
/*
- * After locks have been released and vacuum_defer_cleanup_age has been
- * applied, check whether we need to back up further to make logical
- * decoding possible. We need to do so if we're computing the global limit
- * (rel = NULL) or if the passed relation is a catalog relation of some
- * kind.
+ * The only difference between catalog / data horizons is that the slot's
+ * catalog xmin is applied to the catalog one (so catalogs can be accessed
+ * for logical decoding). Initialize with data horizon, and then back up
+ * further if necessary. Have to back up the shared horizon as well, since
+ * that also can contain catalogs.
*/
- if (!(flags & PROCARRAY_SLOTS_XMIN) &&
- (rel == NULL ||
- RelationIsAccessibleInLogicalDecoding(rel)) &&
- TransactionIdIsValid(replication_slot_catalog_xmin) &&
- NormalTransactionIdPrecedes(replication_slot_catalog_xmin, result))
- result = replication_slot_catalog_xmin;
/*
catalog和data基准线之间的不同是复制槽的xmin会应用到catalog上(因此逻辑解码可以访问catalogs)
使用数据基准线来初始化,然后再需要时进行备份.
同样需要备份共享数据表的基准线(备份到shared_oldest_nonremovable_raw),因为它们会受catalog xmin的影响.
*/
+ h->shared_oldest_nonremovable_raw = h->shared_oldest_nonremovable;//备份shared_oldest_nonremovable
+ h->shared_oldest_nonremovable =
+ TransactionIdOlder(h->shared_oldest_nonremovable,
+ h->slot_catalog_xmin);//取slot_catalog_xmin和原值的较小值
+ h->catalog_oldest_nonremovable = h->data_oldest_nonremovable;//catalog表值初始化为普通表值
+ h->catalog_oldest_nonremovable =
+ TransactionIdOlder(h->catalog_oldest_nonremovable,
+ h->slot_catalog_xmin);//取slot_catalog_xmin和原值的较小值
- return result;
+ /*
+ * It's possible that slots / vacuum_defer_cleanup_age backed up the
+ * horizons further than oldest_considered_running. Fix.
+ */
/*
修正oldest_considered_running,取最旧的那个
*/
+ h->oldest_considered_running =
+ TransactionIdOlder(h->oldest_considered_running,
+ h->shared_oldest_nonremovable);
+ h->oldest_considered_running =
+ TransactionIdOlder(h->oldest_considered_running,
+ h->catalog_oldest_nonremovable);
+ h->oldest_considered_running =
+ TransactionIdOlder(h->oldest_considered_running,
+ h->data_oldest_nonremovable);
+
+ /*
+ * shared horizons have to be at least as old as the oldest visible in
+ * current db
+ */
/*
共享表的基准线在当前db中必须是最小,至少相等
*/
+ Assert(TransactionIdPrecedesOrEquals(h->shared_oldest_nonremovable,
+ h->data_oldest_nonremovable));
+ Assert(TransactionIdPrecedesOrEquals(h->shared_oldest_nonremovable,
+ h->catalog_oldest_nonremovable));
+
+ /*
+ * Horizons need to ensure that pg_subtrans access is still possible for
+ * the relevant backends.
+ */
/*
基准线必须确保pg_subtrans的访问对于相关后台进程来说仍然是可行的
*/
+ Assert(TransactionIdPrecedesOrEquals(h->oldest_considered_running,
+ h->shared_oldest_nonremovable));
+ Assert(TransactionIdPrecedesOrEquals(h->oldest_considered_running,
+ h->catalog_oldest_nonremovable));
+ Assert(TransactionIdPrecedesOrEquals(h->oldest_considered_running,
+ h->data_oldest_nonremovable));
+ Assert(!TransactionIdIsValid(h->slot_xmin) ||
+ TransactionIdPrecedesOrEquals(h->oldest_considered_running,
+ h->slot_xmin));
+ Assert(!TransactionIdIsValid(h->slot_catalog_xmin) ||
+ TransactionIdPrecedesOrEquals(h->oldest_considered_running,
+ h->slot_catalog_xmin));
+
+ /* update approximate horizons with the computed horizons */
/*
使用计算所得的值更新大概的基准线,调用GlobalVisUpdateApply
*/
+ GlobalVisUpdateApply(h);
+}
+
+/*
+ * Return the oldest XID for which deleted tuples must be preserved in the
+ * passed table.
+ *
+ * If rel is not NULL the horizon may be considerably more recent than
+ * otherwise (i.e. fewer tuples will be removable). In the NULL case a horizon
+ * that is correct (but not optimal) for all relations will be returned.
+ *
+ * This is used by VACUUM to decide which deleted tuples must be preserved in
+ * the passed in table.
+ */
/*
GetOldestNonRemovableTransactionId : 返回XID,rel中≥该XID的事务数据必须保留
如rel==NULL,则返回对所有关系通用的基准线
该函数用于VACUUM确定哪些被删除的元组必须保留
*/
+TransactionId
+GetOldestNonRemovableTransactionId(Relation rel)
+{
+ ComputeXidHorizonsResult horizons;
+ //计算基准
+ ComputeXidHorizons(&horizons);
+
+ /* select horizon appropriate for relation */
//rel为NULL或者是全局共享关系,返回shared_oldest_nonremovable
+ if (rel == NULL || rel->rd_rel->relisshared)
+ return horizons.shared_oldest_nonremovable;
+ else if (RelationIsAccessibleInLogicalDecoding(rel))
//逻辑解码,返回catalog_oldest_nonremovable
+ return horizons.catalog_oldest_nonremovable;
+ else
//普通数据表,返回data_oldest_nonremovable
+ return horizons.data_oldest_nonremovable;
//临时表,无需VACUUM清除,自行清除
+}
+
+/*
+ * Return the oldest transaction id any currently running backend might still
+ * consider running. This should not be used for visibility / pruning
+ * determinations (see GetOldestNonRemovableTransactionId()), but for
+ * decisions like up to where pg_subtrans can be truncated.
+ */
+TransactionId
+GetOldestTransactionIdConsideredRunning(void)
+{
+ ComputeXidHorizonsResult horizons;
+
+ ComputeXidHorizons(&horizons);
+
//返回正在运行的最旧的事务ID
+ return horizons.oldest_considered_running;
+}
+
+/*
+ * Return the visibility horizons for a hot standby feedback message.
+ */
+void
+GetReplicationHorizons(TransactionId *xmin, TransactionId *catalog_xmin)
+{
+ ComputeXidHorizonsResult horizons;
+
+ ComputeXidHorizons(&horizons);
+
+ /*
+ * Don't want to use shared_oldest_nonremovable here, as that contains the
+ * effect of replication slot's catalog_xmin. We want to send a separate
+ * feedback for the catalog horizon, so the primary can remove data table
+ * contents more aggressively.
+ */
//返回全局共享基准线和字典表基准线
+ *xmin = horizons.shared_oldest_nonremovable_raw;
+ *catalog_xmin = horizons.slot_catalog_xmin;
}
三、跟踪分析
N/A
四、参考资料
1.
Improving connection scalability: GetSnapshotData()
2.
snapshot scalability: Don’t compute global horizons while building snapshots.