PostgreSQL 14在MVCC上面有所增强,以保证在大批量客户端连接场景下性能不会出现显著下降(详细可参见参考资料中的链接)。本节介绍了提交记录“Don’t compute global horizons while building snapshots.”中heapam.c中的修改以及涉及的相关数据结构。
一、数据结构
GlobalVisState
/*
* State for the GlobalVisTest* family of functions. Those functions can
* e.g. be used to decide if a deleted row can be removed without violating
* MVCC semantics: If the deleted row's xmax is not considered to be running
* by anyone, the row can be removed.
* 用于配合GlobalVisTest*函数族使用的数据结构。
* 这些函数用于确定删除的行在不违反MVCC语义的情况下是否可以被清除:
* 如果被删除的行的xmax与所有运行的事务无关,那么可以清除。
*
* To avoid slowing down GetSnapshotData(), we don't calculate a precise
* cutoff XID while building a snapshot (looking at the frequently changing
* xmins scales badly). Instead we compute two boundaries while building the
* snapshot:
*
* 1) definitely_needed, indicating that rows deleted by XIDs >=
* definitely_needed are definitely still visible.
*
* 2) maybe_needed, indicating that rows deleted by XIDs < maybe_needed can
* definitely be removed
*
* 为了避免影响GetSnapshotData()的性能,在构建快照时我们不会计算精确的cutoff XID,
* 相反,我们会在构建快照时计算两个边界,分别是:
* 1)definitely_needed,表示如果删除事务的XIDs >= definitely_needed,则可见
* 2)maybe_needed,表示被XIDs < maybe_needed 事务删除的,可以确定被清除
*
* When testing an XID that falls in between the two (i.e. XID >= maybe_needed
* && XID < definitely_needed), the boundaries can be recomputed (using
* ComputeXidHorizons()) to get a more accurate answer. This is cheaper than
* maintaining an accurate value all the time.
* 对于XID >= maybe_needed && XID < definitely_needed的XID,
* 边界需要使用函数ComputeXidHorizons重新计算来获取更精确的结果,这比在所有情况下都维护一个精确值成本要低
*
* As it is not cheap to compute accurate boundaries, we limit the number of
* times that happens in short succession. See GlobalVisTestShouldUpdate().
* 计算精确的边界成本高昂,因此需要限制短时间内出现的次数,详细参见GlobalVisTestShouldUpdate函数
*
* There are three backend lifetime instances of this struct, optimized for
* different types of relations. As e.g. a normal user defined table in one
* database is inaccessible to backends connected to another database, a test
* specific to a relation can be more aggressive than a test for a shared
* relation. Currently we track four different states:
* 对于不同的关系,出于优化的目的,该结构有三种不同的backend生命周期。
* 比如某个数据库中用户定义的普通表对于连接到另外一个数据库的backends是不可见的,
* 对某个特定关系的验证比起共享关系的验证要更为积极。我们跟踪四种不同的状态:
*
* 1) GlobalVisSharedRels, which only considers an XID's
* effects visible-to-everyone if neither snapshots in any database, nor a
* replication slot's xmin, nor a replication slot's catalog_xmin might
* still consider XID as running.
* 1)GlobalVisSharedRels,事务对该Relation的操作会影响所有数据库中的快照、复制槽的xmin和复制槽的catalog_xmin。
*
* 2) GlobalVisCatalogRels, which only considers an XID's
* effects visible-to-everyone if neither snapshots in the current
* database, nor a replication slot's xmin, nor a replication slot's
* catalog_xmin might still consider XID as running.
*
* I.e. the difference to GlobalVisSharedRels is that
* snapshot in other databases are ignored.
* 2)GlobalVisCatalogRels,事务对该Relation的操作会影响当前数据库中的快照、复制槽的xmin和复制槽的catalog_xmin
* 与GlobalVisSharedRels不同之处是其他数据库的快照会被忽略
*
* 3) GlobalVisDataRels, which only considers an XID's
* effects visible-to-everyone if neither snapshots in the current
* database, nor a replication slot's xmin consider XID as running.
*
* I.e. the difference to GlobalVisCatalogRels is that
* replication slot's catalog_xmin is not taken into account.
*
* 3)GlobalVisDataRels,事务对该Relation的操作会影响当前数据库中的快照、复制槽的xmin
* 与GlobalVisCatalogRels不同的是catalog_xmin不作考虑
*
* 4) GlobalVisTempRels, which only considers the current session, as temp
* tables are not visible to other sessions.
*
* 4)GlobalVisTempRels,只考虑当前会话,临时表对其他session不可见
*
* GlobalVisTestFor(relation) returns the appropriate state
* for the relation.
* GlobalVisTestFor(relation)返回关系正确的状态
*
* The boundaries are FullTransactionIds instead of TransactionIds to avoid
* wraparound dangers. There e.g. would otherwise exist no procarray state to
* prevent maybe_needed to become old enough after the GetSnapshotData()
* call.
* 边界使用FullTransactionIds而不是TransactionIds以避免因为回卷而导致的错误。
* 例如,在GetSnapshotData()调用后,将不存在procarray状态来防止maybe_needed变得32bit XID中存在的过老现象。
*
* The typedef is in the header.
*/
struct GlobalVisState
{
/* XIDs >= are considered running by some backend */
//大于等于该值的XIDs:其他活动backend可见
FullTransactionId definitely_needed;
/* XIDs < are not considered to be running by any backend */
//小于该值的XIDs:活动backend不可见
FullTransactionId maybe_needed;
};
/*
* A 64 bit value that contains an epoch and a TransactionId. This is
* wrapped in a struct to prevent implicit conversion to/from TransactionId.
* Not all values represent valid normal XIDs.
* 号外:PG 14已支持64 bit XID,令人厌烦的XID回卷问题不复存在
*/
typedef struct FullTransactionId
{
uint64 value;
} FullTransactionId;
二、源码解读
diff heapam.c
修改heapam.c,目的是去掉RecentGlobalXmin的访问,改为通过GlobalVisXXX配合GlobalVisState Struct实现类似的功能
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -1517,6 +1517,7 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer,
bool at_chain_start;
bool valid;
bool skip;
+ GlobalVisState *vistest = NULL;//新引入的数据结构,全局访问状态,详见GlobalVisState Struct
/* If this is not the first call, previous call returned a (live!) tuple */
if (all_dead)
@@ -1527,7 +1528,8 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer,
at_chain_start = first_call;
skip = !first_call;
- Assert(TransactionIdIsValid(RecentGlobalXmin));
+ /* XXX: we should assert that a snapshot is pushed or registered */
+ Assert(TransactionIdIsValid(RecentXmin));//去掉全局变量RecentGlobalXmin的访问,转而确认快照入栈和注册
Assert(BufferGetBlockNumber(buffer) == blkno);
/* Scan through possible multiple members of HOT-chain */
@@ -1616,9 +1618,14 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer,
* Note: if you change the criterion here for what is "dead", fix the
* planner's get_actual_variable_range() function to match.
*/
- if (all_dead && *all_dead &&
- !HeapTupleIsSurelyDead(heapTuple, RecentGlobalXmin))
- *all_dead = false;
+ if (all_dead && *all_dead)//去掉与RecentGlobalXmin相关的代码,改为从全局访问状态结构体中获取信息
+ {
+ if (!vistest)
+ vistest = GlobalVisTestFor(relation);//在后续介绍
+
+ if (!HeapTupleIsSurelyDead(heapTuple, vistest))
+ *all_dead = false;
+ }
/*
* Check to see if HOT chain continues past this tuple; if so fetch
GlobalVisTestFor
/*
* If rel != NULL, return test state appropriate for relation, otherwise
* return state usable for all relations. The latter may consider XIDs as
* not-yet-visible-to-everyone that a state for a specific relation would
* already consider visible-to-everyone.
* 如果rel有效(不为NULL),返回对应该关系的验证状态,否则返回对所有关系均可用的状态。
* 后续可能会认为XIDs为对所有仍未可见的状态,该状态对于某个指定的关系可能已经认为对所有事务可见。
*
* This needs to be called while a snapshot is active or registered, otherwise
* there are wraparound and other dangers.
* 该例程在快照活动或已注册时调用,否则存在回卷或其他风险。
*
* See comment for GlobalVisState for details.
* 详细可参见GlobalVisState
*/
GlobalVisState *
GlobalVisTestFor(Relation rel)
{
bool need_shared;//共享?
bool need_catalog;//数据字典?
GlobalVisState *state;//访问状态
/* XXX: we should assert that a snapshot is pushed or registered */
//快照已入栈或注册
Assert(RecentXmin);
if (!rel)
need_shared = need_catalog = true;//rel为NULL,则均值为true
else
{
/*
* Other kinds currently don't contain xids, nor always the necessary
* logical decoding markers.
* 还没有包含xids的其他类型,或者通常是逻辑解码标记器
*/
Assert(rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_MATVIEW ||
rel->rd_rel->relkind == RELKIND_TOASTVALUE);
need_shared = rel->rd_rel->relisshared || RecoveryInProgress();//共享的关系(跨数据库)或者正处于恢复过程
need_catalog = IsCatalogRelation(rel) || RelationIsAccessibleInLogicalDecoding(rel);//数据字典表或者逻辑解码可访问的关系
}
if (need_shared)
state = &GlobalVisSharedRels;//全局共享
else if (need_catalog)
state = &GlobalVisCatalogRels;//当前数据库中的Catalog
else if (RELATION_IS_LOCAL(rel))
state = &GlobalVisTempRels;//临时表
else
state = &GlobalVisDataRels;//常规表
Assert(FullTransactionIdIsValid(state->definitely_needed) &&
FullTransactionIdIsValid(state->maybe_needed));
return state;
}
三、跟踪分析
N/A
四、参考资料
1.
Improving connection scalability: GetSnapshotData()
2.
snapshot scalability: Don’t compute global horizons while building snapshots.