PostgreSQL 源码解读(249)- PG 14(Improving connection scalability)#1

PostgreSQL 14在MVCC上面有所增强,以保证在大批量客户端连接场景下性能不会出现显著下降(详细可参见参考资料中的链接)。本节介绍了提交记录“Don’t compute global horizons while building snapshots.”中heapam.c中的修改以及涉及的相关数据结构。

一、数据结构

GlobalVisState

/*
 * State for the GlobalVisTest* family of functions. Those functions can
 * e.g. be used to decide if a deleted row can be removed without violating
 * MVCC semantics: If the deleted row's xmax is not considered to be running
 * by anyone, the row can be removed.
 * 用于配合GlobalVisTest*函数族使用的数据结构。
 * 这些函数用于确定删除的行在不违反MVCC语义的情况下是否可以被清除:
 * 如果被删除的行的xmax与所有运行的事务无关,那么可以清除。
 * 
 * To avoid slowing down GetSnapshotData(), we don't calculate a precise
 * cutoff XID while building a snapshot (looking at the frequently changing
 * xmins scales badly). Instead we compute two boundaries while building the
 * snapshot:
 *
 * 1) definitely_needed, indicating that rows deleted by XIDs >=
 *    definitely_needed are definitely still visible.
 *
 * 2) maybe_needed, indicating that rows deleted by XIDs < maybe_needed can
 *    definitely be removed
 *  
 *  为了避免影响GetSnapshotData()的性能,在构建快照时我们不会计算精确的cutoff XID,
 *  相反,我们会在构建快照时计算两个边界,分别是:
 *  1)definitely_needed,表示如果删除事务的XIDs >= definitely_needed,则可见
 *  2)maybe_needed,表示被XIDs < maybe_needed 事务删除的,可以确定被清除
 *
 * When testing an XID that falls in between the two (i.e. XID >= maybe_needed
 * && XID < definitely_needed), the boundaries can be recomputed (using
 * ComputeXidHorizons()) to get a more accurate answer. This is cheaper than
 * maintaining an accurate value all the time.
 * 对于XID >= maybe_needed && XID < definitely_needed的XID,
 * 边界需要使用函数ComputeXidHorizons重新计算来获取更精确的结果,这比在所有情况下都维护一个精确值成本要低
 *
 * As it is not cheap to compute accurate boundaries, we limit the number of
 * times that happens in short succession. See GlobalVisTestShouldUpdate().
 * 计算精确的边界成本高昂,因此需要限制短时间内出现的次数,详细参见GlobalVisTestShouldUpdate函数
 *
 * There are three backend lifetime instances of this struct, optimized for
 * different types of relations. As e.g. a normal user defined table in one
 * database is inaccessible to backends connected to another database, a test
 * specific to a relation can be more aggressive than a test for a shared
 * relation.  Currently we track four different states:
 * 对于不同的关系,出于优化的目的,该结构有三种不同的backend生命周期。
 * 比如某个数据库中用户定义的普通表对于连接到另外一个数据库的backends是不可见的,
 * 对某个特定关系的验证比起共享关系的验证要更为积极。我们跟踪四种不同的状态:
 *
 * 1) GlobalVisSharedRels, which only considers an XID's
 *    effects visible-to-everyone if neither snapshots in any database, nor a
 *    replication slot's xmin, nor a replication slot's catalog_xmin might
 *    still consider XID as running.
 * 1)GlobalVisSharedRels,事务对该Relation的操作会影响所有数据库中的快照、复制槽的xmin和复制槽的catalog_xmin。
 *
 * 2) GlobalVisCatalogRels, which only considers an XID's
 *    effects visible-to-everyone if neither snapshots in the current
 *    database, nor a replication slot's xmin, nor a replication slot's
 *    catalog_xmin might still consider XID as running.
 *
 *    I.e. the difference to GlobalVisSharedRels is that
 *    snapshot in other databases are ignored.
 * 2)GlobalVisCatalogRels,事务对该Relation的操作会影响当前数据库中的快照、复制槽的xmin和复制槽的catalog_xmin
 *    与GlobalVisSharedRels不同之处是其他数据库的快照会被忽略
 *
 * 3) GlobalVisDataRels, which only considers an XID's
 *    effects visible-to-everyone if neither snapshots in the current
 *    database, nor a replication slot's xmin consider XID as running.
 *
 *    I.e. the difference to GlobalVisCatalogRels is that
 *    replication slot's catalog_xmin is not taken into account.
 *
 * 3)GlobalVisDataRels,事务对该Relation的操作会影响当前数据库中的快照、复制槽的xmin
 *    与GlobalVisCatalogRels不同的是catalog_xmin不作考虑
 *
 * 4) GlobalVisTempRels, which only considers the current session, as temp
 *    tables are not visible to other sessions.
 *
 * 4)GlobalVisTempRels,只考虑当前会话,临时表对其他session不可见
 *
 * GlobalVisTestFor(relation) returns the appropriate state
 * for the relation.
 * GlobalVisTestFor(relation)返回关系正确的状态
 *
 * The boundaries are FullTransactionIds instead of TransactionIds to avoid
 * wraparound dangers. There e.g. would otherwise exist no procarray state to
 * prevent maybe_needed to become old enough after the GetSnapshotData()
 * call.
 * 边界使用FullTransactionIds而不是TransactionIds以避免因为回卷而导致的错误。
 * 例如,在GetSnapshotData()调用后,将不存在procarray状态来防止maybe_needed变得32bit XID中存在的过老现象。
 *
 * The typedef is in the header.
 */
struct GlobalVisState
{
    /* XIDs >= are considered running by some backend */
    //大于等于该值的XIDs:其他活动backend可见
    FullTransactionId definitely_needed;
    /* XIDs < are not considered to be running by any backend */
    //小于该值的XIDs:活动backend不可见
    FullTransactionId maybe_needed;
};
/*
 * A 64 bit value that contains an epoch and a TransactionId.  This is
 * wrapped in a struct to prevent implicit conversion to/from TransactionId.
 * Not all values represent valid normal XIDs.
 * 号外:PG 14已支持64 bit XID,令人厌烦的XID回卷问题不复存在
 */
typedef struct FullTransactionId
{
    uint64      value;
} FullTransactionId;

二、源码解读

diff heapam.c
修改heapam.c,目的是去掉RecentGlobalXmin的访问,改为通过GlobalVisXXX配合GlobalVisState Struct实现类似的功能

--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -1517,6 +1517,7 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer,
    bool        at_chain_start;
    bool        valid;
    bool        skip;
+   GlobalVisState *vistest = NULL;//新引入的数据结构,全局访问状态,详见GlobalVisState Struct
    /* If this is not the first call, previous call returned a (live!) tuple */
    if (all_dead)
@@ -1527,7 +1528,8 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer,
    at_chain_start = first_call;
    skip = !first_call;
-   Assert(TransactionIdIsValid(RecentGlobalXmin));
+   /* XXX: we should assert that a snapshot is pushed or registered */
+   Assert(TransactionIdIsValid(RecentXmin));//去掉全局变量RecentGlobalXmin的访问,转而确认快照入栈和注册
    Assert(BufferGetBlockNumber(buffer) == blkno);
    /* Scan through possible multiple members of HOT-chain */
@@ -1616,9 +1618,14 @@ heap_hot_search_buffer(ItemPointer tid, Relation relation, Buffer buffer,
         * Note: if you change the criterion here for what is "dead", fix the
         * planner's get_actual_variable_range() function to match.
         */
-       if (all_dead && *all_dead &&
-           !HeapTupleIsSurelyDead(heapTuple, RecentGlobalXmin))
-           *all_dead = false;
+       if (all_dead && *all_dead)//去掉与RecentGlobalXmin相关的代码,改为从全局访问状态结构体中获取信息
+       {
+           if (!vistest)
+               vistest = GlobalVisTestFor(relation);//在后续介绍
+
+           if (!HeapTupleIsSurelyDead(heapTuple, vistest))
+               *all_dead = false;
+       }
        /*
         * Check to see if HOT chain continues past this tuple; if so fetch

GlobalVisTestFor

/*
 * If rel != NULL, return test state appropriate for relation, otherwise
 * return state usable for all relations.  The latter may consider XIDs as
 * not-yet-visible-to-everyone that a state for a specific relation would
 * already consider visible-to-everyone.
 * 如果rel有效(不为NULL),返回对应该关系的验证状态,否则返回对所有关系均可用的状态。
 * 后续可能会认为XIDs为对所有仍未可见的状态,该状态对于某个指定的关系可能已经认为对所有事务可见。
 *
 * This needs to be called while a snapshot is active or registered, otherwise
 * there are wraparound and other dangers.
 * 该例程在快照活动或已注册时调用,否则存在回卷或其他风险。
 *
 * See comment for GlobalVisState for details.
 * 详细可参见GlobalVisState
 */
GlobalVisState *
GlobalVisTestFor(Relation rel)
{
    bool        need_shared;//共享?
    bool        need_catalog;//数据字典?
    GlobalVisState *state;//访问状态
    /* XXX: we should assert that a snapshot is pushed or registered */
    //快照已入栈或注册
    Assert(RecentXmin);
    if (!rel)
        need_shared = need_catalog = true;//rel为NULL,则均值为true
    else
    {
        /*
         * Other kinds currently don't contain xids, nor always the necessary
         * logical decoding markers.
         * 还没有包含xids的其他类型,或者通常是逻辑解码标记器
         */
        Assert(rel->rd_rel->relkind == RELKIND_RELATION ||
               rel->rd_rel->relkind == RELKIND_MATVIEW ||
               rel->rd_rel->relkind == RELKIND_TOASTVALUE);
        need_shared = rel->rd_rel->relisshared || RecoveryInProgress();//共享的关系(跨数据库)或者正处于恢复过程
        need_catalog = IsCatalogRelation(rel) || RelationIsAccessibleInLogicalDecoding(rel);//数据字典表或者逻辑解码可访问的关系
    }
    if (need_shared)
        state = &GlobalVisSharedRels;//全局共享
    else if (need_catalog)
        state = &GlobalVisCatalogRels;//当前数据库中的Catalog
    else if (RELATION_IS_LOCAL(rel))
        state = &GlobalVisTempRels;//临时表
    else
        state = &GlobalVisDataRels;//常规表
    Assert(FullTransactionIdIsValid(state->definitely_needed) &&
           FullTransactionIdIsValid(state->maybe_needed));
    return state;
}


三、跟踪分析

N/A

四、参考资料

1. Improving connection scalability: GetSnapshotData()
2. snapshot scalability: Don’t compute global horizons while building snapshots.

请使用浏览器的分享功能分享到微信等