PostgreSQL 14在MVCC上面有所增强，以保证在大批量客户端连接场景下性能不会出现显著下降（详细可参见参考资料中的链接）。本节介绍了提交记录“snapshot scalability: Move subxact info to ProcGlobal, remove PGXACT.”中的修改以及涉及的相关数据结构。
一、数据结构
PROC_HDR
/*
* There is one ProcGlobal struct for the whole database cluster.
* 全局的ProcGlobal(PROC_HDR)结构体
*
* Adding/Removing an entry into the procarray requires holding *both*
* ProcArrayLock and XidGenLock in exclusive mode (in that order). Both are
* needed because the dense arrays (see below) are accessed from
* GetNewTransactionId() and GetSnapshotData(), and we don't want to add
* further contention by both using the same lock. Adding/Removing a procarray
* entry is much less frequent.
* 对proc数组的插入和删除,需要以独占的模式持有ProcArrayLock和XidGenLock,
* 原因是GetNewTransactionId()和GetSnapshotData()函数需要访问,并且我们不希望使用同样的锁来增加争用.
* 其实,插入和删除proc数组并不频繁.
*
* Some fields in PGPROC are mirrored into more densely packed arrays (e.g.
* xids), with one entry for each backend. These arrays only contain entries
* for PGPROCs that have been added to the shared array with ProcArrayAdd()
* (in contrast to PGPROC array which has unused PGPROCs interspersed).
* PGPROC中的一些字段镜像到更稠密的数组中(如xids),每一个后台进程都有一项.
* 这些数组只包含使用ProcArrayAdd()加入到共享数组的PGPROCs.
*
* The dense arrays are indexed by PGPROC->pgxactoff. Any concurrent
* ProcArrayAdd() / ProcArrayRemove() can lead to pgxactoff of a procarray
* member to change. Therefore it is only safe to use PGPROC->pgxactoff to
* access the dense array while holding either ProcArrayLock or XidGenLock.
* 稠密数组通过PGPROC->pgxactoff字段进行索引,所有同步ProcArrayAdd和ProcArrayRemove操作
* 会改变procarray中的pgxactoff成员变量.
*
* As long as a PGPROC is in the procarray, the mirrored values need to be
* maintained in both places in a coherent manner.
* 由于PGPROC在proc数组中,镜像值需要以一致的方式进行运维.
*
* The denser separate arrays are beneficial for three main reasons: First, to
* allow for as tight loops accessing the data as possible. Second, to prevent
* updates of frequently changing data (e.g. xmin) from invalidating
* cachelines also containing less frequently changing data (e.g. xid,
* statusFlags). Third to condense frequently accessed data into as few
* cachelines as possible.
* 基于以下3点原因,更稠密单独的数组会带来不少好处.
* 1.紧密的循环访问数据成为可能;
* 2.避免了对修改频度不高的数据进行修改导致CPU缓存无效的情况出现;
* 3.使频繁访问的数据放在小量的CPU缓存中成为可能.
*
* There are two main reasons to have the data mirrored between these dense
* arrays and PGPROC. First, as explained above, a PGPROC's array entries can
* only be accessed with either ProcArrayLock or XidGenLock held, whereas the
* PGPROC entries do not require that (obviously there may still be locking
* requirements around the individual field, separate from the concerns
* here). That is particularly important for a backend to efficiently checks
* it own values, which it often can safely do without locking. Second, the
* PGPROC fields allow to avoid unnecessary accesses and modification to the
* dense arrays. A backend's own PGPROC is more likely to be in a local cache,
* whereas the cachelines for the dense array will be modified by other
* backends (often removing it from the cache for other cores/sockets). At
* commit/abort time a check of the PGPROC value can avoid accessing/dirtying
* the corresponding array value.
* 让这些稠密数组和PGPROC之间生成数据镜像,有两个主要原因:
* 1.如上所述,PGPROC数组条目需要在持有ProcArrayLock和XidGenLock的前提下才能访问,但实际上并不需要.
* 在没有锁的情况下,后台进程有效且安全的检查自己的值是十分重要的.
* 2.PGPROC中的字段避免不必要的访问和修改.后台进程自己的PGPROC更应该在自己的本地缓存中,
* 而无需理会数组会被其他进程修改.在事务提交或回滚时,检查PGPROC值可以避免访问或者弄脏相应的数组值
*
* Basically it makes sense to access the PGPROC variable when checking a
* single backend's data, especially when already looking at the PGPROC for
* other reasons already. It makes sense to look at the "dense" arrays if we
* need to look at many / most entries, because we then benefit from the
* reduced indirection and better cross-process cache-ability.
* 检查单个后台进程数据时,访问PGPROC值是比较敏感的,特别是在已经因为其他原因检索过PGPROC的情况下
*
* When entering a PGPROC for 2PC transactions with ProcArrayAdd(), the data
* in the dense arrays is initialized from the PGPROC while it already holds
* ProcArrayLock.
* 在2PC事务的情况下,通过ProcArrayAdd方法进入PGPROC时,
* 数组中的数据在持有ProcArrayLock锁的情况下初始化数组.
*/
typedef struct PROC_HDR
{
/* Array of PGPROC structures (not including dummies for prepared txns) */
//PGPROC结构体数组
PGPROC *allProcs;
/* Array mirroring PGPROC.xid for each PGPROC currently in the procarray */
// 镜像每一个PGPROC.xid
TransactionId *xids;
/*
* Array mirroring PGPROC.subxidStatus for each PGPROC currently in the
* procarray.
*/
//镜像PGPROC.subxidStatus
XidCacheStatus *subxidStates;
/*
* Array mirroring PGPROC.statusFlags for each PGPROC currently in the
* procarray.
*/
//镜像PGPROC.statusFlags
uint8 *statusFlags;
/* Length of allProcs array */
//allProcs数组的大小
uint32 allProcCount;
/* Head of list of free PGPROC structures */
//空闲PGPROC结构体链表头
PGPROC *freeProcs;
/* Head of list of autovacuum's free PGPROC structures */
//autovacuum进程空闲PGPROC结构体链表头
PGPROC *autovacFreeProcs;
/* Head of list of bgworker free PGPROC structures */
//bgworker进程
PGPROC *bgworkerFreeProcs;
/* Head of list of walsender free PGPROC structures */
//walsender
PGPROC *walsenderFreeProcs;
/* First pgproc waiting for group XID clear */
//等待XID组清理的第一个pgproc
pg_atomic_uint32 procArrayGroupFirst;
/* First pgproc waiting for group transaction status update */
//等待事务状态组更新的首个pgproc
pg_atomic_uint32 clogGroupFirst;
/* WALWriter process's latch */
//WALWriter进程latch
Latch *walwriterLatch;
/* Checkpointer process's latch */
//检查点进程latch
Latch *checkpointerLatch;
/* Current shared estimate of appropriate spins_per_delay value */
//spins_per_delay
int spins_per_delay;
/* The proc of the Startup process, since not in ProcArray */
//Startup进程的proc结构体,不再Proc数组中
PGPROC *startupProc;
int startupProcPid;
/* Buffer id of the buffer that Startup process waits for pin on, or -1 */
//Startup进程等待pin的缓存ID
int startupBufferPinWaitBufId;
} PROC_HDR;
extern PGDLLIMPORT PROC_HDR *ProcGlobal;
二、源码解读
主要修改 : 去掉MyPgXact结构体,改用MyProc
/*
去掉MyPgXact结构体,改用MyProc
*/
diff --git a/src/backend/access/transam/clog.c b/src/backend/access/transam/clog.c
index a4599e966106a4d9a0442924fbe8e2a8659b7992..65aa8841f7ce0175d4b45986690f3d4f90700a89 100644 (file)
--- a/src/backend/access/transam/clog.c
+++ b/src/backend/access/transam/clog.c
@@ -295,7 +295,7 @@ TransactionIdSetPageStatus(TransactionId xid, int nsubxids,
*/
if (all_xact_same_page && xid == MyProc->xid &&
nsubxids <= THRESHOLD_SUBTRANS_CLOG_OPT &&
- nsubxids == MyPgXact->nxids &&
+ nsubxids == MyProc->subxidStatus.count &&
memcmp(subxids, MyProc->subxids.xids,
nsubxids * sizeof(TransactionId)) == 0)
{
@@ -510,16 +510,15 @@ TransactionGroupUpdateXidStatus(TransactionId xid, XidStatus status,
while (nextidx != INVALID_PGPROCNO)
{
PGPROC *proc = &ProcGlobal->allProcs[nextidx];
- PGXACT *pgxact = &ProcGlobal->allPgXact[nextidx];
/*
* Transactions with more than THRESHOLD_SUBTRANS_CLOG_OPT sub-XIDs
* should not use group XID status update mechanism.
*/
- Assert(pgxact->nxids <= THRESHOLD_SUBTRANS_CLOG_OPT);
+ Assert(proc->subxidStatus.count <= THRESHOLD_SUBTRANS_CLOG_OPT);
TransactionIdSetPageStatusInternal(proc->clogGroupMemberXid,
- pgxact->nxids,
+ proc->subxidStatus.count,
proc->subxids.xids,
proc->clogGroupMemberXidStatus,
proc->clogGroupMemberLsn,
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 744b8a7f39352bce409a8f08d71748e84e4b15e4..ef4f9981e359f7eac2d581edf51d08bca565e8af 100644 (file)
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -21,9 +21,9 @@
* GIDs and aborts the transaction if there already is a global
* transaction in prepared state with the same GID.
*
- * A global transaction (gxact) also has dummy PGXACT and PGPROC; this is
- * what keeps the XID considered running by TransactionIdIsInProgress.
- * It is also convenient as a PGPROC to hook the gxact's locks to.
+ * A global transaction (gxact) also has dummy PGPROC; this is what keeps
+ * the XID considered running by TransactionIdIsInProgress. It is also
+ * convenient as a PGPROC to hook the gxact's locks to.
*
* Information to recover prepared transactions in case of crash is
* now stored in WAL for the common case. In some cases there will be
@@ -447,14 +447,12 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
TimestampTz prepared_at, Oid owner, Oid databaseid)
{
PGPROC *proc;
- PGXACT *pgxact;
int i;
Assert(LWLockHeldByMeInMode(TwoPhaseStateLock, LW_EXCLUSIVE));
Assert(gxact != NULL);
proc = &ProcGlobal->allProcs[gxact->pgprocno];
- pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
/* Initialize the PGPROC entry */
MemSet(proc, 0, sizeof(PGPROC));
@@ -480,8 +478,8 @@ MarkAsPreparingGuts(GlobalTransaction gxact, TransactionId xid, const char *gid,
for (i = 0; i < NUM_LOCK_PARTITIONS; i++)
SHMQueueInit(&(proc->myProcLocks[i]));
/* subxid data must be filled later by GXactLoadSubxactData */
- pgxact->overflowed = false;
- pgxact->nxids = 0;
+ proc->subxidStatus.overflowed = false;
+ proc->subxidStatus.count = 0;
gxact->prepared_at = prepared_at;
gxact->xid = xid;
@@ -510,19 +508,18 @@ GXactLoadSubxactData(GlobalTransaction gxact, int nsubxacts,
TransactionId *children)
{
PGPROC *proc = &ProcGlobal->allProcs[gxact->pgprocno];
- PGXACT *pgxact = &ProcGlobal->allPgXact[gxact->pgprocno];
/* We need no extra lock since the GXACT isn't valid yet */
if (nsubxacts > PGPROC_MAX_CACHED_SUBXIDS)
{
- pgxact->overflowed = true;
+ proc->subxidStatus.overflowed = true;
nsubxacts = PGPROC_MAX_CACHED_SUBXIDS;
}
if (nsubxacts > 0)
{
memcpy(proc->subxids.xids, children,
nsubxacts * sizeof(TransactionId));
- pgxact->nxids = nsubxacts;
+ proc->subxidStatus.count = nsubxacts;
}
}
diff --git a/src/backend/access/transam/varsup.c b/src/backend/access/transam/varsup.c
index 4c91b343ecd2251b8b9c7d8797b48242e492764a..2d2b05be36c47f7504dacf42766c0e4ebe7f196f 100644 (file)
--- a/src/backend/access/transam/varsup.c
+++ b/src/backend/access/transam/varsup.c
@@ -222,22 +222,31 @@ GetNewTransactionId(bool isSubXact)
*/
if (!isSubXact)
{
+ Assert(ProcGlobal->subxidStates[MyProc->pgxactoff].count == 0);
+ Assert(!ProcGlobal->subxidStates[MyProc->pgxactoff].overflowed);
+ Assert(MyProc->subxidStatus.count == 0);
+ Assert(!MyProc->subxidStatus.overflowed);
+
/* LWLockRelease acts as barrier */
MyProc->xid = xid;
ProcGlobal->xids[MyProc->pgxactoff] = xid;
}
else
{
- int nxids = MyPgXact->nxids;
+ XidCacheStatus *substat = &ProcGlobal->subxidStates[MyProc->pgxactoff];
+ int nxids = MyProc->subxidStatus.count;
+
+ Assert(substat->count == MyProc->subxidStatus.count);
+ Assert(substat->overflowed == MyProc->subxidStatus.overflowed);
if (nxids < PGPROC_MAX_CACHED_SUBXIDS)
{
MyProc->subxids.xids[nxids] = xid;
pg_write_barrier();
- MyPgXact->nxids = nxids + 1;
+ MyProc->subxidStatus.count = substat->count = nxids + 1;
}
else
- MyPgXact->overflowed = true;
+ MyProc->subxidStatus.overflowed = substat->overflowed = true;
}
LWLockRelease(XidGenLock);
diff --git a/src/backend/storage/ipc/procarray.c b/src/backend/storage/ipc/procarray.c
index 224da4f9510b8985317276d0fc611a96872d9c39..8262abd42e6bd2a95dd13ef44107878dcaf53441 100644 (file)
--- a/src/backend/storage/ipc/procarray.c
+++ b/src/backend/storage/ipc/procarray.c
@@ -4,9 +4,10 @@
* POSTGRES process array code.
*
*
- * This module maintains arrays of the PGPROC and PGXACT structures for all
- * active backends. Although there are several uses for this, the principal
- * one is as a means of determining the set of currently running transactions.
+ * This module maintains arrays of PGPROC substructures, as well as associated
+ * arrays in ProcGlobal, for all active backends. Although there are several
+ * uses for this, the principal one is as a means of determining the set of
+ * currently running transactions.
*
* Because of various subtle race conditions it is critical that a backend
* hold the correct locks while setting or clearing its xid (in
@@ -85,7 +86,7 @@ typedef struct ProcArrayStruct
/*
* Highest subxid that has been removed from KnownAssignedXids array to
* prevent overflow; or InvalidTransactionId if none. We track this for
- * similar reasons to tracking overflowing cached subxids in PGXACT
+ * similar reasons to tracking overflowing cached subxids in PGPROC
* entries. Must hold exclusive ProcArrayLock to change this, and shared
* lock to read it.
*/
@@ -96,7 +97,7 @@ typedef struct ProcArrayStruct
/* oldest catalog xmin of any replication slot */
TransactionId replication_slot_catalog_xmin;
- /* indexes into allPgXact[], has PROCARRAY_MAXPROCS entries */
+ /* indexes into allProcs[], has PROCARRAY_MAXPROCS entries */
int pgprocnos[FLEXIBLE_ARRAY_MEMBER];
} ProcArrayStruct;
@@ -239,7 +240,6 @@ typedef struct ComputeXidHorizonsResult
static ProcArrayStruct *procArray;
static PGPROC *allProcs;
-static PGXACT *allPgXact;
/*
* Bookkeeping for tracking emulated transactions in recovery
@@ -325,8 +325,7 @@ static int KnownAssignedXidsGetAndSetXmin(TransactionId *xarray,
static TransactionId KnownAssignedXidsGetOldestXmin(void);
static void KnownAssignedXidsDisplay(int trace_level);
static void KnownAssignedXidsReset(void);
-static inline void ProcArrayEndTransactionInternal(PGPROC *proc,
- PGXACT *pgxact, TransactionId latestXid);
+static inline void ProcArrayEndTransactionInternal(PGPROC *proc, TransactionId latestXid);
static void ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid);
static void MaintainLatestCompletedXid(TransactionId latestXid);
static void MaintainLatestCompletedXidRecovery(TransactionId latestXid);
@@ -411,7 +410,6 @@ CreateSharedProcArray(void)
}
allProcs = ProcGlobal->allProcs;
- allPgXact = ProcGlobal->allPgXact;
/* Create or attach to the KnownAssignedXids arrays too, if needed */
if (EnableHotStandby)
@@ -476,11 +474,14 @@ ProcArrayAdd(PGPROC *proc)
/*
保持procs数组的有序性
腾出index位置,用例存放proc的相关信息
*/
(arrayP->numProcs - index) * sizeof(*arrayP->pgprocnos));
memmove(&ProcGlobal->xids[index + 1], &ProcGlobal->xids[index],
(arrayP->numProcs - index) * sizeof(*ProcGlobal->xids));
+ memmove(&ProcGlobal->subxidStates[index + 1], &ProcGlobal->subxidStates[index],
+ (arrayP->numProcs - index) * sizeof(*ProcGlobal->subxidStates));
memmove(&ProcGlobal->vacuumFlags[index + 1], &ProcGlobal->vacuumFlags[index],
(arrayP->numProcs - index) * sizeof(*ProcGlobal->vacuumFlags));
arrayP->pgprocnos[index] = proc->pgprocno;
ProcGlobal->xids[index] = proc->xid;
+ ProcGlobal->subxidStates[index] = proc->subxidStatus;
ProcGlobal->vacuumFlags[index] = proc->vacuumFlags;
arrayP->numProcs++;
@@ -534,6 +535,8 @@ ProcArrayRemove(PGPROC *proc, TransactionId latestXid)
MaintainLatestCompletedXid(latestXid);
ProcGlobal->xids[proc->pgxactoff] = 0;
+ ProcGlobal->subxidStates[proc->pgxactoff].overflowed = false;
+ ProcGlobal->subxidStates[proc->pgxactoff].count = 0;
}
else
{
@@ -542,6 +545,8 @@ ProcArrayRemove(PGPROC *proc, TransactionId latestXid)
}
Assert(TransactionIdIsValid(ProcGlobal->xids[proc->pgxactoff] == 0));
+ Assert(TransactionIdIsValid(ProcGlobal->subxidStates[proc->pgxactoff].count == 0));
+ Assert(TransactionIdIsValid(ProcGlobal->subxidStates[proc->pgxactoff].overflowed == false));
ProcGlobal->vacuumFlags[proc->pgxactoff] = 0;
for (index = 0; index < arrayP->numProcs; index++)
@@ -553,6 +558,8 @@ ProcArrayRemove(PGPROC *proc, TransactionId latestXid)
(arrayP->numProcs - index - 1) * sizeof(*arrayP->pgprocnos));
memmove(&ProcGlobal->xids[index], &ProcGlobal->xids[index + 1],
(arrayP->numProcs - index - 1) * sizeof(*ProcGlobal->xids));
+ memmove(&ProcGlobal->subxidStates[index], &ProcGlobal->subxidStates[index + 1],
+ (arrayP->numProcs - index - 1) * sizeof(*ProcGlobal->subxidStates));
memmove(&ProcGlobal->vacuumFlags[index], &ProcGlobal->vacuumFlags[index + 1],
(arrayP->numProcs - index - 1) * sizeof(*ProcGlobal->vacuumFlags));
@@ -597,8 +604,6 @@ ProcArrayRemove(PGPROC *proc, TransactionId latestXid)
/*
去掉MyPgXact结构体
*/
void
ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid)
{
- PGXACT *pgxact = &allPgXact[proc->pgprocno];
-
if (TransactionIdIsValid(latestXid))
{
/*
@@ -616,7 +621,7 @@ ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid)
*/
if (LWLockConditionalAcquire(ProcArrayLock, LW_EXCLUSIVE))
{
- ProcArrayEndTransactionInternal(proc, pgxact, latestXid);
+ ProcArrayEndTransactionInternal(proc, latestXid);
LWLockRelease(ProcArrayLock);
}
else
@@ -630,15 +635,14 @@ ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid)
* estimate of global xmin, but that's OK.
*/
Assert(!TransactionIdIsValid(proc->xid));
+ Assert(proc->subxidStatus.count == 0);
+ Assert(!proc->subxidStatus.overflowed);
proc->lxid = InvalidLocalTransactionId;
proc->xmin = InvalidTransactionId;
proc->delayChkpt = false; /* be sure this is cleared in abort */
proc->recoveryConflictPending = false;
- Assert(pgxact->nxids == 0);
- Assert(pgxact->overflowed == false);
-
/* must be cleared with xid/xmin: */
/* avoid unnecessarily dirtying shared cachelines */
if (proc->vacuumFlags & PROC_VACUUM_STATE_MASK)
@@ -659,8 +663,7 @@ ProcArrayEndTransaction(PGPROC *proc, TransactionId latestXid)
* We don't do any locking here; caller must handle that.
*/
static inline void
-ProcArrayEndTransactionInternal(PGPROC *proc, PGXACT *pgxact,
- TransactionId latestXid)
+ProcArrayEndTransactionInternal(PGPROC *proc, TransactionId latestXid)
{
size_t pgxactoff = proc->pgxactoff;
@@ -683,8 +686,15 @@ ProcArrayEndTransactionInternal(PGPROC *proc, PGXACT *pgxact,
}
/* Clear the subtransaction-XID cache too while holding the lock */
/*
清空Cache
*/
- pgxact->nxids = 0;
- pgxact->overflowed = false;
+ Assert(ProcGlobal->subxidStates[pgxactoff].count == proc->subxidStatus.count &&
+ ProcGlobal->subxidStates[pgxactoff].overflowed == proc->subxidStatus.overflowed);
+ if (proc->subxidStatus.count > 0 || proc->subxidStatus.overflowed)
+ {
+ ProcGlobal->subxidStates[pgxactoff].count = 0;
+ ProcGlobal->subxidStates[pgxactoff].overflowed = false;
+ proc->subxidStatus.count = 0;
+ proc->subxidStatus.overflowed = false;
+ }
/* Also advance global latestCompletedXid while holding the lock */
MaintainLatestCompletedXid(latestXid);
@@ -774,9 +784,8 @@ ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid)
while (nextidx != INVALID_PGPROCNO)
{
PGPROC *proc = &allProcs[nextidx];
- PGXACT *pgxact = &allPgXact[nextidx];
- ProcArrayEndTransactionInternal(proc, pgxact, proc->procArrayGroupMemberXid);
+ ProcArrayEndTransactionInternal(proc, proc->procArrayGroupMemberXid);
/* Move to next proc in list. */
nextidx = pg_atomic_read_u32(&proc->procArrayGroupNext);
@@ -820,7 +829,6 @@ ProcArrayGroupClearXid(PGPROC *proc, TransactionId latestXid)
void
ProcArrayClearTransaction(PGPROC *proc)
{
- PGXACT *pgxact = &allPgXact[proc->pgprocno];
size_t pgxactoff;
/*
@@ -845,8 +853,15 @@ ProcArrayClearTransaction(PGPROC *proc)
Assert(!proc->delayChkpt);
/* Clear the subtransaction-XID cache too */
- pgxact->nxids = 0;
- pgxact->overflowed = false;
+ Assert(ProcGlobal->subxidStates[pgxactoff].count == proc->subxidStatus.count &&
+ ProcGlobal->subxidStates[pgxactoff].overflowed == proc->subxidStatus.overflowed);
+ if (proc->subxidStatus.count > 0 || proc->subxidStatus.overflowed)
+ {
+ ProcGlobal->subxidStates[pgxactoff].count = 0;
+ ProcGlobal->subxidStates[pgxactoff].overflowed = false;
+ proc->subxidStatus.count = 0;
+ proc->subxidStatus.overflowed = false;
+ }
LWLockRelease(ProcArrayLock);
}
@@ -1267,6 +1282,7 @@ TransactionIdIsInProgress(TransactionId xid)
{
static TransactionId *xids = NULL;
static TransactionId *other_xids;
+ XidCacheStatus *other_subxidstates;
int nxids = 0;
ProcArrayStruct *arrayP = procArray;
TransactionId topxid;
@@ -1329,6 +1345,7 @@ TransactionIdIsInProgress(TransactionId xid)
}
other_xids = ProcGlobal->xids;
+ other_subxidstates = ProcGlobal->subxidStates;
LWLockAcquire(ProcArrayLock, LW_SHARED);
@@ -1351,7 +1368,6 @@ TransactionIdIsInProgress(TransactionId xid)
for (size_t pgxactoff = 0; pgxactoff < numProcs; pgxactoff++)
{
int pgprocno;
- PGXACT *pgxact;
PGPROC *proc;
TransactionId pxid;
int pxids;
@@ -1386,9 +1402,7 @@ TransactionIdIsInProgress(TransactionId xid)
/*
* Step 2: check the cached child-Xids arrays
*/
- pgprocno = arrayP->pgprocnos[pgxactoff];
- pgxact = &allPgXact[pgprocno];
- pxids = pgxact->nxids;
+ pxids = other_subxidstates[pgxactoff].count;
pg_read_barrier(); /* pairs with barrier in GetNewTransactionId() */
pgprocno = arrayP->pgprocnos[pgxactoff];
proc = &allProcs[pgprocno];
@@ -1412,7 +1426,7 @@ TransactionIdIsInProgress(TransactionId xid)
* we hold ProcArrayLock. So we can't miss an Xid that we need to
* worry about.)
*/
- if (pgxact->overflowed)
+ if (other_subxidstates[pgxactoff].overflowed)
xids[nxids++] = pxid;
}
@@ -2019,6 +2033,7 @@ GetSnapshotData(Snapshot snapshot)
size_t numProcs = arrayP->numProcs;
TransactionId *xip = snapshot->xip;
int *pgprocnos = arrayP->pgprocnos;
+ XidCacheStatus *subxidStates = ProcGlobal->subxidStates;
uint8 *allVacuumFlags = ProcGlobal->vacuumFlags;
/*
@@ -2095,17 +2110,16 @@ GetSnapshotData(Snapshot snapshot)
*/
if (!suboverflowed)
{
- int pgprocno = pgprocnos[pgxactoff];
- PGXACT *pgxact = &allPgXact[pgprocno];
- if (pgxact->overflowed)
+ if (subxidStates[pgxactoff].overflowed)
suboverflowed = true;
else
{
- int nsubxids = pgxact->nxids;
+ int nsubxids = subxidStates[pgxactoff].count;
if (nsubxids > 0)
{
+ int pgprocno = pgprocnos[pgxactoff];
PGPROC *proc = &allProcs[pgprocno];
pg_read_barrier(); /* pairs with GetNewTransactionId */
@@ -2498,8 +2512,6 @@ GetRunningTransactionData(void)
*/
for (index = 0; index < arrayP->numProcs; index++)
{
- int pgprocno = arrayP->pgprocnos[index];
- PGXACT *pgxact = &allPgXact[pgprocno];
TransactionId xid;
/* Fetch xid just once - see GetNewTransactionId */
@@ -2520,7 +2532,7 @@ GetRunningTransactionData(void)
if (TransactionIdPrecedes(xid, oldestRunningXid))
oldestRunningXid = xid;
- if (pgxact->overflowed)
/*
改为访问Proc
*/
+ if (ProcGlobal->subxidStates[index].overflowed)
suboverflowed = true;
/*
@@ -2540,27 +2552,28 @@ GetRunningTransactionData(void)
*/
if (!suboverflowed)
{
+ XidCacheStatus *other_subxidstates = ProcGlobal->subxidStates;
+
for (index = 0; index < arrayP->numProcs; index++)
{
int pgprocno = arrayP->pgprocnos[index];
PGPROC *proc = &allProcs[pgprocno];
- PGXACT *pgxact = &allPgXact[pgprocno];
- int nxids;
+ int nsubxids;
/*
* Save subtransaction XIDs. Other backends can't add or remove
* entries while we're holding XidGenLock.
*/
- nxids = pgxact->nxids;
- if (nxids > 0)
+ nsubxids = other_subxidstates[index].count;
+ if (nsubxids > 0)
{
/* barrier not really required, as XidGenLock is held, but ... */
pg_read_barrier(); /* pairs with GetNewTransactionId */
memcpy(&xids[count], (void *) proc->subxids.xids,
- nxids * sizeof(TransactionId));
- count += nxids;
- subcount += nxids;
+ nsubxids * sizeof(TransactionId));
+ count += nsubxids;
+ subcount += nsubxids;
/*
* Top-level XID of a transaction is always less than any of
@@ -3627,14 +3640,6 @@ ProcArrayGetReplicationSlotXmin(TransactionId *xmin,
LWLockRelease(ProcArrayLock);
}
-
-#define XidCacheRemove(i) \
- do { \
- MyProc->subxids.xids[i] = MyProc->subxids.xids[MyPgXact->nxids - 1]; \
- pg_write_barrier(); \
- MyPgXact->nxids--; \
- } while (0)
-
/*
* XidCacheRemoveRunningXids
*
@@ -3650,6 +3655,7 @@ XidCacheRemoveRunningXids(TransactionId xid,
{
int i,
j;
+ XidCacheStatus *mysubxidstat;
Assert(TransactionIdIsValid(xid));
@@ -3667,6 +3673,8 @@ XidCacheRemoveRunningXids(TransactionId xid,
*/
LWLockAcquire(ProcArrayLock, LW_EXCLUSIVE);
+ mysubxidstat = &ProcGlobal->subxidStates[MyProc->pgxactoff];
+
/*
* Under normal circumstances xid and xids[] will be in increasing order,
* as will be the entries in subxids. Scan backwards to avoid O(N^2)
@@ -3676,11 +3684,14 @@ XidCacheRemoveRunningXids(TransactionId xid,
{
TransactionId anxid = xids[i];
- for (j = MyPgXact->nxids - 1; j >= 0; j--)
/*
改为访问Proc->subxidXXX
*/
+ for (j = MyProc->subxidStatus.count - 1; j >= 0; j--)
{
if (TransactionIdEquals(MyProc->subxids.xids[j], anxid))
{
- XidCacheRemove(j);
+ MyProc->subxids.xids[j] = MyProc->subxids.xids[MyProc->subxidStatus.count - 1];
+ pg_write_barrier();
+ mysubxidstat->count--;
+ MyProc->subxidStatus.count--;
break;
}
}
@@ -3692,20 +3703,23 @@ XidCacheRemoveRunningXids(TransactionId xid,
* error during AbortSubTransaction. So instead of Assert, emit a
* debug warning.
*/
- if (j < 0 && !MyPgXact->overflowed)
+ if (j < 0 && !MyProc->subxidStatus.overflowed)
elog(WARNING, "did not find subXID %u in MyProc", anxid);
}
- for (j = MyPgXact->nxids - 1; j >= 0; j--)
+ for (j = MyProc->subxidStatus.count - 1; j >= 0; j--)
{
if (TransactionIdEquals(MyProc->subxids.xids[j], xid))
{
- XidCacheRemove(j);
+ MyProc->subxids.xids[j] = MyProc->subxids.xids[MyProc->subxidStatus.count - 1];
+ pg_write_barrier();
+ mysubxidstat->count--;
+ MyProc->subxidStatus.count--;
break;
}
}
/* Ordinarily we should have found it, unless the cache has overflowed */
- if (j < 0 && !MyPgXact->overflowed)
+ if (j < 0 && !MyProc->subxidStatus.overflowed)
elog(WARNING, "did not find subXID %u in MyProc", xid);
/* Also advance global latestCompletedXid while holding the lock */
diff --git a/src/backend/storage/lmgr/proc.c b/src/backend/storage/lmgr/proc.c
index f6113b2d24320573b19240a1fc7af9dee936078c..aa9fbd80545bc2248e7f72710febea8049f41ad1 100644 (file)
--- a/src/backend/storage/lmgr/proc.c
+++ b/src/backend/storage/lmgr/proc.c
@@ -63,9 +63,8 @@ int LockTimeout = 0;
int IdleInTransactionSessionTimeout = 0;
bool log_lock_waits = false;
-/* Pointer to this process's PGPROC and PGXACT structs, if any */
+/* Pointer to this process's PGPROC struct, if any */
PGPROC *MyProc = NULL;
-PGXACT *MyPgXact = NULL;
/*
* This spinlock protects the freelist of recycled PGPROC structures.
@@ -110,10 +109,8 @@ ProcGlobalShmemSize(void)
size = add_size(size, mul_size(TotalProcs, sizeof(PGPROC)));
size = add_size(size, sizeof(slock_t));
- size = add_size(size, mul_size(MaxBackends, sizeof(PGXACT)));
- size = add_size(size, mul_size(NUM_AUXILIARY_PROCS, sizeof(PGXACT)));
- size = add_size(size, mul_size(max_prepared_xacts, sizeof(PGXACT)));
size = add_size(size, mul_size(TotalProcs, sizeof(*ProcGlobal->xids)));
+ size = add_size(size, mul_size(TotalProcs, sizeof(*ProcGlobal->subxidStates)));
size = add_size(size, mul_size(TotalProcs, sizeof(*ProcGlobal->vacuumFlags)));
return size;
@@ -161,7 +158,6 @@ void
InitProcGlobal(void)
{
PGPROC *procs;
- PGXACT *pgxacts;
int i,
j;
bool found;
@@ -202,18 +198,6 @@ InitProcGlobal(void)
/* XXX allProcCount isn't really all of them; it excludes prepared xacts */
ProcGlobal->allProcCount = MaxBackends + NUM_AUXILIARY_PROCS;
- /*
- * Also allocate a separate array of PGXACT structures. This is separate
- * from the main PGPROC array so that the most heavily accessed data is
- * stored contiguously in memory in as few cache lines as possible. This
- * provides significant performance benefits, especially on a
- * multiprocessor system. There is one PGXACT structure for every PGPROC
- * structure.
- */
- pgxacts = (PGXACT *) ShmemAlloc(TotalProcs * sizeof(PGXACT));
- MemSet(pgxacts, 0, TotalProcs * sizeof(PGXACT));
- ProcGlobal->allPgXact = pgxacts;
-
/*
* Allocate arrays mirroring PGPROC fields in a dense manner. See
* PROC_HDR.
@@ -224,6 +208,8 @@ InitProcGlobal(void)
ProcGlobal->xids =
(TransactionId *) ShmemAlloc(TotalProcs * sizeof(*ProcGlobal->xids));
MemSet(ProcGlobal->xids, 0, TotalProcs * sizeof(*ProcGlobal->xids));
+ ProcGlobal->subxidStates = (XidCacheStatus *) ShmemAlloc(TotalProcs * sizeof(*ProcGlobal->subxidStates));
+ MemSet(ProcGlobal->subxidStates, 0, TotalProcs * sizeof(*ProcGlobal->subxidStates));
ProcGlobal->vacuumFlags = (uint8 *) ShmemAlloc(TotalProcs * sizeof(*ProcGlobal->vacuumFlags));
MemSet(ProcGlobal->vacuumFlags, 0, TotalProcs * sizeof(*ProcGlobal->vacuumFlags));
@@ -372,7 +358,6 @@ InitProcess(void)
(errcode(ERRCODE_TOO_MANY_CONNECTIONS),
errmsg("sorry, too many clients already")));
}
- MyPgXact = &ProcGlobal->allPgXact[MyProc->pgprocno];
/*
* Cross-check that the PGPROC is of the type we expect; if this were not
@@ -569,7 +554,6 @@ InitAuxiliaryProcess(void)
((volatile PGPROC *) auxproc)->pid = MyProcPid;
MyProc = auxproc;
- MyPgXact = &ProcGlobal->allPgXact[auxproc->pgprocno];
SpinLockRelease(ProcStructLock);
diff --git a/src/include/storage/proc.h b/src/include/storage/proc.h
index 9f3a8b518eb2f7ab8cc8546ce6536d1dd687022b..9c9a50ae457fd8a580410772abdffcad9cb74e4b 100644 (file)
--- a/src/include/storage/proc.h
+++ b/src/include/storage/proc.h
@@ -35,6 +35,14 @@
*/
#define PGPROC_MAX_CACHED_SUBXIDS 64 /* XXX guessed-at value */
/*
新增的数据结构:XidCacheStatus,专用于subxids,包括subxid数目和是否越界的标记
*/
+typedef struct XidCacheStatus
+{
+ /* number of cached subxids, never more than PGPROC_MAX_CACHED_SUBXIDS */
+ uint8 count;
+ /* has PGPROC->subxids overflowed */
+ bool overflowed;
+} XidCacheStatus;
+
/*
XidCache新增subxidStatus,是ProcGlobal子事务状态的镜像
*/
struct XidCache
{
TransactionId xids[PGPROC_MAX_CACHED_SUBXIDS];
@@ -187,6 +195,8 @@ struct PGPROC
*/
SHM_QUEUE myProcLocks[NUM_LOCK_PARTITIONS];
+ XidCacheStatus subxidStatus; /* mirrored with
+ * ProcGlobal->subxidStates[i] */
struct XidCache subxids; /* cache for subtransaction XIDs */
/* Support for group XID clearing. */
@@ -235,22 +245,6 @@ struct PGPROC
extern PGDLLIMPORT PGPROC *MyProc;
-extern PGDLLIMPORT struct PGXACT *MyPgXact;
-
-/*
- * Prior to PostgreSQL 9.2, the fields below were stored as part of the
- * PGPROC. However, benchmarking revealed that packing these particular
- * members into a separate array as tightly as possible sped up GetSnapshotData
- * considerably on systems with many CPU cores, by reducing the number of
- * cache lines needing to be fetched. Thus, think very carefully before adding
- * anything else here.
- */
-typedef struct PGXACT
-{
- bool overflowed;
-
- uint8 nxids;
-} PGXACT;
/*
* There is one ProcGlobal struct for the whole database cluster.
/*
ProcGlobal结构体全局可见
*/
@@ -310,12 +304,16 @@ typedef struct PROC_HDR
{
/*
PROC_HDR结构体新增subxidStates变量,用于镜像PGPROC.subxidStatus
*/
/* Array of PGPROC structures (not including dummies for prepared txns) */
PGPROC *allProcs;
- /* Array of PGXACT structures (not including dummies for prepared txns) */
- PGXACT *allPgXact;
/* Array mirroring PGPROC.xid for each PGPROC currently in the procarray */
TransactionId *xids;
+ /*
+ * Array mirroring PGPROC.subxidStatus for each PGPROC currently in the
+ * procarray.
+ */
+ XidCacheStatus *subxidStates;
+
/*
* Array mirroring PGPROC.vacuumFlags for each PGPROC currently in the
* procarray.
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index b4948ac675f7868c924d342b752b917e840e4a20..3d990463ce9cfae97b17ddee0a366a559730f113 100644 (file)
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -1536,7 +1536,6 @@ PGSetenvStatusType
PGShmemHeader
PGTransactionStatusType
PGVerbosity
-PGXACT
PG_Locale_Strategy
PG_Lock_Status
PG_init_t
三、跟踪分析
N/A
四、参考资料
1.
Improving connection scalability: GetSnapshotData()
2.
snapshot scalability: Don’t compute global horizons while building snapshots.