ToplingDB fork 自 rocksdb,核心是两个 github 仓库:
| github 仓库 | 内容 |
|---|---|
| ToplingDB | fork 自 RocksDB,仅对 RocksDB 进行最少的必要的修改,从而可以对 RocksDB 保持 100% 的兼容性 |
| rockside | SidePlugin 插件体系,是 ToplingDB 的 submodule |
ToplingDB 对 RocksDB 做了很多改进。
allow_fdatasync 使用场景
toplingDB 添加了参数allow_fdatasync,主要的适应场景是不需要使用文件同步功能时,关闭同步直接返回。
原版rocksdb必需执行同步过程。
fdatasync和fsync的介绍
原网址: https:// man7.org/linux/man-page s/man2/fdatasync.2.html
fsync() transfers ("flushes") all modified in-core data of (i.e.,
modified buffer cache pages for) the file referred to by the file
descriptor fd to the disk device (or other permanent storage
device) so that all changed information can be retrieved even if
the system crashes or is rebooted. This includes writing through
or flushing a disk cache if present. The call blocks until the
device reports that the transfer has completed.
As well as flushing the file data, fsync() also flushes the
metadata information associated with the file (see inode(7)).
Calling fsync() does not necessarily ensure that the entry in the
directory containing the file has also reached disk. For that an
explicit fsync() on a file descriptor for the directory is also
needed.
fdatasync() is similar to fsync(), but does not flush modified
metadata unless that metadata is needed in order to allow a
subsequent data retrieval to be correctly handled. For example,
changes to st_atime or st_mtime (respectively, time of last
access and time of last modification; see inode(7)) do not
require flushing because they are not necessary for a subsequent
data read to be handled correctly. On the other hand, a change
to the file size (st_size, as made by say ftruncate(2)), would
require a metadata flush.
The aim of fdatasync() is to reduce disk activity for
applications that do not require all metadata to be synchronized
with the disk.翻译大意如下:
Fsync()传输(“刷新”)所有已修改的内核内数据(即修改的缓冲区缓存页)由文件引用的文件描述符fd到磁盘设备(或其他永久存储)设备),以便在在系统崩溃或重新启动时,可以检索所有更改的信息。这包括通过或刷新磁盘缓存(如果存在的话)。调用阻塞,直到设备报告传输已经完成。
除了刷新文件数据外,fsync()还会刷新与文件相关的元数据信息(参见inode(7))。
调用fsync()并不一定确保包含文件的目录也已到达磁盘。为此需要显式添加fsync()对目录的文件描述符执行操作。
Fdatasync()类似于fsync(),但不会刷新修改元数据,除非该元数据是为了允许正确处理后续的数据检索。例如,更改为st_atime或st_mtime(分别为最后一次访问权限和上次修改时间;参见inode(7))不要要求刷新,因为后续操作不需要刷新就可以正确处理读取的数据。另一方面,改变转换为文件大小(st_size,如ftruncate(2))需要元数据刷新。
fdatasync()的目的是减少磁盘活动不需要同步所有元数据到磁盘的应用减少磁盘IO操作。
allow_fdatasync参数
通过添加参数allow_fdatasync,可以设置是否使用fdatasync同步数据内容到磁盘的过程。
参数从options参数传递到EnvOptions参数中,通过设置PosixWritableFile中的allow_fdatasync_来完成参数设置过程。
allow_fdatasync_ 用来控制是否使用fdatasync完成数据同步的过程。
通过添加参数allow_fdatasync_,可以设置不使用fdatasync来完成数据同步的过程。
```env/io_posix.cc
IOStatus PosixWritableFile::Sync(const IOOptions& /*opts*/,
IODebugContext* /*dbg*/) {
#ifdef HAVE_FULLFSYNC
if (::fcntl(fd_, F_FULLFSYNC) < 0) {
return IOError("while fcntl(F_FULLFSYNC)", filename_, errno);
}
#else // HAVE_FULLFSYNC
if (!allow_fdatasync_) {
return IOStatus::OK();
}
if (fdatasync(fd_) < 0) {
return IOError("While fdatasync", filename_, errno);
}
#endif // HAVE_FULLFSYNC
return IOStatus::OK();
}
```
更多详细内容待正在熟悉ToplingDB 后再分析,暂时分析到这里啦。