OneProxy的部署和启动:
从平民官网下载最新版本的Oneproxy,官网下载地址http://www.onexsoft.com/download,本说明文档下载使用以及实验使用的是目前最新的v5.5版本。
部署安装
1. 用root登陆到服务器
2. 解压oneproxy压缩文件,tar -zxvf oneproxy-rhel5-linux64-v5.5-ga.tar.gz
3. 解压后生成oneproxy文件夹,进入文件夹,可以看到一下文件:
部署安装
1. 用root登陆到服务器
2. 解压oneproxy压缩文件,tar -zxvf oneproxy-rhel5-linux64-v5.5-ga.tar.gz
3. 解压后生成oneproxy文件夹,进入文件夹,可以看到一下文件:
[root@localhost oneproxy]# ls
4. 重点介绍一下文件:
1. demo.sh 这是oneproxy的启动脚本的demo,可以拷贝这个脚本做修改来作为自己的启动配置文件
2. oneproxy oneproxy启动的二进制文件
3. oneproxy.log oneproxy的日志文件,可以以此监控oneproxy的启动状态以及节点的通信启动成功与否
demo.sh oneproxy part.txt README testadmin.sql testproxy.sql
4. 重点介绍一下文件:
1. demo.sh 这是oneproxy的启动脚本的demo,可以拷贝这个脚本做修改来作为自己的启动配置文件
2. oneproxy oneproxy启动的二进制文件
3. oneproxy.log oneproxy的日志文件,可以以此监控oneproxy的启动状态以及节点的通信启动成功与否
启动
1. 首先通过使用vim查看启动demo脚本:
#/bin/bash
#
export ONEPROXY_HOME=/data/oneproxy
if [ -f ${ONEPROXY_HOME}/oneproxy.pid ]; then
kill -9 `cat ${ONEPROXY_HOME}/oneproxy.pid`
fi
sleep 2
# valgrind --leak-check=full \
${ONEPROXY_HOME}/oneproxy --proxy-address=:3307 --proxy-extra-address=:3308 \
--proxy-master-addresses=192.168.1.119:3306@default \
--proxy-user-list=test/1378F6CC3A8E8A43CA388193FBED5405982FBBD3@test \
--proxy-part-tables=${ONEPROXY_HOME}/part.txt \
--proxy-charset=gbk_chinese_ci --proxy-found-rows \
--proxy-group-policy=default:master-only \
--event-threads=6 --proxy-group-security=default:0 \
--log-file=${ONEPROXY_HOME}/oneproxy.log \
--pid-file=${ONEPROXY_HOME}/oneproxy.pid
2. 接下来必须对启动配置文件作出修改,那么必须了解启动的配置参数,因为本章节只演示启动,那么将不对所有启动参数作出说明,而将在下面章节作出一一说明和参数的作用演示,想要查看所有参数信息可以通过命令查看:
./oneproxy –-help
./oneproxy –help-all
想要成功启动oneproxy那么必须了解一下参数:
--proxy-address
oneproxy的启动地址和端口信息,可以通过本地和远程mysql客户端登陆oneproxy代理接口,格式为IP:端口
--proxy-extra-address
oneproxy的备用启动地址和端口信息,格式为IP:端口
--proxy-master-addresses
在oneproxy中有server group的概念,一个server group其实可以理解为对应的一个DB集群,这个集群的拓扑结构可以对用户透明,比如典型的各种主从复制拓扑,而这个参数对应server group中的master节点,master节点一般负责集群数据的读写,格式为IP:端口@server group名称
--proxy-slave-addresses
这个参数对应对应server group中的slave节点,slave节点一般只负责集群中数据的查询,格式为IP:端口@server group名称
--proxy-user-list
这个参数对应登陆oneproxy的账户,同样的这个账户也是存在于所有server group中的所有节点,需要的注意的是这个账户需要赋予远程登陆的权限,比如:
,另外用户的密码必须是加密的,具体加密后的密码可以登陆到oneproxy管理端使用passwd命令去加密账户的明文密码
mysql> passwd 123;
+------+------------------------------------------+
| TEXT | PASSWORD |
+------+------------------------------------------+
| 123 | 956C97523BADD23B6091B09E332A77B5CBDBBEEF |
+------+------------------------------------------+
1 row in set (0.00 sec)
参数格式:用户名/加密后密码@数据库
--log-file
这个参数是启动日志文件,纪录节点的正常启动和失败信息
下面是实验实验环境的启动脚本配置
#/bin/bash
#
export ONEPROXY_HOME=/mysql/oneproxy
if [ -f ${ONEPROXY_HOME}/oneproxy.pid ]; then
kill -9 `cat ${ONEPROXY_HOME}/oneproxy.pid`
fi
sleep 2
# valgrind --leak-check=full \
${ONEPROXY_HOME}/oneproxy --proxy-address=:3307 --proxy-extra-address=:3308 \
--proxy-master-addresses=10.88.43.98:3306@default\
--proxy-slave-addresses=10.88.33.97:3306@default\
--proxy-master-addresses=10.1.71.90:3306@data\
--proxy-slave-addresses=10.1.71.91:3306@data\
--proxy-user-list=lgs/956C97523BADD23B6091B09E332A77B5CBDBBEEF@test \
--proxy-part-tables=${ONEPROXY_HOME}/part.txt \
--proxy-charset=gbk_chinese_ci --proxy-found-rows \
--proxy-group-policy=default:master-only \
--event-threads=6 --proxy-group-security=default:0 --proxy-enable-firewall --proxy-firewall-query=${ONEPROXY_HOME}/firewall.sql\
--log-file=${ONEPROXY_HOME}/oneproxy.log \
--pid-file=${ONEPROXY_HOME}/oneproxy.pid
1. 启动oneproxy
通过命令sh oneproxy来运行启动脚本,虽然命令端不会打印出太多信息,但是可以通过查看日志文件查看启动的情况
2015-05-16 02:10:48: (critical) plugin oneproxy 5.5.0-20150513 started
2015-05-16 02:10:48: (critical) valid config checksum = 4023582183
2015-05-16 02:10:49: (critical) Ping backend (10.1.71.90:3306) success, mark it up!
2015-05-16 02:10:49: (critical) Ping backend (10.1.71.91:3306) success, mark it up!
2015-05-16 02:10:49: (critical) Ping backend (10.88.43.98:3306) success, mark it up!
2015-05-16 02:10:49: (critical) Ping backend (10.88.33.97:3306) success, mark it up!
通过日志文件可以看到脚本中配置的节点都顺利的被标志为启动。可能会有这样一种情况,如果日志文件中一致显示节点挂起或者宕机可能是用户列表中指定的数据库有误。
2. 现在oneproxy已经正常启动,那么接下来重点介绍oneproxy提供的两个重要接口
1. 代理接口
代理接口其实和一般mysql登陆的客户端一样,所以登陆过程也基本一致,根据配置的启动脚本,登陆命令应该是:
mysql –h 10.88.43.98 –-user=lgs –p123 –P3307
登陆进去后,可以执行所有用户熟悉mysql客户端命令,重点演示show databases命令
mysql> show databases;
+----------+
| Database |
+----------+
| data |
| default |
+----------+
2 rows in set (0.00 sec)
mysql> use data;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+----------------+
| Tables_in_test |
+----------------+
| test |
+----------------+
1 row in set (0.02 sec)
可以发现代理端接口显示的并不是像mysql客户端中显示的数据库信息,而是配置文件中的server group名称,可以进入对应的server group查看其中的表信息,可以看到的是表正是对应启动配置文件中server group映射的数据库test信息。在上面的说明中提到过oneproxy不关心server group的集群拓扑结构,对于用户而言这是对用户透明的,但是需要强调的是最好保证所有数据节点中数据的一致性,也就是说master和slave的数据是一致的,这也是典型主从复制结构所要求的。
代理端口的优势就在于,用户不需要关心server group中发生的数据行为,比如数据的实际的读写是发生在哪个节点上的,当发生节点故障时切换的逻辑是怎样的,乃至分库分表后servergroup展示用户的可见信息甚至没发生改动的,可以做到完全透明化,这样不但能够轻松让用户管理servergroup,而且可以在不改动代码的前提下,轻松实现servergroup背后的数据架构改动。
a. 管理接口
管理接口不同于代理端接口,它是完全用于管理oneproxy中的所有节点以及可以作出所有oneproxy的功能,先登陆管理接口
mysql -h 10.88.43.98 -P4041 -p OneProxy --user=admin
系统默认连接端口是4041,用户名密码分别是admin,OneProxy
成功登陆后,如果想要知道所有管理命令那么执行help查看
mysql> list help;
+---------------------+--------------------------------------------------+------------------------------------------+
| COMMAND | SUMMARY | SAMPLE |
+---------------------+--------------------------------------------------+------------------------------------------+
| LIST HELP | Display the command list. | list |
| LIST CHARSET | Display the charset list. | list charset |
| LIST POLICY | Display the traffic policy for server group. | list policy |
| LIST BACKEND | Display info for all the backends. | list backend |
| LIST GROUP | Display info for all the server groups. | list group |
| LIST POOL | Display info for all the connection pools. | list pool |
| LIST QUEUE | Display statitics data for all the event queues. | list queue |
| LIST THREADS | Display statitics data for all the threads. | list threads |
| LIST TABLEMAP | Display the map of tables and server groups. | list tablemap |
| LIST USERS | Display the user list. | list users |
| LIST SQLSTATS | Display the sql statistics. | list sqlstats |
| LIST TABSTATS | Display the table statistics. | list tabstats |
| LIST USERSTATS | Display the user statistics. | list userstats |
| LIST IPSTATS | Display the IP based perfstats . | list ipstats |
| LIST IPTABSTATS | Display the IP based table statistics. | list iptabstats |
| LIST SQLTEXT | Display the sql text. | list sqltext |
| LIST DMLTEXT | Display the sql text of dml operation. | list dmltext |
| LIST NEWTEXT | Display the sql text gathered by runtime. | list newtext |
| LIST OLDTEXT | Display the sql text loaded by firewall. | list oldtext |
| LIST SQLFAIL | Display the failed sql in dry run. | list sqlfail |
| LIST TRANS_DEBUG | Display the table map of transaction. | list trans_debug |
| LIST TABLES | Display the partition tables. | list tables |
| LIST PARTITIONS | Display the table partitions. | list partitions |
| SET MASTER | Set a given backend as master backend. | set master '192.168.1.119:3306' |
| SET SLAVE | Set a given backend as slave backend. | set slave '192.168.1.119:3306' |
| SET OFFLINE | Set a given backend as offline backend. | set offline '192.168.1.119:3306' |
| SET ONLINE | Set a given backend as online backend. | set online '192.168.1.119:3306' |
| SET GPOLICY | Set a feature policy for given server group. | set gpolicy default 1 |
| SET GMASTER | Set the master backend for given server group. | set gmaster default 1 |
| SET GACCESS | Set the security level for given server group. | set gaccess default 1 |
| SET GMANUAL | Set manual master mode for given server group. | set gmanual default 1 |
| SET POOLMIN | Set the min idle connections for given backend. | set poolmin '192.168.1.119:3306' 5 |
| SET POOLMAX | Set the max idle connections for given backend. | set poolmax '192.168.1.119:3306' 300 |
| SET SQLSTATS | Enable or disable the sql statistics feature. | set sqlstats {on|off|clear} |
| SET IPSTATS | Clear the ip statistics data. | set ipstats {clear} |
| SET TABSTATS | Enable or disable the table statistics feature. | set tabstats {on|off|clear} |
| SET USERSTATS | Enable or disable the user statistics feature. | set userstats {on|off|clear} |
| SET TRANS_DEBUG | Enable or disable the transaction debug mode. | set trans_debug {on|off} |
| SET LOG_SQLERROR | Enable or disable the sql error log feature. | set log_sqlerror {on|off} |
| SET SECURITY_LEVEL | set the security level of the proxy. | set security_level {0-4} |
| SET SQL_FIREWALL | Enable or disable the SQL Firewall feature. | set sql_firewall {on|off} |
| SET SQL_SIGNATURE | Enable or disable the SQL signature feature. | set sql_signature {on|off} |
| SET IP_FIREWALL | Enable or disable the IP Firewall feature. | set ip_firewall {on|off} |
| SET LOGIN_RESTRICT | Enable or disable the login restrict mode. | set login_restrict {on|off} |
| SET UNSAFE_GROUP | Enalbe or disable the unsafe group operation. | set unsafe_group {on|off} |
| SET MULTI_INSERTS | Enalbe or disable cross partition operation. | set multi_inserts {on|off} |
| SET BLACKIP | Disable the access of specific ip address. | set blackip '192.168.0.1' |
| SET GREENIP | Enable the access of specific ip address. | set greenip '192.168.0.1' |
| SET READ_ONLY | Put a partition into read only mode. | set read_only 'my_range_0' |
| SET READ_WRITE | Put a partition into writable mode. | set read_write 'my_range_0' |
| ADD MASTER | Add a new master(rw) backend to oneproxy. | add master '192.168.1.120:3306@default' |
| ADD SLAVE | Add a new slave(ro) backend to oneproxy. | add slave '192.168.1.120:3306@default' |
| SAVE SQL | save sql list to a text file. | save sql '/tmp/firewall.sql' |
| SAVE DML | save dml sql list to a text file. | save dml '/tmp/firewall_dml.sql' |
| SAVE NEW | save new sql list to a text file. | save new '/tmp/firewall_new.sql' |
| SAVE IP | save ip list to a text file. | save ip '/tmp/firewall.sql' |
| LOAD SQL | load firewall sql from text file. | load sql '/tmp/firewall.sql' |
| LOAD IP | load firewall ip list from text file. | load ip '/tmp/firewall_ip.sql' |
| MAP | Map a table name to a given server group. | map my_test1_0 default |
| UNMAP | Remove the server gruop for a given table. | unmap my_test1_0 |
| PASSWD | Encrypt the password of command options. | passwd test |
| SHUTDOWN | Shutdown the proxy. | shutdown force |
+---------------------+--------------------------------------------------+------------------------------------------+
62 rows in set (0.00 sec)
这里不对所有命令进行一一介绍,将会在下面的章节展示OneProxy功能和管理的过程中均有涉及,对应本章的启动例子,这里可以通过使用list backend来查看所有节点的状态
mysql> list backend;
+------+-----------------+-----------+--------+----------+---------+
| INDX | ADDRESS | TYPE | STATUS | REQUESTS | GROUP |
+------+-----------------+-----------+--------+----------+---------+
| 1 | 10.88.7.52:3306 | RW/Master | UP | 0 | default |
| 2 | 10.1.71.90:3306 | RW/Master | UP | 0 | data |
| 3 | 10.88.7.5:3306 | RO/Slave | UP | 0 | default |
| 4 | 10.1.71.91:3306 | RO/Slave | UP | 0 | data |
+------+-----------------+-----------+--------+----------+---------+
4 rows in set (0.00 sec)