Written with StackEdit.
数据库驱动及socket编程说明¶
因为 tcp/ip 并没有在协议内部实现心跳检测, 极为依赖应用程序自己来实现心跳检测. 所以在 web 程序这种一般要 pre connect 的模式下, 一般是初始化后一直使用一条连接, 不同数据库驱动在连接池以及自动重连上的实现有所不同, 但是为了程序的健壮性, 一般都要考虑 auto reconnect 的问题.
在 captcha 中的问题是, 备用集群平常基本没有流量, 导致建立的连接长时间无人使用, 可能会导致连接被重置等之类的问题, 所以如果数据库驱动能够支持自动重连是最好的.
参考资料: tcp+ip高效编程++改善网络程序的44个技巧
下面以我们最常用的几个数据库驱动组件来做说明和测试
使用工具: tcpkill
aioredis¶
aioredis.create_redis_pool¶
把 gt-server 改成了单进程启动, 以及连接池内最大只使用一个连接.
redis.redis = yield from aioredis.create_redis_pool(
SERVER_REDIS_ADDRESS, db=0, maxsize=1)
开启 gt-server 应用程序后, 和 redis 建立的连接如下所示.
ryefccd@fccd:~$ lsof -n -i tcp:6379
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
python3.5 15463 ryefccd 15u IPv4 89364230 0t0 TCP 192.168.1.54:48232->192.168.1.200:6379 (ESTABLISHED)
python3.5 15463 ryefccd 16u IPv4 89364231 0t0 TCP 192.168.1.54:48234->192.168.1.200:6379 (ESTABLISHED)
python3.5 15463 ryefccd 17u IPv4 89364232 0t0 TCP 192.168.1.54:48236->192.168.1.200:6379 (ESTABLISHED)
python3.5 15463 ryefccd 18u IPv4 89364233 0t0 TCP 192.168.1.54:48238->192.168.1.200:6379 (ESTABLISHED)
使用 tcpkill 杀掉 tcp 连接.
ryefccd@fccd:~$ sudo tcpkill -i eth1 -9 port 48232
[sudo] password for ryefccd:
tcpkill: listening on eth1 [port 48232]
192.168.1.54:48232 > 192.168.1.200:6379: R 1804130665:1804130665(0) win 0
192.168.1.54:48232 > 192.168.1.200:6379: R 1804130947:1804130947(0) win 0
192.168.1.54:48232 > 192.168.1.200:6379: R 1804131511:1804131511(0) win 0
...
192.168.1.200:6379 > 192.168.1.54:48232: R 2919458341:2919458341(0) win 0
192.168.1.200:6379 > 192.168.1.54:48232: R 2919460903:2919460903(0) win 0
192.168.1.200:6379 > 192.168.1.54:48232: R 2919463831:2919463831(0) win 0
用于 proof 的连接被 kill
ryefccd@fccd:~$ lsof -n -i tcp:6379
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
python3.5 15463 ryefccd 16u IPv4 89364231 0t0 TCP 192.168.1.54:48234->192.168.1.200:6379 (ESTABLISHED)
python3.5 15463 ryefccd 17u IPv4 89364232 0t0 TCP 192.168.1.54:48236->192.168.1.200:6379 (ESTABLISHED)
python3.5 15463 ryefccd 18u IPv4 89364233 0t0 TCP 192.168.1.54:48238->192.168.1.200:6379 (ESTABLISHED)
请求再次发生后, 连接已经重建.
ryefccd@fccd:~$ lsof -n -i tcp:6379
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
python3.5 15463 ryefccd 16u IPv4 89364231 0t0 TCP 192.168.1.54:48234->192.168.1.200:6379 (ESTABLISHED)
python3.5 15463 ryefccd 17u IPv4 89364232 0t0 TCP 192.168.1.54:48236->192.168.1.200:6379 (ESTABLISHED)
python3.5 15463 ryefccd 18u IPv4 89364233 0t0 TCP 192.168.1.54:48238->192.168.1.200:6379 (ESTABLISHED)
python3.5 15463 ryefccd 23u IPv4 89483848 0t0 TCP 192.168.1.54:49896->192.168.1.200:6379 (ESTABLISHED)
这里是使用的 aioredis 的 aioredis.create_redis_pool 来使用这个驱动内置的连接池, 查看源码也是找到连接池内可用的连接, 如果没有就创建一个新的连接. 这是所谓的高层次的api, 封装了底层相关的网络异常, 建议优先使用这些方法简化业务逻辑的开发.
aioredis.create_connection¶
amp = yield from aioredis.create_connection(
SERVER_REDIS_ADDRESS, db=0)
redis.redis = aioredis.Redis(amp)
# redis.redis = yield from aioredis.create_redis_pool(
# SERVER_REDIS_ADDRESS, db=0, maxsize=1)
下面换成 aioredis.create_connection 要演示
开启 gt-server 应用程序后, 和 redis 建立的连接如.
ryefccd@fccd:~$ lsof -n -i tcp:6379
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
python3.5 16706 ryefccd 15u IPv4 90572954 0t0 TCP 192.168.1.54:38360->192.168.1.200:6379 (ESTABLISHED)
python3.5 16706 ryefccd 16u IPv4 90572955 0t0 TCP 192.168.1.54:38362->192.168.1.200:6379 (ESTABLISHED)
python3.5 16706 ryefccd 17u IPv4 90572956 0t0 TCP 192.168.1.54:38364->192.168.1.200:6379 (ESTABLISHED)
python3.5 16706 ryefccd 18u IPv4 90572957 0t0 TCP 192.168.1.54:38366->192.168.1.200:6379 (ESTABLISHED)
tcpkill 杀掉第一个连接
^Cryefccd@fccd:~$ sudo tcpkill -i eth1 -9 port 38360
[sudo] password for ryefccd:
tcpkill: listening on eth1 [port 38360]
192.168.1.54:38360 > 192.168.1.200:6379: R 908335530:908335530(0) win 0
192.168.1.54:38360 > 192.168.1.200:6379: R 908335759:908335759(0) win 0
192.168.1.54:38360 > 192.168.1.200:6379: R 908336217:908336217(0) win 0
192.168.1.54:38360 > 192.168.1.200:6379: R 908336904:908336904(0) win 0
192.168.1.54:38360 > 192.168.1.200:6379: R 908337820:908337820(0) win 0
192.168.1.54:38360 > 192.168.1.200:6379: R 908338965:908338965(0) win 0
192.168.1.54:38360 > 192.168.1.200:6379: R 908340339:908340339(0) win 0
192.168.1.54:38360 > 192.168.1.200:6379: R 908341942:908341942(0) win 0
192.168.1.54:38360 > 192.168.1.200:6379: R 908343774:908343774(0) win 0
...
报错 Connection reset
[2019-03-19T09:23:39.587172+0800] ERROR handler.py run [Errno 104] Connection reset by peer
Traceback (most recent call last):
File "/home/ryefccd/workspace/server18/server/handler.py", line 350, in run
self.output = yield from self.get_cache()
File "/home/ryefccd/workspace/server18/server/handler.py", line 309, in get_cache
cache = yield from self.redis.get(self.url_md5)
File "/home/ryefccd/workspace/server18/common/myredis.py", line 22, in get
result = yield from MySingleRedis.instance().redis.get(key)
File "/home/ryefccd/env/server18/lib/python3.5/site-packages/aioredis/commands/string.py", line 83, in get
return self.execute(b'GET', key, encoding=encoding)
File "/home/ryefccd/env/server18/lib/python3.5/site-packages/aioredis/commands/__init__.py", line 50, in execute
return self._pool_or_conn.execute(command, *args, **kwargs)
File "/home/ryefccd/env/server18/lib/python3.5/site-packages/aioredis/connection.py", line 287, in execute
raise ConnectionClosedError(msg)
aioredis.errors.ConnectionClosedError: [Errno 104] Connection reset by peer
重新发起请求, 不会建立连接, 任然报 Connection reset 的错误.
ryefccd@fccd:~$ lsof -n -i tcp:6379
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
python3.5 16706 ryefccd 16u IPv4 90572955 0t0 TCP 192.168.1.54:38362->192.168.1.200:6379 (ESTABLISHED)
python3.5 16706 ryefccd 17u IPv4 90572956 0t0 TCP 192.168.1.54:38364->192.168.1.200:6379 (ESTABLISHED)
python3.5 16706 ryefccd 18u IPv4 90572957 0t0 TCP 192.168.1.54:38366->192.168.1.200:6379 (ESTABLISHED)
Traceback (most recent call last):
File "/home/ryefccd/workspace/server18/server/handler.py", line 355, in run
yield from self.updateproof()
File "/home/ryefccd/workspace/server18/server/registerhandler.py", line 71, in updateproof
yield from self.proof_obj.save()
File "/home/ryefccd/workspace/server18/common/myredis.py", line 27, in set
yield from MySingleRedis.instance().redis.setex(key, expire, data)
File "/home/ryefccd/env/server18/lib/python3.5/site-packages/aioredis/commands/string.py", line 218, in setex
fut = self.execute(b'SETEX', key, seconds, value)
File "/home/ryefccd/env/server18/lib/python3.5/site-packages/aioredis/commands/__init__.py", line 50, in execute
return self._pool_or_conn.execute(command, *args, **kwargs)
File "/home/ryefccd/env/server18/lib/python3.5/site-packages/aioredis/connection.py", line 287, in execute
raise ConnectionClosedError(msg)
aioredis.errors.ConnectionClosedError: [Errno 104] Connection reset by peer
mongo¶
pymongo¶
to do
motor¶
# kill 一个连接后
ryefccd@fccd:~$ lsof -w -n -i :32017 |grep python3.5
python3.5 9015 ryefccd 19u IPv4 133604407 0t0 TCP 192.168.1.54:56900->10.0.0.201:32017 (ESTABLISHED)
python3.5 9015 ryefccd 20u IPv4 133604408 0t0 TCP 192.168.1.54:56902->10.0.0.201:32017 (ESTABLISHED)
python3.5 9015 ryefccd 21u IPv4 133760502 0t0 TCP 192.168.1.54:59010->10.0.0.201:32017 (ESTABLISHED)
ryefccd@fccd:~$
ryefccd@fccd:~$
# 再次重新建立连接
ryefccd@fccd:~$ lsof -w -n -i :32017 |grep python3.5
python3.5 9015 ryefccd 19u IPv4 133604407 0t0 TCP 192.168.1.54:56900->10.0.0.201:32017 (ESTABLISHED)
python3.5 9015 ryefccd 20u IPv4 133604408 0t0 TCP 192.168.1.54:56902->10.0.0.201:32017 (ESTABLISHED)
python3.5 9015 ryefccd 21u IPv4 133760502 0t0 TCP 192.168.1.54:59010->10.0.0.201:32017 (ESTABLISHED)
python3.5 9015 ryefccd 22u IPv4 133961102 0t0 TCP 192.168.1.54:33482->10.0.0.201:32017 (ESTABLISHED)
postgresql¶
todo
mysql¶
todo