{"id":201276,"date":"2025-05-10T08:26:49","date_gmt":"2025-05-10T00:26:49","guid":{"rendered":"https:\/\/server.hk\/cnblog\/201276\/"},"modified":"2025-05-10T08:26:49","modified_gmt":"2025-05-10T00:26:49","slug":"redis%e9%9b%86%e7%be%a4%e6%96%b9%e6%a1%88","status":"publish","type":"post","link":"https:\/\/server.hk\/cnblog\/201276\/","title":{"rendered":"Redis\u96c6\u7fa4\u65b9\u6848"},"content":{"rendered":"

<\/b> <\/p>\n

Redis\u96c6\u7fa4\u65b9\u6848<\/h1>\n

<\/i>\u6536\u85cf<\/span> <\/p>\n

\u672c\u7bc7\u6587\u7ae0\u7ed9\u5927\u5bb6\u5206\u4eab\u300aRedis\u96c6\u7fa4\u65b9\u6848\u300b\uff0c\u8986\u76d6\u4e86\u6570\u636e\u5e93\u7684\u5e38\u89c1\u57fa\u7840\u77e5\u8bc6\uff0c\u5176\u5b9e\u4e00\u4e2a\u8bed\u8a00\u7684\u5168\u90e8\u77e5\u8bc6\u70b9\u4e00\u7bc7\u6587\u7ae0\u662f\u4e0d\u53ef\u80fd\u8bf4\u5b8c\u7684\uff0c\u4f46\u5e0c\u671b\u901a\u8fc7\u8fd9\u4e9b\u95ee\u9898\uff0c\u8ba9\u8bfb\u8005\u5bf9\u81ea\u5df1\u7684\u638c\u63e1\u7a0b\u5ea6\u6709\u4e00\u5b9a\u7684\u8ba4\u8bc6(B \u6570)\uff0c\u4ece\u800c\u5f25\u8865\u81ea\u5df1\u7684\u4e0d\u8db3\uff0c\u66f4\u597d\u7684\u638c\u63e1\u5b83\u3002<\/p>\n

\u524d\u6bb5\u65f6\u95f4\u641e\u4e86\u641eRedis\u96c6\u7fa4\uff0c\u60f3\u7528\u505a\u63a8\u8350\u7cfb\u7edf\u7684\u7ebf\u4e0a\u5b58\u50a8\uff0c\u8bf4\u6765\u633a\u6709\u8da3\uff0c\u8fd9\u8fb9\u57fa\u7840\u67b6\u6784\u4e0d\u592a\u5b8c\u5584\uff0c\u56e0\u6b64\u9700\u8981\u6211\u4eec\u505a\u63a8\u8350\u7cfb\u7edf\u7684\u81ea\u5df1\u6765\u642d\u8fd9\u4e2a\u5b58\u50a8\u73af\u5883\uff0c\u5c31\u81ea\u5df1\u6298\u817e\u4e86\u6298\u817e\u3002\u516c\u53f8\u6240\u7ed9\u673a\u5668\u7684\u5355\u673a\u6027\u80fd\u5176\u5b9e\u633a\u7ed9\u529b\uff0c\u5df2\u7ecf\u53ef\u4ee5\u6ee1\u8db3\u76ee\u524d\u7684\u4e1a\u52a1\u9700\u6c42\uff0c\u60f3\u505aredis\u96c6\u7fa4\u4e3b\u8981\u6709\u4ee5\u4e0b\u51e0\u70b9\u8003\u8651\uff1a<\/p>\n

    1\u3001\u6269\u5c55\u6027\uff0cscale-out\uff0c\u4ee5\u540e\u6570\u636e\u91cf\u53d8\u5f97\u5f88\u5927\u4e4b\u540e\uff0c\u4e0d\u81f3\u4e8e\u63a8\u5230\u91cd\u6765\uff0credis\u867d\u7136\u53ef\u4ee5\u5f00\u542f\u865a\u62df\u5185\u5b58\u529f\u80fd\uff0c\u5355\u673a\u4e5f\u80fd\u63d0\u4f9b\u8d85\u8fc7\u7269\u7406\u5185\u5b58\u4e0a\u9650\u7684\u5bb9\u91cf\uff0c\u4f46\u9891\u7e41\u5728\u5185\u5b58\u548c\u786c\u76d8\u95f4swap\u9875\u4f1a\u5927\u5927\u964d\u4f4e\u5176\u6027\u80fd\uff0c\u6709\u70b9\u513f\u8fdd\u80ccredis\u7684\u8bbe\u8ba1\u521d\u8877\u3002<\/p>\n

    2\u3001redis\u662f\u4e00\u4e2a\u5355\u7ebf\u7a0bio\u590d\u7528\u7684\u7ed3\u6784\uff0c\u65e0\u6cd5\u6709\u6548\u5229\u7528\u670d\u52a1\u5668\u7684\u591a\u6838\u7ed3\u6784\uff0c\u5982\u679c\u80fd\u5728\u4e00\u53f0\u591a\u6838\u673a\u5668\u8d77\u591a\u4e2aredis\u8fdb\u7a0b\uff0c\u5171\u540c\u63d0\u4f9b\u670d\u52a1\uff0c\u6548\u7387\u4f1a\u66f4\u9ad8\u4e00\u4e9b\u3002<\/p>\n

    3\u3001\u4e3b\u4ece\uff0c\u6570\u636e\u5907\u4efd\u548c\u5bb9\u707e\u3002\u3002<\/p>\n

\u56e0\u6b64\u8ba1\u5212\u505a\u7684redis\u96c6\u7fa4\u5e0c\u671b\u53ef\u4ee5\u5b9e\u73b0\u4ee5\u4e0b\u529f\u80fd\uff1a<\/p>\n

    1\u3001data sharding\uff0c\u652f\u6301\u6570\u636e\u5207\u7247\u3002<\/p>\n

    2\u3001\u4e3b\u4ece\u5907\u4efd\uff0c\u4e3b\u8282\u70b9\u5199\u6570\u636e\uff0c\u4e3b\u548c\u4ece\u90fd\u63d0\u4f9b\u8bfb\u8bf7\u6c42\u670d\u52a1\uff0c\u5e76\u4e14\u652f\u6301\u4e3b\u4ece\u81ea\u52a8\u5207\u6362\u3002<\/p>\n

    3\u3001\u8bfb\u8bf7\u6c42\u505a\u8d1f\u8f7d\u5747\u8861\u3002<\/p>\n

    4\u3001\u66f4\u597d\u5730\uff0c\u652f\u6301\u8282\u70b9failover\uff0c\u6570\u636e\u81ea\u52a8\u8fc1\u79fb\u3002<\/p>\n

\u4e0b\u9762\u662f\u524d\u540e\u7ecf\u5386\u7684\u4e00\u4e2a\u8fc7\u7a0b\uff1a<\/strong><\/p>\n

\u3010\u7b2c\u4e00\u6b65\u3011\u5c1d\u8bd5\u5b98\u65b9\u65b9\u6848<\/strong><\/p>\n

   \u80af\u5b9a\u60f3\u53bb\u67e5\u770b\u4e00\u4e0bredis\u7684\u5b98\u65b9\u96c6\u7fa4\u65b9\u6848\uff0c\u4f46\u662f\u5f88\u9057\u61be\uff0c\u5b98\u65b9\u5bf9cluster\u7684\u58f0\u660e\u5982\u4e0b\uff1a<\/p>\n

Unfortunately Redis Cluster is currently not production ready, however you can get more information about it reading the specification or checking the partial implementation in the unstable branch of the Redis GitHub repositoriy.<\/p>\n

Once Redis Cluster will be available, and if a Redis Cluster complaint client is available for your language, Redis Cluster will be the de facto standard for Redis partitioning.<\/p>\n

Redis Cluster is a mix between query routing and client side partitioning.<\/p>\n

  \u7531\u4e8e\u8fd9\u8fb9\u60f3\u505a\u751f\u4ea7\u73af\u5883\u90e8\u7f72\uff0cunstable branch\u76ee\u524d\u8fd8\u662f\u4e0d\u6562\u7528\uff0c\u5728\u5b98\u65b9\u76ee\u524d\u7684\u7248\u672c\u4e0a\u505a\u63d0\u524d\u5f00\u53d1\u53c8\u6ca1\u6709\u8d44\u6e90\u548c\u65f6\u95f4\uff0c\u56e0\u6b64\u5c31\u653e\u5f03\u4e86\u3002<\/p>\n

\u3010\u7b2c\u4e8c\u6b65\u3011\u521d\u6b65\u8bbe\u60f3\u7684\u65b9\u6848<\/strong><\/p>\n

   \u820d\u5f03\u4e86\u5b98\u65b9\u7684\u65b9\u6848\u540e\uff0c\u5c31\u60f3\u80fd\u4e0d\u80fd\u81ea\u5df1\u642d\u4e00\u4e2a\uff0c\u5f53\u65f6\u521d\u6b65\u7684\u60f3\u6cd5\u662f\uff1a\u7528lvs\u505a\u8bfb\u8bf7\u6c42\u7684\u8d1f\u8f7d\u5747\u8861\uff0c\u5728\u5ba2\u6237\u7aef\u4ee3\u7801\u91cc\u81ea\u5df1\u5199\u4e00\u4e2a\u4e00\u81f4\u6027hash\u7b97\u6cd5\u505a\u6570\u636e\u5207\u7247\uff0c\u914d\u7f6eredis\u4e3b\u4ece\uff0c\u5e76\u4e14\u914d\u7f6ekeepalived\u505a\u4e3b\u4ece\u81ea\u52a8\u5207\u6362\u3002\u8fd9\u4e2a\u65b9\u6848\u5e94\u8be5\u53ef\u4ee5\u65bd\u884c\u7684\uff0c\u4f46\u5f53\u65f6\u81ea\u5df1\u9047\u5230\u4e00\u4e9b\u7ec6\u8282\u65b9\u9762\u7684\u95ee\u9898\uff0c\u5c31\u5728stackoverflow\u4e0a\u95ee\u4e86\u4e00\u4e0b\uff0c\u95ee\u9898\u5982\u4e0b\uff1a<\/p>\n

Since the redis cluster is still a work in progress, I want to build a simplied one by myselfin the current stage. The system should support data sharding,load balance and master-slave backup. A preliminary plan is as follows:<\/p>\n

    \n
  1. \n

    Master-slave: use multiple master-slave pairs in different locations to enhance the data security. Matsters are responsible for the write operation, while both masters and slaves can provide the read service. Datas are sent to all the masters during one write operation. Use Keepalived between the master and the slave to detect failures and switch master-slave automatically.<\/p>\n<\/li>\n

  2. \n

    Data sharding: write a consistant hash on the client side to support data sharding during write\/read in case the memory is not enougth in single machine.<\/p>\n<\/li>\n

  3. \n

    Load balance: use LVS to redirect the read request to the corresponding server for the load balance.<\/p>\n<\/li>\n<\/ol>\n

    My question is how to combine the LVS and the data sharding together?<\/p>\n

    For example, because of data sharding, all keys are splited and stored in server A,B and C without overlap. Considering the slave backup and other master-slave pairs, the system will contain 1(A,B,C), 2(A,B,C) , 3(A,B,C) and so on, where each one has three servers. How to configure the LVS to support the redirection in such a situation when a read request comes? Or is there other approachs in redis to achieve the same goal?<\/p>\n

    Thanks:)<\/p>\n

    \u6709\u4e2a\u7f51\u53cb\u7ed9\u4e86\u4e24\u4e2a\u5efa\u8bae\uff1a<\/strong><\/p>\n

    You can really close to what you need by using:<\/p>\n

     shard data across multiple redis nodes (it also supports node ejection and connection pooling)<\/p>\n

    redis slave master\/slave replication<\/p>\n

     to handle master failover<\/p>\n

    depending on your needs you probably need some script listening to fail overs (see sentinel docs) and clean things up when a master goes down<\/p>\n

    \u8fd9\u4f4d\u7f51\u53cb\u7684\u4e24\u4e2a\u5efa\u8bae\u633a\u542f\u53d1\u7684\uff0c\u6211\u5728\u770bredis\u7684\u5b98\u65b9doc\u7684\u65f6\u5019\uff0c\u5bf9twemproxy\u6709\u4e00\u4e9b\u5370\u8c61\uff0c\u4f46\u5f53\u65f6\u6ca1\u6709\u592a\u5728\u610f\uff0c\u81f3\u4e8e\u540e\u8005\u7528redis sentinel\u505amaster failover\uff0credis sentinel\u4e5f\u662f\u4e00\u4e2aredis\u6b63\u5728\u5f00\u53d1\u4e2d\u7684\u6a21\u5757\uff0c\u6211\u4e0d\u592a\u6562\u7528\u3002<\/p>\n

    \u53e6\u5916\uff0c\u6211\u820d\u5f03\u81ea\u5df1\u7684\u8fd9\u4e2a\u521d\u6b65\u65b9\u6848\u8fd8\u6709\u4e24\u4e2a\u539f\u56e0\uff1a<\/p>\n

    1\u3001\u81ea\u5df1\u5728\u5199\u5ba2\u6237\u7aefdata sharding\u548c\u5747\u8861\u670d\u52a1\u7684\u65f6\u5019\uff0c\u53d1\u73b0\u5b9e\u9645\u9700\u8981\u8003\u8651\u7684\u95ee\u9898\u6bd4\u5f00\u59cb\u60f3\u7684\u8981\u590d\u6742\u4e00\u4e9b\uff0c\u5982\u679c\u5199\u5b8c\uff0c\u5176\u5b9e\u76f8\u5f53\u4e8e\u5c06twemproxy\u7684\u529f\u80fd\u505a\u4e86\u4e00\u904d\uff0c\u9020\u8f6e\u5b50\u7684\u4e8b\u60c5\u8fd8\u662f\u5c11\u5e72\u3002<\/p>\n

    2\u3001\u529f\u80fd\u505a\u5f97\u6709\u4e9b\u5197\u4f59\uff0c\u4e00\u6b21\u8bfb\u8bf7\u6c42\u8981\u7ecf\u8fc7\u5ba2\u6237\u7aef\u7684sharding\u3001\u7136\u540e\u8fd8\u6709\u7ecf\u8fc7lvs\u518d\u5230\u5b9e\u9645\u7684\u670d\u52a1\u5668\uff0c\u4e0d\u505a\u4f18\u5316\u7684\u8bdd\uff0c\u4f1a\u589e\u52a0\u4e0d\u5c11\u5ef6\u8fdf\u3002<\/p>\n

    \u3010\u7b2c\u4e09\u6b65\u3011\u6700\u7ec8\u7684\u65b9\u6848\uff0c\u5982\u4e0b\u56fe\u6240\u793a<\/strong><\/p>\n

    <\/p>\n

    \u56fe\u4e2d\u753b\u7684\u633a\u660e\u767d\u4e86\uff0c\u5c31\u4e0d\u518d\u591a\u89e3\u91ca\u4e86\u3002<\/p>\n

    twemproxy\u662ftwitter\u5f00\u6e90\u7684\u4e00\u4e2a\u6570\u636e\u5e93\u4ee3\u7406\u670d\u52a1\uff0c\u53ef\u4ee5\u7528\u4e8ememcached\u548credis\u7684sharding\uff0c\u517c\u5bb9\u4e8c\u8005\u7684\u6807\u51c6\u63a5\u53e3\uff0c\u4f46\u662f\u5bf9\u4e8eredis\u7684keys\uff0cdbsize\u7b49\u547d\u4ee4\u4e0d\u652f\u6301\uff0c\u8fd9\u4e2a\u5176\u5b9e\u60f3\u4e00\u4e0b\u4e5f\u5c31\u660e\u767d\u4e86\uff0c\u8fd9\u79cdpool\u5185\u8de8\u673a\u505a\u7edf\u8ba1\u7684\u547d\u4ee4proxy\u4e00\u822c\u4e0d\u4f1a\u652f\u6301\u7684\u3002\u53e6\u5916\uff0ctwemproxy\u5728\u81ea\u8eab\u4e0e\u540e\u53f0redis\u4e4b\u95f4\u4f7f\u7528pipeline\u53d1\u9001\u547d\u4ee4\uff0c\u56e0\u6b64\u6027\u80fd\u635f\u5931\u6bd4\u8f83\u5c0f\u3002\u4f46\u662f\uff0ctwemproxy\u5bf9\u4e8e\u6bcf\u4e00\u4e2a\u5ba2\u6237\u7aef\u8fde\u63a5\u5f00\u542f\u7684mbuf\u6709\u9650\uff0c\u6700\u5927\u53ef\u4ee5\u8bbe\u7f6e\u4e3a64k\uff0c\u5982\u679c\u5728\u5ba2\u6237\u7aef\u4ee3\u7406\u5c42\u4e0etwemproxy\u4e4b\u95f4\u4e5f\u4f7f\u7528pipeline\uff0c\u8fd9\u4e2apipeline\u4e0d\u80fd\u592a\u6df1\uff0c\u800c\u4e14\u4e0d\u652f\u6301pipeline\u7684\u539f\u5b50\u6027\uff08transaction\uff09\uff0c\u5176\u5b9e\uff0c\u8fd9\u4e2a\u65f6\u5019\uff0c\u76f8\u5f53\u4e8e\u5ba2\u6237\u7aef\u8fde\u63a5\u4e0eredis\u6570\u636e\u5e93\u4e4b\u95f4\u5b58\u5728\u4e24\u5c42pipeline\uff0c\u5206\u522b\u662f\u5ba2\u6237\u7aef\u5230twemproxy\u7684pipeline\uff0c\u548ctwemproy\u5230\u540e\u53f0redis\u670d\u52a1\u5668\u7684pipeline\uff0c\u7531\u4e8e\u4e8c\u8005buffer\u6df1\u5ea6\u4e0d\u4e00\u81f4\uff0c\u56e0\u6b64\u4e0d\u652f\u6301pipeline\u7684transaction\u4e5f\u5c31\u597d\u7406\u89e3\u4e86\u3002\u3002\u5728\u5f15\u5165\u4e86twemproxy\uff0c\u63d2\u5165\u5927\u89c4\u6a21\u6570\u636e\u7684\u65f6\u5019\uff0c\u6709\u65f6\u5019\u786e\u5b9e\u633a\u8017\u65f6\uff0c\u800c\u4e14pipeline\u4e0d\u4fdd\u8bc1\u539f\u5b50\u6027\uff0c\u4e22\u6570\u636e\u65f6\u7684\u6062\u590d\u95ee\u9898\u5728\u5ba2\u6237\u7aef\u9700\u8981\u8fdb\u884c\u989d\u5916\u5173\u6ce8\u3002\u5bf9\u4e8e\u975etransaction\u7684pipeline\u603b\u4e22\u6570\u636e\uff0c\u6216\u8005\u5bf9\u4e8e\u6570\u636e\u91cf\u6bd4\u8f83\u5927\u7684key\u4e00\u6b21\u6027\u53d6\u6570\u636e\u5931\u8d25\u7b49\u95ee\u9898\uff0c\u540e\u6765\u7ecf\u67e5\u662ftwemproxy\u7aeftimeou\u503c\u8bbe\u7f6e\u8fc7\u5c0f\uff0c\u6309\u7167\u5b98\u65b9\u793a\u4f8b\u8bbe\u7f6e400ms\uff0c\u4f1a\u5728\u4e00\u6b21\u6027\u64cd\u4f5c\u5927\u6570\u636e\u91cf\u7684\u65f6\u5019\u8fd4\u56detimeout\u5931\u8d25\uff0c\u8fd9\u4e2a\u6570\u503c\u9700\u8981\u614e\u91cd\u6839\u636e\u4e1a\u52a1\uff08\u5177\u4f53\u7684\uff0c\u5c31\u662f\u5ba2\u6237\u7aef\u5355\u6b21\u547d\u4ee4\u64cd\u4f5c\u7684\u6570\u636e\u91cf\uff09\u8fdb\u884c\u8bbe\u7f6e\uff0c\u4e00\u822c2000ms\u5dee\u4e0d\u591a\u5c31\u591f\u7528\u4e86\uff08\u53ef\u4ee5\u652f\u6301\u4e00\u6b21\u64cd\u4f5c\u63a5\u8fd1\u767e\u4e07\u7684\u6570\u636e\uff09\u3002<\/span><\/p>\n

    \u4e0a\u9762\u7684\u7ed3\u6784\uff0c\u5c06\u8bfb\u64cd\u4f5c\u7684\u8d1f\u8f7d\u5747\u8861\u653e\u5230\u4e86\u5ba2\u6237\u7aef\u4ee3\u7801\u6765\u505a\uff0c\u5199\u64cd\u4f5c\u63a7\u5236\u4e5f\u5728\u5ba2\u6237\u7aef\u5c42\u7684\u4ee3\u7801\u91cc\uff0c\u53e6\u5916\uff0c\u5bf9\u4e8etwemproy\u5355\u70b9\u3001\u4e3b\u4ece\u4e4b\u95f4\u53ef\u4ee5\u5f15\u5165keepalived\u6765\u6d88\u9664\u5355\u70b9\u548c\u6545\u969c\u6062\u590d\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"

    Redis\u96c6\u7fa4\u65b9\u6848 \u6536\u85cf \u672c\u7bc7\u6587...<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[101],"tags":[],"class_list":["post-201276","post","type-post","status-publish","format-standard","hentry","category-database"],"_links":{"self":[{"href":"https:\/\/server.hk\/cnblog\/wp-json\/wp\/v2\/posts\/201276","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/server.hk\/cnblog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/server.hk\/cnblog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/server.hk\/cnblog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/server.hk\/cnblog\/wp-json\/wp\/v2\/comments?post=201276"}],"version-history":[{"count":0,"href":"https:\/\/server.hk\/cnblog\/wp-json\/wp\/v2\/posts\/201276\/revisions"}],"wp:attachment":[{"href":"https:\/\/server.hk\/cnblog\/wp-json\/wp\/v2\/media?parent=201276"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/server.hk\/cnblog\/wp-json\/wp\/v2\/categories?post=201276"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/server.hk\/cnblog\/wp-json\/wp\/v2\/tags?post=201276"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}