I have an existing PostgreSQL cluster v11.5 with streaming replication.
I want to set up high availability managed by Patroni.
I installed Patroni on both machines, I set up an etcd cluster on three others Virtuals Machines ['http://10.A.A.191:2379', 'http://10.B.B.99:2379', 'http://10.C.C.170:2379'] but Patroni starting failed with systemctl.
When I start Patroni with patroni -c /etc/patroni.yml
command, I have the errors below:
2019-12-26 16:51:38,207 DEBUG: Selection next machine in cache. Available machines: ['http://10.A.A.191:2379', 'http://10.B.B.99:2379', 'http://10.C.C.170:2379']
2019-12-26 16:51:38,207 INFO: Selected new etcd server http://10.C.C.170:2379
2019-12-26 16:51:38,208 DEBUG: Converted retries value: 0 -> Retry(total=0, connect=None, read=None, redirect=0, status=None)
2019-12-26 16:51:38,208 DEBUG: Starting new HTTP connection (1): 10.C.C.170:2379
2019-12-26 16:51:38,210 DEBUG: http://10.C.C.170:2379 "GET /v2/machines HTTP/1.1" 404 19
2019-12-26 16:51:38,211 ERROR: Failed to get list of machines from http://10.C.C.170:2379/v2: EtcdException('Bad response : 404 page not found\n',)
2019-12-26 16:51:38,211 INFO: Retrying on http://10.A.A.191:2379
2019-12-26 16:51:38,211 DEBUG: Converted retries value: 0 -> Retry(total=0, connect=None, read=None, redirect=0, status=None)
2019-12-26 16:51:38,211 DEBUG: Starting new HTTP connection (1): 10.237.132.191:2379
2019-12-26 16:51:38,214 DEBUG: http://10.A.A.191:2379 "GET /v2/machines HTTP/1.1" 404 19
2019-12-26 16:51:38,214 ERROR: Failed to get list of machines from http://10.A.A.191:2379/v2: EtcdException('Bad response : 404 page not found\n',)
2019-12-26 16:51:38,214 INFO: Retrying on http://10.B.B.99:2379
2019-12-26 16:51:38,215 DEBUG: Converted retries value: 0 -> Retry(total=0, connect=None, read=None, redirect=0, status=None)
2019-12-26 16:51:38,215 DEBUG: Starting new HTTP connection (1): 10.B.B.99:2379
2019-12-26 16:51:38,217 DEBUG: http://10.B.B.99:2379 "GET /v2/machines HTTP/1.1" 404 19
2019-12-26 16:51:38,217 ERROR: Failed to get list of machines from http://10.B.B.99:2379/v2: EtcdException('Bad response : 404 page not found\n',)
2019-12-26 16:51:38,217 INFO: waiting on etcd
The problem seems to be at API level DEBUG: http://10.B.B.99:2379 "GET /v2/machines HTTP/1.1" 404 19
etcd is reacheable from PostgreSQL host:
curl -s http://10.B.B.99:2380/members |jq -r
returns
[
{
"id": 730XXXXXXXXXXX000,
"peerURLs": [
"http://10.B.B.99:2380"
],
"name": "etcd1",
"clientURLs": [
"http://0.0.0.0:2379"
]
},
{
"id": 956XXXXXXXXXXX000,
"peerURLs": [
"http://10.C.C.170:2380"
],
"name": "etcd3",
"clientURLs": [
"http://0.0.0.0:2379"
]
},
{
"id": 1551XXXXXXXXXXX000,
"peerURLs": [
"http://10.A.A.191:2380"
],
"name": "etcd2",
"clientURLs": [
"http://0.0.0.0:2379"
]
}
]
etcd1 start command:
ExecStart=/usr/local/bin/etcd \
--name etcd1 \
--advertise-client-urls http://0.0.0.0:2379 \
--listen-client-urls http://0.0.0.0:2379 \
--initial-advertise-peer-urls http://10.B.B.99:2380 \
--listen-peer-urls http://0.0.0.0:2380 \
--initial-cluster etcd1=http://10.B.B.99:2380,etcd2=http://10.A.A.191:2380,etcd3=http://10.C.C.170:2380 \
--initial-cluster-token etcd-cluster-atom \
--initial-cluster-state new \
--logger zap \
--log-outputs stdout \
--log-level debug
Someone could help me please?
What is wrong?
Many thanks
Versions:
etcd Version: 3.4.0
etcdctl version: 3.4.0
API version: 3.4
patroni 1.6.3
python-etcd==0.4.5
Best Answer
Thanks to CyberDem0n.
Two issues in etcd1 start command:
API VERSION
Etcd v3.4 default API version is v3.
Patroni doesn't currently support the v3 API.
Etcd 3.4 turned API v2 off by default.
I added --enable-v2=true flag on the three etcd.service and this error was fixed.
advertise-client-urls FLAG
And for each etcd server I replaced 0.0.0.0 by the real etcd server ip on the
--advertise-client-urls
flag (like--advertise-client-urls http://10.B.B.99:2379
in the example).