super_read_only makes node to stop when event is received

If super_read_only is set to ON in one node, it will stop and be removed from the cluster when an event is received from other nodes (but not when queries are run in it).

How can we better deal with this situation? If setting super_read_only in any node is not supported, we shouldn't allow for it to be set.

Tested with 5.6.30-76.3-56, Percona XtraDB Cluster (GPL), Release rel76.3, Revision 64742ac, WSREP version 25.16, wsrep_25.16.

How to reproduce:

- Start a two-node cluster.

- Set global super_read_only=ON in one of them

- Issue an INSERT on it. No problems, we get an error as expected, and the cluster is still up and with all nodes.

mysql> insert into test.read_only_test values (1);
ERROR 1290 (HY000): The MySQL server is running with the --read-only (super) option so it cannot execute this statement

- Issue an INSERT on the other node (which doesn't have super_read_only=ON). The row is inserted in the node, but fails on the super_read_only node, causing it to stop with the following messages in the error log. The node is no longer part of the cluster.

2016-10-17 22:41:24 1346 [Warning] WSREP: BF applier failed to open_and_lock_tables: 1290, fatal: 0 wsrep = (exec_mode: 1 conflict_state: 0 seqno: 19)
2016-10-17 22:41:24 1346 [ERROR] Slave SQL: Error executing row event: 'The MySQL server is running with the --read-only (super) option so it cannot execute this statement', Error_code: 1290
2016-10-17 22:41:24 1346 [Warning] WSREP: RBR event 3 Write_rows apply warning: 1290, 19
2016-10-17 22:41:24 1346 [Warning] WSREP: Failed to apply app buffer: seqno: 19, status: 1
  at galera/src/trx_handle.cpp:apply():351
Retrying 2th time
2016-10-17 22:41:24 1346 [Warning] WSREP: BF applier failed to open_and_lock_tables: 1290, fatal: 0 wsrep = (exec_mode: 1 conflict_state: 0 seqno: 19)
2016-10-17 22:41:24 1346 [ERROR] Slave SQL: Error executing row event: 'The MySQL server is running with the --read-only (super) option so it cannot execute this statement', Error_code: 1290
2016-10-17 22:41:24 1346 [Warning] WSREP: RBR event 3 Write_rows apply warning: 1290, 19
2016-10-17 22:41:24 1346 [Warning] WSREP: Failed to apply app buffer: seqno: 19, status: 1
  at galera/src/trx_handle.cpp:apply():351
Retrying 3th time
2016-10-17 22:41:24 1346 [Warning] WSREP: BF applier failed to open_and_lock_tables: 1290, fatal: 0 wsrep = (exec_mode: 1 conflict_state: 0 seqno: 19)
2016-10-17 22:41:24 1346 [ERROR] Slave SQL: Error executing row event: 'The MySQL server is running with the --read-only (super) option so it cannot execute this statement', Error_code: 1290
2016-10-17 22:41:24 1346 [Warning] WSREP: RBR event 3 Write_rows apply warning: 1290, 19
2016-10-17 22:41:24 1346 [Warning] WSREP: Failed to apply app buffer: seqno: 19, status: 1
  at galera/src/trx_handle.cpp:apply():351
Retrying 4th time
2016-10-17 22:41:24 1346 [Warning] WSREP: BF applier failed to open_and_lock_tables: 1290, fatal: 0 wsrep = (exec_mode: 1 conflict_state: 0 seqno: 19)
2016-10-17 22:41:24 1346 [ERROR] Slave SQL: Error executing row event: 'The MySQL server is running with the --read-only (super) option so it cannot execute this statement', Error_code: 1290
2016-10-17 22:41:24 1346 [Warning] WSREP: RBR event 3 Write_rows apply warning: 1290, 19
2016-10-17 22:41:24 1346 [ERROR] WSREP: Failed to apply trx: source: 31920f3d-94ba-11e6-81ab-2a2cc8113fcb version: 3 local: 0 state: APPLYING flags: 1 conn_id: 4 trx_id: 4875 seqnos (l: 7, g: 19, s: 18, d: 18, ts: 355927415497)
2016-10-17 22:41:24 1346 [ERROR] WSREP: Failed to apply trx 19 4 times
2016-10-17 22:41:24 1346 [ERROR] WSREP: Node consistency compromized, aborting...
2016-10-17 22:41:24 1346 [Note] WSREP: Closing send monitor...
2016-10-17 22:41:24 1346 [Note] WSREP: Closed send monitor.
2016-10-17 22:41:24 1346 [Note] WSREP: gcomm: terminating thread
2016-10-17 22:41:24 1346 [Note] WSREP: gcomm: joining thread
2016-10-17 22:41:24 1346 [Note] WSREP: gcomm: closing backend
2016-10-17 22:41:25 1346 [Note] WSREP: view(view_id(NON_PRIM,31920f3d,2) memb {
 55b0a271,0
} joined {
} left {
} partitioned {
 31920f3d,0
})
2016-10-17 22:41:25 1346 [Note] WSREP: New COMPONENT: primary = no, bootstrap = no, my_idx = 0, memb_num = 1
2016-10-17 22:41:25 1346 [Note] WSREP: view((empty))
2016-10-17 22:41:25 1346 [Note] WSREP: gcomm: closed
2016-10-17 22:41:25 1346 [Note] WSREP: Flow-control interval: [16, 16]
2016-10-17 22:41:25 1346 [Note] WSREP: Received NON-PRIMARY.
2016-10-17 22:41:25 1346 [Note] WSREP: Shifting SYNCED -> OPEN (TO: 19)
2016-10-17 22:41:25 1346 [Note] WSREP: Received self-leave message.
2016-10-17 22:41:25 1346 [Note] WSREP: Flow-control interval: [0, 0]
2016-10-17 22:41:25 1346 [Note] WSREP: Received SELF-LEAVE. Closing connection.
2016-10-17 22:41:25 1346 [Note] WSREP: Shifting OPEN -> CLOSED (TO: 19)
2016-10-17 22:41:25 1346 [Note] WSREP: RECV thread exiting 0: Success
2016-10-17 22:41:25 1346 [Note] WSREP: recv_thread() joined.
2016-10-17 22:41:25 1346 [Note] WSREP: Closing replication queue.
2016-10-17 22:41:25 1346 [Note] WSREP: Closing slave action queue.
2016-10-17 22:41:25 1346 [Note] WSREP: /usr/sbin/mysqld: Terminated.
161017 22:41:25 mysqld_safe Number of processes running now: 0
161017 22:41:25 mysqld_safe WSREP: not restarting wsrep node automatically
161017 22:41:25 mysqld_safe mysqld from pid file /var/lib/mysql/node-2.pid ended