* [Drbd-dev] Here's logs 3 volumes case
@ 2016-08-16 1:14 박경민
2016-08-18 11:51 ` Lars Ellenberg
0 siblings, 1 reply; 2+ messages in thread
From: 박경민 @ 2016-08-16 1:14 UTC (permalink / raw)
To: drbd-dev
[-- Attachment #1: Type: text/plain, Size: 56668 bytes --]
There're 3 node, and each had 100MB/100MB/95MB volume, time synced
Please ,Watch DRBD9_3 node's log carefully
Thanks
<DRBD9_1>
...
Aug 16 09:32:30 DRBD9_1 kernel: drbd r0/0 drbd1: current_size: 204800
Aug 16 09:32:30 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: c_size: 194560
u_size: 0 d_size: 194560 max_size: 194560
Aug 16 09:32:30 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: la_size: 0
my_usize: 0 my_max_size: 204800
Aug 16 09:32:30 DRBD9_1 kernel: drbd r0/0 drbd1: my node_id: 0
Aug 16 09:32:30 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: node_id: 1 idx: 0
bm-uuid: 0x0 flags: 0x0 max_size: 0 (DUnknown)
Aug 16 09:32:30 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: node_id: 2 idx: 1
bm-uuid: 0x0 flags: 0x0 max_size: 194560 (DUnknown)
Aug 16 09:32:30 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: calling
drbd_determine_dev_size()
Aug 16 09:32:30 DRBD9_1 kernel: drbd r0/0 drbd1: my node_id: 0
Aug 16 09:32:30 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: node_id: 1 idx: 0
bm-uuid: 0x0 flags: 0x0 max_size: 0 (DUnknown)
Aug 16 09:32:30 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: node_id: 2 idx: 1
bm-uuid: 0x0 flags: 0x0 max_size: 194560 (DUnknown)
Aug 16 09:32:30 DRBD9_1 kernel: drbd r0/0 drbd1: drbd_bm_resize called with
capacity == 194560
Aug 16 09:32:30 DRBD9_1 kernel: drbd r0/0 drbd1: resync bitmap: bits=24320
words=760 pages=2
Aug 16 09:32:30 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3:
drbd_sync_handshake:
Aug 16 09:32:30 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: self
0000000000000004:0000000000000000:0000000000000000:0000000000000000
bits:1280 flags:0
Aug 16 09:32:30 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: peer
0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:24
Aug 16 09:32:30 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: uuid_compare()=0
by rule 10
Aug 16 09:32:30 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: pdsk( DUnknown ->
Inconsistent ) repl( Off -> Established )
Aug 16 09:32:34 DRBD9_1 kernel: drbd r0 DRBD9_2: Handshake to peer 1
successful: Agreed network protocol version 111
Aug 16 09:32:34 DRBD9_1 kernel: drbd r0 DRBD9_2: Feature flags enabled on
protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
Aug 16 09:32:34 DRBD9_1 kernel: drbd r0 DRBD9_2: Starting ack_recv thread
(from drbd_r_r0 [97862])
Aug 16 09:32:34 DRBD9_1 kernel: drbd r0 DRBD9_2: sock was shut down by peer
Aug 16 09:32:34 DRBD9_1 kernel: drbd r0 DRBD9_2: conn( Connecting ->
BrokenPipe )
Aug 16 09:32:34 DRBD9_1 kernel: drbd r0 DRBD9_2: ack_receiver terminated
Aug 16 09:32:34 DRBD9_1 kernel: drbd r0 DRBD9_2: Terminating ack_recv thread
Aug 16 09:32:34 DRBD9_1 kernel: drbd r0 DRBD9_2: Connection closed
Aug 16 09:32:34 DRBD9_1 kernel: drbd r0 DRBD9_2: conn( BrokenPipe ->
Unconnected )
Aug 16 09:32:34 DRBD9_1 kernel: drbd r0 DRBD9_2: Restarting receiver thread
Aug 16 09:32:34 DRBD9_1 kernel: drbd r0 DRBD9_2: conn( Unconnected ->
Connecting )
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0 DRBD9_2: Handshake to peer 1
successful: Agreed network protocol version 111
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0 DRBD9_2: Feature flags enabled on
protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0 DRBD9_2: Starting ack_recv thread
(from drbd_r_r0 [97862])
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0 DRBD9_2: sock was shut down by peer
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0 DRBD9_2: conn( Connecting ->
BrokenPipe )
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0 DRBD9_2: ack_receiver terminated
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0 DRBD9_2: Terminating ack_recv thread
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0 DRBD9_2: Connection closed
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0 DRBD9_2: conn( BrokenPipe ->
Unconnected )
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0 DRBD9_2: Restarting receiver thread
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0 DRBD9_2: conn( Unconnected ->
Connecting )
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0 DRBD9_2: Handshake to peer 1
successful: Agreed network protocol version 111
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0 DRBD9_2: Feature flags enabled on
protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0 DRBD9_2: Starting ack_recv thread
(from drbd_r_r0 [97862])
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0: Preparing cluster-wide state
change 369176076 (0->1 499/146)
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0: State change 369176076:
primary_nodes=0, weak_nodes=0
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0: Committing cluster-wide state
change 369176076 (1ms)
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0 DRBD9_2: conn( Connecting ->
Connected ) peer( Unknown -> Secondary )
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0/0 drbd1: current_size: 194560
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: c_size: 194560
u_size: 0 d_size: 204800 max_size: 204800
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: la_size: 0
my_usize: 0 my_max_size: 204800
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0/0 drbd1: my node_id: 0
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: node_id: 1 idx: 0
bm-uuid: 0x0 flags: 0x0 max_size: 204800 (DUnknown)
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: node_id: 2 idx: 1
bm-uuid: 0x0 flags: 0x13 max_size: 194560 (Inconsistent)
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: calling
drbd_determine_dev_size()
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0/0 drbd1: my node_id: 0
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: node_id: 1 idx: 0
bm-uuid: 0x0 flags: 0x0 max_size: 204800 (DUnknown)
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: node_id: 2 idx: 1
bm-uuid: 0x0 flags: 0x13 max_size: 194560 (Inconsistent)
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2:
drbd_sync_handshake:
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: self
0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:0
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: peer
0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:24
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: uuid_compare()=0
by rule 10
Aug 16 09:32:36 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: pdsk( DUnknown ->
Inconsistent ) repl( Off -> Established )
Aug 16 09:32:56 DRBD9_1 kernel: drbd r0 DRBD9_3: Preparing remote state
change 1953992009 (primary_nodes=0, weak_nodes=0)
Aug 16 09:32:56 DRBD9_1 kernel: drbd r0 DRBD9_3: Committing remote state
change 1953992009
Aug 16 09:32:56 DRBD9_1 kernel: drbd r0 DRBD9_3: peer( Secondary -> Primary
)
Aug 16 09:32:56 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: pdsk( Inconsistent
-> UpToDate )
Aug 16 09:32:56 DRBD9_1 kernel: drbd r0/0 drbd1: size = 95 MB (97280 KB)
Aug 16 09:32:56 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3:
drbd_sync_handshake:
Aug 16 09:32:56 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: self
0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:0
Aug 16 09:32:56 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: peer
D43B0300042F33B9:FFFFFFFFFFFFFFFF:0000000000000000:0000000000000000 bits:0
flags:120
Aug 16 09:32:56 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: uuid_compare()=-3
by rule 20
Aug 16 09:32:56 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: Writing the whole
bitmap, full sync required after drbd_sync_handshake.
Aug 16 09:32:56 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: repl( Established
-> WFBitMapT )
Aug 16 09:32:56 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: receive bitmap
stats [Bytes(packets)]: plain 0(0), RLE 21(1), total 21; compression: 99.4%
Aug 16 09:32:56 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: send bitmap stats
[Bytes(packets)]: plain 0(0), RLE 21(1), total 21; compression: 99.4%
Aug 16 09:32:56 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: helper command:
/sbin/drbdadm before-resync-target
Aug 16 09:32:56 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: helper command:
/sbin/drbdadm before-resync-target exit code 0 (0x0)
Aug 16 09:32:56 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: repl( WFBitMapT ->
SyncTarget )
Aug 16 09:32:56 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: resync-susp( no ->
connection dependency )
Aug 16 09:32:56 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: Began resync as
SyncTarget (will sync 97280 KB [24320 bits set]).
Aug 16 09:33:12 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: Resync done (total
15 sec; paused 0 sec; 6484 K/sec)
Aug 16 09:33:12 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: updated UUIDs
D43B0300042F33B8:0000000000000000:0000000000000000:0000000000000000
Aug 16 09:33:12 DRBD9_1 kernel: drbd r0/0 drbd1: disk( Inconsistent ->
UpToDate )
Aug 16 09:33:12 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: repl( SyncTarget
-> Established )
Aug 16 09:33:12 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: helper command:
/sbin/drbdadm after-resync-target
Aug 16 09:33:12 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: helper command:
/sbin/drbdadm after-resync-target exit code 0 (0x0)
Aug 16 09:33:12 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: updated UUIDs
D43B0300042F33B8:0000000000000000:0000000000000000:0000000000000000
Aug 16 09:33:12 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: pdsk( Inconsistent
-> UpToDate ) resync-susp( peer -> no )
Aug 16 09:33:22 DRBD9_1 kernel: drbd r0 DRBD9_3: peer( Primary -> Secondary
)
Aug 16 09:33:26 DRBD9_1 kernel: drbd r0 DRBD9_3: Preparing remote state
change 3319188407 (primary_nodes=0, weak_nodes=0)
Aug 16 09:33:26 DRBD9_1 kernel: drbd r0 DRBD9_3: Committing remote state
change 3319188407
Aug 16 09:33:26 DRBD9_1 kernel: drbd r0 DRBD9_3: conn( Connected ->
TearDown ) peer( Secondary -> Unknown )
Aug 16 09:33:26 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: pdsk( UpToDate ->
DUnknown ) repl( Established -> Off )
Aug 16 09:33:26 DRBD9_1 kernel: drbd r0 DRBD9_3: ack_receiver terminated
Aug 16 09:33:26 DRBD9_1 kernel: drbd r0 DRBD9_3: Terminating ack_recv thread
Aug 16 09:33:26 DRBD9_1 kernel: drbd r0 DRBD9_3: Connection closed
Aug 16 09:33:26 DRBD9_1 kernel: drbd r0 DRBD9_3: conn( TearDown ->
Unconnected )
Aug 16 09:33:26 DRBD9_1 kernel: drbd r0 DRBD9_3: Restarting receiver thread
Aug 16 09:33:26 DRBD9_1 kernel: drbd r0 DRBD9_3: conn( Unconnected ->
Connecting )
Aug 16 09:33:27 DRBD9_1 kernel: drbd r0 DRBD9_2: Preparing remote state
change 2471101321 (primary_nodes=0, weak_nodes=0)
Aug 16 09:33:27 DRBD9_1 kernel: drbd r0 DRBD9_2: Committing remote state
change 2471101321
Aug 16 09:33:33 DRBD9_1 kernel: drbd r0 DRBD9_2: Preparing remote state
change 4086777357 (primary_nodes=0, weak_nodes=0)
Aug 16 09:33:33 DRBD9_1 kernel: drbd r0 DRBD9_2: Committing remote state
change 4086777357
Aug 16 09:33:33 DRBD9_1 kernel: drbd r0 DRBD9_2: conn( Connected ->
TearDown ) peer( Secondary -> Unknown )
Aug 16 09:33:33 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: pdsk( UpToDate ->
DUnknown ) repl( Established -> Off )
Aug 16 09:33:33 DRBD9_1 kernel: drbd r0 DRBD9_2: ack_receiver terminated
Aug 16 09:33:33 DRBD9_1 kernel: drbd r0 DRBD9_2: Terminating ack_recv thread
Aug 16 09:33:33 DRBD9_1 kernel: drbd r0 DRBD9_2: Connection closed
Aug 16 09:33:33 DRBD9_1 kernel: drbd r0 DRBD9_2: conn( TearDown ->
Unconnected )
Aug 16 09:33:33 DRBD9_1 kernel: drbd r0 DRBD9_2: Restarting receiver thread
Aug 16 09:33:33 DRBD9_1 kernel: drbd r0 DRBD9_2: conn( Unconnected ->
Connecting )
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0 DRBD9_2: Handshake to peer 1
successful: Agreed network protocol version 111
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0 DRBD9_2: Feature flags enabled on
protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0 DRBD9_2: Starting ack_recv thread
(from drbd_r_r0 [97862])
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0: Preparing cluster-wide state
change 1975720708 (0->1 499/146)
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0: State change 1975720708:
primary_nodes=0, weak_nodes=0
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0: Committing cluster-wide state
change 1975720708 (1ms)
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0 DRBD9_2: conn( Connecting ->
Connected ) peer( Unknown -> Secondary )
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1: current_size: 194560
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: c_size: 194560
u_size: 0 d_size: 204800 max_size: 204800
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: la_size: 194560
my_usize: 0 my_max_size: 204800
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1: my node_id: 0
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: node_id: 1 idx: 0
bm-uuid: 0x0 flags: 0x10 max_size: 204800 (DUnknown)
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: node_id: 2 idx: 1
bm-uuid: 0x0 flags: 0x10 max_size: 194560 (DUnknown)
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1: Resize forced while not
fully connected!
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: calling
drbd_determine_dev_size()
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1: my node_id: 0
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: node_id: 1 idx: 0
bm-uuid: 0x0 flags: 0x10 max_size: 204800 (DUnknown)
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: node_id: 2 idx: 1
bm-uuid: 0x0 flags: 0x10 max_size: 194560 (DUnknown)
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1: Resize forced while not
fully connected!
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1: drbd_bm_resize called with
capacity == 204800
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1: resync bitmap: bits=25600
words=800 pages=2
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1: size = 100 MB (102400 KB)
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1: Writing the whole bitmap,
size changed
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2:
drbd_sync_handshake:
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: self
D43B0300042F33B8:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:0
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: peer
D43B0300042F33B8:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:20
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: uuid_compare()=0
by rule 38
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1: No resync, but 1280 bits
in bitmap!
Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: pdsk( DUnknown ->
UpToDate ) repl( Off -> Established )
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0 DRBD9_3: Handshake to peer 2
successful: Agreed network protocol version 111
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0 DRBD9_3: Feature flags enabled on
protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0 DRBD9_3: Starting ack_recv thread
(from drbd_r_r0 [97863])
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0: Preparing cluster-wide state
change 550148406 (0->2 499/146)
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0: State change 550148406:
primary_nodes=0, weak_nodes=0
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0: Committing cluster-wide state
change 550148406 (0ms)
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0 DRBD9_3: conn( Connecting ->
Connected ) peer( Unknown -> Secondary )
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0/0 drbd1: current_size: 204800
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: c_size: 194560
u_size: 0 d_size: 194560 max_size: 194560
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: la_size: 204800
my_usize: 0 my_max_size: 204800
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0/0 drbd1: my node_id: 0
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: node_id: 1 idx: 0
bm-uuid: 0x0 flags: 0x11 max_size: 204800 (UpToDate)
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: node_id: 2 idx: 1
bm-uuid: 0x0 flags: 0x10 max_size: 194560 (DUnknown)
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: The peer's disk
size is too small! (194560 < 204800 sectors)
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0 DRBD9_3: conn( Connected ->
Disconnecting ) peer( Secondary -> Unknown )
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0 DRBD9_3: error receiving P_SIZES,
e: -5 l: 0!
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0 DRBD9_3: ack_receiver terminated
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0 DRBD9_3: Terminating ack_recv thread
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0 DRBD9_2: Preparing remote state
change 1936541420 (primary_nodes=0, weak_nodes=0)
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0 DRBD9_3: Connection closed
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0 DRBD9_3: conn( Disconnecting ->
StandAlone )
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0 DRBD9_3: Terminating receiver thread
Aug 16 09:33:46 DRBD9_1 kernel: drbd r0 DRBD9_2: Committing remote state
change 1936541420
<DRBD9_2>
...
Aug 16 09:32:27 DRBD9_2 kernel: drbd r0/0 drbd1: current_size: 204800
Aug 16 09:32:27 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: c_size: 194560
u_size: 0 d_size: 194560 max_size: 194560
Aug 16 09:32:27 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: la_size: 0
my_usize: 0 my_max_size: 204800
Aug 16 09:32:27 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: node_id: 0 idx: 0
bm-uuid: 0x0 flags: 0x0 max_size: 0 (DUnknown)
Aug 16 09:32:27 DRBD9_2 kernel: drbd r0/0 drbd1: my node_id: 1
Aug 16 09:32:27 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: node_id: 2 idx: 1
bm-uuid: 0x0 flags: 0x0 max_size: 194560 (DUnknown)
Aug 16 09:32:27 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: calling
drbd_determine_dev_size()
Aug 16 09:32:27 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: node_id: 0 idx: 0
bm-uuid: 0x0 flags: 0x0 max_size: 0 (DUnknown)
Aug 16 09:32:27 DRBD9_2 kernel: drbd r0/0 drbd1: my node_id: 1
Aug 16 09:32:27 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: node_id: 2 idx: 1
bm-uuid: 0x0 flags: 0x0 max_size: 194560 (DUnknown)
Aug 16 09:32:27 DRBD9_2 kernel: drbd r0/0 drbd1: drbd_bm_resize called with
capacity == 194560
Aug 16 09:32:27 DRBD9_2 kernel: drbd r0/0 drbd1: resync bitmap: bits=24320
words=760 pages=2
Aug 16 09:32:27 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3:
drbd_sync_handshake:
Aug 16 09:32:27 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: self
0000000000000004:0000000000000000:0000000000000000:0000000000000000
bits:1280 flags:0
Aug 16 09:32:27 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: peer
0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:24
Aug 16 09:32:27 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: uuid_compare()=0
by rule 10
Aug 16 09:32:27 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: pdsk( DUnknown ->
Inconsistent ) repl( Off -> Established )
Aug 16 09:32:30 DRBD9_2 kernel: drbd r0 DRBD9_3: Preparing remote state
change 1968584032 (primary_nodes=0, weak_nodes=0)
Aug 16 09:32:30 DRBD9_2 kernel: drbd r0 DRBD9_3: Committing remote state
change 1968584032
Aug 16 09:32:30 DRBD9_2 kernel: drbd r0/0 drbd1: current_size: 194560
Aug 16 09:32:30 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: c_size: 194560
u_size: 0 d_size: 194560 max_size: 194560
Aug 16 09:32:30 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: la_size: 0
my_usize: 0 my_max_size: 204800
Aug 16 09:32:30 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: node_id: 0 idx: 0
bm-uuid: 0x0 flags: 0x0 max_size: 0 (DUnknown)
Aug 16 09:32:30 DRBD9_2 kernel: drbd r0/0 drbd1: my node_id: 1
Aug 16 09:32:30 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: node_id: 2 idx: 1
bm-uuid: 0x0 flags: 0x13 max_size: 194560 (Inconsistent)
Aug 16 09:32:30 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: calling
drbd_determine_dev_size()
Aug 16 09:32:30 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: node_id: 0 idx: 0
bm-uuid: 0x0 flags: 0x0 max_size: 0 (DUnknown)
Aug 16 09:32:30 DRBD9_2 kernel: drbd r0/0 drbd1: my node_id: 1
Aug 16 09:32:30 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: node_id: 2 idx: 1
bm-uuid: 0x0 flags: 0x13 max_size: 194560 (Inconsistent)
Aug 16 09:32:32 DRBD9_2 kernel: drbd r0 DRBD9_1: sock_recvmsg returned -11
Aug 16 09:32:32 DRBD9_2 kernel: drbd r0 DRBD9_1: conn( Connecting ->
BrokenPipe )
Aug 16 09:32:32 DRBD9_2 kernel: drbd r0 DRBD9_1: short read (expected size
8)
Aug 16 09:32:32 DRBD9_2 kernel: drbd r0 DRBD9_1: Connection closed
Aug 16 09:32:32 DRBD9_2 kernel: drbd r0 DRBD9_1: conn( BrokenPipe ->
Unconnected )
Aug 16 09:32:33 DRBD9_2 kernel: drbd r0 DRBD9_1: conn( Unconnected ->
Connecting )
Aug 16 09:32:34 DRBD9_2 kernel: drbd r0 DRBD9_1: sock was reset by peer
Aug 16 09:32:34 DRBD9_2 kernel: drbd r0 DRBD9_1: conn( Connecting ->
BrokenPipe )
Aug 16 09:32:34 DRBD9_2 kernel: drbd r0 DRBD9_1: short read (expected size
8)
Aug 16 09:32:34 DRBD9_2 kernel: drbd r0 DRBD9_1: Connection closed
Aug 16 09:32:34 DRBD9_2 kernel: drbd r0 DRBD9_1: conn( BrokenPipe ->
Unconnected )
Aug 16 09:32:35 DRBD9_2 kernel: drbd r0 DRBD9_1: conn( Unconnected ->
Connecting )
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0 DRBD9_1: Handshake to peer 0
successful: Agreed network protocol version 111
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0 DRBD9_1: Feature flags enabled on
protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0 DRBD9_1: Starting ack_recv thread
(from drbd_r_r0 [110256])
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0 DRBD9_1: conn( Connecting ->
NetworkFailure )
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0 DRBD9_1: ack_receiver terminated
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0 DRBD9_1: Terminating ack_recv thread
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0 DRBD9_1: Connection closed
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0 DRBD9_1: conn( NetworkFailure ->
Unconnected )
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0 DRBD9_1: Restarting receiver thread
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0 DRBD9_1: conn( Unconnected ->
Connecting )
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0 DRBD9_1: Handshake to peer 0
successful: Agreed network protocol version 111
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0 DRBD9_1: Feature flags enabled on
protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0 DRBD9_1: Starting ack_recv thread
(from drbd_r_r0 [110256])
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0 DRBD9_1: Preparing remote state
change 369176076 (primary_nodes=0, weak_nodes=0)
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0 DRBD9_1: Committing remote state
change 369176076
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0 DRBD9_1: conn( Connecting ->
Connected ) peer( Unknown -> Secondary )
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0/0 drbd1: current_size: 194560
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: c_size: 194560
u_size: 0 d_size: 204800 max_size: 204800
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: la_size: 0
my_usize: 0 my_max_size: 204800
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: node_id: 0 idx: 0
bm-uuid: 0x0 flags: 0x0 max_size: 204800 (DUnknown)
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0/0 drbd1: my node_id: 1
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: node_id: 2 idx: 1
bm-uuid: 0x0 flags: 0x13 max_size: 194560 (Inconsistent)
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: calling
drbd_determine_dev_size()
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: node_id: 0 idx: 0
bm-uuid: 0x0 flags: 0x0 max_size: 204800 (DUnknown)
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0/0 drbd1: my node_id: 1
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: node_id: 2 idx: 1
bm-uuid: 0x0 flags: 0x13 max_size: 194560 (Inconsistent)
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1:
drbd_sync_handshake:
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: self
0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:0
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: peer
0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:24
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: uuid_compare()=0
by rule 10
Aug 16 09:32:36 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: pdsk( DUnknown ->
Inconsistent ) repl( Off -> Established )
Aug 16 09:32:56 DRBD9_2 kernel: drbd r0 DRBD9_3: Preparing remote state
change 1953992009 (primary_nodes=0, weak_nodes=0)
Aug 16 09:32:56 DRBD9_2 kernel: drbd r0 DRBD9_3: Committing remote state
change 1953992009
Aug 16 09:32:56 DRBD9_2 kernel: drbd r0 DRBD9_3: peer( Secondary -> Primary
)
Aug 16 09:32:56 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: pdsk( Inconsistent
-> UpToDate )
Aug 16 09:32:56 DRBD9_2 kernel: drbd r0/0 drbd1: size = 95 MB (97280 KB)
Aug 16 09:32:56 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3:
drbd_sync_handshake:
Aug 16 09:32:56 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: self
0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:0
Aug 16 09:32:56 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: peer
D43B0300042F33B9:FFFFFFFFFFFFFFFF:0000000000000000:0000000000000000 bits:0
flags:120
Aug 16 09:32:56 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: uuid_compare()=-3
by rule 20
Aug 16 09:32:56 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: Writing the whole
bitmap, full sync required after drbd_sync_handshake.
Aug 16 09:32:56 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: repl( Established
-> WFBitMapT )
Aug 16 09:32:56 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: receive bitmap
stats [Bytes(packets)]: plain 0(0), RLE 21(1), total 21; compression: 99.4%
Aug 16 09:32:56 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: send bitmap stats
[Bytes(packets)]: plain 0(0), RLE 21(1), total 21; compression: 99.4%
Aug 16 09:32:56 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: helper command:
/sbin/drbdadm before-resync-target
Aug 16 09:32:56 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: helper command:
/sbin/drbdadm before-resync-target exit code 0 (0x0)
Aug 16 09:32:56 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: repl( WFBitMapT ->
SyncTarget )
Aug 16 09:32:56 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: resync-susp( no ->
connection dependency )
Aug 16 09:32:56 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: Began resync as
SyncTarget (will sync 97280 KB [24320 bits set]).
Aug 16 09:33:12 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: pdsk( Inconsistent
-> UpToDate )
Aug 16 09:33:12 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: Resync done (total
15 sec; paused 0 sec; 6484 K/sec)
Aug 16 09:33:12 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: updated UUIDs
D43B0300042F33B8:0000000000000000:0000000000000000:0000000000000000
Aug 16 09:33:12 DRBD9_2 kernel: drbd r0/0 drbd1: disk( Inconsistent ->
UpToDate )
Aug 16 09:33:12 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: repl( SyncTarget
-> Established )
Aug 16 09:33:12 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: resync-susp(
connection dependency -> no )
Aug 16 09:33:12 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: helper command:
/sbin/drbdadm after-resync-target
Aug 16 09:33:12 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: helper command:
/sbin/drbdadm after-resync-target exit code 0 (0x0)
Aug 16 09:33:22 DRBD9_2 kernel: drbd r0 DRBD9_3: peer( Primary -> Secondary
)
Aug 16 09:33:26 DRBD9_2 kernel: drbd r0 DRBD9_3: Preparing remote state
change 3319188407 (primary_nodes=0, weak_nodes=0)
Aug 16 09:33:26 DRBD9_2 kernel: drbd r0 DRBD9_3: Committing remote state
change 3319188407
Aug 16 09:33:27 DRBD9_2 kernel: drbd r0 DRBD9_3: Preparing remote state
change 2471101321 (primary_nodes=0, weak_nodes=0)
Aug 16 09:33:27 DRBD9_2 kernel: drbd r0 DRBD9_3: Committing remote state
change 2471101321
Aug 16 09:33:27 DRBD9_2 kernel: drbd r0 DRBD9_3: conn( Connected ->
TearDown ) peer( Secondary -> Unknown )
Aug 16 09:33:27 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: pdsk( UpToDate ->
DUnknown ) repl( Established -> Off )
Aug 16 09:33:27 DRBD9_2 kernel: drbd r0 DRBD9_3: ack_receiver terminated
Aug 16 09:33:27 DRBD9_2 kernel: drbd r0 DRBD9_3: Terminating ack_recv thread
Aug 16 09:33:27 DRBD9_2 kernel: drbd r0 DRBD9_3: Connection closed
Aug 16 09:33:27 DRBD9_2 kernel: drbd r0 DRBD9_3: conn( TearDown ->
Unconnected )
Aug 16 09:33:27 DRBD9_2 kernel: drbd r0 DRBD9_3: Restarting receiver thread
Aug 16 09:33:27 DRBD9_2 kernel: drbd r0 DRBD9_3: conn( Unconnected ->
Connecting )
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0: Preparing cluster-wide state
change 4086777357 (1->0 496/16)
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0: State change 4086777357:
primary_nodes=0, weak_nodes=0
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0 DRBD9_1: Cluster is now split
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0: Committing cluster-wide state
change 4086777357 (0ms)
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0 DRBD9_1: conn( Connected ->
Disconnecting ) peer( Secondary -> Unknown )
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: pdsk( UpToDate ->
DUnknown ) repl( Established -> Off )
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0 DRBD9_1: ack_receiver terminated
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0 DRBD9_1: Terminating ack_recv thread
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0 DRBD9_1: Connection closed
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0 DRBD9_1: conn( Disconnecting ->
StandAlone )
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0 DRBD9_1: Terminating receiver thread
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0 DRBD9_1: Terminating sender thread
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0 tcp:DRBD9_3: Closing unexpected
connection from 200.200.200.61
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0 DRBD9_3: conn( Connecting ->
Disconnecting )
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0 DRBD9_3: Connection closed
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0 DRBD9_3: conn( Disconnecting ->
StandAlone )
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0 DRBD9_3: Terminating receiver thread
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0 DRBD9_3: Terminating sender thread
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0/0 drbd1: disk( UpToDate ->
Detaching )
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0/0 drbd1: disk( Detaching ->
Diskless )
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0/0 drbd1: drbd_bm_resize called with
capacity == 0
Aug 16 09:33:33 DRBD9_2 kernel: drbd r0: Terminating worker thread
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0: Starting worker thread (from
drbdsetup [110439])
Aug 16 09:33:38 DRBD9_2 systemd-udevd: error: /dev/drbd1: Wrong medium type
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0/0 drbd1: disk( Diskless ->
Attaching )
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0/0 drbd1: Maximum number of peer
devices = 2
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0: Method to ensure write ordering:
flush
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0/0 drbd1: node_id: 0 idx: 0 bm-uuid:
0x0 flags: 0x10 (not currently reachable)
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0/0 drbd1: my node_id: 1
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0/0 drbd1: node_id: 2 idx: 1 bm-uuid:
0x0 flags: 0x10 (not currently reachable)
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0/0 drbd1: Adjusting my ra_pages to
backing device's (32 -> 1024)
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0/0 drbd1: node_id: 0 idx: 0 bm-uuid:
0x0 flags: 0x10 (not currently reachable)
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0/0 drbd1: my node_id: 1
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0/0 drbd1: node_id: 2 idx: 1 bm-uuid:
0x0 flags: 0x10 (not currently reachable)
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0/0 drbd1: drbd_bm_resize called with
capacity == 194560
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0/0 drbd1: resync bitmap: bits=24320
words=760 pages=2
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0/0 drbd1: size = 95 MB (97280 KB)
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0/0 drbd1: recounting of set bits
took additional 0ms
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0/0 drbd1: disk( Attaching ->
UpToDate )
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0/0 drbd1: attached to current UUID:
D43B0300042F33B8
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0 DRBD9_1: Starting sender thread
(from drbdsetup [110450])
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0 DRBD9_3: Starting sender thread
(from drbdsetup [110453])
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0 DRBD9_1: conn( StandAlone ->
Unconnected )
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0 DRBD9_1: Starting receiver thread
(from drbd_w_r0 [110440])
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0 DRBD9_1: conn( Unconnected ->
Connecting )
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0 DRBD9_3: conn( StandAlone ->
Unconnected )
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0 DRBD9_3: Starting receiver thread
(from drbd_w_r0 [110440])
Aug 16 09:33:38 DRBD9_2 kernel: drbd r0 DRBD9_3: conn( Unconnected ->
Connecting )
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0 DRBD9_1: Handshake to peer 0
successful: Agreed network protocol version 111
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0 DRBD9_1: Feature flags enabled on
protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0 DRBD9_1: Starting ack_recv thread
(from drbd_r_r0 [110459])
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0 DRBD9_1: Preparing remote state
change 1975720708 (primary_nodes=0, weak_nodes=0)
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0 DRBD9_1: Committing remote state
change 1975720708
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0 DRBD9_1: conn( Connecting ->
Connected ) peer( Unknown -> Secondary )
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1: current_size: 194560
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: c_size: 194560
u_size: 0 d_size: 204800 max_size: 204800
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: la_size: 194560
my_usize: 0 my_max_size: 204800
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: node_id: 0 idx: 0
bm-uuid: 0x0 flags: 0x10 max_size: 204800 (DUnknown)
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1: my node_id: 1
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: node_id: 2 idx: 1
bm-uuid: 0x0 flags: 0x10 max_size: 0 (DUnknown)
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1: Resize forced while not
fully connected!
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: calling
drbd_determine_dev_size()
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: node_id: 0 idx: 0
bm-uuid: 0x0 flags: 0x10 max_size: 204800 (DUnknown)
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1: my node_id: 1
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: node_id: 2 idx: 1
bm-uuid: 0x0 flags: 0x10 max_size: 0 (DUnknown)
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1: Resize forced while not
fully connected!
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1: drbd_bm_resize called with
capacity == 204800
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1: resync bitmap: bits=25600
words=800 pages=2
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1: size = 100 MB (102400 KB)
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1: Writing the whole bitmap,
size changed
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1:
drbd_sync_handshake:
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: self
D43B0300042F33B8:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:0
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: peer
D43B0300042F33B8:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:120
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: uuid_compare()=0
by rule 38
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1: No resync, but 1280 bits
in bitmap!
Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: pdsk( DUnknown ->
UpToDate ) repl( Off -> Established )
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0 DRBD9_3: Handshake to peer 2
successful: Agreed network protocol version 111
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0 DRBD9_3: Feature flags enabled on
protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0 DRBD9_3: Starting ack_recv thread
(from drbd_r_r0 [110461])
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0 DRBD9_1: Preparing remote state
change 550148406 (primary_nodes=0, weak_nodes=0)
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0 DRBD9_1: Committing remote state
change 550148406
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0: Preparing cluster-wide state
change 1936541420 (1->2 499/146)
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0: State change 1936541420:
primary_nodes=0, weak_nodes=0
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0: Committing cluster-wide state
change 1936541420 (1ms)
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0 DRBD9_3: conn( Connecting ->
Connected ) peer( Unknown -> Secondary )
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0/0 drbd1: current_size: 204800
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: c_size: 194560
u_size: 0 d_size: 194560 max_size: 194560
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: la_size: 204800
my_usize: 0 my_max_size: 204800
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: node_id: 0 idx: 0
bm-uuid: 0x0 flags: 0x11 max_size: 204800 (UpToDate)
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0/0 drbd1: my node_id: 1
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: node_id: 2 idx: 1
bm-uuid: 0x0 flags: 0x10 max_size: 194560 (DUnknown)
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: The peer's disk
size is too small! (194560 < 204800 sectors)
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0 DRBD9_3: conn( Connected ->
Disconnecting ) peer( Secondary -> Unknown )
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0 DRBD9_3: error receiving P_SIZES,
e: -5 l: 0!
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0 DRBD9_3: ack_receiver terminated
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0 DRBD9_3: Terminating ack_recv thread
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0 DRBD9_3: Connection closed
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0 DRBD9_3: conn( Disconnecting ->
StandAlone )
Aug 16 09:33:46 DRBD9_2 kernel: drbd r0 DRBD9_3: Terminating receiver thread
<DRBD9_3>
...
Aug 16 09:32:27 DRBD9_3 kernel: drbd r0/0 drbd1: current_size: 194560
Aug 16 09:32:27 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: c_size: 204800
u_size: 0 d_size: 204800 max_size: 204800
Aug 16 09:32:27 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: la_size: 0
my_usize: 0
Aug 16 09:32:27 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: node_id: 0 idx: 0
bm-uuid: 0x0 flags: 0x0 max_size: 0 (DUnknown)
Aug 16 09:32:27 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: node_id: 1 idx: 1
bm-uuid: 0x0 flags: 0x0 max_size: 204800 (DUnknown)
Aug 16 09:32:27 DRBD9_3 kernel: drbd r0/0 drbd1: my node_id: 2
Aug 16 09:32:27 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: calling
drbd_determine_dev_size()
Aug 16 09:32:27 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: node_id: 0 idx: 0
bm-uuid: 0x0 flags: 0x0 max_size: 0 (DUnknown)
Aug 16 09:32:27 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: node_id: 1 idx: 1
bm-uuid: 0x0 flags: 0x0 max_size: 204800 (DUnknown)
Aug 16 09:32:27 DRBD9_3 kernel: drbd r0/0 drbd1: my node_id: 2
Aug 16 09:32:27 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2:
drbd_sync_handshake:
Aug 16 09:32:27 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: self
0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:0
Aug 16 09:32:27 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: peer
0000000000000004:0000000000000000:0000000000000000:0000000000000000
bits:1280 flags:24
Aug 16 09:32:27 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: uuid_compare()=0
by rule 10
Aug 16 09:32:27 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: pdsk( DUnknown ->
Inconsistent ) repl( Off -> Established )
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0 DRBD9_1: Handshake successful:
Agreed network protocol version 111
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0 DRBD9_1: Feature flags enabled on
protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0 DRBD9_1: Starting ack_recv thread
(from drbd_r_r0 [109747])
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0 DRBD9_1: Preparing remote state
change 1968584032 (primary_nodes=0, weak_nodes=0)
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0 DRBD9_1: Committing remote state
change 1968584032
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0 DRBD9_1: conn( Connecting ->
Connected ) peer( Unknown -> Secondary )
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0/0 drbd1: current_size: 194560
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: c_size: 204800
u_size: 0 d_size: 204800 max_size: 204800
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: la_size: 0
my_usize: 0
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: node_id: 0 idx: 0
bm-uuid: 0x0 flags: 0x0 max_size: 204800 (DUnknown)
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: node_id: 1 idx: 1
bm-uuid: 0x0 flags: 0x13 max_size: 204800 (Inconsistent)
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0/0 drbd1: my node_id: 2
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: calling
drbd_determine_dev_size()
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: node_id: 0 idx: 0
bm-uuid: 0x0 flags: 0x0 max_size: 204800 (DUnknown)
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: node_id: 1 idx: 1
bm-uuid: 0x0 flags: 0x13 max_size: 204800 (Inconsistent)
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0/0 drbd1: my node_id: 2
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1:
drbd_sync_handshake:
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: self
0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:0
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: peer
0000000000000004:0000000000000000:0000000000000000:0000000000000000
bits:1280 flags:24
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: uuid_compare()=0
by rule 10
Aug 16 09:32:30 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: pdsk( DUnknown ->
Inconsistent ) repl( Off -> Established )
Aug 16 09:32:36 DRBD9_3 kernel: drbd r0 DRBD9_1: Preparing remote state
change 369176076 (primary_nodes=0, weak_nodes=0)
Aug 16 09:32:36 DRBD9_3 kernel: drbd r0 DRBD9_1: Committing remote state
change 369176076
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0: Preparing cluster-wide state
change 1953992009 (2->-1 7683/4609)
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0: State change 1953992009:
primary_nodes=4, weak_nodes=FFFFFFFFFFFFFFF8
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0: Committing cluster-wide state
change 1953992009 (0ms)
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0: role( Secondary -> Primary )
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1: disk( Inconsistent ->
UpToDate )
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1: size = 95 MB (97280 KB)
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0: Forced to consider local data as
UpToDate!
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1: new current UUID:
D43B0300042F33B9 weak: FFFFFFFFFFFFFFF8
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1:
drbd_sync_handshake:
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: self
D43B0300042F33B9:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:0
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: peer
0000000000000004:FFFFFFFFFFFFFFFF:0000000000000000:0000000000000000
bits:24320 flags:104
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: uuid_compare()=3
by rule 30
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: Writing the whole
bitmap, full sync required after drbd_sync_handshake.
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2:
drbd_sync_handshake:
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: self
D43B0300042F33B9:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:0
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: peer
0000000000000004:FFFFFFFFFFFFFFFF:0000000000000000:0000000000000000
bits:24320 flags:104
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: uuid_compare()=3
by rule 30
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: Writing the whole
bitmap, full sync required after drbd_sync_handshake.
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1: drbd_r_r0[109748] going to
'set_n_write from sync_handshake' but bitmap already locked for
'set_n_write from sync_handshake' by drbd_r_r0[109747]
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: repl( Established
-> WFBitMapS )
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: send bitmap stats
[Bytes(packets)]: plain 0(0), RLE 21(1), total 21; compression: 99.4%
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: repl( Established
-> WFBitMapS )
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: send bitmap stats
[Bytes(packets)]: plain 0(0), RLE 21(1), total 21; compression: 99.4%
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: receive bitmap
stats [Bytes(packets)]: plain 0(0), RLE 21(1), total 21; compression: 99.4%
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: helper command:
/sbin/drbdadm before-resync-source
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: receive bitmap
stats [Bytes(packets)]: plain 0(0), RLE 21(1), total 21; compression: 99.4%
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: helper command:
/sbin/drbdadm before-resync-source
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: helper command:
/sbin/drbdadm before-resync-source exit code 0 (0x0)
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: repl( WFBitMapS ->
SyncSource )
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: Began resync as
SyncSource (will sync 97280 KB [24320 bits set]).
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: helper command:
/sbin/drbdadm before-resync-source exit code 0 (0x0)
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: repl( WFBitMapS ->
SyncSource )
Aug 16 09:32:56 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: Began resync as
SyncSource (will sync 97280 KB [24320 bits set]).
Aug 16 09:33:12 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: Resync done (total
15 sec; paused 0 sec; 6484 K/sec)
Aug 16 09:33:12 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: pdsk( Inconsistent
-> UpToDate ) repl( SyncSource -> Established )
Aug 16 09:33:12 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: Resync done (total
15 sec; paused 0 sec; 6484 K/sec)
Aug 16 09:33:12 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: pdsk( Inconsistent
-> UpToDate ) repl( SyncSource -> Established )
Aug 16 09:33:22 DRBD9_3 kernel: drbd r0: role( Primary -> Secondary )
Aug 16 09:33:26 DRBD9_3 kernel: drbd r0: Preparing cluster-wide state
change 3319188407 (2->0 496/16)
Aug 16 09:33:26 DRBD9_3 kernel: drbd r0: State change 3319188407:
primary_nodes=0, weak_nodes=0
Aug 16 09:33:26 DRBD9_3 kernel: drbd r0: Committing cluster-wide state
change 3319188407 (0ms)
Aug 16 09:33:26 DRBD9_3 kernel: drbd r0 DRBD9_1: conn( Connected ->
Disconnecting ) peer( Secondary -> Unknown )
Aug 16 09:33:26 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: pdsk( UpToDate ->
DUnknown ) repl( Established -> Off )
Aug 16 09:33:26 DRBD9_3 kernel: drbd r0 DRBD9_1: ack_receiver terminated
Aug 16 09:33:26 DRBD9_3 kernel: drbd r0 DRBD9_1: Terminating ack_recv thread
Aug 16 09:33:26 DRBD9_3 kernel: drbd r0 DRBD9_1: Connection closed
Aug 16 09:33:26 DRBD9_3 kernel: drbd r0 DRBD9_1: conn( Disconnecting ->
StandAlone )
Aug 16 09:33:26 DRBD9_3 kernel: drbd r0 DRBD9_1: Terminating receiver thread
Aug 16 09:33:26 DRBD9_3 kernel: drbd r0 DRBD9_1: Terminating sender thread
Aug 16 09:33:27 DRBD9_3 kernel: drbd r0: Preparing cluster-wide state
change 2471101321 (2->1 496/16)
Aug 16 09:33:27 DRBD9_3 kernel: drbd r0: State change 2471101321:
primary_nodes=0, weak_nodes=0
Aug 16 09:33:27 DRBD9_3 kernel: drbd r0 DRBD9_2: Cluster is now split
Aug 16 09:33:27 DRBD9_3 kernel: drbd r0: Committing cluster-wide state
change 2471101321 (6ms)
Aug 16 09:33:27 DRBD9_3 kernel: drbd r0 DRBD9_2: conn( Connected ->
Disconnecting ) peer( Secondary -> Unknown )
Aug 16 09:33:27 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: pdsk( UpToDate ->
DUnknown ) repl( Established -> Off )
Aug 16 09:33:27 DRBD9_3 kernel: drbd r0 DRBD9_2: ack_receiver terminated
Aug 16 09:33:27 DRBD9_3 kernel: drbd r0 DRBD9_2: Terminating ack_recv thread
Aug 16 09:33:27 DRBD9_3 kernel: drbd r0 DRBD9_2: Connection closed
Aug 16 09:33:27 DRBD9_3 kernel: drbd r0 DRBD9_2: conn( Disconnecting ->
StandAlone )
Aug 16 09:33:27 DRBD9_3 kernel: drbd r0 DRBD9_2: Terminating receiver thread
Aug 16 09:33:27 DRBD9_3 kernel: drbd r0 DRBD9_2: Terminating sender thread
Aug 16 09:33:27 DRBD9_3 kernel: drbd r0/0 drbd1: disk( UpToDate ->
Detaching )
Aug 16 09:33:27 DRBD9_3 kernel: drbd r0/0 drbd1: disk( Detaching ->
Diskless )
Aug 16 09:33:27 DRBD9_3 kernel: drbd r0/0 drbd1: drbd_bm_resize called with
capacity == 0
Aug 16 09:33:27 DRBD9_3 kernel: drbd r0: Terminating worker thread
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0: Starting worker thread (from
drbdsetup [109966])
Aug 16 09:33:45 DRBD9_3 systemd-udevd: error: /dev/drbd1: Wrong medium type
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0/0 drbd1: disk( Diskless ->
Attaching )
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0/0 drbd1: Maximum number of peer
devices = 2
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0: Method to ensure write ordering:
flush
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0/0 drbd1: node_id: 0 idx: 0 bm-uuid:
0x0 flags: 0x10 (not currently reachable)
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0/0 drbd1: node_id: 1 idx: 1 bm-uuid:
0x0 flags: 0x10 (not currently reachable)
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0/0 drbd1: my node_id: 2
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0/0 drbd1: Adjusting my ra_pages to
backing device's (32 -> 1024)
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0/0 drbd1: node_id: 0 idx: 0 bm-uuid:
0x0 flags: 0x10 (not currently reachable)
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0/0 drbd1: node_id: 1 idx: 1 bm-uuid:
0x0 flags: 0x10 (not currently reachable)
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0/0 drbd1: my node_id: 2
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0/0 drbd1: drbd_bm_resize called with
capacity == 194560
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0/0 drbd1: resync bitmap: bits=24320
words=760 pages=2
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0/0 drbd1: size = 95 MB (97280 KB)
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0/0 drbd1: recounting of set bits
took additional 0ms
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0/0 drbd1: disk( Attaching ->
UpToDate )
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0/0 drbd1: attached to current UUID:
D43B0300042F33B8
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0 DRBD9_1: Starting sender thread
(from drbdsetup [109976])
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0 DRBD9_2: Starting sender thread
(from drbdsetup [109980])
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0 DRBD9_1: conn( StandAlone ->
Unconnected )
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0 DRBD9_1: Starting receiver thread
(from drbd_w_r0 [109967])
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0 DRBD9_1: conn( Unconnected ->
Connecting )
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0 DRBD9_2: conn( StandAlone ->
Unconnected )
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0 DRBD9_2: Starting receiver thread
(from drbd_w_r0 [109967])
Aug 16 09:33:45 DRBD9_3 kernel: drbd r0 DRBD9_2: conn( Unconnected ->
Connecting )
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_1: Handshake successful:
Agreed network protocol version 111
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_1: Feature flags enabled on
protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_1: Starting ack_recv thread
(from drbd_r_r0 [109985])
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_2: Handshake successful:
Agreed network protocol version 111
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_2: Feature flags enabled on
protocol level: 0x7 TRIM THIN_RESYNC WRITE_SAME.
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_2: Starting ack_recv thread
(from drbd_r_r0 [109986])
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_1: Preparing remote state
change 550148406 (primary_nodes=0, weak_nodes=0)
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_1: Committing remote state
change 550148406
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_1: conn( Connecting ->
Connected ) peer( Unknown -> Secondary )
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1: current_size: 194560
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: c_size: 204800
u_size: 0 d_size: 204800 max_size: 204800
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: la_size: 194560
my_usize: 0
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: node_id: 0 idx: 0
bm-uuid: 0x0 flags: 0x10 max_size: 204800 (DUnknown)
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: node_id: 1 idx: 1
bm-uuid: 0x0 flags: 0x10 max_size: 0 (DUnknown)
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1: my node_id: 2
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: calling
drbd_determine_dev_size()
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: node_id: 0 idx: 0
bm-uuid: 0x0 flags: 0x10 max_size: 204800 (DUnknown)
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: node_id: 1 idx: 1
bm-uuid: 0x0 flags: 0x10 max_size: 0 (DUnknown)
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1: my node_id: 2
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1:
drbd_sync_handshake:
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: self
D43B0300042F33B8:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:0
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: peer
D43B0300042F33B8:0000000000000000:0000000000000000:0000000000000000
bits:1280 flags:120
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: uuid_compare()=0
by rule 38
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: pdsk( DUnknown ->
UpToDate ) repl( Off -> Established )
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_1: sock was shut down by peer
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_1: conn( Connected ->
BrokenPipe ) peer( Secondary -> Unknown )
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: pdsk( UpToDate ->
DUnknown ) repl( Established -> Off )
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_1: ack_receiver terminated
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_1: Terminating ack_recv thread
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_2: Preparing remote state
change 1936541420 (primary_nodes=0, weak_nodes=0)
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_2: Committing remote state
change 1936541420
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_2: conn( Connecting ->
Connected ) peer( Unknown -> Secondary )
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1: current_size: 194560
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: c_size: 204800
u_size: 0 d_size: 204800 max_size: 204800
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: la_size: 194560
my_usize: 0
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: node_id: 0 idx: 0
bm-uuid: 0x0 flags: 0x10 max_size: 204800 (DUnknown)
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: node_id: 1 idx: 1
bm-uuid: 0x0 flags: 0x10 max_size: 204800 (DUnknown)
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1: my node_id: 2
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: calling
drbd_determine_dev_size()
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_1: node_id: 0 idx: 0
bm-uuid: 0x0 flags: 0x10 max_size: 204800 (DUnknown)
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: node_id: 1 idx: 1
bm-uuid: 0x0 flags: 0x10 max_size: 204800 (DUnknown)
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1: my node_id: 2
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2:
drbd_sync_handshake:
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: self
D43B0300042F33B8:0000000000000000:0000000000000000:0000000000000000 bits:0
flags:0
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: peer
D43B0300042F33B8:0000000000000000:0000000000000000:0000000000000000
bits:1280 flags:20
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: uuid_compare()=0
by rule 40
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: pdsk( DUnknown ->
UpToDate ) repl( Off -> Established )
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_1: Connection closed
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_1: conn( BrokenPipe ->
Unconnected )
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_1: Restarting receiver thread
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_1: conn( Unconnected ->
Connecting )
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_2: sock was shut down by peer
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_2: conn( Connected ->
BrokenPipe ) peer( Secondary -> Unknown )
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0/0 drbd1 DRBD9_2: pdsk( UpToDate ->
DUnknown ) repl( Established -> Off )
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_2: ack_receiver terminated
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_2: Terminating ack_recv thread
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_2: Connection closed
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_2: conn( BrokenPipe ->
Unconnected )
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_2: Restarting receiver thread
Aug 16 09:33:46 DRBD9_3 kernel: drbd r0 DRBD9_2: conn( Unconnected ->
Connecting )
ᐧ
[-- Attachment #2: Type: text/html, Size: 67168 bytes --]
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: [Drbd-dev] Here's logs 3 volumes case
2016-08-16 1:14 [Drbd-dev] Here's logs 3 volumes case 박경민
@ 2016-08-18 11:51 ` Lars Ellenberg
0 siblings, 0 replies; 2+ messages in thread
From: Lars Ellenberg @ 2016-08-18 11:51 UTC (permalink / raw)
To: drbd-dev
On Tue, Aug 16, 2016 at 10:14:07AM +0900, 박경민 wrote:
> There're 3 node, and each had 100MB/100MB/95MB volume, time synced
> Please ,Watch DRBD9_3 node's log carefully
> <DRBD9_1>
>
> ...
> Aug 16 09:32:30 DRBD9_1 kernel: drbd r0/0 drbd1: current_size: 204800
...
> Aug 16 09:32:56 DRBD9_1 kernel: drbd r0/0 drbd1: size = 95 MB (97280 KB)
> Aug 16 09:33:12 DRBD9_1 kernel: drbd r0/0 drbd1: disk( Inconsistent -> UpToDate )
> Aug 16 09:33:26 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: pdsk( UpToDate -> DUnknown ) repl( Established -> Off )
...
> Aug 16 09:33:39 DRBD9_1 kernel: drbd r0 DRBD9_2: conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1: current_size: 194560
> Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: c_size: 194560 u_size: 0 d_size: 204800 max_size: 204800
> Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: la_size: 194560 my_usize: 0 my_max_size: 204800
> Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1: my node_id: 0
> Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_2: node_id: 1 idx: 0 bm-uuid: 0x0 flags: 0x10 max_size: 204800 (DUnknown)
> Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1 DRBD9_3: node_id: 2 idx: 1 bm-uuid: 0x0 flags: 0x10 max_size: 194560 (DUnknown)
> Aug 16 09:33:39 DRBD9_1 kernel: drbd r0/0 drbd1: Resize forced while not fully connected!
Still knows that DRBD9_3 cannot support it,
but knowingly ignores it.
"Too bad".
> <DRBD9_2>
> ...
> Aug 16 09:32:27 DRBD9_2 kernel: drbd r0/0 drbd1: current_size: 204800
> Aug 16 09:32:56 DRBD9_2 kernel: drbd r0/0 drbd1: size = 95 MB (97280 KB)
> Aug 16 09:33:38 DRBD9_2 kernel: drbd r0/0 drbd1: size = 95 MB (97280 KB)
> Aug 16 09:33:39 DRBD9_2 kernel: drbd r0 DRBD9_1: conn( Connecting -> Connected ) peer( Unknown -> Secondary )
> Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1: current_size: 194560
> Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: c_size: 194560 u_size: 0 d_size: 204800 max_size: 204800
> Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: la_size: 194560 my_usize: 0 my_max_size: 204800
> Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: calling drbd_determine_dev_size()
> Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_1: node_id: 0 idx: 0 bm-uuid: 0x0 flags: 0x10 max_size: 204800 (DUnknown)
> Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1: my node_id: 1
> Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1 DRBD9_3: node_id: 2 idx: 1 bm-uuid: 0x0 flags: 0x10 max_size: 0 (DUnknown)
> Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1: Resize forced while not fully connected!
> Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1: drbd_bm_resize called with capacity == 204800
> Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1: resync bitmap: bits=25600 words=800 pages=2
> Aug 16 09:33:39 DRBD9_2 kernel: drbd r0/0 drbd1: size = 100 MB (102400 KB)
Does not know exactly anymore what DRBD9_3 can support,
but still knows that the last agreed size was 95, and that _3 is
currently not reachable. Still knowingly ignores it.
"Too bad".
Thanks for the logs.
As I said, the (re)size handshake is something we actively work on, but
have not come to a "nice" solution yet.
The code lines you pointed out earlier have already been changed
internally again (I think that'S not yet publically visible,
because we don't know yet if we keep it that way, or evenchange the wire
protocol for this).
In the code that you currently have,
it is entirely possible that what was meant
is the negated logic:
add the flag only if it is NOT the handshake.
You should be able to work around it for now
by explicitly specifying the target size in the config file
(or by patching the line you indicated already).
The real fix for all "possible" cases of (re)size exchanges
turns out to be much more complicated than we would like it to be.
Lars
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2016-08-18 11:51 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-08-16 1:14 [Drbd-dev] Here's logs 3 volumes case 박경민
2016-08-18 11:51 ` Lars Ellenberg
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox