Here you can find some common commands related to DRBD. They will be added once I find use of them on my daily work. Stay tuned!
How to force re-sync by invalidating the data on the node you are in:
root@eave:~# [color=green]drbdadm invalidate nfs[/color] root@eave:~# cat /proc/drbd version: 8.3.11 (api:88/proto:86-96) srcversion: F937DCB2E5D83C6CCE4A6C9 0: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r----- ns:0 nr:28860 dw:24380 dr:0 al:0 bm:2 lo:36 pe:434 ua:35 ap:0 ep:1 wo:f oos:26192512 [>....................] sync'ed: 0.2% (25576/25600)Mfinish: 0:19:50 speed: 21,888 (21,888) want: 51,200 K/sec
How to force re-sync by invalidating the data on the other node:
root@eave:~# [color=green]drbdadm invalidate_remote nfs[/color] root@eave:~# cat /proc/drbd version: 8.3.11 (api:88/proto:86-96) srcversion: F937DCB2E5D83C6CCE4A6C9 1: cs:SyncTarget ro:Secondary/Primary ds:UpToDate/Inconsistent C r----- ns:0 nr:6073892 dw:6066212 dr:0 al:0 bm:370 lo:61 pe:7454 ua:60 ap:0 ep:1 wo:f oos:20155504 [===>................] sync'ed: 23.2% (19680/25600)Mfinish: 0:24:42 speed: 13,596 (11,220) want: 51,200 K/sec
How to fix StandAlone Unknown drbd resource:
1. On primary node: drbdadm connect nfs
2. On secondary node: drbdadm -- --discard-my-data connect nfs
Example:
root@eave:~# cat /proc/drbd version: 8.3.11 (api:88/proto:86-96) srcversion: 00000000000000000000000 0: cs:StandAlone ro:Primary/Unknown ds:UpToDate/DUnknown r----- ns:0 nr:0 dw:0 dr:664 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:524032 root@eave:~# drbdadm connect nfs root@candy:~# drbdadm -- --discard-my-data connect nfs root@eave:~# cat /proc/drbd version: 8.3.11 (api:88/proto:86-96) srcversion: 00000000000000000000000 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----- ns:524032 nr:0 dw:0 dr:525028 al:0 bm:32 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0
Split Brain Recovery
:
How to get pass the WFConnection state
This happens usually when the network connection between nodes was cut off (split brain). In this situation, if you do not have drbd configured to recover automatically, you have to step in.
Version 8.3:
On secondary node (where the data will be discarded):
drbdadm down nfs
drbdadm -- --discard-my-data connect nfs
drbdadm attach nfs
Example:
root@candy:~# drbdadm down nfs root@candy:~# cat /proc/drbd version: 8.3.11 (api:88/proto:86-96) srcversion: 00000000000000000000000 0: cs:Unconfigured root@candy:~# drbdadm -- --discard-my-data connect nfs root@candy:~# cat /proc/drbd version: 8.3.11 (api:88/proto:86-96) srcversion: 00000000000000000000000 0: cs:WFConnection ro:Secondary/Unknown ds:Diskless/DUnknown C r----- ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 root@candy:~# drbdadm attach nfs root@candy:~# cat /proc/drbd version: 8.3.11 (api:88/proto:86-96) srcversion: 00000000000000000000000 0: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r----- ns:0 nr:66680 dw:58476 dr:0 al:0 bm:126 lo:146 pe:547 ua:145 ap:0 ep:1 wo:f oos:66432 [========>...........] sync'ed: 48.4% (66432/124908)K finish: 0:00:11 speed: 5,844 (5,844) want: 51,200 K/sec
For version 8.4:
Secondary node:
drbdadm disconnect nfs
drbdadm -- --discard-my-data connect nfs
Primary node (if not connected):
drbdadm connect nfs
How to fix uuid_compare()=-1000 by rule 100
This happens usually after a split brain and normally this error comes with the explanation. However, in this situation I got only uuid_compare()=-1000 by rule 100 with nothing else.
The fix here was to recreate the meta device:
# drbdadm detach nfs
# drbdadm create-md nfs
# drbdadm up nfs
Example:
root@eave:~# drbdadm detach nfs root@eave:~# drbdadm create-md nfs md_offset 0 al_offset 4096 bm_offset 36864 Found some data ==> This might destroy existing data! <== Do you want to proceed? [need to type 'yes' to confirm] yes Valid meta data seems to be in place. Do you really want to overwrite? [need to type 'yes' to confirm] yes Writing meta data... initializing activity log NOT initialized bitmap New drbd meta data block successfully created. root@eave:~# drbdadm up nfs root@eave:~# cat /proc/drbd version: 8.3.11 (api:88/proto:86-96) srcversion: F937DCB2E5D83C6CCE4A6C9 0: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate B r----- ns:0 nr:748024 dw:716024 dr:0 al:0 bm:42 lo:252 pe:4168 ua:250 ap:0 ep:1 wo:d oos:72684800 [>....................] sync'ed: 1.0% (70980/71680)Mfinish: 1:15:55 speed: 15,936 (13,760) want: 30,720 K/sec 2: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate B r----- ns:0 nr:27316 dw:27316 dr:0 al:0 bm:3 lo:0 pe:0 ua:0 ap:0 ep:1 wo:d oos:0 3: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate B r----- ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:d oos:0 4: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate B r----- ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:d oos:0
How to populate the metadata device and startup drbd:
drbdadm create-md nfs
drbdadm up all
cat /proc/drbd
Example:
root@candy:~# drbdadm create-md nfs Writing meta data... initializing activity log NOT initialized bitmap New drbd meta data block successfully created. success root@candy:~# drbdadm up all root@candy:~# cat /proc/drbd version: 8.3.11 (api:88/proto:86-96) srcversion: 00000000000000000000000 0: cs:WFConnection ro:Secondary/Unknown ds:Inconsistent/DUnknown C r----- ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:26214144
How to start the first sync of drbd resource:
drbdadm -- --overwrite-data-of-peer primary nfs
Example:
root@eave:~# drbdadm -- --overwrite-data-of-peer primary nfs root@eave:~# cat /proc/drbd version: 8.3.11 (api:88/proto:86-96) srcversion: 00000000000000000000000 0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r----- ns:810112 nr:0 dw:0 dr:811672 al:0 bm:49 lo:0 pe:2 ua:7 ap:0 ep:1 wo:f oos:25404416 [>....................] sync'ed: 3.2% (24808/25600)Mfinish: 0:40:52 speed: 10,340 (10,124) K/sec
How to change the state of drbd resource as secondary:
# drbdadm secondary nfs
How to change the state of drbd resource as primary:
# drbdadm primary nfs
How to change sync rate without stop/start drbd:
# drbdsetup /dev/drbd0 syncer -r 10M
You can do it also by modifying the config file and execute afterwards (assuming the resource is nfs):
# drbdadm adjust nfs