Commit ebcf035
DTrace and log polish (#1773)
Cleanup DTrace printing of downstairs state.
Convert the DsState type into a string before we stuff it into the
DTrace probe. We were treating it as a string already on the other side,
so why not just send the string we want to begin with. This avoids
having to do more complicated processing inside the D scripts. Updated
all the DTraces scripts that were using the old method to now expect the
new string that is returned for the default display of DsState type.
Updated the default display for DsState to provide a bit more info for
states that were "Connecting".
Changed a few log messages from error to warn in the case where we can't
send an update message to Nexus.
Fix for various bits in
#1772
Here is some sample output from some of the DTrace scripts:
```
alan@etrium:crucible$ pfexec dtrace -Z -s tools/dtrace/up-info.d
PID UUID SESSION DS0 DS1 DS2 NEXT_JOB DELTA CONN ELR ELC ERR ERN
12140 b088b7c5 2d9b1fec ACT ACT ACT 1004 0 3 0 0 0 0
12140 b088b7c5 2d9b1fec ACT ACT ACT 1004 0 3 0 0 0 0
12140 b088b7c5 2d9b1fec ACT ACT ACT 1004 0 3 0 0 0 0
12140 b088b7c5 2d9b1fec ACT ACT ACT 1004 0 3 0 0 0 0
12146 2d50ec4b 431c0edf ACT ACT ACT 1004 0 3 0 0 0 0
12146 2d50ec4b 431c0edf ACT ACT ACT 1004 0 3 0 0 0 0
12146 2d50ec4b 431c0edf ACT ACT ACT 1004 0 3 0 0 0 0
12146 2d50ec4b 431c0edf ACT ACT ACT 1004 0 3 0 0 0 0
12154 5f6d602e d8035684 WQ WQ NEW 1000 0 2 0 0 0 0
12154 5f6d602e d8035684 WQ WQ NEW 1000 0 2 0 0 0 0
12154 5f6d602e d8035684 WQ WQ NEW 1000 0 2 0 0 0 0
12154 5f6d602e d8035684 WQ WQ NEW 1000 0 2 0 0 0 0
12154 5f6d602e d8035684 WQ WQ NEW 1000 0 2 0 0 0 0
12154 5f6d602e d8035684 WQ WQ NEW 1000 0 2 0 0 0 0
12154 5f6d602e d8035684 WQ WQ NEW 1000 0 2 0 0 0 0
12154 5f6d602e d8035684 WQ WQ NEW 1000 0 2 0 0 0 0
12154 5f6d602e d8035684 WQ WQ NEW 1000 0 2 0 0 0 0
12154 5f6d602e d8035684 REC REC REC 1000 0 3 0 0 2 18
12154 5f6d602e d8035684 ACT ACT ACT 1002 2 3 0 0 20 0
12154 5f6d602e d8035684 ACT ACT ACT 1002 0 3 0 0 20 0
12154 5f6d602e d8035684 ACT ACT ACT 1002 0 3 0 0 20 0
PID UUID SESSION DS0 DS1 DS2 NEXT_JOB DELTA CONN ELR ELC ERR ERN
12154 5f6d602e d8035684 ACT ACT ACT 1002 0 3 0 0 20 0
12179 0817aca3 2a0fd3d4 OFL ACT ACT 1401 0 3 0 0 0 0
12179 0817aca3 2a0fd3d4 OFL ACT ACT 1401 0 3 0 0 0 0
12179 0817aca3 2a0fd3d4 OFL ACT ACT 1401 0 3 0 0 0 0
```
I let the dtrace scripts run while running cargo test and I could see
many of the states showing up.
Here is a bunch of noise from that:
```
alan@etrium:crucible$ pfexec dtrace -Z -s tools/dtrace/sled_upstairs_info.d
PID UUID SESSION DS0 DS1 DS2 UPW DSW NEXT_JOB WRITE_BO IP0 IP1 IP2 D0 D1 D2 S0 S1 S2 ER0 ER1 ER2 EC0 EC1 EC2
12494 7dc3f943 1bb2b697 FLT ACT ACT 0 0 1498 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12491 861995a1 e6a231de OFL OFL OFL 0 0 1000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12497 5bd17c94 fe8e0586 FLT ACT ACT 0 0 2201 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12492 34940348 dd0721be FLT ACT ACT 0 497 1497 0 0 0 0 0 497 497 497 0 0 0 0 0 0 0 0
12501 dbb21d8c 932671af OFL ACT ACT 0 0 1000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12503 da39d261 17b1d5c7 OFL ACT ACT 0 0 1000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12498 09e744d8 e1e347c0 FLT ACT ACT 0 1200 2200 0 0 0 0 0 1200 1200 1200 0 0 0 0 0 0 0 0
12502 4137d113 1912fce1 OFL ACT ACT 0 0 1000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12965 5fed87cb 30bb4c55 ACT ACT ACT 1 40 1184 1048576 1 1 1 39 39 39 0 0 0 0 0 0 0 0 0
12494 7dc3f943 1bb2b697 FLT ACT ACT 0 0 1498 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12497 5bd17c94 fe8e0586 FLT ACT ACT 0 0 2201 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12492 34940348 dd0721be FLT ACT ACT 0 497 1497 0 0 0 0 0 497 497 497 0 0 0 0 0 0 0 0
12501 dbb21d8c 932671af OFL ACT ACT 0 0 1000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12498 09e744d8 e1e347c0 FLT ACT ACT 0 1200 2200 0 0 0 0 0 1200 1200 1200 0 0 0 0 0 0 0 0
12965 5fed87cb 30bb4c55 ACT ACT ACT 1 43 1187 1048576 1 1 1 42 42 42 0 0 0 0 0 0 0 0 0
12494 7dc3f943 1bb2b697 FLT ACT ACT 0 0 1498 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12497 5bd17c94 fe8e0586 FLT ACT ACT 0 0 2201 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12492 34940348 dd0721be FLT ACT ACT 0 497 1497 0 0 0 0 0 497 497 497 0 0 0 0 0 0 0 0
12501 dbb21d8c 932671af OFL ACT ACT 0 0 1000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12498 09e744d8 e1e347c0 FLT ACT ACT 0 0 2201 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12965 5fed87cb 30bb4c55 ACT ACT ACT 1 46 1190 1048576 1 1 1 45 45 45 0 0 0 0 0 0 0 0 0
PID UUID SESSION DS0 DS1 DS2 UPW DSW NEXT_JOB WRITE_BO IP0 IP1 IP2 D0 D1 D2 S0 S1 S2 ER0 ER1 ER2 EC0 EC1 EC2
12494 7dc3f943 1bb2b697 FLT ACT ACT 0 0 1498 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12497 5bd17c94 fe8e0586 FLT ACT ACT 0 0 2201 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12492 34940348 dd0721be FLT ACT ACT 0 0 1498 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12501 dbb21d8c 932671af OFL ACT ACT 0 0 1000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12498 09e744d8 e1e347c0 FLT ACT ACT 0 0 2201 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12494 7dc3f943 1bb2b697 FLT ACT ACT 0 0 1498 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12497 5bd17c94 fe8e0586 FLT ACT ACT 0 0 2201 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12492 34940348 dd0721be FLT ACT ACT 0 0 1498 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12501 dbb21d8c 932671af OFL ACT ACT 0 0 1000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12498 09e744d8 e1e347c0 FLT ACT ACT 0 0 2201 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12494 7dc3f943 1bb2b697 LR ACT ACT 20 199 1699 17920 10 37 37 24 162 162 165 0 0 3 0 0 0 0 0
12497 5bd17c94 fe8e0586 LR ACT ACT 11 181 2384 8704 10 19 19 24 162 162 147 0 0 3 0 0 0 0 0
12492 34940348 dd0721be LR ACT ACT 2 164 1664 0 2 2 2 24 162 162 138 0 0 3 0 0 0 0 0
12501 dbb21d8c 932671af OFL ACT ACT 0 0 1000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12498 09e744d8 e1e347c0 LR ACT ACT 8 176 2379 6144 10 14 14 24 162 162 142 0 0 3 0 0 0 0 0
12501 dbb21d8c 932671af OFL ACT ACT 0 0 1000 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
12965 5fed87cb 30bb4c55 ACT ACT ACT 0 46 1190 0 0 0 0 46 46 46 0 0 0 0 0 0 0 0 0
12501 dbb21d8c 932671af ACT ACT ACT 0 1 1001 512 1 0 0 0 1 1 0 0 0 0 0 0 0 0 0
```
---------
Co-authored-by: Alan Hanson <[email protected]>1 parent 65ca41e commit ebcf035
File tree
14 files changed
+200
-178
lines changed- cmon/src
- tools/dtrace
- upstairs/src
14 files changed
+200
-178
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | | - | |
12 | | - | |
13 | | - | |
| 11 | + | |
14 | 12 | | |
15 | 13 | | |
16 | 14 | | |
| |||
86 | 84 | | |
87 | 85 | | |
88 | 86 | | |
89 | | - | |
90 | | - | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
91 | 90 | | |
92 | | - | |
93 | | - | |
94 | | - | |
95 | | - | |
96 | | - | |
97 | | - | |
98 | | - | |
99 | | - | |
100 | | - | |
101 | | - | |
102 | | - | |
103 | | - | |
104 | | - | |
105 | | - | |
106 | | - | |
107 | | - | |
108 | | - | |
109 | | - | |
110 | | - | |
111 | | - | |
112 | | - | |
113 | | - | |
114 | | - | |
115 | | - | |
116 | | - | |
117 | | - | |
118 | | - | |
119 | | - | |
120 | | - | |
121 | | - | |
122 | | - | |
123 | | - | |
124 | | - | |
125 | | - | |
126 | | - | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
127 | 106 | | |
128 | 107 | | |
129 | 108 | | |
| |||
277 | 256 | | |
278 | 257 | | |
279 | 258 | | |
280 | | - | |
281 | | - | |
282 | | - | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
283 | 262 | | |
284 | 263 | | |
285 | 264 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
60 | | - | |
| 60 | + | |
61 | 61 | | |
62 | | - | |
| 62 | + | |
63 | 63 | | |
64 | 64 | | |
65 | 65 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
6 | | - | |
| 6 | + | |
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
12 | | - | |
13 | | - | |
14 | | - | |
15 | | - | |
16 | | - | |
17 | | - | |
18 | | - | |
19 | | - | |
20 | | - | |
21 | | - | |
22 | | - | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
23 | 26 | | |
24 | 27 | | |
25 | 28 | | |
26 | 29 | | |
27 | 30 | | |
28 | 31 | | |
29 | 32 | | |
30 | | - | |
| 33 | + | |
31 | 34 | | |
32 | 35 | | |
33 | | - | |
| 36 | + | |
34 | 37 | | |
35 | 38 | | |
36 | | - | |
| 39 | + | |
37 | 40 | | |
38 | 41 | | |
39 | 42 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
5 | | - | |
| 5 | + | |
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | | - | |
38 | | - | |
39 | | - | |
40 | | - | |
41 | | - | |
42 | | - | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
48 | 51 | | |
49 | 52 | | |
50 | 53 | | |
| |||
54 | 57 | | |
55 | 58 | | |
56 | 59 | | |
57 | | - | |
| 60 | + | |
58 | 61 | | |
59 | 62 | | |
60 | | - | |
| 63 | + | |
61 | 64 | | |
62 | 65 | | |
63 | | - | |
| 66 | + | |
64 | 67 | | |
65 | 68 | | |
66 | 69 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | | - | |
| 2 | + | |
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| |||
28 | 28 | | |
29 | 29 | | |
30 | 30 | | |
31 | | - | |
32 | | - | |
33 | | - | |
34 | | - | |
35 | | - | |
36 | | - | |
37 | | - | |
38 | | - | |
39 | | - | |
40 | | - | |
41 | | - | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
42 | 45 | | |
43 | 46 | | |
44 | 47 | | |
| |||
48 | 51 | | |
49 | 52 | | |
50 | 53 | | |
51 | | - | |
| 54 | + | |
52 | 55 | | |
53 | 56 | | |
54 | | - | |
| 57 | + | |
55 | 58 | | |
56 | 59 | | |
57 | | - | |
| 60 | + | |
58 | 61 | | |
59 | 62 | | |
60 | 63 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
5 | | - | |
| 5 | + | |
6 | 6 | | |
7 | 7 | | |
8 | 8 | | |
| |||
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
54 | 57 | | |
55 | 58 | | |
56 | 59 | | |
| |||
61 | 64 | | |
62 | 65 | | |
63 | 66 | | |
64 | | - | |
| 67 | + | |
65 | 68 | | |
66 | 69 | | |
67 | | - | |
| 70 | + | |
68 | 71 | | |
69 | 72 | | |
70 | | - | |
| 73 | + | |
71 | 74 | | |
72 | 75 | | |
73 | 76 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
10 | | - | |
| 10 | + | |
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| |||
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
43 | | - | |
44 | | - | |
45 | | - | |
46 | | - | |
47 | | - | |
48 | | - | |
49 | | - | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
54 | 57 | | |
55 | 58 | | |
56 | 59 | | |
| |||
59 | 62 | | |
60 | 63 | | |
61 | 64 | | |
62 | | - | |
| 65 | + | |
63 | 66 | | |
64 | 67 | | |
65 | | - | |
| 68 | + | |
66 | 69 | | |
67 | 70 | | |
68 | | - | |
| 71 | + | |
69 | 72 | | |
70 | 73 | | |
71 | 74 | | |
| |||
0 commit comments