Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Verify if missing modified events from simulator to aggregator #112

Closed
yb01 opened this issue Jul 28, 2022 · 2 comments
Closed

Verify if missing modified events from simulator to aggregator #112

yb01 opened this issue Jul 28, 2022 · 2 comments
Assignees
@q131172019
Milestone
2022-0730

Comments

@yb01
Copy link
Collaborator

yb01 commented Jul 28, 2022

here is the simulator and aggregator log:


I0727 23:55:48.284354   76024 regionNodeEvents.go:77] Total (25000) Added events are to be pulled
I0727 23:55:48.284431   76024 regionNodeEvents.go:83] Pulling Region Node Event with final batch size (25000) for (1) RPs
I0727 23:55:48.470271   76024 regionNodeEvents.go:101] Handle GET all (25000) region node added events via initPull successfully
I0727 23:55:49.239142   76024 regionNodeEvents.go:119] Handle POST CRV to discard all old region node modified events successfully
I0727 23:57:43.763921   76024 regionNodeEvents.go:190] Actually total (5000) new modified events are created.
I0727 23:57:43.763974   76024 regionNodeEvents.go:54] At 2th minute mark, generating 5k modified and added node events is completed
I0727 23:57:43.786886   76024 regionNodeEvents.go:83] Pulling Region Node Event with final batch size (4538) for (1) RPs
I0727 23:57:43.810673   76024 regionNodeEvents.go:103] Handle GET all (4538) region node modified events via SubsequentPull succesfully
I0727 23:57:43.965058   76024 regionNodeEvents.go:119] Handle POST CRV to discard all old region node modified events successfully
I0728 00:00:43.787620   76024 regionNodeEvents.go:190] Actually total (25000) new modified events are created.
I0728 00:00:43.787663   76024 regionNodeEvents.go:59] At 5th time mark, generating 25k modified node events is completed
I0728 00:00:43.816957   76024 regionNodeEvents.go:83] Pulling Region Node Event with final batch size (15791) for (1) RPs
I0728 00:00:43.907375   76024 regionNodeEvents.go:103] Handle GET all (15791) region node modified events via SubsequentPull succesfully
I0728 00:00:44.382160   76024 regionNodeEvents.go:119] Handle POST CRV to discard all old region node modified events successfully
I0728 00:02:43.796601   76024 regionNodeEvents.go:190] Actually total (1000) new modified events are created.
I0728 00:02:43.796652   76024 regionNodeEvents.go:64] At 7th time mark, generating 1k modified events is completed
I0728 00:02:43.888924   76024 regionNodeEvents.go:83] Pulling Region Node Event with final batch size (986) for (1) RPs
I0728 00:02:43.893577   76024 regionNodeEvents.go:103] Handle GET all (986) region node modified events via SubsequentPull succesfully
I0728 00:02:43.935691   76024 regionNodeEvents.go:119] Handle POST CRV to discard all old region node modified events successfully
I0728 00:04:43.909879   76024 regionNodeEvents.go:190] Actually total (5000) new modified events are created.
I0728 00:04:43.909924   76024 regionNodeEvents.go:54] At 2th minute mark, generating 5k modified and added node events is completed
I0728 00:04:43.961471   76024 regionNodeEvents.go:83] Pulling Region Node Event with final batch size (4515) for (1) RPs
I0728 00:04:43.984555   76024 regionNodeEvents.go:103] Handle GET all (4515) region node modified events via SubsequentPull succesfully
I0728 00:04:44.132528   76024 regionNodeEvents.go:119] Handle POST CRV to discard all old region node modified events successfully
 
 
I0727 23:55:48.749475   76042 aggregator.go:97] Total (25000) region node events are pulled successfully in (1) RPs
I0727 23:57:43.864798   76042 aggregator.go:97] Total (4538) region node events are pulled successfully in (1) RPs
I0728 00:00:44.076756   76042 aggregator.go:97] Total (15791) region node events are pulled successfully in (1) RPs
I0728 00:02:43.905979   76042 aggregator.go:97] Total (986) region node events are pulled successfully in (1) RPs
I0728 00:04:44.035163   76042 aggregator.go:97] Total (4515) region node events are pulled successfully in (1) RPs
@yb01 yb01 added this to the 2022-0730 milestone Jul 28, 2022
@yb01
Copy link
Collaborator Author

yb01 commented Jul 28, 2022

Carl to double check

@yb01
Copy link
Collaborator Author

yb01 commented Jul 30, 2022 •
edited
Loading

did some adhoc testing and it looks good per the results below. for 1 RP failure cases, where two PULL()s were done to get the 25k changes total

I0730 21:46:24.947075    2323 installer.go:210] Start processing watch event for client: Client-323048a1-f905-4744-a6f0-8697e7f31121
I0730 21:50:44.966264    2323 aggregator.go:92] Total (22283) region node events are pulled successfully in (10) RPs. pull duration 779.428879ms
I0730 21:50:45.775560    2323 aggregator.go:92] Total (2717) region node events are pulled successfully in (10) RPs. pull duration 518.975899ms


I0730 11:55:06.992833   30967 installer.go:210] Start processing watch event for client: Client-a6cdc27c-76ab-4f22-af1b-6ac5dd9f29e7
I0730 11:55:57.108637   30967 aggregator.go:92] Total (1870) region node events are pulled successfully in (1) RPs. pull duration 8.533193ms
I0730 11:55:57.226482   30967 aggregator.go:92] Total (23130) region node events are pulled successfully in (1) RPs. pull duration 44.013629ms
^C


  9 I0731 00:17:54.480180   31028 stats.go:112] Nodes group by region:
  10 I0731 00:17:54.480191   31028 stats.go:114]      Region: 0, node counts: 25087
  11 I0731 00:17:54.481207   31028 stats.go:90] Nodes group by region, by resource partition:
  12 I0731 00:17:54.481215   31028 stats.go:92] Region: 0
  13 I0731 00:17:54.481222   31028 stats.go:94]       RP: 4, node counts: 2070
  14 I0731 00:17:54.481229   31028 stats.go:94]       RP: 9, node counts: 2859
  15 I0731 00:17:54.481234   31028 stats.go:94]       RP: 3, node counts: 2914
  16 I0731 00:17:54.481240   31028 stats.go:94]       RP: 1, node counts: 1600
  17 I0731 00:17:54.481245   31028 stats.go:94]       RP: 2, node counts: 2587
  18 I0731 00:17:54.481252   31028 stats.go:94]       RP: 8, node counts: 2264
  19 I0731 00:17:54.481258   31028 stats.go:94]       RP: 6, node counts: 2591
  20 I0731 00:17:54.481263   31028 stats.go:94]       RP: 0, node counts: 2215
  21 I0731 00:17:54.481268   31028 stats.go:94]       **RP: 5, node counts: 2839**
  22 I0731 00:17:54.481274   31028 stats.go:94]       RP: 7, node counts: 3148

2869 I0731 00:23:48.772242   31028 stats.go:41] [Metrics][List]ListDuration: 2.409043265s. Number of nodes listed: 25087
2870 I0731 00:23:48.772254   31028 stats.go:60] [Metrics][Watch]Watch session last: 5m54.280675432s. Number of nodes Added :0, Updated: 2839, Deleted: 0. watch prolonged than 1s: 2839
2871 I0731 00:23:48.772450   31028 stats.go:65] [Metrics][Watch] perc50 2.813019364s, perc90 2.864451423s, perc99 2.906315649s. Total count **2839**

@yb01 yb01 closed this as completed Jul 30, 2022
@yb01 yb01 changed the title Missing modified events from simulator to aggregator Verify if missing modified events from simulator to aggregator Jul 31, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@q131172019 q131172019

Labels
None yet
Projects
None yet
Milestone
2022-0730
Development

No branches or pull requests
2 participants
@yb01 @q131172019