Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(config)_: integrate new rpc providers configs #6178

Open
wants to merge 8 commits into
base: develop
Choose a base branch
from

Conversation

friofry
Copy link
Contributor

@friofry friofry commented Dec 6, 2024

This is the follow-up PR for the improved RPC configuration. See description: #6151

@status-im-auto
Copy link
Member

status-im-auto commented Dec 6, 2024

Jenkins Builds

Click to see older builds (72)
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ dc9415f #1 2024-12-06 13:16:17 ~4 min macos 📦zip
✔️ dc9415f #1 2024-12-06 13:16:28 ~4 min linux 📦zip
✔️ dc9415f #1 2024-12-06 13:16:30 ~4 min ios 📦zip
✖️ dc9415f #1 2024-12-06 13:16:35 ~4 min tests 📄log
✔️ dc9415f #1 2024-12-06 13:16:36 ~4 min windows 📦zip
✔️ dc9415f #1 2024-12-06 13:17:45 ~6 min android 📦aar
✖️ dc9415f #1 2024-12-06 13:18:36 ~6 min tests-rpc 📄log
✔️ dc9415f #1 2024-12-06 13:22:02 ~10 min macos 📦zip
✖️ e0e484f #2 2024-12-06 13:20:51 ~1 min tests 📄log
✔️ e0e484f #2 2024-12-06 13:22:37 ~3 min windows 📦zip
✔️ e0e484f #2 2024-12-06 13:23:45 ~4 min ios 📦zip
✔️ e0e484f #2 2024-12-06 13:25:25 ~5 min android 📦aar
✖️ e0e484f #2 2024-12-06 13:25:38 ~6 min tests-rpc 📄log
✔️ e0e484f #2 2024-12-06 13:25:45 ~6 min linux 📦zip
✔️ e0e484f #2 2024-12-06 13:25:54 ~6 min macos 📦zip
✔️ e0e484f #2 2024-12-06 13:30:46 ~8 min macos 📦zip
✖️ d2795bc #3 2024-12-06 13:25:08 ~1 min tests 📄log
✔️ d2795bc #3 2024-12-06 13:26:35 ~3 min windows 📦zip
✔️ d2795bc #3 2024-12-06 13:27:46 ~3 min ios 📦zip
✔️ d2795bc #3 2024-12-06 13:30:06 ~4 min macos 📦zip
✔️ d2795bc #3 2024-12-06 13:31:13 ~5 min android 📦aar
✔️ d2795bc #3 2024-12-06 13:31:24 ~5 min linux 📦zip
✖️ d2795bc #3 2024-12-06 13:31:35 ~5 min tests-rpc 📄log
✔️ d2795bc #3 2024-12-06 13:39:04 ~8 min macos 📦zip
526f19c #4 2025-01-07 14:37:19 ~25 sec windows 📄log
✖️ 526f19c #4 2025-01-07 14:38:26 ~1 min tests 📄log
✔️ 526f19c #4 2025-01-07 14:41:38 ~4 min macos 📦zip
✔️ 526f19c #4 2025-01-07 14:41:39 ~4 min linux 📦zip
✔️ 526f19c #4 2025-01-07 14:42:00 ~5 min android 📦aar
✔️ 526f19c #4 2025-01-07 14:42:02 ~5 min ios 📦zip
✖️ 526f19c #4 2025-01-07 14:42:50 ~5 min tests-rpc 📄log
✔️ 526f19c #4 2025-01-07 14:45:53 ~9 min macos 📦zip
54f6a89 #5 2025-01-09 13:35:45 ~25 sec windows 📄log
✖️ 54f6a89 #5 2025-01-09 13:36:48 ~1 min tests 📄log
✔️ 54f6a89 #5 2025-01-09 13:40:23 ~5 min macos 📦zip
✔️ 54f6a89 #5 2025-01-09 13:40:38 ~5 min linux 📦zip
✔️ 54f6a89 #5 2025-01-09 13:40:39 ~5 min ios 📦zip
✔️ 54f6a89 #5 2025-01-09 13:40:45 ~5 min android 📦aar
✖️ 54f6a89 #5 2025-01-09 13:41:21 ~6 min tests-rpc 📄log
✔️ 54f6a89 #5 2025-01-09 13:44:36 ~9 min macos 📦zip
54f6a89 #6 2025-01-10 22:02:59 ~26 sec windows 📄log
✖️ 54f6a89 #6 2025-01-10 22:03:58 ~1 min tests 📄log
✖️ 54f6a89 #6 2025-01-10 22:07:08 ~4 min tests-rpc 📄log
✔️ 54f6a89 #6 2025-01-10 22:07:33 ~5 min linux 📦zip
✔️ 54f6a89 #6 2025-01-10 22:07:39 ~5 min macos 📦zip
✔️ 54f6a89 #6 2025-01-10 22:07:55 ~5 min ios 📦zip
✔️ 54f6a89 #6 2025-01-10 22:08:13 ~5 min android 📦aar
✔️ 54f6a89 #6 2025-01-10 22:12:46 ~10 min macos 📦zip
8582775 #7 2025-01-13 12:02:36 ~25 sec windows 📄log
✖️ 8582775 #7 2025-01-13 12:03:38 ~1 min tests 📄log
✔️ 8582775 #7 2025-01-13 12:07:04 ~5 min linux 📦zip
✔️ 8582775 #7 2025-01-13 12:07:20 ~5 min android 📦aar
✔️ 8582775 #7 2025-01-13 12:07:21 ~5 min ios 📦zip
✔️ 8582775 #7 2025-01-13 12:07:27 ~5 min macos 📦zip
✖️ 8582775 #7 2025-01-13 12:07:55 ~5 min tests-rpc 📄log
✔️ 8582775 #7 2025-01-13 12:12:08 ~10 min macos 📦zip
0945def #8 2025-01-13 12:16:49 ~24 sec windows 📄log
✖️ 0945def #8 2025-01-13 12:17:37 ~1 min tests 📄log
✔️ 0945def #8 2025-01-13 12:21:07 ~4 min linux 📦zip
✔️ 0945def #8 2025-01-13 12:21:40 ~5 min macos 📦zip
✖️ 0945def #8 2025-01-13 12:21:41 ~5 min tests-rpc 📄log
✔️ 0945def #8 2025-01-13 12:21:49 ~5 min ios 📦zip
✔️ 0945def #8 2025-01-13 12:22:56 ~6 min android 📦aar
✔️ 0945def #8 2025-01-13 12:26:00 ~9 min macos 📦zip
6903ae1 #9 2025-01-13 14:42:50 ~26 sec windows 📄log
✔️ 6903ae1 #9 2025-01-13 14:46:11 ~3 min macos 📦zip
✔️ 6903ae1 #9 2025-01-13 14:47:24 ~5 min linux 📦zip
✖️ 6903ae1 #9 2025-01-13 14:47:33 ~5 min tests-rpc 📄log
✔️ 6903ae1 #9 2025-01-13 14:47:52 ~5 min android 📦aar
✖️ 6903ae1 #9 2025-01-13 14:48:59 ~6 min tests 📄log
✔️ 6903ae1 #9 2025-01-13 14:53:18 ~10 min macos 📦zip
✔️ 6903ae1 #9 2025-01-13 15:56:15 ~1 hr 14 min ios 📦zip
Commit #️⃣ Finished (UTC) Duration Platform Result
fdb7d1b #10 2025-01-14 10:44:18 ~25 sec windows 📄log
✖️ fdb7d1b #10 2025-01-14 10:45:20 ~1 min tests 📄log
✔️ fdb7d1b #10 2025-01-14 10:48:13 ~4 min macos 📦zip
✔️ fdb7d1b #10 2025-01-14 10:48:28 ~4 min linux 📦zip
✔️ fdb7d1b #10 2025-01-14 10:48:31 ~4 min ios 📦zip
✔️ fdb7d1b #10 2025-01-14 10:49:24 ~5 min macos 📦zip
✖️ fdb7d1b #10 2025-01-14 10:49:44 ~5 min tests-rpc 📄log
✔️ fdb7d1b #10 2025-01-14 10:49:46 ~6 min android 📦aar
a8959b8 #11 2025-01-14 10:45:37 ~24 sec windows 📄log
✖️ a8959b8 #11 2025-01-14 10:46:46 ~1 min tests 📄log
✔️ a8959b8 #11 2025-01-14 10:52:47 ~4 min macos 📦zip
✔️ a8959b8 #11 2025-01-14 10:53:09 ~4 min linux 📦zip
✔️ a8959b8 #11 2025-01-14 10:53:24 ~4 min ios 📦zip
✔️ a8959b8 #11 2025-01-14 10:54:47 ~5 min macos 📦zip
✖️ a8959b8 #11 2025-01-14 10:55:06 ~5 min tests-rpc 📄log
✔️ a8959b8 #11 2025-01-14 10:55:43 ~5 min android 📦aar

@friofry friofry force-pushed the ab/issue-5997-rpc-providers-integration branch 2 times, most recently from e0e484f to d2795bc Compare December 6, 2024 13:23
@friofry
Copy link
Contributor Author

friofry commented Jan 6, 2025

todo:

  • > As discussed and agreed, we should ensure now that default status-provided networks are not stored in the database anymore, because it makes no sense and only adds confusion. We should only persist user-provided secrets here, but the status-provided secrets must remain built-in in the binary.

#6151 (comment)

@friofry
Copy link
Contributor Author

friofry commented Jan 6, 2025

@friofry friofry force-pushed the ab/issue-5997-rpc-providers-configuration branch 2 times, most recently from 1ecae6e to afb9730 Compare January 6, 2025 18:16
@friofry friofry force-pushed the ab/issue-5997-rpc-providers-integration branch from d2795bc to 526f19c Compare January 7, 2025 14:36
@friofry friofry requested a review from igor-sirotin January 7, 2025 14:49
@friofry friofry force-pushed the ab/issue-5997-rpc-providers-integration branch from 526f19c to 54f6a89 Compare January 9, 2025 13:34
@friofry friofry force-pushed the ab/issue-5997-rpc-providers-configuration branch 2 times, most recently from 1c51d12 to c1b756f Compare January 10, 2025 15:50
Base automatically changed from ab/issue-5997-rpc-providers-configuration to develop January 10, 2025 22:02
Copy link

codecov bot commented Jan 10, 2025

Codecov Report

Attention: Patch coverage is 59.81162% with 256 lines in your changes missing coverage. Please review.

Project coverage is 19.13%. Comparing base (f7d73be) to head (54f6a89).
Report is 7 commits behind head on develop.

Files with missing lines Patch % Lines
rpc/network/db/rpc_provider_db.go 59.85% 44 Missing and 13 partials ⚠️
params/networkhelper/provider_utils.go 43.52% 45 Missing and 3 partials ⚠️
rpc/network/db/network_db.go 60.65% 32 Missing and 16 partials ⚠️
rpc/network/network.go 42.64% 32 Missing and 7 partials ⚠️
params/networkhelper/validate.go 0.00% 27 Missing ⚠️
rpc/network/db/utils.go 63.63% 10 Missing and 6 partials ⚠️
rpc/client.go 73.80% 8 Missing and 3 partials ⚠️
api/defaults.go 0.00% 6 Missing ⚠️
params/network_config.go 86.66% 2 Missing ⚠️
services/connector/commands/test_helpers.go 0.00% 1 Missing ⚠️
... and 1 more

❗ There is a different number of reports uploaded between BASE (f7d73be) and HEAD (54f6a89). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (f7d73be) HEAD (54f6a89)
unit 1 0
Additional details and impacted files
@@             Coverage Diff              @@
##           develop    #6178       +/-   ##
============================================
- Coverage    60.05%   19.13%   -40.92%     
============================================
  Files          832      817       -15     
  Lines       109840   108221     -1619     
============================================
- Hits         65961    20713    -45248     
- Misses       36148    84685    +48537     
+ Partials      7731     2823     -4908     
Flag Coverage Δ
functional 19.13% <59.81%> (?)
unit ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
api/default_networks.go 100.00% <100.00%> (ø)
node/get_status_node.go 37.64% <100.00%> (-13.24%) ⬇️
params/config.go 23.23% <ø> (-45.80%) ⬇️
protocol/requests/create_account.go 16.66% <ø> (-16.67%) ⬇️
services/connector/commands/test_helpers.go 0.00% <0.00%> (-100.00%) ⬇️
services/connector/test_helpers.go 0.00% <0.00%> (-100.00%) ⬇️
params/network_config.go 86.66% <86.66%> (ø)
api/defaults.go 62.91% <0.00%> (-12.92%) ⬇️
rpc/client.go 57.54% <73.80%> (-6.81%) ⬇️
rpc/network/db/utils.go 63.63% <63.63%> (ø)
... and 5 more

... and 642 files with indirect coverage changes

@friofry friofry force-pushed the ab/issue-5997-rpc-providers-integration branch from 54f6a89 to 8582775 Compare January 13, 2025 12:01
Copy link
Collaborator

@igor-sirotin igor-sirotin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't check the wallet logic at all

api/default_networks.go Outdated Show resolved Hide resolved
api/default_networks_test.go Outdated Show resolved Hide resolved
@@ -117,19 +118,25 @@ func OverrideEmbeddedProxyProviders(networks []params.Network, enabled bool, use
return updatedNetworks
}

func deepCopyNetworks(networks []params.Network) []params.Network {
func DeepCopyNetwork(network params.Network) params.Network {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps this could be a method of params.network?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with Igor's suggestion for this and the function below.

rpc/client.go Outdated Show resolved Hide resolved
return fmt.Errorf("failed to begin transaction: %w", err)
}
defer func() {
if p := recover(); p != nil {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, this is an smart thing to rollback in case of a panic 👍
Perhaps we should do this for all db transactions? Or at least write it down as a guideline?

@@ -102,3 +102,14 @@ func CompareNetworksList(t require.TestingT, expectedNetworks, actualNetworks []
CompareNetworks(t, expectedNetwork, network)
}
}

func ConvertNetworksToPointers(networks []params.Network) []*params.Network {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
func ConvertNetworksToPointers(networks []params.Network) []*params.Network {
func DereferenceNetworks(networks []params.Network) []*params.Network {

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though Dereference is the opposite thing 🤔 😄

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PointerifyNetworks :)

Copy link
Contributor

@saledjenic saledjenic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@friofry nice and clean. Thanks.

*params.NewProxyProvider(chainID, "proxy-grove", proxyUrl(stageName, "grove", chainName, networkName), false),
// Direct providers
*params.NewDirectProvider(chainID, "direct-infura", "https://mainnet.infura.io/v3/", true),
*params.NewDirectProvider(chainID, "direct-grove", "https://eth-archival.rpc.grove.city/v1/", false),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dlipicar mentioned today that grove updated the format of their endpoints, maybe you can align with that?

Copy link
Contributor Author

@friofry friofry Jan 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@@ -117,19 +118,25 @@ func OverrideEmbeddedProxyProviders(networks []params.Network, enabled bool, use
return updatedNetworks
}

func deepCopyNetworks(networks []params.Network) []params.Network {
func DeepCopyNetwork(network params.Network) params.Network {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with Igor's suggestion for this and the function below.

* default_networks.go
  * explicit provider initialization with more granular config (rps limiter, order)
  * token overrides made more flexible, support not only infura and grove
* get_status_node.go
  * override status-proxy auth instead of passing override config to rpc/client.go
* config.go
  * ProviderConfig removed
* client.go
  * Now any provider can be enabled/disabled (if user wants to use only his custom RPC urls)
  * Use bearer auth instead of URL auth
  * Provider order is defined by default_networks.go
* add method to make a deepcopy of a network + tests
* improved logging
* improved memory allocation
@friofry friofry force-pushed the ab/issue-5997-rpc-providers-integration branch from 6903ae1 to fdb7d1b Compare January 14, 2025 10:43
@friofry friofry force-pushed the ab/issue-5997-rpc-providers-integration branch from fdb7d1b to a8959b8 Compare January 14, 2025 10:44
@friofry
Copy link
Contributor Author

friofry commented Jan 14, 2025

As discussed with @dlipicar, this PR requires integration on the desktop side.
We need to split the URL and auth token in the UI.
image

I've added 2 endpoints for this:

  • SetChainEnabled
  • SetChainUserRpcProviders

and maybe show list of all providers (not editable)

@friofry
Copy link
Contributor Author

friofry commented Jan 14, 2025

should be merged after #6228

@@ -18,19 +17,39 @@ const (
ArbitrumSepoliaChainID uint64 = 421614
sntSymbol = "SNT"
sttSymbol = "STT"
proxyNodefleet = "proxy-nodefleet"
Copy link
Contributor

@dlipicar dlipicar Jan 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: declare type/alias type to group all "provider IDs" or add some indication in the const name that we're talking about provider IDs


func mainnet(stageName string) params.Network {
chainID := MainnetChainID
Copy link
Contributor

@dlipicar dlipicar Jan 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: const these three for all chains

@@ -70,6 +70,15 @@ type Network struct {
RelatedChainID uint64 `json:"relatedChainId" validate:"omitempty"`
Copy link
Contributor

@dlipicar dlipicar Jan 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there seems to be a bunch of stuff to support adding user networks, we should probably add a field indicating if a network is embedded/user-added (for example, to see if we enable deleting it client-side) like we do for providers

Copy link
Contributor

@dlipicar dlipicar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

noice

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

5 participants