Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cmd)_: status-backend #5847

Merged
merged 10 commits into from
Sep 27, 2024
Merged

feat(cmd)_: status-backend #5847

merged 10 commits into from
Sep 27, 2024

Conversation

igor-sirotin
Copy link
Collaborator

@igor-sirotin igor-sirotin commented Sep 18, 2024

Closes #5828

Description

This PR introduces a new tool for debugging and testing status-go.
In contrast to existing statusd and status-cli, the status-backend exposes full status-go API through HTTP.

This allows to communicate with status-go through HTTP the same way as status-desktop and status-mobile do, including: create account, restore account, login, logout, start messenger, start wallet and subscribe to status-go signals.

Implementation

  1. I decided to keep statusd as is. Reasoning here.
    Yet, when functional tests migrate to use status-bacend, we can remove the --seed-pharse argument from statusd. It was introduced specifically for it.

  2. @osmaczko I kept signals_server in statusd for easier review of the changes in it.
    After this is merged, I will open a separate PR with just moving and renaming the files.

  3. To continue the idea of a single server for both APIs:
    I found a better solution. When using status-backend, you don't need to start geth's RPC server at all. You can access services/* API through CallRPC endpoint, just the same as desktop/mobile do.

Usage

Start the app with the address to listen to:

status-backend --address localhost:12345

Access the exposed API with any HTTP client you prefer:

Simple flows

Create account and login

  1. InitializeApplication
  2. CreateAccountAndLogin
  3. wakuext_startMessenger
  4. wallet_startWallet
  5. settings_getSettings(temporary workaround, otherwise settings don't get saved into DB)

Login into account

  1. InitializeApplication
  2. LoginAccount
  3. wakuext_startMessenger
  4. wallet_startWallet

Requests collection

Not to pollute this PR, a follow-up PR with some basic requests is here:

@status-im-auto
Copy link
Member

status-im-auto commented Sep 18, 2024

Jenkins Builds

Click to see older builds (98)
Commit #️⃣ Finished (UTC) Duration Platform Result
✖️ 6696e6f #1 2024-09-18 22:44:28 ~2 min tests 📄log
✔️ 6696e6f #1 2024-09-18 22:44:43 ~2 min tests-rpc 📄log
✔️ 6696e6f #1 2024-09-18 22:46:15 ~4 min ios 📦zip
✔️ 6696e6f #1 2024-09-18 22:46:16 ~4 min linux 📦zip
✔️ 6696e6f #1 2024-09-18 22:47:39 ~5 min android 📦aar
✔️ 46ce050 #2 2024-09-18 23:17:25 ~1 min android 📦aar
✔️ 46ce050 #2 2024-09-18 23:17:58 ~2 min tests-rpc 📄log
✔️ 46ce050 #2 2024-09-18 23:18:07 ~2 min linux 📦zip
✖️ 46ce050 #2 2024-09-18 23:18:13 ~2 min tests 📄log
✔️ 46ce050 #2 2024-09-18 23:18:45 ~2 min ios 📦zip
✔️ 860b8dd #3 2024-09-19 07:10:08 ~1 min android 📦aar
✔️ 860b8dd #3 2024-09-19 07:10:30 ~2 min linux 📦zip
✔️ 860b8dd #3 2024-09-19 07:10:51 ~2 min tests-rpc 📄log
✔️ 860b8dd #3 2024-09-19 07:11:42 ~3 min ios 📦zip
✖️ 860b8dd #3 2024-09-19 07:40:40 ~32 min tests 📄log
✖️ 1acd8d8 #4 2024-09-19 14:56:28 ~1 min tests 📄log
✔️ 1acd8d8 #4 2024-09-19 14:56:49 ~1 min android 📦aar
✔️ 1acd8d8 #4 2024-09-19 14:57:20 ~2 min linux 📦zip
✔️ 1acd8d8 #4 2024-09-19 14:57:27 ~2 min tests-rpc 📄log
✔️ 1acd8d8 #4 2024-09-19 14:58:41 ~3 min ios 📦zip
✔️ bdbb9f6 #5 2024-09-19 16:30:06 ~1 min android 📦aar
✔️ bdbb9f6 #5 2024-09-19 16:30:33 ~2 min linux 📦zip
✔️ bdbb9f6 #5 2024-09-19 16:30:36 ~2 min tests-rpc 📄log
✔️ bdbb9f6 #5 2024-09-19 16:31:32 ~3 min ios 📦zip
✔️ bdbb9f6 #5 2024-09-19 16:59:20 ~30 min tests 📄log
✔️ f8c281c #6 2024-09-19 16:41:00 ~1 min android 📦aar
✔️ f8c281c #6 2024-09-19 16:41:31 ~2 min tests-rpc 📄log
✔️ f8c281c #6 2024-09-19 16:41:31 ~2 min linux 📦zip
✔️ f8c281c #6 2024-09-19 16:41:57 ~2 min ios 📦zip
✔️ 9f6bfa6 #7 2024-09-19 16:43:59 ~1 min android 📦aar
✔️ 9f6bfa6 #7 2024-09-19 16:44:26 ~2 min linux 📦zip
✔️ 9f6bfa6 #7 2024-09-19 16:44:34 ~2 min tests-rpc 📄log
✔️ 9f6bfa6 #7 2024-09-19 16:45:33 ~3 min ios 📦zip
✔️ 9f6bfa6 #6 2024-09-19 17:31:41 ~32 min tests 📄log
✖️ 1618aab #7 2024-09-19 22:53:55 ~1 min tests 📄log
✔️ 1618aab #8 2024-09-19 22:54:18 ~1 min android 📦aar
✔️ 1618aab #8 2024-09-19 22:54:45 ~2 min linux 📦zip
✔️ 1618aab #8 2024-09-19 22:54:52 ~2 min tests-rpc 📄log
✔️ 1618aab #8 2024-09-19 22:55:31 ~2 min ios 📦zip
✖️ a346466 #8 2024-09-20 22:59:35 ~1 min tests 📄log
✔️ a346466 #9 2024-09-20 22:59:58 ~1 min android 📦aar
✔️ a346466 #9 2024-09-20 23:00:27 ~2 min linux 📦zip
✔️ a346466 #9 2024-09-20 23:00:34 ~2 min tests-rpc 📄log
✔️ a346466 #9 2024-09-20 23:01:24 ~3 min ios 📦zip
✔️ 3c17a43 #10 2024-09-23 12:43:38 ~1 min android 📦aar
✔️ 3c17a43 #10 2024-09-23 12:44:02 ~2 min linux 📦zip
✔️ 3c17a43 #10 2024-09-23 12:44:11 ~2 min tests-rpc 📄log
✖️ 3c17a43 #9 2024-09-23 12:44:56 ~2 min tests 📄log
✔️ 3c17a43 #10 2024-09-23 12:45:03 ~3 min ios 📦zip
✖️ 963fe09 #10 2024-09-23 20:20:59 ~1 min tests 📄log
✔️ 963fe09 #11 2024-09-23 20:21:08 ~1 min android 📦aar
✔️ 963fe09 #11 2024-09-23 20:21:35 ~1 min linux 📦zip
✔️ 963fe09 #11 2024-09-23 20:21:50 ~2 min tests-rpc 📄log
✔️ 963fe09 #11 2024-09-23 20:22:38 ~3 min ios 📦zip
✖️ b8ee0e8 #11 2024-09-23 20:22:14 ~1 min tests 📄log
✔️ b8ee0e8 #12 2024-09-23 20:22:58 ~1 min android 📦aar
✔️ b8ee0e8 #12 2024-09-23 20:23:54 ~2 min linux 📦zip
✔️ b8ee0e8 #12 2024-09-23 20:24:05 ~2 min tests-rpc 📄log
✖️ 6a17e62 #12 2024-09-23 20:25:58 ~2 min tests 📄log
✔️ 6a17e62 #12 2024-09-23 20:27:59 ~4 min ios 📦zip
✔️ 6a17e62 #13 2024-09-23 20:28:11 ~4 min android 📦aar
✔️ 6a17e62 #13 2024-09-23 20:28:11 ~3 min tests-rpc 📄log
✔️ 6a17e62 #13 2024-09-23 20:28:26 ~4 min linux 📦zip
✖️ ac82091 #13 2024-09-23 20:31:47 ~2 min tests 📄log
✔️ ac82091 #14 2024-09-23 20:31:57 ~2 min tests-rpc 📄log
✔️ ac82091 #13 2024-09-23 20:32:27 ~2 min ios 📦zip
✔️ ac82091 #14 2024-09-23 20:33:28 ~3 min linux 📦zip
✔️ ac82091 #14 2024-09-23 20:34:14 ~4 min android 📦aar
✔️ 5462930 #15 2024-09-23 20:35:59 ~1 min android 📦aar
✔️ 5462930 #15 2024-09-23 20:36:40 ~2 min linux 📦zip
✔️ 5462930 #15 2024-09-23 20:36:56 ~2 min tests-rpc 📄log
✖️ 5462930 #14 2024-09-23 20:37:13 ~2 min tests 📄log
✔️ 5462930 #14 2024-09-23 20:37:28 ~2 min ios 📦zip
✔️ 16a4a48 #16 2024-09-23 21:00:43 ~1 min android 📦aar
✔️ 16a4a48 #16 2024-09-23 21:01:22 ~2 min tests-rpc 📄log
✔️ 16a4a48 #16 2024-09-23 21:01:23 ~2 min linux 📦zip
✖️ 16a4a48 #15 2024-09-23 21:01:49 ~2 min tests 📄log
✔️ 16a4a48 #15 2024-09-23 21:02:25 ~3 min ios 📦zip
✔️ 91cd92a #17 2024-09-23 21:42:10 ~1 min android 📦aar
✔️ 91cd92a #17 2024-09-23 21:42:46 ~2 min linux 📦zip
✔️ 91cd92a #17 2024-09-23 21:42:48 ~2 min tests-rpc 📄log
✔️ 91cd92a #16 2024-09-23 21:43:16 ~2 min ios 📦zip
✔️ 91cd92a #16 2024-09-23 22:11:28 ~30 min tests 📄log
✔️ 1d8e0b2 #18 2024-09-25 13:53:51 ~2 min android 📦aar
✔️ 1d8e0b2 #18 2024-09-25 13:53:57 ~2 min linux 📦zip
✔️ 1d8e0b2 #18 2024-09-25 13:54:24 ~2 min tests-rpc 📄log
✔️ 1d8e0b2 #17 2024-09-25 13:54:41 ~3 min ios 📦zip
✔️ 1d8e0b2 #17 2024-09-25 14:22:34 ~30 min tests 📄log
✔️ 181e58b #19 2024-09-26 14:38:18 ~1 min android 📦aar
✔️ 181e58b #19 2024-09-26 14:39:23 ~2 min tests-rpc 📄log
✔️ 181e58b #18 2024-09-26 14:39:37 ~2 min ios 📦zip
✔️ 181e58b #19 2024-09-26 14:41:03 ~4 min linux 📦zip
✔️ 181e58b #18 2024-09-26 15:09:29 ~32 min tests 📄log
✔️ 1ce9b15 #20 2024-09-26 15:18:44 ~1 min android 📦aar
✔️ 1ce9b15 #20 2024-09-26 15:19:23 ~2 min linux 📦zip
✔️ 1ce9b15 #20 2024-09-26 15:19:55 ~2 min tests-rpc 📄log
✔️ 1ce9b15 #19 2024-09-26 15:20:22 ~3 min ios 📦zip
✔️ 1ce9b15 #19 2024-09-26 15:47:50 ~30 min tests 📄log
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 2fa34eb #21 2024-09-26 15:21:30 ~2 min android 📦aar
✔️ 2fa34eb #21 2024-09-26 15:22:12 ~2 min linux 📦zip
✔️ 2fa34eb #21 2024-09-26 15:22:50 ~2 min tests-rpc 📄log
✔️ 2fa34eb #20 2024-09-26 15:23:39 ~3 min ios 📦zip
✔️ 2fa34eb #20 2024-09-26 16:19:25 ~31 min tests 📄log
✔️ 2ba2613 #22 2024-09-27 13:55:34 ~1 min android 📦aar
✔️ 2ba2613 #22 2024-09-27 13:56:12 ~2 min linux 📦zip
✔️ 2ba2613 #22 2024-09-27 13:56:49 ~2 min tests-rpc 📄log
✔️ 2ba2613 #21 2024-09-27 13:57:03 ~2 min ios 📦zip
✔️ 2ba2613 #21 2024-09-27 14:26:02 ~31 min tests 📄log

@igor-sirotin igor-sirotin force-pushed the feat/5828-status-backend branch 4 times, most recently from ac82091 to 5462930 Compare September 23, 2024 20:34
@@ -99,6 +99,7 @@ func initializeApplication(requestJSON string) string {
return string(data)
}

// Deprecated: Use InitializeApplication instead.
func OpenAccounts(datadir string) string {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing new here, it was deprecated already. It's just done for the private func:

status-go/mobile/status.go

Lines 107 to 109 in b3143ce

// DEPRECATED: use InitializeApplication
// openAccounts opens database and returns accounts list.
func openAccounts(datadir string) string {

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could have gathered the list of funcs manually, but it would be not simple to keep it up to date.

@igor-sirotin igor-sirotin changed the title feat(cmd)_: status-backend feat(cmd)_: status-backend Sep 23, 2024
@igor-sirotin igor-sirotin force-pushed the feat/5828-status-backend branch from 5462930 to 16a4a48 Compare September 23, 2024 20:58
@igor-sirotin igor-sirotin marked this pull request as ready for review September 23, 2024 21:32
Copy link
Contributor

@osmaczko osmaczko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

I like we ended up with one server, CallRPC endpoint is what I had in mind under term of "proxy", more or less.

@igor-sirotin igor-sirotin force-pushed the feat/5828-status-backend branch from 91cd92a to 1d8e0b2 Compare September 25, 2024 13:51
@codecov-commenter
Copy link

codecov-commenter commented Sep 25, 2024

Codecov Report

Attention: Patch coverage is 72.22222% with 15 lines in your changes missing coverage. Please review.

Please upload report for BASE (develop@55d9140). Learn more about missing BASE report.

✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
cmd/statusd/server/signals_server.go 71.69% 8 Missing and 7 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             develop    #5847   +/-   ##
==========================================
  Coverage           ?   46.06%           
==========================================
  Files              ?      884           
  Lines              ?   157156           
  Branches           ?        0           
==========================================
  Hits               ?    72397           
  Misses             ?    76447           
  Partials           ?     8312           
Flag Coverage Δ
functional 11.34% <16.66%> (?)
unit 45.64% <70.37%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
cmd/statusd/main.go 40.17% <100.00%> (ø)
mobile/status.go 4.26% <ø> (ø)
cmd/statusd/server/signals_server.go 71.71% <71.69%> (ø)

Copy link
Contributor

@qfrank qfrank left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the great work, a big step forward for a fast development experience although there still need some changes for mobile frontend. 🚀

}

if isDeprecated {
isDeprecated = false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should support deprecated by default and maybe add a switch to disable it. Consider that the mobile client will use the server one day. That would be a really cool experience for mobile developers as they won't need to endure the long wait to re-build with status-go with each modification to status-go to debug or do something else ...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense indeed, will do 👍

Copy link
Collaborator Author

@igor-sirotin igor-sirotin Sep 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@qfrank, done: 703e8a9

Deprecated methods are now available, but will have Deprecation: true header 🙌

Screenshot 2024-09-26 at 15 32 49

Copy link
Contributor

@fbarbu15 fbarbu15 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you please add a README.md with the Usage you describe in your PR description?

@igor-sirotin
Copy link
Collaborator Author

igor-sirotin commented Sep 26, 2024

Can you please add a README.md with the Usage you describe in your PR description?

@fbarbu15 sure, good point! Added: 2fa34eb


// Function to check if a comment indicates a deprecated function
func isDeprecatedComment(line string) bool {
return strings.Contains(line, "// Deprecated:")
Copy link
Contributor

@friofry friofry Sep 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sometimes it's // DEPRECATED or @deprecated

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True. It will work in this file for all public methods now, but will make it a bit smarter 👍

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

simple case-insensitive first comment word contains "deprecated" should do, right?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done: 2ba2613

Checking it like this:

deprecatedRegex = regexp.MustCompile(`(?i)//\s*Deprecated`)

Which matches https://go.dev/wiki/Deprecated.
Not bothering about @deprecated form, as it's not conventional in Go. Anyway, it's only used in wallet/api.go, and is not parsed by this program.

Copy link
Contributor

@friofry friofry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great tool!

Copy link
Contributor

@dlipicar dlipicar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

power/simplicity ratio off the charts, awesome job!

@igor-sirotin igor-sirotin merged commit fc36a7e into develop Sep 27, 2024
12 checks passed
@igor-sirotin igor-sirotin deleted the feat/5828-status-backend branch September 27, 2024 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Enable calling mobile/status.go API with an HTTP endpoint in statusd
8 participants