Skip to content

Conversation

@matuszsmig
Copy link
Contributor

@matuszsmig matuszsmig commented Oct 20, 2024

This pull request introduces several new features and improvements to the ThreeEditor and WrapperApp components, focusing on enhancing the handling and display of simulation results. The changes include adding new types, updating existing components to fetch job results dynamically, and creating new components for better results presentation.

Key Changes:

Enhancements to Simulation Results Handling:

New Components for Result Presentation:

Updates to Existing Components:

Service Updates:

These changes collectively improve the user experience by ensuring that simulation results are fetched and displayed dynamically, providing a more responsive and informative interface.

@matuszsmig matuszsmig linked an issue Oct 20, 2024 that may be closed by this pull request
@matuszsmig
Copy link
Contributor Author

I am using here endpoint created in this PR: yaptide/yaptide#1140

@grzanka
Copy link
Contributor

grzanka commented Oct 20, 2024

@matuszsmig can I test it already now ?

@matuszsmig
Copy link
Contributor Author

@matuszsmig can I test it already now ?

yes

@matuszsmig
Copy link
Contributor Author

@grzanka do you have any other objections to this PR?

@matuszsmig matuszsmig marked this pull request as ready for review October 20, 2024 19:58
@grzanka
Copy link
Contributor

grzanka commented Oct 20, 2024

Looks OK, but I got a lot of following errors like:

Sort.ts:17 OrderElement peak not found

image

@grzanka
Copy link
Contributor

grzanka commented Oct 20, 2024

@grzanka do you have any other objections to this PR?

only this one: #1790 (comment)

@tudde
Copy link
Contributor

tudde commented Oct 20, 2024

@matuszsmig Aside from the errors mentioned above, the code looks great from what I've seen.

@matuszsmig
Copy link
Contributor Author

Looks OK, but I got a lot of following errors like:

Sort.ts:17 OrderElement peak not found

image

It tries to sort data, but whole data isn't fetched yet, so it isn't important.
Once whole data is being fetched this warning disappears

@grzanka
Copy link
Contributor

grzanka commented Oct 21, 2024

This PR breaks one essential functionality (which BTW needs to be described in the users manual - will you add an issue on that ?). If user sorts the quantities when defining simulation (Editor \ Scoring \ Outputs), the same order should be used when displaying results. See how it works on production now:

Screencast.from.2024-10-21.12-16-51.webm

In the video above I've changed the order of quantities in the simulation definition and the results were presented in a correct way.

This is how it behaves in current PR:

Screencast.from.2024-10-21.12-19-47.webm

I've changed the order of estimators in such way that YZ_Profile should be the last, but when displaying results I've seen that the last one was Peaks.

Copy link
Contributor

@grzanka grzanka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It breaks essential functionality, see my comment here:
#1790 (comment)

@grzanka
Copy link
Contributor

grzanka commented Oct 21, 2024

The order of elements (estimators in results tab) should be extracted from the JSON project only if the user defined project using Editor tab.

If the user defined the project using pyodide converter (which alas is undocumented) then the order of elements is not relevant, we display them in same order as they come from backend. See video below:

Screencast.from.2024-10-21.12-31-16.webm

@grzanka
Copy link
Contributor

grzanka commented Oct 21, 2024

@matuszsmig @tudde both options: defining simulation in editor and using pyodide converter needs to be tested to be sure that this PR is OK

@grzanka
Copy link
Contributor

grzanka commented Oct 21, 2024

Update the documentation here:
https://yaptide.github.io/deploy/sequence/

@github-actions
Copy link

On-demand deploy: succeeded ✅
https://github.com/yaptide/ui/actions/runs/11446212094

@matuszsmig matuszsmig marked this pull request as draft October 22, 2024 07:26
@grzanka
Copy link
Contributor

grzanka commented Oct 22, 2024

The warning:

Sort.ts:17 OrderElement yz_profile not found

is not yet gone.

Try removing yaptide_data volume, adding a fresh one, adding new user and running following scenario:

Screencast.from.2024-10-22.10-16-46.webm

There are following issues:

  • failed request to get results from DB: ShSimulatorService.tsx:369 GET https://127.0.0.1:8443/results?job_id=20241022-081723-23f33521-701d-4196-aa4e-f5475469be5bDIRECT&estimator_name=undefined_ 404 (NOT FOUND)
  • missing times in COMPLETED task - > you can ignore it now, but should be solved in subsequent PR, issue is needed

Here a detailed logs from yaptide_flask container:

grzanka@grzankax1:~/workspace/yaptide$ docker logs yaptide_flask 
/usr/local/lib/python3.12/site-packages/paramiko/pkey.py:100: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from this module in 48.0.0.
  "cipher": algorithms.TripleDES,
/usr/local/lib/python3.12/site-packages/paramiko/transport.py:259: CryptographyDeprecationWarning: TripleDES has been moved to cryptography.hazmat.decrepit.ciphers.algorithms.TripleDES and will be removed from this module in 48.0.0.
  "class": algorithms.TripleDES,
 * Serving Flask app 'yaptide.application'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:6000
 * Running on http://172.19.0.4:6000
Press CTRL+C to quit
127.0.0.1 - - [22/Oct/2024 08:12:53] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [22/Oct/2024 08:13:23] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [22/Oct/2024 08:13:53] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [22/Oct/2024 08:14:23] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [22/Oct/2024 08:14:54] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [22/Oct/2024 08:15:24] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [22/Oct/2024 08:15:54] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [22/Oct/2024 08:16:24] "GET / HTTP/1.1" 200 -
172.19.0.6 - - [22/Oct/2024 08:16:33] "GET / HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:16:33] "GET / HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:16:33] "GET /auth/refresh HTTP/1.0" 403 -
172.19.0.6 - - [22/Oct/2024 08:16:49] "OPTIONS /auth/login HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:16:49] "POST /auth/login HTTP/1.0" 202 -
127.0.0.1 - - [22/Oct/2024 08:16:54] "GET / HTTP/1.1" 200 -
172.19.0.6 - - [22/Oct/2024 08:17:08] "GET / HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:17:08] "GET / HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:17:08] "GET /user/simulations?page_idx=1&page_size=6&order_type=descend&order_by=start_time HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:08] "GET /user/simulations?page_idx=1&page_size=6&order_type=descend&order_by=start_time HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:17:08] "GET /user/simulations?page_idx=1&page_size=6&order_type=descend&order_by=start_time HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:08] "GET /user/simulations?page_idx=1&page_size=6&order_type=descend&order_by=start_time HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:17:11] "OPTIONS /jobs/direct HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:11] "OPTIONS /jobs/direct HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:17:12] "POST /jobs/direct HTTP/1.0" 202 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:12] "POST /jobs/direct HTTP/1.0" 202 -
172.19.0.6 - - [22/Oct/2024 08:17:12] "GET / HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:12] "GET / HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:17:12] "GET /user/simulations?page_idx=1&page_size=6&order_type=descend&order_by=start_time HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:12] "GET /user/simulations?page_idx=1&page_size=6&order_type=descend&order_by=start_time HTTP/1.0" 200 -
WARNING:root:Simulation state not updated, skipping commit
172.19.0.6 - - [22/Oct/2024 08:17:12] "GET /jobs?job_id=20241022-081711-b167a8b2-3bef-4de3-89d0-c253f9b41b92DIRECT HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:12] "GET /jobs?job_id=20241022-081711-b167a8b2-3bef-4de3-89d0-c253f9b41b92DIRECT HTTP/1.0" 200 -
172.19.0.5 - - [22/Oct/2024 08:17:12] "POST /tasks HTTP/1.1" 202 -
INFO:werkzeug:172.19.0.5 - - [22/Oct/2024 08:17:12] "POST /tasks HTTP/1.1" 202 -
WARNING:root:Simulation state not updated, skipping commit
172.19.0.6 - - [22/Oct/2024 08:17:12] "GET /jobs?job_id=20241022-081711-b167a8b2-3bef-4de3-89d0-c253f9b41b92DIRECT HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:12] "GET /jobs?job_id=20241022-081711-b167a8b2-3bef-4de3-89d0-c253f9b41b92DIRECT HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:17:12] "GET /inputs?job_id=20241022-081711-b167a8b2-3bef-4de3-89d0-c253f9b41b92DIRECT HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:12] "GET /inputs?job_id=20241022-081711-b167a8b2-3bef-4de3-89d0-c253f9b41b92DIRECT HTTP/1.0" 200 -
172.19.0.5 - - [22/Oct/2024 08:17:13] "POST /tasks HTTP/1.1" 202 -
INFO:werkzeug:172.19.0.5 - - [22/Oct/2024 08:17:13] "POST /tasks HTTP/1.1" 202 -
WARNING:root:Simulation state not updated, skipping commit
172.19.0.6 - - [22/Oct/2024 08:17:13] "GET /jobs?job_id=20241022-081711-b167a8b2-3bef-4de3-89d0-c253f9b41b92DIRECT HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:13] "GET /jobs?job_id=20241022-081711-b167a8b2-3bef-4de3-89d0-c253f9b41b92DIRECT HTTP/1.0" 200 -
172.19.0.5 - - [22/Oct/2024 08:17:14] "POST /results HTTP/1.1" 202 -
INFO:werkzeug:172.19.0.5 - - [22/Oct/2024 08:17:14] "POST /results HTTP/1.1" 202 -
172.19.0.6 - - [22/Oct/2024 08:17:15] "GET /jobs?job_id=20241022-081711-b167a8b2-3bef-4de3-89d0-c253f9b41b92DIRECT HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:15] "GET /jobs?job_id=20241022-081711-b167a8b2-3bef-4de3-89d0-c253f9b41b92DIRECT HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:17:15] "GET /results?job_id=20241022-081711-b167a8b2-3bef-4de3-89d0-c253f9b41b92DIRECT&estimator_name=z_profile_ HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:15] "GET /results?job_id=20241022-081711-b167a8b2-3bef-4de3-89d0-c253f9b41b92DIRECT&estimator_name=z_profile_ HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:17:23] "OPTIONS /jobs/direct HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:23] "OPTIONS /jobs/direct HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:17:23] "POST /jobs/direct HTTP/1.0" 202 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:23] "POST /jobs/direct HTTP/1.0" 202 -
172.19.0.5 - - [22/Oct/2024 08:17:23] "POST /tasks HTTP/1.1" 202 -
INFO:werkzeug:172.19.0.5 - - [22/Oct/2024 08:17:23] "POST /tasks HTTP/1.1" 202 -
172.19.0.5 - - [22/Oct/2024 08:17:24] "POST /tasks HTTP/1.1" 202 -
INFO:werkzeug:172.19.0.5 - - [22/Oct/2024 08:17:24] "POST /tasks HTTP/1.1" 202 -
127.0.0.1 - - [22/Oct/2024 08:17:24] "GET / HTTP/1.1" 200 -
INFO:werkzeug:127.0.0.1 - - [22/Oct/2024 08:17:24] "GET / HTTP/1.1" 200 -
172.19.0.5 - - [22/Oct/2024 08:17:24] "POST /results HTTP/1.1" 202 -
INFO:werkzeug:172.19.0.5 - - [22/Oct/2024 08:17:24] "POST /results HTTP/1.1" 202 -
172.19.0.6 - - [22/Oct/2024 08:17:28] "GET / HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:28] "GET / HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:17:28] "GET / HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:28] "GET / HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:17:28] "GET /user/simulations?page_idx=1&page_size=6&order_type=descend&order_by=start_time HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:28] "GET /user/simulations?page_idx=1&page_size=6&order_type=descend&order_by=start_time HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:17:28] "GET /user/simulations?page_idx=1&page_size=6&order_type=descend&order_by=start_time HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:28] "GET /user/simulations?page_idx=1&page_size=6&order_type=descend&order_by=start_time HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:17:28] "GET /jobs?job_id=20241022-081723-23f33521-701d-4196-aa4e-f5475469be5bDIRECT HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:28] "GET /jobs?job_id=20241022-081723-23f33521-701d-4196-aa4e-f5475469be5bDIRECT HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:17:28] "GET /jobs?job_id=20241022-081723-23f33521-701d-4196-aa4e-f5475469be5bDIRECT HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:28] "GET /jobs?job_id=20241022-081723-23f33521-701d-4196-aa4e-f5475469be5bDIRECT HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:17:28] "GET /inputs?job_id=20241022-081723-23f33521-701d-4196-aa4e-f5475469be5bDIRECT HTTP/1.0" 200 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:28] "GET /inputs?job_id=20241022-081723-23f33521-701d-4196-aa4e-f5475469be5bDIRECT HTTP/1.0" 200 -
172.19.0.6 - - [22/Oct/2024 08:17:28] "GET /results?job_id=20241022-081723-23f33521-701d-4196-aa4e-f5475469be5bDIRECT&estimator_name=undefined_ HTTP/1.0" 404 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:28] "GET /results?job_id=20241022-081723-23f33521-701d-4196-aa4e-f5475469be5bDIRECT&estimator_name=undefined_ HTTP/1.0" 404 -
172.19.0.6 - - [22/Oct/2024 08:17:28] "GET /results?job_id=20241022-081723-23f33521-701d-4196-aa4e-f5475469be5bDIRECT&estimator_name=undefined_ HTTP/1.0" 404 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:28] "GET /results?job_id=20241022-081723-23f33521-701d-4196-aa4e-f5475469be5bDIRECT&estimator_name=undefined_ HTTP/1.0" 404 -
172.19.0.6 - - [22/Oct/2024 08:17:29] "GET /results?job_id=20241022-081723-23f33521-701d-4196-aa4e-f5475469be5bDIRECT&estimator_name=undefined_ HTTP/1.0" 404 -
INFO:werkzeug:172.19.0.6 - - [22/Oct/2024 08:17:29] "GET /results?job_id=20241022-081723-23f33521-701d-4196-aa4e-f5475469be5bDIRECT&estimator_name=undefined_ HTTP/1.0" 404 -
127.0.0.1 - - [22/Oct/2024 08:17:55] "GET / HTTP/1.1" 200 -
INFO:werkzeug:127.0.0.1 - - [22/Oct/2024 08:17:55] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [22/Oct/2024 08:18:25] "GET / HTTP/1.1" 200 -
INFO:werkzeug:127.0.0.1 - - [22/Oct/2024 08:18:25] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [22/Oct/2024 08:18:55] "GET / HTTP/1.1" 200 -
INFO:werkzeug:127.0.0.1 - - [22/Oct/2024 08:18:55] "GET / HTTP/1.1" 200 -

@grzanka
Copy link
Contributor

grzanka commented Oct 22, 2024

BTW - sorting seems to works, but warning is present.

@grzanka
Copy link
Contributor

grzanka commented Nov 19, 2024

@grzanka I have to map and get names of Estimators for Fluka otherwise results won't be shown

That's correct, but you need to have these "Estimator names" for all other simulators (i.e. SHIELD-HIT12A) as well, when running project from the editor.

I'd put the if on the border between user-uploaded simulations (of any simulator type) and Editor projects than between Fluka and SHIELD-HIT12A.

Below couple of fragments from ex1.json:
image
image

For example z_profile in UI tab for ex1.json should come from 0-th element of outputs list in scoringManager (under name) instead of from 0-th element of estimators (under name).

For SHIELD-HIT12A those two names are the same, but you should make a query to scoringManager, not to estimators.
For Fluka in estimators we find only numbers like 21,22... while in scoringManager we have well defined user names.

This way you can probably avoid dealing with trailing _ ...

Lets summarize, if I understand the situation correctly.

Master branch

Fluka project from Editor

Estimator names extracted from the output filenames (i.e. 21,22...)

SHIELD-HIT12A project from Editor

Estimator names extracted from the output filenames (i.e. 'AlongBeamAxis' )

any project from user-uploaded files or pyodide-generated files

Estimator names extracted from the output filenames (i.e. 21,22... for Fluka, 'MyFileName' for SHIELDHIT12A )

This PR

Fluka project from Editor

Estimator names extracted from the Editor project (i.e. 'AlongAxis')

SHIELD-HIT12A project from Editor

Estimator names extracted from the output filenames (i.e. 'AlongBeamAxis' )

any project from user-uploaded files or pyodide-generated files

Estimator names extracted from the output filenames (i.e. 21,22... for Fluka, 'MyFileName' for SHIELDHIT12A )

Desired state

Fluka or SHIELD-HIT12A project from Editor

Estimator names extracted from the Editor project (i.e. 'AlongAxis')

any project from user-uploaded files or pyodide-generated files

Estimator names extracted from the output filenames (i.e. 21,22... for Fluka, 'MyFileName' for SHIELDHIT12A )

@grzanka
Copy link
Contributor

grzanka commented Nov 19, 2024

@matuszsmig cross-check if all the examples have correctly set inputType, I see something weird in the ex*json files:

grzanka@grzankax1:~/workspace/ui/public/examples$ cat *json | grep inputType
		"inputType": "YAPTIDE_PROJECT",
		"inputType": "YAPTIDE_PROJECT",
		"inputType": "EDITOR",
		"inputType": "EDITOR",
		"inputType": "YAPTIDE_PROJECT",
		"inputType": "YAPTIDE_PROJECT",
		"inputType": "EDITOR",
		"inputType": "EDITOR",
		"inputType": "EDITOR",
		"inputType": "EDITOR",

All examples should be of the same type, namely generated with the Editor.

@grzanka grzanka mentioned this pull request Nov 19, 2024
@matuszsmig matuszsmig deployed to github-pages November 22, 2024 23:18 — with GitHub Actions Active
@matuszsmig matuszsmig requested a review from grzanka November 22, 2024 23:20
@grzanka
Copy link
Contributor

grzanka commented Nov 23, 2024

I've simulated a run with user input files and noticed strange field name on top level in JSON:
image

same for regular simulations:
image

what is that field , why is it in top level ?

JSON obtained by Save to file on the Results tab

Also - why do we see now pages in top level of saved JSON ?

@grzanka
Copy link
Contributor

grzanka commented Nov 23, 2024

After clicking on "Save to file" on Simulation Card we will get a JSON with only one estimator saved:

image

When loading such files in the yaptide (with logged out user) second estimator wants to be downloaded from backend:
image

@grzanka
Copy link
Contributor

grzanka commented Nov 25, 2024

@matuszsmig there is this PR #1782 which was merged today - check it if has some effect on your work

@grzanka
Copy link
Contributor

grzanka commented Dec 1, 2024

I've logged in as an user which has no simulations stored in the DB. I've loaded the example and tried to save it with its results to my hard disk as JSON file. yaptide started to query backend producing a lot of failed POST requests.
Why do yaptide calls backend if I load the data from the example and try to dump it to JSON ?

Screencast.from.2024-12-01.20-54-47.webm

@matuszsmig matuszsmig requested a review from grzanka December 2, 2024 18:45
@grzanka grzanka enabled auto-merge December 3, 2024 09:15
@grzanka grzanka added this pull request to the merge queue Dec 3, 2024
Merged via the queue into master with commit 26e4159 Dec 3, 2024
8 checks passed
@grzanka grzanka deleted the 1771-query-backend-only-for-specific-estimator branch December 3, 2024 09:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Query backend only for specific estimator

5 participants