scripts/runHorusec.sh no longer works

[davewichers]

I just tried running it like so, and I get errors:

BenchmarkJava % scripts/runHorusec.sh
time="2022-05-03T17:55:02Z" level=info msg="Set log file to /tmp/horusec/horusec-log-2022-05-03 17:55:02.log"
time="2022-05-03T17:55:20Z" level=warning msg="{HORUSEC_CLI} When starting the analysis WE SKIP A TOTAL OF 260 FILES that are not considered to be analyzed. To see more details use flag --log-level=debug"

time="2022-05-03T17:55:31Z" level=error msg="{HORUSEC_CLI} Error when copy project to .horusec folder" error="open /src/.horusec/36d20a05-c992-4668-9cac-977a1e716655/owasp-benchmark/src/.horusec/36d20a05-c992-4668-9cac-977a1e716655.zip: no such file or directory"
Error: open /src/.horusec/36d20a05-c992-4668-9cac-977a1e716655/owasp-benchmark/src/.horusec/36d20a05-c992-4668-9cac-977a1e716655.zip: no such file or directory
Usage:
horusec start [flags]

[davewichers]

@darkspirit510 - Hey Sascha - Can you investigate/fix this? I believe you wrote the original.

[darkspirit510]

What platform / docker are you using? Can you run

docker pull horuszup/horusec-cli

before running the script? Maybe you grabbed a broken version? On my Mac (OS 12.3.1 + Docker 4.0.1) it's taking way longer than I remember, but it's doing something.

[davewichers]

If you have to do that first, should we build that command into the script? If you already have the latest, that will be a no-op right? So won't hurt anything.

[darkspirit510]

Definitely, already "fixed" this. Also extended the command to ignore the results folder (which reduced the scan time to about 20 minutes) and fixed an issue with version extraction, but have some issues with pushing. Will create PR tomorrow.

[davewichers]

When I run this, I still don't get a results file in the /results directory. You are supposed to run all these scripts from the project home directory, so scripts/runHorusec.sh. I the results file path in the script is hardcoded to be:

result_file="/src/results/Benchmark_$benchmark_version-horusec-$horusec_version.json".

Where is this putting that results file? Inside the docker container somewhere. I tried this:

result_file="./results/Benchmark_$benchmark_version-horusec-$horusec_version.json"

And that didn't work either. The thought being that you are in the directory above /scripts. So ./results should go into the results folder, but no love.

Can you fix the script so when you run it, you end up with the results file in the /results folder? And this needs to be self relative as you have no idea where people will put this git repo.

[darkspirit510]

Sorry, missed that one.

The path can be absolute because in the docker run-command the current directory is bound to /src/. But tbh, i'm not sure if I ran the command from project directory or within the scripts folder. Will check this tomorrow!

[darkspirit510]

Okay, finally could make it. But - can't reproduce... As you mentioned You are supposed to run all these scripts from the project home directory, so scripts/runHorusec.sh:

root:BenchmarkJava/ # scripts/runHorusec.sh                                                                                                                                                                                                                                  [15:41:08]
Using default tag: latest
latest: Pulling from horuszup/horusec-cli
Digest: sha256:db96efa7ee0a3a23ab3d2c52a899fc46e34a40436e56ca441186b3f07a646f54
Status: Image is up to date for horuszup/horusec-cli:latest
docker.io/horuszup/horusec-cli:latest
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/usr/share/maven/lib/guice.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
time="2022-05-25T13:41:19Z" level=warning msg="{HORUSEC_CLI} Config file not found"
time="2022-05-25T13:41:19Z" level=warning msg="{HORUSEC_CLI} When starting the analysis WE SKIP A TOTAL OF 7342 FILES that are not considered to be analyzed. To see more details use flag --log-level=debug"

time="2022-05-25T13:41:25Z" level=warning msg="Horusec will return a timeout after 3600 seconds. This time can be customized in the cli settings."

time="2022-05-25T13:41:25Z" level=warning msg="{HORUSEC_CLI} PLEASE DON'T REMOVE \".horusec\" FOLDER BEFORE THE ANALYSIS FINISH! Don't worry, we'll remove it after the analysis ends automatically! Project sent to folder in location: [/src/.horusec/fa4cba84-ef1b-41e2-8a1f-0a0fd6d62fe5]"

time="2022-05-25T14:21:36Z" level=info msg="{HORUSEC_CLI} Writing output JSON to file in the path: /src/results/Benchmark_1.2-horusec-v2.7.1.json"

==================================================================================


time="2022-05-25T14:21:36Z" level=warning msg="{HORUSEC_CLI} No authorization token was found, your code it is not going to be sent to horusec. Please enter a token with the -a flag to configure and save your analysis"

time="2022-05-25T14:21:36Z" level=warning msg="{HORUSEC_CLI} 52068 VULNERABILITIES WERE FOUND IN YOUR CODE SENT TO HORUSEC, TO SEE MORE DETAILS USE THE LOG LEVEL AS DEBUG AND TRY AGAIN"

time="2022-05-25T14:21:36Z" level=warning msg="{HORUSEC_CLI} Horusec not show info vulnerabilities in this analysis, to see info vulnerabilities add option \"--information-severity=true\". For more details use (horusec start --help) command."

==================================================================================

time="2022-05-25T14:21:36Z" level=warning msg="{HORUSEC_CLI} During execution we found some problems:"

time="2022-05-25T14:21:36Z" level=warning msg="Error while running tool YarnAudit: {HORUSEC_CLI} Error It looks like your project doesn't have a yarn.lock file. If you use Yarn to handle your dependencies, it would be a good idea to commit it so horusec can check for vulnerabilities"
time="2022-05-25T14:21:36Z" level=warning msg=" Error while running tool NpmAudit: {HORUSEC_CLI} Error It looks like your project doesn't have a package-lock.json file. If you use NPM to handle your dependencies, it would be a good idea to commit it so horusec can check for vulnerabilities"

And afterwards

root:BenchmarkJava/ # ls -l results                                                                                                                                                                                                                               [18:50:50]
total 293M
[...]
-rw-r--r--  1 root   root    58M May 25 16:21 Benchmark_1.2-horusec-v2.7.1.json
[...]

[davewichers]

Why are running things as root? Shouldn’t be doing that.

…

On May 25, 2022, at 12:55 PM, Sascha Knoop ***@***.***> wrote:  Okay, finally could make it. But - can't reproduce... As you mentioned You are supposed to run all these scripts from the project home directory, so scripts/runHorusec.sh: root:BenchmarkJava/ # scripts/runHorusec.sh [15:41:08] Using default tag: latest latest: Pulling from horuszup/horusec-cli Digest: sha256:db96efa7ee0a3a23ab3d2c52a899fc46e34a40436e56ca441186b3f07a646f54 Status: Image is up to date for horuszup/horusec-cli:latest docker.io/horuszup/horusec-cli:latest WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/usr/share/maven/lib/guice.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1 WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release time="2022-05-25T13:41:19Z" level=warning msg="{HORUSEC_CLI} Config file not found" time="2022-05-25T13:41:19Z" level=warning msg="{HORUSEC_CLI} When starting the analysis WE SKIP A TOTAL OF 7342 FILES that are not considered to be analyzed. To see more details use flag --log-level=debug" time="2022-05-25T13:41:25Z" level=warning msg="Horusec will return a timeout after 3600 seconds. This time can be customized in the cli settings." time="2022-05-25T13:41:25Z" level=warning msg="{HORUSEC_CLI} PLEASE DON'T REMOVE \".horusec\" FOLDER BEFORE THE ANALYSIS FINISH! Don't worry, we'll remove it after the analysis ends automatically! Project sent to folder in location: [/src/.horusec/fa4cba84-ef1b-41e2-8a1f-0a0fd6d62fe5]" time="2022-05-25T14:21:36Z" level=info msg="{HORUSEC_CLI} Writing output JSON to file in the path: /src/results/Benchmark_1.2-horusec-v2.7.1.json" ================================================================================== time="2022-05-25T14:21:36Z" level=warning msg="{HORUSEC_CLI} No authorization token was found, your code it is not going to be sent to horusec. Please enter a token with the -a flag to configure and save your analysis" time="2022-05-25T14:21:36Z" level=warning msg="{HORUSEC_CLI} 52068 VULNERABILITIES WERE FOUND IN YOUR CODE SENT TO HORUSEC, TO SEE MORE DETAILS USE THE LOG LEVEL AS DEBUG AND TRY AGAIN" time="2022-05-25T14:21:36Z" level=warning msg="{HORUSEC_CLI} Horusec not show info vulnerabilities in this analysis, to see info vulnerabilities add option \"--information-severity=true\". For more details use (horusec start --help) command." ================================================================================== time="2022-05-25T14:21:36Z" level=warning msg="{HORUSEC_CLI} During execution we found some problems:" time="2022-05-25T14:21:36Z" level=warning msg="Error while running tool YarnAudit: {HORUSEC_CLI} Error It looks like your project doesn't have a yarn.lock file. If you use Yarn to handle your dependencies, it would be a good idea to commit it so horusec can check for vulnerabilities" time="2022-05-25T14:21:36Z" level=warning msg=" Error while running tool NpmAudit: {HORUSEC_CLI} Error It looks like your project doesn't have a package-lock.json file. If you use NPM to handle your dependencies, it would be a good idea to commit it so horusec can check for vulnerabilities" And afterwards root:BenchmarkJava/ # ls -l results [18:50:50] total 293M [...] -rw-r--r-- 1 root root 58M May 25 16:21 Benchmark_1.2-horusec-v2.7.1.json [...] — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were assigned.

[darkspirit510]

On my server I did not add any user to docker (ran it there because it is faster). Just did a clean checkout on my computer and used my personal account, same result:

sknoop:BenchmarkJava/ (master) $ scripts/runHorusec.sh                                                                                                                                   [12:34:44]
Using default tag: latest
latest: Pulling from horuszup/horusec-cli
59bf1c3509f3: Pull complete 
1ea03e1895df: Pull complete 
1ff98835b055: Pull complete 
a3f2dd7b7d65: Pull complete 
d182b62d4a35: Pull complete 
d7a57db2abd7: Pull complete 
73490af52bd3: Pull complete 
69ef757ff51c: Pull complete 
02b1b3930d32: Pull complete 
d7a7d2d6ddd3: Pull complete 
Digest: sha256:db96efa7ee0a3a23ab3d2c52a899fc46e34a40436e56ca441186b3f07a646f54
Status: Downloaded newer image for horuszup/horusec-cli:latest
docker.io/horuszup/horusec-cli:latest
time="2022-05-26T10:36:29Z" level=warning msg="{HORUSEC_CLI} Config file not found"
time="2022-05-26T10:36:36Z" level=warning msg="{HORUSEC_CLI} When starting the analysis WE SKIP A TOTAL OF 80 FILES that are not considered to be analyzed. To see more details use flag --log-level=debug"

time="2022-05-26T10:37:18Z" level=warning msg="Horusec will return a timeout after 3600 seconds. This time can be customized in the cli settings."

time="2022-05-26T10:37:18Z" level=warning msg="{HORUSEC_CLI} PLEASE DON'T REMOVE \".horusec\" FOLDER BEFORE THE ANALYSIS FINISH! Don’t worry, we’ll remove it after the analysis ends automatically! Project sent to folder in location: [/src/.horusec/2aef25f0-42f3-4713-a20a-98035b589bc6]"

time="2022-05-26T10:59:30Z" level=info msg="{HORUSEC_CLI} Writing output JSON to file in the path: /src/results/Benchmark_1.2-horusec-v2.7.1.json"

==================================================================================


time="2022-05-26T10:59:30Z" level=warning msg="{HORUSEC_CLI} No authorization token was found, your code it is not going to be sent to horusec. Please enter a token with the -a flag to configure and save your analysis"

time="2022-05-26T10:59:30Z" level=warning msg="{HORUSEC_CLI} 44772 VULNERABILITIES WERE FOUND IN YOUR CODE SENT TO HORUSEC, TO SEE MORE DETAILS USE THE LOG LEVEL AS DEBUG AND TRY AGAIN"

time="2022-05-26T10:59:30Z" level=warning msg="{HORUSEC_CLI} Horusec not show info vulnerabilities in this analysis, to see info vulnerabilities add option \"--information-severity=true\". For more details use (horusec start --help) command."

==================================================================================

time="2022-05-26T10:59:30Z" level=warning msg="{HORUSEC_CLI} During execution we found some problems:"

time="2022-05-26T10:59:30Z" level=warning msg="Error while running tool NpmAudit: {HORUSEC_CLI} Error It looks like your project doesn't have a package-lock.json file. If you use NPM to handle your dependencies, it would be a good idea to commit it so horusec can check for vulnerabilities"
time="2022-05-26T10:59:30Z" level=warning msg=" Error while running tool YarnAudit: {HORUSEC_CLI} Error It looks like your project doesn't have a yarn.lock file. If you use Yarn to handle your dependencies, it would be a good idea to commit it so horusec can check for vulnerabilities"

sknoop:BenchmarkJava/ (master✗) $ ls -l results                                                                                                                                              [12:59:42]
[...]
-rw-r--r--   1 sknoop  staff    49M May 26 12:59 Benchmark_1.2-horusec-v2.7.1.json
[...]

I really wonder why it works on both my laptop and server, but not on your machine 😞