Fix becnhmark.py and add warning in cmake file

[JackyYin]

1. Currently target testsuite and benchmark rely on package lua and python3,
   I think it's better to give a warning message if any of the package is missing.

2. benchmark.py is still using python2 module commands, I replaced it with python3 module subprocess.

3. fix LWAN_PATH for the testrunner in benchmark.py.

4. remove argument of -j in weighttp command, currently command options in my environment:

root@934a87a992a1:/app/build/weighttp# weighttp
weighttp 0.4 - a lightweight and simple webserver benchmarking tool

error: missing url argument

weighttp <options> <url>
  -n num   number of requests    (mandatory)
  -t num   threadcount           (default: 1)
  -c num   concurrent clients    (default: 1)
  -k       keep alive            (default: no)
  -6       use ipv6              (default: no)
  -H str   add header to request
  -h       show help and exit
  -v       show version and exit

example: weighttpd -n 10000 -c 10 -t 2 -k -H "User-Agent: foo" localhost/index.html

related issue: #309

[lpereira]

Have you tested the script with the current version of weighttp with JSON output as default?

(I'm thinking about moving weighttp to the Lwan source tree and build it alongside Lwan -- this way it's easier to benchmark, as the tools can just call into the built binaries rather than requiring a non-standard version of the tool. It's just a single file so should be easy to hook it up.)

[JackyYin]

Have you tested the script with the current version of weighttp with JSON output as default?

(I'm thinking about moving weighttp to the Lwan source tree and build it alongside Lwan -- this way it's easier to benchmark, as the tools can just call into the built binaries rather than requiring a non-standard version of the tool. It's just a single file so should be easy to hook it up.)

Sorry @lpereira ,
Could you please provide the correct weighttp you are currently using?

I have tried another version of weighttp : https://github.com/gstrauss/weighttp/tree/rewrite.
which have following options:

root@934a87a992a1:/app/build/weighttp# weighttp -h

weighttp  - a lightweight and simple webserver benchmarking tool

weighttp <options> <URI>
  -n num     number of requests      (mandatory)
  -t num     thread count            (default: 1)
  -c num     concurrent clients      (default: 1)
  -k         keep alive              (default: no)
  -K num     num pipelined requests  (default: 1)
  -6         use ipv6                (default: no)
  -i         use HTTP HEAD method    (default: GET)
  -m method  use custom HTTP method  (default: GET)
  -H str     add header to request ("label: value"); repeatable
  -b size    socket buffer sizes (SO_SNDBUF, SO_RCVBUF)
  -B addr    local address to bind to when making outgoing connections
  -C cookie  add cookie to request ("cookie-name=value"); repeatable
  -F         use TCP Fast Open (RFC 7413)
  -T type    Content-Type header to use for POST/PUT data,
             e.g. application/x-www-form-urlencoded
                                     (default: text/plain)
  -A string  add Basic WWW Authorization   (str is username:password)
  -P string  add Basic Proxy-Authorization (str is username:password)
  -X proxy   proxy:port or unix domain socket path beginning w/ '/'
  -p file    make HTTP POST request using file contents for body
  -u file    make HTTP PUT request using file contents for body
  -d         (ignored; compatibility with Apache Bench (ab))
  -l         (ignored; compatibility with Apache Bench (ab))
  -r         (ignored; compatibility with Apache Bench (ab))
  -q         quiet: do not show version header or progress
  -h         show help and exit
  -V         show version and exit

example:
  weighttpd -n 500000 -c 100 -t 2 -K 64 http://localhost/index.html

But still missing the -j option.

[lpereira]

This version should be fine. It doesn't have the -j option because it outputs JSON by default. The check can be removed from the script. Have you tested the script with this weighttp you have?

…

On Mon, Sep 13, 2021, 20:38 7FrogTW ***@***.***> wrote: Have you tested the script with the current version of weighttp with JSON output as default? (I'm thinking about moving weighttp to the Lwan source tree and build it alongside Lwan -- this way it's easier to benchmark, as the tools can just call into the built binaries rather than requiring a non-standard version of the tool. It's just a single file so should be easy to hook it up.) Sorry @lpereira <https://github.com/lpereira> , Could you please provide the correct weighttp you are currently using? I have tried another version of weighttp : https://github.com/gstrauss/weighttp/tree/rewrite. which have following options: ***@***.***:/app/build/weighttp# weighttp -h weighttp - a lightweight and simple webserver benchmarking tool weighttp <options> <URI> -n num number of requests (mandatory) -t num thread count (default: 1) -c num concurrent clients (default: 1) -k keep alive (default: no) -K num num pipelined requests (default: 1) -6 use ipv6 (default: no) -i use HTTP HEAD method (default: GET) -m method use custom HTTP method (default: GET) -H str add header to request ("label: value"); repeatable -b size socket buffer sizes (SO_SNDBUF, SO_RCVBUF) -B addr local address to bind to when making outgoing connections -C cookie add cookie to request ("cookie-name=value"); repeatable -F use TCP Fast Open (RFC 7413) -T type Content-Type header to use for POST/PUT data, e.g. application/x-www-form-urlencoded (default: text/plain) -A string add Basic WWW Authorization (str is username:password) -P string add Basic Proxy-Authorization (str is username:password) -X proxy proxy:port or unix domain socket path beginning w/ '/' -p file make HTTP POST request using file contents for body -u file make HTTP PUT request using file contents for body -d (ignored; compatibility with Apache Bench (ab)) -l (ignored; compatibility with Apache Bench (ab)) -r (ignored; compatibility with Apache Bench (ab)) -q quiet: do not show version header or progress -h show help and exit -V show version and exit example: weighttpd -n 500000 -c 100 -t 2 -K 64 http://localhost/index.html But still missing the -j option. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#311 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAADVGIFA5BHDNTAOZFWE3TUB27TVANCNFSM5D3W4FUQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[JackyYin]

This version should be fine. It doesn't have the -j option because it outputs JSON by default. The check can be removed from the script. Have you tested the script with this weighttp you have?

Yes, I have tested it using make benchmark, but I am currently getting only 400 reply (I think it's bacause of the missing 100.html), is it the expected behavior?
Also, a new commit is added for the new neighttp option and output: 09783cb

[JackyYin]

Sorry for my late update.
After moving testrunner.conf to my project root directory, I am able to receive 200 from make benchmark.

{
  "reqs_per_sec": 13564,
  "kBps_per_sec": 3881,
  "secs_elapsed": 73.724321,
  "request_counts": {
    "started": 1000000,
    "retired": 1000000,
    "total":   1000000
  },
  "response_counts": {
    "pass": 1000000,
    "fail": 0,
    "errs": 0
  },
  "status_codes": {
    "2xx":  1000000,
    "3xx":  0,
    "4xx":  0,
    "5xx":  0
  },
  "traffic": {
    "bytes_total":      293002400,
    "bytes_headers":    193002400,
    "bytes_body":       100000000
  }
}

[lpereira]

I'm sorry about taking so long to respond to this, but we're now building weighttp alongside Lwan, so updating the benchmark.py script should be a lot easier now!