adding small container scenarios

[mangod9]

Looking to add more container scenarios to measure 100m and 250m configurations.

[mangod9]

@sebastienros, want to add a couple of tiny container scenarios. Is there anything else needed for these to start showing up in the dashboard to trend over time?

fyi, @LoopedBard3, @DrewScoggins

[sebastienros]

This will just appear automatically.

My only concerns are that these should throttle a lot, maybe showing numbers that are useless, a dry run would tell. Also this job will take much longer (because all other dimensions are used so even one more value is doubling the number of jobs) so the timeouts might need to be updated, or the jobs reorganized (@LoopedBard3 to keep an eye on the timings).

[mangod9]

My only concerns are that these should throttle a lot, maybe showing numbers that are useless, a dry run would tell

the reason for this is a recent regression with lambda which shows up with < .5 cpu. Need to check whether time_to_first_response is a good metric for the cold start as measured in their scenario.

[sebastienros]

In tis case I think it would be worth creating a different job for that as we don't need to benchmark different max/set throughputs (these will be useless at that level of cpu anyway). And we could then run more often or on different machines.

[LoopedBard3]

I agree that a new containers startup or time to ready job would make sense. @mangod9 is the thought that the time to first response (for example) would include starting the container or is only the app startup what we want, or should either work? My understanding of the current flow from the Agents Startup.cs code in crank, the current flow for startup when docker is used to start the stopwatch on the start of the docker start command and then stopped when we either get the ready text from the app or when we are able to contact the application (which has a 300 ms timeout between attempts).

[mangod9]

chatted with @LoopedBard3 and we decided to merge as-is for now and based on the results possibly make a separate scenario for tiny containers.