Help devise (and agree on) a top-page/view performance-testing list for the Q3 rewrite/redesign

[stephendonner]

Hey team! I'm looking to help with the Q3 rewrite/redesign, performance and page-weight wise, so would you let me know your top page/view concerns, to help me prioritize testing?

(My main focus will be desktop, just FYI.)

I've picked off my obvious choices so far:

1. Homepage
2. Search
3. Discovery Pane
4. Most-popular Extensions and Themes

Anything else to add/remove/comment on?

Thanks!

/cc @krupa (whom I'm sure will see this anyway)

[kumar303]

That's awesome! Performance and page weight are crucial metrics (see also #10534 which is related).

I agree with your list but I'm wondering if we could also measure installation related performance too. I guess the flow is search -> click install button -> download XPI file so maybe just that last part is what we need to add. I believe the last step is fully hosted on the CDN (S3?) but it's probably still worth doing a smoke test on response time because we may need to adjust the region or add mirroring / other tweaks if we see a lot of slowness.

[stephendonner]

Thanks for the feedback, and suggestion/question, Kumar.

I certainly agree that the search -> click -> download flow is an important one, but it's not one I'm going to be able to have time/priority to address in the timeframe of my Q3 goal (I'm out all of October...) until I get the basics out of the way.

Reason being, those are scripted actions, in the world of the tools I'd be using (webpagetest and sitespeed.io, largely through their APIs).

Webpagetest (which is wrapped and used by sitespeed.io) has already been immediately useful in identifying a config issue on the rewrite, which we'll need to get squared away so we can start direct comparisons, where possible: https://bugzilla.mozilla.org/show_bug.cgi?id=1379064

@muffinresearch @tofumatt @digitarald - thoughts?

[stephendonner]

I've got the homepage, add-on detail page, Discovery Pane, search, themes, and extensions top-level pages all set up to test, here: https://github.com/stephendonner/webpagetest-amo/tree/6e5983686ffd18761d2f24742abb5eb61e368e51

Please take a look at, in particular: the set of metrics I'm asserting against in the respective *.json files.

As an example set of data, here's an example of what's available in a testrun against Chrome: https://www.webpagetest.org/result/170808_1Q_N5V/1/details/ and here's all of that same info, available via JSON. I can pretty much assert on/against any of those fields/values.

For now, to give us an idea, I've set this up in the various wpt-amo-* jobs on https://fx-test-jenkins-dev.stage.mozaws.net:8443/. How it works is that they assert that scores/values are equal or greater to a comparative value from prod, typically.

Thanks!

[muffinresearch]

I've got the homepage, add-on detail page, Discovery Pane, search, themes, and extensions top-level pages all set up to test, here: https://github.com/stephendonner/webpagetest-amo/tree/6e5983686ffd18761d2f24742abb5eb61e368e51

I think that's a really good list to start with.

[muffinresearch]

@stephendonner do you need any more information for this issue or are we good to close it?

[stephendonner]

@muffinresearch I'll close this, thanks; I can see from https://docs.google.com/document/d/1DEbsJKB5Q4J3EFrSmmO6I6GFs4qNfIFZ8auHRdS3Yww/edit#heading=h.7b0hgjodw2zz that the pages we care about for the OKRs this quarter are:

• home
• details
• search-results page

I've got those covered, and we have data for them via SpeedCurve, so let's follow-up separately on how we want to proceed with measuring and continuing to log Issues.

Thanks!