Personal Highlight Order Test

[abeddow91]

What does this change?

This PR introduces a client-side A/B/C test to evaluate whether reordering the highlights carousel based on user interactions (clicks and views) can improve engagement metrics.

We’ll measure engagement in two ways:

1. View count – logged whenever a card appears in one of the first two slots of the container.
2. Clicks – logged once when a user clicks on a card in the container.

Personalisation Mechanics

• Click tracking:
  • If a user clicks a card in the highlights container, the next time they visit that front, the clicked card will be moved to the back of the carousel.
• View tracking:
  • If a card has appeared in one of the first two slots three or more times, it will be moved to the back of the carousel on the third visit.

Test Variants
The test will have the following set up :

Variant	Behaviour
Control	No reordering (current experience)
Variant A	Click tracking only
Variant B	View tracking only
Variant C	Click + view tracking combined

Reset conditions

• If Editorial add or remove cards from the highlights container, the user’s local highlights order is reset.
• If Editorial only reorder the existing set of cards (no additions or removals), the user’s personalisation is preserved.
• Only one highlights container history is stored at a time. Moving between fronts (e.g., UK → US homepage) overwrites the previous front’s history.

How does it work?

Storage model
We are storing the users highlights history on their local storage, under the gu.history.highlights key. It is a stringified array of histories objects which take the following model:

{
card: DCRFrontCard;
viewCount: number;
wasClicked: boolean;
}

Reordering Logic
The viewCount metric is incremented whenever a card appears in one of the first two slots of the highlights container.
The wasClicked metric is set to true when a user clicks on the card.

Once a view count reaches 3 or higher, or a card has been clicked, the card is moved to the end of the array to allow "fresher" content to appear in view on next visit. Reordering is applied and persisted immediately in localStorage, but the visual change only takes effect on the next visit

Rendering behaviour
The highlights container is server-side rendered in the original order prescribed by the fronts tool. This ensures we know the heights of the container and prevent any CLS as the cards do not have a fixed height. On this first render, the container has visibility: hidden set that it does not appear to the user until we have validated the order, to prevent any flashing if the order changes.

Once the container is hydrated - which should be immediate as the container is now a critical island - the users highlights history is retrieved from local storage.

If the users history and the fronts trails contain all the same cards, we prefer the users history and rerender the container in the new order, setting visibility to true.

If the users history contains differerent cards to those in the fronts trails, we prefer the front trails in order to preserve editorial source-of-truth. We do not re-render, but we set these trails in local storage and set visibility to true

Why?

We would like to understand how personalising the highlight carousel might impact engagement by demoting unengaged or clicked content for otherwise buried cards in the first two slots.

The hypothesis is that users in the personalised variants will have higher CTR and more clicks per session than control.
We would also like to know if the personalised variants increase the number of unique cards clicked and shortens the time to first click.

Screenshots

• Click and View tracking combined
  https://github.com/user-attachments/assets/36343417-b642-48f3-8d80-74e07da21a1c

[github-actions]

Deploy build 46358 of Mobile::mobile-apps-rendering to CODE

All deployment options

• Deploy build 46358 of Mobile::mobile-apps-rendering to CODE
• Deploy parts of build 46358 to CODE by previewing it first
• What's on CODE right now?

---

From guardian/actions-riff-raff.

[github-actions]

Deploy build 22060 of dotcom:rendering-all to CODE

All deployment options

• Deploy build 22060 of dotcom:rendering-all to CODE
• Deploy parts of build 22060 to CODE by previewing it first
• What's on CODE right now?

---

From guardian/actions-riff-raff.

[github-actions]

Hello 👋! When you're ready to run Chromatic, please apply the run_chromatic label to this PR.

You will need to reapply the label each time you want to run Chromatic.

Click here to see the Chromatic project.

[abeddow91]

This TODO is not required for the test launch but worth considering for the future

[domlander]

Is there some complexity around using the card URL instead of the entire card object?

[domlander]

In the "both" variant, should "viewed" cards have priority over "clicked" cards? Maybe one for product? I think they should but understand it wasn't defined anywhere.

I don’t think we need to consider this for this AB test, but if we make this feature live, we should ask ourselves what behaviour we’d like to see if the user views the page lots of times without clicking on a card in the highlights container. Should the cards keep cycling around every three views, or should they reset to the curated order once every card has been viewed in a visible position at least three times.

What about users that have read an article in the highlights container, but didn’t click on the card in the highlights container, i.e. they clicked on a card elsewhere on the front? It looks like we store recently read articles in gu.history.articleCountsThisWeek.

[domlander]

I'm not sure why this is needed. useEffect will only run on the client and we shouldn't make visible this content until we have determined which order the cards should be displayed to the user.

[abeddow91]

In the "both" variant, should "viewed" cards have priority over "clicked" cards? Maybe one for product? I think they should but understand it wasn't defined anywhere.

I don’t think we need to consider this for this AB test, but if we make this feature live, we should ask ourselves what behaviour we’d like to see if the user views the page lots of times without clicking on a card in the highlights container. Should the cards keep cycling around every three views, or should they reset to the curated order once every card has been viewed in a visible position at least three times.

What about users that have read an article in the highlights container, but didn’t click on the card in the highlights container, i.e. they clicked on a card elsewhere on the front? It looks like we store recently read articles in gu.history.articleCountsThisWeek.

Thanks for writing this down! I 100% agree, there's a lot to think about in terms of how we might make this more sophisiticated. As you say, for the test, we've gone for a more simplified approach but I think testing other variations and optimisations would be a huge benefit.