Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 93 additions & 0 deletions docs/lotus-cases/CHATGPT_MOBILE_WEB_PUBLIC_JUDGMENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# ChatGPT mobile-web public Lotus judgment

**Verdict:** `ALLOW_BOUNDED_DIAGNOSTIC`
**Case:** `chatgpt-mobile-web-public-2026-07-21`
**Source:** `safal207/LiminalQAengineer` PR `#106`
**Exact source head:** `2407be212e19a393fcd0d8dd33d9fe444aea663b`

## Decision

The source audit supports a narrow, human-reviewed conclusion:

- the signed-out ChatGPT mobile-web entry passed the bounded route, horizontal-layout, compact-height, primary-control, layout-stability and event-delivery checks;
- a distinct mobile-user-agent delivery branch is confirmed, but is not a defect;
- one repeated first-party `console.error` exists on the public mobile login page;
- no visible login failure, uncaught page error, security impact or authenticated-chat impact was established.

`ALLOW_BOUNDED_DIAGNOSTIC` permits the P3 diagnostic to be retained for first-party engineering review. It does not permit a claim that login is broken, users are blocked, telemetry is lost, the mobile web is inferior to the native app, or a security vulnerability exists.

## Confirmed passes

1. Five public home profiles returned HTTP `200`.
2. No horizontal overflow was detected in either round.
3. The signed-out composer remained visible at `412×915` and `412×520`.
4. Critical mobile controls exposed `44×44` CSS-pixel boxes in the tested state.
5. The mobile login page retained provider, email and Continue choices without horizontal overflow.
6. Public-home CLS stayed between `0` and `0.0004` in the observation window.
7. `/unauth-mweb/events/` POSTs received successful HTTP `200/204` responses.

A pass is scoped to the exact signed-out profiles and observation window. It is not a global quality claim.

## Allowed observation

### Distinct mobile branch — `CONFIRMED_ARCHITECTURE_NOT_DEFECT`

At the same `412×915` viewport, desktop and Android mobile user-agents received different signed-out headings, header structure and CTA state. This establishes a mobile-user-agent branch that requires independent regression and experiment coverage.

It does not establish a user problem.

## Allowed diagnostic

### Public mobile-login console signal — `P3_DIAGNOSTIC`

The focused probe reproduced one first-party `console.error` in both rounds:

- console text: `JSHandle@error`;
- serialized values: empty object and `undefined`;
- first-party minified bundle source;
- no uncaught page error;
- no visible login-form failure.

The correct statement is:

> A stable first-party console diagnostic exists on the public mobile login page; its semantic cause and user impact are unknown.

The correct next action is first-party source-map resolution and explicit error-code logging, followed by a repeat of the public console-cleanliness check.

## Rejected claims

The judgment blocks the following:

- `/unauth-mweb/events/` delivery failed;
- public login is broken;
- authentication is unavailable;
- the console signal exposes a security vulnerability;
- users lose data or telemetry;
- the mobile composer is obstructed;
- the page has duplicate visible headings;
- the public mobile page has a confirmed touch-target accessibility failure;
- native-app behaviour is proven by the mobile-web run;
- authenticated long-chat, streaming, attachment, sidebar, Search, Projects, Work, billing, settings or offline behaviour has been assessed.

## Rejected detector outputs

- Event-request failure: rejected because successful HTTP `200/204` responses preceded the browser loading-aborted signal.
- Composer overlap: rejected after screenshot review showed an ancestor container rather than obstruction.
- Duplicate visible heading: rejected by the controlled browser matrix.
- Raw small-target count: insufficient because critical icons met the threshold and inline/link context was not adjudicated as a failed task.

## Unknowns preserved

- real virtual-keyboard and browser-chrome interaction;
- authenticated long-conversation navigation;
- streaming interruption and reconnect recovery;
- file, image, camera and attachment states;
- mobile Search sources, widgets, Projects, Work and plan-limit states;
- TalkBack, browser zoom and external-keyboard tasks;
- the semantic meaning of the opaque login Error object.

## Pythia boundary

This packet is public and audit-only. It does not log in, submit a prompt, access a private conversation, contact OpenAI, approve an external report, claim a vulnerability, deploy or merge.

Machine-readable judgment: `examples/lotus-cases/chatgpt-mobile-web-public-judgment-v1.json`.
107 changes: 107 additions & 0 deletions examples/lotus-cases/chatgpt-mobile-web-public-judgment-v1.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
{
"schema_version": "pythia-lotus-public-mobile-web-judgment-v1",
"case_id": "chatgpt-mobile-web-public-2026-07-21",
"source": {
"repository": "safal207/LiminalQAengineer",
"pull_request": 106,
"exact_head": "2407be212e19a393fcd0d8dd33d9fe444aea663b",
"result_path": "audits/chatgpt/mobile-web-public-result.json",
"diagnostic_path": "audits/chatgpt/mobile-web-diagnostics-result.json",
"baseline_run_id": 29783360123,
"diagnostic_run_id": 29783766882,
"baseline_artifact_sha256": "1be5ceda6b73ff4a92ff13fc793c22366a05c97069392ad2ac0ec4a3c5ae7316",
"diagnostic_artifact_sha256": "dc76eadf08f34a03273f95aee2ff3a7256b39c1a600af5cef91c0c0fd799056c"
},
"authority": {
"mode": "audit_only",
"ownership": false,
"approval": false,
"execution": false,
"account_access": false,
"prompt_submission": false,
"login_submission": false,
"private_conversation_access": false,
"external_submission": false,
"security_claim": false,
"deployment": false,
"delivery": false,
"merge": false
},
"verdict": "ALLOW_BOUNDED_DIAGNOSTIC",
"verdict_meaning": "Retain one repeated P3 public-login console diagnostic for first-party engineering review while preserving all user-impact, security, native-app and authenticated-chat claims as unproven.",
"confirmed_passes": [
"Five public home profiles returned HTTP 200.",
"No horizontal document overflow was detected.",
"The signed-out composer remained visible at 412x915 and 412x520.",
"Critical mobile controls exposed 44x44 CSS-pixel boxes in the tested state.",
"The mobile login layout retained provider, email and Continue choices.",
"Observed public-home CLS remained from 0 to 0.0004.",
"The mobile event POST endpoints returned HTTP 200 or 204."
],
"allowed_observations": [
{
"id": "mobile-user-agent-variant",
"judgment": "CONFIRMED_ARCHITECTURE_NOT_DEFECT",
"claim": "The signed-out mobile user-agent selects a materially different public branch at the same viewport.",
"user_impact_established": false
},
{
"id": "mobile-login-console-error",
"judgment": "P3_DIAGNOSTIC",
"claim": "One first-party console.error repeated on the public mobile login page.",
"user_impact_established": false,
"security_impact_established": false,
"visible_login_failure": false,
"next_test": "Resolve the first-party minified stack through source maps, emit an explicit error code/message if intentional, and repeat the public console-cleanliness check."
}
],
"rejected_signals": [
{
"id": "mobile-event-post-aborts",
"judgment": "REJECTED_FALSE_NETWORK_FAILURE",
"reason": "Successful HTTP 200 or 204 responses preceded the browser loading-aborted signal."
},
{
"id": "composer-overlap-detector",
"judgment": "REJECTED_FALSE_POSITIVE",
"reason": "Screenshot review showed an ancestor container rather than composer obstruction."
},
{
"id": "public-text-duplicate-heading-hypothesis",
"judgment": "REJECTED_BY_BROWSER_MATRIX",
"reason": "The controlled browser matrix found no duplicate visible heading."
},
{
"id": "small-target-detector",
"judgment": "INSUFFICIENT_FOR_ACCESSIBILITY_DEFECT",
"reason": "Critical icon controls met 44x44; remaining links and CTA need context and spacing review."
}
],
"blocked_claims": [
"The public mobile login flow is broken.",
"Authentication is unavailable on mobile web.",
"The mobile event requests failed to reach the server.",
"Required analytics or user data was lost.",
"The mobile composer is obstructed.",
"The page has duplicate visible headings.",
"The audit established a touch-target accessibility violation.",
"The audit established a security vulnerability.",
"The native Android or iOS application has the same behaviour.",
"Authenticated long-chat, streaming, attachments, Search, Projects, Work, settings, billing or offline recovery were assessed."
],
"unknowns": [
"Real virtual-keyboard and browser-chrome interaction.",
"Authenticated long-chat scrolling and return-to-latest behaviour.",
"Streaming interruption and reconnect recovery.",
"File, image, camera and attachment states.",
"Mobile Search sources, widgets, Projects, Work and plan-limit states.",
"TalkBack, browser zoom and external-keyboard behaviour.",
"The semantic cause of the opaque public-login Error object."
],
"recommended_human_actions": [
"Treat the signed-out mobile baseline as a scoped pass, not a universal product pass.",
"Review the one P3 diagnostic with first-party source maps.",
"Keep the event-delivery, overlap and duplicate-heading signals rejected.",
"Authorise a separate signed-in mobile-web audit before making claims about the chat product after login."
]
}
117 changes: 117 additions & 0 deletions tests/test_chatgpt_mobile_web_public_judgment.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,117 @@
from __future__ import annotations

import json
from pathlib import Path


ROOT = Path(__file__).resolve().parents[1]
PACKET_PATH = (
ROOT
/ "examples"
/ "lotus-cases"
/ "chatgpt-mobile-web-public-judgment-v1.json"
)
DOC_PATH = (
ROOT
/ "docs"
/ "lotus-cases"
/ "CHATGPT_MOBILE_WEB_PUBLIC_JUDGMENT.md"
)


def _packet() -> dict:
return json.loads(PACKET_PATH.read_text(encoding="utf-8"))


def test_judgment_is_bounded_and_non_executing() -> None:
packet = _packet()
authority = packet["authority"]

assert packet["verdict"] == "ALLOW_BOUNDED_DIAGNOSTIC"
assert authority["mode"] == "audit_only"
for field in (
"ownership",
"approval",
"execution",
"account_access",
"prompt_submission",
"login_submission",
"private_conversation_access",
"external_submission",
"security_claim",
"deployment",
"delivery",
"merge",
):
assert authority[field] is False


def test_only_one_user_impact_unknown_diagnostic_is_allowed() -> None:
packet = _packet()
observations = {item["id"]: item for item in packet["allowed_observations"]}

assert observations["mobile-user-agent-variant"]["judgment"] == (
"CONFIRMED_ARCHITECTURE_NOT_DEFECT"
)
assert observations["mobile-user-agent-variant"]["user_impact_established"] is False

diagnostic = observations["mobile-login-console-error"]
assert diagnostic["judgment"] == "P3_DIAGNOSTIC"
assert diagnostic["user_impact_established"] is False
assert diagnostic["security_impact_established"] is False
assert diagnostic["visible_login_failure"] is False
assert diagnostic["next_test"].strip()


def test_false_positive_signals_remain_rejected() -> None:
packet = _packet()
rejected = {item["id"]: item for item in packet["rejected_signals"]}

assert rejected["mobile-event-post-aborts"]["judgment"] == (
"REJECTED_FALSE_NETWORK_FAILURE"
)
assert rejected["composer-overlap-detector"]["judgment"] == (
"REJECTED_FALSE_POSITIVE"
)
assert rejected["public-text-duplicate-heading-hypothesis"]["judgment"] == (
"REJECTED_BY_BROWSER_MATRIX"
)
assert rejected["small-target-detector"]["judgment"] == (
"INSUFFICIENT_FOR_ACCESSIBILITY_DEFECT"
)


def test_authenticated_native_and_security_claims_are_blocked() -> None:
blocked = "\n".join(_packet()["blocked_claims"])

assert "login flow is broken" in blocked
assert "event requests failed" in blocked
assert "security vulnerability" in blocked
assert "native Android or iOS" in blocked
assert "Authenticated long-chat" in blocked


def test_exact_source_and_artifact_digests_are_preserved() -> None:
source = _packet()["source"]

assert source["repository"] == "safal207/LiminalQAengineer"
assert source["pull_request"] == 106
assert source["exact_head"] == "2407be212e19a393fcd0d8dd33d9fe444aea663b"
assert source["baseline_run_id"] == 29783360123
assert source["diagnostic_run_id"] == 29783766882
assert len(source["baseline_artifact_sha256"]) == 64
assert len(source["diagnostic_artifact_sha256"]) == 64


def test_human_document_preserves_scoped_pass_and_unknowns() -> None:
document = DOC_PATH.read_text(encoding="utf-8")

for required in (
"ALLOW_BOUNDED_DIAGNOSTIC",
"P3 diagnostic",
"not a defect",
"no visible login failure",
"authenticated",
"does not log in",
):
assert required in document
Loading