Skip to content

Bug: Integer overflow in test_cumulative_sums.py causes invalid results on sequences >= ~1,000,000 bits bits #15

Description

@rumasarasool

## Bug Description

test_cumulative_sums.py produces invalid p-values for any
bit sequence of approximately 1,000,000 bits or longer due to
integer overflow in the running cumulative sum calculation.

The test fails for ALL input sources at this length — including
os.urandom(), a cryptographically secure baseline expected to
reliably pass — making the test result meaningless rather than
informative.

## Location

File: nistrng/sp800_22r1a/test_cumulative_sums.py
Lines: 53–56

forward_sum += bits_copy[i]                           # line 53
backward_sum += bits_copy[bits_copy.size - 1 - i]    # line 54
forward_max = max(abs(forward_sum), forward_max)      # line 55
backward_max = max(abs(backward_sum), backward_max)   # line 56

## Root Cause

bits_copy is a numpy array whose element type defaults to a
small integer (e.g. int8 or uint8). Accumulating up to
1,000,000 values of ±1 into forward_sum via Python's +=
operator produces a running total that silently overflows the
storage type, wrapping to an incorrect value.

## Evidence

Python emits this RuntimeWarning during execution:
RuntimeWarning: overflow encountered in scalar add
RuntimeWarning: overflow encountered in scalar absolute

Both warnings point directly to lines 53–55 of this file.

## Reproduction

import numpy as np
from nistrng import run_all_battery, SP800_22R1A_BATTERY
import os

# os.urandom() should pass randomness tests reliably
n = 1_000_000
bits = np.unpackbits(
    np.frombuffer(os.urandom(n // 8), dtype=np.uint8)
).astype(np.int8)

results = run_all_battery(bits, SP800_22R1A_BATTERY, False)
for name, result in results:
    print(f"{name}: {'PASS' if result.passed else 'FAIL'} 
            p={result.score:.6f}")

Expected: cumulative sums PASS (os.urandom is a CSPRNG)
Actual: cumulative sums FAIL with p=0.000000

## Fix

Cast the bit array to int64 before the test runs:

bits_copy = bits_copy.astype(np.int64)

This prevents overflow because int64 can hold values up to
9,223,372,036,854,775,807 — far beyond what 1,000,000
accumulated ±1 values can produce.

## Context

Discovered during a university cybersecurity research project
(CosmicQKG — Cosmic Noise Quantum Key Generator) at COMSATS
University Islamabad, while validating randomness of IBM Quantum
hardware output using NIST SP 800-22. The overflow caused the
test to fail for every source tested including the os.urandom()
reference baseline, which led to the root-cause investigation.

nistrng version: 1.2.3
Python version: 3.11+
numpy version: 2.4.4

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions