APIMART · multi-model AI API gateway

Move from benchmark results to a controlled API test using one multi-model gateway. Test these models through APIMART

Read the APIMART OpenAI-compatible API docs · Transparent disclosure: this repository is maintained by APIMART.

AI Generation Benchmarks

10 prompts × 3 models = 30 raw outputs you can inspect and reproduce.

Compare GPT Image 2, Flux 2 Flex, and Gemini 2.5 Flash Image Preview with the same prompt fixtures. Every result keeps the original image, model identifier, parameters, latency, retry count, reported API cost, and dated review notes together.

Explore the visual benchmark · Browse 50 intent-led guides · View all 30 outputs

Maintained by APIMart Labs. The APIMart links in this repository are promotional. The prompts, raw results, and code examples remain available without creating an account or starring the repository.

Measured snapshot

Tested on 2026-07-21 with ten shared prompt fixtures and one accepted output per model and fixture. This is a dated workflow benchmark, not a universal model ranking.

Model	Outputs	Reported cost	Average latency	Latency range
GPT Image 2	10/10	$0.0850	79.3 s	49–159 s
Flux 2 Flex	10/10	$0.7000	33.2 s	22–43 s
Gemini 2.5 Flash Image Preview	10/10	$0.1250	32.5 s	32–34 s
Total	30/30	$0.9100	—	—

Two Flux tasks and one Gemini task needed a second attempt. Failed attempts returned no cost and are excluded from the total.

Same prompt, three model APIs

Product photography

Open the measured comparison →

GPT Image 2	Flux 2 Flex	Gemini 2.5 Flash

Dashboard UI

Open the measured comparison →

GPT Image 2	Flux 2 Flex	Gemini 2.5 Flash

Architecture rendering

Open the measured comparison →

GPT Image 2	Flux 2 Flex	Gemini 2.5 Flash

Poster typography

Open the measured comparison →

GPT Image 2	Flux 2 Flex	Gemini 2.5 Flash

What is included

• 50 structured prompt fixtures across ecommerce, advertising, characters, design, editing, food, architecture, and technical visualization.
• 50 search-intent guides covering named-model APIs, model comparisons, buyer workflows, reliability, automation, prompt libraries, and production evaluation.
• 10 measured comparison pages with three raw outputs, cost, latency, retry data, a shared prompt, and a dated finding.
• Python, JavaScript, and cURL recipes using environment-based credentials.
• Automated validation and GitHub Pages deployment from the reviewed source data.

Popular starting points:

• GPT Image prompt examples
• Nano Banana Pro prompt examples
• AI product photography prompts
• Character turnaround prompts
• AI image editing prompts
• AI architecture rendering prompts
• GPT Image 2 vs Flux 2 Flex
• AI image API latency comparison
• AI image API output validation

Reproduce a run

git clone https://github.com/luyx-66/ai-generation-benchmarks.git
cd ai-generation-benchmarks
npm install

Store the API key in the environment rather than source control:

export APIMART_API_KEY="your-key"
npm run benchmark:smoke -- --batch --max-cost=0.85 --stop-on-error

The runner submits one task at a time, polls asynchronous status, downloads expiring outputs, resumes completed records, and supports a reported-cost ceiling.

Run through one API gateway

APIMart is a multi-model AI API gateway with an OpenAI-compatible endpoint. Use one account to access supported image models, compare current pricing, and choose a cost-efficient option for each workload.

Create an APIMart account to reproduce the suite →

High-volume AI API toolkit

• LLM API Cost Calculator — forecast token, image, and video API spend
• AI API Load Tester — measure latency, throughput, and rate-limit behavior
• AI Image Generation API Batch — run resumable bulk image jobs with retries and cost caps
• Multi-Provider LLM API Examples — call text, image, and video models through one gateway
• Sora 2 AI Video Generation API Examples — submit, poll, and budget asynchronous video batches

Repository map

content/                 Reviewed keyword and prompt source data
comparisons/             Generated Markdown comparison pages
guides/                  Generated Markdown prompt and API guides
examples/                Runnable Python, JavaScript, and cURL recipes
benchmarks/              Raw images, result records, and result schema
scripts/                 Content generation and validation
site/                    Generated GitHub Pages output
.github/workflows/       Quality checks and Pages deployment

Review-gated automation

content/keyword-expansion.json is the reviewed publishing queue. Every entry has a unique search intent, original description and angle, at least three reusable fixtures, measured-evidence links, and a publishAt date.

Each Monday, the scheduled-content workflow prepares newly due guides from that reviewed queue, runs the uniqueness, evidence, SEO, and conversion-link checks, and opens a draft pull request. Publication waits for review and merge. The Pages workflow deploys only from main. Use CONTENT_AS_OF=YYYY-MM-DD npm run generate:content to preview a future release locally.

Each Wednesday, the APIMART model-catalog monitor reads /v1/models, compares the public identifiers with the committed snapshot, and opens a draft pull request only when the catalog changes. Reading the catalog does not make a paid generation request, and credentials remain in GitHub Actions secrets.

Contribute a result

1. Add or improve a fixture in content/prompts.json.
2. Run the same fixture with the model identifier and parameters recorded.
3. Store the raw output reference and result record using benchmarks/schema.json.
4. Run npm test before opening a pull request.

Generated files are committed so GitHub search, repository visitors, and GitHub Pages see the same reviewed content.

Editorial rules

• Do not copy prompts, images, or tables without compatible licensing and attribution.
• Do not publish a universal model winner from one prompt or an undocumented run.
• Do not hard-code credentials in examples, fixtures, issues, or pull requests.
• Keep raw outputs and rejected findings visible when they affect the conclusion.
• Label commercial relationships and keep promotional links relevant.
• Do not exchange rewards for stars or other engagement.

If this dataset is useful, star the repository to bookmark future measured runs and share a reproducible result through a pull request.

Project directory

This repository is part of the APIMART open-source AI API toolkit. Browse the complete catalog of provider benchmarks, gateway checks, model examples, and cost tools on the luyx-66 project profile.

License

Code is released under the MIT License. Prompt and written dataset content is released under CC BY 4.0.