From lesion segmentation to DR grading, vessel extraction, ImageOnly VLM reading, and structured fusion reporting.
从病灶分割、DR 分级、血管提取，到 ImageOnly 独立阅片与结构化融合报告的一体化本地工作流。
English.
ADVANCED_DOCTOR_FOR_IDRID is a Windows-first local workstation that integrates deep lesion segmentation, optic-disc and fovea localization, DR grading, vessel segmentation, traditional CV baselines, a local MedGemma ImageOnly reading path, and an on-demand DeepSeek fusion report workflow into one browser-based interface.
中文。
ADVANCED_DOCTOR_FOR_IDRID 是一个面向 Windows 本地部署的眼底诊断工作站，把深度学习病灶分割、视盘/中心凹定位、DR 分级、血管分割、传统 CV 对照、MedGemma 的 ImageOnly 独立阅片，以及按需生成的 DeepSeek 融合病例报告，统一到了一个浏览器工作界面中。
Showcase images in this README are generated from real in-project analysis outputs and curated for lightweight repository display.
README 中的展示图来自项目真实分析产物，并经过轻量化整理以适合仓库展示。
- Local workstation / 本地化工作站: Launch from Python or packaged Windows EXE and open directly in the browser.
- Multitask lesion analysis / 多任务病灶分析: Segment MA, HE, EX, SE while preserving the original pretrained lesion pipeline.
- OD + fovea localization / 视盘与中心凹定位: Predict optic-disc segmentation and anatomical center coordinates in the same workstation flow.
- DR grading / DR 自动分级: Use lesion-aware grading logic as part of the structured analysis stack.
- Vessel segmentation / 血管分割: Run a dedicated vessel model alongside the lesion model.
- Traditional CV baselines / 传统 CV 对照: Keep optic-disc, EX, and vessel classical pipelines visible as reference evidence.
- ImageOnly VLM reading / 独立阅片路径: Use local MedGemma for a reading path that intentionally sees only the original fundus image.
- Fusion report / 融合病例报告: Generate a longer Chinese clinical-style report on demand by combining ImageOnly observations with quantitative model outputs.
- One-click packaging / 一键桌面化: Ship a PyInstaller-based Windows workstation bundle through Releases.
flowchart LR
A["Fundus Image<br/>眼底图像输入"] --> B["FastAPI Orchestrator<br/>后端任务编排"]
B --> C["IDRiD Multitask Model<br/>病灶分割 + OD + Fovea + DR Grading"]
B --> D["Vessel Segmentation Model<br/>血管分割模型"]
B --> E["Traditional CV Suite<br/>传统CV对照模块"]
B --> F["MedGemma ImageOnly<br/>仅看原图的独立阅片"]
C --> G["Structured Evidence<br/>结构化量化结果"]
D --> G
E --> G
F --> H["On-demand DeepSeek Fusion<br/>按需中文融合病例报告"]
G --> H
B --> I["React/Vite Workstation UI<br/>浏览器工作站前端"]
G --> I
F --> I
H --> I
English.
The workstation separates the independent image reading path from the structured evidence path. Local MedGemma reads the original image only, while the structured branch aggregates segmentation, grading, vessel ratio, and anatomical localization. A longer fusion report is then generated on demand.
中文。
这套系统明确区分“独立阅片路径”和“结构化证据路径”。本地 MedGemma 只看原始眼底图；结构化分支则汇总分割、分级、血管比例和解剖定位信息。随后再按需生成更长的中文融合病例报告。
English.
Single-image upload entry with a model status overview for the local workstation.
中文。
单图上传入口，同时展示本地工作站的核心模型状态概览。
English.
The workstation keeps the raw fundus and the main lesion overlay side by side for direct visual comparison.
中文。
工作站会把原始眼底图和主综合叠加结果并排展示，方便医生直接对照复核。
English.
The report area combines independent VLM reading, structured evidence, grading signals, and vessel statistics.
中文。
报告页同时容纳独立阅片、结构化证据、分级信号与血管统计，形成完整病例摘要入口。
English.
Masks, probability maps, and traditional CV artifacts remain visible instead of being hidden behind the final narrative.
中文。
掩码、概率图和传统 CV 中间结果不会被最终报告遮住，而是保留为可见的辅助证据层。
- Browser-based workstation / 浏览器工作站: local upload, polling, visualization, and download.
- MA / HE / EX / SE lesion masks / 四类病灶掩码: preserve multi-lesion outputs from the multitask pipeline.
- OD segmentation / 视盘分割: additional branch attached after the encoder.
- Fovea center prediction / 中心凹定位: explicit anatomical coordinate output.
- DR grade probabilities / DR 分级概率: structured class probabilities and threshold probabilities.
- Vessel ratio / 血管比例: vessel mask and vessel burden summary.
- Traditional CV evidence / 传统 CV 证据: optic-disc, EX, and vessel handcrafted baselines.
- ImageOnly report / ImageOnly 报告: original-image-only reading path from local MedGemma.
- Fusion report / 融合报告: on-demand Chinese narrative synthesized from ImageOnly findings and quantitative results.
- Packaging / 桌面打包: PyInstaller-based Windows bundle for one-click startup.
ADVANCED_DOCTOR_FOR_IDRID/
├─ backend/ # FastAPI backend, orchestrator, wrappers / 后端与模型封装
├─ mediseg_UI/ # React + Vite workstation frontend / 工作站前端
├─ Traditional_CV/ # Classical CV baselines / 传统CV基线模块
├─ IDRID/ # Multitask retinal model code + released checkpoint / 多任务模型代码与发布权重
├─ COMBINED_DATASET/ # Vessel model code + released checkpoint / 血管模型代码与发布权重
├─ VLM/ # Local VLM and API report code (without weights) / VLM相关代码（不含本地权重）
├─ docs/assets/ # README showcase images / README 展示素材
├─ main.py # One-click launcher / 一键启动器
├─ build_exe.py # PyInstaller build entry / EXE 打包脚本
└─ FundusWorkstation.spec # PyInstaller spec / 打包配置
English.
The project is tested in a conda environment named torch-eye-disease. You will need Python, PyTorch-compatible dependencies, and Node.js for the frontend source workflow.
中文。
项目默认在名为 torch-eye-disease 的 conda 环境中开发和测试。若从源码运行，需要 Python、PyTorch 相关依赖，以及前端所需的 Node.js。
conda create -n torch-eye-disease python=3.10
conda activate torch-eye-disease
pip install -r backend/requirements.txtcd mediseg_UI
npm install
cd ..python main.pyOptional launcher modes / 可选启动模式：
python main.py --backend-only
python main.py --no-browser
python main.py --integrated-uiEnglish.
The repository includes the PyInstaller entrypoint and build spec. Because a full local bundle with deep-learning runtime and optional local VLM assets can easily exceed normal GitHub release size limits, the most stable public path is: clone the repo, pull LFS checkpoints, and build the EXE locally with python build_exe.py.
中文。
仓库已经包含 PyInstaller 的构建入口和打包 spec。由于完整桌面包在集成深度学习运行时与可选本地 VLM 资源后，很容易超过 GitHub Release 的常规体积限制，当前最稳妥的公开使用方式是：克隆仓库、拉取 LFS 权重，然后本地执行 python build_exe.py 生成 EXE。
English.
The default workflow runs the local ImageOnly report first. The longer Chinese fusion report is generated on demand in the report page after you provide a DeepSeek API key in the UI.
中文。
默认流程会先运行本地 ImageOnly 报告。更长的中文融合病例报告需要在报告页中按需触发，并在前端界面里填写 DeepSeek API Key。
| Asset | Path | Delivery | Notes |
|---|---|---|---|
| Multitask IDRiD checkpoint | IDRID/multitask_runs/default/final_multitask_model_state.pth |
Git LFS | Main lesion + OD + localization + grading checkpoint |
| Vessel segmentation checkpoint | COMBINED_DATASET/checkpoints_combined/best_model.pth |
Git LFS | Dedicated vessel model checkpoint |
| Asset | Why it is not in the repo | Where to get it | Where to place it |
|---|---|---|---|
MedGemma 1.5 4B local weights |
Too large for normal repo publishing and subject to upstream distribution/licensing flow | Download from the official Hugging Face / Google release channel | VLM/models/google-medgemma-1.5-4b-it/ |
| Raw IDRiD dataset | Dataset redistribution should follow the original dataset terms | Obtain from the official IDRiD source | Keep locally; not required inside this repo for inference-only usage |
| Raw vessel training data | Large and not needed for open-source inference packaging | Use your local training data source | Keep outside the repository |
Runtime outputs (outputs/) |
Machine-specific generated artifacts | Produced automatically by the workstation | outputs/jobs/... |
Windows packaged bundle (dist/) |
Too large and binary-heavy for Git history | Download from Releases | Any local folder |
- Repository source / 源码仓库: ADVANCED_DOCTOR_FOR_IDRID
- Release page / 发布页: GitHub Releases
- Checkpoints / 权重文件: pulled via Git LFS when cloning the repository
- Packaged workstation / 桌面版工作站: the repository provides the build recipe; a fully bundled desktop artifact may be distributed separately when it fits the release-size budget
- First launch note / 首次启动提示: any locally built workstation may take noticeably longer on first launch because local models and dependencies are large
- Source code for backend, frontend, traditional CV, training and inference scripts
- Core configuration, launcher, build script, and packaging spec
- README showcase assets
- Two key released checkpoints through Git LFS
- Local MedGemma model weights
- Raw datasets and archived training folders
- Build outputs, packaged binaries, cached node modules, runtime jobs, and logs
- Personal temporary assets and unrelated local experiment folders
python build_exe.pyThis script builds the frontend first and then invokes PyInstaller using FundusWorkstation.spec.
该脚本会先构建前端，再使用 FundusWorkstation.spec 调用 PyInstaller 生成 Windows 工作站包。
- DME grading / DME 分级: extend the current report flow to include macular-edema-specific structured outputs.
- Stronger report alignment / 更稳的报告链路: continue improving agreement between ImageOnly reading, segmentation evidence, and fusion writing.
- Clinical workstation refinement / 医疗工作站增强: richer comparisons, cleaner export packages, and more explicit review ergonomics.
- Release automation / 发布自动化: smoother GitHub release publishing and artifact delivery.
If this repository helps your work, please cite the original dataset and upstream model resources appropriately.
- IDRiD dataset / IDRiD 数据集
- Porwal et al., Indian Diabetic Retinopathy Image Dataset (IDRiD): A Database for Diabetic Retinopathy Screening Research, Data, 2018.
- MedGemma / MedGemma
- Please cite the official Google / Hugging Face release associated with the model snapshot you use locally.
- PyTorch, Transformers, FastAPI, React, Vite
- Please also acknowledge the upstream frameworks that make this workstation possible.
This project stands on the shoulders of open datasets, open deep-learning frameworks, and the broader medical-imaging open-source ecosystem.
这个项目建立在公开数据集、开源深度学习框架，以及整个医学影像开源生态的基础之上。