feat: Set up comprehensive Python testing infrastructure with Poetry

[llbbl]

Set Up Python Testing Infrastructure

Summary

This PR establishes a comprehensive testing infrastructure for the CogAgent project using Poetry as the package manager and pytest as the testing framework. The setup provides a solid foundation for writing and running tests with proper dependency management, code coverage reporting, and test organization.

Changes Made

Package Management

• Migrated to Poetry: Created pyproject.toml with full Poetry configuration
• Consolidated dependencies: Merged dependencies from requirements.txt and app/requirements.txt
• Python version: Set to Python 3.10+ due to Gradio requirements
• Optional dependency groups: Created groups for finetune and vllm dependencies

Testing Framework

• Added testing dependencies:
  • pytest (^7.4.0) - Main testing framework
  • pytest-cov (^4.1.0) - Coverage reporting
  • pytest-mock (^3.11.0) - Mocking utilities

Test Configuration

• pytest configuration in pyproject.toml:

  • Test discovery patterns for test_*.py and *_test.py
  • Coverage reporting with 80% threshold
  • HTML and XML coverage output formats
  • Strict markers and configuration
  • Custom test markers: unit, integration, slow

• Coverage configuration:

  • Source directories: app, inference, finetune
  • Exclusions for test files, caches, and build artifacts
  • Branch coverage enabled
  • Detailed reporting options

Directory Structure

tests/
├── __init__.py
├── conftest.py          # Shared fixtures and configuration
├── test_setup_validation.py  # Infrastructure validation tests
├── unit/                # Unit tests directory
│   └── __init__.py
└── integration/         # Integration tests directory
    └── __init__.py

Test Fixtures (conftest.py)

Created comprehensive fixtures for common testing needs:

• temp_dir - Temporary directory management
• mock_model_config - Model configuration mocking
• mock_transformers_model - Transformers model mocking
• mock_tokenizer - Tokenizer mocking
• mock_image_data - Image data for testing
• mock_openai_client - OpenAI client mocking
• mock_gradio_interface - Gradio interface mocking
• env_setup - Environment variable setup
• mock_torch_cuda - CUDA availability mocking
• And more...

Development Commands

Added Poetry script commands:

• poetry run test - Run all tests
• poetry run tests - Alternative command (both work)

Git Configuration

Updated .gitignore with:

• Testing artifacts (.pytest_cache/, .coverage, htmlcov/, etc.)
• Claude settings (.claude/*)
• Note about not ignoring poetry.lock
• Additional IDE and build artifacts

Validation

• Created tests/test_setup_validation.py to verify the infrastructure
• Created run_validation.py script for quick setup verification

Instructions for Use

1. Install Poetry (if not already installed):

   curl -sSL https://install.python-poetry.org | python3 -

2. Install dependencies:

   poetry install --with dev

3. Run tests:

   poetry run test
   # or
   poetry run tests

4. Run tests with specific markers:

   poetry run pytest -m unit        # Run only unit tests
   poetry run pytest -m integration # Run only integration tests
   poetry run pytest -m "not slow"  # Skip slow tests

5. View coverage report:

   # After running tests, open the HTML report
   open htmlcov/index.html

Additional Notes

• The project requires Python 3.10+ due to Gradio dependency requirements
• Optional dependency groups can be installed with:
  • poetry install --with finetune for finetuning dependencies
  • poetry install --with vllm for VLLM server dependencies
• The testing infrastructure is ready for immediate use - developers can start writing tests in the appropriate directories
• All test fixtures are available globally through conftest.py
• Coverage threshold is set to 80% and will fail the test run if not met

Next Steps

With this infrastructure in place, the team can:

1. Start writing unit tests for individual modules
2. Create integration tests for end-to-end scenarios
3. Add performance benchmarks using the slow marker
4. Integrate testing into CI/CD pipelines
5. Track code coverage over time