Skip to content

Final Test Consolidation Plan

Current Messy Structure (Still!)

tests/
├── unit/              # ✅ 8 files - KEEP
├── coverage/          # ✅ 12 files - KEEP  
├── integration/       # ✅ 6 files - KEEP
├── core/              # ❓ 5 files - CONSOLIDATE
├── features/          # ❓ 7 files - CONSOLIDATE
├── enhanced_types/    # ❓ 3 files - CONSOLIDATE
├── benchmarks/        # ❓ 4 files - CONSOLIDATE
├── security/          # ❓ 1 file - CONSOLIDATE
└── 4 standalone files # ❓ CONSOLIDATE

Target Clean Structure (2-3 folders)

tests/
├── unit/              # Main functionality tests
├── coverage/          # Edge cases & coverage boosters
└── integration/       # Integration, features, security, benchmarks

Consolidation Actions

1. Move tests/core/ → tests/coverage/

Logic: Core edge cases belong with coverage boosters 

tests/core/test_security.py           → tests/coverage/test_security_edge_cases.py
tests/core/test_ultra_fast_path_coverage.py → tests/coverage/test_performance_paths.py  
tests/core/test_edge_cases.py         → tests/coverage/test_core_edge_cases.py (merge)
tests/core/test_dataframe_orientation_regression.py → tests/coverage/test_dataframe_edge_cases.py
tests/core/test_circular_references.py → tests/coverage/test_circular_reference_edge_cases.py

2. Move tests/features/ → tests/integration/

Logic: Feature tests are integration tests 

tests/features/test_ml_serializers.py → tests/integration/test_ml_integration.py
tests/features/test_chunked_streaming.py → tests/integration/test_streaming_integration.py
tests/features/test_auto_detection_and_metadata.py → tests/integration/test_auto_detection.py
tests/features/test_template_deserialization.py → tests/integration/test_template_integration.py
tests/features/test_utils.py → tests/integration/test_utilities_integration.py
tests/features/test_redaction.py → tests/integration/test_redaction_integration.py
tests/features/test_enhanced_error_handling.py → tests/integration/test_error_handling.py

3. Move tests/enhanced_types/ → tests/integration/

Logic: Type-specific tests are integration tests 

tests/enhanced_types/test_pandas_auto_detection.py → tests/integration/test_pandas_integration.py
tests/enhanced_types/test_numpy_auto_detection.py → tests/integration/test_numpy_integration.py
tests/enhanced_types/test_basic_type_enhancements.py → tests/integration/test_type_enhancements.py

4. Move tests/security/ → tests/coverage/

Logic: Security edge cases belong with coverage 

tests/security/test_security_attack_vectors.py → tests/coverage/test_security_edge_cases.py (merge)

5. Move tests/benchmarks/ → tests/integration/

Logic: Performance tests are integration tests 

tests/benchmarks/test_performance.py → tests/integration/test_performance.py
tests/benchmarks/test_memory_usage.py → tests/integration/test_memory_performance.py
tests/benchmarks/test_large_data.py → tests/integration/test_large_data_performance.py
tests/benchmarks/test_optimization_effectiveness.py → tests/integration/test_optimization_performance.py

6. Move Standalone Files → Appropriate Locations

tests/test_modern_api.py → tests/integration/test_api_integration.py
tests/test_configurable_caching.py → tests/integration/test_caching_integration.py
tests/test_new_ml_frameworks.py → tests/integration/test_new_ml_frameworks.py
tests/test_unified_ml_handlers.py → tests/integration/test_unified_ml_integration.py

Implementation Strategy

Phase 1: Merge Files with Same Purpose

  • Merge security files
  • Merge core edge case files
  • Combine similar functionality

Phase 2: Move and Rename

  • Move directories to target locations
  • Use descriptive names that indicate purpose
  • Update imports and references

Phase 3: Clean Up Empty Directories

  • Remove empty source directories
  • Update pytest configuration

Final Result

tests/
├── unit/              # 8 files - Main functionality
├── coverage/          # ~15 files - Edge cases, security, core edge cases
└── integration/       # ~20 files - Features, types, benchmarks, integration

Benefits

  • Clear structure: 3 directories max
  • Logical organization: Know exactly where each test belongs
  • Easy maintenance: No confusion about where to add new tests
  • Consistent naming: All files follow clear patterns

Rules for Future

  • Main functionality → tests/unit/
  • Edge cases/coverage → tests/coverage/
  • Everything else → tests/integration/