Category Field Migration: snake_case → camelCase

Migration Date: 2025-11-26 Status: Code Changes Complete - Pending Database/Index Migration

Overview

This document tracks the migration of category fields from snake_case to camelCase across the CROP microservices codebase.

Fields Being Renamed

Motivation

• Consistency: Most fields in the codebase use camelCase (e.g., brandCode, partNumber, equipmentFitment)
• Legacy cleanup: Category fields were using snake_case from original implementation
• Frontend alignment: Frontend (CROP-front) does not use these snake_case fields
• Small dataset: Only ~10 documents in production MongoDB, trivial to migrate

Scope

In Scope

• ✅ Backend microservices (/Users/vova/Code/CROP/microservices/)
• ✅ TypeScript types and interfaces
• ✅ Zod validation schemas
• ✅ Elasticsearch mapping
• ✅ Database transformers
• ✅ API routes and handlers
• ✅ Test files
• ✅ OpenAPI documentation

Out of Scope

• ❌ Frontend (/Users/vova/Code/CROP/CROP-front/) - confirmed 0 occurrences
• ❌ Worktrees (/Users/vova/Code/CROP/wt-search/)
• ❌ node_modules

Migration Strategy

Phase 1: Code Preparation (COMPLETED)

1. ✅ Update TypeScript types to use camelCase
2. ✅ Update Zod schemas with backward compatibility
3. ✅ Update transformers to use camelCase
4. ✅ Update Elasticsearch mapping
5. ✅ Update all references in code
6. ✅ Update tests
7. ✅ Run typecheck to verify

Phase 2: Data Migration (MANUAL - TO BE EXECUTED)

2.1 Elasticsearch Reindexing

Requirement: Recreate Elasticsearch index with new field names

# 1. Backup current data (optional but recommended)
bun scripts/backup-index.ts --index parts_current

# 2. Run zero-downtime migration
bun scripts/migrate-index.ts --mapping-file scripts/elasticsearch-mapping.json

# This will:
# - Create new versioned index with updated mapping
# - Reindex all documents with new field names
# - Switch alias to new index
# - Delete old index (or keep with --keep-old flag)

Estimated Time: 5-10 minutes for ~10 documents Downtime: Zero (uses alias switching)

2.2 MongoDB Migration

Requirement: Update field names in MongoDB documents

Collections to Update:

• crop_stage.parts (~10 documents) - CURRENT PRODUCTION
• crop_prod.parts (3,740+ documents) - FUTURE PRODUCTION

Migration Script:

// Run this in MongoDB Shell or create a migration script

use crop_stage;
db.parts.updateMany(
  {},
  {
    $rename: {
      "category_name": "categoryName",
      "category_path": "categoryPath",
      "category_id": "categoryId"
    }
  }
);

use crop_prod;
db.parts.updateMany(
  {},
  {
    $rename: {
      "category_name": "categoryName",
      "category_path": "categoryPath",
      "category_id": "categoryId"
    }
  }
);

Estimated Time: < 1 minute per collection Rollback: Keep backup or use $rename to revert

2.3 Shared Types Package

Location: packages/shared-types/src/categories/normalizer.ts

This package is used by other services. Update field names in:

• ProductCategoryResult interface
• buildCategoryResult() function
• getCategorizeStats() function

Phase 3: Verification (MANUAL)

1. API Testing

# Test search with category filters (should work with camelCase)
curl "http://localhost:3001/api/search?categoryId=filters"
curl "http://localhost:3001/api/search?categoryPath=BATTERIES"

# Verify response contains camelCase fields
curl "http://localhost:3001/api/search?q=filter" | jq '.parts[0] | {categoryId, categoryName, categoryPath}'

2. Typecheck

cd /Users/vova/Code/CROP/microservices/services/search
bun run typecheck

3. Test Suite

cd /Users/vova/Code/CROP/microservices/services/search
bun test

Files Changed

Types & Interfaces

• ✅ /microservices/packages/shared-types/src/categories/normalizer.ts
• ✅ /microservices/services/search/src/types/part.ts

Schemas

• ✅ /microservices/services/search/src/schemas/parts.ts
• ✅ /microservices/services/search/src/schemas/search.ts

Elasticsearch

• ✅ /microservices/services/search/scripts/elasticsearch-mapping.json

Transformers & Utils

• ✅ /microservices/services/search/src/sync/transformers.ts
• ✅ /microservices/services/search/src/utils/query-builder.ts
• ✅ /microservices/services/search/src/utils/facets.ts
• ✅ /microservices/services/search/src/utils/search-params-normalizer.ts
• ✅ /microservices/services/search/src/utils/autocomplete-sections-service.ts

Routes

• ✅ /microservices/services/search/src/routes/search.ts
• ✅ /microservices/services/search/src/routes/parts.ts

OpenAPI

• ✅ /microservices/services/search/src/openapi.ts
• ✅ /microservices/services/search/openapi.json

Tests

• ✅ /microservices/services/search/src/__tests__/transformers.test.ts
• ✅ /microservices/services/search/src/__tests__/query-builder.test.ts
• ✅ /microservices/services/search/src/__tests__/routes/search.test.ts
• ✅ /microservices/services/search/src/__tests__/openapi-contract.test.ts

Documentation

• ✅ /microservices/services/search/docs/API.md
• ✅ /microservices/services/search/docs/INCONSISTENCIES_SUMMARY.md
• ✅ /microservices/services/search/docs/ARCHITECTURE_INCONSISTENCIES_ANALYSIS.md
• ⚠️ /microservices/docs/CATEGORY_TAXONOMY.md - may need updates

Backward Compatibility

API Parameters

The search API accepts both snake_case and camelCase for query parameters:

// Both formats work (via Zod schema normalization)
?categoryId=filters           // ✅ New (recommended)
?category_id=filters          // ✅ Legacy (still supported)

?categoryPath=BATTERIES       // ✅ New (recommended)
?category_path=BATTERIES      // ✅ Legacy (still supported)

Response Fields

After migration, API responses will only contain camelCase fields:

{
  "id": "part-123",
  "categoryId": ["filters"],
  "categoryName": ["Air Filters"],
  "categoryPath": ["FILTERS > Air Filters"]
}

Breaking Change: Clients using snake_case response fields must update to camelCase.

Rollback Plan

If issues are discovered:

1. Code Rollback

git revert <commit-hash>
bun run typecheck
bun test

2. Elasticsearch Rollback

# Rollback to previous index version
bun scripts/rollback-migration.ts --delete-failed

3. MongoDB Rollback

// Rename fields back to snake_case
db.parts.updateMany({}, {
  $rename: {
    "categoryName": "category_name",
    "categoryPath": "category_path",
    "categoryId": "category_id"
  }
});

Testing Checklist

• Typecheck passes
• Unit tests pass
• Integration tests pass
• Search API returns camelCase fields
• Category filters work with camelCase params
• Legacy snake_case params still work (backward compatibility)
• Facets aggregation works correctly
• Autocomplete returns category suggestions
• OpenAPI spec reflects new field names

Known Issues

None identified.

Post-Migration Tasks

1. Monitor Logs

   • Watch for any references to old snake_case fields
   • Check error rates in production

2. Update Client Code

   • Notify frontend team of field name changes
   • Update API integration tests

3. Deprecation Notices

   • Add deprecation warnings for snake_case params (Phase 2)
   • Plan to remove snake_case support in 6 months (Phase 3)

4. Documentation Updates

   • Update API documentation
   • Update integration guides
   • Update example code snippets

Timeline

• Phase 1 (Code): ✅ Completed 2025-11-26
• Phase 2 (Data): ⏳ Pending - Ready to execute
• Phase 3 (Verification): ⏳ Pending - After Phase 2

Contact

For questions about this migration:

• Engineer: Claude Code (via vova)
• Documentation: This file and related docs in /microservices/docs/

Last Updated: 2025-11-26 Migration Status: Code Complete, Database Migration Pending