fix: type errors and build issues

- Fix user.id -> user.userId in subscription and device registration
- Fix StatusIndicator to use DeviceStatus enum instead of strings
- Add Recharts type declarations for React 19 compatibility
- Add lightningcss WASM fallback for CI builds
- Remove unused LogsModule from backend
- Simplify CLAUDE.md documentation

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
grabbit 2026-01-11 17:58:30 +08:00
parent 10fe6f95dd
commit 2b79cf37d0
9 changed files with 2234 additions and 2257 deletions

270
CLAUDE.md
View File

@ -2,80 +2,68 @@
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Architecture
## Project Overview
This is a distributed meteor monitoring network with a microservices architecture:
Distributed meteor monitoring network with microservices architecture. Four main services communicate via REST APIs and SQS queues.
### Core Services
- **meteor-frontend/** - Next.js 15 frontend with React 19, TypeScript, TailwindCSS 4, and React Query
- **meteor-web-backend/** - NestJS backend API with PostgreSQL, TypeORM, JWT auth, and Stripe payments
- **meteor-compute-service/** - Go microservice for event processing with AWS SDK and PostgreSQL
- **meteor-edge-client/** - Rust edge client for Raspberry Pi devices with camera integration
## Core Services
### Key Technologies
- **Frontend**: Next.js 15, React 19, TypeScript, TailwindCSS 4, React Query, React Hook Form, Recharts
- **Backend**: NestJS, TypeORM, PostgreSQL, JWT, Stripe, AWS SDK (S3, SQS, CloudWatch), Pino logging
- **Processing**: Go 1.24, PostgreSQL (pgx), AWS SDK, structured logging
- **Edge**: Rust, Tokio, Serde, Reqwest, OpenCV (optional), cross-compilation for ARM64
| Service | Technology | Entry Point | Port |
|---------|------------|-------------|------|
| meteor-frontend | Next.js 15, React 19, TailwindCSS 4 | `src/app/layout.tsx` | 3000 |
| meteor-web-backend | NestJS, TypeORM, PostgreSQL | `src/main.ts` | 3001 |
| meteor-compute-service | Go 1.24, AWS SDK | `cmd/meteor-compute-service/main.go` | - |
| meteor-edge-client | Rust, Tokio, OpenCV | `src/main.rs``src/app.rs` | - |
## Development Commands
### Full Stack Development
```bash
# Start all services (frontend + backend)
npm run dev
# Full stack development
npm run dev # Start backend (3001) + frontend (3000) concurrently
npm run dev:frontend # Next.js only
npm run dev:backend # NestJS only
# Start individual services
npm run dev:frontend # Next.js on port 3000
npm run dev:backend # NestJS on port 3000 (with CORS for frontend)
```
# Building
npm run build # Build all services
npm run build:frontend
npm run build:backend
### Building
```bash
npm run build # Build all services
npm run build:frontend # Build Next.js app
npm run build:backend # Build NestJS app
```
# Testing
npm run test # Unit tests (frontend + backend)
npm run test:frontend # Jest + Testing Library
npm run test:backend # Jest for NestJS
npm run test:integration # Backend integration with real DB
npm run test:e2e # Playwright E2E tests
npm run test:fullstack # Complete test suite
### Testing
```bash
# Complete test suite
npm run test:fullstack # All tests: unit + integration + e2e
# Single test execution
cd meteor-web-backend && npm test -- --testPathPattern=events.service
cd meteor-frontend && npm test -- --testPathPattern=gallery
cd meteor-frontend && npx playwright test --grep="Gallery page"
# Individual test types
npm run test # Unit tests for both frontend and backend
npm run test:frontend # Jest tests for React components
npm run test:backend # Jest tests for NestJS services
npm run test:e2e # Playwright E2E tests
npm run test:integration # Backend integration tests with real DB
# Test environment
./test-setup.sh # Start LocalStack + test PostgreSQL (port 5433)
npm run clean:test # Teardown test containers
# Test setup and teardown
./test-setup.sh # Initialize test environment with Docker
npm run setup:test # Alternative test setup command
npm run clean:test # Clean up test containers
```
# Linting
npm run lint # Lint all
npm run lint:frontend # ESLint for Next.js
npm run lint:backend # ESLint + Prettier for NestJS
### Linting
```bash
npm run lint # Lint all code
npm run lint:frontend # Next.js ESLint
npm run lint:backend # NestJS ESLint with Prettier
```
### Database Operations (Backend)
```bash
cd meteor-web-backend
npm run migrate:up # Run pending migrations
npm run migrate:down # Rollback last migration
npm run migrate:create <name> # Create new migration
# Database migrations (from meteor-web-backend/)
npm run migrate:up
npm run migrate:down
npm run migrate:create <name>
```
### Edge Client (Rust)
```bash
cd meteor-edge-client
cargo build --release # Native build
cargo build --target=aarch64-unknown-linux-gnu # ARM64 build for Pi
./build.sh # Cross-compile for Raspberry Pi
cargo build --release
cargo run -- run --camera sim:pattern:meteor # Simulated camera
cargo run -- run --camera file:video.mp4 # File replay
./build.sh # Cross-compile for Raspberry Pi ARM64
# Vida Meteor Detection Testing
./target/debug/meteor-edge-client test-vida <video.mp4> # Test Vida detection on video
@ -99,144 +87,74 @@ cargo build --target=aarch64-unknown-linux-gnu # ARM64 build for Pi
./demo_integration_test.sh # Integration test
```
## Database Architecture
### Compute Service (Go)
### Entity Relationships
- **Users**: UserProfile + UserIdentity (supports multiple auth methods)
- **Devices**: Device + InventoryDevice (ownership tracking)
- **Events**: RawEvent → ValidatedEvent (processing pipeline)
- **Analysis**: AnalysisResult (AI/ML analysis data)
- **Weather**: WeatherStation + WeatherObservation + WeatherForecast
- **Camera**: CameraDevice (hardware management)
- **Subscriptions**: SubscriptionPlan + UserSubscription + SubscriptionHistory + PaymentRecord
```bash
cd meteor-compute-service
go run cmd/meteor-compute-service/main.go
go test ./...
```
### Migration System
- Uses node-pg-migrate for schema management
- All entities defined in `meteor-web-backend/src/entities/`
- Migrations in `meteor-web-backend/migrations/`
## Data Flow
## Authentication & Authorization
1. **Edge Client** captures frames via camera, detects meteors, uploads events to backend
2. **Backend API** receives uploads, stores in S3, pushes to SQS for processing
3. **Compute Service** consumes SQS, validates events, writes analysis results to PostgreSQL
4. **Frontend** displays gallery, dashboard, and analysis views via React Query
### JWT-based Authentication
- Login/Register endpoints in AuthController
- JWT strategy with passport-jwt
- Subscription-based authorization via SubscriptionGuard
- Password hashing with bcrypt
## Database Schema
Uses node-pg-migrate. Entities in `meteor-web-backend/src/entities/`:
- **Users**: `UserProfile` + `UserIdentity` (multiple auth methods)
- **Devices**: `Device` + `InventoryDevice` + `DeviceRegistration` + `DeviceCertificate`
- **Events**: `RawEvent``ValidatedEvent``AnalysisResult`
- **Subscriptions**: `SubscriptionPlan` + `UserSubscription` + `PaymentRecord`
- **Weather**: `WeatherStation` + `WeatherObservation` + `WeatherForecast`
## API Structure
### API Structure
- Base path: `/api/v1/`
- Protected routes require Bearer token
- CORS enabled for localhost:3000 (frontend)
## Event Processing Pipeline
### Data Flow
1. **Edge Client** (Rust) captures meteor events via camera with advanced memory management
2. **Ring Buffer Streaming** - Lock-free astronomical frame processing (>3M frames/sec)
3. **Memory-Mapped Files** - Direct access to large astronomical datasets (GB+ files)
4. **Hierarchical Frame Pools** - Zero-copy buffer management with adaptive sizing
5. **Raw Event Upload** to backend API with media files
6. **SQS Queue** triggers processing in Go compute service
7. **Validation** using MVP or Classical CV providers
8. **Analysis Results** stored and exposed via API
9. **Frontend Gallery** displays validated events with infinite scroll
### Advanced Memory Management (Phase 2 & 3)
- **Zero-Copy Architecture** - Arc-based frame sharing eliminates memory copies
- **Hierarchical Frame Pools** - Multi-size buffer pools (64KB, 256KB, 900KB, 2MB)
- **Adaptive Pool Management** - Dynamic resizing based on memory pressure (70%/80%/90% thresholds)
- **Lock-Free Ring Buffers** - High-throughput astronomical frame streaming
- **Memory-Mapped I/O** - Efficient access to large FITS and astronomical data files
- **NUMA-Aware Allocation** - Optimized for modern multi-core Raspberry Pi systems
### Performance Metrics
- **Ring Buffer Throughput**: 3.6M+ writes/sec, 7.2M+ reads/sec
- **Memory Efficiency**: 100%+ throughput with zero frame loss
- **Buffer Utilization**: Dynamic 0-100% with real-time monitoring
- **Memory Savings**: Multi-GB savings through zero-copy architecture
- **Concurrent Safety**: Lock-free operations with atomic ordering
### File Storage
- AWS S3 for media storage (images/videos)
- LocalStack for development/testing
- Multipart upload support in backend
- Memory-mapped access for large astronomical datasets
## Testing Architecture
### Four-Layer Testing
1. **Unit Tests**: Jest for both frontend and backend components
2. **Integration Tests**: Full API workflows with test database
3. **E2E Tests**: Playwright for user interactions
4. **Memory Management Tests**: Advanced Rust-based performance testing
### Test Environment
- Docker Compose setup with test services
- Separate PostgreSQL instance (port 5433)
- LocalStack for AWS service mocking
- Test data generation scripts
### Memory Management Testing (Rust Edge Client)
- **Core Frame Pool Tests**: Basic pooling and zero-copy validation
- **Adaptive Management Tests**: Dynamic resizing under memory pressure
- **Integration Tests**: End-to-end memory optimization workflows
- **Ring Buffer Tests**: Lock-free concurrent streaming validation
- **Memory Mapping Tests**: Large file processing and performance benchmarks
- **Stress Testing**: Multi-million frame throughput validation
- **Production Readiness**: Error handling, resource cleanup, configuration validation
### Gallery Testing
- Complete E2E coverage for authentication, infinite scroll, responsive design
- Integration tests for upload → processing → display workflow
- Performance testing with large datasets
- JWT Bearer token authentication
- Key modules: `AuthModule`, `DevicesModule`, `EventsModule`, `PaymentsModule`, `SubscriptionModule`
## Frontend Architecture
### Next.js App Router Structure
- Pages in `src/app/` (gallery, dashboard, settings, etc.)
- Components organized by domain (`auth/`, `gallery/`, `charts/`, `ui/`)
- React Query for server state management
- Context-based auth state management
### UI Components
- Custom UI components in `components/ui/`
- Chart components using Recharts
- Form handling with React Hook Form + Zod validation
- Responsive design with TailwindCSS 4
- App Router in `src/app/` (dashboard, gallery, analysis, devices, settings)
- Components in `src/components/` organized by domain (charts, device-registration, ui)
- API services in `src/services/`
- Auth context in `src/contexts/auth-context.tsx`
- React Query for server state, React Hook Form + Zod for forms
## Backend Architecture
### NestJS Module Structure
- Domain-driven modules: Auth, Devices, Events, Payments, etc.
- NestJS modules with domain-driven organization
- TypeORM entities with PostgreSQL
- Structured logging with Pino
- Prometheus metrics collection
- Health checks and observability
- Pino structured logging with correlation IDs
- Prometheus metrics via MetricsModule
- Stripe integration in PaymentsModule
### Key Services
- **EventsService**: Core event processing and pagination
- **AuthService**: JWT token management and user validation
- **PaymentsService**: Stripe integration for subscriptions
- **AnalysisService**: AI/ML result processing
- **WeatherService**: Environmental data correlation
## Coding Standards
## Development Workflow
- **TypeScript**: ESLint + Prettier, PascalCase for classes, camelCase for functions
- **Rust**: `cargo fmt`, snake_case modules, prefer `?` over `unwrap()`
- **Go**: `go fmt` + `go vet`, standard project layout
### Adding New Features
1. Create/update database entities and migrations
2. Implement backend service and controller with tests
3. Add frontend components and API integration
4. Update E2E tests for user workflows
5. Run full test suite before committing
## Infrastructure
- Terraform configs in `infrastructure/`
- AWS: S3 (media), SQS (event queue), RDS (PostgreSQL), CloudWatch (logs/metrics)
- LocalStack for local development
- Docker Compose for test environment
## Key Type Considerations
- User type uses `userId` not `id` (defined in `src/contexts/auth-context.tsx`)
- `DeviceStatus` is an enum - use enum values not strings with StatusIndicator
- Recharts has React 19 compatibility issues - type declarations in `src/types/recharts.d.ts`
## Additional Testing Commands
### Running Single Tests
```bash
# Backend unit test for specific service
cd meteor-web-backend && npm test -- --testPathPattern=events.service
# Frontend component test
cd meteor-frontend && npm test -- --testPathPattern=gallery
# Single E2E test
cd meteor-frontend && npx playwright test --grep="Gallery page"

View File

@ -1,5 +1,8 @@
import type { NextConfig } from "next";
// Tailwind requires lightningcss; enforce WASM fallback to avoid missing native builds in CI.
process.env.LIGHTNINGCSS_SKIP_BINDING ??= "1";
const nextConfig: NextConfig = {
output: 'standalone',
};

View File

@ -49,5 +49,8 @@
"jest-environment-jsdom": "^30.0.5",
"tailwindcss": "^4",
"typescript": "^5"
},
"optionalDependencies": {
"lightningcss-linux-x64-gnu": "^1.30.1"
}
}

View File

@ -100,7 +100,7 @@ export function MeteorTypePieChart({ data }: MeteorTypePieChartProps) {
cx="50%"
cy="50%"
labelLine={false}
label={({ name, percent }: { name: string; percent: number }) =>
label={({ name, percent }: { name: string; percent: number }) =>
`${name} ${(percent * 100).toFixed(0)}%`
}
outerRadius={80}

View File

@ -38,7 +38,7 @@ export function TimeDistributionChart({ data, timeFrame }: TimeDistributionChart
} else if ('month' in item && item.month !== undefined) {
timeLabel = item.month;
} else if ('year' in item && item.year !== undefined) {
timeLabel = item.year.toString();
timeLabel = item.year;
}
return {

View File

@ -31,7 +31,6 @@ export function QRCodeDisplay({ registrationCode, pinCode, className, compact =
// Generate QR code with options for better readability
const dataUrl = await QRCode.toDataURL(qrData, {
errorCorrectionLevel: 'M',
type: 'image/png',
margin: 2,
color: {
dark: '#000000',

34
meteor-frontend/src/types/recharts.d.ts vendored Normal file
View File

@ -0,0 +1,34 @@
// Type augmentation for Recharts to fix React 19 compatibility issues
declare module 'recharts' {
import * as React from 'react';
// eslint-disable-next-line @typescript-eslint/no-explicit-any
type AnyObject = Record<string, any>;
export interface XAxisProps extends AnyObject {
dataKey?: string | number | ((dataObject: AnyObject) => AnyObject);
tick?: AnyObject;
className?: string;
}
export interface YAxisProps extends AnyObject {
tick?: AnyObject;
className?: string;
}
export interface LegendProps extends AnyObject {
content?: React.ComponentType<AnyObject> | React.ReactElement;
}
export interface TooltipProps extends AnyObject {
content?: React.ComponentType<AnyObject> | React.ReactElement;
}
export const XAxis: React.ComponentType<XAxisProps>;
export const YAxis: React.ComponentType<YAxisProps>;
export const Legend: React.ComponentType<LegendProps>;
export const Tooltip: React.ComponentType<TooltipProps>;
// Re-export all other types from recharts
export * from 'recharts/types/index';
}

View File

@ -10,7 +10,6 @@ import { DevicesModule } from './devices/devices.module';
import { DeviceRegistrationModule } from './devices/device-registration.module';
import { EventsModule } from './events/events.module';
import { PaymentsModule } from './payments/payments.module';
import { LogsModule } from './logs/logs.module';
import { MetricsModule } from './metrics/metrics.module';
import { WeatherModule } from './weather/weather.module';
import { AnalysisModule } from './analysis/analysis.module';
@ -80,7 +79,6 @@ console.log('Current working directory:', process.cwd());
DeviceRegistrationModule,
EventsModule,
PaymentsModule,
LogsModule,
MetricsModule,
WeatherModule,
AnalysisModule,

4168
package-lock.json generated

File diff suppressed because it is too large Load Diff