meteor_detect/docs/cams_ftp_format.md

CAMS FTP Format Documentation

Overview

The CAMS FTP (Cameras for All-sky Meteor Surveillance File Transfer Protocol) format is a data compression technique used in meteor detection systems. It was developed by Peter Jenniskens at the SETI Institute for the NASA-sponsored CAMS meteor survey. The format allows for efficient storage and processing of video data for meteor detection while preserving critical temporal and statistical information.

Format Description

The CAMS FTP format compresses video data by taking a batch of 256 consecutive frames and generating 4 special images that retain the essential data needed for meteor detection:

1. maxpixel: Records the maximum brightness value for each pixel position across all 256 frames
2. avepixel: Calculates the average brightness of each pixel after excluding the maximum value
3. stdpixel: Computes the standard deviation of each pixel after excluding the maximum value
4. maxframe: Records which frame (0-255) contained the maximum brightness value for each pixel

By generating these four images, the storage requirement is reduced from 256 frames to just 4 images while preserving the most important information for meteor detection.

Implementation Details

Our implementation in src/detection/frame_stacker.rs provides a complete solution for generating CAMS FTP format images:

FrameStackerConfig

Configuration options for the frame stacker:

• frames_per_stack: Number of frames to stack (typically 256 for CAMS format)
• save_stacked_frames: Whether to save the produced images to disk
• output_directory: Directory for storing the output images
• write_fits: Option to write in FITS astronomical format instead of PNG
• max_pixel_value: Maximum pixel value (typically 255 for 8-bit images)

Processing Algorithm

The frame stacking algorithm works as follows:

1. The system collects 256 consecutive frames (or the configured number)
2. For each pixel position across all frames:
   • maxpixel: Records the highest brightness value
   • maxframe: Records which frame contained that maximum value
   • avepixel: Calculates the average brightness after excluding the maximum value
   • stdpixel: Computes the standard deviation after excluding the maximum value

Mathematical Formulation

Let's denote the pixel value at position (x,y) in frame i as pi, where i ranges from 0 to 255:

• maxpixel(x,y) = max{p0, p1, ..., p255}
• maxframe(x,y) = argmax{pi} (the i value where pi equals maxpixel(x,y))
• avepixel(x,y) = (sum{pi} - maxpixel(x,y)) / 255
• stdpixel(x,y) = sqrt((sum{(pi - avepixel(x,y))²} - (maxpixel(x,y) - avepixel(x,y))²) / 255)

In the standard deviation calculation, we exclude the maximum value to focus on the background variation.

Special Considerations

1. Multiple Maximum Values: If multiple frames have the same maximum value, our implementation uses the first occurrence.

2. Handling Color Frames: If input frames are not already grayscale, our system automatically converts them.

3. Numerical Precision: For accurate calculations, sums are accumulated in 64-bit floating point.

4. Rounding and Clipping: Results are rounded and clipped to 8-bit values (0-255) for the output images.

Usage for Meteor Detection

The CAMS FTP format is specifically designed for meteor detection:

• maxpixel: Shows the meteor streak across the sky
• maxframe: Provides timing information for the meteor's trajectory
• avepixel: Represents the background sky
• stdpixel: Helps distinguish noise from real meteors

Meteors typically appear as bright streaks against the background in the maxpixel image, with sequential values in the maxframe image. By analyzing these patterns, our detection algorithms can identify meteor events while filtering out noise, aircraft, and other false positives.

Performance Considerations

Processing 256 frames for CAMS FTP format can be computationally intensive. Our implementation:

• Uses efficient pixel-wise operations
• Handles memory carefully for larger frame sizes
• Processes in a background thread to avoid blocking the main application

For real-time applications on resource-constrained devices (like Raspberry Pi), consider reducing resolution or frame rate if processing cannot keep up with the incoming video.

References

1. Jenniskens, P., Gural, P. S., Grigsby, B., et al. (2011). "CAMS: Cameras for Allsky Meteor Surveillance to establish minor meteor showers." Icarus, 216(1), 40-61.
2. CAMS Project Website: http://cams.seti.org/