@page swc_batch_processing Batch SWC Processing

# Batch SWC File Processing with Database Caching

This document describes the functionality for processing multiple SWC files in a directory with automatic database caching.

## Overview

The `swc_batch` module provides functionality to:
1. Scan a directory for all `.swc` files
2. Process each SWC file into a `Dendrite` structure
3. Save all processed dendrites to a SQLite database
4. On subsequent runs, load from the database instead of reprocessing files

## Usage

### As a Library

```rust
use quadrix::swc_batch;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Process all SWC files in a directory
    let collection = swc_batch::process_directory(
        "/path/to/swc/files",
        "dendrites.db"
    )?;

    // Check if loaded from cache
    if collection.loaded_from_cache {
        println!("Loaded from cached database");
    } else {
        println!("Processed SWC files and created database");
    }

    // Access the dendrites
    for (name, dendrite) in collection.dendrites.iter() {
        println!("Dendrite: {}", name);
        println!("  Vertices: {}", dendrite.vertices.len());
        println!("  Edges: {}", dendrite.edges.len());
    }

    Ok(())
}
```

### Using the Binary

```bash
# Process a directory of SWC files
cargo run --bin process_swc_directory --features web -- /path/to/swc/files

# Use a custom database name
cargo run --bin process_swc_directory --features web -- /path/to/swc/files my_dendrites.db
```

### Using the Example

```bash
# Run the example
cargo run --example process_swc_batch --features web -- /path/to/swc/files
```

## How It Works

### First Run (No Database)

1. Scans the directory for `.swc` files
2. Parses each SWC file into a `Dendrite` structure
3. Creates a SQLite database in the same directory
4. Saves each dendrite with all its vertices and edges
5. Returns a `DendriteCollection` with `loaded_from_cache = false`

### Subsequent Runs (Database Exists)

1. Detects existing database in the directory
2. Loads all dendrites from the database
3. Returns a `DendriteCollection` with `loaded_from_cache = true`

This is much faster than reprocessing SWC files!

## Database Schema

The database stores:
- **Neurons**: Top-level container for each SWC file
- **Dendritic Structures**: The dendrite structure metadata
- **Vertices**: All vertices with their coordinates and relationships
- **Edges**: All edges connecting vertices

For detailed schema information, see `README_DATABASE.md`.

## API Reference

### `process_directory`

```rust
pub fn process_directory(
    dir_path: &str,
    db_name: &str,
) -> Result<DendriteCollection, Box<dyn std::error::Error>>
```

Process all SWC files in a directory, with database caching.

**Arguments:**
- `dir_path`: Path to the directory containing SWC files
- `db_name`: Name of the database file (e.g., "dendrites.db")

**Returns:**
- `DendriteCollection`: Contains all loaded dendrites and cache status

### `DendriteCollection`

```rust
pub struct DendriteCollection {
    pub dendrites: HashMap<String, Dendrite>,
    pub loaded_from_cache: bool,
}
```

**Fields:**
- `dendrites`: HashMap mapping SWC filename to `Dendrite` structure
- `loaded_from_cache`: `true` if loaded from database, `false` if processed from SWC files

## Performance

Processing SWC files and building dendrite structures can be time-consuming for large files. The database caching provides significant performance improvements:

- **First run**: O(n x m) where n = number of files, m = average file size
- **Subsequent runs**: O(n x k) where k is database query time (much faster than parsing)

Typical speedup: **10-100x faster** for subsequent loads.

## Error Handling

The function returns a `Result` with descriptive error messages:

- **No SWC files found**: Returns error if directory has no `.swc` files
- **Invalid directory**: Returns error if path is not a directory
- **Parse errors**: Prints warning for individual file errors, continues processing others
- **Database errors**: Returns error if database creation/access fails

## Example Output

```
Processing directory: /path/to/swc/files
Database name: dendrites.db

Found 5 SWC files
Processing: neuron1.swc
  Processed successfully
Processing: neuron2.swc
  Processed successfully
Processing: neuron3.swc
  Processed successfully
Processing: neuron4.swc
  Processed successfully
Processing: neuron5.swc
  Processed successfully

Saved 5 dendrites to database

=== Results ===
Total dendrites loaded: 5
Loaded from cache: false

=== Dendrite Summary ===
neuron1.swc
  Vertices: 1234
  Edges: 1233
  Total length: 5678.90
  Leaf nodes: 45
  Branch points: 23
```

## Features Required

This functionality requires the `web` feature to be enabled:

```toml
[dependencies]
quadrix = { version = "0.1", features = ["web"] }
```

Or when building:

```bash
cargo build --features web
```

## Related Documentation

- [SWC Import](SWC_IMPORT.md) - Details on SWC file format and parsing
- [Database](../README_DATABASE.md) - Database schema and operations
- [API Documentation](../README.md) - Main project documentation
