@page batch_import_apps Batch Import in Apps

# Batch SWC Import in Desktop and Web Apps

This document explains how to use the batch SWC import functionality in both the desktop app and web server.

## Desktop App (Tauri)

### New Command: `process_swc_directory`

The desktop app now includes a command to process all SWC files in a directory.

### JavaScript/TypeScript Usage

```javascript
import { invoke } from '@tauri-apps/api/tauri';

// Process all SWC files in a directory
async function processSWCDirectory(directoryPath) {
  try {
    const result = await invoke('process_swc_directory', {
      directoryPath: directoryPath
    });

    console.log(`Processed ${result.successful} of ${result.totalFiles} files`);

    // Show summaries
    result.dendriteSummaries.forEach(summary => {
      console.log(`${summary.fileName}:`);
      console.log(`  Vertices: ${summary.numVertices}`);
      console.log(`  Edges: ${summary.numEdges}`);
      console.log(`  Total length: ${summary.totalLength.toFixed(2)}`);
      console.log(`  Leaves: ${summary.numLeaves}`);
      console.log(`  Branch points: ${summary.numBranchPoints}`);
    });

    // Show errors if any
    if (result.errors.length > 0) {
      console.error('Errors:', result.errors);
    }

    return result;
  } catch (error) {
    console.error('Failed to process directory:', error);
    throw error;
  }
}

// Example: Use with directory picker
import { open } from '@tauri-apps/api/dialog';

async function selectAndProcessDirectory() {
  const selected = await open({
    directory: true,
    multiple: false,
  });

  if (selected) {
    const result = await processSWCDirectory(selected);
    // Handle the result...
  }
}
```

### Response Format

```typescript
interface DirectoryProcessResult {
  totalFiles: number;
  successful: number;
  failed: number;
  dendriteSummaries: DendriteSummary[];
  errors: string[];
}

interface DendriteSummary {
  fileName: string;
  numVertices: number;
  numEdges: number;
  numLeaves: number;
  numBranchPoints: number;
  totalLength: number;
}
```

### Example UI Component (React)

```jsx
import { useState } from 'react';
import { invoke } from '@tauri-apps/api/tauri';
import { open } from '@tauri-apps/api/dialog';

function SWCBatchProcessor() {
  const [processing, setProcessing] = useState(false);
  const [result, setResult] = useState(null);

  const handleSelectDirectory = async () => {
    try {
      const selected = await open({
        directory: true,
        multiple: false,
        title: 'Select directory containing SWC files'
      });

      if (selected) {
        setProcessing(true);
        const result = await invoke('process_swc_directory', {
          directoryPath: selected
        });
        setResult(result);
      }
    } catch (error) {
      console.error('Error:', error);
      alert(`Error: ${error}`);
    } finally {
      setProcessing(false);
    }
  };

  return (
    <div>
      <button onClick={handleSelectDirectory} disabled={processing}>
        {processing ? 'Processing...' : 'Select SWC Directory'}
      </button>

      {result && (
        <div>
          <h3>Results</h3>
          <p>Processed {result.successful} of {result.totalFiles} files</p>

          {result.failed > 0 && (
            <div>
              <h4>Errors ({result.failed})</h4>
              <ul>
                {result.errors.map((err, i) => (
                  <li key={i}>{err}</li>
                ))}
              </ul>
            </div>
          )}

          <h4>Summaries</h4>
          <table>
            <thead>
              <tr>
                <th>File</th>
                <th>Vertices</th>
                <th>Edges</th>
                <th>Length</th>
                <th>Leaves</th>
                <th>Branch Points</th>
              </tr>
            </thead>
            <tbody>
              {result.dendriteSummaries.map((summary, i) => (
                <tr key={i}>
                  <td>{summary.fileName}</td>
                  <td>{summary.numVertices}</td>
                  <td>{summary.numEdges}</td>
                  <td>{summary.totalLength.toFixed(2)}</td>
                  <td>{summary.numLeaves}</td>
                  <td>{summary.numBranchPoints}</td>
                </tr>
              ))}
            </tbody>
          </table>
        </div>
      )}
    </div>
  );
}
```

## Web Server (Axum)

### New API Endpoint: `POST /api/import/swc/directory`

The web server now includes an endpoint to batch import all SWC files from a directory into the database.

### HTTP Request

```http
POST /api/import/swc/directory
Content-Type: application/json

{
  "directory_path": "/path/to/swc/files",
  "species": "Rat",
  "brain_region": "Hippocampus"
}
```

### Request Body

```typescript
interface ImportDirectoryRequest {
  directory_path: string;
  species?: string;          // Optional
  brain_region?: string;     // Optional
}
```

### Response

```typescript
interface DirectoryImportResponse {
  total_files: number;
  successful: number;
  failed: number;
  imported_neurons: ImportResponse[];
  errors: string[];
}

interface ImportResponse {
  neuron_id: number;
  structure_id: number;
  num_vertices: number;
  num_edges: number;
}
```

### JavaScript/Fetch Example

```javascript
async function importSWCDirectory(directoryPath, species = null, brainRegion = null) {
  const response = await fetch('http://localhost:3000/api/import/swc/directory', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      directory_path: directoryPath,
      species: species,
      brain_region: brainRegion,
    }),
  });

  if (!response.ok) {
    const errorText = await response.text();
    throw new Error(`Import failed: ${errorText}`);
  }

  const result = await response.json();

  console.log(`Imported ${result.successful} of ${result.total_files} files`);
  console.log(`Created ${result.imported_neurons.length} neurons in database`);

  if (result.errors.length > 0) {
    console.warn('Errors occurred:', result.errors);
  }

  return result;
}

// Usage
try {
  const result = await importSWCDirectory(
    '/Users/yossi/Documents/Maths/Projects/DendritesData/Rat',
    'Rat',
    'Hippocampus'
  );
  console.log('Import successful:', result);
} catch (error) {
  console.error('Import failed:', error);
}
```

### cURL Example

```bash
# Import all SWC files from a directory
curl -X POST http://localhost:3000/api/import/swc/directory \
  -H "Content-Type: application/json" \
  -d '{
    "directory_path": "/path/to/swc/files",
    "species": "Rat",
    "brain_region": "Hippocampus"
  }'
```

### React Component Example

```jsx
import { useState } from 'react';

function DirectoryImporter() {
  const [directoryPath, setDirectoryPath] = useState('');
  const [species, setSpecies] = useState('');
  const [brainRegion, setBrainRegion] = useState('');
  const [importing, setImporting] = useState(false);
  const [result, setResult] = useState(null);

  const handleImport = async (e) => {
    e.preventDefault();
    setImporting(true);
    setResult(null);

    try {
      const response = await fetch('http://localhost:3000/api/import/swc/directory', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({
          directory_path: directoryPath,
          species: species || null,
          brain_region: brainRegion || null,
        }),
      });

      if (!response.ok) {
        throw new Error(await response.text());
      }

      const data = await response.json();
      setResult(data);
    } catch (error) {
      alert(`Error: ${error.message}`);
    } finally {
      setImporting(false);
    }
  };

  return (
    <div>
      <h2>Import SWC Directory</h2>

      <form onSubmit={handleImport}>
        <div>
          <label>
            Directory Path:
            <input
              type="text"
              value={directoryPath}
              onChange={(e) => setDirectoryPath(e.target.value)}
              required
              placeholder="/path/to/swc/files"
            />
          </label>
        </div>

        <div>
          <label>
            Species (optional):
            <input
              type="text"
              value={species}
              onChange={(e) => setSpecies(e.target.value)}
              placeholder="e.g., Rat"
            />
          </label>
        </div>

        <div>
          <label>
            Brain Region (optional):
            <input
              type="text"
              value={brainRegion}
              onChange={(e) => setBrainRegion(e.target.value)}
              placeholder="e.g., Hippocampus"
            />
          </label>
        </div>

        <button type="submit" disabled={importing}>
          {importing ? 'Importing...' : 'Import Directory'}
        </button>
      </form>

      {result && (
        <div>
          <h3>Import Results</h3>
          <p>Imported {result.successful} of {result.total_files} files</p>

          {result.failed > 0 && (
            <div>
              <h4>Errors ({result.failed})</h4>
              <ul>
                {result.errors.map((err, i) => (
                  <li key={i}>{err}</li>
                ))}
              </ul>
            </div>
          )}

          <h4>Imported Neurons</h4>
          <ul>
            {result.imported_neurons.map((neuron, i) => (
              <li key={i}>
                Neuron ID: {neuron.neuron_id},
                Structure ID: {neuron.structure_id},
                {neuron.num_vertices} vertices,
                {neuron.num_edges} edges
              </li>
            ))}
          </ul>
        </div>
      )}
    </div>
  );
}
```

## Features

### Desktop App
- Process directory without database (just analyzes files)
- Returns statistics for each file
- Error handling per file
- No database required
- Fast analysis

### Web Server
- Imports all files into database
- Creates neurons and structures
- Persists all vertex and edge data
- Broadcasts events to WebSocket clients
- Supports metadata (species, brain region)
- Continues on individual file errors

## Error Handling

Both implementations handle errors gracefully:

1. **Individual file errors** don't stop the entire batch
2. **Detailed error messages** include the file name
3. **Summary statistics** show success/failure counts
4. **Partial results** are returned even if some files fail

## Performance

- **Desktop App**: Processes files in memory, faster for analysis only
- **Web Server**: Saves to database, takes longer but provides persistence
- Both: Process files sequentially to avoid memory issues with large datasets

## Related Documentation

- [Batch Processing Module](SWC_BATCH_PROCESSING.md) - Core module documentation
- [Web API Documentation](../README_DATABASE.md) - Full API reference
- [SWC Import](SWC_IMPORT.md) - SWC file format details
