Configuration Guide¶

New in v0.1.17.1: Modular Configuration

NetIntel-OCR v0.1.17.1 introduces modular installation which affects configuration. You can now configure only the modules you have installed, and the system will gracefully handle missing modules.

Overview¶

NetIntel-OCR v0.1.17+ introduces comprehensive configuration management through the config capability, enabling environment-specific settings, profiles, and automated configuration handling. Version 0.1.17.1 enhances this with modular configuration that adapts to installed features.

Configuration Architecture¶

graph TB
    A[Default Config] --> B[Profile Configs]
    B --> C[Environment Overrides]
    C --> D[CLI Arguments]
    D --> E[Runtime Config]

Configuration File Structure¶

Main Configuration (`config.json`)¶

{
  "version": "0.1.17.1",
  "profile": "production",

  "modules": {
    "kg": {
      "enabled": true,
      "falkordb": {
        "host": "localhost",
        "port": 6379
      },
      "embeddings": {
        "models": ["TransE", "RotatE", "ComplEx"]
      }
    },
    "vector": {
      "enabled": false
    },
    "api": {
      "enabled": true
    },
    "mcp": {
      "enabled": false
    }
  },

  "server": {
    "api": {
      "host": "0.0.0.0",
      "port": 8000,
      "workers": 4,
      "timeout": 300,
      "max_request_size": "100MB",
      "auth": {
        "enabled": false,
        "key": null
      }
    },
    "mcp": {
      "host": "0.0.0.0",
      "port": 8001,
      "enabled": true
    }
  },

  "models": {
    "default": "qwen2.5vl:7b",
    "text": "Nanonets-OCR-s:latest",
    "network": "NetIntelOCR-7B-0925",
    "flow": "qwen2.5vl:7b",
    "ollama": {
      "host": "http://localhost:11434",
      "timeout": 600
    }
  },

  "db": {
    "type": "milvus",
    "milvus": {
      "host": "localhost",
      "port": 19530,
      "collection": "network_docs",
      "index_type": "IVF_SQ8",
      "metric_type": "L2"
    },
    "lancedb": {
      "path": "./lance_db",
      "table": "documents"
    }
  },

  "processing": {
    "max_parallel": 4,
    "chunk_size": 10,
    "confidence_threshold": 0.5,
    "context_lines": 2,
    "timeout_per_page": 30,
    "cache": {
      "enabled": true,
      "dir": "~/.cache/netintel-ocr",
      "ttl": 3600
    }
  },

  "logging": {
    "level": "INFO",
    "format": "json",
    "file": null,
    "rotation": {
      "enabled": false,
      "max_size": "100MB",
      "max_files": 10
    }
  },

  "performance": {
    "gpu": {
      "enabled": false,
      "device": 0
    },
    "memory": {
      "max_usage": "8GB",
      "swap_enabled": false
    },
    "cpu": {
      "max_cores": null
    }
  }
}

Module-Aware Configuration (v0.1.17.1)¶

Automatic Module Detection¶

NetIntel-OCR v0.1.17.1 automatically detects installed modules and adjusts configuration:

# Check installed modules and their configuration status
netintel-ocr --version --detailed

# Initialize config based on installed modules
netintel-ocr config init --auto-detect

# The system will:
# 1. Detect which modules are installed
# 2. Enable configuration for installed modules
# 3. Disable configuration for missing modules
# 4. Show warnings for missing optional modules

Module-Specific Configuration¶

# Configure Knowledge Graph (if installed)
netintel-ocr config set modules.kg.enabled true
netintel-ocr config set modules.kg.falkordb.host localhost

# Configure Vector Store (if installed)
netintel-ocr config set modules.vector.enabled true
netintel-ocr config set modules.vector.milvus.host localhost

# Check module availability before configuring
netintel-ocr config check-module kg
# Output: ✓ Knowledge Graph module installed
netintel-ocr config check-module vector
# Output: ✗ Vector module not installed. Install with: pip install netintel-ocr[vector]

Configuration Commands¶

Initialize Configuration¶

# Create default configuration (v0.1.17.1 auto-detects modules)
netintel-ocr config init

# Initialize with template
netintel-ocr config init --template production

# Initialize at custom path
netintel-ocr config init --output /etc/netintel/config.json

# Available templates (v0.1.17.1 templates adapt to installed modules)
netintel-ocr config init --list-templates
# - minimal: Base OCR only (500MB install)
# - development: Dev with available modules
# - staging: Staging with KG if available
# - production: Production with all installed modules
# - enterprise: Full features (requires [all] install)
# - cloud: Cloud-optimized (requires [cloud] install)

View Configuration¶

# Show current configuration
netintel-ocr config show

# Show specific section
netintel-ocr config show server
netintel-ocr config show models

# Show effective configuration (with overrides)
netintel-ocr config show --effective

# Output as JSON
netintel-ocr config show --json > current-config.json

Set Configuration Values¶

# Set individual values
netintel-ocr config set server.api.port 8080
netintel-ocr config set models.default llava:13b
netintel-ocr config set db.milvus.host milvus.internal

# Set nested values
netintel-ocr config set server.api.auth.enabled true
netintel-ocr config set server.api.auth.key "secret-key-123"

# Set arrays
netintel-ocr config set server.api.cors.origins '["http://localhost:3000","https://app.example.com"]'

# Remove a setting (use default)
netintel-ocr config unset server.api.auth.key

Get Configuration Values¶

# Get specific value
netintel-ocr config get server.api.port
# Output: 8000

# Get section
netintel-ocr config get models
# Output: JSON object with all model settings

# Get with default
netintel-ocr config get cache.ttl --default 3600

Configuration Profiles¶

Profile Management¶

# List available profiles
netintel-ocr config profile list

# Create new profile
netintel-ocr config profile create staging
netintel-ocr config profile create production --from development

# Switch profile
netintel-ocr config profile use production

# Show current profile
netintel-ocr config profile current

# Delete profile
netintel-ocr config profile delete old-staging

Profile-Specific Settings¶

# Set value for specific profile
netintel-ocr config set server.api.port 8000 --profile production
netintel-ocr config set server.api.port 3000 --profile development

# Copy profile
netintel-ocr config profile copy production production-backup

# Compare profiles
netintel-ocr config profile diff development production

# Export profile
netintel-ocr config profile export production > prod-config.json

# Import profile
netintel-ocr config profile import staging < staging-config.json

Environment Variables¶

Configuration via Environment¶

All configuration values can be overridden using environment variables:

# Pattern: NETINTEL_<SECTION>_<KEY>
export NETINTEL_SERVER_API_PORT=8080
export NETINTEL_MODELS_DEFAULT=llava:13b
export NETINTEL_DB_MILVUS_HOST=milvus.internal
export NETINTEL_LOGGING_LEVEL=DEBUG

# Module-specific variables (v0.1.17.1)
export NETINTEL_MODULES_KG_ENABLED=true
export NETINTEL_MODULES_KG_FALKORDB_HOST=localhost
export NETINTEL_MODULES_VECTOR_ENABLED=false
export NETINTEL_MODULES_API_ENABLED=true

# Special variables
export NETINTEL_CONFIG=/path/to/config.json
export NETINTEL_PROFILE=production
export NETINTEL_DEBUG=true

Environment Management¶

# Export current config as environment variables
netintel-ocr config env export > .env

# Export specific profile
netintel-ocr config env export --profile production > prod.env

# Load environment file
netintel-ocr config env load .env

# Show environment variables
netintel-ocr config env show

# Generate Docker env file
netintel-ocr config env docker > docker.env

# Generate Kubernetes ConfigMap
netintel-ocr config env k8s > configmap.yaml

Advanced Configuration¶

Configuration Validation¶

# Validate current configuration
netintel-ocr config validate

# Validate specific file
netintel-ocr config validate --file custom-config.json

# Strict validation (check all dependencies)
netintel-ocr config validate --strict

# Fix common issues
netintel-ocr config fix

# Check configuration health
netintel-ocr config health

Configuration Migration¶

# Migrate from old format
netintel-ocr config migrate old-config.ini

# Upgrade configuration version
netintel-ocr config upgrade

# Convert between formats
netintel-ocr config convert config.yaml --format json

# Merge configurations
netintel-ocr config merge base.json overrides.json --output final.json

Configuration Backup¶

# Backup current configuration
netintel-ocr config backup

# Backup with timestamp
netintel-ocr config backup config-$(date +%Y%m%d-%H%M%S).json

# List backups
netintel-ocr config backup list

# Restore from backup
netintel-ocr config restore config-20240115-120000.json

# Auto-backup before changes
netintel-ocr config set --backup server.api.port 8080

Configuration Templates¶

Using Templates¶

# List available templates
netintel-ocr config template list

# Show template details
netintel-ocr config template show production

# Create config from template
netintel-ocr config template apply production

# Create custom template
netintel-ocr config template create my-template --from-current

# Share template
netintel-ocr config template export my-template > template.json

Template Examples (v0.1.17.1)¶

Templates now adapt based on installed modules:

Minimal Template (Base Install)¶

{
  "profile": "minimal",
  "modules": {
    "kg": {"enabled": false},
    "vector": {"enabled": false},
    "api": {"enabled": false},
    "mcp": {"enabled": false}
  },
  "server": {
    "api": {"port": 3000, "host": "localhost"}
  },
  "logging": {"level": "INFO"}
}

Development Template (Auto-detects Modules)¶

{
  "profile": "development",
  "modules": {
    "kg": {"enabled": "auto"},
    "vector": {"enabled": "auto"},
    "api": {"enabled": "auto"},
    "mcp": {"enabled": false}
  },
  "server": {
    "api": {"port": 3000, "host": "localhost"}
  },
  "logging": {"level": "DEBUG"},
  "performance": {"gpu": {"enabled": false}}
}

Production Template (Requires Modules)¶

{
  "profile": "production",
  "modules": {
    "kg": {"enabled": true, "required": true},
    "vector": {"enabled": true, "required": false},
    "api": {"enabled": true, "required": true},
    "mcp": {"enabled": false}
  },
  "server": {
    "api": {
      "port": 8000,
      "host": "0.0.0.0",
      "workers": 8,
      "auth": {"enabled": true}
    }
  },
  "logging": {"level": "WARNING"},
  "performance": {"gpu": {"enabled": true}}
}

Configuration Scenarios¶

Multi-Environment Setup¶

# Development
netintel-ocr config profile use development
netintel-ocr config set server.api.port 3000
netintel-ocr config set logging.level DEBUG

# Staging
netintel-ocr config profile use staging
netintel-ocr config set server.api.port 8000
netintel-ocr config set db.milvus.host staging-milvus

# Production
netintel-ocr config profile use production
netintel-ocr config set server.api.auth.enabled true
netintel-ocr config set logging.level WARNING

Docker Configuration¶

# Generate Docker-optimized config
netintel-ocr config init --template docker

# Set Docker-specific settings
netintel-ocr config set server.api.host 0.0.0.0
netintel-ocr config set db.milvus.host milvus
netintel-ocr config set models.ollama.host http://ollama:11434

# Export for Docker
netintel-ocr config env docker > docker.env

Kubernetes Configuration¶

# Generate K8s ConfigMap
netintel-ocr config k8s configmap > configmap.yaml

# Generate K8s Secret
netintel-ocr config k8s secret --keys "server.api.auth.key" > secret.yaml

# Generate Helm values
netintel-ocr config k8s helm > values.yaml

Configuration Best Practices¶

1. Use Profiles for Environments¶

# Separate profiles for each environment
netintel-ocr config profile create development
netintel-ocr config profile create staging
netintel-ocr config profile create production

# Switch based on environment
ENVIRONMENT=production
netintel-ocr config profile use $ENVIRONMENT

2. Secure Sensitive Data¶

# Don't store secrets in config files
netintel-ocr config set server.api.auth.key "$API_KEY" --no-save

# Use environment variables for secrets
export NETINTEL_SERVER_API_AUTH_KEY="secret-key"

# Or use secret management
netintel-ocr config secret set api-key
netintel-ocr config secret get api-key

3. Version Control Configuration¶

# Track configuration changes
git add config.json
git commit -m "Update production configuration"

# Exclude sensitive profiles
echo "config-production.json" >> .gitignore

# Track templates only
git add templates/*.json

4. Validate Before Deployment¶

# Always validate configuration
netintel-ocr config validate --strict

# Test configuration
netintel-ocr --dry-run server all

# Check effective configuration
netintel-ocr config show --effective

Troubleshooting Configuration¶

Common Issues¶

# Configuration not found
netintel-ocr config init

# Invalid configuration
netintel-ocr config validate
netintel-ocr config fix

# Profile not found
netintel-ocr config profile list
netintel-ocr config profile create missing-profile

# Module not installed (v0.1.17.1)
netintel-ocr --version  # Check which modules are installed
# Install missing module
pip install "netintel-ocr[kg]"
pip install "netintel-ocr[vector]"

# Permission denied
sudo chown $USER:$USER ~/.netintel-ocr/config.json
chmod 600 ~/.netintel-ocr/config.json

Debug Configuration¶

# Show configuration resolution
netintel-ocr config debug

# Show configuration sources
netintel-ocr config sources

# Test configuration loading
netintel-ocr config test

# Show default values
netintel-ocr config defaults

Configuration API¶

Programmatic Access¶

from netintel_ocr.config import ConfigManager

# Load configuration
config = ConfigManager()
config.load()

# Get values
api_port = config.get("server.api.port")
models = config.get("models")

# Set values
config.set("server.api.port", 8080)
config.set("models.default", "llava:13b")

# Check module availability (v0.1.17.1)
if config.is_module_installed("kg"):
    config.set("modules.kg.enabled", True)
    # Configure KG settings
    config.set("modules.kg.falkordb.host", "localhost")
else:
    print("KG module not installed. Install with: pip install netintel-ocr[kg]")

# Save configuration
config.save()

# Switch profiles
config.use_profile("production")

Next Steps¶

Deployment Guide - Deploy with configuration
API Configuration - API server settings
Model Configuration - Model management
Performance Tuning - Optimization settings