Skip to main content

Configuration

Chorus loads configurations from .yaml files. Use commands and flags to designate the path to these .yaml files:

  • -config. Example: proxy -config ./my-dir/my-conf.yaml
  • -config-override . Example: proxy -config ./my-dir/my-conf.yaml -config-override ./my-dir/my--secretconf.yaml

Both of these flags are optional. Values provided in -config override the default values. Values provided in -config-override override default values and -config values.

Configuration values can also be set as environment variables. For example: export CFG_REDIS_ADDRESS='127.0.0.1:6379' will override value from yaml config:

redis:
  address: 127.0.0.1:1234

CFG_ - mandatory prefix for chorus config value _ - separator for yaml properties.

log.level: info from .yaml is functionally equivalent to the environment variable CFG_LOG_LEVEL=info.

Configuration Files​

For full configuration options and default values, see the source files:

Key Configuration Sections​

Storage​

Define storages and designate one as main:

storage:
  main: "my_storage"  # name of main storage
  storages:
    my_storage:
      type: S3  # S3 or SWIFT
      address: s3.example.com
      provider: Ceph  # Ceph, Minio, or Other
      isSecure: true
      rateLimit:
        enabled: false
        rpm: 60  # requests per minute
      credentials:
        user1:
          accessKeyID: "..."
          secretAccessKey: "..."

The rateLimit option limits requests to a specific storage, useful when a destination storage has API rate limits or limited capacity.

Redis​

Configure Redis connection:

redis:
  addresses:
    - "127.0.0.1:6379"
  password: ""
  appDB: 0      # metadata storage
  queueDB: 1    # work queue
  lockDB: 2     # distributed locks
  configDB: 3   # policies

Logging, Metrics, Tracing​

log:
  json: true   # JSON format for production
  level: info

metrics:
  enabled: true
  port: 9090

trace:
  enabled: true
  endpoint: "http://jaeger:14268/api/traces"

Dynamic Credentials​

Manage storage credentials via API instead of config files:

storage:
  dynamicCredentials:
    enabled: true
    masterPassword: "secret"  # encrypts credentials in Redis
    pollInterval: 3s

When enabled, use chorctl set-user to add credentials at runtime.

Swift Storage​

OpenStack Swift requires different configuration for proxy and worker:

  • Worker: Uses Keystone authentication (authURL)
  • Proxy: Uses direct Swift endpoint (storageURL)

Example Swift storage:

storage:
  storages:
    swift-main:
      type: SWIFT
      # Worker uses Keystone
      authURL: http://keystone:5000/v3
      storageEndpointName: swift
      storageEndpointType: object-store
      region: RegionOne
      # Proxy uses direct endpoint (cannot use Keystone)
      storageURL: http://swift:8080/v1
      credentials:
        # Key must be OpenStack project ID
        b6ebf758c9894224a105e5531eaa4ce9:
          username: admin
          password: secret
          domainName: Default
          tenantName: admin

See Helm chart examples for complete S3 and Swift configuration examples.

Agent​

The Agent is an alternative to Proxy for capturing S3 changes. Use Agent when:

  • You cannot deploy Proxy in front of your storage (e.g., AWS S3, managed services)
  • Your storage supports S3 bucket notifications

Agent receives bucket notifications via webhook and creates replication tasks.

Limitations

Agent is currently S3 only and supports bucket-level replication only (not user-level). This is because Agent relies on S3 bucket notifications which are configured per-bucket.

Agent Configuration​

port: 9673
url: "http://agent.example.com:9673"  # URL reachable by S3 storage
fromStorage: "main"  # source storage name from storage config

Key settings:

  • url: Public URL where Agent receives notifications. Must be reachable by the S3 storage.
  • fromStorage: Name of the source storage (must match a key in storage.storages)

See service/agent/config.yaml for full configuration options.

Setting Up Agent-based Replication​

  1. Deploy Agent with url pointing to its public endpoint
  2. Configure your S3 storage to send bucket notifications to <agent-url>/webhook:
    • AWS S3: Configure SNS with HTTP endpoint
    • Ceph: Configure bucket notifications
    • MinIO: Configure webhook notifications
  3. Create replication with --agent-url flag:
chorctl repl add --user=user1 --from=main --to=follower \
    --from-bucket=mybucket --agent-url=http://agent.example.com:9673/webhook

See Helm chart agent example for a complete deployment configuration.