Skip to main content

Configuration

Chorus reads config from yaml files. Path to these yaml files can be provided with flags:

  • -config. Example: proxy -config ./my-dir/my-conf.yaml
  • -config-override . Example: proxy -config ./my-dir/my-conf.yaml -config-override ./my-dir/my--secretconf.yaml

Both flags are optional. Values provided in -config override default values. Values provided in -config-override override default values and -config values.

Besides this, config value can be set as env variable: export CFG_REDIS_ADDRESS='127.0.0.1:6379' will override value from yaml config:

redis:
  address: 127.0.0.1:1234

CFG_ - mandatory prefix for chorus config value _ - separator for yaml properties.

Hence, log.level: info from yaml equals to envar CFG_LOG_LEVEL=info.

Common config​

Proxy and worker have common config properties with following default values:

log:
  json: false # false for dev console logger, true - json log for prod to export to Grafana&Loki.
  level: info
metrics:
  enabled: false
  port: 9090
trace:
  enabled: false
  endpoint: # url to Jaeger or other open trace provider
redis:
  address: "127.0.0.1:6379"
  password:
  appDB: 0 # redis db to store replicated objects metadata
  queueDB: 1 # redis db to store worker replication tasks
  lockDB: 2 # redis db for distributed lock
  configDB: 3 # redis db to store configured replication policies
features:
  tagging: true # sync object/bucket tags
  acl: true # sync object/bucket ACLs
storage:
  createRouting: true # create roting rules to route proxy requests to main storage for all users and buckets
  createReplication: false  # create replication rules to replicate data from main to other storages for all users and buckets
  storages:
#    one: # yaml key with some handy storage name
#      address: <s3.chorus.com>
#      credentials:
#        user1:
#          accessKeyID: <user1 v4 accessKey credential>
#          secretAccessKey: <user1 v4 secretKey credential>
#        user2:
#          accessKeyID: <user2 v4 accessKey credential>
#          secretAccessKey: <user2 v4 secretKey credential>
#      provider: <Ceph|Minio|AWS|Other see providers list in rclone config> # https://rclone.org/s3/#configuration
#      isMain: true # <true|false> one of the storages in should be main
#      healthCheckInterval: 10s
#      httpTimeout: 1m
#      isSecure: true #set false for http address
#      rateLimit:
#        enable: true
#        rpm: 60
#    two: # yaml key with some handy storage name
#      address: <news3.chorus.com>
#      credentials:
#        user1:
#          accessKeyID: <user1 v4 accessKey credential>
#          secretAccessKey: <user1 v4 secretKey credential>
#        user2:
#          accessKeyID: <user2 v4 accessKey credential>
#          secretAccessKey: <user2 v4 secretKey credential>
#      provider: <Ceph|Minio|AWS|Other see providers list in rclone config> # https://rclone.org/s3/#configuration
#      isMain: false # <true|false> one of the storages in should be main
#      healthCheckInterval: 10s
#      httpTimeout: 1m
#      isSecure: true #set false for http address
#      rateLimit:
#        enable: true
#        rpm: 60

Worker config​

Worker default values:

concurrency: 10
rclone:
  memoryLimit:
    enabled: true
    limit: 300M # Max RAM memory for file copy. Possible values: 50M <= mem <= (total app RAM - 50M). 
                # More memory -> faster s3 replication. # For example: if worker container resources.memory.limit = 500M, then mem = 450M
    retryMin: 2s # min task retry interval if limit reached. Used in retry sampler
    retryMax: 20s # max task retry interval if limit reached. Used in retry sampler
  localFileLimit:
    enabled: false
    limit: 3 # max number of objects copied in parallel by one instance
    retryMin: 2s
    retryMax: 20s
  globalFileLimit:
    enabled: true
    limit: 3 # max number of objects copied in parallel across all instances - useful if worker replicas share the same network resource
    retryMin: 2s
    retryMax: 20s
api:
  enabled: false # set true to enable management api
  grpcPort: 9670
  httpPort: 9671
  secure: false

Proxy config​

Proxy default values:

address: "http://localhost:9669" # Chorus proxy s3 api address
port: 9669
cors:
  enabled: false
  allowAll: false
  whitelist: # list of allowed urls
auth:
  allowV2Signature: false
  useStorage: # use credentials from one of configured storages <one|two>
  custom: # OR use custom credentials for proxy s3 endpoint
#    user1:
#      accessKeyID: <user1 v4 accessKey credential>
#      secretAccessKey: <user1 v4 secretKey credential>
#    user2:
#      accessKeyID: <user2 v4 accessKey credential>
#      secretAccessKey: <user2 v4 secretKey credential>