Installation

The easiest way to try-out chorus is Standalone installation. It requires no extra tools and has zero dependencies. Standalone version does not persist any data between restarts so it can be used only for education. For production-like environments use Helm chart or Docker.

Helm Chart

Install the latest Helm chart from Clyso registry:

helm install <release-name>  oci://harbor.clyso.com/chorus/chorus

Chart default values.yaml:

proxy:
  enabled: true
  image:
    repository: harbor.clyso.com/chorus/proxy
    tag: "" # Chart appVersion will be used if empty
    pullPolicy: Always
  replicas: 2
  resources:
    limits:
      cpu: 100m
      memory: 300Mi
    requests:
      cpu: 50m
      memory: 150Mi
  proxyService:
    portHttp:
      port: 9669
      targetPort: http
    type: ClusterIP
  config: # for full spec and default values see proxy config: https://github.com/clyso/chorus/blob/main/service/proxy/config.yaml and common config https://github.com/clyso/chorus/blob/main/pkg/config/config.yaml
    metrics:
      enabled: false
    log:
      json: true
      level: info
    trace:
      enabled: false
      endpoint: # url to Jaeger or other open trace provider
    address: http://localhost:9669 # Chorus proxy s3 api address
    port: 9669
    cors:
      enabled: false
      allowAll: false
      whitelist:
    auth:
      allowV2Signature: false
      useStorage: # use credentials from one of configured storages <one|two>
      custom: # or use custom credentials for proxy s3 endpoint
    #    user1:
    #      accessKeyID: <user1 v4 accessKey credential>
    #      secretAccessKey: <user1 v4 secretKey credential>
    #    user2:
    #      accessKeyID: <user2 v4 accessKey credential>
    #      secretAccessKey: <user2 v4 secretKey credential>
worker:
  replicas: 2
  image:
    repository: harbor.clyso.com/chorus/worker
    tag: "" # Chart appVersion will be used if empty
    pullPolicy: Always
  resources:
    limits:
      cpu: 100m
      memory: 300Mi
    requests:
      cpu: 50m
      memory: 150Mi
  restApiService:
    portRest:
      port: 9071
      targetPort: rest
    type: ClusterIP
  grpcApiService:
    portGRPC:
      port: 9670
      targetPort: grpc
    type: ClusterIP
  config: # for full spec and default values see worker config: https://github.com/clyso/chorus/blob/main/service/worker/config.yaml and common config https://github.com/clyso/chorus/blob/main/pkg/config/config.yaml
    metrics:
      enabled: false
    log:
      json: true
      level: info
    trace:
      enabled: false
      endpoint: # url to Jaeger or other open trace provider
    rclone:
      memoryLimit:
        enabled: true
        limit: 300M # Max RAM memory for file copy. Possible values: 50M <= mem <= (total app RAM - 50M). More memory -> faster s3 replication. # For example: if worker container resources.memory.limit = 500M, then mem = 450M
        retryMin: 2s
        retryMax: 20s
      memoryCalc:
        mul: 1.5
        const: 0M
      localFileLimit:
        enabled: false
        limit: 3
        retryMin: 2s
        retryMax: 20s
      globalFileLimit:
        enabled: true
        limit: 3
        retryMin: 2s
        retryMax: 20s
    concurrency: 10
    api:
      enabled: true
commonConfig:
  features:
    tagging: true
    acl: true
  storage:
    createRouting: true # create roting rules to route proxy requests to main storage
    createReplication: false # create replication rules to replicate data from main to other storages
    storages:
      one:
        address: s3.clyso.com
        provider: Ceph
        isMain: true
        isSecure: true
      two:
        address: office.clyso.cloud
        provider: Ceph
        isSecure: true
agent:
  enabled: false
  image:
    repository: harbor.clyso.com/chorus/agent
    tag: "" # Chart appVersion will be used if empty
    pullPolicy: Always
  replicas: 1
  resources:
    limits:
      cpu: 100m
      memory: 128Mi
    requests:
      cpu: 50m
      memory: 64Mi
  agentService:
    portHttp:
      port: 9673
      targetPort: http
    type: ClusterIP
  config: # for full spec and default values see agent config: https://github.com/clyso/chorus/blob/main/service/agent/config.yaml and common config https://github.com/clyso/chorus/blob/main/pkg/config/config.yaml
    metrics:
      enabled: false
    log:
      json: true
      level: info
    trace:
      enabled: false
      endpoint: # url to Jaeger or other open trace provider
    port: 9673 # agent port to listen incoming notifications
    url: http://localhost:9673 # REQUIRED: url to be used by s3 storage to send notifications. The URL should be reachable for s3 storage.
    fromStorage: "" # REQUIRED: notifications source storage name from Chorus config. Normally name of the main storage from storage config.
existingSecret: "" # set name of existing secret with storage credentials or
secret: "" #  set storage credentials here to create secret:
# Example
# secret: |-
#  storage:
#    storages:
#      one:
#        address: s3.clyso.com
#        provider: Ceph
#        isMain: true
#        isSecure: true
#        credentials:
#          user1:
#            accessKeyID: abc
#            secretAccessKey: abc
#          user2:
#            accessKeyID: def
#            secretAccessKey: jhi
#      two:
#        address: office.clyso.cloud
#        provider: Ceph
#        isMain: false
#        isSecure: true
#        credentials:
#          user1:
#            accessKeyID: jkl
#            secretAccessKey: mno
#          user2:
#            accessKeyID: pqr
#            secretAccessKey: stw
redis:
  enabled: true
  architecture: standalone
  auth:
    existingSecret: redis-secret
    existingSecretPasswordKey: password
imagePullSecrets:
serviceAccount:
  # Specifies whether a service account should be created
  create: true
  # Annotations to add to the service account
  annotations: {}
  # The name of the service account to use.
  # If not set and create is true, a name is generated using the fullname template
  name: ""

For more information see Configuration section

TODO: Helm chart does not contain admin-UI so it should be installed separately

Docker compose

See docker-compose example on GitHub.

Management CLI

Chorus provides CLI called chorctl as alternative to web UI.

Installation steps:

1. download suitable chorctl_* binary from the latest release
2. unpack and add to system PATH

Or with homebrew (for MacOS and Linux only):

brew install clyso/tap/chorctl

Usage:

1. By default, chorctl will use localhost:9670 to call management GRPC Api. To override use envar:

   export CHORUS_ADDRESS=<grpc api ip or domain>:<port>
   Or -a or --address flag:

   chorctl -a '<grpc api ip or domain>:<port>'
2. Print help to see available commands and flags:

   chorctl help 
3. See replication dashboard

   chorctl dash 
4. Manage replications per user or per bucket with chorctl repl command

   chorctl repl -h

Standalone

Standalone version does not have persistence so it cannot be used for production

Chorus provides standalone version suitable for local development playground. Standalone version is a single binary containing all Chorus components along with embedded in-memory s3 storages for test purposes:

• Chorus proxy running on port 9669.
• Management web UI on port 9672.
• Management GRPC API on port 9670.
• Management REST API on port 9671.
• 2 Fake s3 storage on ports 9680 [MAIN] and 9681.

Installation steps:

1. download suitable standalone_* binary from the latest release
2. unpack and add to system PATH
3. run chorus -h or chorus -help to print help Usage: If no flags and arguments chorus will start with embedded fake s3 storages. Modify chorus config to run it against custom storages:

• print current fake config into file with chorus print-config > my-conf.yaml command
• open my-conf.yaml with text editor end replace s3 urls and credentials
• run chorus with the config chorus -config ./my-config.yaml
• alternatively place config to default chorus config dir $HOME/.config/chorus/config.yaml

   mkdir -p ~/.config/chorus && mv ./my-config.yaml ~/.config/chorus/config.yaml
  now chorus can be will use the config by default