Log All S3 Operations

Problem

The customer noticed that the default RGW access log does not contain useful info such as the bucket name. How can they log more info for all S3 operations?

Solution

Enable the RGW ops log to file feature as follows:

# ceph config set global rgw_ops_log_rados false
# ceph config set global rgw_ops_log_file_path '/var/log/ceph/ops-log-$cluster-$name.log'
# ceph config set global rgw_enable_ops_log true

If you are using Cephadm and want to output the RGW ops log to the Container Logs then use /dev/stderr or /dev/stdout as File Path:

# ceph config set global rgw_ops_log_file_path '/dev/stdout'

Then restart all radosgw daemons.

# ceph orch ps
# ceph orch daemon restart <rgw>

Following this configuration change, the radosgw will log operations to the file /var/log/ceph/ceph-rgw-ops.json.log, for example:

root@ceph-rgw-1:~# tail -n1 /var/log/ceph/d6e57b01-8e9a-46c6-88ae-14476be461cc/ceph-rgw-ops.json.log
{"bucket":"mybucketname","time":"2023-10-31T22:02:43.565188Z","time_local":"2023-10-31T22:02:43.565188+0000","remote_addr":"1.2.3.4","user":"myusername","operation":"delete_obj","uri":"DELETE /path/to/my/object?x-id=DeleteObject?x-id=DeleteObject HTTP/1.1","http_status":"204","error_code":"NoContent","bytes_sent":0,"bytes_received":0,"object_size":0,"total_time":3,"user_agent":"aws-sdk-js/3.331.0 os/linux/4.19.0-24-amd64 lang/js md/nodejs/18.17.1 api/s3/3.331.0","referrer":"","trans_id":"tx00000****************-**********-*******-default","authentication_type":"Local","access_key_id":"********************","temp_url":false}