Skip to main content

Block Pools vs. Rados Namespaces

Overview​

When creating a Gateway via the Rook-Ceph Operator, currently only a block pool can be specified. However, RADOS Namespaces are not directly supported in the gateway specification.

Gateway Pool Configuration​

The pool specified in the CephNVMeOFGateway specification is strictly for the NVMe-oF Gateway metadata - the pool where the NVMe-oF Gateway configuration lives. The Gateway itself can still perform I/O to RBD images that reside outside of this specific pool.

Example Gateway Specification​

apiVersion: ceph.rook.io/v1
kind: CephNVMeOFGateway
metadata:
  name: nvmeof
  namespace: rook-ceph
spec:
  # Container image for the NVMe-oF gateway daemon
  image: quay.io/ceph/nvmeof:1.5
  # Pool name that will be used by the NVMe-oF gateway
  pool: nvmeof
  # ANA (Asymmetric Namespace Access) group name
  group: group-a
  # Number of gateway instances to run
  instances: 1
  hostNetwork: false

Current Limitations​

The NVMeOFGatewaySpec definition only supports specifying a pool, not a namespace:

// NVMeOFGatewaySpec represents the spec of an NVMe-oF gateway
type NVMeOFGatewaySpec struct {
    // Image is the container image to use for the NVMe-oF gateway daemon.
    // For example, quay.io/ceph/nvmeof:1.5
    // +kubebuilder:validation:MinLength=1
    Image string `json:"image"`

    // The number of active gateway instances
    // +kubebuilder:validation:Minimum=1
    Instances int `json:"instances"`

    // Pool is the RADOS pool where NVMe-oF configuration is stored
    // +kubebuilder:validation:MinLength=1
    Pool string `json:"pool"`
    //...

Workaround: Using Rados Namespaces with RBD Images​

To use RBD images in different Rados Namespaces:

Step 1: Create a Pool​

First, create the block pool where your data will reside.

Step 2: Create a Rados Namespace​

Create a Rados namespace that points to the pool created in Step 1:

apiVersion: ceph.rook.io/v1
kind: CephBlockPoolRadosNamespace
metadata:
  name: namespace-a
  namespace: rook-ceph # namespace:cluster
spec:
  # The name of the CephBlockPool CR where the namespace is created
  blockPoolName: replicapool

Step 3: Create the StorageClass​

When creating the StorageClass for your RBD images, you must specify both the pool name and the clusterID of the namespace.

As noted in the Rook documentation: "Once the RADOS namespace is created, an RBD-based StorageClass can be created to create PVs in this RADOS namespace. For this purpose, the clusterID value from the CephBlockPoolRadosNamespace status needs to be put into the clusterID field of the StorageClass spec."

Step 4: Consume the StorageClass​

Allow the CephNVMeOFGateway to consume this StorageClass. Your RBD images will then safely live in their respective namespaces while clients can consume these RBD images via the CephNVMeOFGateway.

References​