Skip to main content

Deploy Ceph using Cephadm with IPv6

Learn how to successfully deploy Ceph using Cephadm in IPv6 environments and resolve common configuration issues.

Problem Description​

Cephadm bootstrap fails when trying to configure a Ceph cluster with IPv6 addresses, showing network configuration errors.

Common Error Messages​

./cephadm bootstrap --mon-ip xxxx:xxxx:xx:xxxx:x::xx --skip-monitoring-stack -c ceph.conf
Verifying podman|docker is present...
Verifying lvm2 is present...
Verifying time synchronization is in place...
Unit chrony.service is enabled and running
Repeating the final host check...
podman (/usr/bin/podman) version 4.7.2 is present
systemctl is present
lvcreate is present
Unit chrony.service is enabled and running
Host looks OK
Cluster fsid: 19c15c1a-c125-11ee-a33d-17acafae1852
...
The public CIDR network xxxx:xxx:xx:xxxx:x::/80 (from -c conf file) is not configured locally.
ERROR: None of the public CIDR network(s) {'xxxx:xxx:xx:xxxx:x::/80'} (from -c conf file) is configured locally.

Solution​

Step 1: Verify IPv6 Configuration​

First, check the IPv6 address configuration of your node:

ip addr show

Example output:

2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether xx:xx:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    inet6 xxxx:xxx:xx:xxxx:x::xx/128 scope global
       valid_lft forever preferred_lft forever
    inet6 xxx::x/xx scope link
       valid_lft forever preferred_lft forever

Step 2: Bootstrap Without Monitor Network​

Bootstrap the cluster while skipping monitor network configuration:

./cephadm bootstrap --mon-ip xxxx:xxx:xx:xxxx:x::xx --skip-mon-network

Step 3: Configure Monitor Network Post-Bootstrap​

After successful bootstrap, configure the monitor network from the cephadm shell. The network should be in CIDR notation:

./cephadm shell -- ceph config set mon public_network xxxx:xxx:xx:xxxx:x::/80

Step 4: Deploy Additional Daemons​

Add nodes and deploy other daemons following the upstream Ceph documentation.

Troubleshooting Tips​

  • Port accessibility: If daemons cannot join the cluster, verify that required ports are open
  • Network connectivity: Ensure IPv6 connectivity between all cluster nodes
  • CIDR notation: Always use proper CIDR notation for IPv6 network configuration