Skip to main content

Understanding CRC and Secure Messenger Mode Options

Problem​

Ceph's network protocols support two communication modes:

  • crc: network traffic is checksummed for integrity
  • secure: network traffic is encrypted

These are controlled by six options. These six options are explained in this article.

Solution​

Ceph network communications are complex. It is possible to select different modes for different channels of communication: for example, connections with the MON and MGR daemons and connections with OSD and MDS daemons. Options also make it possible to differentiate modes for intra-cluster and client connections.

Ceph has six messenger mode configuration options.

The first two options are read by Ceph clients. Clients include RBD, CephFS clients, and the ceph command line tools. Note that the RGW is also a ceph client.

  • ms_mon_client_mode: When a client connects to a MON or MGR, it reads its ms_mon_client_mode option for its preferred order of allowed connection modes. The default is "secure crc", meaning that a client prefers to encrypt connections with the MON and MGR, but will fall back to crc-only if the MON/MGR does not allow secure mode.
  • ms_client_mode: When a client is connecting to any other daemon (e.g. an OSD or MDS), it reads the ms_client_mode option to decide that prefered order. The default is "crc secure", meaning that the client prefers those connections to be in crc mode, but will fall back to secure mode if the OSD/MDS does not allow crc mode.

Options read by the MON and MGR daemons:

  • ms_mon_cluster_mode: When a MON/MGR is connecting to another MON/MGR, it reads its ms_mon_cluster_mode option. The default is "secure crc", meaning that all intra-MON/MGR traffic is encrypted by default, but that any MON/MGR allowing only crc mode will also be permitted.
  • ms_mon_service_mode: When a MON/MGR is connecting with anything else (e.g. a client, OSD, or MDS), then it reads its ms_mon_service_mode option to determine the list of allowed modes. The default is "secure crc", meaning that MON and MGR daemons allow peers to connect in either secure or crc mode.

Options read by the OSD and MDS daemons:

  • ms_mon_cluster_mode: When an OSD/MDS is connecting with a MON or MGR, it reads its ms_mon_cluster_mode option, which defaults to "secure crc". This means that an OSD/MDS prefers secure mode when communicating with MONs and MGRs, but will fall back to crc if the MON/MGR does not allow secure mode.
  • ms_cluster_mode: When an OSD/MDS is communicating with other OSD or MDS daemons, it reads the ms_cluster_mode option. This defaults to "secure crc", which means that the that daemon prefers secure connections, but will fall back to crc if the other side does not allow secure mode.
  • ms_service_mode: When an OSD/MDS is communicating with a client, it reads its ms_service_mode option to determine which modes it allows for such connections. This defaults to "crc secure", meaning that OSD and MDS daemons allow both crc and secure mode connections with clients.