Appendix B

Command Cheat Sheet

35 min read

Every command that appears in a code block of a chapter is collected here, grouped by tool and subcommand, so you can find it quickly during day-to-day work. The right-hand column shows the chapter of first use; go back to that chapter for the full explanation. Before running state-changing commands on a real cluster, make sure you are connected to the right cluster (ceph fsid) and record the current state with ceph -s. Commands that need a release newer than Reef carry a release label (for example Squid+); the others work unchanged on Reef (18.2) and later. The same labels appear on the code blocks in the chapters.

ceph · main cluster command line#

ceph osd
`ceph osd tree`	Ch. 1
`ceph osd pool create lab-pool`	Ch. 1
`ceph osd pool application enable lab-pool rados-test`	Ch. 1
`ceph osd map lab-pool hello`	Ch. 1
`ceph osd pool ls detail`	Ch. 1
`ceph osd df tree`	Ch. 1
`ceph osd pool autoscale-status`	Ch. 1
`ceph osd pool create rbd`	Ch. 1
`ceph osd pool rm lab-pool lab-pool --yes-i-really-really-mean-it`	Ch. 1
`ceph osd map rbd rbd_data.5e2a.0000000000000001`	Ch. 1
`ceph osd pool create vms && rbd pool init vms`	Ch. 2
`ceph osd blocklist ls`	Ch. 2
`ceph osd pool create nvmeof-pool && rbd pool init nvmeof-pool` Squid+	Ch. 2
`ceph osd pool create $p`	Ch. 3
`ceph osd pool ls detail \| grep -E "images\|volumes\|backups\|vms"`	Ch. 3
`ceph osd pool set volumes target_size_ratio 0.6`	Ch. 3
`ceph osd pool set vms target_size_ratio 0.2`	Ch. 3
`ceph osd pool set images target_size_ratio 0.1`	Ch. 3
`ceph osd pool set backups target_size_ratio 0.1`	Ch. 3
`ceph osd get-require-min-compat-client`	Ch. 3
`ceph osd pool create cephfs.fs02.meta`	Ch. 6
`ceph osd pool create cephfs.fs02.data`	Ch. 6
`ceph osd pool set cephfs.fs02.meta crush_rule ssd-rule`	Ch. 6
`ceph osd pool create cephfs.fs01.ec erasure`	Ch. 6
`ceph osd pool set cephfs.fs01.ec allow_ec_overwrites true`	Ch. 6
`ceph osd set norebalance && ceph osd set nobackfill`	Ch. 7
`ceph osd unset nobackfill && ceph osd unset norebalance`	Ch. 7
`ceph osd safe-to-destroy osd.9`	Ch. 7
`ceph osd ok-to-stop osd.9`	Ch. 7
`ceph osd tree down`	Ch. 7
`ceph osd safe-to-destroy osd.7`	Ch. 7
`ceph osd tree \| grep osd.7`	Ch. 7
`ceph osd dump \| grep -E 'full_ratio\|nearfull'`	Ch. 7
`ceph osd df \| sort -nk17 \| tail`	Ch. 7
`ceph osd pool set rbd target_size_ratio 0.4`	Ch. 7
`ceph osd pool set cephfs.fs01.meta pg_autoscale_mode on`	Ch. 7
`ceph osd pool set images bulk true`	Ch. 7
`ceph osd ok-to-stop $(ceph osd ls-tree ceph-node2 \| tr '\n' ' ')`	Ch. 7
`ceph osd set noout`	Ch. 7
`ceph osd unset noout`	Ch. 7
`ceph osd set-group noout osd.4`	Ch. 7
`ceph osd set-group noout ceph-node3`	Ch. 7
`ceph osd unset-group noout ceph-node3`	Ch. 7
`ceph osd map rbd rbd_data.12ab.0000000000000004`	Ch. 8
`ceph osd crush tree --show-shadow`	Ch. 8
`ceph osd crush add-bucket rack1 rack`	Ch. 8
`ceph osd crush move rack1 root=default`	Ch. 8
`ceph osd crush move ceph-node1 rack=rack1`	Ch. 8
`ceph osd crush set-device-class nvme osd.20`	Ch. 8
`ceph osd crush class ls`	Ch. 8
`ceph osd crush reweight osd.20 3.49309`	Ch. 8
`ceph osd reweight 20 0.9`	Ch. 8
`ceph osd crush rule create-replicated rep-ssd-host default host ssd`	Ch. 8
`ceph osd crush rule create-replicated rep-rack default rack`	Ch. 8
`ceph osd erasure-code-profile set ec42-rack k=4 m=2 crush-failure-domain=rack crush-device-class=hdd`	Ch. 8
`ceph osd crush rule create-erasure ec42-rack ec42-rack`	Ch. 8
`ceph osd crush rule ls`	Ch. 8
`ceph osd crush rule dump rep-ssd-host`	Ch. 8
`ceph osd getcrushmap -o crush.bin`	Ch. 8
`ceph osd setcrushmap -i crush.new`	Ch. 8
`ceph osd crush show-tunables`	Ch. 8
`ceph osd crush tunables optimal`	Ch. 8
`ceph osd crush set-all-straw-buckets-to-straw2`	Ch. 8
`ceph osd crush rm-device-class osd.$id`	Ch. 8
`ceph osd crush set-device-class fastnvme osd.$id`	Ch. 8
`ceph osd crush rule create-replicated fast-host default host fastnvme`	Ch. 8
`ceph osd pool create tenant-a-db 64 64 replicated fast-host`	Ch. 8
`ceph osd pool application enable tenant-a-db rbd`	Ch. 8
`ceph osd pool create analytics-rbd --autoscale-mode=on`	Ch. 9
`ceph osd pool set analytics-rbd crush_rule rep-ssd-host`	Ch. 9
`ceph osd pool application enable analytics-rbd rbd`	Ch. 9
`ceph osd pool create benchpool 128 128`	Ch. 10
`ceph osd pool application enable benchpool rados-bench --yes-i-really-mean-it`	Ch. 10
`ceph osd perf`	Ch. 10
`ceph osd metadata 3 \| jq '{bluefs_dedicated_db, bluefs_db_size, devices}'`	Ch. 12
`ceph osd pool set rgw-data compression_algorithm zstd`	Ch. 12
`ceph osd pool set rgw-data compression_mode aggressive`	Ch. 12
`ceph osd pool set rgw-data compression_required_ratio 0.875`	Ch. 12
`ceph osd erasure-code-profile ls`	Ch. 13
`ceph osd erasure-code-profile get default`	Ch. 13
`ceph osd erasure-code-profile set ec42 plugin=isa technique=reed_sol_van k=4 m=2 \`	Ch. 13
`ceph osd pool create ecpool erasure ec42`	Ch. 13
`ceph osd pool set ecpool allow_ec_overwrites true`	Ch. 13
`ceph osd pool application enable ecpool rgw`	Ch. 13
`ceph osd erasure-code-profile set lrc42 plugin=lrc k=4 m=2 l=3 crush-failure-domain=host`	Ch. 13
`ceph osd erasure-code-profile set clay42 plugin=clay k=4 m=2 d=5 crush-failure-domain=host`	Ch. 13
`ceph osd pool set ecpool allow_ec_optimizations true` Tentacle+	Ch. 13
`ceph osd pool get ecpool allow_ec_optimizations` Tentacle+	Ch. 13
`ceph osd erasure-code-profile set ec42fast plugin=isa technique=reed_sol_van k=4 m=2 \` Tentacle+	Ch. 13
`ceph osd pool create rbd-ec erasure ec42fast` Tentacle+	Ch. 13
`ceph osd pool set rbd-ec allow_ec_overwrites true` Tentacle+	Ch. 13
`ceph osd pool set rbd-ec allow_ec_optimizations true` Tentacle+	Ch. 13
`ceph osd crush rule dump ecpool`	Ch. 13
`ceph osd getcrushmap -o cm && crushtool -i cm --test --rule <id> --num-rep 6 --show-bad-mappings`	Ch. 13
`ceph osd pool set ecpool fast_read 1`	Ch. 13
`ceph osd pool create devpool 32`	Ch. 14
`ceph osd pool application enable devpool myapp`	Ch. 14
`ceph osd tier add cold-storage hot-storage`	Ch. 16
`ceph osd tier cache-mode hot-storage writeback`	Ch. 16
`ceph osd tier set-overlay cold-storage hot-storage`	Ch. 16
`ceph osd pool set hot-storage hit_set_type bloom`	Ch. 16
`ceph osd pool set hot-storage hit_set_count 12`	Ch. 16
`ceph osd pool set hot-storage hit_set_period 14400`	Ch. 16
`ceph osd pool set hot-storage target_max_bytes 1099511627776`	Ch. 16
`ceph osd pool ls detail \| grep -E "tier\|cache_mode\|read_tier\|write_tier"`	Ch. 16
`ceph osd tier cache-mode hot-storage proxy`	Ch. 16
`ceph osd tier remove-overlay cold-storage`	Ch. 16
`ceph osd tier remove cold-storage hot-storage`	Ch. 16
`ceph osd crush rule create-replicated fast-rule default host nvme`	Ch. 16
`ceph osd erasure-code-profile set cold-ec k=4 m=2 crush-failure-domain=host crush-device-class=hdd`	Ch. 16
`ceph osd pool create vm-fast 128 128 replicated fast-rule`	Ch. 16
`ceph osd pool create archive 256 256 erasure cold-ec`	Ch. 16
`ceph osd pool set archive allow_ec_overwrites true`	Ch. 16
`ceph osd find 12`	Ch. 17
`ceph osd dump \| grep -E "^osd.12 "`	Ch. 17
`ceph osd dump \| grep -E 'stretch_mode\|degraded_stretch\|recovering_stretch'`	Ch. 18
`ceph osd pool ls detail \| grep -E 'peering_crush\|min_size'`	Ch. 18
`ceph osd crush add-bucket site1 datacenter`	Ch. 18
`ceph osd crush add-bucket site2 datacenter`	Ch. 18
`ceph osd crush move site1 root=default`	Ch. 18
`ceph osd crush move site2 root=default`	Ch. 18
`ceph osd crush move stor-a1 datacenter=site1`	Ch. 18
`ceph osd tree \| grep -E "down\|destroyed"`	Ch. 19
`ceph osd pool set vm-fast target_size_ratio 0.4`	Ch. 19
`ceph osd pool set archive bulk true`	Ch. 19
`ceph osd pool set archive pg_num_min 256`	Ch. 19
`ceph osd pool get threshold`	Ch. 19
`ceph osd pool set noautoscale`	Ch. 19
`ceph osd pool unset noautoscale`	Ch. 19
`ceph osd df tree \| sort -nk17 \| tail`	Ch. 19
`ceph osd set-group noout stor-b3`	Ch. 19
`ceph osd unset-group noout stor-b3`	Ch. 19
`ceph osd set norebalance && ceph osd unset norebalance`	Ch. 19
`ceph osd metadata 4 \| jq '{bluestore_bdev_type, bluestore_bdev_rotational, bluestore_bdev_dev_node}'`	Ch. 23
`ceph osd erasure-code-profile set ec42 plugin=isa technique=reed_sol_van k=4 m=2 crush-failure-domain=host` Tentacle+	Ch. 27
`ceph osd pool create ecdata erasure ec42` Tentacle+	Ch. 27
`ceph osd pool set ecdata allow_ec_overwrites true` Tentacle+	Ch. 27
`ceph osd pool set ecdata allow_ec_optimizations true` Tentacle+	Ch. 27
`ceph osd pool get ecdata all \| grep -E 'ec_optim\|stripe'` Tentacle+	Ch. 27
`ceph osd pool ls detail \| grep -E 'erasure\|stripe_width'`	Ch. 27
`ceph osd getcrushmap -o cm.bin && crushtool -d cm.bin -o cm.txt`	Ch. 28
`ceph osd map rbd rbd_data.10ab2c3d4e5f.0000000000000000`	Ch. 28
`ceph osd pg-upmap-items 3.15 4 11`	Ch. 28
`ceph osd rm-pg-upmap-items 3.15`	Ch. 28
`ceph osd crush weight-set ls`	Ch. 28
`ceph osd dump \| grep -E '^pg_upmap(_items\|_primary)?' \| head`	Ch. 28
`ceph osd crush rule create-replicated fast default host ssd`	Ch. 28
`ceph osd crush class ls-osd ssd`	Ch. 28
`ceph osd dump \| grep -E 'allow_crimson\|crimson'` main	Ch. 30
`ceph osd numa-status`	Ch. 33
`ceph osd map rbd rbd_header.5f3a2b8c1d4e`	Ch. 34
ceph orch
`ceph orch host add ceph-node2 192.168.56.12 --labels _admin`	Ch. 1
`ceph orch host add ceph-node3 192.168.56.13`	Ch. 1
`ceph orch host ls`	Ch. 1
`ceph orch apply mon --placement="ceph-node1,ceph-node2,ceph-node3"`	Ch. 1
`ceph orch apply mgr --placement="ceph-node1,ceph-node2"`	Ch. 1
`ceph orch ls`	Ch. 1
`ceph orch ps`	Ch. 1
`ceph orch device ls`	Ch. 1
`ceph orch apply osd --all-available-devices`	Ch. 1
`ceph orch daemon add osd ceph-node1:/dev/sdb`	Ch. 1
`ceph orch host add ceph-node4 192.168.56.14`	Ch. 1
`ceph orch host label add ceph-node4 rgw`	Ch. 1
`ceph orch device ls ceph-node4`	Ch. 1
`ceph orch host label add ceph-node4 mon`	Ch. 1
`ceph orch apply -i mon.yaml`	Ch. 1
`ceph orch apply rgw demo --placement="label:rgw" --port=8080`	Ch. 1
`ceph orch apply rbd-mirror --placement="2 ceph-node2 ceph-node3"`	Ch. 2
`ceph orch ps --daemon-type rbd-mirror`	Ch. 2
`ceph orch apply nvmeof nvmeof-pool group1 --placement="ceph-node2,ceph-node3"` Squid+	Ch. 2
`ceph orch host label add ceph-node3 rgw`	Ch. 4
`ceph orch apply rgw objstore --placement="label:rgw count-per-host:1" --port=8080`	Ch. 4
`ceph orch ls rgw`	Ch. 4
`ceph orch ps --daemon-type rgw`	Ch. 4
`ceph orch apply -i ingress.yaml`	Ch. 4
`ceph orch ls ingress`	Ch. 4
`ceph orch restart rgw.objstore`	Ch. 4
`ceph orch apply rgw gold-eu1 --realm=gold --zone=eu-1 \`	Ch. 5
`ceph orch apply rgw gold-eu2 --realm=gold --zone=eu-2 \`	Ch. 5
`ceph orch restart rgw.gold-eu2`	Ch. 5
`ceph orch restart rgw.gold-eu1`	Ch. 5
`ceph orch ps --daemon-type mds`	Ch. 6
`ceph orch apply mds fs02 --placement="2 ceph-node3 ceph-node4"`	Ch. 6
`ceph orch apply cephfs-mirror --placement="1 ceph-node2"`	Ch. 6
`ceph orch ps --daemon-type osd --hostname ceph-node2`	Ch. 7
`ceph orch ls --export > cluster-specs.yaml`	Ch. 7
`ceph orch daemon restart osd.3`	Ch. 7
`ceph orch daemon stop mds.fs01.ceph-node3.xyzabc`	Ch. 7
`ceph orch daemon start mds.fs01.ceph-node3.xyzabc`	Ch. 7
`ceph orch daemon redeploy mgr.ceph-node1.abcdef`	Ch. 7
`ceph orch host add ceph-node5 192.168.10.15 --labels osd`	Ch. 7
`ceph orch device ls ceph-node5 --refresh`	Ch. 7
`ceph orch ls osd`	Ch. 7
`ceph orch apply -i osd-spec.yaml`	Ch. 7
`ceph orch host label add ceph-node5 mon`	Ch. 7
`ceph orch apply -i mon-spec.yaml`	Ch. 7
`ceph orch osd rm 9`	Ch. 7
`ceph orch osd rm status`	Ch. 7
`ceph orch host drain ceph-node5`	Ch. 7
`ceph orch ps ceph-node5`	Ch. 7
`ceph orch host rm ceph-node5`	Ch. 7
`ceph orch osd rm 7 --replace --zap`	Ch. 7
`ceph orch device ls ceph-node2 --refresh`	Ch. 7
`ceph orch upgrade start --image quay.io/ceph/ceph:v20.2.5`	Ch. 7
`ceph orch upgrade status`	Ch. 7
`ceph orch upgrade pause`	Ch. 7
`ceph orch upgrade resume`	Ch. 7
`ceph orch upgrade stop`	Ch. 7
`ceph orch upgrade start --image quay.io/ceph/ceph:v21.2.0 --daemon-types mgr,mon` Umbrella+	Ch. 7
`ceph orch host maintenance enter ceph-node2`	Ch. 7
`ceph orch host maintenance exit ceph-node2`	Ch. 7
`ceph orch apply mgmt-gateway --placement="label:mgmt"` Tentacle+	Ch. 9
`ceph orch apply mgmt-gateway --virtual_ip 192.168.10.60 --enable-auth=true --placement="label:mgmt"` Tentacle+	Ch. 9
`ceph orch ls mgmt-gateway` Tentacle+	Ch. 9
`ceph orch daemon stop osd.12`	Ch. 10
`ceph orch daemon stop osd.20`	Ch. 10
`ceph orch daemon start osd.20`	Ch. 10
`ceph orch apply -i cluster.yaml --dry-run`	Ch. 11
`ceph orch apply -i cluster.yaml`	Ch. 11
`ceph orch ls --export > cluster-current.yaml`	Ch. 11
`ceph orch ls osd --export`	Ch. 11
`ceph orch rm rgw.objstore`	Ch. 11
`ceph orch device ls --wide`	Ch. 11
`ceph orch ls --export > /srv/ceph-gitops/specs/cluster.yaml`	Ch. 11
`ceph orch apply -i osd-hybrid.yaml --dry-run`	Ch. 11
`ceph orch device ls --refresh --wide`	Ch. 11
`ceph orch ls --export > /tmp/live-specs.yaml`	Ch. 11
`ceph orch daemon stop osd.7`	Ch. 12
`ceph orch daemon start osd.7`	Ch. 12
`ceph orch restart osd.<service>`	Ch. 15
`ceph orch apply rbd-mirror --placement=1`	Ch. 18
`ceph orch upgrade start --ceph-version 20.2.5`	Ch. 19
`ceph orch upgrade pause \| resume \| stop`	Ch. 19
`ceph orch upgrade start --image quay.io/ceph/ceph:v20.2.5 --daemon-types mgr,mon --hosts mon1,mon2`	Ch. 19
`ceph orch upgrade start --image quay.io/ceph/ceph:v20.2.5 --services rgw.s3main --limit 2`	Ch. 19
`ceph orch osd rm 17 --replace`	Ch. 19
`ceph orch device ls stor-b3 --refresh`	Ch. 19
`ceph orch device zap stor-b3 /dev/sdk --force`	Ch. 19
`ceph orch ps --daemon-type osd \| grep osd.17`	Ch. 19
`ceph orch host maintenance enter stor-b3`	Ch. 19
`ceph orch host maintenance exit stor-b3`	Ch. 19
`ceph orch host drain stor-b3`	Ch. 19
`ceph orch host rm stor-b3`	Ch. 19
`ceph orch apply node-exporter`	Ch. 20
`ceph orch apply ceph-exporter`	Ch. 20
`ceph orch apply alertmanager`	Ch. 20
`ceph orch apply prometheus`	Ch. 20
`ceph orch apply grafana`	Ch. 20
`ceph orch ls --service-type prometheus`	Ch. 20
`ceph orch apply mgmt-gateway --virtual_ip 192.0.2.50 --enable-auth=true --placement="label:mgmt"` Tentacle+	Ch. 20
`ceph orch apply oauth2-proxy -i oauth2-proxy.yaml` Tentacle+	Ch. 20
`ceph orch ls \| grep -E "mgmt-gateway\|oauth2"` Tentacle+	Ch. 20
`ceph orch host ls --detail`	Ch. 21
`ceph orch apply nvmeof POOL GROUP --placement="host01, host02"` Tentacle+	Ch. 31
ceph config
`ceph config dump`	Ch. 1
`ceph config get osd osd_memory_target`	Ch. 1
`ceph config set osd osd_max_backfills 2`	Ch. 1
`ceph config show osd.0 \| head`	Ch. 1
`ceph config help osd_memory_target`	Ch. 1
`ceph config set mon mon_allow_pool_delete true`	Ch. 1
`ceph config set mon mon_allow_pool_delete false`	Ch. 1
`ceph config generate-minimal-conf > /tmp/ceph.conf`	Ch. 2
`ceph config set client.rgw rgw_dns_name s3.example.com`	Ch. 4
`ceph config set client.rgw rgw_keystone_url https://keystone.example.com:5000`	Ch. 4
`ceph config set client.rgw rgw_keystone_api_version 3`	Ch. 4
`ceph config set client.rgw rgw_keystone_admin_user rgw`	Ch. 4
`ceph config set client.rgw rgw_keystone_admin_password_path /etc/ceph/keystone.pass`	Ch. 4
`ceph config set client.rgw rgw_keystone_admin_domain Default`	Ch. 4
`ceph config set client.rgw rgw_keystone_admin_project service`	Ch. 4
`ceph config set client.rgw rgw_keystone_accepted_roles "member,admin"`	Ch. 4
`ceph config set client.rgw rgw_keystone_implicit_tenants true`	Ch. 4
`ceph config set client.rgw rgw_swift_account_in_url true`	Ch. 4
`ceph config set client.rgw rgw_s3_auth_use_keystone true`	Ch. 4
`ceph config set osd osd_memory_target 6442450944`	Ch. 7
`ceph config set osd.12 osd_memory_target 8589934592`	Ch. 7
`ceph config set osd/class:ssd osd_memory_target 8589934592`	Ch. 7
`ceph config set client.rgw.objstore rgw_max_concurrent_requests 2048`	Ch. 7
`ceph config show osd.3 \| grep memory_target`	Ch. 7
`ceph config log 10`	Ch. 7
`ceph config set mgr target_max_misplaced_ratio .03`	Ch. 7
`ceph config get osd osd_mclock_profile`	Ch. 7
`ceph config set osd osd_mclock_profile high_recovery_ops`	Ch. 7
`ceph config set osd osd_mclock_profile balanced`	Ch. 7
`ceph config set osd osd_scrub_begin_hour 1`	Ch. 7
`ceph config set osd osd_scrub_end_hour 6`	Ch. 7
`ceph config get mon ms_cluster_mode`	Ch. 8
`ceph config get mon ms_service_mode`	Ch. 8
`ceph config get mon ms_client_mode`	Ch. 8
`ceph config set global ms_cluster_mode "secure"`	Ch. 8
`ceph config set global ms_client_mode secure`	Ch. 8
`ceph config set global ms_service_mode secure`	Ch. 8
`ceph config set mgr mgr/prometheus/rbd_stats_pools "analytics-rbd"`	Ch. 9
`ceph config set mgr mgr/pgwatch/interval 10`	Ch. 9
`ceph config set global mgr_stats_period p ; wait 2p+2 seconds`	Ch. 9
`ceph config dump \| grep osd_mclock_max_capacity_iops`	Ch. 10
`ceph config set mgr mgr/cephadm/container_image_prometheus registry.internal:5000/prometheus/prometheus:<tag>`	Ch. 11
`ceph config set mgr mgr/cephadm/container_image_grafana registry.internal:5000/ceph/grafana:<tag>`	Ch. 11
`ceph config set mgr mgr/cephadm/log_to_cluster_level debug`	Ch. 11
`ceph config dump --format json \| jq -S 'sort_by(.section, .name)' > /tmp/live-config.json`	Ch. 11
`ceph config set global osd_pool_default_flag_ec_optimizations true` Tentacle+	Ch. 13
`ceph config get client objecter_inflight_ops`	Ch. 14
`ceph config get client objecter_inflight_op_bytes`	Ch. 14
`ceph config set client.devapp rados_osd_op_timeout 30`	Ch. 14
`ceph config get osd osd_class_dir`	Ch. 15
`ceph config set client.rgw rgw_lifecycle_work_time "00:00-23:59"`	Ch. 16
`ceph config get mon mon_stretch_cluster_recovery_ratio`	Ch. 18
`ceph config set mgr mgr/balancer/upmap_max_deviation 1`	Ch. 19
`ceph config dump \| head`	Ch. 19
`ceph config get osd.12 osd_memory_target`	Ch. 19
`ceph config show osd.12 \| grep -E "osd_memory_target\|osd_max_backfills"`	Ch. 19
`ceph config set osd/class:hdd osd_memory_target 6G`	Ch. 19
`ceph config set osd.12 debug_osd 10/10`	Ch. 19
`ceph config rm osd.12 debug_osd`	Ch. 19
`ceph config log 20`	Ch. 19
`ceph config reset 1234`	Ch. 19
`ceph config help osd_op_queue`	Ch. 19
`ceph config get client.ceph-exporter exporter_prio_limit`	Ch. 20
`ceph config get mgr mgr_stats_period`	Ch. 20
`ceph config set osd osd_mclock_profile high_client_ops`	Ch. 21
`ceph config show osd.0 \| grep -E "osd_mclock_max_capacity_iops\|osd_mclock_scheduler"`	Ch. 21
`ceph config set osd/host:stor-a1 osd_memory_target 8G`	Ch. 21
`ceph config show osd.3 bluestore_throttle_cost_per_io`	Ch. 21
`ceph config set osd.3 debug_osd 20 ; ceph config set osd.3 debug_bluestore 20`	Ch. 22
`ceph config rm osd.3 debug_osd ; ceph config rm osd.3 debug_bluestore`	Ch. 22
`ceph config get osd.4 bdev_ioring`	Ch. 23
`ceph config get osd.4 osd_memory_target`	Ch. 23
`ceph config set osd bluestore_rocksdb_options_annex "max_background_jobs=6"`	Ch. 25
`ceph config set osd crimson_prometheus_port_base 29180` main	Ch. 30
`ceph config set osd debug_osd 20/20` Squid+	Ch. 32
ceph tell
`ceph tell mds.cephfs:0 session ls \| jq '.[] \| {id, num_caps, "client": .client_metadata.hostname}'`	Ch. 6
`ceph tell mds.cephfs:0 dump_ops_in_flight \| jq '.ops[0]'`	Ch. 6
`ceph tell mds.cephfs:0 perf dump mds_log \| jq '.mds_log \| {ev, seg, evtrm}'`	Ch. 6
`ceph tell mds.cephfs:0 cache status`	Ch. 6
`ceph tell mds.fs01:0 session ls`	Ch. 6
`ceph tell mds.fs01:0 perf dump mds_server`	Ch. 6
`ceph tell mds.fs01:0 dump_ops_in_flight`	Ch. 6
`ceph tell mds.fs01:0 cache status`	Ch. 6
`ceph tell mds.fs01:0 client ls \| grep -E '"id"\|hostname'`	Ch. 6
`ceph tell mds.fs01:0 client evict id=4305`	Ch. 6
`ceph tell osd.* config set debug_osd 10/10`	Ch. 7
`ceph tell osd.* config set debug_osd 1/5`	Ch. 7
`ceph tell mon.ceph-node1 mon_status`	Ch. 8
`ceph tell mgr.$(ceph mgr stat \| jq -r .active_name) config set debug_mgr 20`	Ch. 9
`ceph tell osd.0 bench`	Ch. 10
`ceph tell osd.0 bench 12288000 4096 4194304 100`	Ch. 10
`ceph tell osd.3 dump_ops_in_flight`	Ch. 10
`ceph tell osd.3 dump_historic_slow_ops`	Ch. 10
`ceph tell osd.* version`	Ch. 10
`ceph tell mon.* sessions \| jq length`	Ch. 10
`ceph tell mds.fs01:0 perf dump`	Ch. 10
`ceph tell mds.fs01:0 perf dump mds_server \| jq '.mds_server \| {req: .handle_client_request}'`	Ch. 10
`ceph tell osd.4 bench`	Ch. 10
`ceph tell osd.4 bench 12288000 4096 4194304 100`	Ch. 10
`ceph tell osd.3 dump_mempools \| jq '.mempool.by_pool \| to_entries \| sort_by(-.value.bytes) \| .[0:5]'`	Ch. 10
`ceph tell osd.3 heap stats`	Ch. 10
`ceph tell osd.3 heap release`	Ch. 10
`ceph tell osd.3 heap start_profiler`	Ch. 10
`ceph tell osd.3 heap dump`	Ch. 10
`ceph tell osd.3 heap stop_profiler`	Ch. 10
`ceph tell osd.3 bluefs stats`	Ch. 12
`ceph tell osd.3 bluestore allocator score block`	Ch. 12
`ceph tell osd.3 bluestore allocator fragmentation histogram block`	Ch. 12
`ceph tell osd.4 config get osd_backoff_on_peering`	Ch. 14
`ceph tell osd.4 dump_mempools \| jq '.mempool.by_pool \| to_entries \| sort_by(-.value.bytes) \| .[:6]'`	Ch. 23
`ceph tell osd.4 dump_mempools \| jq '.mempool.total'`	Ch. 23
`ceph tell osd.4 compact`	Ch. 25
`ceph tell osd.4 config get osd_recovery_max_active_hdd`	Ch. 27
`ceph tell osd.0 dump_metrics reactor_utilization` main	Ch. 30
`ceph tell osd.0 dump_store_shards` main	Ch. 30
`ceph tell osd.4 dump_scrubs` Squid+	Ch. 32
`ceph tell osd.4 dump_scrub_reservations` Squid+	Ch. 32
`ceph tell osd.4 dump_recovery_reservations` Squid+	Ch. 32
`ceph tell osd.3 dump_mempools \| jq '.mempool.by_pool \| to_entries \| sort_by(-.value.bytes) \| .[:5]'`	Ch. 33
`ceph tell osd.3 perf dump \| jq '."bluestore-pricache:meta"'`	Ch. 33
ceph fs
`ceph fs volume create cephfs01`	Ch. 1
`ceph fs volume create manila-fs`	Ch. 3
`ceph fs volume create fs01 --placement="2 ceph-node3 ceph-node4"`	Ch. 6
`ceph fs ls`	Ch. 6
`ceph fs status fs01`	Ch. 6
`ceph fs new fs02 cephfs.fs02.meta cephfs.fs02.data`	Ch. 6
`ceph fs set fs01 allow_standby_replay true`	Ch. 6
`ceph fs set fs01 standby_count_wanted 1`	Ch. 6
`ceph fs set fs01 max_mds 2`	Ch. 6
`ceph fs authorize fs01 client.webapp /web rw`	Ch. 6
`ceph fs authorize fs01 client.analyst / r /reports rw`	Ch. 6
`ceph fs subvolumegroup create fs01 smb` Tentacle+	Ch. 6
`ceph fs subvolume create fs01 docs --group_name smb --size 107374182400` Tentacle+	Ch. 6
`ceph fs status ; ceph health detail \| grep -E 'MDS_\|CLIENT'`	Ch. 6
`ceph fs subvolumegroup create fs01 teams`	Ch. 6
`ceph fs subvolume create fs01 data-eng --group_name teams --size 53687091200 --mode 770`	Ch. 6
`ceph fs subvolume getpath fs01 data-eng --group_name teams`	Ch. 6
`ceph fs subvolume info fs01 data-eng --group_name teams`	Ch. 6
`ceph fs subvolume resize fs01 data-eng 107374182400 --group_name teams --no_shrink`	Ch. 6
`ceph fs subvolume ls fs01 --group_name teams`	Ch. 6
`ceph fs subvolume snapshot create fs01 data-eng snap-2026-11-15 --group_name teams`	Ch. 6
`ceph fs subvolume snapshot clone fs01 data-eng snap-2026-11-15 data-eng-test --group_name teams`	Ch. 6
`ceph fs clone status fs01 data-eng-test`	Ch. 6
`ceph fs snap-schedule add /projects 1h --fs fs01`	Ch. 6
`ceph fs snap-schedule retention add /projects h 24 --fs fs01`	Ch. 6
`ceph fs snap-schedule retention add /projects d 7 --fs fs01`	Ch. 6
`ceph fs snap-schedule status /projects --fs fs01`	Ch. 6
`ceph fs add_data_pool fs01 cephfs.fs01.ec`	Ch. 6
`ceph fs authorize fs01 client.mirror_remote / rwps`	Ch. 6
`ceph fs snapshot mirror peer_bootstrap create fs01 client.mirror_remote site-b`	Ch. 6
`ceph fs snapshot mirror enable fs01`	Ch. 6
`ceph fs snapshot mirror peer_bootstrap import fs01 <token>`	Ch. 6
`ceph fs snapshot mirror add fs01 /projects`	Ch. 6
`ceph fs snapshot mirror peer_list fs01`	Ch. 6
`ceph fs status`	Ch. 6
`ceph fs subvolumegroup create fs01 analytics`	Ch. 9
`ceph fs subvolume create fs01 shared --group_name analytics --size 2199023255552`	Ch. 9
`ceph fs subvolume getpath fs01 shared --group_name analytics`	Ch. 9
ceph pg
`ceph pg ls-by-pool lab-pool \| head`	Ch. 1
`ceph pg force-recovery 2.1f`	Ch. 7
`ceph pg force-backfill 2.1f 2.20`	Ch. 7
`ceph pg cancel-force-recovery 2.1f`	Ch. 7
`ceph pg deep-scrub 2.1f`	Ch. 7
`ceph pg map 3.1f`	Ch. 8
`ceph pg stat`	Ch. 8
`ceph pg dump_stuck inactive`	Ch. 8
`ceph pg dump_stuck unclean`	Ch. 8
`ceph pg ls degraded`	Ch. 8
`ceph pg 3.1f query \| jq '.recovery_state'`	Ch. 8
`ceph pg ls-by-pool tenant-a-db \| head`	Ch. 8
`ceph pg 5.3a query \| jq .state`	Ch. 10
`ceph pg ls-by-pool ecpool \| grep -c 2147483647`	Ch. 13
`ceph pg 5.1as2 query \| jq '.info \| {last_update, last_complete, log_tail}'`	Ch. 13
`ceph pg 5.1a list_unfound`	Ch. 13
`ceph pg dump_stuck stale`	Ch. 17
`ceph pg ls down`	Ch. 17
`ceph pg 2.1f query \| jq '.state, .recovery_state[0]'`	Ch. 17
`ceph pg repair 2.1f`	Ch. 17
`ceph pg 2.5 list_unfound`	Ch. 17
`ceph pg 2.5 query \| jq '.recovery_state[] \| select(.name\|test("Active")) \| .might_have_unfound'`	Ch. 17
`ceph pg 2.5 mark_unfound_lost revert`	Ch. 17
`ceph pg <pgid> query \| jq '.info.stats.stat_sum.num_objects_degraded'`	Ch. 27
`ceph pg 2.1f query \| jq '.recovery_state'` Squid+	Ch. 32
ceph daemon
`ceph daemon osd.4 bluestore allocator score block`	Ch. 1
`ceph daemon osd.3 config show`	Ch. 7
`ceph daemon mon.a sessions \| jq length`	Ch. 7
`ceph daemon osd.4 dump_ops_in_flight \| jq '.ops[] \| select(.description\|test("ECSub"))'`	Ch. 13
`ceph daemon osd.4 dump_blocked_ops`	Ch. 14
`ceph daemon client.admin.<pid>.<cookie> objecter_requests`	Ch. 14
`ceph daemon osd.12 status`	Ch. 17
`ceph daemon osd.7 dump_ops_in_flight \| jq '.ops[] \| {age, description, last: .type_data.events[-1].event}'`	Ch. 17
`ceph daemon osd.7 dump_blocked_ops`	Ch. 17
`ceph daemon osd.7 dump_historic_slow_ops \| jq '.ops[0].type_data.events'`	Ch. 17
`ceph daemon osd.7 dump_historic_ops_by_duration \| jq '.ops[0:3][] \| {duration, description}'`	Ch. 17
`ceph daemon osd.3 counter schema \| jq 'keys' \| head`	Ch. 20
`ceph daemon osd.3 counter dump \| jq '.osd[0].counters \| {op_w, op_r}'`	Ch. 20
`ceph daemon osd.3 perf dump osd \| jq '.osd.op_w_latency'`	Ch. 20
`ceph daemon osd.3 perf schema \| jq '.osd.op_r_latency'`	Ch. 20
`ceph daemon osd.3 perf dump \| jq '.osd.op_r_latency'`	Ch. 20
`ceph daemon osd.3 counter dump \| jq 'keys' \| head`	Ch. 20
`ceph daemon osd.3 dump_mempools \| jq '.mempool.by_pool \| to_entries \| sort_by(-.value.bytes) \| .[0:5]'`	Ch. 21
`ceph daemon osd.3 perf dump \| jq '.prioritycache'`	Ch. 21
`ceph daemon osd.3 dump_mempools \| jq '.mempool.total'`	Ch. 21
`ceph daemon osd.4 bluefs stats`	Ch. 24
`ceph daemon osd.4 bluestore allocator fragmentation histogram block`	Ch. 24
`ceph daemon osd.4 perf dump bluestore \| jq '.bluestore \| {onodes, onode_hits, onode_misses}'`	Ch. 24
`ceph daemon osd.4 messenger dump`	Ch. 26
`ceph daemon osd.12 dump_ops_in_flight \| jq '.ops[] \| {description, age, flag_point}'`	Ch. 34
ceph dashboard
`ceph dashboard create-self-signed-cert`	Ch. 9
`ceph dashboard set-ssl-certificate -i dashboard.crt`	Ch. 9
`ceph dashboard set-ssl-certificate-key -i dashboard.key`	Ch. 9
`ceph dashboard ac-user-create admin2 -i /root/dash-admin.pw administrator`	Ch. 9
`ceph dashboard ac-user-show admin2`	Ch. 9
`ceph dashboard ac-role-show`	Ch. 9
`ceph dashboard ac-role-create storage-ops "Ops team: OSD and pool only"`	Ch. 9
`ceph dashboard ac-role-add-scope-perms storage-ops osd read update`	Ch. 9
`ceph dashboard ac-role-add-scope-perms storage-ops pool read create update`	Ch. 9
`ceph dashboard ac-user-create --pwd_update_required alice -i /root/alice.pw storage-ops`	Ch. 9
`ceph dashboard ac-user-set-roles alice storage-ops read-only`	Ch. 9
`ceph dashboard ac-user-disable alice`	Ch. 9
`ceph dashboard set-pwd-policy-enabled true`	Ch. 9
`ceph dashboard set-pwd-policy-min-length 12`	Ch. 9
`ceph dashboard set-pwd-policy-check-complexity-enabled true`	Ch. 9
`ceph dashboard motd set warning 2h "Maintenance on ceph-node3 in progress"`	Ch. 9
`ceph dashboard motd clear`	Ch. 9
`ceph dashboard sso setup saml2 https://ceph-dash.example.internal \`	Ch. 9
`ceph dashboard sso status`	Ch. 9
`ceph dashboard sso enable saml2`	Ch. 9
`ceph dashboard feature status`	Ch. 9
`ceph dashboard feature disable iscsi mirroring`	Ch. 9
`ceph dashboard feature enable mirroring`	Ch. 9
`ceph dashboard get-prometheus-api-host`	Ch. 20
ceph auth
`ceph auth get-or-create client.rbd-lab \`	Ch. 2
`ceph auth get client.rbd-lab`	Ch. 2
`ceph auth get-or-create client.glance \`	Ch. 3
`ceph auth get-or-create client.cinder \`	Ch. 3
`ceph auth get-or-create client.cinder-backup \`	Ch. 3
`ceph auth get client.glance \| ssh glance-host sudo tee /etc/ceph/ceph.client.glance.keyring`	Ch. 3
`ceph auth get client.cinder \| ssh cinder-host sudo tee /etc/ceph/ceph.client.cinder.keyring`	Ch. 3
`ceph auth get client.cinder \| ssh compute1 sudo tee /etc/ceph/ceph.client.cinder.keyring`	Ch. 3
`ceph auth get client.cinder-backup \| ssh backup-host sudo tee /etc/ceph/ceph.client.cinder-backup.keyring`	Ch. 3
`ceph auth get-or-create client.manila mon 'allow r' mgr 'allow rw'`	Ch. 3
`ceph auth get client.webapp`	Ch. 6
`ceph auth get client.webapp -o /etc/ceph/ceph.client.webapp.keyring`	Ch. 6
`ceph auth ls`	Ch. 8
`ceph auth get client.admin`	Ch. 8
`ceph auth get-or-create client.vmhost mon 'profile rbd' osd 'profile rbd pool=vms' mgr 'profile rbd pool=vms'`	Ch. 8
`ceph auth get-or-create client.monitor mon 'allow r' mgr 'allow r'`	Ch. 8
`ceph auth get-or-create client.tenant1 mon 'allow r' osd 'allow rw pool=shared namespace=tenant1'`	Ch. 8
`ceph auth caps client.vmhost mon 'profile rbd' osd 'profile rbd pool=vms, profile rbd-read-only pool=images'`	Ch. 8
`ceph auth rm client.old-app`	Ch. 8
`ceph auth get-or-create client.devapp mon 'allow r' osd 'allow rwx pool=devpool' \`	Ch. 14
`ceph auth caps client.devapp mon 'allow r' \`	Ch. 15
`ceph auth caps client.counter mon 'allow r' \`	Ch. 15
ceph mgr
`ceph mgr module enable rgw`	Ch. 5
`ceph mgr module enable nfs`	Ch. 6
`ceph mgr module enable smb` Tentacle+	Ch. 6
`ceph mgr module enable snap_schedule`	Ch. 6
`ceph mgr module enable mirroring`	Ch. 6
`ceph mgr module ls \| grep -i dashboard`	Ch. 9
`ceph mgr module enable dashboard`	Ch. 9
`ceph mgr services`	Ch. 9
`ceph mgr module disable dashboard && ceph mgr module enable dashboard`	Ch. 9
`ceph mgr module disable prometheus && ceph mgr module enable prometheus`	Ch. 9
`ceph mgr module ls \| jq '.enabled_modules'`	Ch. 9
`ceph mgr module enable pgwatch --force`	Ch. 9
`ceph mgr fail`	Ch. 9
`ceph mgr stat`	Ch. 9
`ceph mgr module enable prometheus`	Ch. 20
ceph mon
`ceph mon stat`	Ch. 7
`ceph mon dump \| grep election_strategy`	Ch. 7
`ceph mon set election_strategy connectivity`	Ch. 7
`ceph mon add disallowed_leader e`	Ch. 7
`ceph mon dump`	Ch. 8
`ceph mon dump \| grep -E "v2:\|v1:"`	Ch. 8
`ceph mon dump \| grep -E 'tiebreaker\|stretch\|location'`	Ch. 18
`ceph mon set_location a datacenter=site1`	Ch. 18
`ceph mon set_location b datacenter=site1`	Ch. 18
`ceph mon set_location c datacenter=site2`	Ch. 18
`ceph mon set_location d datacenter=site2`	Ch. 18
`ceph mon set_location e datacenter=site3`	Ch. 18
`ceph mon enable_stretch_mode e stretch_rule datacenter`	Ch. 18
ceph health
`ceph health detail \| grep -A3 LARGE_OMAP`	Ch. 4
`ceph health detail \| grep -i mds`	Ch. 6
`ceph health detail`	Ch. 7
`ceph health detail \| grep -i scrub`	Ch. 7
`ceph health detail \| grep -i cephadm`	Ch. 10
`ceph health mute CEPHADM_STRAY_DAEMON 1h`	Ch. 11
`ceph health detail \| grep -i spillover`	Ch. 12
`ceph health detail --format json-pretty \| jq '.checks \| keys'`	Ch. 17
`ceph health detail \| grep -A3 SLOW_OPS`	Ch. 17
`ceph health detail \| grep -E "inconsistent\|scrub errors"`	Ch. 17
ceph balancer
`ceph balancer status`	Ch. 7
`ceph balancer mode upmap`	Ch. 7
`ceph balancer on`	Ch. 7
`ceph balancer eval`	Ch. 19
`ceph balancer status && ceph balancer eval`	Ch. 28
ceph nfs
`ceph nfs cluster create nfs01 "2 ceph-node2 ceph-node3" --ingress --virtual_ip 192.168.10.50/24`	Ch. 6
`ceph nfs cluster info nfs01`	Ch. 6
`ceph nfs export create cephfs --cluster-id nfs01 --pseudo-path /shared --fsname fs01 --path /web`	Ch. 6
`ceph nfs export ls nfs01 --detailed`	Ch. 6
ceph telemetry
`ceph telemetry preview`	Ch. 9
`ceph telemetry show`	Ch. 9
`ceph telemetry on --license sharing-1-0`	Ch. 9
`ceph telemetry off`	Ch. 9
ceph df
`ceph df`	Ch. 1
`ceph df detail`	Ch. 1
`ceph df detail \| grep -E 'POOL\|rgw-data'`	Ch. 12
ceph rgw
`ceph rgw realm bootstrap -i primary.yaml`	Ch. 5
`ceph rgw realm tokens`	Ch. 5
`ceph rgw zone create -i secondary.yaml`	Ch. 5
ceph smb
`ceph smb cluster create smb01 user --define-user-pass=alice%S3cret-Ch@ngeMe --placement="1 ceph-node2"` Tentacle+	Ch. 6
`ceph smb share create smb01 docs fs01 / --subvolume=smb/docs` Tentacle+	Ch. 6
`ceph smb show` Tentacle+	Ch. 6
ceph quorum_status
`ceph quorum_status -f json-pretty \| jq '.quorum_names'`	Ch. 7
`ceph quorum_status \| jq '.quorum_leader_name, .quorum_names'`	Ch. 7
`ceph quorum_status -f json-pretty`	Ch. 8
ceph log
`ceph log last 20 info cluster`	Ch. 9
`ceph log last cephadm 50`	Ch. 11
`ceph log last 100`	Ch. 17
ceph nvmeof
`ceph nvmeof subsystem add --nqn nqn.2016-06.io.spdk:cnode1` Tentacle+	Ch. 31
`ceph nvmeof listener add --nqn nqn.2016-06.io.spdk:cnode1 --host-name host01 --traddr 10.0.0.11 --trsvcid 4420` Tentacle+	Ch. 31
`ceph nvmeof namespace add --nqn nqn.2016-06.io.spdk:cnode1 --rbd-pool rbd --rbd-image-name vol1` Tentacle+	Ch. 31
ceph device
`ceph device ls-by-daemon osd.7`	Ch. 7
`ceph device light on <devid> ident`	Ch. 7
ceph crash
`ceph crash ls-new`	Ch. 17
`ceph crash ls \| grep osd.12`	Ch. 17
ceph -v
`ceph -v`	Ch. 1
ceph status
`ceph status`	Ch. 1
ceph -s
`ceph -s`	Ch. 1
ceph --id
`ceph --id rbd-lab -s`	Ch. 2
ceph versions
`ceph versions`	Ch. 7
ceph -W
`ceph -W cephadm`	Ch. 7
ceph features
`ceph features`	Ch. 8
ceph pgwatch
`ceph pgwatch status`	Ch. 9
ceph cephadm
`ceph cephadm check-host ceph-node2`	Ch. 11
ceph nvme-gw
`ceph nvme-gw show POOL GROUP` Tentacle+	Ch. 31

rados · direct work with RADOS objects#

rados -p
`rados -p lab-pool put hello /tmp/hello.txt`	Ch. 1
`rados -p lab-pool ls`	Ch. 1
`rados -p lab-pool stat hello`	Ch. 1
`rados -p rbd listomapvals rbd_directory \| head`	Ch. 2
`rados -p rbd listomapkeys rbd_header.$(rados -p rbd get rbd_id.vm1 - \| strings)`	Ch. 2
`rados -p rbd ls \| grep -c "^rbd_data.<id>"`	Ch. 2
`rados -p benchpool cleanup`	Ch. 10
`rados -p hot-storage cache-flush-evict-all`	Ch. 16
`rados -p hot-storage ls \| head`	Ch. 16
`rados -p vmpool ls \| grep '^journal_data\.' \| sort -t. -k4 -n \| tail -4`	Ch. 18
`rados -p test put obj1 /etc/hosts`	Ch. 22
`rados -p rbd getxattr rbd_header.5f3a2b8c1d4e _ > /tmp/oi.bin`	Ch. 34
`rados -p rbd stat rbd_header.5f3a2b8c1d4e`	Ch. 34
`rados -p rbd listwatchers rbd_header.5f3a2b8c1d4e`	Ch. 36
`rados -p rbd notify rbd_header.5f3a2b8c1d4e hello`	Ch. 36
`rados -p rbd listsnaps rbd_data.5f3a.0000000000000000`	Ch. 36
`rados -p rbd listomapkeys rbd_header.5f3a2b8c1d4e`	Ch. 37
`rados -p rbd getomapval rbd_header.5f3a2b8c1d4e object_prefix`	Ch. 37
`rados -p default.rgw.meta -N users.keys ls \| head -3`	Ch. 40
rados bench
`rados bench -p benchpool 60 write -b 4M -t 16 --no-cleanup`	Ch. 10
`rados bench -p benchpool 60 seq -t 16`	Ch. 10
`rados bench -p benchpool 60 rand -t 16`	Ch. 10
`rados bench -p benchpool 60 write -b 4K -t 64 --no-cleanup`	Ch. 10
`rados bench -p bench 60 write -t 32 -b 4M --no-cleanup`	Ch. 10
`rados bench -p bench 60 rand -t 32 ; rados -p bench cleanup`	Ch. 10
`rados bench -t T: always exactly T operations in flight ← each completion sends a new op`	Ch. 10
rados list-inconsistent-pg
`rados list-inconsistent-pg rbd`	Ch. 17
rados list-inconsistent-obj
`rados list-inconsistent-obj 2.1f --format=json-pretty` Squid+	Ch. 32

rbd · RBD image management#

rbd mirror
`rbd mirror pool enable --site-name site-a vms image`	Ch. 2
`rbd mirror pool enable --site-name site-b vms image`	Ch. 2
`rbd mirror pool peer bootstrap create --site-name site-a vms > /root/vms-token`	Ch. 2
`rbd mirror pool peer bootstrap import --site-name site-b --direction rx-only vms /root/vms-token`	Ch. 2
`rbd mirror pool info vms`	Ch. 2
`rbd mirror image enable vms/db01 snapshot`	Ch. 2
`rbd mirror snapshot schedule add --pool vms 15m`	Ch. 2
`rbd mirror snapshot schedule ls --pool vms --recursive`	Ch. 2
`rbd mirror image enable vms/app01 journal`	Ch. 2
`rbd mirror pool status vms --verbose`	Ch. 2
`rbd mirror image status vms/db01`	Ch. 2
`rbd mirror pool peer bootstrap import --site-name site-b --direction rx-tx vms /root/vms-token`	Ch. 2
`rbd mirror pool enable --site-name site-a vmpool image`	Ch. 18
`rbd mirror pool peer bootstrap create --site-name site-a vmpool > /tmp/token`	Ch. 18
`rbd mirror pool enable --site-name site-b vmpool image`	Ch. 18
`rbd mirror pool peer bootstrap import --site-name site-b --direction rx-tx vmpool /tmp/token`	Ch. 18
`rbd mirror image enable vmpool/db01 snapshot`	Ch. 18
`rbd mirror snapshot schedule add --pool vmpool 15m`	Ch. 18
`rbd mirror snapshot schedule status --pool vmpool`	Ch. 18
`rbd mirror pool status vmpool --verbose`	Ch. 18
rbd --cluster
`rbd --cluster site-a mirror image demote vms/db01`	Ch. 2
`rbd --cluster site-b mirror image status vms/db01`	Ch. 2
`rbd --cluster site-b mirror image promote vms/db01`	Ch. 2
`rbd --cluster site-b mirror image promote --force vms/db01`	Ch. 2
`rbd --cluster site-a mirror image resync vms/db01`	Ch. 2
`rbd --cluster site-a mirror image status vms/db01`	Ch. 2
`rbd --cluster site-b mirror image demote vms/db01`	Ch. 2
`rbd --cluster site-a mirror image promote vms/db01`	Ch. 2
`rbd --cluster site-a mirror image demote vmpool/db01`	Ch. 18
`rbd --cluster site-b mirror image status vmpool/db01`	Ch. 18
`rbd --cluster site-b mirror image promote vmpool/db01`	Ch. 18
`rbd --cluster site-b mirror image promote --force vmpool/db01`	Ch. 18
`rbd --cluster site-a mirror image resync vmpool/db01`	Ch. 18
rbd create
`rbd create rbd/disk01 --size 10G`	Ch. 1
`rbd create rbd/golden-os --size 20G`	Ch. 2
`rbd create vms/db01 --size 50G`	Ch. 2
`rbd create vms/app01 --size 20G --image-feature exclusive-lock,journaling`	Ch. 2
`rbd create analytics-rbd/warehouse --size 500G`	Ch. 9
`rbd create rbd/benchimg --size 100G`	Ch. 10
`rbd create rbd/vm01 --size 200G --data-pool ecpool`	Ch. 13
`rbd create --size 1T --data-pool archive vm-fast/backup01`	Ch. 16
`rbd create rbd/vm01 --size 10G --stripe-unit 64K --stripe-count 4 --data-pool rbd-ec`	Ch. 37
rbd info
`rbd info rbd/disk01`	Ch. 1
`rbd info rbd/vm01 \| grep parent`	Ch. 2
`rbd info rbd/vm02 \| grep parent`	Ch. 2
`rbd info rbd/vm1 \| grep -E 'block_name_prefix\|order\|features'`	Ch. 2
`rbd info volumes/volume-aaa \| grep -A3 parent`	Ch. 3
`rbd info volumes/volume-<uuid> --id cinder \| grep parent`	Ch. 3
`rbd info rbd/vm01 \| grep data_pool`	Ch. 13
`rbd info rbd/vm01`	Ch. 37
rbd snap
`rbd snap create rbd/disk01@before-upgrade`	Ch. 2
`rbd snap ls rbd/disk01`	Ch. 2
`rbd snap rollback rbd/disk01@before-upgrade`	Ch. 2
`rbd snap rm rbd/disk01@before-upgrade`	Ch. 2
`rbd snap purge rbd/disk01`	Ch. 2
`rbd snap create rbd/golden-os@v1`	Ch. 2
`rbd snap protect rbd/golden-os@v1`	Ch. 2
`rbd snap unprotect rbd/golden-os@v1`	Ch. 2
rbd --id
`rbd --id rbd-lab ls rbd`	Ch. 2
`rbd --id rbd-lab create rbd/disk01 --size 10G`	Ch. 2
`rbd --id rbd-lab ls -l rbd`	Ch. 2
`rbd --id rbd-lab info rbd/disk01`	Ch. 2
`rbd --id rbd-lab resize rbd/disk01 --size 20G`	Ch. 2
rbd journal
`rbd journal info --pool vmpool --image db01`	Ch. 18
`rbd journal status --pool vmpool --image db01`	Ch. 18
`rbd journal inspect --pool vmpool --image db01 --verbose \| head`	Ch. 18
`rbd journal info --pool rbd --image vm01`	Ch. 39
`rbd journal status --pool rbd --image vm01`	Ch. 39
rbd trash
`rbd trash mv rbd/vm01 --expires-at "2026-12-01"`	Ch. 2
`rbd trash ls rbd`	Ch. 2
`rbd trash restore rbd/<image-id>`	Ch. 2
`rbd trash purge rbd`	Ch. 2
rbd pool
`rbd pool init rbd`	Ch. 1
`rbd pool init $p`	Ch. 3
`rbd pool init analytics-rbd`	Ch. 9
rbd du
`rbd du rbd/disk01`	Ch. 2
`rbd du rbd/vm1`	Ch. 2
`rbd du volumes/volume-aaa`	Ch. 3
rbd migration
`rbd migration prepare rbd/disk01 fast-nvme/disk01`	Ch. 2
`rbd migration execute fast-nvme/disk01`	Ch. 2
`rbd migration commit fast-nvme/disk01`	Ch. 2
rbd config
`rbd config image set rbd/vm01 rbd_qos_iops_limit 2000`	Ch. 2
`rbd config pool set rbd rbd_qos_bps_limit 209715200`	Ch. 2
`rbd config image ls rbd/vm01 \| grep qos`	Ch. 2
rbd bench
`rbd bench --io-type write --io-size 4K --io-threads 32 --io-total 10G --io-pattern rand rbd/benchimg`	Ch. 10
`rbd bench --io-type read --io-size 4K --io-threads 32 --io-total 10G --io-pattern rand rbd/benchimg`	Ch. 10
`rbd bench --io-type readwrite --rw-mix-read 70 --io-size 16K --io-pattern rand rbd/benchimg`	Ch. 10
rbd clone
`rbd clone rbd/golden-os@v1 rbd/vm01`	Ch. 2
`rbd clone rbd/golden-os@v1 rbd/vm02`	Ch. 2
rbd children
`rbd children rbd/golden-os@v1`	Ch. 2
`rbd children images/ubuntu@snap`	Ch. 3
rbd flatten
`rbd flatten rbd/vm02`	Ch. 2
`rbd flatten volumes/volume-aaa`	Ch. 3
rbd status
`rbd status rbd/vm1`	Ch. 2
`rbd status rbd/vm01`	Ch. 38
rbd lock
`rbd lock ls rbd/vm1`	Ch. 2
`rbd lock ls rbd/vm01`	Ch. 38
rbd -p
`rbd -p images ls -l --id glance`	Ch. 3
`rbd -p volumes ls -l --id cinder`	Ch. 3
rbd cache
`rbd cache = true`	Ch. 3
`rbd cache writethrough until flush = true`	Ch. 3
rbd device
`rbd device list`	Ch. 2
rbd resize
`rbd resize rbd/disk01 --size 5G --allow-shrink`	Ch. 2
rbd feature
`rbd feature disable rbd/vm1 fast-diff object-map`	Ch. 2
rbd encryption
`rbd encryption format rbd/secure01 luks2 /root/passphrase.bin`	Ch. 2
rbd concurrent
`rbd concurrent management ops = 20`	Ch. 3
rbd map
`rbd map rbd/img1 -o ms_mode=prefer-crc,queue_depth=256,alloc_size=65536`	Ch. 29
rbd showmapped
`rbd showmapped`	Ch. 29
rbd showmapped;
`rbd showmapped; cat /sys/bus/rbd/devices/0/features`	Ch. 29

radosgw-admin · RGW administration#

radosgw-admin user
`radosgw-admin user create --uid=alice --display-name="Alice" --email=alice@example.com`	Ch. 4
`radosgw-admin user info --uid=alice`	Ch. 4
`radosgw-admin user list`	Ch. 4
`radosgw-admin user suspend --uid=alice`	Ch. 4
`radosgw-admin user enable --uid=alice`	Ch. 4
`radosgw-admin user create --tenant=acme --uid=bob --display-name="Bob at ACME"`	Ch. 4
`radosgw-admin user create --uid=acme-root --display-name="ACME root" \` Squid+	Ch. 4
`radosgw-admin user create --uid=sync-user --display-name="Multisite sync" --system`	Ch. 5
`radosgw-admin user create --uid=tester --display-name="Multisite tester"`	Ch. 5
`radosgw-admin user info --uid=tester`	Ch. 5
`radosgw-admin user create --uid=analytics --display-name="Analytics team"`	Ch. 9
`radosgw-admin user create --uid=bench --display-name="Benchmark user"`	Ch. 10
radosgw-admin sync
`radosgw-admin sync status`	Ch. 5
`radosgw-admin sync status --rgw-zone=eu-2`	Ch. 5
`radosgw-admin sync error list \| head -50`	Ch. 5
`radosgw-admin sync error trim`	Ch. 5
`radosgw-admin sync error list \| jq '.[0].entries[:5]'`	Ch. 5
`radosgw-admin sync group create --group-id=global --status=allowed`	Ch. 5
`radosgw-admin sync group flow create --group-id=global --flow-id=eu-mesh \`	Ch. 5
`radosgw-admin sync group pipe create --group-id=global --pipe-id=all \`	Ch. 5
`radosgw-admin sync group create --bucket=critical-data --group-id=critical --status=enabled`	Ch. 5
`radosgw-admin sync group pipe create --bucket=critical-data --group-id=critical \`	Ch. 5
`radosgw-admin sync info --bucket=critical-data`	Ch. 5
`radosgw-admin sync status \| grep features`	Ch. 5
radosgw-admin bucket
`radosgw-admin bucket stats --bucket=photos \| jq '.num_shards, .usage'`	Ch. 4
`radosgw-admin bucket limit check \| jq '.[].buckets[] \| select(.fill_status!="OK")'`	Ch. 4
`radosgw-admin bucket check --bucket=photos --check-objects`	Ch. 4
`radosgw-admin bucket list`	Ch. 4
`radosgw-admin bucket stats --bucket=photos`	Ch. 4
`radosgw-admin bucket limit check`	Ch. 4
`radosgw-admin bucket check --bucket=photos --check-objects --fix`	Ch. 4
`radosgw-admin bucket sync status --bucket=photos`	Ch. 5
`radosgw-admin bucket sync status --bucket=ms-test`	Ch. 5
`radosgw-admin bucket sync status --bucket logs --source-zone zone-a`	Ch. 42
radosgw-admin quota
`radosgw-admin quota set --quota-scope=user --uid=alice --max-size=500G --max-objects=10000000`	Ch. 4
`radosgw-admin quota enable --quota-scope=user --uid=alice`	Ch. 4
`radosgw-admin quota set --quota-scope=bucket --uid=alice --max-objects=5000000`	Ch. 4
`radosgw-admin quota enable --quota-scope=bucket --uid=alice`	Ch. 4
`radosgw-admin quota set --quota-scope=account --account-id=<account-id> --max-size=10T` Squid+	Ch. 4
`radosgw-admin quota enable --quota-scope=account --account-id=<account-id>` Squid+	Ch. 4
`radosgw-admin quota set --quota-scope=user --uid=analytics --max-size=10T`	Ch. 9
`radosgw-admin quota enable --quota-scope=user --uid=analytics`	Ch. 9
radosgw-admin zone
`radosgw-admin zone placement add --rgw-zone default \`	Ch. 4
`radosgw-admin zone create --rgw-zonegroup=europe --rgw-zone=eu-1 \`	Ch. 5
`radosgw-admin zone modify --rgw-zone=eu-1 --access-key=<AK> --secret=<SK>`	Ch. 5
`radosgw-admin zone create --rgw-zonegroup=europe --rgw-zone=eu-2 \`	Ch. 5
`radosgw-admin zone modify --rgw-zone=eu-2 --master --default`	Ch. 5
`radosgw-admin zone modify --rgw-zone=eu-1 --master --default`	Ch. 5
`radosgw-admin zone create --rgw-zonegroup=europe --rgw-zone=eu-archive \`	Ch. 5
radosgw-admin period
`radosgw-admin period update --commit`	Ch. 4
`radosgw-admin period pull --url=http://rgw-eu1.example.com:80 --access-key=<AK> --secret=<SK>`	Ch. 5
`radosgw-admin period get \| jq '.period_map.zonegroups[].zones[].name'`	Ch. 5
`radosgw-admin period pull --url=http://rgw-eu2.example.com:80 --access-key=<AK> --secret=<SK>`	Ch. 5
`radosgw-admin period get-current`	Ch. 42
`radosgw-admin period get \| jq '{id, epoch, realm_epoch, predecessor_uuid, master_zone}'`	Ch. 42
radosgw-admin ratelimit
`radosgw-admin ratelimit set --ratelimit-scope=user --uid=alice --max-read-ops=1000 --max-write-ops=200`	Ch. 4
`radosgw-admin ratelimit enable --ratelimit-scope=user --uid=alice`	Ch. 4
`radosgw-admin ratelimit set --ratelimit-scope=user --uid=alice --max-read-ops=600 --max-write-bytes=104857600`	Ch. 41
`radosgw-admin ratelimit get --ratelimit-scope=user --uid=alice`	Ch. 41
radosgw-admin lc
`radosgw-admin lc list`	Ch. 4
`radosgw-admin lc process --bucket logs-bucket`	Ch. 4
`radosgw-admin lc get --bucket photos`	Ch. 16
`radosgw-admin lc process --bucket photos`	Ch. 16
radosgw-admin realm
`radosgw-admin realm create --rgw-realm=gold --default`	Ch. 5
`radosgw-admin realm pull --url=http://rgw-eu1.example.com:80 \`	Ch. 5
`radosgw-admin realm default --rgw-realm=gold`	Ch. 5
`radosgw-admin realm pull --url=http://rgw-eu2.example.com:80 --access-key=<AK> --secret=<SK>`	Ch. 5
radosgw-admin reshard
`radosgw-admin reshard list ; radosgw-admin reshard status --bucket=photos`	Ch. 4
`radosgw-admin reshard list`	Ch. 4
`radosgw-admin reshard add --bucket=photos --num-shards=101`	Ch. 4
radosgw-admin gc
`radosgw-admin gc list --include-all \| jq length`	Ch. 4
`radosgw-admin gc list --include-all \| head`	Ch. 4
`radosgw-admin gc process`	Ch. 4
radosgw-admin zonegroup
`radosgw-admin zonegroup placement add --rgw-zonegroup default \`	Ch. 4
`radosgw-admin zonegroup create --rgw-zonegroup=europe \`	Ch. 5
`radosgw-admin zonegroup modify --rgw-zonegroup=europe --enable-feature=resharding`	Ch. 5
radosgw-admin metadata
`radosgw-admin metadata sync status`	Ch. 5
`radosgw-admin metadata get bucket:photos`	Ch. 40
`radosgw-admin metadata get bucket.instance:photos:a1b2c3d4-0000-4000-8000-1234567890ab.4123.7`	Ch. 40
radosgw-admin key
`radosgw-admin key create --uid=alice --key-type=s3 --gen-access-key --gen-secret`	Ch. 4
`radosgw-admin key create --subuser=alice:swift --key-type=swift --gen-secret`	Ch. 4
radosgw-admin data
`radosgw-admin data sync status --source-zone=eu-1`	Ch. 5
`radosgw-admin data sync status --source-zone=us-east --shard-id=7`	Ch. 5
radosgw-admin subuser
`radosgw-admin subuser create --uid=alice --subuser=alice:swift --access=full`	Ch. 4
radosgw-admin account
`radosgw-admin account create --account-name=acme --email=ops@acme.example` Squid+	Ch. 4
radosgw-admin object
`radosgw-admin object stat --bucket=photos --object=a.jpg \| jq '.manifest'`	Ch. 4
radosgw-admin usage
`radosgw-admin usage show --uid=alice --start-date=2026-11-01`	Ch. 4
radosgw-admin datalog
`radosgw-admin datalog status ; radosgw-admin mdlog status`	Ch. 5
radosgw-admin bilog
`radosgw-admin bilog list --bucket=photos --max-entries=5`	Ch. 5

cephadm · deployment and management with cephadm#

cephadm shell
`cephadm shell`	Ch. 1
`cephadm shell -- ceph -s`	Ch. 1
`cephadm shell --name osd.3 -- ceph daemon osd.3 help`	Ch. 10
`cephadm shell --name osd.3 -- ceph daemon osd.3 dump_ops_in_flight`	Ch. 10
`cephadm shell --name osd.3 -- ceph daemon osd.3 dump_historic_ops`	Ch. 10
`cephadm shell --name osd.3 -- ceph daemon osd.3 perf dump bluestore`	Ch. 10
`cephadm shell --name osd.3 -- ceph daemon osd.3 config diff`	Ch. 10
`cephadm shell --name osd.12`	Ch. 10
`cephadm shell --name osd.7`	Ch. 12
cephadm bootstrap
`cephadm bootstrap \`	Ch. 1
`cephadm bootstrap --mon-ip 192.168.10.11 --single-host-defaults`	Ch. 11
`cephadm bootstrap --mon-ip 192.168.10.11 --cluster-network 192.168.20.0/24 --apply-spec cluster.yaml`	Ch. 11
cephadm logs
`cephadm logs --name osd.3`	Ch. 7
`cephadm logs --name osd.12 -- -n 200`	Ch. 17
cephadm install
`cephadm install ceph-common`	Ch. 1
cephadm check-host
`cephadm check-host`	Ch. 10
cephadm ls
`cephadm ls`	Ch. 11
cephadm adopt
`cephadm adopt --style legacy --name mon.ceph-node1`	Ch. 11
cephadm --image
`cephadm --image registry.internal:5000/ceph/ceph:v20.2.5 bootstrap --mon-ip 192.168.10.11 \`	Ch. 11

ceph-volume · disk and OSD preparation#

ceph-volume lvm
`ceph-volume lvm new-db --osd-id 7 --osd-fsid <fsid> --target vg_nvme/db_osd7`	Ch. 12
`ceph-volume lvm migrate --osd-id 7 --osd-fsid <fsid> --from data --target vg_nvme/db_osd7`	Ch. 12

ceph-bluestore-tool · BlueStore inspection and repair#

ceph-bluestore-tool show-label
`ceph-bluestore-tool show-label --dev /dev/sdb`	Ch. 1
`ceph-bluestore-tool show-label --dev /dev/ceph-vg/osd-block-xxxx`	Ch. 12
`ceph-bluestore-tool show-label --dev /dev/ceph-xxx/osd-block-yyy`	Ch. 24
ceph-bluestore-tool bluefs-bdev-sizes
`ceph-bluestore-tool bluefs-bdev-sizes --path /var/lib/ceph/osd/ceph-7`	Ch. 12
`ceph-bluestore-tool bluefs-bdev-sizes --path /var/lib/ceph/osd/ceph-4`	Ch. 24
ceph-bluestore-tool reshard
`ceph-bluestore-tool reshard --path /var/lib/ceph/osd/ceph-7 --sharding "<spec>"`	Ch. 12
`ceph-bluestore-tool reshard --path /var/lib/ceph/osd/ceph-4 \`	Ch. 25
ceph-bluestore-tool fsck
`ceph-bluestore-tool fsck --path /var/lib/ceph/osd/ceph-7 --deep`	Ch. 12
ceph-bluestore-tool bluefs-bdev-expand
`ceph-bluestore-tool bluefs-bdev-expand --path /var/lib/ceph/osd/ceph-7`	Ch. 12
ceph-bluestore-tool bluefs-log-dump
`ceph-bluestore-tool bluefs-log-dump --path /var/lib/ceph/osd/ceph-4`	Ch. 24
ceph-bluestore-tool bluefs-export
`ceph-bluestore-tool bluefs-export --path ... --out-dir /tmp/bluefs`	Ch. 24
ceph-bluestore-tool free-score
`ceph-bluestore-tool free-score --path /var/lib/ceph/osd/ceph-4`	Ch. 24
ceph-bluestore-tool show-sharding
`ceph-bluestore-tool show-sharding --path /var/lib/ceph/osd/ceph-4`	Ch. 25

ceph-objectstore-tool · offline PG and object inspection#

ceph-objectstore-tool --data-path
`ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-4 --op list --pgid 3.1a \| head`	Ch. 1
`ceph-objectstore-tool --data-path ... --pgid 3.1a '<obj-json>' dump \| jq '.onode'`	Ch. 1
`ceph-objectstore-tool --data-path ... --op log --pgid 3.1a \| jq '.pg_log_t.log[-1]'`	Ch. 1
`ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-12 --op list-pgs`	Ch. 10
`ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-12 --pgid 5.3a --op info`	Ch. 10
`ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-12 --pgid 2.5 --op info`	Ch. 18
`ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-12 --pgid 2.5 --op export --file /backup/2.5.export`	Ch. 18
`ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-30 --op import --file /backup/2.5.export`	Ch. 18

ceph-kvstore-tool · RocksDB inspection#

ceph-kvstore-tool bluestore-kv
`ceph-kvstore-tool bluestore-kv /var/lib/ceph/osd/ceph-4 stats`	Ch. 25

crushtool · compile and test CRUSH maps#

crushtool -i
`crushtool -i crush.new --test --rule 1 --num-rep 3 --show-statistics --show-bad-mappings`	Ch. 8
`crushtool -i cm.bin --test --rule 0 --num-rep 3 --min-x 0 --max-x 1023 --show-utilization`	Ch. 28
`crushtool -i cm.bin --test --rule 0 --num-rep 3 --x 0x36d15c32 --show-mappings`	Ch. 28
`crushtool -i cm.bin --test --show-bad-mappings --rule 0 --num-rep 3`	Ch. 28
crushtool -d
`crushtool -d crush.bin -o crush.txt`	Ch. 8
crushtool -c
`crushtool -c crush.txt -o crush.new`	Ch. 8

ceph-dencoder · decode binary structures#

ceph-dencoder list_types
`ceph-dencoder list_types \| grep -i -E 'osdmap\|pg_log'`	Ch. 22
ceph-dencoder type
`ceph-dencoder type object_info_t import /tmp/oi.bin decode dump_json`	Ch. 34

kubectl · Kubernetes (Rook)#

kubectl -n
`kubectl -n rook-ceph get pods`	Ch. 11
`kubectl -n rook-ceph get cephcluster`	Ch. 11
`kubectl -n rook-ceph exec deploy/rook-ceph-tools -- ceph -s`	Ch. 11
kubectl apply
`kubectl apply -f cluster.yaml`	Ch. 11