The Comprehensive Ceph Guide
فارسیGet started
Interactive lab

Replication or erasure coding

In the book: Chapter 13

Lab

Replication or erasure coding

Choose k, m and the plugin and compare usable capacity, overhead, the failures each layout survives and the cost of repairing one chunk.

768 TBraw capacity
8failure domains

EC isa k=4 m=2

usable
512 TB (66.7%)
up to nearfull (85%)
435.2 TB
raw per usable byte
1.5×
OSD/host losses without losing data
2
size / min_size
6 / 5
losses with I/O still served
1
read to rebuild one chunk
4 chunks

replicated, size 3

usable
256 TB (33.3%)
up to nearfull (85%)
217.6 TB
raw per usable byte
3×
OSD/host losses without losing data
2
size / min_size
3 / 2
losses with I/O still served
1
read to rebuild one chunk
1 chunks
usable = raw × k / (k + m) · EC min_size = k + min(1, m − 1) = 5 · replicated min_size = size − ⌊size / 2⌋ = 2 · repair read = k
ceph osd erasure-code-profile set prof-4-2 plugin=isa k=4 m=2 crush-failure-domain=host
ceph osd pool create ecpool erasure prof-4-2
ceph osd pool set ecpool allow_ec_overwrites true   # RBD, CephFS

The default profile on main is plugin=isa technique=reed_sol_van k=2 m=2; profiles cannot be changed once a pool uses them.

Follows:

  • src/mon/OSDMonitor.cc :: prepare_pool_size size and min_size of a new pool
  • src/common/options/global.yaml.in :: osd_pool_default_erasure_code_profile, mon_osd_nearfull_ratio default profile and fullness ratios
  • src/erasure-code/lrc/ErasureCodeLrc.cc :: parse_kml the k, m, l rules of LRC
  • src/erasure-code/clay/ErasureCodeClay.cc :: parse the d range of CLAY
  • src/erasure-code/shec/ErasureCodeShec.cc :: parse the c rule of SHEC
  • src/osd/OSDMap.cc :: pool_raw_used_rate raw space per byte (size, or (k+m)/k)
A fixed snapshot of the simulator; the interactive version is in the web edition's lab. Interactive lab: Replication or erasure coding