Specialist Part (8): RGW — The Request Path, SigV4, Quotas, Rate Limits, GC and Lifecycle

pseudocode
process_request(req):
    s = req_state ; rest handler by dialect (S3, Swift, admin, STS ...) from URL and headers
    op = handler.get_op()                       # e.g. RGWPutObj_ObjStore_S3
    handler.authorize()                         # SigV4/V2, STS, anonymous ...
    normalize buckets and tenants
    rgw_process_authenticated(handler, op):
        init permissions                        # bucket and user policy, ACL (cached)
        reading permissions ; init op ; verifying op mask
        verifying CORS origin matches ; verifying op permissions (IAM + ACL)
        verifying op params ; pre-executing
        check rate limiting → 503 SlowDown + Retry-After when delayed
        executing: op->execute()                # RADOS I/O, quota check inside
        completing: op->complete() → response, ops log, usage log, bucket notification

pseudocode
parse Authorization: AWS4-HMAC-SHA256 Credential=AK/YYYYMMDD/region/s3/aws4_request,
                     SignedHeaders=h1;h2;..., Signature=hex
reject if |now - x-amz-date| > RGW_AUTH_GRACE (15 min)      # RequestTimeTooSkewed
canonical = METHOD \n URI \n sorted query \n "name:value\n" for each signed header \n
            SignedHeaders \n payload hash (or UNSIGNED-PAYLOAD / STREAMING-...)
sts = "AWS4-HMAC-SHA256" \n x-amz-date \n "YYYYMMDD/region/s3/aws4_request" \n hex(sha256(canonical))
kDate = HMAC("AWS4" + secret, YYYYMMDD) ; kRegion = HMAC(kDate, region)
kService = HMAC(kRegion, "s3") ; kSigning = HMAC(kService, "aws4_request")
ok if hex(HMAC(kSigning, sts)) == Signature (constant-time compare)
streaming upload: every chunk carries its own signature chained from the previous one

example:
sha256(canonical request) = 36f24a8179a5bc758bde32b4a6932f313216b6c9f7d84e04e33bd7a2086b87f5
kSigning                  = 38793786cc844d69350de848f15fe23b8f7887e3bcbb48d552b2e0a2c1d9746d
Signature                 = 5e63a875b9d9e0ec540db8784929f6d4c5799c453bd5442804a9a7bc7f57aff8

pseudocode
PUT:
    check quota (estimated size = Content-Length)            # before any data write
    processor = Atomic (or Multipart part / Append)
    loop body chunks: filter chain (compression, encryption, etag md5) → StripeProcessor → ChunkProcessor
        aio write chunk ; keep up to rgw_put_obj_min_window_size (16 MiB) in flight
    complete: wait aio ; index prepare → write head attrs + manifest → index complete (alg-40-1)
    old object's tail (if overwritten) → gc chain with tail_tag
GET:
    read head (attrs + up to first chunk of data) ; check conditions (If-Match, Range ...)
    walk manifest from range start: each RADOS read ≤ rgw_get_obj_max_req_size (4 MiB)
    keep up to rgw_get_obj_window_size (16 MiB) outstanding ; stream to client in order
    filters in reverse (decrypt, decompress)

pseudocode
check_quota(bucket, owner, num_objs = 1, size):
    for scope in (bucket quota, user quota) if enabled:
        stats = quota cache (refreshed after rgw_bucket_quota_ttl = 10 min, or on async update)
        if max_objects >= 0 and stats.num_objects + num_objs > max_objects → QuotaExceeded
        if max_size >= 0:
            if check_on_raw: exceeded = stats.size + size > max_size
            else:            exceeded = stats.size_rounded + round_up(size, 4096) > max_size
        → 403 QuotaExceeded

pseudocode
tokens kept ×1000 (fixed point) ; interval = rgw_ratelimit_interval (60 s)
on request, levels checked in order: account, user, bucket (each with its own bucket of tokens):
    elapsed = now - ts ; refill = limit × elapsed / interval  (capped at limit)
    op type ∈ {read, write, list, delete}            # list/delete classes: main branch
    if ops tokens < 1 or bytes tokens < 0:
        delay = ceil(needed × interval / limit) seconds → 503 SlowDown, Retry-After: delay
    else take 1 op token
after response: subtract transferred bytes from read or write bytes bucket (may go negative)
if a lower level blocks, op tokens already taken at higher levels are given back

# example: setting a limit for a user
radosgw-admin ratelimit set --ratelimit-scope=user --uid=alice --max-read-ops=600 --max-write-bytes=104857600
radosgw-admin ratelimit enable --ratelimit-scope=user --uid=alice
radosgw-admin ratelimit get --ratelimit-scope=user --uid=alice

example (output):
{ "user_ratelimit": { "max_read_ops": 600, "max_write_ops": 0, "max_list_ops": 0, "max_delete_ops": 0,
                      "max_read_bytes": 0, "max_write_bytes": 104857600, "enabled": true } }

pseudocode
enqueue(chain, tag):
    shard = XXH64(tag, seed 8675309) mod rgw_gc_max_objs (32) → object gc.<shard> in <zone>.rgw.log:gc
    cls rgw_gc.queue_enqueue(entry{tag, chain, time = now + rgw_gc_obj_min_wait (2 h)})
    (split into several entries if chain too large for one queue entry)
gc thread every rgw_gc_processor_period (1 h), for up to rgw_gc_processor_max_time (1 h):
    for each shard in random start order:
        try lock "gc_process" on gc.<shard> ; skip if held by another RGW
        list expired entries ; delete each tail object (≤ rgw_gc_max_concurrent_io = 10 in flight)
        remove processed entries from queue (trim in chunks of rgw_gc_max_trim_chunk)

pseudocode
PutBucketLifecycle: store rules in bucket attr ; add entry "<tenant>:<bucket>:<marker>" to shard
    lc.<ceph_str_hash_linux(key) mod 7877 mod rgw_lc_max_objs (32)> in <zone>.rgw.log:lc
rgw_lc_max_worker (3) worker threads, active only inside rgw_lifecycle_work_time (00:00-06:00):
    pick shard ; lock "lc_process" (rgw_lc_lock_max_time 90 s, renewed) ; take next entry not done today
    bucket_lc_process(entry):
        skip if bucket marker != entry marker (bucket was recreated)
        group rules by prefix ; list objects (and versions) per prefix
        for each object, run matching actions with up to rgw_lc_max_wp_worker (128) parallel ops:
            CurrentExpiration, NonCurrentExpiration, DMExpiration (expired delete markers),
            CurrentTransition, NonCurrentTransition (copy tail to class pool, new manifest)
        abort incomplete multipart uploads older than N days
    mark entry complete with start time

example (lc list output):
[ { "bucket": ":logs:a1b2c3d4-...4123.9", "shard": "lc.17", "started": "Sun, 15 Nov 2026 00:00:04 GMT", "status": "COMPLETE" },
  { "bucket": ":photos:a1b2c3d4-...4123.7", "shard": "lc.4",  "started": "Sun, 15 Nov 2026 00:00:06 GMT", "status": "PROCESSING" } ]

# example: a simple lifecycle with a transition and an expiry
cat > lc.json <<'J'
{ "Rules": [ { "ID": "logs", "Status": "Enabled", "Filter": { "Prefix": "logs/" },
  "Transitions": [ { "Days": 30, "StorageClass": "COLD" } ],
  "Expiration": { "Days": 365 },
  "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 } } ] }
J
aws --endpoint-url https://s3.example.test s3api put-bucket-lifecycle-configuration --bucket logs --lifecycle-configuration file://lc.json