37 - A Configuration Distribution Manager
A backend daemon to manage and distribute configurations
Status
Public Draft - brainstorming phase
About
An open design and implementation of a distributed database for rolling out configuration changes to nodes.
Requirements
- Uses AMF
- strong consistency
- assume global deployment
- low data volume (~MB-GB)
- durable storage
- encryption between nodes and at rest
- dynamic node membership
- nodes only distribute data, clients issue writes/final state
- keep n historical revisions
- activation properties: date, percentage of success
- rollback
- partition data via namespaces, client opt into subscribing
Other ideas to explore
- client defines (maybe as a DSL) how rollout proceeds - e.g. not all configuration revisions need the same rollout process, e.g. this change needs immediate effect, or that change needs slow propagation, etc
Consensus
- nodes form clusters (same data center/backend links)
- clusters form groupings
- groupings form deployment
Distribute Data Within Clusters
- Need a way to only distribute data to cluster leaders, leaders distribute to peers (e.g. minimize egress between clusters)
- A Hierarchical Model for Fast Distributed Consensus in Dynamic Networks