adamflott.com

AMF - Application Making Framework

Description: A Rust library to aid in daemon and application development using types...or What If main() Had Rules?
Authored: 2022-03-26;
Permalink: https://adamflott.com/programming/rust/amf/
tags : framework; rust; unix;
categories : programming;

Status

Public Draft - R&D phase

About

An R&D open source project to explore writing a framework to aid in developing daemons, applications, and REPL s in the Rust programming language that can run in various execution environments (Linux, Linux-like, Docker, Kubernetes, etc).

Background

Professionally I've written quite a few daemons that run in various forms (web servers, system managing, etc) and often repeat myself.

At the day job, I was often given asks that sound conceptually easy, but are often wrought with edge cases and complexity an architect gets to gloss over (e.g. Unix signal handling). As a result I spend far too much trying to take care of the ask instead of focusing on the primary goal (e.g. write a daemon that manages firewalls).

As a user of a framework, I don't want to sacrifice my desire to bring my own libraries/techniques instead of being forced by the framework (e.g. logging). My current employer has environments that require special handling. Being able to move an daemon from one environment to another with only a re-compilation has a lot of value (e.g. going from your local desktop to running on one of the cloud platforms).

I attempted an implementation in Haskell in December 2020 and have a partial working version on my GitHub. Given my lost bet on Haskell begetting a revolution for how programs are written, I've re-focused on using Rust. As of 2022 I am not able to find a language with a thriving, growing, and nurturing ecosystem that surpasses Rust.

Goal

Design a system that:

supports all the requirements
alleviates the pain of dealing with the mundane (e.g. configuration loading/updating safely)
to let the user focus on writing code

Terminology

Term	Definition
section	command line options, environment variables, configurations, logging, telemetry
control	Interacting with the app via outside controls (either Unix Domain Socket, HTTP, etc)
environment	Execution environment
app type	daemon, app, REPL, script, etc
computation	a user runs their app type in an environment
event	a user defined notable thing that happened
metric	a piece of data for telemetry
configuration	values relevant to the domain of your application that can be safely changed at run time without restarting

Requirements

Supports overriding (or disabling) sections; tracked via the type system:
- command line options
- environment variables
- configurations (format, handling updates, etc)
- logging
- telemetry
Unix_domain_socket control interface using Session_type 's
Supports app types
- Daemon_(computing)
- App - an application that is not meant to run "forever", e.g. ls,
- REPL
Supports running unchanged code (re-compilation is allowed) under different environments
- Linux, and Linux-like environments (my current employer uses segregated setup on top of a Debian-based environment)
- Docker, and Docker-like environments (my current employer has a Docker-like application running network which forces an app to handle telemetry, logging, and configuration management in an explicit way)
- Kubernetes
- Not a requirement to support any of the cloud providers (as I have very little experience)
Support an async runtime (targeting tokio first)
Handle/report resource limits, e.g. a programmatic ulimit
Periodically/synchronously capture system information and make that information available to the user (e.g. no more parsing ifconfig output)

Non-Requirements

Performance. While Rust can perform very well, the goal of this project is not to strive for maximum performance (in both CPU and memory). Two and a half out of three ain't bad!

Design

An implementer(/user) will:

decide the types used for each section or use one of the provided disable types
supply 3 functions to cover setup, the body, and the teardown (akin to the bracket pattern)
use one of the environment executor libraries to run their computation

Events form the spine of the framework. All notable things that happen get serialized into a channel which n readers and only a single writer. If the user chooses logging output, events become the log entries. Events can be low level: UNIX signals, socket operations, etc or high level: new user added, timer expired, remote host reported down, etc. Event handling is asynchronous. The framework augments each event with the who, when, and where. The data portion of the event should contain the what and/or why.

Metrics

Configuration

Control