AMF - Application Making Framework

Status

Public Draft - R&D phase

About

An R&D open source project to explore writing a framework to aid in developing daemons, applications, and REPL s in the Rust programming language that can run in various execution environments (Linux, Linux-like, Docker, Kubernetes, etc.).

Background

Professionally, I’ve written quite a few daemons that run in various forms (web servers, system managing, etc.) and often repeat myself.

At the day job, I was often given asks that sound conceptually easy, but are often wrought with edge cases and complexity an architect gets to gloss over (e.g., Unix signal handling). As a result I spend far too much trying to take care of the ask instead of focusing on the primary goal (e.g., write a daemon that manages firewalls).

As a user of a framework, I don’t want to sacrifice my desire to bring my own libraries/techniques instead of being forced by the framework (e.g., logging). My current employer has environments that require special handling. Being able to move a daemon from one environment to another with only a re-compilation has a lot of value (e.g., going from your local desktop to running on one of the cloud platforms).

I attempted an implementation in Haskell in December 2020 and have a partial working version on my GitHub. Given my lost bet on Haskell begetting a revolution for how programs are written, I’ve re-focused on using Rust. As of 2022 I am not able to find a language with a thriving, growing, and nurturing ecosystem that surpasses Rust.

Goal

Design a system that:

1. supports all the requirements
2. alleviates the pain of dealing with the mundane (e.g., configuration loading/updating safely)
3. to let the user focus on writing code

Terminology

Requirements

• Supports overriding (or disabling) sections; tracked via the type system:
  • command line options
  • environment variables
  • configurations (format, handling updates, etc.)
  • logging
  • telemetry

Unix_domain_socket control interface using Session_type ’s

• Supports app types * Daemon_(computing)

  • App - an application that is not meant to run “forever”, e.g. ls,

REPL

• Supports running unchanged code (re-compilation is allowed) under different environments
  • Linux and Linux-like environments (my current employer uses segregated setup on top of a Debian-based environment)
  • Docker, and Docker-like environments (my current employer has a Docker-like application running network which forces an app to handle telemetry, logging, and configuration management in an explicit way)
  • Kubernetes
  • Not a requirement to support any of the cloud providers (as I have very little experience)
• Support an async runtime (targeting tokio first)
• Handle/report resource limits, e.g., a programmatic ulimit
• Periodically/synchronously capture system information and make that information available to the user (e.g., no more parsing ifconfig output)

Non-Requirements

• Performance. While Rust can perform very well, the goal of this project is not to strive for maximum performance (in both CPU and memory). Two and a half out of three ain’t bad!

Design

An implementer(/user) will:

• decide the types used for each section or use one of the provided disable types
• supply 3 functions to cover setup, the body, and the teardown (akin to the bracket pattern)
• use one of the environment executor libraries to run their computation

Events form the spine of the framework. All notable things that happen get serialized into a channel which n readers and only a single writer. If the user chooses logging output, events become the log entries. Events can be low level: UNIX signals, socket operations, etc. or high level: new user added, timer expired, remote host reported down, etc. Event handling is asynchronous. The framework augments each event with the who, when, and where. The data portion of the event should contain the what and/or why.

Metrics

Configuration

Control

References

• Linux Daemon Writing HOWTO
• Session Types for Rust