Overview¶

• Status: ✅ Accepted
• Audience: Contributors to flexconf and developers integrating the SDK.

This spec describes the overall architecture of flexconf, the core concepts, and the vocabulary used across all other specs. It is intentionally high-level; details belong in the topic-specific specs linked below.

1. Goal¶

flexconf lets a Go application:

1. Declare the shape of its configuration once, as Go types.
2. Load that configuration from one or more config directories with sensible layering.
3. Resolve templating tokens in config values transparently — pulling from the environment or from secret backends — without the application code needing to know which values are secrets.

The distinguishing idea: the config author, not the application developer, decides what is a secret. A developer says "I need a field Token string". An operator writing the config file writes token: $(secret:artifactory/token) — or token: hardcoded-for-dev in a throwaway environment. The application is unchanged either way.

2. Core concepts¶

Config schema¶

The application-provided description of its configuration, expressed as Go types (structs with tags). flexconf binds loaded values into an instance of these types. The flexconf:"..." struct tag mirrors the semantics of the standard yaml:"..." tag (field name, then comma-separated options); it exists as a format-agnostic abstraction so the same struct can back other config formats later without re-tagging. Details: schema-and-binding.md.

type Config struct {
    Service  string `flexconf:"service"`
    Timeout  time.Duration `flexconf:"timeout,default=30s"`
    Artifactory struct {
        URL   string `flexconf:"url"`
        Token string `flexconf:"token"` // may resolve from a secret, but the type doesn't care
    } `flexconf:"artifactory"`
}

Config directory / layer¶

Configuration is loaded from one or more config directories. A Loader is configured with an ordered list of them; each directory is a layer. When more than one directory provides the same file, the layers are merged into a single tree (a later directory overrides an earlier one, per-key). A single directory is the common case; multiple directories enable base/override layering (e.g. a shipped default dir beneath a site-local one). Details: config-loading.md.

Config file¶

Within the directories, an individual configuration is selected by file name — config.yaml, agents.yaml, … — and bound into a Go struct. One Loader can load several kinds of config for the same app by name (see Loader). Environment variables are not a config layer; they feed $(env:...) tokens (see Resolver), not the directory merge. Details: config-loading.md.

Templating token¶

A placeholder embedded in a config value with the grammar $(scheme:path), e.g. $(env:HOME), $(secret:artifactory/token). Tokens are resolved after layers are merged and before binding. Details: templating.md.

Resolver¶

The component that resolves a token for a given scheme. env and secret are built in; custom schemes can be registered. A resolver maps a path to a concrete value. Details: resolvers.md.

Vault driver¶

The pluggable backend behind the secret: scheme. A driver knows how to fetch a secret by path from a specific system (HashiCorp Vault, AWS/GCP/Azure secret managers, an encrypted file, …). The active driver is selected at runtime via a driver-registry pattern, so swapping backends requires no code change in the loading path. Details: vault-drivers.md.

Loader¶

The top-level orchestrator, configured with the ordered config directories. Its Load(name, &dst) method ties everything together for one named file: read that file from each layer → merge by precedence → resolve tokens (via resolvers, which may call vault drivers) → bind into dst → validate. The same Loader loads several config kinds by calling Load with different names and structs (Load("config.yaml", &cfg), Load("agents.yaml", &agents)). Details: config-loading.md and api.md.

3. How the pieces fit together¶

   Load("config.yaml", &cfg)
                        ┌─────────────────────────────────────────────────┐
   config dirs   ─────▶ │                  Loader                          │
   (ordered layers)     │                                                  │
                        │  1. read <name> from each layer (directory)      │
                        │  2. merge by precedence  →  raw value tree       │
                        │  3. resolve $(scheme:path) tokens ───────────────┼──▶ Resolvers
                        │       • env:    → environment                    │        │
                        │       • secret: → Vault Driver ──────────────────┼──▶ Vault / AWS / file / …
                        │       • custom: → registered resolver            │        │
                        │  4. bind resolved tree → &cfg (the struct)       │◀───────┘
                        │  5. validate (required, types, custom rules)     │
                        └────────────────────────┬────────────────────────┘
                                                 ▼
                                     typed, validated config value

Resolution order matters: tokens are resolved on the merged tree, so a value introduced by a higher-precedence layer can itself contain a token. Nesting and escaping rules are defined in templating.md.

4. Token grammar summary¶

Normative grammar lives in templating.md; summarized here for orientation.

• A token has the form $(scheme:path).
• scheme selects a resolver; path is opaque to flexconf and interpreted by that resolver.
• Built-in schemes:
• env:NAME — value of environment variable NAME. A missing variable is a hard error; there is no :-default syntax (templating.md §10).
• secret:[vault:]namespace/key — value fetched for that two-level secret address (see vault-drivers.md §6). An optional leading vault: segment selects a named vault from the registry; omitting it uses the default vault (see vault-registry.md §4–§5).
• file:path — verbatim contents of a file (relative to the containing config file); non-secret (see resolvers.md §4).
• config:path.yaml — structural include: splices another YAML file's tree in as the whole value (no deep-merge); a different composition axis from directory layering (templating.md §7).
• Scalar tokens (env/secret/file) may appear anywhere inside a string value and be combined with literal text: url: https://$(env:HOST):$(env:PORT)/api. config: must be a whole value. Templating operates on the parsed node tree (values only, never keys; resolved values are inert), and a whole-value token is left untagged so its type is inferred from the resolved content (templating.md §2, §6).

5. Vault driver selection summary¶

Normative rules live in vault-drivers.md.

• Drivers implement a common interface and register under a name.
• Vaults are named in a registry (map[name]VaultConf) and referenced by name; each name maps to a driver + its non-secret config. Definitions are layered across files and code, with a designated default vault. Details: vault-registry.md.
• The secret: resolver delegates to whichever vault the token selects (the named one, else the default). Config authors never name the backend in tokens — only an optional logical vault name and the secret address ([vault:]namespace/key).

6. Module layout¶

The module (go.mod root github.com/sylvanld/go-flexconf) is organized around three core packages plus an optional CLI package. The split follows the dependency direction — flexprompt at the bottom — so there are no import cycles.

Rules:

• Dependencies point downward only: cmd/flexconf → flexcli → internal/agent → flexvault → flexprompt, and flexconf → internal/agent (for the secret: resolver). flexprompt MUST NOT import the others; flexvault MUST NOT import flexconf, flexcli, or internal/agent; internal/agent MUST NOT import flexconf or flexcli; nothing imports flexcli. The agent runtime is a module-internal package so neither flexconf nor flexcli needs to import the other to share it (resolvers.md §5.5).
• An application using only secret management (no config loading) can depend on flexvault (+ flexprompt) alone, without pulling in the config loader or Cobra.
• Concrete drivers live in their own sub-packages (flexvault/driver/keepass) and register with flexvault from init(), so importing a driver for its side effect (_ "…/flexvault/driver/keepass") is all that's needed to make it selectable by name.

Details of the flexvault types are in vault-drivers.md; the flexprompt Prompter in prompter.md; flexcli in cli.md.

7. Glossary¶

8. Resolved design decisions¶

These were tracked here while open; they are now settled for v1 and recorded in the topic specs. Kept here as a pointer so the whole picture is visible.

• Resolution is eager, at load, only. Every token (including secret:) is resolved during Load; there is no lazy/on-access or rotating-secret mode. This is a firm v1 decision, not a placeholder (resolvers.md §5, §9; config-loading.md §5).
• No caching or TTL for vault drivers in v1. Each resolution asks the vault (through the running agent, or the in-process Manager); there is no freshness/TTL/invalidation layer. Explicitly out of scope for v1 (vault-drivers.md §11, resolvers.md §9).
• Behavior on partial failure — some tokens unresolved — Load fails loud and early (config-loading.md §3, §5).