A deployment complains that port in production is 9090, while defaults.yaml says 8080. Was it override.yaml, an environment variable, or a Kubernetes ConfigMap? Without the right tooling, finding out can eat half a working day.

I've been writing Python since 2017, and I've worked with configuration in every form: INI files in legacy projects, environment variables in Docker containers, YAML with templates in Helm, pydantic models in fresh services, Vault and Consul in bigger infrastructure.

In the previous two articles I traced how we got to the way we handle configuration today and why no existing Python library covers all of it, which left me and my team unsatisfied. This one is about what I built in response.

TL;DR

  • dature loads config from many sources into your own stdlib @dataclass, with no base class and no framework.
  • Every value remembers where it came from, so errors name the file and line, and a debug report shows each field's origin.
  • Merging is configurable per field, and sources can be conditional (when=) or depend on each other.
  • It's not a replacement for everything: if you need tight FastAPI dependency injection, pydantic-settings is a more direct path.

From pains to a spec

Six pains became the specification:

  • merging individual fields from different sources;
  • no way to tell where an invalid value came from;
  • validators wanted both inside the type and outside it;
  • secret masking doesn't work everywhere;
  • adding your own source should be easy;
  • it must be a library, not a framework.

My library, dature, grew out of these. If tight integration with FastAPI dependency injection is critical for you, pydantic-settings gives a more direct path. dature has its own niche; it doesn't displace everything else, it closes specific pains that my team and I ran into.

dature overview

dature's philosophy

dature is a library. Its only job is to load data from arbitrary sources into a standard-library @dataclass and return a ready object. It doesn't capture your entry point, doesn't change the working directory, doesn't manage logging.

The dataclass is the single source of truth. Type hints describe the structure, the types, and the constraints. There's no need to duplicate definitions in YAML files, separate validators, or Pydantic models.

from dataclasses import dataclass import dature @dataclass class Config: host: str port: int debug: bool = False config = dature.load(dature.Yaml12Source(file="config.yaml"), schema=Config) # isinstance(config, Config) → True # Full IDE and mypy support, __post_init__ works

Loading from any source while keeping provenance

The main idea of dature is not to lose a value's origin after parsing. For every field it's known which source and which line it came from, and that is available both in validation errors and in the debug report.

The list of supported sources is, as you'd expect, broad:

  • eight file formats with auto-detection by extension: YAML 1.1/1.2, JSON, JSON5, TOML 1.0/1.1, INI, .env;
  • environment variables;
  • Docker secrets;
  • Vault, Consul, etcd, ZooKeeper, AWS SSM, AWS Secrets Manager, Azure App Configuration, Azure Key Vault, GCP Secret Manager;
  • argparse CLI;
  • a custom CLI source.

One call, several sources:

from dataclasses import dataclass import dature @dataclass class Config: host: str port: int secret_key: str debug: bool = False config = dature.load( dature.Yaml12Source(file="defaults.yaml"), dature.Toml11Source(file="config.toml", skip_if_missing=True), dature.EnvFileSource(file=".env", skip_if_broken=True), dature.EnvSource(prefix="APP_"), # environment variables with the APP_ prefix  schema=Config, )

skip_if_broken=True means that if a file is invalid, the source is silently skipped. skip_if_missing=True means that if a file doesn't exist, the source is silently skipped. This lets you describe the chain "defaults → local overrides → environment variables" without checking for file existence by hand.

The format is detected automatically from the extension. If you need a non-standard loader, you write your own Source.

Merge strategies

Four global strategies decide how to resolve conflicts between sources:

  • last_wins: the last source overrides the earlier ones (the default);
  • first_wins: the first source takes priority;
  • first_found: take the first source that isn't broken and ignore the rest;
  • raise_on_conflict: an error if two sources set different values.

You can set separate rules for individual fields. For example, a list of servers from different sources should be combined rather than replaced:

from dataclasses import dataclass import dature @dataclass class Config: servers: list[str] port: int config = dature.load( dature.Yaml12Source(file="defaults.yaml"), dature.Yaml12Source(file="local.yaml", skip_if_broken=True), dature.EnvSource(prefix="APP_"), field_merges={dature.F[Config].servers: "append_unique"}, schema=Config, )

Field groups let you enforce consistency: if one field of a group is overridden in a new source, the rest must be overridden too:

import dature dature.load( dature.Yaml12Source(file="defaults.yaml"), dature.Yaml12Source(file="override.yaml"), field_groups=((dature.F[Config].db_host, dature.F[Config].db_port, dature.F[Config].db_name),), schema=Config, ) # If override.yaml changes db_host but not db_port → error

Validation

dature uses Annotated for built-in validators:

from dataclasses import dataclass from typing import Annotated import dature @dataclass class Config: port: Annotated[int, (dature.V >= 1) & (dature.V <= 65535)] host: Annotated[str, dature.V.len() >= 1] log_level: Annotated[str, dature.V.matches(r"^(DEBUG|INFO|WARNING|ERROR)$")]

The validator lives right in the field's type, and that matters. When you add a new field to the dataclass, you physically can't forget its constraints: they're part of the signature. Validators can also be supplied separately, through a Source, when you want to reuse one schema with different rules or move the checks into another module:

import dature config = dature.load( dature.Yaml12Source( file="config.yaml", validators={ dature.F[Config].port: (dature.V >= 1) & (dature.V <= 65535), }, ), schema=Config, )

For checks across fields there are root validators:

import dature def check_ssl(config: Config) -> bool: return not config.use_ssl or config.cert_path is not None config = dature.load( dature.Yaml12Source( file="config.yaml", root_validators=(dature.V.root(func=check_ssl, error_message="cert_path required when use_ssl is True"),), ), schema=Config, )

The standard __post_init__ works too, because dature returns an ordinary dataclass. When validation fails, the error points at the exact place:

Config loading errors (1) [port] Must be greater than 0 ├── port: -1 │ ^^ └── FILE 'config.yaml', line 2

The message has everything you need to fix it in a minute: the file path, the line number, the value itself, and carets under the offending piece. That's the difference that's worth an hour of debugging in production logs.

Secrets and debugging

Fields of type SecretStr are masked automatically in errors and on output:

from dataclasses import dataclass import dature from dature.fields import SecretStr @dataclass class Config: database_url: str api_key: SecretStr config = dature.load(dature.Yaml12Source(file="config.yaml"), schema=Config) print(config.api_key) # SecretStr('**********') print(config.api_key.get_secret_value()) # the real value

Masking works on three levels:

  • By type: SecretStr, PaymentCardNumber.
  • By field name: by default, fields containing the substrings password, passwd, secret, token, key, auth, credential, uri, url, connection_string are masked. The list is configurable via MaskingConfig.secret_field_names.
  • By heuristic: bigram analysis. The string is lowercased and split into bigrams, and the share of rare bigrams is compared against a threshold (heuristic_threshold, 0.5 by default). If more than half are rare, the string is considered a randomly generated token and is masked. The minimum string length for analysis is 8 characters (min_heuristic_length). The heuristic requires the optional random_string_detector dependency; without it, only the first two levels work.

Debug mode shows where each value came from:

import dature config = dature.load( dature.Yaml12Source(file="defaults.yaml"), dature.EnvSource(prefix="APP_"), schema=Config, debug=True, masking_mode="secrets_only", # so ordinary values appear in the report ) report = dature.load_report(config) for origin in report.field_origins: print(f"{origin.key} = {origin.value} ← {origin.source_file or origin.source_loader_type}") # host = 'localhost' ← defaults.yaml # port = 9090 ← env # debug = False ← defaults.yaml

Debug mode is especially useful when a config is assembled from several sources and it's unclear which one dominates a given field: debug=True shows each value's path from source to final object.

Custom fields and sources

dature works with your own types and formats. Registering a loader for a new type:

from dataclasses import dataclass import dature @dataclass(frozen=True, slots=True) class Rgb: r: int g: int b: int def rgb_from_string(value: str) -> Rgb: parts = value.split(",") return Rgb(r=int(parts[0]), g=int(parts[1]), b=int(parts[2])) @dataclass class AppConfig: name: str color: Rgb config = dature.load( dature.Yaml12Source( file=SOURCES_DIR / "custom_type_common.yaml", type_loaders={Rgb: rgb_from_string}, # applies only to this file source  ), schema=AppConfig, )

A custom format is a subclass of FileSource:

import xml.etree.ElementTree as ET from dataclasses import dataclass from pathlib import Path import dature from adaptix import Provider, loader from dature.loaders import bool_loader, float_from_string from dature.sources.base import FileSource from dature.types import FileOrStream, JSONValue @dataclass(kw_only=True, repr=False) class XmlSource(FileSource): format_name = "xml" def _load_file(self, path: FileOrStream) -> JSONValue: if not isinstance(path, Path): msg = "XmlSource only supports file paths" raise TypeError(msg) tree = ET.parse(path) # noqa: S314  root = tree.getroot() return {child.tag: child.text or "" for child in root} def format_loaders(self) -> list[Provider]: return [ loader(bool, bool_loader), loader(float, float_from_string), ] @dataclass class Config: host: str port: int debug: bool config = dature.load( XmlSource(file="custom_loader.xml"), schema=Config, )

Real-world scenarios where dature helps

So the features don't hang in the air, here are several typical scenarios. In these situations you normally end up writing a lot of boilerplate, while with dature we get by in a few lines.

Scenario: merge lists, replace everything else. An application config: common defaults for CORS origins, plus extra ones for each environment. The behavior you want: lists are combined, while database.host is replaced by source priority.

Before, you had to write a custom SettingsSource in pydantic-settings, override _field_is_complex, and check that deep-merge is enabled for the specific field. That came to twenty lines of infrastructure code for a single business scenario.

With dature it's one field_merges line:

config = dature.load( dature.Yaml12Source(file="defaults.yaml"), dature.Yaml12Source(file=f"config.{env}.yaml", skip_if_broken=True), dature.EnvSource(prefix="APP_"), field_merges={ dature.F[Config].cors_origins: "append_unique", }, schema=Config, )

Everything else is last_wins by default, and no custom source class is needed.

Scenario: the config itself answers where each field came from. The classic production detective story. A deployment complains that port in production is 9090 while defaults.yaml says 8080. You need to quickly find the override's source: override.yaml, environment variables, or a Kubernetes ConfigMap. So the config should not only hold the final value but also show its origin.

Without a suitable tool, the detective work starts with a trip into the container: env | grep APP_, check mounted files, reread docker-compose. Understanding the origin of a single value can sometimes eat half a working day.

With dature:

config = dature.load( dature.Yaml12Source(file="defaults.yaml"), dature.Yaml12Source(file="override.yaml", skip_if_broken=True), dature.EnvSource(prefix="APP_"), schema=Config, debug=True, ) report = dature.load_report(config) for origin in report.field_origins: print(f"{origin.key} = {origin.value} ← {origin.source_file or origin.source_loader_type}")

The output immediately shows port = 9090 ← env. The value came from an environment variable, and the case is closed in a minute rather than half a day of digging through files in a container. It's especially useful in CI: you can run load(..., debug=True) as part of a test stage and verify where each field is supposed to come from in production.

Scenario: a custom source for an internal configuration service. A large company has its own configuration service: a REST endpoint that returns settings for the requesting module. Nobody is going to replace it with "ordinary Vault," and without dature the options are to load it manually and glue it to the other sources by hand, or to write a custom SettingsSource in pydantic-settings with all that entails.

With dature, a custom source is a subclass of Source with one _load method:

from dataclasses import dataclass import requests import dature from dature.sources.base import Source from dature.types import JSONValue @dataclass(kw_only=True, repr=False) class InternalConfigApiSource(Source): service: str def _load(self) -> JSONValue: response = requests.get( f"https://config.internal/api/v1/{self.service}", timeout=5, ) response.raise_for_status() return response.json() config = dature.load( dature.Yaml12Source(file="defaults.yaml"), InternalConfigApiSource(service="payments"), dature.EnvSource(prefix="APP_"), schema=Config, )

No custom error pipeline, no merge strategies, no secret masking of your own. The custom source slots into the common chain and gets everything else automatically.

Scenario: consistent changes to related fields. A team is moving to a staging environment, and all the DB parameters change at once: db_host, db_port, db_name, db_user. You want a guarantee: either staging.yaml overrides all four, or loading fails with a clear error. No half-migrations.

field_groups does exactly that:

config = dature.load( dature.Yaml12Source(file="defaults.yaml"), dature.Yaml12Source(file="staging.yaml", skip_if_broken=True), field_groups=( ( dature.F[Config].db_host, dature.F[Config].db_port, dature.F[Config].db_name, dature.F[Config].db_user, ), ), schema=Config, ) # If staging.yaml changes db_host but not db_port → an error naming # the group and which field stayed from defaults.yaml

The error is caught at load time, not at the moment of the first connection to someone else's database.

Scenario: dynamic configuration without a restart. A long-lived service has request limits in YAML, and you want to change them on the fly without a restart. Re-reading on every request is expensive; re-reading by hand is unreliable.

cache=timedelta(...) solves it in one line: every class that goes through a shared Loader is invalidated at the same moment, and the next call gets fresh values:

from datetime import timedelta import dature loader = dature.Loader( dature.Yaml12Source(file="config.yaml"), schema=Config, cache=timedelta(seconds=30), ) # Anywhere in the code  config = loader.load() # within one bucket, a single real re-parse

Between "re-read on every call" and "once at startup" there is now a middle ground, and its cost is one parameter.

Scenario: the config path comes from outside. A container doesn't know where its config lives: the path is mounted at release time and passed through an environment variable. Sometimes a Vault token or the active profile name comes along too. You get a two-stage load: first read the ENV, then build the arguments for the main sources from those values.

This is usually written by hand: os.environ.get, a check for absence, path concatenation, and only then the loader call. With more than two sources, imperative glue appears between them.

With dature, sources refer to each other via ${@tag.key} right in their arguments:

config = Source：https://dev.to/niccolum/dature-type-safe-configuration-through-plain-dataclasses-24bi
My favorites
PreviousOutlook.com Is Rejecting Mail From Domains With No DMARC Record: What Actually ChangedNextHow long do you get to bid on a German public tender? I checked 187 notices

Related posts

When cleanup jobs pile up: draining a dead letter queue after a night of failed retries

When cleanup jobs pile up: draining a dead letter queue after a night of failed retries

dev.to
Part Size, Not Protocol: Direct Browser Uploads of Large Training Files to Object Storage

Part Size, Not Protocol: Direct Browser Uploads of Large Training Files to Object Storage

dev.to
Support Attachments: Multipart Selection for 100 MB-1 GB Private PDF and ZIP Uploads

Support Attachments: Multipart Selection for 100 MB-1 GB Private PDF and ZIP Uploads

dev.to
I created an interactive TUI log debugger in Python because grep and tail are driving me crazy.

I created an interactive TUI log debugger in Python because grep and tail are driving me crazy.

dev.to
A read-only MCP server for TeamCity

A read-only MCP server for TeamCity

dev.to
Getting the V100 tensor cores back from Triton

Getting the V100 tensor cores back from Triton

dev.to