BigConfig | Blog

The Evolution of DevOps: From Separation by Technology to Separation by Concerns

Thu, 02 Apr 2026 00:00:00 GMT

The current state of DevOps feels eerily similar to the dark ages of Frontend development before the React revolution.

In the early days of the web, we practiced Separation by Technology. We kept our CSS in one folder, our HTML in another, and our JavaScript in a third. We were told this was clean, but in reality, it was a nightmare to maintain. To change a single button, you had to hunt through three different files in three different directories.

React changed the game by introducing Separation by Concerns. It recognized that a Component is the natural unit of work. A component encapsulates its logic, structure, and styling into a single, cohesive module.

The Modern DevOps Bottleneck

Today, DevOps is still stuck in the Separation by Technology phase. We segregate our logic by the tools we use:

Terraform/OpenTofu for infrastructure.
Ansible for configuration.
Kubernetes for orchestration.

While Kubernetes has moved us toward immutability, it has also introduced the YAML trap . Furthermore, the real world isn’t always immutable. We don’t discard a developer’s machine every time we need a minor package upgrade, and stateful services (like databases) require delicate, mutable handling that pure-container strategies struggle to manage.

Enter BigConfig: The Component-Based Approach

BigConfig applies the React philosophy to DevOps. Instead of managing a Terraform repo and an Ansible repo you manage Packages.

For example, the BigConfig Package once utilize four Terraform projects and two Ansible projects under the hood, but to the user, it is a single, functional unit.

The Coupling Problem: A Practical Example

Consider a simple task: checking if the the server can send emails. Ansible needs to create a .mailrc file so the operator can use s-nail CLI to test the SMTP connection during an incident.

The Traditional Way: The infrastructure (SMTP infrastructure) is defined in OpenTofu, but the .mailrc configuration lives in an Ansible folder. This creates strong coupling. If you change your SMTP provider or port, you must manually update two different repositories.
The BigConfig Way: Responsibility is delegated, not segregated. The Tofu SMTP component is responsible for generating the configuration data, while Ansible is simply the delivery mechanism that places the file on the server. The file source path acts as a clean interface. One change in the SMTP component automatically flows through the system.

Modularity and Referential Transparency

The power of a Separation of Concerns architecture lies in its modularity. In the BigConfig Package once, we categorize providers into functional types: Compute, DNS, and SMTP.

Interchangeability: Our Compute component has three implementations: OCI, Hetzner, and DigitalOcean. Because these are built as components, you can swap one for another without touching your DNS or SMTP logic.
Referential Transparency: We utilize “no-infra” components as placeholders. This allows you to bring your own pre-existing infrastructure into the system. You can replace a live resource with its values without changing the behavior of the rest of the stack. In programming, this is called referential transparency; in DevOps, it makes brownfields easier to support.
No Degradation of Experience: Usually, abstractions make debugging harder. BigConfig is designed so that the composition of components doesn’t hide the underlying tools. You can manage the entire Composition as one, or drop down to debug a single component (like a specific Tofu component) without any additional configuration overhead.

Conclusion

We are moving past the era of “Tool-First” DevOps. By adopting a component-based mindset, we can stop managing scripts and start building systems that are as modular, testable, and maintainable as our modern frontend applications.

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

Vibe Coding Meets Vibe Ops: Automating the Last Mile of Deployment

Wed, 01 Apr 2026 00:00:00 GMT

Vibe coding has made building web applications faster and more intuitive than ever, but the Ops side of the house hasn’t quite kept up. While 37signals’ ONCE simplified the deployment architecture for vibe coders, managing the underlying infrastructure—Compute, DNS, and SMTP—remains a fragmented, manual process.

Enter the BigConfig once : the missing link that automates the manual out of your deployment pipeline.

Zero-Friction Infrastructure

BigConfig turns complex infrastructure provisioning into a few simple terminal commands. No more hunting for IP addresses or manually editing SSH configs.

Provision your VPS instantly

git clone https://github.com/amiorin/once
cd once
bb once create

Jump straight into your server (SSH config is auto-populated)
```
ssh once
```
Start ONCE TUI in your server to deploy your vibe coded application
```
sudo once
```

1. Compute

While Oracle and Hetzner are supported out of the box, BigConfig is built for extensibility. Adding a new compute provider is trivial, ensuring you aren’t locked into a single ecosystem.

2. DNS

Cloudflare integration is ready to go. Point your domain and let BigConfig handle the records, ensuring your application is reachable as soon as the compute instance is live.

3. SMTP

Transactional email is handled via Resend by default. Like the other modules, swapping this for another SMTP service is simple and straightforward.

The Vision: Dev & Ops in Plain English

The roadmap for BigConfig is headed toward a Natural Language Infrastructure. The next milestone is teaching Claude Code to utilize the BigConfig once .

Soon, a vibe coder won’t just write code in English—they will manage their entire production environment through conversation, blurring the line between development and operations entirely.

Current Manual Hurdles

We’ve automated the heavy lifting, but a few Human-in-the-Loop steps remain—mostly due to provider security and billing requirements. In the age of agentification, these hurdles may not last long:

Domain Registration: Snagging budget domains (like .website or .online for $0.85) via Namecheap is still a manual checkout process.
Oracle Cloud Setup: Creating an account requires a physical credit card. To unlock the Always Free tier (4 cores, 24 GB RAM), you must manually switch to a Pay-As-You-Go account and generate your API credentials.
The Namecheap-Cloudflare Bridge: Because Namecheap doesn’t offer an API for smaller accounts, pointing your Namecheap domain to Cloudflare DNS and generating an API token remains a manual task.
Resend Setup: Account creation and initial API key generation must be done via their web dashboard.

Conclusion

The gap between vibe coding a masterpiece and actually shipping it to a live URL is narrowing. By wrapping the fragmented worlds of VPS provisioning, DNS routing, and SMTP configuration into a unified, programmable workflow, BigConfig transforms deployment from a weekend chore into a minor detail. While a few manual gates like billing and domain registration remain—reminders of the physical world we still inhabit—the path toward a fully conversational, Natural Language Infrastructure is clear.

We are rapidly approaching a reality where the Ops in DevOps is as effortless as the Dev. If you’re ready to stop wrestling with dashboards and start shipping at the speed of thought, it’s time to give the BigConfig a spin.

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

The Power of Framing: Why BigConfig is Rebranding as a Package Manager

Fri, 13 Mar 2026 00:00:00 GMT

BigConfig began as a simple Babashka script designed to DRY up a complex Terraform project for a data platform. Since those humble beginnings, it has evolved through several iterations into a robust template and workflow engine. But as the tool matured, I realized that technical power wasn’t enough; the way it was framed was the true barrier to adoption.

The Language Barrier (and the Loophole)

BigConfig is powerful as a library, but I’ve faced a hard truth: very few developers will learn a language like Clojure just to use a library. However, history shows that developers will learn a new language if it solves a fundamental deployment problem.

People learned Ruby to master Homebrew ; they learn Nix for reproducible builds. Meanwhile, tools like Helm force users to juggle the awkward marriage of YAML and Go templates—a “solution” many endure only because no better alternative exists. To get developers to cross the language barrier, you have to offer more than a tool; you have to offer a total solution.

The “Package Manager” Epiphany

I noticed a significant shift in engagement depending on how I framed the project. When I describe BigConfig as a library, it feels abstract—like “more work” added to a developer’s plate. When I introduce it as a package manager, the interest is immediate.

In the mind of a developer, a library is a component you have to manage. A package manager is the system that manages things for you. By shifting the perspective, BigConfig goes from being a “Clojure utility” to an “Infrastructure Orchestrator.”

How BigConfig Differs

Like Nix and Guix , BigConfig embraces a full programming language. However, it avoids the “two-language architecture” common in those ecosystems—where you often have a compiled language for the CLI and a separate interpreted one for the user.

BigConfig is Clojure all the way down (in the spirirt of Emacs). This allows it to support three distinct environments seamlessly:

The REPL: For interactive development and real-time exploration.
The Shell: For traditional CLI workflows and CI/CD pipelines.
The Library: For embedding directly into your own control planes or APIs.

Beyond the language, BigConfig introduces robust client-side coordination, featuring an Atlantis-style locking mechanism that uses GitHub tags to prevent developer collisions in shared environments.

Limitless Abstraction

The level of abstraction is where BigConfig truly shines. When you adopt the system, you aren’t locked into a rigid schema; you can adapt the entire engine to your specific needs. Complex tasks—like deploying the same architecture across different hyperscalers—are reduced from massive refactors to simply updating a property. It moves the conversation from how to deploy to what to deploy.

The Roadmap

The next phase is focused on expanding the ecosystem and making package discovery seamless:

Hyperscaler Support: Having already added DigitalOcean , Hetzner , and Oracle Cloud , I am now prioritizing AWS , Google Cloud , and Azure .
Application Packages: While the first “app”—a remote development environment—is a niche use case, I’m expanding into high-demand stacks like Airflow and Rama .
The Ecosystem: I am currently defining the formal package manifest and building a registry where users can discover, version, and publish their own infrastructure packages.

Conclusion

The evolution of BigConfig is a testament to the idea that the right abstraction is just as important as the right code. By reframing the tool from a utility you have to manage into a system that manages for you, we bridge the gap between complex cloud resources and developer productivity.

As we expand our hyperscaler support and formalize our package registry, the goal remains the same: to move infrastructure management away from the “how” and toward the “what.” Whether you are deploying a niche remote environment or a massive data stack like a Data Lake, BigConfig provides the language and the logic to make your infrastructure as versionable and reproducible as your software.

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

Simple over easy for operations

Wed, 11 Mar 2026 00:00:00 GMT

Building a workflow engine for infrastructure operations is not trivial. Most people start with a simple mental model: a desired state and a sequence of functions that produce side effects. In Clojure, this looks like a simple thread-first macro:

(-> {}
    fn1
    fn2
    ...)

Your state {} is threaded through fn1 and fn2. However, real-world operations are rarely linear. They require complex branching, error handling, and conditional jumps (e.g., “if success, continue; otherwise, jump to cleanup”).

Wiring the Engine

To handle non-linear flows, we associate functions with qualified keywords (steps). Together with the next step, they form the “wiring”. You can override sequential execution by providing a next-fn to handle custom branching.

The core execution loop looks like this:

(loop [step first-step
       opts opts]
  (let [[f next-step] (wire-fn step step-fns)
        new-opts (f opts)
        [next-step next-opts] (next-fn step next-step new-opts)]
    (if next-step
      (recur next-step next-opts)
      next-opts)))

Workflow Example

Here is how we use this engine to create a client-side lock for Terraform using Git tags. The opts map represents our “World State”, shared across all functions.

We invoke it like this: (lock [] {}). The first argument is a list of middleware-style step functions, and the second is the starting state.

(->workflow {:first-step ::generate-lock-id
             :wire-fn (fn [step _]
                        (case step
                          ::generate-lock-id [generate-lock-id ::delete-tag]
                          ::delete-tag [delete-tag ::create-tag]
                          ::create-tag [create-tag ::push-tag]
                          ::push-tag [push-tag ::get-remote-tag]
                          ::get-remote-tag [(comp get-remote-tag delete-tag) ::read-tag]
                          ::read-tag [read-tag ::check-tag]
                          ::check-tag [check-tag ::end]
                          ::end [identity]))
             :next-fn (fn [step next-step opts]
                        (case step
                          ::end [nil opts]
                          ::push-tag (choice {:on-success ::end
                                              :on-failure next-step
                                              :opts opts})
                          ::delete-tag [next-step opts]
                          (choice {:on-success next-step
                                   :on-failure ::end
                                   :opts opts})))})

Debugging Made Simple

In many CI/CD systems, debugging is a nightmare of “print” statements and re-running 10-minute pipelines. Because Clojure data structures are immutable and persistent, we can use a debug macro provided by BigConfig and a “spy” function to inspect the state at every step.

(comment
  (debug tap-values
    (create [(fn [f step opts]
               (tap> [step opts]) ;; "Spy" on every state change
               (f step opts))]
            {::bc/env :repl
             ::tools/tofu-opts (workflow/parse-args "render")
             ::tools/ansible-opts (workflow/parse-args "render")})))

Using tap>, you get the result “frozen in time”. You can render templates and inspect them without ever executing a side effect.

Solving the Composability Problem: Nested Options

Operations often require calling the same sub-workflow multiple times. If every workflow uses the same top-level keys, they clash. We solve this with Nested Options.

By using the workflow’s namespace as a key, we isolate state. However, sometimes a child needs data from a sibling (e.g., Ansible needs an IP address generated by Terraform). We use an opts-fn to map these values explicitly at runtime.

The specialized ->workflow* constructor uses this next-fn to manage this state isolation:

(fn [step next-step {:keys [::bc/exit] :as opts}]
  (if (steps-set step)
    (do
      (swap! opts* merge (select-keys opts [::bc/exit ::bc/err]))
      (swap! opts* assoc step opts))
    (reset! opts* opts))
  (cond
    (= step ::end) [nil @opts*]
    (> exit 0)     [::end @opts*] ;; Error handling jump
    :else
    [next-step (let [[new-opts opts-fn]
                     (get step->opts-and-opts-fn next-step [@opts* identity])]
                 (opts-fn new-opts))]))

This logic ensures that if a step is a sub-workflow, its internal state is captured within the parent’s state under its own key. The opts-fn allows us to bridge the gap—for instance, pulling a Terraform-generated IP address into the Ansible configuration dynamically.

The Working Directory and the Maven Diamond Problem

In operations, you must render configuration files before invoking tools. If you compose multiple workflows, you run into the “Maven Diamond Problem”: two different parent workflows sharing the same sub-workflow. To prevent them from overwriting each other’s files, we use dynamic, hashed prefixes for working directories:

.dist/default-f704ed4d/io/github/amiorin/alice/tools/ansible

The hash f704ed4d is dynamic. If a workflow is moved or re-composed, the hash changes, ensuring total isolation during template rendering.

Conclusion: Simple over Easy

Tools like AWS Step Functions , Temporal , or Restate are powerful workflow engines, but for many operational tasks, they are not a good fit. BigConfig has an edge because it is local and synchronous where it counts. It turns infrastructure into a local control loop orchestrating multiple tools.

In the industry, “Easy” (using the same language as the backend, like Go) often wins over “Simple”. But Go lacks a REPL, immutable data structures, and the ability to implement a debug macro that allows for instantaneous feedback.

Infrastructure eventually becomes a mess of “duct tape and prayers” when the underlying tools aren’t built for complexity. If you choose Simple over Easy, Clojure is the best language for operations—even if you’re learning Clojure for the first time.

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

Composability: Orchestrating Infrastructure with Babashka and BigConfig Package

Tue, 10 Mar 2026 00:00:00 GMT

In the world of Clojure and DevOps, Babashka has become the gold standard for fast, scripted automation. Today, we’re exploring how to leverage the BigConfig Package with Babashka to create highly composable infrastructure workflows that are as flexible as they are powerful.

Getting Started

One of BigConfig Package’s greatest strengths is its CLI-first approach. You can invoke the walter package directly from your terminal to run complex sequences in a single line. For example, to cycle through an environment lifecycle:

bb walter create delete create delete

bb: The Babashka runtime.
walter: The specific task defined in your configuration.
create/delete: Individual steps within the workflow. You can chain these steps in any order or frequency required by your pipeline.

Configuration

To integrate walter package into your project, define your dependencies and tasks in your bb.edn file. This acts as the central configuration for your automation.

bb.edn

{:deps {io.github.amiorin/walter {:git/sha "4ea205299cf34c29fa289eacb512a31bfe604d93"}}
 :tasks
 {:requires ([io.github.amiorin.walter.package :as package]
             [io.github.amiorin.walter.tools :as walter]
             [io.github.amiorin.alice.tools :as alice])

  tofu {:doc "Render and apply OpenTofu (Terraform) plans"
        :task (walter/tofu* *command-line-args* (package/walter-opts {:big-config.workflow/params {:hyperscaler "oci"}}))}

  ansible {:doc "Run Ansible playbooks"
           :task (walter/ansible* *command-line-args* (package/walter-opts {}))}

  ansible-local {:doc "Run local Ansible playbooks via the 'alice' package"
                 :task (alice/ansible-local* *command-line-args* (package/walter-opts {}))}

  walter {:doc "Primary Walter workflow"
          :task (package/walter* *command-line-args* {:big-config.workflow/params {:hyperscaler "oci"}})}}}

Key Components

:deps: Targets the specific Git SHA of the walter package for reproducible builds.
Workflow Structure: A package typically splits logic between package.clj and tools.clj. The former composes multiple tools—like OpenTofu and Ansible—into a unified, high-level workflow.

Parameters and Composability

The walter task utilizes a primary parameter: :hyperscaler. Currently, the package offers out-of-the-box support for Oracle Cloud (OCI) and Hetzner.

Uniformity across workflows

The function (package/walter-opts ...) is the “glue” of the system. It handles two critical requirements:

Directory Mapping: Every tool needs a specific location to render configuration files. This function ensures these are correctly mapped based on the parent workflow.
Options Consistency: By populating the :big-config.workflow/params, it ensures that all tasks—whether tool-specific or package-wide—share the same options.

Cross-Package Integration

BigConfig Package excels at mixing and matching. While tofu and ansible are defined inside the walter package, you can seamlessly pull in tasks like ansible-local from entirely different packages (such as alice).

Developer Tip: Even when nesting workflows inside a larger process, you retain the ability to invoke sub-workflows interactively. This makes debugging specific infrastructure segments significantly faster.

Conclusion

By transforming standard Terraform and Ansible code into a managed workflow, you gain the ability to invoke resources multiple times with surgical precision. Since the only persistent state is the Terraform state (ideally stored in an S3 backend), your entire team can operate the same resources from different machines.

Beyond standard deployment, BigConfig Package provides built-in coordination steps like git-check, git-push, lock, and unlock-any. These utilities ensure that whether you are working in a tool-specific workflow or a massive package-wide deployment, your team remains in sync and your iing and configuration management, BigConfig Package removes the friction typically found in multi-cloud deployments. This modular anfrastructure remains stable.

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

BigConfig Package Walter Fork this package or use the package template.

Universal Infrastructure: Solving the Portability Gap with BigConfig

Mon, 09 Mar 2026 00:00:00 GMT

The primary challenge with Terraform and Ansible has always been portability. It is notoriously difficult to take a solution written for one environment and apply it to another without significant manual adjustments. Kubernetes achieved its massive success by leveling this playing field. With tools like Helm, you have a package manager that allows you to install applications without worrying about whether you are on-prem or using a specific hyperscaler.

However, in the world of Kubernetes, a common sentiment is to avoid stateful applications like databases unless you have mastered every technical nuance. Most internal platforms use Kubernetes for stateless services while relying on managed databases provided by the hyperscaler.

BigConfig Package changes this dynamic by enabling the creation of universal applications that are both stateful and stateless. Walter , the first application built with BigConfig, demonstrates this by deploying seamlessly across Oracle Cloud and Hetzner.

The Portability Challenge

Infrastructure is rarely uniform. When moving between providers, you encounter several inconsistencies:

The IP address property is named differently depending on the provider.
The default SSH user varies.
The UID of the default user is often inconsistent.

The solution involves defining a standardized schema for output parameters in every main.tf file:

Gluing Infrastructure to Configuration

This approach effectively glues the OpenTofu infrastructure step to the Ansible configuration step. By parsing the JSON output from the infrastructure layer, we can pass critical connection data directly into the workflow:

(defn opts-fn
  [opts]
  (let [dir (workflow/path opts ::tools/tofu)]
    (merge-with merge opts {::workflow/params (if (fs/exists? dir)
                                                (-> (p/shell {:dir dir
                                                              :out :string} "tofu output --json")
                                                    :out
                                                    (json/parse-string keyword)
                                                    (->> (s/select-one [:params :value])))
                                                {:ip "192.168.0.1"
                                                 :sudoer "ubuntu"})})))

As long as the OpenTofu step adheres to the schema by providing an ip, sudoer, and uid, any new hyperscaler can be integrated into this BigConfig Package.

Handling Distribution Differences

What about variations in Linux distributions? This can be handled within Ansible or, similar to our Terraform approach, by using different source files based on the distribution.

(defn tofu
  [step-fns opts]
  (let [opts (workflow/prepare {::workflow/name ::tofu
                                ::render/templates [{:template (keyword->path ::tofu)
                                                     :overwrite true
                                                     :hyperscaler "hcloud"
                                                     :transform [["{{ hyperscaler }}"]]}]}
                               opts)]
    (workflow/run-steps step-fns opts)))

The power lies in the dynamic folder pathing. By using the template variable "{{ hyperscaler }}" for the hyperscaler, the directory containing the infrastructure code becomes dynamic. This allows us to share core Ansible logic while diverging where necessary, ensuring the code remains clean and manageable.

Conclusion

By standardizing the handshake between infrastructure provisioning and configuration management, BigConfig Package removes the friction typically found in multi-cloud deployments. This modular approach ensures that your automation remains truly portable, allowing stateful workloads to run wherever they are needed most without being locked into a single provider’s ecosystem.

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

BigConfig Package Walter Fork this package or use the package template.

The YAML Trap: Escaping Greenspun’s Tenth Rule with BigConfig

Thu, 26 Feb 2026 00:00:00 GMT

Greenspun’s Tenth Rule is a famous (and delightfully cynical) adage in computer science. While it was born in the era of C and Fortran, it has never been more relevant than it is today in the world of Platform Engineering.

If you’ve ever felt like your CI/CD pipeline is held together by duct tape, YAML-indentation prayers, and sheer willpower, you’ve lived this rule.

What is Greenspun’s Tenth Rule?

In the early 90s, Philip Greenspun stated:

“Any sufficiently complicated C or Fortran program contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.”

The core insight is that once a system reaches a certain level of complexity, it inevitably requires high-level abstraction, automation, and dynamic logic. Instead of starting with a powerful, established language (like Lisp) built for those tasks, developers often “accidentally” reinvent a mediocre version of one using brittle configuration files and makeshift scripts.

The Rule in the DevOps Ecosystem

In DevOps, we strive for Infrastructure as Code (IaC). However, because we started with static configuration formats (YAML/JSON) and tried to force them to perform complex logic, we’ve essentially proven Greenspun right.

1. The YAML “Programming” Trap

Tools like Terraform, Ansible, Helm, and GitHub Actions began as simple configuration formats. But as users demanded loops, conditionals, and variables, these tools evolved into “accidental” languages.

The Problem: You end up writing complex business logic inside strings within a YAML file.
The Reality: You are using a “bug-ridden implementation” of a real programming language, but without the benefit of a debugger, a compiler, or proper unit testing.

2. Kubernetes as a Distributed Lisp

Some architects argue that Kubernetes is the ultimate manifestation of this rule. Its control loop the constant cycle of reconciling desired state vs. actual state mimics the recursive nature of Lisp environments. It is, in essence, a programmable platform designed to manage other programs.

Escaping the Trap: Putting Lisp Back in Ops

The industry has invested massive human capital into building Ansible roles, Helm charts, and Terraform modules. We shouldn’t throw them away, but we must stop trying to make them do things they weren’t designed for.

How do we escape Greenspun’s trap without rebuilding everything from scratch? By assimilating these tools (to borrow a 90s Star Trek reference).

This is the core design principle of BigConfig. Instead of fighting against limited YAML DSLs, BigConfig uses Clojure a modern, production-grade Lisp to wrap and orchestrate existing tools.

The BigConfig Philosophy: Express infrastructure logic with the most powerful dynamic language available, while still leveraging the ecosystem you already have.

Why it Matters: The Power of Assimilation

The Tenth Rule is a warning: Don’t reinvent the wheel poorly. If your infrastructure requires complex logic, stop forcing it into a flat config file.

From Static Files to Fractal Architecture

While a standard Helm package is limited strictly to Kubernetes, a BigConfig package is a Clojure function. Because BigConfig assimilates Ansible and Terraform alongside Helm, it isn’t siloed.

Truly Cloud Native: A Kubernetes application that requires specific cloud resources (like an S3 bucket or an RDS instance) can be abstracted into a single, cohesive unit.
First-Class Functions: In BigConfig, everything is a function. This leads to a fractal architecture where every layer from a single container to a multi-region cloud deployment is governed by the same recursive logic: Observe, Diff, and Act.

Ready to stop writing logic in YAML?

Operations is a hard problem. YAML is too rigid, and Go is too low-level for rapid infrastructure iteration. While Python and JavaScript are popular, they lack the REPL-driven development flow that makes infrastructure-as-code feel truly interactive.

Clojure is the most robust Lisp available today and it won’t let you down.

Conclusion

Greenspun’s Tenth Rule isn’t just a witty observation; it’s a technical debt warning. When we try to solve 21st-century infrastructure challenges using static configuration files, we inevitably end up building “shadow” programming languages that are difficult to test, impossible to debug, and fragile to scale.

By embracing a functional, Lisp-based approach through BigConfig, we stop fighting the limitations of YAML and start leveraging the power of actual logic. Instead of building a “bug-ridden implementation of half of Common Lisp,” we use the real thing Clojure to orchestrate, automate, and scale.

The goal of Platform Engineering shouldn’t be to write more scripts; it should be to create elegant, recursive systems that can manage themselves. It’s time to move past the duct tape and prayers and give our infrastructure the robust, dynamic foundation it deserves.

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

Introducing BigConfig Package

Mon, 23 Feb 2026 00:00:00 GMT

Helm has long been the gold standard for Kubernetes, but it hits a wall the moment you step outside the cluster. While there are projects to bridge this gap, they never quite achieved Helm’s level of ubiquity.

BigConfig Package is taking a different path. It aims to deliver both infrastructure and software as a unified, cohesive delivery.

The Case Against the “CLI Proliferation”

One of the most radical aspects of BigConfig Package is what it lacks: a dedicated CLI. In BigConfig’s philosophy, a CLI is just another tool that end-users should tailor to their specific needs. We argue that the current explosion of specialized CLIs is actually a symptom of unsolved underlying problems:

Bash Limitations: The fragility of shell scripting forces developers to wrap logic in binaries.
Interoperability Gaps: Integrating multiple CLI tools via scripting is often easier than managing complex library dependencies, leading to “scripting fatigue.”
The Go Paradox: While static languages like Go are excellent for single binary deployments, they force the creation of monolithic tools that make composition more difficult (e.g., Terraform plugin is based on GRPC).

BigConfig solves these issues by leveraging Clojure and Babashka. By using a dynamic, high-performance language, we replace rigid tools with flexible Babashka tasks, offering the power of a library with the convenience of a script.

The Three Environments: Shell, REPL, and Lib

Most DevOps professionals only interact with tools (like Helm) in the Shell environment. BigConfig expands this to three distinct planes:

Shell: For quick, command-line execution.
REPL: For real-time, interactive debugging and experimentation. No more “edit-coffee-fail” loops.
Lib: For deep automation. You can orchestrate complex scenarios without reimplementing anything when switching gear from CLI tool to in-house solution.

What is a BigConfig Package?

There is no need to reinvent the wheel. A BigConfig Package is simply a Clojure repository, and the package itself is just a Clojure function written with BigConfig Workflow .

Our first application, Walter , tackles a challenge Helm simply wasn’t built for: Remote Development Environments on machines instead of containers.

While devcontainers are popular, containerizing a full development environment on Kubernetes has significant overhead and limitations. Nix is the superior choice here. Its native multi-user support makes it ideal for high-powered “beefy” machines shared by multiple developers.

In this setup, Terraform and Ansible do the heavy lifting but remain completely hidden once development is complete, leaving behind a clean interface that doesn’t “leak” its underlying complexity.

The use case of Remote Development Environment is getting traction and by integrating a Linux user to run a self-hosted GitHub runner, you unlock massive efficiency:

Cost Reduction: Run Dev and CI on the same hardware.
Instant Caching: CI pipelines never miss a cache hit because the environment is shared.
Absolute Parity: “Works on my machine” becomes a reality because the Dev and CI environments are identical.
Native Speed: Docker runs natively, eliminating the virtualization overhead found on macOS or Windows.

Comparison: Helm vs. Babashka

The Helm Way

In this paradigm, you manage repositories and install charts. It typically requires juggling YAML, Go templates, and Bash.

helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update
helm install my-redis bitnami/redis

The Babashka Way

With BigConfig, helm install becomes bb redis create. The CLI is defined by the package creator but can be customized by the user.

We use create instead of install in this case because the infrastructure is part of the delivery. Using bb instead of bash provides a unified environment where everything is written in Clojure, replacing the fragmented mix of Bash, Compiled Go, and YAML.

;; bb.edn
{:deps {io.github.amiorin/redis {:git/sha "eaff6f"}}
 :tasks
 {:requires ([redis :as redis])
  redis {:doc "Provision a Redis instance"
         :task (comp-wf/redis* *command-line-args*)}}}

bb redis create
bb redis delete

Conclusion

The era of managing a dozen different CLIs just to deploy a single stack is reaching its breaking point. BigConfig offers a way out, not by building a bigger tool, but by providing a more flexible foundation. By unifying the way we handle infrastructure and applications through Clojure, we aren’t just deploying software; we are creating a more cohesive, reproducible, and developer-friendly ecosystem.

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

My Keyboard Endgame: 34 Keys, 3 Layers, and a Macropad

Fri, 13 Feb 2026 00:00:00 GMT

I spend a lot of time analyzing other people’s “endgame” setups, and I honestly don’t understand why so many users accept such steep learning curves.

The secret to a sustainable workflow is simplicity. If your layout is so complex that a month away from your desk “drives you nuts” when you return, it’s not an endgame—it’s a chore. Here is how I stripped my setup down to the essentials without losing productivity.

The Hardware Philosophy

34 Keys is Enough: I’m a Ferris Sweep MX devotee. You can buy the the Silakka54 from Aliexpress if you are on a budget.
2 Thumb Keys: You don’t need a cluster; two well-placed keys are sufficient.
Externalize the Complexity: Don’t cram multimedia into layers. Use a dedicated macropad like the Elgato Stream Deck (or the budget-friendly Ajazz AKP03E). This gives you physical dials and buttons for the stuff that doesn’t belong on a typing layer.
The Trackpad: The Apple Magic Trackpad remains king. I use Mouseless to reduce dependency, but I refuse to add clunky mouse layers to my keyboard.

The Three-Layer Rule

Most people over-engineer their firmware. I’ve cut out the “magic” (No Macros, No Tap Dance, No Key Overrides) in favor of logic.

Layer 0: Alpha. Standard typing.
Layer 1: Numbers & Navigation. I use the zxcv row for numbers. This leaves 15 keys free for my Tiling Window Manager of choice, AeroSpace .
Layer 2: Symbols. I don’t understand splitting symbols across layers. There are exactly 29 essential symbols—they fit perfectly on one layer. Keep them together so your brain doesn’t have to hunt.

Taming the “Necessary Evils”

Home Row Mods: These can be a headache. I’ve contained the “evil” by only using two keys for mods (Z/X and comma/dot).
Combos: Only one—F+J for CAPS WORD.
Input Methods: Stick to Tap-Hold and Layer Toggle only. Anything else increases the likelihood of misfires.

The Terminal-Centric Stack

If you live in the terminal, your hardware is only as good as your software’s ability to interpret it. My stack focuses on modern protocols and efficiency:

Protocol: Use Kitty Keyboard Protocol (kkp) for better key handling.
Terminal: Ghostty (fast, native, modern).
Multiplexer: Zellij over tmux.
Editor: Doomemacs (Vi bindings forever).

My skhd Configuration

Since I avoid complex firmware overrides, I handle my application-specific shortcuts via skhd . This keeps my keyboard “dumb” and my configuration portable.

# Use xxd -psd to find the hex code and remove 0a

# Navigation: Prev/Next tab in Chrome
cmd - i : skhd -k "shift + cmd - 0x21"
cmd - o : skhd -k "shift + cmd - 0x1E"

# Organization: Move tab left/right in Chrome
shift + cmd - i : skhd -k "shift + ctrl - 0x74"
shift + cmd - o : skhd -k "shift + ctrl - 0x79"

# History: Back and Forward
cmd - 0x2B : skhd -k "cmd - 0x21"
cmd - 0x2F : skhd -k "cmd - 0x1E"

# Terminal Fix: Emacs in terminal doesn't support M-[
# Mapping M-5 as a workaround
alt - 0x21 : skhd -k "alt - 5"

Final Thoughts

Keep it simple. If you can’t explain your layout to yourself in 30 seconds, it’s too complicated. Externalize your macros to a macropad, keep your symbols on one layer, and let your software do the heavy lifting.

Replacing Integrant and Docker Compose with BigConfig System

Tue, 10 Feb 2026 00:00:00 GMT

In the Clojure ecosystem, we’ve grown accustomed to a structural divide. We manage our application architecture with libraries like Integrant and our external dependencies (databases, queues, caches) with Docker Compose or Process Compose .

While effective, this separation creates a constant “context switch.” Your application lives in the REPL, but its foundation is buried in YAML and EDN files. One requires evaluating expressions; the other requires multiple steps to refresh, start, or stop using Emacs or a Shell.

To bridge this gap, I’ve been developing BigConfig System. It is a library inside the BigConfig ecosystem that treats system lifecycles as programmable workflows rather than static dependency graphs.

The “Emacs” Philosophy

I view BigConfig through the lens of the Emacs philosophy: the environment is something you should never have to leave.

As projects scale, we traditionally reach for external orchestrators. However, from the perspective of a running REPL, they have foreign interfaces. By keeping the orchestration logic strictly within Clojure I’ve unified the application and its dependencies. Now, I can manage the entire stack, from environment variables to database migrations, without ever dropping out of my REPL session.

Solving the Postgres Dependency

To test the limits of BigConfig System, I moved away from static Postgres containers in favor of a dynamic, code-driven workflow. I needed to handle multiple profiles (like dev for long-running sessions and test for transient fixtures) simultaneously without port collisions or state pollution.

Instead of a binary “started/stopped” toggle, BigConfig manages a granular execution sequence:

Environment Prep: Dynamically set variables for project-root/.postgres/[profile].
Filesystem Cleanup: Ensure a clean slate by wiping stale data from previous runs.
Initialization: Programmatically execute initdb.
Smart Startup: Launch Postgres via babashka/process, grepping logs for the “ready to accept connections” regex.
Provisioning: Chain-load createuser, createdb, and sql-migrate.
Teardown: A dedicated stop-fn ensures the process tree is destroyed cleanly.

Abandoning the “Refresh” Cycle

Many Clojure developers rely on cider-ns-refresh, but for a refined flow, I find it a bit heavy-handed. My development style centers on evaluating expressions directly within (comment ...) blocks.

By using cider-eval-defun-at-point and cider-inspect-last-result, I gain high-granularity control. The beauty of BigConfig is that the CIDER inspector updates automatically as the system state changes. I can see the system evolve in real-time, directly within my editor, without the jarring reset of a full namespace refresh.

The Workflow in Action

Here is how BigConfig System expresses a managed process as a programmable workflow. Notice how the system lifecycle is defined as a series of step functions (background-process and stop):

(comment
  (let [kill-timeout 150
        shutdown-timeout 100
        clj-timeout 3000]
    ;; Define a managed background process with a specific lifecycle
    ;; Start the process in the background and block the main thread
    ;; until the regex is found or the timeout triggers.
    (defn background-process [opts]
      (let [re-opts (into {} [[:cmd (format "clj -X big-config.system/main :shutdown-timeout %s" shutdown-timeout)]
                              [:regex #"token"]
                              [:timeout clj-timeout]
                              [:key ::proc]])
            opts (re-process re-opts opts)]
        (add-stop-fn opts (fn [{:keys [::proc] :as opts}]
                            (when proc
                              (destroy! proc kill-timeout))))))
    ;; Define the system as a stateful, wired workflow
    (def ->system (->workflow {:first-step ::start
                               :wire-fn (fn [step _]
                                          (case step
                                            ::start [background-process ::end]
                                            ::end [stop]))}))
    ;; sys1 starts and stops the system. It useful during the development of the system itself.
    ;; sys2 starts only. This is useful in all the other cases.
    (into {} [[:sys1 (into (sorted-map) (->system [log-step-fn] {::bc/env :repl}))]
              [:sys2 (let [system (atom (into (sorted-map) (->system [log-step-fn] {::bc/env :repl
                                                                                    ::async true})))]
                       (stop! @system)
                       @system)]])))

(defn main [& {:keys [shutdown-timeout]}]
  (assert shutdown-timeout)
  (.addShutdownHook (Runtime/getRuntime)
                    (Thread. (fn []
                               (println "\n[Shutdown Hook] Cleaning up resources before exit...")
                               (Thread/sleep shutdown-timeout))))
  (println "Script is running... (Press Ctrl+C to stop)")
  (println "token")
  @(promise))

Is it better than Integrant and Docker Compose?

It is still early days, and I am currently refining the library. One major advantage is that there is no longer a split between EDN, YAML, and CLJ files—it’s now just a single CLJ file. The shift in developer experience is already palpable. By replacing the fragmented mix of Docker Compose, Process Compose, and Integrant with a unified Clojure-based workflow, I’ve gained a level of introspection and speed that configuration-based tools and libraries simply cannot provide.

Conclusion

My quest to eliminate configuration files has reached a new milestone with system.edn and compose.yaml. By committing to a “never leave the REPL” philosophy, I’ve replaced traditional CLI tools and libraries with direct evaluation. While abandoning namespace reloading in favor of evaluating only the final expression isn’t for everyone, it has fundamentally transformed my productivity.

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

Beyond the Shell: Reimagining DevOps with the Clojure REPL

Mon, 02 Feb 2026 00:00:00 GMT

We are all familiar with the standard REPL experience provided by shells like Bash, Zsh, or Fish. However, far fewer developers have experienced the power of a Clojure REPL. I built BigConfig in Clojure because I believe we should stop relying on fragmented shell environments for operations and development. By moving to a unified Clojure REPL, we can significantly boost productivity.

The Problem: The “Edit/Run” Loop

Let’s look at the standard workflow for a tool like Terraform:

cd into the project root.
Edit the .tf files.
Run terraform plan in the shell.
Repeat until it works.

Now, imagine scaling this across multiple tools. Suddenly, you aren’t just running Terraform; you’re managing ansible, kubectl, and helm. Each tool lives in a different directory. The cognitive cost of switching directories and managing multiple shell sessions becomes a massive bottleneck. You either waste time cd-ing back and forth or you end up with twenty terminal tabs, losing track of which is which.

The BigConfig Alternative: “Edit/Evaluate”

With BigConfig, this friction disappears. Instead of a “shell-per-directory” approach, we use a single, persistent JVM:

Jack-in once: Like a shell, the JVM stays running, but you only need one instance for your entire project.
Unified Workspace: Clojure files and tool-specific configurations (YAML, HCL, etc.) live in the same project.
Instant Feedback: You evaluate Clojure code directly within your editor. No context switching is required.

In the traditional model, “Edit” and “Run” happen in two different programs (the editor and the shell). In BigConfig, “Edit” and “Evaluate” happen in the same program, your editor. Whether you are managing one tool or a hundred, the workflow remains identical.

Workflow: A New Type of Flow Control

To make this possible, BigConfig introduces a specialized flow control: the workflow.

While if is the classic example of flow control in code, a BigConfig workflow operates on a workbench map. You define a series of steps (specialized functions) that execute sequentially, reading from and writing to this shared map.

For example, the workbench map might contain a sequence like ["terraform init", "terraform plan"]. A specific step reads those commands and executes them in a directory defined elsewhere in the map. Even if you prefer writing HCL or YAML manually rather than generating it, BigConfig’s workflow engine automates the orchestration, so you don’t have to.

Real-World Application: Developing Terraform Providers

I use BigConfig for both operations and core development. Currently, I’m building a Terraform Provider in Clojure . Because the gRPC protocol documentation can be sparse, I built a BigConfig workflow to log and analyze traffic.

I use two primary workflows for this:

The hcloud workflow: This starts the Hetzner Terraform Provider in dev mode, launches a gRPC “man-in-the-middle” to log traffic, renders a test main.tf, executes Terraform, and transforms the resulting gRPC Java classes into readable Clojure maps.
The dev workflow: This is identical to the first, but it swaps the Hetzner provider for my own Clojure-based provider in development.

(def hcloud-wf (->workflow {:first-step ::start
                            :wire-fn (fn [step step-fns]
                                       (case step
                                         ::start [start-hcloud ::start-proxy]
                                         ::start-proxy [start-proxy ::prepare]
                                         ::prepare [prepare ::render]
                                         ::render [render/render ::exec]
                                         ::exec [(partial run/run-cmds step-fns) ::fix-messages]
                                         ::fix-messages [fix-messages ::end]
                                         ::end [stop]))}))

(comment
  (into (sorted-map) (hcloud-wf {::bc/env :repl
                                 ::test-name "hcloud"})))

(def dev-wf (->workflow {:first-step ::start
                         :wire-fn (fn [step step-fns]
                                    (case step
                                      ::start [start ::start-proxy]
                                      ::start-proxy [start-proxy ::prepare]
                                      ::prepare [prepare ::render]
                                      ::render [render/render ::exec]
                                      ::exec [(partial run/run-cmds step-fns) ::fix-messages]
                                      ::fix-messages [fix-messages ::end]
                                      ::end [stop]))}))
(comment
  (into (sorted-map) (dev-wf {::bc/env :repl
                              ::test-name "first"})))

Goodbye, Debuggers

I don’t need a traditional debugger for this. Because these workflows are just pure functions that transform a map, I can “attach” any value I want to inspect to the workbench map at any step.

Thanks to Clojure’s qualified keywords, I never have to worry about naming conflicts between different steps. I can see the entire state of my infrastructure and my code in one place, instantly.

Conclusion

Traditional DevOps workflows are often bogged down by the “Edit/Run” loop, requiring developers to constantly switch between editors, directories, and terminal tabs. BigConfig solves this by leveraging a persistent Clojure REPL to create a unified “Edit/Evaluate” environment. By orchestrating tools like Terraform through a centralized “workbench map,” BigConfig eliminates context switching and replaces traditional debugging with instant, stateful feedback. It transforms fragmented operations into a seamless, code-driven flow where infrastructure and development live in a single, interactive workspace.

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

Update

I received an interesting question and wanted to share my response here.

Question

The REPL often suffers from reproducibility issues due to out-of-order execution. Does BigConfig address this in some way?

Answer

This is addressed both by Clojure itself and within BigConfig’s architecture.

Clojure provides the foundation because out-of-order execution would be detrimental to any serious development. By using Clojure for operations—rather than Python or Node—we can leverage the same robust patterns used in traditional Clojure REPL-driven development.

BigConfig specifically solves this by introducing a new control flow: the workflow. A workflow is a function that invokes other functions, called steps, similar to AWS Step Functions (which served as an inspiration for BigConfig). If you are modifying a specific step, such as prepare, you would edit it like this:

(comment
  (do
    (defn prepare [opts]
      ; you are developing the code of this step
      )
    (into (sorted-map) (hcloud-wf {::bc/env :repl
                                   ::test-name "hcloud"}))))

In my editor, I simply press C-c C-c to evaluate everything in the correct order. I can then inspect the workbench map in one window and the stdout in another, ensuring a predictable and reproducible state every time.

Hypermedia: The New/Old Architecture Set to Revolutionize Web and Mobile Development

Tue, 18 Nov 2025 00:00:00 GMT

Hypermedia is the tech everyone is talking about, yet it remains a bit of a mystery for many. After diving deep, I’m convinced this new/old approach is not just a trend—it’s poised to disrupt how we build applications on both the web and mobile.

Here are my thoughts, inspired by the powerful concepts driving this shift.

What is a Hypermedia Driven Application (HDA)?

The Hypermedia Driven Application (HDA) architecture—championed by projects like htmx —is an ingenious fusion of old and new.

It takes the simplicity and flexibility of traditional Multi-Page Applications (MPAs) and marries it with the smooth user experience (UX) typically found only in complex Single-Page Applications (SPAs).

After struggling to grasp this concept myself, I decided the best way to understand it was to build one. That journey led me to the fascinating world of Hyperlith and Datastar . The ideas behind Hyperlith are truly revolutionary. Hyperlith stands for Hypermedia and Monolith.

The Philosophy: Dumb Terminals, Smart Backend

In the HDA model, your application’s logic, state, and rendering all reside in the backend.

Your distributed application runs entirely on the server.
The browser or mobile app becomes a dumb terminal.

Pages are conceptually replaced by views (for both web and mobile). When a user interacts with a control (like a button), it triggers an action on the server. This action recalculates the current view, which is then pushed to all relevant users.

Think about a traditional leaderboard implemented as a web page: instead of polling or building complex real-time infrastructure, the view automatically updates for all online users as soon as the data changes on the server. The multiplayer capabilities are practically trivial!

The Disruption: Why HDAs Win

This server-side approach has several game-changing implications for development:

Unified Codebase for Web and Mobile

With frameworks like Hyperlith and Datastar, you can achieve amazing results without starting from scratch. Instead of managing separate SPAs or native mobile apps, you manage a single backend that renders views suitable for any client.

Lightning-Fast Mobile Development

Mistake a typo in your app? Need to tweak a UI element? In the HDA model, you simply fix the code on the server. You don’t have to upload a new mobile app version and wait for store approval. The changes are live instantly for all users.

Effortless A/B Testing

Since all rendering happens on the server, implementing server-side A/B testing becomes incredibly simple. You can easily serve different versions of a view to different user segments without complicated client-side routing or code splitting.

Simplified Shared & User-Specific Views

The HDA architecture makes it easier to create views that are shared among users (like a stock ticker) and then progressively make portions user-specific (like a profile widget).

It’s easier to start with a shared view and then make it specific than to start with isolated pages and struggle to add shared portions later.

Efficient Updates and Network Performance

Many people worry that sending the complete HTML on every change won’t scale, especially on mobile networks. This concern is often misplaced.

Instead of thinking of it as constant full-page refreshes, imagine it as a stream of bytes, highly compressed with algorithms like Brotli.

The compression acts as your caching and diffing mechanism, achieving compression ratios of 100 or more! This efficiency is critical: a $1.5 MB/s sustained throughput on the server is equivalent to handling $150 MB/s without compression, drastically reducing infrastructure costs.

Seamless User Experience with DOM Morphing

Modern apps like YouTube and Spotify offer that polished mini-player experience, letting you keep watching content while navigating to a new page. This amazing user experience is a natural fit for HDAs, thanks to a technique called DOM morphing.

Instead of completely swapping out the entire page (like in a traditional MPA) or relying on complex client-side routing (like in an SPA), the HDA backend sends the new rendered view. The client then intelligently morphs the existing DOM to match the new version. This means that persistent elements—like an ongoing video player or music widget—can retain their place, focus, and internal state, resulting in a smooth, interruption-free user flow.

BigConfig and Internal Developer portals

The strategic shift towards Hypermedia Driven Applications (HDA) fundamentally alters the economics of development, making internal solutions significantly more viable. As the cost of building web and mobile experiences drops, so too does the traditional calculus of “build vs. buy.”

To capitalize on this change, our next major step is integrating Internal Developer Portal capabilities directly into BigConfig. This addition completes BigConfig’s transformation into the essential, comprehensive library for scalable operations, providing core building blocks like configuration as code, workflow orchestration, scaffolding, robust persistency, auditing, control planes, and now, internal developer portals.

I’m working on the code right now. Stay tuned, I will update the blog post as soon as I have the first version.

See it in Action

I built a simple HDA example to prove the concept. When you open it in two separate browser tabs and interact with one, the other updates instantly.

The most incredible part? Zero JavaScript was required to implement the real-time, multiplayer capabilities. The state is managed centrally and seamlessly on the server.

The future of application development is here, and it looks a lot like the past—just smarter, faster, and much more scalable.

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

Demystifying the control plane: the easy upgrade path from GitOps with BigConfig

Wed, 05 Nov 2025 00:00:00 GMT

For many engineering teams, GitOps has been a game-changer, providing a declarative way to manage infrastructure and applications. But as complexity grows, you may find your processes hitting a ceiling. The natural next step? Upgrading to a control plane.

Often, teams hesitate, believing this shift requires a costly, painful rewrite of their entire configuration management solution when it is written in Terraform or Ansible. They fear increased cognitive load and massive sunk costs.

This is where BigConfig changes the narrative.

With BigConfig, you can upgrade your existing GitOps solution to a full-fledged control plane without rewriting everything from scratch or overwhelming your team.

Control plane components

A control plane is essentially a brain that manages your entire system. It operates on three fundamental components:

The API: The interface where users (or other systems) declare what they want.
The desired state: The single source of truth—the definitive record of how the system should look.
The control loop: The tireless worker that continuously observes the actual state and makes changes to match the desired state.

Desired state persistence

The desired state is the heart of the control plane, and it needs to be persisted in a reliable, flexible way.

When dealing with complex, nested configuration data—like a complete picture of your developers, teams, environments, policies, and services—a traditional SQL data model can quickly become cumbersome. Instead, storing the state as a flexible, nested map is far more efficient and manageable.

Audit log

A key requirement for a robust control plane is a detailed audit log. We don’t just want the final state; we want to know how it got there like we do in GitOps.

BigConfig uses a pattern often referred to as Event Sourcing (though we’ll keep it simple here). Instead of storing only the current state, we store all the events that generated that state. This gives you an immutable, perfect audit trail and allows you to rebuild the state at any point in time.

BigConfig Store

BigConfig tackles this persistence challenge by leveraging Redis Sorted Sets to store both the events and periodic snapshots of the state. This provides high performance and structure for our time-series data.

Let’s look at a conceptual example in code (using Clojure) that demonstrates how BigConfig manages concurrent updates without inconsistency:

(deftest inc-test
  (testing "inc from 2 instances works"
    (let [port (-> (system-state)
                   :redis/server
                   :port)
          default-opts {:store-key (-> (random-uuid) str)
                        :initial-state {:cnt 0}
                        :wcar-opts {:spec {:port port}
                                    :pool :none}
                        :snapshot-every 2
                        :business-fn (fn [state _event _timestamp]
                                       (update state :cnt inc))}
          store1 (sut/store! default-opts)
          store2 (sut/store! default-opts)
          times 10]
      (dotimes [_ times]
        (sut/handle! store1 {:op :inc})
        (sut/handle! store2 {:op :inc}))
      (is (= [{:cnt (- (* times 2) 1)} {:cnt (* times 2)}] [@store1 @store2])))))

In this counter test, we define key parameters:

:store-key: The key used in Redis.
:initial-state: The starting Desired State (e.g., {:cnt 0}).
:snapshot-every: Defines how frequently a state snapshot is saved to Redis, dramatically accelerating recovery.
:business-fn: The pure function that defines how the state changes based on an event. Purity is critical; it guarantees that replaying events (for recovery or auditing) yields the exact same Desired State.

Notice how two concurrent writers (store1 and store2) process events via the handle! function, yet the final state remains consistent. This is the power of a well-architected control plane store. The desired state can grow to encompass everything: your list of developers, teams, services, policies—all in one place.

BigConfig Workflow

The final component is the control loop. It is a dedicated reader of the desired state. Its sole purpose is to observe the actual state of your infrastructure and ensure it matches the desired state specified in the store.

BigConfig seamlessly integrates here.

BigConfig workflow takes the comprehensive desired state, renders it into the necessary configuration files (reused from an existing Terraform or Ansible GitOps project), and applies them to the target environment. You keep your existing configuration logic while BigConfig provides the intelligence and state management above it.

The result? You upgrade from configuration automation to a more dynamic, auditable, and consistent control plane without the nightmare of a complete rewrite.

Example code

I’m working on the code right now. Stay tuned, I will update the blog post as soon as I have the first version.

Conclusion

The shift from GitOps to a full-fledged control plane is the inevitable evolution for engineering teams grappling with growing complexity, but it doesn’t have to be a painful, expensive overhaul. BigConfig provides the crucial bridge, allowing you to move beyond simple configuration automation without sacrificing your investment in existing Terraform or Ansible logic. By introducing a robust control plane built on a flexible desired state, an immutable audit log (stored efficiently in Redis Sorted Sets), and a workflow engine, BigConfig ensures consistency and concurrency even as your system scales. This model separates the declaration of what you want from the execution of how to get it, enabling you to gain the dynamic, auditable, and resilient desired state persistence of a modern control plane while retaining your established configuration workflows.

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

Local-First GitHub Actions Strategy

Tue, 04 Nov 2025 00:00:00 GMT

If you’ve spent any significant time with GitHub Actions (GHA), you know the drill: it can be a massive time-saver, but when things go wrong, the development loop is painfully slow. Committing, pushing, waiting for the run to fail, and then repeating… it’s a productivity killer.

Over time, I’ve refined a strategy that cuts this frustrating cycle short. My philosophy is simple: Avoid any GitHub Actions feature that isn’t available or easy to replicate locally.

Show me the code

The Slow CI Development Loop

The biggest headache with GitHub Actions isn’t writing the YAML; it’s debugging it.

When your build or test step fails, the quickest way to fix it is to reproduce the failure on your local machine. But if your CI heavily relies on GHA features, that local reproduction becomes difficult or impossible.

This leads to the dreaded “commit-and-wait” debugging cycle.

Docker-First CI Recipe

My solution is to treat GitHub Actions purely as an execution environment for a set of simple, executable shell commands. Everything complex—the build, the test environment, the dependencies—is encapsulated within a Docker container.

My Typical CI Workflow Steps:

Free Disk Space: My development container is a beast (around 18 GB), so the first step is necessary cleanup.
Download Devcontainer: Pull the large, pre-built image that looks exactly like my devmachine.
Clone the Repository: Standard checkout action.
Run Tests Inside the Container: Execute the tests by using docker run and mounting the repository volume.

I don’t write my own GitHub Actions. I rely only on standard shell commands and docker run. If I need something complex, I use battle-tested, popular actions written by others (like actions/checkout), but I don’t try to author my own actions.

Developing CI Without Committing

The real game-changer is the ability to run and debug the CI code locally without ever making a commit. This is the inner loop that makes CI development fast.

Since my CI logic is just a single step executed by GHA, I can extract and run that exact step on my machine.

The Local Debug Command

This single line of bash and command-line tools allows me to extract the critical run step from my workflow file and execute it locally:

cat .github/workflows/ci.yml \
  | jet -i yaml -o json \
  | jq -r .jobs.test.steps[3].run \
  | GITHUB_WORKSPACE=$(pwd) bash -s

cat .github/workflows/ci.yml: Reads the workflow file.
jet -i yaml -o json: Converts the YAML to JSON (I prefer using jet, but you could use other tools).
jq -r .jobs.test.steps[3].run: Extracts the exact run command script from the correct job and step.
GITHUB_WORKSPACE=$(pwd) bash -s: Executes the extracted script in a new shell, mimicking how GHA works by defining the necessary GITHUB_WORKSPACE environment variable and feeding the script in.

This technique is incredibly fast. I can iterate on my docker run command or environment variables until the script runs perfectly, all locally, and only then do I commit and push.

Example of CI step extracted with the singe line of bash above.

docker run
--rm
-u vscode
-e GITHUB_WORKSPACE=$GITHUB_WORKSPACE
-v $GITHUB_WORKSPACE:$GITHUB_WORKSPACE
ghcr.io/amiorin/big-container:latest
fish -c 'cd $GITHUB_WORKSPACE
&& bb build
&& direnv allow
&& direnv exec . just test'

The Performance Trade-off and the Ultimate Fix

My strategy has a performance “tax”:

5-10 minutes to free disk space.
5-10 minutes to download the 18 GB devcontainer image.

This is the unavoidable cost of a standardized, containerized environment on GitHub’s shared runners. If this 10-20 minute setup time is simply unacceptable for your project, there is one simple, permanent solution:

Use a Self-Hosted Runner

By using a self-hosted runner, your CI build machine is always ready. Your container is already pulled, and disk space is managed. Your CI job, once triggered, can jump straight to running tests in seconds, avoiding the entire pain of figuring out GitHub’s official caching architecture.

Conclusion

GitHub Actions is a powerful tool, but its debugging experience can be maddening. By adopting a local-first, Docker-centric approach and reducing GHA to a simple execution engine, you can dramatically accelerate your CI development loop and reclaim your productivity.

Think locally, test locally, and only then commit globally. It’s the fastest path to a green build.

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

Control planes in BigConfig

Wed, 29 Oct 2025 00:00:00 GMT

A control plane acts as an API for provisioning and managing infrastructure. Well-known examples include AWS itself and Kubernetes (K8s).

A common pain point when using infrastructure-as-code (IaC) tools like Terraform (using HCL) and GitOps is the difficulty in transitioning or upgrading to an internal control plane. Teams that have invested significant time in writing Terraform’s HCL code traditionally face a complete rewrite to build a custom control plane API over their existing infrastructure definitions.

Transforming Terraform into a Control Plane

This limitation is no longer the case with BigConfig. We’ve demonstrated that a functional control plane can be created with as little as 200 lines of code, leveraging existing Terraform configurations.

Example Scenario: Secure AWS IAM User Management

Consider the need to create a control plane for managing AWS users. Directly exposing the comprehensive IAM API to engineers is a security risk and the GitOps approach is not scaling. Instead, a minimal API is required that allows engineers to only create, delete, and rename users.

The Key to Bridging the Gap

The secret to transforming any Terraform code into a control plane lies in how Terraform manages resource identifiers:

Terraform HCL Specification: Resources are defined with the syntax resource "<TYPE>" "<LABEL>". The <LABEL> is a static, user-defined identifier that acts as a primary key within the configuration file.
Terraform State: The Terraform state file maintains a crucial mapping between this static HCL <LABEL> and the actual primary key used by the cloud provider (e.g., the ARN in AWS).
Control Plane Requirement: In a control plane, the list of managed resources is dynamic, not static like in an HCL file. To function correctly, the equivalent of the Terraform <LABEL> must be persisted between invocations and generated automatically by the control plane logic, effectively mimicking the role of the HCL file’s label but in a dynamic context.

BigConfig addresses this by programmatically managing this dynamic labeling and state mapping, enabling the existing Terraform code to be driven by a new, restrictive, and secure API—the control plane.

Try the Control Plane CLI Locally

You can test this powerful concept right now, even without an AWS account:

Install the Control Plane CLI:

clojure -Ttools install big-config/control-plane '{:git/sha "aff7c186999c19ad2b4f07752abd13ec5b3b6651" :git/url "https://github.com/amiorin/big-config.git"}' :as ctlp

Create a convenient alias:
```
alias p="clojure -Tctlp"
```
Show the help
```
p help
```
Define a new user via the API (CLI) :
```
p create-user :name '"alice"'
```

Like any control plane, it maintains a desired state (saved locally). Let’s inspect the Control Plane State (not the Terraform state):

p print-state

The output shows how the user is now managed by a random UUID that is persisted across commands:

{:region "eu-west-1",
 :aws-account-id "251213589273",
 :module "users",
 :users {#uuid "ee39e089-9697-4db9-80d9-c931af720535" "alice"},
 :target-dir "dist"}

Finally, apply this desired state to your infrastructure using standard Terraform/OpenTofu commands:

# Run tofu plan
p diff-state

# Run tofu apply
p apply-state

This dynamic state management is powered by the Prevalent System design pattern and uses pure functions to handle state logic. You can explore the implementation details in the GitHub repository.

Show me the code

Conclusion

Terraform remains the go-to tool for modern DevOps and infrastructure management. While some platform teams have attempted to replace it with control planes, this demonstration showed the true strength of Terraform: its ability to abstract the low-level cloud provider APIs. This capability actually makes control plane implementation faster and more efficient rather than competing with it.

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

Configuration Hell? How BigConfig Tames the Modern Dev Environment

Mon, 20 Oct 2025 00:00:00 GMT

Setting up a local development environment today is rarely a trivial matter. The days of simply git clone and npm install are long gone. Modern architectures, particularly those embracing microservices, polyglot persistence, and cloud-native practices, have turned the humble setup process into a multi-layered nightmare.

If you’ve ever spent an afternoon debugging why your local database port clashes with your integration environment, or wrestled with five different tools requiring three different credential formats, you know the pain.

Let’s dive into a concrete example — a complex but typical setup — and see how BigConfig transforms this chaos into an automated, zero-cost development experience.

The Configuration Challenge: A Deep Dive into Rama JDBC

In our case, we’re configuring the rama-jdbc development environment. Rama JDBC implements the robust Outbox Pattern using SQL triggers. This already introduces a host of configuration demands:

Cloud-Native Credentials: Connecting to our staging AWS RDS instance to introspect and replicate the target schema requires fetching credentials securely from AWS Secrets Manager.
Polyglot Tooling: Our system is a mix of languages and utilities:
- Backend services are in Go, using sql-migrate for database migrations.
- Automation and utility scripts are implemented using Babashka and the Just task runner.
Local Database Mirroring: To ensure parity, we need a local SQL database. We use Process Compose to manage its lifecycle, which needs distinct configurations for dev and test.
SQL UI Pain Points: For schema inspection, we use the DuckDB UI. Critically, DuckDB only supports connection via an initialization SQL file, not environment variables.
Conflicting Formats: We have tools that require a simple host and port, while others demand a full JDBC URL.

The Dynamic Port Nightmare

The configuration complexity peaks when dealing with TCP ports. To manage multiple concurrent development contexts (e.g., a Dev feature branch and a Test branch), we use Git Worktrees.

This means ports cannot be the default ones; they must be dynamic but deterministic, dependent on:

The service name.
The path of the current working directory.

In total, this setup requires managing five dynamic ports:

SQL Server (Dev & Test instances)
Process Compose (Dev & Test instances)
DuckDB UI

Manually managing and passing these five dynamic ports and all the various configuration formats (JDBC URL, host/port, SQL init file) across Go, Babashka, Just, DuckDB, and the main Clojure application is a recipe for errors and lost development time.

The BigConfig Solution: configuration as code

This is where BigConfig steps in. By leveraging a single, centralized configuration system written in Clojure, we can define the logic for all these variables, ensuring consistency across every tool.

BigConfig allows us to define a master configuration where:

Port Calculation Logic is Centralized: A single Clojure function can take the (service-name path) as input and deterministically calculate the correct TCP port for the current worktree.
Format Transformation is Automated: The same central configuration can:
- Calculate the host and port.
- Automatically assemble the JDBC URL for Clojure tools.
- Generate the specific SQL initialization file required by DuckDB.
Secure Credentials Injection: Logic to securely fetch credentials from AWS Secrets Manager is handled once, and the resulting secrets are injected into the necessary configurations for both the Go migrations and the local server.

Zero-Cost Automation

The biggest win is the developer experience. The entire end-to-end setup, which was a “nightmare” of manual steps, is now reduced to a single, idempotent command:

bb build

The Babashka task runner uses BigConfig to coordinate everything: calculating ports, generating configuration files, and ensuring full parity between the local development container and the GitHub CI Runner. The “build” step becomes a zero-cost operation that simply ensures your environment is perfectly configured and ready to go.

The REPL Advantage: Speed at Scale

Ultimately, development productivity is measured by the speed of the feedback loop. Even with a complex, port-heavy environment, the Clojure REPL workflow remains lightning-fast.

Most of the time, development is spent hot-reloading code:

Change code → Evaluate expression → Inspect result. This takes milliseconds.

Crucially, when a database change is necessary, BigConfig’s automation keeps the environment responsive:

Need a new migration → Rebuild the database. This, too, takes milliseconds, thanks to the deterministic and automated environment setup.

BigConfig doesn’t just manage complexity; it preserves the core advantage of Clojure — the immediate, fluid feedback loop of the REPL — even when faced with the configuration hurdles of a complex, modern, polyglot application. It allows you to focus on the code, not the configuration.

Conclusion

You can verify the developer experience described here. Just clone rama-jdbc inside big-container .

docker run -it --rm ghcr.io/amiorin/big-container
git clone https://github.com/amiorin/rama-jdbc workspaces/rama-jdbc
cd workspaces/rama-jdbc
# First time will take minutes and devenv progress is not showed in the terminal.
bb build
# Second time will take milliseconds
time bb build
# Start postgres and run the migrations for the dev environment
just pc-dev &
# Run the test for the test environment
clojure -M:shared:test

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

A New Approach to Dotfiles management with BigConfig

Tue, 30 Sep 2025 00:00:00 GMT

Managing dotfiles—the configuration files that personalize your user environment—is a crucial part of a developer’s workflow. The go-to tools for this have long been Chezmoi and Stow . While Stow is celebrated for its simplicity, Chezmoi offers powerful templating and secret management. However, what if you need the best of both worlds? This is where BigConfig comes in, offering a new way to manage your configurations by combining the simplicity of a declarative approach with the power of code.

The Developer Experience

The developer experience with BigConfig is centered around Babashka tasks, making it feel like a standard Clojure project. You get a clear set of commands:

bb install -p [macos|ubuntu]
bb diff -p [macos|ubuntu]
bb render -p [macos|ubuntu|all]

The diff command is particularly useful, allowing you to compare your current dotfiles against the rendered versions before installing them. This gives you a clear, human-readable way to see exactly what’s about to change.

By keeping the code and the dotfiles in the same repository, BigConfig provides a cohesive and powerful developer experience.

The Problem with the Old Ways

Like many developers, I use both macOS and Ubuntu, and the dotfiles for these environments are similar but not identical. Some configurations only exist on one platform, while others need slight tweaks. Stow, while easy for symlinking, lacks the flexibility for this kind of conditional logic. Chezmoi, on the other hand, embeds its logic within the filename, which can become unwieldy and less transparent as your needs grow.

This complexity led me to seek a solution that treats dotfile management as an automation problem—one that’s best solved with code.

Introducing BigConfig

BigConfig takes a different approach. Instead of embedding logic in filenames, it uses a data structure to define the rendering pipeline for your dotfiles. This means your configuration files are kept clean, and the logic for how they are applied is externalized in a dedicated file.

The core of this system is a two-stage rendering process:

Stage 1: Common dotfiles are merged with platform-specific ones (e.g., common and macos are merged into resources/stage-2/macos). At this stage, secrets and tokens are not yet resolved.
Stage 2: The merged files from Stage 1 are rendered, and all secrets and environment variables are resolved, creating the final configuration in a dist/ directory. This is the directory used for installation and comparison.

This structure allows you to maintain a clean separation of concerns: your source files (resources/stage-1) are never committed with secrets, and the final rendered output (dist/) is never committed at all. Your private information is stored securely in an .envrc.private file, which is kept out of your Git repository.

A Look at the Code

BigConfig’s rendering logic is defined in a Clojure data structure. Here’s a snippet that shows how this two-step process is defined:

[{:template "stage-1"
  :target-dir (format "resources/stage-2/%s" profile)
  :overwrite :delete
  :transform [["common"
               :raw]
              ["{{ profile }}"
               :raw]]}
 {:template "stage-2"
  :target-dir dir
  :overwrite :delete
  :transform [["{{ profile }}"]]}]

This data structure is a clear “recipe” for how your dotfiles should be built. It tells BigConfig:

:template: The source directory for the files.
:target-dir: The destination directory.
:overwrite :delete: Ensure the target is clean before rendering.
:transform: The core logic for copying and rendering files. For example, ["common" :raw] copies the contents of the common directory without treating them as templates, while ["{{ profile }}"] copies the contents of the macos or ubuntu directory and treats the files within as templates, resolving variables and secrets.

This declarative approach makes the process transparent and easy to debug.

Adopting BigConfig for your dotfiles

If you want to adopt BigConfig for your dotfiles, just follow this tutorial or have a look at mine

It might seem excessive to learn Clojure and a tool like BigConfig just to manage dotfiles, but the real value comes from applying that investment across a broader range of automation tasks. Clojure’s strengths—its functional nature, immutability, and powerful data manipulation capabilities—make it a strong choice for configuration-as-code. By using a single, cohesive language, you can unify your automation stack and create a more maintainable, expressive system.

Why Clojure for Configuration-as-Code?

Clojure’s core design principles make it an excellent fit for complex configuration tasks. Unlike static languages or rigid data formats like YAML or JSON, Clojure’s Lisp-based syntax treats code as data. This allows you to programmatically generate and manipulate configuration files with functions, macros, and conditional logic.

For example, instead of manually copying and pasting large YAML blocks across multiple environments, you could define a single function that takes environment-specific parameters (e.g., development, staging, production) and generates the correct configuration for each. This reduces redundancy and the risk of manual errors.

Expanding Beyond Dotfiles

While managing dotfiles is a great starting point, the true return on investment for learning Clojure and a tool like BigConfig lies in its applicability to other areas of the software development lifecycle.

Local Development Bootstrap

A unified configuration system can automate the setup of a new developer’s machine. Instead of relying on a multi-step, error-prone manual process, you can use Clojure to define a single script that:

Installs necessary dependencies (e.g., Homebrew packages, language runtimes).
Clones required repositories.
Configures local databases, environment variables, and services.
Sets up the development environment, including IDE settings and editor configurations.

CI/CD Pipelines with GitHub Actions

GitHub Actions workflows are defined in YAML, which can become unwieldy and difficult to manage as they grow in complexity. By using a tool that integrates Clojure, you can dynamically generate these workflow files. This allows you to:

Use a single source of truth for your build, test, and deploy steps.
Parameterize workflows to run across different platforms or branches.
Create reusable, composable functions to define common CI/CD patterns, making your pipelines more DRY (Don’t Repeat Yourself).

Infrastructure as Code (IaC) with Terraform and Ansible

The modern approach to managing cloud infrastructure is through code, but traditional IaC tools like Terraform and Ansible have their own configuration languages (HCL and YAML, respectively). While powerful, these languages can lack the full expressiveness of a general-purpose programming language. Using Clojure, you can:

Generate Terraform HCL files: Create complex Terraform configurations for large-scale cloud deployments, such as a Kubernetes cluster, by leveraging Clojure’s data manipulation capabilities.
Create dynamic Ansible playbooks: Instead of a static playbook, you can write Clojure code that generates an Ansible playbook based on dynamic inputs or the state of your infrastructure. This is particularly useful for provisioning environments that vary slightly from one another.

Kubernetes (K8s) Cluster Management

Kubernetes configuration is notoriously verbose, with extensive YAML manifests for deployments, services, and ingresses. By using Clojure as a configuration generator, you can simplify this process by:

Templating YAML manifests: Define a base template in Clojure and generate multiple, consistent Kubernetes manifests from it.
Automating cluster deployments: Use a single script to deploy an entire application stack, from pods and services to persistent volumes and secrets.
Enforcing best practices: Embed validation and sanity checks within your Clojure code to ensure all generated manifests adhere to your organization’s standards before deployment.

Conclusion

Stow and Chezmoi are great, but for those with complex, multi-platform needs, they can fall short. BigConfig doesn’t try to hide the underlying logic; it embraces it. It recognizes that managing dotfiles is an automation task that benefits from explicit, readable code. Just as Astro , a modern static site generator, has gained popularity over tools like Hugo by being more transparent and flexible, BigConfig offers a similar paradigm shift for dotfile management.

Ultimately, your dotfiles are part of your automation workflow. Shouldn’t your tool for managing them be as powerful and flexible as the rest of your toolchain? BigConfig says yes.

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

Why Ansible Still Rules for Your Dev Environment

Thu, 18 Sep 2025 00:00:00 GMT

Back in the day, before Red Hat acquired Ansible, I was using it to provision Cloudera clusters in massive data centers. And let me tell you, its killer feature wasn’t some complex, enterprise-grade capability. It was pure simplicity.

You just needed SSH, and you were ready to go. The feedback loop was in seconds—a refreshing change from the slow, manual processes we were used to. It was a DevOps dream.

Then came Docker. For many use cases, containers were the new king. They offered a more lightweight, portable solution for shipping applications. And for a while, it seemed like Ansible might get relegated to the history books.

But not so fast. While Docker took over for application deployment, Ansible found its true calling: provisioning the remote development environment.

The New Remote Frontier

Remote development has gone mainstream. Whether you’re using a GUI or a terminal, working on a remote machine makes you more productive. It’s not just a trend; it’s a fundamental shift.

Think about it:

No more “it works on my computer!” Everyone’s environment is the same. No more chasing down dependency hell.
No more wasted time. Your environment is always up to date. The days of git pull followed by hours of fixing a broken dev setup are gone.
More resources, less cost. Remote machines can be shared, giving you access to powerful hardware for a fraction of the price.
Easy authentication. With SSH agent forwarding, pulling and pushing changes to your code repositories is seamless.

It’s a developer’s paradise. But there’s a catch…

The Problem with Ansible’s Language

Ansible is fantastic for this, but its configuration language—YAML—has a few pain points:

It’s another language to master. You have to learn the specific syntax and structure, which can feel like a steep climb on top of everything else you already know.
It’s not flexible enough. Manually curating dozens of YAML files—like packages.yml, repos.yml, and ssh-config.yml—can be tedious and error-prone. The more complex your environment, the messier it gets.

Wouldn’t it be great if you could just write code to generate your configurations?

Introducing BigConfig: The Code-First Approach

This is where a tool like BigConfig comes in. Imagine a world where you write a simple script to generate your Ansible inventory and playbook files. No more manual YAML curation. You can leverage the power of a real programming language to create dynamic configurations.

Here’s the secret: JSON is valid YAML.

This simple fact allows us to generate JSON files with a .yml extension. BigConfig can take your code and spit out a perfect, machine-readable Ansible configuration.

The next logical step? Making the provisioning of your remote development environment an API.

You could simply send a request to a service, and it would provision a new, perfect environment for a new team member in minutes. BigConfig can handle this too, turning a manual, file-based process into a programmable, scalable service.

While some tools come and go, others adapt and find their niche. For provisioning remote development environments, Ansible remains a powerhouse. It just needs a little help from the next generation of tools to unleash its full potential.

Show me the code

This project is specific to my setup but it can be forked and adapted to your needs. I use an iMac and two minipc (soyo and firebat) to develop in the terminal using ssh and tailscale so that I can also code when I am on the road with my MacBook Pro. The ansible project provisions multiple users on both minipc. I use Nix and devenv .

Go faster

Imagine that you have to provision hundreds of users per machine. It will take too long but if every host becomes the combination of the user + the host then Ansible will provision these users in parallel because they look like different hosts.

Conclusion

Even in an age dominated by containers and cloud-native solutions, Ansible remains a crucial tool, not for what it once was, but for what it has become. Its core strength—its simplicity—makes it the ideal choice for a new, pervasive use case: provisioning remote development environments.

While Docker excels at application deployment, Ansible found its niche in ensuring developers have consistent, powerful, and reproducible workspaces. This solves the persistent problem of “it works on my machine” and significantly reduces time spent on setup and maintenance. It’s a fundamental shift in how we approach development, making it more efficient and collaborative.

However, Ansible’s YAML configuration language can be cumbersome. The need to manually manage multiple files becomes a bottleneck as environments grow in complexity. This is where a code-first approach, like the one offered by BigConfig, provides a powerful solution. By leveraging the fact that JSON is valid YAML, you can use a real programming language to dynamically generate configurations. This not only makes the process more flexible and less error-prone but also opens the door to treating environment provisioning as an API—a scalable, programmable service that can instantly onboard new team members.

In short, Ansible’s journey is a testament to its adaptability. It has evolved from a tool for provisioning data centers to the cornerstone of modern remote development. Paired with a tool like BigConfig, its simple, powerful core is unlocked, proving that some of the best tools aren’t those that are replaced, but those that find a new purpose.

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

Reimplementing the AWS EKS API with Clojure using BigConfig, Rama, and Pedestal

Wed, 03 Sep 2025 00:00:00 GMT

The world of cloud infrastructure often involves interacting with complex APIs. While services like AWS EKS provide robust management for Kubernetes clusters, there might be scenarios where you need a more tailored or localized control plane. This article will guide you through reimplementing the AWS EKS API using a powerful Clojure stack: Pedestal for the API, BigConfig to wrap Terraform and Ansible in a workflow, and Rama for state and jobs.

Why Reimplement?

Before we dive into the how, let’s consider the why. K8s, Spark, ClickHouse, Postgres, and so on are all good candidates for an in-house software as a service solution. Reimplementing a cloud API might seem counterintuitive, but it can be beneficial for:

Avoiding vendor lock-in: This can be relevant for some companies.
Multi-cloud strategy: You need an EKS-like solution in multiple cloud providers and you need a generic API.
Saas: You have an open source software and the Saas is your source of revenue.
Metal: You cannot use the cloud but you want to provide the same developer experience in your company.
Integration costs: Buying EKS and integrating it with the rest of your infrastructure is not feasible or very expensive. Building an EKS-like solution is cheaper.

Disclaimer: This is a simplified blueprint for educational and experimental purposes. It will not cover the full breadth and complexity of the actual AWS EKS API.

The Clojure Stack

Here’s a quick overview of the tools we’ll be using:

BigConfig: A workflow and a template engine that enables us to have a zero-cost build step before running any devops tool like Terraform or Ansible.
Rama: A distributed stream processing and analytics engine that can also function as a durable, highly concurrent data store. We’ll use Rama to manage our cluster definitions and state.
Pedestal: A comprehensive web framework for Clojure that emphasizes data-driven development and offers excellent support for both synchronous and asynchronous request handling. It will serve as our API gateway.

Let’s imagine the core entities we want to manage: EKS Clusters. For simplicity, we’ll focus on creating and describing clusters.

First principles

Reuse GitOps: Building a single K8s cluster with GitOps should be reuseable. Replacing GitOps with an API should not require to reimplement everything from scratch. The solution should contain a more generalized version of the GitOps one for one cluster.
Declarative when possible: Terraform should be used to create resources instead of the AWS APIs whenever it is possible.

Sequence diagram

Deliverables

Pedestal API: to create and describe clusters.
Rama Module: to store the desired state, and invoke the BigConfig module with the desired state.
BigConfig Module: this is where the heavy lifting is happening:
- Workflow: achieving the desired state will require multiple steps.
- Lock: to be sure that changes are ACID.
- Build: to generate the configuration files for Terraform based on the desired state.
- Apply: to run terraform apply programmatically.

Perks of the solution

Modularity: Every deliverable can be developed in parallel by adopting contracts.
Uniformity: BigConfig, Rama, and Pedestal deliverables are all written in Clojure.
Declarative: Creating an EC2 instance programmatically can be done faster with Terraform and we don’t need to worry about the life cycle management.
Reusability: The GitOps code can be reused. The code to provision one K8s cluster with GitOps or multiple K8s clusters with an API, doesn’t require to change from Terraform to the AWS SDK. The API is just a virtual admin. The GitOps version developed interactively by an admin can be package as a Clojure dependency and reused inside Rama. This is a killer feature of BigConfig.

The code

A first version that uses the Prevalent System design pattern instead of Rama and the CLI instead of Pedestal is available. Instead of EKS, the proof of concept create, delete, and rename AWS users.

Show me the code

Further Enhancements

This is a basic example, but you can extend it significantly:

More EKS Features: Implement more aspects of the EKS API, such as node groups, Fargate profiles, or update operations.
Authentication and Authorization: Integrate with a robust authentication system to secure your API.
Error Handling: Implement more sophisticated error handling and meaningful error messages by adopting OpenTelemetry.

Conclusion

By combining BigConfig, Rama, and Pedestal, we’ve built a foundation for a in-house EKS-like API in Clojure. This approach provides a high degree of control, flexibility, and the ability to tailor your infrastructure management precisely to your needs. This project serves as an excellent starting point for exploring the potential of building custom cloud-native services with Clojure.

Would you like to have a follow-up on this topic? What are your thoughts? I’d love to hear your experiences.

The killer feature of BigConfig

Mon, 01 Sep 2025 00:00:00 GMT

For anyone working with Infrastructure as Code (IaC), managing configurations and deployments efficiently is key. Engineers are constantly seeking ways to enhance their workflows. Today, we’re diving into a powerful combination: OpenTofu and BigConfig , highlighting a killer feature that makes your build step practically invisible!

The Challenge of Configuration Management

IaC tools like OpenTofu (an open-source alternative to Terraform) empower teams to define, provision, and manage infrastructure through code. However, as projects scale, especially in complex environments, the build and deployment process can become a multi-step chore. This often involves:

Git checks: Ensuring your working directory is clean and up-to-date.
Lock acquire: Making sure that changes are apply in order and incrementally.
Execution: Iterate on the infracoding until it works.
Git pushes: Committing changes back to your repository if the change is successful
Environment-specific deployments: Handling different configurations for different environments like staging and production.

This manual orchestration can be time-consuming and prone to errors.

Enter BigConfig: Simplifying Complex Workflows

BigConfig is a fantastic tool designed to encapsulate and automate these complex command sequences. It allows you to define a series of steps and execute them with a single command. Think of it as a smart wrapper for your common IaC operations. By centralizing these tasks, BigConfig significantly reduces cognitive load and improves consistency.

The Killer Feature: An Invisible Build Step with a Shell Alias

Here’s where the magic truly happens! By combining OpenTofu, BigConfig, and a simple shell alias, we can create an invisible build step. Imagine replacing a series of manual operations with just one, familiar invocation.

Consider this powerful shell alias:

alias tofu="bb render git-check lock exec git-push unlock-any -- alpha prod tofu"

Let’s break down what this alias does:

alias tofu="...": This redefines your tofu command for the session. Now, whenever you type tofu, it executes BigConfig instead of tofu. Every step of the workflow is executed only if the previous step is successful.
render: This is the BigConfig step to initiate a build process and achieve DRY like Atmos. If this fails, there is not reason to proceed. That’s why this step is always present and it is the first step.
git-check: render should not make the Git working directory dirty and git-check ensures your Git working directory is clean and up to date.
lock: It then acquires a lock for the module alpha and the profile prod, preventing concurrent changes from other developers.
exec: This is the core execution step, where BigConfig will run your OpenTofu commands. In case of failure of exec, the workflow will stop and the next steps will not be executed. In particular the lock will not be released and the changes will not be pushed so that the developer can fix or revert the change.
git-push: This automatically pushes the just applied change to your Git repository. You should be always one commit ahead of origin when you make changes.
unlock-any: This ensures that any locks are released. any means that the owner is ignored. This step can be used alone if another developer forget to release the lock.
-- alpha prod tofu: -- is the separator between the workflow definition and module, profile, and the shell command, in this case tofu.

How This Transforms Your Workflow

A simple alias is now extending the capabilities of OpenTofu. Now OpenTofu has the capabilities of Atlantis and Atmos. But BigConfig is not specific to OpenTofu and it can be used also with Ansible, K8s, and your dotfiles.

Before: OpenTofu was not enough and other tools were required like Atlantis and Atmos.

After: Any DevOps tool can be augmented to have the capabilities of Atlantis and Atmos.

The sequential workflow, with all its checks, locks, and pushes, becomes completely invisible to the user. You interact with OpenTofu as you normally would, but all the surrounding boilerplate is handled automatically by BigConfig.

Benefits for Your Team

Increased Productivity: Engineers can focus on writing IaC, not on the deployment mechanics.
Reduced Errors: Automated checks and consistent execution minimize human error.
Standardized Deployments: Ensures that every deployment follows the same robust process.
Faster Onboarding: New team members can quickly get up to speed without memorizing complex sequences.

Get Started!

If you’re using OpenTofu and looking to streamline your IaC workflows, exploring BigConfig and implementing a similar shell alias is highly recommended. It’s a small change that yields massive benefits, transforming your change process from a visible chore into an invisible, seamless part of your development process. Happy infrastructure building! 🚀

Are you still using Atlantis or Atmos? What are your thoughts? I’d love to hear your experiences.

BigConfig | Blog

The Evolution of DevOps: From Separation by Technology to Separation by Concerns

The Modern DevOps Bottleneck

Enter BigConfig: The Component-Based Approach

The Coupling Problem: A Practical Example

Modularity and Referential Transparency

Conclusion

Vibe Coding Meets Vibe Ops: Automating the Last Mile of Deployment

Zero-Friction Infrastructure

1. Compute

2. DNS

3. SMTP

The Vision: Dev & Ops in Plain English

Current Manual Hurdles

Conclusion

The Power of Framing: Why BigConfig is Rebranding as a Package Manager

The Language Barrier (and the Loophole)

The “Package Manager” Epiphany

How BigConfig Differs

Limitless Abstraction

The Roadmap

Conclusion

Simple over easy for operations

Wiring the Engine

Workflow Example

Debugging Made Simple

Solving the Composability Problem: Nested Options

The Working Directory and the Maven Diamond Problem

Conclusion: Simple over Easy

Composability: Orchestrating Infrastructure with Babashka and BigConfig Package

Getting Started

Configuration

Key Components

Parameters and Composability

Uniformity across workflows

Cross-Package Integration

Conclusion

Universal Infrastructure: Solving the Portability Gap with BigConfig

The Portability Challenge

Gluing Infrastructure to Configuration

Handling Distribution Differences

Conclusion

The YAML Trap: Escaping Greenspun’s Tenth Rule with BigConfig

What is Greenspun’s Tenth Rule?

The Rule in the DevOps Ecosystem

1. The YAML “Programming” Trap

2. Kubernetes as a Distributed Lisp

Escaping the Trap: Putting Lisp Back in Ops

Why it Matters: The Power of Assimilation

From Static Files to Fractal Architecture

Ready to stop writing logic in YAML?

Conclusion

Introducing BigConfig Package

The Case Against the “CLI Proliferation”

The Three Environments: Shell, REPL, and Lib

What is a BigConfig Package?

Comparison: Helm vs. Babashka

The Helm Way

The Babashka Way

Conclusion

My Keyboard Endgame: 34 Keys, 3 Layers, and a Macropad

The Hardware Philosophy

The Three-Layer Rule

Taming the “Necessary Evils”

The Terminal-Centric Stack

My skhd Configuration

Final Thoughts

Replacing Integrant and Docker Compose with BigConfig System

The “Emacs” Philosophy

Solving the Postgres Dependency

Abandoning the “Refresh” Cycle

The Workflow in Action

Is it better than Integrant and Docker Compose?

Links

Conclusion

Beyond the Shell: Reimagining DevOps with the Clojure REPL

The Problem: The “Edit/Run” Loop

The BigConfig Alternative: “Edit/Evaluate”

Workflow: A New Type of Flow Control

Real-World Application: Developing Terraform Providers