technical Featured 5 min read

Lessons from Organizing 6 AI Agents: Why Single Source of Truth Matters

How we went from chaotic agent workspaces consuming 95% disk space to a clean, audited directory structure that scales. Real lessons from managing multi-agent AI teams.

#AI Agents #DevOps #Infrastructure #Lessons Learned #OpenClaw #Team Management

When you’re running six AI agents (the orchestrator, the programming agent, the marketing agent, the finance agent, the documentation agent, and the autonomous coding agent) as a real development team, you quickly discover that AI agents are digital hoarders.

Our disk hit 95% capacity. Project repositories were duplicated across every agent’s workspace—some with creative suffixes like -clean, -backup, and -v2. Databases lived in random directories. Scripts were scattered everywhere. Agents couldn’t find each other’s work, and frankly, neither could I.

We needed a fundamental redesign. Here’s what we learned building a multi-agent directory structure that actually scales.

The Problem: Chaos at Scale

Picture this: You’ve got an AI agent that needs to “quickly check something” in a repository. It clones the repo to its workspace, does the work, and… never cleans up. Multiply that by six agents across dozens of projects, and you get digital chaos.

The symptoms were clear:

  • 95% disk utilization (19+ GB of duplicate repositories)
  • /tmp accumulated 6.1 GB of agent debris (CDK directories, Playwright profiles, full project clones)
  • No single source of truth for any project
  • Agents working with stale data because they didn’t know where the “real” files lived
  • My morning routine included manually finding which version of which repo actually had the latest changes

Sound familiar? If you’re building AI agent teams, you’ll hit this wall faster than you think.

The Solution: ~/projects/ as Single Source of Truth

We designed a standardized directory structure that every agent must follow:

~/projects/<project>/
  repos/          — authoritative git checkouts ONLY
  repos/archives/ — repos needing reconciliation  
  content/        — blog posts, marketing, docs
  notes/          — research, meeting notes, specs
  images/         — logos, screenshots, assets
  specs/          — technical specifications
  work-log/       — daily work logs (YYYY-MM-DD.md)
  scripts/        — project-specific utilities
  env/            — environment configs

Plus shared infrastructure directories:

  • ~/run/ — production deployments (live systems)
  • ~/databases/ — all SQLite databases
  • ~/scripts/ — cross-project utilities
  • ~/backups/ — database backups, config snapshots
  • ~/.openclaw/workspace-<agent>/ — agent-personal files ONLY

Key Design Decisions That Made the Difference

1. One Authoritative Repo Per Project

No more repositories with suffixes. One project, one repo, one location. If an agent needs to work on a project, it uses the canonical checkout in ~/projects/<project>/repos/. Period.

This was harder to enforce than expected. Agents are pattern-matching creatures—they see a successful workflow (clone repo, do work) and repeat it everywhere, even when it doesn’t make sense.

2. Agents Move Their Own Files

We didn’t play janitor. Each agent was responsible for migrating their own content to the new structure. This forced them to learn the directory layout by doing, not just reading documentation.

The marketing agent moved marketing content from workspace to ~/projects/<project>/content/. The programming agent consolidated repositories from scattered clones to canonical locations. The finance agent moved financial models to appropriate project directories.

Teaching moment: Agents that migrate their own files remember where things belong.

3. Nightly Audit Teaches, Doesn’t Fix

Our audit script runs at 2:30 AM every night. When it finds violations (repos in workspaces, files in wrong locations), it doesn’t fix them automatically. Instead, it sends correction messages to the violating agent explaining WHY the placement is wrong.

Example audit message:

“Found repository project1-mobile in workspace-programming. Repositories belong in ~/projects/project1/repos/ for team access and backup coverage. Please move and update your workflows.”

The “teach don’t janitor” pattern is incredibly powerful — agents that fix their own mistakes remember the rule.

4. trash-put Over rm (With a Gotcha)

We mandated trash-put instead of rm for safety. But here’s the lesson: trash-put doesn’t free disk space until you empty the trash. We spent hours wondering why our cleanup efforts weren’t showing up in df -h.

Sometimes the simple rules have complex implications.

Implementation Reality Check

Migrating 11 active projects across six agent workspaces was… educational.

  • The programming agent consolidated 13 authoritative repositories from dozens of scattered clones
  • The marketing agent moved 76 blog posts from workspace staging directories to proper content locations
  • 19+ GB reclaimed through systematic cleanup of duplicates, stale builds, and orphaned node_modules

The disk utilization dropped from 95% to ~76%. More importantly, every agent now knows exactly where to find project files.

The Audit Loop: Continuous Compliance

Our nightly audit is the secret sauce. It checks:

  • Are repositories only in ~/projects/<project>/repos/?
  • Are agent workspaces clean of project content?
  • Are temp files older than 24 hours cleaned up?
  • Do all projects have required directories (content/, notes/, etc.)?

The audit teaches rather than fixes. When it finds violations, it generates personalized correction messages explaining the proper structure and why it matters.

The orchestrator agent’s morning report includes system health from the audit, so I know immediately if agents are drifting from the standard.

Template-Driven Consistency

New projects get the full directory structure automatically via template. No agent has to remember what directories a project needs—the template ensures consistency from day one.

cp -r ~/projects/templates/project-template ~/projects/new-project/

Simple, but it eliminates “where does this go?” decisions.

Lessons Learned

AI agents are digital hoarders. They clone repositories “just to check something” and never clean up. They accumulate temp files, cache directories, and build artifacts without thinking about disk space.

Policies must be explicit AND enforced. Saying “don’t put repos in your workspace” means nothing without an audit catching violations. Rules without enforcement become suggestions.

The “teach don’t janitor” pattern scales. Agents that fix their own mistakes remember the rule. Auto-fixing violations teaches nothing and creates dependency.

Context matters more than storage. Agents care less about “where files live” and more about “can I find what I need?” The directory structure is just infrastructure—the real value is predictable locations for predictable content.

What’s Next

We’re expanding the audit to catch more edge cases (projects without proper README files, missing environment configs). The template is getting smarter based on observed project patterns.

But the core lesson remains: Multi-agent teams need structure that’s enforced, not just documented.

If you’re building AI agent infrastructure, start with directory structure before you worry about communication protocols or task distribution. Chaos is expensive, and storage is cheap, but organization is priceless.

Your future self (and your disk usage monitor) will thank you.


Want to see more technical lessons from building multi-agent AI teams? Check out OpenClaw, the platform that makes this all possible.

Published by Stuart Bain

Share:

Related Posts

technical

The Move to Jekyll

Explaining the migration from WordPress to Jekyll static site generation during the 2020 COVID-19 pandemic, and the benefits of simplifying web hosting.

Read More
technical

Nagging Numbers

A practical exploration of alphabetical vs. numerical sorting problems, illustrated through auction industry lot numbering - and how to solve them with smart planning.

Read More