Codemie Infrastructure as Code

Infrastructure as Code solution for managing Codemie AI assistants, datasources, and workflows through declarative YAML configuration.

🎯 What This Solves

Key Problems

1. Access Control Limitations 🔒
In Codemie UI, only resource creators can modify assistants and workflows. Team members cannot contribute improvements even when they have ideas.

2. Limited Execution History Visibility 👁️
Only the creator can see execution history of assistants and workflows - even administrators lack access. This makes debugging and monitoring difficult for the team.

3. No Custom Validation ✅
Changes go live immediately without additional checks. There's no way to enforce team-specific validation rules or quality gates before deployment.

What IaC Enables

Democratic Access
Anyone can propose changes via pull requests. Changes are reviewed and deployed by a service account with full access to all resources and their execution history.

Custom Validation Pipeline
Add your own validation rules in CI/CD - from simple YAML checks to AI-powered prompt reviews. Nothing deploys until it passes your criteria.

Team Visibility
Service account has access to execution history of all assistants and workflows, enabling team-wide monitoring, debugging, and analytics.

Additional Benefits

• Code Review Process - Changes reviewed before going live, improving quality
• DRY Configuration - $ref mechanism eliminates duplication (define once, use everywhere)
• Automated Deployment - CI/CD integration removes manual steps and human errors
• Audit Trail - Git history shows who changed what and why
• Environment Consistency - Same configuration can be deployed to dev/staging/prod or can be used by other teams

Overview

This IaC solution enables version-controlled, reproducible deployment of Codemie resources using a GitOps approach. Changes are detected via SHA256 checksums, and resources are managed by their human-readable names.

Installation

npm install

Environment Variables

Create a .env file in the project root:

CODEMIE_API_URL=https://your-instance.codemie.ai/api
CODEMIE_AUTH_URL=https://your-keycloak.auth.com
CODEMIE_REALM=codemie
CODEMIE_CLIENT_ID=your-client-id
CODEMIE_CLIENT_SECRET=your-client-secret

⚠️ Security: DO NOT commit .env to Git!

The .env file is already added to .gitignore and should never be committed to the repository.

For local development:

• Each developer creates their own .env file locally with actual credentials
• Copy the template above and replace with your values
• Use your own credentials (do not share them via Git)
• Optional: Create .env.example in your project (template without actual secrets) to help team members

For CI/CD (GitLab):

• Add environment variables in GitLab: Settings → CI/CD → Variables
• Add each variable separately:
  • CODEMIE_API_URL
  • CODEMIE_AUTH_URL
  • CODEMIE_REALM
  • CODEMIE_CLIENT_ID
  • CODEMIE_CLIENT_SECRET (mark as Masked)

Substitution in codemie.yaml:

• ${VAR} and ${VAR:-default} are substituted in every string value of the configuration, including values nested inside lists such as resources.skills or resources.datasources.
• A ${VAR} without a default whose variable is not set fails the load with an error naming the configuration path, for example resources.datasources[0].name. Use ${VAR:-default} where a value is optional, or ${VAR:-} to substitute an empty string.
• There is no escape syntax: every ${...} occurrence in a string value is substituted, so a configuration cannot carry a literal ${VAR} through the loader.

Quick Start

# Initialize a new Codemie IaC project (creates .env and codemie.yaml via wizard)
codemie init

# Full validation with API connectivity check (default, recommended)
npm run validate

# Basic validation only (fast, no API connection)
npm run validate:basic

# Preview changes
npm run preview

# Deploy resources
npm run deploy

# Deploy resources and delete orphaned ones
npm run deploy -- --prune

# Remove all IaC-managed resources
npm run destroy

# Create backup of all resources
npm run backup

# Use external config file to override default names and locations for: codemie.yaml, .codemie/state.json, backups
npm run <command>  --config codemie/iac.json

Note: All commands work identically whether you use monolithic configuration or modular imports - the $import directives are automatically resolved during config loading.

Project Initialization (codemie init)

Use the interactive wizard to scaffold a project:

codemie init

What it does:

• Creates a .env template (with a safety prompt if it already exists)
• Generates a minimal codemie.yaml using your project name and description
• Saves an optional appConfig file when you pick non-default paths for config/state/backups
• Prints next steps to validate and deploy

If .env or codemie.yaml already exist, the wizard asks before overwriting and will keep your files if you decline.

Config file structure:

{
  "rootDir": "root/of/working/directory",
  "codemieConfig": "path/to/codemie.yaml",
  "codemieState": "path/to/state.json",
  "backupsDirectory": "path/to/backups"
}

It is not mandatory to provide all fields - any missing fields will use default values.

Default Config:

{
  "rootDir": ".",
  "codemieConfig": "codemie.yaml",
  "codemieState": ".codemie/state.json",
  "backupsDirectory": "backups"
}

Validation types:

• Full validation (npm run validate) checks:

  • codemie.yaml syntax is valid
  • All referenced files exist
  • Required fields are present
  • No cyclic dependencies in sub-assistants
  • Assistants with sub-assistants cannot be sub-assistants themselves
  • Plus API checks:
    • Slug conflicts with existing assistants in Codemie (if slugs are provided)
    • Ensures your slugs are globally unique before deployment

• Basic validation (npm run validate:basic) skips API checks (faster, no credentials needed)

CLI Usage

The tool provides a unified codemie command for all operations:

# Show help and available commands
codemie --help

# Deploy resources
codemie deploy
codemie deploy --prune
codemie deploy --skip-datasources

# Other commands
codemie backup
codemie destroy --force
codemie preview
codemie validate --check-api

# Import a single resource from the Codemie platform
codemie import <type> <slug>
codemie import assistant my-assistant
codemie import workflow 11111111-1111-1111-1111-111111111111
codemie import assistant my-assistant --update  # refresh an already-managed resource in place
codemie import assistant my-assistant --with-skills  # also import the project-owned skills it references
# <type> is one of: assistant, datasource, workflow, skill
# <slug> is a resource slug or name; for assistant and workflow it may also be a platform UUID

# Run a command from a subdirectory of the project, or against another project entirely
codemie validate --root-dir ../another-project

--with-skills (assistants only). An imported assistant references its skills by name, so a deploy fails if one of those skills belongs to this project but is not in resources.skills yet. With --with-skills, each referenced skill that the project owns and does not manage yet is imported too - skills/<name>.skill.md plus a resources.skills entry, exactly as codemie import skill would write it - so the assistant deploys with no manual skill import. Skills owned by other projects keep landing in imported.skills as references, and a skill the configuration already names is left untouched (also under --update, which refreshes the assistant only). On any other resource type the flag is ignored with a warning.

Note: Every command finds the project root automatically by walking up from the current directory looking for codemie.yaml, so you can run codemie from any subdirectory of your project. Pass -d, --root-dir <path> to point at a different project root instead. codemie init never auto-detects an existing project - it always scaffolds in the current directory unless -d is given. -c, --config and -d, --root-dir are independent: a -c file wins only where it declares rootDir itself, otherwise an explicit -d still applies.

For local development, npm scripts provide convenient shortcuts:

npm run deploy    # → codemie deploy
npm run backup    # → codemie backup
npm run destroy   # → codemie destroy --force

🔄 Migrating Existing Resources to IaC

If you already have assistants, datasources, and workflows created in Codemie UI, use the backup command to migrate them to IaC management:

npm run backup

What It Does

Automated Export:

• Fetches all resources from your Codemie platform (assistants, datasources, workflows, integrations)
• Generates IaC configuration files in backups/YYYY-MM-DDTHH-MM-SS/ directory:
  • codemie.yaml - Main configuration with all resources (MCP servers use integration aliasing)
  • state.json - State file with resource IDs and checksums
  • system_prompts/*.prompt.md - Assistant system prompts
  • workflows/*.yaml - Workflow definitions
  • openapi_specs/*.json - OpenAPI tool specifications

Transaction Safety:

• Atomic backup process with rollback on failure
• Resume capability - if backup is interrupted, rerunning continues from last checkpoint
• Temporary directory used during backup (.temp-*) - moved to final location only on success
• Progress tracking with transaction.json checkpoint file

Type-Safe Conversions:

• SDK response types → IaC resource formats with proper field mapping
• Workflow YAML includes assistant ID → name resolution for readability

Migration Workflow

Step 1: Create Backup

npm run backup

Output:

🗄️  Creating backup of all Codemie resources...

🔌 Connecting to Codemie API...
✓ Connected to Codemie API

📁 Temporary backup directory: backups/.temp-2025-01-27

🤖 Fetching assistants...
   Found 15 assistant(s)
   • TAD BA Helper (ast-123)
   • TAD Code Analyst (ast-456)
   ...
✓ Backed up 15 assistant(s)

📊 Fetching datasources...
   Found 8 datasource(s)
✓ Backed up 8 datasource(s)

🔄 Fetching workflows...
   Found 3 workflow(s)
✓ Backed up 3 workflow(s)

🔄 Finalizing backup...
✓ Backup finalized at backups/2025-01-27T14-30-45

==================================================
📊 Backup Summary:

  🤖 Assistants: 15
  📊 Datasources: 8
  🔄 Workflows: 3
  🔌 Integrations: 5

  📁 Location: backups/2025-01-27T14-30-45

Step 2: Review Generated Files

cd backups/2025-01-27T14-30-45
ls -la

Files created:

codemie.yaml         # Main IaC configuration
state.json           # Resource state (IDs, checksums)
system_prompts/      # Assistant system prompts
  tad-ba-helper.prompt.md
  tad-code-analyst.prompt.md
workflows/           # Workflow definitions
  code-review-workflow.yaml
openapi_specs/       # OpenAPI tool specs

Step 3: Copy to Your IaC Project

# Option A: Replace entire configuration
cp backups/2025-01-27T14-30-45/* ./
mv backups/2025-01-27T14-30-45/system_prompts/* system_prompts/
mv backups/2025-01-27T14-30-45/workflows/* workflows/
mv backups/2025-01-27T14-30-45/openapi_specs/* openapi_specs/

# Option B: Merge specific resources into existing config
# Manually copy needed sections from backups/2025-01-27T14-30-45/codemie.yaml

Step 4: Validate & Deploy

npm run validate  # Check configuration
npm run preview      # Preview (should show "No changes")
npm run deploy    # Deploy (updates state file with current checksums)

Troubleshooting

Backup Interrupted?

• Just run npm run backup again - it will resume from last checkpoint
• Transaction file (transaction.json) tracks progress
• Temporary directory (.temp-*) will be cleaned up automatically

Backup Failed Completely?

• Check .temp-* directory exists - if yes, partial backup was attempted
• Error message will indicate which resource failed
• Temporary directory will be automatically cleaned up (rollback)
• Fix the issue and rerun backup

Code Datasource Missing Repository Link?

• Some code datasources may show warning: ⚠️ Code datasource missing repository link
• SDK sometimes doesn't return code.link field
• Manually add link: https://github.com/... in codemie.yaml datasource section

Workflow Assistant IDs Not Resolved?

• If workflow YAML contains assistant_id: ast-xxx instead of assistant name
• This happens when assistant doesn't exist in backup (e.g., deleted or from another project)
• Manually replace with correct assistant name or ID in workflow YAML file

All MCP Assistants Show as Changed After an Upgrade?

• After upgrading to the release that normalizes MCP configuration, MCP server configuration is normalized before hashing, so every assistant with mcp_servers gets a new configChecksum once
• preview and deploy list each of them under ⚠️ UPDATE: as - assistant:<name> with Prompt or configuration changed and no per-field diff
• Expected, not a configuration error - deploy stores the new configChecksum in .codemie/state.json and the next run reports them as unchanged

Resume From Specific Point?

• Transaction file tracks what's completed: transaction.json
• Contains: completed: ['ast-1', 'ast-2'] and failed: [{id: 'ast-3', error: '...'}]
• To force fresh backup: delete backups/YYYY-MM-DDTHH-MM-SS/transaction.json before rerunning

📝 Prompt Compilation System

The repository includes an automated prompt compilation system for managing complex prompts with reusable components and configuration.

When to Use Compilation

Use template-based compilation for:

• Prompts with shared components (includes)
• Multiple configuration variants
• Team collaboration on modular components
• Complex prompts that benefit from DRY principles

Use direct editing for:

• Simple, self-contained prompts
• Quick changes or updates
• Prompts that don't need shared components

Quick Start

# Compile specific agent
npm run compile -- --prompt tad-ba-helper

# Compile all prompts
npm run compile:all

# Check if compiled files are up-to-date
npm run compile:check

# Verbose output
npm run compile:all -- --verbose

Note: Compilation functionality is available only as internal npm scripts. The compile command is not exposed in the published codemie-iac package, which focuses exclusively on infrastructure management (deploy, validate, preview, destroy, backup).

Directory Structure

prompt_source/              # Source templates
├── shared/                 # Shared includes
│   ├── jira-processing.include.md
│   └── confluence-processing.include.md
└── {agent-name}/           # Agent directory
    ├── {agent}.prompt.md       # Template with placeholders
    ├── {agent}.config.yaml     # Configuration values
    └── {agent}.requirements.md # Requirements doc

system_prompts/             # Ready-to-deploy prompts
├── README.md               # Documentation
└── *.prompt.md             # Final prompts (some auto-compiled)

Placeholder Types

The compilation system handles three types of placeholders:

1. Config Placeholders - {config.key}

   • Resolved during compilation from .config.yaml
   • Example: {team_ids.prescreening} → 91267

2. Runtime Variables - {{system}}

   • Preserved for Codemie platform (resolved at runtime)
   • Example: {{current_user}}, {{date}}

3. Documentation Examples - <example>

   • Part of prompt text, never resolved
   • Example: <param_name>, <user_id>

Example Template

Source: prompt_source/my-agent/my-agent.prompt.md

---
prompt_name: My Agent
prompt_version: 1.0.0
---

# Instructions

{include:jira-processing}

Use team ID: {team_ids.my_team}
Current user: {{current_user}}

Config: prompt_source/my-agent/my-agent.config.yaml

prompt_version: 1.0.0

team_ids:
  my_team: "12345"

Compiled: system_prompts/my-agent/my-agent.prompt.md

---
prompt_name: My Agent
prompt_version: 1.0.0
---

# Instructions

[Full content of jira-processing.include.md inserted here]

Use team ID: 12345
Current user: {{current_user}}

CI/CD Integration

Compilation is fully automated in the deployment pipeline:

1. Compile Stage - Runs before validation
2. Validation - Checks for unresolved placeholders
3. Deployment - Deploys compiled prompts
4. Auto-Commit - CI commits compiled files back to repository

Workflow:

• Edit source files in prompt_source/
• Push to repository
• CI automatically compiles and deploys
• Compiled files updated in system_prompts/

Extensible Architecture

The compilation system uses a plugin-based architecture for easy extension:

• ProcessorPlugin - Custom content transformations
• ValidatorPlugin - Custom validation rules
• Type-safe - Full TypeScript support
• Well-documented - See ARCHITECTURE.md for details

For complete documentation:

• Source Templates: prompt_source/README.md
• System Prompts: system_prompts/README.md
• Architecture: ARCHITECTURE.md
• Development Guide: CONTRIBUTING.md

Configuration Structure

All resources are defined in codemie.yaml:

version: 1

project:
  name: my-project
  description: Your project description

environment:
  codemie_api_url: ${CODEMIE_API_URL}
  # ... other env vars

imported:
  integrations:    # Platform integrations (referenced via $ref)
  assistants:      # Existing assistants (for workflows)
  datasources:     # Existing datasources

resources:
  datasources:     # Datasources to create/manage
  assistants:      # Assistants to create/manage
  workflows:       # Workflows to create/manage

For detailed examples of each resource type, see:

• examples/datasource-examples.yaml - All datasource types with field descriptions
• examples/assistant-examples.yaml - Assistant patterns and configurations
• examples/toolkit-examples.yaml - All toolkit configurations
• examples/workflow-examples.yaml - Workflow patterns and structures
• examples/skill-examples.yaml - Skill patterns, including expanding a folder of skill directories

📁 Configuration Organization

Modular Configuration with $import

For large projects, you can split configuration into multiple files using the $import directive. The ConfigLoader automatically resolves imports and supports three organization styles:

Style 1: Monolithic

Keep everything in one file - perfect for small projects:

# codemie.yaml
version: 1
project:
  name: my-project

resources:
  datasources:
    - name: repo-1
      type: code
      link: https://github.com/org/repo-1
    - name: repo-2
      type: code
      link: https://github.com/org/repo-2
  
  assistants:
    - name: Assistant 1
      prompt: ./system_prompts/assistant-1.prompt.md
      model: gpt-4o

Style 2: Grouped Resources

Group related resources in array files:

# codemie.yaml
resources:
  datasources:
    - $import: configs/datasources/all-repos.yaml
    - $import: configs/datasources/knowledge-bases.yaml
  
  assistants:
    - $import: configs/assistants/ba-team.yaml
    - $import: configs/assistants/dev-team.yaml

# configs/datasources/all-repos.yaml
- name: repo-1
  type: code
  link: https://github.com/org/repo-1

- name: repo-2
  type: code
  link: https://github.com/org/repo-2

- name: repo-3
  type: code
  link: https://github.com/org/repo-3

# configs/assistants/ba-team.yaml
- name: BA Reviewer
  prompt: ./system_prompts/ba-reviewer.prompt.md
  model: claude-3-7

- name: BA Helper
  prompt: ./system_prompts/ba-helper.prompt.md
  model: gpt-4.1
  sub_assistants:
    - BA Reviewer

Style 3: Individual Files

Each resource in a separate file for maximum modularity:

# codemie.yaml
resources:
  datasources:
    - $import: configs/datasources/repo-1.yaml
    - $import: configs/datasources/repo-2.yaml
    - $import: configs/datasources/jira-kb.yaml
  
  assistants:
    - $import: configs/assistants/ba-reviewer.yaml
    - $import: configs/assistants/ba-helper.yaml
    - $import: configs/assistants/code-reviewer.yaml

# configs/datasources/repo-1.yaml
name: repo-1
type: code
description: Repository 1
link: https://github.com/org/repo-1
branch: main

# configs/assistants/ba-reviewer.yaml
name: BA Reviewer
description: Reviews BA artifacts
prompt: ./system_prompts/ba-reviewer.prompt.md
model: claude-3-7
shared: true
toolkits:
  - toolkit: Project Management
    tools:
      - $ref: tool_definitions.jira_tool

Style 4: Hybrid

Mix all approaches based on your needs:

# codemie.yaml
resources:
  datasources:
    # Inline for quick tests
    - name: test-repo
      type: code
      link: https://github.com/org/test
    
    # Array import for simple repos
    - $import: configs/datasources/simple-repos.yaml
    
    # Individual files for critical resources
    - $import: configs/datasources/production-jira.yaml
    - $import: configs/datasources/production-confluence.yaml
  
  assistants:
    # Team groups
    - $import: configs/assistants/ba-team.yaml
    - $import: configs/assistants/dev-team.yaml
    
    # VIP assistants separately
    - $import: configs/assistants/ceo-assistant.yaml

Import Features

Object Imports:

# Import entire sections
tool_definitions:
  $import: configs/shared/tool-definitions.yaml

datasource_defaults:
  $import: configs/shared/datasource-defaults.yaml

Circular Import Protection:

• Automatically detects circular dependencies (file-a → file-b → file-a)
• Throws clear error with import chain: Circular import detected: [file-a.yaml, file-b.yaml, file-a.yaml]
• Allows same file in different branches (tree structure, not linear chain)

Nested Imports:

# configs/datasources/all-repos.yaml
- $import: base-repos.yaml  # Import from same directory
- name: additional-repo
  type: code
  link: https://github.com/org/additional

Cross-Project Reuse:

# Project A
resources:
  assistants:
    - $import: ../shared-assistants/code-reviewer.yaml
    - $import: configs/assistants/project-a-specific.yaml

# Project B
resources:
  assistants:
    - $import: ../shared-assistants/code-reviewer.yaml
    - $import: configs/assistants/project-b-specific.yaml

Recommended Structure

project-root/
├── codemie.yaml                 # Main config (imports only)
├── configs/
│   ├── project.yaml            # Project metadata
│   ├── environment.yaml        # Environment variables
│   ├── imported.yaml           # Imported resources
│   ├── shared/
│   │   ├── tool-definitions.yaml
│   │   └── datasource-defaults.yaml
│   ├── datasources/
│   │   ├── repos/
│   │   │   ├── frontend-repos.yaml
│   │   │   └── backend-repos.yaml
│   │   └── knowledge-bases/
│   │       ├── jira-production.yaml
│   │       └── confluence-production.yaml
│   ├── assistants/
│   │   ├── ba-team/
│   │   │   ├── ba-reviewer.yaml
│   │   │   └── ba-helper.yaml
│   │   └── dev-team/
│   │       ├── code-reviewer.yaml
│   │       └── codebase-digger.yaml
│   └── workflows/
│       ├── test-automation.yaml
│       └── release-notes.yaml
├── system_prompts/
│   └── *.prompt.md
└── workflows/
    └── *.yaml

🧩 Skills Configuration

Skills are defined under resources.skills. Two shapes are supported, and you can mix both in the same list.

Single-File Skill

The original shape — one entry, one .skill.md file:

resources:
  skills:
    - name: code-review-checklist
      description: Checklist the assistant walks through when reviewing a pull request
      file: ./skills/code-review-checklist.skill.md
      categories: [review, quality]
      visibility: project

• name (required): unique skill name.
• description (required): 10-1000 characters — see "Description length validation" below.
• file (or inline content, see below): at least one of the two is required. A skill expanded from source_dir carries both — file points at its SKILL.md and content holds the body.
• categories (optional): array of category labels.
• visibility (optional): project or public (default: project).

A Folder of Skill Directories (source_dir)

For a directory that already holds one subdirectory per skill (the layout used by skill authoring tools), point source_dir at it instead of listing every skill by hand:

resources:
  skills:
    - source_dir: ./skills/engineering
      ignore_file: ./skills/engineering/.codemie-skills-ignore.yaml
      categories: [engineering]
      visibility: project
      group: engineering-team

skills/engineering/
├── code-review-checklist/
│   └── SKILL.md
├── incident-runbook/
│   └── SKILL.md
└── draft-idea/            # no SKILL.md here → skipped silently
    └── notes.md

# skills/engineering/.codemie-skills-ignore.yaml
skills:
  - draft-idea

Behavior:

• Each immediate subdirectory of source_dir that directly contains a SKILL.md becomes one skill entry. Subdirectories without a SKILL.md are skipped silently. There is no recursion into deeper levels.
• Skills are expanded in sorted order of their subdirectory names, so two loads of the same folder always produce identical results.
• SKILL.md may start with ----delimited YAML front matter (a UTF-8 BOM and CRLF line endings are tolerated). name is taken from the front matter, falling back to the directory name when absent. description is taken from the front matter. Everything after the front matter becomes the skill's content.
• ignore_file (optional) points at a YAML file shaped skills: [<dirName>, ...]; the listed directory names are skipped during expansion.
• categories, visibility and group on the source_dir entry are inherited by every skill expanded from it. group is a label only — it never splits or migrates anything, and it is intentionally excluded from change detection, so relabelling a skill does not mark it as changed. The one command that reads it is deploy --replace-skill-group (see "Replacing a Whole Skill Group" below).
• A source_dir that does not exist is a configuration error naming both the resolved path and the offending source_dir entry. A missing or malformed ignore_file, and malformed SKILL.md front matter, fail the same way — naming the file at fault.
• Skill names must be unique across every source_dir and every hand-written entry. A duplicate is a configuration error naming the name and both sources that produced it.
• Names produced by expansion must be kebab-case, 3-64 characters, starting and ending with a letter or digit. This rule only applies to skills expanded from source_dir — existing hand-written skill names in the single-file shape are unaffected.
• Every other file in a skill's subdirectory ships with it as a companion file — see "Companion Files" below.

Companion Files

A skill whose file points at a SKILL.md — every skill expanded from source_dir, plus any hand-written entry that points at one — ships the rest of its directory too. Every other file beneath that directory, at any depth, becomes a companion file that travels with the skill to the platform:

skills/engineering/code-review-checklist/
├── SKILL.md                  # the skill's own content, never a companion file
├── .codemie-exclude          # optional, never shipped itself
├── checklist.md              # companion file
├── templates/pr-body.md      # companion file (the relative path is preserved)
└── assets/diagram.png        # companion file (binary → shipped base64)

A skill in the single-file shape (file: ./skills/foo.skill.md) or with inline content has no companion files.

Always excluded, whatever patterns you write: SKILL.md and .codemie-exclude themselves, the directories node_modules, .git and evals, and the lockfiles package-lock.json, npm-shrinkwrap.json, yarn.lock, pnpm-lock.yaml, bun.lockb, composer.lock, Gemfile.lock, Cargo.lock and poetry.lock. Symlinks — to files or directories — are skipped rather than followed.

.codemie-exclude is optional and is read only from the skill's own directory (one placed deeper in the tree is not read). Patterns go one per line and are matched against skill-relative POSIX paths:

| Pattern | Matches | |---------|---------| | docs/secret.txt | that exact path | | vendor/lib | vendor/lib itself and everything beneath it | | build/ | everything beneath build/ — a trailing slash marks a directory prefix | | secret.txt | a file named secret.txt at any depth — a pattern without / matches the basename | | *.log | * matches any run of characters except / | | docs/**/notes.md | ** crosses / | | file?.txt | ? is exactly one character, never / | | {foo,bar}.txt | {a,b} is alternation | | # draft/ | nothing — a line whose first non-whitespace character is # is a comment |

Blank lines are ignored, and every other character is literal (the . in file.txt is a dot, not "any character").

Text and binary files are detected automatically. Content that contains no NUL byte and round-trips exactly as UTF-8 is sent as text, byte-identical to the file on disk; anything else — a binary asset, or text in a legacy single-byte encoding — is base64-encoded, so it survives deploy and backup byte-for-byte. The MIME type is derived from the file extension, falling back to application/octet-stream.

Companion files are tracked in state by checksum. .codemie/state.json records one content hash per companion file plus one aggregate checksum — never the file bodies. Adding, editing or removing a companion file therefore marks the skill as changed in preview (which names the files added, removed and modified) and triggers an update in deploy, even when SKILL.md itself is untouched. Removing the last companion file clears the skill's companion files on the platform. When state cannot say what the platform holds — a skill recovered by name after an "already exists" create, or a state entry written before companion files were tracked — the platform's own list is read first, so stale companions are cleared rather than left behind.

backup and import keep the layout. For a skill that has companion files, codemie backup writes skills/<name>/SKILL.md with the companion files beside it under their original relative paths, decoding base64 ones back to bytes; a skill with no companion files keeps the flat skills/<name>.skill.md layout (content files are written for project-owned skills only). codemie import fetches a skill's companion files as well and writes those same two layouts, so a backed-up or imported skill redeploys with its companions intact.

A failed companion lookup never touches the platform's files. When that lookup cannot be answered — a timeout, a server error, or a platform without the companion-files endpoint — deploy warns, naming the skill and the reason, and updates the skill without the companion_files field, leaving whatever the platform holds untouched instead of failing the skill.

Only the SKILL.md layout manages companion files. If a managed entry points at a flat skills/<name>.skill.md while the platform skill has companion files, codemie import --update refreshes the content file, warns, and deliberately records no companion hashes in state: those files stay on the platform, unmanaged locally. Point the entry at a skills/<name>/SKILL.md layout to manage them.

Description Length Validation

Behavior change: a skill description outside 10-1000 characters now fails locally — in validate, preview and deploy — naming the skill, its actual length and the allowed range. Previously an out-of-range description reached the platform and failed there with an opaque API error. There is no placeholder description and no truncation: an out-of-range description is always an error, for both shapes above.

Running codemie validate --check-api additionally checks each skill's content length against the platform's own configured limits, which are read live from the API rather than hard-coded in the CLI.

Replacing a Whole Skill Group (--replace-skill-group)

Some skill sets are regenerated as a whole — a large skill split differently, for example — and have to be deleted and recreated under the same names. deploy updates skills in place and --prune only removes skills that left the config, so neither detaches and recreates a set of skills. --replace-skill-group does:

# See what would happen first
codemie preview --replace-skill-group github-copilot

# Then run it (the confirmation flag is mandatory)
codemie deploy --replace-skill-group github-copilot --yes-replace-skills

Group membership comes only from the group field — the label persisted in .codemie/state.json or the one on the resources.skills entry (including a group inherited from a source_dir entry). Skill names are never pattern-matched against the group name: a skill called github-copilot--body-1 is not part of the group unless it carries the label.

What a run does, in order:

1. Refuses without --yes-replace-skills — nothing is touched, no API call is made. A group name that matches an imported.skills entry, or that matches no skill at all, is refused the same way.
2. Detaches every selected skill from every state-managed assistant and every assistant in imported.assistants, before the first delete.
3. Deletes the selected skills and drops their state entries. A delete failure aborts the run then and there, so no skill is recreated while part of the group is still on the platform; fix the platform error and rerun.
4. Deploys only that group's skills and the assistants that reference them (plus those assistants' sub_assistants, so dependency resolution still works), letting the normal attachment sync reattach the recreated skills. An assistant's skills entries are matched against the group case-insensitively, the same way validation accepts them. Datasources and workflows are not deployed — resources.datasources and imported.datasources are still read, so an assistant's datasource_names resolve — --prune is ignored, and no other resource type is created, updated or deleted.

preview --replace-skill-group <group> reports the same set of skills as the deploy: one delete per selected skill and one create per config entry of the group.

⚠️ Recreated skills get new platform ids. Anything that references them by id rather than by name does not follow the change. In particular, another IaC project that references these skills through its own imported.skills keeps the old ids and must be updated (re-import the skill, or update the id by hand) after a replacement run.

📁 File Structure

assistant-resources/
├── codemie.yaml                 # Main IaC configuration
├── .codemie/
│   └── state.json              # Deployment state (committed to Git)
├── system_prompts/
│   └── *.prompt.md             # Prompt content files
├── workflows/
│   └── *.yaml                  # Workflow definitions
├── examples/                    # Configuration examples (reference)
│   ├── datasource-examples.yaml
│   ├── assistant-examples.yaml
│   ├── toolkit-examples.yaml
│   ├── workflow-examples.yaml
│   └── skill-examples.yaml
└── src/                        # IaC implementation

Key Features

Modular Configuration with $import

Organize your configuration across multiple files for better maintainability and team collaboration. The $import directive automatically resolves imports during config loading:

# Main config - clean and organized
resources:
  datasources:
    - $import: configs/datasources/repo-1.yaml  # Single resource
    - $import: configs/datasources/all-repos.yaml  # Array of resources
  
  assistants:
    - $import: configs/assistants/ba-team.yaml

Supports inline resources, individual files, array files, and any combination. See Configuration Organization for details.

Name-Based Resource Management

Resources are identified by their name field. Renaming a resource in codemie.yaml results in deleting the old resource and creating a new one.

Why names instead of IDs?

1. Platform generates UUIDs - You don't know the ID until after creation, making it impossible to write config before deployment
2. Git-friendly - Human-readable names enable meaningful diffs in pull requests: "Added Customer Support Bot" vs "Added resource a3f2b8c9-..."
3. Team collaboration - Names are self-documenting; team members understand what changed without looking up UUIDs
4. Config portability - Same config can be deployed to different environments without ID conflicts

Why delete+create on rename?

• Without UUIDs in config, there's no way to track that "Old Name" became "New Name"
• Alternative would require manual migration steps, defeating IaC purpose
• State file (state.json) maps names→UUIDs for updates, but rename breaks this mapping

About slugs:

• slug is optional for all assistants
• Platform auto-generates slug from name if not provided
• Recommendation: Don't specify slug unless you have specific requirements
• Sub-assistants use names (not slugs)
• Workflows reference assistants by name

Checksum-Based Change Detection

Changes to prompts and configuration are detected via SHA256 checksums, ensuring only modified resources are updated.

Why checksums instead of timestamps or full comparison?

1. Idempotency - Same config produces same checksums, enabling safe re-runs without unintended changes
2. Efficiency - Avoid costly API calls to fetch resources for comparison when nothing changed
3. Precision - Separate checksums for prompt content and resource config allow granular change detection
4. Git integration - Checksums stored in state.json provide audit trail of what was actually deployed

How it works:

• Prompt checksum: SHA256 of prompt file content
• Config checksum: SHA256 of resource configuration (excluding prompt path, including all other fields)
• On deployment: Compare stored checksums with current checksums
• Only update if either checksum differs

One-time checksum change for assistants with MCP servers

After upgrading to the release that normalizes MCP configuration, MCP server configuration is normalized before it is hashed: keys with no value (null) and an empty top-level mcp_connect_url no longer contribute to the hash, and the remaining keys are hashed in a fixed order at every depth, so the order the platform or your YAML uses for MCP fields never affects the checksum. Every assistant that has mcp_servers therefore gets a new configChecksum once - and only once.

The first npm run preview (or codemie preview) and npm run deploy after the upgrade list each such assistant under ⚠️ UPDATE: as - assistant:<name> followed by Prompt or configuration changed, with no per-field diff. This is expected and is not a configuration error - nothing in your codemie.yaml changed. deploy writes the new configChecksum back to .codemie/state.json, and the second run reports the same assistants as unchanged.

Orphaned Resource Cleanup

Resources removed from codemie.yaml are NOT automatically deleted by default. To remove them, you must explicitly use the --prune flag.

How orphaned resources are detected:

1. Load state.json - contains all resources created by IaC with their names and UUIDs
2. Load codemie.yaml - contains desired state (what should exist)
3. Compare: For each resource in state.json, check if it exists in codemie.yaml
4. If resource is in state but NOT in config → mark as orphaned
5. If --prune is used: Delete orphaned resources from platform using their UUID
6. Remove from state.json

Safety mechanisms:

• Opt-in cleanup - npm run deploy never deletes resources unless you pass --prune.
• Only IaC-created resources - Resources NOT in state.json are never deleted (manually created resources are safe)
• Per-resource-type - Checks assistants, datasources, workflows separately
• Logged deletion - Each deletion is logged with resource name and UUID for audit

Why automatic cleanup?

• GitOps principle - Config is source of truth; platform state should match config
• No manual cleanup - Prevents orphaned resources accumulating over time
• Safe refactoring - Rename/restructure resources without leaving behind old versions

Workflow Assistant Resolution

Workflows reference assistants by assistant_name, which are resolved to platform UUIDs during deployment from state.json or imported.assistants.

Why name-based references in workflows?

1. Readability - assistant_name: "Code Reviewer" vs assistant_id: "a3f2b8c9-..."
2. Maintainability - Update assistant name in one place (codemie.yaml), workflows auto-resolve
3. Versionability - Git diffs show meaningful changes: "Changed to Data Analyzer" vs changed UUID
4. Cross-environment - Same workflow YAML works in dev/prod with different assistant UUIDs

Resolution order:

1. Check state.json - IaC-created assistants (have name→UUID mapping)
2. Check imported.assistants - Pre-existing platform assistants referenced by config
3. If not found → deployment fails with clear error message

This happens during npm run deploy, before sending workflow YAML to API.

Integration Aliasing with $ref

Avoid duplication by referencing integrations:

imported:
  integrations:
    - alias: jira_connection
      id: 00000000-0000-0000-0000-000000000001
      credential_type: Jira

resources:
  # For datasources: use .id suffix to get UUID string
  datasources:
    - name: my-jira-kb
      type: knowledge_base_jira
      setting_id: $ref:imported.integrations.jira_connection.id  # → "00000000..."
      jql: project = MY
  
  # For toolkits: reference full object (no .id suffix)
  assistants:
    - name: My Assistant
      toolkits:
        - toolkit: Project Management
          tools:
            - name: generic_jira_tool
              settings:
                $ref: imported.integrations.jira_connection  # → full object

Why $ref instead of repeating integration IDs?

1. Single source of truth - Integration ID defined once, used everywhere
2. Easier updates - Change integration ID in one place, applies to all references
3. Reduced errors - Copy-paste mistakes eliminated
4. Semantic clarity - $ref:imported.integrations.jira_connection.id explains what it references

How $ref works:

• For datasources: Path MUST end with .id to get the UUID string
  • $ref:imported.integrations.name.id → resolves to UUID string
• For toolkits: Path references the full object (no .id suffix)
  • $ref:imported.integrations.name → resolves to full integration object
• During config load, ConfigLoader resolves all $ref references
• Validation ensures referenced path exists before deployment

State Management

The .codemie/state.json file tracks IaC-created resources:

{
  "version": "1",
  "project": "my-project",
  "lastSync": "2025-01-15T10:30:00Z",
  "resources": {
    "assistants": {
      "My Assistant": {
        "id": "uuid-here",
        "promptChecksum": "sha256-hash",
        "configChecksum": "sha256-hash"
      }
    },
    "datasources": { ... },
    "workflows": { ... }
  }
}

Why commit state.json to Git?

1. CI/CD requirement - Pipeline needs to know what's deployed to calculate changes
2. Team visibility - Everyone sees current production state
3. Audit trail - Git history shows who deployed what and when
4. No external backend - Simpler than S3/Consul/etc.; Git is the backend

Why no state locking?

• GitLab CI/CD resource_group: production ensures sequential execution
• Only one pipeline runs at a time → no concurrent modifications → no race conditions
• State conflicts impossible with proper CI/CD setup

📝 Configuration Guide

Importing Existing Resources from Platform

When you want to reference resources that already exist on the Codemie platform (created manually or by another team), you need to import them into your IaC configuration.

Why Import Resources?

For Integrations:

• Integrations are created in Codemie UI
• IaC only references them - doesn't create or manage them
• Required for datasources and toolkits to authenticate

For Datasources:

• Reuse datasources created by other teams
• Avoid duplicate indexing of the same data
• Share knowledge bases across multiple assistants

For Assistants:

• Reference existing assistants in workflows
• Use assistants as sub-assistants without managing them
• Collaborate across teams without duplicating resources

How to Import Resources

Step 1: Find the resource in Codemie UI

For Integrations:

1. Go to Codemie UI → Integrations
2. Find your integration (e.g., "Git Server Connection")
3. Copy the UUID from the URL or integration details
4. Note the integration type (Git, Jira, Confluence, etc.)

For Datasources:

1. Go to Codemie UI → Data Sources
2. Find your datasource
3. Copy the UUID from the URL (e.g., 00000000-0000-0000-0000-000000000002)
4. Note the datasource name and type

For Assistants:

1. Go to Codemie UI → Assistants
2. Find your assistant
3. Copy the UUID from the URL (e.g., 00000000-0000-0000-0000-000000000003)
4. Note the exact assistant name

Step 2: Add to imported section in codemie.yaml

imported:
  # Integrations (created in Codemie UI, referenced by datasources/toolkits)
  integrations:
    - alias: git_server_connection
      id: 00000000-0000-0000-0000-000000000004  # From Codemie UI
      credential_type: Git
    
    - alias: jira_connection
      id: 00000000-0000-0000-0000-000000000001
      credential_type: Jira
    
    - alias: confluence_connection
      id: 00000000-0000-0000-0000-000000000005
      credential_type: Confluence
  
  # Datasources (existing datasources to reuse)
  datasources:
    - id: 00000000-0000-0000-0000-000000000002  # From Codemie UI
      name: Confluence-6May2025  # Exact name from platform
      type: knowledge_base_confluence
    
    - id: 00000000-0000-0000-0000-000000000006
      name: Jira-Production-KB
      type: knowledge_base_jira
  
  # Assistants (existing assistants to reference in workflows/sub-assistants)
  assistants:
    - id: 00000000-0000-0000-0000-000000000003  # From Codemie UI
      name: Front-End Code Reviewer  # Exact name from platform
    
    - id: 00000000-0000-0000-0000-000000000007
      name: Checklist Generator Assistant

Step 3: Use imported resources in your configuration

resources:
  datasources:
    # Use imported integration
    - name: my-repo
      type: code
      link: https://git.example.com/repo
      branch: master
      setting_id: $ref:imported.integrations.git_server_connection.id  # Reference
  
  assistants:
    # Use imported datasource
    - name: My Assistant
      prompt: ./system_prompts/assistant.prompt.md
      model: gpt-4o
      context:
        - context_type: knowledge_base
          name: Confluence-6May2025  # References imported datasource
      
      # Use imported integration in toolkit
      toolkits:
        - toolkit: Project Management
          tools:
            - name: generic_jira_tool
              label: Generic Jira
              settings_config: false
              settings:
                $ref: imported.integrations.jira_connection  # Reference
      
      # Use imported assistant as sub-assistant
      sub_assistants:
        - Front-End Code Reviewer  # References imported assistant
  
  workflows:
    - name: My Workflow
      definition: ./workflows/workflow.yaml
      mode: Sequential

Important Notes:

• Integration IDs: Find in Codemie UI → Integrations (usually in URL or settings)
• Datasource IDs: Find in Codemie UI → Data Sources → Select datasource → Check URL
• Assistant IDs: Find in Codemie UI → Assistants → Select assistant → Check URL
• Exact names: Must match platform names exactly (case-sensitive)
• Type validation: IaC validates that types match platform resources
• Access permissions: Service account must have access to imported resources

Why use imported instead of creating via IaC?

| Resource Type | When to Import | When to Create via IaC | |---------------|----------------|------------------------| | Integrations | Always (created in UI only) | Not yet supported | | Datasources | Shared across teams, large/expensive to index | Team-specific datasources | | Assistants | Managed by other teams, stable production assistants | Your team's assistants |

Nested Assistants (Sub-Assistants)

You can link assistants together using sub_assistants (name-based):

resources:
  assistants:
    # Sub-assistant (must be defined first)
    - name: Sub Assistant Helper
      prompt: ./system_prompts/sub.prompt.md
      description: "Helper assistant"
      model: gpt-4o
    
    # Main assistant with nested sub-assistant
    - name: Main Assistant
      prompt: ./system_prompts/main.prompt.md
      description: "Main assistant with helper"
      model: gpt-4o
      sub_assistants:
        - "Sub Assistant Helper"  # Reference by NAME

How it works:

• Define sub-assistants before the main assistant in the YAML
• Use sub_assistants to reference them by name (name-based approach)
• During deployment, names are automatically resolved to IDs from state.json
• Sub-assistants must exist (be deployed first) for resolution to work

⚠️ Important restrictions:

• No cyclic dependencies: A → B → C → A is not allowed
• No nested sub-assistants: If an assistant has sub_assistants, it cannot be used as a sub-assistant by another assistant
• Example of invalid configuration:

  # ❌ Invalid: Middle has sub-assistants AND is used as sub-assistant
  assistants:
    - name: Helper
      model: gpt-4o
      
    - name: Middle
      model: gpt-4o
      sub_assistants: ["Helper"]  # Middle has sub-assistants
      
    - name: Main
      model: gpt-4o
      sub_assistants: ["Middle"]  # ❌ Middle cannot be sub-assistant!

📋 Deployment Order:

IaC automatically handles deployment order to resolve dependencies:

1. Datasources first - All datasources are created before assistants
2. Sub-assistants before main - Assistants are deployed in definition order
3. Assistants before workflows - Workflows are deployed last

💡 Best Practice: Define resources in the order they're referenced (datasources → sub-assistants → main assistants → workflows) for clarity.

Toolkits for Assistants

Toolkits provide additional capabilities to assistants, such as Git operations, Project Management (Jira/Confluence), and OpenAPI integration.

🔧 For detailed examples, see examples/toolkit-examples.yaml with all available toolkits and their configurations.

Available Toolkits:

1. Git - Git operations (PR changes, diffs)
2. Project Management - Jira and Confluence integration
3. OpenAPI - OpenAPI specification tools

Available MCP Integrations:

1. Figma MCP - Figma design tool integration
2. Atlassian MCP - Alternative Jira/Confluence integration (more advanced than toolkit)
3. Custom MCP - Any MCP-compatible server

Integration Aliases & Tool Definitions

To avoid repeating integration and tool settings across multiple assistants, you can define them once and reference using aliases:

Two-Level System:

# Level 1: Define integrations (created in Codemie UI)
imported:
  integrations:
    - alias: jira_main
      id: "00000000-0000-0000-0000-000000000001"
      credential_type: Jira
    
    - alias: confluence_main
      id: "00000000-0000-0000-0000-000000000005"
      credential_type: Confluence

# Level 2: Define tools (referencing integrations)
tool_definitions:
  jira_tool:
    name: generic_jira_tool
    label: Generic Jira
    settings_config: false
    settings:
      $ref: imported.integrations.jira_main  # References integration
  
  confluence_tool:
    name: generic_confluence_tool
    label: Generic Confluence
    settings_config: false
    settings:
      $ref: imported.integrations.confluence_main

# Level 3: Use tools in assistants
resources:
  assistants:
    - name: BA Assistant
      model: gpt-4o
      toolkits:
        - toolkit: Project Management
          tools:
            - $ref: tool_definitions.jira_tool
            - $ref: tool_definitions.confluence_tool

Usage Options:

# Option 1: Use tool definitions (RECOMMENDED - cleanest & most explicit)
tools:
  - $ref: tool_definitions.jira_tool
  - $ref: tool_definitions.confluence_tool

# Option 2: Reference integration directly (middle ground)
tools:
  - name: generic_jira_tool
    settings:
      $ref: imported.integrations.jira_main

# Option 3: Inline everything (not recommended, verbose)
tools:
  - name: generic_jira_tool
    settings:
      id: "00000000-0000-0000-0000-000000000001"
      credential_type: Jira
      # ... many more fields

Benefits:

• ✅ DRY Principle: Define once, use everywhere
• ✅ Easy Updates: Change integration/tool in one place
• ✅ Ultra Clean Config: From 15+ lines to 1 line per tool
• ✅ Type Safety: Integration/tool not found? Error at load time
• ✅ Reusability: Share tools across multiple assistants

Note: Integrations are created and managed in Codemie UI. The IaC config only references them by ID.

Basic Toolkit Structure:

resources:
  assistants:
    - name: My Assistant
      model: gpt-4o
      toolkits:
        - toolkit: Git  # Toolkit name
          label: ""
          settings_config: true
          is_external: false
          tools:
            - name: get_pr_changes
              label: "Get PR Changes"
              settings_config: false

MCP (Model Context Protocol) Servers:

In addition to toolkits, assistants can use MCP servers for external integrations. MCP servers use the same integration aliasing pattern as toolkits:

imported:
  integrations:
    - alias: playwright_mcp
      id: "00000000-0000-0000-0000-000000000008"
      credential_type: MCP

resources:
  assistants:
    - name: Browser Automation Assistant
      prompt: ./system_prompts/browser-automation.prompt.md
      description: "Assistant with browser automation via Playwright MCP"
      model: gpt-4o
      mcp_servers:
        - name: Playwright MCP Server
          description: "Playwright browser automation"
          enabled: true
          config:
            url: https://your-mcp-server.example.com/playwright/
            headers:
              X-Api-Key: $API_KEY
          settings:
            $ref: imported.integrations.playwright_mcp

MCP Integration Aliasing:

• MCP integrations are defined once in imported.integrations
• MCP servers reference them via settings: { $ref: imported.integrations.alias }
• Same DRY pattern as toolkits - define once, use everywhere

🔧 For detailed MCP examples, see the MCP section in examples/toolkit-examples.yaml

Workflows

Workflows orchestrate multiple assistants to handle complex, multi-step tasks.

Workflow Configuration in codemie.yaml

resources:
  workflows:
    - name: My Workflow  # No ID needed (name-based management)
      description: Workflow description
      definition: ./workflows/my-workflow.yaml  # Path to workflow definition file
      mode: Sequential  # Sequential or Autonomous
      shared: true  # Share with project members

Required fields:

• name - Unique workflow name (used as identifier)
• description - Workflow description
• definition - Path to workflow definition file
• mode - Execution mode: Sequential or Autonomous
• shared - Whether to share with project (typically true)

Workflow YAML Structure

Create a workflow definition file (e.g., workflows/my-workflow.yaml):

assistants:
  # Reference assistants by NAME (name-based approach)
  - id: analyzer  # Local ID for this workflow
    assistant_name: Data Analysis Assistant  # References assistant from codemie.yaml
  
  - id: reviewer  # Local ID
    assistant_name: Code Reviewer  # Another assistant reference
  
  # Inline assistant (defined directly in workflow)
  - id: parser
    model: gpt-4o-mini
    name: Data Parser
    system_prompt: |
      You are a data parser. Extract structured data from text.

states:
  - id: first_step
    assistant_id: analyzer  # Uses local ID from assistants section
    task: |
      Analyze the following input: {{ user_input }}
    output_schema: |
      {
        "analysis": "string",
        "confidence": "number",
        "recommendations": "array"
      }
    next:
      state_id: second_step
  
  - id: second_step
    assistant_id: reviewer
    task: |
      Review the analysis: {{ analysis }}
      Provide feedback on: {{ recommendations }}
    output_schema: |
      {
        "feedback": "string",
        "approved": "boolean"
      }
    next:
      state_id: end

How assistant resolution works:

1. External assistants (with assistant_name):

   • IaC looks up in state.json (for IaC-managed assistants)
   • Falls back to imported.assistants (for platform-managed assistants)
   • Resolves name → UUID during deployment
   • Workflow YAML stays unchanged (portable!)

2. Inline assistants (with model and system_prompt):

   • Defined directly in workflow
   • No resolution needed
   • Good for simple, workflow-specific assistants

Change detection:

• Workflow YAML checksum (workflowYamlChecksum) - Detects changes to workflow logic
• Metadata checksum (configChecksum) - Detects changes to name, description, mode

📚 For detailed workflow patterns, see examples/workflow-examples.yaml

Linking Datasources to Assistants

You can link datasources to assistants using the context field:

resources:
  datasources:
    - name: my-repo
      type: code
      description: "My repository"
      link: https://git.example.com/repo
      branch: master
      setting_id: $ref:imported.integrations.git_connection.id
  
  assistants:
    - name: My Assistant
      prompt: ./system_prompts/assistant.prompt.md
      description: "Assistant with datasources"
      model: gpt-4o
      context:
        - context_type: code
          name: my-repo  # Reference datasource by name

How it works:

• Use context array to reference datasources by their name field
• Specify context_type matching datasource type (code, knowledge_base, etc.)
• During deployment, datasource names are resolved to platform IDs
• Datasources must be defined in config (resources or imported) for resolution to work

Example with imported datasources:

imported:
  datasources:
    - id: 00000000-0000-0000-0000-000000000006
      name: Existing Jira KB
      type: knowledge_base_jira

resources:
  datasources:
    - name: my-code-repo
      type: code
      description: "My repository"
      link: https://git.example.com/repo
      branch: master
      setting_id: $ref:imported.integrations.git_connection.id
  
  assistants:
    - name: My Assistant
      prompt: ./system_prompts/assistant.prompt.md
      model: gpt-4o
      context:
        - context_type: code
          name: my-code-repo      # From resources (IaC-managed)
        - context_type: knowledge_base
          name: Existing Jira KB  # From imported (platform-managed)

💡 Use imported datasources when:

• Datasource is shared across multiple teams
• You don't want to manage datasource lifecycle via IaC
• Datasource already exists and is maintained elsewhere

Datasource Defaults with $ref

To avoid repetition, define reusable datasource configurations in datasource_defaults and reference them using $ref:

datasource_defaults:
  code:
    type: code
    index_type: code
    embeddings_model: ada-002
    shared_with_project: true
    branch: master
    force_reindex: false
  
  production_code:
    type: code
    index_type: summary
    embeddings_model: text-embedding-3-large
    shared_with_project: true
    branch: main
    force_reindex: false
  
  jira:
    type: knowledge_base_jira
    embeddings_model: ada-002
    shared_with_project: true
    force_reindex: false

resources:
  datasources:
    # Explicit $ref - inherits ALL fields from code
    - $ref: datasource_defaults.code
      name: my-repo
      description: My Git repository
      link: https://git.example.com/repo
      setting_id: $ref:imported.integrations.git_conn.id
      # type, index_type, embeddings_model, branch, etc. from $ref!
    
    # Override specific fields
    - $ref: datasource_defaults.code
      name: special-repo
      description: Special repository
      link: https://git.example.com/repo2
      setting_id: $ref:imported.integrations.git_conn.id
      branch: develop  # ← Overrides "master" from defaults
      embeddings_model: text-embedding-3-large  # ← Override
    
    # Different defaults template
    - $ref: datasource_defaults.production_code
      name: prod-repo
      description: Production repository
      link: https://git.example.com/prod
      setting_id: $ref:imported.integrations.git_conn.id
      # Uses production_code settings (summary index, large embeddings)
    
    # Jira datasource
    - $ref: datasource_defaults.jira
      name: my-jira
      description: Jira knowledge base
      setting_id: $ref:imported.integrations.jira_conn.id
      jql: project = MY

How $ref defaults work:

• Explicit reference: $ref: datasource_defaults.name at the start of datasource
• Merge strategy: { ...defaults, ...datasource } - datasource fields override defaults
• Multiple templates: Create different default sets for different use cases
• Minimal config: Only specify fields unique to each datasource (name, link, description)

Why $ref instead of type-based defaults?

1. Explicit - Clear where values come from
2. Flexible - Multiple default sets for same type (e.g., code vs production_code)
3. Reusable - Same $ref mechanism as integrations and tools
4. DRY - Avoid repeating same fields across 30+ datasources

Required fields (for CODE datasources):

• name - Unique datasource name
• type - Datasource type (determines which defaults to use)
• description - Description (typically repository info)
• link - Git repository URL
• branch - Git branch to index

Common fields (typically in defaults):

• index_type - Indexing type: code, summary, or chunk-summary
• embeddings_model - Model for embeddings (e.g., ada-002)
• summarization_model - LLM model for summarization (e.g., gpt-4o)
• shared_with_project - Share with project (default: true)
• setting_id - Integration reference (use $ref:imported.integrations.name.id)
• force_reindex - Force full reindex on deploy (default: false)

Force Reindex: Set force_reindex: true to trigger a full reindex of the datasource on the next deployment, even if no configuration changes were detected. This is useful for:

• Updating the index with new commits from the repository
• Recovering from failed indexing
• Applying new embeddings model settings

After deployment, set it back to false to avoid unnecessary reindexing.

Note: Datasource creation starts background indexing. Check Codemie UI for indexing status.

📚 For detailed datasource examples, see examples/datasource-examples.yaml with all available fields, descriptions, and common patterns.

Reusing Prompts

Multiple assistants can share the same prompt file:

assistants:
  - name: Production Bot
    prompt: system_prompts/shared.prompt.md
    model: gpt-4o
  
  - name: Development Bot (Copy)
    prompt: system_prompts/shared.prompt.md  # Same file
    model: gpt-4o-mini

Use cases:

• ✅ Creating test/staging copies of production assistants
• ✅ A/B testing with different models but same prompt
• ✅ Team-specific variants with shared core logic
• ✅ Backup/rollback scenarios

🔄 Common Workflows

Adding a New Assistant

1. Create prompt file:

   touch system_prompts/my-new-assistant.prompt.md

2. Add to codemie.yaml:

   resources:
     assistants:
       - name: My New Assistant  # Used as identifier (NO ID!)
         prompt: ./system_prompts/my-new-assistant.prompt.md
         description: "Description here"
         model: gpt-4o
         temperature: 0.7
         shared: true
         toolkits: []
         context: []

3. Deploy:

   npm run validate  # Check slug conflicts (if slug provided)
   npm run preview          # Preview changes
   npm run deploy        # Create assistan