Madomeda - Markdown Document Metadata Manager

A Python tool for managing and normalizing frontmatter in markdown files within git repositories.

Quick Start

# Install dependencies
pip install -r requirements.txt

# Run on current directory (dry run)
python madomeda.py --whatif

# Process files
python madomeda.py

# View database
python inspect_db.py

What It Does

Madomeda automatically:

• ✅ Discovers markdown files in git repositories
• ✅ Parses and normalizes YAML frontmatter
• ✅ Applies rule-based validation (lowercase keys, normalized tags)
• ✅ Extracts git metadata (authors, dates, versions)
• ✅ Generates missing frontmatter from templates
• ✅ Tracks all changes in SQLite database
• ✅ Logs modifications to changelog file
• ✅ (Optional) Uses LLM for intelligent metadata inference

Features

Git Integration

• Tracks files via git ls-files
• Extracts commit history, authors, and dates
• Respects .gitignore patterns
• Ignores hidden files (starting with .)

Rule-Based Normalization

• Tags: Converts to lowercase_with_underscores
• Keys: All frontmatter keys to lowercase
• tag→tags: Renames singular to plural
• summary→description: Renames summary field to description
• Extensible via JSON rule files

Template System

Multi-strategy field resolution:

• heur: Heuristics (git history, document content)
• orig: Original frontmatter value
• ai <prompt>: LLM inference with structured output

Database Tracking

SQLite database stores:

• File paths and discovery dates
• Frontmatter versions and conformance
• Git commit information
• All unique tags across repository

Installation

pip install -r requirements.txt

Optional: LLM Configuration

Edit .env:

OPENAI_API_URL=https://api.openai.com/v1
OPENAI_MODEL=gpt-4
OPENAI_API_KEY=your-api-key-here

Usage

# Process current directory
python madomeda.py

# Dry run (preview changes)
python madomeda.py --whatif

# Specific directory
python madomeda.py --dir /path/to/repo

# Force update all files
python madomeda.py --force

# Skip confirmation prompts
python madomeda.py --no-confirm

# Ignore specific paths
python madomeda.py --ignore-paths docs/archive vendor

# Use custom template
python madomeda.py --template my-template

# Add-only mode (preserve existing keys)
python madomeda.py --add-only

Example Transformation

Before:

---
Title: My Document
Tag: Python, Machine-Learning, Data Science
---

After:

---
title: My Document
created: '2025-10-23T17:27:16+02:00'
changed: '2025-10-23T17:27:16+02:00'
authors:
- John Doe
version: 34d2d05
tags:
- python
- machine_learning
- data_science
---

Documentation

• GETTING_STARTED.md - Step-by-step guide
• USAGE.md - Detailed usage examples
• STRUCTURE.md - Project architecture
• IMPLEMENTATION.md - Technical details
• PROJECT_SUMMARY.md - Complete overview

Project Structure

madomeda/
├── madomeda.py              # Main entry point
├── inspect_db.py            # Database inspection tool
├── requirements.txt         # Dependencies
├── .env                     # LLM configuration
│
├── core/                    # Core modules
│   ├── database.py          # SQLite management
│   ├── repository.py        # Git operations
│   ├── frontmatter.py       # YAML parsing
│   ├── rules.py             # Rule engine
│   ├── template.py          # Template system
│   ├── llm.py              # LLM integration
│   ├── changelog.py         # Change logging
│   └── processor.py         # Main orchestration
│
├── templates/               # Frontmatter templates
├── rules/                   # Validation rules
├── prompts/                 # LLM prompts
│
├── madomeda.db             # Database (gitignored)
└── madomeda_changelog.txt   # Change log

Templates

Create custom templates in templates/:

{
  "title": "heur|orig",
  "created": "heur|orig",
  "changed": "heur|orig",
  "authors": "heur|orig",
  "version": "heur|orig",
  "tags": "heur|orig|ai default"
}

Strategy priority: Left to right until one succeeds.

Rules

Default rules in rules/:

• tags_normalization.json - Normalize tag format
• tag_to_tags.json - Rename singular to plural
• lowercase_keys.json - Lowercase all keys
• summary_to_description.json - Rename summary to description

Heuristic Values

Field	Source
title	# heading or filename
created	First commit date (ISO 8601)
changed	Latest commit date (ISO 8601)
authors	All contributors from git log
version	Latest commit tag (if exists), else short hash (7 chars)
tags	Normalized from original

Command-Line Options

Option	Description
--dir PATH	Repository directory (default: current)
--template NAME	Template to use (default: 'default')
--ignore-paths PATH...	Paths to exclude
--whatif	Dry run mode (no changes)
--no-confirm	Skip confirmation prompts
--force	Recreate all frontmatter
--add-only	Only add missing fields, preserve existing keys

Database Inspection

python inspect_db.py

Or query directly:

import sqlite3
conn = sqlite3.connect('madomeda.db')
cursor = conn.cursor()
cursor.execute('SELECT tag FROM tags ORDER BY tag')
print([row[0] for row in cursor.fetchall()])

Changelog Format

2025-10-23 17:40:36 (commit: 34d2d05)
  sample.md - frontmatter updated
  test.md - frontmatter updated

Requirements

• Python 3.7+
• Git repository
• Dependencies: PyYAML, python-dotenv, openai (optional)

License

Created as a custom tool for markdown documentation management.

Getting Help

1. Read the documentation files
2. Run with --whatif to preview changes
3. Use inspect_db.py to examine database
4. Check madomeda_changelog.txt for history

Contributing

This is a standalone tool. Customize by:

• Creating new templates in templates/
• Adding rules in rules/
• Modifying prompts in prompts/
• Extending code in core/ modules