Scheduled Workflow to Batch-Document Legacy Macros

What This Builds

A script that runs on a schedule, checks your macro library for anything new or changed since the last run, and sends just those macros to an LLM API for a first-draft description, a parameter list, and a flag for anything that looks risky. It writes each result to a documentation stub file for a person to review and correct, not to publish as-is. Instead of a growing backlog of undocumented macros that never gets tackled because there's never a dedicated afternoon for it, documentation stays roughly current on its own.

Prerequisites

• Comfortable writing and running a Python (or R) script, and scheduling it with cron, Task Scheduler, or a similar tool your organization already uses
• An Anthropic API account with billing enabled. This bills per token rather than a flat subscription, so check the provider's pricing page for current rates before pointing this at a large macro library
• Written confirmation from whoever owns your data and vendor policy that sending your macro library to this API is an approved arrangement. Macro code is often confidential intellectual property, not just a technical asset, and that decision belongs to your employer, not to this guide

The Concept

This is less a chatbot conversation and more a standing employee whose entire job is reading macros nobody has time to read. Once a day, or on whatever schedule fits your library's pace of change, the script wakes up, checks which macro files are new or edited since the last time it ran, sends only those to the API with the same documentation prompt every time, and files the result away for someone to check later. Nobody has to remember to run it. The tradeoff is that nobody's watching it in real time either, which is why the failure modes below matter as much as the setup steps.

Build It Step by Step

Part 1: Write the script

Below is a working starting point. Copy it. Then adjust the folder paths, the file extension pattern, and the prompt wording to match your library and house documentation style.

"""
document_macros.py

Scans a folder of SAS macro files. For any macro that is new or has changed
since the last run, sends it to the Claude API and writes a documentation
stub to a separate review folder. Designed to run on a schedule (cron or
Task Scheduler), not interactively.
"""

import hashlib
import json
import os
import pathlib
import sys
from datetime import datetime, timezone

import requests

MACRO_DIR = pathlib.Path("/path/to/macro/library")
OUTPUT_DIR = pathlib.Path("/path/to/documentation/stubs")
MANIFEST_PATH = pathlib.Path("/path/to/documentation/.manifest.json")
LOG_PATH = pathlib.Path("/path/to/documentation/run.log")

# Paste the current model id here, copied from the provider's console or
# model documentation. Do not leave the placeholder in place.
MODEL_ID = "PASTE_THE_MODEL_ID_HERE"
API_KEY = os.environ.get("ANTHROPIC_API_KEY")

# Upper limit on macros sent per run. Keeps a first run against a large
# library, or a lost manifest, from sending everything at once.
MAX_PER_RUN = 8

PROMPT_TEMPLATE = """You are documenting a SAS macro for an internal library.
Read the macro below and produce:
1. A one-paragraph plain-language description of what it does.
2. A list of its parameters, with a best guess at each one's purpose.
3. A flag for anything that looks risky: hard-coded file paths, undocumented
   assumptions, or logic that depends on a specific dataset structure.
Mark this output clearly as a DRAFT for human review, not a final answer.

MACRO CODE:
{macro_code}
"""

def file_hash(path):
    return hashlib.sha256(path.read_bytes()).hexdigest()

def load_manifest():
    if MANIFEST_PATH.exists():
        return json.loads(MANIFEST_PATH.read_text())
    return {}

def save_manifest(manifest):
    MANIFEST_PATH.write_text(json.dumps(manifest, indent=2))

def call_claude(macro_code):
    response = requests.post(
        "https://api.anthropic.com/v1/messages",
        headers={
            "x-api-key": API_KEY,
            "anthropic-version": "2023-06-01",
            "content-type": "application/json",
        },
        json={
            "model": MODEL_ID,
            "max_tokens": 1024,
            "messages": [
                {"role": "user", "content": PROMPT_TEMPLATE.format(macro_code=macro_code)}
            ],
        },
        timeout=60,
    )
    response.raise_for_status()
    return response.json()["content"][0]["text"]

def main():
    if MODEL_ID == "PASTE_THE_MODEL_ID_HERE" or not API_KEY:
        sys.exit("Set MODEL_ID and ANTHROPIC_API_KEY before running this script.")

    manifest = load_manifest()
    processed = 0
    failed = []

    for macro_file in sorted(MACRO_DIR.glob("*.sas")):
        if processed >= MAX_PER_RUN:
            break

        current_hash = file_hash(macro_file)
        if manifest.get(macro_file.name) == current_hash:
            continue

        try:
            doc_text = call_claude(macro_file.read_text())
        except Exception as err:
            # One bad call should not stop the run or lose finished work.
            failed.append(f"{macro_file.name}: {err}")
            continue

        stub_path = OUTPUT_DIR / f"{macro_file.stem}.doc-stub.md"
        stub_path.write_text(doc_text)

        manifest[macro_file.name] = current_hash
        processed += 1

    save_manifest(manifest)

    with LOG_PATH.open("a") as log:
        timestamp = datetime.now(timezone.utc).isoformat()
        log.write(
            f"{timestamp} run complete, {processed} macro(s) documented, "
            f"{len(failed)} failed\n"
        )
        for line in failed:
            log.write(f"{timestamp} FAILED {line}\n")

if __name__ == "__main__":
    main()

If your team standardizes on R instead of Python, the same structure carries over: hash each file, compare against a saved manifest, call the API for anything changed, and write the result to a stub file. The manifest and scheduling logic don't change, only the syntax around them.

Part 2: Set the model id and the API key

Copy the current model id from your Anthropic console or the API's model documentation page, and paste it in place of the placeholder in MODEL_ID. Model names change as new versions ship, so treat whatever is in this guide's example as out of date the moment you read it. Store the API key as an environment variable your scheduler can see: on Linux or macOS, an entry in the crontab's environment or a file loaded before the script runs, on Windows, a system environment variable. Never put the key inside the script itself.

Part 3: Schedule it

On Linux or macOS, add a crontab entry:

# Run every night at 2am, log stdout and stderr for later review
0 2 * * * /usr/bin/python3 /path/to/document_macros.py >> /path/to/documentation/cron.log 2>&1

On Windows, register the same script under Task Scheduler with a daily trigger and the action pointed at your Python interpreter and the script's path. Either way, run the script manually once first and check that a stub file actually appears before trusting the schedule.

Real Example: A Quarterly Macro Library Catch-Up

Setup: A programming team's shared macro library has about 140 SAS macros, roughly a third of them undocumented or documented so sparsely the comments are useless. The script runs nightly against the library folder.

Input: Over the first week, the script finds 46 macros with no entry in its manifest, since everything that existed before the script's first run counts as new, and processes eight of them a night, the MAX_PER_RUN limit the team chose to stay within its token budget.

Output: Each processed macro gets a stub file with a plain-language description, a parameter list, and, for eleven of the forty-six, a flag noting a hard-coded file path that should probably be a parameter. A programmer reviews each stub during normal QC time rather than a dedicated documentation sprint, correcting maybe one detail per stub on average.

Time saved: Reverse-engineering an undocumented macro by hand runs about 20 to 30 minutes each. Reviewing and correcting an AI-drafted stub runs closer to 5 to 10 minutes. Across 46 macros, that's roughly 12 to 15 hours of backlog cleared without a single dedicated documentation week.

What to Do When It Breaks

• The scheduled run stops happening and nobody notices, because a script that doesn't run also doesn't error → This is the failure you won't see. Check the run log's last timestamp on a fixed cadence, a recurring reminder rather than memory, or point a simple monitoring check at the log file's modification time and alert if it's older than your schedule interval. Cron jobs most often go silent because something changed underneath them, a server migration or a credential rotation, not because of a bug in the script itself.
• The API key expires or gets rotated and every run fails quietly into the log → The script writes a FAILED line to the run log for every call that errors and keeps going, so a rotated key shows up as a run with zero documented and every macro failed. Failures land in a file nobody's watching in real time, so check both counts on your monitoring cadence.
• A macro is too large for the model's context and the call fails or returns a truncated result → Split large macros into logical sections before sending, or raise max_tokens and check the response for a truncation flag before writing the stub.
• The manifest file gets corrupted or deleted → The script treats every macro as new again and works back through the whole library, MAX_PER_RUN macros at a time. That costs extra API usage, and it overwrites any stub still sitting in the output folder with a fresh draft. Treat that folder as a review inbox: once a person has corrected a stub, move it into your real documentation location so a later run cannot replace the reviewed text.

Variations

• Simpler version: Run the script manually once a week instead of on a schedule, if your macro library changes slowly enough that unattended automation isn't worth the setup.
• Extended version: Add a step that posts a short summary of each run, how many macros processed, how many risk flags raised, to a team channel, so the backlog's progress stays visible without anyone opening the log file.

What to Do Next

• This week: Run the script once by hand against a small test folder of two or three macros you already know well, and check the drafts against what you'd write yourself.
• This month: Turn on the schedule against the real library, starting with a small daily batch limit while you build confidence in the output.
• Advanced: Route each finished stub through the Custom GPT QC reviewer guide's checklist before a human signs off, catching both undocumented logic and QC-style issues in the same pass.

Advanced guide for statistician professionals. These techniques use more sophisticated AI features that may require paid subscriptions.