# Project P5 — Automation Toolkit By the end of this chapter you will: - Ship a multi-command toolkit as an installable package - Cover it with pytest, ruff, and mypy enforced in CI - Package for pip and Docker with a scheduled-run option - Document it like an open-source project others could adopt --- ## The project goal Build and ship a production-ready Python automation toolkit with three commands: 1. `organize-files`: sort/move files by rules, 2. `api-report`: fetch and summarize API datasets, 3. `run-scheduled`: execute recurring workflows with logs and status output. You are not only writing scripts. You are shipping a maintainable, testable, operational toolset. --- ## What success looks like Minimum acceptance criteria: 1. Toolkit installs as a package and exposes all CLI commands. 2. Test suite covers core logic with deterministic fixtures. 3. CI runs ruff, mypy, and pytest on every pull request. 4. Docker image runs the toolkit in non-root mode. 5. Runbook explains setup, scheduling, and incident handling. Demo one failure path and one recovery path. Interviewers trust production thinking more than a perfect happy path. --- ## Recommended architecture ```text automation-toolkit/ pyproject.toml uv.lock src/ toolkit/ cli.py commands/ organize_files.py api_report.py run_scheduled.py core/ config.py logging.py retry.py telemetry.py services/ file_service.py api_service.py scheduler_service.py models/ report_models.py tests/ test_organize_files.py test_api_report.py test_scheduler.py docker/ Dockerfile compose.yml .github/ workflows/ quality.yml release.yml docs/ RUNBOOK.md ADR-001-cli-framework.md ``` Architecture principle: shared domain logic in `services`, thin command adapters in `commands`. --- ## Implementation phases Phase 1: CLI foundation - Initialize project with uv and pinned dependencies. - Build Typer CLI entrypoint with three command groups. - Define shared config loading and structured logging. Phase 2: Core service logic - File organizer service with dry-run mode. - API report service with pagination, retry, and schema validation. - Scheduler service with lock-file to prevent overlapping runs. Phase 3: Test and quality gates - Add pytest coverage for successful and failure flows. - Enforce Ruff and mypy locally via pre-commit. - Add CI workflow gate for pull requests. Phase 4: Containerization and scheduling - Build multi-stage Docker image with lockfile install. - Add compose profile for local scheduler testing. - Validate cron/Task Scheduler integration runbook. Phase 5: Release and operations - Create tag-based release workflow and build artifacts. - Publish package and image with versioned changelog. - Define incident checklist and rollback procedure. --- ## Example CLI contract ```python import typer app = typer.Typer(help="Automation Toolkit") @app.command("organize-files") def organize_files(source: str, target: str, dry_run: bool = True) -> None: """Move files by policy with optional dry-run.""" @app.command("api-report") def api_report(url: str, output: str) -> None: """Fetch paginated data and write a summary report.""" @app.command("run-scheduled") def run_scheduled(job: str) -> None: """Execute a named scheduled automation job.""" if __name__ == "__main__": app() ``` CLI contracts should expose predictable options and explicit failure codes. --- ## Command runbook ```bash # install dependencies uv sync --frozen # run quality checks ruff check . ruff format . --check mypy src pytest -q # run commands python -m toolkit.cli organize-files --source incoming --target archive --dry-run false python -m toolkit.cli api-report --url https://api.example.com/items --output reports/latest.json python -m toolkit.cli run-scheduled --job weekly-summary # build and run container docker build -t automation-toolkit:local -f docker/Dockerfile . docker run --rm automation-toolkit:local --help ``` --- ## Quality gates | Gate | Requirement | |---|---| | Test reliability | Core command logic validated with deterministic fixtures | | Lint policy | Ruff check and format check pass | | Type safety | mypy passes on source package | | CI enforcement | PR workflow required before merge | | Runtime readiness | Docker image starts and executes commands | | Operational docs | Runbook includes failure recovery and ownership | --- ## Common failure modes | Failure mode | Symptom | Fix | |---|---|---| | Duplicate file moves on rerun | Unexpected repeated side effects | Add idempotency checks and dry-run preview | | API pagination drift | Missing records in report output | Implement robust next-page cursor handling | | Silent scheduler failure | Job appears green but no output | Add explicit exit codes and structured alerts | | Inconsistent local and CI behavior | Works locally, fails in pipeline | Align tooling versions and lock dependency graph | | Unclear operational ownership | Incidents linger without response | Assign on-call owner and escalation runbook | --- ## Extension ideas 1. Add plugin hooks so teams can register custom command modules. 2. Add YAML job definitions for non-developer operators. 3. Add notification adapters (email, Slack, webhook) for job outcomes. 4. Add metrics endpoint for scheduled job latency and success rate. 5. Add interactive TUI dashboard for local operations. --- ## ADR sample ### ADR-001: Typer vs argparse - Context: Toolkit needs multiple commands with readable argument contracts and auto-help. - Decision: Use Typer for command ergonomics and type-driven arguments. - Consequences: - Positive: faster CLI development and cleaner help UX. - Negative: extra dependency versus stdlib argparse. - Revisit trigger: dependency policy demands stdlib-only runtime. ### ADR-002: uv vs Poetry - Context: Need fast local/CI installs with lockfile determinism. - Decision: Use uv as primary dependency workflow. - Consequences: - Positive: faster installs and simplified pipeline timing. - Negative: team must align on uv-specific commands. - Revisit trigger: packaging constraints require workflow features unavailable in current setup. --- ## Interview defense prompts 1. How did you guarantee idempotent behavior for repeated runs? 2. Where do you enforce quality gates and why at those points? 3. What operational signals tell you automation is healthy? 4. How do you handle API failure and partial-data scenarios? 5. What would you improve first if the toolkit had 100x more users? --- ## Summary - P5 proves you can ship automation as a product, not as isolated scripts. - The strongest signal is reproducibility: test gates, CI policy, and container parity. - Operational maturity appears in runbooks, failure handling, and ownership boundaries. - The project is complete when another engineer can install, run, and operate it from docs alone. ## Self-Check Quiz

Project P5 — Automation Toolkit

Part of L5