Contributing to secdiagram

Thank you for your interest in contributing to secdiagram! We welcome contributions from developers, security researchers, and software architects of all backgrounds.

This guide outlines our development workflow, architectural principles, coding standards, and security policies to help you contribute effectively and safely.

---

Table of Contents

• Architectural Principles
• Code of Conduct
• Development Setup
• Development Workflow
  • Running Unit Tests
  • Linting and Formatting
  • Type Checking
  • Security and SAST Scans
• Task Tracking
• Codebase Architecture
• Coding Guidelines
  • Zero Runtime Dependencies
  • Strict Determinism
  • Cross-Platform File Handling
  • Modern Python Typing (3.10+)
  • Security Best Practices
  • Dependency Pinning & Supply Chain
• Branching Model
• Pull Request Process
• Commit Message Conventions
• Reporting Security Vulnerabilities

---

🏛️ Architectural Principles

Before writing code, please keep the four core tenets of secdiagram in mind:

1. Zero External Runtime Dependencies: The core CLI, AST analyzer, graph builder, and renderers must run on the Python standard library only. We do not add third-party runtime wheels to dependencies = [] in pyproject.toml. This eliminates supply chain risk and ensures instant installation.
2. Deterministic Output: Analyzing the same repository state must always yield identical outputs (diagram.mmd, graph.json, security.sarif, etc.). IDs are computed via deterministic hashing (stable_id). Never iterate over unordered sets or dict keys when writing outputs.
3. 100% Offline-First: All core analysis, diagram generation, and security checks must run without internet access. Any remote feature (e.g. Kroki, mermaid.ink, OpenAI API) must be strictly optional with graceful local fallbacks.
4. Security by Design: secdiagram inspects untrusted repositories. It must never execute (eval, exec, importlib) analyzed code. All file reading, path resolutions, and YAML/JSON parses must be hardened against path traversal and deserialization attacks.

---

📜 Code of Conduct

We are committed to providing a friendly, safe, and welcoming environment for all contributors. Please treat others with respect, empathy, and professional courtesy in all project interactions, including issues, pull requests, and discussions.

---

🚀 Development Setup

Prerequisites

• Python: Version 3.10 or higher (tested on 3.10, 3.11, 3.12, 3.13, and 3.14).
• Git: Installed and configured.
• (Optional) Docker: For testing containerized execution and local Ollama reviews.
• (Optional) Node.js & Mermaid CLI (@mermaid-js/mermaid-cli): For testing local offline PNG/SVG/PDF image rendering.

Step-by-Step Local Setup

1. Clone the repository:

   git clone https://github.com/vserey/secdiagram.git
   cd clone-secdiagram
   

2. Create and activate a virtual environment:

   # Linux / macOS
   python3 -m venv .venv
   source .venv/bin/activate
   
   # Windows (PowerShell)
   python -m venv .venv
   .venv\Scripts\Activate.ps1
   

3. Install the package in editable mode with development dependencies:

   python -m pip install "pip==26.3"
   pip install -e ".[dev]"
   

   Development dependencies installed via .[dev] include:

   • ruff: High-speed linter and import formatter.
   • mypy: Static type checker.
   • coverage: Test coverage suite.
   • bandit: Static application security testing (SAST) for Python.
   • pip-audit: Known CVE dependency auditor.

---

🔄 Development Workflow

Running Unit Tests

Run the full test suite using Python's built-in unittest runner:

# Run all unit tests
python -m unittest discover -s tests -v

# Run a specific test module
python -m unittest tests/test_security.py -v

# Run with test coverage
python -m coverage run -m unittest discover -s tests
python -m coverage report

Note: Test suite must achieve ≥ 80% coverage across secdiagram_core. All existing 134+ tests must pass without regressions.

Linting and Formatting

We use ruff for linting and import sorting:

# Check code style and imports
ruff check .

# Automatically apply safe fixes
ruff check . --fix

Type Checking

We use mypy with Python 3.10+ strict typing rules:

mypy

Security and SAST Scans

Run automated security gates before pushing commits:

# Run Bandit SAST gate (high severity)
python -m bandit -r secdiagram_core -lll

# Generate full Bandit JSON report
python -m bandit -r secdiagram_core -f json -o bandit.json

# Check installed dev dependencies for vulnerabilities
python -m pip_audit --strict --skip-editable

---

🗂️ Task Tracking

Every non-trivial change is tracked on GitHub Issues and a Projects v2 board. GitHub is the source of truth; the local cache under .secdiagram/tasks/ is a gitignored, read-only snapshot for offline work.

Lifecycle: Backlog -> Ready -> In Progress -> In Review -> Done, with Blocked while waiting on a dependency or decision.

# Create a task and add it to the board
python scripts/taskctl.py new --type feature --title "Add Go manifest detection" \
  --priority P1 --area discovery --milestone 0.3.0

# Move through the lifecycle
python scripts/taskctl.py start <issue>
python scripts/taskctl.py block <issue> --reason "waiting on upstream API"
python scripts/taskctl.py review <issue>
python scripts/taskctl.py done <issue> --evidence-file evidence.md

# Refresh and validate the offline cache
python scripts/taskctl.py sync
python scripts/taskctl.py validate --offline

Pull requests must reference an issue with Closes #N; the tasks-pr-check.yml workflow fails otherwise. See docs/TASK_TRACKING.md for one-time setup, board fields, and troubleshooting.

---

📂 Codebase Architecture

secdiagram/
├── secdiagram_core/             # Core library (Zero external runtime dependencies)
│   ├── __init__.py              # Version, schema, and public exports
│   ├── analyzer.py              # Orchestrates repo discovery, AST indexing, and analysis
│   ├── ast_index.py             # Python AST parser (classes, functions, calls, imports)
│   ├── cli.py                   # Command-line entrypoint and argument parser
│   ├── discover.py              # File walking, gitignore parsing, polyglot manifest detection
│   ├── features.py              # Feature grouping and subsystem association
│   ├── frameworks.py            # Framework detection (FastAPI, Flask, Django, etc.)
│   ├── graph_builder.py         # Builds dependency graph, pruning, edge weighting
│   ├── high_level.py            # Compiles C4 high-level tier overview diagrams
│   ├── iac.py                   # Infrastructure-as-Code audits (Dockerfile, Compose, K8s, TF)
│   ├── llm.py                   # Citation-preserving LLM narrative review generator
│   ├── model.py                 # Graph, Node, Edge, Finding, and stable_id data models
│   ├── render_image.py          # PNG, SVG, PDF image generation (local mmdc / Kroki / ink)
│   ├── render_json.py           # Structured JSON output serializer
│   ├── render_markdown.py       # Architecture document generator (ARCHITECTURE.md)
│   ├── render_mermaid.py        # Mermaid flowchart code generator (diagram.mmd)
│   ├── render_sarif.py          # OASIS SARIF v2.1.0 security report serializer
│   ├── resolver.py              # Qualified symbol resolution and call attribution
│   └── security.py              # SAST security rules (taint flow, hardcoded secrets, injection)
├── tests/                       # Test suite
│   ├── fixtures/                # Sample applications (Python, .NET, Node.js)
│   ├── test_cli.py              # CLI integration tests
│   ├── test_discover.py         # Discovery & gitignore tests
│   ├── test_graph.py            # Graph construction & pruning tests
│   ├── test_high_level.py       # High-level C4 diagram tests
│   ├── test_image.py            # Image rendering and provider tests
│   ├── test_llm.py              # Ollama and LLM narrative tests
│   ├── test_python_index.py     # AST indexing & resolver tests
│   ├── test_render.py           # Renderers verification
│   └── test_security.py         # SAST heuristics & secret scanner tests
├── .github/
│   ├── dependabot.yml           # Automated dependency updates
│   └── workflows/
│       ├── ci.yml               # Multi-version Python CI, lint, SAST, Trivy
│       ├── codeql.yml           # GitHub CodeQL static analysis
│       ├── docker.yml           # Container build & security scan
│       └── release.yml          # Automated release and Docker Hub delivery
├── Dockerfile                   # Hardened multi-stage unprivileged container
├── docker-compose.yml           # Container run orchestration
└── pyproject.toml               # Package build metadata & tool configurations

---

📐 Coding Guidelines

1. Zero Runtime Dependencies

• Strict Rule: Do not add packages to dependencies = [] in pyproject.toml.
• Use Python's standard library modules (ast, urllib, dataclasses, hashlib, json, pathlib, re, shutil, tempfile, subprocess).
• If a feature requires external tooling (like rendering images via Mermaid or executing local LLMs), interact with the CLI binary or standard HTTP protocols with robust error handling and offline fallbacks.

2. Strict Determinism

• Stable IDs must be created using stable_id(*parts).
• Never iterate over unordered set or dict.keys() where order influences output files.
• Always sort collections before generating Markdown, JSON, Mermaid code, or SARIF findings.

3. Cross-Platform File Handling

• Always use pathlib.Path for filesystem operations.
• Internally normalize graph node and file paths to forward slashes (/) so diagrams and SARIF reports remain byte-for-byte identical across Linux, macOS, and Windows.

4. Modern Python Typing (3.10+)

• Use Python 3.10+ type union syntax: str | None instead of Optional[str].
• Use built-in generic collections: list[str], dict[str, Any], set[int].
• Import abstract collection types from collections.abc (e.g. from collections.abc import Iterable, Sequence), not from typing.
• Provide complete type hints for function signatures and data models.

5. Security Best Practices

• No Code Execution: Never call eval(), exec(), or dynamically __import__() untrusted user code.
• Hashing: When computing non-cryptographic hashes for IDs (e.g., hashlib.sha1), always pass usedforsecurity=False to comply with FIPS and Bandit security gates:

  digest = hashlib.sha1(data.encode("utf-8"), usedforsecurity=False).hexdigest()
  

• Subprocess Safety: Pass commands as argument lists (['mmdc', '-i', ...]): never use shell=True. Set explicit execution timeouts.
• Network Calls: Always set explicit timeout on urllib.request.urlopen.

6. Dependency Pinning & Supply Chain

• Keep dependencies = [] empty; the runtime remains standard-library only.
• Pin every development/build dependency with an exact == version in pyproject.toml. Never use ranges, >=, or latest.
• Pin GitHub Actions to full commit SHAs with a trailing version comment, and let Dependabot keep them current.
• Pin container and Compose images to an exact tag and @sha256: digest.
• Install Node tooling from the committed docker/mermaid/package-lock.json using npm ci --omit=dev; never use floating npm install in the image build.
• Treat any new dependency as a security review item: prefer first-party, actively maintained projects with reproducible lockfiles.

7. Container Security & Image Vulnerability Gating

• Automated container builds and releases to Docker Hub (vserey/secdiagram) require that all CRITICAL and HIGH severity image vulnerabilities are resolved prior to push.
• Container pushes are blocked by default if unresolved Critical or High vulnerabilities are detected.
• If an upstream base image vulnerability cannot be remediated immediately, a documented Contextual Reachability and Exploitability Analysis must prove that the finding is not reachable and not exploitable in secdiagram's container environment (e.g. dead code, lack of network ingress, unprivileged UID 10001 user execution) before publication is permitted.
• Approved exceptions live in .trivyignore.yaml, generated by scripts/refresh_image_baseline.py from a Trivy JSON report. Each entry requires a statement and an expired_at date; fixable findings are rejected by the generator and always block the gate. Refresh the baseline and docs/IMAGE_SECURITY_BASELINE.md each release.

---

Branching Model

| Branch | Purpose | Direct commits | Promotion | |---|---|---|---| | main | Stable, release-ready | Never | develop → main via release PR | | develop | Integration | Never | Feature branches merged locally | | feat/..., fix/..., ci/..., docs/... | Short-lived work | Yes | PR or local merge into develop |

• Branch from develop and keep it current with origin/develop before starting.
• develop must always contain the latest main.
• Merge features into develop locally only after the full verification suite passes; push develop after a clean merge.
• Promote develop to main only when a release is ready and all CI gates pass.
• Tag releases on main after the release merge is verified.
• After a release, sync main back into develop; the develop-sync.yml workflow does this automatically and fails loudly on conflicts.
• Never force-push main, develop, or any shared branch.

---

🤝 Pull Request Process

1. Create or select a task: Open a GitHub issue (or run python scripts/taskctl.py new ...) and move it to In Progress with python scripts/taskctl.py start <issue>.

2. Fork and branch from develop:

   git fetch origin
   git checkout develop
   git merge --ff-only origin/develop
   git checkout -b feat/add-go-manifest-detection
   # or
   git checkout -b fix/sanitize-mermaid-subgraph-ids
   

3. Develop and test:

   • Write clean, commented code conforming to the guidelines above.
   • Add comprehensive unit tests under tests/ covering new functionality and edge cases.
   • Ensure all tests pass: python -m unittest discover -s tests -v.
   • Ensure linter passes: ruff check ..
   • Ensure type check passes: mypy.
   • Ensure security scan passes: python -m bandit -r secdiagram_core -lll.
   • Ensure task metadata validates: python scripts/taskctl.py validate --offline.
   • Ensure containerized build and smoke test pass: docker build -t secdiagram:verify . && docker run --rm secdiagram:verify --help
   • Ensure container image vulnerability scan passes: trivy image --severity CRITICAL,HIGH --ignorefile .trivyignore.yaml --exit-code 1 secdiagram:verify

4. Verify Git Hygiene:

   • Verify no secrets, .env files, temporary screenshots, or dump files are staged:

     git status
     git diff --cached
     

   • Confirm .secdiagram/tasks/ is not staged; it is a local, read-only cache.

5. Submit PR: Open a Pull Request targeting the develop branch. Provide a clear description of:

   • What the PR accomplishes.
   • Why the change is needed.
   • A linked issue using Closes #N.
   • Test results and verification evidence.
   • Move the task to In Review with python scripts/taskctl.py review <issue> before requesting review.

6. Release promotion: Maintainers merge develop into main via a release Pull Request only when the release is ready, then tag the verified commit on main.

---

💬 Commit Message Conventions

We adhere to the Conventional Commits specification:

• feat: A new feature or capability.
• fix: A bug fix.
• docs: Documentation updates.
• test: Adding or updating tests.
• refactor: Code changes that neither fix a bug nor add a feature.
• ci: Changes to CI/CD workflows, build scripts, or release automation.
• perf: Performance improvements.

Example:

feat: add detection for Go modules and Gin route endpoints

- Parse go.mod for module identity and direct dependencies
- Map Gin engine HTTP route bindings to presentation tier
- Add unit tests covering go.mod parsing

---

🛡️ Reporting Security Vulnerabilities

We take security seriously. If you discover a security vulnerability in secdiagram, please do not disclose it publicly in an issue or forum.

Instead, please report it privately:

1. GitHub Security Advisory: Submit a report via GitHub Private Vulnerability Reporting.
2. Email: If you are unable to use GitHub Security Advisories, send an encrypted email to the maintainer at vichhaiyserey@gmail.com.

Include:

• A description of the vulnerability and its potential impact.
• Step-by-step reproduction steps or a minimal proof-of-concept repository.
• Recommended remediation steps, if known.

We will acknowledge receipt within 48 hours and work with you on a coordinated fix and disclosure schedule.