rlnorthcutt/sitemapExport

The tool supports interactive prompts, command-line flags, and environment variables. Flags take priority over environment variables, which take priority over interactive prompts.

Interactive

./sitemapExport

Example session:

Enter the Sitemap or RSS feed URL or file path (required): https://example.com/sitemap.xml
Enter the CSS selector to extract content (default: body):
Enter the output filename (default: output):
Enter the URL filter pattern (default: *):
Enter the output file type (txt, json, jsonl, md) (default: txt): jsonl

Export data with the following settings:
  Input:           https://example.com/sitemap.xml
  CSS Selector:    body
  URL Filter:      *
  Output Filename: output
  Output Filetype: jsonl
  Format:          txt

Do you want to proceed with these settings? (y/n): y

Non-interactive (flags)

./sitemapExport --input="https://example.com/sitemap.xml" --css="article" --filename="output" --type="jsonl" --filter="blog/*"

Short flags are also supported:

./sitemapExport -i "https://example.com/sitemap.xml" -c "article" -n "output" -t "jsonl"

Flags

| Flag | Short | Default | Description | |---|---|---|---| | --input | -i | (required) | Sitemap or RSS feed URL, or local file path | | --css | -c | body | CSS selector to extract content | | --filename | -n | output | Output filename (without extension) | | --type | -t | txt | Output format: txt, json, jsonl, md | | --filter | | * | Only include URLs matching this pattern (e.g., blog/*) | | --verbose | -v | false | Enable verbose output |

Environment variables

All inputs can also be set via environment variables, which are checked when a flag is not explicitly provided:

| Variable | Corresponding flag | |---|---| | SITEMAP_INPUT | --input | | SITEMAP_CSS | --css | | SITEMAP_FILENAME | --filename | | SITEMAP_TYPE | --type | | SITEMAP_FILTER | --filter |

Output formats

• txt — plain text, one page per section
• json — pretty-printed JSON array
• jsonl — one JSON object per line (ideal for AI ingestion pipelines)
• md — Markdown with headings and content

Example output

JSON (output.json):

[
  {
    "Title": "Home",
    "URL": "https://example.com",
    "Description": "Welcome to our homepage",
    "Content": "Welcome to our site!"
  },
  {
    "Title": "About Us",
    "URL": "https://example.com/about",
    "Description": "Learn more about our company",
    "Content": "We are a company..."
  }
]

JSON Lines (output.jsonl):

{"Title":"Home","URL":"https://example.com/","Description":"Welcome to our homepage","Content":"Welcome to our site!"}
{"Title":"About Us","URL":"https://example.com/about","Description":"Learn more about our company","Content":"We are a company..."}