AI crawl & propose

Let the proxy draft the configuration for you.

Site Transformer crawls the site, groups pages by structural similarity, and uses AI to propose a full set of aliases and DOM transforms. You review the diff, accept what you want, and publish.

Auditing a large site by hand is slow. The proxy already sees every page — so it can do the first pass for you.

A crawl maps the site, clusters templates that share a structure, and surfaces the patterns worth fixing. From there, AI drafts concrete configuration you can accept line by line — never a black box that ships on its own.

How a run works

  • Crawl — discover and fetch the site's pages, respecting robots and crawl budget. The crawler identifies itself as SiteTransformerBot and obeys the site's robots.txt — see the crawler page.
  • Cluster — group pages by structural similarity so one rule can cover a whole template.
  • Analyze — AI inspects each cluster for URL and on-page issues a technical audit would flag.
  • Propose — receive a ready-to-review configuration of aliases and transforms.
  • Accept & publish — keep what you want, edit the rest, and deploy in seconds.

You stay in control

Nothing goes live until you approve it. Every proposal is a reviewable change set with an instant rollback, so the AI accelerates the work without ever owning the decision.

crawl · proposal
# crawl summary
pages:    4,812 crawled
clusters: 7 templates detected

# proposed for cluster "legacy-article" (1,204 pages)
alias:    "/index.php?id=*" → "/guides/{slug}"
transform match("^/guides/.*"):
  replace ".post-wrap-legacy" → "article"
  inject  "ld+json" → schema("Article")

# → review · accept · publish

Start from a proposal, not a blank config.

Crawl your site and let Site Transformer draft the first version for you.