Technical FAQ
The questions people arriving from a desktop ripper actually ask.
▸
Do you execute JavaScript and single-page apps?
Yes, optionally. Tick "render JavaScript" and each page is loaded in headless Google Chrome on our server and captured after the DOM settles, so a React, Vue, Angular or Svelte app yields real HTML instead of an empty root div. It is off by default because it is several times slower — leave it off for classic server-rendered sites. Note that links discovered only through user interaction (click-to-load, infinite scroll) still will not be found; the crawler follows markup, it does not drive the UI.
▸
How do I back up a site that requires a login?
Two fields: cookies and user-agent. Sign in to your own site in a browser, copy the session cookie header, paste it into the cookie field, and the crawler sends it with every request to that host — so members areas, staging sites behind a session gate, and logged-in dashboards mirror correctly. We deliberately do not do credential brute-forcing or login-form automation. Your cookie string is stored with the job and deleted with it (72 hours, or immediately if you delete the backup).
▸
Do you respect robots.txt?
There is a per-crawl toggle, and it is off by default — because the normal case is you archiving your own site, where robots.txt is a directive to search engines rather than a permission boundary for the owner. Turn it on and we fetch /robots.txt and skip disallowed paths. Either way, robots.txt is not authorization: if the site is not yours, the Acceptable Use Policy says do not crawl it, whatever robots.txt allows.
▸
How hard does the crawler hit a server?
One request at a time per crawl, over a keep-alive connection, with an optional delay of up to 5 seconds between requests. There is no parallel hammering. You also cap depth, page count and total size per job, and you can pause or cancel mid-run from the app.
▸
What does it actually capture?
Links, images, stylesheets, scripts, media sources, iframes, srcset candidates and CSS url() plus stylesheet import references, followed breadth-first from the URL you gave it. Scope is yours to choose: only under this path, this exact host, this host and its subdomains, or anywhere. Off-site assets you pull in are kept in a separate _ext/ folder so the mirror stays browsable.
▸
How are the downloaded files organised?
The directory structure mirrors the site. Query-string URLs get a stable hashed suffix so nothing collides, and in the ZIP the HTML and CSS are rewritten to relative local paths — open index.html from your desktop and it browses like the live site, offline. A receipt file records what was captured and when.
▸
Can it back up a site that is already down?
No. We fetch over live HTTP, so the origin has to answer. If the server is gone, the pages are gone — which is the argument for taking the backup while it is still up.
▸
What cannot be captured?
Anything the server does not hand to an anonymous HTTP client: database contents, server-side code, admin panels behind two-factor prompts, streaming media served through DRM, and content that only appears after a user interaction. This is an offline copy of the front end, not a server migration.
▸
Where does the crawl run from, and can I be blocked?
From our servers, so the target sees our datacentre IP, not yours. That protects your home or office connection from getting rate-limited or banned while you archive your own property. It also means an aggressive WAF may challenge us — if a crawl stalls with 403s, add a delay, set a realistic user-agent, and if it is your site, allowlist us.
▸
How long do you keep my mirror?
72 hours from the start of the backup, then an hourly scheduled purge deletes the files and the index. Downloaded ZIPs are assembled into a temporary file and removed the moment the transfer completes. You can also delete any backup yourself at once. Details are in the Privacy Policy.
▸
Is there an API?
Not yet. The tool is driven from the web app today. If a scripted or scheduled backup is what you need, tell us — it is on the list and demand decides the order.
▸
What does it cost?
Nothing during the beta. If paid tiers arrive, they will be about higher ceilings and longer retention, and we will not silently start charging for an account you already have.