Need integration help? Contact our engineering team
Please choose the option as per your server configuration from the side menu and follow the instructions

URL Inspector

The URL Inspector puts a powerful magnifying glass over any specific path in your domain. It allows you to see exactly what search engines and AI agents are seeing when they request your page.

Summary / Overview

Sometimes, looking at aggregated domain metrics isn't enough to solve an indexing problem. The URL Inspector gives you the "ground truth" for a single URL. It shows you the most recent copy of the HTML served to bots, the precise timestamp of when it was generated, and the breakdown of which specific crawlers have visited this exact page over time.

Use this tool to debug layout shifts that only happen during rendering, verify that specific meta data and structured data tags made it into the final HTML, and assess organic interest level based on crawler interactions.

Detailed Explanation / How It Works

The Sandboxed Preview

The centerpiece of the URL Inspector is the Sandboxed Preview. Index Render securely fetches the exact pre-rendered HTML currently stored in your cache and renders it inside an isolated iframe. This prevents styles or scripts from leaking onto your dashboard, but gives you an accurate visual representation of the bot-facing page.

Hit Analytics & Crawler Breakdown

Below the preview, the inspector provides a detailed analytical breakdown for the URL:

  • Total Hits: The raw number of times crawlers have requested this page.
  • Crawler Activity Chart (Pie Chart): A visual distribution showing which user agents (e.g., Googlebot, Bingbot, ChatGPT-User) are crawling this page most frequently.
  • Recent Hit Log: A table detailing the last 10 specific hits, including the exact timestamp, the IP address (masked for privacy), and the User-Agent string.

Step-by-Step Usage

Inspecting a URL

  1. From the Domain Details page, search for a specific URL.
  2. Click the Inspect button in the Actions column.
  3. The URL Inspector will load the cached HTML into the preview window.
  4. Toggle between the visual preview and the raw HTML source view (if applicable) to verify specific tags.

Verifying SEO Tags

  1. Use the inspector purely to verify that JavaScript-injected elements (like Open Graph tags, canonical URLs, and schema.org JSON-LD) are present in the final rendered output.
  2. If they are missing, it indicates a rendering timeout or an issue with your hydration process that needs fixing at the origin.
Pro Tip: Search engines heavily rely on the `` content. Even if the visual preview looks slightly unstyled (due to missing static assets designed for human interaction), as long as the content and the metadata in the DOM are correct, your SEO is safe.

Common Pitfalls / Tips

  • Visual discrepancies: The preview might look "broken" if you aggressively block CSS or images from being loaded by Index Render to save resources. This is normal and doesn't affect indexing text.
  • Stale Data: If the rendered content looks outdated, navigate to the Cache Manager and manually purge the URL to force a fresh render on the next request.
  • Crawler Log Empty: If the pie chart and hit logs are empty, it means no search engine has hit this specific cache entry yet. Consider submitting the URL directly via Google Search Console.