Traefik Integration Guide

Configure Traefik Proxy to automatically route crawler traffic to Index Render. This integration uses Traefik's powerful routing rules to identify bots and forward them to the pre-rendering service transparently.

Edge Routing By handling bot detection at the Traefik level, you ensure that your application containers only deal with real human traffic.

1

Define Index Render Service

Add the Index Render API as a service in your Traefik dynamic configuration.

dynamic_conf.yml

http:
  services:
    index-render-api:
      loadBalancer:
        servers:
          - url: "https://api.indexrender.io"

2

Configure Headers Middleware

Create a middleware to inject your API key and set the necessary headers for the Index Render service.

dynamic_conf.yml

http:
  middlewares:
    index-render-headers:
      headers:
        customRequestHeaders:
          x-indexrender-key: "your_api_key_here"
          Accept: "text/html"

3

Set Up Bot Routing Rule

Create a router with a higher priority that matches crawler User-Agents. This router will forward traffic to the Index Render service.

dynamic_conf.yml

http:
  routers:
    # 1. Crawler Router (High Priority)
    my-app-bots:
      rule: "Host(`example.com`) && HeaderRegexp(`User-Agent`, `(?i)(googlebot|googlebot-image|googlebot-video|googlebot-news|googleother|apis-google|bingbot|yandex|baiduspider|facebookexternalhit|facebookcatalog|facebot|twitterbot|rogerbot|linkedinbot|embedly|quora\ link\ preview|showyoubot|outbrain|pinterest|pinterestbot|slackbot|slack-imgproxy|vkshare|w3c_validator|redditbot|applebot|whatsapp|flipboard|tumblr|bitlybot|skypeuripreview|nuzzel|discordbot|google\ page\ speed|qwantify|bitrix\ link\ preview|xing-contenttabreceiver|telegrambot|google-inspectiontool|petalbot|duckduckbot|bytespider|gptbot|chatgpt-user|claudebot|perplexitybot|ahrefsbot|semrushbot|mj12bot|slurp)`)"
      priority: 100
      service: index-render-api
      middlewares:
        - index-render-headers
      # Note: You may need a middleware to rewrite the path if not using the default API structure

    # 2. Regular User Router
    my-app:
      rule: "Host(`example.com`)"
      priority: 10
      service: my-app-service

Note: Adjust the Host and service names to match your existing Traefik setup.

4

Verify Integration

To confirm that your integration is active, send a request to your URL while simulating a search engine bot. Run the following command in your terminal:

Terminal

curl --location 'https://yourdomain.com' \
--header 'User-Agent: googlebot'

In the <head> section of the response, you should see the Index Render metadata tags indicating the page was successfully pre-rendered:

Expected Output

<!--IndexRender-->
<meta name="generator" content="IndexRender/65d6cde" />
<meta name="x-rendered-by" content="IndexRender/65d6cde" />
<meta name="x-rendered-at" content="2026-04-26T18:26:03.1076251Z" />
<meta name="x-render-traceparent" content="00-39f1dd04eeba1243cae01dc5c70e5b47-cdf5cfae992740d8-01" />