robots.txt for WordPress

Optimized robots.txt template for WordPress sites. Blocks admin, login, and plugin directories while allowing important content to be crawled.

robots.txt

User-agent: *
Disallow: /wp-admin/
Disallow: /wp-login.php
Disallow: /wp-includes/
Disallow: /wp-content/plugins/
Disallow: /wp-content/cache/
Disallow: /trackback/
Disallow: /feed/
Disallow: /comments/feed/
Disallow: /?s=
Disallow: /search/
Allow: /wp-content/uploads/
Allow: /wp-admin/admin-ajax.php

Sitemap: https://example.com/sitemap.xml

Line-by-Line Explanation

User-agent: * — applies to all search engine crawlers

Disallow: /wp-admin/ — blocks the WordPress admin dashboard from crawling

Disallow: /wp-login.php — prevents crawling of the login page

Disallow: /wp-includes/ — blocks WordPress core files

Disallow: /wp-content/plugins/ — prevents indexing of plugin files

Disallow: /wp-content/cache/ — blocks cached versions of pages

Disallow: /trackback/ — prevents crawling of trackback URLs

Disallow: /feed/ — blocks RSS feed pages from being crawled

Disallow: /?s= and /search/ — prevents search result pages from being indexed

Allow: /wp-content/uploads/ — ensures media files are crawlable

Allow: /wp-admin/admin-ajax.php — required for some themes and plugins to function

Sitemap — points crawlers to the XML sitemap for efficient discovery

Best Practices for WordPress

✓ If using Yoast SEO or Rank Math, they auto-generate a virtual robots.txt — check for conflicts.
✓ Never block /wp-content/themes/ as search engines need CSS and JS for rendering.
✓ Add specific paths for any custom post type archives you want to exclude.
✓ Test your robots.txt in Google Search Console before deploying.

Build a custom robots.txt for your WordPress site

Open robots.txt Generator

Frequently Asked Questions

Does WordPress create a robots.txt automatically?▾

WordPress generates a virtual robots.txt by default, but it is very basic. A custom robots.txt file uploaded to your root directory will override the virtual one.

Should I block /wp-content/themes/?▾

No. Google needs access to CSS and JavaScript files to render your pages correctly. Blocking theme files can negatively impact your search rankings.

Do I need robots.txt if I use an SEO plugin?▾

SEO plugins like Yoast and Rank Math manage robots.txt for you. If you use a plugin, edit robots.txt through the plugin settings rather than uploading a separate file.

Related Templates

Drupal Drupal-optimized robots.txt template. Blocks admin, system files, and internal paths while allowing content pages. Shopify Recommended robots.txt configuration for Shopify stores. Handles collection filtering, checkout, and admin pages. Wix Recommended robots.txt configuration for Wix websites. Understand what Wix generates and how to customize it.