ແທັກປລັກອິນ: crawlers
-
Block AI Crawlers
Tell AI (Artificial Intelligence) companies not to scrape your site for their AI products.
-
Robots.txt rewrite
Provide the easy managment of your robots.txt from admin side. It propose you the advanced then standard robots.txt content too.
-
AI Content Signals
Control how AI crawlers use your content with Cloudflare Content Signals in robots.txt. Includes EU Directive 2019/790 rights reservation.
-
CodingBunny LLMs.txt Generator
Advanced llms.txt generator with AI metadata, Schema.org types, canonical URLs, and differentiated licenses for optimal LLM indexing.
-
Robots Meta Whiz
Easy way to discourage search engines from indexing only specific pages / posts with custom meta tags.
-
-
CeeWP Robots.txt Manager
Easily manage your WordPress site's virtual robots.txt file with custom rules. No physical file needed!
-
Serve Markdown
Serve Markdown versions of your posts and pages to AI agents and crawlers. Content negotiation, .md URLs, auto-discovery, and crawler logging.
-
Seo Crawlytics Data Export & Delete
Exports all entries of Seo Crawlytics and gives you the option to delete the entries that are older than 1 month.
-
Log-Hero.com Plugin
Analyze how search engines and other bots crawl and understand your web page. The official PHP Wordpress plugin for log-hero.com.
-
-
Botkibble
Serves every published post and page as Markdown for AI agents and crawlers. No configuration, no API keys. Activate and it works.
-
Supertab Connect
Connect your WordPress site to the Supertab Connect platform for RSL license serving and enabling crawler authentication protocol.
-
AI Rights Protection by SimpleFeed
Protect your rights by reviewing and blocking AI bots from using your content without compensation.
