How does longest-match work?

Within the matching UA group, the rule with the longest pattern that still matches the path wins. Allow and Disallow are weighed by pattern length only — an Allow with a longer pattern beats a Disallow with a shorter one.

What does $ at the end of a pattern mean?

It anchors the pattern to the end of the URL path. /foo$ matches /foo exactly, not /foo/bar. Useful for blocking specific file extensions like /*.pdf$.

robots.txt Tester

Network

Implements the matching rules used by Googlebot / Bingbot: groups are keyed by user-agent, the most-specific UA wins (exact > substring > *), and within a group the longest matching pattern wins (Allow ties beat Disallow). Supports the * wildcard and $ end-of-path anchor. Surfaces every Sitemap: declaration too, so you can sanity-check that your sitemap URLs are reachable.

—

robots.txt

Groups parsed: 3Sitemaps: 1

Test cases

User-agentURL path
Allowedrule: Allow: /admin/
User-agentURL path
Allowedrule: Allow: /admin/public/
User-agentURL path
Disallowedrule: Disallow: /
User-agentURL path
Disallowedrule: Disallow: /private/

Sitemaps

https://example.com/sitemap.xml

Substring UA matches are case-insensitive. Longest pattern wins; ties go to Allow.

How to use

Paste your robots.txt into the upper box.
Add (user-agent, path) test cases — set UA to '*' to test the catch-all group.
Read the verdict and the rule that produced it on the right.

Frequently asked questions

How does longest-match work?: Within the matching UA group, the rule with the longest pattern that still matches the path wins. Allow and Disallow are weighed by pattern length only — an Allow with a longer pattern beats a Disallow with a shorter one.
What does $ at the end of a pattern mean?: It anchors the pattern to the end of the URL path. /foo$ matches /foo exactly, not /foo/bar. Useful for blocking specific file extensions like /*.pdf$.

Related tools

IP Address Inspector

Type an IPv4 or IPv6 address and see its class, scope (private / public / loopback / link-local), decimal value, binary, reverse-DNS notation, and /32 CIDR.

Network00

Port Number Reference

Searchable cheat sheet for ~60 standard TCP / UDP port numbers — from 22 (SSH) and 80 (HTTP) to 6379 (Redis) and 27017 (MongoDB).

Network00

DNS Record Reference

Searchable cheat sheet for DNS record types — A, AAAA, CNAME, MX, TXT, NS, SOA, PTR, SRV, CAA, DNSSEC, SVCB / HTTPS — with examples.

Network00

Subnet Calculator (IPv4 / CIDR)

Parse an IPv4 CIDR into network address, broadcast, netmask, wildcard, host range, and class. Shows binary breakdown and private/public status.

Network00

User Agent Parser

Parse a User-Agent string into browser, engine, OS, device, and CPU. Detects 20+ bots including GPTBot, ClaudeBot, PerplexityBot.

Network00

HTTP Status Code Reference

Searchable list of every HTTP status code (1xx-5xx) with summary, RFC, when to use, and common pitfalls.

Network00