Finding #005

Brave Search crawler uses an obfuscated user-agent by design

Date: 25 March 2026, 03:59 UTC Author: Manus AI Radar version: v1.0.2 Status: Open

Observed Hit

The following hit was recorded on the Kaistone Radar dashboard:

Timestamp: 2026-03-25T03:59:45.026Z
Bot: Unknown / Human
IP: 134.199.132.123
User-Agent: Mozilla/5.0 (iPad; CPU OS 26_0_1 like Mac OS X)
AppleWebKit/605.1.15 (KHTML, like Gecko)
Brave/1 Mobile/15E148 Safari/604.1
ID: mn5ilsw2-w5l71u

The hit was classified as Unknown / Human because no bot pattern matched the user-agent. However, several signals indicate this is the Brave Search crawler.

Signal Analysis

SignalObservationInterpretation
Brave/1 token Present in the user-agent string Real Brave browser releases include a full version number (e.g. Brave/1.74.48). A bare Brave/1 is not a valid browser release — it is a minimal self-identification token consistent with a crawler.
OS version 26_0_1 Claims to be iPadOS 26.0.1 iPadOS 26 does not exist. The current release is iPadOS 18.x. This is a synthetic, non-existent OS version — confirming the UA is not from a real device.
IP: 134.199.132.123 DigitalOcean, LLC (ASN 14061), Georgia, USA A real iPad user would connect from a mobile carrier or home ISP. A DigitalOcean datacenter IP confirms this is a server-side automated process.
Beacon image loaded Hit appeared on the dashboard The 1×1 GIF beacon was fetched, meaning the crawler uses a headless browser renderer that loads embedded images — not a plain HTTP fetcher.

Confirmed by Brave's Own Documentation

This behaviour is not accidental. Brave's official crawler documentation explicitly states:

"The Brave Search crawler does not advertise a differentiated user agent because we must avoid discrimination from websites that allow only Google to crawl them."

search.brave.com/help/brave-search-crawler

Brave has made a deliberate policy decision to disguise their crawler as a regular browser user in order to bypass bot-blocking rules. The Brave/1 token appears to be the only intentional self-identification signal embedded in an otherwise generic mobile Safari UA string.

Implications

This finding has several implications for AI crawler detection and web governance:

ImplicationDetail
Detection is possible but non-trivial The Brave/1 token combined with a datacenter IP and an impossible OS version provides a reliable heuristic. None of these signals alone is conclusive, but together they are strongly indicative.
robots.txt compliance is unclear Because the crawler mimics a browser UA, standard bot-blocking rules in robots.txt that target named crawlers (e.g. User-agent: BraveBot) will not apply. Brave states their crawler respects Googlebot rules, but this cannot be independently verified from the UA alone.
Site owners cannot give or withhold consent A site owner who wishes to allow Brave Search but block other AI crawlers has no reliable mechanism to do so, since Brave does not identify itself distinctly.
Kaistone Radar classified it as Unknown / Human The current bot detection patterns do not match this UA. A new pattern targeting Brave/1 on datacenter IPs could be added, but would require IP-range awareness to avoid false positives from real Brave browser users.

Detection Heuristic

A reliable heuristic for identifying the Brave Search crawler combines three conditions:

user-agent contains "Brave/1"
AND user-agent does NOT contain "Brave/1." (i.e. no full version)
AND IP resolves to a known datacenter ASN (e.g. DigitalOcean, AWS, GCP, Azure)

Alternatively, a simpler pattern that catches the minimal version token:

{ pattern: /Brave\/1(?!\.\d)/i, name: "Brave Search (suspected)" }
Note: Adding this pattern to BOT_PATTERNS in beacon.mjs carries a small risk of false positives if a real Brave browser user happens to visit with an unusual UA. Cross-referencing with IP ASN data would reduce this risk but is not currently implemented in the beacon function.

Comparison with Other Crawlers

CrawlerSelf-identifies?Example UA token
GPTBot (OpenAI)YesGPTBot/1.0
ClaudeBot (Anthropic)YesClaudeBot/1.0
PerplexityBotYesPerplexityBot/1.0
GooglebotYesGooglebot/2.1
Brave SearchNo (by policy)Brave/1 (minimal token only)

References

Brave Search Crawler documentation: search.brave.com/help/brave-search-crawler
IP lookup: ipinfo.io/134.199.132.123 — DigitalOcean, ASN 14061
Dashboard: /dashboard/
Findings index: /findings/