Sitebulb is a website crawling tool designed to help SEO professionals efficiently audit websites and find opportunities for improvement. As such, Sitebulb falls under the category of a ‘crawler’, a generic term used to describe bots that discover and scan web pages by following links between them.
The user-agent is part of the HTTP request that a crawler like Sitebulb sends when retrieving page content from the server. It is a string of characters that helps servers identify the application, bot, or system requesting information.
The Sitebulb Bot
It is important to note that Sitebulb is not a public bot. The Sitebulb bot will only crawl a website if a user proactively chooses to use it during an audit in the Sitebulb software.
Sitebulb’s User Agent string
Sitebulb has two user agents, Sitebulb Desktop and Sitebulb Smartphone, which will be selected by default when you first set up an audit with Desktop or Mobile, respectively.
These user agent strings look like this:
Sitebulb smartphone
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 +https://sitebulb.com
Sitebulb desktop
Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Safari/537.36 (compatible; Sitebulb/1.1; +https://sitebulb.com)
Chrome/W.X.Y.Z Placeholder
Like Googlebot, Sitebulb uses the placeholder Chrome/W.X.Y.Z to represent the latest version of Chrome, which is regularly updated when new versions of Sitebulb are released. We do this to keep our Chrome Crawler evergreen, in line with Google's rendering policy.
Read more about Sitebulb’s evergreen Chromium rendering engine here.
This means that if you choose to allowlist Sitebulb using its user agent string, you will need to regularly update your listing rules as the string is updated to reflect the latest version of Chrome. For guidance on how to efficiently allowlist Sitebulb for crawling, read our listing documentation.
Changing the User Agent
When setting up your audit, you can choose among several user agents, including Google, Bing, Yahoo, and Yandex. This changes the user agent string Sitebulb uses when making HTTP requests and can affect what content the website server returns. For more on customising user agent settings, read our Robots Directives settings guide.