Sitebulb is a website crawling tool designed to help SEO professionals efficiently audit websites and find opportunities for improvement. As such, Sitebulb falls under the category of a ‘crawler’, a generic term used to describe bots that discover and scan web pages by following links between them.
The user-agent is part of the HTTP request that a crawler like Sitebulb sends when trying to recover the page content for the server. It is a string of characters that helps servers identify the application, bot, or system requesting information.
The Sitebulb Bot
It is important to note that Sitebulb is not a public bot. The Sitebulb bot will only ever crawl a website if a user proactively chooses to use it while performing an audit within the Sitebulb software.
Sitebulb’s User Agent string
Sitebulb has two user agents, Sitebulb Desktop and Sitebulb Smartphone, which will be selected by default when you first set up an audit with Desktop or Mobile respectively.
These user agents use the same string, which will look like this:
Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z/117.0.5938.88 Safari/537.36 (compatible; Sitebulb/1.1; +https://sitebulb.com)
Chrome/W.X.Y.Z Placeholder
Like Googlebot, Sitebulb uses the placeholder Chrome/W.X.Y.Z to represent the latest version of Chrome, which is regularly updated when new versions of Sitebulb are released. We do this to keep our Chrome Crawler evergreen, in line with Google's rendering policy.
Read more about Sitebulb’s evergreen Chromium rendering engine here.
This means that if you choose to whitelist Sitebulb using Sitebulb’s user agent string, you will need to regularly update your whitelisting rules as the string is updated with the latest version of Chrome. For guidance on how to efficiently whitelist Sitebulb for crawling, read our Whitelisting documentation.
Changing the User Agent
When setting up your audit, you can choose between a number of user-agents, including Google, Bing, Yahoo, and Yandex user-agents. This changes which user-agent string Sitebulb will use when making HTTP requests, and can impact what content is returned by the website server. For more on customizing user agent settings, read our Robots Directives settings guide.