Hacker News new | past | comments | ask | show | jobs | submit

User Agents of Visitors

This page demonstrates showing query result from database.

Below list of user agents are collected from the visitor's HTTP header.

stats ready

User AgentCount
Bots7960372
Android198737
Other72725
MacOS62254
Windows48492
Linux4581
iPhone420
iPad57
ChromeOS42
curl39
Wget27
Links1
Bot AgentCount
SemrushBot5574149
YandexBot651728
AhrefsBot579678
PetalBot428103
DotBot191675
BarkrowlerBot159765
BLEXBot148154
GoogleBot102155
BingBot90841
MJ12Bot17603
DataForSeoBot12853
InternetMeasurement878
DuckDuckGoBot870
CensysInspect507
NeevaBot406
PaloBot320
PythonRequests305
LinkdexBot191
YahooBot93
NetcraftSurveyAgent43
FacebookBot36
QwantBot16
WhatsAppBot2
TelegramBot1
Other AgentCount
Mozilla/5.0 (compatible; Bytespider; spider-feedback@bytedance.com) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.0.0 Safari/537.3635139
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; GPTBot/1.0; +https://openai.com/gptbot)15667
Go-http-client/1.113939
wp_is_mobile3412
MBCrawler/1.0 (https://monitorbacklinks.com/robot)2208
Mozilla/5.0 (compatible; Google-Apps-Script; beanserver; +https://script.google.com; id: UAEmdDd-ws7HzVG5CObvQRdUIkKS0g_LGKw)228
Go-http-client/2.0215
Mozilla/5.0 zgrab/0.x201
Python/3.10 aiohttp/3.8.5177
Mozilla/5.0 (compatible; SeznamBot/4.0; +http://napoveda.seznam.cz/seznambot-intro/)157
SEMrushBot123
axios/1.3.4107
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)86
colly - https://github.com/gocolly/colly72
FuseonBot/1.1 (+http://linkaffinity.io)67
Java/11.0.1060
Mozilla/5.0 (compatible; MojeekBot/0.11; +https://www.mojeek.com/bot.html)54
Mozilla/5.0 (compatible; ImagesiftBot; +imagesift.com)52
Mozilla/5.0 (compatible; VelenPublicWebCrawler/1.0; +https://velen.io)43
Scrapy/2.6.3 (+https://scrapy.org)40
Mozilla/5.0 (compatible; SeznamBot/4.0-RC1; +http://napoveda.seznam.cz/seznambot-intro/)39
Mozilla/5.0 (X11; OpenBSD i386) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.3637
Request-Promise35
Apache-HttpClient/4.5.2 (Java/1.8.0_151)23
Ruby22
Apache-HttpClient/4.5.2 (Java/1.8.0_161)21
got (https://github.com/sindresorhus/got)20
Twitterbot/1.018
Client/36613 CFNetwork/1485 Darwin/23.1.015
YouBot (+http://www.you.com)14
url12
Mozilla/5.0 (keys-so-bot)11
Mozilla/5.0 (compatible; wpbot/1.0; +https://forms.gle/ajBaxygz9jSR8p8G9)11
Screaming Frog SEO Spider/16.710
Mozilla/5.09
GuzzleHttp/79
Symfony BrowserKit9
Slack-ImgProxy (+https://api.slack.com/robots)9
VerticalLeap-BacklinkChecker/2.0 (http://www.apolloinsights.com/) Chrome/56.0.2924.87 Mozilla/5.08
Screaming Frog SEO Spider/17.28
Screaming Frog SEO Spider/18.28
Mozilla/5.0 (l9scan/2.0.934323e2631313e2430313e2237313; +https://leakix.net)8
Apache-HttpClient/5.1.3 (Java/1.8.0_342)8
Safari/18614.3.7.1.5 CFNetwork/1402.0.8 Darwin/22.2.08
Screaming Frog SEO Spider/18.17
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot7
Mozilla/5.0 (compatible; Google-Apps-Script; beanserver; +https://script.google.com; id: UAEmdDd88BOptxfamryWUfQvDkS1pNY218A)7
Apache/2.4.34 (Ubuntu) OpenSSL/1.1.1 (internal dummy connection)7
GoogleProducer; (+http://goo.gl/7y4SX)7
Mozilla/4.0 (compatible; fluid/0.0; +http://www.leak.info/bot.html)6
K7MLWCBot/1.0 (+http://www.k7computing.com)6
Screaming Frog SEO Spider/19.26
Screaming Frog SEO Spider/19.46
Safari/19617.1.17.11.9 CFNetwork/1490.0.4 Darwin/23.2.06
Mozilla/5.0 (compatible; GeedoBot; +http://www.geedo.com/bot.html)6
Safari/18615.1.26.110.1 CFNetwork/1406.0.4 Darwin/22.4.06
okhttp/4.9.25
Chrome/74.0.3729.157 Safari/537.365
Screaming Frog SEO Spider/17.15
Mozilla/5.0 (compatible; Cincraw/1.0; +http://cincrawdata.net/bot/)5
CyotekWebCopy/1.9 CyotekHTTP/6.25
Mozilla/5.0 (compatible; Google-Apps-Script; beanserver; +https://script.google.com; id: UAEmdDd_gfaBLU6m8ghvF28GCIDhv5fWj7A)5
MobileSafari/604.1 CFNetwork/978.0.7 Darwin/18.7.05
colly - https://github.com/gocolly/colly/v24
Screaming Frog SEO Spider/18.04
MetaInspector/5.9.0 (+https://github.com/jaimeiniesta/metainspector)4
Client/33911 CFNetwork/1474 Darwin/23.0.04
IMNTools/2.0 (+http://www.webuildpages.com)4
libwww-perl/6.434
Mozilla/5.0 (compatible; BitSightBot/1.0)4
Screaming Frog SEO Spider/15.24
firefox3
Screaming Frog SEO Spider/16.63
Mozilla/5.0 (X11; FreeBSD amd64; rv:107.0) Gecko/20100101 Firefox/107.03
com.apple.WebKit.Networking/8614.2.9.0.10 CFNetwork/1399 Darwin/22.1.03
Mozilla/5.0 (compatible)3
WebZIP/3.5 (http://www.spidersoft.com)3
com.apple.WebKit.Networking/8615.3.12.10.2 CFNetwork/1410.0.3 Darwin/22.6.03
com.apple.WebKit.Networking/8616.1.27.10.13 CFNetwork/1474 Darwin/23.0.03
undici3
MagpieRSS/0.72 \(+http://magpierss.sf.net\)3
Mozilla/5.0 (compatible; coccocbot-web/1.0; +http://help.coccoc.com/searchengine)3
Screaming Frog SEO Spider/19.83
HackerNews/1500 CFNetwork/1333.0.4 Darwin/21.5.02
Mozilla/5.0 (compatible; Cocolyzebot/1.0; https://cocolyze.com/bot)2
HackerNews/1500 CFNetwork/1335.0.3 Darwin/21.6.02
fasthttp2
Python/3.9 aiohttp/3.8.12
TimeTravelAggregator-lanl;Browser2
Test Certificate Info2
Mozilla/5.0 AppleWebKit (KHTML, like Gecko) Chrome Safari2
axios/0.21.42
HackerNews/1517 CFNetwork/1399 Darwin/22.1.02
BrightEdge Crawler/1.0 (crawler@brightedge.com)2
Apache-HttpClient/4.5.13 (Java/11.0.18)2
Mozilla/5.0 (compatible; NetpeakCheckerBot/3.6; +https://netpeaksoftware.com/checker)2
Mozilla/5.0 (compatible; Google-Apps-Script; beanserver; +https://script.google.com; id: UAEmdDd_u7KpcmIgaHtZ0QMQUl5pHt97_wVw)2
Screaming Frog SEO Spider/18.52
Mozilla/5.0 (compatible; Google-Apps-Script; beanserver; +https://script.google.com; id: UAEmdDd9zIB9tyvQQKAS9My2aOmpA6OwKqzI)2
Mozilla/5.0 researchscan.comsys.rwth-aachen.de2
Mozilla/5.0 (PlayBook; U; RIM Tablet OS 2.1.0; en-US) AppleWebKit/536.2+ (KHTML like Gecko) Version/7.2.1.0 Safari/536.2+2
python-httpx/0.25.12
Python/3.11 aiohttp/3.9.12
ClaudeBot2
CCBot/2.0 (https://commoncrawl.org/faq/)2
Python/3.6 aiohttp/3.6.22
CriteoBot/0.12
Mozilla/5.0 (compatible; SEOkicks; +https://www.seokicks.de/robot.html)2
Mediapartners-Google2
FriendlyCrawler/Nutch-1.20-SNAPSHOT2
ZoomBot (Linkbot 1.0 http://suite.seozoom.it/bot.html)2
Python/3.11 aiohttp/3.9.32
Mozilla/5.0 (compatible; Google-Apps-Script; beanserver; +https://script.google.com; id: UAEmdDd_rdvGWaRJzPZkVOIpi21btyOduBw)1
Chrome/72.0.3626.121 Safari/537.361
Python/3.7 aiohttp/3.7.4.post01
Mozilla/5.0 Firefox/26.01
Mozilla/5.0 (compatible; Atomseobot/2.0; +http://https://error404.atomseo.com/)1
Mozilla/5.0 (compatible; Domains Project/1.3.7; +https://domainsproject.org)1
Scrapy/2.6.2 (+https://scrapy.org)1
Opera/9.80 (J2ME/MIDP; Opera Mini/8.0.35626/37.8918; U; en) Presto/2.12.423 Version/12.161
Opera/3 CFNetwork/1390 Darwin/22.0.01
Screaming Frog SEO Spider/13.21
Discu.eu bot/0.11
Mozilla/5.0 (compatible; proximic; +https://www.comscore.com/Web-Crawler)1
axios/1.1.31
Screaming Frog SEO Spider/17.01
Mozilla/5.0 (Andorid 5SGLOBAL; CPU Andorid OS 7_1_1 ) AppleWebKit/537.51.2 (KHTML, like Gecko) Version/7.0 MQQBrowser/7.4.1 Mobile/11D201 Safari/8536.25 MttCustomUA/2 QBWebViewType/11
NinjBot/2.0 (+http://www.webuildpages.com)1
Java/18.0.2.11
Google1
Mozilla/5.0 (BlackBerry; U; BlackBerry 9900; en) AppleWebKit/534.11+ (KHTML, like Gecko) Version/7.1.0.346 Mobile Safari/534.11+1
axios/0.27.21
Mozilla/5.0 (compatible; Googlebot/2.1; +https://www.google.com/bot.html)1
Mozilla/5.0 (compatible; wmtips.com/2.0; +http://www.wmtips.com/tools/)1
Fuzz Faster U Fool v1.5.0-dev1
Mozilla/5.0 (compatible; Google-Apps-Script; beanserver; +https://script.google.com; id: UAEmdDd-cmG1mZP6IofUJLw13vmPX3ZGnHA)1
HackerNews/1517 CFNetwork/1406.0.4 Darwin/22.4.01
Java/1.8.0_3611
python-httpx/0.23.01
Python/3.9 aiohttp/3.8.41
Python-urllib/3.81
MicroMessenger1
Mozilla/5.0 (compatible; Google-Apps-Script; beanserver; +https://script.google.com; id: UAEmdDd-mS4PbtL8QN6HNiSF29HVx2iWs4lg)1
RepoLookoutBot/v1.1.0-257-g0920a65 (abuse reports to abuse@repo-lookout.org)1
Orion/2 CFNetwork/1410.0.3 Darwin/22.6.01
Screaming Frog SEO Spider/11.21
Mozilla/5.0 (compatible; Konqueror/3.5; NetBSD 4.0_RC3; X11) KHTML/3.5.7 (like Gecko)1
FCVersion/3.0.6.526 OSVersion/10.0.19041.33241
Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)1
Python/3.11 aiohttp/3.8.61
Mozilla/5.0 (compatible; archive.org_bot +http://www.archive.org/details/archive.org_bot)1
Python/3.9 aiohttp/3.9.11
Python/3.10 aiohttp/3.9.01
`Mozilla/5.01
Mozilla/5.0 (compatible; Konqueror/4.5; FreeBSD) KHTML/4.5.4 (like Gecko)1
Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)1
Slackbot 1.0 (+https://api.slack.com/robots)1
Screaming Frog SEO Spider/8.11
SonyEricssonK310iv/R4DA Browser/NetFront/3.3 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Link/6.3.1.13.01
Googlebot/2.1 ( http://www.googlebot.com/bot.html)1
BlackBerry7100i/4.1.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 VendorID/1031
Mozilla/5.0 (X11; FreeBSD amd64) AppleWebKit/535.22+ (KHTML, like Gecko) Chromium/17.0.963.56 Chrome/17.0.963.56 Safari/535.22+ Epiphany/2.30.61
Mozilla/5.0 (X11; Ginux x86_64; rv:75.0) Gecko/20100101 Higherfox/75.01
Mozilla/5.0 (X11; FreeBSD amd64) AppleWebKit/537.4 (KHTML like Gecko) Chrome/22.0.1229.79 Safari/537.41
WebCopier v4.61
Download Demon/3.5.0.111
Mozilla/4.0 (compatible; MSIE 6.0; j2me) ReqwirelessWeb/3.51
python-urllib3/1.26.181
newspaper/0.2.81
Web Downloader/6.91
BlackBerry7520/4.0.0 Profile/MIDP-2.0 Configuration/CLDC-1.1 UP.Browser/5.0.3.3 UP.Link/5.1.2.12 (Google WAP Proxy/1.0)1
Python/3.9 aiohttp/3.9.31
Arc/1.36.0 (Mac OS X Version 14.3.1 (Build 23D60))1
shadowforce.io - sslshed/0.11
zoombot (linkbot 1.0 http://suite.seozoom.it/bot.html)1
Chrome Privacy Preserving Prefetch Proxy1
python-httpx/0.27.01
Iframely/1.3.1 (+https://iframely.com/docs/about) Canva1
Iframely/1.3.1 (+https://iframely.com/docs/about)1
SonyEricssonK550i/R1JD Browser/NetFront/3.3 Profile/MIDP-2.0 Configuration/CLDC-1.11
Source Code of user-agents.tsx
(import statements omitted for simplicity, click to expand)
import { proxy } from '../../../db/proxy.js'
import {
  getOtherUserAgents,
  getUAStatsProgress,
} from '../../../db/user-agent.js'
import SourceCode from '../components/source-code.js'
import Style from '../components/style.js'
import { o } from '../jsx/jsx.js'
function agentTable(title: string, rows: [name: string, count: number][]) {
  if (rows.length === 0) return
  rows.sort((a, b) => b[1] - a[1])
  return (
    <table>
      <thead>
        <tr>
          <th>{title}</th>
          <th>Count</th>
        </tr>
      </thead>
      <tbody>
        {[
          rows.map(([name, count]) => (
            <tr>
              <td>{name}</td>
              <td>{count}</td>
            </tr>
          )),
        ]}
      </tbody>
    </table>
  )
}

function Tables() {
  return (
    <>
      <p>{getUAStatsProgress()}</p>
      {agentTable(
        'User Agent',
        proxy.ua_type.map(row => [row.name, row.count]),
      )}
      {agentTable(
        'Bot Agent',
        proxy.ua_bot.map(row => [row.name, row.count]),
      )}
      {agentTable(
        'Other Agent',
        getOtherUserAgents().map(row => [row.user_agent, row.count]),
      )}
    </>
  )
}

let UserAgents = (
  <div id="user-agents">
    <h1>User Agents of Visitors</h1>
    {Style(/* css */ `
#user-agents table {
  border-collapse: collapse;
  margin: 1rem;
  display: inline;
}
#user-agents th,
#user-agents td {
  border: 1px solid var(--text-color, black);
  padding: 0.25rem 0.5rem;
  max-width: calc(90vw - 8rem);
  word-break: break-word;
}
`)}
    <p>This page demonstrates showing query result from database.</p>
    <p>
      Below list of user agents are collected from the visitor's HTTP header.
    </p>
    <Tables />
    <SourceCode page="user-agents.tsx" />
  </div>
)

export default UserAgents