Skip to content

Filtering bad traffic

Some visits are recorded but left out of the reports, with a reason:

Reason Rule
bot-ua A declared crawler (search engines, AI crawlers) or an HTTP library such as curl or python-requests
headless An automated browser: HeadlessChrome and the like, navigator.webdriver set, or no Accept-Language header
datacenter A visit from a hosting provider's network (AWS, Google Cloud, Azure, OVH, Hetzner…)
country A country blocked by the project (its settings) or by the instance (for every project)
referrer-spam The visit arrived from a referrer-spam domain, or a subdomain of one: Matomo's public list, plus those the instance's administrators add

Visits left out are deleted after 30 days (EXCLUDED_RETENTION_DAYS).

The Excluded traffic page

Each project's Excluded traffic page shows how many visits each rule left out, a sample of them, and which networks the counted visits come from.

  • A hosting provider among those networks can be left out from there, for the project or, for an administrator, for every project.
  • It also lists the referrers of counted visits; an administrator can mark one as spam from there, for every project.

Either way, the rules are then re-applied to the period shown.

Where the rules are set

  • A project's settings: its blocked countries and its own hosting networks.
  • Administration → Filtering rules, for administrators: countries blocked for every project, and the referrer-spam domains and hosting networks that the shipped lists miss. A referrer domain can be pasted as it shows in a report, or as a whole link.
  • The shipped lists (hosting networks, crawler user agents, spam domains): plain text files in src/app/filtering/data/, which change with the application's upgrades.

Re-applying the rules

Rules apply when a visit starts. After changing a rule, re-apply them to the visits already recorded:

  • Re-apply the rules on a project's Excluded traffic page, for the period shown;
  • or abilian-analytics rules reapply, for every project's whole history (optionally --project NAME --since YYYY-MM-DD).

This re-runs the network, country and referrer rules, whose inputs every visit keeps. Visits tagged as bots or headless browsers stay as they are, since user agents aren't stored. Imported visits have no network: country and referrer only.