Day to day¶
Running the instance once it is in production. Commands run on the server as in the first deployment: hop3 run --app abilian-analytics '<command>'.
Upgrades¶
Deploy the new version; migrations run before the app starts. If one stops the start, hop3 logs --app abilian-analytics says why: for instance, the migration that lowercases emails stops when two accounts differ only by case, and names them; merge or rename one, then deploy again.
Backups¶
Everything is in PostgreSQL (the GeoIP volume can be downloaded again). Back it up daily, with Hop3's addon backup if it has one, otherwise pg_dump --format=custom from cron; try a restore once.
Health¶
Administration → Instance health after each upgrade, and when something looks off. It checks collection, GeoIP freshness, the monitoring of sites, the daily housekeeping (salts, retention) and the schema. Monitored sites not checked for 15 minutes mean the background jobs have stopped, and with them alerts and email reports: check hop3 logs.
Mail¶
The app sends alerts and email reports through the SMTP server that SMTP_HOST names, without authentication. On Hop3, the email addon gives it a local relay; the server's operator first chooses where that relay sends (a provider such as Brevo or Scaleway TEM), then:
hop3 addon email create mail --from noreply@example.com
hop3 addon attach mail --app abilian-analytics --type email
The address must be on the relay's verified domain, which needs SPF, DKIM and DMARC records, or the large mailboxes will refuse the mail or file it as spam. The addon isn't declared in hop3.toml. A mail the relay refuses is reported in the app's log, WARNING mail to … failed: or WARNING report to … failed:; a report is tried again at the next turn, an alert at the next check.
The log¶
The app logs what happens, one line per event: a site down (WARNING) or back up, an alert sent (alert by webhook: …), a report sent, a GeoIP database downloaded, a failure with its traceback. A start logs a few lines; after that, the log stays quiet while all goes well. It logs no requests: they would be the visitors' hits, which the app keeps no record of. Hop3's nginx logs them; how long it keeps them is the server's setting. In development (DEBUG), the app logs a line per request: method, path, status and time.
GeoIP¶
The app keeps the databases up to date on its own, in a daily job. When Instance health shows them more than 45 days old, check hop3 logs for jobs.geoip failed: the server can't reach db-ip.com.
Filtering lists¶
The lists of bots, hosting networks and spam referrers come with the application. After an upgrade that changes them, re-apply them to past visits: hop3 run --app abilian-analytics 'abilian-analytics rules reapply' (all projects, all history), or Re-apply the rules on a project's Excluded traffic page (for the period shown). The same goes after changing a project's blocked countries or networks, for the history before the change.
A lost password¶
abilian-analytics user set-password you@example.com --generate sets a new one and prints it, once. Without --generate, it asks for the password, which needs a terminal that hop3 run doesn't provide. Administrators can also reset others' passwords under Administration → Users.
Rate limits¶
One address can send up to 600 hits and 10 password attempts a minute, whatever the number of workers (they are counted in the database). The address is the one nginx appends to X-Forwarded-For (see the first deployment).