Changelog Roadmap

We rebuilt the Raw Logs page around one simple idea: when something strange happens in your traffic, you should be able to see it, isolate it and understand it in seconds. The page now opens with a chart of your hits over time, switchable between line and bar view, with granularity going all the way down to one minute. And here comes our favorite part: peak isolation. Spot a spike? Click once where it starts and once where it ends, and the whole page zooms into that exact window. The chart automatically refines to the finest granularity available, so a bump that looked like a single fat bar becomes a detailed minute by minute picture. Right below the chart you'll find four new panels: Top IDs/IPs, Top Countries, Top Bots and Top Status Codes. Each one shows the top values for the range you're looking at, together with their share of the total hits. So when traffic spikes, the answer is usually right there: a single IP hammering you, a bot gone wild, a wave of 404s. Every entry is clickable, and one click turns it into a filter. Filters are the glue that holds it all together. The query builder at the top lets you filter on any dimension of your events, and everything on the page reacts at once: chart, panels and table always tell the same story. Zoom into the spike, glance at the panels, click the suspect, and you're looking at exactly the requests that caused it.

You can't stare at your dashboard all day, so from today Alerts do it for you. Pick a metric, a threshold and a time window, and we'll email you the moment your logs cross the line. Ready made templates cover the common cases: server error spikes, traffic drops, bot spikes, malicious bot traffic and slow responses. You can tweak everything and even attach the same advanced filters you use in reports to watch just a specific path or status code. We also made sure it won't flood your inbox. Every alert has a cooldown, there's a daily email cap, and before saving you can run a preview that replays your rule over recent traffic and shows exactly how many emails it would have sent. Alerts work on single sites and on whole networks, with up to ten recipients each. Creating your first one takes less than a minute.

AI crawlers are everywhere in your logs. OpenAI, Google, Perplexity and the rest hit your site thousands of times, but until now there was no way to answer the question that really matters: is any of this crawling turning into real visitors? Bot Conversions answers exactly that. For each producer, meaning the company behind one or more bots, we show you how much it crawls your site and how many human visitors its platforms send back. Attribution works on two signals. When a platform tags its outbound links, we match visits through the utm_source parameter, which gives you high precision. When there is no utm tag, we fall back to the referrer, so visits that arrive with a matching referrer source still get counted. Each visit is counted once, never twice. The Overview page gives you the full picture of which companies are scraping you the most and how many human visitors each one of them brings to your site, so you can see at a glance which bots do the heavy crawling and whether that activity brings anything back. When you want to go deeper, the Producer Analysis page lets you pick a single producer and inspect its converted paths. For every page on your site you see the referred human visits, what share of that page's total human traffic they represent, and the split between utm attributed visits and referrer only visits. Columns are sortable and you can filter by host and pathname, which makes it easy to spot which content actually earns you referrals from a given platform. There is also a new dashboard widget, Crawler to Users Conversions, that shows your top producers with crawls received and real users sent, including comparison arrows against the previous period. You'll find everything under Bot Conversions in the sidebar. Pick a date range and see who is really sending you people.

Your sitemap tells crawlers which pages matter. Until now, though, you had no way to know whether they listened. The new Sitemap section answers exactly that question. Everything starts from the Overview page. Add one or more sitemap URLs from your domain and we scan them automatically, collecting every URL they contain. Each collected URL then appears in a single table cross referenced with real traffic: total visits with a comparison against the previous period, first and last visit, and the last HTTP status code the page returned. You can sort by visits or by most recent visit, search for any URL, and filter by page type. The most interesting part is the coverage filters. In one click you can see the URLs that are in your sitemaps, the visited URLs that are missing from your sitemaps, and the sitemap URLs that received no crawls at all in the selected period. That last view is gold: those are the pages you told crawlers about that nobody is coming to read. The Page Stats page lets you zoom in on a single page. Type a pathname and an autocomplete suggests matching entries from your sitemaps, exactly like a search bar. Once selected, you get a visit trend chart, the latest bot visits on that page, and its lifecycle: when it first appeared in your sitemap and when it was last updated, or, if the page is not in any sitemap, when bots first and last reached it. Pages that receive traffic but are missing from your sitemaps are clearly marked with a Not in sitemap badge. Finally, Tags Lists lets you teach us the structure of your site. Add the URLs of your categories, authors, company pages and tags, with the same autocomplete based on your sitemap entries, and those page types become available as filters throughout the section. Add your sitemaps from the Overview page and let the first scan run. From that moment on you will always know which of your pages crawlers visit, which ones they ignore, and which ones they find even though you never listed them.

You block a bot at your CDN, move on, and never find out whether it actually stopped. The new Bots Management page closes that loop. It shows you the bots that crawled your website in the last 7 days, each with its crawl count and its category, such as AI, SEO or monitoring. For every bot you simply record how you are handling it: Allowed, Blocked or Not Managed. This page does not block or allow anything by itself. It tracks your decisions over time, and that is exactly what makes it useful. Once we know what you intended, we can compare it against real traffic and flag problems. For example, if a bot you marked as blocked is still hitting your site, that is a sign your CDN rule is not working, and we can tell you. Filters let you focus on bots in a specific state, whether allowed, blocked or not yet managed, and you can sort the list to bring the most relevant bots to the top. Everything is paginated, so even sites visited by hundreds of different bots stay easy to review. Start by going through your most frequent bots and marking the decisions you already made. From then on, you will always know whether reality matches your intentions.

Your robots.txt is the first thing AI crawlers, SEO spiders and monitoring bots read when they visit your site, yet almost nobody checks it regularly. The new Robots.txt Intelligence section does it for you. At its core is the Trends page, where you can see the current state of your robots.txt at a glance: whether it was found, is missing, or could not be reached at the last check, when it was last verified, and its full content in a clean and readable viewer. You can trigger a fresh check on demand with the Check now button, or simply let the automatic daily check keep everything up to date. If your site serves no robots.txt at all, we tell you, because in that case AI crawlers are free to access everything. Below the file itself, the Crawlers in robots.txt panel breaks the file down bot by bot. Every crawler explicitly named in your robots.txt gets its own card showing whether it is allowed or disallowed, what it is used for, and the exact Allow, Disallow and Crawl delay rules that apply to it. A search box lets you instantly find any bot by name or purpose. Since we keep previous versions of your file, each card is also compared with the prior version: bots that are new, removed or whose rules changed are clearly flagged, with the old rules shown next to the new ones. A silent edit to your robots.txt will never go unnoticed again. The page is completed by a view of the bot traffic from the last 7 days, connecting what the file says with what bots actually do. The section also brings everything together in one place. Alongside Trends you will find Details, History and Tips for your robots.txt, plus Annotations, Bots Management and the Robots Leaderboard, all reachable from the new Intelligence sidebar.