User-agent: * Allow: / # Thank-you pages are excluded via meta robots noindex (not Disallow, so # crawlers can see the noindex directive and fully de-index them). # SEO crawlers are allowed on purpose (Ryan 2026-09-21). DSE's own audits run # through Semrush, and blocking these would break them. A competitor can see # this site through their own tools either way, so blocking costs more than it # saves. User-agent: SemrushBot Allow: / User-agent: AhrefsBot Allow: / User-agent: rogerbot Allow: / User-agent: dotbot Allow: / Sitemap: https://www.dsetransport.au/sitemap-index.xml