About the atmire-dspace-site-checker crawler

This page describes the polite crawler bot operated by Atmire. If you found us in your server logs or robots.txt hits — this is who we are and what we do.

What is it?

atmire-dspace-site-checker is the back-end of the DSpace Analyzer: a free tool that runs SEO and Google Scholar indexability checks on DSpace repositories. The tool was originally demoed at Open Repositories 2014 and has been used by DSpace administrators worldwide to diagnose indexing problems on their own installations.

What does it request?

For each repository we analyse, the bot fetches a small, well-defined set of URLs:

• The base URL of the repository (a HEAD + GET to reach the homepage).
• /robots.txt — to check whether the repository is bot-friendly for Google Scholar.
• The browse-by-title page (typically /browse?type=title or /browse/title).
• The DSpace 7+ REST roots /server/api and /server/api/discover/search/objects — to read version + item count without scraping HTML.
• Optionally one or two specific item pages, if the user pasted an item URL to inspect citation metadata.

All requests respect robots.txt rules for the atmire-dspace-site-checker token. The HTTP request body is never written to — every request is a read.

When does it run?

Two distinct trigger paths:

• User-initiated. Anyone visiting analyzer.atmire.com can paste a repository URL and get a one-off analysis. This is the most common path; one user action = one short burst of requests against the named repository.
• Periodic refresh of repositories already in our database. The analyzer keeps an internal list of repositories that have been analysed at least once. We periodically re-run the same checks against that list to keep aggregate statistics current. A repository is typically refreshed at most a few times per year.

We don't crawl your site unless someone explicitly added it via the form or your site was previously added by a user. We don't follow links beyond the URLs listed above.

How to identify our requests

Both the HTTP User-Agent header and our robots.txt product token use:

User-Agent: atmire-dspace-site-checker/1.0 (+https://analyzer.atmire.com/checker.html)

The bare token (used by RFC 9309 robots.txt parsers) is just atmire-dspace-site-checker.

How to block us

If you don't want us to crawl your repository, add either of the following to your robots.txt:

User-agent: atmire-dspace-site-checker
Disallow: /

We honour these directives on our next visit. If you want us to remove your repository entirely from our database, drop a note to bram@atmire.com with the URL.

Source code

The DSpace Site Checker is internally maintained at Atmire. There is no public source code repository at the moment. If you're interested in the analysis methodology or want to contribute, get in touch.

Last updated: 2026. Operated by atmire.com.