How to Build Reproducible Paper Alerts with OpenAlex, Semantic Scholar, and arXiv

Bottom Line

A reproducible paper alert is a small promise to your future self: when a paper shows up in your reading list, you can explain exactly why it appeared.

That is a narrower problem than verifying every result from an AI literature search tool. Here the goal is cadence, saved parameters, identifiers, deduplication, and version handling. If a commercial AI tool later suggests the same paper, you have a source-traceable baseline to compare against.

For a frontier science topic such as quantum information or quantum error correction, I would split the job across three public metadata layers: arXiv for new e-prints, OpenAlex for DOI and scholarly metadata checks, and Semantic Scholar for citation or recommendation expansion around known seed papers.

The rule is simple: do not save only titles. Save the query, endpoint, run time, result count, DOI, arXiv ID, arXiv version, journal reference status, and source API.

What Changed For Alert Design

OpenAlex announced API changes on February 24, 2026, including advanced search features, semantic search in beta, content download options, new documentation, API keys, and usage-based pricing. That update is easy to read as a pricing story, but for alert workflows the practical issue is traceability.

OpenAlex documentation lists a free key allowance, no listed cost for retrieving a single work by DOI or OpenAlex ID, and example usage prices for list-plus-filter, search, and PDF/XML content download calls. Those details should be treated as documentation-backed operating assumptions, not permanent constants. If a script stores the endpoint, request type, and usage metadata, a future pricing or policy change is easier to notice.

The search model matters too. OpenAlex describes advanced search features such as proximity search, exact matching, wildcards, and long searches up to 8KB, while semantic search is presented as beta. That makes semantic search useful for finding candidate papers, but I would not make it the primary alert trigger when the goal is reproducibility.

Give Each Source One Job

A common failure mode is making OpenAlex, Semantic Scholar, arXiv, and an AI search tool compete as if each were the same kind of answer engine. For paper alerts, they work better as separate layers.

Source	Best alert role	Fields to save	Main caution
arXiv	Track new e-prints in physics, AI, quantum information, and related categories	search_query, id_list, start, max_results, sortBy, sortOrder, arXiv ID, version, published date, updated date	Treat records as e-prints first; DOI and journal reference need separate fields
OpenAlex	Enrich and verify DOI, OpenAlex ID, venue, and open scholarly metadata	Endpoint, filter or search parameters, DOI, OpenAlex ID, API key status, usage information when available	Search and content download may have usage costs; semantic search is beta
Semantic Scholar	Expand from known seed papers through recommendations, citations, authors, and venues	Seed paper, recommendation endpoint, paper identifier, citation direction, returned paper IDs	Recommendations are candidates, not a verified alert set

The Semantic Scholar API overview describes Academic Graph coverage across papers, authors, citations, venues, and embeddings, along with recommendation capabilities. That makes it useful after you already have a seed paper or a known cluster.

The arXiv API manual supports a different role. It defines query parameters such as search_query, id_list, start, and max_results, and it returns Atom-formatted records with fields such as title, ID, published date, updated date, summary, authors, categories, journal reference, and DOI.

In fast-moving areas, the updated date can matter as much as the first appearance date. A new version of an arXiv preprint may change the abstract, author list, or downstream interpretation even when the title looks familiar.

A Concrete arXiv Alert Setup

Suppose you want a daily watchlist for quantum information papers that mention error correction. I would make the alert small enough to inspect by hand before expanding it.

A query ledger row could look like this:

Ledger field	Example value
query_name	qinfo_error_correction_daily
purpose	Track new quantum information e-prints related to error correction
source	arxiv
endpoint	arXiv API query endpoint
search_query	cat:quant-ph AND all:"error correction"
sortBy	submittedDate
sortOrder	descending
max_results	50
rerun_schedule	Once per day
dedupe_priority	DOI, then arXiv ID, then normalized title-author-year
status_labels	new, version_update, duplicate, skim, exclude

The exact query may need tuning. cat:quant-ph is a quantum physics category constraint, while the phrase search narrows the alert to records that mention error correction. The point is not that this is the perfect query. The point is that the query is explicit enough to rerun, compare, and revise.

For each returned record, store at least these fields:

Result field	Why it belongs in the table
source	Shows that the row came from arXiv, OpenAlex, Semantic Scholar, or another candidate source
query_name	Ties the paper back to a specific alert condition
run_timestamp	Separates one alert run from the next
arxiv_id	Anchors the e-print record
arxiv_version	Distinguishes a new version from a new paper
title	Useful for humans, but not enough for deduplication
authors	Helps with fallback matching when DOI is absent
published	Marks first appearance in the source record
updated	Flags later metadata or version changes
categories	Preserves subject classification from the source
doi	Enables publisher and journal metadata checks when present
journal_ref_present	Shows whether journal reference metadata appeared in the arXiv record
decision_label	Keeps your judgment separate from source metadata

This is the practical friction point: the alert should be boring enough to audit. A flashy dashboard that cannot tell whether a paper is new, updated, duplicated, or merely recommended is weaker than a plain table with stable IDs.

Deduplication Rules That Do Not Hide Updates

Start deduplication with DOI. If two records have the same DOI, they probably refer to the same scholarly object, though you should still preserve which source supplied the record.

If no DOI exists, use arXiv ID next. Keep the version separate. arxiv_id answers “which e-print?” while arxiv_version answers “which state of that e-print?”

Only after DOI and arXiv ID should you fall back to normalized title-author-year matching. That fallback is useful, but it can be too aggressive. In frontier science, the same work can appear as an arXiv preprint, a later journal article, an OpenAlex work, and a Semantic Scholar paper. Collapsing those too quickly can erase the publication path you were trying to track.

A clean rule is to deduplicate the identity of the work while preserving source-specific rows or source-specific fields. That lets you say, “this paper entered through the arXiv daily alert, later matched an OpenAlex DOI record, and also appeared in a Semantic Scholar recommendation set.”

Alert Cadence And Saved Parameters

The arXiv API manual says the same query does not need to be run more than once per day if no new articles have been added, and it recommends caching results. For a daily paper alert, that is a good default. Re-running the same query repeatedly during the day is usually noise, not diligence.

OpenAlex needs a different cadence distinction. Broad search calls are not the same as DOI or OpenAlex ID lookups. A practical pattern is to run focused searches sparingly, then use identifier lookups for enrichment when a record is already in your table.

Semantic Scholar’s official API page says many endpoints can be used without authentication, but unauthenticated requests may share limits and face throttling during congestion. Some endpoints require an API key, and the page recommends using one. For reproducible alerts, the endpoint-specific requirement belongs in the ledger rather than in someone’s memory.

At minimum, each alert run should produce a run log:

Run log field	Example
query_name	qinfo_error_correction_daily
source	arxiv
run_timestamp	Stored in your local timezone or UTC, consistently
parameters_hash	Hash of the exact saved parameters
result_count	Number of returned records
new_count	Records not previously seen by DOI or arXiv ID
version_update_count	Known arXiv IDs with changed version or updated date
duplicate_count	Records matched to existing works
error_code	Empty for success; populated for failed runs

This is what makes the alert reproducible. Not the fact that it ran automatically, but the fact that the next run can be compared to the previous one using the same rules.

Where Semantic Search Fits

Semantic search and recommendations are useful when vocabulary is unstable. That happens constantly in frontier science. A relevant paper may not use the exact phrase you expected, and a seed paper can lead to a cluster of work that keyword search misses.

Still, I would give semantic results lower authority in the alert pipeline. Use exact, fielded queries for the primary alert. Use Semantic Scholar Recommendations or OpenAlex semantic search beta as an expansion layer.

Before saving a semantic result as part of the main watchlist, return to checkable fields: DOI, arXiv ID, title, authors, version, and journal reference status. Semantic systems can widen the candidate pool, but reproducibility still comes from saved queries and identifiers.

Evidence Level

This workflow rests on official documentation and primary source descriptions, not a benchmark proving that one discovery system is best.

OpenAlex documentation supports claims about its API changes, authentication, usage model, advanced search, and beta semantic search. Semantic Scholar’s official API page supports claims about Academic Graph, Recommendations, Datasets, API keys, and access patterns. The arXiv API manual supports claims about query parameters, field prefixes, Atom responses, metadata fields, caching, and alert frequency.

What these sources do not prove is that any alert will catch every important paper. Literature discovery is not a closed problem. A reproducible alert system reduces avoidable confusion; it does not replace field knowledge, citation judgment, or actually reading the papers.

What Changes If You Build The Baseline

AI literature tools become easier to evaluate because you are no longer judging them against a vague memory of what should exist.

If an AI tool suggests a paper, match it against your table. Did it find something your arXiv alert missed? Did it surface an older related paper that would have appeared through Semantic Scholar recommendations from a seed? Did it describe an arXiv e-print without a DOI as if it were already a journal article? Did it omit the identifier you need to verify the record?

That comparison is more useful than asking whether the generated list looks plausible. It turns AI search into another candidate source that has to meet the same metadata discipline.

Common Failure Points

The first failure point is mixing preprints and journal articles into one status. arXiv records can include DOI or journal reference metadata, but the safer table design labels the source record as an e-print and stores DOI and journal reference presence separately.

The second failure point is saving only the search terms. Parameters such as start, max_results, sortBy, sortOrder, and id_list affect what the result set means. If the alert changes later, keywords alone are not enough to reconstruct the run.

The third failure point is treating bulk corpus collection and daily alerts as the same job. OpenAlex distinguishes bulk dataset access from API usage. Pulling a large corpus and checking a focused topic every morning have different cost, caching, and reproducibility concerns.

What Remains Uncertain

API policies and prices are not scientific constants. OpenAlex says pricing can change. Semantic Scholar endpoint requirements and throttling behavior need to be checked against current documentation. arXiv query limits, response details, and metadata interpretation should stay anchored to the official manual.

The other uncertainty is semantic coverage. OpenAlex semantic search is beta, and Semantic Scholar Recommendations can be useful around a seed paper, but neither proves that the returned set represents an entire field. In frontier science, terminology shifts quickly, and the same result can appear as a preprint, conference paper, journal article, dataset, or later version.

What To Watch Next

Watch for changes in OpenAlex semantic search status, OpenAlex usage conditions, Semantic Scholar endpoint requirements, and arXiv API guidance. Any of those can affect how often alerts should run, what fields should be saved, or which API calls belong in the enrichment step.

The practical finish is a query ledger, not a perfect tool choice. Pick one narrow topic, store the exact query and parameters, run it on a deliberate cadence, and make every saved paper answerable to a source, an identifier, and a version state.