Understanding Usage and Engagement in AI-Powered Scientific Research Tools: The Asta Interaction Dataset

arXiv 2026 42 method

TLDR

Analyzes 200k+ user queries from two AI-powered research tools, revealing complex query patterns and collaborative usage behaviors.

Reasoning

Strengths: large-scale real-world dataset, detailed analysis of query evolution and engagement. Weaknesses: limited to two specific tools, no comparison to non-AI tools, focus on descriptive analysis rather than causal inference.

Read-first score

Read-first score 42, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 28.

Recency 8%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=analysis,dataset,evaluation

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=artifact,dataset

Topical relevance 42%

23.3

Uses existing LLM keyword relevance scores normalized to 0-100. AI scientist,automated scientific discovery,autonomous research agent,automated research,literature review agent,survey generation,automated experimentation,experiment design agent,AI for scientific research,paper writing agent,research automation,scientific discovery agent

Field roles

FrontierBridge

Rank sensitivity

Stability: volatile; rank range: 27.

Keyword Scores

AI for scientific research

literature review agent

research automation

automated scientific discovery

automated research

paper writing agent

scientific discovery agent

AI scientist

autonomous research agent

survey generation

automated experimentation

experiment design agent

Resources

Paper