{"name":"AskChem-Bench","version":"1.1","total_questions":30,"task_types":{"CA":{"name":"Cross-Paper Condition Aggregation","description":"Aggregate catalysts, conditions, and performance metrics across multiple papers"},"TC":{"name":"Temporal Claim Tracking","description":"Track how scientific understanding of a topic has evolved over time"},"CS":{"name":"Contradiction Surfacing","description":"Identify contradictory evidence and competing claims across papers"}},"methodology":{"evaluation_protocol":"GPT-5.5 answers each question with and without retrieval context. AskChem rewrites the question into 3-4 keyword sub-queries, fans them out to /api/search, diversifies the merged claim pool, and a grounded synthesiser writes the answer strictly from those claims. Paperclip unified uses the same rewriter and synthesiser but retrieves papers via Paperclip (hybrid/bm25 search over PMC+arXiv). Edison Scientific (FutureHouse paperqa3) runs on all 30 questions.","doi_verification":"Every DOI in a model answer is checked via CrossRef (existence, citation count, year). Relevance is judged 0-3 by gemini-3.1-pro-preview against the AskChem-extracted claim (claim-grounded mode) when available, otherwise against the paper's title+abstract (paper-grounded mode). Rubric: 3=directly answers, 2=on topic, 1=loosely related, 0=irrelevant. Each system is judged on what it actually surfaced — AskChem on its claims, Paperclip/Edison on their papers.","modes":[{"id":"alone","name":"LLM only","description":"GPT-5.5 with no AskChem context."},{"id":"unified","name":"+ AskChem","description":"LLM rewriter + /api/search + diversified pool + grounded synthesis."},{"id":"paperclip_unified","name":"+ Paperclip","description":"Same rewriter and GPT-5.5 synthesis as AskChem, but retrieval via Paperclip search (PMC+arXiv)."},{"id":"edison_scientific","name":"Edison Scientific","description":"FutureHouse paperqa3 on all 30 questions."},{"id":"notebooklm","name":"NotebookLM","description":"Google NotebookLM web UI (no public API). Each question pasted by hand into the NotebookLM Deep Research interface; answers stored in data/eval/notebooklm_answers.md and scored by scripts/score_external_answer.py through the same paper-grounded judge as the other systems."}],"metrics":[{"id":"doi_existence_rate","name":"DOI %","description":"Share of cited DOIs that resolve to a real CrossRef record."},{"id":"citation_density","name":"Cites/answer","description":"Average number of verified DOIs cited per answer."},{"id":"grounded_specificity","name":"Grounded specificity","description":"Quantitative tokens (yields, temps, units) that share a sentence with a citation marker."},{"id":"citation_count_mean","name":"Avg cites/paper","description":"Mean CrossRef citation count of the papers cited per answer."},{"id":"recent_high_impact_rate","name":"Recent impact","description":"Fraction of cited papers with >=50 citations published in the last 5 years."},{"id":"paper_relevance_mean","name":"Relevance (mean 0-3)","description":"Mean gemini-3.1-pro-preview judge score on the 0-3 rubric. Claim-grounded for AskChem unified; paper-grounded (title+abstract) for Paperclip and Edison — each system is judged on the evidence it actually surfaces."},{"id":"paper_relevance_high_rate","name":"On-topic (>=2 rate)","description":"Fraction of cited evidence scored >=2 by the judge ('on topic' or 'directly answers'). Complements Relevance: a system that mostly scores 2s gets a high On-topic but middling Relevance."},{"id":"edison_overlap_rate","name":"Edison overlap","description":"Per-question fraction of Edison Scientific's cited DOIs that the system also cited (recall against Edison as a retrieval baseline). 0 means no shared papers; 1 means the system found everything Edison did. Not reported for Edison's own row (trivially 1.0)."}]},"questions":[{"id":"ca01","task":"CA","domain":"coupling","question":"What catalysts and conditions have been used for C-N coupling of heteroaryl chlorides? Give specific catalysts, ligands, solvents, temperatures, and yields from the literature."},{"id":"ca02","task":"CA","domain":"coupling","question":"What catalysts, solvents, bases, and temperatures have been reported for Suzuki-Miyaura cross-coupling of aryl chlorides? Include specific yields."},{"id":"ca03","task":"CA","domain":"catalysis","question":"What homogeneous catalysts have been used for asymmetric hydrogenation? List specific catalyst systems, substrates, ee values, and conditions."},{"id":"ca04","task":"CA","domain":"electrocatalysis","question":"What electrocatalysts have been reported for CO2 reduction to CO or formate? Give specific materials, overpotentials, Faradaic efficiencies, and current densities."},{"id":"ca05","task":"CA","domain":"photocatalysis","question":"What photocatalysts and conditions have been used for visible-light-driven water splitting? Report specific materials, cocatalysts, light sources, and hydrogen evolution rates."},{"id":"ca06","task":"CA","domain":"polymerization","question":"What catalysts and conditions have been reported for ring-opening metathesis polymerization (ROMP)? Include specific initiators, monomers, solvents, and molecular weights."},{"id":"ca07","task":"CA","domain":"oxidation","question":"What catalytic systems have been used for selective oxidation of alcohols to aldehydes? Report catalysts, oxidants, solvents, temperatures, and selectivities."},{"id":"ca08","task":"CA","domain":"synthesis","question":"What conditions have been reported for Heck coupling of aryl halides with olefins? Give catalysts, ligands, bases, solvents, and temperatures with yields."},{"id":"ca09","task":"CA","domain":"biochemistry","question":"What enzyme systems have been used for biocatalytic synthesis? Report specific enzymes, substrates, products, and yields or turnover numbers."},{"id":"ca10","task":"CA","domain":"adsorption","question":"What adsorbent materials and conditions have been used for heavy metal removal from water? Report specific materials, adsorption capacities, pH, and contact times."},{"id":"tc01","task":"TC","domain":"perovskites","question":"How has the scientific understanding of perovskite degradation mechanisms evolved over time? Describe the key shifts in the field's understanding, citing specific findings and years."},{"id":"tc02","task":"TC","domain":"MOFs","question":"How has the understanding of metal-organic framework (MOF) stability in aqueous environments evolved? Trace the key findings and when they were reported."},{"id":"tc03","task":"TC","domain":"batteries","question":"How has the understanding of solid electrolyte interphase (SEI) formation in lithium-ion batteries evolved over time? Cite specific discoveries and their years."},{"id":"tc04","task":"TC","domain":"nanomedicine","question":"How has the scientific understanding of nanoparticle-protein corona formation evolved? Describe shifts in understanding with dates and citations."},{"id":"tc05","task":"TC","domain":"photocatalysis","question":"How has the mechanistic understanding of TiO2 photocatalysis evolved from its discovery to the present? Cite key findings and years."},{"id":"tc06","task":"TC","domain":"CO2_reduction","question":"How has the understanding of CO2 electrochemical reduction mechanisms and selectivity evolved? Trace the key discoveries with citations and years."},{"id":"tc07","task":"TC","domain":"drug_delivery","question":"How has the understanding of nanoparticle drug delivery systems evolved? Trace the progression of key findings with citations and years."},{"id":"tc08","task":"TC","domain":"2D_materials","question":"How has the understanding of defects in graphene and their effects on electronic properties evolved over time? Trace key discoveries."},{"id":"tc09","task":"TC","domain":"nanotoxicology","question":"How has the scientific understanding of nanoparticle cytotoxicity mechanisms evolved? Trace shifts in understanding with dates and citations."},{"id":"tc10","task":"TC","domain":"batteries_electrolyte","question":"How has the understanding of lithium battery electrolyte stability and decomposition evolved? Cite key findings and the years they appeared."},{"id":"cs01","task":"CS","domain":"nanotoxicology","question":"Are silver nanoparticles toxic or safe for biomedical use? What contradictory findings exist about the antimicrobial activity versus cytotoxicity of silver nanoparticles?"},{"id":"cs02","task":"CS","domain":"computational","question":"Which DFT functionals are most accurate for chemical calculations? Are there contradictory results about the accuracy of different density functional methods?"},{"id":"cs03","task":"CS","domain":"MOF_stability","question":"Are metal-organic frameworks stable enough for practical applications? What conflicting reports exist about MOF stability under real-world conditions?"},{"id":"cs04","task":"CS","domain":"solvent_effects","question":"Does the choice of solvent fundamentally alter reaction mechanisms? What contradictory findings exist about solvent effects on reaction selectivity and mechanism?"},{"id":"cs05","task":"CS","domain":"perovskite","question":"Does interface passivation reliably improve perovskite solar cell stability? What conflicting results exist about passivation strategies?"},{"id":"cs06","task":"CS","domain":"polymer","question":"Is polylactic acid (PLA) truly biodegradable in natural environments? What contradictory findings exist about PLA and polymer degradation rates?"},{"id":"cs07","task":"CS","domain":"graphene","question":"Does nitrogen doping improve or degrade the electrocatalytic activity of graphene? What conflicting results have been reported?"},{"id":"cs08","task":"CS","domain":"CO2_reduction","question":"Is copper the best electrocatalyst for CO2 reduction, or do alternatives outperform it? What contradictory findings exist about CO2 reduction selectivity?"},{"id":"cs09","task":"CS","domain":"batteries","question":"Do solid-state electrolytes solve the safety problems of lithium batteries, or introduce new ones? What conflicting results exist about electrolyte stability?"},{"id":"cs10","task":"CS","domain":"nanoparticle_catalysis","question":"Do smaller nanoparticles always have higher catalytic activity? What contradictory findings exist about the nanoparticle size effect in catalysis?"}],"results":{"gpt-5.5":{"CA":{"alone":{"n_questions":10,"doi_existence_rate":{"mean":0.798,"std":0.187},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":4.0,"std":1.789},"specificity_score":{"mean":25.1,"std":6.007},"grounded_specificity":{"mean":21.3,"std":6.067},"citation_count_mean":{"mean":1615.16,"std":1278.546},"citation_count_median":{"mean":1058.6,"std":513.633},"high_impact_rate":{"mean":0.867,"std":0.205},"recent_high_impact_rate":{"mean":0.0,"std":0.0},"paper_relevance_mean":{"mean":1.827,"std":0.417},"paper_relevance_high_rate":{"mean":0.798,"std":0.207},"edison_overlap_rate":{"mean":0.0,"std":0.0}},"unified":{"n_questions":10,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":13.9,"std":5.612},"specificity_score":{"mean":11.4,"std":10.042},"grounded_specificity":{"mean":11.4,"std":10.042},"citation_count_mean":{"mean":152.46,"std":92.864},"citation_count_median":{"mean":94.5,"std":54.688},"high_impact_rate":{"mean":0.446,"std":0.197},"recent_high_impact_rate":{"mean":0.146,"std":0.111},"paper_relevance_mean":{"mean":2.419,"std":0.365},"paper_relevance_high_rate":{"mean":0.938,"std":0.1},"edison_overlap_rate":{"mean":0.02,"std":0.044}},"paperclip_unified":{"n_questions":10,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":10.0,"std":3.924},"specificity_score":{"mean":1.1,"std":1.578},"grounded_specificity":{"mean":1.0,"std":1.549},"citation_count_mean":{"mean":44.01,"std":33.202},"citation_count_median":{"mean":26.25,"std":20.203},"high_impact_rate":{"mean":0.123,"std":0.177},"recent_high_impact_rate":{"mean":0.061,"std":0.111},"paper_relevance_mean":{"mean":2.351,"std":0.257},"paper_relevance_high_rate":{"mean":0.947,"std":0.101},"edison_overlap_rate":{"mean":0.013,"std":0.037}},"edison_scientific":{"n_questions":10,"doi_existence_rate":{"mean":0.983,"std":0.035},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":9.2,"std":3.37},"specificity_score":{"mean":79.9,"std":33.786},"grounded_specificity":{"mean":65.8,"std":29.233},"citation_count_mean":{"mean":215.9,"std":147.086},"citation_count_median":{"mean":113.8,"std":81.907},"high_impact_rate":{"mean":0.447,"std":0.212},"recent_high_impact_rate":{"mean":0.046,"std":0.076},"paper_relevance_mean":{"mean":2.357,"std":0.205},"paper_relevance_high_rate":{"mean":1.0,"std":0.0},"edison_overlap_rate":{"mean":0.0,"std":0.0}},"notebooklm":{"n_questions":10,"doi_existence_rate":{"mean":0.951,"std":0.089},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":7.8,"std":1.887},"specificity_score":{"mean":15.1,"std":5.647},"grounded_specificity":{"mean":0.1,"std":0.3},"citation_count_mean":{"mean":311.04,"std":273.272},"citation_count_median":{"mean":107.9,"std":94.705},"high_impact_rate":{"mean":0.384,"std":0.24},"recent_high_impact_rate":{"mean":0.119,"std":0.088},"paper_relevance_mean":{"mean":1.993,"std":0.311},"paper_relevance_high_rate":{"mean":0.916,"std":0.124},"edison_overlap_rate":{"mean":0.043,"std":0.069}}},"CS":{"alone":{"n_questions":10,"doi_existence_rate":{"mean":0.897,"std":0.127},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":6.8,"std":3.25},"specificity_score":{"mean":6.4,"std":5.181},"grounded_specificity":{"mean":1.8,"std":2.272},"citation_count_mean":{"mean":3791.99,"std":4265.144},"citation_count_median":{"mean":2147.55,"std":1034.284},"high_impact_rate":{"mean":0.982,"std":0.055},"recent_high_impact_rate":{"mean":0.013,"std":0.037},"paper_relevance_mean":{"mean":1.634,"std":0.365},"paper_relevance_high_rate":{"mean":0.612,"std":0.293},"edison_overlap_rate":{"mean":0.029,"std":0.063}},"unified":{"n_questions":10,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":16.2,"std":3.894},"specificity_score":{"mean":4.3,"std":3.551},"grounded_specificity":{"mean":4.3,"std":3.551},"citation_count_mean":{"mean":265.9,"std":128.48},"citation_count_median":{"mean":134.9,"std":66.858},"high_impact_rate":{"mean":0.56,"std":0.198},"recent_high_impact_rate":{"mean":0.208,"std":0.124},"paper_relevance_mean":{"mean":2.229,"std":0.35},"paper_relevance_high_rate":{"mean":0.873,"std":0.198},"edison_overlap_rate":{"mean":0.051,"std":0.121}},"paperclip_unified":{"n_questions":10,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":10.8,"std":2.04},"specificity_score":{"mean":0.3,"std":0.64},"grounded_specificity":{"mean":0.3,"std":0.64},"citation_count_mean":{"mean":121.88,"std":84.342},"citation_count_median":{"mean":51.4,"std":37.362},"high_impact_rate":{"mean":0.269,"std":0.15},"recent_high_impact_rate":{"mean":0.123,"std":0.119},"paper_relevance_mean":{"mean":1.889,"std":0.203},"paper_relevance_high_rate":{"mean":0.787,"std":0.151},"edison_overlap_rate":{"mean":0.036,"std":0.064}},"edison_scientific":{"n_questions":10,"doi_existence_rate":{"mean":0.992,"std":0.025},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":10.6,"std":3.262},"specificity_score":{"mean":26.1,"std":18.075},"grounded_specificity":{"mean":19.3,"std":19.147},"citation_count_mean":{"mean":396.89,"std":217.502},"citation_count_median":{"mean":196.65,"std":128.241},"high_impact_rate":{"mean":0.677,"std":0.142},"recent_high_impact_rate":{"mean":0.206,"std":0.195},"paper_relevance_mean":{"mean":2.04,"std":0.179},"paper_relevance_high_rate":{"mean":0.892,"std":0.103},"edison_overlap_rate":{"mean":0.0,"std":0.0}},"notebooklm":{"n_questions":10,"doi_existence_rate":{"mean":0.99,"std":0.03},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":8.4,"std":1.744},"specificity_score":{"mean":4.4,"std":3.47},"grounded_specificity":{"mean":0.1,"std":0.3},"citation_count_mean":{"mean":519.13,"std":436.889},"citation_count_median":{"mean":192.85,"std":312.208},"high_impact_rate":{"mean":0.471,"std":0.213},"recent_high_impact_rate":{"mean":0.185,"std":0.091},"paper_relevance_mean":{"mean":1.8,"std":0.258},"paper_relevance_high_rate":{"mean":0.737,"std":0.183},"edison_overlap_rate":{"mean":0.029,"std":0.063}}},"TC":{"alone":{"n_questions":10,"doi_existence_rate":{"mean":0.953,"std":0.06},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":17.9,"std":5.629},"specificity_score":{"mean":1.4,"std":2.417},"grounded_specificity":{"mean":1.1,"std":2.385},"citation_count_mean":{"mean":3372.93,"std":1883.704},"citation_count_median":{"mean":2197.9,"std":1374.024},"high_impact_rate":{"mean":0.984,"std":0.025},"recent_high_impact_rate":{"mean":0.005,"std":0.015},"paper_relevance_mean":{"mean":1.531,"std":0.246},"paper_relevance_high_rate":{"mean":0.566,"std":0.237},"edison_overlap_rate":{"mean":0.166,"std":0.104}},"unified":{"n_questions":10,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":24.3,"std":3.348},"specificity_score":{"mean":2.0,"std":1.549},"grounded_specificity":{"mean":2.0,"std":1.549},"citation_count_mean":{"mean":333.72,"std":121.643},"citation_count_median":{"mean":171.45,"std":52.725},"high_impact_rate":{"mean":0.68,"std":0.133},"recent_high_impact_rate":{"mean":0.201,"std":0.117},"paper_relevance_mean":{"mean":1.802,"std":0.15},"paper_relevance_high_rate":{"mean":0.793,"std":0.15},"edison_overlap_rate":{"mean":0.055,"std":0.08}},"paperclip_unified":{"n_questions":10,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":1.8,"std":0.98},"specificity_score":{"mean":0.3,"std":0.64},"grounded_specificity":{"mean":0.2,"std":0.6},"citation_count_mean":{"mean":7.64,"std":11.429},"citation_count_median":{"mean":7.3,"std":11.321},"high_impact_rate":{"mean":0.0,"std":0.0},"recent_high_impact_rate":{"mean":0.0,"std":0.0},"paper_relevance_mean":{"mean":0.925,"std":0.225},"paper_relevance_high_rate":{"mean":0.0,"std":0.0},"edison_overlap_rate":{"mean":0.0,"std":0.0}},"edison_scientific":{"n_questions":10,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":12.4,"std":3.47},"specificity_score":{"mean":3.0,"std":2.366},"grounded_specificity":{"mean":2.4,"std":2.458},"citation_count_mean":{"mean":985.72,"std":950.084},"citation_count_median":{"mean":265.8,"std":93.758},"high_impact_rate":{"mean":0.785,"std":0.125},"recent_high_impact_rate":{"mean":0.086,"std":0.083},"paper_relevance_mean":{"mean":1.816,"std":0.188},"paper_relevance_high_rate":{"mean":0.795,"std":0.187},"edison_overlap_rate":{"mean":0.0,"std":0.0}},"notebooklm":{"n_questions":10,"doi_existence_rate":{"mean":0.87,"std":0.297},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":7.6,"std":3.555},"specificity_score":{"mean":0.9,"std":0.7},"grounded_specificity":{"mean":0.1,"std":0.3},"citation_count_mean":{"mean":1436.71,"std":1447.861},"citation_count_median":{"mean":501.55,"std":357.715},"high_impact_rate":{"mean":0.657,"std":0.28},"recent_high_impact_rate":{"mean":0.058,"std":0.077},"paper_relevance_mean":{"mean":1.703,"std":0.21},"paper_relevance_high_rate":{"mean":0.707,"std":0.187},"edison_overlap_rate":{"mean":0.053,"std":0.046}}},"overall":{"alone":{"n_questions":30,"doi_existence_rate":{"mean":0.883,"std":0.15},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":9.567,"std":7.154},"specificity_score":{"mean":10.967,"std":11.268},"grounded_specificity":{"mean":8.067,"std":10.175},"citation_count_mean":{"mean":2926.693,"std":2946.323},"citation_count_median":{"mean":1801.35,"std":1161.936},"high_impact_rate":{"mean":0.944,"std":0.135},"recent_high_impact_rate":{"mean":0.006,"std":0.024},"paper_relevance_mean":{"mean":1.663,"std":0.371},"paper_relevance_high_rate":{"mean":0.658,"std":0.271},"edison_overlap_rate":{"mean":0.065,"std":0.101}},"unified":{"n_questions":30,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":18.133,"std":6.26},"specificity_score":{"mean":5.9,"std":7.391},"grounded_specificity":{"mean":5.9,"std":7.391},"citation_count_mean":{"mean":250.693,"std":137.48},"citation_count_median":{"mean":133.617,"std":66.342},"high_impact_rate":{"mean":0.562,"std":0.202},"recent_high_impact_rate":{"mean":0.185,"std":0.121},"paper_relevance_mean":{"mean":2.147,"std":0.396},"paper_relevance_high_rate":{"mean":0.866,"std":0.164},"edison_overlap_rate":{"mean":0.042,"std":0.089}},"paperclip_unified":{"n_questions":30,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":7.533,"std":4.836},"specificity_score":{"mean":0.567,"std":1.116},"grounded_specificity":{"mean":0.5,"std":1.088},"citation_count_mean":{"mean":57.843,"std":71.084},"citation_count_median":{"mean":28.317,"std":31.15},"high_impact_rate":{"mean":0.131,"std":0.173},"recent_high_impact_rate":{"mean":0.061,"std":0.106},"paper_relevance_mean":{"mean":1.721,"std":0.636},"paper_relevance_high_rate":{"mean":0.578,"std":0.427},"edison_overlap_rate":{"mean":0.016,"std":0.045}},"edison_scientific":{"n_questions":30,"doi_existence_rate":{"mean":0.991,"std":0.026},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":10.733,"std":3.614},"specificity_score":{"mean":36.333,"std":39.105},"grounded_specificity":{"mean":29.167,"std":33.581},"citation_count_mean":{"mean":532.837,"std":657.174},"citation_count_median":{"mean":192.083,"std":120.455},"high_impact_rate":{"mean":0.636,"std":0.216},"recent_high_impact_rate":{"mean":0.113,"std":0.147},"paper_relevance_mean":{"mean":2.071,"std":0.292},"paper_relevance_high_rate":{"mean":0.897,"std":0.147},"edison_overlap_rate":{"mean":0.0,"std":0.0}},"notebooklm":{"n_questions":30,"doi_existence_rate":{"mean":0.937,"std":0.187},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":7.933,"std":2.555},"specificity_score":{"mean":6.8,"std":7.162},"grounded_specificity":{"mean":0.1,"std":0.3},"citation_count_mean":{"mean":755.627,"std":1013.133},"citation_count_median":{"mean":267.433,"std":326.715},"high_impact_rate":{"mean":0.504,"std":0.271},"recent_high_impact_rate":{"mean":0.121,"std":0.1},"paper_relevance_mean":{"mean":1.837,"std":0.291},"paper_relevance_high_rate":{"mean":0.789,"std":0.191},"edison_overlap_rate":{"mean":0.042,"std":0.061}}}},"gpt-5.4":{"CA":{"alone":{"n_questions":10,"doi_existence_rate":{"mean":0.546,"std":0.305},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":2.6,"std":1.96},"specificity_score":{"mean":39.5,"std":15.022}},"strict_grounded":{"n_questions":10,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":13.2,"std":7.277},"specificity_score":{"mean":13.3,"std":14.064}},"retrieval_assisted":{"n_questions":10,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":12.9,"std":6.818},"specificity_score":{"mean":16.7,"std":16.787}},"edison_scientific":{"n_questions":3,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":12.0,"std":4.32},"specificity_score":{"mean":113.667,"std":34.422}},"grounded":{"n_questions":10,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":13.2,"std":7.277},"specificity_score":{"mean":13.3,"std":14.064}}},"CS":{"alone":{"n_questions":7,"doi_existence_rate":{"mean":0.626,"std":0.342},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":2.857,"std":1.884},"specificity_score":{"mean":4.571,"std":4.625},"contradiction_markers":{"mean":2.571,"std":2.441}},"strict_grounded":{"n_questions":7,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":11.857,"std":6.266},"specificity_score":{"mean":1.0,"std":1.195},"contradiction_markers":{"mean":1.0,"std":0.535}},"retrieval_assisted":{"n_questions":7,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":11.714,"std":7.629},"specificity_score":{"mean":0.857,"std":1.125},"contradiction_markers":{"mean":1.857,"std":1.457}},"grounded":{"n_questions":7,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":11.857,"std":6.266},"specificity_score":{"mean":1.0,"std":1.195},"contradiction_markers":{"mean":1.0,"std":0.535}}},"TC":{"alone":{"n_questions":8,"doi_existence_rate":{"mean":0.92,"std":0.065},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":10.75,"std":4.323},"specificity_score":{"mean":0.375,"std":0.696},"years_mentioned":{"mean":13.875,"std":2.666},"temporal_span":{"mean":32.625,"std":17.592}},"strict_grounded":{"n_questions":8,"doi_existence_rate":{"mean":0.995,"std":0.014},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":18.0,"std":7.483},"specificity_score":{"mean":0.75,"std":1.09},"years_mentioned":{"mean":7.875,"std":2.522},"temporal_span":{"mean":9.25,"std":3.455}},"retrieval_assisted":{"n_questions":8,"doi_existence_rate":{"mean":0.996,"std":0.01},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":22.25,"std":10.94},"specificity_score":{"mean":1.0,"std":1.414},"years_mentioned":{"mean":8.5,"std":3.571},"temporal_span":{"mean":9.375,"std":4.554}},"edison_scientific":{"n_questions":1,"doi_existence_rate":{"mean":1.0,"std":0},"doi_relevance_rate":{"mean":0.0,"std":0},"citation_density":{"mean":9.0,"std":0},"specificity_score":{"mean":2.0,"std":0},"years_mentioned":{"mean":9.0,"std":0},"temporal_span":{"mean":11.0,"std":0}},"grounded":{"n_questions":8,"doi_existence_rate":{"mean":0.995,"std":0.014},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":18.0,"std":7.483},"specificity_score":{"mean":0.75,"std":1.09},"years_mentioned":{"mean":7.875,"std":2.522},"temporal_span":{"mean":9.25,"std":3.455}}}},"gpt-4.1":{"CA":{"alone":{"doi_existence_rate":{"mean":0.547,"std":0.162},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":3.0,"std":1.183},"specificity_score":{"mean":13.3,"std":5.178}},"grounded":{"doi_existence_rate":{"mean":0.995,"std":0.016},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":15.1,"std":7.327},"specificity_score":{"mean":10.4,"std":13.002}}},"CS":{"alone":{"doi_existence_rate":{"mean":0.549,"std":0.2},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":2.9,"std":1.64},"specificity_score":{"mean":2.4,"std":2.375},"contradiction_markers":{"mean":1.3,"std":1.269}},"grounded":{"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":8.4,"std":3.852},"specificity_score":{"mean":0.5,"std":0.671},"contradiction_markers":{"mean":1.5,"std":0.922}}},"TC":{"alone":{"doi_existence_rate":{"mean":0.748,"std":0.118},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":5.7,"std":1.418},"specificity_score":{"mean":0.1,"std":0.3},"years_mentioned":{"mean":8.4,"std":1.744},"temporal_span":{"mean":27.9,"std":13.05}},"grounded":{"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":13.2,"std":4.094},"specificity_score":{"mean":0.2,"std":0.4},"years_mentioned":{"mean":7.2,"std":1.939},"temporal_span":{"mean":13.7,"std":14.464}}}}},"subset_results":{"gpt-5.5":{"balanced_18_edison":{"CA":{"alone":{"n_questions":10,"doi_existence_rate":{"mean":0.783,"std":0.187},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":3.9,"std":1.7},"specificity_score":{"mean":25.1,"std":6.007},"grounded_specificity":{"mean":21.3,"std":6.067},"citation_count_mean":{"mean":1609.15,"std":1274.651},"citation_count_median":{"mean":1052.45,"std":506.59},"high_impact_rate":{"mean":0.863,"std":0.206},"recent_high_impact_rate":{"mean":0.0,"std":0.0},"paper_relevance_mean":{"mean":1.827,"std":0.417},"paper_relevance_high_rate":{"mean":0.797,"std":0.207},"edison_overlap_rate":{"mean":0.0,"std":0.0}},"unified":{"n_questions":10,"doi_existence_rate":{"mean":0.993,"std":0.02},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":13.8,"std":5.6},"specificity_score":{"mean":11.4,"std":10.042},"grounded_specificity":{"mean":11.4,"std":10.042},"citation_count_mean":{"mean":151.79,"std":92.841},"citation_count_median":{"mean":94.05,"std":54.542},"high_impact_rate":{"mean":0.441,"std":0.203},"recent_high_impact_rate":{"mean":0.146,"std":0.111},"paper_relevance_mean":{"mean":2.419,"std":0.365},"paper_relevance_high_rate":{"mean":0.938,"std":0.1},"edison_overlap_rate":{"mean":0.02,"std":0.044}},"paperclip_unified":{"n_questions":10,"doi_existence_rate":{"mean":0.992,"std":0.023},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":9.9,"std":3.859},"specificity_score":{"mean":1.1,"std":1.578},"grounded_specificity":{"mean":1.0,"std":1.549},"citation_count_mean":{"mean":44.07,"std":33.037},"citation_count_median":{"mean":26.35,"std":20.184},"high_impact_rate":{"mean":0.123,"std":0.177},"recent_high_impact_rate":{"mean":0.061,"std":0.111},"paper_relevance_mean":{"mean":2.351,"std":0.257},"paper_relevance_high_rate":{"mean":0.947,"std":0.101},"edison_overlap_rate":{"mean":0.014,"std":0.043}},"edison_scientific":{"n_questions":10,"doi_existence_rate":{"mean":0.942,"std":0.076},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":8.9,"std":3.506},"specificity_score":{"mean":79.9,"std":33.786},"grounded_specificity":{"mean":65.8,"std":29.233},"citation_count_mean":{"mean":225.12,"std":161.772},"citation_count_median":{"mean":132.95,"std":123.282},"high_impact_rate":{"mean":0.464,"std":0.248},"recent_high_impact_rate":{"mean":0.046,"std":0.076},"paper_relevance_mean":{"mean":2.357,"std":0.205},"paper_relevance_high_rate":{"mean":1.0,"std":0.0},"edison_overlap_rate":{"mean":null,"std":null}}},"CS":{"alone":{"n_questions":10,"doi_existence_rate":{"mean":0.875,"std":0.123},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":6.6,"std":3.137},"specificity_score":{"mean":6.4,"std":5.181},"grounded_specificity":{"mean":1.8,"std":2.272},"citation_count_mean":{"mean":3772.44,"std":4274.138},"citation_count_median":{"mean":2166.15,"std":1012.972},"high_impact_rate":{"mean":0.98,"std":0.06},"recent_high_impact_rate":{"mean":0.013,"std":0.038},"paper_relevance_mean":{"mean":1.634,"std":0.365},"paper_relevance_high_rate":{"mean":0.612,"std":0.293},"edison_overlap_rate":{"mean":0.029,"std":0.063},"contradiction_markers":{"mean":1.9,"std":1.64}},"unified":{"n_questions":10,"doi_existence_rate":{"mean":0.983,"std":0.026},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":15.9,"std":3.78},"specificity_score":{"mean":4.3,"std":3.551},"grounded_specificity":{"mean":4.3,"std":3.551},"citation_count_mean":{"mean":266.23,"std":128.465},"citation_count_median":{"mean":139.1,"std":70.886},"high_impact_rate":{"mean":0.565,"std":0.204},"recent_high_impact_rate":{"mean":0.204,"std":0.117},"paper_relevance_mean":{"mean":2.229,"std":0.35},"paper_relevance_high_rate":{"mean":0.873,"std":0.198},"edison_overlap_rate":{"mean":0.051,"std":0.121},"contradiction_markers":{"mean":1.1,"std":0.7}},"paperclip_unified":{"n_questions":10,"doi_existence_rate":{"mean":0.99,"std":0.03},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":10.7,"std":2.1},"specificity_score":{"mean":0.3,"std":0.64},"grounded_specificity":{"mean":0.3,"std":0.64},"citation_count_mean":{"mean":110.25,"std":78.791},"citation_count_median":{"mean":51.3,"std":37.264},"high_impact_rate":{"mean":0.261,"std":0.15},"recent_high_impact_rate":{"mean":0.123,"std":0.119},"paper_relevance_mean":{"mean":1.889,"std":0.203},"paper_relevance_high_rate":{"mean":0.787,"std":0.151},"edison_overlap_rate":{"mean":0.036,"std":0.064},"contradiction_markers":{"mean":0.6,"std":0.663}},"edison_scientific":{"n_questions":10,"doi_existence_rate":{"mean":0.992,"std":0.025},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":10.6,"std":3.262},"specificity_score":{"mean":26.1,"std":18.075},"grounded_specificity":{"mean":19.3,"std":19.147},"citation_count_mean":{"mean":396.03,"std":217.523},"citation_count_median":{"mean":195.6,"std":127.642},"high_impact_rate":{"mean":0.668,"std":0.145},"recent_high_impact_rate":{"mean":0.206,"std":0.195},"paper_relevance_mean":{"mean":2.04,"std":0.179},"paper_relevance_high_rate":{"mean":0.892,"std":0.103},"edison_overlap_rate":{"mean":null,"std":null},"contradiction_markers":{"mean":3.2,"std":2.441}}},"TC":{"alone":{"n_questions":10,"doi_existence_rate":{"mean":0.921,"std":0.075},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":17.3,"std":5.658},"specificity_score":{"mean":1.4,"std":2.417},"grounded_specificity":{"mean":1.1,"std":2.385},"citation_count_mean":{"mean":3401.92,"std":1901.2},"citation_count_median":{"mean":2194.45,"std":1319.96},"high_impact_rate":{"mean":0.984,"std":0.025},"recent_high_impact_rate":{"mean":0.005,"std":0.016},"paper_relevance_mean":{"mean":1.531,"std":0.246},"paper_relevance_high_rate":{"mean":0.566,"std":0.237},"edison_overlap_rate":{"mean":0.179,"std":0.113},"years_mentioned":{"mean":13.4,"std":2.577},"temporal_span":{"mean":33.6,"std":13.566}},"unified":{"n_questions":10,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":24.3,"std":3.348},"specificity_score":{"mean":2.0,"std":1.549},"grounded_specificity":{"mean":2.0,"std":1.549},"citation_count_mean":{"mean":333.04,"std":121.616},"citation_count_median":{"mean":171.1,"std":52.498},"high_impact_rate":{"mean":0.68,"std":0.133},"recent_high_impact_rate":{"mean":0.201,"std":0.117},"paper_relevance_mean":{"mean":1.802,"std":0.15},"paper_relevance_high_rate":{"mean":0.794,"std":0.15},"edison_overlap_rate":{"mean":0.06,"std":0.085},"years_mentioned":{"mean":10.7,"std":1.792},"temporal_span":{"mean":17.6,"std":11.534}},"paperclip_unified":{"n_questions":10,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":1.8,"std":0.98},"specificity_score":{"mean":0.3,"std":0.64},"grounded_specificity":{"mean":0.2,"std":0.6},"citation_count_mean":{"mean":7.64,"std":11.429},"citation_count_median":{"mean":7.3,"std":11.321},"high_impact_rate":{"mean":0.0,"std":0.0},"recent_high_impact_rate":{"mean":0.0,"std":0.0},"paper_relevance_mean":{"mean":0.925,"std":0.225},"paper_relevance_high_rate":{"mean":0.0,"std":0.0},"edison_overlap_rate":{"mean":0.0,"std":0.0},"years_mentioned":{"mean":1.6,"std":0.663},"temporal_span":{"mean":3.2,"std":6.416}},"edison_scientific":{"n_questions":10,"doi_existence_rate":{"mean":0.935,"std":0.074},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":11.6,"std":3.353},"specificity_score":{"mean":3.0,"std":2.366},"grounded_specificity":{"mean":2.4,"std":2.458},"citation_count_mean":{"mean":987.59,"std":951.988},"citation_count_median":{"mean":269.75,"std":108.311},"high_impact_rate":{"mean":0.776,"std":0.136},"recent_high_impact_rate":{"mean":0.089,"std":0.085},"paper_relevance_mean":{"mean":1.816,"std":0.188},"paper_relevance_high_rate":{"mean":0.795,"std":0.187},"edison_overlap_rate":{"mean":null,"std":null},"years_mentioned":{"mean":11.1,"std":2.625},"temporal_span":{"mean":33.2,"std":16.732}}}}},"gpt-5.4":{"balanced_9":{"CA":{"alone":{"n_questions":3,"doi_existence_rate":{"mean":0.25,"std":0.354},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":1.0,"std":1.414},"specificity_score":{"mean":51.333,"std":12.658}},"strict_grounded":{"n_questions":3,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":9.333,"std":5.793},"specificity_score":{"mean":16.0,"std":14.967}},"retrieval_assisted":{"n_questions":3,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":9.333,"std":5.793},"specificity_score":{"mean":14.667,"std":13.199}},"edison_scientific":{"n_questions":3,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":12.0,"std":4.32},"specificity_score":{"mean":113.667,"std":34.422}},"grounded":{"n_questions":3,"doi_existence_rate":{"mean":1.0,"std":0.0},"doi_relevance_rate":{"mean":0.0,"std":0.0},"citation_density":{"mean":9.333,"std":5.793},"specificity_score":{"mean":16.0,"std":14.967}}},"TC":{"alone":{"n_questions":1,"doi_existence_rate":{"mean":1.0,"std":0},"doi_relevance_rate":{"mean":0.0,"std":0},"citation_density":{"mean":6.0,"std":0},"specificity_score":{"mean":1.0,"std":0},"years_mentioned":{"mean":11.0,"std":0},"temporal_span":{"mean":10.0,"std":0}},"strict_grounded":{"n_questions":1,"doi_existence_rate":{"mean":1.0,"std":0},"doi_relevance_rate":{"mean":0.0,"std":0},"citation_density":{"mean":18.0,"std":0},"specificity_score":{"mean":0.0,"std":0},"years_mentioned":{"mean":7.0,"std":0},"temporal_span":{"mean":7.0,"std":0}},"retrieval_assisted":{"n_questions":1,"doi_existence_rate":{"mean":1.0,"std":0},"doi_relevance_rate":{"mean":0.0,"std":0},"citation_density":{"mean":33.0,"std":0},"specificity_score":{"mean":2.0,"std":0},"years_mentioned":{"mean":11.0,"std":0},"temporal_span":{"mean":11.0,"std":0}},"edison_scientific":{"n_questions":1,"doi_existence_rate":{"mean":1.0,"std":0},"doi_relevance_rate":{"mean":0.0,"std":0},"citation_density":{"mean":9.0,"std":0},"specificity_score":{"mean":2.0,"std":0},"years_mentioned":{"mean":9.0,"std":0},"temporal_span":{"mean":11.0,"std":0}},"grounded":{"n_questions":1,"doi_existence_rate":{"mean":1.0,"std":0},"doi_relevance_rate":{"mean":0.0,"std":0},"citation_density":{"mean":18.0,"std":0},"specificity_score":{"mean":0.0,"std":0},"years_mentioned":{"mean":7.0,"std":0},"temporal_span":{"mean":7.0,"std":0}}}}},"gpt-4.1":{}},"runs":{"gpt-5.5":{"generated_at":"2026-05-19T21:05:30+00:00","methods":{"llm_alone":{"label":"GPT-only baseline","description":"Reuse existing GPT answers with no AskChem retrieval."},"unified":{"label":"AskChem (LLM rewriter + hybrid /api/search + grounded synthesis)","description":"Canonical AskChem usage: a small LLM rewriter turns the question into 3-4 short keyword sub-queries, fans them out to the single hybrid retrieval endpoint /api/search (which already exploits FTS, paper-level, taxonomy, and vector signals via RRF), merges and diversifies the claim pool to <= 40 claims with <= 4 per source, then the answer head synthesises strictly from those claims."},"edison_scientific":{"label":"Edison Scientific literature baseline","description":"External literature synthesis baseline (all 30 questions)."},"paperclip_unified":{"label":"Paperclip (hybrid/bm25 search + GPT-5.5 synthesis)","description":"Same rewriter and synthesis as AskChem unified, but retrieval uses Paperclip search (ranking hybrid by default, bm25 for short keyword sub-queries, lookup for DOIs) over pmc+arxiv, then GPT-5.5 grounded synthesis."}},"subsets":{"balanced_9":{"description":"Balanced Edison subset with 3 questions each from CA, TC, and CS.","question_ids":["ca02","ca04","ca10","tc01","tc03","tc05","cs01","cs03","cs08","ca05","ca08","ca09","tc02","tc06","tc08","cs02","cs05","cs09","ca01","ca03","ca06","ca07","tc04","tc07","tc09","tc10","cs04","cs06","cs07","cs10"]}},"askchem_snapshot":{"api":"https://askchem.org/api","retrieved_at":"2026-05-19T20:41:37+00:00","stats":{"total_claims":2337403,"total_sources":140913,"total_views":7,"total_nodes":110626,"claim_types":{"property":938871,"method":322810,"comparison":225223,"mechanism":213888,"computational_result":190376,"reaction":121182,"limitation":52971,"surprising_finding":48461,"experimental_design":46359,"scope_entry":43834,"structure":43505,"future_direction":37270,"hypothesis":31117,"conclusion":12287,"observation":4901,"measurement":384,"equation":331,"performance":316,"parameter":306,"experimental_result":296,"result":269,"model":255,"conclusions":191,"optimization":182,"background":175,"theory":131,"application":106,"data_point":70,"definition":69,"prediction":59,"composition":48,"control_experiment":44,"background_information":40,"relationship":40,"design":38,"theoretical_result":35,"metadata":34,"correlation":33,"outcome":33,"assumption":30,"data":29,"claim":26,"computational_design":26,"negative_result":25,"phenomenon":24,"calculation":23,"material":22,"optimization_entry":21,"performance_metric":21,"derivation":20,"control":19,"model_parameter":18,"concept":17,"behavior":15,"historical":14,"rule":13,"classification":12,"condition":12,"interpretation":12,"material_property":12,"mathematical_result":12,"capability":11,"finding":11,"problem":11,"limit":10,"computational_method":8,"mathematical_derivation":8,"trend":8,"advantage":7,"formula":7,"other":7,"problem_statement":7,"recommendation":7,"analytical_result":6,"approximation":6,"background_knowledge":6,"explanation":6,"implication":6,"literature_result":6,"reference_value":6,"spectroscopic":6,"theoretical_model":6,"conparison":5,"discovery":5,"goal":5,"literature_value":5,"scaling_property":5,"theorem":5,"context":4,"dataset_description":4,"design_principle":4,"device":4,"equation_or_formula":4,"experimental_finding":4,"material_preparation":4,"model_definition":4,"parameter_definition":4,"parameter_fit":4,"statistical_result":4,"validation":4,"architecture":3,"background_property":3,"claim_type:method":3,"design_rule":3,"empirical_result":3,"equation_derivation":3,"historical_fact":3,"historical_milestone":3,"interaction":3,"kinetics":3,"mathematical_relation":3,"parameter_extraction":3,"parameter_setting":3,"property|comparison":3,"scaling_law":3,"threshold":3,"application|method":2,"application|reaction":2,"background/comparison":2,"background_theory":2,"challenge":2,"claim_type":2,"claim_type:comparison":2,"conputational_result":2,"correction":2,"critique":2,"dynamic_behavior":2,"extrapolation":2,"general":2,"limitations":2,"mechanism|property":2,"method|comparison":2,"metric":2,"model_approximation":2,"model_design":2,"outlook":2,"proof":2,"property|mechanism":2,"purpose":2,"reaction|comparison":2,"reasoning":2,"regime_definition":2,"relation":2,"requirement":2,"structural_response":2,"synthesis":2,"system":2,"theoretical_prediction":2,"analysis":1,"application/method":1,"application/reaction":1,"application|comparison":1,"artifact":1,"assignment":1,"background/property":1,"band_structure":1,"benchmark":1,"benefit":1,"biological_property":1,"claim (conclusion)":1,"claim (method/property)":1,"comparative_result":1,"comparision":1,"comparison|mechanism":1,"comparison|mechanism|computational_result":1,"comparison|property":1,"computational_study":1,"conclusion|property":1,"conditions":1,"constraint":1,"dataset":1,"demonstration":1,"design_concept":1,"design_parameter":1,"example":1,"experimental_parameter":1,"field_observations":1,"finding|property":1,"future_opportunity":1,"future_scope":1,"general_fact":1,"goal_statement":1,"historical/comparison":1,"historical_comparison":1,"historical_trend":1,"history":1,"implication/method":1,"initial_state_assignment":1,"literature_observation":1,"material_composition":1,"material_proposal":1,"material_synthesis":1,"mathematical_property":1,"mechanism|computational_result":1,"meta":1,"metadata_based_method":1,"methodology_outlook":1,"method|computational_result":1,"method|computational_result|property":1,"method|prediction":1,"method|property":1,"model_derivation":1,"model_property":1,"model_result":1,"model_structure":1,"motivation":1,"novelty":1,"novelty/comparison":1,"optimization_result":1,"outcome/method":1,"outcome|method":1,"outcome|property":1,"paper_type":1,"parameter_estimation":1,"principle":1,"prior_work":1,"problem/mechanism":1,"problem/property":1,"problem_description":1,"product":1,"projection/method":1,"property|comparison|mechanism":1,"property|method":1,"rationale":1,"reaction|computational_result":1,"reaction|mechanism":1,"reaction|method":1,"recommendation|mechanism":1,"reference":1,"representation":1,"scaling":1,"scope":1,"scope/property":1,"scope/reaction":1,"scope|property":1,"significance":1,"solution":1,"synthesis_condition":1,"theoretical":1,"theoretical_principle":1,"tradeoff":1,"trajectory":1},"year_distribution":{"1925":3,"1926":1,"1927":2,"1928":2,"1929":1,"1930":2,"1931":1,"1932":1,"1937":1,"1940":1,"1941":1,"1942":1,"1943":1,"1944":1,"1949":1,"1953":1,"1954":1,"1958":1,"1959":1,"1960":1,"1961":2,"1962":1,"1963":1,"1964":1,"1965":2,"1967":4,"1968":2,"1969":5,"1970":4,"1971":3,"1972":9,"1973":6,"1974":14,"1975":12,"1976":9,"1977":10,"1978":6,"1979":15,"1980":11,"1981":16,"1982":13,"1983":21,"1984":17,"1985":27,"1986":20,"1987":28,"1988":33,"1989":32,"1990":45,"1991":45,"1992":55,"1993":44,"1994":69,"1995":138,"1996":162,"1997":206,"1998":262,"1999":267,"2000":302,"2001":419,"2002":496,"2003":614,"2004":755,"2005":848,"2006":856,"2007":1070,"2008":1102,"2009":1217,"2010":1461,"2011":1768,"2012":1993,"2013":2150,"2014":2621,"2015":4782,"2016":5123,"2017":5921,"2018":11122,"2019":16415,"2020":18380,"2021":14581,"2022":12172,"2023":12596,"2024":11883,"2025":6438,"2026":22},"citation_source":"","citation_source_url":"","citations_updated_at":""},"quality":{"total_claims":2337403,"total_sources":140913,"claim_type_distribution":{"property":938871,"method":322810,"comparison":225223,"mechanism":213888,"computational_result":190376,"reaction":121182,"limitation":52971,"surprising_finding":48461,"experimental_design":46359,"scope_entry":43834,"structure":43505,"future_direction":37270,"hypothesis":31117,"conclusion":12287,"observation":4901,"measurement":384,"equation":331,"performance":316,"parameter":306,"experimental_result":296,"result":269,"model":255,"conclusions":191,"optimization":182,"background":175,"theory":131,"application":106,"data_point":70,"definition":69,"prediction":59,"composition":48,"control_experiment":44,"background_information":40,"relationship":40,"design":38,"theoretical_result":35,"metadata":34,"correlation":33,"outcome":33,"assumption":30,"data":29,"claim":26,"computational_design":26,"negative_result":25,"phenomenon":24,"calculation":23,"material":22,"optimization_entry":21,"performance_metric":21,"derivation":20,"control":19,"model_parameter":18,"concept":17,"behavior":15,"historical":14,"rule":13,"classification":12,"condition":12,"interpretation":12,"material_property":12,"mathematical_result":12,"capability":11,"finding":11,"problem":11,"limit":10,"computational_method":8,"mathematical_derivation":8,"trend":8,"advantage":7,"formula":7,"other":7,"problem_statement":7,"recommendation":7,"analytical_result":6,"approximation":6,"background_knowledge":6,"explanation":6,"implication":6,"literature_result":6,"reference_value":6,"spectroscopic":6,"theoretical_model":6,"conparison":5,"discovery":5,"goal":5,"literature_value":5,"scaling_property":5,"theorem":5,"context":4,"dataset_description":4,"design_principle":4,"device":4,"equation_or_formula":4,"experimental_finding":4,"material_preparation":4,"model_definition":4,"parameter_definition":4,"parameter_fit":4,"statistical_result":4,"validation":4,"architecture":3,"background_property":3,"claim_type:method":3,"design_rule":3,"empirical_result":3,"equation_derivation":3,"historical_fact":3,"historical_milestone":3,"interaction":3,"kinetics":3,"mathematical_relation":3,"parameter_extraction":3,"parameter_setting":3,"property|comparison":3,"scaling_law":3,"threshold":3,"application|method":2,"application|reaction":2,"background/comparison":2,"background_theory":2,"challenge":2,"claim_type":2,"claim_type:comparison":2,"conputational_result":2,"correction":2,"critique":2,"dynamic_behavior":2,"extrapolation":2,"general":2,"limitations":2,"mechanism|property":2,"method|comparison":2,"metric":2,"model_approximation":2,"model_design":2,"outlook":2,"proof":2,"property|mechanism":2,"purpose":2,"reaction|comparison":2,"reasoning":2,"regime_definition":2,"relation":2,"requirement":2,"structural_response":2,"synthesis":2,"system":2,"theoretical_prediction":2,"analysis":1,"application/method":1,"application/reaction":1,"application|comparison":1,"artifact":1,"assignment":1,"background/property":1,"band_structure":1,"benchmark":1,"benefit":1,"biological_property":1,"claim (conclusion)":1,"claim (method/property)":1,"comparative_result":1,"comparision":1,"comparison|mechanism":1,"comparison|mechanism|computational_result":1,"comparison|property":1,"computational_study":1,"conclusion|property":1,"conditions":1,"constraint":1,"dataset":1,"demonstration":1,"design_concept":1,"design_parameter":1,"example":1,"experimental_parameter":1,"field_observations":1,"finding|property":1,"future_opportunity":1,"future_scope":1,"general_fact":1,"goal_statement":1,"historical/comparison":1,"historical_comparison":1,"historical_trend":1,"history":1,"implication/method":1,"initial_state_assignment":1,"literature_observation":1,"material_composition":1,"material_proposal":1,"material_synthesis":1,"mathematical_property":1,"mechanism|computational_result":1,"meta":1,"metadata_based_method":1,"methodology_outlook":1,"method|computational_result":1,"method|computational_result|property":1,"method|prediction":1,"method|property":1,"model_derivation":1,"model_property":1,"model_result":1,"model_structure":1,"motivation":1,"novelty":1,"novelty/comparison":1,"optimization_result":1,"outcome/method":1,"outcome|method":1,"outcome|property":1,"paper_type":1,"parameter_estimation":1,"principle":1,"prior_work":1,"problem/mechanism":1,"problem/property":1,"problem_description":1,"product":1,"projection/method":1,"property|comparison|mechanism":1,"property|method":1,"rationale":1,"reaction|computational_result":1,"reaction|mechanism":1,"reaction|method":1,"recommendation|mechanism":1,"reference":1,"representation":1,"scaling":1,"scope":1,"scope/property":1,"scope/reaction":1,"scope|property":1,"significance":1,"solution":1,"synthesis_condition":1,"theoretical":1,"theoretical_principle":1,"tradeoff":1,"trajectory":1},"year_range":[1925,2026],"year_distribution":{"1925":3,"1926":1,"1927":2,"1928":2,"1929":1,"1930":2,"1931":1,"1932":1,"1937":1,"1940":1,"1941":1,"1942":1,"1943":1,"1944":1,"1949":1,"1953":1,"1954":1,"1958":1,"1959":1,"1960":1,"1961":2,"1962":1,"1963":1,"1964":1,"1965":2,"1967":4,"1968":2,"1969":5,"1970":4,"1971":3,"1972":9,"1973":6,"1974":14,"1975":12,"1976":9,"1977":10,"1978":6,"1979":15,"1980":11,"1981":16,"1982":13,"1983":21,"1984":17,"1985":27,"1986":20,"1987":28,"1988":33,"1989":32,"1990":45,"1991":45,"1992":55,"1993":44,"1994":69,"1995":138,"1996":162,"1997":206,"1998":262,"1999":267,"2000":302,"2001":419,"2002":496,"2003":614,"2004":755,"2005":848,"2006":856,"2007":1070,"2008":1102,"2009":1217,"2010":1461,"2011":1768,"2012":1993,"2013":2150,"2014":2621,"2015":4782,"2016":5123,"2017":5921,"2018":11122,"2019":16415,"2020":18380,"2021":14581,"2022":12172,"2023":12596,"2024":11883,"2025":6438,"2026":22},"extraction_depth":{"full_paper":1515538,"abstract_only":821372,"full_paper_by_model":{"gemini-3.1-pro":1515538},"full_paper_papers":41076},"doi_verification":{"total_validated":140913,"crossref_verified":140836,"retracted_caught":38,"verification_rate":99.9},"subfield_coverage":{"materials":568084,"other":13084,"computational":188417,"biochemistry":160729,"electrocatalysis":419,"physical_chemistry":188800,"catalysis":67924,"electrochemistry":86624,"analytical":48835,"polymer":26466,"inorganic":12964,"organic_synthesis":34891,"environmental":43345,"photochemistry":14804,"atmospheric_chemistry":635,"green_chemistry":237,"chemical_biology":1214,"methods_paper":157,"medicinal_chemistry":2291,"biophysics":130,"optical":34,"computational_electrochemistry":30,"biocatalysis":102,"supramolecular_chemistry":141,"computational_study":94,"medicinal":83,"spintronics":50,"bioinorganic":125,"organometallic|computational|catalysis":66,"superconductivity":22,"nanomedicine":27,"food_chemistry":251,"combustion":62,"atmospheric|combustion|physical_chemistry|analytical":57,"energy|catalysis|environmental":60,"biosensors":50,"radiochemistry":61,"organometallic_chemistry":37,"geochemistry":63,"instrumentation":16,"atmospheric":110},"flag_stats":{"total":0,"resolved":0,"open":0},"smiles_validation":{"total":0,"valid":0,"invalid":0},"extraction_models":["gemini-3.1-pro (full paper PDF extraction + classification)","gpt-5-mini (abstract extraction)"],"citation_source":"Semantic Scholar Academic Graph API","citation_source_url":"https://api.semanticscholar.org/","citations_updated_at":"","known_limitations":["SMILES validation not yet run on all claims","Contradiction detection: 556 confirmed contradictions detected via PAW pre-filter + Gemini verification","Abstract-only extractions capture headline findings; full-paper extractions from Gemini-3.1-pro-preview and GPT-5.4 include tables, conditions, and mechanistic detail"]}}},"gpt-5.4":{"generated_at":"2026-04-13T19:53:16+00:00","methods":{"llm_alone":{"label":"GPT-only baseline","description":"Reuse existing GPT answers with no AskChem retrieval."},"strict_grounded":{"label":"AskChem strict_grounded","description":"Use only retrieved AskChem claims as evidence."},"retrieval_assisted":{"label":"AskChem retrieval_assisted","description":"Use AskChem claims as primary evidence, with broader diversified retrieval."},"edison_scientific":{"label":"Edison Scientific literature baseline","description":"External literature synthesis baseline on the balanced 9-question subset."}},"subsets":{"balanced_9":{"description":"Balanced Edison subset with 3 questions each from CA, TC, and CS.","question_ids":["ca02","ca04","ca10","tc01","tc03","tc05","cs01","cs03","cs08"]}},"askchem_snapshot":{"api":"https://askchem.org/api","retrieved_at":"2026-04-13T18:19:37+00:00","stats":{"total_claims":1766989,"total_sources":123987,"total_views":7,"total_nodes":4117,"claim_types":{"property":552675,"method":298596,"computational_result":219005,"comparison":211947,"mechanism":178193,"reaction":66514,"scope_entry":46971,"experimental_design":39203,"limitation":34848,"structure":33631,"surprising_finding":29834,"future_direction":18674,"hypothesis":16757,"conclusion":11945,"observation":3970,"measurement":383,"equation":320,"parameter":306,"performance":297,"experimental_result":293,"result":262,"model":255,"conclusions":191,"background":175,"optimization":155,"theory":131,"application":104,"data_point":70,"definition":68,"prediction":59,"composition":47,"control_experiment":44,"background_information":40,"relationship":39,"theoretical_result":35,"metadata":34,"design":33,"assumption":30,"correlation":30,"data":29,"claim":26,"computational_design":26,"negative_result":25,"phenomenon":24,"calculation":23,"material":22,"performance_metric":21,"derivation":20,"control":19,"outcome":19,"model_parameter":18,"concept":17,"behavior":15,"historical":14,"rule":13,"classification":12,"condition":12,"interpretation":12,"material_property":12,"mathematical_result":12,"capability":11,"problem":11,"finding":10,"limit":10,"computational_method":8,"mathematical_derivation":8,"trend":8,"advantage":7,"formula":7,"other":7,"problem_statement":7,"recommendation":7,"analytical_result":6,"approximation":6,"background_knowledge":6,"explanation":6,"implication":6,"literature_result":6,"optimization_entry":6,"reference_value":6,"theoretical_model":6,"conparison":5,"discovery":5,"goal":5,"literature_value":5,"scaling_property":5,"theorem":5,"context":4,"dataset_description":4,"design_principle":4,"device":4,"equation_or_formula":4,"experimental_finding":4,"material_preparation":4,"model_definition":4,"parameter_definition":4,"parameter_fit":4,"statistical_result":4,"validation":4,"architecture":3,"background_property":3,"claim_type:method":3,"design_rule":3,"empirical_result":3,"equation_derivation":3,"historical_fact":3,"historical_milestone":3,"kinetics":3,"mathematical_relation":3,"parameter_extraction":3,"parameter_setting":3,"property|comparison":3,"scaling_law":3,"threshold":3,"application|method":2,"application|reaction":2,"background/comparison":2,"background_theory":2,"challenge":2,"claim_type":2,"claim_type:comparison":2,"conputational_result":2,"correction":2,"critique":2,"extrapolation":2,"general":2,"limitations":2,"mechanism|property":2,"method|comparison":2,"metric":2,"model_approximation":2,"model_design":2,"outlook":2,"proof":2,"property|mechanism":2,"purpose":2,"reaction|comparison":2,"reasoning":2,"regime_definition":2,"relation":2,"requirement":2,"spectroscopic":2,"structural_response":2,"synthesis":2,"theoretical_prediction":2,"analysis":1,"application/method":1,"application/reaction":1,"application|comparison":1,"artifact":1,"assignment":1,"background/property":1,"band_structure":1,"benchmark":1,"benefit":1,"biological_property":1,"claim (conclusion)":1,"claim (method/property)":1,"comparative_result":1,"comparision":1,"comparison|mechanism":1,"comparison|mechanism|computational_result":1,"comparison|property":1,"computational_study":1,"conclusion|property":1,"conditions":1,"constraint":1,"dataset":1,"demonstration":1,"design_concept":1,"design_parameter":1,"example":1,"experimental_parameter":1,"field_observations":1,"finding|property":1,"future_opportunity":1,"future_scope":1,"general_fact":1,"goal_statement":1,"historical/comparison":1,"historical_comparison":1,"historical_trend":1,"history":1,"implication/method":1,"initial_state_assignment":1,"literature_observation":1,"material_composition":1,"material_proposal":1,"material_synthesis":1,"mathematical_property":1,"mechanism|computational_result":1,"meta":1,"metadata_based_method":1,"methodology_outlook":1,"method|computational_result":1,"method|computational_result|property":1,"method|prediction":1,"method|property":1,"model_derivation":1,"model_property":1,"model_result":1,"model_structure":1,"motivation":1,"novelty":1,"novelty/comparison":1,"optimization_result":1,"outcome/method":1,"outcome|method":1,"outcome|property":1,"paper_type":1,"parameter_estimation":1,"principle":1,"prior_work":1,"problem/mechanism":1,"problem/property":1,"problem_description":1,"product":1,"projection/method":1,"property|comparison|mechanism":1,"property|method":1,"rationale":1,"reaction|computational_result":1,"reaction|mechanism":1,"reaction|method":1,"recommendation|mechanism":1,"reference":1,"representation":1,"scaling":1,"scope":1,"scope/property":1,"scope/reaction":1,"scope|property":1,"significance":1,"solution":1,"synthesis_condition":1,"theoretical":1,"theoretical_principle":1,"tradeoff":1,"trajectory":1},"year_distribution":{"1925":3,"1926":1,"1927":2,"1928":2,"1929":1,"1930":2,"1931":1,"1932":1,"1940":1,"1941":1,"1942":1,"1943":1,"1944":1,"1949":1,"1953":1,"1954":1,"1958":1,"1959":1,"1960":1,"1961":2,"1962":1,"1963":1,"1964":1,"1965":2,"1967":4,"1968":2,"1969":5,"1970":4,"1971":3,"1972":9,"1973":6,"1974":14,"1975":12,"1976":9,"1977":10,"1978":6,"1979":15,"1980":11,"1981":15,"1982":13,"1983":21,"1984":17,"1985":27,"1986":19,"1987":28,"1988":33,"1989":32,"1990":45,"1991":44,"1992":55,"1993":44,"1994":56,"1995":94,"1996":99,"1997":113,"1998":111,"1999":96,"2000":97,"2001":122,"2002":157,"2003":226,"2004":233,"2005":283,"2006":289,"2007":376,"2008":360,"2009":451,"2010":609,"2011":842,"2012":998,"2013":1135,"2014":1581,"2015":1896,"2016":2255,"2017":2840,"2018":7421,"2019":12863,"2020":15293,"2021":13525,"2022":10818,"2023":11606,"2024":11693,"2025":6437,"2026":16},"citation_source":"","citation_source_url":"","citations_updated_at":""},"quality":{"total_claims":1766989,"total_sources":123987,"claim_type_distribution":{"property":552675,"method":298596,"computational_result":219005,"comparison":211947,"mechanism":178193,"reaction":66514,"scope_entry":46971,"experimental_design":39203,"limitation":34848,"structure":33631,"surprising_finding":29834,"future_direction":18674,"hypothesis":16757,"conclusion":11945,"observation":3970,"measurement":383,"equation":320,"parameter":306,"performance":297,"experimental_result":293,"result":262,"model":255,"conclusions":191,"background":175,"optimization":155,"theory":131,"application":104,"data_point":70,"definition":68,"prediction":59,"composition":47,"control_experiment":44,"background_information":40,"relationship":39,"theoretical_result":35,"metadata":34,"design":33,"assumption":30,"correlation":30,"data":29,"claim":26,"computational_design":26,"negative_result":25,"phenomenon":24,"calculation":23,"material":22,"performance_metric":21,"derivation":20,"control":19,"outcome":19,"model_parameter":18,"concept":17,"behavior":15,"historical":14,"rule":13,"classification":12,"condition":12,"interpretation":12,"material_property":12,"mathematical_result":12,"capability":11,"problem":11,"finding":10,"limit":10,"computational_method":8,"mathematical_derivation":8,"trend":8,"advantage":7,"formula":7,"other":7,"problem_statement":7,"recommendation":7,"analytical_result":6,"approximation":6,"background_knowledge":6,"explanation":6,"implication":6,"literature_result":6,"optimization_entry":6,"reference_value":6,"theoretical_model":6,"conparison":5,"discovery":5,"goal":5,"literature_value":5,"scaling_property":5,"theorem":5,"context":4,"dataset_description":4,"design_principle":4,"device":4,"equation_or_formula":4,"experimental_finding":4,"material_preparation":4,"model_definition":4,"parameter_definition":4,"parameter_fit":4,"statistical_result":4,"validation":4,"architecture":3,"background_property":3,"claim_type:method":3,"design_rule":3,"empirical_result":3,"equation_derivation":3,"historical_fact":3,"historical_milestone":3,"kinetics":3,"mathematical_relation":3,"parameter_extraction":3,"parameter_setting":3,"property|comparison":3,"scaling_law":3,"threshold":3,"application|method":2,"application|reaction":2,"background/comparison":2,"background_theory":2,"challenge":2,"claim_type":2,"claim_type:comparison":2,"conputational_result":2,"correction":2,"critique":2,"extrapolation":2,"general":2,"limitations":2,"mechanism|property":2,"method|comparison":2,"metric":2,"model_approximation":2,"model_design":2,"outlook":2,"proof":2,"property|mechanism":2,"purpose":2,"reaction|comparison":2,"reasoning":2,"regime_definition":2,"relation":2,"requirement":2,"spectroscopic":2,"structural_response":2,"synthesis":2,"theoretical_prediction":2,"analysis":1,"application/method":1,"application/reaction":1,"application|comparison":1,"artifact":1,"assignment":1,"background/property":1,"band_structure":1,"benchmark":1,"benefit":1,"biological_property":1,"claim (conclusion)":1,"claim (method/property)":1,"comparative_result":1,"comparision":1,"comparison|mechanism":1,"comparison|mechanism|computational_result":1,"comparison|property":1,"computational_study":1,"conclusion|property":1,"conditions":1,"constraint":1,"dataset":1,"demonstration":1,"design_concept":1,"design_parameter":1,"example":1,"experimental_parameter":1,"field_observations":1,"finding|property":1,"future_opportunity":1,"future_scope":1,"general_fact":1,"goal_statement":1,"historical/comparison":1,"historical_comparison":1,"historical_trend":1,"history":1,"implication/method":1,"initial_state_assignment":1,"literature_observation":1,"material_composition":1,"material_proposal":1,"material_synthesis":1,"mathematical_property":1,"mechanism|computational_result":1,"meta":1,"metadata_based_method":1,"methodology_outlook":1,"method|computational_result":1,"method|computational_result|property":1,"method|prediction":1,"method|property":1,"model_derivation":1,"model_property":1,"model_result":1,"model_structure":1,"motivation":1,"novelty":1,"novelty/comparison":1,"optimization_result":1,"outcome/method":1,"outcome|method":1,"outcome|property":1,"paper_type":1,"parameter_estimation":1,"principle":1,"prior_work":1,"problem/mechanism":1,"problem/property":1,"problem_description":1,"product":1,"projection/method":1,"property|comparison|mechanism":1,"property|method":1,"rationale":1,"reaction|computational_result":1,"reaction|mechanism":1,"reaction|method":1,"recommendation|mechanism":1,"reference":1,"representation":1,"scaling":1,"scope":1,"scope/property":1,"scope/reaction":1,"scope|property":1,"significance":1,"solution":1,"synthesis_condition":1,"theoretical":1,"theoretical_principle":1,"tradeoff":1,"trajectory":1},"year_range":[1925,2026],"year_distribution":{"1925":3,"1926":1,"1927":2,"1928":2,"1929":1,"1930":2,"1931":1,"1932":1,"1940":1,"1941":1,"1942":1,"1943":1,"1944":1,"1949":1,"1953":1,"1954":1,"1958":1,"1959":1,"1960":1,"1961":2,"1962":1,"1963":1,"1964":1,"1965":2,"1967":4,"1968":2,"1969":5,"1970":4,"1971":3,"1972":9,"1973":6,"1974":14,"1975":12,"1976":9,"1977":10,"1978":6,"1979":15,"1980":11,"1981":15,"1982":13,"1983":21,"1984":17,"1985":27,"1986":19,"1987":28,"1988":33,"1989":32,"1990":45,"1991":44,"1992":55,"1993":44,"1994":56,"1995":94,"1996":99,"1997":113,"1998":111,"1999":96,"2000":97,"2001":122,"2002":157,"2003":226,"2004":233,"2005":283,"2006":289,"2007":376,"2008":360,"2009":451,"2010":609,"2011":842,"2012":998,"2013":1135,"2014":1581,"2015":1896,"2016":2255,"2017":2840,"2018":7421,"2019":12863,"2020":15293,"2021":13525,"2022":10818,"2023":11606,"2024":11693,"2025":6437,"2026":16},"extraction_depth":{"full_paper":945124,"abstract_only":821372,"full_paper_by_model":{"@vertexai-gemini-kc119-2/gemini-3.1-pro-preview":532001,"gpt-5.4":413123},"full_paper_papers":23216},"doi_verification":{"total_validated":56400,"crossref_verified":56390,"retracted_caught":33,"verification_rate":100.0},"subfield_coverage":{"materials":365026,"other":9604,"computational":172710,"biochemistry":41205,"physical_chemistry":138818,"catalysis":23458,"polymer":15628,"electrochemistry":40175,"environmental":35857,"analytical":23997,"organic_synthesis":7076,"inorganic":2393,"atmospheric_chemistry":635,"photochemistry":10206,"green_chemistry":237,"methods_paper":157,"medicinal_chemistry":2155,"computational_electrochemistry":30,"chemical_biology":703,"computational_study":94,"medicinal":83,"spintronics":50,"bioinorganic":125,"organometallic|computational|catalysis":66,"superconductivity":22,"food_chemistry":251,"combustion":62,"atmospheric|combustion|physical_chemistry|analytical":57,"energy|catalysis|environmental":60,"biosensors":50,"radiochemistry":61,"instrumentation":16,"atmospheric":110},"flag_stats":{"total":0,"resolved":0,"open":0},"smiles_validation":{"total":0,"valid":0,"invalid":0},"extraction_models":["gemini-3.1-pro-preview (full paper PDF extraction + classification)","gpt-5.4 (full paper PDF extraction)","gpt-5-mini (abstract extraction)"],"citation_source":"Semantic Scholar Academic Graph API","citation_source_url":"https://api.semanticscholar.org/","citations_updated_at":"","known_limitations":["SMILES validation not yet run on all claims","Contradiction detection pipeline deployed; batch scan pending","Abstract-only extractions capture headline findings; full-paper extractions from Gemini-3.1-pro-preview and GPT-5.4 include tables, conditions, and mechanistic detail"]}}},"gpt-4.1":{"generated_at":"","methods":{},"subsets":{},"askchem_snapshot":{}}},"reproducibility":{"install":"pip install openai requests","run":"OPENAI_API_KEY=sk-... python scripts/benchmark_chemtree.py","env_vars":{"BENCH_MODEL":"LLM model to benchmark (default: gpt-5.5)","ASKCHEM_API":"AskChem API base URL (default: https://askchem.org/api)","EDISON":"Optional Edison Scientific API key to run the FutureHouse paperqa3 baseline","PAPERCLIP":"Paperclip API key (same as PAPERCLIP_API_KEY) for the Paperclip retrieval baseline","BENCH_RESUME":"Set to 0 to ignore cached partial results and rerun everything"}}}