Exactly. The metadata granularity is often the fatal, hidden assumption. If they're using journal-level categories, then the entire paper gets flattened into a single, noisy label. Your "Bayesian A/B testing" paper gets lumped under "Statistics - Applications" and the algorithm happily serves you any other paper in that bucket, like a clinical stats paper that also uses Bayesian methods for dose-finding.
That's why these feeds fail for interdisciplinary work. The solution you mention, generating embeddings from specific sections, is the right path but a non-starter for most providers. The cost isn't just in the model inference, it's in the data pipeline. You need clean, parsed full-text with reliable section headers for millions of papers, which most aggregators don't have consistently. So they fall back on the abstracts, and we're back to square one with the keyword mess.
APIs are not magic.
The clinical trial versus A/B testing mismatch is a perfect case study of a bad label hierarchy. If their taxonomy flattens both under "Statistics > Trial" or similar, the model can't differentiate. This isn't just a keyword issue, it's a flawed ontology problem.
The lack of citation context weighting you noted is the other critical failure. In a proper graph, an edge where Paper B cites your foundational paper in its "Related Work" section should have a negligible weight compared to a citation in the "Methodology" section. Without that, the graph is just counting connections, not understanding them.
Your systematic approach here is the right one. Most users just get frustrated and leave. Documenting the specific failure modes, like the keyword "trial" pulling from clinical domains, is the only way to pressure for a fix, even if the underlying incentive is to keep you scrolling through low-precision results.
Data over dogma
Your breakdown of that keyword mismatch, "Bayesian A/B testing" versus clinical "trial" design, feels so familiar from the CRM world. We see the exact same thing when trying to build audience segments. If you set a lead scoring rule for anyone who interacts with content tagged "trial," you'll catch both people on free software trials and people reading clinical study reports. It's a taxonomy fail before the algorithm even gets started.
That lack of citation context weighting is the other huge red flag. It's like if my marketing automation treated someone who opened a welcome email the same as someone who clicked a key pricing page link. The action might be the same on the surface, but the intent is completely different. If they're not weighting a "methodology" section citation more heavily than one in the "references," they've built a map with no sense of distance.
Have you tried creating a separate, hyper-specific saved search as a workaround, even if it's more manual? I've had to do that with lead filters when the auto-segmentation gets too noisy.
If it's not measurable, it's not marketing.
That CRM analogy is spot on. It's the same problem with logging and monitoring taxonomies. If you tag every error log with a generic "error" label, your alerting is useless. You need structured fields like `error.domain=authentication` and `error.severity=critical`. Feed algorithms are basically doing bad log aggregation on paper metadata.
> creating a separate, hyper-specific saved search as a workaround
That's exactly what I've done, but the manual maintenance is the killer. It's like building a custom Prometheus query to filter out noisy metrics because your service discovery is mislabeled. It works, but it's technical debt. You have to constantly update the search terms as new jargon or tangential fields pop up, and you miss the serendipity of a good discovery system.
The real failure is that these platforms are offering a managed service but putting the onus of data cleansing back on the user. I wouldn't tolerate that from a vendor providing log ingestion, and I'm not sure why we tolerate it here.
Automate everything. Twice.
I'm struggling with this exact same problem on a different platform. When I search for papers on SaaS subscription billing, I keep getting research on healthcare billing systems.
You mention the feed suggestions are based on superficial keyword matching. Do you think the issue gets worse the more you interact with the feed? Like if you click "not interested" on an irrelevant paper, does it just find a new keyword in that paper to match on?