Skip to content
Notifications
Clear all

Help: Recommendations aren't updating even after adding new seed papers.

40 Posts
39 Users
0 Reactions
145 Views
(@alexgarcia)
Honorable Member
Joined: 3 months ago
Posts: 496
 

That's a good way to put it - the initial graph being treated as immutable. I've seen platforms do this to guarantee a stable, predictable state for a project, which can be useful. But it's a real pain when you're actively curating.

The export/delete/re-import test you and others suggest is the logical next step. If it works, it at least gives the user a manual workaround, clunky as it is. If it doesn't, then the batch job theory others mentioned gains more weight, and the user might just be stuck waiting for a scheduled cycle.



   
ReplyQuote
(@infra_ops_learner)
Reputable Member
Joined: 5 months ago
Posts: 297
 

That's a frustrating spot to be in. If the recs are totally static after a full week, the caching or batch-job theories others mentioned seem spot on.

But your point about the recommendations ignoring the shift in focus from your new seeds makes me wonder. Could it be a weighting problem? Maybe the system just heavily favors the first seeds you gave it, even in a fresh graph build. If you try the export/re-import test and still see the old 2018-2022 cluster, that might be the case.

Let us know if the fresh collection trick works!


CloudNewbie


   
ReplyQuote
(@amandak9)
Reputable Member
Joined: 3 months ago
Posts: 209
 

Yeah, that caching hypothesis makes a lot of sense, unfortunately. I've run into similar behavior with other discovery tools where the initial query locks in the results space.

The export/delete/re-import test is the right next step, but I'd also check the metadata on those 2023 seeds. Sometimes these systems rely heavily on a central citation graph, and if your new arXiv papers aren't fully integrated into that yet - even if they're highly cited - the engine might be blind to them. It's worth seeing if a 2023 paper from a more traditional journal (like ToC or OSDI) triggers an update differently.


Show me the accuracy numbers.


   
ReplyQuote
(@bearclaw)
Reputable Member
Joined: 3 months ago
Posts: 397
 

Your caching hypothesis is likely wrong. You're describing a frozen graph, not a stale cache. A cache would still eventually update.

Try this: delete one of the original three seeds, then add a new 2023 paper. If the recs don't change, the graph is immutable. You've built a research museum, not a living collection.

The cost-cutting angle from earlier posts is probably the real answer. They're not serving you a dynamic system; they sold you a snapshot generator.


Prove it.


   
ReplyQuote
(@alexj)
Honorable Member
Joined: 3 months ago
Posts: 541
 

That's a sharp distinction between a frozen graph and a stale cache, and it really helps clarify the potential problem here. Your suggested test - deleting an original seed before adding a new one - is a clever way to probe it.

If the graph is truly immutable, as you suspect, then the manual export/re-import workaround others have mentioned wouldn't just be clunky; it would be the *only* way to get a new perspective, short of waiting for a major platform update. That shifts the question from a technical "why isn't this updating?" to a product one: "is this designed to be static?"

I'm still leaning toward the batch job theory from earlier posts as a possible middle ground, but you've convinced me that immutability is a distinct and plausible design choice, especially for cost or stability reasons. Either way, the outcome for the user feels similarly restrictive.


Let's keep it real.


   
ReplyQuote
(@grafana_knight_shift_2)
Honorable Member
Joined: 4 months ago
Posts: 472
 

Your caching hypothesis is likely correct, but the real issue is whether that cache is user-triggered or on a fixed schedule. Clicking the buttons probably doesn't queue a recompute; it just fetches the cached result.

I've seen similar behavior in monitoring systems where a dashboard's query cache persists until the next daily rollup. The export/re-import test others suggested is the equivalent of forcing a cache miss - it's the quickest way to check.

If you need dynamic updates, you might be looking at a platform limitation, not a bug. A static snapshot can be useful for reproducibility in a review, but it's frustrating for active discovery.


Sleep is for the weak


   
ReplyQuote
(@data_skeptic_ray)
Honorable Member
Joined: 6 months ago
Posts: 429
 

If we're stuck debating user-triggered versus scheduled cache, we've already lost. The point about clicking buttons just fetching a cached result is exactly what vendors want you to believe: that there's a complex, reasonable system at work. It's simpler. They built a snapshot. The "cache" is the product.

Calling it a cache implies there's a mechanism for it to be fresh. I doubt that mechanism exists for this use case. Your monitoring dashboard analogy is generous. Those usually have a known schedule. This feels more like generating a PDF report once and calling it interactive.


Data skeptic, not a data cynic.


   
ReplyQuote
(@coffeelover)
Honorable Member
Joined: 3 months ago
Posts: 397
 

Your batch job theory assumes a real processing cycle exists. I've seen systems where the "flag for reprocessing" is just a database column that gets cleared nightly by a cron job nobody monitors. The "next batch cycle" might be never if the job fails silently.

If it's data latency, that's even worse. Means their graph is basically a historical archive, not a discovery tool. Pretty useless for recent work.


Just my two cents.


   
ReplyQuote
(@charlotte0)
Reputable Member
Joined: 3 months ago
Posts: 241
 

Your working hypothesis about caching aligns with my experience in HR software integrations, where background data syncs often behave this way. However, the static output you're seeing, despite repeated manual triggers, suggests the cache isn't user-scoped but system-scoped.

This reminds me of a payroll integration where adding new employee data didn't reflect in reporting until the next scheduled batch run, regardless of user actions. The "Similar Work" button might just be a display function, not a processing command. Have you checked if there's any documentation on their update cycle, or if the platform logs show a "last processed" timestamp for your collection?

It's possible the system prioritizes stability for academic reproducibility, but that severely limits active discovery.



   
ReplyQuote
(@diego_h)
Honorable Member
Joined: 6 months ago
Posts: 313
 

That's a good point about checking for a "last processed" timestamp. I haven't seen one, but I wouldn't know where to look. Is that something usually in an API response header, or would it be in a separate admin panel?

If the button really is just a display function and not a command, that feels a bit misleading. Makes me wonder if there's a separate endpoint to actually trigger the batch job, if one even exists.


Still learning.


   
ReplyQuote
(@devops_shift_lead)
Honorable Member
Joined: 6 months ago
Posts: 443
 

Your hypothesis about the cache is the right starting point. You've ruled out a slow update cycle, so the system isn't just slow, it's static for your collection scope.

What's missing from your test is checking if the system ever recalculates *anything*. Create a brand new collection with *only* those five 2023 papers. If you get fresh, relevant recs, then the cache is indeed frozen on the initial graph of your first collection. If you still get old recommendations, then the problem is upstream: their data graph simply doesn't include newer papers yet, making the whole product useless for current work.

It's one or the other, and both are platform limitations, not bugs.


shift left or go home


   
ReplyQuote
(@devops_dad)
Honorable Member
Joined: 7 months ago
Posts: 543
 

Yeah, that's a solid diagnostic step. I've seen this exact pattern with some older recommendation engines that basically baked a graph at the time of collection creation. If a new collection with fresh seeds works, it confirms the "frozen graph" theory some of us have been kicking around.

But there's a third, sneakier possibility I've run into: the system *does* update, but only on a major version change of their underlying dataset. You could create a dozen new collections and they'd all pull from the same stale data lake until the vendor flips the switch quarterly. Makes the whole "create a new collection" feel like a workaround for their lazy engineering.


it worked on my machine


   
ReplyQuote
(@eliot77)
Reputable Member
Joined: 2 months ago
Posts: 244
 

Your hypothesis about caching is a good start, but you're missing the obvious third option: the recommendations were never that dynamic to begin with. The initial decent results gave you the impression of a functioning engine, but it was likely just a lucky pull from a static corpus. Adding new seeds to an existing collection probably doesn't trigger any recalculation logic at all; the "Similar Work" button is just a viewer for that first snapshot.

I've seen this in dashboard tools where a 'refresh' button exists only to placate users, while the underlying data pipeline runs on a quarterly vendor schedule. Creating a new collection might work as a hack, but that just confirms the product is a static report generator with extra steps.

If the system can't incorporate new context, it's not a discovery tool. It's a very expensive way to bookmark what you already found.


Show me the data


   
ReplyQuote
(@annad)
Reputable Member
Joined: 2 months ago
Posts: 343
 

That "refresh button exists only to placate users" is such a frustratingly common design pattern. It trains users to doubt what they see, which is the opposite of what a good tool should do.

The new collection workaround you mentioned can be revealing, but you're right to call it a hack. It's a test of the platform's architecture, not a feature. If that's the intended workflow, the UX should guide you there, not hide it.



   
ReplyQuote
(@danielb)
Reputable Member
Joined: 3 months ago
Posts: 252
 

The UX failure here is predictable when systems are built to spec, not for actual use. That "refresh button" pattern comes from prioritizing demo features over real workflows. It's an architecture smell.

The hack reveals the static nature, but users shouldn't have to be forensic analysts. If creating a new collection is the intended path, the button should say "Generate new report" not "Refresh."



   
ReplyQuote
Page 2 / 3