different rss and same post

[odysseymemoirs]

How to handle this situation?

Suppose when whe add 2 different rss from same website:

1. site.com/tech/rss.xml
2. site.com/cars/rss.xml

feeder.add({
  url: ['site.com/tech/rss.xml', 'site.com/cars/rss.xml' ],
  refresh: 2000,
  eventName: 'world'
}); 

so when a new post appears on website with the following tags:

title: New Tesla will cost 1 bitcoin 

description: some description

tags: cars, tech, news, cyberpunk

This post will appear in these two rss

1. site.com/cars/rss.xml
2. site.com/tech/rss.xml

So, we will have duplicated post on eventName: 'world'

My idea is that the same eventName should have the same cache memory to skip same post

[TobiTenno]

ok, so you added stuff that has crossover and you're expecting something else to handle your dupe checking?

just making sure i'm understanding what you're asking.

[odysseymemoirs]

just making sure i'm understanding what you're asking.

not quite

I know this is not the library's job but only my vision of good rss aggregator library

and i guess above problem is very rare that only i faced it and to handle it i need to write additional wrappers and cases

I came to this solution:
(considering that same rss item come at the same moment(60sec))

let itemsReceived = [];
setInterval(() => { itemsReceived = [] }, 60000)

feeder.on('eventName', function (item) {
  if (itemsReceived.includes(item.title)) return

  itemsReceived.push(item.title);

// dot something with rss item

)}

[odysseymemoirs]

and isn't it more correct to get rss items as an array from feeder.on('eventName') rather than one by one?

[TobiTenno]

and isn't it more correct to get rss items as an array from feeder.on('eventName') rather than one by one?

one rss entry -> one event

itemsReceived.push(item.title);

I'll take a look this weekend with it and see if I can accurately do this on the feed manager, I'll probably do feed url, as those are more likely to be unique/crossed-over.

[TobiTenno]

I've got an initial build working. Would you mind contacting me on discord and I can try to get you a branch to test out? if you'd be willing to manually test it before I get tests written for it?

[TobiTenno]

I've added a gitter for the repo so it's easier to find/talk to me