Commit Graph

7 Commits

Author SHA1 Message Date
iPromKnight
57f4757541 Implement Max Queue and Max Batch size when publishing
MaxPublishBatchSize must be set, but MaxQueueSize can be set to 0 to disable check of the rabbitmq queue size
2024-02-02 14:43:29 +00:00
iPromKnight
8ad6cf731c harden consumer 2024-02-02 14:04:53 +00:00
iPromKnight
68edaba308 Introduce max batch size, and configurable publish window
Still need to implement queue size limit
Also fixes env var consistency between addon and consumer
2024-02-02 13:49:54 +00:00
iPromKnight
0644fe7720 add the jackett backend alternate addon "jackettio" 2024-02-02 13:27:41 +00:00
iPromKnight
188ffd10f3 rework removing providers filters, and clean up project a little
Also wraps in pm2, and introduces linting, and dev watch
2024-02-02 13:27:15 +00:00
iPromKnight
ee994fc8be ignore bin and obj 2024-02-01 16:47:45 +00:00
iPromKnight
ab17ef81be Big rewrite - distributed consumers for ingestion / scraping(scalable) - single producer written in c#.
Changed from page scraping to rss xml scraping
Includes RealDebridManager hashlist decoding (requires a github readonly PAT as requests must be authenticated) - This allows ingestion of 200k+ entries in a few hours.
Simplifies a lot of torrentio to deal with new data
2024-02-01 16:38:45 +00:00