Gabisonfire
5fe3f471e9
Renames project to Knight Crawler
2024-02-03 23:45:06 -05:00
Gabisonfire
a8173a0ba7
Merge pull request #48 from HDwayne/master
...
Monitoring with Grafana and Prometheus
2024-02-03 19:42:49 -05:00
iPromKnight
d660a72d6f
missed a couple of log lines for logger
2024-02-03 22:12:45 +00:00
iPromKnight
8c1b6fc91b
roll logging levels out throughout the consumer
2024-02-03 22:06:38 +00:00
iPromKnight
4bd76d682f
remove pm2 process manager - let the container orchestrator handle the lifetime events (docker, podman, k8s etc)
2024-02-03 22:06:02 +00:00
iPromKnight
678b3857f3
Add pino as a logger, and update cache manager
...
Rewrites the cache service to use the latest implementation of cache manager, and bring in the new mongo package.
Introduce Logger service
Removes bluebird as a dependency
2024-02-03 22:05:15 +00:00
HDwayne
a56b04f04b
Add Temporary statistics endpoint
2024-02-03 22:06:13 +01:00
iPromKnight
57f4757541
Implement Max Queue and Max Batch size when publishing
...
MaxPublishBatchSize must be set, but MaxQueueSize can be set to 0 to disable check of the rabbitmq queue size
2024-02-02 14:43:29 +00:00
iPromKnight
8ad6cf731c
harden consumer
2024-02-02 14:04:53 +00:00
iPromKnight
68edaba308
Introduce max batch size, and configurable publish window
...
Still need to implement queue size limit
Also fixes env var consistency between addon and consumer
2024-02-02 13:49:54 +00:00
iPromKnight
0644fe7720
add the jackett backend alternate addon "jackettio"
2024-02-02 13:27:41 +00:00
iPromKnight
188ffd10f3
rework removing providers filters, and clean up project a little
...
Also wraps in pm2, and introduces linting, and dev watch
2024-02-02 13:27:15 +00:00
iPromKnight
ee994fc8be
ignore bin and obj
2024-02-01 16:47:45 +00:00
iPromKnight
ab17ef81be
Big rewrite - distributed consumers for ingestion / scraping(scalable) - single producer written in c#.
...
Changed from page scraping to rss xml scraping
Includes RealDebridManager hashlist decoding (requires a github readonly PAT as requests must be authenticated) - This allows ingestion of 200k+ entries in a few hours.
Simplifies a lot of torrentio to deal with new data
2024-02-01 16:38:45 +00:00
