Commit Graph

13 Commits

Author SHA1 Message Date
iPromKnight
a25aef8f58 dont restart the metadata service when it completes, and exit if its a single run 2024-02-28 11:51:46 +00:00
iPromKnight
360bc20970 Comment resolution 2024-02-11 21:45:17 +00:00
iPromKnight
0f3e20eef3 trackers needs to be singleton, loggers should. be too 2024-02-11 21:45:17 +00:00
iPromKnight
b773ffcb20 Logging expansion
Adds Promtail and Loki to the metrics stack, configuring them as a datasource in grafana on deployment
Adds a dashboard to monitor the system (logs for producer and consumer)
Restructures the base directory so deployment manifests are moved into a deployments folder
Adds selective labels to producer, consumer and addon so that their logs are ingested by promtail
2024-02-04 19:50:28 +00:00
iPromKnight
7ba38db0cc remove .env and add .env.example - add readme instructions 2024-02-04 15:45:32 +00:00
iPromKnight
8ad6cf731c harden consumer 2024-02-02 14:04:53 +00:00
iPromKnight
ee994fc8be ignore bin and obj 2024-02-01 16:47:45 +00:00
iPromKnight
ab17ef81be Big rewrite - distributed consumers for ingestion / scraping(scalable) - single producer written in c#.
Changed from page scraping to rss xml scraping
Includes RealDebridManager hashlist decoding (requires a github readonly PAT as requests must be authenticated) - This allows ingestion of 200k+ entries in a few hours.
Simplifies a lot of torrentio to deal with new data
2024-02-01 16:38:45 +00:00
TheBeastLT
c3c21e9dc8 [scraper] updates manual functions 2020-12-23 11:28:45 +01:00
TheBeastLT
70c279d7d7 [addon] adds filtering based on provider 2020-03-15 23:49:54 +01:00
TheBeastLT
58aba322c2 moves scraper to a subpackage 2020-03-10 15:23:52 +01:00
TheBeastLT
5cfc82134a updating to use tpb cached torrent files 2019-12-31 19:36:03 +01:00
TheBeastLT
5a122683d7 Initial commit 2019-03-13 22:41:15 +01:00