brock/torrentio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,110 Commits 7 Branches 31 Tags
master
Commit Graph

129 Commits

Author SHA1 Message Date
iPromKnight
9e9e928c80 promises helpers now ts 2024-02-11 21:43:01 +00:00
iPromKnight
948cb8e037 metadata service now ts 2024-02-11 21:43:01 +00:00
iPromKnight
cf25f32cb6 Torrent processing orchestrator now typescript too 
Will start to tackle some of the other services after work tomorrow.
 2024-02-11 21:43:01 +00:00
iPromKnight
204fe51658 Rewritten repository in typescript 2024-02-11 21:43:01 +00:00
David Howell
2cae5296a2 Build multi-platform images 
Refactor GitHub Actions workflow for build
Run Dockle and Trivy, upload sarif reports to GitHub
Refactor Dockerfiles based on best practices
 2024-02-08 06:00:48 +00:00
David Young
8c39b36911 Fix addon collection name 
Signed-off-by: David Young <davidy@funkypenguin.co.nz>
 2024-02-08 13:41:39 +13:00
Gabisonfire
6813c755ea Merge pull request #68 from FunkeCoder23/66-remove-sync-alter 
Remove alter param from database sync
 2024-02-05 08:14:29 -05:00
purple_emily
143754f36f BUGFIX: Change POSTGRES_DATABASE to POSTGRES_DB 2024-02-05 12:49:19 +00:00
purple_emily
0a0ad800fc BUGFIX: Change POSTGRES_USERNAME to POSTGRES_USER. Oops 2024-02-05 11:44:39 +00:00
purple_emily
287507d8e0 Add alter param back to database sync. Default to false. Make toggleable from .env file. 2024-02-05 10:43:20 +00:00
funkecoder23
ecdae702ad Remove alter param from database sync 
Re: [sequelize docs](https://sequelize.org/docs/v6/core-concepts/model-basics/#synchronization-in-production)
Alter is a bad idea in production.

Also print error message instead of sequelize error object as it's a bit verbose
 2024-02-04 23:23:57 -05:00
trulow
d4e82e9878 Updated and replaced selfhostio entries with knightcrawler 
Updated and replaced selfhostio entries with knightcrawler.

Was previously still pointing to the old DB name of selfhostio
 2024-02-04 16:52:05 -08:00
funkecoder23
3ddc922081 remove extraneous v from manifest version 2024-02-04 17:19:11 -05:00
iPromKnight
e461e26b0f Change postgres configuration in the producer to use the env vars from the stack 2024-02-04 15:03:07 +00:00
purple_emily
cd3c2d3fe6 Merge and simplify the environment variables in 
addon and consumer.

Todo: producer

Change DATABASE_URI to be generic POSTGRES variables

DOES NOT WORK - First pass at upgrading environment variables

PostgreSQL environment variables have been split for addon and consumer. ENABLE_SYNC hard coded as `true`

MongoDB variables update.

Make the addon code more similar to the consumer code

Get some parity between addon and consumer
 2024-02-04 08:30:55 +00:00
Gabisonfire
5fe3f471e9 Renames project to Knight Crawler 2024-02-03 23:45:06 -05:00
Gabisonfire
a8173a0ba7 Merge pull request #48 from HDwayne/master 
Monitoring with Grafana and Prometheus
 2024-02-03 19:42:49 -05:00
iPromKnight
d660a72d6f missed a couple of log lines for logger 2024-02-03 22:12:45 +00:00
iPromKnight
8c1b6fc91b roll logging levels out throughout the consumer 2024-02-03 22:06:38 +00:00
iPromKnight
4bd76d682f remove pm2 process manager - let the container orchestrator handle the lifetime events (docker, podman, k8s etc) 2024-02-03 22:06:02 +00:00
iPromKnight
678b3857f3 Add pino as a logger, and update cache manager 
Rewrites the cache service to use the latest implementation of cache manager, and bring in the new mongo package.
Introduce Logger service
Removes bluebird as a dependency
 2024-02-03 22:05:15 +00:00
HDwayne
a56b04f04b Add Temporary statistics endpoint 2024-02-03 22:06:13 +01:00
iPromKnight
57f4757541 Implement Max Queue and Max Batch size when publishing 
MaxPublishBatchSize must be set, but MaxQueueSize can be set to 0 to disable check of the rabbitmq queue size
 2024-02-02 14:43:29 +00:00
iPromKnight
8ad6cf731c harden consumer 2024-02-02 14:04:53 +00:00
iPromKnight
68edaba308 Introduce max batch size, and configurable publish window 
Still need to implement queue size limit
Also fixes env var consistency between addon and consumer
 2024-02-02 13:49:54 +00:00
iPromKnight
0644fe7720 add the jackett backend alternate addon "jackettio" 2024-02-02 13:27:41 +00:00
iPromKnight
188ffd10f3 rework removing providers filters, and clean up project a little 
Also wraps in pm2, and introduces linting, and dev watch
 2024-02-02 13:27:15 +00:00
iPromKnight
ee994fc8be ignore bin and obj 2024-02-01 16:47:45 +00:00
iPromKnight
ab17ef81be Big rewrite - distributed consumers for ingestion / scraping(scalable) - single producer written in c#. 
Changed from page scraping to rss xml scraping
Includes RealDebridManager hashlist decoding (requires a github readonly PAT as requests must be authenticated) - This allows ingestion of 200k+ entries in a few hours.
Simplifies a lot of torrentio to deal with new data
 2024-02-01 16:38:45 +00:00