brock/torrentio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,110 Commits 7 Branches 31 Tags
master
Commit Graph

129 Commits

Author SHA1 Message Date
iPromKnight
79a6aa3cb0 Improve producer matching - Add tissue service 
Tissue service will sanitize the existign database of ingested torrents by matching existing titles with new banned word lists. Now with added kleenex
 2024-03-12 10:29:13 +00:00
iPromKnight
aeb83c19f8 Simplification of parsing in consumer 
should speed up massively especially if imdbIds are found from mongodb
 2024-03-11 10:56:04 +00:00
iPromKnight
5c310427b4 Fix nyaa category 2024-03-11 08:59:55 +00:00
iPromKnight
dda81ec5bf accidentally skipped build on last pr 
tired..
 2024-03-10 23:37:16 +00:00
iPromKnight
75ac89489e [skip ci] Final hotfix 2024-03-10 23:34:35 +00:00
iPromKnight
500dd0d725 patch type 2024-03-10 22:28:06 +00:00
iPromKnight
6f4bc10f5a Fix series parsing 2024-03-10 21:38:55 +00:00
iPromKnight
02150482df reduce cpu cycles in parsing in producer 2024-03-10 15:14:17 +00:00
iPromKnight
2e774058ff Few extra terms getting through 2024-03-10 14:54:25 +00:00
iPromKnight
ad04d323b4 remove log line of adult content 2024-03-10 13:54:35 +00:00
iPromKnight
e2b45e799d [skip ci] Remove Debug logged adult terms found 2024-03-10 13:49:51 +00:00
iPromKnight
6c03f79933 Complete 2024-03-10 13:48:27 +00:00
iPromKnight
320fccc8e8 [skip ci] More work on parsing - seasons to fix still and use banned words 2024-03-10 12:48:19 +00:00
iPromKnight
8d82a17876 re-disable services other than dmm while developing 
re-enable

disable again - will squash dont worry

enable again

disable again
 2024-03-10 12:48:19 +00:00
iPromKnight
f719520b3b [skip ci] Ignore all run profiles to prevent pat leaking 
reenable these, testing only producer should build
 2024-03-10 12:48:19 +00:00
iPromKnight
bacb50e060 [skip ci] remove extra package no longer in use 2024-03-10 12:48:19 +00:00
iPromKnight
6600fceb1a Wip Blacklisting dmm porn 
Create adult text classifier ML Model

wip - starting to write PTN in c#

More work on season, show and movie parsing

Remove ML project
 2024-03-10 12:48:16 +00:00
iPromKnight
856a34a8ef Wait isn't required. 2024-03-09 23:18:15 +00:00
iPromKnight
e48e968fb8 Add database migrations service 2024-03-09 19:52:17 +00:00
iPromKnight
61ec8d7ab5 [skip ci] bearer not basic 2024-03-09 17:46:33 +00:00
iPromKnight
17e092d830 Fix Trackers TTL 
Also introduces the ability to reuse your PAT token. If its set, it'll be used on tracker calls.
 2024-03-09 17:44:17 +00:00
purple_emily
79409915cf Run pre-commit 2024-03-08 14:34:53 +00:00
Gabisonfire
d54b5dbc33 New logo 2024-03-03 19:09:38 -05:00
iPromKnight
a609af66f9 change default retry window to be a larger delay 
Lets not hammer them
 2024-03-03 20:28:53 +00:00
iPromKnight
c3a281c39f retry polic and circuit breaker policy 2024-03-03 19:54:32 +00:00
iPromKnight
62decbf994 Ensure we throw 
when torrentio/knightcrawler instances return invalid status codes on fetch requests for json payloads, pre-parsing of json, polly will catch in the policy wrapped resiliency handler
 2024-03-03 19:30:06 +00:00
iPromKnight
c61e9e94e1 Rethrow so polly captures failures on requests. 2024-03-03 19:22:10 +00:00
iPromKnight
d8f48fcee9 Introduce a circuit breaker, also exit out of loop if mongo failures. 2024-03-03 16:19:56 +00:00
iPromKnight
4b3bb2b5bd hotfix continue, not break - add slight delay - log params 2024-03-03 04:10:14 +00:00
iPromKnight
95fa48c851 Woke up to see a discussion about torrentio scraping: powered by community 
Was a little inspired. Now we have a database (self populating) of imdb id's - why shouldn't we actually have the ability to scrape any other instance of torrentio, or knightcrawler?

Also restructured the producer to be vertically sliced to make it easier to work with
Too much flicking back and forth between Jobs and Crawlers when configuring
 2024-03-02 18:41:57 +00:00
iPromKnight
38d4e5d5ed Allow user control of threshold 2024-02-29 21:47:07 +00:00
iPromKnight
65b8fe588b Further enhance title matching by added fuse with a threshold of a 75% match 2024-02-29 21:39:31 +00:00
Gabisonfire
db1a266a87 Merge pull request #97 from Gabisonfire/fix/premiumize 
Updated premiumize moch from upstream
 2024-02-29 09:24:03 -05:00
iPromKnight
0f0298887c Change startup url to avoid conflict 2024-02-29 11:23:18 +00:00
iPromKnight
fcc4440a7d small hotfix to use fuzzy text lookups and actually utilise the compound index. 2024-02-29 11:23:18 +00:00
iPromKnight
d0346f29bf Add cleanup of initialize mongodb, ensure indexes are created for compound searching 2024-02-28 14:57:26 +00:00
iPromKnight
1b9a01c677 BREAKING: Cleanup RabbitMQ env vars, and Github Pat 2024-02-28 12:57:55 +00:00
iPromKnight
a25aef8f58 dont restart the metadata service when it completes, and exit if its a single run 2024-02-28 11:51:46 +00:00
iPromKnight
42f55318e4 change mongodb env var in addon - breaking 2024-02-28 10:16:34 +00:00
iPromKnight
acff67d7ba Cleanup the mongodb env - breaking change 
Fixes the consumer tests
 2024-02-28 10:14:20 +00:00
iPromKnight
9c8cbf6468 Plug in the new mongo imdb lookups into the consumer 2024-02-28 04:03:50 +00:00
iPromKnight
79d0ef7f4d Create service to populate Imdb data collection within mongo 
We can use this collection as an alternative source to lookup imdb ids, which would be executed before name_to_imdb is called in the consumer.
 2024-02-27 22:39:19 +00:00
Gabisonfire
6c4282b6de Adds Nyaa Crawler 2024-02-27 10:08:39 -05:00
Gabisonfire
c7cf84ae62 Updated premiumize moch from upstream 2024-02-27 09:22:50 -05:00
iPromKnight
49a6283f26 Fix DMM so that all pages are enumerated 
Fixes #95 by switching to git trees instead of the content api.
 2024-02-27 13:51:21 +00:00
Aelfa
62380e5f3c pgrep command is part of the procps package, installed in the builder stage, but it's not present in the final stage. 
This modification adds the installation of procps in the final stage, ensuring that the pgrep command is available for health check.
 2024-02-21 03:58:36 +05:00
Aelfa
0a2c9ad1f0 Update Dockerfile 2024-02-21 03:22:16 +05:00
Aelfa
496b12c680 Resolve "pgrep: not found" issue 
Resolve "pgrep: not found" issue

Added installation of the procps package in the builder stage to ensure the pgrep command is available in the final Docker image. This resolves the "pgrep: not found" issue in the HEALTHCHECK.
 2024-02-19 01:31:14 +05:00
iPromKnight
8a61c0c5b0 Fix the entrypoint 2024-02-18 12:36:04 +00:00
iPromKnight
0925c2fb51 Cleanup boolean helpers 2024-02-13 19:51:49 +00:00