* Process DMM all locally
single call to github to download the repo archive.
remove need for PAT
update RTN to 0.2.13
change to batch_parse for title parsing from RTN
* introduce concurrent dictionary, and parallelism
* update rtn to 025
* Implement movie / show type parsing
* switch to RTN in collectors
* ensure env for pythonnet is loaded, and that requirements copy for qbit
* version bump
* add new indexes, and change year column to int
* Change gist to gin, and change year to int
* Producer changes for new gin query
* Fully map the rtn response using json dump from Pydantic
Also updates Rtn to 0.1.9
* Add housekeeping script to reconcile imdb ids.
* Join Torrent onto the ingested torrent table
Ensure that a torrent can always find the details of where it came from, and how it was parsed.
* Version bump for release
* missing quote on table name
* [wip] bridge python and c# and bring in rank torrent name
* Container restores package now
Includes two dev scripts to install the python packages locally for debugging purposes.
* Introduce slightly turned title matching scoring, by making it length aware
this should help with sequels such as Terminator 2, vs Terminator etc
* Version bump
Also fixes postgres healthcheck so that it utilises the user from the stack.env file
* Ensure the collectors manage sagas in their own keyspace, as we do not want overlap (they have the same correlation ids internally from the exchange)
* version bump
* Converted metadata service to redis
* move to postgres instead
* fix global usings
* [skip ci] optimize wolverine by prebuilding static types
* [skip ci] Stop indexing mac folder indexes
* [skip ci] producer, metadata and migrations
removed mongodb
added redis cache
imdb meta in postgres
Enable pgtrm
Create trigrams index
Add search meta postgres function
* [skip ci] get rid of node folder, replace mongo with redis in consumer
also wire up postgres metadata searches
* [skip ci] change mongo to redis in the addon
* [skip ci] jackettio to redis
* Rest of mongo removed...
* Cleaner rerunning of metadata - without conflicts
* Add akas import as well as basic metadata
* Include episodes file too
* cascade truncate pre-import
* reverse order to avoid cascadeing
* separate out clean to separate handler
* Switch producer to use metadata matching pre-preocessing dmm
* More work
* Still porting PTN
* PTN port, adding tests
* [skip ci] Codec tests
* [skip ci] Complete Collection handler tests
* [skip ci] container tests
* [skip ci] Convert handlers tests
* [skip ci] DateHandler tests
* [skip ci] Dual Audio matching tests
* [skip ci] episode code tests
* [skip ci] Extended handler tests
* [skip ci] group handler tests
* [skip ci] some broken stuff right now
* [skip ci] more ptn
* [skip ci] PTN now in a separate nuget package, rebased this on the redis changes - i need them.
* [skip ci] Wire up PTN port. Tired - will test tomorrow
* [skip ci] Needs a lot of work - too many titles being missed now
* cleaner. done?
* Handle the date in the imdb search
- add integer function to confirm its a valid integer
- use the input date as a range of -+1 year
* [skip ci] Start of collector service for RD
[skip ci] WIP
Implemented metadata saga, along with channels to process up to a maximum of 100 infohashes each time
The saga will rety for each infohas by requeuing up to three times, before just marking as complete for that infoHash - meaning no data will be updated in the db for that torrent.
[skip ci] Ready to test with queue publishing
Will provision a fanout exchange if it doesn't exist, and create and bind a queue to it. Listens to the queue with 50 prefetch count.
Still needs PTN rewrite bringing in to parse the filename response from real debrid, and extract season and episode numbers if the file is a tvshow
[skip ci] Add Debrid Collector Build Job
Debrid Collector ready for testing
New consumer, new collector, producer has meta lookup and anti porn measures
[skip ci] WIP - moving from wolverine to MassTransit.
not happy that wolverine cannot effectively control saga concurrency. we need to really.
[skip ci] Producer and new Consumer moved to MassTransit
Just the debrid collector to go now, then to write the optional qbit collector.
Collector now switched to mass transit too
hide porn titles in logs, clean up cache name in redis for imdb titles
[skip ci] Allow control of queues
[skip ci] Update deployment
Remove old consumer, fix deployment files, fix dockerfiles for shared project import
fix base deployment
* Add collector missing env var
* edits to kick off builds
* Add optional qbit deployment which qbit collector will use
* Qbit collector done
* reorder compose, and bring both qbit and qbitcollector into the compose, with 0 replicas as default
* Clean up compose file
* Ensure debrid collector errors if no debrid api key
