tools/knightcrawler
1
0
Fork 0
mirror of https://github.com/knightcrawler-stremio/knightcrawler.git synced 2024-12-20 03:29:51 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Introduce max batch size, and configurable publish window

Browse Source 
Still need to implement queue size limit
Also fixes env var consistency between addon and consumer
This commit is contained in:
iPromKnight
2024-02-02 13:49:42 +00:00
parent 0644fe7720
commit 68edaba308
41 changed files with 88 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
env/consumer.env vendored
View File

@@ -1,6 +1,6 @@
TZ=London/Europe
MONGODB_URI=mongodb://mongo:mongo@mongodb/selfhostio?tls=false&authSource=admin
POSTGRES_DATABASE_URI=postgres://postgres:postgres@postgres/selfhostio
DATABASE_URI=postgres://postgres:postgres@postgres/selfhostio
RABBIT_URI=amqp://guest:guest@rabbitmq:5672/?heartbeat=30
QUEUE_NAME=ingested
JOB_CONCURRENCY=5

6
env/producer.env vendored
View File

@@ -1,4 +1,10 @@
ScrapeConfiguration__StorageConnectionString=host=postgres;username=postgres;password=postgres;database=selfhostio;
RabbitMqConfiguration__Host=rabbitmq
RabbitMqConfiguration__QueueName=ingested
RabbitMqConfiguration__Username=guest
RabbitMqConfiguration__Password=guest
RabbitMqConfiguration__Durable=true
RabbitMqConfiguration__MaxQueueSize=1000
RabbitMqConfiguration__MaxPublishBatchSize=100
RabbitMqConfiguration__PublishIntervalInSeconds=10
GithubSettings__PAT=