Files
knightcrawler/src/producer/Program.cs
iPromKnight ab17ef81be Big rewrite - distributed consumers for ingestion / scraping(scalable) - single producer written in c#.
Changed from page scraping to rss xml scraping
Includes RealDebridManager hashlist decoding (requires a github readonly PAT as requests must be authenticated) - This allows ingestion of 200k+ entries in a few hours.
Simplifies a lot of torrentio to deal with new data
2024-02-01 16:38:45 +00:00

17 lines
361 B
C#

var builder = WebApplication.CreateBuilder(args);
builder.Configuration
.AddScrapeConfiguration();
builder.Host
.SetupSerilog(builder.Configuration);
builder.Services
.RegisterMassTransit(builder.Configuration)
.AddDataStorage()
.AddCrawlers()
.AddQuartz(builder.Configuration);
var host = builder.Build();
await host.RunAsync();