diff options
| author | Baitinq <[email protected]> | 2022-10-23 12:06:33 +0200 |
|---|---|---|
| committer | Baitinq <[email protected]> | 2022-10-23 12:06:35 +0200 |
| commit | 4445c4d168df1ab71431da9db1a053629ed4d0d9 (patch) | |
| tree | 2c129a481d16ce252d0b634491220b90bd0d9886 /indexer/src | |
| parent | Crawler: Change blockingqueue to channels (diff) | |
| download | OSSE-4445c4d168df1ab71431da9db1a053629ed4d0d9.tar.gz OSSE-4445c4d168df1ab71431da9db1a053629ed4d0d9.tar.bz2 OSSE-4445c4d168df1ab71431da9db1a053629ed4d0d9.zip | |
Crawler: Only crawl 2 urls per url
This makes it so that we dont get rate limited from websites.
Diffstat (limited to 'indexer/src')
0 files changed, 0 insertions, 0 deletions