diff options
| author | Baitinq <manuelpalenzuelamerino@gmail.com> | 2022-10-24 21:22:49 +0200 |
|---|---|---|
| committer | Baitinq <manuelpalenzuelamerino@gmail.com> | 2022-10-25 01:10:34 +0200 |
| commit | 096bc77066533f4b2b11f264e37ff0f9f90fc346 (patch) | |
| tree | 92f1fcea914736a09c46e2b2077912f86a9ee653 /Cargo.lock | |
| parent | Crawler: Add crawled url filter (diff) | |
| download | OSSE-096bc77066533f4b2b11f264e37ff0f9f90fc346.tar.gz OSSE-096bc77066533f4b2b11f264e37ff0f9f90fc346.tar.bz2 OSSE-096bc77066533f4b2b11f264e37ff0f9f90fc346.zip | |
Crawler: Parse urls with the "url" crate
This fixes relative urls, makes url filtering and validation better, and many other improvements.
Diffstat (limited to 'Cargo.lock')
| -rw-r--r-- | Cargo.lock | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/Cargo.lock b/Cargo.lock index 723880c..9282a8a 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -418,6 +418,7 @@ dependencies = [ "scraper", "serde", "tokio 0.2.25", + "url", ] [[package]] |