| Commit message (Collapse) | Author | Age | Files | Lines | |
|---|---|---|---|---|---|
| * | Crawler: Add Err string in the craw_url method | Baitinq | 2022-10-20 | 1 | -3/+3 |
| | | |||||
| * | Crawler: Add indexer interaction skeleton | Baitinq | 2022-10-20 | 1 | -1/+5 |
| | | |||||
| * | Crawler: Wrap crawl response in Result type | Baitinq | 2022-10-20 | 1 | -18/+23 |
| | | |||||
| * | Crawler: Normalise relative urls | Baitinq | 2022-10-20 | 1 | -2/+17 |
| | | | | | | We now normalise urls starting with / (relative to root) and // (relative to protocol) | ||||
| * | Crawler: Remove duplicate parsed urls | Baitinq | 2022-10-20 | 3 | -0/+20 |
| | | |||||
| * | Crawler: Add basic html parsing and link-following | Baitinq | 2022-10-20 | 3 | -9/+1561 |
| | | | | | | Extremely basic implementation. Needs max queue size, error handling, formatting of parsed links. | ||||
| * | Crawler: Add skeleton crawler implementation | Baitinq | 2022-10-20 | 4 | -0/+1051 |
| | | | | | | Starts by filling a queue with the top 1000 most visited sites. "Crawls" each one (empty fn), and blocks for new elements on the queue. | ||||
| * | Misc: Change to use "oxalica/rust-overlay" for the nix development shell | Baitinq | 2022-10-19 | 3 | -26/+90 |
| | | | | | This fixes vscode not being able to find rust-analyzer and rust-src | ||||
| * | Misc: Separate OSSE into components | Baitinq | 2022-10-19 | 9 | -10/+56 |
| | | | | | | We now have a cargo workspace with the Crawler, Client and Indexer packages. | ||||
| * | Initial Commit! | Baitinq | 2022-10-19 | 10 | -0/+136 |
| This is the initial commit for this experiment of a search engine. I hope I can learn a lot from this! | |||||