| Commit message (Collapse) | Author | Age | Files | Lines | |
|---|---|---|---|---|---|
| * | Misc: Remove unneeded dependencies | Baitinq | 2022-10-30 | 1 | -1/+0 |
| | | |||||
| * | Misc: Add local lib crate to share common structs | Baitinq | 2022-10-30 | 1 | -0/+1 |
| | | |||||
| * | Crawler: Use async Client | Baitinq | 2022-10-25 | 1 | -1/+1 |
| | | |||||
| * | Crawler: Shuffle crawled urls | Baitinq | 2022-10-25 | 1 | -1/+2 |
| | | |||||
| * | Crawler: Parse urls with the "url" crate | Baitinq | 2022-10-25 | 1 | -0/+1 |
| | | | | | | This fixes relative urls, makes url filtering and validation better, and many other improvements. | ||||
| * | Crawler: Change blockingqueue to channels | Baitinq | 2022-10-23 | 1 | -1/+1 |
| | | | | | | We now use the async-channel channels implementation. This allows us to have bounded async channels. | ||||
| * | Crawler: Implement basic async functionality | Baitinq | 2022-10-22 | 1 | -0/+1 |
| | | |||||
| * | Crawler: Add basic indexer communication | Baitinq | 2022-10-21 | 1 | -1/+2 |
| | | |||||
| * | Crawler: Remove duplicate parsed urls | Baitinq | 2022-10-20 | 1 | -0/+1 |
| | | |||||
| * | Crawler: Add basic html parsing and link-following | Baitinq | 2022-10-20 | 1 | -0/+2 |
| | | | | | | Extremely basic implementation. Needs max queue size, error handling, formatting of parsed links. | ||||
| * | Crawler: Add skeleton crawler implementation | Baitinq | 2022-10-20 | 1 | -0/+1 |
| | | | | | | Starts by filling a queue with the top 1000 most visited sites. "Crawls" each one (empty fn), and blocks for new elements on the queue. | ||||
| * | Misc: Separate OSSE into components | Baitinq | 2022-10-19 | 1 | -0/+12 |
| We now have a cargo workspace with the Crawler, Client and Indexer packages. | |||||
 |
index : OSSE | |
| Overly Simple Search Engine :) |
| about summary refs log tree commit diff |