about summary refs log tree commit diff
path: root/crawler/src/main.rs (follow)
Commit message (Expand)AuthorAgeFilesLines
* Crawler: Add loging with env_loggerBaitinq2022-11-061-15/+12
* Indexer: Switch back to not serving frontend with actixBaitinq2022-11-051-1/+1
* Indexer+Frontend: Integrate with actixBaitinq2022-11-051-1/+1
* Misc: Cargo fmtBaitinq2022-10-301-6/+6
* Crawler: Set 4 as the maximum "crawl depth"Baitinq2022-10-301-0/+1
* Crawler: Accept max_queue_size as an argument for crawler()Baitinq2022-10-301-3/+5
* Frontend: Move app-specific code to app.rsBaitinq2022-10-301-0/+1
* Misc: Add local lib crate to share common structsBaitinq2022-10-301-7/+1
* Crawler+Indexer+Frontend: Rename structs to follow logical relationsBaitinq2022-10-291-2/+2
* Crawler: Only accept HTTP_STATUS_CODE: 200 as success in crawl_url()Baitinq2022-10-281-3/+4
* Misc: Add TODOsBaitinq2022-10-281-1/+0
* Crawler: Replace String::from with .to_string()Baitinq2022-10-271-3/+6
* Crawler: Fix bad error handling with match handlingBaitinq2022-10-251-6/+9
* Crawler: Use async ClientBaitinq2022-10-251-6/+11
* Crawler: Shuffle crawled urlsBaitinq2022-10-251-3/+2
* Crawler: Add "correct" error handlingBaitinq2022-10-251-21/+23
* Crawler: Parse urls with the "url" crateBaitinq2022-10-251-25/+24
* Crawler: Add crawled url filterBaitinq2022-10-241-1/+8
* Crawler: Set queue size to 2222Baitinq2022-10-241-1/+1
* Crawler+Indexer: Rust cleanupBaitinq2022-10-231-3/+2
* Crawler: Replace println! with dbg!Baitinq2022-10-231-7/+7
* Crawler: Remove prepending of https:// to each urlBaitinq2022-10-231-5/+5
* Crawler: Only crawl 2 urls per urlBaitinq2022-10-231-0/+6
* Crawler: Change blockingqueue to channelsBaitinq2022-10-231-11/+11
* Crawler: Implement basic async functionalityBaitinq2022-10-221-39/+45
* Crawler: Add basic indexer communicationBaitinq2022-10-211-10/+46
* Crawler: Add Err string in the craw_url methodBaitinq2022-10-201-3/+3
* Crawler: Add indexer interaction skeletonBaitinq2022-10-201-1/+5
* Crawler: Wrap crawl response in Result typeBaitinq2022-10-201-18/+23
* Crawler: Normalise relative urlsBaitinq2022-10-201-2/+17
* Crawler: Remove duplicate parsed urlsBaitinq2022-10-201-0/+3
* Crawler: Add basic html parsing and link-followingBaitinq2022-10-201-9/+34
* Crawler: Add skeleton crawler implementationBaitinq2022-10-201-0/+40
* Misc: Separate OSSE into componentsBaitinq2022-10-191-0/+3