about summary refs log tree commit diff
path: root/crawler (follow)
Commit message (Expand)AuthorAgeFilesLines
* Crawler: Add loging with env_loggerBaitinq2022-11-062-15/+14
* Indexer: Switch back to not serving frontend with actixBaitinq2022-11-051-1/+1
* Indexer+Frontend: Integrate with actixBaitinq2022-11-051-1/+1
* Misc: Cargo fmtBaitinq2022-10-301-6/+6
* Crawler: Set 4 as the maximum "crawl depth"Baitinq2022-10-301-0/+1
* Crawler: Accept max_queue_size as an argument for crawler()Baitinq2022-10-301-3/+5
* Frontend: Move app-specific code to app.rsBaitinq2022-10-301-0/+1
* Misc: Remove unneeded dependenciesBaitinq2022-10-301-1/+0
* Misc: Add local lib crate to share common structsBaitinq2022-10-302-7/+2
* Crawler+Indexer+Frontend: Rename structs to follow logical relationsBaitinq2022-10-291-2/+2
* Crawler: Only accept HTTP_STATUS_CODE: 200 as success in crawl_url()Baitinq2022-10-281-3/+4
* Misc: Add TODOsBaitinq2022-10-281-1/+0
* Crawler: Replace String::from with .to_string()Baitinq2022-10-271-3/+6
* Crawler: Fix bad error handling with match handlingBaitinq2022-10-251-6/+9
* Crawler: Use async ClientBaitinq2022-10-252-7/+12
* Crawler: Shuffle crawled urlsBaitinq2022-10-252-4/+4
* Crawler: Add "correct" error handlingBaitinq2022-10-251-21/+23
* Crawler: Parse urls with the "url" crateBaitinq2022-10-252-25/+25
* Crawler: Add crawled url filterBaitinq2022-10-241-1/+8
* Crawler: Set queue size to 2222Baitinq2022-10-241-1/+1
* Crawler+Indexer: Rust cleanupBaitinq2022-10-231-3/+2
* Crawler: Replace println! with dbg!Baitinq2022-10-231-7/+7
* Crawler: Remove prepending of https:// to each urlBaitinq2022-10-232-1006/+1006
* Crawler: Only crawl 2 urls per urlBaitinq2022-10-231-0/+6
* Crawler: Change blockingqueue to channelsBaitinq2022-10-232-12/+12
* Crawler: Implement basic async functionalityBaitinq2022-10-222-39/+46
* Crawler: Add basic indexer communicationBaitinq2022-10-212-11/+48
* Crawler: Add Err string in the craw_url methodBaitinq2022-10-201-3/+3
* Crawler: Add indexer interaction skeletonBaitinq2022-10-201-1/+5
* Crawler: Wrap crawl response in Result typeBaitinq2022-10-201-18/+23
* Crawler: Normalise relative urlsBaitinq2022-10-201-2/+17
* Crawler: Remove duplicate parsed urlsBaitinq2022-10-202-0/+4
* Crawler: Add basic html parsing and link-followingBaitinq2022-10-202-9/+36
* Crawler: Add skeleton crawler implementationBaitinq2022-10-203-0/+1042
* Misc: Separate OSSE into componentsBaitinq2022-10-192-0/+15