OSSE - Overly Simple Search Engine :)

	Commit message (Collapse)	Author	Age	Files	Lines
*	Crawler: Replace println! with dbg!	Baitinq	2022-10-23	1	-7/+7
\|
*	Crawler: Remove prepending of https:// to each url	Baitinq	2022-10-23	2	-1006/+1006
\| \| \| \| \|	We now prepend it to the top-1000-urls list. This fixes crawled urls having two https://
*	Crawler: Only crawl 2 urls per url	Baitinq	2022-10-23	1	-0/+6
\| \| \| \|	This makes it so that we dont get rate limited from websites.
*	Crawler: Change blockingqueue to channels	Baitinq	2022-10-23	3	-19/+45
\| \| \| \| \|	We now use the async-channel channels implementation. This allows us to have bounded async channels.
*	Indexer: Listen on 0.0.0.0	Baitinq	2022-10-23	1	-1/+1
\|
*	Indexer: Implement basic reverse index searching and adding	Baitinq	2022-10-22	3	-15/+163
\| \| \| \|	Very inefficient but kind of functional:::)))))))
*	Crawler: Implement basic async functionality	Baitinq	2022-10-22	3	-93/+285
\|
*	Crawler: Add basic indexer communication	Baitinq	2022-10-21	2	-11/+48
\|
*	Indexer: Add skeleton http rest endpoint functionality	Baitinq	2022-10-21	3	-1/+539
\| \| \| \|	/search and /resource endpoint.
*	Crawler: Add Err string in the craw_url method	Baitinq	2022-10-20	1	-3/+3
\|
*	Crawler: Add indexer interaction skeleton	Baitinq	2022-10-20	1	-1/+5
\|
*	Crawler: Wrap crawl response in Result type	Baitinq	2022-10-20	1	-18/+23
\|
*	Crawler: Normalise relative urls	Baitinq	2022-10-20	1	-2/+17
\| \| \| \| \|	We now normalise urls starting with / (relative to root) and // (relative to protocol)
*	Crawler: Remove duplicate parsed urls	Baitinq	2022-10-20	3	-0/+20
\|
*	Crawler: Add basic html parsing and link-following	Baitinq	2022-10-20	3	-9/+1561
\| \| \| \| \|	Extremely basic implementation. Needs max queue size, error handling, formatting of parsed links.
*	Crawler: Add skeleton crawler implementation	Baitinq	2022-10-20	4	-0/+1051
\| \| \| \| \|	Starts by filling a queue with the top 1000 most visited sites. "Crawls" each one (empty fn), and blocks for new elements on the queue.
*	Misc: Change to use "oxalica/rust-overlay" for the nix development shell	Baitinq	2022-10-19	3	-26/+90
\| \| \| \|	This fixes vscode not being able to find rust-analyzer and rust-src
*	Misc: Separate OSSE into components	Baitinq	2022-10-19	9	-10/+56
\| \| \| \| \|	We now have a cargo workspace with the Crawler, Client and Indexer packages.
*	Initial Commit!	Baitinq	2022-10-19	10	-0/+136
	This is the initial commit for this experiment of a search engine. I hope I can learn a lot from this!