about summary refs log tree commit diff
diff options
context:
space:
mode:
authorBaitinq <manuelpalenzuelamerino@gmail.com>2022-11-07 15:52:58 +0100
committerBaitinq <manuelpalenzuelamerino@gmail.com>2022-11-08 11:45:24 +0100
commit638c940159df496ac7fcd34d45830712c171e827 (patch)
tree391c5de5d203321354af16b9645992f0ce024b59
parentFrontend: Append "..." to truncated titles and descriptions (diff)
downloadOSSE-638c940159df496ac7fcd34d45830712c171e827.tar.gz
OSSE-638c940159df496ac7fcd34d45830712c171e827.tar.bz2
OSSE-638c940159df496ac7fcd34d45830712c171e827.zip
Misc: Update Readme to be more user friendly
-rw-r--r--README.md96
-rw-r--r--docs/frontend.pngbin0 -> 169239 bytes
2 files changed, 83 insertions, 13 deletions
diff --git a/README.md b/README.md
index b16d224..e8c6f66 100644
--- a/README.md
+++ b/README.md
@@ -1,29 +1,99 @@
-# OSSE (Overly Simple Searche Engine) (Oh-Si)
+# OSSE Search Engine
 
-An attempt to make a "functional" search engine while learning Rust :^)
+> Overly Simple Search Engine - Making search engines simple?? \
+> Pronunciation: "oh-see"
 
-## Building
+![contributors](https://img.shields.io/github/contributors/baitinq/OSSE.svg)  [![license](https://img.shields.io/github/license/baitinq/OSSE.svg)](https://github.com/Baitinq/OSSE/blob/master/LICENSE)  [![PRs welcome](https://img.shields.io/badge/PRs-welcome-ff69b4.svg)](https://github.com/baitinq/OSSE/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22)  ![Stars](https://img.shields.io/github/stars/baitinq/OSSE.svg)  ![Forks](https://img.shields.io/github/forks/baitinq/OSSE.svg)  [![code with hearth by Baitinq](https://img.shields.io/badge/%3C%2F%3E%20with%20%E2%99%A5%20by-Baitinq-ff1414.svg)](https://github.com/Baitinq)
 
-### 1. Setup your environment
+<img src="https://raw.githubusercontent.com/Baitinq/OSSE/master/docs/frontend.png" />
+
+
+## 🚩 Table of Contents
+
+- [Why?](#-why)
+- [Usage](#-usage)
+- [Features](#-features)
+- [How it works](#-how-it-works)
+- [Roadmap](#-roadmap)
+- [Contributing](#-contributing)
+- [License](#-license)
+
+## 🐂 Why?
+
+[Just for fun!](https://justforfunnoreally.dev) I really wanted to learn [Rust](https://rust-lang.org) and at the time I was really interested in how search engines worked, so there wasn't any better way of achieving both goals than with this very project!
+
+## 🤖 Usage
+
+This repository is a monorepo formed by the independent components that form the OSSE search engine.
+
+### Installing Dependencies
+
+* #### With [Nix](https://nixos.org/):
 
-#### With Nix:
 ```
 $ nix develop
 ```
-or
-```
-$ nix-shell shell.nix
-```
 
-#### Otherwise:
+* #### Otherwise:
 
-Install cargo through your preferred method.
+Install [cargo](https://doc.rust-lang.org/cargo/) and [trunk](https://trunkrs.dev) with your preferred method (such as your favorite package manager).
 
+### Running
 
-### 2. Run!
+* [Crawler](https://github.com/Baitinq/OSSE/tree/master/crawler)
+```
+$ cargo run --bin crawler
+```
+
+* [Indexer](https://github.com/Baitinq/OSSE/tree/master/indexer)
 
 ```
 $ cargo run --bin indexer
-$ cargo run --bin crawler
+```
+
+* [Frontend](https://github.com/Baitinq/OSSE/tree/master/frontend)
+```
 $ trunk serve frontend/index.html --open
 ```
+
+Once all the components are running, you can navigate to ```127.0.0.1:8080``` on your favorite web browser and start using OSSE!
+
+## 🎨 Features
+
+* [Completely Self-Hosted](https://en.wikipedia.org/wiki/Self-hosting_(web_services)) : OSSE does not use any external services, all you need is its three components (indexer, crawler & frontend) to have a "complete" search engine.
+* [Custom Indexing and Ranking Algorithms](https://github.com/Baitinq/OSSE/tree/master/indexer) : OSSE uses its own open-source indexing and ranking algorithm, meaning that its code is reviewable and improvable by third parties, ensuring its technically and morally correct functionality.
+* [Hackable]() : OSSE is built with extensibility & modularity in mind, so it is entirely feasible to replace or customize its various components.
+* [Privacy Respecting]() : As a result of OSSE being completely independent, it does not send any metadata to any services.
+
+
+## ⚙️  How it works
+
+The OSSE search engine is separated into three independent components:
+
+* ### [Indexer](https://github.com/Baitinq/OSSE/tree/master/indexer)
+This component provides both the actual search engine indexer's implementation and the REST API used to search and add indexed resources. It uses [Actix Web](https://actix.rs) for the REST API (running on port 4444). For the implementation of the actual indexer data structure, we currently use a very simple reverse index implemented with a hashmap, so all the indexed resources are currently lost each time the indexer is restarted.
+
+* ### [Crawler](https://github.com/Baitinq/OSSE/tree/master/crawler)
+This component is a simple recursive crawler that forwards the crawled raw HTML to the indexer. It uses [reqwest](https://docs.rs/reqwest/latest/reqwest) for fetching a predefined list of [root websites](https://github.com/Baitinq/OSSE/blob/master/crawler/top-1000-websites.txt) and parses them with [scraper](https://docs.rs/scraper/latest/scraper), sending the website contents to the indexer and extracting all its links, adding them to a queue of websites to be crawled. This process is "recursively" repeated indefinitely.
+
+* ### [Frontend](https://github.com/Baitinq/OSSE/tree/master/frontend)
+This component is a simple web interface to the indexer. It allows users to search and visualize results in a user friendly way. It is currently built using [Yew](https://yew.rs), which allows us to write the frontend in rust and produce a "blazingly fast" Wasm based web-ui.
+
+## 🐾 Roadmap
+
+- [x] Add frontend
+- [ ] Change indexer to use a ngram index instead of a reverse index
+- [ ] Improve frontend
+- [ ] Improve responsiveness of searching when the indexer is recieving info from crawlers
+- [ ] Rust cleanup
+- [ ] Improve page ranking algorithm
+
+
+## 💬 Contributing
+
+> "If you have any ideas or patches, please do not hesitate to contribute to OSSE!"
+
+## 📜 License
+
+This software is licensed under the [BSD-2-Clause](https://github.com/baitinq/OSSE/blob/master/LICENSE) © [Baitinq](https://github.com/Baitinq).
+
diff --git a/docs/frontend.png b/docs/frontend.png
new file mode 100644
index 0000000..c2742fa
--- /dev/null
+++ b/docs/frontend.png
Binary files differ