ShrimpWorks

Forward: This project came about due to a desire to make the Unreal Archive searchable, without sacrificing user privacy or its stand-alone nature by handing search off to Google or similar “site search” functionality regular search engines implement.

Additionally I wanted data to be searchable using custom criteria rather than hoping Google does the right thing or and trying to convince it to do the right thing based on HTML scraping.

Hosted on GitHub

A simple REST service exposing some RedisSearch functionality to support simple self-hosted site search functionality.

Requirements

• Java 11 JRE
• A RediSearch instance to connect to

Build

The project is built using Gradle:

$ ./gradlew execJar

This generates a fat/uber jar in the build/libs/ directory which may be used to run the service.

Configuration and Running

The service and index schema are configured using a simple YAML config file.

A sample file may be generated by simply running:

java -jar minimum-effort-search-exec.jar > config.yml

The above will write the file config.yml with some example parameters which you may customise.

Thereafter, run the service with the config file as the first parameter:

java -jar minimum-effort-search-exec.jar config.yml

The service will start up, listening on the port you specifier in the config file.

API

Add documents to the index

Add a single document:

POST /index/add

{
  "id": "1",
  "score": 1.0,
  "fields": {
    "title": "Blue T-Shirt",
    "body": "A very basic blue t-shirt you can wear",
    "price": 100,
    "tags": "shirt,blue,clothing"
  }
}

Add multiple documents:

POST /index/addBatch

[
  {
    "id": "2",
    "score": 1.0,
    "fields": {
      "title": "Jean Pant",
      ...
    }
  },
  {
    "id": "3",
    "fields": {
      "title": "Rooi Rokkie",
      ...
    }
  },
]

Search for documents in the index:

GET /search?q=shirt&limit=10&offset=0

Parameters:

• q: Search query string
• limit: Limit the result set to this number of documents
• offset: Return documents starting at this offset. In combination with limit, allows for pagination through results.

{
  "docs": [
    {
      "id": "1",
      "payload": null,
      "score": 1.0,
      "fields": {
        "title": "Blue T-Shirt",
        "body": "A very basic blue t-shirt you can wear",
        "price": 100,
        "tags": "shirt,blue,clothing"
      }
    }
  ],
  "limit": 10,
  "offset": 0,
  "totalResults": 1
}