Writing a Engine

All engines are located in the /engines directory. To write a new engine, you need to create a .py file in this directory.

The first step in writing an engine is to import the BaseEngine class from core.base_engine:

from core.base_engine import BaseEngine

The next prerequisite for writing an engine is to create a class that inherits from BaseEngine with the same name as the engine. For example, for duckduckgo.py, we create a class named DuckDuckGoEngine:

class DuckDuckGoEngine(BaseEngine):

After creating your engine class, you must implement at least one function named search() in that class:

def search(self, query: str, proxy, timeout: int = 10, page: int = 1, time_range: str = None, safesearch: int = 0, **kwargs) -> dict:

The mandatory input parameters for this function are self and query. You should also use **kwargs because there may be inputs received that your engine does not support or does not need.

The parameters sent to all engines are as follows:

• query
• page
• safesearch
• time_range
• num_results
• locale
• country
• proxy

There is no obligation to use all of them, although it is recommended to implement items like proxy. Below, we will explain the purpose of each:

Parameter Descriptions

• query: The main input for each engine, which is a string containing the user's query. Its use in engines is mandatory.

• page: The page number of the search engine, which includes an integer value from 1 to x.

• safesearch: Determines the settings for displaying or filtering explicit results. The input value can be 0, 1, or 2, which correspond to off, moderate, and on filtering, respectively.

• time_range: The value for filtering results based on a time range. Its values can be hour, day, week, month, or year.

• num_results: An integer value to limit the maximum number of results returned. If your engine inherently supports such input, use it; otherwise, you do not need to and can manually remove excess results before sending. This is done in main.py.

• locale: A filter based on language. The input value includes language codes such as en for English.

• country: A filter based on the country.

• proxy: A global proxy for all engines. It is recommended to follow this. The input value should look like this:

{
    "http": "http://127.0.0.1:8080",
    "https": "http://127.0.0.1:8080"
}

If you are using the requests library to send requests, you can directly pass it to the get input:

response = requests.get(
    url,
    headers,
    timeout,
    proxies=proxy
)

Engine Output

The output of your engine is the value returned from the search function. Its syntax should look like this:

{
    "results": [
        {
            "title": "Python Programming Language",
            "url": "www.python.org",
            "content": "Python is a programming language that lets you work quickly..."
        },
        {
            "title": "Learn Python - Free Interactive Course",
            "url": "www.learnpython.org",
            "content": "Learn Python programming with interactive tutorials..."
        },
        {
            "title": "Python for Everybody",
            "url": "www.py4e.com",
            "content": "A free online course to learn Python programming..."
        }
    ]
}

You can use other values for each result depending on the type of engine, but using title, url, and content is mandatory. Results without a title and URL in a combined state will be removed.

For further familiarity with writing engines, you can also review other engines.

Default Configurations

This section is not mandatory, but it is recommended, especially if your engine's category is something other than general.

The default parameters for each engine are retrieved from engine_params.yml in the /configs directory. For example, for Google:

GoogleEngine:
  params:
    max_page: 50
    timeout: 10
    region: "US"
    type: "general"

The main variable name should be similar to the name of your engine class, which in this case is GoogleEngine. The parameter values are placed within params. The important parameter in this section is type, which indicates the category of your engine. If you do not specify this value, your engine will fall into the general category.

The usable values for type can include one of the following:

• general
• images
• videos
• news
• books
• maps
• shaping
• other