mirror of
https://github.com/searxng/searxng
synced 2024-01-01 19:24:07 +01:00
[doc] build from commit a576f69c12
This commit is contained in:
commit
97633487b0
347 changed files with 79365 additions and 0 deletions
190
_sources/dev/contribution_guide.rst.txt
Normal file
190
_sources/dev/contribution_guide.rst.txt
Normal file
|
|
@ -0,0 +1,190 @@
|
|||
.. _how to contribute:
|
||||
|
||||
=================
|
||||
How to contribute
|
||||
=================
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
Prime directives: Privacy, Hackability
|
||||
======================================
|
||||
|
||||
SearXNG has two prime directives, **privacy-by-design and hackability** . The
|
||||
hackability comes in three levels:
|
||||
|
||||
- support of search engines
|
||||
- plugins to alter search behaviour
|
||||
- hacking SearXNG itself
|
||||
|
||||
Note the lack of "world domination" among the directives. SearXNG has no
|
||||
intention of wide mass-adoption, rounded corners, etc. The prime directive
|
||||
"privacy" deserves a separate chapter, as it's quite uncommon unfortunately.
|
||||
|
||||
Privacy-by-design
|
||||
-----------------
|
||||
|
||||
SearXNG was born out of the need for a **privacy-respecting** search tool which
|
||||
can be extended easily to maximize both, its search and its privacy protecting
|
||||
capabilities.
|
||||
|
||||
A few widely used features work differently or turned off by default or not
|
||||
implemented at all **as a consequence of privacy-by-design**.
|
||||
|
||||
If a feature reduces the privacy preserving aspects of searx, it should be
|
||||
switched off by default or should not implemented at all. There are plenty of
|
||||
search engines already providing such features. If a feature reduces the
|
||||
protection of searx, users must be informed about the effect of choosing to
|
||||
enable it. Features that protect privacy but differ from the expectations of
|
||||
the user should also be explained.
|
||||
|
||||
Also, if you think that something works weird with searx, it's might be because
|
||||
of the tool you use is designed in a way to interfere with the privacy respect.
|
||||
Submitting a bugreport to the vendor of the tool that misbehaves might be a good
|
||||
feedback to reconsider the disrespect to its customers (e.g. ``GET`` vs ``POST``
|
||||
requests in various browsers).
|
||||
|
||||
Remember the other prime directive of SearXNG is to be hackable, so if the above
|
||||
privacy concerns do not fancy you, simply fork it.
|
||||
|
||||
*Happy hacking.*
|
||||
|
||||
Code
|
||||
====
|
||||
|
||||
.. _PEP8: https://www.python.org/dev/peps/pep-0008/
|
||||
.. _Conventional Commits: https://www.conventionalcommits.org/
|
||||
.. _Git Commit Good Practice: https://wiki.openstack.org/wiki/GitCommitMessages
|
||||
.. _Structural split of changes:
|
||||
https://wiki.openstack.org/wiki/GitCommitMessages#Structural_split_of_changes
|
||||
.. _gitmoji: https://gitmoji.carloscuesta.me/
|
||||
.. _Semantic PR: https://github.com/zeke/semantic-pull-requests
|
||||
|
||||
.. sidebar:: Create good commits!
|
||||
|
||||
- `Structural split of changes`_
|
||||
- `Conventional Commits`_
|
||||
- `Git Commit Good Practice`_
|
||||
- some like to use: gitmoji_
|
||||
- not yet active: `Semantic PR`_
|
||||
|
||||
In order to submit a patch, please follow the steps below:
|
||||
|
||||
- Follow coding conventions.
|
||||
|
||||
- PEP8_ standards apply, except the convention of line length
|
||||
- Maximum line length is 120 characters
|
||||
|
||||
- The cardinal rule for creating good commits is to ensure there is only one
|
||||
*logical change* per commit / read `Structural split of changes`_
|
||||
|
||||
- Check if your code breaks existing tests. If so, update the tests or fix your
|
||||
code.
|
||||
|
||||
- If your code can be unit-tested, add unit tests.
|
||||
|
||||
- Add yourself to the :origin:`AUTHORS.rst` file.
|
||||
|
||||
- Choose meaningful commit messages, read `Conventional Commits`_
|
||||
|
||||
.. code::
|
||||
|
||||
<type>[optional scope]: <description>
|
||||
|
||||
[optional body]
|
||||
|
||||
[optional footer(s)]
|
||||
|
||||
- Create a pull request.
|
||||
|
||||
For more help on getting started with SearXNG development, see :ref:`devquickstart`.
|
||||
|
||||
|
||||
Translation
|
||||
===========
|
||||
|
||||
Translation currently takes place on :ref:`weblate <translation>`.
|
||||
|
||||
|
||||
.. _contrib docs:
|
||||
|
||||
Documentation
|
||||
=============
|
||||
|
||||
.. _Sphinx: https://www.sphinx-doc.org
|
||||
.. _reST: https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
|
||||
|
||||
.. sidebar:: The reST sources
|
||||
|
||||
has been moved from ``gh-branch`` into ``master`` (:origin:`docs`).
|
||||
|
||||
The documentation is built using Sphinx_. So in order to be able to generate
|
||||
the required files, you have to install it on your system. Much easier, use
|
||||
our :ref:`makefile`.
|
||||
|
||||
Here is an example which makes a complete rebuild:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ make docs.clean docs.html
|
||||
...
|
||||
The HTML pages are in dist/docs.
|
||||
|
||||
.. _make docs.live:
|
||||
|
||||
live build
|
||||
----------
|
||||
|
||||
.. _sphinx-autobuild:
|
||||
https://github.com/executablebooks/sphinx-autobuild/blob/master/README.md
|
||||
|
||||
.. sidebar:: docs.clean
|
||||
|
||||
It is recommended to assert a complete rebuild before deploying (use
|
||||
``docs.clean``).
|
||||
|
||||
Live build is like WYSIWYG. If you want to edit the documentation, its
|
||||
recommended to use. The Makefile target ``docs.live`` builds the docs, opens
|
||||
URL in your favorite browser and rebuilds every time a reST file has been
|
||||
changed (:ref:`make docs.clean`).
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ make docs.live
|
||||
...
|
||||
The HTML pages are in dist/docs.
|
||||
... Serving on http://0.0.0.0:8000
|
||||
... Start watching changes
|
||||
|
||||
Live builds are implemented by sphinx-autobuild_. Use environment
|
||||
``$(SPHINXOPTS)`` to pass arguments to the sphinx-autobuild_ command. Except
|
||||
option ``--host`` (which is always set to ``0.0.0.0``) you can pass any
|
||||
argument. E.g to find and use a free port, use:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ SPHINXOPTS="--port 0" make docs.live
|
||||
...
|
||||
... Serving on http://0.0.0.0:50593
|
||||
...
|
||||
|
||||
|
||||
.. _deploy on github.io:
|
||||
|
||||
deploy on github.io
|
||||
-------------------
|
||||
|
||||
To deploy documentation at :docs:`github.io <.>` use Makefile target :ref:`make
|
||||
docs.gh-pages`, which builds the documentation and runs all the needed git add,
|
||||
commit and push:
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ make docs.clean docs.gh-pages
|
||||
|
||||
.. attention::
|
||||
|
||||
If you are working in your own brand, don't forget to adjust your
|
||||
:ref:`settings brand`.
|
||||
14
_sources/dev/engines/demo/demo_offline.rst.txt
Normal file
14
_sources/dev/engines/demo/demo_offline.rst.txt
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
.. _demo offline engine:
|
||||
|
||||
===================
|
||||
Demo Offline Engine
|
||||
===================
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.demo_offline
|
||||
:members:
|
||||
|
||||
14
_sources/dev/engines/demo/demo_online.rst.txt
Normal file
14
_sources/dev/engines/demo/demo_online.rst.txt
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
.. _demo online engine:
|
||||
|
||||
==================
|
||||
Demo Online Engine
|
||||
==================
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.demo_online
|
||||
:members:
|
||||
|
||||
571
_sources/dev/engines/engine_overview.rst.txt
Normal file
571
_sources/dev/engines/engine_overview.rst.txt
Normal file
|
|
@ -0,0 +1,571 @@
|
|||
.. _engines-dev:
|
||||
|
||||
===============
|
||||
Engine Overview
|
||||
===============
|
||||
|
||||
.. contents::
|
||||
:depth: 3
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. _metasearch-engine: https://en.wikipedia.org/wiki/Metasearch_engine
|
||||
|
||||
.. sidebar:: Further reading ..
|
||||
|
||||
- :ref:`configured engines`
|
||||
- :ref:`settings engine`
|
||||
|
||||
SearXNG is a metasearch-engine_, so it uses different search engines to provide
|
||||
better results.
|
||||
|
||||
Because there is no general search API which could be used for every search
|
||||
engine, an adapter has to be built between SearXNG and the external search
|
||||
engines. Adapters are stored under the folder :origin:`searx/engines`.
|
||||
|
||||
.. _general engine configuration:
|
||||
|
||||
General Engine Configuration
|
||||
============================
|
||||
|
||||
It is required to tell SearXNG the type of results the engine provides. The
|
||||
arguments can be set in the engine file or in the settings file (normally
|
||||
``settings.yml``). The arguments in the settings file override the ones in the
|
||||
engine file.
|
||||
|
||||
It does not matter if an option is stored in the engine file or in the settings.
|
||||
However, the standard way is the following:
|
||||
|
||||
.. _engine file:
|
||||
|
||||
Engine File
|
||||
-----------
|
||||
|
||||
.. table:: Common options in the engine module
|
||||
:width: 100%
|
||||
|
||||
======================= =========== ========================================================
|
||||
argument type information
|
||||
======================= =========== ========================================================
|
||||
categories list categories, in which the engine is working
|
||||
paging boolean support multiple pages
|
||||
time_range_support boolean support search time range
|
||||
engine_type str - ``online`` :ref:`[ref] <online engines>` by
|
||||
default, other possibles values are:
|
||||
- ``offline`` :ref:`[ref] <offline engines>`
|
||||
- ``online_dictionary`` :ref:`[ref] <online dictionary>`
|
||||
- ``online_currency`` :ref:`[ref] <online currency>`
|
||||
- ``online_url_search`` :ref:`[ref] <online url search>`
|
||||
======================= =========== ========================================================
|
||||
|
||||
.. _engine settings:
|
||||
|
||||
Engine ``settings.yml``
|
||||
-----------------------
|
||||
|
||||
For a more detailed description, see :ref:`settings engine` in the :ref:`settings.yml`.
|
||||
|
||||
.. table:: Common options in the engine setup (``settings.yml``)
|
||||
:width: 100%
|
||||
|
||||
======================= =========== ==================================================
|
||||
argument type information
|
||||
======================= =========== ==================================================
|
||||
name string name of search-engine
|
||||
engine string name of searxng-engine (file name without ``.py``)
|
||||
enable_http bool enable HTTP (by default only HTTPS is enabled).
|
||||
shortcut string shortcut of search-engine
|
||||
timeout string specific timeout for search-engine
|
||||
display_error_messages boolean display error messages on the web UI
|
||||
proxies dict set proxies for a specific engine
|
||||
(e.g. ``proxies : {http: socks5://proxy:port,
|
||||
https: socks5://proxy:port}``)
|
||||
======================= =========== ==================================================
|
||||
|
||||
.. _engine overrides:
|
||||
|
||||
Overrides
|
||||
---------
|
||||
|
||||
A few of the options have default values in the namespace of the engine's python
|
||||
module, but are often overwritten by the settings. If ``None`` is assigned to an
|
||||
option in the engine file, it has to be redefined in the settings, otherwise
|
||||
SearXNG will not start with that engine (global names with a leading underline can
|
||||
be ``None``).
|
||||
|
||||
Here is an very simple example of the global names in the namespace of engine's
|
||||
module:
|
||||
|
||||
.. code:: python
|
||||
|
||||
# engine dependent config
|
||||
categories = ['general']
|
||||
paging = True
|
||||
_non_overwritten_global = 'foo'
|
||||
|
||||
|
||||
.. table:: The naming of overrides is arbitrary / recommended overrides are:
|
||||
:width: 100%
|
||||
|
||||
======================= =========== ===========================================
|
||||
argument type information
|
||||
======================= =========== ===========================================
|
||||
base_url string base-url, can be overwritten to use same
|
||||
engine on other URL
|
||||
number_of_results int maximum number of results per request
|
||||
language string ISO code of language and country like en_US
|
||||
api_key string api-key if required by engine
|
||||
======================= =========== ===========================================
|
||||
|
||||
.. _engine request:
|
||||
|
||||
Making a Request
|
||||
================
|
||||
|
||||
To perform a search an URL have to be specified. In addition to specifying an
|
||||
URL, arguments can be passed to the query.
|
||||
|
||||
.. _engine request arguments:
|
||||
|
||||
Passed Arguments (request)
|
||||
--------------------------
|
||||
|
||||
These arguments can be used to construct the search query. Furthermore,
|
||||
parameters with default value can be redefined for special purposes.
|
||||
|
||||
.. _engine request online:
|
||||
|
||||
.. table:: If the ``engine_type`` is :py:obj:`online
|
||||
<searx.search.processors.online.OnlineProcessor.get_params>`
|
||||
:width: 100%
|
||||
|
||||
====================== ============== ========================================================================
|
||||
argument type default-value, information
|
||||
====================== ============== ========================================================================
|
||||
url str ``''``
|
||||
method str ``'GET'``
|
||||
headers set ``{}``
|
||||
data set ``{}``
|
||||
cookies set ``{}``
|
||||
verify bool ``True``
|
||||
headers.User-Agent str a random User-Agent
|
||||
category str current category, like ``'general'``
|
||||
safesearch int ``0``, between ``0`` and ``2`` (normal, moderate, strict)
|
||||
time_range Optional[str] ``None``, can be ``day``, ``week``, ``month``, ``year``
|
||||
pageno int current pagenumber
|
||||
searxng_locale str SearXNG's locale selected by user. Specific language code like
|
||||
``'en'``, ``'en-US'``, or ``'all'`` if unspecified.
|
||||
====================== ============== ========================================================================
|
||||
|
||||
|
||||
.. _engine request online_dictionary:
|
||||
|
||||
.. table:: If the ``engine_type`` is :py:obj:`online_dictionary
|
||||
<searx.search.processors.online_dictionary.OnlineDictionaryProcessor.get_params>`,
|
||||
in addition to the :ref:`online <engine request online>` arguments:
|
||||
:width: 100%
|
||||
|
||||
====================== ============== ========================================================================
|
||||
argument type default-value, information
|
||||
====================== ============== ========================================================================
|
||||
from_lang str specific language code like ``'en_US'``
|
||||
to_lang str specific language code like ``'en_US'``
|
||||
query str the text query without the languages
|
||||
====================== ============== ========================================================================
|
||||
|
||||
.. _engine request online_currency:
|
||||
|
||||
.. table:: If the ``engine_type`` is :py:obj:`online_currency
|
||||
<searx.search.processors.online_currency.OnlineCurrencyProcessor.get_params>`,
|
||||
in addition to the :ref:`online <engine request online>` arguments:
|
||||
:width: 100%
|
||||
|
||||
====================== ============== ========================================================================
|
||||
argument type default-value, information
|
||||
====================== ============== ========================================================================
|
||||
amount float the amount to convert
|
||||
from str ISO 4217 code
|
||||
to str ISO 4217 code
|
||||
from_name str currency name
|
||||
to_name str currency name
|
||||
====================== ============== ========================================================================
|
||||
|
||||
.. _engine request online_url_search:
|
||||
|
||||
.. table:: If the ``engine_type`` is :py:obj:`online_url_search
|
||||
<searx.search.processors.online_url_search.OnlineUrlSearchProcessor.get_params>`,
|
||||
in addition to the :ref:`online <engine request online>` arguments:
|
||||
:width: 100%
|
||||
|
||||
====================== ============== ========================================================================
|
||||
argument type default-value, information
|
||||
====================== ============== ========================================================================
|
||||
search_url dict URLs from the search query:
|
||||
|
||||
.. code:: python
|
||||
|
||||
{
|
||||
'http': str,
|
||||
'ftp': str,
|
||||
'data:image': str
|
||||
}
|
||||
====================== ============== ========================================================================
|
||||
|
||||
Specify Request
|
||||
---------------
|
||||
|
||||
The function :py:func:`def request(query, params):
|
||||
<searx.engines.demo_online.request>` always returns the ``params`` variable, the
|
||||
following parameters can be used to specify a search request:
|
||||
|
||||
.. table::
|
||||
:width: 100%
|
||||
|
||||
=================== =========== ==========================================================================
|
||||
argument type information
|
||||
=================== =========== ==========================================================================
|
||||
url str requested url
|
||||
method str HTTP request method
|
||||
headers set HTTP header information
|
||||
data set HTTP data information
|
||||
cookies set HTTP cookies
|
||||
verify bool Performing SSL-Validity check
|
||||
allow_redirects bool Follow redirects
|
||||
max_redirects int maximum redirects, hard limit
|
||||
soft_max_redirects int maximum redirects, soft limit. Record an error but don't stop the engine
|
||||
raise_for_httperror bool True by default: raise an exception if the HTTP code of response is >= 300
|
||||
=================== =========== ==========================================================================
|
||||
|
||||
|
||||
.. _engine results:
|
||||
.. _engine media types:
|
||||
|
||||
Result Types (``template``)
|
||||
===========================
|
||||
|
||||
Each result item of an engine can be of different media-types. Currently the
|
||||
following media-types are supported. To set another media-type as
|
||||
:ref:`template default`, the parameter ``template`` must be set to the desired
|
||||
type.
|
||||
|
||||
.. _template default:
|
||||
|
||||
``default``
|
||||
-----------
|
||||
|
||||
.. table:: Parameter of the **default** media type:
|
||||
:width: 100%
|
||||
|
||||
========================= =====================================================
|
||||
result-parameter information
|
||||
========================= =====================================================
|
||||
url string, url of the result
|
||||
title string, title of the result
|
||||
content string, general result-text
|
||||
publishedDate :py:class:`datetime.datetime`, time of publish
|
||||
========================= =====================================================
|
||||
|
||||
|
||||
.. _template images:
|
||||
|
||||
``images``
|
||||
----------
|
||||
|
||||
.. list-table:: Parameter of the **images** media type
|
||||
:header-rows: 2
|
||||
:width: 100%
|
||||
|
||||
* - result-parameter
|
||||
- Python type
|
||||
- information
|
||||
|
||||
* - template
|
||||
- :py:class:`str`
|
||||
- is set to ``images.html``
|
||||
|
||||
* - url
|
||||
- :py:class:`str`
|
||||
- url to the result site
|
||||
|
||||
* - title
|
||||
- :py:class:`str`
|
||||
- title of the result
|
||||
|
||||
* - content
|
||||
- :py:class:`str`
|
||||
- description of the image
|
||||
|
||||
* - publishedDate
|
||||
- :py:class:`datetime <datetime.datetime>`
|
||||
- time of publish
|
||||
|
||||
* - img_src
|
||||
- :py:class:`str`
|
||||
- url to the result image
|
||||
|
||||
* - thumbnail_src
|
||||
- :py:class:`str`
|
||||
- url to a small-preview image
|
||||
|
||||
* - resolution
|
||||
- :py:class:`str`
|
||||
- the resolution of the image (e.g. ``1920 x 1080`` pixel)
|
||||
|
||||
* - img_format
|
||||
- :py:class:`str`
|
||||
- the format of the image (e.g. ``png``)
|
||||
|
||||
* - filesize
|
||||
- :py:class:`str`
|
||||
- size of bytes in :py:obj:`human readable <searx.humanize_bytes>` notation
|
||||
(e.g. ``MB`` for 1024 \* 1024 Bytes filesize).
|
||||
|
||||
|
||||
.. _template videos:
|
||||
|
||||
``videos``
|
||||
----------
|
||||
|
||||
.. table:: Parameter of the **videos** media type:
|
||||
:width: 100%
|
||||
|
||||
========================= =====================================================
|
||||
result-parameter information
|
||||
------------------------- -----------------------------------------------------
|
||||
template is set to ``videos.html``
|
||||
========================= =====================================================
|
||||
url string, url of the result
|
||||
title string, title of the result
|
||||
content *(not implemented yet)*
|
||||
publishedDate :py:class:`datetime.datetime`, time of publish
|
||||
thumbnail string, url to a small-preview image
|
||||
length :py:class:`datetime.timedelta`, duration of result
|
||||
views string, view count in humanized number format
|
||||
========================= =====================================================
|
||||
|
||||
|
||||
.. _template torrent:
|
||||
|
||||
``torrent``
|
||||
-----------
|
||||
|
||||
.. _magnetlink: https://en.wikipedia.org/wiki/Magnet_URI_scheme
|
||||
|
||||
.. table:: Parameter of the **torrent** media type:
|
||||
:width: 100%
|
||||
|
||||
========================= =====================================================
|
||||
result-parameter information
|
||||
------------------------- -----------------------------------------------------
|
||||
template is set to ``torrent.html``
|
||||
========================= =====================================================
|
||||
url string, url of the result
|
||||
title string, title of the result
|
||||
content string, general result-text
|
||||
publishedDate :py:class:`datetime.datetime`,
|
||||
time of publish *(not implemented yet)*
|
||||
seed int, number of seeder
|
||||
leech int, number of leecher
|
||||
filesize int, size of file in bytes
|
||||
files int, number of files
|
||||
magnetlink string, magnetlink_ of the result
|
||||
torrentfile string, torrentfile of the result
|
||||
========================= =====================================================
|
||||
|
||||
|
||||
.. _template map:
|
||||
|
||||
``map``
|
||||
-------
|
||||
|
||||
.. table:: Parameter of the **map** media type:
|
||||
:width: 100%
|
||||
|
||||
========================= =====================================================
|
||||
result-parameter information
|
||||
------------------------- -----------------------------------------------------
|
||||
template is set to ``map.html``
|
||||
========================= =====================================================
|
||||
url string, url of the result
|
||||
title string, title of the result
|
||||
content string, general result-text
|
||||
publishedDate :py:class:`datetime.datetime`, time of publish
|
||||
latitude latitude of result (in decimal format)
|
||||
longitude longitude of result (in decimal format)
|
||||
boundingbox boundingbox of result (array of 4. values
|
||||
``[lat-min, lat-max, lon-min, lon-max]``)
|
||||
geojson geojson of result (https://geojson.org/)
|
||||
osm.type type of osm-object (if OSM-Result)
|
||||
osm.id id of osm-object (if OSM-Result)
|
||||
address.name name of object
|
||||
address.road street name of object
|
||||
address.house_number house number of object
|
||||
address.locality city, place of object
|
||||
address.postcode postcode of object
|
||||
address.country country of object
|
||||
========================= =====================================================
|
||||
|
||||
|
||||
.. _template paper:
|
||||
|
||||
``paper``
|
||||
---------
|
||||
|
||||
.. _BibTeX format: https://www.bibtex.com/g/bibtex-format/
|
||||
.. _BibTeX field types: https://en.wikipedia.org/wiki/BibTeX#Field_types
|
||||
|
||||
.. list-table:: Parameter of the **paper** media type /
|
||||
see `BibTeX field types`_ and `BibTeX format`_
|
||||
:header-rows: 2
|
||||
:width: 100%
|
||||
|
||||
* - result-parameter
|
||||
- Python type
|
||||
- information
|
||||
|
||||
* - template
|
||||
- :py:class:`str`
|
||||
- is set to ``paper.html``
|
||||
|
||||
* - title
|
||||
- :py:class:`str`
|
||||
- title of the result
|
||||
|
||||
* - content
|
||||
- :py:class:`str`
|
||||
- abstract
|
||||
|
||||
* - comments
|
||||
- :py:class:`str`
|
||||
- free text display in italic below the content
|
||||
|
||||
* - tags
|
||||
- :py:class:`List <list>`\ [\ :py:class:`str`\ ]
|
||||
- free tag list
|
||||
|
||||
* - publishedDate
|
||||
- :py:class:`datetime <datetime.datetime>`
|
||||
- last publication date
|
||||
|
||||
* - type
|
||||
- :py:class:`str`
|
||||
- short description of medium type, e.g. *book*, *pdf* or *html* ...
|
||||
|
||||
* - authors
|
||||
- :py:class:`List <list>`\ [\ :py:class:`str`\ ]
|
||||
- list of authors of the work (authors with a "s")
|
||||
|
||||
* - editor
|
||||
- :py:class:`str`
|
||||
- list of editors of a book
|
||||
|
||||
* - publisher
|
||||
- :py:class:`str`
|
||||
- name of the publisher
|
||||
|
||||
* - journal
|
||||
- :py:class:`str`
|
||||
- name of the journal or magazine the article was
|
||||
published in
|
||||
|
||||
* - volume
|
||||
- :py:class:`str`
|
||||
- volume number
|
||||
|
||||
* - pages
|
||||
- :py:class:`str`
|
||||
- page range where the article is
|
||||
|
||||
* - number
|
||||
- :py:class:`str`
|
||||
- number of the report or the issue number for a journal article
|
||||
|
||||
* - doi
|
||||
- :py:class:`str`
|
||||
- DOI number (like ``10.1038/d41586-018-07848-2``)
|
||||
|
||||
* - issn
|
||||
- :py:class:`List <list>`\ [\ :py:class:`str`\ ]
|
||||
- ISSN number like ``1476-4687``
|
||||
|
||||
* - isbn
|
||||
- :py:class:`List <list>`\ [\ :py:class:`str`\ ]
|
||||
- ISBN number like ``9780201896831``
|
||||
|
||||
* - pdf_url
|
||||
- :py:class:`str`
|
||||
- URL to the full article, the PDF version
|
||||
|
||||
* - html_url
|
||||
- :py:class:`str`
|
||||
- URL to full article, HTML version
|
||||
|
||||
|
||||
.. _template packages:
|
||||
|
||||
``packages``
|
||||
------------
|
||||
|
||||
.. list-table:: Parameter of the **packages** media type
|
||||
:header-rows: 2
|
||||
:width: 100%
|
||||
|
||||
* - result-parameter
|
||||
- Python type
|
||||
- information
|
||||
|
||||
* - template
|
||||
- :py:class:`str`
|
||||
- is set to ``packages.html``
|
||||
|
||||
* - title
|
||||
- :py:class:`str`
|
||||
- title of the result
|
||||
|
||||
* - content
|
||||
- :py:class:`str`
|
||||
- abstract
|
||||
|
||||
* - package_name
|
||||
- :py:class:`str`
|
||||
- the name of the package
|
||||
|
||||
* - version
|
||||
- :py:class:`str`
|
||||
- the current version of the package
|
||||
|
||||
* - maintainer
|
||||
- :py:class:`str`
|
||||
- the maintainer or author of the project
|
||||
|
||||
* - publishedDate
|
||||
- :py:class:`datetime <datetime.datetime>`
|
||||
- date of latest update or release
|
||||
|
||||
* - tags
|
||||
- :py:class:`List <list>`\ [\ :py:class:`str`\ ]
|
||||
- free tag list
|
||||
|
||||
* - popularity
|
||||
- :py:class:`str`
|
||||
- the popularity of the package, e.g. rating or download count
|
||||
|
||||
* - license_name
|
||||
- :py:class:`str`
|
||||
- the name of the license
|
||||
|
||||
* - license_url
|
||||
- :py:class:`str`
|
||||
- the web location of a license copy
|
||||
|
||||
* - homepage
|
||||
- :py:class:`str`
|
||||
- the url of the project's homepage
|
||||
|
||||
* - source_code_url
|
||||
- :py:class:`str`
|
||||
- the location of the project's source code
|
||||
|
||||
* - links
|
||||
- :py:class:`dict`
|
||||
- additional links in the form of ``{'link_name': 'http://example.com'}``
|
||||
22
_sources/dev/engines/enginelib.rst.txt
Normal file
22
_sources/dev/engines/enginelib.rst.txt
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
.. _searx.enginelib:
|
||||
|
||||
==============
|
||||
Engine Library
|
||||
==============
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.enginelib
|
||||
:members:
|
||||
|
||||
.. _searx.enginelib.traits:
|
||||
|
||||
|
||||
Engine traits
|
||||
=============
|
||||
|
||||
.. automodule:: searx.enginelib.traits
|
||||
:members:
|
||||
9
_sources/dev/engines/engines.rst.txt
Normal file
9
_sources/dev/engines/engines.rst.txt
Normal file
|
|
@ -0,0 +1,9 @@
|
|||
.. _searx.engines loader:
|
||||
|
||||
========================
|
||||
SearXNG's engines loader
|
||||
========================
|
||||
|
||||
.. automodule:: searx.engines
|
||||
:members:
|
||||
|
||||
108
_sources/dev/engines/index.rst.txt
Normal file
108
_sources/dev/engines/index.rst.txt
Normal file
|
|
@ -0,0 +1,108 @@
|
|||
.. _engine implementations:
|
||||
|
||||
======================
|
||||
Engine Implementations
|
||||
======================
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
|
||||
.. toctree::
|
||||
:caption: Framework Components
|
||||
:maxdepth: 2
|
||||
|
||||
enginelib
|
||||
engines
|
||||
engine_overview
|
||||
|
||||
|
||||
Engine Types
|
||||
============
|
||||
|
||||
The :py:obj:`engine_type <searx.enginelib.Engine.engine_type>` of an engine
|
||||
determines which :ref:`search processor <searx.search.processors>` is used by
|
||||
the engine.
|
||||
|
||||
In this section a list of the engines that are documented is given, a complete
|
||||
list of the engines can be found in the source under: :origin:`searx/engines`.
|
||||
|
||||
.. _online engines:
|
||||
|
||||
Online Engines
|
||||
--------------
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :py:obj:`processors.online <searx.search.processors.online>`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:glob:
|
||||
|
||||
demo/demo_online
|
||||
xpath
|
||||
mediawiki
|
||||
json_engine
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:glob:
|
||||
|
||||
online/*
|
||||
|
||||
.. _offline engines:
|
||||
|
||||
Offline Engines
|
||||
---------------
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :py:obj:`processors.offline <searx.search.processors.offline>`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:glob:
|
||||
|
||||
offline_concept
|
||||
demo/demo_offline
|
||||
offline/*
|
||||
|
||||
.. _online url search:
|
||||
|
||||
Online URL Search
|
||||
-----------------
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :py:obj:`processors.online_url_search <searx.search.processors.online_url_search>`
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
:glob:
|
||||
|
||||
online_url_search/*
|
||||
|
||||
.. _online currency:
|
||||
|
||||
Online Currency
|
||||
---------------
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :py:obj:`processors.online_currency <searx.search.processors.online_currency>`
|
||||
|
||||
*no engine of this type is documented yet / coming soon*
|
||||
|
||||
.. _online dictionary:
|
||||
|
||||
Online Dictionary
|
||||
-----------------
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :py:obj:`processors.online_dictionary <searx.search.processors.online_dictionary>`
|
||||
|
||||
*no engine of this type is documented yet / coming soon*
|
||||
13
_sources/dev/engines/json_engine.rst.txt
Normal file
13
_sources/dev/engines/json_engine.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _json_engine engine:
|
||||
|
||||
============
|
||||
JSON Engine
|
||||
============
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.json_engine
|
||||
:members:
|
||||
13
_sources/dev/engines/mediawiki.rst.txt
Normal file
13
_sources/dev/engines/mediawiki.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _mediawiki engine:
|
||||
|
||||
================
|
||||
MediaWiki Engine
|
||||
================
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.mediawiki
|
||||
:members:
|
||||
23
_sources/dev/engines/offline/command-line-engines.rst.txt
Normal file
23
_sources/dev/engines/offline/command-line-engines.rst.txt
Normal file
|
|
@ -0,0 +1,23 @@
|
|||
.. _engine command:
|
||||
|
||||
====================
|
||||
Command Line Engines
|
||||
====================
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :origin:`command.py <searx/engines/command.py>`
|
||||
- :ref:`offline engines`
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
Initial sponsored by `Search and Discovery Fund
|
||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||
|
||||
.. automodule:: searx.engines.command
|
||||
:members:
|
||||
97
_sources/dev/engines/offline/nosql-engines.rst.txt
Normal file
97
_sources/dev/engines/offline/nosql-engines.rst.txt
Normal file
|
|
@ -0,0 +1,97 @@
|
|||
.. _nosql engines:
|
||||
|
||||
===============
|
||||
NoSQL databases
|
||||
===============
|
||||
|
||||
.. sidebar:: further read
|
||||
|
||||
- `NoSQL databases <https://en.wikipedia.org/wiki/NoSQL>`_
|
||||
- `redis.io <https://redis.io/>`_
|
||||
- `MongoDB <https://www.mongodb.com>`_
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
Initial sponsored by `Search and Discovery Fund
|
||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||
|
||||
The following `NoSQL databases`_ are supported:
|
||||
|
||||
- :ref:`engine redis_server`
|
||||
- :ref:`engine mongodb`
|
||||
|
||||
All of the engines above are just commented out in the :origin:`settings.yml
|
||||
<searx/settings.yml>`, as you have to set various options and install
|
||||
dependencies before using them.
|
||||
|
||||
By default, the engines use the ``key-value`` template for displaying results /
|
||||
see :origin:`simple <searx/templates/simple/result_templates/key-value.html>`
|
||||
theme. If you are not satisfied with the original result layout, you can use
|
||||
your own template, set ``result_template`` attribute to ``{template_name}`` and
|
||||
place the templates at::
|
||||
|
||||
searx/templates/{theme_name}/result_templates/{template_name}
|
||||
|
||||
Furthermore, if you do not wish to expose these engines on a public instance, you
|
||||
can still add them and limit the access by setting ``tokens`` as described in
|
||||
section :ref:`private engines`.
|
||||
|
||||
|
||||
Extra Dependencies
|
||||
==================
|
||||
|
||||
For using :ref:`engine redis_server` or :ref:`engine mongodb` you need to
|
||||
install additional packages in Python's Virtual Environment of your SearXNG
|
||||
instance. To switch into the environment (:ref:`searxng-src`) you can use
|
||||
:ref:`searxng.sh`::
|
||||
|
||||
$ sudo utils/searxng.sh instance cmd bash
|
||||
(searxng-pyenv)$ pip install ...
|
||||
|
||||
|
||||
Configure the engines
|
||||
=====================
|
||||
|
||||
`NoSQL databases`_ are used for storing arbitrary data without first defining
|
||||
their structure.
|
||||
|
||||
|
||||
.. _engine redis_server:
|
||||
|
||||
Redis Server
|
||||
------------
|
||||
|
||||
.. _redis: https://github.com/andymccurdy/redis-py#installation
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- ``pip install`` redis_
|
||||
- redis.io_
|
||||
- :origin:`redis_server.py <searx/engines/redis_server.py>`
|
||||
|
||||
.. automodule:: searx.engines.redis_server
|
||||
:members:
|
||||
|
||||
|
||||
.. _engine mongodb:
|
||||
|
||||
MongoDB
|
||||
-------
|
||||
|
||||
.. _pymongo: https://github.com/mongodb/mongo-python-driver#installation
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- ``pip install`` pymongo_
|
||||
- MongoDB_
|
||||
- :origin:`mongodb.py <searx/engines/mongodb.py>`
|
||||
|
||||
|
||||
.. automodule:: searx.engines.mongodb
|
||||
:members:
|
||||
|
||||
62
_sources/dev/engines/offline/search-indexer-engines.rst.txt
Normal file
62
_sources/dev/engines/offline/search-indexer-engines.rst.txt
Normal file
|
|
@ -0,0 +1,62 @@
|
|||
=================
|
||||
Local Search APIs
|
||||
=================
|
||||
|
||||
.. sidebar:: further read
|
||||
|
||||
- `Comparison to alternatives
|
||||
<https://docs.meilisearch.com/learn/what_is_meilisearch/comparison_to_alternatives.html>`_
|
||||
|
||||
.. contents::
|
||||
:depth: 1
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
Initial sponsored by `Search and Discovery Fund
|
||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||
|
||||
Administrators might find themselves wanting to integrate locally running search
|
||||
engines. The following ones are supported for now:
|
||||
|
||||
* `Elasticsearch`_
|
||||
* `Meilisearch`_
|
||||
* `Solr`_
|
||||
|
||||
Each search engine is powerful, capable of full-text search. All of the engines
|
||||
above are added to ``settings.yml`` just commented out, as you have to
|
||||
``base_url`` for all them.
|
||||
|
||||
Please note that if you are not using HTTPS to access these engines, you have to
|
||||
enable HTTP requests by setting ``enable_http`` to ``True``.
|
||||
|
||||
Furthermore, if you do not want to expose these engines on a public instance,
|
||||
you can still add them and limit the access by setting ``tokens`` as described
|
||||
in section :ref:`private engines`.
|
||||
|
||||
.. _engine meilisearch:
|
||||
|
||||
MeiliSearch
|
||||
===========
|
||||
|
||||
.. automodule:: searx.engines.meilisearch
|
||||
:members:
|
||||
|
||||
|
||||
.. _engine elasticsearch:
|
||||
|
||||
Elasticsearch
|
||||
=============
|
||||
|
||||
.. automodule:: searx.engines.elasticsearch
|
||||
:members:
|
||||
|
||||
.. _engine solr:
|
||||
|
||||
Solr
|
||||
====
|
||||
|
||||
.. automodule:: searx.engines.solr
|
||||
:members:
|
||||
|
||||
134
_sources/dev/engines/offline/sql-engines.rst.txt
Normal file
134
_sources/dev/engines/offline/sql-engines.rst.txt
Normal file
|
|
@ -0,0 +1,134 @@
|
|||
.. _sql engines:
|
||||
|
||||
===========
|
||||
SQL Engines
|
||||
===========
|
||||
|
||||
.. sidebar:: further read
|
||||
|
||||
- `SQLite <https://www.sqlite.org/index.html>`_
|
||||
- `PostgreSQL <https://www.postgresql.org>`_
|
||||
- `MySQL <https://www.mysql.com>`_
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
Initial sponsored by `Search and Discovery Fund
|
||||
<https://nlnet.nl/discovery>`_ of `NLnet Foundation <https://nlnet.nl/>`_.
|
||||
|
||||
With the *SQL engines* you can bind SQL databases into SearXNG. The following
|
||||
Relational Database Management System (RDBMS) are supported:
|
||||
|
||||
- :ref:`engine sqlite`
|
||||
- :ref:`engine postgresql`
|
||||
- :ref:`engine mysql_server` & :ref:`engine mariadb_server`
|
||||
|
||||
All of the engines above are just commented out in the :origin:`settings.yml
|
||||
<searx/settings.yml>`, as you have to set the required attributes for the
|
||||
engines, e.g. ``database:`` ...
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
- name: ...
|
||||
engine: {sqlite|postgresql|mysql_server}
|
||||
database: ...
|
||||
result_template: {template_name}
|
||||
query_str: ...
|
||||
|
||||
By default, the engines use the ``key-value`` template for displaying results /
|
||||
see :origin:`simple <searx/templates/simple/result_templates/key-value.html>`
|
||||
theme. If you are not satisfied with the original result layout, you can use
|
||||
your own template, set ``result_template`` attribute to ``{template_name}`` and
|
||||
place the templates at::
|
||||
|
||||
searx/templates/{theme_name}/result_templates/{template_name}
|
||||
|
||||
If you do not wish to expose these engines on a public instance, you can still
|
||||
add them and limit the access by setting ``tokens`` as described in section
|
||||
:ref:`private engines`.
|
||||
|
||||
|
||||
Extra Dependencies
|
||||
==================
|
||||
|
||||
For using :ref:`engine postgresql` or :ref:`engine mysql_server` you need to
|
||||
install additional packages in Python's Virtual Environment of your SearXNG
|
||||
instance. To switch into the environment (:ref:`searxng-src`) you can use
|
||||
:ref:`searxng.sh`::
|
||||
|
||||
$ sudo utils/searxng.sh instance cmd bash
|
||||
(searxng-pyenv)$ pip install ...
|
||||
|
||||
|
||||
Configure the engines
|
||||
=====================
|
||||
|
||||
The configuration of the new database engines are similar. You must put a valid
|
||||
SQL-SELECT query in ``query_str``. At the moment you can only bind at most one
|
||||
parameter in your query. By setting the attribute ``limit`` you can define how
|
||||
many results you want from the SQL server. Basically, it is the same as the
|
||||
``LIMIT`` keyword in SQL.
|
||||
|
||||
Please, do not include ``LIMIT`` or ``OFFSET`` in your SQL query as the engines
|
||||
rely on these keywords during paging. If you want to configure the number of
|
||||
returned results use the option ``limit``.
|
||||
|
||||
.. _engine sqlite:
|
||||
|
||||
SQLite
|
||||
------
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :origin:`sqlite.py <searx/engines/sqlite.py>`
|
||||
|
||||
.. automodule:: searx.engines.sqlite
|
||||
:members:
|
||||
|
||||
|
||||
.. _engine postgresql:
|
||||
|
||||
PostgreSQL
|
||||
----------
|
||||
|
||||
.. _psycopg2: https://www.psycopg.org/install
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :origin:`postgresql.py <searx/engines/postgresql.py>`
|
||||
- ``pip install`` `psycopg2-binary <psycopg2>`_
|
||||
|
||||
.. automodule:: searx.engines.postgresql
|
||||
:members:
|
||||
|
||||
.. _engine mysql_server:
|
||||
|
||||
MySQL
|
||||
-----
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :origin:`mysql_server.py <searx/engines/mysql_server.py>`
|
||||
- ``pip install`` :pypi:`mysql-connector-python <mysql-connector-python>`
|
||||
|
||||
|
||||
.. automodule:: searx.engines.mysql_server
|
||||
:members:
|
||||
|
||||
.. _engine mariadb_server:
|
||||
|
||||
MariaDB
|
||||
--------
|
||||
|
||||
.. sidebar:: info
|
||||
|
||||
- :origin:`mariadb_server.py <searx/engines/mariadb_server.py>`
|
||||
- ``pip install`` :pypi:`mariadb <mariadb>`
|
||||
|
||||
|
||||
.. automodule:: searx.engines.mariadb_server
|
||||
:members:
|
||||
69
_sources/dev/engines/offline_concept.rst.txt
Normal file
69
_sources/dev/engines/offline_concept.rst.txt
Normal file
|
|
@ -0,0 +1,69 @@
|
|||
===============
|
||||
Offline Concept
|
||||
===============
|
||||
|
||||
.. sidebar:: offline engines
|
||||
|
||||
- :ref:`demo offline engine`
|
||||
- :ref:`engine command`
|
||||
- :ref:`sql engines`
|
||||
- :ref:`nosql engines`
|
||||
- :py:obj:`searx.search.processors.offline`
|
||||
|
||||
To extend the functionality of SearXNG, offline engines are going to be
|
||||
introduced. An offline engine is an engine which does not need Internet
|
||||
connection to perform a search and does not use HTTP to communicate.
|
||||
|
||||
Offline engines can be configured, by adding those to the `engines` list of
|
||||
:origin:`settings.yml <searx/settings.yml>`. An example skeleton for offline
|
||||
engines can be found in :ref:`demo offline engine` (:origin:`demo_offline.py
|
||||
<searx/engines/demo_offline.py>`).
|
||||
|
||||
|
||||
Programming Interface
|
||||
=====================
|
||||
|
||||
:py:func:`init(engine_settings=None) <searx.engines.demo_offline.init>`
|
||||
All offline engines can have their own init function to setup the engine before
|
||||
accepting requests. The function gets the settings from settings.yml as a
|
||||
parameter. This function can be omitted, if there is no need to setup anything
|
||||
in advance.
|
||||
|
||||
:py:func:`search(query, params) <searx.engines.demo_offline.searc>`
|
||||
Each offline engine has a function named ``search``. This function is
|
||||
responsible to perform a search and return the results in a presentable
|
||||
format. (Where *presentable* means presentable by the selected result
|
||||
template.)
|
||||
|
||||
The return value is a list of results retrieved by the engine.
|
||||
|
||||
Engine representation in ``/config``
|
||||
If an engine is offline, the attribute ``offline`` is set to ``True``.
|
||||
|
||||
.. _offline requirements:
|
||||
|
||||
Extra Dependencies
|
||||
==================
|
||||
|
||||
If an offline engine depends on an external tool, SearXNG does not install it by
|
||||
default. When an administrator configures such engine and starts the instance,
|
||||
the process returns an error with the list of missing dependencies. Also,
|
||||
required dependencies will be added to the comment/description of the engine, so
|
||||
admins can install packages in advance.
|
||||
|
||||
If there is a need to install additional packages in *Python's Virtual
|
||||
Environment* of your SearXNG instance you need to switch into the environment
|
||||
(:ref:`searxng-src`) first, for this you can use :ref:`searxng.sh`::
|
||||
|
||||
$ sudo utils/searxng.sh instance cmd bash
|
||||
(searxng-pyenv)$ pip install ...
|
||||
|
||||
|
||||
Private engines (Security)
|
||||
==========================
|
||||
|
||||
To limit the access to offline engines, if an instance is available publicly,
|
||||
administrators can set token(s) for each of the :ref:`private engines`. If a
|
||||
query contains a valid token, then SearXNG performs the requested private
|
||||
search. If not, requests from an offline engines return errors.
|
||||
|
||||
13
_sources/dev/engines/online/adobe_stock.rst.txt
Normal file
13
_sources/dev/engines/online/adobe_stock.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _adobe stock engine:
|
||||
|
||||
===========
|
||||
Adobe Stock
|
||||
===========
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.adobe_stock
|
||||
:members:
|
||||
13
_sources/dev/engines/online/alpinelinux.rst.txt
Normal file
13
_sources/dev/engines/online/alpinelinux.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _alpinelinux engine:
|
||||
|
||||
=====================
|
||||
Alpine Linux Packages
|
||||
=====================
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.alpinelinux
|
||||
:members:
|
||||
13
_sources/dev/engines/online/annas_archive.rst.txt
Normal file
13
_sources/dev/engines/online/annas_archive.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _annas_archive engine:
|
||||
|
||||
==============
|
||||
Anna's Archive
|
||||
==============
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.annas_archive
|
||||
:members:
|
||||
14
_sources/dev/engines/online/archlinux.rst.txt
Normal file
14
_sources/dev/engines/online/archlinux.rst.txt
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
.. _archlinux engine:
|
||||
|
||||
==========
|
||||
Arch Linux
|
||||
==========
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.archlinux
|
||||
:members:
|
||||
|
||||
43
_sources/dev/engines/online/bing.rst.txt
Normal file
43
_sources/dev/engines/online/bing.rst.txt
Normal file
|
|
@ -0,0 +1,43 @@
|
|||
.. _bing engines:
|
||||
|
||||
============
|
||||
Bing Engines
|
||||
============
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
|
||||
.. _bing web engine:
|
||||
|
||||
Bing WEB
|
||||
========
|
||||
|
||||
.. automodule:: searx.engines.bing
|
||||
:members:
|
||||
|
||||
.. _bing images engine:
|
||||
|
||||
Bing Images
|
||||
===========
|
||||
|
||||
.. automodule:: searx.engines.bing_images
|
||||
:members:
|
||||
|
||||
.. _bing videos engine:
|
||||
|
||||
Bing Videos
|
||||
===========
|
||||
|
||||
.. automodule:: searx.engines.bing_videos
|
||||
:members:
|
||||
|
||||
.. _bing news engine:
|
||||
|
||||
Bing News
|
||||
=========
|
||||
|
||||
.. automodule:: searx.engines.bing_news
|
||||
:members:
|
||||
13
_sources/dev/engines/online/bpb.rst.txt
Normal file
13
_sources/dev/engines/online/bpb.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _bpb engine:
|
||||
|
||||
===
|
||||
Bpb
|
||||
===
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.bpb
|
||||
:members:
|
||||
13
_sources/dev/engines/online/brave.rst.txt
Normal file
13
_sources/dev/engines/online/brave.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _brave engine:
|
||||
|
||||
=============
|
||||
Brave Engines
|
||||
=============
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.brave
|
||||
:members:
|
||||
14
_sources/dev/engines/online/bt4g.rst.txt
Normal file
14
_sources/dev/engines/online/bt4g.rst.txt
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
.. _bt4g engine:
|
||||
|
||||
====
|
||||
BT4G
|
||||
====
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.bt4g
|
||||
:members:
|
||||
|
||||
13
_sources/dev/engines/online/dailymotion.rst.txt
Normal file
13
_sources/dev/engines/online/dailymotion.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _dailymotion engine:
|
||||
|
||||
===========
|
||||
Dailymotion
|
||||
===========
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.dailymotion
|
||||
:members:
|
||||
8
_sources/dev/engines/online/discourse.rst.txt
Normal file
8
_sources/dev/engines/online/discourse.rst.txt
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
.. _discourse engine:
|
||||
|
||||
================
|
||||
Discourse Forums
|
||||
================
|
||||
|
||||
.. automodule:: searx.engines.discourse
|
||||
:members:
|
||||
22
_sources/dev/engines/online/duckduckgo.rst.txt
Normal file
22
_sources/dev/engines/online/duckduckgo.rst.txt
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
.. _duckduckgo engines:
|
||||
|
||||
==================
|
||||
DuckDuckGo Engines
|
||||
==================
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.duckduckgo
|
||||
:members:
|
||||
|
||||
.. automodule:: searx.engines.duckduckgo_extra
|
||||
:members:
|
||||
|
||||
.. automodule:: searx.engines.duckduckgo_definitions
|
||||
:members:
|
||||
|
||||
.. automodule:: searx.engines.duckduckgo_weather
|
||||
:members:
|
||||
8
_sources/dev/engines/online/geizhals.rst.txt
Normal file
8
_sources/dev/engines/online/geizhals.rst.txt
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
.. _gitea geizhals:
|
||||
|
||||
========
|
||||
Geizhals
|
||||
========
|
||||
|
||||
.. automodule:: searx.engines.geizhals
|
||||
:members:
|
||||
8
_sources/dev/engines/online/gitea.rst.txt
Normal file
8
_sources/dev/engines/online/gitea.rst.txt
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
.. _gitea engine:
|
||||
|
||||
=====
|
||||
Gitea
|
||||
=====
|
||||
|
||||
.. automodule:: searx.engines.gitea
|
||||
:members:
|
||||
8
_sources/dev/engines/online/gitlab.rst.txt
Normal file
8
_sources/dev/engines/online/gitlab.rst.txt
Normal file
|
|
@ -0,0 +1,8 @@
|
|||
.. _gitlab engine:
|
||||
|
||||
======
|
||||
GitLab
|
||||
======
|
||||
|
||||
.. automodule:: searx.engines.gitlab
|
||||
:members:
|
||||
76
_sources/dev/engines/online/google.rst.txt
Normal file
76
_sources/dev/engines/online/google.rst.txt
Normal file
|
|
@ -0,0 +1,76 @@
|
|||
.. _google engines:
|
||||
|
||||
==============
|
||||
Google Engines
|
||||
==============
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
|
||||
.. _google API:
|
||||
|
||||
Google API
|
||||
==========
|
||||
|
||||
.. _Query Parameter Definitions:
|
||||
https://developers.google.com/custom-search/docs/xml_results#WebSearch_Query_Parameter_Definitions
|
||||
|
||||
SearXNG's implementation of the Google API is mainly done in
|
||||
:py:obj:`get_google_info <searx.engines.google.get_google_info>`.
|
||||
|
||||
For detailed description of the *REST-full* API see: `Query Parameter
|
||||
Definitions`_. The linked API documentation can sometimes be helpful during
|
||||
reverse engineering. However, we cannot use it in the freely accessible WEB
|
||||
services; not all parameters can be applied and some engines are more *special*
|
||||
than other (e.g. :ref:`google news engine`).
|
||||
|
||||
|
||||
.. _google web engine:
|
||||
|
||||
Google WEB
|
||||
==========
|
||||
|
||||
.. automodule:: searx.engines.google
|
||||
:members:
|
||||
|
||||
.. _google autocomplete:
|
||||
|
||||
Google Autocomplete
|
||||
====================
|
||||
|
||||
.. autofunction:: searx.autocomplete.google_complete
|
||||
|
||||
.. _google images engine:
|
||||
|
||||
Google Images
|
||||
=============
|
||||
|
||||
.. automodule:: searx.engines.google_images
|
||||
:members:
|
||||
|
||||
.. _google videos engine:
|
||||
|
||||
Google Videos
|
||||
=============
|
||||
|
||||
.. automodule:: searx.engines.google_videos
|
||||
:members:
|
||||
|
||||
.. _google news engine:
|
||||
|
||||
Google News
|
||||
===========
|
||||
|
||||
.. automodule:: searx.engines.google_news
|
||||
:members:
|
||||
|
||||
.. _google scholar engine:
|
||||
|
||||
Google Scholar
|
||||
==============
|
||||
|
||||
.. automodule:: searx.engines.google_scholar
|
||||
:members:
|
||||
13
_sources/dev/engines/online/lemmy.rst.txt
Normal file
13
_sources/dev/engines/online/lemmy.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _lemmy engine:
|
||||
|
||||
=====
|
||||
Lemmy
|
||||
=====
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.lemmy
|
||||
:members:
|
||||
13
_sources/dev/engines/online/loc.rst.txt
Normal file
13
_sources/dev/engines/online/loc.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _loc engine:
|
||||
|
||||
===================
|
||||
Library of Congress
|
||||
===================
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.loc
|
||||
:members:
|
||||
13
_sources/dev/engines/online/mastodon.rst.txt
Normal file
13
_sources/dev/engines/online/mastodon.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _mastodon engine:
|
||||
|
||||
========
|
||||
Mastodon
|
||||
========
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.mastodon
|
||||
:members:
|
||||
13
_sources/dev/engines/online/moviepilot.rst.txt
Normal file
13
_sources/dev/engines/online/moviepilot.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _moviepilot engine:
|
||||
|
||||
==========
|
||||
Moviepilot
|
||||
==========
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.moviepilot
|
||||
:members:
|
||||
13
_sources/dev/engines/online/mrs.rst.txt
Normal file
13
_sources/dev/engines/online/mrs.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _mrs engine:
|
||||
|
||||
=========================
|
||||
Matrix Rooms Search (MRS)
|
||||
=========================
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.mrs
|
||||
:members:
|
||||
13
_sources/dev/engines/online/mullvad_leta.rst.txt
Normal file
13
_sources/dev/engines/online/mullvad_leta.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _voidlinux mullvad_leta:
|
||||
|
||||
============
|
||||
Mullvad-Leta
|
||||
============
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.mullvad_leta
|
||||
:members:
|
||||
27
_sources/dev/engines/online/mwmbl.rst.txt
Normal file
27
_sources/dev/engines/online/mwmbl.rst.txt
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
.. _Mwmbl engine:
|
||||
|
||||
============
|
||||
Mwmbl Engine
|
||||
============
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
|
||||
.. _mwmbl web engine:
|
||||
|
||||
Mwmbl WEB
|
||||
=========
|
||||
|
||||
.. automodule:: searx.engines.mwmbl
|
||||
:members:
|
||||
|
||||
|
||||
.. _mwmbl autocomplete:
|
||||
|
||||
Mwmbl Autocomplete
|
||||
==================
|
||||
|
||||
.. autofunction:: searx.autocomplete.mwmbl
|
||||
13
_sources/dev/engines/online/odysee.rst.txt
Normal file
13
_sources/dev/engines/online/odysee.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _odysee engine:
|
||||
|
||||
======
|
||||
Odysee
|
||||
======
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.odysee
|
||||
:members:
|
||||
27
_sources/dev/engines/online/peertube.rst.txt
Normal file
27
_sources/dev/engines/online/peertube.rst.txt
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
.. _peertube engines:
|
||||
|
||||
================
|
||||
Peertube Engines
|
||||
================
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
|
||||
.. _peertube video engine:
|
||||
|
||||
Peertube Video
|
||||
==============
|
||||
|
||||
.. automodule:: searx.engines.peertube
|
||||
:members:
|
||||
|
||||
.. _sepiasearch engine:
|
||||
|
||||
SepiaSearch
|
||||
===========
|
||||
|
||||
.. automodule:: searx.engines.sepiasearch
|
||||
:members:
|
||||
13
_sources/dev/engines/online/piped.rst.txt
Normal file
13
_sources/dev/engines/online/piped.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _piped engine:
|
||||
|
||||
=====
|
||||
Piped
|
||||
=====
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.piped
|
||||
:members:
|
||||
13
_sources/dev/engines/online/presearch.rst.txt
Normal file
13
_sources/dev/engines/online/presearch.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _engine presearch:
|
||||
|
||||
================
|
||||
Presearch Engine
|
||||
================
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.presearch
|
||||
:members:
|
||||
13
_sources/dev/engines/online/qwant.rst.txt
Normal file
13
_sources/dev/engines/online/qwant.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _qwant engine:
|
||||
|
||||
=====
|
||||
Qwant
|
||||
=====
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.qwant
|
||||
:members:
|
||||
13
_sources/dev/engines/online/radio_browser.rst.txt
Normal file
13
_sources/dev/engines/online/radio_browser.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _RadioBrowser engine:
|
||||
|
||||
============
|
||||
RadioBrowser
|
||||
============
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.radio_browser
|
||||
:members:
|
||||
13
_sources/dev/engines/online/recoll.rst.txt
Normal file
13
_sources/dev/engines/online/recoll.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _engine recoll:
|
||||
|
||||
=============
|
||||
Recoll Engine
|
||||
=============
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.recoll
|
||||
:members:
|
||||
13
_sources/dev/engines/online/seekr.rst.txt
Normal file
13
_sources/dev/engines/online/seekr.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _seekr engine:
|
||||
|
||||
=============
|
||||
Seekr Engines
|
||||
=============
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.seekr
|
||||
:members:
|
||||
13
_sources/dev/engines/online/startpage.rst.txt
Normal file
13
_sources/dev/engines/online/startpage.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _startpage engines:
|
||||
|
||||
=================
|
||||
Startpage Engines
|
||||
=================
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.startpage
|
||||
:members:
|
||||
13
_sources/dev/engines/online/tagesschau.rst.txt
Normal file
13
_sources/dev/engines/online/tagesschau.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _tagesschau engine:
|
||||
|
||||
==============
|
||||
Tagesschau API
|
||||
==============
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.tagesschau
|
||||
:members:
|
||||
13
_sources/dev/engines/online/torznab.rst.txt
Normal file
13
_sources/dev/engines/online/torznab.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _torznab engine:
|
||||
|
||||
==============
|
||||
Torznab WebAPI
|
||||
==============
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.torznab
|
||||
:members:
|
||||
13
_sources/dev/engines/online/void.rst.txt
Normal file
13
_sources/dev/engines/online/void.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _voidlinux engine:
|
||||
|
||||
==========================
|
||||
Void Linux binary packages
|
||||
==========================
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.voidlinux
|
||||
:members:
|
||||
13
_sources/dev/engines/online/wallhaven.rst.txt
Normal file
13
_sources/dev/engines/online/wallhaven.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _wallhaven engine:
|
||||
|
||||
=========
|
||||
Wallhaven
|
||||
=========
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.wallhaven
|
||||
:members:
|
||||
27
_sources/dev/engines/online/wikipedia.rst.txt
Normal file
27
_sources/dev/engines/online/wikipedia.rst.txt
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
.. _wikimedia engines:
|
||||
|
||||
=========
|
||||
Wikimedia
|
||||
=========
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
|
||||
.. _wikipedia engine:
|
||||
|
||||
Wikipedia
|
||||
=========
|
||||
|
||||
.. automodule:: searx.engines.wikipedia
|
||||
:members:
|
||||
|
||||
.. _wikidata engine:
|
||||
|
||||
Wikidata
|
||||
=========
|
||||
|
||||
.. automodule:: searx.engines.wikidata
|
||||
:members:
|
||||
13
_sources/dev/engines/online/yacy.rst.txt
Normal file
13
_sources/dev/engines/online/yacy.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _yacy engine:
|
||||
|
||||
====
|
||||
Yacy
|
||||
====
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.yacy
|
||||
:members:
|
||||
13
_sources/dev/engines/online/yahoo.rst.txt
Normal file
13
_sources/dev/engines/online/yahoo.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _yahoo engine:
|
||||
|
||||
============
|
||||
Yahoo Engine
|
||||
============
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.yahoo
|
||||
:members:
|
||||
13
_sources/dev/engines/online/zlibrary.rst.txt
Normal file
13
_sources/dev/engines/online/zlibrary.rst.txt
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
.. _zlibrary engine:
|
||||
|
||||
=========
|
||||
Z-Library
|
||||
=========
|
||||
|
||||
.. contents:: Contents
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.zlibrary
|
||||
:members:
|
||||
14
_sources/dev/engines/online_url_search/tineye.rst.txt
Normal file
14
_sources/dev/engines/online_url_search/tineye.rst.txt
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
.. _tineye engine:
|
||||
|
||||
======
|
||||
Tineye
|
||||
======
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.tineye
|
||||
:members:
|
||||
|
||||
14
_sources/dev/engines/xpath.rst.txt
Normal file
14
_sources/dev/engines/xpath.rst.txt
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
.. _xpath engine:
|
||||
|
||||
============
|
||||
XPath Engine
|
||||
============
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
.. automodule:: searx.engines.xpath
|
||||
:members:
|
||||
|
||||
18
_sources/dev/index.rst.txt
Normal file
18
_sources/dev/index.rst.txt
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
=======================
|
||||
Developer documentation
|
||||
=======================
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
quickstart
|
||||
rtm_asdf
|
||||
contribution_guide
|
||||
engines/index
|
||||
search_api
|
||||
plugins
|
||||
translation
|
||||
lxcdev
|
||||
makefile
|
||||
reST
|
||||
searxng_extra/index
|
||||
438
_sources/dev/lxcdev.rst.txt
Normal file
438
_sources/dev/lxcdev.rst.txt
Normal file
|
|
@ -0,0 +1,438 @@
|
|||
.. _lxcdev:
|
||||
|
||||
==============================
|
||||
Developing in Linux Containers
|
||||
==============================
|
||||
|
||||
.. _LXC: https://linuxcontainers.org/lxc/introduction/
|
||||
|
||||
In this article we will show, how you can make use of Linux Containers (LXC_) in
|
||||
*distributed and heterogeneous development cycles* (TL;DR; jump to the
|
||||
:ref:`lxcdev summary`).
|
||||
|
||||
.. sidebar:: Audience
|
||||
|
||||
This blog post is written for experienced admins and developers. Readers
|
||||
should have a serious meaning about the terms: *distributed*, *merge* and
|
||||
*linux container*.
|
||||
|
||||
**hint**
|
||||
|
||||
If you have issues with the internet connectivity of your containers read
|
||||
section :ref:`internet connectivity docker`.
|
||||
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
|
||||
Motivation
|
||||
==========
|
||||
|
||||
Most often in our development cycle, we edit the sources and run some test
|
||||
and/or builds by using ``make`` :ref:`[ref] <makefile>` before we commit. This
|
||||
cycle is simple and perfect but might fail in some aspects we should not
|
||||
overlook.
|
||||
|
||||
**The environment in which we run all our development processes matters!**
|
||||
|
||||
The :ref:`makefile` and the :ref:`make install` encapsulate a lot for us, but
|
||||
these tools do not have access to all prerequisites. For example, there may
|
||||
have dependencies on packages that are installed on developer's desktop, but
|
||||
usually are not preinstalled on a server or client system. Another example is;
|
||||
settings have been made to the software on developer's desktop that would never
|
||||
be set on a *production* system.
|
||||
|
||||
**Linux Containers are isolate environments**, we use them to not mix up all
|
||||
the prerequisites from various projects on developer's desktop.
|
||||
|
||||
The scripts from :ref:`searx_utils` can divide in those to install and maintain
|
||||
software
|
||||
|
||||
- :ref:`searxng.sh`
|
||||
|
||||
and the script
|
||||
|
||||
- :ref:`lxc.sh`
|
||||
|
||||
with we can scale our installation, maintenance or even development tasks over a
|
||||
stack of isolated containers / what we call the:
|
||||
|
||||
- :ref:`searxng lxc suite`
|
||||
|
||||
.. _lxcdev install searxng:
|
||||
|
||||
Gentlemen, start your engines!
|
||||
==============================
|
||||
|
||||
.. _LXD: https://linuxcontainers.org/lxd/introduction/
|
||||
.. _archlinux: https://www.archlinux.org/
|
||||
|
||||
Before you can start with containers, you need to install and initiate LXD_
|
||||
once:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ snap install lxd
|
||||
$ lxd init --auto
|
||||
|
||||
And you need to clone from origin or if you have your own fork, clone from your
|
||||
fork:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ cd ~/Downloads
|
||||
$ git clone https://github.com/searxng/searxng.git searxng
|
||||
$ cd searxng
|
||||
|
||||
.. sidebar:: The ``searxng-archlinux`` container
|
||||
|
||||
is the base of all our exercises here.
|
||||
|
||||
The :ref:`lxc-searxng.env` consists of several images, see ``export
|
||||
LXC_SUITE=(...`` near by :origin:`utils/lxc-searxng.env#L19`.
|
||||
For this blog post we exercise on a archlinux_ image. The container of this
|
||||
image is named ``searxng-archlinux``.
|
||||
|
||||
Lets build the container, but be sure that this container does not already
|
||||
exists, so first lets remove possible old one:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh remove searxng-archlinux
|
||||
$ sudo -H ./utils/lxc.sh build searxng-archlinux
|
||||
|
||||
|
||||
.. sidebar:: further read
|
||||
|
||||
- :ref:`lxc.sh install suite`
|
||||
- :ref:`installation nginx`
|
||||
|
||||
To install the complete :ref:`SearXNG suite <searxng lxc suite>` and the HTTP
|
||||
proxy :ref:`installation nginx` into the archlinux container run:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh install suite searxng-archlinux
|
||||
$ sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
|
||||
$ sudo ./utils/lxc.sh show suite | grep SEARXNG_URL
|
||||
...
|
||||
[searxng-archlinux] SEARXNG_URL : http://n.n.n.140/searxng
|
||||
|
||||
.. sidebar:: Fully functional SearXNG suite
|
||||
|
||||
From here on you have a fully functional SearXNG suite (including a
|
||||
:ref:`redis db`).
|
||||
|
||||
In such a SearXNG suite admins can maintain and access the debug log of the
|
||||
services quite easy.
|
||||
|
||||
In the example above the SearXNG instance in the container is wrapped to
|
||||
``http://n.n.n.140/searxng`` to the HOST system. Note, on your HOST system, the
|
||||
IP of your ``searxng-archlinux`` container is different to this example. To
|
||||
test the instance in the container from outside of the container, in your WEB
|
||||
browser on your desktop just open the URL reported in your installation
|
||||
|
||||
.. _working in containers:
|
||||
|
||||
In containers, work as usual
|
||||
============================
|
||||
|
||||
Usually you open a root-bash using ``sudo -H bash``. In case of LXC containers
|
||||
open the root-bash in the container is done by the ``./utils/lxc.sh cmd
|
||||
searxng-archlinux`` command:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux bash
|
||||
INFO: [searxng-archlinux] bash
|
||||
[root@searxng-archlinux SearXNG]$
|
||||
|
||||
The prompt ``[root@searxng-archlinux ...]`` signals, that you are the root user
|
||||
in the container (GUEST). To debug the running SearXNG instance use:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ ./utils/searxng.sh instance inspect
|
||||
...
|
||||
use [CTRL-C] to stop monitoring the log
|
||||
...
|
||||
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh instance inspect
|
||||
...
|
||||
use [CTRL-C] to stop monitoring the log
|
||||
...
|
||||
|
||||
|
||||
Back in the browser on your desktop open the service http://n.n.n.140/searxng
|
||||
and run your application tests while the debug log is shown in the terminal from
|
||||
above. You can stop monitoring using ``CTRL-C``, this also disables the *"debug
|
||||
option"* in SearXNG's settings file and restarts the SearXNG uwsgi application.
|
||||
|
||||
Another point we have to notice is that the service :ref:`SearXNG <searxng.sh>`
|
||||
runs under dedicated system user account with the same name (compare
|
||||
:ref:`create searxng user`). To get a login shell from these accounts, simply
|
||||
call:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ ./utils/searxng.sh instance cmd bash -l
|
||||
(searx-pyenv) [searxng@searxng-archlinux ~]$ pwd
|
||||
/usr/local/searxng
|
||||
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh instance cmd bash -l
|
||||
INFO: [searxng-archlinux] ./utils/searxng.sh instance cmd bash -l
|
||||
(searx-pyenv) [searxng@searxng-archlinux ~]$ pwd
|
||||
/usr/local/searxng
|
||||
|
||||
The prompt ``[searxng@searxng-archlinux]`` signals that you are logged in as system
|
||||
user ``searxng`` in the ``searxng-archlinux`` container and the python *virtualenv*
|
||||
``(searxng-pyenv)`` environment is activated.
|
||||
|
||||
|
||||
Wrap production into developer suite
|
||||
====================================
|
||||
|
||||
In this section we will see how to change the *"Fully functional SearXNG suite"*
|
||||
from a LXC container (which is quite ready for production) into a developer
|
||||
suite. For this, we have to keep an eye on the :ref:`installation basic`:
|
||||
|
||||
- SearXNG setup in: ``/etc/searxng/settings.yml``
|
||||
- SearXNG user's home: ``/usr/local/searxng``
|
||||
- virtualenv in: ``/usr/local/searxng/searxng-pyenv``
|
||||
- SearXNG software in: ``/usr/local/searxng/searxng-src``
|
||||
|
||||
With the use of the :ref:`searxng.sh` the SearXNG service was installed as
|
||||
:ref:`uWSGI application <searxng uwsgi>`. To maintain this service, we can use
|
||||
``systemctl`` (compare :ref:`uWSGI maintenance`).
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: uwsgi@searxng
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux systemctl stop uwsgi@searxng
|
||||
|
||||
With the command above, we stopped the SearXNG uWSGI-App in the archlinux
|
||||
container.
|
||||
|
||||
The uWSGI-App for the archlinux distros is configured in
|
||||
:origin:`utils/templates/etc/uwsgi/apps-archlinux/searxng.ini`, from where at
|
||||
least you should attend the settings of ``uid``, ``chdir``, ``env`` and
|
||||
``http``::
|
||||
|
||||
env = SEARXNG_SETTINGS_PATH=/etc/searxng/settings.yml
|
||||
http = 127.0.0.1:8888
|
||||
|
||||
chdir = /usr/local/searxng/searxng-src/searx
|
||||
virtualenv = /usr/local/searxng/searxng-pyenv
|
||||
pythonpath = /usr/local/searxng/searxng-src
|
||||
|
||||
If you have read the :ref:`Good to know` you remember, that each container
|
||||
shares the root folder of the repository and the command ``utils/lxc.sh cmd``
|
||||
handles relative path names **transparent**.
|
||||
|
||||
To wrap the SearXNG installation in the container into a developer one, we
|
||||
simple have to create a symlink to the **transparent** repository from the
|
||||
desktop. Now lets replace the repository at ``searxng-src`` in the container
|
||||
with the working tree from outside of the container:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old
|
||||
$ ln -s /share/SearXNG/ /usr/local/searxng/searxng-src
|
||||
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
|
||||
mv /usr/local/searxng/searxng-src /usr/local/searxng/searxng-src.old
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux \
|
||||
ln -s /share/SearXNG/ /usr/local/searxng/searxng-src
|
||||
|
||||
Now we can develop as usual in the working tree of our desktop system. Every
|
||||
time the software was changed, you have to restart the SearXNG service (in the
|
||||
container):
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: uwsgi@searxng
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux systemctl restart uwsgi@searxng
|
||||
|
||||
|
||||
Remember: :ref:`working in containers` .. here are just some examples from my
|
||||
daily usage:
|
||||
|
||||
To *inspect* the SearXNG instance (already described above):
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ ./utils/searx.sh inspect service
|
||||
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searx.sh inspect service
|
||||
|
||||
Run :ref:`makefile`, e.g. to test inside the container:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ make test
|
||||
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux make test
|
||||
|
||||
|
||||
|
||||
To install all prerequisites needed for a :ref:`buildhosts`:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ ./utils/searxng.sh install buildhost
|
||||
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux ./utils/searxng.sh install buildhost
|
||||
|
||||
|
||||
To build the docs on a buildhost :ref:`buildhosts`:
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: ``[root@searxng-archlinux SearXNG]`` (GUEST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ make docs.html
|
||||
|
||||
.. group-tab:: desktop (HOST)
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux make docs.html
|
||||
|
||||
|
||||
.. _lxcdev summary:
|
||||
|
||||
Summary
|
||||
=======
|
||||
|
||||
We build up a fully functional SearXNG suite in a archlinux container:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh build searxng-archlinux
|
||||
$ sudo -H ./utils/lxc.sh install suite searxng-archlinux
|
||||
...
|
||||
Developer install? (wraps source from HOST into the running instance) [YES/no]
|
||||
|
||||
To wrap the suite into a developer one answer ``YES`` (or press Enter).
|
||||
|
||||
.. code:: text
|
||||
|
||||
link SearXNG's sources to: /share/SearXNG
|
||||
=========================================
|
||||
|
||||
mv -f "/usr/local/searxng/searxng-src" "/usr/local/searxng/searxng-src.backup"
|
||||
ln -s "/share/SearXNG" "/usr/local/searxng/searxng-src"
|
||||
ls -ld /usr/local/searxng/searxng-src
|
||||
|searxng| lrwxrwxrwx 1 searxng searxng ... /usr/local/searxng/searxng-src -> /share/SearXNG
|
||||
|
||||
On code modification the instance has to be restarted (see :ref:`uWSGI
|
||||
maintenance`):
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd searxng-archlinux systemctl restart uwsgi@searxng
|
||||
|
||||
To access HTTP from the desktop we installed nginx for the services inside the
|
||||
container:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd -- FORCE_TIMEOUT=0 ./utils/searxng.sh install nginx
|
||||
|
||||
To get information about the SearxNG suite in the archlinux container we can
|
||||
use:
|
||||
|
||||
.. code:: text
|
||||
|
||||
$ sudo -H ./utils/lxc.sh show suite searxng-archlinux
|
||||
[searxng-archlinux] INFO: (eth0) docs-live: http:///n.n.n.140:8080/
|
||||
[searxng-archlinux] INFO: (eth0) IPv6: http://[fd42:555b:2af9:e121:216:3eff:fe5b:1744]
|
||||
[searxng-archlinux] uWSGI:
|
||||
[searxng-archlinux] SEARXNG_UWSGI_SOCKET : /usr/local/searxng/run/socket
|
||||
[searxng-archlinux] environment /usr/local/searxng/searxng-src/utils/brand.env:
|
||||
[searxng-archlinux] GIT_URL : https://github.com/searxng/searxng
|
||||
[searxng-archlinux] GIT_BRANCH : master
|
||||
[searxng-archlinux] SEARXNG_URL : http:///n.n.n.140/searxng
|
||||
[searxng-archlinux] SEARXNG_PORT : 8888
|
||||
[searxng-archlinux] SEARXNG_BIND_ADDRESS : 127.0.0.1
|
||||
|
||||
383
_sources/dev/makefile.rst.txt
Normal file
383
_sources/dev/makefile.rst.txt
Normal file
|
|
@ -0,0 +1,383 @@
|
|||
.. _makefile:
|
||||
|
||||
=======================
|
||||
Makefile & ``./manage``
|
||||
=======================
|
||||
|
||||
.. _gnu-make: https://www.gnu.org/software/make/manual/make.html#Introduction
|
||||
|
||||
All relevant build and development tasks are implemented in the
|
||||
:origin:`./manage <manage>` script and for CI or IDE integration a small
|
||||
:origin:`Makefile` wrapper is available. If you are not familiar with
|
||||
Makefiles, we recommend to read gnu-make_ introduction.
|
||||
|
||||
.. sidebar:: build environment
|
||||
|
||||
Before looking deeper at the targets, first read about :ref:`make
|
||||
install`.
|
||||
|
||||
To install developer requirements follow :ref:`buildhosts`.
|
||||
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
The usage is simple, just type ``make {target-name}`` to *build* a target.
|
||||
Calling the ``help`` target gives a first overview (``make help``):
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: ``make``
|
||||
|
||||
.. program-output:: bash -c "cd ..; make --no-print-directory help"
|
||||
|
||||
|
||||
.. group-tab:: ``./manage``
|
||||
|
||||
The Makefile targets are implemented for comfort, if you can do without
|
||||
tab-completion and need to have a more granular control, use
|
||||
:origin:`manage` without the Makefile wrappers.
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ ./manage help
|
||||
|
||||
.. _make install:
|
||||
|
||||
Python environment (``make install``)
|
||||
=====================================
|
||||
|
||||
.. sidebar:: activate environment
|
||||
|
||||
``source ./local/py3/bin/activate``
|
||||
|
||||
We do no longer need to build up the virtualenv manually. Jump into your git
|
||||
working tree and release a ``make install`` to get a virtualenv with a
|
||||
*developer install* of SearXNG (:origin:`setup.py`). ::
|
||||
|
||||
$ cd ~/searxng-clone
|
||||
$ make install
|
||||
PYENV [virtualenv] installing ./requirements*.txt into local/py3
|
||||
...
|
||||
PYENV [install] pip install --use-pep517 --no-build-isolation -e 'searx[test]'
|
||||
...
|
||||
Successfully installed searxng-2023.7.19+a446dea1b
|
||||
|
||||
If you release ``make install`` multiple times the installation will only
|
||||
rebuild if the sha256 sum of the *requirement files* fails. With other words:
|
||||
the check fails if you edit the requirements listed in
|
||||
:origin:`requirements-dev.txt` and :origin:`requirements.txt`). ::
|
||||
|
||||
$ make install
|
||||
PYENV OK
|
||||
PYENV [virtualenv] requirements.sha256 failed
|
||||
[virtualenv] - 6cea6eb6def9e14a18bf32f8a3e... ./requirements-dev.txt
|
||||
[virtualenv] - 471efef6c73558e391c3adb35f4... ./requirements.txt
|
||||
...
|
||||
PYENV [virtualenv] installing ./requirements*.txt into local/py3
|
||||
...
|
||||
PYENV [install] pip install --use-pep517 --no-build-isolation -e 'searx[test]'
|
||||
...
|
||||
Successfully installed searxng-2023.7.19+a446dea1b
|
||||
|
||||
.. sidebar:: drop environment
|
||||
|
||||
To get rid of the existing environment before re-build use :ref:`clean target
|
||||
<make clean>` first.
|
||||
|
||||
If you think, something goes wrong with your ./local environment or you change
|
||||
the :origin:`setup.py` file, you have to call :ref:`make clean`.
|
||||
|
||||
.. _make node.env:
|
||||
|
||||
Node.js environment (``make node.env``)
|
||||
=======================================
|
||||
|
||||
.. _Node.js: https://nodejs.org/
|
||||
.. _nvm: https://github.com/nvm-sh
|
||||
.. _npm: https://www.npmjs.com/
|
||||
|
||||
.. jinja:: searx
|
||||
|
||||
Node.js_ version {{version.node}} or higher is required to build the themes.
|
||||
If the requirement is not met, the build chain uses nvm_ (Node Version
|
||||
Manager) to install latest LTS of Node.js_ locally: there is no need to
|
||||
install nvm_ or npm_ on your system.
|
||||
|
||||
To install NVM_ and Node.js_ in once you can use :ref:`make nvm.nodejs`.
|
||||
|
||||
.. _make nvm:
|
||||
|
||||
NVM ``make nvm.install nvm.status``
|
||||
-----------------------------------
|
||||
|
||||
Use ``make nvm.status`` to get the current status of your Node.js_ and nvm_
|
||||
setup.
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: nvm.install
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ LANG=C make nvm.install
|
||||
INFO: install (update) NVM at ./searxng/.nvm
|
||||
INFO: clone: https://github.com/nvm-sh/nvm.git
|
||||
|| Cloning into './searxng/.nvm'...
|
||||
INFO: checkout v0.39.4
|
||||
|| HEAD is now at 8fbf8ab v0.39.4
|
||||
|
||||
.. group-tab:: nvm.status (ubu2004)
|
||||
|
||||
Here is the output you will typically get on a Ubuntu 20.04 system which
|
||||
serves only a `no longer active <https://nodejs.org/en/about/releases/>`_
|
||||
Release `Node.js v10.19.0 <https://packages.ubuntu.com/focal/nodejs>`_.
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ make nvm.status
|
||||
INFO: Node.js is installed at /usr/bin/node
|
||||
INFO: Node.js is version v10.19.0
|
||||
WARN: minimal Node.js version is 16.13.0
|
||||
INFO: npm is installed at /usr/bin/npm
|
||||
INFO: npm is version 6.14.4
|
||||
WARN: NVM is not installed
|
||||
|
||||
.. _make nvm.nodejs:
|
||||
|
||||
``make nvm.nodejs``
|
||||
-------------------
|
||||
|
||||
Install latest Node.js_ LTS locally (uses nvm_)::
|
||||
|
||||
$ make nvm.nodejs
|
||||
INFO: install (update) NVM at /share/searxng/.nvm
|
||||
INFO: clone: https://github.com/nvm-sh/nvm.git
|
||||
...
|
||||
Downloading and installing node v16.13.0...
|
||||
...
|
||||
INFO: Node.js is installed at searxng/.nvm/versions/node/v16.13.0/bin/node
|
||||
INFO: Node.js is version v16.13.0
|
||||
INFO: npm is installed at searxng/.nvm/versions/node/v16.13.0/bin/npm
|
||||
INFO: npm is version 8.1.0
|
||||
INFO: NVM is installed at searxng/.nvm
|
||||
|
||||
.. _make run:
|
||||
|
||||
``make run``
|
||||
============
|
||||
|
||||
To get up a running a developer instance simply call ``make run``. This enables
|
||||
*debug* option in :origin:`searx/settings.yml`, starts a ``./searx/webapp.py``
|
||||
instance and opens the URL in your favorite WEB browser (:man:`xdg-open`)::
|
||||
|
||||
$ make run
|
||||
|
||||
Changes to theme's HTML templates (jinja2) are instant. Changes to the CSS & JS
|
||||
sources of the theme need to be rebuild. You can do that by running::
|
||||
|
||||
$ make themes.all
|
||||
|
||||
Alternatively to ``themes.all`` you can run *live builds* of the theme you are
|
||||
modify (:ref:`make themes`)::
|
||||
|
||||
$ LIVE_THEME=simple make run
|
||||
|
||||
.. _make format.python:
|
||||
|
||||
``make format.python``
|
||||
======================
|
||||
|
||||
Format Python source code using `Black code style`_. See ``$BLACK_OPTIONS``
|
||||
and ``$BLACK_TARGETS`` in :origin:`Makefile`.
|
||||
|
||||
.. attention::
|
||||
|
||||
We stuck at Black 22.12.0, please read comment in PR `Bump black from 22.12.0
|
||||
to 23.1.0`_
|
||||
|
||||
.. _Bump black from 22.12.0 to 23.1.0:
|
||||
https://github.com/searxng/searxng/pull/2159#pullrequestreview-1284094735
|
||||
|
||||
.. _Black code style:
|
||||
https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html
|
||||
|
||||
.. _make clean:
|
||||
|
||||
``make clean``
|
||||
==============
|
||||
|
||||
Drops all intermediate files, all builds, but keep sources untouched. Before
|
||||
calling ``make clean`` stop all processes using the :ref:`make install` or
|
||||
:ref:`make node.env`. ::
|
||||
|
||||
$ make clean
|
||||
CLEAN pyenv
|
||||
PYENV [virtualenv] drop local/py3
|
||||
CLEAN docs -- build/docs dist/docs
|
||||
CLEAN themes -- locally installed npm dependencies
|
||||
...
|
||||
CLEAN test stuff
|
||||
CLEAN common files
|
||||
|
||||
.. _make docs:
|
||||
|
||||
``make docs``
|
||||
=============
|
||||
|
||||
Target ``docs`` builds the documentation:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ make docs
|
||||
HTML ./docs --> file://
|
||||
DOCS build build/docs/includes
|
||||
...
|
||||
The HTML pages are in dist/docs.
|
||||
|
||||
.. _make docs.clean:
|
||||
|
||||
``make docs.clean docs.live``
|
||||
----------------------------------
|
||||
|
||||
We describe the usage of the ``doc.*`` targets in the :ref:`How to contribute /
|
||||
Documentation <contrib docs>` section. If you want to edit the documentation
|
||||
read our :ref:`make docs.live` section. If you are working in your own brand,
|
||||
adjust your :ref:`settings brand`.
|
||||
|
||||
|
||||
.. _make docs.gh-pages:
|
||||
|
||||
``make docs.gh-pages``
|
||||
----------------------
|
||||
|
||||
To deploy on github.io first adjust your :ref:`settings brand`. For any
|
||||
further read :ref:`deploy on github.io`.
|
||||
|
||||
.. _make test:
|
||||
|
||||
``make test``
|
||||
=============
|
||||
|
||||
Runs a series of tests: :ref:`make test.pylint`, ``test.pep8``, ``test.unit``
|
||||
and ``test.robot``. You can run tests selective, e.g.::
|
||||
|
||||
$ make test.pep8 test.unit test.shell
|
||||
TEST test.pep8 OK
|
||||
...
|
||||
TEST test.unit OK
|
||||
...
|
||||
TEST test.shell OK
|
||||
|
||||
.. _make test.shell:
|
||||
|
||||
``make test.shell``
|
||||
-------------------
|
||||
|
||||
:ref:`sh lint` / if you have changed some bash scripting run this test before
|
||||
commit.
|
||||
|
||||
.. _make test.pylint:
|
||||
|
||||
``make test.pylint``
|
||||
--------------------
|
||||
|
||||
.. _Pylint: https://www.pylint.org/
|
||||
|
||||
Pylint_ is known as one of the best source-code, bug and quality checker for the
|
||||
Python programming language. The pylint profile used in the SearXNG project is
|
||||
found in project's root folder :origin:`.pylintrc`.
|
||||
|
||||
.. _make search.checker:
|
||||
|
||||
``make search.checker.{engine name}``
|
||||
=====================================
|
||||
|
||||
To check all engines::
|
||||
|
||||
make search.checker
|
||||
|
||||
To check a engine with whitespace in the name like *google news* replace space
|
||||
by underline::
|
||||
|
||||
make search.checker.google_news
|
||||
|
||||
To see HTTP requests and more use SEARXNG_DEBUG::
|
||||
|
||||
make SEARXNG_DEBUG=1 search.checker.google_news
|
||||
|
||||
.. _3xx: https://en.wikipedia.org/wiki/List_of_HTTP_status_codes#3xx_redirection
|
||||
|
||||
To filter out HTTP redirects (3xx_)::
|
||||
|
||||
make SEARXNG_DEBUG=1 search.checker.google_news | grep -A1 "HTTP/1.1\" 3[0-9][0-9]"
|
||||
...
|
||||
Engine google news Checking
|
||||
https://news.google.com:443 "GET /search?q=life&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
|
||||
https://news.google.com:443 "GET /search?q=life&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
|
||||
--
|
||||
https://news.google.com:443 "GET /search?q=computer&hl=en&lr=lang_en&ie=utf8&oe=utf8&ceid=US%3Aen&gl=US HTTP/1.1" 302 0
|
||||
https://news.google.com:443 "GET /search?q=computer&hl=en-US&lr=lang_en&ie=utf8&oe=utf8&ceid=US:en&gl=US HTTP/1.1" 200 None
|
||||
--
|
||||
|
||||
.. _make themes:
|
||||
|
||||
``make themes.*``
|
||||
=================
|
||||
|
||||
.. sidebar:: further read
|
||||
|
||||
- :ref:`devquickstart`
|
||||
|
||||
The :origin:`Makefile` targets ``make theme.*`` cover common tasks to build the
|
||||
theme(s). The ``./manage themes.*`` command line can be used to convenient run
|
||||
common theme build tasks.
|
||||
|
||||
.. program-output:: bash -c "cd ..; ./manage themes.help"
|
||||
|
||||
To get live builds while modifying CSS & JS use (:ref:`make run`):
|
||||
|
||||
.. code:: sh
|
||||
|
||||
$ LIVE_THEME=simple make run
|
||||
|
||||
.. _make static.build:
|
||||
|
||||
``make static.build.*``
|
||||
=======================
|
||||
|
||||
.. sidebar:: further read
|
||||
|
||||
- :ref:`devquickstart`
|
||||
|
||||
The :origin:`Makefile` targets ``static.build.*`` cover common tasks to build (a
|
||||
commit of) the static files. The ``./manage static.build..*`` command line
|
||||
can be used to convenient run common build tasks of the static files.
|
||||
|
||||
.. program-output:: bash -c "cd ..; ./manage static.help"
|
||||
|
||||
|
||||
.. _manage redis.help:
|
||||
|
||||
``./manage redis.help``
|
||||
=======================
|
||||
|
||||
The ``./manage redis.*`` command line can be used to convenient run common Redis
|
||||
tasks (:ref:`Redis developer notes`).
|
||||
|
||||
.. program-output:: bash -c "cd ..; ./manage redis.help"
|
||||
|
||||
|
||||
.. _manage go.help:
|
||||
|
||||
``./manage go.help``
|
||||
====================
|
||||
|
||||
The ``./manage go.*`` command line can be used to convenient run common `go
|
||||
(wiki)`_ tasks.
|
||||
|
||||
.. _go (wiki): https://en.wikipedia.org/wiki/Go_(programming_language)
|
||||
|
||||
.. program-output:: bash -c "cd ..; ./manage go.help"
|
||||
106
_sources/dev/plugins.rst.txt
Normal file
106
_sources/dev/plugins.rst.txt
Normal file
|
|
@ -0,0 +1,106 @@
|
|||
.. _dev plugin:
|
||||
|
||||
=======
|
||||
Plugins
|
||||
=======
|
||||
|
||||
.. sidebar:: Further reading ..
|
||||
|
||||
- :ref:`plugins generic`
|
||||
|
||||
Plugins can extend or replace functionality of various components of searx.
|
||||
|
||||
Example plugin
|
||||
==============
|
||||
|
||||
.. code:: python
|
||||
|
||||
name = 'Example plugin'
|
||||
description = 'This plugin extends the suggestions with the word "example"'
|
||||
default_on = False # disabled by default
|
||||
|
||||
# attach callback to the post search hook
|
||||
# request: flask request object
|
||||
# ctx: the whole local context of the post search hook
|
||||
def post_search(request, search):
|
||||
search.result_container.suggestions.add('example')
|
||||
return True
|
||||
|
||||
External plugins
|
||||
================
|
||||
|
||||
SearXNG supports *external plugins* / there is no need to install one, SearXNG
|
||||
runs out of the box. But to demonstrate; in the example below we install the
|
||||
SearXNG plugins from *The Green Web Foundation* `[ref]
|
||||
<https://www.thegreenwebfoundation.org/news/searching-the-green-web-with-searx/>`__:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo utils/searxng.sh instance cmd bash -c
|
||||
(searxng-pyenv)$ pip install git+https://github.com/return42/tgwf-searx-plugins
|
||||
|
||||
In the :ref:`settings.yml` activate the ``plugins:`` section and add module
|
||||
``only_show_green_results`` from ``tgwf-searx-plugins``.
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
plugins:
|
||||
...
|
||||
- only_show_green_results
|
||||
...
|
||||
|
||||
|
||||
Plugin entry points
|
||||
===================
|
||||
|
||||
Entry points (hooks) define when a plugin runs. Right now only three hooks are
|
||||
implemented. So feel free to implement a hook if it fits the behaviour of your
|
||||
plugin. A plugin doesn't need to implement all the hooks.
|
||||
|
||||
|
||||
.. py:function:: pre_search(request, search) -> bool
|
||||
|
||||
Runs BEFORE the search request.
|
||||
|
||||
`search.result_container` can be changed.
|
||||
|
||||
Return a boolean:
|
||||
|
||||
* True to continue the search
|
||||
* False to stop the search
|
||||
|
||||
:param flask.request request:
|
||||
:param searx.search.SearchWithPlugins search:
|
||||
:return: False to stop the search
|
||||
:rtype: bool
|
||||
|
||||
|
||||
.. py:function:: post_search(request, search) -> None
|
||||
|
||||
Runs AFTER the search request.
|
||||
|
||||
:param flask.request request: Flask request.
|
||||
:param searx.search.SearchWithPlugins search: Context.
|
||||
|
||||
|
||||
.. py:function:: on_result(request, search, result) -> bool
|
||||
|
||||
Runs for each result of each engine.
|
||||
|
||||
`result` can be changed.
|
||||
|
||||
If `result["url"]` is defined, then `result["parsed_url"] = urlparse(result['url'])`
|
||||
|
||||
.. warning::
|
||||
`result["url"]` can be changed, but `result["parsed_url"]` must be updated too.
|
||||
|
||||
Return a boolean:
|
||||
|
||||
* True to keep the result
|
||||
* False to remove the result
|
||||
|
||||
:param flask.request request:
|
||||
:param searx.search.SearchWithPlugins search:
|
||||
:param typing.Dict result: Result, see - :ref:`engine results`
|
||||
:return: True to keep the result
|
||||
:rtype: bool
|
||||
82
_sources/dev/quickstart.rst.txt
Normal file
82
_sources/dev/quickstart.rst.txt
Normal file
|
|
@ -0,0 +1,82 @@
|
|||
.. _devquickstart:
|
||||
|
||||
======================
|
||||
Development Quickstart
|
||||
======================
|
||||
|
||||
.. _npm: https://www.npmjs.com/
|
||||
.. _Node.js: https://nodejs.org/
|
||||
.. _eslint: https://eslint.org/
|
||||
.. _stylelint: https://stylelint.io/
|
||||
|
||||
.. sidebar:: further read
|
||||
|
||||
- :ref:`makefile`
|
||||
- :ref:`buildhosts`
|
||||
|
||||
SearXNG loves developers; Developers do not need to worry about tool chains, the
|
||||
usual developer tasks can be comfortably executed via :ref:`make <makefile>`.
|
||||
|
||||
Don't hesitate, just clone SearXNG's sources and start hacking right now ..
|
||||
|
||||
.. code:: bash
|
||||
|
||||
git clone https://github.com/searxng/searxng.git searxng
|
||||
|
||||
Here is how a minimal workflow looks like:
|
||||
|
||||
1. *start* hacking
|
||||
2. *run* your code: :ref:`make run`
|
||||
3. *format & test* your code: :ref:`make format.python` and :ref:`make test`
|
||||
|
||||
If you think at some point something fails, go back to *start*. Otherwise,
|
||||
choose a meaningful commit message and we are happy to receive your pull
|
||||
request. To not end in *wild west* we have some directives, please pay attention
|
||||
to our ":ref:`how to contribute`" guideline.
|
||||
|
||||
.. sidebar:: further read
|
||||
|
||||
- :ref:`make nvm`
|
||||
- :ref:`make themes`
|
||||
|
||||
If you implement themes, you will need to setup a :ref:`Node.js environment
|
||||
<make node.env>`. Before you call *make run* (2.), you need to compile the
|
||||
modified styles and JavaScript: ``make node.clean themes.all``. If eslint_ or
|
||||
stylelint_ report some issues, try ``make themes.fix``.
|
||||
|
||||
Alternatively you can also compile selective the theme you have modified,
|
||||
e.g. the *simple* theme.
|
||||
|
||||
.. code:: bash
|
||||
|
||||
make themes.simple
|
||||
|
||||
.. tip::
|
||||
|
||||
To get live builds while modifying CSS & JS use: ``LIVE_THEME=simple make run``
|
||||
|
||||
.. sidebar:: further read
|
||||
|
||||
- :ref:`make static.build`
|
||||
|
||||
If you finished your *tests* you can start to commit your changes. To separate
|
||||
the modified source code from the build products first run:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
make static.build.restore
|
||||
|
||||
This will restore the old build products and only your changes of the code
|
||||
remain in the working tree which can now be added & committed. When all sources
|
||||
are committed, you can commit the build products simply by:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
make static.build.commit
|
||||
|
||||
Committing the build products should be the last step, just before you send us
|
||||
your PR. There is also a make target to rewind this last build commit:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
make static.build.drop
|
||||
1438
_sources/dev/reST.rst.txt
Normal file
1438
_sources/dev/reST.rst.txt
Normal file
File diff suppressed because it is too large
Load diff
121
_sources/dev/rtm_asdf.rst.txt
Normal file
121
_sources/dev/rtm_asdf.rst.txt
Normal file
|
|
@ -0,0 +1,121 @@
|
|||
==================
|
||||
Runtime Management
|
||||
==================
|
||||
|
||||
The runtimes are managed with asdf and are activated in this project via the
|
||||
`.tool-versions <.tool-versions>`_. If you have not yet installed asdf_, then
|
||||
chapter :ref:`introduce asdf` may be of help to you.
|
||||
|
||||
.. contents::
|
||||
:depth: 2
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
|
||||
Get started
|
||||
===========
|
||||
|
||||
If you have asdf installed you can install the runtimes of this project by:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ cd /path/to/searxng
|
||||
$ asdf install # will install runtimes listed in .tool-versions
|
||||
...
|
||||
|
||||
Manage Versions
|
||||
===============
|
||||
|
||||
If you want to perform a ``test`` with special runtime versions of nodejs,
|
||||
python or shellcheck, you can patch the ``.tool-versions``:
|
||||
|
||||
.. code:: diff
|
||||
|
||||
--- a/.tool-versions
|
||||
+++ b/.tool-versions
|
||||
@@ -1,2 +1,2 @@
|
||||
-python 3.12.0
|
||||
-shellcheck 0.9.0
|
||||
+python 3.11.6
|
||||
+shellcheck 0.8.0
|
||||
|
||||
To install use ``asdf install`` again. If the runtime tools have changed, any
|
||||
existing (nodejs and python) environments should be cleaned up with a ``make
|
||||
clean``.
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ asdf install
|
||||
...
|
||||
$ make clean test
|
||||
|
||||
|
||||
.. _introduce asdf:
|
||||
|
||||
Introduce asdf
|
||||
==============
|
||||
|
||||
To `download asdf`_ and `install asdf`_:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ git clone https://github.com/asdf-vm/asdf.git ~/.asdf --branch <version>
|
||||
$ echo '. "$HOME/.asdf/asdf.sh"' >> ~/.bashrc
|
||||
$ echo '. "$HOME/.asdf/completions/asdf.bash"' >> ~/.bashrc
|
||||
|
||||
Start a new shell and try to `install plugins`_:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ asdf plugin-list-all | grep -E '(golang|python|nodejs|shellcheck).git'
|
||||
golang https://github.com/asdf-community/asdf-golang.git
|
||||
nodejs https://github.com/asdf-vm/asdf-nodejs.git
|
||||
python https://github.com/danhper/asdf-python.git
|
||||
shellcheck https://github.com/luizm/asdf-shellcheck.git
|
||||
|
||||
$ asdf plugin add golang https://github.com/asdf-community/asdf-golang.git
|
||||
$ asdf plugin add nodejs https://github.com/asdf-vm/asdf-nodejs.git
|
||||
$ asdf plugin add python https://github.com/danhper/asdf-python.git
|
||||
$ asdf plugin add shellcheck https://github.com/luizm/asdf-shellcheck.git
|
||||
|
||||
Each plugin has dependencies, to compile runtimes visit the URLs from above and
|
||||
look out for the dependencies you need to install on your OS, on Debian for the
|
||||
runtimes listed above you will need:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ sudo apt update
|
||||
$ sudo apt install \
|
||||
dirmngr gpg curl gawk coreutils build-essential libssl-dev zlib1g-dev \
|
||||
libbz2-dev libreadline-dev libsqlite3-dev \
|
||||
libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev
|
||||
|
||||
With dependencies installed you can install/compile runtimes:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ asdf install golang latest
|
||||
$ asdf install nodejs latest
|
||||
$ asdf install python latest
|
||||
$ asdf install shellcheck latest
|
||||
|
||||
Python will be compiled and will take a while.
|
||||
|
||||
In the repository the version is defined in `.tool-versions`_. Outside the
|
||||
repository, its recommended that the runtime should use the versions of the OS
|
||||
(`Fallback to System Version`_) / if not already done register the system
|
||||
versions global:
|
||||
|
||||
.. code:: bash
|
||||
|
||||
$ cd /
|
||||
$ asdf global golang system
|
||||
$ asdf global nodejs system
|
||||
$ asdf global python system
|
||||
$ asdf global shellcheck system
|
||||
|
||||
.. _asdf: https://asdf-vm.com/
|
||||
.. _download asdf: https://asdf-vm.com/guide/getting-started.html#_2-download-asdf
|
||||
.. _install asdf: https://asdf-vm.com/guide/getting-started.html#_3-install-asdf
|
||||
.. _install plugins: https://asdf-vm.com/guide/getting-started.html#install-the-plugin
|
||||
.. _Fallback to System Version: https://asdf-vm.com/manage/versions.html#fallback-to-system-version
|
||||
124
_sources/dev/search_api.rst.txt
Normal file
124
_sources/dev/search_api.rst.txt
Normal file
|
|
@ -0,0 +1,124 @@
|
|||
.. _search API:
|
||||
|
||||
==========
|
||||
Search API
|
||||
==========
|
||||
|
||||
The search supports both ``GET`` and ``POST``.
|
||||
|
||||
Furthermore, two endpoints ``/`` and ``/search`` are available for querying.
|
||||
|
||||
|
||||
``GET /``
|
||||
|
||||
``GET /search``
|
||||
|
||||
Parameters
|
||||
==========
|
||||
|
||||
.. sidebar:: Further reading ..
|
||||
|
||||
- :ref:`engines-dev`
|
||||
- :ref:`settings.yml`
|
||||
- :ref:`configured engines`
|
||||
|
||||
``q`` : required
|
||||
The search query. This string is passed to external search services. Thus,
|
||||
SearXNG supports syntax of each search service. For example, ``site:github.com
|
||||
SearXNG`` is a valid query for Google. However, if simply the query above is
|
||||
passed to any search engine which does not filter its results based on this
|
||||
syntax, you might not get the results you wanted.
|
||||
|
||||
See more at :ref:`search-syntax`
|
||||
|
||||
``categories`` : optional
|
||||
Comma separated list, specifies the active search categories (see
|
||||
:ref:`configured engines`)
|
||||
|
||||
``engines`` : optional
|
||||
Comma separated list, specifies the active search engines (see
|
||||
:ref:`configured engines`).
|
||||
|
||||
``language`` : default from :ref:`settings search`
|
||||
Code of the language.
|
||||
|
||||
``pageno`` : default ``1``
|
||||
Search page number.
|
||||
|
||||
``time_range`` : optional
|
||||
[ ``day``, ``month``, ``year`` ]
|
||||
|
||||
Time range of search for engines which support it. See if an engine supports
|
||||
time range search in the preferences page of an instance.
|
||||
|
||||
``format`` : optional
|
||||
[ ``json``, ``csv``, ``rss`` ]
|
||||
|
||||
Output format of results. Format needs to be activated in :ref:`settings
|
||||
search`.
|
||||
|
||||
``results_on_new_tab`` : default ``0``
|
||||
[ ``0``, ``1`` ]
|
||||
|
||||
Open search results on new tab.
|
||||
|
||||
``image_proxy`` : default from :ref:`settings server`
|
||||
[ ``True``, ``False`` ]
|
||||
|
||||
Proxy image results through SearXNG.
|
||||
|
||||
``autocomplete`` : default from :ref:`settings search`
|
||||
[ ``google``, ``dbpedia``, ``duckduckgo``, ``mwmbl``, ``startpage``,
|
||||
``wikipedia``, ``stract``, ``swisscows``, ``qwant`` ]
|
||||
|
||||
Service which completes words as you type.
|
||||
|
||||
``safesearch`` : default from :ref:`settings search`
|
||||
[ ``0``, ``1``, ``2`` ]
|
||||
|
||||
Filter search results of engines which support safe search. See if an engine
|
||||
supports safe search in the preferences page of an instance.
|
||||
|
||||
``theme`` : default ``simple``
|
||||
[ ``simple`` ]
|
||||
|
||||
Theme of instance.
|
||||
|
||||
Please note, available themes depend on an instance. It is possible that an
|
||||
instance administrator deleted, created or renamed themes on their instance.
|
||||
See the available options in the preferences page of the instance.
|
||||
|
||||
``enabled_plugins`` : optional
|
||||
List of enabled plugins.
|
||||
|
||||
:default:
|
||||
``Hash_plugin``, ``Self_Information``,
|
||||
``Tracker_URL_remover``, ``Ahmia_blacklist``
|
||||
|
||||
:values:
|
||||
.. enabled by default
|
||||
|
||||
``Hash_plugin``, ``Self_Information``,
|
||||
``Tracker_URL_remover``, ``Ahmia_blacklist``,
|
||||
|
||||
.. disabled by default
|
||||
|
||||
``Hostnames_plugin``, ``Open_Access_DOI_rewrite``,
|
||||
``Vim-like_hotkeys``, ``Tor_check_plugin``
|
||||
|
||||
``disabled_plugins``: optional
|
||||
List of disabled plugins.
|
||||
|
||||
:default:
|
||||
``Hostnames_plugin``, ``Open_Access_DOI_rewrite``,
|
||||
``Vim-like_hotkeys``, ``Tor_check_plugin``
|
||||
|
||||
:values:
|
||||
see values from ``enabled_plugins``
|
||||
|
||||
``enabled_engines`` : optional : *all* :origin:`engines <searx/engines>`
|
||||
List of enabled engines.
|
||||
|
||||
``disabled_engines`` : optional : *all* :origin:`engines <searx/engines>`
|
||||
List of disabled engines.
|
||||
|
||||
14
_sources/dev/searxng_extra/index.rst.txt
Normal file
14
_sources/dev/searxng_extra/index.rst.txt
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
.. _searxng_extra:
|
||||
|
||||
=============================
|
||||
Tooling box ``searxng_extra``
|
||||
=============================
|
||||
|
||||
In the folder :origin:`searxng_extra/` we maintain some tools useful for CI and
|
||||
developers.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
update
|
||||
standalone_searx.py
|
||||
9
_sources/dev/searxng_extra/standalone_searx.py.rst.txt
Normal file
9
_sources/dev/searxng_extra/standalone_searx.py.rst.txt
Normal file
|
|
@ -0,0 +1,9 @@
|
|||
|
||||
.. _standalone_searx.py:
|
||||
|
||||
=====================================
|
||||
``searxng_extra/standalone_searx.py``
|
||||
=====================================
|
||||
|
||||
.. automodule:: searxng_extra.standalone_searx
|
||||
:members:
|
||||
98
_sources/dev/searxng_extra/update.rst.txt
Normal file
98
_sources/dev/searxng_extra/update.rst.txt
Normal file
|
|
@ -0,0 +1,98 @@
|
|||
=========================
|
||||
``searxng_extra/update/``
|
||||
=========================
|
||||
|
||||
:origin:`[source] <searxng_extra/update/__init__.py>`
|
||||
|
||||
Scripts to update static data in :origin:`searx/data/`
|
||||
|
||||
.. _update_ahmia_blacklist.py:
|
||||
|
||||
``update_ahmia_blacklist.py``
|
||||
=============================
|
||||
|
||||
:origin:`[source] <searxng_extra/update/update_ahmia_blacklist.py>`
|
||||
|
||||
.. automodule:: searxng_extra.update.update_ahmia_blacklist
|
||||
:members:
|
||||
|
||||
|
||||
``update_currencies.py``
|
||||
========================
|
||||
|
||||
:origin:`[source] <searxng_extra/update/update_currencies.py>`
|
||||
|
||||
.. automodule:: searxng_extra.update.update_currencies
|
||||
:members:
|
||||
|
||||
``update_engine_descriptions.py``
|
||||
=================================
|
||||
|
||||
:origin:`[source] <searxng_extra/update/update_engine_descriptions.py>`
|
||||
|
||||
.. automodule:: searxng_extra.update.update_engine_descriptions
|
||||
:members:
|
||||
|
||||
|
||||
``update_external_bangs.py``
|
||||
============================
|
||||
|
||||
:origin:`[source] <searxng_extra/update/update_external_bangs.py>`
|
||||
|
||||
.. automodule:: searxng_extra.update.update_external_bangs
|
||||
:members:
|
||||
|
||||
|
||||
``update_firefox_version.py``
|
||||
=============================
|
||||
|
||||
:origin:`[source] <searxng_extra/update/update_firefox_version.py>`
|
||||
|
||||
.. automodule:: searxng_extra.update.update_firefox_version
|
||||
:members:
|
||||
|
||||
|
||||
``update_engine_traits.py``
|
||||
===========================
|
||||
|
||||
:origin:`[source] <searxng_extra/update/update_engine_traits.py>`
|
||||
|
||||
.. automodule:: searxng_extra.update.update_engine_traits
|
||||
:members:
|
||||
|
||||
|
||||
``update_osm_keys_tags.py``
|
||||
===========================
|
||||
|
||||
:origin:`[source] <searxng_extra/update/update_osm_keys_tags.py>`
|
||||
|
||||
.. automodule:: searxng_extra.update.update_osm_keys_tags
|
||||
:members:
|
||||
|
||||
|
||||
``update_pygments.py``
|
||||
======================
|
||||
|
||||
:origin:`[source] <searxng_extra/update/update_pygments.py>`
|
||||
|
||||
.. automodule:: searxng_extra.update.update_pygments
|
||||
:members:
|
||||
|
||||
.. _update_locales.py:
|
||||
|
||||
``update_locales.py``
|
||||
=====================
|
||||
|
||||
:origin:`[source] <searxng_extra/update/update_locales.py>`
|
||||
|
||||
.. automodule:: searxng_extra.update.update_locales
|
||||
:members:
|
||||
|
||||
|
||||
``update_wikidata_units.py``
|
||||
============================
|
||||
|
||||
:origin:`[source] <searxng_extra/update/update_wikidata_units.py>`
|
||||
|
||||
.. automodule:: searxng_extra.update.update_wikidata_units
|
||||
:members:
|
||||
81
_sources/dev/translation.rst.txt
Normal file
81
_sources/dev/translation.rst.txt
Normal file
|
|
@ -0,0 +1,81 @@
|
|||
.. _translation:
|
||||
|
||||
===========
|
||||
Translation
|
||||
===========
|
||||
|
||||
.. _translate.codeberg.org: https://translate.codeberg.org/projects/searxng/
|
||||
.. _Weblate: https://docs.weblate.org
|
||||
.. _translations branch: https://github.com/searxng/searxng/tree/translations
|
||||
.. _orphan branch: https://git-scm.com/docs/git-checkout#Documentation/git-checkout.txt---orphanltnewbranchgt
|
||||
.. _Weblate repository: https://translate.codeberg.org/projects/searxng/searxng/#repository
|
||||
.. _wlc: https://docs.weblate.org/en/latest/wlc.html
|
||||
|
||||
.. |translated| image:: https://translate.codeberg.org/widgets/searxng/-/searxng/svg-badge.svg
|
||||
:target: https://translate.codeberg.org/projects/searxng/
|
||||
|
||||
.. sidebar:: |translated|
|
||||
|
||||
- :ref:`searx.babel_extract`
|
||||
- Weblate_
|
||||
- SearXNG `translations branch`_
|
||||
- SearXNG `Weblate repository`_
|
||||
- Weblate Client: wlc_
|
||||
- Babel Command-Line: `pybabel <http://babel.pocoo.org/en/latest/cmdline.html>`_
|
||||
- `weblate workflow <https://docs.weblate.org/en/latest/workflows.html>`_
|
||||
|
||||
Translation takes place on translate.codeberg.org_.
|
||||
|
||||
Translations which has been added by translators on the translate.codeberg.org_ UI are
|
||||
committed to Weblate's counterpart of the SearXNG *origin* repository which is
|
||||
located at ``https://translate.codeberg.org/git/searxng/searxng``.
|
||||
|
||||
There is no need to clone this repository, :ref:`SearXNG Weblate workflow` take
|
||||
care of the synchronization with the *origin*. To avoid merging commits from
|
||||
the counterpart directly on the ``master`` branch of *SearXNG origin*, a *pull
|
||||
request* (PR) is created by this workflow.
|
||||
|
||||
Weblate monitors the `translations branch`_, not the ``master`` branch. This
|
||||
branch is an `orphan branch`_, decoupled from the master branch (we already know
|
||||
orphan branches from the ``gh-pages``). The `translations branch`_ contains
|
||||
only the
|
||||
|
||||
- ``translation/messages.pot`` and the
|
||||
- ``translation/*/messages.po`` files, nothing else.
|
||||
|
||||
|
||||
.. _SearXNG Weblate workflow:
|
||||
|
||||
.. figure:: translation.svg
|
||||
|
||||
SearXNG's PR workflow to be in sync with Weblate
|
||||
|
||||
Sync from *origin* to *weblate*: using ``make weblate.push.translations``
|
||||
For each commit on the ``master`` branch of SearXNG *origin* the GitHub job
|
||||
:origin:`babel / Update translations branch
|
||||
<.github/workflows/integration.yml>` checks for updated translations.
|
||||
|
||||
Sync from *weblate* to *origin*: using ``make weblate.translations.commit``
|
||||
Every Friday, the GitHub workflow :origin:`babel / create PR for additions from
|
||||
weblate <.github/workflows/translations-update.yml>` creates a PR with the
|
||||
updated translation files:
|
||||
|
||||
- ``translation/messages.pot``,
|
||||
- ``translation/*/messages.po`` and
|
||||
- ``translation/*/messages.mo``
|
||||
|
||||
wlc
|
||||
===
|
||||
|
||||
.. _wlc configuration: https://docs.weblate.org/en/latest/wlc.html#wlc-config
|
||||
.. _API key: https://translate.codeberg.org/accounts/profile/#api
|
||||
|
||||
All weblate integration is done by GitHub workflows, but if you want to use wlc_,
|
||||
copy this content into `wlc configuration`_ in your HOME ``~/.config/weblate``
|
||||
|
||||
.. code-block:: ini
|
||||
|
||||
[keys]
|
||||
https://translate.codeberg.org/api/ = APIKEY
|
||||
|
||||
Replace ``APIKEY`` by your `API key`_.
|
||||
Loading…
Add table
Add a link
Reference in a new issue