Commit Graph

241 Commits

Author SHA1 Message Date
Markus Heiser d965c63429 [docs] reorder chapter "Engines & Settings"
- Split chapter "Engines" and rename it into "Engines & Settings"
- Move docs/admin/engines.rst -> docs/admin/engines/engine_settings.rst

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:04:38 +02:00
Markus Heiser 0d31740cfd [docs] move article "XPath Engine" to to admin/engines/
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:04:38 +02:00
Markus Heiser 274288ea99 [docs] revision of the article "Offline engines"
This patch is a a complete revision of the article "Offline engines", which also
merges the content from the searx-wiki [1] into this article.

[1] https://github.com/searx/searx/wiki/Offline-engines

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:04:38 +02:00
Markus Heiser 1c8cf1d3a8 [docs] add engine "Demo Offline Engine"
This engine just exists for documentation purpose.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:04:38 +02:00
Markus Heiser acb1577202 [docs] move blog article "Offline engines" to dev/offline_engines.rst
The article "Offline engines" should be in developer's documentation next to
chapter "Engine overview".

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:04:38 +02:00
Markus Heiser b88d9d49cc [docs] remove blog article 'Introducing Python 3 support'
This article is obsolete since a long time: Python 2 support has been dropped
and these days, virtualenv is managed by ``make pyenv.install``.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:04:38 +02:00
Markus Heiser 1b3f0c848e [docs] move blog article "SQL engines" to admin/engines/
The article "SQL engines" should be in admin's engine documentation (like the
recoll engine).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:04:38 +02:00
Markus Heiser a561083537 [docs] move blog article "Command line engines" to admin/engines/
The article "Command line engines" should be in admin's engine
documentation (like the recoll engine).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 13:16:34 +02:00
Markus Heiser 523bddf987 [docs] remove blog article about searx-admin
- https://github.com/kvch/searx-admin last maintained 4 years ago
- searx-admin does not support 'use_default_settings' [1] (b4b81a5e)

[1] https://searxng.github.io/searxng/admin/settings.html#use-default-settings

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 13:16:34 +02:00
Alexandre Flament dddf3e2be2 [mod] "make docs" doesn't initialize the engines
avoid external HTTP requests with "make docs"
2021-06-04 11:26:37 +02:00
Markus Heiser 15de8708b7 [docs] highlight source code linked from *automodule* content
sphinx.ext.viewcode won't highlight when 'highlight_language' [1] is set to
string 'none' [2]

[1] https://www.sphinx-doc.org/en/master/usage/extensions/viewcode.html
[2] https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-highlight_language

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-01 16:37:20 +02:00
Markus Heiser ebbd5f7366 [docs] add new rubric 'Source-Code'
- document namespace: searx.engines
- move docs/dev/xpath_engine.rst -> docs/src/searx.engines.xpath.rst

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-01 16:37:20 +02:00
Markus Heiser e7a4d0961b [doc] add blog post about SQL servers
Modified merge of [22a79a4] from searx.

[22a79a4] 22a79a4896

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-28 19:46:54 +02:00
Markus Heiser 85364edaf0 [fix] webapp.py - running without installation is not supported
Remove extension of the sys.path (aka PYTHONPATH).  Running instance directly
from repository's folder is a relict from the early beginning in
2014 (fd651083f) and is no longer supported.

Since commit dd46629 was merged the command line 'searx-run' exists and should
be used.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-26 21:17:19 +02:00
Markus Heiser 2065a324d9 [doc] use a hint instead of a warning about searxng
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-23 11:54:21 +02:00
Markus Heiser 8cd544b2a6 [doc] add documentation about the XPath engine
- pylint searx/engines/xpath.py
- fix indentation of some long lines
- add logging
- add doc-strings

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-23 11:48:21 +02:00
Markus Heiser 12b55320ea [doc] docker: revision and preparation of the documentation
- add sidebars with addition infos about commands and docker in general
- fix long lines & indentation
- correct link to https://github.com/searxng/searxng-docker

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-17 12:02:24 +02:00
Markus Heiser d6c321b830 [mod] docker-entrypoint.sh - add script to make test.sh (shellcheck)
Add script docker-entrypoint.sh to shellcheck and try to _simplify_ and
_normalize_ some parts:

- fix issues reported by shellcheck
- don't mix tab and space indent
- command 'help' replaced by '-h': ./dockerfiles/docker-entrypoint.sh -h
- replace printf in help() by 'cat <<EOF'

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-17 12:02:24 +02:00
Markus Heiser e14a660c9a [fix] sphinx-tabs should not overlap sidebar
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-14 17:19:49 +02:00
Alexandre Flament 20580bcbd4 [docker] multiarch support: linux/amd64,linux/arm64,linux/arm/v7
make docker.buildx : build and push multiarch build.
(it can't be only build)

use buildx with the --cache-from and --cache-to options to cache the layers
(only the last built is cached)
2021-04-28 14:40:00 +02:00
Markus Heiser 28b25185c5 [brand] searxng -- fix links to issue tracker & WEB-GUI
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-25 14:25:08 +02:00
Markus Heiser 3aa153f08e [fix doc] manage.sh update_packages does not exists anymore
Reported-by: https://github.com/searx/searx/issues/2776
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-24 17:03:10 +02:00
Markus Heiser 34badee467 [doc] modify docs to fit to the new build boilerplate
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-23 11:26:22 +02:00
Markus Heiser d0e371f474 [mod] replace makefile boilerplate by 'manage' script
Replaces the make targets with the bash scripts

Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-23 11:26:22 +02:00
Alexandre Flament 9d84025e4f searxng brand 2021-04-22 19:57:02 +02:00
Zackptg5 bed7c3c614 [fix] Fix typo in morty installation instruction 2021-04-22 12:17:49 +02:00
Alexandre Flament 01cefffbf6
Merge pull request #1 from metasearch-lab/httpx_networks
Httpx networks
2021-04-12 17:34:21 +02:00
Alexandre Flament d14994dc73 [httpx] replace searx.poolrequests by searx.network
settings.yml:

* outgoing.networks:
   * can contains network definition
   * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections,
     keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries
   * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time)
   * local_addresses can be "192.168.0.1/24" (it supports IPv6)
   * support_ipv4 & support_ipv6: both True by default
     see https://github.com/searx/searx/pull/1034
* each engine can define a "network" section:
   * either a full network description
   * either reference an existing network

* all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
2021-04-12 17:25:56 +02:00
ColonisationCaptain b2093a8bc0
correct typo/grammatical mistake 2021-04-09 14:56:59 +01:00
Noémi Ványi 547478089f Add blogpost about Elasticsearch, Meilisearch and Solr 2021-04-07 23:21:00 +02:00
Alexandre Flament 99e0651cea [mod] by default allow only HTTPS, not HTTP
Related to https://github.com/searx/searx/pull/2373
2021-03-08 11:35:08 +01:00
Alexandre Flament aaae9a209e
Merge pull request #2600 from dalf/searx-extra
Add searx_extra package
2021-03-05 09:43:39 +01:00
Alexandre Flament b8cd326464 Add searx_extra package
Split the utils directory into:
* searx_extra contains update scripts, standalone_searx.py
* utils contains the files to build and setup searx.
2021-03-04 11:59:14 +01:00
Alex Balgavy 6b59800dc6 Fix security vulnerabilities in suggested nginx configuration
The suggested configurations for nginx found in the documentation and
templates lead to vulnerabilities allowing host spoofing [1] and path
traversal [2], as reported by Gixy [3]. This commit fixes those issues.

[1] https://github.com/yandex/gixy/blob/master/docs/en/plugins/hostspoofing.md
[2] https://github.com/yandex/gixy/blob/master/docs/en/plugins/aliastraversal.md
[3] https://github.com/yandex/gixy
2021-03-03 12:34:22 +01:00
Alexandre Flament c22d4c764c [fix] duckduckgo engine: "!ddg !g" do not redirect to google
* searx understand "!ddg !g time" as : send "!g time" to DDG
* !g a DDG bang for Google: DDG return a HTTP redirect to Google

This commit adds a the allows_redirect param not to follow HTTP redirect.

The DDG engine returns a empty result as before without HTTP redirect.
2021-02-12 11:10:08 +01:00
Markus Heiser 89554e42a9 [fix] make books/user.pdf
Error:

  Configuration error:
  There is a programmable error in your configuration file:
  ...
  NameError: name 'DOCS_URL' is not defined
  make: *** [utils/makefile.sphinx:156: books/user.latex] Fehler 2

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-02-02 20:14:07 +01:00
Alexandre Flament ca93a01844 [mod] dynamically set language_support variable
The language_support variable is set to True by default,
and set to False in only 5 engines.

Except the documentation and the /config URL, this variable is not used.

This commit remove the variable definition in the engines, and
set value according to supported_languages length: False when the length is 0,
True otherwise.

Close #2485
2021-02-01 17:10:37 +01:00
Alexandre Flament 0f18e885bf
Merge pull request #2479 from Tobi823/master
Document workaround for using 2 languages simultaneously #1508
2021-01-27 21:29:42 +01:00
Markus Heiser a69a8a3ed5 [doc] improve admin-docs about result proxy (morty) configuration
[1] https://github.com/searx/searx/pull/1872#issuecomment-768107138

Suggested-by @dalf [1]
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-27 09:58:06 +01:00
Tobi823 16a0a01553 Document workaround for using 2 languages simultaneously #1508 2021-01-18 17:23:09 +01:00
Markus Heiser c4a98862bf [fix] setup.py requires pyyaml installed
pip install -e .
...
Obtaining file:///usr/local/searx/searx-src
    ERROR: Command errored out with exit status 1:
     command: /usr/local/searx/searx-pyenv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/usr/local/searx/searx-src/setup.py'"'"'; __file__='"'"'/usr/local/searx/searx-src/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'rn'"'"', '"'"'n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-vzer91m2
         cwd: /usr/local/searx/searx-src/
    Complete output (9 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/usr/local/searx/searx-src/setup.py", line 10, in <module>
        from searx.version import VERSION_STRING
      File "/usr/local/searx/searx-src/searx/__init__.py", line 19, in <module>
        import searx.settings_loader
      File "/usr/local/searx/searx-src/searx/settings_loader.py", line 8, in <module>
        import yaml
    ModuleNotFoundError: No module named 'yaml'
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-16 08:58:13 +01:00
Alexandre Flament 9c55d772e9
Merge pull request #2408 from return42/rm-brand-make
[mod] move brand options from Makefile to settings.yml
2021-01-12 10:52:42 +01:00
Archie ff2370eead
Fixed typo 2021-01-12 01:33:43 +00:00
Markus Heiser 4853a29803 [mod] improve settings documentation
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-11 22:12:38 +01:00
Markus Heiser 9485179064 [mod] move brand options from Makefile to settings.yml
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-11 22:12:38 +01:00
Alexandre Flament 5fb9a5c7c1
Merge pull request #2411 from dalf/update-secret-key-check
Update secret key check
2020-12-27 18:04:53 +01:00
Markus Heiser b6dcf90816 Revert "[fix] sphinx 3.1 CSS issues"
This reverts commit 0616684baa.

Since PR https://github.com/sphinx-doc/sphinx/pull/7878 has been merged into
Spinx-doc (v3.1.2), this patch is no longer needed:

  See sphinx-doc project, PR 7838 & 7484 with elementary patch to the basic CSS:

  - https://github.com/sphinx-doc/sphinx/issues/7838#issuecomment-646009605
  - https://github.com/sphinx-doc/sphinx/pull/7484#issuecomment-646058972

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-12-27 11:27:42 +01:00
Alexandre Flament eb1b5289d5 [mod] documentation: change the jinja context doesn't depend on searx.webapp
Before this commit, in the documentation, the jinja context is 'webapp' and contains
the global variable in the searx.webapp module.

This commit changes this to include only the mandatory variables to build the
documentation.
2020-12-27 10:00:35 +01:00
Markus Heiser 3beede7ab7 [doc] describe 'make books/{name}.html' and 'books/{name}.pdf'
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-12-24 13:55:22 +01:00
Alexandre Flament 02fc4147ce [mod] dictzone, translated, currency_convert: use engine_type online_curency and online_dictionnary 2020-12-17 11:39:36 +01:00