zaclys
/
searxng
mirror of https://github.com/searxng/searxng.git
1
0
Fork 1
Code Issues Packages Projects Releases Wiki Activity
4,535 Commits 5 Branches 17 Tags 71 MiB
Commit Graph

4535 Commits

Author SHA1 Message Date
Alexandre Flament 94e6ed6110
Merge pull request #66 from return42/docker-buildx 
review of docker entry point & docker documentation
 2021-05-19 17:31:33 +02:00
Alexandre Flament d1cffc55cb [fix] unit test: don't load /etc/searx/settings.yml 
Add a new environment variable SEARX_DISABLE_ETC_SETTINGS
to disable loading of /etc/searx/settings.yml

unit tests:
* set SEARX_DISABLE_ETC_SETTINGS to 1
* remove SEARX_SETTINGS_PATH if it exists
 2021-05-18 17:23:21 +02:00
Alexandre Flament d255e5637b
Merge pull request #81 from return42/csp-bar-graph 
[fix] make /stats more CSP compliant
 2021-05-18 08:55:56 +02:00
Alexandre Flament da2b0562d4
Merge pull request #83 from return42/fix-archive-is 
[fix] engine "archive is" - search_url has been changed
 2021-05-18 08:44:34 +02:00
Markus Heiser 66c8482725 [emacs] don't use tabs, except in makefile-gmake-mode 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 21:26:20 +02:00
Markus Heiser 73d73dcf9b [static] make /stats more CSP compliant- make themes.all 
Based on commits

- 0507e185 [fix] bar graph and rename CSS class engine-scores -> engine-score
- 3e9ad7ae [fix] make /stats more CSP compliant - github issue form
- 34859d0e [fix] make /stats more CSP compliant - oscar theme
- 0a6c4884 [fix] make /stats more CSP compliant - simple theme
- cdfb4b7f [fix] make /stats more CSP compliant - bar graph
- 965817f2 [fix] simple theme - generate missing sourceMap file

this patch is generated by::

     make themes.all

Reported-by: https://github.com/searxng/searxng/issues/57
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 21:25:32 +02:00
Markus Heiser 0507e185a5 [fix] bar graph and rename CSS class engine-scores -> engine-score 
- drop #main_stats selector in stats.less
- 'engine-score' exists before this PR.
- untabify searx/static/themes/__common__/less/stats.less

for details see comment at: d93bec7638..1204e4f07e (r633571496)

Suggested-by: @dalf in commit 1204e4f0
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 21:24:19 +02:00
Markus Heiser 3e9ad7ae0c [fix] make /stats more CSP compliant - github issue form 
Hide textarea from github issue form::

    ./__common__/new_issue.html:6:    <textarea name="body" style="display: none;">{{- '' -}}

BTW: fix indentation.

Reported-by: https://github.com/searxng/searxng/issues/57
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 21:23:27 +02:00
Markus Heiser 34859d0e58 [fix] make /stats more CSP compliant - oscar theme 
Replace oscar theme's *styles* (see below) by CSP compliant implementation in
``searx/static/themes/__common__/less/stats.less`` ::

    ./oscar/stats.html:29:  <th scope="col" style="width:20rem;">{{ th_sort('name', _("Engine name")) }}</th>
    ./oscar/stats.html:30:  <th scope="col" style="width:7rem; text-align: right;">{{ th_sort('score', _('Scores')) }}</th>
    ./oscar/stats.html:33:  <th scope="col" style="text-align: right;">{{ th_sort('reliability', _('Reliability')) }}</th>
    ./oscar/stats.html:38:  <td style="text-align: right;">
    ./oscar/stats.html:91:  <td style="text-align: right;"> {{ engine_reliabilities.get(engine_stat.name, {}).get('reliablity') }}</td>
    ./oscar/stats.html:109: <tbody style="padding-top: 1rem;">
    ./oscar/stats.html:112: <th scope="row" style="width: 10rem">{{ _('Exception') }}</th><td>{{ error.exception_classname }}</td>
    ./oscar/stats.html:114: <th scope="row" style="width: 10rem">{{ _('Message') }}</th><td>{{ error.log_message }}</td>
    ./oscar/stats.html:116: <th scope="row" style="width: 10rem">{{ _('Percentage') }}</th><td style="width: 10rem">{{ error.percentage }}</td>
    ./oscar/stats.html:121: <span style="border-right: 1px solid gray; padding: 0 1rem 0 0; margin: 0 0 0 0.5rem;">{{ param }}</span>
    ./oscar/stats.html:138: <th scope="col" style="width: 10rem">{{ _('Failed test') }}</th>

Reported-by: https://github.com/searxng/searxng/issues/57
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 17:41:22 +02:00
Markus Heiser 0a6c488417 [fix] make /stats more CSP compliant - simple theme 
Replace simple theme's *styles* (see below) by CSP compliant implementation in
``searx/static/themes/simple/less/stats.less`` ::

    ./simple/stats.html:26:  <table style="max-width: 1280px; margin: 0 auto 0 0;">
    ./simple/stats.html:28:  <th scope="col" style="width:20rem;">{{ th_sort('name', _("Engine name")) }}</th>
    ./simple/stats.html:29:  <th scope="col" style="width:7rem; text-align: right;">{{ th_sort('score', _('Scores')) }}</th>
    ./simple/stats.html:32:  <th scope="col" style="text-align: right;">{{ th_sort('reliability', _('Reliability')) }}</th>
    ./simple/stats.html:37:  <td style="text-align: right;">
    ./simple/stats.html:90:  <td style="text-align: right;"> {{ engine_reliabilities.get(engine_stat.name, {}).get('reliablity') }}</td>
    ./simple/stats.html:106: <table style="max-width: 1280px; margin: 1rem; border: 1px solid gray;">
    ./simple/stats.html:107: <tbody style="padding-top: 1rem;">
    ./simple/stats.html:110: <th scope="row" style="width: 10rem">{{ _('Exception') }}</th><td>{{ error.exception_classname }}</td>
    ./simple/stats.html:112: <th scope="row" style="width: 10rem">{{ _('Message') }}</th><td>{{ error.log_message }}</td>
    ./simple/stats.html:114: <th scope="row" style="width: 10rem">{{ _('Percentage') }}</th><td style="width: 10rem">{{ error.percentage }}</td>
    ./simple/stats.html:119: <span style="border-right: 1px solid gray; padding: 0 1rem 0 0; margin: 0 0 0 0.5rem;">{{ param }}</span>
    ./simple/stats.html:136: <th scope="col" style="width: 10rem">{{ _('Failed test') }}</th>

Reported-by: https://github.com/searxng/searxng/issues/57
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 17:41:22 +02:00
Markus Heiser cdfb4b7ff9 [fix] make /stats more CSP compliant - bar graph 
Replace bar graph's *styles* (see below) by CSP compliant implementation in
``searx/static/themes/__common__/less/stats.less`` ::

    ./simple/stats.html:49: <span style="width: calc(max(2px, 100%*{{ (engine_stat.result_count / engine_stats.max_result_count )|round(3) }}))" class="stacked-bar-chart-serie1"></span>{{- "" -}}
    ./simple/stats.html:57: <span style="width: calc(max(2px, 100%*{{ (engine_stat.http / engine_stats.max_time )|round(3) }}))" class="stacked-bar-chart-serie1"></span>{{- "" -}}
    ./simple/stats.html:58: <span style="width: calc(100%*{{ engine_stat.processing / engine_stats.max_time |round(3) }})" class="stacked-bar-chart-serie2"></span>{{- "" -}}
    ./oscar/stats.html:50:  <span style="width: calc(max(2px, 100%*{{ (engine_stat.result_count / engine_stats.max_result_count )|round(3) }}))" class="stacked-bar-chart-serie1"></span>{{- "" -}}
    ./oscar/stats.html:58:  <span style="width: calc(max(2px, 100%*{{ (engine_stat.http / engine_stats.max_time )|round(3) }}))" class="stacked-bar-chart-serie1"></span>{{- "" -}}
    ./oscar/stats.html:59:  <span style="width: calc(100%*{{ engine_stat.processing / engine_stats.max_time |round(3) }})" class="stacked-bar-chart-serie2"></span>{{- "" -}}

Reported-by: https://github.com/searxng/searxng/issues/57
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 17:41:22 +02:00
Markus Heiser 965817f294 [fix] simple theme - generate missing sourceMap file 
C&P from searx/static/themes/oscar/gruntfile.js

Suggested-by: @dalf in commit 1204e4f0
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 17:41:22 +02:00
Markus Heiser 93594a7b42 [less] update grunt-contrib-less v3.3.0 
Upgraded [v3.3.0] otherwise::

`  width: calc(100% - 5rem);`

becomes `width: 95%` once compiled by less version 1.4.1.

[v3.3.0] https://github.com/gruntjs/grunt-contrib-less/releases/tag/v3.0.0

Suggested-by: @dalf in commit 1204e4f0
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 17:41:22 +02:00
Markus Heiser 57cdf180fe [fix] engine archive is - search_url has been changed 
BTW: set soft_max_redirects from *archive is* by 1, to prevent logging::

    DEBUG:httpx._client:HTTP Request: GET https://archive.is/search/?q=www.python.org "HTTP/2 302 Found"
    DEBUG:httpx._client:HTTP Request: GET https://archive.is/www.python.org "HTTP/2 200 OK"
    DEBUG:searx:archive is: ErrorContext('searx/search/processors/online.py', 110, 'count_error(self.engine_name,', None, '1 redirects, maximum: 0', ('200', 'OK', 'archive.is')) True

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 15:21:53 +02:00
Markus Heiser ffcebf5e12 [enh] xpath engine - add request parameter 'soft_max_redirects' 
Make 'soft_max_redirects' configurable per Xpath engine::

    - name : <engine-name>
      engine : xpath
      soft_max_redirects: 1
      ...

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 15:04:55 +02:00
Markus Heiser 130e58f982 [brand] Dockerfile default INSTANCE_NAME=searxng 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 12:02:24 +02:00
Markus Heiser 12b55320ea [doc] docker: revision and preparation of the documentation 
- add sidebars with addition infos about commands and docker in general
- fix long lines & indentation
- correct link to https://github.com/searxng/searxng-docker

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 12:02:24 +02:00
Markus Heiser d6c321b830 [mod] docker-entrypoint.sh - add script to make test.sh (shellcheck) 
Add script docker-entrypoint.sh to shellcheck and try to _simplify_ and
_normalize_ some parts:

- fix issues reported by shellcheck
- don't mix tab and space indent
- command 'help' replaced by '-h': ./dockerfiles/docker-entrypoint.sh -h
- replace printf in help() by 'cat <<EOF'

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 12:02:24 +02:00
Markus Heiser 1970d28a60
Merge pull request #63 from searxng/dependabot/pip/master/pygments-2.9.0 
Bump pygments from 2.8.1 to 2.9.0
 2021-05-15 16:55:54 +00:00
dependabot[bot] 1b26bbcd8f
Bump pygments from 2.8.1 to 2.9.0 
Bumps [pygments](https://github.com/pygments/pygments) from 2.8.1 to 2.9.0.
- [Release notes](https://github.com/pygments/pygments/releases)
- [Changelog](https://github.com/pygments/pygments/blob/master/CHANGES)
- [Commits](https://github.com/pygments/pygments/compare/2.8.1...2.9.0)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-15 07:47:03 +00:00
Markus Heiser 7c21552b1b Revert "Merge pull request #72 from searxng/dependabot/pip/master/jinja2-3.0.0" 
This reverts commit 9e9a1ac6c3, reversing
changes made to 5b1f04fff0.

We can't upgrade to jinja2==3.0.0 until Sphinx 4.1.0 has been released [1][2]

ERROR: Cannot install -r /share/searx/requirements-dev.txt (line 11) and jinja2==3.0.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested jinja2==3.0.0
    sphinx 4.0.1 depends on Jinja2<3.0 and >=2.3

[1] https://github.com/sphinx-doc/sphinx/issues/9088#issuecomment-831020758
[2] https://github.com/sphinx-doc/sphinx/pull/9161

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-15 09:39:32 +02:00
Markus Heiser 606ddafc49
Merge pull request #73 from searxng/dependabot/pip/master/langdetect-1.0.9 
Bump langdetect from 1.0.8 to 1.0.9
 2021-05-15 07:10:56 +00:00
Markus Heiser 9e9a1ac6c3
Merge pull request #72 from searxng/dependabot/pip/master/jinja2-3.0.0 
Bump jinja2 from 2.11.3 to 3.0.0
 2021-05-15 06:52:19 +00:00
Markus Heiser 5b1f04fff0
Merge pull request #78 from return42/fix-docutils 
Revert "[fix] docutils v0.17 incompatibility to previeous v0.16"
 2021-05-14 16:21:06 +00:00
Markus Heiser 780b2ee8c6 Revert "[fix] docutils v0.17 incompatibility to previeous v0.16" 
This reverts commit 4557d58919.

see https://github.com/sphinx-doc/sphinx/issues/9088
 2021-05-14 18:14:16 +02:00
Markus Heiser 89a56a890c
Merge pull request #70 from searxng/dependabot/pip/master/sphinx-4.0.1 
Bump sphinx from 3.5.4 to 4.0.1
 2021-05-14 16:09:34 +00:00
dependabot[bot] 7562f3365b
Bump sphinx from 3.5.4 to 4.0.1 
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 3.5.4 to 4.0.1.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/4.x/CHANGES)
- [Commits](https://github.com/sphinx-doc/sphinx/compare/v3.5.4...v4.0.1)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-14 15:53:32 +00:00
Markus Heiser 776f2e0403
Merge pull request #75 from searxng/dependabot/pip/master/pallets-sphinx-themes-2.0.0 
Bump pallets-sphinx-themes from 1.2.3 to 2.0.0
 2021-05-14 15:52:49 +00:00
dependabot[bot] d9d135e361
Bump pallets-sphinx-themes from 1.2.3 to 2.0.0 
Bumps [pallets-sphinx-themes](https://github.com/pallets/pallets-sphinx-themes) from 1.2.3 to 2.0.0.
- [Release notes](https://github.com/pallets/pallets-sphinx-themes/releases)
- [Changelog](https://github.com/pallets/pallets-sphinx-themes/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/pallets-sphinx-themes/compare/1.2.3...2.0.0)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-14 15:24:40 +00:00
Markus Heiser 7bfa0ea358
Merge pull request #76 from return42/fix-spinx-tabs 
[fix] sphinx-tabs should not overlap sidebar
 2021-05-14 15:23:05 +00:00
Markus Heiser e14a660c9a [fix] sphinx-tabs should not overlap sidebar 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-14 17:19:49 +02:00
Markus Heiser 469c8b4c4b
Merge pull request #71 from searxng/dependabot/pip/master/sphinx-tabs-3.0.0 
Bump sphinx-tabs from 2.1.0 to 3.0.0
 2021-05-14 15:18:30 +00:00
dependabot[bot] c39bce96f9
Bump sphinx-tabs from 2.1.0 to 3.0.0 
Bumps [sphinx-tabs](https://github.com/executablebooks/sphinx-tabs) from 2.1.0 to 3.0.0.
- [Release notes](https://github.com/executablebooks/sphinx-tabs/releases)
- [Changelog](https://github.com/executablebooks/sphinx-tabs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/executablebooks/sphinx-tabs/compare/v2.1.0...v3.0.0)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-14 14:26:55 +00:00
dependabot[bot] be9e704c04
Bump jinja2 from 2.11.3 to 3.0.0 
Bumps [jinja2](https://github.com/pallets/jinja) from 2.11.3 to 3.0.0.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/jinja/compare/2.11.3...3.0.0)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-14 14:26:39 +00:00
dependabot[bot] 9d676f3826
Bump langdetect from 1.0.8 to 1.0.9 
Bumps [langdetect](https://github.com/Mimino666/langdetect) from 1.0.8 to 1.0.9.
- [Release notes](https://github.com/Mimino666/langdetect/releases)
- [Commits](https://github.com/Mimino666/langdetect/commits)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-14 14:26:17 +00:00
Markus Heiser d4ce13ddfd
Merge pull request #74 from searxng/pylint-workaround 
[fix] webapp.py: pylint: disable=assigning-non-slot for request.*
 2021-05-14 14:24:35 +00:00
Alexandre Flament 90fe232e08 [fix] webapp.py: pylint: disable=assigning-non-slot for request.* 
Note: actually, the code should use flask.g.* instead of request.*
 2021-05-14 11:15:35 +02:00
Alexandre Flament 9a383d1533
Merge pull request #67 from searxng/fix-issue-65 
[fix] fix usage of docker environment variable INSTANCE_NAME
 2021-05-10 08:33:32 +02:00
Alexandre Flament a40daec565
Merge pull request #68 from searxng/mod-docker-upgrade-alpine-version 
[mod] Dockerfile: upgrade to Alpine 3.13
 2021-05-10 08:32:31 +02:00
Alexandre Flament 8c5cb5039e [mod] Dockerfile: upgrade to Alpine 3.13 
See:
* https://alpinelinux.org/posts/Alpine-3.13.0-released.html
* musl>=1.2.1 have a new malloc implementation (mallocng) :
 2021-05-09 11:06:07 +02:00
Alexandre Flament b46a3ecfc7 [fix] fix usage of docker environment variable INSTANCE_NAME 
INSTANCE_NAME was ignored.

close #65
 2021-05-09 11:00:28 +02:00
Alexandre Flament 332e3a2a09
Merge pull request #64 from searxng/fix-test-sideeffect 
[fix] fix KeyError: 'ipv6'
 2021-05-07 11:05:09 +02:00
Alexandre Flament 605124679d [fix] fix KeyError: 'ipv6' 
tests/units/network/test_network.py requires a call to searx.network.network.initialize
Depending of the test order execution, this function was sometimes call in another test,
sometimes not.

This commit ensure there is a call to initialize()
 2021-05-07 10:57:55 +02:00
Alexandre Flament f92a8e432a
Merge pull request #60 from searxng/mod-http-header-server-timing 
[mod] Server-Timing header: add render field
 2021-05-07 10:48:06 +02:00
Alexandre Flament d12033e166 [mod] Server-Timing header: add render field 
Add a "render" field to the Server-Timing header
This field is the time spent in the searx.webapp.render function (Jinja2 rendering)

See https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Server-Timing
 2021-05-06 09:39:52 +02:00
Alexandre Flament 39eda78088
Merge pull request #59 from searxng/mod-sslcontext 
[mod] searx.network.client: the same configuration reuses the same ssl.SSLContext
 2021-05-05 20:42:04 +02:00
Alexandre Flament 0f4e995ab4 [mod] searx.network.client: the same configuration reuses the same ssl.SSLContext 
before there was one ssl.SSLContext per client.

see https://github.com/encode/httpx/issues/978
 2021-05-05 20:36:37 +02:00
Alexandre Flament 5b13786abb
Merge pull request #58 from searxng/mod-multithreading 
[mod] multithreading only in searx.search.* packages
 2021-05-05 18:33:16 +02:00
Markus Heiser fa0d05c313 [pylint] checker/__main__.py & checker/background.py 
Lint files that has been touched by [PR #58]

[PR #58] https://github.com/searxng/searxng/pull/58

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-05 16:47:02 +02:00
Alexandre Flament 8c1a65d32f [mod] multithreading only in searx.search.* packages 
it prepares the new architecture change,
everything about multithreading in moved in the searx.search.* packages

previously the call to the "init" function of the engines was done in searx.engines:
* the network was not set (request not sent using the defined proxy)
* it requires to monkey patch the code to avoid HTTP requests during the tests
 2021-05-05 13:12:42 +02:00