zaclys/searxng
1
0
Fork 1
mirror of https://github.com/searxng/searxng synced 2024-01-01 19:24:07 +01:00
Code Issues Projects Releases Packages Wiki Activity
6955 commits 11 branches 17 tags 103 MiB
Commit graph

6955 commits

This branch
This branch
All branches
Author SHA1 Message Date
Markus Heiser
3a71d4b175 [pylint] searx/engines/genius.py, add logger & normalized indentation 
- pylint searx/engines/genius.py
- add logger and log ignored exceptions
- normalized various indentation

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-24 16:19:06 +02:00
Markus Heiser
84a943f867 [enh] XPath engine - add time safe-search support 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-23 22:26:18 +02:00
Markus Heiser
6bfe3fd033 [enh] XPath engine - add time range support 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-23 16:49:30 +02:00
Markus Heiser
1933577c8e [enh] XPath engine - add ISO 639-1 {lang} replacement to search-URL 
BTW: remove obsolte params['query'] and not needed paging condition.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-23 15:05:36 +02:00
Markus Heiser
703f8c4a8b
Merge pull request #91 from return42/xpath-misc 
[doc] add documentation about the XPath engine
 2021-05-23 10:02:16 +00:00
Markus Heiser
2065a324d9 [doc] use a hint instead of a warning about searxng 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-23 11:54:21 +02:00
Markus Heiser
8cd544b2a6 [doc] add documentation about the XPath engine 
- pylint searx/engines/xpath.py
- fix indentation of some long lines
- add logging
- add doc-strings

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-23 11:48:21 +02:00
Markus Heiser
2398e9a1fe
Merge pull request #85 from searxng/fix-metrics-offline-engine 
Fix metrics offline engine
 2021-05-22 13:34:44 +00:00
Markus Heiser
7728e25b11 [refactor] metrics.get_reliabilities() - make code more readable 
- init stat values by None
- drop round_or_none
- don't try to get percentage if base is 'None'

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-22 15:17:18 +02:00
Alexandre Flament
99aaf86b73 [fix] offline engines: fix templates /stats and /preferences 2021-05-22 15:17:18 +02:00
Alexandre Flament
3014463fed [fix] metrics: processing time = total time if there is no http time 
It was previsouly None

Fix /stats
 2021-05-22 15:17:18 +02:00
Alexandre Flament
ec83493538 [fix] offline engine: don't crash on time recording 2021-05-22 15:17:18 +02:00
Alexandre Flament
2f76b570ab
Merge pull request #86 from searxng/remove-call-to-gc 
[mod] remove gc.collect() after each user request
 2021-05-21 18:31:04 +02:00
Markus Heiser
6f1446d55f [pylint] searx/search/__init__.py & replace lic-text by SPDX tag 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-21 17:31:22 +02:00
Alexandre Flament
426fadccb3 [mod] remove gc.collect() after each user request 2021-05-21 17:23:18 +02:00
Markus Heiser
1ab4317f59
Merge pull request #88 from searxng/dependabot/pip/master/sphinx-4.0.2 
Bump sphinx from 4.0.1 to 4.0.2
 2021-05-21 06:33:34 +00:00
dependabot[bot]
1e9efd0822 Bump sphinx from 4.0.1 to 4.0.2 
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 4.0.1 to 4.0.2.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/4.x/CHANGES)
- [Commits](https://github.com/sphinx-doc/sphinx/compare/v4.0.1...v4.0.2)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-21 08:31:50 +02:00
Markus Heiser
1cfb3c3221
Merge pull request #87 from searxng/dependabot/pip/master/pallets-sphinx-themes-2.0.1 
Bump pallets-sphinx-themes from 2.0.0 to 2.0.1
 2021-05-21 06:25:31 +00:00
dependabot[bot]
b565efb48f
Bump pallets-sphinx-themes from 2.0.0 to 2.0.1 
Bumps [pallets-sphinx-themes](https://github.com/pallets/pallets-sphinx-themes) from 2.0.0 to 2.0.1.
- [Release notes](https://github.com/pallets/pallets-sphinx-themes/releases)
- [Changelog](https://github.com/pallets/pallets-sphinx-themes/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/pallets-sphinx-themes/compare/2.0.0...2.0.1)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-21 05:53:03 +00:00
Alexandre Flament
0c1ab0227d
Merge pull request #84 from searxng/fix-test-ignore-etc-settings 
[fix] unit test: don't load /etc/searx/settings.yml
 2021-05-20 13:31:53 +02:00
Alexandre Flament
94e6ed6110
Merge pull request #66 from return42/docker-buildx 
review of docker entry point & docker documentation
 2021-05-19 17:31:33 +02:00
Alexandre Flament
d1cffc55cb [fix] unit test: don't load /etc/searx/settings.yml 
Add a new environment variable SEARX_DISABLE_ETC_SETTINGS
to disable loading of /etc/searx/settings.yml

unit tests:
* set SEARX_DISABLE_ETC_SETTINGS to 1
* remove SEARX_SETTINGS_PATH if it exists
 2021-05-18 17:23:21 +02:00
Alexandre Flament
d255e5637b
Merge pull request #81 from return42/csp-bar-graph 
[fix] make /stats more CSP compliant
 2021-05-18 08:55:56 +02:00
Alexandre Flament
da2b0562d4
Merge pull request #83 from return42/fix-archive-is 
[fix] engine "archive is" - search_url has been changed
 2021-05-18 08:44:34 +02:00
Markus Heiser
66c8482725 [emacs] don't use tabs, except in makefile-gmake-mode 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 21:26:20 +02:00
Markus Heiser
73d73dcf9b [static] make /stats more CSP compliant- make themes.all 
Based on commits

- 0507e185 [fix] bar graph and rename CSS class engine-scores -> engine-score
- 3e9ad7ae [fix] make /stats more CSP compliant - github issue form
- 34859d0e [fix] make /stats more CSP compliant - oscar theme
- 0a6c4884 [fix] make /stats more CSP compliant - simple theme
- cdfb4b7f [fix] make /stats more CSP compliant - bar graph
- 965817f2 [fix] simple theme - generate missing sourceMap file

this patch is generated by::

     make themes.all

Reported-by: https://github.com/searxng/searxng/issues/57
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 21:25:32 +02:00
Markus Heiser
0507e185a5 [fix] bar graph and rename CSS class engine-scores -> engine-score 
- drop #main_stats selector in stats.less
- 'engine-score' exists before this PR.
- untabify searx/static/themes/__common__/less/stats.less

for details see comment at: d93bec7638..1204e4f07e (r633571496)

Suggested-by: @dalf in commit 1204e4f0
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 21:24:19 +02:00
Markus Heiser
3e9ad7ae0c [fix] make /stats more CSP compliant - github issue form 
Hide textarea from github issue form::

    ./__common__/new_issue.html:6:    <textarea name="body" style="display: none;">{{- '' -}}

BTW: fix indentation.

Reported-by: https://github.com/searxng/searxng/issues/57
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 21:23:27 +02:00
Markus Heiser
34859d0e58 [fix] make /stats more CSP compliant - oscar theme 
Replace oscar theme's *styles* (see below) by CSP compliant implementation in
``searx/static/themes/__common__/less/stats.less`` ::

    ./oscar/stats.html:29:  <th scope="col" style="width:20rem;">{{ th_sort('name', _("Engine name")) }}</th>
    ./oscar/stats.html:30:  <th scope="col" style="width:7rem; text-align: right;">{{ th_sort('score', _('Scores')) }}</th>
    ./oscar/stats.html:33:  <th scope="col" style="text-align: right;">{{ th_sort('reliability', _('Reliability')) }}</th>
    ./oscar/stats.html:38:  <td style="text-align: right;">
    ./oscar/stats.html:91:  <td style="text-align: right;"> {{ engine_reliabilities.get(engine_stat.name, {}).get('reliablity') }}</td>
    ./oscar/stats.html:109: <tbody style="padding-top: 1rem;">
    ./oscar/stats.html:112: <th scope="row" style="width: 10rem">{{ _('Exception') }}</th><td>{{ error.exception_classname }}</td>
    ./oscar/stats.html:114: <th scope="row" style="width: 10rem">{{ _('Message') }}</th><td>{{ error.log_message }}</td>
    ./oscar/stats.html:116: <th scope="row" style="width: 10rem">{{ _('Percentage') }}</th><td style="width: 10rem">{{ error.percentage }}</td>
    ./oscar/stats.html:121: <span style="border-right: 1px solid gray; padding: 0 1rem 0 0; margin: 0 0 0 0.5rem;">{{ param }}</span>
    ./oscar/stats.html:138: <th scope="col" style="width: 10rem">{{ _('Failed test') }}</th>

Reported-by: https://github.com/searxng/searxng/issues/57
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 17:41:22 +02:00
Markus Heiser
0a6c488417 [fix] make /stats more CSP compliant - simple theme 
Replace simple theme's *styles* (see below) by CSP compliant implementation in
``searx/static/themes/simple/less/stats.less`` ::

    ./simple/stats.html:26:  <table style="max-width: 1280px; margin: 0 auto 0 0;">
    ./simple/stats.html:28:  <th scope="col" style="width:20rem;">{{ th_sort('name', _("Engine name")) }}</th>
    ./simple/stats.html:29:  <th scope="col" style="width:7rem; text-align: right;">{{ th_sort('score', _('Scores')) }}</th>
    ./simple/stats.html:32:  <th scope="col" style="text-align: right;">{{ th_sort('reliability', _('Reliability')) }}</th>
    ./simple/stats.html:37:  <td style="text-align: right;">
    ./simple/stats.html:90:  <td style="text-align: right;"> {{ engine_reliabilities.get(engine_stat.name, {}).get('reliablity') }}</td>
    ./simple/stats.html:106: <table style="max-width: 1280px; margin: 1rem; border: 1px solid gray;">
    ./simple/stats.html:107: <tbody style="padding-top: 1rem;">
    ./simple/stats.html:110: <th scope="row" style="width: 10rem">{{ _('Exception') }}</th><td>{{ error.exception_classname }}</td>
    ./simple/stats.html:112: <th scope="row" style="width: 10rem">{{ _('Message') }}</th><td>{{ error.log_message }}</td>
    ./simple/stats.html:114: <th scope="row" style="width: 10rem">{{ _('Percentage') }}</th><td style="width: 10rem">{{ error.percentage }}</td>
    ./simple/stats.html:119: <span style="border-right: 1px solid gray; padding: 0 1rem 0 0; margin: 0 0 0 0.5rem;">{{ param }}</span>
    ./simple/stats.html:136: <th scope="col" style="width: 10rem">{{ _('Failed test') }}</th>

Reported-by: https://github.com/searxng/searxng/issues/57
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 17:41:22 +02:00
Markus Heiser
cdfb4b7ff9 [fix] make /stats more CSP compliant - bar graph 
Replace bar graph's *styles* (see below) by CSP compliant implementation in
``searx/static/themes/__common__/less/stats.less`` ::

    ./simple/stats.html:49: <span style="width: calc(max(2px, 100%*{{ (engine_stat.result_count / engine_stats.max_result_count )|round(3) }}))" class="stacked-bar-chart-serie1"></span>{{- "" -}}
    ./simple/stats.html:57: <span style="width: calc(max(2px, 100%*{{ (engine_stat.http / engine_stats.max_time )|round(3) }}))" class="stacked-bar-chart-serie1"></span>{{- "" -}}
    ./simple/stats.html:58: <span style="width: calc(100%*{{ engine_stat.processing / engine_stats.max_time |round(3) }})" class="stacked-bar-chart-serie2"></span>{{- "" -}}
    ./oscar/stats.html:50:  <span style="width: calc(max(2px, 100%*{{ (engine_stat.result_count / engine_stats.max_result_count )|round(3) }}))" class="stacked-bar-chart-serie1"></span>{{- "" -}}
    ./oscar/stats.html:58:  <span style="width: calc(max(2px, 100%*{{ (engine_stat.http / engine_stats.max_time )|round(3) }}))" class="stacked-bar-chart-serie1"></span>{{- "" -}}
    ./oscar/stats.html:59:  <span style="width: calc(100%*{{ engine_stat.processing / engine_stats.max_time |round(3) }})" class="stacked-bar-chart-serie2"></span>{{- "" -}}

Reported-by: https://github.com/searxng/searxng/issues/57
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 17:41:22 +02:00
Markus Heiser
965817f294 [fix] simple theme - generate missing sourceMap file 
C&P from searx/static/themes/oscar/gruntfile.js

Suggested-by: @dalf in commit 1204e4f0
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 17:41:22 +02:00
Markus Heiser
93594a7b42 [less] update grunt-contrib-less v3.3.0 
Upgraded [v3.3.0] otherwise::

`  width: calc(100% - 5rem);`

becomes `width: 95%` once compiled by less version 1.4.1.

[v3.3.0] https://github.com/gruntjs/grunt-contrib-less/releases/tag/v3.0.0

Suggested-by: @dalf in commit 1204e4f0
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 17:41:22 +02:00
Markus Heiser
57cdf180fe [fix] engine archive is - search_url has been changed 
BTW: set soft_max_redirects from *archive is* by 1, to prevent logging::

    DEBUG:httpx._client:HTTP Request: GET https://archive.is/search/?q=www.python.org "HTTP/2 302 Found"
    DEBUG:httpx._client:HTTP Request: GET https://archive.is/www.python.org "HTTP/2 200 OK"
    DEBUG:searx:archive is: ErrorContext('searx/search/processors/online.py', 110, 'count_error(self.engine_name,', None, '1 redirects, maximum: 0', ('200', 'OK', 'archive.is')) True

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 15:21:53 +02:00
Markus Heiser
ffcebf5e12 [enh] xpath engine - add request parameter 'soft_max_redirects' 
Make 'soft_max_redirects' configurable per Xpath engine::

    - name : <engine-name>
      engine : xpath
      soft_max_redirects: 1
      ...

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 15:04:55 +02:00
Markus Heiser
130e58f982 [brand] Dockerfile default INSTANCE_NAME=searxng 
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 12:02:24 +02:00
Markus Heiser
12b55320ea [doc] docker: revision and preparation of the documentation 
- add sidebars with addition infos about commands and docker in general
- fix long lines & indentation
- correct link to https://github.com/searxng/searxng-docker

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 12:02:24 +02:00
Markus Heiser
d6c321b830 [mod] docker-entrypoint.sh - add script to make test.sh (shellcheck) 
Add script docker-entrypoint.sh to shellcheck and try to _simplify_ and
_normalize_ some parts:

- fix issues reported by shellcheck
- don't mix tab and space indent
- command 'help' replaced by '-h': ./dockerfiles/docker-entrypoint.sh -h
- replace printf in help() by 'cat <<EOF'

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-17 12:02:24 +02:00
Markus Heiser
1970d28a60
Merge pull request #63 from searxng/dependabot/pip/master/pygments-2.9.0 
Bump pygments from 2.8.1 to 2.9.0
 2021-05-15 16:55:54 +00:00
dependabot[bot]
1b26bbcd8f
Bump pygments from 2.8.1 to 2.9.0 
Bumps [pygments](https://github.com/pygments/pygments) from 2.8.1 to 2.9.0.
- [Release notes](https://github.com/pygments/pygments/releases)
- [Changelog](https://github.com/pygments/pygments/blob/master/CHANGES)
- [Commits](https://github.com/pygments/pygments/compare/2.8.1...2.9.0)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-15 07:47:03 +00:00
Markus Heiser
7c21552b1b Revert "Merge pull request #72 from searxng/dependabot/pip/master/jinja2-3.0.0" 
This reverts commit 9e9a1ac6c3, reversing
changes made to 5b1f04fff0.

We can't upgrade to jinja2==3.0.0 until Sphinx 4.1.0 has been released [1][2]

ERROR: Cannot install -r /share/searx/requirements-dev.txt (line 11) and jinja2==3.0.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested jinja2==3.0.0
    sphinx 4.0.1 depends on Jinja2<3.0 and >=2.3

[1] https://github.com/sphinx-doc/sphinx/issues/9088#issuecomment-831020758
[2] https://github.com/sphinx-doc/sphinx/pull/9161

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-05-15 09:39:32 +02:00
Markus Heiser
606ddafc49
Merge pull request #73 from searxng/dependabot/pip/master/langdetect-1.0.9 
Bump langdetect from 1.0.8 to 1.0.9
 2021-05-15 07:10:56 +00:00
Markus Heiser
9e9a1ac6c3
Merge pull request #72 from searxng/dependabot/pip/master/jinja2-3.0.0 
Bump jinja2 from 2.11.3 to 3.0.0
 2021-05-15 06:52:19 +00:00
Markus Heiser
5b1f04fff0
Merge pull request #78 from return42/fix-docutils 
Revert "[fix] docutils v0.17 incompatibility to previeous v0.16"
 2021-05-14 16:21:06 +00:00
Markus Heiser
780b2ee8c6 Revert "[fix] docutils v0.17 incompatibility to previeous v0.16" 
This reverts commit 4557d58919.

see https://github.com/sphinx-doc/sphinx/issues/9088
 2021-05-14 18:14:16 +02:00
Markus Heiser
89a56a890c
Merge pull request #70 from searxng/dependabot/pip/master/sphinx-4.0.1 
Bump sphinx from 3.5.4 to 4.0.1
 2021-05-14 16:09:34 +00:00
dependabot[bot]
7562f3365b
Bump sphinx from 3.5.4 to 4.0.1 
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 3.5.4 to 4.0.1.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/4.x/CHANGES)
- [Commits](https://github.com/sphinx-doc/sphinx/compare/v3.5.4...v4.0.1)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-14 15:53:32 +00:00
Markus Heiser
776f2e0403
Merge pull request #75 from searxng/dependabot/pip/master/pallets-sphinx-themes-2.0.0 
Bump pallets-sphinx-themes from 1.2.3 to 2.0.0
 2021-05-14 15:52:49 +00:00
dependabot[bot]
d9d135e361
Bump pallets-sphinx-themes from 1.2.3 to 2.0.0 
Bumps [pallets-sphinx-themes](https://github.com/pallets/pallets-sphinx-themes) from 1.2.3 to 2.0.0.
- [Release notes](https://github.com/pallets/pallets-sphinx-themes/releases)
- [Changelog](https://github.com/pallets/pallets-sphinx-themes/blob/main/CHANGES.rst)
- [Commits](https://github.com/pallets/pallets-sphinx-themes/compare/1.2.3...2.0.0)

Signed-off-by: dependabot[bot] <support@github.com>
 2021-05-14 15:24:40 +00:00
Markus Heiser
7bfa0ea358
Merge pull request #76 from return42/fix-spinx-tabs 
[fix] sphinx-tabs should not overlap sidebar
 2021-05-14 15:23:05 +00:00