Commit Graph

5692 Commits

Author SHA1 Message Date
Markus Heiser 2ac3e5b20b [fix] log messages from: google- images, news, scholar, videos
- HTTP header Accept-Language --> lang_info['headers']['Accept-Language']
- remove obsolete query_url log messages which is already logged by
  httpx._client:HTTP request

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-11 16:31:50 +02:00
Markus Heiser 1ac3961336 [mod] google - get_lang_info add documentataion & comments
BTW: remove obsolete log messages from google engine

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-11 16:06:36 +02:00
Alexandre Flament 1c67b6aece [enh] google engine: supports "default language"
Same behaviour behaviour than Whoogle [1].  Only the google engine with the
"Default language" choice "(all)"" is changed by this patch.

When searching for a locate place, the result are in the expect language,
without missing results [2]:

  > When a language is not specified, the language interpretation is left up to
  > Google to decide how the search results should be delivered.

The query parameters are copied from Whoogle.  With the ``all`` language:

- add parameter ``source=lnt``
- don't use parameter ``lr``
- don't add a ``Accept-Language`` HTTP header.

The new signature of function ``get_lang_info()`` is:

    lang_info = get_lang_info(params, lang_list, custom_aliases, supported_any_language)

Argument ``supported_any_language`` is True for google.py and False for the other
google engines.  With this patch the function now returns:

- query parameters: ``lang_info['params']``
- HTTP headers: ``lang_info['headers']``
- and as before this patch:
  - ``lang_info['subdomain']``
  - ``lang_info['country']``
  - ``lang_info['language']``

[1] https://github.com/benbusby/whoogle-search
[2] https://github.com/benbusby/whoogle-search/releases/tag/v0.5.4
2021-06-10 10:22:01 +02:00
Markus Heiser d0daa87ce6 [coding-style] searx/webapp.py - normalize indentations
No functional change!

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-10 09:35:00 +02:00
Alexandre Flament 73dbee45a6
Merge pull request #90 from searxng/enh-osm-engine
[enh] improve openstreetmap engine
2021-06-10 09:30:41 +02:00
Markus Heiser 15178e5b96 [static] make themes.all - based on commit 12af3b39
Based on commit 12af3b39 this patch is generated by::

    make themes.all

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-10 09:20:51 +02:00
Alexandre Flament 68df982e21
Merge pull request #135 from searxng/refactor-webapp
Refactor webapp
2021-06-10 08:11:15 +02:00
Alexandre Flament 314e09091b
Merge pull request #138 from return42/fix-doc
[fix] section "Overrides" in engine's documentation
2021-06-10 08:10:50 +02:00
Markus Heiser 12af3b39db [fix] make simple/result_templates/map.html more CSP compliant [1]
[1] https://github.com/searxng/searxng/issues/57

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-09 18:08:23 +02:00
Markus Heiser bf10b4a857 [fix] openstreetmap - fix some minor whitespace & indentation issues
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-09 18:08:23 +02:00
Markus Heiser 40956e8e6b [data.all] add searx_extra/update/update_osm_keys_tags.py
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-09 18:08:23 +02:00
Markus Heiser 8871e39122 [pylint] searx_extra/update/update_osm_keys_tags.py
BTW: move some comments into script's  doc-string

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-09 18:08:23 +02:00
Markus Heiser 5cf1ae2672 [pylint] searx/data/__init__.py
BTW: add doc strings and moved __all__ to the top [1]

[1] https://www.python.org/dev/peps/pep-0008/#module-level-dunder-names

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-09 18:08:23 +02:00
Alexandre Flament 2e5d823162 [fix] searx/data/__init__.py: rename __init__ as __all__ 2021-06-09 18:08:23 +02:00
Alexandre Flament c75425655f [enh] openstreetmap / map template: improve results
implements ideas described in #69

* update the engine
* use wikidata
* update map.html template
2021-06-09 18:08:23 +02:00
Alexandre Flament 92c8a8829f [fix] strip spaces from searx user agent
h11 (used by httpx) rejects HTTP request with a trailing space in HTTP headers
2021-06-09 18:08:23 +02:00
Markus Heiser ab259017cc [fix] section "Overrides" in engine's documentation
[1] https://github.com/searxng/searxng/pull/97#discussion_r646094256

Suggested-by: @dalf [1]
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-09 15:16:59 +02:00
Alexandre Flament 2a109d0a5f [mod] refactor searx.webapp.render
render automatically adds these variables to the template context:
* advanced_search
* all_categories
* categories
	before render was checking if the variable was already set
	but it is actually never set by the callers
2021-06-08 15:54:11 +02:00
Alexandre Flament 70bbd14b1a [mod] move hook to set Unix thread name into searx.unixthreadname
requires setproctitle (but optional)
2021-06-08 15:54:11 +02:00
Alexandre Flament 0e23113f70 [mod] remove unused filter searx.webapp.extract_domain
it was used in the pix-art theme
2021-06-08 15:54:11 +02:00
Alexandre Flament cfa07b0a9a [mod] refactor searx.webapp.render
no functional change, reorder declarations
2021-06-08 15:54:11 +02:00
Alexandre Flament 91856e8f6a [mod] refactor searx.webapp.render: remove unused parameters 2021-06-08 15:54:11 +02:00
Alexandre Flament 15567955b4 [mod] remove check of Python 2. 2021-06-08 15:54:11 +02:00
Alexandre Flament aedf0aa5fe [mod] remove searx.webapp.get_base_url function
see the result of:
git grep "base_url" searx/templates
"base_url" is not used any more.
2021-06-08 15:54:11 +02:00
Alexandre Flament eeb0998787 [mod] refactor: move Flask proxy fix to searx.flaskfix module 2021-06-08 15:54:11 +02:00
Markus Heiser 5c5db719d2
Merge pull request #97 from return42/drop-searx-admin
[docs] reorder blog articles
2021-06-08 10:56:18 +00:00
Alexandre Flament 5f76238d5c
Merge pull request #105 from return42/yaml
[enh] add test.yamllint - lint yaml files
2021-06-05 20:36:03 +02:00
Markus Heiser 6b9633098d [emacs] flycheck - activate yamllint and use .yamllint.yml
Open YAML file and use:

  M-x flycheck-mode

Since we use yamllint from local/py3/bin, creating a virtualenv is a
prerequisite:

  make pyenv

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-05 17:42:06 +02:00
Markus Heiser 5a7b12ee6f [yamllint] searx/settings.yml
Fixed messages reported by::

    make test.yamllint

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-05 17:41:24 +02:00
Markus Heiser 4e845966ba [yamllint] searx/settings_robot.yml
Fixed messages reported by::

    make test.yamllint

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-05 17:41:24 +02:00
Markus Heiser b86a3f6303 [yamllint] tests/unit/settings/user_settings_remove2.yml
Fixed messages reported by::

    make test.yamllint

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-05 17:41:24 +02:00
Markus Heiser 008760a8c2 [yamllint] tests/unit/settings/user_settings_remove.yml
Fixed messages reported by::

    make test.yamllint

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-05 17:41:24 +02:00
Markus Heiser 15da3f6591 [yamllint] tests/unit/settings/user_settings_simple.yml
Fixed messages reported by::

    make test.yamllint

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-05 17:41:24 +02:00
Markus Heiser 00b10bb443 [yamllint] tests/unit/settings/user_settings_keep_only.yml
Fixed messages reported by::

    make test.yamllint

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-05 17:41:24 +02:00
Markus Heiser c5d3f542d5 [yamllint] tests/unit/settings/user_settings.yml
Fixed messages reported by::

    make test.yamllint

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-05 17:41:24 +02:00
Markus Heiser 68213f6be4 [yamllint] tests/unit/settings/syntaxerror_settings.yml
Fixed messages reported by::

    make test.yamllint

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-05 17:41:24 +02:00
Markus Heiser 393c06f537 [enh] add test.yamllint - lint yaml files
Usage::

    make test.yamllint

    ./manage test.yamllint

test.yamllint is also added to the `test` makefile target.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-05 17:41:24 +02:00
Markus Heiser fb6dd416e5
Merge pull request #131 from searxng/fix-peertube-languages
Fix peertube languages
2021-06-04 14:26:22 +00:00
Alexandre Flament cf2b549967 Update engines_languages.json 2021-06-04 16:17:22 +02:00
Alexandre Flament 8194db4e21 [fix] peertube fetch supported languages
close #127
2021-06-04 16:17:20 +02:00
Markus Heiser 6f7b0d72c0 [docs] revision of the section 'Command Line Engines'
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:05:58 +02:00
Markus Heiser 5bea04869d [docs] move LXC blog article into developer documentation.
Move article 'Developing in Linux Containers' from blog section do developer
section.  Since there are no more articles in the blog section, remove the
section completely.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:05:58 +02:00
Markus Heiser c5ccd50ffa [docs] revision of the blog article about local search engines
The blog article 'Query your local search engines' has been renamed 'Local
Search Engines', revised and moved into admin's chapter 'Engine & Settings'.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:05:58 +02:00
Markus Heiser 27e5a7437a [docs] rename 'Recoll' to 'Recoll Engine'
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:05:58 +02:00
Markus Heiser e34dc7818f [docs] move 'settings.yml' into admin's 'Engines & Settings'
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:05:58 +02:00
Markus Heiser 340d25b19b [docs] rename 'General Engine Settings' to 'Configured Engines'
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:05:58 +02:00
Markus Heiser c647c8b7a5 [docs] revision of the article "settings.yml"
This patch is a marginal revision of the article "settings.yml", most changes
are from normalizing the YAML syntax.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:05:58 +02:00
Markus Heiser 5caebb379f [docs] revision of the article "Engine Overview"
This patch revision of the article "Engine Overview":

- add links & anchors
- improve formating of the tables

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:05:58 +02:00
Markus Heiser f122cb0e27 [fix] typo: online_dictionnary --> online_dictionary
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:05:58 +02:00
Markus Heiser 97d801e8a0 [fix] docs/conf.py - docutils docs moved to docutils.sourceforge.io
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-04 15:05:58 +02:00