Commit Graph

4818 Commits

Author SHA1 Message Date
Alexandre Flament
7acd7ffc02 [enh] rewrite and enhance metrics 2021-04-21 16:24:46 +02:00
Alexandre Flament
aae7830d14 [mod] refactoring: processors
Report to the user suspended engines.

searx.search.processor.abstract:
* manages suspend time (per network).
* reports suspended time to the ResultContainer (method extend_container_if_suspended)
* adds the results to the ResultContainer (method extend_container)
* handles exceptions (method handle_exception)
2021-04-21 16:24:46 +02:00
Alexandre Flament
ae5954f2da Merge remote-tracking branch 'return42/fix-preference-save' 2021-04-21 16:24:22 +02:00
Alexandre Flament
624d3ec6db Merge remote-tracking branch 'return42/fix-url-bar-suggestion' 2021-04-21 16:23:52 +02:00
Alexandre Flament
92db0227b1 Merge remote-tracking branch 'dalf/oscar-images' 2021-04-19 09:36:54 +02:00
Alexandre Flament
48720e20a8 Merge remote-tracking branch 'searx/master' 2021-04-19 09:35:12 +02:00
Noémi Ványi
8362257b9a
Merge pull request #2736 from plague-doctor/sjp
Add new engine: SJP - Słownik języka polskiego
2021-04-16 17:30:14 +02:00
Noémi Ványi
e56323d3c8
Merge pull request #2759 from ypid/fix/typo
Fix grammar mistake in debug log output
2021-04-16 17:26:45 +02:00
Noémi Ványi
312a51566c
Merge pull request #2764 from mikamp116/patch-1
Fix bug for 'FileNotFoundError' in 'standalone_searx.py'
2021-04-16 17:26:21 +02:00
Noémi Ványi
59df3bec28
Merge pull request #2763 from return42/add-bandcamp
Add Bandcamp search engine
2021-04-16 17:25:59 +02:00
Plague Doctor
d275d7a35e Code refactoring. 2021-04-16 12:23:27 +10:00
Markus Heiser
f637bfc635 [mod] oscar's "default" template should make use of result.thumbnail
Some engine do have set result.img_src, other return a result.thumbnail.  If
result.img_src is unset and a result.thumbnail is given, show it to the UI.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-15 08:52:11 +02:00
Markus Heiser
062d589f86 [fix] xpath expressions to grap all items from bandcamp's response
I also found some items missing a thumbnail and I used text_extract for content
and title, to remove unneeded whitespaces.

BTW: added bandcamp's favicon

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-15 08:52:11 +02:00
Kyle Anthony Williams
4d3c399ee9 [feat] add bandcamp engine 2021-04-15 08:52:11 +02:00
Mikayel Mardanyan Petrosyan
4652ef0f06
Update standalone_searx.py
Fix bug for 'FileNotFoundError: [Errno 2] No such file or directory: 'utils/standalone_searx.py' ' in example to run standalone_searx.py from python
2021-04-13 22:26:45 +02:00
Alexandre Flament
01cefffbf6
Merge pull request #1 from metasearch-lab/httpx_networks
Httpx networks
2021-04-12 17:34:21 +02:00
Alexandre Flament
d14994dc73 [httpx] replace searx.poolrequests by searx.network
settings.yml:

* outgoing.networks:
   * can contains network definition
   * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections,
     keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries
   * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time)
   * local_addresses can be "192.168.0.1/24" (it supports IPv6)
   * support_ipv4 & support_ipv6: both True by default
     see https://github.com/searx/searx/pull/1034
* each engine can define a "network" section:
   * either a full network description
   * either reference an existing network

* all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
2021-04-12 17:25:56 +02:00
Markus Heiser
c4793afadc [fix] https-scheme missing in preferences-page
This patch is an addition to PR #2656 which removed all usage of `base_url` from
the templates, except one was forgotten in the cookie URL of the preferences.

closes: 2740

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-12 16:02:16 +02:00
Markus Heiser
2bf297b19f [fix] redirect when saving preferences
Erroneously commit 87e4c4762 droped the 302 redirect.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-12 15:29:08 +02:00
Robin Schneider
dfc66ff0f0
Fix grammar mistake in debug log output 2021-04-11 22:12:53 +02:00
Noémi Ványi
6c0114567e
Merge pull request #2744 from ColonisationCaptain/patch-1
Correct typo/grammatical mistake in documentation
2021-04-11 14:46:59 +02:00
Alexandre Flament
eaa694fb7d [enh] replace requests by httpx 2021-04-10 15:38:33 +02:00
Alexandre Flament
111180705b [fix] test: avoid HTTP requests
patch engine initialization to skip HTTP request
(engine_init function in searx.engines.initialize_engines)
2021-04-10 15:38:27 +02:00
ColonisationCaptain
b2093a8bc0
correct typo/grammatical mistake 2021-04-09 14:56:59 +01:00
Noémi Ványi
5d5ecdb745
Merge pull request #2717 from 3nprob/configure-config-paths
Allow overriding env vars SEARX_SETTINGS_PATH,UWSGI_SETTINGS_PATH
2021-04-09 12:48:06 +02:00
Plague Doctor
599ff39ddf Fix conflicts 2021-04-09 06:54:03 +10:00
Noémi Ványi
cc359345a8
Merge pull request #2735 from plague-doctor/wordnik
Add new engine: Wordnik.com
2021-04-08 19:48:13 +02:00
Noémi Ványi
a9a51ceb48
Merge pull request #2733 from dalf/fix-2656
SCRIPT_NAME remove trailing slash to avoid infinite redirect
2021-04-08 19:47:28 +02:00
Noémi Ványi
52e08ed777
Merge pull request #2741 from return42/fix-sphinx-theme
[fix] docutils v0.17 incompatibility to previeous v0.16
2021-04-08 19:44:45 +02:00
Markus Heiser
4557d58919 [fix] docutils v0.17 incompatibility to previeous v0.16
With docutils v0.17 a lot of html markup has been changed (see below) what cause
a lot of problems in CSS from Sphinx and other Sphinx extensions & customizing.

For the first this fix pins to previous v0.16. In sphinx 4.0 these problems will
be addressed [2] and we can relax (drop) in the requirements-dev.

HTML5 writer [1]:

  Use the new semantic tags <main>, <section>, <header>, <footer>, <aside>,
  <figure>, and <figcaption>.  See minimal.css and plain.css for styling rule
  examples.

  Change the initial_header_level setting's default to "2", as browsers use the
  same style for <h1> and <h2> when nested in a section.

  Use HTML text-level tags <small>, <s>, <q>, <dfn>, <var>, <samp>, <kbd>, <i>,
  <b>, <u>, <mark>, and <bdi> if a matching class value is found in inline and
  literal elements.  Use <ins> and <del> if a matching class value is found in
  inline, literal, or container elements.

  New optional style responsive.css, adapts to different screen sizes.

  New option embed_images.

[1] https://docutils.sourceforge.io/RELEASE-NOTES.html
[2] https://github.com/sphinx-doc/sphinx/issues/9056

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-08 14:47:24 +02:00
3nprob
0fb423ea59 Allow overriding env vars SEARX_SETTINGS_PATH,UWSGI_SETTINGS_PATH 2021-04-08 12:56:15 +09:00
Plague Doctor
6631f11305 Add new engine: SJP 2021-04-08 10:21:54 +10:00
Plague Doctor
7035bed4ee Add new engine: Wordnik.com 2021-04-08 09:58:00 +10:00
Noémi Ványi
547478089f Add blogpost about Elasticsearch, Meilisearch and Solr 2021-04-07 23:21:00 +02:00
Alexandre Flament
c09ff4faf2 [fix] fix PR 2656
SCRIPT_NAME remove trailing slash to avoid infinite redirect
2021-04-07 13:05:55 +02:00
Noémi Ványi
07f5edce3d Add Meilisearch engine
Website: https://www.meilisearch.com/
2021-04-06 21:57:05 +02:00
Noémi Ványi
dd2b106f94
Merge pull request #2668 from searx/dependabot/pip/master/pycodestyle-2.7.0
Bump pycodestyle from 2.6.0 to 2.7.0
2021-04-05 21:39:36 +02:00
dependabot[bot]
4f869921f2
Bump pycodestyle from 2.6.0 to 2.7.0
Bumps [pycodestyle](https://github.com/PyCQA/pycodestyle) from 2.6.0 to 2.7.0.
- [Release notes](https://github.com/PyCQA/pycodestyle/releases)
- [Changelog](https://github.com/PyCQA/pycodestyle/blob/master/CHANGES.txt)
- [Commits](https://github.com/PyCQA/pycodestyle/compare/2.6.0...2.7.0)

Signed-off-by: dependabot[bot] <support@github.com>
2021-04-05 19:36:05 +00:00
Noémi Ványi
a477a3a687
Merge pull request #2728 from return42/fix-loader
[fix] settings_loader.py - use update_dict only for mapping types
2021-04-05 21:32:41 +02:00
Noémi Ványi
b40af000f3
Merge pull request #2726 from 3nprob/custom-docker-repository
Allow overriding Docker repository when building docker image
2021-04-05 20:48:24 +02:00
Noémi Ványi
736d0656bb
Merge pull request #2723 from 3nprob/clean-hubspot-tracking-urls
Remove hubsbpot tracking URL params

More information here:
https://knowledge.hubspot.com/settings/how-do-i-create-a-tracking-url
https://knowledge.hubspot.com/ads/ad-tracking-in-hubspot
https://knowledge.hubspot.com/ctas/calls-to-action-frequently-asked-questions
https://meta.stackexchange.com/questions/263392/what-are-these-very-long-parameters-in-so-careers-feedback-always-welcome-url
2021-04-05 20:46:13 +02:00
3nprob
2ca0aa4f29 Remove hubsbpot tracking URL params
More information here:
https://knowledge.hubspot.com/settings/how-do-i-create-a-tracking-url
https://knowledge.hubspot.com/ads/ad-tracking-in-hubspot
https://knowledge.hubspot.com/ctas/calls-to-action-frequently-asked-questions
https://meta.stackexchange.com/questions/263392/what-are-these-very-long-parameters-in-so-careers-feedback-always-welcome-url
2021-04-06 02:11:09 +09:00
Adam Tauber
a20141d697 [doc] update authors file 2021-04-05 19:00:06 +02:00
Adam Tauber
ea3eda2640 [doc] update authors file
We would like to thank him for all of his work and we would like to wish
him good luck in his future endeavors.
2021-04-05 19:00:06 +02:00
Noémi Ványi
647c3fb4a5
Merge pull request #2725 from 3nprob/optimize-docker-build
Reduce redundant docker build steps
2021-04-05 18:55:30 +02:00
Markus Heiser
9c10b15096 [fix] settings_loader.py - use update_dict only for mapping types
I can't set `default_doi_resolver` in `settings.yml` if I'm using
`use_default_settings`.  Searx seems to try to interpret all settings at root
level in `settings.yml` as dict, which is correct except for
`default_doi_resolver` which is at root level and a string::

    File "/usr/lib/python3.9/site-packages/searx/settings_loader.py", line 125, in load_settings
        update_settings(default_settings, user_settings)
    File "/usr/lib/python3.9/site-packages/searx/settings_loader.py", line 61, in update_settings
        update_dict(default_settings[k], v)
    File "/usr/lib/python3.9/site-packages/searx/settings_loader.py", line 48, in update_dict
        for k, v in user_dict.items():
    AttributeError: 'str' object has no attribute 'items'

Signed-off-by: Markus Heiser <markus@darmarit.de>
Suggested-by:  @0xhtml https://github.com/searx/searx/issues/2722#issuecomment-813391659
2021-04-05 16:33:48 +02:00
Alexandre Flament
7089526723
Merge pull request #2656 from return42/fix-url_for
[fix] url_for(..., _external=True) in templates
2021-04-05 14:50:39 +02:00
Markus Heiser
87e4c47621 [fix] url_for(..., _external=True) in templates
The `url_for` function in the template context is not the one from Flask, it is
the one from `webapp`.  The `webapp.url_for_theme` is different from its
namesake of Flask and has it quirks, when called with argument `_external=True`.

The `webapp.url_for_theme` can't handle absolute URLs since it pokes a leading
'/', here is the snippet of the old code::

    url = url_for(endpoint, **values)
    if settings['server']['base_url']:
        if url.startswith('/'):
            url = url[1:]
        url = urljoin(settings['server']['base_url'], url)

Next drawback of (Flask's) `_external=True` is, that it will not return the HTTP
scheme when searx (the Flask app) listens on http and is proxied by a https
server.

To get the right scheme `HTTP_X_SCHEME` is needed by Flask (werkzeug).  Since
this is not provided in every environment (e.g. behind Apache mod_wsgi or the
HTTP header is not fully set for some other reasons) it is recommended to
get *script_name*, *server* and *scheme* from the configured `base_url`.  If
`base_url` is specified, then these values from are given preference over any
Flask's generics.

BTW this patch normalize to use `url_for` in the `opensearch.xml` and drop the
need of `host` and `urljoin` in template's context.

Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-05 14:34:45 +02:00
Alexandre Flament
7a0fbdecc4 [enh] oscar: image thumbnail layout
Adjust thumbnail sizes to fill the container width
2021-04-05 11:29:17 +02:00
3nprob
c819ae4d8c Allow overriding Docker repository when building docker image 2021-04-05 15:53:52 +09:00