Commit Graph

3566 Commits

Author SHA1 Message Date
Markus Heiser 1a0760c10a [fix] googel engine - "some results are invalids: invalid content"
Fix google issues listet in the `/stats?engine=google` and message::

    some results are invalids: invalid content

The log is::

    DEBUG   searx                         : result: invalid content: {'url': 'https://de.wikipedia.org/wiki/Foo', 'title': 'Foo - Wikipedia', 'content': None, 'engine': 'google'}
    WARNING searx.engines.google          : ErrorContext('searx/search/processors/abstract.py', 111, 'result_container.extend(self.engine_name, search_results)', None, 'some results are invalids: invalid content', ()) True

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-18 13:23:35 +01:00
Markus Heiser f0102a95c9 [fix] google engine: remove adds and fix mobile_ui selector
1. Fix issue reported in comment [1]
2. Fix XPath selector for the response of google's mobile UI, reported in
   comment [2]

[1] https://github.com/searxng/searxng/pull/777#issuecomment-1015121322
[2] https://github.com/searxng/searxng/pull/777#issuecomment-1015236238

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-18 11:05:45 +01:00
Alexandre Flament b972df255a
Merge pull request #776 from mrpaulblack/simple-ltr-vars
[simple theme] dont hardcode border radius
2022-01-18 07:56:03 +01:00
Émilien Devos 6670063e0d
Update XPath for Google engine 2022-01-17 21:49:57 +00:00
mrpaulblack 6c1a64d4b3 [build] /static 2022-01-17 22:35:22 +01:00
mrpaulblack 7a0f5e6b19 [simple theme] dynamic border radius in CSS 2022-01-17 22:35:02 +01:00
Alexandre Flament 0c036ae294
Merge pull request #750 from dalf/simple-rtl
Simple theme: implement RTL
2022-01-16 20:29:34 +01:00
Alexandre Flament e07417848f
Merge pull request #695 from return42/fix-sp
[fix] startpage engine / modified API
2022-01-16 20:27:36 +01:00
Alexandre Flament 468619abcb [build] /static 2022-01-16 18:52:46 +01:00
Alexandre Flament d000288ad0 [enh] simple theme: RTL support
* mirror all inline SVGs so that direction SVGs display correctly on RTL
* set the bold list element in info box to RTL so the colon gets displayed on the right side
* set correct .ltr function for the left border on the search button in #q
* move text to the right in autocomplete
* move search form in lign with result article on RTL
* add the correct padding for img thumbnails in categories like music on RTL
* apply RTL to result table for map results
* align text in tables part of /preferences on RTL
* move burger menu on index page to the left on RTL
* fix positioning of drop down arrow on select boxes on RTL
* align result URL on the right (written LTR)
* align vim hotkeys help on the left since it is not translated
* image detail:
  * labels (author, format, URL, etc...) are written on the right,
    values are on the left.
  * URL are written LTR and overflow on the right
2022-01-16 18:51:11 +01:00
Alexandre Flament 2084d7b1ed [mod] simple theme: change stylelint configuration
* disable declaration-empty-line-before
  https://stylelint.io/user-guide/rules/list/declaration-empty-line-before/
  this change allows to mix CSS declarations and LESS mixins without empty lines:

  #something {
    display: flex;
    .ltr-left(60rem); // no mandatory empty line before this one
  }

* disable no-invalid-position-at-import-rule
  https://stylelint.io/user-guide/rules/list/no-invalid-position-at-import-rule/

  this change allows to declare some mixins and then import another .less file:
  for example:

  .ltr-left(@offset) {
    left: @offset;
  }
  @import "style.less";
2022-01-16 18:50:19 +01:00
Alexandre Flament f9271d595f [fix] startpage: workaround to use the startpage network
workaround for the issue #762
2022-01-15 22:56:34 +01:00
Alexandre Flament 32874dbf7f
Merge pull request #742 from return42/mysql
[mod] engine mysql_server: make port configurable
2022-01-15 21:35:34 +01:00
Martin Fischer 599d882d0a
Merge pull request #754 from return42/fix-751
[theme] /preferences: rename "Method" to "HTTP Method"
2022-01-14 18:42:49 +01:00
Markus Heiser 580815a9a5 [fix] stop less grunt runner on missing files
The less grunt runner silently ignore missing files and continue with the build[1]::

    Running "less:production" (less) task
    >> Destination css/searxng.min.css not written because no source files were found.
    >> 1 stylesheet created.
    >> 1 sourcemap created.

Add filter function that calls grunt.fail() if the scr file does not exists.

[1] https://github.com/searxng/searxng/pull/750#discussion_r784357031
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-14 13:16:24 +01:00
Markus Heiser 08b38a8bfd [fix] simple theme: description of HTTP method
Suggested-by: @not-my-profile https://github.com/searxng/searxng/pull/754#issuecomment-1012921096
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-14 10:07:09 +01:00
Markus Heiser e45dfb334b [theme] /preferences: rename "Method" to "HTTP Method"
Closes: https://github.com/searxng/searxng/issues/751
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-14 08:29:04 +01:00
searxng-bot ce018d013b [translations] update from Weblate
0ca15e28 - 2022-01-11 - Allan Nordhøy <epost@anotheragency.no>
dd494ce7 - 2022-01-11 - Markus Heiser <markus.heiser@darmarit.de>
7e4e914a - 2022-01-10 - k2s <martin.minka@gmail.com>
b4915a2f - 2022-01-10 - Markus Heiser <markus.heiser@darmarit.de>
832d5cf9 - 2022-01-13 - Genghis Khan <genghiskhan@gmx.ca>
546ad69a - 2022-01-08 - Alexandre Flament <alex@al-f.net>
f14bd241 - 2022-01-09 - Markus Heiser <markus.heiser@darmarit.de>
2815009a - 2022-01-07 - Markus Heiser <markus.heiser@darmarit.de>
2022-01-14 07:17:46 +00:00
Martin Fischer d9709df740 [fix] make BooleanChoices only set cookies that vary from the default
The bug was inadvertently introduced by the refactor in
180d4d068b.

Fixes #746.
2022-01-13 19:43:49 +01:00
Alexandre Flament 687bdef410
Merge pull request #740 from return42/fix-bang
[fix] get_bang_url: handle ambiguous !!bangs without error
2022-01-12 22:00:18 +01:00
Markus Heiser a7c3c352d8 [upd] searx/data/external_bangs.json
Updated external bangs by::

    ./manage pyenv.cmd ./searxng_extra/update/update_external_bangs.py

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-12 19:37:13 +01:00
Markus Heiser 7cdd31440e [fix] external bangs: don't overwrite Bangs in data trie
Bangs with a `*` suffix (e.g. `!!d*`) overwrite Bangs with the same
prefix (e.g. `!!d`) [1].  This can be avoid when a non printable character is
used to tag a LEAF_KEY.

[1] https://github.com/searxng/searxng/pull/740#issuecomment-1010411888

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-12 19:37:13 +01:00
Markus Heiser 50c4b58db6 [hotfix] interim fix to get docker-build of CI without issues
There is an issue with redis v4.1.0 [1] / for the interim lets remove this
python dependency.

[1] https://github.com/searxng/searxng/issues/741

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-12 09:31:12 +01:00
Martin Fischer cd8ce22f86 [themes] add comma before "but" in new sentence
Closes #739.
2022-01-12 08:26:23 +01:00
Markus Heiser bf593af423 [mod] engine mysql_server: make port configurable
Cherry piked from https://github.com/searx/searx/commit/82ac634070

Suggested-by: https://github.com/searx/searx/issues/3117
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-11 23:47:40 +01:00
Markus Heiser 977e9a4330
Merge pull request #686 from return42/lib_redis
Add redis DB and connector
2022-01-11 19:55:14 +01:00
Markus Heiser 6d7e86eece [fix] get_bang_url: handle ambiguous !!bangs without error
An ambiguous bang like `!!d` raises an exception in function get_bang_url().  A
bang is only unique when the bang_definition from get_bang_definition_and_ac() is
a string / for a ambiguous bang the returned bang_definition is a dictionary.

Reported-by: user prg at #searxng:matrix.org on 2022/01/11
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-11 09:58:10 +01:00
Markus Heiser 3457408285 [build] /static 2022-01-10 17:42:13 +01:00
Markus Heiser 85159ad307 [fix] autocomple.js: register li.onmousedown instead li.onclick
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-10 17:30:57 +01:00
Markus Heiser 6420322cd1
Merge pull request #729 from k2s/fix-tokens
fix(security): filtered_engines were not correctly evaluating validate_token()
2022-01-10 11:51:45 +01:00
Markus Heiser df238e944c [mod] starpage engine: add comment about Startpage's FFox add-on
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-10 11:22:38 +01:00
Markus Heiser 21e884f369 [fix] startpage engine: fetch CAPTCHA & issues related to PR-695
In case of CAPTCHA raise a SearxEngineCaptchaException and suspend for 7 days.
When get_sc_code() fails raise a SearxEngineResponseException and suspend for 7
days.

[1] https://github.com/searxng/searxng/pull/695

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-10 11:22:38 +01:00
Markus Heiser 2f4e567e90 [fix] Get an actual `sc` argument from startpage's home page.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-10 11:22:38 +01:00
Markus Heiser 1cbcddb3f7 [pylint] Startpage engine
Fix remarks from pylint

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-10 11:22:38 +01:00
Markus Heiser f1f5e69c42 [fix] startpage engine - avoid captcha
Startpage has introduced new anti-scraping measures that make SearXNG instances
run into captchas:

1. some arguments has been removed and a new `sc` has been added.
2. search path changed from `do/search` to `sp/search`
3. POST request is no longer needed

Closes: https://github.com/searxng/searxng/issues/692
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-10 11:22:12 +01:00
Martin Fischer 576e19dad1 [fix] add default for "about" engine property
Fixes #732.
2022-01-10 08:40:06 +01:00
Martin Minka b76a815827 [fix] filtered_engines were not correctly evaluating validate_token() 2022-01-09 15:53:58 +01:00
Markus Heiser dca83944b5 [fix] redis: don't create a new connection at each client() call
Suggested-by: @dalf https://github.com/searxng/searxng/pull/686#pullrequestreview-844942973
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-07 17:33:40 +01:00
Markus Heiser 4fc5e5299c [fix] ccengine engine - avoid unwanted redirects
api.openverse.engineering is a little picky and wants to have a trailing slash
in the path:

    /v1/images? -->/ v1/images/?

otherwise it redirects, here is the debug log:

    DEBUG   searx.network.openverse       : HTTP Request: GET https://api.openverse.engineering/v1/images?&page=1&page_size=20&format=json&q=foo "HTTP/2 301 Moved Permanently" (text/html; charset=utf-8)
    DEBUG   searx.network.openverse       : HTTP Request: GET https://api.openverse.engineering/v1/images/?&page=1&page_size=20&format=json&q=foo "HTTP/2 200 OK" (application/json)
    WARNING searx.engines.openverse       : ErrorContext('searx/search/processors/online.py', 105, 'count_error(', None, '1 redirects, maximum: 0', ('200', 'OK', 'api.openverse.engineering')) True

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-07 14:14:31 +01:00
Markus Heiser 1461d6a07e [upd] engine_descriptions.json
Updated by:

    ./manage pyenv.cmd python3 \
        ./searxng_extra/update/update_engine_descriptions.py

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-07 13:59:03 +01:00
Léon Tiekötter 37baf46ece [fix] Rename ccengine engine to openverse
The CC engine was merged with WordPress and renamed to Openverse

Source: https://wordpress.org/news/2021/05/welcome-to-openverse/
2022-01-07 13:06:05 +01:00
Léon Tiekötter 4be6deb0a1 [fix] ccengine engine
Change domain to api.openverse.engineering
2022-01-07 13:01:37 +01:00
Markus Heiser ced656606f
Merge pull request #709 from return42/drop-etools
[fix] drop etools engine module
2022-01-07 11:18:47 +01:00
Markus Heiser 5dd3442f83 [fix] drop etools engine module
The implementation of the etools engine is poor.  No date-range support, no
language support and it is broken by a CAPTCHA.

etools is a metasearch engine, the major search engines it supports (google,
bing, wikipedia, Yahoo) are already available in SeaarXNG.

While etools does support several engines we currently don't support directly,
support for them should be added directly to SearXNG if there is demand.

In practice: in SearXNG the worse etools results will be mixed with good results
from other engines we have (as long as there is no captcha).

At best case, what we win with etools is in e.g. results from de.ask.com in a
query from a german request .. in all other cases worse results are bubble up in
SearXNG's result list.

[1] https://github.com/searxng/searxng/issues/696#issuecomment-1005855499

Closes: https://github.com/searxng/searxng/issues/696
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-07 10:41:09 +01:00
Martin Fischer e12525a1fa
Merge pull request #708 from not-my-profile/pref-refactor
Refactor `preferences`
2022-01-07 09:45:23 +01:00
Markus Heiser 2eb95f1532
Merge pull request #718 from searxng/dependabot/pip/master/pygments-2.11.2
Bump pygments from 2.11.0 to 2.11.2
2022-01-07 08:40:55 +01:00
Markus Heiser c956ed6c43 [build] /static 2022-01-07 08:36:32 +01:00
Markus Heiser d8db85638e make pygments.less
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-07 08:35:14 +01:00
searxng-bot 651342d71a [translations] update from Weblate
7781625e - 2022-01-06 - Linerly <linerly@protonmail.com>
dcab2ef1 - 2022-01-06 - Markus Heiser <markus.heiser@darmarit.de>
2022-01-07 07:17:58 +00:00
Léon Tiekötter 3ab826de22 Drop microsoft academic engine
Microsoft academic was discontinued on 2021-12-31.

Source: https://www.microsoft.com/en-us/research/project/academic/articles/microsoft-academic-to-expand-horizons-with-community-driven-approach/
2022-01-07 01:35:13 +01:00