Commit Graph

4764 Commits

Author SHA1 Message Date
Markus Heiser 4cb8cbc6c3 [fix] duckduckgo lite engine: set HTTP header 'Referer'
We have had problems with this before, the bot protection from ddg-lite seems to
have included this referer in the rating [1][2].

From reverse engineering:

- The Referer ``https://google.com/`` was set in commt 257dc7d6c4 --> DDG lite
  does not like this referer anymore!

- The 'Referer' header is only set on second and follow up pages but not on the
  first page

- The vqd value is not needed on the first page, the ddg-lite client sets this
  value only on follow up pages / this can help to reduce the vqd requests from
  SearXNG.

Related to 'Referer' header & ddg requests:

[1] https://github.com/searxng/searxng/pull/2161
[2] https://github.com/searxng/searxng/pull/2081

Closes: https://github.com/searxng/searxng/issues/2796
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-19 18:04:53 +02:00
Bnyro 16ce63b8a2 [mod] yacy: use official instance by default and fix crashes 2023-10-19 18:04:53 +02:00
Alex Balgavy c2f26a560a [mod] add hotkeys option to settings.yml
The change in the hotkey mechanism introduced in 317db5b04 does not allow
configuration via `settings.yml`.  This commit adds that functionality.

Closes: #2898
2023-10-19 18:04:53 +02:00
Hackurei ce6b7c9f75 [feat] implement hackernews engine - news.ycombinator.com 2023-10-19 18:04:53 +02:00
Aine fa830b375b [fix] matrixrooms add proper MRS integration
Related:

- https://github.com/searxng/searxng/issues/2918
2023-10-19 18:04:53 +02:00
Bnyro 34d0d2c9ab [feat] duckduckgo: support for videos and news 2023-10-19 18:04:53 +02:00
Bnyro 84b4932e21 [fix] kickass: crash when no results 2023-10-19 18:04:53 +02:00
Bnyro 057ae1767b [mod] piped: always show video length if available 2023-10-19 18:04:53 +02:00
Bnyro a742cfe2e7 [feat] engine: implementation of mastodon 2023-10-19 18:04:53 +02:00
searxng-bot b6cae7d53f [translations] update from Weblate
68d743281 - 2023-10-05 - return42 <markus.heiser@darmarit.de>
42f091b7f - 2023-10-05 - return42 <markus.heiser@darmarit.de>
2479c0d7b - 2023-10-05 - ghose <correo@xmgz.eu>
a4e6cd592 - 2023-10-05 - return42 <markus.heiser@darmarit.de>
9d4e5f5c3 - 2023-10-05 - return42 <markus.heiser@darmarit.de>
b79d44775 - 2023-10-05 - gallegonovato <fran-carro@hotmail.es>
746291184 - 2023-10-06 - return42 <markus.heiser@darmarit.de>
f24d7e8b1 - 2023-10-05 - return42 <markus.heiser@darmarit.de>
6140911f9 - 2023-10-05 - Fjuro <ifjuro@proton.me>
2023-10-19 18:04:53 +02:00
Markus Heiser ba5f9bd4ac [mod] engine - simplify region & lang handling, make filters configurable
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-19 18:04:53 +02:00
Bnyro 70f6a84aff [feat] engine: implementation of radio-browser.info 2023-10-19 18:04:53 +02:00
Markus Heiser 4de856bf19 [fix] limiter / botdetection: remove http_connection method
Related:

- https://github.com/searxng/searxng/issues/2892#issuecomment-1742153932

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-19 18:04:53 +02:00
Jinyuan Huang 993ea9c1ec [typo] solved a typo in yahoo error message. 2023-10-19 18:04:53 +02:00
Jinyuan Huang 7de4e3b48f [fix] Bug: Yahoo results for simplified Chinese search sometimes have the first character cut off #2866
Co-authored-by: Blair Noctis <n@sail.ng>
2023-10-19 18:04:53 +02:00
Bnyro f6ee574508 [fix] emojipedia: fix engine 2023-10-19 18:04:52 +02:00
Markus Heiser f4f2af7db6 [fix] Revision of the Bing engines
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-19 18:04:52 +02:00
jazzzooo ae91b993b1 [fix] engine - bing fix search, pagination, remove safesearch 2023-10-19 18:04:52 +02:00
Bnyro b732fbfe5b [feat] engine: implementation of pinterest 2023-10-19 18:04:52 +02:00
Bnyro 32e04f05bf [fix] matrixrooms.info: pagination not working properly 2023-10-19 18:04:52 +02:00
Markus Heiser cef109e0e0 [fix] engine - moviepilot instead of thumbnail use img_src
Instead of thumbnail use img_src in the result item, otherwise the "movies"
categories looks clunky.

Related:

- b4e0d2eedc (r128785388)

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-19 18:04:52 +02:00
Bnyro 3b51a0bf0f [mod] tagesschau: add option to only use tagesschau urls 2023-10-19 18:04:52 +02:00
Bnyro 07d932fba0 [feat] engine: implementation of matrixrooms.info 2023-10-19 18:04:52 +02:00
Bnyro 3974a88dae [feat] engine: implementation of tootfinder 2023-10-19 18:04:52 +02:00
Bnyro aad75ae867 [mod] add movies category for tmdb, imdb and moviepilot 2023-10-19 18:04:52 +02:00
Émilien (perso) e0bab6aa63 [mod] warn to info for failed ping limiter 2023-10-19 18:04:52 +02:00
Bnyro da82436054 [mod] settings.yml: remove plugin settings for plugins that don't exist anymore 2023-10-19 18:04:52 +02:00
searxng-bot eb5bcc523c [translations] update from Weblate
80bbf74ef - 2023-09-27 - return42 <markus.heiser@darmarit.de>
5d554ef71 - 2023-09-27 - BernieHuang2008 <berniehuang2008@163.com>
771fd1b90 - 2023-09-27 - return42 <markus.heiser@darmarit.de>
d412012c8 - 2023-09-27 - y0nei <y0nei@proton.me>
1ac171d0a - 2023-09-27 - return42 <markus.heiser@darmarit.de>
a004c40cc - 2023-09-27 - return42 <markus.heiser@darmarit.de>
47e729f89 - 2023-09-27 - return42 <markus.heiser@darmarit.de>
f85b28cc4 - 2023-09-26 - Fjuro <ifjuro@proton.me>
75ce5d581 - 2023-09-27 - return42 <markus.heiser@darmarit.de>
40ce06009 - 2023-09-24 - nonepork <ptohrek@gmail.com>
21f343f71 - 2023-09-25 - tentsbet <remendne@pentrens.jp>
4c80f80fd - 2023-09-22 - return42 <markus.heiser@darmarit.de>
ef3c859e4 - 2023-09-22 - artnay <jiri.gronroos@iki.fi>
2023-10-19 18:04:52 +02:00
dalf 0620c43e87 Update searx.data - update_firefox_version.py 2023-10-19 18:04:52 +02:00
dalf 5e984a2856 Update searx.data - update_ahmia_blacklist.py 2023-10-19 18:04:52 +02:00
dalf cfe1974b65 Update searx.data - update_wikidata_units.py 2023-10-19 18:04:52 +02:00
dalf 442f3f0c06 Update searx.data - update_engine_traits.py 2023-10-19 18:04:52 +02:00
dalf 22822fc179 Update searx.data - update_currencies.py 2023-10-19 18:04:52 +02:00
dalf 46eb440057 Update searx.data - update_engine_descriptions.py 2023-10-19 18:04:52 +02:00
Bnyro e17575112a [fix] categories.html: can't select social media category using search on category select 2023-10-19 18:04:52 +02:00
jazzzooo c4d280d890 [fix] engine - currency fix and simplify 2023-10-19 18:04:52 +02:00
Jinyuan Huang 3c8ad561ff [fix] bilibili new api used 2023-10-19 18:04:52 +02:00
jazzzooo adaf30a5ff [fix] engine - kickass update url, fix parsing, use multiple mirrors 2023-10-19 18:04:52 +02:00
Markus Heiser ba0d5f78b9 Update searx.data - update_engine_traits.py
Update searx.data - update_engine_traits.py

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-19 18:04:52 +02:00
Markus Heiser c61c7f3416 [fix] engine brave - fetch traits (modified settings menu)
Brave has changed it settings menu fundamental.  Region codes are no longer in
the HTML page, we have to read the regional codes from a JS:

  https://cdn.search.brave.com/serp/v2/_app/immutable/chunks/parameters.734c106a.js

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-19 18:04:52 +02:00
Markus Heiser ddc1735d0f [fix] engine annas archive - fetch traits (modified xpath selectors)
Anna’s Archive has cleaned up their languages, available file extensions and
changed the HTML form.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-19 18:04:52 +02:00
Jinyuan Huang fb225803f2 solve #2839 2023-10-19 18:04:52 +02:00
jazzzooo e2da313ab5 [fix] update user agent 2023-10-19 18:04:52 +02:00
Emilien Devos 4ae7b27389 add warning about enabling public_instance setting 2023-10-19 18:04:52 +02:00
Emilien Devos 078d4a4d9c add new parameter called server.public_instance
for enabling by default advanced limiter functions
in the future allow us to add features just for the public instances
2023-10-19 18:04:52 +02:00
Jinyuan Huang 7722a9b40e [fix] settings icon invisble on mobile devices 2023-10-19 18:04:37 +02:00
Alexandre Flament 1182e76b53 /config: expose limiter configuration 2023-10-19 18:04:37 +02:00
Émilien (perso) f9d2b9f5f1 [mod] remove twitter (#2843) 2023-10-19 18:04:37 +02:00
Markus Heiser a7a1e4f866 [doc] move dosc of botdetection from developer to admin section
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-19 18:04:37 +02:00
Bnyro e845c8857b [feat] engine: implementation of imgur 2023-10-19 18:04:37 +02:00