searxngRebrandZaclys/searx
Markus Heiser 4cb8cbc6c3 [fix] duckduckgo lite engine: set HTTP header 'Referer'
We have had problems with this before, the bot protection from ddg-lite seems to
have included this referer in the rating [1][2].

From reverse engineering:

- The Referer ``https://google.com/`` was set in commt 257dc7d6c4 --> DDG lite
  does not like this referer anymore!

- The 'Referer' header is only set on second and follow up pages but not on the
  first page

- The vqd value is not needed on the first page, the ddg-lite client sets this
  value only on follow up pages / this can help to reduce the vqd requests from
  SearXNG.

Related to 'Referer' header & ddg requests:

[1] https://github.com/searxng/searxng/pull/2161
[2] https://github.com/searxng/searxng/pull/2081

Closes: https://github.com/searxng/searxng/issues/2796
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-19 18:04:53 +02:00
..
answerers [mod] random answerer: add random hex color generator 2023-08-28 10:25:44 +02:00
botdetection [fix] limiter / botdetection: remove http_connection method 2023-10-19 18:04:53 +02:00
data [feat] duckduckgo: support for videos and news 2023-10-19 18:04:53 +02:00
enginelib [fix] spelling 2023-10-19 18:02:05 +02:00
engines [fix] duckduckgo lite engine: set HTTP header 'Referer' 2023-10-19 18:04:53 +02:00
infopage [mod] search-syntax.md: add tutorial about the feeling lucky feature 2023-10-19 18:04:21 +02:00
metrics [fix] spelling 2023-10-19 18:02:05 +02:00
network [fix] spelling 2023-10-19 18:02:05 +02:00
plugins add new parameter called server.public_instance 2023-10-19 18:04:52 +02:00
search [feat] implement feeling lucky feature 2023-10-19 18:03:21 +02:00
static [fix] settings icon invisble on mobile devices 2023-10-19 18:04:37 +02:00
templates/simple [fix] categories.html: can't select social media category using search on category select 2023-10-19 18:04:52 +02:00
tools [mod] tools.Config.get(): add missing type annotations 2023-06-05 14:07:19 +02:00
translations [translations] update from Weblate 2023-10-19 18:04:53 +02:00
__init__.py add warning about enabling public_instance setting 2023-10-19 18:04:52 +02:00
autocomplete.py [doc] add documentation of Mwmbl engine & autocompleter 2023-08-28 10:25:44 +02:00
babel_extract.py [mod] move category and names of constants to searx/searxng.msg 2022-03-16 09:55:53 +01:00
compat.py [fix] pyright repported errors 2022-07-30 18:04:44 +02:00
exceptions.py [fix] spelling 2023-10-19 18:02:05 +02:00
external_bang.py [mod] external bang: go to main instead of search page when query is empty 2023-04-25 15:02:34 +02:00
external_urls.py [fix] typos / reported by @kianmeng in searx PR-3366 2022-09-27 18:32:14 +02:00
flaskfix.py [format.python] initial formatting of the python code 2021-12-27 09:26:22 +01:00
locales.py [fix] spelling 2023-10-19 18:02:05 +02:00
preferences.py [mod] add hotkeys option to settings.yml 2023-10-19 18:04:53 +02:00
query.py [feat] implement feeling lucky feature 2023-10-19 18:03:21 +02:00
redisdb.py move searx.shared.redisdb to searx.redisdb 2022-12-10 09:26:38 +01:00
redislib.py [fix] spelling 2023-10-19 18:02:05 +02:00
results.py [mod] move some code from webapp module to webutils module (no functional change) 2023-06-19 19:49:44 +02:00
searxng.msg [mod] clarify the difference of the default category and subgrouping 2023-04-07 11:03:25 +02:00
settings_defaults.py [mod] add hotkeys option to settings.yml 2023-10-19 18:04:53 +02:00
settings_loader.py [fix] spelling 2023-10-19 18:02:05 +02:00
settings.yml [mod] yacy: use official instance by default and fix crashes 2023-10-19 18:04:53 +02:00
sxng_locales.py Update searx.data - update_engine_traits.py 2023-10-19 18:04:52 +02:00
unixthreadname.py [format.python] initial formatting of the python code 2021-12-27 09:26:22 +01:00
utils.py [fix] spelling 2023-10-19 18:02:05 +02:00
version.py [fix] typos in documentation & messages 2023-08-28 10:23:45 +02:00
webadapter.py [feat] implement feeling lucky feature 2023-10-19 18:03:21 +02:00
webapp.py add new parameter called server.public_instance 2023-10-19 18:04:52 +02:00
webutils.py [fix] implement a JSONEncoder for the json format 2023-06-19 19:49:44 +02:00