searxng/searx/data
Markus Heiser 744d96a16c [fix] startpage engine: language/region & time support & fix CAPTCHA
One reason for the often seen CAPTCHA of the startpage requests are the
incomplete requests SearXNG sends to startpage.com.  To avoid CAPTCHA we need to
send a well formed HTTP POST request with a cookie, we need to form a request
that is identical to the request build by startpage.com itself:

- in the cookie the **region** is selected
- in the POST arguments the **language** is selected

Based on the *engine_properties* boilerplate, SearXNG's startpage engine now
implements a `_fetch_engine_properties()` function to fetch regions & languages
from startpage.com.

This patch is a complete new implementation of the request() function, reversed
engineered from the startpage.com page.  The new implementation adds

- time-range support
- save-search support

to the startpage engine which has been missed in the past.

The locale code 'no_NO' from startpage does not exists and is mapped to nb-NO.
For reference see languages-subtag at iana [1], `no` is the macrolanguage::

     type: language
     Subtag: nb
     Description: Norwegian Bokmål
     Added: 2005-10-16
     Suppress-Script: Latn
     Macrolanguage: no

Additional hints:

- To fetch languages from startpage, this patch makes use of the
  EngineProperties implemented in 7bf0d46c

- Te get Startpage's locale & language, the function get_engine_locale from
  9ae409a is used.

[1] https://www.iana.org/assignments/language-subtag-registry/language-subtag-registry
[2] https://www.w3.org/International/questions/qa-choosing-language-tags#langsubtag

Closes: https://github.com/searxng/searxng/issues/1081
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-08-29 19:27:50 +02:00
..
__init__.py [mod] remove deprecate code 2022-01-29 08:54:12 +01:00
ahmia_blacklist.txt Update searx.data - update_ahmia_blacklist.py 2022-08-29 02:09:09 +00:00
currencies.json Update searx.data - update_currencies.py 2022-08-29 02:09:09 +00:00
engine_descriptions.json Update searx.data - update_engine_descriptions.py 2022-08-29 02:17:55 +00:00
engines_languages.json [fix] startpage engine: language/region & time support & fix CAPTCHA 2022-08-29 19:27:50 +02:00
external_bangs.json [upd] searx/data/external_bangs.json 2022-01-12 19:37:13 +01:00
external_urls.json [mod] add external_urls.json and wikidata_units.json 2020-10-28 08:09:25 +01:00
osm_keys_tags.json [enh] openstreetmap / map template: improve results 2021-06-09 18:08:23 +02:00
useragents.json Update searx.data - update_firefox_version.py 2022-08-29 02:09:34 +00:00
wikidata_units.json Update searx.data - update_wikidata_units.py 2022-08-29 02:09:17 +00:00