Most engines response best results if a region is selected, most often a
language is also in the properties of a engine and sometimes the language
argument is just the language of the UI. Most often choosing a language has a
minor effect on the result list.
To summarize:
Some engines have language codes (e.g. `ca`) in their properties, some have
region codes (e.g. `ca-ES`), some have regions and languages in their properties
and other engine do not have any language or region support.
In the past we generalized *language* over all kind of engines without taking
into mind that most engines gave best result when there is a region selected.
This *language-centric* view in SearXNG is misleading when we need
region-codes to parameterize engine request!
This patch replaces the *language-centric* view by a "language / region" view.
Conclusions:
With regions we can't say any longer that a engine supports *this or that*
language, by example: when the user selects 'zh' and a engine supports only
region codes like 'zh-TW' or 'zh-CN' we do not what results the user expects /
similar with 'en' or 'fr when the engine needs a region tag.
- Since it is unclear what the user expects by his language selection, we can't
assert a property that says: "supports_selected_language"
The feature is replaced in the UI by the wider sense of "language_support",
what stands for:
The engine has some kind of language support, either
by a region tag or by a language tag.
- A list of "supported_languages" does not make sense when there are regions
responsible for the result of an engine.
The "supported_languages" has been removed from the /config URL
- The `has_language` test in the `searx/search/checker/impl.py` has been removed
since it does not cover engines with region support.
If there is a need for such a test we can implement new tests after all
engines with language (region) support has been moved to the *supported
properites* scheme (see searxng_extra/update/update_languages.py)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Since https://github.com/searxng/searxng/pull/354
the searx.network.stream(...) returns a tuple
This commits update the checker code according to
this function signature change.
Disable the python code formatting from python-black, where the readability of
code suffers by formatting.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
* download images using the "image_proxy" network (HTTP/1 instead of HTTP/2)
* don't cache data: URL (reduce memory usage)
* after each test: purge image URL cache then call garbage collector
* download only the first 64kb of images
* display the median time instead of the average.
* add a "Reliability" column (sum up the metrics and the checker results).
* the "selected language", "SafeSearch", "Time range" values are displayed as "broken" when the checker tests fail.
Report to the user suspended engines.
searx.search.processor.abstract:
* manages suspend time (per network).
* reports suspended time to the ResultContainer (method extend_container_if_suspended)
* adds the results to the ResultContainer (method extend_container)
* handles exceptions (method handle_exception)
settings.yml:
* outgoing.networks:
* can contains network definition
* propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections,
keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries
* retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time)
* local_addresses can be "192.168.0.1/24" (it supports IPv6)
* support_ipv4 & support_ipv6: both True by default
see https://github.com/searx/searx/pull/1034
* each engine can define a "network" section:
* either a full network description
* either reference an existing network
* all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
