forked from zaclys/searxng
9ae409a05a
The match_language function sometimes returns incorrect results which is why a
new function get_engine_locale is required.
A bugfix of the match_language is not easily possible, because there is almost
no documentation for it and already the call parameters are undefined. E.g. the
function processes values like the ones from yahoo::
"yahoo": [
"ar",
...
"zh_chs",
"zh_cht"
]
The get_engine_locale has been documented in detail, there is a clear
description of the assumptions as well as the requirements and approximation
rules (read doc-string for more details)::
Argument ``engine_locales`` is a python dict that maps *SearXNG locales* to
corresponding *engine locales*:
<engine>: {
# SearXNG string : engine-string
'ca-ES' : 'ca_ES',
'fr-BE' : 'fr_BE',
'fr-CA' : 'fr_CA',
'fr-CH' : 'fr_CH',
'fr' : 'fr_FR',
...
'pl-PL' : 'pl_PL',
'pt-PT' : 'pt_PT'
}
.. hint::
The *SearXNG locale* string has to be known by babel!
In the following you will find a comparison:
>>> import babel.languages
>>> from searx.utils import match_language
>>> from searx.locales import get_engine_locale
Assume we have an engine that supports the follwoing locales:
>>> lang_list = {
... "zh-CN": "zh_CN",
... "zh-HK": "zh_HK",
... "nl-BE": "nl_BE",
... "fr-CA": "fr_CA",
... }
Assumption:
A. When a user selects a language the results should be optimized according to
the selected language.
B. When user selects a language and a territory the results should be
optimized with first priority on territory and second on language.
----
Example: (Assumption A.)
A user selects region 'zh-TW' which should end in zh_HK
hint:
CN is 'Hans' and HK ('Hant') fits better to TW ('Hant')
>>> get_engine_locale('zh-TW', lang_list)
'zh_HK'
>>> lang_list[match_language('zh-TW', lang_list)]
'zh_CN'
----
Example: (Assumption A.)
A user selects only the language 'zh' which should end in CN
>>> get_engine_locale('zh', lang_list)
'zh_CN'
>>> lang_list[match_language('zh', lang_list)]
'zh_CN'
----
Example: (Assumption B.)
A user selects region 'fr-BE' which should end in nl-BE
hint:
priority should be on the territory the user selected. If the user
prefers 'fr' he will select 'fr' without a region tag.
>>> get_engine_locale('fr-BE', lang_list, default='unknown')
'nl_BE'
>>> match_language('fr-BE', lang_list, fallback='unknown')
'fr-CA'
----
Example: (Assumption A.)
A user selects only the language 'fr' which should end in fr_CA
>>> get_engine_locale('fr', lang_list)
'fr_CA'
>>> lang_list[match_language('fr', lang_list)]
'fr_CA'
----
The difference in priority on the territory is best shown with a engine that
supports the following locales:
>>> lang_list = {
... "fr-FR": "fr_FR",
... "fr-CA": "fr_CA",
... "en-GB": "en_GB",
... "nl-BE": "nl_BE",
... }
----
Example: (Assumption A.)
A user selects only a language
>>> get_engine_locale('en', lang_list)
'en_GB'
>>> match_language('en', lang_list)
'en-GB'
hint: the engine supports fr_FR and fr_CA since no territory is given, fr_FR
takes priority ..
>>> get_engine_locale('fr', lang_list)
'fr_FR'
>>> lang_list[match_language('fr', lang_list)]
'fr_FR'
----
Example: (Assumption B.)
A user selects region 'fr-BE' which should end in nl-BE
>>> get_engine_locale('fr-BE', lang_list)
'nl_BE'
>>> lang_list[match_language('fr-BE', lang_list)]
'fr_FR'
----
If the user selects a language and there are two locales like the following:
>>> lang_list = {
... "fr-BE": "fr_BE",
... "fr-CH": "fr_CH",
... }
>>>
>>> get_engine_locale('fr', lang_list)
'fr_BE'
>>> lang_list[match_language('fr', lang_list)]
'fr_BE'
Looks like both functions return the same value, but match_language depends on the
order of the dictionary (which is not predictable):
>>> lang_list = {
... "fr-CH": "fr_CH",
... "fr-BE": "fr_BE",
... }
>>> get_engine_locale('fr', lang_list)
'fr_BE'
>>> lang_list[match_language('fr', lang_list)]
'fr_CH'
>>>
The get_engine_locale selects the locale by looking at the "population percent"
and this percentage has an higher amount in BE (68.%) compared to CH (21%)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
||
|---|---|---|
| .github | ||
| dockerfiles | ||
| docs | ||
| examples | ||
| searx | ||
| searxng_extra | ||
| src/brand | ||
| tests | ||
| utils | ||
| .coveragerc | ||
| .dir-locals.el | ||
| .dockerignore | ||
| .gitattributes | ||
| .gitignore | ||
| .nvmrc | ||
| .pylintrc | ||
| .weblate | ||
| .yamllint.yml | ||
| AUTHORS.rst | ||
| CHANGELOG.rst | ||
| CONTRIBUTING.md | ||
| Dockerfile | ||
| LICENSE | ||
| Makefile | ||
| PULL_REQUEST_TEMPLATE.md | ||
| README.rst | ||
| SECURITY.md | ||
| babel.cfg | ||
| manage | ||
| package.json | ||
| pyrightconfig-ci.json | ||
| pyrightconfig.json | ||
| requirements-dev.txt | ||
| requirements.txt | ||
| setup.py | ||
README.rst
Privacy-respecting, hackable metasearch engine
If you are looking for running instances, ready to use, then visit searx.space. Otherwise jump to the user, admin and developer handbooks you will find on our homepage.
Contact
Come join us if you have questions or just want to chat about SearXNG.
- Matrix
- IRC
#searxng on libera.chat which is bridged to Matrix.
Differences to searx
SearXNG is a fork of searx. Here are some of the changes:
User experience
- Huge update of the simple theme:
- usable on desktop, tablet and mobile
- light and dark versions (you can choose in the preferences)
- support right-to-left languages
- see the screenshots
- the translations are up to date, you can contribute on Weblate
- the preferences page has been updated:
- you can see which engines are reliable or not
- engines are grouped inside each tab
- each engine has a description
- thanks to the anonymous metrics, it is easier to report a bug of an engine and thus engines get fixed more quickly
- if you don't want any metrics to be recorded, you can disable them on the server
- administrator can block and/or replace the URLs in the search results
Setup
- you don't need Morty to proxy the images even on a public instance
- you don't need Filtron to block bots, we implemented the builtin limiter
- you get a well maintained Docker image, now also built for ARM64 and ARM/v7 architectures
- alternatively we have up to date installation scripts
Contributing is easier
- readable debug log
- contributions to the themes are made easier, check out our Development Quickstart guide
- a lot of code cleanup and bug fixes
- the dependencies are up to date
Translations
We need translators, suggestions are welcome at https://weblate.bubu1.eu/projects/searxng/searxng/
Make a donation
You can support the SearXNG project by clicking on the donation page: https://docs.searxng.org/donate.html