zaclys/searxng
1
0
Fork 1
mirror of https://github.com/searxng/searxng synced 2024-01-01 19:24:07 +01:00
Code Issues Projects Releases Packages Wiki Activity

[fix] searx_extra/update scripts: set_loggers(wikidata, 'wikidata') (#331)

Browse source 
* [fix] searx_extra/update scripts: set_loggers(wikidata, 'wikidata')

To test use::

    ./manage pyenv.cmd searx_extra/update/update_currencies.py
    ./manage pyenv.cmd searx_extra/update/update_osm_keys_tags.py
    ./manage pyenv.cmd searx_extra/update/update_wikidata_units.py

The script `update_engine_descriptions.py` seems to have some issues not related
to this patch.

    ./manage pyenv.cmd python -m pip install -U pycld3
    ./manage pyenv.cmd searx_extra/update/update_engine_descriptions.py

Closes: https://github.com/searxng/searxng/issues/328
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This commit is contained in:
Markus Heiser 2021-09-19 09:10:02 +00:00 • committed by GitHub
parent d95c46d646
commit 55b30b5b49
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
4 changed files with 28 additions and 15 deletions
Download patch file Download diff file Expand all files Collapse all files

7
searx_extra/update/update_wikidata_units.py
View file

@ -7,13 +7,14 @@ import collections
from os.path import join
from searx import searx_dir
from searx.engines.wikidata import send_wikidata_query
from searx.engines import wikidata, set_loggers
set_loggers(wikidata, 'wikidata')
# the response contains duplicate ?item with the different ?symbol
# "ORDER BY ?item DESC(?rank) ?symbol" provides a deterministic result
# even if a ?item has different ?symbol of the same rank.
# A deterministic result
# A deterministic result
# see:
# * https://www.wikidata.org/wiki/Help:Ranking
# * https://www.mediawiki.org/wiki/Wikibase/Indexing/RDF_Dump_Format ("Statement representation" section)
@ -36,7 +37,7 @@ ORDER BY ?item DESC(?rank) ?symbol
def get_data():
results = collections.OrderedDict()
response = send_wikidata_query(SARQL_REQUEST)
response = wikidata.send_wikidata_query(SARQL_REQUEST)
for unit in response['results']['bindings']:
name = unit['item']['value'].replace('http://www.wikidata.org/entity/', '')
unit = unit['symbol']['value']