Commit Graph

290 Commits

Author SHA1 Message Date
dalf
4c8ae5b7ed Update searx.data - firefox_version 2021-02-26 08:30:45 +00:00
dalf
d2778b5efe Update searx.data - currencies 2021-02-26 08:30:45 +00:00
Alexandre Flament
7c1847d5f2 [mod] add utils/fetch_external_bangs.py
Based on duckduckgo bangs
Store bangs on a trie to allow autocomplete (not in this commit)
2021-02-24 18:48:36 +01:00
Alexandre Flament
5f4a085fc4
Merge pull request #2595 from dalf/update-wikidata-units
[mod] update wikidata_units.json and fetch_wikidata_units.py
2021-02-23 17:22:37 +01:00
Alexandre Flament
46ca32c3cc [mod] update currencies.json and fetch_currencies.py
use a sparql request on wikidata to get the list of currencies.

currencies.json contains the translation for all supported searx languages.

Supersede #993
2021-02-23 16:42:28 +01:00
Alexandre Flament
93d1da4906 [mod] update wikidata_units.json and fetch_wikidata_units.py
The fetch_wikidata_units.py result won't change randomly.
See comments in the script.
2021-02-23 13:10:38 +01:00
Markus Heiser
ff6804e545 [data] make engines.languages
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-24 09:52:32 +01:00
Alexandre Flament
f5c3cb7afa [mod] drop Python 3.5 support 2021-01-12 09:45:16 +01:00
Markus Heiser
4de276e364 [upd] make SEARX_DEBUG=1 useragents.update
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-12-22 14:23:58 +01:00
Alexandre Flament
ed6696e6bf [mod] add external_urls.json and wikidata_units.json 2020-10-28 08:09:25 +01:00
Alexandre Flament
5e7060053c [mod] ahmia_filter.py: minor changes
- use result['parsed_url']
- load ahmia_blacklist.txt in searx.datae
2020-10-27 20:00:04 +01:00
Marc Abonce Seguin
32957cdf49 add Ahmia filter plugin for onion results 2020-10-25 17:59:43 -07:00
Alexandre Flament
a9dc54bebc [mod] Add searx.data module
Instead of loading the data/*.json in different location,
load these files in the new searx.data module.
2020-10-07 10:29:34 +02:00
Marc Abonce Seguin
ecf5899153 fetch google's search langs rather than ui langs 2020-09-22 11:37:44 +02:00
Marc Abonce Seguin
c86504b47a fix utils/fetch_languages to work with new languages 2020-09-22 11:37:44 +02:00
Michael Ilsaas
a1ce141c99
add peertube engine (#2109) 2020-08-08 19:22:53 +02:00
Dalf
b339b8b6d6 [mod] bangs.json: remove the favicons
fix #2076
2020-07-15 09:15:26 +02:00
Lukas van den Berk
4829a76aae
Created new plugin type custom_results. Added new plugin bang_redirect (#2027)
* Made first attempt at the bangs redirects plugin.

* It redirects. But in a messy way via javascript.

* First version with custom plugin

* Added a help page and a operator to see all the bangs available.

* Changed to .format because of support

* Changed to .format because of support

* Removed : in params

* Fixed path to json file and changed bang operator

* Changed bang operator back to &

* Made first attempt at the bangs redirects plugin.

* It redirects. But in a messy way via javascript.

* First version with custom plugin

* Added a help page and a operator to see all the bangs available.

* Changed to .format because of support

* Changed to .format because of support

* Removed : in params

* Fixed path to json file and changed bang operator

* Changed bang operator back to &

* Refactored getting search query. Also changed bang operator to ! and is now working.

* Removed prints

* Removed temporary bangs_redirect.js file. Updated plugin documentation

* Added unit test for the bangs plugin

* Fixed a unit test and added 2 more for bangs plugin

* Changed back to default settings.yml

* Added myself to AUTHORS.rst

* Refacored working of custom plugin.

* Refactored _get_bangs_data from list to dict to improve search speed.

* Decoupled bangs plugin from webserver with redirect_url

* Refactored bangs unit tests

* Fixed unit test bangs. Removed dubbel parsing in bangs.py

* Removed a dumb print statement

* Refactored bangs plugin to core engine.

* Removed bangs plugin.

* Refactored external bangs unit tests from plugin to core.

* Removed custom_results/bangs documentation from plugins.rst

* Added newline in settings.yml so the PR stays clean.

* Changed searx/plugins/__init__.py back to the old file

* Removed newline search.py

* Refactored get_external_bang_operator from utils to external_bang.py

* Removed unnecessary import form test_plugins.py

* Removed _parseExternalBang and _isExternalBang from query.py

* Removed get_external_bang_operator since it was not necessary

* Simplified external_bang.py

* Simplified external_bang.py

* Moved external_bangs unit tests to test_webapp.py. Fixed return in search with external_bang

* Refactored query parsing to unicode to support python2

* Refactored query parsing to unicode to support python2

* Refactored bangs plugin to core engine.

* Refactored search parameter to search_query in external_bang.py
2020-07-03 13:25:04 +00:00
Émilien Devos
7a50522a82
Update default user agents 2020-04-21 20:50:16 +00:00
Markus Heiser
809e6f7933 make project: re-build generic files of the searx project
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-26 13:35:17 +01:00
Markus Heiser
79412828cd update languages: engines_languages.json languages.py
build by Makefile target:

    make project

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-01 08:06:51 +01:00
Markus Heiser
40843fe95a searx/data/engines_languages.json: sort json file (no content change)
To get meaningfull diffs, the json file has to be sorted.  Before applying any
further content patch, the json file needs a inital sort (without changing any
content).

Sorted by::

  import sys, json

  with open('engines_languages.json') as f:
      j = json.load(f)

  with open('engines_languages.json', 'w') as f:
      json.dump(j, f, indent=2, sort_keys=True)

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-01 07:56:46 +01:00
Marc Abonce Seguin
d0d91fa331 update user agents 2019-11-23 19:27:49 -07:00
Marc Abonce Seguin
94ca30de1b add a shortcut for utils/fetch_firefox_version.py in manage.sh 2019-11-23 19:26:48 -07:00
Marc Abonce Seguin
0169b63e84 [fix] fetch google's supported languages 2019-01-06 21:31:45 -06:00
Ivan Skytte Jørgensen
0b7f0bde62 Use a more human-friendly format in searx/data/engines_languages.json (#1399)
Instead of a single line with 500000 characters use nicely formatted JSON.
Sort the lists in engine_languages.py so when updating it is possible to
more easily see the differences (search engines do change the order their
languages are listed in)
2018-11-29 16:31:05 +01:00
Alex
50c836864a fetch_firefox_version.py : compatible with Python 3 and minor fixes. 2018-08-05 10:55:42 +02:00
Alexandre Flament
066bd916bf [mod] fetch firefox versions in a standalone script 2018-08-05 10:10:15 +02:00
rinpatch
a305dae9d8 switched currency convert to duckduckgo and added Bitcoin to currency list 2018-04-22 13:46:13 +03:00
Marc Abonce Seguin
772c048d01 refactor engine's search language handling
Add match_language function in utils to match any user given
language code with a list of engine's supported languages.

Also add language_aliases dict on each engine to translate
standard language codes into the custom codes used by the engine.
2018-03-27 00:08:03 -06:00
Marc Abonce Seguin
d1eae9359f fix fetch_langauges to be more accurate
Add languages supported by either all default general engines or 10 engines.
2018-03-20 17:58:20 -06:00
marc
44085e31d0 update engines_languages.json and languages.py
Also, fix fetch_languages.py so it can run on python3.
2017-10-10 16:53:28 -05:00
marc
405e5c8f24 [fix] duckduckgo images doesn't fail with countryless language 2017-07-10 18:41:50 +02:00
marc
fd65c12921 make search language handling less strict
languages.py can change, so users may query on a language that is not
on the list anymore, even if it is still recognized by a few engines.

also made no and nb the same because they seem to return the same,
though most engines will only support one or the other.
2017-03-18 23:44:21 +01:00
marc
805fb02ed1 add language support for qwant
closes issue #863
2017-03-18 23:44:21 +01:00
marc
1175b3906f change language list to only include languages with a minimum of engines
that support them.
users can still query lesser supported through the :lang_code bang.
2016-12-29 01:55:30 -06:00
marc
4a1ff56389 minor fixes in utils/fetch_languages.py 2016-12-16 22:14:14 -06:00
marc
af35eee10b tests for _fetch_supported_languages in engines
and refactor method to make it testable without making requests
2016-12-15 00:40:21 -06:00
marc
f62ce21f50 [mod] fetch supported languages for several engines
utils/fetch_languages.py gets languages supported by each engine and
generates engines_languages.json with each engine's supported language.
2016-12-13 19:58:10 -06:00
Alexandre Flament
5525625dae [enh] currency_convert engine: user can write "1 dollars in euros" instead of "1 USD in EUR".
The currency names are fetched wikidata and store into a static file : searx/data/currencies.json
This file is loaded when the currency_converter is loaded.
A database is perhaps more appropiated.
2015-05-12 20:52:08 +02:00