ZaclysIndustries
/
searxngRebrandZaclys
forked from zaclys/searxng
2
0
Fork 0
Code Issues Pull Requests Activity
4,200 Commits 3 Branches 17 Tags 53 MiB
Commit Graph

2781 Commits

Author SHA1 Message Date
Markus Heiser ebfd0eb2b7 [fix] default_doi_resolver in preferences 
Instead of a hard-coded `oadoi.org` default, use the default value from
`settings.yml`.

Fix an issue in the themes: The replacement 'current_doi_resolver' contains the
doi_resolver_url, not the name of the DOI resolver.  Compare return value of::

    searx.plugins.oa_doi_rewrite.get_doi_resolver(...)

Fix a typo in `get_doi_resolver(..)`:  suggested by @kvch:

  *L32 should set doi_resolver not doi_resolvers*

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-04-04 13:36:33 +02:00
Markus Heiser c12826c6d5 [fix] publishedDate: don't try to get date from empty string or None 
Signed-off-by: Markus Heiser <markus@darmarit.de>
 2021-04-04 13:16:38 +02:00
Noémi Ványi 7612e5d9fd
Merge pull request #2712 from searx/update_data_update_firefox_version.py 
Update searx.data - update_firefox_version.py
 2021-04-03 23:15:43 +02:00
Noémi Ványi 985e8b28fe
Merge pull request #2710 from searx/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2021-04-03 23:02:44 +02:00
Noémi Ványi feb2d81874
Merge pull request #2709 from searx/update_data_update_currencies.py 
Update searx.data - update_currencies.py
 2021-04-03 23:02:26 +02:00
Noémi Ványi 9a962c5369
Merge pull request #2711 from searx/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2021-04-03 23:01:43 +02:00
Noémi Ványi 5ca34ac571
Merge pull request #2713 from searx/update_data_update_languages.py 
Update searx.data - update_languages.py
 2021-04-03 23:01:00 +02:00
Markus Heiser 169438137f [fix] url bar autocomplete (opensearch suggestions) 
Since #2593 is merged the OpenSearch-Format is buggy.  The loop in [1] will
change raw_text_query object and this will change also the value of
`raw_text_query.query` on every `raw_text_query.changeQuery(result)`.

This patch fixes this issue by storing the initial query value in `sug_prefix`.

[1] ac0fdc3b96/searx/webapp.py (L804-L806)

OpenSearch-Format::

    [ "<query>",
      [ "<term 1>", "<term 2>", ... "<term n>" ],
      [ "<content 1>", "<content 2>", ..., "<content n>" ],
      [ "<url 1>", "<url 2>", ..., "<url n>" ]
    ]

- https://www.google.com/support/enterprise/static/gsa/docs/admin/current/gsa_doc_set/xml_reference/query_suggestion.html#1080002
- https://developer.mozilla.org/en-US/docs/Archive/Add-ons/Supporting_search_suggestions_in_search_plugins#implementing_search_suggestion_support_on_the_server

Legacy-Format::

    [ "<term 1>", "<term 2>", ..., "<term n>" ]

- https://www.google.com/support/enterprise/static/gsa/docs/admin/current/gsa_doc_set/xml_reference/query_suggestion.html#1081079

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-04-03 18:18:50 +02:00
dalf c0668d248e Update searx.data - update_languages.py 2021-04-01 07:00:46 +00:00
dalf ad74c42aa1 Update searx.data - update_firefox_version.py 2021-04-01 07:00:39 +00:00
dalf 23dc7ef6d1 Update searx.data - update_wikidata_units.py 2021-04-01 07:00:37 +00:00
dalf eb5cd7a543 Update searx.data - update_ahmia_blacklist.py 2021-04-01 07:00:33 +00:00
dalf f804f54ca3 Update searx.data - update_currencies.py 2021-04-01 07:00:29 +00:00
Marc Abonce Seguin 419b907a0b fix dark "expand" button from infobox 2021-03-28 21:54:37 -07:00
Adam Tauber ae122ea943 [enh] release v1.0.0 2021-03-27 20:30:08 +01:00
Adam Tauber 15f2385a7b [enh] update translations from transifex 2021-03-27 19:10:54 +01:00
Alexandre Flament bb316d2698
Merge pull request #2682 from dalf/fix-checker 
[fix] checker: various bug fixes
 2021-03-27 17:44:29 +01:00
Alexandre Flament 725a69616b
Merge pull request #2681 from dalf/fix-wikipedia-title 
[fix] wikipedia: remove HTML from the title
 2021-03-27 17:43:36 +01:00
Noémi Ványi 9bb312c505 Remove duplicated key from dict in Semantic Scholar 2021-03-27 16:58:32 +01:00
Noémi Ványi f596f5767b fix Semantic Scholar engine 2021-03-27 16:54:01 +01:00
Adam Tauber 28286cf3f2 [fix] update seznam engine to be compatible with the new website 2021-03-27 15:29:04 +01:00
Alexandre Flament 0b45afd4d7 [fix] checker: various bug fixes 
* initialize engine_data (youtube engine)
* don't crash if an engine don't set result['url']
 2021-03-25 09:37:37 +01:00
Alexandre Flament fcfcf662ff [fix] wikipedia: remove HTML from the title 
fr.wikipedia.org (and it seems not other wikipedia websites),
adds HTML to api_result['displayTitle'].
(Search for '!wp :fr Braid' for example)

The commit uses api_result['title']
 2021-03-25 08:31:39 +01:00
Adam Tauber 06b754ad67 [mod] increase lobste.rs engine timeout to avoid timeouts most of the time 2021-03-25 01:22:36 +01:00
Adam Tauber 0ba71c3644 [fix] make ina engine compatible with the new response json 2021-03-25 01:20:41 +01:00
Adam Tauber 6255b33c9d [fix] rewrite hoogle to use html/xpath instead of json 
the json response has been changed and it contains html chunks which is
not compatible with our json engine, so we have to switch to html/xpath
parsing
 2021-03-25 01:13:24 +01:00
Adam Tauber 45f0e1a859 [fix] update geektimes.ru url - it redirects to habr.com 2021-03-25 01:02:19 +01:00
Adam Tauber 50ba2b9e87 [fix] update google play movies xpath 2021-03-25 00:55:53 +01:00
Adam Tauber 88657fe9c2 [fix] update google play apps xpath 2021-03-25 00:55:43 +01:00
Adam Tauber 5f450fda74 [enh] add year filter to duckduckgo 2021-03-25 00:25:36 +01:00
Adam Tauber fd737dc9d8 [fix] remove debug code 2021-03-24 23:54:39 +01:00
Alexandre Flament d648001688 [mod] preferences: a tooltip is shown when the mouse is over the engine names 2021-03-22 08:22:59 +01:00
Alexandre Flament 6bd01bf81f [mod] oscar: fix the sourcemap URL in *.min.css 
Close https://github.com/searx/searx/issues/2670

Note: clean-css contains a bug:
* a multiline comment or URL adds "$stdin" to the sourcemap (see src/less/logicodev/search.less)
* in this case when the user opens the devtools, the browser fails to load this "https://.../$stdin" URL
* it is not a problem and the error appears only when the user actively tries to debug the CSS.
* seems related to https://github.com/jakubpawlowicz/clean-css/issues/593
 2021-03-21 18:03:40 +01:00
Alexandre Flament a48ec0b4bd
Merge pull request #2671 from searx/update-soundcloud 
[mod] soundcloud: faster initialization
 2021-03-21 15:10:39 +01:00
Alexandre Flament 30c950a2c7
Merge pull request #2660 from dalf/upd-translations 
[mod] replace /translations.js with an embedded JSON
 2021-03-21 12:39:26 +01:00
Alexandre Flament 38c210d746
[mod] soundcloud: faster initialization 
The get_cliend_id() function:
* fetches https://soundcloud.com
* then fetches each referenced javascript URL to get the client id.

This commit fetches the javascript URLs in the reverse order: the client id is in the last javascript URL.
 2021-03-21 09:29:53 +01:00
James Higginbotham ce6eb81a71
Update settings.yml to enable HTTP for yacy 
Added a line to the yacy entry to enable HTTP if the local yacy instance isn't using HTTPS. Otherwise, an error will be thrown in the logs: "No connection adapters were found for 'http://localhost:8090/yacysearch.json...'". This is likely related to ticket #2641 that forces HTTPS by default.
 2021-03-19 15:06:25 -06:00
Alexandre Flament 2b0dd96bd3 [mod] oscar: remove space 
* reduce by 15% the uncompressed output (on average)
* dos2unix searx/templates/oscar/result_templates/files.html
 2021-03-17 09:22:05 +01:00
Dr. Rolf Jansen 7a9dc63d74
Merge branch 'master' into conditional-sigusr1 2021-03-16 08:45:57 -03:00
Alexandre Flament 6553c79029 [mod] replace /translations.js by embedded JSON 
In webapp.py, there is a new function "get_translations" lists available translations

Close #2064
 2021-03-16 11:22:21 +01:00
Alexandre Flament 32cd0d31b3 [mod] upgrade pygments 
add searx_extra/update/update_pygments.py to update the css style of the oscar and simple themes.
 2021-03-16 09:07:08 +01:00
Dr. Rolf Jansen 2a6dbeb6a5
Merge branch 'master' into conditional-sigusr1 2021-03-15 19:31:44 -03:00
Adam Tauber 4c631ac6d0 [fix] remove debug code 2021-03-15 21:47:27 +01:00
Dr. Rolf Jansen 4a27dabcf7
Merge branch 'master' into conditional-sigusr1 2021-03-15 17:03:36 -03:00
Noémi Ványi 8158d8654a fix Microsoft Academic engine 2021-03-15 20:21:28 +01:00
Adam Tauber f97b4ff7b6 [fix] update youtube_noapi paging 2021-03-15 17:22:31 +01:00
Adam Tauber dd34ac396c
Merge pull request #2652 from kvch/solr-engine 
Add Apache Solr engine
 2021-03-15 15:39:39 +01:00
Alexandre Flament 1664258061
Merge pull request #2655 from return42/fix-imports 
[fix] remove unused import from yahoo-news engine
 2021-03-15 08:38:34 +01:00
Alexandre Flament 5b176b3496
Merge pull request #2659 from MarcAbonce/onions-http-fix 
Fix HTTP error in onion engines
 2021-03-15 08:33:38 +01:00
Marc Abonce Seguin f4a0a4d756 fix HTTP error in onion engines 
regression from https://github.com/searx/searx/pull/2641
most onion websites only serve HTTP, so it must be enabled
 2021-03-14 20:23:07 -07:00
Rolf 80025c3244 Windows does not support SIGUSR1, so don't use it unconditionally. 2021-03-14 19:04:36 -03:00
Markus Heiser 6e1f1085ef [fix] remove unused import from yahoo-news engine 
Signed-off-by: Markus Heiser <markus@darmarit.de>
 2021-03-14 15:13:57 +01:00
Markus Heiser 3703ebb22a [drop] Acgsou engine - www.acgsou.com no longer exists 
- https://www.acgsou.com/ acgsou.com is redirected to 36dm.club
- @rinpatch do not plan on maintaining the engine [1]

[1] https://github.com/searx/searx/pull/1283#issuecomment-798783585

Signed-off-by: Markus Heiser <markus@darmarit.de>
 2021-03-14 11:49:18 +01:00
Noémi Ványi ff527e2681 Add Solr engine 2021-03-13 21:18:09 +01:00
Alexandre Flament 9292571304
Merge pull request #2346 from dalf/upgrade-oscar 
[mod] oscar: upgrade dependencies
 2021-03-13 09:29:13 +01:00
Alexandre Flament 92dd5e245e
Merge pull request #2626 from mikeri/solidtorrents 
Add Solid Torrents engine
 2021-03-12 19:45:22 +01:00
Alexandre Flament a1a492baed
Merge pull request #2641 from dalf/disable_http_by_default 
[mod] by default allow only HTTPS, not HTTP
 2021-03-12 19:21:46 +01:00
Alexandre Flament cb04d42806 [mod] oscar: update README.rst 2021-03-11 09:33:04 +01:00
Alexandre Flament 86912e2272 [mod] oscar: get bootstrap and typeahead from NPM 2021-03-11 09:33:01 +01:00
Alexandre Flament 44407353ef [mod] oscar: get leaflet and jquery from NPM 
easy to upgrade (package.json)
 2021-03-11 09:32:22 +01:00
Alexandre Flament c7133efb12 [mod] oscar: move compiled files to the src directory 2021-03-10 19:28:51 +01:00
Alexandre Flament eda3b513ac [mod] oscar: remove polyfills for Internet Explorer 2021-03-10 19:01:16 +01:00
Alexandre Flament 1268910274 [mod] oscar: remove unused images 2021-03-10 19:01:16 +01:00
Alexandre Flament bdb41bea7b [mod] theme: remove require-2.1.15.min.js 
See https://github.com/requirejs/requirejs/issues/1816

requirejs loads one file: leaflet.

This commit:
* removes requirejs
* load leaflet using <script src...> HTML tag in searx/templates/oscar/base.html
 2021-03-10 19:01:15 +01:00
Alexandre Flament 2f3d5ec2af [mod] oscar: upgrade npm dependencies 2021-03-10 19:01:14 +01:00
Markus Heiser 96422e5c9f [fix] APKMirror engine - update xpath selectors and fix img_src 
BTW: make the code slightly more readable

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-03-09 08:34:57 +01:00
Markus Heiser d2faea423a [fix] rewrite Yahoo-News engine 
Many things have been changed since last review of this engine.  This patch fix
xpath selectors, implements suggestion and is a complete review / rewrite of the
engine.

Signed-off-by: Markus Heiser <markus@darmarit.de>
 2021-03-08 11:43:34 +01:00
Alexandre Flament 99e0651cea [mod] by default allow only HTTPS, not HTTP 
Related to https://github.com/searx/searx/pull/2373
 2021-03-08 11:35:08 +01:00
Michael Ilsaas 5549d58de3 Add Solid Torrents engine 2021-03-07 18:14:30 +01:00
Adam Tauber 44f4a9d49a [enh] add ability to send engine data to subsequent requests 2021-03-06 12:12:35 +01:00
Alexandre Flament 87f4cc4a9e
Merge pull request #2631 from searx/update_data_update_languages.py 
Update searx.data - update_languages.py
 2021-03-06 10:03:00 +01:00
Markus Heiser 4845183128 [mod] don't dump traceback of SearxEngineResponseException on init 
When initing engines a "SearxEngineResponseException" is logged very verbose,
including full traceback information:

    ERROR:searx.engines:yggtorrent engine: Fail to initialize
    Traceback (most recent call last):
      File "share/searx/searx/engines/__init__.py", line 293, in engine_init
        init_fn(get_engine_from_settings(engine_name))
      File "share/searx/searx/engines/yggtorrent.py", line 42, in init
        resp = http_get(url, allow_redirects=False)
      File "share/searx/searx/poolrequests.py", line 197, in get
        return request('get', url, **kwargs)
      File "share/searx/searx/poolrequests.py", line 190, in request
        raise_for_httperror(response)
      File "share/searx/searx/raise_for_httperror.py", line 60, in raise_for_httperror
        raise_for_captcha(resp)
      File "share/searx/searx/raise_for_httperror.py", line 43, in raise_for_captcha
        raise_for_cloudflare_captcha(resp)
      File "share/searx/searx/raise_for_httperror.py", line 30, in raise_for_cloudflare_captcha
        raise SearxEngineCaptchaException(message='Cloudflare CAPTCHA', suspended_time=3600 * 24 * 15)
    searx.exceptions.SearxEngineCaptchaException: Cloudflare CAPTCHA, suspended_time=1296000

For SearxEngineResponseException this is not needed.  Those types of exceptions
can be a normal use case.  E.g. for CAPTCHA errors like shown in the example
above. It should be enough to log a warning for such issues:

    WARNING:searx.engines:yggtorrent engine: Fail to initialize // Cloudflare CAPTCHA, suspended_time=1296000

closes: #2612

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-03-05 17:26:22 +01:00
Alexandre Flament 0165e14a7f
Merge pull request #2632 from searx/update_data_update_wikidata_units.py 
Update searx.data - update_wikidata_units.py
 2021-03-05 11:59:44 +01:00
Alexandre Flament 152f6fc1da
Merge pull request #2630 from searx/update_data_update_ahmia_blacklist.py 
Update searx.data - update_ahmia_blacklist.py
 2021-03-05 11:59:20 +01:00
dalf 1e8b846954 Update searx.data - update_currencies.py 2021-03-05 10:56:57 +00:00
dalf 2f8a708481 Update searx.data - update_wikidata_units.py 2021-03-05 10:56:49 +00:00
dalf d9dc3376d0 Update searx.data - update_languages.py 2021-03-05 10:56:46 +00:00
dalf 2857473553 Update searx.data - update_ahmia_blacklist.py 2021-03-05 10:56:33 +00:00
Alexandre Flament aac37f288f
Merge pull request #2593 from dalf/update-autocomplete 
Update autocomplete
 2021-03-04 10:51:09 +01:00
Alexandre Flament 63f17d2e4c [enh] autocomplete refactoring, autocomplete on external bangs 2021-03-01 19:12:32 +01:00
Markus Heiser d48e2e7b0b [enh] google scholar - python implementation of the engine 
The old xpath configuration for google scholar did not work and is replaced by a
python implementation.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-03-01 15:16:37 +01:00
Alexandre Flament 4fa1290c11 [fix] answers: don't crash when the query is an empty string 2021-03-01 10:52:39 +01:00
Alexandre Flament e2fb500892
Merge pull request #2608 from return42/unittest2 
[py2to3] use unittest from py3, remove unittest2 from py2
 2021-03-01 10:05:38 +01:00
Alexandre Flament 0c663e25fc
Merge pull request #2604 from searx/update_data_firefox_version 
Update searx.data - firefox_version
 2021-03-01 10:03:39 +01:00
Alexandre Flament f77983e174
Merge pull request #2602 from MarcAbonce/fix-bing-fetch-languages 
Fix fetch_languages for Bing
 2021-03-01 09:06:37 +01:00
GazoilKerozen 5f6ac3afa2
Add Freesound engine (#2596) 
Add freesound engine with player.

Co-authored-by: Gazoil <maildeguzel@gmail.com>
 2021-03-01 08:52:36 +01:00
Markus Heiser 3bae35940a [py2to3] use unittest from py3, remove unittest2 from py2 
- unittest2 is a backport of the new features added to the unittest testing
  framework in Python 2.7

- unittest2 was only needed in py2 and can be dropped now

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 2021-02-28 11:37:06 +01:00
Alexandre Flament b05f4d0664
Merge pull request #2605 from searx/update_data_currencies 
Update searx.data - currencies
 2021-02-26 12:56:32 +01:00
Alexandre Flament aec5188b51
Merge pull request #2606 from searx/update_data_wikidata_units 
Update searx.data - wikidata_units
 2021-02-26 12:55:51 +01:00
dalf 893b6e4901 Update searx.data - ahmia_blacklist 2021-02-26 08:31:15 +00:00
dalf 7b9005df31 Update searx.data - wikidata_units 2021-02-26 08:31:01 +00:00
dalf 4c8ae5b7ed Update searx.data - firefox_version 2021-02-26 08:30:45 +00:00
dalf d2778b5efe Update searx.data - currencies 2021-02-26 08:30:45 +00:00
Marc Abonce Seguin d6681fd33b remove articles number from engines_languages.json 2021-02-25 23:54:21 -07:00
Marc Abonce Seguin 9b6ffed061 fix fetch_languages for bing 
Bing has a list of regions that it supports and some of these regions
may have more than one possible language.

In some cases, like Switzerland, these languages are always shown as
options, so there is no issue. But in other cases, like Andorra, Bing
will only show one language at the time, either the region's default or
the request's language if the latter is supported by that region.

For example, if the HTTP request is in French, Andorra will appear as
fr-AD but if the same page is requested in any other language Andorra
will appear as ca-AD.

This is specially a problem when Bing assumes that the request is in
English because it overrides enough language codes to make several major
languages like Arabic dissappear from the languages.py file.

To avoid that issue, I set the Accept-Language header to a language
that's only supported in one region to hopefully avoid these overrides.
 2021-02-25 23:51:49 -07:00
Alexandre Flament 7c1847d5f2 [mod] add utils/fetch_external_bangs.py 
Based on duckduckgo bangs
Store bangs on a trie to allow autocomplete (not in this commit)
 2021-02-24 18:48:36 +01:00
Alexandre Flament 5f4a085fc4
Merge pull request #2595 from dalf/update-wikidata-units 
[mod] update wikidata_units.json and fetch_wikidata_units.py
 2021-02-23 17:22:37 +01:00
Alexandre Flament 46ca32c3cc [mod] update currencies.json and fetch_currencies.py 
use a sparql request on wikidata to get the list of currencies.

currencies.json contains the translation for all supported searx languages.

Supersede #993
 2021-02-23 16:42:28 +01:00
Alexandre Flament 93d1da4906 [mod] update wikidata_units.json and fetch_wikidata_units.py 
The fetch_wikidata_units.py result won't change randomly.
See comments in the script.
 2021-02-23 13:10:38 +01:00
Noémi Ványi 1be6ab2a91 Fix paging of Bing Images 2021-02-22 21:19:34 +01:00