b1fefec40d
[fix] normalize the language & region aspects of all google engines
...
BTW: make the engines ready for search.checker:
- replace eval_xpath by eval_xpath_getindex and eval_xpath_list
- google_images: remove outer try/except block
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-28 10:08:46 +01:00
8cdad5d85d
[fix] google-videos: parse values for 'length' & 'author'
...
The 'video.html' template from the 'oscar' design supports replacement
for *author* and *length*. Google-videos does not have an author, alternatively
the publisher info from is used for the *author*.
Hint: these replacements are not supported by the 'simple' design.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-24 09:51:24 +01:00
89b3050b5c
[fix] revise of the google-Video engine
...
This revise is based on the methods developed in the revise of the google engine
(see commit 410c2f9
2021-01-24 09:39:30 +01:00
8c46b767d0
[fix] google_news: avoid one HTTP redirect except for the English results
...
also add
params['soft_max_redirects'] = 1
to avoid false error reporting in /stats/errors
2021-01-24 08:53:35 +01:00
5f92dfcdbe
[fix] google-news: query uses locale without country tag
...
Wthout country-region tag google will redirect to correct the contry tag [1]:
SEARX_DEBUG=1 searx-checker -v "google news"
...
https://news.google.com:443 "GET /search?q=computer&hl=en... HTTP/1.1" 302 0
https://news.google.com:443 "GET /search?q=computer&hl=en-US&.... HTTP/1.1" 200 None
...
[1] https://github.com/searx/searx/pull/2483#issuecomment-765600849
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-23 11:37:14 +01:00
baec54c492
[fix] revise of the google-news engine
...
This revise is based on the methods developed in the revise of the google engine
(see commit 410c2f9
2021-01-22 18:49:45 +01:00
b405646749
Merge pull request #2451 from mrwormo/invidious-engine
...
[Fix] Invidious Engine
2021-01-16 19:25:45 +01:00
a4dcfa025c
[enh] engines: add about variable
...
move meta information from comment to the about variable
so the preferences, the documentation can show these information
2021-01-14 20:57:17 +01:00
2dff3887f0
[fix] Invidious engine by enabling requests by randomly picking amongst working instances
2021-01-14 12:12:56 +01:00
3f8ebf70b1
[fix] pylint: use "raise ... from ..."
2020-12-20 09:46:53 +01:00
eb33ae6893
[fix] Python 3.9: use html.unescape instead of HTMLParser.unescape
2020-12-20 09:46:53 +01:00
02fc4147ce
[mod] dictzone, translated, currency_convert: use engine_type online_curency and online_dictionnary
2020-12-17 11:39:36 +01:00
7ec8bc3ea7
[mod] split searx.search into different processors
...
see searx.search.processors.abstract.EngineProcessor
First the method searx call the get_params method.
If the return value is not None, then the searx call the method search.
2020-12-17 11:39:36 +01:00
fea8958e99
Fix the StartPage result title is showing the url
...
Fix the issue 2395 where StartPage result title is showing the url. https://github.com/searx/searx/issues/2395
2020-12-16 13:54:14 -08:00
292b73a3fc
Merge pull request #2385 from joshu9h/patch-1
...
[Fix] Startpage
2020-12-14 17:56:48 +01:00
36600118fb
Merge pull request #2372 from dalf/remove-broken-engines
...
[remove] remove searchcode_doc and twitter
2020-12-13 17:11:05 +01:00
8260435c8b
[Fix] Startpage
2020-12-13 15:43:50 +01:00
3c4a9c1188
Merge pull request #2358 from dalf/fix-command
...
[fix] command engine: SearchQuery.query is str not bytes
2020-12-11 14:53:24 +01:00
d703119d3a
[enh] add raise_for_httperror
...
check HTTP response:
* detect some comme CAPTCHA challenge (no solving). In this case the engine is suspended for long a time.
* otherwise raise HTTPError as before
the check is done in poolrequests.py (was before in search.py).
update qwant, wikipedia, wikidata to use raise_for_httperror instead of raise_for_status
2020-12-11 14:37:08 +01:00
033f39bff7
Merge pull request #2376 from dalf/fix-mojeek
...
Fix mojeek
2020-12-11 13:14:54 +01:00
6bc6d5e9fd
Merge pull request #2371 from dalf/mod-genius
...
[mod) genious: return valid results even if contents are empty
2020-12-11 13:14:03 +01:00
d41cafd5f3
[fix] xpath, mojeek: fix commit 58d72f2692
...
before commit 58d72f258d72f2close #2368
2020-12-10 10:52:06 +01:00
3a63dfbdd7
display if an engine does not support https
...
Closes #302
2020-12-09 20:49:54 +01:00
1c9e7cef50
[remove] remove searchcode_doc and twitter
...
* twitter: the API has changed. the engine needs to rewritten.
* searchcode_doc: the API about documentation doesn't exist anymore.
2020-12-09 13:14:31 +01:00
fa73f10f11
[mod) genious: return valid results even if contents are empty
2020-12-09 13:01:34 +01:00
a77d8c8227
Merge pull request #2359 from dalf/update-duden
...
[mod] duden engine
2020-12-08 20:33:38 +01:00
bd4869ecd0
Merge pull request #2366 from dalf/remove-seedpeer
...
[remove] seedpeer engine
2020-12-08 20:33:23 +01:00
56c64d6b64
[remove] seedpeer engine
...
the website is offline.
2020-12-07 21:02:29 +01:00
c1a9732268
Merge pull request #2364 from dalf/fix-youtube-noapi
...
[fix] youtube_noapi engine
2020-12-07 20:26:00 +01:00
13d3004703
Merge pull request #2365 from dalf/fix-soundcloud
...
[fix] soundclound: accept result without content
2020-12-07 20:25:17 +01:00
62073c0e1d
Merge pull request #2361 from dalf/fix-1x
...
[fix] 1x engine
2020-12-07 20:24:47 +01:00
923bc02c17
Merge pull request #2363 from dalf/fix-wikipedia-minor
...
[fix] wikipedia: minor fix: return no result instead of crash in some very few cases.
2020-12-07 18:33:37 +01:00
deb1bde20d
[fix] soundclound: accept result without content
2020-12-07 17:45:36 +01:00
34df0f7910
[fix] youtube_noapi engine
2020-12-07 17:44:31 +01:00
58d51e082d
[fix] wikipedia: minor fix: return no result instead of crash in some very few cases.
...
In few cases, the JSON results doesn't contains the key 'type'.
2020-12-07 17:42:05 +01:00
4ec810749b
[fix] 1x engine
2020-12-07 15:46:00 +01:00
1e781863fa
[fix] command engine: SearchQuery.query is str not bytes
...
see c225db45c8
2020-12-07 10:43:42 +01:00
9bf594cbcf
[mod] duden engine
...
* add params['soft_max_redirects'] = 1 (when there is spelling suggestion)
* avoid try..except
* use eval_xpath_* functions
2020-12-07 10:31:11 +01:00
a458451d20
Merge pull request #2356 from dalf/fix-ddd
...
[fix] duckduckgo_definitions: fix relative image URL
2020-12-07 10:16:53 +01:00
925bb561a2
Merge pull request #2352 from dalf/no_http
...
Remove HTTP connections as much as possible
2020-12-06 10:18:49 +01:00
28cc644f0a
[fix] duckduckgo_definitions: fix relative image URL
...
ddg returns relative URL to https://duckduckgo.com/
2020-12-06 10:14:09 +01:00
cdceec1cbb
Merge pull request #2354 from dalf/fix-wikipedia
...
[fix] wikipedia engine: don't raise an error when the query is not found
2020-12-04 20:42:45 +01:00
f0054d67f1
[fix] wikipedia engine: don't raise an error when the query is not found
...
Add a new parameter "raise_for_status", set by default to True.
When True, any HTTP status code >= 300 raise an exception ( #2332 )
When False, the engine can manage the HTTP status code by itself.
2020-12-04 20:04:39 +01:00
bef2f2efa8
[fix] wikidata: fix crash when the item has no description at all and at least one URL.
2020-12-04 17:17:20 +01:00
244e812f37
[fix] remove searx/engines/filecrop.py (dead code)
2020-12-04 16:48:15 +01:00
fa909c7c02
[mod] stackoverflow & yandex: detect CAPTCHA response
2020-12-03 13:23:19 +01:00
64cccae99e
[mod] various engines: use eval_xpath* functions and searx.exceptions.*
...
Engine list: ahmia, duckduckgo_images, elasticsearch, google, google_images, google_videos, youtube_api
2020-12-03 10:22:48 +01:00
ad72803ed9
[mod] xpath, 1337x, acgsou, apkmirror, archlinux, arxiv: use eval_xpath_* functions
2020-12-03 10:22:48 +01:00
de887c6347
[mod] bing_news: use eval_xpath_getindex
...
remove unused function searx.utils.list_get
2020-12-03 10:22:48 +01:00
1d0c368746
[enh] record details exception per engine
...
add an new API /stats/errors
2020-12-03 10:22:48 +01:00
Markus Heiser