Commit Graph

620 Commits

Author SHA1 Message Date
Alexandre FLAMENT e92755d358 Initialize Redis in searx/webapp.py
settings.yml:
* The default URL was unix:///usr/local/searxng-redis/run/redis.sock?db=0
* The default URL is now "false"

The default URL makes the log difficult to deal with:
if the admin didn't install a Redis instance, the logs record a false error.

It worked before because SearXNG initialized the Redis connection when the limiter started.

In this commit, SearXNG initializes Redis in searx/webapp.py
so various components can use Redis without taking care of the initialization step.
2022-11-05 17:45:52 +01:00
Alexandre Flament 32e8c2cf09 searx.network: add "verify" option to the networks
Each network can define a verify option:
* false to disable certificate verification
* a path to existing certificate.

SearXNG uses SSL_CERT_FILE and SSL_CERT_DIR when they are defined
see https://www.python-httpx.org/environment_variables/#ssl_cert_file
2022-10-14 13:59:22 +00:00
Mohamed Elashri 8d5653e60d
Merge branch 'searxng:master' into master 2022-09-30 23:06:54 +00:00
Markus Heiser ba8959ad7c [fix] typos / reported by @kianmeng in searx PR-3366
[PR-3366] https://github.com/searx/searx/pull/3366

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-09-27 18:32:14 +02:00
Mohamed Elashri 5832c70680
correct sci-hub links/ add `.ru` and remove other 3rd party domains. 2022-09-24 11:03:57 -04:00
Markus Heiser caebafdd06 [fix] typo in crossref settings: disable --> disabled
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-09-24 08:12:36 +02:00
Alexandre Flament d6446be38f [mod] science category: various update of about PR 1705 2022-09-23 20:52:55 +02:00
Alexandre FLAMENT e36f85b836 Science category: update the engines
* use the paper.html template
* fetch more data from the engines
* add crossref.py
2022-09-23 20:45:58 +02:00
Alexandre Flament bef3984d03
Merge pull request #1728 from liimee/eng-ddw
add duckduckgo weather engine
2022-09-23 18:14:09 +02:00
Alexandre Flament d3fec1388c
Merge pull request #1624 from liimee/eng-wttr
Add wttr.in engine
2022-09-23 18:13:37 +02:00
Alexandre FLAMENT 33b43763b9 Brave engine: fix BrotliDecoderDecompressStream error 2022-09-18 22:08:38 +00:00
LencoDigitexer 3f72a79088 add yandex to autocomplete backends settings 2022-09-09 23:50:58 +03:00
Markus Heiser ad8ffd222c [mod] option 'ui: cache_url:' to configure internet cache or archive
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-09-04 09:42:20 +02:00
Alexandre FLAMENT dd0887be18 xpath engine: change raise_for_httperror to no_result_for_http_status
no_result_for_http_status contains a list of HTTP status.
These HTTP status are seen an empty result list.
In other cases an exception is thrown as usual.

Previously raise_for_httperror were ignoring all HTTP error,
which make defective engines invisible in the stats.
2022-09-04 09:07:28 +02:00
Markus Heiser a15dfa5ee1 [fix] engine woxikon.de - don't raise exception on empty result list
Woxikon expects a word in German, so with query "foo" the site finds nothing and
respons a 404:

    httpx.HTTPStatusError: Client error '404 Not Found' \
      for url 'https://synonyme.woxikon.de/synonyme/foo.php'

[1] https://github.com/searxng/searxng/issues/1543#issuecomment-1193317054

Closes: https://github.com/searxng/searxng/issues/1543
Suggested-by: @allendema [1]
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-09-04 09:07:28 +02:00
Markus Heiser 8e9fb0b435
Merge pull request #1647 from return42/deepl-engine
[mod] add deepl translation engine
2022-09-02 14:09:22 +02:00
Émilien Devos fcccf39030
Disable brave by default
Brave is too unstable and will often not work by default. As seen in many issues: https://github.com/searxng/searxng/issues?q=is%3Aissue++sort%3Aupdated-desc+brave+label%3Abug+
2022-08-31 15:47:56 +02:00
ta 12f7d4a46b add duckduckgo weather engine 2022-08-31 17:29:32 +07:00
Alexandre FLAMENT 341ad46303 settings.yml: set default values for result_proxy
* initialize result_proxy with searx/settings_defaults.py
* allow result_proxy.key to be a string

this commit supersedes #1522
2022-08-28 09:27:53 +00:00
Alexandre Flament 56000d5162
Merge pull request #1699 from liimee/eng-app-store
add apple app store engine
2022-08-27 07:43:23 +02:00
Alexandre Flament 44bc94c36e
Merge pull request #1700 from liimee/eng-ddm
add apple maps engine
2022-08-27 07:36:16 +02:00
ta 5dce299b22 add apple maps engine 2022-08-25 17:05:40 +07:00
ta e5c1b64b1d add the apple app store engine
The Apple App Store is the digital app distribution platform for iOS & iPadOS.
2022-08-24 17:27:36 +07:00
ta dd9127492f add 9gag engine
9GAG is a social media website where users upload and share user-generated images and videos
2022-08-22 17:35:07 +07:00
Alexandre Flament 5ed40af3ba
Merge pull request #1661 from liimee/eng-tw
Add twitter engine
2022-08-21 15:21:18 +02:00
Markus Heiser 75bb8c45d0 [mod] decouple qwant's categories from SearXNG's categories
By using new property `qwant_categ:` the category of qwant is no longer bound to
the category of SearXNG.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-08-14 10:26:54 +02:00
ta 96ea355a1f add twitter engine 2022-08-14 08:39:41 +07:00
Léon Tiekötter 8fa84ee9f4 Fix time range support and add soft_max_redirects
Add custom time_range_url and time_range_map

Set soft_max_redirects = 2 to prevent "ErrorContext('searx/search/processors/online.py', 116, 'count_error(', None, '2 redirects, maximum: 0', ('200', 'OK', 'neeva.com')) True"
2022-08-13 07:35:15 +07:00
ta d6dfada1a9 add neeva web engine
Neeva is "the world's first ad-free, private search engine" and uses data from Apple, Bing, Yelp and "others".
They claim to crawl "hundreds of millions" of URLs a day (https://twitter.com/Neeva/status/1536447373903335426).
2022-08-13 07:26:36 +07:00
Johan Planchon e0215e0cc8 add sourcehut engine 2022-08-10 21:48:12 +02:00
Johan Planchon 90bdb23756 add lib.rs engine 2022-08-10 12:17:55 +02:00
Thomas Renard d4acbcfe63 [mod] add deepl translation engine
This implements the Deepl Translation engine. It works nearly like lingva but
directly to the deepl API.  This api only needs a to-lang, from-lang is a fake
by now.

There is a free option to use [1].

[1] https://www.deepl.com/pro-api?cta=header-pro-api for registering a free account.
2022-08-10 09:14:36 +02:00
Markus Heiser 925f3ff487
Merge pull request #1627 from allendema/add-marginalia
[enh] Initial Marginalia.nu support (foss)
2022-08-08 06:53:02 +02:00
Markus Heiser 58f630d312
Merge pull request #1623 from return42/mod-yep.com
[mod] engine yep.com: show all 100 results yep.com has
2022-08-08 06:52:15 +02:00
Allen 5b815b3966 [enh] initial Marginalia.nu support (foss)
Currently it uses a public api_key `/public/` [1]

The 'index' parameter selects the search index, corresponding to the drop down
next to the search field in the main GUI.

    0: popular
    1: blogs
    2: big_sites
    3: default
    4: experimental

'experimental' is more up to date and does not exclude other sites, which is the
case with 'big sites' or 'blogs'.

[1] https://api.marginalia.nu/
[2] https://git.marginalia.nu/marginalia/marginalia.nu
[3] https://news.ycombinator.com/item?id=31536626

Closes: https://github.com/searxng/searxng/issues/1620
2022-08-08 06:31:04 +02:00
Markus Heiser 8b58cd1167 [fix] typo in uWSGI cache: searxcache --> searxngcache
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-08-07 12:31:09 +02:00
ta 8aa018db95 add wttr.in engine 2022-08-07 13:04:18 +07:00
Markus Heiser 647e6187d7 [mod] engine yep.com: show all 100 results yep.com has
yep.com is still in beta, the api.yep.com does not have paging support.  There
is only a 'limit' argument with a maximum of 100 results.

yep.com seems fast; there is nor need for a timeout of 12 sec.

The API returns JSON nevertheless what the HTTP header is, the "show more"
button on yep.com's web site does not set a special HTTP Accept header.

FYI: The index does not support languages, the WEB UI does not offer a language
selection of the results and the entire index seems in English.

Closes: https://github.com/searxng/searxng/issues/1619
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-08-06 13:21:28 +02:00
Allen 58369e41d6 [wip] brave time range support 2022-08-04 04:47:25 +02:00
Markus Heiser 8df1f0c47e [mod] add 'Accept-Language' HTTP header to online processores
Most engines that support languages (and regions) use the Accept-Language from
the WEB browser to build a response that fits to the language (and region).

- add new engine option: send_accept_language_header

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-08-01 17:01:59 +02:00
Markus Heiser 4231a5770b [fix] sjp engine - convert enginename to a latin1 compliance name
The engine name is not only a *name* its also a identifier that is used in
logs, HTTP headers and more.  Unicode characters in the name of an engine could
cause various issues.

Closes: https://github.com/searxng/searxng/issues/1544
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-07-24 21:10:55 +02:00
Solirs 6d646129c3 [mod] add tor_check plugin - convenient tor checking trough searxng 2022-07-19 07:34:54 +02:00
Markus Heiser 4e05197444
Merge pull request #1475 from return42/Emojipedia
[mod] Add engine for Emojipedia
2022-07-15 09:30:40 +02:00
Jay 10edcbe3c2 [mod] Add engine for Emojipedia
Emojipedia is an emoji reference website which documents the meaning and
common usage of emoji characters in the Unicode Standard.  It is owned by Zedge
since 2021. Emojipedia is a voting member of The Unicode Consortium.[1]

Cherry picked from @james-still [2[3] and slightly modified to fit SearXNG's
quality gates.

[1] https://en.wikipedia.org/wiki/Emojipedia
[2] 2fc01eb20f
[3] https://github.com/searx/searx/pull/3278
2022-07-15 09:26:44 +02:00
Brock Vojković 84e2a3bd3f Add infinite scroll as a setting in settings.yml 2022-07-09 17:33:25 +00:00
Alexandre Flament df837d8b1b
Merge pull request #1428 from return42/fix-center_aligment
fix typo and document preference 'center_alignment' in the 'ui' section
2022-07-07 09:43:12 +02:00
ta 14756a2674 [mod] Adds Lingva translate engine
Add the lingva engine (which grabs data from google translate).  Results from
Lingva are added to the infobox results.
2022-07-04 19:06:45 +02:00
Markus Heiser 241bde5ae3 [fix] typo: add missing 'n' in center_aligment --> center_alignment
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-07-03 17:35:54 +02:00
Alexandre Flament f8f239fe1f Donation link: default value to searxng.org, can be hidden or custom
Add a new setting: general.donation_url

By default the value is https://docs.searxng.org/donate.html

When the value is false, the link is hidden

When the value is true, the link goes to the infopage donation,
the administrator can create a custom page.
2022-07-02 11:29:21 +02:00
Markus Heiser ed1ff4b8bc
Merge pull request #1357 from liimee/eng-3
Add curlie.org engine
2022-06-22 18:29:36 +02:00