Commit Graph

4505 Commits

Author SHA1 Message Date
Markus Heiser 66fdec0eb9 [mod] limiter: add config file /etc/searxng/limiter.toml
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-06-01 14:38:53 +02:00
Markus Heiser 1ec325adcc [mod] limiter -> botdetection: modularization and documentation
In order to be able to meet the outstanding requirements, the implementation is
modularized and supplemented with documentation.

This patch does not contain functional change, except it fixes issue #2455

----

Aktivate limiter in the settings.yml and simulate a bot request by::

    curl -H 'Accept-Language: de-DE,en-US;q=0.7,en;q=0.3' \
         -H 'Accept: text/html'
         -H 'User-Agent: xyz' \
         -H 'Accept-Encoding: gzip' \
         'http://127.0.0.1:8888/search?q=foo'

In the LOG:

    DEBUG   searx.botdetection.link_token : missing ping for this request: .....

Since ``BURST_MAX_SUSPICIOUS = 2`` you can repeat the query above two time
before you get a "Too Many Requests" response.

Closes: https://github.com/searxng/searxng/issues/2455
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-05-29 14:54:56 +02:00
Markus Heiser 5226044c13 [mod] limiter: add random token to the limiter URL
By adding a random component in the limiter URL a bot can no longer send a ping
by request a static URL.

Related: https://github.com/searxng/searxng/pull/2357#issuecomment-1518525094
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-05-29 14:54:56 +02:00
Markus Heiser dba569462d [mod] limiter: reduce request rates for requests without a ping
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-05-29 14:54:56 +02:00
dalf c1b5ff7e1c Update searx.data - update_engine_descriptions.py 2023-05-29 07:28:50 +02:00
dalf 2ba50d392e Update searx.data - update_currencies.py 2023-05-29 07:28:18 +02:00
dalf cb843ef13c Update searx.data - update_engine_traits.py 2023-05-29 07:27:40 +02:00
dalf 512e001277 Update searx.data - update_firefox_version.py 2023-05-29 07:27:07 +02:00
dalf f03ac9b152 Update searx.data - update_wikidata_units.py 2023-05-29 07:26:47 +02:00
dalf e12e350f7f Update searx.data - update_ahmia_blacklist.py 2023-05-29 07:26:20 +02:00
Markus Heiser 3ca97cf5e3 [fix] simple theme: move engine alerts in case of no results into sidebar
If there were no results but errors in the engines then the error dialogs of the
engines was displayed in the result list.

With the new design errors of the engines should only be displayed in the
sidebar and at the same time duplications of the (template) code will be
avoided.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-05-28 12:19:32 +02:00
mrpaulblack 60b94dfdca [build] /static 2023-05-28 12:19:32 +02:00
mrpaulblack f087959b02 [mod] simple theme: build design for details (collapsables)
* set border top and bottom on sidebar collasables
* inrease peading on summary so its easier to click on mobile
* remove margins and add flex wrapper to normalize elements in sidebar
2023-05-28 12:19:32 +02:00
Markus Heiser b7e315563d [mod] simple theme: collaps/expand elements in the sidebar
Make elements in the sidebar collapse able.  Except infoboxes all elements in
the sidebar are collapsed by default.

By folding out the sidebar elements, the UI looks less cluttered.  Especially on
small devices like smartphones, where the sidebar is above the results list, the
UX should be improved [1].

[1] https://github.com/searxng/searxng/issues/2140

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-05-28 12:19:32 +02:00
searxng-bot e8054026fb [translations] update from Weblate
69171f12 - 2023-05-25 - fabiosantoscode <fabiosantosart@gmail.com>
2caaed0a - 2023-05-23 - trmx <borcan.cristian1@gmail.com>
84d1702b - 2023-05-21 - return42 <markus.heiser@darmarit.de>
65cc6eb8 - 2023-05-21 - return42 <markus.heiser@darmarit.de>
e0ab3383 - 2023-05-22 - return42 <markus.heiser@darmarit.de>
23e87f15 - 2023-05-21 - return42 <markus.heiser@darmarit.de>
14f0fc6b - 2023-05-21 - return42 <markus.heiser@darmarit.de>
5b7c7b7d - 2023-05-21 - return42 <markus.heiser@darmarit.de>
c725b38d - 2023-05-21 - return42 <markus.heiser@darmarit.de>
2023-05-26 07:08:21 +00:00
Markus Heiser ebe22a4319 [fix] typo: dues --> does
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-05-22 13:18:22 +02:00
Markus Heiser bc647fabaf [fix] ClientPref - don't raise exception if Accept-Language is invalid
If the Accept-Language header [1] is set but empty or holds a value that is
unknown to babel, an excpetion is raised::

    $ curl --header 'Accept-Language: xyz' 'http://127.0.0.1:8888/search?q=foo'
    ...
    Traceback (most recent call last):
      File "searx/preferences.py", line 335, in from_http_request
        return cls(locale=pairs[0][0])
    IndexError: list index out of range

[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept-Language

Reported by: @Eolien55 in https://github.com/searxng/searxng/issues/2434#issuecomment-1556199789
Closes: https://github.com/searxng/searxng/issues/2434
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-05-22 12:38:59 +02:00
Markus Heiser bab730c8a8
Merge pull request #2446 from searxng/translations_update
Update translations
2023-05-19 19:59:58 +02:00
pankaj 4900c091a6 use logger.warning
logger.warn() is depricated.
logger.warning is already being used in some files.
2023-05-19 19:35:29 +05:30
searxng-bot 23b53a03c2 [translations] update from Weblate
2eeec66c - 2023-05-13 - return42 <markus.heiser@darmarit.de>
87058e51 - 2023-05-13 - return42 <markus.heiser@darmarit.de>
2023-05-19 07:07:56 +00:00
Markus Heiser 007a615ffa [mod] donation_url: disable by default
SearXNG's donation campaign has been ended.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-05-15 09:19:17 +02:00
Markus Heiser 6625b6193e
Merge pull request #2420 from searxng/translations_update
Update translations
2023-05-13 06:06:07 +02:00
Markus Heiser caebd297e9 [fix] engine ddg: minor change in the API of ddg
Closes: https://github.com/searxng/searxng/issues/2419
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-05-12 18:58:49 +02:00
searxng-bot f5c407eaa0 [translations] update from Weblate
df7e1be3 - 2023-05-10 - return42 <markus.heiser@darmarit.de>
7ae9877e - 2023-05-08 - return42 <markus.heiser@darmarit.de>
c2fe5131 - 2023-05-07 - KDesp73 <kdesp2003@gmail.com>
2023-05-12 07:07:40 +00:00
Markus Heiser a60851bd59 [fix] version format string generated by 'git show'
Newer versions of git [1] do no longer support a format string that includes a minus
to remove leading zeros [2].  The format string '%Y.%m.%d'  is more version rod.

[1] https://github.com/searxng/searxng/issues/2413#issuecomment-1542320387
[2] https://github.com/searxng/searxng/pull/2122/files

Closes: https://github.com/searxng/searxng/issues/2413
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-05-10 18:33:45 +02:00
Markus Heiser db95c4713b
Merge pull request #2406 from searxng/dependabot/npm_and_yarn/searx/static/themes/simple/master/grunt-contrib-cssmin-5.0.0
Bump grunt-contrib-cssmin from 4.0.0 to 5.0.0 in /searx/static/themes/simple
2023-05-06 05:54:15 +02:00
dependabot[bot] 673837f83e
Bump grunt-contrib-cssmin in /searx/static/themes/simple
Bumps [grunt-contrib-cssmin](https://github.com/gruntjs/grunt-contrib-cssmin) from 4.0.0 to 5.0.0.
- [Release notes](https://github.com/gruntjs/grunt-contrib-cssmin/releases)
- [Changelog](https://github.com/gruntjs/grunt-contrib-cssmin/blob/main/CHANGELOG)
- [Commits](https://github.com/gruntjs/grunt-contrib-cssmin/compare/v4.0.0...v5.0.0)

---
updated-dependencies:
- dependency-name: grunt-contrib-cssmin
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-05-05 07:57:12 +00:00
searxng-bot 54c4d08167 [translations] update from Weblate
70336613 - 2023-05-03 - return42 <markus.heiser@darmarit.de>
55d82b96 - 2023-05-03 - artnay <jiri.gronroos@iki.fi>
3911fe35 - 2023-05-03 - return42 <markus.heiser@darmarit.de>
81b6ebd1 - 2023-05-03 - return42 <markus.heiser@darmarit.de>
6655ac63 - 2023-05-01 - return42 <markus.heiser@darmarit.de>
3b9cccb8 - 2023-04-30 - return42 <markus.heiser@darmarit.de>
51601c0b - 2023-04-30 - return42 <markus.heiser@darmarit.de>
2023-05-05 07:07:41 +00:00
Markus Heiser 823c490c84 [mod] limiter: block requests from PetalBot
Block requests from PetalBlock.  Normally robots.txt is enough to stop
PetalBlock from making requests [1].  However, if SearXNG is offered below a
path (example.org/search), then the robots.txt is not available in the root
paths of the domain / subdomain.

[1] https://webmaster.petalsearch.com/site/petalbot

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-04-30 09:49:26 +02:00
Markus Heiser 7b43f98ebb
Merge pull request #2382 from searxng/translations_update
Update translations
2023-04-30 08:11:00 +02:00
Markus Heiser e0c8e1923d
Merge pull request #2390 from searxng/update_data_update_wikidata_units.py
Update searx.data - update_wikidata_units.py
2023-04-30 07:16:31 +02:00
Markus Heiser d63dbb10fc
Merge pull request #2391 from searxng/update_data_update_firefox_version.py
Update searx.data - update_firefox_version.py
2023-04-30 07:15:55 +02:00
Markus Heiser e9fdfab76e
Merge pull request #2392 from searxng/update_data_update_currencies.py
Update searx.data - update_currencies.py
2023-04-30 07:15:28 +02:00
Markus Heiser 836827517d
Merge pull request #2393 from searxng/update_data_update_ahmia_blacklist.py
Update searx.data - update_ahmia_blacklist.py
2023-04-30 07:14:53 +02:00
Markus Heiser cfc01ea068
Merge pull request #2394 from searxng/update_data_update_engine_traits.py
Update searx.data - update_engine_traits.py
2023-04-30 07:14:23 +02:00
dalf c2fbace534 Update searx.data - update_engine_descriptions.py 2023-04-29 00:24:27 +00:00
dalf 4f31ab7d4b Update searx.data - update_engine_traits.py 2023-04-29 00:14:09 +00:00
dalf df4cc070ec Update searx.data - update_ahmia_blacklist.py 2023-04-29 00:13:21 +00:00
dalf 5b93f97fb2 Update searx.data - update_currencies.py 2023-04-29 00:13:19 +00:00
dalf 7c90a6a222 Update searx.data - update_firefox_version.py 2023-04-29 00:13:09 +00:00
dalf 4336f70b59 Update searx.data - update_wikidata_units.py 2023-04-29 00:13:08 +00:00
searxng-bot 274fcd46b6 [translations] update from Weblate
b6658877 - 2023-04-27 - return42 <markus.heiser@darmarit.de>
d7a3917b - 2023-04-25 - return42 <markus.heiser@darmarit.de>
879248ad - 2023-04-25 - return42 <markus.heiser@darmarit.de>
6ccafe4e - 2023-04-25 - return42 <markus.heiser@darmarit.de>
d202aed8 - 2023-04-23 - Parsa Ranjbar <parsa@disr.it>
2023-04-28 07:08:09 +00:00
Markus Heiser 45529f51a1
Merge pull request #2347 from return42/mod-lang-detection
If language recognition fails use the Accept-Language
2023-04-25 15:46:26 +02:00
Jakub Łukasiewicz 7da47edd93 [mod] external bang: go to main instead of search page when query is empty
Closes: #2368
2023-04-25 15:02:34 +02:00
Markus Heiser 4080eca3dd [build] /static 2023-04-21 13:38:29 +02:00
searxng-bot b41e47eea6 [translations] update from Weblate
72d42638 - 2023-04-15 - tentsbet <remendne@pentrens.jp>
560e1885 - 2023-04-16 - return42 <markus.heiser@darmarit.de>
7370b026 - 2023-04-16 - return42 <markus.heiser@darmarit.de>
20946697 - 2023-04-16 - return42 <markus.heiser@darmarit.de>
2023-04-21 07:07:58 +00:00
Markus Heiser 9b575a997b [fix] doc of locales.get_engine_locale() / zh-classical is missleading
Wikipedia's zh-classical is not zh_Hant (see doc-string of engines.wikipedia).
Fixed the example in the doc-string of locales.get_engine_locale() to 'zh_TW'.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-04-17 08:48:57 +02:00
Markus Heiser f1b6351ae1 [fix] engine: google play movies
Closes: https://github.com/searxng/searxng/pull/1746
Closes: https://github.com/searxng/searxng/issues/1599

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-04-16 19:15:44 +02:00
Marc Abonce Seguin be1f6ee1c4 Update searx.data - update_engine_traits.py 2023-04-16 08:40:44 +02:00
Markus Heiser 8adbc4fcec [mod] settings.yml: enable language detection by default_lang (auto)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-04-15 22:24:59 +02:00