searxng/searx
Markus Heiser 08ef986e58 [POC] limiter: change PING of link_token mehtod from CSS to <img>
while PR #2357 [1] was being implemented the question came up:

    would be better to change the PING resource from CSS to an image so that
    some terminal based browser may still able to pass the test [1]

This patch implements a POC in where a <img src=token> tag is loaded instaed a
CSS.

To test this patch activate limiter and link_token method [3] and start a
developer instance::

    make run

In your terminal browser open http://127.0.0.1:8888/search?q=foo

If the browser is suitable for the link_token method, it loads the image and the
following messages appear::

    DEBUG   searx.botdetection.limiter    : OK 127.0.0.1/32: /clientft61aak7fzyu6o6v.svg ...
    DEBUG   searx.botdetection.link_token : token is valid --> True
    DEBUG   searx.botdetection.link_token : store ping_key for (client) network 127.0.0.1/32 (IP 127.0.0.1) -> SearXNG_limiter.ping[...]

Browsers that do not load images will be blocked: If you try by example::

    lynx http://127.0.0.1:8888/search?q=foo

you will see a WARNING message like::

    WARNING searx.botdetection.link_token : missing ping (IP: 127.0.0.1/32) / request: SearXNG_limiter.ping[...]

Modern terminal WEB browser do support `<img>` tag as well as CSS:

    browsh http://127.0.0.1:8888/search?q=foo

----

[1] 80aaef6c95
[2] https://github.com/searxng/searxng/pull/2357#issuecomment-1574898834
[3] activate limiter and link_token method

```diff
diff --git a/searx/botdetection/limiter.toml b/searx/botdetection/limiter.toml
index 71a231e8f..7e1dba755 100644
--- a/searx/botdetection/limiter.toml
+++ b/searx/botdetection/limiter.toml
@@ -17,6 +17,6 @@ ipv6_prefix = 48
 filter_link_local = false

 # acrivate link_token method in the ip_limit method
-link_token = false
+link_token = true

diff --git a/searx/settings.yml b/searx/settings.yml
index a82a3432d..e7b983afc 100644
--- a/searx/settings.yml
+++ b/searx/settings.yml
@@ -73,7 +73,7 @@ server:
   # public URL of the instance, to ensure correct inbound links. Is overwritten
   # by ${SEARXNG_URL}.
   base_url: false  # "http://example.com/location"
-  limiter: false  # rate limit the number of request on the instance, block some bots
+  limiter: true  # rate limit the number of request on the instance, block some bots

   # If your instance owns a /etc/searxng/settings.yml file, then set the following
   # values there.
```

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2024-11-01 13:39:35 +01:00
..
answerers [fix] setup.py - packaging for 'answerers' to build a valid package 2024-06-17 15:07:07 +02:00
botdetection [fix] add missing tomli to the requirements.txt 2024-10-26 15:59:42 +02:00
data [data] update searx.data - update_firefox_version.py 2024-10-29 09:30:40 +01:00
enginelib [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
engines [fix] annas archive: crash when no thumbnail, differing results, paging 2024-11-01 12:49:33 +01:00
favicons [fix] favicons: msgspec.ValidationError: Expected Path, got str - at $.favicons.cache.db_url 2024-10-29 18:22:22 +01:00
infopage [mod] add French translation for infopage 2024-08-20 07:25:39 +02:00
metrics [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
network [mod] remove py 3.6 leftovers 2024-07-13 17:20:50 +02:00
plugins [fix] calculator: use locale from UI (not from selected language) 2024-10-28 15:53:57 +01:00
search [fix] online_currency.py: if more alternatives, use the last in the list 2024-10-15 15:20:39 +02:00
static [build] /static 2024-10-26 08:34:21 +02:00
templates/simple [POC] limiter: change PING of link_token mehtod from CSS to <img> 2024-11-01 13:39:35 +01:00
translations [l10n] update translations from Weblate 2024-11-01 08:30:38 +01:00
__init__.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
autocomplete.py [fix] html.unescape stract autocomplete suggestions 2024-07-02 18:16:46 +02:00
babel_extract.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
compat.py [mod] Revision of the favicon solution 2024-10-05 08:18:28 +02:00
exceptions.py [fix] engine: duckduckgo - CAPTCHA detection 2024-10-19 14:55:44 +02:00
external_bang.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
external_urls.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
flaskfix.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
limiter.py [fix] limiter: don't hard code settings folder to /etc/searxng 2024-10-28 12:51:05 +01:00
limiter.toml [mod] isolation of botdetection from the limiter 2023-11-01 06:44:56 +01:00
locales.py [fix] fetch_traits: brave, google, annas_archive & radio_browser 2024-09-15 12:48:35 +02:00
preferences.py [mod] Revision of the favicon solution 2024-10-05 08:18:28 +02:00
query.py [refactor] unit tests to utilize paramaterized and break down monolithic tests 2024-10-03 13:20:32 +02:00
redisdb.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
redislib.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
results.py [fix] search: titles including html brackets are not shown properly 2024-08-31 17:11:49 +02:00
searxng.msg [feat] simple theme: pure black theme style 2024-10-03 10:53:17 +02:00
settings.yml [feat] engine: support for openlibrary 2024-10-15 13:06:00 +02:00
settings_defaults.py [feat] add favicons to result urls 2024-10-05 08:18:28 +02:00
settings_loader.py [mod] revision of the settings_loader 2024-07-14 18:10:06 +02:00
sqlitedb.py [mod] sqlitedb: access to SQLite databases a little more convenient. 2024-10-05 08:18:28 +02:00
sxng_locales.py [data] update searx.data - update_engine_traits.py 2024-10-15 06:37:01 +02:00
unixthreadname.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
utils.py add get_embeded_stream_url to searx.utils 2024-10-03 07:10:53 +02:00
version.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
webadapter.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00
webapp.py [POC] limiter: change PING of link_token mehtod from CSS to <img> 2024-11-01 13:39:35 +01:00
webutils.py [mod] pylint all files with one profile / drop PYLINT_SEARXNG_DISABLE_OPTION 2024-03-11 14:55:38 +01:00