while PR #2357 [1] was being implemented the question came up:
would be better to change the PING resource from CSS to an image so that
some terminal based browser may still able to pass the test [1]
This patch implements a POC in where a <img src=token> tag is loaded instaed a
CSS.
To test this patch activate limiter and link_token method [3] and start a
developer instance::
make run
In your terminal browser open http://127.0.0.1:8888/search?q=foo
If the browser is suitable for the link_token method, it loads the image and the
following messages appear::
DEBUG searx.botdetection.limiter : OK 127.0.0.1/32: /clientft61aak7fzyu6o6v.svg ...
DEBUG searx.botdetection.link_token : token is valid --> True
DEBUG searx.botdetection.link_token : store ping_key for (client) network 127.0.0.1/32 (IP 127.0.0.1) -> SearXNG_limiter.ping[...]
Browsers that do not load images will be blocked: If you try by example::
lynx http://127.0.0.1:8888/search?q=foo
you will see a WARNING message like::
WARNING searx.botdetection.link_token : missing ping (IP: 127.0.0.1/32) / request: SearXNG_limiter.ping[...]
Modern terminal WEB browser do support `<img>` tag as well as CSS:
browsh http://127.0.0.1:8888/search?q=foo
----
[1] 80aaef6c95
[2] https://github.com/searxng/searxng/pull/2357#issuecomment-1574898834
[3] activate limiter and link_token method
```diff
diff --git a/searx/botdetection/limiter.toml b/searx/botdetection/limiter.toml
index 71a231e8f..7e1dba755 100644
--- a/searx/botdetection/limiter.toml
+++ b/searx/botdetection/limiter.toml
@@ -17,6 +17,6 @@ ipv6_prefix = 48
filter_link_local = false
# acrivate link_token method in the ip_limit method
-link_token = false
+link_token = true
diff --git a/searx/settings.yml b/searx/settings.yml
index a82a3432d..e7b983afc 100644
--- a/searx/settings.yml
+++ b/searx/settings.yml
@@ -73,7 +73,7 @@ server:
# public URL of the instance, to ensure correct inbound links. Is overwritten
# by ${SEARXNG_URL}.
base_url: false # "http://example.com/location"
- limiter: false # rate limit the number of request on the instance, block some bots
+ limiter: true # rate limit the number of request on the instance, block some bots
# If your instance owns a /etc/searxng/settings.yml file, then set the following
# values there.
```
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
In order to be able to meet the outstanding requirements, the implementation is
modularized and supplemented with documentation.
This patch does not contain functional change, except it fixes issue #2455
----
Aktivate limiter in the settings.yml and simulate a bot request by::
curl -H 'Accept-Language: de-DE,en-US;q=0.7,en;q=0.3' \
-H 'Accept: text/html'
-H 'User-Agent: xyz' \
-H 'Accept-Encoding: gzip' \
'http://127.0.0.1:8888/search?q=foo'
In the LOG:
DEBUG searx.botdetection.link_token : missing ping for this request: .....
Since ``BURST_MAX_SUSPICIOUS = 2`` you can repeat the query above two time
before you get a "Too Many Requests" response.
Closes: https://github.com/searxng/searxng/issues/2455
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
The word "hackable" may arouse interest in programmers to participate in the
development, but it scares the ordinary user.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Some administrators may want to hide the link to public_instances: If you run a
private instance for a company that wants there employees to use the private
instance instead of any of the public instances.
Add a new setting: general.donation_url
By default the value is https://docs.searxng.org/donate.html
When the value is false, the link is hidden
When the value is true, the link goes to the infopage donation,
the administrator can create a custom page.
This patch implements a bolierplate to share content from info-pages of the
SearXNG instance (URL /info) with the project documentation (path /docs/user).
The info pages are using Markdown (CommonMark), to include them in the project
documentation (reST) the myst-parser [1] is used in the Sphinx-doc build chain.
If base_url is known (defined in settings.yml) links to the instance are also
inserted into the project documentation::
searxng_extra/docs_prebuild
[1] https://www.sphinx-doc.org/en/master/usage/markdown.html
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Previously the preferences & stats templates contained the markup:
<a href="{{ url_for('index') }}"><h1><span>SearXNG</span></h1></a>
There are many things wrong with this:
1. the markup was duplicated
2. the CSS needed to be changed whenever a new page wanted to use this
header (since the CSS used page-specific selectors)
3. h1 should be reserved for the actual page title
(e.g. Preferences or Engine stats)
4. the image was set via CSS which also set:
span { visibility: hidden; }
which however removes the alternative text from the accessibility
tree (meaning screen readers will ignore it).
This commit fixes all these problems.
This commit remove the need to update the brand for GIT_URL and GIT_BRANCH:
there are read from the git repository.
It is possible to call python -m searx.version freeze to freeze the current version.
Useful when the code is installed outside git (distro package, docker, etc...)
In the templates and the /config (JSON) the usage of the 'brand.*' name
space is replaced by 'searx.get_setting' function.
- new_issue_url --> get_setting('brand.new_issue_url')
- brand.GIT_URL --> get_setting('brand.git_url')
- brand.PUBLIC_INSTANCES --> get_setting('brand.public_instances')
- brand.DOCS_URL --> get_setting('brand.docs_url')
- brand.ISSUE_URL --> get_setting('brand.issue_url')
- brand.CONTACT_URL --> get_setting('general.contact_url', '')
The macro 'new_issue' from searx/templates/*/messages/no_results.html
is now imported with context::
{% from '__common__/new_issue.html' import new_issue with context %}
To get *public instances URL* from context's 'get_setting()' function::
get_setting('brand.public_instances','')
Macro's prototype does no longer need the 'new_issue_url' argument and has been
changed to::
macro new_issue(engine_name, engine_reliability)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
When the user add searx as a search engine, the browser loads the /opensearch.xml URL without the cookies.
Without the query parameters, the user preferences are ignored (method and autocomplete).
In addition, opensearch.xml is modified to support automatic updates,
see https://developer.mozilla.org/en-US/docs/Web/OpenSearch
We have some variables in the build environment which are also needed in the
templating process. Theses variables are relavant if one creates a fork with
its own branding. We treat these variables under the term 'brands'.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
