mirror of https://github.com/searxng/searxng.git
[doc] move dosc of botdetection from developer to admin section
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This commit is contained in:
parent
75c9de02d1
commit
597c68b4aa
|
@ -15,6 +15,7 @@ Administrator documentation
|
||||||
installation-apache
|
installation-apache
|
||||||
update-searxng
|
update-searxng
|
||||||
answer-captcha
|
answer-captcha
|
||||||
|
searx.botdetection
|
||||||
api
|
api
|
||||||
architecture
|
architecture
|
||||||
plugins
|
plugins
|
||||||
|
|
|
@ -19,6 +19,8 @@ Bot Detection
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
|
|
||||||
|
.. _botdetection rate limit:
|
||||||
|
|
||||||
Rate limit
|
Rate limit
|
||||||
==========
|
==========
|
||||||
|
|
||||||
|
@ -29,6 +31,8 @@ Rate limit
|
||||||
:members:
|
:members:
|
||||||
|
|
||||||
|
|
||||||
|
.. _botdetection probe headers:
|
||||||
|
|
||||||
Probe HTTP headers
|
Probe HTTP headers
|
||||||
==================
|
==================
|
||||||
|
|
|
@ -2,6 +2,23 @@
|
||||||
# lint: pylint
|
# lint: pylint
|
||||||
""".. _botdetection src:
|
""".. _botdetection src:
|
||||||
|
|
||||||
|
The :ref:`limiter <limiter src>` implements several methods to block bots:
|
||||||
|
|
||||||
|
a. Analysis of the HTTP header in the request / can be easily bypassed.
|
||||||
|
|
||||||
|
b. Block and pass lists in which IPs are listed / difficult to maintain, since
|
||||||
|
the IPs of bots are not all known and change over the time.
|
||||||
|
|
||||||
|
c. Detection of bots based on the behavior of the requests and blocking and, if
|
||||||
|
necessary, unblocking of the IPs via a dynamically changeable IP block list.
|
||||||
|
|
||||||
|
For dynamically changeable IP lists a Redis database is needed and for any kind
|
||||||
|
of IP list the determination of the IP of the client is essential. The IP of
|
||||||
|
the client is determined via the X-Forwarded-For_ HTTP header
|
||||||
|
|
||||||
|
.. _X-Forwarded-For:
|
||||||
|
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For
|
||||||
|
|
||||||
X-Forwarded-For
|
X-Forwarded-For
|
||||||
===============
|
===============
|
||||||
|
|
||||||
|
|
Loading…
Reference in New Issue