mirror of https://github.com/searxng/searxng.git
[mod] limiter: block requests from PetalBot
Block requests from PetalBlock. Normally robots.txt is enough to stop PetalBlock from making requests [1]. However, if SearXNG is offered below a path (example.org/search), then the robots.txt is not available in the root paths of the domain / subdomain. [1] https://webmaster.petalsearch.com/site/petalbot Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This commit is contained in:
parent
9967fbb7d1
commit
823c490c84
|
@ -37,6 +37,7 @@ block_user_agent = re.compile(
|
|||
# unmaintained Farside instances
|
||||
+ r'|'
|
||||
+ re.escape(r'Mozilla/5.0 (compatible; Farside/0.1.0; +https://farside.link)')
|
||||
+ '|.*PetalBot.*'
|
||||
+ r')'
|
||||
)
|
||||
|
||||
|
|
Loading…
Reference in New Issue