Commit Graph

9 Commits

Author SHA1 Message Date
Alexandre Flament
b10403d3a1 [mod] searx.network: remove redundant code
searx.client.new_client: the proxies parameter is a dictonnary,
and the protocol (key of the dictionnary) is already normalized
(see usage of searx.network.network.PROXY_PATTERN_MAPPING)
2021-09-17 10:06:24 +02:00
Alexandre Flament
8e73438cbe [upd] upgrade httpx 0.19.0
adjust searx.network module to the new internal API
see https://github.com/encode/httpx/pull/1522
2021-09-17 10:06:22 +02:00
Alexandre Flament
91a6d80e82 [mod] debug mode: log HTTP requests with network name
For example wikipedia requests use the logger name "searx.network.wikipedia"

Log is disable when searx_debug is False
2021-09-11 10:13:14 +02:00
Markus Heiser
2a3b9a2e26 [pylint] searx: drop no longer needed 'missing-function-docstring'
Suggested-by: @dalf https://github.com/searxng/searxng/issues/102#issuecomment-914168470
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-07 13:34:35 +02:00
Markus Heiser
1499002ceb [coding-style] searx/network/client.py - normalized indentations
No functional change!

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-24 17:44:43 +02:00
Markus Heiser
b595c482d0 [pylint] searx/network/client.py & add global (TRANSPORT_KWARGS)
No functional change!

- fix messages from pylint
- add ``global TRANSPORT_KWARGS``
- normalized python_socks imports

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-05-24 17:39:37 +02:00
Alexandre Flament
0f4e995ab4 [mod] searx.network.client: the same configuration reuses the same ssl.SSLContext
before there was one ssl.SSLContext per client.

see https://github.com/encode/httpx/issues/978
2021-05-05 20:36:37 +02:00
Alexandre Flament
283ae7bfad [fix] searx.network: fix rare cases where LOOP is None
* searx.network.client.LOOP is initialized in a thread
* searx.network.__init__ imports LOOP which may happen
  before the thread has initialized LOOP

This commit adds a new function "searx.network.client.get_loop()"
to fix this issue
2021-04-27 17:47:36 +02:00
Alexandre Flament
d14994dc73 [httpx] replace searx.poolrequests by searx.network
settings.yml:

* outgoing.networks:
   * can contains network definition
   * propertiers: enable_http, verify, http2, max_connections, max_keepalive_connections,
     keepalive_expiry, local_addresses, support_ipv4, support_ipv6, proxies, max_redirects, retries
   * retries: 0 by default, number of times searx retries to send the HTTP request (using different IP & proxy each time)
   * local_addresses can be "192.168.0.1/24" (it supports IPv6)
   * support_ipv4 & support_ipv6: both True by default
     see https://github.com/searx/searx/pull/1034
* each engine can define a "network" section:
   * either a full network description
   * either reference an existing network

* all HTTP requests of engine use the same HTTP configuration (it was not the case before, see proxy configuration in master)
2021-04-12 17:25:56 +02:00