Commit Graph

5665 Commits

Author SHA1 Message Date
Alexandre Flament 2134703b4b [enh] settings.yml: implement general.enable_metrics
* allow not to record metrics (response time, etc...)
* this commit doesn't change the UI. If the metrics are disabled
  /stats and /stats/errors will return empty response.
  in /preferences, the columns response time and reliability will be empty.
2022-01-05 19:03:04 +01:00
Alexandre Flament e64c3deab7 [mod] upgrade httpx 0.21.2
httpx 0.21.2 and httpcore 0.14.4 fix multiple issues:
* https://github.com/encode/httpx/releases/tag/0.21.2
* https://github.com/encode/httpcore/releases/tag/0.14.4

so most of the workarounds in searx.network have been removed.
2022-01-05 18:46:00 +01:00
Martin Fischer a7199bc085
Merge pull request #694 from return42/fix-c7f27404
[fix] debug console (pdb) in a session started by `make run`
2022-01-05 18:40:46 +01:00
Markus Heiser 295876abaa [pylint] add scripts from searxng_extra/update to pylint
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-05 16:09:40 +01:00
Markus Heiser ffea5d8ef5 [docs] add documentation for the scripts in searxng_extra/update
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-05 16:09:40 +01:00
Martin Fischer a4c2cfb837 [enh] change categories_as_tabs from a list to a dict
The tab icon names are currently hard coded in the templates.
This commit lets us introduce an icon property in the future, e.g:

categories_as_tabs:
  general:
    icon: search-outline
2022-01-05 11:03:44 +01:00
Martin Fischer b38036d519 [fix] stop setting superfluous cookies for "non tab categories" 2022-01-05 11:03:44 +01:00
Martin Fischer 3dd534e5c0 [refactor] make group_engines_in_tab more readable 2022-01-05 11:03:44 +01:00
Martin Fischer d01e8aa8cc [mod] introduce searx.engines.Engine for type hinting 2022-01-05 11:03:44 +01:00
Martin Fischer 3f76c9c96c [enh] enable English dictionaries by default
These dictionaries are no longer part of the general category,
so they're no longer queried by default -> we can enable them
by default without degrading general query performance.
2022-01-05 11:03:44 +01:00
Martin Fischer abe53946d4 [enh] display help text for "Other" engines tab in preferences 2022-01-05 11:03:44 +01:00
Martin Fischer 6f07d51b62 [mod] improve variable name 2022-01-05 11:03:44 +01:00
Martin Fischer 1e195f5b95 [mod] move group_engines_in_tab to searx.webutils 2022-01-05 11:03:44 +01:00
Martin Fischer 5d74bf3820 [enh] move dictionaries, Erowid & IMDb out of general category
The general category is the category that is searched by default.
From a privacy standpoint it doesn't make sense to send all general
queries to specialized search engines that cannot deal with those
queries anyway.
2022-01-05 11:03:44 +01:00
Martin Fischer ab90e2ac49 [enh] show categories not in any tab category in "Other" preferences tab
Previously we didn't have a good place to put search engines that don't
fit into any of the tab categories. This commit automatically puts
search engines that don't belong to any tab category in an "other"
category, that is only displayed in the user preferences (and not above
search results).
2022-01-05 11:03:44 +01:00
Martin Fischer 4ac6b5d32d [build] /static 2022-01-05 11:03:44 +01:00
Martin Fischer a12cb6a807 [simple] improve styling of subcategory table headers in preferences 2022-01-05 11:03:44 +01:00
Martin Fischer 20a5e43e35 [enh] display language of engines in preferences 2022-01-05 11:03:44 +01:00
Martin Fischer 31e206361f [enh] group engines in preference tabs 2022-01-05 11:03:44 +01:00
Martin Fischer b02f762687 [enh] add more categories 2022-01-05 11:00:11 +01:00
Markus Heiser c9cfa6181b [fix] debug console (pdb) in a session started by `make run`
Commit c7f27404 moves the `python -m searx.webapp` process to the background.  A
background job can't open a simple python-debugger (pdb) console.

This patch moves all processes to the background except the searx.webapp.

Insert a break point somewhere in the webapp application::

    import pdb
    pdb.set_trace()

And start a debug session by::

    make run

and test you break point.

To test that the entire 'make run' stops in the case of an error in the
themes.live [1] background process try:

    make LIVE_THEME=typo-theme-name run

[1] https://github.com/searxng/searxng/pull/664#discussion_r776419585

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-05 10:09:17 +01:00
Markus Heiser 00d3a7477c
Merge pull request #664 from return42/themes.live
[mod] manage: add themes.live command (rebuild on modification)
2022-01-04 19:44:51 +01:00
Markus Heiser d7b4306444 [fix] links in comments - https://docs.searxng.org/
fixed links from https://searxng.github.io/searxng/ to https://docs.searxng.org/
2022-01-04 16:04:49 +01:00
Markus Heiser a6cfab93fa [enh] add redis connector searx/shared/redisdb.py
Add a redis connector, the default DB connector is a socket at::

    unix:///usr/local/searxng-redis/run/redis.sock?db=0

To set up a redis instance simply use::

    $ ./manage redis.build
    $ sudo -H ./manage redis.install

A hint for developers:

To get access rights to this instance, your developer account needs to be added
to the *searxng-redis* group::

    $ sudo -H ./manage redis.addgrp "${USER}"
    # don't forget to logout & login to get member of group

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-03 20:56:39 +01:00
Martin Fischer 8eb3167fba
Merge pull request #687 from dalf/fix-685
[fix] disabling all engines in a category makes the bang search in general
2022-01-03 20:42:05 +01:00
Alexandre Flament 41ae5aea2b [fix] disabling all engines in a category makes the bang search in general
with this commit, there is no result
2022-01-03 20:23:46 +01:00
Alexandre Flament 8e99c11c95
Merge pull request #680 from JamesClonk/master
fix minor typo
2022-01-03 18:25:54 +01:00
Markus Heiser 4f1130d663 [mod] test.shell: add utils/lib_redis.sh to shellcheck procedure
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-03 14:47:24 +01:00
Markus Heiser fe8b88c8a4 [mod] script to build & install a redis instance
A script to build & install a simple & isolated redis service, dedicated to
SearXNG and connected via Unix socket.

    $ ./manage redis.help
    redis.:
      devpkg    : install essential packages to compile redis
      build     : build redis binaries at /800GBPCIex4/share/SearXNG/dist/redis/6.2.6/amd64
      install   : create user (searxng-redis) and install systemd service (searxng-redis)
      remove    : delete user (searxng-redis) and remove service (searxng-redis)
      shell     : start bash interpreter from user searxng-redis
      src       : clone redis source code to <path> and checkput 6.2.6
      useradd   : create user (searxng-redis) at /usr/local/searxng-redis
      userdel   : delete user (searxng-redis)
      addgrp    : add <user> to group (searxng-redis)
      rmgrp     : remove <user> from group (searxng-redis)

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-03 14:34:26 +01:00
Markus Heiser a3332cd550
Merge pull request #684 from return42/fix-makefile
[fix] Makefile: mixed tab & space indentation
2022-01-03 13:57:59 +01:00
Markus Heiser a585f7c66f [fix] Makefile: mixed tab & space indentation
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-03 13:51:35 +01:00
Markus Heiser b630c5d7bc
Merge pull request #666 from return42/mod-doc
[docs] move searxng_extra/ stuff to the developer section.
2022-01-03 10:25:14 +01:00
Markus Heiser de819bb1f5 [docs] move searxng_extra/ stuff to the developer section.
Stuff in folder searxng_extra/ is not suitable for normal users and should only
be used by developers.

The script searxng_extra/standalone_searx.py must not give the impression that
it improves privacy. [1]

[1] https://github.com/searxng/searxng/pull/651#issuecomment-1001389726

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-03 10:20:54 +01:00
Martin Fischer 8e9ad1ccc2 [enh] introduce categories_as_tabs
Previously all categories were displayed as search engine tabs.
This commit changes that so that only the categories listed under
categories_as_tabs in settings.yml are displayed.

This lets us introduce more categories without cluttering up the UI.
Categories not displayed as tabs  can still be searched with !bangs.
2022-01-03 07:01:49 +01:00
Martin Fischer 02e9bdf755 [doc] engine tables: show engines in all categories
Previously the documentation grouped the engines by their first
category so e.g. YouTube and Invidious were only shown in the
in the videos section but not in the music section.

This commit fixes this by iterating over searx.engines.categories,
which also has the added benefit that the sections are now in the
same order as the tabs in the user interface.
2022-01-03 07:01:49 +01:00
Martin Fischer d8af94b721 [doc] engine-table: stop sorting by Disabled
It's only the default value for disabled it's not that important,
since users can easily toggle it anytime in the engine preferences.
2022-01-03 07:01:49 +01:00
Martin Fischer df34b1ddcf [enh] settings.yml: allow granular overwrites for about 2022-01-03 07:01:49 +01:00
Martin Fischer 8a0810cedc [fix] bang syntax for categories with spaces 2022-01-03 07:01:49 +01:00
Fabio Berchtold 6a67beebe1
fix minor typo 2022-01-02 23:13:36 +01:00
Alexandre Flament 458b7fb72d
Merge pull request #610 from return42/img-load-error
[fix] simple theme: hide the image if img load fails
2022-01-02 22:34:15 +01:00
Alexandre Flament d83aa2b0d2
Merge pull request #613 from return42/pylint-bing-images
[pylint] Bing (Images) engine
2022-01-02 22:00:55 +01:00
Alexandre Flament 76cbfbbdda reference docs.searxng.org 2022-01-02 21:18:29 +01:00
Markus Heiser c2e4b95e8d
Merge pull request #672 from searxng/update_data_update_languages.py
Update searx.data - update_languages.py
2022-01-01 18:44:53 +01:00
Markus Heiser 3cdf2d6570 Update searx.data - update_languages.py
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-01 18:37:35 +01:00
Markus Heiser 8191e1a253 [fix] update_languages.py: generate code that passes CI
File searx/languages.py, created by update_languages.py has to pass quality
check from CI::

    make format.python

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-01 18:32:21 +01:00
Markus Heiser 21d7c8b367
Merge pull request #676 from return42/fix-bing-lang
Fix issues on running update_languages.py
2022-01-01 18:12:47 +01:00
Markus Heiser 8a07559ab5 [fix] update_languages.py: no excption on unknown locale & language
Fix exception handling of unknown locales and languages::

    ERROR: ca_ES_valencia --> [Errno 2] No such file or directory: 'local/py3/lib/python3.8/site-packages/babel/locale-data/ca_ES_valencia.dat'
    ERROR: languages['fil-PH'] --> {'name': None, 'english_name': None}
    ERROR: languages['nb-NO'] --> {'name': None, 'english_name': None}

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-01 17:31:38 +01:00
Markus Heiser 61ce0c2244 [fix] bing engines: fetch_supported_languages
The Request to and the Response from https://www.bing.com/account/general has
been changed.

[1] https://github.com/searxng/searxng/pull/672#discussion_r777104919

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-01-01 17:31:38 +01:00
Alexandre Flament 6b49f32d3e
Merge pull request #670 from searxng/update_data_update_firefox_version.py
Update searx.data - update_firefox_version.py
2022-01-01 12:16:50 +01:00
Markus Heiser 3dcba2f67c
Merge pull request #671 from searxng/update_data_update_ahmia_blacklist.py
Update searx.data - update_ahmia_blacklist.py
2022-01-01 12:08:26 +01:00