Commit Graph

51 Commits

Author SHA1 Message Date
Markus Heiser 064b545f14 [mod] utils/filtron.sh - check a filtron installation
Impplement a script to check a filtron installation::

   ./utils/filtron.sh install check

Related-to: https://github.com/searxng/searxng/issues/450
Related-to: https://github.com/searxng/searxng/pull/446#issuecomment-954599668

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-29 15:51:30 +00:00
Alexandre Flament 5c4afdd737 [fix] filtron.sh: fix filtron proxy error
morty & filtron upgrade golang from v1.13.5 to v1.17.2 (#stable go[1])

Closes: #443

[1] https://golang.org/dl/#stable
2021-10-29 14:09:58 +00:00
Alexandre Flament 9e266ecad3 SearXNG: SEARX_BIND_ADDRESS 2021-10-02 16:58:09 +02:00
Alexandre Flament 6443ed7562 SearXNG: SEARXNG_PORT 2021-10-02 16:56:24 +02:00
Allen 08ac175236 [fix] Wording for Morty & Filtron error message 2021-09-04 10:22:52 +02:00
Markus Heiser 8ba4dc26d1 [fix] utils/searx.sh install settings - do not eval the template
The settings template in utils/templates/etc/searx/settings.yml
does not include any replacements (shell environment variables).

BTW: utils/filtron.sh & utils/lib.sh
  Add some more messages to installation procedures, to be more clear.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-08-20 14:21:54 +02:00
Markus Heiser 7196a9b5dd [fix] ./utils/filtron.sh - FILTRON_TARGET from YAML settings
The filtron target is the SearXNG installation and the default of FILTRON_TARGET
is taken from the YAML configuration ('server.port' & 'server.bind_address').

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-24 16:28:15 +02:00
Markus Heiser f61c918dd4 [mod] normalize .config.sh with settings.yml
In commit 94851790 we have centralized all SearXNG setups in the settings.yml
file:

  94851790 [mod] move brand options from Makefile to settings.yml

This step has not yet been completed for the installation procedures!  Since all
SearXNG setups are done in the settings.yml these environment variables needs to
be removed from the ./conf.sh file.  Scripts and other tasks running outside of
an instance got the needed values from the ./utils/brand.env file.

By example: ATM the environment variables of the ./config.sh file are in
conflict with them from settings.yml:

  - PUBLIC_URL          --> {server:base_url}
  - SEARX_INTERNAL_HTTP --> {server:bind_address}.{server:port}
  - GIT_BRANCH          --> {brand:GIT_URL}

These environment variable of a SearXNG instance and additional

  - SEARX_SETTINGS_TEMPLATE

has been remove from the '.config.sh' file.  With this patch, the main focus of
./conf.sh resists on environment variables needed for the installation of morty,
filtron software.

modified  .config.sh:
  - removed no longer supported variables (see above)
  - add comment about: SearXNG setup in settings.yml

modified utils/searx.sh:
  - SEARX_INTERNAL_HTTP no longer take from .config.sh
  - SEARX_SETTINGS_PATH /etc/searx/settings.yml
  - SEARX_SETTINGS_TEMPLATE obsolete

modified utils/lib_install.sh:
  Initialize environment variables SEARX_PYENV, SEARX_SETTINGS_PATH and
  PUBLIC_URL.

modified:  utils/morty.sh
  Add missing hint about SEARX_SETTINGS_PATH and move PUBLIC_URL to
  utils/lib_install.sh

modified:  utils/morty.sh
  Move PUBLIC_URL to utils/lib_install.sh

Renamed utils/templates/etc/searx/use_default_settings.yml -> settings.yml
  - removed option which can't be modified after installation
  - add some comments with examples

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-21 14:38:57 +02:00
Markus Heiser 414a6105e7 [mod] load .config.sh from installation (utils/lib_install.sh)
**new** utils/lib_install.sh:
  Used to initialize installation procedures

  - Modified source_dot_config function that
    - loads .config.sh from an existing installation (at SEARX_SRC).
    - initialize **SEARX_SRC_INIT_FILES**
  - functions like:
    - install_log_searx_instance()
    - install_searx_get_state()

**modified** utils/searx.sh
    - obsolete environment SEARX_INSTANCE_NAME has been replaced
      by string 'SearXNG'.

**modified** utils/filtron.sh, utils/morty.sh, utils/searx.sh
  - source utils/lib_install.sh
  - normalize logging of environment variables using new function
    install_log_searx_instance()

**modified** utils/lib.sh
  - fix marginal typos

**Installation scripts**
  The utils/lib_install.sh is sourced by the installations scripts:

  - utils/searx.sh
  - utils/morty.sh
  - utils/filtron.sh

  If '${SEARX_SRC}/.config.sh' exists, the modified source_dot_config() function
  loads this configuration (instead of './.config.sh').

**SEARX_SRC_INIT_FILES**
  Array of file names to sync into a installation at $SEARX_SRC.  The file names
  are relative to the $REPO_ROOT.  Set by function init_SEARX_SRC_INIT_FILES().
  Most often theses are files like:

  - .config.sh
  - searx/settings.yml
  - utils/brand.env
  - ...

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-21 14:38:33 +02:00
Markus Heiser 666bf1d32b [fix] typo: 'interactiv' --> 'interactive'
Reported-by: https://github.com/searxng/searxng/pull/38#pullrequestreview-646940531
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-28 16:29:03 +02:00
Markus Heiser d9ffb09011 [filtron.sh] make filtron rule file configurable
To select a different file with filtron rules, set environment

    FILTRON_RULES_TEMPLATE

the default is

    utils/templates/etc/filtron/rules.json

The installation is done by the new function install_rules() which offers a
multiple choice in case of collisions (known from searx.sh install setup).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-28 12:48:49 +02:00
Eliesemoule 1986b5ecac
Utility scripts adapted to run on CentOS 7 (#2112) 2020-08-01 22:12:44 +02:00
Markus Heiser 99ff16c465 tooling box: added nginx + polished bash scripts and environment
- add installation method for nginx sites, morty and filtron
- clean up PUBLIC_URL environment in and outside of containers
- clean up comand lines
- handle uWSGI quirks on fedora (emperor mode)
- handle Python quirks on debian (there is no 'python' command anymore)
- lib.sh: add die and die_caller functions
- lxc_suite_install_info is now a function
- lint: shellcheck

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-04-13 11:34:28 +02:00
Markus Heiser 58d5da8b57 nginx: normalize installation (docs and script)s over all distros
This is the revision of the documentation about the varous nginx installation
variants.  It also implements the nginx installation scripts for morty and
filtron.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-04-11 13:19:11 +02:00
Markus Heiser f693149cde Changes from the installation tests on (all) LXC containers.
Tested and fixed HTTP & uWSGI installation on:

  ubu1604 ubu1804 ubu1910 ubu2004 fedora31 archlinux

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-04-08 18:38:36 +02:00
Markus Heiser ee39a098ac apache: normalize installation (docs and script)s over all distros
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-04-07 18:31:51 +02:00
Markus Heiser c81849cb5a filtron.sh & morty.sh: improve usage message (if used in containers)
BTW: normalize soma variable names

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-04-05 17:40:37 +02:00
Markus Heiser 853be4e11c utils/filtron.sh: add generic documentation
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-04-03 20:24:40 +02:00
Markus Heiser dd53c45a2c docs: add utils/lxc.sh docs, normalize filtron, morty & searx docs
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-30 18:47:01 +02:00
Markus Heiser 7b4cf2eb48 tooling box: simplify build enviroments
- no more need for a .config.mk
- docs: use searx.brands environment
- searx.sh, filtron.sh & morty.sh are sourcing utils/brand.env

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-29 15:09:34 +02:00
Markus Heiser e7f69b63f1 LXC: from searx.sh, morty.sh and filtron.sh tests
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-08 02:41:45 +01:00
Markus Heiser b1e90cff23 LXC: separate lxc-suite from lxc & improved command line.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-07 20:24:08 +01:00
Markus Heiser c3e4753ce9 docs: generic documentation from the installation scripts
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-02 19:00:19 +01:00
Markus Heiser 491cb95a1f utils/lxc.env: separate environment that is used in containers
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-01 18:28:10 +01:00
Markus Heiser 37c135f2ce LXC: improved UX when working with a bunch of containers
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-27 19:13:03 +01:00
Markus Heiser af6acd3417 LXC: install searx-suite installs searx, filtron & morty on all containers
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-26 19:07:55 +01:00
Markus Heiser 5fb6d4f508 LXC: normalize package installation & user creation.
utils/lib.sh:
- get DIST_ID & DIST_VERSION from /etc/os-release
- pkg_[install|remove|...] supports ubuntu, debian, archlinux & fedora

utils/lxc.sh
- Workaround for the "setrlimit(RLIMIT_CORE): Operation not permitted" error::
    'Set disable_coredump false' >> /etc/sudo.conf

utils/[searx.sh|filtron.sh|morty.sh]
- switched user creation from 'adduser' perl script to 'useradd' built-in
  command

utils/searx.sh
- install packages for ubuntu, debian, archlinux & fedora

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-23 12:10:45 +01:00
Markus Heiser 59e4026762 searx.sh: install settings at /etc/searx/settings.yml
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-17 18:58:59 +01:00
Markus Heiser 0bfc61dbe9 tooling box: misc fixes from lxc tests
Tested by:

  sudo -H ./utils/lxc.sh build containers
  sudo -H ./utils/lxc.sh cmd /share/searx/utils/searx.sh    install all
  sudo -H ./utils/lxc.sh cmd /share/searx/utils/filtron.sh  install all
  sudo -H ./utils/lxc.sh cmd /share/searx/utils/morty.sh    install all

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-16 22:29:06 +01:00
Markus Heiser ad32739860 shellcheck: fix usse -n instead of ! -z (SC2236 SC2237)
- https://www.shellcheck.net/wiki/SC2236 -- Use -n instead of ! -z.
 - https://www.shellcheck.net/wiki/SC2237 -- Use [ -n .. ] instead of ! [ -z ....

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-16 20:07:37 +01:00
Markus Heiser e6bf2038c3 filtron.sh: add 'install rules' command
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-11 15:58:35 +01:00
Markus Heiser 5c0f9c8c79 tooling box: varius fix from tests
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-08 17:12:01 +01:00
Markus Heiser 71d7550dbe tooling box ./utils/*: minor fix from production test 2020-02-04 19:47:33 +01:00
Markus Heiser 2f40f61f83 /etc/filtron/rules.json: normalize rules from docs & tooling box
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-04 17:59:58 +01:00
Markus Heiser 79e1f837ef utils & Makefile: add .config.mk & .config.sh for searx brands
By isolating the environment of makefiles and bash scripts into .config.mk and
.config.sh it is simple to maintain searx brands by setting some central
environments.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-04 10:39:42 +01:00
Markus Heiser 94ac560dcc doc: add descriptions of our tooling box in ./utils
Initial started with the scripts for searx, filtron and morty installations.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-03 13:25:51 +01:00
Markus Heiser a4437c47ac utils/morty.sh: add script to install morty result proxy
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-02 18:14:10 +01:00
Markus Heiser 709ac51d33 utils/filtron.sh: generalize systemd, accounts and golang tasks
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-01 16:59:27 +01:00
Markus Heiser 56a93ee770 utils/lib.sh: colorize messages
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-01-31 18:24:31 +01:00
Markus Heiser 0bb8847087 utils/filtron.sh: add option to debug filtron requests
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-01-31 17:25:38 +01:00
Markus Heiser cd9b877c8c utils/searx.sh & filtron.sh: misc changes from first tests (WIP)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-01-31 15:53:53 +01:00
Markus Heiser 6274a54746 utils/searx.sh & filtron.sh: misc changes from first tests (WIP)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-01-30 19:55:51 +01:00
Markus Heiser 924bf65517 utils/searx.sh & filtron.sh: misc changes from first tests (WIP)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-01-29 20:00:50 +01:00
Markus Heiser 971a8264b2 utils/searx.sh: add apache site searx.conf:uwsgi (WIP)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-01-27 19:08:40 +01:00
Markus Heiser af2f58fc58 utils/filtron.sh: add script to install filtron middleware (apache)
Preparation for the installation of an apache site.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-01-20 19:08:56 +01:00
Markus Heiser 89df9d9141 utils/searx.sh: add script to install isolated searx service (WIP)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-01-16 14:01:38 +01:00
Markus Heiser 3cf31528f3 utils/searx.sh: add script to install isolated searx service (WIP)
WIP: written from scratch / linted but untested

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-01-14 19:26:54 +01:00
Markus Heiser 3ed82a97af utils/filtron.sh: add command 'update filtron'
BTW: tidy up some code style and fix some issues reported by the linter.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-01-12 15:51:04 +01:00
Markus Heiser 39feb141bc docs(admin): add description of the utils/filtron.sh script
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-01-11 12:50:40 +01:00
Markus Heiser 4990b07b4b utils/filtron.sh: various fix from first installation test (WIP)
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-01-09 16:25:05 +01:00