Commit Graph

52 Commits

Author SHA1 Message Date
Markus Heiser
692708aa77 [clean up] drop obsolete searx, filtron and morty install scripts
Since ./utils/searxng.sh is implemented, the old installation procedures from
filtron, morty and searx can be removed.

For users who want to upgrade, the procedures for removing old installations
have still been retained.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-07-30 13:39:35 +02:00
Markus Heiser
782f73540e [utils/searxng.sh] implement new script to install SearXNG
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-07-30 13:39:35 +02:00
Markus Heiser
7308a6e33c [LXC] ubuntu 20.10 EOL, upgrade to ubunut 21.10 [1]
[1] https://wiki.ubuntu.com/Releases

Tested by::

    # build the container ..
    $ sudo -H ./utils/lxc.sh build searx-ubu2110

    # open a shell in the container
    $ sudo -H ./utils/lxc.sh cmd searx-ubu2110 bash
    [root@searx-ubu2110 SearXNG]#

    # install a complete SearXNG suite ..
    [root@searx-ubu2110 SearXNG]# ./utils/searx.sh install all
    ...

    # install apache to export the SearXNG instance by HTTP
    [root@searx-ubu2110 SearXNG]# ./utils/searx.sh apache install
    ...
    INFO:  got 200 from http://10.174.184.94/searx

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-03-12 18:00:45 +01:00
Markus Heiser
1c79c1a3d2 [LXC] fedora33 EOL [1], upgrade to fedora35 [2]
[1] https://docs.fedoraproject.org/en-US/releases/eol/
[2] https://docs.fedoraproject.org/en-US/releases/f35/

Tested by::

    # build the container ..
    $ sudo -H ./utils/lxc.sh build searx-fedora35

    # open a shell in the container
    $ sudo -H ./utils/lxc.sh cmd searx-fedora35 bash
    [root@searx-fedora35 SearXNG]#

    # install a complete SearXNG suite ..
    [root@searx-fedora35 SearXNG]# ./utils/searx.sh install all
    ...

    # install apache to export the SearXNG instance by HTTP
    [root@searx-fedora35 SearXNG]# ./utils/searx.sh apache install
    ...
    INFO:  got 200 from http://10.174.184.94/searx

To build wheel `python3-devel` needs to be added to SEARX_PACKAGES_fedora::

    |searx|   × Building wheel for setproctitle (pyproject.toml) did not run successfully.
    |searx|   │ exit code: 1
    ...
    |searx|       In file included from src/spt.h:15,
    |searx|                        from src/setproctitle.c:14:
    |searx|       src/spt_python.h:16:10: fatal error: Python.h: No such file or directory
    |searx|          16 | #include <Python.h>
    |searx|             |          ^~~~~~~~~~
    |searx|       compilation terminated.
    |searx|       error: command '/usr/bin/gcc' failed with exit code 1
    |searx|       [end of output]
    ...
    |searx|   ERROR: Failed building wheel for setproctitle

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2022-03-12 17:44:00 +01:00
Alexandre Flament
76cbfbbdda reference docs.searxng.org 2022-01-02 21:18:29 +01:00
Markus Heiser
b6a55e223c [mod] reduce enviroment variables in shell scripts to what is needed
Not all settings from the 'brand:' section of the YAML files are needed in the
shell scripts.  This patch reduce the variables in ./utils/brand.env to what is
needed.  The following ('brand:' settings) can be removed from this file:

- ISSUE_URL
- DOCS_URL
- PUBLIC_INSTANCES
- WIKI_URL

Tasks running outside of an *installed instance*, need the following settings
from the YAML configuration:

- GIT_URL            <--> brand.git_url
- GIT_BRANCH         <--> brand.git_branch
- SEARX_URL          <--> server.base_url  (aka PUBLIC_URL)
- SEARX_PORT         <--> server.port
- SEARX_BIND_ADDRESS <--> server.bind_address

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-24 16:28:15 +02:00
Markus Heiser
b3ff510e10 [fix] pacman --noprogressbar in shell scripts
From pacman's man page:

  --noprogressbar
    Do not show a progress bar when downloading files. This can be useful for
    scripts that call pacman and capture the output.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-03 18:24:55 +02:00
Markus Heiser
c8a6548592 [mod] utils/lxc.sh: detect conflict of docker & LXC in the iptables
Docker is blocking network of existing LXC containers / there is a conflict in
the iptables setup of Docker & LXC.  With this patch:

- utils/lxc.sh checks internet connectivity (instead of silently hang)
- Chapter "Internet Connectivity & Docker" describes the problem and made a
  suggestion for a solution a solution

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-23 12:47:52 +02:00
Markus Heiser
438d35c386 [fix] do not spit out error messages for utils/lxc.sh --help
Error::

  $ utils/lxc.sh --help
  ERROR: missing command lxc
  ...

This breaks also docs build::

  $ make docs
  ...
  WARNING: Unexpected return code 42 from command '../utils/lxc.sh --help'
  ...

Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-23 11:14:29 +02:00
Markus Heiser
0305775e29 [mod] LXC switch to Fedora 33 / Fedora 31 reached its EOL
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-05 13:17:47 +01:00
Alexandre Flament
6ba37777f7
Merge pull request #2623 from return42/fix-centos7
[fix] add package `which` to CentOS-7 boilerplate
2021-03-04 11:19:05 +01:00
Markus Heiser
c355bc3481 [fix] add package 'which' to CentOS-7 boilerplate
Newer CentOS-7 images from https://images.linuxcontainers.org do no longer
include the which command.

Issue:

    $ sudo -H ./utils/lxc.sh cmd searx-centos7 ./utils/filtron.sh install all
    INFO:  [searx-centos7] ./utils/filtron.sh install all
    ...
    Install Go in user's HOME
    -------------------------

    download and install go binary ..
    ...
    -bash: line 1: which: command not found
    -->|ERROR - Go Installation not found in PATH!?!
    -bash: line 2: which: command not found

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-03 19:37:03 +01:00
Markus Heiser
daa4b676df [mod] Drop Ubuntu 16.04 (Xenial Xerus) support
EOL of Ubuntu 16.04 (Xenial Xerus) is in April 2021 but we do not support 16.04
since a longer time.  Issues are comming from dependencies (e.g. pip)::

    [searx-ubu1604]   |searx| SyntaxError: invalid syntax
    [searx-ubu1604]   |searx| Traceback (most recent call last):
    [searx-ubu1604]   |searx|   File "/usr/local/searx/searx-pyenv/bin/pip", line 7, in <module>
    [searx-ubu1604]   |searx|     from pip._internal.cli.main import main
    [searx-ubu1604]   |searx|   File "/usr/local/searx/searx-pyenv/lib/python3.5/site-packages/pip/_internal/cli/main.py", line 60
    [searx-ubu1604]   |searx|     sys.stderr.write(f"ERROR: {exc}")
    [searx-ubu1604]   |searx|                                    ^
    [searx-ubu1604]   |searx| SyntaxError: invalid syntax

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-03 17:52:50 +01:00
Markus Heiser
ea5c992d4f [fix] lxc.sh - SC2034: ubu2010_boilerplate appears unused.
$ make test.sh
  In utils/lxc.sh line 42:
  ubu2010_boilerplate="$ubu1904_boilerplate"
  ^-----------------^ SC2034: ubu2010_boilerplate appears unused. Verify use (or export if used externally).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-24 08:29:13 +01:00
Markus Heiser
89e6ba5af7 [mod] add Ubuntu 20.10 image to the LXC suite
Ubuntu 20.10 image [1] is avialable from https://images.linuxcontainers.org

BTW: replace builhost package 'ttf-dejavu' by more common meta package
'fonts-dejavu' ('ttf-dejavu' is no longer available in 20.10).

[1] d161de601b

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-02 17:27:16 +01:00
Markus Heiser
c7d92d6475 [fix] remove Ubuntu 19.10 from the LXC suite (EOL)
Official support for Ubuntu 19.10 'Eoan Ermine' ended on July 17, 2020.

The image has been dropped [1] from https://images.linuxcontainers.org

[1] d161de601b

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-02 17:23:18 +01:00
Markus Heiser
c89df6739f [fix] determine path to makefile.lxc in a LXC
The path to the makefile.lxc is not always static and
'/share/searx/utils/makefile.lxc' can be wrong.

When target 'lxc.activate' is called within the LXC container the path to
makefile.lxc depends on LXC_SHARE_FOLDER and the basename of the folder where
searx was cloned inital on the HOST system.

This patch determines the path of 'makefile.lxc' next to 'makefile.include'
itself.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-12-18 17:10:49 +01:00
Markus Heiser
623fb87065
[enh] add centos-7 to LXC images (#2118)
## What does this PR do?

To test centos-7 on non centos systems add centos-7 image to the LXC suite.

## Why is this change important?

PR #2112 adds centos-7 support to the install scripts

## How to test this PR locally?

`sudo ./utils/lxc.sh build searx-centos7`
2020-08-08 19:50:11 +02:00
Eliesemoule
1986b5ecac
Utility scripts adapted to run on CentOS 7 (#2112) 2020-08-01 22:12:44 +02:00
Markus Heiser
e50ccc99c0 archlinux: add package 'inetutils' to boilerplate
Even it is a core component, some hoster do not have pre installed the
'inetutils' package.  We do need this package for the 'hostname' command.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-05-04 16:15:23 +02:00
Markus Heiser
99ff16c465 tooling box: added nginx + polished bash scripts and environment
- add installation method for nginx sites, morty and filtron
- clean up PUBLIC_URL environment in and outside of containers
- clean up comand lines
- handle uWSGI quirks on fedora (emperor mode)
- handle Python quirks on debian (there is no 'python' command anymore)
- lib.sh: add die and die_caller functions
- lxc_suite_install_info is now a function
- lint: shellcheck

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-04-13 11:34:28 +02:00
Markus Heiser
58d5da8b57 nginx: normalize installation (docs and script)s over all distros
This is the revision of the documentation about the varous nginx installation
variants.  It also implements the nginx installation scripts for morty and
filtron.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-04-11 13:19:11 +02:00
Markus Heiser
f693149cde Changes from the installation tests on (all) LXC containers.
Tested and fixed HTTP & uWSGI installation on:

  ubu1604 ubu1804 ubu1910 ubu2004 fedora31 archlinux

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-04-08 18:38:36 +02:00
Markus Heiser
ee39a098ac apache: normalize installation (docs and script)s over all distros
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-04-07 18:31:51 +02:00
Markus Heiser
c2caf9569c [fix] creation of /.lxcenv in containers
In utils/lib.sh there are two functions with the same name, but different tasks.
Rename one of them from lxc_init_container() into lxc_init_container_env().

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-04-05 15:15:28 +02:00
Markus Heiser
3c19f2f1ce utils/lxc.sh install: support installation on selected containers
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-04-04 12:40:14 +02:00
Markus Heiser
af988dbf71 utils/lxc.sh: support build of selected containers
$ sudo -H ./utils/lxc.sh build  <name>
  $ sudo -H ./utils/lxc.sh show   [images|suite|info|config [<name>]]

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-04-03 17:08:42 +02:00
Markus Heiser
f27f6c6c5d docs: use make and uitls/searx.sh in containers
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-31 18:25:40 +02:00
Markus Heiser
dd53c45a2c docs: add utils/lxc.sh docs, normalize filtron, morty & searx docs
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-30 18:47:01 +02:00
Markus Heiser
3e4d022d04 [fix] make test.sh: fix various shellcheck error messages
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-30 11:03:46 +02:00
Markus Heiser
62da769952 ./utils/lxc.sh: add command 'install base'
to install basic packages like git and *build essentials*::

  $ ./utils/lxc.sh install base

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-18 17:47:48 +01:00
Markus Heiser
d2cfe9ce5b LXC: add /.lxcenv
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-18 15:34:46 +01:00
Markus Heiser
86e79488aa LXC: utils/makefile.lxc (inital) add /.lxcenv.mk to contaiiners
Get LXC environment when building make targets.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-15 17:01:36 +01:00
Markus Heiser
80f7b65804 searx.sh: add buildhost cmd, installs OS packages for builds
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-09 01:37:26 +01:00
Markus Heiser
d48c7bf678 LXC: fixed raw.idmap & removed obsolete uid/gid subordinate
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-08 18:30:25 +01:00
Markus Heiser
e7f69b63f1 LXC: from searx.sh, morty.sh and filtron.sh tests
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-08 02:41:45 +01:00
Markus Heiser
b1e90cff23 LXC: separate lxc-suite from lxc & improved command line.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-07 20:24:08 +01:00
Markus Heiser
cbc08fdc26 docs: describe uwsgi setup of all suported distributions
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-04 19:56:40 +01:00
Markus Heiser
7af991acda LXC: add ubuntu 19.10 and 20.04 to the list of test images
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-02 19:01:08 +01:00
Markus Heiser
c3e4753ce9 docs: generic documentation from the installation scripts
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-02 19:00:19 +01:00
Markus Heiser
491cb95a1f utils/lxc.env: separate environment that is used in containers
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-03-01 18:28:10 +01:00
Markus Heiser
37c135f2ce LXC: improved UX when working with a bunch of containers
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-27 19:13:03 +01:00
Markus Heiser
af6acd3417 LXC: install searx-suite installs searx, filtron & morty on all containers
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-26 19:07:55 +01:00
Markus Heiser
d5917cc029 utils/lib.sh: make uWSGI installation available for all distros
support: ubuntu, debin, fedora, archlinux

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-25 20:20:17 +01:00
Markus Heiser
5fb6d4f508 LXC: normalize package installation & user creation.
utils/lib.sh:
- get DIST_ID & DIST_VERSION from /etc/os-release
- pkg_[install|remove|...] supports ubuntu, debian, archlinux & fedora

utils/lxc.sh
- Workaround for the "setrlimit(RLIMIT_CORE): Operation not permitted" error::
    'Set disable_coredump false' >> /etc/sudo.conf

utils/[searx.sh|filtron.sh|morty.sh]
- switched user creation from 'adduser' perl script to 'useradd' built-in
  command

utils/searx.sh
- install packages for ubuntu, debian, archlinux & fedora

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-23 12:10:45 +01:00
Markus Heiser
e36e0f80ae LXC: added archlinux and fedora31 containers
BTW: update all packages when installing conatiner's boilerplate

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-18 18:40:34 +01:00
Markus Heiser
ccb96544f7 LXC: improved console messages & prompt
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-18 18:20:03 +01:00
Markus Heiser
a1f5f2ced8 LXC: minor fixes and renaming
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-16 22:26:03 +01:00
Markus Heiser
ad32739860 shellcheck: fix usse -n instead of ! -z (SC2236 SC2237)
- https://www.shellcheck.net/wiki/SC2236 -- Use -n instead of ! -z.
 - https://www.shellcheck.net/wiki/SC2237 -- Use [ -n .. ] instead of ! [ -z ....

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-16 20:07:37 +01:00
Markus Heiser
e8cf225046 LXC: tooling box - add boilerplate to containers
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-02-16 18:18:15 +01:00