forked from zaclys/searxng
docs: use make and uitls/searx.sh in containers
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This commit is contained in:
Markus Heiser
parent
8634eeb9de
commit
f27f6c6c5d
@ -4,9 +4,6 @@
|
||||
Buildhosts
|
||||
==========
|
||||
|
||||
To get best results from build, its recommend to install additional packages
|
||||
on build hosts.
|
||||
|
||||
.. sidebar:: This article needs some work
|
||||
|
||||
If you have any contribution send us your :pull:`PR <../pulls>`, see
|
||||
@ -17,6 +14,23 @@ on build hosts.
|
||||
:local:
|
||||
:backlinks: entry
|
||||
|
||||
To get best results from build, its recommend to install additional packages
|
||||
on build hosts (see :ref:`searx.sh`).::
|
||||
|
||||
sudo -H ./utils/searx.sh buildhost
|
||||
|
||||
This will install packages needed by searx:
|
||||
|
||||
.. include:: ../../build/docs/includes/searx.rst
|
||||
:start-after: START distro-packages
|
||||
:end-before: END distro-packages
|
||||
|
||||
and packages needed to build docuemtation and run tests:
|
||||
|
||||
.. include:: ../../build/docs/includes/searx.rst
|
||||
:start-after: START build-packages
|
||||
:end-before: END build-packages
|
||||
|
||||
.. _docs build:
|
||||
|
||||
Build docs
|
||||
|
||||
@ -28,6 +28,32 @@ ${fedora}
|
||||
|
||||
.. END distro-packages
|
||||
|
||||
.. START build-packages
|
||||
|
||||
.. tabs::
|
||||
|
||||
.. group-tab:: Ubuntu / debian
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
$ sudo -H apt-get install -y \\
|
||||
${debian_build}
|
||||
|
||||
.. group-tab:: Arch Linux
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
$ sudo -H pacman -S --noconfirm \\
|
||||
${arch_build}
|
||||
|
||||
.. group-tab:: Fedora / RHEL
|
||||
|
||||
.. code-block:: sh
|
||||
|
||||
$ sudo -H dnf install -y \\
|
||||
${fedora_build}
|
||||
|
||||
.. END build-packages
|
||||
|
||||
.. START create user
|
||||
|
||||
|
||||
@ -1,5 +1,4 @@
|
||||
|
||||
|
||||
.. _snap: https://snapcraft.io
|
||||
.. _snapcraft LXD: https://snapcraft.io/lxd
|
||||
.. _LXC/LXD Image Server: https://uk.images.linuxcontainers.org/
|
||||
@ -7,6 +6,8 @@
|
||||
.. _LXD: https://linuxcontainers.org/lxd/introduction/
|
||||
.. _`LXD@github`: https://github.com/lxc/lxd
|
||||
|
||||
.. _archlinux: https://www.archlinux.org/
|
||||
|
||||
.. _lxc.sh:
|
||||
|
||||
================
|
||||
@ -21,24 +22,31 @@
|
||||
- `LXD@github`_
|
||||
|
||||
With the use of *Linux Containers* (LXC_) we can scale our tasks over a stack of
|
||||
containers, what we call the: *lxc suite*. Before you can start with
|
||||
containers, you need to install and initiate LXD_ once::
|
||||
containers, what we call the: *lxc suite*. The *searx suite*
|
||||
(:origin:`lxc-searx.env <utils/lxc-searx.env>`) is loaded by default, every time
|
||||
you start the ``lxc.sh`` script (*you do not need to care about*).
|
||||
|
||||
Before you can start with containers, you need to install and initiate LXD_
|
||||
once::
|
||||
|
||||
$ snap install lxd
|
||||
$ lxd init --auto
|
||||
|
||||
The *searx suite* (:origin:`lxc-searx.env <utils/lxc-searx.env>`) is loaded by
|
||||
default, every time you start the ``lxc.sh`` script (you do not need to care
|
||||
about). To make use of the containers from the *searx suite*, you have to build
|
||||
the :ref:`LXC suite containers <lxc.sh --help>` first. But be warned, this
|
||||
might take some time::
|
||||
To make use of the containers from the *searx suite*, you have to build the
|
||||
:ref:`LXC suite containers <lxc.sh help>` initial. But be warned, this might
|
||||
take some time::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh build
|
||||
|
||||
A cup of coffee later, your LXC suite is build up and you can run whatever task
|
||||
you want / in a selected or even in all :ref:`LXC suite containers <lxc.sh
|
||||
--help>`. Each container shares the root folder of the repository and the
|
||||
command ``utils/lxc.sh cmd`` handles relative path names *transparent*::
|
||||
help>`.
|
||||
|
||||
*Good to know ...*
|
||||
|
||||
Eeach container shares the root folder of the repository and the
|
||||
command ``utils/lxc.sh cmd`` **handles relative path names transparent**,
|
||||
compare output of::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd -- ls -la Makefile
|
||||
...
|
||||
@ -46,29 +54,70 @@ command ``utils/lxc.sh cmd`` handles relative path names *transparent*::
|
||||
[searx-fedora31] -rw-r--r-- 1 root root 7603 Mar 30 11:54 Makefile
|
||||
[searx-archlinux] -rw-r--r-- 1 root root 7603 Mar 30 11:54 Makefile
|
||||
|
||||
With this in mind, you can run :ref:`searx.sh` and install packages, needed by
|
||||
searx::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd -- ./utils/searx.sh install packages
|
||||
|
||||
And run one of the :origin:`Makefile` targets::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh cmd -- make test.sh
|
||||
|
||||
You can install a *buildhost environment* into the containers (time for another
|
||||
cup of coffee)::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh install buildhost
|
||||
|
||||
If you want to get rid off all the containers, just type::
|
||||
If there comes the time you want to **get rid off all** the containers and
|
||||
**clean up local images** just type::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh remove
|
||||
|
||||
To clean up your local images use::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh remove images
|
||||
|
||||
.. _lxc.sh --help:
|
||||
|
||||
Install suite
|
||||
=============
|
||||
|
||||
To install the complete :ref:`searx suite (includes searx, morty & filtron)
|
||||
<lxc-searx.env>` into all LXC_ use::
|
||||
|
||||
$ sudo -H ./utils/lxc.sh install suite
|
||||
|
||||
The command above installs a searx suite (see :ref:`installation scripts`). To
|
||||
get the IP (URL) of the filtron service in the containers use ``show suite``
|
||||
command. To test instances from containers just open the URLs in your
|
||||
WEB-Browser::
|
||||
|
||||
$ sudo ./utils/lxc.sh show suite | grep filtron
|
||||
[searx-ubu1604] INFO: (eth0) filtron: http://n.n.n.135:4004/
|
||||
[searx-ubu1804] INFO: (eth0) filtron: http://n.n.n.141:4004/
|
||||
[searx-ubu1910] INFO: (eth0) filtron: http://n.n.n.137:4004/
|
||||
[searx-ubu2004] INFO: (eth0) filtron: http://n.n.n.127:4004/
|
||||
[searx-fedora31] INFO: (eth0) filtron: http://n.n.n.18:4004/
|
||||
[searx-archlinux] INFO: (eth0) filtron: http://n.n.n.12:4004/
|
||||
|
||||
|
||||
Running commands
|
||||
================
|
||||
|
||||
**Inside containers, you can use make or run scripts** from the
|
||||
:ref:`toolboxing`. By example: to setup a :ref:`buildhosts` and run the
|
||||
Makefile target ``test`` in the archlinux_ container::
|
||||
|
||||
sudo -H ./utils/lxc.sh cmd searx-archlinux ./utils/searx.sh buildhost
|
||||
sudo -H ./utils/lxc.sh cmd searx-archlinux make test
|
||||
|
||||
|
||||
Setup searx buildhost
|
||||
=====================
|
||||
|
||||
You can **install the searx buildhost environment** into one or all containers.
|
||||
The installation procedure to set up a :ref:`build host<buildhosts>` takes its
|
||||
time. Installation in all containers will take more time (time for another cup
|
||||
of coffee).::
|
||||
|
||||
sudo -H ./utils/lxc.sh cmd -- ./utils/searx.sh buildhost
|
||||
|
||||
To build (live) documentation inside a archlinux_ container::
|
||||
|
||||
sudo -H ./utils/lxc.sh cmd searx-archlinux make docs-clean docs-live
|
||||
...
|
||||
[I 200331 15:00:42 server:296] Serving on http://0.0.0.0:8080
|
||||
|
||||
To get IP of the container and the port number *live docs* is listening::
|
||||
|
||||
$ sudo ./utils/lxc.sh show suite | grep docs-live
|
||||
...
|
||||
[searx-archlinux] INFO: (eth0) docs-live: http://n.n.n.12:8080/
|
||||
|
||||
|
||||
.. _lxc.sh help:
|
||||
|
||||
Overview
|
||||
========
|
||||
@ -77,3 +126,11 @@ The ``--help`` output of the script is largely self-explanatory:
|
||||
|
||||
.. program-output:: ../utils/lxc.sh --help
|
||||
|
||||
|
||||
.. _lxc-searx.env:
|
||||
|
||||
searx suite
|
||||
===========
|
||||
|
||||
.. literalinclude:: ../../utils/lxc-searx.env
|
||||
:language: bash
|
||||
|
||||
@ -49,18 +49,6 @@ lxc_suite_install() {
|
||||
)
|
||||
}
|
||||
|
||||
lxc_suite_prepare_buildhost() {
|
||||
(
|
||||
lxc_set_suite_env
|
||||
FORCE_TIMEOUT=0
|
||||
export FORCE_TIMEOUT
|
||||
# "${LXC_REPO_ROOT}/utils/searx.sh* install packages
|
||||
"${LXC_REPO_ROOT}/utils/searx.sh" install buildhost
|
||||
rst_title "buildhost installation finished ($(hostname))" part
|
||||
echo
|
||||
)
|
||||
}
|
||||
|
||||
lxc_suite_info() {
|
||||
(
|
||||
lxc_set_suite_env
|
||||
|
||||
10
utils/lxc.sh
10
utils/lxc.sh
@ -82,7 +82,7 @@ usage::
|
||||
$_cmd [start|stop] [containers|<name>]
|
||||
$_cmd show [info|config|suite|images]
|
||||
$_cmd cmd [--|<name>] '...'
|
||||
$_cmd install [suite|base|buildhost]
|
||||
$_cmd install [suite|base]
|
||||
|
||||
build
|
||||
:containers: build, launch and 'install basic' packages on 'containers'
|
||||
@ -106,7 +106,6 @@ cmd
|
||||
install
|
||||
:suite: install LXC suite; ${LXC_SUITE_INSTALL_INFO}
|
||||
:base: prepare LXC; install basic packages
|
||||
:buildhost: prepare LXC; install buildhost packages
|
||||
|
||||
EOF
|
||||
usage_images
|
||||
@ -150,7 +149,9 @@ main() {
|
||||
|
||||
# don't check prerequisite when in recursion
|
||||
if [[ ! $1 == __* ]]; then
|
||||
! required_commands lxc && lxd_info && exit 42
|
||||
if ! in_container; then
|
||||
! required_commands lxc && lxd_info && exit 42
|
||||
fi
|
||||
[[ -z $LXC_SUITE ]] && err_msg "missing LXC_SUITE" && exit 42
|
||||
fi
|
||||
|
||||
@ -240,7 +241,7 @@ main() {
|
||||
install)
|
||||
sudo_or_exit
|
||||
case $2 in
|
||||
suite|base|buildhost)
|
||||
suite|base)
|
||||
lxc_exec "${LXC_REPO_ROOT}/utils/lxc.sh" __install "$2"
|
||||
;;
|
||||
*) usage "$_usage"; exit 42 ;;
|
||||
@ -252,7 +253,6 @@ main() {
|
||||
case $2 in
|
||||
suite) lxc_suite_install ;;
|
||||
base) FORCE_TIMEOUT=0 lxc_install_base_packages ;;
|
||||
buildhost) lxc_suite_prepare_buildhost ;;
|
||||
esac
|
||||
;;
|
||||
doc)
|
||||
|
||||
@ -48,7 +48,7 @@ git build-essential libxslt-dev zlib1g-dev libffi-dev libssl-dev
|
||||
shellcheck"
|
||||
|
||||
BUILD_PACKAGES_debian="\
|
||||
graphviz imagemagick texlive-xetex librsvg2-bin
|
||||
firefox graphviz imagemagick texlive-xetex librsvg2-bin
|
||||
texlive-latex-recommended texlive-extra-utils ttf-dejavu"
|
||||
|
||||
# pacman packages
|
||||
@ -59,7 +59,7 @@ git base-devel libxml2
|
||||
shellcheck"
|
||||
|
||||
BUILD_PACKAGES_arch="\
|
||||
graphviz imagemagick texlive-bin extra/librsvg
|
||||
firefox graphviz imagemagick texlive-bin extra/librsvg
|
||||
texlive-core texlive-latexextra ttf-dejavu"
|
||||
|
||||
# dnf packages
|
||||
@ -70,7 +70,7 @@ git @development-tools libxml2
|
||||
ShellCheck"
|
||||
|
||||
BUILD_PACKAGES_fedora="\
|
||||
graphviz graphviz-gd ImageMagick librsvg2-tools
|
||||
firefox graphviz graphviz-gd ImageMagick librsvg2-tools
|
||||
texlive-xetex-bin texlive-collection-fontsrecommended
|
||||
texlive-collection-latex dejavu-sans-fonts dejavu-serif-fonts
|
||||
dejavu-sans-mono-fonts"
|
||||
@ -116,7 +116,7 @@ usage() {
|
||||
usage::
|
||||
|
||||
$(basename "$0") shell
|
||||
$(basename "$0") install [all|user|searx-src|pyenv|uwsgi|apache|packages]
|
||||
$(basename "$0") install [all|user|searx-src|pyenv|uwsgi|apache|packages|buildhost]
|
||||
$(basename "$0") update [searx]
|
||||
$(basename "$0") remove [all|user|pyenv|searx-src]
|
||||
$(basename "$0") activate [service]
|
||||
@ -135,7 +135,7 @@ install / remove
|
||||
:uwsgi: install searx uWSGI application
|
||||
:settings: reinstall settings from ${REPO_ROOT}/searx/settings.yml
|
||||
:packages: install needed packages from OS package manager
|
||||
:buildhost: install packages from OS package manager needed by buildhost
|
||||
:buildhost: install packages from OS package manager needed by buildhosts
|
||||
update searx
|
||||
Update searx installation ($SERVICE_HOME)
|
||||
activate service
|
||||
@ -688,9 +688,15 @@ rst-doc() {
|
||||
local debian="${SEARX_PACKAGES_debian}"
|
||||
local arch="${SEARX_PACKAGES_arch}"
|
||||
local fedora="${SEARX_PACKAGES_fedora}"
|
||||
local debian_build="${BUILD_PACKAGES_debian}"
|
||||
local arch_build="${BUILD_PACKAGES_arch}"
|
||||
local fedora_build="${BUILD_PACKAGES_fedora}"
|
||||
debian="$(echo "${debian}" | sed 's/.*/ & \\/' | sed '$ s/.$//')"
|
||||
arch="$(echo "${arch}" | sed 's/.*/ & \\/' | sed '$ s/.$//')"
|
||||
fedora="$(echo "${fedora}" | sed 's/.*/ & \\/' | sed '$ s/.$//')"
|
||||
debian_build="$(echo "${debian_build}" | sed 's/.*/ & \\/' | sed '$ s/.$//')"
|
||||
arch_build="$(echo "${arch_build}" | sed 's/.*/ & \\/' | sed '$ s/.$//')"
|
||||
fedora_build="$(echo "${fedora_build}" | sed 's/.*/ & \\/' | sed '$ s/.$//')"
|
||||
|
||||
eval "echo \"$(< "${REPO_ROOT}/docs/build-templates/searx.rst")\""
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user