docs: revision of the installation instructions

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

docs/admin/buildhosts.rst

@@ -4,13 +4,18 @@
 Buildhosts
 ==========
 
+To get best results from build, its recommend to install additional packages
+on build hosts.
+
 .. sidebar:: This article needs some work
 
    If you have any contribution send us your :pull:`PR <../pulls>`, see
    :ref:`how to contribute`.
 
-To get best results from build, its recommend to install additional packages
+.. contents:: Contents
-on build hosts.
+   :depth: 2
+   :local:
+   :backlinks: entry
 
 .. _docs build:
 

docs/admin/filtron.rst

@@ -9,6 +9,11 @@ How to protect an instance
 
    - :ref:`filtron.sh`
 
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
 .. _filtron: https://github.com/asciimoo/filtron
 
 Searx depens on external search services.  To avoid the abuse of these services

docs/admin/index.rst

@@ -3,9 +3,12 @@ Administrator documentation
 ===========================
 
 .. toctree::
-   :maxdepth: 1
+   :maxdepth: 2
+   :caption: Contents
 
    installation
+   installation-nginx
+   installation-apache
    settings
    api
    architecture

docs/admin/installation-apache.rst

@@ -0,0 +1,94 @@
+.. _installation apache:
+
+===================
+Install with apache
+===================
+
+.. sidebar:: public HTTP servers
+
+   On public searx instances use an application firewall (:ref:`filtron
+   <filtron.sh>`).
+
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+Add wsgi mod
+============
+
+.. tabs::
+
+   .. group-tab:: Ubuntu / debian
+
+      .. code:: sh
+
+         sudo -H apt-get install libapache2-mod-uwsgi
+         sudo -H a2enmod uwsgi
+
+Add this configuration in the file ``/etc/apache2/apache2.conf``.  To limit
+acces to your intranet replace ``Allow from all`` directive and replace
+``192.168.0.0/16`` with your subnet IP/class.
+
+.. _inranet apache site:
+
+Note that if your instance of searx is not at the root, you should change
+``<Location />`` by the location of your instance, like ``<Location /searx>``:
+
+.. code:: apache
+
+   # CustomLog /dev/null combined
+
+   <IfModule mod_uwsgi.c>
+
+     <Location />
+
+          Options FollowSymLinks Indexes
+          SetHandler uwsgi-handler
+          uWSGISocket /run/uwsgi/app/searx/socket
+
+          Order deny,allow
+          Deny from all
+          # Allow from fd00::/8 192.168.0.0/16 fe80::/10 127.0.0.0/8 ::1
+          Allow from all
+
+     </Location>
+
+   </IfModule>
+
+Enable apache mod_uwsgi and restart apache:
+
+.. tabs::
+
+   .. group-tab:: Ubuntu / debian
+
+      .. code:: sh
+
+         a2enmod uwsgi
+         sudo -H systemctl restart apache2
+
+disable logs
+============
+
+For better privacy you can disable Apache logs.  Go back to
+``/etc/apache2/apache2.conf`` :ref:`[example] <inranet apache site>` and above
+``<Location />`` activate directive:
+
+.. code:: apache
+
+    CustomLog /dev/null combined
+
+Restart apache:
+
+.. tabs::
+
+   .. group-tab:: Ubuntu / debian
+
+      .. code:: sh
+
+         sudo -H systemctl restart apache2
+
+.. warning::
+
+   You can only disable logs for the whole (virtual) server not for a specific
+   path.

docs/admin/installation-nginx.rst

@@ -0,0 +1,141 @@
+.. _installation nginx:
+
+==================
+Install with nginx
+==================
+
+.. sidebar:: public HTTP servers
+
+   On public searx instances use an application firewall (:ref:`filtron
+   <filtron.sh>`).
+
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+If nginx is not installed (uwsgi will not work with the package
+nginx-light):
+
+.. tabs::
+
+   .. group-tab:: Ubuntu / debian
+
+      .. code:: sh
+
+         sudo -H apt-get install nginx
+
+Hosted at ``/``
+===============
+
+Create the configuration file ``/etc/nginx/sites-available/searx`` with this
+content:
+
+.. code:: nginx
+
+    server {
+        listen 80;
+        server_name searx.example.com;
+        root /usr/local/searx/searx;
+
+        location /static {
+        }
+
+        location / {
+                include uwsgi_params;
+                uwsgi_pass unix:/run/uwsgi/app/searx/socket;
+        }
+    }
+
+Create a symlink to sites-enabled:
+
+.. code:: sh
+
+   sudo -H ln -s /etc/nginx/sites-available/searx /etc/nginx/sites-enabled/searx
+
+Restart service:
+
+.. tabs::
+
+   .. group-tab:: Ubuntu / debian
+
+      .. code:: sh
+
+         sudo -H systemctl restart nginx
+         sudo -H systemctl restart uwsgi
+
+from subdirectory URL (``/searx``)
+==================================
+
+Add this configuration in the server config file
+``/etc/nginx/sites-enabled/default``:
+
+.. code:: nginx
+
+    location /searx/static {
+            alias /usr/local/searx/searx/static;
+    }
+
+    location /searx {
+            uwsgi_param SCRIPT_NAME /searx;
+            include uwsgi_params;
+            uwsgi_pass unix:/run/uwsgi/app/searx/socket;
+    }
+
+
+**OR** using reverse proxy (Please, note that reverse proxy advised to be used
+in case of single-user or low-traffic instances.)
+
+.. code:: nginx
+
+    location /searx/static {
+            alias /usr/local/searx/searx/static;
+    }
+
+    location /searx {
+        proxy_pass http://127.0.0.1:8888;
+        proxy_set_header Host $host;
+        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+        proxy_set_header X-Scheme $scheme;
+        proxy_set_header X-Script-Name /searx;
+        proxy_buffering off;
+    }
+
+Enable ``base_url`` in ``searx/settings.yml``
+
+.. code:: yaml
+
+    base_url : http://your.domain.tld/searx/
+
+Restart service:
+
+.. tabs::
+
+   .. group-tab:: Ubuntu / debian
+
+      .. code:: sh
+
+         sudo -H systemctl restart nginx
+         sudo -H systemctl restart uwsgi
+
+
+disable logs
+============
+
+For better privacy you can disable nginx logs about searx.  How to proceed:
+below ``uwsgi_pass`` in ``/etc/nginx/sites-available/default`` add:
+
+.. code:: nginx
+
+    access_log /dev/null;
+    error_log /dev/null;
+
+Restart service:
+
+.. tabs::
+
+   .. group-tab:: Ubuntu / debian
+
+      .. code:: sh
+
+         sudo -H systemctl restart nginx

docs/admin/installation.rst

@@ -4,50 +4,64 @@
 Installation
 ============
 
-.. contents::
+.. sidebar:: Searx server setup
-   :depth: 3
+
+   - :ref:`installation nginx`
+   - :ref:`installation apache`
+
+   If you do not have any special preferences, it is recommend to use
+   :ref:`searx.sh`.
+
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
 
 .. _installation basic:
 
 Basic installation
 ==================
 
-.. sidebar:: further reading
+Step by step installation with virtualenv.  For Ubuntu, be sure to have enable
-
+universe repository.
-   - :ref:`searx.sh`
-
-Step by step installation for Debian/Ubuntu with virtualenv. For Ubuntu, be sure
-to have enable universe repository.
 
 Install packages:
 
-.. code:: sh
+.. tabs::
+
+   .. group-tab:: Ubuntu / debian
+
+      .. code-block:: sh
 
          $ sudo -H apt-get install \
-           git build-essential libxslt-dev \
+                   git build-essential
-	   python-dev python-virtualenv python-babel \
+                   libxslt-dev python3-dev python3-babel \
                    zlib1g-dev libffi-dev libssl-dev
 
 Install searx:
 
 .. code:: sh
 
-    cd /usr/local
+    sudo -H useradd searx --system --disabled-password -d /usr/local/searx
-    sudo -H git clone https://github.com/asciimoo/searx.git
+    sudo -H usermod -a -G shadow $SERVICE_USER
-    sudo -H useradd searx -d /usr/local/searx
+    cd /usr/local/searx
+    sudo -H git clone https://github.com/asciimoo/searx.git searx-src
     sudo -H chown searx:searx -R /usr/local/searx
 
-Install dependencies in a virtualenv:
+Install virtualenv:
 
 .. code:: sh
 
-    cd /usr/local/searx
     sudo -H -u searx -i
+    (searx)$ python3 -m venv searx-pyenv
+    (searx)$ echo 'source ~/searx-pyenv/bin/activate' > ~/.profile
+
+Exit the searx bash and restart a new to install the searx dependencies:
 
 .. code:: sh
 
-    (searx)$ virtualenv searx-ve
+    sudo -H -u searx -i
-    (searx)$ . ./searx-ve/bin/activate
+    (searx)$ cd searx-src
     (searx)$ ./manage.sh update_packages
 
 Configuration
@@ -55,7 +69,9 @@ Configuration
 
 .. code:: sh
 
-    sed -i -e "s/ultrasecretkey/`openssl rand -hex 16`/g" searx/settings.yml
+    sudo -H -u searx -i
+    (searx)$ cd searx-src
+    (searx)$ sed -i -e "s/ultrasecretkey/`openssl rand -hex 16`/g" searx/settings.yml
 
 Edit searx/settings.yml if necessary.
 
@@ -66,7 +82,9 @@ Start searx:
 
 .. code:: sh
 
-    python searx/webapp.py
+    sudo -H -u searx -i
+    (searx)$ cd searx-src
+    (searx)$ python3 searx/webapp.py
 
 Go to http://localhost:8888
 
@@ -76,20 +94,21 @@ If everything works fine, disable the debug option in settings.yml:
 
     sed -i -e "s/debug : True/debug : False/g" searx/settings.yml
 
-At this point searx is not demonized ; uwsgi allows this.
+At this point searx is not demonized ; uwsgi allows this.  You can exit the
-
+virtualenv and the searx user bash (enter exit command twice).
-You can exit the virtualenv and the searx user bash (enter exit command
-twice).
 
 uwsgi
 =====
 
 Install packages:
 
-.. code:: sh
+.. tabs::
 
-    sudo -H apt-get install \
+   .. group-tab:: Ubuntu / debian
-         uwsgi uwsgi-plugin-python
+
+      .. code-block:: bash
+
+         sudo -H apt-get install uwsgi uwsgi-plugin-python3
 
 Create the configuration file ``/etc/uwsgi/apps-available/searx.ini`` with this
 content:
@@ -97,36 +116,66 @@ content:
 .. code:: ini
 
    [uwsgi]
+
+   # uWSGI core
+   # ----------
+   #
+   # https://uwsgi-docs.readthedocs.io/en/latest/Options.html#uwsgi-core
+
    # Who will run the code
    uid = searx
    gid = searx
 
+   # chdir to specified directory before apps loading
+   chdir = /usr/local/searx/searx-src/searx
+
    # disable logging for privacy
    disable-logging = true
 
-    # Number of workers (usually CPU count)
-    workers = 4
-
    # The right granted on the created socket
    chmod-socket = 666
 
    # Plugin to use and interpretor config
    single-interpreter = true
+
+   # enable master process
    master = true
-    plugin = python
+
+   # load apps in each worker instead of the master
    lazy-apps = true
+
+   # load uWSGI plugins
+   plugin = python3,http
+
+   # By default the Python plugin does not initialize the GIL.  This means your
+   # app-generated threads will not run.  If you need threads, remember to enable
+   # them with enable-threads.  Running uWSGI in multithreading mode (with the
+   # threads options) will automatically enable threading support. This *strange*
+   # default behaviour is for performance reasons.
    enable-threads = true
 
-    # Module to import
+   # plugin: python
+   # --------------
+   #
+   # https://uwsgi-docs.readthedocs.io/en/latest/Options.html#plugin-python
+
+   # load a WSGI module
    module = searx.webapp
 
-    # Support running the module from a webserver subdirectory.
+   # set PYTHONHOME/virtualenv
-    route-run = fixpathinfo:
+   virtualenv = /usr/local/searx/searx-pyenv
 
-    # Virtualenv and python path
+   # add directory (or glob) to pythonpath
-    virtualenv = /usr/local/searx/searx-ve/
+   pythonpath = /usr/local/searx/searx-src
-    pythonpath = /usr/local/searx/
+
-    chdir = /usr/local/searx/searx/
+
+   # plugin http
+   # -----------
+   #
+   # https://uwsgi-docs.readthedocs.io/en/latest/Options.html#plugin-http
+
+   # Native HTTP support: https://uwsgi-docs.readthedocs.io/en/latest/HTTP.html
+   http = 127.0.0.1:8888
 
 Activate the uwsgi application and restart:
 
@@ -136,194 +185,27 @@ Activate the uwsgi application and restart:
     ln -s ../apps-available/searx.ini
     /etc/init.d/uwsgi restart
 
-Web server
-==========
-
-with nginx
-----------
-
-If nginx is not installed (uwsgi will not work with the package
-nginx-light):
-
-.. code:: sh
-
-    sudo -H apt-get install nginx
-
-Hosted at /
-~~~~~~~~~~~
-
-Create the configuration file ``/etc/nginx/sites-available/searx`` with this
-content:
-
-.. code:: nginx
-
-    server {
-        listen 80;
-        server_name searx.example.com;
-        root /usr/local/searx/searx;
-
-        location /static {
-        }
-
-        location / {
-                include uwsgi_params;
-                uwsgi_pass unix:/run/uwsgi/app/searx/socket;
-        }
-    }
-
-Create a symlink to sites-enabled:
-
-.. code:: sh
-
-   sudo -H ln -s /etc/nginx/sites-available/searx /etc/nginx/sites-enabled/searx
-
-Restart service:
-
-.. code:: sh
-
-    sudo -H service nginx restart
-    sudo -H service uwsgi restart
-
-from subdirectory URL (/searx)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Add this configuration in the server config file
-``/etc/nginx/sites-enabled/default``:
-
-.. code:: nginx
-
-    location /searx/static {
-            alias /usr/local/searx/searx/static;
-    }
-
-    location /searx {
-            uwsgi_param SCRIPT_NAME /searx;
-            include uwsgi_params;
-            uwsgi_pass unix:/run/uwsgi/app/searx/socket;
-    }
-
-
-**OR** using reverse proxy (Please, note that reverse proxy advised to be used
-in case of single-user or low-traffic instances.)
-
-.. code:: nginx
-
-    location /searx/static {
-            alias /usr/local/searx/searx/static;
-    }
-
-    location /searx {
-        proxy_pass http://127.0.0.1:8888;
-        proxy_set_header Host $host;
-        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-        proxy_set_header X-Scheme $scheme;
-        proxy_set_header X-Script-Name /searx;
-        proxy_buffering off;
-    }
-
-
-Enable ``base_url`` in ``searx/settings.yml``
-
-.. code:: yaml
-
-    base_url : http://your.domain.tld/searx/
-
-Restart service:
-
-.. code:: sh
-
-    sudo -H service nginx restart
-    sudo -H service uwsgi restart
-
-disable logs
-^^^^^^^^^^^^
-
-for better privacy you can disable nginx logs about searx.
-
-how to proceed: below ``uwsgi_pass`` in ``/etc/nginx/sites-available/default``
-add:
-
-.. code:: nginx
-
-    access_log /dev/null;
-    error_log /dev/null;
-
-Restart service:
-
-.. code:: sh
-
-    sudo -H service nginx restart
-
-with apache
------------
-
-Add wsgi mod:
-
-.. code:: sh
-
-    sudo -H apt-get install libapache2-mod-uwsgi
-    sudo -H a2enmod uwsgi
-
-Add this configuration in the file ``/etc/apache2/apache2.conf``:
-
-.. code:: apache
-
-    <Location />
-        Options FollowSymLinks Indexes
-        SetHandler uwsgi-handler
-        uWSGISocket /run/uwsgi/app/searx/socket
-    </Location>
-
-Note that if your instance of searx is not at the root, you should change
-``<Location />`` by the location of your instance, like ``<Location /searx>``.
-
-Restart Apache:
-
-.. code:: sh
-
-    sudo -H /etc/init.d/apache2 restart
-
-disable logs
-~~~~~~~~~~~~
-
-For better privacy you can disable Apache logs.
-
-.. warning::
-
-   You can only disable logs for the whole (virtual) server not for a specific
-   path.
-
-Go back to ``/etc/apache2/apache2.conf`` and above ``<Location />`` add:
-
-.. code:: apache
-
-    CustomLog /dev/null combined
-
-Restart Apache:
-
-.. code:: sh
-
-    sudo -H /etc/init.d/apache2 restart
 
 How to update
 =============
 
 .. code:: sh
 
-    cd /usr/local/searx
     sudo -H -u searx -i
-
-.. code:: sh
-
-    (searx)$ . ./searx-ve/bin/activate
     (searx)$ git stash
     (searx)$ git pull origin master
     (searx)$ git stash apply
     (searx)$ ./manage.sh update_packages
 
+Restart uwsgi:
+
+.. tabs::
+
+   .. group-tab:: Ubuntu / debian
+
       .. code:: sh
 
-    sudo -H service uwsgi restart
+         sudo -H systemctl restart uwsgi
 
 Docker
 ======

docs/admin/settings.rst

@@ -4,11 +4,17 @@
 ``settings.yml``
 ================
 
+This page describe the options possibilities of the :origin:`searx/settings.yml`
+file.
+
 .. sidebar:: Further reading ..
 
    - :ref:`search API`
 
-This page describe the options possibilities of the settings.yml file.
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
 
 .. _settings global:
 

docs/blog/index.rst

@@ -3,7 +3,8 @@ Blog
 ====
 
 .. toctree::
-   :maxdepth: 1
+   :maxdepth: 2
+   :caption: Contents
 
    python3
    admin

docs/dev/contribution_guide.rst

@@ -4,6 +4,11 @@
 How to contribute
 =================
 
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
 Prime directives: Privacy, Hackability
 ======================================
 

docs/dev/index.rst

@@ -3,7 +3,8 @@ Developer documentation
 =======================
 
 .. toctree::
-   :maxdepth: 1
+   :maxdepth: 2
+   :caption: Contents
 
    quickstart
    contribution_guide

docs/index.rst

@@ -2,7 +2,14 @@
 Welcome to searx
 ================
 
-Search without being tracked.
+    *Search without being tracked.*
+
+Searx is a free internet metasearch engine which aggregates results from more
+than 70 search services.  Users are neither tracked nor profiled.  Additionally,
+searx can be used over Tor for online anonymity.
+
+Get started with searx by using one of the :wiki:`Searx-instances`.  If you
+don't trust anyone, you can set up your own, see :ref:`installation`.
 
 .. sidebar::  Features
 
@@ -16,15 +23,9 @@ Search without being tracked.
    - Hosted by organizations, such as *La Quadrature du Net*, which promote
      digital rights
 
-Searx is a free internet metasearch engine which aggregates results from more
-than 70 search services.  Users are neither tracked nor profiled.  Additionally,
-searx can be used over Tor for online anonymity.
-
-Get started with searx by using one of the :wiki:`Searx-instances`.  If you
-don't trust anyone, you can set up your own, see :ref:`installation`.
-
 .. toctree::
    :maxdepth: 2
+   :caption: Contents
 
    user/index
    admin/index

docs/user/index.rst

@@ -3,7 +3,8 @@ User documentation
 ==================
 
 .. toctree::
-   :maxdepth: 1
+   :maxdepth: 2
+   :caption: Contents
 
    public_instances
    search_syntax

docs/user/own-instance.rst

@@ -2,8 +2,10 @@
 Why use a private instance?
 ===========================
 
-"Is it worth to run my own instance?" is a common question among searx users.
+  *"Is it worth to run my own instance?"*
-Before answering this question, see what options a searx user has.
+
+\.\. is a common question among searx users.  Before answering this question,
+see what options a searx user has.
 
 Public instances are open to everyone who has access to its URL.  Usually, these
 are operated by unknown parties (from the users' point of view).  Private

docs/user/public_instances.rst

@@ -1,16 +1,20 @@
-.. _public instances:
-
 ..
    links has been ported from markdown to reST by::
 
       regexpr:        \[([^\]]*)\]\(([^)]*)\)
       substitution:  `\1 <\2>`__
 
+.. _public instances:
 
 ======================
 Public Searx instances
 ======================
 
+.. contents:: Contents
+   :depth: 2
+   :local:
+   :backlinks: entry
+
 .. _mailing list: mailto:searx-instances@autistici.org
 .. _subscription page: https://www.autistici.org/mailman/listinfo/searx-instances
 

docs/utils/filtron.sh.rst

@@ -7,6 +7,7 @@
 
 .. sidebar:: further reading
 
+   - :ref:`installation`
    - :ref:`searx_filtron`
    - :ref:`architecture`
 
@@ -40,8 +41,6 @@ into this user account:
 Public Reverse Proxy
 ====================
 
-.. tabs::
-
 To install searx in your public HTTP server use:
 
 .. code::  bash

docs/utils/index.rst

@@ -6,17 +6,7 @@
 Tooling box ``utils/*``
 =======================
 
-In the folder :origin:`utils/` we maintain some tools useful for admins and
+.. sidebar:: Work needed!
-developers.
-
-.. toctree::
-   :maxdepth: 1
-
-   searx.sh
-   filtron.sh
-   morty.sh
-
-.. admonition:: Work needed!
 
    Our scripts to maintain services do most support only systemd init process
    used by debian, ubuntu and many other dists.  In general our scripts are only
@@ -24,6 +14,17 @@ developers.
    you have any contribution, please send us your :pull:`PR <../pulls>`, see
    :ref:`how to contribute`.
 
+In the folder :origin:`utils/` we maintain some tools useful for admins and
+developers.
+
+.. toctree::
+   :maxdepth: 2
+   :caption: Contents
+
+   searx.sh
+   filtron.sh
+   morty.sh
+
 .. _toolboxing common:
 
 Common commands

docs/utils/morty.sh.rst

@@ -10,6 +10,7 @@
 
 .. sidebar:: further reading
 
+   - :ref:`installation`
    - :ref:`architecture`
 
 To simplify installation and maintenance of a morty_ instance you can use the

docs/utils/searx.sh.rst

@@ -9,10 +9,15 @@
 
    - :ref:`installation`
    - :ref:`architecture`
+   - :ref:`filtron.sh`
 
 To simplify installation and maintenance of a searx instance you can use the
-script :origin:`utils/searx.sh`.  In most cases you will install searx simply by
+script :origin:`utils/searx.sh`.
-running the command:
+
+Install
+=======
+
+In most cases you will install searx simply by running the command:
 
 .. code::  bash
 
@@ -27,17 +32,17 @@ into this user account.  The installation is described in chapter
 Intranet Reverse Proxy
 ======================
 
-.. warning::
-
-   This setup is **not** suitable **for public instances**, go on with
-   :ref:`reverse proxy`!
-
 To install searx in your intranet HTTP server use:
 
 .. code::  bash
 
    sudo -H ./utils/searx.sh apache install
 
+.. warning::
+
+   This setup is **not** suitable **for public instances**, go on with
+   :ref:`reverse proxy`!
+
 .. tabs::
 
    .. group-tab:: apache

utils/searx.sh

@@ -39,8 +39,9 @@ SEARX_UWSGI_SOCKET="/run/uwsgi/app/searx/socket"
 
 SEARX_APT_PACKAGES="\
   uwsgi uwsgi-plugin-python3 \
-  git build-essential libxslt-dev python3-dev python3-babel zlib1g-dev \
+  git build-essential \
-  libffi-dev libssl-dev \
+  libxslt-dev python3-dev python3-babel\
+  zlib1g-dev libffi-dev libssl-dev \
 "
 
 # Apache Settings