Commit Graph

326 Commits

Author SHA1 Message Date
Markus Heiser ee8b6aac36 [fix] searx.sh update: replace git pull by a hard reset
If the fetched branch has been rebased a 'git pull' will fails.  To get fetched
branch in the working tree, a git reset is needed.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-26 15:15:17 +01:00
Markus Heiser ef3f084c49
Merge pull request #516 from return42/searxng/filtron
[mod] filtron.sh - install github.com/searxng/filtron
2021-11-23 00:23:23 +01:00
Markus Heiser 93e933278a [fix] filtron.sh & morty.sh: quote "$GO_ENV"
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-21 17:13:34 +01:00
Markus Heiser c1bee0d96f [mod] filtron.sh - install github.com/searxng/filtron
SearXNG maintains a fork of filtron at [1]

[1] https://github.com/searxng/filtron

Related: https://github.com/searxng/searxng/issues/507#issuecomment-970254027
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-21 17:13:34 +01:00
Markus Heiser 5242a841a5 [mod] NVM: dev-tools pre-installed in NVM's node installation
The Node.js installation in the NVM environment can be used by IDEs and other
developer tasks.  The required developer packagaes are added to the file
./.nvm_packages and will be installed when Node.js is installed.  Initial we
start with:

- eslint

Having a dedicated developer enviroment, provided by nvm makes it easy to
integrate Node.js packages into various IDEs.  One example is shown in the
.dir-locals.el which is used by emacs.

[1] https://github.com/nvm-sh/nvm#default-global-packages-from-file-while-installing
[2] https://eslint.org

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-20 17:08:22 +01:00
Markus Heiser 69dd025da9 [mod] nvm.cmd ... : run command ... in NVM environment
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-20 15:40:18 +01:00
Markus Heiser 6b4cad768d [mod] NVM: pin Node.js to v16.13.0
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-20 15:19:17 +01:00
Markus Heiser 3e2f3900b8 [fix] nvm.install: add missing popd / node.env exit with error (254)
Issue::

    $ make clean node.env
    ...
    CLEAN     [NVM] drop .nvm/
    ...
    INFO:  install Node.js by NVM
    ...
    Now using node v16.13.0 (npm v8.1.0)
    ...
    INSTALL   searx/static/themes/oscar/package.json
    npm ERR! code ENOENT
    npm ERR! syscall open
        # Here now comes the issue, caused by the missing 'popd' ..
    npm ERR! path SearXNG/.nvm/searx/static/themes/oscar/package.json
    npm ERR! errno -2
    npm ERR! enoent ENOENT: no such file or directory, open 'SearXNG/.nvm/searx/static/themes/oscar/package.json'
    ERROR: node.env exit with error (254)
    make: *** [Makefile:99: node.env] Error 254

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-20 15:16:14 +01:00
Markus Heiser c681570bb9 [fix] brand - partial revert of e82e37ce6
In commit e82e37ce6 accidentally the brand file was touched / partial
revert::

  export GIT_URL='https://github.com/searxng/searxng'
  export GIT_BRANCH='master'

Related: https://github.com/searxng/searxng/pull/515

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-19 14:01:58 +01:00
Markus Heiser 48752815ed [fix] make nvm.clean - don't raise error when NVM is not installed
Issue was::

    $ LANG=C make nvm.clean
    INFO:  NVM is not installed
    make: *** [Makefile:99: nvm.clean] Error 42

Now::

    $ LANG=C make nvm.clean
    CLEAN     [NVM] not installed

BTW: change info_msg to build_msg

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-19 13:19:54 +01:00
Markus Heiser e82e37ce6d
Merge pull request #515 from return42/doc-theme
Various modification of the documentation and the theme sphinx-theme
2021-11-19 11:46:29 +01:00
Markus Heiser 1e997386e9
Merge pull request #508 from return42/arm64
[mod] Tools to install and maintain golang binaries & packages.
2021-11-18 19:49:01 +00:00
Markus Heiser 1dae0c0be0 [brand] SearXNG - docs rename links and fix documentation
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-18 18:27:26 +01:00
Markus Heiser dc1442a2d1 [mod] Tools to install and maintain NVM versions manager for Node.js
[1] https://github.com/nvm-sh/nvm

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-16 23:31:11 +01:00
Markus Heiser 79b41478ac [fix] filtron & morty - install golang binary that fits to arch & os
Closes: https://github.com/searxng/searxng/issues/507
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-16 20:44:53 +01:00
Markus Heiser abd5eee392 [mod] Tools to install and maintain golang [1] binaries & packages.
[1] https://golang.org/doc/devel/release#policy

A simple *helloworld* test with user 'my_user' :

  sudo -H adduser my_user
  ./manage go.golang go1.17.3 my_user
  ./manage go.install github.com/go-training/helloworld@latest my_user
  ./manage go.bash my_user
  $ helloword
  Hello World!!

Don't forget to remove 'my_user':  sudo -H deluser --remove-home my_user

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-16 18:00:47 +01:00
MrPaulBlack 566dfe3330 [simple theme] update to ion icons 5 and drop webfont
* update from ionicons-npm to ionicons ver.5
* drop the webfont built by grunt for icons
* built icons.html template for inlining icons with jinja2 into html
* update icon to use mostly the outline version
* add icons to categories and do not display them on mobile to save space
* remove all legacy ion icon font files from simple theme
* icons.html is added in this commit since make statc.build.restore requires git to know the file already
* cleanup error-dialog
2021-11-13 10:42:07 +01:00
Markus Heiser 3a981257de [simple theme] create favicon.png from src/svg/searxng-wordmark.svg
new bash function convert_if_newer() / usage::

    convert_if_newer <origfile> <outfile> [<options>, ...]
    convert_if_newer "path/to/origin.svg" "path/to/converted.png" -transparent white -resize 64x64

Run's ImageMagik' convert comand to generate <outfile> from <origfile>, if
<origfile> is newer than <outfile>.  The command line is to convert is::

    convert <origfile> [<options>, ...] <outfile>

PNG 'searx/static/themes/simple/img/favicon.png' has been created by::

  $ make themes.simple
  CONVERT   searx/static/themes/simple/src/svg/searxng-wordmark.svg -transparent white -resize 64x64 searx/static/themes/simple/img/favicon.png
  ...

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-13 09:05:20 +01:00
Markus Heiser bf52353a5c [enh] minify searxng-wordmark.svg (HTMLMinifier)
Command::

  ./node_modules/.bin/html-minifier \
      --remove-comments \
      --collapse-whitespace \
      src/svg/searxng-wordmark.svg \
      -o ../../../templates/__common__/searxng-wordmark.min.svg

- html-minifier: https://github.com/kangax/html-minifier
  - onilne: https://kangax.github.io/html-minifier
  - grunt: https://www.npmjs.com/package/grunt-contrib-htmlmin
  - grunt-contrib-htmlmin: https://github.com/gruntjs/grunt-contrib-htmlmin
  - npm: https://www.npmjs.com/package/html-minifier

To test, rebuild your node environment::

  make node.env

Alternatives:

- pretty-data: https://github.com/vkiryukhin/pretty-data
  - grunt: https://www.npmjs.com/package/grunt-xmlmin
  - grunt-xmlming: https://github.com/dtrunk90/grunt-xmlmin
  - npm: https://www.npmjs.com/package/grunt-xmlmin

- minify-xml: https://github.com/kristian/minify-xml
  - no grunt package available
  - npm: https://www.npmjs.com/package/minify-xml

src/svg/searxng-wordmark.svg':
'../../../templates/__common__/searxng-wordmark.min.svg'

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-11-13 09:05:20 +01:00
Markus Heiser 9f30d5b1f8 [fix] morty & filtron: don't use golang installed by package manager
When golang is installed via a package manager the local version, which filtron
downloads, is ignored.

BTW: With the new go1.17.2 (cfea51f4), 'go get' is depricated::

  go get: installing executables with 'go get' in module mode is deprecated.
    Use 'go install pkg@version' instead.
    For more information, see https://golang.org/doc/go-get-install-deprecation
    or run 'go help get' or 'go help install'.

Reported-by: @tiekoetter https://github.com/searxng/searxng/pull/455#issuecomment-954918411
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-30 19:18:32 +02:00
Markus Heiser db6a4d13e7 [mod] searx, filtron & morty scripts: add 'reinstall all' command
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-30 16:11:01 +02:00
Markus Heiser 4167bf15dc [mod] utils/searx.sh - apache, nginx & uwsgi check old 'searx' installations
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-30 16:11:01 +02:00
Markus Heiser 38482739de [mod] utils/searx.sh - check a SearXNG installation
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-30 16:11:01 +02:00
Markus Heiser 2b1252148d [brand] SearXNG - nginx & apache searxng.conf, uwsgi searxng.conf
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-30 16:11:01 +02:00
Markus Heiser a9fc4885f2 [brand] SearXNG - bash env SEARXNG_URL
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-30 16:11:01 +02:00
Markus Heiser cfea51f40f [mod] utils/morty.sh - check golang version is go1.17.2
Related-to: 5c4afdd7 https://github.com/searxng/searxng/issues/455

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-29 15:51:30 +00:00
Markus Heiser f3c161f72a [mod] utils/morty.sh - check a morty installation
Impplement a script to check a morty installation::

   ./utils/morty.sh install check

Related-to: https://github.com/searxng/searxng/issues/450
Related-to: https://github.com/searxng/searxng/pull/446#issuecomment-954599668

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-29 15:51:30 +00:00
Markus Heiser d8d6c3d4ee [mod] utils/filtron.sh - check golang version is go1.17.2
Related-to: 5c4afdd7 https://github.com/searxng/searxng/issues/455

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-29 15:51:30 +00:00
Markus Heiser 064b545f14 [mod] utils/filtron.sh - check a filtron installation
Impplement a script to check a filtron installation::

   ./utils/filtron.sh install check

Related-to: https://github.com/searxng/searxng/issues/450
Related-to: https://github.com/searxng/searxng/pull/446#issuecomment-954599668

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-29 15:51:30 +00:00
Markus Heiser f46508b35f [mod] utils/searxng_check.py - check a SearXNG installation
Impplement a script to check a SearXNG installation::

   ./utils/searx.sh install check

Related-to: https://github.com/searxng/searxng/issues/450
Related-to: https://github.com/searxng/searxng/pull/446#issuecomment-954599668
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-29 15:51:30 +00:00
Alexandre Flament 5c4afdd737 [fix] filtron.sh: fix filtron proxy error
morty & filtron upgrade golang from v1.13.5 to v1.17.2 (#stable go[1])

Closes: #443

[1] https://golang.org/dl/#stable
2021-10-29 14:09:58 +00:00
Markus Heiser 60edf2623d [brand] SearXNG - reference /etc/searxng/settings.yml
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-11 12:28:55 +00:00
Alexandre Flament 3fad483b7a SearXNG: partial update: reference /etc/searxng/settings.yml 2021-10-11 12:28:55 +00:00
Alexandre Flament 253b850376 SearXNG: SEARXNG_SETTINGS_PATH 2021-10-02 17:18:05 +02:00
Alexandre Flament 9e266ecad3 SearXNG: SEARX_BIND_ADDRESS 2021-10-02 16:58:09 +02:00
Alexandre Flament 6443ed7562 SearXNG: SEARXNG_PORT 2021-10-02 16:56:24 +02:00
Alexandre Flament e5e322e8ed [mod] utils/lib.sh: add pyenv.activate
* initialize virtualenv (pyenv.install)
* activate virtualenv

close #310
2021-09-17 14:15:35 +02:00
Allen 08ac175236 [fix] Wording for Morty & Filtron error message 2021-09-04 10:22:52 +02:00
Markus Heiser 8ba4dc26d1 [fix] utils/searx.sh install settings - do not eval the template
The settings template in utils/templates/etc/searx/settings.yml
does not include any replacements (shell environment variables).

BTW: utils/filtron.sh & utils/lib.sh
  Add some more messages to installation procedures, to be more clear.

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-08-20 14:21:54 +02:00
Alexandre Flament a7a181064f manage script: remove unused reference to utils/brand.env and .config.sh 2021-08-01 07:05:00 +02:00
Markus Heiser 5b3eeb6e54 [mod] searx.sh - improve some messages prompted while installation
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-30 14:53:09 +02:00
Markus Heiser 85e0908e4f [fix] searx.sh - get_installed_version_variables()
Fix the quotation marks in the command line to run

    python -m searx.version

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-30 14:53:09 +02:00
Markus Heiser 64d64535a2 [fix] buildenv: first unset environment / then import searx package
The first import of names from the searx package implies loading the
settings.yml.  Before this is done, the enviroment variables must be unset to
not overwrite the values from the settings.yml

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-30 14:53:09 +02:00
Alexandre Flament 4b43775c91 version based on the git repository
This commit remove the need to update the brand for GIT_URL and GIT_BRANCH:
there are read from the git repository.

It is possible to call python -m searx.version freeze to freeze the current version.
Useful when the code is installed outside git (distro package, docker, etc...)
2021-07-30 14:40:09 +02:00
Alexandre Flament 24fcd7475a Revert "[brand] !!DO NOT MERGE!! this patch uses branch fix-searx.sh @return42"
This reverts commit e946c3462c.
2021-07-27 08:40:53 +02:00
Markus Heiser e946c3462c [brand] !!DO NOT MERGE!! this patch uses branch fix-searx.sh @return42
Do not merge this patch in master branch of SearXNG!  This branch exists only
for testing the feature branch fix-searx.sh @return42.

This patch changes the buildenv to::

    GIT_URL='https://github.com/return42/searxng'
    GIT_BRANCH='fix-searx.sh'
    SEARX_PORT='7777'
    SEARX_BIND_ADDRESS='127.0.0.12'

To test installation procedure, clone feature branch (fix-searx.sh)::

    $ cd ~/Downloads
    $ git clone --branch fix-searx.sh https://github.com/return42/searxng searxng
    $ cd searxng

    $ ./utils/searx.sh install all
    ...

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-24 16:28:15 +02:00
Markus Heiser 7196a9b5dd [fix] ./utils/filtron.sh - FILTRON_TARGET from YAML settings
The filtron target is the SearXNG installation and the default of FILTRON_TARGET
is taken from the YAML configuration ('server.port' & 'server.bind_address').

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-24 16:28:15 +02:00
Markus Heiser 61d3914b63 [fix] test.robot - do not use environment from ./utils/brand.env
Some defaults in the settings.yml are taken from the environment.
By example;

The manage scripts sources the ./utils/brand.env and sets SEARX_PORT
environment.  This enviroment *wins over* any settings in a YAML file.
Whe we run a::

    make test.robot

The searx/settings_robot.yml is used, in this file the server settings are::

    server:
      port: 11111
      bind_address: 127.0.0.1

To get in use of the 'port: 11111' we have to unset the SEARX_PORT environment
which was sourced before.

The function buildenv.unset_env() can be called in all use cases where the
enviroment from ./utils/brand.env is not wanted.  ATM it unset the enviroment
variables::

    unset GIT_URL
    unset GIT_BRANCH
    unset SEARX_URL
    unset SEARX_PORT
    unset SEARX_BIND_ADDRESS

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-24 16:28:15 +02:00
Markus Heiser b6a55e223c [mod] reduce enviroment variables in shell scripts to what is needed
Not all settings from the 'brand:' section of the YAML files are needed in the
shell scripts.  This patch reduce the variables in ./utils/brand.env to what is
needed.  The following ('brand:' settings) can be removed from this file:

- ISSUE_URL
- DOCS_URL
- PUBLIC_INSTANCES
- WIKI_URL

Tasks running outside of an *installed instance*, need the following settings
from the YAML configuration:

- GIT_URL            <--> brand.git_url
- GIT_BRANCH         <--> brand.git_branch
- SEARX_URL          <--> server.base_url  (aka PUBLIC_URL)
- SEARX_PORT         <--> server.port
- SEARX_BIND_ADDRESS <--> server.bind_address

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-24 16:28:15 +02:00
Markus Heiser 57b8f340a6 [mod] utils/searx.sh - add command 'inspect settings'
Inspect YAML setting <key> from SearXNG instance (${SEARX_SRC})::

  utils/searx.sh inspect settings server.base_url

utils/lib_install.sh
  should not log on stdout which is used for the output of function
  prompt_installation_setting().  Turned build_msg (stdout) into
  info_msg (stderr).

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-24 16:24:22 +02:00