Commit Graph

5862 Commits

Author SHA1 Message Date
Markus Heiser 14b09c15c4 [fix] simple theme: use stylint to fix common lint errors
This fix was autogenerated by::

     npx stylelint -f unix --fix 'searx/static/themes/simple/src/less/**/*.less'

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-24 13:27:21 +02:00
Markus Heiser dfc5c3bc15 [mod] add stylelint process to target themes.simple
Before build, do lint the LESS files using stylelint [1].  The configuration
'stylelint-config-standard' [2] is added to the root of the simple theme [3].

[1] https://stylelint.io/
[2] https://github.com/stylelint/stylelint-config-standard
[3] https://stylelint.io/user-guide/configure

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-24 13:27:21 +02:00
Alexandre Flament e73a96b5f9
Merge pull request #172 from return42/fix-searx.sh
[fix] utils/searx.sh: signature of YAML settings has been changed
2021-06-23 15:02:08 +02:00
Markus Heiser c8a6548592 [mod] utils/lxc.sh: detect conflict of docker & LXC in the iptables
Docker is blocking network of existing LXC containers / there is a conflict in
the iptables setup of Docker & LXC.  With this patch:

- utils/lxc.sh checks internet connectivity (instead of silently hang)
- Chapter "Internet Connectivity & Docker" describes the problem and made a
  suggestion for a solution a solution

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-23 12:47:52 +02:00
Markus Heiser b5f5ddd5a6 [fix] utils/searx.sh: signature of YAML settings has been changed
In 5a7b12ee we changed the signature of the YAML settings, this patch takes this
into account.

Related-to: 5a7b12ee [yamllint] searx/settings.yml
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-23 09:56:59 +02:00
Markus Heiser 056550d523
Merge pull request #169 from return42/tidy-ignorelists
[fix] tidy up ignore lists .gitignore & .dockerignore
2021-06-22 20:21:30 +00:00
Markus Heiser 98d1661f72
Merge pull request #171 from searxng/build_static_cleanup
Build static cleanup
2021-06-22 19:59:01 +00:00
Alexandre Flament 5742267e35 [fix] update .gitattributes 2021-06-22 20:34:39 +02:00
Alexandre Flament 915f3b0bdd [fix] Makefile: remove themes.bootstrap 2021-06-22 20:34:25 +02:00
Markus Heiser 5081cd4ed7
Merge pull request #170 from return42/emacs
[emacs] .dir-locals.el: add setup for json-mode
2021-06-22 15:48:18 +00:00
Markus Heiser 5a8fed48c9 [emacs] .dir-locals.el: add setup for json-mode
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-22 17:46:46 +02:00
Markus Heiser 7ef6ffa803 [fix] tidy up ignore lists .gitignore & .dockerignore
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-22 16:55:30 +02:00
Alexandre Flament 2530576f24
Merge pull request #157 from searxng/mod_static_build
static build: move generated files to searx/static/themes/*/src/generated
2021-06-22 10:36:19 +02:00
Markus Heiser c1f72b71ba [fix] .gitattributes: bootstrap.css & .js has been moved to oscar
see 86912e22 [mod] oscar: get bootstrap and typeahead from NPM

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-22 10:24:05 +02:00
Markus Heiser 1559ed386e [fix] pygments.less - remove pyenv_OK condition
BTW: add pygments.less to the help message

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-22 10:22:33 +02:00
Alexandre Flament 04492b76de [themes] ignore packages-lock.json 2021-06-22 08:17:06 +02:00
Alexandre Flament 2bc9b00cd7 Static build 2021-06-22 08:15:17 +02:00
Alexandre Flament dca3bcca9e [mod] simple theme: include fonts
"npm run webfont" to build the fonts directory.
It requires fontforge and ttfautohint distro packages.

partial revert of commit 7137d2893f
2021-06-22 08:15:17 +02:00
Alexandre Flament d20f6a1f19 [mod] make themes.all update pygments*.less
Add a searx/static/themes/*/src/generated empty folder
2021-06-22 08:15:11 +02:00
Alexandre Flament fb2f4ba902
Merge pull request #164 from searxng/docker_upgrade_alpine
Docker: upgrade to alpine 3.14
2021-06-22 07:27:19 +02:00
Markus Heiser 089b97156a
Merge pull request #167 from return42/fix-166
[fix] UnicodeEncodeError for preferences ULR
2021-06-21 23:37:09 +00:00
Markus Heiser 5973491701 [fix] UnicodeEncodeError for preferences ULR
Closes: https://github.com/searxng/searxng/issues/166

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-22 01:35:54 +02:00
Alexandre Flament f3e56836d6
Merge pull request #165 from return42/patch-google
improve & document google engine
2021-06-21 20:12:07 +02:00
Markus Heiser 0ef6aa5126 [docs] add documentation from the sources of the google engines
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-21 18:25:52 +02:00
Alexandre Flament ef12c41206
Docker: upgrade to alpine 3.14
Uses Python version 3.9 instead of 3.8
2021-06-21 18:03:54 +02:00
Markus Heiser 05e90f2e57 [fix] google answers: normalize space of the answers.
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-21 16:50:25 +02:00
Markus Heiser f096d68ec6 [mod] google engine: reduce mobile UI parameters to what is needed
Reverse engineering shows that not all of the parameters used by google's mobile
UI (aka "more results" button) are needed [1].

[1] https://github.com/searxng/searxng/pull/160#issuecomment-865013625

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-21 16:50:23 +02:00
Alexandre Flament f4da4bafa7
Merge pull request #160 from searxng/google_mobile_ui
[experimental] google: use the mobile UI
2021-06-21 14:56:50 +02:00
Alexandre Flament bf5f505d74
Merge pull request #161 from return42/fix-typo-manage
[fix] py.build - fix small typo in the build message
2021-06-21 14:55:28 +02:00
Alexandre Flament 7a5c36408a [mod] google: add "use_mobile_ui" parameter to use mobile endpoint.
disable by default, it has to be enabled in settings.yml

related to  #159
2021-06-21 14:52:04 +02:00
Markus Heiser f41b7a7204 [fix] py.build - fix small typo in the build message
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-21 12:52:26 +02:00
Alexandre Flament bb3e67f72b [mod] simple theme: normalize.css becomes a packages.json dependency 2021-06-21 08:28:29 +02:00
Alexandre Flament 0e3a87b73a
Merge pull request #158 from return42/fix-node.clean
[fix] node.clean - ignore npm dependencies when npm is not installed
2021-06-20 07:20:38 +02:00
Markus Heiser a62e1123e2 [fix] node.clean - ignore npm dependencies when npm is not installed
error pattern::

    $ make clean
    CLEAN     pyenv
    PYENV     [virtualenv] drop local/py3
    CLEAN     docs -- build/docs dist/docs
    CLEAN     locally installed npm dependencies
    ./manage: line 318: npm: command not found
    ERROR: node.clean exit with error (127)
    make: *** [Makefile:90: node.clean] Error 127

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-19 16:58:18 +02:00
Alexandre Flament eb6832eb95
Merge pull request #156 from return42/fix-google-news
Fix google news
2021-06-18 14:01:03 +02:00
Markus Heiser 9328c66e93 [fix] google news - send CONSENT Cookie to not be redirected
In the EU there exists a "General Data Protection Regulation" [1] aka GDPR (BTW:
very user friendly!) which requires consent to tracking.  To get the consent
from the user, google-news requests are redirected to confirm and get a CONSENT
Cookie from https://consent.google.de/s?continue=...

This patch adds a CONSENT Cookie to the google-news request to avoid
redirection.

The behavior of the CONTENTS cookies over all google engines seems similar but
the pattern is not yet fully clear to me, here are some random samples from my
analysis ..

Using common google search from different domains::

    google.com:        CONSENT=YES+cb.{{date}}-14-p0.de+FX+816
    google.de:         CONSENT=YES+cb.{{date}}-14-p0.de+FX+333
    google.fr:         CONSENT=YES+srp.gws-{{date}}-0-RC2.fr+FX+826

When searching about videos (google-videos)::

    google.es:         CONSENT=YES+srp.gws-{{date}}-0-RC2.es+FX+076
    google.de:         CONSENT=YES+srp.gws-{{date}}-0-RC2.de+FX+171

Google news has only one domain for all languages::

    news.google.com:   CONSENT=YES+cb.{{date}}-14-p0.de+FX+816

Using google-scholar search from different domains::

    scholar.google.de: CONSENT=YES+cb.{{date}}-14-p0.de+FX+333
    scholar.google.fr: does not use such a cookie / did not ask the user
    scholar.google.es: does not use such a cookie / did not ask the user

Interim summary:

  Pattern is unclear and I won't apply the CONSENT cookie to all google engines.
  More experience is need before we generalize the CONSENT cookies over all
  google engines.

Related:

- e9a6ab401 [fix] youtube - send CONSENT Cookie to not be redirected
- https://github.com/benbusby/whoogle-search/issues/311
- https://github.com/benbusby/whoogle-search/issues/243

[1] https://en.wikipedia.org/wiki/General_Data_Protection_Regulation
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-18 13:21:20 +02:00
Markus Heiser dd7b53d369 [fix] google-news engine - KeyError: 'hl in request
Since we added

- 1c67b6aec [enh] google engine: supports "default language"

there is a KeyError: 'hl in request,error pattern::

    ERROR:searx.searx.search.processor.online:engine google news : exception : 'hl'
    Traceback (most recent call last):
      File "searx/search/processors/online.py", line 144, in search
        search_results = self._search_basic(query, params)
      File "searx/search/processors/online.py", line 118, in _search_basic
        self.engine.request(query, params)
      File "searx/engines/google_news.py", line 97, in request
        if lang_info['hl'] == 'en':
      KeyError: 'hl'

Closes: https://github.com/searxng/searxng/issues/154
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-18 11:34:11 +02:00
Markus Heiser 4417d2b922
Merge pull request #153 from searxng/dependabot/pip/master/sphinx-tabs-3.1.0
Bump sphinx-tabs from 3.0.0 to 3.1.0
2021-06-18 08:19:38 +00:00
dependabot[bot] 4b59d72af8
Bump sphinx-tabs from 3.0.0 to 3.1.0
Bumps [sphinx-tabs](https://github.com/executablebooks/sphinx-tabs) from 3.0.0 to 3.1.0.
- [Release notes](https://github.com/executablebooks/sphinx-tabs/releases)
- [Changelog](https://github.com/executablebooks/sphinx-tabs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/executablebooks/sphinx-tabs/compare/v3.0.0...v3.1.0)

---
updated-dependencies:
- dependency-name: sphinx-tabs
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-06-18 07:05:28 +00:00
Markus Heiser 7ef33c91e0
Merge pull request #136 from searxng/issue-115
/preferences: use simple layout for the oscar theme (Issue 115)
2021-06-17 13:38:50 +00:00
Alexandre Flament 881659ca9d [mod] oscar theme: /preferences : HTML detail order match visual tabs
First details about the general tab, then detail about UI tab, etc...
No functionnal change
2021-06-17 15:29:07 +02:00
Alexandre Flament f83b64270c [mod] oscar theme /preferences: reorganize the preferences
close #115
2021-06-17 15:29:07 +02:00
Alexandre Flament 92c68fe636
Merge pull request #151 from searxng/mod_oscar
make themes: 2 node_modules directories instead of 3
2021-06-17 10:46:50 +02:00
Markus Heiser ae677cb64b [enh] ./manage node.env - check build tools first
The node.env build environment require npm, ttfautohint and fontforge installed
in the OS.  These tools can be installed by::

    sudo -H ./utils/searx.sh install buildhost

If one of the tools is not installed, the script node.env stops with a
appropriate message.

BTW: We ignore CentOS-7 as developer & build platform

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-17 09:23:02 +02:00
Alexandre Flament a447c58218
Merge pull request #152 from return42/fix-typo
[fix] typo in a searx/settings.yml
2021-06-16 17:08:35 +02:00
Markus Heiser 96d4dbf74d [fix] typo in a searx/settings.yml
remove the erroneously leading underline

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-16 16:51:28 +02:00
Alexandre Flament 14ba56fd1a [mod] make node.clean: call the "clean" script from the packages.json
The dependencies and build process are described in the gruntfiles.js and packages.json files.
It makes sense to move the clean up to these files too.
2021-06-16 16:04:58 +02:00
Alexandre Flament c013256881 [mod] remove common grunt install
Before this commit, there are 3 node_modules directory:
* one in .
* two others in ./searx/statics/themes/*

This is no desirable:
* it declares the npm depdenencies in the shell script.
* dependabot can't updates theses dependencies.
* this is a not standard way to build a package (two different locations for the dependencies).

With this commit and the PR  #150 there is one unique node_modules directory per theme.
2021-06-16 15:39:32 +02:00
Alexandre Flament fdc367d58b
Merge pull request #149 from return42/doc-install-plg
[doc] admin manual - describe installation of external plugins
2021-06-16 14:44:56 +02:00
Alexandre Flament 85033f3d09 Static build 2021-06-16 14:30:46 +02:00