zaclys
/
searxng
mirror of https://github.com/searxng/searxng.git
1
0
Fork 1
Code Issues Packages Projects Releases Wiki Activity
SearXNG is a free internet metasearch engine which aggregates results from various search services and databases. Users are neither tracked nor profiled.
4,527 Commits 5 Branches 17 Tags 71 MiB
Python 73.9%
Shell 10.6%
HTML 6.2%
Less 5.1%
JavaScript 2.8%
Other 1.3%
Go to file
Markus Heiser 9328c66e93 [fix] google news - send CONSENT Cookie to not be redirected 
In the EU there exists a "General Data Protection Regulation" [1] aka GDPR (BTW:
very user friendly!) which requires consent to tracking.  To get the consent
from the user, google-news requests are redirected to confirm and get a CONSENT
Cookie from https://consent.google.de/s?continue=...

This patch adds a CONSENT Cookie to the google-news request to avoid
redirection.

The behavior of the CONTENTS cookies over all google engines seems similar but
the pattern is not yet fully clear to me, here are some random samples from my
analysis ..

Using common google search from different domains::

    google.com:        CONSENT=YES+cb.{{date}}-14-p0.de+FX+816
    google.de:         CONSENT=YES+cb.{{date}}-14-p0.de+FX+333
    google.fr:         CONSENT=YES+srp.gws-{{date}}-0-RC2.fr+FX+826

When searching about videos (google-videos)::

    google.es:         CONSENT=YES+srp.gws-{{date}}-0-RC2.es+FX+076
    google.de:         CONSENT=YES+srp.gws-{{date}}-0-RC2.de+FX+171

Google news has only one domain for all languages::

    news.google.com:   CONSENT=YES+cb.{{date}}-14-p0.de+FX+816

Using google-scholar search from different domains::

    scholar.google.de: CONSENT=YES+cb.{{date}}-14-p0.de+FX+333
    scholar.google.fr: does not use such a cookie / did not ask the user
    scholar.google.es: does not use such a cookie / did not ask the user

Interim summary:

  Pattern is unclear and I won't apply the CONSENT cookie to all google engines.
  More experience is need before we generalize the CONSENT cookies over all
  google engines.

Related:

- e9a6ab401 [fix] youtube - send CONSENT Cookie to not be redirected
- https://github.com/benbusby/whoogle-search/issues/311
- https://github.com/benbusby/whoogle-search/issues/243

[1] https://en.wikipedia.org/wiki/General_Data_Protection_Regulation
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
 		2021-06-18 13:21:20 +02:00
.github [mod] simple theme: remove src/less/ion.less 2021-06-16 14:30:09 +02:00
dockerfiles [mod] docker-entrypoint.sh - add script to make test.sh (shellcheck) 2021-05-17 12:02:24 +02:00
docs [doc] admin manual - describe installation of external plugins 2021-06-15 18:00:35 +02:00
examples Fix whitespaces 2016-07-11 18:52:37 +07:00
searx [fix] google news - send CONSENT Cookie to not be redirected 2021-06-18 13:21:20 +02:00
searx_extra [pylint] searx_extra/update/update_osm_keys_tags.py 2021-06-09 18:08:23 +02:00
tests [mod] oscar theme /preferences: reorganize the preferences 2021-06-17 15:29:07 +02:00
utils [enh] ./manage node.env - check build tools first 2021-06-17 09:23:02 +02:00
.config.sh [filtron.sh] make filtron rule file configurable 2021-04-28 12:48:49 +02:00
.coveragerc [mod] use github actions instead of travis 2020-11-17 15:09:06 +01:00
.dir-locals.el [emacs] clean up .dir-locals.el file 2021-06-11 16:50:53 +02:00
.dockerignore [enh] update documentation about docker 2020-07-22 14:58:04 +02:00
.gitattributes [enh] add simple theme (WIP) 2017-08-06 16:04:21 +02:00
.gitignore [enh] Add onions category with Ahmia, Not Evil and Torch 2020-10-25 17:59:05 -07:00
.pylintrc [mod] pylint: add extension-pkg-whitelist=lxml.etree 2020-11-02 15:55:19 +01:00
.yamllint.yml [enh] add test.yamllint - lint yaml files 2021-06-05 17:41:24 +02:00
AUTHORS.rst [brand] searxng -- fix links to issue tracker & WEB-GUI 2021-04-25 14:25:08 +02:00
CHANGELOG.rst [brand] searxng -- fix links to issue tracker & WEB-GUI 2021-04-25 14:25:08 +02:00
CONTRIBUTING.md [brand] searxng -- fix links to issue tracker & WEB-GUI 2021-04-25 14:25:08 +02:00
Dockerfile [brand] Dockerfile default INSTANCE_NAME=searxng 2021-05-17 12:02:24 +02:00
LICENSE [fix] full AGPLv3+ license according to #382 2015-07-04 18:23:54 +02:00
Makefile [data.all] add searx_extra/update/update_osm_keys_tags.py 2021-06-09 18:08:23 +02:00
PULL_REQUEST_TEMPLATE.md Add PR template and contribution guidelines 2020-07-10 17:10:02 +02:00
README.rst [brand] README.rst fix img.shields.io and searxng.github.io URLs 2021-04-26 15:47:56 +02:00
babel.cfg [enh] babel.cfg added 2014-01-22 00:15:42 +01:00
manage [enh] ./manage node.env - check build tools first 2021-06-17 09:23:02 +02:00
requirements-dev.txt Bump sphinx-tabs from 3.0.0 to 3.1.0 2021-06-18 07:05:28 +00:00
requirements.txt Bump certifi from 2020.12.05 to 2021.5.30 2021-06-04 06:05:25 +00:00
setup.py Add searx_extra package 2021-03-04 11:59:14 +01:00

README.rst

searx install searx homepage searx wiki AGPL License Issues commits

Privacy-respecting, hackable metasearch engine

If you are looking for running instances, ready to use, then visit searx.space.

Otherwise jump to the user, admin and developer handbooks you will find on our homepage.