mirror of https://github.com/searxng/searxng.git
2499899554
Partial reverse engineering of the Google engines including a improved language and region handling based on the engine.traits_v1 data. When ever possible the implementations of the Google engines try to make use of the async REST APIs. The get_lang_info() has been generalized to a get_google_info() function / especially the region handling has been improved by adding the cr parameter. searx/data/engine_traits.json Add data type "traits_v1" generated by the fetch_traits() functions from: - Google (WEB), - Google images, - Google news, - Google scholar and - Google videos and remove data from obsolete data type "supported_languages". A traits.custom type that maps region codes to *supported_domains* is fetched from https://www.google.com/supported_domains searx/autocomplete.py: Reversed engineered autocomplete from Google WEB. Supports Google's languages and subdomains. The old API suggestqueries.google.com/complete has been replaced by the async REST API: https://{subdomain}/complete/search?{args} searx/engines/google.py Reverse engineering and extensive testing .. - fetch_traits(): Fetch languages & regions from Google properties. - always use the async REST API (formally known as 'use_mobile_ui') - use *supported_domains* from traits - improved the result list by fetching './/div[@data-content-feature]' and parsing the type of the various *content features* --> thumbnails are added searx/engines/google_images.py Reverse engineering and extensive testing .. - fetch_traits(): Fetch languages & regions from Google properties. - use *supported_domains* from traits - if exists, freshness_date is added to the result - issue 1864: result list has been improved a lot (due to the new cr parameter) searx/engines/google_news.py Reverse engineering and extensive testing .. - fetch_traits(): Fetch languages & regions from Google properties. *supported_domains* is not needed but a ceid list has been added. - different region handling compared to Google WEB - fixed for various languages & regions (due to the new ceid parameter) / avoid CONSENT page - Google News do no longer support time range - result list has been fixed: XPath of pub_date and pub_origin searx/engines/google_videos.py - fetch_traits(): Fetch languages & regions from Google properties. - use *supported_domains* from traits - add paging support - implement a async request ('asearch': 'arc' & 'async': 'use_ac:true,_fmt:html') - simplified code (thanks to '_fmt:html' request) - issue 1359: fixed xpath of video length data searx/engines/google_scholar.py - fetch_traits(): Fetch languages & regions from Google properties. - use *supported_domains* from traits - request(): include patents & citations - response(): fixed CAPTCHA detection (Scholar has its own CATCHA manager) - hardening XPath to iterate over results - fixed XPath of pub_type (has been change from gs_ct1 to gs_cgt2 class) - issue 1769 fixed: new request implementation is no longer incompatible Signed-off-by: Markus Heiser <markus.heiser@darmarit.de> |
||
|---|---|---|
| .devcontainer | ||
| .github | ||
| .vscode | ||
| dockerfiles | ||
| docs | ||
| examples | ||
| searx | ||
| searxng_extra | ||
| src/brand | ||
| tests | ||
| utils | ||
| .coveragerc | ||
| .dir-locals.el | ||
| .dockerignore | ||
| .editorconfig | ||
| .gitattributes | ||
| .gitignore | ||
| .nvmrc | ||
| .pylintrc | ||
| .weblate | ||
| .yamllint.yml | ||
| AUTHORS.rst | ||
| CHANGELOG.rst | ||
| CONTRIBUTING.md | ||
| Dockerfile | ||
| LICENSE | ||
| Makefile | ||
| PULL_REQUEST_TEMPLATE.md | ||
| README.rst | ||
| SECURITY.md | ||
| babel.cfg | ||
| manage | ||
| package.json | ||
| pyrightconfig-ci.json | ||
| pyrightconfig.json | ||
| requirements-dev.txt | ||
| requirements.txt | ||
| setup.py | ||
README.rst
Privacy-respecting, hackable metasearch engine
Searx.space lists ready-to-use running instances.
A user, admin and developer handbook is available on the homepage.
Contact
Ask questions or just chat about SearXNG on
- IRC
#searxng on libera.chat which is bridged to Matrix.
- Matrix
Differences to searx
SearXNG is a fork of searx, with notable changes:
User experience
- Reworked (and still simple) theme:
- Usable on desktop, tablet and mobile.
- Light and dark versions (available in the preferences).
- Right-to-left language support.
- Screenshots
- The translations are up to date, you can contribute on Weblate
- The preferences page has been updated:
- Browse which engines are reliable or not.
- Engines are grouped inside each tab.
- Each engine has a description.
- Thanks to the anonymous metrics, it is easier to report malfunctioning engines, so they get fixed quicker
- Turn off metrics on the server if you don't want them recorded.
- Administrators can block and/or replace the URLs in the search results
Setup
- No need for Morty to proxy images, even on a public instance.
- No need for Filtron to block bots, as there is now a built-in limiter.
- A well maintained Docker image, now also built for ARM64 and ARM/v7 architectures. (Alternatively there are up to date installation scripts.)
Contributing
- Readable debug log.
- Contributing is easier, thanks to the Development Quickstart guide.
- A lot of code cleanup and bugfixes.
- Up to date list dependencies.
Translations
Help translate SearXNG at Weblate
Codespaces
You can contribute from your browser using GitHub Codespaces:
- Fork the repository
- Click on the
<> Codegreen button - Click on the
Codespacestab instead ofLocal - Click on
Create codespace on master - VSCode is going to start in the browser
- Wait for
git pull && make installto appears and then to disapear - You have 120 hours per month (see also your list of existing Codespaces)
- You can start SearXNG using
make runin the terminal or by pressingCtrl+Shift+B.
Donations
Support the SearXNG project by clicking the donation page: https://docs.searxng.org/donate.html
Thank you :)