Bump pygments from 2.11.2 to 2.12.0
After each update of the pygments version, we (or github action) should [1]
1. update less files:
./manage pyenv.cmd searxng_extra/update/update_pygments.py
update: static/themes/oscar/src/generated/pygments-logicodev.less
update: static/themes/oscar/src/generated/pygments-pointhi.less
update: static/themes/simple/src/generated/pygments.less
2. rebuild the static files of each themes:
make static.build.commit
[1] https://github.com/searxng/searxng/pull/2#issuecomment-826958666
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
After each update of the pygments version, we (or github action) should [1]
1. update less files:
./manage pyenv.cmd searxng_extra/update/update_pygments.py
update: static/themes/oscar/src/generated/pygments-logicodev.less
update: static/themes/oscar/src/generated/pygments-pointhi.less
update: static/themes/simple/src/generated/pygments.less
2. rebuild the static files of each themes:
make static.build.commit
[1] https://github.com/searxng/searxng/pull/2#issuecomment-826958666
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Just in case if content is None, the original code will skip extract_text(), and
just append the None value to 'content'. So just add allow_none=True, and this
will return None without raising a ValueError in extract_text().
In PR #1071 the language catalog of dailymotion has been cleaned up, before
there had been over 7000 "languages" in the catalog.
As a side effect of this clean-up the language & region catalog in SearXNG has
been reduced [1].
This patch reduce the ``min_engines_per_lang`` from 13 to 12 to get the missed
languages back in language & region catalog of SearXNG.
[1] 3bb62823ec (diff-f3f00db0f87f95b882624a192e0aac21525638af0b18c9514e765fcf1991678d)
Requested-by: @tiekoetter in a Matrix chat
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
- fix the issue of fetching more the 7000 *languages*
- improve the request function and filter by language & country
- implement time_range_support & safesearch
- add more fields to the response from dailymotion (allow_embed, length)
- better clean up of HTML tags in the 'content' field.
This is more or less a complete rework based on the '/videos' API from [1].
This patch cleans up the language list in SearXNG that has been polluted by the
ISO-639-3 2 and 3 letter codes from dailymotion languages which have never been
used.
[1] https://developers.dailymotion.com/tools/
Closes: https://github.com/searxng/searxng/issues/1065
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
There are several reasons why we should prefer markdown-it-py over mistletoe:
- Get identical rendering results in SearXNG's `/info` pages and the SearXNG's
project documentation which is build by Sphinx-doc.
In the Sphinx-doc we use the MyST parser to render Markdown and the MyST
parser itself is built on top of the markdown-it-py package.
- markdown-it-py has a typographer that supports *replacements*
and *smartquotes* (e.g. em-dash, copyright, ellipsis, ...) [1]
- markdown-it-py is much more flexible compared to mistletoe [2]
- markdown-it-py is the fastest CommonMark compliant parser in python [3]
[1] https://markdown-it-py.readthedocs.io/en/latest/using.html#typographic-components
[2] https://markdown-it-py.readthedocs.io/en/latest/plugins.html
[3] https://markdown-it-py.readthedocs.io/en/latest/other.html#performance
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Markus Heiser