zaclys/searxng
1
0
Fork 1
mirror of https://github.com/searxng/searxng synced 2024-01-01 19:24:07 +01:00
Code Issues Projects Releases Packages Wiki Activity

[mod] improve 'Autodetect search language' plugin

Browse source 
- Add documentation to the plugin
- Harmonize FastText language model with SearXNG's language model

Reosurces::

    import fasttext                                    # --> +10 MB
    fasttext.load_model(str(data_dir / 'lid.176.ftz')) # --> +4MB

Suggested-by: @dalf

- To speed up and simplify the deployment use fasttext-wheel instead of fasttext
- Building numpy on the Alpine Linux of docker-images takes ages --> install
  py3-numpy from Alpines package manager (apk)
- Alpine Linux on docker-images (musl libc) do not support fasttext-wheel (gnu
  libc) --> patch Dockerfile and build from fastetxt:

     sed -i s/fasttext-wheel/fasttext/ requirements.txt

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
This commit is contained in:
Markus Heiser 2022-12-10 16:20:09 +01:00
parent 9925a20950
commit ed901ab18e
5 changed files with 119 additions and 9 deletions
Download patch file Download diff file Expand all files Collapse all files

2
requirements.txt
View file

@ -16,4 +16,4 @@ setproctitle==1.3.2
redis==4.4.0
markdown-it-py==2.1.0
typing_extensions==4.4.0
fasttext==0.9.2
fasttext-wheel==0.9.2