Commit Graph

119 Commits

Author SHA1 Message Date
Alexandre Flament
58a6c045c8 [mod] replace references request.request_data (GET or POST parameters) by request.form (based on merge of POST and GET parameters) 2016-11-02 14:52:22 +01:00
dalf
fbb080f358 Change plugin API :
- pre_search(request, search)
- post_search(request, search)
- on_result(request, search, result)

with
- request is the Flask request
- search a searx.Search instance
- result a searx result as usual
2016-11-02 14:22:16 +01:00
dalf
67e11c42b9 Clean up the architecture
Purposes :
- isolate the plugins calls
- distinction between parsing the web request and running the search (Search class). To be able to test code easily, to run searx code outside a web server, to filter the search query parameters with plugins more easily, etc...

Details :
- request.request_data contains request.form or request.args (initialize inside pre_request() function)
- Query class is renamed RawTextQuery
- SearchQuery class defines all search parameters
- get_search_query_from_webapp create a SearchQuery instance (basically the previous Search.__init__ code)
- Search class and SearchWithPlugins class takes a SearchQuery instance as class constructor parameter
- SearchWithPlugins class inherites from Search class, and run plugins
- A dedicated function search_with_plugins executes plugins to have a well define locals() (which is used by the plugins code).
- All plugins code is executed inside the try...except block (webapp.py, index function)
- advanced_search HTTP parameter value stays in webapp.py (it is only part of UI)
- multiple calls to result_container.get_ordered_results() doesn't compute the order multiple time (note : this method was call only once before)
- paging value is stored in the result_container class (compute in the extend method)
- test about engine.suspend_end_time is done during search method call (instead of __init__)
- check that the format parameter value is one of these : html, rss, json, rss (before the html value was assumed but some text formatting wasn't not done)
2016-11-02 14:22:16 +01:00
Adam Tauber
55abf07a4f [fix] thread safe threaded_requests() function 2016-09-06 00:36:33 +02:00
Adam Tauber
ffbee4bb82 [fix] collect garbage after searches to reduce memory footprint 2016-07-31 23:39:58 +02:00
Noemi Vanyi
16a8641649 advanced-search: panel is visible when checked before 2016-07-25 23:30:48 +02:00
Noemi Vanyi
93c0c49e9a add time range search with yahoo 2016-07-25 23:19:46 +02:00
Noemi Vanyi
5867e97966 more follow ups in new preferences 2016-04-09 19:49:59 +02:00
Noemi Vanyi
8c4db08443 rename blocked engines to diabled engines 2016-04-09 19:49:59 +02:00
Noemi Vanyi
fe691a0988 new preferences handling
Preferences class was introduced in order to handle user preferences. Right now
it parses cookies and the form in preferences. Also it can retrieve settings
based on the name of the setting.

ATTENTION
Please note that engine preferences are handled differently from now on. So it
introduces incompatible changes. Every user who has saved preferences should reset and
save his/her settings again.

This change was needed, because everytime a default disabled engine was
added saved user preferences would broke. Now engine setting tracking is
fixed.
2016-04-09 01:08:44 +02:00
Adam Tauber
10947536aa [enh] suspend engines after error
The duration is based on the number of continuous errors, but maximized in one minute
2016-02-20 00:21:56 +01:00
Adam Tauber
786fe49fab Merge pull request #438 from Cqoicebordel/error_page_format
Don't throw away the request if pageno is wrong
2015-12-01 15:50:47 +01:00
Adam Tauber
b6c3cb0bdd [enh][mod] result handling refactor
Several changes has been made:
 - Parallel result merge
 - Scoring algorithm slightly changed (see result_score())
 - Proper Thread locking on global data manipulation
2015-10-03 17:26:07 +02:00
Cqoicebordel
ed1daa32ea Don't throw away the request if pageno is wrong
Fix #424 but maybe only partially
2015-09-18 20:44:04 +02:00
Adam Tauber
37c3ace309 [fix] add missing scheme to duplicated results too ++ revert gigablasts handling 2015-09-11 18:33:06 +02:00
Adam Tauber
f094188780 [fix] display categories of the selected engines 2015-09-01 15:21:17 +02:00
Adam Tauber
8c4d9b79d5 [fix] engine selection from url 2015-09-01 15:14:02 +02:00
Adam Tauber
42ac2c5b56 [fix] check empty engine language attribute - fixes subtitleseeker 2015-08-25 22:02:18 +02:00
Thomas Pointhuber
5bffa9ca33 [fix] rewrite scheme to http if there is no one, FIX #390 2015-08-24 11:18:58 +02:00
Adam Tauber
7c9f931baf [fix] type error 2015-08-02 20:41:44 +02:00
Adam Tauber
43cd8e0c41 [enh] default settings option to safe_search - #396 2015-08-02 20:32:32 +02:00
Cqoicebordel
00668dd0a1 New feature : language per search engine ! 2015-06-03 17:16:12 +02:00
Adam Tauber
b09aaf126e [mod] allow whitespaces in category/engine list 2015-05-19 11:16:57 +02:00
Adam Tauber
b6de23f540 [enh] cleaner category/engine selection from GET/POST param - closes #289 2015-05-12 08:21:39 +02:00
Adam Tauber
d1b12f9016 [mod] search return value 2015-03-08 23:30:31 +01:00
Adam Tauber
bf09daa31d [enh] log redirects 2015-02-13 13:02:37 +01:00
dalf
5029cb4787 [fix] close #197 for all engines 2015-02-13 11:17:00 +01:00
Thomas Pointhuber
e7de9674b1 [enh] add safesearch functionality
2 = strict
1 = moderate
0 = none
2015-02-08 21:53:37 +01:00
Adam Tauber
1de781a143 [enh] category specific engine deactivation - closes #205 2015-02-03 18:37:38 +01:00
Adam Tauber
b19e681cce [enh] default disabled engines - closes #109 2015-01-31 23:11:45 +01:00
dalf
d07cfd9089 [enh] use one single http connection pool : improve response time. close #100 2015-01-21 11:33:16 +01:00
Thomas Pointhuber
5bfaabeb6b [enh] add nojs support into oscar-template 2015-01-15 15:48:50 +01:00
Adam Tauber
b47b8cc2c6 Merge pull request #166 from Cqoicebordel/non-exclusif-bang
Add non exclusif bang
2015-01-12 14:14:40 +01:00
Adam Tauber
23b4a53248 [enh] engine exception traceback 2015-01-09 04:30:55 +01:00
Adam Tauber
299a80a1eb [enh] using the logger 2015-01-09 04:13:05 +01:00
Cqoicebordel
96c4d52eef Add non exclusif bang
Allow to perform a search while adding an engine (or a category) without adding it "officially" to the request.
'?' is used to add an engine without modifying anything else to the request. For example, you can perform a search in the 'general' category, and if you add '?tw' the result from Twitter will be added to the originals results.
2015-01-03 02:31:23 +01:00
Cqoicebordel
5d977056f7 Flake8 and Twitter corrections
Lots of Flake8 corrections
Maybe we should change the rule to allow lines of 120 chars. It seems more usable.

Big twitter correction : now it outputs the words in right order...
2014-12-29 21:31:04 +01:00
Adam Tauber
1c969186bf [mod] better search request exception handling 2014-12-19 20:01:01 +01:00
Adam Tauber
8c05c00762 [fix] handle empty search requests 2014-12-19 19:40:40 +01:00
Adam Tauber
2b7f3e28e7 [mod] display timeoutted engine name 2014-12-19 13:59:41 +01:00
Adam Tauber
789e46f1c8 [fix] timeout and response parsing order 2014-12-18 12:07:20 +01:00
Adam Tauber
77c3a27f56 [mod] strict timeout handling 2014-12-18 10:11:56 +01:00
Thomas Pointhuber
70fc1002db [fix] fix kickass engine
thanks @Cqoicebordel in #144:
https://github.com/asciimoo/searx/pull/144#issuecomment-67036903
2014-12-15 19:37:58 +01:00
Adam Tauber
d93d703703 [fix] use queues to answers/suggestions/infoboxes 2014-12-14 02:15:04 +01:00
Adam Tauber
a5324d9d82 [fix] proper search timeout handling 2014-12-14 01:18:01 +01:00
Adam Tauber
8e6ae188b8 [mod] search thread names 2014-12-08 23:55:11 +01:00
Adam Tauber
d959cb1c05 [enh] gevent/grequests changed to the built-in threading lib 2014-12-05 19:24:11 +01:00
Thomas Pointhuber
5a7f85ecee [fix] #121, HTTP 500 Error if res["content"] == None 2014-11-01 17:32:26 +01:00
Adam Tauber
5740cfbf1c [fix] pep8 part II. 2014-10-19 12:41:04 +02:00
dalf
599c1137bb [fix] the bang was included in the search string 2014-10-11 12:46:12 +02:00
dalf
829c758970 Merge remote-tracking branch 'upstream/master' 2014-10-01 22:25:11 +02:00
Dalf
0a71525ab6 [enh] add infoboxes and answers (clean up) 2014-10-01 22:17:35 +02:00
Thomas Pointhuber
62d1a70c84 using Query class for parsing of search query 2014-10-01 17:57:53 +02:00
Dalf
6bfd566353 [enh] add infoboxes and answers 2014-09-28 16:51:41 +02:00
Dalf
e39d9fe542 update comment 2014-09-22 23:39:21 +02:00
Dalf
6b058962e1 [fix] when two results are merged, really use the content with more text 2014-09-22 22:55:51 +02:00
Adam Tauber
bd2db71fa6 Merge branch 'comments' of https://github.com/pointhi/searx
Conflicts:
	searx/search.py
2014-09-22 21:40:40 +02:00
Dalf
5d1610d87a [enh] stick results from the same category and template 2014-09-14 14:39:22 +02:00
Thomas Pointhuber
22da73b8bb little update update comments in search.py 2014-09-13 18:44:11 +02:00
Thomas Pointhuber
c9bab0e833 add comments to search.py
* add comments
* add licence-header
2014-09-13 18:25:25 +02:00
Adam Tauber
b0ba367a1a [enh][mod] search refactor 2014-07-07 13:59:27 +02:00
asciimoo
c1d7d30b8e [mod] len() removed from conditions 2014-02-11 13:13:51 +01:00
asciimoo
9ede9b955d [fix] language query string comparison 2014-02-09 11:35:52 +01:00
asciimoo
7c031a3ea6 [enh] better search language selection from query 2014-02-09 11:12:58 +01:00
asciimoo
d11a873d40 [enh] chainable query modifiers 2014-02-09 01:23:01 +01:00
asciimoo
a641971a6f [enh] language selection from query 2014-02-09 01:22:30 +01:00
asciimoo
7f45e8a562 [mod] query parsing refactor vol.1 2014-02-09 01:07:18 +01:00
asciimoo
9eb31c0186 [fix] unicode and pep8 fixes 2014-02-07 02:45:12 +01:00
asciimoo
0540ea9ee2 [enh][mod] code refactor ++ search engine filtering 2014-02-07 01:19:07 +01:00