YaCy Release 1.92

Great progress, more than four hundred bug fixes and enhancements from
YaCy developers luccioman and reger.

Major Changes   
Jump to: Bugfixes / Other Changes

CommitDescription
Tue Dec 20 14:52:33 CET 2016
by luccioman
Enabled displaying results after 14th page for local search queries.

Fixes issue #90 for local queries only: Stealth mode, Portal mode or
Intranet mode. 
For P2p mode, the issue would probably be difficult to solve with
reasonable performance. This is still to dig.

Also switched some InterreputedException catch log messages to warn
level as this is normal behavior when shutting down a peer.

Fixed yacysearch buttons navbar behavior to deal correctly with total
results count or offset over 1000. Also improved the buttons navbar to
be able to navigate over 10th page for local queries. 
Changed Files: htroot/js/yacysearch.js, htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearchitem.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/query/SearchEventCache.java
Fri Dec 16 00:49:19 CET 2016
by reger
upd to ImageIO 3.3.1
Changed Files: .classpath, build.xml, lib/common-image-3.3.1.jar, lib/common-io-3.3.1.jar, lib/common-lang-3.3.1.jar, lib/imageio-bmp-3.3.1.jar, lib/imageio-core-3.3.1.jar, lib/imageio-metadata-3.3.1.jar, lib/imageio-tiff-3.3.1.jar, pom.xml
Mon Dec 05 10:57:37 CET 2016
by luccioman
Reduced locations vocabulary memory footprint.

Reduced this vocabulary memory usage :
 - by using only one map term2entries instead of two maps having the
same key set
 - by generating the location object links on the fly using the
GeoLocation data instead of storing many duplicates of string prefix
"http://www.openstreetmap.org/?lat="
 
Measurements with VisualVM and GeoNames 0 enabled (cities with a
population > 1000) :
 - AutotaggingLibrary retained size :
  - initial : 309 718 763 bytes
  - after refactoring : 159 224 641 bytes
Changed Files: source/net/yacy/cora/lod/vocabulary/LocationTaggingEntry.java, source/net/yacy/cora/lod/vocabulary/SynonymTaggingEntry.java, source/net/yacy/cora/lod/vocabulary/Tagging.java, source/net/yacy/cora/lod/vocabulary/TaggingEntry.java, source/net/yacy/cora/lod/vocabulary/TaggingEntryWithObjectLink.java
Wed Nov 23 18:13:05 CET 2016
by luccioman
Added control over Robots.txt active threads maximum number.

When starting a crawl from a file containing thousands of links,
configuration setting "crawler.MaxActiveThreads" is effective to prevent
saturating the system with too many outgoing HTTP connections threads
launched by the crawler.
But robots.txt was not affected by this setting and was indefinitely
increasing the number of concurrently loading threads until most ot the
connections timed out.

To improve performance control, added a pool of threads for Robots.txt,
consistently used in its ensureExist() and massCrawlCheck() methods.
The Robots.txt threads pool max size can now be configured in the
/PerformanceQueus_p.html page, or with the new
"robots.txt.MaxActiveThreads" setting, initialized with the same default
value as the crawler.
Changed Files: defaults/yacy.init, htroot/CrawlCheck_p.java, htroot/PerformanceQueues_p.java, source/net/yacy/crawler/robots/RobotsTxt.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java, test/java/net/yacy/crawler/HostBalancerTest.java
Mon Nov 21 16:52:53 CET 2016
by reger
make a YearNavigator availabel, useable as SearchEvent.naviator plugin.
It can take any Date field of the index and displays a list of year strings
in reverse order by the year (not the score/count).
To allow to define the index field to use, the fieldname (and title can be 
appended to the navi's name "year" e.g. year:load_date_dt:LoadDate
It works also with dates_in_content_dts field (from the graphical date
navigator). Here the query parameter from: to: are used on selection as
Query modifier (for other dates currently no query parameter available, so
selection won't work to filter search results).
Not included in the UI Searchpage layout config so far (for experiment with
it manual change to conf needed).
Changed Files: defaults/yacy.init, htroot/yacysearchtrailer.java, source/net/yacy/cora/sorting/ConcurrentScoreMap.java, source/net/yacy/search/navigator/LanguageNavigator.java, source/net/yacy/search/navigator/NameSpaceNavigator.java, source/net/yacy/search/navigator/Navigator.java, source/net/yacy/search/navigator/StringNavigator.java, source/net/yacy/search/navigator/YearNavigator.java, source/net/yacy/search/query/SearchEvent.java
Thu Nov 17 14:17:08 CET 2016
by luccioman
Upgraded Bootstrap to 3.3.7 and upgraded its related js dependencies.

Upgraded the following JavaScript libraries dependencies :
 - bootstrap-switch to 3.3.2
 - html5shiv to 3.7.3 and switched to minified version
 - typeahead to 0.10.5
 - jQuery to 1.12.4

Removed unused bootstratp-rtl.css and bootstrap-rtl.min.css.

Tested non regressions on the following systems :
 - Debian Jessie : 
  - Firefox 45.4.0

 - MS Windows 10 :
  - Chrome 54.0.2840.99 
  - Firefox 50.0
  - Edge
  - Emulated IE 11, 10 and 9
Changed Files: htroot/env/bootstrap/css/bootstrap-switch.css, htroot/env/bootstrap/css/bootstrap-switch.min.css, htroot/env/bootstrap/css/bootstrap-theme.css, htroot/env/bootstrap/css/bootstrap-theme.css.map, htroot/env/bootstrap/css/bootstrap-theme.min.css, htroot/env/bootstrap/css/bootstrap-theme.min.css.map, htroot/env/bootstrap/css/bootstrap.css, htroot/env/bootstrap/css/bootstrap.css.map, htroot/env/bootstrap/css/bootstrap.min.css, htroot/env/bootstrap/css/bootstrap.min.css.map, htroot/env/bootstrap/js/bootstrap-switch.js, htroot/env/bootstrap/js/bootstrap-switch.min.js, htroot/env/bootstrap/js/bootstrap.js, htroot/env/bootstrap/js/bootstrap.min.js, htroot/env/bootstrap/js/html5shiv.js, htroot/env/bootstrap/js/html5shiv.min.js, htroot/env/bootstrap/js/jquery.js, htroot/env/bootstrap/js/jquery.min.js, htroot/env/bootstrap/js/jquery.min.map, htroot/env/bootstrap/js/typeahead.jquery.js, htroot/env/bootstrap/js/typeahead.jquery.min.js, htroot/env/templates/metas.template, htroot/jslicense.html, htroot/test.html
Mon Nov 14 01:37:16 CET 2016
by reger
Make ServletRequest implement the standardized HttpServletRequest interface,
to make all readily available information from the original ServletRequest
available to YaCy servlets (without converting data to internal structures).
The implementation of the common interface allows easier integration of
YaCy servlets with the servlet standard (e.g. shared login service with
the servlet container etc.)
Changed Files: htroot/AccessTracker_p.java, htroot/Blog.java, htroot/ConfigBasic.java, htroot/Crawler_p.java, htroot/Network.java, htroot/PerformanceQueues_p.java, htroot/User.java, htroot/ViewFavicon.java, htroot/ViewImage.java, htroot/YaCySearchPluginFF.java, htroot/autoconfig.java, htroot/cytag.java, htroot/yacy/hello.java, htroot/yacy/list.java, htroot/yacy/message.java, htroot/yacy/profile.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacysearch.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/protocol/RequestHeader.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/repository/BlacklistHelper.java, source/net/yacy/server/http/HTTPDemon.java, test/java/net/yacy/cora/protocol/RequestHeaderTest.java
Sun Nov 13 01:39:14 CET 2016
by reger
increase use of header const for custom "EXT" header
Changed Files: htroot/CacheResource_p.java, htroot/Tables_p.java, htroot/ViewFavicon.java, htroot/ViewImage.java, htroot/api/linkstructure.java, htroot/api/snapshot.java, htroot/api/table_p.java, htroot/osm.java, htroot/suggest.java, htroot/yacy/seedlist.java
Sat Nov 12 10:51:54 CET 2016
by luccioman
Converted more URLs to pure relative ones.

Easier YaCy peer configuration behind a reverse proxy subfolder : no
need for the reverse proxy to rewrite HTML links or URLs in css files.

Tested on Debian Jessie with an apache2 reverse proxy.

See related mantis issues http://mantis.tokeek.de/view.php?id=106 and
http://mantis.tokeek.de/view.php?id=701
Changed Files: htroot/api/citation.java, htroot/api/table_p.html, htroot/env/base.css, htroot/env/yacy-ymarks.css, htroot/portalsearch/yacy-portalsearch.css, htroot/yacy/ui/css/yacyui-portalsearch.css, source/net/yacy/cora/federate/solr/responsewriter/GrepHTMLResponseWriter.java
Wed Nov 09 02:40:33 CET 2016
by luccioman
Switched even more URLs to pure relative ones.

Thus a YaCy peer can run behind a reverse proxy subfolder without need
for the reverse proxy to rewrite HTML links (a CPU costly operation).

Tested on Debian Jessie with an apache2 reverse proxy.

See related mantis issues http://mantis.tokeek.de/view.php?id=106 and
http://mantis.tokeek.de/view.php?id=701
Changed Files: defaults/yacy.init, htroot/Status.html, htroot/env/templates/embeddedheader.template, htroot/env/templates/header.template, htroot/env/templates/metas.template, htroot/env/templates/simpleheader.template, source/net/yacy/cora/federate/solr/responsewriter/HTMLResponseWriter.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/server/servletProperties.java, test/java/net/yacy/http/servlets/YaCyDefaultServletTest.java
Tue Nov 08 03:05:51 CET 2016
by luccioman
Switched more URLs to relative ones when possible.

This permits an easier and more flexible reverse proxy configuration.
Some related mantis issues : http://mantis.tokeek.de/view.php?id=106 and
http://mantis.tokeek.de/view.php?id=701
Changed Files: defaults/yacy.init, htroot/ConfigSearchPage_p.html, htroot/Crawler_p.html, htroot/HostBrowser.html, htroot/Network.html, htroot/Status.html, htroot/ViewFile.html, htroot/index.html, htroot/js/Crawler.js, htroot/js/IndexCreate.js, htroot/js/highslide/highslide.js, htroot/js/hypertree.js, htroot/jslicense.html, htroot/rssTerminal.html, htroot/terminal_p.html, htroot/test.html, htroot/yacy/ui/yacyui-admin.html, htroot/yacysearch.html, htroot/yacysearchtrailer.html, source/net/yacy/search/query/QueryParams.java
Fri Nov 04 11:21:20 CET 2016
by luccioman
Generate HTML relative (to each peer) links from hosted WikiCode.

When WikiCode inserted in a peer hosted Blog, Wiki, Messages or Profile
contains relative links (images or any content, hosted in DATA/HTDOCS),
it is more reliable to keep these links relative, especially when the
peer is behind any kind of reverse Proxy.
Changed Files: htroot/Blog.java, htroot/Blog.rss, htroot/BlogComments.java, htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Messages_p.rss, htroot/ViewProfile.java, htroot/Wiki.java, htroot/mediawiki_p.java, source/net/yacy/data/wiki/WikiCode.java, source/net/yacy/server/serverObjects.java
Thu Nov 03 02:33:36 CET 2016
by luccioman
Fixed /News.html and /Wiki.html pages in Search Portal mode (issue #87).

Also fixes theses pages rendering when the peer is not online.

Re-factored code in common with /opensearchdescription.xml and
ConfigPortal.html.
Changed Files: htroot/ConfigPortal.html, htroot/ConfigPortal.java, htroot/News.java, htroot/News.rss, htroot/Wiki.java, htroot/opensearchdescription.java, htroot/opensearchdescription.xml, source/net/yacy/http/servlets/YaCyDefaultServlet.java
Mon Oct 31 02:17:43 CET 2016
by reger
Add search navigator interface to allow for additional navigators (plugins)
Prepared the first basic navigators (for authors and collections) for the
list of SearchEvent.navigatorPlugins and adjusted servlet to use these.
- this allows to configure display order of these navigators (by ordering config string)
- eventually allows for additional and/or custom navigators using any
available index field without need for changing servlets
- the Collection navigation has been adjusted to exclude the internal, 
default robot_*  and dht collections from displaying
- rwi results are now also checked for navigatior by the refactored navi's

So far no config options were added to customize or add navigators (may
come later if route of upcoming modularization/plugin system is defined).
Changed Files: htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java, source/net/yacy/search/navigator/Navigator.java, source/net/yacy/search/navigator/RestrictedStringNavigator.java, source/net/yacy/search/navigator/StringNavigator.java, source/net/yacy/search/query/SearchEvent.java
Sun Oct 23 19:40:02 CEST 2016
by reger
Refacture rwi reference word position and word distance calculation
used for rwi ranking.
Main changes:  
- introduce a  posintext() to access the stored value. This reduces also mem alloc of position array for WordReferenceRow (index access)
- use the positions() array for joined references on multi-word queries if needed (otherwise allow positions() to be null
- adjust assignments and the min() max() and distance() calculation accordingly
Changed Files: htroot/IndexControlRWIs_p.java, source/net/yacy/kelondro/data/citation/CitationReference.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceRow.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceVars.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/rwi/AbstractReference.java, source/net/yacy/kelondro/rwi/Reference.java, source/net/yacy/peers/graphics/WebStructureGraph.java, source/net/yacy/search/ranking/ReferenceOrder.java, test/java/net/yacy/kelondro/data/word/WordReferenceVarsTest.java, test/java/net/yacy/search/index/SegmentTest.java
Sat Oct 22 17:17:21 CEST 2016
by luccioman
Customized name for Threads still using the default "Thread-n" pattern.

This makes threads monitoring easier to read.
Changed Files: source/net/yacy/cora/federate/FederateSearchManager.java, source/net/yacy/cora/federate/yacy/Peers.java, source/net/yacy/crawler/HostBalancer.java, source/net/yacy/crawler/data/Transactions.java, source/net/yacy/crawler/retrieval/RSSLoader.java, source/net/yacy/crawler/retrieval/SitemapImporter.java, source/net/yacy/crawler/robots/RobotsTxt.java, source/net/yacy/data/DidYouMean.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/gui/Tray.java, source/net/yacy/gui/framework/Switchboard.java, source/net/yacy/http/AbstractRemoteHandler.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/util/ConsoleInterface.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/SeedDB.java, source/net/yacy/peers/graphics/OSMTile.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Shutdown.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/ErrorCacheFiller.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/query/AccessTracker.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/query/SecondarySearchSuperviser.java, source/net/yacy/search/ranking/BlockRank.java, source/net/yacy/search/ranking/ReferenceOrder.java, source/net/yacy/utils/loaderThreads.java, source/net/yacy/yacy.java
Fri Oct 21 13:03:31 CEST 2016
by luccioman
Advanced Crawl from local file : better processing of large files.

Applied strategy : when there is no restriction on domains or
sub-path(s), stack anchor links once discovered by the content scraper
instead of waiting the complete parsing of the file. 

This makes it possible to handle a crawling start file with thousands of
links in a reasonable amount of time.

Performance limitation : even if the crawl start faster with a large
file, the content of the parsed file still is fully loaded in memory. 
Changed Files: htroot/Crawler_p.java, source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/crawler/CrawlStarterFromSraper.java, source/net/yacy/crawler/FileCrawlStarterTask.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/ContentScraperListener.java, source/net/yacy/document/parser/html/ScraperListener.java
Sat Oct 15 05:23:18 CEST 2016
by reger
upd to Jetty 9.2.19
Changed Files: .classpath, build.xml, lib/jetty-9.2.19.v20160908.License, lib/jetty-client-9.2.19.v20160908.jar, lib/jetty-continuation-9.2.19.v20160908.jar, lib/jetty-deploy-9.2.19.v20160908.jar, lib/jetty-http-9.2.19.v20160908.jar, lib/jetty-io-9.2.19.v20160908.jar, lib/jetty-jmx-9.2.19.v20160908.jar, lib/jetty-proxy-9.2.19.v20160908.jar, lib/jetty-security-9.2.19.v20160908.jar, lib/jetty-server-9.2.19.v20160908.jar, lib/jetty-servlet-9.2.19.v20160908.jar, lib/jetty-servlets-9.2.19.v20160908.jar, lib/jetty-util-9.2.19.v20160908.jar, lib/jetty-webapp-9.2.19.v20160908.jar, lib/jetty-xml-9.2.19.v20160908.jar, pom.xml
Fri Oct 14 12:00:39 CEST 2016
by luccioman
Merge pull request #39 from luccioman/master

Favicon retrieval and image preview enhancements.
More details on mantis 629 (http://mantis.tokeek.de/view.php?id=629)
Changed Files: defaults/solr.collection.schema, htroot/ViewFavicon.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/api/getpageinfo.java, htroot/api/getpageinfo.json, htroot/api/getpageinfo.xml, htroot/api/getpageinfo_p.java, htroot/api/getpageinfo_p.xml, htroot/portalsearch/yacy-portalsearch.js, htroot/yacy/ui/js/yacyui-portalsearch.js, htroot/yacysearchitem.html, htroot/yacysearchitem.java, htroot/yacysearchitem.json, source/net/yacy/cora/federate/solr/responsewriter/YJsonResponseWriter.java, source/net/yacy/data/InvalidURLLicenceException.java, source/net/yacy/data/URLLicense.java, source/net/yacy/document/Document.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/IconEntry.java, source/net/yacy/document/parser/html/IconLinkRelations.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/http/servlets/SolrSelectServlet.java, source/net/yacy/http/servlets/TemplateMissingParameterException.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/CollectionSchema.java, source/net/yacy/visualization/ImageViewer.java, test/ImageViewerPerfTest.java, test/ImageViewerTest.java, test/java/net/yacy/document/parser/html/ContentScraperTest.java, test/java/net/yacy/document/parser/html/IconEntryTest.java, test/java/net/yacy/kelondro/data/meta/URIMetadataNodeTest.java, test/java/yacysearchitemTest.java, test/net/yacy/data/URLLicenseConcurrentTest.java
Thu Sep 22 16:08:33 CEST 2016
by luccioman
Accessibility : add a customizable alternative text to YaCy log

Applied W3C recommendations :
https://www.w3.org/TR/html51/semantics-embedded-content.html#a-link-or-button-containing-nothing-but-an-image
and
https://www.w3.org/TR/html51/semantics-embedded-content.html#logos-insignia-flags-or-emblems
Changed Files: defaults/yacy.init, htroot/ConfigPortal.html, htroot/ConfigPortal.java, htroot/ConfigSearchPage_p.html, htroot/ConfigSearchPage_p.java, htroot/env/templates/embeddedheader.template, htroot/env/templates/header.template, htroot/env/templates/simpleheader.template, htroot/index.html, htroot/index.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/search/SwitchboardConstants.java
Wed Sep 07 09:28:37 CEST 2016
by luccioman
Merged master into postprocessing branch
Changed Files: addon/YaCy.app/Contents/Info.plist, addon/YaCy.app/Contents/MacOS/startYACYMacOS.sh, build.xml, docker/Dockerfile, docker/Dockerfile.alpine, docker/Readme.md, docker/docker-cloud.yml, htroot/Autocrawl_p.java, htroot/News.html, htroot/TransNews_p.html, htroot/TransNews_p.java, htroot/WebStructurePicture_p.java, htroot/env/templates/submenuComputation.template, htroot/yacy/search.java, htroot/yacy/seedlist.java, locales/fr.lng, locales/master.lng.xlf, source/net/yacy/cora/federate/solr/responsewriter/OpensearchResponseWriter.java, source/net/yacy/cora/federate/solr/responsewriter/YJsonResponseWriter.java, source/net/yacy/cora/lod/vocabulary/Tagging.java, source/net/yacy/cora/sorting/ConcurrentScoreMap.java, source/net/yacy/cora/util/Html2Image.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/crawler/retrieval/Response.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/Tokenizer.java, source/net/yacy/document/WordTokenizer.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/http/servlets/SolrSelectServlet.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/rwi/AbstractReference.java, source/net/yacy/peers/Network.java, source/net/yacy/peers/NewsPool.java, source/net/yacy/peers/PeerActions.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/ranking/ReferenceOrder.java, source/net/yacy/server/serverClassLoader.java, source/net/yacy/server/serverObjects.java, source/net/yacy/yacy.java, startYACY.sh, test/java/net/yacy/cora/sorting/ConcurrentScoreMapTest.java, test/java/net/yacy/data/wiki/WikiCodeTest.java, test/java/net/yacy/document/TokenizerTest.java, test/java/net/yacy/document/WordTokenizerTest.java
Tue Sep 06 18:46:24 CEST 2016
by Michael Peter Christen
removed unused imports
Changed Files: htroot/Autocrawl_p.java, htroot/WebStructurePicture_p.java, htroot/yacy/search.java, source/net/yacy/cora/lod/vocabulary/Tagging.java, source/net/yacy/cora/util/Html2Image.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/crawler/retrieval/Response.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/peers/Network.java, source/net/yacy/repository/Blacklist.java, test/java/net/yacy/data/wiki/WikiCodeTest.java
Tue Sep 06 00:06:02 CEST 2016
by luccioman
Merge remote-tracking branch 'origin/master' into docker

Conflicts solved:
	docker/Readme.md
Changed Files: .classpath, .env, Heroku.md, Procfile, README.md, addon/YaCy.app/Contents/Info.plist, addon/YaCy.app/Contents/MacOS/startYACYMacOS.sh, app.json, build.xml, docker/Readme.md, htroot/ConfigBasic.html, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, htroot/ConfigPortal.html, htroot/ConfigPortal.java, htroot/CookieTest_p.html, htroot/CookieTest_p.java, htroot/DictionaryLoader_p.html, htroot/DictionaryLoader_p.java, htroot/Network.html, htroot/NetworkHistory.java, htroot/News.html, htroot/PerformanceMemory_p.java, htroot/QuickCrawlLink_p.java, htroot/Supporter.java, htroot/TransNews_p.html, htroot/TransNews_p.java, htroot/Translator_p.html, htroot/Translator_p.java, htroot/YaCySearchPluginFF.java, htroot/api/share.html, htroot/api/share.java, htroot/autoconfig.java, htroot/compare_yacy.java, htroot/env/templates/submenuComputation.template, htroot/opensearchdescription.java, htroot/opensearchdescription.xml, htroot/yacy/hello.java, htroot/yacy/message.java, htroot/yacy/search.java, lib/jetty-9.2.18.v20160721.License, lib/jetty-client-9.2.18.v20160721.jar, lib/jetty-continuation-9.2.18.v20160721.jar, lib/jetty-deploy-9.2.18.v20160721.jar, lib/jetty-http-9.2.18.v20160721.jar, lib/jetty-io-9.2.18.v20160721.jar, lib/jetty-jmx-9.2.18.v20160721.jar, lib/jetty-proxy-9.2.18.v20160721.jar, lib/jetty-security-9.2.18.v20160721.jar, lib/jetty-server-9.2.18.v20160721.jar, lib/jetty-servlet-9.2.18.v20160721.jar, lib/jetty-servlets-9.2.18.v20160721.jar, lib/jetty-util-9.2.18.v20160721.jar, lib/jetty-webapp-9.2.18.v20160721.jar, lib/jetty-xml-9.2.18.v20160721.jar, libbuild/GitRevMavenTask/.gitignore, libbuild/J7Zip-modified/.gitignore, libbuild/WebCat-swf/.gitignore, locales/cn.lng, locales/fr.lng, locales/gr.lng, locales/it.lng, locales/ja.lng, locales/master.lng.xlf, locales/ru.lng, locales/uk.lng, pom.xml, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/sorting/ConcurrentScoreMap.java, source/net/yacy/data/Translator.java, source/net/yacy/document/AbstractParser.java, source/net/yacy/document/Document.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/parser/genericParser.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/gui/framework/Browser.java, source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/http/servlets/YaCyProxyServlet.java, source/net/yacy/peers/NewsPool.java, source/net/yacy/peers/PeerActions.java, source/net/yacy/search/ResourceObserver.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/server/http/HTTPDFileHandler.java, source/net/yacy/server/http/HTTPDProxyHandler.java, source/net/yacy/server/http/HTTPDemon.java, source/net/yacy/server/http/TemplateEngine.java, source/net/yacy/server/serverClassLoader.java, source/net/yacy/server/serverSwitch.java, source/net/yacy/utils/translation/TranslationManager.java, source/net/yacy/utils/translation/TranslatorXliff.java, source/net/yacy/yacy.java, startYACY.sh, test/java/net/yacy/cora/sorting/ConcurrentScoreMapTest.java, test/java/net/yacy/document/parser/xlsParserTest.java
Mon Sep 05 22:42:17 CEST 2016
by luccioman
Merge remote-tracking branch 'origin/master' into dist_macOS
Changed Files: htroot/News.html, htroot/TransNews_p.html, htroot/TransNews_p.java, htroot/env/templates/submenuComputation.template, htroot/yacy/search.java, locales/fr.lng, locales/master.lng.xlf, source/net/yacy/cora/sorting/ConcurrentScoreMap.java, source/net/yacy/peers/NewsPool.java, source/net/yacy/peers/PeerActions.java, source/net/yacy/search/query/SearchEvent.java, test/java/net/yacy/cora/sorting/ConcurrentScoreMapTest.java
Fri Aug 26 19:21:47 CEST 2016
by Sergey Stepanov
yacy_search_server from yacy/master

merge
Changed Files: .classpath, .env, .gitignore, .project, .travis.yml, Heroku.md, Procfile, README.md, addon/YaCy.app/Contents/Info.plist, addon/YaCy.app/Contents/Resources/en.lproj/Localizable.strings, addon/synonyms/thesaurus_ru_yacy, app.json, assembly.xml, build.properties, build.xml, debian/control, defaults/heuristicopensearch.conf, defaults/httpd.mime, defaults/solr.collection.schema, defaults/solr/schema.xml, defaults/solr/solrconfig.xml, defaults/yacy.init, defaults/yacy.network.freeworld.unit, defaults/yacy.network.zeronet.unit, defaults/yacy.networks, docker/Dockerfile, docker/Dockerfile.alpine, docker/Readme.md, docker/docker-cloud.yml, examples/SimpleSearchClient/pom.xml, examples/SimpleSearchClient/src/YaCySearchClient.java, htroot/AugmentedBrowsing_p.html, htroot/Autocrawl_p.html, htroot/Autocrawl_p.java, htroot/BlacklistCleaner_p.html, htroot/BlacklistImpExp_p.html, htroot/BlacklistImpExp_p.java, htroot/BlacklistTest_p.html, htroot/Blacklist_p.html, htroot/Blog.html, htroot/BlogComments.html, htroot/Bookmarks.java, htroot/CacheResource_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigBasic.html, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.html, htroot/ConfigLanguage_p.java, htroot/ConfigNetwork_p.html, htroot/ConfigPortal.html, htroot/ConfigPortal.java, htroot/ConfigRobotsTxt_p.html, htroot/ConfigRobotsTxt_p.java, htroot/ConfigSearchBox.java, htroot/ConfigUpdate_p.html, htroot/ContentAnalysis_p.html, htroot/ContentIntegrationPHPBB3_p.html, htroot/CookieTest_p.html, htroot/CookieTest_p.java, htroot/CrawlMonitorRemoteStart.java, htroot/Crawler_p.java, htroot/DictionaryLoader_p.html, htroot/IndexControlRWIs_p.java, htroot/IndexCreateParserErrors_p.html, htroot/IndexExport_p.html, htroot/IndexExport_p.java, htroot/IndexFederated_p.html, htroot/IndexImportMediawiki_p.html, htroot/IndexImportOAIPMHList_p.html, htroot/IndexImportOAIPMH_p.html, htroot/IndexSchema_p.html, htroot/IndexShare_p.java, htroot/Load_MediawikiWiki.html, htroot/Load_MediawikiWiki.java, htroot/Load_PHPBB3.html, htroot/Load_PHPBB3.java, htroot/Load_RSS_p.html, htroot/MessageSend_p.html, htroot/MessageSend_p.java, htroot/Messages_p.html, htroot/Network.html, htroot/Network.java, htroot/NetworkHistory.java, htroot/PerformanceMemory_p.html, htroot/PerformanceMemory_p.java, htroot/PerformanceQueues_p.html, htroot/PerformanceQueues_p.java, htroot/Performance_p.html, htroot/ProxyIndexingMonitor_p.html, htroot/QuickCrawlLink_p.java, htroot/RankingRWI_p.html, htroot/RankingSolr_p.html, htroot/RankingSolr_p.java, htroot/RegexTest.html, htroot/ServerScannerList.html, htroot/SettingsAck_p.html, htroot/SettingsAck_p.java, htroot/Settings_Crawler.inc, htroot/Settings_MessageForwarding.inc, htroot/Settings_Proxy.inc, htroot/Settings_ProxyAccess.inc, htroot/Settings_Seed.inc, htroot/Settings_Seed_UploadFile.inc, htroot/Settings_Seed_UploadFtp.inc, htroot/Settings_Seed_UploadScp.inc, htroot/Settings_ServerAccess.inc, htroot/Settings_p.java, htroot/Status.html, htroot/Status.java, htroot/Status_p.inc, htroot/Steering.html, htroot/Surftips.html, htroot/Tables_p.html, htroot/Threaddump_p.html, htroot/Translator_p.html, htroot/Translator_p.java, htroot/User.html, htroot/ViewFile.html, htroot/ViewFile.java, htroot/ViewImage.java, htroot/ViewProfile.html, htroot/Vocabulary_p.html, htroot/Vocabulary_p.java, htroot/WatchWebStructure_p.html, htroot/WebStructurePicture_p.java, htroot/YaCySearchPluginFF.java, htroot/api/citation.java, htroot/api/push_p.java, htroot/api/schema.xml, htroot/api/share.html, htroot/api/share.java, htroot/api/snapshot.java, htroot/api/ymarks/import_ymark.java, htroot/autoconfig.java, htroot/compare_yacy.html, htroot/compare_yacy.java, htroot/env/base.css, htroot/env/bootstrap/css/bootstrap-theme.css, htroot/env/bootstrap/css/bootstrap-theme.css.map, htroot/env/bootstrap/css/bootstrap-theme.min.css, htroot/env/bootstrap/css/bootstrap.css, htroot/env/bootstrap/css/bootstrap.css.map, htroot/env/bootstrap/css/bootstrap.min.css, htroot/env/bootstrap/fonts/glyphicons-halflings-regular.woff2, htroot/env/bootstrap/js/bootstrap.js, htroot/env/bootstrap/js/bootstrap.min.js, htroot/env/oldie.css, htroot/env/templates/header.template, htroot/env/templates/metas.template, htroot/env/templates/simpleheader.template, htroot/env/templates/submenuConfig.template, htroot/env/templates/submenuIndexCreate.template, htroot/index.html, htroot/js/Bookmarks.js, htroot/js/Crawler.js, htroot/js/IndexCreate.js, htroot/js/WatchWebStructure.js, htroot/js/ajax.js, htroot/js/highslide/highslide.js, htroot/js/html.js, htroot/js/hypertree.js, htroot/js/query.js, htroot/js/raphael-min.js, htroot/js/rss2.js, htroot/js/sorttable.js, htroot/js/xml.js, htroot/js/yacy-ymarks-bookmark-actions.js, htroot/js/yacy-ymarks-tag-actions.js, htroot/js/yacy-ymarks.js, htroot/js/yacyinteractive.js, htroot/js/yacysearch.js, htroot/jslicense.html, htroot/jslicense.java, htroot/opensearchdescription.java, htroot/opensearchdescription.xml, htroot/proxymsg/urlproxyheader.java, htroot/rssTerminal.html, htroot/sharedBlacklist_p.html, htroot/sharedBlacklist_p.java, htroot/yacy/message.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/ui/css/base.css, htroot/yacy/ui/css/widget.css, htroot/yacyinteractive.html, htroot/yacysearch.java, htroot/yacysearchitem.html, htroot/yacysearchitem.java, htroot/yacysearchtrailer.json, installYaCyWindowsService.bat, langdetect/af, langdetect/ar, langdetect/bg, langdetect/bn, langdetect/cs, langdetect/da, langdetect/de, langdetect/el, langdetect/en, langdetect/es, langdetect/et, langdetect/fa, langdetect/fi, langdetect/fr, langdetect/gu, langdetect/he, langdetect/hi, langdetect/hr, langdetect/hu, langdetect/id, langdetect/it, langdetect/ja, langdetect/kn, langdetect/ko, langdetect/lt, langdetect/lv, langdetect/mk, langdetect/ml, langdetect/mr, langdetect/ne, langdetect/nl, langdetect/no, langdetect/pa, langdetect/pl, langdetect/pt, langdetect/ro, langdetect/ru, langdetect/sk, langdetect/sl, langdetect/so, langdetect/sq, langdetect/sv, langdetect/sw, langdetect/ta, langdetect/te, langdetect/th, langdetect/tl, langdetect/tr, langdetect/uk, langdetect/ur, langdetect/vi, langdetect/zh-cn, langdetect/zh-tw, lib/J7Zip-modified.jar, lib/common-image-3.2.1.jar, lib/common-io-3.2.1.jar, lib/common-lang-3.2.1.jar, lib/commons-compress-1.12.License, lib/commons-compress-1.12.jar, lib/commons-fileupload-1.3.2.License, lib/commons-fileupload-1.3.2.jar, lib/commons-io-2.5.License, lib/commons-io-2.5.jar, lib/fontbox-2.0.2.License, lib/fontbox-2.0.2.jar, lib/httpclient-4.5.2.jar, lib/httpcore-4.4.5.License, lib/httpcore-4.4.5.jar, lib/httpmime-4.5.2.jar, lib/icu4j-57_1.jar, lib/imageio-bmp-3.2.1.jar, lib/imageio-core-3.2.1.jar, lib/imageio-metadata-3.2.1.jar, lib/imageio-tiff-3.2.1.jar, lib/jcl-over-slf4j-1.7.21.jar, lib/jetty-9.2.18.v20160721.License, lib/jetty-client-9.2.18.v20160721.jar, lib/jetty-continuation-9.2.18.v20160721.jar, lib/jetty-deploy-9.2.18.v20160721.jar, lib/jetty-http-9.2.18.v20160721.jar, lib/jetty-io-9.2.18.v20160721.jar, lib/jetty-jmx-9.2.18.v20160721.jar, lib/jetty-proxy-9.2.18.v20160721.jar, lib/jetty-security-9.2.18.v20160721.jar, lib/jetty-server-9.2.18.v20160721.jar, lib/jetty-servlet-9.2.18.v20160721.jar, lib/jetty-servlets-9.2.18.v20160721.jar, lib/jetty-util-9.2.18.v20160721.jar, lib/jetty-webapp-9.2.18.v20160721.jar, lib/jetty-xml-9.2.18.v20160721.jar, lib/jsonic-1.2.0.jar, lib/jsoup-1.9.2.jar, lib/langdetect.jar, lib/langdetect.jar.License, lib/log4j-over-slf4j-1.7.21.jar, lib/lucene-analyzers-common-5.5.2.jar, lib/lucene-analyzers-phonetic-5.5.2.jar, lib/lucene-backward-codecs-5.5.2.jar, lib/lucene-classification-5.5.2.jar, lib/lucene-codecs-5.5.2.jar, lib/lucene-core-5.5.2.jar, lib/lucene-facet-5.5.2.jar, lib/lucene-grouping-5.5.2.jar, lib/lucene-highlighter-5.5.2.jar, lib/lucene-join-5.5.2.jar, lib/lucene-memory-5.5.2.jar, lib/lucene-misc-5.5.2.jar, lib/lucene-queries-5.5.2.jar, lib/lucene-queryparser-5.5.2.jar, lib/lucene-spatial-5.5.2.jar, lib/lucene-suggest-5.5.2.jar, lib/metadata-extractor-2.9.1.License, lib/metadata-extractor-2.9.1.jar, lib/pdfbox-2.0.2.License, lib/pdfbox-2.0.2.jar, lib/poi-3.14-20160307.jar, lib/poi-3.14.License, lib/poi-scratchpad-3.14-20160307.jar, lib/slf4j-api-1.7.21.jar, lib/slf4j-jdk14-1.7.21.jar, lib/solr-core-5.5.2.jar, lib/solr-solrj-5.5.2.jar, lib/spatial4j-0.5.jar, lib/webcat-swf-0.1.jar, lib/webcat-swf.License, lib/weupnp-0.1.4.jar, lib/xliff-core-1.2-1.1.jar, libbuild/GitRevMavenTask/.gitignore, libbuild/GitRevMavenTask/pom.xml, libbuild/GitRevMavenTask/src/GitRevMavenTask.java, libbuild/GitRevTask/GitRevTask.java, libbuild/J7Zip-modified/.gitignore, libbuild/J7Zip-modified/pom.xml, libbuild/J7Zip-modified/src/Common/BoolVector.java, libbuild/J7Zip-modified/src/Common/CRC.java, libbuild/J7Zip-modified/src/Common/IntVector.java, libbuild/J7Zip-modified/src/Common/LimitedSequentialInStream.java, libbuild/J7Zip-modified/src/Common/LockedInStream.java, libbuild/J7Zip-modified/src/Common/LockedSequentialInStreamImp.java, libbuild/J7Zip-modified/src/Common/LongVector.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/BindInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/BindPair.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/CoderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/CoderMixer2.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/CoderMixer2ST.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/CoderStreamsInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/FilterCoder.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/OutStreamWithCRC.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/STCoderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/IArchiveExtractCallback.java, libbuild/J7Zip-modified/src/SevenZip/Archive/IInArchive.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/AltCoderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/ArchiveDB.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/ArchiveDatabase.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/ArchiveDatabaseEx.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/BindInfoEx.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/CoderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/ExtractFolderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/FileItem.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Folder.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/FolderOutStream.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Handler.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Header.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/InArchiveInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/InByte2.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/InStream.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/MethodID.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/StreamSwitch.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZipEntry.java, libbuild/J7Zip-modified/src/SevenZip/ArchiveExtractCallback.java, libbuild/J7Zip-modified/src/SevenZip/CRC.java, libbuild/J7Zip-modified/src/SevenZip/Common/InBuffer.java, libbuild/J7Zip-modified/src/SevenZip/Common/LocalCompressProgressInfo.java, libbuild/J7Zip-modified/src/SevenZip/Common/LocalProgress.java, libbuild/J7Zip-modified/src/SevenZip/Common/StreamUtils.java, libbuild/J7Zip-modified/src/SevenZip/Compression/Branch/BCJ2_x86_Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/Branch/BCJ_x86_Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/Copy/Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZ/BinTree.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZ/InWindow.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZ/OutWindow.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZMA/Base.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZMA/Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZMA/Encoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/BitDecoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/BitModel.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/BitTreeDecoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/BitTreeEncoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/Encoder.java, libbuild/J7Zip-modified/src/SevenZip/HRESULT.java, libbuild/J7Zip-modified/src/SevenZip/ICodeProgress.java, libbuild/J7Zip-modified/src/SevenZip/ICompressCoder.java, libbuild/J7Zip-modified/src/SevenZip/ICompressCoder2.java, libbuild/J7Zip-modified/src/SevenZip/ICompressFilter.java, libbuild/J7Zip-modified/src/SevenZip/ICompressGetInStreamProcessedSize.java, libbuild/J7Zip-modified/src/SevenZip/ICompressProgressInfo.java, libbuild/J7Zip-modified/src/SevenZip/ICompressSetInStream.java, libbuild/J7Zip-modified/src/SevenZip/ICompressSetOutStream.java, libbuild/J7Zip-modified/src/SevenZip/ICompressSetOutStreamSize.java, libbuild/J7Zip-modified/src/SevenZip/IInStream.java, libbuild/J7Zip-modified/src/SevenZip/IProgress.java, libbuild/J7Zip-modified/src/SevenZip/J7zip.java, libbuild/J7Zip-modified/src/SevenZip/LzmaAlone.java, libbuild/J7Zip-modified/src/SevenZip/LzmaBench.java, libbuild/J7Zip-modified/src/SevenZip/MyRandomAccessFile.java, libbuild/WebCat-swf/.gitignore, libbuild/WebCat-swf/JavaSWF2-BSD.LICENSE.txt, libbuild/WebCat-swf/pom.xml, libbuild/WebCat-swf/readme.txt, libbuild/WebCat-swf/src/com/anotherbigidea/flash/SWFActionCodes.java, libbuild/WebCat-swf/src/com/anotherbigidea/flash/SWFConstants.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ActionParser.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ActionTextWriter.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ActionWriter.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Actions.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/AlphaColor.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/AlphaTransform.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Base64.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ButtonRecord.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ButtonRecord2.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Color.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ColorTransform.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ExportedSymbol.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/FillStyle.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Font.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/FontDefinition.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Frame.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ImportedSymbol.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/InStream.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Instance.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/LineStyle.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Matrix.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Movie.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/OutStream.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Placement.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Rect.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWF2HTML.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFActions.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFHeader.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFReader.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFShape.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFSpriteTagTypes.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFTagTypes.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFTagTypesImpl.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFTags.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFText.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFVectors.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFWriter.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SaxHandlerBase.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Shape.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SoundInfo.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Style.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Symbol.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/TagParser.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/TagWriter.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/TimeLine.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Transform.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/XMLWriter.java, libbuild/pom.xml, locales/cn.lng, locales/de.lng, locales/fr.lng, locales/gr.lng, locales/hi.lng, locales/it.lng, locales/ja.lng, locales/master.lng.xlf, locales/ru.lng, locales/sk.lng, locales/uk.lng, nbproject/project.xml, pom.xml, source/net/yacy/contentcontrol/SMWListSyncThread.java, source/net/yacy/cora/date/GenericFormatter.java, source/net/yacy/cora/document/WordCache.java, source/net/yacy/cora/document/encoding/UTF8.java, source/net/yacy/cora/document/id/AnchorURL.java, source/net/yacy/cora/document/id/DigestURL.java, source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/cora/federate/FederateSearchManager.java, source/net/yacy/cora/federate/opensearch/OpenSearchConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/instance/EmbeddedInstance.java, source/net/yacy/cora/federate/solr/responsewriter/GSAResponseWriter.java, source/net/yacy/cora/federate/solr/responsewriter/HTMLResponseWriter.java, source/net/yacy/cora/geo/GeonamesLocation.java, source/net/yacy/cora/geo/OpenGeoDBLocation.java, source/net/yacy/cora/lod/vocabulary/Tagging.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/storage/Files.java, source/net/yacy/cora/storage/KeyList.java, source/net/yacy/cora/util/HTTPInputStream.java, source/net/yacy/cora/util/Html2Image.java, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/crawler/HostBalancer.java, source/net/yacy/crawler/HostQueue.java, source/net/yacy/crawler/LegacyBalancer.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/crawler/data/Latency.java, source/net/yacy/crawler/data/ResultURLs.java, source/net/yacy/crawler/data/Transactions.java, source/net/yacy/crawler/retrieval/FileLoader.java, source/net/yacy/crawler/retrieval/HTTPLoader.java, source/net/yacy/crawler/retrieval/Response.java, source/net/yacy/crawler/retrieval/SMBLoader.java, source/net/yacy/data/BlogBoard.java, source/net/yacy/data/BookmarkHelper.java, source/net/yacy/data/MessageBoard.java, source/net/yacy/data/Translator.java, source/net/yacy/data/WorkTables.java, source/net/yacy/data/wiki/AbstractWikiParser.java, source/net/yacy/data/wiki/WikiCode.java, source/net/yacy/data/ymark/YMarkTables.java, source/net/yacy/document/AbstractParser.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Document.java, source/net/yacy/document/ImageParser.java, source/net/yacy/document/LibraryProvider.java, source/net/yacy/document/Parser.java, source/net/yacy/document/ProbabilisticClassifier.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/document/content/dao/PhpBB3Dao.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/language/Identificator.java, source/net/yacy/document/parser/apkParser.java, source/net/yacy/document/parser/audioTagParser.java, source/net/yacy/document/parser/augment/AugmentParser.java, source/net/yacy/document/parser/bzipParser.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/dwgParser.java, source/net/yacy/document/parser/genericParser.java, source/net/yacy/document/parser/gzipParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/ScraperInputStream.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/images/bmpParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/images/icoParser.java, source/net/yacy/document/parser/images/metadataImageParser.java, source/net/yacy/document/parser/images/svgParser.java, source/net/yacy/document/parser/linkScraperParser.java, source/net/yacy/document/parser/mmParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rdfParser.java, source/net/yacy/document/parser/rdfa/impl/RDFaParser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/sidAudioParser.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/gui/framework/Browser.java, source/net/yacy/http/AbstractRemoteHandler.java, source/net/yacy/http/Jetty9HttpServerImpl.java, source/net/yacy/http/ProxyCacheHandler.java, source/net/yacy/http/ProxyHandler.java, source/net/yacy/http/servlets/GSAsearchServlet.java, source/net/yacy/http/servlets/SolrSelectServlet.java, source/net/yacy/http/servlets/SolrServlet.java, source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/http/servlets/YaCyProxyServlet.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/kelondro/util/MemoryControl.java, source/net/yacy/migration.java, source/net/yacy/peers/Network.java, source/net/yacy/peers/PeerActions.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/RemoteSearch.java, source/net/yacy/peers/Seed.java, source/net/yacy/peers/SeedDB.java, source/net/yacy/peers/graphics/EncodedImage.java, source/net/yacy/peers/operation/yacyVersion.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/AutoSearch.java, source/net/yacy/search/ResourceObserver.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java, source/net/yacy/search/index/DocumentIndex.java, source/net/yacy/search/index/ErrorCache.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/query/AccessTracker.java, source/net/yacy/search/query/QueryGoal.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/query/SearchEventCache.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/server/http/ChunkedInputStream.java, source/net/yacy/server/http/HTTPDFileHandler.java, source/net/yacy/server/http/HTTPDProxyHandler.java, source/net/yacy/server/http/HTTPDemon.java, source/net/yacy/server/http/TemplateEngine.java, source/net/yacy/server/serverClassLoader.java, source/net/yacy/server/serverSwitch.java, source/net/yacy/utils/translation/CreateTranslationMasters.java, source/net/yacy/utils/translation/ListNonTranslatedFiles.java, source/net/yacy/utils/translation/TranslateAll.java, source/net/yacy/utils/translation/TranslateAllToOneLanguage.java, source/net/yacy/utils/translation/TranslatorXliff.java, source/net/yacy/visualization/ChartPlotter.java, source/net/yacy/visualization/RasterPlotter.java, source/net/yacy/yacy.java, startYACY.bat, startYACY.sh, startYACY_debug.bat, stopYACY.bat, test/ViewImagePerfTest.java, test/ViewImageTest.java, test/java/net/yacy/cora/document/id/DigestURLTest.java, test/java/net/yacy/cora/document/id/MultiProtocolURLTest.java, test/java/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnectorTest.java, test/java/net/yacy/cora/protocol/HeaderFrameworkTest.java, test/java/net/yacy/cora/protocol/ResponseHeaderTest.java, test/java/net/yacy/cora/util/Html2ImageTest.java, test/java/net/yacy/cora/util/NumberToolsTest.java, test/java/net/yacy/crawler/HostBalancerTest.java, test/java/net/yacy/crawler/HostQueueTest.java, test/java/net/yacy/data/TranslatorTest.java, test/java/net/yacy/data/wiki/WikiCodeTest.java, test/java/net/yacy/document/ParserTest.java, test/java/net/yacy/document/WordTokenizerTest.java, test/java/net/yacy/document/parser/htmlParserTest.java, test/java/net/yacy/document/parser/images/genericImageParserTest.java, test/java/net/yacy/document/parser/images/metadataImageParserTest.java, test/java/net/yacy/document/parser/pdfParserTest.java, test/java/net/yacy/document/parser/xlsParserTest.java, test/java/net/yacy/kelondro/io/RecordsTest.java, test/java/net/yacy/kelondro/util/MemoryControlTest.java, test/java/net/yacy/peers/operation/yacyVersionTest.java, test/java/net/yacy/repository/BlacklistTest.java, test/java/net/yacy/search/query/QueryGoalTest.java, test/java/net/yacy/search/snippet/TextSnippetTest.java, test/java/net/yacy/utils/translation/TranslatorXliffTest.java, test/viewImageTest/ViewImageTest.html
Fri Aug 26 11:57:41 CEST 2016
by Orbiter
Merge pull request #61 from luccioman/heroku_experiments

Deploy YaCy on Heroku
Changed Files: .env, Heroku.md, Procfile, README.md, app.json, docker/Readme.md, htroot/ConfigBasic.html, htroot/ConfigBasic.java, libbuild/GitRevMavenTask/.gitignore, libbuild/J7Zip-modified/.gitignore, libbuild/WebCat-swf/.gitignore, locales/fr.lng, locales/master.lng.xlf, pom.xml, source/net/yacy/server/serverSwitch.java
Tue Aug 23 00:04:24 CEST 2016
by reger
Introduce additional language setting "browser/Browser Language" for UI internationalization.
If language is set to "browser" the client/user browser language is used to choose from
available translation.
simply: one users browser speaks English -> YaCy responds in English, other users browser speaks French -> YaCy responds in French.

! To make a translation/language available you have to activate the language once ! 
(or manually use the utility class TranslateAll)
In ConfigBasic.html availabel translations are marked green on setting language=Browser
The client language is determined by http header Accept-Language (checked in DefaultServlet)

Changed Files: htroot/ConfigBasic.html, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.java, source/net/yacy/data/Translator.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/server/http/TemplateEngine.java, source/net/yacy/yacy.java
Sun Aug 21 19:34:44 CEST 2016
by reger
change unused servletProperties entry CONNECTION_PROP_CLIENT_REQUEST_HEADER to
use directly HttpServletRequest. This is used to get the http protocol version
in HTTPDProxyHandler.fulfillRequestFromWeb() for error response to client.
- adjust YaCyProxyServlet and UrlProxyServlet accordingly
- use more http_version constants in headerframework and httpdeamon
- equalize servlets (3) use of HeaderFramework.CONNECTION_PROP_HOST to HeaderFramework.HOST
Changed Files: htroot/QuickCrawlLink_p.java, htroot/YaCySearchPluginFF.java, htroot/autoconfig.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/http/servlets/YaCyProxyServlet.java, source/net/yacy/server/http/HTTPDProxyHandler.java, source/net/yacy/server/http/HTTPDemon.java
Sun Aug 14 21:25:48 CEST 2016
by reger
upd to Jetty 9.2.18
Changed Files: .classpath, build.xml, lib/jetty-9.2.18.v20160721.License, lib/jetty-client-9.2.18.v20160721.jar, lib/jetty-continuation-9.2.18.v20160721.jar, lib/jetty-deploy-9.2.18.v20160721.jar, lib/jetty-http-9.2.18.v20160721.jar, lib/jetty-io-9.2.18.v20160721.jar, lib/jetty-jmx-9.2.18.v20160721.jar, lib/jetty-proxy-9.2.18.v20160721.jar, lib/jetty-security-9.2.18.v20160721.jar, lib/jetty-server-9.2.18.v20160721.jar, lib/jetty-servlet-9.2.18.v20160721.jar, lib/jetty-servlets-9.2.18.v20160721.jar, lib/jetty-util-9.2.18.v20160721.jar, lib/jetty-webapp-9.2.18.v20160721.jar, lib/jetty-xml-9.2.18.v20160721.jar, pom.xml
Sun Aug 14 03:53:16 CEST 2016
by reger
eleminate dependency on file-extension in storeDocument but use supported mime-type
to also support handling of urls w/o corresponding file-extension.
For this refactor use of document.getParserObject() to alway return a Parser (for clean logic)
and define/move the scraperObject as local var of AbstractParser.
Adjust related calls to getParserObject (where actually a scraperObject is wanted).
Addionally skip appending url token to parsed text for dht metadata entries 
(by default returned as result by rwi index).
Changed Files: source/net/yacy/document/AbstractParser.java, source/net/yacy/document/Document.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/parser/genericParser.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/schema/CollectionConfiguration.java
Tue Aug 09 21:27:57 CEST 2016
by Andreas
Merge pull request #7 from yacy/master

Get my fork synced #7
Changed Files: .classpath, .gitignore, build.properties, build.xml, debian/control, defaults/solr/solrconfig.xml, defaults/yacy.init, defaults/yacy.network.freeworld.unit, docker/Dockerfile, docker/Dockerfile.alpine, docker/Readme.md, docker/docker-cloud.yml, htroot/Blacklist_p.html, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.html, htroot/ConfigLanguage_p.java, htroot/ConfigNetwork_p.html, htroot/ConfigRobotsTxt_p.java, htroot/ConfigSearchBox.java, htroot/CookieTest_p.java, htroot/CrawlMonitorRemoteStart.java, htroot/Crawler_p.java, htroot/IndexShare_p.java, htroot/Load_MediawikiWiki.java, htroot/Load_PHPBB3.java, htroot/MessageSend_p.html, htroot/MessageSend_p.java, htroot/Network.html, htroot/Network.java, htroot/RankingSolr_p.html, htroot/RankingSolr_p.java, htroot/Status.html, htroot/Status.java, htroot/Translator_p.html, htroot/Translator_p.java, htroot/ViewFile.html, htroot/ViewProfile.html, htroot/Vocabulary_p.java, htroot/api/citation.java, htroot/env/templates/header.template, htroot/env/templates/metas.template, htroot/env/templates/simpleheader.template, htroot/index.html, htroot/js/Bookmarks.js, htroot/js/Crawler.js, htroot/js/IndexCreate.js, htroot/js/WatchWebStructure.js, htroot/js/ajax.js, htroot/js/highslide/highslide.js, htroot/js/html.js, htroot/js/hypertree.js, htroot/js/query.js, htroot/js/raphael-min.js, htroot/js/rss2.js, htroot/js/sorttable.js, htroot/js/xml.js, htroot/js/yacy-ymarks-bookmark-actions.js, htroot/js/yacy-ymarks-tag-actions.js, htroot/js/yacy-ymarks.js, htroot/js/yacyinteractive.js, htroot/js/yacysearch.js, htroot/jslicense.html, htroot/jslicense.java, htroot/opensearchdescription.java, htroot/rssTerminal.html, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacysearch.java, lib/commons-compress-1.12.License, lib/commons-compress-1.12.jar, lib/commons-fileupload-1.3.2.License, lib/commons-fileupload-1.3.2.jar, lib/fontbox-2.0.2.License, lib/fontbox-2.0.2.jar, lib/httpcore-4.4.5.License, lib/httpcore-4.4.5.jar, lib/icu4j-57_1.jar, lib/jcl-over-slf4j-1.7.21.jar, lib/jetty-9.2.17.v20160517.License, lib/jetty-client-9.2.17.v20160517.jar, lib/jetty-continuation-9.2.17.v20160517.jar, lib/jetty-deploy-9.2.17.v20160517.jar, lib/jetty-http-9.2.17.v20160517.jar, lib/jetty-io-9.2.17.v20160517.jar, lib/jetty-jmx-9.2.17.v20160517.jar, lib/jetty-proxy-9.2.17.v20160517.jar, lib/jetty-security-9.2.17.v20160517.jar, lib/jetty-server-9.2.17.v20160517.jar, lib/jetty-servlet-9.2.17.v20160517.jar, lib/jetty-servlets-9.2.17.v20160517.jar, lib/jetty-util-9.2.17.v20160517.jar, lib/jetty-webapp-9.2.17.v20160517.jar, lib/jetty-xml-9.2.17.v20160517.jar, lib/jsoup-1.9.2.jar, lib/log4j-over-slf4j-1.7.21.jar, lib/lucene-analyzers-common-5.5.2.jar, lib/lucene-analyzers-phonetic-5.5.2.jar, lib/lucene-backward-codecs-5.5.2.jar, lib/lucene-classification-5.5.2.jar, lib/lucene-codecs-5.5.2.jar, lib/lucene-core-5.5.2.jar, lib/lucene-facet-5.5.2.jar, lib/lucene-grouping-5.5.2.jar, lib/lucene-highlighter-5.5.2.jar, lib/lucene-join-5.5.2.jar, lib/lucene-memory-5.5.2.jar, lib/lucene-misc-5.5.2.jar, lib/lucene-queries-5.5.2.jar, lib/lucene-queryparser-5.5.2.jar, lib/lucene-spatial-5.5.2.jar, lib/lucene-suggest-5.5.2.jar, lib/pdfbox-2.0.2.License, lib/pdfbox-2.0.2.jar, lib/slf4j-api-1.7.21.jar, lib/slf4j-jdk14-1.7.21.jar, lib/solr-core-5.5.2.jar, lib/solr-solrj-5.5.2.jar, libbuild/GitRevMavenTask/src/GitRevMavenTask.java, libbuild/GitRevTask/GitRevTask.java, locales/cn.lng, locales/de.lng, locales/fr.lng, locales/gr.lng, locales/hi.lng, locales/it.lng, locales/ja.lng, locales/master.lng.xlf, locales/ru.lng, locales/sk.lng, locales/uk.lng, pom.xml, readme.mediawiki, source/net/yacy/cora/date/GenericFormatter.java, source/net/yacy/cora/document/id/DigestURL.java, source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/cora/federate/solr/responsewriter/HTMLResponseWriter.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/util/Html2Image.java, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/crawler/HostBalancer.java, source/net/yacy/crawler/HostQueue.java, source/net/yacy/crawler/data/Transactions.java, source/net/yacy/crawler/retrieval/FileLoader.java, source/net/yacy/crawler/retrieval/SMBLoader.java, source/net/yacy/data/MessageBoard.java, source/net/yacy/data/Translator.java, source/net/yacy/data/WorkTables.java, source/net/yacy/data/wiki/WikiCode.java, source/net/yacy/document/Document.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/http/servlets/GSAsearchServlet.java, source/net/yacy/http/servlets/SolrSelectServlet.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/Seed.java, source/net/yacy/peers/SeedDB.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/search/AutoSearch.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/utils/translation/CreateTranslationMasters.java, source/net/yacy/utils/translation/ListNonTranslatedFiles.java, source/net/yacy/utils/translation/TranslateAll.java, source/net/yacy/utils/translation/TranslateAllToOneLanguage.java, source/net/yacy/utils/translation/TranslatorXliff.java, source/net/yacy/yacy.java, test/java/net/yacy/cora/document/id/DigestURLTest.java, test/java/net/yacy/cora/document/id/MultiProtocolURLTest.java, test/java/net/yacy/cora/util/Html2ImageTest.java, test/java/net/yacy/crawler/HostBalancerTest.java, test/java/net/yacy/crawler/HostQueueTest.java, test/java/net/yacy/data/TranslatorTest.java, test/java/net/yacy/data/wiki/WikiCodeTest.java, test/java/net/yacy/repository/BlacklistTest.java, test/java/net/yacy/utils/translation/TranslatorXliffTest.java
Thu Jul 28 02:29:40 CEST 2016
by luccioman
Merge remote-tracking branch 'origin/master' into heroku_experiments
Changed Files: .classpath, build.xml, defaults/solr/solrconfig.xml, defaults/yacy.init, htroot/IndexShare_p.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, lib/commons-compress-1.12.License, lib/commons-compress-1.12.jar, pom.xml, source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Fulltext.java
Wed Jul 20 00:37:31 CEST 2016
by luccioman
Merged changes from master.
Changed Files: locales/de.lng, locales/fr.lng, locales/master.lng.xlf, locales/sk.lng, source/net/yacy/cora/date/GenericFormatter.java, source/net/yacy/cora/document/id/DigestURL.java, source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/document/Document.java, test/java/net/yacy/cora/document/id/DigestURLTest.java, test/java/net/yacy/cora/document/id/MultiProtocolURLTest.java
Mon Jul 04 23:28:46 CEST 2016
by luccioman
Revert : Dockerfiles are now in docker/ folder
Changed Files: .classpath, build.xml, lib/lucene-analyzers-common-5.5.2.jar, lib/lucene-analyzers-phonetic-5.5.2.jar, lib/lucene-backward-codecs-5.5.2.jar, lib/lucene-classification-5.5.2.jar, lib/lucene-codecs-5.5.2.jar, lib/lucene-core-5.5.2.jar, lib/lucene-facet-5.5.2.jar, lib/lucene-grouping-5.5.2.jar, lib/lucene-highlighter-5.5.2.jar, lib/lucene-join-5.5.2.jar, lib/lucene-memory-5.5.2.jar, lib/lucene-misc-5.5.2.jar, lib/lucene-queries-5.5.2.jar, lib/lucene-queryparser-5.5.2.jar, lib/lucene-spatial-5.5.2.jar, lib/lucene-suggest-5.5.2.jar, lib/solr-core-5.5.2.jar, lib/solr-solrj-5.5.2.jar, pom.xml
Mon Jul 04 22:23:54 CEST 2016
by reger
upd to Solr 5.5.2
Changed Files: .classpath, build.xml, lib/lucene-analyzers-common-5.5.2.jar, lib/lucene-analyzers-phonetic-5.5.2.jar, lib/lucene-backward-codecs-5.5.2.jar, lib/lucene-classification-5.5.2.jar, lib/lucene-codecs-5.5.2.jar, lib/lucene-core-5.5.2.jar, lib/lucene-facet-5.5.2.jar, lib/lucene-grouping-5.5.2.jar, lib/lucene-highlighter-5.5.2.jar, lib/lucene-join-5.5.2.jar, lib/lucene-memory-5.5.2.jar, lib/lucene-misc-5.5.2.jar, lib/lucene-queries-5.5.2.jar, lib/lucene-queryparser-5.5.2.jar, lib/lucene-spatial-5.5.2.jar, lib/lucene-suggest-5.5.2.jar, lib/solr-core-5.5.2.jar, lib/solr-solrj-5.5.2.jar, pom.xml
Mon Jul 04 21:07:11 CEST 2016
by Orbiter
Merge pull request #56 from luccioman/LibreJS

 LibreJS compliance : YaCy JavaScript license information
Changed Files: htroot/ViewFile.html, htroot/env/templates/header.template, htroot/env/templates/metas.template, htroot/env/templates/simpleheader.template, htroot/js/Bookmarks.js, htroot/js/Crawler.js, htroot/js/IndexCreate.js, htroot/js/WatchWebStructure.js, htroot/js/ajax.js, htroot/js/highslide/highslide.js, htroot/js/html.js, htroot/js/hypertree.js, htroot/js/query.js, htroot/js/raphael-min.js, htroot/js/rss2.js, htroot/js/sorttable.js, htroot/js/xml.js, htroot/js/yacy-ymarks-bookmark-actions.js, htroot/js/yacy-ymarks-tag-actions.js, htroot/js/yacy-ymarks.js, htroot/js/yacyinteractive.js, htroot/js/yacysearch.js, htroot/jslicense.html, htroot/jslicense.java, source/net/yacy/cora/federate/solr/responsewriter/HTMLResponseWriter.java


Bugfixes   
Jump to: YaCy Release 1.92 top / Other Changes

CommitDescription
Wed Dec 21 08:19:09 CET 2016
by luccioman
Fixed title rendering having non ASCII chars in QuickCrawlLink_p.html.
Changed Files: htroot/QuickCrawlLink_p.java, source/net/yacy/cora/federate/solr/responsewriter/HTMLResponseWriter.java
Tue Dec 20 08:27:17 CET 2016
by luccioman
Improved indentation for easier debugging steps.
Changed Files: source/net/yacy/cora/sorting/WeakPriorityBlockingQueue.java
Sat Dec 03 01:13:47 CET 2016
by reger
fix unresolved pattern in ConfiLanguage drop down list
Changed Files: htroot/ConfigLanguage_p.java
Tue Nov 29 23:32:20 CET 2016
by reger
fix ext2mime to return given default on input=null
Changed Files: source/net/yacy/cora/document/analysis/Classification.java
Tue Nov 29 02:11:44 CET 2016
by luccioman
Fixed a JS undefined error case, occurring when search field is empty.
Changed Files: htroot/yacysearchtrailer.html
Fri Nov 25 11:07:50 CET 2016
by luccioman
Fixed a NullPointerException case.
Changed Files: source/net/yacy/cora/federate/solr/responsewriter/YJsonResponseWriter.java
Fri Nov 25 04:18:36 CET 2016
by reger
fix typing error from commit https://github.com/yacy/yacy_search_server/commit/60ba5c117c4bd7aa30969a9943cc57762c0ab4ba
Changed Files: source/net/yacy/server/http/HTTPDemon.java
Thu Nov 24 00:16:22 CET 2016
by reger
fix authentication by hit in userdb (wrong parameter)
Changed Files: source/net/yacy/search/Switchboard.java
Sat Nov 19 05:56:48 CET 2016
by reger
fix Domains.stripToPort, make ipv6 save
Changed Files: source/net/yacy/cora/protocol/Domains.java
Fri Nov 18 03:00:30 CET 2016
by reger
let RequestHeader.logout succeed w/o throwing exception by default
Changed Files: source/net/yacy/cora/protocol/RequestHeader.java
Tue Nov 15 01:13:31 CET 2016
by luccioman
upd to jsoup-1.10.1 : fixed to Eclipse classpath and Ant configuration.
Changed Files: .classpath, build.xml
Sat Nov 12 21:58:06 CET 2016
by reger
fix cutoff text in button and adjust formatting
Changed Files: htroot/IndexControlURLs_p.html
Thu Nov 10 22:22:37 CET 2016
by reger
bring back/fix: search results and logo without external icon
Changed Files: htroot/YaCySearchPluginFF.html, htroot/env/base.css, htroot/yacyinteractive.html
Fri Oct 14 11:55:49 CEST 2016
by luccioman
Fixed license headers on files created to improve favicon management.
Changed Files: htroot/Load_MediawikiWiki.java, htroot/Load_PHPBB3.java, htroot/ViewFavicon.java, source/net/yacy/data/InvalidURLLicenceException.java, source/net/yacy/document/parser/html/IconEntry.java, source/net/yacy/document/parser/html/IconLinkRelations.java, source/net/yacy/http/servlets/TemplateMissingParameterException.java, source/net/yacy/visualization/ImageViewer.java, test/ImageViewerPerfTest.java, test/ImageViewerTest.java, test/java/net/yacy/document/parser/html/ContentScraperTest.java, test/java/net/yacy/document/parser/html/IconEntryTest.java, test/java/net/yacy/kelondro/data/meta/URIMetadataNodeTest.java, test/java/yacysearchitemTest.java, test/net/yacy/data/URLLicenseConcurrentTest.java
Fri Oct 14 09:42:43 CEST 2016
by luccioman
HTML validation : fixed URL encoding of Pictures link.
Changed Files: htroot/yacysearchitem.html, htroot/yacysearchtrailer.java, test/java/net/yacy/server/serverObjectsTest.java
Thu Oct 13 16:18:24 CEST 2016
by luccioman
Fixed MultiProtocolURL.resolveBackpath to handle remaining '..' segments
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java, test/java/net/yacy/cora/document/id/MultiProtocolURLTest.java
Wed Oct 12 09:16:47 CEST 2016
by luccioman
HTML validation : fixed URL encoding of search results action links.
Changed Files: htroot/yacysearchitem.java
Fri Oct 07 10:43:06 CEST 2016
by luccioman
Fixed HTML validation errors and grouped radios options in fieldsets
Changed Files: htroot/index.html
Thu Sep 29 14:24:15 CEST 2016
by luccioman
Fixed <p> spacers : blocks elements such as <div> are not allowed inside 
Changed Files: htroot/yacysearchtrailer.html
Thu Sep 29 11:42:59 CEST 2016
by luccioman
Fixed HTML validation error : "Stray end tag div"
Changed Files: htroot/Surftips.html, htroot/env/templates/embeddedfooter.template, htroot/env/templates/simplefooter.template
Wed Sep 28 16:08:36 CEST 2016
by reger
fix ConfigBasic servlet parameter name for Japanese _jp->_ja
Changed Files: htroot/ConfigBasic.java
Sat Sep 24 23:33:21 CEST 2016
by reger
fix NPE in QuickCrawlLink_p if param doesn't contain crawl url
Changed Files: htroot/QuickCrawlLink_p.java
Thu Sep 22 16:25:47 CEST 2016
by luccioman
Fixed HTML validation errors : duplicate ids.
Changed Files: htroot/env/templates/header.template, htroot/env/templates/simpleheader.template
Thu Sep 22 16:20:13 CEST 2016
by luccioman
Fixed W3C validation error : percent encode '[' and ']' chars in hrefs.
Changed Files: htroot/env/templates/simpleheader.template
Thu Sep 22 11:25:33 CEST 2016
by luccioman
Fixed NullPointerException cases
Changed Files: source/net/yacy/document/parser/html/ContentScraper.java
Wed Sep 14 16:24:32 CEST 2016
by Michael Peter Christen
fix for java version "9-ea"
Changed Files: source/net/yacy/yacy.java
Wed Sep 07 10:03:48 CEST 2016
by luccioman
Fixed NullPointerException case and added Javadoc
Changed Files: source/net/yacy/cora/federate/solr/responsewriter/OpensearchResponseWriter.java
Wed Sep 07 06:48:52 CEST 2016
by Michael Peter Christen
catch exception and write end of object
Changed Files: source/net/yacy/cora/federate/solr/responsewriter/YJsonResponseWriter.java
Tue Sep 06 17:19:54 CEST 2016
by Michael Peter Christen
fix for broken facet names
Changed Files: source/net/yacy/cora/federate/solr/responsewriter/YJsonResponseWriter.java
Tue Sep 06 09:05:39 CEST 2016
by Michael Peter Christen
fix for too long snippets
Changed Files: source/net/yacy/cora/federate/solr/responsewriter/YJsonResponseWriter.java
Tue Sep 06 09:00:35 CEST 2016
by Michael Peter Christen
fix for bad comma
Changed Files: source/net/yacy/cora/federate/solr/responsewriter/YJsonResponseWriter.java
Tue Sep 06 07:44:38 CEST 2016
by Michael Peter Christen
fix for bad json
Changed Files: source/net/yacy/cora/federate/solr/responsewriter/OpensearchResponseWriter.java, source/net/yacy/cora/federate/solr/responsewriter/YJsonResponseWriter.java
Fri Sep 02 11:23:02 CEST 2016
by luccioman
Fixed indentation for better readability.
Changed Files: source/net/yacy/yacy.java
Tue Aug 30 01:34:32 CEST 2016
by reger
fix deactivation of Russian Thesaurus
Changed Files: htroot/DictionaryLoader_p.java
Wed Aug 24 16:11:52 CEST 2016
by galix
Fixed packaging for YaCy 1.90 on MacOS
Changed Files: addon/YaCy.app/Contents/Info.plist, build.xml
Fri Aug 19 19:24:15 CEST 2016
by Michael Peter Christen
fixed missing browser activation in linux
Changed Files: source/net/yacy/gui/framework/Browser.java
Wed Aug 10 11:02:04 CEST 2016
by luccioman
Fixed description small typing errors.
Changed Files: app.json
Wed Aug 10 06:48:08 CEST 2016
by Michael Peter Christen
fix for NPE and small performance enhancement
Changed Files: source/net/yacy/kelondro/blob/BEncodedHeap.java
Fri Aug 05 12:16:11 CEST 2016
by luccioman
Fixed syntax errors.
Changed Files: docker/Readme.md
Sun Jul 17 22:16:08 CEST 2016
by reger
fix fr.lng typo in submenuRanking.template
Changed Files: locales/fr.lng
Wed Jul 13 03:05:59 CEST 2016
by luccioman
Reverted initial log level modified for debugging on Heroku
Changed Files: defaults/yacy.logging
Wed Jul 13 01:15:25 CEST 2016
by luccioman
Fixed relative Markdown links 
Changed Files: Heroku.md, README.md
Sun Jul 10 05:44:56 CEST 2016
by reger
fix CookieTest_p parameter from ResponseHeader to RequestHeader
Changed Files: htroot/CookieTest_p.java
Sat Jul 09 23:39:43 CEST 2016
by reger
fix GSA servlet clientIP retrival
Changed Files: source/net/yacy/http/servlets/GSAsearchServlet.java
Wed Jul 06 23:52:02 CEST 2016
by reger
fix HostQueueTest - changed open parameter
Changed Files: test/java/net/yacy/crawler/HostQueueTest.java


Other Changes   
Jump to: YaCy Release 1.92 top / Bugfixes

CommitDescription
Mon Dec 26 15:00:55 CET 2016
by Michael Peter Christen
release 1.92
Changed Files: addon/YaCy.app/Contents/Info.plist, pom.xml
Mon Dec 26 14:13:45 CET 2016
by Michael Peter Christen
release 1.92
Changed Files: build.properties
Mon Dec 26 13:50:35 CET 2016
by Michael Peter Christen
added luccioman development release builds as discussed in
http://forum.yacy-websuche.de/viewtopic.php?f=8&t=5906
Changed Files: defaults/yacy.network.freeworld.unit
Mon Dec 26 13:32:36 CET 2016
by Michael Peter Christen
updated seed-list bootstrap locations
Changed Files: defaults/yacy.network.freeworld.unit
Sun Dec 25 23:39:55 CET 2016
by reger
Include itemprop latitude/longitude (see schema.org) in attribute 
parsing for lat/lon.
Harmonize number parsing for lat/lon to parseDouble.
Fix endDate_dts value assignment.
Changed Files: source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/search/schema/CollectionConfiguration.java
Sat Dec 24 06:57:11 CET 2016
by reger
fix html tag attribute parsing containing attribute w/o value
e.g. itemscope or autofocus (in such case the next key was not properly
recognized).
Changed Files: source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/kelondro/io/CharBuffer.java, test/java/net/yacy/kelondro/io/CharBufferTest.java
Sat Dec 24 03:11:35 CET 2016
by reger
include html5 <time> tag in content scraper,
add "datetime" property of <time> tag to scrapers startdate list.
Datetime is parsed as iso8601 (xml) date, html5 allows partial as well
as duration (not handled by this)
Changed Files: source/net/yacy/document/parser/html/ContentScraper.java, test/java/net/yacy/document/parser/html/ContentScraperTest.java
Fri Dec 23 19:03:44 CET 2016
by reger
add/allow to create a bookmark of result viewed via urlproxy.
For this on the header of the viewed result a "add bookmark" button is
available (for authenticated users).
Currently the bookmark is added to a (virtual) bookmark folder "/proxy"
w/o any additional tags etc. 
Changed Files: htroot/proxymsg/urlproxyheader.html, htroot/proxymsg/urlproxyheader.java
Thu Dec 22 19:52:38 CET 2016
by reger
fix and prevent exception on missing required cookie name 
skip cookie creation if name is empty.
Changed Files: htroot/CookieTest_p.java, source/net/yacy/cora/protocol/ResponseHeader.java
Thu Dec 22 16:25:09 CET 2016
by luccioman
FTP crawl start URLs : applied crawl profile depth control

Applied rules :
- when the FTP URL denotes a file resource, stack it as any start URL :
eventually embedded links can be followed applying the usual depth rules
- when the FTP URL denotes a directory, list files under this directory
and stack them for crawl, and repeat the process on sub folders until
crawl depth is reached
Changed Files: source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/search/Switchboard.java
Wed Dec 21 07:22:41 CET 2016
by luccioman
Added a title on the previous and next page pagination buttons.

This is to clarify the meaning of these buttons for users who could
think they link respectively to the first and last results page.
Changed Files: htroot/js/yacysearch.js
Wed Dec 21 02:04:13 CET 2016
by reger
activate filetype navigator plugin and restrict config (append) of navs
to not already actives.
Dht results are now included in count this might over shoot on redundant
dht and solr, while the previous solr facet based was always low.
 
Changed Files: htroot/ConfigSearchPage_p.html, htroot/ConfigSearchPage_p.java, htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java, source/net/yacy/search/navigator/NavigatorPlugins.java, source/net/yacy/search/query/SearchEvent.java
Tue Dec 20 02:06:43 CET 2016
by reger
show earthsearch.png in ConfigSearchPage layout on activated location 
navigator (for more realistic impression)
Changed Files: htroot/ConfigSearchPage_p.html
Mon Dec 19 23:56:03 CET 2016
by reger
add FileTypeNavigator plugin 
Changed Files: source/net/yacy/search/navigator/FileTypeNavigator.java
Mon Dec 19 02:30:55 CET 2016
by reger
remove obsolete put of HttpServletRequest attributes to YaCy servlet 
parameters on SSI (server side includes).
Query parameters are already merged by dispatcher.include, making copy
of parameter (RequestDispatcher.INCLUDE_QUERY_STRING) obsolete.
All other parameter are not used as YaCy servlet arguments.
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Sun Dec 18 21:11:00 CET 2016
by reger
correct DefaultServlet resource pathinContext calculation
exclude servletPath option as resources are always relative to htroot 
or htdocs, the change reflects this.
Theoretically it and the recent adjustments arcording relative urls 
allows to configure the instance to be configurable in a path other as 
root (/) 
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Sun Dec 18 02:38:43 CET 2016
by reger
reduce creation of empty legacy RequestHeader() in situation where null
is acceptable (less for garbage collection).
Changed Files: htroot/api/push_p.java, source/net/yacy/crawler/retrieval/FTPLoader.java, source/net/yacy/crawler/retrieval/FileLoader.java, source/net/yacy/crawler/retrieval/HTTPLoader.java, source/net/yacy/crawler/retrieval/Response.java, source/net/yacy/crawler/retrieval/SMBLoader.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/yacy.java
Sun Dec 18 01:02:56 CET 2016
by reger
adjust Cache getHeader to prev. changes/commit
Changed Files: source/net/yacy/crawler/data/Cache.java
Sun Dec 18 00:57:47 CET 2016
by reger
remove the overhead of unused reverseMappingCache of HeaderFramewor / RequestHeader
Changed Files: source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/protocol/RequestHeader.java, source/net/yacy/cora/protocol/ResponseHeader.java
Sat Dec 17 23:29:57 CET 2016
by reger
reduce redundant declaration for simple date formatter 
using predefined GenericFormatter.SIMPLE_FORMATTER
Changed Files: htroot/MessageSend_p.java, htroot/Messages_p.java, htroot/Wiki.java, htroot/yacy/message.java
Sat Dec 17 21:44:09 CET 2016
by reger
use GenericFormatter SimpleDate for sortable column in table_API
to allow correct chronological sorting (of the date string)
fix for http://mantis.tokeek.de/view.php?id=585
Changed Files: htroot/Table_API_p.java
Sat Dec 17 02:38:52 CET 2016
by reger
del dead menu link to /repository
(directory not created in current distribution -> old)
Changed Files: htroot/Load_PHPBB3.java, htroot/env/templates/submenuPublication.template, source/net/yacy/peers/SeedDB.java
Sat Dec 17 01:33:00 CET 2016
by reger
adjust DomainHandler working on resolved .yacy domain
(remove obsolete check for path on hostname)
Changed Files: source/net/yacy/http/YacyDomainHandler.java
Fri Dec 16 20:38:54 CET 2016
by luccioman
Use loaderDispatcher instead of HTTPClient to download releases.

The default redirection strategy when using directly HTTPClient is
incorrect when redirection is cross host (the original Host header is
still sent when requesting the redirected location).

YaCy LoaderDispatcher handles redirections properly, thus release
archive files using redirected URLs (such as the URLs on a GitHub
Release page) are successfully downloaded.
Changed Files: source/net/yacy/peers/operation/yacyRelease.java
Fri Dec 16 11:03:09 CET 2016
by luccioman
Hardened system update checks.

When a downloaded archive release is corrupted, empty, or can not be
opened for any reason, the update script must not be launched because it
erases the existing lib/*.jar libraries.
Changed Files: htroot/ConfigUpdate_p.html, htroot/ConfigUpdate_p.java, htroot/Steering.html, htroot/Steering.java, source/net/yacy/peers/operation/yacyRelease.java, source/net/yacy/search/Switchboard.java, source/net/yacy/utils/tarTools.java
Fri Dec 16 10:43:00 CET 2016
by luccioman
Added some Javadocs.
Changed Files: source/net/yacy/search/Shutdown.java
Thu Dec 15 23:59:50 CET 2016
by reger
expanded error message on rejected crawl url due to faile dns lookup
close of http://mantis.tokeek.de/view.php?id=678
Changed Files: source/net/yacy/crawler/CrawlStacker.java
Thu Dec 15 15:30:36 CET 2016
by luccioman
Check HTTP status when downloading a release, and report eventual error.
Changed Files: htroot/ConfigUpdate_p.html, htroot/ConfigUpdate_p.java, source/net/yacy/peers/operation/yacyRelease.java
Thu Dec 15 00:45:08 CET 2016
by reger
allow protocol navigator to be unselected if only one button is shown
after activating navi/facetfilter
Changed Files: htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java
Wed Dec 14 12:49:41 CET 2016
by luccioman
Fixed admin navbar rendering at various screen sizes.

Fix mantis 443 (http://mantis.tokeek.de/view.php?id=443).

Tested on :
 - Debian jessie : Firefox ESR 45.5.1
 - MS Windows 10 : Firefox 50.1.0, Chrome 55 and Edge
 - Emulated devices/adaptative views embedded in the previously
mentionned browsers
Changed Files: htroot/env/base.css, htroot/env/templates/header.template
Tue Dec 13 22:14:16 CET 2016
by reger
add HostNavigator plugin
Changed Files: source/net/yacy/search/navigator/HostNavigator.java
Tue Dec 13 22:12:11 CET 2016
by reger
translate languages in ConfigBasic.html
Changed Files: locales/de.lng
Tue Dec 13 16:50:24 CET 2016
by luccioman
Fixed locations search navbar overlapping issues.

This is similar to the main yacysearch navbar issues described in mantis
708 (http://mantis.tokeek.de/view.php?id=708)
Changed Files: htroot/yacysearch_location.html
Tue Dec 13 15:33:18 CET 2016
by luccioman
Fixed resource switch button overlapping at various screen sizes.

Fixes second part of mantis 708
(http://mantis.tokeek.de/view.php?id=708)

The bootstrap-switch component has some sizing issues with long labels,
which are not likely to be solved soon due to a lack of resources on
that project (see issue
https://github.com/nostalgiaz/bootstrap-switch/issues/419 )

This fix works by applying the following ideas :
 - labels are long, so font-size and padding are reduced on small screen
sizes using a media query
 - use relative percent width values on the component wrappers to
prevent overlapping on the neighbour content
 - disable animation because it relies on absolute pixels width values
Changed Files: htroot/env/base.css, htroot/yacysearchtrailer.html
Mon Dec 12 12:55:20 CET 2016
by luccioman
Fixed YaCy logo (no external mark) for the refactored navbar search.

Thanks to reger24 feedback.
Changed Files: htroot/env/base.css
Mon Dec 12 02:29:15 CET 2016
by reger
prepare ConfigSearchPage servlet to append and remove navigator plugins,
keeping order of added nav's.
The search page preview template displays active navs. Therefore a select
and add button has been added below the preview (to keep it close to actual).
This should in future likely be done by drag&drop (html5 feature).
Changed Files: htroot/ConfigSearchPage_p.html, htroot/ConfigSearchPage_p.java
Sun Dec 11 21:46:29 CET 2016
by reger
bundle initialization of search navigation plugins in separate handler
class to allow to use navigator map in config servlets 
(without need to create a search event)
Changed Files: source/net/yacy/search/navigator/NavigatorPlugins.java, source/net/yacy/search/query/SearchEvent.java
Sun Dec 11 04:43:36 CET 2016
by reger
catch ip-format exception on wrong server access setting ip filter
as reported in http://mantis.tokeek.de/view.php?id=713
to prevent abort of initialization.
This jetty/whitelist ipaccesshandler accepts currently only ipv4
Changed Files: source/net/yacy/http/Jetty9HttpServerImpl.java
Sat Dec 10 22:08:09 CET 2016
by reger
list entries in outgoing cookie monitor one per line
for easier readability.
For this adjust outgoingCookies entry to use Cookie[] instead of String[]
Changed Files: htroot/CookieMonitorOutgoing_p.html, htroot/CookieMonitorOutgoing_p.java, source/net/yacy/server/http/HTTPDProxyHandler.java
Fri Dec 09 11:25:09 CET 2016
by luccioman
Fixed yacy search navabar header overlapping at various screen sizes.

- using a icon-only admin button at small and medium screen size
- using a icon-only "Search Interfaces" button at small screen size
- hiding the YaCy brand at extra-small screen size

Fixes the header part of mantis 708
(http://mantis.tokeek.de/view.php?id=708).

Navigator button overlapping is still to fix. 
Changed Files: htroot/env/templates/embeddedSearchHeader.template, htroot/env/templates/simpleSearchHeader.template, htroot/yacysearch.html
Fri Dec 09 00:33:20 CET 2016
by reger
enable getCookies for HeaderFramework reusing Jetty CookieCutter
Changed Files: source/net/yacy/cora/protocol/RequestHeader.java
Wed Dec 07 00:51:07 CET 2016
by reger
add other custom response header entries set by servlets to the response
to the client (not cookies only). This is used by some servlets to mainly 
set "Access-Control-Allow-Origin" header. Added a contains check to be
sure no header set by Defaultservlet is overwritten.
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Wed Dec 07 00:39:53 CET 2016
by reger
use more availabe SwitchboardConstans in ProxyIndexingMonitor_p
(to easily find usage)
Changed Files: htroot/ProxyIndexingMonitor_p.java
Mon Dec 05 18:12:21 CET 2016
by luccioman
Fixed language detector initialization and NullPointerException cases.

NullPointerException occurred when using and Identificator instance
which encountered and error in its constructor.
This error could be caused by a missing "langdetect" folder in the
current folder of the main process, or by simultaneous first calls to
the constructor, initializing concurrently the DetectorFactory.langlist.

Fixes the mantis 714 (http://mantis.tokeek.de/view.php?id=714)
Changed Files: source/net/yacy/document/language/Identificator.java, source/net/yacy/search/Switchboard.java
Mon Dec 05 10:58:23 CET 2016
by luccioman
Added a unit after the vocabulary size value for easier understanding.
Changed Files: htroot/Vocabulary_p.html
Mon Dec 05 03:15:24 CET 2016
by reger
allow request.setAttribute w/o "not implemented" exception by default
skip unused CONNECTION_PROP_ARGS check in getQueryString
Changed Files: source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/protocol/RequestHeader.java
Mon Dec 05 00:11:03 CET 2016
by reger
replace one more arbitrary CONNECTION_PROP_CLIENTIP header with std.
getRemoteAddr()
Changed Files: htroot/Settings_p.java
Sun Dec 04 22:26:02 CET 2016
by reger
use bootstrap button style in MessageSend_p.html
and align buttons with form
Changed Files: htroot/MessageSend_p.html
Sun Dec 04 22:11:22 CET 2016
by reger
skip comparing "xxxxx" on missing authorization header in Blog servlet
Changed Files: htroot/Blog.java, htroot/BlogComments.java
Sun Dec 04 00:58:45 CET 2016
by reger
add UserDB lastaccess update calls on login
Changed Files: source/net/yacy/data/UserDB.java
Sat Dec 03 09:05:19 CET 2016
by luccioman
Reduced GeoNames locations memory footprint.

Using String instead of StringBuilder instances in GeonamesLocation
allows to reuse the same immutable objects in the Tagging class.

Measurements with VisualVM and GeoNames 0 enabled (cities with a
population > 1000) :
 - OverArchingLocation retained size :
  - initial : 164 666 830 bytes
  - after refactoring : 97 736 804 bytes
 - AutotaggingLibrary retained size :
  - initial : 354 713 633 bytes
   - after refactoring : 309 718 763 bytes
Changed Files: source/net/yacy/cora/geo/GeonamesLocation.java
Fri Dec 02 13:45:45 CET 2016
by luccioman
Fixed a NullPointerException case.

Could occur when a search request was performed just after peer startup,
and the Switchboard Thread "LibraryProvider.initialize" had completed,
thus requesting a ProbabilisticClassifier not completely initialized
(and having a null contexts property).
Changed Files: source/net/yacy/document/ProbabilisticClassifier.java
Fri Dec 02 13:12:47 CET 2016
by luccioman
Small memory footprint reduction for GeonamesLocation.

Reusing the same geonameid Integer instance between `id2loc` and
`name2ids` maps reduces (a little) memory footprint.
Measured OverarchigLocation class retained memory with VisualVM on
openJDK 8 :
 - initial : 183 439 490 bytes
 - after refactoring : 164 666 830 bytes
Changed Files: source/net/yacy/cora/geo/GeonamesLocation.java
Wed Nov 30 17:49:51 CET 2016
by luccioman
Small complementary memory footprint improvement for synonyms.

Memory footprint measured with VisualVM and all synonyms enabled :
 - before : 195 015 914 bytes
 - after : 192 548 826 bytes
Changed Files: source/net/yacy/cora/language/synonyms/SynonymLibrary.java
Wed Nov 30 16:50:25 CET 2016
by luccioman
Improved synonyms memory footprint.

The idea is to avoid unnecessary String objects duplication for the same
words. Particularly efficient with the large moby thesaurus.

Memory footprint measurements with VisualVM :
 - openthesaurus_de_yacy :
 	- initial : 19 443 796 bytes
 	- after refactoring : 18 012 606 bytes

 - mobythesaurus_en_yacy :
 	- initial : 343 453 904 bytes
 	- after refactoring : 173 843 780 bytes

 - thesaurus_ru_yacy :
 	- initial : 3 800 706 bytes
 	- after refactoring : 3 466 612 bytes

 - de + en + ru : 
 	- initial : 366 603 450 bytes
 	- after refactoring : 195 015 914 bytes
Changed Files: source/net/yacy/cora/language/synonyms/SynonymLibrary.java
Tue Nov 29 03:20:33 CET 2016
by reger
move Digest auth checks from DefaultServlet to adminAuthenticated,
eliminating the need to modify http header on Servlet container handled 
Digest authentication, to simulate Basic auth for YaCy servlets.
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/search/Switchboard.java
Tue Nov 29 02:56:43 CET 2016
by luccioman
Fixed JavaScript error "hs.htmlExpand is not a function".

This error occurs on /ConfigSearchPage_p.html and on search results page
when Metadata links are enabled.

The fix was to remove unnecessary use of hs.htmlExpand() which is now
part of highslide-full.js library file, currently not distributed with
YaCy (only includes highslide.js). The Metadata links work correctly and
the initial dynamic expansion offered by htmlExpand() did not bring much
usability.
Changed Files: htroot/ConfigSearchPage_p.html, htroot/yacysearchitem.html
Tue Nov 29 01:34:33 CET 2016
by luccioman
Converted "clone" URL links in Table_API_p.html to purely relative ones.

Again for easier YaCy integration when running behind a reverse proxy
subfolder. 
Changed Files: htroot/Table_API_p.java
Mon Nov 28 22:10:05 CET 2016
by luccioman
Fixed image and favicon viewing for unauthenticated local requests.

As reported by @reger24, image and favicon viewing was broken with
unauthenticated requests on peers configured to require authentication
even from localhost.

So I unified viewing rights check in a single new function on
ImageViewer class.
Changed Files: htroot/ViewFavicon.java, htroot/ViewImage.java, htroot/yacysearchitem.java, source/net/yacy/visualization/ImageViewer.java
Sat Nov 26 23:28:30 CET 2016
by reger
remove login cookie generation for static admin ind User servlet
cookieAuth is never successful for static admin, leaving the creation and
handling for login cookies for static admin obsolete.
Changed Files: htroot/User.java, source/net/yacy/data/UserDB.java
Sat Nov 26 19:53:00 CET 2016
by reger
exclude external link icon in Collage servlet
(icons display not close to image in IE)
Changed Files: htroot/Collage.java
Sat Nov 26 18:26:14 CET 2016
by reger
include check to prevent adding username identical with static admin
in ConfigAccounts_p
Changed Files: htroot/ConfigAccounts_p.html, htroot/ConfigAccounts_p.java, locales/master.lng.xlf
Fri Nov 25 11:10:03 CET 2016
by luccioman
Converted ajax URL to relative and added a check on the response status.

This makes YaCy easier to configure when running behind a reverse Proxy.

The check on status avoids trying to update the page with error text
content when the server returned a 404 or 500 error message for example.
Changed Files: htroot/Crawler_p.java, htroot/HostBrowser.java, htroot/IndexControlURLs_p.html
Fri Nov 25 03:05:53 CET 2016
by reger
completing the usage of CONNECTION_PROP_CLIENT_HTTPSERVLETREQUEST in
HTTPDProxyHandlers logging facility.
Changed Files: source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/server/http/HTTPDProxyHandler.java
Fri Nov 25 02:00:20 CET 2016
by reger
update the older ResponseHeader patch to handle cookies,
to work directly with javax.servlet.http.Cookie (rename headerProps to
cookieStore as is only used for this).
(Re)implement set-cookie in DefaultServlet to make cookieAuthentication
work as designed.
Changed Files: htroot/CookieTest_p.java, source/net/yacy/cora/protocol/ResponseHeader.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/server/http/HTTPDemon.java
Fri Nov 25 01:09:42 CET 2016
by reger
make RequestHeader login succeed (without throwing exception by default)
correct getAuthType to return Auth Scheme only after authentication
Changed Files: source/net/yacy/cora/protocol/RequestHeader.java
Wed Nov 23 17:59:52 CET 2016
by luccioman
Fixed thread name consistency for improved monitoring.

Some tasks were modifying the current thread name without restoring it
once finished as it is effectively done elsewhere.
Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/document/Condenser.java, source/net/yacy/peers/operation/yacyRelease.java
Wed Nov 23 11:21:14 CET 2016
by luccioman
Added a name prefix to pooled threads for easier monitoring.

Using JVM monitoring tools, it is then easier to identify tasks running
inside thread pool with a custom prefix rather than the generic one :
"pool-".
Changed Files: source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/TimeoutRequest.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/index/RowHandleMap.java
Wed Nov 23 01:33:01 CET 2016
by reger
ConfigSearchPage, move protocol navi up to better simulate actual design.
Because here btn-group-justified screws up table column width (Explorer
and Firefox) bootstrap btn-group is used.
Changed Files: htroot/ConfigSearchPage_p.html
Tue Nov 22 09:40:22 CET 2016
by luccioman
RWI Ranking Configuration page : fixed missing required dependencies.

Fix for mantis 707 (http://mantis.tokeek.de/view.php?id=707)
Changed Files: htroot/RankingRWI_p.html
Tue Nov 22 01:17:33 CET 2016
by reger
add high scored items first to YearNavigator (to make sure to be included
in sorted view)
Changed Files: source/net/yacy/search/navigator/StringNavigator.java, source/net/yacy/search/navigator/YearNavigator.java
Mon Nov 21 19:55:13 CET 2016
by luccioman
Added charset meta to Solr HTML writers.

Non-ASCII characters are thus correctly rendered in browsers.
This is a fix ro mantis 706 : http://mantis.tokeek.de/view.php?id=706
Changed Files: source/net/yacy/cora/federate/solr/responsewriter/GrepHTMLResponseWriter.java, source/net/yacy/cora/federate/solr/responsewriter/HTMLResponseWriter.java
Mon Nov 21 00:19:11 CET 2016
by reger
make a LanguageNavigator availabel, useable for the SearchEvent.naviator
plugin (not activated yet).
Changed Files: source/net/yacy/search/navigator/LanguageNavigator.java
Sun Nov 20 19:30:38 CET 2016
by reger
implement RequestHeader.setCharacterEncoding for legacy header,
make sure .getProtocol returns a http version
move the patch for Set-Cookie to ResponseHeader (applies only here)
Changed Files: source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/protocol/RequestHeader.java, source/net/yacy/cora/protocol/ResponseHeader.java
Sun Nov 20 12:13:00 CET 2016
by Michael Peter Christen
added+disabled a federate search experiment
Changed Files: source/net/yacy/http/servlets/SolrSelectServlet.java
Sun Nov 20 05:27:33 CET 2016
by reger
use DigestURL in UrlProxyServlet as parameter to pass requested url to 
handler.
UrlProxyServlet splits url in parts to pass it on as parameter and 
HeaderFramework constructs a url from param parts. This is obsolete if
already created url is used (makes HeaderFramework.getRequestURL obsolete
= removed)
Changed Files: source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/http/servlets/YaCyProxyServlet.java, source/net/yacy/server/http/HTTPDProxyHandler.java, source/net/yacy/server/http/HTTPDemon.java
Sun Nov 20 04:02:28 CET 2016
by reger
adjust messages_p servlet to standard header.getPathInfo, too
(replacing non standard http header)
Changed Files: htroot/Messages_p.java
Sun Nov 20 00:20:47 CET 2016
by reger
remove old/obsolete clear text "adminAccount" credential entry from init
and setConfig (.,empty) from servlets/code
Changed Files: defaults/yacy.init, htroot/ConfigAccounts_p.java, htroot/SettingsAck_p.java, source/net/yacy/migration.java, source/net/yacy/search/Switchboard.java
Sun Nov 20 00:15:21 CET 2016
by reger
fix Jetty9YaCySecurityHandler, length check of Basic credential,
add comment to SwitchboardConstants.AdminAccount const
Changed Files: source/net/yacy/http/Jetty9YaCySecurityHandler.java, source/net/yacy/search/SwitchboardConstants.java
Sat Nov 19 16:37:16 CET 2016
by reger
fix Domain.stripToHostName for some IPv6 cases
add unit test for it
Changed Files: source/net/yacy/cora/protocol/Domains.java, test/java/net/yacy/cora/protocol/DomainsTest.java
Sat Nov 19 06:22:55 CET 2016
by reger
add unit test for Domains.stripToPort,
simplify ipv6 check
Changed Files: source/net/yacy/cora/protocol/Domains.java, test/java/net/yacy/cora/protocol/DomainsTest.java
Sat Nov 19 05:08:11 CET 2016
by reger
fix HeaderFramework.getRequestURL on host with port considering ipv6 host
Changed Files: source/net/yacy/cora/protocol/HeaderFramework.java
Sat Nov 19 03:29:23 CET 2016
by reger
refactor: move GSA specific date formatter to GSAservlet
adjust return type to String for HeaderFrameWork.getSingle
Changed Files: source/net/yacy/cora/federate/solr/responsewriter/GSAResponseWriter.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/http/servlets/GSAsearchServlet.java
Sat Nov 19 02:10:21 CET 2016
by reger
refactor: move convertHeaderFromJetty to ProxHandler (only used with active proxy
not needed for standard servlets)
Changed Files: source/net/yacy/http/ProxyCacheHandler.java, source/net/yacy/http/ProxyHandler.java, source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/http/servlets/YaCyProxyServlet.java
Sat Nov 19 01:02:05 CET 2016
by reger
let User servlet detect static admin with (newer) md5 encoded pwd
(complete a old todo)
Changed Files: htroot/User.java
Fri Nov 18 21:44:16 CET 2016
by reger
fix legacy getHeaderCookies to work with cookies from original 
HttpServletRequest, by moving to RequestHeader.
Changed Files: source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/protocol/RequestHeader.java
Fri Nov 18 02:39:53 CET 2016
by reger
remove login request directly after logout,
and add logout from servlet container
make logout button red
Changed Files: htroot/User.html, htroot/User.java
Fri Nov 18 01:12:14 CET 2016
by reger
fix UserDB.proxyAuth from header string
(take care of prefix "Basic" in header entry)
Changed Files: source/net/yacy/data/UserDB.java
Wed Nov 16 15:13:58 CET 2016
by luccioman
Ensure file input stream are closed in both normal and error cases.
Changed Files: htroot/DictionaryLoader_p.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java
Wed Nov 16 15:12:21 CET 2016
by luccioman
FileUtils : added some JavaDocs and unit test cases
Changed Files: source/net/yacy/kelondro/util/FileUtils.java, test/java/net/yacy/kelondro/util/FileUtilsTest.java
Wed Nov 16 01:42:53 CET 2016
by reger
make use of userInRole for quick login verification
Changed Files: source/net/yacy/search/Switchboard.java
Tue Nov 15 02:37:03 CET 2016
by reger
adjust servlet RequestHeader.getLocale
init jvm defaultLocale matching UI language
Changed Files: source/net/yacy/cora/protocol/RequestHeader.java, source/net/yacy/search/Switchboard.java
Mon Nov 14 23:10:32 CET 2016
by reger
upd to jsoup-1.10.1.jar
Changed Files: lib/jsoup-1.10.1.jar, lib/jsoup.License, pom.xml
Mon Nov 14 22:53:26 CET 2016
by reger
remove obsolete X_YACY... header declarations
not in use (no writes, only remove and try to read).
Obsolete parameter setupHttpClient


Changed Files: source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/http/Jetty9YaCySecurityHandler.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/server/http/HTTPDProxyHandler.java
Mon Nov 14 20:17:35 CET 2016
by luccioman
Network Access servlet : render pure relative HTML links.

For better YaCy integration behind reverse proxy with subfolder.

Tested on Debian Jessie with an apache2 reverse proxy.

See related mantis issues http://mantis.tokeek.de/view.php?id=106 and
http://mantis.tokeek.de/view.php?id=701
Changed Files: htroot/AccessTracker_p.html, htroot/AccessTracker_p.java
Mon Nov 14 04:49:30 CET 2016
by reger
more use of RequestHeader constant referer, authorization 
in Jetty9YaCySecurityHandler
Changed Files: source/net/yacy/http/Jetty9YaCySecurityHandler.java
Mon Nov 14 03:01:15 CET 2016
by reger
make more use of the new ServletRequest interface methodes
getScheme, getServerPort (in QuickCrawlLink_p & YaCyDefaultServlet)
Changed Files: htroot/QuickCrawlLink_p.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java
Mon Nov 14 00:18:40 CET 2016
by luccioman
Updated Pattern JavaDoc links to current minimum (1.7) JDK version.
Changed Files: htroot/Blacklist_p.html, htroot/CrawlStartExpert.html, locales/cn.lng, locales/de.lng, locales/hi.lng, locales/ru.lng, locales/uk.lng
Sun Nov 13 23:34:27 CET 2016
by luccioman
Updated links to external Java and Solr docs to currently used versions.
Changed Files: htroot/ContentAnalysis_p.html, htroot/RegexTest.html, locales/ru.lng
Sat Nov 12 23:29:25 CET 2016
by luccioman
Absolute URLs in yacysearch.* : ensure no downgrade from https to http

Also removed unnecessary use of deprecated Seed.getIP().
Changed Files: htroot/yacysearch.atom, htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearch.rss
Sat Nov 12 15:54:35 CET 2016
by luccioman
Converted one more set of URLs to pure relative ones.

Easier YaCy peer configuration behind a reverse proxy subfolder : no
need for the reverse proxy to rewrite HTML links or URLs in css files.

Tested on Debian Jessie with an apache2 reverse proxy.

See related mantis issues http://mantis.tokeek.de/view.php?id=106 and
http://mantis.tokeek.de/view.php?id=701
Changed Files: htroot/CrawlStartExpert.html, htroot/WatchWebStructure_p.html, htroot/api/yacydoc.html, htroot/test.html, locales/master.lng.xlf, locales/ru.lng
Wed Nov 09 02:33:42 CET 2016
by reger
fix Tables.hasIndex check for tablename = key
apply same functionality to hasHeap (to not create new table on call hasHeap)
Changed Files: source/net/yacy/kelondro/blob/Tables.java
Tue Nov 08 23:42:12 CET 2016
by reger
del abandoned indexcleaner.html, servlet deleted with commit
https://github.com/yacy/yacy_search_server/commit/3bcd9d622b1988e221d271bdb088fd1fbcd5e018
crawler servlet log warning line on failure in one of multiple urls (instead of exception msg)
indexcontrolrwi skip not needed type conversion on ranking
Changed Files: htroot/Crawler_p.java, htroot/IndexControlRWIs_p.java
Tue Nov 08 03:12:43 CET 2016
by luccioman
Switched openstreetmap favicon URL to HTTPS.

This avoids mixed content browser error/warning when YaCy is browsed
using HTTPS.
Changed Files: htroot/yacysearch.html
Tue Nov 08 02:39:45 CET 2016
by luccioman
Improved absolute URLs rendering in OpenSearch desc and RSS feeds.

When the peer is behind a reverse proxy providing SSL/TLS encryption,
the rendered absolute URLs should start with https when the user browser
requested https : added limited support to the X-Forwarded-Proto HTTP
header notably provided on Heroku platform.
Also added some unit tests.
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java, test/java/net/yacy/http/servlets/YaCyDefaultServletTest.java
Tue Nov 08 02:31:38 CET 2016
by luccioman
Removed a use of the deprecated Seed.getIP()
Changed Files: htroot/ConfigSearchBox.html, htroot/ConfigSearchBox.java
Sun Nov 06 21:02:58 CET 2016
by reger
upd Column.toString to output encoder "{bytes}"
used for String and binary Column types
Changed Files: source/net/yacy/kelondro/index/Column.java
Sun Nov 06 03:34:24 CET 2016
by reger
fix Column parsing from celldefinition string, without cellwidth def.
(outofbound exception)
Changed Files: source/net/yacy/kelondro/index/Column.java, test/java/net/yacy/kelondro/index/ColumnTest.java
Sat Nov 05 19:40:14 CET 2016
by reger
use human readable date format in CrawlStacker error message 
"double in: local index, oldDate = "
Changed Files: source/net/yacy/crawler/CrawlStacker.java
Fri Nov 04 14:34:52 CET 2016
by luccioman
Use https URLs to load external js scripts in yacysearch_location.html.

Otherwise when https is enabled on the peer, the browser block active
mixed content from these external sites.
Changed Files: htroot/yacysearch_location.html
Fri Nov 04 14:31:22 CET 2016
by luccioman
Fixed errors for Search portal mode or when peer is not reachable.

Same case as reported on issue #87.
Changed Files: htroot/Bookmarks.java, htroot/Bookmarks.rss, htroot/Table_YMark_p.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/get_bookmarks.xml, htroot/api/bookmarks/get_folders.java, htroot/yacysearch_location.java
Fri Nov 04 09:35:03 CET 2016
by luccioman
Used relative URLs in ConfigPortal preview frames.

This is more reliable when YaCy is behind a reverse proxy.

Also updated integration examples to keep the current protocol part
(http or https) in the example address.
Changed Files: htroot/ConfigPortal.html, htroot/ConfigPortal.java
Wed Nov 02 03:43:39 CET 2016
by reger
harmonize used lastmodified date for rwi and fulltext in storeDocument
Changed Files: source/net/yacy/document/Document.java, source/net/yacy/search/index/Segment.java
Tue Nov 01 22:33:20 CET 2016
by reger
include navigator-plugin output data in json and rss/xml output
fix encoding of url for rss
fix unresolved-pattern of url in json & xml for domain navigator
Changed Files: htroot/yacysearchtrailer.java, htroot/yacysearchtrailer.json, htroot/yacysearchtrailer.xml
Tue Nov 01 04:38:47 CET 2016
by reger
refactor namespace navigator as part of navigatorplugin map, this allows
the navigator to include counts all matches (rwi+fulltext).
Fixing also unresolved_pattern in navigators title (of the counter)
The use of inurl: query modifier as filter has not been changed keeping
it as soft (unsharp) filter facet.

Upd StringNavigator to prevent empty string form multivalued solr fields,
removed date value conversion (better handled elsewhere, not need here).
Changed Files: htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java, source/net/yacy/search/navigator/NameSpaceNavigator.java, source/net/yacy/search/navigator/StringNavigator.java, source/net/yacy/search/query/SearchEvent.java
Tue Nov 01 02:19:26 CET 2016
by reger
upd IndexControlRWIs servlet, url list table remove unused word distance
column (table lists always refs for one word).
upd master.lng with recent text changes
Changed Files: htroot/IndexControlRWIs_p.html, htroot/IndexControlRWIs_p.java, locales/master.lng.xlf
Mon Oct 31 18:42:23 CET 2016
by reger
Make navigators underlaying indexfield name accessible in interface
use interface in declaration and extend facet check to include navigator
field.
Changed Files: htroot/yacysearchtrailer.java, source/net/yacy/search/navigator/Navigator.java, source/net/yacy/search/navigator/StringNavigator.java, source/net/yacy/search/query/SearchEvent.java
Mon Oct 31 00:43:01 CET 2016
by reger
improve query modifier parsing of "collection:" and possible collision
with "on:" in case multiple collection modifier were entered (by mistake)
http://mantis.tokeek.de/view.php?id=702
Changed Files: source/net/yacy/search/query/QueryModifier.java
Sun Oct 30 05:19:06 CET 2016
by reger
correct Collection navigatior - search servlet modifier parameter
(navigator entries are single collection names, spaces are removed by crawlstart)
preparation: for abstraction of navi's


Changed Files: htroot/yacysearchtrailer.java
Sat Oct 29 04:19:46 CEST 2016
by reger
Reduce number of default max. search navigator lines (from 10000)
to 100 + make it configurable
Changed Files: defaults/yacy.init, htroot/ConfigSearchPage_p.html, htroot/ConfigSearchPage_p.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java, source/net/yacy/search/query/QueryParams.java, test/java/net/yacy/kelondro/rwi/ReferenceContainerTest.java
Fri Oct 28 07:13:21 CEST 2016
by Sudheesh Singanamalla
Merge pull request #88 from sudheesh001/Patch16

Fixes #16 Updates documentation about cloning and build from source
Changed Files: README.md
Wed Oct 12 16:22:15 CEST 2016
by sudheesh001
Fixes #16 Updates documentation about cloning and build from source
Changed Files: README.md
Wed Oct 26 01:38:40 CEST 2016
by reger
add simple test case for ReferenceContainer helpful for debugging 
calculated ranking parameter
Changed Files: source/net/yacy/kelondro/data/word/WordReferenceRow.java, test/java/net/yacy/kelondro/rwi/ReferenceContainerTest.java
Tue Oct 25 03:02:31 CEST 2016
by reger
reduce the mix of absolute and relative internal html page links 
(prefer relative for same pg or neighbors) to ease proxied access
e.g. http://mantis.tokeek.de/view.php?id=701
Changed Files: htroot/Collage.html, htroot/CrawlProfileEditor_p.html, htroot/Crawler_p.html, htroot/Network.html, htroot/Status.html, htroot/WatchWebStructure_p.html, htroot/env/templates/header.template, htroot/env/templates/simpleheader.template
Sat Oct 22 08:23:48 CEST 2016
by luccioman
Crawl from local file : faster task end when manually terminating crawl.
Changed Files: htroot/yacysearchitem.java, source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/crawler/FileCrawlStarterTask.java, source/net/yacy/crawler/IllegalCrawlProfileException.java
Sat Oct 22 08:23:48 CEST 2016
by luccioman
Fixed NullPointerException case.

As reported by @reger24 , search in Intranet mode was failing due to
this error.
Changed Files: htroot/yacysearchitem.java
Sat Oct 22 00:53:47 CEST 2016
by reger
catch rwi ranking div by zero exception
during rwi search result processing worddistance calculation is effected 
by concurrent update (normalization) of min/max ranking parameter for
wordpositions. On update of min/max the exception is raised in distance calc
and now catched. 
This concurrent update and change of ranking results is needed for speed
but should be further checked for optimization
Changed Files: source/net/yacy/kelondro/rwi/AbstractReference.java
Fri Oct 21 12:48:36 CEST 2016
by luccioman
Updated javadocs : warning about closing stream responsibility.
Changed Files: source/net/yacy/kelondro/util/FileUtils.java
Thu Oct 20 12:12:26 CEST 2016
by luccioman
Fixed redirected URLs processing as crawl start point.

See mantis 699 (http://mantis.tokeek.de/view.php?id=699) for details.
Changed Files: source/net/yacy/crawler/retrieval/HTTPLoader.java
Wed Oct 19 22:55:36 CEST 2016
by reger
div by null in word distance calculation
(again, description in http://mantis.tokeek.de/view.php?id=698)
as root cause was not seen, added just workaround reducing in favour over a 
try catch (for easier followup).
Changed Files: source/net/yacy/kelondro/rwi/AbstractReference.java
Wed Oct 19 14:56:25 CEST 2016
by luccioman
Removed mentions of deprecated LURL-db.

Thanks to LA_FORGE asking about if on YaCy forum (
http://forum.yacy-websuche.de/viewtopic.php?f=5&t=5895 )
Changed Files: htroot/yacy/urls.java, source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/search/index/Segment.java
Wed Oct 19 13:14:20 CEST 2016
by luccioman
Added more examples in Blacklist JUnit test.
Changed Files: test/java/net/yacy/repository/BlacklistTest.java
Mon Oct 17 23:58:28 CEST 2016
by reger
fix min/max calculation of WordReferenceVars.distance() 
Issue was the calculation in AbstractReference with positions.clear() call,
this made distance result always 0 (distance needs min 2 positions) and created concurrency issues.
+ unit test of changes
Changed Files: source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/rwi/AbstractReference.java, test/java/net/yacy/kelondro/data/word/WordReferenceVarsTest.java
Mon Oct 17 22:32:19 CEST 2016
by luccioman
Added fine log level for too long blacklist matching processing. 
Changed Files: source/net/yacy/repository/Blacklist.java
Sun Oct 16 06:31:48 CEST 2016
by reger
adjust ConcurrentScoreMap internal value map to interface and use parameter
Long -> Integer (saves some bytes)
Changed Files: source/net/yacy/cora/sorting/ConcurrentScoreMap.java
Sat Oct 15 19:29:19 CEST 2016
by reger
remove obsolete query variable
leftover from https://github.com/yacy/yacy_search_server/commit/8fb370d9f899a83908e029336690cc6f5bb2059a#diff-1d4259005ebfddc11083387857a86175
harmonize ranking shift parameter to 0xFF
correct addresult weight parameter to long
Changed Files: source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java
Sat Oct 15 11:34:50 CEST 2016
by luccioman
Updated license header file name
Changed Files: test/net/yacy/data/URLLicenseConcurrentTest.java
Sat Oct 15 11:34:23 CEST 2016
by luccioman
Fixed maven compilation error

Removed unit test yacysearchitemTest from default maven Junit tests
path, as yacysearchitem class is not in maven build classpath.
Changed Files: test/yacysearchitemTest.java
Fri Oct 14 14:05:38 CEST 2016
by luccioman
Applied image headers customization to the new ViewFavicon servlet.
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Fri Oct 14 02:36:35 CEST 2016
by reger
add lock symbol to messages UI Trans menu item
Changed Files: htroot/env/templates/submenuComputation.template
Fri Oct 14 01:17:34 CEST 2016
by reger
Avoid frequent data type casting (float/long) for rwi score
refactor to using long in URIMetadataNode too (and related call parameters)
As remote rwi score's are not used (since v1.83) skip reading float-score ,
but keep in toString() for communication with older versions.
Changed Files: source/net/yacy/cora/federate/solr/responsewriter/EnhancedXMLResponseWriter.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/search/index/Fulltext.java
Thu Oct 13 15:39:47 CEST 2016
by luccioman
Added some unit tests for Blacklist.isListed()
Changed Files: test/java/net/yacy/repository/BlacklistTest.java
Thu Oct 13 15:33:31 CEST 2016
by luccioman
Blacklist refactoring : extracted a function for easier unit testing
Changed Files: source/net/yacy/repository/Blacklist.java
Wed Oct 12 11:26:59 CEST 2016
by luccioman
CrawlCheck_p.html : also display info about disallowed URLs.
Changed Files: source/net/yacy/crawler/robots/RobotsTxt.java
Wed Oct 12 11:25:36 CEST 2016
by luccioman
Robots : properly handle URLs including non ASCII characters

This fixes GitHub issue 80 (
https://github.com/yacy/yacy_search_server/issues/80 ) reported by
Lord-Protector.
Changed Files: source/net/yacy/crawler/robots/RobotsTxtEntry.java
Wed Oct 12 09:31:42 CEST 2016
by luccioman
Refactoring : extracted a method to handle authorized action links.
Changed Files: htroot/yacysearchitem.java
Wed Oct 12 08:45:32 CEST 2016
by luccioman
Refactored search result action links construction.

These are long URLS with common parts : it is valuable to build the
common parts only one time.
Changed Files: htroot/yacysearchitem.java
Tue Oct 11 19:54:50 CEST 2016
by reger
prevent negative score on URIMetadataNode - in the special case were no 
solr score is supplied.
+ assert before use & test case
Changed Files: source/net/yacy/search/ranking/ReferenceOrder.java, test/java/net/yacy/search/ranking/ReferenceOrderTest.java
Mon Oct 10 01:10:35 CEST 2016
by luccioman
Fixed loadFromCache with strategy IFFRESH.

This fixes mantis 695 ( http://mantis.tokeek.de/view.php?id=695 ) :
crawl start with 'Link-List of URL' option on websites using cookies.
Changed Files: source/net/yacy/repository/LoaderDispatcher.java
Fri Oct 07 23:49:03 CEST 2016
by reger
remove module for swfparser from maven parent pom
not longer required for the build
see https://github.com/yacy/yacy_search_server/commit/a4465c97d6b222253f34bc2b4e4d4ad276f645a3
Changed Files: libbuild/pom.xml
Fri Oct 07 10:57:31 CEST 2016
by luccioman
Removed invalid empty form action.

A form action URL must not be empty (see
https://www.w3.org/TR/html/sec-forms.html#element-attrdef-form-action ).
No action attribute has the same effect (relaunching the same GET
action) but is valid HTML.
Changed Files: htroot/yacysearch.html
Fri Oct 07 10:46:20 CEST 2016
by luccioman
Removed xmlns attribute which no more makes sense in HTML5 pages.
Changed Files: htroot/ConfigSearchPage_p.html, htroot/Crawler_p.html, htroot/index.html, htroot/yacysearch.html
Fri Oct 07 10:44:45 CEST 2016
by luccioman
Added some accessibility improvements to the main interface.

Tested with NVDA screen reader.
Changed Files: htroot/ConfigSearchPage_p.html, htroot/env/templates/header.template, htroot/env/templates/simpleheader.template, htroot/index.html
Fri Oct 07 10:42:23 CEST 2016
by luccioman
Switched to the short HTML Doctype

This pages were already no more XHTML 1.0 because made use of the HTML5
syntax and elements.
Applied current (2016) HTML standard recommended Doctype declaration
(see https://www.w3.org/TR/html/syntax.html#the-doctype ).
Changed Files: htroot/index.html, htroot/yacysearch.html
Fri Oct 07 03:03:41 CEST 2016
by reger
correct blacklist export as text url to blacklists_p.txt
was using servlet for network access and missing network.unit.name
fix for http://mantis.tokeek.de/view.php?id=694
+ prevent unresoved_pattern in yacy/list servlet
Changed Files: htroot/BlacklistImpExp_p.html, htroot/api/blacklists_p.txt, htroot/yacy/list.java
Thu Oct 06 23:37:12 CEST 2016
by reger
adjust date in text detection to ignore some program version strings
like "3.1.2.0102" see http://mantis.tokeek.de/view.php?id=650
+ expand test case
Changed Files: source/net/yacy/document/DateDetection.java, test/java/net/yacy/document/DateDetectionTest.java
Thu Oct 06 19:03:52 CEST 2016
by reger
optimize condenser language detection a little.
langdetect probabilities take letter case into account, add words from
description and anchors etc. as is.
+ add it to javadoc 
Changed Files: source/net/yacy/document/Condenser.java, source/net/yacy/document/Tokenizer.java, source/net/yacy/document/language/Identificator.java, test/java/net/yacy/document/TokenizerTest.java
Thu Oct 06 03:41:07 CEST 2016
by reger
adjust Tokenizer sentence count to ignore repeated punktuation (like !!!! )
+ remove unused sentenceword map (we use only the count)
+ upd test case for sentence count
Changed Files: source/net/yacy/document/Tokenizer.java, test/java/net/yacy/document/TokenizerTest.java
Wed Oct 05 13:48:22 CEST 2016
by luccioman
Removed unnecessary crawlingDomFilterDepth hidden field.

It had incorrect "-UNRESOLVED_PATTERN-" value (see  second part of
mantis 691 http://mantis.tokeek.de/view.php?id=691 )

Note : crawlingDomFilterDepth is apparently unused in current (2016)
YaCy code-base. It was also unnecessary because crawlingDomFilterCheck
hidden field is set to "off".
Changed Files: htroot/CrawlStartSite.html
Wed Oct 05 12:20:37 CEST 2016
by luccioman
Fixed Recorded action URLs beginning displayed in /Table_API_p.html

Removed scheme, host and port from URL to avoid dealing with http/https,
external host and port retrieving issues.

What's more, this is consistent with how URL are displayed in
/Tables_p.html?table=api&count=100&reverse=on&search= or
Tables_p.xml?table=api&count=100&search=

This fixes mantis 691 first part
(http://mantis.tokeek.de/view.php?id=691)
Changed Files: htroot/Table_API_p.java
Wed Oct 05 05:52:37 CEST 2016
by reger
adjust Tokenizer sentence count on trailing text after last recognized sentence
+ upd test case for rwi multi-word-query  (leaving results known to fail untested)
Changed Files: source/net/yacy/document/Tokenizer.java, test/java/net/yacy/search/index/SegmentTest.java
Tue Oct 04 05:06:18 CEST 2016
by luccioman
Merge pull request #76 from luccioman/crawler

Crawl monitoring : refresh running crawls table
Changed Files: htroot/Crawler_p.html, htroot/js/Crawler.js
Tue Oct 04 03:55:49 CEST 2016
by luccioman
Crawl monitoring : refresh running crawls table

Fix mantis 690 ( http://mantis.tokeek.de/view.php?id=690 ). 
Tested on :
- MS Windows 10 : Edge, Firefox 49, Chrome 53
- Debian Jessie : Firefox ESR 45
Changed Files: htroot/Crawler_p.html, htroot/js/Crawler.js
Mon Oct 03 21:57:02 CEST 2016
by luccioman
Switched to the short HTML Doctype

This page was already no more XHTML 1.0 as it makes use of the HTML5
<progress> element.
Applied current HTML standard recommended Doctype declaration (see
https://www.w3.org/TR/html/syntax.html#the-doctype ).
Changed Files: htroot/Crawler_p.html
Mon Oct 03 21:55:38 CEST 2016
by luccioman
Tables accessibility : added missing <thead> sections.
Changed Files: htroot/Crawler_p.html
Mon Oct 03 21:52:24 CEST 2016
by luccioman
Restricted variables scope to function handleStatus() in Crawler.js

Missing 'var' in declaration was unnecessarily giving global scope to
these variables.
Changed Files: htroot/js/Crawler.js
Tue Oct 04 03:05:26 CEST 2016
by reger
upd maven dependency-check plugin to reflect changes of https://nvd.nist.gov
+ upd unknown ant script with current lib/jsch version
Changed Files: pom.xml, source/net/yacy/peers/operation/yacySeedUploadScp.xml
Tue Oct 04 01:42:18 CEST 2016
by reger
adjust rwi index result word position handling used for rwi ranking
- correct WordReferenceVars.toRowEntry posintext parameter
to set expected min posintext (the difference is on multi-word queries,
while positions are ordered by search word order).
- modified posofphrase/posinphrase join operation
 - to set min posofphrase
 - and keep posinphrase if not same posofphrase (was set to 0, no differentiation during ranking)
+ fix compiler msg (missing type declaration)
Changed Files: source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java
Mon Oct 03 01:52:51 CEST 2016
by reger
add support for older Word versions (Word6/Word95) to docParser
Changed Files: source/net/yacy/document/parser/docParser.java
Mon Oct 03 01:48:35 CEST 2016
by reger
upd to poi-3.15.jar
Changed Files: .classpath, build.xml, lib/poi-3.15.License, lib/poi-3.15.jar, lib/poi-scratchpad-3.15.jar, pom.xml
Sun Oct 02 06:36:34 CEST 2016
by reger
mention date search parameter in search option help (index.html)
Changed Files: htroot/index.html, htroot/index.java, locales/master.lng.xlf
Sun Oct 02 03:19:12 CEST 2016
by reger
generalize DateDetection with holiday date rules readily available in icu
to make sure current dates are recognized (was fixed to 2014 - 2016)
+ adjust holiday date parser from pattern.match to pattern.find to deal with leading and trailing text
+ moved relative date recognition (morgen, tomorrow) to parseline (used by query parser only), as not working and problematic for indexing
+ add test case for parseline (used by query parser)
Changed Files: source/net/yacy/document/DateDetection.java, test/java/net/yacy/document/DateDetectionTest.java
Sat Oct 01 03:16:27 CEST 2016
by reger
correct DateDetection Silvester date
add Thanksgiving
Changed Files: source/net/yacy/document/DateDetection.java
Fri Sep 30 21:46:57 CEST 2016
by reger
have RSSFeed.getChannel return empty message on missing channel element,
a) required b) prevent NPE in rss servlets
+ add test
Changed Files: source/net/yacy/cora/document/feed/RSSFeed.java, source/net/yacy/cora/document/feed/RSSMessage.java, test/java/net/yacy/cora/document/feed/RSSFeedTest.java
Fri Sep 30 21:42:42 CEST 2016
by reger
del double entry in master.lng
Changed Files: locales/master.lng.xlf
Fri Sep 30 17:12:55 CEST 2016
by luccioman
Added some javadocs.
Changed Files: source/net/yacy/search/snippet/TextSnippet.java
Fri Sep 30 17:12:29 CEST 2016
by luccioman
Made use of the constant corresponding to the hard-coded value.
Changed Files: test/java/net/yacy/search/snippet/TextSnippetTest.java
Fri Sep 30 10:54:53 CEST 2016
by luccioman
Fixed some HTML validation errors : Illegal character in query

Now encode space characters in URLs query part.
Changed Files: htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearchtrailer.java, source/net/yacy/search/query/QueryParams.java
Fri Sep 30 09:43:51 CEST 2016
by luccioman
Removed invalid rel="[count]" from links in tagcloud.

These are no valid link relationships, and do not appear to be used in
scripting or styling. 
If necessary, a valid alternative could be to add an attribute such as
data-count="[count]"
Changed Files: htroot/ConfigSearchPage_p.html, htroot/yacysearchtrailer.html
Thu Sep 29 23:59:10 CEST 2016
by reger
display number of documents/rss-items for label "docs" in load_rss_p servlet
(as replacement for the rarely used "docs" rss-tag for a url to the rss-specification)
Changed Files: htroot/Load_RSS_p.java, source/net/yacy/cora/document/feed/Hit.java, source/net/yacy/cora/document/feed/RSSMessage.java
Thu Sep 29 13:42:07 CEST 2016
by luccioman
Removed unnecessary ARIA "form" role on native HTML form elements.

This fixes warnings reported by W3C Nu Html Checker
(https://validator.w3.org/nu/).
Changed Files: htroot/Crawler_p.html, htroot/env/templates/header.template, htroot/env/templates/simpleheader.template, htroot/test.html
Thu Sep 29 10:33:11 CEST 2016
by luccioman
Fixed shutdown for crawler.MaxActiveThreads value greater than 200

Shutdown was hanging in CrawlQueues.close() at
this.workerQueue.put(POISON_REQUEST) when config value
crawler.MaxActiveThreads was greater than 200.

Revealed by "Collision" Threads dumps in mantis 689
(http://mantis.tokeek.de/view.php?id=689#c1312)

Fixed consistency between this.worker.length and this.workerQueue
capacity, and made the process more reliable using non-blocking offer()
function.
Changed Files: source/net/yacy/crawler/data/CrawlQueues.java
Wed Sep 28 14:52:30 CEST 2016
by luccioman
Prevent yacy main thread from hanging on browser opening process.

First fix for mantis 689 (http://mantis.tokeek.de/view.php?id=689).

On Debian Linux, with a headless jre and no open browser,
browser.openBrowserClassic() was called and waited forever the browser
process end (p.waitFor()). YaCy shutdown was therefore not working until
the browser was closed.

Also modified browser opening command for Unix platform to open the
default the browser (with xdg-open util) instead of Firefox.

xdg-open also has the advantage to be asynchronous (not blocking).
Changed Files: source/net/yacy/gui/framework/Browser.java, source/net/yacy/yacy.java
Tue Sep 27 23:12:10 CEST 2016
by reger
upd to pdfbox-2.0.3
Changed Files: .classpath, build.xml, lib/fontbox-2.0.3.License, lib/fontbox-2.0.3.jar, lib/pdfbox-2.0.3.License, lib/pdfbox-2.0.3.jar, pom.xml
Tue Sep 27 22:40:36 CEST 2016
by reger
prevent StringIndexOutOfBounds in getLocalFile()
+ tighten patching of DOS path w/o protocol to drive "LETTER":
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Tue Sep 27 07:53:08 CEST 2016
by luccioman
Avoid duplication of various MS Windows file URLs flavors

Fix for mantis 692 (http://mantis.tokeek.de/view.php?id=692)
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java, test/java/net/yacy/cora/document/id/DigestURLTest.java
Tue Sep 27 07:41:45 CEST 2016
by luccioman
Removed unused import
Changed Files: source/net/yacy/cora/document/id/DigestURL.java
Mon Sep 26 02:54:58 CEST 2016
by reger
remove unused rootpattern, leftover from commit
https://github.com/yacy/yacy_search_server/commit/9a5ab4e2c141ac9ef9f45c6524f2352ab7c3789c#diff-d2b184283abed53ae260fc9eabdaef40
Changed Files: source/net/yacy/cora/document/id/DigestURL.java
Sun Sep 25 22:08:12 CEST 2016
by reger
improve url hash computation for file path with mixed java & windows
file.separator to compute equal hashes (by normalizing path for computation)
+ expand test case for to check mixed java / windows file url notation
like e.g. file:///c:/test/file.html vs. file:///c:\test/file.html
- relates partially to http://mantis.tokeek.de/view.php?id=692
Changed Files: source/net/yacy/cora/document/id/DigestURL.java, test/java/net/yacy/cora/document/id/DigestURLTest.java
Sat Sep 24 05:02:37 CEST 2016
by reger
add missing text for Supporter.html to master.lng
Changed Files: locales/master.lng.xlf
Fri Sep 23 23:37:12 CEST 2016
by reger
fix supported mime XML -> xml for rssParser (mime normalized to lower case for comparison)
+ add mime text/xml as in use for rss in the wild
Changed Files: source/net/yacy/document/parser/rssParser.java
Fri Sep 23 13:27:46 CEST 2016
by luccioman
Added alternative text and title to HostBrowser.html image links

For better accessibility
Changed Files: htroot/ConfigSearchPage_p.html, htroot/yacysearchitem.html
Thu Sep 22 11:28:33 CEST 2016
by luccioman
Updated URL fragment validation rules according to current standards 

See RFC 3986 (https://tools.ietf.org/html/rfc3986) or URL living
standard (https://url.spec.whatwg.org/)
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Thu Sep 22 09:07:07 CEST 2016
by luccioman
Improved ErrorCache behavior when switching networks

Even after network switch, ErroCache was still holding a reference to
the previous Solr cores, thus becoming useless until next YaCy restart.

Initial error cache filling with recent errors from the index was also
missing after the swtich.
Changed Files: source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/search/index/ErrorCache.java, source/net/yacy/search/index/ErrorCacheFiller.java
Thu Sep 22 08:21:14 CEST 2016
by luccioman
Added some JavaDoc and moved crawlStacker close at the right place.
Changed Files: source/net/yacy/search/Switchboard.java
Thu Sep 22 08:20:01 CEST 2016
by luccioman
Log eventual Solr instances close errors.

We do not want to block on this kind of error, but this should not
silently fail as it may have later consequences.
Changed Files: source/net/yacy/search/index/Fulltext.java
Thu Sep 22 00:16:22 CEST 2016
by reger
fix for solr write.lock after mode change http://mantis.tokeek.de/view.php?id=686
The embedded core holds a lock on the index and must be closed. Earlier commit
comment states that core should be closed with solr instance instead on close 
of connector.
Adjusted the InstanceMirror.close() to take care of closing the embedded 
instance to release the lock.
In 2 routines of fulltext this was already explicite implemented (disconnectLocalSolr).
Now this disconnect is part of the InstanceMirror.close().
Changed Files: source/net/yacy/cora/federate/solr/instance/InstanceMirror.java, source/net/yacy/search/index/Fulltext.java, test/java/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnectorTest.java
Wed Sep 21 21:08:21 CEST 2016
by reger
add test case for EmeddedSolrConnector close()
for issue http://mantis.tokeek.de/view.php?id=686
(without solving the issue here)
Changed Files: test/java/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnectorTest.java
Wed Sep 21 01:30:58 CEST 2016
by reger
fix master.lng entry with targetlang 
+ add text to api/citation.html
Changed Files: locales/master.lng.xlf
Tue Sep 20 02:44:28 CEST 2016
by luccioman
Fixed undefined error case in sorttable.js 

Occured when a table with class="sortable" has data cells with colspan
attribute greater than 1
Changed Files: htroot/js/sorttable.js
Tue Sep 20 01:55:56 CEST 2016
by reger
test case: for NewsPool to check the id modificator (for unique id) 
and observe the distribution order .. hands on.
+ add test/DATA to gitignor
Changed Files: .gitignore, source/net/yacy/peers/NewsPool.java, test/java/net/yacy/peers/NewsPoolTest.java
Tue Sep 20 00:17:42 CEST 2016
by luccioman
Removed unused JavaScript file docs.min.js

This file is used by Bootstrap documentation website
(http://getbootstrap.com/) but is not part of the Bootstrap distribution
and has not be included in a Bootstrap based application.
Changed Files: htroot/env/templates/metas.template, htroot/jslicense.html, htroot/test.html, source/net/yacy/cora/federate/solr/responsewriter/HTMLResponseWriter.java
Mon Sep 19 11:01:39 CEST 2016
by Michael Peter Christen
enable fuzzy search, solr style (append a ~ to get a fuzzyness on the
word)
Changed Files: source/net/yacy/search/query/QueryGoal.java
Sun Sep 18 00:59:27 CEST 2016
by reger
test case: simulating multi word query for local rwi index
Purpose of the test case is to be able to (controlled) analyse the rwi ranking for
multi word searches (with focus on posintext and word-distance ranking)
Changed Files: source/net/yacy/kelondro/data/word/WordReferenceRow.java, test/java/net/yacy/search/index/SegmentTest.java
Thu Sep 15 23:26:25 CEST 2016
by reger
avoid NPE on hello message with missing "yourip" key
http://mantis.tokeek.de/view.php?id=684
Changed Files: source/net/yacy/peers/Protocol.java
Thu Sep 15 02:59:32 CEST 2016
by reger
hack to generate a unique message-id for messages created in the same second
by optionally add a 1 second offset counter to the current time (which is
used as the unique id part)
Changed Files: htroot/TransNews_p.java, source/net/yacy/peers/NewsDB.java
Wed Sep 14 16:32:57 CEST 2016
by Michael Peter Christen
removed version number check because it does not work any more if
version numbers are expressed in a different way as we expect. That
could cause that YaCy does not run on systems which are appropriate but
we simply do not understand the version string.
Changed Files: source/net/yacy/yacy.java
Wed Sep 14 09:31:45 CEST 2016
by luccioman
Updated compiling section in Readme
Changed Files: README.md
Wed Sep 14 00:36:19 CEST 2016
by reger
add some javadoc to rwi wordreference distance, position
to remember facts for http://mantis.tokeek.de/view.php?id=683
Init missing word position to 0 like in other non text body words
Changed Files: source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/rwi/AbstractReference.java, source/net/yacy/kelondro/rwi/Reference.java
Tue Sep 13 02:47:36 CEST 2016
by reger
as requested, disable/remove old swf parser
http://forum.yacy-websuche.de/viewtopic.php?f=8&t=5861#p33098
Changed Files: .classpath, build.xml, pom.xml, source/net/yacy/document/TextParser.java
Sun Sep 11 23:25:44 CEST 2016
by reger
prepare a IndexSegment test case for RWI index testing
+ prevent NPE in Segment.clear() on missing embedded solr instance.
Changed Files: source/net/yacy/search/index/Fulltext.java, test/java/net/yacy/search/index/SegmentTest.java
Sat Sep 10 18:23:09 CEST 2016
by reger
remove not needed counter in Tokeninzer (completing last changes)
including a small change, word posintext counting. 
We remember/store 1st posintext. Previously following words got a handle (posintext)
excluding found. Now it just counts and assigns true posintext as handle (posintext)
Changed Files: source/net/yacy/document/Tokenizer.java, source/net/yacy/kelondro/data/word/Word.java
Sat Sep 10 09:33:27 CEST 2016
by luccioman
Fixed remaining OpenSearch link name consistency issues

Hopefully the last complementary fix related to mantis 679
((http://mantis.tokeek.de/view.php?id=679)
Changed Files: htroot/compare_yacy.html, htroot/yacysearch.html, locales/fr.lng
Fri Sep 09 22:42:21 CEST 2016
by reger
Add title attribute to Translator_p.html edit fields
(to display complete text hint)
Thank's @luc
Changed Files: htroot/Translator_p.html
Fri Sep 09 09:00:16 CEST 2016
by luccioman
Updated french translation for some admin pages.
Changed Files: locales/fr.lng
Fri Sep 09 07:43:33 CEST 2016
by luccioman
Fixed french messages encoding in YaCy tray.

Also added the missing french translations. 
Changed Files: source/net/yacy/gui/Tray.java
Thu Sep 08 17:31:28 CEST 2016
by luccioman
OpenSearch short name : made it generic (non internationalized)

This ensure consistency between the index link and the
opensearchdescription, even when switching language after having added
your YaCy peer to the browser engines list.
Changed Files: htroot/index.html, htroot/opensearchdescription.xml, locales/fr.lng
Thu Sep 08 02:29:48 CEST 2016
by reger
adjust the WordReference.join wordsintext calc to take the max (instead of sum)
The reference is for the same url (add same for title and phrases).
+ del redundant join() procedure
Changed Files: source/net/yacy/kelondro/data/word/WordReferenceVars.java
Wed Sep 07 10:07:03 CEST 2016
by luccioman
Merge pull request #71 from luccioman/postprocessing

Postprocessing refactoring + fixed a NPE introduced in the master branch.
Changed Files: source/net/yacy/cora/federate/solr/responsewriter/OpensearchResponseWriter.java, source/net/yacy/search/schema/CollectionConfiguration.java
Wed Sep 07 02:16:16 CEST 2016
by reger
reactivate sentence counter in WordTokenizer for phrasepos ranking,
by counting punktuation (delivered as 1 char word) again.
Changed Files: source/net/yacy/document/Tokenizer.java, source/net/yacy/document/WordTokenizer.java, test/java/net/yacy/document/WordTokenizerTest.java
Tue Sep 06 18:45:29 CEST 2016
by Michael Peter Christen
enhanced json encoding of strings
Changed Files: htroot/yacy/seedlist.java, source/net/yacy/cora/federate/solr/responsewriter/YJsonResponseWriter.java, source/net/yacy/server/serverObjects.java
Tue Sep 06 06:56:51 CEST 2016
by Michael Peter Christen
added default fl to solr query, removed large texts retrieval and
changed snippet to description tag if no other description is available
Changed Files: source/net/yacy/cora/federate/solr/responsewriter/YJsonResponseWriter.java, source/net/yacy/http/servlets/SolrSelectServlet.java
Tue Sep 06 03:18:02 CEST 2016
by reger
implemented RWI WordReference to return the word position value (was always left empty)
This is needed and enables existing word position ranking for RWI.
The upcoming concurrency issue in word position min/max calculation were eliminated
by iterator.hasHext check before next() access.
Changed Files: source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/rwi/AbstractReference.java
Tue Sep 06 00:08:18 CEST 2016
by luccioman
Merge pull request #65 from luccioman/docker

Docker : enable HTTPS support
Changed Files: docker/Dockerfile, docker/Dockerfile.alpine, docker/Readme.md, docker/docker-cloud.yml
Tue Sep 06 00:05:59 CEST 2016
by reger
fix posInText ranking calculation to score 0 on no position info
+ fix Word posInText calc in Tokenizer to start with 1
+ test case
Changed Files: source/net/yacy/document/Tokenizer.java, source/net/yacy/search/ranking/ReferenceOrder.java, test/java/net/yacy/document/TokenizerTest.java
Mon Sep 05 23:36:25 CEST 2016
by luccioman
Merge pull request #72 from luccioman/dist_macOS

YaCy Mac OS bundle : store data in the right place
Changed Files: addon/YaCy.app/Contents/Info.plist, addon/YaCy.app/Contents/MacOS/startYACYMacOS.sh, build.xml, source/net/yacy/server/serverClassLoader.java, source/net/yacy/yacy.java, startYACY.sh
Mon Sep 05 00:07:01 CEST 2016
by reger
adjust the getTopics() and getTopicNavigator() to current useage
- move the maxcount limit restriction completely to getTopicNavigator (as there not used in getTopics)
- let search servlet use getTopics by default (w/o RWI connected check, as of now, Topics are available w/o any additional index interaction)
Changed Files: htroot/yacy/search.java, source/net/yacy/search/query/SearchEvent.java
Sun Sep 04 22:18:07 CEST 2016
by reger
fix ConcurrentScoreMap.set() calculation of totalCount()
+ test case 
Changed Files: source/net/yacy/cora/sorting/ConcurrentScoreMap.java, test/java/net/yacy/cora/sorting/ConcurrentScoreMapTest.java
Sun Sep 04 06:42:48 CEST 2016
by reger
log a error on aborted news publish (due to duplicate news.id)
+ change printed err msg to log entry in PeerAction.processPeerArrival
Changed Files: htroot/TransNews_p.java, source/net/yacy/peers/NewsPool.java, source/net/yacy/peers/PeerActions.java
Sun Sep 04 03:05:25 CEST 2016
by reger
fix fr.lng (missing quotes)
broke sentence appart to reduce inclusion of coding tags in translation
Changed Files: locales/fr.lng, locales/master.lng.xlf
Sun Sep 04 02:29:04 CEST 2016
by reger
include translation news service in status submenue
+ display translation proposal news only for current language (in TransNews servlet)
Changed Files: htroot/News.html, htroot/TransNews_p.html, htroot/TransNews_p.java, htroot/env/templates/submenuComputation.template, locales/master.lng.xlf
Sun Sep 04 01:00:28 CEST 2016
by reger
TranslationNews: take out limitation to send only one text per translated file
(to avoid need of repeated publish button hits)
Changed Files: htroot/TransNews_p.java
Sun Sep 04 00:09:45 CEST 2016
by reger
reactivate the use of included-in-topwords boost in postRanking
+ changed the postRanking to add one score only if word appears more as one time.
+ getTopics() unused code block rem'd (save performace)-> routine needs rework !
Changed Files: source/net/yacy/search/query/SearchEvent.java
Sat Sep 03 15:22:57 CEST 2016
by luccioman
Merged master to dist_macOS branch.
Changed Files: source/net/yacy/search/query/SearchEvent.java
Sat Sep 03 15:21:02 CEST 2016
by luccioman
Fixed serverClassLoader.findClass method

htroot is a supposed to be a subfolder of appPath and not of dataPath,
as assumed in other places where htroot is loaded. This issue was not
visible when dataPath and appPath are equals.
Changed Files: source/net/yacy/server/serverClassLoader.java
Sat Sep 03 14:46:58 CEST 2016
by luccioman
Fixed options processing for Mac OS

- getopt is BSD style and does not support long options
- fixed typing error inparameter value extracting for all platforms
Changed Files: addon/YaCy.app/Contents/MacOS/startYACYMacOS.sh, startYACY.sh
Sat Sep 03 03:37:40 CEST 2016
by reger
apply postranking media search boost only on media queries
Changed Files: source/net/yacy/search/query/SearchEvent.java
Fri Sep 02 11:55:46 CEST 2016
by luccioman
Fix for startup option

- Var initialization
- Declaration in getopt
Changed Files: startYACY.sh
Fri Sep 02 11:22:39 CEST 2016
by luccioman
Explicitely set YaCy data folder when starting in MacOS bundle
Changed Files: addon/YaCy.app/Contents/Info.plist, addon/YaCy.app/Contents/MacOS/startYACYMacOS.sh, build.xml, startYACY.sh
Thu Sep 01 20:33:28 CEST 2016
by reger
upd master.lng for RankingSolr_p.html (add Filter Query txt)
Changed Files: locales/master.lng.xlf
Thu Sep 01 15:40:28 CEST 2016
by luccioman
Postprocessing refactoring

Added Javadocs to refactored methods.
Added log warnings instead of silently failing some errors.
Only fill collection1hosts when required ( shallComputeCR true).
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Wed Aug 31 12:16:25 CEST 2016
by luccioman
Refactored postprocessing

For easier understanding and performances profiling.
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Wed Aug 31 02:36:59 CEST 2016
by reger
skip resolving of host "<unknown>" in hello servlet
Changed Files: htroot/yacy/hello.java
Wed Aug 31 02:24:30 CEST 2016
by reger
correct NewPool rentention calculation
(was still clearing everything after one day)
Changed Files: source/net/yacy/peers/NewsPool.java
Tue Aug 30 02:01:17 CEST 2016
by reger
master.lng: add language names (ConfigBasic.html)
Changed Files: locales/master.lng.xlf
Tue Aug 30 01:10:36 CEST 2016
by Burkhard
Merge pull request #69 from Stepanov-Sergey/master

to add Russian synonyms  
Changed Files: htroot/DictionaryLoader_p.html, htroot/DictionaryLoader_p.java
Tue Aug 30 00:06:42 CEST 2016
by reger
TransNews_p: add ad-hoc translation of target file on positive vote (additon to local translation)
+ errmsg on language=default
Changed Files: htroot/TransNews_p.html, htroot/TransNews_p.java, source/net/yacy/utils/translation/TranslationManager.java
Mon Aug 29 22:10:21 CEST 2016
by reger
upd master.lng (add TransNews_p.html)
Changed Files: locales/master.lng.xlf
Mon Aug 29 02:36:55 CEST 2016
by reger
-fix Supporter log line (instead of System.out)
-TransNews_p.html del debugging text left-over
Changed Files: htroot/Supporter.java, htroot/TransNews_p.html
Mon Aug 29 02:15:06 CEST 2016
by reger
Added news service for easy, community driven UI translation support.
New or modified translation (via /Translator_p.html) can be shared/distributed
via the YaCy internal news service. Remote peers can see and vote on the
translation via the new http://localhost:8090/TransNews_p.html servlet.
A positive vote will add the received translation to the local translation
list and post a voting message to the news service.
(at this no processing of received votings is implemented)

+ fixed the msg service retention time check (NewsPool.automaticProcessP)
Changed Files: htroot/TransNews_p.html, htroot/TransNews_p.java, htroot/Translator_p.html, source/net/yacy/peers/NewsPool.java
Mon Aug 29 01:27:09 CEST 2016
by reger
make error msg part of html (allowing translation) instead of hardcoded text
Changed Files: htroot/Translator_p.html, htroot/Translator_p.java, locales/master.lng.xlf
Sun Aug 28 23:08:03 CEST 2016
by reger
Rename class CreateTranslationMaster to TranslationManager and add 
additional routines and the capability to handle translation maps internally 
(to reduce complexity of handling translation maps for calling servelets)
Changed Files: .classpath, htroot/Translator_p.java, source/net/yacy/utils/translation/TranslationManager.java
Sun Aug 28 22:54:06 CEST 2016
by Sergey Stepanov
to add Russian synonyms

requires health checks
Changed Files: htroot/DictionaryLoader_p.html
Sun Aug 28 22:46:05 CEST 2016
by Sergey Stepanov
to add Russian synonyms

requires health checks
Changed Files: htroot/DictionaryLoader_p.java
Sun Aug 28 02:55:42 CEST 2016
by reger
speed-up reading of xlif language file, by using xmlparser (stax) instead of jaxb
making xliff-core-1.2-1.1.jar obsolete
Changed Files: build.xml, pom.xml, source/net/yacy/utils/translation/TranslatorXliff.java
Sun Aug 28 02:19:29 CEST 2016
by reger
rem some more untranslated text in language files
(to proper display in translation editor)
Changed Files: locales/cn.lng, locales/gr.lng, locales/it.lng, locales/ja.lng, locales/ru.lng, locales/uk.lng
Fri Aug 26 14:51:32 CEST 2016
by Michael Peter Christen
force browser-caching of favicons from search results
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Fri Aug 26 12:15:39 CEST 2016
by Orbiter
Merge pull request #42 from otteresk/master

improve plotting to make better use of y range
Changed Files: htroot/Network.html, htroot/NetworkHistory.java, htroot/PerformanceMemory_p.java
Fri Aug 26 11:49:26 CEST 2016
by Orbiter
Merge pull request #68 from luccioman/dist_macOS

Fixed packaging for YaCy 1.90 on MacOS
Changed Files: addon/YaCy.app/Contents/Info.plist, build.xml
Tue Aug 23 03:37:32 CEST 2016
by reger
Translator_p handle lang "browser" like "default"
Changed Files: htroot/Translator_p.java
Tue Aug 23 00:13:20 CEST 2016
by reger
missing file for prev commint "Introduction of additional language setting browser"
Changed Files: source/net/yacy/server/http/HTTPDFileHandler.java
Sun Aug 21 01:22:25 CEST 2016
by reger
skip overhead to resolve "<unknown>" ip
and protect against NPE in yacy/message servlet
Changed Files: htroot/yacy/message.java
Sat Aug 20 22:23:47 CEST 2016
by reger
fix CookieTest_p formatting (output of <br> as text),
change to dataoutput only by servlet, leave formatting to html.
+ removed link to obsolete env/grafics gif
Changed Files: htroot/CookieTest_p.html, htroot/CookieTest_p.java, source/net/yacy/cora/protocol/HeaderFramework.java
Fri Aug 19 19:23:51 CEST 2016
by Michael Peter Christen
enhanced browser-caching of static content
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Fri Aug 19 17:18:08 CEST 2016
by Michael Peter Christen
fix for bug in network page: the network table pages also loaded the
network images, even if they had not been showed.
Changed Files: htroot/Network.html
Fri Aug 19 15:30:53 CEST 2016
by Michael Peter Christen
don't do solr optimization, this create high IO load. We should leave
this task to solr to do that on it's own instead of forcing it.
Changed Files: source/net/yacy/search/ResourceObserver.java, source/net/yacy/search/Switchboard.java
Fri Aug 19 02:30:09 CEST 2016
by Burkhard
Merge pull request #67 from luccioman/mixedcontent

Fixed HTTPS  mixed content warnings and errors
Changed Files: htroot/ConfigPortal.html, htroot/ConfigPortal.java, htroot/compare_yacy.java, source/net/yacy/yacy.java
Wed Aug 17 17:05:40 CEST 2016
by luccioman
ConfigPortal : fixed mixed-content security issue with https

We now use the same protocol as the one used to display the config page
: so when using https, the content is not blocked by the browser
detecting mixed-content.
Changed Files: htroot/ConfigPortal.html, htroot/ConfigPortal.java
Wed Aug 17 16:13:46 CEST 2016
by luccioman
Search comparison : prefer https URLs

Also disabled currently not working URLs.
Changed Files: htroot/compare_yacy.java
Wed Aug 17 15:15:31 CEST 2016
by luccioman
Copy image resources contained in donation iframe.

Handle eventual images loading errors.
Changed Files: source/net/yacy/yacy.java
Wed Aug 17 15:15:31 CEST 2016
by luccioman
Copy image resources contained in donation iframe.

Also refactored the related code :
- extracted it in a method
- catch Exception and log it, rather than Throwable which is bad
practice
Changed Files: source/net/yacy/yacy.java
Wed Aug 17 03:07:26 CEST 2016
by reger
add error msg to api/share.html
and skip display of url on nothing uploaded
Changed Files: htroot/api/share.html, htroot/api/share.java
Mon Aug 15 04:13:06 CEST 2016
by reger
use config for path to locales in Translator servlet
Changed Files: htroot/Translator_p.java
Sat Aug 13 23:46:36 CEST 2016
by reger
refactor xlsParser to include Excel file attribute (like author) in parser result doc.
Similar to ppt and doc parser, completing a TODO in xlsParser.
Changed Files: source/net/yacy/document/parser/xlsParser.java, test/java/net/yacy/document/parser/xlsParserTest.java
Sat Aug 13 21:46:00 CEST 2016
by Burkhard
Merge pull request #66 from luccioman/opensearch

Opensearch : fix for default ports and naming consistency
Changed Files: htroot/opensearchdescription.java, htroot/opensearchdescription.xml, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java
Fri Aug 12 12:36:44 CEST 2016
by luccioman
Fixed opensearch ShortName consistency with link in index.html

Fix for mantis 679 (http://mantis.tokeek.de/view.php?id=679)
Changed Files: htroot/opensearchdescription.xml
Fri Aug 12 12:18:26 CEST 2016
by luccioman
Opensearch desc : handle https protocol url with default port (443)

This completes modifications made for mantis 669
(http://mantis.tokeek.de/view.php?id=669)
Changed Files: htroot/opensearchdescription.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java
Wed Aug 10 11:03:01 CEST 2016
by luccioman
Merged master to 'heroku' branch.
Changed Files: source/net/yacy/data/WorkTables.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/search/Switchboard.java, test/java/net/yacy/repository/BlacklistTest.java
Wed Aug 10 08:15:41 CEST 2016
by luccioman
Merged master into 'docker' branch.
Changed Files: source/net/yacy/data/WorkTables.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/search/Switchboard.java
Wed Aug 10 08:14:49 CEST 2016
by luccioman
Merge remote-tracking branch 'origin/master' into docker
Changed Files: source/net/yacy/search/Switchboard.java
Tue Aug 09 03:03:04 CEST 2016
by reger
add missing scheduler calc of next exec_date (call of calculateAPIScheduler)
- after last_exec_date is altered, next_exec_date should be recalculated
- makes the recalculation of next_exec in advance (without api call surely made) in Switchbard.schedulerJob() obsolete
Slightly modify next_exec calc. on missed event to now+schedule_time (from fix 10min)

Changed Files: source/net/yacy/data/WorkTables.java, source/net/yacy/search/Switchboard.java
Mon Aug 08 02:19:48 CEST 2016
by reger
keep scheduler selection by repeat entry from https://github.com/yacy/yacy_search_server/commit/07311020d47d42000d5d2488489a7927020739a9
to allow exec schedule on actual exec event.
Iterate on exec date (of advantage after interruption/shutdown) to schedule
older or missed events first.
Changed Files: source/net/yacy/search/Switchboard.java
Mon Aug 08 01:57:31 CEST 2016
by reger
revert due to conflict with double count recording by schedulter / servlet by the commit under normal operation (no shutdown)
Changed Files: source/net/yacy/data/WorkTables.java, source/net/yacy/search/Switchboard.java
Sun Aug 07 05:08:55 CEST 2016
by reger
postpone apicall exec date init until actual call 
fix for http://mantis.tokeek.de/view.php?id=677
The difference is on scheduling a large number of rss feeds and loading 
is not finished before shutdown of YaCy. The change makes sure not already
loaded RSS will be loaded by the scheduler on next startup.
Changed Files: source/net/yacy/data/WorkTables.java, source/net/yacy/search/Switchboard.java
Fri Aug 05 11:59:49 CEST 2016
by luccioman
Merged changes from master.
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/repository/Blacklist.java, test/java/net/yacy/repository/BlacklistTest.java
Fri Aug 05 11:57:38 CEST 2016
by luccioman
Enabled HTTPS as default, and added HTPS related documentation
Changed Files: docker/Dockerfile, docker/Dockerfile.alpine, docker/Readme.md
Thu Aug 04 01:12:49 CEST 2016
by reger
fix Blacklist.contains() matching path pattern to string
similar to https://github.com/yacy/yacy_search_server/commit/5e9e8711927e0beb08881f8096f877d8b82c785d
+ add proof testcase
Changed Files: source/net/yacy/repository/Blacklist.java, test/java/net/yacy/repository/BlacklistTest.java
Wed Aug 03 02:13:26 CEST 2016
by reger
fix Blacklist.remove by using pattern.toString to find pattern to remove,
parameter String path did never equal Pattern.
+ delete unused removeAll, as it does not persist changes after restart
Changed Files: source/net/yacy/repository/Blacklist.java
Tue Aug 02 02:41:03 CEST 2016
by reger
on Blacklist.add pattern to source file also update internal entry maps
as in Blacklist.add(blacklistType) to make entry effective w/o restart
fix for http://mantis.tokeek.de/view.php?id=676
Changed Files: source/net/yacy/repository/Blacklist.java
Sun Jul 31 23:16:24 CEST 2016
by reger
Correct use of _htDocsPath config in YaCyDefaultServlet to use servlet config variable
+ add some javadoc and remove a not useful static declaration
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Sun Jul 31 19:24:52 CEST 2016
by luccioman
Expose HTTPS default port on docker images
Changed Files: docker/Dockerfile, docker/Dockerfile.alpine, docker/docker-cloud.yml
Thu Jul 28 02:04:51 CEST 2016
by luc
Simplified Heroku variables configuration

- Having well defined variables is more readable than configuring
everything in MAVEN_CUSTOM_OPTS
- Do not provide a default admin password value : people deploying on
Heroku for testing or demonstration purpose would likely forget to
customize this password, which would be a security flaw
- reduce Heroku specific impacts on existing pom.xml
Changed Files: Heroku.md, Procfile, app.json, pom.xml
Wed Jul 27 02:16:20 CEST 2016
by reger
add uses of config constant INDEX_RECEIVE_ALLOW
Changed Files: htroot/IndexShare_p.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, source/net/yacy/search/Switchboard.java
Sat Jul 23 20:34:47 CEST 2016
by reger
upd to commons-compress-1.12
Changed Files: .classpath, build.xml, lib/commons-compress-1.12.License, lib/commons-compress-1.12.jar, pom.xml
Sat Jul 23 20:03:13 CEST 2016
by reger
declare poison pill url MultiProtocolURL() as protected to make sure not
used from outside.
After double checking use of poison url revert path init from commit
https://github.com/yacy/yacy_search_server/commit/f8632ad2921fe148e0e9c862be019e1fbb039f40
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Sat Jul 23 19:18:23 CEST 2016
by reger
prevent string index out of bounds MultiProtocolURL.getPaths
as path maybe a empty string
+ init path to "" also in init for poison url (to guarantee success for 
all existing uses of path w/o check for null)
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Sat Jul 23 18:36:43 CEST 2016
by reger
update lucenematchversion to current (5.2.0 -> 5.5.0)
there should be no need for reindex by the update
Changed Files: defaults/solr/solrconfig.xml, source/net/yacy/search/index/Fulltext.java
Thu Jul 21 21:36:00 CEST 2016
by Marc Nause
removed unused properties in default config (CGI capabilities of YaCy's
HTTPd have been removed many moons ago)
Changed Files: defaults/yacy.init
Thu Jul 21 02:14:35 CEST 2016
by reger
deprecate newurl(), not used and already replaced 
instead of making it handle all supported the protocols
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Wed Jul 20 00:35:16 CEST 2016
by luccioman
Moved variables configuration at the right place.

If running "heroku config:set" before git initialization, --app appname
option has to be explicitely passed.
Changed Files: Heroku.md
Wed Jul 20 00:33:19 CEST 2016
by luccioman
Added a message tracing yacy.init eventual customization
Changed Files: pom.xml
Tue Jul 19 02:57:41 CEST 2016
by reger
fix GenericFormatter.parse ("time","timeoffset")
change: UTC offset internally expected in minutes
Changed Files: source/net/yacy/cora/date/GenericFormatter.java
Sun Jul 17 23:42:25 CEST 2016
by reger
improve detection of referenced links by taking http and https link protocol
into account
+ correct query start detection of commit https://github.com/yacy/yacy_search_server/commit/f89d4eb51d998a6e32946804799225d89b91a465
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/document/Document.java
Sun Jul 17 04:17:01 CEST 2016
by reger
fix MultiProtocolURL init (assign of host) for urls with '/' in query part
+ add to test case
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java, test/java/net/yacy/cora/document/id/MultiProtocolURLTest.java
Sat Jul 16 20:34:42 CEST 2016
by reger
correct renamed ConfigAdvanced_p.html -> ConfigProperties_p.html in fr and sk.lng
Changed Files: locales/fr.lng, locales/sk.lng
Sat Jul 16 02:17:58 CEST 2016
by Burkhard
Merge pull request #62 from luccioman/french_translation

Updated french translation for some pages
Changed Files: locales/fr.lng, locales/master.lng.xlf
Sat Jul 16 01:59:09 CEST 2016
by reger
Adjusted hash computation and toNormalform for file:// protocol to deliver
same hash same file on Windows filesystem path with forward- and backslash in path.
Background see http://mantis.tokeek.de/view.php?id=671
+Test case
Changed Files: source/net/yacy/cora/document/id/DigestURL.java, source/net/yacy/cora/document/id/MultiProtocolURL.java, test/java/net/yacy/cora/document/id/DigestURLTest.java
Fri Jul 15 14:44:58 CEST 2016
by luccioman
Updated french translation for some pages

Used Translator_p.html editor and updated french translation for the 8
first files displayed in the combobox.
Changed Files: locales/fr.lng, locales/master.lng.xlf
Fri Jul 15 00:27:41 CEST 2016
by reger
put Autocrawl_p.html to master.lng.xlf
Changed Files: locales/de.lng, locales/master.lng.xlf
Wed Jul 13 01:34:26 CEST 2016
by luccioman
Set 'Deploy to Docker Cloud' url repo parameter to main yacy repo.
Changed Files: README.md
Wed Jul 13 01:07:11 CEST 2016
by luccioman
Removed unnecessary variables for local heroku run
Changed Files: .env
Wed Jul 13 01:06:33 CEST 2016
by luccioman
Added Heroku markdown documentation.
Changed Files: Heroku.md, README.md
Tue Jul 12 09:33:18 CEST 2016
by luccioman
Updated ConfigBasic.html translation.
Changed Files: locales/fr.lng, locales/master.lng.xlf
Tue Jul 12 09:09:55 CEST 2016
by luccioman
Added important log limit options to upgrade procedure example.
Changed Files: docker/Readme.md
Tue Jul 12 09:05:31 CEST 2016
by luccioman
Merged from maain master branch.
Changed Files: htroot/CookieTest_p.java, htroot/opensearchdescription.java, locales/master.lng.xlf, source/net/yacy/crawler/HostBalancer.java, source/net/yacy/crawler/HostQueue.java, source/net/yacy/crawler/retrieval/FileLoader.java, source/net/yacy/crawler/retrieval/SMBLoader.java, source/net/yacy/http/servlets/GSAsearchServlet.java, test/java/net/yacy/crawler/HostQueueTest.java
Tue Jul 12 01:53:01 CEST 2016
by luccioman
Handle more propertly local port configuration by system property

And prefixed property with "net.yacy" to avoid ambiguity.
Changed Files: Procfile, htroot/ConfigBasic.html, htroot/ConfigBasic.java, source/net/yacy/server/serverSwitch.java
Mon Jul 11 04:08:24 CEST 2016
by reger
add missing text for ConfigBasic.html to master.lng.xlf
Changed Files: locales/master.lng.xlf
Mon Jul 11 02:33:12 CEST 2016
by reger
adjust opensearchdescription to return url with protocol it was call on
fix http://mantis.tokeek.de/view.php?id=669
Changed Files: htroot/opensearchdescription.java
Sat Jul 09 19:55:47 CEST 2016
by reger
use directories original lastmodified date in file- & smbloader in response
Changed Files: source/net/yacy/crawler/retrieval/FileLoader.java, source/net/yacy/crawler/retrieval/SMBLoader.java
Fri Jul 08 02:04:38 CEST 2016
by luccioman
Set deploy buttons in one place
Changed Files: README.md, docker/Readme.md
Fri Jul 08 01:51:46 CEST 2016
by luccioman
Added a more detailed description for MAVEN_CUSTOM_OPTS
Changed Files: app.json
Fri Jul 08 01:33:47 CEST 2016
by luccioman
Added a one-click deploy button for Heroku

Transformed main YaCy readme from mediawiki to Markdown format for
better support on GitHub and easier deploy buttons integration.
Changed Files: README.md, app.json
Thu Jul 07 20:50:27 CEST 2016
by luccioman
Handle more properly missing .git directory

Made use of maven profiles.
Changed Files: libbuild/GitRevMavenTask/src/GitRevMavenTask.java, pom.xml
Thu Jul 07 03:13:52 CEST 2016
by luccioman
Fixed inconsistent port check on ConfigBasic

Changed Files: htroot/ConfigBasic.java
Thu Jul 07 02:50:57 CEST 2016
by reger
one more replacement to use cached hosthash vs. calculated
Changed Files: source/net/yacy/crawler/HostBalancer.java
Thu Jul 07 02:33:03 CEST 2016
by luccioman
Possibility to override some yacy.init properties at maven build

It is convenient with Heroky to be able to configure staticIP, http port
and admin initial password at maven build.
Changed Files: defaults/yacy.init, pom.xml
Thu Jul 07 00:03:11 CEST 2016
by luccioman
Local port is set by Heroku, but public port is 80.
Changed Files: defaults/yacy.init
Wed Jul 06 23:35:12 CEST 2016
by luccioman
Revert : Not possible to start on port 80
Changed Files: Procfile
Wed Jul 06 22:33:24 CEST 2016
by luccioman
Try running on port 80 to be in senior mode on Heroku
Changed Files: Procfile
Wed Jul 06 22:22:13 CEST 2016
by luccioman
Now try to run again on Heroku in p2p mode
Changed Files: defaults/yacy.init
Wed Jul 06 22:14:40 CEST 2016
by luccioman
Let Heroku decide which http port to use
Changed Files: Procfile, source/net/yacy/server/serverSwitch.java
Wed Jul 06 22:13:18 CEST 2016
by luccioman
Reverted maven version specification

This was not the cause of build failure.
Changed Files:
Wed Jul 06 21:05:23 CEST 2016
by luccioman
Set log level to FINE to have more details
Changed Files: defaults/yacy.logging
Wed Jul 06 20:16:38 CEST 2016
by luccioman
Experiment on Heroku starting in Intranet mode as default
Changed Files: defaults/yacy.init
Wed Jul 06 02:21:34 CEST 2016
by luccioman
Added PORT env variable for heroku
Changed Files: .env
Wed Jul 06 01:04:44 CEST 2016
by luccioman
Prevent build failing when .git is missing
Changed Files: libbuild/GitRevMavenTask/src/GitRevMavenTask.java
Wed Jul 06 01:04:44 CEST 2016
by luccioman
Prevent build failing when .git is missing
Changed Files: libbuild/GitRevMavenTask/src/GitRevMavenTask.java
Tue Jul 05 23:47:06 CEST 2016
by luccioman
Added a system config file for heroku.

Explicitely set maven version to the same version of Debian Jessie, as
build on herolu dashboard with maven 3.3.9 fails.
Changed Files: system.properties
Tue Jul 05 23:22:35 CEST 2016
by reger
to prevent crawler to concurrently access and alter same crawl queue
after restart, put hosthash in queue's filename (which is used as primary 
key for crawl queue. Hint: initial hosthash from url and recalculated hosthash 
from just hostname:port are not the same. 
fixes http://mantis.tokeek.de/view.php?id=668 (partially)
Changed Files: source/net/yacy/crawler/HostBalancer.java, source/net/yacy/crawler/HostQueue.java
Tue Jul 05 23:15:56 CEST 2016
by Andreas
Upload .gitignore from original rep - 3nd try
Changed Files: .gitignore
Tue Jul 05 23:13:21 CEST 2016
by Andreas
Add files via upload
Changed Files: .gitignore
Tue Jul 05 23:09:35 CEST 2016
by Andreas
Upload .gitignore from original rep
Changed Files: .gitignore
Tue Jul 05 22:49:40 CEST 2016
by luccioman
Added Heroku config files

- .env : environment variables to build and run YaCy with Heroku
- Procfile : command line used to start YaCy with Heroku
Tested with heroku local
Changed Files: .env, Procfile
Tue Jul 05 21:15:33 CEST 2016
by luccioman
Ignore maven build repositories
Changed Files: libbuild/GitRevMavenTask/.gitignore, libbuild/J7Zip-modified/.gitignore, libbuild/WebCat-swf/.gitignore
Mon Jul 04 21:06:54 CEST 2016
by Orbiter
Merge pull request #55 from luccioman/docker

Improve Docker image security, size and reliability
Changed Files: docker/Dockerfile, docker/Dockerfile.alpine, docker/Readme.md, readme.mediawiki
Mon Jul 04 21:05:59 CEST 2016
by Orbiter
Merge pull request #47 from reelsense/patch-2

Distilled sentence - ConfigNetwork_p
Changed Files: htroot/ConfigNetwork_p.html
Mon Jul 04 20:59:37 CEST 2016
by Michael Peter Christen
new development cycle
Changed Files: build.properties