YaCy Release 1.90

After more than a year we have a huge set of new features, too many to
count them all here. To see a complete list, please open
http://yacy.net/release_notes/YaCy_Release_1_90.html

Major Changes   
Jump to: Bugfixes / Other Changes

CommitDescription
Sun Jul 03 18:59:00 CEST 2016
by luccioman
Merge remote-tracking branch 'origin/master'

Conflicts:
	htroot/Load_MediawikiWiki.java
	htroot/Load_PHPBB3.java
	htroot/ViewImage.java
Changed Files: .classpath, .gitignore, assembly.xml, build.xml, debian/control, defaults/yacy.init, defaults/yacy.network.freeworld.unit, defaults/yacy.network.zeronet.unit, defaults/yacy.networks, docker/Dockerfile, docker/Readme.md, docker/docker-cloud.yml, examples/SimpleSearchClient/pom.xml, examples/SimpleSearchClient/src/YaCySearchClient.java, htroot/Blacklist_p.html, htroot/ConfigBasic.html, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.html, htroot/ConfigLanguage_p.java, htroot/ConfigRobotsTxt_p.html, htroot/ConfigRobotsTxt_p.java, htroot/ConfigSearchBox.java, htroot/CrawlMonitorRemoteStart.java, htroot/Crawler_p.java, htroot/IndexExport_p.html, htroot/IndexExport_p.java, htroot/Load_MediawikiWiki.java, htroot/Load_PHPBB3.java, htroot/MessageSend_p.html, htroot/MessageSend_p.java, htroot/Network.html, htroot/Network.java, htroot/RankingSolr_p.html, htroot/RankingSolr_p.java, htroot/SettingsAck_p.html, htroot/SettingsAck_p.java, htroot/Settings_ProxyAccess.inc, htroot/Settings_p.java, htroot/Status.html, htroot/Status.java, htroot/Status_p.inc, htroot/Translator_p.html, htroot/Translator_p.java, htroot/ViewProfile.html, htroot/Vocabulary_p.java, htroot/WebStructurePicture_p.java, htroot/api/citation.java, htroot/api/push_p.java, htroot/api/share.html, htroot/api/share.java, htroot/api/snapshot.java, htroot/env/templates/header.template, htroot/env/templates/metas.template, htroot/env/templates/submenuConfig.template, htroot/index.html, htroot/rssTerminal.html, htroot/yacy/transferRWI.java, htroot/yacysearch.java, htroot/yacysearchitem.java, lib/J7Zip-modified.jar, lib/commons-compress-1.11.License, lib/commons-compress-1.11.jar, lib/commons-fileupload-1.3.2.License, lib/commons-fileupload-1.3.2.jar, lib/commons-io-2.5.License, lib/commons-io-2.5.jar, lib/fontbox-2.0.2.License, lib/fontbox-2.0.2.jar, lib/httpclient-4.5.2.jar, lib/httpcore-4.4.5.License, lib/httpcore-4.4.5.jar, lib/httpmime-4.5.2.jar, lib/icu4j-57_1.jar, lib/jcl-over-slf4j-1.7.21.jar, lib/jetty-9.2.17.v20160517.License, lib/jetty-client-9.2.17.v20160517.jar, lib/jetty-continuation-9.2.17.v20160517.jar, lib/jetty-deploy-9.2.17.v20160517.jar, lib/jetty-http-9.2.17.v20160517.jar, lib/jetty-io-9.2.17.v20160517.jar, lib/jetty-jmx-9.2.17.v20160517.jar, lib/jetty-proxy-9.2.17.v20160517.jar, lib/jetty-security-9.2.17.v20160517.jar, lib/jetty-server-9.2.17.v20160517.jar, lib/jetty-servlet-9.2.17.v20160517.jar, lib/jetty-servlets-9.2.17.v20160517.jar, lib/jetty-util-9.2.17.v20160517.jar, lib/jetty-webapp-9.2.17.v20160517.jar, lib/jetty-xml-9.2.17.v20160517.jar, lib/jsoup-1.9.2.jar, lib/log4j-over-slf4j-1.7.21.jar, lib/lucene-analyzers-common-5.5.1.jar, lib/lucene-analyzers-phonetic-5.5.1.jar, lib/lucene-backward-codecs-5.5.1.jar, lib/lucene-classification-5.5.1.jar, lib/lucene-codecs-5.5.1.jar, lib/lucene-core-5.5.1.jar, lib/lucene-facet-5.5.1.jar, lib/lucene-grouping-5.5.1.jar, lib/lucene-highlighter-5.5.1.jar, lib/lucene-join-5.5.1.jar, lib/lucene-memory-5.5.1.jar, lib/lucene-misc-5.5.1.jar, lib/lucene-queries-5.5.1.jar, lib/lucene-queryparser-5.5.1.jar, lib/lucene-spatial-5.5.1.jar, lib/lucene-suggest-5.5.1.jar, lib/metadata-extractor-2.9.1.License, lib/metadata-extractor-2.9.1.jar, lib/pdfbox-2.0.2.License, lib/pdfbox-2.0.2.jar, lib/poi-3.14-20160307.jar, lib/poi-3.14.License, lib/poi-scratchpad-3.14-20160307.jar, lib/slf4j-api-1.7.21.jar, lib/slf4j-jdk14-1.7.21.jar, lib/solr-core-5.5.1.jar, lib/solr-solrj-5.5.1.jar, lib/spatial4j-0.5.jar, lib/weupnp-0.1.4.jar, lib/xliff-core-1.2-1.1.jar, libbuild/GitRevMavenTask/src/GitRevMavenTask.java, libbuild/GitRevTask/GitRevTask.java, libbuild/J7Zip-modified/pom.xml, libbuild/J7Zip-modified/src/Common/BoolVector.java, libbuild/J7Zip-modified/src/Common/CRC.java, libbuild/J7Zip-modified/src/Common/IntVector.java, libbuild/J7Zip-modified/src/Common/LimitedSequentialInStream.java, libbuild/J7Zip-modified/src/Common/LockedInStream.java, libbuild/J7Zip-modified/src/Common/LockedSequentialInStreamImp.java, libbuild/J7Zip-modified/src/Common/LongVector.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/BindInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/BindPair.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/CoderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/CoderMixer2.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/CoderMixer2ST.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/CoderStreamsInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/FilterCoder.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/OutStreamWithCRC.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/STCoderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/IArchiveExtractCallback.java, libbuild/J7Zip-modified/src/SevenZip/Archive/IInArchive.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/AltCoderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/ArchiveDB.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/ArchiveDatabase.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/ArchiveDatabaseEx.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/BindInfoEx.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/CoderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/ExtractFolderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/FileItem.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Folder.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/FolderOutStream.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Handler.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Header.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/InArchiveInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/InByte2.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/InStream.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/MethodID.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/StreamSwitch.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZipEntry.java, libbuild/J7Zip-modified/src/SevenZip/ArchiveExtractCallback.java, libbuild/J7Zip-modified/src/SevenZip/CRC.java, libbuild/J7Zip-modified/src/SevenZip/Common/InBuffer.java, libbuild/J7Zip-modified/src/SevenZip/Common/LocalCompressProgressInfo.java, libbuild/J7Zip-modified/src/SevenZip/Common/LocalProgress.java, libbuild/J7Zip-modified/src/SevenZip/Common/StreamUtils.java, libbuild/J7Zip-modified/src/SevenZip/Compression/Branch/BCJ2_x86_Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/Branch/BCJ_x86_Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/Copy/Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZ/BinTree.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZ/InWindow.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZ/OutWindow.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZMA/Base.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZMA/Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZMA/Encoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/BitDecoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/BitModel.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/BitTreeDecoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/BitTreeEncoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/Encoder.java, libbuild/J7Zip-modified/src/SevenZip/HRESULT.java, libbuild/J7Zip-modified/src/SevenZip/ICodeProgress.java, libbuild/J7Zip-modified/src/SevenZip/ICompressCoder.java, libbuild/J7Zip-modified/src/SevenZip/ICompressCoder2.java, libbuild/J7Zip-modified/src/SevenZip/ICompressFilter.java, libbuild/J7Zip-modified/src/SevenZip/ICompressGetInStreamProcessedSize.java, libbuild/J7Zip-modified/src/SevenZip/ICompressProgressInfo.java, libbuild/J7Zip-modified/src/SevenZip/ICompressSetInStream.java, libbuild/J7Zip-modified/src/SevenZip/ICompressSetOutStream.java, libbuild/J7Zip-modified/src/SevenZip/ICompressSetOutStreamSize.java, libbuild/J7Zip-modified/src/SevenZip/IInStream.java, libbuild/J7Zip-modified/src/SevenZip/IProgress.java, libbuild/J7Zip-modified/src/SevenZip/J7zip.java, libbuild/J7Zip-modified/src/SevenZip/LzmaAlone.java, libbuild/J7Zip-modified/src/SevenZip/LzmaBench.java, libbuild/J7Zip-modified/src/SevenZip/MyRandomAccessFile.java, libbuild/pom.xml, locales/cn.lng, locales/de.lng, locales/fr.lng, locales/gr.lng, locales/hi.lng, locales/it.lng, locales/ja.lng, locales/master.lng.xlf, locales/ru.lng, locales/sk.lng, locales/uk.lng, nbproject/project.xml, pom.xml, source/net/yacy/cora/document/id/AnchorURL.java, source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/util/Html2Image.java, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/crawler/HostBalancer.java, source/net/yacy/crawler/HostQueue.java, source/net/yacy/crawler/LegacyBalancer.java, source/net/yacy/crawler/data/Latency.java, source/net/yacy/crawler/data/ResultURLs.java, source/net/yacy/crawler/data/Transactions.java, source/net/yacy/crawler/retrieval/Response.java, source/net/yacy/data/MessageBoard.java, source/net/yacy/data/Translator.java, source/net/yacy/data/WorkTables.java, source/net/yacy/data/wiki/WikiCode.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Parser.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/language/Identificator.java, source/net/yacy/document/parser/apkParser.java, source/net/yacy/document/parser/audioTagParser.java, source/net/yacy/document/parser/augment/AugmentParser.java, source/net/yacy/document/parser/bzipParser.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/dwgParser.java, source/net/yacy/document/parser/genericParser.java, source/net/yacy/document/parser/gzipParser.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/images/metadataImageParser.java, source/net/yacy/document/parser/images/svgParser.java, source/net/yacy/document/parser/linkScraperParser.java, source/net/yacy/document/parser/mmParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rdfParser.java, source/net/yacy/document/parser/rdfa/impl/RDFaParser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/sidAudioParser.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/http/AbstractRemoteHandler.java, source/net/yacy/http/Jetty9HttpServerImpl.java, source/net/yacy/http/ProxyCacheHandler.java, source/net/yacy/http/ProxyHandler.java, source/net/yacy/http/servlets/SolrSelectServlet.java, source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/http/servlets/YaCyProxyServlet.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/migration.java, source/net/yacy/peers/Network.java, source/net/yacy/peers/PeerActions.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/Seed.java, source/net/yacy/peers/SeedDB.java, source/net/yacy/peers/operation/yacyVersion.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/AutoSearch.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/server/serverClassLoader.java, source/net/yacy/utils/translation/CreateTranslationMasters.java, source/net/yacy/utils/translation/ListNonTranslatedFiles.java, source/net/yacy/utils/translation/TranslateAll.java, source/net/yacy/utils/translation/TranslateAllToOneLanguage.java, source/net/yacy/utils/translation/TranslatorXliff.java, source/net/yacy/visualization/ChartPlotter.java, source/net/yacy/yacy.java, startYACY.sh, startYACY_debug.bat, test/java/net/yacy/cora/document/id/MultiProtocolURLTest.java, test/java/net/yacy/cora/util/Html2ImageTest.java, test/java/net/yacy/data/TranslatorTest.java, test/java/net/yacy/data/wiki/WikiCodeTest.java, test/java/net/yacy/utils/translation/TranslatorXliffTest.java
Sun Jul 03 16:53:05 CEST 2016
by luccioman
Merge remote-tracking branch 'origin/master' into docker
Changed Files: htroot/CrawlMonitorRemoteStart.java, htroot/Crawler_p.java, htroot/yacysearch.java, libbuild/GitRevMavenTask/src/GitRevMavenTask.java, libbuild/GitRevTask/GitRevTask.java, locales/cn.lng, locales/de.lng, locales/gr.lng, locales/hi.lng, locales/it.lng, locales/master.lng.xlf, locales/ru.lng, locales/uk.lng, pom.xml, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/data/wiki/WikiCode.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/peers/Protocol.java, source/net/yacy/search/Switchboard.java, test/java/net/yacy/data/wiki/WikiCodeTest.java
Tue Jun 21 12:52:18 CEST 2016
by luccioman
Merge remote-tracking branch 'origin/master' into docker
Changed Files: .classpath, build.xml, htroot/Translator_p.java, htroot/env/templates/metas.template, lib/fontbox-2.0.2.License, lib/fontbox-2.0.2.jar, lib/httpcore-4.4.5.License, lib/httpcore-4.4.5.jar, lib/jcl-over-slf4j-1.7.21.jar, lib/log4j-over-slf4j-1.7.21.jar, lib/pdfbox-2.0.2.License, lib/pdfbox-2.0.2.jar, lib/slf4j-api-1.7.21.jar, lib/slf4j-jdk14-1.7.21.jar, locales/de.lng, locales/fr.lng, locales/master.lng.xlf, pom.xml
Thu Jun 09 17:18:19 CEST 2016
by luccioman
Merge remote-tracking branch 'origin/master' into docker
Changed Files: .classpath, build.xml, debian/control, htroot/Translator_p.html, htroot/Translator_p.java, lib/jsoup-1.9.2.jar, locales/de.lng, locales/uk.lng, pom.xml, source/net/yacy/data/MessageBoard.java, source/net/yacy/data/Translator.java, source/net/yacy/utils/translation/CreateTranslationMasters.java, source/net/yacy/utils/translation/TranslatorXliff.java
Thu Jun 09 14:44:51 CEST 2016
by luccioman
Merge remote-tracking branch 'origin/master' into LibreJS
Changed Files: .classpath, build.xml, debian/control, htroot/Translator_p.html, htroot/Translator_p.java, lib/jsoup-1.9.2.jar, locales/de.lng, locales/uk.lng, pom.xml, source/net/yacy/data/MessageBoard.java, source/net/yacy/data/Translator.java, source/net/yacy/utils/translation/CreateTranslationMasters.java, source/net/yacy/utils/translation/TranslatorXliff.java
Wed Jun 08 09:56:16 CEST 2016
by luccioman
Aplied GNU licensing recommendations.
Changed Files: htroot/env/templates/metas.template, htroot/js/Bookmarks.js, htroot/js/Crawler.js, htroot/js/IndexCreate.js, htroot/js/WatchWebStructure.js, htroot/js/ajax.js, htroot/js/html.js, htroot/js/hypertree.js, htroot/js/query.js, htroot/js/rss2.js, htroot/js/xml.js, htroot/js/yacy-ymarks-bookmark-actions.js, htroot/js/yacy-ymarks-tag-actions.js, htroot/js/yacy-ymarks.js, htroot/js/yacyinteractive.js, htroot/js/yacysearch.js
Mon Jun 06 09:20:03 CEST 2016
by luccioman
Added license information for YaCy owned js files
Changed Files: htroot/js/Bookmarks.js, htroot/js/Crawler.js, htroot/js/IndexCreate.js, htroot/js/WatchWebStructure.js, htroot/js/ajax.js, htroot/js/html.js, htroot/js/hypertree.js, htroot/js/query.js, htroot/js/rss2.js, htroot/js/xml.js, htroot/js/yacy-ymarks-bookmark-actions.js, htroot/js/yacy-ymarks-tag-actions.js, htroot/js/yacy-ymarks.js, htroot/js/yacyinteractive.js, htroot/js/yacysearch.js, htroot/jslicense.html
Fri Jun 03 16:07:58 CEST 2016
by luccioman
Merge remote-tracking branch 'origin/master' into docker
Changed Files: .classpath, build.xml, htroot/ConfigBasic.java, htroot/ConfigLanguage_p.html, htroot/ConfigLanguage_p.java, htroot/Translator_p.html, htroot/Translator_p.java, lib/commons-fileupload-1.3.2.License, lib/commons-fileupload-1.3.2.jar, lib/jetty-9.2.17.v20160517.License, lib/jetty-client-9.2.17.v20160517.jar, lib/jetty-continuation-9.2.17.v20160517.jar, lib/jetty-deploy-9.2.17.v20160517.jar, lib/jetty-http-9.2.17.v20160517.jar, lib/jetty-io-9.2.17.v20160517.jar, lib/jetty-jmx-9.2.17.v20160517.jar, lib/jetty-proxy-9.2.17.v20160517.jar, lib/jetty-security-9.2.17.v20160517.jar, lib/jetty-server-9.2.17.v20160517.jar, lib/jetty-servlet-9.2.17.v20160517.jar, lib/jetty-servlets-9.2.17.v20160517.jar, lib/jetty-util-9.2.17.v20160517.jar, lib/jetty-webapp-9.2.17.v20160517.jar, lib/jetty-xml-9.2.17.v20160517.jar, pom.xml, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/data/Translator.java, source/net/yacy/search/AutoSearch.java, source/net/yacy/search/Switchboard.java, source/net/yacy/utils/translation/CreateTranslationMasters.java, source/net/yacy/utils/translation/ListNonTranslatedFiles.java, source/net/yacy/utils/translation/TranslateAll.java, source/net/yacy/utils/translation/TranslateAllToOneLanguage.java, source/net/yacy/utils/translation/TranslatorXliff.java, source/net/yacy/yacy.java, test/java/net/yacy/utils/translation/TranslatorXliffTest.java
Fri Jun 03 01:46:30 CEST 2016
by reger
introduce a translation edit servlet Translator_p.html YaCy's UI text translation
This is the 1st rudimentary approach to support the translatio utilities.
It allows currently to edit untranslated text and save it in a local translation file
in the DATA/LOCALE directory.
+ refactor Translator (less static's) to leverage on class overrides and support garbage collection for this 1 time routine
+ adjust TranslatorXliff to check for local translations in DATA/LOCALE,
  this includes storing manually downloaded translation files in DATA as well 
  (to keep default untouched)
+ on 1st call of Translator_p a master tanslation file is generated, checking
the supported languages for missing translation text (later this masterfile is planned to part of the distribution, to harmonize translation key text between the languages)
Outlook: the local modifications (possibly as translation fragments instead of complete file) to be shared with maintainer using xlif features. 
Changed Files: htroot/ConfigBasic.java, htroot/ConfigLanguage_p.html, htroot/ConfigLanguage_p.java, htroot/Translator_p.html, htroot/Translator_p.java, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/data/Translator.java, source/net/yacy/utils/translation/CreateTranslationMasters.java, source/net/yacy/utils/translation/ListNonTranslatedFiles.java, source/net/yacy/utils/translation/TranslateAll.java, source/net/yacy/utils/translation/TranslateAllToOneLanguage.java, source/net/yacy/utils/translation/TranslatorXliff.java, source/net/yacy/yacy.java, test/java/net/yacy/utils/translation/TranslatorXliffTest.java
Thu Jun 02 21:56:56 CEST 2016
by reger
upd to Jetty v9.2.17.v20160517
Changed Files: .classpath, build.xml, lib/commons-fileupload-1.3.2.jar, lib/jetty-9.2.17.v20160517.License, lib/jetty-client-9.2.17.v20160517.jar, lib/jetty-continuation-9.2.17.v20160517.jar, lib/jetty-deploy-9.2.17.v20160517.jar, lib/jetty-http-9.2.17.v20160517.jar, lib/jetty-io-9.2.17.v20160517.jar, lib/jetty-jmx-9.2.17.v20160517.jar, lib/jetty-proxy-9.2.17.v20160517.jar, lib/jetty-security-9.2.17.v20160517.jar, lib/jetty-server-9.2.17.v20160517.jar, lib/jetty-servlet-9.2.17.v20160517.jar, lib/jetty-servlets-9.2.17.v20160517.jar, lib/jetty-util-9.2.17.v20160517.jar, lib/jetty-webapp-9.2.17.v20160517.jar, lib/jetty-xml-9.2.17.v20160517.jar, pom.xml
Thu May 19 09:57:00 CEST 2016
by luc
Merge remote-tracking branch 'origin/master' into docker
Changed Files: .classpath, build.xml, htroot/index.html, lib/lucene-analyzers-common-5.5.1.jar, lib/lucene-analyzers-phonetic-5.5.1.jar, lib/lucene-backward-codecs-5.5.1.jar, lib/lucene-classification-5.5.1.jar, lib/lucene-codecs-5.5.1.jar, lib/lucene-core-5.5.1.jar, lib/lucene-facet-5.5.1.jar, lib/lucene-grouping-5.5.1.jar, lib/lucene-highlighter-5.5.1.jar, lib/lucene-join-5.5.1.jar, lib/lucene-memory-5.5.1.jar, lib/lucene-misc-5.5.1.jar, lib/lucene-queries-5.5.1.jar, lib/lucene-queryparser-5.5.1.jar, lib/lucene-spatial-5.5.1.jar, lib/lucene-suggest-5.5.1.jar, lib/solr-core-5.5.1.jar, lib/solr-solrj-5.5.1.jar, pom.xml, source/net/yacy/search/Switchboard.java
Thu May 19 02:31:34 CEST 2016
by reger
upd to Solr 5.5.1
Changed Files: .classpath, build.xml, lib/lucene-analyzers-common-5.5.1.jar, lib/lucene-analyzers-phonetic-5.5.1.jar, lib/lucene-backward-codecs-5.5.1.jar, lib/lucene-classification-5.5.1.jar, lib/lucene-codecs-5.5.1.jar, lib/lucene-core-5.5.1.jar, lib/lucene-facet-5.5.1.jar, lib/lucene-grouping-5.5.1.jar, lib/lucene-highlighter-5.5.1.jar, lib/lucene-join-5.5.1.jar, lib/lucene-memory-5.5.1.jar, lib/lucene-misc-5.5.1.jar, lib/lucene-queries-5.5.1.jar, lib/lucene-queryparser-5.5.1.jar, lib/lucene-spatial-5.5.1.jar, lib/lucene-suggest-5.5.1.jar, lib/solr-core-5.5.1.jar, lib/solr-solrj-5.5.1.jar, pom.xml
Tue May 17 23:44:11 CEST 2016
by luc
Merge remote-tracking branch 'origin/master' into docker
Changed Files: .classpath, .gitignore, build.xml, htroot/MessageSend_p.html, htroot/MessageSend_p.java, lib/jetty-9.2.16.v20160414.License, lib/jetty-client-9.2.16.v20160414.jar, lib/jetty-continuation-9.2.16.v20160414.jar, lib/jetty-deploy-9.2.16.v20160414.jar, lib/jetty-http-9.2.16.v20160414.jar, lib/jetty-io-9.2.16.v20160414.jar, lib/jetty-jmx-9.2.16.v20160414.jar, lib/jetty-proxy-9.2.16.v20160414.jar, lib/jetty-security-9.2.16.v20160414.jar, lib/jetty-server-9.2.16.v20160414.jar, lib/jetty-servlet-9.2.16.v20160414.jar, lib/jetty-servlets-9.2.16.v20160414.jar, lib/jetty-util-9.2.16.v20160414.jar, lib/jetty-webapp-9.2.16.v20160414.jar, lib/jetty-xml-9.2.16.v20160414.jar, pom.xml, source/net/yacy/cora/util/Html2Image.java, source/net/yacy/crawler/data/Transactions.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/peers/SeedDB.java, test/java/net/yacy/cora/util/Html2ImageTest.java
Mon May 16 20:34:19 CEST 2016
by reger
upd to Jetty v9.2.16.v20160414
- exclude unused mime4j
- remove unused yacy-cora build
Changed Files: .classpath, .gitignore, build.xml, lib/jetty-9.2.16.v20160414.License, lib/jetty-client-9.2.16.v20160414.jar, lib/jetty-continuation-9.2.16.v20160414.jar, lib/jetty-deploy-9.2.16.v20160414.jar, lib/jetty-http-9.2.16.v20160414.jar, lib/jetty-io-9.2.16.v20160414.jar, lib/jetty-jmx-9.2.16.v20160414.jar, lib/jetty-proxy-9.2.16.v20160414.jar, lib/jetty-security-9.2.16.v20160414.jar, lib/jetty-server-9.2.16.v20160414.jar, lib/jetty-servlet-9.2.16.v20160414.jar, lib/jetty-servlets-9.2.16.v20160414.jar, lib/jetty-util-9.2.16.v20160414.jar, lib/jetty-webapp-9.2.16.v20160414.jar, lib/jetty-xml-9.2.16.v20160414.jar, pom.xml, test/java/net/yacy/cora/util/Html2ImageTest.java
Sun May 08 16:38:17 CEST 2016
by otter
manually pulled - now pushed
Changed Files: .classpath, .project, assembly.xml, build.xml, defaults/yacy.init, htroot/ConfigLanguage_p.html, htroot/ConfigLanguage_p.java, htroot/ConfigRobotsTxt_p.html, htroot/NetworkHistory.java, htroot/SettingsAck_p.html, htroot/SettingsAck_p.java, htroot/Settings_ProxyAccess.inc, htroot/Settings_p.java, htroot/Status.java, htroot/Status_p.inc, htroot/api/snapshot.java, htroot/yacy/transferRWI.java, htroot/yacysearch.java, lib/commons-compress-1.11.License, lib/commons-compress-1.11.jar, lib/commons-io-2.5.License, lib/commons-io-2.5.jar, lib/jsoup-1.9.1.jar, lib/metadata-extractor-2.9.1.License, lib/metadata-extractor-2.9.1.jar, lib/xliff-core-1.2-1.1.jar, locales/cn.lng, locales/de.lng, locales/fr.lng, locales/gr.lng, locales/hi.lng, locales/it.lng, locales/ja.lng, locales/ru.lng, locales/sk.lng, locales/uk.lng, pom.xml, source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/crawler/HostBalancer.java, source/net/yacy/crawler/HostQueue.java, source/net/yacy/crawler/data/ResultURLs.java, source/net/yacy/data/Translator.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/http/AbstractRemoteHandler.java, source/net/yacy/http/Jetty9HttpServerImpl.java, source/net/yacy/http/ProxyCacheHandler.java, source/net/yacy/http/ProxyHandler.java, source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/http/servlets/YaCyProxyServlet.java, source/net/yacy/migration.java, source/net/yacy/peers/PeerActions.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/operation/yacyVersion.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/utils/translation/CreateTranslationMasters.java, source/net/yacy/utils/translation/ListNonTranslatedFiles.java, source/net/yacy/utils/translation/TranslatorXliff.java, source/net/yacy/visualization/ChartPlotter.java, source/net/yacy/yacy.java, test/java/net/yacy/cora/document/id/MultiProtocolURLTest.java, test/java/net/yacy/utils/translation/TranslatorXliffTest.java
Sat Apr 09 23:25:05 CEST 2016
by reger
keep and update existing xlf translation master instead of create new
in utility CreateTranslationMasters
+ small fixes in lng's
Changed Files: locales/cn.lng, locales/de.lng, locales/fr.lng, locales/hi.lng, locales/ja.lng, locales/ru.lng, locales/sk.lng, locales/uk.lng, source/net/yacy/utils/translation/CreateTranslationMasters.java, source/net/yacy/utils/translation/TranslatorXliff.java
Mon Apr 04 02:56:19 CEST 2016
by reger
keep ConfigLanguage_p as 1st entry in exported translation file 
+ rem untranslated text & some typo fixes in several translations
(considering to create a translation master file to harmonize entries)
Changed Files: locales/cn.lng, locales/de.lng, locales/fr.lng, locales/gr.lng, locales/hi.lng, locales/it.lng, locales/ja.lng, locales/ru.lng, locales/sk.lng, locales/uk.lng, source/net/yacy/utils/translation/TranslatorXliff.java
Mon Mar 28 23:26:30 CEST 2016
by reger
Introduce a TranslatorXliff wich can read/write xliff from/to internal translation map.
This eases up suggested initatives from http://mantis.tokeek.de/view.php?id=649
Allows longer term also to store translation maps for the htroot files 
in standardized/reuseable xliff format ( http://docs.oasis-open.org/xliff/xliff-core/xliff-core.html ).
+ added test case creating and comparing xliff file with internal custom prop file.
(currently the introduced class is not used in core code)
Changed Files: .classpath, build.xml, lib/xliff-core-1.2-1.1.jar, pom.xml, source/net/yacy/utils/translation/TranslatorXliff.java, test/java/net/yacy/utils/translation/TranslatorXliffTest.java
Fri Mar 25 05:26:48 CET 2016
by reger
load handler for Transparent Proxy on startup only if feature is activated
to save the resources and keep handler chain small if the feature is not used.
+add a warning message on settingsack_p page to restart on first activation
Changed Files: htroot/SettingsAck_p.html, htroot/SettingsAck_p.java, htroot/Settings_ProxyAccess.inc, htroot/Settings_p.java, htroot/Status.java, htroot/api/snapshot.java, locales/de.lng, source/net/yacy/http/AbstractRemoteHandler.java, source/net/yacy/http/Jetty9HttpServerImpl.java, source/net/yacy/http/ProxyCacheHandler.java, source/net/yacy/http/ProxyHandler.java, source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/http/servlets/YaCyProxyServlet.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java
Sat Mar 19 13:13:04 CET 2016
by Andreas
Merge pull request #4 from yacy/master

Get my fork synced #4
Changed Files: .classpath, .gitignore, build.xml, defaults/yacy.init, defaults/yacy.network.freeworld.unit, defaults/yacy.network.zeronet.unit, defaults/yacy.networks, examples/SimpleSearchClient/pom.xml, examples/SimpleSearchClient/src/YaCySearchClient.java, htroot/ConfigBasic.html, htroot/ConfigBasic.java, htroot/IndexExport_p.html, htroot/IndexExport_p.java, htroot/Status.java, htroot/ViewImage.java, htroot/WebStructurePicture_p.java, htroot/api/push_p.java, htroot/api/share.html, htroot/api/share.java, htroot/compare_yacy.java, htroot/env/templates/header.template, htroot/env/templates/submenuConfig.template, htroot/yacysearchitem.java, installYaCyWindowsService.bat, lib/J7Zip-modified.jar, lib/httpclient-4.5.2.jar, lib/httpmime-4.5.2.jar, lib/jcl-over-slf4j-1.7.18.jar, lib/jetty-9.2.15.v20160210.License, lib/jetty-client-9.2.15.v20160210.jar, lib/jetty-continuation-9.2.15.v20160210.jar, lib/jetty-deploy-9.2.15.v20160210.jar, lib/jetty-http-9.2.15.v20160210.jar, lib/jetty-io-9.2.15.v20160210.jar, lib/jetty-jmx-9.2.15.v20160210.jar, lib/jetty-proxy-9.2.15.v20160210.jar, lib/jetty-security-9.2.15.v20160210.jar, lib/jetty-server-9.2.15.v20160210.jar, lib/jetty-servlet-9.2.15.v20160210.jar, lib/jetty-servlets-9.2.15.v20160210.jar, lib/jetty-util-9.2.15.v20160210.jar, lib/jetty-webapp-9.2.15.v20160210.jar, lib/jetty-xml-9.2.15.v20160210.jar, lib/log4j-over-slf4j-1.7.18.jar, lib/lucene-analyzers-common-5.5.0.jar, lib/lucene-analyzers-phonetic-5.5.0.jar, lib/lucene-backward-codecs-5.5.0.jar, lib/lucene-classification-5.5.0.jar, lib/lucene-codecs-5.5.0.jar, lib/lucene-core-5.5.0.jar, lib/lucene-facet-5.5.0.jar, lib/lucene-grouping-5.5.0.jar, lib/lucene-highlighter-5.5.0.jar, lib/lucene-join-5.5.0.jar, lib/lucene-memory-5.5.0.jar, lib/lucene-misc-5.5.0.jar, lib/lucene-queries-5.5.0.jar, lib/lucene-queryparser-5.5.0.jar, lib/lucene-spatial-5.5.0.jar, lib/lucene-suggest-5.5.0.jar, lib/poi-3.14-20160307.jar, lib/poi-3.14.License, lib/poi-scratchpad-3.14-20160307.jar, lib/slf4j-api-1.7.18.jar, lib/slf4j-jdk14-1.7.18.jar, lib/solr-core-5.5.0.jar, lib/solr-solrj-5.5.0.jar, lib/spatial4j-0.5.jar, lib/weupnp-0.1.4.jar, libbuild/J7Zip-modified/pom.xml, libbuild/J7Zip-modified/src/Common/BoolVector.java, libbuild/J7Zip-modified/src/Common/CRC.java, libbuild/J7Zip-modified/src/Common/IntVector.java, libbuild/J7Zip-modified/src/Common/LimitedSequentialInStream.java, libbuild/J7Zip-modified/src/Common/LockedInStream.java, libbuild/J7Zip-modified/src/Common/LockedSequentialInStreamImp.java, libbuild/J7Zip-modified/src/Common/LongVector.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/BindInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/BindPair.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/CoderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/CoderMixer2.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/CoderMixer2ST.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/CoderStreamsInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/FilterCoder.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/OutStreamWithCRC.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/STCoderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/IArchiveExtractCallback.java, libbuild/J7Zip-modified/src/SevenZip/Archive/IInArchive.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/AltCoderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/ArchiveDB.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/ArchiveDatabase.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/ArchiveDatabaseEx.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/BindInfoEx.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/CoderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/ExtractFolderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/FileItem.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Folder.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/FolderOutStream.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Handler.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Header.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/InArchiveInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/InByte2.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/InStream.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/MethodID.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/StreamSwitch.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZipEntry.java, libbuild/J7Zip-modified/src/SevenZip/ArchiveExtractCallback.java, libbuild/J7Zip-modified/src/SevenZip/CRC.java, libbuild/J7Zip-modified/src/SevenZip/Common/InBuffer.java, libbuild/J7Zip-modified/src/SevenZip/Common/LocalCompressProgressInfo.java, libbuild/J7Zip-modified/src/SevenZip/Common/LocalProgress.java, libbuild/J7Zip-modified/src/SevenZip/Common/StreamUtils.java, libbuild/J7Zip-modified/src/SevenZip/Compression/Branch/BCJ2_x86_Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/Branch/BCJ_x86_Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/Copy/Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZ/BinTree.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZ/InWindow.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZ/OutWindow.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZMA/Base.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZMA/Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZMA/Encoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/BitDecoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/BitModel.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/BitTreeDecoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/BitTreeEncoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/Encoder.java, libbuild/J7Zip-modified/src/SevenZip/HRESULT.java, libbuild/J7Zip-modified/src/SevenZip/ICodeProgress.java, libbuild/J7Zip-modified/src/SevenZip/ICompressCoder.java, libbuild/J7Zip-modified/src/SevenZip/ICompressCoder2.java, libbuild/J7Zip-modified/src/SevenZip/ICompressFilter.java, libbuild/J7Zip-modified/src/SevenZip/ICompressGetInStreamProcessedSize.java, libbuild/J7Zip-modified/src/SevenZip/ICompressProgressInfo.java, libbuild/J7Zip-modified/src/SevenZip/ICompressSetInStream.java, libbuild/J7Zip-modified/src/SevenZip/ICompressSetOutStream.java, libbuild/J7Zip-modified/src/SevenZip/ICompressSetOutStreamSize.java, libbuild/J7Zip-modified/src/SevenZip/IInStream.java, libbuild/J7Zip-modified/src/SevenZip/IProgress.java, libbuild/J7Zip-modified/src/SevenZip/J7zip.java, libbuild/J7Zip-modified/src/SevenZip/LzmaAlone.java, libbuild/J7Zip-modified/src/SevenZip/LzmaBench.java, libbuild/J7Zip-modified/src/SevenZip/MyRandomAccessFile.java, libbuild/pom.xml, locales/ja.lng, nbproject/project.xml, pom.xml, source/net/yacy/cora/document/id/AnchorURL.java, source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/crawler/HostBalancer.java, source/net/yacy/crawler/HostQueue.java, source/net/yacy/crawler/LegacyBalancer.java, source/net/yacy/crawler/data/Latency.java, source/net/yacy/crawler/retrieval/Response.java, source/net/yacy/data/WorkTables.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Parser.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/language/Identificator.java, source/net/yacy/document/parser/apkParser.java, source/net/yacy/document/parser/audioTagParser.java, source/net/yacy/document/parser/augment/AugmentParser.java, source/net/yacy/document/parser/bzipParser.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/dwgParser.java, source/net/yacy/document/parser/genericParser.java, source/net/yacy/document/parser/gzipParser.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/images/metadataImageParser.java, source/net/yacy/document/parser/images/svgParser.java, source/net/yacy/document/parser/linkScraperParser.java, source/net/yacy/document/parser/mmParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rdfParser.java, source/net/yacy/document/parser/rdfa/impl/RDFaParser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/sidAudioParser.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/http/servlets/SolrSelectServlet.java, source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/http/servlets/YaCyProxyServlet.java, source/net/yacy/peers/Network.java, source/net/yacy/peers/PeerActions.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/DocumentIndex.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/server/serverClassLoader.java, source/net/yacy/yacy.java, startYACY.bat, startYACY.sh, startYACY_debug.bat, stopYACY.bat
Sat Mar 12 22:08:22 CET 2016
by reger
upd to Solr 5.5.0
Changed Files: .classpath, build.xml, lib/lucene-analyzers-common-5.5.0.jar, lib/lucene-analyzers-phonetic-5.5.0.jar, lib/lucene-backward-codecs-5.5.0.jar, lib/lucene-classification-5.5.0.jar, lib/lucene-codecs-5.5.0.jar, lib/lucene-core-5.5.0.jar, lib/lucene-facet-5.5.0.jar, lib/lucene-grouping-5.5.0.jar, lib/lucene-highlighter-5.5.0.jar, lib/lucene-join-5.5.0.jar, lib/lucene-memory-5.5.0.jar, lib/lucene-misc-5.5.0.jar, lib/lucene-queries-5.5.0.jar, lib/lucene-queryparser-5.5.0.jar, lib/lucene-spatial-5.5.0.jar, lib/lucene-suggest-5.5.0.jar, lib/solr-core-5.5.0.jar, lib/solr-solrj-5.5.0.jar, lib/spatial4j-0.5.jar, nbproject/project.xml, pom.xml
Tue Mar 08 00:53:38 CET 2016
by reger
fix sevenZip parser exception with LZMA BCJ2 format archives,
see http://mantis.tokeek.de/view.php?id=641
+ sanitize some type cast's in modified sevenZip lib
Changed Files: lib/J7Zip-modified.jar, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/BindInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/CoderMixer2ST.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/ArchiveDB.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/ArchiveDatabase.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Folder.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Handler.java, libbuild/J7Zip-modified/src/SevenZip/Compression/Branch/BCJ2_x86_Decoder.java, libbuild/J7Zip-modified/src/SevenZip/ICompressCoder2.java, libbuild/J7Zip-modified/src/SevenZip/ICompressSetInStream.java
Sun Mar 06 03:19:52 CET 2016
by reger
moving the J7Zip-modified source and Maven build to libbuild
from main pom. 
Using source included in j7zip-modified.jar.
This combines all external lib preparation in the libbuild main pom.
Changed Files: libbuild/J7Zip-modified/pom.xml, libbuild/J7Zip-modified/src/Common/BoolVector.java, libbuild/J7Zip-modified/src/Common/CRC.java, libbuild/J7Zip-modified/src/Common/IntVector.java, libbuild/J7Zip-modified/src/Common/LimitedSequentialInStream.java, libbuild/J7Zip-modified/src/Common/LockedInStream.java, libbuild/J7Zip-modified/src/Common/LockedSequentialInStreamImp.java, libbuild/J7Zip-modified/src/Common/LongVector.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/BindInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/BindPair.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/CoderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/CoderMixer2.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/CoderMixer2ST.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/CoderStreamsInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/FilterCoder.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/OutStreamWithCRC.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/STCoderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/IArchiveExtractCallback.java, libbuild/J7Zip-modified/src/SevenZip/Archive/IInArchive.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/AltCoderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/ArchiveDB.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/ArchiveDatabase.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/ArchiveDatabaseEx.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/BindInfoEx.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/CoderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/ExtractFolderInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/FileItem.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Folder.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/FolderOutStream.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Handler.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/Header.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/InArchiveInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/InByte2.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/InStream.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/MethodID.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZip/StreamSwitch.java, libbuild/J7Zip-modified/src/SevenZip/Archive/SevenZipEntry.java, libbuild/J7Zip-modified/src/SevenZip/ArchiveExtractCallback.java, libbuild/J7Zip-modified/src/SevenZip/CRC.java, libbuild/J7Zip-modified/src/SevenZip/Common/InBuffer.java, libbuild/J7Zip-modified/src/SevenZip/Common/LocalCompressProgressInfo.java, libbuild/J7Zip-modified/src/SevenZip/Common/LocalProgress.java, libbuild/J7Zip-modified/src/SevenZip/Common/StreamUtils.java, libbuild/J7Zip-modified/src/SevenZip/Compression/Branch/BCJ2_x86_Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/Branch/BCJ_x86_Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/Copy/Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZ/BinTree.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZ/InWindow.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZ/OutWindow.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZMA/Base.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZMA/Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/LZMA/Encoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/BitDecoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/BitModel.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/BitTreeDecoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/BitTreeEncoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/Decoder.java, libbuild/J7Zip-modified/src/SevenZip/Compression/RangeCoder/Encoder.java, libbuild/J7Zip-modified/src/SevenZip/HRESULT.java, libbuild/J7Zip-modified/src/SevenZip/ICodeProgress.java, libbuild/J7Zip-modified/src/SevenZip/ICompressCoder.java, libbuild/J7Zip-modified/src/SevenZip/ICompressCoder2.java, libbuild/J7Zip-modified/src/SevenZip/ICompressFilter.java, libbuild/J7Zip-modified/src/SevenZip/ICompressGetInStreamProcessedSize.java, libbuild/J7Zip-modified/src/SevenZip/ICompressProgressInfo.java, libbuild/J7Zip-modified/src/SevenZip/ICompressSetInStream.java, libbuild/J7Zip-modified/src/SevenZip/ICompressSetOutStream.java, libbuild/J7Zip-modified/src/SevenZip/ICompressSetOutStreamSize.java, libbuild/J7Zip-modified/src/SevenZip/IInStream.java, libbuild/J7Zip-modified/src/SevenZip/IProgress.java, libbuild/J7Zip-modified/src/SevenZip/J7zip.java, libbuild/J7Zip-modified/src/SevenZip/LzmaAlone.java, libbuild/J7Zip-modified/src/SevenZip/LzmaBench.java, libbuild/J7Zip-modified/src/SevenZip/MyRandomAccessFile.java, libbuild/pom.xml, pom.xml
Wed Feb 24 02:32:12 CET 2016
by reger
upd to Jetty 9.2.15.v20160210
Changed Files: .classpath, build.xml, lib/jetty-9.2.15.v20160210.License, lib/jetty-client-9.2.15.v20160210.jar, lib/jetty-continuation-9.2.15.v20160210.jar, lib/jetty-deploy-9.2.15.v20160210.jar, lib/jetty-http-9.2.15.v20160210.jar, lib/jetty-io-9.2.15.v20160210.jar, lib/jetty-jmx-9.2.15.v20160210.jar, lib/jetty-proxy-9.2.15.v20160210.jar, lib/jetty-security-9.2.15.v20160210.jar, lib/jetty-server-9.2.15.v20160210.jar, lib/jetty-servlet-9.2.15.v20160210.jar, lib/jetty-servlets-9.2.15.v20160210.jar, lib/jetty-util-9.2.15.v20160210.jar, lib/jetty-webapp-9.2.15.v20160210.jar, lib/jetty-xml-9.2.15.v20160210.jar, nbproject/project.xml, pom.xml
Tue Feb 16 02:05:58 CET 2016
by reger
result heuristic (also used in greedy learning mode) to use outbound links if result is full index doc. Otherwise use default loader methode.
- Above brought up that parser start url parameter, declared as AnchorURL uses only methodes of parent object DigestURL (changed parameter declaration accordingly).
Changed Files: htroot/yacysearchitem.java, source/net/yacy/crawler/retrieval/Response.java, source/net/yacy/document/Parser.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/apkParser.java, source/net/yacy/document/parser/audioTagParser.java, source/net/yacy/document/parser/augment/AugmentParser.java, source/net/yacy/document/parser/bzipParser.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/dwgParser.java, source/net/yacy/document/parser/genericParser.java, source/net/yacy/document/parser/gzipParser.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/images/metadataImageParser.java, source/net/yacy/document/parser/images/svgParser.java, source/net/yacy/document/parser/linkScraperParser.java, source/net/yacy/document/parser/mmParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rdfParser.java, source/net/yacy/document/parser/rdfa/impl/RDFaParser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/sidAudioParser.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java
Tue Feb 09 20:46:44 CET 2016
by luc
Created ViewFavicon class specialized in favicon viewing.

Main image processing is now in ImageViewer, used by both ViewImage and
ViewFavicon.

Fixed URIMetadataNode.getFavicon to use non-standard icons with no size
ass fallback.
Changed Files: htroot/ViewFavicon.java, htroot/ViewImage.java, htroot/yacysearchitem.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/visualization/ImageViewer.java, test/ImageViewerPerfTest.java, test/ImageViewerTest.java, test/java/yacysearchitemTest.java
Sat Feb 06 21:32:54 CET 2016
by reger
upd to Solr 5.4.1
Changed Files: .classpath, build.xml, lib/lucene-analyzers-common-5.4.1.jar, lib/lucene-analyzers-phonetic-5.4.1.jar, lib/lucene-backward-codecs-5.4.1.jar, lib/lucene-classification-5.4.1.jar, lib/lucene-codecs-5.4.1.jar, lib/lucene-core-5.4.1.jar, lib/lucene-facet-5.4.1.jar, lib/lucene-grouping-5.4.1.jar, lib/lucene-highlighter-5.4.1.jar, lib/lucene-join-5.4.1.jar, lib/lucene-memory-5.4.1.jar, lib/lucene-misc-5.4.1.jar, lib/lucene-queries-5.4.1.jar, lib/lucene-queryparser-5.4.1.jar, lib/lucene-spatial-5.4.1.jar, lib/lucene-suggest-5.4.1.jar, lib/solr-core-5.4.1.jar, lib/solr-solrj-5.4.1.jar, nbproject/project.xml, pom.xml
Tue Feb 02 09:57:54 CET 2016
by luc
Improved HTML icons indexing and rendering in search results.

See http://mantis.tokeek.de/view.php?id=629
Changed Files: defaults/solr.collection.schema, htroot/api/getpageinfo.java, htroot/api/getpageinfo_p.java, htroot/yacysearchitem.java, source/net/yacy/document/Document.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/IconEntry.java, source/net/yacy/document/parser/html/IconLinkRelations.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/CollectionSchema.java, test/java/net/yacy/document/parser/html/ContentScraperTest.java, test/java/net/yacy/document/parser/html/IconEntryTest.java, test/java/net/yacy/kelondro/data/meta/URIMetadataNodeTest.java, test/java/yacysearchitemTest.java
Mon Feb 01 10:52:49 CET 2016
by Andreas
Merge pull request #3 from yacy/master

Get my fork synced #3
Changed Files: .classpath, build.xml, htroot/yacysearchtrailer.json, lib/fontbox-1.8.11.License, lib/fontbox-1.8.11.jar, lib/jempbox-1.8.11.License, lib/jempbox-1.8.11.jar, lib/pdfbox-1.8.11.License, lib/pdfbox-1.8.11.jar, nbproject/project.xml, pom.xml, source/net/yacy/search/Switchboard.java
Sun Jan 31 00:30:39 CET 2016
by reger
upd to pdfbox 1.8.11
Changed Files: .classpath, build.xml, lib/fontbox-1.8.11.License, lib/fontbox-1.8.11.jar, lib/jempbox-1.8.11.License, lib/jempbox-1.8.11.jar, lib/pdfbox-1.8.11.License, lib/pdfbox-1.8.11.jar, nbproject/project.xml, pom.xml
Wed Jan 27 17:34:33 CET 2016
by Andreas
Merge pull request #2 from yacy/master

Get my fork synced #2
Changed Files: .classpath, build.xml, defaults/yacy.init, htroot/Autocrawl_p.html, htroot/Autocrawl_p.java, htroot/ConfigPortal.html, htroot/env/templates/submenuIndexCreate.template, htroot/yacysearch.java, lib/webcat-swf-0.1.jar, lib/webcat-swf.License, libbuild/GitRevMavenTask/pom.xml, libbuild/GitRevMavenTask/src/GitRevMavenTask.java, libbuild/WebCat-swf/JavaSWF2-BSD.LICENSE.txt, libbuild/WebCat-swf/pom.xml, libbuild/WebCat-swf/readme.txt, libbuild/WebCat-swf/src/com/anotherbigidea/flash/SWFActionCodes.java, libbuild/WebCat-swf/src/com/anotherbigidea/flash/SWFConstants.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ActionParser.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ActionTextWriter.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ActionWriter.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Actions.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/AlphaColor.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/AlphaTransform.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Base64.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ButtonRecord.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ButtonRecord2.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Color.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ColorTransform.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ExportedSymbol.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/FillStyle.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Font.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/FontDefinition.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Frame.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ImportedSymbol.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/InStream.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Instance.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/LineStyle.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Matrix.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Movie.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/OutStream.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Placement.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Rect.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWF2HTML.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFActions.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFHeader.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFReader.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFShape.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFSpriteTagTypes.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFTagTypes.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFTagTypesImpl.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFTags.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFText.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFVectors.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFWriter.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SaxHandlerBase.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Shape.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SoundInfo.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Style.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Symbol.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/TagParser.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/TagWriter.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/TimeLine.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Transform.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/XMLWriter.java, libbuild/pom.xml, locales/de.lng, locales/fr.lng, nbproject/project.xml, pom.xml, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/crawler/retrieval/HTTPLoader.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/search/AutoSearch.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java, test/java/net/yacy/cora/document/id/DigestURLTest.java, test/java/net/yacy/cora/document/id/MultiProtocolURLTest.java, test/java/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnectorTest.java, test/java/net/yacy/cora/protocol/HeaderFrameworkTest.java, test/java/net/yacy/cora/protocol/ResponseHeaderTest.java, test/java/net/yacy/cora/util/NumberToolsTest.java, test/java/net/yacy/crawler/HostQueueTest.java, test/java/net/yacy/data/wiki/WikiCodeTest.java, test/java/net/yacy/document/ParserTest.java, test/java/net/yacy/document/WordTokenizerTest.java, test/java/net/yacy/document/parser/htmlParserTest.java, test/java/net/yacy/document/parser/images/genericImageParserTest.java, test/java/net/yacy/document/parser/images/metadataImageParserTest.java, test/java/net/yacy/document/parser/pdfParserTest.java, test/java/net/yacy/kelondro/io/RecordsTest.java, test/java/net/yacy/kelondro/util/MemoryControlTest.java, test/java/net/yacy/peers/operation/yacyVersionTest.java, test/java/net/yacy/search/query/QueryGoalTest.java, test/java/net/yacy/search/snippet/TextSnippetTest.java
Wed Jan 20 00:58:29 CET 2016
by reger
add support of compressed swf to swfParser
from JavaSWF2 (source compatible to WebCat).
Moved swf file signature check to parser
Changed use of synced vector to list swf InStream
Changed Files: lib/webcat-swf-0.1.jar, libbuild/WebCat-swf/JavaSWF2-BSD.LICENSE.txt, libbuild/WebCat-swf/pom.xml, libbuild/WebCat-swf/src/com/anotherbigidea/flash/SWFActionCodes.java, libbuild/WebCat-swf/src/com/anotherbigidea/flash/SWFConstants.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ActionParser.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ActionTextWriter.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ActionWriter.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/FillStyle.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Font.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/InStream.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Movie.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFReader.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFSaxParser.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFWriter.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Shape.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/TagParser.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/TagWriter.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Transform.java, source/net/yacy/document/parser/swfParser.java
Mon Jan 18 22:41:49 CET 2016
by reger
import WebCat swf parser custom source package
This package is not available as jar (used jar is a custom compile as we 
use just a portion of the package) 
WebCat package is not maintained. To be able to fix bugs, source extract 
of swf parser imported here.
Changed Files: libbuild/WebCat-swf/pom.xml, libbuild/WebCat-swf/readme.txt, libbuild/WebCat-swf/src/pt/tumba/parser/DocFilter.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ActionParser.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ActionTextWriter.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ActionWriter.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Actions.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/AlphaColor.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/AlphaTransform.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Base64.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ButtonRecord.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ButtonRecord2.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Color.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ColorTransform.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ExportedSymbol.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/FillStyle.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Font.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/FontDefinition.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Frame.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ImportedSymbol.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/InStream.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Instance.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/LineStyle.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Matrix.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Movie.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/OutStream.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Placement.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Rect.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWF2HTML.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFActionCodes.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFActions.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFConstants.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFHeader.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFReader.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFSaxParser.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFShape.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFSpriteTagTypes.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFTagTypes.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFTagTypesImpl.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFTags.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFText.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFVectors.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFWriter.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SaxHandlerBase.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SaxParserBase.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Shape.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SoundInfo.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Style.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Symbol.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/TagParser.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/TagWriter.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/TimeLine.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Transform.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/XMLWriter.java, libbuild/pom.xml
Sat Jan 16 19:22:27 CET 2016
by reger
move test classes to test/java (subdirectory as in Maven standard subdir layout)
because ViewImage*Test.java breaks test run
Changed Files: build.xml, pom.xml, test/java/net/yacy/cora/document/id/DigestURLTest.java, test/java/net/yacy/cora/document/id/MultiProtocolURLTest.java, test/java/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnectorTest.java, test/java/net/yacy/cora/protocol/HeaderFrameworkTest.java, test/java/net/yacy/cora/protocol/ResponseHeaderTest.java, test/java/net/yacy/cora/util/NumberToolsTest.java, test/java/net/yacy/crawler/HostQueueTest.java, test/java/net/yacy/data/wiki/WikiCodeTest.java, test/java/net/yacy/document/ParserTest.java, test/java/net/yacy/document/WordTokenizerTest.java, test/java/net/yacy/document/parser/htmlParserTest.java, test/java/net/yacy/document/parser/images/genericImageParserTest.java, test/java/net/yacy/document/parser/images/metadataImageParserTest.java, test/java/net/yacy/document/parser/pdfParserTest.java, test/java/net/yacy/kelondro/io/RecordsTest.java, test/java/net/yacy/kelondro/util/MemoryControlTest.java, test/java/net/yacy/peers/operation/yacyVersionTest.java, test/java/net/yacy/search/query/QueryGoalTest.java, test/java/net/yacy/search/snippet/TextSnippetTest.java
Wed Jan 06 14:55:54 CET 2016
by Michael Peter Christen
Merge pull request #38 from luccioman/master

Refactoring : use StandardCharsets instead of hardcoded charset names
Changed Files: htroot/CacheResource_p.java, htroot/Vocabulary_p.java, htroot/api/ymarks/import_ymark.java, source/net/yacy/contentcontrol/SMWListSyncThread.java, source/net/yacy/cora/document/WordCache.java, source/net/yacy/cora/document/encoding/UTF8.java, source/net/yacy/cora/federate/opensearch/OpenSearchConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/responsewriter/GSAResponseWriter.java, source/net/yacy/cora/geo/GeonamesLocation.java, source/net/yacy/cora/geo/OpenGeoDBLocation.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/cora/storage/Files.java, source/net/yacy/cora/storage/KeyList.java, source/net/yacy/crawler/retrieval/Response.java, source/net/yacy/data/BookmarkHelper.java, source/net/yacy/data/Translator.java, source/net/yacy/data/wiki/AbstractWikiParser.java, source/net/yacy/document/Document.java, source/net/yacy/document/LibraryProvider.java, source/net/yacy/document/ProbabilisticClassifier.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/document/content/dao/PhpBB3Dao.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/parser/apkParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/ScraperInputStream.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/images/svgParser.java, source/net/yacy/document/parser/mmParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/sidAudioParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/http/servlets/SolrServlet.java, source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/http/servlets/YaCyProxyServlet.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/search/query/AccessTracker.java, source/net/yacy/search/query/QueryGoal.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/server/http/ChunkedInputStream.java, source/net/yacy/server/http/TemplateEngine.java, test/net/yacy/document/parser/htmlParserTest.java
Wed Jan 06 11:36:57 CET 2016
by reger
update on translation files
- delete removed servlets
 AugmentedBrowsingFilters_p.html (de)
 CrawlStartIntranet_p.html (de)
 IndexCreateWWW***Queue_p.html (de)
 Ranking_p.html (de)
- add
 IndexCreateQueues_p.html
- rename
 Settings_Http.inc -> Settings_ProxyAccess.inc
 Language_p.html -> ConfigLanguage_p.html

Changed Files: locales/cn.lng, locales/de.lng, locales/fr.lng, locales/gr.lng, locales/it.lng, locales/ru.lng, locales/sk.lng, locales/uk.lng
Tue Jan 05 23:37:05 CET 2016
by luc
Refactoring : use StandardCharsets constants instead of hard-coded
charset names.
Changed Files: htroot/CacheResource_p.java, htroot/Vocabulary_p.java, htroot/api/ymarks/import_ymark.java, source/net/yacy/contentcontrol/SMWListSyncThread.java, source/net/yacy/cora/document/WordCache.java, source/net/yacy/cora/document/encoding/UTF8.java, source/net/yacy/cora/federate/opensearch/OpenSearchConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/responsewriter/GSAResponseWriter.java, source/net/yacy/cora/geo/GeonamesLocation.java, source/net/yacy/cora/geo/OpenGeoDBLocation.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/cora/storage/Files.java, source/net/yacy/cora/storage/KeyList.java, source/net/yacy/crawler/retrieval/Response.java, source/net/yacy/data/BookmarkHelper.java, source/net/yacy/data/Translator.java, source/net/yacy/data/wiki/AbstractWikiParser.java, source/net/yacy/document/Document.java, source/net/yacy/document/LibraryProvider.java, source/net/yacy/document/ProbabilisticClassifier.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/document/content/dao/PhpBB3Dao.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/parser/apkParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/ScraperInputStream.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/images/svgParser.java, source/net/yacy/document/parser/mmParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/sidAudioParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/http/servlets/SolrServlet.java, source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/http/servlets/YaCyProxyServlet.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/search/query/AccessTracker.java, source/net/yacy/search/query/QueryGoal.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/server/http/ChunkedInputStream.java, source/net/yacy/server/http/TemplateEngine.java, test/net/yacy/document/parser/htmlParserTest.java
Sat Jan 02 21:07:50 CET 2016
by sixcooler
bump to solr/lucene 5.4.0
Changed Files: .classpath, build.xml, lib/lucene-analyzers-common-5.4.0.jar, lib/lucene-analyzers-phonetic-5.4.0.jar, lib/lucene-backward-codecs-5.4.0.jar, lib/lucene-classification-5.4.0.jar, lib/lucene-codecs-5.4.0.jar, lib/lucene-core-5.4.0.jar, lib/lucene-facet-5.4.0.jar, lib/lucene-grouping-5.4.0.jar, lib/lucene-highlighter-5.4.0.jar, lib/lucene-join-5.4.0.jar, lib/lucene-memory-5.4.0.jar, lib/lucene-misc-5.4.0.jar, lib/lucene-queries-5.4.0.jar, lib/lucene-queryparser-5.4.0.jar, lib/lucene-spatial-5.4.0.jar, lib/lucene-suggest-5.4.0.jar, lib/solr-core-5.4.0.jar, lib/solr-solrj-5.4.0.jar, nbproject/project.xml, pom.xml, source/net/yacy/cora/federate/solr/instance/EmbeddedInstance.java
Mon Dec 28 23:20:42 CET 2015
by sixcooler
Merge remote-tracking branch 'choose_remote_name/master'
Changed Files: build.xml, defaults/solr/schema.xml, defaults/yacy.init, htroot/ConfigPortal.html, htroot/ConfigPortal.java, htroot/api/schema.xml, source/net/yacy/kelondro/util/MemoryControl.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/RemoteSearch.java, source/net/yacy/search/SwitchboardConstants.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/query/SearchEventCache.java
Fri Dec 25 01:08:51 CET 2015
by reger
apply default css styles (class btn) to submit buttons 
Changed Files: htroot/ConfigUpdate_p.html, htroot/ContentIntegrationPHPBB3_p.html, htroot/IndexImportOAIPMHList_p.html, htroot/IndexImportOAIPMH_p.html, htroot/Load_RSS_p.html, htroot/Messages_p.html, htroot/Network.html, htroot/RankingRWI_p.html, htroot/ServerScannerList.html, htroot/Settings_Crawler.inc, htroot/Settings_MessageForwarding.inc, htroot/Settings_Proxy.inc, htroot/Settings_Seed.inc, htroot/Settings_Seed_UploadFile.inc, htroot/Settings_Seed_UploadFtp.inc, htroot/Settings_Seed_UploadScp.inc, htroot/Settings_ServerAccess.inc, htroot/Status.html, htroot/Surftips.html, htroot/Tables_p.html, htroot/Threaddump_p.html, htroot/User.html, htroot/ViewFile.html, htroot/compare_yacy.html, htroot/sharedBlacklist_p.html
Fri Dec 25 00:53:16 CET 2015
by reger
upd to ImageIO 3.2.1
Changed Files: .classpath, build.xml, lib/common-image-3.2.1.jar, lib/common-io-3.2.1.jar, lib/common-lang-3.2.1.jar, lib/imageio-bmp-3.2.1.jar, lib/imageio-core-3.2.1.jar, lib/imageio-metadata-3.2.1.jar, lib/imageio-tiff-3.2.1.jar, nbproject/project.xml, pom.xml
Wed Dec 23 00:04:40 CET 2015
by reger
apply more default css styles (class btn) to submit buttons
Changed Files: htroot/AugmentedBrowsing_p.html, htroot/ContentAnalysis_p.html, htroot/DictionaryLoader_p.html, htroot/IndexCreateParserErrors_p.html, htroot/IndexFederated_p.html, htroot/IndexSchema_p.html, htroot/Load_MediawikiWiki.html, htroot/Load_PHPBB3.html, htroot/PerformanceQueues_p.html, htroot/ProxyIndexingMonitor_p.html, htroot/RankingSolr_p.html, htroot/WatchWebStructure_p.html
Tue Nov 24 21:35:58 CET 2015
by reger
upd to Jetty v9.2.14.v20151106
Changed Files: .classpath, build.xml, lib/jetty-9.2.14.v20151106.License, lib/jetty-client-9.2.14.v20151106.jar, lib/jetty-continuation-9.2.14.v20151106.jar, lib/jetty-deploy-9.2.14.v20151106.jar, lib/jetty-http-9.2.14.v20151106.jar, lib/jetty-io-9.2.14.v20151106.jar, lib/jetty-jmx-9.2.14.v20151106.jar, lib/jetty-proxy-9.2.14.v20151106.jar, lib/jetty-security-9.2.14.v20151106.jar, lib/jetty-server-9.2.14.v20151106.jar, lib/jetty-servlet-9.2.14.v20151106.jar, lib/jetty-servlets-9.2.14.v20151106.jar, lib/jetty-util-9.2.14.v20151106.jar, lib/jetty-webapp-9.2.14.v20151106.jar, lib/jetty-xml-9.2.14.v20151106.jar, nbproject/project.xml, pom.xml
Fri Nov 20 09:38:16 CET 2015
by luc
BMP and ICO image formats support : integrated /haraldk/TwelveMonkeys
imageio-bmp-3.2 library.

 - better BMP format flavours support
 - handle PNG encoded icons
 - handle transparency
 
Added some javadoc url references to .classpath
Changed Files: .classpath, build.xml, lib/imageio-bmp-3.2.jar, pom.xml, source/net/yacy/document/ImageParser.java, source/net/yacy/document/parser/images/bmpParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/images/icoParser.java
Tue Nov 17 22:27:07 CET 2015
by reger
avoid File.deleteOnExit() on temp files
JVM registers each file in a list regardless of already deleted and never
cleans up the list during runtime.
This accumulates to a considerable amount of mem during large crawls and/or
long uptime.
To tackle this, all temp files are now created in a subdir of java.io.tmpdir 
and the jvm tmpdir property is set to this subdir, which is deleted by
code on shutdown.
Additionally let pdfParser use this tmp subdir too.
Changed Files: source/net/yacy/document/parser/apkParser.java, source/net/yacy/document/parser/audioTagParser.java, source/net/yacy/document/parser/bzipParser.java, source/net/yacy/document/parser/gzipParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/peers/SeedDB.java, source/net/yacy/yacy.java
Sat Nov 14 01:46:25 CET 2015
by reger
upd to TwelveMonkeys ImageIO 3.2
Changed Files: .classpath, build.xml, lib/common-image-3.2.jar, lib/common-io-3.2.jar, lib/common-lang-3.2.jar, lib/imageio-core-3.2.jar, lib/imageio-metadata-3.2.jar, lib/imageio-tiff-3.2.jar, nbproject/project.xml, pom.xml
Thu Nov 05 09:45:19 CET 2015
by luc
Refactoring : default favicon and image processing errors.
 - moved default favicon processing from ViewImage to
yacysearchitem.html : when previewing ico image search results we don't
want a default favicon be displayed
 - throw an IOException ending in a HTTP 500 error when image processing
fails, rather than returning a null result : behavior is more consistent
accross browsers (for exempla Chrome and Firefox), especially with new
default favicon display system
Changed Files: htroot/ViewImage.java, htroot/env/base.css, htroot/env/oldie.css, htroot/yacy/ui/css/base.css, htroot/yacy/ui/css/widget.css, htroot/yacysearchitem.html
Sat Oct 31 23:09:03 CET 2015
by reger
differentiate api call getLocalPort() from getConfigInt()
Changed Files: htroot/ConfigBasic.java, htroot/ConfigPortal.java, htroot/ConfigSearchBox.java, htroot/CrawlStartScanner_p.java, htroot/Load_MediawikiWiki.java, htroot/Load_PHPBB3.java, htroot/SettingsAck_p.java, htroot/Settings_p.java, htroot/Table_API_p.java, htroot/api/push_p.java, htroot/opensearchdescription.java, htroot/yacysearch.java, htroot/yacysearch_location.java, source/net/yacy/gui/Tray.java, source/net/yacy/http/Jetty9HttpServerImpl.java, source/net/yacy/peers/SeedDB.java, source/net/yacy/search/Switchboard.java, source/net/yacy/server/serverSwitch.java, source/net/yacy/yacy.java
Mon Oct 19 00:53:10 CEST 2015
by reger
upd to jetty 9.2.13.v20150730
Changed Files: .classpath, build.xml, lib/jetty-9.2.13.v20150730.License, lib/jetty-client-9.2.13.v20150730.jar, lib/jetty-continuation-9.2.13.v20150730.jar, lib/jetty-deploy-9.2.13.v20150730.jar, lib/jetty-http-9.2.13.v20150730.jar, lib/jetty-io-9.2.13.v20150730.jar, lib/jetty-jmx-9.2.13.v20150730.jar, lib/jetty-proxy-9.2.13.v20150730.jar, lib/jetty-security-9.2.13.v20150730.jar, lib/jetty-server-9.2.13.v20150730.jar, lib/jetty-servlet-9.2.13.v20150730.jar, lib/jetty-servlets-9.2.13.v20150730.jar, lib/jetty-util-9.2.13.v20150730.jar, lib/jetty-webapp-9.2.13.v20150730.jar, lib/jetty-xml-9.2.13.v20150730.jar, nbproject/project.xml, pom.xml
Sun Oct 18 19:53:39 CEST 2015
by reger
upd httpclient-4.5.1, httpmime-4.5.1, httpcore-4.4.3, commons-compress-1.10
Changed Files: .classpath, build.xml, lib/commons-compress-1.10.License, lib/commons-compress-1.10.jar, lib/httpclient-4.5.1.jar, lib/httpcore-4.4.3.License, lib/httpcore-4.4.3.jar, lib/httpmime-4.5.1.jar, nbproject/project.xml, pom.xml
Sun Oct 18 05:51:01 CEST 2015
by reger
implement ajax crawling scheme for ajax sites which adhere to the proposed use of hash-bangs to provide html content
see freshly deprecated https://developers.google.com/webmasters/ajax-crawling/
Implementation improves parsing of the homepage (ajax page) which uses metatag "fragment" in header and parses supplied html snapshot instead of mostly empty ajax/scripted page.
Implementation supports also hash-bang urls (url with anchor starting with ! like  ...path#!hashfragment) but our crawler filters it
(use of hash-bang is controversly discussed and proposal is deprecated, makes no sense to adjust the crawler, but as long as it is used by some sites the minor change/improvement in htmlparser is good for some time).
Quick - how does it work
- if metatag fragment with content "!" is found
   - htmlparser tries to get content of htmls snapshot (using a different url)
   - htmlparser returns 2 documents (original url and snapshot content - but using same original url)
- after parsing result documents are joined (and stored to index containing content also from snapshot page... as the original ajax page contains typically no parseable html content)
Changed Files: source/net/yacy/document/parser/htmlParser.java
Thu Oct 15 10:06:51 CEST 2015
by luc
Integrated haraldk/TwelveMonkeys library to first add TIF image format
support.
Changed Files: .classpath, build.xml, htroot/ViewImage.java, lib/common-image-3.1.2.jar, lib/common-io-3.1.2.jar, lib/common-lang-3.1.2.jar, lib/imageio-core-3.1.2.jar, lib/imageio-metadata-3.1.2.jar, lib/imageio-tiff-3.1.2.jar, lib/servlet-3.1.2.jar, pom.xml, source/net/yacy/document/ImageParser.java
Sat Oct 03 23:20:33 CEST 2015
by reger
upd to solr/lucene 5.3.1
Changed Files: .classpath, build.xml, htroot/yacysearchtrailer.java, lib/lucene-analyzers-common-5.3.1.jar, lib/lucene-analyzers-phonetic-5.3.1.jar, lib/lucene-backward-codecs-5.3.1.jar, lib/lucene-classification-5.3.1.jar, lib/lucene-codecs-5.3.1.jar, lib/lucene-core-5.3.1.jar, lib/lucene-facet-5.3.1.jar, lib/lucene-grouping-5.3.1.jar, lib/lucene-highlighter-5.3.1.jar, lib/lucene-join-5.3.1.jar, lib/lucene-memory-5.3.1.jar, lib/lucene-misc-5.3.1.jar, lib/lucene-queries-5.3.1.jar, lib/lucene-queryparser-5.3.1.jar, lib/lucene-spatial-5.3.1.jar, lib/lucene-suggest-5.3.1.jar, lib/solr-core-5.3.1.jar, lib/solr-solrj-5.3.1.jar, nbproject/project.xml, pom.xml, source/net/yacy/cora/federate/solr/responsewriter/EnhancedXMLResponseWriter.java, source/net/yacy/cora/federate/solr/responsewriter/HTMLResponseWriter.java, source/net/yacy/crawler/HostQueue.java, source/net/yacy/search/query/QueryModifier.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/server/serverObjects.java
Thu Sep 24 13:50:23 CEST 2015
by Michael Peter Christen
Merge pull request #14 from luccioman/master

Translator refactoring : no more regular expression processing
Changed Files: htroot/Network.html, htroot/Table_API_p.html, htroot/Table_RobotsTxt_p.html, htroot/YMarks.html, htroot/env/style.java, htroot/yacy/ui/yacyui-admin.html, locales/cn.lng, locales/de.lng, locales/fr.lng, locales/gr.lng, locales/hi.lng, locales/it.lng, locales/ru.lng, locales/sk.lng, locales/uk.lng, source/net/yacy/data/Translator.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/utils/translation/ExtensionsFileFilter.java, source/net/yacy/utils/translation/ListNonTranslatedFiles.java, source/net/yacy/utils/translation/TranslateAll.java, source/net/yacy/utils/translation/TranslateAllToOneLanguage.java, source/net/yacy/utils/translation/TranslatorUtil.java
Fri Sep 11 17:20:11 CEST 2015
by luccioman
Translator refactoring : to simplify locale files writing, process keys
as simple string and no more as regular expressions.
Updated all locale files to adapt to refectored Translator : removed
useless escaped characters and did minor corrections.
Performed minor syntax corrections on some html source files.
Added an util to translate all html source files with all locales
without launching full YaCy application.
Corrected main arguments parsing on other translation utils.
Changed Files: htroot/Network.html, htroot/Table_API_p.html, htroot/Table_RobotsTxt_p.html, htroot/YMarks.html, htroot/yacy/ui/yacyui-admin.html, locales/cn.lng, locales/de.lng, locales/fr.lng, locales/gr.lng, locales/hi.lng, locales/it.lng, locales/ru.lng, locales/sk.lng, locales/uk.lng, source/net/yacy/data/Translator.java, source/net/yacy/utils/translation/ExtensionsFileFilter.java, source/net/yacy/utils/translation/ListNonTranslatedFiles.java, source/net/yacy/utils/translation/TranslateAll.java, source/net/yacy/utils/translation/TranslateAllToOneLanguage.java, source/net/yacy/utils/translation/TranslatorUtil.java
Sat Sep 05 14:12:17 CEST 2015
by Michael Peter Christen
when many crawl queues are generated, this NPE can occur; probably
caused as concurrency issue:
W 2015/09/05 14:09:10 ConcurrentLog java.lang.NullPointerException
java.lang.NullPointerException
	at java.util.TreeMap.rotateRight(TreeMap.java:2239)
	at java.util.TreeMap.fixAfterInsertion(TreeMap.java:2271)
	at java.util.TreeMap.put(TreeMap.java:582)
	at net.yacy.kelondro.table.Table.<init>(Table.java:235)
	at net.yacy.crawler.HostQueue.openStack(HostQueue.java:229)
	at net.yacy.crawler.HostQueue.getStack(HostQueue.java:204)
	at net.yacy.crawler.HostQueue.push(HostQueue.java:397)
	at net.yacy.crawler.HostBalancer.push(HostBalancer.java:237)
	at net.yacy.crawler.data.NoticedURL.push(NoticedURL.java:184)
	at net.yacy.crawler.CrawlStacker.stackCrawl(CrawlStacker.java:355)
	at net.yacy.crawler.CrawlStacker.job(CrawlStacker.java:134)
	at sun.reflect.GeneratedMethodAccessor6.invoke(Unknown Source)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at
net.yacy.kelondro.workflow.InstantBlockingThread.job(InstantBlockingThread.java:101)
	at
net.yacy.kelondro.workflow.AbstractBlockingThread.run(AbstractBlockingThread.java:82)
	at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)
	
Changed Files: source/net/yacy/kelondro/table/Table.java
Mon Aug 31 20:24:41 CEST 2015
by sixcooler
fight the fieldcache by usind DocValues: in Solr-5.x the fieldcache has
moved and was not cleared anymore. This results in an huge fieldcache.
(http://lucene.apache.org/#highlights-of-the-lucene-release-include
https://issues.apache.org/jira/browse/LUCENE-5666)
Here I try to use DovValues where it is possible.
For this I used the Api-Scheme as new basis fr the Solr-Schema.
This needs at least a complete optimization of the Solr-Index to get a
smaller FieldCache.
Everything that is indexed with these setting will not use the
Fieldcache at all.
Changed Files: defaults/solr/schema.xml, htroot/api/schema.java, htroot/api/schema.xml, source/net/yacy/cora/federate/solr/SchemaDeclaration.java, source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/schema/CollectionSchema.java, source/net/yacy/search/schema/WebgraphSchema.java
Mon Aug 10 14:27:44 CEST 2015
by Michael Peter Christen
added a new facet type based on a probabilistic classifier using
bayesian filters. This can be used to classify documents during
indexing-time using a pre-definied bayesian filter.

New wordings:
- a context is a class where different categories are possible. The
context name is equal to a facet name.
- a category is a facet type within a facet navigation. Each context
must have several categories, at least one custom name (things you want
to discover) and one with the exact name "negative".

To use this, you must do:
- for each context, you must create a directory within
DATA/CLASSIFICATION with the name of the context (the facet name)
- within each context directory, you must create text files with one
document each per line for every categroy. One of these categories MUST
have the name 'negative.txt'.

Then, each new document is classified to match within one of the given
categories for each context.
Changed Files: defaults/yacy.init, htroot/js/yacysearch.js, htroot/yacysearch.java, source/net/yacy/cora/language/synonyms/AutotaggingLibrary.java, source/net/yacy/cora/lod/vocabulary/Tagging.java, source/net/yacy/document/Document.java, source/net/yacy/document/ProbabilisticClassifier.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/schema/CollectionConfiguration.java
Mon Aug 03 05:37:34 CEST 2015
by Michael Peter Christen
removed warnings
Changed Files: htroot/Bookmarks.java, htroot/ConfigHTCache_p.java, htroot/IndexControlRWIs_p.java, htroot/ViewFile.java, htroot/yacysearch.java, htroot/yacysearchtrailer.java, source/net/yacy/cora/federate/solr/instance/EmbeddedInstance.java, source/net/yacy/cora/federate/solr/responsewriter/EnhancedXMLResponseWriter.java, source/net/yacy/cora/federate/solr/responsewriter/HTMLResponseWriter.java, source/net/yacy/crawler/Balancer.java, source/net/yacy/crawler/RecrawlBusyThread.java, source/net/yacy/data/BookmarksDB.java, source/net/yacy/data/WorkTables.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/kelondro/util/OS.java, source/net/yacy/peers/Protocol.java, source/net/yacy/search/query/QueryModifier.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/ranking/ReferenceOrder.java, source/net/yacy/utils/cryptbig.java
Sun Aug 02 14:52:41 CEST 2015
by Michael Peter Christen
a collection of search query enhancements:
- fixed superfluous space in query field list
- fixed filter query logic
- removed look-ahead query which caused that each new search page
submitted two solr queries
- fixed random solr result orders in case that the solr score was equal:
this was then re-ordered by YaCy using the document hash which came from
the solr object and that appeared to be random. Now the hash of the url
is used and the score is additionally modified by the url length to
prevent that this particular case appears at all.
Changed Files: source/net/yacy/cora/federate/solr/Ranking.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/RemoteSearch.java, source/net/yacy/search/query/QueryModifier.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java
Sat Aug 01 00:25:40 CEST 2015
by reger
upd to pdfbox-1.8.10
Changed Files: .classpath, build.xml, lib/fontbox-1.8.10.License, lib/fontbox-1.8.10.jar, lib/jempbox-1.8.10.License, lib/jempbox-1.8.10.jar, lib/pdfbox-1.8.10.License, lib/pdfbox-1.8.10.jar, nbproject/project.xml, pom.xml
Thu Jul 30 00:16:09 CEST 2015
by reger
upd to Solr-5.2.1
Changed Files: .classpath, build.xml, lib/lucene-analyzers-common-5.2.1.jar, lib/lucene-analyzers-phonetic-5.2.1.jar, lib/lucene-backward-codecs-5.2.1.jar, lib/lucene-classification-5.2.1.jar, lib/lucene-codecs-5.2.1.jar, lib/lucene-core-5.2.1.jar, lib/lucene-facet-5.2.1.jar, lib/lucene-grouping-5.2.1.jar, lib/lucene-highlighter-5.2.1.jar, lib/lucene-join-5.2.1.jar, lib/lucene-memory-5.2.1.jar, lib/lucene-misc-5.2.1.jar, lib/lucene-queries-5.2.1.jar, lib/lucene-queryparser-5.2.1.jar, lib/lucene-spatial-5.2.1.jar, lib/lucene-suggest-5.2.1.jar, lib/solr-core-5.2.1.jar, lib/solr-solrj-5.2.1.jar, nbproject/project.xml, pom.xml
Wed Jul 29 23:30:05 CEST 2015
by reger
move java environment parameter setting disabling SNI (Server Name Indicator) support for https connections from code to startup script allowing admin to ~easy/transparent alter the YaCy default FALSE setting.
Background: some user report problem with connecting/crawling some sites via https which require SNI support (by default switched off in YaCy). On the other hand systems not demanding SNI support are sometimes not properly configured and due to a bug/feature in java 1.7 connection is aborted. The later is more often the case, so the default is still fine. With the java start parameter expert user can no alter the startparameter to -Djsse.enableSNIExtension=true (java default) if they crawl more hosts requiring SNI support.
The alternative to let YaCy try both during https handshake (deep inside the httpclient) is not pursut at this time.
Changed Files: source/net/yacy/yacy.java, startYACY.bat, startYACY.sh, startYACY_debug.bat
Thu Jul 02 18:41:13 CEST 2015
by Ryszard Go?
Use language-detection library for increased accuracy
Changed Files: .classpath, build.xml, langdetect/af, langdetect/ar, langdetect/bg, langdetect/bn, langdetect/cs, langdetect/da, langdetect/de, langdetect/el, langdetect/en, langdetect/es, langdetect/et, langdetect/fa, langdetect/fi, langdetect/fr, langdetect/gu, langdetect/he, langdetect/hi, langdetect/hr, langdetect/hu, langdetect/id, langdetect/it, langdetect/ja, langdetect/kn, langdetect/ko, langdetect/lt, langdetect/lv, langdetect/mk, langdetect/ml, langdetect/mr, langdetect/ne, langdetect/nl, langdetect/no, langdetect/pa, langdetect/pl, langdetect/pt, langdetect/ro, langdetect/ru, langdetect/sk, langdetect/sl, langdetect/so, langdetect/sq, langdetect/sv, langdetect/sw, langdetect/ta, langdetect/te, langdetect/th, langdetect/tl, langdetect/tr, langdetect/uk, langdetect/ur, langdetect/vi, langdetect/zh-cn, langdetect/zh-tw, lib/jsonic-1.2.0.jar, lib/langdetect.jar, lib/langdetect.jar.License, source/net/yacy/document/language/Identificator.java
Wed Jun 24 01:55:51 CEST 2015
by Michael Peter Christen
migration to Solr 5.2: huge benefits - this is a lot faster!

This is a very complex migration: many classes had been renamed or
removed, dependencies changed and the solr index type is now aligned to
be a solr cloud repository.
Together with the Solr 5.2 library update, one other dependent library
had been updated as well: httpclient 4.4->4.4.1

Older indexes are migrated from 4_10 to 5_2. However, the new index
structure is more efficient and we recommend to re-index everything.
Please use the index export before you do the update to a large
surrogate xml file. After the update, start with an empty index and then
initialize this with your dump.
Changed Files: .classpath, build.xml, defaults/solr/schema.xml, defaults/solr/solr.xml, defaults/solr/solrconfig.xml, htroot/IndexControlURLs_p.java, lib/httpclient-4.4.1.License, lib/httpclient-4.4.1.jar, lib/httpcore-4.4.1.License, lib/httpmime-4.4.1.License, lib/httpmime-4.4.1.jar, lib/lucene-analyzers-common-5.2.0.jar, lib/lucene-analyzers-phonetic-5.2.0.jar, lib/lucene-backward-codecs-5.2.0.jar, lib/lucene-classification-5.2.0.jar, lib/lucene-codecs-5.2.0.jar, lib/lucene-core-5.2.0.jar, lib/lucene-facet-5.2.0.jar, lib/lucene-grouping-5.2.0.jar, lib/lucene-highlighter-5.2.0.jar, lib/lucene-join-5.2.0.jar, lib/lucene-memory-5.2.0.jar, lib/lucene-misc-5.2.0.jar, lib/lucene-queries-5.2.0.jar, lib/lucene-queryparser-5.2.0.jar, lib/lucene-spatial-5.2.0.jar, lib/lucene-suggest-5.2.0.jar, lib/noggit-0.6.jar, lib/solr-core-5.2.0.jar, lib/solr-solrj-5.2.0.jar, lib/stax2-api-3.1.4.jar, lib/woodstox-core-asl-4.4.1.jar, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/RemoteSolrConnector.java, source/net/yacy/cora/federate/solr/connector/ShardSelection.java, source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java, source/net/yacy/cora/federate/solr/instance/EmbeddedInstance.java, source/net/yacy/cora/federate/solr/instance/RemoteInstance.java, source/net/yacy/cora/federate/solr/instance/ServerMirror.java, source/net/yacy/cora/federate/solr/instance/ServerShard.java, source/net/yacy/cora/federate/solr/instance/ShardInstance.java, source/net/yacy/cora/federate/solr/instance/SolrInstance.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Fulltext.java
Sun Jun 07 20:09:27 CEST 2015
by sixcooler
Update to Jetty-9.2.11 - a bugfix-release that did not solve my
Problems, but does not harm anything
Changed Files: .classpath, build.xml, lib/jetty-9.2.11.v20150529.License, lib/jetty-client-9.2.11.v20150529.jar, lib/jetty-continuation-9.2.11.v20150529.jar, lib/jetty-deploy-9.2.11.v20150529.jar, lib/jetty-http-9.2.11.v20150529.jar, lib/jetty-io-9.2.11.v20150529.jar, lib/jetty-jmx-9.2.11.v20150529.jar, lib/jetty-proxy-9.2.11.v20150529.jar, lib/jetty-security-9.2.11.v20150529.jar, lib/jetty-server-9.2.11.v20150529.jar, lib/jetty-servlet-9.2.11.v20150529.jar, lib/jetty-servlets-9.2.11.v20150529.jar, lib/jetty-util-9.2.11.v20150529.jar, lib/jetty-webapp-9.2.11.v20150529.jar, lib/jetty-xml-9.2.11.v20150529.jar, nbproject/project.xml, pom.xml
Sat May 30 06:31:08 CEST 2015
by Michael Peter Christen
added intensity option to graphics
Changed Files: htroot/AccessPicture_p.java, htroot/WebStructurePicture_p.java, htroot/imagetest.java, htroot/osm.java, source/net/yacy/peers/graphics/Banner.java, source/net/yacy/peers/graphics/NetworkGraph.java, source/net/yacy/visualization/Captcha.java, source/net/yacy/visualization/ChartPlotter.java, source/net/yacy/visualization/DemoApplet.java, source/net/yacy/visualization/GraphPlotter.java, source/net/yacy/visualization/HexGridPlotter.java, source/net/yacy/visualization/PrintTool.java, source/net/yacy/visualization/RasterPlotter.java
Fri May 29 15:05:52 CEST 2015
by Michael Peter Christen
added a full solr export to the IndexControlURLs_p.html servlet. The
export function is also now the default export option. The export file
format for a full solr export is very similar to a solr search result
xml, only the <lst name="responseHeader"> tag is missing.

The exported xml has a special line termination feature: all documents
will be exported into a single line without any CR in between. That
means that every document is completely inside a single line. While this
is not readable at all for humans, it is very useful for linux line
processing scripts, like grep. Using grep it will be easy to select
single documents which match for a given pattern.

Such dumps shall be importable with the DATA/SURROGATE/in import
function, but that import is not yet adopted to the new file format.
Changed Files: htroot/IndexControlURLs_p.html, htroot/IndexControlURLs_p.java, source/net/yacy/cora/util/CRIgnoreWriter.java, source/net/yacy/search/index/Fulltext.java
Sat May 23 02:06:39 CEST 2015
by reger
Init remote crawler on demand
If remote crawl option is not activated, skip init of remoteCrawlJob to save the resources of queue and ideling thread.
Deploy of the remoteCrawlJob deferred on activation of the option.
Changed Files: htroot/ConfigNetwork_p.java, htroot/RemoteCrawl_p.java, htroot/Status.java, htroot/api/status_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/urls.java, source/net/yacy/crawler/Balancer.java, source/net/yacy/crawler/HostBalancer.java, source/net/yacy/crawler/HostQueue.java, source/net/yacy/crawler/LegacyBalancer.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/crawler/data/NoticedURL.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java
Mon May 11 00:37:04 CEST 2015
by reger
refactor getBookmark 
to consistenly check existance by != null (w/o throwing exception on not found)

Changed Files: htroot/Bookmarks.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/get_folders.java, htroot/api/bookmarks/posts/all.java, htroot/api/bookmarks/posts/get.java, htroot/api/bookmarks/tags/addTag_p.java, htroot/api/bookmarks/xbel/xbel.java, htroot/api/ymarks/import_ymark.java, htroot/yacysearchitem.java, source/net/yacy/data/BookmarksDB.java
Thu Apr 23 06:36:57 CEST 2015
by Michael Peter Christen
update to bootstrap.css 3.3.4
Changed Files: htroot/env/bootstrap/css/bootstrap-theme.css, htroot/env/bootstrap/css/bootstrap-theme.css.map, htroot/env/bootstrap/css/bootstrap-theme.min.css, htroot/env/bootstrap/css/bootstrap.css, htroot/env/bootstrap/css/bootstrap.css.map, htroot/env/bootstrap/css/bootstrap.min.css, htroot/env/bootstrap/fonts/glyphicons-halflings-regular.eot, htroot/env/bootstrap/fonts/glyphicons-halflings-regular.svg, htroot/env/bootstrap/fonts/glyphicons-halflings-regular.ttf, htroot/env/bootstrap/fonts/glyphicons-halflings-regular.woff, htroot/env/bootstrap/js/bootstrap.js, htroot/env/bootstrap/js/bootstrap.min.js, htroot/env/templates/header.template
Wed Apr 15 13:17:23 CEST 2015
by Michael Peter Christen
enhanced timezone managament for indexed data:
to support the new time parser and search functions in YaCy a high
precision detection of date and time on the day is necessary. That
requires that the time zone of the document content and the time zone of
the user, doing a search, is detected. The time zone of the search
request is done automatically using the browsers time zone offset which
is delivered to the search request automatically and invisible to the
user. The time zone for the content of web pages cannot be detected
automatically and must be an attribute of crawl starts. The advanced
crawl start now provides an input field to set the time zone in minutes
as an offset number. All parsers must get a time zone offset passed, so
this required the change of the parser java api. A lot of other changes
had been made which corrects the wrong handling of dates in YaCy which
was to add a correction based on the time zone of the server. Now no
correction is added and all dates in YaCy are UTC/GMT time zone, a
normalized time zone for all peers.
Changed Files: htroot/CrawlStartExpert.html, htroot/CrawlStartSite.html, htroot/Crawler_p.java, htroot/HostBrowser.java, htroot/IndexControlRWIs_p.java, htroot/NetworkHistory.java, htroot/QuickCrawlLink_p.java, htroot/api/bookmarks/posts/get.java, htroot/api/push_p.java, htroot/api/timeline_p.java, htroot/index.html, htroot/rct_p.java, htroot/yacy/search.java, htroot/yacy/transferURL.java, htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearchtrailer.java, source/net/yacy/cora/date/AbstractFormatter.java, source/net/yacy/cora/date/DateFormatter.java, source/net/yacy/cora/date/GenericFormatter.java, source/net/yacy/cora/date/ISO8601Formatter.java, source/net/yacy/cora/document/feed/RSSMessage.java, source/net/yacy/cora/federate/FederateSearchManager.java, source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/crawler/data/CrawlProfile.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/crawler/data/Snapshots.java, source/net/yacy/crawler/retrieval/Request.java, source/net/yacy/crawler/retrieval/Response.java, source/net/yacy/crawler/retrieval/SitemapImporter.java, source/net/yacy/data/BlogBoard.java, source/net/yacy/data/BookmarkHelper.java, source/net/yacy/data/ymark/YMarkAutoTagger.java, source/net/yacy/data/ymark/YMarkCrawlStart.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/DateDetection.java, source/net/yacy/document/Parser.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/importer/ResumptionToken.java, source/net/yacy/document/parser/apkParser.java, source/net/yacy/document/parser/audioTagParser.java, source/net/yacy/document/parser/augment/AugmentParser.java, source/net/yacy/document/parser/bzipParser.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/dwgParser.java, source/net/yacy/document/parser/genericParser.java, source/net/yacy/document/parser/gzipParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/ScraperInputStream.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/images/metadataImageParser.java, source/net/yacy/document/parser/linkScraperParser.java, source/net/yacy/document/parser/mmParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rdfParser.java, source/net/yacy/document/parser/rdfa/impl/RDFaParser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/sidAudioParser.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/http/ProxyCacheHandler.java, source/net/yacy/http/ProxyHandler.java, source/net/yacy/http/servlets/SolrSelectServlet.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/BEncodedHeapBag.java, source/net/yacy/kelondro/blob/Tables.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/peers/NewsDB.java, source/net/yacy/peers/Seed.java, source/net/yacy/peers/graphics/WebStructureGraph.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/EventTracker.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/DocumentIndex.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/query/AccessTracker.java, source/net/yacy/search/query/QueryModifier.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/server/http/HTTPDProxyHandler.java
Sun Apr 05 23:38:14 CEST 2015
by reger
upd to PDFBox 1.8.9
Changed Files: .classpath, build.xml, lib/fontbox-1.8.9.License, lib/fontbox-1.8.9.jar, lib/jempbox-1.8.9.License, lib/jempbox-1.8.9.jar, lib/pdfbox-1.8.9.License, lib/pdfbox-1.8.9.jar, nbproject/project.xml, pom.xml
Sun Mar 22 02:47:12 CET 2015
by reger
upd to Jetty 9.2.10
Changed Files: .classpath, build.xml, lib/jetty-9.2.10.v20150310.License, lib/jetty-client-9.2.10.v20150310.jar, lib/jetty-continuation-9.2.10.v20150310.jar, lib/jetty-deploy-9.2.10.v20150310.jar, lib/jetty-http-9.2.10.v20150310.jar, lib/jetty-io-9.2.10.v20150310.jar, lib/jetty-jmx-9.2.10.v20150310.jar, lib/jetty-proxy-9.2.10.v20150310.jar, lib/jetty-security-9.2.10.v20150310.jar, lib/jetty-server-9.2.10.v20150310.jar, lib/jetty-servlet-9.2.10.v20150310.jar, lib/jetty-servlets-9.2.10.v20150310.jar, lib/jetty-util-9.2.10.v20150310.jar, lib/jetty-webapp-9.2.10.v20150310.jar, lib/jetty-xml-9.2.10.v20150310.jar, nbproject/project.xml, pom.xml
Mon Mar 02 18:00:20 CET 2015
by Michael Peter Christen
Show dates in the content of a document in the search result:
- if an eventDate is given in the search result, replace the document
date with the event date and prefix it with the string "on ".
- the document date is omitted if a date from the cent is shown

Added also the date as fields in the json and rss result sets.
Changed Files: htroot/yacysearch.rss, htroot/yacysearchitem.html, htroot/yacysearchitem.java, htroot/yacysearchitem.json, htroot/yacysearchitem.xml, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/search/snippet/ResultEntry.java
Mon Mar 02 04:30:10 CET 2015
by Michael Peter Christen
added a new way of content browsing in search results:
- date navigation

The date is taken from the CONTENT of the documents / web pages, NOT
from a date submitted in the context of metadata (i.e. http header or
html head form). This makes it possible to search for documents in the
future, i.e. when documents contain event descriptions for future
events.

The date is written to an index field which is now enabled by default.
All documents are scanned for contained date mentions.
To visualize the dates for a specific search results, a histogram
showing the number of documents for each day is displayed. To render
these histograms the morris.js library is used. Morris.js requires also
raphael.js which is now also integrated in YaCy.

The histogram is now also displayed in the index browser by default.

To select a specific range from a search result, the following modifiers
had been introduced:
from:<date>
to:<date>
These modifiers can be used separately (i.e. only 'from' or only 'to')
to describe an open interval or combined to have a closed interval. Both
dates are inclusive. To select a specific single date only, use the
'to:' - modifier.

The histogram shows blue and green lines; the green lines denot weekend
days (saturday and sunday).

Clicking on bars in the histogram has the following reaction:
1st click: add a from:<date> modifier for the date of the bar
2nd click: add a to:<date> modifier for the date of the bar
3rd click: remove from and date modifier and set a on:<date> for the bar
When the on:<date> modifier is used, the histogram shows an unlimited
time period. This makes it possible to click again (4th click) which is
then interpreted as a 1st click again (sets a from modifier).

The display feature is NOT switched on by default; to switch it on use
the /ConfigSearchPage_p.html servlet.
Changed Files: defaults/solr.collection.schema, defaults/yacy.init, htroot/ConfigSearchPage_p.html, htroot/ConfigSearchPage_p.java, htroot/CrawlStartExpert.java, htroot/HostBrowser.html, htroot/HostBrowser.java, htroot/WebStructurePicture_p.java, htroot/env/morris.css, htroot/index.html, htroot/index.java, htroot/js/morris.js, htroot/js/raphael-min.js, htroot/yacysearch.html, htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java, htroot/yacysearchtrailer.json, source/net/yacy/cora/federate/opensearch/OpenSearchConnector.java, source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/cora/federate/solr/SchemaDeclaration.java, source/net/yacy/cora/federate/solr/responsewriter/EnhancedXMLResponseWriter.java, source/net/yacy/cora/federate/solr/responsewriter/HTMLResponseWriter.java, source/net/yacy/cora/sorting/ClusteredScoreMap.java, source/net/yacy/crawler/HostQueue.java, source/net/yacy/crawler/data/ResultURLs.java, source/net/yacy/data/DidYouMean.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/DateDetection.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/Evaluation.java, source/net/yacy/peers/Seed.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/query/QueryModifier.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/CollectionSchema.java, source/net/yacy/search/schema/WebgraphSchema.java
Sat Feb 28 15:46:46 CET 2015
by Michael Peter Christen
Fix for Jetty "JetLeak" bug: update to jetty 9.2.9
The bug was inside the jetty library, for details see:
http://blog.gdssecurity.com/labs/2015/2/25/jetleak-vulnerability-remote-leakage-of-shared-buffers-in-je.html
We recommend to update your YaCy peer with this bugfix.
Changed Files: .classpath, build.xml, lib/jetty-9.2.9.v20150224.License, lib/jetty-client-9.2.9.v20150224.jar, lib/jetty-continuation-9.2.9.v20150224.jar, lib/jetty-deploy-9.2.9.v20150224.jar, lib/jetty-http-9.2.9.v20150224.jar, lib/jetty-io-9.2.9.v20150224.jar, lib/jetty-jmx-9.2.9.v20150224.jar, lib/jetty-proxy-9.2.9.v20150224.jar, lib/jetty-security-9.2.9.v20150224.jar, lib/jetty-server-9.2.9.v20150224.jar, lib/jetty-servlet-9.2.9.v20150224.jar, lib/jetty-servlets-9.2.9.v20150224.jar, lib/jetty-util-9.2.9.v20150224.jar, lib/jetty-webapp-9.2.9.v20150224.jar, lib/jetty-xml-9.2.9.v20150224.jar
Mon Feb 23 22:54:49 CET 2015
by Marc Nause
Changes to improve compatibility with OpenBSD. (see
http://forum.yacy-websuche.de/viewtopic.php?f=8&t=5503)
Changed Files: bin/addrss.sh, bin/apicall.sh, bin/apicat.sh, bin/checkalive.sh, bin/checkindex.sh, bin/clearall.sh, bin/clearapi.sh, bin/clearcache.sh, bin/clearindex.sh, bin/deleteurl.sh, bin/deploy.sh, bin/down.sh, bin/dumpcheck.sh, bin/importmediawiki.sh, bin/importurllist.sh, bin/indexdump.sh, bin/indexrestore.sh, bin/myip.sh, bin/passwd.sh, bin/search.sh, bin/search1.sh, bin/searchall.sh, bin/up.sh, startYACY.sh
Sat Feb 07 00:44:09 CET 2015
by reger
upd to Jetty 9.2.7
Changed Files: build.xml, lib/jetty-9.2.7.v20150116.License, lib/jetty-client-9.2.7.v20150116.jar, lib/jetty-continuation-9.2.7.v20150116.jar, lib/jetty-deploy-9.2.7.v20150116.jar, lib/jetty-http-9.2.7.v20150116.jar, lib/jetty-io-9.2.7.v20150116.jar, lib/jetty-jmx-9.2.7.v20150116.jar, lib/jetty-proxy-9.2.7.v20150116.jar, lib/jetty-security-9.2.7.v20150116.jar, lib/jetty-server-9.2.7.v20150116.jar, lib/jetty-servlet-9.2.7.v20150116.jar, lib/jetty-servlets-9.2.7.v20150116.jar, lib/jetty-util-9.2.7.v20150116.jar, lib/jetty-webapp-9.2.7.v20150116.jar, lib/jetty-xml-9.2.7.v20150116.jar, nbproject/project.xml, pom.xml
Fri Jan 30 13:20:56 CET 2015
by Michael Peter Christen
added a html field scraper which reads text from html entities of a
given css class and extends a given vocabulary with a term consisting
with the text content of the html class tag. Additionally, the term is
included into the semantic facet of the document. This allows the
creation of faceted search to documents without the pre-creation of
vocabularies; instead, the vocabulary is created on-the-fly, possibly
for use in other crawls. If any of the term scraping for a specific
vocabulary is successful on a document, this vocabulary is excluded for
auto-annotation on the page.

To use this feature, do the following:
- create a vocabulary on /Vocabulary_p.html (if not existent)
- in /CrawlStartExpert.html you will now see the vocabularies as column
in a table. The second column provides text fields where you can name
the class of html entities where the literal of the corresponding
vocabulary shall be scraped out
- when doing a search, you will see the content of the scraped fields in
a navigation facet for the given vocabulary
Changed Files: htroot/ConfigHeuristics_p.java, htroot/CrawlStartExpert.html, htroot/CrawlStartExpert.java, htroot/Crawler_p.java, htroot/QuickCrawlLink_p.java, htroot/osm.java, source/net/yacy/cora/language/synonyms/AutotaggingLibrary.java, source/net/yacy/cora/lod/vocabulary/Tagging.java, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/crawler/data/CrawlProfile.java, source/net/yacy/crawler/retrieval/Response.java, source/net/yacy/data/BookmarkHelper.java, source/net/yacy/data/ymark/YMarkAutoTagger.java, source/net/yacy/data/ymark/YMarkCrawlStart.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Parser.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/VocabularyScraper.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/parser/apkParser.java, source/net/yacy/document/parser/audioTagParser.java, source/net/yacy/document/parser/augment/AugmentParser.java, source/net/yacy/document/parser/bzipParser.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/dwgParser.java, source/net/yacy/document/parser/genericParser.java, source/net/yacy/document/parser/gzipParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/ScraperInputStream.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/images/metadataImageParser.java, source/net/yacy/document/parser/linkScraperParser.java, source/net/yacy/document/parser/mmParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rdfParser.java, source/net/yacy/document/parser/rdfa/impl/RDFaParser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/sidAudioParser.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/DocumentIndex.java, source/net/yacy/search/index/Segment.java
Thu Jan 29 02:28:03 CET 2015
by Michael Peter Christen
replace with CommonPattern.SPACE for split
Changed Files: htroot/BlogComments.java, htroot/ViewFile.java, htroot/Vocabulary_p.java, htroot/processing/domaingraph/applet/domaingraph.java, htroot/yacy/message.java, htroot/yacysearch_location.java, source/net/yacy/data/DidYouMean.java, source/net/yacy/data/ymark/YMarkAutoTagger.java, source/net/yacy/search/schema/CollectionConfiguration.java, test/net/yacy/search/snippet/TextSnippetTest.java
Thu Jan 29 01:46:22 CET 2015
by Michael Peter Christen
applying precompiled CommonPattern.COMMA.split to all places where
split(",") was used
Changed Files: htroot/ConfigNetwork_p.java, htroot/IndexControlRWIs_p.java, htroot/SettingsAck_p.java, htroot/ViewFile.java, htroot/WebStructurePicture_p.java, htroot/api/feed.java, htroot/api/table_p.java, htroot/api/timeline_p.java, htroot/yacy/list.java, htroot/yacysearch.java, source/net/yacy/contentcontrol/SMWListSyncThread.java, source/net/yacy/cora/federate/solr/instance/RemoteInstance.java, source/net/yacy/cora/geo/GeonamesLocation.java, source/net/yacy/cora/geo/OpenGeoDBLocation.java, source/net/yacy/cora/lod/vocabulary/Tagging.java, source/net/yacy/cora/storage/Files.java, source/net/yacy/crawler/data/CrawlProfile.java, source/net/yacy/data/BookmarksDB.java, source/net/yacy/data/ListManager.java, source/net/yacy/data/Translator.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/migration.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/SeedDB.java, source/net/yacy/repository/BlacklistFile.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/ranking/RankingProfile.java, source/net/yacy/server/http/HTTPDFileHandler.java
Fri Jan 23 11:30:13 CET 2015
by Michael Peter Christen
replaced old JavaApplicationStub for Mac Application framework with new
script. Adopted the YaCyApp environment and fixed a problem in the
startYACY.sh application wrapper which caused wrong usage of logging
option -l which caused that files had been written to the YaCy
application folder.
As a result of this fix, it is not necessary any more to change path
settings in Info.plist if libraries are changed.
Changed Files: addon/YaCy.app/Contents/Info.plist, addon/YaCy.app/Contents/MacOS/JavaApplicationStub, build.xml, source/net/yacy/gui/Toolkits.java, source/net/yacy/gui/Tray.java, source/net/yacy/gui/YaCyApp.java, source/net/yacy/gui/framework/Application.java, startYACY.sh


Bugfixes   
Jump to: YaCy Release 1.90 top / Other Changes

CommitDescription
Wed Jun 15 09:07:54 CEST 2016
by luccioman
Fixed rendering for many mobile devices.
Changed Files: htroot/env/templates/metas.template
Wed Jun 08 23:16:10 CEST 2016
by luccioman
JavaScript License fixes for LibreJS compatibility
Changed Files: htroot/ViewFile.html, htroot/env/templates/metas.template, htroot/js/IndexCreate.js, htroot/js/highslide/highslide.js, htroot/js/raphael-min.js, htroot/js/sorttable.js, source/net/yacy/cora/federate/solr/responsewriter/HTMLResponseWriter.java
Tue Jun 07 23:34:42 CEST 2016
by reger
fix MessageBoard test for null data
Changed Files: source/net/yacy/data/MessageBoard.java
Fri Jun 03 14:50:03 CEST 2016
by luccioman
Fixed license link for display on DockerHub
Changed Files: docker/Readme.md
Fri Jun 03 14:39:39 CEST 2016
by luccioman
Fixed line end causing build error.
Changed Files: docker/Dockerfile.alpine
Tue May 31 09:38:00 CEST 2016
by luccioman
Fixed link to docker readme (mediawiki format)
Changed Files: readme.mediawiki
Tue May 17 21:59:29 CEST 2016
by luccioman
Fixed as background process title
Changed Files: docker/Readme.md
Tue May 17 02:08:07 CEST 2016
by reger
fix MessageSend servlet response if called for passive peer
Changed Files: htroot/MessageSend_p.html, htroot/MessageSend_p.java
Tue May 17 02:07:49 CEST 2016
by reger
fix SeedDB.get(byte[]) hash string compare (for returning own seed shortcut)
Changed Files: source/net/yacy/peers/SeedDB.java
Sun May 15 01:26:38 CEST 2016
by reger
fix NPE on missing crawldepth_i
Changed Files: source/net/yacy/crawler/data/Transactions.java
Sat May 14 03:51:13 CEST 2016
by reger
fix NPE in citation servlet on empty text field
Changed Files: htroot/api/citation.java
Fri May 13 13:35:38 CEST 2016
by luc
Fixed initial admin password initialization
Changed Files: docker/Dockerfile
Sun Apr 17 04:42:05 CEST 2016
by reger
fix migration, account for ssl port in config (for auto-disable https)
Changed Files: source/net/yacy/migration.java
Sun Apr 17 03:20:14 CEST 2016
by reger
fix logger name
Changed Files: htroot/ConfigLanguage_p.java, source/net/yacy/crawler/data/ResultURLs.java, source/net/yacy/utils/translation/TranslatorXliff.java
Tue Apr 12 03:55:43 CEST 2016
by reger
fix rwi doubledom return best (highest) ranking
Changed Files: source/net/yacy/search/query/SearchEvent.java
Tue Apr 05 03:16:55 CEST 2016
by reger
fix some line breaks and mark additional lines w/o translation
Changed Files: locales/fr.lng, locales/ru.lng, locales/sk.lng, locales/uk.lng
Sat Mar 26 02:20:53 CET 2016
by reger
fix line break errors in translation files
Changed Files: locales/cn.lng, locales/de.lng, locales/hi.lng, locales/ru.lng
Sat Mar 26 00:51:21 CET 2016
by reger
fix some format errors in language files
Changed Files: locales/cn.lng, locales/de.lng, locales/ru.lng, locales/uk.lng
Tue Mar 22 02:24:52 CET 2016
by reger
fix maven packaging (include langdetect directory)
Changed Files: assembly.xml
Sun Mar 06 03:24:09 CET 2016
by reger
fix use of not initialized variable (m_LiteralDecoder)
Changed Files: libbuild/J7Zip-modified/src/SevenZip/Compression/LZMA/Decoder.java
Sat Feb 20 22:30:15 CET 2016
by reger
fix SolrSelectServlet authentication to  default to true
Changed Files: source/net/yacy/http/servlets/SolrSelectServlet.java
Fri Feb 05 08:29:41 CET 2016
by luc
Fixed json search results from commit "Improved URLLicence reliability"
Changed Files: htroot/compare_yacy.java, htroot/portalsearch/yacy-portalsearch.js, htroot/yacy/ui/js/yacyui-portalsearch.js, htroot/yacysearchitem.json, installYaCyWindowsService.bat, source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/search/index/DocumentIndex.java, startYACY.bat, stopYACY.bat
Sun Jan 31 02:38:39 CET 2016
by reger
fix json date facet displayname
Changed Files: htroot/yacysearchtrailer.json
Tue Jan 19 20:57:22 CET 2016
by sixcooler
fix for connections not getting closed to get favicon.ico during seach
Changed Files: source/net/yacy/crawler/retrieval/HTTPLoader.java
Tue Jan 19 20:50:26 CET 2016
by sixcooler
fix for 'move test classes to test/java'
Changed Files: .classpath
Thu Dec 24 04:37:24 CET 2015
by reger
fix typo in Steering.html
Changed Files: htroot/Steering.html, locales/de.lng, locales/fr.lng, locales/ru.lng
Thu Dec 24 02:00:45 CET 2015
by reger
fix NPE on empty blog importfile parameter
Changed Files: htroot/Blog.html, htroot/BlogComments.html, source/net/yacy/data/BlogBoard.java
Mon Dec 21 19:47:40 CET 2015
by reger
just remove debug leftover
Changed Files: htroot/yacysearchitem.html
Sun Dec 20 15:51:54 CET 2015
by reger
fix reading of ippattern config array in URLProxy
Changed Files: source/net/yacy/http/servlets/UrlProxyServlet.java
Fri Dec 18 02:35:44 CET 2015
by reger
fix (todo) remove redundant holding of email link nameproperty in parser document
Changed Files: htroot/ViewFile.java, source/net/yacy/document/Document.java
Thu Dec 17 02:53:10 CET 2015
by reger
fix dc:date in mediawikiimporter/document.writexml to use lastmodified
Changed Files: source/net/yacy/document/Document.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/importer/MediawikiImporter.java
Wed Dec 16 02:09:33 CET 2015
by luc
Added useful debug JVM parameters as comment.
Changed Files: build.xml
Wed Dec 09 01:21:00 CET 2015
by Michael Peter Christen
fix in error handling
Changed Files: htroot/ViewImage.java
Thu Dec 03 00:33:13 CET 2015
by Michael Peter Christen
fix for npe
Changed Files: source/net/yacy/search/index/Segment.java
Tue Dec 01 00:06:50 CET 2015
by reger
prevent exception on repeated ViewImage with same urlLicense
Changed Files: htroot/ViewImage.java
Mon Nov 30 20:13:34 CET 2015
by Sergey Stepanov
fixed typos
Changed Files: addon/synonyms/thesaurus_ru_yacy
Mon Nov 30 13:19:49 CET 2015
by Michael Peter Christen
fixed classpath
Changed Files: .classpath
Fri Nov 20 10:15:54 CET 2015
by luc
Corrected IcedTea version. See http://mantis.tokeek.de/view.php?id=615
Changed Files: readme.mediawiki
Thu Nov 19 21:08:00 CET 2015
by reger
fix link to logo (yacysearch.xsl)
Changed Files: htroot/yacysearch.xsl
Thu Nov 05 09:40:24 CET 2015
by luc
Ensure closing of InputStream even when an exception occurs.
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Sat Oct 31 22:53:59 CET 2015
by reger
fix detection of https port changed after set in System Admin
Changed Files: htroot/SettingsAck_p.java
Sun Oct 25 19:38:42 CET 2015
by reger
fix typo in WikiCode coordinate calculation
Changed Files: source/net/yacy/data/wiki/WikiCode.java
Wed Oct 14 15:16:16 CEST 2015
by Michael Peter Christen
fix for image size field values (must be multi-valued)
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Sun Oct 11 06:06:40 CEST 2015
by reger
fix link target on iframe list in CrawlProfileEditor
Changed Files: htroot/Table_API_p.html
Thu Sep 24 13:53:54 CEST 2015
by Michael Peter Christen
fix for latest merge
Changed Files: htroot/env/style.java
Sat Sep 12 20:07:43 CEST 2015
by reger
fix test methode (add throw for URIMetadataNode)
Changed Files: test/net/yacy/search/snippet/TextSnippetTest.java
Wed Sep 02 02:36:31 CEST 2015
by reger
fix init of error cache, use latest faildates => load_date_dt
Changed Files: source/net/yacy/search/index/ErrorCache.java, source/net/yacy/search/schema/CollectionSchema.java
Sun Aug 23 23:01:20 CEST 2015
by reger
avoid runtime exception by earlier testing for seed.ip=null
Changed Files: source/net/yacy/peers/Protocol.java
Tue Aug 11 00:42:26 CEST 2015
by Michael Peter Christen
fix for filesystem crawl
Changed Files: source/net/yacy/crawler/data/CrawlProfile.java
Mon Aug 03 05:15:34 CEST 2015
by Michael Peter Christen
revert of fq transformation (recent fix)
Changed Files: source/net/yacy/search/query/QueryParams.java
Fri Jul 10 17:34:29 CEST 2015
by Michael Peter Christen
more fixes for special windows paths
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Fri Jul 10 17:14:14 CEST 2015
by Michael Peter Christen
patch for bad windows file paths
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Mon Jun 01 01:56:09 CEST 2015
by Michael Peter Christen
fix for index import
Changed Files: source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/search/Switchboard.java
Thu May 28 17:43:52 CEST 2015
by Michael Peter Christen
fix for unresolved pattern
Changed Files: htroot/CrawlProfileEditor_p.java
Fri May 22 11:15:53 CEST 2015
by Michael Peter Christen
fix for division by zero
Changed Files: htroot/yacysearchtrailer.java
Mon May 11 14:46:09 CEST 2015
by Michael Peter Christen
disabled debug thread dumps
Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java
Sun May 10 05:18:23 CEST 2015
by reger
fix NPE in addToIndex when used outside searchEvent
Changed Files: source/net/yacy/search/Switchboard.java
Fri May 08 15:31:01 CEST 2015
by Michael Peter Christen
added temporary debug output in http client
Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java
Thu May 07 03:25:19 CEST 2015
by reger
precaution against NPE on createorgetBookmark on search result
Changed Files: htroot/yacysearch.java
Sun Apr 12 01:11:47 CEST 2015
by reger
fix typecast for css links
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Sat Apr 11 12:30:29 CEST 2015
by Michael Peter Christen
fix for filter queries
Changed Files: source/net/yacy/search/query/QueryParams.java
Sat Apr 11 12:20:29 CEST 2015
by Michael Peter Christen
fix for not valid json in case that topics are switched off
Changed Files: htroot/yacysearchtrailer.json
Thu Apr 09 14:21:23 CEST 2015
by Michael Peter Christen
logging fix
Changed Files: source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java
Tue Apr 07 17:02:02 CEST 2015
by Michael Peter Christen
better debugging of fq
Changed Files: source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java
Tue Mar 31 02:20:13 CEST 2015
by reger
fix NPE in location search on missing/empty PubDate in underlaying rss data
Changed Files: htroot/yacysearch_location.java, source/net/yacy/cora/document/feed/RSSMessage.java
Sat Mar 28 21:12:00 CET 2015
by reger
shorten exception loggin on not available connection in  Load_RSS_p servlet
Changed Files: htroot/Load_RSS_p.java
Thu Mar 26 00:21:31 CET 2015
by reger
fix NPE on viewfile of url not in index
Changed Files: htroot/ViewFile.java
Wed Mar 25 13:21:36 CET 2015
by Michael Peter Christen
fix: banner did not show link and qph for portal mode
Changed Files: htroot/Banner.java
Wed Mar 18 22:04:03 CET 2015
by reger
fix get fresh_date_dt  to allow returned value to be date in future
Changed Files: source/net/yacy/kelondro/data/meta/URIMetadataNode.java
Fri Mar 13 02:02:53 CET 2015
by reger
fix MultiProtocolURL mailto protocol detection
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Wed Mar 11 21:28:57 CET 2015
by reger
fix link to DeReWo project page
Changed Files: htroot/DictionaryLoader_p.html
Wed Mar 11 20:02:23 CET 2015
by reger
fix link to DeReWo download file
Changed Files: source/net/yacy/document/LibraryProvider.java
Mon Mar 02 04:43:42 CET 2015
by Michael Peter Christen
bugfix for fixed host/port
Changed Files: htroot/ConfigSearchPage_p.html, htroot/HostBrowser.html
Sun Feb 15 05:30:14 CET 2015
by reger
fix NPE in snippet computation
Changed Files: source/net/yacy/search/snippet/TextSnippet.java
Sat Feb 14 01:38:20 CET 2015
by Michael Peter Christen
fixed httpclient lib paths in ant build
Changed Files: build.xml
Tue Feb 10 08:33:30 CET 2015
by Michael Peter Christen
npe fix for latest scraper feature
Changed Files: source/net/yacy/document/Condenser.java
Sun Feb 08 00:15:30 CET 2015
by reger
fix: searchoption hint for heuristic
Changed Files: htroot/index.html, htroot/index.java
Wed Feb 04 15:03:34 CET 2015
by Michael Peter Christen
IPv6 Fix for push interface
Changed Files: htroot/api/push_p.java
Wed Feb 04 11:55:27 CET 2015
by Michael Peter Christen
fix for failed selection of terms in faceted search with vocabularies
Changed Files: htroot/yacysearchtrailer.java
Wed Jan 28 17:45:25 CET 2015
by Michael Peter Christen
fix for wkhtmltopdf (custom header does not work)
Changed Files: source/net/yacy/cora/util/Html2Image.java
Wed Jan 28 13:40:41 CET 2015
by Michael Peter Christen
fix for urlmaskfilter
Changed Files: htroot/yacysearch.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/peers/Protocol.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java
Tue Jan 27 16:24:27 CET 2015
by Michael Peter Christen
fix for vocabulary on/off setting
Changed Files: htroot/Vocabulary_p.java, htroot/api/snapshot.java, source/net/yacy/cora/lod/vocabulary/Tagging.java
Fri Jan 23 18:34:38 CET 2015
by Michael Peter Christen
fix for shell script
Changed Files: startYACY.sh


Other Changes   
Jump to: YaCy Release 1.90 top / Bugfixes

CommitDescription
Mon Jul 04 12:01:41 CEST 2016
by Michael Peter Christen
release 1.90
Changed Files: build.properties
Mon Jul 04 11:02:36 CEST 2016
by Michael Peter Christen
another peer list update
Changed Files: defaults/yacy.network.freeworld.unit
Mon Jul 04 11:02:15 CEST 2016
by Michael Peter Christen
small refactoring + do not accept too old peers during bootstrap
Changed Files: source/net/yacy/peers/SeedDB.java, source/net/yacy/search/Switchboard.java
Mon Jul 04 02:44:58 CEST 2016
by reger
test case for HostBalancer issue in intranet mode
with file:// protocol, 2 hostqueues accessing same cache file concurrently
http://mantis.tokeek.de/view.php?id=668
Reason seems to be diff. hosthash key of hostqueues on reopen. 
Internal queue key and external representation (directoryname currently hostname.port) must be adjusted to fix it (not done yet).
Changed Files: test/java/net/yacy/crawler/HostBalancerTest.java
Sun Jul 03 22:50:50 CEST 2016
by Michael Peter Christen
added another principal peer
Changed Files: defaults/yacy.network.freeworld.unit
Sun Jul 03 19:20:31 CEST 2016
by luccioman
Revert on commit related to another personnal branch.
Changed Files:
Sun Jul 03 17:28:47 CEST 2016
by luccioman
Added instructions for log control and upgrade
Changed Files: docker/Readme.md
Sat Jul 02 20:33:23 CEST 2016
by reger
remove wrong log line in CrawlSwitchboard
+ don't allow CrawlSwitchboard to exit application
making network param unused
Changed Files: source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/search/Switchboard.java
Fri Jul 01 00:02:10 CEST 2016
by reger
prevent dealing with -UNRESOLVED_PATTERN- eventID parameter in html includes
on first landing on search page
Changed Files: htroot/yacysearch.java
Wed Jun 29 23:27:59 CEST 2016
by reger
add de hint translation for CrawlStartScanner_p
rem missing translation line in other lng
Changed Files: locales/cn.lng, locales/de.lng, locales/hi.lng, locales/master.lng.xlf, locales/ru.lng
Tue Jun 28 23:44:28 CEST 2016
by reger
ignore empty input lines in FileUtils.getListArray() to poka joke blacklist read.
equalizes behavior with getListString()
improves: case were blacklist file contained a undesired empty line, not 
fixed by blacklist-cleaner.
Changed Files: source/net/yacy/kelondro/util/FileUtils.java
Tue Jun 28 20:58:58 CEST 2016
by sixcooler
rise limit of reversion to >9999
Changed Files: libbuild/GitRevMavenTask/src/GitRevMavenTask.java, libbuild/GitRevTask/GitRevTask.java
Mon Jun 27 03:12:39 CEST 2016
by reger
fix NPE in CrawlMonitorRemoteStart servlet due to missing startURL
+ add a startURL attribute while generating news record for above (in Crawler_p)
Changed Files: htroot/CrawlMonitorRemoteStart.java, htroot/Crawler_p.java
Mon Jun 27 00:17:18 CEST 2016
by reger
remove redundant/unused translation in ru.lng, unify Network.html header
+ dto. in uk.lng and comment-out some German leftovers.
+ upd master.lng accordingly
Changed Files: locales/master.lng.xlf, locales/ru.lng, locales/uk.lng
Sun Jun 26 06:46:26 CEST 2016
by reger
exclude rejected results from result count 
  (by using the resultcontainer.size instead of input docList.size)
skip waiting for write-search-result-to-local-index
  (by removing the Thread.join - which will bring a small performance increase)
Changed Files: source/net/yacy/peers/Protocol.java
Sat Jun 25 02:59:44 CEST 2016
by reger
wiki header code test string add "closing" tag
Changed Files: test/java/net/yacy/data/wiki/WikiCodeTest.java
Sat Jun 25 02:46:44 CEST 2016
by reger
internal wiki code, require header line to start with markup 
(to allow something like  "one=two"  as text)
+ incl. test case
Changed Files: source/net/yacy/data/wiki/WikiCode.java, test/java/net/yacy/data/wiki/WikiCodeTest.java
Fri Jun 24 23:52:42 CEST 2016
by reger
add maven release profile plugin version (following maven's suggestion)
+ upd some plugin version
Changed Files: pom.xml
Fri Jun 24 01:42:04 CEST 2016
by reger
fix missing quote in gr.lng, it.lng, add sentence in master.lng.xlf
and remove 2 not needed entries
Changed Files: locales/gr.lng, locales/it.lng, locales/master.lng.xlf
Tue Jun 21 00:14:08 CEST 2016
by reger
upd to slf4j-api-1.7.21.jar with related jars
Changed Files: .classpath, build.xml, lib/jcl-over-slf4j-1.7.21.jar, lib/log4j-over-slf4j-1.7.21.jar, lib/slf4j-api-1.7.21.jar, lib/slf4j-jdk14-1.7.21.jar, pom.xml
Sun Jun 19 22:14:51 CEST 2016
by Orbiter
Merge pull request #58 from luccioman/responsive

Make YaCy usable on small screen devices
Changed Files: htroot/env/templates/metas.template
Sun Jun 19 21:49:54 CEST 2016
by reger
upd to pdfbox-2.0.2.jar
Changed Files: .classpath, build.xml, lib/fontbox-2.0.2.License, lib/fontbox-2.0.2.jar, lib/pdfbox-2.0.2.License, lib/pdfbox-2.0.2.jar, pom.xml
Sun Jun 19 21:39:44 CEST 2016
by reger
upd to httpcore-4.4.5.jar
Changed Files: .classpath, build.xml, lib/httpcore-4.4.5.License, lib/httpcore-4.4.5.jar, pom.xml
Sun Jun 19 20:12:28 CEST 2016
by reger
Add a tranlation master (locales/master.lng.xlf) in xlf format.
This is used by Translator servlet to present text to be translated for 
all languages. The file was created by the entries of the 2 best maintained
languages (ru.lng and de.lng). 
+ add view translation to de.lng and harmonize some entries to master.lng.xlf
Changed Files: locales/de.lng, locales/master.lng.xlf
Sun Jun 19 17:24:22 CEST 2016
by reger
fr.lng add entry for Performance_p.html, submenuDesign.template
correct some quoted entries and equalize some keys to common
Changed Files: locales/fr.lng
Sat Jun 18 17:06:40 CEST 2016
by reger
Translator servlet filename drop-down list on "filter untranslated" switch,
filter list to contain only files with not translated items
Changed Files: htroot/Translator_p.java
Thu Jun 16 11:53:22 CEST 2016
by luccioman
Merge remote-tracking branch 'origin/master' into responsive
Changed Files: htroot/Translator_p.html, htroot/Translator_p.java, source/net/yacy/data/Translator.java
Thu Jun 16 10:59:34 CEST 2016
by luccioman
Merge remote-tracking branch 'origin/master' into docker
Changed Files: htroot/Translator_p.html, htroot/Translator_p.java, source/net/yacy/data/Translator.java
Wed Jun 15 22:32:27 CEST 2016
by reger
add button (Translator servlet) to switch one already translated line in edit mode
Changed Files: htroot/Translator_p.html, htroot/Translator_p.java
Tue Jun 14 22:14:46 CEST 2016
by reger
have Translator servlet adhoc apply added translation by translating a single file
+ fix NPE in Translator, coming from translation read by TranslatorXliff 
  which allows null content for not translated key's
Changed Files: htroot/Translator_p.java, source/net/yacy/data/Translator.java
Tue Jun 14 17:03:54 CEST 2016
by luccioman
Merge remote-tracking branch 'origin/master' into docker
Changed Files: locales/sk.lng, source/net/yacy/data/Translator.java, source/net/yacy/search/Switchboard.java, test/java/net/yacy/data/TranslatorTest.java
Tue Jun 14 01:50:32 CEST 2016
by reger
clean out not used Switchboard variables
counter indexedPages, const xstackCrawlSlots
Changed Files: source/net/yacy/search/Switchboard.java
Mon Jun 13 02:03:31 CEST 2016
by reger
rem German lines in sk.lng 
and line not included in file (status_p.inc)
Changed Files: locales/sk.lng
Fri Jun 10 01:14:19 CEST 2016
by reger
Translator to take caution of word boundaries to identify text portion to
be translated. To avoid key="TEST" sourcetext="this is a myTESTcase for it"
translation of partial terms/words.
Add check of word boundary before and after sourcetext (incl. take care
of current praxis for key to be delimetered by > < 
+ add test case
Changed Files: source/net/yacy/data/Translator.java, test/java/net/yacy/data/TranslatorTest.java
Thu Jun 09 09:32:56 CEST 2016
by luccioman
Added last missing JS licences
Changed Files: htroot/jslicense.html
Thu Jun 09 02:22:02 CEST 2016
by reger
upd to jsoup-1.9.2
Changed Files: .classpath, build.xml, lib/jsoup-1.9.2.jar, pom.xml
Wed Jun 08 01:51:51 CEST 2016
by reger
add DE translation Translator_p
+ html conform order form/fieldset
Changed Files: htroot/Translator_p.html, locales/de.lng
Tue Jun 07 23:00:41 CEST 2016
by luccioman
Added license info for jquery.tagsinput.

Also updated some license urls and identifiers to ones recognized by
LibreJS.
Changed Files: htroot/jslicense.html
Mon Jun 06 09:44:48 CEST 2016
by luccioman
Added remaining scripts license information
Changed Files: htroot/jslicense.html
Mon Jun 06 09:44:48 CEST 2016
by luccioman
Added d3.v3.js license information
Changed Files: htroot/jslicense.html
Sun Jun 05 23:01:45 CEST 2016
by reger
Adjust TranslatorXliff to load default 1st and merge downloaded or modified local translation.
process 1. load default from locales/*.* 
        2. load and merge(overwrite) from DATA/LOCALE/*.* (can be partial translation as it is merged)
- include all entries from DATA/LOCAL to be edited in Translator servlet
  and save just modifications (instead of full list) to DATA/LOCALE

This shall make it easy to share modifications.
Changed Files: htroot/Translator_p.java, source/net/yacy/utils/translation/CreateTranslationMasters.java, source/net/yacy/utils/translation/TranslatorXliff.java
Sun Jun 05 21:46:55 CEST 2016
by reger
fix de, uk translation RemoteCrawl_p.html, WatchWebStructure_p.html
harmonize status.html
Changed Files: locales/de.lng, locales/uk.lng
Sun Jun 05 20:50:22 CEST 2016
by Michael Peter Christen
Merge pull request #54 from luccioman/debian

Update debian dependency to allow installing with java 1.8
Changed Files: debian/control
Sun Jun 05 03:57:08 CEST 2016
by reger
optimize Translator
- translateFilesRecursive: load translation once (reduce io), return true on complete success
  - remove resulting unused translateFiles() variant
- translate: use StringBuilder parameter (skip toString conversion)
- remove not needed static declaration
- upd some javadoc
Changed Files: source/net/yacy/data/Translator.java
Sat Jun 04 23:51:28 CEST 2016
by reger
disable input field of translated text in Translator_p servlet
As currently only adding missing translation is supported
Changed Files: htroot/Translator_p.html, htroot/Translator_p.java
Fri Jun 03 22:07:45 CEST 2016
by luccioman
Added JavaScript license information

First pass applied for YaCy index and administration first page, checked
with LibreJS 6.0.13.
Changed Files: htroot/env/templates/header.template, htroot/env/templates/metas.template, htroot/env/templates/simpleheader.template, htroot/js/html.js, htroot/jslicense.html, htroot/jslicense.java
Fri Jun 03 21:42:14 CEST 2016
by reger
fix accidental use of Java 8 function in Translator_p
thanks to luccioman  
Changed Files: htroot/Translator_p.java
Fri Jun 03 15:11:41 CEST 2016
by luccioman
Improve final image size by concatenating some strategic commands
Changed Files: docker/Dockerfile
Fri Jun 03 02:46:11 CEST 2016
by reger
on language file delete, del local editfile too (in ConfigLanguage_p servlet)
Changed Files: htroot/ConfigLanguage_p.java
Thu Jun 02 22:15:57 CEST 2016
by reger
upd to commons-fileupload-1.3.2
Changed Files: .classpath, build.xml, lib/commons-fileupload-1.3.2.License, pom.xml
Thu Jun 02 12:10:12 CEST 2016
by luccioman
git, compile and cleanup in one instruction
Changed Files: docker/Dockerfile.alpine
Thu Jun 02 10:03:28 CEST 2016
by luccioman
Updated docker readme

Added image variants, tags, and license information.
Changed Files: docker/Readme.md
Thu Jun 02 09:24:44 CEST 2016
by luccioman
Improved curl commands (fail if URL is unreachable).
Changed Files: docker/Dockerfile.alpine
Thu Jun 02 01:42:15 CEST 2016
by reger
remove with localHostNames redundant (but unused) publicIPv4HostNames and publicIPv6HostNames
to free unused resources
Changed Files: source/net/yacy/cora/protocol/Domains.java
Wed Jun 01 22:14:58 CEST 2016
by luccioman
Compacted some instructions to reduce final image size
Changed Files: docker/Dockerfile.alpine
Wed Jun 01 21:50:40 CEST 2016
by luccioman
Created a Dockerfile for yacy base on Alpine Linux
Changed Files: docker/Dockerfile.alpine
Wed Jun 01 21:49:47 CEST 2016
by luccioman
Updated comments
Changed Files: docker/Dockerfile
Wed Jun 01 01:14:22 CEST 2016
by reger
Use transparent addToCrawler in AutoSearch instead of addToIndex

This would likely also be of advantage for RSS import/schedule as
following bug-reports suggest
http://mantis.tokeek.de/view.php?id=569
http://mantis.tokeek.de/view.php?id=655
Changed Files: source/net/yacy/search/AutoSearch.java, source/net/yacy/search/Switchboard.java
Tue May 31 09:51:17 CEST 2016
by luc
Added details about volumes management.
Changed Files: docker/Readme.md
Tue May 31 09:50:37 CEST 2016
by luc
Set process as non-root with "USER" instruction.

Thus there is no need to get sudo package.
Changed Files: docker/Dockerfile
Tue May 31 09:28:08 CEST 2016
by luccioman
Added virtual machines and containers block
Changed Files: readme.mediawiki
Tue May 31 00:12:18 CEST 2016
by luc
Use latest official Docker java image as a basis.

This image is a debian Jessie with an openjdk-8 from debian backports.
This avoid downloading openjdk dependencies through apt-get each time
YaCy docker image is built.
Changed Files: docker/Dockerfile
Mon May 30 23:21:35 CEST 2016
by luc
Run yacy daemon as non-root for improved security.
Changed Files: docker/Dockerfile
Mon May 30 09:14:23 CEST 2016
by luc
Added curl dependency to allow running stopYACY.sh and /bin sh scripts.
Changed Files: docker/Dockerfile, docker/Readme.md
Mon May 30 00:53:53 CEST 2016
by reger
correct translation key "About This ...." for simpleheader.template
in de, ja
+ rem obsolete key "Peer Owner Profile" in cn, sk, uk for simpleheader.template
+ hide empty account lines in ViewProfile servlet
Changed Files: htroot/ViewProfile.html, locales/cn.lng, locales/de.lng, locales/ja.lng, locales/sk.lng, locales/uk.lng
Sun May 29 20:35:57 CEST 2016
by reger
fix 2 more servlet RuntimeException in intranet mode thrown due to seed.getIP()
returning null in intranet mode (in servlets: ConfigSearchBox, Load_PHPBB3
+remove unused (const ∅) seed.IPTYPE
Changed Files: htroot/ConfigSearchBox.java, htroot/Load_PHPBB3.java, source/net/yacy/peers/Seed.java
Sun May 29 01:25:47 CEST 2016
by reger
fix: assure close inputstream in TranslatorXliff after reading xlf file
by using try-wiht-resource block
Changed Files: source/net/yacy/utils/translation/TranslatorXliff.java
Fri May 27 17:30:17 CEST 2016
by reger
fix RuntimeException in ConfigRobotsTxt_p in intranet mode
due to getIP returning null, by removing redundant property setting of "clientname"
which is globally set by template engine
Changed Files: htroot/ConfigRobotsTxt_p.java
Fri May 27 10:29:31 CEST 2016
by luc
Set more generic java runtime dependencies.

This is consistent with official Debian java policy :
https://www.debian.org/doc/packaging-manuals/java-policy/x105.html
Changed Files: debian/control
Fri May 27 03:10:25 CEST 2016
by reger
fix NPE in Load_MediawikiWiki servlet in intranet mode
  - in intranet mode getip returns null causing a NPE
  - adjust starturl (which was set to http://localip/repository) which is never the start url for the Mediawiki
+ correct javadoc for seed.getIP()
Changed Files: htroot/Load_MediawikiWiki.java, source/net/yacy/peers/Seed.java
Thu May 26 10:08:04 CEST 2016
by luc
Enable yacy install on debian with openjdk-8 only

Also added curl or wget in Suggests as they are required to run sh
scripts in /bin.
Changed Files: debian/control
Wed May 25 06:58:51 CEST 2016
by Michael Peter Christen
added hint in status to upgrade to java 1.8
Changed Files: htroot/Status.html, htroot/Status.java
Wed May 25 06:47:06 CEST 2016
by Michael Peter Christen
added column in network table for JRE
Changed Files: htroot/Network.html, htroot/Network.java
Tue May 24 23:11:59 CEST 2016
by Michael Peter Christen
add the JRE version in the seed. Purpose: identify if it is possible to
migrate to new JRE version
Changed Files: source/net/yacy/peers/Seed.java, source/net/yacy/peers/SeedDB.java
Tue May 24 13:31:40 CEST 2016
by Michael Peter Christen
Merge pull request #52 from luccioman/docker

Docker image building from latest sources
Changed Files: docker/Dockerfile, docker/Readme.md, docker/docker-cloud.yml
Tue May 24 13:31:03 CEST 2016
by Michael Peter Christen
Merge pull request #53 from luccioman/accessibility

Use aria role to reduce rssTerminal verbosity.
Changed Files: htroot/rssTerminal.html
Tue May 24 00:26:28 CEST 2016
by reger
add InputStream close after reading input file (Vocabulary_p servlet)
Changed Files: htroot/Vocabulary_p.java, source/net/yacy/kelondro/util/FileUtils.java
Mon May 23 07:55:47 CEST 2016
by luccioman
Use aria role to reduce rssTerminal verbosity.

This is a fix for http://mantis.tokeek.de/view.php?id=643
Changed Files: htroot/rssTerminal.html
Mon May 23 02:47:16 CEST 2016
by reger
upd to icu4j-57_1
Changed Files: .classpath, build.xml, lib/icu4j-57_1.jar, pom.xml
Sun May 22 23:40:37 CEST 2016
by luc
Merge remote-tracking branch 'origin/master' into docker
Changed Files: defaults/yacy.init, htroot/RankingSolr_p.html, htroot/RankingSolr_p.java, source/net/yacy/http/servlets/SolrSelectServlet.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/query/QueryParams.java
Sun May 22 23:33:44 CEST 2016
by luc
Detailed how default passord is encoded
Changed Files: docker/Dockerfile
Sun May 22 22:43:56 CEST 2016
by reger
adjust Solr select servlet to lates bugfix for boostquery (bq param)
to split query into multiple parameter on line separator in input query.
e.g. split "crawldepth_i_0^10.0 \n crawldepth_i:1^5.0"
but allow   "url_file_ext_s:jpg OR url_file_ext_s:png"  to be unsplitted
Changed Files: source/net/yacy/http/servlets/SolrSelectServlet.java
Sun May 22 22:24:04 CEST 2016
by Burkhard
Merge pull request #51 from JeremyRand/multiple-boost-query

Fix multiple boost queries
Changed Files: defaults/yacy.init, htroot/RankingSolr_p.html, htroot/RankingSolr_p.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/query/QueryParams.java
Sun May 22 14:41:58 CEST 2016
by luc
Merge remote-tracking branch 'origin/master' into docker
Changed Files: htroot/Vocabulary_p.java, locales/cn.lng, source/net/yacy/kelondro/util/FileUtils.java
Sun May 22 05:38:26 CEST 2016
by reger
make use and activate autodetect charset in Vocabulary input from file
+ revert mistake of empty cn.lng
Changed Files: htroot/Vocabulary_p.java, locales/cn.lng, source/net/yacy/kelondro/util/FileUtils.java
Sat May 21 19:28:25 CEST 2016
by luc
Merge remote-tracking branch 'origin/master' into docker
Changed Files: .classpath, build.xml, lib/fontbox-2.0.1.License, lib/fontbox-2.0.1.jar, lib/pdfbox-2.0.1.License, lib/pdfbox-2.0.1.jar, pom.xml, source/net/yacy/cora/util/Html2Image.java, source/net/yacy/document/parser/pdfParser.java
Wed May 18 04:58:26 CEST 2016
by JeremyRand
Properly support multiple Boost Queries.  (Previous code was broken because it concatenated multiple Boost Queries together rather than passing Solr an array.)
Changed Files: defaults/yacy.init, htroot/RankingSolr_p.html, htroot/RankingSolr_p.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/query/QueryParams.java
Wed May 18 04:51:06 CEST 2016
by JeremyRand
Refactor escaping in config file read/write code.  Now it uses Apache Commons StringUtils instead of RegEx.
Changed Files: source/net/yacy/kelondro/util/FileUtils.java
Fri May 20 23:12:16 CEST 2016
by reger
upd to PDFBox 2.0.1
Changed Files: .classpath, build.xml, lib/fontbox-2.0.1.License, lib/fontbox-2.0.1.jar, lib/pdfbox-2.0.1.License, lib/pdfbox-2.0.1.jar, pom.xml, source/net/yacy/cora/util/Html2Image.java, source/net/yacy/document/parser/pdfParser.java
Fri May 20 15:47:34 CEST 2016
by luc
Enabled autoredeploy, removed useless volume

- Autoredeploy allow dockercloud to redeploy container when a new image
is pushed on dockerhub. 
- volume /opt/yacy_search_server/DATA is already declared in docker
image and allow persistence of data inside deployed container.
Changed Files: docker/docker-cloud.yml
Thu May 19 01:59:00 CEST 2016
by reger
del cytag trail for own index.html (save resource not used by default)
Changed Files: htroot/index.html, source/net/yacy/search/Switchboard.java
Tue May 17 22:07:46 CEST 2016
by luccioman
Added one-click docker cloud deploy
Changed Files: docker/Readme.md
Tue May 17 21:58:24 CEST 2016
by luccioman
Improved titles levels
Changed Files: docker/Readme.md
Tue May 17 21:56:06 CEST 2016
by luccioman
Created a docker cloud stack file

May be used for one click deploy on docker cloud (see https://docs.docker.com/docker-cloud/apps/deploy-to-cloud-btn/)
Changed Files: docker/docker-cloud.yml
Mon May 16 02:13:33 CEST 2016
by reger
extend snapshot Html2Image.pdf2image to use PDFBox image export capability
if no external tool installed (and for Win)
Resulting jpg are not always perfect (if graphic included) but imho sufficient.
Changed Files: source/net/yacy/cora/util/Html2Image.java, source/net/yacy/document/parser/pdfParser.java, test/java/net/yacy/cora/util/Html2ImageTest.java
Sat May 14 09:02:44 CEST 2016
by luccioman
Merge pull request #2 from yacy/master

Merge master
Changed Files: htroot/api/citation.java
Fri May 13 14:47:50 CEST 2016
by luc
Try reducing even more final docker image size
Changed Files: docker/Dockerfile
Fri May 13 14:34:02 CEST 2016
by luc
Reduce docker image size
Changed Files: docker/Dockerfile
Fri May 13 14:32:54 CEST 2016
by luc
Added docker readme
Changed Files: docker/Readme.md
Thu May 12 18:44:41 CEST 2016
by luccioman
Initialize admin password from default or env
Changed Files: docker/Dockerfile
Thu May 12 18:07:39 CEST 2016
by luccioman
save resource for unused blacklistFiles map (#1)
Changed Files: htroot/Blacklist_p.html, source/net/yacy/repository/Blacklist.java
Thu May 12 17:50:21 CEST 2016
by luccioman
Moved again Dockerfile to docker subdirectory
Changed Files:
Thu May 12 17:49:55 CEST 2016
by luccioman
Moved again Dockerfile to docker subdirectory
Changed Files: docker/Dockerfile
Thu May 12 09:30:59 CEST 2016
by luccioman
Modified sources getting
Changed Files: Dockerfile
Thu May 12 00:13:57 CEST 2016
by reger
save resource for unused blacklistFiles map
Changed Files: htroot/Blacklist_p.html, source/net/yacy/repository/Blacklist.java
Wed May 11 18:47:30 CEST 2016
by luccioman
Try for automated build compatibility on dockerhub

Added Dockerfile on master
Changed Files: Dockerfile
Wed May 11 18:01:36 CEST 2016
by luccioman
Try for automated build compatibility on dockerhub

Remove (unnecessary?) clean in ant command
Changed Files: Dockerfile
Wed May 11 17:46:50 CEST 2016
by luccioman
Trace also hidden files in sources
Changed Files: Dockerfile
Wed May 11 17:38:24 CEST 2016
by luccioman
Moved Dockerfile to project root

This is a try for automated build compatibility with Docker Hub
Changed Files:
Wed May 11 17:37:25 CEST 2016
by luccioman
Moved Dockerfile to project root

This is a try for automated build compatibility on Docker Hub
Changed Files: Dockerfile
Wed May 11 17:22:01 CEST 2016
by luccioman
Try for automated build compatibility on dockerhub
Changed Files: docker/Dockerfile
Wed May 11 15:21:57 CEST 2016
by luccioman
remove unused wget package dependency
Changed Files: docker/Dockerfile
Wed May 11 15:05:54 CEST 2016
by luccioman
Corrected COPY instruction
Changed Files: docker/Dockerfile
Wed May 11 14:51:42 CEST 2016
by luccioman
Try modifications for automated build on dockerhub
Changed Files: docker/Dockerfile
Wed May 11 09:40:58 CEST 2016
by luc
Added a Dockerfile

The goal is to automate building of yacy docker image from latest
commits.
Changed Files: docker/Dockerfile
Mon May 09 22:05:50 CEST 2016
by Andreas
Create .gitignore
Changed Files: .gitignore
Mon May 09 21:59:47 CEST 2016
by Andreas
Delete .gitignore
Changed Files:
Mon May 09 21:52:08 CEST 2016
by Andreas
Update .gitignore
Changed Files:
Mon May 09 21:51:29 CEST 2016
by Andreas
Update .gitignore
Changed Files: .gitignore
Mon May 09 21:50:20 CEST 2016
by Andreas
Delete YaCy ottdist [Builder].launch
Changed Files:
Mon May 09 21:42:02 CEST 2016
by otter
Merge remote-tracking branch 'origin/master'
Changed Files: .project
Sun May 08 22:20:24 CEST 2016
by otter
again
Changed Files: .externalToolBuilders/YaCy ottdist [Builder].launch
Sun May 08 20:17:18 CEST 2016
by otter
optimized for small y values
Changed Files: .gitignore, htroot/NetworkHistory.java
Sun May 08 20:07:04 CEST 2016
by Andreas
Update .project
Changed Files: .project
Sun May 08 16:41:51 CEST 2016
by Andreas
Merge pull request #5 from yacy/master

Get my fork synced #5
Changed Files:
Sun May 08 05:27:19 CEST 2016
by reger
optimize surftip data load,
double load (index, loader) not neccessary, getMetadata already suficient
+ lng file adjustments
Changed Files: htroot/yacysearch.java, locales/de.lng, locales/fr.lng
Fri May 06 21:05:15 CEST 2016
by reger
cleanup connectPeer & yacyVersion.latestRelease usage
obsolete since 
https://github.com/yacy/yacy_search_server/commit/527b3decdee2951910bef96f5e5e915fe7c9c73e
Changed Files: source/net/yacy/peers/PeerActions.java, source/net/yacy/peers/operation/yacyVersion.java
Fri May 06 02:47:04 CEST 2016
by reger
cleanup hack for partial Solr update on multivalued datefields
has been fixed in Solr http://issues.apache.org/jira/browse/SOLR-8050
Changed Files: source/net/yacy/search/Switchboard.java
Thu May 05 23:53:46 CEST 2016
by reger
populate translation "index browser"
Changed Files: locales/fr.lng, locales/ja.lng
Thu May 05 00:55:22 CEST 2016
by reger
clean-out unused SwitchboardConstants
Changed Files: source/net/yacy/search/SwitchboardConstants.java
Wed May 04 01:30:10 CEST 2016
by reger
delete obsolete SEARCHRESULT busythread constants
not used since 29.05.2013 18:27:27
0c1a018bbde9c9e67bc000b6a3fd8dbd1706a6f3
Changed Files: defaults/yacy.init, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java
Tue May 03 15:00:10 CEST 2016
by Andreas
Make sure "leftscale" is positive
Changed Files: htroot/NetworkHistory.java
Tue May 03 03:06:06 CEST 2016
by reger
apply bugfix for ChartPlotter from Pullreq 42
https://github.com/yacy/yacy_search_server/pull/42
thanks to otteresk (https://github.com/otteresk)
Changed Files: source/net/yacy/visualization/ChartPlotter.java
Tue May 03 00:25:26 CEST 2016
by reger
upd to commons-io-2.5
Changed Files: .classpath, build.xml, lib/commons-io-2.5.License, lib/commons-io-2.5.jar, pom.xml
Sun May 01 02:24:13 CEST 2016
by reger
move obsolete submenuCookie.template translation to submenuAccessTracker.template entry
Changed Files: locales/de.lng, locales/fr.lng, locales/ja.lng, locales/sk.lng, locales/uk.lng
Sun May 01 02:22:05 CEST 2016
by reger
prevent "unchecked conversion" compiler message
+ include "translate" property in xlf "trans-unit" export
Changed Files: source/net/yacy/utils/translation/CreateTranslationMasters.java, source/net/yacy/utils/translation/TranslatorXliff.java
Thu Apr 28 02:07:15 CEST 2016
by reger
skip caching of knownURL during transferRWI
(cache values not used)
Changed Files: htroot/yacy/transferRWI.java
Wed Apr 27 21:03:01 CEST 2016
by reger
upd to jsoup-1.9.1
Changed Files: .classpath, build.xml, lib/jsoup-1.9.1.jar, pom.xml
Tue Apr 26 23:36:08 CEST 2016
by reger
upd to metadata-extractor-2.9.1
Changed Files: .classpath, build.xml, lib/metadata-extractor-2.9.1.License, lib/metadata-extractor-2.9.1.jar, pom.xml
Mon Apr 25 01:57:33 CEST 2016
by reger
exclude unused protocol param "duetime"
(receiver interpretes param "time" only)
Changed Files: source/net/yacy/peers/Protocol.java
Thu Apr 21 00:51:08 CEST 2016
by reger
keep addon/Notepad++ keyword marker on lng export 
(length of remarks devider line)
+ harmonize status_p.inc lng text
Changed Files: htroot/Status_p.inc, locales/cn.lng, locales/de.lng, locales/fr.lng, locales/ja.lng, locales/ru.lng, locales/uk.lng, source/net/yacy/utils/translation/TranslatorXliff.java
Wed Apr 20 02:07:33 CEST 2016
by reger
take out obsolete "ranking" directory from maven packaging
Changed Files: assembly.xml
Sun Apr 17 22:54:28 CEST 2016
by reger
fix version in locale version file
(it's compared to full version)
Changed Files: locales/de.lng, source/net/yacy/data/Translator.java
Sat Apr 16 23:31:03 CEST 2016
by reger
upd commons-compress 1.11
Changed Files: .classpath, build.xml, lib/commons-compress-1.11.License, lib/commons-compress-1.11.jar, pom.xml, source/net/yacy/document/parser/zipParser.java
Sat Apr 16 00:22:48 CEST 2016
by reger
delete crawl queue on init exception 
(happens occasionally on path name vaiolation and will never get resolved)
Changed Files: source/net/yacy/crawler/HostBalancer.java, source/net/yacy/crawler/HostQueue.java
Thu Apr 14 22:14:32 CEST 2016
by reger
revert call condition f. migration.installSkins 
(a bug introduced in fb8ae14b217cb247d97830ff5237c80a4fe3c93e , 
see comment on that commit )
Changed Files: source/net/yacy/migration.java
Thu Apr 14 02:54:41 CEST 2016
by reger
remove never used Domains.myHostNames set
Changed Files: source/net/yacy/cora/protocol/Domains.java
Sun Apr 10 21:32:53 CEST 2016
by reger
translate ServerScannerList.html
Changed Files: locales/cn.lng, locales/de.lng, locales/hi.lng, locales/ru.lng, locales/uk.lng
Sun Apr 10 05:33:21 CEST 2016
by reger
support xliff for downloaded of language file
Changed Files: htroot/ConfigLanguage_p.html, htroot/ConfigLanguage_p.java, locales/de.lng
Tue Apr 05 01:57:32 CEST 2016
by reger
Added utility class CreateTranslationMasters to create a language independant
translation master as source to harmonize individual translation files
Included a main to create masters in YaCy an xliff format for testing 

+ restrict TranslatorXliff to use only entries with State=translated

P.S. used https://open-language-tools.java.net/editor/about-xliff-editor.html to
experiement with xlf output (haven't a Pootle avail.)
Changed Files: source/net/yacy/utils/translation/CreateTranslationMasters.java, source/net/yacy/utils/translation/TranslatorXliff.java
Tue Apr 05 01:43:05 CEST 2016
by reger
rem/mark German leftovers in translation of other languages files
Changed Files: locales/de.lng, locales/fr.lng, locales/hi.lng, locales/ru.lng, locales/sk.lng
Sun Apr 03 00:45:01 CEST 2016
by reger
clean-up submenuUseCaseAccount.template translation
Changed Files: locales/cn.lng, locales/de.lng, locales/fr.lng, locales/hi.lng, locales/ja.lng, locales/ru.lng, locales/sk.lng, locales/uk.lng
Thu Mar 31 01:51:25 CEST 2016
by reger
fix empty drop down list in ConfigLanguage after wrong/empty download
+ add xliff translated attribut
+ append japanese lng name
Changed Files: htroot/ConfigLanguage_p.java, source/net/yacy/data/Translator.java, source/net/yacy/utils/translation/TranslatorXliff.java
Wed Mar 30 01:09:13 CEST 2016
by reger
translate Network History screen -> de
+ remove leftover debug line
Changed Files: locales/de.lng, source/net/yacy/utils/translation/TranslatorXliff.java
Tue Mar 29 21:31:02 CEST 2016
by reger
remove unused translator routine which also doesn't handle rel path input
+ correct some language file match issues
Changed Files: locales/cn.lng, locales/de.lng, locales/ja.lng, locales/ru.lng, locales/uk.lng, source/net/yacy/data/Translator.java
Mon Mar 28 05:31:17 CEST 2016
by reger
use UTF8 to write imported language files
Changed Files: htroot/ConfigLanguage_p.java
Mon Mar 28 04:38:07 CEST 2016
by reger
upd obsolete translation (blekko)
Changed Files: locales/ru.lng, locales/uk.lng
Thu Mar 24 19:26:38 CET 2016
by reger
add test case for optimized toTokens()
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java, test/java/net/yacy/cora/document/id/MultiProtocolURLTest.java
Wed Mar 23 00:01:43 CET 2016
by reger
adjust build class path (- htroot)
according to comment on https://github.com/yacy/yacy_search_server/commit/0dcda3809ed4cbc7f8a95e95f48814c24c4ed3ff
Changed Files: build.xml
Sun Mar 20 23:46:39 CET 2016
by reger
adjust utility ListNonTranslatedFiles for path compare on windows 
(backslash replace)
Changed Files: source/net/yacy/utils/translation/ListNonTranslatedFiles.java
Sun Mar 20 22:25:57 CET 2016
by reger
update de translation ConfigBasic.html (config router +upnp)
Changed Files: locales/de.lng
Sun Mar 20 18:12:55 CET 2016
by reger
keep svnRevision in .init for convert of .conf until release >1.83
Changed Files: defaults/yacy.init
Sun Mar 20 03:42:33 CET 2016
by reger
include Version config string in yacy.init (replacing svnRevision)
Changed Files: defaults/yacy.init
Sun Mar 20 03:34:28 CET 2016
by reger
make migration version safe
Changed Files: source/net/yacy/migration.java, source/net/yacy/peers/Protocol.java, source/net/yacy/yacy.java
Sat Mar 19 18:29:34 CET 2016
by Burkhard
Merge pull request #46 from reelsense/patch-1

Grammar fix - ConfigRobotsTxt_p.html
Changed Files: htroot/ConfigRobotsTxt_p.html
Sat Mar 19 06:29:44 CET 2016
by Bryan Black
Distilled sentence - ConfigNetwork_p

I distilled the first "Distributed Computing Network for Domain" description sentence and corrected grammar.
Changed Files: htroot/ConfigNetwork_p.html
Sat Mar 19 05:40:11 CET 2016
by Bryan Black
Grammar fix

The sentence sounded odd.
Changed Files: htroot/ConfigRobotsTxt_p.html
Fri Mar 18 14:02:08 CET 2016
by luccioman
Added YaCy icons
Changed Files: images/YaCy.ico, images/YaCyLogo_120ppi.png, images/YaCyLogo_60ppi.png, images/yacy.png
Mon Mar 14 22:34:13 CET 2016
by reger
upd to poi-3.14
Changed Files: .classpath, build.xml, lib/poi-3.14-20160307.jar, lib/poi-3.14.License, lib/poi-scratchpad-3.14-20160307.jar, nbproject/project.xml, pom.xml
Mon Mar 14 22:32:06 CET 2016
by reger
reduce logging (EmbeddedSolrConnector.query)
mainly to reduce the frequent metadat checks like
> EmbeddedSolrConnector.query QUERY: q={!cache=false raw f=id}xXxXxX&rows=1&start=0&fl=id,load_date_dt
(p.s. direct servlet queries logged via AccessTracker.addToDump)
Changed Files: source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java
Mon Mar 14 02:22:06 CET 2016
by reger
move example code SearchClient out of yacycore package
to example directory
Changed Files: examples/SimpleSearchClient/pom.xml, examples/SimpleSearchClient/src/YaCySearchClient.java
Mon Mar 14 01:17:41 CET 2016
by reger
adding Mavens output dir to gitignore
Changed Files: .gitignore
Mon Mar 14 00:57:21 CET 2016
by reger
upd Eclipse classpath for slf4j jars
Changed Files: .classpath
Sun Mar 13 05:50:51 CET 2016
by reger
correct "SLF4J: Defaulting to no-operation (NOP) logger implementation"
introduced earlier https://github.com/yacy/yacy_search_server/commit/d9036bb9b77da1dda0f29df6207eb529f7e3361a
Changed Files: pom.xml
Sun Mar 13 04:03:32 CET 2016
by reger
activate hack in libbuild/pom.xml to include main project as build module
This allows to use libbuild/pom as main maven project to build all needed
libs to compile and run YaCy, including YaCy itself.

Furthermore, maven is well supported by Netbeans IDE so we can discontinue
to maintaine Netbeans IDE specific project settings (in directory nbproject,
which lack config for debug anyway, nbproject can be deleted and NB will
compile YaCy via maven).

Additionally adjusted pom's project name to match the name in the repository.
Changed Files: libbuild/pom.xml, pom.xml
Sun Mar 13 01:34:10 CET 2016
by reger
upd to slf4j-1.7.18
remove direct declaration of slf4j-api in pom (let maven do it's job)
Changed Files: build.xml, lib/jcl-over-slf4j-1.7.18.jar, lib/log4j-over-slf4j-1.7.18.jar, lib/slf4j-api-1.7.18.jar, lib/slf4j-jdk14-1.7.18.jar, nbproject/project.xml, pom.xml
Sat Mar 12 02:12:56 CET 2016
by reger
del outdated lib/dependencies.txt
as alternative you may use maven build site:site goal which produces
a complete list of direct and transitive dependencies.
Changed Files:
Sat Mar 12 02:00:23 CET 2016
by reger
upd to httpclient-4.5.2
Changed Files: .classpath, build.xml, lib/httpclient-4.5.2.jar, lib/httpmime-4.5.2.jar, nbproject/project.xml, pom.xml
Sat Mar 12 01:15:05 CET 2016
by reger
put settings_p servlet (back) as start page for System Administration
to display the options available (as tables_p only used for indepth edit)
see http://mantis.tokeek.de/view.php?id=460
Changed Files: htroot/env/templates/header.template, htroot/env/templates/submenuConfig.template
Fri Mar 11 18:12:13 CET 2016
by Michael Peter Christen
0N - basic dump upload servlet infrastructure, to share index dumps
within an experimental new sharing model
Changed Files: htroot/api/share.html, htroot/api/share.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/yacy.java
Fri Mar 11 08:59:33 CET 2016
by Michael Peter Christen
harmonization of http post of files for one and several files - this had
been differently - and wrong for several files. also: base64-encoding
for gzipped push files because our data structures currently only
supports ASCII POST pushes..
Changed Files: htroot/api/push_p.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java
Fri Mar 11 08:55:51 CET 2016
by Michael Peter Christen
0n - added experimental zeronet network which supports intranet peers
(still needs work)
Changed Files: defaults/yacy.network.zeronet.unit, defaults/yacy.networks
Fri Mar 11 08:54:42 CET 2016
by Michael Peter Christen
0n: modified the p2p bootstraping process - rules had been too tight and
did not support the re-start of a network with just one principal peer.
Changed Files: source/net/yacy/peers/Network.java, source/net/yacy/peers/PeerActions.java, source/net/yacy/search/Switchboard.java
Fri Mar 11 07:20:38 CET 2016
by Michael Peter Christen
update to seed list
Changed Files: defaults/yacy.network.freeworld.unit
Fri Mar 11 01:14:54 CET 2016
by reger
remove unused proxyCookieWhiteList from yacy.init
Changed Files: defaults/yacy.init
Wed Mar 09 01:01:58 CET 2016
by reger
more fixes for sevenZip parser exception on specific archives
Found 2 more cases were modified code throws exceptions while original J7Zip
unpacks it.
Reimplemented error prone CoderMixer2ST from unmodified J7Zip.jar. 
Changed Files: lib/J7Zip-modified.jar, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/BindInfo.java, libbuild/J7Zip-modified/src/SevenZip/Archive/Common/CoderMixer2ST.java, libbuild/J7Zip-modified/src/SevenZip/Compression/Branch/BCJ2_x86_Decoder.java, libbuild/J7Zip-modified/src/SevenZip/ICompressCoder2.java
Wed Mar 09 00:56:34 CET 2016
by reger
remove transitive dependency guava from pom
(let Maven do it's job)
Changed Files: pom.xml
Fri Mar 04 23:05:55 CET 2016
by reger
fix delete of temp file after odt % ooxml parser
Close zipfile after parsing
Changed Files: source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java
Wed Mar 02 00:12:34 CET 2016
by reger
use supplied url port to get robots.txt in crawlers hostqueue
Changed Files: source/net/yacy/crawler/HostBalancer.java, source/net/yacy/crawler/HostQueue.java, source/net/yacy/crawler/LegacyBalancer.java, source/net/yacy/crawler/data/Latency.java
Mon Feb 29 00:04:36 CET 2016
by reger
adjust start/stop classpath in build script
(with servlet classloader no need for htroot in system classpath)
Changed Files: build.xml
Sun Feb 28 22:53:41 CET 2016
by reger
harmonize classpath with startYaCy.bat
(with servlet classloader no need for htroot in system classpath)
Changed Files: startYACY.sh
Sun Feb 28 22:10:43 CET 2016
by reger
harmonize classpath with startYaCy.bat
Changed Files: startYACY_debug.bat
Sat Feb 27 03:44:43 CET 2016
by reger
fix mixed logfactory in UrlProxyServlet,
Class doesn't use functions of declared ancestor, change to extend on httpservlet
Changed Files: source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/http/servlets/YaCyProxyServlet.java
Fri Feb 26 00:50:26 CET 2016
by reger
upd to slf4j-1.7.16
Changed Files: .classpath, build.xml, lib/jcl-over-slf4j-1.7.16.jar, lib/log4j-over-slf4j-1.7.16.jar, lib/slf4j-api-1.7.16.jar, lib/slf4j-jdk14-1.7.16.jar, nbproject/project.xml, pom.xml
Wed Feb 24 15:13:20 CET 2016
by Michael Peter Christen
0N - added recording of dump exports if given time frame is not negative
Changed Files: htroot/IndexExport_p.java, source/net/yacy/data/WorkTables.java
Tue Feb 23 18:56:20 CET 2016
by Michael Peter Christen
0N - added option to generate index export files for a specific number
of minutes in the past and reverted latest change. The export file dump
will now contain four data elements: f - first date of index entry write
date, l - last date of index write date, n - now-date of index dump
time, c - count of numbers inside the dump. '0N' denotes a series of
changes which will lead to the opportunity to exchange index data dumps
in a way that is needed to integrate ZeroNet index data. This will be
based on index dump sharing; that causes this commit.
Changed Files: htroot/IndexExport_p.html, htroot/IndexExport_p.java, source/net/yacy/search/index/Fulltext.java
Sun Feb 21 21:38:11 CET 2016
by reger
making WebStructurePicture_p less vulnerable to faulty host input parameter (like host1,,host3)
by continue host loop on exception

inspired by http://mantis.tokeek.de/view.php?id=637
Changed Files: htroot/WebStructurePicture_p.java
Sun Feb 21 00:23:00 CET 2016
by reger
fix assertion exception in toString of MultiProtocolURL
toString of AnchorURL and MultiProtocolURL are identical code
(no need to override or to protect call to parent)

as reported in https://github.com/yacy/yacy_search_server/issues/43
Changed Files: source/net/yacy/cora/document/id/AnchorURL.java, source/net/yacy/cora/document/id/MultiProtocolURL.java
Sat Feb 20 23:17:59 CET 2016
by reger
include initial japaneese translation to language selection
Changed Files: htroot/ConfigBasic.html, htroot/ConfigBasic.java
Sat Feb 20 22:43:41 CET 2016
by Burkhard
Merge pull request #44 from ImpactCrater/master

Created a translation file ja.lng
Changed Files: locales/ja.lng
Sat Feb 20 19:55:33 CET 2016
by ImpactCrater
Created a translation file ja.lng

I wrote a bit of translation to Japanese.
Changed Files: locales/ja.lng
Fri Feb 19 19:26:02 CET 2016
by Michael Peter Christen
added peer hash to export dump name.
Changed Files: htroot/IndexExport_p.html, htroot/IndexExport_p.java
Thu Feb 18 01:09:29 CET 2016
by reger
upd to weupnp-0.1.4.jar
Changed Files: .classpath, build.xml, lib/weupnp-0.1.4.jar, nbproject/project.xml, pom.xml
Wed Feb 17 22:47:03 CET 2016
by reger
del unused statistic parameter (from status servlet)
Changed Files: htroot/Status.java, source/net/yacy/peers/PeerActions.java
Tue Feb 16 17:36:49 CET 2016
by Michael Peter Christen
enhanced robustnes of image computation
Changed Files: htroot/ViewImage.java
Sun Feb 14 21:13:25 CET 2016
by reger
put metadata dc_publisher in corresponding schema field
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Fri Feb 12 01:20:03 CET 2016
by reger
remove servlet classloder internal cache map (to save the resources, cache hits marginal)
- DefaultServlet includes already a class cache "templateMethodCache" which is emptied 
  on low mem status 
- avoid classloader cache gets has no hits but over time holds all (used) servlet classes 
Changed Files: source/net/yacy/server/serverClassLoader.java
Thu Feb 11 09:33:20 CET 2016
by luc
Also check and index eventual icon url information from metadata.
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Wed Feb 10 10:08:07 CET 2016
by luc
Display icons in ViewFile "links" mode.
Changed Files: htroot/ViewFile.java, source/net/yacy/document/Document.java
Wed Feb 10 09:02:21 CET 2016
by luc
Updated getpageinfo outputs to return page icons list.
Changed Files: htroot/api/getpageinfo.java, htroot/api/getpageinfo.json, htroot/api/getpageinfo.xml, htroot/api/getpageinfo_p.java, htroot/api/getpageinfo_p.xml
Mon Feb 08 17:47:26 CET 2016
by otter
format2
Changed Files: htroot/PerformanceMemory_p.java
Mon Feb 08 17:44:26 CET 2016
by otter
format
Changed Files: htroot/PerformanceMemory_p.java
Sun Feb 07 21:16:22 CET 2016
by reger
override detected language (statistic langdetect) only with TLD determided
language if langdetect probability is not high.
+ additionally truncate zh-cn / zh-tw returned by langdetect to 2 char ISO639-1 zh
used by YaCy
Changed Files: source/net/yacy/document/Condenser.java, source/net/yacy/document/language/Identificator.java, source/net/yacy/search/index/Segment.java
Sun Feb 07 01:52:32 CET 2016
by reger
include use of condenser's content text for language detection.
Language identification may show poor performance on documents with short or no
title but clear lang indication in text content. Using content text too
improves lang detection.
+ remove double caching of text in Identificator
Changed Files: source/net/yacy/document/Condenser.java, source/net/yacy/document/language/Identificator.java
Sat Feb 06 20:22:17 CET 2016
by otter
Correctly handle POSTed parameter also with HTTPS activated
Changed Files: htroot/PerformanceMemory_p.java
Fri Feb 05 17:05:36 CET 2016
by luc
Added favicon url transmission in RWI chunks.
Changed Files: htroot/yacysearchitem.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/search/query/SearchEvent.java, test/java/net/yacy/kelondro/data/meta/URIMetadataNodeTest.java
Thu Feb 04 19:20:27 CET 2016
by reger
correct isHidden() for smb from last commit 
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Thu Feb 04 19:16:00 CET 2016
by reger
respect hidden attribute for file and smb directory listing
(hidden directories are not listed, effects crawling of local file system)
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/search/index/DocumentIndex.java
Thu Feb 04 08:14:49 CET 2016
by luc
Extracted intranet and filtype related rules from getFaviconURL func
Changed Files: htroot/yacysearchitem.java, test/java/yacysearchitemTest.java
Thu Feb 04 00:47:42 CET 2016
by reger
compare search page, remove diminished search target
(romso.de, dbpedia.neofonie.de )
Changed Files: htroot/compare_yacy.java
Wed Feb 03 21:50:50 CET 2016
by reger
upd classpath in batches (remove not necessary htroot)
see prev commit
Changed Files: installYaCyWindowsService.bat, startYACY.bat, stopYACY.bat
Tue Feb 02 03:44:01 CET 2016
by reger
add findClass() to servlet class loader (used in YaCyDefaltServlet)
In the 2 cases where servlet calls servlet the jvm classloader chain is
invoked and servlet class loaded by jvm loader (successful while requiring 
htroot in system classpath). This patch uses the standard override design
for loaders to handle these cases (making in not longer crucial to have htroot 
in system classpath, as this classLoader is mainly used for servlets and
looks in this case for the class in the configured path).
+ As the default classloader is parallelcapable we should register this too.
Changed Files: source/net/yacy/server/serverClassLoader.java
Mon Feb 01 21:04:38 CET 2016
by otter
Improved plotting
Changed Files: htroot/Network.html, htroot/NetworkHistory.java, source/net/yacy/visualization/ChartPlotter.java
Thu Jan 28 23:14:09 CET 2016
by reger
expand initRemoteCrawler() to terminate worker threads if called to deactivate
remote crawl.
On startup we save the resources for remote crawler if disabled. Once started
threads are running idle after disable remote crawl. Now threads are terminated
to save the resources also while disabeling during runtime.
+ remove empty class Channels
Changed Files: source/net/yacy/search/Switchboard.java
Tue Jan 26 21:27:44 CET 2016
by reger
support scraping start-/enddate from html tag with property "datetime"
This may be used in html5 <time> tag (which we don't explicite support yet for date in content scraping).
Changed Files: source/net/yacy/document/parser/html/ContentScraper.java
Mon Jan 25 23:54:20 CET 2016
by reger
complet resource cleanup of lists in contentscraper's close()
Changed Files: source/net/yacy/document/parser/html/ContentScraper.java
Mon Jan 25 22:13:04 CET 2016
by reger
add support parsing swf-metadata to swfparser
flash supports metadata tag in swf file with metadata in xmp (xml) format.
parse some common data to include it in the head section of the html string
of converttohtml.
Changed Files: lib/webcat-swf-0.1.jar, libbuild/WebCat-swf/pom.xml, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWF2HTML.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFTagTypes.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWFTagTypesImpl.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/TagParser.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/TagWriter.java
Sun Jan 24 02:30:27 CET 2016
by reger
replace remaining use of java.util.Vector by ArrayList (WebCat-swf)
Changed Files: libbuild/WebCat-swf/src/pt/tumba/parser/swf/ActionParser.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ActionWriter.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Actions.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Base64.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ButtonRecord.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ButtonRecord2.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Frame.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/Movie.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/OutStream.java
Sat Jan 23 22:46:22 CET 2016
by reger
add support for DEFINEFONT3 (swf8) to webcat parser
experienced issue with JPEGTABLE tag (with length=0) causing abort of parsing (ioexception)
as we don't use/need it for text parsing skip this tag.
Changed Files: lib/webcat-swf-0.1.jar, libbuild/WebCat-swf/src/pt/tumba/parser/swf/TagParser.java
Sat Jan 23 20:19:01 CET 2016
by reger
add missing swf tag constants according to latest spec
reduce use of synced vector in webcat parser
Changed Files: libbuild/WebCat-swf/src/com/anotherbigidea/flash/SWFConstants.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/ActionParser.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/SWF2HTML.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/TagParser.java, libbuild/WebCat-swf/src/pt/tumba/parser/swf/TagWriter.java
Thu Jan 21 02:55:05 CET 2016
by reger
pass parsed swf content trough htmlscraper
Swf may contain subset of html tags which shoul'd appear as text.
Especially <font> tag may totally screw up metadata servlet if not filtered out.
Changed Files: source/net/yacy/document/parser/swfParser.java
Mon Jan 18 23:36:18 CET 2016
by reger
replace bugfixed webcat-swf.jar
Changed Files: .classpath, build.xml, lib/webcat-swf-0.1.jar, lib/webcat-swf.License, libbuild/WebCat-swf/pom.xml, nbproject/project.xml, pom.xml
Mon Jan 18 22:29:17 CET 2016
by reger
refactor libbuild/GitRevMavenTask (marvenize)
to be able to add additional modules to build
Changed Files: libbuild/GitRevMavenTask/pom.xml, libbuild/GitRevMavenTask/src/GitRevMavenTask.java, libbuild/pom.xml
Sun Jan 17 14:52:24 CET 2016
by reger
add translation DE,FR submenuRanking.template
upd translation DE RankingSolr_p
Changed Files: locales/de.lng, locales/fr.lng
Sat Jan 16 17:55:20 CET 2016
by reger
add DE translation for submenuCrawler.template
+ upd submenuIndexControl.template
Changed Files: locales/de.lng
Fri Jan 15 02:10:10 CET 2016
by reger
apply remote result count config value to Bookmark Autosearch
+ prepare to make the widely unused Bookmark feature optional 
Changed Files: source/net/yacy/search/AutoSearch.java
Thu Jan 14 22:19:55 CET 2016
by Michael Peter Christen
Merge pull request #40 from Scarfmonster/autocrawl

Automatic crawling
Changed Files: defaults/yacy.init, htroot/Autocrawl_p.html, htroot/Autocrawl_p.java, htroot/env/templates/submenuIndexCreate.template, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java
Thu Jan 14 03:27:33 CET 2016
by Ryszard Go?
Add missing settings to autocrawl settings page
Changed Files: defaults/yacy.init, htroot/Autocrawl_p.html, htroot/Autocrawl_p.java
Thu Jan 14 02:40:46 CET 2016
by Ryszard Go?
Add autocrawl settings page
Changed Files: htroot/Autocrawl_p.html, htroot/Autocrawl_p.java, htroot/env/templates/submenuIndexCreate.template
Thu Jan 14 00:50:23 CET 2016
by Ryszard Go?
Add the Autocrawl thread
Changed Files: defaults/yacy.init, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java
Wed Jan 13 01:30:49 CET 2016
by reger
altered clac. of search result items per page to display 
taking the existing limits into account but make it consistent with search option screen for admin and public user
changes:
  - configured default number of items per page (ConfigPortal_p.html) is used as is (no hardcoded limit)
  - otherwise requests are limited to 100 results per page ( = search option, index.html)
      (this basically is the major change, inc. limit from 20 to 100 for public user)
P.S. - the older grant of more (1000), if no online snippet calculation, is kept (for the time being)

see http://mantis.tokeek.de/view.php?id=627 
Changed Files: defaults/yacy.init, htroot/ConfigPortal.html, htroot/yacysearch.java
Tue Jan 12 22:09:30 CET 2016
by luc
Corrected access to Load_MediawikiWiki.html and Load_PHPBB3.html

A NullPointerException occured when trying to access theses pages in
Robinson (Search portal) mode
Changed Files: htroot/Load_MediawikiWiki.java, htroot/Load_PHPBB3.java
Tue Jan 12 16:28:34 CET 2016
by Ryszard Go?
Create autocrawl profiles
Changed Files: defaults/yacy.init, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/search/SwitchboardConstants.java
Tue Jan 12 01:04:10 CET 2016
by reger
remove contendom depending override of search result items per page
initially introduced https://github.com/yacy/yacy_search_server/commit/e4570bffaf6e8c7c71d8945dbed746a91015819d#diff-ae6c130fc11088c830b00ed9256ab56b
(as one part of unexpected difference in actual vs requested results, partial bugfix for http://mantis.tokeek.de/view.php?id=627 )
Changed Files: htroot/yacysearch.java
Mon Jan 11 13:18:07 CET 2016
by Andreas
Merge pull request #1 from yacy/master

Get my fork synced
Changed Files: htroot/Status.java, locales/de.lng, locales/hi.lng, locales/ru.lng, pom.xml, source/net/yacy/document/parser/swfParser.java
Mon Jan 11 01:38:45 CET 2016
by reger
upd pom javadoc goal
to not fail a build on javadoc errors
Changed Files: pom.xml
Sun Jan 10 01:16:31 CET 2016
by reger
fix swfParser reading file signature
before passing to library (current version expects data w/o signature)
Changed Files: source/net/yacy/document/parser/swfParser.java
Sat Jan 09 20:32:47 CET 2016
by reger
update Maven pom - add release-profile
to create the release archive only if profile is activated (speeding up normal compilation)
- bind install of the 2 jar's not available in repository to validate phase (was clean)
  to automatically add these to local repository (with disadvantage it's done every time)
Changed Files: pom.xml
Sat Jan 09 01:19:06 CET 2016
by reger
fix tray icon switch
(using predefined/correct config name)
Changed Files: htroot/Status.java
Fri Jan 08 23:18:13 CET 2016
by luc
Return more accurate HTTP status 400 with detail message when some error
occurs on ViewImage :
 - missing required parameters
 - url licence invalid
Changed Files: htroot/ViewImage.java, source/net/yacy/data/InvalidURLLicenceException.java, source/net/yacy/http/servlets/TemplateMissingParameterException.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java
Fri Jan 08 20:46:02 CET 2016
by luc
Corrected NullPointerException cases occuring in YJsonResponseWriter
when no description is available.
Changed Files: source/net/yacy/server/serverObjects.java
Fri Jan 08 20:43:49 CET 2016
by luc
Updated documented sample url
Changed Files: source/net/yacy/http/servlets/SolrSelectServlet.java
Fri Jan 08 20:42:57 CET 2016
by luc
Improved URLLicence reliability for use by conccurrent non authaurized
users.
Removed URLLicence generation when unnecessary (authorized users)
Changed Files: htroot/yacysearchitem.html, htroot/yacysearchitem.java, source/net/yacy/cora/federate/solr/responsewriter/YJsonResponseWriter.java, source/net/yacy/data/URLLicense.java, test/net/yacy/data/URLLicenseConcurrentTest.java
Fri Jan 08 02:46:29 CET 2016
by reger
add de translation for IndexExport_p
Changed Files: locales/de.lng
Fri Jan 08 01:10:09 CET 2016
by reger
fix typo in translation (de,hi) for AccessTracker_p
- rem some not translated in ru (-> currently best maintained translation)
Changed Files: locales/de.lng, locales/hi.lng, locales/ru.lng
Thu Jan 07 16:17:37 CET 2016
by reger
further refactor using standard java / (one) utf-8 charset variable
extending initiative of commit https://github.com/yacy/yacy_search_server/commit/9a2575185027fe822956a4edf538c304862ed0d5 
Changed Files: source/net/yacy/cora/document/encoding/UTF8.java, source/net/yacy/cora/federate/opensearch/OpenSearchConnector.java, source/net/yacy/cora/lod/vocabulary/Tagging.java, source/net/yacy/http/servlets/GSAsearchServlet.java, source/net/yacy/http/servlets/SolrSelectServlet.java, source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/http/servlets/YaCyProxyServlet.java
Wed Jan 06 14:57:26 CET 2016
by Michael Peter Christen
Merge pull request #28 from Stepanov-Sergey/patch-1

fixed typos
Changed Files: addon/synonyms/thesaurus_ru_yacy
Wed Jan 06 14:56:55 CET 2016
by Michael Peter Christen
Merge pull request #37 from sudheesh001/LogFix

Log files are commitable and shouldn't be
Changed Files: .gitignore
Wed Jan 06 00:51:49 CET 2016
by reger
remove obsolete css style for removed file CacheAdmin_p.html
and remove from translations
Changed Files: htroot/env/base.css, locales/de.lng, locales/fr.lng, locales/gr.lng, locales/hi.lng, locales/it.lng, locales/sk.lng
Tue Jan 05 10:48:51 CET 2016
by reger
add missing quote chars in sk.lng translation file
+ minor: del one redundancy
Changed Files: locales/cn.lng, locales/de.lng, locales/fr.lng, locales/ru.lng, locales/sk.lng
Mon Jan 04 04:49:10 CET 2016
by reger
remove blekko from opensearch config (not available)
see https://blekko.com/
http://searchengineland.com/goodbye-blekko-search-engine-joins-ibms-watson-team-217633
Changed Files: defaults/heuristicopensearch.conf, htroot/yacysearch.java, locales/de.lng
Mon Jan 04 03:15:08 CET 2016
by sudheesh001
Log files are commitable and shouldn't be
Changed Files: .gitignore
Sun Jan 03 01:11:27 CET 2016
by reger
remove workaround for Solr bug regarding multivalued date fields
fixed in 5.4.0
http://issues.apache.org/jira/browse/SOLR-8050
Changed Files: source/net/yacy/search/Switchboard.java, test/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnectorTest.java
Sat Jan 02 05:03:57 CET 2016
by reger
check error URL cache before adding errorDoc to index
- del obsolete related switchboardconstant
Changed Files: source/net/yacy/search/SwitchboardConstants.java, source/net/yacy/search/index/ErrorCache.java
Fri Jan 01 06:08:41 CET 2016
by reger
remove local credential on download of config files from remote systems
(blacklists, language, skins) to reduce risk to expose md5-pwd.
- remove NoSuchElementException in loop
Changed Files: htroot/BlacklistImpExp_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigLanguage_p.java, htroot/sharedBlacklist_p.java
Thu Dec 31 03:04:13 CET 2015
by reger
reintroduce special handling of file upload multipart/form-data from HTTPDemon.parseMultipart
- add filename to parameter fieldname
- add filecontent to special parameter fieldname$file
(some servlets use this $file parameter)

fix for http://mantis.tokeek.de/view.php?id=542
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Wed Dec 30 21:27:44 CET 2015
by reger
correct filename input in settings_seed_upload.inc form
to get filename
(recently introduced by change form "text" to the more convenient "file" input  https://github.com/yacy/yacy_search_server/commit/50f64ddc3b87c64e6f29dc6f038023f1fb1da7fc)
Changed Files: htroot/Settings_Seed_UploadFile.inc
Tue Dec 29 20:38:45 CET 2015
by reger
fix low memory status hint (dht-in disabled)
http://mantis.tokeek.de/view.php?id=619
Changed Files: htroot/PerformanceQueues_p.java, htroot/Status.html, locales/cn.lng, locales/de.lng, locales/fr.lng, locales/ru.lng, locales/uk.lng, source/net/yacy/search/SwitchboardConstants.java
Tue Dec 29 15:41:19 CET 2015
by reger
remove obsolete yacy.init entries interaction.*
(related to removed triplestore)
Changed Files: defaults/yacy.init, source/net/yacy/cora/document/id/AnchorURL.java
Mon Dec 28 23:20:37 CET 2015
by Michael Peter Christen
added greeting line to interactive search and harmonized display
position with 'Administration' line in all administration pages
Changed Files: htroot/env/templates/header.template, htroot/env/templates/simpleheader.template, htroot/yacyinteractive.html
Mon Dec 28 22:57:31 CET 2015
by Michael Peter Christen
use curl downloads in download script with silent mode
Changed Files: htroot/js/yacyinteractive.js
Mon Dec 28 02:32:00 CET 2015
by reger
fix poison mediawikiimporter output queue also after ExecutionException
in worker thread.
Writer of importer keeps needs a poison to close the file. On exception (e.g. OOM)
add a poison marker in outer most try/catch to assure output queue will terminate
in this condition too (and closes+renames the surrogate/in/xxx.prt file)
Changed Files: source/net/yacy/document/importer/MediawikiImporter.java
Sun Dec 27 01:59:15 CET 2015
by reger
fix mediawikiimporter number format exception on coordinate parsing
handle uncomplete metadata like "NS=43/50//N". 
For other {expr ... } type entries a try catch added
Changed Files: source/net/yacy/data/wiki/WikiCode.java, source/net/yacy/document/importer/MediawikiImporter.java
Sat Dec 26 22:25:08 CET 2015
by Michael Peter Christen
stub to use a new app launcher for mac
Changed Files: addon/YaCy.app/Contents/Info.plist, addon/YaCy.app/Contents/MacOS/JavaAppLauncher, addon/YaCy.app/Contents/Resources/en.lproj/Localizable.strings, build.xml
Sat Dec 26 17:35:46 CET 2015
by reger
increase http header EXPIRES for css and images in DefaultServlet
to increase browser cache hits for not changing content
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Sat Dec 26 01:15:07 CET 2015
by reger
skip loading document on crawl start for YMark bookmarks
by adding a constructor giving the already loaded document as parameter.
Changed Files: source/net/yacy/data/ymark/YMarkTables.java, source/net/yacy/search/Switchboard.java
Thu Dec 24 19:39:37 CET 2015
by reger
limit bookmark tag cloud font-size to 2.0em
Changed Files: htroot/Bookmarks.java
Wed Dec 23 23:11:37 CET 2015
by reger
apply default css styles (class btn) to blacklist* submit buttons 
Changed Files: htroot/BlacklistCleaner_p.html, htroot/BlacklistImpExp_p.html, htroot/BlacklistTest_p.html, htroot/Blacklist_p.html
Tue Dec 22 20:15:08 CET 2015
by reger
use input type=file for choosing IndexImportMediawiki_p dump file
Changed Files: htroot/IndexImportMediawiki_p.html
Tue Dec 22 17:48:24 CET 2015
by reger
fix mime table for ms office / open office documents
(causing wrong parser detect in intranet mode)
Changed Files: defaults/httpd.mime, source/net/yacy/document/parser/ooxmlParser.java
Tue Dec 22 01:36:23 CET 2015
by reger
change some more submit button to bootstrap btn css class
Changed Files: htroot/Performance_p.html, htroot/RegexTest.html, htroot/Vocabulary_p.html
Mon Dec 21 04:42:26 CET 2015
by reger
adjust MultiProtocolURL.protocol detection to handle mailto with "://" in parameters,
and feeding hyperlinks to webgraph processing.
Changed Files: htroot/Crawler_p.java, source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/document/Document.java, source/net/yacy/search/schema/CollectionConfiguration.java
Mon Dec 21 01:05:59 CET 2015
by reger
quick fix: go back to display search results favicon via <img> tag and ViewImage,
! until better solution is found !.

Reason: in IE-Browser no individual favicon is displayed with <object> tag 
(always the default) and only few individual fav's with Firefox (randomly)

hint: to be able to use <img src=ViewImage/> return of default icon was
added back to it.

relates to http://mantis.tokeek.de/view.php?id=629
Changed Files: htroot/ViewImage.java, htroot/yacysearchitem.html
Sun Dec 20 15:49:24 CET 2015
by reger
make use of header.getContentType where possible (mime is normalized afterwards)
otherwise use header.mime() differentiated in prev. commit. 
Changed Files: source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/crawler/retrieval/Response.java, source/net/yacy/http/servlets/YaCyProxyServlet.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/server/http/HTTPDProxyHandler.java
Sun Dec 20 06:44:16 CET 2015
by reger
fix HeaderFramework.mime() to strip charset parameter.
Differentiate mime() and getContentType() which gives the raw header field.
This improves parser detection if charsets are included in http content-type field.
Changed Files: source/net/yacy/cora/protocol/HeaderFramework.java
Sun Dec 20 02:10:49 CET 2015
by reger
fix (todo): correct doc.id of remote search result if no match with newly 
calculated doc hash if different.
Testing showed that in some cases delivered url doesn't match the local
calculated hash. In this case replace doc.id (and host_id_s) with calculation
from url.
Changed Files: source/net/yacy/kelondro/data/meta/URIMetadataNode.java
Sat Dec 19 21:25:08 CET 2015
by reger
exclude in/outboundlinksnofollowcount_i from default schema fields
(not used in any function)
Changed Files: defaults/solr.collection.schema
Sat Dec 19 20:13:33 CET 2015
by reger
fix: adjust urlstub for mailto links
(skip protocol)
Changed Files: htroot/ViewFile.java, source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/search/query/SearchEvent.java
Sat Dec 19 01:42:35 CET 2015
by reger
drop returning document language "en" if unknown (fix todo)
which also harmonizes handling of query.modifier for rwi and solr results
(to result must match a given language filter)
Changed Files: source/net/yacy/kelondro/data/meta/URIMetadataNode.java
Fri Dec 18 01:11:55 CET 2015
by reger
show mailto links in ViewFile.html linklist
Changed Files: htroot/ViewFile.java, source/net/yacy/cora/document/id/AnchorURL.java, source/net/yacy/cora/document/id/MultiProtocolURL.java
Thu Dec 17 00:36:08 CET 2015
by reger
let html scraper accept html5 <link rel="icon"> for favicon links
Changed Files: source/net/yacy/document/parser/html/ContentScraper.java
Wed Dec 16 02:26:40 CET 2015
by luc
Corrected another case of
org.apache.lucene.store.AlreadyClosedException" occuring when
SearchEvent.cleanup() was called while committing local solr index.
Changed Files: source/net/yacy/peers/Protocol.java, source/net/yacy/peers/RemoteSearch.java
Wed Dec 16 03:01:17 CET 2015
by reger
prevent mailto links getting into parser result document's in/outbound link collection
by checking mailto scheme early.
- fix upper case mailto protocol assignment
- add test case for getProtocol
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/document/Document.java, test/net/yacy/cora/document/id/MultiProtocolURLTest.java
Wed Dec 16 02:23:58 CET 2015
by luc
Corrected frames preview displaying eventually incorrectly in local
administration mode.
Changed Files: htroot/ConfigPortal.html, htroot/ConfigPortal.java
Wed Dec 16 02:20:03 CET 2015
by luc
Added an option to eventually limit size of remote solr documents put to
local index. See mantis #626.
Changed Files: defaults/yacy.init, htroot/ConfigPortal.html, htroot/ConfigPortal.java, source/net/yacy/peers/Protocol.java, source/net/yacy/search/SwitchboardConstants.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/query/SearchEventCache.java
Tue Dec 15 02:09:43 CET 2015
by reger
fix test htmlParserTest, charset parameter
+ upd maven templating-plugin version
Changed Files: pom.xml, test/net/yacy/document/parser/htmlParserTest.java
Mon Dec 14 23:55:12 CET 2015
by luc
Added a note on deprecated default search field and operator.
Changed Files: defaults/solr/schema.xml, htroot/api/schema.xml
Mon Dec 14 23:49:28 CET 2015
by luc
Modified MemoryControl.main() test to properly end for better results
displaying.
Changed Files: source/net/yacy/kelondro/util/MemoryControl.java
Mon Dec 14 21:33:27 CET 2015
by sixcooler
CachedSolrConnector was replaced by ConcurrentUpdateSolrConnector years
ago - time to let it go
Commented out unused table of cache-objects
Changed Files: htroot/PerformanceMemory_p.html, htroot/PerformanceMemory_p.java
Sun Dec 13 20:01:49 CET 2015
by reger
fix use of NETWORK_SEARCHVERIFY for rwi verification
was not used to set the searchevent parameter (done in SearchEventCache.getEvent)
- remove unused corresponding QueryParams.filterfailurls param.
Changed Files: htroot/IndexControlRWIs_p.java, htroot/yacy/search.java, htroot/yacysearch.java, source/net/yacy/cora/federate/FederateSearchManager.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEventCache.java
Sun Dec 13 02:50:08 CET 2015
by reger
remove unused network scanner parameter from queryparameter
Search event is not using networkscanner 
(removed filterscannerfail param always init to false)
Changed Files: htroot/IndexControlRWIs_p.java, htroot/yacy/search.java, htroot/yacysearch.java, source/net/yacy/cora/federate/FederateSearchManager.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java
Wed Dec 09 00:49:38 CET 2015
by reger
add hint to "default max results per page" limit on ConfigPortal
(limit is applied in yacysearch & max. total results by sum result-stack size)
- remove obsolete search.navigation prop (has moved to ConfigSearchPage_p)
Changed Files: htroot/ConfigPortal.html, htroot/ConfigPortal.java
Tue Dec 08 12:19:05 CET 2015
by Michael Peter Christen
Merge pull request #32 from luccioman/master

Fix for MediaWiki import (mantis 625)
Changed Files: source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/search/Switchboard.java
Tue Dec 08 03:34:03 CET 2015
by luc
Fixed MediaWiki import : DCEntry conversion to SolrInputDocument was
failing. Looks like it was broken since Commit
b43811d38c426ce5237df7d37c88285644e6a544
Changed Files: source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/search/Switchboard.java
Tue Dec 08 03:30:51 CET 2015
by luc
Updated MediaWimporter main() : display usage in console and stop
properly without calling System.exit
Changed Files: source/net/yacy/document/importer/MediawikiImporter.java
Tue Dec 08 03:17:32 CET 2015
by reger
make current ranking score value avail. to search interface / api
Update the result score result field with the result queue ranking value to reflect
the actual calculated/used score,
for rwi & solr stack results.
(calc. etc. is unchanged, it's just that result entry carries the latest val
as api retrieves the number from it)
Changed Files: source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/search/query/SearchEvent.java
Mon Dec 07 23:03:59 CET 2015
by reger
upd to Bootstrap v3.3.6
Changed Files: htroot/env/bootstrap/css/bootstrap-theme.css, htroot/env/bootstrap/css/bootstrap-theme.css.map, htroot/env/bootstrap/css/bootstrap-theme.min.css, htroot/env/bootstrap/css/bootstrap.css, htroot/env/bootstrap/css/bootstrap.css.map, htroot/env/bootstrap/css/bootstrap.min.css, htroot/env/bootstrap/js/bootstrap.js, htroot/env/bootstrap/js/bootstrap.min.js
Mon Dec 07 21:58:36 CET 2015
by luc
Fixed isSolrDump function : PushBackInputStream was not unread when
returning false (for example with a WikiMedia dump).
Changed Files: source/net/yacy/document/content/SurrogateReader.java
Mon Dec 07 01:30:00 CET 2015
by reger
add missing bootstrap 3.3.4 glyphicons file
see comment @luc http://mantis.tokeek.de/view.php?id=623#c1151
Changed Files: htroot/env/bootstrap/fonts/glyphicons-halflings-regular.woff2
Thu Dec 03 00:39:15 CET 2015
by Michael Peter Christen
less logging in new language detection
Changed Files: source/net/yacy/document/language/Identificator.java
Wed Dec 02 22:57:59 CET 2015
by reger
Apply collection query constraint/modifiert to rwi result stack.
Collection is not available in pure rwi entries (but in local solr metadata)
But if user wishes to filter by query constraint also rwi shall adhere to this
(even if only rwi entries with parsed or solr received metadata may fit)
 
Changed Files: source/net/yacy/search/query/SearchEvent.java
Tue Dec 01 14:39:59 CET 2015
by Michael Peter Christen
Merge pull request #29 from luccioman/master

Modified images render error management
Changed Files: htroot/ViewImage.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java
Tue Dec 01 09:55:47 CET 2015
by luc
Corrected return type when licence is gone to be consistent with other
error cases.
Changed Files: htroot/ViewImage.java
Tue Dec 01 01:06:01 CET 2015
by luc
Corrected error management for unsupported image formats, parsing
errors, and unavailable resources : avoid logging to much Exceptions as
these errors easily occur when searching images.
Changed Files: htroot/ViewImage.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java
Mon Nov 30 18:57:16 CET 2015
by reger
upd maven pom (add langdetect)
Changed Files: pom.xml
Mon Nov 30 13:45:25 CET 2015
by Michael Peter Christen
Merge pull request #27 from Stepanov-Sergey/master

added Russian synonyms
Changed Files: addon/synonyms/thesaurus_ru_yacy
Mon Nov 30 13:35:41 CET 2015
by Michael Peter Christen
urlproxyheader must be in the default package because all classes in the
htroot path must be in the default package
Changed Files: htroot/proxymsg/urlproxyheader.java
Mon Nov 30 13:18:03 CET 2015
by Michael Peter Christen
Merge pull request #23 from linkerlin/patch-1

Create .travis.yml
Changed Files: .travis.yml
Mon Nov 30 09:37:47 CET 2015
by Sergey Stepanov
added Russian synonyms
Changed Files: addon/synonyms/thesaurus_ru_yacy
Sun Nov 29 05:19:39 CET 2015
by reger
read/init crawl queue in a thread
to speed-up YaCy start on large existing crawler queues
Changed Files: source/net/yacy/crawler/HostBalancer.java
Sun Nov 29 01:24:46 CET 2015
by reger
upd to slf4j-1.7.13
Changed Files: .classpath, build.xml, lib/jcl-over-slf4j-1.7.13.jar, lib/log4j-over-slf4j-1.7.13.jar, lib/slf4j-LICENSE.txt, lib/slf4j-api-1.7.13.jar, lib/slf4j-jdk14-1.7.13.jar, nbproject/project.xml, pom.xml
Sat Nov 28 23:09:15 CET 2015
by reger
remove unused md5 from ViewFile servlet params
Changed Files: htroot/ViewFile.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java
Sat Nov 28 02:43:38 CET 2015
by reger
fix vsdParser (Visio) parser return statement
(final block un-necessary throw)
Changed Files: source/net/yacy/document/parser/vsdParser.java
Fri Nov 27 02:41:02 CET 2015
by reger
remove md5_s from default index fields
it is not assigned a value / not used
Due to above also excluded from transfer protocol.
Changed Files: defaults/solr.collection.schema, source/net/yacy/kelondro/data/meta/URIMetadataNode.java
Thu Nov 26 09:30:43 CET 2015
by luc
- No max dimensions specified : render raw image data when source and
target image format are the same.
- Corrected scaling condition.
Changed Files: htroot/ViewImage.java, source/net/yacy/peers/graphics/EncodedImage.java
Wed Nov 25 01:34:41 CET 2015
by reger
fix flux factor (additional crawl delay by access count) calculation
Changed Files: source/net/yacy/crawler/data/Latency.java
Sun Nov 22 21:26:18 CET 2015
by reger
throw exception if crawler hostqueue can't create hostpath directory.
In rare cases hostname may not be a valid filesystem directory name,
which can't be created (e.g. containing '*' char). To prevent crawl queue
looping on this invalid entry by throwing a malformedurlexception.
Changed Files: source/net/yacy/crawler/HostQueue.java
Fri Nov 20 19:35:39 CET 2015
by luc
Use same max file size when loading all resource bytes or opening stream
content
Changed Files: source/net/yacy/crawler/retrieval/HTTPLoader.java, source/net/yacy/crawler/retrieval/Response.java
Fri Nov 20 15:02:58 CET 2015
by luc
Rendering performance improvement : use EncodedImage constructor with
BufferedImage parameter to avoid re-rerendering BufferedImage.
Changed Files: htroot/ViewImage.java
Fri Nov 20 14:35:36 CET 2015
by luc
Corrected scaling function for non RGB images.
Changed Files: htroot/ViewImage.java, source/net/yacy/peers/graphics/EncodedImage.java
Fri Nov 20 09:42:24 CET 2015
by luc
Refactoring : extracted write InputStream method.
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Fri Nov 20 09:32:30 CET 2015
by luc
Configuration projet eclipse : ajout nature et validation javascript
Changed Files: .project
Fri Nov 20 09:29:02 CET 2015
by luc
Correction erreur de compilation.
Changed Files: htroot/proxymsg/urlproxyheader.java
Fri Nov 20 01:49:56 CET 2015
by reger
start using a template for urlproxy header
It is included as iframe  /proxmsg/urlproxyheader.html
to allow full servlet functionallity and flexibility to display some
index/meta data in future.
Changed Files: htroot/proxymsg/urlproxyheader.html, htroot/proxymsg/urlproxyheader.java, source/net/yacy/http/servlets/UrlProxyServlet.java
Wed Nov 18 10:15:06 CET 2015
by luc
Process large or local file images dealing directly with content
InputStream.
Changed Files: htroot/ViewImage.java, source/net/yacy/cora/util/HTTPInputStream.java, source/net/yacy/crawler/retrieval/HTTPLoader.java, source/net/yacy/repository/LoaderDispatcher.java, test/ViewImagePerfTest.java, test/ViewImageTest.java
Wed Nov 18 10:11:38 CET 2015
by luc
If available, check content length before downloading. Check also
content length is not over Integer.MAX_VALUE.
Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java
Wed Nov 18 10:08:06 CET 2015
by luc
Ensure resource is closed when reading a full file InputStream
Changed Files: source/net/yacy/crawler/retrieval/FileLoader.java, source/net/yacy/crawler/retrieval/SMBLoader.java, source/net/yacy/kelondro/util/FileUtils.java
Tue Nov 17 23:45:29 CET 2015
by reger
have psParser cleanup temp file
Changed Files: source/net/yacy/document/parser/psParser.java
Mon Nov 16 21:37:45 CET 2015
by reger
set tmpfile.deleteOnExit by default,
to make sure files are removed on shutdown.
Changed Files: source/net/yacy/document/parser/psParser.java, source/net/yacy/kelondro/util/FileUtils.java
Mon Nov 16 01:06:20 CET 2015
by reger
Exclude repetitive protocol part in tokenized url 
used as description if none is avail. from parser.
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Sun Nov 15 06:06:37 CET 2015
by reger
harmonize wordsintitle & CollectionSchema.title_words_val calculation,
remove obsolete partial init of wordreference from urimetadata
Changed Files: source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/search/index/Segment.java
Sun Nov 15 00:39:38 CET 2015
by reger
add link to quick select blacklist 
from title list
Changed Files: htroot/Blacklist_p.html
Sun Nov 15 00:34:22 CET 2015
by reger
add German translation to re-crawl job
Changed Files: locales/de.lng
Sat Nov 14 21:16:31 CET 2015
by reger
upd to httpcore 4.4.4
Changed Files: .classpath, build.xml, lib/httpcore-4.4.4.License, lib/httpcore-4.4.4.jar, nbproject/project.xml
Fri Nov 13 20:10:47 CET 2015
by reger
fix yacysearch.json "totalResults"
element "totalResults" is included twice (at begin & end), 
only the element after performing the search holds number > 0
see http://mantis.tokeek.de/view.php?id=608
Changed Files: htroot/yacysearch.json
Fri Nov 13 01:48:28 CET 2015
by reger
Sort out double keywords (dc_subject) early in parsed documents
- by direct using Set vs. List
- remove not neede String[] getter
Changed Files: htroot/api/getpageinfo.java, htroot/api/getpageinfo_p.java, source/net/yacy/document/Document.java, source/net/yacy/search/Switchboard.java
Thu Nov 12 08:21:37 CET 2015
by luc
Added links to more image test suites.
Changed Files: test/viewImageTest/ViewImageTest.html
Wed Nov 11 00:57:51 CET 2015
by reger
improve locale translator
- skip empty line
- robustness file section detection (space independant)
Changed Files: source/net/yacy/data/Translator.java
Tue Nov 10 20:45:33 CET 2015
by sixcooler
do not store subfield *_coordinate + make all num-fields being docvalues
Changed Files: source/net/yacy/search/schema/CollectionSchema.java, source/net/yacy/search/schema/WebgraphSchema.java
Tue Nov 10 20:43:58 CET 2015
by sixcooler
not using 'location' as defaultfacetfield - since we removed it being
default.
Changed Files: source/net/yacy/search/query/QueryParams.java
Tue Nov 10 20:39:46 CET 2015
by sixcooler
simplification / speedup of GenerationMemoryStrategy
Changed Files: source/net/yacy/kelondro/util/GenerationMemoryStrategy.java, source/net/yacy/kelondro/workflow/InstantBusyThread.java
Tue Nov 10 20:32:42 CET 2015
by sixcooler
do not store subfield *_coordinate
Changed Files: defaults/solr/schema.xml
Tue Nov 10 20:32:05 CET 2015
by sixcooler
set startuptype of most solr handlers to lazy
Changed Files: defaults/solr/solrconfig.xml
Tue Nov 10 20:27:17 CET 2015
by sixcooler
fix to not let the AccessTracker-Log grow to much, but have enough data
to monitor.
(+gitignore-correction)
Changed Files: source/net/yacy/search/query/AccessTracker.java
Tue Nov 10 01:29:13 CET 2015
by reger
harmonize document title for archive parsers
Changed Files: source/net/yacy/document/parser/bzipParser.java, source/net/yacy/document/parser/gzipParser.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/zipParser.java
Mon Nov 09 08:18:32 CET 2015
by Linker Lin
Create .travis.yml
Changed Files: .travis.yml
Sat Nov 07 19:13:18 CET 2015
by reger
update bzip and bzip parser process,
to return one document for the file with combined parser results of the
containing file and registers it with supplied url and mime of the archive.
Changed Files: source/net/yacy/document/parser/bzipParser.java, source/net/yacy/document/parser/gzipParser.java
Fri Nov 06 23:58:55 CET 2015
by reger
update zip and tar parser process,
to return one document for the file with combined parser results of the
containing files.
Changed Files: source/net/yacy/document/Document.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/zipParser.java
Wed Nov 04 21:52:02 CET 2015
by reger
optimize order of parsers to try
- start with a parser matching the remote supplied mime
Changed Files: source/net/yacy/document/TextParser.java
Wed Nov 04 02:57:00 CET 2015
by reger
use current tar library for untar files
- remove old source copy
Changed Files: source/net/yacy/utils/tarTools.java
Tue Nov 03 22:14:14 CET 2015
by reger
fix tarParser early exit on looping content
- adjust check of data available according to doc 
- return null on no recognized content (to not exit TextParser next parser try)
- use commons.compress directly
Changed Files: source/net/yacy/document/parser/tarParser.java
Tue Nov 03 03:35:01 CET 2015
by reger
fix bzipParser recognition
- Bzip2Inputstream checks magic byte itself to identify bz2 (leave it in input)
- try to suppy fitting mime for parsing bz2 content
Changed Files: source/net/yacy/document/parser/bzipParser.java
Sat Oct 31 19:44:31 CET 2015
by reger
increase use of pre.defined CATCHALL_QUERY string
Changed Files: source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/query/QueryGoal.java
Sat Oct 31 19:18:46 CET 2015
by reger
Optimize internal imagequery focus on using content_type to select images
(in favor of url file extension)
Changed Files: source/net/yacy/search/query/QueryGoal.java, source/net/yacy/search/schema/CollectionConfiguration.java
Fri Oct 30 16:20:28 CET 2015
by luc
Avoid returning an empty image when target encoding is not supported or
when an error occured while encoding.
Changed Files: htroot/ViewImage.java
Fri Oct 30 16:19:05 CET 2015
by luc
Updated javadocs for warning on target encoding format potential errors.
Changed Files: source/net/yacy/peers/graphics/EncodedImage.java, source/net/yacy/visualization/RasterPlotter.java
Fri Oct 30 05:18:16 CET 2015
by luc
Corrected images alpha channel rendering
Changed Files: htroot/ViewImage.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/peers/graphics/EncodedImage.java
Thu Oct 29 23:24:39 CET 2015
by luc
Made ViewImagePerfTest extend ViewImageTest to ease automated image
render tests
Changed Files: test/ViewImagePerfTest.java, test/ViewImageTest.java
Thu Oct 29 11:41:58 CET 2015
by Michael Peter Christen
removed totalResults in json to prevent message 'Duplicate key
"totalResults"' when parsing the json in JSONObject (json reference
implementation)
Changed Files: htroot/yacysearch.json
Thu Oct 29 02:24:17 CET 2015
by luc
Corrected encoding extension arg parsing
Changed Files: test/ViewImageTest.java
Mon Oct 26 22:19:20 CET 2015
by reger
upd readme.mediawiki min java version 1.7
Changed Files: readme.mediawiki
Mon Oct 26 21:19:35 CET 2015
by reger
adjust MediaWiki importer geo coordinate calculation
- allow lat/long 0.xxx
- south / west assignment
include test class
Changed Files: source/net/yacy/data/wiki/WikiCode.java, test/net/yacy/data/wiki/WikiCodeTest.java
Sun Oct 25 05:41:25 CET 2015
by reger
fix IndexImportMediawiki_p servlet's refresh header
add url parameter to make sure no parameter are included in refresh url 
which could cause unwanted restart of import job

see http://mantis.tokeek.de/view.php?id=591 comments
Changed Files: htroot/IndexImportMediawiki_p.html
Sun Oct 25 03:06:15 CET 2015
by reger
fix MediawikiImporter for bz2 dump
skip reading bz2 file magicbyte to identify bz2 format as inputstream reset would be required. Common compress reads and checks the magicbytes internally and throws ioexception if wrong, making preread obsolete.
Changed Files: source/net/yacy/document/importer/MediawikiImporter.java
Sun Oct 25 00:26:45 CEST 2015
by reger
fix a system.out to log.fine
in bmpParser
Changed Files: source/net/yacy/document/parser/images/bmpParser.java
Sat Oct 24 22:44:28 CEST 2015
by reger
remove override of dynamicField coordinate_p in solr schema
(coordinate_p is not a mandatory field as such doesn't need to be declared as schema.field)
Changed Files: defaults/solr/schema.xml
Sat Oct 24 19:36:33 CEST 2015
by reger
fix init of peer flags
(remove hiding of ssl flag)
Changed Files: source/net/yacy/peers/Seed.java
Fri Oct 23 15:49:07 CEST 2015
by luc
Created a class to test ViewImage rendering against multiple image
files.
Changed Files: test/ViewImageTest.java
Fri Oct 23 14:12:00 CEST 2015
by luc
Corrected APNG test suite link name.
Changed Files: test/viewImageTest/ViewImageTest.html
Fri Oct 23 13:57:24 CEST 2015
by luc
Detailed javadoc.
Changed Files: test/ViewImagePerfTest.java
Fri Oct 23 12:27:52 CEST 2015
by luc
Filled ViewImageTest.html with all remaining IANA image file formats.
Added some links to test suites and specifications.
Changed Files: test/viewImageTest/ViewImageTest.html, test/viewImageTest/test/ACAD_r2000_sample.dwg, test/viewImageTest/test/ACAD_r2000_sample_original.dxf, test/viewImageTest/test/Animated_PNG_example_bouncing_beach_ball.png, test/viewImageTest/test/Specimens_of_calligraphy_and_natural_history_illustration.djvu, test/viewImageTest/test/stone_wall.vtf, test/viewImageTest/test/svflogo.svf
Thu Oct 22 02:35:58 CEST 2015
by reger
fix unnececary set null of peer flags, causing reread
remove obsolete version flags
Changed Files: source/net/yacy/peers/Seed.java, source/net/yacy/peers/operation/yacyVersion.java
Thu Oct 22 00:36:34 CEST 2015
by luc
Patch to manage render or load errors is still needed after highlight.js
version upgrade.
Updated patch for better behavior consistency between browsers.
Changed Files: htroot/js/highslide/highslide.js
Wed Oct 21 02:49:51 CEST 2015
by luc
- Keep aspect ratio of images rendered directly by browser such as gif
and svg.
- Corrected quadratic rendering of landscape images with height smaller
than maxHeight
Changed Files: htroot/ViewImage.java, htroot/env/base.css, htroot/yacysearchitem.html, htroot/yacysearchitem.java
Wed Oct 21 02:14:04 CEST 2015
by reger
upd javascript img viewerto highslide 4.1.13
Changed Files: htroot/js/highslide/highslide.js
Tue Oct 20 01:17:37 CEST 2015
by luc
Display full size preview using ViewImage Servlet.
Changed Files: htroot/yacysearchitem.html, htroot/yacysearchitem.java
Tue Oct 20 01:15:02 CEST 2015
by luc
Added image preview error management.
Changed Files: htroot/js/highslide/highslide.js
Mon Oct 19 14:11:26 CEST 2015
by luc
Corrected NullPointerException case when ImageIO reader is not found for
image format.
Changed Files: source/net/yacy/document/ImageParser.java
Mon Oct 19 03:47:28 CEST 2015
by reger
remove obsolete yacy.init entry "secureHttps"
not used anywhere
Changed Files: defaults/yacy.init
Mon Oct 19 01:06:51 CEST 2015
by reger
upd to icu4j-56_1
Changed Files: .classpath, build.xml, lib/icu4j-56_1.jar, nbproject/project.xml, pom.xml
Sun Oct 18 06:19:12 CEST 2015
by reger
add a log entry on parsing ajax crawling scheme snapshot 
(prev. commit https://github.com/yacy/yacy_search_server/commit/9252e36aeb3765ba06d4dcf5543ad2e64c70bd4e) 
Changed Files: source/net/yacy/document/parser/htmlParser.java
Fri Oct 16 23:30:51 CEST 2015
by Michael Peter Christen
replaced HashMap with LinkedHashMap to preserve the object order
Changed Files: source/net/yacy/cora/util/JSONObject.java
Fri Oct 16 23:30:04 CEST 2015
by Michael Peter Christen
added log lines
Changed Files: source/net/yacy/document/ImageParser.java
Fri Oct 16 03:05:39 CEST 2015
by reger
init Recrawl job chunk size to max crawl loader during job start, to use some system preferences
and allow injection of recrawl urls before queue is empty
During recrawl the balancer hangs on the very last urls often on hosts with huge delay time,
by allowing injection earlier progress is more balanced. Max number of injected crawl urls by recrawl job is 2 * max loader.
Changed Files: source/net/yacy/crawler/RecrawlBusyThread.java
Thu Oct 15 09:18:24 CEST 2015
by luc
Created a generic ViewImage performance render test.
Changed Files: test/ViewImagePerfTest.java
Wed Oct 14 10:17:09 CEST 2015
by luc
Created a ViewImage rendering performance measurement test.
Changed Files: test/ViewImageJPGPerfTest.java
Wed Oct 14 10:15:00 CEST 2015
by luc
Refactoring : split into sub-functions to make it understanding and
performance measurement easier.
Changed Files: htroot/ViewImage.java
Wed Oct 14 10:13:37 CEST 2015
by luc
Updated table headers and SVG file url for case sensitive OS.
Changed Files: test/viewImageTest/ViewImageTest.html
Tue Oct 13 02:43:18 CEST 2015
by reger
unescape MultiProtocolURL getAttributes() return values.
use getAttributes() to get query parameters as clear text (w/o url encoding)
use getSearchpartMap() to get in internal format (url encoded)

fix for http://mantis.tokeek.de/view.php?id=606
Changed Files: htroot/Table_API_p.html, source/net/yacy/cora/document/id/MultiProtocolURL.java
Sun Oct 11 01:23:52 CEST 2015
by reger
refactor special handling (static override) of SUPPORTED_EXTENSIONS/MIME_TYPES
not used for genericImageParser

Changed Files: source/net/yacy/document/parser/images/genericImageParser.java
Sat Oct 10 23:49:58 CEST 2015
by reger
add links with image extension not automatically to image links.
With the wide spread use e.g. of Wikimedia the url file extension of links with image extension often point to html.
Changed Files: source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/TransformerWriter.java
Tue Oct 06 20:48:09 CEST 2015
by luc
Added ico and bmp sample pictures
Changed Files: test/viewImageTest/ViewImageTest.html, test/viewImageTest/test/favicon.ico, test/viewImageTest/test/sails.bmp
Tue Oct 06 09:53:09 CEST 2015
by luc
Added JPEG 2000 and FITS samples
Changed Files: test/viewImageTest/test/WFPC2u5780205r_c0fx.fits, test/viewImageTest/test/relax.jp2
Tue Oct 06 09:51:47 CEST 2015
by luc
Added image formats and informations for each format.
Changed Files: test/viewImageTest/ViewImageTest.html
Tue Oct 06 04:13:04 CEST 2015
by reger
handle image preview for url w empty file extension
fix of commit 688f7b2a5c194124dc2a253be45a1af1b814bef2
Changed Files: htroot/yacysearchitem.java
Mon Oct 05 01:58:31 CEST 2015
by reger
check jpeg file signature in genericImageParser
to fail early without further object allocation if source is not a jpeg.
Changed Files: source/net/yacy/document/parser/images/genericImageParser.java
Sun Oct 04 05:43:40 CEST 2015
by reger
use recrawljob w/o sort results by date
This is a workaround for existing index (not fully reindexed) since intro of schema with docvalues
to prevent solr exception causing recrawljob to fail with
org.apache.solr.core.SolrCore java.lang.IllegalStateException: unexpected docvalues type NONE for field 'load_date_dt' (expected=NUMERIC). Use UninvertingReader or index with docvalues.
Changed Files: source/net/yacy/crawler/RecrawlBusyThread.java
Sun Oct 04 05:43:16 CEST 2015
by Michael Peter Christen
removed unused import
Changed Files: htroot/yacysearchtrailer.java
Sat Oct 03 21:43:41 CEST 2015
by reger
upd to poi-3.13
Changed Files: .classpath, build.xml, lib/poi-3.13-20150929.jar, lib/poi-3.13.License, lib/poi-scratchpad-3.13-20150929.jar, nbproject/project.xml, pom.xml
Fri Oct 02 12:41:30 CEST 2015
by luc
Created a html test page to check ViewImage rendering with different
file formats.
Changed Files: test/viewImageTest/ViewImageTest.html, test/viewImageTest/test/JPEG_example_JPG_RIP_100.jpg, test/viewImageTest/test/PNG_transparency_demonstration_1.png, test/viewImageTest/test/Rotating_earth_(large).gif, test/viewImageTest/test/SVG_Logo.svg, test/viewImageTest/test/Sunflower_as_gif_websafe.gif, test/viewImageTest/test/marbles.tif, test/viewImageTest/test/sample.cgm
Fri Oct 02 01:48:48 CEST 2015
by reger
allow/display svg images in image results previews
svg is not supported by awt but by most browser. Image content is delivered as received (without size adjustment)
Changed Files: htroot/ViewImage.java, htroot/yacysearchitem.java, source/net/yacy/cora/document/analysis/Classification.java
Thu Oct 01 23:11:58 CEST 2015
by reger
remove some unused var allocation in parser
Changed Files: source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/parser/images/metadataImageParser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/document/parser/swfParser.java
Thu Oct 01 13:21:28 CEST 2015
by Michael Peter Christen
follow-up to latest commit: also flush the search cache if all crawls
had been terminated.
Changed Files: source/net/yacy/search/Switchboard.java
Thu Oct 01 13:18:44 CEST 2015
by Michael Peter Christen
every time a crawl is started, the user expects a different search
result behaviour. This requires that the search cache is flushed for
each crawl start. TODO: this should also be done if a crawl is
terminated.
Changed Files: htroot/Crawler_p.java
Thu Oct 01 13:09:33 CEST 2015
by Michael Peter Christen
in case that the include_string contains several entries including
1-char tokens and also more-than-1-char tokens, then remove the 1-char
tokens to prevent that we are to strict. This will make it possible to
be a bit more fuzzy in the search where it is appropriate.
Changed Files: source/net/yacy/search/query/QueryGoal.java
Thu Oct 01 13:03:22 CEST 2015
by Michael Peter Christen
add also 1-character tokens to the token list because that could be also
searched for. A full-string search for a filename may fail if those
1-char tokens are omitted
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Tue Sep 29 22:57:33 CEST 2015
by reger
add a end condition to svgParser for wrong content
(if parser choosen just by file extension)
Changed Files: source/net/yacy/document/parser/images/svgParser.java
Sun Sep 27 03:24:28 CEST 2015
by reger
remove double caching of inputstream in ViewImage
Changed Files: htroot/ViewImage.java
Sun Sep 27 00:17:42 CEST 2015
by reger
fix old/obsolete solr dependency to stax
delete obsolete jar
Changed Files: .classpath, build.xml, nbproject/project.xml, pom.xml
Sat Sep 26 19:58:15 CEST 2015
by reger
Add report profile with OWASP Dependency-Check to maven pom
Changed Files: pom.xml
Sat Sep 26 17:30:34 CEST 2015
by reger
remove rdfParser from init (current function identical with genericParser)
Changed Files: source/net/yacy/document/TextParser.java
Sat Sep 26 17:27:33 CEST 2015
by reger
add svgParser to parse metadate from svg images
Reads document level included title and description and skips the graphic content to save bandwidth.
svg metadata element is not interpreted
- remove rdfParser from init (current function identical with genericParser)
Changed Files: source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/images/svgParser.java
Sat Sep 26 15:42:23 CEST 2015
by reger
optimize parseInt for <img> tag attribute parsing
Performance better as using Numberformat.parse or parseInt(substring())
Changed Files: source/net/yacy/cora/util/NumberTools.java, source/net/yacy/document/parser/html/ContentScraper.java, test/net/yacy/cora/util/NumberToolsTest.java
Thu Sep 24 01:58:19 CEST 2015
by reger
check for loading error (includs unsupported formats)
to prevent blank thumbnail display in image search because of not handled source which don't load on click.
Now the cross icon indicates the problem (inlcuding not supported format)
Changed Files: source/net/yacy/document/ImageParser.java
Wed Sep 23 21:01:51 CEST 2015
by luc
Correction for mantis 535: inurl: parameter doesn't work on URLs with
upper-case letters
Changed Files: source/net/yacy/kelondro/data/meta/URIMetadataNode.java
Wed Sep 23 00:13:10 CEST 2015
by reger
harmonize/correct assignment to Ymarkmeta.mime
replace use of deprecated
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/cora/federate/solr/responsewriter/YJsonResponseWriter.java, source/net/yacy/data/ymark/YMarkMetadata.java
Tue Sep 22 11:56:17 CEST 2015
by Michael Peter Christen
Fix for index entries which have id's not computed as hash from the url.
This makes it possible to operate with outside-computed url hashes in
enterprise environments not using the build-in crawler from YaCy.
Changed Files: source/net/yacy/cora/document/id/DigestURL.java, source/net/yacy/peers/Protocol.java
Tue Sep 22 03:52:15 CEST 2015
by reger
remove unused check for known fileextension in searchtrailer
(check is done on add to filetype-nav)
Changed Files: htroot/yacysearchtrailer.java
Tue Sep 22 00:12:31 CEST 2015
by reger
optionally include mime in p2p url exchange string
if doctype decodes to ambiguous mime and default conversion is not equal to original
 
Changed Files: source/net/yacy/kelondro/data/meta/URIMetadataNode.java
Sun Sep 20 23:28:42 CEST 2015
by reger
add Portuguese month names to date recognition
Changed Files: source/net/yacy/document/DateDetection.java
Sat Sep 19 05:30:55 CEST 2015
by reger
fix html parser taking <style> content as text.
Noticed some result description contain css content from style tag.
Added <style> to tag list to scrape it's content not as text
+ test case included
Changed Files: source/net/yacy/document/parser/html/ContentScraper.java, test/net/yacy/document/parser/htmlParserTest.java
Fri Sep 18 02:25:44 CEST 2015
by Michael Peter Christen
patch for a bug inside of solr since solr 5.0 when using a boost
function with a numeric date field:
"unexpected docvalues type NUMERIC for field 'last_modified' (expected
one of [SORTED, SORTED_SET]). Use UninvertingReader or index with
docvalues."
This is a well-known bug inside solr which prevents that now the 'sort
by date' in the YaCy search interface can be used. Without this patch no
results at all is displayed (since the exception prevents that). Now
there is at least a result but it is not ordered properly.
Changed Files: source/net/yacy/cora/federate/solr/Ranking.java
Tue Sep 15 02:11:30 CEST 2015
by reger
limit css scrolling to result/content window x
from pull request #10
Changed Files: htroot/env/bootstrap-base.css
Tue Sep 15 02:09:17 CEST 2015
by Burkhard
Merge pull request #10 from Raegdan/raegdan-css-layout-fix

Fixed CSS scrolling
Changed Files: htroot/env/bootstrap-base.css
Sun Sep 13 20:23:15 CEST 2015
by reger
Hack to prevent Solr issue on partial update on a document containing multivalued date field
(regardless if these fields part of update).
Switch partial update option off in postprocessing if schema contains *_dts (multivalued date field).
see http://mantis.tokeek.de/view.php?id=601
Changed Files: source/net/yacy/search/Switchboard.java
Sun Sep 13 20:19:50 CEST 2015
by reger
adapt SolrServerConnector.add to handle error on partial update input document.
In case of error we deleted the original document and added the new doc to the index.
This is not valid for partial update documents (which contain only a subset of the fields).
Remove the "delete" error handling step.
Changed Files: source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java
Sun Sep 13 06:02:07 CEST 2015
by reger
add test case for partial update - to discover effect on YaCy for update of documents with multivalued date fields (like dates_in_content_dts) 
current result: loss of fields/information in index document, see EmbeddedSolrConnectorTest.testUdate_withMultivaluedDateField()
Changed Files: test/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnectorTest.java
Sat Sep 12 23:06:13 CEST 2015
by reger
on reindex delete index document with invalid url
if discovered
Changed Files: source/net/yacy/crawler/RecrawlBusyThread.java
Sat Sep 12 22:00:40 CEST 2015
by reger
use a parsed date in Document.toString
Changed Files: source/net/yacy/document/Document.java
Fri Sep 11 17:23:59 CEST 2015
by luccioman
Returned again to main repository location : does anyone want to
consider mantis 597 ?  (http://mantis.tokeek.de/view.php?id=597)
Changed Files: htroot/env/style.java
Mon Sep 07 02:36:22 CEST 2015
by luccioman
Merge from main repository
Changed Files: htroot/style.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java
Mon Sep 07 02:36:22 CEST 2015
by reger
improve filtering by filetype navigator.
The used url-filter for filetype doesn't require ".ext" resulting in too many matches,
add a sort-out filter for RWI results.
Changed Files: source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java
Sun Sep 06 22:19:05 CEST 2015
by reger
prevent metadata records in index w/o valid url
by throwing MalformedURL exception on URIMetadataNode creation
Changed Files: source/net/yacy/cora/federate/AbstractFederateSearchConnector.java, source/net/yacy/cora/federate/SolrFederateSearchConnector.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/peers/Protocol.java, source/net/yacy/search/Switchboard.java
Sun Sep 06 04:28:27 CEST 2015
by reger
extract modification date from vCard (vcfParser)
Changed Files: source/net/yacy/document/parser/vcfParser.java
Sun Sep 06 00:04:54 CEST 2015
by reger
extract lastmodified from openoffice doc
set lastmod date in office document parsers
Changed Files: source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/xml/ODMetaHandler.java
Sat Sep 05 14:07:23 CEST 2015
by Michael Peter Christen
in case that a site crawl is started for urls with file:// path, the
host filter does not work because there is no host given in such urls.
In that case, patch the filter to be a sub-path filter.
Changed Files: htroot/Crawler_p.java
Sat Sep 05 01:57:30 CEST 2015
by reger
fix exception throw after sendError in DefaultServlet
- reduce debug exception logs in crawler
Changed Files: source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java
Fri Sep 04 17:05:06 CEST 2015
by Michael Peter Christen
Merge pull request #12 from luccioman/master

Updated french locale and added new translator utils
Changed Files: htroot/Steering.html, htroot/yacysearchitem.html, locales/fr.lng, source/net/yacy/data/Translator.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/utils/translation/ListNonTranslatedFiles.java, source/net/yacy/utils/translation/SourceFileFilter.java, source/net/yacy/utils/translation/TranslateAll.java, source/net/yacy/utils/translation/TranslatorUtil.java
Fri Sep 04 13:52:03 CEST 2015
by luccioman
Return to mai repository version
Changed Files: htroot/env/style.java
Fri Sep 04 13:44:44 CEST 2015
by luccioman
Added utils to help translation without launching full YaCy application
:
- translate all source files with a locale
- list all non translated files with a locale
Changed Files: source/net/yacy/utils/translation/ListNonTranslatedFiles.java, source/net/yacy/utils/translation/TranslateAll.java, source/net/yacy/utils/translation/TranslatorUtil.java
Fri Sep 04 13:42:57 CEST 2015
by luccioman
Added a function to list files recursively.
Changed Files: source/net/yacy/kelondro/util/FileUtils.java
Fri Sep 04 13:42:10 CEST 2015
by luccioman
Translator refactoring : 
- deleted useless new StringBuilder allocation
- use of a new reusable FileNameFilter
- added javadoc
Changed Files: source/net/yacy/data/Translator.java, source/net/yacy/utils/translation/SourceFileFilter.java
Thu Sep 03 23:36:57 CEST 2015
by reger
fix missing license in image search
see http://mantis.tokeek.de/view.php?id=522
Changed Files: htroot/yacysearchitem.java
Thu Sep 03 09:02:03 CEST 2015
by luccioman
Updated french translations for yacysearhitem.html,
yacysearchtrailer.html and Steering.html files.
Corrected various labels.
Changed Files: locales/fr.lng
Thu Sep 03 08:59:17 CEST 2015
by luccioman
Corrected br markup
Changed Files: htroot/Steering.html
Thu Sep 03 08:58:14 CEST 2015
by luccioman
Corrected bookmark link title
Changed Files: htroot/yacysearchitem.html
Thu Sep 03 00:59:14 CEST 2015
by reger
fix missing onclick in ConfigPortal
to enable checkbox
Changed Files: htroot/ConfigPortal.html
Wed Sep 02 19:10:39 CEST 2015
by sixcooler
ignore /DATA (Eclipse Mars)
Changed Files: .gitignore
Tue Sep 01 23:22:48 CEST 2015
by reger
apply same size constrain on result image from doc
as for linked images
see https://github.com/yacy/yacy_search_server/commit/19f1308bf09172d2be66c58289d52ba2b2c0cf9d
Changed Files: source/net/yacy/search/query/SearchEvent.java
Tue Sep 01 21:47:25 CEST 2015
by reger
enable Solr schema dynamicField _p (type=location) for YaCy coordinate_p field
Changed Files: defaults/solr/schema.xml
Mon Aug 31 23:28:03 CEST 2015
by reger
complete TODO: getFileExtension handle dot in query part
+ testcase
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java, test/net/yacy/cora/document/id/MultiProtocolURLTest.java
Mon Aug 31 19:57:57 CEST 2015
by sixcooler
French Translation update by Luc:
http://forum.yacy-websuche.de/viewtopic.php?f=8&t=5671
Changed Files: locales/fr.lng
Mon Aug 31 01:58:36 CEST 2015
by reger
start recording/indexing pixel size for image document
as for linked images
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Sun Aug 30 23:02:19 CEST 2015
by reger
check mime prior to ext for metadata modification for images
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/search/schema/CollectionConfiguration.java
Sun Aug 30 02:19:52 CEST 2015
by reger
enforce th result images limit to > 16x16px
for linked images
http://mantis.tokeek.de/view.php?id=594
Changed Files: source/net/yacy/search/query/SearchEvent.java
Thu Aug 27 09:25:11 CEST 2015
by luccioman
Updated french translation for index.html, yacysearch.html and
simpleheader.template. Correcte special characters to use HTML entities
instead.
Changed Files: defaults/yacy.network.freeworld.unit, locales/fr.lng
Wed Aug 26 23:58:08 CEST 2015
by reger
remove exired domain titan.deep-one.in from bootstrap.seedlist
Changed Files: defaults/yacy.network.freeworld.unit
Wed Aug 26 13:57:00 CEST 2015
by luccioman
Updated translation of index.html, yacysearch.html and
simpleheader.template, corrected some special characters not written as
HTML entities.
Changed Files: htroot/style.java, locales/fr.lng
Tue Aug 25 23:26:17 CEST 2015
by reger
fix NPE on .yacyh result url of disconnected peer
(cleanup yacyshare remaining)
Changed Files: source/net/yacy/kelondro/data/meta/URIMetadataNode.java
Tue Aug 25 02:19:00 CEST 2015
by reger
log missing seed.port
in favour of exception to prevent repeating throws
Changed Files: source/net/yacy/peers/Seed.java
Tue Aug 25 01:16:41 CEST 2015
by reger
fix: Preserve protocol in url proxy 
to connect to http/https. Display warning if https target is viewed over http
Changed Files: source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/http/servlets/YaCyProxyServlet.java, source/net/yacy/server/http/HTTPDProxyHandler.java
Wed Aug 19 22:46:48 CEST 2015
by reger
upd to jsoup-1.8.3
Changed Files: .classpath, build.xml, lib/jsoup-1.8.3.jar, nbproject/project.xml, pom.xml
Mon Aug 10 20:53:20 CEST 2015
by sixcooler
added / corrected charste to be 1.7 compatible.
@Orbiter: please check is this is ok for you
Changed Files: source/net/yacy/document/ProbabilisticClassifier.java
Sun Aug 09 21:01:30 CEST 2015
by reger
exclude more default search fields from  text copy to text_t
for metadata index documents
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Sat Aug 08 18:35:49 CEST 2015
by reger
remove obsolete interface SearchAccumulator
and unused SRURSSConnector Thread inheritance
Changed Files: source/net/yacy/cora/federate/opensearch/SRURSSConnector.java
Mon Aug 03 05:17:22 CEST 2015
by Michael Peter Christen
enhanced logging
Changed Files: source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/util/ConcurrentLog.java
Mon Aug 03 00:49:24 CEST 2015
by reger
! finish running crawls before applying !
Allow crawl urls up to 2048 character 
fix for http://mantis.tokeek.de/view.php?id=575
Changed Files: source/net/yacy/crawler/retrieval/Request.java
Sun Aug 02 22:56:14 CEST 2015
by reger
use some more declared HeaderFramework constants
Changed Files: source/net/yacy/cora/federate/solr/instance/RemoteInstance.java, source/net/yacy/cora/protocol/http/GzipCompressingEntity.java, source/net/yacy/cora/protocol/http/GzipRequestInterceptor.java, source/net/yacy/cora/protocol/http/GzipResponseInterceptor.java, source/net/yacy/server/http/HTTPDProxyHandler.java
Sun Aug 02 21:36:44 CEST 2015
by reger
add missing ; in base.css
Changed Files: htroot/yacy/ui/css/base.css
Sun Aug 02 03:39:58 CEST 2015
by reger
pom: have Maven dependency management decide on transitive Lucene dependencies
Changed Files: pom.xml
Sun Aug 02 00:53:49 CEST 2015
by reger
replace deprecated myPublicLocalIP() in AbstractRemoteHandler
Changed Files: source/net/yacy/http/AbstractRemoteHandler.java
Sun Aug 02 00:20:14 CEST 2015
by reger
remove unused Transmission hit counter
Changed Files: source/net/yacy/peers/Transmission.java
Sat Aug 01 23:54:26 CEST 2015
by reger
use more absolute path for config file opening
as suggested in pull request 5 (https://github.com/yacy/yacy_search_server/pull/5)
Changed Files: source/net/yacy/search/Switchboard.java
Thu Jul 30 14:10:31 CEST 2015
by Michael Peter Christen
added bayes filter from Philipp Nolte, originally taken from
https://github.com/ptnplanet/Java-Naive-Bayes-Classifier
and modified inside the loklak.org project. After optimization in loklak
it was inserted into the net.yacy.cora.bayes package. It shall be used
to create custom search navigation filters.

The original copyright notice was copied from the README.md from
https://github.com/ptnplanet/Java-Naive-Bayes-Classifier/blob/master/README.md
The original package domain was
de.daslaboratorium.machinelearning.classifier
Changed Files: source/net/yacy/cora/bayes/BayesClassifier.java, source/net/yacy/cora/bayes/Classification.java, source/net/yacy/cora/bayes/Classifier.java
Thu Jul 30 13:39:10 CEST 2015
by Michael Peter Christen
using latest enhanced (un/)gzip methods from loklak for yacy
Changed Files: source/net/yacy/utils/gzip.java
Thu Jul 30 03:21:40 CEST 2015
by Michael Peter Christen
added export option to export the fulltext of the search index text only
Changed Files: htroot/IndexExport_p.html, htroot/IndexExport_p.java, source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/search/index/Fulltext.java
Mon Jul 27 15:16:08 CEST 2015
by Michael Peter Christen
try a healing of the cache if the index file is corrupted
Changed Files: source/net/yacy/crawler/data/Cache.java
Mon Jul 27 15:03:13 CEST 2015
by Michael Peter Christen
added log lines for query performance profiling
Changed Files: source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java
Mon Jul 27 00:57:19 CEST 2015
by reger
upd to SLF4J-1.7.12
Changed Files: .classpath, build.xml, lib/jcl-over-slf4j-1.7.12.jar, lib/log4j-over-slf4j-1.7.12.jar, lib/slf4j-api-1.7.12.jar, lib/slf4j-jdk14-1.7.12.jar, nbproject/project.xml
Sun Jul 26 00:53:40 CEST 2015
by reger
upd to httpclient-4.5 and httpmime-4.5
Changed Files: .classpath, build.xml, lib/httpclient-4.5.License, lib/httpclient-4.5.jar, lib/httpmime-4.5.License, lib/httpmime-4.5.jar, nbproject/project.xml, pom.xml
Sat Jul 25 00:50:41 CEST 2015
by reger
upd to icu4j-55.1.jar
Changed Files: .classpath, build.xml, lib/icu4j-55_1.jar, nbproject/project.xml, pom.xml
Tue Jul 21 22:31:34 CEST 2015
by reger
upd to jsch-0.1.53.jar
Changed Files: .classpath, build.xml, lib/jsch-0.1.53.License, lib/jsch-0.1.53.jar, nbproject/project.xml, pom.xml
Tue Jul 21 07:21:10 CEST 2015
by Kirill Fomchenko
Fixed CSS scrolling

When the sidebar on search page becomes scrollable, the scrollbar shrinks the sidebar and makes the search results weirdly scrollable on X axis by several pixels. Now the sidebar always have a scrollbar, and results are never X-scrollable.
Changed Files: htroot/env/bootstrap-base.css
Mon Jul 20 03:45:23 CEST 2015
by reger
upd to lib/weupnp-0.1.3.jar
Changed Files: .classpath, build.xml, lib/weupnp-0.1.3.jar, nbproject/project.xml, pom.xml
Thu Jul 16 23:42:41 CEST 2015
by Michael Peter Christen
added jsonp to suggest servlet
Changed Files: htroot/suggest.java, htroot/suggest.json
Wed Jul 15 01:04:59 CEST 2015
by reger
upd NB classpath
Changed Files: nbproject/project.xml
Fri Jul 10 16:47:19 CEST 2015
by Michael Peter Christen
remove old vocabularies and synonyms before adding new
Changed Files: .classpath, source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/cora/federate/solr/SchemaDeclaration.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/CollectionSchema.java, source/net/yacy/search/schema/WebgraphSchema.java
Thu Jul 09 16:25:11 CEST 2015
by Michael Peter Christen
added another default network / commented out
Changed Files: defaults/yacy.init
Wed Jul 08 17:36:37 CEST 2015
by Michael Peter Christen
added msg (text emails) format; should be handled by html parser.
Changed Files: source/net/yacy/document/parser/htmlParser.java
Wed Jul 08 03:02:10 CEST 2015
by reger
fix one implicit Integer/Long type conversion
-> causes Java 1.8 compile error
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Sat Jul 04 22:49:01 CEST 2015
by reger
add CommonPattern for multiple spaces 
to eliminate empty split words on following spaces
Changed Files: htroot/ViewFile.java, htroot/Vocabulary_p.java, htroot/yacysearch_location.java, pom.xml, source/net/yacy/cora/document/feed/RSSMessage.java, source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/cora/util/CommonPattern.java, source/net/yacy/data/DidYouMean.java, source/net/yacy/search/schema/CollectionConfiguration.java
Thu Jul 02 00:23:50 CEST 2015
by Michael Peter Christen
added enrichment of synonyms and vocabularies for imported documents
during surrogate reading: those attributes from the dump are removed
during the import process and replaced by new detected attributes
according to the setting of the YaCy peer.
This may cause that all such attributes are removed if the importing
peer has no synonyms and/or no vocabularies defined.
Changed Files: source/net/yacy/document/Document.java, source/net/yacy/document/Tokenizer.java, source/net/yacy/document/VocabularyScraper.java, source/net/yacy/document/WordTokenizer.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/schema/CollectionConfiguration.java
Wed Jul 01 18:28:18 CEST 2015
by Michael Peter Christen
refactoring: separated condenser and tokenizer
Changed Files: source/net/yacy/document/Condenser.java, source/net/yacy/document/Tokenizer.java
Wed Jul 01 00:58:23 CEST 2015
by reger
Rem depreciated AdminHandlers in solrconfig.xml
avoid warning log
W  org.apache.solr.handler.admin.AdminHandlers <requestHandler name="/admin/"  class="solr.admin.AdminHandlers" /> is deprecated . It is not required anymore
Changed Files: defaults/solr/solrconfig.xml
Tue Jun 30 11:12:36 CEST 2015
by Michael Peter Christen
fix for non-authorized view of IndexBrowser: show only the number of
non-failure documents
Changed Files: htroot/HostBrowser.java
Mon Jun 29 12:28:34 CEST 2015
by Michael Peter Christen
enhanced surrogate import process speed (dramatically!)
Changed Files: source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/search/Switchboard.java
Mon Jun 29 02:02:01 CEST 2015
by Michael Peter Christen
fix for
- bad regex computation for crawl start from file (limitation on domain
did not work)
- servlet error when starting crawl from a large list of urls
Changed Files: htroot/Crawler_p.java, source/net/yacy/crawler/data/CrawlProfile.java
Wed Jun 24 13:02:12 CEST 2015
by Michael Peter Christen
suppress access to solr when doing search suggestions in case that the
index has more than two million documents. This protects the index from
beeing flooded with search requests that cannot be resolved before the
real search query has to be computet.
Changed Files: htroot/suggest.java, htroot/yacysearch.java, source/net/yacy/data/DidYouMean.java
Tue Jun 23 23:41:43 CEST 2015
by Michael Peter Christen
remove redundant code
Changed Files: htroot/Crawler_p.java
Sun Jun 14 22:56:26 CEST 2015
by sixcooler
Next Try for a fix for upload-connection staying in blocked state.
This was caused by reading via GZIP from close-wait connection an caused
high cpu- and system-loads.
Instat of implementing handling of the RedListener now I found a
timelimeted 'get' "realy" solving this problem.
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Wed Jun 10 02:35:37 CEST 2015
by reger
Resourceobserver log warning - deleting releases files - only on actual deletes
instead of entering routine
Changed Files: source/net/yacy/peers/operation/yacyRelease.java, source/net/yacy/search/ResourceObserver.java
Tue Jun 09 21:26:10 CEST 2015
by sixcooler
Fix for upload-connection staying in blocked state.
This was caused by reading via GZIP from close-wait connection an caused
high cpu- and system-loads.
Solved by implementing handling of the RedListener.
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Mon Jun 08 03:17:12 CEST 2015
by reger
add log entry on release file delete by ResourceObserver
Changed Files: source/net/yacy/search/ResourceObserver.java
Mon Jun 08 02:52:13 CEST 2015
by reger
implement deleteOldDownloads in RexourceObserver on low diskspace
- direct assign sb.observer (skip redundant InitThread)
Changed Files: source/net/yacy/search/ResourceObserver.java, source/net/yacy/search/Switchboard.java
Sun Jun 07 20:37:37 CEST 2015
by Michael Peter Christen
added link to Snapshots in search results if the snapshot exists and
option is set in ConfigSearchPage_p
(this is a stub: we also need a visualization of pdf files!)
Changed Files: defaults/yacy.init, htroot/ConfigSearchPage_p.html, htroot/ConfigSearchPage_p.java, htroot/yacysearchitem.html, htroot/yacysearchitem.java, source/net/yacy/crawler/data/Transactions.java
Sat Jun 06 18:45:39 CEST 2015
by reger
enhance recrawl job
- allow to modify the query to select documents to  process (after job has started)
- allow to include failed urls (httpstatus <> 200)
Changed Files: htroot/IndexReIndexMonitor_p.html, htroot/IndexReIndexMonitor_p.java, source/net/yacy/crawler/RecrawlBusyThread.java
Fri Jun 05 07:22:35 CEST 2015
by Michael Peter Christen
servlet for latest commit
Changed Files: htroot/IndexExport_p.html, htroot/IndexExport_p.java
Fri Jun 05 03:36:57 CEST 2015
by reger
upd to poi-3.12.jar
Changed Files: .classpath, build.xml, lib/poi-3.12-20150511.jar, lib/poi-3.12.License, lib/poi-scratchpad-3.12-20150511.jar, nbproject/project.xml, pom.xml
Fri Jun 05 00:51:00 CEST 2015
by reger
remove augmented parsing activation from frontend
experimental implementation not used and based on error prone experimental rdfaparser 
Changed Files: htroot/env/templates/submenuSemantic.template, source/net/yacy/document/TextParser.java
Fri Jun 05 00:15:16 CEST 2015
by reger
remove RDFa parser activation from frontend
reason: experimental implementatin of RDFa parser not executed (limited to special urls) but may cause error on normal html parsing due to a inputstream.reset
Changed Files: htroot/AugmentedParsing_p.html, htroot/AugmentedParsing_p.java, source/net/yacy/document/TextParser.java
Thu Jun 04 23:03:46 CEST 2015
by Michael Peter Christen
removed the new index export method from the IndexControlURLs_p.html
servlet and moved it to a new /IndexExport_p.html servlet. This servlet
is now more prominent linked in the main menu under Production -> Index
Export/Import
Changed Files: htroot/IndexControlURLs_p.html, htroot/IndexControlURLs_p.java, htroot/env/templates/header.template, htroot/env/templates/submenuIndexImport.template, locales/de.lng, locales/fr.lng, locales/ru.lng
Thu Jun 04 22:44:46 CEST 2015
by reger
Merge origin/master
Changed Files: skins/27c3.css, skins/28c3.css, skins/dark-blue.css, skins/dark.css, skins/phosphor.css
Thu Jun 04 22:44:01 CEST 2015
by reger
remove obsolete searchfl work table
was used to register urls with not complete words in snippet but is never accessed
Changed Files: htroot/ConfigHTCache_p.html, htroot/ConfigHTCache_p.java, htroot/IndexControlURLs_p.html, htroot/IndexControlURLs_p.java, source/net/yacy/data/WorkTables.java, source/net/yacy/search/ResourceObserver.java, source/net/yacy/search/Switchboard.java
Thu Jun 04 22:15:38 CEST 2015
by sixcooler
correct the dark themes to show also a dark navbar on searchresults
Changed Files: skins/27c3.css, skins/28c3.css, skins/dark-blue.css, skins/dark.css, skins/phosphor.css
Mon Jun 01 01:24:33 CEST 2015
by Michael Peter Christen
gzip compression will perform more efficient and with better compression
level
Changed Files: source/net/yacy/cora/protocol/http/GzipCompressingEntity.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/index/RowHandleMap.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/utils/gzip.java
Sat May 30 19:02:54 CEST 2015
by Michael Peter Christen
full solr xml exports will now be automatically compressed during
export. That makes it possible to export a solr xml dump even if disc
space is low.
Changed Files: source/net/yacy/search/index/Fulltext.java
Sat May 30 17:54:02 CEST 2015
by Michael Peter Christen
wrap HeaReader close() in a catch Throwable block to prevent that an
excpetion during close blocks the whole shotdown process
Changed Files: source/net/yacy/kelondro/blob/HeapReader.java
Sat May 30 13:19:59 CEST 2015
by Michael Peter Christen
added surrogate import process for exported solr dumps.
Just throw your solr dump file into DATA/SURROGATES/in/ and it will be
imported!
Changed Files: htroot/api/ymarks/import_ymark.java, source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Fulltext.java
Sat May 30 06:57:15 CEST 2015
by Michael Peter Christen
prevent disc usage when showing tray animation
Changed Files: source/net/yacy/gui/Tray.java
Sat May 30 06:12:08 CEST 2015
by Michael Peter Christen
re-licensing some of my old visualization classes under LGPL 2.1
Changed Files: source/net/yacy/visualization/ChartPlotter.java, source/net/yacy/visualization/CircleTool.java, source/net/yacy/visualization/FontGenerator3Pixel.java, source/net/yacy/visualization/FontGenerator5Pixel.java, source/net/yacy/visualization/GraphPlotter.java, source/net/yacy/visualization/HexGridPlotter.java, source/net/yacy/visualization/PrintTool.java, source/net/yacy/visualization/RasterPlotter.java
Sat May 30 06:01:52 CEST 2015
by Michael Peter Christen
adding a 3-pixel font generator made some time ago..
Changed Files: source/net/yacy/visualization/FontGenerator3Pixel.java
Thu May 28 16:07:40 CEST 2015
by Michael Peter Christen
All entities of crawl profiles are now editable in the crawl profile
editor.
Changed Files: htroot/CrawlProfileEditor_p.java, source/net/yacy/crawler/data/CrawlProfile.java, source/net/yacy/search/Switchboard.java
Wed May 27 02:31:13 CEST 2015
by reger
- Image search expand box, adjust javascript hs padtominsize parameter, to make sure expand box doesn't shrink on small images
- asure ImageResult.imagetext has value for the link text (use filename if no alt text given)
Changed Files: htroot/js/highslide/highslide.js, source/net/yacy/search/query/SearchEvent.java
Tue May 26 23:57:06 CEST 2015
by reger
remove hard throw exception in makeResultEntry
remove not used "share." peername.yacy url rewrite
Changed Files: source/net/yacy/kelondro/data/meta/URIMetadataNode.java
Tue May 26 23:54:04 CEST 2015
by reger
use available mime (instead null) on imageresult from metadatanode
Changed Files: source/net/yacy/search/query/SearchEvent.java
Tue May 26 04:26:26 CEST 2015
by reger
revert deletion of BinSearch
(accident)
Changed Files: source/net/yacy/kelondro/index/BinSearch.java
Tue May 26 04:15:00 CEST 2015
by reger
Eleminate duplication of values for search ResultEntry 
by instatiation from URIMetadataNode, by eleminating differentiation of ResultEntry/URIMetadataNode.
- moved remaining ResultEntry functionallity to URIMetadataNode
   - for 1:1 functionallity added a function makeResultEntry() 
- removed ResultEntry 
- refactored related code

Main difference is after makeResultEntry the text_t content is removed and alternative title/url strings for display are calculated.


Main difference left is, that 
Changed Files: htroot/yacy/search.java, htroot/yacysearchitem.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/search/query/SearchEvent.java
Mon May 25 21:51:32 CEST 2015
by reger
fix compiler notification of missing serialID 
from last commit
Changed Files: source/net/yacy/search/snippet/ResultEntry.java
Mon May 25 21:28:48 CEST 2015
by reger
refactor ResultEntry to be based on MetadataNode/SolrDocument
to share/reuse common access routines
Changed Files: htroot/yacysearchitem.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/snippet/ResultEntry.java
Mon May 25 19:46:26 CEST 2015
by reger
Implement sharing of ioDispatcher for term & citation index
as proposed in ioDispatcher description
Changed Files: source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/search/index/Segment.java
Mon May 25 00:08:38 CEST 2015
by reger
use doctype() in ViewFile to choose display routines
in preference of getfileExtension()
Changed Files: htroot/ViewFile.java
Sun May 24 21:48:58 CEST 2015
by reger
On imageSearch prefere mime to sort out none-image documents
Generalize the hack to prevent urls with just a img extension beeing returned

improving http://mantis.tokeek.de/view.php?id=528
Changed Files: source/net/yacy/search/query/SearchEvent.java
Sun May 24 19:38:04 CEST 2015
by reger
improve MultiprotocolURL.getFileExtension()
prevent string OOB while querypart contains a dot (return just "")
see log snippet in http://mantis.tokeek.de/view.php?id=533
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Sun May 24 18:03:27 CEST 2015
by reger
Increase IODspatcher dumpQueue size to 2 to reduce risk of concurrent emergency dump,
skip concurrent emergency merge
dealing with/see  http://mantis.tokeek.de/view.php?id=566
Changed Files: source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/kelondro/rwi/IndexCell.java
Sun May 24 01:59:40 CEST 2015
by reger
fix string OoB on getImagelinks with long alttext
in description calculation
Changed Files: source/net/yacy/document/Document.java
Sat May 23 20:31:37 CEST 2015
by reger
Convert content charset for display via CacheResource_p
Cached resource charset encoding might not fit to internal handling (using utf-8),
convert resource to utf-8
see http://mantis.tokeek.de/view.php?id=576
Changed Files: htroot/CacheResource_p.java
Fri May 22 11:22:36 CEST 2015
by Michael Peter Christen
removed a -UNRESOLVED_PATTERN-
Changed Files: htroot/yacysearch.json
Sun May 17 06:21:12 CEST 2015
by reger
Limit extra sleep of BusyThread on LowMemCycle
Changed Files: source/net/yacy/kelondro/workflow/AbstractBusyThread.java
Sun May 17 00:13:00 CEST 2015
by reger
detail optimization of RecrawlThread
Changed Files: source/net/yacy/crawler/RecrawlBusyThread.java
Sat May 16 01:23:08 CEST 2015
by reger
Initial (experimental) implementation of index update/re-crawl job
added to IndexReIndexMonitor_p.html
Selects existing documents from index and feeds it to the crawler.
currently only the field fresh_date_dt is used determine documents for recrawl (fresh_date_dt:[* TO NOW-1DAY]
Documents are  added in small chunks (200) to the crawler, only if no other crawl is running.
Changed Files: htroot/IndexReIndexMonitor_p.html, htroot/IndexReIndexMonitor_p.java, source/net/yacy/crawler/RecrawlBusyThread.java
Sat May 16 00:01:54 CEST 2015
by reger
correct log msg text
Changed Files: source/net/yacy/crawler/retrieval/FileLoader.java
Thu May 14 00:03:09 CEST 2015
by reger
fix extract of inboundlinks_protocol_sxt
url counter maybe > 999
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Wed May 13 21:58:43 CEST 2015
by reger
fix early return in addToCrawler
check / handle all supplied urls after error url
Changed Files: source/net/yacy/search/Switchboard.java
Tue May 12 12:06:21 CEST 2015
by Michael Peter Christen
fix for latest commit, see
https://github.com/yacy/yacy_search_server/commit/f810915717579d490259d70610dc4118b7c6e6e9#commitcomment-11145880
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Tue May 12 01:09:10 CEST 2015
by reger
fix NPE on MultiProtocolURL on url with parameter value and '='
in getAttribute
- added test case for it
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java, test/net/yacy/cora/document/id/MultiProtocolURLTest.java
Mon May 11 16:30:41 CEST 2015
by Michael Peter Christen
added crawl start from a clone with very, very large url: they are now
encoded as post submit form inside a javascript creation function.
Changed Files: htroot/CrawlStartExpert.java, htroot/Table_API_p.html, htroot/Table_API_p.java, source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/data/WorkTables.java
Mon May 11 14:42:21 CEST 2015
by Michael Peter Christen
enable api calls with very long urls
Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/data/WorkTables.java, source/net/yacy/search/index/Fulltext.java
Mon May 11 01:35:12 CEST 2015
by reger
upd library reference of missing jsch-0.1.21 in seeduploadscp.xml
upd to jsch-0.1.52.jar
Changed Files: .classpath, build.xml, lib/jsch-0.1.52.License, lib/jsch-0.1.52.jar, nbproject/project.xml, pom.xml, source/net/yacy/peers/operation/yacySeedUploadScp.xml
Sun May 10 18:52:33 CEST 2015
by reger
add opensearch rss results to dht collection (due to text = snippet)
which is used to differentiate meta from full data
- make sure check for dht is not dependant on number of collection entries
Changed Files: source/net/yacy/cora/federate/opensearch/OpenSearchConnector.java, source/net/yacy/search/index/Fulltext.java
Sun May 10 15:30:21 CEST 2015
by reger
add bookmark.query to edit form
Changed Files: htroot/Bookmarks.html, htroot/Bookmarks.java
Sun May 10 15:29:23 CEST 2015
by reger
persist bookmark timestamp
on setTimeStamp()
Changed Files: source/net/yacy/data/BookmarksDB.java
Sun May 10 03:00:05 CEST 2015
by reger
upd to commons-io-2.4.jar
Changed Files: .classpath, build.xml, lib/commons-io-2.4.License, lib/commons-io-2.4.jar, nbproject/project.xml, pom.xml
Sun May 10 02:29:08 CEST 2015
by reger
update bookmark autosearch description
- add german translation
Changed Files: htroot/Bookmarks.html, locales/de.lng
Fri May 08 15:30:26 CEST 2015
by Michael Peter Christen
added option to re-index exported xml snapshot dumps to
HTCACHE/snapshots by just placing them in the SURROGATES/in path
Changed Files: source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/search/Switchboard.java
Fri May 08 14:01:30 CEST 2015
by Michael Peter Christen
revert of 8a7c68e4c7f6a682e3ef656b423ce1ad76b42caa
keeping surrogates after processing is essential for some users. If the
space they are taking is too high, please set up an automatic deletion
process (like a cronjob).
Changed Files: source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java
Fri May 08 13:46:27 CEST 2015
by Michael Peter Christen
added must-not-match filter to snapshot generation.
also: fixed some bugs
Changed Files: htroot/CrawlStartExpert.html, htroot/CrawlStartExpert.java, htroot/Crawler_p.java, htroot/QuickCrawlLink_p.java, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/crawler/data/CrawlProfile.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/data/ymark/YMarkCrawlStart.java, source/net/yacy/search/index/Segment.java
Fri May 08 10:38:33 CEST 2015
by Michael Peter Christen
adding a try-catch to link graph processing to prevent that a single
malformed url interrupts the storage process
Changed Files: source/net/yacy/search/index/Segment.java
Sun May 03 02:31:50 CEST 2015
by reger
on bookmaring of search result, remember orig. query in separate bookmark property
(instead of using the description field)
- adjust display and autosearch
- don't overwrite existing bookmark but combine info
Changed Files: htroot/Bookmarks.html, htroot/Bookmarks.java, htroot/api/bookmarks/posts/add_p.java, htroot/yacysearch.java, source/net/yacy/data/BookmarksDB.java, source/net/yacy/search/AutoSearch.java, source/net/yacy/search/Switchboard.java
Sat May 02 02:36:18 CEST 2015
by reger
break out of NormalizeDistributor loop on timeout
Changed Files: source/net/yacy/search/ranking/ReferenceOrder.java
Fri May 01 19:24:14 CEST 2015
by reger
harmonize filesearch input box layout
Changed Files: htroot/yacyinteractive.html
Thu Apr 30 00:01:11 CEST 2015
by reger
upd to metadata-extractor-2.8.1
Changed Files: .classpath, build.xml, lib/metadata-extractor-2.8.1.License, lib/metadata-extractor-2.8.1.jar, nbproject/project.xml, pom.xml
Wed Apr 29 01:53:04 CEST 2015
by reger
upd to poi-3.11.jar
Changed Files: .classpath, build.xml, lib/poi-3.11-20141221.jar, lib/poi-3.11.License, lib/poi-scratchpad-3.11-20141221.jar, nbproject/project.xml, pom.xml
Tue Apr 28 03:12:14 CEST 2015
by reger
fix typo in image filter query
(extra bracket)
Changed Files: source/net/yacy/search/query/QueryGoal.java
Mon Apr 27 22:38:40 CEST 2015
by reger
fix String out of range in Collection Nav
see http://mantis.tokeek.de/view.php?id=573
Changed Files: source/net/yacy/search/query/QueryParams.java
Sun Apr 26 17:42:39 CEST 2015
by reger
improve character encoding for urlproxy servlet
for none utf-8 pages
Changed Files: source/net/yacy/http/servlets/UrlProxyServlet.java
Sun Apr 26 17:41:05 CEST 2015
by reger
upd to jsoup-1.8.2
Changed Files: .classpath, build.xml, lib/jsoup-1.8.2.jar, nbproject/project.xml, pom.xml
Sun Apr 26 04:29:32 CEST 2015
by reger
make Quality of Service Servlet available to prioritize requests from local host
This assigns priorities to incoming requests. Higher priority numbers are served before lower.
(disabled by default in defaults/web.xml, 
uncomment or copy entry to DATA/Settings/web.xml)
Changed Files: defaults/web.xml, source/net/yacy/http/Jetty9HttpServerImpl.java, source/net/yacy/http/servlets/YaCyQoSFilter.java
Sat Apr 25 22:38:38 CEST 2015
by reger
correct typo in de.lng
Changed Files: locales/de.lng
Sat Apr 25 03:24:28 CEST 2015
by reger
upd parser calls in test cases
Changed Files: test/net/yacy/document/ParserTest.java, test/net/yacy/document/parser/htmlParserTest.java, test/net/yacy/document/parser/images/genericImageParserTest.java, test/net/yacy/document/parser/images/metadataImageParserTest.java, test/net/yacy/document/parser/pdfParserTest.java
Sat Apr 25 02:45:05 CEST 2015
by reger
add additional links to crawl queue pages
Changed Files: htroot/Crawler_p.html
Thu Apr 23 18:17:28 CEST 2015
by Michael Peter Christen
don't record dump generation calls since that
- is not a change of the index
- happens very often within self-backup strategies from the outside
(i.e. cronjobs)
Changed Files: htroot/IndexControlURLs_p.java
Tue Apr 21 10:13:57 CEST 2015
by Michael Peter Christen
Merge pull request #4 from dertuxmalwieder/master

Readme improvements
Changed Files: readme.mediawiki
Mon Apr 20 14:30:45 CEST 2015
by Sven Knurr
Readme improvements

Now GitHub should display it properly. Also, added OpenBSD.
Changed Files: readme.mediawiki
Mon Apr 20 10:24:34 CEST 2015
by Michael Peter Christen
Merge pull request #2 from Scarfmonster/master

English Synonyms and small fixes
Changed Files: addon/synonyms/mobythesaurus_en_yacy, htroot/CrawlProfileEditor_p.html, htroot/DictionaryLoader_p.html, htroot/DictionaryLoader_p.java, source/net/yacy/crawler/data/CrawlProfile.java
Mon Apr 20 10:23:39 CEST 2015
by Michael Peter Christen
Merge pull request #3 from shaman/master

How about normal font weight in the searched titles & fix RSS icon position?
Changed Files: htroot/env/base.css, htroot/yacysearch.html, locales/uk.lng, skins/27c3.css
Mon Apr 20 04:27:15 CEST 2015
by Eugene Kuligin
fix typos
Changed Files: locales/uk.lng
Mon Apr 20 04:24:50 CEST 2015
by Eugene Kuligin
add vertical margin to the search cloud block
Changed Files: htroot/env/base.css
Mon Apr 20 04:19:37 CEST 2015
by Eugene Kuligin
fix RSS icon displaying
Changed Files: htroot/yacysearch.html
Mon Apr 20 04:13:05 CEST 2015
by Eugene Kuligin
increase view space with normal font weight in the searched titles
Changed Files: skins/27c3.css
Mon Apr 20 00:01:14 CEST 2015
by reger
fix NPE in Vocabulary_p servlet
called w/o parameter
Changed Files: htroot/Vocabulary_p.java
Sun Apr 19 15:55:49 CEST 2015
by Ryszard Go?
fix for Accept '?' URLs column in Crawl Profile List
Changed Files: htroot/CrawlProfileEditor_p.html, source/net/yacy/crawler/data/CrawlProfile.java
Fri Apr 17 15:14:10 CEST 2015
by Ryszard Go?
SynonymLibrary status check fix for multiple files
Changed Files: htroot/DictionaryLoader_p.java
Fri Apr 17 16:15:35 CEST 2015
by Ryszard Go?
added English synonyms
Changed Files: addon/synonyms/mobythesaurus_en_yacy, htroot/DictionaryLoader_p.html, htroot/DictionaryLoader_p.java
Fri Apr 17 02:14:13 CEST 2015
by reger
skip redundant add. of keywords to text
search uses keywords as default search field
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Thu Apr 16 02:36:12 CEST 2015
by reger
put https port in peers dna
as we flag if a peer is accesible via https, we need to know the port if we want to use is (e.g. for interYaCy communication)
start to provide / tansport the port by recording it in peers dna.
- add https link on the Network.html lock symbol
Changed Files: htroot/Network.html, htroot/Network.java, source/net/yacy/peers/Seed.java, source/net/yacy/search/Switchboard.java
Wed Apr 15 02:16:53 CEST 2015
by reger
incoming connection count/text fix
improvement on http://mantis.tokeek.de/view.php?id=570
Changed Files: htroot/AccessTracker_p.html, htroot/AccessTracker_p.java, locales/de.lng
Tue Apr 14 10:33:43 CEST 2015
by Michael Peter Christen
Merge pull request #1 from Scarfmonster/master

Search navigation fix
Changed Files: htroot/yacysearchtrailer.html
Tue Apr 14 03:19:27 CEST 2015
by reger
add info text icon next to Augmented Browsing check-box
with hint to config page
Changed Files: htroot/ConfigSearchPage_p.html
Tue Apr 14 02:07:02 CEST 2015
by reger
show "Augmented Browsing" link in search result only if urlproxy allowed and option switched on in layout
(AugmentedBrowsing_p.html, ConfigSearchPage_p.html)
as user only gets a error page if the option is not enabled
Changed Files: htroot/yacysearchitem.java
Mon Apr 13 23:32:06 CEST 2015
by Ryszard Go?
Search navigation fix
Changed Files: htroot/yacysearchtrailer.html
Mon Apr 13 16:20:00 CEST 2015
by Michael Peter Christen
added parsing of contentprop attribute in html tags for
content='startDate' and content='endDate'. The value of these field is
now written to new solr fields startDates_dts and endDates_dts.
Changed Files: defaults/solr.collection.schema, source/net/yacy/cora/federate/solr/SolrType.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/CollectionSchema.java
Mon Apr 13 16:18:15 CEST 2015
by Michael Peter Christen
reverted json syntax for facet results to version from january
Changed Files: htroot/yacysearchtrailer.java, htroot/yacysearchtrailer.json
Sun Apr 12 22:02:45 CEST 2015
by Michael Peter Christen
added parsing of dd, dt and article html fields. The parsed result is
written to special solr fields which are deactivated by default.
Changed Files: defaults/solr.collection.schema, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/CollectionSchema.java
Sat Apr 11 13:00:32 CEST 2015
by Michael Peter Christen
more logging during start-up
Changed Files: source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/crawler/data/NoticedURL.java
Fri Apr 10 16:16:20 CEST 2015
by Michael Peter Christen
<experimental> added parsing of <article> html element.
Whenever such an element occurs, the complete content of all article
elements replaces the parsed <content> part of documents.
Changed Files: source/net/yacy/document/parser/html/ContentScraper.java
Fri Apr 10 15:59:18 CEST 2015
by Michael Peter Christen
enhanced suggestions
Changed Files: htroot/suggest.java, source/net/yacy/data/DidYouMean.java
Fri Apr 10 15:10:18 CEST 2015
by Michael Peter Christen
replaced "fork me" banner with github banner
Changed Files: htroot/Status.html, htroot/env/grafics/forkme_right_green_007200.png
Wed Apr 08 22:42:30 CEST 2015
by reger
upd to httpcore 4.4.1
Changed Files: .classpath, build.xml, lib/httpcore-4.4.1.jar, nbproject/project.xml, pom.xml
Tue Apr 07 16:10:13 CEST 2015
by Michael Peter Christen
split query into filter query and text query to get better ranking
results and faster results
Changed Files: source/net/yacy/http/servlets/GSAsearchServlet.java, source/net/yacy/http/servlets/SolrSelectServlet.java, source/net/yacy/search/query/QueryGoal.java, source/net/yacy/search/query/QueryParams.java
Tue Apr 07 13:14:41 CEST 2015
by Michael Peter Christen
testing switching off cold searchers; maybe this brings performance
enhancements when using large facets
Changed Files: defaults/solr/solrconfig.xml
Tue Apr 07 13:13:58 CEST 2015
by Michael Peter Christen
when selecting collections in navigation, do show the un-selected
collections in search result. When selecting one of them in another
search, switch off the previously selected collection. This actually
turns the collection navigation modifier into a radio-button like
behaviour
Changed Files: htroot/yacysearchtrailer.java, source/net/yacy/search/query/QueryModifier.java, source/net/yacy/search/query/QueryParams.java
Tue Apr 07 00:10:42 CEST 2015
by reger
replace deprecated getIP with getIPs in AbstractRemoteHandler
Changed Files: source/net/yacy/http/AbstractRemoteHandler.java
Sat Apr 04 21:11:09 CEST 2015
by reger
include htroot (*.class) in maven clean 
harmonize antrun javac call in pom with build.xml
Changed Files: pom.xml
Sat Apr 04 00:24:16 CEST 2015
by reger
add SynonymLibrary status to DictionaryLoader_p servlet 
http://mantis.tokeek.de/view.php?id=564
Changed Files: htroot/DictionaryLoader_p.java
Thu Apr 02 13:27:47 CEST 2015
by Michael Peter Christen
refactoring of filter queries (several queries instead only one)
Changed Files: source/net/yacy/search/query/QueryParams.java
Thu Apr 02 02:10:00 CEST 2015
by reger
show location nav as selectable nav in search page layout
- switch automatically on upon load of geodata provider
- but allow switch on also without geodata file (and display the location nav if search result has lat/lon location)
Changed Files: htroot/ConfigSearchPage_p.html, htroot/ConfigSearchPage_p.java, htroot/DictionaryLoader_p.java, htroot/yacysearchtrailer.java
Wed Apr 01 18:37:45 CEST 2015
by Michael Peter Christen
use a cursor hand on facet headline to show that this is clickable
Changed Files: htroot/yacysearchtrailer.html
Wed Apr 01 18:17:52 CEST 2015
by Michael Peter Christen
added an expansion option to search facets on result page:
- if less or equal of 8 facet options are present, they are shown by
default
- if more facet options are present, they are hidden
To view or hide all facets, just click on the facet header bar
Changed Files: htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java, source/net/yacy/http/servlets/GSAsearchServlet.java, source/net/yacy/http/servlets/SolrSelectServlet.java, source/net/yacy/search/query/QueryGoal.java, source/net/yacy/search/query/QueryParams.java
Wed Apr 01 01:57:56 CEST 2015
by reger
make location facet return results
for location nav facet of field coordinate_p does not return results, now using coordinate_p_0_coordinate as alternative to get facet counts. As the actual facet value is not used this should not harm any analysis (even if facet is a incomplete location).
If facet value is used in future likely *_geohash field could be introduced (for facet and other ... as transport value)
Changed Files: source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java
Sun Mar 29 07:12:23 CEST 2015
by reger
fix display and limit of max server connections after startup
(on restart value returned to default=50)
This has no effect on Jetty but the limit is still respected.
Changed Files: source/net/yacy/yacy.java
Sun Mar 29 05:48:54 CEST 2015
by reger
add err msg on failure during Load_rss
Changed Files: htroot/Load_RSS_p.html, htroot/Load_RSS_p.java
Sat Mar 28 03:05:21 CET 2015
by reger
correct percent encoding for '%' char
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Tue Mar 24 12:32:39 CET 2015
by Michael Peter Christen
added a new collection type 'dht' to all documents from the peer-to-peer
interface to distinguish rich and poor document data.
This also reverts some changes from commit
796770e070daf38289b594f4cbdc65b9ce0ca2b1 because the firstSeen database
is the wrong method to distinguish these types of data
Changed Files: htroot/yacy/crawlReceipt.java, htroot/yacy/transferURL.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/kelondro/logging/ConsoleOutErrHandler.java, source/net/yacy/peers/Protocol.java, source/net/yacy/search/index/Fulltext.java
Tue Mar 24 00:13:05 CET 2015
by reger
fix missing display of   CrawlerMonitor -> robots.txt Monitor
revert delete of file api/table_p.html see https://gitorious.org/yacy/rc1/commit/3ffe19b85cb6f2c1d26922eb61ca73a42096e8e9
(still used in this menu)
Changed Files: htroot/Table_RobotsTxt_p.html, htroot/api/table_p.html
Mon Mar 23 11:12:39 CET 2015
by Marc Nause
Updated Git links from Gitorious to Github.
Changed Files: addon/yacy-svn-4.spec, htroot/ConfigUpdate_p.html, htroot/Status.html, htroot/env/templates/header.template, htroot/env/templates/simpleheader.template, htroot/opensearchdescription.xml, libbuild/pom.xml, pom.xml, readme.txt
Mon Mar 23 03:57:47 CET 2015
by reger
prevent overwrite of crawled or received full documents by (newer) metadata
To protect rich index data (full resource) from overwriting by metadata gathered during remote search,
the newly introduced "firstSeen" index is used to differentiate between full-resource-doc and metadata,
as a "firstSeen" entry is only added on store's of full-resource-docs (during crawl or remote search).
Changed Files: source/net/yacy/peers/Protocol.java
Wed Mar 18 21:57:41 CET 2015
by otter
Fixes hanging FlushThread (see
http://forum.yacy-websuche.de/viewtopic.php?f=5&t=5447)
by replacing put() method by the more robust add() to
add a merge job to the queue.
Changed Files: source/net/yacy/kelondro/rwi/IODispatcher.java
Mon Mar 16 02:03:40 CET 2015
by reger
fix snippet containig number with comma as desmo point  http://mantis.tokeek.de/view.php?id=344
to keep it as one word (by altering the split regex)
- added sniipet test case with number
- regex for word split to match multiple splitcars
Changed Files: source/net/yacy/search/snippet/TextSnippet.java, test/net/yacy/search/snippet/TextSnippetTest.java
Sun Mar 15 22:31:47 CET 2015
by reger
fix error on *abc query input
http://mantis.tokeek.de/view.php?id=486
Changed Files: source/net/yacy/search/query/QueryModifier.java
Sun Mar 15 06:02:45 CET 2015
by reger
apply UTF-8 encoding
copied from escape()
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java, test/net/yacy/cora/document/id/MultiProtocolURLTest.java
Sun Mar 15 03:37:32 CET 2015
by reger
fix for path with char code > 255 
(causing index out of bound exception)
+ test cas for it
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java, test/net/yacy/cora/document/id/MultiProtocolURLTest.java
Sun Mar 15 00:46:07 CET 2015
by reger
fix url encoding for path see http://mantis.tokeek.de/view.php?id=559
So far we used same escape procedure for all parts of the url (which includes x-www-form-urlencoded for all url components)
Added capability to use different encoding rules for the different url components (through specific bitset for each component).
(this is inspired by org.apache.http.client and java.net.uri implementation).
- Added test case for  http://mantis.tokeek.de/view.php?id=559
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java, test/net/yacy/cora/document/id/MultiProtocolURLTest.java
Wed Mar 11 19:36:23 CET 2015
by reger
try to fix hang on index blob merge on shutdown
http://mantis.tokeek.de/view.php?id=505
It happens but not able to reproduce. This change makes sure terminate signal is catched at end of currently running merge jobs
Changed Files: source/net/yacy/kelondro/rwi/IODispatcher.java
Wed Mar 11 01:05:14 CET 2015
by reger
fix url (path) %-decoding http://mantis.tokeek.de/view.php?id=519
- add test case for this
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java, test/net/yacy/cora/document/id/MultiProtocolURLTest.java
Mon Mar 09 00:09:36 CET 2015
by reger
update configheuristics_p.html text 
to state current opensearch heuristic function
Changed Files: htroot/ConfigHeuristics_p.html, locales/de.lng, locales/ru.lng
Sun Mar 08 22:10:51 CET 2015
by reger
fix version conflict in pom
for commons-io
Changed Files: pom.xml
Sun Mar 08 21:49:23 CET 2015
by reger
exclude default search fields from  text copy to text_t
for metadata index documents (reduce text redundance)
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Sun Mar 08 02:34:48 CET 2015
by reger
For remote crawlReceipt add document abstract/description
enhance the returned metadata returned to the originator by description_txt to improve fulltext search result hits.
Changed Files: source/net/yacy/peers/Protocol.java
Thu Mar 05 02:22:05 CET 2015
by reger
harmonize snippet computation
to considere description_txt always (solr hl & internal).
For now just added desc to text list for computation, could be further equalized with hl computation.
Changed Files: source/net/yacy/search/snippet/TextSnippet.java
Thu Mar 05 02:09:27 CET 2015
by reger
remove unused statement
Changed Files: htroot/PerformanceMemory_p.java
Mon Mar 02 13:10:05 CET 2015
by Michael Peter Christen
added special terms for on: - Date modifier: tomorrow, today; i.e.:
search for: "Berlin on:tomorrow" to find events happening tomorrow in
Berlin
Changed Files: source/net/yacy/document/DateDetection.java
Mon Mar 02 12:55:31 CET 2015
by Michael Peter Christen
generalized time period computations
Changed Files: htroot/Crawler_p.java, htroot/IndexDeletion_p.java, htroot/Table_API_p.java, htroot/api/timeline_p.java, htroot/yacysearchtrailer.java, source/net/yacy/cora/date/AbstractFormatter.java
Sun Mar 01 23:50:17 CET 2015
by reger
add test case for snippet html encoding check
Changed Files: test/net/yacy/search/snippet/TextSnippetTest.java
Sat Feb 28 19:48:29 CET 2015
by reger
upd pom
Changed Files: nbproject/project.xml, pom.xml
Sat Feb 28 19:02:18 CET 2015
by reger
postpone raw snippet html encoding upon use
instead of during init of snippet 
adressing http://mantis.tokeek.de/view.php?id=551
Changed Files: source/net/yacy/search/snippet/TextSnippet.java
Fri Feb 27 00:53:20 CET 2015
by reger
apply query parameter getQueryFields() to GSA servlet
Changed Files: source/net/yacy/http/servlets/GSAsearchServlet.java
Wed Feb 25 21:11:59 CET 2015
by Marc Nause
Next try to fix start script for OpenBSD.
Changed Files: startYACY.sh
Wed Feb 25 01:58:42 CET 2015
by reger
fix mimetype of rss items in rss parser
- remove self reference as anchor for items
Changed Files: source/net/yacy/document/parser/rssParser.java
Wed Feb 25 01:05:46 CET 2015
by Michael Peter Christen
enhanced date parsing time
Changed Files: source/net/yacy/document/DateDetection.java
Mon Feb 23 23:12:07 CET 2015
by reger
introduce getQueryFields to return default query fields (queryparamter QF)
calculated from boostfields config, making sure title, description, keywords and content is always searched.
- apply change to solrServlet makes sure every remote query uses at least all locally defined boost fields for search
- apply to local solr search
- simplify select query by using QF defaults
Changed Files: source/net/yacy/cora/federate/solr/Ranking.java, source/net/yacy/http/servlets/SolrSelectServlet.java, source/net/yacy/search/query/QueryGoal.java, source/net/yacy/search/query/QueryParams.java
Sun Feb 22 05:42:04 CET 2015
by reger
add description_txt to default query fields,
Dublin Core Metadata field extracted by most parsers.
Changed Files: defaults/yacy.init, htroot/RankingSolr_p.java
Sun Feb 22 05:31:56 CET 2015
by reger
add extracted description/subject to pptParser
Changed Files: source/net/yacy/document/parser/pptParser.java
Fri Feb 20 02:21:04 CET 2015
by reger
url unescape add check for inconsistent utf8 multibyte parsing
If the url contains special chars (like umlaute ) it's interpreted as multybyte char and actually not converted at all (removed).
Added a check if the multibyte convesion is not complete, just add the char as is.

This fixes http://mantis.tokeek.de/view.php?id=200
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Tue Feb 17 03:16:10 CET 2015
by reger
stop sending crawl receipts if receiver got offline
Changed Files: source/net/yacy/search/Switchboard.java
Mon Feb 16 01:20:12 CET 2015
by reger
upd lucene api doc link
Changed Files: htroot/ContentAnalysis_p.html
Mon Feb 16 00:50:16 CET 2015
by reger
add extracted description/subject to docParser 
Changed Files: source/net/yacy/document/parser/docParser.java
Sun Feb 15 23:09:01 CET 2015
by reger
replace depreciated HTTPClient setStaleConnectionCheckEnabled with setValidateAfterInactivity()
Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java
Sun Feb 15 21:34:01 CET 2015
by reger
replace depriciated HTTPClient ALLOW_ALL_HOSTNAME_VERIFIER with NoopHostnameVerifier()
Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java
Sun Feb 15 20:39:20 CET 2015
by reger
fix formatting issue if snippet contains html code
replacement for reverted commit
https://gitorious.org/yacy/rc1/commit/61f42a792872c7fc83c7d4d64e1f0bb9d063c810
Changed Files: source/net/yacy/search/snippet/TextSnippet.java
Sat Feb 14 23:04:05 CET 2015
by reger
upd to commons-codec-1.10.jar, commons-compress-1.9.jar
Changed Files: .classpath, build.xml, lib/commons-codec-1.10.License, lib/commons-codec-1.10.jar, lib/commons-compress-1.9.License, lib/commons-compress-1.9.jar, nbproject/project.xml, pom.xml
Sat Feb 14 02:43:05 CET 2015
by reger
revert: formatting fix eats also up highlighting
need other solution for snippets with unwanted html code 
Changed Files: htroot/yacysearchitem.java
Fri Feb 13 00:50:32 CET 2015
by reger
upd to httpclient-4.4
Changed Files: .classpath, lib/httpclient-4.4.License, lib/httpclient-4.4.jar, lib/httpcore-4.4.License, lib/httpcore-4.4.jar, lib/httpmime-4.4.License, lib/httpmime-4.4.jar, nbproject/project.xml, pom.xml
Fri Feb 13 00:20:33 CET 2015
by reger
fix formatting issue in search result display 
if description contains html code
noticed e.g. for id=NmNdJ9uApLaQ  http://hswong3i.net/blog/hswong3i/virtualmin-drupal-7-x-ubuntu-12-04-howto
Changed Files: htroot/yacysearchitem.java
Wed Feb 11 23:26:39 CET 2015
by reger
allow/recognize host in file: protocol crawl target
This is useful in intranet indexing while crawling a intranet file server accessed via hostname while e.g. under Windows mapped to different drive letters on individual clients.
Here you can crawl e.g.  file://fileserver/documents having a valid uri in that intranet environment (while e.g. P:/documents might be client dependant).
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Wed Feb 11 01:43:02 CET 2015
by reger
fix parser test cases
(Vocabulary paramete)
Changed Files: test/net/yacy/document/ParserTest.java, test/net/yacy/document/parser/htmlParserTest.java, test/net/yacy/document/parser/images/genericImageParserTest.java, test/net/yacy/document/parser/images/metadataImageParserTest.java, test/net/yacy/document/parser/pdfParserTest.java, test/net/yacy/search/snippet/TextSnippetTest.java
Wed Feb 11 01:42:01 CET 2015
by reger
disable optimistic GC assumption in StandardMemoryStrategy
After several tests found that eom is not prevented. Major reason in testing was assumption future GC will free avg of last 5 GC.
Disabeling this check improved eom exceptions.

Added simplest testcase used for verification
Changed Files: source/net/yacy/kelondro/util/StandardMemoryStrategy.java, test/net/yacy/kelondro/util/MemoryControlTest.java
Tue Feb 10 08:43:45 CET 2015
by Michael Peter Christen
the cleanup process experienced a 100% CPU load situation and the loop
did not terminate:

Occurrences: 100
at java.util.HashMap$KeyIterator.next(HashMap.java:956)
at
net.yacy.cora.protocol.ConnectionInfo.cleanup(ConnectionInfo.java:300)
at
net.yacy.cora.protocol.ConnectionInfo.cleanUp(ConnectionInfo.java:293)
at net.yacy.search.Switchboard.cleanupJob(Switchboard.java:2212)
at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
net.yacy.kelondro.workflow.InstantBusyThread.job(InstantBusyThread.java:105)
at
net.yacy.kelondro.workflow.AbstractBusyThread.run(AbstractBusyThread.java:215)

This tries to fix the problem; the problem should be monitored
Changed Files: source/net/yacy/cora/protocol/ConnectionInfo.java
Mon Feb 09 18:46:06 CET 2015
by Michael Peter Christen
hack to make date detection faster (while it becomes a bit incomplete
regarding language alternatives)
Changed Files: source/net/yacy/document/DateDetection.java
Mon Feb 09 18:45:07 CET 2015
by Michael Peter Christen
enhanced suggest function
Changed Files: htroot/suggest.java, htroot/yacysearch.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/data/DidYouMean.java
Sun Feb 08 23:40:33 CET 2015
by reger
fix Umlaut handling in blekko heuristic search term
http://mantis.tokeek.de/view.php?id=169
observation: blekko seams to block xxxbot agents (=0 results)
Changed Files: defaults/heuristicopensearch.conf, source/net/yacy/cora/federate/opensearch/OpenSearchConnector.java
Sat Feb 07 22:01:54 CET 2015
by reger
url with semicolon or comma handling in proxy request
apply patch supplied with bugreport http://mantis.tokeek.de/view.php?id=540
Changed Files: source/net/yacy/http/ProxyHandler.java
Sat Feb 07 13:47:15 CET 2015
by sixcooler
small correction for last commit
Changed Files: .classpath
Sat Feb 07 00:37:43 CET 2015
by reger
upd error message for proxy
fix http://mantis.tokeek.de/view.php?id=539
Changed Files: source/net/yacy/http/AbstractRemoteHandler.java
Wed Feb 04 11:37:07 CET 2015
by Michael Peter Christen
remove remote indexing option in crawl start if not in p2p mode
Changed Files: htroot/CrawlStartExpert.html, htroot/CrawlStartExpert.java
Wed Feb 04 03:51:34 CET 2015
by reger
adjust table column width to not line wrap crawler traffic line
Changed Files: htroot/Crawler_p.html
Wed Feb 04 01:50:35 CET 2015
by Michael Peter Christen
cloning a crawl now accepts the class name of vocabulary scapers
Changed Files: htroot/CrawlStartExpert.java
Wed Feb 04 01:12:25 CET 2015
by Michael Peter Christen
configuration option for maxload limit for remote search
Changed Files: defaults/yacy.init, source/net/yacy/peers/RemoteSearch.java, source/net/yacy/search/SwitchboardConstants.java
Tue Feb 03 03:08:34 CET 2015
by reger
add shortMemory check to heuristic search 
and skip operation on shortMemory (no request to remote openserch systems)
Changed Files: source/net/yacy/cora/federate/FederateSearchManager.java
Sun Feb 01 05:35:09 CET 2015
by reger
fix: malformed filename in image search
fix for http://mantis.tokeek.de/view.php?id=533
Changed Files: htroot/yacysearchitem.java
Sun Feb 01 04:26:33 CET 2015
by reger
refactor: just some more useages of constant for term ":[* TO *]"
Changed Files: htroot/HostBrowser.java, htroot/RankingSolr_p.java, source/net/yacy/cora/federate/solr/logic/CatchallLiteral.java, source/net/yacy/search/schema/CollectionConfiguration.java
Sun Feb 01 00:29:28 CET 2015
by reger
remove hardcoded initialization of language nav if not used
Changed Files: source/net/yacy/search/query/SearchEvent.java
Fri Jan 30 21:17:23 CET 2015
by Marc Nause
Added & in start script for *NIX which was lost a few commits ago.
Changed Files: startYACY.sh
Thu Jan 29 11:39:47 CET 2015
by Michael Peter Christen
refactoring of autotagging code (combined same code pieces)
Changed Files: source/net/yacy/cora/lod/vocabulary/Tagging.java
Thu Jan 29 02:45:32 CET 2015
by Michael Peter Christen
enhanced initialization speed of vocabularies by using better
normalization and by removal of unused data structures
Changed Files: source/net/yacy/cora/lod/vocabulary/Tagging.java
Thu Jan 29 02:22:28 CET 2015
by Michael Peter Christen
using precompiled CommonPattern.TAB for split
Changed Files: source/net/yacy/cora/geo/GeonamesLocation.java, source/net/yacy/cora/util/CommonPattern.java, source/net/yacy/document/parser/csvParser.java
Thu Jan 29 02:19:41 CET 2015
by Michael Peter Christen
using precompiled pattern CommonPattern.SEMICOLON for splits
Changed Files: htroot/CookieTest_p.java, htroot/Vocabulary_p.java, source/net/yacy/data/UserDB.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/html/ScraperInputStream.java, source/net/yacy/document/parser/vcfParser.java
Thu Jan 29 02:16:42 CET 2015
by Michael Peter Christen
persistency for vocabulary facet switch
Changed Files: htroot/Vocabulary_p.html, htroot/Vocabulary_p.java, source/net/yacy/cora/lod/vocabulary/Tagging.java, source/net/yacy/search/Switchboard.java, source/net/yacy/server/serverSwitch.java
Thu Jan 29 01:53:36 CET 2015
by Michael Peter Christen
introducting a new getConfig method which parses comma-separated llists
from setting fields; refactoring for all places where such lists are
parsed
Changed Files: htroot/IndexControlRWIs_p.java, htroot/yacysearch.java, source/net/yacy/server/http/HTTPDFileHandler.java, source/net/yacy/server/serverSwitch.java
Thu Jan 29 01:35:28 CET 2015
by Michael Peter Christen
refactoring with CommonPattern.COMMA
Changed Files: defaults/yacy.init, source/net/yacy/cora/lod/vocabulary/Tagging.java, source/net/yacy/data/ListManager.java, source/net/yacy/kelondro/index/Row.java
Thu Jan 29 01:22:28 CET 2015
by Michael Peter Christen
do not reindex based on vocabulary fields (there are meanwhile many of
them) and some default settings
Changed Files: defaults/yacy.init, source/net/yacy/migration.java
Thu Jan 29 00:33:07 CET 2015
by Michael Peter Christen
refactoring of reindexSolr (just replaced constant string)
Changed Files: htroot/IndexReIndexMonitor_p.java, source/net/yacy/migration.java, source/net/yacy/search/index/ReindexSolrBusyThread.java, source/net/yacy/server/serverSwitch.java
Wed Jan 28 03:59:01 CET 2015
by reger
Allow to hide linkstructure graphic in crawl monitor 
using/setting the config param DECORATION_GRAFICS_LINKSTRUCTURE
Changed Files: htroot/Crawler_p.html, htroot/Crawler_p.java, htroot/HostBrowser.html, htroot/HostBrowser.java
Tue Jan 27 17:00:20 CET 2015
by Michael Peter Christen
removed some warnings
Changed Files: htroot/ViewImage.java, htroot/api/snapshot.java, source/net/yacy/cora/federate/AbstractFederateSearchConnector.java, source/net/yacy/cora/federate/FederateSearchManager.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/search/AutoSearch.java, source/net/yacy/search/snippet/ResultEntry.java, source/net/yacy/server/http/HTTPDemon.java
Tue Jan 27 16:53:09 CET 2015
by Michael Peter Christen
the LinkedBlockingQueue is much faster than the ArrayBlockingQueue
(strange but this is the result of a test:
ArrayBlockingQueue: 39461 lines / second;
LinkedBlockingQueue: 60774 lines / second)
Changed Files: source/net/yacy/cora/language/synonyms/SynonymLibrary.java, source/net/yacy/cora/lod/vocabulary/Tagging.java, source/net/yacy/cora/storage/Files.java, source/net/yacy/crawler/retrieval/URLRewriterLibrary.java
Sat Jan 24 23:17:07 CET 2015
by reger
fix: eom on parsing ico file by genericImageParser

trace: java.lang.OutOfMemoryError: Java heap space
	at java.awt.image.DataBufferInt.<init>(DataBufferInt.java:75)
	at java.awt.image.Raster.createPackedRaster(Raster.java:467)
	at java.awt.image.DirectColorModel.createCompatibleWritableRaster(DirectColorModel.java:1032)
	at java.awt.image.BufferedImage.<init>(BufferedImage.java:331)
	at net.yacy.document.parser.images.bmpParser$IMAGEMAP.<init>(bmpParser.java:149)
	at net.yacy.document.parser.images.bmpParser.parse(bmpParser.java:69)
	at net.yacy.document.parser.images.genericImageParser.parse(genericImageParser.java:116)
Changed Files: source/net/yacy/document/parser/images/genericImageParser.java
Sat Jan 24 07:10:14 CET 2015
by Michael Peter Christen
update to latest code changes from json.org
Changed Files: source/net/yacy/cora/util/JSONArray.java, source/net/yacy/cora/util/JSONException.java, source/net/yacy/cora/util/JSONObject.java, source/net/yacy/cora/util/JSONString.java, source/net/yacy/cora/util/JSONTokener.java
Sat Jan 24 01:53:58 CET 2015
by reger
Let auto-disabled crawls recover if low resource condition vanished.
Analog to autodisabled DHT switch autodisabled crawls back on upon mem ok
by remembering the autodisable by conf parameter.
Changed Files: pom.xml, source/net/yacy/search/ResourceObserver.java, source/net/yacy/search/SwitchboardConstants.java
Fri Jan 23 17:57:54 CET 2015
by Michael Peter Christen
write java version to status page
Changed Files: htroot/Status.java, htroot/Status_p.inc
Fri Jan 23 11:31:05 CET 2015
by Michael Peter Christen
new development cycle
Changed Files: build.properties