YaCy Release 1.80

Release 1.8

Major Changes   
Jump to: Bugfixes / Other Changes

Fri Sep 05 12:54:21 CEST 2014
by Michael Peter Christen
css skin fix: visited was not set which caused bad colors on new-user
menu design.
Changed Files: htroot/env/base.css, skins/27c3.css, skins/28c3.css, skins/classic-blue.css, skins/generic_pd.css, skins/geo.css, skins/grey.css, skins/mint-green.css, skins/pdblue.css, skins/pdbootstrap.css, skins/phosphor.css, skins/vega-aqua.css
Mon Aug 25 22:47:33 CEST 2014
by Marc Nause
Fix for http://mantis.tokeek.de/view.php?id=432

*) replaced all shebangs with more universal versions
Changed Files: addon/thumbnail.sh, addon/yacyInit.sh, addon/yacyInit2.sh, bin/addrss.sh, bin/apicall.sh, bin/apicat.sh, bin/checkalive.sh, bin/checkindex.sh, bin/clearall.sh, bin/clearapi.sh, bin/clearcache.sh, bin/clearindex.sh, bin/deleteurl.sh, bin/deploy.sh, bin/down.sh, bin/dumpcheck.sh, bin/importOAIList.sh, bin/importmediawiki.sh, bin/importurllist.sh, bin/indexdump.sh, bin/indexrestore.sh, bin/passwd.sh, bin/search.sh, bin/search1.sh, bin/searchall.sh, bin/searchtest.sh, bin/searchtestmulti.sh, bin/up.sh, killYACY.sh, reconfigureYACY.sh, startYACY.sh, stopYACY.sh, updateYACY.sh
Wed Aug 20 22:45:36 CEST 2014
by reger
remove old description for proxy port settings (Settings_p.html?page=ProxyAccess)
- The options were not current (only port number accepted, which is part of ConfigBasic.html)
- Deleted options and the port number input field from the proxyaccess page.
- joined both transparent proxy setup pages (Settings_Http.inc & Settings_ProxyAccess.inc) in one page
- adjustments to the related/linked pages
Changed Files: htroot/ConfigBasic.html, htroot/SettingsAck_p.html, htroot/SettingsAck_p.java, htroot/Settings_ProxyAccess.inc, htroot/Settings_p.html, htroot/Settings_p.java, htroot/Status_p.inc
Tue Aug 19 19:58:31 CEST 2014
by orbiter
added a unlock icon for all protected pages that are unlocked because
the administrator is logged in.
Changed Files: htroot/Bookmarks.html, htroot/Status.html, htroot/YMarks.html, htroot/env/base.css, htroot/env/grafics/lockopen.gif, htroot/env/templates/header.template, htroot/env/templates/submenuAccessTracker.template, htroot/env/templates/submenuBlacklist.template, htroot/env/templates/submenuComputation.template, htroot/env/templates/submenuConfig.template, htroot/env/templates/submenuCookie.template, htroot/env/templates/submenuCrawlMonitor.template, htroot/env/templates/submenuCrawler.template, htroot/env/templates/submenuDesign.template, htroot/env/templates/submenuIndexControl.template, htroot/env/templates/submenuIndexCreate.template, htroot/env/templates/submenuIndexImport.template, htroot/env/templates/submenuMaintenance.template, htroot/env/templates/submenuPortalConfiguration.template, htroot/env/templates/submenuPublication.template, htroot/env/templates/submenuRanking.template, htroot/env/templates/submenuSemantic.template, htroot/env/templates/submenuTargetAnalysis.template, htroot/env/templates/submenuUseCaseAccount.template, htroot/env/templates/submenuWebStructure.template, source/net/yacy/http/servlets/YaCyDefaultServlet.java
Tue Aug 19 14:51:07 CEST 2014
by sixcooler
bump to httpclient-4.3.5
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/dependencies.txt, lib/httpclient-4.3.5.License, lib/httpclient-4.3.5.jar, lib/httpmime-4.3.5.License, lib/httpmime-4.3.5.jar, nbproject/project.xml, pom.xml
Sun Aug 17 00:59:16 CEST 2014
by reger
update to Jetty 9.2.2
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/jetty-9.2.2.v20140723.License, lib/jetty-client-9.2.2.v20140723.jar, lib/jetty-continuation-9.2.2.v20140723.jar, lib/jetty-deploy-9.2.2.v20140723.jar, lib/jetty-http-9.2.2.v20140723.jar, lib/jetty-io-9.2.2.v20140723.jar, lib/jetty-jmx-9.2.2.v20140723.jar, lib/jetty-proxy-9.2.2.v20140723.jar, lib/jetty-security-9.2.2.v20140723.jar, lib/jetty-server-9.2.2.v20140723.jar, lib/jetty-servlet-9.2.2.v20140723.jar, lib/jetty-servlets-9.2.2.v20140723.jar, lib/jetty-util-9.2.2.v20140723.jar, lib/jetty-webapp-9.2.2.v20140723.jar, lib/jetty-xml-9.2.2.v20140723.jar, pom.xml
Sun Aug 10 22:33:10 CEST 2014
by reger
use more predefined Solr query parameter constants
- use CommonParams and DisMaxParams constants
- fix typo in get sort parameter
- getDocumentCountByParams redundant implementation and risk of not optimized call (row parameter unspecified) -> as only used from getCountByQuery removed from interface
Changed Files: source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/RemoteSolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java, source/net/yacy/cora/federate/solr/responsewriter/GrepHTMLResponseWriter.java, source/net/yacy/search/query/QueryParams.java
Sun Aug 10 01:29:16 CEST 2014
by reger
simplify rssreader and improve atom feed  link extraction
- type detection (rss/atom) 
    - init type parameter overwritten during parse, parameter obsolete
    - detection by endtag changed to simpler first-tag evaluation
- channel image not used, removed related extra parser handling
    - remove unused code (set/getImage) in rssfeed
- atom link extraction to account for possible multipe link tags
   - spec limits link to one with rel="alternate" or one without rel attribute
     not accounting for the follwing type & hreflang exception yet:

   o  atom:entry elements MUST NOT contain more than one atom:link
      element with a rel attribute value of "alternate" that has the
      same combination of type and hreflang attribute values.
Changed Files: source/net/yacy/cora/document/feed/RSSFeed.java, source/net/yacy/cora/document/feed/RSSReader.java, source/net/yacy/document/parser/rssParser.java
Mon Aug 04 02:35:38 CEST 2014
by Michael Peter Christen
more logic on Solr queries; usage of the query terms in posprocessing,
saving one query for double document detection now per document
Changed Files: source/net/yacy/cora/federate/solr/logic/AbstractOperations.java, source/net/yacy/cora/federate/solr/logic/AbstractTerm.java, source/net/yacy/cora/federate/solr/logic/Conjunction.java, source/net/yacy/cora/federate/solr/logic/Disjunction.java, source/net/yacy/cora/federate/solr/logic/Literal.java, source/net/yacy/cora/federate/solr/logic/Negation.java, source/net/yacy/cora/federate/solr/logic/Operations.java, source/net/yacy/cora/federate/solr/logic/Term.java, source/net/yacy/cora/federate/solr/logic/TermTools.java, source/net/yacy/kelondro/util/MemoryControl.java, source/net/yacy/search/schema/CollectionConfiguration.java
Fri Aug 01 11:00:10 CEST 2014
by orbiter
better error handling for remote solr queries and exists-checks
Changed Files: htroot/HostBrowser.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/Load_RSS_p.java, htroot/api/citation.java, htroot/api/linkstructure.java, htroot/api/webstructure.java, htroot/api/ymarks/add_ymark.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacysearch.java, source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java, source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/crawler/retrieval/RSSLoader.java, source/net/yacy/crawler/retrieval/SitemapImporter.java, source/net/yacy/data/ymark/YMarkMetadata.java, source/net/yacy/peers/Transmission.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/index/Segment.java
Fri Jul 25 23:55:10 CEST 2014
by reger
update to pdfbox 1.8.6
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/fontbox-1.8.6.License, lib/fontbox-1.8.6.jar, lib/jempbox-1.8.6.License, lib/jempbox-1.8.6.jar, lib/pdfbox-1.8.6.License, lib/pdfbox-1.8.6.jar, pom.xml
Mon Jul 21 23:54:23 CEST 2014
by Michael Peter Christen
refactoring (change Metadata name of load time data structure to avoid
confusion with Node data which is also called metadata)
Changed Files: source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/MirrorSolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrConnector.java, source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/schema/CollectionConfiguration.java
Sun Jul 20 00:00:43 CEST 2014
by reger
Combine /heuristic search modifier with opensearch configured targets
- with search modifier /heuristic a request is send to all configured opensearch target systems (old /heuristic/blekko modifier not longer valid)
- this allows to use opensearch heuristic on individual search request (in contrast to configuration HEURISTIC_OPENSEARCH=true which sends a osd request on all global searches
- the index.html searchoption text adjusted to be displayed only if option configured
- add Archive-It to predefined systems
Changed Files: defaults/heuristicopensearch.conf, htroot/index.html, htroot/index.java, htroot/yacysearch.java, source/net/yacy/cora/federate/opensearch/OpenSearchConnector.java, source/net/yacy/cora/federate/opensearch/SRURSSConnector.java, source/net/yacy/search/Switchboard.java
Fri Jul 18 12:43:01 CEST 2014
by Michael Peter Christen
added an option to set 'obey nofollow' for links with rel="nofollow"
attribute in the <a> tag for each crawl. This introduces a lot of
changes because it extends the usage of the AnchorURL Object type which
now also has a different toString method that the underlying
DigestURL.toString. It is therefore not advised to use .toString at all
for urls, just just toNormalform(false) instead.
Changed Files: defaults/yacy.init, htroot/BlacklistTest_p.java, htroot/CrawlStartExpert.html, htroot/CrawlStartExpert.java, htroot/Crawler_p.java, htroot/QuickCrawlLink_p.java, htroot/ViewFile.java, htroot/api/getpageinfo.java, htroot/api/getpageinfo_p.java, source/net/yacy/cora/document/id/AnchorURL.java, source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/cora/federate/solr/responsewriter/HTMLResponseWriter.java, source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/crawler/data/Cache.java, source/net/yacy/crawler/data/CrawlProfile.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/crawler/retrieval/HTTPLoader.java, source/net/yacy/crawler/retrieval/Request.java, source/net/yacy/crawler/retrieval/SitemapImporter.java, source/net/yacy/data/BookmarkHelper.java, source/net/yacy/data/ymark/YMarkCrawlStart.java, source/net/yacy/document/Document.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/ImageEntry.java, source/net/yacy/document/parser/rdfa/impl/RDFaParser.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/DocumentIndex.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/snippet/MediaSnippet.java, source/net/yacy/search/snippet/ResultEntry.java, source/net/yacy/server/http/HTTPDProxyHandler.java, test/net/yacy/search/snippet/TextSnippetTest.java
Fri Jul 11 04:04:09 CEST 2014
by orbiter
Renamed the transmission cloud to buffer in dispatcher since the name
'cloud' was a bad idea. Changed also the accumulation process for peer
targets so that every dht chunk is not assigned the set of redundant
targets but they are assigned to redundant targets individually. This
enhances the granularity of the target accumulation and should enhance
the efficiency of the process. Finally the dht protocol client was
enriched with the ability to remove the 'accept remote index' flag from
peers or remove peers completely if they do not answer at all.
Changed Files: htroot/IndexControlRWIs_p.java, source/net/yacy/peers/DHTSelection.java, source/net/yacy/peers/Dispatcher.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/Transmission.java, source/net/yacy/search/Switchboard.java
Tue Jul 01 16:39:00 CEST 2014
by Michael Peter Christen
update to solr 4.9.0
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, defaults/solr/solrconfig.xml, lib/lucene-analyzers-common-4.9.0.jar, lib/lucene-analyzers-phonetic-4.9.0.jar, lib/lucene-classification-4.9.0.jar, lib/lucene-codecs-4.9.0.jar, lib/lucene-core-4.9.0.jar, lib/lucene-facet-4.9.0.jar, lib/lucene-grouping-4.9.0.jar, lib/lucene-highlighter-4.9.0.jar, lib/lucene-join-4.9.0.jar, lib/lucene-memory-4.9.0.jar, lib/lucene-misc-4.9.0.jar, lib/lucene-queries-4.9.0.jar, lib/lucene-queryparser-4.9.0.jar, lib/lucene-spatial-4.9.0.jar, lib/lucene-suggest-4.9.0.jar, lib/solr-core-4.9.0.License, lib/solr-core-4.9.0.jar, lib/solr-solrj-4.9.0.License, lib/solr-solrj-4.9.0.jar, source/net/yacy/search/index/Fulltext.java
Thu Jun 26 13:02:35 CEST 2014
by Michael Peter Christen
enhanced metadata enrichment for media file type search:
- Web servers may now deliver YaCy-specific http header field with a
title and keywords. The new http header fields are:
X-YaCy-Media-Title - to be used for media (image, audio, video) titles
X-YaCy-Media-Keywords - to be used for media (image, audio, video)
- both fields are written to document fields title and keywords and are
searched also during image search.
- to make the usage of arbitrary http header fields (including this new
fields) possible in the /api/push_p.json servlet, a new POST argument is
also introduced to push http header fields. The new POST attribute is
named "responseHeader-X" (where X is the counter). It is allowed to use
this attribute as multi-attribute several times, each can be filled with
a http header line.
- see /api/push_p.html for examples
Changed Files: htroot/api/push_p.html, htroot/api/push_p.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/search/schema/CollectionConfiguration.java
Sun Jun 22 16:24:45 CEST 2014
by sixcooler
bump to httpclient-4.3.4
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/dependencies.txt, lib/httpclient-4.3.4.License, lib/httpclient-4.3.4.jar, lib/httpmime-4.3.4.License, lib/httpmime-4.3.4.jar, nbproject/project.xml, pom.xml
Sun Jun 22 00:21:47 CEST 2014
by reger
update to Jetty 9.2.1
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/jetty-9.2.1.v20140609.License, lib/jetty-client-9.2.1.v20140609.jar, lib/jetty-continuation-9.2.1.v20140609.jar, lib/jetty-deploy-9.2.1.v20140609.jar, lib/jetty-http-9.2.1.v20140609.jar, lib/jetty-io-9.2.1.v20140609.jar, lib/jetty-jmx-9.2.1.v20140609.jar, lib/jetty-proxy-9.2.1.v20140609.jar, lib/jetty-security-9.2.1.v20140609.jar, lib/jetty-server-9.2.1.v20140609.jar, lib/jetty-servlet-9.2.1.v20140609.jar, lib/jetty-servlets-9.2.1.v20140609.jar, lib/jetty-util-9.2.1.v20140609.jar, lib/jetty-webapp-9.2.1.v20140609.jar, lib/jetty-xml-9.2.1.v20140609.jar, nbproject/project.xml, pom.xml
Sun Jun 15 12:38:30 CEST 2014
by orbiter
Added filter query options to the ranking servlet /RankingSolr_p.html.
Filter queries are not actually related to ranking, but user requests
have pointed out that specific boost queries to move results to the end
of the result list are not sufficient. Such boost filters may be better
executed as actual filter and therefore such a filter can now be
statically applied to every search request. A typical use could be the
expression "http_unique_b:true AND www_unique_b:true" which uses the
recently introduced fields http_unique_b and www_unique_b which are true
only for one of the alternatives with/without http(s) and with/without
prefix 'www.' in host names.
Changed Files: defaults/yacy.init, htroot/RankingSolr_p.html, htroot/RankingSolr_p.java, source/net/yacy/cora/federate/solr/Ranking.java, source/net/yacy/http/servlets/GSAsearchServlet.java, source/net/yacy/http/servlets/SolrSelectServlet.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java, source/net/yacy/search/query/QueryParams.java
Sun Jun 15 12:11:23 CEST 2014
by Michael Peter Christen
Added collection navigation:
The collection field (can be filled i.e. in Crawl Start) can be used to
add categories to YaCy index entries. The usage of that field was
restricted to solr searches and post argument filters as implemented in
commit f7571386a3367432648f7cbd6e4744556ff445a7.
This commit extends collections to a full navigation option in the
standard YaCy search interface. The field is not active by default but
can be activated easily in the /ConfigSearchPage_p.html servlet (just
check the 'Collection' facet field). Collections can now be used for (at
least) two purposes:
- to provide search tenants (through post argument collection)
- to provide self-made category navigation
Search requests may now have (independently from switched on or off
collection facet) a "collection:<collection-name>" modifier attached;
firthermore collection names may use disjunctions using the '|' pipe
symbol. For example, this is a valid search request:
www collection:user|proxy
Changed Files: defaults/yacy.init, htroot/ConfigPortal.java, htroot/ConfigSearchPage_p.html, htroot/ConfigSearchPage_p.java, htroot/yacy/search.java, htroot/yacysearch.java, htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java, htroot/yacysearchtrailer.json, htroot/yacysearchtrailer.xml, source/net/yacy/cora/federate/solr/responsewriter/OpensearchResponseWriter.java, source/net/yacy/cora/federate/solr/responsewriter/YJsonResponseWriter.java, source/net/yacy/http/servlets/SolrSelectServlet.java, source/net/yacy/search/query/QueryModifier.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java
Thu May 29 13:24:24 CEST 2014
by Michael Peter Christen
fixed several problems with postprocessing:
- unique-postprocessing was destroying results from other
postprocessings; removed cross-updates as they had been not necessary
- unique-postprocessing did not restrict on same protocol
- inefficient concurrent update cache was redesigned completely
- increased limits for concurrent blocking queues to prevent early
Changed Files: htroot/Vocabulary_p.java, source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java, source/net/yacy/cora/federate/solr/instance/InstanceMirror.java, source/net/yacy/cora/federate/solr/instance/RemoteInstance.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/HyperlinkGraph.java
Thu May 22 03:01:07 CEST 2014
by Michael Peter Christen
better handling of ranking parameters and new default values for date
navigation which is done using ranking in solr.
Changed Files: defaults/yacy.init, htroot/IndexControlRWIs_p.java, htroot/RankingSolr_p.html, htroot/yacysearchitem.java, source/net/yacy/cora/sorting/WeakPriorityBlockingQueue.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/snippet/ResultEntry.java
Wed May 21 22:48:41 CEST 2014
by Michael Peter Christen
migrated from pdfbox 1.8.4 to 1.8.5. They have a very long bugfix list
for that update:
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, htroot/api/linkstructure.java, lib/fontbox-1.8.5.License, lib/fontbox-1.8.5.jar, lib/jempbox-1.8.5.License, lib/jempbox-1.8.5.jar, lib/org.restlet.License, lib/pdfbox-1.8.5.License, lib/pdfbox-1.8.5.jar
Wed May 21 13:28:26 CEST 2014
by sixcooler
manual merge
Changed Files: lib/lucene-analyzers-common-4.8.1.jar, lib/lucene-analyzers-phonetic-4.8.1.jar, lib/lucene-classification-4.8.1.jar, lib/lucene-codecs-4.8.1.jar, lib/lucene-core-4.8.1.jar, lib/lucene-facet-4.8.1.jar, lib/lucene-grouping-4.8.1.jar, lib/lucene-highlighter-4.8.1.jar, lib/lucene-join-4.8.1.jar, lib/lucene-memory-4.8.1.jar, lib/lucene-misc-4.8.1.jar, lib/lucene-queries-4.8.1.jar, lib/lucene-queryparser-4.8.1.jar, lib/lucene-spatial-4.8.1.jar, lib/lucene-suggest-4.8.1.jar, lib/org.restlet.jar, lib/solr-core-4.8.1.License, lib/solr-core-4.8.1.jar, lib/solr-solrj-4.8.1.License, lib/solr-solrj-4.8.1.jar, lib/zookeeper-3.4.6.jar
Wed May 21 11:48:08 CEST 2014
by Michael Peter Christen
migration to Solr 4.8.1
This includes also an update to zookeeper 3.4.6 and a new library that
Solr initializes by default: org.restlet from
which is included in version 2.2.1 from may 6th 2014
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, defaults/solr/solrconfig.xml, defaults/yacy.logging, lib/lucene-analyzers-common-4.8.1.jar, lib/lucene-analyzers-phonetic-4.8.1.jar, lib/lucene-classification-4.8.1.jar, lib/lucene-codecs-4.8.1.jar, lib/lucene-core-4.8.1.jar, lib/lucene-facet-4.8.1.jar, lib/lucene-grouping-4.8.1.jar, lib/lucene-highlighter-4.8.1.jar, lib/lucene-join-4.8.1.jar, lib/lucene-memory-4.8.1.jar, lib/lucene-misc-4.8.1.jar, lib/lucene-queries-4.8.1.jar, lib/lucene-queryparser-4.8.1.jar, lib/lucene-spatial-4.8.1.jar, lib/lucene-suggest-4.8.1.jar, lib/org.restlet.jar, lib/solr-core-4.8.1.License, lib/solr-core-4.8.1.jar, lib/solr-solrj-4.8.1.License, lib/solr-solrj-4.8.1.jar, lib/zookeeper-3.4.6.jar
Tue May 20 21:50:16 CEST 2014
by orbiter
fixed generics warnings for generic array instantiation that appeared
after migration to Java 7
Changed Files: source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/cora/storage/ConcurrentARC.java, source/net/yacy/crawler/HostBalancer.java, source/net/yacy/crawler/HostQueue.java, source/net/yacy/data/Diff.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/index/RAMIndexCluster.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/util/MergeIterator.java, source/net/yacy/kelondro/util/StackIterator.java, source/net/yacy/peers/Dispatcher.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/HyperlinkEdges.java
Sun May 11 01:53:11 CEST 2014
by reger
update to Jetty 9
besides adjustments in code it makes the servlet settings in web.xml significant.
This applies to solr, gsa and proxy servlet. There is no longer a default setup in code during init (as jetty 9 checks for double definition).
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, htroot/ConfigAccounts_p.java, lib/javax.servlet-api-3.1.0.jar, lib/jetty-9.1.4.v20140401.License, lib/jetty-client-9.1.4.v20140401.jar, lib/jetty-continuation-9.1.4.v20140401.jar, lib/jetty-deploy-9.1.4.v20140401.jar, lib/jetty-http-9.1.4.v20140401.jar, lib/jetty-io-9.1.4.v20140401.jar, lib/jetty-jmx-9.1.4.v20140401.jar, lib/jetty-proxy-9.1.4.v20140401.jar, lib/jetty-security-9.1.4.v20140401.jar, lib/jetty-server-9.1.4.v20140401.jar, lib/jetty-servlet-9.1.4.v20140401.jar, lib/jetty-servlets-9.1.4.v20140401.jar, lib/jetty-util-9.1.4.v20140401.jar, lib/jetty-webapp-9.1.4.v20140401.jar, lib/jetty-xml-9.1.4.v20140401.jar, nbproject/project.xml, pom.xml, source/net/yacy/http/Jetty9HttpServerImpl.java, source/net/yacy/http/Jetty9YaCySecurityHandler.java, source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/http/servlets/YaCyProxyServlet.java, source/net/yacy/yacy.java

Jump to: YaCy Release 1.80 top / Other Changes

Tue Sep 16 14:53:30 CEST 2014
by Michael Peter Christen
fix for http://forum.yacy-websuche.de/viewtopic.php?p=30720#p30720
Changed Files: source/net/yacy/yacy.java
Mon Sep 15 13:03:08 CEST 2014
by Michael Peter Christen
fix for favicon
Changed Files: htroot/env/templates/metas.template
Sat Sep 13 17:02:26 CEST 2014
by Michael Peter Christen
changed error message, see http://mantis.tokeek.de/view.php?id=439
Changed Files: source/net/yacy/http/MonitorHandler.java
Sat Sep 13 16:26:40 CEST 2014
by Michael Peter Christen
fix for http://mantis.tokeek.de/view.php?id=467
Changed Files: htroot/opensearchdescription.java, htroot/opensearchdescription.xml
Sat Sep 13 00:27:39 CEST 2014
by reger
fix NPE in image search
Changed Files: source/net/yacy/search/query/SearchEvent.java
Fri Sep 12 16:37:01 CEST 2014
by Michael Peter Christen
enhanced image search: bugfixes and performance enhancements
Changed Files: htroot/ViewImage.java, htroot/yacysearchitem.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/query/SearchEvent.java
Wed Sep 10 14:24:03 CEST 2014
by Michael Peter Christen
fixes and speed updates for search process
Changed Files: source/net/yacy/crawler/retrieval/FTPLoader.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java
Wed Sep 10 13:19:08 CEST 2014
by Michael Peter Christen
fixed text in startup tray icon and added shutdown icon during shutdown
Changed Files: source/net/yacy/gui/Tray.java, source/net/yacy/search/Switchboard.java
Mon Sep 08 10:26:28 CEST 2014
by Michael Peter Christen
npe fix for non-tray users
Changed Files: source/net/yacy/gui/Tray.java
Fri Sep 05 23:29:09 CEST 2014
by Michael Peter Christen
very bad language detection hack fix hack
Changed Files: source/net/yacy/kelondro/util/ISO639.java
Mon Sep 01 11:08:35 CEST 2014
by orbiter
fixed wrong submenu title
Changed Files: htroot/env/templates/submenuCrawler.template
Thu Aug 28 16:28:06 CEST 2014
by orbiter
small fix to the welcome message
Changed Files: htroot/Status.html
Wed Aug 27 18:32:44 CEST 2014
by orbiter
another fix to de locale which fixes a html tag glitch
Changed Files: locales/de.lng
Wed Aug 27 18:13:06 CEST 2014
by orbiter
fix for translation mistakes
Changed Files: locales/de.lng
Wed Aug 27 00:50:25 CEST 2014
by orbiter
fix for de locale translation mistake
Changed Files: locales/de.lng
Wed Aug 27 00:16:50 CEST 2014
by orbiter
bugfix to latest commit
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Wed Aug 27 00:03:49 CEST 2014
by orbiter
addon to latest fix
Changed Files: source/net/yacy/crawler/HostBalancer.java
Wed Aug 27 00:03:24 CEST 2014
by orbiter
fix for browsing of file paths in Index Browser
Changed Files: htroot/HostBrowser.java
Wed Aug 13 00:59:11 CEST 2014
by orbiter
concurrency bugfix
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Wed Aug 06 12:55:18 CEST 2014
by Michael Peter Christen
fix for possible interruption of concurrent queries
Changed Files: source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java
Wed Aug 06 01:33:24 CEST 2014
by Michael Peter Christen
fix for subpath crawl filter
Changed Files: source/net/yacy/crawler/data/CrawlProfile.java
Tue Aug 05 22:23:52 CEST 2014
by Michael Peter Christen
fix for termination of all crawls
Changed Files: htroot/Crawler_p.java
Tue Aug 05 21:32:25 CEST 2014
by Michael Peter Christen
fix for crawler attribute maxdompages
Changed Files: source/net/yacy/crawler/data/CrawlProfile.java
Tue Aug 05 21:31:59 CEST 2014
by Michael Peter Christen
npe fix in crawler
Changed Files: source/net/yacy/crawler/HostBalancer.java
Mon Aug 04 12:58:39 CEST 2014
by Michael Peter Christen
toString fix
Changed Files: source/net/yacy/crawler/CrawlStacker.java
Fri Aug 01 12:04:15 CEST 2014
by Michael Peter Christen
fix for image alt attachment to AnchorURLs in html parser.
Changed Files: htroot/ViewFile.java, source/net/yacy/cora/document/id/AnchorURL.java, source/net/yacy/crawler/data/ResultImages.java, source/net/yacy/document/Document.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/ImageEntry.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/schema/WebgraphConfiguration.java
Tue Jul 29 22:02:11 CEST 2014
by Marc Nause
This hopefully fixes http://mantis.tokeek.de/view.php?id=424
Changed Files: build.xml
Tue Jul 29 11:18:45 CEST 2014
by orbiter
fix for malformed hostpath names in crawl balancer
Changed Files: source/net/yacy/crawler/HostBalancer.java, source/net/yacy/crawler/HostQueue.java
Thu Jul 24 00:04:46 CEST 2014
by Michael Peter Christen
toString() fix
Changed Files: source/net/yacy/crawler/robots/RobotsTxt.java
Wed Jul 23 17:54:31 CEST 2014
by orbiter
fix for file urls
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/crawler/retrieval/FileLoader.java, source/net/yacy/crawler/robots/RobotsTxt.java
Tue Jul 22 21:57:38 CEST 2014
by reger
fix localhost link to opensearchdescription.xml
Changed Files: source/net/yacy/cora/federate/solr/responsewriter/OpensearchResponseWriter.java
Tue Jul 22 18:24:10 CEST 2014
by Michael Peter Christen
fixed target_alt_t in webgraph
Changed Files: source/net/yacy/cora/document/id/AnchorURL.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/ImageEntry.java, source/net/yacy/search/schema/WebgraphConfiguration.java
Tue Jul 22 00:59:08 CEST 2014
by Michael Peter Christen
fix for condition for writing the webgraph
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Wed Jul 16 16:04:38 CEST 2014
by Michael Peter Christen
small bugfixes
Changed Files: source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java, source/net/yacy/document/parser/apkParser.java
Sun Jul 13 00:48:50 CEST 2014
by reger
fix button (clear list) text color in CrawlResults
Changed Files: htroot/CrawlResults.html
Fri Jul 11 17:34:41 CEST 2014
by Michael Peter Christen
added debugging of filter queries in thread dump thread names
Changed Files: source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/RemoteSolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java
Thu Jul 10 17:25:33 CEST 2014
by Michael Peter Christen
another fix for unique field computation
Changed Files: source/net/yacy/cora/federate/solr/SchemaConfiguration.java
Thu Jul 10 17:13:35 CEST 2014
by Michael Peter Christen
fix for processing of noindex flag in http header
Changed Files: source/net/yacy/cora/protocol/ResponseHeader.java, source/net/yacy/document/Document.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/schema/CollectionConfiguration.java
Thu Jul 10 15:40:38 CEST 2014
by Michael Peter Christen
fixed bugs in canonical, robots and title/description unique calculation
Changed Files: source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/search/schema/CollectionConfiguration.java
Thu Jul 10 01:59:29 CEST 2014
by Michael Peter Christen
fix for bad URL decoding
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/document/parser/html/CharacterCoding.java
Mon Jul 07 19:15:11 CEST 2014
by Michael Peter Christen
fixed double content postprocessing
Changed Files: defaults/solr.collection.schema, source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/CollectionSchema.java
Wed Jul 02 11:29:50 CEST 2014
by orbiter
fix for long periods in timeline
Changed Files: htroot/api/timeline_p.java, source/net/yacy/search/EventTracker.java
Tue Jul 01 16:06:01 CEST 2014
by Michael Peter Christen
refactoring of AccessTracker events & timeline fix
Changed Files: htroot/PerformanceSearch_p.java, htroot/api/timeline_p.java, htroot/api/timeline_p.xml, source/net/yacy/peers/graphics/ProfilingGraph.java, source/net/yacy/search/EventTracker.java, source/net/yacy/search/query/AccessTracker.java
Fri Jun 27 15:12:53 CEST 2014
by Michael Peter Christen
fixed external link
Changed Files: htroot/CrawlStartExpert.html
Thu Jun 26 12:57:01 CEST 2014
by Michael Peter Christen
enhanced debugging
Changed Files: source/net/yacy/cora/protocol/TimeoutRequest.java, source/net/yacy/crawler/HostBalancer.java
Thu Jun 26 12:56:33 CEST 2014
by Michael Peter Christen
bugfix for image search
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/crawler/retrieval/Response.java, source/net/yacy/search/query/QueryGoal.java, source/net/yacy/search/query/SearchEvent.java
Thu Jun 26 11:58:07 CEST 2014
by Michael Peter Christen
increased fixed limit for http POST request sizes to 100MB
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Fri Jun 20 13:54:18 CEST 2014
by Michael Peter Christen
fixed and enhanced Base64 (en)coder (again)
Changed Files: source/net/yacy/cora/order/Base64Order.java
Tue Jun 17 22:54:47 CEST 2014
by Marc Nause
Improvements and bugfixes for recording actions of blacklist API.
Changed Files: htroot/Blacklist_p.java, htroot/api/blacklists/add_entry_p.java, htroot/api/blacklists/delete_entry_p.java
Tue Jun 17 11:36:20 CEST 2014
by Michael Peter Christen
fix for broken json
Changed Files: htroot/yacysearchtrailer.json
Sun Jun 15 22:50:04 CEST 2014
by orbiter
fix for crawl start check
Changed Files: htroot/CrawlStartSite.html, htroot/api/getpageinfo.java, htroot/js/IndexCreate.js
Wed Jun 04 12:28:28 CEST 2014
by Michael Peter Christen
NPE fix for postprocessing without term index
Changed Files: source/net/yacy/search/ResourceObserver.java
Wed Jun 04 12:07:07 CEST 2014
by Michael Peter Christen
fix for bad String conversion
Changed Files: htroot/IndexDeletion_p.java
Mon Jun 02 15:06:38 CEST 2014
by Michael Peter Christen
fixed webgraph postprocessing and status display in Crawler_p servlet
Changed Files: htroot/api/status_p.java, source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/WebgraphConfiguration.java
Fri May 23 00:49:50 CEST 2014
by Michael Peter Christen
fixed dependency in debian package on java 7
Changed Files: debian/control
Thu May 22 15:01:58 CEST 2014
by orbiter
fix appearance of image search thumbnails.
Changed Files: htroot/ViewImage.java, htroot/yacysearchitem.html, htroot/yacysearchitem.java
Wed May 21 18:20:16 CEST 2014
by Michael Peter Christen
debug code for index distribution testing
Changed Files: htroot/IndexControlRWIs_p.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/peers/Protocol.java, source/net/yacy/search/index/Fulltext.java
Wed May 21 15:31:47 CEST 2014
by Michael Peter Christen
fixed bug in test function
Changed Files: htroot/IndexControlRWIs_p.java
Wed May 21 09:28:23 CEST 2014
by orbiter
fix for latest generic warning fix
Changed Files: source/net/yacy/peers/Dispatcher.java
Sat May 17 22:56:15 CEST 2014
by reger
fix: allow enable of CrawlStartExpert.html #file
Changed Files: htroot/CrawlStartExpert.html
Sun May 11 04:39:59 CEST 2014
by reger
fix xml validation error on defaults/web.xml
Changed Files: defaults/web.xml

Other Changes   
Jump to: YaCy Release 1.80 top / Bugfixes

Tue Sep 16 16:02:45 CEST 2014
by Michael Peter Christen
Release 1.80
Changed Files: build.properties
Tue Sep 16 15:49:03 CEST 2014
by Michael Peter Christen
updated seedlist
Changed Files: defaults/yacy.network.freeworld.unit
Tue Sep 16 14:41:04 CEST 2014
by Michael Peter Christen
replaced warning with info
Changed Files: source/net/yacy/crawler/CrawlStacker.java
Tue Sep 16 13:41:01 CEST 2014
by Michael Peter Christen
more enhancements to image search in case that a restriction to a single
domain is done
Changed Files: source/net/yacy/cora/protocol/Domains.java, source/net/yacy/peers/RemoteSearch.java, source/net/yacy/search/query/QueryGoal.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java
Tue Sep 16 00:13:26 CEST 2014
by Michael Peter Christen
hack to get more results when doing a remote site search
Changed Files: source/net/yacy/peers/Protocol.java, source/net/yacy/search/query/SearchEvent.java
Mon Sep 15 23:29:42 CEST 2014
by reger
modifiy description for Field-Reindex to act only on local index
Changed Files: htroot/IndexReIndexMonitor_p.html, locales/de.lng
Mon Sep 15 20:33:22 CEST 2014
by Michael Peter Christen
increaesed ability ot search for all images on a single server within
the p2p remote search
Changed Files: htroot/yacysearch.java, source/net/yacy/peers/DHTSelection.java, source/net/yacy/peers/RemoteSearch.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/query/QueryGoal.java
Mon Sep 15 14:00:41 CEST 2014
by Michael Peter Christen
better contentdom abstraction
Changed Files: source/net/yacy/peers/Protocol.java, source/net/yacy/peers/RemoteSearch.java
Mon Sep 15 13:50:40 CEST 2014
by Michael Peter Christen
changed default value for maximum number of connections to 50
Changed Files: source/net/yacy/cora/protocol/ConnectionInfo.java
Mon Sep 15 13:38:53 CEST 2014
by Michael Peter Christen
automatically add a wild card if only a search on a single domain is
done. This makes it possible to search all documents on a single domain
even if no search word is given. This is in particular interesting when
searching for all images on a single domain.
Changed Files: htroot/yacysearch.java
Mon Sep 15 12:48:58 CEST 2014
by Michael Peter Christen
removed www and welcome servlet, these had been demo servlets and are
not needed any more
Changed Files: source/net/yacy/yacy.java
Sun Sep 14 23:26:22 CEST 2014
by reger
implement a forward to remote peer link in P2P Network list
Most links in Network.html are only available with transparent proxy = on, which is switched off by default,
to make the provided links useable in default setup a small forward servlet added (goto_p.java),
which takes the peer hash as parameter and forwards to current public ip (optional with path= parameter).
The servlet is protected ( _p ending) to assure forwarding works only for authorized YaCy users.
Changed Files: htroot/Network.html, htroot/goto_p.html, htroot/goto_p.java
Sun Sep 14 01:40:27 CEST 2014
by reger
add recommended link "self" to atom feed output
Changed Files: htroot/yacysearch.atom
Sun Sep 14 01:38:27 CEST 2014
by reger
limit OAI import urn resolver try for Deutsche National Library
The resolver service of National Library uses name space nbn, limit use of nbn-resolving.de accordingly to urn:nbn:
- add resolver for rfc's
Changed Files: source/net/yacy/document/content/DCEntry.java
Sat Sep 13 11:09:35 CEST 2014
by Michael Peter Christen
added a script which retrieves the public ip of the local peer and
prints it out to the terminal using the new servlet /yacy/seedlist.xml.
just call:
Changed Files: bin/myip.sh
Sat Sep 13 11:02:11 CEST 2014
by Michael Peter Christen
Added the option to retrieve only the own seed or a seleceted seed
(selected by peer-hash) in the /yacy/seedlist.[json/xml] api.
Added also a xml-version of the servlet.
The own seed can be retrieved i.e. with
and any other peer can be selected with

Changed Files: htroot/yacy/seedlist.java, htroot/yacy/seedlist.xml
Sat Sep 13 03:21:52 CEST 2014
by reger
- remove repositoryPath post from ConfigBasic (obsolete)
- remove static snippetComputationTime from ResultEntry (not used)
Changed Files: htroot/ConfigBasic.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/snippet/ResultEntry.java
Sat Sep 13 02:15:36 CEST 2014
by Michael Peter Christen
changed link to new tutorial repository (yes, Youtube..). The link does
not point to youtube directly to prevent that the referer to the peer
address is given to youtube. Instead, a forwarder address at yacy.net is
used to redirect to the tutorial repository (and can be changed later).
Changed Files: htroot/env/templates/header.template, htroot/env/templates/simpleheader.template
Fri Sep 12 23:11:25 CEST 2014
by malykhin.dmitry
Update russian translation
Changed Files: locales/ru.lng
Fri Sep 12 02:10:18 CEST 2014
by reger
- skip html template specific servlet post variables (show_xxx) for feeds,
- add <updated> (in required format) to atom feed
Changed Files: htroot/yacysearch.java, htroot/yacysearchitem.atom, htroot/yacysearchitem.java
Fri Sep 12 00:08:22 CEST 2014
by reger
adding totalResults and id to atom feed output
Changed Files: htroot/yacysearch.atom, htroot/yacysearchitem.atom
Thu Sep 11 11:34:17 CEST 2014
by malykhin.dmitry
Update russian translation
Changed Files: locales/ru.lng
Thu Sep 11 03:40:57 CEST 2014
by malykhin.dmitry
Update russian translation
Changed Files: locales/ru.lng
Wed Sep 10 00:52:37 CEST 2014
by reger
use configured search items per page if not specified in  post
- remove verify=cacheonly from admin screen search box to use the configured values
  (otherwise definition overwrites configured behavior and is used for following searches what might give unexpected/confusing different results compared to using /yacysearch )
Changed Files: htroot/env/templates/header.template, htroot/yacyinteractive.java, htroot/yacysearch.java
Mon Sep 08 12:20:43 CEST 2014
by malykhin.dmitry
Update russian translation
Changed Files: locales/ru.lng
Mon Sep 08 11:36:28 CEST 2014
by Michael Peter Christen
using htroot/api/blacklists as source folder: removed package
declaration of some classes in that folder
Changed Files: .classpath, htroot/api/blacklists/add_entry_p.java, htroot/api/blacklists/delete_entry_p.java, htroot/api/blacklists/get_list_p.java, htroot/api/blacklists/get_metadata_p.java
Mon Sep 08 05:00:30 CEST 2014
by reger
changed translation of submenuCustomization.template to submenuDesign.template
- removed obsolete scroogle translation
Changed Files: locales/cn.lng, locales/de.lng, locales/ru.lng, locales/uk.lng
Mon Sep 08 00:21:29 CEST 2014
by orbiter
replaced font for boot tray icon with image and added some more images
for further tray icon displays
Changed Files: addon/progress_booting.png, addon/progress_peertopeer.png, addon/progress_search.png, addon/progress_shutdown.png, addon/searchicon.png, source/net/yacy/gui/Tray.java
Sun Sep 07 12:28:19 CEST 2014
by orbiter
speed-up of start process if remote DNS waits for timeout
Changed Files: source/net/yacy/http/AbstractRemoteHandler.java
Sun Sep 07 12:04:35 CEST 2014
by orbiter
added an animation of the tray icon during the boot phase of YaCy.
Additionally, there is a tooltip and a new headline at the tray menu
which states the current booting status.
Changed Files: addon/progressbar.png, source/net/yacy/gui/Tray.java
Fri Sep 05 14:57:40 CEST 2014
by Michael Peter Christen
adding the buffer size is not wrong but may cause confusing information
when the buffer is cleaned after a buffer flush which is not then
available in Solr since that is waiting for a commit. In such cases the
counter would run backwards which is prevented by ignoring the buffer
Changed Files: source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java
Fri Sep 05 14:50:22 CEST 2014
by Michael Peter Christen
changed strategy to count the number of documents: get the max of
solr+buffer and the hit cache. This shall help during first crawls to
see a running document counter even if there was no commit meanwhile to
solr. To support that strategy, the hit cache must be written earlier.
Changed Files: source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java
Fri Sep 05 14:46:11 CEST 2014
by Michael Peter Christen
set the correct fail time in error docs
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Fri Sep 05 12:50:56 CEST 2014
by malykhin.dmitry
Update russian translation
Changed Files: locales/ru.lng
Thu Sep 04 23:17:41 CEST 2014
by malykhin.dmitry
Update russian translation
Changed Files: locales/ru.lng
Thu Sep 04 18:17:38 CEST 2014
by Michael Peter Christen
removing the [] for ipv6 addresses may be a bad idea..
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Thu Sep 04 09:41:42 CEST 2014
by orbiter
enhanced the apk parser (up to beeing production-ready).
The parser is not yet activated and will be after the next release step.
Changed Files: source/net/yacy/document/parser/apkParser.java
Tue Sep 02 13:25:30 CEST 2014
by Michael Peter Christen
removed solr warning during startup
Changed Files: defaults/yacy.logging
Tue Sep 02 13:22:12 CEST 2014
by Michael Peter Christen
changed method to write the citation index: do not catch up references
during document parsing; instead use the same references that would also
be written into the webgraph. That should cause that the webgraph and
the citation index express the exact same semantic.
Changed Files: source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/schema/CollectionConfiguration.java
Tue Sep 02 13:15:19 CEST 2014
by Michael Peter Christen
fixed localhost authorization and replaced the adminRealm with an info
string which is visible in the browser. That makes it possible that the
browser instructs the user how to change a forgotten admin password
(during runtime).
Changed Files: defaults/yacy.init, htroot/ConfigAccounts_p.java, source/net/yacy/http/Jetty9YaCySecurityHandler.java, source/net/yacy/http/YaCyLoginService.java
Tue Sep 02 12:06:56 CEST 2014
by Michael Peter Christen
moved index administration up ahead of system administration to put more
importance on it. People should not feel that it is more important to
tweak any settings (which may break things) than to look into the index.
Changed Files: htroot/env/templates/header.template
Tue Sep 02 12:04:44 CEST 2014
by Michael Peter Christen
moved Table administration and Busy Queues Config out of mini-submenu of
advanced settings to a top-menu entry. Moved the advanced setting to a
less prominent place of the submenu. Removed the table administration
from target analysis submenu because it appeared double, the table
administration is now the default in the system administration.
Sorry for inconvenience if i constantly move menues around, but this
makes just more sense and YaCy is still not finished :)
Changed Files: htroot/Settings_p.html, htroot/Tables_p.html, htroot/env/templates/header.template, htroot/env/templates/submenuConfig.template, htroot/env/templates/submenuTargetAnalysis.template
Tue Sep 02 11:53:22 CEST 2014
by Michael Peter Christen
moved cookie monitoring to the network monitoring submenu
Changed Files: htroot/CookieMonitorIncoming_p.html, htroot/CookieMonitorOutgoing_p.html, htroot/Settings_p.html, htroot/env/templates/submenuAccessTracker.template
Mon Sep 01 13:01:30 CEST 2014
by orbiter
enhanced date-ordered ranking
Changed Files: defaults/yacy.init, source/net/yacy/search/Switchboard.java
Mon Sep 01 11:04:24 CEST 2014
by orbiter
switched position of API steering and content semantic
Changed Files: htroot/env/templates/header.template
Mon Sep 01 00:16:26 CEST 2014
by reger
fix Querygoal.parse exception on +/-null-term
covers http://mantis.tokeek.de/view.php?id=452
Changed Files: source/net/yacy/search/query/QueryGoal.java
Sun Aug 31 05:04:19 CEST 2014
by reger
exclude nav-header/footer  in ServerScannerList.html?embedded
fixes display of header in yacyinteractive.html
Changed Files: htroot/ServerScannerList.html
Sun Aug 31 00:33:05 CEST 2014
by reger
remove unused legacy procedure from httpserver
(deleted  generateSocketAddress(port) )
Changed Files: source/net/yacy/http/Jetty9HttpServerImpl.java, source/net/yacy/http/YaCyHttpServer.java
Thu Aug 28 12:50:45 CEST 2014
by Michael Peter Christen
bugfix for wrong lock display, see
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Thu Aug 28 12:15:11 CEST 2014
by orbiter
update to kaskelix
Changed Files: htroot/Status.html, htroot/Steering.html, htroot/env/grafics/kaskelix.png
Wed Aug 27 18:44:20 CEST 2014
by orbiter
added hint to the search api examples to remind the user that these
interfaces are examples and not actual variants of the current search
Changed Files: htroot/env/templates/simpleheader.template
Wed Aug 27 18:40:20 CEST 2014
by orbiter
added hint to the regular expression tester
Changed Files: htroot/CrawlStartExpert.html
Wed Aug 27 18:29:40 CEST 2014
by orbiter
changed style of info-icon to be similar to bootstrap.css glyphicons
Changed Files: htroot/env/grafics/i16.gif, htroot/env/grafics/i32.gif
Wed Aug 27 18:14:45 CEST 2014
by orbiter
another translation typo
Changed Files: locales/de.lng
Wed Aug 27 02:31:21 CEST 2014
by reger
add rss version to api/feed.rss
IE11 reports error without
Changed Files: htroot/api/feed.rss, htroot/api/feed.xml
Wed Aug 27 00:45:39 CEST 2014
by orbiter
do not remove the index deletion option from the IndexControlURLs_p.html
servlet after a deletion happend, instead show but disable the option
when the index is empty.
Changed Files: htroot/IndexControlURLs_p.html, htroot/IndexControlURLs_p.java
Wed Aug 27 00:31:03 CEST 2014
by orbiter
added a confirmation dialog to complete index deletion
Changed Files: htroot/IndexControlURLs_p.html
Wed Aug 27 00:22:00 CEST 2014
by orbiter
fix html error occurred by missing symbol in de localization in
Changed Files: locales/de.lng
Wed Aug 27 00:15:31 CEST 2014
by orbiter
enable Crawler_p servlet if a rss feed or a wiki dump import was
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Wed Aug 27 00:10:51 CEST 2014
by orbiter
do not push noindex errors into log if in intranet mode. noindex
attributes are attached to artificial constructed index.html files which
list directories. Such files are naturally rejected by the crawler and
should not appear in the error log because these files are part of the
construction of file crawlers and confuse users if they see them in the
error log.
Changed Files: source/net/yacy/search/Switchboard.java
Tue Aug 26 22:47:13 CEST 2014
by Marc Nause
Improved UPnP. (still not perfect)

*) set HTTPS port if enabled
*) improved data structures (may not be final)
*) moved UPnP to own package
Changed Files: htroot/ConfigBasic.java, source/net/yacy/search/Switchboard.java, source/net/yacy/utils/upnp/UPnP.java, source/net/yacy/utils/upnp/UPnPMapping.java, source/net/yacy/utils/upnp/UPnPMappingType.java
Tue Aug 26 19:02:53 CEST 2014
by orbiter
removed the HTDOCS repository concept because the concept to host files
on the YaCy http server is obsolete; YaCy can index file:// and smb://
Changed Files: defaults/yacy.init, htroot/ConfigBasic.html, htroot/ConfigBasic.java, source/net/yacy/server/http/HTTPDFileHandler.java
Tue Aug 26 18:52:49 CEST 2014
by orbiter
renamed Host Browser to Index Browser (gives a better hint what it
actually does)
Changed Files: htroot/HostBrowser.html, htroot/env/templates/header.template, htroot/env/templates/simpleheader.template, htroot/env/templates/submenuWebStructure.template, locales/de.lng, locales/ru.lng, locales/uk.lng
Tue Aug 26 01:02:38 CEST 2014
by reger
this makes it easier to config exception (for private networks),
like   blacklist= .*
        whitelist= 10\..*,127\..* .....     allows only listed ip pattern
Changed Files: defaults/yacy.network.readme, htroot/yacysearch.java
Mon Aug 25 01:25:22 CEST 2014
by reger
change seedfile upload ip check to allow intranet ip in intranet mode
- this allows to setup a principal peer in intranet environment
Changed Files: source/net/yacy/peers/Network.java
Sun Aug 24 23:05:22 CEST 2014
by reger
update target url for yacy-portalsearch.html to search.yacy.net (peer yacportalsearch2014)
(old www.yacy-suche.de not reacheable)
Changed Files: htroot/portalsearch/yacy-portalsearch.html
Sun Aug 24 21:13:36 CEST 2014
by reger
- update javadoc Seed.getIP
- default mySeed.ip to hostip in SeedDB.initMySeed() if Intranetmode
this allows to become senior status in intranet hosted search network with view peers,
otherwise peer would stay junior because of default init with loopback ip as public (dna) ip.
Changed Files: source/net/yacy/peers/Seed.java, source/net/yacy/peers/SeedDB.java
Sun Aug 24 05:20:06 CEST 2014
by reger
in IntranetMode allow intranet hosted seedlist with Network_Domain "any"
- so far intranet seedlist hosts are always denied but need to be allowed in intranet mode
Changed Files: source/net/yacy/peers/Seed.java
Sat Aug 23 19:05:31 CEST 2014
by sixcooler
germal locale correction
Changed Files: locales/de.lng
Sat Aug 23 10:35:38 CEST 2014
by orbiter
make sure that the postprocessing background thread never dies by any
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Thu Aug 21 01:26:42 CEST 2014
by reger
seedUploadUrl config is lost on restart if no publish event occured
-add a saveMySeed() on uploadurl changes (to keep url setting without retyping even if network down)
Changed Files: htroot/SettingsAck_p.java
Wed Aug 20 00:55:31 CEST 2014
by orbiter
added 0x40 to image of lockopen-gif image palette (light grey)
Changed Files: htroot/env/grafics/lockopen.gif
Wed Aug 20 00:48:48 CEST 2014
by orbiter
changed lock/unlock image design
Changed Files: htroot/env/grafics/lock.gif, htroot/env/grafics/lockopen.gif
Mon Aug 18 02:53:03 CEST 2014
by reger
set default "Search Interfaces"."Solr RSS/Opensearch" query to show latest 10 addition to index
Changed Files: htroot/env/templates/simpleheader.template
Mon Aug 18 00:05:52 CEST 2014
by reger
update Netbeans IDE classpath
Changed Files: nbproject/project.xml
Mon Aug 18 00:04:55 CEST 2014
by reger
update to commons-compress-1.8.1
(bugfix release)
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/commons-compress-1.8.1.License, lib/commons-compress-1.8.1.jar, pom.xml
Mon Aug 18 00:03:24 CEST 2014
by reger
reduce mem buffer overhead for gap files during  r/w
(they are typically small compared to idx allowing to use smaller buffersize -> set to 16k records)
Changed Files: source/net/yacy/kelondro/blob/Gap.java
Sat Aug 16 14:29:52 CEST 2014
by reger
allow single quote " ' " in query
see http://mantis.tokeek.de/view.php?id=379
-add QueryGoal test case for this
Changed Files: source/net/yacy/search/query/QueryGoal.java, test/net/yacy/search/query/QueryGoalTest.java
Sat Aug 16 03:37:36 CEST 2014
by reger
fix: hanging text in input fields of WatchWebStructure_p.html
in IE11
Changed Files: htroot/WatchWebStructure_p.html
Fri Aug 15 01:17:11 CEST 2014
by orbiter
snippet computation update
Changed Files: source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/search/snippet/TextSnippet.java
Fri Aug 15 00:06:22 CEST 2014
by orbiter
reverted template from latest cherry-picked commit
Changed Files: htroot/env/templates/submenuCrawler.template
Wed Aug 13 16:03:40 CEST 2014
by Alex
updated some french translations
Changed Files: htroot/env/templates/submenuCrawler.template, locales/fr.lng
Thu Aug 14 03:06:41 CEST 2014
by reger
fill solr rss writer dc:subject tag with keyword content
Changed Files: source/net/yacy/cora/federate/solr/responsewriter/OpensearchResponseWriter.java
Wed Aug 13 00:39:25 CEST 2014
by Alexandre Vouilloz
updated fr traduction of ConfigBasic.html and header.template + fixed some accents in fr.lng
Changed Files: locales/fr.lng
Mon Aug 11 23:53:12 CEST 2014
by orbiter
removed warnings
Changed Files: source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/cora/util/JSONException.java, source/net/yacy/document/Document.java
Mon Aug 11 21:46:39 CEST 2014
by reger
add type attribute to atom feed <link> tag
(for /yacysearch.atom)
Changed Files: htroot/yacysearchitem.atom, htroot/yacysearchitem.java
Mon Aug 11 20:05:15 CEST 2014
by orbiter
reverted .classpath from cherry-picked
Changed Files: .classpath
Mon Aug 11 00:44:52 CEST 2014
by Alexandre Vouilloz
updated the traduction of Status_p.inc in the french locale
Changed Files: .classpath, locales/fr.lng
Mon Aug 11 04:29:43 CEST 2014
by reger
fix test case MultiProtocolURL.toString()
(only allowed on AnchorURL)
Changed Files: test/net/yacy/cora/document/id/DigestURLTest.java, test/net/yacy/cora/document/id/MultiProtocolURLTest.java
Sun Aug 10 22:35:25 CEST 2014
by reger
eliminate lat/lon type conversion in osm
(define as double)
Changed Files: htroot/osm.java
Sun Aug 10 03:18:15 CEST 2014
by reger
reduce doublecontent check document (prevent out of memory)
see http://mantis.tokeek.de/view.php?id=437

test result (concurrency=7)
2000 docs = eom always
1000 docs = eom always
100 docs = eom never

chosen -> 200 docs (eom not encountered during test with 1GB mem setting)
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Fri Aug 08 00:02:59 CEST 2014
by reger
catch NPE during crawl (OAI import)
- condenseDocument mime=null (allowed)
- collectionconfiguration responseheader = null (allowed) 
Changed Files: source/net/yacy/cora/document/analysis/Classification.java, source/net/yacy/search/schema/CollectionConfiguration.java
Wed Aug 06 23:04:42 CEST 2014
by reger
center searchevent lines on network graph
Changed Files: source/net/yacy/peers/graphics/NetworkGraph.java
Wed Aug 06 12:40:35 CEST 2014
by Michael Peter Christen
increased default logging line count to max
Changed Files: htroot/ViewLog_p.java
Wed Aug 06 12:36:59 CEST 2014
by Michael Peter Christen
reducing the concurrent query stack size and reduced concurrency of
postprocessing to avoid OOM situations
Changed Files: source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/peers/Protocol.java, source/net/yacy/search/schema/CollectionConfiguration.java
Wed Aug 06 12:35:12 CEST 2014
by Michael Peter Christen
bugfix for crawler double-check: if an url is redirected, the
redirect-target was not double-checked. This is now done by replacing
the redirect-URL on the crawl queue again (where it is double-checked)
Changed Files: source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/crawler/retrieval/HTTPLoader.java
Tue Aug 05 21:31:28 CEST 2014
by Michael Peter Christen
added hint to port forwarding videos
Changed Files: htroot/ConfigBasic.html
Tue Aug 05 01:27:27 CEST 2014
by reger
fix ConfigAccounts del user with uppercase letter in name
(usernames are case sensitive, userdb.delete used toLower) 
Changed Files: source/net/yacy/data/UserDB.java
Mon Aug 04 16:13:39 CEST 2014
by Michael Peter Christen
Changed Files: source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/search/Switchboard.java
Mon Aug 04 15:28:58 CEST 2014
by Michael Peter Christen
added concurrency to postprocess rewrite process
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Mon Aug 04 14:44:42 CEST 2014
by Michael Peter Christen
log ppm instead of docs/second
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Mon Aug 04 14:44:20 CEST 2014
by Michael Peter Christen
set process type of web graph according to fields as defined in the
Changed Files: source/net/yacy/search/schema/WebgraphConfiguration.java
Mon Aug 04 14:19:37 CEST 2014
by Michael Peter Christen
log postprocessing constraints in case that postprocessing is not
Changed Files: source/net/yacy/search/Switchboard.java
Mon Aug 04 13:37:34 CEST 2014
by Michael Peter Christen
less logging
Changed Files: source/net/yacy/crawler/CrawlStacker.java
Mon Aug 04 13:00:45 CEST 2014
by Michael Peter Christen
better solr query logging to detect unnecessary sort requests for more
performance profiling
Changed Files: source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java
Mon Aug 04 12:59:38 CEST 2014
by Michael Peter Christen
no sorting if http/www unique fields are not demanded (makes query
faster) and some code restrucuring
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Mon Aug 04 08:29:35 CEST 2014
by Michael Peter Christen
more retries, less sleep
Changed Files: source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java
Mon Aug 04 02:38:58 CEST 2014
by reger
fix "Ljava.lang.String" in crawl queue anchor name
(e.g. IndexCreateQueues_p.html?stack=LOCAL with images in queue)
Changed Files: source/net/yacy/document/Document.java
Mon Aug 04 00:03:42 CEST 2014
by reger
catch IndexControlRWIs_p error if RWI not connected
Changed Files: htroot/IndexControlRWIs_p.java
Sun Aug 03 22:44:22 CEST 2014
by orbiter
added new classes which shall reduce call overhead to Solr (stub)
Changed Files: source/net/yacy/cora/federate/solr/logic/Conjunction.java, source/net/yacy/cora/federate/solr/logic/DNF.java, source/net/yacy/cora/federate/solr/logic/Literal.java, source/net/yacy/search/schema/CollectionConfiguration.java
Sun Aug 03 20:09:23 CEST 2014
by reger
adjust rss lat/lon to double
(common format across other classes)
Changed Files: source/net/yacy/cora/document/feed/Hit.java, source/net/yacy/cora/document/feed/RSSMessage.java
Fri Aug 01 13:21:16 CEST 2014
by orbiter
removed unused images from webgraph edge computation
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/WebgraphConfiguration.java
Fri Aug 01 13:20:25 CEST 2014
by orbiter
target linktexts must be string to enable search facets on these fields
Changed Files: defaults/solr.webgraph.schema, htroot/api/schema.java, source/net/yacy/search/schema/WebgraphConfiguration.java, source/net/yacy/search/schema/WebgraphSchema.java
Fri Aug 01 12:15:14 CEST 2014
by Michael Peter Christen
do not store failure of loading of robots.txt into the index as a fail
Changed Files: source/net/yacy/crawler/robots/RobotsTxt.java, source/net/yacy/search/index/ErrorCache.java
Fri Aug 01 01:47:53 CEST 2014
by reger
- update Maven pom
- add ppt parser test case
Changed Files: pom.xml, test/net/yacy/document/ParserTest.java
Tue Jul 29 21:26:27 CEST 2014
by Marc Nause
Replaced old UPNP library with Weupnp. UPNP should
 work now, at least it does on my network. UPNP code in YaCy can still
be improved though (see TODO comment: make port on gateway configurable
or find free one).

*) removed old code
*) added new lib
*) changed code to work with new lib
Changed Files: .classpath, build.xml, lib/weupnp-0.1.2.jar, lib/weupnp.license, source/net/yacy/utils/UPnP.java
Tue Jul 29 18:49:42 CEST 2014
by orbiter
reverted commit f94c91315bce0fb5a328ce7204a9066ff457e9e3 because the
webgraph has not enough performance for that
Changed Files: source/net/yacy/search/index/Segment.java
Tue Jul 29 11:12:01 CEST 2014
by orbiter
yacystats is gone :(
Changed Files: defaults/yacy.network.allip.unit, defaults/yacy.network.freeworld.unit, defaults/yacy.network.intranet.unit, defaults/yacy.network.metager.unit, defaults/yacy.network.webportal.unit, htroot/env/templates/header.template, htroot/env/templates/simpleheader.template, htroot/yacy/ui/yacyui-admin.html
Mon Jul 28 15:42:57 CEST 2014
by Michael Peter Christen
enhanced snippet computation and test method in ViewFile
Changed Files: htroot/ViewFile.java, htroot/yacysearchitem.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/snippet/TextSnippet.java
Sun Jul 27 04:13:07 CEST 2014
by reger
move bootstrap peers test systems to its test class
var assignment not needed  elsewhere.
Changed Files: source/net/yacy/cora/federate/yacy/Peers.java
Sat Jul 26 00:38:53 CEST 2014
by reger
fix type in .init description
Changed Files: defaults/yacy.init
Sat Jul 26 00:17:41 CEST 2014
by reger
set .ini default boost fields to same as assigned by button "reset to default"
(in RankingSolr_p)
- fix typo http://mantis.tokeek.de/view.php?id=430
Changed Files: defaults/yacy.init
Fri Jul 25 18:53:13 CEST 2014
by orbiter
fix for the usage of ready-prepared solr queries, some queries are
formulated as edismax query but this was not set as query attribut. The
defType=edismax property needs a qf-field, so this was added as well. Do
not remove that field again! This fixes also a problem with title-unique
Changed Files: source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/schema/CollectionConfiguration.java
Thu Jul 24 15:35:53 CEST 2014
by Michael Peter Christen
if the webgraph is used, then use it also for reference computation to
avoid contradictions with references_i in the collection index.
Changed Files: source/net/yacy/search/index/Segment.java
Thu Jul 24 14:59:37 CEST 2014
by Michael Peter Christen
added a snippet test function in ViewFile: you can now search for a
specific word on the document; the servlet returns the snippet in the
same way as it would be shown in a search result.
Changed Files: htroot/ViewFile.html, htroot/ViewFile.java, source/net/yacy/crawler/robots/RobotsTxt.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/search/query/QueryGoal.java, source/net/yacy/search/schema/WebgraphConfiguration.java
Wed Jul 23 17:54:16 CEST 2014
by orbiter
no idea why the words max was an ordered one. This change increaes speed
dunring document processin a bit
Changed Files: source/net/yacy/document/Condenser.java
Tue Jul 22 22:01:01 CEST 2014
by reger
Merge origin/master
Changed Files: reconfigureYACY.sh
Tue Jul 22 21:54:46 CEST 2014
by reger
fix double &query parameter (index.html)
Changed Files: htroot/index.html
Tue Jul 22 21:52:53 CEST 2014
by Marc Nause
Updated commandline reconfiguration tool.

*) fixed "set HTTP port" (root cause was sloppy implementation of method
which gets values from config file)
*) added "set HTTPS port"
Changed Files: reconfigureYACY.sh
Tue Jul 22 00:23:17 CEST 2014
by Michael Peter Christen
changed handling of crawl profile field crawlingIfOlder: this should be
filled with the date, when the url is recognized as to be outdated. That
field was partly misinterpreted and the time interval was filled in. In
case that all the urls which are in the index shall be treated as
outdated, the field is filled now with Long.MAX_VALUE because then all
crawl dates are before that date and therefore outdated.
Changed Files: htroot/Crawler_p.java, htroot/QuickCrawlLink_p.java, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/crawler/data/CrawlProfile.java, source/net/yacy/search/Switchboard.java
Mon Jul 21 22:13:34 CEST 2014
by reger
improve logexception info
- log a message or class name insted of msgtxt "null"
Changed Files: source/net/yacy/cora/util/ConcurrentLog.java
Mon Jul 21 19:26:12 CEST 2014
by reger
fix bookmark.rss
- channel end tag postion
- link with html entity
Changed Files: htroot/Bookmarks.java, htroot/Bookmarks.rss
Mon Jul 21 15:58:29 CEST 2014
by orbiter
reverted toString() change in AnchorURL to prevent mistakenly used
toString(). This fixes also the update link bug.
Changed Files: htroot/ConfigUpdate_p.java, source/net/yacy/cora/document/id/AnchorURL.java
Mon Jul 21 02:00:05 CEST 2014
by reger
Add link to RemoteCrawl_p
suggestion http://mantis.tokeek.de/view.php?id=277
Changed Files: htroot/CrawlResults.html, htroot/RemoteCrawl_p.html
Mon Jul 21 00:47:14 CEST 2014
by reger
Add Solr default query fields (&qf) to select servlet
according to the ranking profiles boost fields defined by the peer (if df/qf is not specified in query).
This allows for pretty simple queries ( q=word) without the need to know about the specific index configuration.
Making sure all relevant fields (as determined by the index owner) are searched, still maintaining the option to query specific fields
and does not relay on the duplication of text to text_t.
- add author to reset-default boost fields (support results for author nav)
Changed Files: htroot/RankingSolr_p.java, source/net/yacy/http/servlets/SolrSelectServlet.java
Mon Jul 21 00:04:54 CEST 2014
by reger
prevent array out of bound exception on getRankingProfile(x)
on faulty &profileNr=  query parameter 
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Sun Jul 20 12:28:59 CEST 2014
by reger
fix input boostfield factor of 0.0 in RankingSolr
- input was accepted and stored but not editeable (added check factor >0.0 during edit)
- make use of some more predefined solr constants
Changed Files: htroot/RankingSolr_p.java
Sun Jul 20 02:49:49 CEST 2014
by reger
remove unused static (RSS)search from protocol
Changed Files: source/net/yacy/peers/Protocol.java
Sun Jul 20 00:06:22 CEST 2014
by reger
use CRLF in generated bat command scripts for windows
- for easier viewing with standard viewers
Changed Files: source/net/yacy/peers/operation/yacyRelease.java
Wed Jul 16 18:13:29 CEST 2014
by Michael Peter Christen
do not write CR values to webgraph if no CR values are computed
Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java
Wed Jul 16 16:04:19 CEST 2014
by Michael Peter Christen
added option to delete documents from the webgraph
Changed Files: htroot/IndexDeletion_p.html, htroot/IndexDeletion_p.java
Wed Jul 16 14:57:51 CEST 2014
by Michael Peter Christen
added SolrCloud access mode and configuration
Changed Files: htroot/IndexFederated_p.html, htroot/IndexFederated_p.java, source/net/yacy/cora/federate/solr/connector/ShardSelection.java, source/net/yacy/cora/federate/solr/instance/ServerShard.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Fulltext.java
Wed Jul 16 14:57:25 CEST 2014
by Michael Peter Christen
enhanced postprocessing status report
Changed Files: htroot/api/status_p.java, source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/search/schema/CollectionConfiguration.java
Sun Jul 13 00:50:16 CEST 2014
by malykhin.dmitry
Update russian translation
Changed Files: locales/ru.lng
Sun Jul 13 00:11:17 CEST 2014
by reger
fix GSA filter query assignment
- use more parameter constants
Changed Files: source/net/yacy/cora/federate/solr/responsewriter/GSAResponseWriter.java, source/net/yacy/http/servlets/GSAsearchServlet.java
Fri Jul 11 19:52:25 CEST 2014
by Michael Peter Christen
removed exist() retrieval functions from error cache and replaced it
with metadata retrieval from connectors directly. This should cause
better usage of the cache. Automatically increase the metadata cache if
more memory is available.
Changed Files: htroot/HostBrowser.java, source/net/yacy/cora/federate/solr/instance/InstanceMirror.java, source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/ErrorCache.java
Fri Jul 11 18:36:04 CEST 2014
by Michael Peter Christen
cleanup of checkAcceptanceInitially in CrawlStacker, should avoid
double-calling of solr
Changed Files: htroot/HostBrowser.java, source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/search/Switchboard.java
Fri Jul 11 18:15:35 CEST 2014
by Michael Peter Christen
reverted filter query hack, it did not work
Changed Files: source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java
Fri Jul 11 18:05:11 CEST 2014
by Michael Peter Christen
reduced number of solr queries during crawling
Changed Files: source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/search/index/ErrorCache.java, source/net/yacy/search/schema/CollectionConfiguration.java
Fri Jul 11 18:04:55 CEST 2014
by Michael Peter Christen
enhanced solr queries for single document extraction
Changed Files: source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java
Fri Jul 11 17:09:46 CEST 2014
by Michael Peter Christen
testing filter queries instead of queries to retrieve documents by id
Changed Files: source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java
Fri Jul 11 16:48:08 CEST 2014
by Michael Peter Christen
removed more unnecessary exist-checks in ErrorCache
Changed Files: source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/search/index/ErrorCache.java
Fri Jul 11 16:26:52 CEST 2014
by Michael Peter Christen
don't do a double check to solr for failed documents if they are not
written to solr
Changed Files: source/net/yacy/search/index/ErrorCache.java
Fri Jul 11 16:01:42 CEST 2014
by Michael Peter Christen
enhanced crawler host round-robin strategy
Changed Files: source/net/yacy/crawler/HostBalancer.java
Thu Jul 10 23:58:47 CEST 2014
by reger
catch input file not found in Mediawiki import
Changed Files: htroot/IndexImportMediawiki_p.java
Thu Jul 10 02:20:01 CEST 2014
by reger
cleanup older unused classes
Changed Files:
Thu Jul 10 02:17:56 CEST 2014
by reger
move startup time from old serverCore to switchboard (most used here)
to make servercore eventually obsolete.
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/peers/Network.java, source/net/yacy/search/Switchboard.java, source/net/yacy/server/serverCore.java
Thu Jul 10 02:14:03 CEST 2014
by reger
remove unused localSearchThread from SearchEvent
Changed Files: source/net/yacy/search/query/SearchEvent.java
Thu Jul 10 02:09:26 CEST 2014
by reger
add local ip to AbstractRemoteHandler local hostname cache
Changed Files: source/net/yacy/http/AbstractRemoteHandler.java
Mon Jul 07 13:37:17 CEST 2014
by Michael Peter Christen
added linkScraperParser, a parser which ignores the text like the
generic parser but extracts links like the htmlParser. This should be
used for ASCII documents without known text format annotation like
source code files or json documents. Probably also good for xml files
without known schema.
Changed Files: source/net/yacy/document/Document.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/linkScraperParser.java
Sun Jul 06 20:49:01 CEST 2014
by reger
fix seek error for 0 file size records file
by add extra check for file size = 0 in cleanlast()
- (http://mantis.tokeek.de/view.php?id=411)
Changed Files: source/net/yacy/kelondro/io/Records.java
Sun Jul 06 20:41:26 CEST 2014
by reger
add test case for Records (used in HostBalancer)
- simulating seek error (http://mantis.tokeek.de/view.php?id=411)
Changed Files: test/net/yacy/kelondro/io/RecordsTest.java
Sun Jul 06 16:04:30 CEST 2014
by reger
fix Crawler HostQueue release of  stackfile
- close stackfile inputstream at end of ChunkIterator
This should solve startup delay while unfinished crawl jobs exist (maybe also too many open file situation)
Changed Files: source/net/yacy/kelondro/table/ChunkIterator.java, source/net/yacy/kelondro/table/Table.java
Sun Jul 06 13:24:36 CEST 2014
by reger
fix NPE in Condenser,
discovered by calling IndexControlRWI, "Word Deletion" with "for every resolvable and deleted URL reference"
Changed Files: source/net/yacy/document/Condenser.java
Sun Jul 06 05:38:32 CEST 2014
by reger
update to JUnit 4.11
- fix build.xml -> parserTest error on Windows due to javac encoding
Changed Files: build.xml, libt/hamcrest-core-1.3.jar, libt/junit-4.11.jar, pom.xml, test/net/yacy/document/ParserTest.java, test/net/yacy/document/parser/htmlParserTest.java
Sun Jul 06 00:38:16 CEST 2014
by reger
adding test for HostQueue crawl stack
- simulating problem with zero length stack file (but not fixing it)
- adding test data clean to maven pom
Changed Files: pom.xml, test/net/yacy/crawler/HostQueueTest.java
Thu Jul 03 22:01:13 CEST 2014
by reger
make search result also avail. as atom feed via /yacysearch.atom
- fix logo in rss feed 
Changed Files: htroot/opensearchdescription.xml, htroot/yacysearch.atom, htroot/yacysearch.java, htroot/yacysearchitem.atom
Thu Jul 03 13:06:29 CEST 2014
by orbiter
timeline performance update
Changed Files: htroot/api/timeline_p.java, source/net/yacy/search/query/AccessTracker.java
Wed Jul 02 14:56:34 CEST 2014
by Michael Peter Christen
reverted latest change for reindexing because that works actually only
for internal Solr indexes. This is mainly caused by the fact that an
external Solr may be also a SolrCloud which do not support LukeRequests,
which are needed to request the old Schema.
Changed Files: htroot/IndexReIndexMonitor_p.java
Wed Jul 02 14:34:48 CEST 2014
by Michael Peter Christen
remove check for local solr. This check was made during a time when Solr
was optional and another alternative metadata store was available. Since
that store is now removed, Solr is always available (internally or
Changed Files: htroot/IndexReIndexMonitor_p.java, source/net/yacy/peers/Protocol.java
Tue Jul 01 23:24:26 CEST 2014
by reger
fix NPE in solr rss where snippet contains only the title text
and adjusted xslt, for solr snippets (&hl=true) to decode the xml encoded html <b> tag by adding disable-output-escaping
(still open item description may be double as dc: tag and rss.description tag)
Changed Files: htroot/yacysearch.xsl, pom.xml, source/net/yacy/cora/federate/solr/responsewriter/OpensearchResponseWriter.java
Tue Jul 01 16:18:12 CEST 2014
by Michael Peter Christen
update of commons-io and slf4j-api (as preparation for Solr 4.9.0)
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/commons-io-2.3.jar, lib/commons-io-2.3.license, lib/slf4j-api-1.7.6.jar
Mon Jun 30 05:36:48 CEST 2014
by reger
added test case for TextSnippet,
removed obsolete/unused parameter and reference to MediaSnippet
Changed Files: source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/snippet/ResultEntry.java, source/net/yacy/search/snippet/TextSnippet.java, test/net/yacy/search/snippet/TextSnippetTest.java
Sun Jun 29 22:26:22 CEST 2014
by Michael Peter Christen
no timeout for large reference collections
Changed Files: source/net/yacy/search/index/Segment.java
Sun Jun 29 02:54:09 CEST 2014
by reger
extract author and keywords in .doc and .ppt parser
Changed Files: source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/pptParser.java
Fri Jun 27 23:05:06 CEST 2014
by reger
enable proper Author navigator
- author facet is based on omitted author_sxt field
- adjust to make author nav available on exist of author field but keep using author_sxt to construct the facet (why!?)
- add check for querymodifier author in searchevent
Changed Files: source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java
Fri Jun 27 14:40:36 CEST 2014
by Michael Peter Christen
Changed Files: htroot/PerformanceSearch_p.java, htroot/api/timeline_p.java, htroot/api/timeline_p.xml, source/net/yacy/peers/graphics/ProfilingGraph.java, source/net/yacy/search/EventTracker.java, source/net/yacy/search/query/AccessTracker.java
Fri Jun 27 10:15:01 CEST 2014
by orbiter
added apkParser stub (work in progress)
Changed Files: source/net/yacy/document/parser/apkParser.java
Fri Jun 27 10:14:36 CEST 2014
by orbiter
added access tracker log reader stub
Changed Files: source/net/yacy/search/query/AccessTracker.java
Thu Jun 26 22:16:15 CEST 2014
by reger
adjust mergeDocument after parsing to
- preserve charset and languages
- fix merge of author
Changed Files: source/net/yacy/document/Document.java
Thu Jun 26 02:56:45 CEST 2014
by reger
fix AbstractParser.singleList not adding null strings
- prevents null titles in oo... parser  (as detected by ParserTest)
- correct ParserTest dc_description check (dc_description allowed to return 0 length array)
Changed Files: source/net/yacy/document/AbstractParser.java, test/net/yacy/document/ParserTest.java
Wed Jun 25 11:51:11 CEST 2014
by Michael Peter Christen
write <em>-tagged texts also into the bold_txt field
Changed Files: source/net/yacy/document/parser/html/ContentScraper.java
Sun Jun 22 23:33:59 CEST 2014
by reger
harmonize use of style for "delete" button
- apply the monstly used btn-danger class 
Changed Files: htroot/ConfigAccounts_p.html, htroot/ConfigAppearance_p.html, htroot/ConfigHTCache_p.html, htroot/ConfigLanguage_p.html, htroot/CrawlProfileEditor_p.html, htroot/IndexControlURLs_p.html
Sat Jun 21 04:21:29 CEST 2014
by reger
add a link to recrawl index entry to metadata html page
- to allow manually renew index content for this url (e.g. in case it is a remote search result with metadata only)
- use simply a  QuickCrawlLink_p javascript snippet (minimalistic 1st solution)
Changed Files: source/net/yacy/cora/federate/solr/responsewriter/HTMLResponseWriter.java
Fri Jun 20 13:56:55 CEST 2014
by Michael Peter Christen
added 'synchronous' and 'commit' flags in push api
Changed Files: htroot/api/push_p.java
Fri Jun 20 13:55:15 CEST 2014
by Michael Peter Christen
using concurrency to do base64 encoding in file POST commands
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Fri Jun 20 13:53:22 CEST 2014
by Michael Peter Christen
enhanced ASCII byte[] -> String conversion
Changed Files: source/net/yacy/cora/document/encoding/ASCII.java
Fri Jun 20 01:02:43 CEST 2014
by reger
move page navigation below results (image search)
force page navigation to be displayed below results in image search for any number of displayed images instead to be displayed to the right of last image.
Changed Files: htroot/yacysearch.html
Thu Jun 19 12:11:02 CEST 2014
by Michael Peter Christen
enhanced base64 encoding
Changed Files: source/net/yacy/cora/order/Base64Order.java
Thu Jun 19 12:10:42 CEST 2014
by Michael Peter Christen
removed superfluous date method
Changed Files: source/net/yacy/cora/date/MicroDate.java
Thu Jun 19 12:10:12 CEST 2014
by Michael Peter Christen
removed superfluous assert
Changed Files: source/net/yacy/kelondro/blob/HeapReader.java
Sun Jun 15 12:41:14 CEST 2014
by orbiter
ignore the api javadoc directory in git commits
Changed Files: .gitignore
Thu Jun 12 18:10:07 CEST 2014
by Michael Peter Christen
added a push api to make it possible to upload files directly without
crawling to the YaCy indexer. Files are uploaded using POST multipart
requests; multiple file uploads are possible as well. Each file has
attached the file date and mime type which is used to get the right
parser for the submitted data. Also an url is submitted which is
assigned to the document.
The CrawlSwitchboard has a new option for default Crawl Profiles which
are assigned dynamically from the new push interface.
Changed Files: htroot/api/push_p.html, htroot/api/push_p.java, htroot/api/push_p.json, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/search/Switchboard.java
Thu Jun 12 18:06:22 CEST 2014
by Michael Peter Christen
re-implementation of file push option in the context of POST http
requests. The internal representation of post-arguments is String and
therefore not appropriate for byte[] object as submitted by file pushes.
Therefore all pushed files are encoded to base64 _after_ uploading with
an http form (you do not need to do that encoding yourself) to hand-over
the byte[] as string in the post argument.
Servlets which read such files must decode the base64 data to get the
original byte[] array.
This is considered as a temporary solution for file uploads and a proper
implementations would need to consider all attributes as handed over as
Objects with either String or byte[] Object instances. This would be a
major code change and is not done at this time here now. The feature was
submitted to realize a feature as pushed with the next commit.
Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java
Thu Jun 12 05:23:26 CEST 2014
by Michael Peter Christen
Changed Files: source/net/yacy/crawler/retrieval/Response.java, source/net/yacy/http/ProxyCacheHandler.java, source/net/yacy/http/ProxyHandler.java, source/net/yacy/search/Switchboard.java, source/net/yacy/server/http/HTTPDProxyHandler.java
Wed Jun 11 04:13:55 CEST 2014
by reger
crawling of filenames with + fails due to url decoding
modified UTF8.decodeURL to apply x-www-form-urlencoded ( space -> + ) to the query part of the url only.
Changed Files: source/net/yacy/cora/document/encoding/UTF8.java
Tue Jun 10 04:25:20 CEST 2014
by reger
optimize pdfParser
skip starting reader thread if all content already read
Changed Files: source/net/yacy/document/parser/pdfParser.java
Sun Jun 08 08:20:30 CEST 2014
by reger
fix pdfParser not closed warning from pdfbox
for encrypted pdf on exit due to missing permission to extract
Changed Files: source/net/yacy/document/parser/pdfParser.java
Sat Jun 07 03:01:26 CEST 2014
by reger
fix unresolved pattern in yacysearch.rss title
and rss xml error due to html & encoding in url entries 
Changed Files: htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearchitem.java
Thu Jun 05 02:13:08 CEST 2014
by reger
Crawler hostbalancer does not delete finished queue files,
use alternative delete to fight the sympthom (and fix deletion of host dirs on startup)
Root cause (which class holds a lock on .stack) not found.
Changed Files: source/net/yacy/crawler/HostQueue.java
Wed Jun 04 13:59:37 CEST 2014
by Michael Peter Christen
in case that the network is switched to any other than freeworld, RWIs
are disabled. This is a temporary fix. There must be a better way to
determine if RWIs are to be switched on or of.
Changed Files: htroot/ConfigNetwork_p.java
Wed Jun 04 12:09:49 CEST 2014
by Michael Peter Christen
doing index deletions using a get string which makes it easier to
copy-paste deletion examples (see: #EuGH :( )
Changed Files: htroot/IndexDeletion_p.html
Mon Jun 02 17:40:56 CEST 2014
by Michael Peter Christen
added option to prefer http over https in unique-protocol ranking
Changed Files: defaults/yacy.init, source/net/yacy/search/Switchboard.java, source/net/yacy/search/schema/CollectionConfiguration.java
Mon Jun 02 13:11:44 CEST 2014
by Michael Peter Christen
removed check on protocol for double content (user request)
Changed Files: source/net/yacy/cora/federate/solr/SchemaConfiguration.java
Mon Jun 02 02:11:54 CEST 2014
by reger
add links to port config in status panel
- pom upd to match javadoc location
Changed Files: htroot/ConfigBasic.html, htroot/Status_p.inc, pom.xml
Sun Jun 01 22:25:55 CEST 2014
by Michael Peter Christen
changed javadoc publishing path from 'api' to 'javadoc' because there
are also other APIs in in YaCy.
Changed Files: build.xml
Sun Jun 01 06:43:50 CEST 2014
by reger
fix logging settings
- add missing .level
- remove obsolete jena settings
- set default level=INFO to prevent debug logging of not explicite specified classes
Changed Files: defaults/yacy.logging, source/net/yacy/document/Document.java
Sun Jun 01 03:15:38 CEST 2014
by reger
make https port setting changeable via front end somewhere
(chosen Http Networking page /Settings_p.html?page=http )
Changed Files: htroot/SettingsAck_p.html, htroot/SettingsAck_p.java, htroot/Settings_Http.inc, htroot/Settings_p.java
Sun Jun 01 01:02:03 CEST 2014
by Michael Peter Christen
added option to configure a custom user agent within allip networks
Changed Files: defaults/yacy.init, htroot/CrawlStartExpert.java, htroot/Crawler_p.java, source/net/yacy/cora/protocol/ClientIdentification.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java
Sun Jun 01 00:20:25 CEST 2014
by reger
- fix typo in Message_p (defaultpath)
- use more existing switchboardconstants for getproperties
- replace depriciated call defaultservlet
Changed Files: htroot/BlogComments.java, htroot/ConfigAppearance_p.java, htroot/Messages_p.java, htroot/www/welcome.java, htroot/yacy/message.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java
Sat May 31 09:29:55 CEST 2014
by orbiter
made number of open files in crawler configurable and increased default
maximum number of open files from 100 to 1000. This number can be
changed with the attribut crawler.onDemandLimit
Changed Files: defaults/yacy.init, source/net/yacy/crawler/HostBalancer.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/crawler/data/NoticedURL.java
Fri May 30 05:12:56 CEST 2014
by malykhin.dmitry
Update russian translation
Changed Files: locales/ru.lng
Thu May 29 00:01:48 CEST 2014
by sixcooler
Changed Windows Firewall Rules to just honor the default Port 8090, but
not use any programm-path.
This should match more installations in different paths and also running
YaCy as service (prunsrv).

This commit was contributed and tested on Windows7 by René.
Changed Files: build.nsi
Wed May 28 18:48:54 CEST 2014
by Michael Peter Christen
a try to fix the mixed up terms 'Active' -> 'Senior' and 'Passive' ->
Changed Files: htroot/Network.html
Wed May 28 03:01:34 CEST 2014
by reger
fix crawl of file:// , also http://mantis.tokeek.de/view.php?id=149
local files can be crawled (intranet mode) url parsing fixed according to  RFC 1738 (for unix and windows)
for win like file:///c:/tmp   or file://localhost/c:/tmp
for linux like file:///tmp  or file://localhost/tmp
Host is ignored and path must be absolute

Changed Files: htroot/Crawler_p.java, source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/crawler/HostQueue.java
Tue May 27 18:57:05 CEST 2014
by Michael Peter Christen
changed debian dependency to openjdk-7-jre-headless
Changed Files: debian/control
Tue May 27 15:28:28 CEST 2014
by Michael Peter Christen
added new fields http_unique_b and www_unique_b which can be used for
ranking to prefer urls containing a www subdomain or using the https
Changed Files: defaults/solr.collection.schema, source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/CollectionSchema.java
Sun May 25 01:06:39 CEST 2014
by reger
make config search page layout closer to actual page appearance
Changed Files: htroot/ConfigSearchPage_p.html
Sat May 24 01:01:27 CEST 2014
by reger
- upd pom to Solr 4.8.1 and latest jar updates
- upd nsis java autodownload package to jre 7u55
Changed Files: build.nsi, nbproject/project.xml, pom.xml
Thu May 22 22:16:39 CEST 2014
by sixcooler
Monitoring and limit connection-count for Jetty
Changed Files: htroot/Connections_p.html, htroot/Connections_p.java, htroot/PerformanceQueues_p.java, htroot/Status.java, source/net/yacy/cora/protocol/ConnectionInfo.java, source/net/yacy/http/Jetty9HttpServerImpl.java, source/net/yacy/http/MonitorHandler.java, source/net/yacy/http/YaCyHttpServer.java
Thu May 22 15:53:07 CEST 2014
by orbiter
upgraded poi library (office document format parser) from 3.9 to 3.10
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/poi-3.10-FINAL-20140208.License, lib/poi-3.10-FINAL-20140208.jar, lib/poi-scratchpad-3.10-FINAL-20140208.License, lib/poi-scratchpad-3.10-FINAL-20140208.jar
Thu May 22 12:55:22 CEST 2014
by orbiter
switch on core.service.rwi when switching back from portal mode to p2p
Changed Files: htroot/ConfigBasic.java, htroot/IndexFederated_p.html, htroot/IndexFederated_p.java
Thu May 22 12:09:21 CEST 2014
by orbiter
upgraded bouncy caste libraries (needed for encrypted pdfs, dependency
in pdfbox) to 1.46
removed the activation.jar library; I don't know which other library
depends on it.
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/bcmail-jdk15-1.46.License, lib/bcmail-jdk15-1.46.jar, lib/bcprov-jdk15-1.46.License, lib/bcprov-jdk15-1.46.jar, lib/dependencies.txt
Thu May 22 00:28:51 CEST 2014
by Michael Peter Christen
tried to make last_modified recognition smarter
Changed Files: source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/protocol/ResponseHeader.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/schema/CollectionConfiguration.java
Wed May 21 23:21:31 CEST 2014
by Michael Peter Christen
'Last-Modified', not 'Last-modified' according to
Changed Files: source/net/yacy/cora/protocol/HeaderFramework.java
Wed May 21 18:37:52 CEST 2014
by Michael Peter Christen
fix for operator precedence bug (cast binds stronger than bitwise AND)
in peer hash hashing. This should not change anything if java casts long
to int by masking with 0xFFFFFFFFL but you never know. The important
thing is, that the hashCode() should not return numbers that have the
same order as the hash code order because hashing of seeds is used to
remove the order in some places.
Changed Files: source/net/yacy/peers/Seed.java
Wed May 21 13:29:46 CEST 2014
by sixcooler
Revert "manual merge"

This reverts commit 3bfab8566c30df5a928ec6c7d3736f8e5d2eb4ca.
Changed Files:
Wed May 21 03:04:07 CEST 2014
by sixcooler
avoid erros from ConnectHandler, correction for #6d16fa9
Changed Files: source/net/yacy/http/AbstractRemoteHandler.java, source/net/yacy/http/Jetty9HttpServerImpl.java, source/net/yacy/http/ProxyHandler.java
Tue May 20 13:27:40 CEST 2014
by orbiter
updated bootstrap seed list
Changed Files: defaults/yacy.init, defaults/yacy.network.freeworld.unit
Mon May 19 20:51:43 CEST 2014
by Marc Nause
Improved blacklist XML/JSON API.
Changed Files: htroot/api/blacklists/get_list_p.java, htroot/api/blacklists/get_list_p.json, htroot/api/blacklists/get_list_p.xml, htroot/api/blacklists/get_metadata_p.java, htroot/api/blacklists/get_metadata_p.json, htroot/api/blacklists/get_metadata_p.xml
Mon May 19 17:55:03 CEST 2014
by sixcooler
lower Segment-size (hope to get Segments of 10GB)
Changed Files: source/net/yacy/search/Switchboard.java
Sun May 18 22:41:09 CEST 2014
by reger
fix: load_Rss remove feed to many parameter for get
use form post methode
Changed Files: htroot/Load_RSS_p.html
Sat May 17 23:36:59 CEST 2014
by reger
"Use remote proxy for yacy" has no function, remove option and related config item
see/fix bug http://mantis.tokeek.de/view.php?id=23
Changed Files: defaults/yacy.init, htroot/SettingsAck_p.java, htroot/Settings_Proxy.inc, htroot/Settings_p.java, htroot/Status.java, source/net/yacy/cora/protocol/http/ProxySettings.java, source/net/yacy/search/Switchboard.java
Sat May 17 21:34:23 CEST 2014
by reger
fix: CrawlStartExpert.html # From File with missing filename
- crawlName must not be empty
- crawlingFile must not be empty
Changed Files: htroot/Crawler_p.java
Wed May 14 23:13:49 CEST 2014
by reger
cleanup obsolete and not used  serverswitch Authentify code
as auth is mostly delegated to Jetty container.
Changed Files: source/net/yacy/server/serverSwitch.java
Tue May 13 22:31:12 CEST 2014
by orbiter
move unnecessary nested else out of condition 
Changed Files: source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/http/servlets/YaCyProxyServlet.java
Tue May 13 22:29:05 CEST 2014
by orbiter
removed warnings
Changed Files: htroot/api/linkstructure.java, source/net/yacy/search/schema/HyperlinkGraph.java
Tue May 13 22:27:31 CEST 2014
by orbiter
removed warnings
Changed Files: .settings/org.eclipse.jdt.core.prefs, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/http/Jetty9YaCySecurityHandler.java, source/net/yacy/kelondro/util/MemoryControl.java, source/net/yacy/peers/Protocol.java
Tue May 13 21:30:47 CEST 2014
by orbiter
another circle computation enhancement
Changed Files: source/net/yacy/visualization/CircleTool.java
Tue May 13 08:14:56 CEST 2014
by orbiter
switched menu position of ConfigPortal.html and ConfigSearchBox.html
Changed Files: htroot/env/templates/header.template, htroot/env/templates/submenuPortalConfiguration.template
Tue May 13 02:33:35 CEST 2014
by reger
Exclude =  from percent-encoding in MultiProtocolURL
fix http://mantis.tokeek.de/view.php?id=185 and http://mantis.tokeek.de/view.php?id=280
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Tue May 13 01:50:15 CEST 2014
by reger
fix truncated urls containing ","
adressing http://mantis.tokeek.de/view.php?id=58

Exclude comma from percent-encoding in MultiProtocolURL (see  RFC 1738 2.2  and  RFC 3986 2.2)
Changed Files: source/net/yacy/cora/document/id/MultiProtocolURL.java
Mon May 12 16:21:24 CEST 2014
by Michael Peter Christen
enhanced circle tool initialization
Changed Files: source/net/yacy/visualization/CircleTool.java
Mon May 12 15:45:25 CEST 2014
by Michael Peter Christen
automatically clear graphics cache
Changed Files: htroot/NetworkPicture.java, source/net/yacy/peers/graphics/NetworkGraph.java, source/net/yacy/search/Switchboard.java, source/net/yacy/visualization/CircleTool.java
Mon May 12 15:34:56 CEST 2014
by Michael Peter Christen
enhanced circle computation time and memory footprint
Changed Files: source/net/yacy/visualization/CircleTool.java
Mon May 12 00:12:05 CEST 2014
by reger
improve IndexDeletion by query
adding transparently text_t as pseudo default search field if no fieldname (no  : ) is included.
adressing bug report  http://mantis.tokeek.de/view.php?id=274
Changed Files: htroot/IndexDeletion_p.java
Sun May 11 03:51:47 CEST 2014
by reger
assign serialVersionUID for proxyservlet, too.
Changed Files: source/net/yacy/http/servlets/YaCyProxyServlet.java
Sun May 11 03:31:07 CEST 2014
by reger
set serialVersionUID on urlproxyservlet to skip compiler warning
- remove commented out code
Changed Files: source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/http/servlets/YaCyProxyServlet.java
Sun May 11 02:37:00 CEST 2014
by reger
remove redundant servlet-api
Changed Files:
Thu May 08 22:21:24 CEST 2014
by reger
respect "index media" switched off in CrawlStartExpert.html
fix http://mantis.tokeek.de/view.php?id=64
Changed Files: source/net/yacy/search/Switchboard.java
Thu May 08 22:19:47 CEST 2014
by reger
Merge origin/master
Changed Files: .settings/org.eclipse.jdt.core.prefs, build.properties, htroot/yacy/seedlist.java, source/net/yacy/cora/protocol/Domains.java
Thu May 08 22:18:12 CEST 2014
by reger
switch pom to v1.73 and java 1.7
Changed Files: pom.xml
Thu May 08 07:41:40 CEST 2014
by orbiter
added a peername=<peername> property in the seedlist API
Changed Files: htroot/yacy/seedlist.java
Thu May 08 07:41:11 CEST 2014
by orbiter
next development step: migration to java 1.7
This includes also a small code change to test generic type inference, a
java 1.7 feature
Changed Files: .settings/org.eclipse.jdt.core.prefs, build.properties, source/net/yacy/cora/protocol/Domains.java