Huge update on web site design in YaCy: we are now using bootstrap.css. A large number of other changes is included as well, most notable an almost perfect russian localization, many bugfixes and performance enhancements.
Commit | Description |
---|---|
Thu Apr 03 14:51:19 CEST 2014 by Michael Peter Christen | - added d3.js for visualizations using embedded svg - added a servlet api/linkstructure.json which generates a link graph information in json - added a javascript link graph renderer hypertree.js using d3 and the new servlet linkstructure.json - embedded the new link graph in the crawler monitor and the host browser Changed Files: htroot/Crawler_p.html, htroot/Crawler_p.java, htroot/HostBrowser.html, htroot/HostBrowser.java, htroot/api/linkstructure.java, htroot/api/linkstructure.json, htroot/env/hypertree.css, htroot/js/d3.v3.min.js, htroot/js/hypertree.js, htroot/test.html |
Mon Mar 31 15:47:58 CEST 2014 by Michael Peter Christen | Complete redesign of administration top-level menu. This follows two principles: - provide an easy tutorial-like "what should I do first" menu - provide all elements which are subject to most first questions to YaCy exibition people on top level: Resource limitation, Parser and Ranking settings I apologize to everyone who are used to the old style and need to find the menu items (again) after this change. I hope that this will make the interface more usable for new users who see a web indexer/crawler the first time. Changed Files: htroot/ConfigAccounts_p.html, htroot/ConfigBasic.html, htroot/ConfigHTCache_p.html, htroot/ConfigParser.html, htroot/ConfigRobotsTxt_p.html, htroot/ConfigUpdate_p.html, htroot/ContentIntegrationPHPBB3_p.html, htroot/CrawlStartSite_p.html, htroot/IndexImportMediawiki_p.html, htroot/IndexImportOAIPMH_p.html, htroot/Load_RSS_p.html, htroot/Performance_p.html, htroot/Status.html, htroot/Threaddump_p.html, htroot/env/templates/SubmenuMaintenance.template, htroot/env/templates/header.template, htroot/env/templates/submenuComputation.template, htroot/env/templates/submenuConfig.template, htroot/env/templates/submenuCrawler.template, htroot/env/templates/submenuIndexControl.template, htroot/env/templates/submenuIndexCreate.template, htroot/env/templates/submenuPortalConfiguration.template, htroot/env/templates/submenuUseCaseAccount.template |
Mon Mar 31 03:18:38 CEST 2014 by Michael Peter Christen | redesign of search page integration menu structure Changed Files: .settings/org.eclipse.jdt.core.prefs, htroot/ConfigAppearance_p.html, htroot/ConfigHeuristics_p.html, htroot/ConfigLanguage_p.html, htroot/ConfigPortal.html, htroot/ConfigProfile_p.html, htroot/ConfigSearchBox.html, htroot/ConfigSearchPage_p.html, htroot/RankingRWI_p.html, htroot/RankingSolr_p.html, htroot/env/templates/header.template, htroot/env/templates/submenuDesign.template, htroot/env/templates/submenuPortalConfiguration.template, htroot/env/templates/submenuRanking.template, locales/de.lng, locales/fr.lng, locales/ru.lng |
Mon Mar 31 01:15:35 CEST 2014 by Michael Peter Christen | replacing direct html table cellspacing with css set-up for cellspacing Changed Files: htroot/AccessTracker_p.html, htroot/Blacklist_p.html, htroot/ConfigHeuristics_p.html, htroot/ConfigParser.html, htroot/Connections_p.html, htroot/CookieMonitorIncoming_p.html, htroot/CookieMonitorOutgoing_p.html, htroot/CrawlCheck_p.html, htroot/CrawlMonitorRemoteStart.html, htroot/CrawlProfileEditor_p.html, htroot/CrawlResults.html, htroot/CrawlStartExpert_p.html, htroot/Crawler_p.html, htroot/HostBrowser.html, htroot/IndexControlRWIs_p.html, htroot/IndexControlURLs_p.html, htroot/IndexCreateLoaderQueue_p.html, htroot/IndexCreateParserErrors_p.html, htroot/IndexCreateQueues_p.html, htroot/IndexFederated_p.html, htroot/IndexImportOAIPMHList_p.html, htroot/IndexSchema_p.html, htroot/Load_RSS_p.html, htroot/Messages_p.html, htroot/Network.html, htroot/News.html, htroot/PerformanceConcurrency_p.html, htroot/PerformanceMemory_p.html, htroot/PerformanceQueues_p.html, htroot/PerformanceSearch_p.html, htroot/Performance_p.html, htroot/ProxyIndexingMonitor_p.html, htroot/RemoteCrawl_p.html, htroot/ServerScannerList.html, htroot/Table_YMark_p.html, htroot/Tables_p.html, htroot/ViewFile.html, htroot/Vocabulary_p.html, htroot/api/table_p.html, htroot/env/base.css, htroot/sharedBlacklist_p.html |
Sun Mar 30 04:04:02 CEST 2014 by reger | alternative UrlProxyServlet (for /proxy.html) using different url rewrite rules - use JSoup parser for selective rewrite of html body <a href= links only, instead of regex which rewrites also header href/src links - this improves display of pages which use header <base> tag - tags with src attribute are taken from original location (like css) improving display and are not routed trough the indexer Disadvantage: scripting links will drop out of proxy Setting of the servlet through web.xml exclusivly (in case one would like to quickly switch back to the YaCyProxyServlet, leaving the existing code of YaCyProxyServlet untouched available) Changed Files: defaults/web.xml, source/net/yacy/http/Jetty8HttpServerImpl.java, source/net/yacy/http/servlets/UrlProxyServlet.java, source/net/yacy/http/servlets/YaCyProxyServlet.java, source/net/yacy/server/http/HTTPDProxyHandler.java |
Fri Mar 28 13:48:37 CET 2014 by Michael Peter Christen | added missing @Override annotation Changed Files: htroot/Blacklist_p.java, htroot/HostBrowser.java, source/net/yacy/cora/date/AbstractFormatter.java, source/net/yacy/cora/date/ISO8601Formatter.java, source/net/yacy/cora/document/analysis/EnhancedTextProfileSignature.java, source/net/yacy/cora/federate/solr/connector/CachedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java, source/net/yacy/cora/federate/solr/instance/EmbeddedInstance.java, source/net/yacy/cora/federate/solr/instance/ServerMirror.java, source/net/yacy/cora/federate/solr/instance/ServerShard.java, source/net/yacy/cora/federate/yacy/Peer.java, source/net/yacy/cora/federate/yacy/Peers.java, source/net/yacy/cora/language/phonetic/ColognePhonetic.java, source/net/yacy/cora/language/phonetic/DoubleMetaphone.java, source/net/yacy/cora/language/phonetic/Metaphone.java, source/net/yacy/cora/language/phonetic/Soundex.java, source/net/yacy/cora/order/AbstractOrder.java, source/net/yacy/cora/order/Order.java, source/net/yacy/cora/protocol/ConnectionInfo.java, source/net/yacy/cora/protocol/http/GzipCompressingEntity.java, source/net/yacy/cora/protocol/http/GzipRequestInterceptor.java, source/net/yacy/cora/protocol/http/GzipResponseInterceptor.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/protocol/http/NonClosingInputStreamEntity.java, source/net/yacy/cora/sorting/AbstractScoreMap.java, source/net/yacy/cora/sorting/Array.java, source/net/yacy/cora/sorting/ConcurrentScoreMap.java, source/net/yacy/cora/sorting/OrderedScoreMap.java, source/net/yacy/cora/sorting/Rating.java, source/net/yacy/cora/sorting/UpDownLatch.java, source/net/yacy/cora/storage/ARC.java, source/net/yacy/cora/storage/ARH.java, source/net/yacy/cora/storage/ConcurrentARC.java, source/net/yacy/cora/storage/SizeLimitedSet.java, source/net/yacy/cora/util/LookAheadIterator.java, source/net/yacy/cora/util/StringBuilderComparator.java, source/net/yacy/crawler/robots/RobotsTxt.java, source/net/yacy/data/MessageBoard.java, source/net/yacy/data/UserDB.java, source/net/yacy/data/ymark/YMarkDMOZImporter.java, source/net/yacy/data/ymark/YMarkHTMLImporter.java, source/net/yacy/data/ymark/YMarkJSONImporter.java, source/net/yacy/data/ymark/YMarkTag.java, source/net/yacy/data/ymark/YMarkXBELImporter.java, source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/document/content/dao/ImportDump.java, source/net/yacy/document/importer/Importer.java, source/net/yacy/document/importer/ResumptionToken.java, source/net/yacy/document/language/LanguageFilenameFilter.java, source/net/yacy/document/parser/rdfa/impl/RDFaTripleContent.java, source/net/yacy/gui/InfoPage.java, source/net/yacy/gui/Tray.java, source/net/yacy/gui/framework/Application.java, source/net/yacy/http/ProxyHandler.java, source/net/yacy/http/servlets/GSAsearchServlet.java, source/net/yacy/kelondro/blob/TablesColumnRAMIndex.java, source/net/yacy/kelondro/data/navigation/NavigationReference.java, source/net/yacy/kelondro/data/navigation/NavigationReferenceRow.java, source/net/yacy/kelondro/index/Index.java, source/net/yacy/kelondro/index/RAMIndexCluster.java, source/net/yacy/kelondro/index/RowHandleMap.java, source/net/yacy/kelondro/io/AbstractReader.java, source/net/yacy/kelondro/io/AbstractWriter.java, source/net/yacy/kelondro/io/ByteCountOutputStream.java, source/net/yacy/kelondro/io/CachedFileReader.java, source/net/yacy/kelondro/rwi/AbstractBufferedIndex.java, source/net/yacy/kelondro/rwi/AbstractReference.java, source/net/yacy/kelondro/rwi/ReferenceContainerOrder.java, source/net/yacy/kelondro/rwi/ReferenceIterator.java, source/net/yacy/kelondro/table/ChunkIterator.java, source/net/yacy/kelondro/util/BDecoder.java, source/net/yacy/kelondro/util/GenerationMemoryStrategy.java, source/net/yacy/kelondro/util/NamePrefixThreadFactory.java, source/net/yacy/kelondro/util/ReverseMapIterator.java, source/net/yacy/kelondro/util/StandardMemoryStrategy.java, source/net/yacy/kelondro/workflow/AbstractBusyThread.java, source/net/yacy/peers/DHTSelection.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/operation/yacySeedUploadFtp.java, source/net/yacy/search/MemoryTracker.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/CollectionSchema.java, source/net/yacy/search/schema/WebgraphSchema.java, source/net/yacy/upnp/Discovery.java, source/net/yacy/upnp/JXPathParser.java, source/net/yacy/upnp/ServicesEventing.java, source/net/yacy/upnp/messages/StateVariableResponseParser.java, source/net/yacy/utils/ListDirs.java, source/net/yacy/utils/enumerateFiles.java, source/net/yacy/utils/loaderCore.java, source/net/yacy/visualization/DemoApplet.java, source/net/yacy/yacy.java |
Fri Mar 21 02:42:09 CET 2014 by Michael Peter Christen | - added new input field in index.html - changed progress bar in yacysearch.html - moved pagination navigation to page bottom - moved search term input field to headline Changed Files: defaults/yacy.init, htroot/env/base.css, htroot/env/templates/header.template, htroot/index.html, htroot/index.java, htroot/js/yacysearch.js, htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearchitem.html |
Thu Mar 20 22:52:01 CET 2014 by orbiter | replaced pdblue-homebrew buttons with bootstrap standard buttons Changed Files: htroot/ConfigAccounts_p.html, htroot/ConfigAppearance_p.html, htroot/ConfigBasic.html, htroot/ConfigHTCache_p.html, htroot/ConfigHeuristics_p.html, htroot/ConfigLanguage_p.html, htroot/ConfigNetwork_p.html, htroot/ConfigParser.html, htroot/ConfigPortal.html, htroot/ConfigProfile_p.html, htroot/ConfigProperties_p.html, htroot/ConfigRobotsTxt_p.html, htroot/ConfigSearchPage_p.html, htroot/ContentControl_p.html, htroot/ContentIntegrationPHPBB3_p.html, htroot/CookieMonitorIncoming_p.html, htroot/CookieMonitorOutgoing_p.html, htroot/CookieTest_p.html, htroot/CrawlCheck_p.html, htroot/CrawlProfileEditor_p.html, htroot/CrawlStartExpert_p.html, htroot/CrawlStartScanner_p.html, htroot/CrawlStartSite_p.html, htroot/HostBrowser.html, htroot/HostBrowserAdmin_p.html, htroot/IndexControlRWIs_p.html, htroot/IndexControlURLs_p.html, htroot/IndexDeletion_p.html, htroot/api/table_p.html, htroot/api/ymarks/test_import.html, htroot/www/welcome.html, skins/pdblue.css |
Thu Mar 20 00:06:28 CET 2014 by Michael Peter Christen | removed underline for anchor refs and replaced it with underline-on-hover. Please update your style (switch and switch back) Changed Files: htroot/env/base.css, skins/27c3.css, skins/28c3.css, skins/classic-blue.css, skins/dark-blue.css, skins/generic_pd.css, skins/geo.css, skins/grey.css, skins/pdblue.css, skins/vega-aqua.css |
Wed Mar 19 10:06:22 CET 2014 by orbiter | - removed usage of jquery and yacy/ui in standard search interfaces as possible - re-implemented formerly jquery-based tagcloud with simple html - fixed minimized menu for mobile/minimized view (try that: make window very narrow...) Changed Files: htroot/env/base.css, htroot/env/bootstrap-base.css, htroot/env/bootstrap/js/jquery.min.js, htroot/env/templates/embeddedfooter.template, htroot/env/templates/footer.template, htroot/env/templates/metas.template, htroot/env/templates/simplefooter.template, htroot/index.html, htroot/test.html, htroot/yacyinteractive.html, htroot/yacysearch.html, htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java |
Tue Mar 18 13:42:31 CET 2014 by Michael Peter Christen | - added bootstrap css framework - adopted all YaCy administration pages to new framework - created new search page layout (working, but still work in progress) - old skin files are fully appliable! (and looking good) - target is a new style based on bootstrap examples, see /test.html - icons in YaCy may be replaced by glyphicons (to be done) Changed Files: htroot/AugmentedBrowsing_p.html, htroot/Bookmarks.html, htroot/ConfigHeuristics_p.html, htroot/ConfigSearchPage_p.html, htroot/CrawlCheck_p.html, htroot/CrawlStartExpert_p.html, htroot/CrawlStartScanner_p.html, htroot/CrawlStartSite_p.html, htroot/Crawler_p.html, htroot/HostBrowser.html, htroot/HostBrowser.java, htroot/IndexControlURLs_p.html, htroot/IndexCreateQueues_p.html, htroot/IndexDeletion_p.html, htroot/IndexFederated_p.html, htroot/IndexSchema_p.html, htroot/Load_RSS_p.html, htroot/Network.html, htroot/RemoteCrawl_p.html, htroot/Status_p.inc, htroot/Table_API_p.html, htroot/Table_RobotsTxt_p.html, htroot/Tables_p.html, htroot/ViewFile.html, htroot/ViewLog_p.html, htroot/ViewProfile.html, htroot/ViewProfile.java, htroot/Vocabulary_p.html, htroot/WatchWebStructure_p.html, htroot/YMarks.html, htroot/env/base.css, htroot/env/bootstrap-base.css, htroot/env/bootstrap/css/bootstrap-rtl.css, htroot/env/bootstrap/css/bootstrap-rtl.min.css, htroot/env/bootstrap/css/bootstrap-theme.css, htroot/env/bootstrap/css/bootstrap-theme.css.map, htroot/env/bootstrap/css/bootstrap-theme.min.css, htroot/env/bootstrap/css/bootstrap.css, htroot/env/bootstrap/css/bootstrap.css.map, htroot/env/bootstrap/css/bootstrap.min.css, htroot/env/bootstrap/fonts/glyphicons-halflings-regular.eot, htroot/env/bootstrap/fonts/glyphicons-halflings-regular.svg, htroot/env/bootstrap/fonts/glyphicons-halflings-regular.ttf, htroot/env/bootstrap/fonts/glyphicons-halflings-regular.woff, htroot/env/bootstrap/js/bootstrap.js, htroot/env/bootstrap/js/bootstrap.min.js, htroot/env/bootstrap/js/docs.min.js, htroot/env/bootstrap/js/html5shiv.js, htroot/env/bootstrap/js/respond.min.js, htroot/env/grafics/external.png, htroot/env/templates/footer.template, htroot/env/templates/header.template, htroot/env/templates/metas.template, htroot/env/templates/portalinput.template, htroot/env/templates/simpleheader.template, htroot/env/templates/submenuIndexCreate.template, htroot/index.html, htroot/index.java, htroot/test.html, htroot/yacyinteractive.html, htroot/yacyinteractive.java, htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearch_location.html, htroot/yacysearchtrailer.html, skins/27c3.css, skins/28c3.css, skins/generic_pd.css, skins/pdblue.css |
Thu Mar 06 01:43:48 CET 2014 by Michael Peter Christen | - added concurrency to postprocessing of webgraph document - bundeled separate webgraph postprocesing steps into one Changed Files: htroot/HostBrowser.java, htroot/IndexDeletion_p.java, source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/MirrorSolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrConnector.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/WebgraphConfiguration.java |
Mon Mar 03 22:13:40 CET 2014 by Michael Peter Christen | enhanced concurrent loading by using a fixed set of concurrent loader processes in favor of throwaway-processes. The control mechanism does less often report a 'queue full' message to the busy loop which then does not perform a long busy waiting; instead all requests are queued and new loader processes are started if necessary up to a given limit (as set before) Changed Files: htroot/IndexCreateLoaderQueue_p.java, htroot/PerformanceQueues_p.java, htroot/Status.java, htroot/api/status_p.java, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/crawler/retrieval/Request.java, source/net/yacy/search/Switchboard.java |
Fri Feb 28 15:02:01 CET 2014 by Michael Peter Christen | attaching names to all Threads to get a better view in profiling tools like VisualVM Changed Files: source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/MirrorSolrConnector.java, source/net/yacy/cora/federate/solr/instance/ServerShard.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/sorting/Array.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/crawler/robots/RobotsTxt.java, source/net/yacy/http/Jetty8HttpServerImpl.java, source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/workflow/InstantBusyThread.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/RemoteSearch.java, source/net/yacy/peers/graphics/WebStructureGraph.java, source/net/yacy/search/MemoryTracker.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/query/SearchEvent.java |
Fri Feb 28 14:01:09 CET 2014 by Michael Peter Christen | - enhanced postprocessing speed and memory footprint (by using HashMaps instead of TreeMaps) - enhanced memory footprint of database indexes (by introduction of optimize calls) - optimize calls shrink the amount of used memory for index sets if they are not changed afterwards any more Changed Files: htroot/HostBrowser.java, htroot/api/webstructure.java, source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/cora/storage/HandleMap.java, source/net/yacy/crawler/Balancer.java, source/net/yacy/crawler/data/NoticedURL.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/BLOB.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/index/BufferedObjectIndex.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/Index.java, source/net/yacy/kelondro/index/RAMIndex.java, source/net/yacy/kelondro/index/RAMIndexCluster.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowHandleMap.java, source/net/yacy/kelondro/index/RowHandleSet.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/snippet/MediaSnippet.java |
Thu Feb 27 18:33:45 CET 2014 by sixcooler | bump to httpclient-4.3.3 Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/dependencies.txt, lib/httpclient-4.3.3.License, lib/httpclient-4.3.3.jar, lib/httpcore-4.3.2.License, lib/httpcore-4.3.2.jar, lib/httpmime-4.3.3.License, lib/httpmime-4.3.3.jar, nbproject/project.xml, pom.xml |
Wed Feb 26 14:30:48 CET 2014 by Michael Peter Christen | do not use luke requests for remote solr servers if the result is different from normal requests. This happens if the remote solr is actually a solrCloud; in such cases the luke request returns only the result of the single solr peer, not the whole cloud. also done: some refactoring. Changed Files: htroot/CrawlStartScanner_p.java, htroot/HostBrowser.java, htroot/IndexDeletion_p.java, htroot/api/status_p.java, source/net/yacy/cora/document/encoding/ASCII.java, source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java, source/net/yacy/cora/federate/solr/instance/ResponseAccumulator.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/ErrorCache.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/index/ReindexSolrBusyThread.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/query/QueryGoal.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/WebgraphConfiguration.java |
Tue Feb 25 00:24:10 CET 2014 by reger | Merge origin/master Changed Files: htroot/Blacklist_p.html, htroot/HostBrowser.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/api/ymarks/add_ymark.java, htroot/yacysearch.java, locales/de.lng, source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/MirrorSolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrConnector.java, source/net/yacy/data/ymark/YMarkMetadata.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/ErrorCache.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/ranking/BlockRank.java, source/net/yacy/search/schema/CollectionConfiguration.java |
Wed Feb 19 04:03:45 CET 2014 by Michael Peter Christen | changed strategy to test existence of documents in Solr: using the update time. The reason for that is a better caching for the crawler double-check, which needs the update time for crawler steering. Changed Files: htroot/HostBrowser.java, htroot/IndexControlRWIs_p.java, htroot/Load_RSS_p.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/MirrorSolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrConnector.java, source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/crawler/retrieval/RSSLoader.java, source/net/yacy/crawler/retrieval/SitemapImporter.java, source/net/yacy/peers/Transmission.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/index/Segment.java |
Wed Feb 12 01:00:44 CET 2014 by Michael Peter Christen | enhanced the resource observer The resource observer is now able to recognize free disk space AND available space for YaCy. The amount of space which is assigned for YaCy are defined in new settings in the configuration file. Furthermore, there is now a cleanup process which deletes files in case that an autodelete is activated. The autodelete is now BY DEFAULT ON if the disk space is low, which means that YaCy starts to delete documents when the disk is full! Changed Files: defaults/yacy.init, htroot/PerformanceQueues_p.java, htroot/Status.java, htroot/Status_p.inc, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/search/ResourceObserver.java, source/net/yacy/search/SwitchboardConstants.java, source/net/yacy/search/index/Fulltext.java |
Tue Feb 11 20:56:40 CET 2014 by orbiter | clear solr caches in case that an exception occurrs. The reason behind this hack is the occurrence of Exceptions like: W 2014/02/11 18:51:33 ConcurrentLog GC overhead limit exceeded java.io.IOException: GC overhead limit exceeded at net.yacy.cora.federate.solr.connector.AbstractSolrConnector.getDocumentById(AbstractSolrConnector.java:334) at net.yacy.cora.federate.solr.connector.MirrorSolrConnector.getDocumentById(MirrorSolrConnector.java:173) at net.yacy.cora.federate.solr.connector.ConcurrentUpdateSolrConnector.getDocumentById(ConcurrentUpdateSolrConnector.java:415) at net.yacy.search.index.Fulltext.getMetadata(Fulltext.java:331) at net.yacy.search.index.Fulltext.getMetadata(Fulltext.java:317) at net.yacy.search.query.SearchEvent.pullOneRWI(SearchEvent.java:1024) at net.yacy.search.query.SearchEvent.pullOneFilteredFromRWI(SearchEvent.java:1047) at net.yacy.search.query.SearchEvent$3.run(SearchEvent.java:1263) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded at java.util.Arrays.copyOfRange(Arrays.java:3077) at java.lang.StringCoding.decode(StringCoding.java:196) at java.lang.String.<init>(String.java:491) at java.lang.String.<init>(String.java:547) at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.readField(CompressingStoredFieldsReader.java:187) at org.apache.lucene.codecs.compressing.CompressingStoredFieldsReader.visitDocument(CompressingStoredFieldsReader.java:351) at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:276) at org.apache.lucene.index.BaseCompositeReader.document(BaseCompositeReader.java:110) at org.apache.lucene.index.IndexReader.document(IndexReader.java:436) at org.apache.solr.search.SolrIndexSearcher.doc(SolrIndexSearcher.java:657) at net.yacy.cora.federate.solr.connector.EmbeddedSolrConnector.SolrQueryResponse2SolrDocumentList(EmbeddedSolrConnector.java:230) at net.yacy.cora.federate.solr.connector.EmbeddedSolrConnector.getDocumentListByParams(EmbeddedSolrConnector.java:320) at net.yacy.cora.federate.solr.connector.AbstractSolrConnector.getDocumentById(AbstractSolrConnector.java:330) ... 7 more This problem was analysed with the Eclipse Memory Analyser after a heap dump, where the following problem was reported as the main Problem Suspect: One instance of "org.apache.solr.util.ConcurrentLRUCache" loaded by "sun.misc.Launcher$AppClassLoader @ 0x42e940a0" occupies 902.898.256 (61,80%) bytes. The memory is accumulated in one instance of "java.util.concurrent.ConcurrentHashMap$Segment[]" loaded by "<system class loader>". This memory is part of the result cache of Solr. Flushing this cache appears the most appropriate solution to that problem. Changed Files: source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java |
Mon Feb 10 21:40:42 CET 2014 by reger | make internal page links relative to ease any future development for context aware servlets note also http://bugs.yacy.net/view.php?id=106 Changed Files: htroot/AccessTracker_p.html, htroot/Bookmarks.html, htroot/ConfigAppearance_p.html, htroot/ConfigPortal.java, htroot/ConfigProfile_p.html, htroot/ConfigSearchPage_p.html, htroot/ConfigUpdate_p.html, htroot/Connections_p.html, htroot/CookieTest_p.html, htroot/CrawlCheck_p.html, htroot/CrawlMonitorRemoteStart.html, htroot/CrawlResults.html, htroot/CrawlStartExpert_p.html, htroot/CrawlStartScanner_p.html, htroot/CrawlStartSite_p.html, htroot/Crawler_p.html, htroot/Help.html, htroot/HostBrowser.html, htroot/IndexControlRWIs_p.html, htroot/IndexControlURLs_p.html, htroot/IndexCreateQueues_p.html, htroot/IndexFederated_p.html, htroot/IndexImportOAIPMHList_p.html, htroot/IndexImportOAIPMHList_p.java, htroot/IndexImportOAIPMH_p.html, htroot/IndexSchema_p.html, htroot/Load_RSS_p.html, htroot/Network.html, htroot/Performance_p.html, htroot/ProxyIndexingMonitor_p.html, htroot/RankingRWI_p.html, htroot/RegexTest.html, htroot/Status.html, htroot/Steering.html, htroot/Supporter.html, htroot/Surftips.html, htroot/Table_API_p.html, htroot/Table_RobotsTxt_p.html, htroot/User.html, htroot/ViewFile.html, htroot/ViewProfile.html, htroot/Vocabulary_p.html, htroot/WatchWebStructure_p.html, htroot/YMarks.html, htroot/YaCySearchPluginFF.html, htroot/env/templates/header.template, htroot/env/templates/metas.template, htroot/env/templates/portalheader.template, htroot/env/templates/simpleheader.template, htroot/env/templates/submenuAccessTracker.template, htroot/env/templates/submenuBlacklist.template, htroot/env/templates/submenuComputation.template, htroot/env/templates/submenuConfig.template, htroot/env/templates/submenuCookie.template, htroot/env/templates/submenuCrawlMonitor.template, htroot/env/templates/submenuIndexControl.template, htroot/env/templates/submenuIndexCreate.template, htroot/env/templates/submenuPublication.template, htroot/env/templates/submenuSearchConfiguration.template, htroot/env/templates/submenuSearchIntegration.template, htroot/env/templates/submenuSemantic.template, htroot/env/templates/submenuTargetAnalysis.template, htroot/env/templates/submenuWebStructure.template, htroot/index.html, htroot/rssTerminal.html, htroot/terminal_p.html, htroot/www/welcome.html, htroot/yacyinteractive.html, htroot/yacysearch.xsl, htroot/yacysearch_location.html, htroot/yacysearchitem.html, htroot/yacysearchitem.java, locales/cn.lng, locales/de.lng, locales/fr.lng, locales/hi.lng, locales/ru.lng, locales/sk.lng, locales/uk.lng |
Mon Feb 10 11:42:11 CET 2014 by Michael Peter Christen | replaced solr 4.6.0. with 4.6.1 Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/lucene-analyzers-common-4.6.1.jar, lib/lucene-analyzers-phonetic-4.6.1.jar, lib/lucene-classification-4.6.1.jar, lib/lucene-codecs-4.6.1.jar, lib/lucene-core-4.6.1.jar, lib/lucene-facet-4.6.1.jar, lib/lucene-grouping-4.6.1.jar, lib/lucene-highlighter-4.6.1.jar, lib/lucene-join-4.6.1.jar, lib/lucene-memory-4.6.1.jar, lib/lucene-misc-4.6.1.jar, lib/lucene-queries-4.6.1.jar, lib/lucene-queryparser-4.6.1.jar, lib/lucene-spatial-4.6.1.jar, lib/lucene-suggest-4.6.1.jar, lib/solr-core-4.6.1.jar, lib/solr-solr-4.6.1.License, lib/solr-solrj-4.6.1.License |
Commit | Description |
---|---|
Fri Apr 04 14:43:35 CEST 2014 by Michael Peter Christen | fixed a bug which prevented crawl starts after a network switch Changed Files: source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/search/index/ErrorCache.java |
Fri Apr 04 12:47:29 CEST 2014 by Michael Peter Christen | fix for linkstructure computation: now also detecting dead links Changed Files: htroot/api/linkstructure.java |
Fri Apr 04 12:41:59 CEST 2014 by Michael Peter Christen | another fix for inbound/outbound detection Changed Files: source/net/yacy/search/schema/WebgraphConfiguration.java |
Fri Apr 04 12:34:34 CEST 2014 by Michael Peter Christen | NPE fix Changed Files: source/net/yacy/crawler/data/CrawlQueues.java |
Fri Apr 04 11:59:51 CEST 2014 by Michael Peter Christen | fix for parsing of numeric value in case that boolean values are given Changed Files: source/net/yacy/server/http/TemplateEngine.java |
Fri Apr 04 10:54:56 CEST 2014 by orbiter | fixed thread dump in case of wrong seeds Changed Files: source/net/yacy/search/Switchboard.java |
Thu Apr 03 01:21:43 CEST 2014 by Michael Peter Christen | bugfix to inbound/outbound identification Changed Files: htroot/api/webstructure.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/WebgraphConfiguration.java |
Wed Apr 02 17:10:59 CEST 2014 by orbiter | fix for first start options and added german translation for popup texts Changed Files: htroot/env/templates/header.template, locales/de.lng, source/net/yacy/http/servlets/YaCyDefaultServlet.java |
Wed Apr 02 16:06:29 CEST 2014 by orbiter | fixed crawl start path for cloned crawls Changed Files: htroot/Table_API_p.java |
Wed Apr 02 04:06:35 CEST 2014 by Michael Peter Christen | fix for naming bug Changed Files: htroot/env/templates/submenuMaintenance.template |
Wed Apr 02 02:34:55 CEST 2014 by Michael Peter Christen | npe fix Changed Files: source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/peers/NewsDB.java |
Wed Apr 02 01:16:34 CEST 2014 by Michael Peter Christen | fix for crawler column width Changed Files: htroot/Crawler_p.html |
Tue Apr 01 15:55:20 CEST 2014 by Michael Peter Christen | fix for paths in locale files Changed Files: locales/cn.lng, locales/de.lng, locales/fr.lng, locales/hi.lng, locales/ru.lng, locales/sk.lng, locales/uk.lng |
Tue Apr 01 00:35:58 CEST 2014 by Michael Peter Christen | fixed crawl start links after renaming them in latest commit Changed Files: htroot/ConfigBasic.html, htroot/CrawlStartSite.html, htroot/Status.html, htroot/index.html, htroot/terminal_p.html |
Mon Mar 31 02:58:43 CEST 2014 by reger | fix: 3 more links pointing to old /xml path Changed Files: htroot/api/bookmarks/get_bookmarks.xml, htroot/yacy/ui/yacyui-search.html, htroot/yacy/ui/yacyuisearch.html |
Mon Mar 31 02:48:28 CEST 2014 by reger | fix: blacklist xml export path to xml template Changed Files: htroot/BlacklistImpExp_p.html |
Fri Mar 28 17:25:39 CET 2014 by Michael Peter Christen | fixed visibility of custom icon Changed Files: htroot/HostBrowser.html, htroot/HostBrowser.java, htroot/ViewFile.html, htroot/ViewProfile.html, htroot/ViewProfile.java, htroot/compare_yacy.html, htroot/env/templates/embeddedheader.template, htroot/env/templates/simpleheader.template, htroot/index.html, htroot/yacyinteractive.html, htroot/yacyinteractive.java, htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearch_location.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java |
Fri Mar 28 14:44:52 CET 2014 by Michael Peter Christen | fix for relative sitemap urls Changed Files: htroot/Crawler_p.java |
Fri Mar 28 14:34:49 CET 2014 by Michael Peter Christen | added close to fix possible resource leak warning Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/server/http/HTTPDemon.java, source/net/yacy/server/http/TemplateEngine.java |
Fri Mar 28 03:08:43 CET 2014 by reger | fix: button style IndexReIndexMonitor_p.html Changed Files: htroot/IndexReIndexMonitor_p.html |
Fri Mar 28 02:05:58 CET 2014 by reger | fix: proxyservlet path header setting Changed Files: source/net/yacy/http/ProxyHandler.java, source/net/yacy/http/servlets/YaCyProxyServlet.java |
Thu Mar 27 23:01:28 CET 2014 by Michael Peter Christen | one more Thread Dump fix for new bootstrap css style Changed Files: source/net/yacy/kelondro/logging/ThreadDump.java |
Thu Mar 27 22:13:37 CET 2014 by Michael Peter Christen | fix for thread dump line spacing Changed Files: htroot/Threaddump_p.java, source/net/yacy/kelondro/logging/ThreadDump.java |
Thu Mar 27 13:36:56 CET 2014 by Michael Peter Christen | fixed auto-activation of search field Changed Files: htroot/yacysearch.html |
Wed Mar 26 16:00:05 CET 2014 by Michael Peter Christen | fix for source and target clickdepth in webgraph index Changed Files: source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/WebgraphConfiguration.java |
Mon Mar 24 09:12:42 CET 2014 by Michael Peter Christen | fixed search field Changed Files: htroot/yacysearch.html |
Sat Mar 22 07:02:26 CET 2014 by Michael Peter Christen | misc style bugfixes Changed Files: htroot/env/templates/header.template, htroot/js/yacysearch.js, htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java, locales/de.lng, locales/ru.lng, locales/uk.lng |
Fri Mar 21 16:17:56 CET 2014 by Michael Peter Christen | fix for page navigation bar Changed Files: htroot/js/yacysearch.js, htroot/yacysearch.html, htroot/yacysearch.java |
Fri Mar 21 14:33:42 CET 2014 by Michael Peter Christen | fixes to progess bar Changed Files: htroot/js/yacysearch.js, htroot/yacysearch.html |
Wed Mar 19 12:42:24 CET 2014 by Michael Peter Christen | fixed bold label setting in bootstrap which causes a bad look in YaCy Changed Files: htroot/env/bootstrap-base.css, htroot/index.html, htroot/yacysearch.html |
Tue Mar 18 14:08:52 CET 2014 by Michael Peter Christen | fixes to locale after renaming of some entities in online interface Changed Files: locales/cn.lng, locales/de.lng, locales/ru.lng, locales/uk.lng |
Sun Mar 16 20:51:32 CET 2014 by reger | fix: set YaCy CONNECTION_PROP_HOST Header in ProxyServlet to host incl. port Changed Files: source/net/yacy/http/servlets/YaCyProxyServlet.java |
Sat Mar 15 21:52:42 CET 2014 by reger | prevent exception on Site Crawl if no start url is given Changed Files: htroot/Crawler_p.java |
Tue Mar 11 09:51:04 CET 2014 by Michael Peter Christen | fix for bad/missing values of size_i Changed Files: htroot/HostBrowser.java, htroot/QuickCrawlLink_p.java, htroot/rct_p.java, source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/crawler/retrieval/Request.java, source/net/yacy/crawler/retrieval/Response.java, source/net/yacy/crawler/retrieval/SitemapImporter.java, source/net/yacy/data/ymark/YMarkCrawlStart.java, source/net/yacy/http/ProxyCacheHandler.java, source/net/yacy/http/ProxyHandler.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java, source/net/yacy/server/http/HTTPDProxyHandler.java |
Tue Mar 04 01:36:38 CET 2014 by Michael Peter Christen | fixed another shutdown problem (only occurs if webgraph core is enabled) Changed Files: source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/MirrorSolrConnector.java, source/net/yacy/cora/federate/solr/connector/RemoteSolrConnector.java, source/net/yacy/cora/federate/solr/instance/EmbeddedInstance.java, source/net/yacy/cora/federate/solr/instance/InstanceMirror.java, source/net/yacy/cora/federate/solr/instance/RemoteInstance.java |
Sun Mar 02 21:13:38 CET 2014 by orbiter | bugfix-bugfix Changed Files: source/net/yacy/http/AbstractRemoteHandler.java |
Sun Mar 02 12:10:19 CET 2014 by orbiter | bugfix to proxy handler: recognize the own yacyh-host Changed Files: source/net/yacy/http/AbstractRemoteHandler.java |
Sun Mar 02 11:20:48 CET 2014 by orbiter | NPE fix for Thread name (just commited yesterday, sorry) Changed Files: source/net/yacy/peers/RemoteSearch.java |
Fri Feb 28 02:51:14 CET 2014 by Michael Peter Christen | fixes and enhancements for postprocessing Changed Files: source/net/yacy/search/Switchboard.java, source/net/yacy/search/schema/CollectionConfiguration.java |
Fri Feb 28 01:53:32 CET 2014 by Michael Peter Christen | another fix for the shutdown exceptions Changed Files: source/net/yacy/cora/federate/solr/instance/EmbeddedInstance.java, source/net/yacy/cora/federate/solr/instance/InstanceMirror.java, source/net/yacy/search/index/Fulltext.java |
Thu Feb 27 02:14:46 CET 2014 by reger | fix: de translation of menu link Changed Files: locales/de.lng |
Wed Feb 26 22:47:16 CET 2014 by Michael Peter Christen | fixed shutdown error when closing solr connectors Changed Files: source/net/yacy/cora/federate/solr/instance/InstanceMirror.java |
Tue Feb 25 15:19:04 CET 2014 by Michael Peter Christen | fix for bad query shortcut hack Changed Files: source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/cora/federate/solr/connector/CachedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java, source/net/yacy/cora/federate/solr/connector/MirrorSolrConnector.java |
Tue Feb 25 13:38:39 CET 2014 by Michael Peter Christen | fixed wrong used memory display Changed Files: htroot/Status.java |
Tue Feb 25 02:16:22 CET 2014 by orbiter | fixed error cache Changed Files: source/net/yacy/crawler/Balancer.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/search/index/ErrorCache.java |
Sat Feb 22 01:44:08 CET 2014 by Michael Peter Christen | fixed usage of media flag when started by automated process Changed Files: htroot/Crawler_p.java |
Sat Feb 22 01:35:01 CET 2014 by Michael Peter Christen | fixed cleaning of index Changed Files: source/net/yacy/cora/federate/solr/instance/InstanceMirror.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Fulltext.java |
Fri Feb 21 23:48:56 CET 2014 by Michael Peter Christen | one more SolrIndexSearcher bugfix Changed Files: source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java |
Fri Feb 21 17:28:32 CET 2014 by sixcooler | fix for 'AlreadyClosedException: this IndexReader is closed' Changed Files: source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java |
Tue Feb 11 22:14:46 CET 2014 by Michael Peter Christen | fixed shortcut self-reference bug Changed Files: source/net/yacy/kelondro/rwi/IndexCell.java |
Commit | Description |
---|---|
Fri Apr 04 17:05:03 CEST 2014 by Michael Peter Christen | release 1.7 Changed Files: build.properties |
Fri Apr 04 17:04:49 CEST 2014 by Michael Peter Christen | update of seed list Changed Files: defaults/yacy.network.freeworld.unit |
Fri Apr 04 16:28:57 CEST 2014 by malykhin.dmitry | Update russian translation Changed Files: locales/ru.lng |
Fri Apr 04 15:45:08 CEST 2014 by Michael Peter Christen | update to german locale Changed Files: locales/de.lng |
Fri Apr 04 15:27:29 CEST 2014 by Michael Peter Christen | fixed a time-out problem in the default servlet which is also a logging problem because the error log showed the wrong reason (file not found) instead the actual reason (time-out). Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java |
Fri Apr 04 15:26:39 CEST 2014 by Michael Peter Christen | fixed a problem with resource observer; probably coming from uncatched exceptions within the apache library which appear only in concurrency environments. Changed Files: source/net/yacy/search/ResourceObserver.java |
Fri Apr 04 15:25:37 CEST 2014 by Michael Peter Christen | eliminating the word 'Facet' from the interface because it is ugly. If people do not know what search navigation is, then they also do not know what a 'facet' is. Changed Files: htroot/ConfigSearchPage_p.html, htroot/yacysearchtrailer.html, locales/cn.lng, locales/de.lng, locales/ru.lng, locales/uk.lng |
Fri Apr 04 14:43:54 CEST 2014 by Michael Peter Christen | enhanced the new link structure graph Changed Files: htroot/api/linkstructure.java, htroot/env/hypertree.css, htroot/js/hypertree.js, source/net/yacy/search/schema/HyperlinkEdge.java |
Fri Apr 04 12:48:55 CEST 2014 by Michael Peter Christen | better link structure limit calibration Changed Files: htroot/Crawler_p.html, htroot/HostBrowser.html |
Fri Apr 04 10:55:46 CEST 2014 by orbiter | better logging in template engine: shows filename of servlets where errors in templates occur Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/server/http/HTTPDemon.java, source/net/yacy/server/http/TemplateEngine.java |
Fri Apr 04 10:54:11 CEST 2014 by orbiter | moved Edge class out of linkstructure servlet as this does not work on non-eclipse driven environments (all non-dev cases) Changed Files: htroot/api/linkstructure.java, source/net/yacy/search/schema/HyperlinkEdge.java |
Fri Apr 04 10:08:44 CEST 2014 by orbiter | reduced search activity corona strength in network image Changed Files: source/net/yacy/peers/graphics/NetworkGraph.java |
Fri Apr 04 03:11:16 CEST 2014 by reger | move baseurl after parsing orig source in urlproxyservlet to calculate absolute href links for rewrite from unmodified source. Changed Files: source/net/yacy/http/servlets/UrlProxyServlet.java |
Fri Apr 04 02:45:04 CEST 2014 by reger | fix: remove obsolete ref to yacy.home (use Switchboard instead) Changed Files: source/net/yacy/document/parser/rdfa/impl/RDFaTripleImpl.java, source/net/yacy/yacy.java |
Thu Apr 03 13:42:10 CEST 2014 by Michael Peter Christen | fix for solr query size; too many documents had been retrieved in case that less than _pagesize_ had been requested. Changed Files: source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java |
Thu Apr 03 00:35:15 CEST 2014 by reger | eleminate obsolete URIMetaDataRow class by joining it with/into URIMetaDataNode. Changed Files: htroot/yacy/crawlReceipt.java, htroot/yacy/transferURL.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/peers/Protocol.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/schema/CollectionConfiguration.java |
Wed Apr 02 23:37:01 CEST 2014 by Michael Peter Christen | introduced new solr field crawldepth_i which records the crawl depth of a document. This is the upper limit for the clickdepth_i value which may be shorter in case that the crawler did not take the shortest path to the document. Changed Files: defaults/solr.collection.schema, htroot/HostBrowser.java, source/net/yacy/document/Document.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/CollectionSchema.java |
Wed Apr 02 17:37:21 CEST 2014 by Michael Peter Christen | added missing servlet html Changed Files: htroot/ContentAnalysis_p.html |
Wed Apr 02 16:05:44 CEST 2014 by orbiter | - added COLS field in GSA response (non-gsa standard by customer request) - updated document link in GSA response writer Changed Files: source/net/yacy/cora/federate/solr/responsewriter/GSAResponseWriter.java, source/net/yacy/http/servlets/GSAsearchServlet.java |
Wed Apr 02 13:33:43 CEST 2014 by Michael Peter Christen | - added popover to p2p/stealth mode button to explain the peer mode and privacy issues. - added popover to first-time use case to explain that specific servlets are only visible after customization and/or crawl starts Changed Files: htroot/env/bootstrap-base.css, htroot/env/templates/header.template, htroot/test.html, htroot/yacysearchtrailer.html, source/net/yacy/http/servlets/YaCyDefaultServlet.java |
Wed Apr 02 12:19:26 CEST 2014 by Michael Peter Christen | transition.js is also included in bootstrap.js Changed Files: |
Wed Apr 02 12:18:33 CEST 2014 by Michael Peter Christen | removed alert.js, modal.js and tooltip.js as these libraries are all included in bootstrap.js Changed Files: |
Wed Apr 02 03:21:51 CEST 2014 by reger | fix: application link count (URIMetadataNode) Changed Files: source/net/yacy/kelondro/data/meta/URIMetadataNode.java |
Wed Apr 02 02:34:39 CEST 2014 by Michael Peter Christen | free configuration of postprocessing clickdepth maximum depth and time Changed Files: source/net/yacy/cora/document/id/DigestURL.java, source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/schema/CollectionConfiguration.java |
Wed Apr 02 02:33:17 CEST 2014 by Michael Peter Christen | added tutorial mode - some menu items will only appear if you 'qualify' for them. Thus, the first-time user will only see four menu items. The other items will unfold as the user interacts. Changed Files: defaults/yacy.init, htroot/Status.java, htroot/env/templates/header.template, source/net/yacy/http/servlets/YaCyDefaultServlet.java |
Wed Apr 02 01:24:15 CEST 2014 by sixcooler | fix receiving DHT / parse pultipart + another close to fix possible resource leak warning Changed Files: source/net/yacy/server/http/HTTPDemon.java |
Tue Apr 01 20:01:22 CEST 2014 by malykhin.dmitry | Update russian translation Changed Files: locales/ru.lng |
Tue Apr 01 17:18:26 CEST 2014 by Michael Peter Christen | added more bootstrap libraries Changed Files: htroot/env/bootstrap/js/alert.js, htroot/env/bootstrap/js/modal.js, htroot/env/bootstrap/js/tooltip.js, htroot/env/bootstrap/js/transition.js |
Tue Apr 01 16:55:22 CEST 2014 by Michael Peter Christen | show host list by default in host browser Changed Files: htroot/env/templates/header.template |
Tue Apr 01 04:02:34 CEST 2014 by reger | make use of detected charset in htmlParser if none is given. Changed Files: source/net/yacy/document/parser/htmlParser.java |
Tue Apr 01 02:55:22 CEST 2014 by reger | add test case htmlParser.parse - getCharset (which fails) Changed Files: test/net/yacy/document/parser/htmlParserTest.java |
Mon Mar 31 20:42:39 CEST 2014 by orbiter | made crawl start pages public since they do not reveal individual information and they are also not used as servlet to actually start the crawl (which is Crawler_p.html). Changed Files: htroot/CrawlStartExpert.html, htroot/CrawlStartExpert.java, htroot/CrawlStartSite.html, htroot/CrawlStartSite.java, htroot/env/templates/header.template, htroot/env/templates/submenuCrawler.template, htroot/env/templates/submenuIndexCreate.template |
Mon Mar 31 18:25:11 CEST 2014 by Michael Peter Christen | added (again) underline to a tags Changed Files: htroot/env/base.css |
Mon Mar 31 18:19:24 CEST 2014 by Michael Peter Christen | Integrated HostBrowser back to administration interface; it can appear with and without navigation bar. Changed Files: htroot/HostBrowser.html, htroot/HostBrowser.java, htroot/env/templates/header.template, htroot/env/templates/submenuWebStructure.template |
Mon Mar 31 16:00:13 CEST 2014 by Michael Peter Christen | added template for latest commit Changed Files: htroot/env/templates/submenuIndexImport.template |
Sun Mar 30 05:11:42 CEST 2014 by reger | adding a hint to javascript browser short cut on Url-Proxy page (AugmentedBrowsing_p.html) Changed Files: htroot/AugmentedBrowsing_p.html, source/net/yacy/http/servlets/UrlProxyServlet.java |
Sun Mar 30 00:23:54 CET 2014 by reger | fix: cut-off button in WatchWebStructure_p.html (by header css dd hight/line-hight) Changed Files: htroot/WatchWebStructure_p.html |
Sat Mar 29 00:34:01 CET 2014 by reger | fix: port in local target adress & button style Changed Files: htroot/AugmentedBrowsing_p.html, htroot/AugmentedParsing_p.html, source/net/yacy/peers/SeedDB.java |
Fri Mar 28 14:31:43 CET 2014 by Michael Peter Christen | prevent opening of new files as that could be a cause for the latest too-many-open-files exception. The old file is just truncated if the table is cleaned. Changed Files: source/net/yacy/kelondro/io/BufferedRecords.java, source/net/yacy/kelondro/io/Records.java, source/net/yacy/kelondro/table/Table.java |
Thu Mar 27 23:36:08 CET 2014 by Michael Peter Christen | added linkscount_i and linksnofollowcount_i to the default solr schema Changed Files: defaults/solr.collection.schema, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/CollectionSchema.java |
Thu Mar 27 22:00:57 CET 2014 by Michael Peter Christen | removed performance settings - they are incomplete and buggy - it was not easy to explain - it did not comply with a KISS strategy - setting a performance of low priority actually caused crashing of a peer - there was nobody who would maintain that functionality Changed Files: defaults/solr.collection.schema, htroot/PerformanceQueues_p.java, htroot/Performance_p.html |
Thu Mar 27 13:50:13 CET 2014 by Michael Peter Christen | changed the protocol navigator: turned into switch buttons Changed Files: htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java |
Thu Mar 27 13:29:14 CET 2014 by Michael Peter Christen | changed protocol facet to show also all other counts if one facet is selected Changed Files: htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchlatestinfo.java, htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java, source/net/yacy/search/query/QueryGoal.java, source/net/yacy/search/query/QueryModifier.java, source/net/yacy/search/query/QueryParams.java |
Thu Mar 27 12:03:51 CET 2014 by Michael Peter Christen | moved host facet up Changed Files: htroot/yacysearchtrailer.html |
Thu Mar 27 11:56:33 CET 2014 by Michael Peter Christen | added typeahead library to re-implement search suggestions for bootstrap framework. Changed Files: htroot/env/bootstrap-base.css, htroot/env/bootstrap/js/typeahead.jquery.js, htroot/env/bootstrap/js/typeahead.jquery.min.js, htroot/index.html, htroot/suggest.java, htroot/yacysearch.html |
Wed Mar 26 23:28:19 CET 2014 by reger | remove unused private header entries (HeaderFramework) X_YACY_ORIGINAL_REQUEST_LINE X_YACY_KEEP_ALIVE_REQUEST_COUNT CONNECTION_PROP_REQUESTLINE Changed Files: source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/http/servlets/YaCyProxyServlet.java, source/net/yacy/server/http/HTTPDProxyHandler.java, source/net/yacy/server/http/HTTPDemon.java |
Wed Mar 26 20:01:15 CET 2014 by sixcooler | make transparent proxy handle https-connections: the implemented handle for connect did not work for me - so lets try the connectHandler Changed Files: source/net/yacy/http/CrashProtectionHandler.java, source/net/yacy/http/Jetty8HttpServerImpl.java, source/net/yacy/http/ProxyHandler.java |
Wed Mar 26 16:39:09 CET 2014 by malykhin.dmitry | Update russian translation Changed Files: locales/ru.lng |
Wed Mar 26 10:54:23 CET 2014 by Michael Peter Christen | - added (previously removed) autocomplete - removed autocomplete from yacyinteractive.html - todo: replace old jquery-1.3.2.min.js with latest which is already a part of bootstrap.css Changed Files: htroot/index.html, htroot/yacyinteractive.html, htroot/yacysearch.html |
Tue Mar 25 22:14:54 CET 2014 by Marc Nause | Team added support for URLs with unicode characters in host part to blacklist. Punycode is used to handle unicode characters. Changed Files: htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.java, htroot/CrawlResults.java, htroot/IndexControlRWIs_p.java, source/net/yacy/cora/document/id/MultiProtocolURL.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/search/snippet/MediaSnippet.java |
Mon Mar 24 09:52:27 CET 2014 by Michael Peter Christen | corrected line-height in tagcloud Changed Files: htroot/env/base.css, htroot/yacy/ui/css/base.css, htroot/yacysearchtrailer.html |
Mon Mar 24 02:32:17 CET 2014 by reger | add some InputFileStream close at end of reads to make sure file is released Changed Files: source/net/yacy/cora/document/WordCache.java, source/net/yacy/cora/storage/KeyList.java, source/net/yacy/gui/framework/Switchboard.java, source/net/yacy/yacy.java |
Sun Mar 23 23:10:29 CET 2014 by reger | limit filetype nav to known extension also on image/media search - on text search we limit filetype nav already to known extension, apply filter to image search Changed Files: source/net/yacy/search/query/SearchEvent.java |
Sun Mar 23 17:28:34 CET 2014 by orbiter | fix for embedded header: the new css framework always need a headline, which is the location for the search input frame. The new embedded header is a clone of the simpleHeader with removed menus and without a link to the index administration. Changed Files: htroot/env/templates/embeddedfooter.template, htroot/env/templates/embeddedheader.template, htroot/env/templates/simplefooter.template |
Sun Mar 23 16:32:47 CET 2014 by malykhin.dmitry | Update russian translation Changed Files: locales/ru.lng |
Sun Mar 23 13:01:25 CET 2014 by Michael Peter Christen | added /date -ranking button Changed Files: htroot/yacysearchtrailer.html |
Sun Mar 23 12:29:46 CET 2014 by Michael Peter Christen | - added text/image/audio/video/app search option to new navigation bar - changed colors of privacy selector Changed Files: htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java |
Sun Mar 23 11:39:46 CET 2014 by Michael Peter Christen | activated language facet by default Changed Files: defaults/yacy.init |
Sun Mar 23 09:32:42 CET 2014 by Michael Peter Christen | from the administration search bar, do global searches Changed Files: htroot/env/templates/header.template |
Sun Mar 23 00:40:36 CET 2014 by reger | surrogate import translate dc:language 3-char codes OAI records often use 3-char language codes, start converting some 3-char lang's to the internal ISO639-1 2-char code Changed Files: source/net/yacy/document/content/DCEntry.java |
Sat Mar 22 21:01:20 CET 2014 by reger | fix: set text color black in select field in table.th in CrawlResults (default inherite white) Changed Files: htroot/CrawlResults.html |
Sat Mar 22 05:45:44 CET 2014 by Michael Peter Christen | - do not show external link icon on navbar icon - do not show navbar icon on index.html Changed Files: htroot/env/base.css, htroot/env/templates/header.template, htroot/env/templates/simpleheader.template, htroot/index.html |
Fri Mar 21 20:28:34 CET 2014 by orbiter | restrict write buffer size to a limit Changed Files: source/net/yacy/cora/federate/solr/instance/RemoteInstance.java |
Fri Mar 21 20:28:06 CET 2014 by orbiter | bring api icons to the front Changed Files: htroot/env/base.css |
Fri Mar 21 18:01:26 CET 2014 by Michael Peter Christen | - added stealth button to navigation menu - more fixes to progress bar Changed Files: htroot/yacysearch.html, htroot/yacysearch.java, htroot/yacysearchitem.java, htroot/yacysearchlatestinfo.java, htroot/yacysearchtrailer.html, htroot/yacysearchtrailer.java, source/net/yacy/search/query/QueryParams.java |
Fri Mar 21 03:02:19 CET 2014 by reger | remove check for unused virtual path /currentyacypeer/ - del jqueryheader.template (not used) Changed Files: source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/server/http/HTTPDProxyHandler.java |
Fri Mar 21 00:04:22 CET 2014 by reger | ConfigSearchPage.html first adjustment to new layout Changed Files: htroot/ConfigSearchPage_p.html |
Thu Mar 20 22:11:49 CET 2014 by orbiter | added adminAccount switch to ConfigAccounts_p servlet to switch on protection of all pages; some refactoring as well Changed Files: htroot/ConfigAccounts_p.html, htroot/ConfigAccounts_p.java, htroot/ConfigPortal.java, htroot/HostBrowser.java, htroot/yacysearch.java, source/net/yacy/http/Jetty8YaCySecurityHandler.java, source/net/yacy/http/servlets/GSAsearchServlet.java, source/net/yacy/search/SwitchboardConstants.java |
Thu Mar 20 22:10:22 CET 2014 by orbiter | added bootstrap-switch to turn checkboxes into toggle switches Changed Files: htroot/env/bootstrap/css/bootstrap-switch.css, htroot/env/bootstrap/css/bootstrap-switch.min.css, htroot/env/bootstrap/js/bootstrap-switch.js, htroot/env/bootstrap/js/bootstrap-switch.min.js, htroot/env/templates/footer.template, htroot/env/templates/metas.template |
Thu Mar 20 19:09:47 CET 2014 by orbiter | added flag to require that all web pages, even such without a "_p" extension require authorization. (default off) Changed Files: defaults/yacy.init, source/net/yacy/http/Jetty8YaCySecurityHandler.java, source/net/yacy/search/SwitchboardConstants.java |
Thu Mar 20 02:34:52 CET 2014 by malykhin.dmitry | Update russian translation Changed Files: locales/ru.lng |
Thu Mar 20 02:12:56 CET 2014 by Michael Peter Christen | reduce number of facets to 10 Changed Files: htroot/yacysearchtrailer.java |
Thu Mar 20 02:10:27 CET 2014 by Michael Peter Christen | replaced old navbar with bootstrap pagination Changed Files: htroot/js/yacysearch.js, htroot/yacysearch.java |
Wed Mar 19 22:40:35 CET 2014 by reger | remove AugmentedBrowsing_p.html augmented browsing switch (has no function in code, previously used in conjuction with http://reflect.ws) Changed Files: defaults/yacy.init, htroot/AugmentedBrowsing_p.html, htroot/AugmentedBrowsing_p.java |
Wed Mar 19 16:32:32 CET 2014 by Michael Peter Christen | enhanced api menu entries Changed Files: htroot/env/templates/header.template, htroot/env/templates/simpleheader.template |
Wed Mar 19 16:21:03 CET 2014 by Michael Peter Christen | removed synchronization in embedded solr connection (that was probably a mistake?) Changed Files: source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java |
Wed Mar 19 16:20:17 CET 2014 by Michael Peter Christen | redesign of menus; usage of bootstrap buttons and dropdown-menus Changed Files: htroot/ViewFile.html, htroot/compare_yacy.html, htroot/env/base.css, htroot/env/templates/header.template, htroot/env/templates/simpleheader.template, htroot/test.html, htroot/yacysearch.html |
Wed Mar 19 12:54:56 CET 2014 by Michael Peter Christen | added dropdown menu in test dashboard Changed Files: htroot/env/bootstrap-base.css, htroot/test.html |
Wed Mar 19 12:23:40 CET 2014 by Michael Peter Christen | fixed shutdown of solr cores in case that more than one local core is to be closed (this happens if webgraph is enabled and the index is dumped using /IndexControlURLs_p.html Changed Files: source/net/yacy/cora/federate/solr/instance/InstanceMirror.java |
Wed Mar 19 12:19:58 CET 2014 by Michael Peter Christen | use default header for user front-ends and inverted header for administration front-end Changed Files: htroot/env/bootstrap-base.css, htroot/env/templates/header.template, htroot/env/templates/simpleheader.template |
Wed Mar 19 11:55:48 CET 2014 by Michael Peter Christen | enlarged facet column slightly Changed Files: htroot/yacysearch.html |
Wed Mar 19 09:29:01 CET 2014 by Michael Peter Christen | removed unused variables and warnings Changed Files: htroot/ConfigHeuristics_p.java, htroot/Connections_p.java, htroot/yacy/ui/yacyui-welcome.html, source/net/yacy/document/content/DCEntry.java, source/net/yacy/server/http/HTTPDFileHandler.java, source/net/yacy/server/http/HTTPDProxyHandler.java |
Tue Mar 18 20:12:33 CET 2014 by malykhin.dmitry | Update russian translate Changed Files: locales/ru.lng |
Tue Mar 18 16:56:05 CET 2014 by orbiter | removed superfluous import of portalheader.template Changed Files: htroot/HostBrowser.html, htroot/env/templates/metas.template |
Mon Mar 17 20:27:04 CET 2014 by reger | remove AugmentedBrowsingFilters_p.html as none of the settings are used currently config settings frome the page also removed from yacy.init augmentation.reflect augmentation.addDoctype augmentation.reparse interaction.overlayinteraction.enabled Changed Files: defaults/yacy.init, htroot/env/templates/submenuSemantic.template |
Sun Mar 16 22:11:19 CET 2014 by reger | fix: remove link to triplestore in Vocabulary_p (triplestore does not longer exist) - should be investigated in more detail to look for additional implications Remove "yacyaction" from proxyservlet as it was only needed for removed interaction routines. Changed Files: htroot/Vocabulary_p.html, source/net/yacy/http/servlets/YaCyProxyServlet.java |
Sun Mar 16 14:56:30 CET 2014 by orbiter | tried javascript hack to make statistic divs equal height Changed Files: htroot/Crawler_p.html |
Sun Mar 16 05:19:01 CET 2014 by reger | remove unused ProxySettings ref. from loader clean unused whois test code Changed Files: source/net/yacy/utils/loaderThreads.java |
Sun Mar 16 00:15:12 CET 2014 by reger | cleanup: remove unrelated and unused code Changed Files: |
Sat Mar 15 23:02:17 CET 2014 by reger | upd pom to latest jars Changed Files: pom.xml |
Fri Mar 14 13:33:23 CET 2014 by Michael Peter Christen | support for multiple sitemaps in robots.txt Changed Files: htroot/CrawlCheck_p.java, htroot/api/getpageinfo.java, htroot/api/getpageinfo.xml, htroot/api/getpageinfo_p.java, htroot/api/getpageinfo_p.xml, source/net/yacy/crawler/robots/RobotsTxtEntry.java, source/net/yacy/crawler/robots/RobotsTxtParser.java |
Fri Mar 14 03:12:04 CET 2014 by reger | remove more unused from legacy server.http - triggerOnlineAction not used - useTemplateCache not used Changed Files: defaults/yacy.init, source/net/yacy/peers/Network.java, source/net/yacy/server/http/HTTPDFileHandler.java, source/net/yacy/server/http/HTTPDProxyHandler.java |
Fri Mar 14 02:24:12 CET 2014 by reger | remove redundant url string manipulation in HTTPDProxyHandler (still used by ProxyServlet) Changed Files: source/net/yacy/server/http/HTTPDProxyHandler.java |
Thu Mar 13 02:46:05 CET 2014 by reger | use servletPath in ProxyServlet instead of fixed name to allow servlet-mapping via web.xml Changed Files: source/net/yacy/http/servlets/YaCyProxyServlet.java |
Wed Mar 12 23:35:57 CET 2014 by reger | fix DCEntry.getIdentifier on successful url parameter Changed Files: source/net/yacy/document/content/DCEntry.java |
Sun Mar 09 03:15:35 CET 2014 by reger | OAI import get multivalued keywords (dc:subject) Changed Files: source/net/yacy/document/content/DCEntry.java |
Fri Mar 07 20:06:14 CET 2014 by reger | allow YaCy to start w/o ssl if JKS init fails Changed Files: source/net/yacy/http/Jetty8HttpServerImpl.java |
Fri Mar 07 17:47:09 CET 2014 by sixcooler | Speed up the ProxyHandler: simplified cache-storing and make it concurrent in order to free the clientconnection asap let other prozesses wait on proxy-access like it was bevore Changed Files: source/net/yacy/http/ProxyHandler.java |
Fri Mar 07 01:21:38 CET 2014 by Michael Peter Christen | fix for facets in search rss result: some facets had been missing, all had missing url properties Changed Files: htroot/yacysearchtrailer.xml |
Fri Mar 07 00:16:57 CET 2014 by reger | fix: to allow unix scripts (bin/*.sh) to allways submit http admin apicalls using auth via config hash (legacy requirement) Changed Files: source/net/yacy/http/YaCyLegacyCredential.java |
Thu Mar 06 03:08:04 CET 2014 by reger | cleanup old reference to HTTPDemon.setAlternativeResolver optimize .yacyh check in AbstractRemoteHandler Changed Files: source/net/yacy/crawler/retrieval/HTTPLoader.java, source/net/yacy/http/AbstractRemoteHandler.java, source/net/yacy/peers/SeedDB.java, source/net/yacy/server/http/HTTPDProxyHandler.java, source/net/yacy/server/http/HTTPDemon.java |
Wed Mar 05 23:28:19 CET 2014 by Michael Peter Christen | enhanced RowSet re-sort limit for small sets Changed Files: source/net/yacy/kelondro/index/RowSet.java |
Wed Mar 05 15:58:15 CET 2014 by malykhin.dmitry | Update russian translation Changed Files: htroot/CrawlResults.java, source/org/apache/tools/tar/TarBuffer.java |
Wed Mar 05 15:53:23 CET 2014 by malykhin.dmitry | Update russian translation Changed Files: htroot/CrawlResults.java, locales/ru.lng, source/org/apache/tools/tar/TarBuffer.java |
Wed Mar 05 02:59:27 CET 2014 by reger | fix: make sure adminAccount changes made via ConfigAccounts_p are effective immediately force to remove current credentials from knownuser cache Changed Files: htroot/ConfigAccounts_p.java, source/net/yacy/http/Jetty8HttpServerImpl.java |
Tue Mar 04 18:37:00 CET 2014 by malykhin.dmitry | Update russian translation Changed Files: locales/ru.lng |
Tue Mar 04 11:24:04 CET 2014 by orbiter | add index on _val fields, this affects especially title length an index on fields make search facets on that field possible Changed Files: defaults/solr/schema.xml |
Tue Mar 04 04:19:35 CET 2014 by malykhin.dmitry | Update russian translation Changed Files: locales/ru.lng |
Tue Mar 04 03:08:37 CET 2014 by reger | improve OAI-PMH import identifier recognition - find best fittng identifier (url) by checking all given dc:identifier in record (many entries proviede several identifiers) as identifier is currently a multivalued field use "getParams" in preference of splitting the 1st string by ";" - add resolve DOI:... identifier via http://dx.doi.org/ Changed Files: htroot/IndexImportOAIPMHList_p.java, source/net/yacy/document/content/DCEntry.java |
Tue Mar 04 00:33:13 CET 2014 by Michael Peter Christen | - let crawl loader jobs die after 10 seconds without new jobs - corrected shutdown order t prevent a deadlock during shutdown Changed Files: source/net/yacy/cora/protocol/Domains.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/search/Switchboard.java |
Mon Mar 03 22:44:55 CET 2014 by Michael Benz | Added Index Deletion and Reindex Monitor to German translation Changed Files: locales/de.lng |
Mon Mar 03 04:16:53 CET 2014 by malykhin.dmitry | Update russian translation Changed Files: locales/ru.lng |
Sun Mar 02 23:46:24 CET 2014 by Michael Benz | Added German translation for Config Search and Ranking RWI and Solr Changed Files: htroot/ConfigSearchPage_p.html, htroot/RankingRWI_p.html, locales/de.lng |
Sun Mar 02 06:13:21 CET 2014 by reger | fix: IPv6 INTRANET_PATTERNS for local ip (see http://bugs.yacy.net/view.php?id=378) requiring following ":" for fc and fd prefix and made pattern match case insesitive - add some more ipv6 test cases to MultiProtocolURLTest.java Changed Files: source/net/yacy/cora/protocol/Domains.java, test/net/yacy/cora/document/id/MultiProtocolURLTest.java |
Sat Mar 01 23:50:35 CET 2014 by reger | fix: unused / incorrect default username parameter (removed setting) Changed Files: htroot/Settings_p.java |
Sat Mar 01 17:28:36 CET 2014 by malykhin.dmitry | Update russian translation Changed Files: locales/ru.lng |
Sat Mar 01 13:37:17 CET 2014 by Michael Benz | German translation for Augmented Browser, Parser, Content Control and Vocabulary Changed Files: locales/de.lng |
Sat Mar 01 11:32:14 CET 2014 by Michael Benz | Improved Image Collage and WatchWebStructure page display with CSS changes Changed Files: htroot/Collage.html, htroot/WatchWebStructure_p.html |
Sat Mar 01 11:31:23 CET 2014 by Michael Benz | Added German translation for AccessGrid_p.html, CrawlCheck_p.html, RegexTest.html and env/templates/submenuTargetAnalysis.template Changed Files: htroot/CrawlCheck_p.html, htroot/RegexTest.html, locales/de.lng |
Sat Mar 01 00:56:45 CET 2014 by reger | status panel link to set max mem +url proxy same error text as in transparent Changed Files: htroot/Status_p.inc, source/net/yacy/http/servlets/YaCyProxyServlet.java |
Fri Feb 28 21:03:23 CET 2014 by malykhin.dmitry | update russian translation Changed Files: locales/ru.lng |
Fri Feb 28 13:58:05 CET 2014 by Michael Peter Christen | small change to crawler monitor to fit in larger translations Changed Files: htroot/Crawler_p.html, htroot/js/Crawler.js |
Fri Feb 28 03:14:07 CET 2014 by reger | little more verbose proxy 403 error message Changed Files: source/net/yacy/http/AbstractRemoteHandler.java |
Fri Feb 28 00:19:30 CET 2014 by orbiter | (again) full redesign of ConcurrentUpdateSolrConnector to remove out-of-order transactions regarding add and delete operations. Now all operations (add and delete) are executed concurrently in-order. Changed Files: source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java |
Fri Feb 28 00:17:50 CET 2014 by orbiter | better height for statistic windows Changed Files: htroot/Crawler_p.html |
Fri Feb 28 00:11:20 CET 2014 by orbiter | better width for Progress table Changed Files: htroot/Crawler_p.html |
Thu Feb 27 23:30:56 CET 2014 by orbiter | fix for switch back to localhost (why did this not work? it worked all the time) Changed Files: htroot/ConfigAccounts_p.java |
Thu Feb 27 22:48:17 CET 2014 by reger | clean up test sources rename to current package names and move to default location Changed Files: test/net/yacy/cora/document/id/DigestURLTest.java, test/net/yacy/cora/document/id/MultiProtocolURLTest.java, test/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnectorTest.java, test/net/yacy/cora/protocol/HeaderFrameworkTest.java, test/net/yacy/document/ParserTest.java, test/net/yacy/document/parser/htmlParserTest.java, test/net/yacy/peers/operation/yacyVersionTest.java |
Thu Feb 27 18:28:40 CET 2014 by Michael Benz | Updated German translation with translated submenu include env/templates/submenuComputation.template Changed Files: locales/de.lng |
Thu Feb 27 12:50:40 CET 2014 by Michael Peter Christen | added a forced commit to index deletion to enable synchronized index updates Changed Files: htroot/IndexDeletion_p.java, source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java |
Thu Feb 27 12:27:15 CET 2014 by Michael Peter Christen | make sure that postprocessed documents are overwritten Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java |
Thu Feb 27 04:43:17 CET 2014 by malykhin.dmitry | update russian lang-file and small improve web-interface Changed Files: htroot/Crawler_p.html, htroot/IndexControlURLs_p.html, locales/ru.lng |
Thu Feb 27 01:27:06 CET 2014 by Michael Peter Christen | a concurrency enhancement which was not used because tests showed worse indexing speed. I leave the code there since it may be useful in SolrCloud environments. Changed Files: source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java |
Thu Feb 27 00:34:13 CET 2014 by Michael Peter Christen | added one more postprocessing low memory check Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java |
Thu Feb 27 00:32:39 CET 2014 by Michael Peter Christen | added a properties file format for the status_p api to support reading of that information with the java Properties class (very easy for small clients) Changed Files: htroot/api/status_p.properties |
Thu Feb 27 00:31:13 CET 2014 by Michael Peter Christen | extended the status_p api with disk space information Changed Files: htroot/api/status_p.java, htroot/api/status_p.xml |
Thu Feb 27 00:24:30 CET 2014 by reger | Merge origin/master Changed Files: source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/cora/federate/solr/instance/InstanceMirror.java, source/net/yacy/peers/Protocol.java, source/net/yacy/search/index/ReindexSolrBusyThread.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/schema/CollectionConfiguration.java |
Thu Feb 27 00:23:34 CET 2014 by reger | use config value htroot in Jetty init (was hardcoded) - move htroot exist check from old httpdfilehandler to startup, remove from filehandler and legacy proxyhandler - use SwitchboardConstant.htroot where appropriate Changed Files: source/net/yacy/data/Translator.java, source/net/yacy/http/Jetty8HttpServerImpl.java, source/net/yacy/http/servlets/YaCyDefaultServlet.java, source/net/yacy/server/http/HTTPDFileHandler.java, source/net/yacy/server/http/HTTPDProxyHandler.java, source/net/yacy/server/http/HTTPDemon.java, source/net/yacy/yacy.java |
Wed Feb 26 23:02:56 CET 2014 by Michael Peter Christen | added short memory protection during postprocessing Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java |
Wed Feb 26 22:21:00 CET 2014 by Michael Peter Christen | removed double concurrency to put Solr documents into the index. The writings to the solr index are also buffered in ConcurrentUpdateSolrConnector Changed Files: source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/peers/Protocol.java, source/net/yacy/search/index/ReindexSolrBusyThread.java, source/net/yacy/search/index/Segment.java |
Wed Feb 26 17:12:53 CET 2014 by sixcooler | fix for switching 'simulate short memory status' and 'Memory Strategy' thx Thomas Changed Files: htroot/PerformanceMemory_p.html |
Wed Feb 26 02:01:40 CET 2014 by reger | reorg URL test classes add isLocal test with some IPv6 examples - putting in default location and clean old package names - add some valid RFC IPv6 sample urls (which don't pass the isLocal test) Changed Files: test/net/yacy/cora/document/id/DigestURLTest.java, test/net/yacy/cora/document/id/MultiProtocolURLTest.java |
Tue Feb 25 16:35:11 CET 2014 by malykhin.dmitry | edit russian locale Changed Files: locales/ru.lng |
Tue Feb 25 15:23:45 CET 2014 by Michael Peter Christen | recover from OOM Changed Files: source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java |
Tue Feb 25 14:37:30 CET 2014 by Michael Peter Christen | - better logging for postprocessing - fixed collection bug in postprocessing Changed Files: source/net/yacy/search/Switchboard.java, source/net/yacy/search/schema/CollectionConfiguration.java |
Tue Feb 25 14:17:33 CET 2014 by Michael Peter Christen | as crawling has a low-cpu demand, we want it to run even if the CPU load is VERY high. This applies also if the CPU load is high because of in-cache crawling; in that case we want to experience a high-CPU load as much as possible Changed Files: defaults/yacy.init |
Tue Feb 25 03:26:20 CET 2014 by orbiter | prevent npe Changed Files: htroot/HostBrowser.java, source/net/yacy/crawler/CrawlStacker.java |
Tue Feb 25 03:19:41 CET 2014 by orbiter | prevent npe Changed Files: source/net/yacy/crawler/CrawlStacker.java |
Tue Feb 25 00:23:06 CET 2014 by reger | fix: use timeout = proxy.ClientTimeout in ProxyHandler (was 10sec fix) see http://bugs.yacy.net/view.php?id=236 Changed Files: source/net/yacy/http/ProxyHandler.java |
Mon Feb 24 23:47:12 CET 2014 by Michael Benz | Updated German translation and Blacklist_p.html Changed Files: htroot/Blacklist_p.html, locales/de.lng |
Mon Feb 24 23:42:50 CET 2014 by orbiter | - introduced a miss cache in ConcurrentUpdateSolrConnector - better usage of cache - bugfix for postprocessing Changed Files: source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/schema/CollectionConfiguration.java |
Mon Feb 24 23:19:17 CET 2014 by reger | make use of new -config cmd-line parameter in reconfgureYACY.sh to asure pwd encoding is compatible with DIGEST auth. in future. Changed Files: reconfigureYACY.sh |
Mon Feb 24 22:59:58 CET 2014 by orbiter | changed caching of ConcurrentUpdateSolrConnector: it caches now also the url along with the load date. While this takes much more memory, it eliminates database lookups for getURL() requests, which happen equally often. This speeds up remote solr configurations. Changed Files: source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/MirrorSolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrConnector.java, source/net/yacy/search/index/ErrorCache.java, source/net/yacy/search/index/Fulltext.java |
Mon Feb 24 21:01:56 CET 2014 by orbiter | refactoring Changed Files: htroot/HostBrowser.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/api/ymarks/add_ymark.java, htroot/yacysearch.java, source/net/yacy/data/ymark/YMarkMetadata.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/index/Segment.java |
Mon Feb 24 20:51:43 CET 2014 by orbiter | removed unused code (HostStat) Changed Files: source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/ranking/BlockRank.java |
Mon Feb 24 20:26:02 CET 2014 by orbiter | introducing ConcurrentUpdateSolrServer for remote solr servers. Scaling of write buffers and update queue size is made according to assigned memory. Changed Files: source/net/yacy/cora/federate/solr/instance/InstanceMirror.java, source/net/yacy/cora/federate/solr/instance/RemoteInstance.java |
Mon Feb 24 20:02:15 CET 2014 by orbiter | removed host increment on stored documents from crawler (that was wrong) Changed Files: htroot/HostBrowser.java |
Mon Feb 24 13:50:28 CET 2014 by malykhin.dmitry | edit russian locale Changed Files: locales/ru.lng |
Mon Feb 24 05:19:40 CET 2014 by reger | remove unused java imports form yacy.java Changed Files: source/net/yacy/yacy.java |
Mon Feb 24 05:16:31 CET 2014 by reger | add a -config command line parameter e.g. -config "port=9090" "port.ssl=8043" - useful for remote installation to set any config file property - multipe parameter can be set at once, on Windows enclose parameter in doublequotes - special handling "adminAccount=adminuser:adminpwd" sets adminusername and md5 encoded admin-pwd - adjusted windows startbatch to allow command line parameter handling - remove not needed classpath calculation from startYACY_debug.bat Changed Files: source/net/yacy/yacy.java, startYACY.bat, startYACY_debug.bat |
Mon Feb 24 04:38:28 CET 2014 by Michael Benz | Finished translation of changed CrawlStartExpert_p.html Changed Files: locales/de.lng |
Mon Feb 24 00:01:11 CET 2014 by reger | fix: allow 4 character admin user name (was min 5 char) Changed Files: source/net/yacy/search/Switchboard.java |
Sat Feb 22 04:42:05 CET 2014 by malykhin.dmitry | edit russian locale Changed Files: locales/ru.lng |
Sat Feb 22 00:52:49 CET 2014 by Michael Peter Christen | removed warning "sun.misc.BASE64Encoder is internal proprietary API and may be removed in a future release" Changed Files: source/net/yacy/cora/order/Base64Order.java |
Thu Feb 20 08:35:51 CET 2014 by Michael Benz | Improved German de.lng translation and fixed adresses -> addresses in \htroot\CrawlStartScanner_p.html Changed Files: htroot/CrawlStartScanner_p.html, locales/de.lng |
Thu Feb 20 00:39:19 CET 2014 by Michael Peter Christen | better control over close-state of remote solr connections Changed Files: source/net/yacy/cora/federate/solr/connector/CachedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/MirrorSolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java, source/net/yacy/peers/Protocol.java |
Thu Feb 20 00:32:20 CET 2014 by Michael Benz | Edit German de locale and improved translation Changed Files: htroot/IndexControlRWIs_p.html, htroot/IndexImportMediawiki_p.html, htroot/Status.java, htroot/Status_p.inc, locales/de.lng |
Thu Feb 20 00:03:35 CET 2014 by Michael Peter Christen | fix for "org.apache.solr.core.SolrCore Too many close [count:-1] on org.apache.solr.core.SolrCore@51af7c57" -error Changed Files: source/net/yacy/cora/federate/solr/connector/CachedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/instance/EmbeddedInstance.java |
Tue Feb 18 01:38:56 CET 2014 by Michael Peter Christen | delete fail-docs during postprocessing to prevent that they will appear again and stay in postprocessing forever. Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java |
Mon Feb 17 23:01:34 CET 2014 by malykhin.dmitry | edit russian locale Changed Files: locales/ru.lng |
Sun Feb 16 02:26:11 CET 2014 by reger | add edit-link to PK field of table admin Changed Files: htroot/Tables_p.html, htroot/Tables_p.java |
Sat Feb 15 04:29:49 CET 2014 by r | edit russian locale edit russian locale Changed Files: locales/ru.lng |
Fri Feb 14 11:12:25 CET 2014 by Anatoliy Evladov | Fixed my error Changed Files: locales/ru.lng |
Fri Feb 14 11:09:25 CET 2014 by Anatoliy Evladov | Edit ru locales Changed Files: locales/ru.lng |
Wed Feb 12 01:01:23 CET 2014 by Michael Peter Christen | explicit call to optimize to add a expungeDeleted flag Changed Files: source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java |
Tue Feb 11 22:02:10 CET 2014 by Michael Peter Christen | in case that the crawler is running and the search user is the peer admin, we expect that the user wants to check recently crawled document to ensure that recent crawl results are inside the search results, we do a soft commit here. Changed Files: htroot/yacysearch.java |
Tue Feb 11 22:01:25 CET 2014 by Michael Peter Christen | removed clearURLIndex, which is a stub remaining from the old metadata database and not needed any more Changed Files: htroot/IndexControlURLs_p.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/index/Segment.java |
Tue Feb 11 03:20:03 CET 2014 by Michael Peter Christen | Added methods in resource observer to calculate the available and the occupied disc space. These values are also shown on the status page. The disc space calculation shall be used for a disk-limitation of the search index. Changed Files: htroot/Status.java, htroot/Status_p.inc, source/net/yacy/search/ResourceObserver.java |
Tue Feb 11 00:53:10 CET 2014 by Michael Peter Christen | adopted network image drawing to increased number of peers Changed Files: source/net/yacy/peers/graphics/NetworkGraph.java |
Mon Feb 10 21:25:01 CET 2014 by reger | upd: Maven pom to dev v1.69 Changed Files: pom.xml |
Mon Feb 10 14:48:07 CET 2014 by sixcooler | add forgotten solrj-4.6.1 Changed Files: lib/solr-solrj-4.6.1.jar |
Mon Feb 10 12:59:30 CET 2014 by Michael Peter Christen | run the cleanup process even when load is high, do postprocessing even if load > 1 (but < 2) but only if there is enough memory (now: 0.5 GB RAM available). The memory amount of the postprocessing is the cause that systems block because they run into a frequent-GC chain which almost locks the peer. If running with enough memory, the postprocessing is fast and not damaging to the system. Because the required RAM of 0.5 GB is never available in default setting, the postprocessing will not run if the peer is not reconfigured to use more memory. Changed Files: defaults/yacy.init, source/net/yacy/search/Switchboard.java |
Mon Feb 10 11:55:40 CET 2014 by Michael Peter Christen | replaced guava-15.0 with guava-16.0.1 Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/guava-16.0.1.jar |
Mon Feb 10 11:49:48 CET 2014 by Michael Peter Christen | replaced pdfbox 1.8.3 with 1.8.4 Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/fontbox-1.8.4.License, lib/fontbox-1.8.4.jar, lib/jempbox-1.8.4.License, lib/jempbox-1.8.4.jar, lib/pdfbox-1.8.4.License, lib/pdfbox-1.8.4.jar |
Mon Feb 10 11:29:22 CET 2014 by Michael Peter Christen | next dev release Changed Files: build.properties |
Sun Feb 09 20:42:11 CET 2014 by reger | upd: Maven pom Changed Files: pom.xml |