Main Release 1.6
Commit | Description |
---|---|
Sat Jul 27 10:19:46 CEST 2013 by Roland Haeder | Fix for NPE: E 2013/07/26 20:29:29 BUSYTHREAD Runtime Error in serverInstantThread.job, thread 'net.yacy.search.Switchboard.cleanupJob': null; target exception: null java.lang.NullPointerException at net.yacy.search.schema.CollectionConfiguration.convergenceStep(CollectionConfiguration.java:1116) at net.yacy.search.schema.CollectionConfiguration.postprocessing(CollectionConfiguration.java:897) at net.yacy.search.Switchboard.cleanupJob(Switchboard.java:2296) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at net.yacy.kelondro.workflow.InstantBusyThread.job(InstantBusyThread.java:107) at net.yacy.kelondro.workflow.AbstractBusyThread.run(AbstractBusyThread.java:165) Conflicts: source/net/yacy/search/schema/CollectionConfiguration.java Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java |
Thu Jul 25 12:17:29 CEST 2013 by Michael Peter Christen | always use a default value for boolean options to have transparency for the outcome if the attribute is missing in servlets Changed Files: htroot/AccessPicture_p.java, htroot/ConfigPortal.java, htroot/ConfigSearchPage_p.java, htroot/ConfigUpdate_p.java, htroot/Connections_p.java, htroot/Crawler_p.java, htroot/HostBrowser.java, htroot/IndexFederated_p.java, htroot/NetworkPicture.java, htroot/PeerLoadPicture.java, htroot/Status.java, htroot/Table_API_p.java, htroot/Threaddump_p.java, htroot/ViewFile.java, htroot/api/ymarks/import_ymark.java, htroot/opensearchdescription.java, source/net/yacy/server/serverObjects.java |
Wed Jul 17 18:37:34 CEST 2013 by Roland Haeder | Fixes and avoid of catching bad exceptions (some): - Rewrote usage of HashMap/Map to concurrent versions (to avoid a CME=ConcurrentModificationException) - Rewrote ConnectionInfo (as an example) to use a synchronized iterator instead of synchronizing an already synced HashSet (see Collections call) - This avoids catching CMEs again - Commented out noisy ConcurrentLog.logException() call Conflicts: source/net/yacy/repository/LoaderDispatcher.java Changed Files: htroot/AccessPicture_p.java, htroot/AccessTracker_p.java, htroot/CrawlStartScanner_p.java, htroot/PerformanceMemory_p.java, htroot/ServerScannerList.java, htroot/suggest.java, htroot/yacy/hello.java, htroot/yacysearch.java, source/net/yacy/cora/document/RSSFeed.java, source/net/yacy/cora/document/WordCache.java, source/net/yacy/cora/protocol/ConnectionInfo.java, source/net/yacy/peers/DHTSelection.java, source/net/yacy/repository/LoaderDispatcher.java |
Wed Jul 17 18:31:30 CEST 2013 by Roland Haeder | Added 'final' for all exception blocks as this helps the Java compiler to optimize memory usage Conflicts: source/net/yacy/search/Switchboard.java Changed Files: htroot/AccessTracker_p.java, htroot/Bookmarks.java, htroot/CacheResource_p.java, htroot/ConfigHeuristics_p.java, htroot/ContentIntegrationPHPBB3_p.java, htroot/CrawlCheck_p.java, htroot/CrawlStartScanner_p.java, htroot/Crawler_p.java, htroot/HostBrowser.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexDeletion_p.java, htroot/IndexFederated_p.java, htroot/IndexSchema_p.java, htroot/RankingSolr_p.java, htroot/RegexTest.java, htroot/ServerScannerList.java, htroot/Table_API_p.java, htroot/Table_YMark_p.java, htroot/Threaddump_p.java, htroot/Triple_p.java, htroot/Vocabulary_p.java, htroot/YMarks.java, htroot/api/bookmarks/get_bookmarks.java, htroot/api/bookmarks/get_folders.java, htroot/api/bookmarks/posts/all.java, htroot/api/bookmarks/posts/get.java, htroot/api/bookmarks/tags/addTag_p.java, htroot/api/bookmarks/xbel/xbel.java, htroot/api/citation.java, htroot/api/timeline.java, htroot/api/webstructure.java, htroot/api/ymarks/import_ymark.java, htroot/api/ynetSearch.java, htroot/gsa/searchresult.java, htroot/interaction/GetRDF.java, htroot/processing/domaingraph/applet/domaingraph.java, htroot/solr/select.java, htroot/suggest.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacysearch.java, htroot/yacysearchitem.java, libbuild/GitRevTask/GitRevTask.java, libbuild/svnRevNr/svnRevNrParser.java, source/net/yacy/YaCySearchClient.java, source/net/yacy/contentcontrol/ContentControlFilterUpdateThread.java, source/net/yacy/contentcontrol/SMWListImporter.java, source/net/yacy/contentcontrol/SMWListImporterFormatObsolete.java, source/net/yacy/contentcontrol/SMWListSyncThread.java, source/net/yacy/cora/ai/example/testorder.java, source/net/yacy/cora/ai/greedy/Agent.java, source/net/yacy/cora/ai/greedy/Attempts.java, source/net/yacy/cora/ai/greedy/Context.java, source/net/yacy/cora/ai/greedy/Engine.java, source/net/yacy/cora/document/JSONArray.java, source/net/yacy/cora/document/JSONObject.java, source/net/yacy/cora/document/JSONTokener.java, source/net/yacy/cora/document/RSSFeed.java, source/net/yacy/cora/document/RSSReader.java, source/net/yacy/cora/federate/opensearch/OpenSearchConnector.java, source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/cora/federate/solr/SolrServlet.java, source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/MirrorSolrConnector.java, source/net/yacy/cora/federate/solr/connector/RemoteSolrConnector.java, source/net/yacy/cora/federate/solr/connector/ShardSelection.java, source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java, source/net/yacy/cora/federate/solr/instance/EmbeddedInstance.java, source/net/yacy/cora/federate/solr/instance/RemoteInstance.java, source/net/yacy/cora/federate/solr/instance/ServerShard.java, source/net/yacy/cora/federate/solr/responsewriter/OpensearchResponseWriter.java, source/net/yacy/cora/federate/solr/responsewriter/YJsonResponseWriter.java, source/net/yacy/cora/federate/yacy/Peer.java, source/net/yacy/cora/federate/yacy/Peers.java, source/net/yacy/cora/federate/yacy/api/Network.java, source/net/yacy/cora/geo/GeonamesLocation.java, source/net/yacy/cora/geo/OpenGeoDBLocation.java, source/net/yacy/cora/language/phonetic/Phonetic.java, source/net/yacy/cora/language/synonyms/AutotaggingLibrary.java, source/net/yacy/cora/language/synonyms/SynonymLibrary.java, source/net/yacy/cora/lod/JenaTripleStore.java, source/net/yacy/cora/lod/vocabulary/CreativeCommons.java, source/net/yacy/cora/lod/vocabulary/Tagging.java, source/net/yacy/cora/order/Digest.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/protocol/ResponseHeader.java, source/net/yacy/cora/protocol/Scanner.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/protocol/http/LinkExtractor.java, source/net/yacy/cora/sorting/WeakPriorityBlockingQueue.java, source/net/yacy/cora/storage/ConcurrentARC.java, source/net/yacy/cora/storage/Configuration.java, source/net/yacy/cora/storage/Files.java, source/net/yacy/cora/storage/SizeLimitedSet.java, source/net/yacy/cora/storage/ZIPWriter.java, source/net/yacy/crawler/Balancer.java, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/crawler/data/Cache.java, source/net/yacy/crawler/data/CrawlProfile.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/crawler/data/ZURL.java, source/net/yacy/crawler/retrieval/FileLoader.java, source/net/yacy/crawler/retrieval/Request.java, source/net/yacy/crawler/retrieval/SMBLoader.java, source/net/yacy/crawler/retrieval/URLRewriterLibrary.java, source/net/yacy/crawler/robots/RobotsTxt.java, source/net/yacy/data/BlogBoard.java, source/net/yacy/data/BookmarkDate.java, source/net/yacy/data/BookmarksDB.java, source/net/yacy/data/DidYouMean.java, source/net/yacy/data/MessageBoard.java, source/net/yacy/data/UserDB.java, source/net/yacy/data/WorkTables.java, source/net/yacy/data/wiki/WikiBoard.java, source/net/yacy/data/ymark/YMarkAutoTagger.java, source/net/yacy/data/ymark/YMarkDMOZImporter.java, source/net/yacy/data/ymark/YMarkEntry.java, source/net/yacy/data/ymark/YMarkHTMLImporter.java, source/net/yacy/data/ymark/YMarkImporter.java, source/net/yacy/data/ymark/YMarkJSONImporter.java, source/net/yacy/data/ymark/YMarkRDF.java, source/net/yacy/data/ymark/YMarkTables.java, source/net/yacy/data/ymark/YMarkXBELImporter.java, source/net/yacy/document/Document.java, source/net/yacy/document/ImageParser.java, source/net/yacy/document/LibraryProvider.java, source/net/yacy/document/SnippetExtractor.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/document/content/dao/DatabaseConnection.java, source/net/yacy/document/content/dao/ImportDump.java, source/net/yacy/document/content/dao/PhpBB3Dao.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/importer/OAIListFriendsLoader.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/document/importer/OAIPMHLoader.java, source/net/yacy/document/importer/ResumptionToken.java, source/net/yacy/document/parser/audioTagParser.java, source/net/yacy/document/parser/augment/AugmentParser.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/html/CharacterCoding.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/ScraperInputStream.java, source/net/yacy/document/parser/mmParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/rdfa/impl/RDFaParser.java, source/net/yacy/document/parser/rdfa/impl/RDFaTripleImpl.java, source/net/yacy/document/parser/rssParser.java, source/net/yacy/document/parser/sidAudioParser.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xml/ODContentHandler.java, source/net/yacy/document/parser/xml/opensearchdescriptionReader.java, source/net/yacy/gui/framework/Application.java, source/net/yacy/gui/framework/Browser.java, source/net/yacy/gui/framework/Switchboard.java, source/net/yacy/interaction/AugmentHtmlStream.java, source/net/yacy/interaction/Interaction.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/BEncodedHeapBag.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/Gap.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/blob/HeapModifier.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/HeapWriter.java, source/net/yacy/kelondro/blob/MapColumnIndex.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/blob/Stack.java, source/net/yacy/kelondro/blob/Stacks.java, source/net/yacy/kelondro/blob/Tables.java, source/net/yacy/kelondro/blob/TablesColumnBLOBIndex.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/index/IndexTest.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowHandleSet.java, source/net/yacy/kelondro/io/ByteCountInputStream.java, source/net/yacy/kelondro/io/CachedFileReader.java, source/net/yacy/kelondro/io/CachedFileWriter.java, source/net/yacy/kelondro/io/Records.java, source/net/yacy/kelondro/logging/LogalizerHandler.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/kelondro/util/BDecoder.java, source/net/yacy/kelondro/util/BEncoder.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/kelondro/util/ReverseMapIterator.java, source/net/yacy/kelondro/util/SetTools.java, source/net/yacy/kelondro/workflow/WorkflowProcessor.java, source/net/yacy/migration.java, source/net/yacy/peers/DHTSelection.java, source/net/yacy/peers/Network.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/Seed.java, source/net/yacy/peers/SeedDB.java, source/net/yacy/peers/graphics/EncodedImage.java, source/net/yacy/peers/graphics/WebStructureGraph.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/search/ResourceObserver.java, source/net/yacy/search/Shutdown.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/DocumentIndex.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/index/ReindexSolrBusyThread.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/query/AccessTracker.java, source/net/yacy/search/query/QueryGoal.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/query/SecondarySearchSuperviser.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/WebgraphConfiguration.java, source/net/yacy/search/snippet/ResultEntry.java, source/net/yacy/search/snippet/TextSnippet.java, source/net/yacy/server/http/ChunkedInputStream.java, source/net/yacy/server/http/HTTPDFileHandler.java, source/net/yacy/server/http/HTTPDemon.java, source/net/yacy/server/serverObjects.java, source/net/yacy/server/serverSwitch.java, source/net/yacy/upnp/Discovery.java, source/net/yacy/upnp/DiscoveryAdvertisement.java, source/net/yacy/upnp/DiscoveryListener.java, source/net/yacy/upnp/JXPathParser.java, source/net/yacy/upnp/ServicesEventing.java, source/net/yacy/upnp/devices/UPNPRootDevice.java, source/net/yacy/upnp/impls/InternetGatewayDevice.java, source/net/yacy/upnp/messages/ActionMessage.java, source/net/yacy/upnp/messages/ActionMessageResponseParser.java, source/net/yacy/upnp/messages/StateVariableMessage.java, source/net/yacy/upnp/messages/StateVariableResponseParser.java, source/net/yacy/upnp/services/ISO8601Date.java, source/net/yacy/upnp/services/UPNPService.java, source/net/yacy/utils/CryptoLib.java, source/net/yacy/utils/SignatureOutputStream.java, source/net/yacy/utils/UPnP.java, source/net/yacy/utils/cryptbig.java, source/net/yacy/utils/tarTools.java, source/net/yacy/visualization/AnimationGIF.java, source/net/yacy/visualization/RasterPlotter.java, source/net/yacy/yacy.java, test/de/anomic/document/ParserTest.java, test/de/anomic/yacy/yacyURLTest.java, test/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnectorTest.java |
Wed Jul 17 15:20:56 CEST 2013 by Michael Peter Christen | added collection_sxt to error documents Changed Files: htroot/Crawler_p.java, htroot/yacy/crawlReceipt.java, htroot/yacy/urls.java, source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/crawler/data/ZURL.java, source/net/yacy/crawler/retrieval/FTPLoader.java, source/net/yacy/crawler/retrieval/HTTPLoader.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/snippet/MediaSnippet.java |
Sat Jul 13 13:12:36 CEST 2013 by Roland Haeder | Fixed CHMOD on many files + added missing loggers (e.g. jena) and made some noisy loggers quiet Changed Files: addon/YaCy.app/Contents/MacOS/JavaApplicationStub, addon/testkeys, defaults/solr/schema.xml, defaults/solr/solrconfig.xml, defaults/yacy.logging, htroot/api/feed.rss, htroot/env/terminal.css, htroot/js/highslide/graphics/controlbar2.gif, htroot/js/highslide/graphics/controlbar3.gif, htroot/js/highslide/graphics/controlbar4-hover.gif, htroot/js/highslide/graphics/controlbar4.gif, htroot/js/highslide/graphics/fullexpand.gif, htroot/js/highslide/graphics/geckodimmer.png, htroot/js/highslide/graphics/loader.gif, htroot/js/highslide/graphics/loader.white.gif, htroot/js/highslide/graphics/outlines/Outlines.psd, htroot/js/highslide/graphics/outlines/beveled.png, htroot/js/highslide/graphics/outlines/drop-shadow.png, htroot/js/highslide/graphics/outlines/glossy-dark.png, htroot/js/highslide/graphics/outlines/outer-glow.png, htroot/js/highslide/graphics/outlines/rounded-black.png, htroot/js/highslide/graphics/outlines/rounded-white.png, htroot/js/highslide/graphics/resize.gif, htroot/js/highslide/graphics/zoomin.cur, htroot/js/highslide/graphics/zoomout.cur, htroot/js/highslide/highslide.js, htroot/js/query.js, htroot/js/rss2.js, htroot/processing/domaingraph/applet/domaingraph.jar, htroot/processing/domaingraph/applet/domaingraph.pde, htroot/processing/domaingraph/applet/loading.gif, htroot/processing/domaingraph/domaingraph.pde, htroot/processing/put_in_libraries/animation.zip, htroot/processing/put_in_libraries/physics.zip, htroot/yacy/ui/colorpicker_images/Thumbs.db, htroot/yacy/ui/colorpicker_images/blank.gif, htroot/yacy/ui/colorpicker_images/colorpicker_background.png, htroot/yacy/ui/colorpicker_images/colorpicker_hex.png, htroot/yacy/ui/colorpicker_images/colorpicker_hsb_b.png, htroot/yacy/ui/colorpicker_images/colorpicker_hsb_h.png, htroot/yacy/ui/colorpicker_images/colorpicker_hsb_s.png, htroot/yacy/ui/colorpicker_images/colorpicker_indic.gif, htroot/yacy/ui/colorpicker_images/colorpicker_overlay.png, htroot/yacy/ui/colorpicker_images/colorpicker_rgb_b.png, htroot/yacy/ui/colorpicker_images/colorpicker_rgb_g.png, htroot/yacy/ui/colorpicker_images/colorpicker_rgb_r.png, htroot/yacy/ui/colorpicker_images/colorpicker_select.gif, htroot/yacy/ui/colorpicker_images/colorpicker_submit.png, htroot/yacy/ui/colorpicker_images/custom_background.png, htroot/yacy/ui/colorpicker_images/custom_hex.png, htroot/yacy/ui/colorpicker_images/custom_hsb_b.png, htroot/yacy/ui/colorpicker_images/custom_hsb_h.png, htroot/yacy/ui/colorpicker_images/custom_hsb_s.png, htroot/yacy/ui/colorpicker_images/custom_indic.gif, htroot/yacy/ui/colorpicker_images/custom_rgb_b.png, htroot/yacy/ui/colorpicker_images/custom_rgb_g.png, htroot/yacy/ui/colorpicker_images/custom_rgb_r.png, htroot/yacy/ui/colorpicker_images/custom_submit.png, htroot/yacy/ui/colorpicker_images/select.png, htroot/yacy/ui/colorpicker_images/select2.png, htroot/yacy/ui/colorpicker_images/slider.png, htroot/yacy/ui/css/colorpicker.css, htroot/yacy/ui/js/jquery-1.3.2.min.js, htroot/yacy/ui/js/jquery-ui-1.7.2.min.js, htroot/yacy/ui/js/jquery.tagcloud.min.js, htroot/yacy/ui/js/jquery.tinysort.min.js, lib/jcifs-1.3.15-LICENSE.txt |
Fri Jul 12 16:24:56 CEST 2013 by Michael Peter Christen | refactoring of load_delay: this is a matter of client identification Changed Files: htroot/CrawlCheck_p.java, htroot/Crawler_p.java, htroot/DictionaryLoader_p.java, htroot/Load_RSS_p.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/api/getpageinfo.java, htroot/api/getpageinfo_p.java, htroot/api/webstructure.java, source/net/yacy/cora/protocol/ClientIdentification.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/crawler/retrieval/RSSLoader.java, source/net/yacy/peers/operation/yacyRelease.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Segment.java |
Tue Jul 09 14:42:39 CEST 2013 by Michael Peter Christen | replaced usage of log4j by ConcurrentLog where possible Changed Files: source/net/yacy/cora/document/WordCache.java, source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java, source/net/yacy/cora/federate/yacy/Peers.java, source/net/yacy/cora/geo/GeonamesLocation.java, source/net/yacy/cora/language/synonyms/AutotaggingLibrary.java, source/net/yacy/cora/language/synonyms/SynonymLibrary.java, source/net/yacy/cora/lod/JenaTripleStore.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/ResponseHeader.java, source/net/yacy/cora/protocol/TimeoutRequest.java, source/net/yacy/crawler/retrieval/URLRewriterLibrary.java, source/net/yacy/crawler/robots/RobotsTxt.java, source/net/yacy/gui/framework/Switchboard.java, source/net/yacy/kelondro/logging/SimpleLogFormatter.java |
Tue Jul 09 14:28:25 CEST 2013 by Michael Peter Christen | - refactoring of log to ConcurrentLog: jdk-based logger tend to block at java.util.logging.Logger.log(Logger.java:476) in concurrent environments. This makes logging a main performance issue. To overcome this problem, this is a add-on to jdk logging to put log entries on a concurrent message queue and log the messages one by one using a separate process. - FTPClient uses the concurrent logging instead of the log4j logger Changed Files: htroot/BlacklistCleaner_p.java, htroot/Blacklist_p.java, htroot/BlogComments.java, htroot/Bookmarks.java, htroot/CacheResource_p.java, htroot/ConfigAccounts_p.java, htroot/ConfigHeuristics_p.java, htroot/ConfigPortal.java, htroot/ConfigSearchPage_p.java, htroot/ConfigUpdate_p.java, htroot/Connections_p.java, htroot/ContentIntegrationPHPBB3_p.java, htroot/CrawlCheck_p.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlResults.java, htroot/CrawlStartScanner_p.java, htroot/Crawler_p.java, htroot/DictionaryLoader_p.java, htroot/HostBrowser.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateQueues_p.java, htroot/IndexFederated_p.java, htroot/IndexImportOAIPMH_p.java, htroot/IndexSchema_p.java, htroot/Load_RSS_p.java, htroot/NetworkPicture.java, htroot/News.java, htroot/ProxyIndexingMonitor_p.java, htroot/ServerScannerList.java, htroot/Steering.java, htroot/Table_API_p.java, htroot/Table_YMark_p.java, htroot/Tables_p.java, htroot/Triple_p.java, htroot/User.java, htroot/ViewImage.java, htroot/ViewLog_p.java, htroot/ViewProfile.java, htroot/Vocabulary_p.java, htroot/WebStructurePicture_p.java, htroot/YMarks.java, htroot/api/getpageinfo.java, htroot/api/getpageinfo_p.java, htroot/api/table_p.java, htroot/api/termlist_p.java, htroot/api/timeline.java, htroot/api/webstructure.java, htroot/api/yacydoc.java, htroot/api/ymarks/add_ymark.java, htroot/api/ymarks/delete_ymark.java, htroot/api/ymarks/get_metadata.java, htroot/api/ymarks/get_treeview.java, htroot/api/ymarks/get_xbel.java, htroot/api/ymarks/get_ymark.java, htroot/api/ymarks/import_ymark.java, htroot/api/ymarks/manage_tags.java, htroot/gsa/searchresult.java, htroot/rct_p.java, htroot/solr/select.java, htroot/solr/update.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/message.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/net/yacy/contentcontrol/SMWListImporter.java, source/net/yacy/contentcontrol/SMWListImporterFormatObsolete.java, source/net/yacy/contentcontrol/SMWListSyncThread.java, source/net/yacy/cora/federate/opensearch/OpenSearchConnector.java, source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java, source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/ShardSelection.java, source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java, source/net/yacy/cora/federate/solr/instance/EmbeddedInstance.java, source/net/yacy/cora/federate/solr/instance/RemoteInstance.java, source/net/yacy/cora/geo/OpenGeoDBLocation.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/cora/util/ConcurrentLog.java, source/net/yacy/crawler/Balancer.java, source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/crawler/data/Cache.java, source/net/yacy/crawler/data/CrawlProfile.java, source/net/yacy/crawler/data/CrawlQueues.java, source/net/yacy/crawler/data/NoticedURL.java, source/net/yacy/crawler/data/ZURL.java, source/net/yacy/crawler/retrieval/FTPLoader.java, source/net/yacy/crawler/retrieval/FileLoader.java, source/net/yacy/crawler/retrieval/HTTPLoader.java, source/net/yacy/crawler/retrieval/RSSLoader.java, source/net/yacy/crawler/retrieval/SMBLoader.java, source/net/yacy/crawler/retrieval/SitemapImporter.java, source/net/yacy/data/BlogBoard.java, source/net/yacy/data/BlogBoardComments.java, source/net/yacy/data/BookmarkDate.java, source/net/yacy/data/BookmarkHelper.java, source/net/yacy/data/BookmarksDB.java, source/net/yacy/data/DidYouMean.java, source/net/yacy/data/MessageBoard.java, source/net/yacy/data/Translator.java, source/net/yacy/data/UserDB.java, source/net/yacy/data/WorkTables.java, source/net/yacy/data/wiki/WikiBoard.java, source/net/yacy/data/ymark/YMarkAutoTagger.java, source/net/yacy/data/ymark/YMarkEntry.java, source/net/yacy/data/ymark/YMarkHTMLImporter.java, source/net/yacy/data/ymark/YMarkImporter.java, source/net/yacy/data/ymark/YMarkJSONImporter.java, source/net/yacy/data/ymark/YMarkTables.java, source/net/yacy/data/ymark/YMarkXBELImporter.java, source/net/yacy/dbtest.java, source/net/yacy/document/AbstractParser.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Document.java, source/net/yacy/document/LibraryProvider.java, source/net/yacy/document/SnippetExtractor.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/content/SurrogateReader.java, source/net/yacy/document/content/dao/DatabaseConnection.java, source/net/yacy/document/content/dao/ImportDump.java, source/net/yacy/document/content/dao/PhpBB3Dao.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/importer/OAIListFriendsLoader.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/document/importer/OAIPMHLoader.java, source/net/yacy/document/importer/ResumptionToken.java, source/net/yacy/document/language/LanguageStatistics.java, source/net/yacy/document/language/LanguageStatisticsHolder.java, source/net/yacy/document/parser/audioTagParser.java, source/net/yacy/document/parser/augment/AugmentParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/Evaluation.java, source/net/yacy/document/parser/html/TransformerWriter.java, source/net/yacy/document/parser/images/bmpParser.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/images/icoParser.java, source/net/yacy/document/parser/mmParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rdfa/impl/RDFaParser.java, source/net/yacy/document/parser/rdfa/impl/RDFaTripleImpl.java, source/net/yacy/document/parser/sevenzipParser.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/document/parser/xml/ODContentHandler.java, source/net/yacy/document/parser/xml/opensearchdescriptionReader.java, source/net/yacy/document/parser/zipParser.java, source/net/yacy/gui/framework/Browser.java, source/net/yacy/interaction/AugmentHtmlStream.java, source/net/yacy/interaction/Interaction.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/BEncodedHeapBag.java, source/net/yacy/kelondro/blob/BEncodedHeapShard.java, source/net/yacy/kelondro/blob/Compressor.java, source/net/yacy/kelondro/blob/Heap.java, source/net/yacy/kelondro/blob/HeapModifier.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/kelondro/blob/HeapWriter.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/blob/Stack.java, source/net/yacy/kelondro/blob/Stacks.java, source/net/yacy/kelondro/blob/Tables.java, source/net/yacy/kelondro/blob/TablesColumnBLOBIndex.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/data/meta/URIMetadataNode.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/index/BufferedObjectIndex.java, source/net/yacy/kelondro/index/RAMIndexCluster.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/index/RowCollection.java, source/net/yacy/kelondro/index/RowHandleMap.java, source/net/yacy/kelondro/index/RowHandleSet.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/io/BufferedRecords.java, source/net/yacy/kelondro/io/ByteCountInputStream.java, source/net/yacy/kelondro/io/CachedFileReader.java, source/net/yacy/kelondro/io/CachedFileWriter.java, source/net/yacy/kelondro/io/CachedRecords.java, source/net/yacy/kelondro/io/Records.java, source/net/yacy/kelondro/logging/ConsoleOutErrHandler.java, source/net/yacy/kelondro/logging/SimpleLogFormatter.java, source/net/yacy/kelondro/rwi/AbstractIndex.java, source/net/yacy/kelondro/rwi/IODispatcher.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/kelondro/rwi/ReferenceIterator.java, source/net/yacy/kelondro/table/ChunkIterator.java, source/net/yacy/kelondro/table/Relations.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/kelondro/util/ConsoleInterface.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/kelondro/util/GenerationMemoryStrategy.java, source/net/yacy/kelondro/util/MemoryStrategy.java, source/net/yacy/kelondro/util/MergeIterator.java, source/net/yacy/kelondro/util/OS.java, source/net/yacy/kelondro/util/SetTools.java, source/net/yacy/kelondro/util/StandardMemoryStrategy.java, source/net/yacy/kelondro/workflow/AbstractBlockingThread.java, source/net/yacy/kelondro/workflow/AbstractBusyThread.java, source/net/yacy/kelondro/workflow/AbstractThread.java, source/net/yacy/kelondro/workflow/InstantBlockingThread.java, source/net/yacy/kelondro/workflow/InstantBusyThread.java, source/net/yacy/kelondro/workflow/WorkflowProcessor.java, source/net/yacy/migration.java, source/net/yacy/peers/DHTSelection.java, source/net/yacy/peers/Dispatcher.java, source/net/yacy/peers/Network.java, source/net/yacy/peers/NewsDB.java, source/net/yacy/peers/NewsPool.java, source/net/yacy/peers/NewsQueue.java, source/net/yacy/peers/PeerActions.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/RemoteSearch.java, source/net/yacy/peers/Seed.java, source/net/yacy/peers/SeedDB.java, source/net/yacy/peers/Transmission.java, source/net/yacy/peers/graphics/NetworkGraph.java, source/net/yacy/peers/graphics/OSMTile.java, source/net/yacy/peers/graphics/WebStructureGraph.java, source/net/yacy/peers/operation/yacyRelease.java, source/net/yacy/peers/operation/yacyVersion.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/repository/FilterEngine.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/MemoryTracker.java, source/net/yacy/search/ResourceObserver.java, source/net/yacy/search/Shutdown.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/DocumentIndex.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/index/ReindexSolrBusyThread.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/query/AccessTracker.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/query/SearchEventCache.java, source/net/yacy/search/query/SecondarySearchSuperviser.java, source/net/yacy/search/ranking/BlockRank.java, source/net/yacy/search/ranking/RankingProfile.java, source/net/yacy/search/ranking/ReferenceOrder.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/WebgraphConfiguration.java, source/net/yacy/search/snippet/MediaSnippet.java, source/net/yacy/search/snippet/ResultEntry.java, source/net/yacy/server/http/HTTPDFileHandler.java, source/net/yacy/server/http/HTTPDProxyHandler.java, source/net/yacy/server/http/HTTPDemon.java, source/net/yacy/server/http/TemplateEngine.java, source/net/yacy/server/serverCore.java, source/net/yacy/server/serverSwitch.java, source/net/yacy/server/serverSwitchAbstractAction.java, source/net/yacy/utils/UPnP.java, source/net/yacy/utils/crypt.java, source/net/yacy/utils/cryptbig.java, source/net/yacy/utils/gzip.java, source/net/yacy/utils/tarTools.java, source/net/yacy/visualization/Captcha.java, source/net/yacy/visualization/ChartPlotter.java, source/net/yacy/visualization/RasterPlotter.java, source/net/yacy/yacy.java |
Wed Jul 03 14:50:06 CEST 2013 by Michael Peter Christen | added a crawl option to obey html-meta-robots-noindex. This is on by default. Changed Files: defaults/yacy.init, htroot/CrawlStartExpert_p.html, htroot/CrawlStartExpert_p.java, htroot/CrawlStartSite_p.html, htroot/Crawler_p.java, htroot/QuickCrawlLink_p.html, htroot/QuickCrawlLink_p.java, source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/crawler/data/CrawlProfile.java, source/net/yacy/data/ymark/YMarkCrawlStart.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/schema/CollectionConfiguration.java |
Tue Jun 25 16:27:20 CEST 2013 by Michael Peter Christen | added url_file_name_s in default collection schema for the file name without the file extension. This part of the file path is removed from the multi-field url_paths_sxt, which has now not the file name as last part of the path list. The same applies to the new fields source_file_name_s and target_file_name_s in the webgraph schema. Changed Files: defaults/solr.collection.schema, defaults/solr.webgraph.schema, htroot/ViewFile.java, htroot/yacysearchitem.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/document/analysis/Classification.java, source/net/yacy/cora/federate/solr/SchemaConfiguration.java, source/net/yacy/cora/federate/solr/responsewriter/JsonResponseWriter.java, source/net/yacy/crawler/data/Latency.java, source/net/yacy/crawler/data/ResultImages.java, source/net/yacy/crawler/retrieval/FileLoader.java, source/net/yacy/crawler/retrieval/Response.java, source/net/yacy/crawler/retrieval/SMBLoader.java, source/net/yacy/data/ymark/YMarkAutoTagger.java, source/net/yacy/document/Document.java, source/net/yacy/document/LibraryProvider.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/audioTagParser.java, source/net/yacy/document/parser/genericParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/CollectionSchema.java, source/net/yacy/search/schema/WebgraphConfiguration.java, source/net/yacy/search/schema/WebgraphSchema.java |
Commit | Description |
---|---|
Thu Jul 25 09:53:19 CEST 2013 by Michael Peter Christen | fix in forced deletion: forced commit needed Changed Files: source/net/yacy/search/Switchboard.java |
Thu Jul 25 09:31:48 CEST 2013 by Michael Peter Christen | new test case for http://bugs.yacy.net/view.php?id=141 Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java |
Thu Jul 25 09:26:24 CEST 2013 by Michael Peter Christen | fix for http://bugs.yacy.net/view.php?id=267 Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java |
Thu Jul 25 09:21:37 CEST 2013 by Michael Peter Christen | fix for http://bugs.yacy.net/view.php?id=268 Changed Files: source/net/yacy/search/index/Segment.java |
Tue Jul 23 16:46:44 CEST 2013 by orbiter | fixes for problems with remote solr and non-activated webgraph index Changed Files: htroot/IndexDeletion_p.java, htroot/IndexFederated_p.java, source/net/yacy/cora/federate/opensearch/OpenSearchConnector.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/DocumentIndex.java, source/net/yacy/search/index/Fulltext.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/schema/WebgraphConfiguration.java |
Mon Jul 22 15:55:20 CEST 2013 by Michael Peter Christen | fixed problem with remote luke requests Changed Files: source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java, source/net/yacy/cora/federate/solr/instance/ResponseAccumulator.java |
Thu Jul 18 12:37:32 CEST 2013 by Michael Peter Christen | fix for NPE & bug #264 Changed Files: source/net/yacy/cora/federate/solr/instance/InstanceMirror.java |
Wed Jul 17 12:41:54 CEST 2013 by Michael Peter Christen | fixes using the embedded connector Changed Files: htroot/IndexReIndexMonitor_p.java, source/net/yacy/search/index/ReindexSolrBusyThread.java |
Tue Jul 16 18:18:55 CEST 2013 by Michael Peter Christen | fix for crawl stack domain counter Changed Files: source/net/yacy/crawler/Balancer.java, source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/crawler/data/NoticedURL.java, source/net/yacy/search/Switchboard.java |
Sun Jul 14 17:40:28 CEST 2013 by orbiter | fix for bug 260 Changed Files: source/net/yacy/cora/federate/solr/instance/InstanceMirror.java, source/net/yacy/search/index/ReindexSolrBusyThread.java |
Sat Jul 13 11:10:05 CEST 2013 by orbiter | npe fix Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java |
Fri Jul 12 14:38:30 CEST 2013 by orbiter | fix for http://bugs.yacy.net/view.php?id=255 Changed Files: source/net/yacy/crawler/data/CrawlQueues.java |
Thu Jul 11 12:53:39 CEST 2013 by Michael Peter Christen | another patch to prevent CLOSE_WAIT status on solr connections Changed Files: source/net/yacy/cora/federate/solr/instance/RemoteInstance.java |
Thu Jul 11 12:53:16 CEST 2013 by Michael Peter Christen | fixed parsing of peer flags Changed Files: source/net/yacy/peers/Seed.java |
Thu Jul 04 21:46:46 CEST 2013 by orbiter | fix for rssTerminal coloring Changed Files: htroot/rssTerminal.html |
Sun Jun 30 03:50:07 CEST 2013 by reger | bugfix: location (lat/lon) meta data retrival (Double.NaN check) Changed Files: source/net/yacy/kelondro/data/meta/URIMetadataNode.java |
Fri Jun 28 13:16:25 CEST 2013 by Michael Peter Christen | fix for solr requests if no query part is given (prevent npe) Changed Files: htroot/solr/select.java, source/net/yacy/peers/Protocol.java, source/net/yacy/server/serverObjects.java |
Tue Jun 25 16:36:16 CEST 2013 by Michael Peter Christen | npe fix Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java |
Fri Jun 21 15:47:12 CEST 2013 by Michael Peter Christen | fix for daterange: the to-date is inclusive Changed Files: htroot/gsa/searchresult.java |
Commit | Description |
---|---|
Sat Jul 27 11:26:14 CEST 2013 by orbiter | Release 1.6 Changed Files: build.properties |
Sat Jul 27 11:22:40 CEST 2013 by orbiter | we should not do too much greedylearning at this time as we don't have enough experience with it. set greedylearning.limit.doccount to a much lower limit. Changed Files: defaults/yacy.network.freeworld.unit, defaults/yacy.network.intranet.unit, defaults/yacy.network.metager.unit, defaults/yacy.network.webportal.unit |
Sat Jul 27 11:00:35 CEST 2013 by orbiter | wait a bit longer before doing the first peer ping Changed Files: source/net/yacy/search/Switchboard.java |
Sat Jul 27 10:56:02 CEST 2013 by orbiter | re-try to fetch url after a soft commit Changed Files: htroot/CrawlResults.java |
Sat Jul 27 10:51:58 CEST 2013 by orbiter | don't be noisy when doing background tasks that may be allowed to fail Changed Files: source/net/yacy/search/Switchboard.java |
Sat Jul 27 10:42:38 CEST 2013 by orbiter | apply load limitation also to dht-in Changed Files: defaults/yacy.init, htroot/yacy/transferRWI.java |
Fri Jul 26 19:51:34 CEST 2013 by Roland Haeder | Some cleanups: - added SKINS_PATH_DEFAULT as same as LISTS_PATH_DEFAULT was added - Added 'final' keyword to a string Changed Files: htroot/ConfigAppearance_p.java, source/net/yacy/migration.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/search/SwitchboardConstants.java |
Sat Jul 27 10:12:58 CEST 2013 by Roland Haeder | Use SwitchboardConstants.LISTS_PATH_DEFAULT instead of 'DATA/LISTS' Conflicts: htroot/api/blacklists_p.java Changed Files: htroot/BlacklistCleaner_p.java, htroot/BlacklistImpExp_p.java, htroot/BlacklistTest_p.java, htroot/Blacklist_p.java, htroot/api/blacklists.java, htroot/api/blacklists_p.java, htroot/sharedBlacklist_p.java |
Fri Jul 26 19:17:31 CEST 2013 by Roland Haeder | Removed another duplicate occurance of Blacklist.BLACKLIST_FILENAME_FILTER Changed Files: htroot/sharedBlacklist_p.java |
Fri Jul 26 19:09:38 CEST 2013 by Roland Haeder | Fix for blacklist export, also applied the filename filter here Changed Files: htroot/api/blacklists_p.java |
Fri Jul 26 19:09:00 CEST 2013 by Roland Haeder | Removed superfluous check for files ending '.black' as the previous commit already excluded all other files (e.g. .ser dumps), added logging in catch-all block Changed Files: htroot/BlacklistImpExp_p.java |
Fri Jul 26 19:07:42 CEST 2013 by Roland Haeder | Reuse Blacklist.BLACKLIST_FILENAME_FILTER Changed Files: htroot/BlacklistCleaner_p.java |
Fri Jul 26 18:32:04 CEST 2013 by Roland Haeder | Possible fix for ticket http://bugs.yacy.net/view.php?id=270, the filter for only including *.black must be applied Changed Files: htroot/BlacklistImpExp_p.java |
Thu Jul 25 12:02:11 CEST 2013 by Roland Haeder | Fix for NPE on shutdown: java.lang.NullPointerException at net.yacy.search.Switchboard.storeDocumentIndex(Switchboard.java:2732) at net.yacy.search.Switchboard.access00(Switchboard.java:207) at net.yacy.search.Switchboard.run(Switchboard.java:3049) Changed Files: source/net/yacy/search/Switchboard.java |
Sat Jul 27 09:54:51 CEST 2013 by Roland Haeder | Log this exception better Conflicts: source/net/yacy/kelondro/blob/Tables.java Changed Files: source/net/yacy/kelondro/blob/Tables.java |
Fri Jul 26 17:53:28 CEST 2013 by orbiter | do not write an empty failreason in case that there is no fail. Because of the lazy instantiation rule this value was not actually written, but if lazy instantiation is switched on, then this causes that all crawl starts delete all crawl-start-hosts completely because this looks for filled error reasons. Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java |
Tue Jul 23 20:24:13 CEST 2013 by reger | - small adjustment to make sure genericParser is tried last -- for some documents genericParser grabs document instead of specific available parser due to unordered pick of 1st to try parser (like .ps .rdf files and other) - remove redundant file extension registration Changed Files: source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/augment/AugmentParser.java, source/net/yacy/document/parser/rdfa/impl/RDFaParser.java |
Tue Jul 23 19:32:32 CEST 2013 by orbiter | do not run dht ditribution if system load is over 2.5 Changed Files: defaults/yacy.init, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java |
Tue Jul 23 19:10:11 CEST 2013 by orbiter | added segment sizes, postprocessing status and cpu load to crawler monitor Changed Files: htroot/Crawler_p.html, htroot/Crawler_p.java, htroot/js/Crawler.js |
Tue Jul 23 18:20:07 CEST 2013 by orbiter | added load info to status_p.xml Changed Files: htroot/api/status_p.java, htroot/api/status_p.xml |
Tue Jul 23 18:03:33 CEST 2013 by orbiter | - added information about segment count to status_p.xml - also moved this information from the old index structure, which is still in use for the RWI/DHT index to that front-end Changed Files: htroot/api/status_p.java, htroot/api/status_p.xml, source/net/yacy/cora/federate/solr/connector/CachedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/ConcurrentUpdateSolrConnector.java, source/net/yacy/cora/federate/solr/connector/MirrorSolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrConnector.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/search/index/Segment.java |
Tue Jul 23 14:21:12 CEST 2013 by sixcooler | changed optimization to a segment-size of index-size/5.000.000 + one if not idle + one (and force) if postprocessing Changed Files: source/net/yacy/search/Switchboard.java |
Tue Jul 23 12:21:27 CEST 2013 by Michael Peter Christen | replaced StringBuffer with StringBuilder Changed Files: source/net/yacy/cora/document/UTF8.java, source/net/yacy/cora/language/phonetic/DoubleMetaphone.java, source/net/yacy/cora/language/phonetic/Metaphone.java, source/net/yacy/interaction/AugmentHtmlStream.java, source/net/yacy/kelondro/logging/SimpleLogFormatter.java, source/net/yacy/server/http/AugmentedHtmlStream.java, source/net/yacy/upnp/ServicesEventing.java, source/net/yacy/upnp/messages/ActionMessage.java, source/net/yacy/upnp/messages/StateVariableMessage.java |
Tue Jul 23 12:16:51 CEST 2013 by Michael Peter Christen | replaced more containsKey() -> get() usages by a simple get(), followed by a test for NULL. This should increase the application speed and reduces the lookup time for the affected methods by 50% Changed Files: source/net/yacy/kelondro/blob/Tables.java |
Mon Jul 22 18:21:37 CEST 2013 by orbiter | delayed first peer-ping to send the first ping out after the http got up; if the ping comes before the http is up, it cannot be recognized as senior peer (if at all). See also: http://bugs.yacy.net/view.php?id=266 Changed Files: source/net/yacy/search/Switchboard.java |
Mon Jul 22 17:45:12 CEST 2013 by orbiter | added the property federated.service.solr.indexing.timeout to yacy.init to provide a configurable time-out for solr; see also: http://bugs.yacy.net/view.php?id=254 Changed Files: defaults/yacy.init, htroot/IndexFederated_p.java, source/net/yacy/peers/Protocol.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java |
Mon Jul 22 17:28:20 CEST 2013 by orbiter | removed forced non-soft commits to prevent index fragmentation Changed Files: source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/schema/CollectionConfiguration.java |
Mon Jul 22 14:17:38 CEST 2013 by Michael Peter Christen | removed failreason_t removal patch because that causes too much confusion using an external solr. to clean up the index after a schema change, use the index cleaner function from the online servlet Changed Files: source/net/yacy/search/Switchboard.java |
Sun Jul 21 22:57:21 CEST 2013 by reger | extend content type detection on file extension for .tif .tiff .htm Changed Files: source/net/yacy/crawler/retrieval/Response.java |
Thu Jul 18 14:31:42 CEST 2013 by Michael Peter Christen | added a getSegmentCount method and use it to disable optimize if wanted current segment count is below optimization level Changed Files: source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java |
Thu Jul 18 13:26:07 CEST 2013 by Michael Peter Christen | - used reger's LukeRequest to generalize the index info in SolrServerConnector - used the LukeRequest in SolrServerConnector to replace the index size method by a getNumDocs request to a LukeRequest result Changed Files: source/net/yacy/cora/federate/solr/connector/EmbeddedSolrConnector.java, source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java, source/net/yacy/migration.java |
Thu Jul 18 12:29:04 CEST 2013 by Michael Peter Christen | enhancements to crawler Changed Files: source/net/yacy/crawler/Balancer.java, source/net/yacy/kelondro/index/BufferedObjectIndex.java |
Wed Jul 17 19:09:25 CEST 2013 by orbiter | removed double-ocurring value assignments Changed Files: htroot/api/status_p.java |
Mon Jul 15 18:22:35 CEST 2013 by Roland Haeder | Removed trailing spaces + some more final Changed Files: source/net/yacy/search/index/Fulltext.java |
Sun Jul 14 09:01:10 CEST 2013 by Roland Haeder | Added some cora package loggers Changed Files: defaults/yacy.logging |
Sun Jul 14 08:04:10 CEST 2013 by Roland Haeder | Recommended cleanup (please, one day, execute this cleanup) Changed Files: .settings/org.eclipse.jdt.ui.prefs |
Wed Jul 17 16:42:10 CEST 2013 by Felix Ableitner | Fixed (?i) appearing in entries, fixed multiple equal lines in file. Changed Files: source/net/yacy/repository/Blacklist.java |
Wed Jul 17 12:38:58 CEST 2013 by orbiter | logging of deadlocks (if any) during cleanup process Changed Files: source/net/yacy/search/Switchboard.java |
Wed Jul 17 12:38:12 CEST 2013 by orbiter | added cpu load information to status page Changed Files: htroot/Status.java, htroot/Status_p.inc |
Tue Jul 16 17:16:48 CEST 2013 by Michael Peter Christen | do a more frequent optimization, reduces IO after optimization Changed Files: build.properties, source/net/yacy/search/Switchboard.java |
Tue Jul 16 10:49:20 CEST 2013 by orbiter | added load and deadlock detector in Memory util Changed Files: source/net/yacy/cora/util/Memory.java |
Sun Jul 14 18:43:32 CEST 2013 by Lotus | partly revert latest windows changes: YaCy has to be installed to a directory with write access for the running user. DATA folder is now used in the YaCy folder again. For using another location, the start script has to be heavily modified for loading proper start parameters after YaCy has been started once. Changed Files: startYACY.bat, startYACY_debug.bat |
Sun Jul 14 10:15:47 CEST 2013 by Michael Peter Christen | grr Changed Files: source/net/yacy/search/Switchboard.java |
Sun Jul 14 10:02:08 CEST 2013 by Michael Peter Christen | automatically running optimize on solr if user/search is idle for some time Changed Files: source/net/yacy/search/Switchboard.java |
Sat Jul 13 19:09:46 CEST 2013 by orbiter | do a light optimization at the end of a crawl postprocessing Changed Files: source/net/yacy/search/Switchboard.java, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/WebgraphConfiguration.java |
Sat Jul 13 18:54:09 CEST 2013 by orbiter | replaced type of solr schema object sku of text_en_splitting_tight by string Changed Files: defaults/solr/schema.xml, source/net/yacy/search/schema/CollectionSchema.java |
Sat Jul 13 15:12:51 CEST 2013 by Roland Haeder | Merge remote-tracking branch 'upstream/master' Changed Files: htroot/yacysearch.java, source/net/yacy/search/schema/CollectionConfiguration.java |
Sat Jul 13 14:49:36 CEST 2013 by orbiter | stealth mode shall only be available for authorized users, because unauthorized users can otherwise be monitored by authorized users Changed Files: htroot/yacysearch.java |
Fri Jul 12 16:05:48 CEST 2013 by orbiter | activate api actions after postprocessing of crawls Changed Files: source/net/yacy/search/Switchboard.java |
Fri Jul 12 14:29:22 CEST 2013 by orbiter | added a postprocessing field into api/status_p.xml to show if the postprocessing task is running at that time (status: busy) or not (status:idle) Changed Files: htroot/api/status_p.java, htroot/api/status_p.xml, source/net/yacy/search/Switchboard.java |
Fri Jul 12 14:16:51 CEST 2013 by orbiter | added list of crawls to status_p.xml Changed Files: htroot/api/status_p.java, htroot/api/status_p.xml |
Thu Jul 11 22:04:01 CEST 2013 by Lotus | added files to uninstall Changed Files: build.nsi |
Thu Jul 11 21:50:06 CEST 2013 by Lotus | Do not allow automatic update in case YaCy is installed to the Program Files folder on Windows. There are no permissions to write that folder and update would fail. Changed Files: htroot/ConfigUpdate_p.html, htroot/ConfigUpdate_p.java |
Thu Jul 11 21:48:49 CEST 2013 by Lotus | Revert "Revert "Windows installer: update logo"" This reverts commit c66631d407758b2157001ec5abcf98a2fc36c9ad. Changed Files: addon/installer/logo.bmp |
Thu Jul 11 21:48:13 CEST 2013 by Lotus | Revert "Windows installer: update logo" This reverts commit 41cc9be62babcea4c4fecb7c0eac8e624f394b70. Changed Files: addon/installer/logo.bmp |
Thu Jul 11 21:46:46 CEST 2013 by Lotus | Windows installer: update logo Changed Files: addon/installer/logo.bmp |
Thu Jul 11 21:46:17 CEST 2013 by Lotus | adapt windows start script parameters to linux start script parameter Changed Files: startYACY.bat, startYACY_debug.bat |
Thu Jul 11 17:04:24 CEST 2013 by orbiter | minimum access time has a tight connection to ClientIdentification, therefore it is defined there. Changed Files: htroot/api/latency_p.java, source/net/yacy/cora/protocol/ClientIdentification.java, source/net/yacy/crawler/data/NoticedURL.java |
Thu Jul 11 13:10:23 CEST 2013 by Felix Ableitner | Fixed problems with blacklist entry insertion. Changed Files: htroot/Blacklist_p.java |
Thu Jul 11 11:54:11 CEST 2013 by Michael Peter Christen | added Connection:close to all http requests in our http client to prevent CLOSE_WAIT states (as seen in lsof) Changed Files: source/net/yacy/cora/protocol/http/HTTPClient.java |
Wed Jul 10 18:32:12 CEST 2013 by sixcooler | re-insert this file - was deleted by mistake + correct an other case-typo Changed Files: htroot/IndexReIndexMonitor_p.html |
Wed Jul 10 15:13:04 CEST 2013 by orbiter | metager whitelist update Changed Files: defaults/yacy.network.metager.unit |
Tue Jul 09 11:42:13 CEST 2013 by orbiter | enhanced the network scanner: find more hosts automatically by removal of common subdomains before application of protocol-specific prefix Changed Files: htroot/CrawlStartScanner_p.html, htroot/CrawlStartScanner_p.java |
Tue Jul 09 11:01:37 CEST 2013 by orbiter | better handling of time-out at solrj in case that a commit is done in a fail-over case during add Changed Files: source/net/yacy/cora/federate/solr/connector/SolrServerConnector.java |
Tue Jul 09 10:50:47 CEST 2013 by orbiter | fix for bug #252 the naming of the servlet was wrong, the bug may not be present on systems where upper/lowercase matching is lazy (windows) Changed Files: htroot/IndexReIndexMonitor_p.html, htroot/env/templates/submenuIndexControl.template |
Mon Jul 08 23:40:31 CEST 2013 by Michael Peter Christen | increase crawler speed by reduction if the busysleep time Changed Files: defaults/yacy.init |
Mon Jul 08 23:32:15 CEST 2013 by Michael Peter Christen | Merge remote-tracking branch 'nutomics/blacklist_structure' Changed Files: htroot/Blacklist_p.java, source/net/yacy/repository/Blacklist.java |
Mon Jul 08 16:29:30 CEST 2013 by Michael Peter Christen | - better concurrency for network scanner - network scanner can now start from the list of all hosts in the search index Changed Files: htroot/CrawlStartScanner_p.html, htroot/CrawlStartScanner_p.java, source/net/yacy/cora/protocol/Scanner.java |
Mon Jul 08 16:26:11 CEST 2013 by Michael Peter Christen | fix for citation index generation in case that entry.referrerhash() is null. This is especially the case if ftp sites are crawled Changed Files: source/net/yacy/crawler/CrawlStacker.java |
Mon Jul 08 16:20:40 CEST 2013 by Michael Peter Christen | accept (but ignore) a '+' sign in front of search words Changed Files: source/net/yacy/search/query/QueryGoal.java |
Mon Jul 08 13:17:09 CEST 2013 by orbiter | enhanced network scanner - textarea input field can now be used to paste in a large list of hosts - /31er subnet is possible (only one host) - auto-detect subdomains for ftp and www subdomains Changed Files: htroot/CrawlStartScanner_p.html, htroot/CrawlStartScanner_p.java, source/net/yacy/cora/protocol/Scanner.java |
Sun Jul 07 19:17:29 CEST 2013 by Lotus | do not cut text on submit buttons Changed Files: skins/pdblue.css |
Thu Jul 04 19:16:13 CEST 2013 by sixcooler | do not use remote proxy if not switched on - regardless of the proto Changed Files: source/net/yacy/cora/protocol/http/ProxySettings.java |
Thu Jul 04 19:13:51 CEST 2013 by sixcooler | Revert "do not use remote proxy if not switched on - regardless of the proto" This reverts commit 20f452d22809ccb55f177c9b1651f878922e495c. Changed Files: .gitignore, source/net/yacy/cora/protocol/http/ProxySettings.java |
Thu Jul 04 19:12:50 CEST 2013 by sixcooler | do not use remote proxy if not switched on - regardless of the proto Changed Files: .gitignore, source/net/yacy/cora/protocol/http/ProxySettings.java |
Thu Jul 04 19:10:57 CEST 2013 by sixcooler | re-enable saved setting for proxy-crawl-profile Changed Files: source/net/yacy/crawler/CrawlSwitchboard.java |
Thu Jul 04 19:08:53 CEST 2013 by sixcooler | For indexes that are changing rapidly in NRT situations, fcs (stands for Field Cache per Segment) may be a better choice than the default fc. (saves memory) see: http://wiki.apache.org/solr/SimpleFacetParameters#facet.method Changed Files: source/net/yacy/cora/federate/solr/connector/AbstractSolrConnector.java, source/net/yacy/search/query/QueryParams.java |
Thu Jul 04 18:37:57 CEST 2013 by Felix Ableitner | Changed class structure of Blacklist. Changed Files: htroot/Blacklist_p.java, source/net/yacy/repository/Blacklist.java |
Thu Jul 04 17:12:19 CEST 2013 by Michael Peter Christen | added a patch from Sebastian M.B., submitted by email for coloring of rss terminal Changed Files: htroot/rssTerminal.html |
Thu Jul 04 16:56:54 CEST 2013 by Michael Peter Christen | Merge commit 'fd90fcc4e08f80acbfd1c9a7ec62ce04cd309594' Changed Files: htroot/yacysearch.html |
Tue Jul 02 20:45:41 CEST 2013 by Felix Ableitner | Fixes #196. Changed Files: htroot/yacysearch.html |
Tue Jul 02 14:29:13 CEST 2013 by Michael Peter Christen | new robots_i attribute fields Changed Files: defaults/solr.collection.schema, source/net/yacy/search/schema/CollectionConfiguration.java |
Mon Jul 01 13:24:52 CEST 2013 by Michael Peter Christen | hack to prevent long waiting times in crawler Changed Files: source/net/yacy/crawler/Balancer.java |
Mon Jul 01 13:10:09 CEST 2013 by Michael Peter Christen | prepartion for greedy crawl profiles and refactoring Changed Files: htroot/CrawlProfileEditor_p.java, htroot/Crawler_p.java, htroot/IndexCreateQueues_p.java, htroot/WatchWebStructure_p.java, source/net/yacy/crawler/CrawlSwitchboard.java, source/net/yacy/crawler/data/CrawlProfile.java, source/net/yacy/search/Switchboard.java |
Mon Jul 01 13:09:41 CEST 2013 by Michael Peter Christen | adding the canonical tag to crawl queues Changed Files: htroot/solr/select.java, source/net/yacy/document/Document.java, source/net/yacy/search/schema/CollectionConfiguration.java |
Sun Jun 30 11:30:39 CEST 2013 by orbiter | updated copyright message; included LGPL for 'cora' and a warranty warning. Changed Files: COPYRIGHT |
Sun Jun 30 02:11:46 CEST 2013 by Michael Peter Christen | redesign of citation index storage Changed Files: htroot/HostBrowser.java, source/net/yacy/crawler/CrawlStacker.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/schema/CollectionConfiguration.java |
Sat Jun 29 20:28:14 CEST 2013 by orbiter | set crawlingQ to true by default because most webpages are dynamic and crawlingQ should only be switched off in case of crawler traps Changed Files: defaults/yacy.init |
Sat Jun 29 18:54:30 CEST 2013 by Lotus | windows installer: update to latest jre Changed Files: build.nsi |
Sat Jun 29 00:35:43 CEST 2013 by reger | jpeg parser: extract GPS location from meta data Changed Files: source/net/yacy/document/parser/images/genericImageParser.java |
Fri Jun 28 15:33:19 CEST 2013 by Michael Peter Christen | fixed greedy learning mode - global is not a search attribute in searchitems Changed Files: htroot/yacysearchitem.java |
Fri Jun 28 15:32:36 CEST 2013 by Michael Peter Christen | removed unused option for queries Changed Files: htroot/IndexControlRWIs_p.java, htroot/yacy/search.java, htroot/yacysearch.java, source/net/yacy/search/query/QueryParams.java |
Fri Jun 28 15:31:36 CEST 2013 by Michael Peter Christen | added configuration option for greedy learning function to ConfigPortal servlet Changed Files: htroot/ConfigPortal.html, htroot/ConfigPortal.java |
Fri Jun 28 15:19:50 CEST 2013 by sixcooler | possible helpful closing of solr-requests Changed Files: htroot/gsa/searchresult.java, htroot/solr/select.java |
Fri Jun 28 14:51:37 CEST 2013 by sixcooler | added optimize-option Changed Files: htroot/IndexControlURLs_p.html, htroot/IndexControlURLs_p.java, source/net/yacy/search/index/Fulltext.java |
Fri Jun 28 14:17:14 CEST 2013 by Michael Peter Christen | removed attributes from ConfigPortal.html which are redundant to ConfigSearchPage_p.html Changed Files: htroot/ConfigPortal.html, htroot/ConfigPortal.java |
Fri Jun 28 12:39:36 CEST 2013 by Michael Peter Christen | fixed false multiple-generation of remote facet search which caused high cpu usage on remote side. Changed Files: source/net/yacy/peers/Protocol.java, source/net/yacy/peers/RemoteSearch.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java |
Fri Jun 28 12:22:33 CEST 2013 by Michael Peter Christen | - fix to gsa searchresult answer in case that no query part is given - fix to gsa default number of results (is 'num') Changed Files: htroot/env/templates/header.template, htroot/gsa/searchresult.java |
Wed Jun 26 16:51:36 CEST 2013 by Michael Peter Christen | added url_protocol_s to error documents Changed Files: source/net/yacy/search/schema/CollectionConfiguration.java |
Wed Jun 26 09:27:22 CEST 2013 by Michael Peter Christen | refactoring Changed Files: htroot/solr/select.java, source/net/yacy/cora/federate/solr/responsewriter/YJsonResponseWriter.java |
Wed Jun 26 09:26:34 CEST 2013 by Michael Peter Christen | added metadata-extractor-2.6.2.jar to eclipse classpath, removed old lib Changed Files: .classpath, source/net/yacy/document/parser/images/genericImageParser.java |
Tue Jun 25 23:24:02 CEST 2013 by reger | bump up lib/metadata-extractor-2.6.2.jar (used for image parser) with needed code adjustments Changed Files: addon/YaCy.app/Contents/Info.plist, build.xml, lib/metadata-extractor-2.6.2.License, lib/metadata-extractor-2.6.2.jar, source/net/yacy/document/parser/images/genericImageParser.java |
Tue Jun 25 16:36:05 CEST 2013 by Michael Peter Christen | added url_file_name_s in qeury for live-search of urls Changed Files: htroot/HostBrowser.html, htroot/IndexControlURLs_p.html |
Sun Jun 23 00:39:15 CEST 2013 by reger | make imageparser fileextension detection case insensitive (extensions are often upper case) Changed Files: source/net/yacy/document/parser/images/genericImageParser.java |
Tue Jun 18 16:51:56 CEST 2013 by Michael Peter Christen | now writing image alt texts and (camelcase-)parsed urls into a text search field for a better image retrieval Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/schema/CollectionConfiguration.java |
Tue Jun 18 16:25:00 CEST 2013 by Michael Peter Christen | added daterange option to gsa api Changed Files: htroot/gsa/searchresult.java |
Tue Jun 18 16:22:06 CEST 2013 by Michael Peter Christen | to avoid confusion, the gsa api is available at /search? and /searchresult? Changed Files: source/net/yacy/server/http/HTTPDFileHandler.java |
Tue Jun 18 13:28:30 CEST 2013 by orbiter | stub for better image indexing Changed Files: defaults/solr.collection.schema, source/net/yacy/search/schema/CollectionConfiguration.java, source/net/yacy/search/schema/CollectionSchema.java |
Mon Jun 17 13:29:03 CEST 2013 by orbiter | added hint to gsa response writer that the result comes from a yacy peer Changed Files: source/net/yacy/cora/federate/solr/responsewriter/GSAResponseWriter.java |
Mon Jun 17 13:28:28 CEST 2013 by orbiter | start of next development cycle with small version number 0.01 (as in the past) Changed Files: build.properties |