Release_1.04
Commit | Description |
---|---|
Thu Jul 05 12:38:41 CEST 2012 by Michael Peter Christen | made class methods static where possible Changed Files: .settings/org.eclipse.jdt.core.prefs, htroot/BlacklistCleaner_p.java, htroot/BlacklistImpExp_p.java, htroot/BlacklistTest_p.java, htroot/Blacklist_p.java, htroot/Connections_p.java, htroot/Network.java, htroot/api/status_p.java, htroot/yacysearch.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/data/wiki/WikiCode.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverCoreSocket.java, source/de/anomic/server/serverSwitch.java, source/net/yacy/YaCySearchClient.java, source/net/yacy/cora/document/JSONTokener.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/geolocation/GeoLocation.java, source/net/yacy/document/parser/augment/AugmentParser.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/images/bmpParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rdfa/impl/RDFaParser.java, source/net/yacy/kelondro/blob/MapColumnIndex.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/io/CachedRecords.java, source/net/yacy/kelondro/io/RandomAccessIO.java, source/net/yacy/kelondro/io/Records.java, source/net/yacy/kelondro/logging/GuiHandler.java, source/net/yacy/kelondro/logging/LogParser.java, source/net/yacy/kelondro/logging/LogalizerHandler.java, source/net/yacy/kelondro/util/MemoryControl.java, source/net/yacy/kelondro/util/MemoryStrategy.java, source/net/yacy/kelondro/util/StandardMemoryStrategy.java, source/net/yacy/peers/NewsPool.java, source/net/yacy/peers/SeedDB.java, source/net/yacy/peers/operation/yacySeedUploadScp.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java, source/net/yacy/upnp/impls/InternetGatewayDevice.java |
Thu Jul 05 11:18:31 CEST 2012 by Michael Peter Christen | - removed unnecessary semicolons - added default case for switch Changed Files: .settings/org.eclipse.jdt.core.prefs, htroot/Surftips.java, htroot/api/bookmarks/xbel/xbel.java, source/de/anomic/data/ymark/YMarkXBELImporter.java, source/de/anomic/http/server/ChunkedInputStream.java, source/de/anomic/tools/UPnP.java, source/net/yacy/cora/document/RSSReader.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/index/Column.java, source/net/yacy/kelondro/workflow/AbstractThread.java, source/net/yacy/repository/FilterEngine.java, source/net/yacy/yacy.java |
Thu Jul 05 10:44:30 CEST 2012 by Michael Peter Christen | removed more unused method parameters Changed Files: htroot/Bookmarks.java, htroot/Crawler_p.java, htroot/PerformanceConcurrency_p.java, htroot/PerformanceSearch_p.java, htroot/interaction/Table.java, htroot/yacysearch.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/http/server/HTTPDemon.java, source/net/yacy/document/parser/augment/AugmentParser.java, source/net/yacy/interaction/Interaction.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/snippet/MediaSnippet.java |
Thu Jul 05 10:23:07 CEST 2012 by Michael Peter Christen | removed unused method parameters Changed Files: defaults/yacy.init, htroot/ConfigBasic.java, htroot/CrawlMonitorRemoteStart.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlStartScanner_p.java, htroot/Crawler_p.java, htroot/IndexImportOAIPMH_p.java, htroot/Network.java, htroot/NetworkPicture.java, htroot/News.java, htroot/Supporter.java, htroot/Surftips.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/message.java, htroot/yacy/search.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/data/wiki/AbstractWikiParser.java, source/de/anomic/http/server/AugmentedHtmlStream.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/tools/crypt.java, source/de/anomic/tools/cryptbig.java, source/net/yacy/ai/example/ConnectFour.java, source/net/yacy/cora/document/RSSMessage.java, source/net/yacy/cora/protocol/ByteArrayBody.java, source/net/yacy/cora/protocol/Scanner.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/document/importer/OAIPMHLoader.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/gui/framework/Browser.java, source/net/yacy/interaction/AugmentHtmlStream.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/data/citation/CitationReference.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/index/RAMIndex.java, source/net/yacy/kelondro/index/RAMIndexCluster.java, source/net/yacy/kelondro/order/MergeIterator.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/kelondro/workflow/InstantBusyThread.java, source/net/yacy/peers/NewsPool.java, source/net/yacy/peers/NewsQueue.java, source/net/yacy/peers/PeerActions.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/Seed.java, source/net/yacy/peers/SeedDB.java, source/net/yacy/peers/graphics/NetworkGraph.java, source/net/yacy/peers/graphics/WebStructureGraph.java, source/net/yacy/repository/FilterEngine.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java, source/net/yacy/search/query/SnippetProcess.java, source/net/yacy/search/snippet/TextSnippet.java, source/net/yacy/yacy.java |
Thu Jul 05 09:14:04 CEST 2012 by Michael Peter Christen | - added @SuppressWarnings to unused servlet method parameters - removed unnecessary casts - removed unnecessary throw statements Changed Files: htroot/AccessGrid_p.java, htroot/AccessPicture_p.java, htroot/AugmentedBrowsingFilters_p.java, htroot/AugmentedBrowsing_p.java, htroot/AugmentedParsing_p.java, htroot/Banner.java, htroot/BlacklistCleaner_p.java, htroot/BlacklistImpExp_p.java, htroot/BlacklistTest_p.java, htroot/CacheResource_p.java, htroot/ConfigAccounts_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigHTCache_p.java, htroot/ConfigHeuristics_p.java, htroot/ConfigLanguage_p.java, htroot/ConfigLiveSearch.java, htroot/ConfigNetwork_p.java, htroot/ConfigProfile_p.java, htroot/ConfigProperties_p.java, htroot/ConfigRobotsTxt_p.java, htroot/ConfigSearchBox.java, htroot/ConfigUpdate_p.java, htroot/Connections_p.java, htroot/ContentIntegrationPHPBB3_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/CrawlMonitorRemoteStart.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlStartExpert_p.java, htroot/CrawlStartScanner_p.java, htroot/Crawler_p.java, htroot/DemoServlet.java, htroot/DemoServletInteraction.java, htroot/DemoServletRDF.java, htroot/DictionaryLoader_p.java, htroot/Help.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateDomainCrawl_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateParserErrors_p.java, htroot/IndexCreateQueues_p.java, htroot/IndexFederated_p.java, htroot/IndexImportMediawiki_p.java, htroot/IndexImportOAIPMHList_p.java, htroot/IndexImportOAIPMH_p.java, htroot/IndexShare_p.java, htroot/Load_MediawikiWiki.java, htroot/Load_PHPBB3.java, htroot/Load_RSS_p.java, htroot/MessageSend_p.java, htroot/PeerLoadPicture.java, htroot/PerformanceConcurrency_p.java, htroot/PerformanceGraph.java, htroot/PerformanceMemory_p.java, htroot/PerformanceSearch_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/Ranking_p.java, htroot/RemoteCrawl_p.java, htroot/SearchEventPicture.java, htroot/ServerScannerList.java, htroot/Table_API_p.java, htroot/Table_RobotsTxt_p.java, htroot/Table_YMark_p.java, htroot/Tables_p.java, htroot/Threaddump_p.java, htroot/Trails.java, htroot/Triple_p.java, htroot/Triplestore_p.java, htroot/User.java, htroot/ViewLog_p.java, htroot/Vocabulary_p.java, htroot/WatchWebStructure_p.java, htroot/WebStructurePicture_p.java, htroot/WikiHelp.java, htroot/YBRFetch_p.java, htroot/YMarks.java, htroot/api/blacklists.java, htroot/api/blacklists_p.java, htroot/api/config_p.java, htroot/api/getpageinfo.java, htroot/api/getpageinfo_p.java, htroot/api/latency_p.java, htroot/api/schema_p.java, htroot/api/status_p.java, htroot/api/termlist_p.java, htroot/api/trail_p.java, htroot/api/version.java, htroot/env/style.java, htroot/imagetest.java, htroot/interaction/GetRDF.java, htroot/interaction/PutRDF.java, htroot/interaction_elements/Document_part.java, htroot/interaction_elements/Footer.java, htroot/interaction_elements/Loginstatus_part.java, htroot/interaction_elements/OverlayInteraction.java, htroot/interaction_elements/Tag_part.java, htroot/mediawiki_p.java, htroot/osm.java, htroot/rct_p.java, htroot/robots.java, htroot/sharedBlacklist_p.java, htroot/ssitestservlet.java, htroot/test.java, htroot/www/welcome.java, htroot/yacy/crawlReceipt.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacyinteractive.java, htroot/yacysearchlatestinfo.java, source/de/anomic/data/Translator.java, source/de/anomic/data/ymark/YMarkJSONImporter.java, source/net/yacy/cora/sorting/OrderedScoreMap.java, source/net/yacy/document/importer/OAIListFriendsLoader.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/kelondro/io/CachedFileReader.java, source/net/yacy/kelondro/util/GenerationMemoryStrategy.java, source/net/yacy/peers/dht/FlatWordPartitionScheme.java, source/net/yacy/search/index/DocumentReference.java, source/net/yacy/yacy.java, source/org/apache/tools/tar/TarInputStream.java |
Thu Jul 05 08:44:39 CEST 2012 by Michael Peter Christen | cleaned unnecessary nested code Changed Files: .settings/org.eclipse.jdt.core.prefs, htroot/CacheResource_p.java, htroot/ViewFile.java, htroot/api/ynetSearch.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/data/ymark/TablesRowComparator.java, source/de/anomic/data/ymark/YMarkAutoTagger.java, source/de/anomic/data/ymark/YMarkCrawlStart.java, source/de/anomic/data/ymark/YMarkDate.java, source/de/anomic/http/server/AugmentedHtmlStream.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/server/serverSwitch.java, source/net/yacy/ai/example/SchwarzerPeter.java, source/net/yacy/ai/greedy/Battle.java, source/net/yacy/ai/greedy/Context.java, source/net/yacy/cora/document/JSONObject.java, source/net/yacy/cora/lod/vocabulary/Tagging.java, source/net/yacy/cora/protocol/ResponseHeader.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/cora/storage/ConfigurationSet.java, source/net/yacy/cora/storage/SimpleARC.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/WordCache.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/parser/rdfa/impl/RDFaParser.java, source/net/yacy/document/parser/sidAudioParser.java, source/net/yacy/interaction/AugmentHtmlStream.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/blob/ObjectBuffer.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/util/ByteArray.java, source/net/yacy/kelondro/util/ByteBuffer.java, source/net/yacy/kelondro/util/StandardMemoryStrategy.java, source/net/yacy/kelondro/workflow/InstantBusyThread.java, source/net/yacy/peers/Seed.java, source/net/yacy/peers/dht/PeerSelection.java, source/net/yacy/repository/FilterEngine.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/query/SnippetProcess.java, source/net/yacy/search/snippet/TextSnippet.java, source/net/yacy/upnp/services/ISO8601Date.java |
Wed Jul 04 21:15:10 CEST 2012 by orbiter | refactoring and new usage of SentenceReader: this class appeared as one of the major CPU users during snippet verification. The class was not efficient for two reasons: - it used a too complex input stream; generated from sources and UTF8 byte-conversions. The BufferedReader applied a strong overhead. - to feed data into the SentenceReader, multiple toString/getBytes had been applied until a buffered Reader from an input stream was possible. These superfluous conversions had been removed. - the best source for the Sentence Reader is a String. Therefore the production of Strings had been forced inside the Document class. Changed Files: htroot/ViewFile.java, source/de/anomic/data/ymark/YMarkAutoTagger.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Document.java, source/net/yacy/document/SentenceReader.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/WordTokenizer.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/parser/augment/AugmentParser.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/genericParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/rdfParser.java, source/net/yacy/document/parser/rdfa/impl/RDFaParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/search/snippet/TextSnippet.java |
Tue Jul 03 17:06:20 CEST 2012 by Michael Peter Christen | Adding a limit of 1000 links that a parser shall store during indexing. A limit was necessary because some web pages have such huge numbers of links that it can easily cause a OOM just by the number of links. The quesion if the number of 1000 links is sufficient or too weak must be answered with the result of testing this feature. Changed Files: htroot/Crawler_p.java, source/de/anomic/data/BookmarkHelper.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/ScraperInputStream.java, source/net/yacy/document/parser/html/TransformerWriter.java, source/net/yacy/document/parser/htmlParser.java |
Mon Jul 02 15:40:40 CEST 2012 by Michael Peter Christen | - smaller caches to save memory - close cloneable iterators to free memory Changed Files: source/net/yacy/cora/order/CloneableIterator.java, source/net/yacy/cora/order/CloneableMapIterator.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/order/Digest.java, source/net/yacy/kelondro/order/MergeIterator.java, source/net/yacy/kelondro/order/RotateIterator.java, source/net/yacy/kelondro/order/StackIterator.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/search/index/MetadataRepository.java, source/net/yacy/search/ranking/BlockRank.java |
Mon Jul 02 13:57:29 CEST 2012 by Michael Peter Christen | better integration of blacklist according to use case Changed Files: htroot/Bookmarks.java, htroot/Crawler_p.java, htroot/DictionaryLoader_p.java, htroot/Load_RSS_p.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/api/getpageinfo.java, htroot/api/getpageinfo_p.java, htroot/api/webstructure.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/RSSLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/data/ymark/YMarkAutoTagger.java, source/de/anomic/data/ymark/YMarkMetadata.java, source/net/yacy/document/importer/OAIListFriendsLoader.java, source/net/yacy/document/importer/OAIPMHLoader.java, source/net/yacy/peers/graphics/OSMTile.java, source/net/yacy/peers/operation/yacyRelease.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/snippet/MediaSnippet.java, source/net/yacy/search/snippet/TextSnippet.java |
Mon Jul 02 09:51:43 CEST 2012 by Michael Peter Christen | giving threads name so its easier to see whats happening during debugging and within a thread dump Changed Files: source/de/anomic/crawler/Cache.java, source/de/anomic/crawler/CrawlStacker.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/cora/services/federated/opensearch/SRURSSConnector.java, source/net/yacy/cora/storage/Files.java, source/net/yacy/document/content/dao/PhpBB3Dao.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/gui/YaCyApp.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/query/SearchEvent.java |
Thu Jun 28 14:27:29 CEST 2012 by Michael Peter Christen | removed segments-concept and the Segments class: the segments had been there to create a tenant-infrastructure but were never be used since that was all much too complex. There will be a replacement using a solr navigation using a segment field in the search index. Changed Files: htroot/Bookmarks.java, htroot/CrawlResults.java, htroot/Crawler_p.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.html, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.html, htroot/IndexControlURLs_p.java, htroot/IndexFederated_p.java, htroot/IndexShare_p.java, htroot/Load_RSS_p.java, htroot/PerformanceGraph.java, htroot/PerformanceQueues_p.java, htroot/QuickCrawlLink_p.java, htroot/ViewFile.java, htroot/Vocabulary_p.java, htroot/YBRFetch_p.java, htroot/api/status_p.java, htroot/api/termlist_p.java, htroot/api/timeline.java, htroot/api/webstructure.java, htroot/api/yacydoc.java, htroot/api/ymarks/add_ymark.java, htroot/api/ymarks/get_metadata.java, htroot/api/ymarks/get_treeview.java, htroot/suggest.java, htroot/yacy/crawlReceipt.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacyinteractive.java, htroot/yacysearch.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/RSSLoader.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/data/ymark/YMarkMetadata.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Segment.java |
Wed Jun 27 12:17:58 CEST 2012 by Michael Peter Christen | - allow lazy initialization of solr value (if using 'lazy', then no 0-values and no empty strings are written). This may save a lot of memory (in ram and on disc) if excessive 0-values or empty strings appear) - do not allow default boolean values for checkboxes because that does not make sense: browsers may omit the checkbox attribute name if the box is not checked. A default value 'true' would not comply with the semantic of the browsers response. - add a checkbox in IndexFederated_p for the lazy initialization of solr fields. Changed Files: defaults/yacy.init, htroot/AccessPicture_p.java, htroot/ConfigPortal.java, htroot/ConfigUpdate_p.java, htroot/Connections_p.java, htroot/Crawler_p.java, htroot/IndexFederated_p.html, htroot/IndexFederated_p.java, htroot/NetworkPicture.java, htroot/PeerLoadPicture.java, htroot/Status.java, htroot/Table_API_p.java, htroot/Threaddump_p.java, htroot/ViewFile.java, htroot/api/ymarks/import_ymark.java, htroot/opensearchdescription.java, source/de/anomic/server/serverObjects.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/SolrConfiguration.java |
Tue Jun 26 10:13:13 CEST 2012 by cominch | Merge remote-tracking branch 'original yacy/master' Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, bin/checkalive.sh, build.xml, defaults/RDFaParser.xsl, defaults/solr/currency.xml, defaults/solr/elevate.xml, defaults/solr/lang/contractions_ca.txt, defaults/solr/lang/contractions_fr.txt, defaults/solr/lang/contractions_ga.txt, defaults/solr/lang/contractions_it.txt, defaults/solr/lang/hyphenations_ga.txt, defaults/solr/lang/stemdict_nl.txt, defaults/solr/lang/stoptags_ja.txt, defaults/solr/lang/stopwords_ar.txt, defaults/solr/lang/stopwords_bg.txt, defaults/solr/lang/stopwords_ca.txt, defaults/solr/lang/stopwords_cz.txt, defaults/solr/lang/stopwords_da.txt, defaults/solr/lang/stopwords_de.txt, defaults/solr/lang/stopwords_el.txt, defaults/solr/lang/stopwords_en.txt, defaults/solr/lang/stopwords_es.txt, defaults/solr/lang/stopwords_eu.txt, defaults/solr/lang/stopwords_fa.txt, defaults/solr/lang/stopwords_fi.txt, defaults/solr/lang/stopwords_fr.txt, defaults/solr/lang/stopwords_ga.txt, defaults/solr/lang/stopwords_gl.txt, defaults/solr/lang/stopwords_hi.txt, defaults/solr/lang/stopwords_hu.txt, defaults/solr/lang/stopwords_hy.txt, defaults/solr/lang/stopwords_id.txt, defaults/solr/lang/stopwords_it.txt, defaults/solr/lang/stopwords_ja.txt, defaults/solr/lang/stopwords_lv.txt, defaults/solr/lang/stopwords_nl.txt, defaults/solr/lang/stopwords_no.txt, defaults/solr/lang/stopwords_pt.txt, defaults/solr/lang/stopwords_ro.txt, defaults/solr/lang/stopwords_ru.txt, defaults/solr/lang/stopwords_sv.txt, defaults/solr/lang/stopwords_th.txt, defaults/solr/lang/stopwords_tr.txt, defaults/solr/protwords.txt, defaults/solr/schema.xml, defaults/solr/solr.xml, defaults/solr/solrconfig.xml, defaults/solr/stopwords.txt, defaults/solr/synonyms.txt, defaults/yacy.init, htroot/Blog.java, htroot/BlogComments.java, htroot/CacheResource_p.java, htroot/ConfigAccounts_p.java, htroot/ConfigAppearance_p.java, htroot/CookieTest_p.java, htroot/CrawlStartScanner_p.java, htroot/Crawler_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexFederated_p.html, htroot/IndexFederated_p.java, htroot/Messages_p.java, htroot/QuickCrawlLink_p.java, htroot/SettingsAck_p.java, htroot/Steering.java, htroot/Table_API_p.java, htroot/User.java, htroot/ViewImage.java, htroot/Wiki.java, htroot/opensearchdescription.java, htroot/suggest.java, htroot/yacy/message.java, htroot/yacysearch.java, htroot/yacysearch_location.java, htroot/yacysearchitem.java, lib/apache-solr-core-3.6.0.jar, lib/commons-httpclient-3.1.jar, lib/commons-lang-2.6.jar, lib/dependencies.txt, lib/fontbox-1.7.0.License, lib/fontbox-1.7.0.jar, lib/guava-r05.jar, lib/jempbox-1.7.0.License, lib/jempbox-1.7.0.jar, lib/jetty-6.1.26-patched-JETTY-1340.jar, lib/jetty-LICENSE-ASL.txt, lib/jetty-util-6.1.26-patched-JETTY-1340.jar, lib/jetty-util-LICENSE-ASL.txt, lib/log4j-over-slf4j-1.6.1.jar, lib/lucene-analyzers-3.6.0.jar, lib/lucene-core-3.6.0.jar, lib/lucene-highlighter-3.6.0.jar, lib/lucene-phonetic-3.6.0.jar, lib/lucene-spatial-3.6.0.jar, lib/lucene-spellchecker-3.6.0.jar, lib/pdfbox-1.7.0.License, lib/pdfbox-1.7.0.jar, lib/servlet-api-2.5-20081211.jar, lib/servlet-api-LICENSE-ASL.txt, nbproject/project.xml, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/data/BlogBoard.java, source/de/anomic/data/BlogBoardComments.java, source/de/anomic/data/wiki/WikiBoard.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/servletProperties.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/protocol/ResponseHeader.java, source/net/yacy/cora/protocol/TimeoutRequest.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/services/federated/solr/AbstractSolrConnector.java, source/net/yacy/cora/services/federated/solr/MultipleSolrConnector.java, source/net/yacy/cora/services/federated/solr/RetrySolrConnector.java, source/net/yacy/cora/services/federated/solr/ShardSelection.java, source/net/yacy/cora/services/federated/solr/ShardSolrConnector.java, source/net/yacy/cora/services/federated/solr/SingleSolrConnector.java, source/net/yacy/cora/services/federated/solr/SolrConnector.java, source/net/yacy/cora/storage/ConfigurationSet.java, source/net/yacy/document/Document.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/rdfa/impl/RDFaTripleImpl.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/interaction/AugmentHtmlStream.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/migration.java, source/net/yacy/peers/Network.java, source/net/yacy/peers/Seed.java, source/net/yacy/peers/operation/yacyRelease.java, source/net/yacy/peers/operation/yacySeedUploadFile.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/DocumentIndex.java, source/net/yacy/search/index/MetadataRepository.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/index/SolrConfiguration.java, source/net/yacy/search/query/SnippetProcess.java, source/net/yacy/search/solr/EmbeddedSolrConnector.java, source/net/yacy/yacy.java |
Tue Jun 26 00:08:25 CEST 2012 by Michael Peter Christen | generalized localhost naming. this is also a preparation for a better IPv6 implementation. Changed Files: htroot/Blog.java, htroot/BlogComments.java, htroot/ConfigAccounts_p.java, htroot/CrawlStartScanner_p.java, htroot/QuickCrawlLink_p.java, htroot/SettingsAck_p.java, htroot/Steering.java, htroot/Table_API_p.java, htroot/ViewImage.java, htroot/Wiki.java, htroot/opensearchdescription.java, htroot/yacysearch.java, htroot/yacysearch_location.java, htroot/yacysearchitem.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/data/BlogBoard.java, source/de/anomic/data/BlogBoardComments.java, source/de/anomic/data/wiki/WikiBoard.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/server/serverCore.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/services/federated/solr/ShardSolrConnector.java, source/net/yacy/cora/services/federated/solr/SingleSolrConnector.java, source/net/yacy/interaction/AugmentHtmlStream.java, source/net/yacy/peers/Seed.java, source/net/yacy/search/Switchboard.java |
Mon Jun 25 18:17:31 CEST 2012 by Michael Peter Christen | fixing redirects and status codes: storing of status code in ResponseHeader to make it available for late evaluations, like storage in solr. Changed Files: htroot/CacheResource_p.java, htroot/CookieTest_p.java, htroot/Crawler_p.java, htroot/User.java, htroot/suggest.java, htroot/yacysearch.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/server/servletProperties.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/protocol/ResponseHeader.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/services/federated/solr/AbstractSolrConnector.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/peers/operation/yacyRelease.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/SolrConfiguration.java |
Mon Jun 25 11:34:38 CEST 2012 by Michael Peter Christen | - fixed IndexFederated Servlet / a embedded Solr can now be selected - added code stub for an embedded Solr but generation of Solr store is still commented out (it works but is not yet ready for usage) Changed Files: defaults/yacy.init, htroot/IndexControlRWIs_p.java, htroot/IndexFederated_p.html, htroot/IndexFederated_p.java, source/de/anomic/crawler/CrawlQueues.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/MetadataRepository.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/query/SnippetProcess.java, source/net/yacy/search/solr/EmbeddedSolrConnector.java |
Fri Jun 22 16:49:58 CEST 2012 by Michael Peter Christen | upgraded to pdfbox 1.7.0 changes in http://www.apache.org/dist/pdfbox/1.7.0/RELEASE-NOTES.txt with many bugfixes, including performance related Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/fontbox-1.7.0.License, lib/fontbox-1.7.0.jar, lib/jempbox-1.7.0.License, lib/jempbox-1.7.0.jar, lib/pdfbox-1.7.0.License, lib/pdfbox-1.7.0.jar |
Fri Jun 22 15:31:17 CEST 2012 by Michael Peter Christen | added jetty libraries, needed for future use as web server and as application server for the solr search interface Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, htroot/IndexFederated_p.html, lib/dependencies.txt, lib/jetty-6.1.26-patched-JETTY-1340.jar, lib/jetty-LICENSE-ASL.txt, lib/jetty-util-6.1.26-patched-JETTY-1340.jar, lib/jetty-util-LICENSE-ASL.txt, lib/servlet-api-2.5-20081211.jar, lib/servlet-api-LICENSE-ASL.txt, source/net/yacy/search/solr/EmbeddedSolrConnector.java |
Fri Jun 22 11:39:17 CEST 2012 by Michael Peter Christen | using com.google.common.io.Files instead of homebrew methods Changed Files: .classpath, build.xml, htroot/BlogComments.java, htroot/ConfigAppearance_p.java, htroot/Messages_p.java, htroot/yacy/message.java, source/net/yacy/cora/storage/ConfigurationSet.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/migration.java, source/net/yacy/peers/operation/yacyRelease.java, source/net/yacy/peers/operation/yacySeedUploadFile.java, source/net/yacy/search/Switchboard.java, source/net/yacy/yacy.java |
Fri Jun 22 00:36:49 CEST 2012 by Michael Peter Christen | - added test for EmbeddedSolrConnector - added needed libraries for this test this includes most (all) files needed for an embedded solr Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, defaults/solr/currency.xml, defaults/solr/elevate.xml, defaults/solr/lang/contractions_ca.txt, defaults/solr/lang/contractions_fr.txt, defaults/solr/lang/contractions_ga.txt, defaults/solr/lang/contractions_it.txt, defaults/solr/lang/hyphenations_ga.txt, defaults/solr/lang/stemdict_nl.txt, defaults/solr/lang/stoptags_ja.txt, defaults/solr/lang/stopwords_ar.txt, defaults/solr/lang/stopwords_bg.txt, defaults/solr/lang/stopwords_ca.txt, defaults/solr/lang/stopwords_cz.txt, defaults/solr/lang/stopwords_da.txt, defaults/solr/lang/stopwords_de.txt, defaults/solr/lang/stopwords_el.txt, defaults/solr/lang/stopwords_en.txt, defaults/solr/lang/stopwords_es.txt, defaults/solr/lang/stopwords_eu.txt, defaults/solr/lang/stopwords_fa.txt, defaults/solr/lang/stopwords_fi.txt, defaults/solr/lang/stopwords_fr.txt, defaults/solr/lang/stopwords_ga.txt, defaults/solr/lang/stopwords_gl.txt, defaults/solr/lang/stopwords_hi.txt, defaults/solr/lang/stopwords_hu.txt, defaults/solr/lang/stopwords_hy.txt, defaults/solr/lang/stopwords_id.txt, defaults/solr/lang/stopwords_it.txt, defaults/solr/lang/stopwords_ja.txt, defaults/solr/lang/stopwords_lv.txt, defaults/solr/lang/stopwords_nl.txt, defaults/solr/lang/stopwords_no.txt, defaults/solr/lang/stopwords_pt.txt, defaults/solr/lang/stopwords_ro.txt, defaults/solr/lang/stopwords_ru.txt, defaults/solr/lang/stopwords_sv.txt, defaults/solr/lang/stopwords_th.txt, defaults/solr/lang/stopwords_tr.txt, defaults/solr/protwords.txt, defaults/solr/schema.xml, defaults/solr/solr.xml, defaults/solr/solrconfig.xml, defaults/solr/stopwords.txt, defaults/solr/synonyms.txt, lib/commons-httpclient-3.1.jar, lib/dependencies.txt, lib/lucene-analyzers-3.6.0.jar, lib/lucene-core-3.6.0.jar, lib/lucene-highlighter-3.6.0.jar, lib/lucene-phonetic-3.6.0.jar, lib/lucene-spatial-3.6.0.jar, lib/lucene-spellchecker-3.6.0.jar, source/net/yacy/search/solr/EmbeddedSolrConnector.java |
Thu Jun 21 14:55:38 CEST 2012 by Michael Peter Christen | - added solr core and libraries that solr needs (lucene is missing, will follow later) - added embedded solr connector which can connect to solr programmatically (without using a server in between) Changed Files: .classpath, build.xml, lib/apache-solr-core-3.6.0.jar, lib/commons-lang-2.6.jar, lib/dependencies.txt, lib/guava-r05.jar, lib/log4j-over-slf4j-1.6.1.jar, source/net/yacy/cora/services/federated/solr/AbstractSolrConnector.java, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java, source/net/yacy/peers/Network.java, source/net/yacy/search/solr/EmbeddedSolrConnector.java |
Wed Jun 20 18:04:23 CEST 2012 by cominch | Show additional interaction elements in footer section on each page, if activated in ConfigPortal.html. This footer is also visible in augmented browsing proxy mode. Changed Files: htroot/ConfigPortal.html, htroot/ConfigPortal.java, htroot/env/templates/embeddedfooter.template, htroot/env/templates/footer.template, htroot/env/templates/simplefooter.template, htroot/interaction_elements/Footer.html, htroot/interaction_elements/Footer.java, htroot/interaction_elements/Loginstatus_part.html, htroot/interaction_elements/Loginstatus_part.java, htroot/interaction_elements/OverlayInteraction.html, htroot/interaction_elements/OverlayInteraction.java, htroot/interaction_elements/login_admin.png, htroot/interaction_elements/login_empty.png, htroot/interaction_elements/login_user.png, htroot/yacysearch.html, source/net/yacy/interaction/AugmentHtmlStream.java |
Wed Jun 20 07:58:27 CEST 2012 by cominch | Merge remote-tracking branch 'original yacy/master' Changed Files: build.properties, defaults/yacy.init, htroot/Crawler_p.html, htroot/PerformanceQueues_p.java, source/de/anomic/crawler/RobotsTxt.java, source/net/yacy/document/TextParser.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/peers/SeedDB.java, source/net/yacy/upnp/DiscoveryAdvertisement.java, startYACY.sh |
Commit | Description |
---|---|
Sun Jul 08 16:48:09 CEST 2012 by Michael Peter Christen | fix for url camel case parser and sentence reader Changed Files: source/net/yacy/document/Condenser.java, source/net/yacy/document/SentenceReader.java |
Sun Jul 08 16:11:19 CEST 2012 by Michael Peter Christen | fix for sevenzip parser Changed Files: source/net/yacy/document/parser/sevenzipParser.java |
Fri Jul 06 01:29:13 CEST 2012 by Michael Peter Christen | fix to solr configuration (case where the external solr was not online) Changed Files: htroot/IndexFederated_p.java |
Thu Jul 05 14:24:03 CEST 2012 by Michael Peter Christen | fix for pattern matcher in html parser Changed Files: source/net/yacy/document/parser/html/ContentScraper.java |
Thu Jul 05 14:23:43 CEST 2012 by Michael Peter Christen | fix for solr shutdown Changed Files: source/net/yacy/cora/services/federated/solr/MultipleSolrConnector.java |
Thu Jul 05 14:23:29 CEST 2012 by Michael Peter Christen | fix for urls beginning with "//" Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java |
Thu Jul 05 14:06:00 CEST 2012 by sixcooler | fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=4430 Changed Files: htroot/IndexControlRWIs_p.java |
Mon Jul 02 14:37:57 CEST 2012 by Michael Peter Christen | bugfix for concurrent seed loader Changed Files: source/net/yacy/search/Switchboard.java |
Mon Jul 02 10:27:46 CEST 2012 by Michael Peter Christen | fixes for new eclipse 'Juno' warning 'Resource leak'. Changed Files: htroot/interaction_elements/Document_part.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/TemplateEngine.java, source/de/anomic/tools/CryptoLib.java, source/net/yacy/cora/storage/ConfigurationSet.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/order/Digest.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/query/AccessTracker.java |
Tue Jun 26 16:11:39 CEST 2012 by sixcooler | fix crawl start from file Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java |
Thu Jun 21 14:22:32 CEST 2012 by cominch | Augmented browsing: Small CSS fix Changed Files: htroot/interaction_elements/OverlayInteraction.html |
Thu Jun 21 12:02:14 CEST 2012 by cominch | Augmented browsing: small js fix Changed Files: htroot/interaction_elements/Tag_part.html |
Thu Jun 21 11:19:55 CEST 2012 by cominch | Augmented browsing: CSS fix Changed Files: htroot/interaction_elements/OverlayInteraction.html |
Commit | Description |
---|---|
Mon Jul 09 00:13:59 CEST 2012 by Michael Peter Christen | Release 1.04 Changed Files: build.properties |
Sun Jul 08 22:05:04 CEST 2012 by Michael Peter Christen | use less memory for md5 cache Changed Files: source/net/yacy/kelondro/order/Digest.java |
Sun Jul 08 22:04:36 CEST 2012 by Michael Peter Christen | more logging Changed Files: source/net/yacy/kelondro/logging/Log.java |
Sun Jul 08 21:25:22 CEST 2012 by Michael Peter Christen | filter old peers from bootstrap (now stronger: 60 minutes instead of 240). Changed Files: source/net/yacy/search/Switchboard.java |
Sun Jul 08 21:17:33 CEST 2012 by Michael Peter Christen | added classification for control file types which shall not be loaded but placed onto the noload-queue Changed Files: source/de/anomic/crawler/CrawlStacker.java, source/net/yacy/cora/document/Classification.java |
Sun Jul 08 17:59:20 CEST 2012 by Michael Peter Christen | added webm mime-type Changed Files: defaults/httpd.mime |
Sun Jul 08 17:58:05 CEST 2012 by Michael Peter Christen | added webm Changed Files: source/net/yacy/cora/document/Classification.java |
Sun Jul 08 16:11:50 CEST 2012 by Michael Peter Christen | fix for sitemap importer: can now also import very large sitemaps within small memory configurations Changed Files: source/de/anomic/crawler/SitemapImporter.java, source/net/yacy/document/parser/sitemapParser.java |
Fri Jul 06 09:21:12 CEST 2012 by Michael Peter Christen | catch and log a warning in RasterPlotter Changed Files: source/net/yacy/visualization/RasterPlotter.java |
Fri Jul 06 09:05:41 CEST 2012 by Michael Peter Christen | - fixed a memory leak (or bad usage) during parsing/snippet fetch - more logging for errors Changed Files: source/de/anomic/http/server/HTTPDFileHandler.java, source/net/yacy/document/parser/html/TransformerWriter.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/kelondro/workflow/InstantBlockingThread.java |
Fri Jul 06 08:29:41 CEST 2012 by Michael Peter Christen | prevent loading of content from the cache when retrieval with IFFRESH is used and cache is stale. Should speed up snippet generation when cache strategy is IFFRESH. Changed Files: source/de/anomic/crawler/Cache.java, source/net/yacy/repository/LoaderDispatcher.java |
Thu Jul 05 14:50:37 CEST 2012 by sixcooler | more abstraction of error message Changed Files: htroot/IndexControlRWIs_p.java |
Thu Jul 05 14:27:28 CEST 2012 by Michael Peter Christen | abstraction of error message Changed Files: htroot/IndexControlRWIs_p.java |
Thu Jul 05 11:09:44 CEST 2012 by Michael Peter Christen | removed unaccessible code Changed Files: .settings/org.eclipse.jdt.core.prefs, source/net/yacy/kelondro/table/Table.java |
Thu Jul 05 10:24:52 CEST 2012 by Michael Peter Christen | removed unused ImageReference package Changed Files: |
Thu Jul 05 09:21:27 CEST 2012 by Michael Peter Christen | removed snippet pattern filter - it was not used Changed Files: htroot/api/ymarks/manage_tags.java, htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/data/ymark/YMarkTables.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/RemoteSearch.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/query/SnippetProcess.java |
Thu Jul 05 01:02:51 CEST 2012 by Michael Peter Christen | replaced non-generic array with collection Changed Files: source/net/yacy/kelondro/workflow/InstantBusyThread.java, source/net/yacy/peers/Protocol.java |
Thu Jul 05 00:43:41 CEST 2012 by Michael Peter Christen | adding more principal peers for bootstraping Changed Files: defaults/yacy.network.freeworld.unit |
Thu Jul 05 00:20:58 CEST 2012 by orbiter | More SentenceReader cleanup Changed Files: source/net/yacy/document/Document.java, source/net/yacy/document/SentenceReader.java, source/net/yacy/search/snippet/TextSnippet.java |
Wed Jul 04 22:06:20 CEST 2012 by orbiter | Simplified SentenceReader (no more Reader inside..) Changed Files: source/net/yacy/document/SentenceReader.java |
Wed Jul 04 21:56:25 CEST 2012 by orbiter | replaced HashARC with SizeLimited Objects which are less costly Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/storage/SizeLimitedMap.java, source/net/yacy/cora/storage/SizeLimitedSet.java, source/net/yacy/document/parser/html/ContentScraper.java |
Wed Jul 04 21:15:38 CEST 2012 by orbiter | more tolerance when creating solar document Changed Files: source/net/yacy/search/index/SolrConfiguration.java |
Tue Jul 03 18:22:25 CEST 2012 by orbiter | automatically adopt size of word cache to available memory Changed Files: source/net/yacy/cora/sorting/OrderedScoreMap.java, source/net/yacy/document/WordCache.java |
Tue Jul 03 17:20:41 CEST 2012 by Michael Peter Christen | clean up parser data Changed Files: source/net/yacy/document/Document.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/htmlParser.java |
Tue Jul 03 07:12:20 CEST 2012 by Michael Peter Christen | - better data structures in secondary search - fixed a big memory leak in secondary search Changed Files: source/net/yacy/kelondro/data/word/WordReferenceFactory.java, source/net/yacy/peers/RemoteSearch.java, source/net/yacy/search/query/SearchEvent.java |
Tue Jul 03 06:06:38 CEST 2012 by Michael Peter Christen | parser refactoring & hacks Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/html/AbstractScraper.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/search/snippet/TextSnippet.java |
Mon Jul 02 14:27:37 CEST 2012 by Michael Peter Christen | concurrently initialize the seed list during p2p network bootstrap Changed Files: source/net/yacy/search/Switchboard.java |
Sun Jul 01 00:12:20 CEST 2012 by reger | add search result heuristic. adding a crawl job with depth-1 for every displayed search result (crawling every external linked page of displayed search result pages) Changed Files: defaults/yacy.init, htroot/ConfigHeuristics_p.html, htroot/ConfigHeuristics_p.java, htroot/yacysearchitem.java, source/net/yacy/search/Switchboard.java |
Sat Jun 30 10:30:01 CEST 2012 by Michael Peter Christen | more logging Changed Files: source/de/anomic/http/server/HTTPDemon.java, source/net/yacy/http/SSIHandler.java, source/net/yacy/yacy.java |
Thu Jun 28 13:27:45 CEST 2012 by Michael Peter Christen | added solr field 'refresh_s' which stores the refresh url contained in the meta-refresh html header field. Changed Files: defaults/solr.keys.list, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/search/index/SolrConfiguration.java, source/net/yacy/search/index/SolrField.java |
Wed Jun 27 13:07:02 CEST 2012 by Michael Peter Christen | do not fill the keywords with title content if keywords do not exist. Changed Files: source/net/yacy/document/parser/html/ContentScraper.java |
Tue Jun 26 14:53:45 CEST 2012 by Michael Peter Christen | shorter autocommit time (now: 1 second) to prevent that user cannot see results in solr the first time they try it out. The value can now be easily set to a higher number using the IndexFederated_p interface. Changed Files: defaults/yacy.init |
Tue Jun 26 14:51:57 CEST 2012 by Michael Peter Christen | - add canonical field only if requested by solr schema - remove canonical url from in/outbound urls if present Changed Files: source/net/yacy/search/index/SolrConfiguration.java |
Tue Jun 26 13:54:48 CEST 2012 by Michael Peter Christen | added option to record urls that are forwarded to the solr index Changed Files: defaults/solr.keys.list, defaults/yacy.init, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/net/yacy/search/SwitchboardConstants.java, source/net/yacy/search/index/MetadataRepository.java |
Tue Jun 26 11:18:29 CEST 2012 by Michael Peter Christen | fixed bad referer computation in SSIs which causes a NPE during host computation. This error was there before the latest IPv6 hack but did not cause a NPE. The IPv6 hack was not the cause for this bug, but it discovered the misconfiguration of the 'referer' referrer. Changed Files: source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/ServerSideIncludes.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/RequestHeader.java |
Tue Jun 26 00:25:46 CEST 2012 by Michael Peter Christen | more IPv6 hacks Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/Domains.java |
Mon Jun 25 14:59:46 CEST 2012 by Michael Peter Christen | added option to configure the autocommit delay time of solr on-the-fly Changed Files: defaults/yacy.init, htroot/IndexFederated_p.html, htroot/IndexFederated_p.java, source/net/yacy/cora/services/federated/solr/AbstractSolrConnector.java, source/net/yacy/cora/services/federated/solr/MultipleSolrConnector.java, source/net/yacy/cora/services/federated/solr/RetrySolrConnector.java, source/net/yacy/cora/services/federated/solr/ShardSolrConnector.java, source/net/yacy/cora/services/federated/solr/SolrConnector.java, source/net/yacy/search/Switchboard.java |
Mon Jun 25 11:37:32 CEST 2012 by Michael Peter Christen | Merge remote-tracking branch 'origin/master' Changed Files: nbproject/project.xml |
Sun Jun 24 22:50:08 CEST 2012 by reger | adjusted NetBeans classpath for new and updated libraries in lib Changed Files: nbproject/project.xml |
Sun Jun 24 10:58:09 CEST 2012 by Michael Peter Christen | root, not yacy Changed Files: bin/checkalive.sh |
Sun Jun 24 10:57:18 CEST 2012 by Michael Peter Christen | changed recommended line in /etc/crontab for high-availability Changed Files: bin/checkalive.sh |
Fri Jun 22 11:40:02 CEST 2012 by Michael Peter Christen | extended embedded solr tests to ensure that it will be usable within a jetty instance Changed Files: source/net/yacy/cora/services/federated/solr/AbstractSolrConnector.java, source/net/yacy/search/solr/EmbeddedSolrConnector.java |
Fri Jun 22 00:49:32 CEST 2012 by Michael Peter Christen | refactoring Changed Files: htroot/IndexFederated_p.java, source/de/anomic/crawler/ZURL.java, source/net/yacy/cora/services/federated/solr/MultipleSolrConnector.java, source/net/yacy/cora/services/federated/solr/RetrySolrConnector.java, source/net/yacy/cora/services/federated/solr/ShardSelection.java, source/net/yacy/cora/services/federated/solr/ShardSolrConnector.java, source/net/yacy/cora/services/federated/solr/SingleSolrConnector.java, source/net/yacy/search/Switchboard.java |
Thu Jun 21 16:09:12 CEST 2012 by Michael Peter Christen | moved RDFaParser.xsl configuration file to defaults Changed Files: build.xml, defaults/RDFaParser.xsl, source/net/yacy/document/parser/rdfa/impl/RDFaTripleImpl.java |
Thu Jun 21 16:04:48 CEST 2012 by Michael Peter Christen | using guava for host resolution (non-blocking for ips) and time-out Changed Files: .classpath, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/TimeoutRequest.java |
Thu Jun 21 14:59:55 CEST 2012 by Michael Peter Christen | added new class libraries to mac app Changed Files: addon/YaCy.app/Contents/Info.plist |
Thu Jun 21 11:01:02 CEST 2012 by cominch | Augmented browsing: small UI modifications Changed Files: htroot/interaction_elements/Document_part.html, htroot/interaction_elements/OverlayInteraction.html, htroot/interaction_elements/Tag_part.html |
Wed Jun 20 16:39:04 CEST 2012 by Michael Peter Christen | better integration of RDFaParser Changed Files: build.xml, source/net/yacy/document/Document.java, source/net/yacy/document/TextParser.java, source/net/yacy/search/index/DocumentIndex.java |
Wed Jun 20 09:10:39 CEST 2012 by cominch | Augmented Browsing: changed the settings page Changed Files: htroot/AugmentedBrowsingFilters_p.html, htroot/AugmentedBrowsingFilters_p.java |
Wed Jun 20 07:55:28 CEST 2012 by cominch | Corrected loading of default page settings on ConfigPortal.html Changed Files: htroot/ConfigPortal.java |
Tue Jun 19 13:13:00 CEST 2012 by sixcooler | correct table in new look of Crawler_p Changed Files: htroot/Crawler_p.html |