YaCy Release 1.04

Release_1.04

Major Changes   
Jump to: Bugfixes / Other Changes

CommitDescription
Thu Jul 05 12:38:41 CEST 2012
by Michael Peter Christen
made class methods static where possible
Changed Files: .settings/org.eclipse.jdt.core.prefs, htroot/BlacklistCleaner_p.java, htroot/BlacklistImpExp_p.java, htroot/BlacklistTest_p.java, htroot/Blacklist_p.java, htroot/Connections_p.java, htroot/Network.java, htroot/api/status_p.java, htroot/yacysearch.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/CrawlSwitchboard.java, source/de/anomic/crawler/NoticedURL.java, source/de/anomic/data/wiki/WikiCode.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/serverCoreSocket.java, source/de/anomic/server/serverSwitch.java, source/net/yacy/YaCySearchClient.java, source/net/yacy/cora/document/JSONTokener.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/geolocation/GeoLocation.java, source/net/yacy/document/parser/augment/AugmentParser.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/images/bmpParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/rdfa/impl/RDFaParser.java, source/net/yacy/kelondro/blob/MapColumnIndex.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/index/HandleMap.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/io/CachedRecords.java, source/net/yacy/kelondro/io/RandomAccessIO.java, source/net/yacy/kelondro/io/Records.java, source/net/yacy/kelondro/logging/GuiHandler.java, source/net/yacy/kelondro/logging/LogParser.java, source/net/yacy/kelondro/logging/LogalizerHandler.java, source/net/yacy/kelondro/util/MemoryControl.java, source/net/yacy/kelondro/util/MemoryStrategy.java, source/net/yacy/kelondro/util/StandardMemoryStrategy.java, source/net/yacy/peers/NewsPool.java, source/net/yacy/peers/SeedDB.java, source/net/yacy/peers/operation/yacySeedUploadScp.java, source/net/yacy/repository/Blacklist.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java, source/net/yacy/upnp/impls/InternetGatewayDevice.java
Thu Jul 05 11:18:31 CEST 2012
by Michael Peter Christen
- removed unnecessary semicolons
- added default case for switch
Changed Files: .settings/org.eclipse.jdt.core.prefs, htroot/Surftips.java, htroot/api/bookmarks/xbel/xbel.java, source/de/anomic/data/ymark/YMarkXBELImporter.java, source/de/anomic/http/server/ChunkedInputStream.java, source/de/anomic/tools/UPnP.java, source/net/yacy/cora/document/RSSReader.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/index/Column.java, source/net/yacy/kelondro/workflow/AbstractThread.java, source/net/yacy/repository/FilterEngine.java, source/net/yacy/yacy.java
Thu Jul 05 10:44:30 CEST 2012
by Michael Peter Christen
removed more unused method parameters
Changed Files: htroot/Bookmarks.java, htroot/Crawler_p.java, htroot/PerformanceConcurrency_p.java, htroot/PerformanceSearch_p.java, htroot/interaction/Table.java, htroot/yacysearch.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/http/server/HTTPDemon.java, source/net/yacy/document/parser/augment/AugmentParser.java, source/net/yacy/interaction/Interaction.java, source/net/yacy/kelondro/data/word/WordReferenceVars.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/snippet/MediaSnippet.java
Thu Jul 05 10:23:07 CEST 2012
by Michael Peter Christen
removed unused method parameters
Changed Files: defaults/yacy.init, htroot/ConfigBasic.java, htroot/CrawlMonitorRemoteStart.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlStartScanner_p.java, htroot/Crawler_p.java, htroot/IndexImportOAIPMH_p.java, htroot/Network.java, htroot/NetworkPicture.java, htroot/News.java, htroot/Supporter.java, htroot/Surftips.java, htroot/yacy/crawlReceipt.java, htroot/yacy/hello.java, htroot/yacy/message.java, htroot/yacy/search.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/data/wiki/AbstractWikiParser.java, source/de/anomic/http/server/AugmentedHtmlStream.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/tools/crypt.java, source/de/anomic/tools/cryptbig.java, source/net/yacy/ai/example/ConnectFour.java, source/net/yacy/cora/document/RSSMessage.java, source/net/yacy/cora/protocol/ByteArrayBody.java, source/net/yacy/cora/protocol/Scanner.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/document/importer/OAIPMHLoader.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/odtParser.java, source/net/yacy/document/parser/ooxmlParser.java, source/net/yacy/document/parser/psParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/gui/framework/Browser.java, source/net/yacy/interaction/AugmentHtmlStream.java, source/net/yacy/kelondro/blob/MapDataMining.java, source/net/yacy/kelondro/data/citation/CitationReference.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/index/RAMIndex.java, source/net/yacy/kelondro/index/RAMIndexCluster.java, source/net/yacy/kelondro/order/MergeIterator.java, source/net/yacy/kelondro/table/SQLTable.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/kelondro/workflow/InstantBusyThread.java, source/net/yacy/peers/NewsPool.java, source/net/yacy/peers/NewsQueue.java, source/net/yacy/peers/PeerActions.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/Seed.java, source/net/yacy/peers/SeedDB.java, source/net/yacy/peers/graphics/NetworkGraph.java, source/net/yacy/peers/graphics/WebStructureGraph.java, source/net/yacy/repository/FilterEngine.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/SwitchboardConstants.java, source/net/yacy/search/query/SnippetProcess.java, source/net/yacy/search/snippet/TextSnippet.java, source/net/yacy/yacy.java
Thu Jul 05 09:14:04 CEST 2012
by Michael Peter Christen
- added @SuppressWarnings to unused servlet method parameters
- removed unnecessary casts
- removed unnecessary throw statements
Changed Files: htroot/AccessGrid_p.java, htroot/AccessPicture_p.java, htroot/AugmentedBrowsingFilters_p.java, htroot/AugmentedBrowsing_p.java, htroot/AugmentedParsing_p.java, htroot/Banner.java, htroot/BlacklistCleaner_p.java, htroot/BlacklistImpExp_p.java, htroot/BlacklistTest_p.java, htroot/CacheResource_p.java, htroot/ConfigAccounts_p.java, htroot/ConfigAppearance_p.java, htroot/ConfigHTCache_p.java, htroot/ConfigHeuristics_p.java, htroot/ConfigLanguage_p.java, htroot/ConfigLiveSearch.java, htroot/ConfigNetwork_p.java, htroot/ConfigProfile_p.java, htroot/ConfigProperties_p.java, htroot/ConfigRobotsTxt_p.java, htroot/ConfigSearchBox.java, htroot/ConfigUpdate_p.java, htroot/Connections_p.java, htroot/ContentIntegrationPHPBB3_p.java, htroot/CookieMonitorIncoming_p.java, htroot/CookieMonitorOutgoing_p.java, htroot/CrawlMonitorRemoteStart.java, htroot/CrawlProfileEditor_p.java, htroot/CrawlStartExpert_p.java, htroot/CrawlStartScanner_p.java, htroot/Crawler_p.java, htroot/DemoServlet.java, htroot/DemoServletInteraction.java, htroot/DemoServletRDF.java, htroot/DictionaryLoader_p.java, htroot/Help.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.java, htroot/IndexCreateDomainCrawl_p.java, htroot/IndexCreateLoaderQueue_p.java, htroot/IndexCreateParserErrors_p.java, htroot/IndexCreateQueues_p.java, htroot/IndexFederated_p.java, htroot/IndexImportMediawiki_p.java, htroot/IndexImportOAIPMHList_p.java, htroot/IndexImportOAIPMH_p.java, htroot/IndexShare_p.java, htroot/Load_MediawikiWiki.java, htroot/Load_PHPBB3.java, htroot/Load_RSS_p.java, htroot/MessageSend_p.java, htroot/PeerLoadPicture.java, htroot/PerformanceConcurrency_p.java, htroot/PerformanceGraph.java, htroot/PerformanceMemory_p.java, htroot/PerformanceSearch_p.java, htroot/ProxyIndexingMonitor_p.java, htroot/Ranking_p.java, htroot/RemoteCrawl_p.java, htroot/SearchEventPicture.java, htroot/ServerScannerList.java, htroot/Table_API_p.java, htroot/Table_RobotsTxt_p.java, htroot/Table_YMark_p.java, htroot/Tables_p.java, htroot/Threaddump_p.java, htroot/Trails.java, htroot/Triple_p.java, htroot/Triplestore_p.java, htroot/User.java, htroot/ViewLog_p.java, htroot/Vocabulary_p.java, htroot/WatchWebStructure_p.java, htroot/WebStructurePicture_p.java, htroot/WikiHelp.java, htroot/YBRFetch_p.java, htroot/YMarks.java, htroot/api/blacklists.java, htroot/api/blacklists_p.java, htroot/api/config_p.java, htroot/api/getpageinfo.java, htroot/api/getpageinfo_p.java, htroot/api/latency_p.java, htroot/api/schema_p.java, htroot/api/status_p.java, htroot/api/termlist_p.java, htroot/api/trail_p.java, htroot/api/version.java, htroot/env/style.java, htroot/imagetest.java, htroot/interaction/GetRDF.java, htroot/interaction/PutRDF.java, htroot/interaction_elements/Document_part.java, htroot/interaction_elements/Footer.java, htroot/interaction_elements/Loginstatus_part.java, htroot/interaction_elements/OverlayInteraction.java, htroot/interaction_elements/Tag_part.java, htroot/mediawiki_p.java, htroot/osm.java, htroot/rct_p.java, htroot/robots.java, htroot/sharedBlacklist_p.java, htroot/ssitestservlet.java, htroot/test.java, htroot/www/welcome.java, htroot/yacy/crawlReceipt.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacyinteractive.java, htroot/yacysearchlatestinfo.java, source/de/anomic/data/Translator.java, source/de/anomic/data/ymark/YMarkJSONImporter.java, source/net/yacy/cora/sorting/OrderedScoreMap.java, source/net/yacy/document/importer/OAIListFriendsLoader.java, source/net/yacy/document/importer/OAIPMHImporter.java, source/net/yacy/kelondro/io/CachedFileReader.java, source/net/yacy/kelondro/util/GenerationMemoryStrategy.java, source/net/yacy/peers/dht/FlatWordPartitionScheme.java, source/net/yacy/search/index/DocumentReference.java, source/net/yacy/yacy.java, source/org/apache/tools/tar/TarInputStream.java
Thu Jul 05 08:44:39 CEST 2012
by Michael Peter Christen
cleaned unnecessary nested code
Changed Files: .settings/org.eclipse.jdt.core.prefs, htroot/CacheResource_p.java, htroot/ViewFile.java, htroot/api/ynetSearch.java, source/de/anomic/crawler/CrawlProfile.java, source/de/anomic/crawler/CrawlStacker.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/data/ymark/TablesRowComparator.java, source/de/anomic/data/ymark/YMarkAutoTagger.java, source/de/anomic/data/ymark/YMarkCrawlStart.java, source/de/anomic/data/ymark/YMarkDate.java, source/de/anomic/http/server/AugmentedHtmlStream.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/server/serverSwitch.java, source/net/yacy/ai/example/SchwarzerPeter.java, source/net/yacy/ai/greedy/Battle.java, source/net/yacy/ai/greedy/Context.java, source/net/yacy/cora/document/JSONObject.java, source/net/yacy/cora/lod/vocabulary/Tagging.java, source/net/yacy/cora/protocol/ResponseHeader.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/cora/storage/ConfigurationSet.java, source/net/yacy/cora/storage/SimpleARC.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/WordCache.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/parser/rdfa/impl/RDFaParser.java, source/net/yacy/document/parser/sidAudioParser.java, source/net/yacy/interaction/AugmentHtmlStream.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/blob/ObjectBuffer.java, source/net/yacy/kelondro/data/meta/DigestURI.java, source/net/yacy/kelondro/index/Cache.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/rwi/IndexCell.java, source/net/yacy/kelondro/rwi/ReferenceContainer.java, source/net/yacy/kelondro/util/ByteArray.java, source/net/yacy/kelondro/util/ByteBuffer.java, source/net/yacy/kelondro/util/StandardMemoryStrategy.java, source/net/yacy/kelondro/workflow/InstantBusyThread.java, source/net/yacy/peers/Seed.java, source/net/yacy/peers/dht/PeerSelection.java, source/net/yacy/repository/FilterEngine.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/query/SnippetProcess.java, source/net/yacy/search/snippet/TextSnippet.java, source/net/yacy/upnp/services/ISO8601Date.java
Wed Jul 04 21:15:10 CEST 2012
by orbiter
refactoring and new usage of SentenceReader: this class appeared as one
of the major CPU users during snippet verification. The class was not
efficient for two reasons:
- it used a too complex input stream; generated from sources and UTF8
byte-conversions. The BufferedReader applied a strong overhead.
- to feed data into the SentenceReader, multiple toString/getBytes had
been applied until a buffered Reader from an input stream was possible.
These superfluous conversions had been removed.
- the best source for the Sentence Reader is a String. Therefore the
production of Strings had been forced inside the Document class.
Changed Files: htroot/ViewFile.java, source/de/anomic/data/ymark/YMarkAutoTagger.java, source/net/yacy/document/Condenser.java, source/net/yacy/document/Document.java, source/net/yacy/document/SentenceReader.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/WordTokenizer.java, source/net/yacy/document/content/DCEntry.java, source/net/yacy/document/parser/augment/AugmentParser.java, source/net/yacy/document/parser/csvParser.java, source/net/yacy/document/parser/docParser.java, source/net/yacy/document/parser/genericParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/images/genericImageParser.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/document/parser/pptParser.java, source/net/yacy/document/parser/rdfParser.java, source/net/yacy/document/parser/rdfa/impl/RDFaParser.java, source/net/yacy/document/parser/rtfParser.java, source/net/yacy/document/parser/swfParser.java, source/net/yacy/document/parser/torrentParser.java, source/net/yacy/document/parser/vsdParser.java, source/net/yacy/document/parser/xlsParser.java, source/net/yacy/search/snippet/TextSnippet.java
Tue Jul 03 17:06:20 CEST 2012
by Michael Peter Christen
Adding a limit of 1000 links that a parser shall store during indexing.
A limit was necessary because some web pages have such huge numbers of
links that it can easily cause a OOM just by the number of links.
The quesion if the number of 1000 links is sufficient or too weak must
be answered with the result of testing this feature.
Changed Files: htroot/Crawler_p.java, source/de/anomic/data/BookmarkHelper.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/html/ScraperInputStream.java, source/net/yacy/document/parser/html/TransformerWriter.java, source/net/yacy/document/parser/htmlParser.java
Mon Jul 02 15:40:40 CEST 2012
by Michael Peter Christen
- smaller caches to save memory
- close cloneable iterators to free memory
Changed Files: source/net/yacy/cora/order/CloneableIterator.java, source/net/yacy/cora/order/CloneableMapIterator.java, source/net/yacy/kelondro/blob/ArrayStack.java, source/net/yacy/kelondro/blob/BEncodedHeap.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/data/word/Word.java, source/net/yacy/kelondro/index/RowSet.java, source/net/yacy/kelondro/order/Digest.java, source/net/yacy/kelondro/order/MergeIterator.java, source/net/yacy/kelondro/order/RotateIterator.java, source/net/yacy/kelondro/order/StackIterator.java, source/net/yacy/kelondro/rwi/ReferenceContainerArray.java, source/net/yacy/kelondro/rwi/ReferenceContainerCache.java, source/net/yacy/kelondro/table/Table.java, source/net/yacy/search/index/MetadataRepository.java, source/net/yacy/search/ranking/BlockRank.java
Mon Jul 02 13:57:29 CEST 2012
by Michael Peter Christen
better integration of blacklist according to use case
Changed Files: htroot/Bookmarks.java, htroot/Crawler_p.java, htroot/DictionaryLoader_p.java, htroot/Load_RSS_p.java, htroot/ViewFile.java, htroot/ViewImage.java, htroot/api/getpageinfo.java, htroot/api/getpageinfo_p.java, htroot/api/webstructure.java, htroot/yacysearch.java, htroot/yacysearchitem.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/RSSLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/data/ymark/YMarkAutoTagger.java, source/de/anomic/data/ymark/YMarkMetadata.java, source/net/yacy/document/importer/OAIListFriendsLoader.java, source/net/yacy/document/importer/OAIPMHLoader.java, source/net/yacy/peers/graphics/OSMTile.java, source/net/yacy/peers/operation/yacyRelease.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/snippet/MediaSnippet.java, source/net/yacy/search/snippet/TextSnippet.java
Mon Jul 02 09:51:43 CEST 2012
by Michael Peter Christen
giving threads name so its easier to see whats happening during
debugging and within a thread dump
Changed Files: source/de/anomic/crawler/Cache.java, source/de/anomic/crawler/CrawlStacker.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/ftp/FTPClient.java, source/net/yacy/cora/services/federated/opensearch/SRURSSConnector.java, source/net/yacy/cora/storage/Files.java, source/net/yacy/document/content/dao/PhpBB3Dao.java, source/net/yacy/document/parser/pdfParser.java, source/net/yacy/gui/YaCyApp.java, source/net/yacy/kelondro/blob/MapHeap.java, source/net/yacy/kelondro/data/word/WordReferenceRow.java, source/net/yacy/kelondro/table/SplitTable.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/query/SearchEvent.java
Thu Jun 28 14:27:29 CEST 2012
by Michael Peter Christen
removed segments-concept and the Segments class:
the segments had been there to create a tenant-infrastructure but were
never be used since that was all much too complex. There will be a
replacement using a solr navigation using a segment field in the search
index.
Changed Files: htroot/Bookmarks.java, htroot/CrawlResults.java, htroot/Crawler_p.java, htroot/IndexCleaner_p.java, htroot/IndexControlRWIs_p.html, htroot/IndexControlRWIs_p.java, htroot/IndexControlURLs_p.html, htroot/IndexControlURLs_p.java, htroot/IndexFederated_p.java, htroot/IndexShare_p.java, htroot/Load_RSS_p.java, htroot/PerformanceGraph.java, htroot/PerformanceQueues_p.java, htroot/QuickCrawlLink_p.java, htroot/ViewFile.java, htroot/Vocabulary_p.java, htroot/YBRFetch_p.java, htroot/api/status_p.java, htroot/api/termlist_p.java, htroot/api/timeline.java, htroot/api/webstructure.java, htroot/api/yacydoc.java, htroot/api/ymarks/add_ymark.java, htroot/api/ymarks/get_metadata.java, htroot/api/ymarks/get_treeview.java, htroot/suggest.java, htroot/yacy/crawlReceipt.java, htroot/yacy/query.java, htroot/yacy/search.java, htroot/yacy/transferRWI.java, htroot/yacy/transferURL.java, htroot/yacy/urls.java, htroot/yacyinteractive.java, htroot/yacysearch.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/RSSLoader.java, source/de/anomic/crawler/SitemapImporter.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/data/ymark/YMarkMetadata.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/Segment.java
Wed Jun 27 12:17:58 CEST 2012
by Michael Peter Christen
- allow lazy initialization of solr value (if using 'lazy', then no
0-values and no empty strings are written). This may save a lot of
memory (in ram and on disc) if excessive 0-values or empty strings
appear)
- do not allow default boolean values for checkboxes because that does
not make sense: browsers may omit the checkbox attribute name if the box
is not checked. A default value 'true' would not comply with the
semantic of the browsers response.
- add a checkbox in IndexFederated_p for the lazy initialization of solr
fields.
Changed Files: defaults/yacy.init, htroot/AccessPicture_p.java, htroot/ConfigPortal.java, htroot/ConfigUpdate_p.java, htroot/Connections_p.java, htroot/Crawler_p.java, htroot/IndexFederated_p.html, htroot/IndexFederated_p.java, htroot/NetworkPicture.java, htroot/PeerLoadPicture.java, htroot/Status.java, htroot/Table_API_p.java, htroot/Threaddump_p.java, htroot/ViewFile.java, htroot/api/ymarks/import_ymark.java, htroot/opensearchdescription.java, source/de/anomic/server/serverObjects.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/SolrConfiguration.java
Tue Jun 26 10:13:13 CEST 2012
by cominch
Merge remote-tracking branch 'original yacy/master'
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, bin/checkalive.sh, build.xml, defaults/RDFaParser.xsl, defaults/solr/currency.xml, defaults/solr/elevate.xml, defaults/solr/lang/contractions_ca.txt, defaults/solr/lang/contractions_fr.txt, defaults/solr/lang/contractions_ga.txt, defaults/solr/lang/contractions_it.txt, defaults/solr/lang/hyphenations_ga.txt, defaults/solr/lang/stemdict_nl.txt, defaults/solr/lang/stoptags_ja.txt, defaults/solr/lang/stopwords_ar.txt, defaults/solr/lang/stopwords_bg.txt, defaults/solr/lang/stopwords_ca.txt, defaults/solr/lang/stopwords_cz.txt, defaults/solr/lang/stopwords_da.txt, defaults/solr/lang/stopwords_de.txt, defaults/solr/lang/stopwords_el.txt, defaults/solr/lang/stopwords_en.txt, defaults/solr/lang/stopwords_es.txt, defaults/solr/lang/stopwords_eu.txt, defaults/solr/lang/stopwords_fa.txt, defaults/solr/lang/stopwords_fi.txt, defaults/solr/lang/stopwords_fr.txt, defaults/solr/lang/stopwords_ga.txt, defaults/solr/lang/stopwords_gl.txt, defaults/solr/lang/stopwords_hi.txt, defaults/solr/lang/stopwords_hu.txt, defaults/solr/lang/stopwords_hy.txt, defaults/solr/lang/stopwords_id.txt, defaults/solr/lang/stopwords_it.txt, defaults/solr/lang/stopwords_ja.txt, defaults/solr/lang/stopwords_lv.txt, defaults/solr/lang/stopwords_nl.txt, defaults/solr/lang/stopwords_no.txt, defaults/solr/lang/stopwords_pt.txt, defaults/solr/lang/stopwords_ro.txt, defaults/solr/lang/stopwords_ru.txt, defaults/solr/lang/stopwords_sv.txt, defaults/solr/lang/stopwords_th.txt, defaults/solr/lang/stopwords_tr.txt, defaults/solr/protwords.txt, defaults/solr/schema.xml, defaults/solr/solr.xml, defaults/solr/solrconfig.xml, defaults/solr/stopwords.txt, defaults/solr/synonyms.txt, defaults/yacy.init, htroot/Blog.java, htroot/BlogComments.java, htroot/CacheResource_p.java, htroot/ConfigAccounts_p.java, htroot/ConfigAppearance_p.java, htroot/CookieTest_p.java, htroot/CrawlStartScanner_p.java, htroot/Crawler_p.java, htroot/IndexControlRWIs_p.java, htroot/IndexFederated_p.html, htroot/IndexFederated_p.java, htroot/Messages_p.java, htroot/QuickCrawlLink_p.java, htroot/SettingsAck_p.java, htroot/Steering.java, htroot/Table_API_p.java, htroot/User.java, htroot/ViewImage.java, htroot/Wiki.java, htroot/opensearchdescription.java, htroot/suggest.java, htroot/yacy/message.java, htroot/yacysearch.java, htroot/yacysearch_location.java, htroot/yacysearchitem.java, lib/apache-solr-core-3.6.0.jar, lib/commons-httpclient-3.1.jar, lib/commons-lang-2.6.jar, lib/dependencies.txt, lib/fontbox-1.7.0.License, lib/fontbox-1.7.0.jar, lib/guava-r05.jar, lib/jempbox-1.7.0.License, lib/jempbox-1.7.0.jar, lib/jetty-6.1.26-patched-JETTY-1340.jar, lib/jetty-LICENSE-ASL.txt, lib/jetty-util-6.1.26-patched-JETTY-1340.jar, lib/jetty-util-LICENSE-ASL.txt, lib/log4j-over-slf4j-1.6.1.jar, lib/lucene-analyzers-3.6.0.jar, lib/lucene-core-3.6.0.jar, lib/lucene-highlighter-3.6.0.jar, lib/lucene-phonetic-3.6.0.jar, lib/lucene-spatial-3.6.0.jar, lib/lucene-spellchecker-3.6.0.jar, lib/pdfbox-1.7.0.License, lib/pdfbox-1.7.0.jar, lib/servlet-api-2.5-20081211.jar, lib/servlet-api-LICENSE-ASL.txt, nbproject/project.xml, source/de/anomic/crawler/Balancer.java, source/de/anomic/crawler/CrawlQueues.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/data/BlogBoard.java, source/de/anomic/data/BlogBoardComments.java, source/de/anomic/data/wiki/WikiBoard.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/server/serverCore.java, source/de/anomic/server/servletProperties.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/protocol/ResponseHeader.java, source/net/yacy/cora/protocol/TimeoutRequest.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/services/federated/solr/AbstractSolrConnector.java, source/net/yacy/cora/services/federated/solr/MultipleSolrConnector.java, source/net/yacy/cora/services/federated/solr/RetrySolrConnector.java, source/net/yacy/cora/services/federated/solr/ShardSelection.java, source/net/yacy/cora/services/federated/solr/ShardSolrConnector.java, source/net/yacy/cora/services/federated/solr/SingleSolrConnector.java, source/net/yacy/cora/services/federated/solr/SolrConnector.java, source/net/yacy/cora/storage/ConfigurationSet.java, source/net/yacy/document/Document.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/rdfa/impl/RDFaTripleImpl.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/interaction/AugmentHtmlStream.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/migration.java, source/net/yacy/peers/Network.java, source/net/yacy/peers/Seed.java, source/net/yacy/peers/operation/yacyRelease.java, source/net/yacy/peers/operation/yacySeedUploadFile.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/DocumentIndex.java, source/net/yacy/search/index/MetadataRepository.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/index/SolrConfiguration.java, source/net/yacy/search/query/SnippetProcess.java, source/net/yacy/search/solr/EmbeddedSolrConnector.java, source/net/yacy/yacy.java
Tue Jun 26 00:08:25 CEST 2012
by Michael Peter Christen
generalized localhost naming.
this is also a preparation for a better IPv6 implementation.
Changed Files: htroot/Blog.java, htroot/BlogComments.java, htroot/ConfigAccounts_p.java, htroot/CrawlStartScanner_p.java, htroot/QuickCrawlLink_p.java, htroot/SettingsAck_p.java, htroot/Steering.java, htroot/Table_API_p.java, htroot/ViewImage.java, htroot/Wiki.java, htroot/opensearchdescription.java, htroot/yacysearch.java, htroot/yacysearch_location.java, htroot/yacysearchitem.java, source/de/anomic/crawler/Balancer.java, source/de/anomic/data/BlogBoard.java, source/de/anomic/data/BlogBoardComments.java, source/de/anomic/data/wiki/WikiBoard.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/server/serverCore.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/services/federated/solr/ShardSolrConnector.java, source/net/yacy/cora/services/federated/solr/SingleSolrConnector.java, source/net/yacy/interaction/AugmentHtmlStream.java, source/net/yacy/peers/Seed.java, source/net/yacy/search/Switchboard.java
Mon Jun 25 18:17:31 CEST 2012
by Michael Peter Christen
fixing redirects and status codes: storing of status code in
ResponseHeader to make it available for late evaluations, like storage
in solr.
Changed Files: htroot/CacheResource_p.java, htroot/CookieTest_p.java, htroot/Crawler_p.java, htroot/User.java, htroot/suggest.java, htroot/yacysearch.java, source/de/anomic/crawler/RobotsTxt.java, source/de/anomic/crawler/retrieval/FTPLoader.java, source/de/anomic/crawler/retrieval/FileLoader.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/de/anomic/crawler/retrieval/Response.java, source/de/anomic/crawler/retrieval/SMBLoader.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDProxyHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/server/servletProperties.java, source/net/yacy/cora/protocol/HeaderFramework.java, source/net/yacy/cora/protocol/ResponseHeader.java, source/net/yacy/cora/protocol/http/HTTPClient.java, source/net/yacy/cora/services/federated/solr/AbstractSolrConnector.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/sitemapParser.java, source/net/yacy/peers/operation/yacyRelease.java, source/net/yacy/repository/LoaderDispatcher.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/SolrConfiguration.java
Mon Jun 25 11:34:38 CEST 2012
by Michael Peter Christen
- fixed IndexFederated Servlet / a embedded Solr can now be selected
- added code stub for an embedded Solr but generation of Solr store is
still commented out (it works but is not yet ready for usage)
Changed Files: defaults/yacy.init, htroot/IndexControlRWIs_p.java, htroot/IndexFederated_p.html, htroot/IndexFederated_p.java, source/de/anomic/crawler/CrawlQueues.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/index/MetadataRepository.java, source/net/yacy/search/index/Segment.java, source/net/yacy/search/query/SnippetProcess.java, source/net/yacy/search/solr/EmbeddedSolrConnector.java
Fri Jun 22 16:49:58 CEST 2012
by Michael Peter Christen
upgraded to pdfbox 1.7.0
changes in http://www.apache.org/dist/pdfbox/1.7.0/RELEASE-NOTES.txt
with many bugfixes, including performance related
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, lib/fontbox-1.7.0.License, lib/fontbox-1.7.0.jar, lib/jempbox-1.7.0.License, lib/jempbox-1.7.0.jar, lib/pdfbox-1.7.0.License, lib/pdfbox-1.7.0.jar
Fri Jun 22 15:31:17 CEST 2012
by Michael Peter Christen
added jetty libraries, needed for future use as web server and as
application server for the solr search interface
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, htroot/IndexFederated_p.html, lib/dependencies.txt, lib/jetty-6.1.26-patched-JETTY-1340.jar, lib/jetty-LICENSE-ASL.txt, lib/jetty-util-6.1.26-patched-JETTY-1340.jar, lib/jetty-util-LICENSE-ASL.txt, lib/servlet-api-2.5-20081211.jar, lib/servlet-api-LICENSE-ASL.txt, source/net/yacy/search/solr/EmbeddedSolrConnector.java
Fri Jun 22 11:39:17 CEST 2012
by Michael Peter Christen
using com.google.common.io.Files instead of homebrew methods
Changed Files: .classpath, build.xml, htroot/BlogComments.java, htroot/ConfigAppearance_p.java, htroot/Messages_p.java, htroot/yacy/message.java, source/net/yacy/cora/storage/ConfigurationSet.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/migration.java, source/net/yacy/peers/operation/yacyRelease.java, source/net/yacy/peers/operation/yacySeedUploadFile.java, source/net/yacy/search/Switchboard.java, source/net/yacy/yacy.java
Fri Jun 22 00:36:49 CEST 2012
by Michael Peter Christen
- added test for EmbeddedSolrConnector
- added needed libraries for this test
this includes most (all) files needed for an embedded solr
Changed Files: .classpath, addon/YaCy.app/Contents/Info.plist, build.xml, defaults/solr/currency.xml, defaults/solr/elevate.xml, defaults/solr/lang/contractions_ca.txt, defaults/solr/lang/contractions_fr.txt, defaults/solr/lang/contractions_ga.txt, defaults/solr/lang/contractions_it.txt, defaults/solr/lang/hyphenations_ga.txt, defaults/solr/lang/stemdict_nl.txt, defaults/solr/lang/stoptags_ja.txt, defaults/solr/lang/stopwords_ar.txt, defaults/solr/lang/stopwords_bg.txt, defaults/solr/lang/stopwords_ca.txt, defaults/solr/lang/stopwords_cz.txt, defaults/solr/lang/stopwords_da.txt, defaults/solr/lang/stopwords_de.txt, defaults/solr/lang/stopwords_el.txt, defaults/solr/lang/stopwords_en.txt, defaults/solr/lang/stopwords_es.txt, defaults/solr/lang/stopwords_eu.txt, defaults/solr/lang/stopwords_fa.txt, defaults/solr/lang/stopwords_fi.txt, defaults/solr/lang/stopwords_fr.txt, defaults/solr/lang/stopwords_ga.txt, defaults/solr/lang/stopwords_gl.txt, defaults/solr/lang/stopwords_hi.txt, defaults/solr/lang/stopwords_hu.txt, defaults/solr/lang/stopwords_hy.txt, defaults/solr/lang/stopwords_id.txt, defaults/solr/lang/stopwords_it.txt, defaults/solr/lang/stopwords_ja.txt, defaults/solr/lang/stopwords_lv.txt, defaults/solr/lang/stopwords_nl.txt, defaults/solr/lang/stopwords_no.txt, defaults/solr/lang/stopwords_pt.txt, defaults/solr/lang/stopwords_ro.txt, defaults/solr/lang/stopwords_ru.txt, defaults/solr/lang/stopwords_sv.txt, defaults/solr/lang/stopwords_th.txt, defaults/solr/lang/stopwords_tr.txt, defaults/solr/protwords.txt, defaults/solr/schema.xml, defaults/solr/solr.xml, defaults/solr/solrconfig.xml, defaults/solr/stopwords.txt, defaults/solr/synonyms.txt, lib/commons-httpclient-3.1.jar, lib/dependencies.txt, lib/lucene-analyzers-3.6.0.jar, lib/lucene-core-3.6.0.jar, lib/lucene-highlighter-3.6.0.jar, lib/lucene-phonetic-3.6.0.jar, lib/lucene-spatial-3.6.0.jar, lib/lucene-spellchecker-3.6.0.jar, source/net/yacy/search/solr/EmbeddedSolrConnector.java
Thu Jun 21 14:55:38 CEST 2012
by Michael Peter Christen
- added solr core and libraries that solr needs (lucene is missing, will
follow later)
- added embedded solr connector which can connect to solr
programmatically (without using a server in between)
Changed Files: .classpath, build.xml, lib/apache-solr-core-3.6.0.jar, lib/commons-lang-2.6.jar, lib/dependencies.txt, lib/guava-r05.jar, lib/log4j-over-slf4j-1.6.1.jar, source/net/yacy/cora/services/federated/solr/AbstractSolrConnector.java, source/net/yacy/cora/services/federated/solr/SolrSingleConnector.java, source/net/yacy/peers/Network.java, source/net/yacy/search/solr/EmbeddedSolrConnector.java
Wed Jun 20 18:04:23 CEST 2012
by cominch
Show additional interaction elements in footer section on each page, if
activated in ConfigPortal.html.
This footer is also visible in augmented browsing proxy mode.
Changed Files: htroot/ConfigPortal.html, htroot/ConfigPortal.java, htroot/env/templates/embeddedfooter.template, htroot/env/templates/footer.template, htroot/env/templates/simplefooter.template, htroot/interaction_elements/Footer.html, htroot/interaction_elements/Footer.java, htroot/interaction_elements/Loginstatus_part.html, htroot/interaction_elements/Loginstatus_part.java, htroot/interaction_elements/OverlayInteraction.html, htroot/interaction_elements/OverlayInteraction.java, htroot/interaction_elements/login_admin.png, htroot/interaction_elements/login_empty.png, htroot/interaction_elements/login_user.png, htroot/yacysearch.html, source/net/yacy/interaction/AugmentHtmlStream.java
Wed Jun 20 07:58:27 CEST 2012
by cominch
Merge remote-tracking branch 'original yacy/master'
Changed Files: build.properties, defaults/yacy.init, htroot/Crawler_p.html, htroot/PerformanceQueues_p.java, source/de/anomic/crawler/RobotsTxt.java, source/net/yacy/document/TextParser.java, source/net/yacy/kelondro/blob/HeapReader.java, source/net/yacy/peers/SeedDB.java, source/net/yacy/upnp/DiscoveryAdvertisement.java, startYACY.sh


Bugfixes   
Jump to: YaCy Release 1.04 top / Other Changes

CommitDescription
Sun Jul 08 16:48:09 CEST 2012
by Michael Peter Christen
fix for url camel case parser and sentence reader
Changed Files: source/net/yacy/document/Condenser.java, source/net/yacy/document/SentenceReader.java
Sun Jul 08 16:11:19 CEST 2012
by Michael Peter Christen
fix for sevenzip parser
Changed Files: source/net/yacy/document/parser/sevenzipParser.java
Fri Jul 06 01:29:13 CEST 2012
by Michael Peter Christen
fix to solr configuration (case where the external solr was not online)
Changed Files: htroot/IndexFederated_p.java
Thu Jul 05 14:24:03 CEST 2012
by Michael Peter Christen
fix for pattern matcher in html parser
Changed Files: source/net/yacy/document/parser/html/ContentScraper.java
Thu Jul 05 14:23:43 CEST 2012
by Michael Peter Christen
fix for solr shutdown
Changed Files: source/net/yacy/cora/services/federated/solr/MultipleSolrConnector.java
Thu Jul 05 14:23:29 CEST 2012
by Michael Peter Christen
fix for urls beginning with "//"
Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java
Thu Jul 05 14:06:00 CEST 2012
by sixcooler
fix for http://forum.yacy-websuche.de/viewtopic.php?f=5&t=4430
Changed Files: htroot/IndexControlRWIs_p.java
Mon Jul 02 14:37:57 CEST 2012
by Michael Peter Christen
bugfix for concurrent seed loader
Changed Files: source/net/yacy/search/Switchboard.java
Mon Jul 02 10:27:46 CEST 2012
by Michael Peter Christen
fixes for new eclipse 'Juno' warning 'Resource leak'.
Changed Files: htroot/interaction_elements/Document_part.java, source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/HTTPDemon.java, source/de/anomic/http/server/TemplateEngine.java, source/de/anomic/tools/CryptoLib.java, source/net/yacy/cora/storage/ConfigurationSet.java, source/net/yacy/document/importer/MediawikiImporter.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/document/parser/tarParser.java, source/net/yacy/document/parser/vcfParser.java, source/net/yacy/kelondro/data/meta/URIMetadataRow.java, source/net/yacy/kelondro/index/Row.java, source/net/yacy/kelondro/order/Digest.java, source/net/yacy/kelondro/util/FileUtils.java, source/net/yacy/search/Switchboard.java, source/net/yacy/search/query/AccessTracker.java
Tue Jun 26 16:11:39 CEST 2012
by sixcooler
fix crawl start from file
Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java
Thu Jun 21 14:22:32 CEST 2012
by cominch
Augmented browsing: Small CSS fix
Changed Files: htroot/interaction_elements/OverlayInteraction.html
Thu Jun 21 12:02:14 CEST 2012
by cominch
Augmented browsing: small js fix
Changed Files: htroot/interaction_elements/Tag_part.html
Thu Jun 21 11:19:55 CEST 2012
by cominch
Augmented browsing: CSS fix
Changed Files: htroot/interaction_elements/OverlayInteraction.html


Other Changes   
Jump to: YaCy Release 1.04 top / Bugfixes

CommitDescription
Mon Jul 09 00:13:59 CEST 2012
by Michael Peter Christen
Release 1.04
Changed Files: build.properties
Sun Jul 08 22:05:04 CEST 2012
by Michael Peter Christen
use less memory for md5 cache
Changed Files: source/net/yacy/kelondro/order/Digest.java
Sun Jul 08 22:04:36 CEST 2012
by Michael Peter Christen
more logging
Changed Files: source/net/yacy/kelondro/logging/Log.java
Sun Jul 08 21:25:22 CEST 2012
by Michael Peter Christen
filter old peers from bootstrap (now stronger: 60 minutes instead of
240).
Changed Files: source/net/yacy/search/Switchboard.java
Sun Jul 08 21:17:33 CEST 2012
by Michael Peter Christen
added classification for control file types which shall not be loaded
but placed onto the noload-queue
Changed Files: source/de/anomic/crawler/CrawlStacker.java, source/net/yacy/cora/document/Classification.java
Sun Jul 08 17:59:20 CEST 2012
by Michael Peter Christen
added webm mime-type
Changed Files: defaults/httpd.mime
Sun Jul 08 17:58:05 CEST 2012
by Michael Peter Christen
added webm
Changed Files: source/net/yacy/cora/document/Classification.java
Sun Jul 08 16:11:50 CEST 2012
by Michael Peter Christen
fix for sitemap importer: can now also import very large sitemaps within
small memory configurations
Changed Files: source/de/anomic/crawler/SitemapImporter.java, source/net/yacy/document/parser/sitemapParser.java
Fri Jul 06 09:21:12 CEST 2012
by Michael Peter Christen
catch and log a warning in RasterPlotter
Changed Files: source/net/yacy/visualization/RasterPlotter.java
Fri Jul 06 09:05:41 CEST 2012
by Michael Peter Christen
- fixed a memory leak (or bad usage) during parsing/snippet fetch
- more logging for errors
Changed Files: source/de/anomic/http/server/HTTPDFileHandler.java, source/net/yacy/document/parser/html/TransformerWriter.java, source/net/yacy/document/parser/htmlParser.java, source/net/yacy/kelondro/workflow/InstantBlockingThread.java
Fri Jul 06 08:29:41 CEST 2012
by Michael Peter Christen
prevent loading of content from the cache when retrieval with IFFRESH is
used and cache is stale. Should speed up snippet generation when cache
strategy is IFFRESH.
Changed Files: source/de/anomic/crawler/Cache.java, source/net/yacy/repository/LoaderDispatcher.java
Thu Jul 05 14:50:37 CEST 2012
by sixcooler
more abstraction of error message
Changed Files: htroot/IndexControlRWIs_p.java
Thu Jul 05 14:27:28 CEST 2012
by Michael Peter Christen
abstraction of error message
Changed Files: htroot/IndexControlRWIs_p.java
Thu Jul 05 11:09:44 CEST 2012
by Michael Peter Christen
removed unaccessible code
Changed Files: .settings/org.eclipse.jdt.core.prefs, source/net/yacy/kelondro/table/Table.java
Thu Jul 05 10:24:52 CEST 2012
by Michael Peter Christen
removed unused ImageReference package
Changed Files:
Thu Jul 05 09:21:27 CEST 2012
by Michael Peter Christen
removed snippet pattern filter - it was not used
Changed Files: htroot/api/ymarks/manage_tags.java, htroot/yacy/search.java, htroot/yacysearch.java, source/de/anomic/data/ymark/YMarkTables.java, source/net/yacy/peers/Protocol.java, source/net/yacy/peers/RemoteSearch.java, source/net/yacy/search/query/QueryParams.java, source/net/yacy/search/query/SearchEvent.java, source/net/yacy/search/query/SnippetProcess.java
Thu Jul 05 01:02:51 CEST 2012
by Michael Peter Christen
replaced non-generic array with collection
Changed Files: source/net/yacy/kelondro/workflow/InstantBusyThread.java, source/net/yacy/peers/Protocol.java
Thu Jul 05 00:43:41 CEST 2012
by Michael Peter Christen
adding more principal peers for bootstraping
Changed Files: defaults/yacy.network.freeworld.unit
Thu Jul 05 00:20:58 CEST 2012
by orbiter
More SentenceReader cleanup
Changed Files: source/net/yacy/document/Document.java, source/net/yacy/document/SentenceReader.java, source/net/yacy/search/snippet/TextSnippet.java
Wed Jul 04 22:06:20 CEST 2012
by orbiter
Simplified SentenceReader (no more Reader inside..)
Changed Files: source/net/yacy/document/SentenceReader.java
Wed Jul 04 21:56:25 CEST 2012
by orbiter
replaced HashARC with SizeLimited Objects which are less costly
Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/storage/SizeLimitedMap.java, source/net/yacy/cora/storage/SizeLimitedSet.java, source/net/yacy/document/parser/html/ContentScraper.java
Wed Jul 04 21:15:38 CEST 2012
by orbiter
more tolerance when creating solar document
Changed Files: source/net/yacy/search/index/SolrConfiguration.java
Tue Jul 03 18:22:25 CEST 2012
by orbiter
automatically adopt size of word cache to available memory
Changed Files: source/net/yacy/cora/sorting/OrderedScoreMap.java, source/net/yacy/document/WordCache.java
Tue Jul 03 17:20:41 CEST 2012
by Michael Peter Christen
clean up parser data
Changed Files: source/net/yacy/document/Document.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/document/parser/htmlParser.java
Tue Jul 03 07:12:20 CEST 2012
by Michael Peter Christen
- better data structures in secondary search
- fixed a big memory leak in secondary search
Changed Files: source/net/yacy/kelondro/data/word/WordReferenceFactory.java, source/net/yacy/peers/RemoteSearch.java, source/net/yacy/search/query/SearchEvent.java
Tue Jul 03 06:06:38 CEST 2012
by Michael Peter Christen
parser refactoring & hacks
Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/document/TextParser.java, source/net/yacy/document/parser/html/AbstractScraper.java, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/search/snippet/TextSnippet.java
Mon Jul 02 14:27:37 CEST 2012
by Michael Peter Christen
concurrently initialize the seed list during p2p network bootstrap
Changed Files: source/net/yacy/search/Switchboard.java
Sun Jul 01 00:12:20 CEST 2012
by reger
add search result heuristic. adding a crawl job with depth-1 for every displayed search result (crawling every external linked page of displayed search result pages)
Changed Files: defaults/yacy.init, htroot/ConfigHeuristics_p.html, htroot/ConfigHeuristics_p.java, htroot/yacysearchitem.java, source/net/yacy/search/Switchboard.java
Sat Jun 30 10:30:01 CEST 2012
by Michael Peter Christen
more logging
Changed Files: source/de/anomic/http/server/HTTPDemon.java, source/net/yacy/http/SSIHandler.java, source/net/yacy/yacy.java
Thu Jun 28 13:27:45 CEST 2012
by Michael Peter Christen
added solr field 'refresh_s' which stores the refresh url contained in
the meta-refresh html header field.
Changed Files: defaults/solr.keys.list, source/net/yacy/document/parser/html/ContentScraper.java, source/net/yacy/search/index/SolrConfiguration.java, source/net/yacy/search/index/SolrField.java
Wed Jun 27 13:07:02 CEST 2012
by Michael Peter Christen
do not fill the keywords with title content if keywords do not exist.
Changed Files: source/net/yacy/document/parser/html/ContentScraper.java
Tue Jun 26 14:53:45 CEST 2012
by Michael Peter Christen
shorter autocommit time (now: 1 second) to prevent that user cannot see
results in solr the first time they try it out. The value can now be
easily set to a higher number using the IndexFederated_p interface.
Changed Files: defaults/yacy.init
Tue Jun 26 14:51:57 CEST 2012
by Michael Peter Christen
- add canonical field only if requested by solr schema
- remove canonical url from in/outbound urls if present
Changed Files: source/net/yacy/search/index/SolrConfiguration.java
Tue Jun 26 13:54:48 CEST 2012
by Michael Peter Christen
added option to record urls that are forwarded to the solr index
Changed Files: defaults/solr.keys.list, defaults/yacy.init, source/de/anomic/crawler/ZURL.java, source/de/anomic/crawler/retrieval/HTTPLoader.java, source/net/yacy/search/SwitchboardConstants.java, source/net/yacy/search/index/MetadataRepository.java
Tue Jun 26 11:18:29 CEST 2012
by Michael Peter Christen
fixed bad referer computation in SSIs which causes a NPE during host
computation. This error was there before the latest IPv6 hack but did
not cause a NPE. The IPv6 hack was not the cause for this bug, but it
discovered the misconfiguration of the 'referer' referrer.
Changed Files: source/de/anomic/http/server/HTTPDFileHandler.java, source/de/anomic/http/server/ServerSideIncludes.java, source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/RequestHeader.java
Tue Jun 26 00:25:46 CEST 2012
by Michael Peter Christen
more IPv6 hacks
Changed Files: source/net/yacy/cora/document/MultiProtocolURI.java, source/net/yacy/cora/protocol/Domains.java
Mon Jun 25 14:59:46 CEST 2012
by Michael Peter Christen
added option to configure the autocommit delay time of solr on-the-fly
Changed Files: defaults/yacy.init, htroot/IndexFederated_p.html, htroot/IndexFederated_p.java, source/net/yacy/cora/services/federated/solr/AbstractSolrConnector.java, source/net/yacy/cora/services/federated/solr/MultipleSolrConnector.java, source/net/yacy/cora/services/federated/solr/RetrySolrConnector.java, source/net/yacy/cora/services/federated/solr/ShardSolrConnector.java, source/net/yacy/cora/services/federated/solr/SolrConnector.java, source/net/yacy/search/Switchboard.java
Mon Jun 25 11:37:32 CEST 2012
by Michael Peter Christen
Merge remote-tracking branch 'origin/master'
Changed Files: nbproject/project.xml
Sun Jun 24 22:50:08 CEST 2012
by reger
adjusted NetBeans classpath for  new and updated libraries in lib 
Changed Files: nbproject/project.xml
Sun Jun 24 10:58:09 CEST 2012
by Michael Peter Christen
root, not yacy
Changed Files: bin/checkalive.sh
Sun Jun 24 10:57:18 CEST 2012
by Michael Peter Christen
changed recommended line in /etc/crontab for high-availability
Changed Files: bin/checkalive.sh
Fri Jun 22 11:40:02 CEST 2012
by Michael Peter Christen
extended embedded solr tests to ensure that it will be usable within a
jetty instance
Changed Files: source/net/yacy/cora/services/federated/solr/AbstractSolrConnector.java, source/net/yacy/search/solr/EmbeddedSolrConnector.java
Fri Jun 22 00:49:32 CEST 2012
by Michael Peter Christen
refactoring
Changed Files: htroot/IndexFederated_p.java, source/de/anomic/crawler/ZURL.java, source/net/yacy/cora/services/federated/solr/MultipleSolrConnector.java, source/net/yacy/cora/services/federated/solr/RetrySolrConnector.java, source/net/yacy/cora/services/federated/solr/ShardSelection.java, source/net/yacy/cora/services/federated/solr/ShardSolrConnector.java, source/net/yacy/cora/services/federated/solr/SingleSolrConnector.java, source/net/yacy/search/Switchboard.java
Thu Jun 21 16:09:12 CEST 2012
by Michael Peter Christen
moved RDFaParser.xsl configuration file to defaults
Changed Files: build.xml, defaults/RDFaParser.xsl, source/net/yacy/document/parser/rdfa/impl/RDFaTripleImpl.java
Thu Jun 21 16:04:48 CEST 2012
by Michael Peter Christen
using guava for host resolution (non-blocking for ips) and time-out
Changed Files: .classpath, source/net/yacy/cora/protocol/Domains.java, source/net/yacy/cora/protocol/TimeoutRequest.java
Thu Jun 21 14:59:55 CEST 2012
by Michael Peter Christen
added new class libraries to mac app
Changed Files: addon/YaCy.app/Contents/Info.plist
Thu Jun 21 11:01:02 CEST 2012
by cominch
Augmented browsing: small UI modifications
Changed Files: htroot/interaction_elements/Document_part.html, htroot/interaction_elements/OverlayInteraction.html, htroot/interaction_elements/Tag_part.html
Wed Jun 20 16:39:04 CEST 2012
by Michael Peter Christen
better integration of RDFaParser
Changed Files: build.xml, source/net/yacy/document/Document.java, source/net/yacy/document/TextParser.java, source/net/yacy/search/index/DocumentIndex.java
Wed Jun 20 09:10:39 CEST 2012
by cominch
Augmented Browsing: changed the settings page
Changed Files: htroot/AugmentedBrowsingFilters_p.html, htroot/AugmentedBrowsingFilters_p.java
Wed Jun 20 07:55:28 CEST 2012
by cominch
Corrected loading of default page settings on ConfigPortal.html
Changed Files: htroot/ConfigPortal.java
Tue Jun 19 13:13:00 CEST 2012
by sixcooler
correct table in new look of Crawler_p
Changed Files: htroot/Crawler_p.html